[PATCH 00/36] drm: xlnx: zynqmp_dpsub: Initial live video input support

[PATCH 00/36] drm: xlnx: zynqmp_dpsub: Initial live video input support

[Laurent Pinchart]

Hello,

The DPSUB is the DisplayPort subsystem, a set of hard IP cores found in
the ZynqMP family of SoCs. It combines a DisplayPort encoder, a video
blender with two input channels, and a DMA engine. The zynqmp_dpsub
driver exposes this as a DRM device with one CRTC and two planes.

In addition to those features, the DPSUB can interface with the
programmable logic (PL) found in the ZynqMP SoC. Each input to the video
blender can come from the PL instead of the DMA engine, and the blender
output can also be routed to the PL. This creates a very configurable
device that can accommodate lots of use cases, but it also makes it
difficult to model it as a DRM/KMS device.

This patch series implements initial support for live video inputs, by
restricting the supported use cases to a single live video input. In
that mode, the video blender is configured in pass-through mode, with
the whole DPSUB essentially operating as a DisplayPort encoder only. The
CRTC and plane functions are then implemented by IP cores in the PL.

To support this, the series start with patch 01/36 to model the
connections to the PL in DT using OF graph bindings. This fixes a
historical mistake that forgot to model the connection to the DP
connector in DT.

With that in place, patches 02/36 to 10/36 refactor the driver to turn
the DisplayPort encoder implementation, modelled as a DRM encoder, into
a DRM bridge. Please see individual patches for details. The rework is
internal only, simplifies the code by making use of the DRM bridge
connector helper, but doesn't bring any functional change.

Patches 11/36 to 30/36 continue refactoring of the driver, to cleanly
separate the DRM planes, CRTC, encoder and connector from the DRM bridge
implementation. The goal is to make the latter available to a DRM driver
for the PL display pipeline without registering any DRM device in the
DPSUB driver itself. Patches 31/36 to 34/36 implement this, reading
information about the connection to the PL from the device tree to
decide in which mode to operate.

Finally, patch 35/36 and 36/36 update the ZynqMP core and ZCU106A board
device tree files to create ports and connect the DPSUB to the
DisplayPort connector. I have tested the whole series without these two
patches to ensure that backward compatibility with older DT isn't
broken.

With this series applied, the DPSUB can be used as a DisplayPort encoder
by a PL display pipeline. A careful reviewer may ask me where drivers
for such a display pipelines are, and that would be a very good
question.

PL display pipelines are currently supported in the Xilinx downstream
kernel only, which is something I want to address next. That road will
be full of challenges, as in theory anything can be implemented in the
PL, including pipelines that connect cameras and displays together. If
anynoe is interested in discussing this topic, please let me know.

Laurent Pinchart (36):
  dt-bindings: display: xlnx: zynqmp-dpsub: Add OF graph ports
  drm: xlnx: zynqmp_dpsub: Switch to atomic encoder enable/disable
  drm: xlnx: zynqmp_dpsub: Constify mode argument to function
  drm: xlnx: zynqmp_dpsub: Create DRM bridge to model DP encoder
  drm: xlnx: zynqmp_dpsub: Don't access connector in
    zynqmp_dp_set_format()
  drm: xlnx: zynqmp_dpsub: Move connector registration to bridge attach
  drm: xlnx: zynqmp_dpsub: Move encoder to DPSUB core
  drm: xlnx: zynqmp_dpsub: Attach to the next bridge
  drm: xlnx: zynqmp_dpsub: Use DRM connector bridge helper
  drm: xlnx: zynqmp_dpsub: Report HPD through the bridge
  drm: xlnx: zynqmp_dpsub: Drop unused zynqmp_disp.event field
  drm: xlnx: zynqmp_dpsub: Drop unused zynqmp_disp_format.bus_fmt field
  drm: xlnx: zynqmp_dpsub: Don't pass CRTC to zynqmp_disp_setup_clock()
  drm: xlnx: zynqmp_dpsub: Configure blender in zynqmp_disp_enable()
  drm: xlnx: zynqmp_dpsub: Use local variable in
    zynqmp_disp_layer_update()
  drm: xlnx: zynqmp_dpsub: Pass format info to
    zynqmp_disp_layer_set_format()
  drm: xlnx: zynqmp_dpsub: Remplace hardcoded values with ARRAY_SIZE()
  drm: xlnx: zynqmp_dpsub: Don't use drmm_kcalloc() for temporary data
  drm: xlnx: zynqmp_dpsub: Move pclk from zynqmp_disp to zynqmp_dpsub
  drm: xlnx: zynqmp_dpsub: Move audio clk from zynqmp_disp to
    zynqmp_dpsub
  drm: xlnx: zynqmp_dpsub: Move CRTC to zynqmp_dpsub structure
  drm: xlnx: zynqmp_dpsub: Move planes to zynqmp_dpsub structure
  drm: xlnx: zynqmp_dpsub: Move CRTC handling to zynqmp_kms.c
  drm: xlnx: zynqmp_dpsub: Move planes handling to zynqmp_kms.c
  drm: xlnx: zynqmp_dpsub: Register AUX bus at bridge attach time
  drm: xlnx: zynqmp_dpsub: Move DP bridge init to zynqmp_dp_probe()
  drm: xlnx: zynqmp_dpsub: Manage DP and DISP allocations manually
  drm: xlnx: zynqmp_dpsub: Move all DRM init and cleanup to zynqmp_kms.c
  drm: xlnx: zynqmp_dpsub: Decouple DRM device from zynqmp_dpsub
  drm: xlnx: zynqmp_dpsub: Rename zynqmp_dpsub_handle_vblank with DRM
    prefix
  drm: xlnx: zynqmp_dpsub: Parse DT to find connected ports
  drm: xlnx: zynqmp_dpsub: Allow configuration of layer mode
  drm: xlnx: zynqmp_dpsub: Support operation without DMA engine
  drm: xlnx: zynqmp_dpsub: Add support for live video input
  arm64: dts: zynqmp: Add ports for the DisplayPort subsystem
  arm64: dts: zynqmp: zcu106a: Describe DisplayPort connector

 .../display/xlnx/xlnx,zynqmp-dpsub.yaml       |  67 ++
 .../boot/dts/xilinx/zynqmp-zcu106-revA.dts    |  20 +
 arch/arm64/boot/dts/xilinx/zynqmp.dtsi        |  24 +
 drivers/gpu/drm/xlnx/Makefile                 |   2 +-
 drivers/gpu/drm/xlnx/zynqmp_disp.c            | 646 ++++--------------
 drivers/gpu/drm/xlnx/zynqmp_disp.h            |  48 +-
 drivers/gpu/drm/xlnx/zynqmp_dp.c              | 482 +++++++------
 drivers/gpu/drm/xlnx/zynqmp_dp.h              |   4 +-
 drivers/gpu/drm/xlnx/zynqmp_dpsub.c           | 316 +++++----
 drivers/gpu/drm/xlnx/zynqmp_dpsub.h           |  46 +-
 drivers/gpu/drm/xlnx/zynqmp_kms.c             | 539 +++++++++++++++
 drivers/gpu/drm/xlnx/zynqmp_kms.h             |  48 ++
 12 files changed, 1343 insertions(+), 899 deletions(-)
 create mode 100644 drivers/gpu/drm/xlnx/zynqmp_kms.c
 create mode 100644 drivers/gpu/drm/xlnx/zynqmp_kms.h

-- 
Regards,

Laurent Pinchart

[PATCH 01/36] dt-bindings: display: xlnx: zynqmp-dpsub: Add OF graph ports

[Laurent Pinchart]

The DPSUB doesn't live in isolation, but is connected to the
programmable logic for live inputs and outputs, and also has a
DisplayPort output. Model all those using OF graph.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 .../display/xlnx/xlnx,zynqmp-dpsub.yaml       | 67 +++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/Documentation/devicetree/bindings/display/xlnx/xlnx,zynqmp-dpsub.yaml b/Documentation/devicetree/bindings/display/xlnx/xlnx,zynqmp-dpsub.yaml
index d88bd93f4b80..5000c5fda027 100644
--- a/Documentation/devicetree/bindings/display/xlnx/xlnx,zynqmp-dpsub.yaml
+++ b/Documentation/devicetree/bindings/display/xlnx/xlnx,zynqmp-dpsub.yaml
@@ -117,6 +117,45 @@ properties:
       - const: dp-phy0
       - const: dp-phy1
 
+  ports:
+    $ref: /schemas/graph.yaml#/properties/ports
+    description: |
+      Connections to the programmable logic and the DisplayPort PHYs. Each port
+      shall have a single endpoint.
+
+    properties:
+      port@0:
+        $ref: /schemas/graph.yaml#/properties/port
+        description: The live video input from the programmable logic
+
+      port@1:
+        $ref: /schemas/graph.yaml#/properties/port
+        description: The live graphics input from the programmable logic
+
+      port@2:
+        $ref: /schemas/graph.yaml#/properties/port
+        description: The live audio input from the programmable logic
+
+      port@3:
+        $ref: /schemas/graph.yaml#/properties/port
+        description: The blended video output to the programmable logic
+
+      port@4:
+        $ref: /schemas/graph.yaml#/properties/port
+        description: The mixed audio output to the programmable logic
+
+      port@5:
+        $ref: /schemas/graph.yaml#/properties/port
+        description: The DisplayPort output
+
+    required:
+      - port@0
+      - port@1
+      - port@2
+      - port@3
+      - port@4
+      - port@5
+
 required:
   - compatible
   - reg
@@ -130,6 +169,7 @@ required:
   - dma-names
   - phys
   - phy-names
+  - ports
 
 additionalProperties: false
 
@@ -164,6 +204,33 @@ examples:
                <&psgtr 0 PHY_TYPE_DP 1 3 27000000>;
 
         phy-names = "dp-phy0", "dp-phy1";
+
+        ports {
+            #address-cells = <1>;
+            #size-cells = <0>;
+
+            port@0 {
+                reg = <0>;
+            };
+            port@1 {
+                reg = <1>;
+            };
+            port@2 {
+                reg = <2>;
+            };
+            port@3 {
+                reg = <3>;
+            };
+            port@4 {
+                reg = <4>;
+            };
+            port@5 {
+                reg = <5>;
+                dpsub_dp_out: endpoint {
+                    remote-endpoint = <&dp_connector>;
+                };
+            };
+        };
     };
 
 ...
-- 
Regards,

Laurent Pinchart

Re: [PATCH 01/36] dt-bindings: display: xlnx: zynqmp-dpsub: Add OF graph ports

[Rob Herring]

On Mon, 09 Aug 2021 04:34:22 +0300, Laurent Pinchart wrote:
> The DPSUB doesn't live in isolation, but is connected to the
> programmable logic for live inputs and outputs, and also has a
> DisplayPort output. Model all those using OF graph.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  .../display/xlnx/xlnx,zynqmp-dpsub.yaml       | 67 +++++++++++++++++++
>  1 file changed, 67 insertions(+)
> 

Reviewed-by: Rob Herring <robh@kernel.org>

[PATCH 35/36] arm64: dts: zynqmp: Add ports for the DisplayPort subsystem

[Laurent Pinchart]

The DPSUB DT bindings now specify ports to model the connections with
the programmable logic and the DisplayPort output. Add them to the
device tree.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
index 28dccb891a53..5128beb1dac0 100644
--- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
+++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
@@ -879,6 +879,30 @@ zynqmp_dpsub: display@fd4a0000 {
 			       <&zynqmp_dpdma ZYNQMP_DPDMA_VIDEO1>,
 			       <&zynqmp_dpdma ZYNQMP_DPDMA_VIDEO2>,
 			       <&zynqmp_dpdma ZYNQMP_DPDMA_GRAPHICS>;
+
+			ports {
+				#address-cells = <1>;
+				#size-cells = <0>;
+
+				port@0 {
+					reg = <0>;
+				};
+				port@1 {
+					reg = <1>;
+				};
+				port@2 {
+					reg = <2>;
+				};
+				port@3 {
+					reg = <3>;
+				};
+				port@4 {
+					reg = <4>;
+				};
+				port@5 {
+					reg = <5>;
+				};
+			};
 		};
 	};
 };
-- 
Regards,

Laurent Pinchart

[PATCH 36/36] arm64: dts: zynqmp: zcu106a: Describe DisplayPort connector

[Laurent Pinchart]

Add a device tree node to describe the DisplayPort connector, and
connect it to the DPSUB output.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 .../boot/dts/xilinx/zynqmp-zcu106-revA.dts    | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/arch/arm64/boot/dts/xilinx/zynqmp-zcu106-revA.dts b/arch/arm64/boot/dts/xilinx/zynqmp-zcu106-revA.dts
index eff7c6447087..aaf44ab256a1 100644
--- a/arch/arm64/boot/dts/xilinx/zynqmp-zcu106-revA.dts
+++ b/arch/arm64/boot/dts/xilinx/zynqmp-zcu106-revA.dts
@@ -146,6 +146,18 @@ refhdmi: refhdmi {
 		#clock-cells = <0>;
 		clock-frequency = <114285000>;
 	};
+
+	dpcon {
+		compatible = "dp-connector";
+		label = "P11";
+		type = "full-size";
+
+		port {
+			dpcon_in: endpoint {
+				remote-endpoint = <&dpsub_dp_out>;
+			};
+		};
+	};
 };
 
 &can1 {
@@ -165,6 +177,14 @@ &zynqmp_dpsub {
 	phy-names = "dp-phy0", "dp-phy1";
 	phys = <&psgtr 1 PHY_TYPE_DP 0 3>,
 	       <&psgtr 0 PHY_TYPE_DP 1 3>;
+
+	ports {
+		port@5 {
+			dpsub_dp_out: endpoint {
+				remote-endpoint = <&dpcon_in>;
+			};
+		};
+	};
 };
 
 /* fpd_dma clk 667MHz, lpd_dma 500MHz */
-- 
Regards,

Laurent Pinchart