[PATCH v3 0/5] Fixing live video input in ZynqMP DPSUB

[PATCH v3 0/5] Fixing live video input in ZynqMP DPSUB

[Anatoliy Klymenko]

Add few missing pieces to support ZynqMP DPSUB live video in mode.

ZynqMP DPSUB supports 2 modes of operations in regard to video data
input.
    
In the first mode, DPSUB uses DMA engine to pull video data from memory
buffers. To support this the driver implements CRTC and DRM bridge
representing DP encoder.
    
In the second mode, DPSUB acquires video data pushed from FPGA and 
passes it downstream to DP output. This mode of operation is modeled in
the driver as a DRM bridge that should be attached to some external
CRTC.

Patches 1/5,2/5,3/5,4/5 are minor fixes.

DPSUB requires input live video format to be configured.
Patch 5/5: The DP Subsystem requires the input live video format to be
configured. In this patch, we are assuming that the CRTC's bus format is
fixed (typical for FPGA CRTC) and comes from the device tree. This is a
proposed solution, as there is no API to query CRTC output bus format
or negotiate it in any other way.

Changes in v2: 
- Address reviewers' comments:
  - More elaborate and consistent comments / commit messages
  - Fix includes' order
  - Replace of_property_read_u32_index() with of_property_read_u32()

Changes in v3:
- Split patch #3 into 3) moving status register clear immediately after
  read; 4) masking status against interrupts' mask

Link to v1: https://lore.kernel.org/all/20240112234222.913138-1-anatoliy.klymenko@amd.com/
Link to v2: https://lore.kernel.org/all/20240119055437.2549149-1-anatoliy.klymenko@amd.com/

Anatoliy Klymenko (5):
  drm: xlnx: zynqmp_dpsub: Make drm bridge discoverable
  drm: xlnx: zynqmp_dpsub: Fix timing for live mode
  drm: xlnx: zynqmp_dpsub: Clear status register ASAP
  drm: xlnx: zynqmp_dpsub: Filter interrupts against mask
  drm: xlnx: zynqmp_dpsub: Set live video in format

 drivers/gpu/drm/xlnx/zynqmp_disp.c      | 111 +++++++++++++++++++++---
 drivers/gpu/drm/xlnx/zynqmp_disp.h      |   3 +-
 drivers/gpu/drm/xlnx/zynqmp_disp_regs.h |   8 +-
 drivers/gpu/drm/xlnx/zynqmp_dp.c        |  16 +++-
 drivers/gpu/drm/xlnx/zynqmp_kms.c       |   2 +-
 5 files changed, 119 insertions(+), 21 deletions(-)

-- 
2.25.1

[PATCH v3 1/5] drm: xlnx: zynqmp_dpsub: Make drm bridge discoverable

[Anatoliy Klymenko]

ZynqMP DPSUB supports 2 input modes: DMA based and live video.

In the first mode, the driver implements CRTC and DP encoder DRM bridge
to model the complete display pipeline. In this case, DRM bridge is
being directly instantiated within the driver, not using any bridge
discovery mechanisms.

In the live video input mode video signal is generated by FPGA fabric
and passed into DPSUB over the connected bus. In this mode driver
exposes the DP encoder as a DRM bridge, expecting external CRTC to
discover it via drm_of_find_panel_or_bridge() or a similar call. This
discovery relies on drm_bridge.of_node being properly set.

Assign device OF node to the bridge prior to registering it. This will
make said bridge discoverable by an external CRTC driver.

Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>
---
 drivers/gpu/drm/xlnx/zynqmp_dp.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/gpu/drm/xlnx/zynqmp_dp.c b/drivers/gpu/drm/xlnx/zynqmp_dp.c
index a0606fab0e22..d60b7431603f 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_dp.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_dp.c
@@ -1721,6 +1721,7 @@ int zynqmp_dp_probe(struct zynqmp_dpsub *dpsub)
 	bridge->ops = DRM_BRIDGE_OP_DETECT | DRM_BRIDGE_OP_EDID
 		    | DRM_BRIDGE_OP_HPD;
 	bridge->type = DRM_MODE_CONNECTOR_DisplayPort;
+	bridge->of_node = dp->dev->of_node;
 	dpsub->bridge = bridge;
 
 	/*
-- 
2.25.1

[PATCH v3 2/5] drm: xlnx: zynqmp_dpsub: Fix timing for live mode

[Anatoliy Klymenko]

Expect external video timing in live video input mode, program
DPSUB acordingly.

Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>
---
 drivers/gpu/drm/xlnx/zynqmp_disp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/xlnx/zynqmp_disp.c b/drivers/gpu/drm/xlnx/zynqmp_disp.c
index 407bc07cec69..8a39b3accce5 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_disp.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_disp.c
@@ -1166,7 +1166,7 @@ void zynqmp_disp_enable(struct zynqmp_disp *disp)
 	/* Choose clock source based on the DT clock handle. */
 	zynqmp_disp_avbuf_set_clocks_sources(disp, disp->dpsub->vid_clk_from_ps,
 					     disp->dpsub->aud_clk_from_ps,
-					     true);
+					     disp->dpsub->vid_clk_from_ps);
 	zynqmp_disp_avbuf_enable_channels(disp);
 	zynqmp_disp_avbuf_enable_audio(disp);
 
-- 
2.25.1

Re: [PATCH v3 2/5] drm: xlnx: zynqmp_dpsub: Fix timing for live mode

[Laurent Pinchart]

Hi Anatoliy,

Thank you for the patch.

On Tue, Jan 23, 2024 at 06:53:59PM -0800, Anatoliy Klymenko wrote:
> Expect external video timing in live video input mode, program
> DPSUB acordingly.

Are there no designs where the DPSUB operates in non-live mode, but uses
a video clock from the PL, for instance to use a different clock
frequency ?

I don't think that use case is very common, so I'm fine with this patch
in order to properly support the more common use case of live input, and
leave the PL clock without live input use case for later.

> 
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> 

No need for a blank line here.

> Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/gpu/drm/xlnx/zynqmp_disp.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/xlnx/zynqmp_disp.c b/drivers/gpu/drm/xlnx/zynqmp_disp.c
> index 407bc07cec69..8a39b3accce5 100644
> --- a/drivers/gpu/drm/xlnx/zynqmp_disp.c
> +++ b/drivers/gpu/drm/xlnx/zynqmp_disp.c
> @@ -1166,7 +1166,7 @@ void zynqmp_disp_enable(struct zynqmp_disp *disp)
>  	/* Choose clock source based on the DT clock handle. */
>  	zynqmp_disp_avbuf_set_clocks_sources(disp, disp->dpsub->vid_clk_from_ps,
>  					     disp->dpsub->aud_clk_from_ps,
> -					     true);
> +					     disp->dpsub->vid_clk_from_ps);
>  	zynqmp_disp_avbuf_enable_channels(disp);
>  	zynqmp_disp_avbuf_enable_audio(disp);
>  

-- 
Regards,

Laurent Pinchart

RE: [PATCH v3 2/5] drm: xlnx: zynqmp_dpsub: Fix timing for live mode

[Klymenko, Anatoliy]

Hi Laurent,

Thanks a lot for the review.

> -----Original Message-----
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Sent: Monday, February 5, 2024 12:08 AM
> To: Klymenko, Anatoliy <Anatoliy.Klymenko@amd.com>
> Cc: maarten.lankhorst@linux.intel.com; mripard@kernel.org;
> tzimmermann@suse.de; airlied@gmail.com; daniel@ffwll.ch; Simek, Michal
> <michal.simek@amd.com>; dri-devel@lists.freedesktop.org; linux-arm-
> kernel@lists.infradead.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v3 2/5] drm: xlnx: zynqmp_dpsub: Fix timing for live mode
> 
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
> 
> 
> Hi Anatoliy,
> 
> Thank you for the patch.
> 
> On Tue, Jan 23, 2024 at 06:53:59PM -0800, Anatoliy Klymenko wrote:
> > Expect external video timing in live video input mode, program DPSUB
> > acordingly.
> 
> Are there no designs where the DPSUB operates in non-live mode, but uses a
> video clock from the PL, for instance to use a different clock frequency ?
> 
> I don't think that use case is very common, so I'm fine with this patch in order to
> properly support the more common use case of live input, and leave the PL clock
> without live input use case for later.
> 
Theoretically, we can create such a design, but it wouldn't make too much sense since the PS clock is the best choice here. Probably, in some complicated scenarios like hybrid live-in/DMA or live-out, we would consider using the PL video clock, but the DPSUB driver is not supporting them yet.
> >
> > Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >
> 
> No need for a blank line here.
> 
Sure, I'll fix this in the new version. Thank you.
> > Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> > ---
> >  drivers/gpu/drm/xlnx/zynqmp_disp.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/gpu/drm/xlnx/zynqmp_disp.c
> > b/drivers/gpu/drm/xlnx/zynqmp_disp.c
> > index 407bc07cec69..8a39b3accce5 100644
> > --- a/drivers/gpu/drm/xlnx/zynqmp_disp.c
> > +++ b/drivers/gpu/drm/xlnx/zynqmp_disp.c
> > @@ -1166,7 +1166,7 @@ void zynqmp_disp_enable(struct zynqmp_disp *disp)
> >       /* Choose clock source based on the DT clock handle. */
> >       zynqmp_disp_avbuf_set_clocks_sources(disp, disp->dpsub-
> >vid_clk_from_ps,
> >                                            disp->dpsub->aud_clk_from_ps,
> > -                                          true);
> > +
> > + disp->dpsub->vid_clk_from_ps);
> >       zynqmp_disp_avbuf_enable_channels(disp);
> >       zynqmp_disp_avbuf_enable_audio(disp);
> >
> 
> --
> Regards,
> 
> Laurent Pinchart

[PATCH v3 3/5] drm: xlnx: zynqmp_dpsub: Clear status register ASAP

[Anatoliy Klymenko]

Clear status register as soon as we read it.

Addressing comments from
https://lore.kernel.org/dri-devel/beb551c7-bb7e-4cd0-b166-e9aad90c4620@ideasonboard.com/

Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>
---
 drivers/gpu/drm/xlnx/zynqmp_dp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/xlnx/zynqmp_dp.c b/drivers/gpu/drm/xlnx/zynqmp_dp.c
index d60b7431603f..5a3335e1fffa 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_dp.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_dp.c
@@ -1624,6 +1624,8 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
 	u32 status, mask;
 
 	status = zynqmp_dp_read(dp, ZYNQMP_DP_INT_STATUS);
+	/* clear status register as soon as we read it */
+	zynqmp_dp_write(dp, ZYNQMP_DP_INT_STATUS, status);
 	mask = zynqmp_dp_read(dp, ZYNQMP_DP_INT_MASK);
 	if (!(status & ~mask))
 		return IRQ_NONE;
@@ -1634,8 +1636,6 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
 	if (status & ZYNQMP_DP_INT_CHBUF_OVERFLW_MASK)
 		dev_dbg_ratelimited(dp->dev, "overflow interrupt\n");
 
-	zynqmp_dp_write(dp, ZYNQMP_DP_INT_STATUS, status);
-
 	if (status & ZYNQMP_DP_INT_VBLANK_START)
 		zynqmp_dpsub_drm_handle_vblank(dp->dpsub);
 
-- 
2.25.1

Re: [PATCH v3 3/5] drm: xlnx: zynqmp_dpsub: Clear status register ASAP

[Tomi Valkeinen]

On 24/01/2024 04:54, Anatoliy Klymenko wrote:
> Clear status register as soon as we read it.
> 
> Addressing comments from
> https://lore.kernel.org/dri-devel/beb551c7-bb7e-4cd0-b166-e9aad90c4620@ideasonboard.com/
> 
> Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>
> ---
>   drivers/gpu/drm/xlnx/zynqmp_dp.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xlnx/zynqmp_dp.c b/drivers/gpu/drm/xlnx/zynqmp_dp.c
> index d60b7431603f..5a3335e1fffa 100644
> --- a/drivers/gpu/drm/xlnx/zynqmp_dp.c
> +++ b/drivers/gpu/drm/xlnx/zynqmp_dp.c
> @@ -1624,6 +1624,8 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
>   	u32 status, mask;
>   
>   	status = zynqmp_dp_read(dp, ZYNQMP_DP_INT_STATUS);
> +	/* clear status register as soon as we read it */
> +	zynqmp_dp_write(dp, ZYNQMP_DP_INT_STATUS, status);
>   	mask = zynqmp_dp_read(dp, ZYNQMP_DP_INT_MASK);
>   	if (!(status & ~mask))
>   		return IRQ_NONE;
> @@ -1634,8 +1636,6 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
>   	if (status & ZYNQMP_DP_INT_CHBUF_OVERFLW_MASK)
>   		dev_dbg_ratelimited(dp->dev, "overflow interrupt\n");
>   
> -	zynqmp_dp_write(dp, ZYNQMP_DP_INT_STATUS, status);
> -
>   	if (status & ZYNQMP_DP_INT_VBLANK_START)
>   		zynqmp_dpsub_drm_handle_vblank(dp->dpsub);
>   

Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

  Tomi

Re: [PATCH v3 3/5] drm: xlnx: zynqmp_dpsub: Clear status register ASAP

[Laurent Pinchart]

Hi Anatoliy,

Thank you for the patch.

On Tue, Jan 23, 2024 at 06:54:00PM -0800, Anatoliy Klymenko wrote:
> Clear status register as soon as we read it.
> 
> Addressing comments from
> https://lore.kernel.org/dri-devel/beb551c7-bb7e-4cd0-b166-e9aad90c4620@ideasonboard.com/
> 
> Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>
> ---
>  drivers/gpu/drm/xlnx/zynqmp_dp.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xlnx/zynqmp_dp.c b/drivers/gpu/drm/xlnx/zynqmp_dp.c
> index d60b7431603f..5a3335e1fffa 100644
> --- a/drivers/gpu/drm/xlnx/zynqmp_dp.c
> +++ b/drivers/gpu/drm/xlnx/zynqmp_dp.c
> @@ -1624,6 +1624,8 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
>  	u32 status, mask;
>  
>  	status = zynqmp_dp_read(dp, ZYNQMP_DP_INT_STATUS);
> +	/* clear status register as soon as we read it */

I don't think a comment is strictly required, but I don't mind it.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +	zynqmp_dp_write(dp, ZYNQMP_DP_INT_STATUS, status);
>  	mask = zynqmp_dp_read(dp, ZYNQMP_DP_INT_MASK);
>  	if (!(status & ~mask))
>  		return IRQ_NONE;
> @@ -1634,8 +1636,6 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
>  	if (status & ZYNQMP_DP_INT_CHBUF_OVERFLW_MASK)
>  		dev_dbg_ratelimited(dp->dev, "overflow interrupt\n");
>  
> -	zynqmp_dp_write(dp, ZYNQMP_DP_INT_STATUS, status);
> -
>  	if (status & ZYNQMP_DP_INT_VBLANK_START)
>  		zynqmp_dpsub_drm_handle_vblank(dp->dpsub);
>  

-- 
Regards,

Laurent Pinchart

[PATCH v3 4/5] drm: xlnx: zynqmp_dpsub: Filter interrupts against mask

[Anatoliy Klymenko]

Filter out status register against the interrupts' mask.

Some events are being reported via DP status register, even if
corresponding interrupts have been disabled. One instance of such event
leads to generation of VBLANK when the driver is in DRM bridge mode,
which in turn results in NULL pointer dereferencing. We should avoid
processing such events in an interrupt handler context.

This problem is less noticeable when the driver operates in DMA mode, as
in this case we have DRM CRTC object instantiated and DRM framework
simply discards unwanted VBLANKs in drm_handle_vblank().

Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>
---
 drivers/gpu/drm/xlnx/zynqmp_dp.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/xlnx/zynqmp_dp.c b/drivers/gpu/drm/xlnx/zynqmp_dp.c
index 5a3335e1fffa..9f48e5bbcdec 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_dp.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_dp.c
@@ -1627,7 +1627,14 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
 	/* clear status register as soon as we read it */
 	zynqmp_dp_write(dp, ZYNQMP_DP_INT_STATUS, status);
 	mask = zynqmp_dp_read(dp, ZYNQMP_DP_INT_MASK);
-	if (!(status & ~mask))
+
+	/*
+	 * Status register may report some events, which corresponding interrupts
+	 * have been disabled. Filter out those events against interrupts' mask.
+	 */
+	status &= ~mask;
+
+	if (!status)
 		return IRQ_NONE;
 
 	/* dbg for diagnostic, but not much that the driver can do */
-- 
2.25.1

Re: [PATCH v3 4/5] drm: xlnx: zynqmp_dpsub: Filter interrupts against mask

[Tomi Valkeinen]

On 24/01/2024 04:54, Anatoliy Klymenko wrote:
> Filter out status register against the interrupts' mask.
> 
> Some events are being reported via DP status register, even if
> corresponding interrupts have been disabled. One instance of such event
> leads to generation of VBLANK when the driver is in DRM bridge mode,
> which in turn results in NULL pointer dereferencing. We should avoid
> processing such events in an interrupt handler context.
> 
> This problem is less noticeable when the driver operates in DMA mode, as
> in this case we have DRM CRTC object instantiated and DRM framework
> simply discards unwanted VBLANKs in drm_handle_vblank().
> 
> Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>
> ---
>   drivers/gpu/drm/xlnx/zynqmp_dp.c | 9 ++++++++-
>   1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/xlnx/zynqmp_dp.c b/drivers/gpu/drm/xlnx/zynqmp_dp.c
> index 5a3335e1fffa..9f48e5bbcdec 100644
> --- a/drivers/gpu/drm/xlnx/zynqmp_dp.c
> +++ b/drivers/gpu/drm/xlnx/zynqmp_dp.c
> @@ -1627,7 +1627,14 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
>   	/* clear status register as soon as we read it */
>   	zynqmp_dp_write(dp, ZYNQMP_DP_INT_STATUS, status);
>   	mask = zynqmp_dp_read(dp, ZYNQMP_DP_INT_MASK);
> -	if (!(status & ~mask))
> +
> +	/*
> +	 * Status register may report some events, which corresponding interrupts
> +	 * have been disabled. Filter out those events against interrupts' mask.
> +	 */
> +	status &= ~mask;
> +
> +	if (!status)
>   		return IRQ_NONE;
>   
>   	/* dbg for diagnostic, but not much that the driver can do */

Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

  Tomi

Re: [PATCH v3 4/5] drm: xlnx: zynqmp_dpsub: Filter interrupts against mask

[Laurent Pinchart]

Hi Anatoliy,

Thank you for the patch.

On Tue, Jan 23, 2024 at 06:54:01PM -0800, Anatoliy Klymenko wrote:
> Filter out status register against the interrupts' mask.
> 
> Some events are being reported via DP status register, even if
> corresponding interrupts have been disabled. One instance of such event
> leads to generation of VBLANK when the driver is in DRM bridge mode,
> which in turn results in NULL pointer dereferencing. We should avoid
> processing such events in an interrupt handler context.
> 
> This problem is less noticeable when the driver operates in DMA mode, as
> in this case we have DRM CRTC object instantiated and DRM framework
> simply discards unwanted VBLANKs in drm_handle_vblank().
> 
> Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/gpu/drm/xlnx/zynqmp_dp.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/xlnx/zynqmp_dp.c b/drivers/gpu/drm/xlnx/zynqmp_dp.c
> index 5a3335e1fffa..9f48e5bbcdec 100644
> --- a/drivers/gpu/drm/xlnx/zynqmp_dp.c
> +++ b/drivers/gpu/drm/xlnx/zynqmp_dp.c
> @@ -1627,7 +1627,14 @@ static irqreturn_t zynqmp_dp_irq_handler(int irq, void *data)
>  	/* clear status register as soon as we read it */
>  	zynqmp_dp_write(dp, ZYNQMP_DP_INT_STATUS, status);
>  	mask = zynqmp_dp_read(dp, ZYNQMP_DP_INT_MASK);
> -	if (!(status & ~mask))
> +
> +	/*
> +	 * Status register may report some events, which corresponding interrupts
> +	 * have been disabled. Filter out those events against interrupts' mask.
> +	 */
> +	status &= ~mask;
> +
> +	if (!status)
>  		return IRQ_NONE;
>  
>  	/* dbg for diagnostic, but not much that the driver can do */

-- 
Regards,

Laurent Pinchart

[PATCH v3 5/5] drm: xlnx: zynqmp_dpsub: Set live video in format

[Anatoliy Klymenko]

ZynqMP DPSUB supports 2 modes of operations in regard to video data
input.

In the first mode, DPSUB uses DMA engine to pull video data from memory
buffers. To support this the driver implements CRTC and DRM bridge
representing DP encoder.

In the second mode, DPSUB acquires video data pushed from FPGA and
passes it downstream to DP output. This mode of operation is modeled in
the driver as a DRM bridge that should be attached to some external
CRTC. DPSUB supports multiple input data formats. In order to properly
operate exact media bus format should be negotiated between external
CRTC and DPSUB bridge. DRM framework provides a mechanism to negotiate
media bus formats between bridges connected into a chain, but not
between CRTC and encoder (first bridge in the chain). This change
mitigates the issue for FPGA based CRTC, which would typically have a
single fixed media bus format.

Expect live video input format to be set as "bus-format" property in
connected remote endpoint.

Set display layer mode in the layer creation context.

Signed-off-by: Anatoliy Klymenko <anatoliy.klymenko@amd.com>
---
 drivers/gpu/drm/xlnx/zynqmp_disp.c      | 109 ++++++++++++++++++++++--
 drivers/gpu/drm/xlnx/zynqmp_disp.h      |   3 +-
 drivers/gpu/drm/xlnx/zynqmp_disp_regs.h |   8 +-
 drivers/gpu/drm/xlnx/zynqmp_dp.c        |   2 +-
 drivers/gpu/drm/xlnx/zynqmp_kms.c       |   2 +-
 5 files changed, 107 insertions(+), 17 deletions(-)

diff --git a/drivers/gpu/drm/xlnx/zynqmp_disp.c b/drivers/gpu/drm/xlnx/zynqmp_disp.c
index 8a39b3accce5..a7115321b3fb 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_disp.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_disp.c
@@ -18,8 +18,10 @@
 #include <linux/dma/xilinx_dpdma.h>
 #include <linux/dma-mapping.h>
 #include <linux/dmaengine.h>
+#include <linux/media-bus-format.h>
 #include <linux/module.h>
 #include <linux/of.h>
+#include <linux/of_graph.h>
 #include <linux/platform_device.h>
 #include <linux/slab.h>
 
@@ -67,12 +69,16 @@
 /**
  * struct zynqmp_disp_format - Display subsystem format information
  * @drm_fmt: DRM format (4CC)
+ * @bus_fmt: Live video media bus format
  * @buf_fmt: AV buffer format
  * @swap: Flag to swap R & B for RGB formats, and U & V for YUV formats
  * @sf: Scaling factors for color components
  */
 struct zynqmp_disp_format {
-	u32 drm_fmt;
+	union {
+		u32 drm_fmt;
+		u32 bus_fmt;
+	};
 	u32 buf_fmt;
 	bool swap;
 	const u32 *sf;
@@ -354,6 +360,16 @@ static const struct zynqmp_disp_format avbuf_gfx_fmts[] = {
 	},
 };
 
+/* TODO: add support for different formats */
+static const struct zynqmp_disp_format avbuf_live_vid_fmts[] = {
+	{
+		.bus_fmt	= MEDIA_BUS_FMT_UYVY8_1X16,
+		.buf_fmt	= ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_BPC_8 |
+				  ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_FMT_YUV422,
+		.sf		= scaling_factors_888,
+	}
+};
+
 static u32 zynqmp_disp_avbuf_read(struct zynqmp_disp *disp, int reg)
 {
 	return readl(disp->avbuf.base + reg);
@@ -369,6 +385,34 @@ static bool zynqmp_disp_layer_is_video(const struct zynqmp_disp_layer *layer)
 	return layer->id == ZYNQMP_DPSUB_LAYER_VID;
 }
 
+/**
+ * zynqmp_disp_avbuf_set_live_format - Set live input format for a layer
+ * @disp: Display controller
+ * @layer: The layer
+ * @fmt: The format information
+ *
+ * Set the live video input format for @layer to @fmt.
+ */
+static void zynqmp_disp_avbuf_set_live_format(struct zynqmp_disp *disp,
+					      struct zynqmp_disp_layer *layer,
+					      const struct zynqmp_disp_format *fmt)
+{
+	u32 reg, i;
+
+	reg = zynqmp_disp_layer_is_video(layer)
+	    ? ZYNQMP_DISP_AV_BUF_LIVE_VID_CONFIG
+	    : ZYNQMP_DISP_AV_BUF_LIVE_GFX_CONFIG;
+	zynqmp_disp_avbuf_write(disp, reg, fmt->buf_fmt);
+
+	for (i = 0; i < ZYNQMP_DISP_AV_BUF_NUM_SF; ++i) {
+		reg = zynqmp_disp_layer_is_video(layer)
+		    ? ZYNQMP_DISP_AV_BUF_LIVD_VID_COMP_SF(i)
+		    : ZYNQMP_DISP_AV_BUF_LIVD_GFX_COMP_SF(i);
+		zynqmp_disp_avbuf_write(disp, reg, fmt->sf[i]);
+	}
+	layer->disp_fmt = fmt;
+}
+
 /**
  * zynqmp_disp_avbuf_set_format - Set the input format for a layer
  * @disp: Display controller
@@ -902,15 +946,12 @@ u32 *zynqmp_disp_layer_drm_formats(struct zynqmp_disp_layer *layer,
 /**
  * zynqmp_disp_layer_enable - Enable a layer
  * @layer: The layer
- * @mode: Operating mode of layer
  *
  * Enable the @layer in the audio/video buffer manager and the blender. DMA
  * channels are started separately by zynqmp_disp_layer_update().
  */
-void zynqmp_disp_layer_enable(struct zynqmp_disp_layer *layer,
-			      enum zynqmp_dpsub_layer_mode mode)
+void zynqmp_disp_layer_enable(struct zynqmp_disp_layer *layer)
 {
-	layer->mode = mode;
 	zynqmp_disp_avbuf_enable_video(layer->disp, layer);
 	zynqmp_disp_blend_layer_enable(layer->disp, layer);
 }
@@ -950,11 +991,12 @@ void zynqmp_disp_layer_set_format(struct zynqmp_disp_layer *layer,
 	layer->disp_fmt = zynqmp_disp_layer_find_format(layer, info->format);
 	layer->drm_fmt = info;
 
-	zynqmp_disp_avbuf_set_format(layer->disp, layer, layer->disp_fmt);
-
-	if (!layer->disp->dpsub->dma_enabled)
+	/* Live format set during layer creation */
+	if (layer->mode == ZYNQMP_DPSUB_LAYER_LIVE)
 		return;
 
+	zynqmp_disp_avbuf_set_format(layer->disp, layer, layer->disp_fmt);
+
 	/*
 	 * Set pconfig for each DMA channel to indicate they're part of a
 	 * video group.
@@ -1083,7 +1125,7 @@ static int zynqmp_disp_layer_request_dma(struct zynqmp_disp *disp,
 	unsigned int i;
 	int ret;
 
-	if (!disp->dpsub->dma_enabled)
+	if (layer->mode == ZYNQMP_DPSUB_LAYER_LIVE)
 		return 0;
 
 	for (i = 0; i < layer->info->num_channels; i++) {
@@ -1104,6 +1146,43 @@ static int zynqmp_disp_layer_request_dma(struct zynqmp_disp *disp,
 	return 0;
 }
 
+/**
+ * zynqmp_disp_get_live_fmt - Get live video format
+ * @disp: Display controller
+ * @layer: Display layer
+ *
+ * Parse connected remote endpoint and retrieve configured bus-format
+ *
+ * Return: live format pointer on success, NULL otherwise
+ */
+static const struct zynqmp_disp_format *zynqmp_disp_get_live_fmt(struct zynqmp_disp *disp,
+								 struct zynqmp_disp_layer *layer)
+{
+	struct device_node *local, *remote, *dpsub = disp->dev->of_node;
+	int rc, i;
+	u32 fmt;
+
+	local = of_graph_get_endpoint_by_regs(dpsub, layer->id, -1);
+	if (!local)
+		return NULL;
+
+	remote = of_graph_get_remote_endpoint(local);
+	of_node_put(local);
+	if (!remote)
+		return NULL;
+
+	rc = of_property_read_u32(remote, "bus-format", &fmt);
+	of_node_put(remote);
+	if (rc)
+		return NULL;
+
+	for (i = 0; i < ARRAY_SIZE(avbuf_live_vid_fmts); ++i)
+		if (avbuf_live_vid_fmts[i].bus_fmt == fmt)
+			return &avbuf_live_vid_fmts[i];
+
+	return NULL;
+}
+
 /**
  * zynqmp_disp_create_layers - Create and initialize all layers
  * @disp: Display controller
@@ -1130,9 +1209,15 @@ static int zynqmp_disp_create_layers(struct zynqmp_disp *disp)
 
 	for (i = 0; i < ARRAY_SIZE(disp->layers); i++) {
 		struct zynqmp_disp_layer *layer = &disp->layers[i];
+		const struct zynqmp_disp_format *disp_fmt;
 
 		layer->id = i;
 		layer->disp = disp;
+		/* For now we assume dpsub works in either live or non-live mode for both layers.
+		 * Hybrid mode is not supported yet.
+		 */
+		layer->mode = disp->dpsub->dma_enabled ? ZYNQMP_DPSUB_LAYER_NONLIVE
+						       : ZYNQMP_DPSUB_LAYER_LIVE;
 		layer->info = &layer_info[i];
 
 		ret = zynqmp_disp_layer_request_dma(disp, layer);
@@ -1140,6 +1225,12 @@ static int zynqmp_disp_create_layers(struct zynqmp_disp *disp)
 			goto err;
 
 		disp->dpsub->layers[i] = layer;
+
+		if (layer->mode == ZYNQMP_DPSUB_LAYER_LIVE) {
+			disp_fmt = zynqmp_disp_get_live_fmt(disp, layer);
+			if (disp_fmt)
+				zynqmp_disp_avbuf_set_live_format(disp, layer, disp_fmt);
+		}
 	}
 
 	return 0;
diff --git a/drivers/gpu/drm/xlnx/zynqmp_disp.h b/drivers/gpu/drm/xlnx/zynqmp_disp.h
index 123cffac08be..f3357b2d5c09 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_disp.h
+++ b/drivers/gpu/drm/xlnx/zynqmp_disp.h
@@ -62,8 +62,7 @@ void zynqmp_disp_blend_set_global_alpha(struct zynqmp_disp *disp,
 
 u32 *zynqmp_disp_layer_drm_formats(struct zynqmp_disp_layer *layer,
 				   unsigned int *num_formats);
-void zynqmp_disp_layer_enable(struct zynqmp_disp_layer *layer,
-			      enum zynqmp_dpsub_layer_mode mode);
+void zynqmp_disp_layer_enable(struct zynqmp_disp_layer *layer);
 void zynqmp_disp_layer_disable(struct zynqmp_disp_layer *layer);
 void zynqmp_disp_layer_set_format(struct zynqmp_disp_layer *layer,
 				  const struct drm_format_info *info);
diff --git a/drivers/gpu/drm/xlnx/zynqmp_disp_regs.h b/drivers/gpu/drm/xlnx/zynqmp_disp_regs.h
index f92a006d5070..926e07c255bb 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_disp_regs.h
+++ b/drivers/gpu/drm/xlnx/zynqmp_disp_regs.h
@@ -165,10 +165,10 @@
 #define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_BPC_10		0x2
 #define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_BPC_12		0x3
 #define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_BPC_MASK		GENMASK(2, 0)
-#define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_FMT_RGB		0x0
-#define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_FMT_YUV444	0x1
-#define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_FMT_YUV422	0x2
-#define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_FMT_YONLY	0x3
+#define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_FMT_RGB		0x00
+#define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_FMT_YUV444	0x10
+#define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_FMT_YUV422	0x20
+#define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_FMT_YONLY	0x30
 #define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_FMT_MASK		GENMASK(5, 4)
 #define ZYNQMP_DISP_AV_BUF_LIVE_CONFIG_CB_FIRST		BIT(8)
 #define ZYNQMP_DISP_AV_BUF_PALETTE_MEMORY		0x400
diff --git a/drivers/gpu/drm/xlnx/zynqmp_dp.c b/drivers/gpu/drm/xlnx/zynqmp_dp.c
index 9f48e5bbcdec..6fd7eab0f6db 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_dp.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_dp.c
@@ -1295,7 +1295,7 @@ static void zynqmp_dp_disp_enable(struct zynqmp_dp *dp,
 	/* TODO: Make the format configurable. */
 	info = drm_format_info(DRM_FORMAT_YUV422);
 	zynqmp_disp_layer_set_format(layer, info);
-	zynqmp_disp_layer_enable(layer, ZYNQMP_DPSUB_LAYER_LIVE);
+	zynqmp_disp_layer_enable(layer);
 
 	if (layer_id == ZYNQMP_DPSUB_LAYER_GFX)
 		zynqmp_disp_blend_set_global_alpha(dp->dpsub->disp, true, 255);
diff --git a/drivers/gpu/drm/xlnx/zynqmp_kms.c b/drivers/gpu/drm/xlnx/zynqmp_kms.c
index db3bb4afbfc4..43bf416b33d5 100644
--- a/drivers/gpu/drm/xlnx/zynqmp_kms.c
+++ b/drivers/gpu/drm/xlnx/zynqmp_kms.c
@@ -122,7 +122,7 @@ static void zynqmp_dpsub_plane_atomic_update(struct drm_plane *plane,
 
 	/* Enable or re-enable the plane if the format has changed. */
 	if (format_changed)
-		zynqmp_disp_layer_enable(layer, ZYNQMP_DPSUB_LAYER_NONLIVE);
+		zynqmp_disp_layer_enable(layer);
 }
 
 static const struct drm_plane_helper_funcs zynqmp_dpsub_plane_helper_funcs = {
-- 
2.25.1

Re: [PATCH v3 0/5] Fixing live video input in ZynqMP DPSUB

[Laurent Pinchart]

Hello,

This series looks good. Tomi, could you get it merged through drm-misc ?

On Tue, Jan 23, 2024 at 06:53:57PM -0800, Anatoliy Klymenko wrote:
> Add few missing pieces to support ZynqMP DPSUB live video in mode.
> 
> ZynqMP DPSUB supports 2 modes of operations in regard to video data
> input.
>     
> In the first mode, DPSUB uses DMA engine to pull video data from memory
> buffers. To support this the driver implements CRTC and DRM bridge
> representing DP encoder.
>     
> In the second mode, DPSUB acquires video data pushed from FPGA and 
> passes it downstream to DP output. This mode of operation is modeled in
> the driver as a DRM bridge that should be attached to some external
> CRTC.
> 
> Patches 1/5,2/5,3/5,4/5 are minor fixes.
> 
> DPSUB requires input live video format to be configured.
> Patch 5/5: The DP Subsystem requires the input live video format to be
> configured. In this patch, we are assuming that the CRTC's bus format is
> fixed (typical for FPGA CRTC) and comes from the device tree. This is a
> proposed solution, as there is no API to query CRTC output bus format
> or negotiate it in any other way.
> 
> Changes in v2: 
> - Address reviewers' comments:
>   - More elaborate and consistent comments / commit messages
>   - Fix includes' order
>   - Replace of_property_read_u32_index() with of_property_read_u32()
> 
> Changes in v3:
> - Split patch #3 into 3) moving status register clear immediately after
>   read; 4) masking status against interrupts' mask
> 
> Link to v1: https://lore.kernel.org/all/20240112234222.913138-1-anatoliy.klymenko@amd.com/
> Link to v2: https://lore.kernel.org/all/20240119055437.2549149-1-anatoliy.klymenko@amd.com/
> 
> Anatoliy Klymenko (5):
>   drm: xlnx: zynqmp_dpsub: Make drm bridge discoverable
>   drm: xlnx: zynqmp_dpsub: Fix timing for live mode
>   drm: xlnx: zynqmp_dpsub: Clear status register ASAP
>   drm: xlnx: zynqmp_dpsub: Filter interrupts against mask
>   drm: xlnx: zynqmp_dpsub: Set live video in format
> 
>  drivers/gpu/drm/xlnx/zynqmp_disp.c      | 111 +++++++++++++++++++++---
>  drivers/gpu/drm/xlnx/zynqmp_disp.h      |   3 +-
>  drivers/gpu/drm/xlnx/zynqmp_disp_regs.h |   8 +-
>  drivers/gpu/drm/xlnx/zynqmp_dp.c        |  16 +++-
>  drivers/gpu/drm/xlnx/zynqmp_kms.c       |   2 +-
>  5 files changed, 119 insertions(+), 21 deletions(-)

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 0/5] Fixing live video input in ZynqMP DPSUB

[Laurent Pinchart]

On Mon, Feb 05, 2024 at 09:29:09AM +0200, Laurent Pinchart wrote:
> Hello,
> 
> This series looks good. Tomi, could you get it merged through drm-misc ?

I got things mixed up, sorry. Patches 1/5 to 4/5 are fine, but 5/5 needs
a different approach. I've reviewed the first four patches, which I
think are fine and can be applied already.

> On Tue, Jan 23, 2024 at 06:53:57PM -0800, Anatoliy Klymenko wrote:
> > Add few missing pieces to support ZynqMP DPSUB live video in mode.
> > 
> > ZynqMP DPSUB supports 2 modes of operations in regard to video data
> > input.
> >     
> > In the first mode, DPSUB uses DMA engine to pull video data from memory
> > buffers. To support this the driver implements CRTC and DRM bridge
> > representing DP encoder.
> >     
> > In the second mode, DPSUB acquires video data pushed from FPGA and 
> > passes it downstream to DP output. This mode of operation is modeled in
> > the driver as a DRM bridge that should be attached to some external
> > CRTC.
> > 
> > Patches 1/5,2/5,3/5,4/5 are minor fixes.
> > 
> > DPSUB requires input live video format to be configured.
> > Patch 5/5: The DP Subsystem requires the input live video format to be
> > configured. In this patch, we are assuming that the CRTC's bus format is
> > fixed (typical for FPGA CRTC) and comes from the device tree. This is a
> > proposed solution, as there is no API to query CRTC output bus format
> > or negotiate it in any other way.
> > 
> > Changes in v2: 
> > - Address reviewers' comments:
> >   - More elaborate and consistent comments / commit messages
> >   - Fix includes' order
> >   - Replace of_property_read_u32_index() with of_property_read_u32()
> > 
> > Changes in v3:
> > - Split patch #3 into 3) moving status register clear immediately after
> >   read; 4) masking status against interrupts' mask
> > 
> > Link to v1: https://lore.kernel.org/all/20240112234222.913138-1-anatoliy.klymenko@amd.com/
> > Link to v2: https://lore.kernel.org/all/20240119055437.2549149-1-anatoliy.klymenko@amd.com/
> > 
> > Anatoliy Klymenko (5):
> >   drm: xlnx: zynqmp_dpsub: Make drm bridge discoverable
> >   drm: xlnx: zynqmp_dpsub: Fix timing for live mode
> >   drm: xlnx: zynqmp_dpsub: Clear status register ASAP
> >   drm: xlnx: zynqmp_dpsub: Filter interrupts against mask
> >   drm: xlnx: zynqmp_dpsub: Set live video in format
> > 
> >  drivers/gpu/drm/xlnx/zynqmp_disp.c      | 111 +++++++++++++++++++++---
> >  drivers/gpu/drm/xlnx/zynqmp_disp.h      |   3 +-
> >  drivers/gpu/drm/xlnx/zynqmp_disp_regs.h |   8 +-
> >  drivers/gpu/drm/xlnx/zynqmp_dp.c        |  16 +++-
> >  drivers/gpu/drm/xlnx/zynqmp_kms.c       |   2 +-
> >  5 files changed, 119 insertions(+), 21 deletions(-)

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 0/5] Fixing live video input in ZynqMP DPSUB

[Tomi Valkeinen]

On 24/01/2024 04:53, Anatoliy Klymenko wrote:
> Add few missing pieces to support ZynqMP DPSUB live video in mode.
> 
> ZynqMP DPSUB supports 2 modes of operations in regard to video data
> input.
>      
> In the first mode, DPSUB uses DMA engine to pull video data from memory
> buffers. To support this the driver implements CRTC and DRM bridge
> representing DP encoder.
>      
> In the second mode, DPSUB acquires video data pushed from FPGA and
> passes it downstream to DP output. This mode of operation is modeled in
> the driver as a DRM bridge that should be attached to some external
> CRTC.
> 
> Patches 1/5,2/5,3/5,4/5 are minor fixes.
> 
> DPSUB requires input live video format to be configured.
> Patch 5/5: The DP Subsystem requires the input live video format to be
> configured. In this patch, we are assuming that the CRTC's bus format is
> fixed (typical for FPGA CRTC) and comes from the device tree. This is a
> proposed solution, as there is no API to query CRTC output bus format
> or negotiate it in any other way.
> 
> Changes in v2:
> - Address reviewers' comments:
>    - More elaborate and consistent comments / commit messages
>    - Fix includes' order
>    - Replace of_property_read_u32_index() with of_property_read_u32()
> 
> Changes in v3:
> - Split patch #3 into 3) moving status register clear immediately after
>    read; 4) masking status against interrupts' mask
> 
> Link to v1: https://lore.kernel.org/all/20240112234222.913138-1-anatoliy.klymenko@amd.com/
> Link to v2: https://lore.kernel.org/all/20240119055437.2549149-1-anatoliy.klymenko@amd.com/
> 
> Anatoliy Klymenko (5):
>    drm: xlnx: zynqmp_dpsub: Make drm bridge discoverable
>    drm: xlnx: zynqmp_dpsub: Fix timing for live mode
>    drm: xlnx: zynqmp_dpsub: Clear status register ASAP
>    drm: xlnx: zynqmp_dpsub: Filter interrupts against mask
>    drm: xlnx: zynqmp_dpsub: Set live video in format
> 
>   drivers/gpu/drm/xlnx/zynqmp_disp.c      | 111 +++++++++++++++++++++---
>   drivers/gpu/drm/xlnx/zynqmp_disp.h      |   3 +-
>   drivers/gpu/drm/xlnx/zynqmp_disp_regs.h |   8 +-
>   drivers/gpu/drm/xlnx/zynqmp_dp.c        |  16 +++-
>   drivers/gpu/drm/xlnx/zynqmp_kms.c       |   2 +-
>   5 files changed, 119 insertions(+), 21 deletions(-)
> 

Thanks! I have pushed patches 1 to 4 to drm-misc-next. As Laurent said, 
the fifth one needs some more work.

  Tomi