[PATCH 00/11] media: renesas: rcar-csi2: Use the subdev active state

[PATCH 00/11] media: renesas: rcar-csi2: Use the subdev active state

[Jacopo Mondi]

As a follow-up to the recently sent
"media: renesas: rcar-csi2: Support multiple streams" series, this smaller
version collects some fixes and implement usage of the subdev active state
to simplify the R-Car CSI-2, ADV748x and MAX9286 drivers implementations.

Below the possibly more controversial points:

- R-Car CSI-2

There's a user-visible change here, as in the past it was possible to set
format on all pads, and the vin-test scripts do that. In this version I decided
to make the format configurable on the sink and propagate to sources. I'm not
sure if this can be considered a user visible change and we need to maintain
backward compatibility

- ADV748x

Add a list of supported formats and validated them in s_fmt. The ADV748x driver
configures the chip to automatically detect the output format based on what the
front-end uses, so setting a format doesn't actually modify the output image
formats. Regardless I presume making sure the format is actually supported is a
good thing.

- MAX9286

The driver uses frame_interval = 0/0 to detect if automatic frame sync detection
should be used. For this reason I initialized the frame interval of the active
state to 0/0 which might sound weird.

Tested with GMSL on Eagle V3M
Tested with HDMI on Salvator-X

Jacopo Mondi (11):
  media: rcar-vin: Fix YUYV8_1X16 handling for CSI-2
  media: rcar-csi2: Disable runtime_pm in probe error
  media: rcar-csi2: Cleanup subdevice in remove()
  media: rcar-csi2: Use the subdev active state
  media: adv748x-csi2: Initialize subdev format
  media: adv748x-csi2: Implement enum_mbus_codes
  media: adv748x-csi2: Validate the image format
  media: adv748x-csi2: Use the subdev active state
  media: max9286: Fix enum_mbus_code
  media: max9286: Use the subdev active state
  media: max9286: Use frame interval from subdev state

 drivers/media/i2c/adv748x/adv748x-csi2.c      | 141 ++++++++-----
 drivers/media/i2c/adv748x/adv748x.h           |   1 -
 drivers/media/i2c/max9286.c                   | 185 +++++++-----------
 drivers/media/platform/renesas/rcar-csi2.c    | 141 +++++++------
 .../platform/renesas/rcar-vin/rcar-dma.c      |  16 +-
 5 files changed, 251 insertions(+), 233 deletions(-)

--
2.44.0

[PATCH 01/11] media: rcar-vin: Fix YUYV8_1X16 handling for CSI-2

[Jacopo Mondi]

The YUYV8_1X16 and UYVY8_1X16 formats are treated as 'ITU-R
BT.601/BT.1358 16-bit YCbCr-422 input' (YUV16 - 0x5) in the R-Car VIN
driver and are thus disallowed when capturing frames from the R-Car
CSI-2 interface according to the hardware manual.

As the 1X16 format variants are meant to be used with serial busses they
have to be treated as 'YCbCr-422 8-bit data input' (0x1) when capturing
from CSI-2, which is a valid setting for CSI-2.

Commit 78b3f9d75a62 ("media: rcar-vin: Add check that input interface
and format are valid") disallowed capturing YUV16 when using the CSI-2
interface. Fix this by using YUV8_BT601 for YCbCr422 when CSI-2 is in
use.

Fixes: 78b3f9d75a62 ("media: rcar-vin: Add check that input interface and format are valid")
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 .../media/platform/renesas/rcar-vin/rcar-dma.c   | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-vin/rcar-dma.c b/drivers/media/platform/renesas/rcar-vin/rcar-dma.c
index e2c40abc6d3d..21d5b2815e86 100644
--- a/drivers/media/platform/renesas/rcar-vin/rcar-dma.c
+++ b/drivers/media/platform/renesas/rcar-vin/rcar-dma.c
@@ -742,12 +742,22 @@ static int rvin_setup(struct rvin_dev *vin)
 	 */
 	switch (vin->mbus_code) {
 	case MEDIA_BUS_FMT_YUYV8_1X16:
-		/* BT.601/BT.1358 16bit YCbCr422 */
-		vnmc |= VNMC_INF_YUV16;
+		if (vin->is_csi)
+			/* YCbCr422 8-bit */
+			vnmc |= VNMC_INF_YUV8_BT601;
+		else
+			/* BT.601/BT.1358 16bit YCbCr422 */
+			vnmc |= VNMC_INF_YUV16;
 		input_is_yuv = true;
 		break;
 	case MEDIA_BUS_FMT_UYVY8_1X16:
-		vnmc |= VNMC_INF_YUV16 | VNMC_YCAL;
+		if (vin->is_csi)
+			/* YCbCr422 8-bit */
+			vnmc |= VNMC_INF_YUV8_BT601;
+		else
+			/* BT.601/BT.1358 16bit YCbCr422 */
+			vnmc |= VNMC_INF_YUV16;
+		vnmc |= VNMC_YCAL;
 		input_is_yuv = true;
 		break;
 	case MEDIA_BUS_FMT_UYVY8_2X8:
-- 
2.44.0

Re: [PATCH 01/11] media: rcar-vin: Fix YUYV8_1X16 handling for CSI-2

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:16PM +0200, Jacopo Mondi wrote:
> The YUYV8_1X16 and UYVY8_1X16 formats are treated as 'ITU-R
> BT.601/BT.1358 16-bit YCbCr-422 input' (YUV16 - 0x5) in the R-Car VIN
> driver and are thus disallowed when capturing frames from the R-Car
> CSI-2 interface according to the hardware manual.
> 
> As the 1X16 format variants are meant to be used with serial busses they
> have to be treated as 'YCbCr-422 8-bit data input' (0x1) when capturing
> from CSI-2, which is a valid setting for CSI-2.
> 
> Commit 78b3f9d75a62 ("media: rcar-vin: Add check that input interface
> and format are valid") disallowed capturing YUV16 when using the CSI-2
> interface. Fix this by using YUV8_BT601 for YCbCr422 when CSI-2 is in
> use.
> 
> Fixes: 78b3f9d75a62 ("media: rcar-vin: Add check that input interface and format are valid")
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  .../media/platform/renesas/rcar-vin/rcar-dma.c   | 16 +++++++++++++---
>  1 file changed, 13 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/platform/renesas/rcar-vin/rcar-dma.c b/drivers/media/platform/renesas/rcar-vin/rcar-dma.c
> index e2c40abc6d3d..21d5b2815e86 100644
> --- a/drivers/media/platform/renesas/rcar-vin/rcar-dma.c
> +++ b/drivers/media/platform/renesas/rcar-vin/rcar-dma.c
> @@ -742,12 +742,22 @@ static int rvin_setup(struct rvin_dev *vin)
>  	 */
>  	switch (vin->mbus_code) {
>  	case MEDIA_BUS_FMT_YUYV8_1X16:
> -		/* BT.601/BT.1358 16bit YCbCr422 */
> -		vnmc |= VNMC_INF_YUV16;
> +		if (vin->is_csi)
> +			/* YCbCr422 8-bit */
> +			vnmc |= VNMC_INF_YUV8_BT601;
> +		else
> +			/* BT.601/BT.1358 16bit YCbCr422 */
> +			vnmc |= VNMC_INF_YUV16;
>  		input_is_yuv = true;
>  		break;
>  	case MEDIA_BUS_FMT_UYVY8_1X16:
> -		vnmc |= VNMC_INF_YUV16 | VNMC_YCAL;
> +		if (vin->is_csi)
> +			/* YCbCr422 8-bit */
> +			vnmc |= VNMC_INF_YUV8_BT601;
> +		else
> +			/* BT.601/BT.1358 16bit YCbCr422 */
> +			vnmc |= VNMC_INF_YUV16;
> +		vnmc |= VNMC_YCAL;

You could also write

	case MEDIA_BUS_FMT_UYVY8_1X16:
		vnmc |= VNMC_YCAL;
		fallthrough;
	case MEDIA_BUS_FMT_YUYV8_1X16:
		if (vin->is_csi)
			/* YCbCr422 8-bit */
			vnmc |= VNMC_INF_YUV8_BT601;
		else
			/* BT.601/BT.1358 16bit YCbCr422 */
			vnmc |= VNMC_INF_YUV16;
		input_is_yuv = true;
		break;

Up to you.

On a side note, CSI-2 isn't supposed to support
MEDIA_BUS_FMT_YUYV8_1X16. The native format is MEDIA_BUS_FMT_UYVY8_1X16.
I wonder if we should trim down the list of supported formats. That's a
candidate for another patch though.

Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

>  		input_is_yuv = true;
>  		break;
>  	case MEDIA_BUS_FMT_UYVY8_2X8:

-- 
Regards,

Laurent Pinchart

Re: [PATCH 01/11] media: rcar-vin: Fix YUYV8_1X16 handling for CSI-2

[Niklas Söderlund]

Hello,

On 2024-05-05 23:50:43 +0300, Laurent Pinchart wrote:
> Hi Jacopo,
> 
> Thank you for the patch.
> 
> On Fri, May 03, 2024 at 05:51:16PM +0200, Jacopo Mondi wrote:
> > The YUYV8_1X16 and UYVY8_1X16 formats are treated as 'ITU-R
> > BT.601/BT.1358 16-bit YCbCr-422 input' (YUV16 - 0x5) in the R-Car VIN
> > driver and are thus disallowed when capturing frames from the R-Car
> > CSI-2 interface according to the hardware manual.
> > 
> > As the 1X16 format variants are meant to be used with serial busses they
> > have to be treated as 'YCbCr-422 8-bit data input' (0x1) when capturing
> > from CSI-2, which is a valid setting for CSI-2.
> > 
> > Commit 78b3f9d75a62 ("media: rcar-vin: Add check that input interface
> > and format are valid") disallowed capturing YUV16 when using the CSI-2
> > interface. Fix this by using YUV8_BT601 for YCbCr422 when CSI-2 is in
> > use.
> > 
> > Fixes: 78b3f9d75a62 ("media: rcar-vin: Add check that input interface and format are valid")
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >  .../media/platform/renesas/rcar-vin/rcar-dma.c   | 16 +++++++++++++---
> >  1 file changed, 13 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/media/platform/renesas/rcar-vin/rcar-dma.c b/drivers/media/platform/renesas/rcar-vin/rcar-dma.c
> > index e2c40abc6d3d..21d5b2815e86 100644
> > --- a/drivers/media/platform/renesas/rcar-vin/rcar-dma.c
> > +++ b/drivers/media/platform/renesas/rcar-vin/rcar-dma.c
> > @@ -742,12 +742,22 @@ static int rvin_setup(struct rvin_dev *vin)
> >  	 */
> >  	switch (vin->mbus_code) {
> >  	case MEDIA_BUS_FMT_YUYV8_1X16:
> > -		/* BT.601/BT.1358 16bit YCbCr422 */
> > -		vnmc |= VNMC_INF_YUV16;
> > +		if (vin->is_csi)
> > +			/* YCbCr422 8-bit */
> > +			vnmc |= VNMC_INF_YUV8_BT601;
> > +		else
> > +			/* BT.601/BT.1358 16bit YCbCr422 */
> > +			vnmc |= VNMC_INF_YUV16;
> >  		input_is_yuv = true;
> >  		break;
> >  	case MEDIA_BUS_FMT_UYVY8_1X16:
> > -		vnmc |= VNMC_INF_YUV16 | VNMC_YCAL;
> > +		if (vin->is_csi)
> > +			/* YCbCr422 8-bit */
> > +			vnmc |= VNMC_INF_YUV8_BT601;
> > +		else
> > +			/* BT.601/BT.1358 16bit YCbCr422 */
> > +			vnmc |= VNMC_INF_YUV16;
> > +		vnmc |= VNMC_YCAL;
> 
> You could also write
> 
> 	case MEDIA_BUS_FMT_UYVY8_1X16:
> 		vnmc |= VNMC_YCAL;
> 		fallthrough;
> 	case MEDIA_BUS_FMT_YUYV8_1X16:
> 		if (vin->is_csi)
> 			/* YCbCr422 8-bit */
> 			vnmc |= VNMC_INF_YUV8_BT601;
> 		else
> 			/* BT.601/BT.1358 16bit YCbCr422 */
> 			vnmc |= VNMC_INF_YUV16;
> 		input_is_yuv = true;
> 		break;
> 
> Up to you.

I prefers Jacopo's version. This function should likely be reworked in 
the future to be separate Gen2/Gen3/Gen4 versions so we can remove all 
these ugly checks as not all formats are supported on all hardware 
generations.

I think that work would be easier if we keep the current ugly and dumb 
structure of these switches.

> 
> On a side note, CSI-2 isn't supposed to support
> MEDIA_BUS_FMT_YUYV8_1X16. The native format is MEDIA_BUS_FMT_UYVY8_1X16.
> I wonder if we should trim down the list of supported formats. That's a
> candidate for another patch though.
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> 
> >  		input_is_yuv = true;
> >  		break;
> >  	case MEDIA_BUS_FMT_UYVY8_2X8:
> 
> -- 
> Regards,
> 
> Laurent Pinchart

-- 
Kind Regards,
Niklas Söderlund

[PATCH 02/11] media: rcar-csi2: Disable runtime_pm in probe error

[Jacopo Mondi]

Disable pm_runtime in the probe() function error path.

Fixes: 769afd212b16 ("media: rcar-csi2: add Renesas R-Car MIPI CSI-2 receiver driver")
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-csi2.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index 582d5e35db0e..249e58c77176 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -1914,12 +1914,14 @@ static int rcsi2_probe(struct platform_device *pdev)
 
 	ret = v4l2_async_register_subdev(&priv->subdev);
 	if (ret < 0)
-		goto error_async;
+		goto error_pm_runtime;
 
 	dev_info(priv->dev, "%d lanes found\n", priv->lanes);
 
 	return 0;
 
+error_pm_runtime:
+	pm_runtime_disable(&pdev->dev);
 error_async:
 	v4l2_async_nf_unregister(&priv->notifier);
 	v4l2_async_nf_cleanup(&priv->notifier);
-- 
2.44.0

Re: [PATCH 02/11] media: rcar-csi2: Disable runtime_pm in probe error

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:17PM +0200, Jacopo Mondi wrote:
> Disable pm_runtime in the probe() function error path.
> 
> Fixes: 769afd212b16 ("media: rcar-csi2: add Renesas R-Car MIPI CSI-2 receiver driver")
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

> ---
>  drivers/media/platform/renesas/rcar-csi2.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
> index 582d5e35db0e..249e58c77176 100644
> --- a/drivers/media/platform/renesas/rcar-csi2.c
> +++ b/drivers/media/platform/renesas/rcar-csi2.c
> @@ -1914,12 +1914,14 @@ static int rcsi2_probe(struct platform_device *pdev)
>  
>  	ret = v4l2_async_register_subdev(&priv->subdev);
>  	if (ret < 0)
> -		goto error_async;
> +		goto error_pm_runtime;
>  
>  	dev_info(priv->dev, "%d lanes found\n", priv->lanes);
>  
>  	return 0;
>  
> +error_pm_runtime:
> +	pm_runtime_disable(&pdev->dev);
>  error_async:
>  	v4l2_async_nf_unregister(&priv->notifier);
>  	v4l2_async_nf_cleanup(&priv->notifier);

-- 
Regards,

Laurent Pinchart

[PATCH 03/11] media: rcar-csi2: Cleanup subdevice in remove()

[Jacopo Mondi]

Cleanup the V4L2 subdevice in the driver's remove function to
ensure its async connection are freed, and guarantee in future that
the subdev active state is cleaned up.

Fixes: 769afd212b16 ("media: rcar-csi2: add Renesas R-Car MIPI CSI-2 receiver driver")
Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-csi2.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index 249e58c77176..2d464e43a5be 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -1938,6 +1938,7 @@ static void rcsi2_remove(struct platform_device *pdev)
 	v4l2_async_nf_unregister(&priv->notifier);
 	v4l2_async_nf_cleanup(&priv->notifier);
 	v4l2_async_unregister_subdev(&priv->subdev);
+	v4l2_subdev_cleanup(&priv->subdev);
 
 	pm_runtime_disable(&pdev->dev);
 
-- 
2.44.0

Re: [PATCH 03/11] media: rcar-csi2: Cleanup subdevice in remove()

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:18PM +0200, Jacopo Mondi wrote:
> Cleanup the V4L2 subdevice in the driver's remove function to
> ensure its async connection are freed, and guarantee in future that
> the subdev active state is cleaned up.
> 
> Fixes: 769afd212b16 ("media: rcar-csi2: add Renesas R-Car MIPI CSI-2 receiver driver")
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

> ---
>  drivers/media/platform/renesas/rcar-csi2.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
> index 249e58c77176..2d464e43a5be 100644
> --- a/drivers/media/platform/renesas/rcar-csi2.c
> +++ b/drivers/media/platform/renesas/rcar-csi2.c
> @@ -1938,6 +1938,7 @@ static void rcsi2_remove(struct platform_device *pdev)
>  	v4l2_async_nf_unregister(&priv->notifier);
>  	v4l2_async_nf_cleanup(&priv->notifier);
>  	v4l2_async_unregister_subdev(&priv->subdev);
> +	v4l2_subdev_cleanup(&priv->subdev);
>  
>  	pm_runtime_disable(&pdev->dev);
>  

-- 
Regards,

Laurent Pinchart

[PATCH 04/11] media: rcar-csi2: Use the subdev active state

[Jacopo Mondi]

Create the subdevice state with v4l2_subdev_init_finalize() and
implement the init_state() operation to guarantee the state is initialized.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-csi2.c | 138 +++++++++++----------
 1 file changed, 75 insertions(+), 63 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index 2d464e43a5be..4d4dfb59b025 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -587,7 +587,8 @@ enum rcar_csi2_pads {
 struct rcar_csi2_info {
 	int (*init_phtw)(struct rcar_csi2 *priv, unsigned int mbps);
 	int (*phy_post_init)(struct rcar_csi2 *priv);
-	int (*start_receiver)(struct rcar_csi2 *priv);
+	int (*start_receiver)(struct rcar_csi2 *priv,
+			      struct v4l2_subdev_state *state);
 	void (*enter_standby)(struct rcar_csi2 *priv);
 	const struct rcsi2_mbps_reg *hsfreqrange;
 	unsigned int csi0clkfreqrange;
@@ -613,8 +614,6 @@ struct rcar_csi2 {
 
 	int channel_vc[4];
 
-	struct mutex lock; /* Protects mf and stream_count. */
-	struct v4l2_mbus_framefmt mf;
 	int stream_count;
 
 	bool cphy;
@@ -808,20 +807,25 @@ static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
 	return 0;
 }
 
-static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
+static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
+				     struct v4l2_subdev_state *state)
 {
 	const struct rcar_csi2_format *format;
 	u32 phycnt, vcdt = 0, vcdt2 = 0, fld = 0;
+	struct v4l2_mbus_framefmt *fmt;
 	unsigned int lanes;
 	unsigned int i;
 	int mbps, ret;
 
+	/* Use the format on the sink pad to compute the receiver config. */
+	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
+
 	dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
-		priv->mf.width, priv->mf.height,
-		priv->mf.field == V4L2_FIELD_NONE ? 'p' : 'i');
+		fmt->width, fmt->height,
+		fmt->field == V4L2_FIELD_NONE ? 'p' : 'i');
 
 	/* Code is validated in set_fmt. */
-	format = rcsi2_code_to_fmt(priv->mf.code);
+	format = rcsi2_code_to_fmt(fmt->code);
 	if (!format)
 		return -EINVAL;
 
@@ -849,11 +853,11 @@ static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
 			vcdt2 |= vcdt_part << ((i % 2) * 16);
 	}
 
-	if (priv->mf.field == V4L2_FIELD_ALTERNATE) {
+	if (fmt->field == V4L2_FIELD_ALTERNATE) {
 		fld = FLD_DET_SEL(1) | FLD_FLD_EN4 | FLD_FLD_EN3 | FLD_FLD_EN2
 			| FLD_FLD_EN;
 
-		if (priv->mf.height == 240)
+		if (fmt->height == 240)
 			fld |= FLD_FLD_NUM(0);
 		else
 			fld |= FLD_FLD_NUM(1);
@@ -1049,15 +1053,18 @@ static int rcsi2_c_phy_setting_v4h(struct rcar_csi2 *priv, int msps)
 	return 0;
 }
 
-static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
+static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv,
+				    struct v4l2_subdev_state *state)
 {
 	const struct rcar_csi2_format *format;
+	struct v4l2_mbus_framefmt *fmt;
 	unsigned int lanes;
 	int msps;
 	int ret;
 
-	/* Calculate parameters */
-	format = rcsi2_code_to_fmt(priv->mf.code);
+	/* Use the format on the sink pad to compute the receiver config. */
+	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
+	format = rcsi2_code_to_fmt(fmt->code);
 	if (!format)
 		return -EINVAL;
 
@@ -1114,7 +1121,7 @@ static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
 	return 0;
 }
 
-static int rcsi2_start(struct rcar_csi2 *priv)
+static int rcsi2_start(struct rcar_csi2 *priv, struct v4l2_subdev_state *state)
 {
 	int ret;
 
@@ -1122,7 +1129,7 @@ static int rcsi2_start(struct rcar_csi2 *priv)
 	if (ret < 0)
 		return ret;
 
-	ret = priv->info->start_receiver(priv);
+	ret = priv->info->start_receiver(priv, state);
 	if (ret) {
 		rcsi2_enter_standby(priv);
 		return ret;
@@ -1146,17 +1153,16 @@ static void rcsi2_stop(struct rcar_csi2 *priv)
 static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
 {
 	struct rcar_csi2 *priv = sd_to_csi2(sd);
+	struct v4l2_subdev_state *state;
 	int ret = 0;
 
-	mutex_lock(&priv->lock);
+	if (!priv->remote)
+		return -ENODEV;
 
-	if (!priv->remote) {
-		ret = -ENODEV;
-		goto out;
-	}
+	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
 
 	if (enable && priv->stream_count == 0) {
-		ret = rcsi2_start(priv);
+		ret = rcsi2_start(priv, state);
 		if (ret)
 			goto out;
 	} else if (!enable && priv->stream_count == 1) {
@@ -1165,49 +1171,26 @@ static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
 
 	priv->stream_count += enable ? 1 : -1;
 out:
-	mutex_unlock(&priv->lock);
+	v4l2_subdev_unlock_state(state);
 
 	return ret;
 }
 
 static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
-				struct v4l2_subdev_state *sd_state,
+				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_format *format)
 {
-	struct rcar_csi2 *priv = sd_to_csi2(sd);
-	struct v4l2_mbus_framefmt *framefmt;
-
-	mutex_lock(&priv->lock);
+	if (format->pad > RCAR_CSI2_SINK)
+		return v4l2_subdev_get_fmt(sd, state, format);
 
 	if (!rcsi2_code_to_fmt(format->format.code))
 		format->format.code = rcar_csi2_formats[0].code;
 
-	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
-		priv->mf = format->format;
-	} else {
-		framefmt = v4l2_subdev_state_get_format(sd_state, 0);
-		*framefmt = format->format;
-	}
-
-	mutex_unlock(&priv->lock);
-
-	return 0;
-}
-
-static int rcsi2_get_pad_format(struct v4l2_subdev *sd,
-				struct v4l2_subdev_state *sd_state,
-				struct v4l2_subdev_format *format)
-{
-	struct rcar_csi2 *priv = sd_to_csi2(sd);
-
-	mutex_lock(&priv->lock);
-
-	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE)
-		format->format = priv->mf;
-	else
-		format->format = *v4l2_subdev_state_get_format(sd_state, 0);
+	*v4l2_subdev_state_get_format(state, format->pad) = format->format;
 
-	mutex_unlock(&priv->lock);
+	/* Propagate the format to the source pads. */
+	for (unsigned int i = RCAR_CSI2_SOURCE_VC0; i < NR_OF_RCAR_CSI2_PAD; i++)
+		*v4l2_subdev_state_get_format(state, i) = format->format;
 
 	return 0;
 }
@@ -1218,7 +1201,7 @@ static const struct v4l2_subdev_video_ops rcar_csi2_video_ops = {
 
 static const struct v4l2_subdev_pad_ops rcar_csi2_pad_ops = {
 	.set_fmt = rcsi2_set_pad_format,
-	.get_fmt = rcsi2_get_pad_format,
+	.get_fmt = v4l2_subdev_get_fmt,
 };
 
 static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
@@ -1226,6 +1209,30 @@ static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
 	.pad	= &rcar_csi2_pad_ops,
 };
 
+static int rcsi2_init_state(struct v4l2_subdev *sd,
+			    struct v4l2_subdev_state *state)
+{
+	static const struct v4l2_mbus_framefmt rcar_csi2_default_fmt = {
+		.width		= 1920,
+		.height		= 1080,
+		.code		= MEDIA_BUS_FMT_RGB888_1X24,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.field		= V4L2_FIELD_NONE,
+		.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
+		.quantization	= V4L2_QUANTIZATION_DEFAULT,
+		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
+	};
+
+	for (unsigned int i = RCAR_CSI2_SINK; i < NR_OF_RCAR_CSI2_PAD; i++)
+		*v4l2_subdev_state_get_format(state, i) = rcar_csi2_default_fmt;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_internal_ops rcar_csi2_internal_ops = {
+	.init_state = rcsi2_init_state,
+};
+
 static irqreturn_t rcsi2_irq(int irq, void *data)
 {
 	struct rcar_csi2 *priv = data;
@@ -1251,14 +1258,17 @@ static irqreturn_t rcsi2_irq(int irq, void *data)
 
 static irqreturn_t rcsi2_irq_thread(int irq, void *data)
 {
+	struct v4l2_subdev_state *state;
 	struct rcar_csi2 *priv = data;
 
-	mutex_lock(&priv->lock);
+	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
+
 	rcsi2_stop(priv);
 	usleep_range(1000, 2000);
-	if (rcsi2_start(priv))
+	if (rcsi2_start(priv, state))
 		dev_warn(priv->dev, "Failed to restart CSI-2 receiver\n");
-	mutex_unlock(&priv->lock);
+
+	v4l2_subdev_unlock_state(state);
 
 	return IRQ_HANDLED;
 }
@@ -1870,23 +1880,23 @@ static int rcsi2_probe(struct platform_device *pdev)
 
 	priv->dev = &pdev->dev;
 
-	mutex_init(&priv->lock);
 	priv->stream_count = 0;
 
 	ret = rcsi2_probe_resources(priv, pdev);
 	if (ret) {
 		dev_err(priv->dev, "Failed to get resources\n");
-		goto error_mutex;
+		return ret;
 	}
 
 	platform_set_drvdata(pdev, priv);
 
 	ret = rcsi2_parse_dt(priv);
 	if (ret)
-		goto error_mutex;
+		return ret;
 
 	priv->subdev.owner = THIS_MODULE;
 	priv->subdev.dev = &pdev->dev;
+	priv->subdev.internal_ops = &rcar_csi2_internal_ops;
 	v4l2_subdev_init(&priv->subdev, &rcar_csi2_subdev_ops);
 	v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
 	snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s",
@@ -1912,21 +1922,25 @@ static int rcsi2_probe(struct platform_device *pdev)
 
 	pm_runtime_enable(&pdev->dev);
 
+	ret = v4l2_subdev_init_finalize(&priv->subdev);
+	if (ret)
+		goto error_pm_runtime;
+
 	ret = v4l2_async_register_subdev(&priv->subdev);
 	if (ret < 0)
-		goto error_pm_runtime;
+		goto error_subdev;
 
 	dev_info(priv->dev, "%d lanes found\n", priv->lanes);
 
 	return 0;
 
+error_subdev:
+	v4l2_subdev_cleanup(&priv->subdev);
 error_pm_runtime:
 	pm_runtime_disable(&pdev->dev);
 error_async:
 	v4l2_async_nf_unregister(&priv->notifier);
 	v4l2_async_nf_cleanup(&priv->notifier);
-error_mutex:
-	mutex_destroy(&priv->lock);
 
 	return ret;
 }
@@ -1941,8 +1955,6 @@ static void rcsi2_remove(struct platform_device *pdev)
 	v4l2_subdev_cleanup(&priv->subdev);
 
 	pm_runtime_disable(&pdev->dev);
-
-	mutex_destroy(&priv->lock);
 }
 
 static struct platform_driver rcar_csi2_pdrv = {
-- 
2.44.0

Re: [PATCH 04/11] media: rcar-csi2: Use the subdev active state

[Niklas Söderlund]

Hi Jacopo,

Thanks for your work.

On 2024-05-03 17:51:19 +0200, Jacopo Mondi wrote:
> Create the subdevice state with v4l2_subdev_init_finalize() and
> implement the init_state() operation to guarantee the state is initialized.

This fails and produces -EPIPE error when trying to capture on R-Car M3N 
using ADV748x. If I only apply patch 1, 2, 3 of this series it works as 
expected, when I apply this as well capture breaks.

You can test this on a remote Gen3 board without any HDMI or CVBS 
source, just enable a test patterns on the HDMI and AFE sources, see [1] 
for my test-case output which demonstrates how I produce the failure.

Oddly enough if I use the old vin-test package I can capture from HDMI 
while CVBS capture still fails with -EPIPE. Not sure what's going on.

1. Ignore the "Unable to get format: Inappropriate ioctl for device 
   (25)." from the "yavta --set-control ..." commands, they are 
   unimportant and not new. Focus in the "Unable to start streaming: 
   Broken pipe (32)." errors from "yavta -c10 --file=..  /dev/videoX" 
   calls, they are not new ;-)

===> star <===
Last login: Fri May  3 17:48:06 2024

Renesas Salvator-X 2nd version board based on r8a77965
0;root@arm64:~
arm64 ~ # echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?
echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?

wkbfirytku 403
ygxezpiinz 0
echo agyxksooma $$; mkdir -p /tmp/vin-capture; echo kvwostutox $?
0;root@arm64:~
arm64 ~ # echo agyxksooma $$; mkdir -p /tmp/vin-capture; echo kvwostutox $?

agyxksooma 403
kvwostutox 0
echo vtmzcdhjtc $$; grep -l 'adv748x 4-0070 afe' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo mwqkcmnuju $?
0;root@arm64:~
arm64 ~ # echo vtmzcdhjtc $$; grep -l 'adv748x 4-0070 afe' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo mwqkcmnuju $?

vtmzcdhjtc 403
/dev/v4l-subdev1
mwqkcmnuju 0
echo yavkazxvyn $$; grep -l 'VIN0 output' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo wpijcgewrd $?
0;root@arm64:~
arm64 ~ # echo yavkazxvyn $$; grep -l 'VIN0 output' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo wpijcgewrd $?

yavkazxvyn 403
/dev/video0
wpijcgewrd 0
echo nkitspntrm $$; for mdev in /dev/media*; do if [[ '$(media-ctl -d $mdev -p | grep 'rcar_vin')' != '' ]]; then echo $mdev; break; fi; done; echo ngoqmbhfqr $?
0;root@arm64:~
arm64 ~ # echo nkitspntrm $$; for mdev in /dev/media*; do if [[ '$(media-ctl -d $mdev -p | grep 'rcar_vin')' != '' ]]; then echo $mdev; break; fi; done; echo ngoqmbhfqr $?

nkitspntrm 403
/dev/media0
ngoqmbhfqr 0
echo ggokwokuyh $$; media-ctl -d /dev/media0 -r; echo yydmasvhba $?
0;root@arm64:~
arm64 ~ # echo ggokwokuyh $$; media-ctl -d /dev/media0 -r; echo yydmasvhba $?

ggokwokuyh 403
yydmasvhba 0
0;root@arm64:~
echo ftqgotsvwt $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 afe':8 -> 'adv748x 4-0070 txb':0 [1]"; echo ohfyvhjfrt $?
arm64 ~ # echo ftqgotsvwt $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 afe':8 -> 'adv748x 4-0070 txb':0 [1]"; echo ohfyvhjfrt $?

ftqgotsvwt 403
ohfyvhjfrt 0
0;root@arm64:~
echo nhtnyteajn $$; media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"; echo vovincdzqi $?
arm64 ~ # echo nhtnyteajn $$; media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"; echo vovincdzqi $?

nhtnyteajn 403
vovincdzqi 0
0;root@arm64:~
echo rulkcoorky $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"; echo esoiygqdfq $?
arm64 ~ # echo rulkcoorky $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"; echo esoiygqdfq $?

rulkcoorky 403
		[stream:0 fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]
esoiygqdfq 0
0;root@arm64:~
echo dmdgcctvqh $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo ighfsicfux $?
arm64 ~ # echo dmdgcctvqh $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo ighfsicfux $?

dmdgcctvqh 403
ighfsicfux 0
0;root@arm64:~
echo mqlvxrvync $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txb':0 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo elpiosmlzj $?
arm64 ~ # echo mqlvxrvync $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txb':0 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo elpiosmlzj $?

mqlvxrvync 403
elpiosmlzj 0
0;root@arm64:~
echo aoldaiatak $$; media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo iymsospuex $?
arm64 ~ # echo aoldaiatak $$; media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo iymsospuex $?

aoldaiatak 403
iymsospuex 0
0;root@arm64:~
echo nxpbvfvtbv $$; yavta -f RGB565 -s 720x480 --field interlaced /dev/video0; echo dppdatfote $?
arm64 ~ # echo nxpbvfvtbv $$; yavta -f RGB565 -s 720x480 --field interlaced /dev/video0; echo dppdatfote $?

nxpbvfvtbv 403
Device /dev/video0 opened.
Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
Video format set: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
dppdatfote 0
echo zsqtrrojrr $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev1; echo xkrxhzzdcq $?
0;root@arm64:~
arm64 ~ # echo zsqtrrojrr $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev1; echo xkrxhzzdcq $?

zsqtrrojrr 403
Device /dev/v4l-subdev1 opened.
Control 0x009f0903 set to 1, is 1
Unable to get format: Inappropriate ioctl for device (25).
xkrxhzzdcq 0
echo jifvldisax $$; yavta -c10 --file=/tmp/vin-capture/cvbs-solid-#.bin /dev/video0; echo tmzmlnjlsq $?
0;root@arm64:~
arm64 ~ # echo jifvldisax $$; yavta -c10 --file=/tmp/vin-capture/cvbs-solid-#.bin /dev/video0; echo tmzmlnjlsq $?

jifvldisax 403
Device /dev/video0 opened.
Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
8 buffers requested.
length: 691200 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffffb1037000.
length: 691200 offset: 32768 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xffffb0f8e000.
length: 691200 offset: 65536 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffffb0ee5000.
length: 691200 offset: 98304 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffffb0e3c000.
length: 691200 offset: 131072 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffffb0d93000.
length: 691200 offset: 163840 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffffb0cea000.
length: 691200 offset: 196608 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffffb0c41000.
length: 691200 offset: 229376 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffffb0b98000.
Unable to start streaming: Broken pipe (32).
8 buffers released.
tmzmlnjlsq 0
echo wojwgdpxnx $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev1; echo dghaavszch $?
0;root@arm64:~
arm64 ~ # echo wojwgdpxnx $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev1; echo dghaavszch $?

wojwgdpxnx 403
Device /dev/v4l-subdev1 opened.
Control 0x009f0903 set to 2, is 2
Unable to get format: Inappropriate ioctl for device (25).
dghaavszch 0
0;root@arm64:~
echo tldblgblou $$; yavta -c10 --file=/tmp/vin-capture/cvbs-colorbars-#.bin /dev/video0; echo pofbqvttib $?
arm64 ~ # echo tldblgblou $$; yavta -c10 --file=/tmp/vin-capture/cvbs-colorbars-#.bin /dev/video0; echo pofbqvttib $?

tldblgblou 403
Device /dev/video0 opened.
Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
8 buffers requested.
length: 691200 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffffb75f7000.
length: 691200 offset: 32768 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xffffb754e000.
length: 691200 offset: 65536 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffffb74a5000.
length: 691200 offset: 98304 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffffb73fc000.
length: 691200 offset: 131072 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffffb7353000.
length: 691200 offset: 163840 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffffb72aa000.
length: 691200 offset: 196608 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffffb7201000.
length: 691200 offset: 229376 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffffb7158000.
Unable to start streaming: Broken pipe (32).
8 buffers released.
pofbqvttib 0
0;root@arm64:~
echo maubyyzesr $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev1; echo dzbvegabvy $?
arm64 ~ # echo maubyyzesr $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev1; echo dzbvegabvy $?

maubyyzesr 403
Device /dev/v4l-subdev1 opened.
Control 0x009f0903 set to 0, is 0
Unable to get format: Inappropriate ioctl for device (25).
dzbvegabvy 0
0;root@arm64:~
echo pffcsadwlt $$; test -e /tmp/vin-capture/cvbs-solid-000007.bin; echo xilwxnvrrd $?
arm64 ~ # echo pffcsadwlt $$; test -e /tmp/vin-capture/cvbs-solid-000007.bin; echo xilwxnvrrd $?

pffcsadwlt 403
xilwxnvrrd 1
echo grglvpwocj $$; grep -l 'adv748x 4-0070 hdmi' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo hvcawnkqdt $?
0;root@arm64:~
arm64 ~ # echo grglvpwocj $$; grep -l 'adv748x 4-0070 hdmi' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo hvcawnkqdt $?

grglvpwocj 403
/dev/v4l-subdev4
hvcawnkqdt 0
echo nnpccbuzak $$; media-ctl -d /dev/media0 -r; echo zigvuehatt $?
0;root@arm64:~
arm64 ~ # echo nnpccbuzak $$; media-ctl -d /dev/media0 -r; echo zigvuehatt $?

nnpccbuzak 403
zigvuehatt 0
echo xyngygxuyr $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 hdmi':1 -> 'adv748x 4-0070 txa':0 [1]"; echo oknofsbyvn $?
0;root@arm64:~
arm64 ~ # echo xyngygxuyr $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 hdmi':1 -> 'adv748x 4-0070 txa':0 [1]"; echo oknofsbyvn $?

xyngygxuyr 403
oknofsbyvn 0
0;root@arm64:~
echo jkzuanypyp $$; media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"; echo onsaxggllz $?
arm64 ~ # echo jkzuanypyp $$; media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"; echo onsaxggllz $?

jkzuanypyp 403
onsaxggllz 0
0;root@arm64:~
echo lttzkqwwoa $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"; echo imxtuqbxgf $?
arm64 ~ # echo lttzkqwwoa $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"; echo imxtuqbxgf $?

lttzkqwwoa 403
		[stream:0 fmt:RGB888_1X24/1280x720 field:none colorspace:srgb]
imxtuqbxgf 0
0;root@arm64:~
echo degzwngmzk $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo vgqpdhbhjt $?
arm64 ~ # echo degzwngmzk $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo vgqpdhbhjt $?

degzwngmzk 403
vgqpdhbhjt 0
0;root@arm64:~
echo iwtingcfbz $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txa':0 [fmt:RGB888_1X24/1280x720 field:none]"; echo qircbjupbq $?
arm64 ~ # echo iwtingcfbz $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txa':0 [fmt:RGB888_1X24/1280x720 field:none]"; echo qircbjupbq $?

iwtingcfbz 403
qircbjupbq 0
0;root@arm64:~
echo twrjqisqff $$; media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo ocaivovrxk $?
arm64 ~ # echo twrjqisqff $$; media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo ocaivovrxk $?

twrjqisqff 403
ocaivovrxk 0
echo ajqwydmzik $$; yavta -f RGB565 -s 1280x720 --field none /dev/video0; echo fayawxvllj $?
0;root@arm64:~
arm64 ~ # echo ajqwydmzik $$; yavta -f RGB565 -s 1280x720 --field none /dev/video0; echo fayawxvllj $?

ajqwydmzik 403
Device /dev/video0 opened.
Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
Video format set: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
fayawxvllj 0
echo pealtxybun $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev4; echo juhrczaeao $?
0;root@arm64:~
arm64 ~ # echo pealtxybun $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev4; echo juhrczaeao $?

pealtxybun 403
Device /dev/v4l-subdev4 opened.
Control 0x009f0903 set to 1, is 1
Unable to get format: Inappropriate ioctl for device (25).
juhrczaeao 0
0;root@arm64:~
echo oaxoraqdid $$; yavta -c10 --file=/tmp/vin-capture/hdmi-solid-#.bin /dev/video0; echo bfsxvmqitj $?
arm64 ~ # echo oaxoraqdid $$; yavta -c10 --file=/tmp/vin-capture/hdmi-solid-#.bin /dev/video0; echo bfsxvmqitj $?

oaxoraqdid 403
Device /dev/video0 opened.
Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
8 buffers requested.
length: 1843200 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffffafa8e000.
length: 1843200 offset: 32768 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xffffaf8cc000.
length: 1843200 offset: 65536 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffffaf70a000.
length: 1843200 offset: 98304 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffffaf548000.
length: 1843200 offset: 131072 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffffaf386000.
length: 1843200 offset: 163840 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffffaf1c4000.
length: 1843200 offset: 196608 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffffaf002000.
length: 1843200 offset: 229376 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffffaee40000.
Unable to start streaming: Broken pipe (32).
8 buffers released.
bfsxvmqitj 0
echo pmbwbnrkkw $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev4; echo ucgivtjwvw $?
0;root@arm64:~
arm64 ~ # echo pmbwbnrkkw $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev4; echo ucgivtjwvw $?

pmbwbnrkkw 403
Device /dev/v4l-subdev4 opened.
Control 0x009f0903 set to 2, is 2
Unable to get format: Inappropriate ioctl for device (25).
ucgivtjwvw 0
0;root@arm64:~
echo rvkbtaiuan $$; yavta -c10 --file=/tmp/vin-capture/hdmi-colorbars-#.bin /dev/video0; echo fdlcrvrylc $?
arm64 ~ # echo rvkbtaiuan $$; yavta -c10 --file=/tmp/vin-capture/hdmi-colorbars-#.bin /dev/video0; echo fdlcrvrylc $?

rvkbtaiuan 403
Device /dev/video0 opened.
Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
8 buffers requested.
length: 1843200 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xffff86f8e000.
length: 1843200 offset: 32768 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xffff86dcc000.
length: 1843200 offset: 65536 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xffff86c0a000.
length: 1843200 offset: 98304 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xffff86a48000.
length: 1843200 offset: 131072 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xffff86886000.
length: 1843200 offset: 163840 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xffff866c4000.
length: 1843200 offset: 196608 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xffff86502000.
length: 1843200 offset: 229376 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xffff86340000.
Unable to start streaming: Broken pipe (32).
8 buffers released.
fdlcrvrylc 0
0;root@arm64:~
echo hrgnrsupul $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev4; echo bvgysmcxve $?
arm64 ~ # echo hrgnrsupul $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev4; echo bvgysmcxve $?

hrgnrsupul 403
Device /dev/v4l-subdev4 opened.
Control 0x009f0903 set to 0, is 0
Unable to get format: Inappropriate ioctl for device (25).
bvgysmcxve 0
echo znlqovrrra $$; test -e /tmp/vin-capture/hdmi-solid-000007.bin; echo sjpcuiptaf $?
0;root@arm64:~
arm64 ~ # echo znlqovrrra $$; test -e /tmp/vin-capture/hdmi-solid-000007.bin; echo sjpcuiptaf $?

znlqovrrra 403
sjpcuiptaf 1
exit
===> end <===

> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  drivers/media/platform/renesas/rcar-csi2.c | 138 +++++++++++----------
>  1 file changed, 75 insertions(+), 63 deletions(-)
> 
> diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
> index 2d464e43a5be..4d4dfb59b025 100644
> --- a/drivers/media/platform/renesas/rcar-csi2.c
> +++ b/drivers/media/platform/renesas/rcar-csi2.c
> @@ -587,7 +587,8 @@ enum rcar_csi2_pads {
>  struct rcar_csi2_info {
>  	int (*init_phtw)(struct rcar_csi2 *priv, unsigned int mbps);
>  	int (*phy_post_init)(struct rcar_csi2 *priv);
> -	int (*start_receiver)(struct rcar_csi2 *priv);
> +	int (*start_receiver)(struct rcar_csi2 *priv,
> +			      struct v4l2_subdev_state *state);
>  	void (*enter_standby)(struct rcar_csi2 *priv);
>  	const struct rcsi2_mbps_reg *hsfreqrange;
>  	unsigned int csi0clkfreqrange;
> @@ -613,8 +614,6 @@ struct rcar_csi2 {
>  
>  	int channel_vc[4];
>  
> -	struct mutex lock; /* Protects mf and stream_count. */
> -	struct v4l2_mbus_framefmt mf;
>  	int stream_count;
>  
>  	bool cphy;
> @@ -808,20 +807,25 @@ static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
>  	return 0;
>  }
>  
> -static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
> +static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
> +				     struct v4l2_subdev_state *state)
>  {
>  	const struct rcar_csi2_format *format;
>  	u32 phycnt, vcdt = 0, vcdt2 = 0, fld = 0;
> +	struct v4l2_mbus_framefmt *fmt;
>  	unsigned int lanes;
>  	unsigned int i;
>  	int mbps, ret;
>  
> +	/* Use the format on the sink pad to compute the receiver config. */
> +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> +
>  	dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
> -		priv->mf.width, priv->mf.height,
> -		priv->mf.field == V4L2_FIELD_NONE ? 'p' : 'i');
> +		fmt->width, fmt->height,
> +		fmt->field == V4L2_FIELD_NONE ? 'p' : 'i');
>  
>  	/* Code is validated in set_fmt. */
> -	format = rcsi2_code_to_fmt(priv->mf.code);
> +	format = rcsi2_code_to_fmt(fmt->code);
>  	if (!format)
>  		return -EINVAL;
>  
> @@ -849,11 +853,11 @@ static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
>  			vcdt2 |= vcdt_part << ((i % 2) * 16);
>  	}
>  
> -	if (priv->mf.field == V4L2_FIELD_ALTERNATE) {
> +	if (fmt->field == V4L2_FIELD_ALTERNATE) {
>  		fld = FLD_DET_SEL(1) | FLD_FLD_EN4 | FLD_FLD_EN3 | FLD_FLD_EN2
>  			| FLD_FLD_EN;
>  
> -		if (priv->mf.height == 240)
> +		if (fmt->height == 240)
>  			fld |= FLD_FLD_NUM(0);
>  		else
>  			fld |= FLD_FLD_NUM(1);
> @@ -1049,15 +1053,18 @@ static int rcsi2_c_phy_setting_v4h(struct rcar_csi2 *priv, int msps)
>  	return 0;
>  }
>  
> -static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
> +static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv,
> +				    struct v4l2_subdev_state *state)
>  {
>  	const struct rcar_csi2_format *format;
> +	struct v4l2_mbus_framefmt *fmt;
>  	unsigned int lanes;
>  	int msps;
>  	int ret;
>  
> -	/* Calculate parameters */
> -	format = rcsi2_code_to_fmt(priv->mf.code);
> +	/* Use the format on the sink pad to compute the receiver config. */
> +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> +	format = rcsi2_code_to_fmt(fmt->code);
>  	if (!format)
>  		return -EINVAL;
>  
> @@ -1114,7 +1121,7 @@ static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
>  	return 0;
>  }
>  
> -static int rcsi2_start(struct rcar_csi2 *priv)
> +static int rcsi2_start(struct rcar_csi2 *priv, struct v4l2_subdev_state *state)
>  {
>  	int ret;
>  
> @@ -1122,7 +1129,7 @@ static int rcsi2_start(struct rcar_csi2 *priv)
>  	if (ret < 0)
>  		return ret;
>  
> -	ret = priv->info->start_receiver(priv);
> +	ret = priv->info->start_receiver(priv, state);
>  	if (ret) {
>  		rcsi2_enter_standby(priv);
>  		return ret;
> @@ -1146,17 +1153,16 @@ static void rcsi2_stop(struct rcar_csi2 *priv)
>  static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
>  {
>  	struct rcar_csi2 *priv = sd_to_csi2(sd);
> +	struct v4l2_subdev_state *state;
>  	int ret = 0;
>  
> -	mutex_lock(&priv->lock);
> +	if (!priv->remote)
> +		return -ENODEV;
>  
> -	if (!priv->remote) {
> -		ret = -ENODEV;
> -		goto out;
> -	}
> +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
>  
>  	if (enable && priv->stream_count == 0) {
> -		ret = rcsi2_start(priv);
> +		ret = rcsi2_start(priv, state);
>  		if (ret)
>  			goto out;
>  	} else if (!enable && priv->stream_count == 1) {
> @@ -1165,49 +1171,26 @@ static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
>  
>  	priv->stream_count += enable ? 1 : -1;
>  out:
> -	mutex_unlock(&priv->lock);
> +	v4l2_subdev_unlock_state(state);
>  
>  	return ret;
>  }
>  
>  static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
> -				struct v4l2_subdev_state *sd_state,
> +				struct v4l2_subdev_state *state,
>  				struct v4l2_subdev_format *format)
>  {
> -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> -	struct v4l2_mbus_framefmt *framefmt;
> -
> -	mutex_lock(&priv->lock);
> +	if (format->pad > RCAR_CSI2_SINK)
> +		return v4l2_subdev_get_fmt(sd, state, format);
>  
>  	if (!rcsi2_code_to_fmt(format->format.code))
>  		format->format.code = rcar_csi2_formats[0].code;
>  
> -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> -		priv->mf = format->format;
> -	} else {
> -		framefmt = v4l2_subdev_state_get_format(sd_state, 0);
> -		*framefmt = format->format;
> -	}
> -
> -	mutex_unlock(&priv->lock);
> -
> -	return 0;
> -}
> -
> -static int rcsi2_get_pad_format(struct v4l2_subdev *sd,
> -				struct v4l2_subdev_state *sd_state,
> -				struct v4l2_subdev_format *format)
> -{
> -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> -
> -	mutex_lock(&priv->lock);
> -
> -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE)
> -		format->format = priv->mf;
> -	else
> -		format->format = *v4l2_subdev_state_get_format(sd_state, 0);
> +	*v4l2_subdev_state_get_format(state, format->pad) = format->format;
>  
> -	mutex_unlock(&priv->lock);
> +	/* Propagate the format to the source pads. */
> +	for (unsigned int i = RCAR_CSI2_SOURCE_VC0; i < NR_OF_RCAR_CSI2_PAD; i++)
> +		*v4l2_subdev_state_get_format(state, i) = format->format;
>  
>  	return 0;
>  }
> @@ -1218,7 +1201,7 @@ static const struct v4l2_subdev_video_ops rcar_csi2_video_ops = {
>  
>  static const struct v4l2_subdev_pad_ops rcar_csi2_pad_ops = {
>  	.set_fmt = rcsi2_set_pad_format,
> -	.get_fmt = rcsi2_get_pad_format,
> +	.get_fmt = v4l2_subdev_get_fmt,
>  };
>  
>  static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
> @@ -1226,6 +1209,30 @@ static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
>  	.pad	= &rcar_csi2_pad_ops,
>  };
>  
> +static int rcsi2_init_state(struct v4l2_subdev *sd,
> +			    struct v4l2_subdev_state *state)
> +{
> +	static const struct v4l2_mbus_framefmt rcar_csi2_default_fmt = {
> +		.width		= 1920,
> +		.height		= 1080,
> +		.code		= MEDIA_BUS_FMT_RGB888_1X24,
> +		.colorspace	= V4L2_COLORSPACE_SRGB,
> +		.field		= V4L2_FIELD_NONE,
> +		.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
> +		.quantization	= V4L2_QUANTIZATION_DEFAULT,
> +		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
> +	};
> +
> +	for (unsigned int i = RCAR_CSI2_SINK; i < NR_OF_RCAR_CSI2_PAD; i++)
> +		*v4l2_subdev_state_get_format(state, i) = rcar_csi2_default_fmt;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_internal_ops rcar_csi2_internal_ops = {
> +	.init_state = rcsi2_init_state,
> +};
> +
>  static irqreturn_t rcsi2_irq(int irq, void *data)
>  {
>  	struct rcar_csi2 *priv = data;
> @@ -1251,14 +1258,17 @@ static irqreturn_t rcsi2_irq(int irq, void *data)
>  
>  static irqreturn_t rcsi2_irq_thread(int irq, void *data)
>  {
> +	struct v4l2_subdev_state *state;
>  	struct rcar_csi2 *priv = data;
>  
> -	mutex_lock(&priv->lock);
> +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
> +
>  	rcsi2_stop(priv);
>  	usleep_range(1000, 2000);
> -	if (rcsi2_start(priv))
> +	if (rcsi2_start(priv, state))
>  		dev_warn(priv->dev, "Failed to restart CSI-2 receiver\n");
> -	mutex_unlock(&priv->lock);
> +
> +	v4l2_subdev_unlock_state(state);
>  
>  	return IRQ_HANDLED;
>  }
> @@ -1870,23 +1880,23 @@ static int rcsi2_probe(struct platform_device *pdev)
>  
>  	priv->dev = &pdev->dev;
>  
> -	mutex_init(&priv->lock);
>  	priv->stream_count = 0;
>  
>  	ret = rcsi2_probe_resources(priv, pdev);
>  	if (ret) {
>  		dev_err(priv->dev, "Failed to get resources\n");
> -		goto error_mutex;
> +		return ret;
>  	}
>  
>  	platform_set_drvdata(pdev, priv);
>  
>  	ret = rcsi2_parse_dt(priv);
>  	if (ret)
> -		goto error_mutex;
> +		return ret;
>  
>  	priv->subdev.owner = THIS_MODULE;
>  	priv->subdev.dev = &pdev->dev;
> +	priv->subdev.internal_ops = &rcar_csi2_internal_ops;
>  	v4l2_subdev_init(&priv->subdev, &rcar_csi2_subdev_ops);
>  	v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
>  	snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s",
> @@ -1912,21 +1922,25 @@ static int rcsi2_probe(struct platform_device *pdev)
>  
>  	pm_runtime_enable(&pdev->dev);
>  
> +	ret = v4l2_subdev_init_finalize(&priv->subdev);
> +	if (ret)
> +		goto error_pm_runtime;
> +
>  	ret = v4l2_async_register_subdev(&priv->subdev);
>  	if (ret < 0)
> -		goto error_pm_runtime;
> +		goto error_subdev;
>  
>  	dev_info(priv->dev, "%d lanes found\n", priv->lanes);
>  
>  	return 0;
>  
> +error_subdev:
> +	v4l2_subdev_cleanup(&priv->subdev);
>  error_pm_runtime:
>  	pm_runtime_disable(&pdev->dev);
>  error_async:
>  	v4l2_async_nf_unregister(&priv->notifier);
>  	v4l2_async_nf_cleanup(&priv->notifier);
> -error_mutex:
> -	mutex_destroy(&priv->lock);
>  
>  	return ret;
>  }
> @@ -1941,8 +1955,6 @@ static void rcsi2_remove(struct platform_device *pdev)
>  	v4l2_subdev_cleanup(&priv->subdev);
>  
>  	pm_runtime_disable(&pdev->dev);
> -
> -	mutex_destroy(&priv->lock);
>  }
>  
>  static struct platform_driver rcar_csi2_pdrv = {
> -- 
> 2.44.0
> 

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH 04/11] media: rcar-csi2: Use the subdev active state

[Laurent Pinchart]

On Fri, May 03, 2024 at 08:03:21PM +0200, Niklas Söderlund wrote:
> Hi Jacopo,
> 
> Thanks for your work.
> 
> On 2024-05-03 17:51:19 +0200, Jacopo Mondi wrote:
> > Create the subdevice state with v4l2_subdev_init_finalize() and
> > implement the init_state() operation to guarantee the state is initialized.
> 
> This fails and produces -EPIPE error when trying to capture on R-Car M3N 
> using ADV748x. If I only apply patch 1, 2, 3 of this series it works as 
> expected, when I apply this as well capture breaks.
> 
> You can test this on a remote Gen3 board without any HDMI or CVBS 
> source, just enable a test patterns on the HDMI and AFE sources, see [1] 
> for my test-case output which demonstrates how I produce the failure.
> 
> Oddly enough if I use the old vin-test package I can capture from HDMI 
> while CVBS capture still fails with -EPIPE. Not sure what's going on.
> 
> 1. Ignore the "Unable to get format: Inappropriate ioctl for device 
>    (25)." from the "yavta --set-control ..." commands, they are 
>    unimportant and not new. Focus in the "Unable to start streaming: 
>    Broken pipe (32)." errors from "yavta -c10 --file=..  /dev/videoX" 
>    calls, they are not new ;-)
> 
> ===> star <===
> Last login: Fri May  3 17:48:06 2024
> 
> Renesas Salvator-X 2nd version board based on r8a77965
> 0;root@arm64:~
> arm64 ~ # echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?
> echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?
> 
> wkbfirytku 403
> ygxezpiinz 0
> echo agyxksooma $$; mkdir -p /tmp/vin-capture; echo kvwostutox $?
> 0;root@arm64:~
> arm64 ~ # echo agyxksooma $$; mkdir -p /tmp/vin-capture; echo kvwostutox $?
> 
> agyxksooma 403
> kvwostutox 0
> echo vtmzcdhjtc $$; grep -l 'adv748x 4-0070 afe' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo mwqkcmnuju $?
> 0;root@arm64:~
> arm64 ~ # echo vtmzcdhjtc $$; grep -l 'adv748x 4-0070 afe' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo mwqkcmnuju $?
> 
> vtmzcdhjtc 403
> /dev/v4l-subdev1
> mwqkcmnuju 0
> echo yavkazxvyn $$; grep -l 'VIN0 output' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo wpijcgewrd $?
> 0;root@arm64:~
> arm64 ~ # echo yavkazxvyn $$; grep -l 'VIN0 output' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo wpijcgewrd $?
> 
> yavkazxvyn 403
> /dev/video0
> wpijcgewrd 0
> echo nkitspntrm $$; for mdev in /dev/media*; do if [[ '$(media-ctl -d $mdev -p | grep 'rcar_vin')' != '' ]]; then echo $mdev; break; fi; done; echo ngoqmbhfqr $?
> 0;root@arm64:~
> arm64 ~ # echo nkitspntrm $$; for mdev in /dev/media*; do if [[ '$(media-ctl -d $mdev -p | grep 'rcar_vin')' != '' ]]; then echo $mdev; break; fi; done; echo ngoqmbhfqr $?
> 
> nkitspntrm 403
> /dev/media0
> ngoqmbhfqr 0
> echo ggokwokuyh $$; media-ctl -d /dev/media0 -r; echo yydmasvhba $?
> 0;root@arm64:~
> arm64 ~ # echo ggokwokuyh $$; media-ctl -d /dev/media0 -r; echo yydmasvhba $?
> 
> ggokwokuyh 403
> yydmasvhba 0
> 0;root@arm64:~
> echo ftqgotsvwt $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 afe':8 -> 'adv748x 4-0070 txb':0 [1]"; echo ohfyvhjfrt $?
> arm64 ~ # echo ftqgotsvwt $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 afe':8 -> 'adv748x 4-0070 txb':0 [1]"; echo ohfyvhjfrt $?
> 
> ftqgotsvwt 403
> ohfyvhjfrt 0
> 0;root@arm64:~
> echo nhtnyteajn $$; media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"; echo vovincdzqi $?
> arm64 ~ # echo nhtnyteajn $$; media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"; echo vovincdzqi $?
> 
> nhtnyteajn 403
> vovincdzqi 0
> 0;root@arm64:~
> echo rulkcoorky $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"; echo esoiygqdfq $?
> arm64 ~ # echo rulkcoorky $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"; echo esoiygqdfq $?
> 
> rulkcoorky 403
> 		[stream:0 fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]
> esoiygqdfq 0
> 0;root@arm64:~
> echo dmdgcctvqh $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo ighfsicfux $?
> arm64 ~ # echo dmdgcctvqh $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo ighfsicfux $?
> 
> dmdgcctvqh 403
> ighfsicfux 0
> 0;root@arm64:~
> echo mqlvxrvync $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txb':0 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo elpiosmlzj $?

Should this be pad 1 instead of pad 0 ?

> arm64 ~ # echo mqlvxrvync $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txb':0 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo elpiosmlzj $?
> 
> mqlvxrvync 403
> elpiosmlzj 0
> 0;root@arm64:~
> echo aoldaiatak $$; media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo iymsospuex $?
> arm64 ~ # echo aoldaiatak $$; media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo iymsospuex $?
> 
> aoldaiatak 403
> iymsospuex 0
> 0;root@arm64:~
> echo nxpbvfvtbv $$; yavta -f RGB565 -s 720x480 --field interlaced /dev/video0; echo dppdatfote $?
> arm64 ~ # echo nxpbvfvtbv $$; yavta -f RGB565 -s 720x480 --field interlaced /dev/video0; echo dppdatfote $?
> 
> nxpbvfvtbv 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format set: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> dppdatfote 0
> echo zsqtrrojrr $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev1; echo xkrxhzzdcq $?
> 0;root@arm64:~
> arm64 ~ # echo zsqtrrojrr $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev1; echo xkrxhzzdcq $?
> 
> zsqtrrojrr 403
> Device /dev/v4l-subdev1 opened.
> Control 0x009f0903 set to 1, is 1
> Unable to get format: Inappropriate ioctl for device (25).
> xkrxhzzdcq 0
> echo jifvldisax $$; yavta -c10 --file=/tmp/vin-capture/cvbs-solid-#.bin /dev/video0; echo tmzmlnjlsq $?
> 0;root@arm64:~
> arm64 ~ # echo jifvldisax $$; yavta -c10 --file=/tmp/vin-capture/cvbs-solid-#.bin /dev/video0; echo tmzmlnjlsq $?
> 
> jifvldisax 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> 8 buffers requested.
> length: 691200 offset: 0 timestamp type/source: mono/EoF
> Buffer 0/0 mapped at address 0xffffb1037000.
> length: 691200 offset: 32768 timestamp type/source: mono/EoF
> Buffer 1/0 mapped at address 0xffffb0f8e000.
> length: 691200 offset: 65536 timestamp type/source: mono/EoF
> Buffer 2/0 mapped at address 0xffffb0ee5000.
> length: 691200 offset: 98304 timestamp type/source: mono/EoF
> Buffer 3/0 mapped at address 0xffffb0e3c000.
> length: 691200 offset: 131072 timestamp type/source: mono/EoF
> Buffer 4/0 mapped at address 0xffffb0d93000.
> length: 691200 offset: 163840 timestamp type/source: mono/EoF
> Buffer 5/0 mapped at address 0xffffb0cea000.
> length: 691200 offset: 196608 timestamp type/source: mono/EoF
> Buffer 6/0 mapped at address 0xffffb0c41000.
> length: 691200 offset: 229376 timestamp type/source: mono/EoF
> Buffer 7/0 mapped at address 0xffffb0b98000.
> Unable to start streaming: Broken pipe (32).
> 8 buffers released.
> tmzmlnjlsq 0
> echo wojwgdpxnx $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev1; echo dghaavszch $?
> 0;root@arm64:~
> arm64 ~ # echo wojwgdpxnx $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev1; echo dghaavszch $?
> 
> wojwgdpxnx 403
> Device /dev/v4l-subdev1 opened.
> Control 0x009f0903 set to 2, is 2
> Unable to get format: Inappropriate ioctl for device (25).
> dghaavszch 0
> 0;root@arm64:~
> echo tldblgblou $$; yavta -c10 --file=/tmp/vin-capture/cvbs-colorbars-#.bin /dev/video0; echo pofbqvttib $?
> arm64 ~ # echo tldblgblou $$; yavta -c10 --file=/tmp/vin-capture/cvbs-colorbars-#.bin /dev/video0; echo pofbqvttib $?
> 
> tldblgblou 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> 8 buffers requested.
> length: 691200 offset: 0 timestamp type/source: mono/EoF
> Buffer 0/0 mapped at address 0xffffb75f7000.
> length: 691200 offset: 32768 timestamp type/source: mono/EoF
> Buffer 1/0 mapped at address 0xffffb754e000.
> length: 691200 offset: 65536 timestamp type/source: mono/EoF
> Buffer 2/0 mapped at address 0xffffb74a5000.
> length: 691200 offset: 98304 timestamp type/source: mono/EoF
> Buffer 3/0 mapped at address 0xffffb73fc000.
> length: 691200 offset: 131072 timestamp type/source: mono/EoF
> Buffer 4/0 mapped at address 0xffffb7353000.
> length: 691200 offset: 163840 timestamp type/source: mono/EoF
> Buffer 5/0 mapped at address 0xffffb72aa000.
> length: 691200 offset: 196608 timestamp type/source: mono/EoF
> Buffer 6/0 mapped at address 0xffffb7201000.
> length: 691200 offset: 229376 timestamp type/source: mono/EoF
> Buffer 7/0 mapped at address 0xffffb7158000.
> Unable to start streaming: Broken pipe (32).
> 8 buffers released.
> pofbqvttib 0
> 0;root@arm64:~
> echo maubyyzesr $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev1; echo dzbvegabvy $?
> arm64 ~ # echo maubyyzesr $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev1; echo dzbvegabvy $?
> 
> maubyyzesr 403
> Device /dev/v4l-subdev1 opened.
> Control 0x009f0903 set to 0, is 0
> Unable to get format: Inappropriate ioctl for device (25).
> dzbvegabvy 0
> 0;root@arm64:~
> echo pffcsadwlt $$; test -e /tmp/vin-capture/cvbs-solid-000007.bin; echo xilwxnvrrd $?
> arm64 ~ # echo pffcsadwlt $$; test -e /tmp/vin-capture/cvbs-solid-000007.bin; echo xilwxnvrrd $?
> 
> pffcsadwlt 403
> xilwxnvrrd 1
> echo grglvpwocj $$; grep -l 'adv748x 4-0070 hdmi' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo hvcawnkqdt $?
> 0;root@arm64:~
> arm64 ~ # echo grglvpwocj $$; grep -l 'adv748x 4-0070 hdmi' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo hvcawnkqdt $?
> 
> grglvpwocj 403
> /dev/v4l-subdev4
> hvcawnkqdt 0
> echo nnpccbuzak $$; media-ctl -d /dev/media0 -r; echo zigvuehatt $?
> 0;root@arm64:~
> arm64 ~ # echo nnpccbuzak $$; media-ctl -d /dev/media0 -r; echo zigvuehatt $?
> 
> nnpccbuzak 403
> zigvuehatt 0
> echo xyngygxuyr $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 hdmi':1 -> 'adv748x 4-0070 txa':0 [1]"; echo oknofsbyvn $?
> 0;root@arm64:~
> arm64 ~ # echo xyngygxuyr $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 hdmi':1 -> 'adv748x 4-0070 txa':0 [1]"; echo oknofsbyvn $?
> 
> xyngygxuyr 403
> oknofsbyvn 0
> 0;root@arm64:~
> echo jkzuanypyp $$; media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"; echo onsaxggllz $?
> arm64 ~ # echo jkzuanypyp $$; media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"; echo onsaxggllz $?
> 
> jkzuanypyp 403
> onsaxggllz 0
> 0;root@arm64:~
> echo lttzkqwwoa $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"; echo imxtuqbxgf $?
> arm64 ~ # echo lttzkqwwoa $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"; echo imxtuqbxgf $?
> 
> lttzkqwwoa 403
> 		[stream:0 fmt:RGB888_1X24/1280x720 field:none colorspace:srgb]
> imxtuqbxgf 0
> 0;root@arm64:~
> echo degzwngmzk $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo vgqpdhbhjt $?
> arm64 ~ # echo degzwngmzk $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo vgqpdhbhjt $?
> 
> degzwngmzk 403
> vgqpdhbhjt 0
> 0;root@arm64:~
> echo iwtingcfbz $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txa':0 [fmt:RGB888_1X24/1280x720 field:none]"; echo qircbjupbq $?
> arm64 ~ # echo iwtingcfbz $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txa':0 [fmt:RGB888_1X24/1280x720 field:none]"; echo qircbjupbq $?
> 
> iwtingcfbz 403
> qircbjupbq 0
> 0;root@arm64:~
> echo twrjqisqff $$; media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo ocaivovrxk $?
> arm64 ~ # echo twrjqisqff $$; media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo ocaivovrxk $?
> 
> twrjqisqff 403
> ocaivovrxk 0
> echo ajqwydmzik $$; yavta -f RGB565 -s 1280x720 --field none /dev/video0; echo fayawxvllj $?
> 0;root@arm64:~
> arm64 ~ # echo ajqwydmzik $$; yavta -f RGB565 -s 1280x720 --field none /dev/video0; echo fayawxvllj $?
> 
> ajqwydmzik 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format set: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> fayawxvllj 0
> echo pealtxybun $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev4; echo juhrczaeao $?
> 0;root@arm64:~
> arm64 ~ # echo pealtxybun $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev4; echo juhrczaeao $?
> 
> pealtxybun 403
> Device /dev/v4l-subdev4 opened.
> Control 0x009f0903 set to 1, is 1
> Unable to get format: Inappropriate ioctl for device (25).
> juhrczaeao 0
> 0;root@arm64:~
> echo oaxoraqdid $$; yavta -c10 --file=/tmp/vin-capture/hdmi-solid-#.bin /dev/video0; echo bfsxvmqitj $?
> arm64 ~ # echo oaxoraqdid $$; yavta -c10 --file=/tmp/vin-capture/hdmi-solid-#.bin /dev/video0; echo bfsxvmqitj $?
> 
> oaxoraqdid 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> 8 buffers requested.
> length: 1843200 offset: 0 timestamp type/source: mono/EoF
> Buffer 0/0 mapped at address 0xffffafa8e000.
> length: 1843200 offset: 32768 timestamp type/source: mono/EoF
> Buffer 1/0 mapped at address 0xffffaf8cc000.
> length: 1843200 offset: 65536 timestamp type/source: mono/EoF
> Buffer 2/0 mapped at address 0xffffaf70a000.
> length: 1843200 offset: 98304 timestamp type/source: mono/EoF
> Buffer 3/0 mapped at address 0xffffaf548000.
> length: 1843200 offset: 131072 timestamp type/source: mono/EoF
> Buffer 4/0 mapped at address 0xffffaf386000.
> length: 1843200 offset: 163840 timestamp type/source: mono/EoF
> Buffer 5/0 mapped at address 0xffffaf1c4000.
> length: 1843200 offset: 196608 timestamp type/source: mono/EoF
> Buffer 6/0 mapped at address 0xffffaf002000.
> length: 1843200 offset: 229376 timestamp type/source: mono/EoF
> Buffer 7/0 mapped at address 0xffffaee40000.
> Unable to start streaming: Broken pipe (32).
> 8 buffers released.
> bfsxvmqitj 0
> echo pmbwbnrkkw $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev4; echo ucgivtjwvw $?
> 0;root@arm64:~
> arm64 ~ # echo pmbwbnrkkw $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev4; echo ucgivtjwvw $?
> 
> pmbwbnrkkw 403
> Device /dev/v4l-subdev4 opened.
> Control 0x009f0903 set to 2, is 2
> Unable to get format: Inappropriate ioctl for device (25).
> ucgivtjwvw 0
> 0;root@arm64:~
> echo rvkbtaiuan $$; yavta -c10 --file=/tmp/vin-capture/hdmi-colorbars-#.bin /dev/video0; echo fdlcrvrylc $?
> arm64 ~ # echo rvkbtaiuan $$; yavta -c10 --file=/tmp/vin-capture/hdmi-colorbars-#.bin /dev/video0; echo fdlcrvrylc $?
> 
> rvkbtaiuan 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> 8 buffers requested.
> length: 1843200 offset: 0 timestamp type/source: mono/EoF
> Buffer 0/0 mapped at address 0xffff86f8e000.
> length: 1843200 offset: 32768 timestamp type/source: mono/EoF
> Buffer 1/0 mapped at address 0xffff86dcc000.
> length: 1843200 offset: 65536 timestamp type/source: mono/EoF
> Buffer 2/0 mapped at address 0xffff86c0a000.
> length: 1843200 offset: 98304 timestamp type/source: mono/EoF
> Buffer 3/0 mapped at address 0xffff86a48000.
> length: 1843200 offset: 131072 timestamp type/source: mono/EoF
> Buffer 4/0 mapped at address 0xffff86886000.
> length: 1843200 offset: 163840 timestamp type/source: mono/EoF
> Buffer 5/0 mapped at address 0xffff866c4000.
> length: 1843200 offset: 196608 timestamp type/source: mono/EoF
> Buffer 6/0 mapped at address 0xffff86502000.
> length: 1843200 offset: 229376 timestamp type/source: mono/EoF
> Buffer 7/0 mapped at address 0xffff86340000.
> Unable to start streaming: Broken pipe (32).
> 8 buffers released.
> fdlcrvrylc 0
> 0;root@arm64:~
> echo hrgnrsupul $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev4; echo bvgysmcxve $?
> arm64 ~ # echo hrgnrsupul $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev4; echo bvgysmcxve $?
> 
> hrgnrsupul 403
> Device /dev/v4l-subdev4 opened.
> Control 0x009f0903 set to 0, is 0
> Unable to get format: Inappropriate ioctl for device (25).
> bvgysmcxve 0
> echo znlqovrrra $$; test -e /tmp/vin-capture/hdmi-solid-000007.bin; echo sjpcuiptaf $?
> 0;root@arm64:~
> arm64 ~ # echo znlqovrrra $$; test -e /tmp/vin-capture/hdmi-solid-000007.bin; echo sjpcuiptaf $?
> 
> znlqovrrra 403
> sjpcuiptaf 1
> exit
> ===> end <===
> 
> > 
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >  drivers/media/platform/renesas/rcar-csi2.c | 138 +++++++++++----------
> >  1 file changed, 75 insertions(+), 63 deletions(-)
> > 
> > diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
> > index 2d464e43a5be..4d4dfb59b025 100644
> > --- a/drivers/media/platform/renesas/rcar-csi2.c
> > +++ b/drivers/media/platform/renesas/rcar-csi2.c
> > @@ -587,7 +587,8 @@ enum rcar_csi2_pads {
> >  struct rcar_csi2_info {
> >  	int (*init_phtw)(struct rcar_csi2 *priv, unsigned int mbps);
> >  	int (*phy_post_init)(struct rcar_csi2 *priv);
> > -	int (*start_receiver)(struct rcar_csi2 *priv);
> > +	int (*start_receiver)(struct rcar_csi2 *priv,
> > +			      struct v4l2_subdev_state *state);
> >  	void (*enter_standby)(struct rcar_csi2 *priv);
> >  	const struct rcsi2_mbps_reg *hsfreqrange;
> >  	unsigned int csi0clkfreqrange;
> > @@ -613,8 +614,6 @@ struct rcar_csi2 {
> >  
> >  	int channel_vc[4];
> >  
> > -	struct mutex lock; /* Protects mf and stream_count. */
> > -	struct v4l2_mbus_framefmt mf;
> >  	int stream_count;
> >  
> >  	bool cphy;
> > @@ -808,20 +807,25 @@ static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
> >  	return 0;
> >  }
> >  
> > -static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
> > +static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
> > +				     struct v4l2_subdev_state *state)
> >  {
> >  	const struct rcar_csi2_format *format;
> >  	u32 phycnt, vcdt = 0, vcdt2 = 0, fld = 0;
> > +	struct v4l2_mbus_framefmt *fmt;
> >  	unsigned int lanes;
> >  	unsigned int i;
> >  	int mbps, ret;
> >  
> > +	/* Use the format on the sink pad to compute the receiver config. */
> > +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> > +
> >  	dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
> > -		priv->mf.width, priv->mf.height,
> > -		priv->mf.field == V4L2_FIELD_NONE ? 'p' : 'i');
> > +		fmt->width, fmt->height,
> > +		fmt->field == V4L2_FIELD_NONE ? 'p' : 'i');
> >  
> >  	/* Code is validated in set_fmt. */
> > -	format = rcsi2_code_to_fmt(priv->mf.code);
> > +	format = rcsi2_code_to_fmt(fmt->code);
> >  	if (!format)
> >  		return -EINVAL;
> >  
> > @@ -849,11 +853,11 @@ static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
> >  			vcdt2 |= vcdt_part << ((i % 2) * 16);
> >  	}
> >  
> > -	if (priv->mf.field == V4L2_FIELD_ALTERNATE) {
> > +	if (fmt->field == V4L2_FIELD_ALTERNATE) {
> >  		fld = FLD_DET_SEL(1) | FLD_FLD_EN4 | FLD_FLD_EN3 | FLD_FLD_EN2
> >  			| FLD_FLD_EN;
> >  
> > -		if (priv->mf.height == 240)
> > +		if (fmt->height == 240)
> >  			fld |= FLD_FLD_NUM(0);
> >  		else
> >  			fld |= FLD_FLD_NUM(1);
> > @@ -1049,15 +1053,18 @@ static int rcsi2_c_phy_setting_v4h(struct rcar_csi2 *priv, int msps)
> >  	return 0;
> >  }
> >  
> > -static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
> > +static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv,
> > +				    struct v4l2_subdev_state *state)
> >  {
> >  	const struct rcar_csi2_format *format;
> > +	struct v4l2_mbus_framefmt *fmt;
> >  	unsigned int lanes;
> >  	int msps;
> >  	int ret;
> >  
> > -	/* Calculate parameters */
> > -	format = rcsi2_code_to_fmt(priv->mf.code);
> > +	/* Use the format on the sink pad to compute the receiver config. */
> > +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> > +	format = rcsi2_code_to_fmt(fmt->code);
> >  	if (!format)
> >  		return -EINVAL;
> >  
> > @@ -1114,7 +1121,7 @@ static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
> >  	return 0;
> >  }
> >  
> > -static int rcsi2_start(struct rcar_csi2 *priv)
> > +static int rcsi2_start(struct rcar_csi2 *priv, struct v4l2_subdev_state *state)
> >  {
> >  	int ret;
> >  
> > @@ -1122,7 +1129,7 @@ static int rcsi2_start(struct rcar_csi2 *priv)
> >  	if (ret < 0)
> >  		return ret;
> >  
> > -	ret = priv->info->start_receiver(priv);
> > +	ret = priv->info->start_receiver(priv, state);
> >  	if (ret) {
> >  		rcsi2_enter_standby(priv);
> >  		return ret;
> > @@ -1146,17 +1153,16 @@ static void rcsi2_stop(struct rcar_csi2 *priv)
> >  static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
> >  {
> >  	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > +	struct v4l2_subdev_state *state;
> >  	int ret = 0;
> >  
> > -	mutex_lock(&priv->lock);
> > +	if (!priv->remote)
> > +		return -ENODEV;
> >  
> > -	if (!priv->remote) {
> > -		ret = -ENODEV;
> > -		goto out;
> > -	}
> > +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
> >  
> >  	if (enable && priv->stream_count == 0) {
> > -		ret = rcsi2_start(priv);
> > +		ret = rcsi2_start(priv, state);
> >  		if (ret)
> >  			goto out;
> >  	} else if (!enable && priv->stream_count == 1) {
> > @@ -1165,49 +1171,26 @@ static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
> >  
> >  	priv->stream_count += enable ? 1 : -1;
> >  out:
> > -	mutex_unlock(&priv->lock);
> > +	v4l2_subdev_unlock_state(state);
> >  
> >  	return ret;
> >  }
> >  
> >  static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
> > -				struct v4l2_subdev_state *sd_state,
> > +				struct v4l2_subdev_state *state,
> >  				struct v4l2_subdev_format *format)
> >  {
> > -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > -	struct v4l2_mbus_framefmt *framefmt;
> > -
> > -	mutex_lock(&priv->lock);
> > +	if (format->pad > RCAR_CSI2_SINK)
> > +		return v4l2_subdev_get_fmt(sd, state, format);
> >  
> >  	if (!rcsi2_code_to_fmt(format->format.code))
> >  		format->format.code = rcar_csi2_formats[0].code;
> >  
> > -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> > -		priv->mf = format->format;
> > -	} else {
> > -		framefmt = v4l2_subdev_state_get_format(sd_state, 0);
> > -		*framefmt = format->format;
> > -	}
> > -
> > -	mutex_unlock(&priv->lock);
> > -
> > -	return 0;
> > -}
> > -
> > -static int rcsi2_get_pad_format(struct v4l2_subdev *sd,
> > -				struct v4l2_subdev_state *sd_state,
> > -				struct v4l2_subdev_format *format)
> > -{
> > -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > -
> > -	mutex_lock(&priv->lock);
> > -
> > -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE)
> > -		format->format = priv->mf;
> > -	else
> > -		format->format = *v4l2_subdev_state_get_format(sd_state, 0);
> > +	*v4l2_subdev_state_get_format(state, format->pad) = format->format;
> >  
> > -	mutex_unlock(&priv->lock);
> > +	/* Propagate the format to the source pads. */
> > +	for (unsigned int i = RCAR_CSI2_SOURCE_VC0; i < NR_OF_RCAR_CSI2_PAD; i++)
> > +		*v4l2_subdev_state_get_format(state, i) = format->format;
> >  
> >  	return 0;
> >  }
> > @@ -1218,7 +1201,7 @@ static const struct v4l2_subdev_video_ops rcar_csi2_video_ops = {
> >  
> >  static const struct v4l2_subdev_pad_ops rcar_csi2_pad_ops = {
> >  	.set_fmt = rcsi2_set_pad_format,
> > -	.get_fmt = rcsi2_get_pad_format,
> > +	.get_fmt = v4l2_subdev_get_fmt,
> >  };
> >  
> >  static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
> > @@ -1226,6 +1209,30 @@ static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
> >  	.pad	= &rcar_csi2_pad_ops,
> >  };
> >  
> > +static int rcsi2_init_state(struct v4l2_subdev *sd,
> > +			    struct v4l2_subdev_state *state)
> > +{
> > +	static const struct v4l2_mbus_framefmt rcar_csi2_default_fmt = {
> > +		.width		= 1920,
> > +		.height		= 1080,
> > +		.code		= MEDIA_BUS_FMT_RGB888_1X24,
> > +		.colorspace	= V4L2_COLORSPACE_SRGB,
> > +		.field		= V4L2_FIELD_NONE,
> > +		.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
> > +		.quantization	= V4L2_QUANTIZATION_DEFAULT,
> > +		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
> > +	};
> > +
> > +	for (unsigned int i = RCAR_CSI2_SINK; i < NR_OF_RCAR_CSI2_PAD; i++)
> > +		*v4l2_subdev_state_get_format(state, i) = rcar_csi2_default_fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_internal_ops rcar_csi2_internal_ops = {
> > +	.init_state = rcsi2_init_state,
> > +};
> > +
> >  static irqreturn_t rcsi2_irq(int irq, void *data)
> >  {
> >  	struct rcar_csi2 *priv = data;
> > @@ -1251,14 +1258,17 @@ static irqreturn_t rcsi2_irq(int irq, void *data)
> >  
> >  static irqreturn_t rcsi2_irq_thread(int irq, void *data)
> >  {
> > +	struct v4l2_subdev_state *state;
> >  	struct rcar_csi2 *priv = data;
> >  
> > -	mutex_lock(&priv->lock);
> > +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
> > +
> >  	rcsi2_stop(priv);
> >  	usleep_range(1000, 2000);
> > -	if (rcsi2_start(priv))
> > +	if (rcsi2_start(priv, state))
> >  		dev_warn(priv->dev, "Failed to restart CSI-2 receiver\n");
> > -	mutex_unlock(&priv->lock);
> > +
> > +	v4l2_subdev_unlock_state(state);
> >  
> >  	return IRQ_HANDLED;
> >  }
> > @@ -1870,23 +1880,23 @@ static int rcsi2_probe(struct platform_device *pdev)
> >  
> >  	priv->dev = &pdev->dev;
> >  
> > -	mutex_init(&priv->lock);
> >  	priv->stream_count = 0;
> >  
> >  	ret = rcsi2_probe_resources(priv, pdev);
> >  	if (ret) {
> >  		dev_err(priv->dev, "Failed to get resources\n");
> > -		goto error_mutex;
> > +		return ret;
> >  	}
> >  
> >  	platform_set_drvdata(pdev, priv);
> >  
> >  	ret = rcsi2_parse_dt(priv);
> >  	if (ret)
> > -		goto error_mutex;
> > +		return ret;
> >  
> >  	priv->subdev.owner = THIS_MODULE;
> >  	priv->subdev.dev = &pdev->dev;
> > +	priv->subdev.internal_ops = &rcar_csi2_internal_ops;
> >  	v4l2_subdev_init(&priv->subdev, &rcar_csi2_subdev_ops);
> >  	v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
> >  	snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s",
> > @@ -1912,21 +1922,25 @@ static int rcsi2_probe(struct platform_device *pdev)
> >  
> >  	pm_runtime_enable(&pdev->dev);
> >  
> > +	ret = v4l2_subdev_init_finalize(&priv->subdev);
> > +	if (ret)
> > +		goto error_pm_runtime;
> > +
> >  	ret = v4l2_async_register_subdev(&priv->subdev);
> >  	if (ret < 0)
> > -		goto error_pm_runtime;
> > +		goto error_subdev;
> >  
> >  	dev_info(priv->dev, "%d lanes found\n", priv->lanes);
> >  
> >  	return 0;
> >  
> > +error_subdev:
> > +	v4l2_subdev_cleanup(&priv->subdev);
> >  error_pm_runtime:
> >  	pm_runtime_disable(&pdev->dev);
> >  error_async:
> >  	v4l2_async_nf_unregister(&priv->notifier);
> >  	v4l2_async_nf_cleanup(&priv->notifier);
> > -error_mutex:
> > -	mutex_destroy(&priv->lock);
> >  
> >  	return ret;
> >  }
> > @@ -1941,8 +1955,6 @@ static void rcsi2_remove(struct platform_device *pdev)
> >  	v4l2_subdev_cleanup(&priv->subdev);
> >  
> >  	pm_runtime_disable(&pdev->dev);
> > -
> > -	mutex_destroy(&priv->lock);
> >  }
> >  
> >  static struct platform_driver rcar_csi2_pdrv = {

-- 
Regards,

Laurent Pinchart

Re: [PATCH 04/11] media: rcar-csi2: Use the subdev active state

[Niklas Söderlund]

Hello,

On 2024-05-06 00:52:21 +0300, Laurent Pinchart wrote:
> On Fri, May 03, 2024 at 08:03:21PM +0200, Niklas Söderlund wrote:
> > Hi Jacopo,
> > 
> > Thanks for your work.
> > 
> > On 2024-05-03 17:51:19 +0200, Jacopo Mondi wrote:
> > > Create the subdevice state with v4l2_subdev_init_finalize() and
> > > implement the init_state() operation to guarantee the state is initialized.
> > 
> > This fails and produces -EPIPE error when trying to capture on R-Car M3N 
> > using ADV748x. If I only apply patch 1, 2, 3 of this series it works as 
> > expected, when I apply this as well capture breaks.
> > 
> > You can test this on a remote Gen3 board without any HDMI or CVBS 
> > source, just enable a test patterns on the HDMI and AFE sources, see [1] 
> > for my test-case output which demonstrates how I produce the failure.
> > 
> > Oddly enough if I use the old vin-test package I can capture from HDMI 
> > while CVBS capture still fails with -EPIPE. Not sure what's going on.
> > 
> > 1. Ignore the "Unable to get format: Inappropriate ioctl for device 
> >    (25)." from the "yavta --set-control ..." commands, they are 
> >    unimportant and not new. Focus in the "Unable to start streaming: 
> >    Broken pipe (32)." errors from "yavta -c10 --file=..  /dev/videoX" 
> >    calls, they are not new ;-)
> > 
> > ===> star <===
> > Last login: Fri May  3 17:48:06 2024
> > 
> > Renesas Salvator-X 2nd version board based on r8a77965
> > 0;root@arm64:~
> > arm64 ~ # echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?
> > echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?
> > 
> > wkbfirytku 403
> > ygxezpiinz 0
> > echo agyxksooma $$; mkdir -p /tmp/vin-capture; echo kvwostutox $?
> > 0;root@arm64:~
> > arm64 ~ # echo agyxksooma $$; mkdir -p /tmp/vin-capture; echo kvwostutox $?
> > 
> > agyxksooma 403
> > kvwostutox 0
> > echo vtmzcdhjtc $$; grep -l 'adv748x 4-0070 afe' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo mwqkcmnuju $?
> > 0;root@arm64:~
> > arm64 ~ # echo vtmzcdhjtc $$; grep -l 'adv748x 4-0070 afe' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo mwqkcmnuju $?
> > 
> > vtmzcdhjtc 403
> > /dev/v4l-subdev1
> > mwqkcmnuju 0
> > echo yavkazxvyn $$; grep -l 'VIN0 output' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo wpijcgewrd $?
> > 0;root@arm64:~
> > arm64 ~ # echo yavkazxvyn $$; grep -l 'VIN0 output' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo wpijcgewrd $?
> > 
> > yavkazxvyn 403
> > /dev/video0
> > wpijcgewrd 0
> > echo nkitspntrm $$; for mdev in /dev/media*; do if [[ '$(media-ctl -d $mdev -p | grep 'rcar_vin')' != '' ]]; then echo $mdev; break; fi; done; echo ngoqmbhfqr $?
> > 0;root@arm64:~
> > arm64 ~ # echo nkitspntrm $$; for mdev in /dev/media*; do if [[ '$(media-ctl -d $mdev -p | grep 'rcar_vin')' != '' ]]; then echo $mdev; break; fi; done; echo ngoqmbhfqr $?
> > 
> > nkitspntrm 403
> > /dev/media0
> > ngoqmbhfqr 0
> > echo ggokwokuyh $$; media-ctl -d /dev/media0 -r; echo yydmasvhba $?
> > 0;root@arm64:~
> > arm64 ~ # echo ggokwokuyh $$; media-ctl -d /dev/media0 -r; echo yydmasvhba $?
> > 
> > ggokwokuyh 403
> > yydmasvhba 0
> > 0;root@arm64:~
> > echo ftqgotsvwt $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 afe':8 -> 'adv748x 4-0070 txb':0 [1]"; echo ohfyvhjfrt $?
> > arm64 ~ # echo ftqgotsvwt $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 afe':8 -> 'adv748x 4-0070 txb':0 [1]"; echo ohfyvhjfrt $?
> > 
> > ftqgotsvwt 403
> > ohfyvhjfrt 0
> > 0;root@arm64:~
> > echo nhtnyteajn $$; media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"; echo vovincdzqi $?
> > arm64 ~ # echo nhtnyteajn $$; media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"; echo vovincdzqi $?
> > 
> > nhtnyteajn 403
> > vovincdzqi 0
> > 0;root@arm64:~
> > echo rulkcoorky $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"; echo esoiygqdfq $?
> > arm64 ~ # echo rulkcoorky $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"; echo esoiygqdfq $?
> > 
> > rulkcoorky 403
> > 		[stream:0 fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]
> > esoiygqdfq 0
> > 0;root@arm64:~
> > echo dmdgcctvqh $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo ighfsicfux $?
> > arm64 ~ # echo dmdgcctvqh $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo ighfsicfux $?
> > 
> > dmdgcctvqh 403
> > ighfsicfux 0
> > 0;root@arm64:~
> > echo mqlvxrvync $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txb':0 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo elpiosmlzj $?
> 
> Should this be pad 1 instead of pad 0 ?

Indeed, before this patch I could set the format on either pad 0 or 1, 
after this only on 1. That is OK and setting it on 1 allows capture of 
both HDMI and CVBS on Gen3. However it uncovers other issues in patch 
7/11, will reply to that patch.

> 
> > arm64 ~ # echo mqlvxrvync $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txb':0 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo elpiosmlzj $?
> > 
> > mqlvxrvync 403
> > elpiosmlzj 0
> > 0;root@arm64:~
> > echo aoldaiatak $$; media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo iymsospuex $?
> > arm64 ~ # echo aoldaiatak $$; media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo iymsospuex $?
> > 
> > aoldaiatak 403
> > iymsospuex 0
> > 0;root@arm64:~
> > echo nxpbvfvtbv $$; yavta -f RGB565 -s 720x480 --field interlaced /dev/video0; echo dppdatfote $?
> > arm64 ~ # echo nxpbvfvtbv $$; yavta -f RGB565 -s 720x480 --field interlaced /dev/video0; echo dppdatfote $?
> > 
> > nxpbvfvtbv 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format set: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> > Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> > dppdatfote 0
> > echo zsqtrrojrr $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev1; echo xkrxhzzdcq $?
> > 0;root@arm64:~
> > arm64 ~ # echo zsqtrrojrr $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev1; echo xkrxhzzdcq $?
> > 
> > zsqtrrojrr 403
> > Device /dev/v4l-subdev1 opened.
> > Control 0x009f0903 set to 1, is 1
> > Unable to get format: Inappropriate ioctl for device (25).
> > xkrxhzzdcq 0
> > echo jifvldisax $$; yavta -c10 --file=/tmp/vin-capture/cvbs-solid-#.bin /dev/video0; echo tmzmlnjlsq $?
> > 0;root@arm64:~
> > arm64 ~ # echo jifvldisax $$; yavta -c10 --file=/tmp/vin-capture/cvbs-solid-#.bin /dev/video0; echo tmzmlnjlsq $?
> > 
> > jifvldisax 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> > 8 buffers requested.
> > length: 691200 offset: 0 timestamp type/source: mono/EoF
> > Buffer 0/0 mapped at address 0xffffb1037000.
> > length: 691200 offset: 32768 timestamp type/source: mono/EoF
> > Buffer 1/0 mapped at address 0xffffb0f8e000.
> > length: 691200 offset: 65536 timestamp type/source: mono/EoF
> > Buffer 2/0 mapped at address 0xffffb0ee5000.
> > length: 691200 offset: 98304 timestamp type/source: mono/EoF
> > Buffer 3/0 mapped at address 0xffffb0e3c000.
> > length: 691200 offset: 131072 timestamp type/source: mono/EoF
> > Buffer 4/0 mapped at address 0xffffb0d93000.
> > length: 691200 offset: 163840 timestamp type/source: mono/EoF
> > Buffer 5/0 mapped at address 0xffffb0cea000.
> > length: 691200 offset: 196608 timestamp type/source: mono/EoF
> > Buffer 6/0 mapped at address 0xffffb0c41000.
> > length: 691200 offset: 229376 timestamp type/source: mono/EoF
> > Buffer 7/0 mapped at address 0xffffb0b98000.
> > Unable to start streaming: Broken pipe (32).
> > 8 buffers released.
> > tmzmlnjlsq 0
> > echo wojwgdpxnx $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev1; echo dghaavszch $?
> > 0;root@arm64:~
> > arm64 ~ # echo wojwgdpxnx $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev1; echo dghaavszch $?
> > 
> > wojwgdpxnx 403
> > Device /dev/v4l-subdev1 opened.
> > Control 0x009f0903 set to 2, is 2
> > Unable to get format: Inappropriate ioctl for device (25).
> > dghaavszch 0
> > 0;root@arm64:~
> > echo tldblgblou $$; yavta -c10 --file=/tmp/vin-capture/cvbs-colorbars-#.bin /dev/video0; echo pofbqvttib $?
> > arm64 ~ # echo tldblgblou $$; yavta -c10 --file=/tmp/vin-capture/cvbs-colorbars-#.bin /dev/video0; echo pofbqvttib $?
> > 
> > tldblgblou 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> > 8 buffers requested.
> > length: 691200 offset: 0 timestamp type/source: mono/EoF
> > Buffer 0/0 mapped at address 0xffffb75f7000.
> > length: 691200 offset: 32768 timestamp type/source: mono/EoF
> > Buffer 1/0 mapped at address 0xffffb754e000.
> > length: 691200 offset: 65536 timestamp type/source: mono/EoF
> > Buffer 2/0 mapped at address 0xffffb74a5000.
> > length: 691200 offset: 98304 timestamp type/source: mono/EoF
> > Buffer 3/0 mapped at address 0xffffb73fc000.
> > length: 691200 offset: 131072 timestamp type/source: mono/EoF
> > Buffer 4/0 mapped at address 0xffffb7353000.
> > length: 691200 offset: 163840 timestamp type/source: mono/EoF
> > Buffer 5/0 mapped at address 0xffffb72aa000.
> > length: 691200 offset: 196608 timestamp type/source: mono/EoF
> > Buffer 6/0 mapped at address 0xffffb7201000.
> > length: 691200 offset: 229376 timestamp type/source: mono/EoF
> > Buffer 7/0 mapped at address 0xffffb7158000.
> > Unable to start streaming: Broken pipe (32).
> > 8 buffers released.
> > pofbqvttib 0
> > 0;root@arm64:~
> > echo maubyyzesr $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev1; echo dzbvegabvy $?
> > arm64 ~ # echo maubyyzesr $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev1; echo dzbvegabvy $?
> > 
> > maubyyzesr 403
> > Device /dev/v4l-subdev1 opened.
> > Control 0x009f0903 set to 0, is 0
> > Unable to get format: Inappropriate ioctl for device (25).
> > dzbvegabvy 0
> > 0;root@arm64:~
> > echo pffcsadwlt $$; test -e /tmp/vin-capture/cvbs-solid-000007.bin; echo xilwxnvrrd $?
> > arm64 ~ # echo pffcsadwlt $$; test -e /tmp/vin-capture/cvbs-solid-000007.bin; echo xilwxnvrrd $?
> > 
> > pffcsadwlt 403
> > xilwxnvrrd 1
> > echo grglvpwocj $$; grep -l 'adv748x 4-0070 hdmi' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo hvcawnkqdt $?
> > 0;root@arm64:~
> > arm64 ~ # echo grglvpwocj $$; grep -l 'adv748x 4-0070 hdmi' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo hvcawnkqdt $?
> > 
> > grglvpwocj 403
> > /dev/v4l-subdev4
> > hvcawnkqdt 0
> > echo nnpccbuzak $$; media-ctl -d /dev/media0 -r; echo zigvuehatt $?
> > 0;root@arm64:~
> > arm64 ~ # echo nnpccbuzak $$; media-ctl -d /dev/media0 -r; echo zigvuehatt $?
> > 
> > nnpccbuzak 403
> > zigvuehatt 0
> > echo xyngygxuyr $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 hdmi':1 -> 'adv748x 4-0070 txa':0 [1]"; echo oknofsbyvn $?
> > 0;root@arm64:~
> > arm64 ~ # echo xyngygxuyr $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 hdmi':1 -> 'adv748x 4-0070 txa':0 [1]"; echo oknofsbyvn $?
> > 
> > xyngygxuyr 403
> > oknofsbyvn 0
> > 0;root@arm64:~
> > echo jkzuanypyp $$; media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"; echo onsaxggllz $?
> > arm64 ~ # echo jkzuanypyp $$; media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"; echo onsaxggllz $?
> > 
> > jkzuanypyp 403
> > onsaxggllz 0
> > 0;root@arm64:~
> > echo lttzkqwwoa $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"; echo imxtuqbxgf $?
> > arm64 ~ # echo lttzkqwwoa $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"; echo imxtuqbxgf $?
> > 
> > lttzkqwwoa 403
> > 		[stream:0 fmt:RGB888_1X24/1280x720 field:none colorspace:srgb]
> > imxtuqbxgf 0
> > 0;root@arm64:~
> > echo degzwngmzk $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo vgqpdhbhjt $?
> > arm64 ~ # echo degzwngmzk $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo vgqpdhbhjt $?
> > 
> > degzwngmzk 403
> > vgqpdhbhjt 0
> > 0;root@arm64:~
> > echo iwtingcfbz $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txa':0 [fmt:RGB888_1X24/1280x720 field:none]"; echo qircbjupbq $?
> > arm64 ~ # echo iwtingcfbz $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txa':0 [fmt:RGB888_1X24/1280x720 field:none]"; echo qircbjupbq $?
> > 
> > iwtingcfbz 403
> > qircbjupbq 0
> > 0;root@arm64:~
> > echo twrjqisqff $$; media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo ocaivovrxk $?
> > arm64 ~ # echo twrjqisqff $$; media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo ocaivovrxk $?
> > 
> > twrjqisqff 403
> > ocaivovrxk 0
> > echo ajqwydmzik $$; yavta -f RGB565 -s 1280x720 --field none /dev/video0; echo fayawxvllj $?
> > 0;root@arm64:~
> > arm64 ~ # echo ajqwydmzik $$; yavta -f RGB565 -s 1280x720 --field none /dev/video0; echo fayawxvllj $?
> > 
> > ajqwydmzik 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format set: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> > Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> > fayawxvllj 0
> > echo pealtxybun $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev4; echo juhrczaeao $?
> > 0;root@arm64:~
> > arm64 ~ # echo pealtxybun $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev4; echo juhrczaeao $?
> > 
> > pealtxybun 403
> > Device /dev/v4l-subdev4 opened.
> > Control 0x009f0903 set to 1, is 1
> > Unable to get format: Inappropriate ioctl for device (25).
> > juhrczaeao 0
> > 0;root@arm64:~
> > echo oaxoraqdid $$; yavta -c10 --file=/tmp/vin-capture/hdmi-solid-#.bin /dev/video0; echo bfsxvmqitj $?
> > arm64 ~ # echo oaxoraqdid $$; yavta -c10 --file=/tmp/vin-capture/hdmi-solid-#.bin /dev/video0; echo bfsxvmqitj $?
> > 
> > oaxoraqdid 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> > 8 buffers requested.
> > length: 1843200 offset: 0 timestamp type/source: mono/EoF
> > Buffer 0/0 mapped at address 0xffffafa8e000.
> > length: 1843200 offset: 32768 timestamp type/source: mono/EoF
> > Buffer 1/0 mapped at address 0xffffaf8cc000.
> > length: 1843200 offset: 65536 timestamp type/source: mono/EoF
> > Buffer 2/0 mapped at address 0xffffaf70a000.
> > length: 1843200 offset: 98304 timestamp type/source: mono/EoF
> > Buffer 3/0 mapped at address 0xffffaf548000.
> > length: 1843200 offset: 131072 timestamp type/source: mono/EoF
> > Buffer 4/0 mapped at address 0xffffaf386000.
> > length: 1843200 offset: 163840 timestamp type/source: mono/EoF
> > Buffer 5/0 mapped at address 0xffffaf1c4000.
> > length: 1843200 offset: 196608 timestamp type/source: mono/EoF
> > Buffer 6/0 mapped at address 0xffffaf002000.
> > length: 1843200 offset: 229376 timestamp type/source: mono/EoF
> > Buffer 7/0 mapped at address 0xffffaee40000.
> > Unable to start streaming: Broken pipe (32).
> > 8 buffers released.
> > bfsxvmqitj 0
> > echo pmbwbnrkkw $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev4; echo ucgivtjwvw $?
> > 0;root@arm64:~
> > arm64 ~ # echo pmbwbnrkkw $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev4; echo ucgivtjwvw $?
> > 
> > pmbwbnrkkw 403
> > Device /dev/v4l-subdev4 opened.
> > Control 0x009f0903 set to 2, is 2
> > Unable to get format: Inappropriate ioctl for device (25).
> > ucgivtjwvw 0
> > 0;root@arm64:~
> > echo rvkbtaiuan $$; yavta -c10 --file=/tmp/vin-capture/hdmi-colorbars-#.bin /dev/video0; echo fdlcrvrylc $?
> > arm64 ~ # echo rvkbtaiuan $$; yavta -c10 --file=/tmp/vin-capture/hdmi-colorbars-#.bin /dev/video0; echo fdlcrvrylc $?
> > 
> > rvkbtaiuan 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> > 8 buffers requested.
> > length: 1843200 offset: 0 timestamp type/source: mono/EoF
> > Buffer 0/0 mapped at address 0xffff86f8e000.
> > length: 1843200 offset: 32768 timestamp type/source: mono/EoF
> > Buffer 1/0 mapped at address 0xffff86dcc000.
> > length: 1843200 offset: 65536 timestamp type/source: mono/EoF
> > Buffer 2/0 mapped at address 0xffff86c0a000.
> > length: 1843200 offset: 98304 timestamp type/source: mono/EoF
> > Buffer 3/0 mapped at address 0xffff86a48000.
> > length: 1843200 offset: 131072 timestamp type/source: mono/EoF
> > Buffer 4/0 mapped at address 0xffff86886000.
> > length: 1843200 offset: 163840 timestamp type/source: mono/EoF
> > Buffer 5/0 mapped at address 0xffff866c4000.
> > length: 1843200 offset: 196608 timestamp type/source: mono/EoF
> > Buffer 6/0 mapped at address 0xffff86502000.
> > length: 1843200 offset: 229376 timestamp type/source: mono/EoF
> > Buffer 7/0 mapped at address 0xffff86340000.
> > Unable to start streaming: Broken pipe (32).
> > 8 buffers released.
> > fdlcrvrylc 0
> > 0;root@arm64:~
> > echo hrgnrsupul $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev4; echo bvgysmcxve $?
> > arm64 ~ # echo hrgnrsupul $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev4; echo bvgysmcxve $?
> > 
> > hrgnrsupul 403
> > Device /dev/v4l-subdev4 opened.
> > Control 0x009f0903 set to 0, is 0
> > Unable to get format: Inappropriate ioctl for device (25).
> > bvgysmcxve 0
> > echo znlqovrrra $$; test -e /tmp/vin-capture/hdmi-solid-000007.bin; echo sjpcuiptaf $?
> > 0;root@arm64:~
> > arm64 ~ # echo znlqovrrra $$; test -e /tmp/vin-capture/hdmi-solid-000007.bin; echo sjpcuiptaf $?
> > 
> > znlqovrrra 403
> > sjpcuiptaf 1
> > exit
> > ===> end <===
> > 
> > > 
> > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > ---
> > >  drivers/media/platform/renesas/rcar-csi2.c | 138 +++++++++++----------
> > >  1 file changed, 75 insertions(+), 63 deletions(-)
> > > 
> > > diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
> > > index 2d464e43a5be..4d4dfb59b025 100644
> > > --- a/drivers/media/platform/renesas/rcar-csi2.c
> > > +++ b/drivers/media/platform/renesas/rcar-csi2.c
> > > @@ -587,7 +587,8 @@ enum rcar_csi2_pads {
> > >  struct rcar_csi2_info {
> > >  	int (*init_phtw)(struct rcar_csi2 *priv, unsigned int mbps);
> > >  	int (*phy_post_init)(struct rcar_csi2 *priv);
> > > -	int (*start_receiver)(struct rcar_csi2 *priv);
> > > +	int (*start_receiver)(struct rcar_csi2 *priv,
> > > +			      struct v4l2_subdev_state *state);
> > >  	void (*enter_standby)(struct rcar_csi2 *priv);
> > >  	const struct rcsi2_mbps_reg *hsfreqrange;
> > >  	unsigned int csi0clkfreqrange;
> > > @@ -613,8 +614,6 @@ struct rcar_csi2 {
> > >  
> > >  	int channel_vc[4];
> > >  
> > > -	struct mutex lock; /* Protects mf and stream_count. */
> > > -	struct v4l2_mbus_framefmt mf;
> > >  	int stream_count;
> > >  
> > >  	bool cphy;
> > > @@ -808,20 +807,25 @@ static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
> > >  	return 0;
> > >  }
> > >  
> > > -static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
> > > +static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
> > > +				     struct v4l2_subdev_state *state)
> > >  {
> > >  	const struct rcar_csi2_format *format;
> > >  	u32 phycnt, vcdt = 0, vcdt2 = 0, fld = 0;
> > > +	struct v4l2_mbus_framefmt *fmt;
> > >  	unsigned int lanes;
> > >  	unsigned int i;
> > >  	int mbps, ret;
> > >  
> > > +	/* Use the format on the sink pad to compute the receiver config. */
> > > +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> > > +
> > >  	dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
> > > -		priv->mf.width, priv->mf.height,
> > > -		priv->mf.field == V4L2_FIELD_NONE ? 'p' : 'i');
> > > +		fmt->width, fmt->height,
> > > +		fmt->field == V4L2_FIELD_NONE ? 'p' : 'i');
> > >  
> > >  	/* Code is validated in set_fmt. */
> > > -	format = rcsi2_code_to_fmt(priv->mf.code);
> > > +	format = rcsi2_code_to_fmt(fmt->code);
> > >  	if (!format)
> > >  		return -EINVAL;
> > >  
> > > @@ -849,11 +853,11 @@ static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
> > >  			vcdt2 |= vcdt_part << ((i % 2) * 16);
> > >  	}
> > >  
> > > -	if (priv->mf.field == V4L2_FIELD_ALTERNATE) {
> > > +	if (fmt->field == V4L2_FIELD_ALTERNATE) {
> > >  		fld = FLD_DET_SEL(1) | FLD_FLD_EN4 | FLD_FLD_EN3 | FLD_FLD_EN2
> > >  			| FLD_FLD_EN;
> > >  
> > > -		if (priv->mf.height == 240)
> > > +		if (fmt->height == 240)
> > >  			fld |= FLD_FLD_NUM(0);
> > >  		else
> > >  			fld |= FLD_FLD_NUM(1);
> > > @@ -1049,15 +1053,18 @@ static int rcsi2_c_phy_setting_v4h(struct rcar_csi2 *priv, int msps)
> > >  	return 0;
> > >  }
> > >  
> > > -static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
> > > +static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv,
> > > +				    struct v4l2_subdev_state *state)
> > >  {
> > >  	const struct rcar_csi2_format *format;
> > > +	struct v4l2_mbus_framefmt *fmt;
> > >  	unsigned int lanes;
> > >  	int msps;
> > >  	int ret;
> > >  
> > > -	/* Calculate parameters */
> > > -	format = rcsi2_code_to_fmt(priv->mf.code);
> > > +	/* Use the format on the sink pad to compute the receiver config. */
> > > +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> > > +	format = rcsi2_code_to_fmt(fmt->code);
> > >  	if (!format)
> > >  		return -EINVAL;
> > >  
> > > @@ -1114,7 +1121,7 @@ static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
> > >  	return 0;
> > >  }
> > >  
> > > -static int rcsi2_start(struct rcar_csi2 *priv)
> > > +static int rcsi2_start(struct rcar_csi2 *priv, struct v4l2_subdev_state *state)
> > >  {
> > >  	int ret;
> > >  
> > > @@ -1122,7 +1129,7 @@ static int rcsi2_start(struct rcar_csi2 *priv)
> > >  	if (ret < 0)
> > >  		return ret;
> > >  
> > > -	ret = priv->info->start_receiver(priv);
> > > +	ret = priv->info->start_receiver(priv, state);
> > >  	if (ret) {
> > >  		rcsi2_enter_standby(priv);
> > >  		return ret;
> > > @@ -1146,17 +1153,16 @@ static void rcsi2_stop(struct rcar_csi2 *priv)
> > >  static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
> > >  {
> > >  	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > > +	struct v4l2_subdev_state *state;
> > >  	int ret = 0;
> > >  
> > > -	mutex_lock(&priv->lock);
> > > +	if (!priv->remote)
> > > +		return -ENODEV;
> > >  
> > > -	if (!priv->remote) {
> > > -		ret = -ENODEV;
> > > -		goto out;
> > > -	}
> > > +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
> > >  
> > >  	if (enable && priv->stream_count == 0) {
> > > -		ret = rcsi2_start(priv);
> > > +		ret = rcsi2_start(priv, state);
> > >  		if (ret)
> > >  			goto out;
> > >  	} else if (!enable && priv->stream_count == 1) {
> > > @@ -1165,49 +1171,26 @@ static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
> > >  
> > >  	priv->stream_count += enable ? 1 : -1;
> > >  out:
> > > -	mutex_unlock(&priv->lock);
> > > +	v4l2_subdev_unlock_state(state);
> > >  
> > >  	return ret;
> > >  }
> > >  
> > >  static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
> > > -				struct v4l2_subdev_state *sd_state,
> > > +				struct v4l2_subdev_state *state,
> > >  				struct v4l2_subdev_format *format)
> > >  {
> > > -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > > -	struct v4l2_mbus_framefmt *framefmt;
> > > -
> > > -	mutex_lock(&priv->lock);
> > > +	if (format->pad > RCAR_CSI2_SINK)
> > > +		return v4l2_subdev_get_fmt(sd, state, format);
> > >  
> > >  	if (!rcsi2_code_to_fmt(format->format.code))
> > >  		format->format.code = rcar_csi2_formats[0].code;
> > >  
> > > -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> > > -		priv->mf = format->format;
> > > -	} else {
> > > -		framefmt = v4l2_subdev_state_get_format(sd_state, 0);
> > > -		*framefmt = format->format;
> > > -	}
> > > -
> > > -	mutex_unlock(&priv->lock);
> > > -
> > > -	return 0;
> > > -}
> > > -
> > > -static int rcsi2_get_pad_format(struct v4l2_subdev *sd,
> > > -				struct v4l2_subdev_state *sd_state,
> > > -				struct v4l2_subdev_format *format)
> > > -{
> > > -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > > -
> > > -	mutex_lock(&priv->lock);
> > > -
> > > -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE)
> > > -		format->format = priv->mf;
> > > -	else
> > > -		format->format = *v4l2_subdev_state_get_format(sd_state, 0);
> > > +	*v4l2_subdev_state_get_format(state, format->pad) = format->format;
> > >  
> > > -	mutex_unlock(&priv->lock);
> > > +	/* Propagate the format to the source pads. */
> > > +	for (unsigned int i = RCAR_CSI2_SOURCE_VC0; i < NR_OF_RCAR_CSI2_PAD; i++)
> > > +		*v4l2_subdev_state_get_format(state, i) = format->format;
> > >  
> > >  	return 0;
> > >  }
> > > @@ -1218,7 +1201,7 @@ static const struct v4l2_subdev_video_ops rcar_csi2_video_ops = {
> > >  
> > >  static const struct v4l2_subdev_pad_ops rcar_csi2_pad_ops = {
> > >  	.set_fmt = rcsi2_set_pad_format,
> > > -	.get_fmt = rcsi2_get_pad_format,
> > > +	.get_fmt = v4l2_subdev_get_fmt,
> > >  };
> > >  
> > >  static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
> > > @@ -1226,6 +1209,30 @@ static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
> > >  	.pad	= &rcar_csi2_pad_ops,
> > >  };
> > >  
> > > +static int rcsi2_init_state(struct v4l2_subdev *sd,
> > > +			    struct v4l2_subdev_state *state)
> > > +{
> > > +	static const struct v4l2_mbus_framefmt rcar_csi2_default_fmt = {
> > > +		.width		= 1920,
> > > +		.height		= 1080,
> > > +		.code		= MEDIA_BUS_FMT_RGB888_1X24,
> > > +		.colorspace	= V4L2_COLORSPACE_SRGB,
> > > +		.field		= V4L2_FIELD_NONE,
> > > +		.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
> > > +		.quantization	= V4L2_QUANTIZATION_DEFAULT,
> > > +		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
> > > +	};
> > > +
> > > +	for (unsigned int i = RCAR_CSI2_SINK; i < NR_OF_RCAR_CSI2_PAD; i++)
> > > +		*v4l2_subdev_state_get_format(state, i) = rcar_csi2_default_fmt;
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static const struct v4l2_subdev_internal_ops rcar_csi2_internal_ops = {
> > > +	.init_state = rcsi2_init_state,
> > > +};
> > > +
> > >  static irqreturn_t rcsi2_irq(int irq, void *data)
> > >  {
> > >  	struct rcar_csi2 *priv = data;
> > > @@ -1251,14 +1258,17 @@ static irqreturn_t rcsi2_irq(int irq, void *data)
> > >  
> > >  static irqreturn_t rcsi2_irq_thread(int irq, void *data)
> > >  {
> > > +	struct v4l2_subdev_state *state;
> > >  	struct rcar_csi2 *priv = data;
> > >  
> > > -	mutex_lock(&priv->lock);
> > > +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
> > > +
> > >  	rcsi2_stop(priv);
> > >  	usleep_range(1000, 2000);
> > > -	if (rcsi2_start(priv))
> > > +	if (rcsi2_start(priv, state))
> > >  		dev_warn(priv->dev, "Failed to restart CSI-2 receiver\n");
> > > -	mutex_unlock(&priv->lock);
> > > +
> > > +	v4l2_subdev_unlock_state(state);
> > >  
> > >  	return IRQ_HANDLED;
> > >  }
> > > @@ -1870,23 +1880,23 @@ static int rcsi2_probe(struct platform_device *pdev)
> > >  
> > >  	priv->dev = &pdev->dev;
> > >  
> > > -	mutex_init(&priv->lock);
> > >  	priv->stream_count = 0;
> > >  
> > >  	ret = rcsi2_probe_resources(priv, pdev);
> > >  	if (ret) {
> > >  		dev_err(priv->dev, "Failed to get resources\n");
> > > -		goto error_mutex;
> > > +		return ret;
> > >  	}
> > >  
> > >  	platform_set_drvdata(pdev, priv);
> > >  
> > >  	ret = rcsi2_parse_dt(priv);
> > >  	if (ret)
> > > -		goto error_mutex;
> > > +		return ret;
> > >  
> > >  	priv->subdev.owner = THIS_MODULE;
> > >  	priv->subdev.dev = &pdev->dev;
> > > +	priv->subdev.internal_ops = &rcar_csi2_internal_ops;
> > >  	v4l2_subdev_init(&priv->subdev, &rcar_csi2_subdev_ops);
> > >  	v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
> > >  	snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s",
> > > @@ -1912,21 +1922,25 @@ static int rcsi2_probe(struct platform_device *pdev)
> > >  
> > >  	pm_runtime_enable(&pdev->dev);
> > >  
> > > +	ret = v4l2_subdev_init_finalize(&priv->subdev);
> > > +	if (ret)
> > > +		goto error_pm_runtime;
> > > +
> > >  	ret = v4l2_async_register_subdev(&priv->subdev);
> > >  	if (ret < 0)
> > > -		goto error_pm_runtime;
> > > +		goto error_subdev;
> > >  
> > >  	dev_info(priv->dev, "%d lanes found\n", priv->lanes);
> > >  
> > >  	return 0;
> > >  
> > > +error_subdev:
> > > +	v4l2_subdev_cleanup(&priv->subdev);
> > >  error_pm_runtime:
> > >  	pm_runtime_disable(&pdev->dev);
> > >  error_async:
> > >  	v4l2_async_nf_unregister(&priv->notifier);
> > >  	v4l2_async_nf_cleanup(&priv->notifier);
> > > -error_mutex:
> > > -	mutex_destroy(&priv->lock);
> > >  
> > >  	return ret;
> > >  }
> > > @@ -1941,8 +1955,6 @@ static void rcsi2_remove(struct platform_device *pdev)
> > >  	v4l2_subdev_cleanup(&priv->subdev);
> > >  
> > >  	pm_runtime_disable(&pdev->dev);
> > > -
> > > -	mutex_destroy(&priv->lock);
> > >  }
> > >  
> > >  static struct platform_driver rcar_csi2_pdrv = {
> 
> -- 
> Regards,
> 
> Laurent Pinchart

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH 04/11] media: rcar-csi2: Use the subdev active state

[Jacopo Mondi]

Hi Niklas


On Fri, May 03, 2024 at 08:03:21PM GMT, Niklas Söderlund wrote:
> Hi Jacopo,
>
> Thanks for your work.
>
> On 2024-05-03 17:51:19 +0200, Jacopo Mondi wrote:
> > Create the subdevice state with v4l2_subdev_init_finalize() and
> > implement the init_state() operation to guarantee the state is initialized.
>
> This fails and produces -EPIPE error when trying to capture on R-Car M3N
> using ADV748x. If I only apply patch 1, 2, 3 of this series it works as
> expected, when I apply this as well capture breaks.
>
> You can test this on a remote Gen3 board without any HDMI or CVBS
> source, just enable a test patterns on the HDMI and AFE sources, see [1]
> for my test-case output which demonstrates how I produce the failure.
>
> Oddly enough if I use the old vin-test package I can capture from HDMI
> while CVBS capture still fails with -EPIPE. Not sure what's going on.
>
> 1. Ignore the "Unable to get format: Inappropriate ioctl for device
>    (25)." from the "yavta --set-control ..." commands, they are
>    unimportant and not new. Focus in the "Unable to start streaming:
>    Broken pipe (32)." errors from "yavta -c10 --file=..  /dev/videoX"
>    calls, they are not new ;-)
>
> ===> star <===
> Last login: Fri May  3 17:48:06 2024
>
> Renesas Salvator-X 2nd version board based on r8a77965
> 0;root@arm64:~
> arm64 ~ # echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?
> echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?

What are these random strings in test logs ? (out of curiosity)

>
> wkbfirytku 403
> ygxezpiinz 0
> echo agyxksooma $$; mkdir -p /tmp/vin-capture; echo kvwostutox $?
> 0;root@arm64:~
> arm64 ~ # echo agyxksooma $$; mkdir -p /tmp/vin-capture; echo kvwostutox $?
>
> agyxksooma 403
> kvwostutox 0
> echo vtmzcdhjtc $$; grep -l 'adv748x 4-0070 afe' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo mwqkcmnuju $?
> 0;root@arm64:~
> arm64 ~ # echo vtmzcdhjtc $$; grep -l 'adv748x 4-0070 afe' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo mwqkcmnuju $?
>
> vtmzcdhjtc 403
> /dev/v4l-subdev1
> mwqkcmnuju 0
> echo yavkazxvyn $$; grep -l 'VIN0 output' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo wpijcgewrd $?
> 0;root@arm64:~
> arm64 ~ # echo yavkazxvyn $$; grep -l 'VIN0 output' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo wpijcgewrd $?
>
> yavkazxvyn 403
> /dev/video0
> wpijcgewrd 0
> echo nkitspntrm $$; for mdev in /dev/media*; do if [[ '$(media-ctl -d $mdev -p | grep 'rcar_vin')' != '' ]]; then echo $mdev; break; fi; done; echo ngoqmbhfqr $?
> 0;root@arm64:~
> arm64 ~ # echo nkitspntrm $$; for mdev in /dev/media*; do if [[ '$(media-ctl -d $mdev -p | grep 'rcar_vin')' != '' ]]; then echo $mdev; break; fi; done; echo ngoqmbhfqr $?
>
> nkitspntrm 403
> /dev/media0
> ngoqmbhfqr 0
> echo ggokwokuyh $$; media-ctl -d /dev/media0 -r; echo yydmasvhba $?
> 0;root@arm64:~
> arm64 ~ # echo ggokwokuyh $$; media-ctl -d /dev/media0 -r; echo yydmasvhba $?
>
> ggokwokuyh 403
> yydmasvhba 0
> 0;root@arm64:~
> echo ftqgotsvwt $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 afe':8 -> 'adv748x 4-0070 txb':0 [1]"; echo ohfyvhjfrt $?
> arm64 ~ # echo ftqgotsvwt $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 afe':8 -> 'adv748x 4-0070 txb':0 [1]"; echo ohfyvhjfrt $?
>
> ftqgotsvwt 403
> ohfyvhjfrt 0
> 0;root@arm64:~
> echo nhtnyteajn $$; media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"; echo vovincdzqi $?
> arm64 ~ # echo nhtnyteajn $$; media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"; echo vovincdzqi $?
>
> nhtnyteajn 403
> vovincdzqi 0
> 0;root@arm64:~
> echo rulkcoorky $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"; echo esoiygqdfq $?
> arm64 ~ # echo rulkcoorky $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"; echo esoiygqdfq $?
>
> rulkcoorky 403
> 		[stream:0 fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]
> esoiygqdfq 0
> 0;root@arm64:~
> echo dmdgcctvqh $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo ighfsicfux $?
> arm64 ~ # echo dmdgcctvqh $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo ighfsicfux $?
>
> dmdgcctvqh 403
> ighfsicfux 0
> 0;root@arm64:~
> echo mqlvxrvync $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txb':0 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo elpiosmlzj $?
> arm64 ~ # echo mqlvxrvync $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txb':0 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo elpiosmlzj $?
>
> mqlvxrvync 403
> elpiosmlzj 0
> 0;root@arm64:~
> echo aoldaiatak $$; media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo iymsospuex $?
> arm64 ~ # echo aoldaiatak $$; media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo iymsospuex $?

Please read the cover letter

-------------------------------------------------------------------------------
- R-Car CSI-2

There's a user-visible change here, as in the past it was possible to set
format on all pads, and the vin-test scripts do that. In this version I decided
to make the format configurable on the sink and propagate to sources. I'm not
sure if this can be considered a user visible change and we need to maintain
backward compatibility
-------------------------------------------------------------------------------

You should set format on pad #0

How do you feel about that ?


>
> aoldaiatak 403
> iymsospuex 0
> 0;root@arm64:~
> echo nxpbvfvtbv $$; yavta -f RGB565 -s 720x480 --field interlaced /dev/video0; echo dppdatfote $?
> arm64 ~ # echo nxpbvfvtbv $$; yavta -f RGB565 -s 720x480 --field interlaced /dev/video0; echo dppdatfote $?
>
> nxpbvfvtbv 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format set: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> dppdatfote 0
> echo zsqtrrojrr $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev1; echo xkrxhzzdcq $?
> 0;root@arm64:~
> arm64 ~ # echo zsqtrrojrr $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev1; echo xkrxhzzdcq $?
>
> zsqtrrojrr 403
> Device /dev/v4l-subdev1 opened.
> Control 0x009f0903 set to 1, is 1
> Unable to get format: Inappropriate ioctl for device (25).
> xkrxhzzdcq 0
> echo jifvldisax $$; yavta -c10 --file=/tmp/vin-capture/cvbs-solid-#.bin /dev/video0; echo tmzmlnjlsq $?
> 0;root@arm64:~
> arm64 ~ # echo jifvldisax $$; yavta -c10 --file=/tmp/vin-capture/cvbs-solid-#.bin /dev/video0; echo tmzmlnjlsq $?
>
> jifvldisax 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> 8 buffers requested.
> length: 691200 offset: 0 timestamp type/source: mono/EoF
> Buffer 0/0 mapped at address 0xffffb1037000.
> length: 691200 offset: 32768 timestamp type/source: mono/EoF
> Buffer 1/0 mapped at address 0xffffb0f8e000.
> length: 691200 offset: 65536 timestamp type/source: mono/EoF
> Buffer 2/0 mapped at address 0xffffb0ee5000.
> length: 691200 offset: 98304 timestamp type/source: mono/EoF
> Buffer 3/0 mapped at address 0xffffb0e3c000.
> length: 691200 offset: 131072 timestamp type/source: mono/EoF
> Buffer 4/0 mapped at address 0xffffb0d93000.
> length: 691200 offset: 163840 timestamp type/source: mono/EoF
> Buffer 5/0 mapped at address 0xffffb0cea000.
> length: 691200 offset: 196608 timestamp type/source: mono/EoF
> Buffer 6/0 mapped at address 0xffffb0c41000.
> length: 691200 offset: 229376 timestamp type/source: mono/EoF
> Buffer 7/0 mapped at address 0xffffb0b98000.
> Unable to start streaming: Broken pipe (32).
> 8 buffers released.
> tmzmlnjlsq 0
> echo wojwgdpxnx $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev1; echo dghaavszch $?
> 0;root@arm64:~
> arm64 ~ # echo wojwgdpxnx $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev1; echo dghaavszch $?
>
> wojwgdpxnx 403
> Device /dev/v4l-subdev1 opened.
> Control 0x009f0903 set to 2, is 2
> Unable to get format: Inappropriate ioctl for device (25).
> dghaavszch 0
> 0;root@arm64:~
> echo tldblgblou $$; yavta -c10 --file=/tmp/vin-capture/cvbs-colorbars-#.bin /dev/video0; echo pofbqvttib $?
> arm64 ~ # echo tldblgblou $$; yavta -c10 --file=/tmp/vin-capture/cvbs-colorbars-#.bin /dev/video0; echo pofbqvttib $?
>
> tldblgblou 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> 8 buffers requested.
> length: 691200 offset: 0 timestamp type/source: mono/EoF
> Buffer 0/0 mapped at address 0xffffb75f7000.
> length: 691200 offset: 32768 timestamp type/source: mono/EoF
> Buffer 1/0 mapped at address 0xffffb754e000.
> length: 691200 offset: 65536 timestamp type/source: mono/EoF
> Buffer 2/0 mapped at address 0xffffb74a5000.
> length: 691200 offset: 98304 timestamp type/source: mono/EoF
> Buffer 3/0 mapped at address 0xffffb73fc000.
> length: 691200 offset: 131072 timestamp type/source: mono/EoF
> Buffer 4/0 mapped at address 0xffffb7353000.
> length: 691200 offset: 163840 timestamp type/source: mono/EoF
> Buffer 5/0 mapped at address 0xffffb72aa000.
> length: 691200 offset: 196608 timestamp type/source: mono/EoF
> Buffer 6/0 mapped at address 0xffffb7201000.
> length: 691200 offset: 229376 timestamp type/source: mono/EoF
> Buffer 7/0 mapped at address 0xffffb7158000.
> Unable to start streaming: Broken pipe (32).
> 8 buffers released.
> pofbqvttib 0
> 0;root@arm64:~
> echo maubyyzesr $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev1; echo dzbvegabvy $?
> arm64 ~ # echo maubyyzesr $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev1; echo dzbvegabvy $?
>
> maubyyzesr 403
> Device /dev/v4l-subdev1 opened.
> Control 0x009f0903 set to 0, is 0
> Unable to get format: Inappropriate ioctl for device (25).
> dzbvegabvy 0
> 0;root@arm64:~
> echo pffcsadwlt $$; test -e /tmp/vin-capture/cvbs-solid-000007.bin; echo xilwxnvrrd $?
> arm64 ~ # echo pffcsadwlt $$; test -e /tmp/vin-capture/cvbs-solid-000007.bin; echo xilwxnvrrd $?
>
> pffcsadwlt 403
> xilwxnvrrd 1
> echo grglvpwocj $$; grep -l 'adv748x 4-0070 hdmi' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo hvcawnkqdt $?
> 0;root@arm64:~
> arm64 ~ # echo grglvpwocj $$; grep -l 'adv748x 4-0070 hdmi' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo hvcawnkqdt $?
>
> grglvpwocj 403
> /dev/v4l-subdev4
> hvcawnkqdt 0
> echo nnpccbuzak $$; media-ctl -d /dev/media0 -r; echo zigvuehatt $?
> 0;root@arm64:~
> arm64 ~ # echo nnpccbuzak $$; media-ctl -d /dev/media0 -r; echo zigvuehatt $?
>
> nnpccbuzak 403
> zigvuehatt 0
> echo xyngygxuyr $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 hdmi':1 -> 'adv748x 4-0070 txa':0 [1]"; echo oknofsbyvn $?
> 0;root@arm64:~
> arm64 ~ # echo xyngygxuyr $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 hdmi':1 -> 'adv748x 4-0070 txa':0 [1]"; echo oknofsbyvn $?
>
> xyngygxuyr 403
> oknofsbyvn 0
> 0;root@arm64:~
> echo jkzuanypyp $$; media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"; echo onsaxggllz $?
> arm64 ~ # echo jkzuanypyp $$; media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"; echo onsaxggllz $?
>
> jkzuanypyp 403
> onsaxggllz 0
> 0;root@arm64:~
> echo lttzkqwwoa $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"; echo imxtuqbxgf $?
> arm64 ~ # echo lttzkqwwoa $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"; echo imxtuqbxgf $?
>
> lttzkqwwoa 403
> 		[stream:0 fmt:RGB888_1X24/1280x720 field:none colorspace:srgb]
> imxtuqbxgf 0
> 0;root@arm64:~
> echo degzwngmzk $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo vgqpdhbhjt $?
> arm64 ~ # echo degzwngmzk $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo vgqpdhbhjt $?
>
> degzwngmzk 403
> vgqpdhbhjt 0
> 0;root@arm64:~
> echo iwtingcfbz $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txa':0 [fmt:RGB888_1X24/1280x720 field:none]"; echo qircbjupbq $?
> arm64 ~ # echo iwtingcfbz $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txa':0 [fmt:RGB888_1X24/1280x720 field:none]"; echo qircbjupbq $?
>
> iwtingcfbz 403
> qircbjupbq 0
> 0;root@arm64:~
> echo twrjqisqff $$; media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo ocaivovrxk $?
> arm64 ~ # echo twrjqisqff $$; media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo ocaivovrxk $?
>
> twrjqisqff 403
> ocaivovrxk 0
> echo ajqwydmzik $$; yavta -f RGB565 -s 1280x720 --field none /dev/video0; echo fayawxvllj $?
> 0;root@arm64:~
> arm64 ~ # echo ajqwydmzik $$; yavta -f RGB565 -s 1280x720 --field none /dev/video0; echo fayawxvllj $?
>
> ajqwydmzik 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format set: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> fayawxvllj 0
> echo pealtxybun $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev4; echo juhrczaeao $?
> 0;root@arm64:~
> arm64 ~ # echo pealtxybun $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev4; echo juhrczaeao $?
>
> pealtxybun 403
> Device /dev/v4l-subdev4 opened.
> Control 0x009f0903 set to 1, is 1
> Unable to get format: Inappropriate ioctl for device (25).
> juhrczaeao 0
> 0;root@arm64:~
> echo oaxoraqdid $$; yavta -c10 --file=/tmp/vin-capture/hdmi-solid-#.bin /dev/video0; echo bfsxvmqitj $?
> arm64 ~ # echo oaxoraqdid $$; yavta -c10 --file=/tmp/vin-capture/hdmi-solid-#.bin /dev/video0; echo bfsxvmqitj $?
>
> oaxoraqdid 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> 8 buffers requested.
> length: 1843200 offset: 0 timestamp type/source: mono/EoF
> Buffer 0/0 mapped at address 0xffffafa8e000.
> length: 1843200 offset: 32768 timestamp type/source: mono/EoF
> Buffer 1/0 mapped at address 0xffffaf8cc000.
> length: 1843200 offset: 65536 timestamp type/source: mono/EoF
> Buffer 2/0 mapped at address 0xffffaf70a000.
> length: 1843200 offset: 98304 timestamp type/source: mono/EoF
> Buffer 3/0 mapped at address 0xffffaf548000.
> length: 1843200 offset: 131072 timestamp type/source: mono/EoF
> Buffer 4/0 mapped at address 0xffffaf386000.
> length: 1843200 offset: 163840 timestamp type/source: mono/EoF
> Buffer 5/0 mapped at address 0xffffaf1c4000.
> length: 1843200 offset: 196608 timestamp type/source: mono/EoF
> Buffer 6/0 mapped at address 0xffffaf002000.
> length: 1843200 offset: 229376 timestamp type/source: mono/EoF
> Buffer 7/0 mapped at address 0xffffaee40000.
> Unable to start streaming: Broken pipe (32).
> 8 buffers released.
> bfsxvmqitj 0
> echo pmbwbnrkkw $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev4; echo ucgivtjwvw $?
> 0;root@arm64:~
> arm64 ~ # echo pmbwbnrkkw $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev4; echo ucgivtjwvw $?
>
> pmbwbnrkkw 403
> Device /dev/v4l-subdev4 opened.
> Control 0x009f0903 set to 2, is 2
> Unable to get format: Inappropriate ioctl for device (25).
> ucgivtjwvw 0
> 0;root@arm64:~
> echo rvkbtaiuan $$; yavta -c10 --file=/tmp/vin-capture/hdmi-colorbars-#.bin /dev/video0; echo fdlcrvrylc $?
> arm64 ~ # echo rvkbtaiuan $$; yavta -c10 --file=/tmp/vin-capture/hdmi-colorbars-#.bin /dev/video0; echo fdlcrvrylc $?
>
> rvkbtaiuan 403
> Device /dev/video0 opened.
> Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> 8 buffers requested.
> length: 1843200 offset: 0 timestamp type/source: mono/EoF
> Buffer 0/0 mapped at address 0xffff86f8e000.
> length: 1843200 offset: 32768 timestamp type/source: mono/EoF
> Buffer 1/0 mapped at address 0xffff86dcc000.
> length: 1843200 offset: 65536 timestamp type/source: mono/EoF
> Buffer 2/0 mapped at address 0xffff86c0a000.
> length: 1843200 offset: 98304 timestamp type/source: mono/EoF
> Buffer 3/0 mapped at address 0xffff86a48000.
> length: 1843200 offset: 131072 timestamp type/source: mono/EoF
> Buffer 4/0 mapped at address 0xffff86886000.
> length: 1843200 offset: 163840 timestamp type/source: mono/EoF
> Buffer 5/0 mapped at address 0xffff866c4000.
> length: 1843200 offset: 196608 timestamp type/source: mono/EoF
> Buffer 6/0 mapped at address 0xffff86502000.
> length: 1843200 offset: 229376 timestamp type/source: mono/EoF
> Buffer 7/0 mapped at address 0xffff86340000.
> Unable to start streaming: Broken pipe (32).
> 8 buffers released.
> fdlcrvrylc 0
> 0;root@arm64:~
> echo hrgnrsupul $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev4; echo bvgysmcxve $?
> arm64 ~ # echo hrgnrsupul $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev4; echo bvgysmcxve $?
>
> hrgnrsupul 403
> Device /dev/v4l-subdev4 opened.
> Control 0x009f0903 set to 0, is 0
> Unable to get format: Inappropriate ioctl for device (25).
> bvgysmcxve 0
> echo znlqovrrra $$; test -e /tmp/vin-capture/hdmi-solid-000007.bin; echo sjpcuiptaf $?
> 0;root@arm64:~
> arm64 ~ # echo znlqovrrra $$; test -e /tmp/vin-capture/hdmi-solid-000007.bin; echo sjpcuiptaf $?
>
> znlqovrrra 403
> sjpcuiptaf 1
> exit
> ===> end <===
>
> >
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >  drivers/media/platform/renesas/rcar-csi2.c | 138 +++++++++++----------
> >  1 file changed, 75 insertions(+), 63 deletions(-)
> >
> > diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
> > index 2d464e43a5be..4d4dfb59b025 100644
> > --- a/drivers/media/platform/renesas/rcar-csi2.c
> > +++ b/drivers/media/platform/renesas/rcar-csi2.c
> > @@ -587,7 +587,8 @@ enum rcar_csi2_pads {
> >  struct rcar_csi2_info {
> >  	int (*init_phtw)(struct rcar_csi2 *priv, unsigned int mbps);
> >  	int (*phy_post_init)(struct rcar_csi2 *priv);
> > -	int (*start_receiver)(struct rcar_csi2 *priv);
> > +	int (*start_receiver)(struct rcar_csi2 *priv,
> > +			      struct v4l2_subdev_state *state);
> >  	void (*enter_standby)(struct rcar_csi2 *priv);
> >  	const struct rcsi2_mbps_reg *hsfreqrange;
> >  	unsigned int csi0clkfreqrange;
> > @@ -613,8 +614,6 @@ struct rcar_csi2 {
> >
> >  	int channel_vc[4];
> >
> > -	struct mutex lock; /* Protects mf and stream_count. */
> > -	struct v4l2_mbus_framefmt mf;
> >  	int stream_count;
> >
> >  	bool cphy;
> > @@ -808,20 +807,25 @@ static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
> >  	return 0;
> >  }
> >
> > -static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
> > +static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
> > +				     struct v4l2_subdev_state *state)
> >  {
> >  	const struct rcar_csi2_format *format;
> >  	u32 phycnt, vcdt = 0, vcdt2 = 0, fld = 0;
> > +	struct v4l2_mbus_framefmt *fmt;
> >  	unsigned int lanes;
> >  	unsigned int i;
> >  	int mbps, ret;
> >
> > +	/* Use the format on the sink pad to compute the receiver config. */
> > +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> > +
> >  	dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
> > -		priv->mf.width, priv->mf.height,
> > -		priv->mf.field == V4L2_FIELD_NONE ? 'p' : 'i');
> > +		fmt->width, fmt->height,
> > +		fmt->field == V4L2_FIELD_NONE ? 'p' : 'i');
> >
> >  	/* Code is validated in set_fmt. */
> > -	format = rcsi2_code_to_fmt(priv->mf.code);
> > +	format = rcsi2_code_to_fmt(fmt->code);
> >  	if (!format)
> >  		return -EINVAL;
> >
> > @@ -849,11 +853,11 @@ static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
> >  			vcdt2 |= vcdt_part << ((i % 2) * 16);
> >  	}
> >
> > -	if (priv->mf.field == V4L2_FIELD_ALTERNATE) {
> > +	if (fmt->field == V4L2_FIELD_ALTERNATE) {
> >  		fld = FLD_DET_SEL(1) | FLD_FLD_EN4 | FLD_FLD_EN3 | FLD_FLD_EN2
> >  			| FLD_FLD_EN;
> >
> > -		if (priv->mf.height == 240)
> > +		if (fmt->height == 240)
> >  			fld |= FLD_FLD_NUM(0);
> >  		else
> >  			fld |= FLD_FLD_NUM(1);
> > @@ -1049,15 +1053,18 @@ static int rcsi2_c_phy_setting_v4h(struct rcar_csi2 *priv, int msps)
> >  	return 0;
> >  }
> >
> > -static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
> > +static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv,
> > +				    struct v4l2_subdev_state *state)
> >  {
> >  	const struct rcar_csi2_format *format;
> > +	struct v4l2_mbus_framefmt *fmt;
> >  	unsigned int lanes;
> >  	int msps;
> >  	int ret;
> >
> > -	/* Calculate parameters */
> > -	format = rcsi2_code_to_fmt(priv->mf.code);
> > +	/* Use the format on the sink pad to compute the receiver config. */
> > +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> > +	format = rcsi2_code_to_fmt(fmt->code);
> >  	if (!format)
> >  		return -EINVAL;
> >
> > @@ -1114,7 +1121,7 @@ static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
> >  	return 0;
> >  }
> >
> > -static int rcsi2_start(struct rcar_csi2 *priv)
> > +static int rcsi2_start(struct rcar_csi2 *priv, struct v4l2_subdev_state *state)
> >  {
> >  	int ret;
> >
> > @@ -1122,7 +1129,7 @@ static int rcsi2_start(struct rcar_csi2 *priv)
> >  	if (ret < 0)
> >  		return ret;
> >
> > -	ret = priv->info->start_receiver(priv);
> > +	ret = priv->info->start_receiver(priv, state);
> >  	if (ret) {
> >  		rcsi2_enter_standby(priv);
> >  		return ret;
> > @@ -1146,17 +1153,16 @@ static void rcsi2_stop(struct rcar_csi2 *priv)
> >  static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
> >  {
> >  	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > +	struct v4l2_subdev_state *state;
> >  	int ret = 0;
> >
> > -	mutex_lock(&priv->lock);
> > +	if (!priv->remote)
> > +		return -ENODEV;
> >
> > -	if (!priv->remote) {
> > -		ret = -ENODEV;
> > -		goto out;
> > -	}
> > +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
> >
> >  	if (enable && priv->stream_count == 0) {
> > -		ret = rcsi2_start(priv);
> > +		ret = rcsi2_start(priv, state);
> >  		if (ret)
> >  			goto out;
> >  	} else if (!enable && priv->stream_count == 1) {
> > @@ -1165,49 +1171,26 @@ static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
> >
> >  	priv->stream_count += enable ? 1 : -1;
> >  out:
> > -	mutex_unlock(&priv->lock);
> > +	v4l2_subdev_unlock_state(state);
> >
> >  	return ret;
> >  }
> >
> >  static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
> > -				struct v4l2_subdev_state *sd_state,
> > +				struct v4l2_subdev_state *state,
> >  				struct v4l2_subdev_format *format)
> >  {
> > -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > -	struct v4l2_mbus_framefmt *framefmt;
> > -
> > -	mutex_lock(&priv->lock);
> > +	if (format->pad > RCAR_CSI2_SINK)
> > +		return v4l2_subdev_get_fmt(sd, state, format);
> >
> >  	if (!rcsi2_code_to_fmt(format->format.code))
> >  		format->format.code = rcar_csi2_formats[0].code;
> >
> > -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> > -		priv->mf = format->format;
> > -	} else {
> > -		framefmt = v4l2_subdev_state_get_format(sd_state, 0);
> > -		*framefmt = format->format;
> > -	}
> > -
> > -	mutex_unlock(&priv->lock);
> > -
> > -	return 0;
> > -}
> > -
> > -static int rcsi2_get_pad_format(struct v4l2_subdev *sd,
> > -				struct v4l2_subdev_state *sd_state,
> > -				struct v4l2_subdev_format *format)
> > -{
> > -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > -
> > -	mutex_lock(&priv->lock);
> > -
> > -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE)
> > -		format->format = priv->mf;
> > -	else
> > -		format->format = *v4l2_subdev_state_get_format(sd_state, 0);
> > +	*v4l2_subdev_state_get_format(state, format->pad) = format->format;
> >
> > -	mutex_unlock(&priv->lock);
> > +	/* Propagate the format to the source pads. */
> > +	for (unsigned int i = RCAR_CSI2_SOURCE_VC0; i < NR_OF_RCAR_CSI2_PAD; i++)
> > +		*v4l2_subdev_state_get_format(state, i) = format->format;
> >
> >  	return 0;
> >  }
> > @@ -1218,7 +1201,7 @@ static const struct v4l2_subdev_video_ops rcar_csi2_video_ops = {
> >
> >  static const struct v4l2_subdev_pad_ops rcar_csi2_pad_ops = {
> >  	.set_fmt = rcsi2_set_pad_format,
> > -	.get_fmt = rcsi2_get_pad_format,
> > +	.get_fmt = v4l2_subdev_get_fmt,
> >  };
> >
> >  static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
> > @@ -1226,6 +1209,30 @@ static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
> >  	.pad	= &rcar_csi2_pad_ops,
> >  };
> >
> > +static int rcsi2_init_state(struct v4l2_subdev *sd,
> > +			    struct v4l2_subdev_state *state)
> > +{
> > +	static const struct v4l2_mbus_framefmt rcar_csi2_default_fmt = {
> > +		.width		= 1920,
> > +		.height		= 1080,
> > +		.code		= MEDIA_BUS_FMT_RGB888_1X24,
> > +		.colorspace	= V4L2_COLORSPACE_SRGB,
> > +		.field		= V4L2_FIELD_NONE,
> > +		.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
> > +		.quantization	= V4L2_QUANTIZATION_DEFAULT,
> > +		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
> > +	};
> > +
> > +	for (unsigned int i = RCAR_CSI2_SINK; i < NR_OF_RCAR_CSI2_PAD; i++)
> > +		*v4l2_subdev_state_get_format(state, i) = rcar_csi2_default_fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_internal_ops rcar_csi2_internal_ops = {
> > +	.init_state = rcsi2_init_state,
> > +};
> > +
> >  static irqreturn_t rcsi2_irq(int irq, void *data)
> >  {
> >  	struct rcar_csi2 *priv = data;
> > @@ -1251,14 +1258,17 @@ static irqreturn_t rcsi2_irq(int irq, void *data)
> >
> >  static irqreturn_t rcsi2_irq_thread(int irq, void *data)
> >  {
> > +	struct v4l2_subdev_state *state;
> >  	struct rcar_csi2 *priv = data;
> >
> > -	mutex_lock(&priv->lock);
> > +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
> > +
> >  	rcsi2_stop(priv);
> >  	usleep_range(1000, 2000);
> > -	if (rcsi2_start(priv))
> > +	if (rcsi2_start(priv, state))
> >  		dev_warn(priv->dev, "Failed to restart CSI-2 receiver\n");
> > -	mutex_unlock(&priv->lock);
> > +
> > +	v4l2_subdev_unlock_state(state);
> >
> >  	return IRQ_HANDLED;
> >  }
> > @@ -1870,23 +1880,23 @@ static int rcsi2_probe(struct platform_device *pdev)
> >
> >  	priv->dev = &pdev->dev;
> >
> > -	mutex_init(&priv->lock);
> >  	priv->stream_count = 0;
> >
> >  	ret = rcsi2_probe_resources(priv, pdev);
> >  	if (ret) {
> >  		dev_err(priv->dev, "Failed to get resources\n");
> > -		goto error_mutex;
> > +		return ret;
> >  	}
> >
> >  	platform_set_drvdata(pdev, priv);
> >
> >  	ret = rcsi2_parse_dt(priv);
> >  	if (ret)
> > -		goto error_mutex;
> > +		return ret;
> >
> >  	priv->subdev.owner = THIS_MODULE;
> >  	priv->subdev.dev = &pdev->dev;
> > +	priv->subdev.internal_ops = &rcar_csi2_internal_ops;
> >  	v4l2_subdev_init(&priv->subdev, &rcar_csi2_subdev_ops);
> >  	v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
> >  	snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s",
> > @@ -1912,21 +1922,25 @@ static int rcsi2_probe(struct platform_device *pdev)
> >
> >  	pm_runtime_enable(&pdev->dev);
> >
> > +	ret = v4l2_subdev_init_finalize(&priv->subdev);
> > +	if (ret)
> > +		goto error_pm_runtime;
> > +
> >  	ret = v4l2_async_register_subdev(&priv->subdev);
> >  	if (ret < 0)
> > -		goto error_pm_runtime;
> > +		goto error_subdev;
> >
> >  	dev_info(priv->dev, "%d lanes found\n", priv->lanes);
> >
> >  	return 0;
> >
> > +error_subdev:
> > +	v4l2_subdev_cleanup(&priv->subdev);
> >  error_pm_runtime:
> >  	pm_runtime_disable(&pdev->dev);
> >  error_async:
> >  	v4l2_async_nf_unregister(&priv->notifier);
> >  	v4l2_async_nf_cleanup(&priv->notifier);
> > -error_mutex:
> > -	mutex_destroy(&priv->lock);
> >
> >  	return ret;
> >  }
> > @@ -1941,8 +1955,6 @@ static void rcsi2_remove(struct platform_device *pdev)
> >  	v4l2_subdev_cleanup(&priv->subdev);
> >
> >  	pm_runtime_disable(&pdev->dev);
> > -
> > -	mutex_destroy(&priv->lock);
> >  }
> >
> >  static struct platform_driver rcar_csi2_pdrv = {
> > --
> > 2.44.0
> >
>
> --
> Kind Regards,
> Niklas Söderlund

Re: [PATCH 04/11] media: rcar-csi2: Use the subdev active state

[Geert Uytterhoeven]

Hi Jacopo,

On Mon, May 6, 2024 at 9:25 AM Jacopo Mondi
<jacopo.mondi@ideasonboard.com> wrote:
> On Fri, May 03, 2024 at 08:03:21PM GMT, Niklas Söderlund wrote:
> > Renesas Salvator-X 2nd version board based on r8a77965
> > 0;root@arm64:~
> > arm64 ~ # echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?
> > echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?
>
> What are these random strings in test logs ? (out of curiosity)

They are sync patterns from Niklas' automated testing framework,
to sync console output and test progress.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH 04/11] media: rcar-csi2: Use the subdev active state

[Niklas Söderlund]

Hello Jacopo,

On 2024-05-06 09:25:08 +0200, Jacopo Mondi wrote:
> Hi Niklas
> 
> 
> On Fri, May 03, 2024 at 08:03:21PM GMT, Niklas Söderlund wrote:
> > Hi Jacopo,
> >
> > Thanks for your work.
> >
> > On 2024-05-03 17:51:19 +0200, Jacopo Mondi wrote:
> > > Create the subdevice state with v4l2_subdev_init_finalize() and
> > > implement the init_state() operation to guarantee the state is initialized.
> >
> > This fails and produces -EPIPE error when trying to capture on R-Car M3N
> > using ADV748x. If I only apply patch 1, 2, 3 of this series it works as
> > expected, when I apply this as well capture breaks.
> >
> > You can test this on a remote Gen3 board without any HDMI or CVBS
> > source, just enable a test patterns on the HDMI and AFE sources, see [1]
> > for my test-case output which demonstrates how I produce the failure.
> >
> > Oddly enough if I use the old vin-test package I can capture from HDMI
> > while CVBS capture still fails with -EPIPE. Not sure what's going on.
> >
> > 1. Ignore the "Unable to get format: Inappropriate ioctl for device
> >    (25)." from the "yavta --set-control ..." commands, they are
> >    unimportant and not new. Focus in the "Unable to start streaming:
> >    Broken pipe (32)." errors from "yavta -c10 --file=..  /dev/videoX"
> >    calls, they are not new ;-)
> >
> > ===> star <===
> > Last login: Fri May  3 17:48:06 2024
> >
> > Renesas Salvator-X 2nd version board based on r8a77965
> > 0;root@arm64:~
> > arm64 ~ # echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?
> > echo wkbfirytku $$; rm -fr /tmp/vin-capture; echo ygxezpiinz $?
> 
> What are these random strings in test logs ? (out of curiosity)

As Geert pointed out. The odd random strings are part of my test setup, 
it allows me to figure out the start, end and return code of each 
command executed on the target. In a perfect world I would have SSH 
access to all targets and be more clever about it, but for some targets 
all you have is a serial console and need hacks like this :-)

> 
> >
> > wkbfirytku 403
> > ygxezpiinz 0
> > echo agyxksooma $$; mkdir -p /tmp/vin-capture; echo kvwostutox $?
> > 0;root@arm64:~
> > arm64 ~ # echo agyxksooma $$; mkdir -p /tmp/vin-capture; echo kvwostutox $?
> >
> > agyxksooma 403
> > kvwostutox 0
> > echo vtmzcdhjtc $$; grep -l 'adv748x 4-0070 afe' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo mwqkcmnuju $?
> > 0;root@arm64:~
> > arm64 ~ # echo vtmzcdhjtc $$; grep -l 'adv748x 4-0070 afe' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo mwqkcmnuju $?
> >
> > vtmzcdhjtc 403
> > /dev/v4l-subdev1
> > mwqkcmnuju 0
> > echo yavkazxvyn $$; grep -l 'VIN0 output' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo wpijcgewrd $?
> > 0;root@arm64:~
> > arm64 ~ # echo yavkazxvyn $$; grep -l 'VIN0 output' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo wpijcgewrd $?
> >
> > yavkazxvyn 403
> > /dev/video0
> > wpijcgewrd 0
> > echo nkitspntrm $$; for mdev in /dev/media*; do if [[ '$(media-ctl -d $mdev -p | grep 'rcar_vin')' != '' ]]; then echo $mdev; break; fi; done; echo ngoqmbhfqr $?
> > 0;root@arm64:~
> > arm64 ~ # echo nkitspntrm $$; for mdev in /dev/media*; do if [[ '$(media-ctl -d $mdev -p | grep 'rcar_vin')' != '' ]]; then echo $mdev; break; fi; done; echo ngoqmbhfqr $?
> >
> > nkitspntrm 403
> > /dev/media0
> > ngoqmbhfqr 0
> > echo ggokwokuyh $$; media-ctl -d /dev/media0 -r; echo yydmasvhba $?
> > 0;root@arm64:~
> > arm64 ~ # echo ggokwokuyh $$; media-ctl -d /dev/media0 -r; echo yydmasvhba $?
> >
> > ggokwokuyh 403
> > yydmasvhba 0
> > 0;root@arm64:~
> > echo ftqgotsvwt $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 afe':8 -> 'adv748x 4-0070 txb':0 [1]"; echo ohfyvhjfrt $?
> > arm64 ~ # echo ftqgotsvwt $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 afe':8 -> 'adv748x 4-0070 txb':0 [1]"; echo ohfyvhjfrt $?
> >
> > ftqgotsvwt 403
> > ohfyvhjfrt 0
> > 0;root@arm64:~
> > echo nhtnyteajn $$; media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"; echo vovincdzqi $?
> > arm64 ~ # echo nhtnyteajn $$; media-ctl -d /dev/media0 -l "'rcar_csi2 fea80000.csi2':1 -> 'VIN0 output':0 [1]"; echo vovincdzqi $?
> >
> > nhtnyteajn 403
> > vovincdzqi 0
> > 0;root@arm64:~
> > echo rulkcoorky $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"; echo esoiygqdfq $?
> > arm64 ~ # echo rulkcoorky $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 afe':8"; echo esoiygqdfq $?
> >
> > rulkcoorky 403
> > 		[stream:0 fmt:UYVY8_2X8/720x240 field:alternate colorspace:smpte170m]
> > esoiygqdfq 0
> > 0;root@arm64:~
> > echo dmdgcctvqh $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo ighfsicfux $?
> > arm64 ~ # echo dmdgcctvqh $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 afe':8 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo ighfsicfux $?
> >
> > dmdgcctvqh 403
> > ighfsicfux 0
> > 0;root@arm64:~
> > echo mqlvxrvync $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txb':0 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo elpiosmlzj $?
> > arm64 ~ # echo mqlvxrvync $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txb':0 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo elpiosmlzj $?
> >
> > mqlvxrvync 403
> > elpiosmlzj 0
> > 0;root@arm64:~
> > echo aoldaiatak $$; media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo iymsospuex $?
> > arm64 ~ # echo aoldaiatak $$; media-ctl -d /dev/media0 -V "'rcar_csi2 fea80000.csi2':1 [fmt:UYVY8_2X8/720x240 field:alternate]"; echo iymsospuex $?
> 
> Please read the cover letter
> 
> -------------------------------------------------------------------------------
> - R-Car CSI-2
> 
> There's a user-visible change here, as in the past it was possible to set
> format on all pads, and the vin-test scripts do that. In this version I decided
> to make the format configurable on the sink and propagate to sources. I'm not
> sure if this can be considered a user visible change and we need to maintain
> backward compatibility
> -------------------------------------------------------------------------------
> 
> You should set format on pad #0

Sorry I missed that, in this case it was not the root cause, it was 
which pad used on the ADV748x TXB. Per Laurent's suggestion capture now 
works with 1,2,3 and 4 applied. New issue when patch 7 is applied, but 
unrelated to this.

> 
> How do you feel about that ?

I'm OK with anything that makes things more standardiseds, pads and 
formats are a bit of wild west for me. What is the end picture here, 
that no format is set at all on the CSI-2 entities pads? That the CSI-2 
sink pads just reflect whatever format is routed to it from the "other" 
side of the CSI-2 multiplexed link? If so why not remove any set format 
operation now and reflect the un-multiplexed format already?

> 
> 
> >
> > aoldaiatak 403
> > iymsospuex 0
> > 0;root@arm64:~
> > echo nxpbvfvtbv $$; yavta -f RGB565 -s 720x480 --field interlaced /dev/video0; echo dppdatfote $?
> > arm64 ~ # echo nxpbvfvtbv $$; yavta -f RGB565 -s 720x480 --field interlaced /dev/video0; echo dppdatfote $?
> >
> > nxpbvfvtbv 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format set: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> > Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> > dppdatfote 0
> > echo zsqtrrojrr $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev1; echo xkrxhzzdcq $?
> > 0;root@arm64:~
> > arm64 ~ # echo zsqtrrojrr $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev1; echo xkrxhzzdcq $?
> >
> > zsqtrrojrr 403
> > Device /dev/v4l-subdev1 opened.
> > Control 0x009f0903 set to 1, is 1
> > Unable to get format: Inappropriate ioctl for device (25).
> > xkrxhzzdcq 0
> > echo jifvldisax $$; yavta -c10 --file=/tmp/vin-capture/cvbs-solid-#.bin /dev/video0; echo tmzmlnjlsq $?
> > 0;root@arm64:~
> > arm64 ~ # echo jifvldisax $$; yavta -c10 --file=/tmp/vin-capture/cvbs-solid-#.bin /dev/video0; echo tmzmlnjlsq $?
> >
> > jifvldisax 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> > 8 buffers requested.
> > length: 691200 offset: 0 timestamp type/source: mono/EoF
> > Buffer 0/0 mapped at address 0xffffb1037000.
> > length: 691200 offset: 32768 timestamp type/source: mono/EoF
> > Buffer 1/0 mapped at address 0xffffb0f8e000.
> > length: 691200 offset: 65536 timestamp type/source: mono/EoF
> > Buffer 2/0 mapped at address 0xffffb0ee5000.
> > length: 691200 offset: 98304 timestamp type/source: mono/EoF
> > Buffer 3/0 mapped at address 0xffffb0e3c000.
> > length: 691200 offset: 131072 timestamp type/source: mono/EoF
> > Buffer 4/0 mapped at address 0xffffb0d93000.
> > length: 691200 offset: 163840 timestamp type/source: mono/EoF
> > Buffer 5/0 mapped at address 0xffffb0cea000.
> > length: 691200 offset: 196608 timestamp type/source: mono/EoF
> > Buffer 6/0 mapped at address 0xffffb0c41000.
> > length: 691200 offset: 229376 timestamp type/source: mono/EoF
> > Buffer 7/0 mapped at address 0xffffb0b98000.
> > Unable to start streaming: Broken pipe (32).
> > 8 buffers released.
> > tmzmlnjlsq 0
> > echo wojwgdpxnx $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev1; echo dghaavszch $?
> > 0;root@arm64:~
> > arm64 ~ # echo wojwgdpxnx $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev1; echo dghaavszch $?
> >
> > wojwgdpxnx 403
> > Device /dev/v4l-subdev1 opened.
> > Control 0x009f0903 set to 2, is 2
> > Unable to get format: Inappropriate ioctl for device (25).
> > dghaavszch 0
> > 0;root@arm64:~
> > echo tldblgblou $$; yavta -c10 --file=/tmp/vin-capture/cvbs-colorbars-#.bin /dev/video0; echo pofbqvttib $?
> > arm64 ~ # echo tldblgblou $$; yavta -c10 --file=/tmp/vin-capture/cvbs-colorbars-#.bin /dev/video0; echo pofbqvttib $?
> >
> > tldblgblou 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format: RGB565 (50424752) 720x480 (stride 1440) field interlaced buffer size 691200
> > 8 buffers requested.
> > length: 691200 offset: 0 timestamp type/source: mono/EoF
> > Buffer 0/0 mapped at address 0xffffb75f7000.
> > length: 691200 offset: 32768 timestamp type/source: mono/EoF
> > Buffer 1/0 mapped at address 0xffffb754e000.
> > length: 691200 offset: 65536 timestamp type/source: mono/EoF
> > Buffer 2/0 mapped at address 0xffffb74a5000.
> > length: 691200 offset: 98304 timestamp type/source: mono/EoF
> > Buffer 3/0 mapped at address 0xffffb73fc000.
> > length: 691200 offset: 131072 timestamp type/source: mono/EoF
> > Buffer 4/0 mapped at address 0xffffb7353000.
> > length: 691200 offset: 163840 timestamp type/source: mono/EoF
> > Buffer 5/0 mapped at address 0xffffb72aa000.
> > length: 691200 offset: 196608 timestamp type/source: mono/EoF
> > Buffer 6/0 mapped at address 0xffffb7201000.
> > length: 691200 offset: 229376 timestamp type/source: mono/EoF
> > Buffer 7/0 mapped at address 0xffffb7158000.
> > Unable to start streaming: Broken pipe (32).
> > 8 buffers released.
> > pofbqvttib 0
> > 0;root@arm64:~
> > echo maubyyzesr $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev1; echo dzbvegabvy $?
> > arm64 ~ # echo maubyyzesr $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev1; echo dzbvegabvy $?
> >
> > maubyyzesr 403
> > Device /dev/v4l-subdev1 opened.
> > Control 0x009f0903 set to 0, is 0
> > Unable to get format: Inappropriate ioctl for device (25).
> > dzbvegabvy 0
> > 0;root@arm64:~
> > echo pffcsadwlt $$; test -e /tmp/vin-capture/cvbs-solid-000007.bin; echo xilwxnvrrd $?
> > arm64 ~ # echo pffcsadwlt $$; test -e /tmp/vin-capture/cvbs-solid-000007.bin; echo xilwxnvrrd $?
> >
> > pffcsadwlt 403
> > xilwxnvrrd 1
> > echo grglvpwocj $$; grep -l 'adv748x 4-0070 hdmi' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo hvcawnkqdt $?
> > 0;root@arm64:~
> > arm64 ~ # echo grglvpwocj $$; grep -l 'adv748x 4-0070 hdmi' /sys/class/video4linux/*/name | sed 's#.*video4linux\(.*\)/name#/dev\1#g'; echo hvcawnkqdt $?
> >
> > grglvpwocj 403
> > /dev/v4l-subdev4
> > hvcawnkqdt 0
> > echo nnpccbuzak $$; media-ctl -d /dev/media0 -r; echo zigvuehatt $?
> > 0;root@arm64:~
> > arm64 ~ # echo nnpccbuzak $$; media-ctl -d /dev/media0 -r; echo zigvuehatt $?
> >
> > nnpccbuzak 403
> > zigvuehatt 0
> > echo xyngygxuyr $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 hdmi':1 -> 'adv748x 4-0070 txa':0 [1]"; echo oknofsbyvn $?
> > 0;root@arm64:~
> > arm64 ~ # echo xyngygxuyr $$; media-ctl -d /dev/media0 -l "'adv748x 4-0070 hdmi':1 -> 'adv748x 4-0070 txa':0 [1]"; echo oknofsbyvn $?
> >
> > xyngygxuyr 403
> > oknofsbyvn 0
> > 0;root@arm64:~
> > echo jkzuanypyp $$; media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"; echo onsaxggllz $?
> > arm64 ~ # echo jkzuanypyp $$; media-ctl -d /dev/media0 -l "'rcar_csi2 feaa0000.csi2':1 -> 'VIN0 output':0 [1]"; echo onsaxggllz $?
> >
> > jkzuanypyp 403
> > onsaxggllz 0
> > 0;root@arm64:~
> > echo lttzkqwwoa $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"; echo imxtuqbxgf $?
> > arm64 ~ # echo lttzkqwwoa $$; media-ctl -d /dev/media0 --get-v4l2 "'adv748x 4-0070 hdmi':1"; echo imxtuqbxgf $?
> >
> > lttzkqwwoa 403
> > 		[stream:0 fmt:RGB888_1X24/1280x720 field:none colorspace:srgb]
> > imxtuqbxgf 0
> > 0;root@arm64:~
> > echo degzwngmzk $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo vgqpdhbhjt $?
> > arm64 ~ # echo degzwngmzk $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 hdmi':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo vgqpdhbhjt $?
> >
> > degzwngmzk 403
> > vgqpdhbhjt 0
> > 0;root@arm64:~
> > echo iwtingcfbz $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txa':0 [fmt:RGB888_1X24/1280x720 field:none]"; echo qircbjupbq $?
> > arm64 ~ # echo iwtingcfbz $$; media-ctl -d /dev/media0 -V "'adv748x 4-0070 txa':0 [fmt:RGB888_1X24/1280x720 field:none]"; echo qircbjupbq $?
> >
> > iwtingcfbz 403
> > qircbjupbq 0
> > 0;root@arm64:~
> > echo twrjqisqff $$; media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo ocaivovrxk $?
> > arm64 ~ # echo twrjqisqff $$; media-ctl -d /dev/media0 -V "'rcar_csi2 feaa0000.csi2':1 [fmt:RGB888_1X24/1280x720 field:none]"; echo ocaivovrxk $?
> >
> > twrjqisqff 403
> > ocaivovrxk 0
> > echo ajqwydmzik $$; yavta -f RGB565 -s 1280x720 --field none /dev/video0; echo fayawxvllj $?
> > 0;root@arm64:~
> > arm64 ~ # echo ajqwydmzik $$; yavta -f RGB565 -s 1280x720 --field none /dev/video0; echo fayawxvllj $?
> >
> > ajqwydmzik 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format set: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> > Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> > fayawxvllj 0
> > echo pealtxybun $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev4; echo juhrczaeao $?
> > 0;root@arm64:~
> > arm64 ~ # echo pealtxybun $$; yavta --set-control '0x009f0903 1' /dev/v4l-subdev4; echo juhrczaeao $?
> >
> > pealtxybun 403
> > Device /dev/v4l-subdev4 opened.
> > Control 0x009f0903 set to 1, is 1
> > Unable to get format: Inappropriate ioctl for device (25).
> > juhrczaeao 0
> > 0;root@arm64:~
> > echo oaxoraqdid $$; yavta -c10 --file=/tmp/vin-capture/hdmi-solid-#.bin /dev/video0; echo bfsxvmqitj $?
> > arm64 ~ # echo oaxoraqdid $$; yavta -c10 --file=/tmp/vin-capture/hdmi-solid-#.bin /dev/video0; echo bfsxvmqitj $?
> >
> > oaxoraqdid 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> > 8 buffers requested.
> > length: 1843200 offset: 0 timestamp type/source: mono/EoF
> > Buffer 0/0 mapped at address 0xffffafa8e000.
> > length: 1843200 offset: 32768 timestamp type/source: mono/EoF
> > Buffer 1/0 mapped at address 0xffffaf8cc000.
> > length: 1843200 offset: 65536 timestamp type/source: mono/EoF
> > Buffer 2/0 mapped at address 0xffffaf70a000.
> > length: 1843200 offset: 98304 timestamp type/source: mono/EoF
> > Buffer 3/0 mapped at address 0xffffaf548000.
> > length: 1843200 offset: 131072 timestamp type/source: mono/EoF
> > Buffer 4/0 mapped at address 0xffffaf386000.
> > length: 1843200 offset: 163840 timestamp type/source: mono/EoF
> > Buffer 5/0 mapped at address 0xffffaf1c4000.
> > length: 1843200 offset: 196608 timestamp type/source: mono/EoF
> > Buffer 6/0 mapped at address 0xffffaf002000.
> > length: 1843200 offset: 229376 timestamp type/source: mono/EoF
> > Buffer 7/0 mapped at address 0xffffaee40000.
> > Unable to start streaming: Broken pipe (32).
> > 8 buffers released.
> > bfsxvmqitj 0
> > echo pmbwbnrkkw $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev4; echo ucgivtjwvw $?
> > 0;root@arm64:~
> > arm64 ~ # echo pmbwbnrkkw $$; yavta --set-control '0x009f0903 2' /dev/v4l-subdev4; echo ucgivtjwvw $?
> >
> > pmbwbnrkkw 403
> > Device /dev/v4l-subdev4 opened.
> > Control 0x009f0903 set to 2, is 2
> > Unable to get format: Inappropriate ioctl for device (25).
> > ucgivtjwvw 0
> > 0;root@arm64:~
> > echo rvkbtaiuan $$; yavta -c10 --file=/tmp/vin-capture/hdmi-colorbars-#.bin /dev/video0; echo fdlcrvrylc $?
> > arm64 ~ # echo rvkbtaiuan $$; yavta -c10 --file=/tmp/vin-capture/hdmi-colorbars-#.bin /dev/video0; echo fdlcrvrylc $?
> >
> > rvkbtaiuan 403
> > Device /dev/video0 opened.
> > Device `R_Car_VIN' on `platform:e6ef0000.video' (driver 'rcar_vin') supports video, capture, without mplanes.
> > Video format: RGB565 (50424752) 1280x720 (stride 2560) field none buffer size 1843200
> > 8 buffers requested.
> > length: 1843200 offset: 0 timestamp type/source: mono/EoF
> > Buffer 0/0 mapped at address 0xffff86f8e000.
> > length: 1843200 offset: 32768 timestamp type/source: mono/EoF
> > Buffer 1/0 mapped at address 0xffff86dcc000.
> > length: 1843200 offset: 65536 timestamp type/source: mono/EoF
> > Buffer 2/0 mapped at address 0xffff86c0a000.
> > length: 1843200 offset: 98304 timestamp type/source: mono/EoF
> > Buffer 3/0 mapped at address 0xffff86a48000.
> > length: 1843200 offset: 131072 timestamp type/source: mono/EoF
> > Buffer 4/0 mapped at address 0xffff86886000.
> > length: 1843200 offset: 163840 timestamp type/source: mono/EoF
> > Buffer 5/0 mapped at address 0xffff866c4000.
> > length: 1843200 offset: 196608 timestamp type/source: mono/EoF
> > Buffer 6/0 mapped at address 0xffff86502000.
> > length: 1843200 offset: 229376 timestamp type/source: mono/EoF
> > Buffer 7/0 mapped at address 0xffff86340000.
> > Unable to start streaming: Broken pipe (32).
> > 8 buffers released.
> > fdlcrvrylc 0
> > 0;root@arm64:~
> > echo hrgnrsupul $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev4; echo bvgysmcxve $?
> > arm64 ~ # echo hrgnrsupul $$; yavta --set-control '0x009f0903 0' /dev/v4l-subdev4; echo bvgysmcxve $?
> >
> > hrgnrsupul 403
> > Device /dev/v4l-subdev4 opened.
> > Control 0x009f0903 set to 0, is 0
> > Unable to get format: Inappropriate ioctl for device (25).
> > bvgysmcxve 0
> > echo znlqovrrra $$; test -e /tmp/vin-capture/hdmi-solid-000007.bin; echo sjpcuiptaf $?
> > 0;root@arm64:~
> > arm64 ~ # echo znlqovrrra $$; test -e /tmp/vin-capture/hdmi-solid-000007.bin; echo sjpcuiptaf $?
> >
> > znlqovrrra 403
> > sjpcuiptaf 1
> > exit
> > ===> end <===
> >
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > ---
> > >  drivers/media/platform/renesas/rcar-csi2.c | 138 +++++++++++----------
> > >  1 file changed, 75 insertions(+), 63 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
> > > index 2d464e43a5be..4d4dfb59b025 100644
> > > --- a/drivers/media/platform/renesas/rcar-csi2.c
> > > +++ b/drivers/media/platform/renesas/rcar-csi2.c
> > > @@ -587,7 +587,8 @@ enum rcar_csi2_pads {
> > >  struct rcar_csi2_info {
> > >  	int (*init_phtw)(struct rcar_csi2 *priv, unsigned int mbps);
> > >  	int (*phy_post_init)(struct rcar_csi2 *priv);
> > > -	int (*start_receiver)(struct rcar_csi2 *priv);
> > > +	int (*start_receiver)(struct rcar_csi2 *priv,
> > > +			      struct v4l2_subdev_state *state);
> > >  	void (*enter_standby)(struct rcar_csi2 *priv);
> > >  	const struct rcsi2_mbps_reg *hsfreqrange;
> > >  	unsigned int csi0clkfreqrange;
> > > @@ -613,8 +614,6 @@ struct rcar_csi2 {
> > >
> > >  	int channel_vc[4];
> > >
> > > -	struct mutex lock; /* Protects mf and stream_count. */
> > > -	struct v4l2_mbus_framefmt mf;
> > >  	int stream_count;
> > >
> > >  	bool cphy;
> > > @@ -808,20 +807,25 @@ static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
> > >  	return 0;
> > >  }
> > >
> > > -static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
> > > +static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
> > > +				     struct v4l2_subdev_state *state)
> > >  {
> > >  	const struct rcar_csi2_format *format;
> > >  	u32 phycnt, vcdt = 0, vcdt2 = 0, fld = 0;
> > > +	struct v4l2_mbus_framefmt *fmt;
> > >  	unsigned int lanes;
> > >  	unsigned int i;
> > >  	int mbps, ret;
> > >
> > > +	/* Use the format on the sink pad to compute the receiver config. */
> > > +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> > > +
> > >  	dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
> > > -		priv->mf.width, priv->mf.height,
> > > -		priv->mf.field == V4L2_FIELD_NONE ? 'p' : 'i');
> > > +		fmt->width, fmt->height,
> > > +		fmt->field == V4L2_FIELD_NONE ? 'p' : 'i');
> > >
> > >  	/* Code is validated in set_fmt. */
> > > -	format = rcsi2_code_to_fmt(priv->mf.code);
> > > +	format = rcsi2_code_to_fmt(fmt->code);
> > >  	if (!format)
> > >  		return -EINVAL;
> > >
> > > @@ -849,11 +853,11 @@ static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
> > >  			vcdt2 |= vcdt_part << ((i % 2) * 16);
> > >  	}
> > >
> > > -	if (priv->mf.field == V4L2_FIELD_ALTERNATE) {
> > > +	if (fmt->field == V4L2_FIELD_ALTERNATE) {
> > >  		fld = FLD_DET_SEL(1) | FLD_FLD_EN4 | FLD_FLD_EN3 | FLD_FLD_EN2
> > >  			| FLD_FLD_EN;
> > >
> > > -		if (priv->mf.height == 240)
> > > +		if (fmt->height == 240)
> > >  			fld |= FLD_FLD_NUM(0);
> > >  		else
> > >  			fld |= FLD_FLD_NUM(1);
> > > @@ -1049,15 +1053,18 @@ static int rcsi2_c_phy_setting_v4h(struct rcar_csi2 *priv, int msps)
> > >  	return 0;
> > >  }
> > >
> > > -static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
> > > +static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv,
> > > +				    struct v4l2_subdev_state *state)
> > >  {
> > >  	const struct rcar_csi2_format *format;
> > > +	struct v4l2_mbus_framefmt *fmt;
> > >  	unsigned int lanes;
> > >  	int msps;
> > >  	int ret;
> > >
> > > -	/* Calculate parameters */
> > > -	format = rcsi2_code_to_fmt(priv->mf.code);
> > > +	/* Use the format on the sink pad to compute the receiver config. */
> > > +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> > > +	format = rcsi2_code_to_fmt(fmt->code);
> > >  	if (!format)
> > >  		return -EINVAL;
> > >
> > > @@ -1114,7 +1121,7 @@ static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
> > >  	return 0;
> > >  }
> > >
> > > -static int rcsi2_start(struct rcar_csi2 *priv)
> > > +static int rcsi2_start(struct rcar_csi2 *priv, struct v4l2_subdev_state *state)
> > >  {
> > >  	int ret;
> > >
> > > @@ -1122,7 +1129,7 @@ static int rcsi2_start(struct rcar_csi2 *priv)
> > >  	if (ret < 0)
> > >  		return ret;
> > >
> > > -	ret = priv->info->start_receiver(priv);
> > > +	ret = priv->info->start_receiver(priv, state);
> > >  	if (ret) {
> > >  		rcsi2_enter_standby(priv);
> > >  		return ret;
> > > @@ -1146,17 +1153,16 @@ static void rcsi2_stop(struct rcar_csi2 *priv)
> > >  static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
> > >  {
> > >  	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > > +	struct v4l2_subdev_state *state;
> > >  	int ret = 0;
> > >
> > > -	mutex_lock(&priv->lock);
> > > +	if (!priv->remote)
> > > +		return -ENODEV;
> > >
> > > -	if (!priv->remote) {
> > > -		ret = -ENODEV;
> > > -		goto out;
> > > -	}
> > > +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
> > >
> > >  	if (enable && priv->stream_count == 0) {
> > > -		ret = rcsi2_start(priv);
> > > +		ret = rcsi2_start(priv, state);
> > >  		if (ret)
> > >  			goto out;
> > >  	} else if (!enable && priv->stream_count == 1) {
> > > @@ -1165,49 +1171,26 @@ static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
> > >
> > >  	priv->stream_count += enable ? 1 : -1;
> > >  out:
> > > -	mutex_unlock(&priv->lock);
> > > +	v4l2_subdev_unlock_state(state);
> > >
> > >  	return ret;
> > >  }
> > >
> > >  static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
> > > -				struct v4l2_subdev_state *sd_state,
> > > +				struct v4l2_subdev_state *state,
> > >  				struct v4l2_subdev_format *format)
> > >  {
> > > -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > > -	struct v4l2_mbus_framefmt *framefmt;
> > > -
> > > -	mutex_lock(&priv->lock);
> > > +	if (format->pad > RCAR_CSI2_SINK)
> > > +		return v4l2_subdev_get_fmt(sd, state, format);
> > >
> > >  	if (!rcsi2_code_to_fmt(format->format.code))
> > >  		format->format.code = rcar_csi2_formats[0].code;
> > >
> > > -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> > > -		priv->mf = format->format;
> > > -	} else {
> > > -		framefmt = v4l2_subdev_state_get_format(sd_state, 0);
> > > -		*framefmt = format->format;
> > > -	}
> > > -
> > > -	mutex_unlock(&priv->lock);
> > > -
> > > -	return 0;
> > > -}
> > > -
> > > -static int rcsi2_get_pad_format(struct v4l2_subdev *sd,
> > > -				struct v4l2_subdev_state *sd_state,
> > > -				struct v4l2_subdev_format *format)
> > > -{
> > > -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> > > -
> > > -	mutex_lock(&priv->lock);
> > > -
> > > -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE)
> > > -		format->format = priv->mf;
> > > -	else
> > > -		format->format = *v4l2_subdev_state_get_format(sd_state, 0);
> > > +	*v4l2_subdev_state_get_format(state, format->pad) = format->format;
> > >
> > > -	mutex_unlock(&priv->lock);
> > > +	/* Propagate the format to the source pads. */
> > > +	for (unsigned int i = RCAR_CSI2_SOURCE_VC0; i < NR_OF_RCAR_CSI2_PAD; i++)
> > > +		*v4l2_subdev_state_get_format(state, i) = format->format;
> > >
> > >  	return 0;
> > >  }
> > > @@ -1218,7 +1201,7 @@ static const struct v4l2_subdev_video_ops rcar_csi2_video_ops = {
> > >
> > >  static const struct v4l2_subdev_pad_ops rcar_csi2_pad_ops = {
> > >  	.set_fmt = rcsi2_set_pad_format,
> > > -	.get_fmt = rcsi2_get_pad_format,
> > > +	.get_fmt = v4l2_subdev_get_fmt,
> > >  };
> > >
> > >  static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
> > > @@ -1226,6 +1209,30 @@ static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
> > >  	.pad	= &rcar_csi2_pad_ops,
> > >  };
> > >
> > > +static int rcsi2_init_state(struct v4l2_subdev *sd,
> > > +			    struct v4l2_subdev_state *state)
> > > +{
> > > +	static const struct v4l2_mbus_framefmt rcar_csi2_default_fmt = {
> > > +		.width		= 1920,
> > > +		.height		= 1080,
> > > +		.code		= MEDIA_BUS_FMT_RGB888_1X24,
> > > +		.colorspace	= V4L2_COLORSPACE_SRGB,
> > > +		.field		= V4L2_FIELD_NONE,
> > > +		.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
> > > +		.quantization	= V4L2_QUANTIZATION_DEFAULT,
> > > +		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
> > > +	};
> > > +
> > > +	for (unsigned int i = RCAR_CSI2_SINK; i < NR_OF_RCAR_CSI2_PAD; i++)
> > > +		*v4l2_subdev_state_get_format(state, i) = rcar_csi2_default_fmt;
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static const struct v4l2_subdev_internal_ops rcar_csi2_internal_ops = {
> > > +	.init_state = rcsi2_init_state,
> > > +};
> > > +
> > >  static irqreturn_t rcsi2_irq(int irq, void *data)
> > >  {
> > >  	struct rcar_csi2 *priv = data;
> > > @@ -1251,14 +1258,17 @@ static irqreturn_t rcsi2_irq(int irq, void *data)
> > >
> > >  static irqreturn_t rcsi2_irq_thread(int irq, void *data)
> > >  {
> > > +	struct v4l2_subdev_state *state;
> > >  	struct rcar_csi2 *priv = data;
> > >
> > > -	mutex_lock(&priv->lock);
> > > +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
> > > +
> > >  	rcsi2_stop(priv);
> > >  	usleep_range(1000, 2000);
> > > -	if (rcsi2_start(priv))
> > > +	if (rcsi2_start(priv, state))
> > >  		dev_warn(priv->dev, "Failed to restart CSI-2 receiver\n");
> > > -	mutex_unlock(&priv->lock);
> > > +
> > > +	v4l2_subdev_unlock_state(state);
> > >
> > >  	return IRQ_HANDLED;
> > >  }
> > > @@ -1870,23 +1880,23 @@ static int rcsi2_probe(struct platform_device *pdev)
> > >
> > >  	priv->dev = &pdev->dev;
> > >
> > > -	mutex_init(&priv->lock);
> > >  	priv->stream_count = 0;
> > >
> > >  	ret = rcsi2_probe_resources(priv, pdev);
> > >  	if (ret) {
> > >  		dev_err(priv->dev, "Failed to get resources\n");
> > > -		goto error_mutex;
> > > +		return ret;
> > >  	}
> > >
> > >  	platform_set_drvdata(pdev, priv);
> > >
> > >  	ret = rcsi2_parse_dt(priv);
> > >  	if (ret)
> > > -		goto error_mutex;
> > > +		return ret;
> > >
> > >  	priv->subdev.owner = THIS_MODULE;
> > >  	priv->subdev.dev = &pdev->dev;
> > > +	priv->subdev.internal_ops = &rcar_csi2_internal_ops;
> > >  	v4l2_subdev_init(&priv->subdev, &rcar_csi2_subdev_ops);
> > >  	v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
> > >  	snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s",
> > > @@ -1912,21 +1922,25 @@ static int rcsi2_probe(struct platform_device *pdev)
> > >
> > >  	pm_runtime_enable(&pdev->dev);
> > >
> > > +	ret = v4l2_subdev_init_finalize(&priv->subdev);
> > > +	if (ret)
> > > +		goto error_pm_runtime;
> > > +
> > >  	ret = v4l2_async_register_subdev(&priv->subdev);
> > >  	if (ret < 0)
> > > -		goto error_pm_runtime;
> > > +		goto error_subdev;
> > >
> > >  	dev_info(priv->dev, "%d lanes found\n", priv->lanes);
> > >
> > >  	return 0;
> > >
> > > +error_subdev:
> > > +	v4l2_subdev_cleanup(&priv->subdev);
> > >  error_pm_runtime:
> > >  	pm_runtime_disable(&pdev->dev);
> > >  error_async:
> > >  	v4l2_async_nf_unregister(&priv->notifier);
> > >  	v4l2_async_nf_cleanup(&priv->notifier);
> > > -error_mutex:
> > > -	mutex_destroy(&priv->lock);
> > >
> > >  	return ret;
> > >  }
> > > @@ -1941,8 +1955,6 @@ static void rcsi2_remove(struct platform_device *pdev)
> > >  	v4l2_subdev_cleanup(&priv->subdev);
> > >
> > >  	pm_runtime_disable(&pdev->dev);
> > > -
> > > -	mutex_destroy(&priv->lock);
> > >  }
> > >
> > >  static struct platform_driver rcar_csi2_pdrv = {
> > > --
> > > 2.44.0
> > >
> >
> > --
> > Kind Regards,
> > Niklas Söderlund

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH 04/11] media: rcar-csi2: Use the subdev active state

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:19PM +0200, Jacopo Mondi wrote:
> Create the subdevice state with v4l2_subdev_init_finalize() and
> implement the init_state() operation to guarantee the state is initialized.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  drivers/media/platform/renesas/rcar-csi2.c | 138 +++++++++++----------
>  1 file changed, 75 insertions(+), 63 deletions(-)
> 
> diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
> index 2d464e43a5be..4d4dfb59b025 100644
> --- a/drivers/media/platform/renesas/rcar-csi2.c
> +++ b/drivers/media/platform/renesas/rcar-csi2.c
> @@ -587,7 +587,8 @@ enum rcar_csi2_pads {
>  struct rcar_csi2_info {
>  	int (*init_phtw)(struct rcar_csi2 *priv, unsigned int mbps);
>  	int (*phy_post_init)(struct rcar_csi2 *priv);
> -	int (*start_receiver)(struct rcar_csi2 *priv);
> +	int (*start_receiver)(struct rcar_csi2 *priv,
> +			      struct v4l2_subdev_state *state);
>  	void (*enter_standby)(struct rcar_csi2 *priv);
>  	const struct rcsi2_mbps_reg *hsfreqrange;
>  	unsigned int csi0clkfreqrange;
> @@ -613,8 +614,6 @@ struct rcar_csi2 {
>  
>  	int channel_vc[4];
>  
> -	struct mutex lock; /* Protects mf and stream_count. */
> -	struct v4l2_mbus_framefmt mf;
>  	int stream_count;
>  
>  	bool cphy;
> @@ -808,20 +807,25 @@ static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
>  	return 0;
>  }
>  
> -static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
> +static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
> +				     struct v4l2_subdev_state *state)
>  {
>  	const struct rcar_csi2_format *format;
>  	u32 phycnt, vcdt = 0, vcdt2 = 0, fld = 0;
> +	struct v4l2_mbus_framefmt *fmt;

const

>  	unsigned int lanes;
>  	unsigned int i;
>  	int mbps, ret;
>  
> +	/* Use the format on the sink pad to compute the receiver config. */
> +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> +
>  	dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
> -		priv->mf.width, priv->mf.height,
> -		priv->mf.field == V4L2_FIELD_NONE ? 'p' : 'i');
> +		fmt->width, fmt->height,
> +		fmt->field == V4L2_FIELD_NONE ? 'p' : 'i');
>  
>  	/* Code is validated in set_fmt. */
> -	format = rcsi2_code_to_fmt(priv->mf.code);
> +	format = rcsi2_code_to_fmt(fmt->code);
>  	if (!format)
>  		return -EINVAL;
>  
> @@ -849,11 +853,11 @@ static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv)
>  			vcdt2 |= vcdt_part << ((i % 2) * 16);
>  	}
>  
> -	if (priv->mf.field == V4L2_FIELD_ALTERNATE) {
> +	if (fmt->field == V4L2_FIELD_ALTERNATE) {
>  		fld = FLD_DET_SEL(1) | FLD_FLD_EN4 | FLD_FLD_EN3 | FLD_FLD_EN2
>  			| FLD_FLD_EN;
>  
> -		if (priv->mf.height == 240)
> +		if (fmt->height == 240)
>  			fld |= FLD_FLD_NUM(0);
>  		else
>  			fld |= FLD_FLD_NUM(1);
> @@ -1049,15 +1053,18 @@ static int rcsi2_c_phy_setting_v4h(struct rcar_csi2 *priv, int msps)
>  	return 0;
>  }
>  
> -static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
> +static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv,
> +				    struct v4l2_subdev_state *state)
>  {
>  	const struct rcar_csi2_format *format;
> +	struct v4l2_mbus_framefmt *fmt;

const

>  	unsigned int lanes;
>  	int msps;
>  	int ret;
>  
> -	/* Calculate parameters */
> -	format = rcsi2_code_to_fmt(priv->mf.code);
> +	/* Use the format on the sink pad to compute the receiver config. */
> +	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
> +	format = rcsi2_code_to_fmt(fmt->code);
>  	if (!format)
>  		return -EINVAL;
>  
> @@ -1114,7 +1121,7 @@ static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv)
>  	return 0;
>  }
>  
> -static int rcsi2_start(struct rcar_csi2 *priv)
> +static int rcsi2_start(struct rcar_csi2 *priv, struct v4l2_subdev_state *state)
>  {
>  	int ret;
>  
> @@ -1122,7 +1129,7 @@ static int rcsi2_start(struct rcar_csi2 *priv)
>  	if (ret < 0)
>  		return ret;
>  
> -	ret = priv->info->start_receiver(priv);
> +	ret = priv->info->start_receiver(priv, state);
>  	if (ret) {
>  		rcsi2_enter_standby(priv);
>  		return ret;
> @@ -1146,17 +1153,16 @@ static void rcsi2_stop(struct rcar_csi2 *priv)
>  static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
>  {
>  	struct rcar_csi2 *priv = sd_to_csi2(sd);
> +	struct v4l2_subdev_state *state;
>  	int ret = 0;
>  
> -	mutex_lock(&priv->lock);
> +	if (!priv->remote)
> +		return -ENODEV;
>  
> -	if (!priv->remote) {
> -		ret = -ENODEV;
> -		goto out;
> -	}
> +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
>  
>  	if (enable && priv->stream_count == 0) {
> -		ret = rcsi2_start(priv);
> +		ret = rcsi2_start(priv, state);
>  		if (ret)
>  			goto out;
>  	} else if (!enable && priv->stream_count == 1) {
> @@ -1165,49 +1171,26 @@ static int rcsi2_s_stream(struct v4l2_subdev *sd, int enable)
>  
>  	priv->stream_count += enable ? 1 : -1;
>  out:
> -	mutex_unlock(&priv->lock);
> +	v4l2_subdev_unlock_state(state);
>  
>  	return ret;
>  }
>  
>  static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
> -				struct v4l2_subdev_state *sd_state,
> +				struct v4l2_subdev_state *state,
>  				struct v4l2_subdev_format *format)
>  {
> -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> -	struct v4l2_mbus_framefmt *framefmt;
> -
> -	mutex_lock(&priv->lock);
> +	if (format->pad > RCAR_CSI2_SINK)
> +		return v4l2_subdev_get_fmt(sd, state, format);
>  
>  	if (!rcsi2_code_to_fmt(format->format.code))
>  		format->format.code = rcar_csi2_formats[0].code;
>  
> -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> -		priv->mf = format->format;
> -	} else {
> -		framefmt = v4l2_subdev_state_get_format(sd_state, 0);
> -		*framefmt = format->format;
> -	}
> -
> -	mutex_unlock(&priv->lock);
> -
> -	return 0;
> -}
> -
> -static int rcsi2_get_pad_format(struct v4l2_subdev *sd,
> -				struct v4l2_subdev_state *sd_state,
> -				struct v4l2_subdev_format *format)
> -{
> -	struct rcar_csi2 *priv = sd_to_csi2(sd);
> -
> -	mutex_lock(&priv->lock);
> -
> -	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE)
> -		format->format = priv->mf;
> -	else
> -		format->format = *v4l2_subdev_state_get_format(sd_state, 0);
> +	*v4l2_subdev_state_get_format(state, format->pad) = format->format;
>  
> -	mutex_unlock(&priv->lock);
> +	/* Propagate the format to the source pads. */
> +	for (unsigned int i = RCAR_CSI2_SOURCE_VC0; i < NR_OF_RCAR_CSI2_PAD; i++)
> +		*v4l2_subdev_state_get_format(state, i) = format->format;
>  
>  	return 0;
>  }
> @@ -1218,7 +1201,7 @@ static const struct v4l2_subdev_video_ops rcar_csi2_video_ops = {
>  
>  static const struct v4l2_subdev_pad_ops rcar_csi2_pad_ops = {
>  	.set_fmt = rcsi2_set_pad_format,
> -	.get_fmt = rcsi2_get_pad_format,
> +	.get_fmt = v4l2_subdev_get_fmt,
>  };
>  
>  static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
> @@ -1226,6 +1209,30 @@ static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
>  	.pad	= &rcar_csi2_pad_ops,
>  };
>  
> +static int rcsi2_init_state(struct v4l2_subdev *sd,
> +			    struct v4l2_subdev_state *state)
> +{
> +	static const struct v4l2_mbus_framefmt rcar_csi2_default_fmt = {
> +		.width		= 1920,
> +		.height		= 1080,
> +		.code		= MEDIA_BUS_FMT_RGB888_1X24,
> +		.colorspace	= V4L2_COLORSPACE_SRGB,
> +		.field		= V4L2_FIELD_NONE,
> +		.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
> +		.quantization	= V4L2_QUANTIZATION_DEFAULT,
> +		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
> +	};
> +
> +	for (unsigned int i = RCAR_CSI2_SINK; i < NR_OF_RCAR_CSI2_PAD; i++)

I would write 'i = 0'.

Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

> +		*v4l2_subdev_state_get_format(state, i) = rcar_csi2_default_fmt;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_internal_ops rcar_csi2_internal_ops = {
> +	.init_state = rcsi2_init_state,
> +};
> +
>  static irqreturn_t rcsi2_irq(int irq, void *data)
>  {
>  	struct rcar_csi2 *priv = data;
> @@ -1251,14 +1258,17 @@ static irqreturn_t rcsi2_irq(int irq, void *data)
>  
>  static irqreturn_t rcsi2_irq_thread(int irq, void *data)
>  {
> +	struct v4l2_subdev_state *state;
>  	struct rcar_csi2 *priv = data;
>  
> -	mutex_lock(&priv->lock);
> +	state = v4l2_subdev_lock_and_get_active_state(&priv->subdev);
> +
>  	rcsi2_stop(priv);
>  	usleep_range(1000, 2000);
> -	if (rcsi2_start(priv))
> +	if (rcsi2_start(priv, state))
>  		dev_warn(priv->dev, "Failed to restart CSI-2 receiver\n");
> -	mutex_unlock(&priv->lock);
> +
> +	v4l2_subdev_unlock_state(state);
>  
>  	return IRQ_HANDLED;
>  }
> @@ -1870,23 +1880,23 @@ static int rcsi2_probe(struct platform_device *pdev)
>  
>  	priv->dev = &pdev->dev;
>  
> -	mutex_init(&priv->lock);
>  	priv->stream_count = 0;
>  
>  	ret = rcsi2_probe_resources(priv, pdev);
>  	if (ret) {
>  		dev_err(priv->dev, "Failed to get resources\n");
> -		goto error_mutex;
> +		return ret;
>  	}
>  
>  	platform_set_drvdata(pdev, priv);
>  
>  	ret = rcsi2_parse_dt(priv);
>  	if (ret)
> -		goto error_mutex;
> +		return ret;
>  
>  	priv->subdev.owner = THIS_MODULE;
>  	priv->subdev.dev = &pdev->dev;
> +	priv->subdev.internal_ops = &rcar_csi2_internal_ops;
>  	v4l2_subdev_init(&priv->subdev, &rcar_csi2_subdev_ops);
>  	v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
>  	snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s",
> @@ -1912,21 +1922,25 @@ static int rcsi2_probe(struct platform_device *pdev)
>  
>  	pm_runtime_enable(&pdev->dev);
>  
> +	ret = v4l2_subdev_init_finalize(&priv->subdev);
> +	if (ret)
> +		goto error_pm_runtime;
> +
>  	ret = v4l2_async_register_subdev(&priv->subdev);
>  	if (ret < 0)
> -		goto error_pm_runtime;
> +		goto error_subdev;
>  
>  	dev_info(priv->dev, "%d lanes found\n", priv->lanes);
>  
>  	return 0;
>  
> +error_subdev:
> +	v4l2_subdev_cleanup(&priv->subdev);
>  error_pm_runtime:
>  	pm_runtime_disable(&pdev->dev);
>  error_async:
>  	v4l2_async_nf_unregister(&priv->notifier);
>  	v4l2_async_nf_cleanup(&priv->notifier);
> -error_mutex:
> -	mutex_destroy(&priv->lock);
>  
>  	return ret;
>  }
> @@ -1941,8 +1955,6 @@ static void rcsi2_remove(struct platform_device *pdev)
>  	v4l2_subdev_cleanup(&priv->subdev);
>  
>  	pm_runtime_disable(&pdev->dev);
> -
> -	mutex_destroy(&priv->lock);
>  }
>  
>  static struct platform_driver rcar_csi2_pdrv = {

-- 
Regards,

Laurent Pinchart

[PATCH 05/11] media: adv748x-csi2: Initialize subdev format

[Jacopo Mondi]

Initialize the subdevice format to the default 1280x720 YUV 4:2:2.

The subdevice format is propagated from the frontend (HDMI or AFE) to
the CSI-2 tx, but it was not initialized during the CSI-2 tx
registration.

Use YUYV8 1280x720 as default format as it's supported by both TXes in
all supported configurations (1, 2 or 4 CSI-2 lanes).

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/i2c/adv748x/adv748x-csi2.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
index 5b265b722394..9da7f6742a2b 100644
--- a/drivers/media/i2c/adv748x/adv748x-csi2.c
+++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
@@ -14,6 +14,13 @@
 
 #include "adv748x.h"
 
+static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
+	.width = 1280,
+	.height = 720,
+	.code = MEDIA_BUS_FMT_YUYV8_1X16,
+	.field = V4L2_FIELD_NONE,
+};
+
 int adv748x_csi2_set_virtual_channel(struct adv748x_csi2 *tx, unsigned int vc)
 {
 	return tx_write(tx, ADV748X_CSI_VC_REF, vc << ADV748X_CSI_VC_REF_SHIFT);
@@ -311,6 +318,8 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx)
 	if (ret)
 		return ret;
 
+	tx->format = adv748x_csi2_default_fmt;
+
 	ret = v4l2_async_subdev_endpoint_add(&tx->sd,
 					     of_fwnode_handle(state->endpoints[tx->port]));
 	if (ret)
-- 
2.44.0

Re: [PATCH 05/11] media: adv748x-csi2: Initialize subdev format

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:20PM +0200, Jacopo Mondi wrote:
> Initialize the subdevice format to the default 1280x720 YUV 4:2:2.
> 
> The subdevice format is propagated from the frontend (HDMI or AFE) to
> the CSI-2 tx, but it was not initialized during the CSI-2 tx
> registration.
> 
> Use YUYV8 1280x720 as default format as it's supported by both TXes in
> all supported configurations (1, 2 or 4 CSI-2 lanes).
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  drivers/media/i2c/adv748x/adv748x-csi2.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> index 5b265b722394..9da7f6742a2b 100644
> --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> @@ -14,6 +14,13 @@
>  
>  #include "adv748x.h"
>  
> +static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
> +	.width = 1280,
> +	.height = 720,
> +	.code = MEDIA_BUS_FMT_YUYV8_1X16,
> +	.field = V4L2_FIELD_NONE,
> +};
> +
>  int adv748x_csi2_set_virtual_channel(struct adv748x_csi2 *tx, unsigned int vc)
>  {
>  	return tx_write(tx, ADV748X_CSI_VC_REF, vc << ADV748X_CSI_VC_REF_SHIFT);
> @@ -311,6 +318,8 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx)
>  	if (ret)
>  		return ret;
>  
> +	tx->format = adv748x_csi2_default_fmt;
> +

This looks OK, so

Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

But patch 08/11 removes the tx->format field, so I think you could as
well drop this patch.

>  	ret = v4l2_async_subdev_endpoint_add(&tx->sd,
>  					     of_fwnode_handle(state->endpoints[tx->port]));
>  	if (ret)

-- 
Regards,

Laurent Pinchart

[PATCH 06/11] media: adv748x-csi2: Implement enum_mbus_codes

[Jacopo Mondi]

Define a list of supported mbus codes for the TXA and TXB CSI-2
transmitters and implement the enum_mbus_code operation.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/i2c/adv748x/adv748x-csi2.c | 35 ++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
index 9da7f6742a2b..219417b319a6 100644
--- a/drivers/media/i2c/adv748x/adv748x-csi2.c
+++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
@@ -21,6 +21,18 @@ static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
 	.field = V4L2_FIELD_NONE,
 };
 
+static const unsigned int adv748x_csi2_txa_fmts[] = {
+	MEDIA_BUS_FMT_YUYV8_1X16,
+	MEDIA_BUS_FMT_YUYV10_1X20,
+	MEDIA_BUS_FMT_RGB565_1X16,
+	MEDIA_BUS_FMT_RGB666_1X18,
+	MEDIA_BUS_FMT_RGB888_1X24,
+};
+
+static const unsigned int adv748x_csi2_txb_fmts[] = {
+	MEDIA_BUS_FMT_YUYV8_1X16,
+};
+
 int adv748x_csi2_set_virtual_channel(struct adv748x_csi2 *tx, unsigned int vc)
 {
 	return tx_write(tx, ADV748X_CSI_VC_REF, vc << ADV748X_CSI_VC_REF_SHIFT);
@@ -146,6 +158,28 @@ static const struct v4l2_subdev_video_ops adv748x_csi2_video_ops = {
  * But we must support setting the pad formats for format propagation.
  */
 
+static int adv748x_csi2_enum_mbus_code(struct v4l2_subdev *sd,
+				       struct v4l2_subdev_state *sd_state,
+				       struct v4l2_subdev_mbus_code_enum *code)
+{
+	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
+	const unsigned int *codes = is_txa(tx) ?
+				    adv748x_csi2_txa_fmts :
+				    adv748x_csi2_txb_fmts;
+	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
+				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
+
+	if (code->pad != ADV748X_CSI2_SOURCE)
+		return -EINVAL;
+
+	if (code->index >= num_fmts)
+		return -EINVAL;
+
+	code->code = codes[code->index];
+
+	return 0;
+}
+
 static struct v4l2_mbus_framefmt *
 adv748x_csi2_get_pad_format(struct v4l2_subdev *sd,
 			    struct v4l2_subdev_state *sd_state,
@@ -235,6 +269,7 @@ static int adv748x_csi2_get_mbus_config(struct v4l2_subdev *sd, unsigned int pad
 }
 
 static const struct v4l2_subdev_pad_ops adv748x_csi2_pad_ops = {
+	.enum_mbus_code = adv748x_csi2_enum_mbus_code,
 	.get_fmt = adv748x_csi2_get_format,
 	.set_fmt = adv748x_csi2_set_format,
 	.get_mbus_config = adv748x_csi2_get_mbus_config,
-- 
2.44.0

Re: [PATCH 06/11] media: adv748x-csi2: Implement enum_mbus_codes

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:21PM +0200, Jacopo Mondi wrote:
> Define a list of supported mbus codes for the TXA and TXB CSI-2
> transmitters and implement the enum_mbus_code operation.

The commit message should explain why, not just what. Explaining why the
formats for TXA and TXB differ would also be useful.

> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  drivers/media/i2c/adv748x/adv748x-csi2.c | 35 ++++++++++++++++++++++++
>  1 file changed, 35 insertions(+)
> 
> diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> index 9da7f6742a2b..219417b319a6 100644
> --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> @@ -21,6 +21,18 @@ static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
>  	.field = V4L2_FIELD_NONE,
>  };
>  
> +static const unsigned int adv748x_csi2_txa_fmts[] = {
> +	MEDIA_BUS_FMT_YUYV8_1X16,
> +	MEDIA_BUS_FMT_YUYV10_1X20,

CSI-2 uses UYVY, not YUYV.

> +	MEDIA_BUS_FMT_RGB565_1X16,
> +	MEDIA_BUS_FMT_RGB666_1X18,
> +	MEDIA_BUS_FMT_RGB888_1X24,
> +};
> +
> +static const unsigned int adv748x_csi2_txb_fmts[] = {
> +	MEDIA_BUS_FMT_YUYV8_1X16,
> +};
> +
>  int adv748x_csi2_set_virtual_channel(struct adv748x_csi2 *tx, unsigned int vc)
>  {
>  	return tx_write(tx, ADV748X_CSI_VC_REF, vc << ADV748X_CSI_VC_REF_SHIFT);
> @@ -146,6 +158,28 @@ static const struct v4l2_subdev_video_ops adv748x_csi2_video_ops = {
>   * But we must support setting the pad formats for format propagation.
>   */
>  
> +static int adv748x_csi2_enum_mbus_code(struct v4l2_subdev *sd,
> +				       struct v4l2_subdev_state *sd_state,
> +				       struct v4l2_subdev_mbus_code_enum *code)
> +{
> +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> +	const unsigned int *codes = is_txa(tx) ?
> +				    adv748x_csi2_txa_fmts :
> +				    adv748x_csi2_txb_fmts;
> +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> +
> +	if (code->pad != ADV748X_CSI2_SOURCE)
> +		return -EINVAL;
> +
> +	if (code->index >= num_fmts)
> +		return -EINVAL;
> +
> +	code->code = codes[code->index];
> +
> +	return 0;
> +}
> +
>  static struct v4l2_mbus_framefmt *
>  adv748x_csi2_get_pad_format(struct v4l2_subdev *sd,
>  			    struct v4l2_subdev_state *sd_state,
> @@ -235,6 +269,7 @@ static int adv748x_csi2_get_mbus_config(struct v4l2_subdev *sd, unsigned int pad
>  }
>  
>  static const struct v4l2_subdev_pad_ops adv748x_csi2_pad_ops = {
> +	.enum_mbus_code = adv748x_csi2_enum_mbus_code,
>  	.get_fmt = adv748x_csi2_get_format,
>  	.set_fmt = adv748x_csi2_set_format,
>  	.get_mbus_config = adv748x_csi2_get_mbus_config,

-- 
Regards,

Laurent Pinchart

Re: [PATCH 06/11] media: adv748x-csi2: Implement enum_mbus_codes

[Jacopo Mondi]

Hi Laurent

On Mon, May 06, 2024 at 12:07:40AM GMT, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Fri, May 03, 2024 at 05:51:21PM +0200, Jacopo Mondi wrote:
> > Define a list of supported mbus codes for the TXA and TXB CSI-2
> > transmitters and implement the enum_mbus_code operation.
>
> The commit message should explain why, not just what. Explaining why the

Should I explain why the driver has to implement enum_mbus_codes ?

> formats for TXA and TXB differ would also be useful.

ok

>
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >  drivers/media/i2c/adv748x/adv748x-csi2.c | 35 ++++++++++++++++++++++++
> >  1 file changed, 35 insertions(+)
> >
> > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > index 9da7f6742a2b..219417b319a6 100644
> > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > @@ -21,6 +21,18 @@ static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
> >  	.field = V4L2_FIELD_NONE,
> >  };
> >
> > +static const unsigned int adv748x_csi2_txa_fmts[] = {
> > +	MEDIA_BUS_FMT_YUYV8_1X16,
> > +	MEDIA_BUS_FMT_YUYV10_1X20,
>
> CSI-2 uses UYVY, not YUYV.
>

As this a recurring comment in the series (rightfully, I'm not
questioning that) how does it work with existing test scripts assuming
YUYV ? The same question could be asked about the issue Niklas had: is
changing what pad an operation is allowed to be called on legit ?

My answer would be yes, otherwise we'll be forever stuck, but I would
like to check, especially with Niklas which maintains vin-tests.

> > +	MEDIA_BUS_FMT_RGB565_1X16,
> > +	MEDIA_BUS_FMT_RGB666_1X18,
> > +	MEDIA_BUS_FMT_RGB888_1X24,
> > +};
> > +
> > +static const unsigned int adv748x_csi2_txb_fmts[] = {
> > +	MEDIA_BUS_FMT_YUYV8_1X16,
> > +};
> > +
> >  int adv748x_csi2_set_virtual_channel(struct adv748x_csi2 *tx, unsigned int vc)
> >  {
> >  	return tx_write(tx, ADV748X_CSI_VC_REF, vc << ADV748X_CSI_VC_REF_SHIFT);
> > @@ -146,6 +158,28 @@ static const struct v4l2_subdev_video_ops adv748x_csi2_video_ops = {
> >   * But we must support setting the pad formats for format propagation.
> >   */
> >
> > +static int adv748x_csi2_enum_mbus_code(struct v4l2_subdev *sd,
> > +				       struct v4l2_subdev_state *sd_state,
> > +				       struct v4l2_subdev_mbus_code_enum *code)
> > +{
> > +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > +	const unsigned int *codes = is_txa(tx) ?
> > +				    adv748x_csi2_txa_fmts :
> > +				    adv748x_csi2_txb_fmts;
> > +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> > +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> > +
> > +	if (code->pad != ADV748X_CSI2_SOURCE)
> > +		return -EINVAL;
> > +
> > +	if (code->index >= num_fmts)
> > +		return -EINVAL;
> > +
> > +	code->code = codes[code->index];
> > +
> > +	return 0;
> > +}
> > +
> >  static struct v4l2_mbus_framefmt *
> >  adv748x_csi2_get_pad_format(struct v4l2_subdev *sd,
> >  			    struct v4l2_subdev_state *sd_state,
> > @@ -235,6 +269,7 @@ static int adv748x_csi2_get_mbus_config(struct v4l2_subdev *sd, unsigned int pad
> >  }
> >
> >  static const struct v4l2_subdev_pad_ops adv748x_csi2_pad_ops = {
> > +	.enum_mbus_code = adv748x_csi2_enum_mbus_code,
> >  	.get_fmt = adv748x_csi2_get_format,
> >  	.set_fmt = adv748x_csi2_set_format,
> >  	.get_mbus_config = adv748x_csi2_get_mbus_config,
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 06/11] media: adv748x-csi2: Implement enum_mbus_codes

[Laurent Pinchart]

On Mon, May 06, 2024 at 10:10:00AM +0200, Jacopo Mondi wrote:
> On Mon, May 06, 2024 at 12:07:40AM GMT, Laurent Pinchart wrote:
> > On Fri, May 03, 2024 at 05:51:21PM +0200, Jacopo Mondi wrote:
> > > Define a list of supported mbus codes for the TXA and TXB CSI-2
> > > transmitters and implement the enum_mbus_code operation.
> >
> > The commit message should explain why, not just what. Explaining why the
> 
> Should I explain why the driver has to implement enum_mbus_codes ?

You can just note it's mandatory to implement ?

> > formats for TXA and TXB differ would also be useful.
> 
> ok
> 
> > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > ---
> > >  drivers/media/i2c/adv748x/adv748x-csi2.c | 35 ++++++++++++++++++++++++
> > >  1 file changed, 35 insertions(+)
> > >
> > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > index 9da7f6742a2b..219417b319a6 100644
> > > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > @@ -21,6 +21,18 @@ static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
> > >  	.field = V4L2_FIELD_NONE,
> > >  };
> > >
> > > +static const unsigned int adv748x_csi2_txa_fmts[] = {
> > > +	MEDIA_BUS_FMT_YUYV8_1X16,
> > > +	MEDIA_BUS_FMT_YUYV10_1X20,
> >
> > CSI-2 uses UYVY, not YUYV.
> 
> As this a recurring comment in the series (rightfully, I'm not
> questioning that) how does it work with existing test scripts assuming
> YUYV ? The same question could be asked about the issue Niklas had: is
> changing what pad an operation is allowed to be called on legit ?
> 
> My answer would be yes, otherwise we'll be forever stuck, but I would
> like to check, especially with Niklas which maintains vin-tests.

As usual, changes of behaviour are only regressions if someone complains
about them. We can keep backward-compatibility, but it would then be
nice to also support the right media bus codes, and to add a comment
that clearly indicates which ones are for backward-compatibility.

That being said, given that userspace should use UYVY, and given that
the driver currently accepts UYVY, it should at least be enumerated. I
would skip enumeration of the media bus codes that we keep for
backward-compatibility, even if we accept them at runtime.

> > > +	MEDIA_BUS_FMT_RGB565_1X16,
> > > +	MEDIA_BUS_FMT_RGB666_1X18,
> > > +	MEDIA_BUS_FMT_RGB888_1X24,
> > > +};
> > > +
> > > +static const unsigned int adv748x_csi2_txb_fmts[] = {
> > > +	MEDIA_BUS_FMT_YUYV8_1X16,
> > > +};
> > > +
> > >  int adv748x_csi2_set_virtual_channel(struct adv748x_csi2 *tx, unsigned int vc)
> > >  {
> > >  	return tx_write(tx, ADV748X_CSI_VC_REF, vc << ADV748X_CSI_VC_REF_SHIFT);
> > > @@ -146,6 +158,28 @@ static const struct v4l2_subdev_video_ops adv748x_csi2_video_ops = {
> > >   * But we must support setting the pad formats for format propagation.
> > >   */
> > >
> > > +static int adv748x_csi2_enum_mbus_code(struct v4l2_subdev *sd,
> > > +				       struct v4l2_subdev_state *sd_state,
> > > +				       struct v4l2_subdev_mbus_code_enum *code)
> > > +{
> > > +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > > +	const unsigned int *codes = is_txa(tx) ?
> > > +				    adv748x_csi2_txa_fmts :
> > > +				    adv748x_csi2_txb_fmts;
> > > +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> > > +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> > > +
> > > +	if (code->pad != ADV748X_CSI2_SOURCE)
> > > +		return -EINVAL;
> > > +
> > > +	if (code->index >= num_fmts)
> > > +		return -EINVAL;
> > > +
> > > +	code->code = codes[code->index];
> > > +
> > > +	return 0;
> > > +}
> > > +
> > >  static struct v4l2_mbus_framefmt *
> > >  adv748x_csi2_get_pad_format(struct v4l2_subdev *sd,
> > >  			    struct v4l2_subdev_state *sd_state,
> > > @@ -235,6 +269,7 @@ static int adv748x_csi2_get_mbus_config(struct v4l2_subdev *sd, unsigned int pad
> > >  }
> > >
> > >  static const struct v4l2_subdev_pad_ops adv748x_csi2_pad_ops = {
> > > +	.enum_mbus_code = adv748x_csi2_enum_mbus_code,
> > >  	.get_fmt = adv748x_csi2_get_format,
> > >  	.set_fmt = adv748x_csi2_set_format,
> > >  	.get_mbus_config = adv748x_csi2_get_mbus_config,

-- 
Regards,

Laurent Pinchart

Re: [PATCH 06/11] media: adv748x-csi2: Implement enum_mbus_codes

[Jacopo Mondi]

Hi Laurent

On Mon, May 06, 2024 at 11:38:41AM GMT, Laurent Pinchart wrote:
> On Mon, May 06, 2024 at 10:10:00AM +0200, Jacopo Mondi wrote:
> > On Mon, May 06, 2024 at 12:07:40AM GMT, Laurent Pinchart wrote:
> > > On Fri, May 03, 2024 at 05:51:21PM +0200, Jacopo Mondi wrote:
> > > > Define a list of supported mbus codes for the TXA and TXB CSI-2
> > > > transmitters and implement the enum_mbus_code operation.
> > >
> > > The commit message should explain why, not just what. Explaining why the
> >
> > Should I explain why the driver has to implement enum_mbus_codes ?
>
> You can just note it's mandatory to implement ?
>
> > > formats for TXA and TXB differ would also be useful.
> >
> > ok
> >
> > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > ---
> > > >  drivers/media/i2c/adv748x/adv748x-csi2.c | 35 ++++++++++++++++++++++++
> > > >  1 file changed, 35 insertions(+)
> > > >
> > > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > index 9da7f6742a2b..219417b319a6 100644
> > > > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > @@ -21,6 +21,18 @@ static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
> > > >  	.field = V4L2_FIELD_NONE,
> > > >  };
> > > >
> > > > +static const unsigned int adv748x_csi2_txa_fmts[] = {
> > > > +	MEDIA_BUS_FMT_YUYV8_1X16,
> > > > +	MEDIA_BUS_FMT_YUYV10_1X20,
> > >
> > > CSI-2 uses UYVY, not YUYV.
> >
> > As this a recurring comment in the series (rightfully, I'm not
> > questioning that) how does it work with existing test scripts assuming
> > YUYV ? The same question could be asked about the issue Niklas had: is
> > changing what pad an operation is allowed to be called on legit ?
> >
> > My answer would be yes, otherwise we'll be forever stuck, but I would
> > like to check, especially with Niklas which maintains vin-tests.
>
> As usual, changes of behaviour are only regressions if someone complains
> about them. We can keep backward-compatibility, but it would then be
> nice to also support the right media bus codes, and to add a comment
> that clearly indicates which ones are for backward-compatibility.
>
> That being said, given that userspace should use UYVY, and given that
> the driver currently accepts UYVY, it should at least be enumerated. I
> would skip enumeration of the media bus codes that we keep for
> backward-compatibility, even if we accept them at runtime.
>

What about the change to the pad on which s_fmt is allowed on R-Car
CSI-2 ?

> > > > +	MEDIA_BUS_FMT_RGB565_1X16,
> > > > +	MEDIA_BUS_FMT_RGB666_1X18,
> > > > +	MEDIA_BUS_FMT_RGB888_1X24,
> > > > +};
> > > > +
> > > > +static const unsigned int adv748x_csi2_txb_fmts[] = {
> > > > +	MEDIA_BUS_FMT_YUYV8_1X16,
> > > > +};
> > > > +
> > > >  int adv748x_csi2_set_virtual_channel(struct adv748x_csi2 *tx, unsigned int vc)
> > > >  {
> > > >  	return tx_write(tx, ADV748X_CSI_VC_REF, vc << ADV748X_CSI_VC_REF_SHIFT);
> > > > @@ -146,6 +158,28 @@ static const struct v4l2_subdev_video_ops adv748x_csi2_video_ops = {
> > > >   * But we must support setting the pad formats for format propagation.
> > > >   */
> > > >
> > > > +static int adv748x_csi2_enum_mbus_code(struct v4l2_subdev *sd,
> > > > +				       struct v4l2_subdev_state *sd_state,
> > > > +				       struct v4l2_subdev_mbus_code_enum *code)
> > > > +{
> > > > +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > > > +	const unsigned int *codes = is_txa(tx) ?
> > > > +				    adv748x_csi2_txa_fmts :
> > > > +				    adv748x_csi2_txb_fmts;
> > > > +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> > > > +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> > > > +
> > > > +	if (code->pad != ADV748X_CSI2_SOURCE)
> > > > +		return -EINVAL;
> > > > +
> > > > +	if (code->index >= num_fmts)
> > > > +		return -EINVAL;
> > > > +
> > > > +	code->code = codes[code->index];
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > >  static struct v4l2_mbus_framefmt *
> > > >  adv748x_csi2_get_pad_format(struct v4l2_subdev *sd,
> > > >  			    struct v4l2_subdev_state *sd_state,
> > > > @@ -235,6 +269,7 @@ static int adv748x_csi2_get_mbus_config(struct v4l2_subdev *sd, unsigned int pad
> > > >  }
> > > >
> > > >  static const struct v4l2_subdev_pad_ops adv748x_csi2_pad_ops = {
> > > > +	.enum_mbus_code = adv748x_csi2_enum_mbus_code,
> > > >  	.get_fmt = adv748x_csi2_get_format,
> > > >  	.set_fmt = adv748x_csi2_set_format,
> > > >  	.get_mbus_config = adv748x_csi2_get_mbus_config,
>
> --
> Regards,
>
> Laurent Pinchart

Re: [PATCH 06/11] media: adv748x-csi2: Implement enum_mbus_codes

[Niklas Söderlund]

On 2024-05-06 10:42:32 +0200, Jacopo Mondi wrote:
> Hi Laurent
> 
> On Mon, May 06, 2024 at 11:38:41AM GMT, Laurent Pinchart wrote:
> > On Mon, May 06, 2024 at 10:10:00AM +0200, Jacopo Mondi wrote:
> > > On Mon, May 06, 2024 at 12:07:40AM GMT, Laurent Pinchart wrote:
> > > > On Fri, May 03, 2024 at 05:51:21PM +0200, Jacopo Mondi wrote:
> > > > > Define a list of supported mbus codes for the TXA and TXB CSI-2
> > > > > transmitters and implement the enum_mbus_code operation.
> > > >
> > > > The commit message should explain why, not just what. Explaining why the
> > >
> > > Should I explain why the driver has to implement enum_mbus_codes ?
> >
> > You can just note it's mandatory to implement ?
> >
> > > > formats for TXA and TXB differ would also be useful.
> > >
> > > ok
> > >
> > > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > > ---
> > > > >  drivers/media/i2c/adv748x/adv748x-csi2.c | 35 ++++++++++++++++++++++++
> > > > >  1 file changed, 35 insertions(+)
> > > > >
> > > > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > index 9da7f6742a2b..219417b319a6 100644
> > > > > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > @@ -21,6 +21,18 @@ static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
> > > > >  	.field = V4L2_FIELD_NONE,
> > > > >  };
> > > > >
> > > > > +static const unsigned int adv748x_csi2_txa_fmts[] = {
> > > > > +	MEDIA_BUS_FMT_YUYV8_1X16,
> > > > > +	MEDIA_BUS_FMT_YUYV10_1X20,
> > > >
> > > > CSI-2 uses UYVY, not YUYV.
> > >
> > > As this a recurring comment in the series (rightfully, I'm not
> > > questioning that) how does it work with existing test scripts assuming
> > > YUYV ? The same question could be asked about the issue Niklas had: is
> > > changing what pad an operation is allowed to be called on legit ?
> > >
> > > My answer would be yes, otherwise we'll be forever stuck, but I would
> > > like to check, especially with Niklas which maintains vin-tests.
> >
> > As usual, changes of behaviour are only regressions if someone complains
> > about them. We can keep backward-compatibility, but it would then be
> > nice to also support the right media bus codes, and to add a comment
> > that clearly indicates which ones are for backward-compatibility.
> >
> > That being said, given that userspace should use UYVY, and given that
> > the driver currently accepts UYVY, it should at least be enumerated. I
> > would skip enumeration of the media bus codes that we keep for
> > backward-compatibility, even if we accept them at runtime.

I'm OK just dropping them and update the use-cases, better get this 
right for the future. If we find use-cases we can't update, keeping 
accepting the formats but not enumerating them seems like a good option.

> >
> 
> What about the change to the pad on which s_fmt is allowed on R-Car
> CSI-2 ?
> 
> > > > > +	MEDIA_BUS_FMT_RGB565_1X16,
> > > > > +	MEDIA_BUS_FMT_RGB666_1X18,
> > > > > +	MEDIA_BUS_FMT_RGB888_1X24,
> > > > > +};
> > > > > +
> > > > > +static const unsigned int adv748x_csi2_txb_fmts[] = {
> > > > > +	MEDIA_BUS_FMT_YUYV8_1X16,
> > > > > +};
> > > > > +
> > > > >  int adv748x_csi2_set_virtual_channel(struct adv748x_csi2 *tx, unsigned int vc)
> > > > >  {
> > > > >  	return tx_write(tx, ADV748X_CSI_VC_REF, vc << ADV748X_CSI_VC_REF_SHIFT);
> > > > > @@ -146,6 +158,28 @@ static const struct v4l2_subdev_video_ops adv748x_csi2_video_ops = {
> > > > >   * But we must support setting the pad formats for format propagation.
> > > > >   */
> > > > >
> > > > > +static int adv748x_csi2_enum_mbus_code(struct v4l2_subdev *sd,
> > > > > +				       struct v4l2_subdev_state *sd_state,
> > > > > +				       struct v4l2_subdev_mbus_code_enum *code)
> > > > > +{
> > > > > +	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > > > > +	const unsigned int *codes = is_txa(tx) ?
> > > > > +				    adv748x_csi2_txa_fmts :
> > > > > +				    adv748x_csi2_txb_fmts;
> > > > > +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> > > > > +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> > > > > +
> > > > > +	if (code->pad != ADV748X_CSI2_SOURCE)
> > > > > +		return -EINVAL;
> > > > > +
> > > > > +	if (code->index >= num_fmts)
> > > > > +		return -EINVAL;
> > > > > +
> > > > > +	code->code = codes[code->index];
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > > +
> > > > >  static struct v4l2_mbus_framefmt *
> > > > >  adv748x_csi2_get_pad_format(struct v4l2_subdev *sd,
> > > > >  			    struct v4l2_subdev_state *sd_state,
> > > > > @@ -235,6 +269,7 @@ static int adv748x_csi2_get_mbus_config(struct v4l2_subdev *sd, unsigned int pad
> > > > >  }
> > > > >
> > > > >  static const struct v4l2_subdev_pad_ops adv748x_csi2_pad_ops = {
> > > > > +	.enum_mbus_code = adv748x_csi2_enum_mbus_code,
> > > > >  	.get_fmt = adv748x_csi2_get_format,
> > > > >  	.set_fmt = adv748x_csi2_set_format,
> > > > >  	.get_mbus_config = adv748x_csi2_get_mbus_config,
> >
> > --
> > Regards,
> >
> > Laurent Pinchart

-- 
Kind Regards,
Niklas Söderlund

[PATCH 07/11] media: adv748x-csi2: Validate the image format

[Jacopo Mondi]

The adv748x-csi2 driver configures the CSI-2 transmitter to
automatically infer the image stream format from the connected
frontend (HDMI or AFE).

Setting a new format on the subdevice hence does not actually control
the CSI-2 output format, but it's only there for the purpose of
pipeline validation.

However, there is currently no validation that the supplied media bus
code is valid and supported by the device.

With the introduction of enum_mbus_codes a list of supported format is
now available, use it to validate that the supplied format is correct
and use the default YUYV8 one if that's not the case.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/i2c/adv748x/adv748x-csi2.c | 27 +++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
index 219417b319a6..1aae6467ca62 100644
--- a/drivers/media/i2c/adv748x/adv748x-csi2.c
+++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
@@ -215,6 +215,23 @@ static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
+					 unsigned int code)
+{
+	const unsigned int *codes = is_txa(tx) ?
+				    adv748x_csi2_txa_fmts :
+				    adv748x_csi2_txb_fmts;
+	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
+				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
+
+	for (unsigned int i = 0; i < num_fmts; i++) {
+		if (codes[i] == code)
+			return 0;
+	}
+
+	return -EINVAL;
+}
+
 static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
 				   struct v4l2_subdev_state *sd_state,
 				   struct v4l2_subdev_format *sdformat)
@@ -222,7 +239,15 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
 	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
 	struct adv748x_state *state = tx->state;
 	struct v4l2_mbus_framefmt *mbusformat;
-	int ret = 0;
+	int ret;
+
+	/*
+	 * Make sure the format is supported, if not default it to
+	 * YUYV8 as it's supported by both TXes.
+	 */
+	ret = adv748x_csi2_is_fmt_supported(tx, sdformat->format.code);
+	if (ret)
+		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;
 
 	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
 						 sdformat->which);
-- 
2.44.0

Re: [PATCH 07/11] media: adv748x-csi2: Validate the image format

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:22PM +0200, Jacopo Mondi wrote:
> The adv748x-csi2 driver configures the CSI-2 transmitter to
> automatically infer the image stream format from the connected
> frontend (HDMI or AFE).
> 
> Setting a new format on the subdevice hence does not actually control
> the CSI-2 output format, but it's only there for the purpose of
> pipeline validation.
> 
> However, there is currently no validation that the supplied media bus
> code is valid and supported by the device.
> 
> With the introduction of enum_mbus_codes a list of supported format is
> now available, use it to validate that the supplied format is correct
> and use the default YUYV8 one if that's not the case.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  drivers/media/i2c/adv748x/adv748x-csi2.c | 27 +++++++++++++++++++++++-
>  1 file changed, 26 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> index 219417b319a6..1aae6467ca62 100644
> --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> @@ -215,6 +215,23 @@ static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
>  	return 0;
>  }
>  
> +static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
> +					 unsigned int code)
> +{
> +	const unsigned int *codes = is_txa(tx) ?
> +				    adv748x_csi2_txa_fmts :
> +				    adv748x_csi2_txb_fmts;
> +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> +
> +	for (unsigned int i = 0; i < num_fmts; i++) {
> +		if (codes[i] == code)
> +			return 0;
> +	}
> +
> +	return -EINVAL;
> +}
> +
>  static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
>  				   struct v4l2_subdev_state *sd_state,
>  				   struct v4l2_subdev_format *sdformat)
> @@ -222,7 +239,15 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
>  	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
>  	struct adv748x_state *state = tx->state;
>  	struct v4l2_mbus_framefmt *mbusformat;
> -	int ret = 0;
> +	int ret;
> +
> +	/*
> +	 * Make sure the format is supported, if not default it to
> +	 * YUYV8 as it's supported by both TXes.

As indicated in the review if 06/11, I think this should be UYVY8

> +	 */
> +	ret = adv748x_csi2_is_fmt_supported(tx, sdformat->format.code);
> +	if (ret)
> +		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;
>  
>  	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
>  						 sdformat->which);

-- 
Regards,

Laurent Pinchart

Re: [PATCH 07/11] media: adv748x-csi2: Validate the image format

[Niklas Söderlund]

Hi Jacopo,

Thanks for your work.

On 2024-05-03 17:51:22 +0200, Jacopo Mondi wrote:
> The adv748x-csi2 driver configures the CSI-2 transmitter to
> automatically infer the image stream format from the connected
> frontend (HDMI or AFE).
> 
> Setting a new format on the subdevice hence does not actually control
> the CSI-2 output format, but it's only there for the purpose of
> pipeline validation.
> 
> However, there is currently no validation that the supplied media bus
> code is valid and supported by the device.
> 
> With the introduction of enum_mbus_codes a list of supported format is
> now available, use it to validate that the supplied format is correct
> and use the default YUYV8 one if that's not the case.

With the update tests for the change in patch 4 I hit multiple issues 
with this patch for CVBS capture.

> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  drivers/media/i2c/adv748x/adv748x-csi2.c | 27 +++++++++++++++++++++++-
>  1 file changed, 26 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> index 219417b319a6..1aae6467ca62 100644
> --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> @@ -215,6 +215,23 @@ static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
>  	return 0;
>  }
>  
> +static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
> +					 unsigned int code)
> +{
> +	const unsigned int *codes = is_txa(tx) ?
> +				    adv748x_csi2_txa_fmts :
> +				    adv748x_csi2_txb_fmts;
> +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> +
> +	for (unsigned int i = 0; i < num_fmts; i++) {
> +		if (codes[i] == code)
> +			return 0;
> +	}
> +
> +	return -EINVAL;
> +}
> +
>  static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
>  				   struct v4l2_subdev_state *sd_state,
>  				   struct v4l2_subdev_format *sdformat)
> @@ -222,7 +239,15 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
>  	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
>  	struct adv748x_state *state = tx->state;
>  	struct v4l2_mbus_framefmt *mbusformat;
> -	int ret = 0;
> +	int ret;
> +
> +	/*
> +	 * Make sure the format is supported, if not default it to
> +	 * YUYV8 as it's supported by both TXes.
> +	 */
> +	ret = adv748x_csi2_is_fmt_supported(tx, sdformat->format.code);
> +	if (ret)
> +		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;

If adv748x_csi2_is_fmt_supported() returns non-zero you default to 
MEDIA_BUS_FMT_YUYV8_1X16, which is fine. But the non-zero return code is 
propagated at the end of this function and to user-space falling the 
IOCTL.

Fixing that I hit another issue that kind of shows we need this format 
validation ;-)

The TXB entity only supports MEDIA_BUS_FMT_YUYV8_1X16 formats, the AFE 
entity only outputs MEDIA_BUS_FMT_UYVY8_2X8... So with format validation 
in place it becomes impossible to connect AFE to TXB and breaking CBVS 
capture on Gen3. As a hack I added MEDIA_BUS_FMT_UYVY8_2X8 support to 
TXB and I can again capture CVBS with patch 1-8 applied.

>  
>  	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
>  						 sdformat->which);
> -- 
> 2.44.0
> 

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH 07/11] media: adv748x-csi2: Validate the image format

[Jacopo Mondi]

Hi Niklas

On Mon, May 06, 2024 at 01:37:30PM GMT, Niklas Söderlund wrote:
> Hi Jacopo,
>
> Thanks for your work.
>
> On 2024-05-03 17:51:22 +0200, Jacopo Mondi wrote:
> > The adv748x-csi2 driver configures the CSI-2 transmitter to
> > automatically infer the image stream format from the connected
> > frontend (HDMI or AFE).
> >
> > Setting a new format on the subdevice hence does not actually control
> > the CSI-2 output format, but it's only there for the purpose of
> > pipeline validation.
> >
> > However, there is currently no validation that the supplied media bus
> > code is valid and supported by the device.
> >
> > With the introduction of enum_mbus_codes a list of supported format is
> > now available, use it to validate that the supplied format is correct
> > and use the default YUYV8 one if that's not the case.
>
> With the update tests for the change in patch 4 I hit multiple issues
> with this patch for CVBS capture.
>
> >
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >  drivers/media/i2c/adv748x/adv748x-csi2.c | 27 +++++++++++++++++++++++-
> >  1 file changed, 26 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > index 219417b319a6..1aae6467ca62 100644
> > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > @@ -215,6 +215,23 @@ static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
> >  	return 0;
> >  }
> >
> > +static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
> > +					 unsigned int code)
> > +{
> > +	const unsigned int *codes = is_txa(tx) ?
> > +				    adv748x_csi2_txa_fmts :
> > +				    adv748x_csi2_txb_fmts;
> > +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> > +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> > +
> > +	for (unsigned int i = 0; i < num_fmts; i++) {
> > +		if (codes[i] == code)
> > +			return 0;
> > +	}
> > +
> > +	return -EINVAL;
> > +}
> > +
> >  static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> >  				   struct v4l2_subdev_state *sd_state,
> >  				   struct v4l2_subdev_format *sdformat)
> > @@ -222,7 +239,15 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> >  	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> >  	struct adv748x_state *state = tx->state;
> >  	struct v4l2_mbus_framefmt *mbusformat;
> > -	int ret = 0;
> > +	int ret;
> > +
> > +	/*
> > +	 * Make sure the format is supported, if not default it to
> > +	 * YUYV8 as it's supported by both TXes.
> > +	 */
> > +	ret = adv748x_csi2_is_fmt_supported(tx, sdformat->format.code);
> > +	if (ret)
> > +		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;
>
> If adv748x_csi2_is_fmt_supported() returns non-zero you default to
> MEDIA_BUS_FMT_YUYV8_1X16, which is fine. But the non-zero return code is
> propagated at the end of this function and to user-space falling the
> IOCTL.

Ouch, I think in my tests the error message got ignored

>
> Fixing that I hit another issue that kind of shows we need this format
> validation ;-)
>
> The TXB entity only supports MEDIA_BUS_FMT_YUYV8_1X16 formats, the AFE
> entity only outputs MEDIA_BUS_FMT_UYVY8_2X8... So with format validation
> in place it becomes impossible to connect AFE to TXB and breaking CBVS
> capture on Gen3. As a hack I added MEDIA_BUS_FMT_UYVY8_2X8 support to
> TXB and I can again capture CVBS with patch 1-8 applied.

Thanks for testing analog capture, I don't have a setup to easily do
so.

Should we make the AFE front-end produce 1X16 instead ? The format is
only used between the AFE and TXs after all, changing it shouldn't
have any implication on the interoperability with other devices.
>
> >
> >  	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
> >  						 sdformat->which);
> > --
> > 2.44.0
> >
>
> --
> Kind Regards,
> Niklas Söderlund

Re: [PATCH 07/11] media: adv748x-csi2: Validate the image format

[Niklas Söderlund]

Hi Jacopo,

On 2024-05-06 15:21:30 +0200, Jacopo Mondi wrote:
> Hi Niklas
> 
> On Mon, May 06, 2024 at 01:37:30PM GMT, Niklas Söderlund wrote:
> > Hi Jacopo,
> >
> > Thanks for your work.
> >
> > On 2024-05-03 17:51:22 +0200, Jacopo Mondi wrote:
> > > The adv748x-csi2 driver configures the CSI-2 transmitter to
> > > automatically infer the image stream format from the connected
> > > frontend (HDMI or AFE).
> > >
> > > Setting a new format on the subdevice hence does not actually control
> > > the CSI-2 output format, but it's only there for the purpose of
> > > pipeline validation.
> > >
> > > However, there is currently no validation that the supplied media bus
> > > code is valid and supported by the device.
> > >
> > > With the introduction of enum_mbus_codes a list of supported format is
> > > now available, use it to validate that the supplied format is correct
> > > and use the default YUYV8 one if that's not the case.
> >
> > With the update tests for the change in patch 4 I hit multiple issues
> > with this patch for CVBS capture.
> >
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > ---
> > >  drivers/media/i2c/adv748x/adv748x-csi2.c | 27 +++++++++++++++++++++++-
> > >  1 file changed, 26 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > index 219417b319a6..1aae6467ca62 100644
> > > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > @@ -215,6 +215,23 @@ static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
> > >  	return 0;
> > >  }
> > >
> > > +static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
> > > +					 unsigned int code)
> > > +{
> > > +	const unsigned int *codes = is_txa(tx) ?
> > > +				    adv748x_csi2_txa_fmts :
> > > +				    adv748x_csi2_txb_fmts;
> > > +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> > > +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> > > +
> > > +	for (unsigned int i = 0; i < num_fmts; i++) {
> > > +		if (codes[i] == code)
> > > +			return 0;
> > > +	}
> > > +
> > > +	return -EINVAL;
> > > +}
> > > +
> > >  static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> > >  				   struct v4l2_subdev_state *sd_state,
> > >  				   struct v4l2_subdev_format *sdformat)
> > > @@ -222,7 +239,15 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> > >  	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > >  	struct adv748x_state *state = tx->state;
> > >  	struct v4l2_mbus_framefmt *mbusformat;
> > > -	int ret = 0;
> > > +	int ret;
> > > +
> > > +	/*
> > > +	 * Make sure the format is supported, if not default it to
> > > +	 * YUYV8 as it's supported by both TXes.
> > > +	 */
> > > +	ret = adv748x_csi2_is_fmt_supported(tx, sdformat->format.code);
> > > +	if (ret)
> > > +		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;
> >
> > If adv748x_csi2_is_fmt_supported() returns non-zero you default to
> > MEDIA_BUS_FMT_YUYV8_1X16, which is fine. But the non-zero return code is
> > propagated at the end of this function and to user-space falling the
> > IOCTL.
> 
> Ouch, I think in my tests the error message got ignored
> 
> >
> > Fixing that I hit another issue that kind of shows we need this format
> > validation ;-)
> >
> > The TXB entity only supports MEDIA_BUS_FMT_YUYV8_1X16 formats, the AFE
> > entity only outputs MEDIA_BUS_FMT_UYVY8_2X8... So with format validation
> > in place it becomes impossible to connect AFE to TXB and breaking CBVS
> > capture on Gen3. As a hack I added MEDIA_BUS_FMT_UYVY8_2X8 support to
> > TXB and I can again capture CVBS with patch 1-8 applied.
> 
> Thanks for testing analog capture, I don't have a setup to easily do
> so.

You can test it with the pattern generator on any Gen3 board.

> 
> Should we make the AFE front-end produce 1X16 instead ? The format is
> only used between the AFE and TXs after all, changing it shouldn't
> have any implication on the interoperability with other devices.

Not sure, the list of formats supported by each entity in the ADV748x is 
added by patch 'media: adv748x-csi2: Implement enum_mbus_codes' in this 
series. Where did the list come from? What formats do the AFE support?  
Why is the formats supported different between TXA and TXB ?

> >
> > >
> > >  	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
> > >  						 sdformat->which);
> > > --
> > > 2.44.0
> > >
> >
> > --
> > Kind Regards,
> > Niklas Söderlund

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH 07/11] media: adv748x-csi2: Validate the image format

[Jacopo Mondi]

Hi Niklas

On Mon, May 06, 2024 at 04:12:01PM GMT, Niklas Söderlund wrote:
> Hi Jacopo,
>
> On 2024-05-06 15:21:30 +0200, Jacopo Mondi wrote:
> > Hi Niklas
> >
> > On Mon, May 06, 2024 at 01:37:30PM GMT, Niklas Söderlund wrote:
> > > Hi Jacopo,
> > >
> > > Thanks for your work.
> > >
> > > On 2024-05-03 17:51:22 +0200, Jacopo Mondi wrote:
> > > > The adv748x-csi2 driver configures the CSI-2 transmitter to
> > > > automatically infer the image stream format from the connected
> > > > frontend (HDMI or AFE).
> > > >
> > > > Setting a new format on the subdevice hence does not actually control
> > > > the CSI-2 output format, but it's only there for the purpose of
> > > > pipeline validation.
> > > >
> > > > However, there is currently no validation that the supplied media bus
> > > > code is valid and supported by the device.
> > > >
> > > > With the introduction of enum_mbus_codes a list of supported format is
> > > > now available, use it to validate that the supplied format is correct
> > > > and use the default YUYV8 one if that's not the case.
> > >
> > > With the update tests for the change in patch 4 I hit multiple issues
> > > with this patch for CVBS capture.
> > >
> > > >
> > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > ---
> > > >  drivers/media/i2c/adv748x/adv748x-csi2.c | 27 +++++++++++++++++++++++-
> > > >  1 file changed, 26 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > index 219417b319a6..1aae6467ca62 100644
> > > > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > @@ -215,6 +215,23 @@ static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
> > > >  	return 0;
> > > >  }
> > > >
> > > > +static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
> > > > +					 unsigned int code)
> > > > +{
> > > > +	const unsigned int *codes = is_txa(tx) ?
> > > > +				    adv748x_csi2_txa_fmts :
> > > > +				    adv748x_csi2_txb_fmts;
> > > > +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> > > > +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> > > > +
> > > > +	for (unsigned int i = 0; i < num_fmts; i++) {
> > > > +		if (codes[i] == code)
> > > > +			return 0;
> > > > +	}
> > > > +
> > > > +	return -EINVAL;
> > > > +}
> > > > +
> > > >  static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> > > >  				   struct v4l2_subdev_state *sd_state,
> > > >  				   struct v4l2_subdev_format *sdformat)
> > > > @@ -222,7 +239,15 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> > > >  	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > > >  	struct adv748x_state *state = tx->state;
> > > >  	struct v4l2_mbus_framefmt *mbusformat;
> > > > -	int ret = 0;
> > > > +	int ret;
> > > > +
> > > > +	/*
> > > > +	 * Make sure the format is supported, if not default it to
> > > > +	 * YUYV8 as it's supported by both TXes.
> > > > +	 */
> > > > +	ret = adv748x_csi2_is_fmt_supported(tx, sdformat->format.code);
> > > > +	if (ret)
> > > > +		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;
> > >
> > > If adv748x_csi2_is_fmt_supported() returns non-zero you default to
> > > MEDIA_BUS_FMT_YUYV8_1X16, which is fine. But the non-zero return code is
> > > propagated at the end of this function and to user-space falling the
> > > IOCTL.
> >
> > Ouch, I think in my tests the error message got ignored
> >
> > >
> > > Fixing that I hit another issue that kind of shows we need this format
> > > validation ;-)
> > >
> > > The TXB entity only supports MEDIA_BUS_FMT_YUYV8_1X16 formats, the AFE
> > > entity only outputs MEDIA_BUS_FMT_UYVY8_2X8... So with format validation
> > > in place it becomes impossible to connect AFE to TXB and breaking CBVS
> > > capture on Gen3. As a hack I added MEDIA_BUS_FMT_UYVY8_2X8 support to
> > > TXB and I can again capture CVBS with patch 1-8 applied.
> >
> > Thanks for testing analog capture, I don't have a setup to easily do
> > so.
>
> You can test it with the pattern generator on any Gen3 board.
>

ah well

> >
> > Should we make the AFE front-end produce 1X16 instead ? The format is
> > only used between the AFE and TXs after all, changing it shouldn't
> > have any implication on the interoperability with other devices.
>
> Not sure, the list of formats supported by each entity in the ADV748x is
> added by patch 'media: adv748x-csi2: Implement enum_mbus_codes' in this
> series.

> Where did the list come from?

From the chip datasheet ?
Section 9.7 "MIPI Ouput format"

>What formats do the AFE support?

The AFE doesn't really "support" any format in my understanding. It
connects to one of the two TXes with an internal processing pipeline,
and the TX transmits the received video stream on the serial bus.

> Why is the formats supported different between TXA and TXB ?

You should ask the chip producer :)

>
> > >
> > > >
> > > >  	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
> > > >  						 sdformat->which);
> > > > --
> > > > 2.44.0
> > > >
> > >
> > > --
> > > Kind Regards,
> > > Niklas Söderlund
>
> --
> Kind Regards,
> Niklas Söderlund
>

Re: [PATCH 07/11] media: adv748x-csi2: Validate the image format

[Niklas Söderlund]

Hello Jacopo,

On 2024-05-06 16:36:01 +0200, Jacopo Mondi wrote:
> Hi Niklas
> 
> On Mon, May 06, 2024 at 04:12:01PM GMT, Niklas Söderlund wrote:
> > Hi Jacopo,
> >
> > On 2024-05-06 15:21:30 +0200, Jacopo Mondi wrote:
> > > Hi Niklas
> > >
> > > On Mon, May 06, 2024 at 01:37:30PM GMT, Niklas Söderlund wrote:
> > > > Hi Jacopo,
> > > >
> > > > Thanks for your work.
> > > >
> > > > On 2024-05-03 17:51:22 +0200, Jacopo Mondi wrote:
> > > > > The adv748x-csi2 driver configures the CSI-2 transmitter to
> > > > > automatically infer the image stream format from the connected
> > > > > frontend (HDMI or AFE).
> > > > >
> > > > > Setting a new format on the subdevice hence does not actually control
> > > > > the CSI-2 output format, but it's only there for the purpose of
> > > > > pipeline validation.
> > > > >
> > > > > However, there is currently no validation that the supplied media bus
> > > > > code is valid and supported by the device.
> > > > >
> > > > > With the introduction of enum_mbus_codes a list of supported format is
> > > > > now available, use it to validate that the supplied format is correct
> > > > > and use the default YUYV8 one if that's not the case.
> > > >
> > > > With the update tests for the change in patch 4 I hit multiple issues
> > > > with this patch for CVBS capture.
> > > >
> > > > >
> > > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > > ---
> > > > >  drivers/media/i2c/adv748x/adv748x-csi2.c | 27 +++++++++++++++++++++++-
> > > > >  1 file changed, 26 insertions(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > index 219417b319a6..1aae6467ca62 100644
> > > > > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > @@ -215,6 +215,23 @@ static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
> > > > >  	return 0;
> > > > >  }
> > > > >
> > > > > +static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
> > > > > +					 unsigned int code)
> > > > > +{
> > > > > +	const unsigned int *codes = is_txa(tx) ?
> > > > > +				    adv748x_csi2_txa_fmts :
> > > > > +				    adv748x_csi2_txb_fmts;
> > > > > +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> > > > > +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> > > > > +
> > > > > +	for (unsigned int i = 0; i < num_fmts; i++) {
> > > > > +		if (codes[i] == code)
> > > > > +			return 0;
> > > > > +	}
> > > > > +
> > > > > +	return -EINVAL;
> > > > > +}
> > > > > +
> > > > >  static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> > > > >  				   struct v4l2_subdev_state *sd_state,
> > > > >  				   struct v4l2_subdev_format *sdformat)
> > > > > @@ -222,7 +239,15 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> > > > >  	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > > > >  	struct adv748x_state *state = tx->state;
> > > > >  	struct v4l2_mbus_framefmt *mbusformat;
> > > > > -	int ret = 0;
> > > > > +	int ret;
> > > > > +
> > > > > +	/*
> > > > > +	 * Make sure the format is supported, if not default it to
> > > > > +	 * YUYV8 as it's supported by both TXes.
> > > > > +	 */
> > > > > +	ret = adv748x_csi2_is_fmt_supported(tx, sdformat->format.code);
> > > > > +	if (ret)
> > > > > +		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;
> > > >
> > > > If adv748x_csi2_is_fmt_supported() returns non-zero you default to
> > > > MEDIA_BUS_FMT_YUYV8_1X16, which is fine. But the non-zero return code is
> > > > propagated at the end of this function and to user-space falling the
> > > > IOCTL.
> > >
> > > Ouch, I think in my tests the error message got ignored
> > >
> > > >
> > > > Fixing that I hit another issue that kind of shows we need this format
> > > > validation ;-)
> > > >
> > > > The TXB entity only supports MEDIA_BUS_FMT_YUYV8_1X16 formats, the AFE
> > > > entity only outputs MEDIA_BUS_FMT_UYVY8_2X8... So with format validation
> > > > in place it becomes impossible to connect AFE to TXB and breaking CBVS
> > > > capture on Gen3. As a hack I added MEDIA_BUS_FMT_UYVY8_2X8 support to
> > > > TXB and I can again capture CVBS with patch 1-8 applied.
> > >
> > > Thanks for testing analog capture, I don't have a setup to easily do
> > > so.
> >
> > You can test it with the pattern generator on any Gen3 board.
> >
> 
> ah well
> 
> > >
> > > Should we make the AFE front-end produce 1X16 instead ? The format is
> > > only used between the AFE and TXs after all, changing it shouldn't
> > > have any implication on the interoperability with other devices.
> >
> > Not sure, the list of formats supported by each entity in the ADV748x is
> > added by patch 'media: adv748x-csi2: Implement enum_mbus_codes' in this
> > series.
> 
> > Where did the list come from?
> 
> From the chip datasheet ?
> Section 9.7 "MIPI Ouput format"

Thanks I found it now, maybe add that to the commit message of that 
patch? I was hunting for register values in the register control manual 
and found nothing ;-)
> 
> >What formats do the AFE support?
> 
> The AFE doesn't really "support" any format in my understanding. It
> connects to one of the two TXes with an internal processing pipeline,
> and the TX transmits the received video stream on the serial bus.

Ahh! I think we found the root of the issue we talked about the other 
day in the VIN format handling about 1X16 vs 2x8 and CSI-2 ;-) That 
likely came from this setting.

Yes, with the information from the datahseet I do think we should change 
adv748x_afe_enum_mbus_code() to report MEDIA_BUS_FMT_YUYV8_1X16 instead 
of MEDIA_BUS_FMT_UYVY8_2X8.

> 
> > Why is the formats supported different between TXA and TXB ?
> 
> You should ask the chip producer :)

If only we could. Imagine how much time we save if we could talk to them 
and have datasheets instead of guessing half the time ;-)

> 
> >
> > > >
> > > > >
> > > > >  	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
> > > > >  						 sdformat->which);
> > > > > --
> > > > > 2.44.0
> > > > >
> > > >
> > > > --
> > > > Kind Regards,
> > > > Niklas Söderlund
> >
> > --
> > Kind Regards,
> > Niklas Söderlund
> >

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH 07/11] media: adv748x-csi2: Validate the image format

[Jacopo Mondi]

Hi Niklas

On Mon, May 06, 2024 at 04:58:30PM GMT, Niklas Söderlund wrote:
> Hello Jacopo,
>
> On 2024-05-06 16:36:01 +0200, Jacopo Mondi wrote:
> > Hi Niklas
> >
> > On Mon, May 06, 2024 at 04:12:01PM GMT, Niklas Söderlund wrote:
> > > Hi Jacopo,
> > >
> > > On 2024-05-06 15:21:30 +0200, Jacopo Mondi wrote:
> > > > Hi Niklas
> > > >
> > > > On Mon, May 06, 2024 at 01:37:30PM GMT, Niklas Söderlund wrote:
> > > > > Hi Jacopo,
> > > > >
> > > > > Thanks for your work.
> > > > >
> > > > > On 2024-05-03 17:51:22 +0200, Jacopo Mondi wrote:
> > > > > > The adv748x-csi2 driver configures the CSI-2 transmitter to
> > > > > > automatically infer the image stream format from the connected
> > > > > > frontend (HDMI or AFE).
> > > > > >
> > > > > > Setting a new format on the subdevice hence does not actually control
> > > > > > the CSI-2 output format, but it's only there for the purpose of
> > > > > > pipeline validation.
> > > > > >
> > > > > > However, there is currently no validation that the supplied media bus
> > > > > > code is valid and supported by the device.
> > > > > >
> > > > > > With the introduction of enum_mbus_codes a list of supported format is
> > > > > > now available, use it to validate that the supplied format is correct
> > > > > > and use the default YUYV8 one if that's not the case.
> > > > >
> > > > > With the update tests for the change in patch 4 I hit multiple issues
> > > > > with this patch for CVBS capture.
> > > > >
> > > > > >
> > > > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > > > ---
> > > > > >  drivers/media/i2c/adv748x/adv748x-csi2.c | 27 +++++++++++++++++++++++-
> > > > > >  1 file changed, 26 insertions(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > > index 219417b319a6..1aae6467ca62 100644
> > > > > > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > > @@ -215,6 +215,23 @@ static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
> > > > > >  	return 0;
> > > > > >  }
> > > > > >
> > > > > > +static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
> > > > > > +					 unsigned int code)
> > > > > > +{
> > > > > > +	const unsigned int *codes = is_txa(tx) ?
> > > > > > +				    adv748x_csi2_txa_fmts :
> > > > > > +				    adv748x_csi2_txb_fmts;
> > > > > > +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> > > > > > +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> > > > > > +
> > > > > > +	for (unsigned int i = 0; i < num_fmts; i++) {
> > > > > > +		if (codes[i] == code)
> > > > > > +			return 0;
> > > > > > +	}
> > > > > > +
> > > > > > +	return -EINVAL;
> > > > > > +}
> > > > > > +
> > > > > >  static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> > > > > >  				   struct v4l2_subdev_state *sd_state,
> > > > > >  				   struct v4l2_subdev_format *sdformat)
> > > > > > @@ -222,7 +239,15 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> > > > > >  	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > > > > >  	struct adv748x_state *state = tx->state;
> > > > > >  	struct v4l2_mbus_framefmt *mbusformat;
> > > > > > -	int ret = 0;
> > > > > > +	int ret;
> > > > > > +
> > > > > > +	/*
> > > > > > +	 * Make sure the format is supported, if not default it to
> > > > > > +	 * YUYV8 as it's supported by both TXes.
> > > > > > +	 */
> > > > > > +	ret = adv748x_csi2_is_fmt_supported(tx, sdformat->format.code);
> > > > > > +	if (ret)
> > > > > > +		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;
> > > > >
> > > > > If adv748x_csi2_is_fmt_supported() returns non-zero you default to
> > > > > MEDIA_BUS_FMT_YUYV8_1X16, which is fine. But the non-zero return code is
> > > > > propagated at the end of this function and to user-space falling the
> > > > > IOCTL.
> > > >
> > > > Ouch, I think in my tests the error message got ignored
> > > >
> > > > >
> > > > > Fixing that I hit another issue that kind of shows we need this format
> > > > > validation ;-)
> > > > >
> > > > > The TXB entity only supports MEDIA_BUS_FMT_YUYV8_1X16 formats, the AFE
> > > > > entity only outputs MEDIA_BUS_FMT_UYVY8_2X8... So with format validation
> > > > > in place it becomes impossible to connect AFE to TXB and breaking CBVS
> > > > > capture on Gen3. As a hack I added MEDIA_BUS_FMT_UYVY8_2X8 support to
> > > > > TXB and I can again capture CVBS with patch 1-8 applied.
> > > >
> > > > Thanks for testing analog capture, I don't have a setup to easily do
> > > > so.
> > >
> > > You can test it with the pattern generator on any Gen3 board.
> > >
> >
> > ah well
> >
> > > >
> > > > Should we make the AFE front-end produce 1X16 instead ? The format is
> > > > only used between the AFE and TXs after all, changing it shouldn't
> > > > have any implication on the interoperability with other devices.
> > >
> > > Not sure, the list of formats supported by each entity in the ADV748x is
> > > added by patch 'media: adv748x-csi2: Implement enum_mbus_codes' in this
> > > series.
> >
> > > Where did the list come from?
> >
> > From the chip datasheet ?
> > Section 9.7 "MIPI Ouput format"
>
> Thanks I found it now, maybe add that to the commit message of that
> patch? I was hunting for register values in the register control manual
> and found nothing ;-)

ok..

> >
> > >What formats do the AFE support?
> >
> > The AFE doesn't really "support" any format in my understanding. It
> > connects to one of the two TXes with an internal processing pipeline,
> > and the TX transmits the received video stream on the serial bus.
>
> Ahh! I think we found the root of the issue we talked about the other
> day in the VIN format handling about 1X16 vs 2x8 and CSI-2 ;-) That
> likely came from this setting.
>
> Yes, with the information from the datahseet I do think we should change
> adv748x_afe_enum_mbus_code() to report MEDIA_BUS_FMT_YUYV8_1X16 instead

nit: MEDIA_BUS_FMT_UYVY8_1X16

> of MEDIA_BUS_FMT_UYVY8_2X8.
>
> >
> > > Why is the formats supported different between TXA and TXB ?
> >
> > You should ask the chip producer :)
>
> If only we could. Imagine how much time we save if we could talk to them
> and have datasheets instead of guessing half the time ;-)
>

:')

> >
> > >
> > > > >
> > > > > >
> > > > > >  	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
> > > > > >  						 sdformat->which);
> > > > > > --
> > > > > > 2.44.0
> > > > > >
> > > > >
> > > > > --
> > > > > Kind Regards,
> > > > > Niklas Söderlund
> > >
> > > --
> > > Kind Regards,
> > > Niklas Söderlund
> > >
>
> --
> Kind Regards,
> Niklas Söderlund

Re: [PATCH 07/11] media: adv748x-csi2: Validate the image format

[Laurent Pinchart]

On Mon, May 06, 2024 at 05:04:52PM +0200, Jacopo Mondi wrote:
> On Mon, May 06, 2024 at 04:58:30PM GMT, Niklas Söderlund wrote:
> > On 2024-05-06 16:36:01 +0200, Jacopo Mondi wrote:
> > > On Mon, May 06, 2024 at 04:12:01PM GMT, Niklas Söderlund wrote:
> > > > On 2024-05-06 15:21:30 +0200, Jacopo Mondi wrote:
> > > > > On Mon, May 06, 2024 at 01:37:30PM GMT, Niklas Söderlund wrote:
> > > > > > On 2024-05-03 17:51:22 +0200, Jacopo Mondi wrote:
> > > > > > > The adv748x-csi2 driver configures the CSI-2 transmitter to
> > > > > > > automatically infer the image stream format from the connected
> > > > > > > frontend (HDMI or AFE).
> > > > > > >
> > > > > > > Setting a new format on the subdevice hence does not actually control
> > > > > > > the CSI-2 output format, but it's only there for the purpose of
> > > > > > > pipeline validation.
> > > > > > >
> > > > > > > However, there is currently no validation that the supplied media bus
> > > > > > > code is valid and supported by the device.
> > > > > > >
> > > > > > > With the introduction of enum_mbus_codes a list of supported format is
> > > > > > > now available, use it to validate that the supplied format is correct
> > > > > > > and use the default YUYV8 one if that's not the case.
> > > > > >
> > > > > > With the update tests for the change in patch 4 I hit multiple issues
> > > > > > with this patch for CVBS capture.
> > > > > >
> > > > > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > > > > ---
> > > > > > >  drivers/media/i2c/adv748x/adv748x-csi2.c | 27 +++++++++++++++++++++++-
> > > > > > >  1 file changed, 26 insertions(+), 1 deletion(-)
> > > > > > >
> > > > > > > diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > > > index 219417b319a6..1aae6467ca62 100644
> > > > > > > --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > > > +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> > > > > > > @@ -215,6 +215,23 @@ static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
> > > > > > >  	return 0;
> > > > > > >  }
> > > > > > >
> > > > > > > +static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
> > > > > > > +					 unsigned int code)
> > > > > > > +{
> > > > > > > +	const unsigned int *codes = is_txa(tx) ?
> > > > > > > +				    adv748x_csi2_txa_fmts :
> > > > > > > +				    adv748x_csi2_txb_fmts;
> > > > > > > +	size_t num_fmts = is_txa(tx) ? ARRAY_SIZE(adv748x_csi2_txa_fmts)
> > > > > > > +				     : ARRAY_SIZE(adv748x_csi2_txb_fmts);
> > > > > > > +
> > > > > > > +	for (unsigned int i = 0; i < num_fmts; i++) {
> > > > > > > +		if (codes[i] == code)
> > > > > > > +			return 0;
> > > > > > > +	}
> > > > > > > +
> > > > > > > +	return -EINVAL;
> > > > > > > +}
> > > > > > > +
> > > > > > >  static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> > > > > > >  				   struct v4l2_subdev_state *sd_state,
> > > > > > >  				   struct v4l2_subdev_format *sdformat)
> > > > > > > @@ -222,7 +239,15 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
> > > > > > >  	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> > > > > > >  	struct adv748x_state *state = tx->state;
> > > > > > >  	struct v4l2_mbus_framefmt *mbusformat;
> > > > > > > -	int ret = 0;
> > > > > > > +	int ret;
> > > > > > > +
> > > > > > > +	/*
> > > > > > > +	 * Make sure the format is supported, if not default it to
> > > > > > > +	 * YUYV8 as it's supported by both TXes.
> > > > > > > +	 */
> > > > > > > +	ret = adv748x_csi2_is_fmt_supported(tx, sdformat->format.code);
> > > > > > > +	if (ret)
> > > > > > > +		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;
> > > > > >
> > > > > > If adv748x_csi2_is_fmt_supported() returns non-zero you default to
> > > > > > MEDIA_BUS_FMT_YUYV8_1X16, which is fine. But the non-zero return code is
> > > > > > propagated at the end of this function and to user-space falling the
> > > > > > IOCTL.
> > > > >
> > > > > Ouch, I think in my tests the error message got ignored
> > > > >
> > > > > > Fixing that I hit another issue that kind of shows we need this format
> > > > > > validation ;-)
> > > > > >
> > > > > > The TXB entity only supports MEDIA_BUS_FMT_YUYV8_1X16 formats, the AFE
> > > > > > entity only outputs MEDIA_BUS_FMT_UYVY8_2X8... So with format validation
> > > > > > in place it becomes impossible to connect AFE to TXB and breaking CBVS
> > > > > > capture on Gen3. As a hack I added MEDIA_BUS_FMT_UYVY8_2X8 support to
> > > > > > TXB and I can again capture CVBS with patch 1-8 applied.
> > > > >
> > > > > Thanks for testing analog capture, I don't have a setup to easily do
> > > > > so.
> > > >
> > > > You can test it with the pattern generator on any Gen3 board.
> > >
> > > ah well
> > >
> > > > > Should we make the AFE front-end produce 1X16 instead ? The format is
> > > > > only used between the AFE and TXs after all, changing it shouldn't
> > > > > have any implication on the interoperability with other devices.
> > > >
> > > > Not sure, the list of formats supported by each entity in the ADV748x is
> > > > added by patch 'media: adv748x-csi2: Implement enum_mbus_codes' in this
> > > > series.
> > >
> > > > Where did the list come from?
> > >
> > > From the chip datasheet ?
> > > Section 9.7 "MIPI Ouput format"
> >
> > Thanks I found it now, maybe add that to the commit message of that
> > patch? I was hunting for register values in the register control manual
> > and found nothing ;-)
> 
> ok..
> 
> > > > What formats do the AFE support?
> > >
> > > The AFE doesn't really "support" any format in my understanding. It
> > > connects to one of the two TXes with an internal processing pipeline,
> > > and the TX transmits the received video stream on the serial bus.
> >
> > Ahh! I think we found the root of the issue we talked about the other
> > day in the VIN format handling about 1X16 vs 2x8 and CSI-2 ;-) That
> > likely came from this setting.
> >
> > Yes, with the information from the datahseet I do think we should change
> > adv748x_afe_enum_mbus_code() to report MEDIA_BUS_FMT_YUYV8_1X16 instead
> 
> nit: MEDIA_BUS_FMT_UYVY8_1X16

A general note about internal formats: in the absence of information
telling the exact format used in internal buses, and actually even when
that information is available but the exact format doesn't affect
anything, you're free to pick whatever seems logical enough and makes
life the easiest for userspace and kernel drivers. In this specific
case, MEDIA_BUS_FMT_YUYV8_1X16 would work, but I think
MEDIA_BUS_FMT_UYVY8_1X16 is better as it will match the CSI-2 TX output,
saving us from writing conversion code in the CSI-2 TX subdev.

> > of MEDIA_BUS_FMT_UYVY8_2X8.
> >
> > > > Why is the formats supported different between TXA and TXB ?
> > >
> > > You should ask the chip producer :)
> >
> > If only we could. Imagine how much time we save if we could talk to them
> > and have datasheets instead of guessing half the time ;-)
> 
> :')
> 
> > > > > > >
> > > > > > >  	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
> > > > > > >  						 sdformat->which);

-- 
Regards,

Laurent Pinchart

[PATCH 08/11] media: adv748x-csi2: Use the subdev active state

[Jacopo Mondi]

Initialize and use the subdev active state to store the subdevice
format.

This simplifies the implementation of the get_fmt and set_fmt pad
operations.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/i2c/adv748x/adv748x-csi2.c | 114 ++++++++---------------
 drivers/media/i2c/adv748x/adv748x.h      |   1 -
 2 files changed, 41 insertions(+), 74 deletions(-)

diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
index 1aae6467ca62..15a77387b32e 100644
--- a/drivers/media/i2c/adv748x/adv748x-csi2.c
+++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
@@ -6,7 +6,6 @@
  */
 
 #include <linux/module.h>
-#include <linux/mutex.h>
 
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-device.h>
@@ -14,13 +13,6 @@
 
 #include "adv748x.h"
 
-static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
-	.width = 1280,
-	.height = 720,
-	.code = MEDIA_BUS_FMT_YUYV8_1X16,
-	.field = V4L2_FIELD_NONE,
-};
-
 static const unsigned int adv748x_csi2_txa_fmts[] = {
 	MEDIA_BUS_FMT_YUYV8_1X16,
 	MEDIA_BUS_FMT_YUYV10_1X20,
@@ -78,7 +70,32 @@ static int adv748x_csi2_register_link(struct adv748x_csi2 *tx,
 
 /* -----------------------------------------------------------------------------
  * v4l2_subdev_internal_ops
- *
+ */
+
+static int adv748x_csi2_init_state(struct v4l2_subdev *sd,
+				   struct v4l2_subdev_state *state)
+{
+	static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
+		.width		= 1280,
+		.height		= 720,
+		.code		= MEDIA_BUS_FMT_YUYV8_1X16,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.field		= V4L2_FIELD_NONE,
+		.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
+		.quantization	= V4L2_QUANTIZATION_DEFAULT,
+		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
+	};
+	struct v4l2_mbus_framefmt *fmt;
+
+	fmt = v4l2_subdev_state_get_format(state, ADV748X_CSI2_SINK);
+	*fmt = adv748x_csi2_default_fmt;
+
+	*v4l2_subdev_state_get_format(state, ADV748X_CSI2_SOURCE) = *fmt;
+
+	return 0;
+}
+
+/*
  * We use the internal registered operation to be able to ensure that our
  * incremental subdevices (not connected in the forward path) can be registered
  * against the resulting video path and media device.
@@ -128,6 +145,7 @@ static int adv748x_csi2_registered(struct v4l2_subdev *sd)
 }
 
 static const struct v4l2_subdev_internal_ops adv748x_csi2_internal_ops = {
+	.init_state = adv748x_csi2_init_state,
 	.registered = adv748x_csi2_registered,
 };
 
@@ -180,41 +198,6 @@ static int adv748x_csi2_enum_mbus_code(struct v4l2_subdev *sd,
 	return 0;
 }
 
-static struct v4l2_mbus_framefmt *
-adv748x_csi2_get_pad_format(struct v4l2_subdev *sd,
-			    struct v4l2_subdev_state *sd_state,
-			    unsigned int pad, u32 which)
-{
-	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
-
-	if (which == V4L2_SUBDEV_FORMAT_TRY)
-		return v4l2_subdev_state_get_format(sd_state, pad);
-
-	return &tx->format;
-}
-
-static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
-				   struct v4l2_subdev_state *sd_state,
-				   struct v4l2_subdev_format *sdformat)
-{
-	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
-	struct adv748x_state *state = tx->state;
-	struct v4l2_mbus_framefmt *mbusformat;
-
-	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
-						 sdformat->which);
-	if (!mbusformat)
-		return -EINVAL;
-
-	mutex_lock(&state->mutex);
-
-	sdformat->format = *mbusformat;
-
-	mutex_unlock(&state->mutex);
-
-	return 0;
-}
-
 static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
 					 unsigned int code)
 {
@@ -237,10 +220,12 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
 				   struct v4l2_subdev_format *sdformat)
 {
 	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
-	struct adv748x_state *state = tx->state;
 	struct v4l2_mbus_framefmt *mbusformat;
 	int ret;
 
+	if (sdformat->pad == ADV748X_CSI2_SOURCE)
+		return v4l2_subdev_get_fmt(sd, sd_state, sdformat);
+
 	/*
 	 * Make sure the format is supported, if not default it to
 	 * YUYV8 as it's supported by both TXes.
@@ -249,34 +234,14 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
 	if (ret)
 		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;
 
-	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
-						 sdformat->which);
-	if (!mbusformat)
-		return -EINVAL;
-
-	mutex_lock(&state->mutex);
-
-	if (sdformat->pad == ADV748X_CSI2_SOURCE) {
-		const struct v4l2_mbus_framefmt *sink_fmt;
-
-		sink_fmt = adv748x_csi2_get_pad_format(sd, sd_state,
-						       ADV748X_CSI2_SINK,
-						       sdformat->which);
-
-		if (!sink_fmt) {
-			ret = -EINVAL;
-			goto unlock;
-		}
-
-		sdformat->format = *sink_fmt;
-	}
-
+	mbusformat = v4l2_subdev_state_get_format(sd_state, sdformat->pad);
 	*mbusformat = sdformat->format;
 
-unlock:
-	mutex_unlock(&state->mutex);
+	/* Propagate format to the source pad. */
+	mbusformat = v4l2_subdev_state_get_format(sd_state, ADV748X_CSI2_SOURCE);
+	*mbusformat = sdformat->format;
 
-	return ret;
+	return 0;
 }
 
 static int adv748x_csi2_get_mbus_config(struct v4l2_subdev *sd, unsigned int pad,
@@ -295,7 +260,7 @@ static int adv748x_csi2_get_mbus_config(struct v4l2_subdev *sd, unsigned int pad
 
 static const struct v4l2_subdev_pad_ops adv748x_csi2_pad_ops = {
 	.enum_mbus_code = adv748x_csi2_enum_mbus_code,
-	.get_fmt = adv748x_csi2_get_format,
+	.get_fmt = v4l2_subdev_get_fmt,
 	.set_fmt = adv748x_csi2_set_format,
 	.get_mbus_config = adv748x_csi2_get_mbus_config,
 };
@@ -378,8 +343,6 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx)
 	if (ret)
 		return ret;
 
-	tx->format = adv748x_csi2_default_fmt;
-
 	ret = v4l2_async_subdev_endpoint_add(&tx->sd,
 					     of_fwnode_handle(state->endpoints[tx->port]));
 	if (ret)
@@ -389,6 +352,11 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx)
 	if (ret)
 		goto err_cleanup_subdev;
 
+	tx->sd.state_lock = &state->mutex;
+	ret = v4l2_subdev_init_finalize(&tx->sd);
+	if (ret)
+		goto err_free_ctrl;
+
 	ret = v4l2_async_register_subdev(&tx->sd);
 	if (ret)
 		goto err_free_ctrl;
diff --git a/drivers/media/i2c/adv748x/adv748x.h b/drivers/media/i2c/adv748x/adv748x.h
index d2b5e722e997..9bc0121d0eff 100644
--- a/drivers/media/i2c/adv748x/adv748x.h
+++ b/drivers/media/i2c/adv748x/adv748x.h
@@ -75,7 +75,6 @@ enum adv748x_csi2_pads {
 
 struct adv748x_csi2 {
 	struct adv748x_state *state;
-	struct v4l2_mbus_framefmt format;
 	unsigned int page;
 	unsigned int port;
 	unsigned int num_lanes;
-- 
2.44.0

Re: [PATCH 08/11] media: adv748x-csi2: Use the subdev active state

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:23PM +0200, Jacopo Mondi wrote:
> Initialize and use the subdev active state to store the subdevice
> format.
> 
> This simplifies the implementation of the get_fmt and set_fmt pad
> operations.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  drivers/media/i2c/adv748x/adv748x-csi2.c | 114 ++++++++---------------
>  drivers/media/i2c/adv748x/adv748x.h      |   1 -
>  2 files changed, 41 insertions(+), 74 deletions(-)
> 
> diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
> index 1aae6467ca62..15a77387b32e 100644
> --- a/drivers/media/i2c/adv748x/adv748x-csi2.c
> +++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
> @@ -6,7 +6,6 @@
>   */
>  
>  #include <linux/module.h>
> -#include <linux/mutex.h>
>  
>  #include <media/v4l2-ctrls.h>
>  #include <media/v4l2-device.h>
> @@ -14,13 +13,6 @@
>  
>  #include "adv748x.h"
>  
> -static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
> -	.width = 1280,
> -	.height = 720,
> -	.code = MEDIA_BUS_FMT_YUYV8_1X16,
> -	.field = V4L2_FIELD_NONE,
> -};
> -
>  static const unsigned int adv748x_csi2_txa_fmts[] = {
>  	MEDIA_BUS_FMT_YUYV8_1X16,
>  	MEDIA_BUS_FMT_YUYV10_1X20,
> @@ -78,7 +70,32 @@ static int adv748x_csi2_register_link(struct adv748x_csi2 *tx,
>  
>  /* -----------------------------------------------------------------------------
>   * v4l2_subdev_internal_ops
> - *
> + */
> +
> +static int adv748x_csi2_init_state(struct v4l2_subdev *sd,
> +				   struct v4l2_subdev_state *state)
> +{
> +	static const struct v4l2_mbus_framefmt adv748x_csi2_default_fmt = {
> +		.width		= 1280,
> +		.height		= 720,
> +		.code		= MEDIA_BUS_FMT_YUYV8_1X16,

I think this should be UYVY8.

> +		.colorspace	= V4L2_COLORSPACE_SRGB,
> +		.field		= V4L2_FIELD_NONE,
> +		.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
> +		.quantization	= V4L2_QUANTIZATION_DEFAULT,
> +		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
> +	};
> +	struct v4l2_mbus_framefmt *fmt;
> +
> +	fmt = v4l2_subdev_state_get_format(state, ADV748X_CSI2_SINK);
> +	*fmt = adv748x_csi2_default_fmt;
> +
> +	*v4l2_subdev_state_get_format(state, ADV748X_CSI2_SOURCE) = *fmt;

For consistency, I'd write

	fmt = v4l2_subdev_state_get_format(state, ADV748X_CSI2_SOURCE);
	*fmt = adv748x_csi2_default_fmt;

> +
> +	return 0;
> +}
> +
> +/*
>   * We use the internal registered operation to be able to ensure that our
>   * incremental subdevices (not connected in the forward path) can be registered
>   * against the resulting video path and media device.
> @@ -128,6 +145,7 @@ static int adv748x_csi2_registered(struct v4l2_subdev *sd)
>  }
>  
>  static const struct v4l2_subdev_internal_ops adv748x_csi2_internal_ops = {
> +	.init_state = adv748x_csi2_init_state,
>  	.registered = adv748x_csi2_registered,
>  };
>  
> @@ -180,41 +198,6 @@ static int adv748x_csi2_enum_mbus_code(struct v4l2_subdev *sd,
>  	return 0;
>  }
>  
> -static struct v4l2_mbus_framefmt *
> -adv748x_csi2_get_pad_format(struct v4l2_subdev *sd,
> -			    struct v4l2_subdev_state *sd_state,
> -			    unsigned int pad, u32 which)
> -{
> -	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> -
> -	if (which == V4L2_SUBDEV_FORMAT_TRY)
> -		return v4l2_subdev_state_get_format(sd_state, pad);
> -
> -	return &tx->format;
> -}
> -
> -static int adv748x_csi2_get_format(struct v4l2_subdev *sd,
> -				   struct v4l2_subdev_state *sd_state,
> -				   struct v4l2_subdev_format *sdformat)
> -{
> -	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> -	struct adv748x_state *state = tx->state;
> -	struct v4l2_mbus_framefmt *mbusformat;
> -
> -	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
> -						 sdformat->which);
> -	if (!mbusformat)
> -		return -EINVAL;
> -
> -	mutex_lock(&state->mutex);
> -
> -	sdformat->format = *mbusformat;
> -
> -	mutex_unlock(&state->mutex);
> -
> -	return 0;
> -}
> -
>  static int adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx,
>  					 unsigned int code)
>  {
> @@ -237,10 +220,12 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
>  				   struct v4l2_subdev_format *sdformat)
>  {
>  	struct adv748x_csi2 *tx = adv748x_sd_to_csi2(sd);
> -	struct adv748x_state *state = tx->state;
>  	struct v4l2_mbus_framefmt *mbusformat;
>  	int ret;
>  
> +	if (sdformat->pad == ADV748X_CSI2_SOURCE)
> +		return v4l2_subdev_get_fmt(sd, sd_state, sdformat);
> +
>  	/*
>  	 * Make sure the format is supported, if not default it to
>  	 * YUYV8 as it's supported by both TXes.
> @@ -249,34 +234,14 @@ static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
>  	if (ret)
>  		sdformat->format.code = MEDIA_BUS_FMT_YUYV8_1X16;
>  
> -	mbusformat = adv748x_csi2_get_pad_format(sd, sd_state, sdformat->pad,
> -						 sdformat->which);
> -	if (!mbusformat)
> -		return -EINVAL;
> -
> -	mutex_lock(&state->mutex);

The function was locking the device-wide mutex, now it's only locking
the state of this particular subdev. Have you checked this doesn't
introduce any race condition ? If so, please indicate it in the commit
message. If so, please check it :-)

> -
> -	if (sdformat->pad == ADV748X_CSI2_SOURCE) {
> -		const struct v4l2_mbus_framefmt *sink_fmt;
> -
> -		sink_fmt = adv748x_csi2_get_pad_format(sd, sd_state,
> -						       ADV748X_CSI2_SINK,
> -						       sdformat->which);
> -
> -		if (!sink_fmt) {
> -			ret = -EINVAL;
> -			goto unlock;
> -		}
> -
> -		sdformat->format = *sink_fmt;
> -	}
> -
> +	mbusformat = v4l2_subdev_state_get_format(sd_state, sdformat->pad);
>  	*mbusformat = sdformat->format;
>  
> -unlock:
> -	mutex_unlock(&state->mutex);
> +	/* Propagate format to the source pad. */
> +	mbusformat = v4l2_subdev_state_get_format(sd_state, ADV748X_CSI2_SOURCE);
> +	*mbusformat = sdformat->format;
>  
> -	return ret;
> +	return 0;
>  }
>  
>  static int adv748x_csi2_get_mbus_config(struct v4l2_subdev *sd, unsigned int pad,
> @@ -295,7 +260,7 @@ static int adv748x_csi2_get_mbus_config(struct v4l2_subdev *sd, unsigned int pad
>  
>  static const struct v4l2_subdev_pad_ops adv748x_csi2_pad_ops = {
>  	.enum_mbus_code = adv748x_csi2_enum_mbus_code,
> -	.get_fmt = adv748x_csi2_get_format,
> +	.get_fmt = v4l2_subdev_get_fmt,
>  	.set_fmt = adv748x_csi2_set_format,
>  	.get_mbus_config = adv748x_csi2_get_mbus_config,
>  };
> @@ -378,8 +343,6 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx)
>  	if (ret)
>  		return ret;
>  
> -	tx->format = adv748x_csi2_default_fmt;
> -
>  	ret = v4l2_async_subdev_endpoint_add(&tx->sd,
>  					     of_fwnode_handle(state->endpoints[tx->port]));
>  	if (ret)
> @@ -389,6 +352,11 @@ int adv748x_csi2_init(struct adv748x_state *state, struct adv748x_csi2 *tx)
>  	if (ret)
>  		goto err_cleanup_subdev;
>  
> +	tx->sd.state_lock = &state->mutex;

Ah, that answers my question above.

Can you also convert the AFE and HDMI subdevs to use the active state ?

> +	ret = v4l2_subdev_init_finalize(&tx->sd);
> +	if (ret)
> +		goto err_free_ctrl;
> +
>  	ret = v4l2_async_register_subdev(&tx->sd);
>  	if (ret)
>  		goto err_free_ctrl;
> diff --git a/drivers/media/i2c/adv748x/adv748x.h b/drivers/media/i2c/adv748x/adv748x.h
> index d2b5e722e997..9bc0121d0eff 100644
> --- a/drivers/media/i2c/adv748x/adv748x.h
> +++ b/drivers/media/i2c/adv748x/adv748x.h
> @@ -75,7 +75,6 @@ enum adv748x_csi2_pads {
>  
>  struct adv748x_csi2 {
>  	struct adv748x_state *state;
> -	struct v4l2_mbus_framefmt format;
>  	unsigned int page;
>  	unsigned int port;
>  	unsigned int num_lanes;

-- 
Regards,

Laurent Pinchart

[PATCH 09/11] media: max9286: Fix enum_mbus_code

[Jacopo Mondi]

The max9286 driver supports multiple output formats but only a single
one is reported through the .enum_mbus_code operation.

Fix that.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/i2c/max9286.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c
index d685d445cf23..5321238cad60 100644
--- a/drivers/media/i2c/max9286.c
+++ b/drivers/media/i2c/max9286.c
@@ -914,10 +914,10 @@ static int max9286_enum_mbus_code(struct v4l2_subdev *sd,
 				  struct v4l2_subdev_state *sd_state,
 				  struct v4l2_subdev_mbus_code_enum *code)
 {
-	if (code->pad || code->index > 0)
+	if (code->pad || code->index >= ARRAY_SIZE(max9286_formats))
 		return -EINVAL;
 
-	code->code = MEDIA_BUS_FMT_UYVY8_1X16;
+	code->code = max9286_formats[code->index].code;
 
 	return 0;
 }
-- 
2.44.0

Re: [PATCH 09/11] media: max9286: Fix enum_mbus_code

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:24PM +0200, Jacopo Mondi wrote:
> The max9286 driver supports multiple output formats but only a single
> one is reported through the .enum_mbus_code operation.
> 
> Fix that.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

> ---
>  drivers/media/i2c/max9286.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c
> index d685d445cf23..5321238cad60 100644
> --- a/drivers/media/i2c/max9286.c
> +++ b/drivers/media/i2c/max9286.c
> @@ -914,10 +914,10 @@ static int max9286_enum_mbus_code(struct v4l2_subdev *sd,
>  				  struct v4l2_subdev_state *sd_state,
>  				  struct v4l2_subdev_mbus_code_enum *code)
>  {
> -	if (code->pad || code->index > 0)
> +	if (code->pad || code->index >= ARRAY_SIZE(max9286_formats))
>  		return -EINVAL;
>  
> -	code->code = MEDIA_BUS_FMT_UYVY8_1X16;
> +	code->code = max9286_formats[code->index].code;
>  
>  	return 0;
>  }

-- 
Regards,

Laurent Pinchart

[PATCH 10/11] media: max9286: Use the subdev active state

[Jacopo Mondi]

Use the subdev active state in the max9286 driver to store the
image format.

Replace the .open() function call with the .init_state() one and
simplify the set/get_pad_fmt() operations.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/i2c/max9286.c | 131 +++++++++++++-----------------------
 1 file changed, 46 insertions(+), 85 deletions(-)

diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c
index 5321238cad60..cc7ee35560fc 100644
--- a/drivers/media/i2c/max9286.c
+++ b/drivers/media/i2c/max9286.c
@@ -19,7 +19,6 @@
 #include <linux/i2c.h>
 #include <linux/i2c-mux.h>
 #include <linux/module.h>
-#include <linux/mutex.h>
 #include <linux/of_graph.h>
 #include <linux/regulator/consumer.h>
 #include <linux/slab.h>
@@ -198,12 +197,8 @@ struct max9286_priv {
 	struct v4l2_ctrl *pixelrate_ctrl;
 	unsigned int pixelrate;
 
-	struct v4l2_mbus_framefmt fmt[MAX9286_N_SINKS];
 	struct v4l2_fract interval;
 
-	/* Protects controls and fmt structures */
-	struct mutex mutex;
-
 	unsigned int nsources;
 	unsigned int source_mask;
 	unsigned int route_mask;
@@ -788,19 +783,22 @@ static void max9286_v4l2_notifier_unregister(struct max9286_priv *priv)
 static int max9286_s_stream(struct v4l2_subdev *sd, int enable)
 {
 	struct max9286_priv *priv = sd_to_max9286(sd);
+	struct v4l2_subdev_state *state;
 	struct max9286_source *source;
 	unsigned int i;
 	bool sync = false;
-	int ret;
+	int ret = 0;
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
 
 	if (enable) {
 		const struct v4l2_mbus_framefmt *format;
 
 		/*
-		 * Get the format from the first used sink pad, as all sink
-		 * formats must be identical.
+		 * Get the format from the source pad, as all formats must be
+		 * identical.
 		 */
-		format = &priv->fmt[__ffs(priv->bound_sources)];
+		format = v4l2_subdev_state_get_format(state, MAX9286_SRC_PAD);
 
 		max9286_set_video_format(priv, format);
 		max9286_set_fsync_period(priv);
@@ -816,12 +814,12 @@ static int max9286_s_stream(struct v4l2_subdev *sd, int enable)
 		for_each_source(priv, source) {
 			ret = v4l2_subdev_call(source->sd, video, s_stream, 1);
 			if (ret)
-				return ret;
+				goto unlock;
 		}
 
 		ret = max9286_check_video_links(priv);
 		if (ret)
-			return ret;
+			goto unlock;
 
 		/*
 		 * Wait until frame synchronization is locked.
@@ -842,7 +840,8 @@ static int max9286_s_stream(struct v4l2_subdev *sd, int enable)
 		if (!sync) {
 			dev_err(&priv->client->dev,
 				"Failed to get frame synchronization\n");
-			return -EXDEV; /* Invalid cross-device link */
+			ret = -EXDEV; /* Invalid cross-device link */
+			goto unlock;
 		}
 
 		/*
@@ -865,7 +864,10 @@ static int max9286_s_stream(struct v4l2_subdev *sd, int enable)
 		max9286_i2c_mux_close(priv);
 	}
 
-	return 0;
+unlock:
+	v4l2_subdev_unlock_state(state);
+
+	return ret;
 }
 
 static int max9286_get_frame_interval(struct v4l2_subdev *sd,
@@ -922,31 +924,20 @@ static int max9286_enum_mbus_code(struct v4l2_subdev *sd,
 	return 0;
 }
 
-static struct v4l2_mbus_framefmt *
-max9286_get_pad_format(struct max9286_priv *priv,
-		       struct v4l2_subdev_state *sd_state,
-		       unsigned int pad, u32 which)
-{
-	switch (which) {
-	case V4L2_SUBDEV_FORMAT_TRY:
-		return v4l2_subdev_state_get_format(sd_state, pad);
-	case V4L2_SUBDEV_FORMAT_ACTIVE:
-		return &priv->fmt[pad];
-	default:
-		return NULL;
-	}
-}
-
 static int max9286_set_fmt(struct v4l2_subdev *sd,
-			   struct v4l2_subdev_state *sd_state,
+			   struct v4l2_subdev_state *state,
 			   struct v4l2_subdev_format *format)
 {
 	struct max9286_priv *priv = sd_to_max9286(sd);
-	struct v4l2_mbus_framefmt *cfg_fmt;
+	unsigned int pad;
 	unsigned int i;
 
+	/*
+	 * Disable setting format on the source pad: format is propagated
+	 * from the sinks.
+	 */
 	if (format->pad == MAX9286_SRC_PAD)
-		return -EINVAL;
+		return v4l2_subdev_get_fmt(sd, state, format);
 
 	/* Validate the format. */
 	for (i = 0; i < ARRAY_SIZE(max9286_formats); ++i) {
@@ -957,42 +948,16 @@ static int max9286_set_fmt(struct v4l2_subdev *sd,
 	if (i == ARRAY_SIZE(max9286_formats))
 		format->format.code = max9286_formats[0].code;
 
-	cfg_fmt = max9286_get_pad_format(priv, sd_state, format->pad,
-					 format->which);
-	if (!cfg_fmt)
-		return -EINVAL;
-
-	mutex_lock(&priv->mutex);
-	*cfg_fmt = format->format;
-	mutex_unlock(&priv->mutex);
-
-	return 0;
-}
-
-static int max9286_get_fmt(struct v4l2_subdev *sd,
-			   struct v4l2_subdev_state *sd_state,
-			   struct v4l2_subdev_format *format)
-{
-	struct max9286_priv *priv = sd_to_max9286(sd);
-	struct v4l2_mbus_framefmt *cfg_fmt;
-	unsigned int pad = format->pad;
+	*v4l2_subdev_state_get_format(state, format->pad) = format->format;
 
 	/*
-	 * Multiplexed Stream Support: Support link validation by returning the
-	 * format of the first bound link. All links must have the same format,
-	 * as we do not support mixing and matching of cameras connected to the
-	 * max9286.
+	 * Apply the same format on the source pad. As all links must have the
+	 * same format we do so only when the first source format is set.
 	 */
-	if (pad == MAX9286_SRC_PAD)
-		pad = __ffs(priv->bound_sources);
-
-	cfg_fmt = max9286_get_pad_format(priv, sd_state, pad, format->which);
-	if (!cfg_fmt)
-		return -EINVAL;
-
-	mutex_lock(&priv->mutex);
-	format->format = *cfg_fmt;
-	mutex_unlock(&priv->mutex);
+	pad = __ffs(priv->bound_sources);
+	if (pad == format->pad)
+		*v4l2_subdev_state_get_format(state,
+					      MAX9286_SRC_PAD) = format->format;
 
 	return 0;
 }
@@ -1003,7 +968,7 @@ static const struct v4l2_subdev_video_ops max9286_video_ops = {
 
 static const struct v4l2_subdev_pad_ops max9286_pad_ops = {
 	.enum_mbus_code = max9286_enum_mbus_code,
-	.get_fmt	= max9286_get_fmt,
+	.get_fmt	= v4l2_subdev_get_fmt,
 	.set_fmt	= max9286_set_fmt,
 	.get_frame_interval = max9286_get_frame_interval,
 	.set_frame_interval = max9286_set_frame_interval,
@@ -1025,26 +990,20 @@ static const struct v4l2_mbus_framefmt max9286_default_format = {
 	.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
 };
 
-static void max9286_init_format(struct v4l2_mbus_framefmt *fmt)
-{
-	*fmt = max9286_default_format;
-}
-
-static int max9286_open(struct v4l2_subdev *subdev, struct v4l2_subdev_fh *fh)
+static int max9286_init_state(struct v4l2_subdev *sd,
+			      struct v4l2_subdev_state *state)
 {
-	struct v4l2_mbus_framefmt *format;
-	unsigned int i;
+	for (unsigned int i = 0; i < MAX9286_N_SINKS; i++)
+		*v4l2_subdev_state_get_format(state, i) = max9286_default_format;
 
-	for (i = 0; i < MAX9286_N_SINKS; i++) {
-		format = v4l2_subdev_state_get_format(fh->state, i);
-		max9286_init_format(format);
-	}
+	*v4l2_subdev_state_get_format(state,
+				      MAX9286_SRC_PAD) = max9286_default_format;
 
 	return 0;
 }
 
 static const struct v4l2_subdev_internal_ops max9286_subdev_internal_ops = {
-	.open = max9286_open,
+	.init_state = max9286_init_state,
 };
 
 static const struct media_entity_operations max9286_media_ops = {
@@ -1079,10 +1038,6 @@ static int max9286_v4l2_register(struct max9286_priv *priv)
 	}
 
 	/* Configure V4L2 for the MAX9286 itself */
-
-	for (i = 0; i < MAX9286_N_SINKS; i++)
-		max9286_init_format(&priv->fmt[i]);
-
 	v4l2_i2c_subdev_init(&priv->sd, priv->client, &max9286_subdev_ops);
 	priv->sd.internal_ops = &max9286_subdev_internal_ops;
 	priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
@@ -1109,14 +1064,21 @@ static int max9286_v4l2_register(struct max9286_priv *priv)
 	if (ret)
 		goto err_async;
 
+	priv->sd.state_lock = priv->ctrls.lock;
+	ret = v4l2_subdev_init_finalize(&priv->sd);
+	if (ret)
+		goto err_async;
+
 	ret = v4l2_async_register_subdev(&priv->sd);
 	if (ret < 0) {
 		dev_err(dev, "Unable to register subdevice\n");
-		goto err_async;
+		goto err_subdev;
 	}
 
 	return 0;
 
+err_subdev:
+	v4l2_subdev_cleanup(&priv->sd);
 err_async:
 	v4l2_ctrl_handler_free(&priv->ctrls);
 	max9286_v4l2_notifier_unregister(priv);
@@ -1126,6 +1088,7 @@ static int max9286_v4l2_register(struct max9286_priv *priv)
 
 static void max9286_v4l2_unregister(struct max9286_priv *priv)
 {
+	v4l2_subdev_cleanup(&priv->sd);
 	v4l2_ctrl_handler_free(&priv->ctrls);
 	v4l2_async_unregister_subdev(&priv->sd);
 	max9286_v4l2_notifier_unregister(priv);
@@ -1629,8 +1592,6 @@ static int max9286_probe(struct i2c_client *client)
 	if (!priv)
 		return -ENOMEM;
 
-	mutex_init(&priv->mutex);
-
 	priv->client = client;
 
 	/* GPIO values default to high */
-- 
2.44.0

Re: [PATCH 10/11] media: max9286: Use the subdev active state

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:25PM +0200, Jacopo Mondi wrote:
> Use the subdev active state in the max9286 driver to store the
> image format.
> 
> Replace the .open() function call with the .init_state() one and
> simplify the set/get_pad_fmt() operations.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  drivers/media/i2c/max9286.c | 131 +++++++++++++-----------------------
>  1 file changed, 46 insertions(+), 85 deletions(-)
> 
> diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c
> index 5321238cad60..cc7ee35560fc 100644
> --- a/drivers/media/i2c/max9286.c
> +++ b/drivers/media/i2c/max9286.c
> @@ -19,7 +19,6 @@
>  #include <linux/i2c.h>
>  #include <linux/i2c-mux.h>
>  #include <linux/module.h>
> -#include <linux/mutex.h>
>  #include <linux/of_graph.h>
>  #include <linux/regulator/consumer.h>
>  #include <linux/slab.h>
> @@ -198,12 +197,8 @@ struct max9286_priv {
>  	struct v4l2_ctrl *pixelrate_ctrl;
>  	unsigned int pixelrate;
>  
> -	struct v4l2_mbus_framefmt fmt[MAX9286_N_SINKS];
>  	struct v4l2_fract interval;
>  
> -	/* Protects controls and fmt structures */
> -	struct mutex mutex;
> -
>  	unsigned int nsources;
>  	unsigned int source_mask;
>  	unsigned int route_mask;
> @@ -788,19 +783,22 @@ static void max9286_v4l2_notifier_unregister(struct max9286_priv *priv)
>  static int max9286_s_stream(struct v4l2_subdev *sd, int enable)
>  {
>  	struct max9286_priv *priv = sd_to_max9286(sd);
> +	struct v4l2_subdev_state *state;
>  	struct max9286_source *source;
>  	unsigned int i;
>  	bool sync = false;
> -	int ret;
> +	int ret = 0;
> +
> +	state = v4l2_subdev_lock_and_get_active_state(sd);
>  
>  	if (enable) {
>  		const struct v4l2_mbus_framefmt *format;
>  
>  		/*
> -		 * Get the format from the first used sink pad, as all sink
> -		 * formats must be identical.
> +		 * Get the format from the source pad, as all formats must be
> +		 * identical.
>  		 */
> -		format = &priv->fmt[__ffs(priv->bound_sources)];
> +		format = v4l2_subdev_state_get_format(state, MAX9286_SRC_PAD);
>  
>  		max9286_set_video_format(priv, format);
>  		max9286_set_fsync_period(priv);
> @@ -816,12 +814,12 @@ static int max9286_s_stream(struct v4l2_subdev *sd, int enable)
>  		for_each_source(priv, source) {
>  			ret = v4l2_subdev_call(source->sd, video, s_stream, 1);
>  			if (ret)
> -				return ret;
> +				goto unlock;
>  		}
>  
>  		ret = max9286_check_video_links(priv);
>  		if (ret)
> -			return ret;
> +			goto unlock;
>  
>  		/*
>  		 * Wait until frame synchronization is locked.
> @@ -842,7 +840,8 @@ static int max9286_s_stream(struct v4l2_subdev *sd, int enable)
>  		if (!sync) {
>  			dev_err(&priv->client->dev,
>  				"Failed to get frame synchronization\n");
> -			return -EXDEV; /* Invalid cross-device link */
> +			ret = -EXDEV; /* Invalid cross-device link */
> +			goto unlock;
>  		}
>  
>  		/*
> @@ -865,7 +864,10 @@ static int max9286_s_stream(struct v4l2_subdev *sd, int enable)
>  		max9286_i2c_mux_close(priv);
>  	}
>  
> -	return 0;
> +unlock:
> +	v4l2_subdev_unlock_state(state);
> +
> +	return ret;
>  }
>  
>  static int max9286_get_frame_interval(struct v4l2_subdev *sd,
> @@ -922,31 +924,20 @@ static int max9286_enum_mbus_code(struct v4l2_subdev *sd,
>  	return 0;
>  }
>  
> -static struct v4l2_mbus_framefmt *
> -max9286_get_pad_format(struct max9286_priv *priv,
> -		       struct v4l2_subdev_state *sd_state,
> -		       unsigned int pad, u32 which)
> -{
> -	switch (which) {
> -	case V4L2_SUBDEV_FORMAT_TRY:
> -		return v4l2_subdev_state_get_format(sd_state, pad);
> -	case V4L2_SUBDEV_FORMAT_ACTIVE:
> -		return &priv->fmt[pad];
> -	default:
> -		return NULL;
> -	}
> -}
> -
>  static int max9286_set_fmt(struct v4l2_subdev *sd,
> -			   struct v4l2_subdev_state *sd_state,
> +			   struct v4l2_subdev_state *state,
>  			   struct v4l2_subdev_format *format)
>  {
>  	struct max9286_priv *priv = sd_to_max9286(sd);
> -	struct v4l2_mbus_framefmt *cfg_fmt;
> +	unsigned int pad;
>  	unsigned int i;
>  
> +	/*
> +	 * Disable setting format on the source pad: format is propagated
> +	 * from the sinks.
> +	 */
>  	if (format->pad == MAX9286_SRC_PAD)
> -		return -EINVAL;
> +		return v4l2_subdev_get_fmt(sd, state, format);
>  
>  	/* Validate the format. */
>  	for (i = 0; i < ARRAY_SIZE(max9286_formats); ++i) {
> @@ -957,42 +948,16 @@ static int max9286_set_fmt(struct v4l2_subdev *sd,
>  	if (i == ARRAY_SIZE(max9286_formats))
>  		format->format.code = max9286_formats[0].code;
>  
> -	cfg_fmt = max9286_get_pad_format(priv, sd_state, format->pad,
> -					 format->which);
> -	if (!cfg_fmt)
> -		return -EINVAL;
> -
> -	mutex_lock(&priv->mutex);
> -	*cfg_fmt = format->format;
> -	mutex_unlock(&priv->mutex);
> -
> -	return 0;
> -}
> -
> -static int max9286_get_fmt(struct v4l2_subdev *sd,
> -			   struct v4l2_subdev_state *sd_state,
> -			   struct v4l2_subdev_format *format)
> -{
> -	struct max9286_priv *priv = sd_to_max9286(sd);
> -	struct v4l2_mbus_framefmt *cfg_fmt;
> -	unsigned int pad = format->pad;
> +	*v4l2_subdev_state_get_format(state, format->pad) = format->format;
>  
>  	/*
> -	 * Multiplexed Stream Support: Support link validation by returning the
> -	 * format of the first bound link. All links must have the same format,
> -	 * as we do not support mixing and matching of cameras connected to the
> -	 * max9286.
> +	 * Apply the same format on the source pad. As all links must have the
> +	 * same format we do so only when the first source format is set.
>  	 */
> -	if (pad == MAX9286_SRC_PAD)
> -		pad = __ffs(priv->bound_sources);
> -
> -	cfg_fmt = max9286_get_pad_format(priv, sd_state, pad, format->which);
> -	if (!cfg_fmt)
> -		return -EINVAL;
> -
> -	mutex_lock(&priv->mutex);
> -	format->format = *cfg_fmt;
> -	mutex_unlock(&priv->mutex);
> +	pad = __ffs(priv->bound_sources);
> +	if (pad == format->pad)
> +		*v4l2_subdev_state_get_format(state,
> +					      MAX9286_SRC_PAD) = format->format;

We don't enforce the requirement that formats on all sink pads must be
identical. Should we fix that ? One way to do so would be to propagate
formats to all the other sinks, in addition to propagating it to the
source.

>  
>  	return 0;
>  }
> @@ -1003,7 +968,7 @@ static const struct v4l2_subdev_video_ops max9286_video_ops = {
>  
>  static const struct v4l2_subdev_pad_ops max9286_pad_ops = {
>  	.enum_mbus_code = max9286_enum_mbus_code,
> -	.get_fmt	= max9286_get_fmt,
> +	.get_fmt	= v4l2_subdev_get_fmt,
>  	.set_fmt	= max9286_set_fmt,
>  	.get_frame_interval = max9286_get_frame_interval,
>  	.set_frame_interval = max9286_set_frame_interval,
> @@ -1025,26 +990,20 @@ static const struct v4l2_mbus_framefmt max9286_default_format = {
>  	.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
>  };
>  
> -static void max9286_init_format(struct v4l2_mbus_framefmt *fmt)
> -{
> -	*fmt = max9286_default_format;
> -}
> -
> -static int max9286_open(struct v4l2_subdev *subdev, struct v4l2_subdev_fh *fh)
> +static int max9286_init_state(struct v4l2_subdev *sd,
> +			      struct v4l2_subdev_state *state)
>  {
> -	struct v4l2_mbus_framefmt *format;
> -	unsigned int i;
> +	for (unsigned int i = 0; i < MAX9286_N_SINKS; i++)

How about looping over MAX9286_N_PADS instead, and dropping the line
below ?

> +		*v4l2_subdev_state_get_format(state, i) = max9286_default_format;
>  
> -	for (i = 0; i < MAX9286_N_SINKS; i++) {
> -		format = v4l2_subdev_state_get_format(fh->state, i);
> -		max9286_init_format(format);
> -	}
> +	*v4l2_subdev_state_get_format(state,
> +				      MAX9286_SRC_PAD) = max9286_default_format;
>  
>  	return 0;
>  }
>  
>  static const struct v4l2_subdev_internal_ops max9286_subdev_internal_ops = {
> -	.open = max9286_open,
> +	.init_state = max9286_init_state,
>  };
>  
>  static const struct media_entity_operations max9286_media_ops = {
> @@ -1079,10 +1038,6 @@ static int max9286_v4l2_register(struct max9286_priv *priv)
>  	}
>  
>  	/* Configure V4L2 for the MAX9286 itself */
> -
> -	for (i = 0; i < MAX9286_N_SINKS; i++)
> -		max9286_init_format(&priv->fmt[i]);
> -
>  	v4l2_i2c_subdev_init(&priv->sd, priv->client, &max9286_subdev_ops);
>  	priv->sd.internal_ops = &max9286_subdev_internal_ops;
>  	priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> @@ -1109,14 +1064,21 @@ static int max9286_v4l2_register(struct max9286_priv *priv)
>  	if (ret)
>  		goto err_async;
>  
> +	priv->sd.state_lock = priv->ctrls.lock;
> +	ret = v4l2_subdev_init_finalize(&priv->sd);
> +	if (ret)
> +		goto err_async;
> +
>  	ret = v4l2_async_register_subdev(&priv->sd);
>  	if (ret < 0) {
>  		dev_err(dev, "Unable to register subdevice\n");
> -		goto err_async;
> +		goto err_subdev;
>  	}
>  
>  	return 0;
>  
> +err_subdev:
> +	v4l2_subdev_cleanup(&priv->sd);
>  err_async:
>  	v4l2_ctrl_handler_free(&priv->ctrls);
>  	max9286_v4l2_notifier_unregister(priv);
> @@ -1126,6 +1088,7 @@ static int max9286_v4l2_register(struct max9286_priv *priv)
>  
>  static void max9286_v4l2_unregister(struct max9286_priv *priv)
>  {
> +	v4l2_subdev_cleanup(&priv->sd);
>  	v4l2_ctrl_handler_free(&priv->ctrls);
>  	v4l2_async_unregister_subdev(&priv->sd);
>  	max9286_v4l2_notifier_unregister(priv);
> @@ -1629,8 +1592,6 @@ static int max9286_probe(struct i2c_client *client)
>  	if (!priv)
>  		return -ENOMEM;
>  
> -	mutex_init(&priv->mutex);
> -
>  	priv->client = client;
>  
>  	/* GPIO values default to high */

-- 
Regards,

Laurent Pinchart

[PATCH 11/11] media: max9286: Use frame interval from subdev state

[Jacopo Mondi]

Use the frame interval stored in the subdev state instead of storing
a copy in the driver private structure.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/i2c/max9286.c | 52 +++++++++++++++++--------------------
 1 file changed, 24 insertions(+), 28 deletions(-)

diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c
index cc7ee35560fc..4ddbc247395f 100644
--- a/drivers/media/i2c/max9286.c
+++ b/drivers/media/i2c/max9286.c
@@ -197,8 +197,6 @@ struct max9286_priv {
 	struct v4l2_ctrl *pixelrate_ctrl;
 	unsigned int pixelrate;
 
-	struct v4l2_fract interval;
-
 	unsigned int nsources;
 	unsigned int source_mask;
 	unsigned int route_mask;
@@ -571,11 +569,14 @@ static void max9286_set_video_format(struct max9286_priv *priv,
 		      MAX9286_INVVS | MAX9286_HVSRC_D14);
 }
 
-static void max9286_set_fsync_period(struct max9286_priv *priv)
+static void max9286_set_fsync_period(struct max9286_priv *priv,
+				     struct v4l2_subdev_state *state)
 {
+	struct v4l2_fract *interval;
 	u32 fsync;
 
-	if (!priv->interval.numerator || !priv->interval.denominator) {
+	interval = v4l2_subdev_state_get_interval(state, MAX9286_SRC_PAD);
+	if (!interval->numerator || !interval->denominator) {
 		/*
 		 * Special case, a null interval enables automatic FRAMESYNC
 		 * mode. FRAMESYNC is taken from the slowest link.
@@ -591,8 +592,8 @@ static void max9286_set_fsync_period(struct max9286_priv *priv)
 	 * The FRAMESYNC generator is configured with a period expressed as a
 	 * number of PCLK periods.
 	 */
-	fsync = div_u64((u64)priv->pixelrate * priv->interval.numerator,
-			priv->interval.denominator);
+	fsync = div_u64((u64)priv->pixelrate * interval->numerator,
+			interval->denominator);
 
 	dev_dbg(&priv->client->dev, "fsync period %u (pclk %u)\n", fsync,
 		priv->pixelrate);
@@ -801,7 +802,7 @@ static int max9286_s_stream(struct v4l2_subdev *sd, int enable)
 		format = v4l2_subdev_state_get_format(state, MAX9286_SRC_PAD);
 
 		max9286_set_video_format(priv, format);
-		max9286_set_fsync_period(priv);
+		max9286_set_fsync_period(priv, state);
 
 		/*
 		 * The frame sync between cameras is transmitted across the
@@ -874,19 +875,11 @@ static int max9286_get_frame_interval(struct v4l2_subdev *sd,
 				      struct v4l2_subdev_state *sd_state,
 				      struct v4l2_subdev_frame_interval *interval)
 {
-	struct max9286_priv *priv = sd_to_max9286(sd);
-
-	/*
-	 * FIXME: Implement support for V4L2_SUBDEV_FORMAT_TRY, using the V4L2
-	 * subdev active state API.
-	 */
-	if (interval->which != V4L2_SUBDEV_FORMAT_ACTIVE)
-		return -EINVAL;
-
 	if (interval->pad != MAX9286_SRC_PAD)
 		return -EINVAL;
 
-	interval->interval = priv->interval;
+	interval->interval = *v4l2_subdev_state_get_interval(sd_state,
+							     interval->pad);
 
 	return 0;
 }
@@ -895,19 +888,11 @@ static int max9286_set_frame_interval(struct v4l2_subdev *sd,
 				      struct v4l2_subdev_state *sd_state,
 				      struct v4l2_subdev_frame_interval *interval)
 {
-	struct max9286_priv *priv = sd_to_max9286(sd);
-
-	/*
-	 * FIXME: Implement support for V4L2_SUBDEV_FORMAT_TRY, using the V4L2
-	 * subdev active state API.
-	 */
-	if (interval->which != V4L2_SUBDEV_FORMAT_ACTIVE)
-		return -EINVAL;
-
 	if (interval->pad != MAX9286_SRC_PAD)
 		return -EINVAL;
 
-	priv->interval = interval->interval;
+	*v4l2_subdev_state_get_interval(sd_state,
+					interval->pad) = interval->interval;
 
 	return 0;
 }
@@ -993,12 +978,24 @@ static const struct v4l2_mbus_framefmt max9286_default_format = {
 static int max9286_init_state(struct v4l2_subdev *sd,
 			      struct v4l2_subdev_state *state)
 {
+	struct v4l2_fract *interval;
+
 	for (unsigned int i = 0; i < MAX9286_N_SINKS; i++)
 		*v4l2_subdev_state_get_format(state, i) = max9286_default_format;
 
 	*v4l2_subdev_state_get_format(state,
 				      MAX9286_SRC_PAD) = max9286_default_format;
 
+	/*
+	 * Special case: a null interval enables automatic FRAMESYNC mode.
+	 *
+	 * FRAMESYNC is taken from the slowest link. See register 0x01
+	 * configuration.
+	 */
+	interval = v4l2_subdev_state_get_interval(state, MAX9286_SRC_PAD);
+	interval->numerator = 0;
+	interval->denominator = 0;
+
 	return 0;
 }
 
@@ -1145,7 +1142,6 @@ static int max9286_setup(struct max9286_priv *priv)
 	max9286_write(priv, 0x69, (0xf & ~priv->route_mask));
 
 	max9286_set_video_format(priv, &max9286_default_format);
-	max9286_set_fsync_period(priv);
 
 	cfg = max9286_read(priv, 0x1c);
 	if (cfg < 0)
-- 
2.44.0

Re: [PATCH 11/11] media: max9286: Use frame interval from subdev state

[Laurent Pinchart]

Hi Jacopo,

Thank you for the patch.

On Fri, May 03, 2024 at 05:51:26PM +0200, Jacopo Mondi wrote:
> Use the frame interval stored in the subdev state instead of storing
> a copy in the driver private structure.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  drivers/media/i2c/max9286.c | 52 +++++++++++++++++--------------------
>  1 file changed, 24 insertions(+), 28 deletions(-)
> 
> diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c
> index cc7ee35560fc..4ddbc247395f 100644
> --- a/drivers/media/i2c/max9286.c
> +++ b/drivers/media/i2c/max9286.c
> @@ -197,8 +197,6 @@ struct max9286_priv {
>  	struct v4l2_ctrl *pixelrate_ctrl;
>  	unsigned int pixelrate;
>  
> -	struct v4l2_fract interval;
> -
>  	unsigned int nsources;
>  	unsigned int source_mask;
>  	unsigned int route_mask;
> @@ -571,11 +569,14 @@ static void max9286_set_video_format(struct max9286_priv *priv,
>  		      MAX9286_INVVS | MAX9286_HVSRC_D14);
>  }
>  
> -static void max9286_set_fsync_period(struct max9286_priv *priv)
> +static void max9286_set_fsync_period(struct max9286_priv *priv,
> +				     struct v4l2_subdev_state *state)
>  {
> +	struct v4l2_fract *interval;

const

>  	u32 fsync;
>  
> -	if (!priv->interval.numerator || !priv->interval.denominator) {
> +	interval = v4l2_subdev_state_get_interval(state, MAX9286_SRC_PAD);
> +	if (!interval->numerator || !interval->denominator) {
>  		/*
>  		 * Special case, a null interval enables automatic FRAMESYNC
>  		 * mode. FRAMESYNC is taken from the slowest link.
> @@ -591,8 +592,8 @@ static void max9286_set_fsync_period(struct max9286_priv *priv)
>  	 * The FRAMESYNC generator is configured with a period expressed as a
>  	 * number of PCLK periods.
>  	 */
> -	fsync = div_u64((u64)priv->pixelrate * priv->interval.numerator,
> -			priv->interval.denominator);
> +	fsync = div_u64((u64)priv->pixelrate * interval->numerator,
> +			interval->denominator);
>  
>  	dev_dbg(&priv->client->dev, "fsync period %u (pclk %u)\n", fsync,
>  		priv->pixelrate);
> @@ -801,7 +802,7 @@ static int max9286_s_stream(struct v4l2_subdev *sd, int enable)
>  		format = v4l2_subdev_state_get_format(state, MAX9286_SRC_PAD);
>  
>  		max9286_set_video_format(priv, format);
> -		max9286_set_fsync_period(priv);
> +		max9286_set_fsync_period(priv, state);
>  
>  		/*
>  		 * The frame sync between cameras is transmitted across the
> @@ -874,19 +875,11 @@ static int max9286_get_frame_interval(struct v4l2_subdev *sd,
>  				      struct v4l2_subdev_state *sd_state,
>  				      struct v4l2_subdev_frame_interval *interval)
>  {
> -	struct max9286_priv *priv = sd_to_max9286(sd);
> -
> -	/*
> -	 * FIXME: Implement support for V4L2_SUBDEV_FORMAT_TRY, using the V4L2
> -	 * subdev active state API.
> -	 */
> -	if (interval->which != V4L2_SUBDEV_FORMAT_ACTIVE)
> -		return -EINVAL;
> -
>  	if (interval->pad != MAX9286_SRC_PAD)
>  		return -EINVAL;
>  
> -	interval->interval = priv->interval;
> +	interval->interval = *v4l2_subdev_state_get_interval(sd_state,
> +							     interval->pad);
>  
>  	return 0;
>  }
> @@ -895,19 +888,11 @@ static int max9286_set_frame_interval(struct v4l2_subdev *sd,
>  				      struct v4l2_subdev_state *sd_state,
>  				      struct v4l2_subdev_frame_interval *interval)
>  {
> -	struct max9286_priv *priv = sd_to_max9286(sd);
> -
> -	/*
> -	 * FIXME: Implement support for V4L2_SUBDEV_FORMAT_TRY, using the V4L2
> -	 * subdev active state API.
> -	 */
> -	if (interval->which != V4L2_SUBDEV_FORMAT_ACTIVE)
> -		return -EINVAL;
> -
>  	if (interval->pad != MAX9286_SRC_PAD)
>  		return -EINVAL;
>  
> -	priv->interval = interval->interval;
> +	*v4l2_subdev_state_get_interval(sd_state,
> +					interval->pad) = interval->interval;
>  
>  	return 0;
>  }
> @@ -993,12 +978,24 @@ static const struct v4l2_mbus_framefmt max9286_default_format = {
>  static int max9286_init_state(struct v4l2_subdev *sd,
>  			      struct v4l2_subdev_state *state)
>  {
> +	struct v4l2_fract *interval;
> +
>  	for (unsigned int i = 0; i < MAX9286_N_SINKS; i++)
>  		*v4l2_subdev_state_get_format(state, i) = max9286_default_format;
>  
>  	*v4l2_subdev_state_get_format(state,
>  				      MAX9286_SRC_PAD) = max9286_default_format;
>  
> +	/*
> +	 * Special case: a null interval enables automatic FRAMESYNC mode.
> +	 *
> +	 * FRAMESYNC is taken from the slowest link. See register 0x01
> +	 * configuration.
> +	 */
> +	interval = v4l2_subdev_state_get_interval(state, MAX9286_SRC_PAD);
> +	interval->numerator = 0;
> +	interval->denominator = 0;
> +
>  	return 0;
>  }
>  
> @@ -1145,7 +1142,6 @@ static int max9286_setup(struct max9286_priv *priv)
>  	max9286_write(priv, 0x69, (0xf & ~priv->route_mask));
>  
>  	max9286_set_video_format(priv, &max9286_default_format);
> -	max9286_set_fsync_period(priv);

The commit message should indicate why this line can be dropped.

>  
>  	cfg = max9286_read(priv, 0x1c);
>  	if (cfg < 0)

-- 
Regards,

Laurent Pinchart

Re: [PATCH 11/11] media: max9286: Use frame interval from subdev state

[Jacopo Mondi]

Hi Laurent

On Mon, May 06, 2024 at 12:36:11AM GMT, Laurent Pinchart wrote:
> Hi Jacopo,
>
> Thank you for the patch.
>
> On Fri, May 03, 2024 at 05:51:26PM +0200, Jacopo Mondi wrote:
> > Use the frame interval stored in the subdev state instead of storing
> > a copy in the driver private structure.
> >
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >  drivers/media/i2c/max9286.c | 52 +++++++++++++++++--------------------
> >  1 file changed, 24 insertions(+), 28 deletions(-)
> >
> > diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c
> > index cc7ee35560fc..4ddbc247395f 100644
> > --- a/drivers/media/i2c/max9286.c
> > +++ b/drivers/media/i2c/max9286.c
> > @@ -197,8 +197,6 @@ struct max9286_priv {
> >  	struct v4l2_ctrl *pixelrate_ctrl;
> >  	unsigned int pixelrate;
> >
> > -	struct v4l2_fract interval;
> > -
> >  	unsigned int nsources;
> >  	unsigned int source_mask;
> >  	unsigned int route_mask;
> > @@ -571,11 +569,14 @@ static void max9286_set_video_format(struct max9286_priv *priv,
> >  		      MAX9286_INVVS | MAX9286_HVSRC_D14);
> >  }
> >
> > -static void max9286_set_fsync_period(struct max9286_priv *priv)
> > +static void max9286_set_fsync_period(struct max9286_priv *priv,
> > +				     struct v4l2_subdev_state *state)
> >  {
> > +	struct v4l2_fract *interval;
>
> const
>

../drivers/media/i2c/max9286.c:993:29: error: assignment of member ‘numerator’ in read-only object
  993 |         interval->numerator = 0;

Re: [PATCH 11/11] media: max9286: Use frame interval from subdev state

[Laurent Pinchart]

On Mon, May 06, 2024 at 11:37:13AM +0200, Jacopo Mondi wrote:
> On Mon, May 06, 2024 at 12:36:11AM GMT, Laurent Pinchart wrote:
> > On Fri, May 03, 2024 at 05:51:26PM +0200, Jacopo Mondi wrote:
> > > Use the frame interval stored in the subdev state instead of storing
> > > a copy in the driver private structure.
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > ---
> > >  drivers/media/i2c/max9286.c | 52 +++++++++++++++++--------------------
> > >  1 file changed, 24 insertions(+), 28 deletions(-)
> > >
> > > diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c
> > > index cc7ee35560fc..4ddbc247395f 100644
> > > --- a/drivers/media/i2c/max9286.c
> > > +++ b/drivers/media/i2c/max9286.c
> > > @@ -197,8 +197,6 @@ struct max9286_priv {
> > >  	struct v4l2_ctrl *pixelrate_ctrl;
> > >  	unsigned int pixelrate;
> > >
> > > -	struct v4l2_fract interval;
> > > -
> > >  	unsigned int nsources;
> > >  	unsigned int source_mask;
> > >  	unsigned int route_mask;
> > > @@ -571,11 +569,14 @@ static void max9286_set_video_format(struct max9286_priv *priv,
> > >  		      MAX9286_INVVS | MAX9286_HVSRC_D14);
> > >  }
> > >
> > > -static void max9286_set_fsync_period(struct max9286_priv *priv)
> > > +static void max9286_set_fsync_period(struct max9286_priv *priv,
> > > +				     struct v4l2_subdev_state *state)
> > >  {
> > > +	struct v4l2_fract *interval;
> >
> > const
> 
> ../drivers/media/i2c/max9286.c:993:29: error: assignment of member ‘numerator’ in read-only object
>   993 |         interval->numerator = 0;

That's the .init_state() implementation. There you can't make it const,
but in max9286_set_fsync_period() it should be fine.

-- 
Regards,

Laurent Pinchart