[PATCH] media: staging: max96712: Store format in subdev active state

[PATCH] media: staging: max96712: Store format in subdev active state

[Niklas Söderlund]

Create and store the subdevice format in the subdevices active state.
This change do not have a huge effect on the driver as it do not yet
support changing the format.

Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
Hello,

This change is mostly to align all driver used in the R-Car VIN pipeline
to use the same set of API and to make it easier to extend them in
tandem going forward.

This should be seen as a compliment to Jacopo's larger work in "[PATCH
v3 00/11] media: renesas: rcar-csi2: Use the subdev active state" which
updates other drivers in the VIN pipeline to use the active state.
---
 drivers/staging/media/max96712/max96712.c | 36 +++++++++++++++++------
 1 file changed, 27 insertions(+), 9 deletions(-)

diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c
index c44145284aa1..2da65ccd2c57 100644
--- a/drivers/staging/media/max96712/max96712.c
+++ b/drivers/staging/media/max96712/max96712.c
@@ -242,21 +242,34 @@ static const struct v4l2_subdev_video_ops max96712_video_ops = {
 	.s_stream = max96712_s_stream,
 };
 
-static int max96712_get_pad_format(struct v4l2_subdev *sd,
-				   struct v4l2_subdev_state *sd_state,
-				   struct v4l2_subdev_format *format)
+static int max96712_init_state(struct v4l2_subdev *sd,
+			       struct v4l2_subdev_state *state)
 {
-	format->format.width = 1920;
-	format->format.height = 1080;
-	format->format.code = MEDIA_BUS_FMT_RGB888_1X24;
-	format->format.field = V4L2_FIELD_NONE;
+	static const struct v4l2_mbus_framefmt default_fmt = {
+		.width          = 1920,
+		.height         = 1080,
+		.code           = MEDIA_BUS_FMT_RGB888_1X24,
+		.colorspace     = V4L2_COLORSPACE_SRGB,
+		.field          = V4L2_FIELD_NONE,
+		.ycbcr_enc      = V4L2_YCBCR_ENC_DEFAULT,
+		.quantization   = V4L2_QUANTIZATION_DEFAULT,
+		.xfer_func      = V4L2_XFER_FUNC_DEFAULT,
+	};
+	struct v4l2_mbus_framefmt *fmt;
+
+	fmt = v4l2_subdev_state_get_format(state, 0);
+	*fmt = default_fmt;
 
 	return 0;
 }
 
+static const struct v4l2_subdev_internal_ops max96712_internal_ops = {
+	.init_state = max96712_init_state,
+};
+
 static const struct v4l2_subdev_pad_ops max96712_pad_ops = {
-	.get_fmt = max96712_get_pad_format,
-	.set_fmt = max96712_get_pad_format,
+	.get_fmt = v4l2_subdev_get_fmt,
+	.set_fmt = v4l2_subdev_get_fmt,
 };
 
 static const struct v4l2_subdev_ops max96712_subdev_ops = {
@@ -293,6 +306,7 @@ static int max96712_v4l2_register(struct max96712_priv *priv)
 	long pixel_rate;
 	int ret;
 
+	priv->sd.internal_ops = &max96712_internal_ops;
 	v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops);
 	priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
 	priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
@@ -324,6 +338,10 @@ static int max96712_v4l2_register(struct max96712_priv *priv)
 
 	v4l2_set_subdevdata(&priv->sd, priv);
 
+	ret = v4l2_subdev_init_finalize(&priv->sd);
+	if (ret)
+		goto error;
+
 	ret = v4l2_async_register_subdev(&priv->sd);
 	if (ret < 0) {
 		dev_err(&priv->client->dev, "Unable to register subdevice\n");
-- 
2.45.0

Re: [PATCH] media: staging: max96712: Store format in subdev active state

[Laurent Pinchart]

Hi Niklas,

Thank you for the patch.

On Fri, May 10, 2024 at 12:03:47PM +0200, Niklas Söderlund wrote:
> Create and store the subdevice format in the subdevices active state.
> This change do not have a huge effect on the driver as it do not yet
> support changing the format.
> 
> Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> ---
> Hello,
> 
> This change is mostly to align all driver used in the R-Car VIN pipeline
> to use the same set of API and to make it easier to extend them in
> tandem going forward.
> 
> This should be seen as a compliment to Jacopo's larger work in "[PATCH
> v3 00/11] media: renesas: rcar-csi2: Use the subdev active state" which
> updates other drivers in the VIN pipeline to use the active state.
> ---
>  drivers/staging/media/max96712/max96712.c | 36 +++++++++++++++++------
>  1 file changed, 27 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c
> index c44145284aa1..2da65ccd2c57 100644
> --- a/drivers/staging/media/max96712/max96712.c
> +++ b/drivers/staging/media/max96712/max96712.c
> @@ -242,21 +242,34 @@ static const struct v4l2_subdev_video_ops max96712_video_ops = {
>  	.s_stream = max96712_s_stream,
>  };
>  
> -static int max96712_get_pad_format(struct v4l2_subdev *sd,
> -				   struct v4l2_subdev_state *sd_state,
> -				   struct v4l2_subdev_format *format)
> +static int max96712_init_state(struct v4l2_subdev *sd,
> +			       struct v4l2_subdev_state *state)
>  {
> -	format->format.width = 1920;
> -	format->format.height = 1080;
> -	format->format.code = MEDIA_BUS_FMT_RGB888_1X24;
> -	format->format.field = V4L2_FIELD_NONE;
> +	static const struct v4l2_mbus_framefmt default_fmt = {
> +		.width          = 1920,
> +		.height         = 1080,
> +		.code           = MEDIA_BUS_FMT_RGB888_1X24,
> +		.colorspace     = V4L2_COLORSPACE_SRGB,
> +		.field          = V4L2_FIELD_NONE,
> +		.ycbcr_enc      = V4L2_YCBCR_ENC_DEFAULT,
> +		.quantization   = V4L2_QUANTIZATION_DEFAULT,
> +		.xfer_func      = V4L2_XFER_FUNC_DEFAULT,
> +	};
> +	struct v4l2_mbus_framefmt *fmt;
> +
> +	fmt = v4l2_subdev_state_get_format(state, 0);
> +	*fmt = default_fmt;
>  
>  	return 0;
>  }
>  
> +static const struct v4l2_subdev_internal_ops max96712_internal_ops = {
> +	.init_state = max96712_init_state,
> +};
> +
>  static const struct v4l2_subdev_pad_ops max96712_pad_ops = {
> -	.get_fmt = max96712_get_pad_format,
> -	.set_fmt = max96712_get_pad_format,
> +	.get_fmt = v4l2_subdev_get_fmt,
> +	.set_fmt = v4l2_subdev_get_fmt,
>  };
>  
>  static const struct v4l2_subdev_ops max96712_subdev_ops = {
> @@ -293,6 +306,7 @@ static int max96712_v4l2_register(struct max96712_priv *priv)
>  	long pixel_rate;
>  	int ret;
>  
> +	priv->sd.internal_ops = &max96712_internal_ops;
>  	v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops);
>  	priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>  	priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> @@ -324,6 +338,10 @@ static int max96712_v4l2_register(struct max96712_priv *priv)
>  
>  	v4l2_set_subdevdata(&priv->sd, priv);
>  

I think it would be nice to already use the control handler lock as the
active state lock. Apart from that, the patch looks good to me.

> +	ret = v4l2_subdev_init_finalize(&priv->sd);
> +	if (ret)
> +		goto error;
> +
>  	ret = v4l2_async_register_subdev(&priv->sd);
>  	if (ret < 0) {
>  		dev_err(&priv->client->dev, "Unable to register subdevice\n");

-- 
Regards,

Laurent Pinchart

Re: [PATCH] media: staging: max96712: Store format in subdev active state

[Niklas Söderlund]

Hello Laurent,

Thanks for your feedback.

On 2024-05-10 13:13:40 +0300, Laurent Pinchart wrote:
> Hi Niklas,
> 
> Thank you for the patch.
> 
> On Fri, May 10, 2024 at 12:03:47PM +0200, Niklas Söderlund wrote:
> > Create and store the subdevice format in the subdevices active state.
> > This change do not have a huge effect on the driver as it do not yet
> > support changing the format.
> > 
> > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > ---
> > Hello,
> > 
> > This change is mostly to align all driver used in the R-Car VIN pipeline
> > to use the same set of API and to make it easier to extend them in
> > tandem going forward.
> > 
> > This should be seen as a compliment to Jacopo's larger work in "[PATCH
> > v3 00/11] media: renesas: rcar-csi2: Use the subdev active state" which
> > updates other drivers in the VIN pipeline to use the active state.
> > ---
> >  drivers/staging/media/max96712/max96712.c | 36 +++++++++++++++++------
> >  1 file changed, 27 insertions(+), 9 deletions(-)
> > 
> > diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c
> > index c44145284aa1..2da65ccd2c57 100644
> > --- a/drivers/staging/media/max96712/max96712.c
> > +++ b/drivers/staging/media/max96712/max96712.c
> > @@ -242,21 +242,34 @@ static const struct v4l2_subdev_video_ops max96712_video_ops = {
> >  	.s_stream = max96712_s_stream,
> >  };
> >  
> > -static int max96712_get_pad_format(struct v4l2_subdev *sd,
> > -				   struct v4l2_subdev_state *sd_state,
> > -				   struct v4l2_subdev_format *format)
> > +static int max96712_init_state(struct v4l2_subdev *sd,
> > +			       struct v4l2_subdev_state *state)
> >  {
> > -	format->format.width = 1920;
> > -	format->format.height = 1080;
> > -	format->format.code = MEDIA_BUS_FMT_RGB888_1X24;
> > -	format->format.field = V4L2_FIELD_NONE;
> > +	static const struct v4l2_mbus_framefmt default_fmt = {
> > +		.width          = 1920,
> > +		.height         = 1080,
> > +		.code           = MEDIA_BUS_FMT_RGB888_1X24,
> > +		.colorspace     = V4L2_COLORSPACE_SRGB,
> > +		.field          = V4L2_FIELD_NONE,
> > +		.ycbcr_enc      = V4L2_YCBCR_ENC_DEFAULT,
> > +		.quantization   = V4L2_QUANTIZATION_DEFAULT,
> > +		.xfer_func      = V4L2_XFER_FUNC_DEFAULT,
> > +	};
> > +	struct v4l2_mbus_framefmt *fmt;
> > +
> > +	fmt = v4l2_subdev_state_get_format(state, 0);
> > +	*fmt = default_fmt;
> >  
> >  	return 0;
> >  }
> >  
> > +static const struct v4l2_subdev_internal_ops max96712_internal_ops = {
> > +	.init_state = max96712_init_state,
> > +};
> > +
> >  static const struct v4l2_subdev_pad_ops max96712_pad_ops = {
> > -	.get_fmt = max96712_get_pad_format,
> > -	.set_fmt = max96712_get_pad_format,
> > +	.get_fmt = v4l2_subdev_get_fmt,
> > +	.set_fmt = v4l2_subdev_get_fmt,
> >  };
> >  
> >  static const struct v4l2_subdev_ops max96712_subdev_ops = {
> > @@ -293,6 +306,7 @@ static int max96712_v4l2_register(struct max96712_priv *priv)
> >  	long pixel_rate;
> >  	int ret;
> >  
> > +	priv->sd.internal_ops = &max96712_internal_ops;
> >  	v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops);
> >  	priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> >  	priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> > @@ -324,6 +338,10 @@ static int max96712_v4l2_register(struct max96712_priv *priv)
> >  
> >  	v4l2_set_subdevdata(&priv->sd, priv);
> >  
> 
> I think it would be nice to already use the control handler lock as the
> active state lock. Apart from that, the patch looks good to me.

I also worked on this to learn a bit more about the active state API. I 
assume what you are asking for here is ?

    priv->sd.state_lock = priv->ctrl_handler.lock;

What is the rational for using the control handler lock over the default 
state lock? I checked Jacopo's series when working on this and there he 
uses no lock for the R-Car CSI-2 driver and a private driver global 
state lock for ADV748x.

> 
> > +	ret = v4l2_subdev_init_finalize(&priv->sd);
> > +	if (ret)
> > +		goto error;
> > +
> >  	ret = v4l2_async_register_subdev(&priv->sd);
> >  	if (ret < 0) {
> >  		dev_err(&priv->client->dev, "Unable to register subdevice\n");
> 
> -- 
> Regards,
> 
> Laurent Pinchart

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH] media: staging: max96712: Store format in subdev active state

[Laurent Pinchart]

Hi Niklas,

On Fri, May 10, 2024 at 12:36:56PM +0200, Niklas Söderlund wrote:
> On 2024-05-10 13:13:40 +0300, Laurent Pinchart wrote:
> > On Fri, May 10, 2024 at 12:03:47PM +0200, Niklas Söderlund wrote:
> > > Create and store the subdevice format in the subdevices active state.
> > > This change do not have a huge effect on the driver as it do not yet
> > > support changing the format.
> > > 
> > > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > ---
> > > Hello,
> > > 
> > > This change is mostly to align all driver used in the R-Car VIN pipeline
> > > to use the same set of API and to make it easier to extend them in
> > > tandem going forward.
> > > 
> > > This should be seen as a compliment to Jacopo's larger work in "[PATCH
> > > v3 00/11] media: renesas: rcar-csi2: Use the subdev active state" which
> > > updates other drivers in the VIN pipeline to use the active state.
> > > ---
> > >  drivers/staging/media/max96712/max96712.c | 36 +++++++++++++++++------
> > >  1 file changed, 27 insertions(+), 9 deletions(-)
> > > 
> > > diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c
> > > index c44145284aa1..2da65ccd2c57 100644
> > > --- a/drivers/staging/media/max96712/max96712.c
> > > +++ b/drivers/staging/media/max96712/max96712.c
> > > @@ -242,21 +242,34 @@ static const struct v4l2_subdev_video_ops max96712_video_ops = {
> > >  	.s_stream = max96712_s_stream,
> > >  };
> > >  
> > > -static int max96712_get_pad_format(struct v4l2_subdev *sd,
> > > -				   struct v4l2_subdev_state *sd_state,
> > > -				   struct v4l2_subdev_format *format)
> > > +static int max96712_init_state(struct v4l2_subdev *sd,
> > > +			       struct v4l2_subdev_state *state)
> > >  {
> > > -	format->format.width = 1920;
> > > -	format->format.height = 1080;
> > > -	format->format.code = MEDIA_BUS_FMT_RGB888_1X24;
> > > -	format->format.field = V4L2_FIELD_NONE;
> > > +	static const struct v4l2_mbus_framefmt default_fmt = {
> > > +		.width          = 1920,
> > > +		.height         = 1080,
> > > +		.code           = MEDIA_BUS_FMT_RGB888_1X24,
> > > +		.colorspace     = V4L2_COLORSPACE_SRGB,
> > > +		.field          = V4L2_FIELD_NONE,
> > > +		.ycbcr_enc      = V4L2_YCBCR_ENC_DEFAULT,
> > > +		.quantization   = V4L2_QUANTIZATION_DEFAULT,
> > > +		.xfer_func      = V4L2_XFER_FUNC_DEFAULT,
> > > +	};
> > > +	struct v4l2_mbus_framefmt *fmt;
> > > +
> > > +	fmt = v4l2_subdev_state_get_format(state, 0);
> > > +	*fmt = default_fmt;
> > >  
> > >  	return 0;
> > >  }
> > >  
> > > +static const struct v4l2_subdev_internal_ops max96712_internal_ops = {
> > > +	.init_state = max96712_init_state,
> > > +};
> > > +
> > >  static const struct v4l2_subdev_pad_ops max96712_pad_ops = {
> > > -	.get_fmt = max96712_get_pad_format,
> > > -	.set_fmt = max96712_get_pad_format,
> > > +	.get_fmt = v4l2_subdev_get_fmt,
> > > +	.set_fmt = v4l2_subdev_get_fmt,
> > >  };
> > >  
> > >  static const struct v4l2_subdev_ops max96712_subdev_ops = {
> > > @@ -293,6 +306,7 @@ static int max96712_v4l2_register(struct max96712_priv *priv)
> > >  	long pixel_rate;
> > >  	int ret;
> > >  
> > > +	priv->sd.internal_ops = &max96712_internal_ops;
> > >  	v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops);
> > >  	priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> > >  	priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> > > @@ -324,6 +338,10 @@ static int max96712_v4l2_register(struct max96712_priv *priv)
> > >  
> > >  	v4l2_set_subdevdata(&priv->sd, priv);
> > >  
> > 
> > I think it would be nice to already use the control handler lock as the
> > active state lock. Apart from that, the patch looks good to me.
> 
> I also worked on this to learn a bit more about the active state API. I 
> assume what you are asking for here is ?
> 
>     priv->sd.state_lock = priv->ctrl_handler.lock;

That's right.

> What is the rational for using the control handler lock over the default 
> state lock?

When there are interactions between controls and formats, using separate
locks can lead to AB/BA deadlocks if you need to access controls from a
context where the state is locked (e.g. calling
v4l2_ctrl_handler_setup() at stream enable time), and access the active
state from a context where the control handler is locked (e.g. accessing
the state in .s_ctrl()). Using the same lock helps there, you can then
call __v4l2_ctrl_handler_setup() at stream enable time, and get the
state without locking it in .s_ctrl().

At some point in the not too distant future I would like to move storage
of controls to the state. I don't know how painful that will be though,
the control framework has grown to be a very complex beast, with lots of
features that are not useful for MC-based drivers (such as control
handler inheritance). I'm worried that we'll need a full rewrite...
We'll see :-)

> I checked Jacopo's series when working on this and there he 
> uses no lock for the R-Car CSI-2 driver and a private driver global 
> state lock for ADV748x.

For the CSI-2 RX driver, that's because it doesn't have a control
handler. For the ADV748x driver, that's because the control handler is
configured to use the device-wide mutex.

> > > +	ret = v4l2_subdev_init_finalize(&priv->sd);
> > > +	if (ret)
> > > +		goto error;
> > > +
> > >  	ret = v4l2_async_register_subdev(&priv->sd);
> > >  	if (ret < 0) {
> > >  		dev_err(&priv->client->dev, "Unable to register subdevice\n");

-- 
Regards,

Laurent Pinchart

Re: [PATCH] media: staging: max96712: Store format in subdev active state

[Niklas Söderlund]

Hi Laurent,

On 2024-05-10 18:27:24 +0300, Laurent Pinchart wrote:
> Hi Niklas,
> 
> On Fri, May 10, 2024 at 12:36:56PM +0200, Niklas Söderlund wrote:
> > On 2024-05-10 13:13:40 +0300, Laurent Pinchart wrote:
> > > On Fri, May 10, 2024 at 12:03:47PM +0200, Niklas Söderlund wrote:
> > > > Create and store the subdevice format in the subdevices active state.
> > > > This change do not have a huge effect on the driver as it do not yet
> > > > support changing the format.
> > > > 
> > > > Signed-off-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > > ---
> > > > Hello,
> > > > 
> > > > This change is mostly to align all driver used in the R-Car VIN pipeline
> > > > to use the same set of API and to make it easier to extend them in
> > > > tandem going forward.
> > > > 
> > > > This should be seen as a compliment to Jacopo's larger work in "[PATCH
> > > > v3 00/11] media: renesas: rcar-csi2: Use the subdev active state" which
> > > > updates other drivers in the VIN pipeline to use the active state.
> > > > ---
> > > >  drivers/staging/media/max96712/max96712.c | 36 +++++++++++++++++------
> > > >  1 file changed, 27 insertions(+), 9 deletions(-)
> > > > 
> > > > diff --git a/drivers/staging/media/max96712/max96712.c b/drivers/staging/media/max96712/max96712.c
> > > > index c44145284aa1..2da65ccd2c57 100644
> > > > --- a/drivers/staging/media/max96712/max96712.c
> > > > +++ b/drivers/staging/media/max96712/max96712.c
> > > > @@ -242,21 +242,34 @@ static const struct v4l2_subdev_video_ops max96712_video_ops = {
> > > >  	.s_stream = max96712_s_stream,
> > > >  };
> > > >  
> > > > -static int max96712_get_pad_format(struct v4l2_subdev *sd,
> > > > -				   struct v4l2_subdev_state *sd_state,
> > > > -				   struct v4l2_subdev_format *format)
> > > > +static int max96712_init_state(struct v4l2_subdev *sd,
> > > > +			       struct v4l2_subdev_state *state)
> > > >  {
> > > > -	format->format.width = 1920;
> > > > -	format->format.height = 1080;
> > > > -	format->format.code = MEDIA_BUS_FMT_RGB888_1X24;
> > > > -	format->format.field = V4L2_FIELD_NONE;
> > > > +	static const struct v4l2_mbus_framefmt default_fmt = {
> > > > +		.width          = 1920,
> > > > +		.height         = 1080,
> > > > +		.code           = MEDIA_BUS_FMT_RGB888_1X24,
> > > > +		.colorspace     = V4L2_COLORSPACE_SRGB,
> > > > +		.field          = V4L2_FIELD_NONE,
> > > > +		.ycbcr_enc      = V4L2_YCBCR_ENC_DEFAULT,
> > > > +		.quantization   = V4L2_QUANTIZATION_DEFAULT,
> > > > +		.xfer_func      = V4L2_XFER_FUNC_DEFAULT,
> > > > +	};
> > > > +	struct v4l2_mbus_framefmt *fmt;
> > > > +
> > > > +	fmt = v4l2_subdev_state_get_format(state, 0);
> > > > +	*fmt = default_fmt;
> > > >  
> > > >  	return 0;
> > > >  }
> > > >  
> > > > +static const struct v4l2_subdev_internal_ops max96712_internal_ops = {
> > > > +	.init_state = max96712_init_state,
> > > > +};
> > > > +
> > > >  static const struct v4l2_subdev_pad_ops max96712_pad_ops = {
> > > > -	.get_fmt = max96712_get_pad_format,
> > > > -	.set_fmt = max96712_get_pad_format,
> > > > +	.get_fmt = v4l2_subdev_get_fmt,
> > > > +	.set_fmt = v4l2_subdev_get_fmt,
> > > >  };
> > > >  
> > > >  static const struct v4l2_subdev_ops max96712_subdev_ops = {
> > > > @@ -293,6 +306,7 @@ static int max96712_v4l2_register(struct max96712_priv *priv)
> > > >  	long pixel_rate;
> > > >  	int ret;
> > > >  
> > > > +	priv->sd.internal_ops = &max96712_internal_ops;
> > > >  	v4l2_i2c_subdev_init(&priv->sd, priv->client, &max96712_subdev_ops);
> > > >  	priv->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> > > >  	priv->sd.entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> > > > @@ -324,6 +338,10 @@ static int max96712_v4l2_register(struct max96712_priv *priv)
> > > >  
> > > >  	v4l2_set_subdevdata(&priv->sd, priv);
> > > >  
> > > 
> > > I think it would be nice to already use the control handler lock as the
> > > active state lock. Apart from that, the patch looks good to me.
> > 
> > I also worked on this to learn a bit more about the active state API. I 
> > assume what you are asking for here is ?
> > 
> >     priv->sd.state_lock = priv->ctrl_handler.lock;
> 
> That's right.

I will send a v2 of this patch using the controller lock.

> 
> > What is the rational for using the control handler lock over the default 
> > state lock?
> 
> When there are interactions between controls and formats, using separate
> locks can lead to AB/BA deadlocks if you need to access controls from a
> context where the state is locked (e.g. calling
> v4l2_ctrl_handler_setup() at stream enable time), and access the active
> state from a context where the control handler is locked (e.g. accessing
> the state in .s_ctrl()). Using the same lock helps there, you can then
> call __v4l2_ctrl_handler_setup() at stream enable time, and get the
> state without locking it in .s_ctrl().
> 
> At some point in the not too distant future I would like to move storage
> of controls to the state. I don't know how painful that will be though,
> the control framework has grown to be a very complex beast, with lots of
> features that are not useful for MC-based drivers (such as control
> handler inheritance). I'm worried that we'll need a full rewrite...
> We'll see :-)
> 
> > I checked Jacopo's series when working on this and there he 
> > uses no lock for the R-Car CSI-2 driver and a private driver global 
> > state lock for ADV748x.
> 
> For the CSI-2 RX driver, that's because it doesn't have a control
> handler. For the ADV748x driver, that's because the control handler is
> configured to use the device-wide mutex.

Thanks for this through description of why using the control handler 
lock for the stat is a good idea. I really like your idea of trying to 
make the control framework less complex for MC-based drivers in the 
future.

> 
> > > > +	ret = v4l2_subdev_init_finalize(&priv->sd);
> > > > +	if (ret)
> > > > +		goto error;
> > > > +
> > > >  	ret = v4l2_async_register_subdev(&priv->sd);
> > > >  	if (ret < 0) {
> > > >  		dev_err(&priv->client->dev, "Unable to register subdevice\n");
> 
> -- 
> Regards,
> 
> Laurent Pinchart

-- 
Kind Regards,
Niklas Söderlund