[PATCH v5 00/10] Metadata series preparation

[PATCH v5 00/10] Metadata series preparation

[Sakari Ailus]

Hi folks,

This smallish set contains patches that prepare for merging the metadata
series.

There are simple cleanups but also a noteworthy change: the addition of
the new struct v4l2_subdev_client_info struct to the set_fmt,
get_selection and set_selection pad operation arguments.

The series now begins with a set of imx219 patches, fixes and cleanups
accrued during the conversion of the imx219 driver to use the common raw
sensor model. I wonder if all these should be backported.

The new struct v4l2_subdev_client_info enables passing around file handler
specific client capability information, which is used to differentiate
UAPI between existing users and those that are aware of the new common raw
sensor model. This is effectively required if we want to add support for
the new model to existing raw sensor drivers: the new model is in a direct
conflict with how the imx219 driver in particular worked before the common
raw sensor model.

There still needs to be a single driver internal state, the different
UAPIs simply offer a different view to that state. The get_fmt() pad op
does not have the client_info argument as the get_fmt() is called by other
drivers inside the kernel so there's no concept of "client info" there nor
the format returned by the callback should have depended on it.

The Coccinelle spatch used to generate most of the driver changes in the
last patch is below, as is a Perl script to generate another Coccinelle
patch doing the rest of the job:

caps.cocci
------------------8<----------------------
@ fmt @
identifier a1, a2, a3;
identifier fn =~ ".*_set.*";

@@

  fn(struct v4l2_subdev *a1,
+ const struct v4l2_subdev_client_info *ci,
  struct v4l2_subdev_state *a2, struct v4l2_subdev_format *a3) {...}

@ sel @
identifier a1, a2, a3;
identifier fn2;

@@

  fn2(struct v4l2_subdev *a1,
+ const struct v4l2_subdev_client_info *ci,
  struct v4l2_subdev_state *a2, struct v4l2_subdev_selection *a3) {...}

@ fmt_call @
expression a1, a2, a3;
identifier fmt.fn;
@@

  fn(a1
+ , NULL
  , a2, a3);

@ fmt_call2 @
expression a1, a2, a3;
identifier fmt.fn;
@@

  return fn(a1
+ , NULL
  , a2, a3);


@ fmt_call3 @
expression ret, a1, a2, a3;
identifier fmt.fn;

@@

  ret = fn(a1
+ , NULL
  , a2, a3);

@ fmt_call4 @
expression a1, a2, a3;
identifier sel.fn2;
@@

  fn2(a1
+ , NULL
  , a2, a3);

@ fmt_call5 @
expression a1, a2, a3;
identifier sel.fn2;
@@

  return fn2(a1
+ , NULL
  , a2, a3);


@ fmt_call6 @
expression ret, a1, a2, a3;
identifier sel.fn2;

@@

  ret = fn2(a1
+ , NULL
  , a2, a3);

------------------8<----------------------

mk-caps2.pl
------------------8<----------------------
#!/usr/bin/perl -w

my @prefix = (
	"(void)",
	"",
	"ret = ",
	"return "
);

my $state_args = [
	[ "sd", "state", "fmt" ]
];

my $pad_active_args = [
	#[ "sd", "pad", "get_fmt", "fmt" ],
	[ "sd", "pad", "set_fmt", "fmt" ],
	[ "sd", "pad", "get_selection", "fmt" ],
	[ "sd", "pad", "set_selection", "fmt" ],
];

my $pad_args = [
	#[ "sd", "pad", "get_fmt", "state", "fmt" ],
	[ "sd", "pad", "set_fmt", "state", "fmt" ],
	[ "sd", "pad", "get_selection", "state", "fmt" ],
	[ "sd", "pad", "set_selection", "state", "fmt" ],
];

my $group_args = [
	#[ "dev", "group", "pad", "get_fmt", "state", "fmt" ],
	[ "dev", "group", "pad", "set_fmt", "state", "fmt" ],
	[ "dev", "group", "pad", "get_selection", "state", "fmt" ],
	[ "dev", "group", "pad", "set_selection", "state", "fmt" ],
];

my %funcs = (
	# "function_name" => [ arg ref, new arg position from the end ],
	"v4l2_subdev_call" => { args => $pad_args, pos => 1 },
	"v4l2_device_call_all" => { args => $group_args, pos => 1 },
	#"v4l2_subdev_get_fmt" => { args => $state_args, pos => 1 },
	"sensor_call" => { args => $pad_args, pos => 1 },
	"saa_call_all" => { args => $pad_args, pos => 1 },
	"call_all" => { args => $pad_args, pos => 1 },
	"v4l2_subdev_call_state_active" => { args => $pad_active_args, newname => "v4l2_subdev_call_ci_state_active" },
);

my $rule;

foreach my $p (@prefix) {
	foreach my $f (keys %funcs) {
		my $aref = $funcs{$f};

		foreach my $i (@{$aref->{args}}) {
			my $newname = $aref->{newname};

			print "@ rule" . $rule++ . " @\n";
			print "symbol $f;\n";
			print map {
				((/_/ || /^pad$/) ?
				"symbol" : "expression") . " $_;\n";
			} @$i;
			my @pre = @$i;
			my @post = splice @pre, $#pre - ($aref->{pos} // 0);
			print "identifier ret;\n" if $p =~ "ret =";
			print "@@\n";
			if (! defined $newname) {
				print "  $p$f(" . join(", ", @pre) . "\n";
				print "+ , NULL\n";
				print "  , " . join(", ", @post) . ");\n";
			} else {
				print "  $p\n";
				print "- $f(\n";
				print "+ $newname(\n";
				print "  " . join(", ", (@pre, @post)) . ");\n";
			}
		}
	}
}
------------------8<----------------------

since v4:

- Merge mergeable patches that have received acks in time.

- Drop the VALIDATE_LATE MC link flag from this set.

- Move the frame descriptor patches out of the set.

- Do not add the client_info argument to the get_fmt() pad op.

- Introduce v4l2_subdev_get_fmt_ci() which returns the pad format but uses
  the same arguments (i.e. also the client info argument) as the set_fmt()
  pad op.

- Call the VISIBLE area in v4 as ACTIVE_AREA (imx219 driver; patch
  merged).

- Set the pixel_rate control's ops to NULL (the control is read-only and
  not applied on device).

- Fix imx219 driver's limits for VBLANK, HBLANK and EXPOSURE controls.

- Postpone the patch bumping the STREAMS capability bit.

- Fix stopping streams documentation in patch "media: Improve
  enable_streams and disable_streams documentation".

since v3:

- Include the patch adding v4l2_subdev_get_frame_desc(); compared to the
  version posted earlier, this one uses get_fmt() pad op.

- Removed kernel-doc documentation from sub-device ops in drivers. These
  comments are redundant.

- Include v4l2_subdev_client_info changes for ov2732 and t4ka3 drivers.

- Fix setting media link flags.

- Fix the condition for skipping early validation (and doing it later).
  The old condition sometimes caused the link being validated too early.

since v2:

- Include imx219 driver fixes accrued during imx219 Common Raw Sensor
  Model support implementation. This includes fixes to control ranges and
  the removal of the rate_factor, among other things.

- Reword documentation of the VALIDATE_LATE link flag.

- Require that VALIDATE_LATE links sink (origin entity in this case) may
  have a single pad only.

- Drop the pipe argument from __media_pipeline_validate_one() as the
  function doesn't use it.

- Fix MUST_CONNECT flag validation for links that also have VALIDATE_LATE
  flag.

- Rename the macros to create wrapper functions, define one for "! defined
  CONFIG_MEDIA_CONTROLLER" case as well.

- Call v4l2_subdev_call_waive_fourth() v4l2_subdev_call_drop_fourth() now.

- Comment the link validation code a little better.

- Fix copying the routes in the routing cleanup patch (copy the same
  number of routes as before), rename the function doing it well. Make the
  new function copy_routes_state_to_routing() to take just routing and
  state arguments, for all callers have this information.

- Add a patch bumping the STREAMS sub-device capability bit and document
  that bit 0x2 is reserved.

since v1:

- Address compilation issues. I had only tested on arm64 previously.

- Introduce v4l2_subdev_call_ci_state_active(), to call pad ops with
  client info argument with active state. Getting around needing a new
  function doesn't seem easy (probably requires one more layer of
  indirection I'd like to do without).

Sakari Ailus (10):
  media: Documentation: Improve pixel rate calculation documentation
  media: imx219: Scale the vblank limits according to rate_factor
  media: imx219: Account rate_factor in setting upper exposure limit
  media: imx219: Make control handler ops for PIXEL_RATE NULL
  media: imx219: Rename "binning" as "bin_hv" in imx219_set_pad_format
  media: imx219: Fix vertical blanking and exposure for analogue binning
  media: Improve enable_streams and disable_streams documentation
  media: v4l2-subdev: Move subdev client capabilities into a new struct
  media: v4l2-subdev: Add v4l2_subdev_get_fmt_ci()
  media: v4l2-subdev: Add struct v4l2_subdev_client_info pointer to pad
    ops

 Documentation/driver-api/media/tx-rx.rst      |  6 +-
 drivers/media/i2c/adv7170.c                   |  1 +
 drivers/media/i2c/adv7175.c                   |  1 +
 drivers/media/i2c/adv7180.c                   |  3 +-
 drivers/media/i2c/adv7183.c                   |  3 +-
 drivers/media/i2c/adv748x/adv748x-afe.c       |  1 +
 drivers/media/i2c/adv748x/adv748x-csi2.c      |  1 +
 drivers/media/i2c/adv748x/adv748x-hdmi.c      |  1 +
 drivers/media/i2c/adv7511-v4l2.c              |  1 +
 drivers/media/i2c/adv7604.c                   |  2 +
 drivers/media/i2c/adv7842.c                   |  1 +
 drivers/media/i2c/ak881x.c                    |  3 +-
 drivers/media/i2c/alvium-csi2.c               |  3 +
 drivers/media/i2c/ar0521.c                    |  1 +
 drivers/media/i2c/ccs/ccs-core.c              | 12 ++-
 drivers/media/i2c/cx25840/cx25840-core.c      |  1 +
 drivers/media/i2c/ds90ub913.c                 |  1 +
 drivers/media/i2c/ds90ub953.c                 |  1 +
 drivers/media/i2c/ds90ub960.c                 |  1 +
 drivers/media/i2c/et8ek8/et8ek8_driver.c      |  1 +
 drivers/media/i2c/gc0308.c                    |  1 +
 drivers/media/i2c/gc0310.c                    |  3 +-
 drivers/media/i2c/gc05a2.c                    |  4 +-
 drivers/media/i2c/gc08a3.c                    |  4 +-
 drivers/media/i2c/gc2145.c                    |  2 +
 drivers/media/i2c/hi556.c                     |  2 +
 drivers/media/i2c/hi846.c                     |  2 +
 drivers/media/i2c/hi847.c                     |  1 +
 drivers/media/i2c/imx111.c                    |  1 +
 drivers/media/i2c/imx208.c                    |  1 +
 drivers/media/i2c/imx214.c                    |  4 +-
 drivers/media/i2c/imx219.c                    | 48 ++++++------
 drivers/media/i2c/imx258.c                    |  2 +
 drivers/media/i2c/imx274.c                    |  3 +
 drivers/media/i2c/imx283.c                    |  2 +
 drivers/media/i2c/imx290.c                    |  4 +-
 drivers/media/i2c/imx296.c                    |  7 +-
 drivers/media/i2c/imx319.c                    |  1 +
 drivers/media/i2c/imx334.c                    |  3 +-
 drivers/media/i2c/imx335.c                    |  4 +-
 drivers/media/i2c/imx355.c                    |  1 +
 drivers/media/i2c/imx412.c                    |  3 +-
 drivers/media/i2c/imx415.c                    |  4 +-
 drivers/media/i2c/isl7998x.c                  |  1 +
 drivers/media/i2c/lt6911uxe.c                 |  5 +-
 drivers/media/i2c/max9286.c                   |  1 +
 drivers/media/i2c/max96714.c                  |  1 +
 drivers/media/i2c/max96717.c                  |  1 +
 drivers/media/i2c/ml86v7667.c                 |  2 +-
 drivers/media/i2c/mt9m001.c                   |  5 +-
 drivers/media/i2c/mt9m111.c                   |  3 +
 drivers/media/i2c/mt9m114.c                   |  6 ++
 drivers/media/i2c/mt9p031.c                   |  3 +
 drivers/media/i2c/mt9t112.c                   |  3 +
 drivers/media/i2c/mt9v011.c                   |  1 +
 drivers/media/i2c/mt9v032.c                   |  3 +
 drivers/media/i2c/mt9v111.c                   |  1 +
 drivers/media/i2c/og01a1b.c                   |  3 +-
 drivers/media/i2c/og0ve1b.c                   |  3 +-
 drivers/media/i2c/os05b10.c                   |  2 +
 drivers/media/i2c/ov01a10.c                   |  3 +
 drivers/media/i2c/ov02a10.c                   |  3 +-
 drivers/media/i2c/ov02c10.c                   |  1 +
 drivers/media/i2c/ov02e10.c                   |  1 +
 drivers/media/i2c/ov08d10.c                   |  1 +
 drivers/media/i2c/ov08x40.c                   |  1 +
 drivers/media/i2c/ov13858.c                   |  1 +
 drivers/media/i2c/ov13b10.c                   |  1 +
 drivers/media/i2c/ov2640.c                    |  2 +
 drivers/media/i2c/ov2659.c                    |  1 +
 drivers/media/i2c/ov2680.c                    |  3 +
 drivers/media/i2c/ov2685.c                    |  2 +
 drivers/media/i2c/ov2732.c                    |  4 +-
 drivers/media/i2c/ov2735.c                    |  4 +-
 drivers/media/i2c/ov2740.c                    |  1 +
 drivers/media/i2c/ov4689.c                    |  2 +
 drivers/media/i2c/ov5640.c                    |  2 +
 drivers/media/i2c/ov5645.c                    |  4 +-
 drivers/media/i2c/ov5647.c                    |  2 +
 drivers/media/i2c/ov5648.c                    |  1 +
 drivers/media/i2c/ov5670.c                    |  2 +
 drivers/media/i2c/ov5675.c                    |  2 +
 drivers/media/i2c/ov5693.c                    |  3 +
 drivers/media/i2c/ov5695.c                    |  1 +
 drivers/media/i2c/ov6211.c                    |  3 +-
 drivers/media/i2c/ov64a40.c                   |  2 +
 drivers/media/i2c/ov7251.c                    |  4 +-
 drivers/media/i2c/ov7670.c                    |  1 +
 drivers/media/i2c/ov772x.c                    |  2 +
 drivers/media/i2c/ov7740.c                    |  1 +
 drivers/media/i2c/ov8856.c                    |  1 +
 drivers/media/i2c/ov8858.c                    |  3 +-
 drivers/media/i2c/ov8865.c                    |  2 +
 drivers/media/i2c/ov9282.c                    |  4 +-
 drivers/media/i2c/ov9640.c                    |  2 +
 drivers/media/i2c/ov9650.c                    |  1 +
 drivers/media/i2c/ov9734.c                    |  1 +
 drivers/media/i2c/rdacm20.c                   |  2 +-
 drivers/media/i2c/rdacm21.c                   |  2 +-
 drivers/media/i2c/rj54n1cb0c.c                |  3 +
 drivers/media/i2c/s5c73m3/s5c73m3-core.c      |  2 +
 drivers/media/i2c/s5k3m5.c                    |  4 +-
 drivers/media/i2c/s5k5baf.c                   |  3 +
 drivers/media/i2c/s5k6a3.c                    |  1 +
 drivers/media/i2c/s5kjn1.c                    |  4 +-
 drivers/media/i2c/saa6752hs.c                 |  1 +
 drivers/media/i2c/saa7115.c                   |  1 +
 drivers/media/i2c/saa717x.c                   |  1 +
 drivers/media/i2c/st-mipid02.c                |  1 +
 drivers/media/i2c/t4ka3.c                     |  3 +
 drivers/media/i2c/tc358743.c                  |  1 +
 drivers/media/i2c/tc358746.c                  |  1 +
 drivers/media/i2c/tda1997x.c                  |  1 +
 drivers/media/i2c/thp7312.c                   |  1 +
 drivers/media/i2c/tvp514x.c                   |  1 +
 drivers/media/i2c/tvp5150.c                   |  4 +-
 drivers/media/i2c/tvp7002.c                   |  1 +
 drivers/media/i2c/tw9900.c                    |  1 +
 drivers/media/i2c/tw9910.c                    |  2 +
 drivers/media/i2c/vd55g1.c                    |  4 +-
 drivers/media/i2c/vd56g3.c                    |  4 +-
 drivers/media/i2c/vgxy61.c                    |  4 +-
 drivers/media/pci/cobalt/cobalt-driver.c      |  8 +-
 drivers/media/pci/cobalt/cobalt-v4l2.c        | 10 +--
 drivers/media/pci/cx18/cx18-av-core.c         |  1 +
 drivers/media/pci/cx18/cx18-controls.c        |  2 +-
 drivers/media/pci/cx18/cx18-ioctl.c           |  2 +-
 drivers/media/pci/cx23885/cx23885-video.c     |  4 +-
 drivers/media/pci/intel/ipu3/ipu3-cio2.c      |  1 +
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c |  2 +
 .../media/pci/intel/ipu6/ipu6-isys-subdev.c   |  1 +
 .../media/pci/intel/ipu6/ipu6-isys-subdev.h   |  1 +
 drivers/media/pci/intel/ivsc/mei_csi.c        |  1 +
 drivers/media/pci/ivtv/ivtv-controls.c        |  2 +-
 drivers/media/pci/ivtv/ivtv-ioctl.c           |  2 +-
 drivers/media/pci/saa7134/saa7134-empress.c   |  4 +-
 drivers/media/platform/amd/isp4/isp4_subdev.c |  1 +
 drivers/media/platform/amd/isp4/isp4_video.c  |  2 +-
 .../platform/amlogic/c3/isp/c3-isp-core.c     |  1 +
 .../platform/amlogic/c3/isp/c3-isp-resizer.c  |  3 +
 .../amlogic/c3/mipi-adapter/c3-mipi-adap.c    |  1 +
 .../amlogic/c3/mipi-csi2/c3-mipi-csi2.c       |  1 +
 .../platform/arm/mali-c55/mali-c55-isp.c      |  3 +
 .../platform/arm/mali-c55/mali-c55-resizer.c  | 15 +++-
 .../platform/arm/mali-c55/mali-c55-tpg.c      |  1 +
 drivers/media/platform/atmel/atmel-isi.c      |  4 +-
 .../media/platform/broadcom/bcm2835-unicam.c  |  1 +
 drivers/media/platform/cadence/cdns-csi2rx.c  |  1 +
 drivers/media/platform/cadence/cdns-csi2tx.c  |  1 +
 drivers/media/platform/intel/pxa_camera.c     |  6 +-
 drivers/media/platform/marvell/mcam-core.c    |  4 +-
 .../platform/microchip/microchip-csi2dc.c     |  1 +
 .../platform/microchip/microchip-isc-scaler.c |  2 +
 drivers/media/platform/nxp/imx-mipi-csis.c    |  3 +-
 drivers/media/platform/nxp/imx7-media-csi.c   |  1 +
 .../platform/nxp/imx8-isi/imx8-isi-crossbar.c |  1 +
 .../platform/nxp/imx8-isi/imx8-isi-pipe.c     |  3 +
 drivers/media/platform/nxp/imx8mq-mipi-csi2.c |  1 +
 .../media/platform/qcom/camss/camss-csid.c    |  3 +-
 .../media/platform/qcom/camss/camss-csiphy.c  |  3 +-
 .../media/platform/qcom/camss/camss-ispif.c   |  3 +-
 drivers/media/platform/qcom/camss/camss-tpg.c |  3 +-
 drivers/media/platform/qcom/camss/camss-vfe.c | 12 ++-
 .../media/platform/raspberrypi/rp1-cfe/csi2.c |  1 +
 .../platform/raspberrypi/rp1-cfe/pisp-fe.c    |  1 +
 drivers/media/platform/renesas/rcar-csi2.c    |  1 +
 .../media/platform/renesas/rcar-isp/csisp.c   |  1 +
 drivers/media/platform/renesas/renesas-ceu.c  |  7 +-
 .../platform/renesas/rzg2l-cru/rzg2l-csi2.c   |  3 +-
 .../platform/renesas/rzg2l-cru/rzg2l-ip.c     |  3 +-
 .../renesas/rzv2h-ivc/rzv2h-ivc-subdev.c      |  1 +
 drivers/media/platform/renesas/sh_vou.c       |  4 +-
 .../media/platform/renesas/vsp1/vsp1_brx.c    |  3 +
 .../media/platform/renesas/vsp1/vsp1_drm.c    | 18 +++--
 .../media/platform/renesas/vsp1/vsp1_entity.c |  4 +-
 .../media/platform/renesas/vsp1/vsp1_entity.h |  1 +
 .../media/platform/renesas/vsp1/vsp1_histo.c  |  3 +
 .../media/platform/renesas/vsp1/vsp1_hsit.c   |  1 +
 .../media/platform/renesas/vsp1/vsp1_rwpf.c   |  3 +
 .../media/platform/renesas/vsp1/vsp1_sru.c    |  1 +
 .../media/platform/renesas/vsp1/vsp1_uds.c    |  1 +
 .../media/platform/renesas/vsp1/vsp1_uif.c    |  2 +
 .../media/platform/renesas/vsp1/vsp1_vspx.c   |  3 +-
 .../platform/rockchip/rkcif/rkcif-interface.c |  3 +
 .../platform/rockchip/rkisp1/rkisp1-csi.c     |  1 +
 .../platform/rockchip/rkisp1/rkisp1-isp.c     |  3 +
 .../platform/rockchip/rkisp1/rkisp1-resizer.c |  3 +
 .../samsung/exynos4-is/fimc-capture.c         |  7 +-
 .../platform/samsung/exynos4-is/fimc-isp.c    |  1 +
 .../platform/samsung/exynos4-is/fimc-lite.c   |  3 +
 .../platform/samsung/exynos4-is/mipi-csis.c   |  1 +
 .../samsung/s3c-camif/camif-capture.c         |  3 +
 .../platform/samsung/s3c-camif/camif-core.c   |  2 +-
 drivers/media/platform/st/stm32/stm32-csi.c   |  1 +
 drivers/media/platform/st/stm32/stm32-dcmi.c  |  7 +-
 .../st/stm32/stm32-dcmipp/dcmipp-byteproc.c   |  3 +
 .../st/stm32/stm32-dcmipp/dcmipp-input.c      |  1 +
 .../platform/sunxi/sun4i-csi/sun4i_v4l2.c     |  1 +
 .../sunxi/sun6i-csi/sun6i_csi_bridge.c        |  1 +
 .../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c   |  1 +
 .../sun8i_a83t_mipi_csi2.c                    |  1 +
 .../media/platform/synopsys/dw-mipi-csi2rx.c  |  1 +
 .../media/platform/ti/am437x/am437x-vpfe.c    |  2 +-
 drivers/media/platform/ti/cal/cal-camerarx.c  |  1 +
 drivers/media/platform/ti/cal/cal-video.c     |  4 +-
 .../platform/ti/j721e-csi2rx/j721e-csi2rx.c   |  1 +
 drivers/media/platform/ti/omap3isp/ispccdc.c  |  5 +-
 drivers/media/platform/ti/omap3isp/ispccp2.c  |  3 +-
 drivers/media/platform/ti/omap3isp/ispcsi2.c  |  3 +-
 .../media/platform/ti/omap3isp/isppreview.c   |  5 +-
 .../media/platform/ti/omap3isp/ispresizer.c   |  5 +-
 drivers/media/platform/ti/omap3isp/ispvideo.c |  4 +-
 drivers/media/platform/ti/vpe/vip.c           |  4 +-
 drivers/media/platform/via/via-camera.c       |  4 +-
 drivers/media/platform/video-mux.c            |  1 +
 .../media/platform/xilinx/xilinx-csi2rxss.c   |  1 +
 drivers/media/platform/xilinx/xilinx-tpg.c    |  1 +
 .../media/test-drivers/vimc/vimc-debayer.c    |  1 +
 drivers/media/test-drivers/vimc/vimc-scaler.c |  3 +
 drivers/media/test-drivers/vimc/vimc-sensor.c |  1 +
 drivers/media/usb/cx231xx/cx231xx-417.c       |  2 +-
 drivers/media/usb/cx231xx/cx231xx-video.c     |  4 +-
 drivers/media/usb/dvb-usb/cxusb-analog.c      |  6 +-
 drivers/media/usb/em28xx/em28xx-camera.c      |  2 +-
 drivers/media/usb/go7007/go7007-v4l2.c        |  2 +-
 drivers/media/usb/go7007/s2250-board.c        |  1 +
 drivers/media/usb/pvrusb2/pvrusb2-hdw.c       |  2 +-
 drivers/media/v4l2-core/v4l2-subdev.c         | 78 ++++++++++++++-----
 .../media/atomisp/i2c/atomisp-gc2235.c        |  1 +
 .../media/atomisp/i2c/atomisp-ov2722.c        |  1 +
 .../staging/media/atomisp/pci/atomisp_cmd.c   | 16 ++--
 .../staging/media/atomisp/pci/atomisp_csi2.c  |  1 +
 .../media/atomisp/pci/atomisp_subdev.c        |  3 +
 .../staging/media/atomisp/pci/atomisp_v4l2.c  |  8 +-
 drivers/staging/media/imx/imx-ic-prp.c        |  1 +
 drivers/staging/media/imx/imx-ic-prpencvf.c   |  1 +
 drivers/staging/media/imx/imx-media-csi.c     |  3 +
 drivers/staging/media/imx/imx-media-vdic.c    |  1 +
 drivers/staging/media/imx/imx6-mipi-csi2.c    |  1 +
 drivers/staging/media/ipu3/ipu3-v4l2.c        |  3 +
 drivers/staging/media/ipu7/ipu7-isys-csi2.c   |  2 +
 drivers/staging/media/ipu7/ipu7-isys-subdev.c |  1 +
 drivers/staging/media/ipu7/ipu7-isys-subdev.h |  1 +
 .../media/sunxi/sun6i-isp/sun6i_isp_proc.c    |  1 +
 drivers/staging/media/tegra-video/csi.c       |  1 +
 drivers/staging/media/tegra-video/vi.c        | 10 +--
 include/media/v4l2-subdev.h                   | 41 +++++++++-
 247 files changed, 612 insertions(+), 183 deletions(-)

-- 
2.47.3

[PATCH v5 01/10] media: Documentation: Improve pixel rate calculation documentation

[Sakari Ailus]

Improve documentation on calculating the pixel rate, by adding references
to relevant functions and mentioning V4L2 fwnode endpoint instead of OF
endpoint.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 Documentation/driver-api/media/tx-rx.rst | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/Documentation/driver-api/media/tx-rx.rst b/Documentation/driver-api/media/tx-rx.rst
index 7df2407817b3..9b231fa0216a 100644
--- a/Documentation/driver-api/media/tx-rx.rst
+++ b/Documentation/driver-api/media/tx-rx.rst
@@ -104,7 +104,11 @@ where
    * - k
      - 16 for D-PHY and 7 for C-PHY.
 
-Information on whether D-PHY or C-PHY is used, and the value of ``nr_of_lanes``, can be obtained from the OF endpoint configuration.
+Information on whether D-PHY or C-PHY is used as well as the value of
+``nr_of_lanes`` can be obtained from the V4L2 endpoint configuration; see
+:c:func:`v4l2_fwnode_endpoint_alloc_parse()`,
+:c:func:`v4l2_fwnode_endpoint_parse()` and
+:c:func:`v4l2_get_active_data_lanes()`.
 
 .. note::
 
-- 
2.47.3

[PATCH v5 02/10] media: imx219: Scale the vblank limits according to rate_factor

[Sakari Ailus]

The limits for vertical blanking (and frame length in pixels) is related
to the properties of the hardware, it's not in half-line units the driver
uses. Multiply the vertical blanking limits by the rate_factor to satisty
hardware requirements.

Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
Cc: stable@vger.kernel.org
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 223d3753cc93..d8fe7db18b6c 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -878,14 +878,17 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
 
 	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
+		unsigned int rate_factor = imx219_get_rate_factor(state);
 		int exposure_max;
 		int exposure_def;
 		int llp_min;
 		int pixel_rate;
 
 		/* Update limits and set FPS to default */
-		ret = __v4l2_ctrl_modify_range(imx219->vblank, IMX219_VBLANK_MIN,
-					       IMX219_FLL_MAX - mode->height, 1,
+		ret = __v4l2_ctrl_modify_range(imx219->vblank,
+					       IMX219_VBLANK_MIN * rate_factor,
+					       (IMX219_FLL_MAX - mode->height) *
+					       rate_factor, rate_factor,
 					       mode->fll_def - mode->height);
 		if (ret)
 			return ret;
@@ -928,8 +931,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 			return ret;
 
 		/* Scale the pixel rate based on the mode specific factor */
-		pixel_rate = imx219_get_pixel_rate(imx219) *
-			     imx219_get_rate_factor(state);
+		pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
 		ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
 					       pixel_rate, 1, pixel_rate);
 		if (ret)
-- 
2.47.3

Re: [PATCH v5 02/10] media: imx219: Scale the vblank limits according to rate_factor

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Jun 08, 2026 at 12:53:48AM +0300, Sakari Ailus wrote:
> The limits for vertical blanking (and frame length in pixels) is related
> to the properties of the hardware, it's not in half-line units the driver
> uses.

The driver only uses half-line units when binning in the analog domain.
The commit message would benefit from making this clearer.

> Multiply the vertical blanking limits by the rate_factor to satisty
> hardware requirements.
> 
> Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> Cc: stable@vger.kernel.org
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/i2c/imx219.c | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index 223d3753cc93..d8fe7db18b6c 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -878,14 +878,17 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
>  
>  	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> +		unsigned int rate_factor = imx219_get_rate_factor(state);
>  		int exposure_max;
>  		int exposure_def;
>  		int llp_min;
>  		int pixel_rate;
>  
>  		/* Update limits and set FPS to default */
> -		ret = __v4l2_ctrl_modify_range(imx219->vblank, IMX219_VBLANK_MIN,
> -					       IMX219_FLL_MAX - mode->height, 1,
> +		ret = __v4l2_ctrl_modify_range(imx219->vblank,
> +					       IMX219_VBLANK_MIN * rate_factor,
> +					       (IMX219_FLL_MAX - mode->height) *
> +					       rate_factor, rate_factor,

Without judging whether or not dividing LLP by the rate factor in
imx219_set_ctrl() is correct or not, this patch brings the control
limits in line with how the control is applied, so

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  					       mode->fll_def - mode->height);
>  		if (ret)
>  			return ret;
> @@ -928,8 +931,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  			return ret;
>  
>  		/* Scale the pixel rate based on the mode specific factor */
> -		pixel_rate = imx219_get_pixel_rate(imx219) *
> -			     imx219_get_rate_factor(state);
> +		pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
>  		ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
>  					       pixel_rate, 1, pixel_rate);
>  		if (ret)

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 02/10] media: imx219: Scale the vblank limits according to rate_factor

[Dave Stevenson]

Hi Sakari

On Sun, 7 Jun 2026 at 22:54, Sakari Ailus <sakari.ailus@linux.intel.com> wrote:
>
> The limits for vertical blanking (and frame length in pixels) is related
> to the properties of the hardware, it's not in half-line units the driver
> uses. Multiply the vertical blanking limits by the rate_factor to satisty
> hardware requirements.

Whilst that would be a logical interpretation, it doesn't match with
how the hardware performs, nor the docs.

The datasheet for register FRM_LENGTH_A 0x160 says
frame_length_lines
BINNING_MODE = 0,1,2
Unit: 1Lines
BINNING_MODE = 3
Units: 2Lines

That's not units of 2 lines for active lines only, that is units of 2
lines for ALL lines.

I have tested it, and the sensor works fine with FRM_LENGTH_A being
0x278 in the 1640x1232 mode, and 0x100 in 640x480 modes with all the
tests I've thrown at it. Drop them any lower and it does stall or give
corrupt horizontal lines.
(Please note that the sensor extends the frame length automatically to
accommodate the exposure time requested, so do ensure the exposure
time doesn't interact with the frame length if you're testing).

This patch drops the maximum frame rate from 81.07 to 79.07fps in
1640x1232 (2.5%), and 200.1 to 188.39fps in 640x480 (6%) for no good
reason that I can see.
Unless anyone can produce a genuine situation where they see the
sensor behave incorrectly with the old setup, I'll be very sad to see
this merged.

  Dave

> Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> Cc: stable@vger.kernel.org
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/i2c/imx219.c | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index 223d3753cc93..d8fe7db18b6c 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -878,14 +878,17 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>         crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
>
>         if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> +               unsigned int rate_factor = imx219_get_rate_factor(state);
>                 int exposure_max;
>                 int exposure_def;
>                 int llp_min;
>                 int pixel_rate;
>
>                 /* Update limits and set FPS to default */
> -               ret = __v4l2_ctrl_modify_range(imx219->vblank, IMX219_VBLANK_MIN,
> -                                              IMX219_FLL_MAX - mode->height, 1,
> +               ret = __v4l2_ctrl_modify_range(imx219->vblank,
> +                                              IMX219_VBLANK_MIN * rate_factor,
> +                                              (IMX219_FLL_MAX - mode->height) *
> +                                              rate_factor, rate_factor,
>                                                mode->fll_def - mode->height);
>                 if (ret)
>                         return ret;
> @@ -928,8 +931,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>                         return ret;
>
>                 /* Scale the pixel rate based on the mode specific factor */
> -               pixel_rate = imx219_get_pixel_rate(imx219) *
> -                            imx219_get_rate_factor(state);
> +               pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
>                 ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
>                                                pixel_rate, 1, pixel_rate);
>                 if (ret)
> --
> 2.47.3
>

Re: [PATCH v5 02/10] media: imx219: Scale the vblank limits according to rate_factor

[Laurent Pinchart]

Hi Dave,

Thanks a lot for taking the time to investigate and provide very
valuable insight. I really appreciate that.

On Mon, Jun 08, 2026 at 04:29:46PM +0100, Dave Stevenson wrote:
> On Sun, 7 Jun 2026 at 22:54, Sakari Ailus wrote:
> >
> > The limits for vertical blanking (and frame length in pixels) is related
> > to the properties of the hardware, it's not in half-line units the driver
> > uses. Multiply the vertical blanking limits by the rate_factor to satisty
> > hardware requirements.
> 
> Whilst that would be a logical interpretation, it doesn't match with
> how the hardware performs, nor the docs.
> 
> The datasheet for register FRM_LENGTH_A 0x160 says
> frame_length_lines
> BINNING_MODE = 0,1,2
> Unit: 1Lines
> BINNING_MODE = 3
> Units: 2Lines
> 
> That's not units of 2 lines for active lines only, that is units of 2
> lines for ALL lines.
> 
> I have tested it, and the sensor works fine with FRM_LENGTH_A being
> 0x278 in the 1640x1232 mode, and 0x100 in 640x480 modes with all the
> tests I've thrown at it. Drop them any lower and it does stall or give
> corrupt horizontal lines.

If I understand this correctly, it indicates that

- The frame length is expressed as a number of lines at the output of
  the sensor (after binning).

- The minimum margin between the output height (after binning) and the
  frame length is 32 lines.

- In special binning mode, the FRM_LENGTH_A register needs to be
  programmed with frame_length_lines / 2.

The last constraint is device-specific, and as far as I understand it
can be handled directly in the driver without affecting the userspace
API by simply dividing the frame length value by 2 before writing it to
the register.

It's getting a bit late, I'll test the "regular" binning mode
(BINNING_MODE = 1) tomorrow to see how it compares (unless someone beats
me to it). Unless I get very unexpected results, it seems that the
existing implementation is correct and the patch should be dropped.

One thing that may not be implemented correctly is different binning
modes horizontally and vertically. imx219_get_rate_factor() will return
2 only when both the horizontal and vertical binning modes are "special
analog binning". When used to scale the frame length and exposure time,
I wonder if only vertical binning should be taken into account.

> (Please note that the sensor extends the frame length automatically to
> accommodate the exposure time requested, so do ensure the exposure
> time doesn't interact with the frame length if you're testing).
> 
> This patch drops the maximum frame rate from 81.07 to 79.07fps in
> 1640x1232 (2.5%), and 200.1 to 188.39fps in 640x480 (6%) for no good
> reason that I can see.
> Unless anyone can produce a genuine situation where they see the
> sensor behave incorrectly with the old setup, I'll be very sad to see
> this merged.
> 
> > Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/i2c/imx219.c | 10 ++++++----
> >  1 file changed, 6 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > index 223d3753cc93..d8fe7db18b6c 100644
> > --- a/drivers/media/i2c/imx219.c
> > +++ b/drivers/media/i2c/imx219.c
> > @@ -878,14 +878,17 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >         crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
> >
> >         if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> > +               unsigned int rate_factor = imx219_get_rate_factor(state);
> >                 int exposure_max;
> >                 int exposure_def;
> >                 int llp_min;
> >                 int pixel_rate;
> >
> >                 /* Update limits and set FPS to default */
> > -               ret = __v4l2_ctrl_modify_range(imx219->vblank, IMX219_VBLANK_MIN,
> > -                                              IMX219_FLL_MAX - mode->height, 1,
> > +               ret = __v4l2_ctrl_modify_range(imx219->vblank,
> > +                                              IMX219_VBLANK_MIN * rate_factor,
> > +                                              (IMX219_FLL_MAX - mode->height) *
> > +                                              rate_factor, rate_factor,
> >                                                mode->fll_def - mode->height);
> >                 if (ret)
> >                         return ret;
> > @@ -928,8 +931,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >                         return ret;
> >
> >                 /* Scale the pixel rate based on the mode specific factor */
> > -               pixel_rate = imx219_get_pixel_rate(imx219) *
> > -                            imx219_get_rate_factor(state);
> > +               pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
> >                 ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
> >                                                pixel_rate, 1, pixel_rate);
> >                 if (ret)

-- 
Regards,

Laurent Pinchart

[PATCH v5 03/10] media: imx219: Account rate_factor in setting upper exposure limit

[Sakari Ailus]

The rate_factor multiplier is used to multiply a few values in the
sensor's timing configuration and the exposure time is one of them. This
also needs to be taken into account in exposure time margin: multiply it
by rate_factor so that sensor's exposure time margin is respected.

Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
Cc: stable@vger.kernel.org
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index d8fe7db18b6c..e681f80f9e3e 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -451,7 +451,8 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
 		int exposure_max, exposure_def;
 
 		/* Update max exposure while meeting expected vblanking */
-		exposure_max = format->height + ctrl->val - IMX219_EXPOSURE_OFFSET;
+		exposure_max = format->height + ctrl->val -
+			IMX219_EXPOSURE_OFFSET * rate_factor;
 		exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
 				exposure_max : IMX219_EXPOSURE_DEFAULT;
 		ret = __v4l2_ctrl_modify_range(imx219->exposure,
-- 
2.47.3

Re: [PATCH v5 03/10] media: imx219: Account rate_factor in setting upper exposure limit

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The patch updates the exposure maximum limit calculation in `imx219_set_ctrl()` to account for analog binning (initially via `rate_factor`, later rewritten using `bin_hv` in commit "media: imx219: Fix vertical blanking and exposure for analogue binning"), but misses the identical calculation in `imx219_set_pad_format()`. This logic error persists through the end of the series. When a binned format is set, `__v4l2_ctrl_s_ctrl(vblank)` correctly updates the exposure limit via `imx219_set_ctrl()`. However, immediately afterward, `imx219_set_pad_format()` recalculates and overwrites `exposure_max` using the unscaled formula `mode->fll_def - IMX219_EXPOSURE_OFFSET` (instead of `(mode->fll_def / bin_hv) - IMX219_EXPOSURE_OFFSET`). This allows userspace to set an exposure time greater than the frame length, violating sensor hardware constraints.
--

commit 5813edc0a3e8ed48f469607d03f8489ba7a6eb19
Author: Sakari Ailus <sakari.ailus@linux.intel.com>

media: imx219: Account rate_factor in setting upper exposure limit

This commit modifies the maximum exposure time calculation in imx219_set_ctrl()
to account for the rate_factor multiplier when analog binning is used. This
ensures the sensor's exposure time margin is correctly respected.

> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index d8fe7db18b6cd..e681f80f9e3ef 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -451,7 +451,8 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
>  		int exposure_max, exposure_def;
>  
>  		/* Update max exposure while meeting expected vblanking */
> -		exposure_max = format->height + ctrl->val - IMX219_EXPOSURE_OFFSET;
> +		exposure_max = format->height + ctrl->val -
> +			IMX219_EXPOSURE_OFFSET * rate_factor;

[Severity: High]
This is a pre-existing issue, but does imx219_set_pad_format() also need to
be updated to apply scaling to its exposure maximum calculation?

When a binned format is set, __v4l2_ctrl_s_ctrl(vblank) correctly updates
the exposure limit here in imx219_set_ctrl(). However, immediately afterward,
imx219_set_pad_format() recalculates and overwrites exposure_max using the
unscaled formula:

drivers/media/i2c/imx219.c:imx219_set_pad_format() {
	...
		/* Update max exposure while meeting expected vblanking */
		exposure_max = mode->fll_def - IMX219_EXPOSURE_OFFSET;
		exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
				exposure_max : IMX219_EXPOSURE_DEFAULT;
		ret = __v4l2_ctrl_modify_range(imx219->exposure,
					       imx219->exposure->minimum,
					       exposure_max,
					       imx219->exposure->step,
					       exposure_def);
	...
}

It appears this logic error persists until it is rewritten using bin_hv in
the later commit "media: imx219: Fix vertical blanking and exposure for
analogue binning".

Until then, does this allow userspace to set an exposure time greater than
the actual programmed frame length, violating sensor hardware constraints and
potentially resulting in sensor streaming failures or I2C timeouts?

>  		exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
>  				exposure_max : IMX219_EXPOSURE_DEFAULT;
>  		ret = __v4l2_ctrl_modify_range(imx219->exposure,

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260607215356.842932-1-sakari.ailus@linux.intel.com?part=3

Re: [PATCH v5 03/10] media: imx219: Account rate_factor in setting upper exposure limit

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Jun 08, 2026 at 12:53:49AM +0300, Sakari Ailus wrote:
> The rate_factor multiplier is used to multiply a few values in the
> sensor's timing configuration and the exposure time is one of them. This
> also needs to be taken into account in exposure time margin: multiply it
> by rate_factor so that sensor's exposure time margin is respected.
> 
> Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> Cc: stable@vger.kernel.org
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/i2c/imx219.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index d8fe7db18b6c..e681f80f9e3e 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -451,7 +451,8 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
>  		int exposure_max, exposure_def;
>  
>  		/* Update max exposure while meeting expected vblanking */
> -		exposure_max = format->height + ctrl->val - IMX219_EXPOSURE_OFFSET;
> +		exposure_max = format->height + ctrl->val -
> +			IMX219_EXPOSURE_OFFSET * rate_factor;

I'm not sure if this is quite right, I need some more time to research.
Regardless of that, isn't the same change also needed in
imx219_set_pad_format() ?

>  		exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
>  				exposure_max : IMX219_EXPOSURE_DEFAULT;
>  		ret = __v4l2_ctrl_modify_range(imx219->exposure,

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 03/10] media: imx219: Account rate_factor in setting upper exposure limit

[Sakari Ailus]

Hi Laurent,

On Mon, Jun 08, 2026 at 12:06:25PM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.
> 
> On Mon, Jun 08, 2026 at 12:53:49AM +0300, Sakari Ailus wrote:
> > The rate_factor multiplier is used to multiply a few values in the
> > sensor's timing configuration and the exposure time is one of them. This
> > also needs to be taken into account in exposure time margin: multiply it
> > by rate_factor so that sensor's exposure time margin is respected.
> > 
> > Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/i2c/imx219.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > index d8fe7db18b6c..e681f80f9e3e 100644
> > --- a/drivers/media/i2c/imx219.c
> > +++ b/drivers/media/i2c/imx219.c
> > @@ -451,7 +451,8 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> >  		int exposure_max, exposure_def;
> >  
> >  		/* Update max exposure while meeting expected vblanking */
> > -		exposure_max = format->height + ctrl->val - IMX219_EXPOSURE_OFFSET;
> > +		exposure_max = format->height + ctrl->val -
> > +			IMX219_EXPOSURE_OFFSET * rate_factor;
> 
> I'm not sure if this is quite right, I need some more time to research.
> Regardless of that, isn't the same change also needed in
> imx219_set_pad_format() ?

Indeed. I'll take that into account in v6.

Controls are initialised for non-binned default mode so that part is ok.

> 
> >  		exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
> >  				exposure_max : IMX219_EXPOSURE_DEFAULT;
> >  		ret = __v4l2_ctrl_modify_range(imx219->exposure,
> 

-- 
Regards,

Sakari Ailus

Re: [PATCH v5 03/10] media: imx219: Account rate_factor in setting upper exposure limit

[Dave Stevenson]

Hi Sakari

On Sun, 7 Jun 2026 at 22:54, Sakari Ailus <sakari.ailus@linux.intel.com> wrote:
>
> The rate_factor multiplier is used to multiply a few values in the
> sensor's timing configuration and the exposure time is one of them. This
> also needs to be taken into account in exposure time margin: multiply it
> by rate_factor so that sensor's exposure time margin is respected.

Testing the 1640x1232 mode with FRM_LENGTH_A set to 0x288 (79.07fps),
I can write register 0x15a (COARSE_INTEGRATION_TIME_A) with values up
to and including 0x284 without it affecting the output frame rate, and
without image corruption.
With IMX219_EXPOSURE_OFFSET being 4, the current code implements
exactly those limits, so why do you believe the offset should be
increased?

To my mind section 5-5 Frame Rate Calculation Formula of the datasheet
is fairly clear with
[ In the case of (frame_length_lines - 4 > coarse_integration_time) ]:
Frame_Length = frame_length_lines
[ In the case of (frame_length_lines - 4 < coarse_integration_time) ]:
Frame_Length = coarse_integration_time + 4

The register FRM_LENGTH_A (0x160) being in units of 2 lines doesn't
change that calculation.

  Dave

> Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> Cc: stable@vger.kernel.org
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/i2c/imx219.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index d8fe7db18b6c..e681f80f9e3e 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -451,7 +451,8 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
>                 int exposure_max, exposure_def;
>
>                 /* Update max exposure while meeting expected vblanking */
> -               exposure_max = format->height + ctrl->val - IMX219_EXPOSURE_OFFSET;
> +               exposure_max = format->height + ctrl->val -
> +                       IMX219_EXPOSURE_OFFSET * rate_factor;
>                 exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
>                                 exposure_max : IMX219_EXPOSURE_DEFAULT;
>                 ret = __v4l2_ctrl_modify_range(imx219->exposure,
> --
> 2.47.3
>

Re: [PATCH v5 03/10] media: imx219: Account rate_factor in setting upper exposure limit

[Laurent Pinchart]

On Mon, Jun 08, 2026 at 04:42:44PM +0100, Dave Stevenson wrote:
> On Sun, 7 Jun 2026 at 22:54, Sakari Ailus wrote:
> >
> > The rate_factor multiplier is used to multiply a few values in the
> > sensor's timing configuration and the exposure time is one of them. This
> > also needs to be taken into account in exposure time margin: multiply it
> > by rate_factor so that sensor's exposure time margin is respected.
> 
> Testing the 1640x1232 mode with FRM_LENGTH_A set to 0x288 (79.07fps),
> I can write register 0x15a (COARSE_INTEGRATION_TIME_A) with values up
> to and including 0x284 without it affecting the output frame rate, and
> without image corruption.
> With IMX219_EXPOSURE_OFFSET being 4, the current code implements
> exactly those limits, so why do you believe the offset should be
> increased?
> 
> To my mind section 5-5 Frame Rate Calculation Formula of the datasheet
> is fairly clear with
> [ In the case of (frame_length_lines - 4 > coarse_integration_time) ]:
> Frame_Length = frame_length_lines
> [ In the case of (frame_length_lines - 4 < coarse_integration_time) ]:
> Frame_Length = coarse_integration_time + 4
> 
> The register FRM_LENGTH_A (0x160) being in units of 2 lines doesn't
> change that calculation.

Readnig Jai's and your analysis, I agree. This patch should be dropped.

> > Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/i2c/imx219.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > index d8fe7db18b6c..e681f80f9e3e 100644
> > --- a/drivers/media/i2c/imx219.c
> > +++ b/drivers/media/i2c/imx219.c
> > @@ -451,7 +451,8 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> >                 int exposure_max, exposure_def;
> >
> >                 /* Update max exposure while meeting expected vblanking */
> > -               exposure_max = format->height + ctrl->val - IMX219_EXPOSURE_OFFSET;
> > +               exposure_max = format->height + ctrl->val -
> > +                       IMX219_EXPOSURE_OFFSET * rate_factor;
> >                 exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
> >                                 exposure_max : IMX219_EXPOSURE_DEFAULT;
> >                 ret = __v4l2_ctrl_modify_range(imx219->exposure,

-- 
Regards,

Laurent Pinchart

[PATCH v5 04/10] media: imx219: Make control handler ops for PIXEL_RATE NULL

[Sakari Ailus]

The PIXEL_RATE control exists to convey the value to the userspace and has
no configuration that would need to be programmed to the sensor. Make the
control handler ops for the PIXEL_RATE control NULL and avoid a warning
(as well as returning an error) from the driver.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index e681f80f9e3e..86683fe8a79e 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -556,7 +556,7 @@ static int imx219_init_controls(struct imx219 *imx219)
 		return ret;
 
 	/* By default, PIXEL_RATE is read only */
-	imx219->pixel_rate = v4l2_ctrl_new_std(ctrl_hdlr, &imx219_ctrl_ops,
+	imx219->pixel_rate = v4l2_ctrl_new_std(ctrl_hdlr, NULL,
 					       V4L2_CID_PIXEL_RATE,
 					       imx219_get_pixel_rate(imx219),
 					       imx219_get_pixel_rate(imx219), 1,
-- 
2.47.3

Re: [PATCH v5 04/10] media: imx219: Make control handler ops for PIXEL_RATE NULL

[Laurent Pinchart]

On Mon, Jun 08, 2026 at 12:53:50AM +0300, Sakari Ailus wrote:
> The PIXEL_RATE control exists to convey the value to the userspace and has
> no configuration that would need to be programmed to the sensor. Make the
> control handler ops for the PIXEL_RATE control NULL and avoid a warning
> (as well as returning an error) from the driver.

I thought the standard way to handle pixel rate being read only was to
set the V4L2_CTRL_FLAG_READ_ONLY flag, like we do for e.g.
V4L2_CID_LINK_FREQ. Is that not correct ?

> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/i2c/imx219.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index e681f80f9e3e..86683fe8a79e 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -556,7 +556,7 @@ static int imx219_init_controls(struct imx219 *imx219)
>  		return ret;
>  
>  	/* By default, PIXEL_RATE is read only */
> -	imx219->pixel_rate = v4l2_ctrl_new_std(ctrl_hdlr, &imx219_ctrl_ops,
> +	imx219->pixel_rate = v4l2_ctrl_new_std(ctrl_hdlr, NULL,
>  					       V4L2_CID_PIXEL_RATE,
>  					       imx219_get_pixel_rate(imx219),
>  					       imx219_get_pixel_rate(imx219), 1,

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 04/10] media: imx219: Make control handler ops for PIXEL_RATE NULL

[Jacopo Mondi]

Hi Laurent
  sorry if I reply in place of Sakari but I got this fresh

On Mon, Jun 08, 2026 at 10:36:53AM +0300, Laurent Pinchart wrote:
> On Mon, Jun 08, 2026 at 12:53:50AM +0300, Sakari Ailus wrote:
> > The PIXEL_RATE control exists to convey the value to the userspace and has
> > no configuration that would need to be programmed to the sensor. Make the
> > control handler ops for the PIXEL_RATE control NULL and avoid a warning
> > (as well as returning an error) from the driver.
>
> I thought the standard way to handle pixel rate being read only was to
> set the V4L2_CTRL_FLAG_READ_ONLY flag, like we do for e.g.
> V4L2_CID_LINK_FREQ. Is that not correct ?

PIXEL_RATE is RO by default

drivers/media/v4l2-core/v4l2-ctrls-defs.c:      case V4L2_CID_PIXEL_RATE:
drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *type = V4L2_CTRL_TYPE_INTEGER64;
drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *flags |= V4L2_CTRL_FLAG_READ_ONLY;
drivers/media/v4l2-core/v4l2-ctrls-defs.c-              break;

The purpose of setting the ctrl_ops member to NULL is to avoid having
to handle RO controls in the driver implementation of .s_ctrl().

>
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/i2c/imx219.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > index e681f80f9e3e..86683fe8a79e 100644
> > --- a/drivers/media/i2c/imx219.c
> > +++ b/drivers/media/i2c/imx219.c
> > @@ -556,7 +556,7 @@ static int imx219_init_controls(struct imx219 *imx219)
> >  		return ret;
> >
> >  	/* By default, PIXEL_RATE is read only */
> > -	imx219->pixel_rate = v4l2_ctrl_new_std(ctrl_hdlr, &imx219_ctrl_ops,
> > +	imx219->pixel_rate = v4l2_ctrl_new_std(ctrl_hdlr, NULL,
> >  					       V4L2_CID_PIXEL_RATE,
> >  					       imx219_get_pixel_rate(imx219),
> >  					       imx219_get_pixel_rate(imx219), 1,
>
> --
> Regards,
>
> Laurent Pinchart
>

Re: [PATCH v5 04/10] media: imx219: Make control handler ops for PIXEL_RATE NULL

[Laurent Pinchart]

On Mon, Jun 08, 2026 at 09:53:17AM +0200, Jacopo Mondi wrote:
> Hi Laurent
>   sorry if I reply in place of Sakari but I got this fresh

Thanks :-)

> On Mon, Jun 08, 2026 at 10:36:53AM +0300, Laurent Pinchart wrote:
> > On Mon, Jun 08, 2026 at 12:53:50AM +0300, Sakari Ailus wrote:
> > > The PIXEL_RATE control exists to convey the value to the userspace and has
> > > no configuration that would need to be programmed to the sensor. Make the
> > > control handler ops for the PIXEL_RATE control NULL and avoid a warning
> > > (as well as returning an error) from the driver.
> >
> > I thought the standard way to handle pixel rate being read only was to
> > set the V4L2_CTRL_FLAG_READ_ONLY flag, like we do for e.g.
> > V4L2_CID_LINK_FREQ. Is that not correct ?
> 
> PIXEL_RATE is RO by default
> 
> drivers/media/v4l2-core/v4l2-ctrls-defs.c:      case V4L2_CID_PIXEL_RATE:
> drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *type = V4L2_CTRL_TYPE_INTEGER64;
> drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *flags |= V4L2_CTRL_FLAG_READ_ONLY;
> drivers/media/v4l2-core/v4l2-ctrls-defs.c-              break;
> 
> The purpose of setting the ctrl_ops member to NULL is to avoid having
> to handle RO controls in the driver implementation of .s_ctrl().

Shouldn't the V4L2 control framework avoid .s_ctrl() calls for read-only
controls ? I thought it did already.

> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  drivers/media/i2c/imx219.c | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > > index e681f80f9e3e..86683fe8a79e 100644
> > > --- a/drivers/media/i2c/imx219.c
> > > +++ b/drivers/media/i2c/imx219.c
> > > @@ -556,7 +556,7 @@ static int imx219_init_controls(struct imx219 *imx219)
> > >  		return ret;
> > >
> > >  	/* By default, PIXEL_RATE is read only */
> > > -	imx219->pixel_rate = v4l2_ctrl_new_std(ctrl_hdlr, &imx219_ctrl_ops,
> > > +	imx219->pixel_rate = v4l2_ctrl_new_std(ctrl_hdlr, NULL,
> > >  					       V4L2_CID_PIXEL_RATE,
> > >  					       imx219_get_pixel_rate(imx219),
> > >  					       imx219_get_pixel_rate(imx219), 1,

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 04/10] media: imx219: Make control handler ops for PIXEL_RATE NULL

[Sakari Ailus]

Hi Laurent,

On Mon, Jun 08, 2026 at 11:03:38AM +0300, Laurent Pinchart wrote:
> On Mon, Jun 08, 2026 at 09:53:17AM +0200, Jacopo Mondi wrote:
> > Hi Laurent
> >   sorry if I reply in place of Sakari but I got this fresh
> 
> Thanks :-)
> 
> > On Mon, Jun 08, 2026 at 10:36:53AM +0300, Laurent Pinchart wrote:
> > > On Mon, Jun 08, 2026 at 12:53:50AM +0300, Sakari Ailus wrote:
> > > > The PIXEL_RATE control exists to convey the value to the userspace and has
> > > > no configuration that would need to be programmed to the sensor. Make the
> > > > control handler ops for the PIXEL_RATE control NULL and avoid a warning
> > > > (as well as returning an error) from the driver.
> > >
> > > I thought the standard way to handle pixel rate being read only was to
> > > set the V4L2_CTRL_FLAG_READ_ONLY flag, like we do for e.g.
> > > V4L2_CID_LINK_FREQ. Is that not correct ?
> > 
> > PIXEL_RATE is RO by default
> > 
> > drivers/media/v4l2-core/v4l2-ctrls-defs.c:      case V4L2_CID_PIXEL_RATE:
> > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *type = V4L2_CTRL_TYPE_INTEGER64;
> > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *flags |= V4L2_CTRL_FLAG_READ_ONLY;
> > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              break;
> > 
> > The purpose of setting the ctrl_ops member to NULL is to avoid having
> > to handle RO controls in the driver implementation of .s_ctrl().
> 
> Shouldn't the V4L2 control framework avoid .s_ctrl() calls for read-only
> controls ? I thought it did already.

The control may be read-only on the UAPI but the driver could still do
something about it in its s_ctrl() callback. I don't know if any driver
depends on this though.

-- 
Regards,

Sakari Ailus

Re: [PATCH v5 04/10] media: imx219: Make control handler ops for PIXEL_RATE NULL

[Laurent Pinchart]

On Mon, Jun 08, 2026 at 11:14:32AM +0300, Sakari Ailus wrote:
> On Mon, Jun 08, 2026 at 11:03:38AM +0300, Laurent Pinchart wrote:
> > On Mon, Jun 08, 2026 at 09:53:17AM +0200, Jacopo Mondi wrote:
> > > Hi Laurent
> > >   sorry if I reply in place of Sakari but I got this fresh
> > 
> > Thanks :-)
> > 
> > > On Mon, Jun 08, 2026 at 10:36:53AM +0300, Laurent Pinchart wrote:
> > > > On Mon, Jun 08, 2026 at 12:53:50AM +0300, Sakari Ailus wrote:
> > > > > The PIXEL_RATE control exists to convey the value to the userspace and has
> > > > > no configuration that would need to be programmed to the sensor. Make the
> > > > > control handler ops for the PIXEL_RATE control NULL and avoid a warning
> > > > > (as well as returning an error) from the driver.
> > > >
> > > > I thought the standard way to handle pixel rate being read only was to
> > > > set the V4L2_CTRL_FLAG_READ_ONLY flag, like we do for e.g.
> > > > V4L2_CID_LINK_FREQ. Is that not correct ?
> > > 
> > > PIXEL_RATE is RO by default
> > > 
> > > drivers/media/v4l2-core/v4l2-ctrls-defs.c:      case V4L2_CID_PIXEL_RATE:
> > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *type = V4L2_CTRL_TYPE_INTEGER64;
> > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *flags |= V4L2_CTRL_FLAG_READ_ONLY;
> > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              break;
> > > 
> > > The purpose of setting the ctrl_ops member to NULL is to avoid having
> > > to handle RO controls in the driver implementation of .s_ctrl().
> > 
> > Shouldn't the V4L2 control framework avoid .s_ctrl() calls for read-only
> > controls ? I thought it did already.
> 
> The control may be read-only on the UAPI but the driver could still do
> something about it in its s_ctrl() callback. I don't know if any driver
> depends on this though.

It seems to be one of the many areas where control handling should be
simplified for drivers.

In any case, the imx219 driver creates the V4L2_CID_LINK_FREQ control
with a non-NULL ops pointer, sets the V4L2_CTRL_FLAG_READ_ONLY flag, and
does not handle V4L2_CID_LINK_FREQ in imx219_set_ctrl(). If there's an
issue for V4L2_CID_PIXEL_RATE there is also an issue for
V4L2_CID_LINK_FREQ.

Maybe the best short term fix would be to drop the dev_info() in the
default case of the ctrl->id switch in imx219_set_ctrl() ?

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 04/10] media: imx219: Make control handler ops for PIXEL_RATE NULL

[Sakari Ailus]

On Mon, Jun 08, 2026 at 11:24:26AM +0300, Laurent Pinchart wrote:
> On Mon, Jun 08, 2026 at 11:14:32AM +0300, Sakari Ailus wrote:
> > On Mon, Jun 08, 2026 at 11:03:38AM +0300, Laurent Pinchart wrote:
> > > On Mon, Jun 08, 2026 at 09:53:17AM +0200, Jacopo Mondi wrote:
> > > > Hi Laurent
> > > >   sorry if I reply in place of Sakari but I got this fresh
> > > 
> > > Thanks :-)
> > > 
> > > > On Mon, Jun 08, 2026 at 10:36:53AM +0300, Laurent Pinchart wrote:
> > > > > On Mon, Jun 08, 2026 at 12:53:50AM +0300, Sakari Ailus wrote:
> > > > > > The PIXEL_RATE control exists to convey the value to the userspace and has
> > > > > > no configuration that would need to be programmed to the sensor. Make the
> > > > > > control handler ops for the PIXEL_RATE control NULL and avoid a warning
> > > > > > (as well as returning an error) from the driver.
> > > > >
> > > > > I thought the standard way to handle pixel rate being read only was to
> > > > > set the V4L2_CTRL_FLAG_READ_ONLY flag, like we do for e.g.
> > > > > V4L2_CID_LINK_FREQ. Is that not correct ?
> > > > 
> > > > PIXEL_RATE is RO by default
> > > > 
> > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c:      case V4L2_CID_PIXEL_RATE:
> > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *type = V4L2_CTRL_TYPE_INTEGER64;
> > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *flags |= V4L2_CTRL_FLAG_READ_ONLY;
> > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              break;
> > > > 
> > > > The purpose of setting the ctrl_ops member to NULL is to avoid having
> > > > to handle RO controls in the driver implementation of .s_ctrl().
> > > 
> > > Shouldn't the V4L2 control framework avoid .s_ctrl() calls for read-only
> > > controls ? I thought it did already.
> > 
> > The control may be read-only on the UAPI but the driver could still do
> > something about it in its s_ctrl() callback. I don't know if any driver
> > depends on this though.
> 
> It seems to be one of the many areas where control handling should be
> simplified for drivers.
> 
> In any case, the imx219 driver creates the V4L2_CID_LINK_FREQ control
> with a non-NULL ops pointer, sets the V4L2_CTRL_FLAG_READ_ONLY flag, and
> does not handle V4L2_CID_LINK_FREQ in imx219_set_ctrl(). If there's an
> issue for V4L2_CID_PIXEL_RATE there is also an issue for
> V4L2_CID_LINK_FREQ.

The ops should be set to NULL for link_freq as well.

> 
> Maybe the best short term fix would be to drop the dev_info() in the
> default case of the ctrl->id switch in imx219_set_ctrl() ?

Any reason why not to set ops NULL instead?

-- 
Sakari Ailus

Re: [PATCH v5 04/10] media: imx219: Make control handler ops for PIXEL_RATE NULL

[Laurent Pinchart]

On Mon, Jun 08, 2026 at 01:21:22PM +0300, Sakari Ailus wrote:
> On Mon, Jun 08, 2026 at 11:24:26AM +0300, Laurent Pinchart wrote:
> > On Mon, Jun 08, 2026 at 11:14:32AM +0300, Sakari Ailus wrote:
> > > On Mon, Jun 08, 2026 at 11:03:38AM +0300, Laurent Pinchart wrote:
> > > > On Mon, Jun 08, 2026 at 09:53:17AM +0200, Jacopo Mondi wrote:
> > > > > Hi Laurent
> > > > >   sorry if I reply in place of Sakari but I got this fresh
> > > > 
> > > > Thanks :-)
> > > > 
> > > > > On Mon, Jun 08, 2026 at 10:36:53AM +0300, Laurent Pinchart wrote:
> > > > > > On Mon, Jun 08, 2026 at 12:53:50AM +0300, Sakari Ailus wrote:
> > > > > > > The PIXEL_RATE control exists to convey the value to the userspace and has
> > > > > > > no configuration that would need to be programmed to the sensor. Make the
> > > > > > > control handler ops for the PIXEL_RATE control NULL and avoid a warning
> > > > > > > (as well as returning an error) from the driver.
> > > > > >
> > > > > > I thought the standard way to handle pixel rate being read only was to
> > > > > > set the V4L2_CTRL_FLAG_READ_ONLY flag, like we do for e.g.
> > > > > > V4L2_CID_LINK_FREQ. Is that not correct ?
> > > > > 
> > > > > PIXEL_RATE is RO by default
> > > > > 
> > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c:      case V4L2_CID_PIXEL_RATE:
> > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *type = V4L2_CTRL_TYPE_INTEGER64;
> > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *flags |= V4L2_CTRL_FLAG_READ_ONLY;
> > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              break;
> > > > > 
> > > > > The purpose of setting the ctrl_ops member to NULL is to avoid having
> > > > > to handle RO controls in the driver implementation of .s_ctrl().
> > > > 
> > > > Shouldn't the V4L2 control framework avoid .s_ctrl() calls for read-only
> > > > controls ? I thought it did already.
> > > 
> > > The control may be read-only on the UAPI but the driver could still do
> > > something about it in its s_ctrl() callback. I don't know if any driver
> > > depends on this though.
> > 
> > It seems to be one of the many areas where control handling should be
> > simplified for drivers.
> > 
> > In any case, the imx219 driver creates the V4L2_CID_LINK_FREQ control
> > with a non-NULL ops pointer, sets the V4L2_CTRL_FLAG_READ_ONLY flag, and
> > does not handle V4L2_CID_LINK_FREQ in imx219_set_ctrl(). If there's an
> > issue for V4L2_CID_PIXEL_RATE there is also an issue for
> > V4L2_CID_LINK_FREQ.
> 
> The ops should be set to NULL for link_freq as well.
> 
> > Maybe the best short term fix would be to drop the dev_info() in the
> > default case of the ctrl->id switch in imx219_set_ctrl() ?
> 
> Any reason why not to set ops NULL instead?

Because that seems to be a hack. Drivers shouldn't have to set a NULL
ops pointer for read-only controls, when there's already a read-only
flag. I'd like to simplify the code on the driver side and handle this
in the control framework, not adding yet another arcane rule that most
driver authors will not be aware of.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 04/10] media: imx219: Make control handler ops for PIXEL_RATE NULL

[Sakari Ailus]

On Mon, Jun 08, 2026 at 01:27:55PM +0300, Laurent Pinchart wrote:
> On Mon, Jun 08, 2026 at 01:21:22PM +0300, Sakari Ailus wrote:
> > On Mon, Jun 08, 2026 at 11:24:26AM +0300, Laurent Pinchart wrote:
> > > On Mon, Jun 08, 2026 at 11:14:32AM +0300, Sakari Ailus wrote:
> > > > On Mon, Jun 08, 2026 at 11:03:38AM +0300, Laurent Pinchart wrote:
> > > > > On Mon, Jun 08, 2026 at 09:53:17AM +0200, Jacopo Mondi wrote:
> > > > > > Hi Laurent
> > > > > >   sorry if I reply in place of Sakari but I got this fresh
> > > > > 
> > > > > Thanks :-)
> > > > > 
> > > > > > On Mon, Jun 08, 2026 at 10:36:53AM +0300, Laurent Pinchart wrote:
> > > > > > > On Mon, Jun 08, 2026 at 12:53:50AM +0300, Sakari Ailus wrote:
> > > > > > > > The PIXEL_RATE control exists to convey the value to the userspace and has
> > > > > > > > no configuration that would need to be programmed to the sensor. Make the
> > > > > > > > control handler ops for the PIXEL_RATE control NULL and avoid a warning
> > > > > > > > (as well as returning an error) from the driver.
> > > > > > >
> > > > > > > I thought the standard way to handle pixel rate being read only was to
> > > > > > > set the V4L2_CTRL_FLAG_READ_ONLY flag, like we do for e.g.
> > > > > > > V4L2_CID_LINK_FREQ. Is that not correct ?
> > > > > > 
> > > > > > PIXEL_RATE is RO by default
> > > > > > 
> > > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c:      case V4L2_CID_PIXEL_RATE:
> > > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *type = V4L2_CTRL_TYPE_INTEGER64;
> > > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *flags |= V4L2_CTRL_FLAG_READ_ONLY;
> > > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              break;
> > > > > > 
> > > > > > The purpose of setting the ctrl_ops member to NULL is to avoid having
> > > > > > to handle RO controls in the driver implementation of .s_ctrl().
> > > > > 
> > > > > Shouldn't the V4L2 control framework avoid .s_ctrl() calls for read-only
> > > > > controls ? I thought it did already.
> > > > 
> > > > The control may be read-only on the UAPI but the driver could still do
> > > > something about it in its s_ctrl() callback. I don't know if any driver
> > > > depends on this though.
> > > 
> > > It seems to be one of the many areas where control handling should be
> > > simplified for drivers.
> > > 
> > > In any case, the imx219 driver creates the V4L2_CID_LINK_FREQ control
> > > with a non-NULL ops pointer, sets the V4L2_CTRL_FLAG_READ_ONLY flag, and
> > > does not handle V4L2_CID_LINK_FREQ in imx219_set_ctrl(). If there's an
> > > issue for V4L2_CID_PIXEL_RATE there is also an issue for
> > > V4L2_CID_LINK_FREQ.
> > 
> > The ops should be set to NULL for link_freq as well.
> > 
> > > Maybe the best short term fix would be to drop the dev_info() in the
> > > default case of the ctrl->id switch in imx219_set_ctrl() ?
> > 
> > Any reason why not to set ops NULL instead?
> 
> Because that seems to be a hack. Drivers shouldn't have to set a NULL
> ops pointer for read-only controls, when there's already a read-only
> flag. I'd like to simplify the code on the driver side and handle this
> in the control framework, not adding yet another arcane rule that most
> driver authors will not be aware of.

I don't think I'd necessarily call it a hack.

The control may be changeable, but not by the user. If the driver is just
setting the value without going through the control framework, control
events will be omitted.

-- 
Sakari Ailus

Re: [PATCH v5 04/10] media: imx219: Make control handler ops for PIXEL_RATE NULL

[Laurent Pinchart]

On Mon, Jun 08, 2026 at 04:47:03PM +0300, Sakari Ailus wrote:
> On Mon, Jun 08, 2026 at 01:27:55PM +0300, Laurent Pinchart wrote:
> > On Mon, Jun 08, 2026 at 01:21:22PM +0300, Sakari Ailus wrote:
> > > On Mon, Jun 08, 2026 at 11:24:26AM +0300, Laurent Pinchart wrote:
> > > > On Mon, Jun 08, 2026 at 11:14:32AM +0300, Sakari Ailus wrote:
> > > > > On Mon, Jun 08, 2026 at 11:03:38AM +0300, Laurent Pinchart wrote:
> > > > > > On Mon, Jun 08, 2026 at 09:53:17AM +0200, Jacopo Mondi wrote:
> > > > > > > Hi Laurent
> > > > > > >   sorry if I reply in place of Sakari but I got this fresh
> > > > > > 
> > > > > > Thanks :-)
> > > > > > 
> > > > > > > On Mon, Jun 08, 2026 at 10:36:53AM +0300, Laurent Pinchart wrote:
> > > > > > > > On Mon, Jun 08, 2026 at 12:53:50AM +0300, Sakari Ailus wrote:
> > > > > > > > > The PIXEL_RATE control exists to convey the value to the userspace and has
> > > > > > > > > no configuration that would need to be programmed to the sensor. Make the
> > > > > > > > > control handler ops for the PIXEL_RATE control NULL and avoid a warning
> > > > > > > > > (as well as returning an error) from the driver.
> > > > > > > >
> > > > > > > > I thought the standard way to handle pixel rate being read only was to
> > > > > > > > set the V4L2_CTRL_FLAG_READ_ONLY flag, like we do for e.g.
> > > > > > > > V4L2_CID_LINK_FREQ. Is that not correct ?
> > > > > > > 
> > > > > > > PIXEL_RATE is RO by default
> > > > > > > 
> > > > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c:      case V4L2_CID_PIXEL_RATE:
> > > > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *type = V4L2_CTRL_TYPE_INTEGER64;
> > > > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              *flags |= V4L2_CTRL_FLAG_READ_ONLY;
> > > > > > > drivers/media/v4l2-core/v4l2-ctrls-defs.c-              break;
> > > > > > > 
> > > > > > > The purpose of setting the ctrl_ops member to NULL is to avoid having
> > > > > > > to handle RO controls in the driver implementation of .s_ctrl().
> > > > > > 
> > > > > > Shouldn't the V4L2 control framework avoid .s_ctrl() calls for read-only
> > > > > > controls ? I thought it did already.
> > > > > 
> > > > > The control may be read-only on the UAPI but the driver could still do
> > > > > something about it in its s_ctrl() callback. I don't know if any driver
> > > > > depends on this though.
> > > > 
> > > > It seems to be one of the many areas where control handling should be
> > > > simplified for drivers.
> > > > 
> > > > In any case, the imx219 driver creates the V4L2_CID_LINK_FREQ control
> > > > with a non-NULL ops pointer, sets the V4L2_CTRL_FLAG_READ_ONLY flag, and
> > > > does not handle V4L2_CID_LINK_FREQ in imx219_set_ctrl(). If there's an
> > > > issue for V4L2_CID_PIXEL_RATE there is also an issue for
> > > > V4L2_CID_LINK_FREQ.
> > > 
> > > The ops should be set to NULL for link_freq as well.
> > > 
> > > > Maybe the best short term fix would be to drop the dev_info() in the
> > > > default case of the ctrl->id switch in imx219_set_ctrl() ?
> > > 
> > > Any reason why not to set ops NULL instead?
> > 
> > Because that seems to be a hack. Drivers shouldn't have to set a NULL
> > ops pointer for read-only controls, when there's already a read-only
> > flag. I'd like to simplify the code on the driver side and handle this
> > in the control framework, not adding yet another arcane rule that most
> > driver authors will not be aware of.
> 
> I don't think I'd necessarily call it a hack.

It's still yet another undocumented behaviour to will be copied through
cargo-cult in a subset of drivers, making the code base more difficult
to understand and maintain. The fact that this patch addressed the
PIXEL_RATE control but not the LINK_FREQUENCY control proves my concerns
are valid :-)

I'd like to see one scheme clearly documented, and used by all drivers.
Let's first focus on selecting one scheme and documenting it. Hans'
opinion would be useful.

> The control may be changeable, but not by the user. If the driver is just
> setting the value without going through the control framework, control
> events will be omitted.

-- 
Regards,

Laurent Pinchart

[PATCH v5 05/10] media: imx219: Rename "binning" as "bin_hv" in imx219_set_pad_format

[Sakari Ailus]

Rename "binning" as "bin_hv" in anticipation of having a variable called
"binning" for another purpose.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/i2c/imx219.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 86683fe8a79e..3aebcbaa3fcd 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -837,7 +837,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	const struct imx219_mode *mode;
 	struct v4l2_mbus_framefmt *format;
 	struct v4l2_rect *crop;
-	u8 bin_h, bin_v, binning;
+	u8 bin_h, bin_v, bin_hv;
 	int ret;
 
 	format = v4l2_subdev_state_get_format(state, 0);
@@ -870,11 +870,11 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	bin_v = min(IMX219_ACTIVE_AREA_HEIGHT / format->height, 2U);
 
 	/* Ensure bin_h and bin_v are same to avoid 1:2 or 2:1 stretching */
-	binning = min(bin_h, bin_v);
+	bin_hv = min(bin_h, bin_v);
 
 	crop = v4l2_subdev_state_get_crop(state, 0);
-	crop->width = format->width * binning;
-	crop->height = format->height * binning;
+	crop->width = format->width * bin_hv;
+	crop->height = format->height * bin_hv;
 	crop->left = (IMX219_NATIVE_WIDTH - crop->width) / 2;
 	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
 
-- 
2.47.3

Re: [PATCH v5 05/10] media: imx219: Rename "binning" as "bin_hv" in imx219_set_pad_format

[Dave Stevenson]

On Sun, 7 Jun 2026 at 22:54, Sakari Ailus <sakari.ailus@linux.intel.com> wrote:
>
> Rename "binning" as "bin_hv" in anticipation of having a variable called
> "binning" for another purpose.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

Reviewed-by: Dave Stevenson <dave.stevenson@raspberrypi.com>

> ---
>  drivers/media/i2c/imx219.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index 86683fe8a79e..3aebcbaa3fcd 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -837,7 +837,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>         const struct imx219_mode *mode;
>         struct v4l2_mbus_framefmt *format;
>         struct v4l2_rect *crop;
> -       u8 bin_h, bin_v, binning;
> +       u8 bin_h, bin_v, bin_hv;
>         int ret;
>
>         format = v4l2_subdev_state_get_format(state, 0);
> @@ -870,11 +870,11 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>         bin_v = min(IMX219_ACTIVE_AREA_HEIGHT / format->height, 2U);
>
>         /* Ensure bin_h and bin_v are same to avoid 1:2 or 2:1 stretching */
> -       binning = min(bin_h, bin_v);
> +       bin_hv = min(bin_h, bin_v);
>
>         crop = v4l2_subdev_state_get_crop(state, 0);
> -       crop->width = format->width * binning;
> -       crop->height = format->height * binning;
> +       crop->width = format->width * bin_hv;
> +       crop->height = format->height * bin_hv;
>         crop->left = (IMX219_NATIVE_WIDTH - crop->width) / 2;
>         crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
>
> --
> 2.47.3
>

[PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[Sakari Ailus]

When vertical analogue binning is in use, the minimum frame length in
lines decreases to around half of the normal. In relation to the sensor's
output size this means vertical blanking can be negative but that's not an
issue as control values are signed. Remove the workaround for this
non-issue that doubled the pixel rate, frame length in lines and exposure
time.

The resulting change also fixes the minimum, the maximum and the step
values for the control.

Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 37 ++++++++++++++-----------------------
 1 file changed, 14 insertions(+), 23 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 3aebcbaa3fcd..3cee31758b7e 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -420,15 +420,6 @@ static void imx219_get_binning(struct v4l2_subdev_state *state, u8 *bin_h,
 
 }
 
-static inline u32 imx219_get_rate_factor(struct v4l2_subdev_state *state)
-{
-	u8 bin_h, bin_v;
-
-	imx219_get_binning(state, &bin_h, &bin_v);
-
-	return (bin_h & bin_v) == IMX219_BINNING_X2_ANALOG ? 2 : 1;
-}
-
 /* -----------------------------------------------------------------------------
  * Controls
  */
@@ -440,19 +431,17 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
 	struct i2c_client *client = v4l2_get_subdevdata(&imx219->sd);
 	const struct v4l2_mbus_framefmt *format;
 	struct v4l2_subdev_state *state;
-	u32 rate_factor;
 	int ret = 0;
 
 	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
 	format = v4l2_subdev_state_get_format(state, 0);
-	rate_factor = imx219_get_rate_factor(state);
 
 	if (ctrl->id == V4L2_CID_VBLANK) {
 		int exposure_max, exposure_def;
 
 		/* Update max exposure while meeting expected vblanking */
 		exposure_max = format->height + ctrl->val -
-			IMX219_EXPOSURE_OFFSET * rate_factor;
+			IMX219_EXPOSURE_OFFSET;
 		exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
 				exposure_max : IMX219_EXPOSURE_DEFAULT;
 		ret = __v4l2_ctrl_modify_range(imx219->exposure,
@@ -479,7 +468,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
 		break;
 	case V4L2_CID_EXPOSURE:
 		cci_write(imx219->regmap, IMX219_REG_EXPOSURE,
-			  ctrl->val / rate_factor, &ret);
+			  ctrl->val, &ret);
 		break;
 	case V4L2_CID_DIGITAL_GAIN:
 		cci_write(imx219->regmap, IMX219_REG_DIGITAL_GAIN,
@@ -496,7 +485,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
 		break;
 	case V4L2_CID_VBLANK:
 		cci_write(imx219->regmap, IMX219_REG_FRM_LENGTH_A,
-			  (format->height + ctrl->val) / rate_factor, &ret);
+			  format->height + ctrl->val, &ret);
 		break;
 	case V4L2_CID_HBLANK:
 		cci_write(imx219->regmap, IMX219_REG_LINE_LENGTH_A,
@@ -837,8 +826,8 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	const struct imx219_mode *mode;
 	struct v4l2_mbus_framefmt *format;
 	struct v4l2_rect *crop;
-	u8 bin_h, bin_v, bin_hv;
-	int ret;
+	u8 bin_h, bin_v;
+	int ret, bin_hv;
 
 	format = v4l2_subdev_state_get_format(state, 0);
 
@@ -879,23 +868,25 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
 
 	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
-		unsigned int rate_factor = imx219_get_rate_factor(state);
 		int exposure_max;
 		int exposure_def;
 		int llp_min;
 		int pixel_rate;
 
 		/* Update limits and set FPS to default */
+		int vblank_min = ((int)mode->height * (1 - bin_hv) / bin_hv) +
+			IMX219_VBLANK_MIN;
 		ret = __v4l2_ctrl_modify_range(imx219->vblank,
-					       IMX219_VBLANK_MIN * rate_factor,
-					       (IMX219_FLL_MAX - mode->height) *
-					       rate_factor, rate_factor,
-					       mode->fll_def - mode->height);
+					       vblank_min,
+					       IMX219_FLL_MAX - mode->height, 1,
+					       (int)(mode->fll_def / bin_hv) -
+					       (int)mode->height);
 		if (ret)
 			return ret;
 
 		ret = __v4l2_ctrl_s_ctrl(imx219->vblank,
-					 mode->fll_def - mode->height);
+					 (int)(mode->fll_def / bin_hv) -
+					 (int)mode->height);
 		if (ret)
 			return ret;
 
@@ -932,7 +923,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 			return ret;
 
 		/* Scale the pixel rate based on the mode specific factor */
-		pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
+		pixel_rate = imx219_get_pixel_rate(imx219);
 		ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
 					       pixel_rate, 1, pixel_rate);
 		if (ret)
-- 
2.47.3

Re: [PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Unscaled `exposure_max` allows userspace to request exposure times that exceed the physical frame length during analog binning.
--

commit 5bb0595912a9dcd70c3d849b873e9a213a70ec05
Author: Sakari Ailus <sakari.ailus@linux.intel.com>

media: imx219: Fix vertical blanking and exposure for analogue binning

This commit removes a workaround that doubled the pixel rate, frame length
in lines, and exposure time for analog binning. It updates the minimum, maximum,
and step values for the control.

> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index 3aebcbaa3fcd6..3cee31758b7e9 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
[ ... ]
> @@ -879,23 +868,25 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
>  
>  	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
[ ... ]
>  		/* Update limits and set FPS to default */
> +		int vblank_min = ((int)mode->height * (1 - bin_hv) / bin_hv) +
> +			IMX219_VBLANK_MIN;
>  		ret = __v4l2_ctrl_modify_range(imx219->vblank,
> -					       IMX219_VBLANK_MIN * rate_factor,
> -					       (IMX219_FLL_MAX - mode->height) *
> -					       rate_factor, rate_factor,
> -					       mode->fll_def - mode->height);
> +					       vblank_min,
> +					       IMX219_FLL_MAX - mode->height, 1,
> +					       (int)(mode->fll_def / bin_hv) -
> +					       (int)mode->height);
>  		if (ret)
>  			return ret;
>  
>  		ret = __v4l2_ctrl_s_ctrl(imx219->vblank,
> -					 mode->fll_def - mode->height);
> +					 (int)(mode->fll_def / bin_hv) -
> +					 (int)mode->height);
>  		if (ret)
>  			return ret;

[Severity: High]
Does this code allow userspace to request an exposure time that exceeds the
physical frame length?

When vblank is initialized via __v4l2_ctrl_s_ctrl() above, it dynamically
computes and sets the correct upper limit for the exposure control under
the hood.

However, immediately following this in imx219_set_pad_format(), the code uses
the unscaled mode->fll_def to modify the exposure limits:

		/* Update max exposure while meeting expected vblanking */
		exposure_max = mode->fll_def - IMX219_EXPOSURE_OFFSET;
		exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
				exposure_max : IMX219_EXPOSURE_DEFAULT;
		ret = __v4l2_ctrl_modify_range(imx219->exposure,
					       imx219->exposure->minimum,
					       exposure_max,
					       imx219->exposure->step,
					       exposure_def);

Since this patch removes the rate_factor division when writing V4L2_CID_EXPOSURE
to the sensor hardware in imx219_set_ctrl(), and exposure_max is not scaled
by bin_hv here, could this overwrite the safely computed hardware bound with an
unscaled maximum?

If so, userspace could successfully request an exposure value (e.g., 1703) while
the physical frame length is only 853, which might violate the fundamental
sensor constraint that exposure time must be less than the frame length.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260607215356.842932-1-sakari.ailus@linux.intel.com?part=6

Re: [PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[Jacopo Mondi]

Hi Sakari

On Mon, Jun 08, 2026 at 12:53:52AM +0300, Sakari Ailus wrote:
> When vertical analogue binning is in use, the minimum frame length in
> lines decreases to around half of the normal. In relation to the sensor's
> output size this means vertical blanking can be negative but that's not an
> issue as control values are signed. Remove the workaround for this

Didn't we just discussed two weeks ago in media summit how negative
blankings are a bad idea, and of all drivers one could decide to play
with imx219 is probably the worse due it's large use base and the fact
libcamera doesn't support negative blankings ?

Have I missed something ?

> non-issue that doubled the pixel rate, frame length in lines and exposure
> time.
>
> The resulting change also fixes the minimum, the maximum and the step
> values for the control.
>
> Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/i2c/imx219.c | 37 ++++++++++++++-----------------------
>  1 file changed, 14 insertions(+), 23 deletions(-)
>
> diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> index 3aebcbaa3fcd..3cee31758b7e 100644
> --- a/drivers/media/i2c/imx219.c
> +++ b/drivers/media/i2c/imx219.c
> @@ -420,15 +420,6 @@ static void imx219_get_binning(struct v4l2_subdev_state *state, u8 *bin_h,
>
>  }
>
> -static inline u32 imx219_get_rate_factor(struct v4l2_subdev_state *state)
> -{
> -	u8 bin_h, bin_v;
> -
> -	imx219_get_binning(state, &bin_h, &bin_v);
> -
> -	return (bin_h & bin_v) == IMX219_BINNING_X2_ANALOG ? 2 : 1;
> -}
> -
>  /* -----------------------------------------------------------------------------
>   * Controls
>   */
> @@ -440,19 +431,17 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
>  	struct i2c_client *client = v4l2_get_subdevdata(&imx219->sd);
>  	const struct v4l2_mbus_framefmt *format;
>  	struct v4l2_subdev_state *state;
> -	u32 rate_factor;
>  	int ret = 0;
>
>  	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
>  	format = v4l2_subdev_state_get_format(state, 0);
> -	rate_factor = imx219_get_rate_factor(state);
>
>  	if (ctrl->id == V4L2_CID_VBLANK) {
>  		int exposure_max, exposure_def;
>
>  		/* Update max exposure while meeting expected vblanking */
>  		exposure_max = format->height + ctrl->val -
> -			IMX219_EXPOSURE_OFFSET * rate_factor;
> +			IMX219_EXPOSURE_OFFSET;
>  		exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
>  				exposure_max : IMX219_EXPOSURE_DEFAULT;
>  		ret = __v4l2_ctrl_modify_range(imx219->exposure,
> @@ -479,7 +468,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
>  		break;
>  	case V4L2_CID_EXPOSURE:
>  		cci_write(imx219->regmap, IMX219_REG_EXPOSURE,
> -			  ctrl->val / rate_factor, &ret);
> +			  ctrl->val, &ret);
>  		break;
>  	case V4L2_CID_DIGITAL_GAIN:
>  		cci_write(imx219->regmap, IMX219_REG_DIGITAL_GAIN,
> @@ -496,7 +485,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
>  		break;
>  	case V4L2_CID_VBLANK:
>  		cci_write(imx219->regmap, IMX219_REG_FRM_LENGTH_A,
> -			  (format->height + ctrl->val) / rate_factor, &ret);
> +			  format->height + ctrl->val, &ret);
>  		break;
>  	case V4L2_CID_HBLANK:
>  		cci_write(imx219->regmap, IMX219_REG_LINE_LENGTH_A,
> @@ -837,8 +826,8 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  	const struct imx219_mode *mode;
>  	struct v4l2_mbus_framefmt *format;
>  	struct v4l2_rect *crop;
> -	u8 bin_h, bin_v, bin_hv;
> -	int ret;
> +	u8 bin_h, bin_v;
> +	int ret, bin_hv;
>
>  	format = v4l2_subdev_state_get_format(state, 0);
>
> @@ -879,23 +868,25 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
>
>  	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> -		unsigned int rate_factor = imx219_get_rate_factor(state);
>  		int exposure_max;
>  		int exposure_def;
>  		int llp_min;
>  		int pixel_rate;
>
>  		/* Update limits and set FPS to default */
> +		int vblank_min = ((int)mode->height * (1 - bin_hv) / bin_hv) +
> +			IMX219_VBLANK_MIN;
>  		ret = __v4l2_ctrl_modify_range(imx219->vblank,
> -					       IMX219_VBLANK_MIN * rate_factor,
> -					       (IMX219_FLL_MAX - mode->height) *
> -					       rate_factor, rate_factor,
> -					       mode->fll_def - mode->height);
> +					       vblank_min,
> +					       IMX219_FLL_MAX - mode->height, 1,
> +					       (int)(mode->fll_def / bin_hv) -
> +					       (int)mode->height);
>  		if (ret)
>  			return ret;
>
>  		ret = __v4l2_ctrl_s_ctrl(imx219->vblank,
> -					 mode->fll_def - mode->height);
> +					 (int)(mode->fll_def / bin_hv) -
> +					 (int)mode->height);
>  		if (ret)
>  			return ret;
>
> @@ -932,7 +923,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
>  			return ret;
>
>  		/* Scale the pixel rate based on the mode specific factor */
> -		pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
> +		pixel_rate = imx219_get_pixel_rate(imx219);
>  		ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
>  					       pixel_rate, 1, pixel_rate);
>  		if (ret)
> --
> 2.47.3
>
>

Re: [PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[Laurent Pinchart]

On Mon, Jun 08, 2026 at 08:58:46AM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Jun 08, 2026 at 12:53:52AM +0300, Sakari Ailus wrote:
> > When vertical analogue binning is in use, the minimum frame length in
> > lines decreases to around half of the normal. In relation to the sensor's
> > output size this means vertical blanking can be negative but that's not an
> > issue as control values are signed. Remove the workaround for this
> 
> Didn't we just discussed two weeks ago in media summit how negative
> blankings are a bad idea, and of all drivers one could decide to play
> with imx219 is probably the worse due it's large use base and the fact
> libcamera doesn't support negative blankings ?

I also think that negative blanking values are a bad idea, for this
driver or any other driver. I still haven't seen any compelling
argument.

> Have I missed something ?
> 
> > non-issue that doubled the pixel rate, frame length in lines and exposure
> > time.
> >
> > The resulting change also fixes the minimum, the maximum and the step
> > values for the control.
> >
> > Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/i2c/imx219.c | 37 ++++++++++++++-----------------------
> >  1 file changed, 14 insertions(+), 23 deletions(-)
> >
> > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > index 3aebcbaa3fcd..3cee31758b7e 100644
> > --- a/drivers/media/i2c/imx219.c
> > +++ b/drivers/media/i2c/imx219.c
> > @@ -420,15 +420,6 @@ static void imx219_get_binning(struct v4l2_subdev_state *state, u8 *bin_h,
> >
> >  }
> >
> > -static inline u32 imx219_get_rate_factor(struct v4l2_subdev_state *state)
> > -{
> > -	u8 bin_h, bin_v;
> > -
> > -	imx219_get_binning(state, &bin_h, &bin_v);
> > -
> > -	return (bin_h & bin_v) == IMX219_BINNING_X2_ANALOG ? 2 : 1;
> > -}
> > -
> >  /* -----------------------------------------------------------------------------
> >   * Controls
> >   */
> > @@ -440,19 +431,17 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> >  	struct i2c_client *client = v4l2_get_subdevdata(&imx219->sd);
> >  	const struct v4l2_mbus_framefmt *format;
> >  	struct v4l2_subdev_state *state;
> > -	u32 rate_factor;
> >  	int ret = 0;
> >
> >  	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
> >  	format = v4l2_subdev_state_get_format(state, 0);
> > -	rate_factor = imx219_get_rate_factor(state);
> >
> >  	if (ctrl->id == V4L2_CID_VBLANK) {
> >  		int exposure_max, exposure_def;
> >
> >  		/* Update max exposure while meeting expected vblanking */
> >  		exposure_max = format->height + ctrl->val -
> > -			IMX219_EXPOSURE_OFFSET * rate_factor;
> > +			IMX219_EXPOSURE_OFFSET;
> >  		exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
> >  				exposure_max : IMX219_EXPOSURE_DEFAULT;
> >  		ret = __v4l2_ctrl_modify_range(imx219->exposure,
> > @@ -479,7 +468,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> >  		break;
> >  	case V4L2_CID_EXPOSURE:
> >  		cci_write(imx219->regmap, IMX219_REG_EXPOSURE,
> > -			  ctrl->val / rate_factor, &ret);
> > +			  ctrl->val, &ret);
> >  		break;
> >  	case V4L2_CID_DIGITAL_GAIN:
> >  		cci_write(imx219->regmap, IMX219_REG_DIGITAL_GAIN,
> > @@ -496,7 +485,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> >  		break;
> >  	case V4L2_CID_VBLANK:
> >  		cci_write(imx219->regmap, IMX219_REG_FRM_LENGTH_A,
> > -			  (format->height + ctrl->val) / rate_factor, &ret);
> > +			  format->height + ctrl->val, &ret);
> >  		break;
> >  	case V4L2_CID_HBLANK:
> >  		cci_write(imx219->regmap, IMX219_REG_LINE_LENGTH_A,
> > @@ -837,8 +826,8 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >  	const struct imx219_mode *mode;
> >  	struct v4l2_mbus_framefmt *format;
> >  	struct v4l2_rect *crop;
> > -	u8 bin_h, bin_v, bin_hv;
> > -	int ret;
> > +	u8 bin_h, bin_v;
> > +	int ret, bin_hv;
> >
> >  	format = v4l2_subdev_state_get_format(state, 0);
> >
> > @@ -879,23 +868,25 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >  	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
> >
> >  	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> > -		unsigned int rate_factor = imx219_get_rate_factor(state);
> >  		int exposure_max;
> >  		int exposure_def;
> >  		int llp_min;
> >  		int pixel_rate;
> >
> >  		/* Update limits and set FPS to default */
> > +		int vblank_min = ((int)mode->height * (1 - bin_hv) / bin_hv) +
> > +			IMX219_VBLANK_MIN;
> >  		ret = __v4l2_ctrl_modify_range(imx219->vblank,
> > -					       IMX219_VBLANK_MIN * rate_factor,
> > -					       (IMX219_FLL_MAX - mode->height) *
> > -					       rate_factor, rate_factor,
> > -					       mode->fll_def - mode->height);
> > +					       vblank_min,
> > +					       IMX219_FLL_MAX - mode->height, 1,
> > +					       (int)(mode->fll_def / bin_hv) -
> > +					       (int)mode->height);
> >  		if (ret)
> >  			return ret;
> >
> >  		ret = __v4l2_ctrl_s_ctrl(imx219->vblank,
> > -					 mode->fll_def - mode->height);
> > +					 (int)(mode->fll_def / bin_hv) -
> > +					 (int)mode->height);
> >  		if (ret)
> >  			return ret;
> >
> > @@ -932,7 +923,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >  			return ret;
> >
> >  		/* Scale the pixel rate based on the mode specific factor */
> > -		pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
> > +		pixel_rate = imx219_get_pixel_rate(imx219);
> >  		ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
> >  					       pixel_rate, 1, pixel_rate);
> >  		if (ret)

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[Sakari Ailus]

Hi Laurent, Jacopo,

On Mon, Jun 08, 2026 at 12:10:26PM +0300, Laurent Pinchart wrote:
> On Mon, Jun 08, 2026 at 08:58:46AM +0200, Jacopo Mondi wrote:
> > Hi Sakari
> > 
> > On Mon, Jun 08, 2026 at 12:53:52AM +0300, Sakari Ailus wrote:
> > > When vertical analogue binning is in use, the minimum frame length in
> > > lines decreases to around half of the normal. In relation to the sensor's
> > > output size this means vertical blanking can be negative but that's not an
> > > issue as control values are signed. Remove the workaround for this
> > 
> > Didn't we just discussed two weeks ago in media summit how negative
> > blankings are a bad idea, and of all drivers one could decide to play
> > with imx219 is probably the worse due it's large use base and the fact
> > libcamera doesn't support negative blankings ?
> 
> I also think that negative blanking values are a bad idea, for this
> driver or any other driver. I still haven't seen any compelling
> argument.

Note that the blanking controls haven't expressed blanking in other
configurations than those that do not use binning, either analogue or
digital, or cropping. The fact that negative values would result due to
sensor configuration does not mean the values would be somehow incorrect,
they simply do not reflect actual blanking configuration on the sensor.

In retrospect, we should have always had frame length in lines and line
length in pixels controls instead, or possibly besides the blanking
controls. But as the two blanking controls have been in use for conveying
frame length in lines and line length in pixels, relative to a reference
size, we can't re-purpose them for something else anymore without breaking
pretty much all userspace.

Multiplying the pixel rate and either frame length in lines or line length
in pixels by a constant does not make the blanking controls magically
correct.

I hope the above summarises my position in an understandable way.

I believe removing the rate_factor is necesary if we want to add support
for the Common Raw Sensor Model to the imx219 driver. Otherwise, we'll be
left with a single example driver only, the ov2740, which is an entirely
register list based driver.

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[Jai Luthra]

Hi Sakari,

Quoting Sakari Ailus (2026-06-08 19:37:34)
> Hi Laurent, Jacopo,
> 
> On Mon, Jun 08, 2026 at 12:10:26PM +0300, Laurent Pinchart wrote:
> > On Mon, Jun 08, 2026 at 08:58:46AM +0200, Jacopo Mondi wrote:
> > > Hi Sakari
> > > 
> > > On Mon, Jun 08, 2026 at 12:53:52AM +0300, Sakari Ailus wrote:
> > > > When vertical analogue binning is in use, the minimum frame length in
> > > > lines decreases to around half of the normal. In relation to the sensor's
> > > > output size this means vertical blanking can be negative but that's not an
> > > > issue as control values are signed. Remove the workaround for this
> > > 
> > > Didn't we just discussed two weeks ago in media summit how negative
> > > blankings are a bad idea, and of all drivers one could decide to play
> > > with imx219 is probably the worse due it's large use base and the fact
> > > libcamera doesn't support negative blankings ?
> > 
> > I also think that negative blanking values are a bad idea, for this
> > driver or any other driver. I still haven't seen any compelling
> > argument.
> 
> Note that the blanking controls haven't expressed blanking in other
> configurations than those that do not use binning, either analogue or
> digital, or cropping. The fact that negative values would result due to
> sensor configuration does not mean the values would be somehow incorrect,
> they simply do not reflect actual blanking configuration on the sensor.

I agree.. although what is the actual blanking configuration on the sensor
in this case?

I've been banging my head for a while to figure it out (my best guess in
the sibling thread)

> 
> In retrospect, we should have always had frame length in lines and line
> length in pixels controls instead, or possibly besides the blanking
> controls. But as the two blanking controls have been in use for conveying
> frame length in lines and line length in pixels, relative to a reference
> size, we can't re-purpose them for something else anymore without breaking
> pretty much all userspace.
> 
> Multiplying the pixel rate and either frame length in lines or line length
> in pixels by a constant does not make the blanking controls magically
> correct.
> 

Sure.. but the sensor must be reading out pixels along with non-negative
blanking internally. If that happens to not match the register values, do
you object to multiplying LLP or FLL with a constant?

Also what do you think of sensors like IMX678/IMX283 where the HTOT (LLP)
register is not in units of pixels at all?

> I hope the above summarises my position in an understandable way.
> 
> I believe removing the rate_factor is necesary if we want to add support
> for the Common Raw Sensor Model to the imx219 driver. Otherwise, we'll be
> left with a single example driver only, the ov2740, which is an entirely
> register list based driver.
> 

Agreed.

Thanks,
    Jai

> -- 
> Kind regards,
> 
> Sakari Ailus

Re: [PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[Laurent Pinchart]

On Mon, Jun 08, 2026 at 09:53:46PM +0530, Jai Luthra wrote:
> Quoting Sakari Ailus (2026-06-08 19:37:34)
> > On Mon, Jun 08, 2026 at 12:10:26PM +0300, Laurent Pinchart wrote:
> > > On Mon, Jun 08, 2026 at 08:58:46AM +0200, Jacopo Mondi wrote:
> > > > Hi Sakari
> > > > 
> > > > On Mon, Jun 08, 2026 at 12:53:52AM +0300, Sakari Ailus wrote:
> > > > > When vertical analogue binning is in use, the minimum frame length in
> > > > > lines decreases to around half of the normal. In relation to the sensor's
> > > > > output size this means vertical blanking can be negative but that's not an
> > > > > issue as control values are signed. Remove the workaround for this
> > > > 
> > > > Didn't we just discussed two weeks ago in media summit how negative
> > > > blankings are a bad idea, and of all drivers one could decide to play
> > > > with imx219 is probably the worse due it's large use base and the fact
> > > > libcamera doesn't support negative blankings ?
> > > 
> > > I also think that negative blanking values are a bad idea, for this
> > > driver or any other driver. I still haven't seen any compelling
> > > argument.
> > 
> > Note that the blanking controls haven't expressed blanking in other
> > configurations than those that do not use binning, either analogue or
> > digital, or cropping. The fact that negative values would result due to
> > sensor configuration does not mean the values would be somehow incorrect,
> > they simply do not reflect actual blanking configuration on the sensor.
> 
> I agree.. although what is the actual blanking configuration on the sensor
> in this case?
> 
> I've been banging my head for a while to figure it out (my best guess in
> the sibling thread)
> 
> > In retrospect, we should have always had frame length in lines and line
> > length in pixels controls instead, or possibly besides the blanking
> > controls. But as the two blanking controls have been in use for conveying
> > frame length in lines and line length in pixels, relative to a reference
> > size,

What's the reference size ? The controls are documented as

``V4L2_CID_VBLANK (integer)``
    Vertical blanking. The idle period after every frame during which no
    image data is produced. The unit of vertical blanking is a line.
    Every line has length of the image width plus horizontal blanking at
    the pixel rate defined by ``V4L2_CID_PIXEL_RATE`` control in the
    same sub-device.

``V4L2_CID_HBLANK (integer)``
    Horizontal blanking. The idle period after every line of image data
    during which no image data is produced. The unit of horizontal
    blanking is pixels.

This is compatible with usage of the output size as a reference. And
doing so wouldn't require negative blanking, would it ?

> > we can't re-purpose them for something else anymore without breaking
> > pretty much all userspace.
> > 
> > Multiplying the pixel rate and either frame length in lines or line length
> > in pixels by a constant does not make the blanking controls magically
> > correct.

The multiplication of the pixel rate may be the core of the issue. It's
getting late, I'll try to look at that tomorrow.

> Sure.. but the sensor must be reading out pixels along with non-negative
> blanking internally. If that happens to not match the register values, do
> you object to multiplying LLP or FLL with a constant?
> 
> Also what do you think of sensors like IMX678/IMX283 where the HTOT (LLP)
> register is not in units of pixels at all?
> 
> > I hope the above summarises my position in an understandable way.
> > 
> > I believe removing the rate_factor is necesary if we want to add support
> > for the Common Raw Sensor Model to the imx219 driver. Otherwise, we'll be
> > left with a single example driver only, the ov2740, which is an entirely
> > register list based driver.
> 
> Agreed.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[Jai Luthra]

Hi Jacopo, Sakari,
++ Dave, Hans and Laurent,

Quoting Jacopo Mondi (2026-06-08 12:28:46)
> Hi Sakari
> 
> On Mon, Jun 08, 2026 at 12:53:52AM +0300, Sakari Ailus wrote:
> > When vertical analogue binning is in use, the minimum frame length in
> > lines decreases to around half of the normal. In relation to the sensor's
> > output size this means vertical blanking can be negative but that's not an
> > issue as control values are signed. Remove the workaround for this
> 
> Didn't we just discussed two weeks ago in media summit how negative
> blankings are a bad idea, and of all drivers one could decide to play
> with imx219 is probably the worse due it's large use base and the fact
> libcamera doesn't support negative blankings ?
> 
> Have I missed something ?
> 

I think it would be helpful if I write down clearly how this sensor
operates (to the best of my knowledge) so we can decide on the correct
fix:

--------------------

IMX219 sensor has an active resolution of 3280x2464.

The driver currently programs the sensor VT pixel clock as fixed for a
given lane configuration.

In 2-lane mode it reads 182.4 MPixel/second:

    #define IMX219_PIXEL_RATE		182400000

And the framerate is given by:

    PIXEL_RATE / (FRAME_LENGTH * LINE_LENGTH)

where FRAME_LENGTH and LINE_LENGTH are registers that include the active
height and width along with blankings.

There are restrictions on the minimum of the LINE_LENGTH register and
minimum vertical blanking (32), which cap the framerate for the full resolution
mode.

    MIN_LINE_LENGTH: 0xd78 => 3448 pixels
    MIN_FRAME_LENGTH: ACTIVE_HEIGHT + MIN_VBLANK (32) = 2464 + 32
                           => 2496 lines

The maximum frame rate is

    182400000/(2496*3448) => ~ 21.2 frames/second

--------------------

A user might want to stream a lower resolution with the full field-of-view,
let's take 1640x1232 (which is exactly 1/2 of active area) as an example.

The sensor hardware can achieve this using two different binning modes:

    2x2-binning (regval: 0x1)
    2x2-analog-(special)-binning (regval: 0x3)

The sensor pipeline looks like:

active pixel array ->
    analogue crop (none) ->
        2x2 binning and ADC readout ->
                output to CSI-2 bus

The mode names suggest that binning can happen either before or after ADC,
but the datasheet is not very clear about the process. We can infer
some details though. See below..

--------------------

With the "normal" 2x2-binning mode the sensor allows programming
FRAME_LENGTH to a lower value. The driver still uses the min blanking of 32
lines, but the height is now 1232, half of the 2464 before.

    MIN_LINE_LENGTH: 0xd78 => 3448 pixels
    MIN_FRAME_LENGTH: READOUT_HEIGHT + MIN_VBLANK = 1232 + 32
                           => 1264 lines

The maximum frame rate is

    182400000/(1264*3448) => ~ 41.8 frames/second

--------------------

With the "special" 2x2-binning mode, the datasheet notes that FRAME_LENGTH
register should be in units of 2 Lines instead of 1 Line. This means
cutting it down by half once more:

    MIN_FRAME_LENGTH: (READOUT_HEIGHT + MIN_VBLANK)/2 = (1232 + 32)/2
                           => 632 lines

While there is a slightly higher minimum enforced for the line length:

    MIN_LINE_LENGTH: 0xde8 => 3560 pixels

The maximum frame rate is

    182400000/(632*3560) => ~ 81.0 frames/second

===================

Through the minimum allowed values of the FRAME_LENGTH and LINE_LENGTH
registers and maximum possible framerate, I think it is safe to say that:

    2x2-binning => Readout half the pixels (do vertical averaging in the
                                            analogue domain, before ADC
                                            reads out the voltages)

    2x2-special-binning => Readout a quarter of the pixels (???)

FRAME_LENGTH being 1/4th of normal would seem to suggest that it is a 4x1
binning (combining 4 lines instead of blocks of 2x2).. which does not make
sense to me.

My best guess is that in 2x2-special-binning mode the sensor does *both
horizontal and vertical* averaging in the analogue domain, before the ADC
reads out the voltages.

A higher minimum LINE_LENGTH value for this mode is the best "hard"
evidence I have for this guess unfortunately, as the datasheet is quite
lacking on this topic.

====================

The APIs before Sakari's series expose HBLANK and VBLANK controls instead
of the actual FRAME_LENGTH/LINE_LENGTH registers to the userspace.

The minimum value of FRAME_LENGTH is 632 when we are streaming 1640x1232,
so the sensor registers would suggest that we have a *negative vertical
blanking*. Which as Jacopo and Laurent both point out, does not make any
conceptual sense whatsoever.

What the driver does today to avoid these negative values is to double the
PIXEL_RATE control value to 364800000 when using 2x2-special-binning mode
(and thus userspace has no idea the FRAME_LENGTH/VBLANK is in units of
 2xLines)

As Sakari pointed out, that does not make any sense either. The sensor PLL
values are completely unchanged, so the pixel readout must still be
happening at the same rate. Moreover, the new raw sensor model will expose
FRAME_LENGTH and LINE_LENGTH directly to userspace, and this hack of
doubling the PIXEL_RATE breaks those calculations.

===================

If we want to support the new raw sensor model (that mandates the new
FRAME_LENGTH and LINE_LENGTH controls) for this sensor we have to
fix the PIXEL_RATE for sure. I see two options going forward:

OPTION 1 (as proposed by Sakari):

    Fix PIXEL_RATE to 182400000 and allow **negative values** for HBLANK
    and VBLANK controls when using 2x2-special-binning mode.

    This will break any userspace tools, many libcamera pipelines included,
    that never expected those control values to be negative (even though
    the API has always permitted those)

OPTION 2 (something that struck me today morning discussing with Jacopo):
    
    Fix PIXEL_RATE to 182400000 but **adjust the HBLANK values** to go
    lower to compensate, which will diverge from the sensor registers which
    keep MIN_LINE_LENGTH fixed across both binning modes.

    This will make the driver quite more complicated, but userspace
    expectations of non-negative blankings will be met. And it's likely
    that the sensor is internally doing pre-ADC averaging horizontally as
    well, or so my best guess is.

Of course, there are other options to just leave this highly used sensor
alone, or support embedded data and internal pads without mandating the new
FRAME_LENGTH and LINE_LENGTH controls. But I personally would leave that as
a last resort.

> > non-issue that doubled the pixel rate, frame length in lines and exposure
> > time.
> >
> > The resulting change also fixes the minimum, the maximum and the step
> > values for the control.
> >
> > Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Thanks,
    Jai

> > ---
> >  drivers/media/i2c/imx219.c | 37 ++++++++++++++-----------------------
> >  1 file changed, 14 insertions(+), 23 deletions(-)
> >
> > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > index 3aebcbaa3fcd..3cee31758b7e 100644
> > --- a/drivers/media/i2c/imx219.c
> > +++ b/drivers/media/i2c/imx219.c
> > @@ -420,15 +420,6 @@ static void imx219_get_binning(struct v4l2_subdev_state *state, u8 *bin_h,
> >
> >  }
> >
> > -static inline u32 imx219_get_rate_factor(struct v4l2_subdev_state *state)
> > -{
> > -     u8 bin_h, bin_v;
> > -
> > -     imx219_get_binning(state, &bin_h, &bin_v);
> > -
> > -     return (bin_h & bin_v) == IMX219_BINNING_X2_ANALOG ? 2 : 1;
> > -}
> > -
> >  /* -----------------------------------------------------------------------------
> >   * Controls
> >   */
> > @@ -440,19 +431,17 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> >       struct i2c_client *client = v4l2_get_subdevdata(&imx219->sd);
> >       const struct v4l2_mbus_framefmt *format;
> >       struct v4l2_subdev_state *state;
> > -     u32 rate_factor;
> >       int ret = 0;
> >
> >       state = v4l2_subdev_get_locked_active_state(&imx219->sd);
> >       format = v4l2_subdev_state_get_format(state, 0);
> > -     rate_factor = imx219_get_rate_factor(state);
> >
> >       if (ctrl->id == V4L2_CID_VBLANK) {
> >               int exposure_max, exposure_def;
> >
> >               /* Update max exposure while meeting expected vblanking */
> >               exposure_max = format->height + ctrl->val -
> > -                     IMX219_EXPOSURE_OFFSET * rate_factor;
> > +                     IMX219_EXPOSURE_OFFSET;
> >               exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
> >                               exposure_max : IMX219_EXPOSURE_DEFAULT;
> >               ret = __v4l2_ctrl_modify_range(imx219->exposure,
> > @@ -479,7 +468,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> >               break;
> >       case V4L2_CID_EXPOSURE:
> >               cci_write(imx219->regmap, IMX219_REG_EXPOSURE,
> > -                       ctrl->val / rate_factor, &ret);
> > +                       ctrl->val, &ret);
> >               break;
> >       case V4L2_CID_DIGITAL_GAIN:
> >               cci_write(imx219->regmap, IMX219_REG_DIGITAL_GAIN,
> > @@ -496,7 +485,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> >               break;
> >       case V4L2_CID_VBLANK:
> >               cci_write(imx219->regmap, IMX219_REG_FRM_LENGTH_A,
> > -                       (format->height + ctrl->val) / rate_factor, &ret);
> > +                       format->height + ctrl->val, &ret);
> >               break;
> >       case V4L2_CID_HBLANK:
> >               cci_write(imx219->regmap, IMX219_REG_LINE_LENGTH_A,
> > @@ -837,8 +826,8 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >       const struct imx219_mode *mode;
> >       struct v4l2_mbus_framefmt *format;
> >       struct v4l2_rect *crop;
> > -     u8 bin_h, bin_v, bin_hv;
> > -     int ret;
> > +     u8 bin_h, bin_v;
> > +     int ret, bin_hv;
> >
> >       format = v4l2_subdev_state_get_format(state, 0);
> >
> > @@ -879,23 +868,25 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >       crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
> >
> >       if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> > -             unsigned int rate_factor = imx219_get_rate_factor(state);
> >               int exposure_max;
> >               int exposure_def;
> >               int llp_min;
> >               int pixel_rate;
> >
> >               /* Update limits and set FPS to default */
> > +             int vblank_min = ((int)mode->height * (1 - bin_hv) / bin_hv) +
> > +                     IMX219_VBLANK_MIN;
> >               ret = __v4l2_ctrl_modify_range(imx219->vblank,
> > -                                            IMX219_VBLANK_MIN * rate_factor,
> > -                                            (IMX219_FLL_MAX - mode->height) *
> > -                                            rate_factor, rate_factor,
> > -                                            mode->fll_def - mode->height);
> > +                                            vblank_min,
> > +                                            IMX219_FLL_MAX - mode->height, 1,
> > +                                            (int)(mode->fll_def / bin_hv) -
> > +                                            (int)mode->height);
> >               if (ret)
> >                       return ret;
> >
> >               ret = __v4l2_ctrl_s_ctrl(imx219->vblank,
> > -                                      mode->fll_def - mode->height);
> > +                                      (int)(mode->fll_def / bin_hv) -
> > +                                      (int)mode->height);
> >               if (ret)
> >                       return ret;
> >
> > @@ -932,7 +923,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> >                       return ret;
> >
> >               /* Scale the pixel rate based on the mode specific factor */
> > -             pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
> > +             pixel_rate = imx219_get_pixel_rate(imx219);
> >               ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
> >                                              pixel_rate, 1, pixel_rate);
> >               if (ret)
> > --
> > 2.47.3
> >
> >

Re: [PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[Jai Luthra]

Quoting Jai Luthra (2026-06-08 16:01:06)
> Hi Jacopo, Sakari,
> ++ Dave, Hans and Laurent,
> 
> Quoting Jacopo Mondi (2026-06-08 12:28:46)
> > Hi Sakari
> > 
> > On Mon, Jun 08, 2026 at 12:53:52AM +0300, Sakari Ailus wrote:
> > > When vertical analogue binning is in use, the minimum frame length in
> > > lines decreases to around half of the normal. In relation to the sensor's
> > > output size this means vertical blanking can be negative but that's not an
> > > issue as control values are signed. Remove the workaround for this
> > 
> > Didn't we just discussed two weeks ago in media summit how negative
> > blankings are a bad idea, and of all drivers one could decide to play
> > with imx219 is probably the worse due it's large use base and the fact
> > libcamera doesn't support negative blankings ?
> > 
> > Have I missed something ?
> > 
> 
> I think it would be helpful if I write down clearly how this sensor
> operates (to the best of my knowledge) so we can decide on the correct
> fix:
> 
> --------------------
> 
> IMX219 sensor has an active resolution of 3280x2464.
> 
> The driver currently programs the sensor VT pixel clock as fixed for a
> given lane configuration.
> 
> In 2-lane mode it reads 182.4 MPixel/second:
> 
>     #define IMX219_PIXEL_RATE           182400000
> 
> And the framerate is given by:
> 
>     PIXEL_RATE / (FRAME_LENGTH * LINE_LENGTH)
> 
> where FRAME_LENGTH and LINE_LENGTH are registers that include the active
> height and width along with blankings.
> 
> There are restrictions on the minimum of the LINE_LENGTH register and
> minimum vertical blanking (32), which cap the framerate for the full resolution
> mode.
> 
>     MIN_LINE_LENGTH: 0xd78 => 3448 pixels
>     MIN_FRAME_LENGTH: ACTIVE_HEIGHT + MIN_VBLANK (32) = 2464 + 32
>                            => 2496 lines
> 
> The maximum frame rate is
> 
>     182400000/(2496*3448) => ~ 21.2 frames/second
> 
> --------------------
> 
> A user might want to stream a lower resolution with the full field-of-view,
> let's take 1640x1232 (which is exactly 1/2 of active area) as an example.
> 
> The sensor hardware can achieve this using two different binning modes:
> 
>     2x2-binning (regval: 0x1)
>     2x2-analog-(special)-binning (regval: 0x3)
> 
> The sensor pipeline looks like:
> 
> active pixel array ->
>     analogue crop (none) ->
>         2x2 binning and ADC readout ->
>                 output to CSI-2 bus
> 
> The mode names suggest that binning can happen either before or after ADC,
> but the datasheet is not very clear about the process. We can infer
> some details though. See below..
> 
> --------------------
> 
> With the "normal" 2x2-binning mode the sensor allows programming
> FRAME_LENGTH to a lower value. The driver still uses the min blanking of 32
> lines, but the height is now 1232, half of the 2464 before.
> 
>     MIN_LINE_LENGTH: 0xd78 => 3448 pixels
>     MIN_FRAME_LENGTH: READOUT_HEIGHT + MIN_VBLANK = 1232 + 32
>                            => 1264 lines
> 
> The maximum frame rate is
> 
>     182400000/(1264*3448) => ~ 41.8 frames/second
> 
> --------------------
> 
> With the "special" 2x2-binning mode, the datasheet notes that FRAME_LENGTH
> register should be in units of 2 Lines instead of 1 Line. This means
> cutting it down by half once more:
> 
>     MIN_FRAME_LENGTH: (READOUT_HEIGHT + MIN_VBLANK)/2 = (1232 + 32)/2
>                            => 632 lines
> 
> While there is a slightly higher minimum enforced for the line length:
> 
>     MIN_LINE_LENGTH: 0xde8 => 3560 pixels
> 
> The maximum frame rate is
> 
>     182400000/(632*3560) => ~ 81.0 frames/second
> 
> ===================
> 
> Through the minimum allowed values of the FRAME_LENGTH and LINE_LENGTH
> registers and maximum possible framerate, I think it is safe to say that:
> 
>     2x2-binning => Readout half the pixels (do vertical averaging in the
>                                             analogue domain, before ADC
>                                             reads out the voltages)
> 
>     2x2-special-binning => Readout a quarter of the pixels (???)
> 
> FRAME_LENGTH being 1/4th of normal would seem to suggest that it is a 4x1
> binning (combining 4 lines instead of blocks of 2x2).. which does not make
> sense to me.
> 
> My best guess is that in 2x2-special-binning mode the sensor does *both
> horizontal and vertical* averaging in the analogue domain, before the ADC
> reads out the voltages.
> 
> A higher minimum LINE_LENGTH value for this mode is the best "hard"
> evidence I have for this guess unfortunately, as the datasheet is quite
> lacking on this topic.
> 

Found another evidence for this, see "Table 16 Mode Example" in the
datasheet.

It mentions "H Binning = Analog" for x2 binning mode (with ~4x the FPS)
And "H Binning = Digital" for x4 binning mode

It doesn't mention a ~2x FPS mode, which I assume is the normal 2x2-binning
mode, where I again assume that horizontal binning is done digitally.


> ====================
> 
> The APIs before Sakari's series expose HBLANK and VBLANK controls instead
> of the actual FRAME_LENGTH/LINE_LENGTH registers to the userspace.
> 
> The minimum value of FRAME_LENGTH is 632 when we are streaming 1640x1232,
> so the sensor registers would suggest that we have a *negative vertical
> blanking*. Which as Jacopo and Laurent both point out, does not make any
> conceptual sense whatsoever.
> 
> What the driver does today to avoid these negative values is to double the
> PIXEL_RATE control value to 364800000 when using 2x2-special-binning mode
> (and thus userspace has no idea the FRAME_LENGTH/VBLANK is in units of
>  2xLines)
> 
> As Sakari pointed out, that does not make any sense either. The sensor PLL
> values are completely unchanged, so the pixel readout must still be
> happening at the same rate. Moreover, the new raw sensor model will expose
> FRAME_LENGTH and LINE_LENGTH directly to userspace, and this hack of
> doubling the PIXEL_RATE breaks those calculations.
> 
> ===================
> 
> If we want to support the new raw sensor model (that mandates the new
> FRAME_LENGTH and LINE_LENGTH controls) for this sensor we have to
> fix the PIXEL_RATE for sure. I see two options going forward:
> 
> OPTION 1 (as proposed by Sakari):
> 
>     Fix PIXEL_RATE to 182400000 and allow **negative values** for HBLANK
>     and VBLANK controls when using 2x2-special-binning mode.
> 
>     This will break any userspace tools, many libcamera pipelines included,
>     that never expected those control values to be negative (even though
>     the API has always permitted those)
> 
> OPTION 2 (something that struck me today morning discussing with Jacopo):
>     
>     Fix PIXEL_RATE to 182400000 but **adjust the HBLANK values** to go
>     lower to compensate, which will diverge from the sensor registers which
>     keep MIN_LINE_LENGTH fixed across both binning modes.
> 
>     This will make the driver quite more complicated, but userspace
>     expectations of non-negative blankings will be met. And it's likely
>     that the sensor is internally doing pre-ADC averaging horizontally as
>     well, or so my best guess is.
> 

Which makes me lean more on OPTION 2 now.

In our case above with 2x2-special-binning mode for 1640x1232, the register
values are:

    LINE_LENGTH = 3560 pixels
    FRAME_LENGTH = 632 lines


So OPTION 1 would give us:

    HBLANK = 1920, VBLANK = -600

When actually the datasheet's "H Binning = Analog" would suggest to me that
reality looks more like:

    LINE_LENGTH = 1780 pixels
    FRAME_LENGTH = 1264 lines

So OPTION 2 would give us:

    HBLANK = 140, VBLANK = 32

> Of course, there are other options to just leave this highly used sensor
> alone, or support embedded data and internal pads without mandating the new
> FRAME_LENGTH and LINE_LENGTH controls. But I personally would leave that as
> a last resort.

[snip]

Thanks,
    Jai

Re: [PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[Dave Stevenson]

Hi Jai, Jacopo, Sakari, and Laurent

On Mon, 8 Jun 2026 at 11:31, Jai Luthra <jai.luthra@ideasonboard.com> wrote:
>
> Hi Jacopo, Sakari,
> ++ Dave, Hans and Laurent,
>
> Quoting Jacopo Mondi (2026-06-08 12:28:46)
> > Hi Sakari
> >
> > On Mon, Jun 08, 2026 at 12:53:52AM +0300, Sakari Ailus wrote:
> > > When vertical analogue binning is in use, the minimum frame length in
> > > lines decreases to around half of the normal. In relation to the sensor's
> > > output size this means vertical blanking can be negative but that's not an
> > > issue as control values are signed. Remove the workaround for this
> >
> > Didn't we just discussed two weeks ago in media summit how negative
> > blankings are a bad idea, and of all drivers one could decide to play
> > with imx219 is probably the worse due it's large use base and the fact
> > libcamera doesn't support negative blankings ?
> >
> > Have I missed something ?

I'll agree that I don't like the concept of negative blanking as it is
very counter-intuitive.

However libcamera *does* appear to support negative blanking through
the Pi pipeline handler.
Running with mainline Pi5 CFE driver, mainline libcamera, and imx219
with this patch series. If I ask for 1640x1232 at 79.07fps then
libcamera is setting V4L2_CID_VBLANK to -584 and giving me 79.07fps.
The code could do with an audit though as I do see some uint32_t
variables kicking around, so it could be more luck than judgement
(hmm, I see a value of 4294966712 being set).

> I think it would be helpful if I write down clearly how this sensor
> operates (to the best of my knowledge) so we can decide on the correct
> fix:
>
> --------------------
>
> IMX219 sensor has an active resolution of 3280x2464.
>
> The driver currently programs the sensor VT pixel clock as fixed for a
> given lane configuration.
>
> In 2-lane mode it reads 182.4 MPixel/second:
>
>     #define IMX219_PIXEL_RATE           182400000
>
> And the framerate is given by:
>
>     PIXEL_RATE / (FRAME_LENGTH * LINE_LENGTH)
>
> where FRAME_LENGTH and LINE_LENGTH are registers that include the active
> height and width along with blankings.
>
> There are restrictions on the minimum of the LINE_LENGTH register and
> minimum vertical blanking (32), which cap the framerate for the full resolution
> mode.
>
>     MIN_LINE_LENGTH: 0xd78 => 3448 pixels
>     MIN_FRAME_LENGTH: ACTIVE_HEIGHT + MIN_VBLANK (32) = 2464 + 32
>                            => 2496 lines
>
> The maximum frame rate is
>
>     182400000/(2496*3448) => ~ 21.2 frames/second
>
> --------------------
>
> A user might want to stream a lower resolution with the full field-of-view,
> let's take 1640x1232 (which is exactly 1/2 of active area) as an example.
>
> The sensor hardware can achieve this using two different binning modes:
>
>     2x2-binning (regval: 0x1)
>     2x2-analog-(special)-binning (regval: 0x3)
>
> The sensor pipeline looks like:
>
> active pixel array ->
>     analogue crop (none) ->
>         2x2 binning and ADC readout ->
>                 output to CSI-2 bus
>
> The mode names suggest that binning can happen either before or after ADC,
> but the datasheet is not very clear about the process. We can infer
> some details though. See below..

We did previously ask Sony for more information on the special binning
mode, but there wasn't anything more to be had. imx219 is now at least
12 years old (we started working with it in 2014), so those who knew
about it in Sony have generally moved on.

> --------------------
>
> With the "normal" 2x2-binning mode the sensor allows programming
> FRAME_LENGTH to a lower value. The driver still uses the min blanking of 32
> lines, but the height is now 1232, half of the 2464 before.
>
>     MIN_LINE_LENGTH: 0xd78 => 3448 pixels
>     MIN_FRAME_LENGTH: READOUT_HEIGHT + MIN_VBLANK = 1232 + 32
>                            => 1264 lines
>
> The maximum frame rate is
>
>     182400000/(1264*3448) => ~ 41.8 frames/second
>
> --------------------
>
> With the "special" 2x2-binning mode, the datasheet notes that FRAME_LENGTH
> register should be in units of 2 Lines instead of 1 Line. This means
> cutting it down by half once more:
>
>     MIN_FRAME_LENGTH: (READOUT_HEIGHT + MIN_VBLANK)/2 = (1232 + 32)/2
>                            => 632 lines
>
> While there is a slightly higher minimum enforced for the line length:
>
>     MIN_LINE_LENGTH: 0xde8 => 3560 pixels
>
> The maximum frame rate is
>
>     182400000/(632*3560) => ~ 81.0 frames/second
>
> ===================
>
> Through the minimum allowed values of the FRAME_LENGTH and LINE_LENGTH
> registers and maximum possible framerate, I think it is safe to say that:
>
>     2x2-binning => Readout half the pixels (do vertical averaging in the
>                                             analogue domain, before ADC
>                                             reads out the voltages)
>
>     2x2-special-binning => Readout a quarter of the pixels (???)
>
> FRAME_LENGTH being 1/4th of normal would seem to suggest that it is a 4x1
> binning (combining 4 lines instead of blocks of 2x2).. which does not make
> sense to me.
>
> My best guess is that in 2x2-special-binning mode the sensor does *both
> horizontal and vertical* averaging in the analogue domain, before the ADC
> reads out the voltages.
>
> A higher minimum LINE_LENGTH value for this mode is the best "hard"
> evidence I have for this guess unfortunately, as the datasheet is quite
> lacking on this topic.
>
> ====================
>
> The APIs before Sakari's series expose HBLANK and VBLANK controls instead
> of the actual FRAME_LENGTH/LINE_LENGTH registers to the userspace.
>
> The minimum value of FRAME_LENGTH is 632 when we are streaming 1640x1232,
> so the sensor registers would suggest that we have a *negative vertical
> blanking*. Which as Jacopo and Laurent both point out, does not make any
> conceptual sense whatsoever.
>
> What the driver does today to avoid these negative values is to double the
> PIXEL_RATE control value to 364800000 when using 2x2-special-binning mode
> (and thus userspace has no idea the FRAME_LENGTH/VBLANK is in units of
>  2xLines)
>
> As Sakari pointed out, that does not make any sense either. The sensor PLL
> values are completely unchanged, so the pixel readout must still be
> happening at the same rate. Moreover, the new raw sensor model will expose
> FRAME_LENGTH and LINE_LENGTH directly to userspace, and this hack of
> doubling the PIXEL_RATE breaks those calculations.

A model needs to describe the functional behaviour, even if it isn't
how the hardware actually implements it.

If the new model can't describe imx219 with a change of pixel clock
when binning, is it going to cope with the Starvis 2 sensors (eg
imx415, imx662 and imx678) where the HMAX register is in units related
to the input clock rather than pixel clock? Perhaps it can, but one of
those probably ought to be converted to prove the point (imx415 is in
mainline and fairly easily available).

A second case that needs to be considered by this generic model: there
are a bunch of sensors (imx378, imx477, imx519, and imx708 to name 4)
that have an exposure multiplication factor to allow for very long
exposure times / very low frame rates. How would that get represented?
The register is CIT_LSHIFT with permitted values from 0 (for off) to 7
(*128). "exposure time = COARSE_INTEG_TIME * 2^CIT_LSHIFT", and
likewise frame_length_lines gets multiplied by that value.

Is there a proposal on how to handle eg ov5647 where the PLL
configuration is changed between some of the modes for both pixel
clock and link frequency?
Or imx290 which changes link frequency for the modes, although it does
keep a common pixel clock? (I've never come up with a good reason for
the change in link frequency, just that the datasheet does it that
way).

> ===================
>
> If we want to support the new raw sensor model (that mandates the new
> FRAME_LENGTH and LINE_LENGTH controls) for this sensor we have to
> fix the PIXEL_RATE for sure. I see two options going forward:
>
> OPTION 1 (as proposed by Sakari):
>
>     Fix PIXEL_RATE to 182400000 and allow **negative values** for HBLANK
>     and VBLANK controls when using 2x2-special-binning mode.
>
>     This will break any userspace tools, many libcamera pipelines included,
>     that never expected those control values to be negative (even though
>     the API has always permitted those)

Will it actually break that much?
As much as I conceptually don't like negative values, as above the Pi
pipeline handlers in libcamera seem to just work.
Anyone using the sensor in a raw capacity and just setting a
precomputed value is going to break, but that's going to be the case
with any of the proposed changes.

> OPTION 2 (something that struck me today morning discussing with Jacopo):
>
>     Fix PIXEL_RATE to 182400000 but **adjust the HBLANK values** to go
>     lower to compensate, which will diverge from the sensor registers which
>     keep MIN_LINE_LENGTH fixed across both binning modes.
>
>     This will make the driver quite more complicated, but userspace
>     expectations of non-negative blankings will be met. And it's likely
>     that the sensor is internally doing pre-ADC averaging horizontally as
>     well, or so my best guess is.

It is true that the line length configured is sufficient that it could
be halved in order to keep the pixel rate the same. That would seem
feasible.

> Of course, there are other options to just leave this highly used sensor
> alone, or support embedded data and internal pads without mandating the new
> FRAME_LENGTH and LINE_LENGTH controls. But I personally would leave that as
> a last resort.

Is there a dependency stopping the embedded data and internal pads
patches being merged without the common raw sensor model?

  Dave

> > > non-issue that doubled the pixel rate, frame length in lines and exposure
> > > time.
> > >
> > > The resulting change also fixes the minimum, the maximum and the step
> > > values for the control.
> > >
> > > Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>
> Thanks,
>     Jai
>
> > > ---
> > >  drivers/media/i2c/imx219.c | 37 ++++++++++++++-----------------------
> > >  1 file changed, 14 insertions(+), 23 deletions(-)
> > >
> > > diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
> > > index 3aebcbaa3fcd..3cee31758b7e 100644
> > > --- a/drivers/media/i2c/imx219.c
> > > +++ b/drivers/media/i2c/imx219.c
> > > @@ -420,15 +420,6 @@ static void imx219_get_binning(struct v4l2_subdev_state *state, u8 *bin_h,
> > >
> > >  }
> > >
> > > -static inline u32 imx219_get_rate_factor(struct v4l2_subdev_state *state)
> > > -{
> > > -     u8 bin_h, bin_v;
> > > -
> > > -     imx219_get_binning(state, &bin_h, &bin_v);
> > > -
> > > -     return (bin_h & bin_v) == IMX219_BINNING_X2_ANALOG ? 2 : 1;
> > > -}
> > > -
> > >  /* -----------------------------------------------------------------------------
> > >   * Controls
> > >   */
> > > @@ -440,19 +431,17 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> > >       struct i2c_client *client = v4l2_get_subdevdata(&imx219->sd);
> > >       const struct v4l2_mbus_framefmt *format;
> > >       struct v4l2_subdev_state *state;
> > > -     u32 rate_factor;
> > >       int ret = 0;
> > >
> > >       state = v4l2_subdev_get_locked_active_state(&imx219->sd);
> > >       format = v4l2_subdev_state_get_format(state, 0);
> > > -     rate_factor = imx219_get_rate_factor(state);
> > >
> > >       if (ctrl->id == V4L2_CID_VBLANK) {
> > >               int exposure_max, exposure_def;
> > >
> > >               /* Update max exposure while meeting expected vblanking */
> > >               exposure_max = format->height + ctrl->val -
> > > -                     IMX219_EXPOSURE_OFFSET * rate_factor;
> > > +                     IMX219_EXPOSURE_OFFSET;
> > >               exposure_def = (exposure_max < IMX219_EXPOSURE_DEFAULT) ?
> > >                               exposure_max : IMX219_EXPOSURE_DEFAULT;
> > >               ret = __v4l2_ctrl_modify_range(imx219->exposure,
> > > @@ -479,7 +468,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> > >               break;
> > >       case V4L2_CID_EXPOSURE:
> > >               cci_write(imx219->regmap, IMX219_REG_EXPOSURE,
> > > -                       ctrl->val / rate_factor, &ret);
> > > +                       ctrl->val, &ret);
> > >               break;
> > >       case V4L2_CID_DIGITAL_GAIN:
> > >               cci_write(imx219->regmap, IMX219_REG_DIGITAL_GAIN,
> > > @@ -496,7 +485,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
> > >               break;
> > >       case V4L2_CID_VBLANK:
> > >               cci_write(imx219->regmap, IMX219_REG_FRM_LENGTH_A,
> > > -                       (format->height + ctrl->val) / rate_factor, &ret);
> > > +                       format->height + ctrl->val, &ret);
> > >               break;
> > >       case V4L2_CID_HBLANK:
> > >               cci_write(imx219->regmap, IMX219_REG_LINE_LENGTH_A,
> > > @@ -837,8 +826,8 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> > >       const struct imx219_mode *mode;
> > >       struct v4l2_mbus_framefmt *format;
> > >       struct v4l2_rect *crop;
> > > -     u8 bin_h, bin_v, bin_hv;
> > > -     int ret;
> > > +     u8 bin_h, bin_v;
> > > +     int ret, bin_hv;
> > >
> > >       format = v4l2_subdev_state_get_format(state, 0);
> > >
> > > @@ -879,23 +868,25 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> > >       crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
> > >
> > >       if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> > > -             unsigned int rate_factor = imx219_get_rate_factor(state);
> > >               int exposure_max;
> > >               int exposure_def;
> > >               int llp_min;
> > >               int pixel_rate;
> > >
> > >               /* Update limits and set FPS to default */
> > > +             int vblank_min = ((int)mode->height * (1 - bin_hv) / bin_hv) +
> > > +                     IMX219_VBLANK_MIN;
> > >               ret = __v4l2_ctrl_modify_range(imx219->vblank,
> > > -                                            IMX219_VBLANK_MIN * rate_factor,
> > > -                                            (IMX219_FLL_MAX - mode->height) *
> > > -                                            rate_factor, rate_factor,
> > > -                                            mode->fll_def - mode->height);
> > > +                                            vblank_min,
> > > +                                            IMX219_FLL_MAX - mode->height, 1,
> > > +                                            (int)(mode->fll_def / bin_hv) -
> > > +                                            (int)mode->height);
> > >               if (ret)
> > >                       return ret;
> > >
> > >               ret = __v4l2_ctrl_s_ctrl(imx219->vblank,
> > > -                                      mode->fll_def - mode->height);
> > > +                                      (int)(mode->fll_def / bin_hv) -
> > > +                                      (int)mode->height);
> > >               if (ret)
> > >                       return ret;
> > >
> > > @@ -932,7 +923,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
> > >                       return ret;
> > >
> > >               /* Scale the pixel rate based on the mode specific factor */
> > > -             pixel_rate = imx219_get_pixel_rate(imx219) * rate_factor;
> > > +             pixel_rate = imx219_get_pixel_rate(imx219);
> > >               ret = __v4l2_ctrl_modify_range(imx219->pixel_rate, pixel_rate,
> > >                                              pixel_rate, 1, pixel_rate);
> > >               if (ret)
> > > --
> > > 2.47.3
> > >
> > >

Re: [PATCH v5 06/10] media: imx219: Fix vertical blanking and exposure for analogue binning

[Laurent Pinchart]

I wanted to check the valid of the static read-only registers, and ended
up dumping all registers while the sensor is running (using the
1920x1080 cropped mode, no binning). Here are the results, for the
non-zero registers. A large number of registers follow the CCS
specifications, registers marked with a (*) differ.

I will reply to the ongoing discussion separately.

0x0000 - module_model_id = 0x0219
0x0002 - ??? = 0x20
0x0004 - Lot_ID (*) = 0x686b17
0x0007 - Wafer_Num (*) = 0x0f
0x000d - Chip_Number (*) = 0x0945
0x0018 - FRM_CNT (*) = 0x56
0x0019 - PX_ORDER (*) = RGGB (0x01)
0x001b - DT_PEDESTAL (*) = 64
0x0040 - frame_format_model_type = 2-Byte Generic Frame Format (0x01)
0x0041 - frame_format_model_subtype = 1 Column, 2 Rows (0x12)
0x0042 - frame_format_descriptor_0 = Visible Pixel data: 3280 (0x5cd0)
0x0044 - frame_format_descriptor_1 = Embedded data: 2 (0x1002)
0x0046 - frame_format_descriptor_2 = Visible Pixel data: 2464 (0x59a0)
0x0086 - analog_gain_code_max = 224
0x0088 - analog_gain_code_step_size = 1
0x008e - analog_gain_c0 = 256
0x0090 - analog_gain_m1 = -1
0x0092 - analog_gain_c1 = 256
0x00c0 - data_format_model_type = 2-byte data format (0x01)
0x00c1 - data_format_model_subtype = 1 descriptor
0x00c2 - data_format_descriptor_0 = 10 bit uncompressed, 8 bit compressed (0x0a08)
0x00c4 - data_format_descriptor_1 = 10 bit uncompressed, 10 bit compressed (0x0a0a)
0x0100 - mode_select = 0x01
0x0105 - mask_corrupted_frames = 0x01
0x0114 - CSI_lane_mode (*) = 2 lanes (0x01)
0x0118 - TCLK_POST (*) = 119
0x011a - THS_PREPARE (*) = 71
0x011c - THS_ZERO_MIN (*) = 103
0x011e - THS_TRAIL (*) = 63
0x0120 - TCLK_TRAIL_MIN (*) = 55
0x0122 - TCLK_PREPARE (*) = 63
0x0124 - TCLK_ZERO (*) = 255
0x0126 - TLPX (*) = 55
0x012a - EXCK_FREQ (*) = 24
0x0142 - READOUT_V_CNT (*) = 0x5c70
0x0157 - ANA_GAIN_GLOBAL_A (*) = 232
0x0158 - DIG_GAIN_GLOBAL_A (*) = 256
0x015a - COARSE_INTEGRATION_TIME_A (*) = 1108
0x0160 - FRM_LENGTH_A (*) = 1112
0x0162 - LINE_LENGTH_A (*) = 3448
0x0164 - X_ADD_STA_A (*) = 680
0x0166 - X_ADD_END_A (*) = 2599
0x0168 - Y_ADD_STA_A (*) = 692
0x016a - Y_ADD_END_A (*) = 1771
0x016c - x_output_size (*) = 1920
0x016e - y_output_size (*) = 1080
0x0170 - X_ODD_INC_A (*) = 1
0x0171 - Y_ODD_INC_A (*) = 1
0x018a - COARSE_INTEG_TIME_SHORT_A (*) = 500
0x018c - CSI_DATA_FORMAT_A (*) = 0x0a0a
0x0192 - LSC_SELECT_TABLE_A (*) = 0x01
0x0194 - LSC_WHITE_BALANCE_RG_A (*) = 0x1000
0x0196 - ??? (*) = 0x10
0x0258 - DIG_GAIN_GLOBAL_B (*) = 256
0x025a - COARSE_INTEGRATION_TIME_B (*) = 1000
0x0260 - FRM_LENGTH_B (*) = 2728
0x0262 - LINE_LENGTH_B (*) = 3448
0x0266 - X_ADD_END_B (*) = 3279
0x026a - Y_ADD_END_B (*) = 2463
0x026c - x_output_size (*) = 3280
0x026e - y_output_size (*) = 2464
0x0270 - X_ODD_INC_B (*) = 1
0x0271 - Y_ODD_INC_B (*) = 1
0x028a - COARSE_INTEG_TIME_SHORT_B (*) = 500
0x028c - CSI_DATA_FORMAT_B (*) = 0x0a0a
0x0292 - LSC_SELECT_TABLE_B (*) = 0x01
0x0294 - LSC_WHITE_BALANCE_RG_B (*) = 0x10
0x0296 - ??? = 0x10
0x0300 - vt_pix_clk_div = 5
0x0302 - vt_sys_clk_div = 1
0x0304 - vt_pre_pll_clk_div = 3
0x0305 - op_pre_pll_clk_div = 3
0x0306 - vt_pll_multiplier = 57
0x0308 - op_pix_clk_div = 10
0x030a - op_sys_clk_div = 1
0x030d - op_pll_multiplier = 114
0x0319 - ??? = 0x03
0x031a - ??? = 0x01
0x031b - ??? = 0x4c
0x031d - ??? = 0x03
0x031e - ??? = 0x01
0x031f - ??? = 0x4c
0x0322 - FLASH_STROBE_DIV (*) = 1
0x0334 - FLASH_STROBE_HI_PERIOD_RS (*) = 1
0x0336 - FLASH_STROBE_LO_PERIOD_RS (*) = 1
0x0338 - FLASH_STROBE_COUNT_RS (*) = 1
0x0380 - x_even_inc = 0x01
0x0382 - x_odd_inc = 0x01
0x0388 - FINE_INTEG_TIME (*) = 500
0x0602 - test_data_red = 0x03ff
0x0604 - test_data_greenR = 0x03ff
0x0606 - test_data_blue = 0x03ff
0x0608 - test_data_greenB = 0x03ff
0x0624 - TP_WINDOW_WIDTH (*) = 1920
0x0626 - TP_WINDOW_HEIGHT (*) = 1080
0x1004 - coarse_integration_time_min = 1
0x1006 - coarse_integration_time_max_margin = 4
0x1081 - digital_gain_capability = 1
0x1084 - digital_gain_min = 256
0x1086 - digital_gain_max = 4095
0x1088 - digital_gain_step_size = 1
0x1100 - min_ext_clk_freq_mhz = 6.0 (0x40c00000)
0x1104 - max_ext_clk_freq_mhz = 27.0 (0x41d80000)
0x1108 - min_vt_pre_pll_clk_div = 1
0x110a - max_vt_pre_pll_clk_div = 13
0x110c - min_vt_pll_ip_freq_mhz = 6.0 (0x40c00000)
0x1110 - max_vt_pll_ip_freq_mhz = 27.0 (0x41d80000)
0x1114 - min_vt_pll_multiplier = 8
0x1116 - max_vt_pll_multiplier = 2047
0x1118 - min_pll_op_freq_mhz = 400.0 (0x43c80000)
0x111c - max_pll_op_freq_mhz = 916.0 (0x44650000)
0x1120 - min_vt_sys_clk_div = 1
0x1122 - max_vt_sys_clk_div = 2
0x1124 - min_vt_sys_clk_freq_mhz = 200.0 (0x43480000)
0x1128 - max_vt_sys_clk_freq_mhz = 700.0 (0x442f0000)
0x112c - min_vt_pix_clk_freq_mhz = 80.0 (0x42a00000)
0x1130 - max_vt_pix_clk_freq_mhz = 140.0 (0x430c0000)
0x1134 - min_vt_pix_clk_div = 5
0x1136 - max_vt_pix_clk_div = 5
0x1140 - min_frame_length_lines = 256
0x1142 - max_frame_length_lines = 65534
0x1144 - min_line_length_pck = 3448
0x1146 - max_line_length_pck = 32752
0x1148 - min_line_blanking_pck = 168
0x114a - min_frame_blanking_lines = 32
0x1160 - min_op_sys_clk_div = 1
0x1162 - max_op_sys_clk_div = 2
0x1164 - min_op_sys_clk_freq_mhz = 200.0 (0x43480000)
0x1168 - max_op_sys_clk_freq_mhz = 916.0 (0x44650000)
0x116c - min_op_pix_clk_freq_mhz (*) = 20.0 (0x41a00000)
0x1170 - max_op_pix_clk_freq_mhz (*) = 114.5 (0x42e50000)
0x1174 - min_op_pix_clk_div (*) = 8
0x1176 - max_op_pix_clk_div (*) = 10
0x1184 - x_addr_max = 3279
0x1186 - y_addr_max = 2463
0x1188 - min_x_output_size = 256
0x118a - min_y_output_size = 256
0x118c - max_x_output_size = 3280
0x118e - max_y_output_size = 2464
0x11c0 - min_even_inc = 1
0x11c2 - max_even_inc = 1
0x11c4 - min_odd_inc = 1
0x11c6 - max_odd_inc = 3
0x1211 - ??? = 1
0x1300 - compression_capability = 1

On Mon, Jun 08, 2026 at 04:01:06PM +0530, Jai Luthra wrote:
> Hi Jacopo, Sakari,
> ++ Dave, Hans and Laurent,
> 
> Quoting Jacopo Mondi (2026-06-08 12:28:46)
> > Hi Sakari
> > 
> > On Mon, Jun 08, 2026 at 12:53:52AM +0300, Sakari Ailus wrote:
> > > When vertical analogue binning is in use, the minimum frame length in
> > > lines decreases to around half of the normal. In relation to the sensor's
> > > output size this means vertical blanking can be negative but that's not an
> > > issue as control values are signed. Remove the workaround for this
> > 
> > Didn't we just discussed two weeks ago in media summit how negative
> > blankings are a bad idea, and of all drivers one could decide to play
> > with imx219 is probably the worse due it's large use base and the fact
> > libcamera doesn't support negative blankings ?
> > 
> > Have I missed something ?
> > 
> 
> I think it would be helpful if I write down clearly how this sensor
> operates (to the best of my knowledge) so we can decide on the correct
> fix:
> 
> --------------------
> 
> IMX219 sensor has an active resolution of 3280x2464.
> 
> The driver currently programs the sensor VT pixel clock as fixed for a
> given lane configuration.
> 
> In 2-lane mode it reads 182.4 MPixel/second:
> 
>     #define IMX219_PIXEL_RATE		182400000
> 
> And the framerate is given by:
> 
>     PIXEL_RATE / (FRAME_LENGTH * LINE_LENGTH)
> 
> where FRAME_LENGTH and LINE_LENGTH are registers that include the active
> height and width along with blankings.
> 
> There are restrictions on the minimum of the LINE_LENGTH register and
> minimum vertical blanking (32), which cap the framerate for the full resolution
> mode.
> 
>     MIN_LINE_LENGTH: 0xd78 => 3448 pixels
>     MIN_FRAME_LENGTH: ACTIVE_HEIGHT + MIN_VBLANK (32) = 2464 + 32
>                            => 2496 lines
> 
> The maximum frame rate is
> 
>     182400000/(2496*3448) => ~ 21.2 frames/second
> 
> --------------------
> 
> A user might want to stream a lower resolution with the full field-of-view,
> let's take 1640x1232 (which is exactly 1/2 of active area) as an example.
> 
> The sensor hardware can achieve this using two different binning modes:
> 
>     2x2-binning (regval: 0x1)
>     2x2-analog-(special)-binning (regval: 0x3)
> 
> The sensor pipeline looks like:
> 
> active pixel array ->
>     analogue crop (none) ->
>         2x2 binning and ADC readout ->
>                 output to CSI-2 bus
> 
> The mode names suggest that binning can happen either before or after ADC,
> but the datasheet is not very clear about the process. We can infer
> some details though. See below..
> 
> --------------------
> 
> With the "normal" 2x2-binning mode the sensor allows programming
> FRAME_LENGTH to a lower value. The driver still uses the min blanking of 32
> lines, but the height is now 1232, half of the 2464 before.
> 
>     MIN_LINE_LENGTH: 0xd78 => 3448 pixels
>     MIN_FRAME_LENGTH: READOUT_HEIGHT + MIN_VBLANK = 1232 + 32
>                            => 1264 lines
> 
> The maximum frame rate is
> 
>     182400000/(1264*3448) => ~ 41.8 frames/second
> 
> --------------------
> 
> With the "special" 2x2-binning mode, the datasheet notes that FRAME_LENGTH
> register should be in units of 2 Lines instead of 1 Line. This means
> cutting it down by half once more:
> 
>     MIN_FRAME_LENGTH: (READOUT_HEIGHT + MIN_VBLANK)/2 = (1232 + 32)/2
>                            => 632 lines
> 
> While there is a slightly higher minimum enforced for the line length:
> 
>     MIN_LINE_LENGTH: 0xde8 => 3560 pixels
> 
> The maximum frame rate is
> 
>     182400000/(632*3560) => ~ 81.0 frames/second
> 
> ===================
> 
> Through the minimum allowed values of the FRAME_LENGTH and LINE_LENGTH
> registers and maximum possible framerate, I think it is safe to say that:
> 
>     2x2-binning => Readout half the pixels (do vertical averaging in the
>                                             analogue domain, before ADC
>                                             reads out the voltages)
> 
>     2x2-special-binning => Readout a quarter of the pixels (???)
> 
> FRAME_LENGTH being 1/4th of normal would seem to suggest that it is a 4x1
> binning (combining 4 lines instead of blocks of 2x2).. which does not make
> sense to me.
> 
> My best guess is that in 2x2-special-binning mode the sensor does *both
> horizontal and vertical* averaging in the analogue domain, before the ADC
> reads out the voltages.
> 
> A higher minimum LINE_LENGTH value for this mode is the best "hard"
> evidence I have for this guess unfortunately, as the datasheet is quite
> lacking on this topic.
> 
> ====================
> 
> The APIs before Sakari's series expose HBLANK and VBLANK controls instead
> of the actual FRAME_LENGTH/LINE_LENGTH registers to the userspace.
> 
> The minimum value of FRAME_LENGTH is 632 when we are streaming 1640x1232,
> so the sensor registers would suggest that we have a *negative vertical
> blanking*. Which as Jacopo and Laurent both point out, does not make any
> conceptual sense whatsoever.
> 
> What the driver does today to avoid these negative values is to double the
> PIXEL_RATE control value to 364800000 when using 2x2-special-binning mode
> (and thus userspace has no idea the FRAME_LENGTH/VBLANK is in units of
>  2xLines)
> 
> As Sakari pointed out, that does not make any sense either. The sensor PLL
> values are completely unchanged, so the pixel readout must still be
> happening at the same rate. Moreover, the new raw sensor model will expose
> FRAME_LENGTH and LINE_LENGTH directly to userspace, and this hack of
> doubling the PIXEL_RATE breaks those calculations.
> 
> ===================
> 
> If we want to support the new raw sensor model (that mandates the new
> FRAME_LENGTH and LINE_LENGTH controls) for this sensor we have to
> fix the PIXEL_RATE for sure. I see two options going forward:
> 
> OPTION 1 (as proposed by Sakari):
> 
>     Fix PIXEL_RATE to 182400000 and allow **negative values** for HBLANK
>     and VBLANK controls when using 2x2-special-binning mode.
> 
>     This will break any userspace tools, many libcamera pipelines included,
>     that never expected those control values to be negative (even though
>     the API has always permitted those)
> 
> OPTION 2 (something that struck me today morning discussing with Jacopo):
>     
>     Fix PIXEL_RATE to 182400000 but **adjust the HBLANK values** to go
>     lower to compensate, which will diverge from the sensor registers which
>     keep MIN_LINE_LENGTH fixed across both binning modes.
> 
>     This will make the driver quite more complicated, but userspace
>     expectations of non-negative blankings will be met. And it's likely
>     that the sensor is internally doing pre-ADC averaging horizontally as
>     well, or so my best guess is.
> 
> Of course, there are other options to just leave this highly used sensor
> alone, or support embedded data and internal pads without mandating the new
> FRAME_LENGTH and LINE_LENGTH controls. But I personally would leave that as
> a last resort.
> 
> > > non-issue that doubled the pixel rate, frame length in lines and exposure
> > > time.
> > >
> > > The resulting change also fixes the minimum, the maximum and the step
> > > values for the control.
> > >
> > > Fixes: f513997119f4 ("media: i2c: imx219: Scale the pixel rate for analog binning")
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

-- 
Regards,

Laurent Pinchart

[PATCH v5 07/10] media: Improve enable_streams and disable_streams documentation

[Sakari Ailus]

Document that enable_streams may start additional streams and
disable_streams may not disable requested streams if other related streams
are still enabled.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
---
 include/media/v4l2-subdev.h | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 0361c8bbee38..a25234befbe9 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -828,6 +828,10 @@ struct v4l2_subdev_state {
  *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
  *	argument.
  *
+ *	Starting the requested streams may require starting additional
+ *	streams. Streams that are started and stopped together due to hardware
+ *	are called a stream group.
+ *
  * @disable_streams: Disable the streams defined in streams_mask on the given
  *	source pad. Subdevs that implement this operation must use the active
  *	state management provided by the subdev core (enabled through a call to
@@ -837,6 +841,10 @@ struct v4l2_subdev_state {
  *	Drivers that support only a single stream without setting the
  *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
  *	argument.
+ *
+ *	Stopping the requested streams may be delayed due to other enabled
+ *	streams in the stream group, where all streams are started and stopped
+ *	together.
  */
 struct v4l2_subdev_pad_ops {
 	int (*enum_mbus_code)(struct v4l2_subdev *sd,
-- 
2.47.3

Re: [PATCH v5 07/10] media: Improve enable_streams and disable_streams documentation

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Jun 08, 2026 at 12:53:53AM +0300, Sakari Ailus wrote:
> Document that enable_streams may start additional streams and
> disable_streams may not disable requested streams if other related streams
> are still enabled.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
> ---
>  include/media/v4l2-subdev.h | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 0361c8bbee38..a25234befbe9 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -828,6 +828,10 @@ struct v4l2_subdev_state {
>   *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
>   *	argument.
>   *
> + *	Starting the requested streams may require starting additional
> + *	streams. Streams that are started and stopped together due to hardware
> + *	are called a stream group.

This is too vague. "may require starting additional streams" can be
understood as the driver starting more streams than the requested ones,
or as the caller having to start more streams before the requests
streams are effectively started.

Here's a proposal to clarify the documentation:

 * Due to device constraints, starting the requested streams may result in
 * additional streams also being started by the driver. Streams that are started
 * and stopped together due to the nature of the hardware are called a stream
 * group.

"hardware" may be a bit restrictive, I wonder if those constraints could
also sometimes come from the software implementation instead of being
pure hardware constraints. I think it's fine for now though, we can
clarify this later if needed.

> + *
>   * @disable_streams: Disable the streams defined in streams_mask on the given
>   *	source pad. Subdevs that implement this operation must use the active
>   *	state management provided by the subdev core (enabled through a call to
> @@ -837,6 +841,10 @@ struct v4l2_subdev_state {
>   *	Drivers that support only a single stream without setting the
>   *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
>   *	argument.
> + *
> + *	Stopping the requested streams may be delayed due to other enabled
> + *	streams in the stream group, where all streams are started and stopped
> + *	together.

This is better. I would have written

 * When the requested stream are part of a stream group, they will be stopped
 * once all streams in the group are stopped.

to make it clearer that the streams won't be stopped immediately ("may"
in the original text implies they may or may not be stopped
immediately), but I'm also OK with your text.

>   */
>  struct v4l2_subdev_pad_ops {
>  	int (*enum_mbus_code)(struct v4l2_subdev *sd,

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 07/10] media: Improve enable_streams and disable_streams documentation

[Sakari Ailus]

Hi Laurent,

On Mon, Jun 08, 2026 at 12:29:16PM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.
> 
> On Mon, Jun 08, 2026 at 12:53:53AM +0300, Sakari Ailus wrote:
> > Document that enable_streams may start additional streams and
> > disable_streams may not disable requested streams if other related streams
> > are still enabled.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
> > ---
> >  include/media/v4l2-subdev.h | 8 ++++++++
> >  1 file changed, 8 insertions(+)
> > 
> > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> > index 0361c8bbee38..a25234befbe9 100644
> > --- a/include/media/v4l2-subdev.h
> > +++ b/include/media/v4l2-subdev.h
> > @@ -828,6 +828,10 @@ struct v4l2_subdev_state {
> >   *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
> >   *	argument.
> >   *
> > + *	Starting the requested streams may require starting additional
> > + *	streams. Streams that are started and stopped together due to hardware
> > + *	are called a stream group.
> 
> This is too vague. "may require starting additional streams" can be
> understood as the driver starting more streams than the requested ones,
> or as the caller having to start more streams before the requests
> streams are effectively started.
> 
> Here's a proposal to clarify the documentation:
> 
>  * Due to device constraints, starting the requested streams may result in
>  * additional streams also being started by the driver. Streams that are started
>  * and stopped together due to the nature of the hardware are called a stream
>  * group.
> 
> "hardware" may be a bit restrictive, I wonder if those constraints could
> also sometimes come from the software implementation instead of being
> pure hardware constraints. I think it's fine for now though, we can
> clarify this later if needed.
> 
> > + *
> >   * @disable_streams: Disable the streams defined in streams_mask on the given
> >   *	source pad. Subdevs that implement this operation must use the active
> >   *	state management provided by the subdev core (enabled through a call to
> > @@ -837,6 +841,10 @@ struct v4l2_subdev_state {
> >   *	Drivers that support only a single stream without setting the
> >   *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
> >   *	argument.
> > + *
> > + *	Stopping the requested streams may be delayed due to other enabled
> > + *	streams in the stream group, where all streams are started and stopped
> > + *	together.
> 
> This is better. I would have written
> 
>  * When the requested stream are part of a stream group, they will be stopped
>  * once all streams in the group are stopped.
> 
> to make it clearer that the streams won't be stopped immediately ("may"
> in the original text implies they may or may not be stopped
> immediately), but I'm also OK with your text.

I'll use your text in v6. I presume Jacopo and Mirela agree. :-)

-- 
Regards,

Sakari Ailus

[PATCH v5 08/10] media: v4l2-subdev: Move subdev client capabilities into a new struct

[Sakari Ailus]

Add struct v4l2_subdev_client_info to hold sub-device client capability
bits that used to be stored in the client_caps field of struct
v4l2_subdev_fh. The intent is to enable passing this struct to sub-device
pad operation callbacks for capability information. The main reason why
this is a new struct instead of a u64 field is that modifying the callback
arguments requires touching almost every sub-device driver and that is
desirable to avoid in the future, should more than the client capability bits
need to be known to the callbacks.

To be squashed to the previous patch.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c |  8 ++++----
 include/media/v4l2-subdev.h           | 12 ++++++++++--
 2 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index dc4ac08c210f..e4ac6981e950 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -634,7 +634,7 @@ subdev_ioctl_get_state(struct v4l2_subdev *sd, struct v4l2_subdev_fh *subdev_fh,
 	case VIDIOC_SUBDEV_S_FRAME_INTERVAL: {
 		struct v4l2_subdev_frame_interval *fi = arg;
 
-		if (!(subdev_fh->client_caps &
+		if (!(subdev_fh->ci.client_caps &
 		      V4L2_SUBDEV_CLIENT_CAP_INTERVAL_USES_WHICH))
 			fi->which = V4L2_SUBDEV_FORMAT_ACTIVE;
 
@@ -673,7 +673,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 	struct v4l2_subdev_fh *subdev_fh = to_v4l2_subdev_fh(vfh);
 	bool ro_subdev = test_bit(V4L2_FL_SUBDEV_RO_DEVNODE, &vdev->flags);
 	bool streams_subdev = sd->flags & V4L2_SUBDEV_FL_STREAMS;
-	bool client_supports_streams = subdev_fh->client_caps &
+	bool client_supports_streams = subdev_fh->ci.client_caps &
 				       V4L2_SUBDEV_CLIENT_CAP_STREAMS;
 	int rval;
 
@@ -1133,7 +1133,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 	case VIDIOC_SUBDEV_G_CLIENT_CAP: {
 		struct v4l2_subdev_client_capability *client_cap = arg;
 
-		client_cap->capabilities = subdev_fh->client_caps;
+		client_cap->capabilities = subdev_fh->ci.client_caps;
 
 		return 0;
 	}
@@ -1153,7 +1153,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 		client_cap->capabilities &= (V4L2_SUBDEV_CLIENT_CAP_STREAMS |
 					     V4L2_SUBDEV_CLIENT_CAP_INTERVAL_USES_WHICH);
 
-		subdev_fh->client_caps = client_cap->capabilities;
+		subdev_fh->ci.client_caps = client_cap->capabilities;
 
 		return 0;
 	}
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index a25234befbe9..e83ef88fe12c 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -741,6 +741,14 @@ struct v4l2_subdev_state {
 	struct v4l2_subdev_stream_configs stream_configs;
 };
 
+/**
+ * struct v4l2_subdev_client_info - Sub-device client information
+ * @client_caps: bitmask of ``V4L2_SUBDEV_CLIENT_CAP_*``
+ */
+struct v4l2_subdev_client_info {
+	u64 client_caps;
+};
+
 /**
  * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
  *
@@ -1144,14 +1152,14 @@ struct v4l2_subdev {
  * @vfh: pointer to &struct v4l2_fh
  * @state: pointer to &struct v4l2_subdev_state
  * @owner: module pointer to the owner of this file handle
- * @client_caps: bitmask of ``V4L2_SUBDEV_CLIENT_CAP_*``
+ * @ci: sub-device client info related to this file handle
  */
 struct v4l2_subdev_fh {
 	struct v4l2_fh vfh;
 	struct module *owner;
 #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
 	struct v4l2_subdev_state *state;
-	u64 client_caps;
+	struct v4l2_subdev_client_info ci;
 #endif
 };
 
-- 
2.47.3

Re: [PATCH v5 08/10] media: v4l2-subdev: Move subdev client capabilities into a new struct

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Jun 08, 2026 at 12:53:54AM +0300, Sakari Ailus wrote:
> Add struct v4l2_subdev_client_info to hold sub-device client capability
> bits that used to be stored in the client_caps field of struct
> v4l2_subdev_fh. The intent is to enable passing this struct to sub-device
> pad operation callbacks for capability information. The main reason why
> this is a new struct instead of a u64 field is that modifying the callback
> arguments requires touching almost every sub-device driver and that is
> desirable to avoid in the future, should more than the client capability bits
> need to be known to the callbacks.
> 
> To be squashed to the previous patch.

This seems to be a leftover, patch 07/10 is unrelated.

> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c |  8 ++++----
>  include/media/v4l2-subdev.h           | 12 ++++++++++--
>  2 files changed, 14 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index dc4ac08c210f..e4ac6981e950 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -634,7 +634,7 @@ subdev_ioctl_get_state(struct v4l2_subdev *sd, struct v4l2_subdev_fh *subdev_fh,
>  	case VIDIOC_SUBDEV_S_FRAME_INTERVAL: {
>  		struct v4l2_subdev_frame_interval *fi = arg;
>  
> -		if (!(subdev_fh->client_caps &
> +		if (!(subdev_fh->ci.client_caps &
>  		      V4L2_SUBDEV_CLIENT_CAP_INTERVAL_USES_WHICH))
>  			fi->which = V4L2_SUBDEV_FORMAT_ACTIVE;
>  
> @@ -673,7 +673,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
>  	struct v4l2_subdev_fh *subdev_fh = to_v4l2_subdev_fh(vfh);
>  	bool ro_subdev = test_bit(V4L2_FL_SUBDEV_RO_DEVNODE, &vdev->flags);
>  	bool streams_subdev = sd->flags & V4L2_SUBDEV_FL_STREAMS;
> -	bool client_supports_streams = subdev_fh->client_caps &
> +	bool client_supports_streams = subdev_fh->ci.client_caps &
>  				       V4L2_SUBDEV_CLIENT_CAP_STREAMS;
>  	int rval;
>  
> @@ -1133,7 +1133,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
>  	case VIDIOC_SUBDEV_G_CLIENT_CAP: {
>  		struct v4l2_subdev_client_capability *client_cap = arg;
>  
> -		client_cap->capabilities = subdev_fh->client_caps;
> +		client_cap->capabilities = subdev_fh->ci.client_caps;
>  
>  		return 0;
>  	}
> @@ -1153,7 +1153,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
>  		client_cap->capabilities &= (V4L2_SUBDEV_CLIENT_CAP_STREAMS |
>  					     V4L2_SUBDEV_CLIENT_CAP_INTERVAL_USES_WHICH);
>  
> -		subdev_fh->client_caps = client_cap->capabilities;
> +		subdev_fh->ci.client_caps = client_cap->capabilities;
>  
>  		return 0;
>  	}
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index a25234befbe9..e83ef88fe12c 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -741,6 +741,14 @@ struct v4l2_subdev_state {
>  	struct v4l2_subdev_stream_configs stream_configs;
>  };
>  
> +/**
> + * struct v4l2_subdev_client_info - Sub-device client information
> + * @client_caps: bitmask of ``V4L2_SUBDEV_CLIENT_CAP_*``
> + */
> +struct v4l2_subdev_client_info {
> +	u64 client_caps;

As this is now part of a structure called *client* info, maybe the field
could be named just "caps" ?

> +};
> +
>  /**
>   * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
>   *
> @@ -1144,14 +1152,14 @@ struct v4l2_subdev {
>   * @vfh: pointer to &struct v4l2_fh
>   * @state: pointer to &struct v4l2_subdev_state
>   * @owner: module pointer to the owner of this file handle
> - * @client_caps: bitmask of ``V4L2_SUBDEV_CLIENT_CAP_*``
> + * @ci: sub-device client info related to this file handle

I'm not a fan of "ci", I would write client_info to make the code more
explicit.

>   */
>  struct v4l2_subdev_fh {
>  	struct v4l2_fh vfh;
>  	struct module *owner;
>  #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
>  	struct v4l2_subdev_state *state;
> -	u64 client_caps;
> +	struct v4l2_subdev_client_info ci;
>  #endif
>  };
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 08/10] media: v4l2-subdev: Move subdev client capabilities into a new struct

[Sakari Ailus]

Hi Laurent,

On Mon, Jun 08, 2026 at 12:34:36PM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.
> 
> On Mon, Jun 08, 2026 at 12:53:54AM +0300, Sakari Ailus wrote:
> > Add struct v4l2_subdev_client_info to hold sub-device client capability
> > bits that used to be stored in the client_caps field of struct
> > v4l2_subdev_fh. The intent is to enable passing this struct to sub-device
> > pad operation callbacks for capability information. The main reason why
> > this is a new struct instead of a u64 field is that modifying the callback
> > arguments requires touching almost every sub-device driver and that is
> > desirable to avoid in the future, should more than the client capability bits
> > need to be known to the callbacks.
> > 
> > To be squashed to the previous patch.
> 
> This seems to be a leftover, patch 07/10 is unrelated.

Yes, I'll drop it.

> 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-subdev.c |  8 ++++----
> >  include/media/v4l2-subdev.h           | 12 ++++++++++--
> >  2 files changed, 14 insertions(+), 6 deletions(-)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > index dc4ac08c210f..e4ac6981e950 100644
> > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > @@ -634,7 +634,7 @@ subdev_ioctl_get_state(struct v4l2_subdev *sd, struct v4l2_subdev_fh *subdev_fh,
> >  	case VIDIOC_SUBDEV_S_FRAME_INTERVAL: {
> >  		struct v4l2_subdev_frame_interval *fi = arg;
> >  
> > -		if (!(subdev_fh->client_caps &
> > +		if (!(subdev_fh->ci.client_caps &
> >  		      V4L2_SUBDEV_CLIENT_CAP_INTERVAL_USES_WHICH))
> >  			fi->which = V4L2_SUBDEV_FORMAT_ACTIVE;
> >  
> > @@ -673,7 +673,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
> >  	struct v4l2_subdev_fh *subdev_fh = to_v4l2_subdev_fh(vfh);
> >  	bool ro_subdev = test_bit(V4L2_FL_SUBDEV_RO_DEVNODE, &vdev->flags);
> >  	bool streams_subdev = sd->flags & V4L2_SUBDEV_FL_STREAMS;
> > -	bool client_supports_streams = subdev_fh->client_caps &
> > +	bool client_supports_streams = subdev_fh->ci.client_caps &
> >  				       V4L2_SUBDEV_CLIENT_CAP_STREAMS;
> >  	int rval;
> >  
> > @@ -1133,7 +1133,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
> >  	case VIDIOC_SUBDEV_G_CLIENT_CAP: {
> >  		struct v4l2_subdev_client_capability *client_cap = arg;
> >  
> > -		client_cap->capabilities = subdev_fh->client_caps;
> > +		client_cap->capabilities = subdev_fh->ci.client_caps;
> >  
> >  		return 0;
> >  	}
> > @@ -1153,7 +1153,7 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
> >  		client_cap->capabilities &= (V4L2_SUBDEV_CLIENT_CAP_STREAMS |
> >  					     V4L2_SUBDEV_CLIENT_CAP_INTERVAL_USES_WHICH);
> >  
> > -		subdev_fh->client_caps = client_cap->capabilities;
> > +		subdev_fh->ci.client_caps = client_cap->capabilities;
> >  
> >  		return 0;
> >  	}
> > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> > index a25234befbe9..e83ef88fe12c 100644
> > --- a/include/media/v4l2-subdev.h
> > +++ b/include/media/v4l2-subdev.h
> > @@ -741,6 +741,14 @@ struct v4l2_subdev_state {
> >  	struct v4l2_subdev_stream_configs stream_configs;
> >  };
> >  
> > +/**
> > + * struct v4l2_subdev_client_info - Sub-device client information
> > + * @client_caps: bitmask of ``V4L2_SUBDEV_CLIENT_CAP_*``
> > + */
> > +struct v4l2_subdev_client_info {
> > +	u64 client_caps;
> 
> As this is now part of a structure called *client* info, maybe the field
> could be named just "caps" ?

Sounds good.

> 
> > +};
> > +
> >  /**
> >   * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
> >   *
> > @@ -1144,14 +1152,14 @@ struct v4l2_subdev {
> >   * @vfh: pointer to &struct v4l2_fh
> >   * @state: pointer to &struct v4l2_subdev_state
> >   * @owner: module pointer to the owner of this file handle
> > - * @client_caps: bitmask of ``V4L2_SUBDEV_CLIENT_CAP_*``
> > + * @ci: sub-device client info related to this file handle
> 
> I'm not a fan of "ci", I would write client_info to make the code more
> explicit.

Shorter names are typically used for function arguments and I'd prefer to
use the same name here. I don't have a strong opinion about this though.

> 
> >   */
> >  struct v4l2_subdev_fh {
> >  	struct v4l2_fh vfh;
> >  	struct module *owner;
> >  #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
> >  	struct v4l2_subdev_state *state;
> > -	u64 client_caps;
> > +	struct v4l2_subdev_client_info ci;
> >  #endif
> >  };
> >  
> 

-- 
Regards,

Sakari Ailus

[PATCH v5 09/10] media: v4l2-subdev: Add v4l2_subdev_get_fmt_ci()

[Sakari Ailus]

v4l2_subdev_get_fmt_ci() is just as v4l2_subdev_get_fmt(), but it also
takes const struct v4l2_subdev_client_info * as its second argument. It
can be used as set_fmt() pad op callback that simply returns the format
from sub-device state.

To be squashed to the previous patch.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c |  9 +++++++++
 include/media/v4l2-subdev.h           | 18 ++++++++++++++++++
 2 files changed, 27 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index e4ac6981e950..c1e1fb9d6773 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1953,6 +1953,15 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_get_fmt);
 
+int v4l2_subdev_get_fmt_ci(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
+			   struct v4l2_subdev_state *state,
+			   struct v4l2_subdev_format *format)
+{
+	return v4l2_subdev_call(sd, pad, get_fmt, state, format);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_get_fmt_ci);
+
 int v4l2_subdev_get_frame_interval(struct v4l2_subdev *sd,
 				   struct v4l2_subdev_state *state,
 				   struct v4l2_subdev_frame_interval *fi)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index e83ef88fe12c..e29defed6409 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1488,6 +1488,24 @@ __v4l2_subdev_state_get_interval(struct v4l2_subdev_state *state,
 int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
 			struct v4l2_subdev_format *format);
 
+/**
+ * v4l2_subdev_get_fmt_ci() - Get pad format using get_fmt pad op
+ * @sd: subdevice
+ * @ci: client info
+ * @state: subdevice state
+ * @format: pointer to &struct v4l2_subdev_format
+ *
+ * Obtain the sub-device format by calling the sub-device get_fmt pad op. This
+ * is intended to be used as the set_fmt callback when the callback just needs
+ * to return the format.
+ *
+ * Returns 0 on success, error value otherwise.
+ */
+int v4l2_subdev_get_fmt_ci(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
+			   struct v4l2_subdev_state *state,
+			   struct v4l2_subdev_format *format);
+
 /**
  * v4l2_subdev_get_frame_interval() - Fill frame interval based on state
  * @sd: subdevice
-- 
2.47.3

Re: [PATCH v5 09/10] media: v4l2-subdev: Add v4l2_subdev_get_fmt_ci()

[Laurent Pinchart]

On Mon, Jun 08, 2026 at 12:53:55AM +0300, Sakari Ailus wrote:
> v4l2_subdev_get_fmt_ci() is just as v4l2_subdev_get_fmt(), but it also
> takes const struct v4l2_subdev_client_info * as its second argument. It
> can be used as set_fmt() pad op callback that simply returns the format
> from sub-device state.

If the sole purpose of this function is to be used as a .set_fmt()
handler for drivers that don't support setting the format, I'm wondering
if we shouldn't instead simplify drivers and allow .set_fmt() being
NULL. We could redirect .set_fmt() to .get_fmt() in the call_set_fmt()
wrapper. Except that v4l2_subdev_call() returns -ENOIOCTLCMD when the
operation is NULL. We could swap the two conditions in the
v4l2_subdev_call() macro:

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index d256b7ec8f84..5392eca46a24 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1935,12 +1935,12 @@ extern const struct v4l2_subdev_ops v4l2_subdev_call_wrappers;
 		int __result;						\
 		if (!__sd)						\
 			__result = -ENODEV;				\
-		else if (!(__sd->ops->o && __sd->ops->o->f))		\
-			__result = -ENOIOCTLCMD;			\
 		else if (v4l2_subdev_call_wrappers.o &&			\
 			 v4l2_subdev_call_wrappers.o->f)		\
 			__result = v4l2_subdev_call_wrappers.o->f(	\
 							__sd, ##args);	\
+		else if (!(__sd->ops->o && __sd->ops->o->f))		\
+			__result = -ENOIOCTLCMD;			\
 		else							\
 			__result = __sd->ops->o->f(__sd, ##args);	\
 		__result;						\


and add a NULL checks in the call_*() wrappers in v4l2-subdev.c.

Would that be too complex ?

> To be squashed to the previous patch.

Why so ?

> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c |  9 +++++++++
>  include/media/v4l2-subdev.h           | 18 ++++++++++++++++++
>  2 files changed, 27 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index e4ac6981e950..c1e1fb9d6773 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1953,6 +1953,15 @@ int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
>  }
>  EXPORT_SYMBOL_GPL(v4l2_subdev_get_fmt);
>  
> +int v4l2_subdev_get_fmt_ci(struct v4l2_subdev *sd,
> +			   const struct v4l2_subdev_client_info *ci,
> +			   struct v4l2_subdev_state *state,
> +			   struct v4l2_subdev_format *format)
> +{
> +	return v4l2_subdev_call(sd, pad, get_fmt, state, format);
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_get_fmt_ci);
> +
>  int v4l2_subdev_get_frame_interval(struct v4l2_subdev *sd,
>  				   struct v4l2_subdev_state *state,
>  				   struct v4l2_subdev_frame_interval *fi)
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index e83ef88fe12c..e29defed6409 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1488,6 +1488,24 @@ __v4l2_subdev_state_get_interval(struct v4l2_subdev_state *state,
>  int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
>  			struct v4l2_subdev_format *format);
>  
> +/**
> + * v4l2_subdev_get_fmt_ci() - Get pad format using get_fmt pad op
> + * @sd: subdevice
> + * @ci: client info
> + * @state: subdevice state
> + * @format: pointer to &struct v4l2_subdev_format
> + *
> + * Obtain the sub-device format by calling the sub-device get_fmt pad op. This
> + * is intended to be used as the set_fmt callback when the callback just needs
> + * to return the format.
> + *
> + * Returns 0 on success, error value otherwise.
> + */
> +int v4l2_subdev_get_fmt_ci(struct v4l2_subdev *sd,
> +			   const struct v4l2_subdev_client_info *ci,
> +			   struct v4l2_subdev_state *state,
> +			   struct v4l2_subdev_format *format);
> +
>  /**
>   * v4l2_subdev_get_frame_interval() - Fill frame interval based on state
>   * @sd: subdevice

-- 
Regards,

Laurent Pinchart

[PATCH v5 10/10] media: v4l2-subdev: Add struct v4l2_subdev_client_info pointer to pad ops

[Sakari Ailus]

Add a pointer to const struct v4l2_subdev_client_info to the set_fmt,
get_selection and set_selection sub-device pad ops. The client info struct
will soon be used to differentiate UAPI based on client capabilities.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/adv7170.c                   |  1 +
 drivers/media/i2c/adv7175.c                   |  1 +
 drivers/media/i2c/adv7180.c                   |  3 +-
 drivers/media/i2c/adv7183.c                   |  3 +-
 drivers/media/i2c/adv748x/adv748x-afe.c       |  1 +
 drivers/media/i2c/adv748x/adv748x-csi2.c      |  1 +
 drivers/media/i2c/adv748x/adv748x-hdmi.c      |  1 +
 drivers/media/i2c/adv7511-v4l2.c              |  1 +
 drivers/media/i2c/adv7604.c                   |  2 +
 drivers/media/i2c/adv7842.c                   |  1 +
 drivers/media/i2c/ak881x.c                    |  3 +-
 drivers/media/i2c/alvium-csi2.c               |  3 +
 drivers/media/i2c/ar0521.c                    |  1 +
 drivers/media/i2c/ccs/ccs-core.c              | 12 +++-
 drivers/media/i2c/cx25840/cx25840-core.c      |  1 +
 drivers/media/i2c/ds90ub913.c                 |  1 +
 drivers/media/i2c/ds90ub953.c                 |  1 +
 drivers/media/i2c/ds90ub960.c                 |  1 +
 drivers/media/i2c/et8ek8/et8ek8_driver.c      |  1 +
 drivers/media/i2c/gc0308.c                    |  1 +
 drivers/media/i2c/gc0310.c                    |  3 +-
 drivers/media/i2c/gc05a2.c                    |  4 +-
 drivers/media/i2c/gc08a3.c                    |  4 +-
 drivers/media/i2c/gc2145.c                    |  2 +
 drivers/media/i2c/hi556.c                     |  2 +
 drivers/media/i2c/hi846.c                     |  2 +
 drivers/media/i2c/hi847.c                     |  1 +
 drivers/media/i2c/imx111.c                    |  1 +
 drivers/media/i2c/imx208.c                    |  1 +
 drivers/media/i2c/imx214.c                    |  4 +-
 drivers/media/i2c/imx219.c                    |  4 +-
 drivers/media/i2c/imx258.c                    |  2 +
 drivers/media/i2c/imx274.c                    |  3 +
 drivers/media/i2c/imx283.c                    |  2 +
 drivers/media/i2c/imx290.c                    |  4 +-
 drivers/media/i2c/imx296.c                    |  7 ++-
 drivers/media/i2c/imx319.c                    |  1 +
 drivers/media/i2c/imx334.c                    |  3 +-
 drivers/media/i2c/imx335.c                    |  4 +-
 drivers/media/i2c/imx355.c                    |  1 +
 drivers/media/i2c/imx412.c                    |  3 +-
 drivers/media/i2c/imx415.c                    |  4 +-
 drivers/media/i2c/isl7998x.c                  |  1 +
 drivers/media/i2c/lt6911uxe.c                 |  5 +-
 drivers/media/i2c/max9286.c                   |  1 +
 drivers/media/i2c/max96714.c                  |  1 +
 drivers/media/i2c/max96717.c                  |  1 +
 drivers/media/i2c/ml86v7667.c                 |  2 +-
 drivers/media/i2c/mt9m001.c                   |  5 +-
 drivers/media/i2c/mt9m111.c                   |  3 +
 drivers/media/i2c/mt9m114.c                   |  6 ++
 drivers/media/i2c/mt9p031.c                   |  3 +
 drivers/media/i2c/mt9t112.c                   |  3 +
 drivers/media/i2c/mt9v011.c                   |  1 +
 drivers/media/i2c/mt9v032.c                   |  3 +
 drivers/media/i2c/mt9v111.c                   |  1 +
 drivers/media/i2c/og01a1b.c                   |  3 +-
 drivers/media/i2c/og0ve1b.c                   |  3 +-
 drivers/media/i2c/os05b10.c                   |  2 +
 drivers/media/i2c/ov01a10.c                   |  3 +
 drivers/media/i2c/ov02a10.c                   |  3 +-
 drivers/media/i2c/ov02c10.c                   |  1 +
 drivers/media/i2c/ov02e10.c                   |  1 +
 drivers/media/i2c/ov08d10.c                   |  1 +
 drivers/media/i2c/ov08x40.c                   |  1 +
 drivers/media/i2c/ov13858.c                   |  1 +
 drivers/media/i2c/ov13b10.c                   |  1 +
 drivers/media/i2c/ov2640.c                    |  2 +
 drivers/media/i2c/ov2659.c                    |  1 +
 drivers/media/i2c/ov2680.c                    |  3 +
 drivers/media/i2c/ov2685.c                    |  2 +
 drivers/media/i2c/ov2732.c                    |  4 +-
 drivers/media/i2c/ov2735.c                    |  4 +-
 drivers/media/i2c/ov2740.c                    |  1 +
 drivers/media/i2c/ov4689.c                    |  2 +
 drivers/media/i2c/ov5640.c                    |  2 +
 drivers/media/i2c/ov5645.c                    |  4 +-
 drivers/media/i2c/ov5647.c                    |  2 +
 drivers/media/i2c/ov5648.c                    |  1 +
 drivers/media/i2c/ov5670.c                    |  2 +
 drivers/media/i2c/ov5675.c                    |  2 +
 drivers/media/i2c/ov5693.c                    |  3 +
 drivers/media/i2c/ov5695.c                    |  1 +
 drivers/media/i2c/ov6211.c                    |  3 +-
 drivers/media/i2c/ov64a40.c                   |  2 +
 drivers/media/i2c/ov7251.c                    |  4 +-
 drivers/media/i2c/ov7670.c                    |  1 +
 drivers/media/i2c/ov772x.c                    |  2 +
 drivers/media/i2c/ov7740.c                    |  1 +
 drivers/media/i2c/ov8856.c                    |  1 +
 drivers/media/i2c/ov8858.c                    |  3 +-
 drivers/media/i2c/ov8865.c                    |  2 +
 drivers/media/i2c/ov9282.c                    |  4 +-
 drivers/media/i2c/ov9640.c                    |  2 +
 drivers/media/i2c/ov9650.c                    |  1 +
 drivers/media/i2c/ov9734.c                    |  1 +
 drivers/media/i2c/rdacm20.c                   |  2 +-
 drivers/media/i2c/rdacm21.c                   |  2 +-
 drivers/media/i2c/rj54n1cb0c.c                |  3 +
 drivers/media/i2c/s5c73m3/s5c73m3-core.c      |  2 +
 drivers/media/i2c/s5k3m5.c                    |  4 +-
 drivers/media/i2c/s5k5baf.c                   |  3 +
 drivers/media/i2c/s5k6a3.c                    |  1 +
 drivers/media/i2c/s5kjn1.c                    |  4 +-
 drivers/media/i2c/saa6752hs.c                 |  1 +
 drivers/media/i2c/saa7115.c                   |  1 +
 drivers/media/i2c/saa717x.c                   |  1 +
 drivers/media/i2c/st-mipid02.c                |  1 +
 drivers/media/i2c/t4ka3.c                     |  3 +
 drivers/media/i2c/tc358743.c                  |  1 +
 drivers/media/i2c/tc358746.c                  |  1 +
 drivers/media/i2c/tda1997x.c                  |  1 +
 drivers/media/i2c/thp7312.c                   |  1 +
 drivers/media/i2c/tvp514x.c                   |  1 +
 drivers/media/i2c/tvp5150.c                   |  4 +-
 drivers/media/i2c/tvp7002.c                   |  1 +
 drivers/media/i2c/tw9900.c                    |  1 +
 drivers/media/i2c/tw9910.c                    |  2 +
 drivers/media/i2c/vd55g1.c                    |  4 +-
 drivers/media/i2c/vd56g3.c                    |  4 +-
 drivers/media/i2c/vgxy61.c                    |  4 +-
 drivers/media/pci/cobalt/cobalt-driver.c      |  8 +--
 drivers/media/pci/cobalt/cobalt-v4l2.c        | 10 +--
 drivers/media/pci/cx18/cx18-av-core.c         |  1 +
 drivers/media/pci/cx18/cx18-controls.c        |  2 +-
 drivers/media/pci/cx18/cx18-ioctl.c           |  2 +-
 drivers/media/pci/cx23885/cx23885-video.c     |  4 +-
 drivers/media/pci/intel/ipu3/ipu3-cio2.c      |  1 +
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c |  2 +
 .../media/pci/intel/ipu6/ipu6-isys-subdev.c   |  1 +
 .../media/pci/intel/ipu6/ipu6-isys-subdev.h   |  1 +
 drivers/media/pci/intel/ivsc/mei_csi.c        |  1 +
 drivers/media/pci/ivtv/ivtv-controls.c        |  2 +-
 drivers/media/pci/ivtv/ivtv-ioctl.c           |  2 +-
 drivers/media/pci/saa7134/saa7134-empress.c   |  4 +-
 drivers/media/platform/amd/isp4/isp4_subdev.c |  1 +
 drivers/media/platform/amd/isp4/isp4_video.c  |  2 +-
 .../platform/amlogic/c3/isp/c3-isp-core.c     |  1 +
 .../platform/amlogic/c3/isp/c3-isp-resizer.c  |  3 +
 .../amlogic/c3/mipi-adapter/c3-mipi-adap.c    |  1 +
 .../amlogic/c3/mipi-csi2/c3-mipi-csi2.c       |  1 +
 .../platform/arm/mali-c55/mali-c55-isp.c      |  3 +
 .../platform/arm/mali-c55/mali-c55-resizer.c  | 15 +++--
 .../platform/arm/mali-c55/mali-c55-tpg.c      |  1 +
 drivers/media/platform/atmel/atmel-isi.c      |  4 +-
 .../media/platform/broadcom/bcm2835-unicam.c  |  1 +
 drivers/media/platform/cadence/cdns-csi2rx.c  |  1 +
 drivers/media/platform/cadence/cdns-csi2tx.c  |  1 +
 drivers/media/platform/intel/pxa_camera.c     |  6 +-
 drivers/media/platform/marvell/mcam-core.c    |  4 +-
 .../platform/microchip/microchip-csi2dc.c     |  1 +
 .../platform/microchip/microchip-isc-scaler.c |  2 +
 drivers/media/platform/nxp/imx-mipi-csis.c    |  3 +-
 drivers/media/platform/nxp/imx7-media-csi.c   |  1 +
 .../platform/nxp/imx8-isi/imx8-isi-crossbar.c |  1 +
 .../platform/nxp/imx8-isi/imx8-isi-pipe.c     |  3 +
 drivers/media/platform/nxp/imx8mq-mipi-csi2.c |  1 +
 .../media/platform/qcom/camss/camss-csid.c    |  3 +-
 .../media/platform/qcom/camss/camss-csiphy.c  |  3 +-
 .../media/platform/qcom/camss/camss-ispif.c   |  3 +-
 drivers/media/platform/qcom/camss/camss-tpg.c |  3 +-
 drivers/media/platform/qcom/camss/camss-vfe.c | 12 ++--
 .../media/platform/raspberrypi/rp1-cfe/csi2.c |  1 +
 .../platform/raspberrypi/rp1-cfe/pisp-fe.c    |  1 +
 drivers/media/platform/renesas/rcar-csi2.c    |  1 +
 .../media/platform/renesas/rcar-isp/csisp.c   |  1 +
 drivers/media/platform/renesas/renesas-ceu.c  |  7 ++-
 .../platform/renesas/rzg2l-cru/rzg2l-csi2.c   |  3 +-
 .../platform/renesas/rzg2l-cru/rzg2l-ip.c     |  3 +-
 .../renesas/rzv2h-ivc/rzv2h-ivc-subdev.c      |  1 +
 drivers/media/platform/renesas/sh_vou.c       |  4 +-
 .../media/platform/renesas/vsp1/vsp1_brx.c    |  3 +
 .../media/platform/renesas/vsp1/vsp1_drm.c    | 18 +++---
 .../media/platform/renesas/vsp1/vsp1_entity.c |  4 +-
 .../media/platform/renesas/vsp1/vsp1_entity.h |  1 +
 .../media/platform/renesas/vsp1/vsp1_histo.c  |  3 +
 .../media/platform/renesas/vsp1/vsp1_hsit.c   |  1 +
 .../media/platform/renesas/vsp1/vsp1_rwpf.c   |  3 +
 .../media/platform/renesas/vsp1/vsp1_sru.c    |  1 +
 .../media/platform/renesas/vsp1/vsp1_uds.c    |  1 +
 .../media/platform/renesas/vsp1/vsp1_uif.c    |  2 +
 .../media/platform/renesas/vsp1/vsp1_vspx.c   |  3 +-
 .../platform/rockchip/rkcif/rkcif-interface.c |  3 +
 .../platform/rockchip/rkisp1/rkisp1-csi.c     |  1 +
 .../platform/rockchip/rkisp1/rkisp1-isp.c     |  3 +
 .../platform/rockchip/rkisp1/rkisp1-resizer.c |  3 +
 .../samsung/exynos4-is/fimc-capture.c         |  7 ++-
 .../platform/samsung/exynos4-is/fimc-isp.c    |  1 +
 .../platform/samsung/exynos4-is/fimc-lite.c   |  3 +
 .../platform/samsung/exynos4-is/mipi-csis.c   |  1 +
 .../samsung/s3c-camif/camif-capture.c         |  3 +
 .../platform/samsung/s3c-camif/camif-core.c   |  2 +-
 drivers/media/platform/st/stm32/stm32-csi.c   |  1 +
 drivers/media/platform/st/stm32/stm32-dcmi.c  |  7 ++-
 .../st/stm32/stm32-dcmipp/dcmipp-byteproc.c   |  3 +
 .../st/stm32/stm32-dcmipp/dcmipp-input.c      |  1 +
 .../platform/sunxi/sun4i-csi/sun4i_v4l2.c     |  1 +
 .../sunxi/sun6i-csi/sun6i_csi_bridge.c        |  1 +
 .../sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c   |  1 +
 .../sun8i_a83t_mipi_csi2.c                    |  1 +
 .../media/platform/synopsys/dw-mipi-csi2rx.c  |  1 +
 .../media/platform/ti/am437x/am437x-vpfe.c    |  2 +-
 drivers/media/platform/ti/cal/cal-camerarx.c  |  1 +
 drivers/media/platform/ti/cal/cal-video.c     |  4 +-
 .../platform/ti/j721e-csi2rx/j721e-csi2rx.c   |  1 +
 drivers/media/platform/ti/omap3isp/ispccdc.c  |  5 +-
 drivers/media/platform/ti/omap3isp/ispccp2.c  |  3 +-
 drivers/media/platform/ti/omap3isp/ispcsi2.c  |  3 +-
 .../media/platform/ti/omap3isp/isppreview.c   |  5 +-
 .../media/platform/ti/omap3isp/ispresizer.c   |  5 +-
 drivers/media/platform/ti/omap3isp/ispvideo.c |  4 +-
 drivers/media/platform/ti/vpe/vip.c           |  4 +-
 drivers/media/platform/via/via-camera.c       |  4 +-
 drivers/media/platform/video-mux.c            |  1 +
 .../media/platform/xilinx/xilinx-csi2rxss.c   |  1 +
 drivers/media/platform/xilinx/xilinx-tpg.c    |  1 +
 .../media/test-drivers/vimc/vimc-debayer.c    |  1 +
 drivers/media/test-drivers/vimc/vimc-scaler.c |  3 +
 drivers/media/test-drivers/vimc/vimc-sensor.c |  1 +
 drivers/media/usb/cx231xx/cx231xx-417.c       |  2 +-
 drivers/media/usb/cx231xx/cx231xx-video.c     |  4 +-
 drivers/media/usb/dvb-usb/cxusb-analog.c      |  6 +-
 drivers/media/usb/em28xx/em28xx-camera.c      |  2 +-
 drivers/media/usb/go7007/go7007-v4l2.c        |  2 +-
 drivers/media/usb/go7007/s2250-board.c        |  1 +
 drivers/media/usb/pvrusb2/pvrusb2-hdw.c       |  2 +-
 drivers/media/v4l2-core/v4l2-subdev.c         | 61 ++++++++++++++-----
 .../media/atomisp/i2c/atomisp-gc2235.c        |  1 +
 .../media/atomisp/i2c/atomisp-ov2722.c        |  1 +
 .../staging/media/atomisp/pci/atomisp_cmd.c   | 16 +++--
 .../staging/media/atomisp/pci/atomisp_csi2.c  |  1 +
 .../media/atomisp/pci/atomisp_subdev.c        |  3 +
 .../staging/media/atomisp/pci/atomisp_v4l2.c  |  8 +--
 drivers/staging/media/imx/imx-ic-prp.c        |  1 +
 drivers/staging/media/imx/imx-ic-prpencvf.c   |  1 +
 drivers/staging/media/imx/imx-media-csi.c     |  3 +
 drivers/staging/media/imx/imx-media-vdic.c    |  1 +
 drivers/staging/media/imx/imx6-mipi-csi2.c    |  1 +
 drivers/staging/media/ipu3/ipu3-v4l2.c        |  3 +
 drivers/staging/media/ipu7/ipu7-isys-csi2.c   |  2 +
 drivers/staging/media/ipu7/ipu7-isys-subdev.c |  1 +
 drivers/staging/media/ipu7/ipu7-isys-subdev.h |  1 +
 .../media/sunxi/sun6i-isp/sun6i_isp_proc.c    |  1 +
 drivers/staging/media/tegra-video/csi.c       |  1 +
 drivers/staging/media/tegra-video/vi.c        | 10 +--
 include/media/v4l2-subdev.h                   |  3 +
 246 files changed, 539 insertions(+), 151 deletions(-)

diff --git a/drivers/media/i2c/adv7170.c b/drivers/media/i2c/adv7170.c
index 812998729207..9a7edd6583c1 100644
--- a/drivers/media/i2c/adv7170.c
+++ b/drivers/media/i2c/adv7170.c
@@ -284,6 +284,7 @@ static int adv7170_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int adv7170_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/adv7175.c b/drivers/media/i2c/adv7175.c
index f1caab8e2abd..aedb54f1857b 100644
--- a/drivers/media/i2c/adv7175.c
+++ b/drivers/media/i2c/adv7175.c
@@ -322,6 +322,7 @@ static int adv7175_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int adv7175_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/adv7180.c b/drivers/media/i2c/adv7180.c
index e5d11a6e6766..79a592f1831d 100644
--- a/drivers/media/i2c/adv7180.c
+++ b/drivers/media/i2c/adv7180.c
@@ -771,6 +771,7 @@ static int adv7180_get_pad_format(struct v4l2_subdev *sd,
 }
 
 static int adv7180_set_pad_format(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *sd_state,
 				  struct v4l2_subdev_format *format)
 {
@@ -808,7 +809,7 @@ static int adv7180_init_state(struct v4l2_subdev *sd,
 		: V4L2_SUBDEV_FORMAT_ACTIVE,
 	};
 
-	return adv7180_set_pad_format(sd, sd_state, &fmt);
+	return adv7180_set_pad_format(sd, NULL, sd_state, &fmt);
 }
 
 static int adv7180_get_mbus_config(struct v4l2_subdev *sd,
diff --git a/drivers/media/i2c/adv7183.c b/drivers/media/i2c/adv7183.c
index a04a1a205fe0..9e4cbebe6e5a 100644
--- a/drivers/media/i2c/adv7183.c
+++ b/drivers/media/i2c/adv7183.c
@@ -420,6 +420,7 @@ static int adv7183_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int adv7183_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
@@ -598,7 +599,7 @@ static int adv7183_probe(struct i2c_client *client)
 	adv7183_s_std(sd, decoder->std);
 	fmt.format.width = 720;
 	fmt.format.height = 576;
-	adv7183_set_fmt(sd, NULL, &fmt);
+	adv7183_set_fmt(sd, NULL, NULL, &fmt);
 
 	/* initialize the hardware to the default control values */
 	ret = v4l2_ctrl_handler_setup(hdl);
diff --git a/drivers/media/i2c/adv748x/adv748x-afe.c b/drivers/media/i2c/adv748x/adv748x-afe.c
index 678199196b84..28b755df30cf 100644
--- a/drivers/media/i2c/adv748x/adv748x-afe.c
+++ b/drivers/media/i2c/adv748x/adv748x-afe.c
@@ -349,6 +349,7 @@ static int adv748x_afe_get_format(struct v4l2_subdev *sd,
 }
 
 static int adv748x_afe_set_format(struct v4l2_subdev *sd,
+				      const struct v4l2_subdev_client_info *ci,
 				      struct v4l2_subdev_state *sd_state,
 				      struct v4l2_subdev_format *sdformat)
 {
diff --git a/drivers/media/i2c/adv748x/adv748x-csi2.c b/drivers/media/i2c/adv748x/adv748x-csi2.c
index ebe7da8ebed7..1d7ab685d229 100644
--- a/drivers/media/i2c/adv748x/adv748x-csi2.c
+++ b/drivers/media/i2c/adv748x/adv748x-csi2.c
@@ -226,6 +226,7 @@ static bool adv748x_csi2_is_fmt_supported(struct adv748x_csi2 *tx, u32 code)
 }
 
 static int adv748x_csi2_set_format(struct v4l2_subdev *sd,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *sd_state,
 				   struct v4l2_subdev_format *sdformat)
 {
diff --git a/drivers/media/i2c/adv748x/adv748x-hdmi.c b/drivers/media/i2c/adv748x/adv748x-hdmi.c
index b154dea29ba2..8d5ebb369961 100644
--- a/drivers/media/i2c/adv748x/adv748x-hdmi.c
+++ b/drivers/media/i2c/adv748x/adv748x-hdmi.c
@@ -440,6 +440,7 @@ static int adv748x_hdmi_get_format(struct v4l2_subdev *sd,
 }
 
 static int adv748x_hdmi_set_format(struct v4l2_subdev *sd,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *sd_state,
 				   struct v4l2_subdev_format *sdformat)
 {
diff --git a/drivers/media/i2c/adv7511-v4l2.c b/drivers/media/i2c/adv7511-v4l2.c
index 860cff50c522..27e0e2c31dbd 100644
--- a/drivers/media/i2c/adv7511-v4l2.c
+++ b/drivers/media/i2c/adv7511-v4l2.c
@@ -1273,6 +1273,7 @@ static int adv7511_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int adv7511_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/adv7604.c b/drivers/media/i2c/adv7604.c
index ac9c69ce438f..433b8c1d598c 100644
--- a/drivers/media/i2c/adv7604.c
+++ b/drivers/media/i2c/adv7604.c
@@ -1943,6 +1943,7 @@ static int adv76xx_get_format(struct v4l2_subdev *sd,
 }
 
 static int adv76xx_get_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -1963,6 +1964,7 @@ static int adv76xx_get_selection(struct v4l2_subdev *sd,
 }
 
 static int adv76xx_set_format(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/adv7842.c b/drivers/media/i2c/adv7842.c
index 3cfae89ce944..7fe394da0e6a 100644
--- a/drivers/media/i2c/adv7842.c
+++ b/drivers/media/i2c/adv7842.c
@@ -2110,6 +2110,7 @@ static int adv7842_get_format(struct v4l2_subdev *sd,
 }
 
 static int adv7842_set_format(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/ak881x.c b/drivers/media/i2c/ak881x.c
index cea46f01997d..f18a42c1fa5c 100644
--- a/drivers/media/i2c/ak881x.c
+++ b/drivers/media/i2c/ak881x.c
@@ -122,6 +122,7 @@ static int ak881x_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int ak881x_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -216,7 +217,7 @@ static const struct v4l2_subdev_video_ops ak881x_subdev_video_ops = {
 static const struct v4l2_subdev_pad_ops ak881x_subdev_pad_ops = {
 	.enum_mbus_code = ak881x_enum_mbus_code,
 	.get_selection	= ak881x_get_selection,
-	.set_fmt	= ak881x_fill_fmt,
+	.set_fmt	= v4l2_subdev_get_fmt_ci,
 	.get_fmt	= ak881x_fill_fmt,
 };
 
diff --git a/drivers/media/i2c/alvium-csi2.c b/drivers/media/i2c/alvium-csi2.c
index dd991c2ee700..d9a5497855f7 100644
--- a/drivers/media/i2c/alvium-csi2.c
+++ b/drivers/media/i2c/alvium-csi2.c
@@ -1887,6 +1887,7 @@ static int alvium_init_state(struct v4l2_subdev *sd,
 }
 
 static int alvium_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
@@ -1922,6 +1923,7 @@ static int alvium_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int alvium_set_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -1957,6 +1959,7 @@ static int alvium_set_selection(struct v4l2_subdev *sd,
 }
 
 static int alvium_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ar0521.c b/drivers/media/i2c/ar0521.c
index ed324c2d87aa..6e07bc923105 100644
--- a/drivers/media/i2c/ar0521.c
+++ b/drivers/media/i2c/ar0521.c
@@ -457,6 +457,7 @@ static int ar0521_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int ar0521_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 8e25f970fd12..9f3c5f00b37c 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -2189,6 +2189,7 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 }
 
 static int ccs_set_format_source(struct v4l2_subdev *subdev,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -2242,6 +2243,7 @@ static int ccs_set_format_source(struct v4l2_subdev *subdev,
 }
 
 static int ccs_set_format(struct v4l2_subdev *subdev,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
@@ -2252,7 +2254,7 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
 	if (fmt->pad == ssd->source_pad) {
 		int rval;
 
-		rval = ccs_set_format_source(subdev, sd_state, fmt);
+		rval = ccs_set_format_source(subdev, NULL, sd_state, fmt);
 
 		return rval;
 	}
@@ -2467,6 +2469,7 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,
 }
 /* We're only called on source pads. This function sets scaling. */
 static int ccs_set_compose(struct v4l2_subdev *subdev,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_selection *sel)
 {
@@ -2536,6 +2539,7 @@ static int ccs_sel_supported(struct v4l2_subdev *subdev,
 }
 
 static int ccs_set_crop(struct v4l2_subdev *subdev,
+			const struct v4l2_subdev_client_info *ci,
 			struct v4l2_subdev_state *sd_state,
 			struct v4l2_subdev_selection *sel)
 {
@@ -2587,6 +2591,7 @@ static void ccs_get_native_size(struct ccs_subdev *ssd, struct v4l2_rect *r)
 }
 
 static int ccs_get_selection(struct v4l2_subdev *subdev,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel)
 {
@@ -2633,6 +2638,7 @@ static int ccs_get_selection(struct v4l2_subdev *subdev,
 }
 
 static int ccs_set_selection(struct v4l2_subdev *subdev,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel)
 {
@@ -2655,10 +2661,10 @@ static int ccs_set_selection(struct v4l2_subdev *subdev,
 
 	switch (sel->target) {
 	case V4L2_SEL_TGT_CROP:
-		ret = ccs_set_crop(subdev, sd_state, sel);
+		ret = ccs_set_crop(subdev, NULL, sd_state, sel);
 		break;
 	case V4L2_SEL_TGT_COMPOSE:
-		ret = ccs_set_compose(subdev, sd_state, sel);
+		ret = ccs_set_compose(subdev, NULL, sd_state, sel);
 		break;
 	default:
 		ret = -EINVAL;
diff --git a/drivers/media/i2c/cx25840/cx25840-core.c b/drivers/media/i2c/cx25840/cx25840-core.c
index 8110d40931d9..f2cd3ff208b0 100644
--- a/drivers/media/i2c/cx25840/cx25840-core.c
+++ b/drivers/media/i2c/cx25840/cx25840-core.c
@@ -1771,6 +1771,7 @@ static int cx25840_s_ctrl(struct v4l2_ctrl *ctrl)
 /* ----------------------------------------------------------------------- */
 
 static int cx25840_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/ds90ub913.c b/drivers/media/i2c/ds90ub913.c
index 6abb5e324ae1..ca0b07d7109b 100644
--- a/drivers/media/i2c/ds90ub913.c
+++ b/drivers/media/i2c/ds90ub913.c
@@ -373,6 +373,7 @@ static int ub913_set_routing(struct v4l2_subdev *sd,
 }
 
 static int ub913_set_fmt(struct v4l2_subdev *sd,
+			 const struct v4l2_subdev_client_info *ci,
 			 struct v4l2_subdev_state *state,
 			 struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/ds90ub953.c b/drivers/media/i2c/ds90ub953.c
index d4228e1134ff..e6ae9cadefa2 100644
--- a/drivers/media/i2c/ds90ub953.c
+++ b/drivers/media/i2c/ds90ub953.c
@@ -426,6 +426,7 @@ static int ub953_set_routing(struct v4l2_subdev *sd,
 
 
 static int ub953_set_fmt(struct v4l2_subdev *sd,
+			 const struct v4l2_subdev_client_info *ci,
 			 struct v4l2_subdev_state *state,
 			 struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/ds90ub960.c b/drivers/media/i2c/ds90ub960.c
index 15a9797b47ac..7d7ba6d349fc 100644
--- a/drivers/media/i2c/ds90ub960.c
+++ b/drivers/media/i2c/ds90ub960.c
@@ -4045,6 +4045,7 @@ static int ub960_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
 }
 
 static int ub960_set_fmt(struct v4l2_subdev *sd,
+			 const struct v4l2_subdev_client_info *ci,
 			 struct v4l2_subdev_state *state,
 			 struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/et8ek8/et8ek8_driver.c b/drivers/media/i2c/et8ek8/et8ek8_driver.c
index 738e2801016a..e8a7b7d87bb1 100644
--- a/drivers/media/i2c/et8ek8/et8ek8_driver.c
+++ b/drivers/media/i2c/et8ek8/et8ek8_driver.c
@@ -1013,6 +1013,7 @@ static int et8ek8_get_pad_format(struct v4l2_subdev *subdev,
 }
 
 static int et8ek8_set_pad_format(struct v4l2_subdev *subdev,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/gc0308.c b/drivers/media/i2c/gc0308.c
index cbcda0e18ff1..064766b6f627 100644
--- a/drivers/media/i2c/gc0308.c
+++ b/drivers/media/i2c/gc0308.c
@@ -1044,6 +1044,7 @@ static void gc0308_update_pad_format(const struct gc0308_frame_size *mode,
 }
 
 static int gc0308_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/gc0310.c b/drivers/media/i2c/gc0310.c
index 7af4d66f42a0..55db5dee22c8 100644
--- a/drivers/media/i2c/gc0310.c
+++ b/drivers/media/i2c/gc0310.c
@@ -362,6 +362,7 @@ static void gc0310_fill_format(struct v4l2_mbus_framefmt *fmt)
 }
 
 static int gc0310_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -534,7 +535,7 @@ static const struct v4l2_subdev_pad_ops gc0310_pad_ops = {
 	.enum_mbus_code = gc0310_enum_mbus_code,
 	.enum_frame_size = gc0310_enum_frame_size,
 	.get_fmt = v4l2_subdev_get_fmt,
-	.set_fmt = v4l2_subdev_get_fmt, /* Only 1 fixed mode supported */
+	.set_fmt = v4l2_subdev_get_fmt_ci, /* Only 1 fixed mode supported */
 	.get_selection = gc0310_get_selection,
 	.set_selection = gc0310_get_selection,
 	.enable_streams = gc0310_enable_streams,
diff --git a/drivers/media/i2c/gc05a2.c b/drivers/media/i2c/gc05a2.c
index 8ba17f80fffe..e2cae5f3f77e 100644
--- a/drivers/media/i2c/gc05a2.c
+++ b/drivers/media/i2c/gc05a2.c
@@ -731,6 +731,7 @@ static void gc05a2_update_pad_format(struct gc05a2 *gc08a3,
 }
 
 static int gc05a2_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *fmt)
 {
@@ -762,6 +763,7 @@ static int gc05a2_set_format(struct v4l2_subdev *sd,
 }
 
 static int gc05a2_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -796,7 +798,7 @@ static int gc05a2_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	gc05a2_set_format(sd, state, &fmt);
+	gc05a2_set_format(sd, NULL, state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/gc08a3.c b/drivers/media/i2c/gc08a3.c
index 11fd936db9c3..ce80976e3a2a 100644
--- a/drivers/media/i2c/gc08a3.c
+++ b/drivers/media/i2c/gc08a3.c
@@ -706,6 +706,7 @@ static void gc08a3_update_pad_format(struct gc08a3 *gc08a3,
 }
 
 static int gc08a3_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *fmt)
 {
@@ -738,6 +739,7 @@ static int gc08a3_set_format(struct v4l2_subdev *sd,
 }
 
 static int gc08a3_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -772,7 +774,7 @@ static int gc08a3_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	gc08a3_set_format(sd, state, &fmt);
+	gc08a3_set_format(sd, NULL, state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/gc2145.c b/drivers/media/i2c/gc2145.c
index b215963a2648..661a59641f19 100644
--- a/drivers/media/i2c/gc2145.c
+++ b/drivers/media/i2c/gc2145.c
@@ -709,6 +709,7 @@ static int gc2145_init_state(struct v4l2_subdev *sd,
 }
 
 static int gc2145_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -774,6 +775,7 @@ static int gc2145_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int gc2145_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/hi556.c b/drivers/media/i2c/hi556.c
index de573cee4451..4c13adf4a900 100644
--- a/drivers/media/i2c/hi556.c
+++ b/drivers/media/i2c/hi556.c
@@ -960,6 +960,7 @@ __hi556_get_pad_crop(struct hi556 *hi556,
 }
 
 static int hi556_get_selection(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_selection *sel)
 {
@@ -1071,6 +1072,7 @@ static int hi556_set_stream(struct v4l2_subdev *sd, int enable)
 }
 
 static int hi556_set_format(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/hi846.c b/drivers/media/i2c/hi846.c
index a3f77b8434ca..025f8bcd17bc 100644
--- a/drivers/media/i2c/hi846.c
+++ b/drivers/media/i2c/hi846.c
@@ -1688,6 +1688,7 @@ static int __maybe_unused hi846_resume(struct device *dev)
 }
 
 static int hi846_set_format(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *format)
 {
@@ -1840,6 +1841,7 @@ static int hi846_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int hi846_get_selection(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/hi847.c b/drivers/media/i2c/hi847.c
index def01aa07b2f..601de43be92f 100644
--- a/drivers/media/i2c/hi847.c
+++ b/drivers/media/i2c/hi847.c
@@ -2639,6 +2639,7 @@ static int hi847_set_stream(struct v4l2_subdev *sd, int enable)
 }
 
 static int hi847_set_format(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/imx111.c b/drivers/media/i2c/imx111.c
index 8eb919788ef7..5b860fa052b1 100644
--- a/drivers/media/i2c/imx111.c
+++ b/drivers/media/i2c/imx111.c
@@ -1122,6 +1122,7 @@ static int imx111_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int imx111_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/imx208.c b/drivers/media/i2c/imx208.c
index d5350bb46f14..3a899b41a9d2 100644
--- a/drivers/media/i2c/imx208.c
+++ b/drivers/media/i2c/imx208.c
@@ -574,6 +574,7 @@ static int imx208_get_pad_format(struct v4l2_subdev *sd,
 }
 
 static int imx208_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/imx214.c b/drivers/media/i2c/imx214.c
index d4945b192776..96833f2bf287 100644
--- a/drivers/media/i2c/imx214.c
+++ b/drivers/media/i2c/imx214.c
@@ -662,6 +662,7 @@ static const struct v4l2_subdev_core_ops imx214_core_ops = {
 };
 
 static int imx214_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_format *format)
 {
@@ -717,6 +718,7 @@ static int imx214_set_format(struct v4l2_subdev *sd,
 }
 
 static int imx214_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -754,7 +756,7 @@ static int imx214_entity_init_state(struct v4l2_subdev *subdev,
 	fmt.format.width = imx214_modes[0].width;
 	fmt.format.height = imx214_modes[0].height;
 
-	imx214_set_format(subdev, sd_state, &fmt);
+	imx214_set_format(subdev, NULL, sd_state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 3cee31758b7e..a7d4c25e4e6e 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -819,6 +819,7 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int imx219_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -934,6 +935,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 }
 
 static int imx219_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -976,7 +978,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	return imx219_set_pad_format(sd, state, &fmt);
+	return imx219_set_pad_format(sd, NULL, state, &fmt);
 }
 
 static const struct v4l2_subdev_video_ops imx219_video_ops = {
diff --git a/drivers/media/i2c/imx258.c b/drivers/media/i2c/imx258.c
index bc9ee449a87c..065c33380f60 100644
--- a/drivers/media/i2c/imx258.c
+++ b/drivers/media/i2c/imx258.c
@@ -913,6 +913,7 @@ static int imx258_get_pad_format(struct v4l2_subdev *sd,
 }
 
 static int imx258_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -988,6 +989,7 @@ __imx258_get_pad_crop(struct imx258 *imx258,
 }
 
 static int imx258_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/imx274.c b/drivers/media/i2c/imx274.c
index 76da647f9cf9..3245afc8ae5b 100644
--- a/drivers/media/i2c/imx274.c
+++ b/drivers/media/i2c/imx274.c
@@ -1059,6 +1059,7 @@ static int imx274_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int imx274_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
@@ -1092,6 +1093,7 @@ static int imx274_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int imx274_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -1208,6 +1210,7 @@ static int imx274_set_selection_crop(struct stimx274 *imx274,
 }
 
 static int imx274_set_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/imx283.c b/drivers/media/i2c/imx283.c
index 38ea1902f2c2..e3cd27ca238e 100644
--- a/drivers/media/i2c/imx283.c
+++ b/drivers/media/i2c/imx283.c
@@ -958,6 +958,7 @@ static void imx283_set_framing_limits(struct imx283 *imx283,
 }
 
 static int imx283_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -1260,6 +1261,7 @@ static int imx283_identify_module(struct imx283 *imx283)
 }
 
 static int imx283_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/imx290.c b/drivers/media/i2c/imx290.c
index 21cbc81cb2ed..d562a9aa455c 100644
--- a/drivers/media/i2c/imx290.c
+++ b/drivers/media/i2c/imx290.c
@@ -1149,6 +1149,7 @@ static int imx290_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int imx290_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
@@ -1187,6 +1188,7 @@ static int imx290_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int imx290_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -1246,7 +1248,7 @@ static int imx290_entity_init_state(struct v4l2_subdev *subdev,
 		},
 	};
 
-	imx290_set_fmt(subdev, sd_state, &fmt);
+	imx290_set_fmt(subdev, NULL, sd_state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/imx296.c b/drivers/media/i2c/imx296.c
index 69636db11a2b..74bb295799fd 100644
--- a/drivers/media/i2c/imx296.c
+++ b/drivers/media/i2c/imx296.c
@@ -675,6 +675,7 @@ static int imx296_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int imx296_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *fmt)
 {
@@ -726,6 +727,7 @@ static int imx296_set_format(struct v4l2_subdev *sd,
 }
 
 static int imx296_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -751,6 +753,7 @@ static int imx296_get_selection(struct v4l2_subdev *sd,
 }
 
 static int imx296_set_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -812,8 +815,8 @@ static int imx296_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	imx296_set_selection(sd, state, &sel);
-	imx296_set_format(sd, state, &format);
+	imx296_set_selection(sd, NULL, state, &sel);
+	imx296_set_format(sd, NULL, state, &format);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/imx319.c b/drivers/media/i2c/imx319.c
index 953310ef3046..281d39ed4194 100644
--- a/drivers/media/i2c/imx319.c
+++ b/drivers/media/i2c/imx319.c
@@ -2030,6 +2030,7 @@ static int imx319_get_pad_format(struct v4l2_subdev *sd,
 
 static int
 imx319_set_pad_format(struct v4l2_subdev *sd,
+		      const struct v4l2_subdev_client_info *ci,
 		      struct v4l2_subdev_state *sd_state,
 		      struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/imx334.c b/drivers/media/i2c/imx334.c
index 553a16b84f4d..ef66d14b33b8 100644
--- a/drivers/media/i2c/imx334.c
+++ b/drivers/media/i2c/imx334.c
@@ -741,6 +741,7 @@ static int imx334_get_pad_format(struct v4l2_subdev *sd,
 }
 
 static int imx334_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -786,7 +787,7 @@ static int imx334_init_state(struct v4l2_subdev *sd,
 				 ~(imx334->link_freq_bitmap),
 				 __ffs(imx334->link_freq_bitmap));
 
-	return imx334_set_pad_format(sd, sd_state, &fmt);
+	return imx334_set_pad_format(sd, NULL, sd_state, &fmt);
 }
 
 static int imx334_set_framefmt(struct imx334 *imx334)
diff --git a/drivers/media/i2c/imx335.c b/drivers/media/i2c/imx335.c
index 1f777a1a8192..8f29d9f2da83 100644
--- a/drivers/media/i2c/imx335.c
+++ b/drivers/media/i2c/imx335.c
@@ -844,6 +844,7 @@ static void imx335_fill_pad_format(struct imx335 *imx335,
 }
 
 static int imx335_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -901,10 +902,11 @@ static int imx335_init_state(struct v4l2_subdev *sd,
 				 ~(imx335->link_freq_bitmap),
 				 __ffs(imx335->link_freq_bitmap));
 
-	return imx335_set_pad_format(sd, sd_state, &fmt);
+	return imx335_set_pad_format(sd, NULL, sd_state, &fmt);
 }
 
 static int imx335_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/imx355.c b/drivers/media/i2c/imx355.c
index 27a5c212a527..d5961c5ccc67 100644
--- a/drivers/media/i2c/imx355.c
+++ b/drivers/media/i2c/imx355.c
@@ -1345,6 +1345,7 @@ static int imx355_get_pad_format(struct v4l2_subdev *sd,
 
 static int
 imx355_set_pad_format(struct v4l2_subdev *sd,
+		      const struct v4l2_subdev_client_info *ci,
 		      struct v4l2_subdev_state *sd_state,
 		      struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/imx412.c b/drivers/media/i2c/imx412.c
index 2705af2f16c0..f269bbaddc96 100644
--- a/drivers/media/i2c/imx412.c
+++ b/drivers/media/i2c/imx412.c
@@ -696,6 +696,7 @@ static int imx412_get_pad_format(struct v4l2_subdev *sd,
 }
 
 static int imx412_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -733,7 +734,7 @@ static int imx412_init_state(struct v4l2_subdev *sd,
 	fmt.which = sd_state ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE;
 	imx412_fill_pad_format(imx412, &supported_mode, &fmt);
 
-	return imx412_set_pad_format(sd, sd_state, &fmt);
+	return imx412_set_pad_format(sd, NULL, sd_state, &fmt);
 }
 
 /**
diff --git a/drivers/media/i2c/imx415.c b/drivers/media/i2c/imx415.c
index 0b424c17e880..f70c36e45a25 100644
--- a/drivers/media/i2c/imx415.c
+++ b/drivers/media/i2c/imx415.c
@@ -1021,6 +1021,7 @@ static int imx415_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int imx415_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *fmt)
 {
@@ -1042,6 +1043,7 @@ static int imx415_set_format(struct v4l2_subdev *sd,
 }
 
 static int imx415_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -1070,7 +1072,7 @@ static int imx415_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	imx415_set_format(sd, state, &format);
+	imx415_set_format(sd, NULL, state, &format);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/isl7998x.c b/drivers/media/i2c/isl7998x.c
index a77538d2343c..cb2b752aaffe 100644
--- a/drivers/media/i2c/isl7998x.c
+++ b/drivers/media/i2c/isl7998x.c
@@ -1027,6 +1027,7 @@ static int isl7998x_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int isl7998x_set_fmt(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/lt6911uxe.c b/drivers/media/i2c/lt6911uxe.c
index bdefdd157e69..c9da174dbfa6 100644
--- a/drivers/media/i2c/lt6911uxe.c
+++ b/drivers/media/i2c/lt6911uxe.c
@@ -384,6 +384,7 @@ static int lt6911uxe_disable_streams(struct v4l2_subdev *sd,
 }
 
 static int lt6911uxe_set_format(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_format *fmt)
 {
@@ -439,7 +440,7 @@ static int lt6911uxe_init_state(struct v4l2_subdev *sd,
 		: V4L2_SUBDEV_FORMAT_ACTIVE,
 	};
 
-	return lt6911uxe_set_format(sd, sd_state, &fmt);
+	return lt6911uxe_set_format(sd, NULL, sd_state, &fmt);
 }
 
 static const struct v4l2_subdev_video_ops lt6911uxe_video_ops = {
@@ -562,7 +563,7 @@ static irqreturn_t lt6911uxe_threaded_irq_fn(int irq, void *dev_id)
 	 * As a HDMI to CSI2 bridge, it needs to update the format in time
 	 * when the HDMI source changes.
 	 */
-	lt6911uxe_set_format(sd, state, &fmt);
+	lt6911uxe_set_format(sd, NULL, state, &fmt);
 	v4l2_subdev_unlock_state(state);
 
 	return IRQ_HANDLED;
diff --git a/drivers/media/i2c/max9286.c b/drivers/media/i2c/max9286.c
index ac0712ce1e65..87c2b2a76d04 100644
--- a/drivers/media/i2c/max9286.c
+++ b/drivers/media/i2c/max9286.c
@@ -910,6 +910,7 @@ static int max9286_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int max9286_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *state,
 			   struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/max96714.c b/drivers/media/i2c/max96714.c
index e3e625e6f11a..0c2ff62a77ea 100644
--- a/drivers/media/i2c/max96714.c
+++ b/drivers/media/i2c/max96714.c
@@ -327,6 +327,7 @@ static int max96714_disable_streams(struct v4l2_subdev *sd,
 }
 
 static int max96714_set_fmt(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *state,
 			    struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/max96717.c b/drivers/media/i2c/max96717.c
index 72f021b1a7b9..ff1df1d84049 100644
--- a/drivers/media/i2c/max96717.c
+++ b/drivers/media/i2c/max96717.c
@@ -414,6 +414,7 @@ static int max96717_set_routing(struct v4l2_subdev *sd,
 }
 
 static int max96717_set_fmt(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *state,
 			    struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/ml86v7667.c b/drivers/media/i2c/ml86v7667.c
index 48b7d589df31..88542996da3b 100644
--- a/drivers/media/i2c/ml86v7667.c
+++ b/drivers/media/i2c/ml86v7667.c
@@ -298,7 +298,7 @@ static const struct v4l2_subdev_video_ops ml86v7667_subdev_video_ops = {
 static const struct v4l2_subdev_pad_ops ml86v7667_subdev_pad_ops = {
 	.enum_mbus_code = ml86v7667_enum_mbus_code,
 	.get_fmt = ml86v7667_fill_fmt,
-	.set_fmt = ml86v7667_fill_fmt,
+	.set_fmt = v4l2_subdev_get_fmt_ci,
 	.get_mbus_config = ml86v7667_get_mbus_config,
 };
 
diff --git a/drivers/media/i2c/mt9m001.c b/drivers/media/i2c/mt9m001.c
index 0ade967b357b..c7d2e05ed6ed 100644
--- a/drivers/media/i2c/mt9m001.c
+++ b/drivers/media/i2c/mt9m001.c
@@ -248,6 +248,7 @@ static int mt9m001_s_stream(struct v4l2_subdev *sd, int enable)
 }
 
 static int mt9m001_set_selection(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_selection *sel)
 {
@@ -289,6 +290,7 @@ static int mt9m001_set_selection(struct v4l2_subdev *sd,
 }
 
 static int mt9m001_get_selection(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_selection *sel)
 {
@@ -359,7 +361,7 @@ static int mt9m001_s_fmt(struct v4l2_subdev *sd,
 	int ret;
 
 	/* No support for scaling so far, just crop. TODO: use skipping */
-	ret = mt9m001_set_selection(sd, NULL, &sel);
+	ret = mt9m001_set_selection(sd, NULL, NULL, &sel);
 	if (!ret) {
 		mf->width	= mt9m001->rect.width;
 		mf->height	= mt9m001->rect.height;
@@ -371,6 +373,7 @@ static int mt9m001_s_fmt(struct v4l2_subdev *sd,
 }
 
 static int mt9m001_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/mt9m111.c b/drivers/media/i2c/mt9m111.c
index 4e748080b798..f911c1cb43e1 100644
--- a/drivers/media/i2c/mt9m111.c
+++ b/drivers/media/i2c/mt9m111.c
@@ -446,6 +446,7 @@ static int mt9m111_reset(struct mt9m111 *mt9m111)
 }
 
 static int mt9m111_set_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -490,6 +491,7 @@ static int mt9m111_set_selection(struct v4l2_subdev *sd,
 }
 
 static int mt9m111_get_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -617,6 +619,7 @@ static int mt9m111_set_pixfmt(struct mt9m111 *mt9m111,
 }
 
 static int mt9m111_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/mt9m114.c b/drivers/media/i2c/mt9m114.c
index e395e2d14e97..813345bfa195 100644
--- a/drivers/media/i2c/mt9m114.c
+++ b/drivers/media/i2c/mt9m114.c
@@ -1255,6 +1255,7 @@ static int mt9m114_pa_enum_framesizes(struct v4l2_subdev *sd,
 }
 
 static int mt9m114_pa_set_fmt(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *state,
 			      struct v4l2_subdev_format *fmt)
 {
@@ -1282,6 +1283,7 @@ static int mt9m114_pa_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int mt9m114_pa_get_selection(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *state,
 				    struct v4l2_subdev_selection *sel)
 {
@@ -1305,6 +1307,7 @@ static int mt9m114_pa_get_selection(struct v4l2_subdev *sd,
 }
 
 static int mt9m114_pa_set_selection(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *state,
 				    struct v4l2_subdev_selection *sel)
 {
@@ -1878,6 +1881,7 @@ static void mt9m114_ifp_update_sel_and_src_fmt(struct v4l2_subdev_state *state)
 }
 
 static int mt9m114_ifp_set_fmt(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *state,
 			       struct v4l2_subdev_format *fmt)
 {
@@ -1924,6 +1928,7 @@ static int mt9m114_ifp_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int mt9m114_ifp_get_selection(struct v4l2_subdev *sd,
+				     const struct v4l2_subdev_client_info *ci,
 				     struct v4l2_subdev_state *state,
 				     struct v4l2_subdev_selection *sel)
 {
@@ -1983,6 +1988,7 @@ static int mt9m114_ifp_get_selection(struct v4l2_subdev *sd,
 }
 
 static int mt9m114_ifp_set_selection(struct v4l2_subdev *sd,
+				     const struct v4l2_subdev_client_info *ci,
 				     struct v4l2_subdev_state *state,
 				     struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/mt9p031.c b/drivers/media/i2c/mt9p031.c
index 8dc57eeba606..ea006afd4aea 100644
--- a/drivers/media/i2c/mt9p031.c
+++ b/drivers/media/i2c/mt9p031.c
@@ -585,6 +585,7 @@ static int mt9p031_get_format(struct v4l2_subdev *subdev,
 }
 
 static int mt9p031_set_format(struct v4l2_subdev *subdev,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *format)
 {
@@ -623,6 +624,7 @@ static int mt9p031_set_format(struct v4l2_subdev *subdev,
 }
 
 static int mt9p031_get_selection(struct v4l2_subdev *subdev,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -647,6 +649,7 @@ static int mt9p031_get_selection(struct v4l2_subdev *subdev,
 }
 
 static int mt9p031_set_selection(struct v4l2_subdev *subdev,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/mt9t112.c b/drivers/media/i2c/mt9t112.c
index bd2268154ca7..f6452dd070fa 100644
--- a/drivers/media/i2c/mt9t112.c
+++ b/drivers/media/i2c/mt9t112.c
@@ -872,6 +872,7 @@ static int mt9t112_set_params(struct mt9t112_priv *priv,
 }
 
 static int mt9t112_get_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -897,6 +898,7 @@ static int mt9t112_get_selection(struct v4l2_subdev *sd,
 }
 
 static int mt9t112_set_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -953,6 +955,7 @@ static int mt9t112_s_fmt(struct v4l2_subdev *sd,
 }
 
 static int mt9t112_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/mt9v011.c b/drivers/media/i2c/mt9v011.c
index 985517f1cff7..c3483e268357 100644
--- a/drivers/media/i2c/mt9v011.c
+++ b/drivers/media/i2c/mt9v011.c
@@ -336,6 +336,7 @@ static int mt9v011_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int mt9v011_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/mt9v032.c b/drivers/media/i2c/mt9v032.c
index d4359d5b92bb..16c995540d07 100644
--- a/drivers/media/i2c/mt9v032.c
+++ b/drivers/media/i2c/mt9v032.c
@@ -503,6 +503,7 @@ static unsigned int mt9v032_calc_ratio(unsigned int input, unsigned int output)
 }
 
 static int mt9v032_set_format(struct v4l2_subdev *subdev,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *format)
 {
@@ -547,6 +548,7 @@ static int mt9v032_set_format(struct v4l2_subdev *subdev,
 }
 
 static int mt9v032_get_selection(struct v4l2_subdev *subdev,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -561,6 +563,7 @@ static int mt9v032_get_selection(struct v4l2_subdev *subdev,
 }
 
 static int mt9v032_set_selection(struct v4l2_subdev *subdev,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/mt9v111.c b/drivers/media/i2c/mt9v111.c
index 64a758c95ab7..6e9b3e190bca 100644
--- a/drivers/media/i2c/mt9v111.c
+++ b/drivers/media/i2c/mt9v111.c
@@ -886,6 +886,7 @@ static int mt9v111_get_format(struct v4l2_subdev *subdev,
 }
 
 static int mt9v111_set_format(struct v4l2_subdev *subdev,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/og01a1b.c b/drivers/media/i2c/og01a1b.c
index 1675f0460969..d18b10fb0a1f 100644
--- a/drivers/media/i2c/og01a1b.c
+++ b/drivers/media/i2c/og01a1b.c
@@ -677,6 +677,7 @@ static int og01a1b_disable_streams(struct v4l2_subdev *sd,
 }
 
 static int og01a1b_set_format(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
@@ -763,7 +764,7 @@ static int og01a1b_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	og01a1b_set_format(sd, state, &fmt);
+	og01a1b_set_format(sd, NULL, state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/og0ve1b.c b/drivers/media/i2c/og0ve1b.c
index 84a28cdcade1..84682389d989 100644
--- a/drivers/media/i2c/og0ve1b.c
+++ b/drivers/media/i2c/og0ve1b.c
@@ -481,6 +481,7 @@ static int og0ve1b_disable_streams(struct v4l2_subdev *sd,
 }
 
 static int og0ve1b_set_pad_format(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *state,
 				  struct v4l2_subdev_format *fmt)
 {
@@ -544,7 +545,7 @@ static int og0ve1b_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	og0ve1b_set_pad_format(sd, state, &fmt);
+	og0ve1b_set_pad_format(sd, NULL, state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/os05b10.c b/drivers/media/i2c/os05b10.c
index e0453c988e4a..a46cfa513e98 100644
--- a/drivers/media/i2c/os05b10.c
+++ b/drivers/media/i2c/os05b10.c
@@ -595,6 +595,7 @@ static int os05b10_set_framing_limits(struct os05b10 *os05b10,
 }
 
 static int os05b10_set_pad_format(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *sd_state,
 				  struct v4l2_subdev_format *fmt)
 {
@@ -624,6 +625,7 @@ static int os05b10_set_pad_format(struct v4l2_subdev *sd,
 }
 
 static int os05b10_get_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov01a10.c b/drivers/media/i2c/ov01a10.c
index 8a29e5b4b6ba..b9a7dacd603e 100644
--- a/drivers/media/i2c/ov01a10.c
+++ b/drivers/media/i2c/ov01a10.c
@@ -634,6 +634,7 @@ static void ov01a10_update_blank_ctrls(struct ov01a10 *ov01a10,
 }
 
 static int ov01a10_set_format(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
@@ -715,6 +716,7 @@ static int ov01a10_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int ov01a10_get_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -747,6 +749,7 @@ static int ov01a10_get_selection(struct v4l2_subdev *sd,
 }
 
 static int ov01a10_set_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov02a10.c b/drivers/media/i2c/ov02a10.c
index 143dcfe10445..2f5bcf323515 100644
--- a/drivers/media/i2c/ov02a10.c
+++ b/drivers/media/i2c/ov02a10.c
@@ -296,6 +296,7 @@ static void ov02a10_fill_fmt(const struct ov02a10_mode *mode,
 }
 
 static int ov02a10_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
@@ -523,7 +524,7 @@ static int ov02a10_init_state(struct v4l2_subdev *sd,
 		}
 	};
 
-	ov02a10_set_fmt(sd, sd_state, &fmt);
+	ov02a10_set_fmt(sd, NULL, sd_state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/ov02c10.c b/drivers/media/i2c/ov02c10.c
index cf93d36032e1..d622f5dcac60 100644
--- a/drivers/media/i2c/ov02c10.c
+++ b/drivers/media/i2c/ov02c10.c
@@ -701,6 +701,7 @@ static int ov02c10_power_on(struct device *dev)
 }
 
 static int ov02c10_set_format(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov02e10.c b/drivers/media/i2c/ov02e10.c
index 4a64cba99991..5c1e7013f559 100644
--- a/drivers/media/i2c/ov02e10.c
+++ b/drivers/media/i2c/ov02e10.c
@@ -590,6 +590,7 @@ static int ov02e10_power_on(struct device *dev)
 }
 
 static int ov02e10_set_format(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov08d10.c b/drivers/media/i2c/ov08d10.c
index 9adef5446a61..0710d2243cc0 100644
--- a/drivers/media/i2c/ov08d10.c
+++ b/drivers/media/i2c/ov08d10.c
@@ -1178,6 +1178,7 @@ static int ov08d10_set_stream(struct v4l2_subdev *sd, int enable)
 }
 
 static int ov08d10_set_format(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov08x40.c b/drivers/media/i2c/ov08x40.c
index 5eaf454f4763..6de0c17e633d 100644
--- a/drivers/media/i2c/ov08x40.c
+++ b/drivers/media/i2c/ov08x40.c
@@ -1844,6 +1844,7 @@ static int ov08x40_get_pad_format(struct v4l2_subdev *sd,
 
 static int
 ov08x40_set_pad_format(struct v4l2_subdev *sd,
+		       const struct v4l2_subdev_client_info *ci,
 		       struct v4l2_subdev_state *sd_state,
 		       struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov13858.c b/drivers/media/i2c/ov13858.c
index 09d58e8b1c7f..de2b79a9a0e3 100644
--- a/drivers/media/i2c/ov13858.c
+++ b/drivers/media/i2c/ov13858.c
@@ -1345,6 +1345,7 @@ static int ov13858_get_pad_format(struct v4l2_subdev *sd,
 
 static int
 ov13858_set_pad_format(struct v4l2_subdev *sd,
+		       const struct v4l2_subdev_client_info *ci,
 		       struct v4l2_subdev_state *sd_state,
 		       struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov13b10.c b/drivers/media/i2c/ov13b10.c
index b0d34141a13a..242a95834126 100644
--- a/drivers/media/i2c/ov13b10.c
+++ b/drivers/media/i2c/ov13b10.c
@@ -1118,6 +1118,7 @@ static int ov13b10_get_pad_format(struct v4l2_subdev *sd,
 
 static int
 ov13b10_set_pad_format(struct v4l2_subdev *sd,
+		       const struct v4l2_subdev_client_info *ci,
 		       struct v4l2_subdev_state *sd_state,
 		       struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov2640.c b/drivers/media/i2c/ov2640.c
index 50feb608b92b..3dbefac6d305 100644
--- a/drivers/media/i2c/ov2640.c
+++ b/drivers/media/i2c/ov2640.c
@@ -938,6 +938,7 @@ static int ov2640_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov2640_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
@@ -1028,6 +1029,7 @@ static int ov2640_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int ov2640_get_selection(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov2659.c b/drivers/media/i2c/ov2659.c
index 7d8c7c3465a4..5bc30e7b8325 100644
--- a/drivers/media/i2c/ov2659.c
+++ b/drivers/media/i2c/ov2659.c
@@ -1080,6 +1080,7 @@ static void __ov2659_try_frame_size(struct v4l2_mbus_framefmt *mf,
 }
 
 static int ov2659_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 78e63bd1b35b..6c13cf7fa409 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -640,6 +640,7 @@ static int ov2680_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov2680_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
@@ -704,6 +705,7 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov2680_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -734,6 +736,7 @@ static int ov2680_get_selection(struct v4l2_subdev *sd,
 }
 
 static int ov2680_set_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov2685.c b/drivers/media/i2c/ov2685.c
index 4911a4eea126..162859a140eb 100644
--- a/drivers/media/i2c/ov2685.c
+++ b/drivers/media/i2c/ov2685.c
@@ -340,6 +340,7 @@ static void ov2685_fill_fmt(const struct ov2685_mode *mode,
 }
 
 static int ov2685_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
@@ -413,6 +414,7 @@ __ov2685_get_pad_crop(struct ov2685 *ov2685,
 }
 
 static int ov2685_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov2732.c b/drivers/media/i2c/ov2732.c
index 40035320fec6..57dd0dd3fd85 100644
--- a/drivers/media/i2c/ov2732.c
+++ b/drivers/media/i2c/ov2732.c
@@ -281,6 +281,7 @@ static int ov2732_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int ov2732_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *state,
 			  struct v4l2_subdev_format *fmt)
 {
@@ -317,6 +318,7 @@ static int ov2732_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov2732_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -442,7 +444,7 @@ static int ov2732_init_state(struct v4l2_subdev *sd,
 		}
 	};
 
-	return ov2732_set_fmt(sd, sd_state, &fmt);
+	return ov2732_set_fmt(sd, NULL, sd_state, &fmt);
 }
 
 static const struct v4l2_subdev_internal_ops ov2732_internal_ops = {
diff --git a/drivers/media/i2c/ov2735.c b/drivers/media/i2c/ov2735.c
index dcb1add1fd9f..90d8be72b559 100644
--- a/drivers/media/i2c/ov2735.c
+++ b/drivers/media/i2c/ov2735.c
@@ -674,6 +674,7 @@ static int ov2735_disable_streams(struct v4l2_subdev *sd,
 }
 
 static int ov2735_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -742,6 +743,7 @@ static int ov2735_set_framing_limits(struct ov2735 *ov2735,
 }
 
 static int ov2735_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -792,7 +794,7 @@ static int ov2735_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	ov2735_set_pad_format(sd, state, &fmt);
+	ov2735_set_pad_format(sd, NULL, state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/ov2740.c b/drivers/media/i2c/ov2740.c
index fb590dfadda1..4023b5b7ca82 100644
--- a/drivers/media/i2c/ov2740.c
+++ b/drivers/media/i2c/ov2740.c
@@ -1023,6 +1023,7 @@ static int ov2740_set_stream(struct v4l2_subdev *sd, int enable)
 }
 
 static int ov2740_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov4689.c b/drivers/media/i2c/ov4689.c
index a59d25b09b5b..466b6f74d982 100644
--- a/drivers/media/i2c/ov4689.c
+++ b/drivers/media/i2c/ov4689.c
@@ -330,6 +330,7 @@ static void ov4689_fill_fmt(const struct ov4689_mode *mode,
 }
 
 static int ov4689_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
@@ -385,6 +386,7 @@ static int ov4689_enable_test_pattern(struct ov4689 *ov4689, u32 pattern)
 }
 
 static int ov4689_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 92d2d6cd4ba4..5a57c0b292ad 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -2948,6 +2948,7 @@ static int ov5640_update_pixel_rate(struct ov5640_dev *sensor)
 }
 
 static int ov5640_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
@@ -2994,6 +2995,7 @@ static int ov5640_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov5640_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov5645.c b/drivers/media/i2c/ov5645.c
index c772ef6e51d2..a8d740ed4dc9 100644
--- a/drivers/media/i2c/ov5645.c
+++ b/drivers/media/i2c/ov5645.c
@@ -848,6 +848,7 @@ static int ov5645_enum_frame_size(struct v4l2_subdev *subdev,
 }
 
 static int ov5645_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_format *format)
 {
@@ -905,12 +906,13 @@ static int ov5645_init_state(struct v4l2_subdev *subdev,
 		},
 	};
 
-	ov5645_set_format(subdev, sd_state, &fmt);
+	ov5645_set_format(subdev, NULL, sd_state, &fmt);
 
 	return 0;
 }
 
 static int ov5645_get_selection(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov5647.c b/drivers/media/i2c/ov5647.c
index 3facf92b3841..4824013454f5 100644
--- a/drivers/media/i2c/ov5647.c
+++ b/drivers/media/i2c/ov5647.c
@@ -787,6 +787,7 @@ static int ov5647_get_pad_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov5647_set_pad_fmt(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *format)
 {
@@ -837,6 +838,7 @@ static int ov5647_set_pad_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov5647_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov5648.c b/drivers/media/i2c/ov5648.c
index f0b839cd65f1..d06f5ebd9464 100644
--- a/drivers/media/i2c/ov5648.c
+++ b/drivers/media/i2c/ov5648.c
@@ -2215,6 +2215,7 @@ static int ov5648_get_fmt(struct v4l2_subdev *subdev,
 }
 
 static int ov5648_set_fmt(struct v4l2_subdev *subdev,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/ov5670.c b/drivers/media/i2c/ov5670.c
index 04b3183b7bcb..c28d19f7bc2f 100644
--- a/drivers/media/i2c/ov5670.c
+++ b/drivers/media/i2c/ov5670.c
@@ -2286,6 +2286,7 @@ static int ov5670_get_pad_format(struct v4l2_subdev *sd,
 }
 
 static int ov5670_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -2554,6 +2555,7 @@ __ov5670_get_pad_crop(struct ov5670 *sensor, struct v4l2_subdev_state *state,
 }
 
 static int ov5670_get_selection(struct v4l2_subdev *subdev,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov5675.c b/drivers/media/i2c/ov5675.c
index 508149485248..883a3df0cc42 100644
--- a/drivers/media/i2c/ov5675.c
+++ b/drivers/media/i2c/ov5675.c
@@ -1016,6 +1016,7 @@ static int ov5675_power_on(struct device *dev)
 }
 
 static int ov5675_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_format *fmt)
 {
@@ -1075,6 +1076,7 @@ static int ov5675_get_format(struct v4l2_subdev *sd,
 }
 
 static int ov5675_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov5693.c b/drivers/media/i2c/ov5693.c
index 4cc796bbee92..af44cb03beaa 100644
--- a/drivers/media/i2c/ov5693.c
+++ b/drivers/media/i2c/ov5693.c
@@ -807,6 +807,7 @@ static int ov5693_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov5693_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *state,
 			  struct v4l2_subdev_format *format)
 {
@@ -882,6 +883,7 @@ static int ov5693_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov5693_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -915,6 +917,7 @@ static int ov5693_get_selection(struct v4l2_subdev *sd,
 }
 
 static int ov5693_set_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov5695.c b/drivers/media/i2c/ov5695.c
index 5bb6ce7b3237..5b23f04ab9f8 100644
--- a/drivers/media/i2c/ov5695.c
+++ b/drivers/media/i2c/ov5695.c
@@ -805,6 +805,7 @@ ov5695_find_best_fit(struct v4l2_subdev_format *fmt)
 }
 
 static int ov5695_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov6211.c b/drivers/media/i2c/ov6211.c
index 034d5d57d67e..2ad31b2c5249 100644
--- a/drivers/media/i2c/ov6211.c
+++ b/drivers/media/i2c/ov6211.c
@@ -459,6 +459,7 @@ static int ov6211_disable_streams(struct v4l2_subdev *sd,
 }
 
 static int ov6211_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -522,7 +523,7 @@ static int ov6211_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	ov6211_set_pad_format(sd, state, &fmt);
+	ov6211_set_pad_format(sd, NULL, state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/ov64a40.c b/drivers/media/i2c/ov64a40.c
index 78b62c169b99..7beddf480b8a 100644
--- a/drivers/media/i2c/ov64a40.c
+++ b/drivers/media/i2c/ov64a40.c
@@ -3107,6 +3107,7 @@ static int ov64a40_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int ov64a40_get_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -3138,6 +3139,7 @@ static int ov64a40_get_selection(struct v4l2_subdev *sd,
 }
 
 static int ov64a40_set_format(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *state,
 			      struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov7251.c b/drivers/media/i2c/ov7251.c
index 27afc3fc0175..bffe64f1e458 100644
--- a/drivers/media/i2c/ov7251.c
+++ b/drivers/media/i2c/ov7251.c
@@ -1214,6 +1214,7 @@ ov7251_find_mode_by_ival(struct ov7251 *ov7251, struct v4l2_fract *timeperframe)
 }
 
 static int ov7251_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_format *format)
 {
@@ -1296,12 +1297,13 @@ static int ov7251_init_state(struct v4l2_subdev *subdev,
 		}
 	};
 
-	ov7251_set_format(subdev, sd_state, &fmt);
+	ov7251_set_format(subdev, NULL, sd_state, &fmt);
 
 	return 0;
 }
 
 static int ov7251_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov7670.c b/drivers/media/i2c/ov7670.c
index b6d238ba0d53..a3e5e315dcde 100644
--- a/drivers/media/i2c/ov7670.c
+++ b/drivers/media/i2c/ov7670.c
@@ -1098,6 +1098,7 @@ static int ov7670_apply_fmt(struct v4l2_subdev *sd)
  * Set a format.
  */
 static int ov7670_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/ov772x.c b/drivers/media/i2c/ov772x.c
index be3ba284ee0b..76aa562213a1 100644
--- a/drivers/media/i2c/ov772x.c
+++ b/drivers/media/i2c/ov772x.c
@@ -1171,6 +1171,7 @@ static int ov772x_set_params(struct ov772x_priv *priv,
 }
 
 static int ov772x_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -1212,6 +1213,7 @@ static int ov772x_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov772x_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/ov7740.c b/drivers/media/i2c/ov7740.c
index c2e02f191816..059435c7b341 100644
--- a/drivers/media/i2c/ov7740.c
+++ b/drivers/media/i2c/ov7740.c
@@ -766,6 +766,7 @@ static int ov7740_try_fmt_internal(struct v4l2_subdev *sd,
 }
 
 static int ov7740_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/ov8856.c b/drivers/media/i2c/ov8856.c
index 8bedb47cd7cf..741fd60a93c2 100644
--- a/drivers/media/i2c/ov8856.c
+++ b/drivers/media/i2c/ov8856.c
@@ -2132,6 +2132,7 @@ static int ov8856_power_off(struct device *dev)
 }
 
 static int ov8856_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov8858.c b/drivers/media/i2c/ov8858.c
index 3f45f7fab833..0bfc4350a8c9 100644
--- a/drivers/media/i2c/ov8858.c
+++ b/drivers/media/i2c/ov8858.c
@@ -1409,6 +1409,7 @@ static const struct v4l2_subdev_video_ops ov8858_video_ops = {
  */
 
 static int ov8858_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *state,
 			  struct v4l2_subdev_format *fmt)
 {
@@ -1486,7 +1487,7 @@ static int ov8858_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	ov8858_set_fmt(sd, sd_state, &fmt);
+	ov8858_set_fmt(sd, NULL, sd_state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/ov8865.c b/drivers/media/i2c/ov8865.c
index a8586df14f77..7d0d2353cdd5 100644
--- a/drivers/media/i2c/ov8865.c
+++ b/drivers/media/i2c/ov8865.c
@@ -2689,6 +2689,7 @@ static int ov8865_get_fmt(struct v4l2_subdev *subdev,
 }
 
 static int ov8865_set_fmt(struct v4l2_subdev *subdev,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
@@ -2797,6 +2798,7 @@ __ov8865_get_pad_crop(struct ov8865_sensor *sensor,
 }
 
 static int ov8865_get_selection(struct v4l2_subdev *subdev,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov9282.c b/drivers/media/i2c/ov9282.c
index 5b6f897a74fc..4b4aee8d4fb6 100644
--- a/drivers/media/i2c/ov9282.c
+++ b/drivers/media/i2c/ov9282.c
@@ -780,6 +780,7 @@ static int ov9282_get_pad_format(struct v4l2_subdev *sd,
 }
 
 static int ov9282_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -826,7 +827,7 @@ static int ov9282_init_state(struct v4l2_subdev *sd,
 	ov9282_fill_pad_format(ov9282, &supported_modes[DEFAULT_MODE],
 			       ov9282->code, &fmt);
 
-	return ov9282_set_pad_format(sd, sd_state, &fmt);
+	return ov9282_set_pad_format(sd, NULL, sd_state, &fmt);
 }
 
 static const struct v4l2_rect *
@@ -845,6 +846,7 @@ __ov9282_get_pad_crop(struct ov9282 *ov9282,
 }
 
 static int ov9282_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov9640.c b/drivers/media/i2c/ov9640.c
index 122f411044ce..d24b3be1e0b6 100644
--- a/drivers/media/i2c/ov9640.c
+++ b/drivers/media/i2c/ov9640.c
@@ -519,6 +519,7 @@ static int ov9640_s_fmt(struct v4l2_subdev *sd,
 }
 
 static int ov9640_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
@@ -563,6 +564,7 @@ static int ov9640_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int ov9640_get_selection(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/ov9650.c b/drivers/media/i2c/ov9650.c
index 5c85db8a4a38..b596cfb2d7f3 100644
--- a/drivers/media/i2c/ov9650.c
+++ b/drivers/media/i2c/ov9650.c
@@ -1226,6 +1226,7 @@ static void __ov965x_try_frame_size(struct v4l2_mbus_framefmt *mf,
 }
 
 static int ov965x_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/ov9734.c b/drivers/media/i2c/ov9734.c
index 0eaf33807fc9..8f39609ad1b6 100644
--- a/drivers/media/i2c/ov9734.c
+++ b/drivers/media/i2c/ov9734.c
@@ -681,6 +681,7 @@ static int ov9734_set_stream(struct v4l2_subdev *sd, int enable)
 }
 
 static int ov9734_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/rdacm20.c b/drivers/media/i2c/rdacm20.c
index 52e8e2620b4d..d7f899c71d85 100644
--- a/drivers/media/i2c/rdacm20.c
+++ b/drivers/media/i2c/rdacm20.c
@@ -442,7 +442,7 @@ static const struct v4l2_subdev_video_ops rdacm20_video_ops = {
 static const struct v4l2_subdev_pad_ops rdacm20_subdev_pad_ops = {
 	.enum_mbus_code = rdacm20_enum_mbus_code,
 	.get_fmt	= rdacm20_get_fmt,
-	.set_fmt	= rdacm20_get_fmt,
+	.set_fmt	= v4l2_subdev_get_fmt_ci,
 };
 
 static const struct v4l2_subdev_ops rdacm20_subdev_ops = {
diff --git a/drivers/media/i2c/rdacm21.c b/drivers/media/i2c/rdacm21.c
index bcab462708c7..afcb7e43a694 100644
--- a/drivers/media/i2c/rdacm21.c
+++ b/drivers/media/i2c/rdacm21.c
@@ -322,7 +322,7 @@ static const struct v4l2_subdev_video_ops rdacm21_video_ops = {
 static const struct v4l2_subdev_pad_ops rdacm21_subdev_pad_ops = {
 	.enum_mbus_code = rdacm21_enum_mbus_code,
 	.get_fmt	= rdacm21_get_fmt,
-	.set_fmt	= rdacm21_get_fmt,
+	.set_fmt	= v4l2_subdev_get_fmt_ci,
 };
 
 static const struct v4l2_subdev_ops rdacm21_subdev_ops = {
diff --git a/drivers/media/i2c/rj54n1cb0c.c b/drivers/media/i2c/rj54n1cb0c.c
index 23352d71a108..211f890a9cbd 100644
--- a/drivers/media/i2c/rj54n1cb0c.c
+++ b/drivers/media/i2c/rj54n1cb0c.c
@@ -541,6 +541,7 @@ static int rj54n1_sensor_scale(struct v4l2_subdev *sd, s32 *in_w, s32 *in_h,
 			       s32 *out_w, s32 *out_h);
 
 static int rj54n1_set_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -578,6 +579,7 @@ static int rj54n1_set_selection(struct v4l2_subdev *sd,
 }
 
 static int rj54n1_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -973,6 +975,7 @@ static int rj54n1_reg_init(struct i2c_client *client)
 }
 
 static int rj54n1_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/s5c73m3/s5c73m3-core.c b/drivers/media/i2c/s5c73m3/s5c73m3-core.c
index 551387cea521..680add1da244 100644
--- a/drivers/media/i2c/s5c73m3/s5c73m3-core.c
+++ b/drivers/media/i2c/s5c73m3/s5c73m3-core.c
@@ -1069,6 +1069,7 @@ static int s5c73m3_oif_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int s5c73m3_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
@@ -1108,6 +1109,7 @@ static int s5c73m3_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int s5c73m3_oif_set_fmt(struct v4l2_subdev *sd,
+			 const struct v4l2_subdev_client_info *ci,
 			 struct v4l2_subdev_state *sd_state,
 			 struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/s5k3m5.c b/drivers/media/i2c/s5k3m5.c
index c591b580d2e7..bdf5bea78685 100644
--- a/drivers/media/i2c/s5k3m5.c
+++ b/drivers/media/i2c/s5k3m5.c
@@ -958,6 +958,7 @@ static void s5k3m5_update_pad_format(struct s5k3m5 *s5k3m5,
 }
 
 static int s5k3m5_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -1038,6 +1039,7 @@ static int s5k3m5_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int s5k3m5_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -1075,7 +1077,7 @@ static int s5k3m5_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	s5k3m5_set_pad_format(sd, state, &fmt);
+	s5k3m5_set_pad_format(sd, NULL, state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/s5k5baf.c b/drivers/media/i2c/s5k5baf.c
index 378d273055ee..b13c4754ec3c 100644
--- a/drivers/media/i2c/s5k5baf.c
+++ b/drivers/media/i2c/s5k5baf.c
@@ -1307,6 +1307,7 @@ static int s5k5baf_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int s5k5baf_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
@@ -1370,6 +1371,7 @@ static int s5k5baf_is_bound_target(u32 target)
 }
 
 static int s5k5baf_get_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -1463,6 +1465,7 @@ static bool s5k5baf_cmp_rect(const struct v4l2_rect *r1,
 }
 
 static int s5k5baf_set_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/s5k6a3.c b/drivers/media/i2c/s5k6a3.c
index ba6477e88da3..071b31728a00 100644
--- a/drivers/media/i2c/s5k6a3.c
+++ b/drivers/media/i2c/s5k6a3.c
@@ -131,6 +131,7 @@ static struct v4l2_mbus_framefmt *__s5k6a3_get_format(
 }
 
 static int s5k6a3_set_fmt(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *sd_state,
 				  struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/s5kjn1.c b/drivers/media/i2c/s5kjn1.c
index a707cb740556..80d713ba3a1e 100644
--- a/drivers/media/i2c/s5kjn1.c
+++ b/drivers/media/i2c/s5kjn1.c
@@ -985,6 +985,7 @@ static void s5kjn1_update_pad_format(struct s5kjn1 *s5kjn1,
 }
 
 static int s5kjn1_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -1065,6 +1066,7 @@ static int s5kjn1_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int s5kjn1_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -1102,7 +1104,7 @@ static int s5kjn1_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	s5kjn1_set_pad_format(sd, state, &fmt);
+	s5kjn1_set_pad_format(sd, NULL, state, &fmt);
 
 	return 0;
 }
diff --git a/drivers/media/i2c/saa6752hs.c b/drivers/media/i2c/saa6752hs.c
index c6bf0b0902e8..6e2a3e4a63f7 100644
--- a/drivers/media/i2c/saa6752hs.c
+++ b/drivers/media/i2c/saa6752hs.c
@@ -563,6 +563,7 @@ static int saa6752hs_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int saa6752hs_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/saa7115.c b/drivers/media/i2c/saa7115.c
index 7cce90750c93..821684e909b2 100644
--- a/drivers/media/i2c/saa7115.c
+++ b/drivers/media/i2c/saa7115.c
@@ -1159,6 +1159,7 @@ static int saa711x_s_sliced_fmt(struct v4l2_subdev *sd, struct v4l2_sliced_vbi_f
 }
 
 static int saa711x_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/saa717x.c b/drivers/media/i2c/saa717x.c
index 0536ceb54650..367539667948 100644
--- a/drivers/media/i2c/saa717x.c
+++ b/drivers/media/i2c/saa717x.c
@@ -980,6 +980,7 @@ static int saa717x_s_register(struct v4l2_subdev *sd, const struct v4l2_dbg_regi
 #endif
 
 static int saa717x_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/st-mipid02.c b/drivers/media/i2c/st-mipid02.c
index 4675181af5fb..f9eabd9632d3 100644
--- a/drivers/media/i2c/st-mipid02.c
+++ b/drivers/media/i2c/st-mipid02.c
@@ -597,6 +597,7 @@ static int mipid02_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int mipid02_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/t4ka3.c b/drivers/media/i2c/t4ka3.c
index 746548868bb0..3c07319015ec 100644
--- a/drivers/media/i2c/t4ka3.c
+++ b/drivers/media/i2c/t4ka3.c
@@ -364,6 +364,7 @@ static void t4ka3_get_vblank_limits(struct t4ka3_data *sensor,
 }
 
 static int t4ka3_set_pad_format(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_format *format)
 {
@@ -627,6 +628,7 @@ static int t4ka3_disable_stream(struct v4l2_subdev *sd,
 }
 
 static int t4ka3_get_selection(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *state,
 			       struct v4l2_subdev_selection *sel)
 {
@@ -652,6 +654,7 @@ static int t4ka3_get_selection(struct v4l2_subdev *sd,
 }
 
 static int t4ka3_set_selection(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *state,
 			       struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/i2c/tc358743.c b/drivers/media/i2c/tc358743.c
index fbd38bbfee03..c79ce2f7a6b9 100644
--- a/drivers/media/i2c/tc358743.c
+++ b/drivers/media/i2c/tc358743.c
@@ -1816,6 +1816,7 @@ static int tc358743_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int tc358743_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/tc358746.c b/drivers/media/i2c/tc358746.c
index 86d9ba3ea4e5..d55539c35a26 100644
--- a/drivers/media/i2c/tc358746.c
+++ b/drivers/media/i2c/tc358746.c
@@ -873,6 +873,7 @@ static int tc358746_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int tc358746_set_fmt(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/tda1997x.c b/drivers/media/i2c/tda1997x.c
index afa1d6f34c9c..bbc8e3ff5175 100644
--- a/drivers/media/i2c/tda1997x.c
+++ b/drivers/media/i2c/tda1997x.c
@@ -1798,6 +1798,7 @@ static int tda1997x_get_format(struct v4l2_subdev *sd,
 }
 
 static int tda1997x_set_format(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/thp7312.c b/drivers/media/i2c/thp7312.c
index 775cfba188d8..f1c7f149c06c 100644
--- a/drivers/media/i2c/thp7312.c
+++ b/drivers/media/i2c/thp7312.c
@@ -729,6 +729,7 @@ static int thp7312_enum_frame_interval(struct v4l2_subdev *sd,
 }
 
 static int thp7312_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/tvp514x.c b/drivers/media/i2c/tvp514x.c
index 376eecb0b673..ad3908560349 100644
--- a/drivers/media/i2c/tvp514x.c
+++ b/drivers/media/i2c/tvp514x.c
@@ -880,6 +880,7 @@ static int tvp514x_get_pad_format(struct v4l2_subdev *sd,
 }
 
 static int tvp514x_set_pad_format(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *sd_state,
 				  struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/tvp5150.c b/drivers/media/i2c/tvp5150.c
index 9c204f38935d..d9237a8357e3 100644
--- a/drivers/media/i2c/tvp5150.c
+++ b/drivers/media/i2c/tvp5150.c
@@ -1104,6 +1104,7 @@ static void tvp5150_set_hw_selection(struct v4l2_subdev *sd,
 }
 
 static int tvp5150_set_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -1156,6 +1157,7 @@ static int tvp5150_set_selection(struct v4l2_subdev *sd,
 }
 
 static int tvp5150_get_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -1724,7 +1726,7 @@ static const struct v4l2_subdev_vbi_ops tvp5150_vbi_ops = {
 static const struct v4l2_subdev_pad_ops tvp5150_pad_ops = {
 	.enum_mbus_code = tvp5150_enum_mbus_code,
 	.enum_frame_size = tvp5150_enum_frame_size,
-	.set_fmt = tvp5150_fill_fmt,
+	.set_fmt = v4l2_subdev_get_fmt_ci,
 	.get_fmt = tvp5150_fill_fmt,
 	.get_selection = tvp5150_get_selection,
 	.set_selection = tvp5150_set_selection,
diff --git a/drivers/media/i2c/tvp7002.c b/drivers/media/i2c/tvp7002.c
index 3979ccde5a95..ada28cb8af7c 100644
--- a/drivers/media/i2c/tvp7002.c
+++ b/drivers/media/i2c/tvp7002.c
@@ -854,6 +854,7 @@ tvp7002_get_pad_format(struct v4l2_subdev *sd,
  */
 static int
 tvp7002_set_pad_format(struct v4l2_subdev *sd,
+		       const struct v4l2_subdev_client_info *ci,
 		       struct v4l2_subdev_state *sd_state,
 		       struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/tw9900.c b/drivers/media/i2c/tw9900.c
index 617fcf7f0b45..60c102aba750 100644
--- a/drivers/media/i2c/tw9900.c
+++ b/drivers/media/i2c/tw9900.c
@@ -198,6 +198,7 @@ static int tw9900_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int tw9900_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/i2c/tw9910.c b/drivers/media/i2c/tw9910.c
index 872207e688bf..731c97a40c82 100644
--- a/drivers/media/i2c/tw9910.c
+++ b/drivers/media/i2c/tw9910.c
@@ -715,6 +715,7 @@ static int tw9910_set_frame(struct v4l2_subdev *sd, u32 *width, u32 *height)
 }
 
 static int tw9910_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -792,6 +793,7 @@ static int tw9910_s_fmt(struct v4l2_subdev *sd,
 }
 
 static int tw9910_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/i2c/vd55g1.c b/drivers/media/i2c/vd55g1.c
index 78d18c028154..513d2262b401 100644
--- a/drivers/media/i2c/vd55g1.c
+++ b/drivers/media/i2c/vd55g1.c
@@ -1217,6 +1217,7 @@ static int vd55g1_patch(struct vd55g1 *sensor)
 }
 
 static int vd55g1_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -1300,6 +1301,7 @@ static int vd55g1_new_format_change_controls(struct vd55g1 *sensor,
 }
 
 static int vd55g1_set_pad_fmt(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *sd_fmt)
 {
@@ -1365,7 +1367,7 @@ static int vd55g1_init_state(struct v4l2_subdev *sd,
 			      vd55g1_get_fmt_code(sensor, VD55G1_MBUS_CODE_DEF),
 			      &fmt.format);
 
-	return vd55g1_set_pad_fmt(sd, sd_state, &fmt);
+	return vd55g1_set_pad_fmt(sd, NULL, sd_state, &fmt);
 }
 
 static int vd55g1_enum_frame_size(struct v4l2_subdev *sd,
diff --git a/drivers/media/i2c/vd56g3.c b/drivers/media/i2c/vd56g3.c
index 157acea9e286..ca0dd1b24072 100644
--- a/drivers/media/i2c/vd56g3.c
+++ b/drivers/media/i2c/vd56g3.c
@@ -823,6 +823,7 @@ static void vd56g3_update_img_pad_format(struct vd56g3 *sensor,
 }
 
 static int vd56g3_set_pad_fmt(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *sd_fmt)
 {
@@ -858,6 +859,7 @@ static int vd56g3_set_pad_fmt(struct v4l2_subdev *sd,
 }
 
 static int vd56g3_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -1038,7 +1040,7 @@ static int vd56g3_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	return vd56g3_set_pad_fmt(sd, sd_state, &fmt);
+	return vd56g3_set_pad_fmt(sd, NULL, sd_state, &fmt);
 }
 
 static const struct v4l2_subdev_video_ops vd56g3_video_ops = {
diff --git a/drivers/media/i2c/vgxy61.c b/drivers/media/i2c/vgxy61.c
index 3fb2166c81ef..e7819691723d 100644
--- a/drivers/media/i2c/vgxy61.c
+++ b/drivers/media/i2c/vgxy61.c
@@ -652,6 +652,7 @@ static int vgxy61_try_fmt_internal(struct v4l2_subdev *sd,
 }
 
 static int vgxy61_get_selection(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_selection *sel)
 {
@@ -1197,6 +1198,7 @@ static int vgxy61_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
 }
 
 static int vgxy61_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
@@ -1260,7 +1262,7 @@ static int vgxy61_init_state(struct v4l2_subdev *sd,
 	vgxy61_fill_framefmt(sensor, sensor->current_mode, &fmt.format,
 			     VGXY61_MEDIA_BUS_FMT_DEF);
 
-	return vgxy61_set_fmt(sd, sd_state, &fmt);
+	return vgxy61_set_fmt(sd, NULL, sd_state, &fmt);
 }
 
 static int vgxy61_s_ctrl(struct v4l2_ctrl *ctrl)
diff --git a/drivers/media/pci/cobalt/cobalt-driver.c b/drivers/media/pci/cobalt/cobalt-driver.c
index 9b9f69ff4016..2053849c9b4c 100644
--- a/drivers/media/pci/cobalt/cobalt-driver.c
+++ b/drivers/media/pci/cobalt/cobalt-driver.c
@@ -525,8 +525,8 @@ static int cobalt_subdevs_init(struct cobalt *cobalt)
 				&cobalt_edid);
 		if (err)
 			return err;
-		err = v4l2_subdev_call(s[i].sd, pad, set_fmt, NULL,
-				&sd_fmt);
+		err = v4l2_subdev_call(s[i].sd, pad, set_fmt, NULL, NULL,
+				       &sd_fmt);
 		if (err)
 			return err;
 		/* Reset channel video module */
@@ -609,8 +609,8 @@ static int cobalt_subdevs_hsma_init(struct cobalt *cobalt)
 
 		if (err)
 			return err;
-		err = v4l2_subdev_call(s->sd, pad, set_fmt, NULL,
-				&sd_fmt);
+		err = v4l2_subdev_call(s->sd, pad, set_fmt, NULL, NULL,
+				       &sd_fmt);
 		if (err)
 			return err;
 		cobalt->have_hsma_rx = true;
diff --git a/drivers/media/pci/cobalt/cobalt-v4l2.c b/drivers/media/pci/cobalt/cobalt-v4l2.c
index 51fd9576c6c2..ad89e62a2747 100644
--- a/drivers/media/pci/cobalt/cobalt-v4l2.c
+++ b/drivers/media/pci/cobalt/cobalt-v4l2.c
@@ -172,7 +172,7 @@ static void cobalt_enable_output(struct cobalt_stream *s)
 		sd_fmt.format.code = MEDIA_BUS_FMT_RGB888_1X24;
 		break;
 	}
-	v4l2_subdev_call(s->sd, pad, set_fmt, NULL, &sd_fmt);
+	v4l2_subdev_call(s->sd, pad, set_fmt, NULL, NULL, &sd_fmt);
 
 	iowrite32(0, &vo->control);
 	/* 1080p60 */
@@ -223,14 +223,14 @@ static void cobalt_enable_input(struct cobalt_stream *s)
 		iowrite32(M00235_CONTROL_BITMAP_ENABLE_MSK |
 			  (1 << M00235_CONTROL_BITMAP_PACK_FORMAT_OFST),
 			  &packer->control);
-		v4l2_subdev_call(s->sd, pad, set_fmt, NULL,
+		v4l2_subdev_call(s->sd, pad, set_fmt, NULL, NULL,
 				 &sd_fmt_yuyv);
 		break;
 	case V4L2_PIX_FMT_RGB24:
 		iowrite32(M00235_CONTROL_BITMAP_ENABLE_MSK |
 			  (2 << M00235_CONTROL_BITMAP_PACK_FORMAT_OFST),
 			  &packer->control);
-		v4l2_subdev_call(s->sd, pad, set_fmt, NULL,
+		v4l2_subdev_call(s->sd, pad, set_fmt, NULL, NULL,
 				 &sd_fmt_rgb);
 		break;
 	case V4L2_PIX_FMT_BGR32:
@@ -238,7 +238,7 @@ static void cobalt_enable_input(struct cobalt_stream *s)
 			  M00235_CONTROL_BITMAP_ENDIAN_FORMAT_MSK |
 			  (3 << M00235_CONTROL_BITMAP_PACK_FORMAT_OFST),
 			  &packer->control);
-		v4l2_subdev_call(s->sd, pad, set_fmt, NULL,
+		v4l2_subdev_call(s->sd, pad, set_fmt, NULL, NULL,
 				 &sd_fmt_rgb);
 		break;
 	}
@@ -938,7 +938,7 @@ static int cobalt_s_fmt_vid_out(struct file *file, void *priv,
 	s->ycbcr_enc = pix->ycbcr_enc;
 	s->quantization = pix->quantization;
 	v4l2_fill_mbus_format(&sd_fmt.format, pix, code);
-	v4l2_subdev_call(s->sd, pad, set_fmt, NULL, &sd_fmt);
+	v4l2_subdev_call(s->sd, pad, set_fmt, NULL, NULL, &sd_fmt);
 	return 0;
 }
 
diff --git a/drivers/media/pci/cx18/cx18-av-core.c b/drivers/media/pci/cx18/cx18-av-core.c
index 4fb19d26ee29..f5961853eeae 100644
--- a/drivers/media/pci/cx18/cx18-av-core.c
+++ b/drivers/media/pci/cx18/cx18-av-core.c
@@ -930,6 +930,7 @@ static int cx18_av_s_ctrl(struct v4l2_ctrl *ctrl)
 }
 
 static int cx18_av_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/pci/cx18/cx18-controls.c b/drivers/media/pci/cx18/cx18-controls.c
index 78eadad8b6e8..41f388ba22ab 100644
--- a/drivers/media/pci/cx18/cx18-controls.c
+++ b/drivers/media/pci/cx18/cx18-controls.c
@@ -85,7 +85,7 @@ static int cx18_s_video_encoding(struct cx2341x_handler *cxhdl, u32 val)
 	fmt->width = cxhdl->width / (is_mpeg1 ? 2 : 1);
 	fmt->height = cxhdl->height;
 	fmt->code = MEDIA_BUS_FMT_FIXED;
-	v4l2_subdev_call(cx->sd_av, pad, set_fmt, NULL, &format);
+	v4l2_subdev_call(cx->sd_av, pad, set_fmt, NULL, NULL, &format);
 	return 0;
 }
 
diff --git a/drivers/media/pci/cx18/cx18-ioctl.c b/drivers/media/pci/cx18/cx18-ioctl.c
index 0d676a57e24e..6f8d7f700016 100644
--- a/drivers/media/pci/cx18/cx18-ioctl.c
+++ b/drivers/media/pci/cx18/cx18-ioctl.c
@@ -150,7 +150,7 @@ static int cx18_s_fmt_vid_cap(struct file *file, void *fh,
 	format.format.width = cx->cxhdl.width = w;
 	format.format.height = cx->cxhdl.height = h;
 	format.format.code = MEDIA_BUS_FMT_FIXED;
-	v4l2_subdev_call(cx->sd_av, pad, set_fmt, NULL, &format);
+	v4l2_subdev_call(cx->sd_av, pad, set_fmt, NULL, NULL, &format);
 	return cx18_g_fmt_vid_cap(file, fh, fmt);
 }
 
diff --git a/drivers/media/pci/cx23885/cx23885-video.c b/drivers/media/pci/cx23885/cx23885-video.c
index 14d219fd1d8a..94ce00154e1f 100644
--- a/drivers/media/pci/cx23885/cx23885-video.c
+++ b/drivers/media/pci/cx23885/cx23885-video.c
@@ -134,7 +134,7 @@ int cx23885_set_tvnorm(struct cx23885_dev *dev, v4l2_std_id norm)
 	format.format.width = dev->width;
 	format.format.height = dev->height;
 	format.format.field = dev->field;
-	call_all(dev, pad, set_fmt, NULL, &format);
+	call_all(dev, pad, set_fmt, NULL, NULL, &format);
 
 	return 0;
 }
@@ -619,7 +619,7 @@ static int vidioc_s_fmt_vid_cap(struct file *file, void *priv,
 	dprintk(2, "%s() width=%d height=%d field=%d\n", __func__,
 		dev->width, dev->height, dev->field);
 	v4l2_fill_mbus_format(&format.format, &f->fmt.pix, MEDIA_BUS_FMT_FIXED);
-	call_all(dev, pad, set_fmt, NULL, &format);
+	call_all(dev, pad, set_fmt, NULL, NULL, &format);
 	v4l2_fill_pix_format(&f->fmt.pix, &format.format);
 	/* set_fmt overwrites f->fmt.pix.field, restore it */
 	f->fmt.pix.field = dev->field;
diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c b/drivers/media/pci/intel/ipu3/ipu3-cio2.c
index 986b9afd7cb5..3333cdd81c13 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c
@@ -1229,6 +1229,7 @@ static int cio2_subdev_init_state(struct v4l2_subdev *sd,
 }
 
 static int cio2_subdev_set_fmt(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
index 5317b01935a3..897271ba6104 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
@@ -404,6 +404,7 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 }
 
 static int ipu6_isys_csi2_set_sel(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *state,
 				  struct v4l2_subdev_selection *sel)
 {
@@ -455,6 +456,7 @@ static int ipu6_isys_csi2_set_sel(struct v4l2_subdev *sd,
 }
 
 static int ipu6_isys_csi2_get_sel(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *state,
 				  struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c b/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c
index dbd6f76a066d..ab04b5b0ad08 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c
@@ -160,6 +160,7 @@ u32 ipu6_isys_convert_bayer_order(u32 code, int x, int y)
 }
 
 int ipu6_isys_subdev_set_fmt(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.h b/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.h
index 35069099c364..d4f76d513dc6 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.h
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.h
@@ -31,6 +31,7 @@ bool ipu6_isys_is_bayer_format(u32 code);
 u32 ipu6_isys_convert_bayer_order(u32 code, int x, int y);
 
 int ipu6_isys_subdev_set_fmt(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *fmt);
 int ipu6_isys_subdev_enum_mbus_code(struct v4l2_subdev *sd,
diff --git a/drivers/media/pci/intel/ivsc/mei_csi.c b/drivers/media/pci/intel/ivsc/mei_csi.c
index c2917e156345..f1dcf498cae7 100644
--- a/drivers/media/pci/intel/ivsc/mei_csi.c
+++ b/drivers/media/pci/intel/ivsc/mei_csi.c
@@ -338,6 +338,7 @@ static int mei_csi_init_state(struct v4l2_subdev *sd,
 }
 
 static int mei_csi_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/pci/ivtv/ivtv-controls.c b/drivers/media/pci/ivtv/ivtv-controls.c
index f087a12c4ebd..816e634de1eb 100644
--- a/drivers/media/pci/ivtv/ivtv-controls.c
+++ b/drivers/media/pci/ivtv/ivtv-controls.c
@@ -60,7 +60,7 @@ static int ivtv_s_video_encoding(struct cx2341x_handler *cxhdl, u32 val)
 	format.format.width = cxhdl->width / (is_mpeg1 ? 2 : 1);
 	format.format.height = cxhdl->height;
 	format.format.code = MEDIA_BUS_FMT_FIXED;
-	v4l2_subdev_call(itv->sd_video, pad, set_fmt, NULL, &format);
+	v4l2_subdev_call(itv->sd_video, pad, set_fmt, NULL, NULL, &format);
 	return 0;
 }
 
diff --git a/drivers/media/pci/ivtv/ivtv-ioctl.c b/drivers/media/pci/ivtv/ivtv-ioctl.c
index fc95f0bf48d5..aba55fcc1e22 100644
--- a/drivers/media/pci/ivtv/ivtv-ioctl.c
+++ b/drivers/media/pci/ivtv/ivtv-ioctl.c
@@ -587,7 +587,7 @@ static int ivtv_s_fmt_vid_cap(struct file *file, void *fh, struct v4l2_format *f
 	format.format.width = fmt->fmt.pix.width;
 	format.format.height = h;
 	format.format.code = MEDIA_BUS_FMT_FIXED;
-	v4l2_subdev_call(itv->sd_video, pad, set_fmt, NULL, &format);
+	v4l2_subdev_call(itv->sd_video, pad, set_fmt, NULL, NULL, &format);
 	return ivtv_g_fmt_vid_cap(file, fh, fmt);
 }
 
diff --git a/drivers/media/pci/saa7134/saa7134-empress.c b/drivers/media/pci/saa7134/saa7134-empress.c
index 8c4f70e4177d..d04a68bb05d8 100644
--- a/drivers/media/pci/saa7134/saa7134-empress.c
+++ b/drivers/media/pci/saa7134/saa7134-empress.c
@@ -122,7 +122,7 @@ static int empress_s_fmt_vid_cap(struct file *file, void *priv,
 	};
 
 	v4l2_fill_mbus_format(&format.format, &f->fmt.pix, MEDIA_BUS_FMT_FIXED);
-	saa_call_all(dev, pad, set_fmt, NULL, &format);
+	saa_call_all(dev, pad, set_fmt, NULL, NULL, &format);
 	v4l2_fill_pix_format(&f->fmt.pix, &format.format);
 
 	f->fmt.pix.pixelformat  = V4L2_PIX_FMT_MPEG;
@@ -145,7 +145,7 @@ static int empress_try_fmt_vid_cap(struct file *file, void *priv,
 	};
 
 	v4l2_fill_mbus_format(&format.format, &f->fmt.pix, MEDIA_BUS_FMT_FIXED);
-	saa_call_all(dev, pad, set_fmt, &pad_state, &format);
+	saa_call_all(dev, pad, set_fmt, NULL, &pad_state, &format);
 	v4l2_fill_pix_format(&f->fmt.pix, &format.format);
 
 	f->fmt.pix.pixelformat  = V4L2_PIX_FMT_MPEG;
diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.c b/drivers/media/platform/amd/isp4/isp4_subdev.c
index 48deea79ce6c..03752d88c0b6 100644
--- a/drivers/media/platform/amd/isp4/isp4_subdev.c
+++ b/drivers/media/platform/amd/isp4/isp4_subdev.c
@@ -890,6 +890,7 @@ static const struct v4l2_subdev_video_ops isp4sd_video_ops = {
 };
 
 static int isp4sd_set_pad_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/amd/isp4/isp4_video.c b/drivers/media/platform/amd/isp4/isp4_video.c
index 0cebb39f98e1..856a2a0b4a12 100644
--- a/drivers/media/platform/amd/isp4/isp4_video.c
+++ b/drivers/media/platform/amd/isp4/isp4_video.c
@@ -240,7 +240,7 @@ static int isp4vid_set_fmt_2_isp(struct v4l2_subdev *sdev,
 	fmt.pad = ISP4VID_PAD_VIDEO_OUTPUT;
 	fmt.format.width = pix_fmt->width;
 	fmt.format.height = pix_fmt->height;
-	return v4l2_subdev_call(sdev, pad, set_fmt, NULL, &fmt);
+	return v4l2_subdev_call(sdev, pad, set_fmt, NULL, NULL, &fmt);
 }
 
 static int isp4vid_s_fmt_vid_cap(struct file *file, void *priv,
diff --git a/drivers/media/platform/amlogic/c3/isp/c3-isp-core.c b/drivers/media/platform/amlogic/c3/isp/c3-isp-core.c
index ff6413fff889..553c808f8f3b 100644
--- a/drivers/media/platform/amlogic/c3/isp/c3-isp-core.c
+++ b/drivers/media/platform/amlogic/c3/isp/c3-isp-core.c
@@ -461,6 +461,7 @@ static void c3_isp_core_set_source_fmt(struct v4l2_subdev_state *state,
 }
 
 static int c3_isp_core_set_fmt(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *state,
 			       struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/amlogic/c3/isp/c3-isp-resizer.c b/drivers/media/platform/amlogic/c3/isp/c3-isp-resizer.c
index 453a889e0b27..1f9c16eb0842 100644
--- a/drivers/media/platform/amlogic/c3/isp/c3-isp-resizer.c
+++ b/drivers/media/platform/amlogic/c3/isp/c3-isp-resizer.c
@@ -621,6 +621,7 @@ static void c3_isp_rsz_set_source_fmt(struct v4l2_subdev_state *state,
 }
 
 static int c3_isp_rsz_set_fmt(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *state,
 			      struct v4l2_subdev_format *format)
 {
@@ -633,6 +634,7 @@ static int c3_isp_rsz_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int c3_isp_rsz_get_selection(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *state,
 				    struct v4l2_subdev_selection *sel)
 {
@@ -674,6 +676,7 @@ static int c3_isp_rsz_get_selection(struct v4l2_subdev *sd,
 }
 
 static int c3_isp_rsz_set_selection(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *state,
 				    struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/amlogic/c3/mipi-adapter/c3-mipi-adap.c b/drivers/media/platform/amlogic/c3/mipi-adapter/c3-mipi-adap.c
index 4bd98fb9c7e9..6f75112723b5 100644
--- a/drivers/media/platform/amlogic/c3/mipi-adapter/c3-mipi-adap.c
+++ b/drivers/media/platform/amlogic/c3/mipi-adapter/c3-mipi-adap.c
@@ -521,6 +521,7 @@ static int c3_mipi_adap_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int c3_mipi_adap_set_fmt(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/amlogic/c3/mipi-csi2/c3-mipi-csi2.c b/drivers/media/platform/amlogic/c3/mipi-csi2/c3-mipi-csi2.c
index b9e4ef3fc308..0c399665f7fa 100644
--- a/drivers/media/platform/amlogic/c3/mipi-csi2/c3-mipi-csi2.c
+++ b/drivers/media/platform/amlogic/c3/mipi-csi2/c3-mipi-csi2.c
@@ -491,6 +491,7 @@ static int c3_mipi_csi_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int c3_mipi_csi_set_fmt(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *state,
 			       struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
index e128adf6ee37..c8464dec9a21 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
@@ -203,6 +203,7 @@ static int mali_c55_isp_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int mali_c55_isp_set_fmt(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_format *format)
 {
@@ -265,6 +266,7 @@ static int mali_c55_isp_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int mali_c55_isp_get_selection(struct v4l2_subdev *sd,
+				      const struct v4l2_subdev_client_info *ci,
 				      struct v4l2_subdev_state *state,
 				      struct v4l2_subdev_selection *sel)
 {
@@ -278,6 +280,7 @@ static int mali_c55_isp_get_selection(struct v4l2_subdev *sd,
 }
 
 static int mali_c55_isp_set_selection(struct v4l2_subdev *sd,
+				      const struct v4l2_subdev_client_info *ci,
 				      struct v4l2_subdev_state *state,
 				      struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
index c4f46651dcee..b1351f053b87 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
@@ -715,6 +715,7 @@ static int mali_c55_rsz_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int mali_c55_rsz_set_sink_fmt(struct v4l2_subdev *sd,
+				     const struct v4l2_subdev_client_info *ci,
 				     struct v4l2_subdev_state *state,
 				     struct v4l2_subdev_format *format)
 {
@@ -773,6 +774,7 @@ static int mali_c55_rsz_set_sink_fmt(struct v4l2_subdev *sd,
 }
 
 static int mali_c55_rsz_set_source_fmt(struct v4l2_subdev *sd,
+				       const struct v4l2_subdev_client_info *ci,
 				       struct v4l2_subdev_state *state,
 				       struct v4l2_subdev_format *format)
 {
@@ -812,6 +814,7 @@ static int mali_c55_rsz_set_source_fmt(struct v4l2_subdev *sd,
 }
 
 static int mali_c55_rsz_set_fmt(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_format *format)
 {
@@ -826,12 +829,13 @@ static int mali_c55_rsz_set_fmt(struct v4l2_subdev *sd,
 
 	if (format->pad == MALI_C55_RSZ_SINK_PAD ||
 	    format->pad == MALI_C55_RSZ_SINK_BYPASS_PAD)
-		return mali_c55_rsz_set_sink_fmt(sd, state, format);
+		return mali_c55_rsz_set_sink_fmt(sd, NULL, state, format);
 
-	return mali_c55_rsz_set_source_fmt(sd, state, format);
+	return mali_c55_rsz_set_source_fmt(sd, NULL, state, format);
 }
 
 static int mali_c55_rsz_get_selection(struct v4l2_subdev *sd,
+				      const struct v4l2_subdev_client_info *ci,
 				      struct v4l2_subdev_state *state,
 				      struct v4l2_subdev_selection *sel)
 {
@@ -850,6 +854,7 @@ static int mali_c55_rsz_get_selection(struct v4l2_subdev *sd,
 }
 
 static int mali_c55_rsz_set_crop(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -908,6 +913,7 @@ static int mali_c55_rsz_set_crop(struct v4l2_subdev *sd,
 }
 
 static int mali_c55_rsz_set_compose(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *state,
 				    struct v4l2_subdev_selection *sel)
 {
@@ -956,6 +962,7 @@ static int mali_c55_rsz_set_compose(struct v4l2_subdev *sd,
 }
 
 static int mali_c55_rsz_set_selection(struct v4l2_subdev *sd,
+				      const struct v4l2_subdev_client_info *ci,
 				      struct v4l2_subdev_state *state,
 				      struct v4l2_subdev_selection *sel)
 {
@@ -963,10 +970,10 @@ static int mali_c55_rsz_set_selection(struct v4l2_subdev *sd,
 		return -EINVAL;
 
 	if (sel->target == V4L2_SEL_TGT_CROP)
-		return mali_c55_rsz_set_crop(sd, state, sel);
+		return mali_c55_rsz_set_crop(sd, NULL, state, sel);
 
 	if (sel->target == V4L2_SEL_TGT_COMPOSE)
-		return mali_c55_rsz_set_compose(sd, state, sel);
+		return mali_c55_rsz_set_compose(sd, NULL, state, sel);
 
 	return -EINVAL;
 }
diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c b/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
index 894f4cf377af..cc390a3fc253 100644
--- a/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
+++ b/drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
@@ -180,6 +180,7 @@ static int mali_c55_tpg_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int mali_c55_tpg_set_fmt(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/atmel/atmel-isi.c b/drivers/media/platform/atmel/atmel-isi.c
index a05a744cbb75..f5e85271a275 100644
--- a/drivers/media/platform/atmel/atmel-isi.c
+++ b/drivers/media/platform/atmel/atmel-isi.c
@@ -609,7 +609,7 @@ static int isi_try_fmt(struct atmel_isi *isi, struct v4l2_format *f,
 
 	isi_try_fse(isi, isi_fmt, &pad_state);
 
-	ret = v4l2_subdev_call(isi->entity.subdev, pad, set_fmt,
+	ret = v4l2_subdev_call(isi->entity.subdev, pad, set_fmt, NULL,
 			       &pad_state, &format);
 	if (ret < 0)
 		return ret;
@@ -641,7 +641,7 @@ static int isi_set_fmt(struct atmel_isi *isi, struct v4l2_format *f)
 	v4l2_fill_mbus_format(&format.format, &f->fmt.pix,
 			      current_fmt->mbus_code);
 	ret = v4l2_subdev_call(isi->entity.subdev, pad,
-			       set_fmt, NULL, &format);
+			       set_fmt, NULL, NULL, &format);
 	if (ret < 0)
 		return ret;
 
diff --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
index 7e299ebf3470..a13efd617e33 100644
--- a/drivers/media/platform/broadcom/bcm2835-unicam.c
+++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
@@ -1325,6 +1325,7 @@ static int unicam_subdev_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int unicam_subdev_set_format(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *state,
 				    struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/cadence/cdns-csi2rx.c b/drivers/media/platform/cadence/cdns-csi2rx.c
index 1ff2d8f78d5b..07cba5eea5fc 100644
--- a/drivers/media/platform/cadence/cdns-csi2rx.c
+++ b/drivers/media/platform/cadence/cdns-csi2rx.c
@@ -622,6 +622,7 @@ static int csi2rx_set_routing(struct v4l2_subdev *subdev,
 }
 
 static int csi2rx_set_fmt(struct v4l2_subdev *subdev,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *state,
 			  struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/cadence/cdns-csi2tx.c b/drivers/media/platform/cadence/cdns-csi2tx.c
index 629b0fa838a2..07277866dca7 100644
--- a/drivers/media/platform/cadence/cdns-csi2tx.c
+++ b/drivers/media/platform/cadence/cdns-csi2tx.c
@@ -201,6 +201,7 @@ static int csi2tx_get_pad_format(struct v4l2_subdev *subdev,
 }
 
 static int csi2tx_set_pad_format(struct v4l2_subdev *subdev,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/intel/pxa_camera.c b/drivers/media/platform/intel/pxa_camera.c
index 2269d852ab0e..9e1fdd963b55 100644
--- a/drivers/media/platform/intel/pxa_camera.c
+++ b/drivers/media/platform/intel/pxa_camera.c
@@ -1819,7 +1819,7 @@ static int pxac_vidioc_try_fmt_vid_cap(struct file *filp, void *priv,
 			      pixfmt == V4L2_PIX_FMT_YUV422P ? 4 : 0);
 
 	v4l2_fill_mbus_format(mf, pix, xlate->code);
-	ret = sensor_call(pcdev, pad, set_fmt, &pad_state, &format);
+	ret = sensor_call(pcdev, pad, set_fmt, NULL, &pad_state, &format);
 	if (ret < 0)
 		return ret;
 
@@ -1882,7 +1882,7 @@ static int pxac_vidioc_s_fmt_vid_cap(struct file *filp, void *priv,
 	xlate = pxa_mbus_xlate_by_fourcc(pcdev->user_formats,
 					 pix->pixelformat);
 	v4l2_fill_mbus_format(&format.format, pix, xlate->code);
-	ret = sensor_call(pcdev, pad, set_fmt, NULL, &format);
+	ret = sensor_call(pcdev, pad, set_fmt, NULL, NULL, &format);
 	if (ret < 0) {
 		dev_warn(pcdev_to_dev(pcdev),
 			 "Failed to configure for format %x\n",
@@ -2091,7 +2091,7 @@ static int pxa_camera_sensor_bound(struct v4l2_async_notifier *notifier,
 	if (err)
 		goto out;
 
-	err = sensor_call(pcdev, pad, set_fmt, NULL, &format);
+	err = sensor_call(pcdev, pad, set_fmt, NULL, NULL, &format);
 	if (err)
 		goto out_sensor_poweroff;
 
diff --git a/drivers/media/platform/marvell/mcam-core.c b/drivers/media/platform/marvell/mcam-core.c
index b8360d37000a..227fbecd8488 100644
--- a/drivers/media/platform/marvell/mcam-core.c
+++ b/drivers/media/platform/marvell/mcam-core.c
@@ -1022,7 +1022,7 @@ static int mcam_cam_configure(struct mcam_camera *cam)
 	v4l2_fill_mbus_format(&format.format, &cam->pix_format, cam->mbus_code);
 	ret = sensor_call(cam, core, init, 0);
 	if (ret == 0)
-		ret = sensor_call(cam, pad, set_fmt, NULL, &format);
+		ret = sensor_call(cam, pad, set_fmt, NULL, NULL, &format);
 	/*
 	 * OV7670 does weird things if flip is set *before* format...
 	 */
@@ -1362,7 +1362,7 @@ static int mcam_vidioc_try_fmt_vid_cap(struct file *filp, void *priv,
 	f = mcam_find_format(pix->pixelformat);
 	pix->pixelformat = f->pixelformat;
 	v4l2_fill_mbus_format(&format.format, pix, f->mbus_code);
-	ret = sensor_call(cam, pad, set_fmt, &pad_state, &format);
+	ret = sensor_call(cam, pad, set_fmt, NULL, &pad_state, &format);
 	v4l2_fill_pix_format(pix, &format.format);
 	pix->bytesperline = pix->width * f->bpp;
 	switch (f->pixelformat) {
diff --git a/drivers/media/platform/microchip/microchip-csi2dc.c b/drivers/media/platform/microchip/microchip-csi2dc.c
index 70303a0b6919..347e44fcd9ee 100644
--- a/drivers/media/platform/microchip/microchip-csi2dc.c
+++ b/drivers/media/platform/microchip/microchip-csi2dc.c
@@ -245,6 +245,7 @@ static int csi2dc_get_fmt(struct v4l2_subdev *csi2dc_sd,
 }
 
 static int csi2dc_set_fmt(struct v4l2_subdev *csi2dc_sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *req_fmt)
 {
diff --git a/drivers/media/platform/microchip/microchip-isc-scaler.c b/drivers/media/platform/microchip/microchip-isc-scaler.c
index e83463543e21..192532f87271 100644
--- a/drivers/media/platform/microchip/microchip-isc-scaler.c
+++ b/drivers/media/platform/microchip/microchip-isc-scaler.c
@@ -46,6 +46,7 @@ static int isc_scaler_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int isc_scaler_set_fmt(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *req_fmt)
 {
@@ -124,6 +125,7 @@ static int isc_scaler_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int isc_scaler_g_sel(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/nxp/imx-mipi-csis.c b/drivers/media/platform/nxp/imx-mipi-csis.c
index 5f9691d76434..1b9012dbae39 100644
--- a/drivers/media/platform/nxp/imx-mipi-csis.c
+++ b/drivers/media/platform/nxp/imx-mipi-csis.c
@@ -1104,6 +1104,7 @@ static int mipi_csis_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int mipi_csis_set_fmt(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *sdformat)
 {
@@ -1225,7 +1226,7 @@ static int mipi_csis_init_state(struct v4l2_subdev *sd,
 		V4L2_MAP_QUANTIZATION_DEFAULT(false, fmt.format.colorspace,
 					      fmt.format.ycbcr_enc);
 
-	return mipi_csis_set_fmt(sd, state, &fmt);
+	return mipi_csis_set_fmt(sd, NULL, state, &fmt);
 }
 
 static int mipi_csis_log_status(struct v4l2_subdev *sd)
diff --git a/drivers/media/platform/nxp/imx7-media-csi.c b/drivers/media/platform/nxp/imx7-media-csi.c
index 7ddc7ba06e3d..0b2ee4dfdcbb 100644
--- a/drivers/media/platform/nxp/imx7-media-csi.c
+++ b/drivers/media/platform/nxp/imx7-media-csi.c
@@ -1890,6 +1890,7 @@ static void imx7_csi_try_fmt(struct v4l2_subdev *sd,
 }
 
 static int imx7_csi_set_fmt(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *sdformat)
 {
diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
index a1cd1e159468..37c147f00038 100644
--- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
+++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
@@ -249,6 +249,7 @@ static int mxc_isi_crossbar_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int mxc_isi_crossbar_set_fmt(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *state,
 				    struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c
index 2d0843c86534..e4ec2bc4fd96 100644
--- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c
+++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c
@@ -448,6 +448,7 @@ static int mxc_isi_pipe_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int mxc_isi_pipe_set_fmt(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_format *fmt)
 {
@@ -543,6 +544,7 @@ static int mxc_isi_pipe_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int mxc_isi_pipe_get_selection(struct v4l2_subdev *sd,
+				      const struct v4l2_subdev_client_info *ci,
 				      struct v4l2_subdev_state *state,
 				      struct v4l2_subdev_selection *sel)
 {
@@ -602,6 +604,7 @@ static int mxc_isi_pipe_get_selection(struct v4l2_subdev *sd,
 }
 
 static int mxc_isi_pipe_set_selection(struct v4l2_subdev *sd,
+				      const struct v4l2_subdev_client_info *ci,
 				      struct v4l2_subdev_state *state,
 				      struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/nxp/imx8mq-mipi-csi2.c b/drivers/media/platform/nxp/imx8mq-mipi-csi2.c
index 04ebed8a0493..17756b7fe1dd 100644
--- a/drivers/media/platform/nxp/imx8mq-mipi-csi2.c
+++ b/drivers/media/platform/nxp/imx8mq-mipi-csi2.c
@@ -597,6 +597,7 @@ static int imx8mq_mipi_csi_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int imx8mq_mipi_csi_set_fmt(struct v4l2_subdev *sd,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *sd_state,
 				   struct v4l2_subdev_format *sdformat)
 {
diff --git a/drivers/media/platform/qcom/camss/camss-csid.c b/drivers/media/platform/qcom/camss/camss-csid.c
index 48459b46a981..a00791b00feb 100644
--- a/drivers/media/platform/qcom/camss/camss-csid.c
+++ b/drivers/media/platform/qcom/camss/camss-csid.c
@@ -987,6 +987,7 @@ static int csid_get_format(struct v4l2_subdev *sd,
  * Return -EINVAL or zero on success
  */
 static int csid_set_format(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
@@ -1036,7 +1037,7 @@ static int csid_init_formats(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
 		}
 	};
 
-	return csid_set_format(sd, fh ? fh->state : NULL, &format);
+	return csid_set_format(sd, NULL, fh ? fh->state : NULL, &format);
 }
 
 /*
diff --git a/drivers/media/platform/qcom/camss/camss-csiphy.c b/drivers/media/platform/qcom/camss/camss-csiphy.c
index 539ac4888b60..0dd50f3879d9 100644
--- a/drivers/media/platform/qcom/camss/camss-csiphy.c
+++ b/drivers/media/platform/qcom/camss/camss-csiphy.c
@@ -503,6 +503,7 @@ static int csiphy_get_format(struct v4l2_subdev *sd,
  * Return -EINVAL or zero on success
  */
 static int csiphy_set_format(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_format *fmt)
 {
@@ -555,7 +556,7 @@ static int csiphy_init_formats(struct v4l2_subdev *sd,
 		}
 	};
 
-	return csiphy_set_format(sd, fh ? fh->state : NULL, &format);
+	return csiphy_set_format(sd, NULL, fh ? fh->state : NULL, &format);
 }
 
 static bool __printf(2, 3)
diff --git a/drivers/media/platform/qcom/camss/camss-ispif.c b/drivers/media/platform/qcom/camss/camss-ispif.c
index aaf3caa42d33..b45080e98123 100644
--- a/drivers/media/platform/qcom/camss/camss-ispif.c
+++ b/drivers/media/platform/qcom/camss/camss-ispif.c
@@ -1037,6 +1037,7 @@ static int ispif_get_format(struct v4l2_subdev *sd,
  * Return -EINVAL or zero on success
  */
 static int ispif_set_format(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *fmt)
 {
@@ -1085,7 +1086,7 @@ static int ispif_init_formats(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
 		}
 	};
 
-	return ispif_set_format(sd, fh ? fh->state : NULL, &format);
+	return ispif_set_format(sd, NULL, fh ? fh->state : NULL, &format);
 }
 
 /*
diff --git a/drivers/media/platform/qcom/camss/camss-tpg.c b/drivers/media/platform/qcom/camss/camss-tpg.c
index c5b75132add4..5764a60fa34c 100644
--- a/drivers/media/platform/qcom/camss/camss-tpg.c
+++ b/drivers/media/platform/qcom/camss/camss-tpg.c
@@ -292,6 +292,7 @@ static int tpg_get_format(struct v4l2_subdev *sd,
 }
 
 static int tpg_set_format(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
@@ -322,7 +323,7 @@ static int tpg_init_formats(struct v4l2_subdev *sd,
 		}
 	};
 
-	return tpg_set_format(sd, fh ? fh->state : NULL, &format);
+	return tpg_set_format(sd, NULL, fh ? fh->state : NULL, &format);
 }
 
 static int tpg_s_ctrl(struct v4l2_ctrl *ctrl)
diff --git a/drivers/media/platform/qcom/camss/camss-vfe.c b/drivers/media/platform/qcom/camss/camss-vfe.c
index 319d19158988..0ba5bf3abeda 100644
--- a/drivers/media/platform/qcom/camss/camss-vfe.c
+++ b/drivers/media/platform/qcom/camss/camss-vfe.c
@@ -1562,6 +1562,7 @@ static int vfe_get_format(struct v4l2_subdev *sd,
 }
 
 static int vfe_set_selection(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel);
 
@@ -1574,6 +1575,7 @@ static int vfe_set_selection(struct v4l2_subdev *sd,
  * Return -EINVAL or zero on success
  */
 static int vfe_set_format(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
@@ -1608,7 +1610,7 @@ static int vfe_set_format(struct v4l2_subdev *sd,
 		sel.target = V4L2_SEL_TGT_COMPOSE;
 		sel.r.width = fmt->format.width;
 		sel.r.height = fmt->format.height;
-		ret = vfe_set_selection(sd, sd_state, &sel);
+		ret = vfe_set_selection(sd, NULL, sd_state, &sel);
 		if (ret < 0)
 			return ret;
 	}
@@ -1625,6 +1627,7 @@ static int vfe_set_format(struct v4l2_subdev *sd,
  * Return -EINVAL or zero on success
  */
 static int vfe_get_selection(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel)
 {
@@ -1695,6 +1698,7 @@ static int vfe_get_selection(struct v4l2_subdev *sd,
  * Return -EINVAL or zero on success
  */
 static int vfe_set_selection(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel)
 {
@@ -1721,7 +1725,7 @@ static int vfe_set_selection(struct v4l2_subdev *sd,
 		crop.pad = MSM_VFE_PAD_SRC;
 		crop.target = V4L2_SEL_TGT_CROP;
 		crop.r = *rect;
-		ret = vfe_set_selection(sd, sd_state, &crop);
+		ret = vfe_set_selection(sd, NULL, sd_state, &crop);
 	} else if (sel->target == V4L2_SEL_TGT_CROP &&
 		sel->pad == MSM_VFE_PAD_SRC) {
 		struct v4l2_subdev_format fmt = { 0 };
@@ -1742,7 +1746,7 @@ static int vfe_set_selection(struct v4l2_subdev *sd,
 
 		fmt.format.width = rect->width;
 		fmt.format.height = rect->height;
-		ret = vfe_set_format(sd, sd_state, &fmt);
+		ret = vfe_set_format(sd, NULL, sd_state, &fmt);
 	} else {
 		ret = -EINVAL;
 	}
@@ -1772,7 +1776,7 @@ static int vfe_init_formats(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
 		}
 	};
 
-	return vfe_set_format(sd, fh ? fh->state : NULL, &format);
+	return vfe_set_format(sd, NULL, fh ? fh->state : NULL, &format);
 }
 
 /*
diff --git a/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c b/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
index 104908afbf41..66d0dde6d840 100644
--- a/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
+++ b/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
@@ -404,6 +404,7 @@ static int csi2_init_state(struct v4l2_subdev *sd,
 }
 
 static int csi2_pad_set_fmt(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *state,
 			    struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/raspberrypi/rp1-cfe/pisp-fe.c b/drivers/media/platform/raspberrypi/rp1-cfe/pisp-fe.c
index 05762b1be2bc..ea21fb1f60cf 100644
--- a/drivers/media/platform/raspberrypi/rp1-cfe/pisp-fe.c
+++ b/drivers/media/platform/raspberrypi/rp1-cfe/pisp-fe.c
@@ -429,6 +429,7 @@ static int pisp_fe_init_state(struct v4l2_subdev *sd,
 }
 
 static int pisp_fe_pad_set_fmt(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *state,
 			       struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index 7305cc4a04cb..ffbe9ded82f3 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -1884,6 +1884,7 @@ static int rcsi2_disable_streams(struct v4l2_subdev *sd,
 }
 
 static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/renesas/rcar-isp/csisp.c b/drivers/media/platform/renesas/rcar-isp/csisp.c
index 8fb2cc3b5650..b21940a8a18c 100644
--- a/drivers/media/platform/renesas/rcar-isp/csisp.c
+++ b/drivers/media/platform/renesas/rcar-isp/csisp.c
@@ -331,6 +331,7 @@ static int risp_disable_streams(struct v4l2_subdev *sd,
 }
 
 static int risp_set_pad_format(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *state,
 			       struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/renesas/renesas-ceu.c b/drivers/media/platform/renesas/renesas-ceu.c
index 65f7659a9e02..47b9e4753301 100644
--- a/drivers/media/platform/renesas/renesas-ceu.c
+++ b/drivers/media/platform/renesas/renesas-ceu.c
@@ -841,12 +841,13 @@ static int __ceu_try_fmt(struct ceu_device *ceudev, struct v4l2_format *v4l2_fmt
 	 * time.
 	 */
 	sd_format.format.code = mbus_code;
-	ret = v4l2_subdev_call(v4l2_sd, pad, set_fmt, &pad_state, &sd_format);
+	ret = v4l2_subdev_call(v4l2_sd, pad, set_fmt, NULL, &pad_state,
+			       &sd_format);
 	if (ret) {
 		if (ret == -EINVAL) {
 			/* fallback */
 			sd_format.format.code = mbus_code_old;
-			ret = v4l2_subdev_call(v4l2_sd, pad, set_fmt,
+			ret = v4l2_subdev_call(v4l2_sd, pad, set_fmt, NULL,
 					       &pad_state, &sd_format);
 		}
 
@@ -900,7 +901,7 @@ static int ceu_set_fmt(struct ceu_device *ceudev, struct v4l2_format *v4l2_fmt)
 
 	format.format.code = mbus_code;
 	v4l2_fill_mbus_format_mplane(&format.format, &v4l2_fmt->fmt.pix_mp);
-	ret = v4l2_subdev_call(v4l2_sd, pad, set_fmt, NULL, &format);
+	ret = v4l2_subdev_call(v4l2_sd, pad, set_fmt, NULL, NULL, &format);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/renesas/rzg2l-cru/rzg2l-csi2.c b/drivers/media/platform/renesas/rzg2l-cru/rzg2l-csi2.c
index 6dc4b53607b4..c17283cba3a6 100644
--- a/drivers/media/platform/renesas/rzg2l-cru/rzg2l-csi2.c
+++ b/drivers/media/platform/renesas/rzg2l-cru/rzg2l-csi2.c
@@ -633,6 +633,7 @@ static int rzg2l_csi2_post_streamoff(struct v4l2_subdev *sd)
 }
 
 static int rzg2l_csi2_set_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *state,
 				 struct v4l2_subdev_format *fmt)
 {
@@ -687,7 +688,7 @@ static int rzg2l_csi2_init_state(struct v4l2_subdev *sd,
 	fmt.format.quantization = V4L2_QUANTIZATION_DEFAULT;
 	fmt.format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
 
-	return rzg2l_csi2_set_format(sd, sd_state, &fmt);
+	return rzg2l_csi2_set_format(sd, NULL, sd_state, &fmt);
 }
 
 static int rzg2l_csi2_enum_mbus_code(struct v4l2_subdev *sd,
diff --git a/drivers/media/platform/renesas/rzg2l-cru/rzg2l-ip.c b/drivers/media/platform/renesas/rzg2l-cru/rzg2l-ip.c
index 5f2c87858bfe..21dd0162fd6e 100644
--- a/drivers/media/platform/renesas/rzg2l-cru/rzg2l-ip.c
+++ b/drivers/media/platform/renesas/rzg2l-cru/rzg2l-ip.c
@@ -200,6 +200,7 @@ static int rzg2l_cru_ip_s_stream(struct v4l2_subdev *sd, int enable)
 }
 
 static int rzg2l_cru_ip_set_format(struct v4l2_subdev *sd,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *state,
 				   struct v4l2_subdev_format *fmt)
 {
@@ -300,7 +301,7 @@ static int rzg2l_cru_ip_init_state(struct v4l2_subdev *sd,
 	fmt.format.quantization = V4L2_QUANTIZATION_DEFAULT;
 	fmt.format.xfer_func = V4L2_XFER_FUNC_DEFAULT;
 
-	return rzg2l_cru_ip_set_format(sd, sd_state, &fmt);
+	return rzg2l_cru_ip_set_format(sd, NULL, sd_state, &fmt);
 }
 
 static const struct v4l2_subdev_video_ops rzg2l_cru_ip_video_ops = {
diff --git a/drivers/media/platform/renesas/rzv2h-ivc/rzv2h-ivc-subdev.c b/drivers/media/platform/renesas/rzv2h-ivc/rzv2h-ivc-subdev.c
index b1659544eaa0..04e168b9526f 100644
--- a/drivers/media/platform/renesas/rzv2h-ivc/rzv2h-ivc-subdev.c
+++ b/drivers/media/platform/renesas/rzv2h-ivc/rzv2h-ivc-subdev.c
@@ -157,6 +157,7 @@ static int rzv2h_ivc_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int rzv2h_ivc_set_fmt(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/renesas/sh_vou.c b/drivers/media/platform/renesas/sh_vou.c
index 4ad7ae188d5b..c06a8fdcd11f 100644
--- a/drivers/media/platform/renesas/sh_vou.c
+++ b/drivers/media/platform/renesas/sh_vou.c
@@ -714,7 +714,7 @@ static int sh_vou_set_fmt_vid_out(struct sh_vou_device *vou_dev,
 	mbfmt->width = geo.output.width;
 	mbfmt->height = geo.output.height;
 	ret = v4l2_device_call_until_err(&vou_dev->v4l2_dev, 0, pad,
-					 set_fmt, NULL, &format);
+					 set_fmt, NULL, NULL, &format);
 	/* Must be implemented, so, don't check for -ENOIOCTLCMD */
 	if (ret < 0)
 		return ret;
@@ -974,7 +974,7 @@ static int sh_vou_s_selection(struct file *file, void *fh,
 	 * final encoder configuration.
 	 */
 	v4l2_device_call_until_err(&vou_dev->v4l2_dev, 0, pad,
-				   set_selection, NULL, &sd_sel);
+				   set_selection, NULL, NULL, &sd_sel);
 	format.format.width = geo.output.width;
 	format.format.height = geo.output.height;
 	ret = v4l2_device_call_until_err(&vou_dev->v4l2_dev, 0, pad,
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_brx.c b/drivers/media/platform/renesas/vsp1/vsp1_brx.c
index d150a92b2617..e5781c67ff80 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_brx.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_brx.c
@@ -124,6 +124,7 @@ static void brx_try_format(struct vsp1_brx *brx,
 }
 
 static int brx_set_format(struct v4l2_subdev *subdev,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
@@ -173,6 +174,7 @@ static int brx_set_format(struct v4l2_subdev *subdev,
 }
 
 static int brx_get_selection(struct v4l2_subdev *subdev,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel)
 {
@@ -208,6 +210,7 @@ static int brx_get_selection(struct v4l2_subdev *subdev,
 }
 
 static int brx_set_selection(struct v4l2_subdev *subdev,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drm.c b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
index f6fbd3475329..5eb262a88f9f 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
@@ -137,7 +137,8 @@ static int vsp1_du_insert_uif(struct vsp1_device *vsp1,
 
 	format.pad = UIF_PAD_SINK;
 
-	ret = v4l2_subdev_call(&uif->subdev, pad, set_fmt, NULL, &format);
+	ret = v4l2_subdev_call(&uif->subdev, pad, set_fmt, NULL, NULL,
+			       &format);
 	if (ret < 0)
 		return ret;
 
@@ -184,7 +185,7 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
 	format.format.ycbcr_enc = input->ycbcr_enc;
 	format.format.quantization = input->quantization;
 
-	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_fmt, NULL,
+	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_fmt, NULL, NULL,
 			       &format);
 	if (ret < 0)
 		return ret;
@@ -199,6 +200,7 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
 	sel.r = input->crop;
 
 	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_selection, NULL,
+			       NULL,
 			       &sel);
 	if (ret < 0)
 		return ret;
@@ -226,7 +228,7 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
 
 	format.format.code = MEDIA_BUS_FMT_ARGB8888_1X32;
 
-	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_fmt, NULL,
+	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_fmt, NULL, NULL,
 			       &format);
 	if (ret < 0)
 		return ret;
@@ -240,7 +242,7 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
 	/* BRx sink, propagate the format from the RPF source. */
 	format.pad = brx_input;
 
-	ret = v4l2_subdev_call(&pipe->brx->subdev, pad, set_fmt, NULL,
+	ret = v4l2_subdev_call(&pipe->brx->subdev, pad, set_fmt, NULL, NULL,
 			       &format);
 	if (ret < 0)
 		return ret;
@@ -254,6 +256,7 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
 	sel.r = vsp1->drm->inputs[rpf->entity.index].compose;
 
 	ret = v4l2_subdev_call(&pipe->brx->subdev, pad, set_selection, NULL,
+			       NULL,
 			       &sel);
 	if (ret < 0)
 		return ret;
@@ -387,7 +390,7 @@ static int vsp1_du_pipeline_setup_brx(struct vsp1_device *vsp1,
 	format.format.height = drm_pipe->height;
 	format.format.field = V4L2_FIELD_NONE;
 
-	ret = v4l2_subdev_call(&brx->subdev, pad, set_fmt, NULL,
+	ret = v4l2_subdev_call(&brx->subdev, pad, set_fmt, NULL, NULL,
 			       &format);
 	if (ret < 0)
 		return ret;
@@ -539,7 +542,8 @@ static int vsp1_du_pipeline_setup_output(struct vsp1_device *vsp1,
 	format.format.code = MEDIA_BUS_FMT_ARGB8888_1X32;
 	format.format.field = V4L2_FIELD_NONE;
 
-	ret = v4l2_subdev_call(&pipe->output->entity.subdev, pad, set_fmt, NULL,
+	ret = v4l2_subdev_call(&pipe->output->entity.subdev, pad, set_fmt,
+			       NULL, NULL,
 			       &format);
 	if (ret < 0)
 		return ret;
@@ -559,7 +563,7 @@ static int vsp1_du_pipeline_setup_output(struct vsp1_device *vsp1,
 		format.format.code, pipe->output->entity.index);
 
 	format.pad = LIF_PAD_SINK;
-	ret = v4l2_subdev_call(&pipe->lif->subdev, pad, set_fmt, NULL,
+	ret = v4l2_subdev_call(&pipe->lif->subdev, pad, set_fmt, NULL, NULL,
 			       &format);
 	if (ret < 0)
 		return ret;
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_entity.c b/drivers/media/platform/renesas/vsp1/vsp1_entity.c
index 50aae8ee724f..a6e508e82987 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_entity.c
@@ -300,6 +300,7 @@ int vsp1_subdev_enum_frame_size(struct v4l2_subdev *subdev,
  * entity's limits, and propagates the sink pad format to the source pad.
  */
 int vsp1_subdev_set_pad_format(struct v4l2_subdev *subdev,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_format *fmt)
 {
@@ -382,7 +383,8 @@ static int vsp1_entity_init_state(struct v4l2_subdev *subdev,
 			       : V4L2_SUBDEV_FORMAT_ACTIVE,
 		};
 
-		v4l2_subdev_call(subdev, pad, set_fmt, sd_state, &format);
+		v4l2_subdev_call(subdev, pad, set_fmt, NULL, sd_state,
+				 &format);
 	}
 
 	return 0;
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_entity.h b/drivers/media/platform/renesas/vsp1/vsp1_entity.h
index c0c1fe7d3e40..677a1efc104a 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/renesas/vsp1/vsp1_entity.h
@@ -188,6 +188,7 @@ int vsp1_subdev_get_pad_format(struct v4l2_subdev *subdev,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_format *fmt);
 int vsp1_subdev_set_pad_format(struct v4l2_subdev *subdev,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_format *fmt);
 int vsp1_subdev_enum_mbus_code(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_histo.c b/drivers/media/platform/renesas/vsp1/vsp1_histo.c
index 97dbfb93abe9..bd9efb42fd1f 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_histo.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_histo.c
@@ -185,6 +185,7 @@ static int histo_enum_frame_size(struct v4l2_subdev *subdev,
 }
 
 static int histo_get_selection(struct v4l2_subdev *subdev,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_selection *sel)
 {
@@ -306,6 +307,7 @@ static int histo_set_compose(struct v4l2_subdev *subdev,
 }
 
 static int histo_set_selection(struct v4l2_subdev *subdev,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_selection *sel)
 {
@@ -330,6 +332,7 @@ static int histo_set_selection(struct v4l2_subdev *subdev,
 }
 
 static int histo_set_format(struct v4l2_subdev *subdev,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_hsit.c b/drivers/media/platform/renesas/vsp1/vsp1_hsit.c
index df069c228243..d734b5775af6 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_hsit.c
@@ -109,6 +109,7 @@ static int hsit_enum_frame_size(struct v4l2_subdev *subdev,
 }
 
 static int hsit_set_format(struct v4l2_subdev *subdev,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
index ced01870acd6..ff5894265b54 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
@@ -110,6 +110,7 @@ static int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
 }
 
 static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_format *fmt)
 {
@@ -214,6 +215,7 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 }
 
 static int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *sd_state,
 				   struct v4l2_subdev_selection *sel)
 {
@@ -255,6 +257,7 @@ static int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
 }
 
 static int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *sd_state,
 				   struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_sru.c b/drivers/media/platform/renesas/vsp1/vsp1_sru.c
index 3fd9fde5c724..5e1cf60be311 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_sru.c
@@ -210,6 +210,7 @@ static void sru_try_format(struct vsp1_sru *sru,
 }
 
 static int sru_set_format(struct v4l2_subdev *subdev,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_uds.c b/drivers/media/platform/renesas/vsp1/vsp1_uds.c
index 9f7bb112929e..d129964ec14d 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_uds.c
@@ -193,6 +193,7 @@ static void uds_try_format(struct vsp1_uds *uds,
 }
 
 static int uds_set_format(struct v4l2_subdev *subdev,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_uif.c b/drivers/media/platform/renesas/vsp1/vsp1_uif.c
index 52dbfe58a70d..16ddcd594134 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_uif.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_uif.c
@@ -54,6 +54,7 @@ static const unsigned int uif_codes[] = {
 };
 
 static int uif_get_selection(struct v4l2_subdev *subdev,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel)
 {
@@ -92,6 +93,7 @@ static int uif_get_selection(struct v4l2_subdev *subdev,
 }
 
 static int uif_set_selection(struct v4l2_subdev *subdev,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_vspx.c b/drivers/media/platform/renesas/vsp1/vsp1_vspx.c
index 1673479be0ff..5c39bacfb13f 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_vspx.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_vspx.c
@@ -120,7 +120,8 @@ static int vsp1_vspx_rwpf_set_subdev_fmt(struct vsp1_device *vsp1,
 	format.format.field = V4L2_FIELD_NONE;
 	format.format.code = rwpf->fmtinfo->mbus;
 
-	return v4l2_subdev_call(&ent->subdev, pad, set_fmt, NULL, &format);
+	return v4l2_subdev_call(&ent->subdev, pad, set_fmt, NULL, NULL,
+				&format);
 }
 
 /* Configure the RPF->IIF->WPF pipeline for ConfigDMA or RAW image transfer. */
diff --git a/drivers/media/platform/rockchip/rkcif/rkcif-interface.c b/drivers/media/platform/rockchip/rkcif/rkcif-interface.c
index 414a9980cf2e..2cba42d7c5d9 100644
--- a/drivers/media/platform/rockchip/rkcif/rkcif-interface.c
+++ b/drivers/media/platform/rockchip/rkcif/rkcif-interface.c
@@ -25,6 +25,7 @@ static const struct media_entity_operations rkcif_interface_media_ops = {
 };
 
 static int rkcif_interface_set_fmt(struct v4l2_subdev *sd,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *state,
 				   struct v4l2_subdev_format *format)
 {
@@ -77,6 +78,7 @@ static int rkcif_interface_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int rkcif_interface_get_sel(struct v4l2_subdev *sd,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *state,
 				   struct v4l2_subdev_selection *sel)
 {
@@ -115,6 +117,7 @@ static int rkcif_interface_get_sel(struct v4l2_subdev *sd,
 }
 
 static int rkcif_interface_set_sel(struct v4l2_subdev *sd,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *state,
 				   struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-csi.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-csi.c
index ddc6182f3e4b..9fa8a0407c90 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-csi.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-csi.c
@@ -304,6 +304,7 @@ static int rkisp1_csi_init_state(struct v4l2_subdev *sd,
 }
 
 static int rkisp1_csi_set_fmt(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
index 21d9f7f41411..0aaee32ec64d 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
@@ -818,6 +818,7 @@ static void rkisp1_isp_set_sink_fmt(struct rkisp1_isp *isp,
 }
 
 static int rkisp1_isp_set_fmt(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
@@ -835,6 +836,7 @@ static int rkisp1_isp_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int rkisp1_isp_get_selection(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *sd_state,
 				    struct v4l2_subdev_selection *sel)
 {
@@ -873,6 +875,7 @@ static int rkisp1_isp_get_selection(struct v4l2_subdev *sd,
 }
 
 static int rkisp1_isp_set_selection(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *sd_state,
 				    struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c
index 8e6b753d3081..5926917434af 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c
@@ -543,6 +543,7 @@ static void rkisp1_rsz_set_sink_fmt(struct rkisp1_resizer *rsz,
 }
 
 static int rkisp1_rsz_set_fmt(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
@@ -558,6 +559,7 @@ static int rkisp1_rsz_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int rkisp1_rsz_get_selection(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *sd_state,
 				    struct v4l2_subdev_selection *sel)
 {
@@ -591,6 +593,7 @@ static int rkisp1_rsz_get_selection(struct v4l2_subdev *sd,
 }
 
 static int rkisp1_rsz_set_selection(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *sd_state,
 				    struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-capture.c b/drivers/media/platform/samsung/exynos4-is/fimc-capture.c
index b5749f9cba39..61200c1e9681 100644
--- a/drivers/media/platform/samsung/exynos4-is/fimc-capture.c
+++ b/drivers/media/platform/samsung/exynos4-is/fimc-capture.c
@@ -797,7 +797,8 @@ static int fimc_pipeline_try_format(struct fimc_ctx *ctx,
 			sd = media_entity_to_v4l2_subdev(me);
 
 			sfmt.pad = 0;
-			ret = v4l2_subdev_call(sd, pad, set_fmt, NULL, &sfmt);
+			ret = v4l2_subdev_call(sd, pad, set_fmt, NULL, NULL,
+					       &sfmt);
 			if (ret)
 				return ret;
 
@@ -805,6 +806,7 @@ static int fimc_pipeline_try_format(struct fimc_ctx *ctx,
 				sfmt.pad = me->num_pads - 1;
 				mf->code = tfmt->code;
 				ret = v4l2_subdev_call(sd, pad, set_fmt, NULL,
+									NULL,
 									&sfmt);
 				if (ret)
 					return ret;
@@ -1519,6 +1521,7 @@ static int fimc_subdev_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int fimc_subdev_set_fmt(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_format *fmt)
 {
@@ -1585,6 +1588,7 @@ static int fimc_subdev_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int fimc_subdev_get_selection(struct v4l2_subdev *sd,
+				     const struct v4l2_subdev_client_info *ci,
 				     struct v4l2_subdev_state *sd_state,
 				     struct v4l2_subdev_selection *sel)
 {
@@ -1641,6 +1645,7 @@ static int fimc_subdev_get_selection(struct v4l2_subdev *sd,
 }
 
 static int fimc_subdev_set_selection(struct v4l2_subdev *sd,
+				     const struct v4l2_subdev_client_info *ci,
 				     struct v4l2_subdev_state *sd_state,
 				     struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-isp.c b/drivers/media/platform/samsung/exynos4-is/fimc-isp.c
index 3c5d7bee2655..a0a9093081dd 100644
--- a/drivers/media/platform/samsung/exynos4-is/fimc-isp.c
+++ b/drivers/media/platform/samsung/exynos4-is/fimc-isp.c
@@ -191,6 +191,7 @@ static void __isp_subdev_try_format(struct fimc_isp *isp,
 }
 
 static int fimc_isp_subdev_set_fmt(struct v4l2_subdev *sd,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *sd_state,
 				   struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-lite.c b/drivers/media/platform/samsung/exynos4-is/fimc-lite.c
index 8be20fd32d1c..1b7bae15189b 100644
--- a/drivers/media/platform/samsung/exynos4-is/fimc-lite.c
+++ b/drivers/media/platform/samsung/exynos4-is/fimc-lite.c
@@ -1052,6 +1052,7 @@ static int fimc_lite_subdev_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int fimc_lite_subdev_set_fmt(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *sd_state,
 				    struct v4l2_subdev_format *fmt)
 {
@@ -1113,6 +1114,7 @@ static int fimc_lite_subdev_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int fimc_lite_subdev_get_selection(struct v4l2_subdev *sd,
+					  const struct v4l2_subdev_client_info *ci,
 					  struct v4l2_subdev_state *sd_state,
 					  struct v4l2_subdev_selection *sel)
 {
@@ -1148,6 +1150,7 @@ static int fimc_lite_subdev_get_selection(struct v4l2_subdev *sd,
 }
 
 static int fimc_lite_subdev_set_selection(struct v4l2_subdev *sd,
+					  const struct v4l2_subdev_client_info *ci,
 					  struct v4l2_subdev_state *sd_state,
 					  struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/samsung/exynos4-is/mipi-csis.c b/drivers/media/platform/samsung/exynos4-is/mipi-csis.c
index 452880b5350c..da6e67a79c09 100644
--- a/drivers/media/platform/samsung/exynos4-is/mipi-csis.c
+++ b/drivers/media/platform/samsung/exynos4-is/mipi-csis.c
@@ -575,6 +575,7 @@ static struct v4l2_mbus_framefmt *__s5pcsis_get_format(
 }
 
 static int s5pcsis_set_fmt(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/samsung/s3c-camif/camif-capture.c b/drivers/media/platform/samsung/s3c-camif/camif-capture.c
index ed1a1d693293..98dfd394a1aa 100644
--- a/drivers/media/platform/samsung/s3c-camif/camif-capture.c
+++ b/drivers/media/platform/samsung/s3c-camif/camif-capture.c
@@ -1275,6 +1275,7 @@ static void __camif_subdev_try_format(struct camif_dev *camif,
 }
 
 static int s3c_camif_subdev_set_fmt(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *sd_state,
 				    struct v4l2_subdev_format *fmt)
 {
@@ -1342,6 +1343,7 @@ static int s3c_camif_subdev_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int s3c_camif_subdev_get_selection(struct v4l2_subdev *sd,
+					  const struct v4l2_subdev_client_info *ci,
 					  struct v4l2_subdev_state *sd_state,
 					  struct v4l2_subdev_selection *sel)
 {
@@ -1429,6 +1431,7 @@ static void __camif_try_crop(struct camif_dev *camif, struct v4l2_rect *r)
 }
 
 static int s3c_camif_subdev_set_selection(struct v4l2_subdev *sd,
+					  const struct v4l2_subdev_client_info *ci,
 					  struct v4l2_subdev_state *sd_state,
 					  struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/platform/samsung/s3c-camif/camif-core.c b/drivers/media/platform/samsung/s3c-camif/camif-core.c
index 221e3c447f36..2ab5a2008818 100644
--- a/drivers/media/platform/samsung/s3c-camif/camif-core.c
+++ b/drivers/media/platform/samsung/s3c-camif/camif-core.c
@@ -228,7 +228,7 @@ static int camif_register_sensor(struct camif_dev *camif)
 		return 0;
 
 	format.pad = CAMIF_SD_PAD_SINK;
-	v4l2_subdev_call(&camif->subdev, pad, set_fmt, NULL, &format);
+	v4l2_subdev_call(&camif->subdev, pad, set_fmt, NULL, NULL, &format);
 
 	v4l2_info(sd, "Initial format from sensor: %dx%d, %#x\n",
 		  format.format.width, format.format.height,
diff --git a/drivers/media/platform/st/stm32/stm32-csi.c b/drivers/media/platform/st/stm32/stm32-csi.c
index fd2b6dfbd44c..8a07032646ca 100644
--- a/drivers/media/platform/st/stm32/stm32-csi.c
+++ b/drivers/media/platform/st/stm32/stm32-csi.c
@@ -736,6 +736,7 @@ static int stm32_csi_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int stm32_csi_set_pad_format(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *state,
 				    struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/st/stm32/stm32-dcmi.c b/drivers/media/platform/st/stm32/stm32-dcmi.c
index eeb0199864dd..115675bd5a06 100644
--- a/drivers/media/platform/st/stm32/stm32-dcmi.c
+++ b/drivers/media/platform/st/stm32/stm32-dcmi.c
@@ -738,7 +738,7 @@ static int dcmi_pipeline_s_fmt(struct stm32_dcmi *dcmi,
 			format->format.width, format->format.height);
 
 		fmt.pad = pad->index;
-		ret = v4l2_subdev_call(subdev, pad, set_fmt, NULL, &fmt);
+		ret = v4l2_subdev_call(subdev, pad, set_fmt, NULL, NULL, &fmt);
 		if (ret < 0) {
 			dev_err(dcmi->dev, "%s: Failed to set format 0x%x %ux%u on \"%s\":%d pad (%d)\n",
 				__func__, format->format.code,
@@ -1224,7 +1224,8 @@ static int dcmi_set_sensor_format(struct stm32_dcmi *dcmi,
 	}
 
 	v4l2_fill_mbus_format(&format.format, pix, sd_fmt->mbus_code);
-	ret = v4l2_subdev_call_state_try(dcmi->source, pad, set_fmt, &format);
+	ret = v4l2_subdev_call_state_try(dcmi->source, pad, set_fmt, NULL,
+					 &format);
 	if (ret < 0)
 		return ret;
 
@@ -1246,7 +1247,7 @@ static int dcmi_get_sensor_bounds(struct stm32_dcmi *dcmi,
 	/*
 	 * Get sensor bounds first
 	 */
-	ret = v4l2_subdev_call(dcmi->source, pad, get_selection,
+	ret = v4l2_subdev_call(dcmi->source, pad, get_selection, NULL,
 			       NULL, &bounds);
 	if (!ret)
 		*r = bounds.r;
diff --git a/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-byteproc.c b/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-byteproc.c
index 0c7aeb0888f6..5da8b70750ff 100644
--- a/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-byteproc.c
+++ b/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-byteproc.c
@@ -264,6 +264,7 @@ dcmipp_byteproc_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int dcmipp_byteproc_set_fmt(struct v4l2_subdev *sd,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *sd_state,
 				   struct v4l2_subdev_format *fmt)
 {
@@ -298,6 +299,7 @@ static int dcmipp_byteproc_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int dcmipp_byteproc_get_selection(struct v4l2_subdev *sd,
+					 const struct v4l2_subdev_client_info *ci,
 					 struct v4l2_subdev_state *sd_state,
 					 struct v4l2_subdev_selection *s)
 {
@@ -351,6 +353,7 @@ static int dcmipp_byteproc_get_selection(struct v4l2_subdev *sd,
 }
 
 static int dcmipp_byteproc_set_selection(struct v4l2_subdev *sd,
+					 const struct v4l2_subdev_client_info *ci,
 					 struct v4l2_subdev_state *sd_state,
 					 struct v4l2_subdev_selection *s)
 {
diff --git a/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-input.c b/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-input.c
index 7d3f5857cbfe..b8aff94162ad 100644
--- a/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-input.c
+++ b/drivers/media/platform/st/stm32/stm32-dcmipp/dcmipp-input.c
@@ -269,6 +269,7 @@ static void dcmipp_inp_adjust_fmt(struct dcmipp_inp_device *inp,
 }
 
 static int dcmipp_inp_set_fmt(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/sunxi/sun4i-csi/sun4i_v4l2.c b/drivers/media/platform/sunxi/sun4i-csi/sun4i_v4l2.c
index 744197b0fccb..af35ece1d5cf 100644
--- a/drivers/media/platform/sunxi/sun4i-csi/sun4i_v4l2.c
+++ b/drivers/media/platform/sunxi/sun4i-csi/sun4i_v4l2.c
@@ -295,6 +295,7 @@ static int sun4i_csi_subdev_get_fmt(struct v4l2_subdev *subdev,
 }
 
 static int sun4i_csi_subdev_set_fmt(struct v4l2_subdev *subdev,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *sd_state,
 				    struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
index d006d9dd0170..b80b350bb6c3 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c
@@ -558,6 +558,7 @@ static int sun6i_csi_bridge_get_fmt(struct v4l2_subdev *subdev,
 }
 
 static int sun6i_csi_bridge_set_fmt(struct v4l2_subdev *subdev,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *state,
 				    struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c b/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
index b06cb73015cd..82ad1a42b978 100644
--- a/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
+++ b/drivers/media/platform/sunxi/sun6i-mipi-csi2/sun6i_mipi_csi2.c
@@ -362,6 +362,7 @@ static int sun6i_mipi_csi2_get_fmt(struct v4l2_subdev *subdev,
 }
 
 static int sun6i_mipi_csi2_set_fmt(struct v4l2_subdev *subdev,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *state,
 				   struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_mipi_csi2.c b/drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_mipi_csi2.c
index dbc51daa4fe3..e1b961246ced 100644
--- a/drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_mipi_csi2.c
+++ b/drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/sun8i_a83t_mipi_csi2.c
@@ -398,6 +398,7 @@ static int sun8i_a83t_mipi_csi2_get_fmt(struct v4l2_subdev *subdev,
 }
 
 static int sun8i_a83t_mipi_csi2_set_fmt(struct v4l2_subdev *subdev,
+					const struct v4l2_subdev_client_info *ci,
 					struct v4l2_subdev_state *state,
 					struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/synopsys/dw-mipi-csi2rx.c b/drivers/media/platform/synopsys/dw-mipi-csi2rx.c
index 41e48365167e..b7247d597c9c 100644
--- a/drivers/media/platform/synopsys/dw-mipi-csi2rx.c
+++ b/drivers/media/platform/synopsys/dw-mipi-csi2rx.c
@@ -479,6 +479,7 @@ dw_mipi_csi2rx_enum_mbus_code(struct v4l2_subdev *sd,
 }
 
 static int dw_mipi_csi2rx_set_fmt(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *state,
 				  struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/ti/am437x/am437x-vpfe.c b/drivers/media/platform/ti/am437x/am437x-vpfe.c
index 1ca559df7e59..3cd5ee69d3c4 100644
--- a/drivers/media/platform/ti/am437x/am437x-vpfe.c
+++ b/drivers/media/platform/ti/am437x/am437x-vpfe.c
@@ -1314,7 +1314,7 @@ static int __subdev_set_format(struct vpfe_device *vpfe,
 
 	*mbus_fmt = *fmt;
 
-	ret = v4l2_subdev_call(sd, pad, set_fmt, NULL, &sd_fmt);
+	ret = v4l2_subdev_call(sd, pad, set_fmt, NULL, NULL, &sd_fmt);
 	if (ret)
 		return ret;
 
diff --git a/drivers/media/platform/ti/cal/cal-camerarx.c b/drivers/media/platform/ti/cal/cal-camerarx.c
index 9ea1f3551d22..fd0d08b94716 100644
--- a/drivers/media/platform/ti/cal/cal-camerarx.c
+++ b/drivers/media/platform/ti/cal/cal-camerarx.c
@@ -763,6 +763,7 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
+				   const struct v4l2_subdev_client_info *ci,
 				   struct v4l2_subdev_state *state,
 				   struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/ti/cal/cal-video.c b/drivers/media/platform/ti/cal/cal-video.c
index d40e24ab1127..fa18d532730e 100644
--- a/drivers/media/platform/ti/cal/cal-video.c
+++ b/drivers/media/platform/ti/cal/cal-video.c
@@ -136,7 +136,7 @@ static int __subdev_set_format(struct cal_ctx *ctx,
 
 	*mbus_fmt = *fmt;
 
-	ret = v4l2_subdev_call_state_active(sd, pad, set_fmt, &sd_fmt);
+	ret = v4l2_subdev_call_state_active(sd, pad, set_fmt, NULL, &sd_fmt);
 	if (ret)
 		return ret;
 
@@ -281,7 +281,7 @@ static int cal_legacy_s_fmt_vid_cap(struct file *file, void *priv,
 	ctx->v_fmt.fmt.pix.field = sd_fmt.format.field;
 	cal_calc_format_size(ctx, fmtinfo, &ctx->v_fmt);
 
-	v4l2_subdev_call_state_active(sd, pad, set_fmt, &sd_fmt);
+	v4l2_subdev_call_state_active(sd, pad, set_fmt, NULL, &sd_fmt);
 
 	ctx->fmtinfo = fmtinfo;
 	*f = ctx->v_fmt;
diff --git a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
index 4769931b1930..0070cda59182 100644
--- a/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
+++ b/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c
@@ -1043,6 +1043,7 @@ static int ti_csi2rx_enum_mbus_code(struct v4l2_subdev *subdev,
 }
 
 static int ti_csi2rx_sd_set_fmt(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/platform/ti/omap3isp/ispccdc.c b/drivers/media/platform/ti/omap3isp/ispccdc.c
index 4708b6303493..f422d70ab6e4 100644
--- a/drivers/media/platform/ti/omap3isp/ispccdc.c
+++ b/drivers/media/platform/ti/omap3isp/ispccdc.c
@@ -2232,6 +2232,7 @@ static int ccdc_enum_frame_size(struct v4l2_subdev *sd,
  * Return 0 on success or a negative error code otherwise.
  */
 static int ccdc_get_selection(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_selection *sel)
 {
@@ -2276,6 +2277,7 @@ static int ccdc_get_selection(struct v4l2_subdev *sd,
  * Return 0 on success or a negative error code otherwise.
  */
 static int ccdc_set_selection(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_selection *sel)
 {
@@ -2346,6 +2348,7 @@ static int ccdc_get_format(struct v4l2_subdev *sd,
  * to the format type.
  */
 static int ccdc_set_format(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
@@ -2472,7 +2475,7 @@ static int ccdc_init_formats(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
 	format.format.code = MEDIA_BUS_FMT_SGRBG10_1X10;
 	format.format.width = 4096;
 	format.format.height = 4096;
-	ccdc_set_format(sd, fh ? fh->state : NULL, &format);
+	ccdc_set_format(sd, NULL, fh ? fh->state : NULL, &format);
 
 	return 0;
 }
diff --git a/drivers/media/platform/ti/omap3isp/ispccp2.c b/drivers/media/platform/ti/omap3isp/ispccp2.c
index d668111b44f4..c0da5cca9ddc 100644
--- a/drivers/media/platform/ti/omap3isp/ispccp2.c
+++ b/drivers/media/platform/ti/omap3isp/ispccp2.c
@@ -776,6 +776,7 @@ static int ccp2_get_format(struct v4l2_subdev *sd,
  * returns zero
  */
 static int ccp2_set_format(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
@@ -820,7 +821,7 @@ static int ccp2_init_formats(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
 	format.format.code = MEDIA_BUS_FMT_SGRBG10_1X10;
 	format.format.width = 4096;
 	format.format.height = 4096;
-	ccp2_set_format(sd, fh ? fh->state : NULL, &format);
+	ccp2_set_format(sd, NULL, fh ? fh->state : NULL, &format);
 
 	return 0;
 }
diff --git a/drivers/media/platform/ti/omap3isp/ispcsi2.c b/drivers/media/platform/ti/omap3isp/ispcsi2.c
index f227042b61b6..8fc2948d654c 100644
--- a/drivers/media/platform/ti/omap3isp/ispcsi2.c
+++ b/drivers/media/platform/ti/omap3isp/ispcsi2.c
@@ -994,6 +994,7 @@ static int csi2_get_format(struct v4l2_subdev *sd,
  * return -EINVAL or zero on success
  */
 static int csi2_set_format(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
@@ -1038,7 +1039,7 @@ static int csi2_init_formats(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
 	format.format.code = MEDIA_BUS_FMT_SGRBG10_1X10;
 	format.format.width = 4096;
 	format.format.height = 4096;
-	csi2_set_format(sd, fh ? fh->state : NULL, &format);
+	csi2_set_format(sd, NULL, fh ? fh->state : NULL, &format);
 
 	return 0;
 }
diff --git a/drivers/media/platform/ti/omap3isp/isppreview.c b/drivers/media/platform/ti/omap3isp/isppreview.c
index 3f3b5bd9cdc7..ac5025ce4315 100644
--- a/drivers/media/platform/ti/omap3isp/isppreview.c
+++ b/drivers/media/platform/ti/omap3isp/isppreview.c
@@ -1925,6 +1925,7 @@ static int preview_enum_frame_size(struct v4l2_subdev *sd,
  * Return 0 on success or a negative error code otherwise.
  */
 static int preview_get_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -1968,6 +1969,7 @@ static int preview_get_selection(struct v4l2_subdev *sd,
  * Return 0 on success or a negative error code otherwise.
  */
 static int preview_set_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -2035,6 +2037,7 @@ static int preview_get_format(struct v4l2_subdev *sd,
  * return -EINVAL or zero on success
  */
 static int preview_set_format(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
@@ -2090,7 +2093,7 @@ static int preview_init_formats(struct v4l2_subdev *sd,
 	format.format.code = MEDIA_BUS_FMT_SGRBG10_1X10;
 	format.format.width = 4096;
 	format.format.height = 4096;
-	preview_set_format(sd, fh ? fh->state : NULL, &format);
+	preview_set_format(sd, NULL, fh ? fh->state : NULL, &format);
 
 	return 0;
 }
diff --git a/drivers/media/platform/ti/omap3isp/ispresizer.c b/drivers/media/platform/ti/omap3isp/ispresizer.c
index ad0127f5b5cb..874aa451a2d0 100644
--- a/drivers/media/platform/ti/omap3isp/ispresizer.c
+++ b/drivers/media/platform/ti/omap3isp/ispresizer.c
@@ -1222,6 +1222,7 @@ static void resizer_try_crop(const struct v4l2_mbus_framefmt *sink,
  * Return 0 on success or a negative error code otherwise.
  */
 static int resizer_get_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -1275,6 +1276,7 @@ static int resizer_get_selection(struct v4l2_subdev *sd,
  * Return 0 on success or a negative error code otherwise.
  */
 static int resizer_set_selection(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_selection *sel)
 {
@@ -1500,6 +1502,7 @@ static int resizer_get_format(struct v4l2_subdev *sd,
  * return -EINVAL or zero on success
  */
 static int resizer_set_format(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_format *fmt)
 {
@@ -1577,7 +1580,7 @@ static int resizer_init_formats(struct v4l2_subdev *sd,
 	format.format.code = MEDIA_BUS_FMT_YUYV8_1X16;
 	format.format.width = 4096;
 	format.format.height = 4096;
-	resizer_set_format(sd, fh ? fh->state : NULL, &format);
+	resizer_set_format(sd, NULL, fh ? fh->state : NULL, &format);
 
 	return 0;
 }
diff --git a/drivers/media/platform/ti/omap3isp/ispvideo.c b/drivers/media/platform/ti/omap3isp/ispvideo.c
index b946c8087c77..cd324c730318 100644
--- a/drivers/media/platform/ti/omap3isp/ispvideo.c
+++ b/drivers/media/platform/ti/omap3isp/ispvideo.c
@@ -839,7 +839,7 @@ isp_video_get_selection(struct file *file, void *fh, struct v4l2_selection *sel)
 	 * implemented.
 	 */
 	sdsel.pad = pad;
-	ret = v4l2_subdev_call(subdev, pad, get_selection, NULL, &sdsel);
+	ret = v4l2_subdev_call(subdev, pad, get_selection, NULL, NULL, &sdsel);
 	if (!ret)
 		sel->r = sdsel.r;
 	if (ret != -ENOIOCTLCMD)
@@ -890,7 +890,7 @@ isp_video_set_selection(struct file *file, void *fh, struct v4l2_selection *sel)
 
 	sdsel.pad = pad;
 	mutex_lock(&video->mutex);
-	ret = v4l2_subdev_call(subdev, pad, set_selection, NULL, &sdsel);
+	ret = v4l2_subdev_call(subdev, pad, set_selection, NULL, NULL, &sdsel);
 	mutex_unlock(&video->mutex);
 	if (!ret)
 		sel->r = sdsel.r;
diff --git a/drivers/media/platform/ti/vpe/vip.c b/drivers/media/platform/ti/vpe/vip.c
index cb0a5a07a3d4..baa4c8cc5447 100644
--- a/drivers/media/platform/ti/vpe/vip.c
+++ b/drivers/media/platform/ti/vpe/vip.c
@@ -1731,7 +1731,7 @@ static int vip_s_fmt_vid_cap(struct file *file, void *priv,
 
 	sfmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	sfmt.pad = 0;
-	ret = v4l2_subdev_call(port->subdev, pad, set_fmt, NULL, &sfmt);
+	ret = v4l2_subdev_call(port->subdev, pad, set_fmt, NULL, NULL, &sfmt);
 	if (ret) {
 		v4l2_dbg(1, debug, &dev->v4l2_dev, "set_fmt failed in subdev\n");
 		return ret;
@@ -2598,7 +2598,7 @@ static int vip_init_port(struct vip_port *port)
 		mbus_fmt->code = fmt->code;
 		sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 		sd_fmt.pad = 0;
-		ret = v4l2_subdev_call(port->subdev, pad, set_fmt,
+		ret = v4l2_subdev_call(port->subdev, pad, set_fmt, NULL,
 				       NULL, &sd_fmt);
 		if (ret)
 			v4l2_dbg(1, debug, &dev->v4l2_dev, "init_port set_fmt failed in subdev: (%d)\n",
diff --git a/drivers/media/platform/via/via-camera.c b/drivers/media/platform/via/via-camera.c
index 1b81acba7da0..6a6e10c6f649 100644
--- a/drivers/media/platform/via/via-camera.c
+++ b/drivers/media/platform/via/via-camera.c
@@ -249,7 +249,7 @@ static int viacam_configure_sensor(struct via_camera *cam)
 	v4l2_fill_mbus_format(&format.format, &cam->sensor_format, cam->mbus_code);
 	ret = sensor_call(cam, core, init, 0);
 	if (ret == 0)
-		ret = sensor_call(cam, pad, set_fmt, NULL, &format);
+		ret = sensor_call(cam, pad, set_fmt, NULL, NULL, &format);
 	/*
 	 * OV7670 does weird things if flip is set *before* format...
 	 */
@@ -842,7 +842,7 @@ static int viacam_do_try_fmt(struct via_camera *cam,
 	upix->pixelformat = f->pixelformat;
 	viacam_fmt_pre(upix, spix);
 	v4l2_fill_mbus_format(&format.format, spix, f->mbus_code);
-	ret = sensor_call(cam, pad, set_fmt, &pad_state, &format);
+	ret = sensor_call(cam, pad, set_fmt, NULL, &pad_state, &format);
 	v4l2_fill_pix_format(spix, &format.format);
 	viacam_fmt_post(upix, spix);
 	return ret;
diff --git a/drivers/media/platform/video-mux.c b/drivers/media/platform/video-mux.c
index cba34893258a..b253b260b605 100644
--- a/drivers/media/platform/video-mux.c
+++ b/drivers/media/platform/video-mux.c
@@ -146,6 +146,7 @@ static const struct v4l2_subdev_video_ops video_mux_subdev_video_ops = {
 };
 
 static int video_mux_set_format(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *sdformat)
 {
diff --git a/drivers/media/platform/xilinx/xilinx-csi2rxss.c b/drivers/media/platform/xilinx/xilinx-csi2rxss.c
index 146131b8f37e..2f6f3af4e492 100644
--- a/drivers/media/platform/xilinx/xilinx-csi2rxss.c
+++ b/drivers/media/platform/xilinx/xilinx-csi2rxss.c
@@ -694,6 +694,7 @@ static int xcsi2rxss_get_format(struct v4l2_subdev *sd,
 }
 
 static int xcsi2rxss_set_format(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/platform/xilinx/xilinx-tpg.c b/drivers/media/platform/xilinx/xilinx-tpg.c
index 7deec6e37edc..bf39ddb9c296 100644
--- a/drivers/media/platform/xilinx/xilinx-tpg.c
+++ b/drivers/media/platform/xilinx/xilinx-tpg.c
@@ -278,6 +278,7 @@ static int xtpg_get_format(struct v4l2_subdev *subdev,
 }
 
 static int xtpg_set_format(struct v4l2_subdev *subdev,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/test-drivers/vimc/vimc-debayer.c b/drivers/media/test-drivers/vimc/vimc-debayer.c
index 0c2e715a8a16..cfda447a5b76 100644
--- a/drivers/media/test-drivers/vimc/vimc-debayer.c
+++ b/drivers/media/test-drivers/vimc/vimc-debayer.c
@@ -240,6 +240,7 @@ static void vimc_debayer_adjust_sink_fmt(struct v4l2_mbus_framefmt *fmt)
 }
 
 static int vimc_debayer_set_fmt(struct v4l2_subdev *sd,
+				const struct v4l2_subdev_client_info *ci,
 				struct v4l2_subdev_state *sd_state,
 				struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/test-drivers/vimc/vimc-scaler.c b/drivers/media/test-drivers/vimc/vimc-scaler.c
index e2037c67e423..d8dc65d33f62 100644
--- a/drivers/media/test-drivers/vimc/vimc-scaler.c
+++ b/drivers/media/test-drivers/vimc/vimc-scaler.c
@@ -140,6 +140,7 @@ static int vimc_scaler_enum_frame_size(struct v4l2_subdev *sd,
 }
 
 static int vimc_scaler_set_fmt(struct v4l2_subdev *sd,
+			    const struct v4l2_subdev_client_info *ci,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *format)
 {
@@ -204,6 +205,7 @@ static int vimc_scaler_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int vimc_scaler_get_selection(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *sd_state,
 				  struct v4l2_subdev_selection *sel)
 {
@@ -239,6 +241,7 @@ static void vimc_scaler_adjust_sink_crop(struct v4l2_rect *r,
 }
 
 static int vimc_scaler_set_selection(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *sd_state,
 				  struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/media/test-drivers/vimc/vimc-sensor.c b/drivers/media/test-drivers/vimc/vimc-sensor.c
index 5deebcc78a33..c58012093142 100644
--- a/drivers/media/test-drivers/vimc/vimc-sensor.c
+++ b/drivers/media/test-drivers/vimc/vimc-sensor.c
@@ -152,6 +152,7 @@ static u32 vimc_calc_vblank(u32 width, u32 height,
 }
 
 static int vimc_sensor_set_fmt(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/media/usb/cx231xx/cx231xx-417.c b/drivers/media/usb/cx231xx/cx231xx-417.c
index c695a97e202b..d0955ea18195 100644
--- a/drivers/media/usb/cx231xx/cx231xx-417.c
+++ b/drivers/media/usb/cx231xx/cx231xx-417.c
@@ -1666,7 +1666,7 @@ static int cx231xx_s_video_encoding(struct cx2341x_handler *cxhdl, u32 val)
 	format.format.width = cxhdl->width / (is_mpeg1 ? 2 : 1);
 	format.format.height = cxhdl->height;
 	format.format.code = MEDIA_BUS_FMT_FIXED;
-	v4l2_subdev_call(dev->sd_cx25840, pad, set_fmt, NULL, &format);
+	v4l2_subdev_call(dev->sd_cx25840, pad, set_fmt, NULL, NULL, &format);
 	return 0;
 }
 
diff --git a/drivers/media/usb/cx231xx/cx231xx-video.c b/drivers/media/usb/cx231xx/cx231xx-video.c
index 2cd4e333bc4b..ec9306a940a8 100644
--- a/drivers/media/usb/cx231xx/cx231xx-video.c
+++ b/drivers/media/usb/cx231xx/cx231xx-video.c
@@ -909,7 +909,7 @@ static int vidioc_s_fmt_vid_cap(struct file *file, void *priv,
 	dev->format = format_by_fourcc(f->fmt.pix.pixelformat);
 
 	v4l2_fill_mbus_format(&format.format, &f->fmt.pix, MEDIA_BUS_FMT_FIXED);
-	call_all(dev, pad, set_fmt, NULL, &format);
+	call_all(dev, pad, set_fmt, NULL, NULL, &format);
 	v4l2_fill_pix_format(&f->fmt.pix, &format.format);
 
 	return rc;
@@ -950,7 +950,7 @@ static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id norm)
 	format.format.code = MEDIA_BUS_FMT_FIXED;
 	format.format.width = dev->width;
 	format.format.height = dev->height;
-	call_all(dev, pad, set_fmt, NULL, &format);
+	call_all(dev, pad, set_fmt, NULL, NULL, &format);
 
 	/* do mode control overrides */
 	cx231xx_do_mode_ctrl_overrides(dev);
diff --git a/drivers/media/usb/dvb-usb/cxusb-analog.c b/drivers/media/usb/dvb-usb/cxusb-analog.c
index 3bbee1fcbc8d..8c7d0888c721 100644
--- a/drivers/media/usb/dvb-usb/cxusb-analog.c
+++ b/drivers/media/usb/dvb-usb/cxusb-analog.c
@@ -1031,7 +1031,8 @@ static int cxusb_medion_try_s_fmt_vid_cap(struct file *file,
 	subfmt.format.field = field;
 	subfmt.format.colorspace = V4L2_COLORSPACE_SMPTE170M;
 
-	ret = v4l2_subdev_call(cxdev->cx25840, pad, set_fmt, NULL, &subfmt);
+	ret = v4l2_subdev_call(cxdev->cx25840, pad, set_fmt, NULL, NULL,
+			       &subfmt);
 	if (ret != 0)
 		return ret;
 
@@ -1513,7 +1514,8 @@ int cxusb_medion_analog_init(struct dvb_usb_device *dvbdev)
 	subfmt.format.field = V4L2_FIELD_SEQ_TB;
 	subfmt.format.colorspace = V4L2_COLORSPACE_SMPTE170M;
 
-	ret = v4l2_subdev_call(cxdev->cx25840, pad, set_fmt, NULL, &subfmt);
+	ret = v4l2_subdev_call(cxdev->cx25840, pad, set_fmt, NULL, NULL,
+			       &subfmt);
 	if (ret != 0)
 		dev_warn(&dvbdev->udev->dev,
 			 "cx25840 format set failed (%d)\n", ret);
diff --git a/drivers/media/usb/em28xx/em28xx-camera.c b/drivers/media/usb/em28xx/em28xx-camera.c
index b5f58dc6dd0f..bc55ad66658c 100644
--- a/drivers/media/usb/em28xx/em28xx-camera.c
+++ b/drivers/media/usb/em28xx/em28xx-camera.c
@@ -392,7 +392,7 @@ int em28xx_init_camera(struct em28xx *dev)
 		format.format.code = MEDIA_BUS_FMT_YUYV8_2X8;
 		format.format.width = 640;
 		format.format.height = 480;
-		v4l2_subdev_call(subdev, pad, set_fmt, NULL, &format);
+		v4l2_subdev_call(subdev, pad, set_fmt, NULL, NULL, &format);
 
 		/* NOTE: for UXGA=1600x1200 switch to 12MHz */
 		dev->board.xclk = EM28XX_XCLK_FREQUENCY_24MHZ;
diff --git a/drivers/media/usb/go7007/go7007-v4l2.c b/drivers/media/usb/go7007/go7007-v4l2.c
index 2087ffcb85a5..86b853e09ecb 100644
--- a/drivers/media/usb/go7007/go7007-v4l2.c
+++ b/drivers/media/usb/go7007/go7007-v4l2.c
@@ -252,7 +252,7 @@ static int set_capture_size(struct go7007 *go, struct v4l2_format *fmt, int try)
 		go->encoder_h_halve = 0;
 		go->encoder_v_halve = 0;
 		go->encoder_subsample = 0;
-		call_all(&go->v4l2_dev, pad, set_fmt, NULL, &format);
+		call_all(&go->v4l2_dev, pad, set_fmt, NULL, NULL, &format);
 	} else {
 		if (width <= sensor_width / 4) {
 			go->encoder_h_halve = 1;
diff --git a/drivers/media/usb/go7007/s2250-board.c b/drivers/media/usb/go7007/s2250-board.c
index 0901d79e827d..a67e995f9628 100644
--- a/drivers/media/usb/go7007/s2250-board.c
+++ b/drivers/media/usb/go7007/s2250-board.c
@@ -398,6 +398,7 @@ static int s2250_s_ctrl(struct v4l2_ctrl *ctrl)
 }
 
 static int s2250_set_fmt(struct v4l2_subdev *sd,
+		const struct v4l2_subdev_client_info *ci,
 		struct v4l2_subdev_state *sd_state,
 		struct v4l2_subdev_format *format)
 {
diff --git a/drivers/media/usb/pvrusb2/pvrusb2-hdw.c b/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
index 3c270ef00752..21a0dfbd455e 100644
--- a/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
+++ b/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
@@ -2924,7 +2924,7 @@ static void pvr2_subdev_update(struct pvr2_hdw *hdw)
 		format.format.code = MEDIA_BUS_FMT_FIXED;
 		pvr2_trace(PVR2_TRACE_CHIPS, "subdev v4l2 set_size(%dx%d)",
 			   format.format.width, format.format.height);
-		v4l2_device_call_all(&hdw->v4l2_dev, 0, pad, set_fmt,
+		v4l2_device_call_all(&hdw->v4l2_dev, 0, pad, set_fmt, NULL,
 				     NULL, &format);
 	}
 
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index c1e1fb9d6773..b384c338940b 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -250,11 +250,12 @@ static int call_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int call_set_fmt(struct v4l2_subdev *sd,
+			const struct v4l2_subdev_client_info *ci,
 			struct v4l2_subdev_state *state,
 			struct v4l2_subdev_format *format)
 {
 	return check_format(sd, state, format) ? :
-	       sd->ops->pad->set_fmt(sd, state, format);
+		sd->ops->pad->set_fmt(sd, ci, state, format);
 }
 
 static int call_enum_mbus_code(struct v4l2_subdev *sd,
@@ -305,19 +306,21 @@ static inline int check_selection(struct v4l2_subdev *sd,
 }
 
 static int call_get_selection(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *state,
 			      struct v4l2_subdev_selection *sel)
 {
 	return check_selection(sd, state, sel) ? :
-	       sd->ops->pad->get_selection(sd, state, sel);
+		sd->ops->pad->get_selection(sd, ci, state, sel);
 }
 
 static int call_set_selection(struct v4l2_subdev *sd,
+			      const struct v4l2_subdev_client_info *ci,
 			      struct v4l2_subdev_state *state,
 			      struct v4l2_subdev_selection *sel)
 {
 	return check_selection(sd, state, sel) ? :
-	       sd->ops->pad->set_selection(sd, state, sel);
+		sd->ops->pad->set_selection(sd, ci, state, sel);
 }
 
 static inline int check_frame_interval(struct v4l2_subdev *sd,
@@ -546,6 +549,21 @@ static int call_s_stream(struct v4l2_subdev *sd, int enable)
 			v4l2_subdev_unlock_state(state);                   \
 		return ret;                                                \
 	}
+#define DEFINE_CI_STATE_WRAPPER(f, arg_type)                               \
+	static int call_##f##_state(struct v4l2_subdev *sd,		   \
+				    const struct v4l2_subdev_client_info *ci, \
+				    struct v4l2_subdev_state *_state,      \
+				    arg_type *arg) \
+	{                                                                  \
+		struct v4l2_subdev_state *state = _state;                  \
+		int ret;                                                   \
+		if (!_state)                                               \
+			state = v4l2_subdev_lock_and_get_active_state(sd); \
+		ret = call_##f(sd, ci, state, arg);			   \
+		if (!_state && state)                                      \
+			v4l2_subdev_unlock_state(state);                   \
+		return ret;                                                \
+	}
 
 #else /* CONFIG_MEDIA_CONTROLLER */
 
@@ -557,15 +575,24 @@ static int call_s_stream(struct v4l2_subdev *sd, int enable)
 		return call_##f(sd, state, arg);                     \
 	}
 
+#define DEFINE_CI_STATE_WRAPPER(f, arg_type)                            \
+	static int call_##f##_state(struct v4l2_subdev *sd,          \
+				    const struct v4l2_subdev_client_info *ci, \
+				    struct v4l2_subdev_state *state, \
+				    arg_type *arg)                   \
+	{                                                            \
+		return call_##f(sd, ci, state, arg);		     \
+	}
+
 #endif /* CONFIG_MEDIA_CONTROLLER */
 
 DEFINE_STATE_WRAPPER(get_fmt, struct v4l2_subdev_format);
-DEFINE_STATE_WRAPPER(set_fmt, struct v4l2_subdev_format);
+DEFINE_CI_STATE_WRAPPER(set_fmt, struct v4l2_subdev_format);
 DEFINE_STATE_WRAPPER(enum_mbus_code, struct v4l2_subdev_mbus_code_enum);
 DEFINE_STATE_WRAPPER(enum_frame_size, struct v4l2_subdev_frame_size_enum);
 DEFINE_STATE_WRAPPER(enum_frame_interval, struct v4l2_subdev_frame_interval_enum);
-DEFINE_STATE_WRAPPER(get_selection, struct v4l2_subdev_selection);
-DEFINE_STATE_WRAPPER(set_selection, struct v4l2_subdev_selection);
+DEFINE_CI_STATE_WRAPPER(get_selection, struct v4l2_subdev_selection);
+DEFINE_CI_STATE_WRAPPER(set_selection, struct v4l2_subdev_selection);
 
 static const struct v4l2_subdev_pad_ops v4l2_subdev_call_pad_wrappers = {
 	.get_fmt		= call_get_fmt_state,
@@ -840,7 +867,8 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 
 		memset(format->reserved, 0, sizeof(format->reserved));
 		memset(format->format.reserved, 0, sizeof(format->format.reserved));
-		return v4l2_subdev_call(sd, pad, set_fmt, state, format);
+		return v4l2_subdev_call(sd, pad, set_fmt, &subdev_fh->ci, state,
+					format);
 	}
 
 	case VIDIOC_SUBDEV_G_CROP: {
@@ -857,8 +885,8 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 		sel.stream = crop->stream;
 		sel.target = V4L2_SEL_TGT_CROP;
 
-		rval = v4l2_subdev_call(
-			sd, pad, get_selection, state, &sel);
+		rval = v4l2_subdev_call(sd, pad, get_selection, &subdev_fh->ci,
+					state, &sel);
 
 		crop->rect = sel.r;
 
@@ -883,8 +911,8 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 		sel.target = V4L2_SEL_TGT_CROP;
 		sel.r = crop->rect;
 
-		rval = v4l2_subdev_call(
-			sd, pad, set_selection, state, &sel);
+		rval = v4l2_subdev_call(sd, pad, set_selection, &subdev_fh->ci,
+					state, &sel);
 
 		crop->rect = sel.r;
 
@@ -954,8 +982,8 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 			sel->stream = 0;
 
 		memset(sel->reserved, 0, sizeof(sel->reserved));
-		return v4l2_subdev_call(
-			sd, pad, get_selection, state, sel);
+		return v4l2_subdev_call(sd, pad, get_selection, &subdev_fh->ci,
+					state, sel);
 	}
 
 	case VIDIOC_SUBDEV_S_SELECTION: {
@@ -968,8 +996,8 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 			sel->stream = 0;
 
 		memset(sel->reserved, 0, sizeof(sel->reserved));
-		return v4l2_subdev_call(
-			sd, pad, set_selection, state, sel);
+		return v4l2_subdev_call(sd, pad, set_selection, &subdev_fh->ci,
+					state, sel);
 	}
 
 	case VIDIOC_G_EDID: {
@@ -1938,7 +1966,8 @@ v4l2_subdev_init_stream_configs(struct v4l2_subdev_stream_configs *stream_config
 	return 0;
 }
 
-int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
+int v4l2_subdev_get_fmt(struct v4l2_subdev *sd,
+			struct v4l2_subdev_state *state,
 			struct v4l2_subdev_format *format)
 {
 	struct v4l2_mbus_framefmt *fmt;
diff --git a/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c b/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c
index f9cc5f45fe00..38f69582c174 100644
--- a/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c
+++ b/drivers/staging/media/atomisp/i2c/atomisp-gc2235.c
@@ -520,6 +520,7 @@ static int gc2235_startup(struct v4l2_subdev *sd)
 }
 
 static int gc2235_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
diff --git a/drivers/staging/media/atomisp/i2c/atomisp-ov2722.c b/drivers/staging/media/atomisp/i2c/atomisp-ov2722.c
index 2c41c496daa6..e3137fbc5802 100644
--- a/drivers/staging/media/atomisp/i2c/atomisp-ov2722.c
+++ b/drivers/staging/media/atomisp/i2c/atomisp-ov2722.c
@@ -623,6 +623,7 @@ static int ov2722_startup(struct v4l2_subdev *sd)
 }
 
 static int ov2722_set_fmt(struct v4l2_subdev *sd,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *format)
 {
diff --git a/drivers/staging/media/atomisp/pci/atomisp_cmd.c b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
index 6cd500d9fd26..52c71cd86dec 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_cmd.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_cmd.c
@@ -3745,14 +3745,16 @@ static int atomisp_set_sensor_crop_and_fmt(struct atomisp_device *isp,
 	sel.r.left = ((input->native_rect.width - sel.r.width) / 2) & ~1;
 	sel.r.top = ((input->native_rect.height - sel.r.height) / 2) & ~1;
 
-	ret = v4l2_subdev_call(input->sensor, pad, set_selection, sd_state, &sel);
+	ret = v4l2_subdev_call(input->sensor, pad, set_selection, NULL,
+			       sd_state, &sel);
 	if (ret)
 		dev_err(isp->dev, "Error setting crop to (%d,%d)/%ux%u: %d\n",
 			sel.r.left, sel.r.top, sel.r.width, sel.r.height, ret);
 
 set_fmt:
 	if (ret == 0) {
-		ret = v4l2_subdev_call(input->sensor, pad, set_fmt, sd_state, &format);
+		ret = v4l2_subdev_call(input->sensor, pad, set_fmt, NULL,
+				       sd_state, &format);
 		dev_dbg(isp->dev, "Set sensor format ret: %d size %dx%d\n",
 			ret, format.format.width, format.format.height);
 	}
@@ -3765,13 +3767,16 @@ static int atomisp_set_sensor_crop_and_fmt(struct atomisp_device *isp,
 		sd_state = v4l2_subdev_lock_and_get_active_state(input->sensor_isp);
 
 		format.pad = SENSOR_ISP_PAD_SINK;
-		ret = v4l2_subdev_call(input->sensor_isp, pad, set_fmt, sd_state, &format);
+		ret = v4l2_subdev_call(input->sensor_isp, pad, set_fmt, NULL,
+				       sd_state, &format);
 		dev_dbg(isp->dev, "Set sensor ISP sink format ret: %d size %dx%d\n",
 			ret, format.format.width, format.format.height);
 
 		if (ret == 0) {
 			format.pad = SENSOR_ISP_PAD_SOURCE;
-			ret = v4l2_subdev_call(input->sensor_isp, pad, set_fmt, sd_state, &format);
+			ret = v4l2_subdev_call(input->sensor_isp, pad,
+					       set_fmt, NULL, sd_state,
+					       &format);
 			dev_dbg(isp->dev, "Set sensor ISP source format ret: %d size %dx%d\n",
 				ret, format.format.width, format.format.height);
 		}
@@ -3783,7 +3788,8 @@ static int atomisp_set_sensor_crop_and_fmt(struct atomisp_device *isp,
 	/* Propagate new fmt to CSI port */
 	if (ret == 0 && which == V4L2_SUBDEV_FORMAT_ACTIVE) {
 		format.pad = CSI2_PAD_SINK;
-		ret = v4l2_subdev_call(input->csi_port, pad, set_fmt, NULL, &format);
+		ret = v4l2_subdev_call(input->csi_port, pad, set_fmt, NULL,
+				       NULL, &format);
 		if (ret)
 			return ret;
 	}
diff --git a/drivers/staging/media/atomisp/pci/atomisp_csi2.c b/drivers/staging/media/atomisp/pci/atomisp_csi2.c
index 95b9113d75e9..71df4ef629c2 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_csi2.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_csi2.c
@@ -123,6 +123,7 @@ int atomisp_csi2_set_ffmt(struct v4l2_subdev *sd,
  * return -EINVAL or zero on success
  */
 static int csi2_set_format(struct v4l2_subdev *sd,
+			   const struct v4l2_subdev_client_info *ci,
 			   struct v4l2_subdev_state *sd_state,
 			   struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/staging/media/atomisp/pci/atomisp_subdev.c b/drivers/staging/media/atomisp/pci/atomisp_subdev.c
index 9de9cd884d99..bdd2d9b18da2 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_subdev.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_subdev.c
@@ -259,6 +259,7 @@ static void isp_get_fmt_rect(struct v4l2_subdev *sd,
 }
 
 static int isp_subdev_get_selection(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *sd_state,
 				    struct v4l2_subdev_selection *sel)
 {
@@ -452,6 +453,7 @@ int atomisp_subdev_set_selection(struct v4l2_subdev *sd,
 }
 
 static int isp_subdev_set_selection(struct v4l2_subdev *sd,
+				    const struct v4l2_subdev_client_info *ci,
 				    struct v4l2_subdev_state *sd_state,
 				    struct v4l2_subdev_selection *sel)
 {
@@ -555,6 +557,7 @@ static int isp_subdev_get_format(struct v4l2_subdev *sd,
  * to the format type.
  */
 static int isp_subdev_set_format(struct v4l2_subdev *sd,
+				 const struct v4l2_subdev_client_info *ci,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/staging/media/atomisp/pci/atomisp_v4l2.c b/drivers/staging/media/atomisp/pci/atomisp_v4l2.c
index 812230397409..476fe143a591 100644
--- a/drivers/staging/media/atomisp/pci/atomisp_v4l2.c
+++ b/drivers/staging/media/atomisp/pci/atomisp_v4l2.c
@@ -913,7 +913,7 @@ static void atomisp_init_sensor(struct atomisp_input_subdev *input)
 
 	sel.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	sel.target = V4L2_SEL_TGT_NATIVE_SIZE;
-	err = v4l2_subdev_call(input->sensor, pad, get_selection,
+	err = v4l2_subdev_call(input->sensor, pad, get_selection, NULL,
 			       act_sd_state, &sel);
 	if (err)
 		goto unlock_act_sd_state;
@@ -922,7 +922,7 @@ static void atomisp_init_sensor(struct atomisp_input_subdev *input)
 
 	sel.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	sel.target = V4L2_SEL_TGT_CROP_DEFAULT;
-	err = v4l2_subdev_call(input->sensor, pad, get_selection,
+	err = v4l2_subdev_call(input->sensor, pad, get_selection, NULL,
 			       act_sd_state, &sel);
 	if (err)
 		goto unlock_act_sd_state;
@@ -968,7 +968,7 @@ static void atomisp_init_sensor(struct atomisp_input_subdev *input)
 	if (!input->sensor->state_lock)
 		v4l2_subdev_lock_state(input->try_sd_state);
 
-	err = v4l2_subdev_call(input->sensor, pad, set_selection,
+	err = v4l2_subdev_call(input->sensor, pad, set_selection, NULL,
 			       input->try_sd_state, &sel);
 
 	if (!input->sensor->state_lock)
@@ -980,7 +980,7 @@ static void atomisp_init_sensor(struct atomisp_input_subdev *input)
 	sel.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	sel.target = V4L2_SEL_TGT_CROP;
 	sel.r = input->native_rect;
-	err = v4l2_subdev_call(input->sensor, pad, set_selection,
+	err = v4l2_subdev_call(input->sensor, pad, set_selection, NULL,
 			       act_sd_state, &sel);
 	if (err)
 		goto unlock_act_sd_state;
diff --git a/drivers/staging/media/imx/imx-ic-prp.c b/drivers/staging/media/imx/imx-ic-prp.c
index 2b80d54006b3..46ac33821f89 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -151,6 +151,7 @@ static int prp_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int prp_set_fmt(struct v4l2_subdev *sd,
+		       const struct v4l2_subdev_client_info *ci,
 		       struct v4l2_subdev_state *sd_state,
 		       struct v4l2_subdev_format *sdformat)
 {
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 77360bfe081a..5ee59b418a5e 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -919,6 +919,7 @@ static void prp_try_fmt(struct prp_priv *priv,
 }
 
 static int prp_set_fmt(struct v4l2_subdev *sd,
+		       const struct v4l2_subdev_client_info *ci,
 		       struct v4l2_subdev_state *sd_state,
 		       struct v4l2_subdev_format *sdformat)
 {
diff --git a/drivers/staging/media/imx/imx-media-csi.c b/drivers/staging/media/imx/imx-media-csi.c
index ef22a083f8eb..43c637b333f2 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1525,6 +1525,7 @@ static void csi_try_fmt(struct csi_priv *priv,
 }
 
 static int csi_set_fmt(struct v4l2_subdev *sd,
+		       const struct v4l2_subdev_client_info *ci,
 		       struct v4l2_subdev_state *sd_state,
 		       struct v4l2_subdev_format *sdformat)
 {
@@ -1593,6 +1594,7 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
 }
 
 static int csi_get_selection(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel)
 {
@@ -1657,6 +1659,7 @@ static int csi_set_scale(u32 *compose, u32 crop, u32 flags)
 }
 
 static int csi_set_selection(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *sd_state,
 			     struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/staging/media/imx/imx-media-vdic.c b/drivers/staging/media/imx/imx-media-vdic.c
index 58f1112e28e5..8dcbb6fe7649 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -566,6 +566,7 @@ static void vdic_try_fmt(struct vdic_priv *priv,
 }
 
 static int vdic_set_fmt(struct v4l2_subdev *sd,
+			const struct v4l2_subdev_client_info *ci,
 			struct v4l2_subdev_state *sd_state,
 			struct v4l2_subdev_format *sdformat)
 {
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c b/drivers/staging/media/imx/imx6-mipi-csi2.c
index 211f67fb92b5..f6cbf0310e89 100644
--- a/drivers/staging/media/imx/imx6-mipi-csi2.c
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -525,6 +525,7 @@ static int csi2_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int csi2_set_fmt(struct v4l2_subdev *sd,
+			const struct v4l2_subdev_client_info *ci,
 			struct v4l2_subdev_state *sd_state,
 			struct v4l2_subdev_format *sdformat)
 {
diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c b/drivers/staging/media/ipu3/ipu3-v4l2.c
index 2f6041d342f4..61fdefe78c3d 100644
--- a/drivers/staging/media/ipu3/ipu3-v4l2.c
+++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
@@ -145,6 +145,7 @@ static int imgu_subdev_get_fmt(struct v4l2_subdev *sd,
 }
 
 static int imgu_subdev_set_fmt(struct v4l2_subdev *sd,
+			       const struct v4l2_subdev_client_info *ci,
 			       struct v4l2_subdev_state *sd_state,
 			       struct v4l2_subdev_format *fmt)
 {
@@ -212,6 +213,7 @@ imgu_subdev_get_compose(struct imgu_v4l2_subdev *sd,
 }
 
 static int imgu_subdev_get_selection(struct v4l2_subdev *sd,
+				     const struct v4l2_subdev_client_info *ci,
 				     struct v4l2_subdev_state *sd_state,
 				     struct v4l2_subdev_selection *sel)
 {
@@ -236,6 +238,7 @@ static int imgu_subdev_get_selection(struct v4l2_subdev *sd,
 }
 
 static int imgu_subdev_set_selection(struct v4l2_subdev *sd,
+				     const struct v4l2_subdev_client_info *ci,
 				     struct v4l2_subdev_state *sd_state,
 				     struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/staging/media/ipu7/ipu7-isys-csi2.c b/drivers/staging/media/ipu7/ipu7-isys-csi2.c
index 08f4ca1e0289..92abb2e651c6 100644
--- a/drivers/staging/media/ipu7/ipu7-isys-csi2.c
+++ b/drivers/staging/media/ipu7/ipu7-isys-csi2.c
@@ -191,6 +191,7 @@ static int ipu7_isys_csi2_enable_stream(struct ipu7_isys_csi2 *csi2)
 }
 
 static int ipu7_isys_csi2_set_sel(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *state,
 				  struct v4l2_subdev_selection *sel)
 {
@@ -242,6 +243,7 @@ static int ipu7_isys_csi2_set_sel(struct v4l2_subdev *sd,
 }
 
 static int ipu7_isys_csi2_get_sel(struct v4l2_subdev *sd,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *state,
 				  struct v4l2_subdev_selection *sel)
 {
diff --git a/drivers/staging/media/ipu7/ipu7-isys-subdev.c b/drivers/staging/media/ipu7/ipu7-isys-subdev.c
index 67a776033d5b..2cb0521b4240 100644
--- a/drivers/staging/media/ipu7/ipu7-isys-subdev.c
+++ b/drivers/staging/media/ipu7/ipu7-isys-subdev.c
@@ -99,6 +99,7 @@ u32 ipu7_isys_convert_bayer_order(u32 code, int x, int y)
 }
 
 int ipu7_isys_subdev_set_fmt(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *format)
 {
diff --git a/drivers/staging/media/ipu7/ipu7-isys-subdev.h b/drivers/staging/media/ipu7/ipu7-isys-subdev.h
index faa50031cf24..c3585d1c4128 100644
--- a/drivers/staging/media/ipu7/ipu7-isys-subdev.h
+++ b/drivers/staging/media/ipu7/ipu7-isys-subdev.h
@@ -31,6 +31,7 @@ bool ipu7_isys_is_bayer_format(u32 code);
 u32 ipu7_isys_convert_bayer_order(u32 code, int x, int y);
 
 int ipu7_isys_subdev_set_fmt(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_format *format);
 int ipu7_isys_subdev_enum_mbus_code(struct v4l2_subdev *sd,
diff --git a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
index 46a334b602f1..3f376af6c228 100644
--- a/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
+++ b/drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c
@@ -313,6 +313,7 @@ static int sun6i_isp_proc_get_fmt(struct v4l2_subdev *subdev,
 }
 
 static int sun6i_isp_proc_set_fmt(struct v4l2_subdev *subdev,
+				  const struct v4l2_subdev_client_info *ci,
 				  struct v4l2_subdev_state *state,
 				  struct v4l2_subdev_format *format)
 {
diff --git a/drivers/staging/media/tegra-video/csi.c b/drivers/staging/media/tegra-video/csi.c
index 41d57dac61a7..5a5fca378ae6 100644
--- a/drivers/staging/media/tegra-video/csi.c
+++ b/drivers/staging/media/tegra-video/csi.c
@@ -170,6 +170,7 @@ static int csi_enum_frameintervals(struct v4l2_subdev *subdev,
 }
 
 static int csi_set_format(struct v4l2_subdev *subdev,
+			  const struct v4l2_subdev_client_info *ci,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
 {
diff --git a/drivers/staging/media/tegra-video/vi.c b/drivers/staging/media/tegra-video/vi.c
index 456134a9e8cf..c4d42fbac038 100644
--- a/drivers/staging/media/tegra-video/vi.c
+++ b/drivers/staging/media/tegra-video/vi.c
@@ -477,7 +477,7 @@ static int __tegra_channel_try_format(struct tegra_vi_channel *chan,
 	ret = v4l2_subdev_call(subdev, pad, enum_frame_size, sd_state, &fse);
 	if (ret) {
 		if (!v4l2_subdev_has_op(subdev, pad, get_selection) ||
-		    v4l2_subdev_call(subdev, pad, get_selection, NULL, &sdsel)) {
+		    v4l2_subdev_call(subdev, pad, get_selection, NULL, NULL, &sdsel)) {
 			try_crop->width = 0;
 			try_crop->height = 0;
 		} else {
@@ -489,7 +489,7 @@ static int __tegra_channel_try_format(struct tegra_vi_channel *chan,
 		try_crop->height = fse.max_height;
 	}
 
-	ret = v4l2_subdev_call(subdev, pad, set_fmt, sd_state, &fmt);
+	ret = v4l2_subdev_call(subdev, pad, set_fmt, NULL, sd_state, &fmt);
 	if (ret < 0)
 		goto out_free;
 
@@ -543,7 +543,7 @@ static int tegra_channel_set_format(struct file *file, void *fh,
 	fmt.pad = 0;
 	v4l2_fill_mbus_format(&fmt.format, pix, fmtinfo->code);
 	subdev = tegra_channel_get_remote_source_subdev(chan);
-	ret = v4l2_subdev_call(subdev, pad, set_fmt, NULL, &fmt);
+	ret = v4l2_subdev_call(subdev, pad, set_fmt, NULL, NULL, &fmt);
 	if (ret < 0)
 		return ret;
 
@@ -626,7 +626,7 @@ static int tegra_channel_g_selection(struct file *file, void *priv,
 	 * Try the get selection operation and fallback to get format if not
 	 * implemented.
 	 */
-	ret = v4l2_subdev_call(subdev, pad, get_selection, NULL, &sdsel);
+	ret = v4l2_subdev_call(subdev, pad, get_selection, NULL, NULL, &sdsel);
 	if (!ret)
 		sel->r = sdsel.r;
 	if (ret != -ENOIOCTLCMD)
@@ -667,7 +667,7 @@ static int tegra_channel_s_selection(struct file *file, void *fh,
 	if (vb2_is_busy(&chan->queue))
 		return -EBUSY;
 
-	ret = v4l2_subdev_call(subdev, pad, set_selection, NULL, &sdsel);
+	ret = v4l2_subdev_call(subdev, pad, set_selection, NULL, NULL, &sdsel);
 	if (!ret) {
 		sel->r = sdsel.r;
 		/*
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index e29defed6409..13b490386d90 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -868,12 +868,15 @@ struct v4l2_subdev_pad_ops {
 		       struct v4l2_subdev_state *state,
 		       struct v4l2_subdev_format *format);
 	int (*set_fmt)(struct v4l2_subdev *sd,
+		       const struct v4l2_subdev_client_info *ci,
 		       struct v4l2_subdev_state *state,
 		       struct v4l2_subdev_format *format);
 	int (*get_selection)(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_selection *sel);
 	int (*set_selection)(struct v4l2_subdev *sd,
+			     const struct v4l2_subdev_client_info *ci,
 			     struct v4l2_subdev_state *state,
 			     struct v4l2_subdev_selection *sel);
 	int (*get_frame_interval)(struct v4l2_subdev *sd,
-- 
2.47.3

Re: [PATCH v5 10/10] media: v4l2-subdev: Add struct v4l2_subdev_client_info pointer to pad ops

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

CC'ing Bryan, Vladimir and Loic for the camss driver (please see below
for a dedicated comment).

On Mon, Jun 08, 2026 at 12:53:56AM +0300, Sakari Ailus wrote:
> Add a pointer to const struct v4l2_subdev_client_info to the set_fmt,
> get_selection and set_selection sub-device pad ops. The client info struct
> will soon be used to differentiate UAPI based on client capabilities.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/i2c/adv7170.c                   |  1 +

[snip] 

>  include/media/v4l2-subdev.h                   |  3 +
>  246 files changed, 539 insertions(+), 151 deletions(-)

Ouch :-/

[snip]

> diff --git a/drivers/media/i2c/adv7180.c b/drivers/media/i2c/adv7180.c
> index e5d11a6e6766..79a592f1831d 100644
> --- a/drivers/media/i2c/adv7180.c
> +++ b/drivers/media/i2c/adv7180.c
> @@ -771,6 +771,7 @@ static int adv7180_get_pad_format(struct v4l2_subdev *sd,
>  }
>  
>  static int adv7180_set_pad_format(struct v4l2_subdev *sd,
> +				  const struct v4l2_subdev_client_info *ci,
>  				  struct v4l2_subdev_state *sd_state,
>  				  struct v4l2_subdev_format *format)
>  {
> @@ -808,7 +809,7 @@ static int adv7180_init_state(struct v4l2_subdev *sd,
>  		: V4L2_SUBDEV_FORMAT_ACTIVE,
>  	};
>  
> -	return adv7180_set_pad_format(sd, sd_state, &fmt);
> +	return adv7180_set_pad_format(sd, NULL, sd_state, &fmt);

I'm a bit concerned about setting the pointer to NULL blindly. I suppose
it's fine in drivers that don't use the client info at all (and that's
all drivers today, so this series doesn't cause any immediate issue),
but I fear we will later regret not having thought out a migration
strategy.

What should happen to drivers that need the client info when their
.set_fmt(), .get_selection() or .set_selection() operations are called
by another driver (as opposed to being called by userspace through
subdev ioctls) ? Will they need to be able to handle a NULL client info
? Or can be forbid usage of those drivers in non-MC pipelines ? This is
an important question to answer now, as it could hinder migration of
sensor drivers used by the non-MC bridges.

Calling .set_fmt() from within the kernel is a common pattern to
implement .init_state(). Will we deprecate that ? And if so, what will
be the recommended pattern for .init_state() ?

>  }
>  
>  static int adv7180_get_mbus_config(struct v4l2_subdev *sd,
> diff --git a/drivers/media/i2c/adv7183.c b/drivers/media/i2c/adv7183.c
> index a04a1a205fe0..9e4cbebe6e5a 100644
> --- a/drivers/media/i2c/adv7183.c
> +++ b/drivers/media/i2c/adv7183.c
> @@ -420,6 +420,7 @@ static int adv7183_enum_mbus_code(struct v4l2_subdev *sd,
>  }
>  
>  static int adv7183_set_fmt(struct v4l2_subdev *sd,
> +		const struct v4l2_subdev_client_info *ci,
>  		struct v4l2_subdev_state *sd_state,
>  		struct v4l2_subdev_format *format)
>  {
> @@ -598,7 +599,7 @@ static int adv7183_probe(struct i2c_client *client)
>  	adv7183_s_std(sd, decoder->std);
>  	fmt.format.width = 720;
>  	fmt.format.height = 576;
> -	adv7183_set_fmt(sd, NULL, &fmt);
> +	adv7183_set_fmt(sd, NULL, NULL, &fmt);

When the status is already NULL we have a bigger issue anyway, so I'm
less concerned here :-)

>  
>  	/* initialize the hardware to the default control values */
>  	ret = v4l2_ctrl_handler_setup(hdl);

[snip]

> diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
> index 8e25f970fd12..9f3c5f00b37c 100644
> --- a/drivers/media/i2c/ccs/ccs-core.c
> +++ b/drivers/media/i2c/ccs/ccs-core.c
> @@ -2189,6 +2189,7 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
>  }
>  
>  static int ccs_set_format_source(struct v4l2_subdev *subdev,
> +				 const struct v4l2_subdev_client_info *ci,
>  				 struct v4l2_subdev_state *sd_state,
>  				 struct v4l2_subdev_format *fmt)
>  {
> @@ -2242,6 +2243,7 @@ static int ccs_set_format_source(struct v4l2_subdev *subdev,
>  }
>  
>  static int ccs_set_format(struct v4l2_subdev *subdev,
> +			  const struct v4l2_subdev_client_info *ci,
>  			  struct v4l2_subdev_state *sd_state,
>  			  struct v4l2_subdev_format *fmt)
>  {
> @@ -2252,7 +2254,7 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
>  	if (fmt->pad == ssd->source_pad) {
>  		int rval;
>  
> -		rval = ccs_set_format_source(subdev, sd_state, fmt);
> +		rval = ccs_set_format_source(subdev, NULL, sd_state, fmt);

Here you get a client info structure from the caller, you should pass it
to ccs_set_format_source().

>  
>  		return rval;
>  	}
> @@ -2467,6 +2469,7 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,
>  }
>  /* We're only called on source pads. This function sets scaling. */
>  static int ccs_set_compose(struct v4l2_subdev *subdev,
> +			   const struct v4l2_subdev_client_info *ci,
>  			   struct v4l2_subdev_state *sd_state,
>  			   struct v4l2_subdev_selection *sel)
>  {
> @@ -2536,6 +2539,7 @@ static int ccs_sel_supported(struct v4l2_subdev *subdev,
>  }
>  
>  static int ccs_set_crop(struct v4l2_subdev *subdev,
> +			const struct v4l2_subdev_client_info *ci,
>  			struct v4l2_subdev_state *sd_state,
>  			struct v4l2_subdev_selection *sel)
>  {
> @@ -2587,6 +2591,7 @@ static void ccs_get_native_size(struct ccs_subdev *ssd, struct v4l2_rect *r)
>  }
>  
>  static int ccs_get_selection(struct v4l2_subdev *subdev,
> +			     const struct v4l2_subdev_client_info *ci,
>  			     struct v4l2_subdev_state *sd_state,
>  			     struct v4l2_subdev_selection *sel)
>  {
> @@ -2633,6 +2638,7 @@ static int ccs_get_selection(struct v4l2_subdev *subdev,
>  }
>  
>  static int ccs_set_selection(struct v4l2_subdev *subdev,
> +			     const struct v4l2_subdev_client_info *ci,
>  			     struct v4l2_subdev_state *sd_state,
>  			     struct v4l2_subdev_selection *sel)
>  {
> @@ -2655,10 +2661,10 @@ static int ccs_set_selection(struct v4l2_subdev *subdev,
>  
>  	switch (sel->target) {
>  	case V4L2_SEL_TGT_CROP:
> -		ret = ccs_set_crop(subdev, sd_state, sel);
> +		ret = ccs_set_crop(subdev, NULL, sd_state, sel);
>  		break;
>  	case V4L2_SEL_TGT_COMPOSE:
> -		ret = ccs_set_compose(subdev, sd_state, sel);
> +		ret = ccs_set_compose(subdev, NULL, sd_state, sel);

Same here. There are other drivers touched by this series that can also
forward the client info.

>  		break;
>  	default:
>  		ret = -EINVAL;

[snip]

> diff --git a/drivers/media/i2c/mt9m001.c b/drivers/media/i2c/mt9m001.c
> index 0ade967b357b..c7d2e05ed6ed 100644
> --- a/drivers/media/i2c/mt9m001.c
> +++ b/drivers/media/i2c/mt9m001.c
> @@ -248,6 +248,7 @@ static int mt9m001_s_stream(struct v4l2_subdev *sd, int enable)
>  }
>  
>  static int mt9m001_set_selection(struct v4l2_subdev *sd,
> +		const struct v4l2_subdev_client_info *ci,
>  		struct v4l2_subdev_state *sd_state,
>  		struct v4l2_subdev_selection *sel)
>  {
> @@ -289,6 +290,7 @@ static int mt9m001_set_selection(struct v4l2_subdev *sd,
>  }
>  
>  static int mt9m001_get_selection(struct v4l2_subdev *sd,
> +		const struct v4l2_subdev_client_info *ci,
>  		struct v4l2_subdev_state *sd_state,
>  		struct v4l2_subdev_selection *sel)
>  {
> @@ -359,7 +361,7 @@ static int mt9m001_s_fmt(struct v4l2_subdev *sd,
>  	int ret;
>  
>  	/* No support for scaling so far, just crop. TODO: use skipping */
> -	ret = mt9m001_set_selection(sd, NULL, &sel);
> +	ret = mt9m001_set_selection(sd, NULL, NULL, &sel);

Here for instance, you could pass the client info to mt9m001_s_fmt()
(although the driver's logic seems wrong, but that's a different
matter). While at it you could also forward the state pointer.

>  	if (!ret) {
>  		mf->width	= mt9m001->rect.width;
>  		mf->height	= mt9m001->rect.height;
> @@ -371,6 +373,7 @@ static int mt9m001_s_fmt(struct v4l2_subdev *sd,
>  }
>  
>  static int mt9m001_set_fmt(struct v4l2_subdev *sd,
> +		const struct v4l2_subdev_client_info *ci,
>  		struct v4l2_subdev_state *sd_state,
>  		struct v4l2_subdev_format *format)
>  {

[snip]

>  static int vgxy61_s_ctrl(struct v4l2_ctrl *ctrl)
> diff --git a/drivers/media/pci/cobalt/cobalt-driver.c b/drivers/media/pci/cobalt/cobalt-driver.c
> index 9b9f69ff4016..2053849c9b4c 100644
> --- a/drivers/media/pci/cobalt/cobalt-driver.c
> +++ b/drivers/media/pci/cobalt/cobalt-driver.c
> @@ -525,8 +525,8 @@ static int cobalt_subdevs_init(struct cobalt *cobalt)
>  				&cobalt_edid);
>  		if (err)
>  			return err;
> -		err = v4l2_subdev_call(s[i].sd, pad, set_fmt, NULL,
> -				&sd_fmt);
> +		err = v4l2_subdev_call(s[i].sd, pad, set_fmt, NULL, NULL,
> +				       &sd_fmt);

Another case of an in-kernel caller that doesn't pass a state. This one
is a cross-driver call, it will already break with subdev drivers that
require a state, so not passing a client info isn't a big issue (I
assume we will require drivers that use client info to be state-based).

>  		if (err)
>  			return err;
>  		/* Reset channel video module */
> @@ -609,8 +609,8 @@ static int cobalt_subdevs_hsma_init(struct cobalt *cobalt)
>  
>  		if (err)
>  			return err;
> -		err = v4l2_subdev_call(s->sd, pad, set_fmt, NULL,
> -				&sd_fmt);
> +		err = v4l2_subdev_call(s->sd, pad, set_fmt, NULL, NULL,
> +				       &sd_fmt);
>  		if (err)
>  			return err;
>  		cobalt->have_hsma_rx = true;

[snip]

> diff --git a/drivers/media/pci/saa7134/saa7134-empress.c b/drivers/media/pci/saa7134/saa7134-empress.c
> index 8c4f70e4177d..d04a68bb05d8 100644
> --- a/drivers/media/pci/saa7134/saa7134-empress.c
> +++ b/drivers/media/pci/saa7134/saa7134-empress.c
> @@ -122,7 +122,7 @@ static int empress_s_fmt_vid_cap(struct file *file, void *priv,
>  	};
>  
>  	v4l2_fill_mbus_format(&format.format, &f->fmt.pix, MEDIA_BUS_FMT_FIXED);
> -	saa_call_all(dev, pad, set_fmt, NULL, &format);
> +	saa_call_all(dev, pad, set_fmt, NULL, NULL, &format);
>  	v4l2_fill_pix_format(&f->fmt.pix, &format.format);
>  
>  	f->fmt.pix.pixelformat  = V4L2_PIX_FMT_MPEG;
> @@ -145,7 +145,7 @@ static int empress_try_fmt_vid_cap(struct file *file, void *priv,
>  	};
>  
>  	v4l2_fill_mbus_format(&format.format, &f->fmt.pix, MEDIA_BUS_FMT_FIXED);
> -	saa_call_all(dev, pad, set_fmt, &pad_state, &format);
> +	saa_call_all(dev, pad, set_fmt, NULL, &pad_state, &format);

Fake states are really a hack, so I'm not worried here either (except by
the amount of legacy code we have, but that's again a separate issue).

>  	v4l2_fill_pix_format(&f->fmt.pix, &format.format);
>  
>  	f->fmt.pix.pixelformat  = V4L2_PIX_FMT_MPEG;

[snip]

> diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.c b/drivers/media/platform/amd/isp4/isp4_subdev.c
> index 48deea79ce6c..03752d88c0b6 100644
> --- a/drivers/media/platform/amd/isp4/isp4_subdev.c
> +++ b/drivers/media/platform/amd/isp4/isp4_subdev.c
> @@ -890,6 +890,7 @@ static const struct v4l2_subdev_video_ops isp4sd_video_ops = {
>  };
>  
>  static int isp4sd_set_pad_format(struct v4l2_subdev *sd,
> +				 const struct v4l2_subdev_client_info *ci,
>  				 struct v4l2_subdev_state *sd_state,
>  				 struct v4l2_subdev_format *format)
>  {
> diff --git a/drivers/media/platform/amd/isp4/isp4_video.c b/drivers/media/platform/amd/isp4/isp4_video.c
> index 0cebb39f98e1..856a2a0b4a12 100644
> --- a/drivers/media/platform/amd/isp4/isp4_video.c
> +++ b/drivers/media/platform/amd/isp4/isp4_video.c
> @@ -240,7 +240,7 @@ static int isp4vid_set_fmt_2_isp(struct v4l2_subdev *sdev,
>  	fmt.pad = ISP4VID_PAD_VIDEO_OUTPUT;
>  	fmt.format.width = pix_fmt->width;
>  	fmt.format.height = pix_fmt->height;
> -	return v4l2_subdev_call(sdev, pad, set_fmt, NULL, &fmt);
> +	return v4l2_subdev_call(sdev, pad, set_fmt, NULL, NULL, &fmt);
>  }
>  
>  static int isp4vid_s_fmt_vid_cap(struct file *file, void *priv,
> diff --git a/drivers/media/platform/amlogic/c3/isp/c3-isp-core.c b/drivers/media/platform/amlogic/c3/isp/c3-isp-core.c
> index ff6413fff889..553c808f8f3b 100644
> --- a/drivers/media/platform/amlogic/c3/isp/c3-isp-core.c
> +++ b/drivers/media/platform/amlogic/c3/isp/c3-isp-core.c
> @@ -461,6 +461,7 @@ static void c3_isp_core_set_source_fmt(struct v4l2_subdev_state *state,
>  }
>  
>  static int c3_isp_core_set_fmt(struct v4l2_subdev *sd,
> +			       const struct v4l2_subdev_client_info *ci,
>  			       struct v4l2_subdev_state *state,
>  			       struct v4l2_subdev_format *format)
>  {
> diff --git a/drivers/media/platform/amlogic/c3/isp/c3-isp-resizer.c b/drivers/media/platform/amlogic/c3/isp/c3-isp-resizer.c
> index 453a889e0b27..1f9c16eb0842 100644
> --- a/drivers/media/platform/amlogic/c3/isp/c3-isp-resizer.c
> +++ b/drivers/media/platform/amlogic/c3/isp/c3-isp-resizer.c
> @@ -621,6 +621,7 @@ static void c3_isp_rsz_set_source_fmt(struct v4l2_subdev_state *state,
>  }
>  
>  static int c3_isp_rsz_set_fmt(struct v4l2_subdev *sd,
> +			      const struct v4l2_subdev_client_info *ci,
>  			      struct v4l2_subdev_state *state,
>  			      struct v4l2_subdev_format *format)
>  {
> @@ -633,6 +634,7 @@ static int c3_isp_rsz_set_fmt(struct v4l2_subdev *sd,
>  }
>  
>  static int c3_isp_rsz_get_selection(struct v4l2_subdev *sd,
> +				    const struct v4l2_subdev_client_info *ci,
>  				    struct v4l2_subdev_state *state,
>  				    struct v4l2_subdev_selection *sel)
>  {
> @@ -674,6 +676,7 @@ static int c3_isp_rsz_get_selection(struct v4l2_subdev *sd,
>  }
>  
>  static int c3_isp_rsz_set_selection(struct v4l2_subdev *sd,
> +				    const struct v4l2_subdev_client_info *ci,
>  				    struct v4l2_subdev_state *state,
>  				    struct v4l2_subdev_selection *sel)
>  {
> diff --git a/drivers/media/platform/amlogic/c3/mipi-adapter/c3-mipi-adap.c b/drivers/media/platform/amlogic/c3/mipi-adapter/c3-mipi-adap.c
> index 4bd98fb9c7e9..6f75112723b5 100644
> --- a/drivers/media/platform/amlogic/c3/mipi-adapter/c3-mipi-adap.c
> +++ b/drivers/media/platform/amlogic/c3/mipi-adapter/c3-mipi-adap.c
> @@ -521,6 +521,7 @@ static int c3_mipi_adap_enum_mbus_code(struct v4l2_subdev *sd,
>  }
>  
>  static int c3_mipi_adap_set_fmt(struct v4l2_subdev *sd,
> +				const struct v4l2_subdev_client_info *ci,
>  				struct v4l2_subdev_state *state,
>  				struct v4l2_subdev_format *format)
>  {
> diff --git a/drivers/media/platform/amlogic/c3/mipi-csi2/c3-mipi-csi2.c b/drivers/media/platform/amlogic/c3/mipi-csi2/c3-mipi-csi2.c
> index b9e4ef3fc308..0c399665f7fa 100644
> --- a/drivers/media/platform/amlogic/c3/mipi-csi2/c3-mipi-csi2.c
> +++ b/drivers/media/platform/amlogic/c3/mipi-csi2/c3-mipi-csi2.c
> @@ -491,6 +491,7 @@ static int c3_mipi_csi_enum_mbus_code(struct v4l2_subdev *sd,
>  }
>  
>  static int c3_mipi_csi_set_fmt(struct v4l2_subdev *sd,
> +			       const struct v4l2_subdev_client_info *ci,
>  			       struct v4l2_subdev_state *state,
>  			       struct v4l2_subdev_format *format)
>  {
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> index e128adf6ee37..c8464dec9a21 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-isp.c
> @@ -203,6 +203,7 @@ static int mali_c55_isp_enum_frame_size(struct v4l2_subdev *sd,
>  }
>  
>  static int mali_c55_isp_set_fmt(struct v4l2_subdev *sd,
> +				const struct v4l2_subdev_client_info *ci,
>  				struct v4l2_subdev_state *state,
>  				struct v4l2_subdev_format *format)
>  {
> @@ -265,6 +266,7 @@ static int mali_c55_isp_set_fmt(struct v4l2_subdev *sd,
>  }
>  
>  static int mali_c55_isp_get_selection(struct v4l2_subdev *sd,
> +				      const struct v4l2_subdev_client_info *ci,
>  				      struct v4l2_subdev_state *state,
>  				      struct v4l2_subdev_selection *sel)
>  {
> @@ -278,6 +280,7 @@ static int mali_c55_isp_get_selection(struct v4l2_subdev *sd,
>  }
>  
>  static int mali_c55_isp_set_selection(struct v4l2_subdev *sd,
> +				      const struct v4l2_subdev_client_info *ci,
>  				      struct v4l2_subdev_state *state,
>  				      struct v4l2_subdev_selection *sel)
>  {
> diff --git a/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
> index c4f46651dcee..b1351f053b87 100644
> --- a/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
> +++ b/drivers/media/platform/arm/mali-c55/mali-c55-resizer.c

[snip]

> @@ -826,12 +829,13 @@ static int mali_c55_rsz_set_fmt(struct v4l2_subdev *sd,
>  
>  	if (format->pad == MALI_C55_RSZ_SINK_PAD ||
>  	    format->pad == MALI_C55_RSZ_SINK_BYPASS_PAD)
> -		return mali_c55_rsz_set_sink_fmt(sd, state, format);
> +		return mali_c55_rsz_set_sink_fmt(sd, NULL, state, format);
>  
> -	return mali_c55_rsz_set_source_fmt(sd, state, format);
> +	return mali_c55_rsz_set_source_fmt(sd, NULL, state, format);

You can forward the client info here too.

>  }
>  
>  static int mali_c55_rsz_get_selection(struct v4l2_subdev *sd,
> +				      const struct v4l2_subdev_client_info *ci,
>  				      struct v4l2_subdev_state *state,
>  				      struct v4l2_subdev_selection *sel)
>  {

[snip]

> @@ -963,10 +970,10 @@ static int mali_c55_rsz_set_selection(struct v4l2_subdev *sd,
>  		return -EINVAL;
>  
>  	if (sel->target == V4L2_SEL_TGT_CROP)
> -		return mali_c55_rsz_set_crop(sd, state, sel);
> +		return mali_c55_rsz_set_crop(sd, NULL, state, sel);
>  
>  	if (sel->target == V4L2_SEL_TGT_COMPOSE)
> -		return mali_c55_rsz_set_compose(sd, state, sel);
> +		return mali_c55_rsz_set_compose(sd, NULL, state, sel);

And here.

>  
>  	return -EINVAL;
>  }

[snip

> diff --git a/drivers/media/platform/nxp/imx-mipi-csis.c b/drivers/media/platform/nxp/imx-mipi-csis.c
> index 5f9691d76434..1b9012dbae39 100644
> --- a/drivers/media/platform/nxp/imx-mipi-csis.c
> +++ b/drivers/media/platform/nxp/imx-mipi-csis.c
> @@ -1104,6 +1104,7 @@ static int mipi_csis_enum_mbus_code(struct v4l2_subdev *sd,
>  }
>  
>  static int mipi_csis_set_fmt(struct v4l2_subdev *sd,
> +			     const struct v4l2_subdev_client_info *ci,
>  			     struct v4l2_subdev_state *state,
>  			     struct v4l2_subdev_format *sdformat)
>  {
> @@ -1225,7 +1226,7 @@ static int mipi_csis_init_state(struct v4l2_subdev *sd,
>  		V4L2_MAP_QUANTIZATION_DEFAULT(false, fmt.format.colorspace,
>  					      fmt.format.ycbcr_enc);
>  
> -	return mipi_csis_set_fmt(sd, state, &fmt);
> +	return mipi_csis_set_fmt(sd, NULL, state, &fmt);
>  }
>  
>  static int mipi_csis_log_status(struct v4l2_subdev *sd)
> diff --git a/drivers/media/platform/nxp/imx7-media-csi.c b/drivers/media/platform/nxp/imx7-media-csi.c
> index 7ddc7ba06e3d..0b2ee4dfdcbb 100644
> --- a/drivers/media/platform/nxp/imx7-media-csi.c
> +++ b/drivers/media/platform/nxp/imx7-media-csi.c
> @@ -1890,6 +1890,7 @@ static void imx7_csi_try_fmt(struct v4l2_subdev *sd,
>  }
>  
>  static int imx7_csi_set_fmt(struct v4l2_subdev *sd,
> +			    const struct v4l2_subdev_client_info *ci,
>  			    struct v4l2_subdev_state *sd_state,
>  			    struct v4l2_subdev_format *sdformat)
>  {
> diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> index a1cd1e159468..37c147f00038 100644
> --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> @@ -249,6 +249,7 @@ static int mxc_isi_crossbar_enum_mbus_code(struct v4l2_subdev *sd,
>  }
>  
>  static int mxc_isi_crossbar_set_fmt(struct v4l2_subdev *sd,
> +				    const struct v4l2_subdev_client_info *ci,
>  				    struct v4l2_subdev_state *state,
>  				    struct v4l2_subdev_format *fmt)
>  {
> diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c
> index 2d0843c86534..e4ec2bc4fd96 100644
> --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c
> +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c
> @@ -448,6 +448,7 @@ static int mxc_isi_pipe_enum_mbus_code(struct v4l2_subdev *sd,
>  }
>  
>  static int mxc_isi_pipe_set_fmt(struct v4l2_subdev *sd,
> +				const struct v4l2_subdev_client_info *ci,
>  				struct v4l2_subdev_state *state,
>  				struct v4l2_subdev_format *fmt)
>  {
> @@ -543,6 +544,7 @@ static int mxc_isi_pipe_set_fmt(struct v4l2_subdev *sd,
>  }
>  
>  static int mxc_isi_pipe_get_selection(struct v4l2_subdev *sd,
> +				      const struct v4l2_subdev_client_info *ci,
>  				      struct v4l2_subdev_state *state,
>  				      struct v4l2_subdev_selection *sel)
>  {
> @@ -602,6 +604,7 @@ static int mxc_isi_pipe_get_selection(struct v4l2_subdev *sd,
>  }
>  
>  static int mxc_isi_pipe_set_selection(struct v4l2_subdev *sd,
> +				      const struct v4l2_subdev_client_info *ci,
>  				      struct v4l2_subdev_state *state,
>  				      struct v4l2_subdev_selection *sel)
>  {
> diff --git a/drivers/media/platform/nxp/imx8mq-mipi-csi2.c b/drivers/media/platform/nxp/imx8mq-mipi-csi2.c
> index 04ebed8a0493..17756b7fe1dd 100644
> --- a/drivers/media/platform/nxp/imx8mq-mipi-csi2.c
> +++ b/drivers/media/platform/nxp/imx8mq-mipi-csi2.c
> @@ -597,6 +597,7 @@ static int imx8mq_mipi_csi_enum_mbus_code(struct v4l2_subdev *sd,
>  }
>  
>  static int imx8mq_mipi_csi_set_fmt(struct v4l2_subdev *sd,
> +				   const struct v4l2_subdev_client_info *ci,
>  				   struct v4l2_subdev_state *sd_state,
>  				   struct v4l2_subdev_format *sdformat)
>  {
> diff --git a/drivers/media/platform/qcom/camss/camss-csid.c b/drivers/media/platform/qcom/camss/camss-csid.c
> index 48459b46a981..a00791b00feb 100644
> --- a/drivers/media/platform/qcom/camss/camss-csid.c
> +++ b/drivers/media/platform/qcom/camss/camss-csid.c
> @@ -987,6 +987,7 @@ static int csid_get_format(struct v4l2_subdev *sd,
>   * Return -EINVAL or zero on success
>   */
>  static int csid_set_format(struct v4l2_subdev *sd,
> +			   const struct v4l2_subdev_client_info *ci,
>  			   struct v4l2_subdev_state *sd_state,
>  			   struct v4l2_subdev_format *fmt)
>  {
> @@ -1036,7 +1037,7 @@ static int csid_init_formats(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
>  		}
>  	};
>  
> -	return csid_set_format(sd, fh ? fh->state : NULL, &format);
> +	return csid_set_format(sd, NULL, fh ? fh->state : NULL, &format);

The driver really needs to be converted to the active state API. Bryan,
would you be able to address this (or work with someone else to do so, I
know you have a few other cleanup tasks for camss that you'd like to see
being addressed) ? Same for the rest of the camss driver, .open() needs
to go and be replaced with .init_state().

>  }
>  
>  /*

[snip]

> diff --git a/drivers/media/platform/qcom/camss/camss-vfe.c b/drivers/media/platform/qcom/camss/camss-vfe.c
> index 319d19158988..0ba5bf3abeda 100644
> --- a/drivers/media/platform/qcom/camss/camss-vfe.c
> +++ b/drivers/media/platform/qcom/camss/camss-vfe.c
> @@ -1562,6 +1562,7 @@ static int vfe_get_format(struct v4l2_subdev *sd,
>  }
>  
>  static int vfe_set_selection(struct v4l2_subdev *sd,
> +			     const struct v4l2_subdev_client_info *ci,
>  			     struct v4l2_subdev_state *sd_state,
>  			     struct v4l2_subdev_selection *sel);
>  
> @@ -1574,6 +1575,7 @@ static int vfe_set_selection(struct v4l2_subdev *sd,
>   * Return -EINVAL or zero on success
>   */
>  static int vfe_set_format(struct v4l2_subdev *sd,
> +			  const struct v4l2_subdev_client_info *ci,
>  			  struct v4l2_subdev_state *sd_state,
>  			  struct v4l2_subdev_format *fmt)
>  {
> @@ -1608,7 +1610,7 @@ static int vfe_set_format(struct v4l2_subdev *sd,
>  		sel.target = V4L2_SEL_TGT_COMPOSE;
>  		sel.r.width = fmt->format.width;
>  		sel.r.height = fmt->format.height;
> -		ret = vfe_set_selection(sd, sd_state, &sel);
> +		ret = vfe_set_selection(sd, NULL, sd_state, &sel);

Forward client info here too. Same below in vfe_set_selection().

>  		if (ret < 0)
>  			return ret;
>  	}
> @@ -1625,6 +1627,7 @@ static int vfe_set_format(struct v4l2_subdev *sd,
>   * Return -EINVAL or zero on success
>   */
>  static int vfe_get_selection(struct v4l2_subdev *sd,
> +			     const struct v4l2_subdev_client_info *ci,
>  			     struct v4l2_subdev_state *sd_state,
>  			     struct v4l2_subdev_selection *sel)
>  {
> @@ -1695,6 +1698,7 @@ static int vfe_get_selection(struct v4l2_subdev *sd,
>   * Return -EINVAL or zero on success
>   */
>  static int vfe_set_selection(struct v4l2_subdev *sd,
> +			     const struct v4l2_subdev_client_info *ci,
>  			     struct v4l2_subdev_state *sd_state,
>  			     struct v4l2_subdev_selection *sel)
>  {
> @@ -1721,7 +1725,7 @@ static int vfe_set_selection(struct v4l2_subdev *sd,
>  		crop.pad = MSM_VFE_PAD_SRC;
>  		crop.target = V4L2_SEL_TGT_CROP;
>  		crop.r = *rect;
> -		ret = vfe_set_selection(sd, sd_state, &crop);
> +		ret = vfe_set_selection(sd, NULL, sd_state, &crop);
>  	} else if (sel->target == V4L2_SEL_TGT_CROP &&
>  		sel->pad == MSM_VFE_PAD_SRC) {
>  		struct v4l2_subdev_format fmt = { 0 };
> @@ -1742,7 +1746,7 @@ static int vfe_set_selection(struct v4l2_subdev *sd,
>  
>  		fmt.format.width = rect->width;
>  		fmt.format.height = rect->height;
> -		ret = vfe_set_format(sd, sd_state, &fmt);
> +		ret = vfe_set_format(sd, NULL, sd_state, &fmt);
>  	} else {
>  		ret = -EINVAL;
>  	}

[snip]

> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drm.c b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
> index f6fbd3475329..5eb262a88f9f 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_drm.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
> @@ -137,7 +137,8 @@ static int vsp1_du_insert_uif(struct vsp1_device *vsp1,
>  
>  	format.pad = UIF_PAD_SINK;
>  
> -	ret = v4l2_subdev_call(&uif->subdev, pad, set_fmt, NULL, &format);
> +	ret = v4l2_subdev_call(&uif->subdev, pad, set_fmt, NULL, NULL,

This one is on me to handle later if/when the vsp1 driver switches to
the active state API.

> +			       &format);
>  	if (ret < 0)
>  		return ret;
>  
> @@ -184,7 +185,7 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
>  	format.format.ycbcr_enc = input->ycbcr_enc;
>  	format.format.quantization = input->quantization;
>  
> -	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_fmt, NULL,
> +	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_fmt, NULL, NULL,
>  			       &format);
>  	if (ret < 0)
>  		return ret;
> @@ -199,6 +200,7 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
>  	sel.r = input->crop;
>  
>  	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_selection, NULL,
> +			       NULL,
>  			       &sel);

Drop that line break.

>  	if (ret < 0)
>  		return ret;

[snip]

> diff --git a/drivers/media/platform/samsung/exynos4-is/fimc-capture.c b/drivers/media/platform/samsung/exynos4-is/fimc-capture.c
> index b5749f9cba39..61200c1e9681 100644
> --- a/drivers/media/platform/samsung/exynos4-is/fimc-capture.c
> +++ b/drivers/media/platform/samsung/exynos4-is/fimc-capture.c
> @@ -797,7 +797,8 @@ static int fimc_pipeline_try_format(struct fimc_ctx *ctx,
>  			sd = media_entity_to_v4l2_subdev(me);
>  
>  			sfmt.pad = 0;
> -			ret = v4l2_subdev_call(sd, pad, set_fmt, NULL, &sfmt);
> +			ret = v4l2_subdev_call(sd, pad, set_fmt, NULL, NULL,
> +					       &sfmt);
>  			if (ret)
>  				return ret;
>  
> @@ -805,6 +806,7 @@ static int fimc_pipeline_try_format(struct fimc_ctx *ctx,
>  				sfmt.pad = me->num_pads - 1;
>  				mf->code = tfmt->code;
>  				ret = v4l2_subdev_call(sd, pad, set_fmt, NULL,
> +									NULL,
>  									&sfmt);

				ret = v4l2_subdev_call(sd, pad, set_fmt, NULL,
						       NULL, &sfmt);

>  				if (ret)
>  					return ret;

[snip]

> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index c1e1fb9d6773..b384c338940b 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c

[snip]

> @@ -1938,7 +1966,8 @@ v4l2_subdev_init_stream_configs(struct v4l2_subdev_stream_configs *stream_config
>  	return 0;
>  }
>  
> -int v4l2_subdev_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> +int v4l2_subdev_get_fmt(struct v4l2_subdev *sd,
> +			struct v4l2_subdev_state *state,

Unrelated change (probably from a previous version where you added a
client_info argument to .get_fmt() too ?

>  			struct v4l2_subdev_format *format)
>  {
>  	struct v4l2_mbus_framefmt *fmt;

[snip]

-- 
Regards,

Laurent Pinchart