[PATCH v11 00/66] Generic line based metadata support, internal pads

[PATCH v11 00/66] Generic line based metadata support, internal pads

[Sakari Ailus]

Hi folks,

Here are a few patches to add support generic, line based metadata as well
as internal pads, sub-device configuration models (including the common
raw sensor model) and a streaming control helper for MC-enabled drivers.
The amount of code and documentation is in this version is no longer small
but it still requires some explaining.

Metadata mbus codes and formats have existed for some time in V4L2. They
however have been only used by drivers that produce the data itself and
effectively this metadata has always been statistics of some sort (at
least when it comes to ISPs). What is different here is that we intend to
add support for metadata originating from camera sensors.

Camera sensors produce different kinds of metadata, embedded data (usually
register address--value pairs used to capture the frame, in a more or less
sensor specific format), histograms (in a very sensor specific format),
dark pixels etc. The number of these formats is probably going to be about
as large as image data formats if not larger, as the image data formats
are much better standardised but a smaller subset of them will be
supported by V4L2, at least initially but possibly much more in the long
run.

Having this many device specific formats would be a major problem for all
the other drivers along that pipeline (not to mention the users of those
drivers), including bridge (e.g. CSI-2 to parallel) but especially CSI-2
receiver drivers that have DMA: the poor driver developer would not only
need to know camera sensor specific formats but to choose the specific
packing of that format suitable for the DMA used by the hardware. It is
unlikely many of these would ever get tested while being present on the
driver API. Also adding new sensors with new embedded data formats would
involve updating all bridge and CSI-2 receiver drivers. I don't expect
this to be a workable approach.

Instead what I'm proposing is to use controls to convey the colour pattern
and metadata layout. Generic mbus codes convey the width of the data
transferred on serial buses (for now, corresponding parallel formats may
be added when needed) but the control is only relevant on the UAPI, not
between drivers. This approach unsnarls the layout of the data and its
colour components, defining what data there is (specific mbus code) and
how that is transported and packed (generic mbus codes and V4L2 formats)
separately.

The user space would be required to "know" the path of that data from the
sensor's internal pad to the V4L2 video node. I do not see this as these
devices require at least some knowledge of the pipeline, i.e. hardware at
hand. Separating what the data means and how it is packed may even be
beneficial: it allows separating code that interprets the data (sensor
internal mbus code) from the code that accesses it (packing).

These formats are in practice line based, meaning that there may be
padding at the end of the line, depending on the bus as well as the DMA.
If non-line based formats are needed, it is always possible to set the
"height" field to 1.

Internal sink pads behave mostly like regular sink pads, but they describe
a flow of data that originates from a sub-device instead of arriving to a
sub-device. The SUBDEV_S_ROUTING IOCTLs are used to enable and disable
routes from internal sink pads to sub-device's source pads. The subdev
format IOCTLs are usable, too, so one can find which subdev format is
available on given internal sink pad.

I've tested the patches using the IPU6 and ov2740 drivers, meaning not all
of this has been tested after making changes (e.g. the imx219 patches have
been only compile tested).

Questions and comments are most welcome.

Preliminary media-ctl and yavta patches can be found here:

<URL:https://git.retiisi.eu/?p=~sailus/yavta.git;a=shortlog;h=refs/heads/metadata>
<URL:https://git.retiisi.eu/?p=~sailus/v4l-utils.git;a=shortlog;h=refs/heads/metadata-l>

I have used IMX219 as an example on routing in a sensor driver in this
version.

The patches are on my metadata branch
<URL:https://git.linuxtv.org/sailus/media_tree.git/log/>.

These patches are on top of my v4l2-mc streaming control helper set, all
of these can be found in the GitWeb URL above.

The documentation build output can be found here:
<URL:https://www.retiisi.eu/~sailus/v4l2/tmp/common-raw/>.

since v10:

- Merge a few patches that aren't strictly related to metadata, mostly
  documentation cleanups and improvements.

- Add V4L2_SUBDEV_ROUTE_FL_STATIC, to indicate a route is mutable but
  hardware specific so it remains even if disabled. An example of this are
  embedded data routes on camera sensors. IMMUTABLE implies STATIC.

- Postpone adding formats higher than 14 bpp and remove reference to
  "little endian" for the CSI-2 packed formats. It's not entirely clear
  how the data on higher bit depth formats are laid out in memory. The
  documentation suggests these are in big endian format but that would be
  impractical for CPU processing.

- Add references to controls in flipping documentation in
  subdev-formats.rst.

- Add -ENXIO return code for device specific invalid routing configuration
  and clarify routing error conditions for -EINVAL.

- Add braces for the case in imx219_enum_mbus_code() to make Clang happy.

- Rebase off of mc-streamon set (to my cleanup branch). The problem of
  when to stream on and off still needs to be addressed but I'm looking
  into a different, better integrated approach right now.

- Rename generic metadata formats at bit depths 16 and 24 as it seems
  these have more generic packing than CSI-2.

since v9:

- Introduce controls for sub-device configuration models (separately
  discussed in <20250203085853.1361401-1-sakari.ailus@linux.intel.com>),
  with the following changes:

	- Fix spelling and improve wording in documentation based on
	  comments.
	
	- Use US English spelling for V4L2 control names.
	
	- Allow for more than one internal pad for pixel data.
	
	- Introduce (or change a bit) generic mbus codes for raw pixel
	  data and metadata.
	
	- Introduce generic and CSI-2 specifc V4L2 pixelformats for image
	  data and metadata. Also 16- and 24-bit formats are generic, not
	  specific to CSI-2.

	- Introduce new controls to convey metadata layout on bus and by
	  extension, in memory, instead of relying on separate metadata
	  mbus codes, aligning with pixel data.

	- Use generic metadata codes in the example.

- Include imx219 embedded data support patches from Laurent, with
  modifications.

- Patches to IPU6 ISYS driver needed to support metadata capture. Luma
  formats added, too.

- Improved debug prints in v4l2_subdev_{en,dis}able_streams().

- Added a patch to make media_entity_to_video_device() NULL-safe.

- Drop _media-glossary-data-unit reference, use :term: to reference it.

- Include a patch to set IPU6 ISYS minimum height to 1.

- Patch 'media: uapi: Document which mbus format fields are valid for
  metadata' already merged.

- Patch 'media: uapi: v4l: Add generic 8-bit metadata format definitions'
  already merged.

- Patch 'media: v4l: Support line-based metadata capture' already merged.

- Patch 'media: Documentation: Document S_ROUTING behaviour' already merged.

- Patch 'media: v4l: subdev: Add len_routes field to struct
  v4l2_subdev_routing' already merged.

since v8:

- Move the patch adding internal pad flag past the routing API reworks, as
  well as a few other patches, in order to separate the patches to those
  that could still be merged for v6.10 (routing changes) and those that
  couldn't (sensor API related). The patch on the edge is "media: uapi:
  v4l: subdev: Enable streams API".

- Include Laurent's two patches to address crop API issues wrt. streams.

- Add two patches to prepare for CCS driver rework (media: ccs: Move
  ccs_pm_get_init function up and media: ccs: Rename out label of
  ccs_start_streaming).

- Address issues in the ov2740 driver patches (as well as the driver
  itself), 4 more patches towards the end of the set.

- Improved generic metadata format names, align with other existing
  formats.

- Improved ov2740 embedded data documentation.

- Reworked streams and camera sensor documentation based on Laurent's
  comments mainly. In particular, the contradictory concept of internal
  source pads no longer should exist in the patches.

- Fixed pad numbering in the CCS example.

- Fixed S_ROUTING behaviour when len_routes is too small and when
  S_ROUTING isn't implemented by the driver.

- Reorder sections in meta-formats.rst alphabetically.

- Add a note per struct fields that certain struct v4l2_subdev_format are
  zero for metadata mbus codes.

- CCS driver patch cleanups.

- CCS driver metadata width fix for space-efficient embedded data at 16
  bpp and over.

- Postpone CCS frame descriptor quirk for now.

- Use MIPI_CSI2_DT_USER_DEFINED(0) instead of a numerical value for
  compressed data datatype.

since v7:

- Add embedded data support for the ov2740 driver.

- Add three patches on top, to add an IMMUTABLE flag to source streams
  when they cannot be disabled.

- Improved documentation of len_routes and num_routes arguments of
  [GS]_ROUTING.

- Remove one inclusion of twice-included media/v4l2-fwnode.h in
  drivers/media/i2c/ccs/ccs-core.c .

- Add missing forward declaration of ccs_internal_ops in
  drivers/media/i2c/ccs/ccs-core.c .

since v6:

- Improve embedded data UAPI documentation on camera sensors.

- Improve wording of stream glossary entry.

- Improve internal pad flag documentation.

- Fix definition of "data unit" and remove an extra "only" in INTERNAL pad
  flag description (1st patch).

- Use IMX219 driver in examples consistently.

- Remove the CSI-2 to parallel bridge from the example to simplify the
  example.

- Minor rewording of some parts of the routing examples.

- Rebase on unified sub-device state information access functions:
  <URL:https://lore.kernel.org/linux-media/20231027095913.1010187-1-sakari.ailus@linux.intel.com/T/#t>

- In CCS driver, do not maintain current active configuration in driver's
  device context struct (apart from mbus codes). Rely on sub-device state
  locking and clean up the code. (Multiple patches towards the end of the
  set.)

- Arrange the CCS patches early in the set towards the end of the set.

- Move the patch enabling streams API to the end of the set.

- Rework IOCTL argument copying condition for [GS]_ROUTING).

- Handle copying back routes in S_ROUTING, do not rely on G_ROUTING
  IOCTL implementation.

- Rebase on metadata preparation patchset v6:
  <URL:https://lore.kernel.org/linux-media/20231106121805.1266696-1-sakari.ailus@linux.intel.com/T/#t>.

since v5:

- Rebase on new set of preparation patches.

- Switch CCS driver from s_stream to enable_streams/disable_streams. Keep
  streaming state information --- the sensor remains in streaming state if
  any of the streams is enabled.

- Fix setting mbus code on embedded data in get_frame_desc() op in the CCS
  driver.

since v4:

- Add a patch to acquire two sub-device states that may use the same lock.

- Add a patch for CCS driver to remove ccs_get_crop_compose() helper.

- Add a patch for CCS driver moving acquiring and releasing the mutex to
  the s_stream callback.

- Add a patch for CCS driver to rely on sub-device state locking using a
  single driver-provided lock.

- Fixed calculating minimum number of routes in copying the routes
  (thanks, Laurent).

- Moved a label in S_ROUTING handling to make Clang happy (hopefully).

- Fixed setting emb_data_ctrl register for CCS embedded data support.

- Rebase on Laurent's cleanup patches.

- Wrap a few long lines.

- Write in embedded data documentation sensor drivers generally don't
  allow configuring it.

since v3:

- Separate preparation patches from this set.

- Add a definition for "Data unit", a pixel that is not image data and use
  it instead in format documentation.

- Fix more numbered lists in dev-subdev.rst.

- Remove a redundant definition for V4L2_META_FMT_GENERIC_CSI2_2_24 ---
  V4L2_META_FMT_GENERIC_CSI2_12 can be used instead.

- Use "X" instead of "p" to denote padding in format documentation.

- Use IMX219 in examples instead of CCS.

- Document that the generic V4L2 CSI-2 metadata formats use padding
  defined in CSI-2 spec and packing defined in CCS spec.

- Add patches to align [GS]_ROUTING behaviour with V4L2. This means mainly
  returning configured routes as part of S_ROUTING as well. "len_routes"
  field is added to denote the length of the array and having more routes
  than fits in the array is no longer an error. Also added more reserved
  fields.

- Added trivial support for S_ROUTING (via G_ROUTING implementation) for
  use in drivers with static-only routes.

- Added helper functions to obtain mbus format as well as crop and compose
  rectangles that are streams-independent.

- Added a patch to define generic CSI-2 long packet types.

- Removed MEDIA_BUS_FMT_IS_META() macro. It didn't seem useful in the end.

- Use a single CCS embedded data format. The bit depth can be selected
  using the meta stream on the source pad.

- Fix mbus code numbers (there were holes due to removed redundant
  formats).

- Fix generic mbus code documentation (byte was being used instead of
  bit).

- Fix spelling of "length".

- Added a patch to remove v4l2_subdev_enable_streams_api that disables
  streams API. This should be merged once libcamera support for streams
  works nicely.

- Don't use strings in printing frame descriptor flags.

- Warn on string truncation in printing frame descriptor.

since v2:

- Add a better example, with formats.

- Add CCS static data media bus codes.

- Added an example demonstrating the use of internal pads. --- Is the
  level of detail enough for the purpose?

- Improved documentation.

- Added a macro to tell whether a format is a metadata format.
  (Documentation could be added.)

- A small ReST syntax fix in the same section.

- Drop leftovers of a patch checking for the INTERNAL_SOURCE flag.

since v1:

- Make the new pad flag just "INTERNAL", requiring either SINK or SOURCE
  pad flag to accompany it. Removed the union in struct v4l2_subdev_route.

- Add the term "stream" to MC glossary.

- Improved and fixed documentation (according to comments).

- Note these formats are little endian.

- Remove 1X8 from the names of the mbus codes. These formats have generally
  8 bits per pixel.

- Fix mbus code numbering (had holes in RFC).

- Add new metadata fields to debug prints.

- Fix a minor documentation build issue.

Laurent Pinchart (6):
  media: i2c: imx219: Inline imx219_update_pad_format() in its caller
  media: i2c: imx219: Add internal image sink pad
  media: i2c: imx219: Add image stream
  media: i2c: imx219: Report internal routes to userspace
  media: i2c: imx219: Report streams using frame descriptors
  media: i2c: imx219: Add embedded data support

Sakari Ailus (60):
  media: Documentation: Clean up figure titles
  media: Documentation: Fix routing documentation flag references
  media: Documentation: There are either immutable or mutable routes
  media: Documentation: Document -ENXIO for VIDIOC_SUBDEV_S_ROUTING
  media: v4l2-subdev: Extend VIDIOC_SUBDEV_S_ROUTING error codes
  media: ccs: No need to set streaming to false in power off
  media: ccs: Move ccs_pm_get_init function up
  media: ccs: Rename out label of ccs_start_streaming
  media: ccs: Move ccs_validate_csi_data_format up
  media: ccs: Use {enable,disable}_streams operations
  media: ccs: Track streaming state
  media: ccs: Support frame descriptors
  media: mc: Add INTERNAL pad flag
  media: Documentation: Refer to internal pads in metadata documentation
  media: uapi: Add generic CSI-2 raw pixelformats
  media: Documentation: Reference color pattern control in format docs
  media: uapi: Add new media bus codes for generic raw formats
  media: uapi: Add V4L2_CID_CONFIG_MODEL control
  media: uapi: Add V4L2_CID_COLOR_PATTERN for describing color patterns
  media: Documentation: Reference COLOR_PATTERN control in raw format
    docs
  media: v4l: uapi: Add a control for color pattern flipping effect
  media: Documentation: Reference flipping controls in raw format docs
  media: Documentation: Document raw mbus codes and CFA for cameras
  media: uapi: Add V4L2_CID_METADATA_LAYOUT control
  media: Documentation: v4l: Document internal sink pads
  media: Documentation: Document embedded data guidelines for camera
    sensors
  media: uapi: ccs: Add metadata layout for MIPI CCS embedded data
  media: Documentation: Document non-CCS use of CCS embedded data layout
  media: ccs: Add support for embedded data stream
  media: Documentation: ccs: Document routing
  media: ccs: Remove ccs_get_crop_compose helper
  media: ccs: Rely on sub-device state locking
  media: ccs: Compute binning configuration from sub-device state
  media: ccs: Compute scaling configuration from sub-device state
  media: ccs: Remove which parameter from ccs_propagate
  media: ccs: Detemine emb_data_ctrl register from sub-device state
  media: uapi: Correct generic CSI-2 metadata format 4cc
  Revert "media: uapi: v4l: Don't expose generic metadata formats to
    userspace"
  media: Documentation: Add subdev configuration models, raw sensor
    model
  media: Document enable_streams and disable_streams behaviour
  media: Documentation: Add scaling and post-scaler crop for common raw
  media: uapi: Add MIPI CCS configuration model
  media: uapi: Add V4L2_CID_BINNING control for binning configuration
  media: uapi: Add controls for sub-sampling configuration
  media: Documentation: Add binning and sub-sampling controls
  media: uapi: Add metadata layout for ov2740 embedded data
  media: ov2740: Add support for embedded data
  media: ov2740: Add support for generic raw formats
  media: ov2740: Add metadata layout control
  media: ov2740: Add support for G_SELECTION IOCTL
  media: v4l: Add V4L2_SUBDEV_ROUTE_FL_IMMUTABLE sub-device routing flag
  media: v4l: Add V4L2_SUBDEV_ROUTE_FL_STATIC sub-device routing flag
  media: Documentation: Document IMMUTABLE and STATIC route flags
  media: uapi: v4l: subdev: Enable streams API
  media: ccs: Add IMMUTABLE and STATIC route flags
  media: ov2740: Add IMMUTABLE and STATIC route flags
  media: imx219: Add support for generic raw formats
  media: ccs: Add frame descriptor quirk
  media: ipu6: Add support for luma-only formats
  media: ipu6: Add support for raw CFA-agnostic formats

 .../media/drivers/camera-sensor.rst           |   71 ++
 .../userspace-api/media/drivers/ccs.rst       |   43 +-
 .../media/mediactl/media-types.rst            |    9 +
 .../media/v4l/common-raw-sensor.dia           |  442 +++++++
 .../media/v4l/common-raw-sensor.svg           |  134 ++
 .../userspace-api/media/v4l/dev-raw-vbi.rst   |    6 +-
 .../userspace-api/media/v4l/dev-subdev.rst    |  182 ++-
 .../media/v4l/ext-ctrls-camera.rst            |   40 +
 .../media/v4l/ext-ctrls-image-process.rst     |    4 +
 .../media/v4l/ext-ctrls-image-source.rst      |   76 ++
 .../userspace-api/media/v4l/meta-formats.rst  |    1 +
 .../media/v4l/metadata-layouts.rst            |  113 ++
 .../media/v4l/metafmt-generic.rst             |   44 +-
 .../media/v4l/pixfmt-raw-generic.rst          |  178 +++
 .../userspace-api/media/v4l/pixfmt.rst        |    1 +
 .../media/v4l/subdev-config-model.rst         |  260 ++++
 .../media/v4l/subdev-formats.rst              |   42 +-
 .../media/v4l/vidioc-subdev-g-routing.rst     |   21 +-
 drivers/media/i2c/ccs/ccs-core.c              | 1090 +++++++++++------
 drivers/media/i2c/ccs/ccs-quirk.h             |    7 +
 drivers/media/i2c/ccs/ccs.h                   |   26 +-
 drivers/media/i2c/imx219.c                    |  445 ++++++-
 drivers/media/i2c/ov2740.c                    |  321 ++++-
 drivers/media/mc/mc-entity.c                  |   15 +-
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c |    6 +
 .../media/pci/intel/ipu6/ipu6-isys-subdev.c   |   10 +
 .../media/pci/intel/ipu6/ipu6-isys-video.c    |   14 +-
 .../platform/nxp/imx8-isi/imx8-isi-crossbar.c |    2 +-
 .../media/platform/raspberrypi/rp1-cfe/csi2.c |    2 +-
 drivers/media/v4l2-core/v4l2-ctrls-defs.c     |   13 +
 drivers/media/v4l2-core/v4l2-ioctl.c          |   12 +-
 drivers/media/v4l2-core/v4l2-subdev.c         |   32 +-
 include/media/v4l2-subdev.h                   |    7 +
 include/uapi/linux/media-bus-format.h         |    6 +
 include/uapi/linux/media.h                    |    1 +
 include/uapi/linux/v4l2-controls.h            |   23 +
 include/uapi/linux/v4l2-subdev.h              |   10 +
 include/uapi/linux/videodev2.h                |   20 +-
 38 files changed, 3174 insertions(+), 555 deletions(-)
 create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.dia
 create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.svg
 create mode 100644 Documentation/userspace-api/media/v4l/metadata-layouts.rst
 create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
 create mode 100644 Documentation/userspace-api/media/v4l/subdev-config-model.rst

-- 
2.47.2

[PATCH v11 01/66] media: Documentation: Clean up figure titles

[Sakari Ailus]

Remove DocBook (?) conversion leftovers from figure titles, including
obsolete numbering and highlighting. This aligns these titles with others.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 Documentation/userspace-api/media/v4l/dev-raw-vbi.rst    | 6 +++---
 Documentation/userspace-api/media/v4l/dev-subdev.rst     | 6 +++---
 Documentation/userspace-api/media/v4l/subdev-formats.rst | 2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst b/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst
index 2bec20d87928..1f7bb8fd15e7 100644
--- a/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst
+++ b/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst
@@ -221,7 +221,7 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
     :alt:   vbi_hsync.svg
     :align: center
 
-    **Figure 4.1. Line synchronization**
+    Line synchronization
 
 .. _vbi-525:
 
@@ -229,7 +229,7 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
     :alt:   vbi_525.svg
     :align: center
 
-    **Figure 4.2. ITU-R 525 line numbering (M/NTSC and M/PAL)**
+    ITU-R 525 line numbering (M/NTSC and M/PAL)
 
 .. _vbi-625:
 
@@ -237,7 +237,7 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
     :alt:   vbi_625.svg
     :align: center
 
-    **Figure 4.3. ITU-R 625 line numbering**
+    ITU-R 625 line numbering
 
 Remember the VBI image format depends on the selected video standard,
 therefore the application must choose a new standard or query the
diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index 225a45fef994..24a69c419dfe 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -459,7 +459,7 @@ selection will refer to the sink pad format dimensions instead.
     :alt:   subdev-image-processing-crop.svg
     :align: center
 
-    **Figure 4.5. Image processing in subdevs: simple crop example**
+    Image processing in subdevs: simple crop example
 
 In the above example, the subdev supports cropping on its sink pad. To
 configure it, the user sets the media bus format on the subdev's sink
@@ -476,7 +476,7 @@ pad.
     :alt:   subdev-image-processing-scaling-multi-source.svg
     :align: center
 
-    **Figure 4.6. Image processing in subdevs: scaling with multiple sources**
+    Image processing in subdevs: scaling with multiple sources
 
 In this example, the subdev is capable of first cropping, then scaling
 and finally cropping for two source pads individually from the resulting
@@ -492,7 +492,7 @@ an area at location specified by the source crop rectangle from it.
     :alt:    subdev-image-processing-full.svg
     :align:  center
 
-    **Figure 4.7. Image processing in subdevs: scaling and composition with multiple sinks and sources**
+    Image processing in subdevs: scaling and composition with multiple sinks and sources
 
 The subdev driver supports two sink pads and two source pads. The images
 from both of the sink pads are individually cropped, then scaled and
diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
index 2a94371448dc..9ef1bc22ad9c 100644
--- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
@@ -2631,7 +2631,7 @@ be named ``MEDIA_BUS_FMT_SRGGB10_2X8_PADHI_LE``.
     :alt:    bayer.svg
     :align:  center
 
-    **Figure 4.8 Bayer Patterns**
+    Bayer Patterns
 
 The following table lists existing packed Bayer formats. The data
 organization is given as an example for the first pixel only.
-- 
2.47.2

Re: [PATCH v11 01/66] media: Documentation: Clean up figure titles

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:02PM +0300, Sakari Ailus wrote:
> Remove DocBook (?) conversion leftovers from figure titles, including
> obsolete numbering and highlighting. This aligns these titles with others.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

I checked the html output and indeed the comments still show up as
figure titles

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j

> ---
>  Documentation/userspace-api/media/v4l/dev-raw-vbi.rst    | 6 +++---
>  Documentation/userspace-api/media/v4l/dev-subdev.rst     | 6 +++---
>  Documentation/userspace-api/media/v4l/subdev-formats.rst | 2 +-
>  3 files changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst b/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst
> index 2bec20d87928..1f7bb8fd15e7 100644
> --- a/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst
> @@ -221,7 +221,7 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>      :alt:   vbi_hsync.svg
>      :align: center
>
> -    **Figure 4.1. Line synchronization**
> +    Line synchronization
>
>  .. _vbi-525:
>
> @@ -229,7 +229,7 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>      :alt:   vbi_525.svg
>      :align: center
>
> -    **Figure 4.2. ITU-R 525 line numbering (M/NTSC and M/PAL)**
> +    ITU-R 525 line numbering (M/NTSC and M/PAL)
>
>  .. _vbi-625:
>
> @@ -237,7 +237,7 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>      :alt:   vbi_625.svg
>      :align: center
>
> -    **Figure 4.3. ITU-R 625 line numbering**
> +    ITU-R 625 line numbering
>
>  Remember the VBI image format depends on the selected video standard,
>  therefore the application must choose a new standard or query the
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index 225a45fef994..24a69c419dfe 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -459,7 +459,7 @@ selection will refer to the sink pad format dimensions instead.
>      :alt:   subdev-image-processing-crop.svg
>      :align: center
>
> -    **Figure 4.5. Image processing in subdevs: simple crop example**
> +    Image processing in subdevs: simple crop example
>
>  In the above example, the subdev supports cropping on its sink pad. To
>  configure it, the user sets the media bus format on the subdev's sink
> @@ -476,7 +476,7 @@ pad.
>      :alt:   subdev-image-processing-scaling-multi-source.svg
>      :align: center
>
> -    **Figure 4.6. Image processing in subdevs: scaling with multiple sources**
> +    Image processing in subdevs: scaling with multiple sources
>
>  In this example, the subdev is capable of first cropping, then scaling
>  and finally cropping for two source pads individually from the resulting
> @@ -492,7 +492,7 @@ an area at location specified by the source crop rectangle from it.
>      :alt:    subdev-image-processing-full.svg
>      :align:  center
>
> -    **Figure 4.7. Image processing in subdevs: scaling and composition with multiple sinks and sources**
> +    Image processing in subdevs: scaling and composition with multiple sinks and sources
>
>  The subdev driver supports two sink pads and two source pads. The images
>  from both of the sink pads are individually cropped, then scaled and
> diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> index 2a94371448dc..9ef1bc22ad9c 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> @@ -2631,7 +2631,7 @@ be named ``MEDIA_BUS_FMT_SRGGB10_2X8_PADHI_LE``.
>      :alt:    bayer.svg
>      :align:  center
>
> -    **Figure 4.8 Bayer Patterns**
> +    Bayer Patterns
>
>  The following table lists existing packed Bayer formats. The data
>  organization is given as an example for the first pixel only.
> --
> 2.47.2
>
>

Re: [PATCH v11 01/66] media: Documentation: Clean up figure titles

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Aug 25, 2025 at 12:50:02PM +0300, Sakari Ailus wrote:
> Remove DocBook (?) conversion leftovers from figure titles, including
> obsolete numbering and highlighting. This aligns these titles with others.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  Documentation/userspace-api/media/v4l/dev-raw-vbi.rst    | 6 +++---
>  Documentation/userspace-api/media/v4l/dev-subdev.rst     | 6 +++---
>  Documentation/userspace-api/media/v4l/subdev-formats.rst | 2 +-
>  3 files changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst b/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst
> index 2bec20d87928..1f7bb8fd15e7 100644
> --- a/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-raw-vbi.rst
> @@ -221,7 +221,7 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>      :alt:   vbi_hsync.svg
>      :align: center
>  
> -    **Figure 4.1. Line synchronization**
> +    Line synchronization
>  
>  .. _vbi-525:
>  
> @@ -229,7 +229,7 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>      :alt:   vbi_525.svg
>      :align: center
>  
> -    **Figure 4.2. ITU-R 525 line numbering (M/NTSC and M/PAL)**
> +    ITU-R 525 line numbering (M/NTSC and M/PAL)
>  
>  .. _vbi-625:
>  
> @@ -237,7 +237,7 @@ and always returns default parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does
>      :alt:   vbi_625.svg
>      :align: center
>  
> -    **Figure 4.3. ITU-R 625 line numbering**
> +    ITU-R 625 line numbering
>  
>  Remember the VBI image format depends on the selected video standard,
>  therefore the application must choose a new standard or query the
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index 225a45fef994..24a69c419dfe 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -459,7 +459,7 @@ selection will refer to the sink pad format dimensions instead.
>      :alt:   subdev-image-processing-crop.svg
>      :align: center
>  
> -    **Figure 4.5. Image processing in subdevs: simple crop example**
> +    Image processing in subdevs: simple crop example
>  
>  In the above example, the subdev supports cropping on its sink pad. To
>  configure it, the user sets the media bus format on the subdev's sink
> @@ -476,7 +476,7 @@ pad.
>      :alt:   subdev-image-processing-scaling-multi-source.svg
>      :align: center
>  
> -    **Figure 4.6. Image processing in subdevs: scaling with multiple sources**
> +    Image processing in subdevs: scaling with multiple sources
>  
>  In this example, the subdev is capable of first cropping, then scaling
>  and finally cropping for two source pads individually from the resulting
> @@ -492,7 +492,7 @@ an area at location specified by the source crop rectangle from it.
>      :alt:    subdev-image-processing-full.svg
>      :align:  center
>  
> -    **Figure 4.7. Image processing in subdevs: scaling and composition with multiple sinks and sources**
> +    Image processing in subdevs: scaling and composition with multiple sinks and sources
>  
>  The subdev driver supports two sink pads and two source pads. The images
>  from both of the sink pads are individually cropped, then scaled and
> diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> index 2a94371448dc..9ef1bc22ad9c 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> @@ -2631,7 +2631,7 @@ be named ``MEDIA_BUS_FMT_SRGGB10_2X8_PADHI_LE``.
>      :alt:    bayer.svg
>      :align:  center
>  
> -    **Figure 4.8 Bayer Patterns**
> +    Bayer Patterns
>  
>  The following table lists existing packed Bayer formats. The data
>  organization is given as an example for the first pixel only.

-- 
Regards,

Laurent Pinchart

[PATCH v11 02/66] media: Documentation: Fix routing documentation flag references

[Sakari Ailus]

The routing flag indicating an active route is called
V4L2_SUBDEV_ROUTE_FL_ACTIVE, not V4L2_SUBDEV_STREAM_FL_ACTIVE. Fix this.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 Documentation/userspace-api/media/v4l/dev-subdev.rst | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index 24a69c419dfe..f30a98a9cf78 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -577,15 +577,14 @@ Device types and routing setup
 
 Different kinds of sub-devices have differing behaviour for route activation,
 depending on the hardware. In all cases, however, only routes that have the
-``V4L2_SUBDEV_STREAM_FL_ACTIVE`` flag set are active.
+``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag set are active.
 
 Devices generating the streams may allow enabling and disabling some of the
 routes or have a fixed routing configuration. If the routes can be disabled, not
-declaring the routes (or declaring them without
-``V4L2_SUBDEV_STREAM_FL_ACTIVE`` flag set) in ``VIDIOC_SUBDEV_S_ROUTING`` will
-disable the routes. ``VIDIOC_SUBDEV_S_ROUTING`` will still return such routes
-back to the user in the routes array, with the ``V4L2_SUBDEV_STREAM_FL_ACTIVE``
-flag unset.
+declaring the routes (or declaring them without ``V4L2_SUBDEV_ROUTE_FL_ACTIVE``
+flag set) in ``VIDIOC_SUBDEV_S_ROUTING`` will disable the routes.
+``VIDIOC_SUBDEV_S_ROUTING`` will still return such routes back to the user in
+the routes array, with the ``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag unset.
 
 Devices transporting the streams almost always have more configurability with
 respect to routing. Typically any route between the sub-device's sink and source
-- 
2.47.2

Re: [PATCH v11 02/66] media: Documentation: Fix routing documentation flag references

[Jacopo Mondi]

Hi Sakari,

On Mon, Aug 25, 2025 at 12:50:03PM +0300, Sakari Ailus wrote:
> The routing flag indicating an active route is called
> V4L2_SUBDEV_ROUTE_FL_ACTIVE, not V4L2_SUBDEV_STREAM_FL_ACTIVE. Fix this.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>


> ---
>  Documentation/userspace-api/media/v4l/dev-subdev.rst | 11 +++++------
>  1 file changed, 5 insertions(+), 6 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index 24a69c419dfe..f30a98a9cf78 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -577,15 +577,14 @@ Device types and routing setup
>
>  Different kinds of sub-devices have differing behaviour for route activation,
>  depending on the hardware. In all cases, however, only routes that have the
> -``V4L2_SUBDEV_STREAM_FL_ACTIVE`` flag set are active.
> +``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag set are active.
>
>  Devices generating the streams may allow enabling and disabling some of the
>  routes or have a fixed routing configuration. If the routes can be disabled, not
> -declaring the routes (or declaring them without
> -``V4L2_SUBDEV_STREAM_FL_ACTIVE`` flag set) in ``VIDIOC_SUBDEV_S_ROUTING`` will
> -disable the routes. ``VIDIOC_SUBDEV_S_ROUTING`` will still return such routes
> -back to the user in the routes array, with the ``V4L2_SUBDEV_STREAM_FL_ACTIVE``
> -flag unset.
> +declaring the routes (or declaring them without ``V4L2_SUBDEV_ROUTE_FL_ACTIVE``
> +flag set) in ``VIDIOC_SUBDEV_S_ROUTING`` will disable the routes.
> +``VIDIOC_SUBDEV_S_ROUTING`` will still return such routes back to the user in
> +the routes array, with the ``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag unset.
>
>  Devices transporting the streams almost always have more configurability with
>  respect to routing. Typically any route between the sub-device's sink and source
> --
> 2.47.2
>
>

Re: [PATCH v11 02/66] media: Documentation: Fix routing documentation flag references

[Laurent Pinchart]

Hi Sakari,

On Mon, Aug 25, 2025 at 12:50:03PM +0300, Sakari Ailus wrote:
> The routing flag indicating an active route is called
> V4L2_SUBDEV_ROUTE_FL_ACTIVE, not V4L2_SUBDEV_STREAM_FL_ACTIVE. Fix this.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  Documentation/userspace-api/media/v4l/dev-subdev.rst | 11 +++++------
>  1 file changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index 24a69c419dfe..f30a98a9cf78 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -577,15 +577,14 @@ Device types and routing setup
>  
>  Different kinds of sub-devices have differing behaviour for route activation,
>  depending on the hardware. In all cases, however, only routes that have the
> -``V4L2_SUBDEV_STREAM_FL_ACTIVE`` flag set are active.
> +``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag set are active.
>  
>  Devices generating the streams may allow enabling and disabling some of the
>  routes or have a fixed routing configuration. If the routes can be disabled, not
> -declaring the routes (or declaring them without
> -``V4L2_SUBDEV_STREAM_FL_ACTIVE`` flag set) in ``VIDIOC_SUBDEV_S_ROUTING`` will
> -disable the routes. ``VIDIOC_SUBDEV_S_ROUTING`` will still return such routes
> -back to the user in the routes array, with the ``V4L2_SUBDEV_STREAM_FL_ACTIVE``
> -flag unset.
> +declaring the routes (or declaring them without ``V4L2_SUBDEV_ROUTE_FL_ACTIVE``
> +flag set) in ``VIDIOC_SUBDEV_S_ROUTING`` will disable the routes.
> +``VIDIOC_SUBDEV_S_ROUTING`` will still return such routes back to the user in
> +the routes array, with the ``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag unset.
>  
>  Devices transporting the streams almost always have more configurability with
>  respect to routing. Typically any route between the sub-device's sink and source

-- 
Regards,

Laurent Pinchart

[PATCH v11 03/66] media: Documentation: There are either immutable or mutable routes

[Sakari Ailus]

Document that each sub-device may have either immutable or mutable routes,
not both of them.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 Documentation/userspace-api/media/v4l/dev-subdev.rst | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index f30a98a9cf78..4da67ee0b290 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -594,6 +594,11 @@ and user-created routes are fully replaced when ``VIDIOC_SUBDEV_S_ROUTING`` is
 called on the sub-device. Such newly created routes have the device's default
 configuration for format and selection rectangles.
 
+A sub-device may only have either immutable routes (routes that have
+``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``) flag set or routes that are all user-created
+or user-removable (routes that do not have ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``
+flag). This is subject to change in the future.
+
 Configuring streams
 -------------------
 
-- 
2.47.2

Re: [PATCH v11 03/66] media: Documentation: There are either immutable or mutable routes

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:04PM +0300, Sakari Ailus wrote:
> Document that each sub-device may have either immutable or mutable routes,
> not both of them.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  Documentation/userspace-api/media/v4l/dev-subdev.rst | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index f30a98a9cf78..4da67ee0b290 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -594,6 +594,11 @@ and user-created routes are fully replaced when ``VIDIOC_SUBDEV_S_ROUTING`` is
>  called on the sub-device. Such newly created routes have the device's default
>  configuration for format and selection rectangles.
>
> +A sub-device may only have either immutable routes (routes that have
> +``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``) flag set or routes that are all user-created

The closing ) should go after 'flag set'

> +or user-removable (routes that do not have ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``
> +flag). This is subject to change in the future.

I take this as:
1) Either all routes are created by the driver and have the
  V4L2_SUBDEV_ROUTE_FL_IMMUTABLE flag set
2) Or all routes are user created (and do not have the
   V4L2_SUBDEV_ROUTE_FL_IMMUTABLE flag set)

Is this intended ? Can't we have routes that are driver created and
are immutable and routes created by the user on the same sub-device ?

> +
>  Configuring streams
>  -------------------
>
> --
> 2.47.2
>
>

Re: [PATCH v11 03/66] media: Documentation: There are either immutable or mutable routes

[Sakari Ailus]

Hi Jacopo,

Thank you for the review.

On Mon, Sep 01, 2025 at 01:18:58PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:04PM +0300, Sakari Ailus wrote:
> > Document that each sub-device may have either immutable or mutable routes,
> > not both of them.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  Documentation/userspace-api/media/v4l/dev-subdev.rst | 5 +++++
> >  1 file changed, 5 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > index f30a98a9cf78..4da67ee0b290 100644
> > --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > @@ -594,6 +594,11 @@ and user-created routes are fully replaced when ``VIDIOC_SUBDEV_S_ROUTING`` is
> >  called on the sub-device. Such newly created routes have the device's default
> >  configuration for format and selection rectangles.
> >
> > +A sub-device may only have either immutable routes (routes that have
> > +``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``) flag set or routes that are all user-created
> 
> The closing ) should go after 'flag set'

Ack.

> 
> > +or user-removable (routes that do not have ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``
> > +flag). This is subject to change in the future.
> 
> I take this as:
> 1) Either all routes are created by the driver and have the
>   V4L2_SUBDEV_ROUTE_FL_IMMUTABLE flag set
> 2) Or all routes are user created (and do not have the
>    V4L2_SUBDEV_ROUTE_FL_IMMUTABLE flag set)
> 
> Is this intended ? Can't we have routes that are driver created and
> are immutable and routes created by the user on the same sub-device ?

We could allow that later on but as I don't think we have a use case yet,
I'd explicitly deny that for the time being.

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v11 03/66] media: Documentation: There are either immutable or mutable routes

[Laurent Pinchart]

On Mon, Aug 25, 2025 at 12:50:04PM +0300, Sakari Ailus wrote:
> Document that each sub-device may have either immutable or mutable routes,
> not both of them.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  Documentation/userspace-api/media/v4l/dev-subdev.rst | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index f30a98a9cf78..4da67ee0b290 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -594,6 +594,11 @@ and user-created routes are fully replaced when ``VIDIOC_SUBDEV_S_ROUTING`` is
>  called on the sub-device. Such newly created routes have the device's default
>  configuration for format and selection rectangles.
>  
> +A sub-device may only have either immutable routes (routes that have
> +``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``) flag set or routes that are all user-created

``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE`` flag set) or routes that are all user-created

> +or user-removable (routes that do not have ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``
> +flag). This is subject to change in the future.

I'd add

Mixing immutable and non-immutable routes is not allowed.

between the two sentences.

But... that's actually not right. An immutable route is always active.
What we don't allow is combining driver-created "static" routes (as in
routes that can be disabled but can't be removed) and user-created
"dynamic" routes.

> +
>  Configuring streams
>  -------------------
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH v11 03/66] media: Documentation: There are either immutable or mutable routes

[Sakari Ailus]

On Wed, Sep 03, 2025 at 03:22:26PM +0200, Laurent Pinchart wrote:
> On Mon, Aug 25, 2025 at 12:50:04PM +0300, Sakari Ailus wrote:
> > Document that each sub-device may have either immutable or mutable routes,
> > not both of them.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  Documentation/userspace-api/media/v4l/dev-subdev.rst | 5 +++++
> >  1 file changed, 5 insertions(+)
> > 
> > diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > index f30a98a9cf78..4da67ee0b290 100644
> > --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > @@ -594,6 +594,11 @@ and user-created routes are fully replaced when ``VIDIOC_SUBDEV_S_ROUTING`` is
> >  called on the sub-device. Such newly created routes have the device's default
> >  configuration for format and selection rectangles.
> >  
> > +A sub-device may only have either immutable routes (routes that have
> > +``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``) flag set or routes that are all user-created
> 
> ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE`` flag set) or routes that are all user-created
> 
> > +or user-removable (routes that do not have ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``
> > +flag). This is subject to change in the future.
> 
> I'd add
> 
> Mixing immutable and non-immutable routes is not allowed.
> 
> between the two sentences.
> 
> But... that's actually not right. An immutable route is always active.
> What we don't allow is combining driver-created "static" routes (as in
> routes that can be disabled but can't be removed) and user-created
> "dynamic" routes.

Indeed, this should be discussing the STATIC flag. The two aren't the same
even though often set (or unset) in the same occasion.

-- 
Sakari Ailus

Re: [PATCH v11 03/66] media: Documentation: There are either immutable or mutable routes

[Sakari Ailus]

Hi Laurent,

On Wed, Sep 03, 2025 at 03:22:26PM +0200, Laurent Pinchart wrote:
> On Mon, Aug 25, 2025 at 12:50:04PM +0300, Sakari Ailus wrote:
> > Document that each sub-device may have either immutable or mutable routes,
> > not both of them.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  Documentation/userspace-api/media/v4l/dev-subdev.rst | 5 +++++
> >  1 file changed, 5 insertions(+)
> > 
> > diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > index f30a98a9cf78..4da67ee0b290 100644
> > --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > @@ -594,6 +594,11 @@ and user-created routes are fully replaced when ``VIDIOC_SUBDEV_S_ROUTING`` is
> >  called on the sub-device. Such newly created routes have the device's default
> >  configuration for format and selection rectangles.
> >  
> > +A sub-device may only have either immutable routes (routes that have
> > +``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``) flag set or routes that are all user-created
> 
> ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE`` flag set) or routes that are all user-created
> 
> > +or user-removable (routes that do not have ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``
> > +flag). This is subject to change in the future.
> 
> I'd add
> 
> Mixing immutable and non-immutable routes is not allowed.
> 
> between the two sentences.

This became:

Mixing static and non-static routes is not allowed in the same sub-device.

-- 
Sakari Ailus

[PATCH v11 04/66] media: Documentation: Document -ENXIO for VIDIOC_SUBDEV_S_ROUTING

[Sakari Ailus]

Document that -ENXIO is returned when the user tries to set a routing
configuration not supported by the hardware (or rather the driver). The
documentation details the exact cases of this, besides -EINVAL that is
already documented for VIDIOC_SUBDEV_S_ROUTING.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../media/v4l/vidioc-subdev-g-routing.rst             | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
index 1cf795480602..15f448664e6b 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
@@ -155,9 +155,18 @@ appropriately. The generic error codes are described at the
 :ref:`Generic Error Codes <gen-errors>` chapter.
 
 EINVAL
+
    The sink or source pad identifiers reference a non-existing pad or reference
    pads of different types (ie. the sink_pad identifiers refers to a source
-   pad), or the ``which`` field has an unsupported value.
+   pad), the ``which`` field has an unsupported value, or, for
+   ``VIDIOC_SUBDEV_S_ROUTING``, the num_routes field value is larger than that
+   of the len_routes field.
+
+ENXIO
+   No such link can be created or such link state change can be made. Either the
+   sink or source (pad, stream) pair or the combination of the sink and source
+   is not supported by the hardware, or no multiple routes from or to the same
+   (pad, stream) pair are supported.
 
 E2BIG
    The application provided ``num_routes`` for ``VIDIOC_SUBDEV_S_ROUTING`` is
-- 
2.47.2

Re: [PATCH v11 04/66] media: Documentation: Document -ENXIO for VIDIOC_SUBDEV_S_ROUTING

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:05PM +0300, Sakari Ailus wrote:
> Document that -ENXIO is returned when the user tries to set a routing
> configuration not supported by the hardware (or rather the driver). The
> documentation details the exact cases of this, besides -EINVAL that is
> already documented for VIDIOC_SUBDEV_S_ROUTING.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../media/v4l/vidioc-subdev-g-routing.rst             | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> index 1cf795480602..15f448664e6b 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> @@ -155,9 +155,18 @@ appropriately. The generic error codes are described at the
>  :ref:`Generic Error Codes <gen-errors>` chapter.
>
>  EINVAL
> +

Is this intended ?

>     The sink or source pad identifiers reference a non-existing pad or reference
>     pads of different types (ie. the sink_pad identifiers refers to a source
> -   pad), or the ``which`` field has an unsupported value.
> +   pad), the ``which`` field has an unsupported value, or, for
> +   ``VIDIOC_SUBDEV_S_ROUTING``, the num_routes field value is larger than that
> +   of the len_routes field.

Doesn't this contradicts the above

The kernel can return a ``num_routes`` value larger than ``len_routes`` from
both ioctls. This indicates thare are more routes in the routing table than fits
the ``routes`` array. In this case, the ``routes`` array is filled by the kernel
with the first ``len_routes`` entries of the subdevice routing table. This is
not considered to be an error, and the ioctl call succeeds. If the applications
wants to retrieve the missing routes, it can issue a new
``VIDIOC_SUBDEV_G_ROUTING`` call with a large enough ``routes`` array.

Looking at the handler for VIDIOC_SUBDEV_S_ROUTING in v4l2-subdev.c I
think this should specify that is invalid to have userspace set a num_routes
value larger than len_routes.

I would simply add this to the above hunk

-   pad), or the ``which`` field has an unsupported value.
+   pad), the ``which`` field has an unsupported value, or, for
+   ``VIDIOC_SUBDEV_S_ROUTING``, the num_routes field provided by the
+   application is larger than the len_routes field value.

> +
> +ENXIO
> +   No such link can be created or such link state change can be made. Either the
> +   sink or source (pad, stream) pair or the combination of the sink and source
> +   is not supported by the hardware, or no multiple routes from or to the same
> +   (pad, stream) pair are supported.

If I'm not mistaken this only applies to VIDIOC_SUBDEV_S_ROUTING, so I
would mention that. Also ENXIO is not returned by the core but by
drivers, so I would not limit the possible cases where ENXIO is
returned by making examples.

What about a simpler:

+ENXIO
+   The application requested routes cannot be created or the state of
+   the specified routes cannot be modified. Only returned for
+   ``VIDIOC_SUBDEV_S_ROUTING``.

>
>  E2BIG
>     The application provided ``num_routes`` for ``VIDIOC_SUBDEV_S_ROUTING`` is
> --
> 2.47.2
>
>

Re: [PATCH v11 04/66] media: Documentation: Document -ENXIO for VIDIOC_SUBDEV_S_ROUTING

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 01:33:56PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:05PM +0300, Sakari Ailus wrote:
> > Document that -ENXIO is returned when the user tries to set a routing
> > configuration not supported by the hardware (or rather the driver). The
> > documentation details the exact cases of this, besides -EINVAL that is
> > already documented for VIDIOC_SUBDEV_S_ROUTING.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../media/v4l/vidioc-subdev-g-routing.rst             | 11 ++++++++++-
> >  1 file changed, 10 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> > index 1cf795480602..15f448664e6b 100644
> > --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> > +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> > @@ -155,9 +155,18 @@ appropriately. The generic error codes are described at the
> >  :ref:`Generic Error Codes <gen-errors>` chapter.
> >
> >  EINVAL
> > +
> 
> Is this intended ?

Oops.

> 
> >     The sink or source pad identifiers reference a non-existing pad or reference
> >     pads of different types (ie. the sink_pad identifiers refers to a source
> > -   pad), or the ``which`` field has an unsupported value.
> > +   pad), the ``which`` field has an unsupported value, or, for
> > +   ``VIDIOC_SUBDEV_S_ROUTING``, the num_routes field value is larger than that
> > +   of the len_routes field.
> 
> Doesn't this contradicts the above
> 
> The kernel can return a ``num_routes`` value larger than ``len_routes`` from
> both ioctls. This indicates thare are more routes in the routing table than fits
> the ``routes`` array. In this case, the ``routes`` array is filled by the kernel
> with the first ``len_routes`` entries of the subdevice routing table. This is
> not considered to be an error, and the ioctl call succeeds. If the applications
> wants to retrieve the missing routes, it can issue a new
> ``VIDIOC_SUBDEV_G_ROUTING`` call with a large enough ``routes`` array.
> 
> Looking at the handler for VIDIOC_SUBDEV_S_ROUTING in v4l2-subdev.c I
> think this should specify that is invalid to have userspace set a num_routes
> value larger than len_routes.
> 
> I would simply add this to the above hunk
> 
> -   pad), or the ``which`` field has an unsupported value.
> +   pad), the ``which`` field has an unsupported value, or, for
> +   ``VIDIOC_SUBDEV_S_ROUTING``, the num_routes field provided by the
> +   application is larger than the len_routes field value.

s/provided/set/ ?

Looks good to me.

> 
> > +
> > +ENXIO
> > +   No such link can be created or such link state change can be made. Either the
> > +   sink or source (pad, stream) pair or the combination of the sink and source
> > +   is not supported by the hardware, or no multiple routes from or to the same
> > +   (pad, stream) pair are supported.
> 
> If I'm not mistaken this only applies to VIDIOC_SUBDEV_S_ROUTING, so I
> would mention that. Also ENXIO is not returned by the core but by
> drivers, so I would not limit the possible cases where ENXIO is
> returned by making examples.
> 
> What about a simpler:
> 
> +ENXIO
> +   The application requested routes cannot be created or the state of
> +   the specified routes cannot be modified. Only returned for
> +   ``VIDIOC_SUBDEV_S_ROUTING``.

Sounds reasonable. I'll use this in the next version.

> 
> >
> >  E2BIG
> >     The application provided ``num_routes`` for ``VIDIOC_SUBDEV_S_ROUTING`` is

-- 
Regards,

Sakari Ailus

Re: [PATCH v11 04/66] media: Documentation: Document -ENXIO for VIDIOC_SUBDEV_S_ROUTING

[Laurent Pinchart]

On Tue, Sep 02, 2025 at 01:00:20PM +0300, Sakari Ailus wrote:
> On Mon, Sep 01, 2025 at 01:33:56PM +0200, Jacopo Mondi wrote:
> > On Mon, Aug 25, 2025 at 12:50:05PM +0300, Sakari Ailus wrote:
> > > Document that -ENXIO is returned when the user tries to set a routing
> > > configuration not supported by the hardware (or rather the driver). The
> > > documentation details the exact cases of this, besides -EINVAL that is
> > > already documented for VIDIOC_SUBDEV_S_ROUTING.
> > >
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  .../media/v4l/vidioc-subdev-g-routing.rst             | 11 ++++++++++-
> > >  1 file changed, 10 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> > > index 1cf795480602..15f448664e6b 100644
> > > --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> > > +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> > > @@ -155,9 +155,18 @@ appropriately. The generic error codes are described at the
> > >  :ref:`Generic Error Codes <gen-errors>` chapter.
> > >
> > >  EINVAL
> > > +
> > 
> > Is this intended ?
> 
> Oops.
> 
> > >     The sink or source pad identifiers reference a non-existing pad or reference
> > >     pads of different types (ie. the sink_pad identifiers refers to a source
> > > -   pad), or the ``which`` field has an unsupported value.
> > > +   pad), the ``which`` field has an unsupported value, or, for
> > > +   ``VIDIOC_SUBDEV_S_ROUTING``, the num_routes field value is larger than that
> > > +   of the len_routes field.
> > 
> > Doesn't this contradicts the above
> > 
> > The kernel can return a ``num_routes`` value larger than ``len_routes`` from
> > both ioctls. This indicates thare are more routes in the routing table than fits
> > the ``routes`` array. In this case, the ``routes`` array is filled by the kernel
> > with the first ``len_routes`` entries of the subdevice routing table. This is
> > not considered to be an error, and the ioctl call succeeds. If the applications
> > wants to retrieve the missing routes, it can issue a new
> > ``VIDIOC_SUBDEV_G_ROUTING`` call with a large enough ``routes`` array.
> > 
> > Looking at the handler for VIDIOC_SUBDEV_S_ROUTING in v4l2-subdev.c I
> > think this should specify that is invalid to have userspace set a num_routes
> > value larger than len_routes.
> > 
> > I would simply add this to the above hunk
> > 
> > -   pad), or the ``which`` field has an unsupported value.
> > +   pad), the ``which`` field has an unsupported value, or, for
> > +   ``VIDIOC_SUBDEV_S_ROUTING``, the num_routes field provided by the
> > +   application is larger than the len_routes field value.
> 
> s/provided/set/ ?
> 
> Looks good to me.
> 
> > > +
> > > +ENXIO
> > > +   No such link can be created or such link state change can be made. Either the
> > > +   sink or source (pad, stream) pair or the combination of the sink and source
> > > +   is not supported by the hardware, or no multiple routes from or to the same
> > > +   (pad, stream) pair are supported.
> > 
> > If I'm not mistaken this only applies to VIDIOC_SUBDEV_S_ROUTING, so I
> > would mention that. Also ENXIO is not returned by the core but by
> > drivers, so I would not limit the possible cases where ENXIO is
> > returned by making examples.
> > 
> > What about a simpler:
> > 
> > +ENXIO
> > +   The application requested routes cannot be created or the state of
> > +   the specified routes cannot be modified. Only returned for
> > +   ``VIDIOC_SUBDEV_S_ROUTING``.

This sounds fairly vague. I'd prefer keeping the description of the
error conditions, but turning them from an exhaustive list into
examples. How about the following ?

ENXIO
   The routes specified by the application can't be applied because they violate
   the driver's validity requirements. For instance, the application may be
   trying to disable an immutable route, create a route that is not supported by
   the device, or that conflicts with other routes. Only returned for
   ``VIDIOC_SUBDEV_S_ROUTING``.

I'm also not too sure about ENXIO ("No such device or address"). There's
no other error code that immediately strikes me as a good match though.
If I were to propose a hack, I'd say EBADR could be used to mean "bad
route" instead of "bad request" :-)

> Sounds reasonable. I'll use this in the next version.
> 
> > >
> > >  E2BIG
> > >     The application provided ``num_routes`` for ``VIDIOC_SUBDEV_S_ROUTING`` is

-- 
Regards,

Laurent Pinchart

Re: [PATCH v11 04/66] media: Documentation: Document -ENXIO for VIDIOC_SUBDEV_S_ROUTING

[Sakari Ailus]

On Wed, Sep 03, 2025 at 03:47:55PM +0200, Laurent Pinchart wrote:
> On Tue, Sep 02, 2025 at 01:00:20PM +0300, Sakari Ailus wrote:
> > On Mon, Sep 01, 2025 at 01:33:56PM +0200, Jacopo Mondi wrote:
> > > On Mon, Aug 25, 2025 at 12:50:05PM +0300, Sakari Ailus wrote:
> > > > Document that -ENXIO is returned when the user tries to set a routing
> > > > configuration not supported by the hardware (or rather the driver). The
> > > > documentation details the exact cases of this, besides -EINVAL that is
> > > > already documented for VIDIOC_SUBDEV_S_ROUTING.
> > > >
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > ---
> > > >  .../media/v4l/vidioc-subdev-g-routing.rst             | 11 ++++++++++-
> > > >  1 file changed, 10 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> > > > index 1cf795480602..15f448664e6b 100644
> > > > --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> > > > @@ -155,9 +155,18 @@ appropriately. The generic error codes are described at the
> > > >  :ref:`Generic Error Codes <gen-errors>` chapter.
> > > >
> > > >  EINVAL
> > > > +
> > > 
> > > Is this intended ?
> > 
> > Oops.
> > 
> > > >     The sink or source pad identifiers reference a non-existing pad or reference
> > > >     pads of different types (ie. the sink_pad identifiers refers to a source
> > > > -   pad), or the ``which`` field has an unsupported value.
> > > > +   pad), the ``which`` field has an unsupported value, or, for
> > > > +   ``VIDIOC_SUBDEV_S_ROUTING``, the num_routes field value is larger than that
> > > > +   of the len_routes field.
> > > 
> > > Doesn't this contradicts the above
> > > 
> > > The kernel can return a ``num_routes`` value larger than ``len_routes`` from
> > > both ioctls. This indicates thare are more routes in the routing table than fits
> > > the ``routes`` array. In this case, the ``routes`` array is filled by the kernel
> > > with the first ``len_routes`` entries of the subdevice routing table. This is
> > > not considered to be an error, and the ioctl call succeeds. If the applications
> > > wants to retrieve the missing routes, it can issue a new
> > > ``VIDIOC_SUBDEV_G_ROUTING`` call with a large enough ``routes`` array.
> > > 
> > > Looking at the handler for VIDIOC_SUBDEV_S_ROUTING in v4l2-subdev.c I
> > > think this should specify that is invalid to have userspace set a num_routes
> > > value larger than len_routes.
> > > 
> > > I would simply add this to the above hunk
> > > 
> > > -   pad), or the ``which`` field has an unsupported value.
> > > +   pad), the ``which`` field has an unsupported value, or, for
> > > +   ``VIDIOC_SUBDEV_S_ROUTING``, the num_routes field provided by the
> > > +   application is larger than the len_routes field value.
> > 
> > s/provided/set/ ?
> > 
> > Looks good to me.
> > 
> > > > +
> > > > +ENXIO
> > > > +   No such link can be created or such link state change can be made. Either the
> > > > +   sink or source (pad, stream) pair or the combination of the sink and source
> > > > +   is not supported by the hardware, or no multiple routes from or to the same
> > > > +   (pad, stream) pair are supported.
> > > 
> > > If I'm not mistaken this only applies to VIDIOC_SUBDEV_S_ROUTING, so I
> > > would mention that. Also ENXIO is not returned by the core but by
> > > drivers, so I would not limit the possible cases where ENXIO is
> > > returned by making examples.
> > > 
> > > What about a simpler:
> > > 
> > > +ENXIO
> > > +   The application requested routes cannot be created or the state of
> > > +   the specified routes cannot be modified. Only returned for
> > > +   ``VIDIOC_SUBDEV_S_ROUTING``.
> 
> This sounds fairly vague. I'd prefer keeping the description of the
> error conditions, but turning them from an exhaustive list into
> examples. How about the following ?
> 
> ENXIO
>    The routes specified by the application can't be applied because they violate
>    the driver's validity requirements. For instance, the application may be
>    trying to disable an immutable route, create a route that is not supported by
>    the device, or that conflicts with other routes. Only returned for
>    ``VIDIOC_SUBDEV_S_ROUTING``.
> 
> I'm also not too sure about ENXIO ("No such device or address"). There's
> no other error code that immediately strikes me as a good match though.
> If I were to propose a hack, I'd say EBADR could be used to mean "bad
> route" instead of "bad request" :-)

:-) What I think really counts is that it's different from EINVAL, albeit
I'm open to better suggestions. We could use e.g. EBADSLT, at least it'd
be easy to spot. Same goes for EBADR probably.

-- 
Sakari Ailus

[PATCH v11 05/66] media: v4l2-subdev: Extend VIDIOC_SUBDEV_S_ROUTING error codes

[Sakari Ailus]

Return -ENXIO from routing configuration errors other than generic IOCTL
argument validation.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 2 +-
 drivers/media/platform/raspberrypi/rp1-cfe/csi2.c       | 2 +-
 drivers/media/v4l2-core/v4l2-subdev.c                   | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
index ede6cc74c023..3c26cfef93d1 100644
--- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
+++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
@@ -114,7 +114,7 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
 				"invalid route from memory input (%u) to pipe %u\n",
 				route->sink_pad,
 				route->source_pad - xbar->num_sinks);
-			return -EINVAL;
+			return -ENXIO;
 		}
 	}
 
diff --git a/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c b/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
index 35c2ab1e2cd4..997b75aa5958 100644
--- a/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
+++ b/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
@@ -492,7 +492,7 @@ static int csi2_set_routing(struct v4l2_subdev *sd,
 		const struct v4l2_subdev_route *route = &routing->routes[i];
 
 		if (route->source_stream != 0)
-			return -EINVAL;
+			return -ENXIO;
 	}
 
 	ret = v4l2_subdev_set_routing_with_fmt(sd, state, routing,
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 1da953629010..8d4c484109fb 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2112,7 +2112,7 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
 {
 	u32 *remote_pads = NULL;
 	unsigned int i, j;
-	int ret = -EINVAL;
+	int ret = -ENXIO;
 
 	if (disallow & (V4L2_SUBDEV_ROUTING_NO_STREAM_MIX |
 			V4L2_SUBDEV_ROUTING_NO_MULTIPLEXING)) {
-- 
2.47.2

Re: [PATCH v11 05/66] media: v4l2-subdev: Extend VIDIOC_SUBDEV_S_ROUTING error codes

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:06PM +0300, Sakari Ailus wrote:
> Return -ENXIO from routing configuration errors other than generic IOCTL
> argument validation.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j

> ---
>  drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 2 +-
>  drivers/media/platform/raspberrypi/rp1-cfe/csi2.c       | 2 +-
>  drivers/media/v4l2-core/v4l2-subdev.c                   | 2 +-
>  3 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> index ede6cc74c023..3c26cfef93d1 100644
> --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> @@ -114,7 +114,7 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
>  				"invalid route from memory input (%u) to pipe %u\n",
>  				route->sink_pad,
>  				route->source_pad - xbar->num_sinks);
> -			return -EINVAL;
> +			return -ENXIO;
>  		}
>  	}
>
> diff --git a/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c b/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
> index 35c2ab1e2cd4..997b75aa5958 100644
> --- a/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
> +++ b/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
> @@ -492,7 +492,7 @@ static int csi2_set_routing(struct v4l2_subdev *sd,
>  		const struct v4l2_subdev_route *route = &routing->routes[i];
>
>  		if (route->source_stream != 0)
> -			return -EINVAL;
> +			return -ENXIO;
>  	}
>
>  	ret = v4l2_subdev_set_routing_with_fmt(sd, state, routing,
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 1da953629010..8d4c484109fb 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2112,7 +2112,7 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
>  {
>  	u32 *remote_pads = NULL;
>  	unsigned int i, j;
> -	int ret = -EINVAL;
> +	int ret = -ENXIO;
>
>  	if (disallow & (V4L2_SUBDEV_ROUTING_NO_STREAM_MIX |
>  			V4L2_SUBDEV_ROUTING_NO_MULTIPLEXING)) {
> --
> 2.47.2
>
>

Re: [PATCH v11 05/66] media: v4l2-subdev: Extend VIDIOC_SUBDEV_S_ROUTING error codes

[Laurent Pinchart]

Hi Sakari,

On Mon, Aug 25, 2025 at 12:50:06PM +0300, Sakari Ailus wrote:
> Return -ENXIO from routing configuration errors other than generic IOCTL
> argument validation.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 2 +-
>  drivers/media/platform/raspberrypi/rp1-cfe/csi2.c       | 2 +-
>  drivers/media/v4l2-core/v4l2-subdev.c                   | 2 +-
>  3 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> index ede6cc74c023..3c26cfef93d1 100644
> --- a/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> +++ b/drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c
> @@ -114,7 +114,7 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
>  				"invalid route from memory input (%u) to pipe %u\n",
>  				route->sink_pad,
>  				route->source_pad - xbar->num_sinks);
> -			return -EINVAL;
> +			return -ENXIO;
>  		}
>  	}
>  
> diff --git a/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c b/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
> index 35c2ab1e2cd4..997b75aa5958 100644
> --- a/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
> +++ b/drivers/media/platform/raspberrypi/rp1-cfe/csi2.c
> @@ -492,7 +492,7 @@ static int csi2_set_routing(struct v4l2_subdev *sd,
>  		const struct v4l2_subdev_route *route = &routing->routes[i];
>  
>  		if (route->source_stream != 0)
> -			return -EINVAL;
> +			return -ENXIO;
>  	}
>  
>  	ret = v4l2_subdev_set_routing_with_fmt(sd, state, routing,
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 1da953629010..8d4c484109fb 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2112,7 +2112,7 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
>  {
>  	u32 *remote_pads = NULL;
>  	unsigned int i, j;
> -	int ret = -EINVAL;
> +	int ret = -ENXIO;
>  
>  	if (disallow & (V4L2_SUBDEV_ROUTING_NO_STREAM_MIX |
>  			V4L2_SUBDEV_ROUTING_NO_MULTIPLEXING)) {

-- 
Regards,

Laurent Pinchart

[PATCH v11 06/66] media: ccs: No need to set streaming to false in power off

[Sakari Ailus]

Streaming will have been stopped before the sensor is powered off, and so
sensor->streaming is also false already. Do not set it as part of the
runtime suspend callback.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 487bcabb4a19..48d1f49781ea 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -1706,7 +1706,6 @@ static int ccs_power_off(struct device *dev)
 	usleep_range(5000, 5000);
 	regulator_bulk_disable(ARRAY_SIZE(ccs_regulators),
 			       sensor->regulators);
-	sensor->streaming = false;
 
 	return 0;
 }
-- 
2.47.2

[PATCH v11 07/66] media: ccs: Move ccs_pm_get_init function up

[Sakari Ailus]

Prepare for using ccs_pm_get_init from locations earlier than its the
current place.

Also add a missing newline while at it.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 73 ++++++++++++++++----------------
 1 file changed, 37 insertions(+), 36 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 48d1f49781ea..2a55af67ed13 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -1714,6 +1714,43 @@ static int ccs_power_off(struct device *dev)
  * Video stream management
  */
 
+static int ccs_pm_get_init(struct ccs_sensor *sensor)
+{
+	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
+	int rval;
+
+	/*
+	 * It can't use pm_runtime_resume_and_get() here, as the driver
+	 * relies at the returned value to detect if the device was already
+	 * active or not.
+	 */
+	rval = pm_runtime_get_sync(&client->dev);
+	if (rval < 0)
+		goto error;
+
+	/* Device was already active, so don't set controls */
+	if (rval == 1 && !sensor->handler_setup_needed)
+		return 0;
+
+	sensor->handler_setup_needed = false;
+
+	/* Restore V4L2 controls to the previously suspended device */
+	rval = v4l2_ctrl_handler_setup(&sensor->pixel_array->ctrl_handler);
+	if (rval)
+		goto error;
+
+	rval = v4l2_ctrl_handler_setup(&sensor->src->ctrl_handler);
+	if (rval)
+		goto error;
+
+	/* Keep PM runtime usage_count incremented on success */
+	return 0;
+
+error:
+	pm_runtime_put(&client->dev);
+	return rval;
+}
+
 static int ccs_start_streaming(struct ccs_sensor *sensor)
 {
 	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
@@ -1868,42 +1905,6 @@ static int ccs_stop_streaming(struct ccs_sensor *sensor)
  * V4L2 subdev video operations
  */
 
-static int ccs_pm_get_init(struct ccs_sensor *sensor)
-{
-	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
-	int rval;
-
-	/*
-	 * It can't use pm_runtime_resume_and_get() here, as the driver
-	 * relies at the returned value to detect if the device was already
-	 * active or not.
-	 */
-	rval = pm_runtime_get_sync(&client->dev);
-	if (rval < 0)
-		goto error;
-
-	/* Device was already active, so don't set controls */
-	if (rval == 1 && !sensor->handler_setup_needed)
-		return 0;
-
-	sensor->handler_setup_needed = false;
-
-	/* Restore V4L2 controls to the previously suspended device */
-	rval = v4l2_ctrl_handler_setup(&sensor->pixel_array->ctrl_handler);
-	if (rval)
-		goto error;
-
-	rval = v4l2_ctrl_handler_setup(&sensor->src->ctrl_handler);
-	if (rval)
-		goto error;
-
-	/* Keep PM runtime usage_count incremented on success */
-	return 0;
-error:
-	pm_runtime_put(&client->dev);
-	return rval;
-}
-
 static int ccs_set_stream(struct v4l2_subdev *subdev, int enable)
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
-- 
2.47.2

[PATCH v11 08/66] media: ccs: Rename out label of ccs_start_streaming

[Sakari Ailus]

In preparation for upcoming changes in the function, rename the out label
as err_pm_put. The purpose of the label is changed to match its name in
the next patch.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 38 ++++++++++++++++----------------
 1 file changed, 19 insertions(+), 19 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 2a55af67ed13..1aa5a06d7cde 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -1763,7 +1763,7 @@ static int ccs_start_streaming(struct ccs_sensor *sensor)
 			 (sensor->csi_format->width << 8) |
 			 sensor->csi_format->compressed);
 	if (rval)
-		goto out;
+		goto err_pm_put;
 
 	/* Binning configuration */
 	if (sensor->binning_horizontal == 1 &&
@@ -1776,38 +1776,38 @@ static int ccs_start_streaming(struct ccs_sensor *sensor)
 
 		rval = ccs_write(sensor, BINNING_TYPE, binning_type);
 		if (rval < 0)
-			goto out;
+			goto err_pm_put;
 
 		binning_mode = 1;
 	}
 	rval = ccs_write(sensor, BINNING_MODE, binning_mode);
 	if (rval < 0)
-		goto out;
+		goto err_pm_put;
 
 	/* Set up PLL */
 	rval = ccs_pll_configure(sensor);
 	if (rval)
-		goto out;
+		goto err_pm_put;
 
 	/* Analog crop start coordinates */
 	rval = ccs_write(sensor, X_ADDR_START, sensor->pa_src.left);
 	if (rval < 0)
-		goto out;
+		goto err_pm_put;
 
 	rval = ccs_write(sensor, Y_ADDR_START, sensor->pa_src.top);
 	if (rval < 0)
-		goto out;
+		goto err_pm_put;
 
 	/* Analog crop end coordinates */
 	rval = ccs_write(sensor, X_ADDR_END,
 			 sensor->pa_src.left + sensor->pa_src.width - 1);
 	if (rval < 0)
-		goto out;
+		goto err_pm_put;
 
 	rval = ccs_write(sensor, Y_ADDR_END,
 			 sensor->pa_src.top + sensor->pa_src.height - 1);
 	if (rval < 0)
-		goto out;
+		goto err_pm_put;
 
 	/*
 	 * Output from pixel array, including blanking, is set using
@@ -1820,22 +1820,22 @@ static int ccs_start_streaming(struct ccs_sensor *sensor)
 		rval = ccs_write(sensor, DIGITAL_CROP_X_OFFSET,
 				 sensor->scaler_sink.left);
 		if (rval < 0)
-			goto out;
+			goto err_pm_put;
 
 		rval = ccs_write(sensor, DIGITAL_CROP_Y_OFFSET,
 				 sensor->scaler_sink.top);
 		if (rval < 0)
-			goto out;
+			goto err_pm_put;
 
 		rval = ccs_write(sensor, DIGITAL_CROP_IMAGE_WIDTH,
 				 sensor->scaler_sink.width);
 		if (rval < 0)
-			goto out;
+			goto err_pm_put;
 
 		rval = ccs_write(sensor, DIGITAL_CROP_IMAGE_HEIGHT,
 				 sensor->scaler_sink.height);
 		if (rval < 0)
-			goto out;
+			goto err_pm_put;
 	}
 
 	/* Scaling */
@@ -1843,20 +1843,20 @@ static int ccs_start_streaming(struct ccs_sensor *sensor)
 	    != CCS_SCALING_CAPABILITY_NONE) {
 		rval = ccs_write(sensor, SCALING_MODE, sensor->scaling_mode);
 		if (rval < 0)
-			goto out;
+			goto err_pm_put;
 
 		rval = ccs_write(sensor, SCALE_M, sensor->scale_m);
 		if (rval < 0)
-			goto out;
+			goto err_pm_put;
 	}
 
 	/* Output size from sensor */
 	rval = ccs_write(sensor, X_OUTPUT_SIZE, sensor->src_src.width);
 	if (rval < 0)
-		goto out;
+		goto err_pm_put;
 	rval = ccs_write(sensor, Y_OUTPUT_SIZE, sensor->src_src.height);
 	if (rval < 0)
-		goto out;
+		goto err_pm_put;
 
 	if (CCS_LIM(sensor, FLASH_MODE_CAPABILITY) &
 	    (CCS_FLASH_MODE_CAPABILITY_SINGLE_STROBE |
@@ -1865,18 +1865,18 @@ static int ccs_start_streaming(struct ccs_sensor *sensor)
 	    sensor->hwcfg.strobe_setup->trigger != 0) {
 		rval = ccs_setup_flash_strobe(sensor);
 		if (rval)
-			goto out;
+			goto err_pm_put;
 	}
 
 	rval = ccs_call_quirk(sensor, pre_streamon);
 	if (rval) {
 		dev_err(&client->dev, "pre_streamon quirks failed\n");
-		goto out;
+		goto err_pm_put;
 	}
 
 	rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_STREAMING);
 
-out:
+err_pm_put:
 	mutex_unlock(&sensor->mutex);
 
 	return rval;
-- 
2.47.2

[PATCH v11 09/66] media: ccs: Move ccs_validate_csi_data_format up

[Sakari Ailus]

ccs_validate_csi_data_format() will soon be needed elsewhere, above its
current location. Move it up.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Julien Massot <julien.massot@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 1aa5a06d7cde..6cf60e1758b4 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -1981,6 +1981,20 @@ static int ccs_post_streamoff(struct v4l2_subdev *subdev)
 	return pm_runtime_put(&client->dev);
 }
 
+static const struct ccs_csi_data_format
+*ccs_validate_csi_data_format(struct ccs_sensor *sensor, u32 code)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(ccs_csi_data_formats); i++) {
+		if (sensor->mbus_frame_fmts & (1 << i) &&
+		    ccs_csi_data_formats[i].code == code)
+			return &ccs_csi_data_formats[i];
+	}
+
+	return sensor->csi_format;
+}
+
 static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_mbus_code_enum *code)
@@ -2116,20 +2130,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 	}
 }
 
-static const struct ccs_csi_data_format
-*ccs_validate_csi_data_format(struct ccs_sensor *sensor, u32 code)
-{
-	unsigned int i;
-
-	for (i = 0; i < ARRAY_SIZE(ccs_csi_data_formats); i++) {
-		if (sensor->mbus_frame_fmts & (1 << i) &&
-		    ccs_csi_data_formats[i].code == code)
-			return &ccs_csi_data_formats[i];
-	}
-
-	return sensor->csi_format;
-}
-
 static int ccs_set_format_source(struct v4l2_subdev *subdev,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_format *fmt)
-- 
2.47.2

[PATCH v11 10/66] media: ccs: Use {enable,disable}_streams operations

[Sakari Ailus]

Switch from s_stream() video op to enable_streams() and disable_streams()
pad operations. They are preferred and required for streams support.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 83 ++++++++++++++------------------
 1 file changed, 36 insertions(+), 47 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 6cf60e1758b4..56c9f7124b2a 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -1711,7 +1711,7 @@ static int ccs_power_off(struct device *dev)
 }
 
 /* -----------------------------------------------------------------------------
- * Video stream management
+ * V4L2 subdev video operations
  */
 
 static int ccs_pm_get_init(struct ccs_sensor *sensor)
@@ -1735,11 +1735,11 @@ static int ccs_pm_get_init(struct ccs_sensor *sensor)
 	sensor->handler_setup_needed = false;
 
 	/* Restore V4L2 controls to the previously suspended device */
-	rval = v4l2_ctrl_handler_setup(&sensor->pixel_array->ctrl_handler);
+	rval = __v4l2_ctrl_handler_setup(&sensor->pixel_array->ctrl_handler);
 	if (rval)
 		goto error;
 
-	rval = v4l2_ctrl_handler_setup(&sensor->src->ctrl_handler);
+	rval = __v4l2_ctrl_handler_setup(&sensor->src->ctrl_handler);
 	if (rval)
 		goto error;
 
@@ -1751,13 +1751,21 @@ static int ccs_pm_get_init(struct ccs_sensor *sensor)
 	return rval;
 }
 
-static int ccs_start_streaming(struct ccs_sensor *sensor)
+static int ccs_enable_streams(struct v4l2_subdev *subdev,
+			      struct v4l2_subdev_state *state, u32 pad,
+			      u64 streams_mask)
 {
+	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
 	unsigned int binning_mode;
 	int rval;
 
-	mutex_lock(&sensor->mutex);
+	if (pad != CCS_PAD_SRC)
+		return -EINVAL;
+
+	rval = ccs_pm_get_init(sensor);
+	if (rval)
+		return rval;
 
 	rval = ccs_write(sensor, CSI_DATA_FORMAT,
 			 (sensor->csi_format->width << 8) |
@@ -1876,64 +1884,41 @@ static int ccs_start_streaming(struct ccs_sensor *sensor)
 
 	rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_STREAMING);
 
-err_pm_put:
-	mutex_unlock(&sensor->mutex);
-
-	return rval;
-}
-
-static int ccs_stop_streaming(struct ccs_sensor *sensor)
-{
-	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
-	int rval;
+	sensor->streaming = true;
 
-	mutex_lock(&sensor->mutex);
-	rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_SOFTWARE_STANDBY);
-	if (rval)
-		goto out;
+	return 0;
 
-	rval = ccs_call_quirk(sensor, post_streamoff);
-	if (rval)
-		dev_err(&client->dev, "post_streamoff quirks failed\n");
+err_pm_put:
+	pm_runtime_mark_last_busy(&client->dev);
+	pm_runtime_put_autosuspend(&client->dev);
 
-out:
-	mutex_unlock(&sensor->mutex);
 	return rval;
 }
 
-/* -----------------------------------------------------------------------------
- * V4L2 subdev video operations
- */
-
-static int ccs_set_stream(struct v4l2_subdev *subdev, int enable)
+static int ccs_disable_streams(struct v4l2_subdev *subdev,
+			       struct v4l2_subdev_state *state, u32 pad,
+			       u64 streams_mask)
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
 	int rval;
 
-	if (!enable) {
-		ccs_stop_streaming(sensor);
-		sensor->streaming = false;
-		pm_runtime_mark_last_busy(&client->dev);
-		pm_runtime_put_autosuspend(&client->dev);
-
-		return 0;
-	}
+	if (pad != CCS_PAD_SRC)
+		return -EINVAL;
 
-	rval = ccs_pm_get_init(sensor);
+	rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_SOFTWARE_STANDBY);
 	if (rval)
 		return rval;
 
-	sensor->streaming = true;
+	rval = ccs_call_quirk(sensor, post_streamoff);
+	if (rval)
+		dev_err(&client->dev, "post_streamoff quirks failed\n");
 
-	rval = ccs_start_streaming(sensor);
-	if (rval < 0) {
-		sensor->streaming = false;
-		pm_runtime_mark_last_busy(&client->dev);
-		pm_runtime_put_autosuspend(&client->dev);
-	}
+	sensor->streaming = false;
+	pm_runtime_mark_last_busy(&client->dev);
+	pm_runtime_put_autosuspend(&client->dev);
 
-	return rval;
+	return 0;
 }
 
 static int ccs_pre_streamon(struct v4l2_subdev *subdev, u32 flags)
@@ -1959,7 +1944,9 @@ static int ccs_pre_streamon(struct v4l2_subdev *subdev, u32 flags)
 		}
 	}
 
+	mutex_lock(&sensor->mutex);
 	rval = ccs_pm_get_init(sensor);
+	mutex_unlock(&sensor->mutex);
 	if (rval)
 		return rval;
 
@@ -3039,7 +3026,7 @@ static int ccs_init_state(struct v4l2_subdev *sd,
 }
 
 static const struct v4l2_subdev_video_ops ccs_video_ops = {
-	.s_stream = ccs_set_stream,
+	.s_stream = v4l2_subdev_s_stream_helper,
 	.pre_streamon = ccs_pre_streamon,
 	.post_streamoff = ccs_post_streamoff,
 };
@@ -3050,6 +3037,8 @@ static const struct v4l2_subdev_pad_ops ccs_pad_ops = {
 	.set_fmt = ccs_set_format,
 	.get_selection = ccs_get_selection,
 	.set_selection = ccs_set_selection,
+	.enable_streams = ccs_enable_streams,
+	.disable_streams = ccs_disable_streams,
 };
 
 static const struct v4l2_subdev_sensor_ops ccs_sensor_ops = {
-- 
2.47.2

[PATCH v11 11/66] media: ccs: Track streaming state

[Sakari Ailus]

With enable_streams and disable_streams, the driver for a device where
streams are not independently started and stopped needs to maintain state
information on streams that have been requested to be started. Do that
now.

In the future, a helper function in the framework is a desirable way to do
this instead.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Julien Massot <julien.massot@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 13 ++++++++++---
 drivers/media/i2c/ccs/ccs.h      |  2 +-
 2 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 56c9f7124b2a..bd7d2a8d23dd 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -1763,6 +1763,11 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 	if (pad != CCS_PAD_SRC)
 		return -EINVAL;
 
+	if (sensor->streaming) {
+		sensor->streaming |= streams_mask;
+		return 0;
+	}
+
 	rval = ccs_pm_get_init(sensor);
 	if (rval)
 		return rval;
@@ -1884,7 +1889,7 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 
 	rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_STREAMING);
 
-	sensor->streaming = true;
+	sensor->streaming |= streams_mask;
 
 	return 0;
 
@@ -1906,6 +1911,10 @@ static int ccs_disable_streams(struct v4l2_subdev *subdev,
 	if (pad != CCS_PAD_SRC)
 		return -EINVAL;
 
+	sensor->streaming &= ~streams_mask;
+	if (sensor->streaming)
+		return 0;
+
 	rval = ccs_write(sensor, MODE_SELECT, CCS_MODE_SELECT_SOFTWARE_STANDBY);
 	if (rval)
 		return rval;
@@ -1914,7 +1923,6 @@ static int ccs_disable_streams(struct v4l2_subdev *subdev,
 	if (rval)
 		dev_err(&client->dev, "post_streamoff quirks failed\n");
 
-	sensor->streaming = false;
 	pm_runtime_mark_last_busy(&client->dev);
 	pm_runtime_put_autosuspend(&client->dev);
 
@@ -3519,7 +3527,6 @@ static int ccs_probe(struct i2c_client *client)
 		goto out_cleanup;
 	}
 
-	sensor->streaming = false;
 	sensor->dev_init_done = true;
 	sensor->handler_setup_needed = true;
 
diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
index 0726c4687f0f..518482758da6 100644
--- a/drivers/media/i2c/ccs/ccs.h
+++ b/drivers/media/i2c/ccs/ccs.h
@@ -238,7 +238,7 @@ struct ccs_sensor {
 	u16 image_start; /* image data start line */
 	u16 visible_pixel_start; /* start pixel of the visible image */
 
-	bool streaming;
+	u8 streaming;
 	bool dev_init_done;
 	bool handler_setup_needed;
 	u8 compressed_min_bpp;
-- 
2.47.2

[PATCH v11 12/66] media: ccs: Support frame descriptors

[Sakari Ailus]

Provide information on the frame layout using frame descriptors.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Julien Massot <julien.massot@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 74 ++++++++++++++++++++++++++++++--
 drivers/media/i2c/ccs/ccs.h      |  4 ++
 2 files changed, 74 insertions(+), 4 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index bd7d2a8d23dd..109b74476b8c 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -25,6 +25,7 @@
 #include <linux/slab.h>
 #include <linux/smiapp.h>
 #include <linux/v4l2-mediabus.h>
+#include <media/mipi-csi2.h>
 #include <media/v4l2-cci.h>
 #include <media/v4l2-device.h>
 #include <media/v4l2-fwnode.h>
@@ -245,6 +246,33 @@ static int ccs_read_all_limits(struct ccs_sensor *sensor)
 	return ret;
 }
 
+static u8 ccs_mipi_csi2_data_type(unsigned int bpp)
+{
+	switch (bpp) {
+	case 6:
+		return MIPI_CSI2_DT_RAW6;
+	case 7:
+		return MIPI_CSI2_DT_RAW7;
+	case 8:
+		return MIPI_CSI2_DT_RAW8;
+	case 10:
+		return MIPI_CSI2_DT_RAW10;
+	case 12:
+		return MIPI_CSI2_DT_RAW12;
+	case 14:
+		return MIPI_CSI2_DT_RAW14;
+	case 16:
+		return MIPI_CSI2_DT_RAW16;
+	case 20:
+		return MIPI_CSI2_DT_RAW20;
+	case 24:
+		return MIPI_CSI2_DT_RAW24;
+	default:
+		WARN_ON(1);
+		return 0;
+	}
+}
+
 static int ccs_read_frame_fmt(struct ccs_sensor *sensor)
 {
 	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
@@ -2078,10 +2106,11 @@ static void ccs_get_crop_compose(struct v4l2_subdev *subdev,
 	if (crops)
 		for (i = 0; i < subdev->entity.num_pads; i++)
 			crops[i] =
-				v4l2_subdev_state_get_crop(sd_state, i);
+				v4l2_subdev_state_get_crop(sd_state, i,
+							   CCS_STREAM_PIXEL);
 	if (comps)
-		*comps = v4l2_subdev_state_get_compose(sd_state,
-						       ssd->sink_pad);
+		*comps = v4l2_subdev_state_get_compose(sd_state, ssd->sink_pad,
+						       CCS_STREAM_PIXEL);
 }
 
 /* Changes require propagation only on sink pad. */
@@ -2114,7 +2143,8 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 		fallthrough;
 	case V4L2_SEL_TGT_COMPOSE:
 		*crops[CCS_PAD_SRC] = *comp;
-		fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC);
+		fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC,
+						   CCS_STREAM_PIXEL);
 		fmt->width = comp->width;
 		fmt->height = comp->height;
 		if (which == V4L2_SUBDEV_FORMAT_ACTIVE && ssd == sensor->src)
@@ -2629,6 +2659,41 @@ static int ccs_set_selection(struct v4l2_subdev *subdev,
 	return ret;
 }
 
+static int ccs_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
+				 struct v4l2_mbus_frame_desc *desc)
+{
+	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
+	struct v4l2_mbus_frame_desc_entry *entry = desc->entry;
+	struct v4l2_subdev_state *sd_state;
+
+	switch (sensor->hwcfg.csi_signalling_mode) {
+	case CCS_CSI_SIGNALING_MODE_CSI_2_DPHY:
+	case CCS_CSI_SIGNALING_MODE_CSI_2_CPHY:
+		desc->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
+		break;
+	default:
+		/* FIXME: CCP2 support */
+		return -EINVAL;
+	}
+
+	sd_state = v4l2_subdev_lock_and_get_active_state(subdev);
+	if (!sd_state)
+		return -EINVAL;
+
+	entry->pixelcode = sensor->csi_format->code;
+	entry->stream = CCS_STREAM_PIXEL;
+	entry->bus.csi2.dt =
+		sensor->csi_format->width == sensor->csi_format->compressed ?
+		ccs_mipi_csi2_data_type(sensor->csi_format->width) :
+		CCS_DEFAULT_COMPRESSED_DT;
+	entry++;
+	desc->num_entries++;
+
+	v4l2_subdev_unlock_state(sd_state);
+
+	return 0;
+}
+
 static int ccs_get_skip_frames(struct v4l2_subdev *subdev, u32 *frames)
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
@@ -3047,6 +3112,7 @@ static const struct v4l2_subdev_pad_ops ccs_pad_ops = {
 	.set_selection = ccs_set_selection,
 	.enable_streams = ccs_enable_streams,
 	.disable_streams = ccs_disable_streams,
+	.get_frame_desc = ccs_get_frame_desc,
 };
 
 static const struct v4l2_subdev_sensor_ops ccs_sensor_ops = {
diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
index 518482758da6..0bdb8cd3accb 100644
--- a/drivers/media/i2c/ccs/ccs.h
+++ b/drivers/media/i2c/ccs/ccs.h
@@ -48,6 +48,8 @@
 
 #define CCS_COLOUR_COMPONENTS		4
 
+#define CCS_DEFAULT_COMPRESSED_DT	MIPI_CSI2_DT_USER_DEFINED(0)
+
 #define SMIAPP_NAME			"smiapp"
 #define CCS_NAME			"ccs"
 
@@ -177,6 +179,8 @@ struct ccs_csi_data_format {
 #define CCS_PAD_SRC			1
 #define CCS_PADS			2
 
+#define CCS_STREAM_PIXEL		0
+
 struct ccs_binning_subtype {
 	u8 horizontal:4;
 	u8 vertical:4;
-- 
2.47.2

Re: [PATCH v11 12/66] media: ccs: Support frame descriptors

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:13PM +0300, Sakari Ailus wrote:
> Provide information on the frame layout using frame descriptors.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Julien Massot <julien.massot@collabora.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Are there dependencies on previous patches from this series for these
CCS-only changes ? They look reviewed to me, can they be fast-tracked ?

> ---
>  drivers/media/i2c/ccs/ccs-core.c | 74 ++++++++++++++++++++++++++++++--
>  drivers/media/i2c/ccs/ccs.h      |  4 ++
>  2 files changed, 74 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
> index bd7d2a8d23dd..109b74476b8c 100644
> --- a/drivers/media/i2c/ccs/ccs-core.c
> +++ b/drivers/media/i2c/ccs/ccs-core.c
> @@ -25,6 +25,7 @@
>  #include <linux/slab.h>
>  #include <linux/smiapp.h>
>  #include <linux/v4l2-mediabus.h>
> +#include <media/mipi-csi2.h>
>  #include <media/v4l2-cci.h>
>  #include <media/v4l2-device.h>
>  #include <media/v4l2-fwnode.h>
> @@ -245,6 +246,33 @@ static int ccs_read_all_limits(struct ccs_sensor *sensor)
>  	return ret;
>  }
>
> +static u8 ccs_mipi_csi2_data_type(unsigned int bpp)
> +{
> +	switch (bpp) {
> +	case 6:
> +		return MIPI_CSI2_DT_RAW6;
> +	case 7:
> +		return MIPI_CSI2_DT_RAW7;
> +	case 8:
> +		return MIPI_CSI2_DT_RAW8;
> +	case 10:
> +		return MIPI_CSI2_DT_RAW10;
> +	case 12:
> +		return MIPI_CSI2_DT_RAW12;
> +	case 14:
> +		return MIPI_CSI2_DT_RAW14;
> +	case 16:
> +		return MIPI_CSI2_DT_RAW16;
> +	case 20:
> +		return MIPI_CSI2_DT_RAW20;
> +	case 24:
> +		return MIPI_CSI2_DT_RAW24;
> +	default:
> +		WARN_ON(1);
> +		return 0;
> +	}
> +}
> +
>  static int ccs_read_frame_fmt(struct ccs_sensor *sensor)
>  {
>  	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
> @@ -2078,10 +2106,11 @@ static void ccs_get_crop_compose(struct v4l2_subdev *subdev,
>  	if (crops)
>  		for (i = 0; i < subdev->entity.num_pads; i++)
>  			crops[i] =
> -				v4l2_subdev_state_get_crop(sd_state, i);
> +				v4l2_subdev_state_get_crop(sd_state, i,
> +							   CCS_STREAM_PIXEL);
>  	if (comps)
> -		*comps = v4l2_subdev_state_get_compose(sd_state,
> -						       ssd->sink_pad);
> +		*comps = v4l2_subdev_state_get_compose(sd_state, ssd->sink_pad,
> +						       CCS_STREAM_PIXEL);
>  }
>
>  /* Changes require propagation only on sink pad. */
> @@ -2114,7 +2143,8 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
>  		fallthrough;
>  	case V4L2_SEL_TGT_COMPOSE:
>  		*crops[CCS_PAD_SRC] = *comp;
> -		fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC);
> +		fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC,
> +						   CCS_STREAM_PIXEL);
>  		fmt->width = comp->width;
>  		fmt->height = comp->height;
>  		if (which == V4L2_SUBDEV_FORMAT_ACTIVE && ssd == sensor->src)
> @@ -2629,6 +2659,41 @@ static int ccs_set_selection(struct v4l2_subdev *subdev,
>  	return ret;
>  }
>
> +static int ccs_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
> +				 struct v4l2_mbus_frame_desc *desc)
> +{
> +	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
> +	struct v4l2_mbus_frame_desc_entry *entry = desc->entry;
> +	struct v4l2_subdev_state *sd_state;
> +
> +	switch (sensor->hwcfg.csi_signalling_mode) {
> +	case CCS_CSI_SIGNALING_MODE_CSI_2_DPHY:
> +	case CCS_CSI_SIGNALING_MODE_CSI_2_CPHY:
> +		desc->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
> +		break;
> +	default:
> +		/* FIXME: CCP2 support */
> +		return -EINVAL;
> +	}
> +
> +	sd_state = v4l2_subdev_lock_and_get_active_state(subdev);
> +	if (!sd_state)
> +		return -EINVAL;
> +
> +	entry->pixelcode = sensor->csi_format->code;
> +	entry->stream = CCS_STREAM_PIXEL;
> +	entry->bus.csi2.dt =
> +		sensor->csi_format->width == sensor->csi_format->compressed ?
> +		ccs_mipi_csi2_data_type(sensor->csi_format->width) :
> +		CCS_DEFAULT_COMPRESSED_DT;
> +	entry++;
> +	desc->num_entries++;
> +
> +	v4l2_subdev_unlock_state(sd_state);
> +
> +	return 0;
> +}
> +
>  static int ccs_get_skip_frames(struct v4l2_subdev *subdev, u32 *frames)
>  {
>  	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
> @@ -3047,6 +3112,7 @@ static const struct v4l2_subdev_pad_ops ccs_pad_ops = {
>  	.set_selection = ccs_set_selection,
>  	.enable_streams = ccs_enable_streams,
>  	.disable_streams = ccs_disable_streams,
> +	.get_frame_desc = ccs_get_frame_desc,
>  };
>
>  static const struct v4l2_subdev_sensor_ops ccs_sensor_ops = {
> diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
> index 518482758da6..0bdb8cd3accb 100644
> --- a/drivers/media/i2c/ccs/ccs.h
> +++ b/drivers/media/i2c/ccs/ccs.h
> @@ -48,6 +48,8 @@
>
>  #define CCS_COLOUR_COMPONENTS		4
>
> +#define CCS_DEFAULT_COMPRESSED_DT	MIPI_CSI2_DT_USER_DEFINED(0)
> +
>  #define SMIAPP_NAME			"smiapp"
>  #define CCS_NAME			"ccs"
>
> @@ -177,6 +179,8 @@ struct ccs_csi_data_format {
>  #define CCS_PAD_SRC			1
>  #define CCS_PADS			2
>
> +#define CCS_STREAM_PIXEL		0
> +
>  struct ccs_binning_subtype {
>  	u8 horizontal:4;
>  	u8 vertical:4;
> --
> 2.47.2
>
>

Re: [PATCH v11 12/66] media: ccs: Support frame descriptors

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 02:06:40PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:13PM +0300, Sakari Ailus wrote:
> > Provide information on the frame layout using frame descriptors.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Julien Massot <julien.massot@collabora.com>
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> Are there dependencies on previous patches from this series for these
> CCS-only changes ? They look reviewed to me, can they be fast-tracked ?

I'll try to do further testing on these so they could be merged.

-- 
Kind regards,

Sakari Ailus

[PATCH v11 13/66] media: mc: Add INTERNAL pad flag

[Sakari Ailus]

Internal sink pads will be used as routing endpoints in V4L2 [GS]_ROUTING
IOCTLs, to indicate that the stream begins in the entity. Internal sink
pads are pads that have both SINK and INTERNAL flags set.

Also prevent creating links to pads that have been flagged as internal and
initialising SOURCE pads with INTERNAL flag set.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 .../userspace-api/media/mediactl/media-types.rst  |  9 +++++++++
 drivers/media/mc/mc-entity.c                      | 15 ++++++++++++---
 include/uapi/linux/media.h                        |  1 +
 3 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/Documentation/userspace-api/media/mediactl/media-types.rst
index 6332e8395263..200c37a1da26 100644
--- a/Documentation/userspace-api/media/mediactl/media-types.rst
+++ b/Documentation/userspace-api/media/mediactl/media-types.rst
@@ -361,6 +361,7 @@ Types and flags used to represent the media graph elements
 .. _MEDIA-PAD-FL-SINK:
 .. _MEDIA-PAD-FL-SOURCE:
 .. _MEDIA-PAD-FL-MUST-CONNECT:
+.. _MEDIA-PAD-FL-INTERNAL:
 
 .. flat-table:: Media pad flags
     :header-rows:  0
@@ -381,6 +382,14 @@ Types and flags used to represent the media graph elements
 	  enabled links even when this flag isn't set; the absence of the flag
 	  doesn't imply there is none.
 
+    *  -  ``MEDIA_PAD_FL_INTERNAL``
+       -  The internal flag indicates an internal pad that has no external
+	  connections. As they are internal to entities, internal pads shall not
+	  be connected with links.
+
+	  The internal flag may currently be present only in a sink pad where it
+	  indicates that the :ref:``stream <media-glossary-stream>`` originates
+	  from within the entity.
 
 One and only one of ``MEDIA_PAD_FL_SINK`` and ``MEDIA_PAD_FL_SOURCE``
 must be set for every pad.
diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c
index 9519a537bfa2..df20356fa98b 100644
--- a/drivers/media/mc/mc-entity.c
+++ b/drivers/media/mc/mc-entity.c
@@ -209,11 +209,16 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads,
 		mutex_lock(&mdev->graph_mutex);
 
 	media_entity_for_each_pad(entity, iter) {
+		const u32 pad_flags = iter->flags & (MEDIA_PAD_FL_SINK |
+						     MEDIA_PAD_FL_SOURCE |
+						     MEDIA_PAD_FL_INTERNAL);
+
 		iter->entity = entity;
 		iter->index = i++;
 
-		if (hweight32(iter->flags & (MEDIA_PAD_FL_SINK |
-					     MEDIA_PAD_FL_SOURCE)) != 1) {
+		if (pad_flags != MEDIA_PAD_FL_SINK &&
+		    pad_flags != (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_INTERNAL) &&
+		    pad_flags != MEDIA_PAD_FL_SOURCE) {
 			ret = -EINVAL;
 			break;
 		}
@@ -1118,7 +1123,8 @@ int media_get_pad_index(struct media_entity *entity, u32 pad_type,
 
 	for (i = 0; i < entity->num_pads; i++) {
 		if ((entity->pads[i].flags &
-		     (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE)) != pad_type)
+		     (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE |
+		      MEDIA_PAD_FL_INTERNAL)) != pad_type)
 			continue;
 
 		if (entity->pads[i].sig_type == sig_type)
@@ -1148,6 +1154,9 @@ media_create_pad_link(struct media_entity *source, u16 source_pad,
 		return -EINVAL;
 	if (WARN_ON(!(sink->pads[sink_pad].flags & MEDIA_PAD_FL_SINK)))
 		return -EINVAL;
+	if (WARN_ON(source->pads[source_pad].flags & MEDIA_PAD_FL_INTERNAL) ||
+	    WARN_ON(sink->pads[sink_pad].flags & MEDIA_PAD_FL_INTERNAL))
+		return -EINVAL;
 
 	link = media_add_link(&source->links);
 	if (link == NULL)
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 1c80b1d6bbaf..80cfd12a43fc 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -208,6 +208,7 @@ struct media_entity_desc {
 #define MEDIA_PAD_FL_SINK			(1U << 0)
 #define MEDIA_PAD_FL_SOURCE			(1U << 1)
 #define MEDIA_PAD_FL_MUST_CONNECT		(1U << 2)
+#define MEDIA_PAD_FL_INTERNAL			(1U << 3)
 
 struct media_pad_desc {
 	__u32 entity;		/* entity ID */
-- 
2.47.2

[PATCH v11 14/66] media: Documentation: Refer to internal pads in metadata documentation

[Sakari Ailus]

Metadata is intended to be used with internal pads when it comes to the
source of the data. Document this.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 Documentation/userspace-api/media/v4l/metafmt-generic.rst | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
index 39d529c910a8..e8824b5a65d6 100644
--- a/Documentation/userspace-api/media/v4l/metafmt-generic.rst
+++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
@@ -7,12 +7,17 @@ V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_M
 
 Generic line-based metadata formats
 
+.. _v4l2-format-generic-meta:
 
 Description
 ===========
 
 These generic line-based metadata formats define the memory layout of the data
-without defining the format or meaning of the metadata itself.
+without defining the format or meaning of the metadata itself. These formats may
+only be used with a Media Controller pipeline where the more specific metadata
+layout is reported by an :ref:`V4L2_CID_METADATA_LAYOUT control
+<image_source_control_metadata_layout>` of the source sub-device. See also
+:ref:`source routes <subdev-routing>`.
 
 .. _v4l2-meta-fmt-generic-8:
 
-- 
2.47.2

Re: [PATCH v11 14/66] media: Documentation: Refer to internal pads in metadata documentation

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:15PM +0300, Sakari Ailus wrote:
> Metadata is intended to be used with internal pads when it comes to the
> source of the data. Document this.

The commit title and description do not match the patch content
anymore

With these updated:
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j

>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  Documentation/userspace-api/media/v4l/metafmt-generic.rst | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> index 39d529c910a8..e8824b5a65d6 100644
> --- a/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> +++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> @@ -7,12 +7,17 @@ V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_M
>
>  Generic line-based metadata formats
>
> +.. _v4l2-format-generic-meta:
>
>  Description
>  ===========
>
>  These generic line-based metadata formats define the memory layout of the data
> -without defining the format or meaning of the metadata itself.
> +without defining the format or meaning of the metadata itself. These formats may
> +only be used with a Media Controller pipeline where the more specific metadata
> +layout is reported by an :ref:`V4L2_CID_METADATA_LAYOUT control
> +<image_source_control_metadata_layout>` of the source sub-device. See also
> +:ref:`source routes <subdev-routing>`.
>
>  .. _v4l2-meta-fmt-generic-8:
>
> --
> 2.47.2
>
>

Re: [PATCH v11 14/66] media: Documentation: Refer to internal pads in metadata documentation

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 02:08:01PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:15PM +0300, Sakari Ailus wrote:
> > Metadata is intended to be used with internal pads when it comes to the
> > source of the data. Document this.
> 
> The commit title and description do not match the patch content
> anymore
> 
> With these updated:
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks for the review. I used this:

media: Documentation: Refer to metadata layout in metadata documentation

Using generic metadata mbus codes requires the use of
V4L2_CID_METADATA_LAYOUT control to define the metadata layout. Document
this in generic metadata format documentation.

-- 
Sakari Ailus

Re: [PATCH v11 14/66] media: Documentation: Refer to internal pads in metadata documentation

[Laurent Pinchart]

On Mon, Aug 25, 2025 at 12:50:15PM +0300, Sakari Ailus wrote:
> Metadata is intended to be used with internal pads when it comes to the
> source of the data. Document this.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  Documentation/userspace-api/media/v4l/metafmt-generic.rst | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> index 39d529c910a8..e8824b5a65d6 100644
> --- a/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> +++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> @@ -7,12 +7,17 @@ V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_M
>  
>  Generic line-based metadata formats
>  
> +.. _v4l2-format-generic-meta:
>  
>  Description
>  ===========
>  
>  These generic line-based metadata formats define the memory layout of the data
> -without defining the format or meaning of the metadata itself.
> +without defining the format or meaning of the metadata itself. These formats may
> +only be used with a Media Controller pipeline where the more specific metadata

Maybe "exact" matadata layout instead of "more specific".

> +layout is reported by an :ref:`V4L2_CID_METADATA_LAYOUT control

s/an/the/

> +<image_source_control_metadata_layout>` of the source sub-device. See also

I would write

layout is reported using the :ref:`V4L2_CID_METADATA_LAYOUT control
<image_source_control_metadata_layout>` by the sub-device generating the
metadata. See also

(or possibly s/using/through/).

With the updated commit message you proposed,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +:ref:`source routes <subdev-routing>`.
>  
>  .. _v4l2-meta-fmt-generic-8:
>  

-- 
Regards,

Laurent Pinchart

[PATCH v11 15/66] media: uapi: Add generic CSI-2 raw pixelformats

[Sakari Ailus]

Add generic raw pixelformats for bit depths 8, 10, 12 and 14. These
formats are CSI-2 packed, apart from the 8-bit format.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../media/v4l/pixfmt-raw-generic.rst          | 175 ++++++++++++++++++
 .../userspace-api/media/v4l/pixfmt.rst        |   1 +
 drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +
 include/uapi/linux/videodev2.h                |   6 +
 4 files changed, 186 insertions(+)
 create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
new file mode 100644
index 000000000000..9fda7a36cd3a
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
@@ -0,0 +1,175 @@
+.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
+
+************************************************************************************************************************************************************************************************************************************************************************************
+V4L2_PIX_FMT_RAW_8 ('RAW8'), V4L2_PIX_FMT_RAW_CSI2_10 ('RACA'), V4L2_PIX_FMT_RAW_CSI2_12 ('RACC'), V4L2_PIX_FMT_RAW_CSI2_14 ('RACE'), V4L2_PIX_FMT_RAW_16 ('RAWG'), V4L2_PIX_FMT_RAW_CSI2_20 ('RACK'), V4L2_PIX_FMT_RAW_24 ('RAWO'), V4L2_PIX_FMT_RAW_CSI2_28 ('RACS')
+************************************************************************************************************************************************************************************************************************************************************************************
+
+
+Generic line-based image data formats
+
+
+Description
+===========
+
+These generic raw image data formats define the memory layout of the data
+without defining the order of the pixels in the format or even the CFA (Colour
+Filter Array) itself. See also :ref:`source routes <subdev-routing>`.
+
+.. _v4l2-pix-fmt-raw-8:
+
+V4L2_PIX_FMT_RAW_8
+------------------
+
+The V4L2_PIX_FMT_GENERIC_8 format is a plain 8-bit raw pixel data format. This
+format is used on CSI-2 for 8 bits per :term:`Data Unit`.
+
+**Byte Order Of V4L2_PIX_FMT_RAW_8.**
+Each cell is one byte. "P" denotes a pixel.
+
+.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|
+
+.. flat-table:: Sample 4x2 Image Frame
+    :header-rows:  0
+    :stub-columns: 0
+    :widths: 12 8 8 8 8
+
+    * - start + 0:
+      - P\ :sub:`00`
+      - P\ :sub:`10`
+      - P\ :sub:`20`
+      - P\ :sub:`30`
+    * - start + 4:
+      - P\ :sub:`01`
+      - P\ :sub:`11`
+      - P\ :sub:`21`
+      - P\ :sub:`31`
+
+.. _v4l2-pix-fmt-raw-csi2-10:
+
+V4L2_PIX_FMT_RAW_CSI2_10
+------------------------
+
+V4L2_PIX_FMT_RAW_CSI2_10 contains 10-bit packed image data, with four bytes
+containing the top 8 bits of the pixels followed by lowest 2 bits of the pixels
+packed into one byte. This format is typically used by CSI-2 receivers with
+a source that transmits MEDIA_BUS_FMT_RAW_10 and the CSI-2 receiver writes the
+received data to memory as-is.
+
+The packing of the data follows the MIPI CSI-2 specification.
+
+**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_10.**
+Each cell is one byte. "P" denotes a pixel.
+
+.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
+
+.. flat-table:: Sample 4x2 Image Frame
+    :header-rows:  0
+    :stub-columns: 0
+    :widths: 12 8 8 8 8 8
+
+    * - start + 0:
+      - P\ :sub:`00 bits 9--2`
+      - P\ :sub:`10 bits 9--2`
+      - P\ :sub:`20 bits 9--2`
+      - P\ :sub:`30 bits 9--2`
+      - P\ :sub:`00 bits 1--0` (bits 1--0)
+        P\ :sub:`10 bits 1--0` (bits 3--2)
+        P\ :sub:`20 bits 1--0` (bits 5--4)
+        P\ :sub:`30 bits 1--0` (bits 7--6)
+    * - start + 5:
+      - P\ :sub:`01 bits 9--2`
+      - P\ :sub:`11 bits 9--2`
+      - P\ :sub:`21 bits 9--2`
+      - P\ :sub:`31 bits 9--2`
+      - P\ :sub:`01 bits 1--0` (bits 1--0)
+        P\ :sub:`11 bits 1--0` (bits 3--2)
+        P\ :sub:`21 bits 1--0` (bits 5--4)
+        P\ :sub:`31 bits 1--0` (bits 7--6)
+
+.. _v4l2-pix-fmt-raw-csi2-12:
+
+V4L2_PIX_FMT_RAW_CSI2_12
+------------------------
+
+V4L2_PIX_FMT_RAW_CSI2_12 contains 12-bit packed image data, with two bytes
+containing the top 8 bits of the pixels followed by lowest 4 bits of the pixels
+packed into 1 byte. This format is typically used by CSI-2 receivers with
+a source that transmits MEDIA_BUS_FMT_RAW_12 and the CSI-2 receiver writes the
+received data to memory as-is.
+
+The packing of the data follows the MIPI CSI-2 specification.
+
+**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_12.**
+Each cell is one byte. "P" denotes a pixel.
+
+.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
+
+.. flat-table:: Sample 4x2 Image Frame
+    :header-rows:  0
+    :stub-columns: 0
+    :widths: 12 8 8 8 8 8 8
+
+    * - start + 0:
+      - P\ :sub:`00 bits 11--4`
+      - P\ :sub:`10 bits 11--4`
+      - P\ :sub:`00 bits 3--0` (bits 3--0)
+        P\ :sub:`10 bits 3--0` (bits 7--4)
+      - P\ :sub:`20 bits 11--4`
+      - P\ :sub:`30 bits 11--4`
+      - P\ :sub:`20 bits 3--0` (bits 3--0)
+        P\ :sub:`30 bits 3--0` (bits 7--4)
+    * - start + 6:
+      - P\ :sub:`01 bits 11--4`
+      - P\ :sub:`11 bits 11--4`
+      - P\ :sub:`01 bits 3--0` (bits 3--0)
+        P\ :sub:`11 bits 3--0` (bits 7--4)
+      - P\ :sub:`21 bits 11--4`
+      - P\ :sub:`31 bits 11--4`
+      - P\ :sub:`21 bits 3--0` (bits 3--0)
+        P\ :sub:`31 bits 3--0` (bits 7--4)
+
+.. _v4l2-pix-fmt-raw-csi2-14:
+
+V4L2_PIX_FMT_RAW_CSI2_14
+------------------------
+
+V4L2_PIX_FMT_RAW_CSI2_14 contains 14-bit packed image data, with four bytes
+containing the top 8 bits of the pixels followed by lowest 6 bits of the pixels
+packed into three bytes. This format is typically used by CSI-2 receivers with a
+source that transmits MEDIA_BUS_FMT_RAW_14 and the CSI-2 receiver writes the
+received data to memory as-is.
+
+The packing of the data follows the MIPI CSI-2 specification.
+
+**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_14.**
+Each cell is one byte. "P" denotes a pixel.
+
+.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.8cm}|p{1.8cm}|
+
+.. flat-table:: Sample 4x2 Image Frame
+    :header-rows:  0
+    :stub-columns: 0
+    :widths: 12 8 8 8 8 8 8 8
+
+    * - start + 0:
+      - P\ :sub:`00 bits 13--6`
+      - P\ :sub:`10 bits 13--6`
+      - P\ :sub:`20 bits 13--6`
+      - P\ :sub:`30 bits 13--6`
+      - P\ :sub:`10 bits 1--0` (bits 7--6)
+        P\ :sub:`00 bits 5--0` (bits 5--0)
+      - P\ :sub:`20 bits 3--0` (bits 7--4)
+        P\ :sub:`10 bits 5--2` (bits 3--0)
+      - P\ :sub:`30 bits 5--0` (bits 7--2)
+        P\ :sub:`20 bits 5--4` (bits 1--0)
+    * - start + 7:
+      - P\ :sub:`01 bits 13--6`
+      - P\ :sub:`11 bits 13--6`
+      - P\ :sub:`21 bits 13--6`
+      - P\ :sub:`31 bits 13--6`
+      - P\ :sub:`11 bits 1--0` (bits 7--6)
+        P\ :sub:`01 bits 5--0` (bits 5--0)
+      - P\ :sub:`21 bits 3--0` (bits 7--4)
+        P\ :sub:`11 bits 5--2` (bits 3--0)
+      - P\ :sub:`31 bits 5--0` (bits 7--2)
+        P\ :sub:`21 bits 5--4` (bits 1--0)
diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
index 11dab4a90630..d917190c717d 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
@@ -25,6 +25,7 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
     pixfmt-indexed
     pixfmt-rgb
     pixfmt-bayer
+    pixfmt-raw-generic
     yuv-formats
     hsv-formats
     depth-formats
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 01cf52c3ea33..e538c1230631 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1434,6 +1434,10 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_SGBRG16:	descr = "16-bit Bayer GBGB/RGRG"; break;
 	case V4L2_PIX_FMT_SGRBG16:	descr = "16-bit Bayer GRGR/BGBG"; break;
 	case V4L2_PIX_FMT_SRGGB16:	descr = "16-bit Bayer RGRG/GBGB"; break;
+	case V4L2_PIX_FMT_RAW_8:	descr = "8-bit Raw"; break;
+	case V4L2_PIX_FMT_RAW_CSI2_10:	descr = "10-bit Raw, CSI-2 Packed"; break;
+	case V4L2_PIX_FMT_RAW_CSI2_12:	descr = "12-bit Raw, CSI-2 Packed"; break;
+	case V4L2_PIX_FMT_RAW_CSI2_14:	descr = "14-bit Raw, CSI-2 Packed"; break;
 	case V4L2_PIX_FMT_RAW_CRU20:	descr = "14-bit Raw CRU Packed"; break;
 	case V4L2_PIX_FMT_SN9C20X_I420:	descr = "GSPCA SN9C20X I420"; break;
 	case V4L2_PIX_FMT_SPCA501:	descr = "GSPCA SPCA501"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 3dd9fa45dde1..c4c4f3eb67e1 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -745,6 +745,12 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_SGRBG16 v4l2_fourcc('G', 'R', '1', '6') /* 16  GRGR.. BGBG.. */
 #define V4L2_PIX_FMT_SRGGB16 v4l2_fourcc('R', 'G', '1', '6') /* 16  RGRG.. GBGB.. */
 
+/* Generic CSI-2 packed raw pixel formats */
+#define V4L2_PIX_FMT_RAW_8 v4l2_fourcc('R', 'A', 'W', '8')
+#define V4L2_PIX_FMT_RAW_CSI2_10 v4l2_fourcc('R', 'A', 'C', 'A')
+#define V4L2_PIX_FMT_RAW_CSI2_12 v4l2_fourcc('R', 'A', 'C', 'C')
+#define V4L2_PIX_FMT_RAW_CSI2_14 v4l2_fourcc('R', 'A', 'C', 'E')
+
 /* HSV formats */
 #define V4L2_PIX_FMT_HSV24 v4l2_fourcc('H', 'S', 'V', '3')
 #define V4L2_PIX_FMT_HSV32 v4l2_fourcc('H', 'S', 'V', '4')
-- 
2.47.2

Re: [PATCH v11 15/66] media: uapi: Add generic CSI-2 raw pixelformats

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:16PM +0300, Sakari Ailus wrote:
> Add generic raw pixelformats for bit depths 8, 10, 12 and 14. These
> formats are CSI-2 packed, apart from the 8-bit format.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../media/v4l/pixfmt-raw-generic.rst          | 175 ++++++++++++++++++
>  .../userspace-api/media/v4l/pixfmt.rst        |   1 +
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +
>  include/uapi/linux/videodev2.h                |   6 +
>  4 files changed, 186 insertions(+)
>  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
>
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> new file mode 100644
> index 000000000000..9fda7a36cd3a
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> @@ -0,0 +1,175 @@
> +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
> +
> +************************************************************************************************************************************************************************************************************************************************************************************
> +V4L2_PIX_FMT_RAW_8 ('RAW8'), V4L2_PIX_FMT_RAW_CSI2_10 ('RACA'), V4L2_PIX_FMT_RAW_CSI2_12 ('RACC'), V4L2_PIX_FMT_RAW_CSI2_14 ('RACE'), V4L2_PIX_FMT_RAW_16 ('RAWG'), V4L2_PIX_FMT_RAW_CSI2_20 ('RACK'), V4L2_PIX_FMT_RAW_24 ('RAWO'), V4L2_PIX_FMT_RAW_CSI2_28 ('RACS')
> +************************************************************************************************************************************************************************************************************************************************************************************

I see in your branch
"media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp"

Which adds pixel formats for the additional formats not documented in
this patch but mentioned here above in the header.

Should you squash it with this patch ?

> +
> +
> +Generic line-based image data formats

Does "line-based" applies to image formats as well or to metadata only ?

> +
> +
> +Description
> +===========
> +
> +These generic raw image data formats define the memory layout of the data
> +without defining the order of the pixels in the format or even the CFA (Colour

What about s/in the format// ?

> +Filter Array) itself. See also :ref:`source routes <subdev-routing>`.

I might have missed why the reference to the routing documentation.

> +
> +.. _v4l2-pix-fmt-raw-8:
> +
> +V4L2_PIX_FMT_RAW_8
> +------------------
> +
> +The V4L2_PIX_FMT_GENERIC_8 format is a plain 8-bit raw pixel data format. This
> +format is used on CSI-2 for 8 bits per :term:`Data Unit`.
> +
> +**Byte Order Of V4L2_PIX_FMT_RAW_8.**
> +Each cell is one byte. "P" denotes a pixel.
> +
> +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|
> +
> +.. flat-table:: Sample 4x2 Image Frame
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths: 12 8 8 8 8
> +
> +    * - start + 0:
> +      - P\ :sub:`00`
> +      - P\ :sub:`10`
> +      - P\ :sub:`20`
> +      - P\ :sub:`30`
> +    * - start + 4:
> +      - P\ :sub:`01`
> +      - P\ :sub:`11`
> +      - P\ :sub:`21`
> +      - P\ :sub:`31`
> +
> +.. _v4l2-pix-fmt-raw-csi2-10:
> +
> +V4L2_PIX_FMT_RAW_CSI2_10
> +------------------------
> +
> +V4L2_PIX_FMT_RAW_CSI2_10 contains 10-bit packed image data, with four bytes
> +containing the top 8 bits of the pixels followed by lowest 2 bits of the pixels
> +packed into one byte. This format is typically used by CSI-2 receivers with
> +a source that transmits MEDIA_BUS_FMT_RAW_10 and the CSI-2 receiver writes the
> +received data to memory as-is.
> +
> +The packing of the data follows the MIPI CSI-2 specification.
> +
> +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_10.**
> +Each cell is one byte. "P" denotes a pixel.
> +
> +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
> +
> +.. flat-table:: Sample 4x2 Image Frame
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths: 12 8 8 8 8 8
> +
> +    * - start + 0:
> +      - P\ :sub:`00 bits 9--2`
> +      - P\ :sub:`10 bits 9--2`
> +      - P\ :sub:`20 bits 9--2`
> +      - P\ :sub:`30 bits 9--2`
> +      - P\ :sub:`00 bits 1--0` (bits 1--0)
> +        P\ :sub:`10 bits 1--0` (bits 3--2)
> +        P\ :sub:`20 bits 1--0` (bits 5--4)
> +        P\ :sub:`30 bits 1--0` (bits 7--6)
> +    * - start + 5:
> +      - P\ :sub:`01 bits 9--2`
> +      - P\ :sub:`11 bits 9--2`
> +      - P\ :sub:`21 bits 9--2`
> +      - P\ :sub:`31 bits 9--2`
> +      - P\ :sub:`01 bits 1--0` (bits 1--0)
> +        P\ :sub:`11 bits 1--0` (bits 3--2)
> +        P\ :sub:`21 bits 1--0` (bits 5--4)
> +        P\ :sub:`31 bits 1--0` (bits 7--6)
> +
> +.. _v4l2-pix-fmt-raw-csi2-12:
> +
> +V4L2_PIX_FMT_RAW_CSI2_12
> +------------------------
> +
> +V4L2_PIX_FMT_RAW_CSI2_12 contains 12-bit packed image data, with two bytes
> +containing the top 8 bits of the pixels followed by lowest 4 bits of the pixels
> +packed into 1 byte. This format is typically used by CSI-2 receivers with
> +a source that transmits MEDIA_BUS_FMT_RAW_12 and the CSI-2 receiver writes the
> +received data to memory as-is.
> +
> +The packing of the data follows the MIPI CSI-2 specification.
> +
> +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_12.**
> +Each cell is one byte. "P" denotes a pixel.
> +
> +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
> +
> +.. flat-table:: Sample 4x2 Image Frame
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths: 12 8 8 8 8 8 8
> +
> +    * - start + 0:
> +      - P\ :sub:`00 bits 11--4`
> +      - P\ :sub:`10 bits 11--4`
> +      - P\ :sub:`00 bits 3--0` (bits 3--0)
> +        P\ :sub:`10 bits 3--0` (bits 7--4)
> +      - P\ :sub:`20 bits 11--4`
> +      - P\ :sub:`30 bits 11--4`
> +      - P\ :sub:`20 bits 3--0` (bits 3--0)
> +        P\ :sub:`30 bits 3--0` (bits 7--4)
> +    * - start + 6:
> +      - P\ :sub:`01 bits 11--4`
> +      - P\ :sub:`11 bits 11--4`
> +      - P\ :sub:`01 bits 3--0` (bits 3--0)
> +        P\ :sub:`11 bits 3--0` (bits 7--4)
> +      - P\ :sub:`21 bits 11--4`
> +      - P\ :sub:`31 bits 11--4`
> +      - P\ :sub:`21 bits 3--0` (bits 3--0)
> +        P\ :sub:`31 bits 3--0` (bits 7--4)
> +
> +.. _v4l2-pix-fmt-raw-csi2-14:
> +
> +V4L2_PIX_FMT_RAW_CSI2_14
> +------------------------
> +
> +V4L2_PIX_FMT_RAW_CSI2_14 contains 14-bit packed image data, with four bytes
> +containing the top 8 bits of the pixels followed by lowest 6 bits of the pixels
> +packed into three bytes. This format is typically used by CSI-2 receivers with a
> +source that transmits MEDIA_BUS_FMT_RAW_14 and the CSI-2 receiver writes the
> +received data to memory as-is.
> +
> +The packing of the data follows the MIPI CSI-2 specification.
> +
> +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_14.**
> +Each cell is one byte. "P" denotes a pixel.
> +
> +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.8cm}|p{1.8cm}|
> +
> +.. flat-table:: Sample 4x2 Image Frame
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths: 12 8 8 8 8 8 8 8
> +
> +    * - start + 0:
> +      - P\ :sub:`00 bits 13--6`
> +      - P\ :sub:`10 bits 13--6`
> +      - P\ :sub:`20 bits 13--6`
> +      - P\ :sub:`30 bits 13--6`
> +      - P\ :sub:`10 bits 1--0` (bits 7--6)
> +        P\ :sub:`00 bits 5--0` (bits 5--0)
> +      - P\ :sub:`20 bits 3--0` (bits 7--4)
> +        P\ :sub:`10 bits 5--2` (bits 3--0)
> +      - P\ :sub:`30 bits 5--0` (bits 7--2)
> +        P\ :sub:`20 bits 5--4` (bits 1--0)
> +    * - start + 7:
> +      - P\ :sub:`01 bits 13--6`
> +      - P\ :sub:`11 bits 13--6`
> +      - P\ :sub:`21 bits 13--6`
> +      - P\ :sub:`31 bits 13--6`
> +      - P\ :sub:`11 bits 1--0` (bits 7--6)
> +        P\ :sub:`01 bits 5--0` (bits 5--0)
> +      - P\ :sub:`21 bits 3--0` (bits 7--4)
> +        P\ :sub:`11 bits 5--2` (bits 3--0)
> +      - P\ :sub:`31 bits 5--0` (bits 7--2)
> +        P\ :sub:`21 bits 5--4` (bits 1--0)
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
> index 11dab4a90630..d917190c717d 100644
> --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> @@ -25,6 +25,7 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
>      pixfmt-indexed
>      pixfmt-rgb
>      pixfmt-bayer
> +    pixfmt-raw-generic
>      yuv-formats
>      hsv-formats
>      depth-formats
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 01cf52c3ea33..e538c1230631 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1434,6 +1434,10 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  	case V4L2_PIX_FMT_SGBRG16:	descr = "16-bit Bayer GBGB/RGRG"; break;
>  	case V4L2_PIX_FMT_SGRBG16:	descr = "16-bit Bayer GRGR/BGBG"; break;
>  	case V4L2_PIX_FMT_SRGGB16:	descr = "16-bit Bayer RGRG/GBGB"; break;
> +	case V4L2_PIX_FMT_RAW_8:	descr = "8-bit Raw"; break;
> +	case V4L2_PIX_FMT_RAW_CSI2_10:	descr = "10-bit Raw, CSI-2 Packed"; break;
> +	case V4L2_PIX_FMT_RAW_CSI2_12:	descr = "12-bit Raw, CSI-2 Packed"; break;
> +	case V4L2_PIX_FMT_RAW_CSI2_14:	descr = "14-bit Raw, CSI-2 Packed"; break;
>  	case V4L2_PIX_FMT_RAW_CRU20:	descr = "14-bit Raw CRU Packed"; break;
>  	case V4L2_PIX_FMT_SN9C20X_I420:	descr = "GSPCA SN9C20X I420"; break;
>  	case V4L2_PIX_FMT_SPCA501:	descr = "GSPCA SPCA501"; break;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 3dd9fa45dde1..c4c4f3eb67e1 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -745,6 +745,12 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_SGRBG16 v4l2_fourcc('G', 'R', '1', '6') /* 16  GRGR.. BGBG.. */
>  #define V4L2_PIX_FMT_SRGGB16 v4l2_fourcc('R', 'G', '1', '6') /* 16  RGRG.. GBGB.. */
>
> +/* Generic CSI-2 packed raw pixel formats */
> +#define V4L2_PIX_FMT_RAW_8 v4l2_fourcc('R', 'A', 'W', '8')
> +#define V4L2_PIX_FMT_RAW_CSI2_10 v4l2_fourcc('R', 'A', 'C', 'A')
> +#define V4L2_PIX_FMT_RAW_CSI2_12 v4l2_fourcc('R', 'A', 'C', 'C')
> +#define V4L2_PIX_FMT_RAW_CSI2_14 v4l2_fourcc('R', 'A', 'C', 'E')
> +
With
"media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp"

and the above minors clarified
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

>  /* HSV formats */
>  #define V4L2_PIX_FMT_HSV24 v4l2_fourcc('H', 'S', 'V', '3')
>  #define V4L2_PIX_FMT_HSV32 v4l2_fourcc('H', 'S', 'V', '4')
> --
> 2.47.2
>
>

Re: [PATCH v11 15/66] media: uapi: Add generic CSI-2 raw pixelformats

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 03:25:01PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:16PM +0300, Sakari Ailus wrote:
> > Add generic raw pixelformats for bit depths 8, 10, 12 and 14. These
> > formats are CSI-2 packed, apart from the 8-bit format.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../media/v4l/pixfmt-raw-generic.rst          | 175 ++++++++++++++++++
> >  .../userspace-api/media/v4l/pixfmt.rst        |   1 +
> >  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +
> >  include/uapi/linux/videodev2.h                |   6 +
> >  4 files changed, 186 insertions(+)
> >  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > new file mode 100644
> > index 000000000000..9fda7a36cd3a
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > @@ -0,0 +1,175 @@
> > +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
> > +
> > +************************************************************************************************************************************************************************************************************************************************************************************
> > +V4L2_PIX_FMT_RAW_8 ('RAW8'), V4L2_PIX_FMT_RAW_CSI2_10 ('RACA'), V4L2_PIX_FMT_RAW_CSI2_12 ('RACC'), V4L2_PIX_FMT_RAW_CSI2_14 ('RACE'), V4L2_PIX_FMT_RAW_16 ('RAWG'), V4L2_PIX_FMT_RAW_CSI2_20 ('RACK'), V4L2_PIX_FMT_RAW_24 ('RAWO'), V4L2_PIX_FMT_RAW_CSI2_28 ('RACS')
> > +************************************************************************************************************************************************************************************************************************************************************************************
> 
> I see in your branch
> "media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp"
> 
> Which adds pixel formats for the additional formats not documented in
> this patch but mentioned here above in the header.
> 
> Should you squash it with this patch ?

The reason these are being postponed is that we haven't yet seen devices
using them. The systems these are used in tend to be little endian while
the format documentation suggests big endian byte order. That'd be
inconvenient. So we'd like to see what do they actually use.

I'll move the extra ones to the other patch.

> 
> > +
> > +
> > +Generic line-based image data formats
> 
> Does "line-based" applies to image formats as well or to metadata only ?

These, too.

> 
> > +
> > +
> > +Description
> > +===========
> > +
> > +These generic raw image data formats define the memory layout of the data
> > +without defining the order of the pixels in the format or even the CFA (Colour
> 
> What about s/in the format// ?

Yes.

> 
> > +Filter Array) itself. See also :ref:`source routes <subdev-routing>`.
> 
> I might have missed why the reference to the routing documentation.

I think this bit can be removed now.

> 
> > +
> > +.. _v4l2-pix-fmt-raw-8:
> > +
> > +V4L2_PIX_FMT_RAW_8
> > +------------------
> > +
> > +The V4L2_PIX_FMT_GENERIC_8 format is a plain 8-bit raw pixel data format. This
> > +format is used on CSI-2 for 8 bits per :term:`Data Unit`.
> > +
> > +**Byte Order Of V4L2_PIX_FMT_RAW_8.**
> > +Each cell is one byte. "P" denotes a pixel.
> > +
> > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|
> > +
> > +.. flat-table:: Sample 4x2 Image Frame
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths: 12 8 8 8 8
> > +
> > +    * - start + 0:
> > +      - P\ :sub:`00`
> > +      - P\ :sub:`10`
> > +      - P\ :sub:`20`
> > +      - P\ :sub:`30`
> > +    * - start + 4:
> > +      - P\ :sub:`01`
> > +      - P\ :sub:`11`
> > +      - P\ :sub:`21`
> > +      - P\ :sub:`31`
> > +
> > +.. _v4l2-pix-fmt-raw-csi2-10:
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_10
> > +------------------------
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_10 contains 10-bit packed image data, with four bytes
> > +containing the top 8 bits of the pixels followed by lowest 2 bits of the pixels
> > +packed into one byte. This format is typically used by CSI-2 receivers with
> > +a source that transmits MEDIA_BUS_FMT_RAW_10 and the CSI-2 receiver writes the
> > +received data to memory as-is.
> > +
> > +The packing of the data follows the MIPI CSI-2 specification.
> > +
> > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_10.**
> > +Each cell is one byte. "P" denotes a pixel.
> > +
> > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
> > +
> > +.. flat-table:: Sample 4x2 Image Frame
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths: 12 8 8 8 8 8
> > +
> > +    * - start + 0:
> > +      - P\ :sub:`00 bits 9--2`
> > +      - P\ :sub:`10 bits 9--2`
> > +      - P\ :sub:`20 bits 9--2`
> > +      - P\ :sub:`30 bits 9--2`
> > +      - P\ :sub:`00 bits 1--0` (bits 1--0)
> > +        P\ :sub:`10 bits 1--0` (bits 3--2)
> > +        P\ :sub:`20 bits 1--0` (bits 5--4)
> > +        P\ :sub:`30 bits 1--0` (bits 7--6)
> > +    * - start + 5:
> > +      - P\ :sub:`01 bits 9--2`
> > +      - P\ :sub:`11 bits 9--2`
> > +      - P\ :sub:`21 bits 9--2`
> > +      - P\ :sub:`31 bits 9--2`
> > +      - P\ :sub:`01 bits 1--0` (bits 1--0)
> > +        P\ :sub:`11 bits 1--0` (bits 3--2)
> > +        P\ :sub:`21 bits 1--0` (bits 5--4)
> > +        P\ :sub:`31 bits 1--0` (bits 7--6)
> > +
> > +.. _v4l2-pix-fmt-raw-csi2-12:
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_12
> > +------------------------
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_12 contains 12-bit packed image data, with two bytes
> > +containing the top 8 bits of the pixels followed by lowest 4 bits of the pixels
> > +packed into 1 byte. This format is typically used by CSI-2 receivers with
> > +a source that transmits MEDIA_BUS_FMT_RAW_12 and the CSI-2 receiver writes the
> > +received data to memory as-is.
> > +
> > +The packing of the data follows the MIPI CSI-2 specification.
> > +
> > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_12.**
> > +Each cell is one byte. "P" denotes a pixel.
> > +
> > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
> > +
> > +.. flat-table:: Sample 4x2 Image Frame
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths: 12 8 8 8 8 8 8
> > +
> > +    * - start + 0:
> > +      - P\ :sub:`00 bits 11--4`
> > +      - P\ :sub:`10 bits 11--4`
> > +      - P\ :sub:`00 bits 3--0` (bits 3--0)
> > +        P\ :sub:`10 bits 3--0` (bits 7--4)
> > +      - P\ :sub:`20 bits 11--4`
> > +      - P\ :sub:`30 bits 11--4`
> > +      - P\ :sub:`20 bits 3--0` (bits 3--0)
> > +        P\ :sub:`30 bits 3--0` (bits 7--4)
> > +    * - start + 6:
> > +      - P\ :sub:`01 bits 11--4`
> > +      - P\ :sub:`11 bits 11--4`
> > +      - P\ :sub:`01 bits 3--0` (bits 3--0)
> > +        P\ :sub:`11 bits 3--0` (bits 7--4)
> > +      - P\ :sub:`21 bits 11--4`
> > +      - P\ :sub:`31 bits 11--4`
> > +      - P\ :sub:`21 bits 3--0` (bits 3--0)
> > +        P\ :sub:`31 bits 3--0` (bits 7--4)
> > +
> > +.. _v4l2-pix-fmt-raw-csi2-14:
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_14
> > +------------------------
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_14 contains 14-bit packed image data, with four bytes
> > +containing the top 8 bits of the pixels followed by lowest 6 bits of the pixels
> > +packed into three bytes. This format is typically used by CSI-2 receivers with a
> > +source that transmits MEDIA_BUS_FMT_RAW_14 and the CSI-2 receiver writes the
> > +received data to memory as-is.
> > +
> > +The packing of the data follows the MIPI CSI-2 specification.
> > +
> > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_14.**
> > +Each cell is one byte. "P" denotes a pixel.
> > +
> > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.8cm}|p{1.8cm}|
> > +
> > +.. flat-table:: Sample 4x2 Image Frame
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths: 12 8 8 8 8 8 8 8
> > +
> > +    * - start + 0:
> > +      - P\ :sub:`00 bits 13--6`
> > +      - P\ :sub:`10 bits 13--6`
> > +      - P\ :sub:`20 bits 13--6`
> > +      - P\ :sub:`30 bits 13--6`
> > +      - P\ :sub:`10 bits 1--0` (bits 7--6)
> > +        P\ :sub:`00 bits 5--0` (bits 5--0)
> > +      - P\ :sub:`20 bits 3--0` (bits 7--4)
> > +        P\ :sub:`10 bits 5--2` (bits 3--0)
> > +      - P\ :sub:`30 bits 5--0` (bits 7--2)
> > +        P\ :sub:`20 bits 5--4` (bits 1--0)
> > +    * - start + 7:
> > +      - P\ :sub:`01 bits 13--6`
> > +      - P\ :sub:`11 bits 13--6`
> > +      - P\ :sub:`21 bits 13--6`
> > +      - P\ :sub:`31 bits 13--6`
> > +      - P\ :sub:`11 bits 1--0` (bits 7--6)
> > +        P\ :sub:`01 bits 5--0` (bits 5--0)
> > +      - P\ :sub:`21 bits 3--0` (bits 7--4)
> > +        P\ :sub:`11 bits 5--2` (bits 3--0)
> > +      - P\ :sub:`31 bits 5--0` (bits 7--2)
> > +        P\ :sub:`21 bits 5--4` (bits 1--0)
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > index 11dab4a90630..d917190c717d 100644
> > --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > @@ -25,6 +25,7 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
> >      pixfmt-indexed
> >      pixfmt-rgb
> >      pixfmt-bayer
> > +    pixfmt-raw-generic
> >      yuv-formats
> >      hsv-formats
> >      depth-formats
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 01cf52c3ea33..e538c1230631 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1434,6 +1434,10 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >  	case V4L2_PIX_FMT_SGBRG16:	descr = "16-bit Bayer GBGB/RGRG"; break;
> >  	case V4L2_PIX_FMT_SGRBG16:	descr = "16-bit Bayer GRGR/BGBG"; break;
> >  	case V4L2_PIX_FMT_SRGGB16:	descr = "16-bit Bayer RGRG/GBGB"; break;
> > +	case V4L2_PIX_FMT_RAW_8:	descr = "8-bit Raw"; break;
> > +	case V4L2_PIX_FMT_RAW_CSI2_10:	descr = "10-bit Raw, CSI-2 Packed"; break;
> > +	case V4L2_PIX_FMT_RAW_CSI2_12:	descr = "12-bit Raw, CSI-2 Packed"; break;
> > +	case V4L2_PIX_FMT_RAW_CSI2_14:	descr = "14-bit Raw, CSI-2 Packed"; break;
> >  	case V4L2_PIX_FMT_RAW_CRU20:	descr = "14-bit Raw CRU Packed"; break;
> >  	case V4L2_PIX_FMT_SN9C20X_I420:	descr = "GSPCA SN9C20X I420"; break;
> >  	case V4L2_PIX_FMT_SPCA501:	descr = "GSPCA SPCA501"; break;
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 3dd9fa45dde1..c4c4f3eb67e1 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -745,6 +745,12 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_SGRBG16 v4l2_fourcc('G', 'R', '1', '6') /* 16  GRGR.. BGBG.. */
> >  #define V4L2_PIX_FMT_SRGGB16 v4l2_fourcc('R', 'G', '1', '6') /* 16  RGRG.. GBGB.. */
> >
> > +/* Generic CSI-2 packed raw pixel formats */
> > +#define V4L2_PIX_FMT_RAW_8 v4l2_fourcc('R', 'A', 'W', '8')
> > +#define V4L2_PIX_FMT_RAW_CSI2_10 v4l2_fourcc('R', 'A', 'C', 'A')
> > +#define V4L2_PIX_FMT_RAW_CSI2_12 v4l2_fourcc('R', 'A', 'C', 'C')
> > +#define V4L2_PIX_FMT_RAW_CSI2_14 v4l2_fourcc('R', 'A', 'C', 'E')
> > +
> With
> "media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp"
> 
> and the above minors clarified
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thank you!

> 
> >  /* HSV formats */
> >  #define V4L2_PIX_FMT_HSV24 v4l2_fourcc('H', 'S', 'V', '3')
> >  #define V4L2_PIX_FMT_HSV32 v4l2_fourcc('H', 'S', 'V', '4')
> > --
> > 2.47.2
> >
> >

-- 
Sakari Ailus

Re: [PATCH v11 15/66] media: uapi: Add generic CSI-2 raw pixelformats

[Laurent Pinchart]

On Tue, Sep 02, 2025 at 04:16:21PM +0300, Sakari Ailus wrote:
> On Mon, Sep 01, 2025 at 03:25:01PM +0200, Jacopo Mondi wrote:
> > On Mon, Aug 25, 2025 at 12:50:16PM +0300, Sakari Ailus wrote:
> > > Add generic raw pixelformats for bit depths 8, 10, 12 and 14. These
> > > formats are CSI-2 packed, apart from the 8-bit format.
> > >
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  .../media/v4l/pixfmt-raw-generic.rst          | 175 ++++++++++++++++++
> > >  .../userspace-api/media/v4l/pixfmt.rst        |   1 +
> > >  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +
> > >  include/uapi/linux/videodev2.h                |   6 +
> > >  4 files changed, 186 insertions(+)
> > >  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > > new file mode 100644
> > > index 000000000000..9fda7a36cd3a
> > > --- /dev/null
> > > +++ b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > > @@ -0,0 +1,175 @@
> > > +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
> > > +
> > > +************************************************************************************************************************************************************************************************************************************************************************************
> > > +V4L2_PIX_FMT_RAW_8 ('RAW8'), V4L2_PIX_FMT_RAW_CSI2_10 ('RACA'), V4L2_PIX_FMT_RAW_CSI2_12 ('RACC'), V4L2_PIX_FMT_RAW_CSI2_14 ('RACE'), V4L2_PIX_FMT_RAW_16 ('RAWG'), V4L2_PIX_FMT_RAW_CSI2_20 ('RACK'), V4L2_PIX_FMT_RAW_24 ('RAWO'), V4L2_PIX_FMT_RAW_CSI2_28 ('RACS')
> > > +************************************************************************************************************************************************************************************************************************************************************************************

You seem to have too many stars.

> > 
> > I see in your branch
> > "media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp"
> > 
> > Which adds pixel formats for the additional formats not documented in
> > this patch but mentioned here above in the header.
> > 
> > Should you squash it with this patch ?
> 
> The reason these are being postponed is that we haven't yet seen devices
> using them. The systems these are used in tend to be little endian while
> the format documentation suggests big endian byte order. That'd be
> inconvenient. So we'd like to see what do they actually use.
> 
> I'll move the extra ones to the other patch.
> 
> > > +
> > > +
> > > +Generic line-based image data formats
> > 
> > Does "line-based" applies to image formats as well or to metadata only ?
> 
> These, too.

I would write "Generic line-based raw image data formats".

> > > +
> > > +
> > > +Description
> > > +===========
> > > +
> > > +These generic raw image data formats define the memory layout of the data
> > > +without defining the order of the pixels in the format or even the CFA (Colour
> > 
> > What about s/in the format// ?
> 
> Yes.
> 
> > > +Filter Array) itself. See also :ref:`source routes <subdev-routing>`.
> > 
> > I might have missed why the reference to the routing documentation.
> 
> I think this bit can be removed now.

Agreed.

> > > +
> > > +.. _v4l2-pix-fmt-raw-8:
> > > +
> > > +V4L2_PIX_FMT_RAW_8
> > > +------------------
> > > +
> > > +The V4L2_PIX_FMT_GENERIC_8 format is a plain 8-bit raw pixel data format. This

s/V4L2_PIX_FMT_GENERIC_8/V4L2_PIX_FMT_RAW_8/

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> > > +format is used on CSI-2 for 8 bits per :term:`Data Unit`.
> > > +
> > > +**Byte Order Of V4L2_PIX_FMT_RAW_8.**
> > > +Each cell is one byte. "P" denotes a pixel.
> > > +
> > > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|
> > > +
> > > +.. flat-table:: Sample 4x2 Image Frame
> > > +    :header-rows:  0
> > > +    :stub-columns: 0
> > > +    :widths: 12 8 8 8 8
> > > +
> > > +    * - start + 0:
> > > +      - P\ :sub:`00`
> > > +      - P\ :sub:`10`
> > > +      - P\ :sub:`20`
> > > +      - P\ :sub:`30`
> > > +    * - start + 4:
> > > +      - P\ :sub:`01`
> > > +      - P\ :sub:`11`
> > > +      - P\ :sub:`21`
> > > +      - P\ :sub:`31`
> > > +
> > > +.. _v4l2-pix-fmt-raw-csi2-10:
> > > +
> > > +V4L2_PIX_FMT_RAW_CSI2_10
> > > +------------------------
> > > +
> > > +V4L2_PIX_FMT_RAW_CSI2_10 contains 10-bit packed image data, with four bytes
> > > +containing the top 8 bits of the pixels followed by lowest 2 bits of the pixels
> > > +packed into one byte. This format is typically used by CSI-2 receivers with
> > > +a source that transmits MEDIA_BUS_FMT_RAW_10 and the CSI-2 receiver writes the
> > > +received data to memory as-is.
> > > +
> > > +The packing of the data follows the MIPI CSI-2 specification.
> > > +
> > > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_10.**
> > > +Each cell is one byte. "P" denotes a pixel.
> > > +
> > > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
> > > +
> > > +.. flat-table:: Sample 4x2 Image Frame
> > > +    :header-rows:  0
> > > +    :stub-columns: 0
> > > +    :widths: 12 8 8 8 8 8
> > > +
> > > +    * - start + 0:
> > > +      - P\ :sub:`00 bits 9--2`
> > > +      - P\ :sub:`10 bits 9--2`
> > > +      - P\ :sub:`20 bits 9--2`
> > > +      - P\ :sub:`30 bits 9--2`
> > > +      - P\ :sub:`00 bits 1--0` (bits 1--0)
> > > +        P\ :sub:`10 bits 1--0` (bits 3--2)
> > > +        P\ :sub:`20 bits 1--0` (bits 5--4)
> > > +        P\ :sub:`30 bits 1--0` (bits 7--6)
> > > +    * - start + 5:
> > > +      - P\ :sub:`01 bits 9--2`
> > > +      - P\ :sub:`11 bits 9--2`
> > > +      - P\ :sub:`21 bits 9--2`
> > > +      - P\ :sub:`31 bits 9--2`
> > > +      - P\ :sub:`01 bits 1--0` (bits 1--0)
> > > +        P\ :sub:`11 bits 1--0` (bits 3--2)
> > > +        P\ :sub:`21 bits 1--0` (bits 5--4)
> > > +        P\ :sub:`31 bits 1--0` (bits 7--6)
> > > +
> > > +.. _v4l2-pix-fmt-raw-csi2-12:
> > > +
> > > +V4L2_PIX_FMT_RAW_CSI2_12
> > > +------------------------
> > > +
> > > +V4L2_PIX_FMT_RAW_CSI2_12 contains 12-bit packed image data, with two bytes
> > > +containing the top 8 bits of the pixels followed by lowest 4 bits of the pixels
> > > +packed into 1 byte. This format is typically used by CSI-2 receivers with
> > > +a source that transmits MEDIA_BUS_FMT_RAW_12 and the CSI-2 receiver writes the
> > > +received data to memory as-is.
> > > +
> > > +The packing of the data follows the MIPI CSI-2 specification.
> > > +
> > > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_12.**
> > > +Each cell is one byte. "P" denotes a pixel.
> > > +
> > > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
> > > +
> > > +.. flat-table:: Sample 4x2 Image Frame
> > > +    :header-rows:  0
> > > +    :stub-columns: 0
> > > +    :widths: 12 8 8 8 8 8 8
> > > +
> > > +    * - start + 0:
> > > +      - P\ :sub:`00 bits 11--4`
> > > +      - P\ :sub:`10 bits 11--4`
> > > +      - P\ :sub:`00 bits 3--0` (bits 3--0)
> > > +        P\ :sub:`10 bits 3--0` (bits 7--4)
> > > +      - P\ :sub:`20 bits 11--4`
> > > +      - P\ :sub:`30 bits 11--4`
> > > +      - P\ :sub:`20 bits 3--0` (bits 3--0)
> > > +        P\ :sub:`30 bits 3--0` (bits 7--4)
> > > +    * - start + 6:
> > > +      - P\ :sub:`01 bits 11--4`
> > > +      - P\ :sub:`11 bits 11--4`
> > > +      - P\ :sub:`01 bits 3--0` (bits 3--0)
> > > +        P\ :sub:`11 bits 3--0` (bits 7--4)
> > > +      - P\ :sub:`21 bits 11--4`
> > > +      - P\ :sub:`31 bits 11--4`
> > > +      - P\ :sub:`21 bits 3--0` (bits 3--0)
> > > +        P\ :sub:`31 bits 3--0` (bits 7--4)
> > > +
> > > +.. _v4l2-pix-fmt-raw-csi2-14:
> > > +
> > > +V4L2_PIX_FMT_RAW_CSI2_14
> > > +------------------------
> > > +
> > > +V4L2_PIX_FMT_RAW_CSI2_14 contains 14-bit packed image data, with four bytes
> > > +containing the top 8 bits of the pixels followed by lowest 6 bits of the pixels
> > > +packed into three bytes. This format is typically used by CSI-2 receivers with a
> > > +source that transmits MEDIA_BUS_FMT_RAW_14 and the CSI-2 receiver writes the
> > > +received data to memory as-is.
> > > +
> > > +The packing of the data follows the MIPI CSI-2 specification.
> > > +
> > > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_14.**
> > > +Each cell is one byte. "P" denotes a pixel.
> > > +
> > > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.8cm}|p{1.8cm}|
> > > +
> > > +.. flat-table:: Sample 4x2 Image Frame
> > > +    :header-rows:  0
> > > +    :stub-columns: 0
> > > +    :widths: 12 8 8 8 8 8 8 8
> > > +
> > > +    * - start + 0:
> > > +      - P\ :sub:`00 bits 13--6`
> > > +      - P\ :sub:`10 bits 13--6`
> > > +      - P\ :sub:`20 bits 13--6`
> > > +      - P\ :sub:`30 bits 13--6`
> > > +      - P\ :sub:`10 bits 1--0` (bits 7--6)
> > > +        P\ :sub:`00 bits 5--0` (bits 5--0)
> > > +      - P\ :sub:`20 bits 3--0` (bits 7--4)
> > > +        P\ :sub:`10 bits 5--2` (bits 3--0)
> > > +      - P\ :sub:`30 bits 5--0` (bits 7--2)
> > > +        P\ :sub:`20 bits 5--4` (bits 1--0)
> > > +    * - start + 7:
> > > +      - P\ :sub:`01 bits 13--6`
> > > +      - P\ :sub:`11 bits 13--6`
> > > +      - P\ :sub:`21 bits 13--6`
> > > +      - P\ :sub:`31 bits 13--6`
> > > +      - P\ :sub:`11 bits 1--0` (bits 7--6)
> > > +        P\ :sub:`01 bits 5--0` (bits 5--0)
> > > +      - P\ :sub:`21 bits 3--0` (bits 7--4)
> > > +        P\ :sub:`11 bits 5--2` (bits 3--0)
> > > +      - P\ :sub:`31 bits 5--0` (bits 7--2)
> > > +        P\ :sub:`21 bits 5--4` (bits 1--0)
> > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > > index 11dab4a90630..d917190c717d 100644
> > > --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> > > +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > > @@ -25,6 +25,7 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
> > >      pixfmt-indexed
> > >      pixfmt-rgb
> > >      pixfmt-bayer
> > > +    pixfmt-raw-generic
> > >      yuv-formats
> > >      hsv-formats
> > >      depth-formats
> > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > index 01cf52c3ea33..e538c1230631 100644
> > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > @@ -1434,6 +1434,10 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> > >  	case V4L2_PIX_FMT_SGBRG16:	descr = "16-bit Bayer GBGB/RGRG"; break;
> > >  	case V4L2_PIX_FMT_SGRBG16:	descr = "16-bit Bayer GRGR/BGBG"; break;
> > >  	case V4L2_PIX_FMT_SRGGB16:	descr = "16-bit Bayer RGRG/GBGB"; break;
> > > +	case V4L2_PIX_FMT_RAW_8:	descr = "8-bit Raw"; break;
> > > +	case V4L2_PIX_FMT_RAW_CSI2_10:	descr = "10-bit Raw, CSI-2 Packed"; break;
> > > +	case V4L2_PIX_FMT_RAW_CSI2_12:	descr = "12-bit Raw, CSI-2 Packed"; break;
> > > +	case V4L2_PIX_FMT_RAW_CSI2_14:	descr = "14-bit Raw, CSI-2 Packed"; break;
> > >  	case V4L2_PIX_FMT_RAW_CRU20:	descr = "14-bit Raw CRU Packed"; break;
> > >  	case V4L2_PIX_FMT_SN9C20X_I420:	descr = "GSPCA SN9C20X I420"; break;
> > >  	case V4L2_PIX_FMT_SPCA501:	descr = "GSPCA SPCA501"; break;
> > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > index 3dd9fa45dde1..c4c4f3eb67e1 100644
> > > --- a/include/uapi/linux/videodev2.h
> > > +++ b/include/uapi/linux/videodev2.h
> > > @@ -745,6 +745,12 @@ struct v4l2_pix_format {
> > >  #define V4L2_PIX_FMT_SGRBG16 v4l2_fourcc('G', 'R', '1', '6') /* 16  GRGR.. BGBG.. */
> > >  #define V4L2_PIX_FMT_SRGGB16 v4l2_fourcc('R', 'G', '1', '6') /* 16  RGRG.. GBGB.. */
> > >
> > > +/* Generic CSI-2 packed raw pixel formats */
> > > +#define V4L2_PIX_FMT_RAW_8 v4l2_fourcc('R', 'A', 'W', '8')
> > > +#define V4L2_PIX_FMT_RAW_CSI2_10 v4l2_fourcc('R', 'A', 'C', 'A')
> > > +#define V4L2_PIX_FMT_RAW_CSI2_12 v4l2_fourcc('R', 'A', 'C', 'C')
> > > +#define V4L2_PIX_FMT_RAW_CSI2_14 v4l2_fourcc('R', 'A', 'C', 'E')
> > > +
> >
> > With
> > "media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp"
> > 
> > and the above minors clarified
> > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> 
> Thank you!
> 
> > >  /* HSV formats */
> > >  #define V4L2_PIX_FMT_HSV24 v4l2_fourcc('H', 'S', 'V', '3')
> > >  #define V4L2_PIX_FMT_HSV32 v4l2_fourcc('H', 'S', 'V', '4')

-- 
Regards,

Laurent Pinchart

Re: [PATCH v11 15/66] media: uapi: Add generic CSI-2 raw pixelformats

[Sakari Ailus]

Hi Laurent,

On Wed, Sep 03, 2025 at 04:20:52PM +0200, Laurent Pinchart wrote:
> On Tue, Sep 02, 2025 at 04:16:21PM +0300, Sakari Ailus wrote:
> > On Mon, Sep 01, 2025 at 03:25:01PM +0200, Jacopo Mondi wrote:
> > > On Mon, Aug 25, 2025 at 12:50:16PM +0300, Sakari Ailus wrote:
> > > > Add generic raw pixelformats for bit depths 8, 10, 12 and 14. These
> > > > formats are CSI-2 packed, apart from the 8-bit format.
> > > >
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > ---
> > > >  .../media/v4l/pixfmt-raw-generic.rst          | 175 ++++++++++++++++++
> > > >  .../userspace-api/media/v4l/pixfmt.rst        |   1 +
> > > >  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +
> > > >  include/uapi/linux/videodev2.h                |   6 +
> > > >  4 files changed, 186 insertions(+)
> > > >  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > > >
> > > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > > > new file mode 100644
> > > > index 000000000000..9fda7a36cd3a
> > > > --- /dev/null
> > > > +++ b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > > > @@ -0,0 +1,175 @@
> > > > +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
> > > > +
> > > > +************************************************************************************************************************************************************************************************************************************************************************************
> > > > +V4L2_PIX_FMT_RAW_8 ('RAW8'), V4L2_PIX_FMT_RAW_CSI2_10 ('RACA'), V4L2_PIX_FMT_RAW_CSI2_12 ('RACC'), V4L2_PIX_FMT_RAW_CSI2_14 ('RACE'), V4L2_PIX_FMT_RAW_16 ('RAWG'), V4L2_PIX_FMT_RAW_CSI2_20 ('RACK'), V4L2_PIX_FMT_RAW_24 ('RAWO'), V4L2_PIX_FMT_RAW_CSI2_28 ('RACS')
> > > > +************************************************************************************************************************************************************************************************************************************************************************************
> 
> You seem to have too many stars.

I'll try to find someone who would be interested in the extra stars.

> 
> > > 
> > > I see in your branch
> > > "media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp"
> > > 
> > > Which adds pixel formats for the additional formats not documented in
> > > this patch but mentioned here above in the header.
> > > 
> > > Should you squash it with this patch ?
> > 
> > The reason these are being postponed is that we haven't yet seen devices
> > using them. The systems these are used in tend to be little endian while
> > the format documentation suggests big endian byte order. That'd be
> > inconvenient. So we'd like to see what do they actually use.
> > 
> > I'll move the extra ones to the other patch.
> > 
> > > > +
> > > > +
> > > > +Generic line-based image data formats
> > > 
> > > Does "line-based" applies to image formats as well or to metadata only ?
> > 
> > These, too.
> 
> I would write "Generic line-based raw image data formats".

Sounds good.

> 
> > > > +
> > > > +
> > > > +Description
> > > > +===========
> > > > +
> > > > +These generic raw image data formats define the memory layout of the data
> > > > +without defining the order of the pixels in the format or even the CFA (Colour
> > > 
> > > What about s/in the format// ?
> > 
> > Yes.
> > 
> > > > +Filter Array) itself. See also :ref:`source routes <subdev-routing>`.
> > > 
> > > I might have missed why the reference to the routing documentation.
> > 
> > I think this bit can be removed now.
> 
> Agreed.
> 
> > > > +
> > > > +.. _v4l2-pix-fmt-raw-8:
> > > > +
> > > > +V4L2_PIX_FMT_RAW_8
> > > > +------------------
> > > > +
> > > > +The V4L2_PIX_FMT_GENERIC_8 format is a plain 8-bit raw pixel data format. This
> 
> s/V4L2_PIX_FMT_GENERIC_8/V4L2_PIX_FMT_RAW_8/

Fixed.

> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thanks!

> 
> > > > +format is used on CSI-2 for 8 bits per :term:`Data Unit`.
> > > > +
> > > > +**Byte Order Of V4L2_PIX_FMT_RAW_8.**
> > > > +Each cell is one byte. "P" denotes a pixel.
> > > > +
> > > > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|
> > > > +
> > > > +.. flat-table:: Sample 4x2 Image Frame
> > > > +    :header-rows:  0
> > > > +    :stub-columns: 0
> > > > +    :widths: 12 8 8 8 8
> > > > +
> > > > +    * - start + 0:
> > > > +      - P\ :sub:`00`
> > > > +      - P\ :sub:`10`
> > > > +      - P\ :sub:`20`
> > > > +      - P\ :sub:`30`
> > > > +    * - start + 4:
> > > > +      - P\ :sub:`01`
> > > > +      - P\ :sub:`11`
> > > > +      - P\ :sub:`21`
> > > > +      - P\ :sub:`31`
> > > > +
> > > > +.. _v4l2-pix-fmt-raw-csi2-10:
> > > > +
> > > > +V4L2_PIX_FMT_RAW_CSI2_10
> > > > +------------------------
> > > > +
> > > > +V4L2_PIX_FMT_RAW_CSI2_10 contains 10-bit packed image data, with four bytes
> > > > +containing the top 8 bits of the pixels followed by lowest 2 bits of the pixels
> > > > +packed into one byte. This format is typically used by CSI-2 receivers with
> > > > +a source that transmits MEDIA_BUS_FMT_RAW_10 and the CSI-2 receiver writes the
> > > > +received data to memory as-is.
> > > > +
> > > > +The packing of the data follows the MIPI CSI-2 specification.
> > > > +
> > > > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_10.**
> > > > +Each cell is one byte. "P" denotes a pixel.
> > > > +
> > > > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
> > > > +
> > > > +.. flat-table:: Sample 4x2 Image Frame
> > > > +    :header-rows:  0
> > > > +    :stub-columns: 0
> > > > +    :widths: 12 8 8 8 8 8
> > > > +
> > > > +    * - start + 0:
> > > > +      - P\ :sub:`00 bits 9--2`
> > > > +      - P\ :sub:`10 bits 9--2`
> > > > +      - P\ :sub:`20 bits 9--2`
> > > > +      - P\ :sub:`30 bits 9--2`
> > > > +      - P\ :sub:`00 bits 1--0` (bits 1--0)
> > > > +        P\ :sub:`10 bits 1--0` (bits 3--2)
> > > > +        P\ :sub:`20 bits 1--0` (bits 5--4)
> > > > +        P\ :sub:`30 bits 1--0` (bits 7--6)
> > > > +    * - start + 5:
> > > > +      - P\ :sub:`01 bits 9--2`
> > > > +      - P\ :sub:`11 bits 9--2`
> > > > +      - P\ :sub:`21 bits 9--2`
> > > > +      - P\ :sub:`31 bits 9--2`
> > > > +      - P\ :sub:`01 bits 1--0` (bits 1--0)
> > > > +        P\ :sub:`11 bits 1--0` (bits 3--2)
> > > > +        P\ :sub:`21 bits 1--0` (bits 5--4)
> > > > +        P\ :sub:`31 bits 1--0` (bits 7--6)
> > > > +
> > > > +.. _v4l2-pix-fmt-raw-csi2-12:
> > > > +
> > > > +V4L2_PIX_FMT_RAW_CSI2_12
> > > > +------------------------
> > > > +
> > > > +V4L2_PIX_FMT_RAW_CSI2_12 contains 12-bit packed image data, with two bytes
> > > > +containing the top 8 bits of the pixels followed by lowest 4 bits of the pixels
> > > > +packed into 1 byte. This format is typically used by CSI-2 receivers with
> > > > +a source that transmits MEDIA_BUS_FMT_RAW_12 and the CSI-2 receiver writes the
> > > > +received data to memory as-is.
> > > > +
> > > > +The packing of the data follows the MIPI CSI-2 specification.
> > > > +
> > > > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_12.**
> > > > +Each cell is one byte. "P" denotes a pixel.
> > > > +
> > > > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
> > > > +
> > > > +.. flat-table:: Sample 4x2 Image Frame
> > > > +    :header-rows:  0
> > > > +    :stub-columns: 0
> > > > +    :widths: 12 8 8 8 8 8 8
> > > > +
> > > > +    * - start + 0:
> > > > +      - P\ :sub:`00 bits 11--4`
> > > > +      - P\ :sub:`10 bits 11--4`
> > > > +      - P\ :sub:`00 bits 3--0` (bits 3--0)
> > > > +        P\ :sub:`10 bits 3--0` (bits 7--4)
> > > > +      - P\ :sub:`20 bits 11--4`
> > > > +      - P\ :sub:`30 bits 11--4`
> > > > +      - P\ :sub:`20 bits 3--0` (bits 3--0)
> > > > +        P\ :sub:`30 bits 3--0` (bits 7--4)
> > > > +    * - start + 6:
> > > > +      - P\ :sub:`01 bits 11--4`
> > > > +      - P\ :sub:`11 bits 11--4`
> > > > +      - P\ :sub:`01 bits 3--0` (bits 3--0)
> > > > +        P\ :sub:`11 bits 3--0` (bits 7--4)
> > > > +      - P\ :sub:`21 bits 11--4`
> > > > +      - P\ :sub:`31 bits 11--4`
> > > > +      - P\ :sub:`21 bits 3--0` (bits 3--0)
> > > > +        P\ :sub:`31 bits 3--0` (bits 7--4)
> > > > +
> > > > +.. _v4l2-pix-fmt-raw-csi2-14:
> > > > +
> > > > +V4L2_PIX_FMT_RAW_CSI2_14
> > > > +------------------------
> > > > +
> > > > +V4L2_PIX_FMT_RAW_CSI2_14 contains 14-bit packed image data, with four bytes
> > > > +containing the top 8 bits of the pixels followed by lowest 6 bits of the pixels
> > > > +packed into three bytes. This format is typically used by CSI-2 receivers with a
> > > > +source that transmits MEDIA_BUS_FMT_RAW_14 and the CSI-2 receiver writes the
> > > > +received data to memory as-is.
> > > > +
> > > > +The packing of the data follows the MIPI CSI-2 specification.
> > > > +
> > > > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_14.**
> > > > +Each cell is one byte. "P" denotes a pixel.
> > > > +
> > > > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.8cm}|p{1.8cm}|
> > > > +
> > > > +.. flat-table:: Sample 4x2 Image Frame
> > > > +    :header-rows:  0
> > > > +    :stub-columns: 0
> > > > +    :widths: 12 8 8 8 8 8 8 8
> > > > +
> > > > +    * - start + 0:
> > > > +      - P\ :sub:`00 bits 13--6`
> > > > +      - P\ :sub:`10 bits 13--6`
> > > > +      - P\ :sub:`20 bits 13--6`
> > > > +      - P\ :sub:`30 bits 13--6`
> > > > +      - P\ :sub:`10 bits 1--0` (bits 7--6)
> > > > +        P\ :sub:`00 bits 5--0` (bits 5--0)
> > > > +      - P\ :sub:`20 bits 3--0` (bits 7--4)
> > > > +        P\ :sub:`10 bits 5--2` (bits 3--0)
> > > > +      - P\ :sub:`30 bits 5--0` (bits 7--2)
> > > > +        P\ :sub:`20 bits 5--4` (bits 1--0)
> > > > +    * - start + 7:
> > > > +      - P\ :sub:`01 bits 13--6`
> > > > +      - P\ :sub:`11 bits 13--6`
> > > > +      - P\ :sub:`21 bits 13--6`
> > > > +      - P\ :sub:`31 bits 13--6`
> > > > +      - P\ :sub:`11 bits 1--0` (bits 7--6)
> > > > +        P\ :sub:`01 bits 5--0` (bits 5--0)
> > > > +      - P\ :sub:`21 bits 3--0` (bits 7--4)
> > > > +        P\ :sub:`11 bits 5--2` (bits 3--0)
> > > > +      - P\ :sub:`31 bits 5--0` (bits 7--2)
> > > > +        P\ :sub:`21 bits 5--4` (bits 1--0)
> > > > diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > > > index 11dab4a90630..d917190c717d 100644
> > > > --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > > > @@ -25,6 +25,7 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
> > > >      pixfmt-indexed
> > > >      pixfmt-rgb
> > > >      pixfmt-bayer
> > > > +    pixfmt-raw-generic
> > > >      yuv-formats
> > > >      hsv-formats
> > > >      depth-formats
> > > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > index 01cf52c3ea33..e538c1230631 100644
> > > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > @@ -1434,6 +1434,10 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> > > >  	case V4L2_PIX_FMT_SGBRG16:	descr = "16-bit Bayer GBGB/RGRG"; break;
> > > >  	case V4L2_PIX_FMT_SGRBG16:	descr = "16-bit Bayer GRGR/BGBG"; break;
> > > >  	case V4L2_PIX_FMT_SRGGB16:	descr = "16-bit Bayer RGRG/GBGB"; break;
> > > > +	case V4L2_PIX_FMT_RAW_8:	descr = "8-bit Raw"; break;
> > > > +	case V4L2_PIX_FMT_RAW_CSI2_10:	descr = "10-bit Raw, CSI-2 Packed"; break;
> > > > +	case V4L2_PIX_FMT_RAW_CSI2_12:	descr = "12-bit Raw, CSI-2 Packed"; break;
> > > > +	case V4L2_PIX_FMT_RAW_CSI2_14:	descr = "14-bit Raw, CSI-2 Packed"; break;
> > > >  	case V4L2_PIX_FMT_RAW_CRU20:	descr = "14-bit Raw CRU Packed"; break;
> > > >  	case V4L2_PIX_FMT_SN9C20X_I420:	descr = "GSPCA SN9C20X I420"; break;
> > > >  	case V4L2_PIX_FMT_SPCA501:	descr = "GSPCA SPCA501"; break;
> > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > > > index 3dd9fa45dde1..c4c4f3eb67e1 100644
> > > > --- a/include/uapi/linux/videodev2.h
> > > > +++ b/include/uapi/linux/videodev2.h
> > > > @@ -745,6 +745,12 @@ struct v4l2_pix_format {
> > > >  #define V4L2_PIX_FMT_SGRBG16 v4l2_fourcc('G', 'R', '1', '6') /* 16  GRGR.. BGBG.. */
> > > >  #define V4L2_PIX_FMT_SRGGB16 v4l2_fourcc('R', 'G', '1', '6') /* 16  RGRG.. GBGB.. */
> > > >
> > > > +/* Generic CSI-2 packed raw pixel formats */
> > > > +#define V4L2_PIX_FMT_RAW_8 v4l2_fourcc('R', 'A', 'W', '8')
> > > > +#define V4L2_PIX_FMT_RAW_CSI2_10 v4l2_fourcc('R', 'A', 'C', 'A')
> > > > +#define V4L2_PIX_FMT_RAW_CSI2_12 v4l2_fourcc('R', 'A', 'C', 'C')
> > > > +#define V4L2_PIX_FMT_RAW_CSI2_14 v4l2_fourcc('R', 'A', 'C', 'E')
> > > > +
> > >
> > > With
> > > "media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp"
> > > 
> > > and the above minors clarified
> > > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > 
> > Thank you!
> > 
> > > >  /* HSV formats */
> > > >  #define V4L2_PIX_FMT_HSV24 v4l2_fourcc('H', 'S', 'V', '3')
> > > >  #define V4L2_PIX_FMT_HSV32 v4l2_fourcc('H', 'S', 'V', '4')
> 

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v11 15/66] media: uapi: Add generic CSI-2 raw pixelformats

[Jacopo Mondi]

Hi again

On Mon, Sep 01, 2025 at 03:25:06PM +0200, Jacopo Mondi wrote:
> Hi Sakari
>
> On Mon, Aug 25, 2025 at 12:50:16PM +0300, Sakari Ailus wrote:
> > Add generic raw pixelformats for bit depths 8, 10, 12 and 14. These
> > formats are CSI-2 packed, apart from the 8-bit format.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../media/v4l/pixfmt-raw-generic.rst          | 175 ++++++++++++++++++
> >  .../userspace-api/media/v4l/pixfmt.rst        |   1 +
> >  drivers/media/v4l2-core/v4l2-ioctl.c          |   4 +
> >  include/uapi/linux/videodev2.h                |   6 +
> >  4 files changed, 186 insertions(+)
> >  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > new file mode 100644
> > index 000000000000..9fda7a36cd3a
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > @@ -0,0 +1,175 @@
> > +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
> > +
> > +************************************************************************************************************************************************************************************************************************************************************************************
> > +V4L2_PIX_FMT_RAW_8 ('RAW8'), V4L2_PIX_FMT_RAW_CSI2_10 ('RACA'), V4L2_PIX_FMT_RAW_CSI2_12 ('RACC'), V4L2_PIX_FMT_RAW_CSI2_14 ('RACE'), V4L2_PIX_FMT_RAW_16 ('RAWG'), V4L2_PIX_FMT_RAW_CSI2_20 ('RACK'), V4L2_PIX_FMT_RAW_24 ('RAWO'), V4L2_PIX_FMT_RAW_CSI2_28 ('RACS')
> > +************************************************************************************************************************************************************************************************************************************************************************************
>
> I see in your branch
> "media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp"
>
> Which adds pixel formats for the additional formats not documented in
> this patch but mentioned here above in the header.
>
> Should you squash it with this patch ?

Oh, I just read in the cover letter:
- Postpone adding formats higher than 14 bpp

So I think you should adjust the title of the page here to remove
these formats with bpp > 14
>
> > +
> > +
> > +Generic line-based image data formats
>
> Does "line-based" applies to image formats as well or to metadata only ?
>
> > +
> > +
> > +Description
> > +===========
> > +
> > +These generic raw image data formats define the memory layout of the data
> > +without defining the order of the pixels in the format or even the CFA (Colour
>
> What about s/in the format// ?
>
> > +Filter Array) itself. See also :ref:`source routes <subdev-routing>`.
>
> I might have missed why the reference to the routing documentation.
>
> > +
> > +.. _v4l2-pix-fmt-raw-8:
> > +
> > +V4L2_PIX_FMT_RAW_8
> > +------------------
> > +
> > +The V4L2_PIX_FMT_GENERIC_8 format is a plain 8-bit raw pixel data format. This
> > +format is used on CSI-2 for 8 bits per :term:`Data Unit`.
> > +
> > +**Byte Order Of V4L2_PIX_FMT_RAW_8.**
> > +Each cell is one byte. "P" denotes a pixel.
> > +
> > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|
> > +
> > +.. flat-table:: Sample 4x2 Image Frame
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths: 12 8 8 8 8
> > +
> > +    * - start + 0:
> > +      - P\ :sub:`00`
> > +      - P\ :sub:`10`
> > +      - P\ :sub:`20`
> > +      - P\ :sub:`30`
> > +    * - start + 4:
> > +      - P\ :sub:`01`
> > +      - P\ :sub:`11`
> > +      - P\ :sub:`21`
> > +      - P\ :sub:`31`
> > +
> > +.. _v4l2-pix-fmt-raw-csi2-10:
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_10
> > +------------------------
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_10 contains 10-bit packed image data, with four bytes
> > +containing the top 8 bits of the pixels followed by lowest 2 bits of the pixels
> > +packed into one byte. This format is typically used by CSI-2 receivers with
> > +a source that transmits MEDIA_BUS_FMT_RAW_10 and the CSI-2 receiver writes the
> > +received data to memory as-is.
> > +
> > +The packing of the data follows the MIPI CSI-2 specification.
> > +
> > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_10.**
> > +Each cell is one byte. "P" denotes a pixel.
> > +
> > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
> > +
> > +.. flat-table:: Sample 4x2 Image Frame
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths: 12 8 8 8 8 8
> > +
> > +    * - start + 0:
> > +      - P\ :sub:`00 bits 9--2`
> > +      - P\ :sub:`10 bits 9--2`
> > +      - P\ :sub:`20 bits 9--2`
> > +      - P\ :sub:`30 bits 9--2`
> > +      - P\ :sub:`00 bits 1--0` (bits 1--0)
> > +        P\ :sub:`10 bits 1--0` (bits 3--2)
> > +        P\ :sub:`20 bits 1--0` (bits 5--4)
> > +        P\ :sub:`30 bits 1--0` (bits 7--6)
> > +    * - start + 5:
> > +      - P\ :sub:`01 bits 9--2`
> > +      - P\ :sub:`11 bits 9--2`
> > +      - P\ :sub:`21 bits 9--2`
> > +      - P\ :sub:`31 bits 9--2`
> > +      - P\ :sub:`01 bits 1--0` (bits 1--0)
> > +        P\ :sub:`11 bits 1--0` (bits 3--2)
> > +        P\ :sub:`21 bits 1--0` (bits 5--4)
> > +        P\ :sub:`31 bits 1--0` (bits 7--6)
> > +
> > +.. _v4l2-pix-fmt-raw-csi2-12:
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_12
> > +------------------------
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_12 contains 12-bit packed image data, with two bytes
> > +containing the top 8 bits of the pixels followed by lowest 4 bits of the pixels
> > +packed into 1 byte. This format is typically used by CSI-2 receivers with
> > +a source that transmits MEDIA_BUS_FMT_RAW_12 and the CSI-2 receiver writes the
> > +received data to memory as-is.
> > +
> > +The packing of the data follows the MIPI CSI-2 specification.
> > +
> > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_12.**
> > +Each cell is one byte. "P" denotes a pixel.
> > +
> > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
> > +
> > +.. flat-table:: Sample 4x2 Image Frame
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths: 12 8 8 8 8 8 8
> > +
> > +    * - start + 0:
> > +      - P\ :sub:`00 bits 11--4`
> > +      - P\ :sub:`10 bits 11--4`
> > +      - P\ :sub:`00 bits 3--0` (bits 3--0)
> > +        P\ :sub:`10 bits 3--0` (bits 7--4)
> > +      - P\ :sub:`20 bits 11--4`
> > +      - P\ :sub:`30 bits 11--4`
> > +      - P\ :sub:`20 bits 3--0` (bits 3--0)
> > +        P\ :sub:`30 bits 3--0` (bits 7--4)
> > +    * - start + 6:
> > +      - P\ :sub:`01 bits 11--4`
> > +      - P\ :sub:`11 bits 11--4`
> > +      - P\ :sub:`01 bits 3--0` (bits 3--0)
> > +        P\ :sub:`11 bits 3--0` (bits 7--4)
> > +      - P\ :sub:`21 bits 11--4`
> > +      - P\ :sub:`31 bits 11--4`
> > +      - P\ :sub:`21 bits 3--0` (bits 3--0)
> > +        P\ :sub:`31 bits 3--0` (bits 7--4)
> > +
> > +.. _v4l2-pix-fmt-raw-csi2-14:
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_14
> > +------------------------
> > +
> > +V4L2_PIX_FMT_RAW_CSI2_14 contains 14-bit packed image data, with four bytes
> > +containing the top 8 bits of the pixels followed by lowest 6 bits of the pixels
> > +packed into three bytes. This format is typically used by CSI-2 receivers with a
> > +source that transmits MEDIA_BUS_FMT_RAW_14 and the CSI-2 receiver writes the
> > +received data to memory as-is.
> > +
> > +The packing of the data follows the MIPI CSI-2 specification.
> > +
> > +**Byte Order Of V4L2_PIX_FMT_RAW_CSI2_14.**
> > +Each cell is one byte. "P" denotes a pixel.
> > +
> > +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.8cm}|p{1.8cm}|
> > +
> > +.. flat-table:: Sample 4x2 Image Frame
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths: 12 8 8 8 8 8 8 8
> > +
> > +    * - start + 0:
> > +      - P\ :sub:`00 bits 13--6`
> > +      - P\ :sub:`10 bits 13--6`
> > +      - P\ :sub:`20 bits 13--6`
> > +      - P\ :sub:`30 bits 13--6`
> > +      - P\ :sub:`10 bits 1--0` (bits 7--6)
> > +        P\ :sub:`00 bits 5--0` (bits 5--0)
> > +      - P\ :sub:`20 bits 3--0` (bits 7--4)
> > +        P\ :sub:`10 bits 5--2` (bits 3--0)
> > +      - P\ :sub:`30 bits 5--0` (bits 7--2)
> > +        P\ :sub:`20 bits 5--4` (bits 1--0)
> > +    * - start + 7:
> > +      - P\ :sub:`01 bits 13--6`
> > +      - P\ :sub:`11 bits 13--6`
> > +      - P\ :sub:`21 bits 13--6`
> > +      - P\ :sub:`31 bits 13--6`
> > +      - P\ :sub:`11 bits 1--0` (bits 7--6)
> > +        P\ :sub:`01 bits 5--0` (bits 5--0)
> > +      - P\ :sub:`21 bits 3--0` (bits 7--4)
> > +        P\ :sub:`11 bits 5--2` (bits 3--0)
> > +      - P\ :sub:`31 bits 5--0` (bits 7--2)
> > +        P\ :sub:`21 bits 5--4` (bits 1--0)
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > index 11dab4a90630..d917190c717d 100644
> > --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > @@ -25,6 +25,7 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
> >      pixfmt-indexed
> >      pixfmt-rgb
> >      pixfmt-bayer
> > +    pixfmt-raw-generic
> >      yuv-formats
> >      hsv-formats
> >      depth-formats
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 01cf52c3ea33..e538c1230631 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1434,6 +1434,10 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >  	case V4L2_PIX_FMT_SGBRG16:	descr = "16-bit Bayer GBGB/RGRG"; break;
> >  	case V4L2_PIX_FMT_SGRBG16:	descr = "16-bit Bayer GRGR/BGBG"; break;
> >  	case V4L2_PIX_FMT_SRGGB16:	descr = "16-bit Bayer RGRG/GBGB"; break;
> > +	case V4L2_PIX_FMT_RAW_8:	descr = "8-bit Raw"; break;
> > +	case V4L2_PIX_FMT_RAW_CSI2_10:	descr = "10-bit Raw, CSI-2 Packed"; break;
> > +	case V4L2_PIX_FMT_RAW_CSI2_12:	descr = "12-bit Raw, CSI-2 Packed"; break;
> > +	case V4L2_PIX_FMT_RAW_CSI2_14:	descr = "14-bit Raw, CSI-2 Packed"; break;
> >  	case V4L2_PIX_FMT_RAW_CRU20:	descr = "14-bit Raw CRU Packed"; break;
> >  	case V4L2_PIX_FMT_SN9C20X_I420:	descr = "GSPCA SN9C20X I420"; break;
> >  	case V4L2_PIX_FMT_SPCA501:	descr = "GSPCA SPCA501"; break;
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 3dd9fa45dde1..c4c4f3eb67e1 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -745,6 +745,12 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_SGRBG16 v4l2_fourcc('G', 'R', '1', '6') /* 16  GRGR.. BGBG.. */
> >  #define V4L2_PIX_FMT_SRGGB16 v4l2_fourcc('R', 'G', '1', '6') /* 16  RGRG.. GBGB.. */
> >
> > +/* Generic CSI-2 packed raw pixel formats */
> > +#define V4L2_PIX_FMT_RAW_8 v4l2_fourcc('R', 'A', 'W', '8')
> > +#define V4L2_PIX_FMT_RAW_CSI2_10 v4l2_fourcc('R', 'A', 'C', 'A')
> > +#define V4L2_PIX_FMT_RAW_CSI2_12 v4l2_fourcc('R', 'A', 'C', 'C')
> > +#define V4L2_PIX_FMT_RAW_CSI2_14 v4l2_fourcc('R', 'A', 'C', 'E')
> > +
> With

Or without

> "media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp"
>
> and the above minors clarified
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>
> >  /* HSV formats */
> >  #define V4L2_PIX_FMT_HSV24 v4l2_fourcc('H', 'S', 'V', '3')
> >  #define V4L2_PIX_FMT_HSV32 v4l2_fourcc('H', 'S', 'V', '4')
> > --
> > 2.47.2
> >
> >

[PATCH v11 16/66] media: Documentation: Reference color pattern control in format docs

[Sakari Ailus]

Refer to the color pattern control in the generic raw pixelformat
documentation.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
index 9fda7a36cd3a..034d4f3e0499 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
@@ -13,7 +13,10 @@ Description
 
 These generic raw image data formats define the memory layout of the data
 without defining the order of the pixels in the format or even the CFA (Colour
-Filter Array) itself. See also :ref:`source routes <subdev-routing>`.
+Filter Array) itself. These formats may only be used with a Media Controller
+pipeline where the more specific format is reported by the
+:ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control on
+the source sub-device. See also :ref:`source routes <subdev-routing>`.
 
 .. _v4l2-pix-fmt-raw-8:
 
-- 
2.47.2

Re: [PATCH v11 16/66] media: Documentation: Reference color pattern control in format docs

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:17PM +0300, Sakari Ailus wrote:
> Refer to the color pattern control in the generic raw pixelformat
> documentation.

Either move this after patch #19 that introduces the control, or
squash it with #15 that introduce the generic RAW pixel format

Either ways
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> index 9fda7a36cd3a..034d4f3e0499 100644
> --- a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> @@ -13,7 +13,10 @@ Description
>
>  These generic raw image data formats define the memory layout of the data
>  without defining the order of the pixels in the format or even the CFA (Colour
> -Filter Array) itself. See also :ref:`source routes <subdev-routing>`.
> +Filter Array) itself. These formats may only be used with a Media Controller
> +pipeline where the more specific format is reported by the
> +:ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control on
> +the source sub-device. See also :ref:`source routes <subdev-routing>`.
>
>  .. _v4l2-pix-fmt-raw-8:
>
> --
> 2.47.2
>
>

Re: [PATCH v11 16/66] media: Documentation: Reference color pattern control in format docs

[Laurent Pinchart]

On Mon, Sep 01, 2025 at 03:51:21PM +0200, Jacopo Mondi wrote:
> On Mon, Aug 25, 2025 at 12:50:17PM +0300, Sakari Ailus wrote:
> > Refer to the color pattern control in the generic raw pixelformat
> > documentation.
> 
> Either move this after patch #19 that introduces the control, or
> squash it with #15 that introduce the generic RAW pixel format

Agreed, this should go after the introduction of the control.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> Either ways
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> 
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst | 5 ++++-
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > index 9fda7a36cd3a..034d4f3e0499 100644
> > --- a/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-raw-generic.rst
> > @@ -13,7 +13,10 @@ Description
> >
> >  These generic raw image data formats define the memory layout of the data
> >  without defining the order of the pixels in the format or even the CFA (Colour
> > -Filter Array) itself. See also :ref:`source routes <subdev-routing>`.
> > +Filter Array) itself. These formats may only be used with a Media Controller
> > +pipeline where the more specific format is reported by the
> > +:ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control on
> > +the source sub-device. See also :ref:`source routes <subdev-routing>`.
> >
> >  .. _v4l2-pix-fmt-raw-8:
> >

-- 
Regards,

Laurent Pinchart

[PATCH v11 17/66] media: uapi: Add new media bus codes for generic raw formats

[Sakari Ailus]

Add new media bus codes for generic raw formats that are not specific to
the colour filter array but that simply specify the bit depth. The layout
(packing) of the data is interface specific.

The rest of the properties of the format are specified with controls in
the image source.

The mbus codes added by this patch have bit depth of 8, 10, 12 and 14.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../media/v4l/subdev-formats.rst              | 27 +++++++++++++++++++
 include/uapi/linux/media-bus-format.h         |  6 +++++
 2 files changed, 33 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
index 9ef1bc22ad9c..1a51b5c817f1 100644
--- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
@@ -3434,6 +3434,33 @@ organization is given as an example for the first pixel only.
 
     \endgroup
 
+Generic raw formats on serial interfaces
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Those formats transfer (largely) unprocessed raw pixel data typically from raw
+camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
+interfaces. The packing of the data on the bus is determined by the hardware,
+however the bit depth is still specific to the format.
+
+.. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: Generic raw formats on serial buses
+    :header-rows:  1
+    :stub-columns: 0
+    :widths:       1 1
+
+    * - Format name
+      - Bit depth
+    * - MEDIA_BUS_FMT_RAW_8
+      - 8
+    * - MEDIA_BUS_FMT_RAW_10
+      - 10
+    * - MEDIA_BUS_FMT_RAW_12
+      - 12
+    * - MEDIA_BUS_FMT_RAW_14
+      - 14
 
 Packed YUV Formats
 ^^^^^^^^^^^^^^^^^^
diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h
index ff62056feed5..b71120bc1788 100644
--- a/include/uapi/linux/media-bus-format.h
+++ b/include/uapi/linux/media-bus-format.h
@@ -185,4 +185,10 @@
 #define MEDIA_BUS_FMT_META_20			0x8006
 #define MEDIA_BUS_FMT_META_24			0x8007
 
+/* Generic (CFA independent) pixel data formats. Next is 0xa009. */
+#define MEDIA_BUS_FMT_RAW_8			0xa001
+#define MEDIA_BUS_FMT_RAW_10			0xa002
+#define MEDIA_BUS_FMT_RAW_12			0xa003
+#define MEDIA_BUS_FMT_RAW_14			0xa004
+
 #endif /* __LINUX_MEDIA_BUS_FORMAT_H */
-- 
2.47.2

Re: [PATCH v11 17/66] media: uapi: Add new media bus codes for generic raw formats

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:18PM +0300, Sakari Ailus wrote:
> Add new media bus codes for generic raw formats that are not specific to
> the colour filter array but that simply specify the bit depth. The layout
> (packing) of the data is interface specific.
>
> The rest of the properties of the format are specified with controls in
> the image source.
>
> The mbus codes added by this patch have bit depth of 8, 10, 12 and 14.

Same as per the pixel formats, your branch contains
"media: uapi: Add more media bus codes for generic raw formats"

Should you squash it in ?

>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../media/v4l/subdev-formats.rst              | 27 +++++++++++++++++++
>  include/uapi/linux/media-bus-format.h         |  6 +++++
>  2 files changed, 33 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> index 9ef1bc22ad9c..1a51b5c817f1 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> @@ -3434,6 +3434,33 @@ organization is given as an example for the first pixel only.
>
>      \endgroup
>
> +Generic raw formats on serial interfaces
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +Those formats transfer (largely) unprocessed raw pixel data typically from raw

I would remove "(largely) unprocessed", mostly because I'm not sure
what you mean here.

> +camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
> +interfaces. The packing of the data on the bus is determined by the hardware,
> +however the bit depth is still specific to the format.
> +
> +.. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: Generic raw formats on serial buses
> +    :header-rows:  1
> +    :stub-columns: 0
> +    :widths:       1 1
> +
> +    * - Format name
> +      - Bit depth
> +    * - MEDIA_BUS_FMT_RAW_8
> +      - 8
> +    * - MEDIA_BUS_FMT_RAW_10
> +      - 10
> +    * - MEDIA_BUS_FMT_RAW_12
> +      - 12
> +    * - MEDIA_BUS_FMT_RAW_14
> +      - 14

Should we now say in the "Bayer Formats" section that the existing
media bus codes that convey the color components ordering are now
superseded by these ones ?

Nits apart
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
   j

>
>  Packed YUV Formats
>  ^^^^^^^^^^^^^^^^^^
> diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h
> index ff62056feed5..b71120bc1788 100644
> --- a/include/uapi/linux/media-bus-format.h
> +++ b/include/uapi/linux/media-bus-format.h
> @@ -185,4 +185,10 @@
>  #define MEDIA_BUS_FMT_META_20			0x8006
>  #define MEDIA_BUS_FMT_META_24			0x8007
>
> +/* Generic (CFA independent) pixel data formats. Next is 0xa009. */
> +#define MEDIA_BUS_FMT_RAW_8			0xa001
> +#define MEDIA_BUS_FMT_RAW_10			0xa002
> +#define MEDIA_BUS_FMT_RAW_12			0xa003
> +#define MEDIA_BUS_FMT_RAW_14			0xa004
> +
>  #endif /* __LINUX_MEDIA_BUS_FORMAT_H */
> --
> 2.47.2
>
>

Re: [PATCH v11 17/66] media: uapi: Add new media bus codes for generic raw formats

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 04:12:33PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:18PM +0300, Sakari Ailus wrote:
> > Add new media bus codes for generic raw formats that are not specific to
> > the colour filter array but that simply specify the bit depth. The layout
> > (packing) of the data is interface specific.
> >
> > The rest of the properties of the format are specified with controls in
> > the image source.
> >
> > The mbus codes added by this patch have bit depth of 8, 10, 12 and 14.
> 
> Same as per the pixel formats, your branch contains
> "media: uapi: Add more media bus codes for generic raw formats"
> 
> Should you squash it in ?

Related to my comment on bit depths and what we know of CSI-2 receiver
implementations, no. But let me know if there's a need to further discuss
this.

> 
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../media/v4l/subdev-formats.rst              | 27 +++++++++++++++++++
> >  include/uapi/linux/media-bus-format.h         |  6 +++++
> >  2 files changed, 33 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > index 9ef1bc22ad9c..1a51b5c817f1 100644
> > --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > @@ -3434,6 +3434,33 @@ organization is given as an example for the first pixel only.
> >
> >      \endgroup
> >
> > +Generic raw formats on serial interfaces
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +Those formats transfer (largely) unprocessed raw pixel data typically from raw
> 
> I would remove "(largely) unprocessed", mostly because I'm not sure
> what you mean here.

I'm fine with removing that.

> 
> > +camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
> > +interfaces. The packing of the data on the bus is determined by the hardware,
> > +however the bit depth is still specific to the format.
> > +
> > +.. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table:: Generic raw formats on serial buses
> > +    :header-rows:  1
> > +    :stub-columns: 0
> > +    :widths:       1 1
> > +
> > +    * - Format name
> > +      - Bit depth
> > +    * - MEDIA_BUS_FMT_RAW_8
> > +      - 8
> > +    * - MEDIA_BUS_FMT_RAW_10
> > +      - 10
> > +    * - MEDIA_BUS_FMT_RAW_12
> > +      - 12
> > +    * - MEDIA_BUS_FMT_RAW_14
> > +      - 14
> 
> Should we now say in the "Bayer Formats" section that the existing
> media bus codes that convey the color components ordering are now
> superseded by these ones ?

I can add a note that new drivers need to use these codes but may use older
mbus codes, too. I'd add that to the camera sensor driver documentation
though.

> 
> Nits apart
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thank you!

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v11 17/66] media: uapi: Add new media bus codes for generic raw formats

[Laurent Pinchart]

On Wed, Sep 03, 2025 at 12:27:41PM +0300, Sakari Ailus wrote:
> On Mon, Sep 01, 2025 at 04:12:33PM +0200, Jacopo Mondi wrote:
> > On Mon, Aug 25, 2025 at 12:50:18PM +0300, Sakari Ailus wrote:
> > > Add new media bus codes for generic raw formats that are not specific to
> > > the colour filter array but that simply specify the bit depth. The layout
> > > (packing) of the data is interface specific.
> > >
> > > The rest of the properties of the format are specified with controls in
> > > the image source.
> > >
> > > The mbus codes added by this patch have bit depth of 8, 10, 12 and 14.
> > 
> > Same as per the pixel formats, your branch contains
> > "media: uapi: Add more media bus codes for generic raw formats"
> > 
> > Should you squash it in ?
> 
> Related to my comment on bit depths and what we know of CSI-2 receiver
> implementations, no. But let me know if there's a need to further discuss
> this.
> 
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  .../media/v4l/subdev-formats.rst              | 27 +++++++++++++++++++
> > >  include/uapi/linux/media-bus-format.h         |  6 +++++
> > >  2 files changed, 33 insertions(+)
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > index 9ef1bc22ad9c..1a51b5c817f1 100644
> > > --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > @@ -3434,6 +3434,33 @@ organization is given as an example for the first pixel only.
> > >
> > >      \endgroup
> > >
> > > +Generic raw formats on serial interfaces
> > > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > +
> > > +Those formats transfer (largely) unprocessed raw pixel data typically from raw
> > 
> > I would remove "(largely) unprocessed", mostly because I'm not sure
> > what you mean here.
> 
> I'm fine with removing that.
> 
> > > +camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
> > > +interfaces. The packing of the data on the bus is determined by the hardware,
> > > +however the bit depth is still specific to the format.
> > > +
> > > +.. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
> > > +
> > > +.. cssclass:: longtable
> > > +
> > > +.. flat-table:: Generic raw formats on serial buses
> > > +    :header-rows:  1
> > > +    :stub-columns: 0
> > > +    :widths:       1 1
> > > +
> > > +    * - Format name
> > > +      - Bit depth
> > > +    * - MEDIA_BUS_FMT_RAW_8
> > > +      - 8
> > > +    * - MEDIA_BUS_FMT_RAW_10
> > > +      - 10
> > > +    * - MEDIA_BUS_FMT_RAW_12
> > > +      - 12
> > > +    * - MEDIA_BUS_FMT_RAW_14
> > > +      - 14
> > 
> > Should we now say in the "Bayer Formats" section that the existing
> > media bus codes that convey the color components ordering are now
> > superseded by these ones ?
> 
> I can add a note that new drivers need to use these codes but may use older
> mbus codes, too. I'd add that to the camera sensor driver documentation
> though.

Agreed. Let's make the new formats mandatory for new drivers, an the old
formats optional. Implementing the old formats is required when porting
existing drivers to new formats, but I would discourage it for new
drivers in order to get userspace to move forward.

> > Nits apart
> > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

-- 
Regards,

Laurent Pinchart

Re: [PATCH v11 17/66] media: uapi: Add new media bus codes for generic raw formats

[Sakari Ailus]

On Wed, Sep 03, 2025 at 04:32:15PM +0200, Laurent Pinchart wrote:
> On Wed, Sep 03, 2025 at 12:27:41PM +0300, Sakari Ailus wrote:
> > On Mon, Sep 01, 2025 at 04:12:33PM +0200, Jacopo Mondi wrote:
> > > On Mon, Aug 25, 2025 at 12:50:18PM +0300, Sakari Ailus wrote:
> > > > Add new media bus codes for generic raw formats that are not specific to
> > > > the colour filter array but that simply specify the bit depth. The layout
> > > > (packing) of the data is interface specific.
> > > >
> > > > The rest of the properties of the format are specified with controls in
> > > > the image source.
> > > >
> > > > The mbus codes added by this patch have bit depth of 8, 10, 12 and 14.
> > > 
> > > Same as per the pixel formats, your branch contains
> > > "media: uapi: Add more media bus codes for generic raw formats"
> > > 
> > > Should you squash it in ?
> > 
> > Related to my comment on bit depths and what we know of CSI-2 receiver
> > implementations, no. But let me know if there's a need to further discuss
> > this.
> > 
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > ---
> > > >  .../media/v4l/subdev-formats.rst              | 27 +++++++++++++++++++
> > > >  include/uapi/linux/media-bus-format.h         |  6 +++++
> > > >  2 files changed, 33 insertions(+)
> > > >
> > > > diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > > index 9ef1bc22ad9c..1a51b5c817f1 100644
> > > > --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > > @@ -3434,6 +3434,33 @@ organization is given as an example for the first pixel only.
> > > >
> > > >      \endgroup
> > > >
> > > > +Generic raw formats on serial interfaces
> > > > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > +
> > > > +Those formats transfer (largely) unprocessed raw pixel data typically from raw
> > > 
> > > I would remove "(largely) unprocessed", mostly because I'm not sure
> > > what you mean here.
> > 
> > I'm fine with removing that.
> > 
> > > > +camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
> > > > +interfaces. The packing of the data on the bus is determined by the hardware,
> > > > +however the bit depth is still specific to the format.
> > > > +
> > > > +.. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
> > > > +
> > > > +.. cssclass:: longtable
> > > > +
> > > > +.. flat-table:: Generic raw formats on serial buses
> > > > +    :header-rows:  1
> > > > +    :stub-columns: 0
> > > > +    :widths:       1 1
> > > > +
> > > > +    * - Format name
> > > > +      - Bit depth
> > > > +    * - MEDIA_BUS_FMT_RAW_8
> > > > +      - 8
> > > > +    * - MEDIA_BUS_FMT_RAW_10
> > > > +      - 10
> > > > +    * - MEDIA_BUS_FMT_RAW_12
> > > > +      - 12
> > > > +    * - MEDIA_BUS_FMT_RAW_14
> > > > +      - 14
> > > 
> > > Should we now say in the "Bayer Formats" section that the existing
> > > media bus codes that convey the color components ordering are now
> > > superseded by these ones ?
> > 
> > I can add a note that new drivers need to use these codes but may use older
> > mbus codes, too. I'd add that to the camera sensor driver documentation
> > though.
> 
> Agreed. Let's make the new formats mandatory for new drivers, an the old
> formats optional. Implementing the old formats is required when porting
> existing drivers to new formats, but I would discourage it for new
> drivers in order to get userspace to move forward.

I'll also add that receiver drivers should be amended with support for the
new formats instead of adding support for the old ones in the sensor
drivers. The latter is likely much easier in any case.

-- 
Sakari Ailus

Re: [PATCH v11 17/66] media: uapi: Add new media bus codes for generic raw formats

[Laurent Pinchart]

On Wed, Sep 03, 2025 at 05:35:18PM +0300, Sakari Ailus wrote:
> On Wed, Sep 03, 2025 at 04:32:15PM +0200, Laurent Pinchart wrote:
> > On Wed, Sep 03, 2025 at 12:27:41PM +0300, Sakari Ailus wrote:
> > > On Mon, Sep 01, 2025 at 04:12:33PM +0200, Jacopo Mondi wrote:
> > > > On Mon, Aug 25, 2025 at 12:50:18PM +0300, Sakari Ailus wrote:
> > > > > Add new media bus codes for generic raw formats that are not specific to
> > > > > the colour filter array but that simply specify the bit depth. The layout
> > > > > (packing) of the data is interface specific.
> > > > >
> > > > > The rest of the properties of the format are specified with controls in
> > > > > the image source.
> > > > >
> > > > > The mbus codes added by this patch have bit depth of 8, 10, 12 and 14.
> > > > 
> > > > Same as per the pixel formats, your branch contains
> > > > "media: uapi: Add more media bus codes for generic raw formats"
> > > > 
> > > > Should you squash it in ?
> > > 
> > > Related to my comment on bit depths and what we know of CSI-2 receiver
> > > implementations, no. But let me know if there's a need to further discuss
> > > this.
> > > 
> > > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > ---
> > > > >  .../media/v4l/subdev-formats.rst              | 27 +++++++++++++++++++
> > > > >  include/uapi/linux/media-bus-format.h         |  6 +++++
> > > > >  2 files changed, 33 insertions(+)
> > > > >
> > > > > diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > > > index 9ef1bc22ad9c..1a51b5c817f1 100644
> > > > > --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > > > +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > > > @@ -3434,6 +3434,33 @@ organization is given as an example for the first pixel only.
> > > > >
> > > > >      \endgroup
> > > > >
> > > > > +Generic raw formats on serial interfaces
> > > > > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > > +
> > > > > +Those formats transfer (largely) unprocessed raw pixel data typically from raw
> > > > 
> > > > I would remove "(largely) unprocessed", mostly because I'm not sure
> > > > what you mean here.
> > > 
> > > I'm fine with removing that.
> > > 
> > > > > +camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
> > > > > +interfaces. The packing of the data on the bus is determined by the hardware,
> > > > > +however the bit depth is still specific to the format.
> > > > > +
> > > > > +.. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
> > > > > +
> > > > > +.. cssclass:: longtable
> > > > > +
> > > > > +.. flat-table:: Generic raw formats on serial buses
> > > > > +    :header-rows:  1
> > > > > +    :stub-columns: 0
> > > > > +    :widths:       1 1
> > > > > +
> > > > > +    * - Format name
> > > > > +      - Bit depth
> > > > > +    * - MEDIA_BUS_FMT_RAW_8
> > > > > +      - 8
> > > > > +    * - MEDIA_BUS_FMT_RAW_10
> > > > > +      - 10
> > > > > +    * - MEDIA_BUS_FMT_RAW_12
> > > > > +      - 12
> > > > > +    * - MEDIA_BUS_FMT_RAW_14
> > > > > +      - 14
> > > > 
> > > > Should we now say in the "Bayer Formats" section that the existing
> > > > media bus codes that convey the color components ordering are now
> > > > superseded by these ones ?
> > > 
> > > I can add a note that new drivers need to use these codes but may use older
> > > mbus codes, too. I'd add that to the camera sensor driver documentation
> > > though.
> > 
> > Agreed. Let's make the new formats mandatory for new drivers, an the old
> > formats optional. Implementing the old formats is required when porting
> > existing drivers to new formats, but I would discourage it for new
> > drivers in order to get userspace to move forward.
> 
> I'll also add that receiver drivers should be amended with support for the
> new formats instead of adding support for the old ones in the sensor
> drivers. The latter is likely much easier in any case.

Good idea.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v11 17/66] media: uapi: Add new media bus codes for generic raw formats

[Laurent Pinchart]

On Mon, Aug 25, 2025 at 12:50:18PM +0300, Sakari Ailus wrote:
> Add new media bus codes for generic raw formats that are not specific to
> the colour filter array but that simply specify the bit depth. The layout
> (packing) of the data is interface specific.
> 
> The rest of the properties of the format are specified with controls in
> the image source.
> 
> The mbus codes added by this patch have bit depth of 8, 10, 12 and 14.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../media/v4l/subdev-formats.rst              | 27 +++++++++++++++++++
>  include/uapi/linux/media-bus-format.h         |  6 +++++
>  2 files changed, 33 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> index 9ef1bc22ad9c..1a51b5c817f1 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> @@ -3434,6 +3434,33 @@ organization is given as an example for the first pixel only.
>  
>      \endgroup
>  
> +Generic raw formats on serial interfaces
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +Those formats transfer (largely) unprocessed raw pixel data typically from raw
> +camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
> +interfaces. The packing of the data on the bus is determined by the hardware,
> +however the bit depth is still specific to the format.
> +
> +.. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: Generic raw formats on serial buses
> +    :header-rows:  1
> +    :stub-columns: 0
> +    :widths:       1 1
> +
> +    * - Format name
> +      - Bit depth
> +    * - MEDIA_BUS_FMT_RAW_8
> +      - 8
> +    * - MEDIA_BUS_FMT_RAW_10
> +      - 10
> +    * - MEDIA_BUS_FMT_RAW_12
> +      - 12
> +    * - MEDIA_BUS_FMT_RAW_14
> +      - 14
>  
>  Packed YUV Formats
>  ^^^^^^^^^^^^^^^^^^
> diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h
> index ff62056feed5..b71120bc1788 100644
> --- a/include/uapi/linux/media-bus-format.h
> +++ b/include/uapi/linux/media-bus-format.h
> @@ -185,4 +185,10 @@
>  #define MEDIA_BUS_FMT_META_20			0x8006
>  #define MEDIA_BUS_FMT_META_24			0x8007
>  
> +/* Generic (CFA independent) pixel data formats. Next is 0xa009. */

The next format is 0xa005. Any reason not to use 0x9000 as a base though
? Or maybe adding these to the 0x3000 (bayer formats) group ? We're
running out of bases.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +#define MEDIA_BUS_FMT_RAW_8			0xa001
> +#define MEDIA_BUS_FMT_RAW_10			0xa002
> +#define MEDIA_BUS_FMT_RAW_12			0xa003
> +#define MEDIA_BUS_FMT_RAW_14			0xa004
> +
>  #endif /* __LINUX_MEDIA_BUS_FORMAT_H */

-- 
Regards,

Laurent Pinchart

[PATCH v11 18/66] media: uapi: Add V4L2_CID_CONFIG_MODEL control

[Sakari Ailus]

Add the V4L2_CID_CONFIG_MODEL control for the configuration model.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
---
 .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 4 ++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c                    | 5 +++++
 include/uapi/linux/v4l2-controls.h                           | 3 +++
 3 files changed, 12 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
index 27803dca8d3e..2ae17ed99729 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
@@ -55,3 +55,7 @@ Image Process Control IDs
     control value divided by e.g. 0x100, meaning that to get no
     digital gain the control value needs to be 0x100. The no-gain
     configuration is also typically the default.
+
+``V4L2_CID_CONFIG_MODEL (bitmask)``
+    Which configuration models the sub-device supports. Please see
+    :ref:`media_subdev_config_model`. This is a read-only control.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 1ea52011247a..dd5f06546773 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN:		return "Test Pattern";
 	case V4L2_CID_DEINTERLACING_MODE:	return "Deinterlacing Mode";
 	case V4L2_CID_DIGITAL_GAIN:		return "Digital Gain";
+	case V4L2_CID_CONFIG_MODEL:		return "Sub-device Configuration Model";
 
 	/* DV controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_DV_RX_POWER_PRESENT:
 		*type = V4L2_CTRL_TYPE_BITMASK;
 		break;
+	case V4L2_CID_CONFIG_MODEL:
+		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
+		*type = V4L2_CTRL_TYPE_BITMASK;
+		break;
 	case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
 	case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
 		*type = V4L2_CTRL_TYPE_INTEGER;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index f836512e9deb..3c0f6afe7500 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1238,6 +1238,9 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_TEST_PATTERN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)
 #define V4L2_CID_DEINTERLACING_MODE		(V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)
 #define V4L2_CID_DIGITAL_GAIN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 5)
+#define V4L2_CID_CONFIG_MODEL			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 6)
+
+#define V4L2_CONFIG_MODEL_COMMON_RAW_SENSOR	(1U << 0)
 
 /*  DV-class control IDs defined by V4L2 */
 #define V4L2_CID_DV_CLASS_BASE			(V4L2_CTRL_CLASS_DV | 0x900)
-- 
2.47.2

[PATCH v11 19/66] media: uapi: Add V4L2_CID_COLOR_PATTERN for describing color patterns

[Sakari Ailus]

Add V4L2_CID_COLOR_PATTERN to tell the camera sensor's native color
pattern.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 .../media/v4l/ext-ctrls-image-source.rst      | 39 +++++++++++++++++++
 .../media/v4l/subdev-formats.rst              |  2 +
 drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  1 +
 include/uapi/linux/v4l2-controls.h            |  6 +++
 4 files changed, 48 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index 71f23f131f97..b19aaaffbce0 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -92,3 +92,42 @@ Image Source Control IDs
     representing a gain of exactly 1.0. For example, if this default value
     is reported as being (say) 128, then a value of 192 would represent
     a gain of exactly 1.5.
+
+``V4L2_CID_COLOR_PATTERN (integer)``
+    This control determines the color components and native pixel order in the
+    sensor's CFA (Color Filter Array) when used in conjunction with
+    :ref:`generic raw mbus codes MEDIA_BUS_FMT_RAW_x (where 'x' is the bit
+    depth) <v4l2-mbus-pixelcode-generic-raw>` pixelcodes. The native pixel
+    order is constant for a given device, it is not affected by cropping or
+    flipping.
+
+    This control may only be used on V4L2 sub-devices.
+
+    This is a read-only control.
+
+    Available patterns:
+
+.. flat-table:: V4L2 color patterns
+    :header-rows:  1
+    :stub-columns: 0
+    :widths:       1 2 1
+
+    * - Macro name
+      - Synopsis
+      - Value
+    * - V4L2_COLOR_PATTERN_GRBG
+      - Raw Bayer, with alternating lines beginning with green, red pixels and
+        blue, green pixels.
+      - 0
+    * - V4L2_COLOR_PATTERN_RGGB
+      - Raw Bayer, with alternating lines beginning with red, green pixels and
+        green, blue pixels.
+      - 1
+    * - V4L2_COLOR_PATTERN_BGGR
+      - Raw Bayer, with alternating lines beginning with blue, green pixels and
+        green, red pixels.
+      - 2
+    * - V4L2_COLOR_PATTERN_GBRG
+      - Raw Bayer, with alternating lines beginning with green, blue pixels and
+        red, green pixels.
+      - 3
diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
index 1a51b5c817f1..e77b4ad19737 100644
--- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
@@ -3434,6 +3434,8 @@ organization is given as an example for the first pixel only.
 
     \endgroup
 
+.. _v4l2-mbus-pixelcode-generic-raw:
+
 Generic raw formats on serial interfaces
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index dd5f06546773..8119012c2638 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1155,6 +1155,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN_BLUE:	return "Blue Pixel Value";
 	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
 	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
+	case V4L2_CID_COLOR_PATTERN:		return "Color Pattern";
 
 	/* Image processing controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 3c0f6afe7500..e552f6655fd0 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1227,6 +1227,12 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_UNIT_CELL_SIZE			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
 #define V4L2_CID_NOTIFY_GAINS			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
 
+#define V4L2_CID_COLOR_PATTERN			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10)
+#define V4L2_COLOR_PATTERN_GRBG			0U
+#define V4L2_COLOR_PATTERN_RGGB			1U
+#define V4L2_COLOR_PATTERN_BGGR			2U
+#define V4L2_COLOR_PATTERN_GBRG			3U
+
 
 /* Image processing controls */
 
-- 
2.47.2

Re: [PATCH v11 19/66] media: uapi: Add V4L2_CID_COLOR_PATTERN for describing color patterns

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Aug 25, 2025 at 12:50:20PM +0300, Sakari Ailus wrote:
> Add V4L2_CID_COLOR_PATTERN to tell the camera sensor's native color
> pattern.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  .../media/v4l/ext-ctrls-image-source.rst      | 39 +++++++++++++++++++
>  .../media/v4l/subdev-formats.rst              |  2 +
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  1 +
>  include/uapi/linux/v4l2-controls.h            |  6 +++
>  4 files changed, 48 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index 71f23f131f97..b19aaaffbce0 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -92,3 +92,42 @@ Image Source Control IDs
>      representing a gain of exactly 1.0. For example, if this default value
>      is reported as being (say) 128, then a value of 192 would represent
>      a gain of exactly 1.5.
> +
> +``V4L2_CID_COLOR_PATTERN (integer)``
> +    This control determines the color components and native pixel order in the
> +    sensor's CFA (Color Filter Array) when used in conjunction with
> +    :ref:`generic raw mbus codes MEDIA_BUS_FMT_RAW_x (where 'x' is the bit
> +    depth) <v4l2-mbus-pixelcode-generic-raw>` pixelcodes. The native pixel
> +    order is constant for a given device, it is not affected by cropping or
> +    flipping.
> +
> +    This control may only be used on V4L2 sub-devices.
> +
> +    This is a read-only control.
> +
> +    Available patterns:
> +
> +.. flat-table:: V4L2 color patterns
> +    :header-rows:  1
> +    :stub-columns: 0
> +    :widths:       1 2 1
> +
> +    * - Macro name
> +      - Synopsis
> +      - Value
> +    * - V4L2_COLOR_PATTERN_GRBG
> +      - Raw Bayer, with alternating lines beginning with green, red pixels and
> +        blue, green pixels.

I know what you mean, but it's a bit vague. Let's try to be more
precise:

      - Bayer pattern with 2x2 macroblocks. Each macroblock stores pixels in
        green, red, blue, green order.

Or even more precise

      - Bayer pattern with 2x2 macroblocks. Each macroblock has a green filter
        in the top-left and bottom-right pixels, a red filter in the top-right
	pixel, and a blue filter in the bottom-left pixel.

> +      - 0
> +    * - V4L2_COLOR_PATTERN_RGGB
> +      - Raw Bayer, with alternating lines beginning with red, green pixels and
> +        green, blue pixels.
> +      - 1
> +    * - V4L2_COLOR_PATTERN_BGGR
> +      - Raw Bayer, with alternating lines beginning with blue, green pixels and
> +        green, red pixels.
> +      - 2
> +    * - V4L2_COLOR_PATTERN_GBRG
> +      - Raw Bayer, with alternating lines beginning with green, blue pixels and
> +        red, green pixels.
> +      - 3
> diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> index 1a51b5c817f1..e77b4ad19737 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> @@ -3434,6 +3434,8 @@ organization is given as an example for the first pixel only.
>  
>      \endgroup
>  
> +.. _v4l2-mbus-pixelcode-generic-raw:
> +

I'd move this to the patch that adds the raw formats. No big deal
though.

>  Generic raw formats on serial interfaces
>  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>  
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index dd5f06546773..8119012c2638 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1155,6 +1155,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_TEST_PATTERN_BLUE:	return "Blue Pixel Value";
>  	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
>  	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
> +	case V4L2_CID_COLOR_PATTERN:		return "Color Pattern";
>  

Shouldn't you also patch v4l2_ctrl_fill() ?

>  	/* Image processing controls */
>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 3c0f6afe7500..e552f6655fd0 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1227,6 +1227,12 @@ enum v4l2_jpeg_chroma_subsampling {
>  #define V4L2_CID_UNIT_CELL_SIZE			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
>  #define V4L2_CID_NOTIFY_GAINS			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
>  
> +#define V4L2_CID_COLOR_PATTERN			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10)
> +#define V4L2_COLOR_PATTERN_GRBG			0U
> +#define V4L2_COLOR_PATTERN_RGGB			1U
> +#define V4L2_COLOR_PATTERN_BGGR			2U
> +#define V4L2_COLOR_PATTERN_GBRG			3U

How about V4L2_CID_CFA_PATTERN and V4L2_CFA_PATTERN_* ?

> +
>  
>  /* Image processing controls */
>  

-- 
Regards,

Laurent Pinchart

[PATCH v11 20/66] media: Documentation: Reference COLOR_PATTERN control in raw format docs

[Sakari Ailus]

Add a reference to the V4L2_CID_COLOR_PATTERN control in the generic raw
pixelformat documentation.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../userspace-api/media/v4l/ext-ctrls-image-source.rst         | 2 ++
 Documentation/userspace-api/media/v4l/subdev-formats.rst       | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index b19aaaffbce0..0fcd60d8c621 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -93,6 +93,8 @@ Image Source Control IDs
     is reported as being (say) 128, then a value of 192 would represent
     a gain of exactly 1.5.
 
+.. _image-source-control-color-pattern:
+
 ``V4L2_CID_COLOR_PATTERN (integer)``
     This control determines the color components and native pixel order in the
     sensor's CFA (Color Filter Array) when used in conjunction with
diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
index e77b4ad19737..77571adeb21e 100644
--- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
@@ -3444,6 +3444,9 @@ camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
 interfaces. The packing of the data on the bus is determined by the hardware,
 however the bit depth is still specific to the format.
 
+The colour components and the native pixel order are determined by the
+:ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control.
+
 .. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
 
 .. cssclass:: longtable
-- 
2.47.2

Re: [PATCH v11 20/66] media: Documentation: Reference COLOR_PATTERN control in raw format docs

[Jacopo Mondi]

On Mon, Aug 25, 2025 at 12:50:21PM +0300, Sakari Ailus wrote:
> Add a reference to the V4L2_CID_COLOR_PATTERN control in the generic raw
> pixelformat documentation.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../userspace-api/media/v4l/ext-ctrls-image-source.rst         | 2 ++
>  Documentation/userspace-api/media/v4l/subdev-formats.rst       | 3 +++
>  2 files changed, 5 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index b19aaaffbce0..0fcd60d8c621 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -93,6 +93,8 @@ Image Source Control IDs
>      is reported as being (say) 128, then a value of 192 would represent
>      a gain of exactly 1.5.
>
> +.. _image-source-control-color-pattern:
> +
>  ``V4L2_CID_COLOR_PATTERN (integer)``
>      This control determines the color components and native pixel order in the
>      sensor's CFA (Color Filter Array) when used in conjunction with
> diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> index e77b4ad19737..77571adeb21e 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> @@ -3444,6 +3444,9 @@ camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
>  interfaces. The packing of the data on the bus is determined by the hardware,
>  however the bit depth is still specific to the format.
>
> +The colour components and the native pixel order are determined by the

Isn't
  +The native colour components ordering are determined by ...

As I don't see how "colour components and the native pixel order"
information can be conveyed separately.

Up to you

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
   j

> +:ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control.
> +
>  .. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
>
>  .. cssclass:: longtable
> --
> 2.47.2
>
>

Re: [PATCH v11 20/66] media: Documentation: Reference COLOR_PATTERN control in raw format docs

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 04:21:53PM +0200, Jacopo Mondi wrote:
> On Mon, Aug 25, 2025 at 12:50:21PM +0300, Sakari Ailus wrote:
> > Add a reference to the V4L2_CID_COLOR_PATTERN control in the generic raw
> > pixelformat documentation.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst         | 2 ++
> >  Documentation/userspace-api/media/v4l/subdev-formats.rst       | 3 +++
> >  2 files changed, 5 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > index b19aaaffbce0..0fcd60d8c621 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > @@ -93,6 +93,8 @@ Image Source Control IDs
> >      is reported as being (say) 128, then a value of 192 would represent
> >      a gain of exactly 1.5.
> >
> > +.. _image-source-control-color-pattern:
> > +
> >  ``V4L2_CID_COLOR_PATTERN (integer)``
> >      This control determines the color components and native pixel order in the
> >      sensor's CFA (Color Filter Array) when used in conjunction with
> > diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > index e77b4ad19737..77571adeb21e 100644
> > --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > @@ -3444,6 +3444,9 @@ camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
> >  interfaces. The packing of the data on the bus is determined by the hardware,
> >  however the bit depth is still specific to the format.
> >
> > +The colour components and the native pixel order are determined by the
> 
> Isn't
>   +The native colour components ordering are determined by ...
> 
> As I don't see how "colour components and the native pixel order"
> information can be conveyed separately.

How about:

The native Colour Filter Array (CFA) pattern is determined by the

> 
> Up to you
> 
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks!

> 
> Thanks
>    j
> 
> > +:ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control.
> > +
> >  .. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
> >
> >  .. cssclass:: longtable

-- 
Kind regrads,

Sakari Ailus

Re: [PATCH v11 20/66] media: Documentation: Reference COLOR_PATTERN control in raw format docs

[Jacopo Mondi]

Hi Sakari

On Wed, Sep 03, 2025 at 01:03:01PM +0300, Sakari Ailus wrote:
> Hi Jacopo,
>
> On Mon, Sep 01, 2025 at 04:21:53PM +0200, Jacopo Mondi wrote:
> > On Mon, Aug 25, 2025 at 12:50:21PM +0300, Sakari Ailus wrote:
> > > Add a reference to the V4L2_CID_COLOR_PATTERN control in the generic raw
> > > pixelformat documentation.
> > >
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst         | 2 ++
> > >  Documentation/userspace-api/media/v4l/subdev-formats.rst       | 3 +++
> > >  2 files changed, 5 insertions(+)
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > index b19aaaffbce0..0fcd60d8c621 100644
> > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > @@ -93,6 +93,8 @@ Image Source Control IDs
> > >      is reported as being (say) 128, then a value of 192 would represent
> > >      a gain of exactly 1.5.
> > >
> > > +.. _image-source-control-color-pattern:
> > > +
> > >  ``V4L2_CID_COLOR_PATTERN (integer)``
> > >      This control determines the color components and native pixel order in the
> > >      sensor's CFA (Color Filter Array) when used in conjunction with
> > > diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > index e77b4ad19737..77571adeb21e 100644
> > > --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > @@ -3444,6 +3444,9 @@ camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
> > >  interfaces. The packing of the data on the bus is determined by the hardware,
> > >  however the bit depth is still specific to the format.
> > >
> > > +The colour components and the native pixel order are determined by the
> >
> > Isn't
> >   +The native colour components ordering are determined by ...
> >
> > As I don't see how "colour components and the native pixel order"
> > information can be conveyed separately.
>
> How about:
>
> The native Colour Filter Array (CFA) pattern is determined by the

Works for me, thanks!

>
> >
> > Up to you
> >
> > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>
> Thanks!
>
> >
> > Thanks
> >    j
> >
> > > +:ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control.
> > > +
> > >  .. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
> > >
> > >  .. cssclass:: longtable
>
> --
> Kind regrads,
>
> Sakari Ailus

Re: [PATCH v11 20/66] media: Documentation: Reference COLOR_PATTERN control in raw format docs

[Laurent Pinchart]

On Wed, Sep 03, 2025 at 02:03:39PM +0200, Jacopo Mondi wrote:
> On Wed, Sep 03, 2025 at 01:03:01PM +0300, Sakari Ailus wrote:
> > On Mon, Sep 01, 2025 at 04:21:53PM +0200, Jacopo Mondi wrote:
> > > On Mon, Aug 25, 2025 at 12:50:21PM +0300, Sakari Ailus wrote:
> > > > Add a reference to the V4L2_CID_COLOR_PATTERN control in the generic raw
> > > > pixelformat documentation.
> > > >
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > ---
> > > >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst         | 2 ++
> > > >  Documentation/userspace-api/media/v4l/subdev-formats.rst       | 3 +++
> > > >  2 files changed, 5 insertions(+)
> > > >
> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > index b19aaaffbce0..0fcd60d8c621 100644
> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > @@ -93,6 +93,8 @@ Image Source Control IDs
> > > >      is reported as being (say) 128, then a value of 192 would represent
> > > >      a gain of exactly 1.5.
> > > >
> > > > +.. _image-source-control-color-pattern:
> > > > +

I'd move this to the previous patch.

> > > >  ``V4L2_CID_COLOR_PATTERN (integer)``
> > > >      This control determines the color components and native pixel order in the
> > > >      sensor's CFA (Color Filter Array) when used in conjunction with
> > > > diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > > index e77b4ad19737..77571adeb21e 100644
> > > > --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> > > > @@ -3444,6 +3444,9 @@ camera sensors using Bayer and other Colour Filter Arrays (CFAs) on serial
> > > >  interfaces. The packing of the data on the bus is determined by the hardware,
> > > >  however the bit depth is still specific to the format.
> > > >
> > > > +The colour components and the native pixel order are determined by the
> > >
> > > Isn't
> > >   +The native colour components ordering are determined by ...
> > >
> > > As I don't see how "colour components and the native pixel order"
> > > information can be conveyed separately.
> >
> > How about:
> >
> > The native Colour Filter Array (CFA) pattern is determined by the

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> Works for me, thanks!
> 
> > > Up to you
> > >
> > > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > >
> > > > +:ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control.
> > > > +
> > > >  .. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
> > > >
> > > >  .. cssclass:: longtable

-- 
Regards,

Laurent Pinchart

[PATCH v11 21/66] media: v4l: uapi: Add a control for color pattern flipping effect

[Sakari Ailus]

Add a bitmask control (V4L2_CID_COLOR_PATTERN_FLIP) to tell whether
flipping results in a change in the sensor's color pattern, separately
horizontally and vertically. The information is essential for raw formats
when using generic raw mbus codes.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  2 ++
 include/uapi/linux/v4l2-controls.h                     |  6 ++++++
 3 files changed, 18 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index 0fcd60d8c621..ef3784d1119a 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -133,3 +133,13 @@ Image Source Control IDs
       - Raw Bayer, with alternating lines beginning with green, blue pixels and
         red, green pixels.
       - 3
+
+``V4L2_CID_COLOR_PATTERN_FLIP (bitmask)``
+    Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
+    V4L2_CID_VFLIP) have an effect on the pixel order of the output color
+    pattern. Macros ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
+    ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` define bitmasks for both bits. If
+    either horizontal or vertical bit is set, the readout pattern order is that
+    of the reversed readout. ``V4L2_COLOR_PATTERN_FLIP_BOTH`` for setting both
+    ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
+    ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 8119012c2638..9ec65998a8f7 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1156,6 +1156,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
 	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
 	case V4L2_CID_COLOR_PATTERN:		return "Color Pattern";
+	case V4L2_CID_COLOR_PATTERN_FLIP:	return "Color Pattern Flip";
 
 	/* Image processing controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1484,6 +1485,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 		*type = V4L2_CTRL_TYPE_BITMASK;
 		break;
 	case V4L2_CID_CONFIG_MODEL:
+	case V4L2_CID_COLOR_PATTERN_FLIP:
 		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
 		*type = V4L2_CTRL_TYPE_BITMASK;
 		break;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index e552f6655fd0..eb9ffdd74d32 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1233,6 +1233,12 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_COLOR_PATTERN_BGGR			2U
 #define V4L2_COLOR_PATTERN_GBRG			3U
 
+#define V4L2_CID_COLOR_PATTERN_FLIP		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 11)
+#define V4L2_COLOR_PATTERN_FLIP_HORIZONTAL	(1U << 0)
+#define V4L2_COLOR_PATTERN_FLIP_VERTICAL	(1U << 1)
+#define V4L2_COLOR_PATTERN_FLIP_BOTH \
+	(V4L2_COLOR_PATTERN_FLIP_HORIZONTAL | V4L2_COLOR_PATTERN_FLIP_VERTICAL)
+
 
 /* Image processing controls */
 
-- 
2.47.2

Re: [PATCH v11 21/66] media: v4l: uapi: Add a control for color pattern flipping effect

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:22PM +0300, Sakari Ailus wrote:
> Add a bitmask control (V4L2_CID_COLOR_PATTERN_FLIP) to tell whether
> flipping results in a change in the sensor's color pattern, separately
> horizontally and vertically. The information is essential for raw formats
> when using generic raw mbus codes.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  2 ++
>  include/uapi/linux/v4l2-controls.h                     |  6 ++++++
>  3 files changed, 18 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index 0fcd60d8c621..ef3784d1119a 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -133,3 +133,13 @@ Image Source Control IDs
>        - Raw Bayer, with alternating lines beginning with green, blue pixels and
>          red, green pixels.
>        - 3
> +
> +``V4L2_CID_COLOR_PATTERN_FLIP (bitmask)``

I still believe that having this and V4L2_CTRL_FLAG_MODIFY_LAYOUT
might be confusing. Should sensor driver that registers H/VFLIP still
mark those controls as V4L2_CTRL_FLAG_MODIFY_LAYOUT or should they
implement V4L2_CID_COLOR_PATTERN_FLIP, or do both ?

Should we deprectate V4L2_CTRL_FLAG_MODIFY_LAYOUT for flip controls ?
In this case I think the documentation of the flag should be updated.

> +    Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
> +    V4L2_CID_VFLIP) have an effect on the pixel order of the output color
> +    pattern. Macros ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> +    ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` define bitmasks for both bits. If
> +    either horizontal or vertical bit is set, the readout pattern order is that
> +    of the reversed readout. ``V4L2_COLOR_PATTERN_FLIP_BOTH`` for setting both

I don't think

"the readout pattern order is that of the reversed readout." is very
clear.

Providing examples might be convenient

In libcamera we have:

 * Applying a transform to an image stored in a Bayer format affects the Bayer
 * order. For example, performing a horizontal flip on the Bayer pattern RGGB
 * causes the RG rows of pixels to become GR, and the GB rows to become BG. The
 * transformed image would have a GRBG order. Performing a vertical flip on the
 * Bayer pattern RGGB causes the GB rows to come before the RG ones and the
 * transformed image would have GBRG order. Applying both vertical and
 * horizontal flips on the Bayer patter RGGB results in transformed images with
 * BGGR order. The bit depth and modifiers are not affected.

Should we at least make a table for the most common Bayer CFA ?


> +    ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> +    ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.

Personally I don't like V4L2_COLOR_PATTERN_FLIP_BOTH much, and would
prefer to have the two flags separate, but it might be just personal
tastes.

> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 8119012c2638..9ec65998a8f7 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1156,6 +1156,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
>  	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
>  	case V4L2_CID_COLOR_PATTERN:		return "Color Pattern";
> +	case V4L2_CID_COLOR_PATTERN_FLIP:	return "Color Pattern Flip";
>
>  	/* Image processing controls */
>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> @@ -1484,6 +1485,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  		*type = V4L2_CTRL_TYPE_BITMASK;
>  		break;
>  	case V4L2_CID_CONFIG_MODEL:
> +	case V4L2_CID_COLOR_PATTERN_FLIP:
>  		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
>  		*type = V4L2_CTRL_TYPE_BITMASK;
>  		break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index e552f6655fd0..eb9ffdd74d32 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1233,6 +1233,12 @@ enum v4l2_jpeg_chroma_subsampling {
>  #define V4L2_COLOR_PATTERN_BGGR			2U
>  #define V4L2_COLOR_PATTERN_GBRG			3U
>
> +#define V4L2_CID_COLOR_PATTERN_FLIP		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 11)
> +#define V4L2_COLOR_PATTERN_FLIP_HORIZONTAL	(1U << 0)
> +#define V4L2_COLOR_PATTERN_FLIP_VERTICAL	(1U << 1)
> +#define V4L2_COLOR_PATTERN_FLIP_BOTH \
> +	(V4L2_COLOR_PATTERN_FLIP_HORIZONTAL | V4L2_COLOR_PATTERN_FLIP_VERTICAL)
> +
>
>  /* Image processing controls */
>
> --
> 2.47.2
>
>

Re: [PATCH v11 21/66] media: v4l: uapi: Add a control for color pattern flipping effect

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 04:34:56PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:22PM +0300, Sakari Ailus wrote:
> > Add a bitmask control (V4L2_CID_COLOR_PATTERN_FLIP) to tell whether
> > flipping results in a change in the sensor's color pattern, separately
> > horizontally and vertically. The information is essential for raw formats
> > when using generic raw mbus codes.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
> >  drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  2 ++
> >  include/uapi/linux/v4l2-controls.h                     |  6 ++++++
> >  3 files changed, 18 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > index 0fcd60d8c621..ef3784d1119a 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > @@ -133,3 +133,13 @@ Image Source Control IDs
> >        - Raw Bayer, with alternating lines beginning with green, blue pixels and
> >          red, green pixels.
> >        - 3
> > +
> > +``V4L2_CID_COLOR_PATTERN_FLIP (bitmask)``
> 
> I still believe that having this and V4L2_CTRL_FLAG_MODIFY_LAYOUT
> might be confusing. Should sensor driver that registers H/VFLIP still
> mark those controls as V4L2_CTRL_FLAG_MODIFY_LAYOUT or should they
> implement V4L2_CID_COLOR_PATTERN_FLIP, or do both ?
> 
> Should we deprectate V4L2_CTRL_FLAG_MODIFY_LAYOUT for flip controls ?
> In this case I think the documentation of the flag should be updated.

Good question. I don't think the flag provides any useful information to
the userspace in this case, it's just a matter of UAPI consistency.

How about setting set the flag to these controls in the control framework?
I can add a patch for that.

> 
> > +    Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
> > +    V4L2_CID_VFLIP) have an effect on the pixel order of the output color
> > +    pattern. Macros ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> > +    ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` define bitmasks for both bits. If
> > +    either horizontal or vertical bit is set, the readout pattern order is that
> > +    of the reversed readout. ``V4L2_COLOR_PATTERN_FLIP_BOTH`` for setting both
> 
> I don't think
> 
> "the readout pattern order is that of the reversed readout." is very
> clear.
> 
> Providing examples might be convenient
> 
> In libcamera we have:
> 
>  * Applying a transform to an image stored in a Bayer format affects the Bayer
>  * order. For example, performing a horizontal flip on the Bayer pattern RGGB
>  * causes the RG rows of pixels to become GR, and the GB rows to become BG. The
>  * transformed image would have a GRBG order. Performing a vertical flip on the
>  * Bayer pattern RGGB causes the GB rows to come before the RG ones and the
>  * transformed image would have GBRG order. Applying both vertical and
>  * horizontal flips on the Bayer patter RGGB results in transformed images with
>  * BGGR order. The bit depth and modifiers are not affected.
> 
> Should we at least make a table for the most common Bayer CFA ?

Do you think this is an issue in practice? The text is meant to apply to
non-Bayer patterns as well.

How about simply, for the part you commented on:

the readout pattern is reversed

> 
> 
> > +    ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> > +    ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
> 
> Personally I don't like V4L2_COLOR_PATTERN_FLIP_BOTH much, and would
> prefer to have the two flags separate, but it might be just personal
> tastes.

I only added it once I realised I needed it in sensor drivers. We don't
have to document it here in fact.

> 
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > index 8119012c2638..9ec65998a8f7 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > @@ -1156,6 +1156,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >  	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
> >  	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
> >  	case V4L2_CID_COLOR_PATTERN:		return "Color Pattern";
> > +	case V4L2_CID_COLOR_PATTERN_FLIP:	return "Color Pattern Flip";
> >
> >  	/* Image processing controls */
> >  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> > @@ -1484,6 +1485,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> >  		*type = V4L2_CTRL_TYPE_BITMASK;
> >  		break;
> >  	case V4L2_CID_CONFIG_MODEL:
> > +	case V4L2_CID_COLOR_PATTERN_FLIP:
> >  		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
> >  		*type = V4L2_CTRL_TYPE_BITMASK;
> >  		break;
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > index e552f6655fd0..eb9ffdd74d32 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -1233,6 +1233,12 @@ enum v4l2_jpeg_chroma_subsampling {
> >  #define V4L2_COLOR_PATTERN_BGGR			2U
> >  #define V4L2_COLOR_PATTERN_GBRG			3U
> >
> > +#define V4L2_CID_COLOR_PATTERN_FLIP		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 11)
> > +#define V4L2_COLOR_PATTERN_FLIP_HORIZONTAL	(1U << 0)
> > +#define V4L2_COLOR_PATTERN_FLIP_VERTICAL	(1U << 1)
> > +#define V4L2_COLOR_PATTERN_FLIP_BOTH \
> > +	(V4L2_COLOR_PATTERN_FLIP_HORIZONTAL | V4L2_COLOR_PATTERN_FLIP_VERTICAL)
> > +
> >
> >  /* Image processing controls */
> >

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v11 21/66] media: v4l: uapi: Add a control for color pattern flipping effect

[Sakari Ailus]

Hi Jacopo,

On Wed, Sep 03, 2025 at 01:25:30PM +0300, Sakari Ailus wrote:
> Hi Jacopo,
> 
> On Mon, Sep 01, 2025 at 04:34:56PM +0200, Jacopo Mondi wrote:
> > Hi Sakari
> > 
> > On Mon, Aug 25, 2025 at 12:50:22PM +0300, Sakari Ailus wrote:
> > > Add a bitmask control (V4L2_CID_COLOR_PATTERN_FLIP) to tell whether
> > > flipping results in a change in the sensor's color pattern, separately
> > > horizontally and vertically. The information is essential for raw formats
> > > when using generic raw mbus codes.
> > >
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
> > >  drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  2 ++
> > >  include/uapi/linux/v4l2-controls.h                     |  6 ++++++
> > >  3 files changed, 18 insertions(+)
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > index 0fcd60d8c621..ef3784d1119a 100644
> > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > @@ -133,3 +133,13 @@ Image Source Control IDs
> > >        - Raw Bayer, with alternating lines beginning with green, blue pixels and
> > >          red, green pixels.
> > >        - 3
> > > +
> > > +``V4L2_CID_COLOR_PATTERN_FLIP (bitmask)``
> > 
> > I still believe that having this and V4L2_CTRL_FLAG_MODIFY_LAYOUT
> > might be confusing. Should sensor driver that registers H/VFLIP still
> > mark those controls as V4L2_CTRL_FLAG_MODIFY_LAYOUT or should they
> > implement V4L2_CID_COLOR_PATTERN_FLIP, or do both ?
> > 
> > Should we deprectate V4L2_CTRL_FLAG_MODIFY_LAYOUT for flip controls ?
> > In this case I think the documentation of the flag should be updated.
> 
> Good question. I don't think the flag provides any useful information to
> the userspace in this case, it's just a matter of UAPI consistency.
> 
> How about setting set the flag to these controls in the control framework?
> I can add a patch for that.

Of course this is not workable as-is as the flag would be present on the
*FLIP controls.

This would be still worked around in the control framework, based on the
value of the flip control, but I think it'd be more practical to document
the flag does not apply to generic raw mbus codes or as you suggested, to
media bus codes at all. The patch that added it is
bf587efff165e0563cc9d2227e49eeba7eb50434 .

-- 
Regards,

Sakari Ailus

Re: [PATCH v11 21/66] media: v4l: uapi: Add a control for color pattern flipping effect

[Jacopo Mondi]

Hi Sakari

On Wed, Sep 03, 2025 at 01:40:51PM +0300, Sakari Ailus wrote:
> Hi Jacopo,
>
> On Wed, Sep 03, 2025 at 01:25:30PM +0300, Sakari Ailus wrote:
> > Hi Jacopo,
> >
> > On Mon, Sep 01, 2025 at 04:34:56PM +0200, Jacopo Mondi wrote:
> > > Hi Sakari
> > >
> > > On Mon, Aug 25, 2025 at 12:50:22PM +0300, Sakari Ailus wrote:
> > > > Add a bitmask control (V4L2_CID_COLOR_PATTERN_FLIP) to tell whether
> > > > flipping results in a change in the sensor's color pattern, separately
> > > > horizontally and vertically. The information is essential for raw formats
> > > > when using generic raw mbus codes.
> > > >
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > ---
> > > >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
> > > >  drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  2 ++
> > > >  include/uapi/linux/v4l2-controls.h                     |  6 ++++++
> > > >  3 files changed, 18 insertions(+)
> > > >
> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > index 0fcd60d8c621..ef3784d1119a 100644
> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > @@ -133,3 +133,13 @@ Image Source Control IDs
> > > >        - Raw Bayer, with alternating lines beginning with green, blue pixels and
> > > >          red, green pixels.
> > > >        - 3
> > > > +
> > > > +``V4L2_CID_COLOR_PATTERN_FLIP (bitmask)``
> > >
> > > I still believe that having this and V4L2_CTRL_FLAG_MODIFY_LAYOUT
> > > might be confusing. Should sensor driver that registers H/VFLIP still
> > > mark those controls as V4L2_CTRL_FLAG_MODIFY_LAYOUT or should they
> > > implement V4L2_CID_COLOR_PATTERN_FLIP, or do both ?
> > >
> > > Should we deprectate V4L2_CTRL_FLAG_MODIFY_LAYOUT for flip controls ?
> > > In this case I think the documentation of the flag should be updated.
> >
> > Good question. I don't think the flag provides any useful information to
> > the userspace in this case, it's just a matter of UAPI consistency.

"in this case" you mean if the sub-dev registers
V4L2_CID_COLOR_PATTERN_FLIP ?

> >
> > How about setting set the flag to these controls in the control framework?
> > I can add a patch for that.
>
> Of course this is not workable as-is as the flag would be present on the
> *FLIP controls.
>
> This would be still worked around in the control framework, based on the
> value of the flip control, but I think it'd be more practical to document
> the flag does not apply to generic raw mbus codes or as you suggested, to
> media bus codes at all. The patch that added it is
> bf587efff165e0563cc9d2227e49eeba7eb50434 .

I think documenting it as deprecated for sub-devs that use generic raw
mbus codes and register V4L2_CID_COLOR_PATTERN_FLIP it's possibily the
best we can do ?

>
> --
> Regards,
>
> Sakari Ailus

Re: [PATCH v11 21/66] media: v4l: uapi: Add a control for color pattern flipping effect

[Sakari Ailus]

Hi Jacopo,

On Wed, Sep 03, 2025 at 02:18:35PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Wed, Sep 03, 2025 at 01:40:51PM +0300, Sakari Ailus wrote:
> > Hi Jacopo,
> >
> > On Wed, Sep 03, 2025 at 01:25:30PM +0300, Sakari Ailus wrote:
> > > Hi Jacopo,
> > >
> > > On Mon, Sep 01, 2025 at 04:34:56PM +0200, Jacopo Mondi wrote:
> > > > Hi Sakari
> > > >
> > > > On Mon, Aug 25, 2025 at 12:50:22PM +0300, Sakari Ailus wrote:
> > > > > Add a bitmask control (V4L2_CID_COLOR_PATTERN_FLIP) to tell whether
> > > > > flipping results in a change in the sensor's color pattern, separately
> > > > > horizontally and vertically. The information is essential for raw formats
> > > > > when using generic raw mbus codes.
> > > > >
> > > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > ---
> > > > >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
> > > > >  drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  2 ++
> > > > >  include/uapi/linux/v4l2-controls.h                     |  6 ++++++
> > > > >  3 files changed, 18 insertions(+)
> > > > >
> > > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > > index 0fcd60d8c621..ef3784d1119a 100644
> > > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > > @@ -133,3 +133,13 @@ Image Source Control IDs
> > > > >        - Raw Bayer, with alternating lines beginning with green, blue pixels and
> > > > >          red, green pixels.
> > > > >        - 3
> > > > > +
> > > > > +``V4L2_CID_COLOR_PATTERN_FLIP (bitmask)``
> > > >
> > > > I still believe that having this and V4L2_CTRL_FLAG_MODIFY_LAYOUT
> > > > might be confusing. Should sensor driver that registers H/VFLIP still
> > > > mark those controls as V4L2_CTRL_FLAG_MODIFY_LAYOUT or should they
> > > > implement V4L2_CID_COLOR_PATTERN_FLIP, or do both ?
> > > >
> > > > Should we deprectate V4L2_CTRL_FLAG_MODIFY_LAYOUT for flip controls ?
> > > > In this case I think the documentation of the flag should be updated.
> > >
> > > Good question. I don't think the flag provides any useful information to
> > > the userspace in this case, it's just a matter of UAPI consistency.
> 
> "in this case" you mean if the sub-dev registers
> V4L2_CID_COLOR_PATTERN_FLIP ?

When used with generic raw formats.

> 
> > >
> > > How about setting set the flag to these controls in the control framework?
> > > I can add a patch for that.
> >
> > Of course this is not workable as-is as the flag would be present on the
> > *FLIP controls.
> >
> > This would be still worked around in the control framework, based on the
> > value of the flip control, but I think it'd be more practical to document
> > the flag does not apply to generic raw mbus codes or as you suggested, to
> > media bus codes at all. The patch that added it is
> > bf587efff165e0563cc9d2227e49eeba7eb50434 .
> 
> I think documenting it as deprecated for sub-devs that use generic raw
> mbus codes and register V4L2_CID_COLOR_PATTERN_FLIP it's possibily the
> best we can do ?

It is used currently to tell that the pattern (and so the mbus code)
changes with flipping controls.

-- 
Regards,

Sakari Ailus

Re: [PATCH v11 21/66] media: v4l: uapi: Add a control for color pattern flipping effect

[Jacopo Mondi]

Hi Sakari

On Wed, Sep 03, 2025 at 01:25:29PM +0300, Sakari Ailus wrote:
> Hi Jacopo,
>
> On Mon, Sep 01, 2025 at 04:34:56PM +0200, Jacopo Mondi wrote:
> > Hi Sakari
> >
> > On Mon, Aug 25, 2025 at 12:50:22PM +0300, Sakari Ailus wrote:
> > > Add a bitmask control (V4L2_CID_COLOR_PATTERN_FLIP) to tell whether
> > > flipping results in a change in the sensor's color pattern, separately
> > > horizontally and vertically. The information is essential for raw formats
> > > when using generic raw mbus codes.
> > >
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
> > >  drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  2 ++
> > >  include/uapi/linux/v4l2-controls.h                     |  6 ++++++
> > >  3 files changed, 18 insertions(+)
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > index 0fcd60d8c621..ef3784d1119a 100644
> > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > @@ -133,3 +133,13 @@ Image Source Control IDs
> > >        - Raw Bayer, with alternating lines beginning with green, blue pixels and
> > >          red, green pixels.
> > >        - 3
> > > +
> > > +``V4L2_CID_COLOR_PATTERN_FLIP (bitmask)``
> >
> > I still believe that having this and V4L2_CTRL_FLAG_MODIFY_LAYOUT
> > might be confusing. Should sensor driver that registers H/VFLIP still
> > mark those controls as V4L2_CTRL_FLAG_MODIFY_LAYOUT or should they
> > implement V4L2_CID_COLOR_PATTERN_FLIP, or do both ?
> >
> > Should we deprectate V4L2_CTRL_FLAG_MODIFY_LAYOUT for flip controls ?
> > In this case I think the documentation of the flag should be updated.
>
> Good question. I don't think the flag provides any useful information to
> the userspace in this case, it's just a matter of UAPI consistency.
>
> How about setting set the flag to these controls in the control framework?
> I can add a patch for that.
>
> >
> > > +    Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
> > > +    V4L2_CID_VFLIP) have an effect on the pixel order of the output color
> > > +    pattern. Macros ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> > > +    ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` define bitmasks for both bits. If
> > > +    either horizontal or vertical bit is set, the readout pattern order is that
> > > +    of the reversed readout. ``V4L2_COLOR_PATTERN_FLIP_BOTH`` for setting both
> >
> > I don't think
> >
> > "the readout pattern order is that of the reversed readout." is very
> > clear.
> >
> > Providing examples might be convenient
> >
> > In libcamera we have:
> >
> >  * Applying a transform to an image stored in a Bayer format affects the Bayer
> >  * order. For example, performing a horizontal flip on the Bayer pattern RGGB
> >  * causes the RG rows of pixels to become GR, and the GB rows to become BG. The
> >  * transformed image would have a GRBG order. Performing a vertical flip on the
> >  * Bayer pattern RGGB causes the GB rows to come before the RG ones and the
> >  * transformed image would have GBRG order. Applying both vertical and
> >  * horizontal flips on the Bayer patter RGGB results in transformed images with
> >  * BGGR order. The bit depth and modifiers are not affected.
> >
> > Should we at least make a table for the most common Bayer CFA ?
>
> Do you think this is an issue in practice? The text is meant to apply to
> non-Bayer patterns as well.

Not an issue... I just have found myself multiple times having to go
and check that piece of documentation to check how the Bayer pattern
ordering is affected by flips. If we use generic raw media bus codes
along the whole pipeline, knowing the correct ordering is only useful
for further processing steps (converting a RAW image to something
user-consumable, or queueing the buffer with format V4L2_PIX_FMT_RAW_*
to a device that needs to know the bayer order, this is typical for a
m2m ISP in example).

I'm not sure how many non-Bayer CFA cameras we do support, indeed we
should account for them, but am I wrong assuming the majority, if not
all, are Bayer, so having a quick reference might be useful.

Anyway, not feeling strong about this, up to you.


>
> How about simply, for the part you commented on:
>
> the readout pattern is reversed
>
> >
> >
> > > +    ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> > > +    ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
> >
> > Personally I don't like V4L2_COLOR_PATTERN_FLIP_BOTH much, and would
> > prefer to have the two flags separate, but it might be just personal
> > tastes.
>
> I only added it once I realised I needed it in sensor drivers. We don't
> have to document it here in fact.
>
> >
> > > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > > index 8119012c2638..9ec65998a8f7 100644
> > > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > > @@ -1156,6 +1156,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> > >  	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
> > >  	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
> > >  	case V4L2_CID_COLOR_PATTERN:		return "Color Pattern";
> > > +	case V4L2_CID_COLOR_PATTERN_FLIP:	return "Color Pattern Flip";
> > >
> > >  	/* Image processing controls */
> > >  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> > > @@ -1484,6 +1485,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> > >  		*type = V4L2_CTRL_TYPE_BITMASK;
> > >  		break;
> > >  	case V4L2_CID_CONFIG_MODEL:
> > > +	case V4L2_CID_COLOR_PATTERN_FLIP:
> > >  		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
> > >  		*type = V4L2_CTRL_TYPE_BITMASK;
> > >  		break;
> > > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > > index e552f6655fd0..eb9ffdd74d32 100644
> > > --- a/include/uapi/linux/v4l2-controls.h
> > > +++ b/include/uapi/linux/v4l2-controls.h
> > > @@ -1233,6 +1233,12 @@ enum v4l2_jpeg_chroma_subsampling {
> > >  #define V4L2_COLOR_PATTERN_BGGR			2U
> > >  #define V4L2_COLOR_PATTERN_GBRG			3U
> > >
> > > +#define V4L2_CID_COLOR_PATTERN_FLIP		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 11)
> > > +#define V4L2_COLOR_PATTERN_FLIP_HORIZONTAL	(1U << 0)
> > > +#define V4L2_COLOR_PATTERN_FLIP_VERTICAL	(1U << 1)
> > > +#define V4L2_COLOR_PATTERN_FLIP_BOTH \
> > > +	(V4L2_COLOR_PATTERN_FLIP_HORIZONTAL | V4L2_COLOR_PATTERN_FLIP_VERTICAL)
> > > +
> > >
> > >  /* Image processing controls */
> > >
>
> --
> Kind regards,
>
> Sakari Ailus

Re: [PATCH v11 21/66] media: v4l: uapi: Add a control for color pattern flipping effect

[Sakari Ailus]

Hi Jacopo,

On Wed, Sep 03, 2025 at 02:37:56PM +0200, Jacopo Mondi wrote:
> I'm not sure how many non-Bayer CFA cameras we do support, indeed we
> should account for them, but am I wrong assuming the majority, if not
> all, are Bayer, so having a quick reference might be useful.

Well, one important reason for not having non-Bayer pattern cameras is that
we effectively only support Bayer patterns. :-) There are lots of different
patterns, including different filters, but most devices can be configured
to generate Bayer output. Also the size of the pattern may be larger than
2x2.

> 
> Anyway, not feeling strong about this, up to you.

Thanks!

-- 
Regards,

Sakari Ailus

Re: [PATCH v11 21/66] media: v4l: uapi: Add a control for color pattern flipping effect

[Laurent Pinchart]

On Wed, Sep 03, 2025 at 04:25:11PM +0300, Sakari Ailus wrote:
> Hi Jacopo,
> 
> On Wed, Sep 03, 2025 at 02:37:56PM +0200, Jacopo Mondi wrote:
> > I'm not sure how many non-Bayer CFA cameras we do support, indeed we
> > should account for them, but am I wrong assuming the majority, if not
> > all, are Bayer, so having a quick reference might be useful.
> 
> Well, one important reason for not having non-Bayer pattern cameras is that
> we effectively only support Bayer patterns. :-) There are lots of different
> patterns, including different filters, but most devices can be configured
> to generate Bayer output. Also the size of the pattern may be larger than
> 2x2.

Quad bayer comes to mind, as well as 4x4 patterns where two of the 2x2
regions are regular Bayer and the other two are Bayer filters with green
replaced by Ir. Quad bayer may need extra thoughts, as some sensors
support "rebayering" the pixels and output regular Bayer.

> > Anyway, not feeling strong about this, up to you.
> 
> Thanks!

-- 
Regards,

Laurent Pinchart

[PATCH v11 22/66] media: Documentation: Reference flipping controls in raw format docs

[Sakari Ailus]

Add a reference to the flipping related controls (V4L2_CID_HFLIP,
V4L2_CID_VFLIP and V4L2_CID_COLOR_PATTERN_FLIP) to the generic raw format
documentation.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../userspace-api/media/v4l/ext-ctrls-image-source.rst         | 2 ++
 Documentation/userspace-api/media/v4l/subdev-formats.rst       | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index ef3784d1119a..3cb7ee1b0aed 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -134,6 +134,8 @@ Image Source Control IDs
         red, green pixels.
       - 3
 
+.. _image-source-control-color-pattern-flip:
+
 ``V4L2_CID_COLOR_PATTERN_FLIP (bitmask)``
     Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
     V4L2_CID_VFLIP) have an effect on the pixel order of the output color
diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
index 77571adeb21e..c36cd42b09c6 100644
--- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
@@ -3446,6 +3446,9 @@ however the bit depth is still specific to the format.
 
 The colour components and the native pixel order are determined by the
 :ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control.
+Whether or not flipping controls (``V4L2_CID_HFLIP`` and ``V4L2_CID_VFLIP``)
+affect the pattern is conveyed via the :ref:`V4L2_CID_COLOR_PATTERN_FLIP
+<image-source-control-color-pattern-flip>` control.
 
 .. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
 
-- 
2.47.2

Re: [PATCH v11 22/66] media: Documentation: Reference flipping controls in raw format docs

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:23PM +0300, Sakari Ailus wrote:
> Add a reference to the flipping related controls (V4L2_CID_HFLIP,
> V4L2_CID_VFLIP and V4L2_CID_COLOR_PATTERN_FLIP) to the generic raw format
> documentation.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j

> ---
>  .../userspace-api/media/v4l/ext-ctrls-image-source.rst         | 2 ++
>  Documentation/userspace-api/media/v4l/subdev-formats.rst       | 3 +++
>  2 files changed, 5 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index ef3784d1119a..3cb7ee1b0aed 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -134,6 +134,8 @@ Image Source Control IDs
>          red, green pixels.
>        - 3
>
> +.. _image-source-control-color-pattern-flip:
> +
>  ``V4L2_CID_COLOR_PATTERN_FLIP (bitmask)``
>      Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
>      V4L2_CID_VFLIP) have an effect on the pixel order of the output color
> diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> index 77571adeb21e..c36cd42b09c6 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> @@ -3446,6 +3446,9 @@ however the bit depth is still specific to the format.
>
>  The colour components and the native pixel order are determined by the
>  :ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control.
> +Whether or not flipping controls (``V4L2_CID_HFLIP`` and ``V4L2_CID_VFLIP``)
> +affect the pattern is conveyed via the :ref:`V4L2_CID_COLOR_PATTERN_FLIP
> +<image-source-control-color-pattern-flip>` control.
>
>  .. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
>
> --
> 2.47.2
>
>

Re: [PATCH v11 22/66] media: Documentation: Reference flipping controls in raw format docs

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Aug 25, 2025 at 12:50:23PM +0300, Sakari Ailus wrote:
> Add a reference to the flipping related controls (V4L2_CID_HFLIP,
> V4L2_CID_VFLIP and V4L2_CID_COLOR_PATTERN_FLIP) to the generic raw format
> documentation.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Assuming we'll agree on the V4L2_CID_COLOR_PATTERN_FLIP,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  .../userspace-api/media/v4l/ext-ctrls-image-source.rst         | 2 ++
>  Documentation/userspace-api/media/v4l/subdev-formats.rst       | 3 +++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index ef3784d1119a..3cb7ee1b0aed 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -134,6 +134,8 @@ Image Source Control IDs
>          red, green pixels.
>        - 3
>  
> +.. _image-source-control-color-pattern-flip:
> +
>  ``V4L2_CID_COLOR_PATTERN_FLIP (bitmask)``
>      Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
>      V4L2_CID_VFLIP) have an effect on the pixel order of the output color
> diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> index 77571adeb21e..c36cd42b09c6 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
> @@ -3446,6 +3446,9 @@ however the bit depth is still specific to the format.
>  
>  The colour components and the native pixel order are determined by the
>  :ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` control.
> +Whether or not flipping controls (``V4L2_CID_HFLIP`` and ``V4L2_CID_VFLIP``)
> +affect the pattern is conveyed via the :ref:`V4L2_CID_COLOR_PATTERN_FLIP
> +<image-source-control-color-pattern-flip>` control.
>  
>  .. tabularcolumns:: |p{2.0cm}|p{4.0cm}|p{11.3cm}|
>  

-- 
Regards,

Laurent Pinchart

[PATCH v11 23/66] media: Documentation: Document raw mbus codes and CFA for cameras

[Sakari Ailus]

Document the use of raw mbus codes for camera sensors and how the
V4L2_CID_COLOR_PATTERN and V4L2_CID_COLOR_PATTERN_FLIP controls are used
to convey the color filter array pattern on UAPI.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../userspace-api/media/drivers/camera-sensor.rst   | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index 75fd9166383f..a8e0e4accbc9 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -39,6 +39,19 @@ format set on a source pad at the end of the device's internal pipeline.
 
 Most sensor drivers are implemented this way.
 
+V4L2_CID_COLOR_PATTERN, raw mbus formats, flipping and cropping
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For raw image data originating from camera sensors, specific :ref:`raw mbus
+codes MEDIA_BUS_FMT_RAW_x (where 'x' is the bit depth)
+<v4l2-mbus-pixelcode-generic-raw>` are used as Color Filter Array (CFA) agnostic
+raw formats. The ``V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>``
+control in the same sub-device defines the native color pattern of the
+device. Flipping may further affect the readout pattern as indicated by the
+``V4L2_CID_COLOR_PATTERN_FLIP <image-source-control-color-pattern-flip>``
+control. Further on, cropping also has an effect on the pattern if cropped
+amount is not divisible by the size of the pattern, horizontally and vertically.
+
 Frame interval configuration
 ----------------------------
 
-- 
2.47.2

Re: [PATCH v11 23/66] media: Documentation: Document raw mbus codes and CFA for cameras

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:24PM +0300, Sakari Ailus wrote:
> Document the use of raw mbus codes for camera sensors and how the
> V4L2_CID_COLOR_PATTERN and V4L2_CID_COLOR_PATTERN_FLIP controls are used
> to convey the color filter array pattern on UAPI.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../userspace-api/media/drivers/camera-sensor.rst   | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index 75fd9166383f..a8e0e4accbc9 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -39,6 +39,19 @@ format set on a source pad at the end of the device's internal pipeline.
>
>  Most sensor drivers are implemented this way.
>
> +V4L2_CID_COLOR_PATTERN, raw mbus formats, flipping and cropping
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This will appear as a subsection of

"2.1. Sensor internal pipeline configuration"

With

2.1.1. Freely configurable camera sensor drivers
2.1.2. Register list-based drivers
2.1.3. V4L2_CID_COLOR_PATTERN, raw mbus formats, flipping and cropping

I don't know if it's intentional or not, but I would rather make out
of this a paragraph of its own, as it will be done for

2.3. Rotation, orientation and flipping
2.4. Binning and sub-sampling

etc

> +
> +For raw image data originating from camera sensors, specific :ref:`raw mbus
> +codes MEDIA_BUS_FMT_RAW_x (where 'x' is the bit depth)
> +<v4l2-mbus-pixelcode-generic-raw>` are used as Color Filter Array (CFA) agnostic
> +raw formats. The ``V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>``

This and the below ``V4L2_CID_COLOR_PATTERN_FLIP ...`` links are
broken.

Shuould this be

:ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>`

> +control in the same sub-device defines the native color pattern of the
> +device. Flipping may further affect the readout pattern as indicated by the
> +``V4L2_CID_COLOR_PATTERN_FLIP <image-source-control-color-pattern-flip>``

and this

:ref:`V4L2_CID_COLOR_PATTERN_FLIP <image-source-control-color-pattern-flip>`

instead ?

> +control. Further on, cropping also has an effect on the pattern if cropped
> +amount is not divisible by the size of the pattern, horizontally and vertically.
> +
>  Frame interval configuration
>  ----------------------------
>
> --
> 2.47.2
>
>

Re: [PATCH v11 23/66] media: Documentation: Document raw mbus codes and CFA for cameras

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 04:59:42PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:24PM +0300, Sakari Ailus wrote:
> > Document the use of raw mbus codes for camera sensors and how the
> > V4L2_CID_COLOR_PATTERN and V4L2_CID_COLOR_PATTERN_FLIP controls are used
> > to convey the color filter array pattern on UAPI.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../userspace-api/media/drivers/camera-sensor.rst   | 13 +++++++++++++
> >  1 file changed, 13 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > index 75fd9166383f..a8e0e4accbc9 100644
> > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > @@ -39,6 +39,19 @@ format set on a source pad at the end of the device's internal pipeline.
> >
> >  Most sensor drivers are implemented this way.
> >
> > +V4L2_CID_COLOR_PATTERN, raw mbus formats, flipping and cropping
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> This will appear as a subsection of
> 
> "2.1. Sensor internal pipeline configuration"
> 
> With
> 
> 2.1.1. Freely configurable camera sensor drivers
> 2.1.2. Register list-based drivers
> 2.1.3. V4L2_CID_COLOR_PATTERN, raw mbus formats, flipping and cropping
> 
> I don't know if it's intentional or not, but I would rather make out
> of this a paragraph of its own, as it will be done for
> 
> 2.3. Rotation, orientation and flipping
> 2.4. Binning and sub-sampling

It's all related to sensor internal pipeline configuration. It's not wrong
to put them after it instead of under though. I don't have a strong opinion
either way.

> 
> etc
> 
> > +
> > +For raw image data originating from camera sensors, specific :ref:`raw mbus
> > +codes MEDIA_BUS_FMT_RAW_x (where 'x' is the bit depth)
> > +<v4l2-mbus-pixelcode-generic-raw>` are used as Color Filter Array (CFA) agnostic
> > +raw formats. The ``V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>``
> 
> This and the below ``V4L2_CID_COLOR_PATTERN_FLIP ...`` links are
> broken.
> 
> Shuould this be
> 
> :ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>`
> 
> > +control in the same sub-device defines the native color pattern of the
> > +device. Flipping may further affect the readout pattern as indicated by the
> > +``V4L2_CID_COLOR_PATTERN_FLIP <image-source-control-color-pattern-flip>``
> 
> and this
> 
> :ref:`V4L2_CID_COLOR_PATTERN_FLIP <image-source-control-color-pattern-flip>`
> 
> instead ?

Yes.

> 
> > +control. Further on, cropping also has an effect on the pattern if cropped
> > +amount is not divisible by the size of the pattern, horizontally and vertically.
> > +
> >  Frame interval configuration
> >  ----------------------------
> >

-- 
Regards,

Sakari Ailus

Re: [PATCH v11 23/66] media: Documentation: Document raw mbus codes and CFA for cameras

[Laurent Pinchart]

On Wed, Sep 03, 2025 at 02:00:17PM +0300, Sakari Ailus wrote:
> On Mon, Sep 01, 2025 at 04:59:42PM +0200, Jacopo Mondi wrote:
> > On Mon, Aug 25, 2025 at 12:50:24PM +0300, Sakari Ailus wrote:
> > > Document the use of raw mbus codes for camera sensors and how the
> > > V4L2_CID_COLOR_PATTERN and V4L2_CID_COLOR_PATTERN_FLIP controls are used
> > > to convey the color filter array pattern on UAPI.
> > >
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  .../userspace-api/media/drivers/camera-sensor.rst   | 13 +++++++++++++
> > >  1 file changed, 13 insertions(+)
> > >
> > > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > index 75fd9166383f..a8e0e4accbc9 100644
> > > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > @@ -39,6 +39,19 @@ format set on a source pad at the end of the device's internal pipeline.
> > >
> > >  Most sensor drivers are implemented this way.
> > >
> > > +V4L2_CID_COLOR_PATTERN, raw mbus formats, flipping and cropping
> > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > 
> > This will appear as a subsection of
> > 
> > "2.1. Sensor internal pipeline configuration"
> > 
> > With
> > 
> > 2.1.1. Freely configurable camera sensor drivers
> > 2.1.2. Register list-based drivers
> > 2.1.3. V4L2_CID_COLOR_PATTERN, raw mbus formats, flipping and cropping
> > 
> > I don't know if it's intentional or not, but I would rather make out
> > of this a paragraph of its own, as it will be done for
> > 
> > 2.3. Rotation, orientation and flipping
> > 2.4. Binning and sub-sampling
> 
> It's all related to sensor internal pipeline configuration. It's not wrong
> to put them after it instead of under though. I don't have a strong opinion
> either way.
> 
> > etc
> > 
> > > +
> > > +For raw image data originating from camera sensors, specific :ref:`raw mbus
> > > +codes MEDIA_BUS_FMT_RAW_x (where 'x' is the bit depth)
> > > +<v4l2-mbus-pixelcode-generic-raw>` are used as Color Filter Array (CFA) agnostic
> > > +raw formats. The ``V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>``
> > 
> > This and the below ``V4L2_CID_COLOR_PATTERN_FLIP ...`` links are
> > broken.
> > 
> > Shuould this be
> > 
> > :ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>`
> > 
> > > +control in the same sub-device defines the native color pattern of the
> > > +device. Flipping may further affect the readout pattern as indicated by the
> > > +``V4L2_CID_COLOR_PATTERN_FLIP <image-source-control-color-pattern-flip>``
> > 
> > and this
> > 
> > :ref:`V4L2_CID_COLOR_PATTERN_FLIP <image-source-control-color-pattern-flip>`
> > 
> > instead ?
> 
> Yes.
> 
> > 
> > > +control. Further on, cropping also has an effect on the pattern if cropped
> > > +amount is not divisible by the size of the pattern, horizontally and vertically.

We could expand this, but it's a good start.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> > > +
> > >  Frame interval configuration
> > >  ----------------------------
> > >

-- 
Regards,

Laurent Pinchart

[PATCH v11 24/66] media: uapi: Add V4L2_CID_METADATA_LAYOUT control

[Sakari Ailus]

The metadata layout control defines the layout of the metadata on the bus.
It is used on sub-devices that use generic metadata mbus codes on at least
on one of the pads.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../userspace-api/media/v4l/ext-ctrls-image-source.rst      | 6 ++++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c                   | 1 +
 include/uapi/linux/v4l2-controls.h                          | 2 ++
 3 files changed, 9 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index 3cb7ee1b0aed..64c0f9ff5b1b 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -145,3 +145,9 @@ Image Source Control IDs
     of the reversed readout. ``V4L2_COLOR_PATTERN_FLIP_BOTH`` for setting both
     ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
     ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
+
+``V4L2_CID_METADATA_LAYOUT (integer)``
+    The metadata layout control defines the on-bus metadata layout for metadata
+    streams. The control is used in conjunction with :ref:`generic metadata
+    formats <media-bus-format-generic-meta>` formats to specify the layout of the
+    data.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 9ec65998a8f7..a7ea380de5ee 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1157,6 +1157,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
 	case V4L2_CID_COLOR_PATTERN:		return "Color Pattern";
 	case V4L2_CID_COLOR_PATTERN_FLIP:	return "Color Pattern Flip";
+	case V4L2_CID_METADATA_LAYOUT:		return "Metadata Layout";
 
 	/* Image processing controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index eb9ffdd74d32..74e22a930be4 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1239,6 +1239,8 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_COLOR_PATTERN_FLIP_BOTH \
 	(V4L2_COLOR_PATTERN_FLIP_HORIZONTAL | V4L2_COLOR_PATTERN_FLIP_VERTICAL)
 
+#define V4L2_CID_METADATA_LAYOUT		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 12)
+
 
 /* Image processing controls */
 
-- 
2.47.2

Re: [PATCH v11 24/66] media: uapi: Add V4L2_CID_METADATA_LAYOUT control

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:25PM +0300, Sakari Ailus wrote:
> The metadata layout control defines the layout of the metadata on the bus.
> It is used on sub-devices that use generic metadata mbus codes on at least
> on one of the pads.

Ideally:
1) we have per pad controls
2) s/V4L2_CID_COLOR_PATTERN/V4L2_CID_DATA_PATTERN
3) we use V4L2_CID_DATA_PATTERN for both color components ordering and
   metadata layouts. A sub-device will expose the same control, but on
   different pads

I know we don't want to block this series to wait for per-pad
controls, but I'm a bit afraid V4L2_CID_METADATA_LAYOUT will be made
redundant before it gets used for real (which maybe it's not a big
deal as if no one uses it, we will be able to easily replace it).

>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../userspace-api/media/v4l/ext-ctrls-image-source.rst      | 6 ++++++
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c                   | 1 +
>  include/uapi/linux/v4l2-controls.h                          | 2 ++
>  3 files changed, 9 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index 3cb7ee1b0aed..64c0f9ff5b1b 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -145,3 +145,9 @@ Image Source Control IDs
>      of the reversed readout. ``V4L2_COLOR_PATTERN_FLIP_BOTH`` for setting both
>      ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
>      ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
> +
> +``V4L2_CID_METADATA_LAYOUT (integer)``
> +    The metadata layout control defines the on-bus metadata layout for metadata
> +    streams. The control is used in conjunction with :ref:`generic metadata
> +    formats <media-bus-format-generic-meta>` formats to specify the layout of the
> +    data.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 9ec65998a8f7..a7ea380de5ee 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1157,6 +1157,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
>  	case V4L2_CID_COLOR_PATTERN:		return "Color Pattern";
>  	case V4L2_CID_COLOR_PATTERN_FLIP:	return "Color Pattern Flip";
> +	case V4L2_CID_METADATA_LAYOUT:		return "Metadata Layout";
>
>  	/* Image processing controls */
>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index eb9ffdd74d32..74e22a930be4 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1239,6 +1239,8 @@ enum v4l2_jpeg_chroma_subsampling {
>  #define V4L2_COLOR_PATTERN_FLIP_BOTH \
>  	(V4L2_COLOR_PATTERN_FLIP_HORIZONTAL | V4L2_COLOR_PATTERN_FLIP_VERTICAL)
>
> +#define V4L2_CID_METADATA_LAYOUT		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 12)
> +

Intentional empty line ?

For the time being
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

>
>  /* Image processing controls */
>
> --
> 2.47.2
>
>

Re: [PATCH v11 24/66] media: uapi: Add V4L2_CID_METADATA_LAYOUT control

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 05:07:55PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:25PM +0300, Sakari Ailus wrote:
> > The metadata layout control defines the layout of the metadata on the bus.
> > It is used on sub-devices that use generic metadata mbus codes on at least
> > on one of the pads.
> 
> Ideally:
> 1) we have per pad controls
> 2) s/V4L2_CID_COLOR_PATTERN/V4L2_CID_DATA_PATTERN
> 3) we use V4L2_CID_DATA_PATTERN for both color components ordering and
>    metadata layouts. A sub-device will expose the same control, but on
>    different pads
> 
> I know we don't want to block this series to wait for per-pad
> controls, but I'm a bit afraid V4L2_CID_METADATA_LAYOUT will be made
> redundant before it gets used for real (which maybe it's not a big
> deal as if no one uses it, we will be able to easily replace it).

I wouldn't do that, for the reason that the mbus codes in the two cases
will be different in any case so both will have separate namespaces.

Metadata is also unaffected by flipping, unlike the colour patterns.

I wonder what others think.

> 
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst      | 6 ++++++
> >  drivers/media/v4l2-core/v4l2-ctrls-defs.c                   | 1 +
> >  include/uapi/linux/v4l2-controls.h                          | 2 ++
> >  3 files changed, 9 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > index 3cb7ee1b0aed..64c0f9ff5b1b 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > @@ -145,3 +145,9 @@ Image Source Control IDs
> >      of the reversed readout. ``V4L2_COLOR_PATTERN_FLIP_BOTH`` for setting both
> >      ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> >      ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
> > +
> > +``V4L2_CID_METADATA_LAYOUT (integer)``
> > +    The metadata layout control defines the on-bus metadata layout for metadata
> > +    streams. The control is used in conjunction with :ref:`generic metadata
> > +    formats <media-bus-format-generic-meta>` formats to specify the layout of the
> > +    data.
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > index 9ec65998a8f7..a7ea380de5ee 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > @@ -1157,6 +1157,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >  	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
> >  	case V4L2_CID_COLOR_PATTERN:		return "Color Pattern";
> >  	case V4L2_CID_COLOR_PATTERN_FLIP:	return "Color Pattern Flip";
> > +	case V4L2_CID_METADATA_LAYOUT:		return "Metadata Layout";
> >
> >  	/* Image processing controls */
> >  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > index eb9ffdd74d32..74e22a930be4 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -1239,6 +1239,8 @@ enum v4l2_jpeg_chroma_subsampling {
> >  #define V4L2_COLOR_PATTERN_FLIP_BOTH \
> >  	(V4L2_COLOR_PATTERN_FLIP_HORIZONTAL | V4L2_COLOR_PATTERN_FLIP_VERTICAL)
> >
> > +#define V4L2_CID_METADATA_LAYOUT		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 12)
> > +
> 
> Intentional empty line ?

I'll drop it.

> 
> For the time being
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thank you.

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v11 24/66] media: uapi: Add V4L2_CID_METADATA_LAYOUT control

[Laurent Pinchart]

On Wed, Sep 03, 2025 at 02:13:07PM +0300, Sakari Ailus wrote:
> On Mon, Sep 01, 2025 at 05:07:55PM +0200, Jacopo Mondi wrote:
> > On Mon, Aug 25, 2025 at 12:50:25PM +0300, Sakari Ailus wrote:
> > > The metadata layout control defines the layout of the metadata on the bus.
> > > It is used on sub-devices that use generic metadata mbus codes on at least
> > > on one of the pads.
> > 
> > Ideally:
> > 1) we have per pad controls
> > 2) s/V4L2_CID_COLOR_PATTERN/V4L2_CID_DATA_PATTERN
> > 3) we use V4L2_CID_DATA_PATTERN for both color components ordering and
> >    metadata layouts. A sub-device will expose the same control, but on
> >    different pads
> > 
> > I know we don't want to block this series to wait for per-pad
> > controls, but I'm a bit afraid V4L2_CID_METADATA_LAYOUT will be made
> > redundant before it gets used for real (which maybe it's not a big
> > deal as if no one uses it, we will be able to easily replace it).
> 
> I wouldn't do that, for the reason that the mbus codes in the two cases
> will be different in any case so both will have separate namespaces.
> 
> Metadata is also unaffected by flipping, unlike the colour patterns.
> 
> I wonder what others think.

Per-pad controls could possibly be used here, but they're not there yet,
and the CFA pattern and metadata layout are two different concepts.

This being said, if you consider a sensor that produces multiple
metadata streams (e.g. embedded data and NPU tensors, transported over
different DTs), a single V4L2_CID_METADATA_LAYOUT control won't scale.
We will likely need per-pad, or even per-stream controls for that.

We could start with V4L2_CID_METADATA_LAYOUT and later extend it with
per-pad/stream controls for additional metadata streams, I don't think
that would cause trouble in userspace.

> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst      | 6 ++++++
> > >  drivers/media/v4l2-core/v4l2-ctrls-defs.c                   | 1 +
> > >  include/uapi/linux/v4l2-controls.h                          | 2 ++
> > >  3 files changed, 9 insertions(+)
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > index 3cb7ee1b0aed..64c0f9ff5b1b 100644
> > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > @@ -145,3 +145,9 @@ Image Source Control IDs
> > >      of the reversed readout. ``V4L2_COLOR_PATTERN_FLIP_BOTH`` for setting both
> > >      ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> > >      ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
> > > +
> > > +``V4L2_CID_METADATA_LAYOUT (integer)``
> > > +    The metadata layout control defines the on-bus metadata layout for metadata

This needs to be clearer. "on-bus metadata layout" left me guessing what
values this control would take. Before the patches further in the series
that add layout values, I wasn't sure if the control would use media bus
codes or other types of values.

> > > +    streams. The control is used in conjunction with :ref:`generic metadata
> > > +    formats <media-bus-format-generic-meta>` formats to specify the layout of the
> > > +    data.
> > > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > > index 9ec65998a8f7..a7ea380de5ee 100644
> > > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > > @@ -1157,6 +1157,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> > >  	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
> > >  	case V4L2_CID_COLOR_PATTERN:		return "Color Pattern";
> > >  	case V4L2_CID_COLOR_PATTERN_FLIP:	return "Color Pattern Flip";
> > > +	case V4L2_CID_METADATA_LAYOUT:		return "Metadata Layout";

How about v4l2_ctrl_fill() ?

> > >
> > >  	/* Image processing controls */
> > >  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> > > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > > index eb9ffdd74d32..74e22a930be4 100644
> > > --- a/include/uapi/linux/v4l2-controls.h
> > > +++ b/include/uapi/linux/v4l2-controls.h
> > > @@ -1239,6 +1239,8 @@ enum v4l2_jpeg_chroma_subsampling {
> > >  #define V4L2_COLOR_PATTERN_FLIP_BOTH \
> > >  	(V4L2_COLOR_PATTERN_FLIP_HORIZONTAL | V4L2_COLOR_PATTERN_FLIP_VERTICAL)
> > >
> > > +#define V4L2_CID_METADATA_LAYOUT		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 12)
> > > +
> > 
> > Intentional empty line ?
> 
> I'll drop it.
> 
> > For the time being
> > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> 
> Thank you.

-- 
Regards,

Laurent Pinchart

[PATCH v11 25/66] media: Documentation: v4l: Document internal sink pads

[Sakari Ailus]

Document internal sink pads, pads that have both SINK and INTERNAL flags
set. Use the IMX219 camera sensor as an example.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by Julien Massot <julien.massot@collabora.com>
---
 .../userspace-api/media/v4l/dev-subdev.rst    | 151 ++++++++++++++++++
 .../media/v4l/ext-ctrls-image-source.rst      |   2 +
 2 files changed, 153 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index 4da67ee0b290..bb86cadfad1c 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the
 sub-device and a (pad, stream) pair. For sub-devices that do not support
 multiplexed streams the 'stream' field is always 0.
 
+.. _v4l2-subdev-internal-source-pads:
+
+Internal sink pads and routing
+------------------------------
+
+Cases where a single sub-device source pad is traversed by multiple streams, one
+or more of which originate from within the sub-device itself, are special as
+there is no external sink pad for such routes. In those cases, the sources of
+the internally generated streams are represented by internal sink pads, which
+are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`
+pad flag set.
+
+Internal pads have all the properties of an external pad, including formats and
+selections. The format in this case is the source format of the stream. An
+internal pad always has a single stream only (0).
+
+Routes from an internal sink pad to an external source pad are typically not
+modifiable but they can be activated and deactivated using the
+:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag, depending
+on driver capabilities.
+
 Interaction between routes, streams, formats and selections
 -----------------------------------------------------------
 
@@ -695,3 +716,133 @@ To configure this pipeline, the userspace must take the following steps:
    the configurations along the stream towards the receiver, using
    :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each
    stream endpoint in each sub-device.
+
+   In case generic raw and metadata formats are used,
+   :ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` and
+   :ref:`V4L2_CID_METADATA_LAYOUT <image_source_control_metadata_layout>`
+   controls are present on the source sub-device to obtain the pixel data color
+   pattern and metadata layout.
+
+Internal pads setup example
+---------------------------
+
+A simple example of a multiplexed stream setup might be as follows:
+
+- An IMX219 camera sensor source sub-device, with one source pad (0), one
+  internal sink pad (1) as the source of the image data and an internal sink
+  pad (2) as the source of the embedded data. There are two routes, one from the
+  internal sink pad 1 to the source pad 0 (image data) and another from the
+  internal sink pad 2 to the source pad 0 (embedded data). Both streams are
+  always active, i.e. there is no need to separately enable the embedded data
+  stream. The sensor uses the CSI-2 interface.
+
+- A CSI-2 receiver in the SoC. The receiver has a single sink pad (pad 0),
+  connected to the sensor, and two source pads (pads 1 and 2), to the two DMA
+  engines. The receiver demultiplexes the incoming streams to the source pads.
+
+- DMA engines in the SoC, one for each stream. Each DMA engine is connected to a
+  single source pad of the receiver.
+
+The sensor and the receiver are modelled as V4L2 sub-devices, exposed to
+userspace via /dev/v4l-subdevX device nodes. The DMA engines are modelled as
+V4L2 devices, exposed to userspace via /dev/videoX nodes.
+
+To configure this pipeline, the userspace must take the following steps:
+
+1) Set up media links between entities: enable the links from the sensor to the
+   receiver and from the receiver to the DMA engines. This step does not differ
+   from normal non-multiplexed media controller setup.
+
+2) Configure routing
+
+.. flat-table:: Camera sensor. There are no configurable routes.
+    :header-rows: 1
+
+    * - Sink Pad/Stream
+      - Source Pad/Stream
+      - Routing Flags
+      - Comments
+    * - 1/0
+      - 0/0
+      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+      - Pixel data stream from the internal image sink pad
+    * - 2/0
+      - 0/1
+      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+      - Metadata stream from the internal embedded data sink pad
+
+The options available in the sensor's routing configuration are dictated by
+hardware capabilities: typically camera sensors always produce an image data
+stream while it may be possible to enable and disable the embedded data stream.
+
+.. flat-table:: Receiver routing table. Typically both routes need to be
+		explicitly set.
+    :header-rows:  1
+
+    * - Sink Pad/Stream
+      - Source Pad/Stream
+      - Routing Flags
+      - Comments
+    * - 0/0
+      - 1/0
+      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+      - Pixel data stream from camera sensor
+    * - 0/1
+      - 2/0
+      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+      - Metadata stream from camera sensor
+
+3) Configure formats and selections
+
+   This example assumes that the formats are propagated from sink pad to the
+   source pad as-is. The tables contain fields of both struct v4l2_subdev_format
+   and struct v4l2_mbus_framefmt.
+
+.. flat-table:: Formats set on the sub-devices. Bold values are set, others are
+                static or propagated. The order is aligned with configured
+                routes.
+    :header-rows: 1
+    :fill-cells:
+
+    * - Sub-device
+      - Pad/Stream
+      - Width
+      - Height
+      - Code
+    * - :rspan:`3` IMX219
+      - 1/0
+      - 3296
+      - 2480
+      - MEDIA_BUS_FMT_RAW_10
+    * - 0/0
+      - **3296**
+      - **2480**
+      - **MEDIA_BUS_FMT_RAW_10**
+    * - 2/0
+      - 3296
+      - 2
+      - MEDIA_BUS_FMT_META_10
+    * - 0/1
+      - 3296
+      - 2
+      - MEDIA_BUS_FMT_META_10
+    * - :rspan:`3` CSI-2 receiver
+      - 0/0
+      - **3296**
+      - **2480**
+      - **MEDIA_BUS_FMT_RAW_10**
+    * - 1/0
+      - 3296
+      - 2480
+      - MEDIA_BUS_FMT_RAW_10
+    * - 0/1
+      - **3296**
+      - **2**
+      - **MEDIA_BUS_FMT_META_10**
+    * - 2/0
+      - 3296
+      - 2
+      - MEDIA_BUS_FMT_META_10
+
+The embedded data format does not need to be configured on the sensor's pads as
+the format is dictated by the pixel data format in this case.
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index 64c0f9ff5b1b..d803a2f0f2f9 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -146,6 +146,8 @@ Image Source Control IDs
     ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
     ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
 
+.. _image_source_control_metadata_layout:
+
 ``V4L2_CID_METADATA_LAYOUT (integer)``
     The metadata layout control defines the on-bus metadata layout for metadata
     streams. The control is used in conjunction with :ref:`generic metadata
-- 
2.47.2

Re: [PATCH v11 25/66] media: Documentation: v4l: Document internal sink pads

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:26PM +0300, Sakari Ailus wrote:
> Document internal sink pads, pads that have both SINK and INTERNAL flags
> set. Use the IMX219 camera sensor as an example.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by Julien Massot <julien.massot@collabora.com>
> ---
>  .../userspace-api/media/v4l/dev-subdev.rst    | 151 ++++++++++++++++++
>  .../media/v4l/ext-ctrls-image-source.rst      |   2 +
>  2 files changed, 153 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index 4da67ee0b290..bb86cadfad1c 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the
>  sub-device and a (pad, stream) pair. For sub-devices that do not support
>  multiplexed streams the 'stream' field is always 0.
>
> +.. _v4l2-subdev-internal-source-pads:
> +
> +Internal sink pads and routing
> +------------------------------
> +
> +Cases where a single sub-device source pad is traversed by multiple streams, one
> +or more of which originate from within the sub-device itself, are special as
> +there is no external sink pad for such routes. In those cases, the sources of
> +the internally generated streams are represented by internal sink pads, which
> +are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`
> +pad flag set.
> +
> +Internal pads have all the properties of an external pad, including formats and
> +selections. The format in this case is the source format of the stream. An
> +internal pad always has a single stream only (0).
> +
> +Routes from an internal sink pad to an external source pad are typically not
> +modifiable but they can be activated and deactivated using the
> +:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag, depending
> +on driver capabilities.

Ah, so they are modifiable :)

What about

Routes from an internal sink pad to an external source pad are
typically created by the driver and can be activated and deactivated
using the :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE
<v4l2-subdev-routing-flags>` flag, depending on the device
capabilities.

> +
>  Interaction between routes, streams, formats and selections
>  -----------------------------------------------------------
>
> @@ -695,3 +716,133 @@ To configure this pipeline, the userspace must take the following steps:
>     the configurations along the stream towards the receiver, using
>     :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each
>     stream endpoint in each sub-device.
> +
> +   In case generic raw and metadata formats are used,
> +   :ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` and
> +   :ref:`V4L2_CID_METADATA_LAYOUT <image_source_control_metadata_layout>`
> +   controls are present on the source sub-device to obtain the pixel data color
> +   pattern and metadata layout.
> +
> +Internal pads setup example
> +---------------------------
> +
> +A simple example of a multiplexed stream setup might be as follows:
> +
> +- An IMX219 camera sensor source sub-device, with one source pad (0), one

s/sensor source/sensor/

Should we even mention imx219 or can this be a generic "RAW camera
sensor" ?


> +  internal sink pad (1) as the source of the image data and an internal sink
> +  pad (2) as the source of the embedded data. There are two routes, one from the

I would provide a rational for the reason why the external source pad
is preferably assigned to id 0.

   - A RAW camera sensor driver modeled as a single sub-device with
     three pads. The external source pas is assigned id #0 for
     compatibility reasons with existing user-space applications
     developed to work on drivers that pre-dates the introduction of
     internal pads, where the only available pad was the external
     source one. The sub-device also has two internal sink pads, pad
     #1 that represents the pixel array and produces image data and
     pad #2 that produces embedded data.

     There are two routes, one from the internal sink pad 1...

> +  internal sink pad 1 to the source pad 0 (image data) and another from the
> +  internal sink pad 2 to the source pad 0 (embedded data). Both streams are
> +  always active, i.e. there is no need to separately enable the embedded data
> +  stream. The sensor uses the CSI-2 interface.
> +
> +- A CSI-2 receiver in the SoC. The receiver has a single sink pad (pad 0),
> +  connected to the sensor, and two source pads (pads 1 and 2), to the two DMA
> +  engines. The receiver demultiplexes the incoming streams to the source pads.
> +
> +- DMA engines in the SoC, one for each stream. Each DMA engine is connected to a
> +  single source pad of the receiver.
> +
> +The sensor and the receiver are modelled as V4L2 sub-devices, exposed to
> +userspace via /dev/v4l-subdevX device nodes. The DMA engines are modelled as
> +V4L2 devices, exposed to userspace via /dev/videoX nodes.
> +
> +To configure this pipeline, the userspace must take the following steps:
> +
> +1) Set up media links between entities: enable the links from the sensor to the
> +   receiver and from the receiver to the DMA engines. This step does not differ
> +   from normal non-multiplexed media controller setup.
> +
> +2) Configure routing
> +
> +.. flat-table:: Camera sensor. There are no configurable routes.
> +    :header-rows: 1
> +
> +    * - Sink Pad/Stream
> +      - Source Pad/Stream
> +      - Routing Flags
> +      - Comments
> +    * - 1/0
> +      - 0/0
> +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE

Is this also V4L2_SUBDEV_ROUTE_FL_IMMUTABLE ?

> +      - Pixel data stream from the internal image sink pad
> +    * - 2/0
> +      - 0/1
> +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> +      - Metadata stream from the internal embedded data sink pad
> +
> +The options available in the sensor's routing configuration are dictated by
> +hardware capabilities: typically camera sensors always produce an image data

s/typically/some

> +stream while it may be possible to enable and disable the embedded data stream.

s/it may be possible/other might allow

if you accept my suggestion in the line above

> +
> +.. flat-table:: Receiver routing table. Typically both routes need to be
> +		explicitly set.

set or enabled ?

> +    :header-rows:  1
> +
> +    * - Sink Pad/Stream
> +      - Source Pad/Stream
> +      - Routing Flags
> +      - Comments
> +    * - 0/0
> +      - 1/0
> +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> +      - Pixel data stream from camera sensor
> +    * - 0/1
> +      - 2/0
> +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> +      - Metadata stream from camera sensor
> +
> +3) Configure formats and selections
> +
> +   This example assumes that the formats are propagated from sink pad to the
> +   source pad as-is. The tables contain fields of both struct v4l2_subdev_format
> +   and struct v4l2_mbus_framefmt.
> +
> +.. flat-table:: Formats set on the sub-devices. Bold values are set, others are
> +                static or propagated. The order is aligned with configured
> +                routes.
> +    :header-rows: 1
> +    :fill-cells:
> +
> +    * - Sub-device
> +      - Pad/Stream
> +      - Width
> +      - Height
> +      - Code
> +    * - :rspan:`3` IMX219

If you want to make this generic, I would replace IMX219 with just
"Sensor"

> +      - 1/0
> +      - 3296
> +      - 2480
> +      - MEDIA_BUS_FMT_RAW_10
> +    * - 0/0
> +      - **3296**
> +      - **2480**
> +      - **MEDIA_BUS_FMT_RAW_10**
> +    * - 2/0
> +      - 3296
> +      - 2
> +      - MEDIA_BUS_FMT_META_10
> +    * - 0/1
> +      - 3296
> +      - 2
> +      - MEDIA_BUS_FMT_META_10
> +    * - :rspan:`3` CSI-2 receiver
> +      - 0/0
> +      - **3296**
> +      - **2480**
> +      - **MEDIA_BUS_FMT_RAW_10**
> +    * - 1/0
> +      - 3296
> +      - 2480
> +      - MEDIA_BUS_FMT_RAW_10
> +    * - 0/1
> +      - **3296**
> +      - **2**
> +      - **MEDIA_BUS_FMT_META_10**
> +    * - 2/0
> +      - 3296
> +      - 2
> +      - MEDIA_BUS_FMT_META_10
> +
> +The embedded data format does not need to be configured on the sensor's pads as
> +the format is dictated by the pixel data format in this case.
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index 64c0f9ff5b1b..d803a2f0f2f9 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -146,6 +146,8 @@ Image Source Control IDs
>      ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
>      ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
>
> +.. _image_source_control_metadata_layout:
> +

All comments I made are just suggestions, so take in what you like.

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

>  ``V4L2_CID_METADATA_LAYOUT (integer)``
>      The metadata layout control defines the on-bus metadata layout for metadata
>      streams. The control is used in conjunction with :ref:`generic metadata
> --
> 2.47.2
>
>

Re: [PATCH v11 25/66] media: Documentation: v4l: Document internal sink pads

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 06:39:29PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:26PM +0300, Sakari Ailus wrote:
> > Document internal sink pads, pads that have both SINK and INTERNAL flags
> > set. Use the IMX219 camera sensor as an example.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by Julien Massot <julien.massot@collabora.com>
> > ---
> >  .../userspace-api/media/v4l/dev-subdev.rst    | 151 ++++++++++++++++++
> >  .../media/v4l/ext-ctrls-image-source.rst      |   2 +
> >  2 files changed, 153 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > index 4da67ee0b290..bb86cadfad1c 100644
> > --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > @@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the
> >  sub-device and a (pad, stream) pair. For sub-devices that do not support
> >  multiplexed streams the 'stream' field is always 0.
> >
> > +.. _v4l2-subdev-internal-source-pads:
> > +
> > +Internal sink pads and routing
> > +------------------------------
> > +
> > +Cases where a single sub-device source pad is traversed by multiple streams, one
> > +or more of which originate from within the sub-device itself, are special as
> > +there is no external sink pad for such routes. In those cases, the sources of
> > +the internally generated streams are represented by internal sink pads, which
> > +are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`
> > +pad flag set.
> > +
> > +Internal pads have all the properties of an external pad, including formats and
> > +selections. The format in this case is the source format of the stream. An
> > +internal pad always has a single stream only (0).
> > +
> > +Routes from an internal sink pad to an external source pad are typically not
> > +modifiable but they can be activated and deactivated using the
> > +:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag, depending
> > +on driver capabilities.
> 
> Ah, so they are modifiable :)
> 
> What about
> 
> Routes from an internal sink pad to an external source pad are
> typically created by the driver and can be activated and deactivated
> using the :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE
> <v4l2-subdev-routing-flags>` flag, depending on the device
> capabilities.

I'll use that.

> 
> > +
> >  Interaction between routes, streams, formats and selections
> >  -----------------------------------------------------------
> >
> > @@ -695,3 +716,133 @@ To configure this pipeline, the userspace must take the following steps:
> >     the configurations along the stream towards the receiver, using
> >     :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each
> >     stream endpoint in each sub-device.
> > +
> > +   In case generic raw and metadata formats are used,
> > +   :ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` and
> > +   :ref:`V4L2_CID_METADATA_LAYOUT <image_source_control_metadata_layout>`
> > +   controls are present on the source sub-device to obtain the pixel data color
> > +   pattern and metadata layout.
> > +
> > +Internal pads setup example
> > +---------------------------
> > +
> > +A simple example of a multiplexed stream setup might be as follows:
> > +
> > +- An IMX219 camera sensor source sub-device, with one source pad (0), one
> 
> s/sensor source/sensor/

Source sub-device is referred to in documentation elsewhere; I think it's
appropriate here, too.

> 
> Should we even mention imx219 or can this be a generic "RAW camera
> sensor" ?

As this is an example, I think it's a good idea to use an actual device and
a driver that has support for internal pads.

> 
> 
> > +  internal sink pad (1) as the source of the image data and an internal sink
> > +  pad (2) as the source of the embedded data. There are two routes, one from the
> 
> I would provide a rational for the reason why the external source pad
> is preferably assigned to id 0.
> 
>    - A RAW camera sensor driver modeled as a single sub-device with
>      three pads. The external source pas is assigned id #0 for
>      compatibility reasons with existing user-space applications
>      developed to work on drivers that pre-dates the introduction of
>      internal pads, where the only available pad was the external
>      source one. The sub-device also has two internal sink pads, pad
>      #1 that represents the pixel array and produces image data and
>      pad #2 that produces embedded data.

I'd put this to driver documentation as it's not part of the UAPI.

> 
>      There are two routes, one from the internal sink pad 1...
> 
> > +  internal sink pad 1 to the source pad 0 (image data) and another from the
> > +  internal sink pad 2 to the source pad 0 (embedded data). Both streams are
> > +  always active, i.e. there is no need to separately enable the embedded data
> > +  stream. The sensor uses the CSI-2 interface.
> > +
> > +- A CSI-2 receiver in the SoC. The receiver has a single sink pad (pad 0),
> > +  connected to the sensor, and two source pads (pads 1 and 2), to the two DMA
> > +  engines. The receiver demultiplexes the incoming streams to the source pads.
> > +
> > +- DMA engines in the SoC, one for each stream. Each DMA engine is connected to a
> > +  single source pad of the receiver.
> > +
> > +The sensor and the receiver are modelled as V4L2 sub-devices, exposed to
> > +userspace via /dev/v4l-subdevX device nodes. The DMA engines are modelled as
> > +V4L2 devices, exposed to userspace via /dev/videoX nodes.
> > +
> > +To configure this pipeline, the userspace must take the following steps:
> > +
> > +1) Set up media links between entities: enable the links from the sensor to the
> > +   receiver and from the receiver to the DMA engines. This step does not differ
> > +   from normal non-multiplexed media controller setup.
> > +
> > +2) Configure routing
> > +
> > +.. flat-table:: Camera sensor. There are no configurable routes.
> > +    :header-rows: 1
> > +
> > +    * - Sink Pad/Stream
> > +      - Source Pad/Stream
> > +      - Routing Flags
> > +      - Comments
> > +    * - 1/0
> > +      - 0/0
> > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> 
> Is this also V4L2_SUBDEV_ROUTE_FL_IMMUTABLE ?

Yes, and V4L2_SUBDEV_ROUTE_FL_STATIC. I'll update the example.

In practice the patch needs to be moved forwards in the set so we'll have
all the flags.

> 
> > +      - Pixel data stream from the internal image sink pad
> > +    * - 2/0
> > +      - 0/1
> > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > +      - Metadata stream from the internal embedded data sink pad
> > +
> > +The options available in the sensor's routing configuration are dictated by
> > +hardware capabilities: typically camera sensors always produce an image data
> 
> s/typically/some

"Some" would suggest this may not be very common whereas I think this
applies virtually to all camera sensors.

> 
> > +stream while it may be possible to enable and disable the embedded data stream.
> 
> s/it may be possible/other might allow
> 
> if you accept my suggestion in the line above
> 
> > +
> > +.. flat-table:: Receiver routing table. Typically both routes need to be
> > +		explicitly set.
> 
> set or enabled ?

On the receiver side, the routes are also created, not just enabled. I'll
use "created".

> 
> > +    :header-rows:  1
> > +
> > +    * - Sink Pad/Stream
> > +      - Source Pad/Stream
> > +      - Routing Flags
> > +      - Comments
> > +    * - 0/0
> > +      - 1/0
> > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > +      - Pixel data stream from camera sensor
> > +    * - 0/1
> > +      - 2/0
> > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > +      - Metadata stream from camera sensor
> > +
> > +3) Configure formats and selections
> > +
> > +   This example assumes that the formats are propagated from sink pad to the
> > +   source pad as-is. The tables contain fields of both struct v4l2_subdev_format
> > +   and struct v4l2_mbus_framefmt.
> > +
> > +.. flat-table:: Formats set on the sub-devices. Bold values are set, others are
> > +                static or propagated. The order is aligned with configured
> > +                routes.
> > +    :header-rows: 1
> > +    :fill-cells:
> > +
> > +    * - Sub-device
> > +      - Pad/Stream
> > +      - Width
> > +      - Height
> > +      - Code
> > +    * - :rspan:`3` IMX219
> 
> If you want to make this generic, I would replace IMX219 with just
> "Sensor"
> 
> > +      - 1/0
> > +      - 3296
> > +      - 2480
> > +      - MEDIA_BUS_FMT_RAW_10
> > +    * - 0/0
> > +      - **3296**
> > +      - **2480**
> > +      - **MEDIA_BUS_FMT_RAW_10**
> > +    * - 2/0
> > +      - 3296
> > +      - 2
> > +      - MEDIA_BUS_FMT_META_10
> > +    * - 0/1
> > +      - 3296
> > +      - 2
> > +      - MEDIA_BUS_FMT_META_10
> > +    * - :rspan:`3` CSI-2 receiver
> > +      - 0/0
> > +      - **3296**
> > +      - **2480**
> > +      - **MEDIA_BUS_FMT_RAW_10**
> > +    * - 1/0
> > +      - 3296
> > +      - 2480
> > +      - MEDIA_BUS_FMT_RAW_10
> > +    * - 0/1
> > +      - **3296**
> > +      - **2**
> > +      - **MEDIA_BUS_FMT_META_10**
> > +    * - 2/0
> > +      - 3296
> > +      - 2
> > +      - MEDIA_BUS_FMT_META_10
> > +
> > +The embedded data format does not need to be configured on the sensor's pads as
> > +the format is dictated by the pixel data format in this case.
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > index 64c0f9ff5b1b..d803a2f0f2f9 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > @@ -146,6 +146,8 @@ Image Source Control IDs
> >      ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> >      ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
> >
> > +.. _image_source_control_metadata_layout:
> > +
> 
> All comments I made are just suggestions, so take in what you like.
> 
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks!

> 
> >  ``V4L2_CID_METADATA_LAYOUT (integer)``
> >      The metadata layout control defines the on-bus metadata layout for metadata
> >      streams. The control is used in conjunction with :ref:`generic metadata

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v11 25/66] media: Documentation: v4l: Document internal sink pads

[Jacopo Mondi]

Hi Sakari

On Wed, Sep 03, 2025 at 03:17:29PM +0300, Sakari Ailus wrote:
> Hi Jacopo,
>
> On Mon, Sep 01, 2025 at 06:39:29PM +0200, Jacopo Mondi wrote:
> > Hi Sakari
> >
> > On Mon, Aug 25, 2025 at 12:50:26PM +0300, Sakari Ailus wrote:
> > > Document internal sink pads, pads that have both SINK and INTERNAL flags
> > > set. Use the IMX219 camera sensor as an example.
> > >
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > Reviewed-by Julien Massot <julien.massot@collabora.com>
> > > ---
> > >  .../userspace-api/media/v4l/dev-subdev.rst    | 151 ++++++++++++++++++
> > >  .../media/v4l/ext-ctrls-image-source.rst      |   2 +
> > >  2 files changed, 153 insertions(+)
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > > index 4da67ee0b290..bb86cadfad1c 100644
> > > --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > > +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > > @@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the
> > >  sub-device and a (pad, stream) pair. For sub-devices that do not support
> > >  multiplexed streams the 'stream' field is always 0.
> > >
> > > +.. _v4l2-subdev-internal-source-pads:
> > > +
> > > +Internal sink pads and routing
> > > +------------------------------
> > > +
> > > +Cases where a single sub-device source pad is traversed by multiple streams, one
> > > +or more of which originate from within the sub-device itself, are special as
> > > +there is no external sink pad for such routes. In those cases, the sources of
> > > +the internally generated streams are represented by internal sink pads, which
> > > +are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`
> > > +pad flag set.
> > > +
> > > +Internal pads have all the properties of an external pad, including formats and
> > > +selections. The format in this case is the source format of the stream. An
> > > +internal pad always has a single stream only (0).
> > > +
> > > +Routes from an internal sink pad to an external source pad are typically not
> > > +modifiable but they can be activated and deactivated using the
> > > +:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag, depending
> > > +on driver capabilities.
> >
> > Ah, so they are modifiable :)
> >
> > What about
> >
> > Routes from an internal sink pad to an external source pad are
> > typically created by the driver and can be activated and deactivated
> > using the :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > <v4l2-subdev-routing-flags>` flag, depending on the device
> > capabilities.
>
> I'll use that.
>
> >
> > > +
> > >  Interaction between routes, streams, formats and selections
> > >  -----------------------------------------------------------
> > >
> > > @@ -695,3 +716,133 @@ To configure this pipeline, the userspace must take the following steps:
> > >     the configurations along the stream towards the receiver, using
> > >     :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each
> > >     stream endpoint in each sub-device.
> > > +
> > > +   In case generic raw and metadata formats are used,
> > > +   :ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` and
> > > +   :ref:`V4L2_CID_METADATA_LAYOUT <image_source_control_metadata_layout>`
> > > +   controls are present on the source sub-device to obtain the pixel data color
> > > +   pattern and metadata layout.
> > > +
> > > +Internal pads setup example
> > > +---------------------------
> > > +
> > > +A simple example of a multiplexed stream setup might be as follows:
> > > +
> > > +- An IMX219 camera sensor source sub-device, with one source pad (0), one
> >
> > s/sensor source/sensor/
>
> Source sub-device is referred to in documentation elsewhere; I think it's
> appropriate here, too.
>
> >
> > Should we even mention imx219 or can this be a generic "RAW camera
> > sensor" ?
>
> As this is an example, I think it's a good idea to use an actual device and
> a driver that has support for internal pads.
>
> >
> >
> > > +  internal sink pad (1) as the source of the image data and an internal sink
> > > +  pad (2) as the source of the embedded data. There are two routes, one from the
> >
> > I would provide a rational for the reason why the external source pad
> > is preferably assigned to id 0.
> >
> >    - A RAW camera sensor driver modeled as a single sub-device with
> >      three pads. The external source pas is assigned id #0 for
> >      compatibility reasons with existing user-space applications
> >      developed to work on drivers that pre-dates the introduction of
> >      internal pads, where the only available pad was the external
> >      source one. The sub-device also has two internal sink pads, pad
> >      #1 that represents the pixel array and produces image data and
> >      pad #2 that produces embedded data.
>
> I'd put this to driver documentation as it's not part of the UAPI.
>

mmmm, it's true we don't mandate for new drivers to have
the source pad at index #0, and it's done only for existing drivers..
Adn userspace should not make assumptions about pad numbers but rather
inspect the per-pad flags to know which one is the external source pad.

So yes, it's no uAPI you're right.

> >
> >      There are two routes, one from the internal sink pad 1...
> >
> > > +  internal sink pad 1 to the source pad 0 (image data) and another from the
> > > +  internal sink pad 2 to the source pad 0 (embedded data). Both streams are
> > > +  always active, i.e. there is no need to separately enable the embedded data
> > > +  stream. The sensor uses the CSI-2 interface.
> > > +
> > > +- A CSI-2 receiver in the SoC. The receiver has a single sink pad (pad 0),
> > > +  connected to the sensor, and two source pads (pads 1 and 2), to the two DMA
> > > +  engines. The receiver demultiplexes the incoming streams to the source pads.
> > > +
> > > +- DMA engines in the SoC, one for each stream. Each DMA engine is connected to a
> > > +  single source pad of the receiver.
> > > +
> > > +The sensor and the receiver are modelled as V4L2 sub-devices, exposed to
> > > +userspace via /dev/v4l-subdevX device nodes. The DMA engines are modelled as
> > > +V4L2 devices, exposed to userspace via /dev/videoX nodes.
> > > +
> > > +To configure this pipeline, the userspace must take the following steps:
> > > +
> > > +1) Set up media links between entities: enable the links from the sensor to the
> > > +   receiver and from the receiver to the DMA engines. This step does not differ
> > > +   from normal non-multiplexed media controller setup.
> > > +
> > > +2) Configure routing
> > > +
> > > +.. flat-table:: Camera sensor. There are no configurable routes.
> > > +    :header-rows: 1
> > > +
> > > +    * - Sink Pad/Stream
> > > +      - Source Pad/Stream
> > > +      - Routing Flags
> > > +      - Comments
> > > +    * - 1/0
> > > +      - 0/0
> > > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> >
> > Is this also V4L2_SUBDEV_ROUTE_FL_IMMUTABLE ?
>
> Yes, and V4L2_SUBDEV_ROUTE_FL_STATIC. I'll update the example.
>
> In practice the patch needs to be moved forwards in the set so we'll have
> all the flags.
>

Yeah, I noticed later those two flags have been introduced later in
the series..

> >
> > > +      - Pixel data stream from the internal image sink pad
> > > +    * - 2/0
> > > +      - 0/1
> > > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > > +      - Metadata stream from the internal embedded data sink pad
> > > +
> > > +The options available in the sensor's routing configuration are dictated by
> > > +hardware capabilities: typically camera sensors always produce an image data
> >
> > s/typically/some
>
> "Some" would suggest this may not be very common whereas I think this
> applies virtually to all camera sensors.
>

Ok!

> >
> > > +stream while it may be possible to enable and disable the embedded data stream.
> >
> > s/it may be possible/other might allow
> >
> > if you accept my suggestion in the line above
> >
> > > +
> > > +.. flat-table:: Receiver routing table. Typically both routes need to be
> > > +		explicitly set.
> >
> > set or enabled ?
>
> On the receiver side, the routes are also created, not just enabled. I'll
> use "created".
>

fine as well

> >
> > > +    :header-rows:  1
> > > +
> > > +    * - Sink Pad/Stream
> > > +      - Source Pad/Stream
> > > +      - Routing Flags
> > > +      - Comments
> > > +    * - 0/0
> > > +      - 1/0
> > > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > > +      - Pixel data stream from camera sensor
> > > +    * - 0/1
> > > +      - 2/0
> > > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > > +      - Metadata stream from camera sensor
> > > +
> > > +3) Configure formats and selections
> > > +
> > > +   This example assumes that the formats are propagated from sink pad to the
> > > +   source pad as-is. The tables contain fields of both struct v4l2_subdev_format
> > > +   and struct v4l2_mbus_framefmt.
> > > +
> > > +.. flat-table:: Formats set on the sub-devices. Bold values are set, others are
> > > +                static or propagated. The order is aligned with configured
> > > +                routes.
> > > +    :header-rows: 1
> > > +    :fill-cells:
> > > +
> > > +    * - Sub-device
> > > +      - Pad/Stream
> > > +      - Width
> > > +      - Height
> > > +      - Code
> > > +    * - :rspan:`3` IMX219
> >
> > If you want to make this generic, I would replace IMX219 with just
> > "Sensor"
> >
> > > +      - 1/0
> > > +      - 3296
> > > +      - 2480
> > > +      - MEDIA_BUS_FMT_RAW_10
> > > +    * - 0/0
> > > +      - **3296**
> > > +      - **2480**
> > > +      - **MEDIA_BUS_FMT_RAW_10**
> > > +    * - 2/0
> > > +      - 3296
> > > +      - 2
> > > +      - MEDIA_BUS_FMT_META_10
> > > +    * - 0/1
> > > +      - 3296
> > > +      - 2
> > > +      - MEDIA_BUS_FMT_META_10
> > > +    * - :rspan:`3` CSI-2 receiver
> > > +      - 0/0
> > > +      - **3296**
> > > +      - **2480**
> > > +      - **MEDIA_BUS_FMT_RAW_10**
> > > +    * - 1/0
> > > +      - 3296
> > > +      - 2480
> > > +      - MEDIA_BUS_FMT_RAW_10
> > > +    * - 0/1
> > > +      - **3296**
> > > +      - **2**
> > > +      - **MEDIA_BUS_FMT_META_10**
> > > +    * - 2/0
> > > +      - 3296
> > > +      - 2
> > > +      - MEDIA_BUS_FMT_META_10
> > > +
> > > +The embedded data format does not need to be configured on the sensor's pads as
> > > +the format is dictated by the pixel data format in this case.
> > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > index 64c0f9ff5b1b..d803a2f0f2f9 100644
> > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > @@ -146,6 +146,8 @@ Image Source Control IDs
> > >      ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> > >      ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
> > >
> > > +.. _image_source_control_metadata_layout:
> > > +
> >
> > All comments I made are just suggestions, so take in what you like.
> >
> > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>
> Thanks!
>
> >
> > >  ``V4L2_CID_METADATA_LAYOUT (integer)``
> > >      The metadata layout control defines the on-bus metadata layout for metadata
> > >      streams. The control is used in conjunction with :ref:`generic metadata
>
> --
> Kind regards,
>
> Sakari Ailus

Re: [PATCH v11 25/66] media: Documentation: v4l: Document internal sink pads

[Sakari Ailus]

Hi Jacopo,

On Wed, Sep 03, 2025 at 02:29:09PM +0200, Jacopo Mondi wrote:
> > > > +  internal sink pad (1) as the source of the image data and an internal sink
> > > > +  pad (2) as the source of the embedded data. There are two routes, one from the
> > >
> > > I would provide a rational for the reason why the external source pad
> > > is preferably assigned to id 0.
> > >
> > >    - A RAW camera sensor driver modeled as a single sub-device with
> > >      three pads. The external source pas is assigned id #0 for
> > >      compatibility reasons with existing user-space applications
> > >      developed to work on drivers that pre-dates the introduction of
> > >      internal pads, where the only available pad was the external
> > >      source one. The sub-device also has two internal sink pads, pad
> > >      #1 that represents the pixel array and produces image data and
> > >      pad #2 that produces embedded data.
> >
> > I'd put this to driver documentation as it's not part of the UAPI.
> >
> 
> mmmm, it's true we don't mandate for new drivers to have
> the source pad at index #0, and it's done only for existing drivers..

I'd still say that's what we tend to do, for source sub-devices, but it's
not part of the UAPI. :-)

> Adn userspace should not make assumptions about pad numbers but rather
> inspect the per-pad flags to know which one is the external source pad.
> 
> So yes, it's no uAPI you're right.

I agree.

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v11 25/66] media: Documentation: v4l: Document internal sink pads

[Laurent Pinchart]

On Wed, Sep 03, 2025 at 02:29:09PM +0200, Jacopo Mondi wrote:
> On Wed, Sep 03, 2025 at 03:17:29PM +0300, Sakari Ailus wrote:
> > On Mon, Sep 01, 2025 at 06:39:29PM +0200, Jacopo Mondi wrote:
> > > On Mon, Aug 25, 2025 at 12:50:26PM +0300, Sakari Ailus wrote:
> > > > Document internal sink pads, pads that have both SINK and INTERNAL flags
> > > > set. Use the IMX219 camera sensor as an example.
> > > >
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > Reviewed-by Julien Massot <julien.massot@collabora.com>
> > > > ---
> > > >  .../userspace-api/media/v4l/dev-subdev.rst    | 151 ++++++++++++++++++
> > > >  .../media/v4l/ext-ctrls-image-source.rst      |   2 +
> > > >  2 files changed, 153 insertions(+)
> > > >
> > > > diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > > > index 4da67ee0b290..bb86cadfad1c 100644
> > > > --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> > > > @@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the
> > > >  sub-device and a (pad, stream) pair. For sub-devices that do not support
> > > >  multiplexed streams the 'stream' field is always 0.
> > > >
> > > > +.. _v4l2-subdev-internal-source-pads:
> > > > +
> > > > +Internal sink pads and routing
> > > > +------------------------------
> > > > +
> > > > +Cases where a single sub-device source pad is traversed by multiple streams, one

s/is traversed by/carries/

> > > > +or more of which originate from within the sub-device itself, are special as
> > > > +there is no external sink pad for such routes. In those cases, the sources of
> > > > +the internally generated streams are represented by internal sink pads, which
> > > > +are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`
> > > > +pad flag set.
> > > > +
> > > > +Internal pads have all the properties of an external pad, including formats and
> > > > +selections. The format in this case is the source format of the stream. An

I'd add "[...] and selections, but can not be connect to other subdevs
through links."

> > > > +internal pad always has a single stream only (0).

I already have one use case (related to HDR) that could violate this
rule. I'm fine with it for now, we can always relax it later, but I'd
like to know your opinion on the concept.

> > > > +
> > > > +Routes from an internal sink pad to an external source pad are typically not
> > > > +modifiable but they can be activated and deactivated using the
> > > > +:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag, depending
> > > > +on driver capabilities.
> > >
> > > Ah, so they are modifiable :)

I was going to mention the same :-)

> > > What about
> > >
> > > Routes from an internal sink pad to an external source pad are
> > > typically created by the driver and can be activated and deactivated
> > > using the :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > > <v4l2-subdev-routing-flags>` flag, depending on the device
> > > capabilities.

I'd drop "typically" as (I think) they are always created by the driver.
We could use the word "static" already.

Routes from an internal sink pad to an external source pad are
statically created by the driver. They are or are not immutable
depending on the device capabities. The routes that are not immutable
can be activated and deactivated using the
:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag.

(we could possibly drop the last sentence as it's implied by not being
immutable, but it doesn't hurt to have the information here)

> > I'll use that.
> >
> > > > +
> > > >  Interaction between routes, streams, formats and selections
> > > >  -----------------------------------------------------------
> > > >
> > > > @@ -695,3 +716,133 @@ To configure this pipeline, the userspace must take the following steps:
> > > >     the configurations along the stream towards the receiver, using
> > > >     :ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each
> > > >     stream endpoint in each sub-device.
> > > > +
> > > > +   In case generic raw and metadata formats are used,
> > > > +   :ref:`V4L2_CID_COLOR_PATTERN <image-source-control-color-pattern>` and
> > > > +   :ref:`V4L2_CID_METADATA_LAYOUT <image_source_control_metadata_layout>`
> > > > +   controls are present on the source sub-device to obtain the pixel data color
> > > > +   pattern and metadata layout.

s/pixel data color pattern/pixel array CFA pattern/

(or you can spell out color filter array)

> > > > +
> > > > +Internal pads setup example
> > > > +---------------------------
> > > > +
> > > > +A simple example of a multiplexed stream setup might be as follows:
> > > > +
> > > > +- An IMX219 camera sensor source sub-device, with one source pad (0), one
> > >
> > > s/sensor source/sensor/
> >
> > Source sub-device is referred to in documentation elsewhere; I think it's
> > appropriate here, too.
> >
> > > Should we even mention imx219 or can this be a generic "RAW camera
> > > sensor" ?
> >
> > As this is an example, I think it's a good idea to use an actual device and
> > a driver that has support for internal pads.
> >
> > > > +  internal sink pad (1) as the source of the image data and an internal sink
> > > > +  pad (2) as the source of the embedded data. There are two routes, one from the
> > >
> > > I would provide a rational for the reason why the external source pad
> > > is preferably assigned to id 0.
> > >
> > >    - A RAW camera sensor driver modeled as a single sub-device with
> > >      three pads. The external source pas is assigned id #0 for
> > >      compatibility reasons with existing user-space applications
> > >      developed to work on drivers that pre-dates the introduction of
> > >      internal pads, where the only available pad was the external
> > >      source one. The sub-device also has two internal sink pads, pad
> > >      #1 that represents the pixel array and produces image data and
> > >      pad #2 that produces embedded data.
> >
> > I'd put this to driver documentation as it's not part of the UAPI.
> 
> mmmm, it's true we don't mandate for new drivers to have
> the source pad at index #0, and it's done only for existing drivers..
> Adn userspace should not make assumptions about pad numbers but rather
> inspect the per-pad flags to know which one is the external source pad.
> 
> So yes, it's no uAPI you're right.

That may be possible for camera sensors, but in general the pad numbers
may be part of the UAPI as there may not be any other way to tell pads
apart on some subdevs.

For camera sensors, I would document the pad numbers explicitly in the
raw camera sensor subdev configuration model, and make them part of the
UAPI. I wouldn't explain the rationale here.

> > >
> > >      There are two routes, one from the internal sink pad 1...
> > >
> > > > +  internal sink pad 1 to the source pad 0 (image data) and another from the
> > > > +  internal sink pad 2 to the source pad 0 (embedded data). Both streams are
> > > > +  always active, i.e. there is no need to separately enable the embedded data
> > > > +  stream. The sensor uses the CSI-2 interface.
> > > > +
> > > > +- A CSI-2 receiver in the SoC. The receiver has a single sink pad (pad 0),
> > > > +  connected to the sensor, and two source pads (pads 1 and 2), to the two DMA

s/the sensor/the sensor's source pad (0)/
s/the two/two/

> > > > +  engines. The receiver demultiplexes the incoming streams to the source pads.
> > > > +
> > > > +- DMA engines in the SoC, one for each stream. Each DMA engine is connected to a

s/DMA engines/Two DMA engines/

> > > > +  single source pad of the receiver.

s/single/different/

> > > > +
> > > > +The sensor and the receiver are modelled as V4L2 sub-devices, exposed to
> > > > +userspace via /dev/v4l-subdevX device nodes. The DMA engines are modelled as
> > > > +V4L2 devices, exposed to userspace via /dev/videoX nodes.

s/V4L2 devices/V4L2 video devices/

> > > > +
> > > > +To configure this pipeline, the userspace must take the following steps:
> > > > +
> > > > +1) Set up media links between entities: enable the links from the sensor to the
> > > > +   receiver and from the receiver to the DMA engines. This step does not differ
> > > > +   from normal non-multiplexed media controller setup.
> > > > +
> > > > +2) Configure routing
> > > > +
> > > > +.. flat-table:: Camera sensor. There are no configurable routes.
> > > > +    :header-rows: 1
> > > > +
> > > > +    * - Sink Pad/Stream
> > > > +      - Source Pad/Stream
> > > > +      - Routing Flags
> > > > +      - Comments
> > > > +    * - 1/0
> > > > +      - 0/0
> > > > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > >
> > > Is this also V4L2_SUBDEV_ROUTE_FL_IMMUTABLE ?
> >
> > Yes, and V4L2_SUBDEV_ROUTE_FL_STATIC. I'll update the example.
> >
> > In practice the patch needs to be moved forwards in the set so we'll have
> > all the flags.
> 
> Yeah, I noticed later those two flags have been introduced later in
> the series..
> 
> > > > +      - Pixel data stream from the internal image sink pad
> > > > +    * - 2/0
> > > > +      - 0/1
> > > > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE

I assume this one is also static and immutable.

> > > > +      - Metadata stream from the internal embedded data sink pad
> > > > +
> > > > +The options available in the sensor's routing configuration are dictated by
> > > > +hardware capabilities: typically camera sensors always produce an image data
> > >
> > > s/typically/some
> >
> > "Some" would suggest this may not be very common whereas I think this
> > applies virtually to all camera sensors.
> 
> Ok!
> 
> > > > +stream while it may be possible to enable and disable the embedded data stream.

The section provides one particular example, based on the imx219, and
then this paragraph documents a more generic concept. It's unclear if it
applies to the imx219 as well. If you want to keep it, I would phrase it
along the lines of

While both routes are immutable for the IMX219, other camera sensors may
offer more flexible configuration options. Routing configuration is
dictated by the hardware capabilities: camera sensors typically always
produce an image data stream, but some of them support enabling and
disabling the embedded data stream.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> > >
> > > s/it may be possible/other might allow
> > >
> > > if you accept my suggestion in the line above
> > >
> > > > +
> > > > +.. flat-table:: Receiver routing table. Typically both routes need to be
> > > > +		explicitly set.
> > >
> > > set or enabled ?
> >
> > On the receiver side, the routes are also created, not just enabled. I'll
> > use "created".
> 
> fine as well
> 
> > > > +    :header-rows:  1
> > > > +
> > > > +    * - Sink Pad/Stream
> > > > +      - Source Pad/Stream
> > > > +      - Routing Flags
> > > > +      - Comments
> > > > +    * - 0/0
> > > > +      - 1/0
> > > > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > > > +      - Pixel data stream from camera sensor
> > > > +    * - 0/1
> > > > +      - 2/0
> > > > +      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
> > > > +      - Metadata stream from camera sensor
> > > > +
> > > > +3) Configure formats and selections
> > > > +
> > > > +   This example assumes that the formats are propagated from sink pad to the
> > > > +   source pad as-is. The tables contain fields of both struct v4l2_subdev_format
> > > > +   and struct v4l2_mbus_framefmt.
> > > > +
> > > > +.. flat-table:: Formats set on the sub-devices. Bold values are set, others are
> > > > +                static or propagated. The order is aligned with configured
> > > > +                routes.
> > > > +    :header-rows: 1
> > > > +    :fill-cells:
> > > > +
> > > > +    * - Sub-device
> > > > +      - Pad/Stream
> > > > +      - Width
> > > > +      - Height
> > > > +      - Code
> > > > +    * - :rspan:`3` IMX219
> > >
> > > If you want to make this generic, I would replace IMX219 with just
> > > "Sensor"
> > >
> > > > +      - 1/0
> > > > +      - 3296
> > > > +      - 2480
> > > > +      - MEDIA_BUS_FMT_RAW_10
> > > > +    * - 0/0
> > > > +      - **3296**
> > > > +      - **2480**
> > > > +      - **MEDIA_BUS_FMT_RAW_10**
> > > > +    * - 2/0
> > > > +      - 3296
> > > > +      - 2
> > > > +      - MEDIA_BUS_FMT_META_10
> > > > +    * - 0/1
> > > > +      - 3296
> > > > +      - 2
> > > > +      - MEDIA_BUS_FMT_META_10
> > > > +    * - :rspan:`3` CSI-2 receiver
> > > > +      - 0/0
> > > > +      - **3296**
> > > > +      - **2480**
> > > > +      - **MEDIA_BUS_FMT_RAW_10**
> > > > +    * - 1/0
> > > > +      - 3296
> > > > +      - 2480
> > > > +      - MEDIA_BUS_FMT_RAW_10
> > > > +    * - 0/1
> > > > +      - **3296**
> > > > +      - **2**
> > > > +      - **MEDIA_BUS_FMT_META_10**
> > > > +    * - 2/0
> > > > +      - 3296
> > > > +      - 2
> > > > +      - MEDIA_BUS_FMT_META_10
> > > > +
> > > > +The embedded data format does not need to be configured on the sensor's pads as
> > > > +the format is dictated by the pixel data format in this case.
> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > index 64c0f9ff5b1b..d803a2f0f2f9 100644
> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > > > @@ -146,6 +146,8 @@ Image Source Control IDs
> > > >      ``V4L2_COLOR_PATTERN_FLIP_HORIZONTAL`` and
> > > >      ``V4L2_COLOR_PATTERN_FLIP_VERTICAL`` is provided as well.
> > > >
> > > > +.. _image_source_control_metadata_layout:
> > > > +
> > >
> > > All comments I made are just suggestions, so take in what you like.
> > >
> > > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> >
> > Thanks!
> >
> > > >  ``V4L2_CID_METADATA_LAYOUT (integer)``
> > > >      The metadata layout control defines the on-bus metadata layout for metadata
> > > >      streams. The control is used in conjunction with :ref:`generic metadata

-- 
Regards,

Laurent Pinchart

[PATCH v11 26/66] media: Documentation: Document embedded data guidelines for camera sensors

[Sakari Ailus]

Document how embedded data support should be implemented for camera
sensors, and when and how CCS embedded data layout should be referenced.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Julien Massot <julien.massot@collabora.com>
---
 .../media/drivers/camera-sensor.rst           | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index a8e0e4accbc9..a719c6f88f11 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -117,3 +117,23 @@ register programming sequences shall initialize the :ref:`V4L2_CID_HFLIP
 values programmed by the register sequences. The default values of these
 controls shall be 0 (disabled). Especially these controls shall not be inverted,
 independently of the sensor's mounting rotation.
+
+Embedded data
+-------------
+
+Many sensors, mostly raw sensors, support embedded data which is used to convey
+the sensor configuration for the captured frame back to the host. While CSI-2 is
+the most common data interface used by such sensors, embedded data can be
+available on other interfaces as well.
+
+Embedded data support is indicated by the presence of an internal sink pad (pad
+that has both the ``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and
+``MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`` flags set) with a metadata
+format to model the embedded data stream. If the sub-device driver supports
+disabling embedded data, this can be done by disabling the embedded data route
+via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL.
+
+In general, changing the embedded data layout from the driver-configured values
+is not supported. The height of the metadata is device-specific and the width
+is that (or less of that) of the image width, as configured on the pixel data
+stream.
-- 
2.47.2

Re: [PATCH v11 26/66] media: Documentation: Document embedded data guidelines for camera sensors

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:27PM +0300, Sakari Ailus wrote:
> Document how embedded data support should be implemented for camera
> sensors, and when and how CCS embedded data layout should be referenced.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Julien Massot <julien.massot@collabora.com>
> ---
>  .../media/drivers/camera-sensor.rst           | 20 +++++++++++++++++++
>  1 file changed, 20 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index a8e0e4accbc9..a719c6f88f11 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -117,3 +117,23 @@ register programming sequences shall initialize the :ref:`V4L2_CID_HFLIP
>  values programmed by the register sequences. The default values of these
>  controls shall be 0 (disabled). Especially these controls shall not be inverted,
>  independently of the sensor's mounting rotation.
> +
> +Embedded data
> +-------------
> +
> +Many sensors, mostly raw sensors, support embedded data which is used to convey
> +the sensor configuration for the captured frame back to the host. While CSI-2 is
> +the most common data interface used by such sensors, embedded data can be
> +available on other interfaces as well.
> +
> +Embedded data support is indicated by the presence of an internal sink pad (pad
> +that has both the ``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and
> +``MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`` flags set) with a metadata

These are rendered as verbatim text, not links

> +format to model the embedded data stream. If the sub-device driver supports
> +disabling embedded data, this can be done by disabling the embedded data route
> +via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL.
> +
> +In general, changing the embedded data layout from the driver-configured values
> +is not supported. The height of the metadata is device-specific and the width
> +is that (or less of that) of the image width, as configured on the pixel data

s/is that (or less of that) of/is the same (or smaller) than/

reads better in my possibily broke English


> +stream.

With the links fixed
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j

> --
> 2.47.2
>
>

Re: [PATCH v11 26/66] media: Documentation: Document embedded data guidelines for camera sensors

[Sakari Ailus]

Hi Jacopo,

On Mon, Sep 01, 2025 at 06:42:33PM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Mon, Aug 25, 2025 at 12:50:27PM +0300, Sakari Ailus wrote:
> > Document how embedded data support should be implemented for camera
> > sensors, and when and how CCS embedded data layout should be referenced.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Julien Massot <julien.massot@collabora.com>
> > ---
> >  .../media/drivers/camera-sensor.rst           | 20 +++++++++++++++++++
> >  1 file changed, 20 insertions(+)
> >
> > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > index a8e0e4accbc9..a719c6f88f11 100644
> > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > @@ -117,3 +117,23 @@ register programming sequences shall initialize the :ref:`V4L2_CID_HFLIP
> >  values programmed by the register sequences. The default values of these
> >  controls shall be 0 (disabled). Especially these controls shall not be inverted,
> >  independently of the sensor's mounting rotation.
> > +
> > +Embedded data
> > +-------------
> > +
> > +Many sensors, mostly raw sensors, support embedded data which is used to convey
> > +the sensor configuration for the captured frame back to the host. While CSI-2 is
> > +the most common data interface used by such sensors, embedded data can be
> > +available on other interfaces as well.
> > +
> > +Embedded data support is indicated by the presence of an internal sink pad (pad
> > +that has both the ``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and
> > +``MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`` flags set) with a metadata
> 
> These are rendered as verbatim text, not links

Will fix for v12.

> 
> > +format to model the embedded data stream. If the sub-device driver supports
> > +disabling embedded data, this can be done by disabling the embedded data route
> > +via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL.
> > +
> > +In general, changing the embedded data layout from the driver-configured values
> > +is not supported. The height of the metadata is device-specific and the width
> > +is that (or less of that) of the image width, as configured on the pixel data
> 
> s/is that (or less of that) of/is the same (or smaller) than/
> 
> reads better in my possibily broke English

How about:

The height of the metadata is device-specific and equal to or less than the
image width, as configured on the pixel data stream.

> With the links fixed
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thank you.

-- 
Sakari Ailus

Re: [PATCH v11 26/66] media: Documentation: Document embedded data guidelines for camera sensors

[Laurent Pinchart]

On Wed, Sep 03, 2025 at 04:37:12PM +0300, Sakari Ailus wrote:
> On Mon, Sep 01, 2025 at 06:42:33PM +0200, Jacopo Mondi wrote:
> > On Mon, Aug 25, 2025 at 12:50:27PM +0300, Sakari Ailus wrote:
> > > Document how embedded data support should be implemented for camera
> > > sensors, and when and how CCS embedded data layout should be referenced.
> > >
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > Reviewed-by: Julien Massot <julien.massot@collabora.com>
> > > ---
> > >  .../media/drivers/camera-sensor.rst           | 20 +++++++++++++++++++
> > >  1 file changed, 20 insertions(+)
> > >
> > > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > index a8e0e4accbc9..a719c6f88f11 100644
> > > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > > @@ -117,3 +117,23 @@ register programming sequences shall initialize the :ref:`V4L2_CID_HFLIP
> > >  values programmed by the register sequences. The default values of these
> > >  controls shall be 0 (disabled). Especially these controls shall not be inverted,
> > >  independently of the sensor's mounting rotation.
> > > +
> > > +Embedded data
> > > +-------------
> > > +
> > > +Many sensors, mostly raw sensors, support embedded data which is used to convey
> > > +the sensor configuration for the captured frame back to the host. While CSI-2 is
> > > +the most common data interface used by such sensors, embedded data can be
> > > +available on other interfaces as well.
> > > +
> > > +Embedded data support is indicated by the presence of an internal sink pad (pad
> > > +that has both the ``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and
> > > +``MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`` flags set) with a metadata
> > 
> > These are rendered as verbatim text, not links
> 
> Will fix for v12.
> 
> > > +format to model the embedded data stream. If the sub-device driver supports
> > > +disabling embedded data, this can be done by disabling the embedded data route
> > > +via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL.
> > > +
> > > +In general, changing the embedded data layout from the driver-configured values
> > > +is not supported. The height of the metadata is device-specific and the width
> > > +is that (or less of that) of the image width, as configured on the pixel data
> > 
> > s/is that (or less of that) of/is the same (or smaller) than/
> > 
> > reads better in my possibily broke English
> 
> How about:
> 
> The height of the metadata is device-specific and equal to or less than the
> image width, as configured on the pixel data stream.

That doesn't sound right. "The *height* is [...] than the image
*width*". I'll give it a try:

----
In general, changing the embedded data layout from the driver-configured
values is not supported. The height of the metadata is typically a small
fixed device-specific value, and the width is equal to the image width,
as configured on the pixel data.
----

That drops the "or less" though. When is the embedded data width smaller
than the image data width, and is it configurable in that case, or does
it have a fixed value that depends on the image data width ?

> > With the links fixed
> > Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> 
> Thank you.

-- 
Regards,

Laurent Pinchart

[PATCH v11 27/66] media: uapi: ccs: Add metadata layout for MIPI CCS embedded data

[Sakari Ailus]

Add new metadata layout (V4L2_METADATA_LAYOUT_CCS) for MIPI CCS embedded
data.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Julien Massot <julien.massot@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 .../media/v4l/ext-ctrls-image-source.rst      | 12 ++++++
 .../userspace-api/media/v4l/meta-formats.rst  |  1 +
 .../media/v4l/metadata-layouts.rst            | 39 +++++++++++++++++++
 .../media/v4l/subdev-formats.rst              |  5 +++
 include/uapi/linux/v4l2-controls.h            |  1 +
 5 files changed, 58 insertions(+)
 create mode 100644 Documentation/userspace-api/media/v4l/metadata-layouts.rst

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index d803a2f0f2f9..371c155a88f7 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -153,3 +153,15 @@ Image Source Control IDs
     streams. The control is used in conjunction with :ref:`generic metadata
     formats <media-bus-format-generic-meta>` formats to specify the layout of the
     data.
+
+.. flat-table:: V4L2 metadata layouts
+    :header-rows:  1
+    :stub-columns: 0
+    :widths:       1 2 1
+
+    * - Macro name
+      - Synopsis
+      - Value
+    * - V4L2_METADATA_LAYOUT_CCS
+      - :ref:`CCS embedded data <media-metadata-layout-ccs>`
+      - 1
diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
index 0de80328c36b..542fbecab2dc 100644
--- a/Documentation/userspace-api/media/v4l/meta-formats.rst
+++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
@@ -15,6 +15,7 @@ These formats are used for the :ref:`metadata` interface only.
     metafmt-c3-isp
     metafmt-d4xx
     metafmt-generic
+    metadata-layouts
     metafmt-intel-ipu3
     metafmt-pisp-be
     metafmt-pisp-fe
diff --git a/Documentation/userspace-api/media/v4l/metadata-layouts.rst b/Documentation/userspace-api/media/v4l/metadata-layouts.rst
new file mode 100644
index 000000000000..5f717d54d1ca
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/metadata-layouts.rst
@@ -0,0 +1,39 @@
+.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
+
+.. _media_metadata_layouts:
+
+Metadata Layouts
+----------------
+
+The :ref:`metadata layout control <image_source_control_metadata_layout>`
+specifies the on-bus layout of the metadata on pads with a :ref:`generic
+metadata mbus code <media-bus-format-generic-meta>` independently of the bit
+depth.
+
+.. _media-metadata-layout-ccs:
+
+MIPI CCS Embedded Data Layout (``V4L2_METADATA_LAYOUT_CCS``)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+`MIPI CCS <https://www.mipi.org/specifications/camera-command-set>`_ defines a
+metadata layout for sensor embedded data, identified by
+``V4L2_CID_METADATA_LAYOUT`` control value ``V4L2_METADATA_LAYOUT_CCS``, which
+is used to store the register configuration used for capturing a given
+frame. The layout itself is defined in the CCS specification.
+
+The CCS embedded data format (code ``0xa``) definition includes three levels:
+
+1. Padding within CSI-2 bus :term:`Data Unit` as documented in the MIPI CCS
+   specification.
+
+2. The tagged data format as documented in the MIPI CCS specification.
+
+3. Register addresses and register documentation as documented in the MIPI CCS
+   specification.
+
+The ``V4L2_METADATA_LAYOUT_CCS`` metadata layout value shall be used only by
+devices that fulfill all three levels above.
+
+This metadata layout code is only used for "2-byte simplified tagged data
+format" (code ``0xa``) but their use may be extended further in the future, to
+cover other CCS embedded data format codes.
diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst
index c36cd42b09c6..f027a37f1774 100644
--- a/Documentation/userspace-api/media/v4l/subdev-formats.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst
@@ -8503,6 +8503,11 @@ content is more or less device specific but the data is transmitted and received
 by multiple devices that do not process the data in any way, simply writing
 it to system memory for processing in software at the end of the pipeline.
 
+The exact layout of the data generated by the device is reported by the
+:ref:`V4L2_CID_METADATA_LAYOUT <image_source_control_metadata_layout>`
+control. For in-memory formats, :ref:`Generic line-based metadata formats
+<v4l2-format-generic-meta>` are used.
+
 "b" in an array cell signifies a byte of data, followed by the number of the bit
 and finally the bit number in subscript. "x" indicates a padding bit.
 
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 74e22a930be4..d31cd22d1fa9 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1240,6 +1240,7 @@ enum v4l2_jpeg_chroma_subsampling {
 	(V4L2_COLOR_PATTERN_FLIP_HORIZONTAL | V4L2_COLOR_PATTERN_FLIP_VERTICAL)
 
 #define V4L2_CID_METADATA_LAYOUT		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 12)
+#define V4L2_METADATA_LAYOUT_CCS		1U
 
 
 /* Image processing controls */
-- 
2.47.2

[PATCH v11 28/66] media: Documentation: Document non-CCS use of CCS embedded data layout

[Sakari Ailus]

The CCS embedded data layout has multiple aspects (packing, encoding and
the rest, including register addresses and semantics). Explicitly allow
non-compliant embedded data to use the two former to reduce redundant
documentation.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 .../userspace-api/media/drivers/camera-sensor.rst    | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index a719c6f88f11..cbbfbb0d8273 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -137,3 +137,15 @@ In general, changing the embedded data layout from the driver-configured values
 is not supported. The height of the metadata is device-specific and the width
 is that (or less of that) of the image width, as configured on the pixel data
 stream.
+
+CCS and non-CCS embedded data layout
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Embedded data which is fully compliant with CCS definitions uses ``CCS embedded
+data layout <media-metadata-layout-ccs>`` (level 3) for :ref:`the metadata
+layout control <image_source_control_metadata_layout>`. Device-specific embedded
+data compliant with either MIPI CCS embedded data levels 1 or 2 only shall not
+use CCS embedded data mbus code, but may refer to CCS embedded data
+documentation with the level of conformance specified and omit documenting these
+aspects of the layout. The rest of the device-specific embedded data layout is
+documented in the context of the data layout itself.
-- 
2.47.2

[PATCH v11 29/66] media: ccs: Add support for embedded data stream

[Sakari Ailus]

Add support for embedded data stream, in UAPI and frame descriptor.

This patch adds also a new embedded data pad (2) to the source sub-device.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 383 +++++++++++++++++++++++++++++--
 drivers/media/i2c/ccs/ccs.h      |  17 +-
 2 files changed, 375 insertions(+), 25 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 109b74476b8c..5c53651f82df 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -833,7 +833,7 @@ static int ccs_init_controls(struct ccs_sensor *sensor)
 	struct v4l2_fwnode_device_properties props;
 	int rval;
 
-	rval = v4l2_ctrl_handler_init(&sensor->pixel_array->ctrl_handler, 19);
+	rval = v4l2_ctrl_handler_init(&sensor->pixel_array->ctrl_handler, 22);
 	if (rval)
 		return rval;
 
@@ -1028,6 +1028,23 @@ static int ccs_init_controls(struct ccs_sensor *sensor)
 
 	v4l2_ctrl_cluster(2, &sensor->hflip);
 
+	v4l2_ctrl_new_std(&sensor->pixel_array->ctrl_handler, NULL,
+			  V4L2_CID_COLOR_PATTERN, sensor->default_pixel_order,
+			  sensor->default_pixel_order, 1,
+			  sensor->default_pixel_order);
+
+	v4l2_ctrl_new_std(&sensor->pixel_array->ctrl_handler, NULL,
+			  V4L2_CID_COLOR_PATTERN_FLIP,
+			  V4L2_COLOR_PATTERN_FLIP_BOTH,
+			  V4L2_COLOR_PATTERN_FLIP_BOTH, 1,
+			  V4L2_COLOR_PATTERN_FLIP_BOTH);
+
+	v4l2_ctrl_new_std(&sensor->pixel_array->ctrl_handler, NULL,
+			  V4L2_CID_METADATA_LAYOUT,
+			  V4L2_METADATA_LAYOUT_CCS,
+			  V4L2_METADATA_LAYOUT_CCS, 1,
+			  V4L2_METADATA_LAYOUT_CCS);
+
 	rval = v4l2_ctrl_handler_init(&sensor->src->ctrl_handler, 0);
 	if (rval)
 		return rval;
@@ -1899,6 +1916,13 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 	if (rval < 0)
 		goto err_pm_put;
 
+	/* Configure embedded data */
+	if (sensor->csi_format->compressed >= 16) {
+		rval = ccs_write(sensor, EMB_DATA_CTRL, sensor->emb_data_ctrl);
+		if (rval < 0)
+			goto err_pm_put;
+	}
+
 	if (CCS_LIM(sensor, FLASH_MODE_CAPABILITY) &
 	    (CCS_FLASH_MODE_CAPABILITY_SINGLE_STROBE |
 	     SMIAPP_FLASH_MODE_CAPABILITY_MULTIPLE_STROBE) &&
@@ -2018,6 +2042,57 @@ static const struct ccs_csi_data_format
 	return sensor->csi_format;
 }
 
+#define CCS_EMBEDDED_CODE_DEPTH(depth, half_depth)			\
+	depth,								\
+	CCS_EMB_DATA_CAPABILITY_TWO_BYTES_PER_RAW##depth,		\
+	CCS_EMB_DATA_CAPABILITY_NO_ONE_BYTE_PER_RAW##depth,		\
+	CCS_EMB_DATA_CTRL_RAW##half_depth##_PACKING_FOR_RAW##depth,	\
+	MEDIA_BUS_FMT_META_##half_depth,				\
+	MEDIA_BUS_FMT_META_##depth,					\
+
+static const struct ccs_embedded_code {
+	u8 depth;
+	u8 cap_two_bytes_per_sample;
+	u8 cap_no_legacy;
+	u8 ctrl;
+	u32 code_two_bytes;
+	u32 code_legacy;
+} ccs_embedded_codes[] = {
+	{ CCS_EMBEDDED_CODE_DEPTH(16, 8) },
+	{ CCS_EMBEDDED_CODE_DEPTH(20, 10) },
+	{ CCS_EMBEDDED_CODE_DEPTH(24, 12) },
+};
+
+static const struct ccs_embedded_code *ccs_embedded_code(unsigned int bpp)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(ccs_embedded_codes); i++)
+		if (ccs_embedded_codes[i].depth == bpp)
+			return ccs_embedded_codes + i;
+
+	WARN_ON(1);
+
+	return ccs_embedded_codes;
+}
+
+static u32
+ccs_default_embedded_code(struct ccs_sensor *sensor,
+			  const struct ccs_embedded_code *embedded_code)
+{
+	if (CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
+	    BIT(embedded_code->cap_two_bytes_per_sample))
+		return embedded_code->code_two_bytes;
+
+	if (!(CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
+	      BIT(embedded_code->cap_no_legacy)))
+		return embedded_code->code_legacy;
+
+	WARN_ON(1);
+
+	return embedded_code->code_legacy;
+}
+
 static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_mbus_code_enum *code)
@@ -2033,6 +2108,61 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,
 	dev_err(&client->dev, "subdev %s, pad %u, index %u\n",
 		subdev->name, code->pad, code->index);
 
+	if (subdev == &sensor->src->sd) {
+		if (code->pad == CCS_PAD_META ||
+		    code->stream == CCS_STREAM_META) {
+			struct v4l2_mbus_framefmt *pix_fmt =
+				v4l2_subdev_state_get_format(sd_state,
+							     CCS_PAD_SRC,
+							     CCS_STREAM_PIXEL);
+			const struct ccs_csi_data_format *csi_format =
+				ccs_validate_csi_data_format(sensor,
+							     pix_fmt->code);
+			unsigned int i = 0;
+			u32 codes[2];
+
+			switch (csi_format->compressed) {
+			case 8:
+				codes[i++] = MEDIA_BUS_FMT_META_8;
+				break;
+			case 10:
+				codes[i++] = MEDIA_BUS_FMT_META_10;
+				break;
+			case 12:
+				codes[i++] = MEDIA_BUS_FMT_META_12;
+				break;
+			case 14:
+				codes[i++] = MEDIA_BUS_FMT_META_14;
+				break;
+			case 16:
+			case 20:
+			case 24: {
+				const struct ccs_embedded_code *embedded_code =
+					ccs_embedded_code(csi_format->compressed);
+
+				if (CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
+				    BIT(embedded_code->cap_two_bytes_per_sample))
+					codes[i++] =
+						embedded_code->code_two_bytes;
+
+				if (!(CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
+				      BIT(embedded_code->cap_no_legacy)))
+					codes[i++] = embedded_code->code_legacy;
+				break;
+			}
+			default:
+				WARN_ON(1);
+			}
+
+			if (WARN_ON(i > ARRAY_SIZE(codes)) || code->index >= i)
+				goto out;
+
+			code->code = codes[code->index];
+			rval = 0;
+			goto out;
+		}
+	}
+
 	if (subdev != &sensor->src->sd || code->pad != CCS_PAD_SRC) {
 		if (code->index)
 			goto out;
@@ -2075,8 +2205,11 @@ static int __ccs_get_format(struct v4l2_subdev *subdev,
 			    struct v4l2_subdev_state *sd_state,
 			    struct v4l2_subdev_format *fmt)
 {
-	fmt->format = *v4l2_subdev_state_get_format(sd_state, fmt->pad);
-	fmt->format.code = __ccs_get_mbus_code(subdev, fmt->pad);
+	fmt->format = *v4l2_subdev_state_get_format(sd_state, fmt->pad,
+						    fmt->stream);
+
+	if (fmt->pad != CCS_PAD_META && fmt->stream != CCS_STREAM_META)
+		fmt->format.code = __ccs_get_mbus_code(subdev, fmt->pad);
 
 	return 0;
 }
@@ -2208,6 +2341,89 @@ static int ccs_set_format_source(struct v4l2_subdev *subdev,
 	return ccs_pll_update(sensor);
 }
 
+static inline unsigned int ccs_embedded_data_lines(struct ccs_sensor *sensor)
+{
+	return sensor->embedded_end - sensor->embedded_start;
+}
+
+static int ccs_set_format_meta(struct v4l2_subdev *subdev,
+			       struct v4l2_subdev_state *sd_state,
+			       struct v4l2_mbus_framefmt *fmt)
+{
+	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
+	const struct ccs_csi_data_format *csi_format;
+	struct v4l2_mbus_framefmt *pix_fmt;
+	struct v4l2_mbus_framefmt *meta_fmt;
+	struct v4l2_mbus_framefmt *meta_out_fmt;
+	u32 code;
+
+	pix_fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC,
+					       CCS_STREAM_PIXEL);
+	meta_fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_META, 0);
+	meta_out_fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC,
+						    CCS_STREAM_META);
+
+	code = fmt ? fmt->code : 0;
+
+	meta_fmt->width = pix_fmt->width;
+	meta_fmt->height = ccs_embedded_data_lines(sensor);
+
+	csi_format = ccs_validate_csi_data_format(sensor, pix_fmt->code);
+
+	switch (csi_format->compressed) {
+	case 8:
+		meta_fmt->code = MEDIA_BUS_FMT_META_8;
+		break;
+	case 10:
+		meta_fmt->code = MEDIA_BUS_FMT_META_10;
+		break;
+	case 12:
+		meta_fmt->code = MEDIA_BUS_FMT_META_12;
+		break;
+	case 14:
+		meta_fmt->code = MEDIA_BUS_FMT_META_14;
+		break;
+	case 16:
+	case 20:
+	case 24: {
+		const struct ccs_embedded_code *embedded_code;
+
+		embedded_code = ccs_embedded_code(csi_format->compressed);
+		meta_fmt->code =
+			ccs_default_embedded_code(sensor, embedded_code);
+
+		if (!(CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
+		      BIT(embedded_code->cap_no_legacy)) &&
+		    code == embedded_code->code_legacy) {
+			meta_fmt->code = embedded_code->code_legacy;
+			sensor->emb_data_ctrl = 0;
+		}
+
+		if (CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
+		    BIT(embedded_code->cap_two_bytes_per_sample) &&
+		    code == embedded_code->code_two_bytes) {
+			meta_fmt->code = embedded_code->code_two_bytes;
+			sensor->emb_data_ctrl = embedded_code->ctrl;
+			meta_fmt->width <<= 1;
+		}
+
+		break;
+	}
+	default:
+		WARN_ON(1);
+		return 0;
+	}
+
+	meta_out_fmt->width = meta_fmt->width;
+	meta_out_fmt->height = meta_fmt->height;
+	meta_out_fmt->code = meta_fmt->code;
+
+	if (fmt)
+		*fmt = *meta_out_fmt;
+
+	return 0;
+}
+
 static int ccs_set_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_state *sd_state,
 			  struct v4l2_subdev_format *fmt)
@@ -2216,12 +2432,25 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
 	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
 	struct v4l2_rect *crops[CCS_PADS];
 
+	if (ssd == sensor->src && fmt->pad == CCS_PAD_META)
+		return ccs_get_format(subdev, sd_state, fmt);
+
 	mutex_lock(&sensor->mutex);
 
+	if (ssd == sensor->src && fmt->stream == CCS_STREAM_META) {
+		ccs_set_format_meta(subdev, sd_state, &fmt->format);
+
+		mutex_unlock(&sensor->mutex);
+
+		return 0;
+	}
+
 	if (fmt->pad == ssd->source_pad) {
 		int rval;
 
 		rval = ccs_set_format_source(subdev, sd_state, fmt);
+		if (ccs_embedded_data_lines(sensor) && ssd == sensor->src)
+			ccs_set_format_meta(subdev, sd_state, NULL);
 
 		mutex_unlock(&sensor->mutex);
 
@@ -2496,6 +2725,12 @@ static int ccs_sel_supported(struct v4l2_subdev *subdev,
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
 
+	if (sel->stream != CCS_STREAM_PIXEL)
+		return -EINVAL;
+
+	if (sel->pad == CCS_PAD_META)
+		return -EINVAL;
+
 	/* We only implement crop in three places. */
 	switch (sel->target) {
 	case V4L2_SEL_TGT_CROP:
@@ -2540,7 +2775,8 @@ static int ccs_set_crop(struct v4l2_subdev *subdev,
 
 	if (sel->pad == ssd->sink_pad) {
 		struct v4l2_mbus_framefmt *mfmt =
-			v4l2_subdev_state_get_format(sd_state, sel->pad);
+			v4l2_subdev_state_get_format(sd_state, sel->pad,
+						     CCS_STREAM_PIXEL);
 
 		src_size.width = mfmt->width;
 		src_size.height = mfmt->height;
@@ -2601,7 +2837,9 @@ static int ccs_get_selection(struct v4l2_subdev *subdev,
 		} else if (sel->pad == ssd->sink_pad) {
 			struct v4l2_mbus_framefmt *sink_fmt =
 				v4l2_subdev_state_get_format(sd_state,
-							     ssd->sink_pad);
+							     ssd->sink_pad,
+							     CCS_STREAM_PIXEL);
+
 			sel->r.top = sel->r.left = 0;
 			sel->r.width = sink_fmt->width;
 			sel->r.height = sink_fmt->height;
@@ -2689,6 +2927,18 @@ static int ccs_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
 	entry++;
 	desc->num_entries++;
 
+	if (ccs_embedded_data_lines(sensor)) {
+		struct v4l2_mbus_framefmt *meta_out_fmt =
+			v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC,
+						     CCS_STREAM_META);
+
+		entry->pixelcode = meta_out_fmt->code;
+		entry->stream = CCS_STREAM_META;
+		entry->bus.csi2.dt = MIPI_CSI2_DT_EMBEDDED_8B;
+		entry++;
+		desc->num_entries++;
+	}
+
 	v4l2_subdev_unlock_state(sd_state);
 
 	return 0;
@@ -3005,6 +3255,8 @@ static void ccs_cleanup(struct ccs_sensor *sensor)
 	ccs_free_controls(sensor);
 }
 
+static const struct v4l2_subdev_internal_ops ccs_internal_ops;
+
 static int ccs_init_subdev(struct ccs_sensor *sensor,
 			   struct ccs_subdev *ssd, const char *name,
 			   unsigned short num_pads, u32 function,
@@ -3017,15 +3269,18 @@ static int ccs_init_subdev(struct ccs_sensor *sensor,
 	if (!ssd)
 		return 0;
 
-	if (ssd != sensor->src)
+	if (ssd != sensor->src) {
 		v4l2_subdev_init(&ssd->sd, &ccs_ops);
+		ssd->sd.internal_ops = &ccs_internal_ops;
+	}
 
 	ssd->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
 	ssd->sd.entity.function = function;
 	ssd->sensor = sensor;
 
 	ssd->npads = num_pads;
-	ssd->source_pad = num_pads - 1;
+	ssd->source_pad =
+		ssd == sensor->pixel_array ? CCS_PA_PAD_SRC : CCS_PAD_SRC;
 
 	v4l2_i2c_subdev_set_name(&ssd->sd, client, sensor->minfo.name, name);
 
@@ -3039,6 +3294,10 @@ static int ccs_init_subdev(struct ccs_sensor *sensor,
 		ssd->sd.owner = THIS_MODULE;
 		ssd->sd.dev = &client->dev;
 		v4l2_set_subdevdata(&ssd->sd, client);
+	} else {
+		ssd->sd.flags |= V4L2_SUBDEV_FL_STREAMS;
+		ssd->pads[CCS_PAD_META].flags =
+			MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_INTERNAL;
 	}
 
 	rval = media_entity_pads_init(&ssd->sd.entity, ssd->npads, ssd->pads);
@@ -3056,21 +3315,19 @@ static int ccs_init_subdev(struct ccs_sensor *sensor,
 	return 0;
 }
 
-static int ccs_init_state(struct v4l2_subdev *sd,
-			  struct v4l2_subdev_state *sd_state)
+static int __ccs_init_state(struct v4l2_subdev *sd,
+			    struct v4l2_subdev_state *sd_state)
 {
 	struct ccs_subdev *ssd = to_ccs_subdev(sd);
 	struct ccs_sensor *sensor = ssd->sensor;
 	unsigned int pad = ssd == sensor->pixel_array ?
 		CCS_PA_PAD_SRC : CCS_PAD_SINK;
 	struct v4l2_mbus_framefmt *fmt =
-		v4l2_subdev_state_get_format(sd_state, pad);
+		v4l2_subdev_state_get_format(sd_state, pad, CCS_STREAM_PIXEL);
 	struct v4l2_rect *crop =
-		v4l2_subdev_state_get_crop(sd_state, pad);
+		v4l2_subdev_state_get_crop(sd_state, pad, CCS_STREAM_PIXEL);
 	bool is_active = !sd->active_state || sd->active_state == sd_state;
 
-	mutex_lock(&sensor->mutex);
-
 	ccs_get_native_size(ssd, crop);
 
 	fmt->width = crop->width;
@@ -3082,20 +3339,78 @@ static int ccs_init_state(struct v4l2_subdev *sd,
 		if (is_active)
 			sensor->pa_src = *crop;
 
-		mutex_unlock(&sensor->mutex);
 		return 0;
 	}
 
-	fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC);
+	fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC,
+					   CCS_STREAM_PIXEL);
 	fmt->code = ssd == sensor->src ?
 		sensor->csi_format->code : sensor->internal_csi_format->code;
 	fmt->field = V4L2_FIELD_NONE;
 
 	ccs_propagate(sd, sd_state, is_active, V4L2_SEL_TGT_CROP);
 
+	return 0;
+}
+
+static int ccs_init_state(struct v4l2_subdev *sd,
+			  struct v4l2_subdev_state *sd_state)
+{
+	struct ccs_subdev *ssd = to_ccs_subdev(sd);
+	struct ccs_sensor *sensor = ssd->sensor;
+	int rval;
+
+	mutex_lock(&sensor->mutex);
+	rval = __ccs_init_state(sd, sd_state);
 	mutex_unlock(&sensor->mutex);
 
-	return 0;
+	return rval;
+}
+
+static int ccs_src_init_state(struct v4l2_subdev *sd,
+			      struct v4l2_subdev_state *sd_state)
+{
+	struct v4l2_subdev_route routes[] = {
+		{
+			.sink_pad = CCS_PAD_SINK,
+			.source_pad = CCS_PAD_SRC,
+			.source_stream = CCS_STREAM_PIXEL,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+		}, {
+			.sink_pad = CCS_PAD_META,
+			.source_pad = CCS_PAD_SRC,
+			.source_stream = CCS_STREAM_META,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+		}
+	};
+	struct v4l2_subdev_krouting routing = {
+		.routes = routes,
+		.num_routes = 1,
+	};
+	struct ccs_subdev *ssd = to_ccs_subdev(sd);
+	struct ccs_sensor *sensor = ssd->sensor;
+	int rval;
+
+	mutex_lock(&sensor->mutex);
+
+	if (ccs_embedded_data_lines(sensor))
+		routing.num_routes++;
+
+	rval = v4l2_subdev_set_routing(sd, sd_state, &routing);
+	if (rval)
+		goto out;
+
+	rval = __ccs_init_state(sd, sd_state);
+	if (rval)
+		goto out;
+
+	if (ccs_embedded_data_lines(sensor))
+		ccs_set_format_meta(sd, sd_state, NULL);
+
+out:
+	mutex_unlock(&sensor->mutex);
+
+	return rval;
 }
 
 static const struct v4l2_subdev_video_ops ccs_video_ops = {
@@ -3110,6 +3425,14 @@ static const struct v4l2_subdev_pad_ops ccs_pad_ops = {
 	.set_fmt = ccs_set_format,
 	.get_selection = ccs_get_selection,
 	.set_selection = ccs_set_selection,
+};
+
+static const struct v4l2_subdev_pad_ops ccs_src_pad_ops = {
+	.enum_mbus_code = ccs_enum_mbus_code,
+	.get_fmt = ccs_get_format,
+	.set_fmt = ccs_set_format,
+	.get_selection = ccs_get_selection,
+	.set_selection = ccs_set_selection,
 	.enable_streams = ccs_enable_streams,
 	.disable_streams = ccs_disable_streams,
 	.get_frame_desc = ccs_get_frame_desc,
@@ -3126,12 +3449,22 @@ static const struct v4l2_subdev_ops ccs_ops = {
 	.sensor = &ccs_sensor_ops,
 };
 
+static const struct v4l2_subdev_ops ccs_src_ops = {
+	.video = &ccs_video_ops,
+	.pad = &ccs_src_pad_ops,
+	.sensor = &ccs_sensor_ops,
+};
+
 static const struct media_entity_operations ccs_entity_ops = {
 	.link_validate = v4l2_subdev_link_validate,
 };
 
-static const struct v4l2_subdev_internal_ops ccs_internal_src_ops = {
+static const struct v4l2_subdev_internal_ops ccs_internal_ops = {
 	.init_state = ccs_init_state,
+};
+
+static const struct v4l2_subdev_internal_ops ccs_src_internal_ops = {
+	.init_state = ccs_src_init_state,
 	.registered = ccs_registered,
 	.unregistered = ccs_unregistered,
 };
@@ -3280,8 +3613,8 @@ static int ccs_probe(struct i2c_client *client)
 
 	sensor->src = &sensor->ssds[sensor->ssds_used];
 
-	v4l2_i2c_subdev_init(&sensor->src->sd, client, &ccs_ops);
-	sensor->src->sd.internal_ops = &ccs_internal_src_ops;
+	v4l2_i2c_subdev_init(&sensor->src->sd, client, &ccs_src_ops);
+	sensor->src->sd.internal_ops = &ccs_src_internal_ops;
 
 	sensor->regulators = devm_kcalloc(&client->dev,
 					  ARRAY_SIZE(ccs_regulators),
@@ -3555,12 +3888,20 @@ static int ccs_probe(struct i2c_client *client)
 		goto out_cleanup;
 	}
 
-	rval = ccs_init_subdev(sensor, sensor->scaler, " scaler", 2,
+	rval = ccs_init_subdev(sensor, sensor->scaler, " scaler",
+			       sensor->ssds_used != CCS_SUBDEVS ?
+			       CCS_PADS_NOMETA :
+			       ccs_embedded_data_lines(sensor) ?
+			       CCS_PADS : CCS_PADS_NOMETA,
 			       MEDIA_ENT_F_PROC_VIDEO_SCALER,
 			       "ccs scaler mutex", &scaler_lock_key);
 	if (rval)
 		goto out_cleanup;
-	rval = ccs_init_subdev(sensor, sensor->binner, " binner", 2,
+	rval = ccs_init_subdev(sensor, sensor->binner, " binner",
+			       sensor->ssds_used == CCS_SUBDEVS ?
+			       CCS_PADS_NOMETA :
+			       ccs_embedded_data_lines(sensor) ?
+			       CCS_PADS : CCS_PADS_NOMETA,
 			       MEDIA_ENT_F_PROC_VIDEO_SCALER,
 			       "ccs binner mutex", &binner_lock_key);
 	if (rval)
diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
index 0bdb8cd3accb..1bc524de3aa2 100644
--- a/drivers/media/i2c/ccs/ccs.h
+++ b/drivers/media/i2c/ccs/ccs.h
@@ -175,11 +175,18 @@ struct ccs_csi_data_format {
 #define CCS_SUBDEVS			3
 
 #define CCS_PA_PAD_SRC			0
-#define CCS_PAD_SINK			0
-#define CCS_PAD_SRC			1
-#define CCS_PADS			2
+enum {
+	CCS_PAD_SINK,
+	CCS_PAD_SRC,
+	CCS_PAD_META,
+	CCS_PADS_NOMETA = CCS_PAD_META,
+	CCS_PADS,
+};
 
-#define CCS_STREAM_PIXEL		0
+enum {
+	CCS_STREAM_PIXEL,
+	CCS_STREAM_META,
+};
 
 struct ccs_binning_subtype {
 	u8 horizontal:4;
@@ -230,6 +237,8 @@ struct ccs_sensor {
 	int default_pixel_order;
 	struct ccs_data_container sdata, mdata;
 
+	u8 emb_data_ctrl;
+
 	u8 binning_horizontal;
 	u8 binning_vertical;
 
-- 
2.47.2

[PATCH v11 30/66] media: Documentation: ccs: Document routing

[Sakari Ailus]

Document which routes are available for the CCS driver (source) sub-device
and what configuration are possible.

Also update copyright.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 .../userspace-api/media/drivers/ccs.rst       | 43 ++++++++++++++++++-
 .../media/v4l/metadata-layouts.rst            |  2 +
 2 files changed, 44 insertions(+), 1 deletion(-)

diff --git a/Documentation/userspace-api/media/drivers/ccs.rst b/Documentation/userspace-api/media/drivers/ccs.rst
index 03015b33d5ab..67c7d2b62ed5 100644
--- a/Documentation/userspace-api/media/drivers/ccs.rst
+++ b/Documentation/userspace-api/media/drivers/ccs.rst
@@ -111,4 +111,45 @@ than in the centre.
 Shading correction needs to be enabled for luminance correction level to have an
 effect.
 
-**Copyright** |copy| 2020 Intel Corporation
+.. _media-ccs-routes:
+
+Routes
+------
+
+The CCS driver implements one or two :ref:`routes <subdev-routing>` in
+its source sub-device (scaler sub-device if it exists for the device, otherwise
+binner) depending on whether the sensor supports embedded data. (All CCS
+compliant sensors do but the CCS driver supports preceding standards that did
+not require embedded data support, too.)
+
+The first route of the CCS source sub-device is for pixel data (sink pad
+0/stream 0 -> source pad 1/stream 0) and the second one is for embedded data
+(internal sink pad 2/stream 0 -> source pad 1/stream 1).
+
+Embedded data
+~~~~~~~~~~~~~
+
+MIPI CCS supports generation of camera sensor embedded data. The metadata layout
+used for this format on the internal sink pad is :ref:`V4L2_METADATA_LAYOUT_CCS
+<media-metadata-layout-ccs>`.
+
+The bit depth of the CCS pixel data affects how the sensor will output the
+embedded data, adding padding to align with CSI-2 bus :term:`Data Unit` for that
+particular bit depth. This is indicated by the generic metadata format on the
+source pad of the sensor's source sub-device.
+
+Devices supporting embedded data for bit depths greater than or equal to 16 may
+support more dense packing or legacy single metadata byte per data unit, or both
+of these. The supported embedded data formats can be enumerated and configured
+on stream 1 of the source pad (1) of the CCS source sub-device.
+
+The use of the denser packing results in embedded data lines being longer than
+the pixel data in data units since the data units are smaller. In bytes the
+embedded data lines are still not longer than the image data lines.
+
+The embedded data format is determined by the sub-device image data format
+configured on the source sub-device. The embedded data mbus code is only changed
+when the bit depth of the image data changes in the source pad of the source
+sub-device.
+
+**Copyright** |copy| 2020, 2024--2025 Intel Corporation
diff --git a/Documentation/userspace-api/media/v4l/metadata-layouts.rst b/Documentation/userspace-api/media/v4l/metadata-layouts.rst
index 5f717d54d1ca..a4f7e93e6a29 100644
--- a/Documentation/userspace-api/media/v4l/metadata-layouts.rst
+++ b/Documentation/userspace-api/media/v4l/metadata-layouts.rst
@@ -37,3 +37,5 @@ devices that fulfill all three levels above.
 This metadata layout code is only used for "2-byte simplified tagged data
 format" (code ``0xa``) but their use may be extended further in the future, to
 cover other CCS embedded data format codes.
+
+Also see :ref:`CCS driver documentation <media-ccs-routes>`.
-- 
2.47.2

[PATCH v11 31/66] media: ccs: Remove ccs_get_crop_compose helper

[Sakari Ailus]

As it's now easier to obtain the necessary information on crop and compose
rectangles after moving to sub-device state, remove the
ccs_get_crop_compose helper.

Also remove the comp arguments of the compose goodness calculators and
make related local variables and function arguments const where
applicable.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 164 ++++++++++++++-----------------
 1 file changed, 72 insertions(+), 92 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 5c53651f82df..0f0588e0c2da 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -2228,24 +2228,6 @@ static int ccs_get_format(struct v4l2_subdev *subdev,
 	return rval;
 }
 
-static void ccs_get_crop_compose(struct v4l2_subdev *subdev,
-				 struct v4l2_subdev_state *sd_state,
-				 struct v4l2_rect **crops,
-				 struct v4l2_rect **comps)
-{
-	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
-	unsigned int i;
-
-	if (crops)
-		for (i = 0; i < subdev->entity.num_pads; i++)
-			crops[i] =
-				v4l2_subdev_state_get_crop(sd_state, i,
-							   CCS_STREAM_PIXEL);
-	if (comps)
-		*comps = v4l2_subdev_state_get_compose(sd_state, ssd->sink_pad,
-						       CCS_STREAM_PIXEL);
-}
-
 /* Changes require propagation only on sink pad. */
 static void ccs_propagate(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_state *sd_state, int which,
@@ -2253,15 +2235,17 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
-	struct v4l2_rect *comp, *crops[CCS_PADS];
+	struct v4l2_rect *comp, *crop;
 	struct v4l2_mbus_framefmt *fmt;
 
-	ccs_get_crop_compose(subdev, sd_state, crops, &comp);
-
+	comp = v4l2_subdev_state_get_compose(sd_state, ssd->sink_pad,
+					     CCS_STREAM_PIXEL);
 	switch (target) {
 	case V4L2_SEL_TGT_CROP:
-		comp->width = crops[CCS_PAD_SINK]->width;
-		comp->height = crops[CCS_PAD_SINK]->height;
+		crop = v4l2_subdev_state_get_crop(sd_state, CCS_PAD_SINK,
+						  CCS_STREAM_PIXEL);
+		comp->width = crop->width;
+		comp->height = crop->height;
 		if (which == V4L2_SUBDEV_FORMAT_ACTIVE) {
 			if (ssd == sensor->scaler) {
 				sensor->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
@@ -2275,13 +2259,15 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 		}
 		fallthrough;
 	case V4L2_SEL_TGT_COMPOSE:
-		*crops[CCS_PAD_SRC] = *comp;
+		crop = v4l2_subdev_state_get_crop(sd_state, CCS_PAD_SRC,
+						  CCS_STREAM_PIXEL);
+		*crop = *comp;
 		fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC,
 						   CCS_STREAM_PIXEL);
 		fmt->width = comp->width;
 		fmt->height = comp->height;
 		if (which == V4L2_SUBDEV_FORMAT_ACTIVE && ssd == sensor->src)
-			sensor->src_src = *crops[CCS_PAD_SRC];
+			sensor->src_src = *crop;
 		break;
 	default:
 		WARN_ON_ONCE(1);
@@ -2430,7 +2416,7 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
-	struct v4l2_rect *crops[CCS_PADS];
+	struct v4l2_rect *crop;
 
 	if (ssd == sensor->src && fmt->pad == CCS_PAD_META)
 		return ccs_get_format(subdev, sd_state, fmt);
@@ -2472,12 +2458,13 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
 		      CCS_LIM(sensor, MIN_Y_OUTPUT_SIZE),
 		      CCS_LIM(sensor, MAX_Y_OUTPUT_SIZE));
 
-	ccs_get_crop_compose(subdev, sd_state, crops, NULL);
+	crop = v4l2_subdev_state_get_crop(sd_state, ssd->sink_pad,
+					  CCS_STREAM_PIXEL);
 
-	crops[ssd->sink_pad]->left = 0;
-	crops[ssd->sink_pad]->top = 0;
-	crops[ssd->sink_pad]->width = fmt->format.width;
-	crops[ssd->sink_pad]->height = fmt->format.height;
+	crop->left = 0;
+	crop->top = 0;
+	crop->width = fmt->format.width;
+	crop->height = fmt->format.height;
 	ccs_propagate(subdev, sd_state, fmt->which, V4L2_SEL_TGT_CROP);
 
 	mutex_unlock(&sensor->mutex);
@@ -2532,26 +2519,23 @@ static int scaling_goodness(struct v4l2_subdev *subdev, int w, int ask_w,
 static void ccs_set_compose_binner(struct v4l2_subdev *subdev,
 				   struct v4l2_subdev_state *sd_state,
 				   struct v4l2_subdev_selection *sel,
-				   struct v4l2_rect **crops,
-				   struct v4l2_rect *comp)
+				   const struct v4l2_rect *sink_crop)
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	unsigned int i;
 	unsigned int binh = 1, binv = 1;
-	int best = scaling_goodness(
-		subdev,
-		crops[CCS_PAD_SINK]->width, sel->r.width,
-		crops[CCS_PAD_SINK]->height, sel->r.height, sel->flags);
+	int best = scaling_goodness(subdev, sink_crop->width, sel->r.width,
+				    sink_crop->height, sel->r.height,
+				    sel->flags);
 
 	for (i = 0; i < sensor->nbinning_subtypes; i++) {
-		int this = scaling_goodness(
-			subdev,
-			crops[CCS_PAD_SINK]->width
-			/ sensor->binning_subtypes[i].horizontal,
-			sel->r.width,
-			crops[CCS_PAD_SINK]->height
-			/ sensor->binning_subtypes[i].vertical,
-			sel->r.height, sel->flags);
+		int this = scaling_goodness(subdev,
+					    sink_crop->width
+					    / sensor->binning_subtypes[i].horizontal,
+					    sel->r.width,
+					    sink_crop->height
+					    / sensor->binning_subtypes[i].vertical,
+					    sel->r.height, sel->flags);
 
 		if (this > best) {
 			binh = sensor->binning_subtypes[i].horizontal;
@@ -2564,8 +2548,8 @@ static void ccs_set_compose_binner(struct v4l2_subdev *subdev,
 		sensor->binning_horizontal = binh;
 	}
 
-	sel->r.width = (crops[CCS_PAD_SINK]->width / binh) & ~1;
-	sel->r.height = (crops[CCS_PAD_SINK]->height / binv) & ~1;
+	sel->r.width = (sink_crop->width / binh) & ~1;
+	sel->r.height = (sink_crop->height / binv) & ~1;
 }
 
 /*
@@ -2580,8 +2564,7 @@ static void ccs_set_compose_binner(struct v4l2_subdev *subdev,
 static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,
 				   struct v4l2_subdev_state *sd_state,
 				   struct v4l2_subdev_selection *sel,
-				   struct v4l2_rect **crops,
-				   struct v4l2_rect *comp)
+				   const struct v4l2_rect *sink_crop)
 {
 	struct i2c_client *client = v4l2_get_subdevdata(subdev);
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
@@ -2593,17 +2576,12 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,
 	unsigned int i;
 	int best = INT_MIN;
 
-	sel->r.width = min_t(unsigned int, sel->r.width,
-			     crops[CCS_PAD_SINK]->width);
-	sel->r.height = min_t(unsigned int, sel->r.height,
-			      crops[CCS_PAD_SINK]->height);
-
-	a = crops[CCS_PAD_SINK]->width
-		* CCS_LIM(sensor, SCALER_N_MIN) / sel->r.width;
-	b = crops[CCS_PAD_SINK]->height
-		* CCS_LIM(sensor, SCALER_N_MIN) / sel->r.height;
-	max_m = crops[CCS_PAD_SINK]->width
-		* CCS_LIM(sensor, SCALER_N_MIN)
+	sel->r.width = min_t(unsigned int, sel->r.width, sink_crop->width);
+	sel->r.height = min_t(unsigned int, sel->r.height, sink_crop->height);
+
+	a = sink_crop->width * CCS_LIM(sensor, SCALER_N_MIN) / sel->r.width;
+	b = sink_crop->height * CCS_LIM(sensor, SCALER_N_MIN) / sel->r.height;
+	max_m = sink_crop->width * CCS_LIM(sensor, SCALER_N_MIN)
 		/ CCS_LIM(sensor, MIN_X_OUTPUT_SIZE);
 
 	a = clamp(a, CCS_LIM(sensor, SCALER_M_MIN),
@@ -2634,14 +2612,12 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,
 	}
 
 	for (i = 0; i < ntry; i++) {
-		int this = scaling_goodness(
-			subdev,
-			crops[CCS_PAD_SINK]->width
-			/ try[i] * CCS_LIM(sensor, SCALER_N_MIN),
-			sel->r.width,
-			crops[CCS_PAD_SINK]->height,
-			sel->r.height,
-			sel->flags);
+		int this = scaling_goodness(subdev,
+					    sink_crop->width
+					    / try[i]
+					    * CCS_LIM(sensor, SCALER_N_MIN),
+					    sel->r.width, sink_crop->height,
+					    sel->r.height, sel->flags);
 
 		dev_dbg(&client->dev, "trying factor %u (%u)\n", try[i], i);
 
@@ -2656,12 +2632,10 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,
 			continue;
 
 		this = scaling_goodness(
-			subdev, crops[CCS_PAD_SINK]->width
-			/ try[i]
+			subdev, sink_crop->width / try[i]
 			* CCS_LIM(sensor, SCALER_N_MIN),
 			sel->r.width,
-			crops[CCS_PAD_SINK]->height
-			/ try[i]
+			sink_crop->height / try[i]
 			* CCS_LIM(sensor, SCALER_N_MIN),
 			sel->r.height,
 			sel->flags);
@@ -2673,18 +2647,13 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,
 		}
 	}
 
-	sel->r.width =
-		(crops[CCS_PAD_SINK]->width
-		 / scale_m
-		 * CCS_LIM(sensor, SCALER_N_MIN)) & ~1;
+	sel->r.width = (sink_crop->width / scale_m
+			* CCS_LIM(sensor, SCALER_N_MIN)) & ~1;
 	if (mode == SMIAPP_SCALING_MODE_BOTH)
-		sel->r.height =
-			(crops[CCS_PAD_SINK]->height
-			 / scale_m
-			 * CCS_LIM(sensor, SCALER_N_MIN))
-			& ~1;
+		sel->r.height = (sink_crop->height / scale_m
+				 * CCS_LIM(sensor, SCALER_N_MIN)) & ~1;
 	else
-		sel->r.height = crops[CCS_PAD_SINK]->height;
+		sel->r.height = sink_crop->height;
 
 	if (sel->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
 		sensor->scale_m = scale_m;
@@ -2698,17 +2667,21 @@ static int ccs_set_compose(struct v4l2_subdev *subdev,
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
-	struct v4l2_rect *comp, *crops[CCS_PADS];
+	const struct v4l2_rect *sink_crop;
+	struct v4l2_rect *comp;
 
-	ccs_get_crop_compose(subdev, sd_state, crops, &comp);
+	sink_crop = v4l2_subdev_state_get_crop(sd_state, CCS_PAD_SINK,
+					       CCS_STREAM_PIXEL);
+	comp = v4l2_subdev_state_get_compose(sd_state, ssd->sink_pad,
+					     CCS_STREAM_PIXEL);
 
 	sel->r.top = 0;
 	sel->r.left = 0;
 
 	if (ssd == sensor->binner)
-		ccs_set_compose_binner(subdev, sd_state, sel, crops, comp);
+		ccs_set_compose_binner(subdev, sd_state, sel, sink_crop);
 	else
-		ccs_set_compose_scaler(subdev, sd_state, sel, crops, comp);
+		ccs_set_compose_scaler(subdev, sd_state, sel, sink_crop);
 
 	*comp = sel->r;
 	ccs_propagate(subdev, sd_state, sel->which, V4L2_SEL_TGT_COMPOSE);
@@ -2769,9 +2742,13 @@ static int ccs_set_crop(struct v4l2_subdev *subdev,
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
-	struct v4l2_rect src_size = { 0 }, *crops[CCS_PADS], *comp;
+	struct v4l2_rect src_size = { 0 }, *crop;
+	const struct v4l2_rect *comp;
 
-	ccs_get_crop_compose(subdev, sd_state, crops, &comp);
+	crop = v4l2_subdev_state_get_crop(sd_state, sel->pad,
+					  CCS_STREAM_PIXEL);
+	comp = v4l2_subdev_state_get_compose(sd_state, ssd->sink_pad,
+					     CCS_STREAM_PIXEL);
 
 	if (sel->pad == ssd->sink_pad) {
 		struct v4l2_mbus_framefmt *mfmt =
@@ -2795,7 +2772,7 @@ static int ccs_set_crop(struct v4l2_subdev *subdev,
 	sel->r.left = min_t(int, sel->r.left, src_size.width - sel->r.width);
 	sel->r.top = min_t(int, sel->r.top, src_size.height - sel->r.height);
 
-	*crops[sel->pad] = sel->r;
+	*crop = sel->r;
 
 	if (ssd != sensor->pixel_array && sel->pad == CCS_PAD_SINK)
 		ccs_propagate(subdev, sd_state, sel->which, V4L2_SEL_TGT_CROP);
@@ -2820,14 +2797,17 @@ static int ccs_get_selection(struct v4l2_subdev *subdev,
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
-	struct v4l2_rect *comp, *crops[CCS_PADS];
+	const struct v4l2_rect *crop, *comp;
 	int ret;
 
 	ret = ccs_sel_supported(subdev, sel);
 	if (ret)
 		return ret;
 
-	ccs_get_crop_compose(subdev, sd_state, crops, &comp);
+	crop = v4l2_subdev_state_get_crop(sd_state, sel->pad,
+					  CCS_STREAM_PIXEL);
+	comp = v4l2_subdev_state_get_compose(sd_state, ssd->sink_pad,
+					     CCS_STREAM_PIXEL);
 
 	switch (sel->target) {
 	case V4L2_SEL_TGT_CROP_BOUNDS:
@@ -2849,7 +2829,7 @@ static int ccs_get_selection(struct v4l2_subdev *subdev,
 		break;
 	case V4L2_SEL_TGT_CROP:
 	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
-		sel->r = *crops[sel->pad];
+		sel->r = *crop;
 		break;
 	case V4L2_SEL_TGT_COMPOSE:
 		sel->r = *comp;
-- 
2.47.2

[PATCH v11 32/66] media: ccs: Rely on sub-device state locking

[Sakari Ailus]

Rely on sub-device state locking to serialise access to driver's data
structures. The driver-provided mutex is used as the state lock for all
driver sub-devices.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 187 +++++++++++++------------------
 drivers/media/i2c/ccs/ccs.h      |   1 -
 2 files changed, 79 insertions(+), 109 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 0f0588e0c2da..2d2ae568739b 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -541,12 +541,13 @@ static int ccs_pll_update(struct ccs_sensor *sensor)
  *
  */
 
-static void __ccs_update_exposure_limits(struct ccs_sensor *sensor)
+static void __ccs_update_exposure_limits(struct ccs_sensor *sensor,
+					 struct v4l2_rect *pa_src)
 {
 	struct v4l2_ctrl *ctrl = sensor->exposure;
 	int max;
 
-	max = sensor->pa_src.height + sensor->vblank->val -
+	max = pa_src->height + sensor->vblank->val -
 		CCS_LIM(sensor, COARSE_INTEGRATION_TIME_MAX_MARGIN);
 
 	__v4l2_ctrl_modify_range(ctrl, ctrl->minimum, max, ctrl->step, max);
@@ -649,12 +650,20 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl)
 		container_of(ctrl->handler, struct ccs_subdev, ctrl_handler)
 			->sensor;
 	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
+	struct v4l2_subdev_state *state;
+	struct v4l2_rect *pa_src;
 	int pm_status;
 	u32 orient = 0;
 	unsigned int i;
 	int exposure;
 	int rval;
 
+	if (ctrl->id == V4L2_CID_VBLANK || ctrl->id == V4L2_CID_HBLANK) {
+		state = v4l2_subdev_get_locked_active_state(&sensor->pixel_array->sd);
+		pa_src = v4l2_subdev_state_get_crop(state, CCS_PA_PAD_SRC,
+						    CCS_STREAM_PIXEL);
+	}
+
 	switch (ctrl->id) {
 	case V4L2_CID_HFLIP:
 	case V4L2_CID_VFLIP:
@@ -673,7 +682,7 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl)
 	case V4L2_CID_VBLANK:
 		exposure = sensor->exposure->val;
 
-		__ccs_update_exposure_limits(sensor);
+		__ccs_update_exposure_limits(sensor, pa_src);
 
 		if (exposure > sensor->exposure->maximum) {
 			sensor->exposure->val =	sensor->exposure->maximum;
@@ -765,12 +774,12 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl)
 		break;
 	case V4L2_CID_VBLANK:
 		rval = ccs_write(sensor, FRAME_LENGTH_LINES,
-				 sensor->pa_src.height + ctrl->val);
+				 pa_src->height + ctrl->val);
 
 		break;
 	case V4L2_CID_HBLANK:
 		rval = ccs_write(sensor, LINE_LENGTH_PCK,
-				 sensor->pa_src.width + ctrl->val);
+				 pa_src->width + ctrl->val);
 
 		break;
 	case V4L2_CID_TEST_PATTERN:
@@ -1242,7 +1251,8 @@ static int ccs_get_mbus_formats(struct ccs_sensor *sensor)
 	return 0;
 }
 
-static void ccs_update_blanking(struct ccs_sensor *sensor)
+static void ccs_update_blanking(struct ccs_sensor *sensor,
+				struct v4l2_rect *pa_src)
 {
 	struct v4l2_ctrl *vblank = sensor->vblank;
 	struct v4l2_ctrl *hblank = sensor->hblank;
@@ -1265,21 +1275,26 @@ static void ccs_update_blanking(struct ccs_sensor *sensor)
 
 	min = max_t(int,
 		    CCS_LIM(sensor, MIN_FRAME_BLANKING_LINES),
-		    min_fll - sensor->pa_src.height);
-	max = max_fll -	sensor->pa_src.height;
+		    min_fll - pa_src->height);
+	max = max_fll -	pa_src->height;
 
 	__v4l2_ctrl_modify_range(vblank, min, max, vblank->step, min);
 
-	min = max_t(int, min_llp - sensor->pa_src.width, min_lbp);
-	max = max_llp - sensor->pa_src.width;
+	min = max_t(int, min_llp - pa_src->width, min_lbp);
+	max = max_llp - pa_src->width;
 
 	__v4l2_ctrl_modify_range(hblank, min, max, hblank->step, min);
 
-	__ccs_update_exposure_limits(sensor);
+	__ccs_update_exposure_limits(sensor, pa_src);
 }
 
 static int ccs_pll_blanking_update(struct ccs_sensor *sensor)
 {
+	struct v4l2_subdev_state *state =
+		v4l2_subdev_get_locked_active_state(&sensor->pixel_array->sd);
+	struct v4l2_rect *pa_src =
+		v4l2_subdev_state_get_crop(state, CCS_PA_PAD_SRC,
+					   CCS_STREAM_PIXEL);
 	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
 	int rval;
 
@@ -1288,15 +1303,15 @@ static int ccs_pll_blanking_update(struct ccs_sensor *sensor)
 		return rval;
 
 	/* Output from pixel array, including blanking */
-	ccs_update_blanking(sensor);
+	ccs_update_blanking(sensor, pa_src);
 
 	dev_dbg(&client->dev, "vblank\t\t%d\n", sensor->vblank->val);
 	dev_dbg(&client->dev, "hblank\t\t%d\n", sensor->hblank->val);
 
 	dev_dbg(&client->dev, "real timeperframe\t100/%d\n",
 		sensor->pll.pixel_rate_pixel_array /
-		((sensor->pa_src.width + sensor->hblank->val) *
-		 (sensor->pa_src.height + sensor->vblank->val) / 100));
+		((pa_src->width + sensor->hblank->val) *
+		 (pa_src->height + sensor->vblank->val) / 100));
 
 	return 0;
 }
@@ -1801,6 +1816,16 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 			      u64 streams_mask)
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
+	struct v4l2_subdev_state *pa_state =
+		v4l2_subdev_get_locked_active_state(&sensor->pixel_array->sd);
+	struct v4l2_subdev_state *src_state =
+		v4l2_subdev_get_locked_active_state(&sensor->src->sd);
+	struct v4l2_rect *pa_src =
+		v4l2_subdev_state_get_crop(pa_state, CCS_PA_PAD_SRC,
+					   CCS_STREAM_PIXEL);
+	struct v4l2_rect *src_src =
+		v4l2_subdev_state_get_crop(src_state, CCS_PAD_SRC,
+					   CCS_STREAM_PIXEL);
 	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
 	unsigned int binning_mode;
 	int rval;
@@ -1848,22 +1873,20 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 		goto err_pm_put;
 
 	/* Analog crop start coordinates */
-	rval = ccs_write(sensor, X_ADDR_START, sensor->pa_src.left);
+	rval = ccs_write(sensor, X_ADDR_START, pa_src->left);
 	if (rval < 0)
 		goto err_pm_put;
 
-	rval = ccs_write(sensor, Y_ADDR_START, sensor->pa_src.top);
+	rval = ccs_write(sensor, Y_ADDR_START, pa_src->top);
 	if (rval < 0)
 		goto err_pm_put;
 
 	/* Analog crop end coordinates */
-	rval = ccs_write(sensor, X_ADDR_END,
-			 sensor->pa_src.left + sensor->pa_src.width - 1);
+	rval = ccs_write(sensor, X_ADDR_END, pa_src->left + pa_src->width - 1);
 	if (rval < 0)
 		goto err_pm_put;
 
-	rval = ccs_write(sensor, Y_ADDR_END,
-			 sensor->pa_src.top + sensor->pa_src.height - 1);
+	rval = ccs_write(sensor, Y_ADDR_END, pa_src->top + pa_src->height - 1);
 	if (rval < 0)
 		goto err_pm_put;
 
@@ -1875,23 +1898,30 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 	/* Digital crop */
 	if (CCS_LIM(sensor, DIGITAL_CROP_CAPABILITY)
 	    == CCS_DIGITAL_CROP_CAPABILITY_INPUT_CROP) {
+		struct v4l2_subdev_state *scaler_state =
+			v4l2_subdev_get_locked_active_state(&sensor->scaler->sd);
+		struct v4l2_rect *scaler_sink =
+			v4l2_subdev_state_get_crop(scaler_state,
+						   sensor->scaler->sink_pad,
+						   CCS_STREAM_PIXEL);
+
 		rval = ccs_write(sensor, DIGITAL_CROP_X_OFFSET,
-				 sensor->scaler_sink.left);
+				 scaler_sink->left);
 		if (rval < 0)
 			goto err_pm_put;
 
 		rval = ccs_write(sensor, DIGITAL_CROP_Y_OFFSET,
-				 sensor->scaler_sink.top);
+				 scaler_sink->top);
 		if (rval < 0)
 			goto err_pm_put;
 
 		rval = ccs_write(sensor, DIGITAL_CROP_IMAGE_WIDTH,
-				 sensor->scaler_sink.width);
+				 scaler_sink->width);
 		if (rval < 0)
 			goto err_pm_put;
 
 		rval = ccs_write(sensor, DIGITAL_CROP_IMAGE_HEIGHT,
-				 sensor->scaler_sink.height);
+				 scaler_sink->height);
 		if (rval < 0)
 			goto err_pm_put;
 	}
@@ -1909,10 +1939,10 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 	}
 
 	/* Output size from sensor */
-	rval = ccs_write(sensor, X_OUTPUT_SIZE, sensor->src_src.width);
+	rval = ccs_write(sensor, X_OUTPUT_SIZE, src_src->width);
 	if (rval < 0)
 		goto err_pm_put;
-	rval = ccs_write(sensor, Y_OUTPUT_SIZE, sensor->src_src.height);
+	rval = ccs_write(sensor, Y_OUTPUT_SIZE, src_src->height);
 	if (rval < 0)
 		goto err_pm_put;
 
@@ -2101,9 +2131,6 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	unsigned int i;
 	int idx = -1;
-	int rval = -EINVAL;
-
-	mutex_lock(&sensor->mutex);
 
 	dev_err(&client->dev, "subdev %s, pad %u, index %u\n",
 		subdev->name, code->pad, code->index);
@@ -2155,21 +2182,21 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,
 			}
 
 			if (WARN_ON(i > ARRAY_SIZE(codes)) || code->index >= i)
-				goto out;
+				return -EINVAL;
 
 			code->code = codes[code->index];
-			rval = 0;
-			goto out;
+
+			return 0;
 		}
 	}
 
 	if (subdev != &sensor->src->sd || code->pad != CCS_PAD_SRC) {
 		if (code->index)
-			goto out;
+			return -EINVAL;
 
 		code->code = sensor->internal_csi_format->code;
-		rval = 0;
-		goto out;
+
+		return 0;
 	}
 
 	for (i = 0; i < ARRAY_SIZE(ccs_csi_data_formats); i++) {
@@ -2180,18 +2207,14 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,
 			code->code = ccs_csi_data_formats[i].code;
 			dev_err(&client->dev, "found index %u, i %u, code %x\n",
 				code->index, i, code->code);
-			rval = 0;
-			break;
+			return 0;
 		}
 	}
 
-out:
-	mutex_unlock(&sensor->mutex);
-
-	return rval;
+	return -EINVAL;
 }
 
-static u32 __ccs_get_mbus_code(struct v4l2_subdev *subdev, unsigned int pad)
+static u32 ccs_get_mbus_code(struct v4l2_subdev *subdev, unsigned int pad)
 {
 	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 
@@ -2201,33 +2224,19 @@ static u32 __ccs_get_mbus_code(struct v4l2_subdev *subdev, unsigned int pad)
 		return sensor->internal_csi_format->code;
 }
 
-static int __ccs_get_format(struct v4l2_subdev *subdev,
-			    struct v4l2_subdev_state *sd_state,
-			    struct v4l2_subdev_format *fmt)
+static int ccs_get_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_state *sd_state,
+			  struct v4l2_subdev_format *fmt)
 {
 	fmt->format = *v4l2_subdev_state_get_format(sd_state, fmt->pad,
 						    fmt->stream);
 
 	if (fmt->pad != CCS_PAD_META && fmt->stream != CCS_STREAM_META)
-		fmt->format.code = __ccs_get_mbus_code(subdev, fmt->pad);
+		fmt->format.code = ccs_get_mbus_code(subdev, fmt->pad);
 
 	return 0;
 }
 
-static int ccs_get_format(struct v4l2_subdev *subdev,
-			  struct v4l2_subdev_state *sd_state,
-			  struct v4l2_subdev_format *fmt)
-{
-	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
-	int rval;
-
-	mutex_lock(&sensor->mutex);
-	rval = __ccs_get_format(subdev, sd_state, fmt);
-	mutex_unlock(&sensor->mutex);
-
-	return rval;
-}
-
 /* Changes require propagation only on sink pad. */
 static void ccs_propagate(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_state *sd_state, int which,
@@ -2251,7 +2260,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 				sensor->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
 				sensor->scaling_mode =
 					CCS_SCALING_MODE_NO_SCALING;
-				sensor->scaler_sink = *comp;
 			} else if (ssd == sensor->binner) {
 				sensor->binning_horizontal = 1;
 				sensor->binning_vertical = 1;
@@ -2266,8 +2274,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 						   CCS_STREAM_PIXEL);
 		fmt->width = comp->width;
 		fmt->height = comp->height;
-		if (which == V4L2_SUBDEV_FORMAT_ACTIVE && ssd == sensor->src)
-			sensor->src_src = *crop;
 		break;
 	default:
 		WARN_ON_ONCE(1);
@@ -2286,7 +2292,7 @@ static int ccs_set_format_source(struct v4l2_subdev *subdev,
 	unsigned int i;
 	int rval;
 
-	rval = __ccs_get_format(subdev, sd_state, fmt);
+	rval = ccs_get_format(subdev, sd_state, fmt);
 	if (rval)
 		return rval;
 
@@ -2421,13 +2427,9 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
 	if (ssd == sensor->src && fmt->pad == CCS_PAD_META)
 		return ccs_get_format(subdev, sd_state, fmt);
 
-	mutex_lock(&sensor->mutex);
-
 	if (ssd == sensor->src && fmt->stream == CCS_STREAM_META) {
 		ccs_set_format_meta(subdev, sd_state, &fmt->format);
 
-		mutex_unlock(&sensor->mutex);
-
 		return 0;
 	}
 
@@ -2438,13 +2440,12 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
 		if (ccs_embedded_data_lines(sensor) && ssd == sensor->src)
 			ccs_set_format_meta(subdev, sd_state, NULL);
 
-		mutex_unlock(&sensor->mutex);
-
 		return rval;
 	}
 
 	/* Sink pad. Width and height are changeable here. */
-	fmt->format.code = __ccs_get_mbus_code(subdev, fmt->pad);
+	fmt->format.code = ccs_get_mbus_code(subdev, fmt->pad);
+
 	fmt->format.width &= ~1;
 	fmt->format.height &= ~1;
 	fmt->format.field = V4L2_FIELD_NONE;
@@ -2467,8 +2468,6 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
 	crop->height = fmt->format.height;
 	ccs_propagate(subdev, sd_state, fmt->which, V4L2_SEL_TGT_CROP);
 
-	mutex_unlock(&sensor->mutex);
-
 	return 0;
 }
 
@@ -2776,9 +2775,6 @@ static int ccs_set_crop(struct v4l2_subdev *subdev,
 
 	if (ssd != sensor->pixel_array && sel->pad == CCS_PAD_SINK)
 		ccs_propagate(subdev, sd_state, sel->which, V4L2_SEL_TGT_CROP);
-	else if (sel->which == V4L2_SUBDEV_FORMAT_ACTIVE &&
-		 ssd == sensor->pixel_array)
-		sensor->pa_src = sel->r;
 
 	return 0;
 }
@@ -2850,8 +2846,6 @@ static int ccs_set_selection(struct v4l2_subdev *subdev,
 	if (ret)
 		return ret;
 
-	mutex_lock(&sensor->mutex);
-
 	sel->r.left = max(0, sel->r.left & ~1);
 	sel->r.top = max(0, sel->r.top & ~1);
 	sel->r.width = CCS_ALIGN_DIM(sel->r.width, sel->flags);
@@ -2873,7 +2867,6 @@ static int ccs_set_selection(struct v4l2_subdev *subdev,
 		ret = -EINVAL;
 	}
 
-	mutex_unlock(&sensor->mutex);
 	return ret;
 }
 
@@ -3256,6 +3249,7 @@ static int ccs_init_subdev(struct ccs_sensor *sensor,
 
 	ssd->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
 	ssd->sd.entity.function = function;
+	ssd->sd.state_lock = &sensor->mutex;
 	ssd->sensor = sensor;
 
 	ssd->npads = num_pads;
@@ -3295,8 +3289,8 @@ static int ccs_init_subdev(struct ccs_sensor *sensor,
 	return 0;
 }
 
-static int __ccs_init_state(struct v4l2_subdev *sd,
-			    struct v4l2_subdev_state *sd_state)
+static int ccs_init_state(struct v4l2_subdev *sd,
+			  struct v4l2_subdev_state *sd_state)
 {
 	struct ccs_subdev *ssd = to_ccs_subdev(sd);
 	struct ccs_sensor *sensor = ssd->sensor;
@@ -3315,12 +3309,8 @@ static int __ccs_init_state(struct v4l2_subdev *sd,
 	fmt->code = sensor->internal_csi_format->code;
 	fmt->field = V4L2_FIELD_NONE;
 
-	if (ssd == sensor->pixel_array) {
-		if (is_active)
-			sensor->pa_src = *crop;
-
+	if (ssd == sensor->pixel_array)
 		return 0;
-	}
 
 	fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC,
 					   CCS_STREAM_PIXEL);
@@ -3333,20 +3323,6 @@ static int __ccs_init_state(struct v4l2_subdev *sd,
 	return 0;
 }
 
-static int ccs_init_state(struct v4l2_subdev *sd,
-			  struct v4l2_subdev_state *sd_state)
-{
-	struct ccs_subdev *ssd = to_ccs_subdev(sd);
-	struct ccs_sensor *sensor = ssd->sensor;
-	int rval;
-
-	mutex_lock(&sensor->mutex);
-	rval = __ccs_init_state(sd, sd_state);
-	mutex_unlock(&sensor->mutex);
-
-	return rval;
-}
-
 static int ccs_src_init_state(struct v4l2_subdev *sd,
 			      struct v4l2_subdev_state *sd_state)
 {
@@ -3371,25 +3347,20 @@ static int ccs_src_init_state(struct v4l2_subdev *sd,
 	struct ccs_sensor *sensor = ssd->sensor;
 	int rval;
 
-	mutex_lock(&sensor->mutex);
-
 	if (ccs_embedded_data_lines(sensor))
 		routing.num_routes++;
 
 	rval = v4l2_subdev_set_routing(sd, sd_state, &routing);
 	if (rval)
-		goto out;
+		return 0;
 
-	rval = __ccs_init_state(sd, sd_state);
+	rval = ccs_init_state(sd, sd_state);
 	if (rval)
-		goto out;
+		return 0;
 
 	if (ccs_embedded_data_lines(sensor))
 		ccs_set_format_meta(sd, sd_state, NULL);
 
-out:
-	mutex_unlock(&sensor->mutex);
-
 	return rval;
 }
 
diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
index 1bc524de3aa2..338209950fcb 100644
--- a/drivers/media/i2c/ccs/ccs.h
+++ b/drivers/media/i2c/ccs/ccs.h
@@ -232,7 +232,6 @@ struct ccs_sensor {
 	u32 mbus_frame_fmts;
 	const struct ccs_csi_data_format *csi_format;
 	const struct ccs_csi_data_format *internal_csi_format;
-	struct v4l2_rect pa_src, scaler_sink, src_src;
 	u32 default_mbus_frame_fmts;
 	int default_pixel_order;
 	struct ccs_data_container sdata, mdata;
-- 
2.47.2

Re: [PATCH v11 32/66] media: ccs: Rely on sub-device state locking

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Aug 25, 2025 at 12:50:33PM +0300, Sakari Ailus wrote:
> Rely on sub-device state locking to serialise access to driver's data
> structures. The driver-provided mutex is used as the state lock for all
> driver sub-devices.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/i2c/ccs/ccs-core.c | 187 +++++++++++++------------------
>  drivers/media/i2c/ccs/ccs.h      |   1 -
>  2 files changed, 79 insertions(+), 109 deletions(-)
> 
> diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
> index 0f0588e0c2da..2d2ae568739b 100644
> --- a/drivers/media/i2c/ccs/ccs-core.c
> +++ b/drivers/media/i2c/ccs/ccs-core.c
> @@ -541,12 +541,13 @@ static int ccs_pll_update(struct ccs_sensor *sensor)
>   *
>   */
>  
> -static void __ccs_update_exposure_limits(struct ccs_sensor *sensor)
> +static void __ccs_update_exposure_limits(struct ccs_sensor *sensor,
> +					 struct v4l2_rect *pa_src)

This can be const.

>  {
>  	struct v4l2_ctrl *ctrl = sensor->exposure;
>  	int max;
>  
> -	max = sensor->pa_src.height + sensor->vblank->val -
> +	max = pa_src->height + sensor->vblank->val -
>  		CCS_LIM(sensor, COARSE_INTEGRATION_TIME_MAX_MARGIN);
>  
>  	__v4l2_ctrl_modify_range(ctrl, ctrl->minimum, max, ctrl->step, max);
> @@ -649,12 +650,20 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl)
>  		container_of(ctrl->handler, struct ccs_subdev, ctrl_handler)
>  			->sensor;
>  	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
> +	struct v4l2_subdev_state *state;
> +	struct v4l2_rect *pa_src;
>  	int pm_status;
>  	u32 orient = 0;
>  	unsigned int i;
>  	int exposure;
>  	int rval;
>  
> +	if (ctrl->id == V4L2_CID_VBLANK || ctrl->id == V4L2_CID_HBLANK) {
> +		state = v4l2_subdev_get_locked_active_state(&sensor->pixel_array->sd);
> +		pa_src = v4l2_subdev_state_get_crop(state, CCS_PA_PAD_SRC,
> +						    CCS_STREAM_PIXEL);
> +	}
> +
>  	switch (ctrl->id) {
>  	case V4L2_CID_HFLIP:
>  	case V4L2_CID_VFLIP:
> @@ -673,7 +682,7 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl)
>  	case V4L2_CID_VBLANK:
>  		exposure = sensor->exposure->val;
>  
> -		__ccs_update_exposure_limits(sensor);
> +		__ccs_update_exposure_limits(sensor, pa_src);
>  
>  		if (exposure > sensor->exposure->maximum) {
>  			sensor->exposure->val =	sensor->exposure->maximum;
> @@ -765,12 +774,12 @@ static int ccs_set_ctrl(struct v4l2_ctrl *ctrl)
>  		break;
>  	case V4L2_CID_VBLANK:
>  		rval = ccs_write(sensor, FRAME_LENGTH_LINES,
> -				 sensor->pa_src.height + ctrl->val);
> +				 pa_src->height + ctrl->val);
>  
>  		break;
>  	case V4L2_CID_HBLANK:
>  		rval = ccs_write(sensor, LINE_LENGTH_PCK,
> -				 sensor->pa_src.width + ctrl->val);
> +				 pa_src->width + ctrl->val);
>  
>  		break;
>  	case V4L2_CID_TEST_PATTERN:
> @@ -1242,7 +1251,8 @@ static int ccs_get_mbus_formats(struct ccs_sensor *sensor)
>  	return 0;
>  }
>  
> -static void ccs_update_blanking(struct ccs_sensor *sensor)
> +static void ccs_update_blanking(struct ccs_sensor *sensor,
> +				struct v4l2_rect *pa_src)

Same here.

>  {
>  	struct v4l2_ctrl *vblank = sensor->vblank;
>  	struct v4l2_ctrl *hblank = sensor->hblank;
> @@ -1265,21 +1275,26 @@ static void ccs_update_blanking(struct ccs_sensor *sensor)
>  
>  	min = max_t(int,
>  		    CCS_LIM(sensor, MIN_FRAME_BLANKING_LINES),
> -		    min_fll - sensor->pa_src.height);
> -	max = max_fll -	sensor->pa_src.height;
> +		    min_fll - pa_src->height);
> +	max = max_fll -	pa_src->height;
>  
>  	__v4l2_ctrl_modify_range(vblank, min, max, vblank->step, min);
>  
> -	min = max_t(int, min_llp - sensor->pa_src.width, min_lbp);
> -	max = max_llp - sensor->pa_src.width;
> +	min = max_t(int, min_llp - pa_src->width, min_lbp);
> +	max = max_llp - pa_src->width;
>  
>  	__v4l2_ctrl_modify_range(hblank, min, max, hblank->step, min);
>  
> -	__ccs_update_exposure_limits(sensor);
> +	__ccs_update_exposure_limits(sensor, pa_src);
>  }
>  
>  static int ccs_pll_blanking_update(struct ccs_sensor *sensor)
>  {
> +	struct v4l2_subdev_state *state =
> +		v4l2_subdev_get_locked_active_state(&sensor->pixel_array->sd);

As one of the callers (ccs_set_compose()) already has a state pointer, I
would pass it to this function.

> +	struct v4l2_rect *pa_src =

const

I now realize you seem to have missed my review of v9. Please take all
the comments from there into account for v12.

> +		v4l2_subdev_state_get_crop(state, CCS_PA_PAD_SRC,
> +					   CCS_STREAM_PIXEL);
>  	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
>  	int rval;
>  
> @@ -1288,15 +1303,15 @@ static int ccs_pll_blanking_update(struct ccs_sensor *sensor)
>  		return rval;
>  
>  	/* Output from pixel array, including blanking */
> -	ccs_update_blanking(sensor);
> +	ccs_update_blanking(sensor, pa_src);
>  
>  	dev_dbg(&client->dev, "vblank\t\t%d\n", sensor->vblank->val);
>  	dev_dbg(&client->dev, "hblank\t\t%d\n", sensor->hblank->val);
>  
>  	dev_dbg(&client->dev, "real timeperframe\t100/%d\n",
>  		sensor->pll.pixel_rate_pixel_array /
> -		((sensor->pa_src.width + sensor->hblank->val) *
> -		 (sensor->pa_src.height + sensor->vblank->val) / 100));
> +		((pa_src->width + sensor->hblank->val) *
> +		 (pa_src->height + sensor->vblank->val) / 100));
>  
>  	return 0;
>  }
> @@ -1801,6 +1816,16 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
>  			      u64 streams_mask)
>  {
>  	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
> +	struct v4l2_subdev_state *pa_state =
> +		v4l2_subdev_get_locked_active_state(&sensor->pixel_array->sd);
> +	struct v4l2_subdev_state *src_state =
> +		v4l2_subdev_get_locked_active_state(&sensor->src->sd);
> +	struct v4l2_rect *pa_src =
> +		v4l2_subdev_state_get_crop(pa_state, CCS_PA_PAD_SRC,
> +					   CCS_STREAM_PIXEL);
> +	struct v4l2_rect *src_src =
> +		v4l2_subdev_state_get_crop(src_state, CCS_PAD_SRC,
> +					   CCS_STREAM_PIXEL);
>  	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
>  	unsigned int binning_mode;
>  	int rval;
> @@ -1848,22 +1873,20 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
>  		goto err_pm_put;
>  
>  	/* Analog crop start coordinates */
> -	rval = ccs_write(sensor, X_ADDR_START, sensor->pa_src.left);
> +	rval = ccs_write(sensor, X_ADDR_START, pa_src->left);
>  	if (rval < 0)
>  		goto err_pm_put;
>  
> -	rval = ccs_write(sensor, Y_ADDR_START, sensor->pa_src.top);
> +	rval = ccs_write(sensor, Y_ADDR_START, pa_src->top);
>  	if (rval < 0)
>  		goto err_pm_put;
>  
>  	/* Analog crop end coordinates */
> -	rval = ccs_write(sensor, X_ADDR_END,
> -			 sensor->pa_src.left + sensor->pa_src.width - 1);
> +	rval = ccs_write(sensor, X_ADDR_END, pa_src->left + pa_src->width - 1);
>  	if (rval < 0)
>  		goto err_pm_put;
>  
> -	rval = ccs_write(sensor, Y_ADDR_END,
> -			 sensor->pa_src.top + sensor->pa_src.height - 1);
> +	rval = ccs_write(sensor, Y_ADDR_END, pa_src->top + pa_src->height - 1);
>  	if (rval < 0)
>  		goto err_pm_put;
>  
> @@ -1875,23 +1898,30 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
>  	/* Digital crop */
>  	if (CCS_LIM(sensor, DIGITAL_CROP_CAPABILITY)
>  	    == CCS_DIGITAL_CROP_CAPABILITY_INPUT_CROP) {
> +		struct v4l2_subdev_state *scaler_state =
> +			v4l2_subdev_get_locked_active_state(&sensor->scaler->sd);
> +		struct v4l2_rect *scaler_sink =
> +			v4l2_subdev_state_get_crop(scaler_state,
> +						   sensor->scaler->sink_pad,
> +						   CCS_STREAM_PIXEL);
> +
>  		rval = ccs_write(sensor, DIGITAL_CROP_X_OFFSET,
> -				 sensor->scaler_sink.left);
> +				 scaler_sink->left);
>  		if (rval < 0)
>  			goto err_pm_put;
>  
>  		rval = ccs_write(sensor, DIGITAL_CROP_Y_OFFSET,
> -				 sensor->scaler_sink.top);
> +				 scaler_sink->top);
>  		if (rval < 0)
>  			goto err_pm_put;
>  
>  		rval = ccs_write(sensor, DIGITAL_CROP_IMAGE_WIDTH,
> -				 sensor->scaler_sink.width);
> +				 scaler_sink->width);
>  		if (rval < 0)
>  			goto err_pm_put;
>  
>  		rval = ccs_write(sensor, DIGITAL_CROP_IMAGE_HEIGHT,
> -				 sensor->scaler_sink.height);
> +				 scaler_sink->height);
>  		if (rval < 0)
>  			goto err_pm_put;
>  	}
> @@ -1909,10 +1939,10 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
>  	}
>  
>  	/* Output size from sensor */
> -	rval = ccs_write(sensor, X_OUTPUT_SIZE, sensor->src_src.width);
> +	rval = ccs_write(sensor, X_OUTPUT_SIZE, src_src->width);
>  	if (rval < 0)
>  		goto err_pm_put;
> -	rval = ccs_write(sensor, Y_OUTPUT_SIZE, sensor->src_src.height);
> +	rval = ccs_write(sensor, Y_OUTPUT_SIZE, src_src->height);
>  	if (rval < 0)
>  		goto err_pm_put;
>  
> @@ -2101,9 +2131,6 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,
>  	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
>  	unsigned int i;
>  	int idx = -1;
> -	int rval = -EINVAL;
> -
> -	mutex_lock(&sensor->mutex);
>  
>  	dev_err(&client->dev, "subdev %s, pad %u, index %u\n",
>  		subdev->name, code->pad, code->index);
> @@ -2155,21 +2182,21 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,
>  			}
>  
>  			if (WARN_ON(i > ARRAY_SIZE(codes)) || code->index >= i)
> -				goto out;
> +				return -EINVAL;
>  
>  			code->code = codes[code->index];
> -			rval = 0;
> -			goto out;
> +
> +			return 0;
>  		}
>  	}
>  
>  	if (subdev != &sensor->src->sd || code->pad != CCS_PAD_SRC) {
>  		if (code->index)
> -			goto out;
> +			return -EINVAL;
>  
>  		code->code = sensor->internal_csi_format->code;
> -		rval = 0;
> -		goto out;
> +
> +		return 0;
>  	}
>  
>  	for (i = 0; i < ARRAY_SIZE(ccs_csi_data_formats); i++) {
> @@ -2180,18 +2207,14 @@ static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,
>  			code->code = ccs_csi_data_formats[i].code;
>  			dev_err(&client->dev, "found index %u, i %u, code %x\n",
>  				code->index, i, code->code);
> -			rval = 0;
> -			break;
> +			return 0;
>  		}
>  	}
>  
> -out:
> -	mutex_unlock(&sensor->mutex);
> -
> -	return rval;
> +	return -EINVAL;
>  }
>  
> -static u32 __ccs_get_mbus_code(struct v4l2_subdev *subdev, unsigned int pad)
> +static u32 ccs_get_mbus_code(struct v4l2_subdev *subdev, unsigned int pad)
>  {
>  	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
>  
> @@ -2201,33 +2224,19 @@ static u32 __ccs_get_mbus_code(struct v4l2_subdev *subdev, unsigned int pad)
>  		return sensor->internal_csi_format->code;
>  }
>  
> -static int __ccs_get_format(struct v4l2_subdev *subdev,
> -			    struct v4l2_subdev_state *sd_state,
> -			    struct v4l2_subdev_format *fmt)
> +static int ccs_get_format(struct v4l2_subdev *subdev,
> +			  struct v4l2_subdev_state *sd_state,
> +			  struct v4l2_subdev_format *fmt)
>  {
>  	fmt->format = *v4l2_subdev_state_get_format(sd_state, fmt->pad,
>  						    fmt->stream);
>  
>  	if (fmt->pad != CCS_PAD_META && fmt->stream != CCS_STREAM_META)
> -		fmt->format.code = __ccs_get_mbus_code(subdev, fmt->pad);
> +		fmt->format.code = ccs_get_mbus_code(subdev, fmt->pad);
>  
>  	return 0;
>  }
>  
> -static int ccs_get_format(struct v4l2_subdev *subdev,
> -			  struct v4l2_subdev_state *sd_state,
> -			  struct v4l2_subdev_format *fmt)
> -{
> -	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
> -	int rval;
> -
> -	mutex_lock(&sensor->mutex);
> -	rval = __ccs_get_format(subdev, sd_state, fmt);
> -	mutex_unlock(&sensor->mutex);
> -
> -	return rval;
> -}
> -
>  /* Changes require propagation only on sink pad. */
>  static void ccs_propagate(struct v4l2_subdev *subdev,
>  			  struct v4l2_subdev_state *sd_state, int which,
> @@ -2251,7 +2260,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
>  				sensor->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
>  				sensor->scaling_mode =
>  					CCS_SCALING_MODE_NO_SCALING;
> -				sensor->scaler_sink = *comp;
>  			} else if (ssd == sensor->binner) {
>  				sensor->binning_horizontal = 1;
>  				sensor->binning_vertical = 1;
> @@ -2266,8 +2274,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
>  						   CCS_STREAM_PIXEL);
>  		fmt->width = comp->width;
>  		fmt->height = comp->height;
> -		if (which == V4L2_SUBDEV_FORMAT_ACTIVE && ssd == sensor->src)
> -			sensor->src_src = *crop;
>  		break;
>  	default:
>  		WARN_ON_ONCE(1);
> @@ -2286,7 +2292,7 @@ static int ccs_set_format_source(struct v4l2_subdev *subdev,
>  	unsigned int i;
>  	int rval;
>  
> -	rval = __ccs_get_format(subdev, sd_state, fmt);
> +	rval = ccs_get_format(subdev, sd_state, fmt);
>  	if (rval)
>  		return rval;
>  
> @@ -2421,13 +2427,9 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
>  	if (ssd == sensor->src && fmt->pad == CCS_PAD_META)
>  		return ccs_get_format(subdev, sd_state, fmt);
>  
> -	mutex_lock(&sensor->mutex);
> -
>  	if (ssd == sensor->src && fmt->stream == CCS_STREAM_META) {
>  		ccs_set_format_meta(subdev, sd_state, &fmt->format);
>  
> -		mutex_unlock(&sensor->mutex);
> -
>  		return 0;
>  	}
>  
> @@ -2438,13 +2440,12 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
>  		if (ccs_embedded_data_lines(sensor) && ssd == sensor->src)
>  			ccs_set_format_meta(subdev, sd_state, NULL);
>  
> -		mutex_unlock(&sensor->mutex);
> -
>  		return rval;
>  	}
>  
>  	/* Sink pad. Width and height are changeable here. */
> -	fmt->format.code = __ccs_get_mbus_code(subdev, fmt->pad);
> +	fmt->format.code = ccs_get_mbus_code(subdev, fmt->pad);
> +
>  	fmt->format.width &= ~1;
>  	fmt->format.height &= ~1;
>  	fmt->format.field = V4L2_FIELD_NONE;
> @@ -2467,8 +2468,6 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
>  	crop->height = fmt->format.height;
>  	ccs_propagate(subdev, sd_state, fmt->which, V4L2_SEL_TGT_CROP);
>  
> -	mutex_unlock(&sensor->mutex);
> -
>  	return 0;
>  }
>  
> @@ -2776,9 +2775,6 @@ static int ccs_set_crop(struct v4l2_subdev *subdev,
>  
>  	if (ssd != sensor->pixel_array && sel->pad == CCS_PAD_SINK)
>  		ccs_propagate(subdev, sd_state, sel->which, V4L2_SEL_TGT_CROP);
> -	else if (sel->which == V4L2_SUBDEV_FORMAT_ACTIVE &&
> -		 ssd == sensor->pixel_array)
> -		sensor->pa_src = sel->r;
>  
>  	return 0;
>  }
> @@ -2850,8 +2846,6 @@ static int ccs_set_selection(struct v4l2_subdev *subdev,
>  	if (ret)
>  		return ret;
>  
> -	mutex_lock(&sensor->mutex);
> -
>  	sel->r.left = max(0, sel->r.left & ~1);
>  	sel->r.top = max(0, sel->r.top & ~1);
>  	sel->r.width = CCS_ALIGN_DIM(sel->r.width, sel->flags);
> @@ -2873,7 +2867,6 @@ static int ccs_set_selection(struct v4l2_subdev *subdev,
>  		ret = -EINVAL;
>  	}
>  
> -	mutex_unlock(&sensor->mutex);
>  	return ret;
>  }
>  
> @@ -3256,6 +3249,7 @@ static int ccs_init_subdev(struct ccs_sensor *sensor,
>  
>  	ssd->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>  	ssd->sd.entity.function = function;
> +	ssd->sd.state_lock = &sensor->mutex;
>  	ssd->sensor = sensor;
>  
>  	ssd->npads = num_pads;
> @@ -3295,8 +3289,8 @@ static int ccs_init_subdev(struct ccs_sensor *sensor,
>  	return 0;
>  }
>  
> -static int __ccs_init_state(struct v4l2_subdev *sd,
> -			    struct v4l2_subdev_state *sd_state)
> +static int ccs_init_state(struct v4l2_subdev *sd,
> +			  struct v4l2_subdev_state *sd_state)
>  {
>  	struct ccs_subdev *ssd = to_ccs_subdev(sd);
>  	struct ccs_sensor *sensor = ssd->sensor;
> @@ -3315,12 +3309,8 @@ static int __ccs_init_state(struct v4l2_subdev *sd,
>  	fmt->code = sensor->internal_csi_format->code;
>  	fmt->field = V4L2_FIELD_NONE;
>  
> -	if (ssd == sensor->pixel_array) {
> -		if (is_active)
> -			sensor->pa_src = *crop;
> -
> +	if (ssd == sensor->pixel_array)
>  		return 0;
> -	}
>  
>  	fmt = v4l2_subdev_state_get_format(sd_state, CCS_PAD_SRC,
>  					   CCS_STREAM_PIXEL);
> @@ -3333,20 +3323,6 @@ static int __ccs_init_state(struct v4l2_subdev *sd,
>  	return 0;
>  }
>  
> -static int ccs_init_state(struct v4l2_subdev *sd,
> -			  struct v4l2_subdev_state *sd_state)
> -{
> -	struct ccs_subdev *ssd = to_ccs_subdev(sd);
> -	struct ccs_sensor *sensor = ssd->sensor;
> -	int rval;
> -
> -	mutex_lock(&sensor->mutex);
> -	rval = __ccs_init_state(sd, sd_state);
> -	mutex_unlock(&sensor->mutex);
> -
> -	return rval;
> -}
> -
>  static int ccs_src_init_state(struct v4l2_subdev *sd,
>  			      struct v4l2_subdev_state *sd_state)
>  {
> @@ -3371,25 +3347,20 @@ static int ccs_src_init_state(struct v4l2_subdev *sd,
>  	struct ccs_sensor *sensor = ssd->sensor;
>  	int rval;
>  
> -	mutex_lock(&sensor->mutex);
> -
>  	if (ccs_embedded_data_lines(sensor))
>  		routing.num_routes++;
>  
>  	rval = v4l2_subdev_set_routing(sd, sd_state, &routing);
>  	if (rval)
> -		goto out;
> +		return 0;
>  
> -	rval = __ccs_init_state(sd, sd_state);
> +	rval = ccs_init_state(sd, sd_state);
>  	if (rval)
> -		goto out;
> +		return 0;
>  
>  	if (ccs_embedded_data_lines(sensor))
>  		ccs_set_format_meta(sd, sd_state, NULL);
>  
> -out:
> -	mutex_unlock(&sensor->mutex);
> -
>  	return rval;
>  }
>  
> diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
> index 1bc524de3aa2..338209950fcb 100644
> --- a/drivers/media/i2c/ccs/ccs.h
> +++ b/drivers/media/i2c/ccs/ccs.h
> @@ -232,7 +232,6 @@ struct ccs_sensor {
>  	u32 mbus_frame_fmts;
>  	const struct ccs_csi_data_format *csi_format;
>  	const struct ccs_csi_data_format *internal_csi_format;
> -	struct v4l2_rect pa_src, scaler_sink, src_src;
>  	u32 default_mbus_frame_fmts;
>  	int default_pixel_order;
>  	struct ccs_data_container sdata, mdata;

-- 
Regards,

Laurent Pinchart

[PATCH v11 33/66] media: ccs: Compute binning configuration from sub-device state

[Sakari Ailus]

Calculate binning configuration from sub-device state so the state related
configuration can be removed from the driver's device context struct.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 56 +++++++++++++++++++-------------
 drivers/media/i2c/ccs/ccs.h      |  3 --
 2 files changed, 33 insertions(+), 26 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 2d2ae568739b..1eb63956ef1f 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -511,13 +511,36 @@ static int ccs_pll_try(struct ccs_sensor *sensor, struct ccs_pll *pll)
 	return ccs_pll_calculate(&client->dev, &lim, pll);
 }
 
+static void
+ccs_get_binning(struct ccs_sensor *sensor, u8 *binning_mode, u8 *binh, u8 *binv)
+{
+	struct v4l2_subdev_state *state =
+		v4l2_subdev_get_locked_active_state(&sensor->binner->sd);
+	const struct v4l2_rect *sink_crop =
+		v4l2_subdev_state_get_crop(state, CCS_PAD_SINK,
+					   CCS_STREAM_PIXEL);
+	const struct v4l2_rect *sink_comp =
+		v4l2_subdev_state_get_compose(state, CCS_PAD_SINK,
+					      CCS_STREAM_PIXEL);
+
+	if (binning_mode)
+		*binning_mode =	sink_crop->width == sink_comp->width &&
+				sink_crop->height == sink_comp->height ? 0 : 1;
+
+	*binh = sink_crop->width / sink_comp->width;
+	*binv = sink_crop->height / sink_comp->height;
+}
+
 static int ccs_pll_update(struct ccs_sensor *sensor)
 {
 	struct ccs_pll *pll = &sensor->pll;
+	u8 binh, binv;
 	int rval;
 
-	pll->binning_horizontal = sensor->binning_horizontal;
-	pll->binning_vertical = sensor->binning_vertical;
+	ccs_get_binning(sensor, NULL, &binh, &binv);
+
+	pll->binning_horizontal = binh;
+	pll->binning_vertical = binv;
 	pll->link_freq =
 		sensor->link_freq->qmenu_int[sensor->link_freq->val];
 	pll->scale_m = sensor->scale_m;
@@ -1258,8 +1281,11 @@ static void ccs_update_blanking(struct ccs_sensor *sensor,
 	struct v4l2_ctrl *hblank = sensor->hblank;
 	u16 min_fll, max_fll, min_llp, max_llp, min_lbp;
 	int min, max;
+	u8 binh, binv;
+
+	ccs_get_binning(sensor, NULL, &binh, &binv);
 
-	if (sensor->binning_vertical > 1 || sensor->binning_horizontal > 1) {
+	if (binv > 1 || binh > 1) {
 		min_fll = CCS_LIM(sensor, MIN_FRAME_LENGTH_LINES_BIN);
 		max_fll = CCS_LIM(sensor, MAX_FRAME_LENGTH_LINES_BIN);
 		min_llp = CCS_LIM(sensor, MIN_LINE_LENGTH_PCK_BIN);
@@ -1827,7 +1853,7 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 		v4l2_subdev_state_get_crop(src_state, CCS_PAD_SRC,
 					   CCS_STREAM_PIXEL);
 	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
-	unsigned int binning_mode;
+	u8 binning_mode, binh, binv;
 	int rval;
 
 	if (pad != CCS_PAD_SRC)
@@ -1849,19 +1875,12 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 		goto err_pm_put;
 
 	/* Binning configuration */
-	if (sensor->binning_horizontal == 1 &&
-	    sensor->binning_vertical == 1) {
-		binning_mode = 0;
-	} else {
-		u8 binning_type =
-			(sensor->binning_horizontal << 4)
-			| sensor->binning_vertical;
+	ccs_get_binning(sensor,	&binning_mode, &binh, &binv);
 
-		rval = ccs_write(sensor, BINNING_TYPE, binning_type);
+	if (binning_mode) {
+		rval = ccs_write(sensor, BINNING_TYPE, (binh << 4) | binv);
 		if (rval < 0)
 			goto err_pm_put;
-
-		binning_mode = 1;
 	}
 	rval = ccs_write(sensor, BINNING_MODE, binning_mode);
 	if (rval < 0)
@@ -2260,9 +2279,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 				sensor->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
 				sensor->scaling_mode =
 					CCS_SCALING_MODE_NO_SCALING;
-			} else if (ssd == sensor->binner) {
-				sensor->binning_horizontal = 1;
-				sensor->binning_vertical = 1;
 			}
 		}
 		fallthrough;
@@ -2542,10 +2558,6 @@ static void ccs_set_compose_binner(struct v4l2_subdev *subdev,
 			best = this;
 		}
 	}
-	if (sel->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
-		sensor->binning_vertical = binv;
-		sensor->binning_horizontal = binh;
-	}
 
 	sel->r.width = (sink_crop->width / binh) & ~1;
 	sel->r.height = (sink_crop->height / binv) & ~1;
@@ -3736,8 +3748,6 @@ static int ccs_probe(struct i2c_client *client)
 				sensor->binning_subtypes[i].vertical);
 		}
 	}
-	sensor->binning_horizontal = 1;
-	sensor->binning_vertical = 1;
 
 	if (device_create_file(&client->dev, &dev_attr_ident) != 0) {
 		dev_err(&client->dev, "sysfs ident entry creation failed\n");
diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
index 338209950fcb..9671883e4885 100644
--- a/drivers/media/i2c/ccs/ccs.h
+++ b/drivers/media/i2c/ccs/ccs.h
@@ -238,9 +238,6 @@ struct ccs_sensor {
 
 	u8 emb_data_ctrl;
 
-	u8 binning_horizontal;
-	u8 binning_vertical;
-
 	u8 scale_m;
 	u8 scaling_mode;
 
-- 
2.47.2

Re: [PATCH v11 33/66] media: ccs: Compute binning configuration from sub-device state

[Laurent Pinchart]

On Mon, Aug 25, 2025 at 12:50:34PM +0300, Sakari Ailus wrote:
> Calculate binning configuration from sub-device state so the state related
> configuration can be removed from the driver's device context struct.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

I gave a

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

on v9.

> ---
>  drivers/media/i2c/ccs/ccs-core.c | 56 +++++++++++++++++++-------------
>  drivers/media/i2c/ccs/ccs.h      |  3 --
>  2 files changed, 33 insertions(+), 26 deletions(-)
> 
> diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
> index 2d2ae568739b..1eb63956ef1f 100644
> --- a/drivers/media/i2c/ccs/ccs-core.c
> +++ b/drivers/media/i2c/ccs/ccs-core.c
> @@ -511,13 +511,36 @@ static int ccs_pll_try(struct ccs_sensor *sensor, struct ccs_pll *pll)
>  	return ccs_pll_calculate(&client->dev, &lim, pll);
>  }
>  
> +static void
> +ccs_get_binning(struct ccs_sensor *sensor, u8 *binning_mode, u8 *binh, u8 *binv)
> +{
> +	struct v4l2_subdev_state *state =
> +		v4l2_subdev_get_locked_active_state(&sensor->binner->sd);
> +	const struct v4l2_rect *sink_crop =
> +		v4l2_subdev_state_get_crop(state, CCS_PAD_SINK,
> +					   CCS_STREAM_PIXEL);
> +	const struct v4l2_rect *sink_comp =
> +		v4l2_subdev_state_get_compose(state, CCS_PAD_SINK,
> +					      CCS_STREAM_PIXEL);
> +
> +	if (binning_mode)
> +		*binning_mode =	sink_crop->width == sink_comp->width &&
> +				sink_crop->height == sink_comp->height ? 0 : 1;
> +
> +	*binh = sink_crop->width / sink_comp->width;
> +	*binv = sink_crop->height / sink_comp->height;
> +}
> +
>  static int ccs_pll_update(struct ccs_sensor *sensor)
>  {
>  	struct ccs_pll *pll = &sensor->pll;
> +	u8 binh, binv;
>  	int rval;
>  
> -	pll->binning_horizontal = sensor->binning_horizontal;
> -	pll->binning_vertical = sensor->binning_vertical;
> +	ccs_get_binning(sensor, NULL, &binh, &binv);
> +
> +	pll->binning_horizontal = binh;
> +	pll->binning_vertical = binv;
>  	pll->link_freq =
>  		sensor->link_freq->qmenu_int[sensor->link_freq->val];
>  	pll->scale_m = sensor->scale_m;
> @@ -1258,8 +1281,11 @@ static void ccs_update_blanking(struct ccs_sensor *sensor,
>  	struct v4l2_ctrl *hblank = sensor->hblank;
>  	u16 min_fll, max_fll, min_llp, max_llp, min_lbp;
>  	int min, max;
> +	u8 binh, binv;
> +
> +	ccs_get_binning(sensor, NULL, &binh, &binv);
>  
> -	if (sensor->binning_vertical > 1 || sensor->binning_horizontal > 1) {
> +	if (binv > 1 || binh > 1) {
>  		min_fll = CCS_LIM(sensor, MIN_FRAME_LENGTH_LINES_BIN);
>  		max_fll = CCS_LIM(sensor, MAX_FRAME_LENGTH_LINES_BIN);
>  		min_llp = CCS_LIM(sensor, MIN_LINE_LENGTH_PCK_BIN);
> @@ -1827,7 +1853,7 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
>  		v4l2_subdev_state_get_crop(src_state, CCS_PAD_SRC,
>  					   CCS_STREAM_PIXEL);
>  	struct i2c_client *client = v4l2_get_subdevdata(&sensor->src->sd);
> -	unsigned int binning_mode;
> +	u8 binning_mode, binh, binv;
>  	int rval;
>  
>  	if (pad != CCS_PAD_SRC)
> @@ -1849,19 +1875,12 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
>  		goto err_pm_put;
>  
>  	/* Binning configuration */
> -	if (sensor->binning_horizontal == 1 &&
> -	    sensor->binning_vertical == 1) {
> -		binning_mode = 0;
> -	} else {
> -		u8 binning_type =
> -			(sensor->binning_horizontal << 4)
> -			| sensor->binning_vertical;
> +	ccs_get_binning(sensor,	&binning_mode, &binh, &binv);
>  
> -		rval = ccs_write(sensor, BINNING_TYPE, binning_type);
> +	if (binning_mode) {
> +		rval = ccs_write(sensor, BINNING_TYPE, (binh << 4) | binv);
>  		if (rval < 0)
>  			goto err_pm_put;
> -
> -		binning_mode = 1;
>  	}
>  	rval = ccs_write(sensor, BINNING_MODE, binning_mode);
>  	if (rval < 0)
> @@ -2260,9 +2279,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
>  				sensor->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
>  				sensor->scaling_mode =
>  					CCS_SCALING_MODE_NO_SCALING;
> -			} else if (ssd == sensor->binner) {
> -				sensor->binning_horizontal = 1;
> -				sensor->binning_vertical = 1;
>  			}
>  		}
>  		fallthrough;
> @@ -2542,10 +2558,6 @@ static void ccs_set_compose_binner(struct v4l2_subdev *subdev,
>  			best = this;
>  		}
>  	}
> -	if (sel->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> -		sensor->binning_vertical = binv;
> -		sensor->binning_horizontal = binh;
> -	}
>  
>  	sel->r.width = (sink_crop->width / binh) & ~1;
>  	sel->r.height = (sink_crop->height / binv) & ~1;
> @@ -3736,8 +3748,6 @@ static int ccs_probe(struct i2c_client *client)
>  				sensor->binning_subtypes[i].vertical);
>  		}
>  	}
> -	sensor->binning_horizontal = 1;
> -	sensor->binning_vertical = 1;
>  
>  	if (device_create_file(&client->dev, &dev_attr_ident) != 0) {
>  		dev_err(&client->dev, "sysfs ident entry creation failed\n");
> diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
> index 338209950fcb..9671883e4885 100644
> --- a/drivers/media/i2c/ccs/ccs.h
> +++ b/drivers/media/i2c/ccs/ccs.h
> @@ -238,9 +238,6 @@ struct ccs_sensor {
>  
>  	u8 emb_data_ctrl;
>  
> -	u8 binning_horizontal;
> -	u8 binning_vertical;
> -
>  	u8 scale_m;
>  	u8 scaling_mode;
>  

-- 
Regards,

Laurent Pinchart

[PATCH v11 34/66] media: ccs: Compute scaling configuration from sub-device state

[Sakari Ailus]

Compute scaling configuration from sub-device state instead of storing it
to the driver's device context struct.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 59 ++++++++++++++++++++++----------
 drivers/media/i2c/ccs/ccs.h      |  3 --
 2 files changed, 40 insertions(+), 22 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 1eb63956ef1f..7b451684e1ed 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -531,19 +531,51 @@ ccs_get_binning(struct ccs_sensor *sensor, u8 *binning_mode, u8 *binh, u8 *binv)
 	*binv = sink_crop->height / sink_comp->height;
 }
 
+static void ccs_get_scaling(struct ccs_sensor *sensor, u8 *scaling_mode,
+			    u8 *scale_m)
+{
+	struct v4l2_subdev_state *state =
+		v4l2_subdev_get_locked_active_state(&sensor->scaler->sd);
+	const struct v4l2_rect *sink_crop =
+		v4l2_subdev_state_get_crop(state, CCS_PAD_SINK,
+					   CCS_STREAM_PIXEL);
+	const struct v4l2_rect *sink_comp =
+		v4l2_subdev_state_get_compose(state, CCS_PAD_SINK,
+					      CCS_STREAM_PIXEL);
+
+	*scale_m = sink_crop->width * CCS_LIM(sensor, SCALER_N_MIN) /
+		sink_comp->width;
+
+	if (!scaling_mode)
+		return;
+
+	if (sink_crop->width == sink_comp->width)
+		*scaling_mode = CCS_SCALING_MODE_NO_SCALING;
+	else if (sink_crop->height == sink_comp->height)
+		*scaling_mode = CCS_SCALING_MODE_HORIZONTAL;
+	else
+		*scaling_mode = SMIAPP_SCALING_MODE_BOTH;
+}
+
 static int ccs_pll_update(struct ccs_sensor *sensor)
 {
 	struct ccs_pll *pll = &sensor->pll;
 	u8 binh, binv;
+	u8 scale_m;
 	int rval;
 
 	ccs_get_binning(sensor, NULL, &binh, &binv);
 
+	if (sensor->scaler)
+		ccs_get_scaling(sensor, NULL, &scale_m);
+	else
+		scale_m = CCS_LIM(sensor, SCALER_N_MIN);
+
 	pll->binning_horizontal = binh;
 	pll->binning_vertical = binv;
 	pll->link_freq =
 		sensor->link_freq->qmenu_int[sensor->link_freq->val];
-	pll->scale_m = sensor->scale_m;
+	pll->scale_m = scale_m;
 	pll->bits_per_pixel = sensor->csi_format->compressed;
 
 	rval = ccs_pll_try(sensor, pll);
@@ -1203,7 +1235,7 @@ static int ccs_get_mbus_formats(struct ccs_sensor *sensor)
 	/* Figure out which BPP values can be used with which formats. */
 	pll->binning_horizontal = 1;
 	pll->binning_vertical = 1;
-	pll->scale_m = sensor->scale_m;
+	pll->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
 
 	for (i = 0; i < ARRAY_SIZE(ccs_csi_data_formats); i++) {
 		sensor->compressed_min_bpp =
@@ -1948,11 +1980,15 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 	/* Scaling */
 	if (CCS_LIM(sensor, SCALING_CAPABILITY)
 	    != CCS_SCALING_CAPABILITY_NONE) {
-		rval = ccs_write(sensor, SCALING_MODE, sensor->scaling_mode);
+		u8 scaling_mode, scale_m;
+
+		ccs_get_scaling(sensor, &scaling_mode, &scale_m);
+
+		rval = ccs_write(sensor, SCALING_MODE, scaling_mode);
 		if (rval < 0)
 			goto err_pm_put;
 
-		rval = ccs_write(sensor, SCALE_M, sensor->scale_m);
+		rval = ccs_write(sensor, SCALE_M, scale_m);
 		if (rval < 0)
 			goto err_pm_put;
 	}
@@ -2261,7 +2297,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_state *sd_state, int which,
 			  int target)
 {
-	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
 	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
 	struct v4l2_rect *comp, *crop;
 	struct v4l2_mbus_framefmt *fmt;
@@ -2274,13 +2309,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
 						  CCS_STREAM_PIXEL);
 		comp->width = crop->width;
 		comp->height = crop->height;
-		if (which == V4L2_SUBDEV_FORMAT_ACTIVE) {
-			if (ssd == sensor->scaler) {
-				sensor->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
-				sensor->scaling_mode =
-					CCS_SCALING_MODE_NO_SCALING;
-			}
-		}
 		fallthrough;
 	case V4L2_SEL_TGT_COMPOSE:
 		crop = v4l2_subdev_state_get_crop(sd_state, CCS_PAD_SRC,
@@ -2665,11 +2693,6 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,
 				 * CCS_LIM(sensor, SCALER_N_MIN)) & ~1;
 	else
 		sel->r.height = sink_crop->height;
-
-	if (sel->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
-		sensor->scale_m = scale_m;
-		sensor->scaling_mode = mode;
-	}
 }
 /* We're only called on source pads. This function sets scaling. */
 static int ccs_set_compose(struct v4l2_subdev *subdev,
@@ -3784,8 +3807,6 @@ static int ccs_probe(struct i2c_client *client)
 	sensor->pixel_array = &sensor->ssds[sensor->ssds_used];
 	sensor->ssds_used++;
 
-	sensor->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
-
 	/* prepare PLL configuration input values */
 	sensor->pll.bus_type = CCS_PLL_BUS_TYPE_CSI2_DPHY;
 	sensor->pll.csi2.lanes = sensor->hwcfg.lanes;
diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
index 9671883e4885..3fddbe267409 100644
--- a/drivers/media/i2c/ccs/ccs.h
+++ b/drivers/media/i2c/ccs/ccs.h
@@ -238,9 +238,6 @@ struct ccs_sensor {
 
 	u8 emb_data_ctrl;
 
-	u8 scale_m;
-	u8 scaling_mode;
-
 	u8 frame_skip;
 	u16 embedded_start; /* embedded data start line */
 	u16 embedded_end;
-- 
2.47.2

Re: [PATCH v11 34/66] media: ccs: Compute scaling configuration from sub-device state

[Laurent Pinchart]

Hi Sakari,

You seem to have missed taking my review comments from v9 into account
here. As that was the case for other ccs patches earlier in this series
I'll assume the rest of the ccs patches are also affected and will skip
reviewing them. Please look at v9 and address the comments in v12.

On Mon, Aug 25, 2025 at 12:50:35PM +0300, Sakari Ailus wrote:
> Compute scaling configuration from sub-device state instead of storing it
> to the driver's device context struct.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/i2c/ccs/ccs-core.c | 59 ++++++++++++++++++++++----------
>  drivers/media/i2c/ccs/ccs.h      |  3 --
>  2 files changed, 40 insertions(+), 22 deletions(-)
> 
> diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
> index 1eb63956ef1f..7b451684e1ed 100644
> --- a/drivers/media/i2c/ccs/ccs-core.c
> +++ b/drivers/media/i2c/ccs/ccs-core.c
> @@ -531,19 +531,51 @@ ccs_get_binning(struct ccs_sensor *sensor, u8 *binning_mode, u8 *binh, u8 *binv)
>  	*binv = sink_crop->height / sink_comp->height;
>  }
>  
> +static void ccs_get_scaling(struct ccs_sensor *sensor, u8 *scaling_mode,
> +			    u8 *scale_m)
> +{
> +	struct v4l2_subdev_state *state =
> +		v4l2_subdev_get_locked_active_state(&sensor->scaler->sd);
> +	const struct v4l2_rect *sink_crop =
> +		v4l2_subdev_state_get_crop(state, CCS_PAD_SINK,
> +					   CCS_STREAM_PIXEL);
> +	const struct v4l2_rect *sink_comp =
> +		v4l2_subdev_state_get_compose(state, CCS_PAD_SINK,
> +					      CCS_STREAM_PIXEL);
> +
> +	*scale_m = sink_crop->width * CCS_LIM(sensor, SCALER_N_MIN) /
> +		sink_comp->width;
> +
> +	if (!scaling_mode)
> +		return;
> +
> +	if (sink_crop->width == sink_comp->width)
> +		*scaling_mode = CCS_SCALING_MODE_NO_SCALING;
> +	else if (sink_crop->height == sink_comp->height)
> +		*scaling_mode = CCS_SCALING_MODE_HORIZONTAL;
> +	else
> +		*scaling_mode = SMIAPP_SCALING_MODE_BOTH;
> +}
> +
>  static int ccs_pll_update(struct ccs_sensor *sensor)
>  {
>  	struct ccs_pll *pll = &sensor->pll;
>  	u8 binh, binv;
> +	u8 scale_m;
>  	int rval;
>  
>  	ccs_get_binning(sensor, NULL, &binh, &binv);
>  
> +	if (sensor->scaler)
> +		ccs_get_scaling(sensor, NULL, &scale_m);
> +	else
> +		scale_m = CCS_LIM(sensor, SCALER_N_MIN);
> +
>  	pll->binning_horizontal = binh;
>  	pll->binning_vertical = binv;
>  	pll->link_freq =
>  		sensor->link_freq->qmenu_int[sensor->link_freq->val];
> -	pll->scale_m = sensor->scale_m;
> +	pll->scale_m = scale_m;
>  	pll->bits_per_pixel = sensor->csi_format->compressed;
>  
>  	rval = ccs_pll_try(sensor, pll);
> @@ -1203,7 +1235,7 @@ static int ccs_get_mbus_formats(struct ccs_sensor *sensor)
>  	/* Figure out which BPP values can be used with which formats. */
>  	pll->binning_horizontal = 1;
>  	pll->binning_vertical = 1;
> -	pll->scale_m = sensor->scale_m;
> +	pll->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
>  
>  	for (i = 0; i < ARRAY_SIZE(ccs_csi_data_formats); i++) {
>  		sensor->compressed_min_bpp =
> @@ -1948,11 +1980,15 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
>  	/* Scaling */
>  	if (CCS_LIM(sensor, SCALING_CAPABILITY)
>  	    != CCS_SCALING_CAPABILITY_NONE) {
> -		rval = ccs_write(sensor, SCALING_MODE, sensor->scaling_mode);
> +		u8 scaling_mode, scale_m;
> +
> +		ccs_get_scaling(sensor, &scaling_mode, &scale_m);
> +
> +		rval = ccs_write(sensor, SCALING_MODE, scaling_mode);
>  		if (rval < 0)
>  			goto err_pm_put;
>  
> -		rval = ccs_write(sensor, SCALE_M, sensor->scale_m);
> +		rval = ccs_write(sensor, SCALE_M, scale_m);
>  		if (rval < 0)
>  			goto err_pm_put;
>  	}
> @@ -2261,7 +2297,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
>  			  struct v4l2_subdev_state *sd_state, int which,
>  			  int target)
>  {
> -	struct ccs_sensor *sensor = to_ccs_sensor(subdev);
>  	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
>  	struct v4l2_rect *comp, *crop;
>  	struct v4l2_mbus_framefmt *fmt;
> @@ -2274,13 +2309,6 @@ static void ccs_propagate(struct v4l2_subdev *subdev,
>  						  CCS_STREAM_PIXEL);
>  		comp->width = crop->width;
>  		comp->height = crop->height;
> -		if (which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> -			if (ssd == sensor->scaler) {
> -				sensor->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
> -				sensor->scaling_mode =
> -					CCS_SCALING_MODE_NO_SCALING;
> -			}
> -		}
>  		fallthrough;
>  	case V4L2_SEL_TGT_COMPOSE:
>  		crop = v4l2_subdev_state_get_crop(sd_state, CCS_PAD_SRC,
> @@ -2665,11 +2693,6 @@ static void ccs_set_compose_scaler(struct v4l2_subdev *subdev,
>  				 * CCS_LIM(sensor, SCALER_N_MIN)) & ~1;
>  	else
>  		sel->r.height = sink_crop->height;
> -
> -	if (sel->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
> -		sensor->scale_m = scale_m;
> -		sensor->scaling_mode = mode;
> -	}
>  }
>  /* We're only called on source pads. This function sets scaling. */
>  static int ccs_set_compose(struct v4l2_subdev *subdev,
> @@ -3784,8 +3807,6 @@ static int ccs_probe(struct i2c_client *client)
>  	sensor->pixel_array = &sensor->ssds[sensor->ssds_used];
>  	sensor->ssds_used++;
>  
> -	sensor->scale_m = CCS_LIM(sensor, SCALER_N_MIN);
> -
>  	/* prepare PLL configuration input values */
>  	sensor->pll.bus_type = CCS_PLL_BUS_TYPE_CSI2_DPHY;
>  	sensor->pll.csi2.lanes = sensor->hwcfg.lanes;
> diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
> index 9671883e4885..3fddbe267409 100644
> --- a/drivers/media/i2c/ccs/ccs.h
> +++ b/drivers/media/i2c/ccs/ccs.h
> @@ -238,9 +238,6 @@ struct ccs_sensor {
>  
>  	u8 emb_data_ctrl;
>  
> -	u8 scale_m;
> -	u8 scaling_mode;
> -
>  	u8 frame_skip;
>  	u16 embedded_start; /* embedded data start line */
>  	u16 embedded_end;

-- 
Regards,

Laurent Pinchart

[PATCH v11 35/66] media: ccs: Remove which parameter from ccs_propagate

[Sakari Ailus]

ccs_propagate() no longer stores information in the driver's context
struct. The which parameter can thus be removed.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 7b451684e1ed..1193cde1c98c 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -2294,8 +2294,7 @@ static int ccs_get_format(struct v4l2_subdev *subdev,
 
 /* Changes require propagation only on sink pad. */
 static void ccs_propagate(struct v4l2_subdev *subdev,
-			  struct v4l2_subdev_state *sd_state, int which,
-			  int target)
+			  struct v4l2_subdev_state *sd_state, int target)
 {
 	struct ccs_subdev *ssd = to_ccs_subdev(subdev);
 	struct v4l2_rect *comp, *crop;
@@ -2510,7 +2509,7 @@ static int ccs_set_format(struct v4l2_subdev *subdev,
 	crop->top = 0;
 	crop->width = fmt->format.width;
 	crop->height = fmt->format.height;
-	ccs_propagate(subdev, sd_state, fmt->which, V4L2_SEL_TGT_CROP);
+	ccs_propagate(subdev, sd_state, V4L2_SEL_TGT_CROP);
 
 	return 0;
 }
@@ -2718,7 +2717,7 @@ static int ccs_set_compose(struct v4l2_subdev *subdev,
 		ccs_set_compose_scaler(subdev, sd_state, sel, sink_crop);
 
 	*comp = sel->r;
-	ccs_propagate(subdev, sd_state, sel->which, V4L2_SEL_TGT_COMPOSE);
+	ccs_propagate(subdev, sd_state, V4L2_SEL_TGT_COMPOSE);
 
 	if (sel->which == V4L2_SUBDEV_FORMAT_ACTIVE)
 		return ccs_pll_blanking_update(sensor);
@@ -2809,7 +2808,7 @@ static int ccs_set_crop(struct v4l2_subdev *subdev,
 	*crop = sel->r;
 
 	if (ssd != sensor->pixel_array && sel->pad == CCS_PAD_SINK)
-		ccs_propagate(subdev, sd_state, sel->which, V4L2_SEL_TGT_CROP);
+		ccs_propagate(subdev, sd_state, V4L2_SEL_TGT_CROP);
 
 	return 0;
 }
@@ -3335,7 +3334,6 @@ static int ccs_init_state(struct v4l2_subdev *sd,
 		v4l2_subdev_state_get_format(sd_state, pad, CCS_STREAM_PIXEL);
 	struct v4l2_rect *crop =
 		v4l2_subdev_state_get_crop(sd_state, pad, CCS_STREAM_PIXEL);
-	bool is_active = !sd->active_state || sd->active_state == sd_state;
 
 	ccs_get_native_size(ssd, crop);
 
@@ -3353,7 +3351,7 @@ static int ccs_init_state(struct v4l2_subdev *sd,
 		sensor->csi_format->code : sensor->internal_csi_format->code;
 	fmt->field = V4L2_FIELD_NONE;
 
-	ccs_propagate(sd, sd_state, is_active, V4L2_SEL_TGT_CROP);
+	ccs_propagate(sd, sd_state, V4L2_SEL_TGT_CROP);
 
 	return 0;
 }
-- 
2.47.2

[PATCH v11 36/66] media: ccs: Detemine emb_data_ctrl register from sub-device state

[Sakari Ailus]

Determine the value to write to emb_data_ctrl register from sub-device
state.

This also moves up the embedded data definitions. There are no changes to
the definitions themselves.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 118 ++++++++++++++++---------------
 drivers/media/i2c/ccs/ccs.h      |   2 -
 2 files changed, 62 insertions(+), 58 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 1193cde1c98c..87a1614a020b 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -1869,6 +1869,57 @@ static int ccs_pm_get_init(struct ccs_sensor *sensor)
 	return rval;
 }
 
+#define CCS_EMBEDDED_CODE_DEPTH(depth, half_depth)			\
+	depth,								\
+	CCS_EMB_DATA_CAPABILITY_TWO_BYTES_PER_RAW##depth,		\
+	CCS_EMB_DATA_CAPABILITY_NO_ONE_BYTE_PER_RAW##depth,		\
+	CCS_EMB_DATA_CTRL_RAW##half_depth##_PACKING_FOR_RAW##depth,	\
+	MEDIA_BUS_FMT_META_##half_depth,				\
+	MEDIA_BUS_FMT_META_##depth,					\
+
+static const struct ccs_embedded_code {
+	u8 depth;
+	u8 cap_two_bytes_per_sample;
+	u8 cap_no_legacy;
+	u8 ctrl;
+	u32 code_two_bytes;
+	u32 code_legacy;
+} ccs_embedded_codes[] = {
+	{ CCS_EMBEDDED_CODE_DEPTH(16, 8) },
+	{ CCS_EMBEDDED_CODE_DEPTH(20, 10) },
+	{ CCS_EMBEDDED_CODE_DEPTH(24, 12) },
+};
+
+static const struct ccs_embedded_code *ccs_embedded_code(unsigned int bpp)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(ccs_embedded_codes); i++)
+		if (ccs_embedded_codes[i].depth == bpp)
+			return ccs_embedded_codes + i;
+
+	WARN_ON(1);
+
+	return ccs_embedded_codes;
+}
+
+static u32
+ccs_default_embedded_code(struct ccs_sensor *sensor,
+			  const struct ccs_embedded_code *embedded_code)
+{
+	if (CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
+	    BIT(embedded_code->cap_two_bytes_per_sample))
+		return embedded_code->code_two_bytes;
+
+	if (!(CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
+	      BIT(embedded_code->cap_no_legacy)))
+		return embedded_code->code_legacy;
+
+	WARN_ON(1);
+
+	return embedded_code->code_legacy;
+}
+
 static int ccs_enable_streams(struct v4l2_subdev *subdev,
 			      struct v4l2_subdev_state *state, u32 pad,
 			      u64 streams_mask)
@@ -2003,7 +2054,16 @@ static int ccs_enable_streams(struct v4l2_subdev *subdev,
 
 	/* Configure embedded data */
 	if (sensor->csi_format->compressed >= 16) {
-		rval = ccs_write(sensor, EMB_DATA_CTRL, sensor->emb_data_ctrl);
+		const struct ccs_embedded_code *embedded_code =
+			ccs_embedded_code(sensor->csi_format->compressed);
+		const struct v4l2_mbus_framefmt *meta_out_fmt =
+			v4l2_subdev_state_get_format(src_state, CCS_PAD_SRC,
+						     CCS_STREAM_META);
+
+		rval = ccs_write(sensor, EMB_DATA_CTRL,
+				 meta_out_fmt->code ==
+				 embedded_code->code_legacy ?
+				 0 : embedded_code->ctrl);
 		if (rval < 0)
 			goto err_pm_put;
 	}
@@ -2127,57 +2187,6 @@ static const struct ccs_csi_data_format
 	return sensor->csi_format;
 }
 
-#define CCS_EMBEDDED_CODE_DEPTH(depth, half_depth)			\
-	depth,								\
-	CCS_EMB_DATA_CAPABILITY_TWO_BYTES_PER_RAW##depth,		\
-	CCS_EMB_DATA_CAPABILITY_NO_ONE_BYTE_PER_RAW##depth,		\
-	CCS_EMB_DATA_CTRL_RAW##half_depth##_PACKING_FOR_RAW##depth,	\
-	MEDIA_BUS_FMT_META_##half_depth,				\
-	MEDIA_BUS_FMT_META_##depth,					\
-
-static const struct ccs_embedded_code {
-	u8 depth;
-	u8 cap_two_bytes_per_sample;
-	u8 cap_no_legacy;
-	u8 ctrl;
-	u32 code_two_bytes;
-	u32 code_legacy;
-} ccs_embedded_codes[] = {
-	{ CCS_EMBEDDED_CODE_DEPTH(16, 8) },
-	{ CCS_EMBEDDED_CODE_DEPTH(20, 10) },
-	{ CCS_EMBEDDED_CODE_DEPTH(24, 12) },
-};
-
-static const struct ccs_embedded_code *ccs_embedded_code(unsigned int bpp)
-{
-	unsigned int i;
-
-	for (i = 0; i < ARRAY_SIZE(ccs_embedded_codes); i++)
-		if (ccs_embedded_codes[i].depth == bpp)
-			return ccs_embedded_codes + i;
-
-	WARN_ON(1);
-
-	return ccs_embedded_codes;
-}
-
-static u32
-ccs_default_embedded_code(struct ccs_sensor *sensor,
-			  const struct ccs_embedded_code *embedded_code)
-{
-	if (CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
-	    BIT(embedded_code->cap_two_bytes_per_sample))
-		return embedded_code->code_two_bytes;
-
-	if (!(CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
-	      BIT(embedded_code->cap_no_legacy)))
-		return embedded_code->code_legacy;
-
-	WARN_ON(1);
-
-	return embedded_code->code_legacy;
-}
-
 static int ccs_enum_mbus_code(struct v4l2_subdev *subdev,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_mbus_code_enum *code)
@@ -2429,16 +2438,13 @@ static int ccs_set_format_meta(struct v4l2_subdev *subdev,
 
 		if (!(CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
 		      BIT(embedded_code->cap_no_legacy)) &&
-		    code == embedded_code->code_legacy) {
+		    code == embedded_code->code_legacy)
 			meta_fmt->code = embedded_code->code_legacy;
-			sensor->emb_data_ctrl = 0;
-		}
 
 		if (CCS_LIM(sensor, EMB_DATA_CAPABILITY) &
 		    BIT(embedded_code->cap_two_bytes_per_sample) &&
 		    code == embedded_code->code_two_bytes) {
 			meta_fmt->code = embedded_code->code_two_bytes;
-			sensor->emb_data_ctrl = embedded_code->ctrl;
 			meta_fmt->width <<= 1;
 		}
 
diff --git a/drivers/media/i2c/ccs/ccs.h b/drivers/media/i2c/ccs/ccs.h
index 3fddbe267409..27d38cf927ac 100644
--- a/drivers/media/i2c/ccs/ccs.h
+++ b/drivers/media/i2c/ccs/ccs.h
@@ -236,8 +236,6 @@ struct ccs_sensor {
 	int default_pixel_order;
 	struct ccs_data_container sdata, mdata;
 
-	u8 emb_data_ctrl;
-
 	u8 frame_skip;
 	u16 embedded_start; /* embedded data start line */
 	u16 embedded_end;
-- 
2.47.2

[PATCH v11 37/66] media: uapi: Correct generic CSI-2 metadata format 4cc

[Sakari Ailus]

Rework the pixelformat 4cc for CSI-2 generic metadata, the 16- and 24-bit
variants are not specific to CSI-2. This can be done as no driver uses
this yet and the interface is disabled.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../media/v4l/metafmt-generic.rst             | 37 +++++++++----------
 .../media/pci/intel/ipu6/ipu6-isys-video.c    |  2 +-
 drivers/media/v4l2-core/v4l2-ioctl.c          |  8 ++--
 include/uapi/linux/videodev2.h                | 12 +++---
 4 files changed, 29 insertions(+), 30 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
index e8824b5a65d6..ca960907efde 100644
--- a/Documentation/userspace-api/media/v4l/metafmt-generic.rst
+++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
@@ -1,7 +1,7 @@
 .. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
 
 ********************************************************************************************************************************************************************************************************************************************************************************
-V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_META_FMT_GENERIC_CSI2_12 ('MC1C'), V4L2_META_FMT_GENERIC_CSI2_14 ('MC1E'), V4L2_META_FMT_GENERIC_CSI2_16 ('MC1G'), V4L2_META_FMT_GENERIC_CSI2_20 ('MC1K'), V4L2_META_FMT_GENERIC_CSI2_24 ('MC1O')
+V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MECA'), V4L2_META_FMT_GENERIC_CSI2_12 ('MECC'), V4L2_META_FMT_GENERIC_CSI2_14 ('MECE'), V4L2_META_FMT_GENERIC_16 ('METG'), V4L2_META_FMT_GENERIC_CSI2_20 ('MECK'), V4L2_META_FMT_GENERIC_24 ('METO')
 ********************************************************************************************************************************************************************************************************************************************************************************
 
 
@@ -29,7 +29,7 @@ is used on CSI-2 for 8 bits per :term:`Data Unit`.
 
 Additionally it is used for 16 bits per Data Unit when two bytes of metadata are
 packed into one 16-bit Data Unit. Otherwise the 16 bits per pixel dataformat is
-:ref:`V4L2_META_FMT_GENERIC_CSI2_16 <v4l2-meta-fmt-generic-csi2-16>`.
+:ref:`V4L2_META_FMT_GENERIC_16 <v4l2-meta-fmt-generic-16>`.
 
 **Byte Order Of V4L2_META_FMT_GENERIC_8.**
 Each cell is one byte. "M" denotes a byte of metadata.
@@ -112,8 +112,8 @@ the data is defined in the MIPI CCS specification.
 
 This format is also used in conjunction with 24 bits per :term:`Data Unit`
 formats that pack two bytes of metadata into one Data Unit. Otherwise the
-24 bits per pixel dataformat is :ref:`V4L2_META_FMT_GENERIC_CSI2_24
-<v4l2-meta-fmt-generic-csi2-24>`.
+24 bits per pixel dataformat is :ref:`V4L2_META_FMT_GENERIC_24
+<v4l2-meta-fmt-generic-24>`.
 
 This format is little endian.
 
@@ -185,14 +185,14 @@ Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
       - x
       - x
 
-.. _v4l2-meta-fmt-generic-csi2-16:
+.. _v4l2-meta-fmt-generic-16:
 
-V4L2_META_FMT_GENERIC_CSI2_16
------------------------------
+V4L2_META_FMT_GENERIC_16
+------------------------
 
-V4L2_META_FMT_GENERIC_CSI2_16 contains 8-bit generic metadata packed in 16-bit
-Data Units, with one padding byte after every byte of metadata. This format is
-typically used by CSI-2 receivers with a source that transmits
+V4L2_META_FMT_GENERIC_16 contains 8-bit generic metadata packed in 16-bit Data
+Units, with one padding byte after every byte of metadata. This format is
+also used by CSI-2 receivers with a source that transmits
 MEDIA_BUS_FMT_META_16 and the CSI-2 receiver writes the received data to memory
 as-is.
 
@@ -205,7 +205,7 @@ Some devices support more efficient packing of metadata in conjunction with
 
 This format is little endian.
 
-**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_16.**
+**Byte Order Of V4L2_META_FMT_GENERIC_16.**
 Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
 
 .. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|
@@ -287,16 +287,15 @@ Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
       - x
       - x
 
-.. _v4l2-meta-fmt-generic-csi2-24:
+.. _v4l2-meta-fmt-generic-24:
 
-V4L2_META_FMT_GENERIC_CSI2_24
------------------------------
+V4L2_META_FMT_GENERIC_24
+------------------------
 
-V4L2_META_FMT_GENERIC_CSI2_24 contains 8-bit generic metadata packed in 24-bit
-Data Units, with two padding bytes after every byte of metadata. This format is
-typically used by CSI-2 receivers with a source that transmits
-MEDIA_BUS_FMT_META_24 and the CSI-2 receiver writes the received data to memory
-as-is.
+V4L2_META_FMT_GENERIC_24 contains 8-bit generic metadata packed in 24-bit Data
+Units, with two padding bytes after every byte of metadata. This format is also
+used by CSI-2 receivers with a source that transmits MEDIA_BUS_FMT_META_24 and
+the CSI-2 receiver writes the received data to memory as-is.
 
 The packing of the data follows the MIPI CSI-2 specification and the padding of
 the data is defined in the MIPI CCS specification.
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
index f3f3bc0615e5..e1306b9510e7 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
@@ -91,7 +91,7 @@ const struct ipu6_isys_pixelformat ipu6_isys_pfmts[] = {
 	  IPU6_FW_ISYS_FRAME_FORMAT_RAW10, true },
 	{ V4L2_META_FMT_GENERIC_CSI2_12, 12, 12, MEDIA_BUS_FMT_META_12,
 	  IPU6_FW_ISYS_FRAME_FORMAT_RAW12, true },
-	{ V4L2_META_FMT_GENERIC_CSI2_16, 16, 16, MEDIA_BUS_FMT_META_16,
+	{ V4L2_META_FMT_GENERIC_16, 16, 16, MEDIA_BUS_FMT_META_16,
 	  IPU6_FW_ISYS_FRAME_FORMAT_RAW16, true },
 };
 
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index e538c1230631..d197002d640d 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1487,9 +1487,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_GENERIC_CSI2_10:	descr = "8-bit Generic Meta, 10b CSI-2"; break;
 	case V4L2_META_FMT_GENERIC_CSI2_12:	descr = "8-bit Generic Meta, 12b CSI-2"; break;
 	case V4L2_META_FMT_GENERIC_CSI2_14:	descr = "8-bit Generic Meta, 14b CSI-2"; break;
-	case V4L2_META_FMT_GENERIC_CSI2_16:	descr = "8-bit Generic Meta, 16b CSI-2"; break;
+	case V4L2_META_FMT_GENERIC_16:	descr = "8-bit Generic Meta, 16b"; break;
 	case V4L2_META_FMT_GENERIC_CSI2_20:	descr = "8-bit Generic Meta, 20b CSI-2"; break;
-	case V4L2_META_FMT_GENERIC_CSI2_24:	descr = "8-bit Generic Meta, 24b CSI-2"; break;
+	case V4L2_META_FMT_GENERIC_24:	descr = "8-bit Generic Meta, 24b"; break;
 
 	default:
 		/* Compressed formats */
@@ -1576,9 +1576,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_META_FMT_GENERIC_CSI2_10:
 		case V4L2_META_FMT_GENERIC_CSI2_12:
 		case V4L2_META_FMT_GENERIC_CSI2_14:
-		case V4L2_META_FMT_GENERIC_CSI2_16:
+		case V4L2_META_FMT_GENERIC_16:
 		case V4L2_META_FMT_GENERIC_CSI2_20:
-		case V4L2_META_FMT_GENERIC_CSI2_24:
+		case V4L2_META_FMT_GENERIC_24:
 			fmt->flags |= V4L2_FMT_FLAG_META_LINE_BASED;
 			break;
 		default:
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index c4c4f3eb67e1..6a5d9ac7bdb9 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -896,12 +896,12 @@ struct v4l2_pix_format {
  * adding new ones!
  */
 #define V4L2_META_FMT_GENERIC_8		v4l2_fourcc('M', 'E', 'T', '8') /* Generic 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_10	v4l2_fourcc('M', 'C', '1', 'A') /* 10-bit CSI-2 packed 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_12	v4l2_fourcc('M', 'C', '1', 'C') /* 12-bit CSI-2 packed 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_14	v4l2_fourcc('M', 'C', '1', 'E') /* 14-bit CSI-2 packed 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_16	v4l2_fourcc('M', 'C', '1', 'G') /* 16-bit CSI-2 packed 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'C', '1', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_24	v4l2_fourcc('M', 'C', '1', 'O') /* 24-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_CSI2_10	v4l2_fourcc('M', 'E', 'C', 'A') /* 10-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_CSI2_12	v4l2_fourcc('M', 'E', 'C', 'C') /* 12-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_CSI2_14	v4l2_fourcc('M', 'E', 'C', 'E') /* 14-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_16	v4l2_fourcc('M', 'E', 'T', 'G') /* Generic 16-bit 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'E', 'C', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_24	v4l2_fourcc('M', 'E', 'T', 'O') /* Generic 24-bit 8-bit metadata */
 #endif
 
 /* priv field value to indicates that subsequent fields are valid. */
-- 
2.47.2

Re: [PATCH v11 37/66] media: uapi: Correct generic CSI-2 metadata format 4cc

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:38PM +0300, Sakari Ailus wrote:
> Rework the pixelformat 4cc for CSI-2 generic metadata, the 16- and 24-bit
> variants are not specific to CSI-2. This can be done as no driver uses
> this yet and the interface is disabled.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j

> ---
>  .../media/v4l/metafmt-generic.rst             | 37 +++++++++----------
>  .../media/pci/intel/ipu6/ipu6-isys-video.c    |  2 +-
>  drivers/media/v4l2-core/v4l2-ioctl.c          |  8 ++--
>  include/uapi/linux/videodev2.h                | 12 +++---
>  4 files changed, 29 insertions(+), 30 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> index e8824b5a65d6..ca960907efde 100644
> --- a/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> +++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> @@ -1,7 +1,7 @@
>  .. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
>
>  ********************************************************************************************************************************************************************************************************************************************************************************
> -V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_META_FMT_GENERIC_CSI2_12 ('MC1C'), V4L2_META_FMT_GENERIC_CSI2_14 ('MC1E'), V4L2_META_FMT_GENERIC_CSI2_16 ('MC1G'), V4L2_META_FMT_GENERIC_CSI2_20 ('MC1K'), V4L2_META_FMT_GENERIC_CSI2_24 ('MC1O')
> +V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MECA'), V4L2_META_FMT_GENERIC_CSI2_12 ('MECC'), V4L2_META_FMT_GENERIC_CSI2_14 ('MECE'), V4L2_META_FMT_GENERIC_16 ('METG'), V4L2_META_FMT_GENERIC_CSI2_20 ('MECK'), V4L2_META_FMT_GENERIC_24 ('METO')
>  ********************************************************************************************************************************************************************************************************************************************************************************
>
>
> @@ -29,7 +29,7 @@ is used on CSI-2 for 8 bits per :term:`Data Unit`.
>
>  Additionally it is used for 16 bits per Data Unit when two bytes of metadata are
>  packed into one 16-bit Data Unit. Otherwise the 16 bits per pixel dataformat is
> -:ref:`V4L2_META_FMT_GENERIC_CSI2_16 <v4l2-meta-fmt-generic-csi2-16>`.
> +:ref:`V4L2_META_FMT_GENERIC_16 <v4l2-meta-fmt-generic-16>`.
>
>  **Byte Order Of V4L2_META_FMT_GENERIC_8.**
>  Each cell is one byte. "M" denotes a byte of metadata.
> @@ -112,8 +112,8 @@ the data is defined in the MIPI CCS specification.
>
>  This format is also used in conjunction with 24 bits per :term:`Data Unit`
>  formats that pack two bytes of metadata into one Data Unit. Otherwise the
> -24 bits per pixel dataformat is :ref:`V4L2_META_FMT_GENERIC_CSI2_24
> -<v4l2-meta-fmt-generic-csi2-24>`.
> +24 bits per pixel dataformat is :ref:`V4L2_META_FMT_GENERIC_24
> +<v4l2-meta-fmt-generic-24>`.
>
>  This format is little endian.
>
> @@ -185,14 +185,14 @@ Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
>        - x
>        - x
>
> -.. _v4l2-meta-fmt-generic-csi2-16:
> +.. _v4l2-meta-fmt-generic-16:
>
> -V4L2_META_FMT_GENERIC_CSI2_16
> ------------------------------
> +V4L2_META_FMT_GENERIC_16
> +------------------------
>
> -V4L2_META_FMT_GENERIC_CSI2_16 contains 8-bit generic metadata packed in 16-bit
> -Data Units, with one padding byte after every byte of metadata. This format is
> -typically used by CSI-2 receivers with a source that transmits
> +V4L2_META_FMT_GENERIC_16 contains 8-bit generic metadata packed in 16-bit Data
> +Units, with one padding byte after every byte of metadata. This format is
> +also used by CSI-2 receivers with a source that transmits
>  MEDIA_BUS_FMT_META_16 and the CSI-2 receiver writes the received data to memory
>  as-is.
>
> @@ -205,7 +205,7 @@ Some devices support more efficient packing of metadata in conjunction with
>
>  This format is little endian.
>
> -**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_16.**
> +**Byte Order Of V4L2_META_FMT_GENERIC_16.**
>  Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
>
>  .. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|
> @@ -287,16 +287,15 @@ Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
>        - x
>        - x
>
> -.. _v4l2-meta-fmt-generic-csi2-24:
> +.. _v4l2-meta-fmt-generic-24:
>
> -V4L2_META_FMT_GENERIC_CSI2_24
> ------------------------------
> +V4L2_META_FMT_GENERIC_24
> +------------------------
>
> -V4L2_META_FMT_GENERIC_CSI2_24 contains 8-bit generic metadata packed in 24-bit
> -Data Units, with two padding bytes after every byte of metadata. This format is
> -typically used by CSI-2 receivers with a source that transmits
> -MEDIA_BUS_FMT_META_24 and the CSI-2 receiver writes the received data to memory
> -as-is.
> +V4L2_META_FMT_GENERIC_24 contains 8-bit generic metadata packed in 24-bit Data
> +Units, with two padding bytes after every byte of metadata. This format is also
> +used by CSI-2 receivers with a source that transmits MEDIA_BUS_FMT_META_24 and
> +the CSI-2 receiver writes the received data to memory as-is.
>
>  The packing of the data follows the MIPI CSI-2 specification and the padding of
>  the data is defined in the MIPI CCS specification.
> diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> index f3f3bc0615e5..e1306b9510e7 100644
> --- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> @@ -91,7 +91,7 @@ const struct ipu6_isys_pixelformat ipu6_isys_pfmts[] = {
>  	  IPU6_FW_ISYS_FRAME_FORMAT_RAW10, true },
>  	{ V4L2_META_FMT_GENERIC_CSI2_12, 12, 12, MEDIA_BUS_FMT_META_12,
>  	  IPU6_FW_ISYS_FRAME_FORMAT_RAW12, true },
> -	{ V4L2_META_FMT_GENERIC_CSI2_16, 16, 16, MEDIA_BUS_FMT_META_16,
> +	{ V4L2_META_FMT_GENERIC_16, 16, 16, MEDIA_BUS_FMT_META_16,
>  	  IPU6_FW_ISYS_FRAME_FORMAT_RAW16, true },
>  };
>
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index e538c1230631..d197002d640d 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1487,9 +1487,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  	case V4L2_META_FMT_GENERIC_CSI2_10:	descr = "8-bit Generic Meta, 10b CSI-2"; break;
>  	case V4L2_META_FMT_GENERIC_CSI2_12:	descr = "8-bit Generic Meta, 12b CSI-2"; break;
>  	case V4L2_META_FMT_GENERIC_CSI2_14:	descr = "8-bit Generic Meta, 14b CSI-2"; break;
> -	case V4L2_META_FMT_GENERIC_CSI2_16:	descr = "8-bit Generic Meta, 16b CSI-2"; break;
> +	case V4L2_META_FMT_GENERIC_16:	descr = "8-bit Generic Meta, 16b"; break;
>  	case V4L2_META_FMT_GENERIC_CSI2_20:	descr = "8-bit Generic Meta, 20b CSI-2"; break;
> -	case V4L2_META_FMT_GENERIC_CSI2_24:	descr = "8-bit Generic Meta, 24b CSI-2"; break;
> +	case V4L2_META_FMT_GENERIC_24:	descr = "8-bit Generic Meta, 24b"; break;
>
>  	default:
>  		/* Compressed formats */
> @@ -1576,9 +1576,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_META_FMT_GENERIC_CSI2_10:
>  		case V4L2_META_FMT_GENERIC_CSI2_12:
>  		case V4L2_META_FMT_GENERIC_CSI2_14:
> -		case V4L2_META_FMT_GENERIC_CSI2_16:
> +		case V4L2_META_FMT_GENERIC_16:
>  		case V4L2_META_FMT_GENERIC_CSI2_20:
> -		case V4L2_META_FMT_GENERIC_CSI2_24:
> +		case V4L2_META_FMT_GENERIC_24:
>  			fmt->flags |= V4L2_FMT_FLAG_META_LINE_BASED;
>  			break;
>  		default:
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index c4c4f3eb67e1..6a5d9ac7bdb9 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -896,12 +896,12 @@ struct v4l2_pix_format {
>   * adding new ones!
>   */
>  #define V4L2_META_FMT_GENERIC_8		v4l2_fourcc('M', 'E', 'T', '8') /* Generic 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_10	v4l2_fourcc('M', 'C', '1', 'A') /* 10-bit CSI-2 packed 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_12	v4l2_fourcc('M', 'C', '1', 'C') /* 12-bit CSI-2 packed 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_14	v4l2_fourcc('M', 'C', '1', 'E') /* 14-bit CSI-2 packed 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_16	v4l2_fourcc('M', 'C', '1', 'G') /* 16-bit CSI-2 packed 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'C', '1', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_24	v4l2_fourcc('M', 'C', '1', 'O') /* 24-bit CSI-2 packed 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_CSI2_10	v4l2_fourcc('M', 'E', 'C', 'A') /* 10-bit CSI-2 packed 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_CSI2_12	v4l2_fourcc('M', 'E', 'C', 'C') /* 12-bit CSI-2 packed 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_CSI2_14	v4l2_fourcc('M', 'E', 'C', 'E') /* 14-bit CSI-2 packed 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_16	v4l2_fourcc('M', 'E', 'T', 'G') /* Generic 16-bit 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'E', 'C', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_24	v4l2_fourcc('M', 'E', 'T', 'O') /* Generic 24-bit 8-bit metadata */
>  #endif
>
>  /* priv field value to indicates that subsequent fields are valid. */
> --
> 2.47.2
>
>

Re: [PATCH v11 37/66] media: uapi: Correct generic CSI-2 metadata format 4cc

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Aug 25, 2025 at 12:50:38PM +0300, Sakari Ailus wrote:
> Rework the pixelformat 4cc for CSI-2 generic metadata, the 16- and 24-bit
> variants are not specific to CSI-2. This can be done as no driver uses
> this yet and the interface is disabled.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../media/v4l/metafmt-generic.rst             | 37 +++++++++----------
>  .../media/pci/intel/ipu6/ipu6-isys-video.c    |  2 +-
>  drivers/media/v4l2-core/v4l2-ioctl.c          |  8 ++--
>  include/uapi/linux/videodev2.h                | 12 +++---
>  4 files changed, 29 insertions(+), 30 deletions(-)
> 
> diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> index e8824b5a65d6..ca960907efde 100644
> --- a/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> +++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
> @@ -1,7 +1,7 @@
>  .. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
>  
>  ********************************************************************************************************************************************************************************************************************************************************************************
> -V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_META_FMT_GENERIC_CSI2_12 ('MC1C'), V4L2_META_FMT_GENERIC_CSI2_14 ('MC1E'), V4L2_META_FMT_GENERIC_CSI2_16 ('MC1G'), V4L2_META_FMT_GENERIC_CSI2_20 ('MC1K'), V4L2_META_FMT_GENERIC_CSI2_24 ('MC1O')
> +V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MECA'), V4L2_META_FMT_GENERIC_CSI2_12 ('MECC'), V4L2_META_FMT_GENERIC_CSI2_14 ('MECE'), V4L2_META_FMT_GENERIC_16 ('METG'), V4L2_META_FMT_GENERIC_CSI2_20 ('MECK'), V4L2_META_FMT_GENERIC_24 ('METO')
>  ********************************************************************************************************************************************************************************************************************************************************************************
>  
>  
> @@ -29,7 +29,7 @@ is used on CSI-2 for 8 bits per :term:`Data Unit`.
>  
>  Additionally it is used for 16 bits per Data Unit when two bytes of metadata are
>  packed into one 16-bit Data Unit. Otherwise the 16 bits per pixel dataformat is
> -:ref:`V4L2_META_FMT_GENERIC_CSI2_16 <v4l2-meta-fmt-generic-csi2-16>`.
> +:ref:`V4L2_META_FMT_GENERIC_16 <v4l2-meta-fmt-generic-16>`.
>  
>  **Byte Order Of V4L2_META_FMT_GENERIC_8.**
>  Each cell is one byte. "M" denotes a byte of metadata.
> @@ -112,8 +112,8 @@ the data is defined in the MIPI CCS specification.
>  
>  This format is also used in conjunction with 24 bits per :term:`Data Unit`
>  formats that pack two bytes of metadata into one Data Unit. Otherwise the
> -24 bits per pixel dataformat is :ref:`V4L2_META_FMT_GENERIC_CSI2_24
> -<v4l2-meta-fmt-generic-csi2-24>`.
> +24 bits per pixel dataformat is :ref:`V4L2_META_FMT_GENERIC_24
> +<v4l2-meta-fmt-generic-24>`.
>  
>  This format is little endian.
>  
> @@ -185,14 +185,14 @@ Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
>        - x
>        - x
>  
> -.. _v4l2-meta-fmt-generic-csi2-16:
> +.. _v4l2-meta-fmt-generic-16:
>  
> -V4L2_META_FMT_GENERIC_CSI2_16
> ------------------------------
> +V4L2_META_FMT_GENERIC_16
> +------------------------
>  
> -V4L2_META_FMT_GENERIC_CSI2_16 contains 8-bit generic metadata packed in 16-bit
> -Data Units, with one padding byte after every byte of metadata. This format is
> -typically used by CSI-2 receivers with a source that transmits
> +V4L2_META_FMT_GENERIC_16 contains 8-bit generic metadata packed in 16-bit Data
> +Units, with one padding byte after every byte of metadata. This format is
> +also used by CSI-2 receivers with a source that transmits
>  MEDIA_BUS_FMT_META_16 and the CSI-2 receiver writes the received data to memory

MEDIA_BUS_FMT_META_16 fits on the previous line.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  as-is.
>  
> @@ -205,7 +205,7 @@ Some devices support more efficient packing of metadata in conjunction with
>  
>  This format is little endian.
>  
> -**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_16.**
> +**Byte Order Of V4L2_META_FMT_GENERIC_16.**
>  Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
>  
>  .. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|
> @@ -287,16 +287,15 @@ Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
>        - x
>        - x
>  
> -.. _v4l2-meta-fmt-generic-csi2-24:
> +.. _v4l2-meta-fmt-generic-24:
>  
> -V4L2_META_FMT_GENERIC_CSI2_24
> ------------------------------
> +V4L2_META_FMT_GENERIC_24
> +------------------------
>  
> -V4L2_META_FMT_GENERIC_CSI2_24 contains 8-bit generic metadata packed in 24-bit
> -Data Units, with two padding bytes after every byte of metadata. This format is
> -typically used by CSI-2 receivers with a source that transmits
> -MEDIA_BUS_FMT_META_24 and the CSI-2 receiver writes the received data to memory
> -as-is.
> +V4L2_META_FMT_GENERIC_24 contains 8-bit generic metadata packed in 24-bit Data
> +Units, with two padding bytes after every byte of metadata. This format is also
> +used by CSI-2 receivers with a source that transmits MEDIA_BUS_FMT_META_24 and
> +the CSI-2 receiver writes the received data to memory as-is.
>  
>  The packing of the data follows the MIPI CSI-2 specification and the padding of
>  the data is defined in the MIPI CCS specification.
> diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> index f3f3bc0615e5..e1306b9510e7 100644
> --- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> @@ -91,7 +91,7 @@ const struct ipu6_isys_pixelformat ipu6_isys_pfmts[] = {
>  	  IPU6_FW_ISYS_FRAME_FORMAT_RAW10, true },
>  	{ V4L2_META_FMT_GENERIC_CSI2_12, 12, 12, MEDIA_BUS_FMT_META_12,
>  	  IPU6_FW_ISYS_FRAME_FORMAT_RAW12, true },
> -	{ V4L2_META_FMT_GENERIC_CSI2_16, 16, 16, MEDIA_BUS_FMT_META_16,
> +	{ V4L2_META_FMT_GENERIC_16, 16, 16, MEDIA_BUS_FMT_META_16,
>  	  IPU6_FW_ISYS_FRAME_FORMAT_RAW16, true },
>  };
>  
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index e538c1230631..d197002d640d 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1487,9 +1487,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  	case V4L2_META_FMT_GENERIC_CSI2_10:	descr = "8-bit Generic Meta, 10b CSI-2"; break;
>  	case V4L2_META_FMT_GENERIC_CSI2_12:	descr = "8-bit Generic Meta, 12b CSI-2"; break;
>  	case V4L2_META_FMT_GENERIC_CSI2_14:	descr = "8-bit Generic Meta, 14b CSI-2"; break;
> -	case V4L2_META_FMT_GENERIC_CSI2_16:	descr = "8-bit Generic Meta, 16b CSI-2"; break;
> +	case V4L2_META_FMT_GENERIC_16:	descr = "8-bit Generic Meta, 16b"; break;
>  	case V4L2_META_FMT_GENERIC_CSI2_20:	descr = "8-bit Generic Meta, 20b CSI-2"; break;
> -	case V4L2_META_FMT_GENERIC_CSI2_24:	descr = "8-bit Generic Meta, 24b CSI-2"; break;
> +	case V4L2_META_FMT_GENERIC_24:	descr = "8-bit Generic Meta, 24b"; break;
>  
>  	default:
>  		/* Compressed formats */
> @@ -1576,9 +1576,9 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_META_FMT_GENERIC_CSI2_10:
>  		case V4L2_META_FMT_GENERIC_CSI2_12:
>  		case V4L2_META_FMT_GENERIC_CSI2_14:
> -		case V4L2_META_FMT_GENERIC_CSI2_16:
> +		case V4L2_META_FMT_GENERIC_16:
>  		case V4L2_META_FMT_GENERIC_CSI2_20:
> -		case V4L2_META_FMT_GENERIC_CSI2_24:
> +		case V4L2_META_FMT_GENERIC_24:
>  			fmt->flags |= V4L2_FMT_FLAG_META_LINE_BASED;
>  			break;
>  		default:
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index c4c4f3eb67e1..6a5d9ac7bdb9 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -896,12 +896,12 @@ struct v4l2_pix_format {
>   * adding new ones!
>   */
>  #define V4L2_META_FMT_GENERIC_8		v4l2_fourcc('M', 'E', 'T', '8') /* Generic 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_10	v4l2_fourcc('M', 'C', '1', 'A') /* 10-bit CSI-2 packed 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_12	v4l2_fourcc('M', 'C', '1', 'C') /* 12-bit CSI-2 packed 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_14	v4l2_fourcc('M', 'C', '1', 'E') /* 14-bit CSI-2 packed 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_16	v4l2_fourcc('M', 'C', '1', 'G') /* 16-bit CSI-2 packed 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'C', '1', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
> -#define V4L2_META_FMT_GENERIC_CSI2_24	v4l2_fourcc('M', 'C', '1', 'O') /* 24-bit CSI-2 packed 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_CSI2_10	v4l2_fourcc('M', 'E', 'C', 'A') /* 10-bit CSI-2 packed 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_CSI2_12	v4l2_fourcc('M', 'E', 'C', 'C') /* 12-bit CSI-2 packed 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_CSI2_14	v4l2_fourcc('M', 'E', 'C', 'E') /* 14-bit CSI-2 packed 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_16	v4l2_fourcc('M', 'E', 'T', 'G') /* Generic 16-bit 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'E', 'C', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
> +#define V4L2_META_FMT_GENERIC_24	v4l2_fourcc('M', 'E', 'T', 'O') /* Generic 24-bit 8-bit metadata */
>  #endif
>  
>  /* priv field value to indicates that subsequent fields are valid. */

-- 
Regards,

Laurent Pinchart

[PATCH v11 38/66] Revert "media: uapi: v4l: Don't expose generic metadata formats to userspace"

[Sakari Ailus]

This reverts commit d69c8429ea80af02e89e5b3eecb78e417ad049c8.

Make the generic metadata formats visible for the userspace again.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 include/uapi/linux/videodev2.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 6a5d9ac7bdb9..2b2e070f272c 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -890,7 +890,6 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_RPI_FE_CFG	v4l2_fourcc('R', 'P', 'F', 'C') /* PiSP FE configuration */
 #define V4L2_META_FMT_RPI_FE_STATS	v4l2_fourcc('R', 'P', 'F', 'S') /* PiSP FE stats */
 
-#ifdef __KERNEL__
 /*
  * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when
  * adding new ones!
@@ -902,7 +901,6 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_GENERIC_16	v4l2_fourcc('M', 'E', 'T', 'G') /* Generic 16-bit 8-bit metadata */
 #define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'E', 'C', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
 #define V4L2_META_FMT_GENERIC_24	v4l2_fourcc('M', 'E', 'T', 'O') /* Generic 24-bit 8-bit metadata */
-#endif
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
2.47.2

Re: [PATCH v11 38/66] Revert "media: uapi: v4l: Don't expose generic metadata formats to userspace"

[Jacopo Mondi]

On Mon, Aug 25, 2025 at 12:50:39PM +0300, Sakari Ailus wrote:
> This reverts commit d69c8429ea80af02e89e5b3eecb78e417ad049c8.
>
> Make the generic metadata formats visible for the userspace again.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

 \o/

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  include/uapi/linux/videodev2.h | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 6a5d9ac7bdb9..2b2e070f272c 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -890,7 +890,6 @@ struct v4l2_pix_format {
>  #define V4L2_META_FMT_RPI_FE_CFG	v4l2_fourcc('R', 'P', 'F', 'C') /* PiSP FE configuration */
>  #define V4L2_META_FMT_RPI_FE_STATS	v4l2_fourcc('R', 'P', 'F', 'S') /* PiSP FE stats */
>
> -#ifdef __KERNEL__
>  /*
>   * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when
>   * adding new ones!
> @@ -902,7 +901,6 @@ struct v4l2_pix_format {
>  #define V4L2_META_FMT_GENERIC_16	v4l2_fourcc('M', 'E', 'T', 'G') /* Generic 16-bit 8-bit metadata */
>  #define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'E', 'C', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
>  #define V4L2_META_FMT_GENERIC_24	v4l2_fourcc('M', 'E', 'T', 'O') /* Generic 24-bit 8-bit metadata */
> -#endif
>
>  /* priv field value to indicates that subsequent fields are valid. */
>  #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
> --
> 2.47.2
>
>

Re: [PATCH v11 38/66] Revert "media: uapi: v4l: Don't expose generic metadata formats to userspace"

[Laurent Pinchart]

On Mon, Aug 25, 2025 at 12:50:39PM +0300, Sakari Ailus wrote:
> This reverts commit d69c8429ea80af02e89e5b3eecb78e417ad049c8.
> 
> Make the generic metadata formats visible for the userspace again.

"Now that the API has stabilized, make ..."

> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  include/uapi/linux/videodev2.h | 2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 6a5d9ac7bdb9..2b2e070f272c 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -890,7 +890,6 @@ struct v4l2_pix_format {
>  #define V4L2_META_FMT_RPI_FE_CFG	v4l2_fourcc('R', 'P', 'F', 'C') /* PiSP FE configuration */
>  #define V4L2_META_FMT_RPI_FE_STATS	v4l2_fourcc('R', 'P', 'F', 'S') /* PiSP FE stats */
>  
> -#ifdef __KERNEL__
>  /*
>   * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when
>   * adding new ones!
> @@ -902,7 +901,6 @@ struct v4l2_pix_format {
>  #define V4L2_META_FMT_GENERIC_16	v4l2_fourcc('M', 'E', 'T', 'G') /* Generic 16-bit 8-bit metadata */
>  #define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'E', 'C', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
>  #define V4L2_META_FMT_GENERIC_24	v4l2_fourcc('M', 'E', 'T', 'O') /* Generic 24-bit 8-bit metadata */
> -#endif
>  
>  /* priv field value to indicates that subsequent fields are valid. */
>  #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe

-- 
Regards,

Laurent Pinchart

[PATCH v11 39/66] media: Documentation: Add subdev configuration models, raw sensor model

[Sakari Ailus]

Sub-device configuration models define what V4L2 API elements are
available on a compliant sub-device and how do they behave.

The patch also adds a model for common raw sensors.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
---
 .../media/drivers/camera-sensor.rst           |   4 +
 .../media/v4l/common-raw-sensor.dia           | 442 ++++++++++++++++++
 .../media/v4l/common-raw-sensor.svg           | 134 ++++++
 .../userspace-api/media/v4l/dev-subdev.rst    |   2 +
 .../media/v4l/subdev-config-model.rst         | 230 +++++++++
 5 files changed, 812 insertions(+)
 create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.dia
 create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.svg
 create mode 100644 Documentation/userspace-api/media/v4l/subdev-config-model.rst

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index cbbfbb0d8273..39f3f91c6733 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -18,6 +18,8 @@ binning functionality. The sensor drivers belong to two distinct classes, freely
 configurable and register list-based drivers, depending on how the driver
 configures this functionality.
 
+Also see :ref:`media_subdev_config_model_common_raw_sensor`.
+
 Freely configurable camera sensor drivers
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -118,6 +120,8 @@ values programmed by the register sequences. The default values of these
 controls shall be 0 (disabled). Especially these controls shall not be inverted,
 independently of the sensor's mounting rotation.
 
+.. _media_using_camera_sensor_drivers_embedded_data:
+
 Embedded data
 -------------
 
diff --git a/Documentation/userspace-api/media/v4l/common-raw-sensor.dia b/Documentation/userspace-api/media/v4l/common-raw-sensor.dia
new file mode 100644
index 000000000000..24b3f2b2a626
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/common-raw-sensor.dia
@@ -0,0 +1,442 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/">
+  <dia:diagramdata>
+    <dia:attribute name="background">
+      <dia:color val="#ffffffff"/>
+    </dia:attribute>
+    <dia:attribute name="pagebreak">
+      <dia:color val="#000099ff"/>
+    </dia:attribute>
+    <dia:attribute name="paper">
+      <dia:composite type="paper">
+        <dia:attribute name="name">
+          <dia:string>#A4#</dia:string>
+        </dia:attribute>
+        <dia:attribute name="tmargin">
+          <dia:real val="2.8222"/>
+        </dia:attribute>
+        <dia:attribute name="bmargin">
+          <dia:real val="2.8222"/>
+        </dia:attribute>
+        <dia:attribute name="lmargin">
+          <dia:real val="2.8222"/>
+        </dia:attribute>
+        <dia:attribute name="rmargin">
+          <dia:real val="2.8222"/>
+        </dia:attribute>
+        <dia:attribute name="is_portrait">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="scaling">
+          <dia:real val="1"/>
+        </dia:attribute>
+        <dia:attribute name="fitto">
+          <dia:boolean val="false"/>
+        </dia:attribute>
+      </dia:composite>
+    </dia:attribute>
+    <dia:attribute name="grid">
+      <dia:composite type="grid">
+        <dia:attribute name="dynamic">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="width_x">
+          <dia:real val="1"/>
+        </dia:attribute>
+        <dia:attribute name="width_y">
+          <dia:real val="1"/>
+        </dia:attribute>
+        <dia:attribute name="visible_x">
+          <dia:int val="1"/>
+        </dia:attribute>
+        <dia:attribute name="visible_y">
+          <dia:int val="1"/>
+        </dia:attribute>
+        <dia:composite type="color"/>
+      </dia:composite>
+    </dia:attribute>
+    <dia:attribute name="color">
+      <dia:color val="#d8e5e5ff"/>
+    </dia:attribute>
+    <dia:attribute name="guides"/>
+    <dia:attribute name="guide_color">
+      <dia:color val="#00ff00ff"/>
+    </dia:attribute>
+    <dia:attribute name="display">
+      <dia:composite type="display">
+        <dia:attribute name="antialiased">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="snap-to-grid">
+          <dia:boolean val="false"/>
+        </dia:attribute>
+        <dia:attribute name="snap-to-guides">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="snap-to-object">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="show-grid">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="show-guides">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="show-connection-points">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+      </dia:composite>
+    </dia:attribute>
+  </dia:diagramdata>
+  <dia:layer name="Background" visible="true" connectable="false"/>
+  <dia:layer name="Background" visible="true" connectable="false"/>
+  <dia:layer name="Background" visible="true" connectable="false"/>
+  <dia:layer name="Background" visible="true" connectable="true" active="true">
+    <dia:object type="Standard - Box" version="0" id="O0">
+      <dia:attribute name="obj_pos">
+        <dia:point val="16.35,11.5"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="16.3,11.45;25.8,18.45"/>
+      </dia:attribute>
+      <dia:attribute name="elem_corner">
+        <dia:point val="16.35,11.5"/>
+      </dia:attribute>
+      <dia:attribute name="elem_width">
+        <dia:real val="9.4000000000000021"/>
+      </dia:attribute>
+      <dia:attribute name="elem_height">
+        <dia:real val="6.9000000000000021"/>
+      </dia:attribute>
+      <dia:attribute name="show_background">
+        <dia:boolean val="true"/>
+      </dia:attribute>
+    </dia:object>
+    <dia:object type="Geometric - Perfect Circle" version="1" id="O1">
+      <dia:attribute name="obj_pos">
+        <dia:point val="25.5627,14.578"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="25.4627,14.478;26.0191,15.0344"/>
+      </dia:attribute>
+      <dia:attribute name="meta">
+        <dia:composite type="dict"/>
+      </dia:attribute>
+      <dia:attribute name="elem_corner">
+        <dia:point val="25.5627,14.578"/>
+      </dia:attribute>
+      <dia:attribute name="elem_width">
+        <dia:real val="0.35638032780853468"/>
+      </dia:attribute>
+      <dia:attribute name="elem_height">
+        <dia:real val="0.35638032780853468"/>
+      </dia:attribute>
+      <dia:attribute name="line_width">
+        <dia:real val="0.10000000000000001"/>
+      </dia:attribute>
+      <dia:attribute name="line_colour">
+        <dia:color val="#000000ff"/>
+      </dia:attribute>
+      <dia:attribute name="fill_colour">
+        <dia:color val="#ffffffff"/>
+      </dia:attribute>
+      <dia:attribute name="show_background">
+        <dia:boolean val="true"/>
+      </dia:attribute>
+      <dia:attribute name="line_style">
+        <dia:enum val="0"/>
+        <dia:real val="1"/>
+      </dia:attribute>
+      <dia:attribute name="flip_horizontal">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="flip_vertical">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="subscale">
+        <dia:real val="1"/>
+      </dia:attribute>
+    </dia:object>
+    <dia:object type="Geometric - Perfect Circle" version="1" id="O2">
+      <dia:attribute name="obj_pos">
+        <dia:point val="17.7702,13.2656"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="17.6702,13.1656;18.24,13.7354"/>
+      </dia:attribute>
+      <dia:attribute name="meta">
+        <dia:composite type="dict"/>
+      </dia:attribute>
+      <dia:attribute name="elem_corner">
+        <dia:point val="17.7702,13.2656"/>
+      </dia:attribute>
+      <dia:attribute name="elem_width">
+        <dia:real val="0.36980024191863681"/>
+      </dia:attribute>
+      <dia:attribute name="elem_height">
+        <dia:real val="0.36980024191863681"/>
+      </dia:attribute>
+      <dia:attribute name="line_width">
+        <dia:real val="0.10000000000000001"/>
+      </dia:attribute>
+      <dia:attribute name="line_colour">
+        <dia:color val="#000000ff"/>
+      </dia:attribute>
+      <dia:attribute name="fill_colour">
+        <dia:color val="#ffffffff"/>
+      </dia:attribute>
+      <dia:attribute name="show_background">
+        <dia:boolean val="true"/>
+      </dia:attribute>
+      <dia:attribute name="line_style">
+        <dia:enum val="0"/>
+        <dia:real val="1"/>
+      </dia:attribute>
+      <dia:attribute name="flip_horizontal">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="flip_vertical">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="subscale">
+        <dia:real val="1"/>
+      </dia:attribute>
+    </dia:object>
+    <dia:object type="Geometric - Perfect Circle" version="1" id="O3">
+      <dia:attribute name="obj_pos">
+        <dia:point val="17.7464,16.2056"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="17.6464,16.1056;18.2162,16.6754"/>
+      </dia:attribute>
+      <dia:attribute name="meta">
+        <dia:composite type="dict"/>
+      </dia:attribute>
+      <dia:attribute name="elem_corner">
+        <dia:point val="17.7464,16.2056"/>
+      </dia:attribute>
+      <dia:attribute name="elem_width">
+        <dia:real val="0.36980024191863681"/>
+      </dia:attribute>
+      <dia:attribute name="elem_height">
+        <dia:real val="0.36980024191863681"/>
+      </dia:attribute>
+      <dia:attribute name="line_width">
+        <dia:real val="0.10000000000000001"/>
+      </dia:attribute>
+      <dia:attribute name="line_colour">
+        <dia:color val="#000000ff"/>
+      </dia:attribute>
+      <dia:attribute name="fill_colour">
+        <dia:color val="#ffffffff"/>
+      </dia:attribute>
+      <dia:attribute name="show_background">
+        <dia:boolean val="true"/>
+      </dia:attribute>
+      <dia:attribute name="line_style">
+        <dia:enum val="0"/>
+        <dia:real val="1"/>
+      </dia:attribute>
+      <dia:attribute name="flip_horizontal">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="flip_vertical">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="subscale">
+        <dia:real val="1"/>
+      </dia:attribute>
+    </dia:object>
+    <dia:object type="Standard - Line" version="0" id="O4">
+      <dia:attribute name="obj_pos">
+        <dia:point val="18.1609,16.3413"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="18.1016,14.5712;25.6221,16.4007"/>
+      </dia:attribute>
+      <dia:attribute name="conn_endpoints">
+        <dia:point val="18.1609,16.3413"/>
+        <dia:point val="25.5627,14.7562"/>
+      </dia:attribute>
+      <dia:attribute name="numcp">
+        <dia:int val="1"/>
+      </dia:attribute>
+      <dia:attribute name="line_style">
+        <dia:enum val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow">
+        <dia:enum val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_length">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_width">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:connections>
+        <dia:connection handle="0" to="O3" connection="8"/>
+        <dia:connection handle="1" to="O1" connection="2"/>
+      </dia:connections>
+    </dia:object>
+    <dia:object type="Standard - Line" version="0" id="O5">
+      <dia:attribute name="obj_pos">
+        <dia:point val="18.14,13.4505"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="18.0821,13.3926;25.6206,14.9674"/>
+      </dia:attribute>
+      <dia:attribute name="conn_endpoints">
+        <dia:point val="18.14,13.4505"/>
+        <dia:point val="25.5627,14.7562"/>
+      </dia:attribute>
+      <dia:attribute name="numcp">
+        <dia:int val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow">
+        <dia:enum val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_length">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_width">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:connections>
+        <dia:connection handle="0" to="O2" connection="3"/>
+        <dia:connection handle="1" to="O1" connection="2"/>
+      </dia:connections>
+    </dia:object>
+    <dia:object type="Standard - Line" version="0" id="O6">
+      <dia:attribute name="obj_pos">
+        <dia:point val="25.919,14.7562"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="25.8686,14.3879;31.0068,15.0497"/>
+      </dia:attribute>
+      <dia:attribute name="conn_endpoints">
+        <dia:point val="25.919,14.7562"/>
+        <dia:point val="30.9564,14.7131"/>
+      </dia:attribute>
+      <dia:attribute name="numcp">
+        <dia:int val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow">
+        <dia:enum val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_length">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_width">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:connections>
+        <dia:connection handle="0" to="O1" connection="3"/>
+      </dia:connections>
+    </dia:object>
+    <dia:object type="Standard - Text" version="1" id="O7">
+      <dia:attribute name="obj_pos">
+        <dia:point val="17.9551,13.2656"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="17.9551,12.5181;22.7051,13.2656"/>
+      </dia:attribute>
+      <dia:attribute name="text">
+        <dia:composite type="text">
+          <dia:attribute name="string">
+            <dia:string>#image data (1)#</dia:string>
+          </dia:attribute>
+          <dia:attribute name="font">
+            <dia:font family="sans" style="0" name="Helvetica"/>
+          </dia:attribute>
+          <dia:attribute name="height">
+            <dia:real val="0.80000000000000004"/>
+          </dia:attribute>
+          <dia:attribute name="pos">
+            <dia:point val="17.9551,13.1131"/>
+          </dia:attribute>
+          <dia:attribute name="color">
+            <dia:color val="#000000ff"/>
+          </dia:attribute>
+          <dia:attribute name="alignment">
+            <dia:enum val="0"/>
+          </dia:attribute>
+        </dia:composite>
+      </dia:attribute>
+      <dia:attribute name="valign">
+        <dia:enum val="1"/>
+      </dia:attribute>
+      <dia:connections>
+        <dia:connection handle="0" to="O2" connection="1"/>
+      </dia:connections>
+    </dia:object>
+    <dia:object type="Standard - Text" version="1" id="O8">
+      <dia:attribute name="obj_pos">
+        <dia:point val="17.9313,16.5754"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="17.9313,16.5754;25.1825,17.379"/>
+      </dia:attribute>
+      <dia:attribute name="text">
+        <dia:composite type="text">
+          <dia:attribute name="string">
+            <dia:string>#embedded data (n - 1)#</dia:string>
+          </dia:attribute>
+          <dia:attribute name="font">
+            <dia:font family="sans" style="0" name="Helvetica"/>
+          </dia:attribute>
+          <dia:attribute name="height">
+            <dia:real val="0.80000000000000004"/>
+          </dia:attribute>
+          <dia:attribute name="pos">
+            <dia:point val="17.9313,17.1891"/>
+          </dia:attribute>
+          <dia:attribute name="color">
+            <dia:color val="#000000ff"/>
+          </dia:attribute>
+          <dia:attribute name="alignment">
+            <dia:enum val="0"/>
+          </dia:attribute>
+        </dia:composite>
+      </dia:attribute>
+      <dia:attribute name="valign">
+        <dia:enum val="0"/>
+      </dia:attribute>
+      <dia:connections>
+        <dia:connection handle="0" to="O3" connection="0"/>
+      </dia:connections>
+    </dia:object>
+    <dia:object type="Standard - Text" version="1" id="O9">
+      <dia:attribute name="obj_pos">
+        <dia:point val="26.1,14.125"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="26.1,13.53;30.7475,14.2775"/>
+      </dia:attribute>
+      <dia:attribute name="text">
+        <dia:composite type="text">
+          <dia:attribute name="string">
+            <dia:string>#source pad (0)#</dia:string>
+          </dia:attribute>
+          <dia:attribute name="font">
+            <dia:font family="sans" style="0" name="Helvetica"/>
+          </dia:attribute>
+          <dia:attribute name="height">
+            <dia:real val="0.80000000000000004"/>
+          </dia:attribute>
+          <dia:attribute name="pos">
+            <dia:point val="26.1,14.125"/>
+          </dia:attribute>
+          <dia:attribute name="color">
+            <dia:color val="#000000ff"/>
+          </dia:attribute>
+          <dia:attribute name="alignment">
+            <dia:enum val="0"/>
+          </dia:attribute>
+        </dia:composite>
+      </dia:attribute>
+      <dia:attribute name="valign">
+        <dia:enum val="3"/>
+      </dia:attribute>
+    </dia:object>
+  </dia:layer>
+</dia:diagram>
diff --git a/Documentation/userspace-api/media/v4l/common-raw-sensor.svg b/Documentation/userspace-api/media/v4l/common-raw-sensor.svg
new file mode 100644
index 000000000000..1d6055da2519
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/common-raw-sensor.svg
@@ -0,0 +1,134 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="296pt" height="142pt" viewBox="0 0 296 142" version="1.1">
+<defs>
+<g>
+<symbol overflow="visible" id="glyph0-0">
+<path style="stroke:none;" d="M 0.640625 2.265625 L 0.640625 -9.015625 L 7.03125 -9.015625 L 7.03125 2.265625 Z M 1.359375 1.546875 L 6.328125 1.546875 L 6.328125 -8.296875 L 1.359375 -8.296875 Z M 1.359375 1.546875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-1">
+<path style="stroke:none;" d="M 1.203125 -7 L 2.359375 -7 L 2.359375 0 L 1.203125 0 Z M 1.203125 -9.71875 L 2.359375 -9.71875 L 2.359375 -8.265625 L 1.203125 -8.265625 Z M 1.203125 -9.71875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-2">
+<path style="stroke:none;" d="M 6.65625 -5.65625 C 6.9375 -6.164062 7.273438 -6.546875 7.671875 -6.796875 C 8.078125 -7.046875 8.550781 -7.171875 9.09375 -7.171875 C 9.820312 -7.171875 10.382812 -6.914062 10.78125 -6.40625 C 11.175781 -5.894531 11.375 -5.164062 11.375 -4.21875 L 11.375 0 L 10.21875 0 L 10.21875 -4.1875 C 10.21875 -4.851562 10.097656 -5.347656 9.859375 -5.671875 C 9.628906 -6.003906 9.269531 -6.171875 8.78125 -6.171875 C 8.1875 -6.171875 7.710938 -5.972656 7.359375 -5.578125 C 7.015625 -5.179688 6.84375 -4.640625 6.84375 -3.953125 L 6.84375 0 L 5.6875 0 L 5.6875 -4.1875 C 5.6875 -4.863281 5.566406 -5.363281 5.328125 -5.6875 C 5.097656 -6.007812 4.734375 -6.171875 4.234375 -6.171875 C 3.648438 -6.171875 3.179688 -5.96875 2.828125 -5.5625 C 2.484375 -5.164062 2.3125 -4.628906 2.3125 -3.953125 L 2.3125 0 L 1.15625 0 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.90625 C 2.582031 -6.332031 2.898438 -6.648438 3.265625 -6.859375 C 3.628906 -7.066406 4.0625 -7.171875 4.5625 -7.171875 C 5.070312 -7.171875 5.503906 -7.039062 5.859375 -6.78125 C 6.222656 -6.519531 6.488281 -6.144531 6.65625 -5.65625 Z M 6.65625 -5.65625 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-3">
+<path style="stroke:none;" d="M 4.390625 -3.515625 C 3.460938 -3.515625 2.816406 -3.40625 2.453125 -3.1875 C 2.097656 -2.976562 1.921875 -2.617188 1.921875 -2.109375 C 1.921875 -1.703125 2.050781 -1.378906 2.3125 -1.140625 C 2.582031 -0.898438 2.953125 -0.78125 3.421875 -0.78125 C 4.054688 -0.78125 4.566406 -1.003906 4.953125 -1.453125 C 5.335938 -1.910156 5.53125 -2.515625 5.53125 -3.265625 L 5.53125 -3.515625 Z M 6.671875 -4 L 6.671875 0 L 5.53125 0 L 5.53125 -1.0625 C 5.269531 -0.632812 4.941406 -0.316406 4.546875 -0.109375 C 4.160156 0.0859375 3.679688 0.1875 3.109375 0.1875 C 2.390625 0.1875 1.816406 -0.015625 1.390625 -0.421875 C 0.972656 -0.828125 0.765625 -1.363281 0.765625 -2.03125 C 0.765625 -2.820312 1.023438 -3.414062 1.546875 -3.8125 C 2.078125 -4.21875 2.867188 -4.421875 3.921875 -4.421875 L 5.53125 -4.421875 L 5.53125 -4.53125 C 5.53125 -5.0625 5.351562 -5.46875 5 -5.75 C 4.65625 -6.039062 4.171875 -6.1875 3.546875 -6.1875 C 3.140625 -6.1875 2.75 -6.140625 2.375 -6.046875 C 2 -5.953125 1.632812 -5.8125 1.28125 -5.625 L 1.28125 -6.671875 C 1.695312 -6.835938 2.101562 -6.960938 2.5 -7.046875 C 2.894531 -7.128906 3.28125 -7.171875 3.65625 -7.171875 C 4.675781 -7.171875 5.429688 -6.90625 5.921875 -6.375 C 6.421875 -5.851562 6.671875 -5.0625 6.671875 -4 Z M 6.671875 -4 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-4">
+<path style="stroke:none;" d="M 5.8125 -3.578125 C 5.8125 -4.410156 5.640625 -5.054688 5.296875 -5.515625 C 4.953125 -5.972656 4.46875 -6.203125 3.84375 -6.203125 C 3.226562 -6.203125 2.75 -5.972656 2.40625 -5.515625 C 2.0625 -5.054688 1.890625 -4.410156 1.890625 -3.578125 C 1.890625 -2.753906 2.0625 -2.113281 2.40625 -1.65625 C 2.75 -1.195312 3.226562 -0.96875 3.84375 -0.96875 C 4.46875 -0.96875 4.953125 -1.195312 5.296875 -1.65625 C 5.640625 -2.113281 5.8125 -2.753906 5.8125 -3.578125 Z M 6.953125 -0.875 C 6.953125 0.320312 6.6875 1.207031 6.15625 1.78125 C 5.632812 2.363281 4.828125 2.65625 3.734375 2.65625 C 3.328125 2.65625 2.945312 2.625 2.59375 2.5625 C 2.238281 2.507812 1.890625 2.421875 1.546875 2.296875 L 1.546875 1.171875 C 1.890625 1.359375 2.222656 1.492188 2.546875 1.578125 C 2.878906 1.671875 3.21875 1.71875 3.5625 1.71875 C 4.3125 1.71875 4.875 1.519531 5.25 1.125 C 5.625 0.726562 5.8125 0.132812 5.8125 -0.65625 L 5.8125 -1.234375 C 5.570312 -0.816406 5.265625 -0.503906 4.890625 -0.296875 C 4.523438 -0.0976562 4.082031 0 3.5625 0 C 2.707031 0 2.015625 -0.328125 1.484375 -0.984375 C 0.960938 -1.640625 0.703125 -2.503906 0.703125 -3.578125 C 0.703125 -4.660156 0.960938 -5.53125 1.484375 -6.1875 C 2.015625 -6.84375 2.707031 -7.171875 3.5625 -7.171875 C 4.082031 -7.171875 4.523438 -7.066406 4.890625 -6.859375 C 5.265625 -6.648438 5.570312 -6.34375 5.8125 -5.9375 L 5.8125 -7 L 6.953125 -7 Z M 6.953125 -0.875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-5">
+<path style="stroke:none;" d="M 7.1875 -3.78125 L 7.1875 -3.21875 L 1.90625 -3.21875 C 1.957031 -2.425781 2.195312 -1.820312 2.625 -1.40625 C 3.050781 -1 3.644531 -0.796875 4.40625 -0.796875 C 4.84375 -0.796875 5.269531 -0.847656 5.6875 -0.953125 C 6.101562 -1.066406 6.515625 -1.226562 6.921875 -1.4375 L 6.921875 -0.359375 C 6.515625 -0.179688 6.09375 -0.046875 5.65625 0.046875 C 5.21875 0.140625 4.78125 0.1875 4.34375 0.1875 C 3.21875 0.1875 2.328125 -0.132812 1.671875 -0.78125 C 1.023438 -1.4375 0.703125 -2.320312 0.703125 -3.4375 C 0.703125 -4.582031 1.007812 -5.488281 1.625 -6.15625 C 2.25 -6.832031 3.085938 -7.171875 4.140625 -7.171875 C 5.078125 -7.171875 5.816406 -6.863281 6.359375 -6.25 C 6.910156 -5.644531 7.1875 -4.820312 7.1875 -3.78125 Z M 6.046875 -4.125 C 6.035156 -4.75 5.859375 -5.25 5.515625 -5.625 C 5.171875 -6 4.71875 -6.1875 4.15625 -6.1875 C 3.507812 -6.1875 2.992188 -6.003906 2.609375 -5.640625 C 2.222656 -5.285156 2 -4.78125 1.9375 -4.125 Z M 6.046875 -4.125 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-6">
+<path style="stroke:none;" d=""/>
+</symbol>
+<symbol overflow="visible" id="glyph0-7">
+<path style="stroke:none;" d="M 5.8125 -5.9375 L 5.8125 -9.71875 L 6.953125 -9.71875 L 6.953125 0 L 5.8125 0 L 5.8125 -1.046875 C 5.570312 -0.628906 5.265625 -0.316406 4.890625 -0.109375 C 4.523438 0.0859375 4.082031 0.1875 3.5625 0.1875 C 2.71875 0.1875 2.03125 -0.148438 1.5 -0.828125 C 0.96875 -1.503906 0.703125 -2.394531 0.703125 -3.5 C 0.703125 -4.59375 0.96875 -5.476562 1.5 -6.15625 C 2.03125 -6.832031 2.71875 -7.171875 3.5625 -7.171875 C 4.082031 -7.171875 4.523438 -7.066406 4.890625 -6.859375 C 5.265625 -6.660156 5.570312 -6.351562 5.8125 -5.9375 Z M 1.890625 -3.5 C 1.890625 -2.644531 2.0625 -1.976562 2.40625 -1.5 C 2.757812 -1.019531 3.238281 -0.78125 3.84375 -0.78125 C 4.457031 -0.78125 4.9375 -1.019531 5.28125 -1.5 C 5.632812 -1.976562 5.8125 -2.644531 5.8125 -3.5 C 5.8125 -4.34375 5.632812 -5.003906 5.28125 -5.484375 C 4.9375 -5.960938 4.457031 -6.203125 3.84375 -6.203125 C 3.238281 -6.203125 2.757812 -5.960938 2.40625 -5.484375 C 2.0625 -5.003906 1.890625 -4.34375 1.890625 -3.5 Z M 1.890625 -3.5 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-8">
+<path style="stroke:none;" d="M 2.34375 -8.984375 L 2.34375 -7 L 4.71875 -7 L 4.71875 -6.109375 L 2.34375 -6.109375 L 2.34375 -2.3125 C 2.34375 -1.738281 2.421875 -1.367188 2.578125 -1.203125 C 2.734375 -1.046875 3.050781 -0.96875 3.53125 -0.96875 L 4.71875 -0.96875 L 4.71875 0 L 3.53125 0 C 2.644531 0 2.03125 -0.164062 1.6875 -0.5 C 1.351562 -0.832031 1.1875 -1.4375 1.1875 -2.3125 L 1.1875 -6.109375 L 0.34375 -6.109375 L 0.34375 -7 L 1.1875 -7 L 1.1875 -8.984375 Z M 2.34375 -8.984375 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-9">
+<path style="stroke:none;" d="M 3.96875 -9.703125 C 3.40625 -8.742188 2.988281 -7.796875 2.71875 -6.859375 C 2.445312 -5.929688 2.3125 -4.984375 2.3125 -4.015625 C 2.3125 -3.054688 2.445312 -2.101562 2.71875 -1.15625 C 3 -0.21875 3.414062 0.726562 3.96875 1.6875 L 2.96875 1.6875 C 2.34375 0.707031 1.875 -0.253906 1.5625 -1.203125 C 1.25 -2.148438 1.09375 -3.085938 1.09375 -4.015625 C 1.09375 -4.941406 1.25 -5.875 1.5625 -6.8125 C 1.875 -7.757812 2.34375 -8.722656 2.96875 -9.703125 Z M 3.96875 -9.703125 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-10">
+<path style="stroke:none;" d="M 1.59375 -1.0625 L 3.65625 -1.0625 L 3.65625 -8.171875 L 1.40625 -7.734375 L 1.40625 -8.875 L 3.640625 -9.328125 L 4.90625 -9.328125 L 4.90625 -1.0625 L 6.953125 -1.0625 L 6.953125 0 L 1.59375 0 Z M 1.59375 -1.0625 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-11">
+<path style="stroke:none;" d="M 1.03125 -9.703125 L 2.03125 -9.703125 C 2.65625 -8.722656 3.117188 -7.757812 3.421875 -6.8125 C 3.734375 -5.875 3.890625 -4.941406 3.890625 -4.015625 C 3.890625 -3.085938 3.734375 -2.148438 3.421875 -1.203125 C 3.117188 -0.253906 2.65625 0.707031 2.03125 1.6875 L 1.03125 1.6875 C 1.582031 0.726562 1.992188 -0.21875 2.265625 -1.15625 C 2.535156 -2.101562 2.671875 -3.054688 2.671875 -4.015625 C 2.671875 -4.984375 2.535156 -5.929688 2.265625 -6.859375 C 1.992188 -7.796875 1.582031 -8.742188 1.03125 -9.703125 Z M 1.03125 -9.703125 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-12">
+<path style="stroke:none;" d="M 6.234375 -3.5 C 6.234375 -4.34375 6.054688 -5.003906 5.703125 -5.484375 C 5.359375 -5.960938 4.882812 -6.203125 4.28125 -6.203125 C 3.664062 -6.203125 3.179688 -5.960938 2.828125 -5.484375 C 2.484375 -5.003906 2.3125 -4.34375 2.3125 -3.5 C 2.3125 -2.644531 2.484375 -1.976562 2.828125 -1.5 C 3.179688 -1.019531 3.664062 -0.78125 4.28125 -0.78125 C 4.882812 -0.78125 5.359375 -1.019531 5.703125 -1.5 C 6.054688 -1.976562 6.234375 -2.644531 6.234375 -3.5 Z M 2.3125 -5.9375 C 2.5625 -6.351562 2.867188 -6.660156 3.234375 -6.859375 C 3.597656 -7.066406 4.039062 -7.171875 4.5625 -7.171875 C 5.40625 -7.171875 6.09375 -6.832031 6.625 -6.15625 C 7.15625 -5.476562 7.421875 -4.59375 7.421875 -3.5 C 7.421875 -2.394531 7.15625 -1.503906 6.625 -0.828125 C 6.09375 -0.148438 5.40625 0.1875 4.5625 0.1875 C 4.039062 0.1875 3.597656 0.0859375 3.234375 -0.109375 C 2.867188 -0.316406 2.5625 -0.628906 2.3125 -1.046875 L 2.3125 0 L 1.15625 0 L 1.15625 -9.71875 L 2.3125 -9.71875 Z M 2.3125 -5.9375 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-13">
+<path style="stroke:none;" d="M 2.453125 -1.0625 L 6.859375 -1.0625 L 6.859375 0 L 0.9375 0 L 0.9375 -1.0625 C 1.414062 -1.5625 2.066406 -2.226562 2.890625 -3.0625 C 3.722656 -3.894531 4.242188 -4.429688 4.453125 -4.671875 C 4.859375 -5.128906 5.140625 -5.515625 5.296875 -5.828125 C 5.460938 -6.140625 5.546875 -6.445312 5.546875 -6.75 C 5.546875 -7.25 5.367188 -7.65625 5.015625 -7.96875 C 4.671875 -8.28125 4.21875 -8.4375 3.65625 -8.4375 C 3.257812 -8.4375 2.84375 -8.363281 2.40625 -8.21875 C 1.96875 -8.082031 1.5 -7.878906 1 -7.609375 L 1 -8.875 C 1.507812 -9.082031 1.984375 -9.238281 2.421875 -9.34375 C 2.867188 -9.445312 3.273438 -9.5 3.640625 -9.5 C 4.609375 -9.5 5.378906 -9.253906 5.953125 -8.765625 C 6.523438 -8.285156 6.8125 -7.640625 6.8125 -6.828125 C 6.8125 -6.453125 6.738281 -6.09375 6.59375 -5.75 C 6.445312 -5.40625 6.1875 -5 5.8125 -4.53125 C 5.707031 -4.40625 5.375 -4.054688 4.8125 -3.484375 C 4.257812 -2.910156 3.472656 -2.101562 2.453125 -1.0625 Z M 2.453125 -1.0625 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-14">
+<path style="stroke:none;" d="M 5.671875 -6.796875 L 5.671875 -5.703125 C 5.347656 -5.867188 5.007812 -5.992188 4.65625 -6.078125 C 4.300781 -6.160156 3.9375 -6.203125 3.5625 -6.203125 C 3 -6.203125 2.570312 -6.113281 2.28125 -5.9375 C 2 -5.769531 1.859375 -5.507812 1.859375 -5.15625 C 1.859375 -4.882812 1.957031 -4.671875 2.15625 -4.515625 C 2.363281 -4.367188 2.773438 -4.226562 3.390625 -4.09375 L 3.78125 -4 C 4.601562 -3.832031 5.1875 -3.585938 5.53125 -3.265625 C 5.875 -2.941406 6.046875 -2.5 6.046875 -1.9375 C 6.046875 -1.28125 5.785156 -0.757812 5.265625 -0.375 C 4.753906 0 4.050781 0.1875 3.15625 0.1875 C 2.78125 0.1875 2.390625 0.148438 1.984375 0.078125 C 1.578125 0.00390625 1.144531 -0.101562 0.6875 -0.25 L 0.6875 -1.4375 C 1.113281 -1.21875 1.53125 -1.050781 1.9375 -0.9375 C 2.351562 -0.832031 2.765625 -0.78125 3.171875 -0.78125 C 3.710938 -0.78125 4.128906 -0.875 4.421875 -1.0625 C 4.710938 -1.25 4.859375 -1.507812 4.859375 -1.84375 C 4.859375 -2.15625 4.753906 -2.394531 4.546875 -2.5625 C 4.335938 -2.726562 3.875 -2.890625 3.15625 -3.046875 L 2.765625 -3.140625 C 2.046875 -3.285156 1.53125 -3.515625 1.21875 -3.828125 C 0.90625 -4.140625 0.75 -4.566406 0.75 -5.109375 C 0.75 -5.765625 0.976562 -6.269531 1.4375 -6.625 C 1.90625 -6.988281 2.570312 -7.171875 3.4375 -7.171875 C 3.851562 -7.171875 4.25 -7.140625 4.625 -7.078125 C 5 -7.015625 5.347656 -6.921875 5.671875 -6.796875 Z M 5.671875 -6.796875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-15">
+<path style="stroke:none;" d="M 3.921875 -6.1875 C 3.304688 -6.1875 2.816406 -5.945312 2.453125 -5.46875 C 2.097656 -4.988281 1.921875 -4.332031 1.921875 -3.5 C 1.921875 -2.65625 2.097656 -1.992188 2.453125 -1.515625 C 2.804688 -1.035156 3.296875 -0.796875 3.921875 -0.796875 C 4.535156 -0.796875 5.019531 -1.035156 5.375 -1.515625 C 5.726562 -2.003906 5.90625 -2.664062 5.90625 -3.5 C 5.90625 -4.320312 5.726562 -4.972656 5.375 -5.453125 C 5.019531 -5.941406 4.535156 -6.1875 3.921875 -6.1875 Z M 3.921875 -7.171875 C 4.921875 -7.171875 5.703125 -6.84375 6.265625 -6.1875 C 6.835938 -5.539062 7.125 -4.644531 7.125 -3.5 C 7.125 -2.351562 6.835938 -1.453125 6.265625 -0.796875 C 5.703125 -0.140625 4.921875 0.1875 3.921875 0.1875 C 2.910156 0.1875 2.117188 -0.140625 1.546875 -0.796875 C 0.984375 -1.453125 0.703125 -2.351562 0.703125 -3.5 C 0.703125 -4.644531 0.984375 -5.539062 1.546875 -6.1875 C 2.117188 -6.84375 2.910156 -7.171875 3.921875 -7.171875 Z M 3.921875 -7.171875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-16">
+<path style="stroke:none;" d="M 1.09375 -2.765625 L 1.09375 -7 L 2.234375 -7 L 2.234375 -2.8125 C 2.234375 -2.144531 2.363281 -1.644531 2.625 -1.3125 C 2.882812 -0.976562 3.269531 -0.8125 3.78125 -0.8125 C 4.40625 -0.8125 4.894531 -1.007812 5.25 -1.40625 C 5.613281 -1.800781 5.796875 -2.34375 5.796875 -3.03125 L 5.796875 -7 L 6.953125 -7 L 6.953125 0 L 5.796875 0 L 5.796875 -1.078125 C 5.515625 -0.648438 5.191406 -0.332031 4.828125 -0.125 C 4.460938 0.0820312 4.035156 0.1875 3.546875 0.1875 C 2.742188 0.1875 2.132812 -0.0625 1.71875 -0.5625 C 1.300781 -1.0625 1.09375 -1.796875 1.09375 -2.765625 Z M 3.984375 -7.171875 Z M 3.984375 -7.171875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-17">
+<path style="stroke:none;" d="M 5.265625 -5.921875 C 5.128906 -5.992188 4.984375 -6.046875 4.828125 -6.078125 C 4.679688 -6.117188 4.519531 -6.140625 4.34375 -6.140625 C 3.6875 -6.140625 3.179688 -5.925781 2.828125 -5.5 C 2.484375 -5.082031 2.3125 -4.476562 2.3125 -3.6875 L 2.3125 0 L 1.15625 0 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.90625 C 2.5625 -6.332031 2.878906 -6.648438 3.265625 -6.859375 C 3.648438 -7.066406 4.117188 -7.171875 4.671875 -7.171875 C 4.753906 -7.171875 4.84375 -7.164062 4.9375 -7.15625 C 5.03125 -7.144531 5.132812 -7.128906 5.25 -7.109375 Z M 5.265625 -5.921875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-18">
+<path style="stroke:none;" d="M 6.25 -6.734375 L 6.25 -5.65625 C 5.914062 -5.832031 5.585938 -5.960938 5.265625 -6.046875 C 4.941406 -6.140625 4.613281 -6.1875 4.28125 -6.1875 C 3.53125 -6.1875 2.945312 -5.953125 2.53125 -5.484375 C 2.125 -5.015625 1.921875 -4.351562 1.921875 -3.5 C 1.921875 -2.644531 2.125 -1.976562 2.53125 -1.5 C 2.945312 -1.03125 3.53125 -0.796875 4.28125 -0.796875 C 4.613281 -0.796875 4.941406 -0.835938 5.265625 -0.921875 C 5.585938 -1.015625 5.914062 -1.148438 6.25 -1.328125 L 6.25 -0.265625 C 5.925781 -0.117188 5.59375 -0.0078125 5.25 0.0625 C 4.90625 0.144531 4.539062 0.1875 4.15625 0.1875 C 3.09375 0.1875 2.25 -0.144531 1.625 -0.8125 C 1.007812 -1.476562 0.703125 -2.375 0.703125 -3.5 C 0.703125 -4.632812 1.015625 -5.53125 1.640625 -6.1875 C 2.273438 -6.84375 3.132812 -7.171875 4.21875 -7.171875 C 4.570312 -7.171875 4.914062 -7.132812 5.25 -7.0625 C 5.59375 -6.988281 5.925781 -6.878906 6.25 -6.734375 Z M 6.25 -6.734375 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-19">
+<path style="stroke:none;" d="M 2.3125 -1.046875 L 2.3125 2.65625 L 1.15625 2.65625 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.9375 C 2.5625 -6.351562 2.867188 -6.660156 3.234375 -6.859375 C 3.597656 -7.066406 4.039062 -7.171875 4.5625 -7.171875 C 5.40625 -7.171875 6.09375 -6.832031 6.625 -6.15625 C 7.15625 -5.476562 7.421875 -4.59375 7.421875 -3.5 C 7.421875 -2.394531 7.15625 -1.503906 6.625 -0.828125 C 6.09375 -0.148438 5.40625 0.1875 4.5625 0.1875 C 4.039062 0.1875 3.597656 0.0859375 3.234375 -0.109375 C 2.867188 -0.316406 2.5625 -0.628906 2.3125 -1.046875 Z M 6.234375 -3.5 C 6.234375 -4.34375 6.054688 -5.003906 5.703125 -5.484375 C 5.359375 -5.960938 4.882812 -6.203125 4.28125 -6.203125 C 3.664062 -6.203125 3.179688 -5.960938 2.828125 -5.484375 C 2.484375 -5.003906 2.3125 -4.34375 2.3125 -3.5 C 2.3125 -2.644531 2.484375 -1.976562 2.828125 -1.5 C 3.179688 -1.019531 3.664062 -0.78125 4.28125 -0.78125 C 4.882812 -0.78125 5.359375 -1.019531 5.703125 -1.5 C 6.054688 -1.976562 6.234375 -2.644531 6.234375 -3.5 Z M 6.234375 -3.5 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-20">
+<path style="stroke:none;" d="M 4.0625 -8.5 C 3.414062 -8.5 2.925781 -8.175781 2.59375 -7.53125 C 2.269531 -6.894531 2.109375 -5.9375 2.109375 -4.65625 C 2.109375 -3.375 2.269531 -2.410156 2.59375 -1.765625 C 2.925781 -1.128906 3.414062 -0.8125 4.0625 -0.8125 C 4.71875 -0.8125 5.207031 -1.128906 5.53125 -1.765625 C 5.863281 -2.410156 6.03125 -3.375 6.03125 -4.65625 C 6.03125 -5.9375 5.863281 -6.894531 5.53125 -7.53125 C 5.207031 -8.175781 4.71875 -8.5 4.0625 -8.5 Z M 4.0625 -9.5 C 5.113281 -9.5 5.914062 -9.082031 6.46875 -8.25 C 7.019531 -7.425781 7.296875 -6.226562 7.296875 -4.65625 C 7.296875 -3.082031 7.019531 -1.878906 6.46875 -1.046875 C 5.914062 -0.222656 5.113281 0.1875 4.0625 0.1875 C 3.019531 0.1875 2.222656 -0.222656 1.671875 -1.046875 C 1.117188 -1.878906 0.84375 -3.082031 0.84375 -4.65625 C 0.84375 -6.226562 1.117188 -7.425781 1.671875 -8.25 C 2.222656 -9.082031 3.019531 -9.5 4.0625 -9.5 Z M 4.0625 -9.5 "/>
+</symbol>
+</g>
+</defs>
+<g id="surface11910">
+<rect x="0" y="0" width="296" height="142" style="fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;"/>
+<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 16.35 11.5 L 25.75 11.5 L 25.75 18.4 L 16.35 18.4 Z M 16.35 11.5 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.919141 14.75625 C 25.919141 14.99375 25.562695 14.99375 25.562695 14.75625 C 25.562695 14.518555 25.919141 14.518555 25.919141 14.75625 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.140039 13.450586 C 18.140039 13.69707 17.770117 13.69707 17.770117 13.450586 C 17.770117 13.203906 18.140039 13.203906 18.140039 13.450586 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.116211 16.39043 C 18.116211 16.637109 17.746484 16.637109 17.746484 16.39043 C 17.746484 16.143945 18.116211 16.143945 18.116211 16.39043 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-dasharray:1,1;stroke-miterlimit:10;" d="M 18.160937 16.341406 L 25.344141 14.80293 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.016797 15.128711 L 25.45332 14.779687 L 24.912109 14.639844 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.140039 13.450586 L 25.342383 14.717383 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 24.916797 14.896484 L 25.452539 14.736914 L 25.003516 14.403906 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.918945 14.75625 L 30.732813 14.715039 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 30.34668 14.968359 L 30.844531 14.714062 L 30.342383 14.468359 " transform="matrix(20,0,0,20,-325,-228)"/>
+<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
+  <use xlink:href="#glyph0-1" x="34.101562" y="34.267687"/>
+  <use xlink:href="#glyph0-2" x="37.712674" y="34.267687"/>
+  <use xlink:href="#glyph0-3" x="50.212674" y="34.267687"/>
+  <use xlink:href="#glyph0-4" x="57.990451" y="34.267687"/>
+  <use xlink:href="#glyph0-5" x="66.046007" y="34.267687"/>
+  <use xlink:href="#glyph0-6" x="73.823785" y="34.267687"/>
+  <use xlink:href="#glyph0-7" x="77.990451" y="34.267687"/>
+  <use xlink:href="#glyph0-3" x="86.046007" y="34.267687"/>
+  <use xlink:href="#glyph0-8" x="93.823785" y="34.267687"/>
+  <use xlink:href="#glyph0-3" x="98.823785" y="34.267687"/>
+  <use xlink:href="#glyph0-6" x="106.601562" y="34.267687"/>
+  <use xlink:href="#glyph0-9" x="110.768229" y="34.267687"/>
+  <use xlink:href="#glyph0-10" x="115.768229" y="34.267687"/>
+  <use xlink:href="#glyph0-11" x="123.823785" y="34.267687"/>
+</g>
+<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
+  <use xlink:href="#glyph0-5" x="33.625" y="115.412218"/>
+  <use xlink:href="#glyph0-2" x="41.402778" y="115.412218"/>
+  <use xlink:href="#glyph0-12" x="53.902778" y="115.412218"/>
+  <use xlink:href="#glyph0-5" x="61.958333" y="115.412218"/>
+  <use xlink:href="#glyph0-7" x="69.736111" y="115.412218"/>
+  <use xlink:href="#glyph0-7" x="77.791667" y="115.412218"/>
+  <use xlink:href="#glyph0-5" x="85.847222" y="115.412218"/>
+  <use xlink:href="#glyph0-7" x="93.625" y="115.412218"/>
+  <use xlink:href="#glyph0-6" x="101.680556" y="115.412218"/>
+  <use xlink:href="#glyph0-7" x="105.847222" y="115.412218"/>
+  <use xlink:href="#glyph0-3" x="113.902778" y="115.412218"/>
+  <use xlink:href="#glyph0-8" x="121.680556" y="115.412218"/>
+  <use xlink:href="#glyph0-3" x="126.680556" y="115.412218"/>
+  <use xlink:href="#glyph0-6" x="134.458333" y="115.412218"/>
+  <use xlink:href="#glyph0-9" x="138.625" y="115.412218"/>
+  <use xlink:href="#glyph0-13" x="143.625" y="115.412218"/>
+  <use xlink:href="#glyph0-11" x="151.680556" y="115.412218"/>
+</g>
+<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
+  <use xlink:href="#glyph0-14" x="197" y="54.505968"/>
+  <use xlink:href="#glyph0-15" x="203.666667" y="54.505968"/>
+  <use xlink:href="#glyph0-16" x="211.444444" y="54.505968"/>
+  <use xlink:href="#glyph0-17" x="219.5" y="54.505968"/>
+  <use xlink:href="#glyph0-18" x="224.5" y="54.505968"/>
+  <use xlink:href="#glyph0-5" x="231.444444" y="54.505968"/>
+  <use xlink:href="#glyph0-6" x="239.222222" y="54.505968"/>
+  <use xlink:href="#glyph0-19" x="243.388889" y="54.505968"/>
+  <use xlink:href="#glyph0-3" x="251.444444" y="54.505968"/>
+  <use xlink:href="#glyph0-7" x="259.222222" y="54.505968"/>
+  <use xlink:href="#glyph0-6" x="267.277778" y="54.505968"/>
+  <use xlink:href="#glyph0-9" x="271.444444" y="54.505968"/>
+  <use xlink:href="#glyph0-20" x="276.444444" y="54.505968"/>
+  <use xlink:href="#glyph0-11" x="284.5" y="54.505968"/>
+</g>
+</g>
+</svg>
diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index bb86cadfad1c..b0774b9a9b71 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -846,3 +846,5 @@ stream while it may be possible to enable and disable the embedded data stream.
 
 The embedded data format does not need to be configured on the sensor's pads as
 the format is dictated by the pixel data format in this case.
+
+.. include:: subdev-config-model.rst
diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
new file mode 100644
index 000000000000..1e6c58931ea0
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
@@ -0,0 +1,230 @@
+.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
+
+.. _media_subdev_config_model:
+
+Sub-device configuration models
+===============================
+
+The V4L2 specification defines a subdev API that exposes three type of
+configuration elements: formats, selection rectangles and controls. The
+specification contains generic information about how those configuration
+elements behave, but not precisely how they apply to particular hardware
+features. We leave some leeway to drivers to decide how to map selection
+rectangles to device features, as long as they comply with the V4L2
+specification. This is needed as hardware features differ between devices, so
+it's the driver's responsibility to handle this mapping.
+
+Unfortunately, this lack of clearly defined mapping in the specification has led
+to different drivers mapping the same hardware features to different API
+elements, or implementing the API elements with slightly different
+behaviours. Furthermore, many drivers have implemented selection rectangles in
+ways that do not comply with the V4L2 specification. All of this makes userspace
+development difficult.
+
+Sub-device configuration models specify in detail what the user space can expect
+from a sub-device in terms of V4L2 sub-device interface support, semantics
+included.
+
+A sub-device may implement more than one configuration model at the same
+time. The implemented configuration models can be obtained from the sub-device's
+``V4L2_CID_CONFIG_MODEL`` control.
+
+.. _media_subdev_config_model_common_raw_sensor:
+
+Common raw camera sensor model
+------------------------------
+
+The common raw camera sensor model defines a set of enumeration and
+configuration interfaces (formats, selections etc.) that cover the vast majority
+of functionality of raw camera sensors. Not all of the interfaces are
+necessarily offered by all drivers.
+
+A sub-device complies with the common raw sensor model if the
+``V4L2_CONFIG_MODEL_COMMON_RAW_SENSOR`` bit is set in the
+``V4L2_CID_CONFIG_MODEL`` control of the sub-device.
+
+The common raw camera sensor model is aligned with
+:ref:`media_using_camera_sensor_drivers`. Please refer to that regarding aspects
+not specified here.
+
+Each camera sensor implementing the common raw sensor model exposes a single
+V4L2 sub-device. The sub-device contains a single source pad (0) and two or more
+internal pads: one or more image data internal pads (starting from 1) and
+optionally an embedded data pad.
+
+Additionally, further internal pads may be supported for other features. Using
+more than one image data internal pad or more than one non-image data pad
+requires these pads documented separately for the given device. The indices of
+the image data internal pads shall be lower than those of the non-image data
+pads.
+
+This is shown in :ref:`media_subdev_config_model_common_raw_sensor_subdev`.
+
+.. _media_subdev_config_model_common_raw_sensor_subdev:
+
+.. kernel-figure:: common-raw-sensor.svg
+    :alt:    common-raw-sensor.svg
+    :align:  center
+
+    **Common raw sensor sub-device with n pads (n == 2)**
+
+Routes
+^^^^^^
+
+A sub-device conforming to common raw camera sensor model implements the
+following routes.
+
+.. flat-table:: Routes
+    :header-rows: 1
+
+    * - Sink pad/stream
+      - Source pad/stream
+      - Static (X/M(aybe)/-)
+      - Mandatory (X/-)
+      - Synopsis
+    * - 1/0
+      - 0/0
+      - X
+      - X
+      - Image data
+    * - 2/0
+      - 0/1
+      - M
+      - \-
+      - Embedded data
+
+Support for the embedded data stream is optional. Drivers supporting the
+embedded data stream may allow disabling and enabling the route when the
+streaming is disabled.
+
+Sensor pixel array size, cropping and binning
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The sensor's pixel array is divided into one or more areas. The areas around the
+edge of the pixel array, usually one or more sides, may contain optical black
+pixels, dummy pixels and other non-image pixels. The entire pixel array size is
+conveyed by the format on (pad, stream) pair 1/0.
+
+A rectangle within the pixel array contains the visible pixels. Capturing the
+non-visible pixels outside the visible pixel area may be supported by the
+sensor. The visible pixel area corresponds to the ``V4L2_SEL_TGT_CROP_DEFAULT``
+selection target on (pad, stream) pair 1/0.
+
+Sensors can perform multiple operations that affect the output image size. First
+of these is the analogue crop. Analogue crop limits the area of the pixel array
+which the sensor will read, affecting sensor timing as well. The granularity of
+the analogue crop configuration varies greatly across sensors: some sensors
+support only a few different analogue crop configurations whereas others may
+support anything divisible by a given number of pixels. The analogue crop
+configuration corresponds to the ``V4L2_SEL_TGT_CROP`` selection target on (pad,
+stream) pair 1/0. The default analogue crop rectangle corresponds to the visible
+pixel area.
+
+In the next step, binning is performed on the image data read from camera
+sensor's pixel array, as determined by the analogue crop configuration. Enabling
+binning will effectively result in an image smaller than the original by given
+binning factors horizontally and vertically. Typical values are 1/2 and 1/3 but
+others may well be supported by the hardware as well.
+
+Sub-sampling follows binning. Sub-sampling, like binning, reduces the size of
+the image by including only a subset of samples read from the sensor's pixel
+matrix, typically every n'th pixel horizontally and vertically, taking the
+sensor's color pattern into account. Sub-sampling is generally configurable
+separately horizontally and vertically.
+
+Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE``
+rectangle, relative to the analogue crop rectangle, on (pad, stream) pair
+1/0. The driver implementation determines how to configure binning and
+sub-sampling to achieve the desired size.
+
+The digital crop operation takes place after binning and sub-sampling. It is
+configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
+0/0. The resulting image size is further output by the sensor.
+
+The sensor's output mbus code is configured by setting the format on the (pad,
+stream) pair 0/0. When setting the format, always use the same width and height
+as for the digital crop setting.
+
+Drivers may only support some or even none of these configurations, in which
+case they do not expose the corresponding selection rectangles. If any selection
+targets are omitted, the further selection rectangle or format is instead
+related to the previous implemented selection rectangle. For instance, if the
+sensor supports binning but not analogue crop, then the binning configuration
+(``V4L2_SEL_TGT_COMPOSE`` selection target) is done in relation to the visible
+pixel area (``V4L2_SEL_TGT_CROP_DEFAULT`` selection target).
+
+Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
+
+.. flat-table:: Selection targets on pads
+    :header-rows: 1
+
+    * - Pad/Stream
+      - Selection target/format
+      - Mandatory (X/-)
+      - Modifiable (X/-)
+      - Synopsis
+    * - 1/0
+      - Format
+      - X
+      - \-
+      - Image data format. The width and the height fields indicates the full
+        size of the pixel array, including non-visible pixels. The media bus
+        code of this format reflects the native pixel depth of the sensor.
+    * - 1/0
+      - ``V4L2_SEL_TGT_CROP_DEFAULT``
+      - X
+      - \
+      - The visible pixel area. This rectangle is relative to the format on the
+        same (pad, stream).
+    * - 1/0
+      - ``V4L2_SEL_TGT_CROP``
+      - \-
+      - X
+      - Analogue crop. Analogue crop typically has a coarse granularity. This
+        rectangle is relative to the format on the same (pad, stream).
+    * - 1/0
+      - ``V4L2_SEL_TGT_COMPOSE``
+      - \-
+      - X
+      - Binning and sub-sampling. This rectangle is relative to the
+        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The
+        combination of binning and sub-sampling is configured using this
+        selection target.
+    * - 2/0
+      - Format
+      - X
+      - \-
+      - Embedded data format.
+    * - 0/0
+      - ``V4L2_SEL_TGT_CROP``
+      - \-
+      - X
+      - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE``
+        rectangle on (pad, stream) pair 1/0.
+    * - 0/0
+      - Format
+      - X
+      - X
+      - Image data source format. Always assign the width and height fields of
+        the format to the same values than for the ``V4L2_SEL_TGT_CROP``
+        rectangle on (pad, stream) pair 0/0. The media bus code reflects the
+        pixel data output of the sensor.
+    * - 0/1
+      - Format
+      - X
+      - \-
+      - Embedded data source format.
+
+Embedded data
+^^^^^^^^^^^^^
+
+The embedded data stream is produced by the sensor when the corresponding route
+is enabled. The embedded data route may also be immutable or not exist at all,
+in case the sensor (or the driver) does not support it.
+
+Generally the sensor embedded data width is determined by the width of the image
+data whereas the number of lines are constant for the embedded data. The user
+space may obtain the size of the embedded data once the image data size on the
+source pad has been configured.
+
+Also see :ref:`media_using_camera_sensor_drivers_embedded_data`.
-- 
2.47.2

Re: [PATCH v11 39/66] media: Documentation: Add subdev configuration models, raw sensor model

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:40PM +0300, Sakari Ailus wrote:
> Sub-device configuration models define what V4L2 API elements are
> available on a compliant sub-device and how do they behave.
>
> The patch also adds a model for common raw sensors.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
> Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> ---
>  .../media/drivers/camera-sensor.rst           |   4 +
>  .../media/v4l/common-raw-sensor.dia           | 442 ++++++++++++++++++
>  .../media/v4l/common-raw-sensor.svg           | 134 ++++++
>  .../userspace-api/media/v4l/dev-subdev.rst    |   2 +
>  .../media/v4l/subdev-config-model.rst         | 230 +++++++++
>  5 files changed, 812 insertions(+)
>  create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.dia
>  create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.svg
>  create mode 100644 Documentation/userspace-api/media/v4l/subdev-config-model.rst
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index cbbfbb0d8273..39f3f91c6733 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -18,6 +18,8 @@ binning functionality. The sensor drivers belong to two distinct classes, freely
>  configurable and register list-based drivers, depending on how the driver
>  configures this functionality.
>
> +Also see :ref:`media_subdev_config_model_common_raw_sensor`.
> +
>  Freely configurable camera sensor drivers
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> @@ -118,6 +120,8 @@ values programmed by the register sequences. The default values of these
>  controls shall be 0 (disabled). Especially these controls shall not be inverted,
>  independently of the sensor's mounting rotation.
>
> +.. _media_using_camera_sensor_drivers_embedded_data:
> +
>  Embedded data
>  -------------
>
> diff --git a/Documentation/userspace-api/media/v4l/common-raw-sensor.dia b/Documentation/userspace-api/media/v4l/common-raw-sensor.dia
> new file mode 100644
> index 000000000000..24b3f2b2a626
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/common-raw-sensor.dia

[snip]

> diff --git a/Documentation/userspace-api/media/v4l/common-raw-sensor.svg b/Documentation/userspace-api/media/v4l/common-raw-sensor.svg
> new file mode 100644
> index 000000000000..1d6055da2519
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/common-raw-sensor.svg

[snip]

> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index bb86cadfad1c..b0774b9a9b71 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -846,3 +846,5 @@ stream while it may be possible to enable and disable the embedded data stream.
>
>  The embedded data format does not need to be configured on the sensor's pads as
>  the format is dictated by the pixel data format in this case.
> +
> +.. include:: subdev-config-model.rst
> diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> new file mode 100644
> index 000000000000..1e6c58931ea0
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> @@ -0,0 +1,230 @@
> +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
> +
> +.. _media_subdev_config_model:
> +
> +Sub-device configuration models
> +===============================
> +
> +The V4L2 specification defines a subdev API that exposes three type of
> +configuration elements: formats, selection rectangles and controls. The
> +specification contains generic information about how those configuration
> +elements behave, but not precisely how they apply to particular hardware
> +features. We leave some leeway to drivers to decide how to map selection
> +rectangles to device features, as long as they comply with the V4L2
> +specification. This is needed as hardware features differ between devices, so
> +it's the driver's responsibility to handle this mapping.
> +
> +Unfortunately, this lack of clearly defined mapping in the specification has led
> +to different drivers mapping the same hardware features to different API
> +elements, or implementing the API elements with slightly different
> +behaviours. Furthermore, many drivers have implemented selection rectangles in
> +ways that do not comply with the V4L2 specification. All of this makes userspace
> +development difficult.
> +
> +Sub-device configuration models specify in detail what the user space can expect
> +from a sub-device in terms of V4L2 sub-device interface support, semantics
> +included.
> +
> +A sub-device may implement more than one configuration model at the same
> +time. The implemented configuration models can be obtained from the sub-device's
> +``V4L2_CID_CONFIG_MODEL`` control.
> +
> +.. _media_subdev_config_model_common_raw_sensor:
> +
> +Common raw camera sensor model
> +------------------------------
> +
> +The common raw camera sensor model defines a set of enumeration and
> +configuration interfaces (formats, selections etc.) that cover the vast majority
> +of functionality of raw camera sensors. Not all of the interfaces are
> +necessarily offered by all drivers.
> +
> +A sub-device complies with the common raw sensor model if the
> +``V4L2_CONFIG_MODEL_COMMON_RAW_SENSOR`` bit is set in the
> +``V4L2_CID_CONFIG_MODEL`` control of the sub-device.
> +
> +The common raw camera sensor model is aligned with
> +:ref:`media_using_camera_sensor_drivers`. Please refer to that regarding aspects
> +not specified here.
> +
> +Each camera sensor implementing the common raw sensor model exposes a single
> +V4L2 sub-device. The sub-device contains a single source pad (0) and two or more
> +internal pads: one or more image data internal pads (starting from 1) and
> +optionally an embedded data pad.
> +
> +Additionally, further internal pads may be supported for other features. Using
> +more than one image data internal pad or more than one non-image data pad
> +requires these pads documented separately for the given device. The indices of
> +the image data internal pads shall be lower than those of the non-image data
> +pads.
> +
> +This is shown in :ref:`media_subdev_config_model_common_raw_sensor_subdev`.

possibly doesn't need a link as the image is just here below

> +
> +.. _media_subdev_config_model_common_raw_sensor_subdev:
> +
> +.. kernel-figure:: common-raw-sensor.svg
> +    :alt:    common-raw-sensor.svg
> +    :align:  center
> +
> +    **Common raw sensor sub-device with n pads (n == 2)**
> +
> +Routes
> +^^^^^^
> +
> +A sub-device conforming to common raw camera sensor model implements the
> +following routes.
> +
> +.. flat-table:: Routes
> +    :header-rows: 1
> +
> +    * - Sink pad/stream
> +      - Source pad/stream
> +      - Static (X/M(aybe)/-)
> +      - Mandatory (X/-)
> +      - Synopsis
> +    * - 1/0
> +      - 0/0
> +      - X
> +      - X
> +      - Image data
> +    * - 2/0
> +      - 0/1
> +      - M
> +      - \-
> +      - Embedded data
> +
> +Support for the embedded data stream is optional. Drivers supporting the
> +embedded data stream may allow disabling and enabling the route when the
> +streaming is disabled.

I would

s/when the streaming is disabled//

> +
> +Sensor pixel array size, cropping and binning
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +The sensor's pixel array is divided into one or more areas. The areas around the
> +edge of the pixel array, usually one or more sides, may contain optical black

You say that "the pixel array is divided into one or more areas" and
then list "the areas around the edge of the pixel array" which is confusing

I think it would be better as

The sensor's full pixel array is divided into one or more areas, one
(or multiple) active area which contains visible pixels surrounded,
usually on one or more sides, by non-active areas which may contain
optical black pixels, dummy pixels and other non-image pixels. The
entire pixel array areas size, including the active and non-active
portions is conveyed by the format on (pad, stream) pair 1/0.

This would also better define the "visible pixels" term which is used
in the rest of the documentation.

> +
> +A rectangle within the pixel array contains the visible pixels. Capturing the

If you accept the above, you can drop the first sentence here

> +non-visible pixels outside the visible pixel area may be supported by the
> +sensor. The visible pixel area corresponds to the ``V4L2_SEL_TGT_CROP_DEFAULT``
> +selection target on (pad, stream) pair 1/0.
> +
> +Sensors can perform multiple operations that affect the output image size. First
> +of these is the analogue crop. Analogue crop limits the area of the pixel array

s/First one of these/The first one of these/

> +which the sensor will read, affecting sensor timing as well. The granularity of
> +the analogue crop configuration varies greatly across sensors: some sensors
> +support only a few different analogue crop configurations whereas others may
> +support anything divisible by a given number of pixels. The analogue crop
> +configuration corresponds to the ``V4L2_SEL_TGT_CROP`` selection target on (pad,
> +stream) pair 1/0. The default analogue crop rectangle corresponds to the visible
> +pixel area.
> +
> +In the next step, binning is performed on the image data read from camera
> +sensor's pixel array, as determined by the analogue crop configuration. Enabling
> +binning will effectively result in an image smaller than the original by given
> +binning factors horizontally and vertically. Typical values are 1/2 and 1/3 but
> +others may well be supported by the hardware as well.
> +
> +Sub-sampling follows binning. Sub-sampling, like binning, reduces the size of
> +the image by including only a subset of samples read from the sensor's pixel
> +matrix, typically every n'th pixel horizontally and vertically, taking the
> +sensor's color pattern into account. Sub-sampling is generally configurable
> +separately horizontally and vertically.
> +
> +Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE``

s/Binning and sub-sampling are/The combined effect of binning and
sub-sampling is/

> +rectangle, relative to the analogue crop rectangle, on (pad, stream) pair
> +1/0. The driver implementation determines how to configure binning and
> +sub-sampling to achieve the desired size.
> +
> +The digital crop operation takes place after binning and sub-sampling. It is
> +configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
> +0/0. The resulting image size is further output by the sensor.

by the sensor on the bus.

?

> +
> +The sensor's output mbus code is configured by setting the format on the (pad,
> +stream) pair 0/0. When setting the format, always use the same width and height
> +as for the digital crop setting.
> +
> +Drivers may only support some or even none of these configurations, in which
> +case they do not expose the corresponding selection rectangles. If any selection
> +targets are omitted, the further selection rectangle or format is instead
> +related to the previous implemented selection rectangle. For instance, if the
> +sensor supports binning but not analogue crop, then the binning configuration
> +(``V4L2_SEL_TGT_COMPOSE`` selection target) is done in relation to the visible
> +pixel area (``V4L2_SEL_TGT_CROP_DEFAULT`` selection target).
> +
> +Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
> +
> +.. flat-table:: Selection targets on pads
> +    :header-rows: 1
> +
> +    * - Pad/Stream
> +      - Selection target/format
> +      - Mandatory (X/-)
> +      - Modifiable (X/-)
> +      - Synopsis
> +    * - 1/0
> +      - Format
> +      - X
> +      - \-
> +      - Image data format. The width and the height fields indicates the full
> +        size of the pixel array, including non-visible pixels. The media bus
> +        code of this format reflects the native pixel depth of the sensor.
> +    * - 1/0
> +      - ``V4L2_SEL_TGT_CROP_DEFAULT``
> +      - X
> +      - \
> +      - The visible pixel area. This rectangle is relative to the format on the
> +        same (pad, stream).
> +    * - 1/0
> +      - ``V4L2_SEL_TGT_CROP``
> +      - \-
> +      - X
> +      - Analogue crop. Analogue crop typically has a coarse granularity. This
> +        rectangle is relative to the format on the same (pad, stream).
> +    * - 1/0
> +      - ``V4L2_SEL_TGT_COMPOSE``
> +      - \-
> +      - X
> +      - Binning and sub-sampling. This rectangle is relative to the
> +        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The
> +        combination of binning and sub-sampling is configured using this
> +        selection target.
> +    * - 2/0
> +      - Format
> +      - X
> +      - \-
> +      - Embedded data format.
> +    * - 0/0
> +      - ``V4L2_SEL_TGT_CROP``
> +      - \-
> +      - X
> +      - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE``
> +        rectangle on (pad, stream) pair 1/0.
> +    * - 0/0
> +      - Format
> +      - X
> +      - X
> +      - Image data source format. Always assign the width and height fields of
> +        the format to the same values than for the ``V4L2_SEL_TGT_CROP``
> +        rectangle on (pad, stream) pair 0/0. The media bus code reflects the
> +        pixel data output of the sensor.
> +    * - 0/1
> +      - Format
> +      - X
> +      - \-
> +      - Embedded data source format.
> +
> +Embedded data
> +^^^^^^^^^^^^^
> +
> +The embedded data stream is produced by the sensor when the corresponding route
> +is enabled. The embedded data route may also be immutable or not exist at all,
> +in case the sensor (or the driver) does not support it.
> +
> +Generally the sensor embedded data width is determined by the width of the image
> +data whereas the number of lines are constant for the embedded data. The user
> +space may obtain the size of the embedded data once the image data size on the
> +source pad has been configured.
> +
> +Also see :ref:`media_using_camera_sensor_drivers_embedded_data`.

Neat!

Take in any of the suggestions I made you like.

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j


> --
> 2.47.2
>
>

[PATCH v11 40/66] media: Document enable_streams and disable_streams behaviour

[Sakari Ailus]

Document that enable_streams may start additional streams and
disable_streams may not disable requested streams if other related streams
are still enabled.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 include/media/v4l2-subdev.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 4b28086808c9..96a9c869ffb1 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -814,6 +814,10 @@ struct v4l2_subdev_state {
  *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
  *	argument.
  *
+ *	Starting the requested streams may require starting additional
+ *	streams. Streams that are started together due to hardware are called a
+ *	stream group.
+ *
  * @disable_streams: Disable the streams defined in streams_mask on the given
  *	source pad. Subdevs that implement this operation must use the active
  *	state management provided by the subdev core (enabled through a call to
@@ -823,6 +827,9 @@ struct v4l2_subdev_state {
  *	Drivers that support only a single stream without setting the
  *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
  *	argument.
+ *
+ *	A stream is only disabled once all streams in the stream group are
+ *	disabled.
  */
 struct v4l2_subdev_pad_ops {
 	int (*enum_mbus_code)(struct v4l2_subdev *sd,
-- 
2.47.2

Re: [PATCH v11 40/66] media: Document enable_streams and disable_streams behaviour

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:41PM +0300, Sakari Ailus wrote:
> Document that enable_streams may start additional streams and
> disable_streams may not disable requested streams if other related streams
> are still enabled.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Can be fast-tracked outside of this series even

> ---
>  include/media/v4l2-subdev.h | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 4b28086808c9..96a9c869ffb1 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -814,6 +814,10 @@ struct v4l2_subdev_state {
>   *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
>   *	argument.
>   *
> + *	Starting the requested streams may require starting additional
> + *	streams. Streams that are started together due to hardware are called a
> + *	stream group.
> + *
>   * @disable_streams: Disable the streams defined in streams_mask on the given
>   *	source pad. Subdevs that implement this operation must use the active
>   *	state management provided by the subdev core (enabled through a call to
> @@ -823,6 +827,9 @@ struct v4l2_subdev_state {
>   *	Drivers that support only a single stream without setting the
>   *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
>   *	argument.
> + *
> + *	A stream is only disabled once all streams in the stream group are
> + *	disabled.
>   */
>  struct v4l2_subdev_pad_ops {
>  	int (*enum_mbus_code)(struct v4l2_subdev *sd,
> --
> 2.47.2
>
>

Re: [PATCH v11 40/66] media: Document enable_streams and disable_streams behaviour

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Aug 25, 2025 at 12:50:41PM +0300, Sakari Ailus wrote:
> Document that enable_streams may start additional streams and
> disable_streams may not disable requested streams if other related streams
> are still enabled.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  include/media/v4l2-subdev.h | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 4b28086808c9..96a9c869ffb1 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -814,6 +814,10 @@ struct v4l2_subdev_state {
>   *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
>   *	argument.
>   *
> + *	Starting the requested streams may require starting additional
> + *	streams. Streams that are started together due to hardware are called a
> + *	stream group.

I initially interpreted this as meaning that, to start the requested
streams at the hardware level, the caller may need to start additional
streams. That is, the .enable_streams() call may not have any effect
until other streams are also started by the caller.

> + *
>   * @disable_streams: Disable the streams defined in streams_mask on the given
>   *	source pad. Subdevs that implement this operation must use the active
>   *	state management provided by the subdev core (enabled through a call to
> @@ -823,6 +827,9 @@ struct v4l2_subdev_state {
>   *	Drivers that support only a single stream without setting the
>   *	V4L2_SUBDEV_CAP_STREAMS sub-device capability flag can ignore the mask
>   *	argument.
> + *
> + *	A stream is only disabled once all streams in the stream group are
> + *	disabled.

But this implies it's the other way around. A clarification is needed.
Which behaviour are you trying to document ?

>   */
>  struct v4l2_subdev_pad_ops {
>  	int (*enum_mbus_code)(struct v4l2_subdev *sd,

-- 
Regards,

Laurent Pinchart

[PATCH v11 41/66] media: Documentation: Add scaling and post-scaler crop for common raw

[Sakari Ailus]

Document scaling and post-scaler digital crop operations for the common
raw sensor model.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
---
 .../media/v4l/subdev-config-model.rst         | 24 +++++++++++++++----
 1 file changed, 19 insertions(+), 5 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
index 1e6c58931ea0..c1b9b74cbcef 100644
--- a/Documentation/userspace-api/media/v4l/subdev-config-model.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
@@ -139,11 +139,18 @@ sub-sampling to achieve the desired size.
 
 The digital crop operation takes place after binning and sub-sampling. It is
 configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
-0/0. The resulting image size is further output by the sensor.
+0/0.
+
+The scaling operation is performed after the digital crop. It is configured by
+setting the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 0/0,
+relative to the digital crop.
 
 The sensor's output mbus code is configured by setting the format on the (pad,
-stream) pair 0/0. When setting the format, always use the same width and height
-as for the digital crop setting.
+stream) pair 0/0. The width and height fields are used to configure post-scaler
+digital crop if supported by the driver, affecting the right and bottom edges of
+the frame. If post-scaler digital crop is not supported, the width and height
+fields of the format will match the compose rectangle sizes applied on the same
+0/0 (pad, stream) pair.
 
 Drivers may only support some or even none of these configurations, in which
 case they do not expose the corresponding selection rectangles. If any selection
@@ -201,12 +208,19 @@ Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
       - X
       - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE``
         rectangle on (pad, stream) pair 1/0.
+    * - 0/0
+      - ``V4L2_SEL_TGT_COMPOSE``
+      - \-
+      - X
+      - Scaling. This rectangle is relative to the ``V4L2_SEL_TGT_CROP``
+        rectangle on (pad, stream) pair 0/0.
     * - 0/0
       - Format
       - X
       - X
-      - Image data source format. Always assign the width and height fields of
-        the format to the same values than for the ``V4L2_SEL_TGT_CROP``
+      - Image data source format and post-scaler crop. The width and height
+        fields of the format, used to configure post-scaler crop on the right
+        and bottom edges of the image, are related to the ``V4L2_SEL_TGT_COMPOSE``
         rectangle on (pad, stream) pair 0/0. The media bus code reflects the
         pixel data output of the sensor.
     * - 0/1
-- 
2.47.2

Re: [PATCH v11 41/66] media: Documentation: Add scaling and post-scaler crop for common raw

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:42PM +0300, Sakari Ailus wrote:
> Document scaling and post-scaler digital crop operations for the common
> raw sensor model.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
> ---
>  .../media/v4l/subdev-config-model.rst         | 24 +++++++++++++++----
>  1 file changed, 19 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> index 1e6c58931ea0..c1b9b74cbcef 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> @@ -139,11 +139,18 @@ sub-sampling to achieve the desired size.
>
>  The digital crop operation takes place after binning and sub-sampling. It is
>  configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
> -0/0. The resulting image size is further output by the sensor.
> +0/0.
> +
> +The scaling operation is performed after the digital crop. It is configured by
> +setting the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 0/0,
> +relative to the digital crop.
>
>  The sensor's output mbus code is configured by setting the format on the (pad,
> -stream) pair 0/0. When setting the format, always use the same width and height
> -as for the digital crop setting.
> +stream) pair 0/0. The width and height fields are used to configure post-scaler
> +digital crop if supported by the driver, affecting the right and bottom edges of
> +the frame. If post-scaler digital crop is not supported, the width and height
> +fields of the format will match the compose rectangle sizes applied on the same
> +0/0 (pad, stream) pair.
>
>  Drivers may only support some or even none of these configurations, in which
>  case they do not expose the corresponding selection rectangles. If any selection
> @@ -201,12 +208,19 @@ Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
>        - X
>        - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE``
>          rectangle on (pad, stream) pair 1/0.
> +    * - 0/0
> +      - ``V4L2_SEL_TGT_COMPOSE``
> +      - \-
> +      - X
> +      - Scaling. This rectangle is relative to the ``V4L2_SEL_TGT_CROP``
> +        rectangle on (pad, stream) pair 0/0.
>      * - 0/0
>        - Format
>        - X
>        - X
> -      - Image data source format. Always assign the width and height fields of
> -        the format to the same values than for the ``V4L2_SEL_TGT_CROP``
> +      - Image data source format and post-scaler crop. The width and height
> +        fields of the format, used to configure post-scaler crop on the right
> +        and bottom edges of the image, are related to the ``V4L2_SEL_TGT_COMPOSE``

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Why not squashing it with #40 that has just introduced the section you
are here modifying ?

>          rectangle on (pad, stream) pair 0/0. The media bus code reflects the
>          pixel data output of the sensor.
>      * - 0/1
> --
> 2.47.2
>
>

[PATCH v11 42/66] media: uapi: Add MIPI CCS configuration model

[Sakari Ailus]

Add a configuration model for MIPI CCS sensors and refer to the CCS driver
documentation from it. If more drivers start implementing the CCS model
the documentation should be split into two but that appears to be unlikely
at the moment.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../userspace-api/media/v4l/subdev-config-model.rst         | 6 ++++++
 include/uapi/linux/v4l2-controls.h                          | 1 +
 2 files changed, 7 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
index c1b9b74cbcef..ee59bf3c5761 100644
--- a/Documentation/userspace-api/media/v4l/subdev-config-model.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
@@ -242,3 +242,9 @@ space may obtain the size of the embedded data once the image data size on the
 source pad has been configured.
 
 Also see :ref:`media_using_camera_sensor_drivers_embedded_data`.
+
+MIPI Camera Command Set (CCS) model
+-----------------------------------
+
+The MIPI Camera Command Set configuration model is implemented by the :ref:`CCS
+driver <media-ccs-uapi>`.
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index d31cd22d1fa9..e25c9d669687 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1256,6 +1256,7 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_CONFIG_MODEL			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 6)
 
 #define V4L2_CONFIG_MODEL_COMMON_RAW_SENSOR	(1U << 0)
+#define V4L2_CONFIG_MODEL_MIPI_CCS		(1U << 1)
 
 /*  DV-class control IDs defined by V4L2 */
 #define V4L2_CID_DV_CLASS_BASE			(V4L2_CTRL_CLASS_DV | 0x900)
-- 
2.47.2

[PATCH v11 43/66] media: uapi: Add V4L2_CID_BINNING control for binning configuration

[Sakari Ailus]

Add V4L2_CID_BINNING control for configuring binning and enumerating a
camera sensor's binning capabilities. The control combines horizontal and
vertical binning into a single control as the two are generally related.

New drivers should use this control to configure binning.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 .../media/drivers/camera-sensor.rst           | 12 ++++++++
 .../media/v4l/ext-ctrls-camera.rst            | 29 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  2 ++
 include/uapi/linux/v4l2-controls.h            |  1 +
 4 files changed, 44 insertions(+)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index 39f3f91c6733..ef1f51862980 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -120,6 +120,18 @@ values programmed by the register sequences. The default values of these
 controls shall be 0 (disabled). Especially these controls shall not be inverted,
 independently of the sensor's mounting rotation.
 
+Binning
+-------
+
+Binning has traditionally been configured using :ref:`the compose selection
+rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING
+<v4l2-cid-camera-sensor-binning>` is also available for binning configuration and
+users should use it when it's available. Drivers supporting the control shall
+also support the compose rectangle, albeit the rectangle may be read-only when
+the control is present.
+
+Binning isn't affected by flipping.
+
 .. _media_using_camera_sensor_drivers_embedded_data:
 
 Embedded data
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
index cdc515c60468..18b484ff5d75 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
@@ -672,3 +672,32 @@ enum v4l2_scene_mode -
 
     As modes differ for each sensor, menu items are not standardized by this
     control and are left to the programmer.
+
+.. _v4l2-cid-camera-sensor-binning:
+
+``V4L2_CID_BINNING_FACTORS (integer menu)``
+
+    Horizontal and vertical binning factors. Binning combines several
+    horizontal, vertical or both pixel values into a single pixel. It is a way
+    to scale an image. Binning typically produces fairly good quality output.
+
+    Determines both horizontal and vertical binning factors for a camera
+    sensor. The values are encoded in the following way:
+
+.. flat-table::
+    :header-rows:  1
+    :stub-columns: 0
+
+    * - Bits
+      - Synopsis
+    * - 48--63
+      - Horizontal binning numerator.
+    * - 32--47
+      - Horizontal binning denominator.
+    * - 16--31
+      - Vertical binning numerator.
+    * - 0--15
+      - Vertical binning denominator.
+
+For instance, a value of ``0x0001000300020003`` indicates binning by 3
+(horizontally) * 3/2 (vertically).
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index a7ea380de5ee..5e1c28850e87 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1087,6 +1087,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_CAMERA_ORIENTATION:	return "Camera Orientation";
 	case V4L2_CID_CAMERA_SENSOR_ROTATION:	return "Camera Sensor Rotation";
 	case V4L2_CID_HDR_SENSOR_MODE:		return "HDR Sensor Mode";
+	case V4L2_CID_BINNING_FACTORS:		return "Binning Factors";
 
 	/* FM Radio Modulator controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1427,6 +1428,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_HDR_SENSOR_MODE:
 		*type = V4L2_CTRL_TYPE_MENU;
 		break;
+	case V4L2_CID_BINNING_FACTORS:
 	case V4L2_CID_LINK_FREQ:
 		*type = V4L2_CTRL_TYPE_INTEGER_MENU;
 		break;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index e25c9d669687..90f47f4780e5 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1101,6 +1101,7 @@ enum v4l2_auto_focus_range {
 #define V4L2_CID_CAMERA_SENSOR_ROTATION		(V4L2_CID_CAMERA_CLASS_BASE+35)
 
 #define V4L2_CID_HDR_SENSOR_MODE		(V4L2_CID_CAMERA_CLASS_BASE+36)
+#define V4L2_CID_BINNING_FACTORS		(V4L2_CID_CAMERA_CLASS_BASE+37)
 
 /* FM Modulator class control IDs */
 
-- 
2.47.2

Re: [PATCH v11 43/66] media: uapi: Add V4L2_CID_BINNING control for binning configuration

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:44PM +0300, Sakari Ailus wrote:
> Add V4L2_CID_BINNING control for configuring binning and enumerating a
> camera sensor's binning capabilities. The control combines horizontal and
> vertical binning into a single control as the two are generally related.
>
> New drivers should use this control to configure binning.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  .../media/drivers/camera-sensor.rst           | 12 ++++++++
>  .../media/v4l/ext-ctrls-camera.rst            | 29 +++++++++++++++++++
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  2 ++
>  include/uapi/linux/v4l2-controls.h            |  1 +
>  4 files changed, 44 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index 39f3f91c6733..ef1f51862980 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -120,6 +120,18 @@ values programmed by the register sequences. The default values of these
>  controls shall be 0 (disabled). Especially these controls shall not be inverted,
>  independently of the sensor's mounting rotation.
>
> +Binning
> +-------
> +
> +Binning has traditionally been configured using :ref:`the compose selection
> +rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING

I would add "... >` on the internal sink pad that represents the image
data source.

> +<v4l2-cid-camera-sensor-binning>` is also available for binning configuration and

missing 'control' after the :ref:`...` so that it reads

The V4L2_CID_BINNING -control- is also available

when rendered


> +users should use it when it's available. Drivers supporting the control shall

I would not make it just about users, but also for drivers.

"The V4L2_CID_BINNING control has been introduced as the preferred way
for drivers and userspace applications for configuring binning."

> +also support the compose rectangle, albeit the rectangle may be read-only when
> +the control is present.
> +
> +Binning isn't affected by flipping.
> +
>  .. _media_using_camera_sensor_drivers_embedded_data:
>
>  Embedded data
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> index cdc515c60468..18b484ff5d75 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> @@ -672,3 +672,32 @@ enum v4l2_scene_mode -
>
>      As modes differ for each sensor, menu items are not standardized by this
>      control and are left to the programmer.
> +
> +.. _v4l2-cid-camera-sensor-binning:
> +
> +``V4L2_CID_BINNING_FACTORS (integer menu)``
> +
> +    Horizontal and vertical binning factors. Binning combines several
> +    horizontal, vertical or both pixel values into a single pixel. It is a way
> +    to scale an image. Binning typically produces fairly good quality output.
> +
> +    Determines both horizontal and vertical binning factors for a camera
> +    sensor. The values are encoded in the following way:
> +
> +.. flat-table::
> +    :header-rows:  1
> +    :stub-columns: 0
> +
> +    * - Bits
> +      - Synopsis
> +    * - 48--63
> +      - Horizontal binning numerator.
> +    * - 32--47
> +      - Horizontal binning denominator.
> +    * - 16--31
> +      - Vertical binning numerator.
> +    * - 0--15
> +      - Vertical binning denominator.
> +
> +For instance, a value of ``0x0001000300020003`` indicates binning by 3
> +(horizontally) * 3/2 (vertically).

indicates an horizontal binning factor of 3 and 3/2 vertical one.

> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index a7ea380de5ee..5e1c28850e87 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1087,6 +1087,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_CAMERA_ORIENTATION:	return "Camera Orientation";
>  	case V4L2_CID_CAMERA_SENSOR_ROTATION:	return "Camera Sensor Rotation";
>  	case V4L2_CID_HDR_SENSOR_MODE:		return "HDR Sensor Mode";
> +	case V4L2_CID_BINNING_FACTORS:		return "Binning Factors";
>
>  	/* FM Radio Modulator controls */
>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> @@ -1427,6 +1428,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_HDR_SENSOR_MODE:
>  		*type = V4L2_CTRL_TYPE_MENU;
>  		break;
> +	case V4L2_CID_BINNING_FACTORS:
>  	case V4L2_CID_LINK_FREQ:
>  		*type = V4L2_CTRL_TYPE_INTEGER_MENU;
>  		break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index e25c9d669687..90f47f4780e5 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1101,6 +1101,7 @@ enum v4l2_auto_focus_range {
>  #define V4L2_CID_CAMERA_SENSOR_ROTATION		(V4L2_CID_CAMERA_CLASS_BASE+35)
>
>  #define V4L2_CID_HDR_SENSOR_MODE		(V4L2_CID_CAMERA_CLASS_BASE+36)
> +#define V4L2_CID_BINNING_FACTORS		(V4L2_CID_CAMERA_CLASS_BASE+37)

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j

>
>  /* FM Modulator class control IDs */
>
> --
> 2.47.2
>
>

[PATCH v11 44/66] media: uapi: Add controls for sub-sampling configuration

[Sakari Ailus]

Sub-sampling is a way to decrease the data rates after the pixel array by
systematically discarding some samples, either vertically or horizontally
or both. Add two controls for the purpose and document them. The
sub-sampling configuration is taken into account in the compose rectangle.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 .../media/drivers/camera-sensor.rst              | 16 +++++++++++++---
 .../userspace-api/media/v4l/ext-ctrls-camera.rst | 11 +++++++++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c        |  2 ++
 include/uapi/linux/v4l2-controls.h               |  2 ++
 4 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index ef1f51862980..b0ad0d778396 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -120,8 +120,8 @@ values programmed by the register sequences. The default values of these
 controls shall be 0 (disabled). Especially these controls shall not be inverted,
 independently of the sensor's mounting rotation.
 
-Binning
--------
+Binning and sub-sampling
+------------------------
 
 Binning has traditionally been configured using :ref:`the compose selection
 rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING
@@ -130,7 +130,17 @@ users should use it when it's available. Drivers supporting the control shall
 also support the compose rectangle, albeit the rectangle may be read-only when
 the control is present.
 
-Binning isn't affected by flipping.
+Sub-sampling is often supported as part of a camera sensor's binning
+functionality and performed after the binning operation. Sub-sampling typically
+produces quality-wise worse results than binning. Sub-sampling factors are
+independent horizontally and vertically and they are controlled using two
+controls, :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and
+V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>`. In
+sub-sampling, the image size before sub-sampling is horizontally and vertically
+divided by the respective sub-sampling factors. Drivers supporting the control shall
+also reflect the sub-sampling configuration in the compose rectangle.
+
+Binning and sub-sampling aren't affected by flipping.
 
 .. _media_using_camera_sensor_drivers_embedded_data:
 
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
index 18b484ff5d75..577b73045bee 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
@@ -701,3 +701,14 @@ enum v4l2_scene_mode -
 
 For instance, a value of ``0x0001000300020003`` indicates binning by 3
 (horizontally) * 3/2 (vertically).
+
+.. _v4l2-cid-camera-sensor-subsampling:
+
+``V4L2_CID_SUBSAMPLING_HORIZONTAL`` and ``V4L2_CID_SUBSAMPLING_VERTICAL``
+(integer)
+
+    Horizontal and vertical subsampling factors.
+
+    Sub-sampling is used to downscale an image, horizontally and vertically, by
+    discarding a part of the image data. Typically sub-sampling produces lower
+    quality images than binning.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 5e1c28850e87..9f8816bfffbe 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1088,6 +1088,8 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_CAMERA_SENSOR_ROTATION:	return "Camera Sensor Rotation";
 	case V4L2_CID_HDR_SENSOR_MODE:		return "HDR Sensor Mode";
 	case V4L2_CID_BINNING_FACTORS:		return "Binning Factors";
+	case V4L2_CID_SUBSAMPLING_HORIZONTAL:	return "Sub-Sampling Factor, Horizontal";
+	case V4L2_CID_SUBSAMPLING_VERTICAL:	return "Sub-Sampling Factor, Vertical";
 
 	/* FM Radio Modulator controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 90f47f4780e5..51d43d4a3151 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1102,6 +1102,8 @@ enum v4l2_auto_focus_range {
 
 #define V4L2_CID_HDR_SENSOR_MODE		(V4L2_CID_CAMERA_CLASS_BASE+36)
 #define V4L2_CID_BINNING_FACTORS		(V4L2_CID_CAMERA_CLASS_BASE+37)
+#define V4L2_CID_SUBSAMPLING_HORIZONTAL		(V4L2_CID_CAMERA_CLASS_BASE+38)
+#define V4L2_CID_SUBSAMPLING_VERTICAL		(V4L2_CID_CAMERA_CLASS_BASE+39)
 
 /* FM Modulator class control IDs */
 
-- 
2.47.2

Re: [PATCH v11 44/66] media: uapi: Add controls for sub-sampling configuration

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:45PM +0300, Sakari Ailus wrote:
> Sub-sampling is a way to decrease the data rates after the pixel array by
> systematically discarding some samples, either vertically or horizontally
> or both. Add two controls for the purpose and document them. The
> sub-sampling configuration is taken into account in the compose rectangle.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  .../media/drivers/camera-sensor.rst              | 16 +++++++++++++---
>  .../userspace-api/media/v4l/ext-ctrls-camera.rst | 11 +++++++++++
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c        |  2 ++
>  include/uapi/linux/v4l2-controls.h               |  2 ++
>  4 files changed, 28 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index ef1f51862980..b0ad0d778396 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -120,8 +120,8 @@ values programmed by the register sequences. The default values of these
>  controls shall be 0 (disabled). Especially these controls shall not be inverted,
>  independently of the sensor's mounting rotation.
>
> -Binning
> --------
> +Binning and sub-sampling
> +------------------------
>
>  Binning has traditionally been configured using :ref:`the compose selection
>  rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING
> @@ -130,7 +130,17 @@ users should use it when it's available. Drivers supporting the control shall
>  also support the compose rectangle, albeit the rectangle may be read-only when
>  the control is present.
>
> -Binning isn't affected by flipping.
> +Sub-sampling is often supported as part of a camera sensor's binning
> +functionality and performed after the binning operation. Sub-sampling typically
> +produces quality-wise worse results than binning. Sub-sampling factors are
> +independent horizontally and vertically and they are controlled using two
> +controls, :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and
> +V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>`. In
> +sub-sampling, the image size before sub-sampling is horizontally and vertically
> +divided by the respective sub-sampling factors. Drivers supporting the control shall
> +also reflect the sub-sampling configuration in the compose rectangle.
> +
> +Binning and sub-sampling aren't affected by flipping.
>
>  .. _media_using_camera_sensor_drivers_embedded_data:
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> index 18b484ff5d75..577b73045bee 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> @@ -701,3 +701,14 @@ enum v4l2_scene_mode -
>
>  For instance, a value of ``0x0001000300020003`` indicates binning by 3
>  (horizontally) * 3/2 (vertically).
> +
> +.. _v4l2-cid-camera-sensor-subsampling:
> +
> +``V4L2_CID_SUBSAMPLING_HORIZONTAL`` and ``V4L2_CID_SUBSAMPLING_VERTICAL``
> +(integer)

Shouldn't these be menus as well, with the driver reporting the
supported sub-sampling factors ?

> +
> +    Horizontal and vertical subsampling factors.
> +
> +    Sub-sampling is used to downscale an image, horizontally and vertically, by
> +    discarding a part of the image data. Typically sub-sampling produces lower
> +    quality images than binning.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 5e1c28850e87..9f8816bfffbe 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1088,6 +1088,8 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_CAMERA_SENSOR_ROTATION:	return "Camera Sensor Rotation";
>  	case V4L2_CID_HDR_SENSOR_MODE:		return "HDR Sensor Mode";
>  	case V4L2_CID_BINNING_FACTORS:		return "Binning Factors";
> +	case V4L2_CID_SUBSAMPLING_HORIZONTAL:	return "Sub-Sampling Factor, Horizontal";
> +	case V4L2_CID_SUBSAMPLING_VERTICAL:	return "Sub-Sampling Factor, Vertical";
>
>  	/* FM Radio Modulator controls */
>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 90f47f4780e5..51d43d4a3151 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1102,6 +1102,8 @@ enum v4l2_auto_focus_range {
>
>  #define V4L2_CID_HDR_SENSOR_MODE		(V4L2_CID_CAMERA_CLASS_BASE+36)
>  #define V4L2_CID_BINNING_FACTORS		(V4L2_CID_CAMERA_CLASS_BASE+37)
> +#define V4L2_CID_SUBSAMPLING_HORIZONTAL		(V4L2_CID_CAMERA_CLASS_BASE+38)
> +#define V4L2_CID_SUBSAMPLING_VERTICAL		(V4L2_CID_CAMERA_CLASS_BASE+39)
>
>  /* FM Modulator class control IDs */
>
> --
> 2.47.2
>
>

[PATCH v11 45/66] media: Documentation: Add binning and sub-sampling controls

[Sakari Ailus]

Document the binning and scaling controls (V4L2_CID_BINNING and
V4L2_CID_SUBSAMPLING_{HORIZONTAL,VERTICAL}) in the common raw sensor
model.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 .../media/v4l/subdev-config-model.rst         | 20 ++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
index ee59bf3c5761..c1c9c9b8bc14 100644
--- a/Documentation/userspace-api/media/v4l/subdev-config-model.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
@@ -134,8 +134,12 @@ separately horizontally and vertically.
 
 Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE``
 rectangle, relative to the analogue crop rectangle, on (pad, stream) pair
-1/0. The driver implementation determines how to configure binning and
-sub-sampling to achieve the desired size.
+1/0. It depends on the driver which of these operations are being used to
+achieve the resulting size. Binning and sub-sampling are also directly
+configured using :ref:`V4L2_CID_BINNING_FACTORS
+<v4l2-cid-camera-sensor-binning>` and :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and
+V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>` controls on
+drivers that support them.
 
 The digital crop operation takes place after binning and sub-sampling. It is
 configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
@@ -194,9 +198,15 @@ Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
       - \-
       - X
       - Binning and sub-sampling. This rectangle is relative to the
-        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The
-        combination of binning and sub-sampling is configured using this
-        selection target.
+        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). Binning is
+        configured using the :ref:`V4L2_CID_BINNING_FACTORS
+        <v4l2-cid-camera-sensor-binning>` control and sub-sampling is configured
+        using the :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and
+        V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>`
+        controls on drivers that support these controls. To configure binning
+        and sub-sampling on drivers that do not support these controls, the
+        selection rectangle may be changed directly to configure the combined
+        effect on the image size.
     * - 2/0
       - Format
       - X
-- 
2.47.2

Re: [PATCH v11 45/66] media: Documentation: Add binning and sub-sampling controls

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:46PM +0300, Sakari Ailus wrote:
> Document the binning and scaling controls (V4L2_CID_BINNING and
> V4L2_CID_SUBSAMPLING_{HORIZONTAL,VERTICAL}) in the common raw sensor
> model.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j

> ---
>  .../media/v4l/subdev-config-model.rst         | 20 ++++++++++++++-----
>  1 file changed, 15 insertions(+), 5 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> index ee59bf3c5761..c1c9c9b8bc14 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> @@ -134,8 +134,12 @@ separately horizontally and vertically.
>
>  Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE``
>  rectangle, relative to the analogue crop rectangle, on (pad, stream) pair
> -1/0. The driver implementation determines how to configure binning and
> -sub-sampling to achieve the desired size.
> +1/0. It depends on the driver which of these operations are being used to
> +achieve the resulting size. Binning and sub-sampling are also directly
> +configured using :ref:`V4L2_CID_BINNING_FACTORS
> +<v4l2-cid-camera-sensor-binning>` and :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and
> +V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>` controls on
> +drivers that support them.
>
>  The digital crop operation takes place after binning and sub-sampling. It is
>  configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
> @@ -194,9 +198,15 @@ Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
>        - \-
>        - X
>        - Binning and sub-sampling. This rectangle is relative to the
> -        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The
> -        combination of binning and sub-sampling is configured using this
> -        selection target.
> +        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). Binning is
> +        configured using the :ref:`V4L2_CID_BINNING_FACTORS
> +        <v4l2-cid-camera-sensor-binning>` control and sub-sampling is configured
> +        using the :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and
> +        V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>`
> +        controls on drivers that support these controls. To configure binning
> +        and sub-sampling on drivers that do not support these controls, the
> +        selection rectangle may be changed directly to configure the combined
> +        effect on the image size.
>      * - 2/0
>        - Format
>        - X
> --
> 2.47.2
>
>

[PATCH v11 46/66] media: uapi: Add metadata layout for ov2740 embedded data

[Sakari Ailus]

Add a metadata layout for ov2740 camera sensor embedded data and document
it.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Julien Massot <julien.massot@collabora.com>
---
 .../media/v4l/ext-ctrls-image-source.rst      |  3 +
 .../media/v4l/metadata-layouts.rst            | 72 +++++++++++++++++++
 include/uapi/linux/v4l2-controls.h            |  1 +
 3 files changed, 76 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index 371c155a88f7..ced358d7c250 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -165,3 +165,6 @@ Image Source Control IDs
     * - V4L2_METADATA_LAYOUT_CCS
       - :ref:`CCS embedded data <media-metadata-layout-ccs>`
       - 1
+    * - V4L2_METADATA_LAYOUT_OV2740
+      - :ref:`OV2740 embedded data <media-metadata-layout-ov2740>`
+      - 2
diff --git a/Documentation/userspace-api/media/v4l/metadata-layouts.rst b/Documentation/userspace-api/media/v4l/metadata-layouts.rst
index a4f7e93e6a29..e667148cc3cb 100644
--- a/Documentation/userspace-api/media/v4l/metadata-layouts.rst
+++ b/Documentation/userspace-api/media/v4l/metadata-layouts.rst
@@ -39,3 +39,75 @@ format" (code ``0xa``) but their use may be extended further in the future, to
 cover other CCS embedded data format codes.
 
 Also see :ref:`CCS driver documentation <media-ccs-routes>`.
+
+.. _media-metadata-layout-ov2740:
+
+Omnivision OV2740 Embedded Data Layout (``V4L2_METADATA_LAYOUT_OV2740``)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The Omnivision OV2740 camera sensor produces the following embedded data layout,
+indicated by ``V4L2_METADATA_LAYOUT_CCS`` metadata layout. The format conforms
+to :ref:`CCS embedded data layout <media-metadata-layout-ccs>` up to level 1.
+
+.. flat-table:: Omnivision OV2740 Embedded Data Layout. Octets at indices marked
+                reserved or unused have been omitted from the table. The values
+                are big endian byte order.
+    :header-rows: 1
+
+    * - Offset
+      - Size in bits (active bits if not the same as size)
+      - Content description
+    * - 4
+      - 16 (10--0)
+      - Analogue gain
+    * - 6
+      - 16
+      - Coarse integration time
+    * - 10
+      - 8
+      - Dpc correction threshold
+    * - 15
+      - 16
+      - Output image width
+    * - 17
+      - 16
+      - Output image height
+    * - 23
+      - 8
+      - MIPI header revision number (2)
+    * - 31
+      - 8
+      - Vertical (bit 1) and horizontal flip (bit 0)
+    * - 32
+      - 8
+      - Frame duration A
+    * - 33
+      - 8
+      - Frame duration B
+    * - 34
+      - 8
+      - Context count (2)
+    * - 35
+      - 8
+      - Context select
+    * - 54
+      - 8
+      - Data pedestal bits 9--2
+    * - 63
+      - 8
+      - Frame average bits 9--2
+    * - 64
+      - 16
+      - Digital gain red
+    * - 66
+      - 16
+      - Digital gain greenr
+    * - 68
+      - 16
+      - Digital gain blue
+    * - 70
+      - 16
+      - Digital gain greenb
+    * - 89
+      - 8
+      - Frame counter (starts at 1, wraps to 0 after 255)
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 51d43d4a3151..bac2fdcbd202 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1244,6 +1244,7 @@ enum v4l2_jpeg_chroma_subsampling {
 
 #define V4L2_CID_METADATA_LAYOUT		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 12)
 #define V4L2_METADATA_LAYOUT_CCS		1U
+#define V4L2_METADATA_LAYOUT_OV2740		2U
 
 
 /* Image processing controls */
-- 
2.47.2

Re: [PATCH v11 46/66] media: uapi: Add metadata layout for ov2740 embedded data

[Laurent Pinchart]

On Mon, Aug 25, 2025 at 12:50:47PM +0300, Sakari Ailus wrote:
> Add a metadata layout for ov2740 camera sensor embedded data and document
> it.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Julien Massot <julien.massot@collabora.com>
> ---
>  .../media/v4l/ext-ctrls-image-source.rst      |  3 +
>  .../media/v4l/metadata-layouts.rst            | 72 +++++++++++++++++++
>  include/uapi/linux/v4l2-controls.h            |  1 +
>  3 files changed, 76 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index 371c155a88f7..ced358d7c250 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -165,3 +165,6 @@ Image Source Control IDs
>      * - V4L2_METADATA_LAYOUT_CCS
>        - :ref:`CCS embedded data <media-metadata-layout-ccs>`
>        - 1
> +    * - V4L2_METADATA_LAYOUT_OV2740
> +      - :ref:`OV2740 embedded data <media-metadata-layout-ov2740>`
> +      - 2
> diff --git a/Documentation/userspace-api/media/v4l/metadata-layouts.rst b/Documentation/userspace-api/media/v4l/metadata-layouts.rst
> index a4f7e93e6a29..e667148cc3cb 100644
> --- a/Documentation/userspace-api/media/v4l/metadata-layouts.rst
> +++ b/Documentation/userspace-api/media/v4l/metadata-layouts.rst
> @@ -39,3 +39,75 @@ format" (code ``0xa``) but their use may be extended further in the future, to
>  cover other CCS embedded data format codes.
>  
>  Also see :ref:`CCS driver documentation <media-ccs-routes>`.
> +
> +.. _media-metadata-layout-ov2740:
> +
> +Omnivision OV2740 Embedded Data Layout (``V4L2_METADATA_LAYOUT_OV2740``)
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +The Omnivision OV2740 camera sensor produces the following embedded data layout,
> +indicated by ``V4L2_METADATA_LAYOUT_CCS`` metadata layout. The format conforms

s/V4L2_METADATA_LAYOUT_CCS/V4L2_METADATA_LAYOUT_OV2740/

> +to :ref:`CCS embedded data layout <media-metadata-layout-ccs>` up to level 1.
> +
> +.. flat-table:: Omnivision OV2740 Embedded Data Layout. Octets at indices marked
> +                reserved or unused have been omitted from the table. The values
> +                are big endian byte order.
> +    :header-rows: 1
> +
> +    * - Offset
> +      - Size in bits (active bits if not the same as size)
> +      - Content description
> +    * - 4
> +      - 16 (10--0)
> +      - Analogue gain
> +    * - 6
> +      - 16
> +      - Coarse integration time
> +    * - 10
> +      - 8
> +      - Dpc correction threshold

s/Dpc/DPC/

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +    * - 15
> +      - 16
> +      - Output image width
> +    * - 17
> +      - 16
> +      - Output image height
> +    * - 23
> +      - 8
> +      - MIPI header revision number (2)
> +    * - 31
> +      - 8
> +      - Vertical (bit 1) and horizontal flip (bit 0)
> +    * - 32
> +      - 8
> +      - Frame duration A
> +    * - 33
> +      - 8
> +      - Frame duration B
> +    * - 34
> +      - 8
> +      - Context count (2)
> +    * - 35
> +      - 8
> +      - Context select
> +    * - 54
> +      - 8
> +      - Data pedestal bits 9--2
> +    * - 63
> +      - 8
> +      - Frame average bits 9--2
> +    * - 64
> +      - 16
> +      - Digital gain red
> +    * - 66
> +      - 16
> +      - Digital gain greenr
> +    * - 68
> +      - 16
> +      - Digital gain blue
> +    * - 70
> +      - 16
> +      - Digital gain greenb
> +    * - 89
> +      - 8
> +      - Frame counter (starts at 1, wraps to 0 after 255)
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 51d43d4a3151..bac2fdcbd202 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1244,6 +1244,7 @@ enum v4l2_jpeg_chroma_subsampling {
>  
>  #define V4L2_CID_METADATA_LAYOUT		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 12)
>  #define V4L2_METADATA_LAYOUT_CCS		1U
> +#define V4L2_METADATA_LAYOUT_OV2740		2U
>  
>  
>  /* Image processing controls */

-- 
Regards,

Laurent Pinchart

[PATCH v11 47/66] media: ov2740: Add support for embedded data

[Sakari Ailus]

Add support for embedded data. This introduces two internal pads for pixel
and embedded data streams. As the driver supports a single mode only,
there's no need for backward compatibility in mode selection.

The embedded data is configured to be placed before the image data whereas
after the image data is the default.

This patch includes switching to {enable,disable}_streams as well.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ov2740.c | 245 +++++++++++++++++++++++++++++--------
 1 file changed, 191 insertions(+), 54 deletions(-)

diff --git a/drivers/media/i2c/ov2740.c b/drivers/media/i2c/ov2740.c
index 4e959534e6e7..684e6999dbd9 100644
--- a/drivers/media/i2c/ov2740.c
+++ b/drivers/media/i2c/ov2740.c
@@ -12,6 +12,7 @@
 #include <linux/nvmem-provider.h>
 #include <linux/regmap.h>
 #include <linux/regulator/consumer.h>
+#include <media/mipi-csi2.h>
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-device.h>
 #include <media/v4l2-fwnode.h>
@@ -72,6 +73,15 @@
 #define OV2740_REG_ISP_CTRL00		0x5000
 /* ISP CTRL01 */
 #define OV2740_REG_ISP_CTRL01		0x5001
+
+/* Embedded data line location control */
+#define OV2740_REG_EMBEDDED_FLAG	0x5a08
+#define OV2740_EMBEDDED_FLAG_FOOTER	BIT(2) /* otherwise it's in header */
+#define OV2740_EMBEDDED_FLAG_MYSTERY	BIT(1)
+#define OV2740_REG_EMBEDDED_DT		0x3036
+#define OV2740_REG_EMBEDDED_CTRL	0x4307
+#define OV2740_EMBEDDED_CTRL_MYSTERY	0x30
+#define OV2740_EMBEDDED_CTRL_ENABLE	BIT(0)
 /* Customer Addresses: 0x7010 - 0x710F */
 #define CUSTOMER_USE_OTP_SIZE		0x100
 /* OTP registers from sensor */
@@ -83,6 +93,21 @@ static const char * const ov2740_supply_name[] = {
 	"DVDD",
 };
 
+enum {
+	OV2740_PAD_SOURCE,
+	OV2740_PAD_PIXEL,
+	OV2740_PAD_META,
+	OV2740_NUM_PADS,
+};
+
+enum {
+	OV2740_STREAM_PIXEL,
+	OV2740_STREAM_META,
+};
+
+#define OV2740_META_WIDTH		100U
+#define OV2740_META_HEIGHT		1U
+
 struct nvm_data {
 	struct nvmem_device *nvmem;
 	struct regmap *regmap;
@@ -156,6 +181,7 @@ static const struct ov2740_reg mode_1932x1092_regs_360mhz[] = {
 	{0x3000, 0x00},
 	{0x3018, 0x32},
 	{0x3031, 0x0a},
+	{OV2740_REG_EMBEDDED_DT, MIPI_CSI2_DT_EMBEDDED_8B},
 	{0x3080, 0x08},
 	{0x3083, 0xB4},
 	{0x3103, 0x00},
@@ -260,7 +286,8 @@ static const struct ov2740_reg mode_1932x1092_regs_360mhz[] = {
 	{0x4017, 0x10},
 	{0x4044, 0x02},
 	{0x4304, 0x08},
-	{0x4307, 0x30},
+	{OV2740_REG_EMBEDDED_CTRL,
+	 OV2740_EMBEDDED_CTRL_MYSTERY | OV2740_EMBEDDED_CTRL_ENABLE},
 	{0x4320, 0x80},
 	{0x4322, 0x00},
 	{0x4323, 0x00},
@@ -309,6 +336,7 @@ static const struct ov2740_reg mode_1932x1092_regs_180mhz[] = {
 	{0x3000, 0x00},
 	{0x3018, 0x32},	/* 0x32 for 2 lanes, 0x12 for 1 lane */
 	{0x3031, 0x0a},
+	{OV2740_REG_EMBEDDED_DT, MIPI_CSI2_DT_EMBEDDED_8B},
 	{0x3080, 0x08},
 	{0x3083, 0xB4},
 	{0x3103, 0x00},
@@ -413,7 +441,8 @@ static const struct ov2740_reg mode_1932x1092_regs_180mhz[] = {
 	{0x4017, 0x10},
 	{0x4044, 0x02},
 	{0x4304, 0x08},
-	{0x4307, 0x30},
+	{OV2740_REG_EMBEDDED_CTRL,
+	 OV2740_EMBEDDED_CTRL_MYSTERY | OV2740_EMBEDDED_CTRL_ENABLE},
 	{0x4320, 0x80},
 	{0x4322, 0x00},
 	{0x4323, 0x00},
@@ -520,7 +549,7 @@ static const struct ov2740_mode supported_modes_180mhz[] = {
 
 struct ov2740 {
 	struct v4l2_subdev sd;
-	struct media_pad pad;
+	struct media_pad pads[OV2740_NUM_PADS];
 	struct v4l2_ctrl_handler ctrl_handler;
 
 	/* V4L2 Controls */
@@ -548,6 +577,9 @@ struct ov2740 {
 
 	/* True if the device has been identified */
 	bool identified;
+
+	/* Track streaming state */
+	u8 streaming;
 };
 
 static inline struct ov2740 *to_ov2740(struct v4l2_subdev *subdev)
@@ -938,16 +970,28 @@ static int ov2740_load_otp_data(struct nvm_data *nvm)
 	return ret;
 }
 
-static int ov2740_start_streaming(struct ov2740 *ov2740)
+static int ov2740_enable_streams(struct v4l2_subdev *sd,
+				 struct v4l2_subdev_state *state, u32 pad,
+				 u64 streams_mask)
 {
-	struct i2c_client *client = v4l2_get_subdevdata(&ov2740->sd);
+	struct i2c_client *client = v4l2_get_subdevdata(sd);
+	struct ov2740 *ov2740 = to_ov2740(sd);
 	const struct ov2740_reg_list *reg_list;
 	int link_freq_index;
 	int ret;
 
+	if (ov2740->streaming) {
+		ov2740->streaming |= streams_mask;
+		return 0;
+	}
+
+	ret = pm_runtime_resume_and_get(&client->dev);
+	if (ret < 0)
+		return ret;
+
 	ret = ov2740_identify_module(ov2740);
 	if (ret)
-		return ret;
+		goto out_pm_put;
 
 	if (ov2740->nvm)
 		ov2740_load_otp_data(ov2740->nvm);
@@ -956,7 +1000,7 @@ static int ov2740_start_streaming(struct ov2740 *ov2740)
 	ret = ov2740_write_reg(ov2740, 0x0103, 1, 0x01);
 	if (ret) {
 		dev_err(&client->dev, "failed to reset\n");
-		return ret;
+		goto out_pm_put;
 	}
 
 	usleep_range(10000, 15000);
@@ -966,85 +1010,104 @@ static int ov2740_start_streaming(struct ov2740 *ov2740)
 	ret = ov2740_write_reg_list(ov2740, reg_list);
 	if (ret) {
 		dev_err(&client->dev, "failed to set plls\n");
-		return ret;
+		goto out_pm_put;
 	}
 
 	reg_list = &ov2740->cur_mode->reg_list;
 	ret = ov2740_write_reg_list(ov2740, reg_list);
 	if (ret) {
 		dev_err(&client->dev, "failed to set mode\n");
-		return ret;
+		goto out_pm_put;
 	}
 
 	ret = __v4l2_ctrl_handler_setup(ov2740->sd.ctrl_handler);
 	if (ret)
-		return ret;
+		goto out_pm_put;
+
+	ret = ov2740_write_reg(ov2740, OV2740_REG_EMBEDDED_FLAG, 1,
+			       OV2740_EMBEDDED_FLAG_MYSTERY);
+	if (ret)
+		goto out_pm_put;
 
 	ret = ov2740_write_reg(ov2740, OV2740_REG_MODE_SELECT, 1,
 			       OV2740_MODE_STREAMING);
-	if (ret)
+	if (ret) {
 		dev_err(&client->dev, "failed to start streaming\n");
+		goto out_pm_put;
+	}
 
-	return ret;
-}
+	ov2740->streaming |= streams_mask;
 
-static void ov2740_stop_streaming(struct ov2740 *ov2740)
-{
-	struct i2c_client *client = v4l2_get_subdevdata(&ov2740->sd);
+	return 0;
 
-	if (ov2740_write_reg(ov2740, OV2740_REG_MODE_SELECT, 1,
-			     OV2740_MODE_STANDBY))
-		dev_err(&client->dev, "failed to stop streaming\n");
+out_pm_put:
+	pm_runtime_put(&client->dev);
+
+	return ret;
 }
 
-static int ov2740_set_stream(struct v4l2_subdev *sd, int enable)
+static int ov2740_disable_streams(struct v4l2_subdev *sd,
+				  struct v4l2_subdev_state *state, u32 pad,
+				  u64 streams_mask)
 {
-	struct ov2740 *ov2740 = to_ov2740(sd);
 	struct i2c_client *client = v4l2_get_subdevdata(sd);
-	struct v4l2_subdev_state *sd_state;
-	int ret = 0;
-
-	sd_state = v4l2_subdev_lock_and_get_active_state(&ov2740->sd);
+	struct ov2740 *ov2740 = to_ov2740(sd);
+	int ret;
 
-	if (enable) {
-		ret = pm_runtime_resume_and_get(&client->dev);
-		if (ret < 0)
-			goto out_unlock;
+	ov2740->streaming &= ~streams_mask;
+	if (ov2740->streaming)
+		return 0;
 
-		ret = ov2740_start_streaming(ov2740);
-		if (ret) {
-			enable = 0;
-			ov2740_stop_streaming(ov2740);
-			pm_runtime_put(&client->dev);
-		}
-	} else {
-		ov2740_stop_streaming(ov2740);
-		pm_runtime_put(&client->dev);
-	}
+	ret = ov2740_write_reg(ov2740, OV2740_REG_MODE_SELECT, 1,
+			       OV2740_MODE_STANDBY);
 
-out_unlock:
-	v4l2_subdev_unlock_state(sd_state);
+	pm_runtime_put(&client->dev);
 
 	return ret;
 }
 
-static int ov2740_set_format(struct v4l2_subdev *sd,
-			     struct v4l2_subdev_state *sd_state,
-			     struct v4l2_subdev_format *fmt)
+static int __ov2740_set_format(struct v4l2_subdev *sd,
+			       struct v4l2_subdev_state *sd_state,
+			       struct v4l2_mbus_framefmt *format,
+			       enum v4l2_subdev_format_whence which,
+			       unsigned int pad, unsigned int stream)
 {
+	struct v4l2_mbus_framefmt *src_pix_fmt, *src_meta_fmt, *pix_fmt,
+		*meta_fmt;
 	struct ov2740 *ov2740 = to_ov2740(sd);
 	const struct ov2740_mode *mode;
 	s32 vblank_def, h_blank;
 
+	/*
+	 * Allow setting format on internal pixel pad as well as the source
+	 * pad's pixel stream (for compatibility).
+	 */
+	if ((pad == OV2740_PAD_SOURCE && stream == OV2740_STREAM_META) ||
+	    pad == OV2740_PAD_META) {
+		*format = *v4l2_subdev_state_get_format(sd_state, pad, stream);
+		return 0;
+	}
+
+	pix_fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_PIXEL, 0);
+	meta_fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_META, 0);
+	src_pix_fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_SOURCE,
+						   OV2740_STREAM_PIXEL);
+	src_meta_fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_SOURCE,
+						    OV2740_STREAM_META);
+
 	mode = v4l2_find_nearest_size(ov2740->supported_modes,
 				      ov2740->supported_modes_count,
 				      width, height,
-				      fmt->format.width, fmt->format.height);
+				      format->width, format->height);
+	ov2740_update_pad_format(mode, pix_fmt);
+	*format = *src_pix_fmt = *pix_fmt;
 
-	ov2740_update_pad_format(mode, &fmt->format);
-	*v4l2_subdev_state_get_format(sd_state, fmt->pad) = fmt->format;
+	meta_fmt->code = MEDIA_BUS_FMT_META_10;
+	meta_fmt->width = OV2740_META_WIDTH;
+	meta_fmt->height = OV2740_META_HEIGHT;
+	*src_meta_fmt = *meta_fmt;
 
-	if (fmt->which == V4L2_SUBDEV_FORMAT_TRY)
+	if (which == V4L2_SUBDEV_FORMAT_TRY)
 		return 0;
 
 	ov2740->cur_mode = mode;
@@ -1064,6 +1127,14 @@ static int ov2740_set_format(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int ov2740_set_format(struct v4l2_subdev *sd,
+			     struct v4l2_subdev_state *sd_state,
+			     struct v4l2_subdev_format *fmt)
+{
+	return __ov2740_set_format(sd, sd_state, &fmt->format, fmt->which,
+				   fmt->pad, fmt->stream);
+}
+
 static int ov2740_enum_mbus_code(struct v4l2_subdev *sd,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_mbus_code_enum *code)
@@ -1100,15 +1171,73 @@ static int ov2740_enum_frame_size(struct v4l2_subdev *sd,
 static int ov2740_init_state(struct v4l2_subdev *sd,
 			     struct v4l2_subdev_state *sd_state)
 {
+	struct v4l2_subdev_route routes[] = {
+		{
+			.sink_pad = OV2740_PAD_PIXEL,
+			.source_pad = OV2740_PAD_SOURCE,
+			.source_stream = OV2740_STREAM_PIXEL,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+		}, {
+			.sink_pad = OV2740_PAD_META,
+			.source_pad = OV2740_PAD_SOURCE,
+			.source_stream = OV2740_STREAM_META,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+		},
+	};
+	struct v4l2_subdev_krouting routing = {
+		.routes = routes,
+		.num_routes = ARRAY_SIZE(routes),
+	};
+	struct v4l2_subdev_state *active_state;
+	struct v4l2_mbus_framefmt format = { 0 };
 	struct ov2740 *ov2740 = to_ov2740(sd);
+	int ret;
+
+	ret = v4l2_subdev_set_routing(sd, sd_state, &routing);
+	if (ret)
+		return ret;
+
+	active_state = v4l2_subdev_get_locked_active_state(sd);
+
+	ov2740_update_pad_format(&ov2740->supported_modes[0], &format);
+
+	return __ov2740_set_format(sd, sd_state, &format,
+				   active_state == sd_state ?
+				   V4L2_SUBDEV_FORMAT_ACTIVE :
+				   V4L2_SUBDEV_FORMAT_TRY, OV2740_PAD_PIXEL, 0);
+}
+
+static int ov2740_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
+				 struct v4l2_mbus_frame_desc *desc)
+{
+	struct v4l2_mbus_frame_desc_entry *entry = desc->entry;
+	struct v4l2_subdev_state *sd_state;
+	struct v4l2_mbus_framefmt *fmt;
+
+	desc->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
+
+	sd_state = v4l2_subdev_lock_and_get_active_state(sd);
+	fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_SOURCE,
+					   OV2740_STREAM_PIXEL);
+	entry->pixelcode = fmt->code;
+	v4l2_subdev_unlock_state(sd_state);
+
+	entry->stream = OV2740_STREAM_PIXEL;
+	entry->bus.csi2.dt = MIPI_CSI2_DT_RAW10;
+	entry++;
+	desc->num_entries++;
+
+	entry->pixelcode = MEDIA_BUS_FMT_META_10;
+	entry->stream = OV2740_STREAM_META;
+	entry->bus.csi2.dt = MIPI_CSI2_DT_EMBEDDED_8B;
+	entry++;
+	desc->num_entries++;
 
-	ov2740_update_pad_format(&ov2740->supported_modes[0],
-				 v4l2_subdev_state_get_format(sd_state, 0));
 	return 0;
 }
 
 static const struct v4l2_subdev_video_ops ov2740_video_ops = {
-	.s_stream = ov2740_set_stream,
+	.s_stream = v4l2_subdev_s_stream_helper,
 };
 
 static const struct v4l2_subdev_pad_ops ov2740_pad_ops = {
@@ -1116,6 +1245,9 @@ static const struct v4l2_subdev_pad_ops ov2740_pad_ops = {
 	.set_fmt = ov2740_set_format,
 	.enum_mbus_code = ov2740_enum_mbus_code,
 	.enum_frame_size = ov2740_enum_frame_size,
+	.enable_streams = ov2740_enable_streams,
+	.disable_streams = ov2740_disable_streams,
+	.get_frame_desc = ov2740_get_frame_desc,
 };
 
 static const struct v4l2_subdev_ops ov2740_subdev_ops = {
@@ -1419,11 +1551,16 @@ static int ov2740_probe(struct i2c_client *client)
 	}
 
 	ov2740->sd.state_lock = ov2740->ctrl_handler.lock;
-	ov2740->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+	ov2740->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
 	ov2740->sd.entity.ops = &ov2740_subdev_entity_ops;
 	ov2740->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
-	ov2740->pad.flags = MEDIA_PAD_FL_SOURCE;
-	ret = media_entity_pads_init(&ov2740->sd.entity, 1, &ov2740->pad);
+	ov2740->pads[OV2740_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+	ov2740->pads[OV2740_PAD_PIXEL].flags =
+		MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_INTERNAL;
+	ov2740->pads[OV2740_PAD_META].flags =
+		MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_INTERNAL;
+	ret = media_entity_pads_init(&ov2740->sd.entity,
+				     ARRAY_SIZE(ov2740->pads), ov2740->pads);
 	if (ret) {
 		dev_err_probe(dev, ret, "failed to init entity pads\n");
 		goto probe_error_v4l2_ctrl_handler_free;
-- 
2.47.2

[PATCH v11 48/66] media: ov2740: Add support for generic raw formats

[Sakari Ailus]

Add Y10 media bus code and V4L2_CID_COLOR_PATTERN as well as
V4L2_CID_COLOR_PATTERN_FLIP control support.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ov2740.c | 44 ++++++++++++++++++++++++++++----------
 1 file changed, 33 insertions(+), 11 deletions(-)

diff --git a/drivers/media/i2c/ov2740.c b/drivers/media/i2c/ov2740.c
index 684e6999dbd9..0b9309fd0461 100644
--- a/drivers/media/i2c/ov2740.c
+++ b/drivers/media/i2c/ov2740.c
@@ -741,7 +741,8 @@ static int ov2740_set_ctrl(struct v4l2_ctrl *ctrl)
 	int ret;
 
 	/* Propagate change of current control to all related controls */
-	if (ctrl->id == V4L2_CID_VBLANK) {
+	switch (ctrl->id) {
+	case V4L2_CID_VBLANK:
 		/* Update max exposure while meeting expected vblanking */
 		exposure_max = ov2740->cur_mode->height + ctrl->val -
 			       OV2740_EXPOSURE_MAX_MARGIN;
@@ -749,6 +750,9 @@ static int ov2740_set_ctrl(struct v4l2_ctrl *ctrl)
 					 ov2740->exposure->minimum,
 					 exposure_max, ov2740->exposure->step,
 					 exposure_max);
+		break;
+	case V4L2_CID_COLOR_PATTERN:
+		return 0;
 	}
 
 	/* V4L2 controls values will be applied only when power is already up */
@@ -804,7 +808,7 @@ static int ov2740_init_controls(struct ov2740 *ov2740)
 	int ret;
 
 	ctrl_hdlr = &ov2740->ctrl_handler;
-	ret = v4l2_ctrl_handler_init(ctrl_hdlr, 10);
+	ret = v4l2_ctrl_handler_init(ctrl_hdlr, 13);
 	if (ret)
 		return ret;
 
@@ -852,6 +856,16 @@ static int ov2740_init_controls(struct ov2740 *ov2740)
 				     V4L2_CID_TEST_PATTERN,
 				     ARRAY_SIZE(ov2740_test_pattern_menu) - 1,
 				     0, 0, ov2740_test_pattern_menu);
+	v4l2_ctrl_new_std(ctrl_hdlr, NULL, V4L2_CID_COLOR_PATTERN,
+			  V4L2_COLOR_PATTERN_GRBG, V4L2_COLOR_PATTERN_GRBG,
+			  1, V4L2_COLOR_PATTERN_GRBG);
+	v4l2_ctrl_new_std(ctrl_hdlr, NULL,
+			  V4L2_CID_COLOR_PATTERN_FLIP,
+			  0, V4L2_COLOR_PATTERN_FLIP_BOTH,
+			  0, V4L2_COLOR_PATTERN_FLIP_BOTH);
+	v4l2_ctrl_new_std(ctrl_hdlr, NULL, V4L2_CID_CONFIG_MODEL,
+			  0, V4L2_CONFIG_MODEL_COMMON_RAW_SENSOR,
+			  0, V4L2_CONFIG_MODEL_COMMON_RAW_SENSOR);
 
 	ret = v4l2_fwnode_device_parse(&client->dev, &props);
 	if (ret) {
@@ -876,7 +890,8 @@ static void ov2740_update_pad_format(const struct ov2740_mode *mode,
 {
 	fmt->width = mode->width;
 	fmt->height = mode->height;
-	fmt->code = MEDIA_BUS_FMT_SGRBG10_1X10;
+	fmt->code = fmt->code == MEDIA_BUS_FMT_RAW_10 ?
+		MEDIA_BUS_FMT_RAW_10 : MEDIA_BUS_FMT_SGRBG10_1X10;
 	fmt->field = V4L2_FIELD_NONE;
 }
 
@@ -1072,7 +1087,7 @@ static int __ov2740_set_format(struct v4l2_subdev *sd,
 			       enum v4l2_subdev_format_whence which,
 			       unsigned int pad, unsigned int stream)
 {
-	struct v4l2_mbus_framefmt *src_pix_fmt, *src_meta_fmt, *pix_fmt,
+	struct v4l2_mbus_framefmt *src_img_fmt, *src_meta_fmt, *img_fmt,
 		*meta_fmt;
 	struct ov2740 *ov2740 = to_ov2740(sd);
 	const struct ov2740_mode *mode;
@@ -1088,9 +1103,9 @@ static int __ov2740_set_format(struct v4l2_subdev *sd,
 		return 0;
 	}
 
-	pix_fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_PIXEL, 0);
+	img_fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_PIXEL, 0);
 	meta_fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_META, 0);
-	src_pix_fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_SOURCE,
+	src_img_fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_SOURCE,
 						   OV2740_STREAM_PIXEL);
 	src_meta_fmt = v4l2_subdev_state_get_format(sd_state, OV2740_PAD_SOURCE,
 						    OV2740_STREAM_META);
@@ -1099,8 +1114,10 @@ static int __ov2740_set_format(struct v4l2_subdev *sd,
 				      ov2740->supported_modes_count,
 				      width, height,
 				      format->width, format->height);
-	ov2740_update_pad_format(mode, pix_fmt);
-	*format = *src_pix_fmt = *pix_fmt;
+
+	img_fmt->code = format->code;
+	ov2740_update_pad_format(mode, img_fmt);
+	*format = *src_img_fmt = *img_fmt;
 
 	meta_fmt->code = MEDIA_BUS_FMT_META_10;
 	meta_fmt->width = OV2740_META_WIDTH;
@@ -1139,10 +1156,14 @@ static int ov2740_enum_mbus_code(struct v4l2_subdev *sd,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_mbus_code_enum *code)
 {
-	if (code->index > 0)
+	const u32 codes[] = {
+		MEDIA_BUS_FMT_SGRBG10_1X10,
+		MEDIA_BUS_FMT_RAW_10,
+	};
+	if (code->index >= ARRAY_SIZE(codes))
 		return -EINVAL;
 
-	code->code = MEDIA_BUS_FMT_SGRBG10_1X10;
+	code->code = codes[code->index];
 
 	return 0;
 }
@@ -1157,7 +1178,8 @@ static int ov2740_enum_frame_size(struct v4l2_subdev *sd,
 	if (fse->index >= ov2740->supported_modes_count)
 		return -EINVAL;
 
-	if (fse->code != MEDIA_BUS_FMT_SGRBG10_1X10)
+	if (fse->code != MEDIA_BUS_FMT_SGRBG10_1X10 &&
+	    fse->code != MEDIA_BUS_FMT_RAW_10)
 		return -EINVAL;
 
 	fse->min_width = supported_modes[fse->index].width;
-- 
2.47.2

[PATCH v11 49/66] media: ov2740: Add metadata layout control

[Sakari Ailus]

Add the V4L2_CID_METADATA_LAYOUT control to the sub-device's control
handler, to convey the metadata layout for the sensor-generated embedded
data.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ov2740.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/media/i2c/ov2740.c b/drivers/media/i2c/ov2740.c
index 0b9309fd0461..00319eacf89e 100644
--- a/drivers/media/i2c/ov2740.c
+++ b/drivers/media/i2c/ov2740.c
@@ -863,6 +863,9 @@ static int ov2740_init_controls(struct ov2740 *ov2740)
 			  V4L2_CID_COLOR_PATTERN_FLIP,
 			  0, V4L2_COLOR_PATTERN_FLIP_BOTH,
 			  0, V4L2_COLOR_PATTERN_FLIP_BOTH);
+	v4l2_ctrl_new_std(ctrl_hdlr, NULL, V4L2_CID_METADATA_LAYOUT,
+			  0, V4L2_METADATA_LAYOUT_OV2740,
+			  1, V4L2_METADATA_LAYOUT_OV2740);
 	v4l2_ctrl_new_std(ctrl_hdlr, NULL, V4L2_CID_CONFIG_MODEL,
 			  0, V4L2_CONFIG_MODEL_COMMON_RAW_SENSOR,
 			  0, V4L2_CONFIG_MODEL_COMMON_RAW_SENSOR);
-- 
2.47.2

[PATCH v11 50/66] media: ov2740: Add support for G_SELECTION IOCTL

[Sakari Ailus]

Add support for the G_SELECTION IOCTL in the ov2740 driver.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ov2740.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/drivers/media/i2c/ov2740.c b/drivers/media/i2c/ov2740.c
index 00319eacf89e..2480813113cb 100644
--- a/drivers/media/i2c/ov2740.c
+++ b/drivers/media/i2c/ov2740.c
@@ -24,6 +24,11 @@
 #define OV2740_DATA_LANES		2
 #define OV2740_RGB_DEPTH		10
 
+#define OV2740_DUMMY_LINES_PRE		8U
+#define OV2740_DUMMY_LINES_POST		8U
+#define OV2740_ACTIVE_WIDTH		1932U
+#define OV2740_ACTIVE_HEIGHT		1092U
+
 #define OV2740_REG_CHIP_ID		0x300a
 #define OV2740_CHIP_ID			0x2740
 
@@ -1155,6 +1160,35 @@ static int ov2740_set_format(struct v4l2_subdev *sd,
 				   fmt->pad, fmt->stream);
 }
 
+static int ov2740_get_selection(struct v4l2_subdev *sd,
+				struct v4l2_subdev_state *state,
+				struct v4l2_subdev_selection *sel)
+{
+	if (sel->stream != OV2740_STREAM_PIXEL ||
+	    sel->pad != OV2740_PAD_SOURCE)
+		return -EINVAL;
+
+	switch (sel->target) {
+	case V4L2_SEL_TGT_NATIVE_SIZE:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+		sel->r.top = 0;
+		sel->r.left = 0;
+		sel->r.width = OV2740_ACTIVE_WIDTH;
+		sel->r.height = OV2740_DUMMY_LINES_PRE + OV2740_ACTIVE_HEIGHT +
+			OV2740_DUMMY_LINES_POST;
+		return 0;
+	case V4L2_SEL_TGT_CROP:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+		sel->r.top = OV2740_DUMMY_LINES_PRE;
+		sel->r.left = 0;
+		sel->r.width = OV2740_ACTIVE_WIDTH;
+		sel->r.height = OV2740_ACTIVE_HEIGHT;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
 static int ov2740_enum_mbus_code(struct v4l2_subdev *sd,
 				 struct v4l2_subdev_state *sd_state,
 				 struct v4l2_subdev_mbus_code_enum *code)
@@ -1268,6 +1302,7 @@ static const struct v4l2_subdev_video_ops ov2740_video_ops = {
 static const struct v4l2_subdev_pad_ops ov2740_pad_ops = {
 	.get_fmt = v4l2_subdev_get_fmt,
 	.set_fmt = ov2740_set_format,
+	.get_selection = ov2740_get_selection,
 	.enum_mbus_code = ov2740_enum_mbus_code,
 	.enum_frame_size = ov2740_enum_frame_size,
 	.enable_streams = ov2740_enable_streams,
-- 
2.47.2

[PATCH v11 51/66] media: v4l: Add V4L2_SUBDEV_ROUTE_FL_IMMUTABLE sub-device routing flag

[Sakari Ailus]

Add a flag to denote immutable routes, V4L2_SUBDEV_ROUTE_FL_IMMUTABLE.
Such routes cannot be changed and they're always active.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 Documentation/userspace-api/media/v4l/dev-subdev.rst         | 4 +++-
 .../userspace-api/media/v4l/vidioc-subdev-g-routing.rst      | 5 +++++
 include/uapi/linux/v4l2-subdev.h                             | 5 +++++
 3 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index b0774b9a9b71..684ef0844517 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -572,7 +572,9 @@ internal pad always has a single stream only (0).
 Routes from an internal sink pad to an external source pad are typically not
 modifiable but they can be activated and deactivated using the
 :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag, depending
-on driver capabilities.
+on driver capabilities. The :ref:`V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
+<v4l2-subdev-routing-flags>` flag indicates that the
+``V4L2_SUBDEV_ROUTE_FLAG_ACTIVE`` of the route may not be unset.
 
 Interaction between routes, streams, formats and selections
 -----------------------------------------------------------
diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
index 15f448664e6b..0b90be43af57 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
@@ -146,6 +146,11 @@ wants to retrieve the missing routes, it can issue a new
     * - V4L2_SUBDEV_ROUTE_FL_ACTIVE
       - 0x0001
       - The route is enabled. Set by applications.
+    * - V4L2_SUBDEV_ROUTE_FL_IMMUTABLE
+      - 0x0002
+      - The route is immutable. Set by the driver. Indicates that the
+	``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag of an immutable route may not be
+	unset.
 
 Return Value
 ============
diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
index 2347e266cf75..839b1329afb2 100644
--- a/include/uapi/linux/v4l2-subdev.h
+++ b/include/uapi/linux/v4l2-subdev.h
@@ -204,6 +204,11 @@ struct v4l2_subdev_capability {
  * on a video node.
  */
 #define V4L2_SUBDEV_ROUTE_FL_ACTIVE		(1U << 0)
+/*
+ * Is the route immutable? The ACTIVE flag of an immutable route may not be
+ * unset.
+ */
+#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE		(1U << 1)
 
 /**
  * struct v4l2_subdev_route - A route inside a subdev
-- 
2.47.2

[PATCH v11 52/66] media: v4l: Add V4L2_SUBDEV_ROUTE_FL_STATIC sub-device routing flag

[Sakari Ailus]

Add a flag to denote static routes, V4L2_SUBDEV_ROUTE_FL_STATIC. Such
routes cannot be removed as they are related to hardware features.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../userspace-api/media/v4l/vidioc-subdev-g-routing.rst      | 5 +++++
 include/uapi/linux/v4l2-subdev.h                             | 5 +++++
 2 files changed, 10 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
index 0b90be43af57..3c7fde673b34 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
@@ -151,6 +151,11 @@ wants to retrieve the missing routes, it can issue a new
       - The route is immutable. Set by the driver. Indicates that the
 	``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag of an immutable route may not be
 	unset.
+    * - ``V4L2_SUBDEV_ROUTE_FL_STATIC``
+      - 0x0004
+      - The route is static. Set by the driver. Indicates that the route is
+        always present and cannot be removed. Implies
+        ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``.
 
 Return Value
 ============
diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
index 839b1329afb2..cb353fa4a4a6 100644
--- a/include/uapi/linux/v4l2-subdev.h
+++ b/include/uapi/linux/v4l2-subdev.h
@@ -209,6 +209,11 @@ struct v4l2_subdev_capability {
  * unset.
  */
 #define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE		(1U << 1)
+/*
+ * Is the route static? The STATIC flag of an immutable route may not be
+ * unset.
+ */
+#define V4L2_SUBDEV_ROUTE_FL_STATIC		(1U << 2)
 
 /**
  * struct v4l2_subdev_route - A route inside a subdev
-- 
2.47.2

Re: [PATCH v11 52/66] media: v4l: Add V4L2_SUBDEV_ROUTE_FL_STATIC sub-device routing flag

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:53PM +0300, Sakari Ailus wrote:
> Add a flag to denote static routes, V4L2_SUBDEV_ROUTE_FL_STATIC. Such
> routes cannot be removed as they are related to hardware features.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../userspace-api/media/v4l/vidioc-subdev-g-routing.rst      | 5 +++++
>  include/uapi/linux/v4l2-subdev.h                             | 5 +++++
>  2 files changed, 10 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> index 0b90be43af57..3c7fde673b34 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> @@ -151,6 +151,11 @@ wants to retrieve the missing routes, it can issue a new
>        - The route is immutable. Set by the driver. Indicates that the
>  	``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag of an immutable route may not be
>  	unset.
> +    * - ``V4L2_SUBDEV_ROUTE_FL_STATIC``
> +      - 0x0004
> +      - The route is static. Set by the driver. Indicates that the route is
> +        always present and cannot be removed. Implies
> +        ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``.
>
>  Return Value
>  ============
> diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
> index 839b1329afb2..cb353fa4a4a6 100644
> --- a/include/uapi/linux/v4l2-subdev.h
> +++ b/include/uapi/linux/v4l2-subdev.h
> @@ -209,6 +209,11 @@ struct v4l2_subdev_capability {
>   * unset.
>   */
>  #define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE		(1U << 1)
> +/*
> + * Is the route static? The STATIC flag of an immutable route may not be

Mentioning "immutable" here might be confusing, but it might just be
me

Can we fast-track this and the previous one ?

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

> + * unset.
> + */
> +#define V4L2_SUBDEV_ROUTE_FL_STATIC		(1U << 2)
>
>  /**
>   * struct v4l2_subdev_route - A route inside a subdev
> --
> 2.47.2
>
>

Re: [PATCH v11 52/66] media: v4l: Add V4L2_SUBDEV_ROUTE_FL_STATIC sub-device routing flag

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Mon, Aug 25, 2025 at 12:50:53PM +0300, Sakari Ailus wrote:
> Add a flag to denote static routes, V4L2_SUBDEV_ROUTE_FL_STATIC. Such
> routes cannot be removed as they are related to hardware features.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../userspace-api/media/v4l/vidioc-subdev-g-routing.rst      | 5 +++++
>  include/uapi/linux/v4l2-subdev.h                             | 5 +++++
>  2 files changed, 10 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> index 0b90be43af57..3c7fde673b34 100644
> --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-routing.rst
> @@ -151,6 +151,11 @@ wants to retrieve the missing routes, it can issue a new
>        - The route is immutable. Set by the driver. Indicates that the
>  	``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag of an immutable route may not be
>  	unset.
> +    * - ``V4L2_SUBDEV_ROUTE_FL_STATIC``
> +      - 0x0004
> +      - The route is static. Set by the driver. Indicates that the route is
> +        always present and cannot be removed. Implies
> +        ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE``.

It's the other way around. IMMUTABLE implies STATIC, but not all static
routes are immutable.

      - The route is static. Set by the driver. Indicates that the route is
        created by the driver and cannot be removed. Static routes may
	be immutable, or may allow userspace to enable and disable them.

I'm not fully happy with the last sentence, but "enable-able" isn't a
word :-)

>  
>  Return Value
>  ============
> diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
> index 839b1329afb2..cb353fa4a4a6 100644
> --- a/include/uapi/linux/v4l2-subdev.h
> +++ b/include/uapi/linux/v4l2-subdev.h
> @@ -209,6 +209,11 @@ struct v4l2_subdev_capability {
>   * unset.
>   */
>  #define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE		(1U << 1)
> +/*
> + * Is the route static? The STATIC flag of an immutable route may not be
> + * unset.

The static flag can never be unset. We need to document this better
above. What happens if an application tries to unset it ?

> + */
> +#define V4L2_SUBDEV_ROUTE_FL_STATIC		(1U << 2)
>  
>  /**
>   * struct v4l2_subdev_route - A route inside a subdev

-- 
Regards,

Laurent Pinchart

[PATCH v11 53/66] media: Documentation: Document IMMUTABLE and STATIC route flags

[Sakari Ailus]

Include the V4L2_SUBDEV_ROUTE_FL_IMMUTABLE and V4L2_SUBDEV_ROUTE_FL_STATIC
flags in V4L2 sub-device documentation that discusses routing.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 Documentation/userspace-api/media/v4l/dev-subdev.rst | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index 684ef0844517..2e94e00978c0 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -603,11 +603,14 @@ depending on the hardware. In all cases, however, only routes that have the
 ``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag set are active.
 
 Devices generating the streams may allow enabling and disabling some of the
-routes or have a fixed routing configuration. If the routes can be disabled, not
+routes or have a fixed routing configuration. Such routes that are directly tied
+to hardware resources are marked with ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE`` route
+flag. Routes that are always present but can be disabled are marked with
+``V4L2_SUBDEV_ROUTE_FL_STATIC`` flag. Static routes can be disabled by not
 declaring the routes (or declaring them without ``V4L2_SUBDEV_ROUTE_FL_ACTIVE``
-flag set) in ``VIDIOC_SUBDEV_S_ROUTING`` will disable the routes.
-``VIDIOC_SUBDEV_S_ROUTING`` will still return such routes back to the user in
-the routes array, with the ``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag unset.
+flag set) in ``VIDIOC_SUBDEV_S_ROUTING``. ``VIDIOC_SUBDEV_S_ROUTING`` will still
+return such routes back to the user in the routes array, with the
+``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag unset.
 
 Devices transporting the streams almost always have more configurability with
 respect to routing. Typically any route between the sub-device's sink and source
-- 
2.47.2

Re: [PATCH v11 53/66] media: Documentation: Document IMMUTABLE and STATIC route flags

[Jacopo Mondi]

Hi Sakari

On Mon, Aug 25, 2025 at 12:50:54PM +0300, Sakari Ailus wrote:
> Include the V4L2_SUBDEV_ROUTE_FL_IMMUTABLE and V4L2_SUBDEV_ROUTE_FL_STATIC
> flags in V4L2 sub-device documentation that discusses routing.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  Documentation/userspace-api/media/v4l/dev-subdev.rst | 11 +++++++----
>  1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index 684ef0844517..2e94e00978c0 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -603,11 +603,14 @@ depending on the hardware. In all cases, however, only routes that have the
>  ``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag set are active.
>
>  Devices generating the streams may allow enabling and disabling some of the
> -routes or have a fixed routing configuration. If the routes can be disabled, not
> +routes or have a fixed routing configuration. Such routes that are directly tied

s/Such routes/Routes

> +to hardware resources are marked with ``V4L2_SUBDEV_ROUTE_FL_IMMUTABLE`` route
   to hardware resources and cannot be disabled by userspace are ...

> +flag. Routes that are always present but can be disabled are marked with
> +``V4L2_SUBDEV_ROUTE_FL_STATIC`` flag. Static routes can be disabled by not
>  declaring the routes (or declaring them without ``V4L2_SUBDEV_ROUTE_FL_ACTIVE``
> -flag set) in ``VIDIOC_SUBDEV_S_ROUTING`` will disable the routes.
> -``VIDIOC_SUBDEV_S_ROUTING`` will still return such routes back to the user in
> -the routes array, with the ``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag unset.
> +flag set) in ``VIDIOC_SUBDEV_S_ROUTING``. ``VIDIOC_SUBDEV_S_ROUTING`` will still
> +return such routes back to the user in the routes array, with the
> +``V4L2_SUBDEV_ROUTE_FL_ACTIVE`` flag unset.

Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

This can be fast-tracked along with the previous two patches I guess

>
>  Devices transporting the streams almost always have more configurability with
>  respect to routing. Typically any route between the sub-device's sink and source
> --
> 2.47.2
>
>

[PATCH v11 54/66] media: uapi: v4l: subdev: Enable streams API

[Sakari Ailus]

Remove v4l2_subdev_enable_streams_api variable that was used to easily
enable streams API for development, and conditions that use the variable.

This patch enables the streams API for V4L2 sub-device interface which
allows transporting multiple streams on a single MC link.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 30 ---------------------------
 1 file changed, 30 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 8d4c484109fb..8100a812c766 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -50,15 +50,6 @@ struct v4l2_subdev_stream_config {
 	struct v4l2_fract interval;
 };
 
-#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
-/*
- * The Streams API is an experimental feature. To use the Streams API, set
- * 'v4l2_subdev_enable_streams_api' to 1 below.
- */
-
-static bool v4l2_subdev_enable_streams_api;
-#endif
-
 /*
  * Maximum stream ID is 63 for now, as we use u64 bitmask to represent a set
  * of streams.
@@ -642,13 +633,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 				       V4L2_SUBDEV_CLIENT_CAP_STREAMS;
 	int rval;
 
-	/*
-	 * If the streams API is not enabled, remove V4L2_SUBDEV_CAP_STREAMS.
-	 * Remove this when the API is no longer experimental.
-	 */
-	if (!v4l2_subdev_enable_streams_api)
-		streams_subdev = false;
-
 	switch (cmd) {
 	case VIDIOC_SUBDEV_QUERYCAP: {
 		struct v4l2_subdev_capability *cap = arg;
@@ -1002,9 +986,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 		struct v4l2_subdev_routing *routing = arg;
 		struct v4l2_subdev_krouting *krouting;
 
-		if (!v4l2_subdev_enable_streams_api)
-			return -ENOIOCTLCMD;
-
 		if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS))
 			return -ENOIOCTLCMD;
 
@@ -1029,9 +1010,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 		unsigned int num_active_routes = 0;
 		unsigned int i;
 
-		if (!v4l2_subdev_enable_streams_api)
-			return -ENOIOCTLCMD;
-
 		if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS))
 			return -ENOIOCTLCMD;
 
@@ -1122,14 +1100,6 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg,
 	case VIDIOC_SUBDEV_S_CLIENT_CAP: {
 		struct v4l2_subdev_client_capability *client_cap = arg;
 
-		/*
-		 * Clear V4L2_SUBDEV_CLIENT_CAP_STREAMS if streams API is not
-		 * enabled. Remove this when streams API is no longer
-		 * experimental.
-		 */
-		if (!v4l2_subdev_enable_streams_api)
-			client_cap->capabilities &= ~V4L2_SUBDEV_CLIENT_CAP_STREAMS;
-
 		/* Filter out unsupported capabilities */
 		client_cap->capabilities &= (V4L2_SUBDEV_CLIENT_CAP_STREAMS |
 					     V4L2_SUBDEV_CLIENT_CAP_INTERVAL_USES_WHICH);
-- 
2.47.2

[PATCH v11 55/66] media: ccs: Add IMMUTABLE and STATIC route flags

[Sakari Ailus]

Add immutable and static route flags to the routing table. The embedded
data stream is always there.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Julien Massot <julien.massot@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/i2c/ccs/ccs-core.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 87a1614a020b..44e7c3a2f95a 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -3370,12 +3370,16 @@ static int ccs_src_init_state(struct v4l2_subdev *sd,
 			.sink_pad = CCS_PAD_SINK,
 			.source_pad = CCS_PAD_SRC,
 			.source_stream = CCS_STREAM_PIXEL,
-			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE |
+				 V4L2_SUBDEV_ROUTE_FL_IMMUTABLE	|
+				 V4L2_SUBDEV_ROUTE_FL_STATIC,
 		}, {
 			.sink_pad = CCS_PAD_META,
 			.source_pad = CCS_PAD_SRC,
 			.source_stream = CCS_STREAM_META,
-			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE |
+				 V4L2_SUBDEV_ROUTE_FL_IMMUTABLE	|
+				 V4L2_SUBDEV_ROUTE_FL_STATIC,
 		}
 	};
 	struct v4l2_subdev_krouting routing = {
-- 
2.47.2

[PATCH v11 56/66] media: ov2740: Add IMMUTABLE and STATIC route flags

[Sakari Ailus]

Add immutable and static route flags to the routing table. The driver does
not support disabling the embedded data whereas the sensor itself does.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Julien Massot <julien.massot@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/i2c/ov2740.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/media/i2c/ov2740.c b/drivers/media/i2c/ov2740.c
index 2480813113cb..8a2f22a7cf1e 100644
--- a/drivers/media/i2c/ov2740.c
+++ b/drivers/media/i2c/ov2740.c
@@ -1235,12 +1235,16 @@ static int ov2740_init_state(struct v4l2_subdev *sd,
 			.sink_pad = OV2740_PAD_PIXEL,
 			.source_pad = OV2740_PAD_SOURCE,
 			.source_stream = OV2740_STREAM_PIXEL,
-			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE |
+				 V4L2_SUBDEV_ROUTE_FL_IMMUTABLE |
+				 V4L2_SUBDEV_ROUTE_FL_STATIC,
 		}, {
 			.sink_pad = OV2740_PAD_META,
 			.source_pad = OV2740_PAD_SOURCE,
 			.source_stream = OV2740_STREAM_META,
-			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE |
+				 V4L2_SUBDEV_ROUTE_FL_IMMUTABLE |
+				 V4L2_SUBDEV_ROUTE_FL_STATIC,
 		},
 	};
 	struct v4l2_subdev_krouting routing = {
-- 
2.47.2

[PATCH v11 57/66] media: i2c: imx219: Inline imx219_update_pad_format() in its caller

[Sakari Ailus]

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

The imx219_update_pad_format() is short and called from a single place,
in imx219_set_pad_format(). Inline the code in the caller to keep all
format adjustments grouped in a single place and improve readability.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 05f0c2c69fd2..cdc56673fa5f 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -799,21 +799,6 @@ static int imx219_disable_streams(struct v4l2_subdev *sd,
 	return ret;
 }
 
-static void imx219_update_pad_format(struct imx219 *imx219,
-				     const struct imx219_mode *mode,
-				     struct v4l2_mbus_framefmt *fmt, u32 code)
-{
-	/* Bayer order varies with flips */
-	fmt->code = imx219_get_format_code(imx219, code);
-	fmt->width = mode->width;
-	fmt->height = mode->height;
-	fmt->field = V4L2_FIELD_NONE;
-	fmt->colorspace = V4L2_COLORSPACE_RAW;
-	fmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
-	fmt->quantization = V4L2_QUANTIZATION_FULL_RANGE;
-	fmt->xfer_func = V4L2_XFER_FUNC_NONE;
-}
-
 static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
 				 struct v4l2_subdev_state *state,
 				 struct v4l2_subdev_mbus_code_enum *code)
@@ -864,12 +849,24 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	format = v4l2_subdev_state_get_format(state, 0);
 	prev_line_len = format->width + imx219->hblank->val;
 
+	/*
+	 * Adjust the requested format to match the closest mode. The Bayer
+	 * order varies with flips.
+	 */
 	mode = v4l2_find_nearest_size(supported_modes,
 				      ARRAY_SIZE(supported_modes),
 				      width, height,
 				      fmt->format.width, fmt->format.height);
 
-	imx219_update_pad_format(imx219, mode, &fmt->format, fmt->format.code);
+	fmt->format.code = imx219_get_format_code(imx219, fmt->format.code);
+	fmt->format.width = mode->width;
+	fmt->format.height = mode->height;
+	fmt->format.field = V4L2_FIELD_NONE;
+	fmt->format.colorspace = V4L2_COLORSPACE_RAW;
+	fmt->format.ycbcr_enc = V4L2_YCBCR_ENC_601;
+	fmt->format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
+	fmt->format.xfer_func = V4L2_XFER_FUNC_NONE;
+
 	*format = fmt->format;
 
 	/*
-- 
2.47.2

[PATCH v11 58/66] media: i2c: imx219: Add internal image sink pad

[Sakari Ailus]

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Use the newly added internal pad API to expose the internal
configuration of the sensor to userspace in a standard manner. This also
paves the way for adding support for embedded data with an additional
internal pad.

To maintain compatibility with existing userspace that may operate on
pad 0 unconditionally, keep the source pad numbered 0 and number the
internal image pad 1.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 170 +++++++++++++++++++++++++++++--------
 1 file changed, 134 insertions(+), 36 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index cdc56673fa5f..1ab7fb1f220c 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -139,6 +139,7 @@
 #define IMX219_DEFAULT_LINK_FREQ_4LANE	364000000
 
 /* IMX219 native and active pixel array size. */
+#define IMX219_NATIVE_FORMAT		MEDIA_BUS_FMT_SRGGB10_1X10
 #define IMX219_NATIVE_WIDTH		3296U
 #define IMX219_NATIVE_HEIGHT		2480U
 #define IMX219_PIXEL_ARRAY_LEFT		8U
@@ -334,9 +335,15 @@ static const struct imx219_mode supported_modes[] = {
 	},
 };
 
+enum imx219_pad_ids {
+	IMX219_PAD_SOURCE = 0,
+	IMX219_PAD_IMAGE,
+	IMX219_NUM_PADS,
+};
+
 struct imx219 {
 	struct v4l2_subdev sd;
-	struct media_pad pad;
+	struct media_pad pads[IMX219_NUM_PADS];
 
 	struct regmap *regmap;
 	struct clk *xclk; /* system clock to IMX219 */
@@ -453,7 +460,7 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
 	int ret = 0;
 
 	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
-	format = v4l2_subdev_state_get_format(state, 0);
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
 	rate_factor = imx219_get_rate_factor(state);
 
 	if (ctrl->id == V4L2_CID_VBLANK) {
@@ -675,8 +682,8 @@ static int imx219_set_framefmt(struct imx219 *imx219,
 	u32 bpp;
 	int ret = 0;
 
-	format = v4l2_subdev_state_get_format(state, 0);
-	crop = v4l2_subdev_state_get_crop(state, 0);
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
+	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
 	bpp = imx219_get_format_bpp(format);
 
 	cci_write(imx219->regmap, IMX219_REG_X_ADD_STA_A,
@@ -805,10 +812,25 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
 {
 	struct imx219 *imx219 = to_imx219(sd);
 
-	if (code->index >= (ARRAY_SIZE(imx219_mbus_formats) / 4))
-		return -EINVAL;
+	if (code->pad == IMX219_PAD_IMAGE) {
+		/* The internal image pad is hardwired to the native format. */
+		if (code->index > 0)
+			return -EINVAL;
 
-	code->code = imx219_get_format_code(imx219, imx219_mbus_formats[code->index * 4]);
+		code->code = IMX219_NATIVE_FORMAT;
+	} else {
+		/*
+		 * On the source pad, the sensor supports multiple raw formats
+		 * with different bit depths.
+		 */
+		u32 format;
+
+		if (code->index >= ARRAY_SIZE(imx219_mbus_formats) / 4)
+			return -EINVAL;
+
+		format = imx219_mbus_formats[code->index * 4];
+		code->code = imx219_get_format_code(imx219, format);
+	}
 
 	return 0;
 }
@@ -818,19 +840,25 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
 				  struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct imx219 *imx219 = to_imx219(sd);
-	u32 code;
 
-	if (fse->index >= ARRAY_SIZE(supported_modes))
-		return -EINVAL;
-
-	code = imx219_get_format_code(imx219, fse->code);
-	if (fse->code != code)
-		return -EINVAL;
-
-	fse->min_width = supported_modes[fse->index].width;
-	fse->max_width = fse->min_width;
-	fse->min_height = supported_modes[fse->index].height;
-	fse->max_height = fse->min_height;
+	if (fse->pad == IMX219_PAD_IMAGE) {
+		if (fse->code != IMX219_NATIVE_FORMAT || fse->index > 0)
+			return -EINVAL;
+
+		fse->min_width = IMX219_NATIVE_WIDTH;
+		fse->max_width = IMX219_NATIVE_WIDTH;
+		fse->min_height = IMX219_NATIVE_HEIGHT;
+		fse->max_height = IMX219_NATIVE_HEIGHT;
+	} else {
+		if (fse->code != imx219_get_format_code(imx219, fse->code) ||
+		    fse->index >= ARRAY_SIZE(supported_modes))
+			return -EINVAL;
+
+		fse->min_width = supported_modes[fse->index].width;
+		fse->max_width = fse->min_width;
+		fse->min_height = supported_modes[fse->index].height;
+		fse->max_height = fse->min_height;
+	}
 
 	return 0;
 }
@@ -842,6 +870,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	struct imx219 *imx219 = to_imx219(sd);
 	const struct imx219_mode *mode;
 	struct v4l2_mbus_framefmt *format;
+	struct v4l2_rect *compose;
 	struct v4l2_rect *crop;
 	u8 bin_h, bin_v;
 	u32 prev_line_len;
@@ -849,6 +878,13 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	format = v4l2_subdev_state_get_format(state, 0);
 	prev_line_len = format->width + imx219->hblank->val;
 
+	/*
+	 * The driver is mode-based, the format can be set on the source pad
+	 * only.
+	 */
+	if (fmt->pad != IMX219_PAD_SOURCE)
+		return v4l2_subdev_get_fmt(sd, state, fmt);
+
 	/*
 	 * Adjust the requested format to match the closest mode. The Bayer
 	 * order varies with flips.
@@ -867,21 +903,50 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	fmt->format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
 	fmt->format.xfer_func = V4L2_XFER_FUNC_NONE;
 
+	/* Propagate the format through the sensor. */
+
+	/* The image pad models the pixel array, and thus has a fixed format. */
 	*format = fmt->format;
+	format->code = IMX219_NATIVE_FORMAT;
+	format->width = IMX219_NATIVE_WIDTH;
+	format->height = IMX219_NATIVE_HEIGHT;
 
 	/*
 	 * Use binning to maximize the crop rectangle size, and centre it in the
 	 * sensor.
 	 */
-	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / format->width, 2U);
-	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / format->height, 2U);
+	bin_h = min(IMX219_PIXEL_ARRAY_WIDTH / fmt->format.width, 2U);
+	bin_v = min(IMX219_PIXEL_ARRAY_HEIGHT / fmt->format.height, 2U);
 
-	crop = v4l2_subdev_state_get_crop(state, 0);
-	crop->width = format->width * bin_h;
-	crop->height = format->height * bin_v;
+	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
+	crop->width = fmt->format.width * bin_h;
+	crop->height = fmt->format.height * bin_v;
 	crop->left = (IMX219_NATIVE_WIDTH - crop->width) / 2;
 	crop->top = (IMX219_NATIVE_HEIGHT - crop->height) / 2;
 
+	/*
+	 * The compose rectangle models binning, its size is the sensor output
+	 * size.
+	 */
+	compose = v4l2_subdev_state_get_compose(state, IMX219_PAD_IMAGE);
+	compose->left = 0;
+	compose->top = 0;
+	compose->width = fmt->format.width;
+	compose->height = fmt->format.height;
+
+	/*
+	 * No mode use digital crop, the source pad crop rectangle size and
+	 * format are thus identical to the image pad compose rectangle.
+	 */
+	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE);
+	crop->left = 0;
+	crop->top = 0;
+	crop->width = fmt->format.width;
+	crop->height = fmt->format.height;
+
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
+	*format = fmt->format;
+
 	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
 		int exposure_max;
 		int exposure_def;
@@ -939,12 +1004,13 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_selection *sel)
 {
-	switch (sel->target) {
-	case V4L2_SEL_TGT_CROP:
-		sel->r = *v4l2_subdev_state_get_crop(state, 0);
-		return 0;
+	struct v4l2_rect *compose;
 
+	switch (sel->target) {
 	case V4L2_SEL_TGT_NATIVE_SIZE:
+		if (sel->pad != IMX219_PAD_IMAGE)
+			return -EINVAL;
+
 		sel->r.top = 0;
 		sel->r.left = 0;
 		sel->r.width = IMX219_NATIVE_WIDTH;
@@ -954,11 +1020,35 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 
 	case V4L2_SEL_TGT_CROP_DEFAULT:
 	case V4L2_SEL_TGT_CROP_BOUNDS:
-		sel->r.top = IMX219_PIXEL_ARRAY_TOP;
-		sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
-		sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
-		sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
+		switch (sel->pad) {
+		case IMX219_PAD_IMAGE:
+			sel->r.top = IMX219_PIXEL_ARRAY_TOP;
+			sel->r.left = IMX219_PIXEL_ARRAY_LEFT;
+			sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;
+			sel->r.height = IMX219_PIXEL_ARRAY_HEIGHT;
+			return 0;
+
+		case IMX219_PAD_SOURCE:
+			compose = v4l2_subdev_state_get_compose(state,
+								IMX219_PAD_IMAGE);
+			sel->r.top = 0;
+			sel->r.left = 0;
+			sel->r.width = compose->width;
+			sel->r.height = compose->height;
+			return 0;
+		}
+
+		break;
+
+	case V4L2_SEL_TGT_CROP:
+		sel->r = *v4l2_subdev_state_get_crop(state, sel->pad);
+		return 0;
+
+	case V4L2_SEL_TGT_COMPOSE:
+		if (sel->pad != IMX219_PAD_IMAGE)
+			return -EINVAL;
 
+		sel->r = *v4l2_subdev_state_get_compose(state, sel->pad);
 		return 0;
 	}
 
@@ -970,7 +1060,7 @@ static int imx219_init_state(struct v4l2_subdev *sd,
 {
 	struct v4l2_subdev_format fmt = {
 		.which = V4L2_SUBDEV_FORMAT_TRY,
-		.pad = 0,
+		.pad = IMX219_PAD_SOURCE,
 		.format = {
 			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
 			.width = supported_modes[0].width,
@@ -1248,10 +1338,18 @@ static int imx219_probe(struct i2c_client *client)
 	imx219->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
 	imx219->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
 
-	/* Initialize source pad */
-	imx219->pad.flags = MEDIA_PAD_FL_SOURCE;
+	/*
+	 * Initialize the pads. To preserve backward compatibility with
+	 * userspace that used the sensor before the introduction of the
+	 * internal image pad, the external source pad is numbered 0 and the
+	 * internal image pad numbered 1.
+	 */
+	imx219->pads[IMX219_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+	imx219->pads[IMX219_PAD_IMAGE].flags = MEDIA_PAD_FL_SINK
+					     | MEDIA_PAD_FL_INTERNAL;
 
-	ret = media_entity_pads_init(&imx219->sd.entity, 1, &imx219->pad);
+	ret = media_entity_pads_init(&imx219->sd.entity,
+				     ARRAY_SIZE(imx219->pads), imx219->pads);
 	if (ret) {
 		dev_err_probe(dev, ret, "failed to init entity pads\n");
 		goto error_handler_free;
-- 
2.47.2

[PATCH v11 59/66] media: i2c: imx219: Add image stream

[Sakari Ailus]

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

In preparation for embedded data stream support, introduce a new
imx219_stream_ids enumeration for stream IDs, with a single value,
IMX219_STREAM_IMAGE for the image data stream. Use it when accessing the
formats, crop and compose rectangles on the source pad. This is meant to
reduce the size of further commits, and doesn't introduce any functional
change.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 1ab7fb1f220c..bb972fd90026 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -341,6 +341,10 @@ enum imx219_pad_ids {
 	IMX219_NUM_PADS,
 };
 
+enum imx219_stream_ids {
+	IMX219_STREAM_IMAGE,
+};
+
 struct imx219 {
 	struct v4l2_subdev sd;
 	struct media_pad pads[IMX219_NUM_PADS];
@@ -460,7 +464,8 @@ static int imx219_set_ctrl(struct v4l2_ctrl *ctrl)
 	int ret = 0;
 
 	state = v4l2_subdev_get_locked_active_state(&imx219->sd);
-	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+					      IMX219_STREAM_IMAGE);
 	rate_factor = imx219_get_rate_factor(state);
 
 	if (ctrl->id == V4L2_CID_VBLANK) {
@@ -682,7 +687,8 @@ static int imx219_set_framefmt(struct imx219 *imx219,
 	u32 bpp;
 	int ret = 0;
 
-	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+					      IMX219_STREAM_IMAGE);
 	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_IMAGE);
 	bpp = imx219_get_format_bpp(format);
 
@@ -938,13 +944,15 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 	 * No mode use digital crop, the source pad crop rectangle size and
 	 * format are thus identical to the image pad compose rectangle.
 	 */
-	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE);
+	crop = v4l2_subdev_state_get_crop(state, IMX219_PAD_SOURCE,
+					  IMX219_STREAM_IMAGE);
 	crop->left = 0;
 	crop->top = 0;
 	crop->width = fmt->format.width;
 	crop->height = fmt->format.height;
 
-	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE);
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+					      IMX219_STREAM_IMAGE);
 	*format = fmt->format;
 
 	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
-- 
2.47.2

[PATCH v11 60/66] media: i2c: imx219: Report internal routes to userspace

[Sakari Ailus]

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Usage of internal pads creates a route internal to the subdev, and the
V4L2 camera sensor API requires such routes to be reported to userspace.
Create the route in the .init_state() operation.

Internal routing support requires stream support, so set the
V4L2_SUBDEV_FL_STREAMS flag. As the route is immutable, there's no need
to implement the .set_routing() operation.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index bb972fd90026..136c084bb930 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -1066,15 +1066,37 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 static int imx219_init_state(struct v4l2_subdev *sd,
 			     struct v4l2_subdev_state *state)
 {
+	struct v4l2_subdev_route routes[1] = {
+		{
+			.sink_pad = IMX219_PAD_IMAGE,
+			.sink_stream = 0,
+			.source_pad = IMX219_PAD_SOURCE,
+			.source_stream = IMX219_STREAM_IMAGE,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE |
+				 V4L2_SUBDEV_ROUTE_FL_IMMUTABLE |
+				 V4L2_SUBDEV_ROUTE_FL_STATIC,
+		},
+	};
+	struct v4l2_subdev_krouting routing = {
+		.len_routes = ARRAY_SIZE(routes),
+		.num_routes = ARRAY_SIZE(routes),
+		.routes = routes,
+	};
 	struct v4l2_subdev_format fmt = {
 		.which = V4L2_SUBDEV_FORMAT_TRY,
 		.pad = IMX219_PAD_SOURCE,
+		.stream = IMX219_STREAM_IMAGE,
 		.format = {
 			.code = MEDIA_BUS_FMT_SRGGB10_1X10,
 			.width = supported_modes[0].width,
 			.height = supported_modes[0].height,
 		},
 	};
+	int ret;
+
+	ret = v4l2_subdev_set_routing(sd, state, &routing);
+	if (ret)
+		return ret;
 
 	imx219_set_pad_format(sd, state, &fmt);
 
@@ -1343,7 +1365,8 @@ static int imx219_probe(struct i2c_client *client)
 		goto error_power_off;
 
 	/* Initialize subdev */
-	imx219->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+	imx219->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
+			    V4L2_SUBDEV_FL_STREAMS;
 	imx219->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
 
 	/*
-- 
2.47.2

[PATCH v11 61/66] media: i2c: imx219: Report streams using frame descriptors

[Sakari Ailus]

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Implement the .get_frame_desc() subdev operation to report information
about streams to the connected CSI-2 receiver. This is required to let
the CSI-2 receiver driver know about virtual channels and data types for
each stream.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 48 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 136c084bb930..8ed1febc7b21 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -23,6 +23,7 @@
 #include <linux/pm_runtime.h>
 #include <linux/regulator/consumer.h>
 
+#include <media/mipi-csi2.h>
 #include <media/v4l2-cci.h>
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-device.h>
@@ -678,6 +679,24 @@ static void imx219_free_controls(struct imx219 *imx219)
  * Subdev operations
  */
 
+static unsigned int imx219_format_bpp(u32 code)
+{
+	switch (code) {
+	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_SGRBG8_1X8:
+	case MEDIA_BUS_FMT_SGBRG8_1X8:
+	case MEDIA_BUS_FMT_SBGGR8_1X8:
+		return 8;
+
+	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_SGRBG10_1X10:
+	case MEDIA_BUS_FMT_SGBRG10_1X10:
+	case MEDIA_BUS_FMT_SBGGR10_1X10:
+	default:
+		return 10;
+	}
+}
+
 static int imx219_set_framefmt(struct imx219 *imx219,
 			       struct v4l2_subdev_state *state)
 {
@@ -1063,6 +1082,34 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 	return -EINVAL;
 }
 
+static int imx219_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
+				 struct v4l2_mbus_frame_desc *fd)
+{
+	const struct v4l2_mbus_framefmt *fmt;
+	struct v4l2_subdev_state *state;
+	u32 code;
+
+	if (pad != IMX219_PAD_SOURCE)
+		return -EINVAL;
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+					   IMX219_STREAM_IMAGE);
+	code = fmt->code;
+	v4l2_subdev_unlock_state(state);
+
+	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
+	fd->num_entries = 1;
+
+	fd->entry[0].pixelcode = code;
+	fd->entry[0].stream = IMX219_STREAM_IMAGE;
+	fd->entry[0].bus.csi2.vc = 0;
+	fd->entry[0].bus.csi2.dt = imx219_format_bpp(code) == 8 ?
+		MIPI_CSI2_DT_RAW8 : MIPI_CSI2_DT_RAW10;
+
+	return 0;
+}
+
 static int imx219_init_state(struct v4l2_subdev *sd,
 			     struct v4l2_subdev_state *state)
 {
@@ -1113,6 +1160,7 @@ static const struct v4l2_subdev_pad_ops imx219_pad_ops = {
 	.set_fmt = imx219_set_pad_format,
 	.get_selection = imx219_get_selection,
 	.enum_frame_size = imx219_enum_frame_size,
+	.get_frame_desc = imx219_get_frame_desc,
 	.enable_streams = imx219_enable_streams,
 	.disable_streams = imx219_disable_streams,
 };
-- 
2.47.2

[PATCH v11 62/66] media: i2c: imx219: Add embedded data support

[Sakari Ailus]

From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

The IMX219 generates embedded data unconditionally. Report it as an
additional stream, with a new internal embedded data pad, and update
subdev operations accordingly.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Co-developed-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 174 ++++++++++++++++++++++++++++++++-----
 1 file changed, 153 insertions(+), 21 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index 8ed1febc7b21..d2d64a1644a9 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -148,6 +148,9 @@
 #define IMX219_PIXEL_ARRAY_WIDTH	3280U
 #define IMX219_PIXEL_ARRAY_HEIGHT	2464U
 
+/* Embedded metadata stream height */
+#define IMX219_EMBEDDED_DATA_HEIGHT	2U
+
 /* Mode : resolution and related config&values */
 struct imx219_mode {
 	/* Frame width */
@@ -339,11 +342,13 @@ static const struct imx219_mode supported_modes[] = {
 enum imx219_pad_ids {
 	IMX219_PAD_SOURCE = 0,
 	IMX219_PAD_IMAGE,
+	IMX219_PAD_EDATA,
 	IMX219_NUM_PADS,
 };
 
 enum imx219_stream_ids {
 	IMX219_STREAM_IMAGE,
+	IMX219_STREAM_EDATA,
 };
 
 struct imx219 {
@@ -697,6 +702,19 @@ static unsigned int imx219_format_bpp(u32 code)
 	}
 }
 
+/* Return the embedded data format corresponding to an image format. */
+static u32 imx219_format_edata(u32 code)
+{
+	switch (imx219_format_bpp(code)) {
+	case 8:
+		return MEDIA_BUS_FMT_META_8;
+
+	case 10:
+	default:
+		return MEDIA_BUS_FMT_META_10;
+	}
+}
+
 static int imx219_set_framefmt(struct imx219 *imx219,
 			       struct v4l2_subdev_state *state)
 {
@@ -758,6 +776,13 @@ static int imx219_enable_streams(struct v4l2_subdev *sd,
 	struct i2c_client *client = v4l2_get_subdevdata(&imx219->sd);
 	int ret;
 
+	/*
+	 * The image stream controls sensor streaming, as embedded data isn't
+	 * controllable independently.
+	 */
+	if (!(streams_mask & BIT(IMX219_STREAM_IMAGE)))
+		return 0;
+
 	ret = pm_runtime_resume_and_get(&client->dev);
 	if (ret < 0)
 		return ret;
@@ -816,6 +841,13 @@ static int imx219_disable_streams(struct v4l2_subdev *sd,
 	struct i2c_client *client = v4l2_get_subdevdata(&imx219->sd);
 	int ret;
 
+	/*
+	 * The image stream controls sensor streaming, as embedded data isn't
+	 * controllable independently.
+	 */
+	if (!(streams_mask & BIT(IMX219_STREAM_IMAGE)))
+		return 0;
+
 	/* set stream off register */
 	ret = cci_write(imx219->regmap, IMX219_REG_MODE_SELECT,
 			IMX219_MODE_STANDBY, NULL);
@@ -837,17 +869,33 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
 {
 	struct imx219 *imx219 = to_imx219(sd);
 
-	if (code->pad == IMX219_PAD_IMAGE) {
+	switch (code->pad) {
+	case IMX219_PAD_IMAGE:
 		/* The internal image pad is hardwired to the native format. */
 		if (code->index > 0)
 			return -EINVAL;
 
 		code->code = IMX219_NATIVE_FORMAT;
-	} else {
-		/*
-		 * On the source pad, the sensor supports multiple raw formats
-		 * with different bit depths.
-		 */
+		return 0;
+
+	case IMX219_PAD_EDATA:
+		if (code->index > 0)
+			return -EINVAL;
+
+		code->code = MEDIA_BUS_FMT_META_8;
+		return 0;
+
+	case IMX219_PAD_SOURCE:
+	default:
+		break;
+	}
+
+	/*
+	 * On the source pad, the sensor supports multiple image raw formats
+	 * with different bit depths. The embedded data format bit depth
+	 * follows the image stream.
+	 */
+	if (code->stream == IMX219_STREAM_IMAGE) {
 		u32 format;
 
 		if (code->index >= ARRAY_SIZE(imx219_mbus_formats) / 4)
@@ -855,6 +903,15 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
 
 		format = imx219_mbus_formats[code->index * 4];
 		code->code = imx219_get_format_code(imx219, format);
+	} else {
+		struct v4l2_mbus_framefmt *fmt;
+
+		if (code->index > 0)
+			return -EINVAL;
+
+		fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+						   IMX219_STREAM_EDATA);
+		code->code = fmt->code;
 	}
 
 	return 0;
@@ -866,7 +923,8 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
 {
 	struct imx219 *imx219 = to_imx219(sd);
 
-	if (fse->pad == IMX219_PAD_IMAGE) {
+	switch (fse->pad) {
+	case IMX219_PAD_IMAGE:
 		if (fse->code != IMX219_NATIVE_FORMAT || fse->index > 0)
 			return -EINVAL;
 
@@ -874,7 +932,24 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
 		fse->max_width = IMX219_NATIVE_WIDTH;
 		fse->min_height = IMX219_NATIVE_HEIGHT;
 		fse->max_height = IMX219_NATIVE_HEIGHT;
-	} else {
+		return 0;
+
+	case IMX219_PAD_EDATA:
+		if (fse->code != MEDIA_BUS_FMT_META_8 || fse->index > 0)
+			return -EINVAL;
+
+		fse->min_width = IMX219_NATIVE_WIDTH;
+		fse->max_width = IMX219_NATIVE_WIDTH;
+		fse->min_height = IMX219_EMBEDDED_DATA_HEIGHT;
+		fse->max_height = IMX219_EMBEDDED_DATA_HEIGHT;
+		return 0;
+
+	case IMX219_PAD_SOURCE:
+	default:
+		break;
+	}
+
+	if (fse->stream == IMX219_STREAM_IMAGE) {
 		if (fse->code != imx219_get_format_code(imx219, fse->code) ||
 		    fse->index >= ARRAY_SIZE(supported_modes))
 			return -EINVAL;
@@ -883,6 +958,21 @@ static int imx219_enum_frame_size(struct v4l2_subdev *sd,
 		fse->max_width = fse->min_width;
 		fse->min_height = supported_modes[fse->index].height;
 		fse->max_height = fse->min_height;
+	} else {
+		struct v4l2_mbus_framefmt *fmt;
+
+		fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+						   IMX219_STREAM_EDATA);
+		if (fse->code != fmt->code)
+			return -EINVAL;
+
+		if (fse->index > 0)
+			return -EINVAL;
+
+		fse->min_width = fmt->width;
+		fse->max_width = fmt->width;
+		fse->min_height = IMX219_EMBEDDED_DATA_HEIGHT;
+		fse->max_height = IMX219_EMBEDDED_DATA_HEIGHT;
 	}
 
 	return 0;
@@ -894,6 +984,7 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 {
 	struct imx219 *imx219 = to_imx219(sd);
 	const struct imx219_mode *mode;
+	struct v4l2_mbus_framefmt *ed_format;
 	struct v4l2_mbus_framefmt *format;
 	struct v4l2_rect *compose;
 	struct v4l2_rect *crop;
@@ -905,9 +996,9 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 
 	/*
 	 * The driver is mode-based, the format can be set on the source pad
-	 * only.
+	 * only, and only for the image streeam.
 	 */
-	if (fmt->pad != IMX219_PAD_SOURCE)
+	if (fmt->pad != IMX219_PAD_SOURCE || fmt->stream != IMX219_STREAM_IMAGE)
 		return v4l2_subdev_get_fmt(sd, state, fmt);
 
 	/*
@@ -974,6 +1065,20 @@ static int imx219_set_pad_format(struct v4l2_subdev *sd,
 					      IMX219_STREAM_IMAGE);
 	*format = fmt->format;
 
+	/*
+	 * Finally, update the formats on the sink and source sides of the
+	 * embedded data stream.
+	 */
+	ed_format = v4l2_subdev_state_get_format(state, IMX219_PAD_EDATA);
+	ed_format->code = imx219_format_edata(format->code);
+	ed_format->width = format->width;
+	ed_format->height = IMX219_EMBEDDED_DATA_HEIGHT;
+	ed_format->field = V4L2_FIELD_NONE;
+
+	format = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+					      IMX219_STREAM_EDATA);
+	*format = *ed_format;
+
 	if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
 		int exposure_max;
 		int exposure_def;
@@ -1033,6 +1138,13 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 {
 	struct v4l2_rect *compose;
 
+	/*
+	 * The embedded data stream doesn't support selection rectangles,
+	 * neither on the embedded data pad nor on the source pad.
+	 */
+	if (sel->pad == IMX219_PAD_EDATA || sel->stream != 0)
+		return -EINVAL;
+
 	switch (sel->target) {
 	case V4L2_SEL_TGT_NATIVE_SIZE:
 		if (sel->pad != IMX219_PAD_IMAGE)
@@ -1085,35 +1197,41 @@ static int imx219_get_selection(struct v4l2_subdev *sd,
 static int imx219_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
 				 struct v4l2_mbus_frame_desc *fd)
 {
-	const struct v4l2_mbus_framefmt *fmt;
 	struct v4l2_subdev_state *state;
-	u32 code;
+	u32 img_code;
+	u32 ed_code;
 
 	if (pad != IMX219_PAD_SOURCE)
 		return -EINVAL;
 
 	state = v4l2_subdev_lock_and_get_active_state(sd);
-	fmt = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
-					   IMX219_STREAM_IMAGE);
-	code = fmt->code;
+	img_code = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+						IMX219_STREAM_IMAGE)->code;
+	ed_code = v4l2_subdev_state_get_format(state, IMX219_PAD_SOURCE,
+					       IMX219_STREAM_EDATA)->code;
 	v4l2_subdev_unlock_state(state);
 
 	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
-	fd->num_entries = 1;
+	fd->num_entries = 2;
 
-	fd->entry[0].pixelcode = code;
+	fd->entry[0].pixelcode = img_code;
 	fd->entry[0].stream = IMX219_STREAM_IMAGE;
 	fd->entry[0].bus.csi2.vc = 0;
-	fd->entry[0].bus.csi2.dt = imx219_format_bpp(code) == 8 ?
+	fd->entry[0].bus.csi2.dt = imx219_format_bpp(img_code) == 8 ?
 		MIPI_CSI2_DT_RAW8 : MIPI_CSI2_DT_RAW10;
 
+	fd->entry[1].pixelcode = ed_code;
+	fd->entry[1].stream = IMX219_STREAM_EDATA;
+	fd->entry[1].bus.csi2.vc = 0;
+	fd->entry[1].bus.csi2.dt = MIPI_CSI2_DT_EMBEDDED_8B;
+
 	return 0;
 }
 
 static int imx219_init_state(struct v4l2_subdev *sd,
 			     struct v4l2_subdev_state *state)
 {
-	struct v4l2_subdev_route routes[1] = {
+	struct v4l2_subdev_route routes[] = {
 		{
 			.sink_pad = IMX219_PAD_IMAGE,
 			.sink_stream = 0,
@@ -1122,6 +1240,14 @@ static int imx219_init_state(struct v4l2_subdev *sd,
 			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE |
 				 V4L2_SUBDEV_ROUTE_FL_IMMUTABLE |
 				 V4L2_SUBDEV_ROUTE_FL_STATIC,
+		}, {
+			.sink_pad = IMX219_PAD_EDATA,
+			.sink_stream = 0,
+			.source_pad = IMX219_PAD_SOURCE,
+			.source_stream = IMX219_STREAM_EDATA,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE |
+				 V4L2_SUBDEV_ROUTE_FL_IMMUTABLE |
+				 V4L2_SUBDEV_ROUTE_FL_STATIC,
 		},
 	};
 	struct v4l2_subdev_krouting routing = {
@@ -1145,6 +1271,10 @@ static int imx219_init_state(struct v4l2_subdev *sd,
 	if (ret)
 		return ret;
 
+	/*
+	 * Set the image stream format on the source pad. This will be
+	 * propagated to all formats and selection rectangles internally.
+	 */
 	imx219_set_pad_format(sd, state, &fmt);
 
 	return 0;
@@ -1420,12 +1550,14 @@ static int imx219_probe(struct i2c_client *client)
 	/*
 	 * Initialize the pads. To preserve backward compatibility with
 	 * userspace that used the sensor before the introduction of the
-	 * internal image pad, the external source pad is numbered 0 and the
-	 * internal image pad numbered 1.
+	 * internal pads, the external source pad is numbered 0 and the internal
+	 * image and embedded data pads numbered 1 and 2 respectively.
 	 */
 	imx219->pads[IMX219_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
 	imx219->pads[IMX219_PAD_IMAGE].flags = MEDIA_PAD_FL_SINK
 					     | MEDIA_PAD_FL_INTERNAL;
+	imx219->pads[IMX219_PAD_EDATA].flags = MEDIA_PAD_FL_SINK
+					     | MEDIA_PAD_FL_INTERNAL;
 
 	ret = media_entity_pads_init(&imx219->sd.entity,
 				     ARRAY_SIZE(imx219->pads), imx219->pads);
-- 
2.47.2

[PATCH v11 63/66] media: imx219: Add support for generic raw formats

[Sakari Ailus]

Add support for colour pattern agnostic raw formats, including controls to
convey the colour pattern and related information.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/imx219.c | 47 +++++++++++++++++++++++++++++++++-----
 1 file changed, 41 insertions(+), 6 deletions(-)

diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c
index d2d64a1644a9..b0e1e02f5210 100644
--- a/drivers/media/i2c/imx219.c
+++ b/drivers/media/i2c/imx219.c
@@ -284,6 +284,12 @@ static const u32 imx219_mbus_formats[] = {
 	MEDIA_BUS_FMT_SBGGR8_1X8,
 };
 
+static const u32 imx219_mbus_formats_raw[] = {
+	IMX219_NATIVE_FORMAT,
+	MEDIA_BUS_FMT_RAW_10,
+	MEDIA_BUS_FMT_RAW_8,
+};
+
 /*
  * Initialisation delay between XCLR low->high and the moment when the sensor
  * can start capture (i.e. can leave software stanby) must be not less than:
@@ -386,6 +392,10 @@ static u32 imx219_get_format_code(struct imx219 *imx219, u32 code)
 {
 	unsigned int i;
 
+	if (code == MEDIA_BUS_FMT_RAW_10 ||
+	    code == MEDIA_BUS_FMT_RAW_8)
+		return code;
+
 	for (i = 0; i < ARRAY_SIZE(imx219_mbus_formats); i++)
 		if (imx219_mbus_formats[i] == code)
 			break;
@@ -650,6 +660,20 @@ static int imx219_init_controls(struct imx219 *imx219)
 		/* The "Solid color" pattern is white by default */
 	}
 
+	v4l2_ctrl_new_std(ctrl_hdlr, NULL, V4L2_CID_COLOR_PATTERN,
+			  V4L2_COLOR_PATTERN_RGGB, V4L2_COLOR_PATTERN_RGGB,
+			  1, V4L2_COLOR_PATTERN_RGGB);
+	v4l2_ctrl_new_std(ctrl_hdlr, NULL,
+			  V4L2_CID_COLOR_PATTERN_FLIP,
+			  0, V4L2_COLOR_PATTERN_FLIP_BOTH,
+			  0, V4L2_COLOR_PATTERN_FLIP_BOTH);
+	v4l2_ctrl_new_std(ctrl_hdlr, NULL, V4L2_CID_METADATA_LAYOUT,
+			  0, V4L2_METADATA_LAYOUT_CCS,
+			  1, V4L2_METADATA_LAYOUT_CCS);
+	v4l2_ctrl_new_std(ctrl_hdlr, NULL, V4L2_CID_CONFIG_MODEL,
+			  0, V4L2_CONFIG_MODEL_COMMON_RAW_SENSOR,
+			  0, V4L2_CONFIG_MODEL_COMMON_RAW_SENSOR);
+
 	if (ctrl_hdlr->error) {
 		ret = ctrl_hdlr->error;
 		dev_err_probe(&client->dev, ret, "Control init failed\n");
@@ -870,13 +894,19 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
 	struct imx219 *imx219 = to_imx219(sd);
 
 	switch (code->pad) {
-	case IMX219_PAD_IMAGE:
+	case IMX219_PAD_IMAGE: {
+		const u32 img_codes[] = {
+			IMX219_NATIVE_FORMAT,
+			MEDIA_BUS_FMT_RAW_10,
+		};
+
 		/* The internal image pad is hardwired to the native format. */
-		if (code->index > 0)
+		if (code->index > ARRAY_SIZE(img_codes))
 			return -EINVAL;
 
-		code->code = IMX219_NATIVE_FORMAT;
+		code->code = img_codes[code->index];
 		return 0;
+	}
 
 	case IMX219_PAD_EDATA:
 		if (code->index > 0)
@@ -898,10 +928,15 @@ static int imx219_enum_mbus_code(struct v4l2_subdev *sd,
 	if (code->stream == IMX219_STREAM_IMAGE) {
 		u32 format;
 
-		if (code->index >= ARRAY_SIZE(imx219_mbus_formats) / 4)
-			return -EINVAL;
+		if (code->index >= ARRAY_SIZE(imx219_mbus_formats) / 4) {
+			u32 idx = code->index - ARRAY_SIZE(imx219_mbus_formats);
+			if (idx >= ARRAY_SIZE(imx219_mbus_formats_raw))
+				return -EINVAL;
 
-		format = imx219_mbus_formats[code->index * 4];
+			format = imx219_mbus_formats_raw[idx];
+		} else {
+			format = imx219_mbus_formats[code->index * 4];
+		}
 		code->code = imx219_get_format_code(imx219, format);
 	} else {
 		struct v4l2_mbus_framefmt *fmt;
-- 
2.47.2

[PATCH v11 64/66] media: ccs: Add frame descriptor quirk

[Sakari Ailus]

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ccs/ccs-core.c  | 3 +++
 drivers/media/i2c/ccs/ccs-quirk.h | 7 +++++++
 2 files changed, 10 insertions(+)

diff --git a/drivers/media/i2c/ccs/ccs-core.c b/drivers/media/i2c/ccs/ccs-core.c
index 44e7c3a2f95a..ed63a8501723 100644
--- a/drivers/media/i2c/ccs/ccs-core.c
+++ b/drivers/media/i2c/ccs/ccs-core.c
@@ -2917,6 +2917,9 @@ static int ccs_get_frame_desc(struct v4l2_subdev *subdev, unsigned int pad,
 	struct v4l2_mbus_frame_desc_entry *entry = desc->entry;
 	struct v4l2_subdev_state *sd_state;
 
+	if (ccs_has_quirk(sensor, frame_desc))
+		return ccs_call_quirk(sensor, frame_desc, desc);
+
 	switch (sensor->hwcfg.csi_signalling_mode) {
 	case CCS_CSI_SIGNALING_MODE_CSI_2_DPHY:
 	case CCS_CSI_SIGNALING_MODE_CSI_2_CPHY:
diff --git a/drivers/media/i2c/ccs/ccs-quirk.h b/drivers/media/i2c/ccs/ccs-quirk.h
index 392c97109617..3e1d9eaa33fa 100644
--- a/drivers/media/i2c/ccs/ccs-quirk.h
+++ b/drivers/media/i2c/ccs/ccs-quirk.h
@@ -36,6 +36,7 @@ struct ccs_sensor;
  *			 access may be done by the caller (default read
  *			 value is zero), else negative error code on error
  * @flags: Quirk flags
+ * @frame_desc: Obtain the frame descriptor
  */
 struct ccs_quirk {
 	int (*limits)(struct ccs_sensor *sensor);
@@ -46,6 +47,8 @@ struct ccs_quirk {
 	int (*init)(struct ccs_sensor *sensor);
 	int (*reg_access)(struct ccs_sensor *sensor, bool write, u32 *reg,
 			  u32 *val);
+	int (*frame_desc)(struct ccs_sensor *sensor,
+			  struct v4l2_mbus_frame_desc *desc);
 	unsigned long flags;
 };
 
@@ -62,6 +65,10 @@ struct ccs_reg_8 {
 		.val = _val,		\
 	}
 
+#define ccs_has_quirk(sensor, _quirk)					\
+	((sensor)->minfo.quirk &&					\
+	 (sensor)->minfo.quirk->_quirk)
+
 #define ccs_call_quirk(sensor, _quirk, ...)				\
 	((sensor)->minfo.quirk &&					\
 	 (sensor)->minfo.quirk->_quirk ?				\
-- 
2.47.2

[PATCH v11 65/66] media: ipu6: Add support for luma-only formats

[Sakari Ailus]

Add support for luma-only formats.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c   | 3 +++
 drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c | 7 +++++++
 drivers/media/pci/intel/ipu6/ipu6-isys-video.c  | 6 ++++++
 3 files changed, 16 insertions(+)

diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
index d1fece6210ab..b6421d6b978a 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
@@ -34,14 +34,17 @@ static const u32 csi2_supported_codes[] = {
 	MEDIA_BUS_FMT_SGBRG10_1X10,
 	MEDIA_BUS_FMT_SGRBG10_1X10,
 	MEDIA_BUS_FMT_SRGGB10_1X10,
+	MEDIA_BUS_FMT_Y10_1X10,
 	MEDIA_BUS_FMT_SBGGR12_1X12,
 	MEDIA_BUS_FMT_SGBRG12_1X12,
 	MEDIA_BUS_FMT_SGRBG12_1X12,
 	MEDIA_BUS_FMT_SRGGB12_1X12,
+	MEDIA_BUS_FMT_Y12_1X12,
 	MEDIA_BUS_FMT_SBGGR8_1X8,
 	MEDIA_BUS_FMT_SGBRG8_1X8,
 	MEDIA_BUS_FMT_SGRBG8_1X8,
 	MEDIA_BUS_FMT_SRGGB8_1X8,
+	MEDIA_BUS_FMT_Y8_1X8,
 	MEDIA_BUS_FMT_META_8,
 	MEDIA_BUS_FMT_META_10,
 	MEDIA_BUS_FMT_META_12,
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c b/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c
index 463a0adf9e13..5db59cec821e 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c
@@ -31,18 +31,21 @@ unsigned int ipu6_isys_mbus_code_to_bpp(u32 code)
 	case MEDIA_BUS_FMT_SGBRG12_1X12:
 	case MEDIA_BUS_FMT_SGRBG12_1X12:
 	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_Y12_1X12:
 	case MEDIA_BUS_FMT_META_12:
 		return 12;
 	case MEDIA_BUS_FMT_SBGGR10_1X10:
 	case MEDIA_BUS_FMT_SGBRG10_1X10:
 	case MEDIA_BUS_FMT_SGRBG10_1X10:
 	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_Y10_1X10:
 	case MEDIA_BUS_FMT_META_10:
 		return 10;
 	case MEDIA_BUS_FMT_SBGGR8_1X8:
 	case MEDIA_BUS_FMT_SGBRG8_1X8:
 	case MEDIA_BUS_FMT_SGRBG8_1X8:
 	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_Y8_1X8:
 	case MEDIA_BUS_FMT_META_8:
 		return 8;
 	default:
@@ -65,21 +68,25 @@ unsigned int ipu6_isys_mbus_code_to_mipi(u32 code)
 	case MEDIA_BUS_FMT_SGBRG16_1X16:
 	case MEDIA_BUS_FMT_SGRBG16_1X16:
 	case MEDIA_BUS_FMT_SRGGB16_1X16:
+	case MEDIA_BUS_FMT_Y16_1X16:
 		return MIPI_CSI2_DT_RAW16;
 	case MEDIA_BUS_FMT_SBGGR12_1X12:
 	case MEDIA_BUS_FMT_SGBRG12_1X12:
 	case MEDIA_BUS_FMT_SGRBG12_1X12:
 	case MEDIA_BUS_FMT_SRGGB12_1X12:
+	case MEDIA_BUS_FMT_Y12_1X12:
 		return MIPI_CSI2_DT_RAW12;
 	case MEDIA_BUS_FMT_SBGGR10_1X10:
 	case MEDIA_BUS_FMT_SGBRG10_1X10:
 	case MEDIA_BUS_FMT_SGRBG10_1X10:
 	case MEDIA_BUS_FMT_SRGGB10_1X10:
+	case MEDIA_BUS_FMT_Y10_1X10:
 		return MIPI_CSI2_DT_RAW10;
 	case MEDIA_BUS_FMT_SBGGR8_1X8:
 	case MEDIA_BUS_FMT_SGBRG8_1X8:
 	case MEDIA_BUS_FMT_SGRBG8_1X8:
 	case MEDIA_BUS_FMT_SRGGB8_1X8:
+	case MEDIA_BUS_FMT_Y8_1X8:
 		return MIPI_CSI2_DT_RAW8;
 	case MEDIA_BUS_FMT_META_8:
 	case MEDIA_BUS_FMT_META_10:
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
index e1306b9510e7..9a6b4736cd67 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
@@ -85,6 +85,12 @@ const struct ipu6_isys_pixelformat ipu6_isys_pfmts[] = {
 	  IPU6_FW_ISYS_FRAME_FORMAT_RGB565 },
 	{ V4L2_PIX_FMT_BGR24, 24, 24, MEDIA_BUS_FMT_RGB888_1X24,
 	  IPU6_FW_ISYS_FRAME_FORMAT_RGBA888 },
+	{ V4L2_PIX_FMT_GREY, 8, 8, MEDIA_BUS_FMT_Y8_1X8,
+	  IPU6_FW_ISYS_FRAME_FORMAT_RAW8 },
+	{ V4L2_PIX_FMT_Y10, 10, 10, MEDIA_BUS_FMT_Y10_1X10,
+	  IPU6_FW_ISYS_FRAME_FORMAT_RAW10 },
+	{ V4L2_PIX_FMT_Y12, 12, 12, MEDIA_BUS_FMT_Y12_1X12,
+	  IPU6_FW_ISYS_FRAME_FORMAT_RAW12 },
 	{ V4L2_META_FMT_GENERIC_8, 8, 8, MEDIA_BUS_FMT_META_8,
 	  IPU6_FW_ISYS_FRAME_FORMAT_RAW8, true },
 	{ V4L2_META_FMT_GENERIC_CSI2_10, 10, 10, MEDIA_BUS_FMT_META_10,
-- 
2.47.2

[PATCH v11 66/66] media: ipu6: Add support for raw CFA-agnostic formats

[Sakari Ailus]

Add support for CFA-agnostic mbus codes and V4L2 pixelformats.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c   | 3 +++
 drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c | 3 +++
 drivers/media/pci/intel/ipu6/ipu6-isys-video.c  | 6 ++++++
 3 files changed, 12 insertions(+)

diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
index b6421d6b978a..21a4f130bb34 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
@@ -35,16 +35,19 @@ static const u32 csi2_supported_codes[] = {
 	MEDIA_BUS_FMT_SGRBG10_1X10,
 	MEDIA_BUS_FMT_SRGGB10_1X10,
 	MEDIA_BUS_FMT_Y10_1X10,
+	MEDIA_BUS_FMT_RAW_10,
 	MEDIA_BUS_FMT_SBGGR12_1X12,
 	MEDIA_BUS_FMT_SGBRG12_1X12,
 	MEDIA_BUS_FMT_SGRBG12_1X12,
 	MEDIA_BUS_FMT_SRGGB12_1X12,
 	MEDIA_BUS_FMT_Y12_1X12,
+	MEDIA_BUS_FMT_RAW_12,
 	MEDIA_BUS_FMT_SBGGR8_1X8,
 	MEDIA_BUS_FMT_SGBRG8_1X8,
 	MEDIA_BUS_FMT_SGRBG8_1X8,
 	MEDIA_BUS_FMT_SRGGB8_1X8,
 	MEDIA_BUS_FMT_Y8_1X8,
+	MEDIA_BUS_FMT_RAW_8,
 	MEDIA_BUS_FMT_META_8,
 	MEDIA_BUS_FMT_META_10,
 	MEDIA_BUS_FMT_META_12,
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c b/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c
index 5db59cec821e..407db2ebe6ee 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-subdev.c
@@ -32,6 +32,7 @@ unsigned int ipu6_isys_mbus_code_to_bpp(u32 code)
 	case MEDIA_BUS_FMT_SGRBG12_1X12:
 	case MEDIA_BUS_FMT_SRGGB12_1X12:
 	case MEDIA_BUS_FMT_Y12_1X12:
+	case MEDIA_BUS_FMT_RAW_12:
 	case MEDIA_BUS_FMT_META_12:
 		return 12;
 	case MEDIA_BUS_FMT_SBGGR10_1X10:
@@ -39,6 +40,7 @@ unsigned int ipu6_isys_mbus_code_to_bpp(u32 code)
 	case MEDIA_BUS_FMT_SGRBG10_1X10:
 	case MEDIA_BUS_FMT_SRGGB10_1X10:
 	case MEDIA_BUS_FMT_Y10_1X10:
+	case MEDIA_BUS_FMT_RAW_10:
 	case MEDIA_BUS_FMT_META_10:
 		return 10;
 	case MEDIA_BUS_FMT_SBGGR8_1X8:
@@ -46,6 +48,7 @@ unsigned int ipu6_isys_mbus_code_to_bpp(u32 code)
 	case MEDIA_BUS_FMT_SGRBG8_1X8:
 	case MEDIA_BUS_FMT_SRGGB8_1X8:
 	case MEDIA_BUS_FMT_Y8_1X8:
+	case MEDIA_BUS_FMT_RAW_8:
 	case MEDIA_BUS_FMT_META_8:
 		return 8;
 	default:
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
index 9a6b4736cd67..362608f89fc5 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
@@ -91,6 +91,12 @@ const struct ipu6_isys_pixelformat ipu6_isys_pfmts[] = {
 	  IPU6_FW_ISYS_FRAME_FORMAT_RAW10 },
 	{ V4L2_PIX_FMT_Y12, 12, 12, MEDIA_BUS_FMT_Y12_1X12,
 	  IPU6_FW_ISYS_FRAME_FORMAT_RAW12 },
+	{ V4L2_PIX_FMT_RAW_8, 8, 8, MEDIA_BUS_FMT_RAW_8,
+	  IPU6_FW_ISYS_FRAME_FORMAT_RAW8 },
+	{ V4L2_PIX_FMT_RAW_CSI2_10, 10, 10, MEDIA_BUS_FMT_RAW_10,
+	  IPU6_FW_ISYS_FRAME_FORMAT_RAW10 },
+	{ V4L2_PIX_FMT_RAW_CSI2_12, 12, 12, MEDIA_BUS_FMT_RAW_12,
+	  IPU6_FW_ISYS_FRAME_FORMAT_RAW12 },
 	{ V4L2_META_FMT_GENERIC_8, 8, 8, MEDIA_BUS_FMT_META_8,
 	  IPU6_FW_ISYS_FRAME_FORMAT_RAW8, true },
 	{ V4L2_META_FMT_GENERIC_CSI2_10, 10, 10, MEDIA_BUS_FMT_META_10,
-- 
2.47.2