[RFC v5 00/15] Sub-device configuration models

[RFC v5 00/15] Sub-device configuration models

[Sakari Ailus]

Hello everyone,

I've been recently working (with others) on sub-device streams support as
well as on internal pads. The two can be used to make sub-device
configuration more versatile.

At the same time, the added interfaces are much more useful if we require
specific semantics of those interfaces, so that the user space knows
exactly what e.g. a given selection target signifies. However, as the same
selection rectangle could be used for a different purpose on a non-raw
sensor device, we need a way to tell how should the user space determine
how to use a given interface.

I'm proposing to solve this problem by introducing sub-device
configuration models, and by the common raw sensor model, also present in
this patchset, in particular.

This has been (and will, for some time, continue to be) the reason why I
have reviewed few sensor driver related patches lately. As we're
introducing a new interface, it's beneficial to be able to use that
interface right from the start, rather than trying to later on offer
compatibility support, which is almost always a fair amount of work with
less than desirable results in the driver.

With this solved, I believe we can enable the use of the streams UAPI.

Comments are welcome.

RFC v4 is <20241220132419.1027206-1-sakari.ailus@linux.intel.com> on LMML.

RFC v3 is <20241129095142.87196-1-sakari.ailus@linux.intel.com> on LMML.

RFC v2 (labelled as PATCH) is
<20241122100633.8971-1-sakari.ailus@linux.intel.com> on LMML.

RFC v1 is <20241011075535.588140-1-sakari.ailus@linux.intel.com> on LMML.

We may end up postponing merging three last patches (those adding
controls) until current control values are moved to sub-device (and video
device) state. Discussion is needed on the topic.

The compiled documentation can be found here
<URL:https://www.retiisi.eu/~sailus/v4l2/tmp/common-raw/output/userspace-api/media/v4l/dev-subdev.html#media-subdev-config-model-common-raw-sensor>.

since RFC v4:

- Document the use of luma formats for generic raw data on both mbus codes
  and V4L2 pixelformats, with new controls V4L2_CID_COLOUR_PATTERN and
  V4L2_CID_COLOUR_PATTERN_FLIP determine the pattern in different flipping
  configurations. This enables the use of generic pattern-agnostic formats
  whereas controls are used to express colour components and their order.

- Add a patch that reworks the metadata format 4ccs. These haven't been in
  use in any case so far.

- Improve documentation column widths.

since RFC v3:

- Fix binning control name in documentation.

- Note V4L2_CID_CONFIG_MODEL is read-only.

- Clarify wording in config model documentation based on Jacopo's
  comments.

- Clarify post-scaler digital crop behaviour.

since RFC v2:

- Remove the CROP_BOUNDS rectangle on the image data sink pad. It's
  redundant.

- Remove scaling from the sink compose rectangle, leaving binning and
  sub-sampling.

- Introduce scaling as well as post-scaler digital crop on source pad
  (image stream). This is a new patch, preceding control changes.

- Use monospace font for selection targets.

- Improvements in referring to previous configuration steps in text
  description.

since RFC v1:

- Rework the documentation according to the comments (spelling fixes,
  alignment mostly).
  
- Include separate patches for binning and sub-sampling controls.

- Allow binning, scaling and scaling to be configured using the compose
  rectangle. This will change to just scaling in the three last patches.

Sakari Ailus (15):
  media: Documentation: Rework embedded data documentation
  media: Documentation: Add a hyphen to list-based
  media: Documentation: Reword split of sensor driver to two classes
  media: Documentation: Add subdev configuration models, raw sensor
    model
  media: Documentation: Add scaling and post-scaler crop for common raw
  media: uapi: Add V4L2_CID_CONFIG_MODEL control
  media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour
    patterns
  media: uapi: Correct generic CSI-2 metadata format 4cc
  media: uapi: Documentation: Improve column width hints for examples
  media: v4l: uapi: Add a control for colour pattern flipping effect
  media: Documentation: Document luma-only mbus codes and CFA for
    cameras
  media: uapi: Documentation: Use luma formats with CFA pattern control
  media: uapi: Add V4L2_CID_BINNING control for binning configuration
  media: uapi: Add controls for sub-sampling configuration
  media: Documentation: Add binning and sub-sampling controls

 .../media/drivers/camera-sensor.rst           |  66 ++-
 .../media/v4l/common-raw-sensor.dia           | 441 ++++++++++++++++++
 .../media/v4l/common-raw-sensor.svg           | 134 ++++++
 .../userspace-api/media/v4l/dev-subdev.rst    |   2 +
 .../media/v4l/ext-ctrls-camera.rst            |  40 ++
 .../media/v4l/ext-ctrls-image-process.rst     |   4 +
 .../media/v4l/ext-ctrls-image-source.rst      |  22 +
 .../media/v4l/metafmt-generic.rst             |  10 +-
 .../media/v4l/pixfmt-yuv-luma.rst             |   5 +
 .../media/v4l/subdev-config-model.rst         | 233 +++++++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  12 +
 include/uapi/linux/v4l2-controls.h            |  27 ++
 include/uapi/linux/videodev2.h                |  12 +-
 13 files changed, 983 insertions(+), 25 deletions(-)
 create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.dia
 create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.svg
 create mode 100644 Documentation/userspace-api/media/v4l/subdev-config-model.rst

-- 
2.39.5

[RFC v5 01/15] media: Documentation: Rework embedded data documentation

[Sakari Ailus]

Rework embedded data documentation by removing the reference to the pixel
data stream. The specific documentation of the embedded data interface
will be elsewhere, in the near future either CCS or common raw camera
sensor model documentation.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 .../userspace-api/media/drivers/camera-sensor.rst   | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index dc415b8f6c8e..8e1083417ae1 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -111,13 +111,12 @@ the sensor configuration for the captured frame back to the host. While CSI-2 is
 the most common data interface used by such sensors, embedded data can be
 available on other interfaces as well.
 
-Such sensors expose two internal sink pads (pads that have both the
-``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and ``MEDIA_PAD_FL_INTERNAL
-<MEDIA-PAD-FL-INTERNAL>`` flags set) to model the source of the image and
-embedded data streams. Both of these pads produces a single stream, and the
-sub-device routes those streams to the external (source) pad. If the sub-device
-driver supports disabling embedded data, this can be done by disabling the
-embedded data route via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL.
+Embedded data support is indicated by the precence of an internal sink pad (pad
+that has both the ``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and
+``MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`` flags set) with a metadata
+format to model the embedded data stream. If the sub-device driver supports
+disabling embedded data, this can be done by disabling the embedded data route
+via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL.
 
 In general, changing the embedded data format from the driver-configured values
 is not supported. The height of the metadata is device-specific and the width
-- 
2.39.5

[RFC v5 02/15] media: Documentation: Add a hyphen to list-based

[Sakari Ailus]

Add a hyphen to list-based for uniform spelling in camera-sensor.rst.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 Documentation/userspace-api/media/drivers/camera-sensor.rst | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index 8e1083417ae1..9a1e8aa9fc77 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -26,10 +26,10 @@ of cropping and scaling operations from the device's pixel array's size.
 
 An example of such a driver is the CCS driver.
 
-Register list based drivers
+Register list-based drivers
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Register list based drivers generally, instead of able to configure the device
+Register list-based drivers generally, instead of able to configure the device
 they control based on user requests, are limited to a number of preset
 configurations that combine a number of different parameters that on hardware
 level are independent. How a driver picks such configuration is based on the
@@ -67,7 +67,7 @@ is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
 the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
 sub-device. The unit of that control is pixels per second.
 
-Register list based drivers need to implement read-only sub-device nodes for the
+Register list-based drivers need to implement read-only sub-device nodes for the
 purpose. Devices that are not register list based need these to configure the
 device's internal processing pipeline.
 
-- 
2.39.5

[RFC v5 03/15] media: Documentation: Reword split of sensor driver to two classes

[Sakari Ailus]

The sensor drivers do not configure the output size of the sensors but the
entire internal pipeline. Reflect this in the documentation.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 .../userspace-api/media/drivers/camera-sensor.rst      | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index 9a1e8aa9fc77..bc55c861fb69 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -10,11 +10,13 @@ used to control the camera sensor drivers.
 
 You may also find :ref:`media_writing_camera_sensor_drivers` useful.
 
-Frame size
-----------
+Sensor internal pipeline configuration
+--------------------------------------
 
-There are two distinct ways to configure the frame size produced by camera
-sensors.
+Camera sensors have an internal processing pipeline including cropping and
+binning functionality. The sensor drivers belong to two distinct classes, freely
+configurable and register list-based drivers, depending on how the driver
+configures this functionality.
 
 Freely configurable camera sensor drivers
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- 
2.39.5

[RFC v5 04/15] media: Documentation: Add subdev configuration models, raw sensor model

[Sakari Ailus]

Sub-device configuration models define what V4L2 API elements are
available on a compliant sub-device and how do they behave.

The patch also adds a model for common raw sensors.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 .../media/drivers/camera-sensor.rst           |   4 +
 .../media/v4l/common-raw-sensor.dia           | 441 ++++++++++++++++++
 .../media/v4l/common-raw-sensor.svg           | 134 ++++++
 .../userspace-api/media/v4l/dev-subdev.rst    |   2 +
 .../media/v4l/subdev-config-model.rst         | 209 +++++++++
 5 files changed, 790 insertions(+)
 create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.dia
 create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.svg
 create mode 100644 Documentation/userspace-api/media/v4l/subdev-config-model.rst

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index bc55c861fb69..5bc4c79d230c 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -18,6 +18,8 @@ binning functionality. The sensor drivers belong to two distinct classes, freely
 configurable and register list-based drivers, depending on how the driver
 configures this functionality.
 
+Also see :ref:`media_subdev_config_model_common_raw_sensor`.
+
 Freely configurable camera sensor drivers
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -105,6 +107,8 @@ values programmed by the register sequences. The default values of these
 controls shall be 0 (disabled). Especially these controls shall not be inverted,
 independently of the sensor's mounting rotation.
 
+.. _media_using_camera_sensor_drivers_embedded_data:
+
 Embedded data
 -------------
 
diff --git a/Documentation/userspace-api/media/v4l/common-raw-sensor.dia b/Documentation/userspace-api/media/v4l/common-raw-sensor.dia
new file mode 100644
index 000000000000..aa927527eae3
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/common-raw-sensor.dia
@@ -0,0 +1,441 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/">
+  <dia:diagramdata>
+    <dia:attribute name="background">
+      <dia:color val="#ffffffff"/>
+    </dia:attribute>
+    <dia:attribute name="pagebreak">
+      <dia:color val="#000099ff"/>
+    </dia:attribute>
+    <dia:attribute name="paper">
+      <dia:composite type="paper">
+        <dia:attribute name="name">
+          <dia:string>#A4#</dia:string>
+        </dia:attribute>
+        <dia:attribute name="tmargin">
+          <dia:real val="2.8222"/>
+        </dia:attribute>
+        <dia:attribute name="bmargin">
+          <dia:real val="2.8222"/>
+        </dia:attribute>
+        <dia:attribute name="lmargin">
+          <dia:real val="2.8222"/>
+        </dia:attribute>
+        <dia:attribute name="rmargin">
+          <dia:real val="2.8222"/>
+        </dia:attribute>
+        <dia:attribute name="is_portrait">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="scaling">
+          <dia:real val="1"/>
+        </dia:attribute>
+        <dia:attribute name="fitto">
+          <dia:boolean val="false"/>
+        </dia:attribute>
+      </dia:composite>
+    </dia:attribute>
+    <dia:attribute name="grid">
+      <dia:composite type="grid">
+        <dia:attribute name="dynamic">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="width_x">
+          <dia:real val="1"/>
+        </dia:attribute>
+        <dia:attribute name="width_y">
+          <dia:real val="1"/>
+        </dia:attribute>
+        <dia:attribute name="visible_x">
+          <dia:int val="1"/>
+        </dia:attribute>
+        <dia:attribute name="visible_y">
+          <dia:int val="1"/>
+        </dia:attribute>
+        <dia:composite type="color"/>
+      </dia:composite>
+    </dia:attribute>
+    <dia:attribute name="color">
+      <dia:color val="#d8e5e5ff"/>
+    </dia:attribute>
+    <dia:attribute name="guides"/>
+    <dia:attribute name="guide_color">
+      <dia:color val="#00ff00ff"/>
+    </dia:attribute>
+    <dia:attribute name="display">
+      <dia:composite type="display">
+        <dia:attribute name="antialiased">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="snap-to-grid">
+          <dia:boolean val="false"/>
+        </dia:attribute>
+        <dia:attribute name="snap-to-guides">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="snap-to-object">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="show-grid">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="show-guides">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+        <dia:attribute name="show-connection-points">
+          <dia:boolean val="true"/>
+        </dia:attribute>
+      </dia:composite>
+    </dia:attribute>
+  </dia:diagramdata>
+  <dia:layer name="Background" visible="true" connectable="false"/>
+  <dia:layer name="Background" visible="true" connectable="false"/>
+  <dia:layer name="Background" visible="true" connectable="true" active="true">
+    <dia:object type="Standard - Box" version="0" id="O0">
+      <dia:attribute name="obj_pos">
+        <dia:point val="16.35,11.5"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="16.3,11.45;25.8,18.45"/>
+      </dia:attribute>
+      <dia:attribute name="elem_corner">
+        <dia:point val="16.35,11.5"/>
+      </dia:attribute>
+      <dia:attribute name="elem_width">
+        <dia:real val="9.4000000000000021"/>
+      </dia:attribute>
+      <dia:attribute name="elem_height">
+        <dia:real val="6.9000000000000021"/>
+      </dia:attribute>
+      <dia:attribute name="show_background">
+        <dia:boolean val="true"/>
+      </dia:attribute>
+    </dia:object>
+    <dia:object type="Geometric - Perfect Circle" version="1" id="O1">
+      <dia:attribute name="obj_pos">
+        <dia:point val="25.5627,14.578"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="25.4627,14.478;26.0191,15.0344"/>
+      </dia:attribute>
+      <dia:attribute name="meta">
+        <dia:composite type="dict"/>
+      </dia:attribute>
+      <dia:attribute name="elem_corner">
+        <dia:point val="25.5627,14.578"/>
+      </dia:attribute>
+      <dia:attribute name="elem_width">
+        <dia:real val="0.35638032780853468"/>
+      </dia:attribute>
+      <dia:attribute name="elem_height">
+        <dia:real val="0.35638032780853468"/>
+      </dia:attribute>
+      <dia:attribute name="line_width">
+        <dia:real val="0.10000000000000001"/>
+      </dia:attribute>
+      <dia:attribute name="line_colour">
+        <dia:color val="#000000ff"/>
+      </dia:attribute>
+      <dia:attribute name="fill_colour">
+        <dia:color val="#ffffffff"/>
+      </dia:attribute>
+      <dia:attribute name="show_background">
+        <dia:boolean val="true"/>
+      </dia:attribute>
+      <dia:attribute name="line_style">
+        <dia:enum val="0"/>
+        <dia:real val="1"/>
+      </dia:attribute>
+      <dia:attribute name="flip_horizontal">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="flip_vertical">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="subscale">
+        <dia:real val="1"/>
+      </dia:attribute>
+    </dia:object>
+    <dia:object type="Geometric - Perfect Circle" version="1" id="O2">
+      <dia:attribute name="obj_pos">
+        <dia:point val="17.7702,13.2656"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="17.6702,13.1656;18.24,13.7354"/>
+      </dia:attribute>
+      <dia:attribute name="meta">
+        <dia:composite type="dict"/>
+      </dia:attribute>
+      <dia:attribute name="elem_corner">
+        <dia:point val="17.7702,13.2656"/>
+      </dia:attribute>
+      <dia:attribute name="elem_width">
+        <dia:real val="0.36980024191863681"/>
+      </dia:attribute>
+      <dia:attribute name="elem_height">
+        <dia:real val="0.36980024191863681"/>
+      </dia:attribute>
+      <dia:attribute name="line_width">
+        <dia:real val="0.10000000000000001"/>
+      </dia:attribute>
+      <dia:attribute name="line_colour">
+        <dia:color val="#000000ff"/>
+      </dia:attribute>
+      <dia:attribute name="fill_colour">
+        <dia:color val="#ffffffff"/>
+      </dia:attribute>
+      <dia:attribute name="show_background">
+        <dia:boolean val="true"/>
+      </dia:attribute>
+      <dia:attribute name="line_style">
+        <dia:enum val="0"/>
+        <dia:real val="1"/>
+      </dia:attribute>
+      <dia:attribute name="flip_horizontal">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="flip_vertical">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="subscale">
+        <dia:real val="1"/>
+      </dia:attribute>
+    </dia:object>
+    <dia:object type="Geometric - Perfect Circle" version="1" id="O3">
+      <dia:attribute name="obj_pos">
+        <dia:point val="17.7464,16.2056"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="17.6464,16.1056;18.2162,16.6754"/>
+      </dia:attribute>
+      <dia:attribute name="meta">
+        <dia:composite type="dict"/>
+      </dia:attribute>
+      <dia:attribute name="elem_corner">
+        <dia:point val="17.7464,16.2056"/>
+      </dia:attribute>
+      <dia:attribute name="elem_width">
+        <dia:real val="0.36980024191863681"/>
+      </dia:attribute>
+      <dia:attribute name="elem_height">
+        <dia:real val="0.36980024191863681"/>
+      </dia:attribute>
+      <dia:attribute name="line_width">
+        <dia:real val="0.10000000000000001"/>
+      </dia:attribute>
+      <dia:attribute name="line_colour">
+        <dia:color val="#000000ff"/>
+      </dia:attribute>
+      <dia:attribute name="fill_colour">
+        <dia:color val="#ffffffff"/>
+      </dia:attribute>
+      <dia:attribute name="show_background">
+        <dia:boolean val="true"/>
+      </dia:attribute>
+      <dia:attribute name="line_style">
+        <dia:enum val="0"/>
+        <dia:real val="1"/>
+      </dia:attribute>
+      <dia:attribute name="flip_horizontal">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="flip_vertical">
+        <dia:boolean val="false"/>
+      </dia:attribute>
+      <dia:attribute name="subscale">
+        <dia:real val="1"/>
+      </dia:attribute>
+    </dia:object>
+    <dia:object type="Standard - Line" version="0" id="O4">
+      <dia:attribute name="obj_pos">
+        <dia:point val="18.1609,16.3413"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="18.1016,14.5712;25.6221,16.4007"/>
+      </dia:attribute>
+      <dia:attribute name="conn_endpoints">
+        <dia:point val="18.1609,16.3413"/>
+        <dia:point val="25.5627,14.7562"/>
+      </dia:attribute>
+      <dia:attribute name="numcp">
+        <dia:int val="1"/>
+      </dia:attribute>
+      <dia:attribute name="line_style">
+        <dia:enum val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow">
+        <dia:enum val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_length">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_width">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:connections>
+        <dia:connection handle="0" to="O3" connection="8"/>
+        <dia:connection handle="1" to="O1" connection="2"/>
+      </dia:connections>
+    </dia:object>
+    <dia:object type="Standard - Line" version="0" id="O5">
+      <dia:attribute name="obj_pos">
+        <dia:point val="18.14,13.4505"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="18.0821,13.3926;25.6206,14.9674"/>
+      </dia:attribute>
+      <dia:attribute name="conn_endpoints">
+        <dia:point val="18.14,13.4505"/>
+        <dia:point val="25.5627,14.7562"/>
+      </dia:attribute>
+      <dia:attribute name="numcp">
+        <dia:int val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow">
+        <dia:enum val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_length">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_width">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:connections>
+        <dia:connection handle="0" to="O2" connection="3"/>
+        <dia:connection handle="1" to="O1" connection="2"/>
+      </dia:connections>
+    </dia:object>
+    <dia:object type="Standard - Line" version="0" id="O6">
+      <dia:attribute name="obj_pos">
+        <dia:point val="25.919,14.7562"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="25.8686,14.3879;31.0068,15.0497"/>
+      </dia:attribute>
+      <dia:attribute name="conn_endpoints">
+        <dia:point val="25.919,14.7562"/>
+        <dia:point val="30.9564,14.7131"/>
+      </dia:attribute>
+      <dia:attribute name="numcp">
+        <dia:int val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow">
+        <dia:enum val="1"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_length">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:attribute name="end_arrow_width">
+        <dia:real val="0.5"/>
+      </dia:attribute>
+      <dia:connections>
+        <dia:connection handle="0" to="O1" connection="3"/>
+      </dia:connections>
+    </dia:object>
+    <dia:object type="Standard - Text" version="1" id="O7">
+      <dia:attribute name="obj_pos">
+        <dia:point val="17.9551,13.2656"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="17.9551,12.5181;22.7051,13.2656"/>
+      </dia:attribute>
+      <dia:attribute name="text">
+        <dia:composite type="text">
+          <dia:attribute name="string">
+            <dia:string>#image data (1)#</dia:string>
+          </dia:attribute>
+          <dia:attribute name="font">
+            <dia:font family="sans" style="0" name="Helvetica"/>
+          </dia:attribute>
+          <dia:attribute name="height">
+            <dia:real val="0.80000000000000004"/>
+          </dia:attribute>
+          <dia:attribute name="pos">
+            <dia:point val="17.9551,13.1131"/>
+          </dia:attribute>
+          <dia:attribute name="color">
+            <dia:color val="#000000ff"/>
+          </dia:attribute>
+          <dia:attribute name="alignment">
+            <dia:enum val="0"/>
+          </dia:attribute>
+        </dia:composite>
+      </dia:attribute>
+      <dia:attribute name="valign">
+        <dia:enum val="1"/>
+      </dia:attribute>
+      <dia:connections>
+        <dia:connection handle="0" to="O2" connection="1"/>
+      </dia:connections>
+    </dia:object>
+    <dia:object type="Standard - Text" version="1" id="O8">
+      <dia:attribute name="obj_pos">
+        <dia:point val="17.9313,16.5754"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="17.9313,16.5754;24.1238,17.3229"/>
+      </dia:attribute>
+      <dia:attribute name="text">
+        <dia:composite type="text">
+          <dia:attribute name="string">
+            <dia:string>#embedded data (2)#</dia:string>
+          </dia:attribute>
+          <dia:attribute name="font">
+            <dia:font family="sans" style="0" name="Helvetica"/>
+          </dia:attribute>
+          <dia:attribute name="height">
+            <dia:real val="0.80000000000000004"/>
+          </dia:attribute>
+          <dia:attribute name="pos">
+            <dia:point val="17.9313,17.1704"/>
+          </dia:attribute>
+          <dia:attribute name="color">
+            <dia:color val="#000000ff"/>
+          </dia:attribute>
+          <dia:attribute name="alignment">
+            <dia:enum val="0"/>
+          </dia:attribute>
+        </dia:composite>
+      </dia:attribute>
+      <dia:attribute name="valign">
+        <dia:enum val="0"/>
+      </dia:attribute>
+      <dia:connections>
+        <dia:connection handle="0" to="O3" connection="0"/>
+      </dia:connections>
+    </dia:object>
+    <dia:object type="Standard - Text" version="1" id="O9">
+      <dia:attribute name="obj_pos">
+        <dia:point val="26.1,14.125"/>
+      </dia:attribute>
+      <dia:attribute name="obj_bb">
+        <dia:rectangle val="26.1,13.53;30.7475,14.2775"/>
+      </dia:attribute>
+      <dia:attribute name="text">
+        <dia:composite type="text">
+          <dia:attribute name="string">
+            <dia:string>#source pad (0)#</dia:string>
+          </dia:attribute>
+          <dia:attribute name="font">
+            <dia:font family="sans" style="0" name="Helvetica"/>
+          </dia:attribute>
+          <dia:attribute name="height">
+            <dia:real val="0.80000000000000004"/>
+          </dia:attribute>
+          <dia:attribute name="pos">
+            <dia:point val="26.1,14.125"/>
+          </dia:attribute>
+          <dia:attribute name="color">
+            <dia:color val="#000000ff"/>
+          </dia:attribute>
+          <dia:attribute name="alignment">
+            <dia:enum val="0"/>
+          </dia:attribute>
+        </dia:composite>
+      </dia:attribute>
+      <dia:attribute name="valign">
+        <dia:enum val="3"/>
+      </dia:attribute>
+    </dia:object>
+  </dia:layer>
+</dia:diagram>
diff --git a/Documentation/userspace-api/media/v4l/common-raw-sensor.svg b/Documentation/userspace-api/media/v4l/common-raw-sensor.svg
new file mode 100644
index 000000000000..1d6055da2519
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/common-raw-sensor.svg
@@ -0,0 +1,134 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="296pt" height="142pt" viewBox="0 0 296 142" version="1.1">
+<defs>
+<g>
+<symbol overflow="visible" id="glyph0-0">
+<path style="stroke:none;" d="M 0.640625 2.265625 L 0.640625 -9.015625 L 7.03125 -9.015625 L 7.03125 2.265625 Z M 1.359375 1.546875 L 6.328125 1.546875 L 6.328125 -8.296875 L 1.359375 -8.296875 Z M 1.359375 1.546875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-1">
+<path style="stroke:none;" d="M 1.203125 -7 L 2.359375 -7 L 2.359375 0 L 1.203125 0 Z M 1.203125 -9.71875 L 2.359375 -9.71875 L 2.359375 -8.265625 L 1.203125 -8.265625 Z M 1.203125 -9.71875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-2">
+<path style="stroke:none;" d="M 6.65625 -5.65625 C 6.9375 -6.164062 7.273438 -6.546875 7.671875 -6.796875 C 8.078125 -7.046875 8.550781 -7.171875 9.09375 -7.171875 C 9.820312 -7.171875 10.382812 -6.914062 10.78125 -6.40625 C 11.175781 -5.894531 11.375 -5.164062 11.375 -4.21875 L 11.375 0 L 10.21875 0 L 10.21875 -4.1875 C 10.21875 -4.851562 10.097656 -5.347656 9.859375 -5.671875 C 9.628906 -6.003906 9.269531 -6.171875 8.78125 -6.171875 C 8.1875 -6.171875 7.710938 -5.972656 7.359375 -5.578125 C 7.015625 -5.179688 6.84375 -4.640625 6.84375 -3.953125 L 6.84375 0 L 5.6875 0 L 5.6875 -4.1875 C 5.6875 -4.863281 5.566406 -5.363281 5.328125 -5.6875 C 5.097656 -6.007812 4.734375 -6.171875 4.234375 -6.171875 C 3.648438 -6.171875 3.179688 -5.96875 2.828125 -5.5625 C 2.484375 -5.164062 2.3125 -4.628906 2.3125 -3.953125 L 2.3125 0 L 1.15625 0 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.90625 C 2.582031 -6.332031 2.898438 -6.648438 3.265625 -6.859375 C 3.628906 -7.066406 4.0625 -7.171875 4.5625 -7.171875 C 5.070312 -7.171875 5.503906 -7.039062 5.859375 -6.78125 C 6.222656 -6.519531 6.488281 -6.144531 6.65625 -5.65625 Z M 6.65625 -5.65625 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-3">
+<path style="stroke:none;" d="M 4.390625 -3.515625 C 3.460938 -3.515625 2.816406 -3.40625 2.453125 -3.1875 C 2.097656 -2.976562 1.921875 -2.617188 1.921875 -2.109375 C 1.921875 -1.703125 2.050781 -1.378906 2.3125 -1.140625 C 2.582031 -0.898438 2.953125 -0.78125 3.421875 -0.78125 C 4.054688 -0.78125 4.566406 -1.003906 4.953125 -1.453125 C 5.335938 -1.910156 5.53125 -2.515625 5.53125 -3.265625 L 5.53125 -3.515625 Z M 6.671875 -4 L 6.671875 0 L 5.53125 0 L 5.53125 -1.0625 C 5.269531 -0.632812 4.941406 -0.316406 4.546875 -0.109375 C 4.160156 0.0859375 3.679688 0.1875 3.109375 0.1875 C 2.390625 0.1875 1.816406 -0.015625 1.390625 -0.421875 C 0.972656 -0.828125 0.765625 -1.363281 0.765625 -2.03125 C 0.765625 -2.820312 1.023438 -3.414062 1.546875 -3.8125 C 2.078125 -4.21875 2.867188 -4.421875 3.921875 -4.421875 L 5.53125 -4.421875 L 5.53125 -4.53125 C 5.53125 -5.0625 5.351562 -5.46875 5 -5.75 C 4.65625 -6.039062 4.171875 -6.1875 3.546875 -6.1875 C 3.140625 -6.1875 2.75 -6.140625 2.375 -6.046875 C 2 -5.953125 1.632812 -5.8125 1.28125 -5.625 L 1.28125 -6.671875 C 1.695312 -6.835938 2.101562 -6.960938 2.5 -7.046875 C 2.894531 -7.128906 3.28125 -7.171875 3.65625 -7.171875 C 4.675781 -7.171875 5.429688 -6.90625 5.921875 -6.375 C 6.421875 -5.851562 6.671875 -5.0625 6.671875 -4 Z M 6.671875 -4 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-4">
+<path style="stroke:none;" d="M 5.8125 -3.578125 C 5.8125 -4.410156 5.640625 -5.054688 5.296875 -5.515625 C 4.953125 -5.972656 4.46875 -6.203125 3.84375 -6.203125 C 3.226562 -6.203125 2.75 -5.972656 2.40625 -5.515625 C 2.0625 -5.054688 1.890625 -4.410156 1.890625 -3.578125 C 1.890625 -2.753906 2.0625 -2.113281 2.40625 -1.65625 C 2.75 -1.195312 3.226562 -0.96875 3.84375 -0.96875 C 4.46875 -0.96875 4.953125 -1.195312 5.296875 -1.65625 C 5.640625 -2.113281 5.8125 -2.753906 5.8125 -3.578125 Z M 6.953125 -0.875 C 6.953125 0.320312 6.6875 1.207031 6.15625 1.78125 C 5.632812 2.363281 4.828125 2.65625 3.734375 2.65625 C 3.328125 2.65625 2.945312 2.625 2.59375 2.5625 C 2.238281 2.507812 1.890625 2.421875 1.546875 2.296875 L 1.546875 1.171875 C 1.890625 1.359375 2.222656 1.492188 2.546875 1.578125 C 2.878906 1.671875 3.21875 1.71875 3.5625 1.71875 C 4.3125 1.71875 4.875 1.519531 5.25 1.125 C 5.625 0.726562 5.8125 0.132812 5.8125 -0.65625 L 5.8125 -1.234375 C 5.570312 -0.816406 5.265625 -0.503906 4.890625 -0.296875 C 4.523438 -0.0976562 4.082031 0 3.5625 0 C 2.707031 0 2.015625 -0.328125 1.484375 -0.984375 C 0.960938 -1.640625 0.703125 -2.503906 0.703125 -3.578125 C 0.703125 -4.660156 0.960938 -5.53125 1.484375 -6.1875 C 2.015625 -6.84375 2.707031 -7.171875 3.5625 -7.171875 C 4.082031 -7.171875 4.523438 -7.066406 4.890625 -6.859375 C 5.265625 -6.648438 5.570312 -6.34375 5.8125 -5.9375 L 5.8125 -7 L 6.953125 -7 Z M 6.953125 -0.875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-5">
+<path style="stroke:none;" d="M 7.1875 -3.78125 L 7.1875 -3.21875 L 1.90625 -3.21875 C 1.957031 -2.425781 2.195312 -1.820312 2.625 -1.40625 C 3.050781 -1 3.644531 -0.796875 4.40625 -0.796875 C 4.84375 -0.796875 5.269531 -0.847656 5.6875 -0.953125 C 6.101562 -1.066406 6.515625 -1.226562 6.921875 -1.4375 L 6.921875 -0.359375 C 6.515625 -0.179688 6.09375 -0.046875 5.65625 0.046875 C 5.21875 0.140625 4.78125 0.1875 4.34375 0.1875 C 3.21875 0.1875 2.328125 -0.132812 1.671875 -0.78125 C 1.023438 -1.4375 0.703125 -2.320312 0.703125 -3.4375 C 0.703125 -4.582031 1.007812 -5.488281 1.625 -6.15625 C 2.25 -6.832031 3.085938 -7.171875 4.140625 -7.171875 C 5.078125 -7.171875 5.816406 -6.863281 6.359375 -6.25 C 6.910156 -5.644531 7.1875 -4.820312 7.1875 -3.78125 Z M 6.046875 -4.125 C 6.035156 -4.75 5.859375 -5.25 5.515625 -5.625 C 5.171875 -6 4.71875 -6.1875 4.15625 -6.1875 C 3.507812 -6.1875 2.992188 -6.003906 2.609375 -5.640625 C 2.222656 -5.285156 2 -4.78125 1.9375 -4.125 Z M 6.046875 -4.125 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-6">
+<path style="stroke:none;" d=""/>
+</symbol>
+<symbol overflow="visible" id="glyph0-7">
+<path style="stroke:none;" d="M 5.8125 -5.9375 L 5.8125 -9.71875 L 6.953125 -9.71875 L 6.953125 0 L 5.8125 0 L 5.8125 -1.046875 C 5.570312 -0.628906 5.265625 -0.316406 4.890625 -0.109375 C 4.523438 0.0859375 4.082031 0.1875 3.5625 0.1875 C 2.71875 0.1875 2.03125 -0.148438 1.5 -0.828125 C 0.96875 -1.503906 0.703125 -2.394531 0.703125 -3.5 C 0.703125 -4.59375 0.96875 -5.476562 1.5 -6.15625 C 2.03125 -6.832031 2.71875 -7.171875 3.5625 -7.171875 C 4.082031 -7.171875 4.523438 -7.066406 4.890625 -6.859375 C 5.265625 -6.660156 5.570312 -6.351562 5.8125 -5.9375 Z M 1.890625 -3.5 C 1.890625 -2.644531 2.0625 -1.976562 2.40625 -1.5 C 2.757812 -1.019531 3.238281 -0.78125 3.84375 -0.78125 C 4.457031 -0.78125 4.9375 -1.019531 5.28125 -1.5 C 5.632812 -1.976562 5.8125 -2.644531 5.8125 -3.5 C 5.8125 -4.34375 5.632812 -5.003906 5.28125 -5.484375 C 4.9375 -5.960938 4.457031 -6.203125 3.84375 -6.203125 C 3.238281 -6.203125 2.757812 -5.960938 2.40625 -5.484375 C 2.0625 -5.003906 1.890625 -4.34375 1.890625 -3.5 Z M 1.890625 -3.5 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-8">
+<path style="stroke:none;" d="M 2.34375 -8.984375 L 2.34375 -7 L 4.71875 -7 L 4.71875 -6.109375 L 2.34375 -6.109375 L 2.34375 -2.3125 C 2.34375 -1.738281 2.421875 -1.367188 2.578125 -1.203125 C 2.734375 -1.046875 3.050781 -0.96875 3.53125 -0.96875 L 4.71875 -0.96875 L 4.71875 0 L 3.53125 0 C 2.644531 0 2.03125 -0.164062 1.6875 -0.5 C 1.351562 -0.832031 1.1875 -1.4375 1.1875 -2.3125 L 1.1875 -6.109375 L 0.34375 -6.109375 L 0.34375 -7 L 1.1875 -7 L 1.1875 -8.984375 Z M 2.34375 -8.984375 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-9">
+<path style="stroke:none;" d="M 3.96875 -9.703125 C 3.40625 -8.742188 2.988281 -7.796875 2.71875 -6.859375 C 2.445312 -5.929688 2.3125 -4.984375 2.3125 -4.015625 C 2.3125 -3.054688 2.445312 -2.101562 2.71875 -1.15625 C 3 -0.21875 3.414062 0.726562 3.96875 1.6875 L 2.96875 1.6875 C 2.34375 0.707031 1.875 -0.253906 1.5625 -1.203125 C 1.25 -2.148438 1.09375 -3.085938 1.09375 -4.015625 C 1.09375 -4.941406 1.25 -5.875 1.5625 -6.8125 C 1.875 -7.757812 2.34375 -8.722656 2.96875 -9.703125 Z M 3.96875 -9.703125 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-10">
+<path style="stroke:none;" d="M 1.59375 -1.0625 L 3.65625 -1.0625 L 3.65625 -8.171875 L 1.40625 -7.734375 L 1.40625 -8.875 L 3.640625 -9.328125 L 4.90625 -9.328125 L 4.90625 -1.0625 L 6.953125 -1.0625 L 6.953125 0 L 1.59375 0 Z M 1.59375 -1.0625 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-11">
+<path style="stroke:none;" d="M 1.03125 -9.703125 L 2.03125 -9.703125 C 2.65625 -8.722656 3.117188 -7.757812 3.421875 -6.8125 C 3.734375 -5.875 3.890625 -4.941406 3.890625 -4.015625 C 3.890625 -3.085938 3.734375 -2.148438 3.421875 -1.203125 C 3.117188 -0.253906 2.65625 0.707031 2.03125 1.6875 L 1.03125 1.6875 C 1.582031 0.726562 1.992188 -0.21875 2.265625 -1.15625 C 2.535156 -2.101562 2.671875 -3.054688 2.671875 -4.015625 C 2.671875 -4.984375 2.535156 -5.929688 2.265625 -6.859375 C 1.992188 -7.796875 1.582031 -8.742188 1.03125 -9.703125 Z M 1.03125 -9.703125 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-12">
+<path style="stroke:none;" d="M 6.234375 -3.5 C 6.234375 -4.34375 6.054688 -5.003906 5.703125 -5.484375 C 5.359375 -5.960938 4.882812 -6.203125 4.28125 -6.203125 C 3.664062 -6.203125 3.179688 -5.960938 2.828125 -5.484375 C 2.484375 -5.003906 2.3125 -4.34375 2.3125 -3.5 C 2.3125 -2.644531 2.484375 -1.976562 2.828125 -1.5 C 3.179688 -1.019531 3.664062 -0.78125 4.28125 -0.78125 C 4.882812 -0.78125 5.359375 -1.019531 5.703125 -1.5 C 6.054688 -1.976562 6.234375 -2.644531 6.234375 -3.5 Z M 2.3125 -5.9375 C 2.5625 -6.351562 2.867188 -6.660156 3.234375 -6.859375 C 3.597656 -7.066406 4.039062 -7.171875 4.5625 -7.171875 C 5.40625 -7.171875 6.09375 -6.832031 6.625 -6.15625 C 7.15625 -5.476562 7.421875 -4.59375 7.421875 -3.5 C 7.421875 -2.394531 7.15625 -1.503906 6.625 -0.828125 C 6.09375 -0.148438 5.40625 0.1875 4.5625 0.1875 C 4.039062 0.1875 3.597656 0.0859375 3.234375 -0.109375 C 2.867188 -0.316406 2.5625 -0.628906 2.3125 -1.046875 L 2.3125 0 L 1.15625 0 L 1.15625 -9.71875 L 2.3125 -9.71875 Z M 2.3125 -5.9375 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-13">
+<path style="stroke:none;" d="M 2.453125 -1.0625 L 6.859375 -1.0625 L 6.859375 0 L 0.9375 0 L 0.9375 -1.0625 C 1.414062 -1.5625 2.066406 -2.226562 2.890625 -3.0625 C 3.722656 -3.894531 4.242188 -4.429688 4.453125 -4.671875 C 4.859375 -5.128906 5.140625 -5.515625 5.296875 -5.828125 C 5.460938 -6.140625 5.546875 -6.445312 5.546875 -6.75 C 5.546875 -7.25 5.367188 -7.65625 5.015625 -7.96875 C 4.671875 -8.28125 4.21875 -8.4375 3.65625 -8.4375 C 3.257812 -8.4375 2.84375 -8.363281 2.40625 -8.21875 C 1.96875 -8.082031 1.5 -7.878906 1 -7.609375 L 1 -8.875 C 1.507812 -9.082031 1.984375 -9.238281 2.421875 -9.34375 C 2.867188 -9.445312 3.273438 -9.5 3.640625 -9.5 C 4.609375 -9.5 5.378906 -9.253906 5.953125 -8.765625 C 6.523438 -8.285156 6.8125 -7.640625 6.8125 -6.828125 C 6.8125 -6.453125 6.738281 -6.09375 6.59375 -5.75 C 6.445312 -5.40625 6.1875 -5 5.8125 -4.53125 C 5.707031 -4.40625 5.375 -4.054688 4.8125 -3.484375 C 4.257812 -2.910156 3.472656 -2.101562 2.453125 -1.0625 Z M 2.453125 -1.0625 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-14">
+<path style="stroke:none;" d="M 5.671875 -6.796875 L 5.671875 -5.703125 C 5.347656 -5.867188 5.007812 -5.992188 4.65625 -6.078125 C 4.300781 -6.160156 3.9375 -6.203125 3.5625 -6.203125 C 3 -6.203125 2.570312 -6.113281 2.28125 -5.9375 C 2 -5.769531 1.859375 -5.507812 1.859375 -5.15625 C 1.859375 -4.882812 1.957031 -4.671875 2.15625 -4.515625 C 2.363281 -4.367188 2.773438 -4.226562 3.390625 -4.09375 L 3.78125 -4 C 4.601562 -3.832031 5.1875 -3.585938 5.53125 -3.265625 C 5.875 -2.941406 6.046875 -2.5 6.046875 -1.9375 C 6.046875 -1.28125 5.785156 -0.757812 5.265625 -0.375 C 4.753906 0 4.050781 0.1875 3.15625 0.1875 C 2.78125 0.1875 2.390625 0.148438 1.984375 0.078125 C 1.578125 0.00390625 1.144531 -0.101562 0.6875 -0.25 L 0.6875 -1.4375 C 1.113281 -1.21875 1.53125 -1.050781 1.9375 -0.9375 C 2.351562 -0.832031 2.765625 -0.78125 3.171875 -0.78125 C 3.710938 -0.78125 4.128906 -0.875 4.421875 -1.0625 C 4.710938 -1.25 4.859375 -1.507812 4.859375 -1.84375 C 4.859375 -2.15625 4.753906 -2.394531 4.546875 -2.5625 C 4.335938 -2.726562 3.875 -2.890625 3.15625 -3.046875 L 2.765625 -3.140625 C 2.046875 -3.285156 1.53125 -3.515625 1.21875 -3.828125 C 0.90625 -4.140625 0.75 -4.566406 0.75 -5.109375 C 0.75 -5.765625 0.976562 -6.269531 1.4375 -6.625 C 1.90625 -6.988281 2.570312 -7.171875 3.4375 -7.171875 C 3.851562 -7.171875 4.25 -7.140625 4.625 -7.078125 C 5 -7.015625 5.347656 -6.921875 5.671875 -6.796875 Z M 5.671875 -6.796875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-15">
+<path style="stroke:none;" d="M 3.921875 -6.1875 C 3.304688 -6.1875 2.816406 -5.945312 2.453125 -5.46875 C 2.097656 -4.988281 1.921875 -4.332031 1.921875 -3.5 C 1.921875 -2.65625 2.097656 -1.992188 2.453125 -1.515625 C 2.804688 -1.035156 3.296875 -0.796875 3.921875 -0.796875 C 4.535156 -0.796875 5.019531 -1.035156 5.375 -1.515625 C 5.726562 -2.003906 5.90625 -2.664062 5.90625 -3.5 C 5.90625 -4.320312 5.726562 -4.972656 5.375 -5.453125 C 5.019531 -5.941406 4.535156 -6.1875 3.921875 -6.1875 Z M 3.921875 -7.171875 C 4.921875 -7.171875 5.703125 -6.84375 6.265625 -6.1875 C 6.835938 -5.539062 7.125 -4.644531 7.125 -3.5 C 7.125 -2.351562 6.835938 -1.453125 6.265625 -0.796875 C 5.703125 -0.140625 4.921875 0.1875 3.921875 0.1875 C 2.910156 0.1875 2.117188 -0.140625 1.546875 -0.796875 C 0.984375 -1.453125 0.703125 -2.351562 0.703125 -3.5 C 0.703125 -4.644531 0.984375 -5.539062 1.546875 -6.1875 C 2.117188 -6.84375 2.910156 -7.171875 3.921875 -7.171875 Z M 3.921875 -7.171875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-16">
+<path style="stroke:none;" d="M 1.09375 -2.765625 L 1.09375 -7 L 2.234375 -7 L 2.234375 -2.8125 C 2.234375 -2.144531 2.363281 -1.644531 2.625 -1.3125 C 2.882812 -0.976562 3.269531 -0.8125 3.78125 -0.8125 C 4.40625 -0.8125 4.894531 -1.007812 5.25 -1.40625 C 5.613281 -1.800781 5.796875 -2.34375 5.796875 -3.03125 L 5.796875 -7 L 6.953125 -7 L 6.953125 0 L 5.796875 0 L 5.796875 -1.078125 C 5.515625 -0.648438 5.191406 -0.332031 4.828125 -0.125 C 4.460938 0.0820312 4.035156 0.1875 3.546875 0.1875 C 2.742188 0.1875 2.132812 -0.0625 1.71875 -0.5625 C 1.300781 -1.0625 1.09375 -1.796875 1.09375 -2.765625 Z M 3.984375 -7.171875 Z M 3.984375 -7.171875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-17">
+<path style="stroke:none;" d="M 5.265625 -5.921875 C 5.128906 -5.992188 4.984375 -6.046875 4.828125 -6.078125 C 4.679688 -6.117188 4.519531 -6.140625 4.34375 -6.140625 C 3.6875 -6.140625 3.179688 -5.925781 2.828125 -5.5 C 2.484375 -5.082031 2.3125 -4.476562 2.3125 -3.6875 L 2.3125 0 L 1.15625 0 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.90625 C 2.5625 -6.332031 2.878906 -6.648438 3.265625 -6.859375 C 3.648438 -7.066406 4.117188 -7.171875 4.671875 -7.171875 C 4.753906 -7.171875 4.84375 -7.164062 4.9375 -7.15625 C 5.03125 -7.144531 5.132812 -7.128906 5.25 -7.109375 Z M 5.265625 -5.921875 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-18">
+<path style="stroke:none;" d="M 6.25 -6.734375 L 6.25 -5.65625 C 5.914062 -5.832031 5.585938 -5.960938 5.265625 -6.046875 C 4.941406 -6.140625 4.613281 -6.1875 4.28125 -6.1875 C 3.53125 -6.1875 2.945312 -5.953125 2.53125 -5.484375 C 2.125 -5.015625 1.921875 -4.351562 1.921875 -3.5 C 1.921875 -2.644531 2.125 -1.976562 2.53125 -1.5 C 2.945312 -1.03125 3.53125 -0.796875 4.28125 -0.796875 C 4.613281 -0.796875 4.941406 -0.835938 5.265625 -0.921875 C 5.585938 -1.015625 5.914062 -1.148438 6.25 -1.328125 L 6.25 -0.265625 C 5.925781 -0.117188 5.59375 -0.0078125 5.25 0.0625 C 4.90625 0.144531 4.539062 0.1875 4.15625 0.1875 C 3.09375 0.1875 2.25 -0.144531 1.625 -0.8125 C 1.007812 -1.476562 0.703125 -2.375 0.703125 -3.5 C 0.703125 -4.632812 1.015625 -5.53125 1.640625 -6.1875 C 2.273438 -6.84375 3.132812 -7.171875 4.21875 -7.171875 C 4.570312 -7.171875 4.914062 -7.132812 5.25 -7.0625 C 5.59375 -6.988281 5.925781 -6.878906 6.25 -6.734375 Z M 6.25 -6.734375 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-19">
+<path style="stroke:none;" d="M 2.3125 -1.046875 L 2.3125 2.65625 L 1.15625 2.65625 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.9375 C 2.5625 -6.351562 2.867188 -6.660156 3.234375 -6.859375 C 3.597656 -7.066406 4.039062 -7.171875 4.5625 -7.171875 C 5.40625 -7.171875 6.09375 -6.832031 6.625 -6.15625 C 7.15625 -5.476562 7.421875 -4.59375 7.421875 -3.5 C 7.421875 -2.394531 7.15625 -1.503906 6.625 -0.828125 C 6.09375 -0.148438 5.40625 0.1875 4.5625 0.1875 C 4.039062 0.1875 3.597656 0.0859375 3.234375 -0.109375 C 2.867188 -0.316406 2.5625 -0.628906 2.3125 -1.046875 Z M 6.234375 -3.5 C 6.234375 -4.34375 6.054688 -5.003906 5.703125 -5.484375 C 5.359375 -5.960938 4.882812 -6.203125 4.28125 -6.203125 C 3.664062 -6.203125 3.179688 -5.960938 2.828125 -5.484375 C 2.484375 -5.003906 2.3125 -4.34375 2.3125 -3.5 C 2.3125 -2.644531 2.484375 -1.976562 2.828125 -1.5 C 3.179688 -1.019531 3.664062 -0.78125 4.28125 -0.78125 C 4.882812 -0.78125 5.359375 -1.019531 5.703125 -1.5 C 6.054688 -1.976562 6.234375 -2.644531 6.234375 -3.5 Z M 6.234375 -3.5 "/>
+</symbol>
+<symbol overflow="visible" id="glyph0-20">
+<path style="stroke:none;" d="M 4.0625 -8.5 C 3.414062 -8.5 2.925781 -8.175781 2.59375 -7.53125 C 2.269531 -6.894531 2.109375 -5.9375 2.109375 -4.65625 C 2.109375 -3.375 2.269531 -2.410156 2.59375 -1.765625 C 2.925781 -1.128906 3.414062 -0.8125 4.0625 -0.8125 C 4.71875 -0.8125 5.207031 -1.128906 5.53125 -1.765625 C 5.863281 -2.410156 6.03125 -3.375 6.03125 -4.65625 C 6.03125 -5.9375 5.863281 -6.894531 5.53125 -7.53125 C 5.207031 -8.175781 4.71875 -8.5 4.0625 -8.5 Z M 4.0625 -9.5 C 5.113281 -9.5 5.914062 -9.082031 6.46875 -8.25 C 7.019531 -7.425781 7.296875 -6.226562 7.296875 -4.65625 C 7.296875 -3.082031 7.019531 -1.878906 6.46875 -1.046875 C 5.914062 -0.222656 5.113281 0.1875 4.0625 0.1875 C 3.019531 0.1875 2.222656 -0.222656 1.671875 -1.046875 C 1.117188 -1.878906 0.84375 -3.082031 0.84375 -4.65625 C 0.84375 -6.226562 1.117188 -7.425781 1.671875 -8.25 C 2.222656 -9.082031 3.019531 -9.5 4.0625 -9.5 Z M 4.0625 -9.5 "/>
+</symbol>
+</g>
+</defs>
+<g id="surface11910">
+<rect x="0" y="0" width="296" height="142" style="fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;"/>
+<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 16.35 11.5 L 25.75 11.5 L 25.75 18.4 L 16.35 18.4 Z M 16.35 11.5 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.919141 14.75625 C 25.919141 14.99375 25.562695 14.99375 25.562695 14.75625 C 25.562695 14.518555 25.919141 14.518555 25.919141 14.75625 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.140039 13.450586 C 18.140039 13.69707 17.770117 13.69707 17.770117 13.450586 C 17.770117 13.203906 18.140039 13.203906 18.140039 13.450586 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.116211 16.39043 C 18.116211 16.637109 17.746484 16.637109 17.746484 16.39043 C 17.746484 16.143945 18.116211 16.143945 18.116211 16.39043 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-dasharray:1,1;stroke-miterlimit:10;" d="M 18.160937 16.341406 L 25.344141 14.80293 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.016797 15.128711 L 25.45332 14.779687 L 24.912109 14.639844 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.140039 13.450586 L 25.342383 14.717383 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 24.916797 14.896484 L 25.452539 14.736914 L 25.003516 14.403906 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.918945 14.75625 L 30.732813 14.715039 " transform="matrix(20,0,0,20,-325,-228)"/>
+<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 30.34668 14.968359 L 30.844531 14.714062 L 30.342383 14.468359 " transform="matrix(20,0,0,20,-325,-228)"/>
+<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
+  <use xlink:href="#glyph0-1" x="34.101562" y="34.267687"/>
+  <use xlink:href="#glyph0-2" x="37.712674" y="34.267687"/>
+  <use xlink:href="#glyph0-3" x="50.212674" y="34.267687"/>
+  <use xlink:href="#glyph0-4" x="57.990451" y="34.267687"/>
+  <use xlink:href="#glyph0-5" x="66.046007" y="34.267687"/>
+  <use xlink:href="#glyph0-6" x="73.823785" y="34.267687"/>
+  <use xlink:href="#glyph0-7" x="77.990451" y="34.267687"/>
+  <use xlink:href="#glyph0-3" x="86.046007" y="34.267687"/>
+  <use xlink:href="#glyph0-8" x="93.823785" y="34.267687"/>
+  <use xlink:href="#glyph0-3" x="98.823785" y="34.267687"/>
+  <use xlink:href="#glyph0-6" x="106.601562" y="34.267687"/>
+  <use xlink:href="#glyph0-9" x="110.768229" y="34.267687"/>
+  <use xlink:href="#glyph0-10" x="115.768229" y="34.267687"/>
+  <use xlink:href="#glyph0-11" x="123.823785" y="34.267687"/>
+</g>
+<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
+  <use xlink:href="#glyph0-5" x="33.625" y="115.412218"/>
+  <use xlink:href="#glyph0-2" x="41.402778" y="115.412218"/>
+  <use xlink:href="#glyph0-12" x="53.902778" y="115.412218"/>
+  <use xlink:href="#glyph0-5" x="61.958333" y="115.412218"/>
+  <use xlink:href="#glyph0-7" x="69.736111" y="115.412218"/>
+  <use xlink:href="#glyph0-7" x="77.791667" y="115.412218"/>
+  <use xlink:href="#glyph0-5" x="85.847222" y="115.412218"/>
+  <use xlink:href="#glyph0-7" x="93.625" y="115.412218"/>
+  <use xlink:href="#glyph0-6" x="101.680556" y="115.412218"/>
+  <use xlink:href="#glyph0-7" x="105.847222" y="115.412218"/>
+  <use xlink:href="#glyph0-3" x="113.902778" y="115.412218"/>
+  <use xlink:href="#glyph0-8" x="121.680556" y="115.412218"/>
+  <use xlink:href="#glyph0-3" x="126.680556" y="115.412218"/>
+  <use xlink:href="#glyph0-6" x="134.458333" y="115.412218"/>
+  <use xlink:href="#glyph0-9" x="138.625" y="115.412218"/>
+  <use xlink:href="#glyph0-13" x="143.625" y="115.412218"/>
+  <use xlink:href="#glyph0-11" x="151.680556" y="115.412218"/>
+</g>
+<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
+  <use xlink:href="#glyph0-14" x="197" y="54.505968"/>
+  <use xlink:href="#glyph0-15" x="203.666667" y="54.505968"/>
+  <use xlink:href="#glyph0-16" x="211.444444" y="54.505968"/>
+  <use xlink:href="#glyph0-17" x="219.5" y="54.505968"/>
+  <use xlink:href="#glyph0-18" x="224.5" y="54.505968"/>
+  <use xlink:href="#glyph0-5" x="231.444444" y="54.505968"/>
+  <use xlink:href="#glyph0-6" x="239.222222" y="54.505968"/>
+  <use xlink:href="#glyph0-19" x="243.388889" y="54.505968"/>
+  <use xlink:href="#glyph0-3" x="251.444444" y="54.505968"/>
+  <use xlink:href="#glyph0-7" x="259.222222" y="54.505968"/>
+  <use xlink:href="#glyph0-6" x="267.277778" y="54.505968"/>
+  <use xlink:href="#glyph0-9" x="271.444444" y="54.505968"/>
+  <use xlink:href="#glyph0-20" x="276.444444" y="54.505968"/>
+  <use xlink:href="#glyph0-11" x="284.5" y="54.505968"/>
+</g>
+</g>
+</svg>
diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index dcfcbd52490d..4d145bd3bd09 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -838,3 +838,5 @@ stream while it may be possible to enable and disable the embedded data stream.
 
 The embedded data format does not need to be configured on the sensor's pads as
 the format is dictated by the pixel data format in this case.
+
+.. include:: subdev-config-model.rst
diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
new file mode 100644
index 000000000000..b0bd09772ceb
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
@@ -0,0 +1,209 @@
+.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
+
+.. _media_subdev_config_model:
+
+Sub-device configuration models
+===============================
+
+A sub-device configuration model specifies in detail what the user space can
+expect from a sub-device in terms of V4L2 sub-device interface support,
+semantics included.
+
+A sub-device may implement more than one configuration model at the same
+time. The implemented configuration models can be obtained from the sub-device's
+``V4L2_CID_CONFIG_MODEL`` control.
+
+.. _media_subdev_config_model_common_raw_sensor:
+
+Common raw camera sensor model
+------------------------------
+
+The common raw camera sensor model defines a set of enumeration and
+configuration interfaces (formats, selections etc.) that cover the vast majority
+of functionality of raw camera sensors. Not all of the interfaces are
+necessarily offered by all drivers.
+
+A sub-device complies with the common raw sensor model if the
+``V4L2_CONFIG_MODEL_COMMON_RAW`` bit is set in the ``V4L2_CID_CONFIG_MODEL``
+control of the sub-device.
+
+The common raw camera sensor model is aligned with
+:ref:`media_using_camera_sensor_drivers`. Please refer to that regarding aspects
+not specified here.
+
+Each camera sensor implementing the common raw sensor model exposes a single
+V4L2 sub-device. The sub-device contains a single source pad (0) and two or more
+internal pads: an image data internal pad (1) and optionally an embedded data
+pad (2). Additionally, further internal pads may be supported for other
+features, in which case they are documented separately for the given device.
+
+This is shown in :ref:`media_subdev_config_model_common_raw_sensor_subdev`.
+
+.. _media_subdev_config_model_common_raw_sensor_subdev:
+
+.. kernel-figure:: common-raw-sensor.svg
+    :alt:    common-raw-sensor.svg
+    :align:  center
+
+    **Common raw sensor sub-device**
+
+Routes
+^^^^^^
+
+A sub-device conforming to common raw camera sensor model implements the
+following routes.
+
+.. flat-table:: Routes
+    :header-rows: 1
+
+    * - Sink pad/stream
+      - Source pad/stream
+      - Static (X/M(aybe)/-)
+      - Mandatory (X/-)
+      - Synopsis
+    * - 1/0
+      - 0/0
+      - X
+      - X
+      - Image data
+    * - 2/0
+      - 0/1
+      - M
+      - \-
+      - Embedded data
+
+Support for the embedded data stream is optional. Drivers supporting the
+embedded data stream may allow disabling and enabling the route when the
+streaming is disabled.
+
+Sensor pixel array size, cropping and binning
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The sensor's pixel array is divided into one or more areas. The areas around the
+edge of the pixel array, usually one or more sides, may contain optical black
+pixels, dummy pixels and other non-image pixels. The entire pixel array size is
+conveyed by the format on (pad, stream) pair 1/0.
+
+A rectangle within the pixel array contains the visible pixels. Capturing the
+non-visible pixels outside the visible pixel area may be supported by the
+sensor. The visible pixel area corresponds to the ``V4L2_SEL_TGT_CROP_DEFAULT``
+selection target on (pad, stream) pair 1/0.
+
+Sensors can perform multiple operations that affect the output image size. First
+of these is the analogue crop. Analogue crop limits the area of the pixel array
+which the sensor will read, affecting sensor timing as well. The granularity of
+the analogue crop configuration varies greatly across sensors: some sensors
+support only a few different analogue crop configurations whereas others may
+support anything divisible by a given number of pixels. The analogue crop
+configuration corresponds the ``V4L2_SEL_TGT_CROP`` selection target on (pad,
+stream) pair 1/0. The default analogue crop rectangle corresponds to the visible
+pixel area.
+
+In the next step, binning is performed on the image data read from camera
+sensor's pixel array, as determined by the analogue crop configuration. Enabling
+binning will effectively result in an image smaller than the original by given
+binning factors horizontally and vertically. Typical values are 1/2 and 1/3 but
+others may well be supported by the hardware as well.
+
+Sub-sampling follows binning. Sub-sampling, like binning, reduces the size of
+the image by including only a subset of samples read from the sensor's pixel
+matrix, typically every n'th pixel horizontally and vertically, taking the
+sensor's colour pattern into account. Sub-sampling is generally configurable
+separately horizontally and vertically.
+
+Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE``
+rectangle, relative to the analogue crop rectangle, on (pad, stream) pair
+1/0. The driver implementation determines how to configure binning and
+sub-sampling to achieve the desired size.
+
+The digital crop operation takes place after binning and sub-sampling. It is
+configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
+0/0. The resulting image size is further output by the sensor.
+
+The sensor's output mbus code is configured by setting the format on the (pad,
+stream) pair 0/0. When setting the format, always use the same width and height
+as for the digital crop setting.
+
+Drivers may only support some of even none of these configurations, in which
+case they do not expose the corresponding selection rectangles. If any selection
+targets are omitted, the further selection rectangle or format is instead
+related to the previous implemented selection rectangle. For instance, if the
+sensor supports binning but not analogue crop, then the binning configuration
+(``V4L2_SEL_TGT_COMPOSE`` selection target) is done in relation to the visible
+pixel area (``V4L2_SEL_TGT_CROP_DEFAULT`` selection target).
+
+Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
+
+.. flat-table:: Selection targets on pads
+    :header-rows: 1
+
+    * - Pad/Stream
+      - Selection target/format
+      - Mandatory (X/-)
+      - Modifiable (X/-)
+      - Synopsis
+    * - 1/0
+      - Format
+      - X
+      - \-
+      - Image data format. The width and the height fields indicates the full
+        size of the pixel array, including non-visible pixels. The media bus
+        code of this format reflects the native pixel depth of the sensor.
+    * - 1/0
+      - ``V4L2_SEL_TGT_CROP_DEFAULT``
+      - X
+      - \
+      - The visible pixel area. This rectangle is relative to the format on the
+        same (pad, stream).
+    * - 1/0
+      - ``V4L2_SEL_TGT_CROP``
+      - \-
+      - X
+      - Analogue crop. Analogue crop typically has a coarse granularity. This
+        rectangle is relative to the format on the same (pad, stream).
+    * - 1/0
+      - ``V4L2_SEL_TGT_COMPOSE``
+      - \-
+      - X
+      - Binning and sub-sampling. This rectangle is relative to the
+        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The
+        combination of binning and sub-sampling is configured using this
+        selection target.
+    * - 2/0
+      - Format
+      - X
+      - \-
+      - Embedded data format.
+    * - 0/0
+      - ``V4L2_SEL_TGT_CROP``
+      - \-
+      - X
+      - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE``
+        rectangle on (pad, stream) pair 1/0.
+    * - 0/0
+      - Format
+      - X
+      - X
+      - Image data source format. Always assign the width and height fields of
+        the format to the same values than for the ``V4L2_SEL_TGT_CROP``
+        rectangle on (pad, stream) pair 0/0. The media bus code reflects the
+        pixel data output of the sensor.
+    * - 0/1
+      - Format
+      - X
+      - \-
+      - Embedded data source format.
+
+Embedded data
+^^^^^^^^^^^^^
+
+The embedded data stream is produced by the sensor when the corresponding route
+is enabled. The embedded data route may also be immutable or not exist at all,
+in case the sensor (or the driver) does not support it.
+
+Generally the sensor embedded data width is determined by the width of the image
+data whereas the number of lines are constant for the embedded data. The user
+space may obtain the size of the embedded data once the image data size on the
+source pad has been configured.
+
+Also see :ref:`media_using_camera_sensor_drivers_embedded_data`.
-- 
2.39.5

[RFC v5 05/15] media: Documentation: Add scaling and post-scaler crop for common raw

[Sakari Ailus]

Document scaling and post-scaler digital crop operations for the common
raw sensor model.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 .../media/v4l/subdev-config-model.rst         | 24 +++++++++++++++----
 1 file changed, 19 insertions(+), 5 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
index b0bd09772ceb..f15e5495cc34 100644
--- a/Documentation/userspace-api/media/v4l/subdev-config-model.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
@@ -118,11 +118,18 @@ sub-sampling to achieve the desired size.
 
 The digital crop operation takes place after binning and sub-sampling. It is
 configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
-0/0. The resulting image size is further output by the sensor.
+0/0.
+
+The scaling operation is performed after the digital crop. It is configured by
+setting the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 0/0,
+relative to the digital crop.
 
 The sensor's output mbus code is configured by setting the format on the (pad,
-stream) pair 0/0. When setting the format, always use the same width and height
-as for the digital crop setting.
+stream) pair 0/0. The width and height fields are used to configure post-scaler
+digital crop if supported by the driver, affecting the right and bottom edges of
+the frame. If post-scaler digital crop is not supported, the width and height
+fields of the format will match the compose rectangle sizes applied on the same
+0/0 (pad, stream) pair.
 
 Drivers may only support some of even none of these configurations, in which
 case they do not expose the corresponding selection rectangles. If any selection
@@ -180,12 +187,19 @@ Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
       - X
       - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE``
         rectangle on (pad, stream) pair 1/0.
+    * - 0/0
+      - ``V4L2_SEL_TGT_COMPOSE``
+      - \-
+      - X
+      - Scaling. This rectangle is relative to the ``V4L2_SEL_TGT_CROP``
+        rectangle on (pad, stream) pair 0/0.
     * - 0/0
       - Format
       - X
       - X
-      - Image data source format. Always assign the width and height fields of
-        the format to the same values than for the ``V4L2_SEL_TGT_CROP``
+      - Image data source format and post-scaler crop. The width and height
+        fields of the format, used to configure post-scaler crop on the right
+        and bottom edges of the image, are related to the ``V4L2_SEL_TGT_COMPOSE``
         rectangle on (pad, stream) pair 0/0. The media bus code reflects the
         pixel data output of the sensor.
     * - 0/1
-- 
2.39.5

[RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control

[Sakari Ailus]

Add the V4L2_CID_CONFIG_MODEL control for the configuration model.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 4 ++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c                    | 5 +++++
 include/uapi/linux/v4l2-controls.h                           | 3 +++
 3 files changed, 12 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
index 27803dca8d3e..2ae17ed99729 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
@@ -55,3 +55,7 @@ Image Process Control IDs
     control value divided by e.g. 0x100, meaning that to get no
     digital gain the control value needs to be 0x100. The no-gain
     configuration is also typically the default.
+
+``V4L2_CID_CONFIG_MODEL (bitmask)``
+    Which configuration models the sub-device supports. Please see
+    :ref:`media_subdev_config_model`. This is a read-only control.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 1ea52011247a..24c9c25e20d1 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN:		return "Test Pattern";
 	case V4L2_CID_DEINTERLACING_MODE:	return "Deinterlacing Mode";
 	case V4L2_CID_DIGITAL_GAIN:		return "Digital Gain";
+	case V4L2_CID_CONFIG_MODEL:		return "Sub-device configuration model";
 
 	/* DV controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_DV_RX_POWER_PRESENT:
 		*type = V4L2_CTRL_TYPE_BITMASK;
 		break;
+	case V4L2_CID_CONFIG_MODEL:
+		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
+		*type = V4L2_CTRL_TYPE_BITMASK;
+		break;
 	case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
 	case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
 		*type = V4L2_CTRL_TYPE_INTEGER;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 974fd254e573..731add75d9ee 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_TEST_PATTERN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)
 #define V4L2_CID_DEINTERLACING_MODE		(V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)
 #define V4L2_CID_DIGITAL_GAIN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 5)
+#define V4L2_CID_CONFIG_MODEL			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 6)
+
+#define V4L2_CID_CONFIG_MODEL_COMMON_RAW	(1U << 0)
 
 /*  DV-class control IDs defined by V4L2 */
 #define V4L2_CID_DV_CLASS_BASE			(V4L2_CTRL_CLASS_DV | 0x900)
-- 
2.39.5

[RFC v5 07/15] media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour patterns

[Sakari Ailus]

Add V4L2_CID_COLOUR_PATTERN to tell the camera sensor's native colour
pattern.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  1 +
 include/uapi/linux/v4l2-controls.h                     |  6 ++++++
 3 files changed, 17 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index 71f23f131f97..fca729512b6f 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -92,3 +92,13 @@ Image Source Control IDs
     representing a gain of exactly 1.0. For example, if this default value
     is reported as being (say) 128, then a value of 192 would represent
     a gain of exactly 1.5.
+
+``V4L2_CID_COLOUR_PATTERN (integer)``
+    This control determines the colour components and pixel order in the
+    sensor's CFA (Colour Filter Array) when used in conjunction with
+    :ref:`luma-only mbus codes MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth)
+    <v4l2-mbus-pixelcode-yuv8>` pixelformats.
+
+    This control may only be used on a V4L2 sub-device.
+
+    This is a read-only control.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 24c9c25e20d1..5b6a4a94f18f 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1155,6 +1155,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN_BLUE:	return "Blue Pixel Value";
 	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
 	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
+	case V4L2_CID_COLOUR_PATTERN:		return "Colour Pattern";
 
 	/* Image processing controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 731add75d9ee..8e761c38b995 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1214,6 +1214,12 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_UNIT_CELL_SIZE			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
 #define V4L2_CID_NOTIFY_GAINS			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
 
+#define V4L2_CID_COLOUR_PATTERN			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10)
+#define V4L2_COLOUR_PATTERN_GRBG		0
+#define V4L2_COLOUR_PATTERN_RGGB		1
+#define V4L2_COLOUR_PATTERN_BGGR		2
+#define V4L2_COLOUR_PATTERN_GBRG		3
+
 
 /* Image processing controls */
 
-- 
2.39.5

[RFC v5 08/15] media: uapi: Correct generic CSI-2 metadata format 4cc

[Sakari Ailus]

Rework the pixelformat 4cc for CSI-2 generic metadata. This can be done as
no driver uses this yet and the interface is disabled.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../userspace-api/media/v4l/metafmt-generic.rst      |  2 +-
 include/uapi/linux/videodev2.h                       | 12 ++++++------
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
index 5116038af13d..1d46f1c4add1 100644
--- a/Documentation/userspace-api/media/v4l/metafmt-generic.rst
+++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
@@ -1,7 +1,7 @@
 .. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
 
 ********************************************************************************************************************************************************************************************************************************************************************************
-V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_META_FMT_GENERIC_CSI2_12 ('MC1C'), V4L2_META_FMT_GENERIC_CSI2_14 ('MC1E'), V4L2_META_FMT_GENERIC_CSI2_16 ('MC1G'), V4L2_META_FMT_GENERIC_CSI2_20 ('MC1K'), V4L2_META_FMT_GENERIC_CSI2_24 ('MC1O')
+V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MECA'), V4L2_META_FMT_GENERIC_CSI2_12 ('MECC'), V4L2_META_FMT_GENERIC_CSI2_14 ('MECE'), V4L2_META_FMT_GENERIC_CSI2_16 ('MECG'), V4L2_META_FMT_GENERIC_CSI2_20 ('MECK'), V4L2_META_FMT_GENERIC_CSI2_24 ('MECO')
 ********************************************************************************************************************************************************************************************************************************************************************************
 
 
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index e7c4dce39007..c8e30ab1de7b 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -869,12 +869,12 @@ struct v4l2_pix_format {
  * adding new ones!
  */
 #define V4L2_META_FMT_GENERIC_8		v4l2_fourcc('M', 'E', 'T', '8') /* Generic 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_10	v4l2_fourcc('M', 'C', '1', 'A') /* 10-bit CSI-2 packed 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_12	v4l2_fourcc('M', 'C', '1', 'C') /* 12-bit CSI-2 packed 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_14	v4l2_fourcc('M', 'C', '1', 'E') /* 14-bit CSI-2 packed 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_16	v4l2_fourcc('M', 'C', '1', 'G') /* 16-bit CSI-2 packed 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'C', '1', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
-#define V4L2_META_FMT_GENERIC_CSI2_24	v4l2_fourcc('M', 'C', '1', 'O') /* 24-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_CSI2_10	v4l2_fourcc('M', 'E', 'C', 'A') /* 10-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_CSI2_12	v4l2_fourcc('M', 'E', 'C', 'C') /* 12-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_CSI2_14	v4l2_fourcc('M', 'E', 'C', 'E') /* 14-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_CSI2_16	v4l2_fourcc('M', 'E', 'C', 'G') /* 16-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_CSI2_20	v4l2_fourcc('M', 'E', 'C', 'K') /* 20-bit CSI-2 packed 8-bit metadata */
+#define V4L2_META_FMT_GENERIC_CSI2_24	v4l2_fourcc('M', 'E', 'C', 'O') /* 24-bit CSI-2 packed 8-bit metadata */
 #endif
 
 /* priv field value to indicates that subsequent fields are valid. */
-- 
2.39.5

[RFC v5 09/15] media: uapi: Documentation: Improve column width hints for examples

[Sakari Ailus]

Use less arbitrary widths for the columns in metadata layout examples.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 Documentation/userspace-api/media/v4l/metafmt-generic.rst | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
index 1d46f1c4add1..a800342c8071 100644
--- a/Documentation/userspace-api/media/v4l/metafmt-generic.rst
+++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst
@@ -74,7 +74,7 @@ This format is little endian.
 **Byte Order Of V4L2_META_FMT_GENERIC_CSI2_10.**
 Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
 
-.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|
+.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
 
 .. flat-table:: Sample 4x2 Metadata Frame
     :header-rows:  0
@@ -118,7 +118,7 @@ This format is little endian.
 **Byte Order Of V4L2_META_FMT_GENERIC_CSI2_12.**
 Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
 
-.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}|
+.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|
 
 .. flat-table:: Sample 4x2 Metadata Frame
     :header-rows:  0
@@ -159,7 +159,7 @@ This format is little endian.
 **Byte Order Of V4L2_META_FMT_GENERIC_CSI2_14.**
 Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
 
-.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{.8cm}|
+.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.8cm}|p{1.8cm}|
 
 .. flat-table:: Sample 4x2 Metadata Frame
     :header-rows:  0
@@ -255,7 +255,7 @@ This format is little endian.
 **Byte Order Of V4L2_META_FMT_GENERIC_CSI2_20.**
 Each cell is one byte. "M" denotes a byte of metadata and "x" a byte of padding.
 
-.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|
+.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.8cm}
 
 .. flat-table:: Sample 4x2 Metadata Frame
     :header-rows:  0
-- 
2.39.5

[RFC v5 10/15] media: v4l: uapi: Add a control for colour pattern flipping effect

[Sakari Ailus]

Add a bitmask control (V4L2_CID_COLOUR_PATTERN_FLIP) to tell whether
flipping results in a change in the sensor's colour pattern, separately
horizontally and vertically. The information is essential for raw formats
when using generic raw mbus codes.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../media/v4l/ext-ctrls-image-source.rst          |  8 ++++++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c         |  2 ++
 include/uapi/linux/v4l2-controls.h                | 15 +++++++++++++++
 3 files changed, 25 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index fca729512b6f..ecfa38c118e3 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -102,3 +102,11 @@ Image Source Control IDs
     This control may only be used on a V4L2 sub-device.
 
     This is a read-only control.
+
+``V4L2_CID_COLOUR_PATTERN_FLIP (bitmask)``
+    Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
+    V4L2_CID_VFLIP) have an effect on the pixel order of the output colour
+    pattern. Macros ``V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL`` and
+    ``V4L2_COLOUR_PATTERN_FLIP_VERTICAL`` define bitmasks for both bits. If
+    either horizontal or vertical bit is set, the readout pattern order is that
+    of the reversed readout.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 5b6a4a94f18f..3f0704a982b8 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1156,6 +1156,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
 	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
 	case V4L2_CID_COLOUR_PATTERN:		return "Colour Pattern";
+	case V4L2_CID_COLOUR_PATTERN_FLIP:	return "Colour Pattern Flip";
 
 	/* Image processing controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1474,6 +1475,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 		*max = 0xffff;
 		break;
 	case V4L2_CID_FLASH_FAULT:
+	case V4L2_CID_COLOUR_PATTERN_FLIP:
 	case V4L2_CID_JPEG_ACTIVE_MARKER:
 	case V4L2_CID_3A_LOCK:
 	case V4L2_CID_AUTO_FOCUS_STATUS:
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 8e761c38b995..fd6465e9a743 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1215,11 +1215,26 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_NOTIFY_GAINS			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
 
 #define V4L2_CID_COLOUR_PATTERN			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10)
+/*
+ * CFA pattern start shall be aligned with the number of patterns for colour
+ * components. Patterns shall be calculable based on flipping when it affects
+ * the pattern the following way:
+ *
+ * flipped_pattern = native_pattern ^
+ *	((hflip ? V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL : 0) |
+ *	 (vflip ? V4L2_COLOUR_PATTERN_FLIP_VERTICAL : 0))
+ *
+ * where hflip and vflip are the values of V4L2_CID_HFLIP and V4L2_CID_VFLIP
+ * controls, respectively.
+ */
 #define V4L2_COLOUR_PATTERN_GRBG		0
 #define V4L2_COLOUR_PATTERN_RGGB		1
 #define V4L2_COLOUR_PATTERN_BGGR		2
 #define V4L2_COLOUR_PATTERN_GBRG		3
 
+#define V4L2_CID_COLOUR_PATTERN_FLIP		(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 11)
+#define V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL	(1U << 0)
+#define V4L2_COLOUR_PATTERN_FLIP_VERTICAL	(1U << 1)
 
 /* Image processing controls */
 
-- 
2.39.5

[RFC v5 11/15] media: Documentation: Document luma-only mbus codes and CFA for cameras

[Sakari Ailus]

Document the use of luma-only mbus codes for camera sensors and how the
V4L2_CID_COLOUR_PATTERN and V4L2_CID_COLOUR_PATTERN_FLIP controls are used
to convey the colour filter array pattern on UAPI.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../userspace-api/media/drivers/camera-sensor.rst   | 13 +++++++++++++
 .../media/v4l/ext-ctrls-image-source.rst            |  4 ++++
 2 files changed, 17 insertions(+)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index 5bc4c79d230c..91e5305458b9 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -41,6 +41,19 @@ format set on a source pad at the end of the device's internal pipeline.
 
 Most sensor drivers are implemented this way.
 
+V4L2_CID_COLOUR_PATTERN, luma-only mbus formats, flipping and cropping
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For raw image data originating from camera sensors, :ref:`luma-only mbus codes
+MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth) <v4l2-mbus-pixelcode-yuv8>`
+are used as Colour Filter Array (CFA) agnostic raw formats. The
+``V4L2_CID_COLOUR_PATTERN <image-source-control-colour-pattern>`` control in the
+same sub-device defines the native colour pattern of the device. Flipping may
+further affect the readout pattern as indicated by the
+``V4L2_CID_COLOUR_PATTERN_FLIP <image-source-control-colour-pattern-flip>``
+control. Further on, cropping also has an effect on the pattern if cropped
+amount is not divisible by the size of the pattern, horizontally and vertically.
+
 Frame interval configuration
 ----------------------------
 
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
index ecfa38c118e3..4b98a740236a 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
@@ -93,6 +93,8 @@ Image Source Control IDs
     is reported as being (say) 128, then a value of 192 would represent
     a gain of exactly 1.5.
 
+.. _image-source-control-colour-pattern:
+
 ``V4L2_CID_COLOUR_PATTERN (integer)``
     This control determines the colour components and pixel order in the
     sensor's CFA (Colour Filter Array) when used in conjunction with
@@ -103,6 +105,8 @@ Image Source Control IDs
 
     This is a read-only control.
 
+.. _image-source-control-colour-pattern-flip:
+
 ``V4L2_CID_COLOUR_PATTERN_FLIP (bitmask)``
     Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
     V4L2_CID_VFLIP) have an effect on the pixel order of the output colour
-- 
2.39.5

[RFC v5 12/15] media: uapi: Documentation: Use luma formats with CFA pattern control

[Sakari Ailus]

Document that the luma-only, greyscale formats (Y10 etc.) are used also
for non-greyscale images with the CFA pattern specified by the
V4L2_CID_COLOUR_PATTERN control.

The advantage of this approach is that there is little (or none at all)
work needed to be able to capture such images with existing CSI-2 receiver
drivers as many already support greyscale formats when compared to new
CFA-agnostic pixelformats.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
index 74df19be91f6..ae4e431b5fd9 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
@@ -9,6 +9,11 @@ Luma-Only Formats
 This family of formats only store the luma component of a Y'CbCr image. They
 are often referred to as greyscale formats.
 
+These pixel formats are also used on :term:`MC-centric` devices when the CFA
+pattern is specified by the :ref:`V4L2_CID_COLOUR_PATTERN
+<image-source-control-colour-pattern>` control in the image source, typically a
+camera sensor.
+
 .. note::
 
    - In all the tables that follow, bit 7 is the most significant bit in a byte.
-- 
2.39.5

[RFC v5 13/15] media: uapi: Add V4L2_CID_BINNING control for binning configuration

[Sakari Ailus]

Add V4L2_CID_BINNING control for configuring binning and enumerating a
camera sensor's binning capabilities. The control combines horizontal and
vertical binning into a single control as the two are generally related.

New drivers should use this control to configure binning.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 .../media/drivers/camera-sensor.rst           | 10 +++++++
 .../media/v4l/ext-ctrls-camera.rst            | 29 +++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  2 ++
 include/uapi/linux/v4l2-controls.h            |  1 +
 4 files changed, 42 insertions(+)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index 91e5305458b9..59431ab0b923 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -120,6 +120,16 @@ values programmed by the register sequences. The default values of these
 controls shall be 0 (disabled). Especially these controls shall not be inverted,
 independently of the sensor's mounting rotation.
 
+Binning
+-------
+
+Binning has traditionally been configured using :ref:`the compose selection
+rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING
+<v4l2-cid-camera-sensor-binning>` is also available for binning configuration and
+users should use it when it's available. Drivers supporting the control shall
+also support the compose rectangle, albeit the rectangle may be read-only when
+the control is present.
+
 .. _media_using_camera_sensor_drivers_embedded_data:
 
 Embedded data
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
index cdc515c60468..18b484ff5d75 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
@@ -672,3 +672,32 @@ enum v4l2_scene_mode -
 
     As modes differ for each sensor, menu items are not standardized by this
     control and are left to the programmer.
+
+.. _v4l2-cid-camera-sensor-binning:
+
+``V4L2_CID_BINNING_FACTORS (integer menu)``
+
+    Horizontal and vertical binning factors. Binning combines several
+    horizontal, vertical or both pixel values into a single pixel. It is a way
+    to scale an image. Binning typically produces fairly good quality output.
+
+    Determines both horizontal and vertical binning factors for a camera
+    sensor. The values are encoded in the following way:
+
+.. flat-table::
+    :header-rows:  1
+    :stub-columns: 0
+
+    * - Bits
+      - Synopsis
+    * - 48--63
+      - Horizontal binning numerator.
+    * - 32--47
+      - Horizontal binning denominator.
+    * - 16--31
+      - Vertical binning numerator.
+    * - 0--15
+      - Vertical binning denominator.
+
+For instance, a value of ``0x0001000300020003`` indicates binning by 3
+(horizontally) * 3/2 (vertically).
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 3f0704a982b8..6be3d4a37127 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1087,6 +1087,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_CAMERA_ORIENTATION:	return "Camera Orientation";
 	case V4L2_CID_CAMERA_SENSOR_ROTATION:	return "Camera Sensor Rotation";
 	case V4L2_CID_HDR_SENSOR_MODE:		return "HDR Sensor Mode";
+	case V4L2_CID_BINNING_FACTORS:		return "Binning Factors";
 
 	/* FM Radio Modulator controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1426,6 +1427,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_HDR_SENSOR_MODE:
 		*type = V4L2_CTRL_TYPE_MENU;
 		break;
+	case V4L2_CID_BINNING_FACTORS:
 	case V4L2_CID_LINK_FREQ:
 		*type = V4L2_CTRL_TYPE_INTEGER_MENU;
 		break;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index fd6465e9a743..ac0e38e98a38 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1088,6 +1088,7 @@ enum v4l2_auto_focus_range {
 #define V4L2_CID_CAMERA_SENSOR_ROTATION		(V4L2_CID_CAMERA_CLASS_BASE+35)
 
 #define V4L2_CID_HDR_SENSOR_MODE		(V4L2_CID_CAMERA_CLASS_BASE+36)
+#define V4L2_CID_BINNING_FACTORS		(V4L2_CID_CAMERA_CLASS_BASE+37)
 
 /* FM Modulator class control IDs */
 
-- 
2.39.5

[RFC v5 14/15] media: uapi: Add controls for sub-sampling configuration

[Sakari Ailus]

Sub-sampling is a way to decrease the data rates after the pixel array by
systematically discarding some samples, either vertically or horizontally
or both. Add two controls for the purpose and document them. The
sub-sampling configuration is taken into account in the compose rectangle.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 .../userspace-api/media/drivers/camera-sensor.rst  | 14 ++++++++++++--
 .../userspace-api/media/v4l/ext-ctrls-camera.rst   | 11 +++++++++++
 drivers/media/v4l2-core/v4l2-ctrls-defs.c          |  2 ++
 include/uapi/linux/v4l2-controls.h                 |  2 ++
 4 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
index 59431ab0b923..9e74eb02e02a 100644
--- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
+++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
@@ -120,8 +120,8 @@ values programmed by the register sequences. The default values of these
 controls shall be 0 (disabled). Especially these controls shall not be inverted,
 independently of the sensor's mounting rotation.
 
-Binning
--------
+Binning and sub-sampling
+------------------------
 
 Binning has traditionally been configured using :ref:`the compose selection
 rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING
@@ -130,6 +130,16 @@ users should use it when it's available. Drivers supporting the control shall
 also support the compose rectangle, albeit the rectangle may be read-only when
 the control is present.
 
+Sub-sampling is often supported as part of a camera sensor's binning
+functionality and performed after the binning operation. Sub-sampling typically
+produces quality-wise worse results than binning. Sub-sampling factors are
+independent horizontally and vertically and they are controlled using two
+controls, :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and
+V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>`. In
+sub-sampling, the image size before sub-sampling is horizontally and vertically
+divided by the respective sub-sampling factors. Drivers supporting the control shall
+also reflect the sub-sampling configuration in the compose rectangle.
+
 .. _media_using_camera_sensor_drivers_embedded_data:
 
 Embedded data
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
index 18b484ff5d75..577b73045bee 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
@@ -701,3 +701,14 @@ enum v4l2_scene_mode -
 
 For instance, a value of ``0x0001000300020003`` indicates binning by 3
 (horizontally) * 3/2 (vertically).
+
+.. _v4l2-cid-camera-sensor-subsampling:
+
+``V4L2_CID_SUBSAMPLING_HORIZONTAL`` and ``V4L2_CID_SUBSAMPLING_VERTICAL``
+(integer)
+
+    Horizontal and vertical subsampling factors.
+
+    Sub-sampling is used to downscale an image, horizontally and vertically, by
+    discarding a part of the image data. Typically sub-sampling produces lower
+    quality images than binning.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 6be3d4a37127..5f4a8cc9f53b 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -1088,6 +1088,8 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_CAMERA_SENSOR_ROTATION:	return "Camera Sensor Rotation";
 	case V4L2_CID_HDR_SENSOR_MODE:		return "HDR Sensor Mode";
 	case V4L2_CID_BINNING_FACTORS:		return "Binning Factors";
+	case V4L2_CID_SUBSAMPLING_HORIZONTAL:	return "Sub-Sampling Factor, Horizontal";
+	case V4L2_CID_SUBSAMPLING_VERTICAL:	return "Sub-Sampling Factor, Vertical";
 
 	/* FM Radio Modulator controls */
 	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index ac0e38e98a38..4eade177d40f 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1089,6 +1089,8 @@ enum v4l2_auto_focus_range {
 
 #define V4L2_CID_HDR_SENSOR_MODE		(V4L2_CID_CAMERA_CLASS_BASE+36)
 #define V4L2_CID_BINNING_FACTORS		(V4L2_CID_CAMERA_CLASS_BASE+37)
+#define V4L2_CID_SUBSAMPLING_HORIZONTAL		(V4L2_CID_CAMERA_CLASS_BASE+38)
+#define V4L2_CID_SUBSAMPLING_VERTICAL		(V4L2_CID_CAMERA_CLASS_BASE+39)
 
 /* FM Modulator class control IDs */
 
-- 
2.39.5

[RFC v5 15/15] media: Documentation: Add binning and sub-sampling controls

[Sakari Ailus]

Document the binning and scaling controls (V4L2_CID_BINNING and
V4L2_CID_SUBSAMPLING_{HORIZONTAL,VERTICAL}) in the common raw sensor
model.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 .../media/v4l/subdev-config-model.rst         | 20 ++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
index f15e5495cc34..84957dc62800 100644
--- a/Documentation/userspace-api/media/v4l/subdev-config-model.rst
+++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
@@ -113,8 +113,12 @@ separately horizontally and vertically.
 
 Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE``
 rectangle, relative to the analogue crop rectangle, on (pad, stream) pair
-1/0. The driver implementation determines how to configure binning and
-sub-sampling to achieve the desired size.
+1/0. It depends on the driver which of these operations are being used to
+achieve the resulting size. Binning and sub-sampling are also directly
+configured using :ref:`V4L2_CID_BINNING_FACTORS
+<v4l2-cid-camera-sensor-binning>` and :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and
+V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>` controls on
+drivers that support them.
 
 The digital crop operation takes place after binning and sub-sampling. It is
 configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
@@ -173,9 +177,15 @@ Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
       - \-
       - X
       - Binning and sub-sampling. This rectangle is relative to the
-        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The
-        combination of binning and sub-sampling is configured using this
-        selection target.
+        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). Binning is
+        configured using the :ref:`V4L2_CID_BINNING_FACTORS
+        <v4l2-cid-camera-sensor-binning>` control and sub-sampling is configured
+        using the :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and
+        V4L2_CID_SUBSAMPLING_VERTICAL <v4l2-cid-camera-sensor-subsampling>`
+        controls on drivers that support these controls. To configure binning
+        and sub-sampling on drivers that do not support these controls, the
+        selection rectangle may be changed directly to configure the combined
+        effect on the image size.
     * - 2/0
       - Format
       - X
-- 
2.39.5

Re: [RFC v5 01/15] media: Documentation: Rework embedded data documentation

[Lad, Prabhakar]

On Mon, Feb 3, 2025 at 8:59 AM Sakari Ailus
<sakari.ailus@linux.intel.com> wrote:
>
> Rework embedded data documentation by removing the reference to the pixel
> data stream. The specific documentation of the embedded data interface
> will be elsewhere, in the near future either CCS or common raw camera
> sensor model documentation.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  .../userspace-api/media/drivers/camera-sensor.rst   | 13 ++++++-------
>  1 file changed, 6 insertions(+), 7 deletions(-)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index dc415b8f6c8e..8e1083417ae1 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -111,13 +111,12 @@ the sensor configuration for the captured frame back to the host. While CSI-2 is
>  the most common data interface used by such sensors, embedded data can be
>  available on other interfaces as well.
>
> -Such sensors expose two internal sink pads (pads that have both the
> -``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and ``MEDIA_PAD_FL_INTERNAL
> -<MEDIA-PAD-FL-INTERNAL>`` flags set) to model the source of the image and
> -embedded data streams. Both of these pads produces a single stream, and the
> -sub-device routes those streams to the external (source) pad. If the sub-device
> -driver supports disabling embedded data, this can be done by disabling the
> -embedded data route via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL.
> +Embedded data support is indicated by the precence of an internal sink pad (pad
> +that has both the ``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and
> +``MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`` flags set) with a metadata
> +format to model the embedded data stream. If the sub-device driver supports
> +disabling embedded data, this can be done by disabling the embedded data route
> +via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL.
>
>  In general, changing the embedded data format from the driver-configured values
>  is not supported. The height of the metadata is device-specific and the width
> --
> 2.39.5
>

Re: [RFC v5 02/15] media: Documentation: Add a hyphen to list-based

[Lad, Prabhakar]

On Mon, Feb 3, 2025 at 8:59 AM Sakari Ailus
<sakari.ailus@linux.intel.com> wrote:
>
> Add a hyphen to list-based for uniform spelling in camera-sensor.rst.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  Documentation/userspace-api/media/drivers/camera-sensor.rst | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index 8e1083417ae1..9a1e8aa9fc77 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -26,10 +26,10 @@ of cropping and scaling operations from the device's pixel array's size.
>
>  An example of such a driver is the CCS driver.
>
> -Register list based drivers
> +Register list-based drivers
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> -Register list based drivers generally, instead of able to configure the device
> +Register list-based drivers generally, instead of able to configure the device
>  they control based on user requests, are limited to a number of preset
>  configurations that combine a number of different parameters that on hardware
>  level are independent. How a driver picks such configuration is based on the
> @@ -67,7 +67,7 @@ is pixels and the unit of the ``V4L2_CID_VBLANK`` is lines. The pixel rate in
>  the sensor's **pixel array** is specified by ``V4L2_CID_PIXEL_RATE`` in the same
>  sub-device. The unit of that control is pixels per second.
>
> -Register list based drivers need to implement read-only sub-device nodes for the
> +Register list-based drivers need to implement read-only sub-device nodes for the
>  purpose. Devices that are not register list based need these to configure the
>  device's internal processing pipeline.
>
> --
> 2.39.5
>

Re: [RFC v5 03/15] media: Documentation: Reword split of sensor driver to two classes

[Lad, Prabhakar]

On Mon, Feb 3, 2025 at 8:59 AM Sakari Ailus
<sakari.ailus@linux.intel.com> wrote:
>
> The sensor drivers do not configure the output size of the sensors but the
> entire internal pipeline. Reflect this in the documentation.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  .../userspace-api/media/drivers/camera-sensor.rst      | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index 9a1e8aa9fc77..bc55c861fb69 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -10,11 +10,13 @@ used to control the camera sensor drivers.
>
>  You may also find :ref:`media_writing_camera_sensor_drivers` useful.
>
> -Frame size
> -----------
> +Sensor internal pipeline configuration
> +--------------------------------------
>
> -There are two distinct ways to configure the frame size produced by camera
> -sensors.
> +Camera sensors have an internal processing pipeline including cropping and
> +binning functionality. The sensor drivers belong to two distinct classes, freely
> +configurable and register list-based drivers, depending on how the driver
> +configures this functionality.
>
>  Freely configurable camera sensor drivers
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> --
> 2.39.5
>

Re: [RFC v5 04/15] media: Documentation: Add subdev configuration models, raw sensor model

[Lad, Prabhakar]

Hi Sakari,

Thank you for the patch.

On Mon, Feb 3, 2025 at 8:59 AM Sakari Ailus
<sakari.ailus@linux.intel.com> wrote:
>
> Sub-device configuration models define what V4L2 API elements are
> available on a compliant sub-device and how do they behave.
>
> The patch also adds a model for common raw sensors.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  .../media/drivers/camera-sensor.rst           |   4 +
>  .../media/v4l/common-raw-sensor.dia           | 441 ++++++++++++++++++
>  .../media/v4l/common-raw-sensor.svg           | 134 ++++++
>  .../userspace-api/media/v4l/dev-subdev.rst    |   2 +
>  .../media/v4l/subdev-config-model.rst         | 209 +++++++++
>  5 files changed, 790 insertions(+)
>  create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.dia
>  create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.svg
>  create mode 100644 Documentation/userspace-api/media/v4l/subdev-config-model.rst
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index bc55c861fb69..5bc4c79d230c 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -18,6 +18,8 @@ binning functionality. The sensor drivers belong to two distinct classes, freely
>  configurable and register list-based drivers, depending on how the driver
>  configures this functionality.
>
> +Also see :ref:`media_subdev_config_model_common_raw_sensor`.
> +
>  Freely configurable camera sensor drivers
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> @@ -105,6 +107,8 @@ values programmed by the register sequences. The default values of these
>  controls shall be 0 (disabled). Especially these controls shall not be inverted,
>  independently of the sensor's mounting rotation.
>
> +.. _media_using_camera_sensor_drivers_embedded_data:
> +
>  Embedded data
>  -------------
<snip>
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index dcfcbd52490d..4d145bd3bd09 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -838,3 +838,5 @@ stream while it may be possible to enable and disable the embedded data stream.
>
>  The embedded data format does not need to be configured on the sensor's pads as
>  the format is dictated by the pixel data format in this case.
> +
> +.. include:: subdev-config-model.rst
> diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> new file mode 100644
> index 000000000000..b0bd09772ceb
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> @@ -0,0 +1,209 @@
> +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
> +
> +.. _media_subdev_config_model:
> +
> +Sub-device configuration models
> +===============================
> +
> +A sub-device configuration model specifies in detail what the user space can
> +expect from a sub-device in terms of V4L2 sub-device interface support,
> +semantics included.
> +
> +A sub-device may implement more than one configuration model at the same
> +time. The implemented configuration models can be obtained from the sub-device's
> +``V4L2_CID_CONFIG_MODEL`` control.
> +
> +.. _media_subdev_config_model_common_raw_sensor:
> +
> +Common raw camera sensor model
> +------------------------------
> +
> +The common raw camera sensor model defines a set of enumeration and
> +configuration interfaces (formats, selections etc.) that cover the vast majority
> +of functionality of raw camera sensors. Not all of the interfaces are
> +necessarily offered by all drivers.
> +
> +A sub-device complies with the common raw sensor model if the
> +``V4L2_CONFIG_MODEL_COMMON_RAW`` bit is set in the ``V4L2_CID_CONFIG_MODEL``
> +control of the sub-device.
> +
> +The common raw camera sensor model is aligned with
> +:ref:`media_using_camera_sensor_drivers`. Please refer to that regarding aspects
> +not specified here.
> +
> +Each camera sensor implementing the common raw sensor model exposes a single
> +V4L2 sub-device. The sub-device contains a single source pad (0) and two or more
> +internal pads: an image data internal pad (1) and optionally an embedded data
> +pad (2). Additionally, further internal pads may be supported for other
> +features, in which case they are documented separately for the given device.
> +
> +This is shown in :ref:`media_subdev_config_model_common_raw_sensor_subdev`.
> +
> +.. _media_subdev_config_model_common_raw_sensor_subdev:
> +
> +.. kernel-figure:: common-raw-sensor.svg
> +    :alt:    common-raw-sensor.svg
> +    :align:  center
> +
> +    **Common raw sensor sub-device**
> +
> +Routes
> +^^^^^^
> +
> +A sub-device conforming to common raw camera sensor model implements the
> +following routes.
> +
> +.. flat-table:: Routes
> +    :header-rows: 1
> +
> +    * - Sink pad/stream
> +      - Source pad/stream
> +      - Static (X/M(aybe)/-)
> +      - Mandatory (X/-)
> +      - Synopsis
> +    * - 1/0
> +      - 0/0
> +      - X
> +      - X
> +      - Image data
> +    * - 2/0
> +      - 0/1
> +      - M
> +      - \-
> +      - Embedded data
> +
> +Support for the embedded data stream is optional. Drivers supporting the
> +embedded data stream may allow disabling and enabling the route when the
> +streaming is disabled.
> +
> +Sensor pixel array size, cropping and binning
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +The sensor's pixel array is divided into one or more areas. The areas around the
> +edge of the pixel array, usually one or more sides, may contain optical black
> +pixels, dummy pixels and other non-image pixels. The entire pixel array size is
> +conveyed by the format on (pad, stream) pair 1/0.
> +
> +A rectangle within the pixel array contains the visible pixels. Capturing the
> +non-visible pixels outside the visible pixel area may be supported by the
> +sensor. The visible pixel area corresponds to the ``V4L2_SEL_TGT_CROP_DEFAULT``
> +selection target on (pad, stream) pair 1/0.
> +
> +Sensors can perform multiple operations that affect the output image size. First
> +of these is the analogue crop. Analogue crop limits the area of the pixel array
> +which the sensor will read, affecting sensor timing as well. The granularity of
> +the analogue crop configuration varies greatly across sensors: some sensors
> +support only a few different analogue crop configurations whereas others may
> +support anything divisible by a given number of pixels. The analogue crop
> +configuration corresponds the ``V4L2_SEL_TGT_CROP`` selection target on (pad,
s/ corresponds the/ corresponds to the

> +stream) pair 1/0. The default analogue crop rectangle corresponds to the visible
> +pixel area.
> +
> +In the next step, binning is performed on the image data read from camera
> +sensor's pixel array, as determined by the analogue crop configuration. Enabling
> +binning will effectively result in an image smaller than the original by given
> +binning factors horizontally and vertically. Typical values are 1/2 and 1/3 but
> +others may well be supported by the hardware as well.
> +
> +Sub-sampling follows binning. Sub-sampling, like binning, reduces the size of
> +the image by including only a subset of samples read from the sensor's pixel
> +matrix, typically every n'th pixel horizontally and vertically, taking the
> +sensor's colour pattern into account. Sub-sampling is generally configurable
> +separately horizontally and vertically.
> +
> +Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE``
> +rectangle, relative to the analogue crop rectangle, on (pad, stream) pair
> +1/0. The driver implementation determines how to configure binning and
> +sub-sampling to achieve the desired size.
> +
> +The digital crop operation takes place after binning and sub-sampling. It is
> +configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
> +0/0. The resulting image size is further output by the sensor.
> +
> +The sensor's output mbus code is configured by setting the format on the (pad,
> +stream) pair 0/0. When setting the format, always use the same width and height
> +as for the digital crop setting.
> +
> +Drivers may only support some of even none of these configurations, in which
s/some of/some or

> +case they do not expose the corresponding selection rectangles. If any selection
> +targets are omitted, the further selection rectangle or format is instead
> +related to the previous implemented selection rectangle. For instance, if the
> +sensor supports binning but not analogue crop, then the binning configuration
> +(``V4L2_SEL_TGT_COMPOSE`` selection target) is done in relation to the visible
> +pixel area (``V4L2_SEL_TGT_CROP_DEFAULT`` selection target).
> +
> +Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
> +
> +.. flat-table:: Selection targets on pads
> +    :header-rows: 1
> +
> +    * - Pad/Stream
> +      - Selection target/format
> +      - Mandatory (X/-)
> +      - Modifiable (X/-)
> +      - Synopsis
> +    * - 1/0
> +      - Format
> +      - X
> +      - \-
> +      - Image data format. The width and the height fields indicates the full
> +        size of the pixel array, including non-visible pixels. The media bus
> +        code of this format reflects the native pixel depth of the sensor.
> +    * - 1/0
> +      - ``V4L2_SEL_TGT_CROP_DEFAULT``
> +      - X
> +      - \
> +      - The visible pixel area. This rectangle is relative to the format on the
> +        same (pad, stream).
> +    * - 1/0
> +      - ``V4L2_SEL_TGT_CROP``
> +      - \-
> +      - X
> +      - Analogue crop. Analogue crop typically has a coarse granularity. This
> +        rectangle is relative to the format on the same (pad, stream).
> +    * - 1/0
> +      - ``V4L2_SEL_TGT_COMPOSE``
> +      - \-
> +      - X
> +      - Binning and sub-sampling. This rectangle is relative to the
> +        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The
> +        combination of binning and sub-sampling is configured using this
> +        selection target.
> +    * - 2/0
> +      - Format
> +      - X
> +      - \-
> +      - Embedded data format.
> +    * - 0/0
> +      - ``V4L2_SEL_TGT_CROP``
> +      - \-
> +      - X
> +      - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE``
> +        rectangle on (pad, stream) pair 1/0.
> +    * - 0/0
> +      - Format
> +      - X
> +      - X
> +      - Image data source format. Always assign the width and height fields of
> +        the format to the same values than for the ``V4L2_SEL_TGT_CROP``
> +        rectangle on (pad, stream) pair 0/0. The media bus code reflects the
> +        pixel data output of the sensor.
> +    * - 0/1
> +      - Format
> +      - X
> +      - \-
> +      - Embedded data source format.
> +
> +Embedded data
> +^^^^^^^^^^^^^
> +
> +The embedded data stream is produced by the sensor when the corresponding route
> +is enabled. The embedded data route may also be immutable or not exist at all,
> +in case the sensor (or the driver) does not support it.
> +
> +Generally the sensor embedded data width is determined by the width of the image
> +data whereas the number of lines are constant for the embedded data. The user
> +space may obtain the size of the embedded data once the image data size on the
> +source pad has been configured.
> +
> +Also see :ref:`media_using_camera_sensor_drivers_embedded_data`.
> --
> 2.39.5
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>

Cheers,
Prabhakar

Re: [RFC v5 05/15] media: Documentation: Add scaling and post-scaler crop for common raw

[Lad, Prabhakar]

On Mon, Feb 3, 2025 at 8:59 AM Sakari Ailus
<sakari.ailus@linux.intel.com> wrote:
>
> Document scaling and post-scaler digital crop operations for the common
> raw sensor model.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  .../media/v4l/subdev-config-model.rst         | 24 +++++++++++++++----
>  1 file changed, 19 insertions(+), 5 deletions(-)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> index b0bd09772ceb..f15e5495cc34 100644
> --- a/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> +++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> @@ -118,11 +118,18 @@ sub-sampling to achieve the desired size.
>
>  The digital crop operation takes place after binning and sub-sampling. It is
>  configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
> -0/0. The resulting image size is further output by the sensor.
> +0/0.
> +
> +The scaling operation is performed after the digital crop. It is configured by
> +setting the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 0/0,
> +relative to the digital crop.
>
>  The sensor's output mbus code is configured by setting the format on the (pad,
> -stream) pair 0/0. When setting the format, always use the same width and height
> -as for the digital crop setting.
> +stream) pair 0/0. The width and height fields are used to configure post-scaler
> +digital crop if supported by the driver, affecting the right and bottom edges of
> +the frame. If post-scaler digital crop is not supported, the width and height
> +fields of the format will match the compose rectangle sizes applied on the same
> +0/0 (pad, stream) pair.
>
>  Drivers may only support some of even none of these configurations, in which
>  case they do not expose the corresponding selection rectangles. If any selection
> @@ -180,12 +187,19 @@ Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
>        - X
>        - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE``
>          rectangle on (pad, stream) pair 1/0.
> +    * - 0/0
> +      - ``V4L2_SEL_TGT_COMPOSE``
> +      - \-
> +      - X
> +      - Scaling. This rectangle is relative to the ``V4L2_SEL_TGT_CROP``
> +        rectangle on (pad, stream) pair 0/0.
>      * - 0/0
>        - Format
>        - X
>        - X
> -      - Image data source format. Always assign the width and height fields of
> -        the format to the same values than for the ``V4L2_SEL_TGT_CROP``
> +      - Image data source format and post-scaler crop. The width and height
> +        fields of the format, used to configure post-scaler crop on the right
> +        and bottom edges of the image, are related to the ``V4L2_SEL_TGT_COMPOSE``
>          rectangle on (pad, stream) pair 0/0. The media bus code reflects the
>          pixel data output of the sensor.
>      * - 0/1
> --
> 2.39.5
>

Re: [RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control

[Lad, Prabhakar]

On Mon, Feb 3, 2025 at 8:59 AM Sakari Ailus
<sakari.ailus@linux.intel.com> wrote:
>
> Add the V4L2_CID_CONFIG_MODEL control for the configuration model.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 4 ++++
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c                    | 5 +++++
>  include/uapi/linux/v4l2-controls.h                           | 3 +++
>  3 files changed, 12 insertions(+)
>
Reviewed-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>

Cheers,
Prabhakar

> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> index 27803dca8d3e..2ae17ed99729 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> @@ -55,3 +55,7 @@ Image Process Control IDs
>      control value divided by e.g. 0x100, meaning that to get no
>      digital gain the control value needs to be 0x100. The no-gain
>      configuration is also typically the default.
> +
> +``V4L2_CID_CONFIG_MODEL (bitmask)``
> +    Which configuration models the sub-device supports. Please see
> +    :ref:`media_subdev_config_model`. This is a read-only control.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 1ea52011247a..24c9c25e20d1 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>         case V4L2_CID_TEST_PATTERN:             return "Test Pattern";
>         case V4L2_CID_DEINTERLACING_MODE:       return "Deinterlacing Mode";
>         case V4L2_CID_DIGITAL_GAIN:             return "Digital Gain";
> +       case V4L2_CID_CONFIG_MODEL:             return "Sub-device configuration model";
>
>         /* DV controls */
>         /* Keep the order of the 'case's the same as in v4l2-controls.h! */
> @@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>         case V4L2_CID_DV_RX_POWER_PRESENT:
>                 *type = V4L2_CTRL_TYPE_BITMASK;
>                 break;
> +       case V4L2_CID_CONFIG_MODEL:
> +               *flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +               *type = V4L2_CTRL_TYPE_BITMASK;
> +               break;
>         case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
>         case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
>                 *type = V4L2_CTRL_TYPE_INTEGER;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 974fd254e573..731add75d9ee 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {
>  #define V4L2_CID_TEST_PATTERN                  (V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)
>  #define V4L2_CID_DEINTERLACING_MODE            (V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)
>  #define V4L2_CID_DIGITAL_GAIN                  (V4L2_CID_IMAGE_PROC_CLASS_BASE + 5)
> +#define V4L2_CID_CONFIG_MODEL                  (V4L2_CID_IMAGE_PROC_CLASS_BASE + 6)
> +
> +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW       (1U << 0)
>
>  /*  DV-class control IDs defined by V4L2 */
>  #define V4L2_CID_DV_CLASS_BASE                 (V4L2_CTRL_CLASS_DV | 0x900)
> --
> 2.39.5
>

Re: RFC v5 01/15] media: Documentation: Rework embedded data documentation

[Mirela Rabulea]

Hi Sakari,

On 03.02.2025 10:58, Sakari Ailus wrote:
> Rework embedded data documentation by removing the reference to the pixel
> data stream. The specific documentation of the embedded data interface
> will be elsewhere, in the near future either CCS or common raw camera
> sensor model documentation.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>   .../userspace-api/media/drivers/camera-sensor.rst   | 13 ++++++-------
>   1 file changed, 6 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index dc415b8f6c8e..8e1083417ae1 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -111,13 +111,12 @@ the sensor configuration for the captured frame back to the host. While CSI-2 is
>   the most common data interface used by such sensors, embedded data can be
>   available on other interfaces as well.
>
> -Such sensors expose two internal sink pads (pads that have both the
> -``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and ``MEDIA_PAD_FL_INTERNAL
> -<MEDIA-PAD-FL-INTERNAL>`` flags set) to model the source of the image and
> -embedded data streams. Both of these pads produces a single stream, and the
> -sub-device routes those streams to the external (source) pad. If the sub-device
> -driver supports disabling embedded data, this can be done by disabling the
> -embedded data route via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL.
> +Embedded data support is indicated by the precence of an internal sink pad (pad

Just a small typo here: precence.

Other than that, Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>

Regards,

Mirela

> +that has both the ``MEDIA_PAD_FL_SINK <MEDIA-PAD-FL-SINK>`` and
> +``MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`` flags set) with a metadata
> +format to model the embedded data stream. If the sub-device driver supports
> +disabling embedded data, this can be done by disabling the embedded data route
> +via the ``VIDIOC_SUBDEV_S_ROUTING`` IOCTL.
>
>   In general, changing the embedded data format from the driver-configured values
>   is not supported. The height of the metadata is device-specific and the width
> --
> 2.39.5
>

Re: [RFC v5 04/15] media: Documentation: Add subdev configuration models, raw sensor model

[Mirela Rabulea]

Hi Sakari,

Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>

Regards,

Mirela

On 03.02.2025 10:58, Sakari Ailus wrote:
> Sub-device configuration models define what V4L2 API elements are
> available on a compliant sub-device and how do they behave.
>
> The patch also adds a model for common raw sensors.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>   .../media/drivers/camera-sensor.rst           |   4 +
>   .../media/v4l/common-raw-sensor.dia           | 441 ++++++++++++++++++
>   .../media/v4l/common-raw-sensor.svg           | 134 ++++++
>   .../userspace-api/media/v4l/dev-subdev.rst    |   2 +
>   .../media/v4l/subdev-config-model.rst         | 209 +++++++++
>   5 files changed, 790 insertions(+)
>   create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.dia
>   create mode 100644 Documentation/userspace-api/media/v4l/common-raw-sensor.svg
>   create mode 100644 Documentation/userspace-api/media/v4l/subdev-config-model.rst
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index bc55c861fb69..5bc4c79d230c 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -18,6 +18,8 @@ binning functionality. The sensor drivers belong to two distinct classes, freely
>   configurable and register list-based drivers, depending on how the driver
>   configures this functionality.
>
> +Also see :ref:`media_subdev_config_model_common_raw_sensor`.
> +
>   Freely configurable camera sensor drivers
>   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> @@ -105,6 +107,8 @@ values programmed by the register sequences. The default values of these
>   controls shall be 0 (disabled). Especially these controls shall not be inverted,
>   independently of the sensor's mounting rotation.
>
> +.. _media_using_camera_sensor_drivers_embedded_data:
> +
>   Embedded data
>   -------------
>
> diff --git a/Documentation/userspace-api/media/v4l/common-raw-sensor.dia b/Documentation/userspace-api/media/v4l/common-raw-sensor.dia
> new file mode 100644
> index 000000000000..aa927527eae3
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/common-raw-sensor.dia
> @@ -0,0 +1,441 @@
> +<?xml version="1.0" encoding="UTF-8"?>
> +<dia:diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/">
> +  <dia:diagramdata>
> +    <dia:attribute name="background">
> +      <dia:color val="#ffffffff"/>
> +    </dia:attribute>
> +    <dia:attribute name="pagebreak">
> +      <dia:color val="#000099ff"/>
> +    </dia:attribute>
> +    <dia:attribute name="paper">
> +      <dia:composite type="paper">
> +        <dia:attribute name="name">
> +          <dia:string>#A4#</dia:string>
> +        </dia:attribute>
> +        <dia:attribute name="tmargin">
> +          <dia:real val="2.8222"/>
> +        </dia:attribute>
> +        <dia:attribute name="bmargin">
> +          <dia:real val="2.8222"/>
> +        </dia:attribute>
> +        <dia:attribute name="lmargin">
> +          <dia:real val="2.8222"/>
> +        </dia:attribute>
> +        <dia:attribute name="rmargin">
> +          <dia:real val="2.8222"/>
> +        </dia:attribute>
> +        <dia:attribute name="is_portrait">
> +          <dia:boolean val="true"/>
> +        </dia:attribute>
> +        <dia:attribute name="scaling">
> +          <dia:real val="1"/>
> +        </dia:attribute>
> +        <dia:attribute name="fitto">
> +          <dia:boolean val="false"/>
> +        </dia:attribute>
> +      </dia:composite>
> +    </dia:attribute>
> +    <dia:attribute name="grid">
> +      <dia:composite type="grid">
> +        <dia:attribute name="dynamic">
> +          <dia:boolean val="true"/>
> +        </dia:attribute>
> +        <dia:attribute name="width_x">
> +          <dia:real val="1"/>
> +        </dia:attribute>
> +        <dia:attribute name="width_y">
> +          <dia:real val="1"/>
> +        </dia:attribute>
> +        <dia:attribute name="visible_x">
> +          <dia:int val="1"/>
> +        </dia:attribute>
> +        <dia:attribute name="visible_y">
> +          <dia:int val="1"/>
> +        </dia:attribute>
> +        <dia:composite type="color"/>
> +      </dia:composite>
> +    </dia:attribute>
> +    <dia:attribute name="color">
> +      <dia:color val="#d8e5e5ff"/>
> +    </dia:attribute>
> +    <dia:attribute name="guides"/>
> +    <dia:attribute name="guide_color">
> +      <dia:color val="#00ff00ff"/>
> +    </dia:attribute>
> +    <dia:attribute name="display">
> +      <dia:composite type="display">
> +        <dia:attribute name="antialiased">
> +          <dia:boolean val="true"/>
> +        </dia:attribute>
> +        <dia:attribute name="snap-to-grid">
> +          <dia:boolean val="false"/>
> +        </dia:attribute>
> +        <dia:attribute name="snap-to-guides">
> +          <dia:boolean val="true"/>
> +        </dia:attribute>
> +        <dia:attribute name="snap-to-object">
> +          <dia:boolean val="true"/>
> +        </dia:attribute>
> +        <dia:attribute name="show-grid">
> +          <dia:boolean val="true"/>
> +        </dia:attribute>
> +        <dia:attribute name="show-guides">
> +          <dia:boolean val="true"/>
> +        </dia:attribute>
> +        <dia:attribute name="show-connection-points">
> +          <dia:boolean val="true"/>
> +        </dia:attribute>
> +      </dia:composite>
> +    </dia:attribute>
> +  </dia:diagramdata>
> +  <dia:layer name="Background" visible="true" connectable="false"/>
> +  <dia:layer name="Background" visible="true" connectable="false"/>
> +  <dia:layer name="Background" visible="true" connectable="true" active="true">
> +    <dia:object type="Standard - Box" version="0" id="O0">
> +      <dia:attribute name="obj_pos">
> +        <dia:point val="16.35,11.5"/>
> +      </dia:attribute>
> +      <dia:attribute name="obj_bb">
> +        <dia:rectangle val="16.3,11.45;25.8,18.45"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_corner">
> +        <dia:point val="16.35,11.5"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_width">
> +        <dia:real val="9.4000000000000021"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_height">
> +        <dia:real val="6.9000000000000021"/>
> +      </dia:attribute>
> +      <dia:attribute name="show_background">
> +        <dia:boolean val="true"/>
> +      </dia:attribute>
> +    </dia:object>
> +    <dia:object type="Geometric - Perfect Circle" version="1" id="O1">
> +      <dia:attribute name="obj_pos">
> +        <dia:point val="25.5627,14.578"/>
> +      </dia:attribute>
> +      <dia:attribute name="obj_bb">
> +        <dia:rectangle val="25.4627,14.478;26.0191,15.0344"/>
> +      </dia:attribute>
> +      <dia:attribute name="meta">
> +        <dia:composite type="dict"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_corner">
> +        <dia:point val="25.5627,14.578"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_width">
> +        <dia:real val="0.35638032780853468"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_height">
> +        <dia:real val="0.35638032780853468"/>
> +      </dia:attribute>
> +      <dia:attribute name="line_width">
> +        <dia:real val="0.10000000000000001"/>
> +      </dia:attribute>
> +      <dia:attribute name="line_colour">
> +        <dia:color val="#000000ff"/>
> +      </dia:attribute>
> +      <dia:attribute name="fill_colour">
> +        <dia:color val="#ffffffff"/>
> +      </dia:attribute>
> +      <dia:attribute name="show_background">
> +        <dia:boolean val="true"/>
> +      </dia:attribute>
> +      <dia:attribute name="line_style">
> +        <dia:enum val="0"/>
> +        <dia:real val="1"/>
> +      </dia:attribute>
> +      <dia:attribute name="flip_horizontal">
> +        <dia:boolean val="false"/>
> +      </dia:attribute>
> +      <dia:attribute name="flip_vertical">
> +        <dia:boolean val="false"/>
> +      </dia:attribute>
> +      <dia:attribute name="subscale">
> +        <dia:real val="1"/>
> +      </dia:attribute>
> +    </dia:object>
> +    <dia:object type="Geometric - Perfect Circle" version="1" id="O2">
> +      <dia:attribute name="obj_pos">
> +        <dia:point val="17.7702,13.2656"/>
> +      </dia:attribute>
> +      <dia:attribute name="obj_bb">
> +        <dia:rectangle val="17.6702,13.1656;18.24,13.7354"/>
> +      </dia:attribute>
> +      <dia:attribute name="meta">
> +        <dia:composite type="dict"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_corner">
> +        <dia:point val="17.7702,13.2656"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_width">
> +        <dia:real val="0.36980024191863681"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_height">
> +        <dia:real val="0.36980024191863681"/>
> +      </dia:attribute>
> +      <dia:attribute name="line_width">
> +        <dia:real val="0.10000000000000001"/>
> +      </dia:attribute>
> +      <dia:attribute name="line_colour">
> +        <dia:color val="#000000ff"/>
> +      </dia:attribute>
> +      <dia:attribute name="fill_colour">
> +        <dia:color val="#ffffffff"/>
> +      </dia:attribute>
> +      <dia:attribute name="show_background">
> +        <dia:boolean val="true"/>
> +      </dia:attribute>
> +      <dia:attribute name="line_style">
> +        <dia:enum val="0"/>
> +        <dia:real val="1"/>
> +      </dia:attribute>
> +      <dia:attribute name="flip_horizontal">
> +        <dia:boolean val="false"/>
> +      </dia:attribute>
> +      <dia:attribute name="flip_vertical">
> +        <dia:boolean val="false"/>
> +      </dia:attribute>
> +      <dia:attribute name="subscale">
> +        <dia:real val="1"/>
> +      </dia:attribute>
> +    </dia:object>
> +    <dia:object type="Geometric - Perfect Circle" version="1" id="O3">
> +      <dia:attribute name="obj_pos">
> +        <dia:point val="17.7464,16.2056"/>
> +      </dia:attribute>
> +      <dia:attribute name="obj_bb">
> +        <dia:rectangle val="17.6464,16.1056;18.2162,16.6754"/>
> +      </dia:attribute>
> +      <dia:attribute name="meta">
> +        <dia:composite type="dict"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_corner">
> +        <dia:point val="17.7464,16.2056"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_width">
> +        <dia:real val="0.36980024191863681"/>
> +      </dia:attribute>
> +      <dia:attribute name="elem_height">
> +        <dia:real val="0.36980024191863681"/>
> +      </dia:attribute>
> +      <dia:attribute name="line_width">
> +        <dia:real val="0.10000000000000001"/>
> +      </dia:attribute>
> +      <dia:attribute name="line_colour">
> +        <dia:color val="#000000ff"/>
> +      </dia:attribute>
> +      <dia:attribute name="fill_colour">
> +        <dia:color val="#ffffffff"/>
> +      </dia:attribute>
> +      <dia:attribute name="show_background">
> +        <dia:boolean val="true"/>
> +      </dia:attribute>
> +      <dia:attribute name="line_style">
> +        <dia:enum val="0"/>
> +        <dia:real val="1"/>
> +      </dia:attribute>
> +      <dia:attribute name="flip_horizontal">
> +        <dia:boolean val="false"/>
> +      </dia:attribute>
> +      <dia:attribute name="flip_vertical">
> +        <dia:boolean val="false"/>
> +      </dia:attribute>
> +      <dia:attribute name="subscale">
> +        <dia:real val="1"/>
> +      </dia:attribute>
> +    </dia:object>
> +    <dia:object type="Standard - Line" version="0" id="O4">
> +      <dia:attribute name="obj_pos">
> +        <dia:point val="18.1609,16.3413"/>
> +      </dia:attribute>
> +      <dia:attribute name="obj_bb">
> +        <dia:rectangle val="18.1016,14.5712;25.6221,16.4007"/>
> +      </dia:attribute>
> +      <dia:attribute name="conn_endpoints">
> +        <dia:point val="18.1609,16.3413"/>
> +        <dia:point val="25.5627,14.7562"/>
> +      </dia:attribute>
> +      <dia:attribute name="numcp">
> +        <dia:int val="1"/>
> +      </dia:attribute>
> +      <dia:attribute name="line_style">
> +        <dia:enum val="1"/>
> +      </dia:attribute>
> +      <dia:attribute name="end_arrow">
> +        <dia:enum val="1"/>
> +      </dia:attribute>
> +      <dia:attribute name="end_arrow_length">
> +        <dia:real val="0.5"/>
> +      </dia:attribute>
> +      <dia:attribute name="end_arrow_width">
> +        <dia:real val="0.5"/>
> +      </dia:attribute>
> +      <dia:connections>
> +        <dia:connection handle="0" to="O3" connection="8"/>
> +        <dia:connection handle="1" to="O1" connection="2"/>
> +      </dia:connections>
> +    </dia:object>
> +    <dia:object type="Standard - Line" version="0" id="O5">
> +      <dia:attribute name="obj_pos">
> +        <dia:point val="18.14,13.4505"/>
> +      </dia:attribute>
> +      <dia:attribute name="obj_bb">
> +        <dia:rectangle val="18.0821,13.3926;25.6206,14.9674"/>
> +      </dia:attribute>
> +      <dia:attribute name="conn_endpoints">
> +        <dia:point val="18.14,13.4505"/>
> +        <dia:point val="25.5627,14.7562"/>
> +      </dia:attribute>
> +      <dia:attribute name="numcp">
> +        <dia:int val="1"/>
> +      </dia:attribute>
> +      <dia:attribute name="end_arrow">
> +        <dia:enum val="1"/>
> +      </dia:attribute>
> +      <dia:attribute name="end_arrow_length">
> +        <dia:real val="0.5"/>
> +      </dia:attribute>
> +      <dia:attribute name="end_arrow_width">
> +        <dia:real val="0.5"/>
> +      </dia:attribute>
> +      <dia:connections>
> +        <dia:connection handle="0" to="O2" connection="3"/>
> +        <dia:connection handle="1" to="O1" connection="2"/>
> +      </dia:connections>
> +    </dia:object>
> +    <dia:object type="Standard - Line" version="0" id="O6">
> +      <dia:attribute name="obj_pos">
> +        <dia:point val="25.919,14.7562"/>
> +      </dia:attribute>
> +      <dia:attribute name="obj_bb">
> +        <dia:rectangle val="25.8686,14.3879;31.0068,15.0497"/>
> +      </dia:attribute>
> +      <dia:attribute name="conn_endpoints">
> +        <dia:point val="25.919,14.7562"/>
> +        <dia:point val="30.9564,14.7131"/>
> +      </dia:attribute>
> +      <dia:attribute name="numcp">
> +        <dia:int val="1"/>
> +      </dia:attribute>
> +      <dia:attribute name="end_arrow">
> +        <dia:enum val="1"/>
> +      </dia:attribute>
> +      <dia:attribute name="end_arrow_length">
> +        <dia:real val="0.5"/>
> +      </dia:attribute>
> +      <dia:attribute name="end_arrow_width">
> +        <dia:real val="0.5"/>
> +      </dia:attribute>
> +      <dia:connections>
> +        <dia:connection handle="0" to="O1" connection="3"/>
> +      </dia:connections>
> +    </dia:object>
> +    <dia:object type="Standard - Text" version="1" id="O7">
> +      <dia:attribute name="obj_pos">
> +        <dia:point val="17.9551,13.2656"/>
> +      </dia:attribute>
> +      <dia:attribute name="obj_bb">
> +        <dia:rectangle val="17.9551,12.5181;22.7051,13.2656"/>
> +      </dia:attribute>
> +      <dia:attribute name="text">
> +        <dia:composite type="text">
> +          <dia:attribute name="string">
> +            <dia:string>#image data (1)#</dia:string>
> +          </dia:attribute>
> +          <dia:attribute name="font">
> +            <dia:font family="sans" style="0" name="Helvetica"/>
> +          </dia:attribute>
> +          <dia:attribute name="height">
> +            <dia:real val="0.80000000000000004"/>
> +          </dia:attribute>
> +          <dia:attribute name="pos">
> +            <dia:point val="17.9551,13.1131"/>
> +          </dia:attribute>
> +          <dia:attribute name="color">
> +            <dia:color val="#000000ff"/>
> +          </dia:attribute>
> +          <dia:attribute name="alignment">
> +            <dia:enum val="0"/>
> +          </dia:attribute>
> +        </dia:composite>
> +      </dia:attribute>
> +      <dia:attribute name="valign">
> +        <dia:enum val="1"/>
> +      </dia:attribute>
> +      <dia:connections>
> +        <dia:connection handle="0" to="O2" connection="1"/>
> +      </dia:connections>
> +    </dia:object>
> +    <dia:object type="Standard - Text" version="1" id="O8">
> +      <dia:attribute name="obj_pos">
> +        <dia:point val="17.9313,16.5754"/>
> +      </dia:attribute>
> +      <dia:attribute name="obj_bb">
> +        <dia:rectangle val="17.9313,16.5754;24.1238,17.3229"/>
> +      </dia:attribute>
> +      <dia:attribute name="text">
> +        <dia:composite type="text">
> +          <dia:attribute name="string">
> +            <dia:string>#embedded data (2)#</dia:string>
> +          </dia:attribute>
> +          <dia:attribute name="font">
> +            <dia:font family="sans" style="0" name="Helvetica"/>
> +          </dia:attribute>
> +          <dia:attribute name="height">
> +            <dia:real val="0.80000000000000004"/>
> +          </dia:attribute>
> +          <dia:attribute name="pos">
> +            <dia:point val="17.9313,17.1704"/>
> +          </dia:attribute>
> +          <dia:attribute name="color">
> +            <dia:color val="#000000ff"/>
> +          </dia:attribute>
> +          <dia:attribute name="alignment">
> +            <dia:enum val="0"/>
> +          </dia:attribute>
> +        </dia:composite>
> +      </dia:attribute>
> +      <dia:attribute name="valign">
> +        <dia:enum val="0"/>
> +      </dia:attribute>
> +      <dia:connections>
> +        <dia:connection handle="0" to="O3" connection="0"/>
> +      </dia:connections>
> +    </dia:object>
> +    <dia:object type="Standard - Text" version="1" id="O9">
> +      <dia:attribute name="obj_pos">
> +        <dia:point val="26.1,14.125"/>
> +      </dia:attribute>
> +      <dia:attribute name="obj_bb">
> +        <dia:rectangle val="26.1,13.53;30.7475,14.2775"/>
> +      </dia:attribute>
> +      <dia:attribute name="text">
> +        <dia:composite type="text">
> +          <dia:attribute name="string">
> +            <dia:string>#source pad (0)#</dia:string>
> +          </dia:attribute>
> +          <dia:attribute name="font">
> +            <dia:font family="sans" style="0" name="Helvetica"/>
> +          </dia:attribute>
> +          <dia:attribute name="height">
> +            <dia:real val="0.80000000000000004"/>
> +          </dia:attribute>
> +          <dia:attribute name="pos">
> +            <dia:point val="26.1,14.125"/>
> +          </dia:attribute>
> +          <dia:attribute name="color">
> +            <dia:color val="#000000ff"/>
> +          </dia:attribute>
> +          <dia:attribute name="alignment">
> +            <dia:enum val="0"/>
> +          </dia:attribute>
> +        </dia:composite>
> +      </dia:attribute>
> +      <dia:attribute name="valign">
> +        <dia:enum val="3"/>
> +      </dia:attribute>
> +    </dia:object>
> +  </dia:layer>
> +</dia:diagram>
> diff --git a/Documentation/userspace-api/media/v4l/common-raw-sensor.svg b/Documentation/userspace-api/media/v4l/common-raw-sensor.svg
> new file mode 100644
> index 000000000000..1d6055da2519
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/common-raw-sensor.svg
> @@ -0,0 +1,134 @@
> +<?xml version="1.0" encoding="UTF-8"?>
> +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="296pt" height="142pt" viewBox="0 0 296 142" version="1.1">
> +<defs>
> +<g>
> +<symbol overflow="visible" id="glyph0-0">
> +<path style="stroke:none;" d="M 0.640625 2.265625 L 0.640625 -9.015625 L 7.03125 -9.015625 L 7.03125 2.265625 Z M 1.359375 1.546875 L 6.328125 1.546875 L 6.328125 -8.296875 L 1.359375 -8.296875 Z M 1.359375 1.546875 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-1">
> +<path style="stroke:none;" d="M 1.203125 -7 L 2.359375 -7 L 2.359375 0 L 1.203125 0 Z M 1.203125 -9.71875 L 2.359375 -9.71875 L 2.359375 -8.265625 L 1.203125 -8.265625 Z M 1.203125 -9.71875 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-2">
> +<path style="stroke:none;" d="M 6.65625 -5.65625 C 6.9375 -6.164062 7.273438 -6.546875 7.671875 -6.796875 C 8.078125 -7.046875 8.550781 -7.171875 9.09375 -7.171875 C 9.820312 -7.171875 10.382812 -6.914062 10.78125 -6.40625 C 11.175781 -5.894531 11.375 -5.164062 11.375 -4.21875 L 11.375 0 L 10.21875 0 L 10.21875 -4.1875 C 10.21875 -4.851562 10.097656 -5.347656 9.859375 -5.671875 C 9.628906 -6.003906 9.269531 -6.171875 8.78125 -6.171875 C 8.1875 -6.171875 7.710938 -5.972656 7.359375 -5.578125 C 7.015625 -5.179688 6.84375 -4.640625 6.84375 -3.953125 L 6.84375 0 L 5.6875 0 L 5.6875 -4.1875 C 5.6875 -4.863281 5.566406 -5.363281 5.328125 -5.6875 C 5.097656 -6.007812 4.734375 -6.171875 4.234375 -6.171875 C 3.648438 -6.171875 3.179688 -5.96875 2.828125 -5.5625 C 2.484375 -5.164062 2.3125 -4.628906 2.3125 -3.953125 L 2.3125 0 L 1.15625 0 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.90625 C 2.582031 -6.332031 2.898438 -6.648438 3.265625 -6.859375 C 3.628906 -7.066406 4.0625 -7.171875 4.5625 -7.171875 C 5.070312 -7.171875 5.503906 -7.039062 5.859375 -6.78125 C 6.222656 -6.519531 6.488281 -6.144531 6.65625 -5.65625 Z M 6.65625 -5.65625 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-3">
> +<path style="stroke:none;" d="M 4.390625 -3.515625 C 3.460938 -3.515625 2.816406 -3.40625 2.453125 -3.1875 C 2.097656 -2.976562 1.921875 -2.617188 1.921875 -2.109375 C 1.921875 -1.703125 2.050781 -1.378906 2.3125 -1.140625 C 2.582031 -0.898438 2.953125 -0.78125 3.421875 -0.78125 C 4.054688 -0.78125 4.566406 -1.003906 4.953125 -1.453125 C 5.335938 -1.910156 5.53125 -2.515625 5.53125 -3.265625 L 5.53125 -3.515625 Z M 6.671875 -4 L 6.671875 0 L 5.53125 0 L 5.53125 -1.0625 C 5.269531 -0.632812 4.941406 -0.316406 4.546875 -0.109375 C 4.160156 0.0859375 3.679688 0.1875 3.109375 0.1875 C 2.390625 0.1875 1.816406 -0.015625 1.390625 -0.421875 C 0.972656 -0.828125 0.765625 -1.363281 0.765625 -2.03125 C 0.765625 -2.820312 1.023438 -3.414062 1.546875 -3.8125 C 2.078125 -4.21875 2.867188 -4.421875 3.921875 -4.421875 L 5.53125 -4.421875 L 5.53125 -4.53125 C 5.53125 -5.0625 5.351562 -5.46875 5 -5.75 C 4.65625 -6.039062 4.171875 -6.1875 3.546875 -6.1875 C 3.140625 -6.1875 2.75 -6.140625 2.375 -6.046875 C 2 -5.953125 1.632812 -5.8125 1.28125 -5.625 L 1.28125 -6.671875 C 1.695312 -6.835938 2.101562 -6.960938 2.5 -7.046875 C 2.894531 -7.128906 3.28125 -7.171875 3.65625 -7.171875 C 4.675781 -7.171875 5.429688 -6.90625 5.921875 -6.375 C 6.421875 -5.851562 6.671875 -5.0625 6.671875 -4 Z M 6.671875 -4 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-4">
> +<path style="stroke:none;" d="M 5.8125 -3.578125 C 5.8125 -4.410156 5.640625 -5.054688 5.296875 -5.515625 C 4.953125 -5.972656 4.46875 -6.203125 3.84375 -6.203125 C 3.226562 -6.203125 2.75 -5.972656 2.40625 -5.515625 C 2.0625 -5.054688 1.890625 -4.410156 1.890625 -3.578125 C 1.890625 -2.753906 2.0625 -2.113281 2.40625 -1.65625 C 2.75 -1.195312 3.226562 -0.96875 3.84375 -0.96875 C 4.46875 -0.96875 4.953125 -1.195312 5.296875 -1.65625 C 5.640625 -2.113281 5.8125 -2.753906 5.8125 -3.578125 Z M 6.953125 -0.875 C 6.953125 0.320312 6.6875 1.207031 6.15625 1.78125 C 5.632812 2.363281 4.828125 2.65625 3.734375 2.65625 C 3.328125 2.65625 2.945312 2.625 2.59375 2.5625 C 2.238281 2.507812 1.890625 2.421875 1.546875 2.296875 L 1.546875 1.171875 C 1.890625 1.359375 2.222656 1.492188 2.546875 1.578125 C 2.878906 1.671875 3.21875 1.71875 3.5625 1.71875 C 4.3125 1.71875 4.875 1.519531 5.25 1.125 C 5.625 0.726562 5.8125 0.132812 5.8125 -0.65625 L 5.8125 -1.234375 C 5.570312 -0.816406 5.265625 -0.503906 4.890625 -0.296875 C 4.523438 -0.0976562 4.082031 0 3.5625 0 C 2.707031 0 2.015625 -0.328125 1.484375 -0.984375 C 0.960938 -1.640625 0.703125 -2.503906 0.703125 -3.578125 C 0.703125 -4.660156 0.960938 -5.53125 1.484375 -6.1875 C 2.015625 -6.84375 2.707031 -7.171875 3.5625 -7.171875 C 4.082031 -7.171875 4.523438 -7.066406 4.890625 -6.859375 C 5.265625 -6.648438 5.570312 -6.34375 5.8125 -5.9375 L 5.8125 -7 L 6.953125 -7 Z M 6.953125 -0.875 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-5">
> +<path style="stroke:none;" d="M 7.1875 -3.78125 L 7.1875 -3.21875 L 1.90625 -3.21875 C 1.957031 -2.425781 2.195312 -1.820312 2.625 -1.40625 C 3.050781 -1 3.644531 -0.796875 4.40625 -0.796875 C 4.84375 -0.796875 5.269531 -0.847656 5.6875 -0.953125 C 6.101562 -1.066406 6.515625 -1.226562 6.921875 -1.4375 L 6.921875 -0.359375 C 6.515625 -0.179688 6.09375 -0.046875 5.65625 0.046875 C 5.21875 0.140625 4.78125 0.1875 4.34375 0.1875 C 3.21875 0.1875 2.328125 -0.132812 1.671875 -0.78125 C 1.023438 -1.4375 0.703125 -2.320312 0.703125 -3.4375 C 0.703125 -4.582031 1.007812 -5.488281 1.625 -6.15625 C 2.25 -6.832031 3.085938 -7.171875 4.140625 -7.171875 C 5.078125 -7.171875 5.816406 -6.863281 6.359375 -6.25 C 6.910156 -5.644531 7.1875 -4.820312 7.1875 -3.78125 Z M 6.046875 -4.125 C 6.035156 -4.75 5.859375 -5.25 5.515625 -5.625 C 5.171875 -6 4.71875 -6.1875 4.15625 -6.1875 C 3.507812 -6.1875 2.992188 -6.003906 2.609375 -5.640625 C 2.222656 -5.285156 2 -4.78125 1.9375 -4.125 Z M 6.046875 -4.125 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-6">
> +<path style="stroke:none;" d=""/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-7">
> +<path style="stroke:none;" d="M 5.8125 -5.9375 L 5.8125 -9.71875 L 6.953125 -9.71875 L 6.953125 0 L 5.8125 0 L 5.8125 -1.046875 C 5.570312 -0.628906 5.265625 -0.316406 4.890625 -0.109375 C 4.523438 0.0859375 4.082031 0.1875 3.5625 0.1875 C 2.71875 0.1875 2.03125 -0.148438 1.5 -0.828125 C 0.96875 -1.503906 0.703125 -2.394531 0.703125 -3.5 C 0.703125 -4.59375 0.96875 -5.476562 1.5 -6.15625 C 2.03125 -6.832031 2.71875 -7.171875 3.5625 -7.171875 C 4.082031 -7.171875 4.523438 -7.066406 4.890625 -6.859375 C 5.265625 -6.660156 5.570312 -6.351562 5.8125 -5.9375 Z M 1.890625 -3.5 C 1.890625 -2.644531 2.0625 -1.976562 2.40625 -1.5 C 2.757812 -1.019531 3.238281 -0.78125 3.84375 -0.78125 C 4.457031 -0.78125 4.9375 -1.019531 5.28125 -1.5 C 5.632812 -1.976562 5.8125 -2.644531 5.8125 -3.5 C 5.8125 -4.34375 5.632812 -5.003906 5.28125 -5.484375 C 4.9375 -5.960938 4.457031 -6.203125 3.84375 -6.203125 C 3.238281 -6.203125 2.757812 -5.960938 2.40625 -5.484375 C 2.0625 -5.003906 1.890625 -4.34375 1.890625 -3.5 Z M 1.890625 -3.5 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-8">
> +<path style="stroke:none;" d="M 2.34375 -8.984375 L 2.34375 -7 L 4.71875 -7 L 4.71875 -6.109375 L 2.34375 -6.109375 L 2.34375 -2.3125 C 2.34375 -1.738281 2.421875 -1.367188 2.578125 -1.203125 C 2.734375 -1.046875 3.050781 -0.96875 3.53125 -0.96875 L 4.71875 -0.96875 L 4.71875 0 L 3.53125 0 C 2.644531 0 2.03125 -0.164062 1.6875 -0.5 C 1.351562 -0.832031 1.1875 -1.4375 1.1875 -2.3125 L 1.1875 -6.109375 L 0.34375 -6.109375 L 0.34375 -7 L 1.1875 -7 L 1.1875 -8.984375 Z M 2.34375 -8.984375 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-9">
> +<path style="stroke:none;" d="M 3.96875 -9.703125 C 3.40625 -8.742188 2.988281 -7.796875 2.71875 -6.859375 C 2.445312 -5.929688 2.3125 -4.984375 2.3125 -4.015625 C 2.3125 -3.054688 2.445312 -2.101562 2.71875 -1.15625 C 3 -0.21875 3.414062 0.726562 3.96875 1.6875 L 2.96875 1.6875 C 2.34375 0.707031 1.875 -0.253906 1.5625 -1.203125 C 1.25 -2.148438 1.09375 -3.085938 1.09375 -4.015625 C 1.09375 -4.941406 1.25 -5.875 1.5625 -6.8125 C 1.875 -7.757812 2.34375 -8.722656 2.96875 -9.703125 Z M 3.96875 -9.703125 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-10">
> +<path style="stroke:none;" d="M 1.59375 -1.0625 L 3.65625 -1.0625 L 3.65625 -8.171875 L 1.40625 -7.734375 L 1.40625 -8.875 L 3.640625 -9.328125 L 4.90625 -9.328125 L 4.90625 -1.0625 L 6.953125 -1.0625 L 6.953125 0 L 1.59375 0 Z M 1.59375 -1.0625 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-11">
> +<path style="stroke:none;" d="M 1.03125 -9.703125 L 2.03125 -9.703125 C 2.65625 -8.722656 3.117188 -7.757812 3.421875 -6.8125 C 3.734375 -5.875 3.890625 -4.941406 3.890625 -4.015625 C 3.890625 -3.085938 3.734375 -2.148438 3.421875 -1.203125 C 3.117188 -0.253906 2.65625 0.707031 2.03125 1.6875 L 1.03125 1.6875 C 1.582031 0.726562 1.992188 -0.21875 2.265625 -1.15625 C 2.535156 -2.101562 2.671875 -3.054688 2.671875 -4.015625 C 2.671875 -4.984375 2.535156 -5.929688 2.265625 -6.859375 C 1.992188 -7.796875 1.582031 -8.742188 1.03125 -9.703125 Z M 1.03125 -9.703125 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-12">
> +<path style="stroke:none;" d="M 6.234375 -3.5 C 6.234375 -4.34375 6.054688 -5.003906 5.703125 -5.484375 C 5.359375 -5.960938 4.882812 -6.203125 4.28125 -6.203125 C 3.664062 -6.203125 3.179688 -5.960938 2.828125 -5.484375 C 2.484375 -5.003906 2.3125 -4.34375 2.3125 -3.5 C 2.3125 -2.644531 2.484375 -1.976562 2.828125 -1.5 C 3.179688 -1.019531 3.664062 -0.78125 4.28125 -0.78125 C 4.882812 -0.78125 5.359375 -1.019531 5.703125 -1.5 C 6.054688 -1.976562 6.234375 -2.644531 6.234375 -3.5 Z M 2.3125 -5.9375 C 2.5625 -6.351562 2.867188 -6.660156 3.234375 -6.859375 C 3.597656 -7.066406 4.039062 -7.171875 4.5625 -7.171875 C 5.40625 -7.171875 6.09375 -6.832031 6.625 -6.15625 C 7.15625 -5.476562 7.421875 -4.59375 7.421875 -3.5 C 7.421875 -2.394531 7.15625 -1.503906 6.625 -0.828125 C 6.09375 -0.148438 5.40625 0.1875 4.5625 0.1875 C 4.039062 0.1875 3.597656 0.0859375 3.234375 -0.109375 C 2.867188 -0.316406 2.5625 -0.628906 2.3125 -1.046875 L 2.3125 0 L 1.15625 0 L 1.15625 -9.71875 L 2.3125 -9.71875 Z M 2.3125 -5.9375 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-13">
> +<path style="stroke:none;" d="M 2.453125 -1.0625 L 6.859375 -1.0625 L 6.859375 0 L 0.9375 0 L 0.9375 -1.0625 C 1.414062 -1.5625 2.066406 -2.226562 2.890625 -3.0625 C 3.722656 -3.894531 4.242188 -4.429688 4.453125 -4.671875 C 4.859375 -5.128906 5.140625 -5.515625 5.296875 -5.828125 C 5.460938 -6.140625 5.546875 -6.445312 5.546875 -6.75 C 5.546875 -7.25 5.367188 -7.65625 5.015625 -7.96875 C 4.671875 -8.28125 4.21875 -8.4375 3.65625 -8.4375 C 3.257812 -8.4375 2.84375 -8.363281 2.40625 -8.21875 C 1.96875 -8.082031 1.5 -7.878906 1 -7.609375 L 1 -8.875 C 1.507812 -9.082031 1.984375 -9.238281 2.421875 -9.34375 C 2.867188 -9.445312 3.273438 -9.5 3.640625 -9.5 C 4.609375 -9.5 5.378906 -9.253906 5.953125 -8.765625 C 6.523438 -8.285156 6.8125 -7.640625 6.8125 -6.828125 C 6.8125 -6.453125 6.738281 -6.09375 6.59375 -5.75 C 6.445312 -5.40625 6.1875 -5 5.8125 -4.53125 C 5.707031 -4.40625 5.375 -4.054688 4.8125 -3.484375 C 4.257812 -2.910156 3.472656 -2.101562 2.453125 -1.0625 Z M 2.453125 -1.0625 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-14">
> +<path style="stroke:none;" d="M 5.671875 -6.796875 L 5.671875 -5.703125 C 5.347656 -5.867188 5.007812 -5.992188 4.65625 -6.078125 C 4.300781 -6.160156 3.9375 -6.203125 3.5625 -6.203125 C 3 -6.203125 2.570312 -6.113281 2.28125 -5.9375 C 2 -5.769531 1.859375 -5.507812 1.859375 -5.15625 C 1.859375 -4.882812 1.957031 -4.671875 2.15625 -4.515625 C 2.363281 -4.367188 2.773438 -4.226562 3.390625 -4.09375 L 3.78125 -4 C 4.601562 -3.832031 5.1875 -3.585938 5.53125 -3.265625 C 5.875 -2.941406 6.046875 -2.5 6.046875 -1.9375 C 6.046875 -1.28125 5.785156 -0.757812 5.265625 -0.375 C 4.753906 0 4.050781 0.1875 3.15625 0.1875 C 2.78125 0.1875 2.390625 0.148438 1.984375 0.078125 C 1.578125 0.00390625 1.144531 -0.101562 0.6875 -0.25 L 0.6875 -1.4375 C 1.113281 -1.21875 1.53125 -1.050781 1.9375 -0.9375 C 2.351562 -0.832031 2.765625 -0.78125 3.171875 -0.78125 C 3.710938 -0.78125 4.128906 -0.875 4.421875 -1.0625 C 4.710938 -1.25 4.859375 -1.507812 4.859375 -1.84375 C 4.859375 -2.15625 4.753906 -2.394531 4.546875 -2.5625 C 4.335938 -2.726562 3.875 -2.890625 3.15625 -3.046875 L 2.765625 -3.140625 C 2.046875 -3.285156 1.53125 -3.515625 1.21875 -3.828125 C 0.90625 -4.140625 0.75 -4.566406 0.75 -5.109375 C 0.75 -5.765625 0.976562 -6.269531 1.4375 -6.625 C 1.90625 -6.988281 2.570312 -7.171875 3.4375 -7.171875 C 3.851562 -7.171875 4.25 -7.140625 4.625 -7.078125 C 5 -7.015625 5.347656 -6.921875 5.671875 -6.796875 Z M 5.671875 -6.796875 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-15">
> +<path style="stroke:none;" d="M 3.921875 -6.1875 C 3.304688 -6.1875 2.816406 -5.945312 2.453125 -5.46875 C 2.097656 -4.988281 1.921875 -4.332031 1.921875 -3.5 C 1.921875 -2.65625 2.097656 -1.992188 2.453125 -1.515625 C 2.804688 -1.035156 3.296875 -0.796875 3.921875 -0.796875 C 4.535156 -0.796875 5.019531 -1.035156 5.375 -1.515625 C 5.726562 -2.003906 5.90625 -2.664062 5.90625 -3.5 C 5.90625 -4.320312 5.726562 -4.972656 5.375 -5.453125 C 5.019531 -5.941406 4.535156 -6.1875 3.921875 -6.1875 Z M 3.921875 -7.171875 C 4.921875 -7.171875 5.703125 -6.84375 6.265625 -6.1875 C 6.835938 -5.539062 7.125 -4.644531 7.125 -3.5 C 7.125 -2.351562 6.835938 -1.453125 6.265625 -0.796875 C 5.703125 -0.140625 4.921875 0.1875 3.921875 0.1875 C 2.910156 0.1875 2.117188 -0.140625 1.546875 -0.796875 C 0.984375 -1.453125 0.703125 -2.351562 0.703125 -3.5 C 0.703125 -4.644531 0.984375 -5.539062 1.546875 -6.1875 C 2.117188 -6.84375 2.910156 -7.171875 3.921875 -7.171875 Z M 3.921875 -7.171875 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-16">
> +<path style="stroke:none;" d="M 1.09375 -2.765625 L 1.09375 -7 L 2.234375 -7 L 2.234375 -2.8125 C 2.234375 -2.144531 2.363281 -1.644531 2.625 -1.3125 C 2.882812 -0.976562 3.269531 -0.8125 3.78125 -0.8125 C 4.40625 -0.8125 4.894531 -1.007812 5.25 -1.40625 C 5.613281 -1.800781 5.796875 -2.34375 5.796875 -3.03125 L 5.796875 -7 L 6.953125 -7 L 6.953125 0 L 5.796875 0 L 5.796875 -1.078125 C 5.515625 -0.648438 5.191406 -0.332031 4.828125 -0.125 C 4.460938 0.0820312 4.035156 0.1875 3.546875 0.1875 C 2.742188 0.1875 2.132812 -0.0625 1.71875 -0.5625 C 1.300781 -1.0625 1.09375 -1.796875 1.09375 -2.765625 Z M 3.984375 -7.171875 Z M 3.984375 -7.171875 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-17">
> +<path style="stroke:none;" d="M 5.265625 -5.921875 C 5.128906 -5.992188 4.984375 -6.046875 4.828125 -6.078125 C 4.679688 -6.117188 4.519531 -6.140625 4.34375 -6.140625 C 3.6875 -6.140625 3.179688 -5.925781 2.828125 -5.5 C 2.484375 -5.082031 2.3125 -4.476562 2.3125 -3.6875 L 2.3125 0 L 1.15625 0 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.90625 C 2.5625 -6.332031 2.878906 -6.648438 3.265625 -6.859375 C 3.648438 -7.066406 4.117188 -7.171875 4.671875 -7.171875 C 4.753906 -7.171875 4.84375 -7.164062 4.9375 -7.15625 C 5.03125 -7.144531 5.132812 -7.128906 5.25 -7.109375 Z M 5.265625 -5.921875 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-18">
> +<path style="stroke:none;" d="M 6.25 -6.734375 L 6.25 -5.65625 C 5.914062 -5.832031 5.585938 -5.960938 5.265625 -6.046875 C 4.941406 -6.140625 4.613281 -6.1875 4.28125 -6.1875 C 3.53125 -6.1875 2.945312 -5.953125 2.53125 -5.484375 C 2.125 -5.015625 1.921875 -4.351562 1.921875 -3.5 C 1.921875 -2.644531 2.125 -1.976562 2.53125 -1.5 C 2.945312 -1.03125 3.53125 -0.796875 4.28125 -0.796875 C 4.613281 -0.796875 4.941406 -0.835938 5.265625 -0.921875 C 5.585938 -1.015625 5.914062 -1.148438 6.25 -1.328125 L 6.25 -0.265625 C 5.925781 -0.117188 5.59375 -0.0078125 5.25 0.0625 C 4.90625 0.144531 4.539062 0.1875 4.15625 0.1875 C 3.09375 0.1875 2.25 -0.144531 1.625 -0.8125 C 1.007812 -1.476562 0.703125 -2.375 0.703125 -3.5 C 0.703125 -4.632812 1.015625 -5.53125 1.640625 -6.1875 C 2.273438 -6.84375 3.132812 -7.171875 4.21875 -7.171875 C 4.570312 -7.171875 4.914062 -7.132812 5.25 -7.0625 C 5.59375 -6.988281 5.925781 -6.878906 6.25 -6.734375 Z M 6.25 -6.734375 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-19">
> +<path style="stroke:none;" d="M 2.3125 -1.046875 L 2.3125 2.65625 L 1.15625 2.65625 L 1.15625 -7 L 2.3125 -7 L 2.3125 -5.9375 C 2.5625 -6.351562 2.867188 -6.660156 3.234375 -6.859375 C 3.597656 -7.066406 4.039062 -7.171875 4.5625 -7.171875 C 5.40625 -7.171875 6.09375 -6.832031 6.625 -6.15625 C 7.15625 -5.476562 7.421875 -4.59375 7.421875 -3.5 C 7.421875 -2.394531 7.15625 -1.503906 6.625 -0.828125 C 6.09375 -0.148438 5.40625 0.1875 4.5625 0.1875 C 4.039062 0.1875 3.597656 0.0859375 3.234375 -0.109375 C 2.867188 -0.316406 2.5625 -0.628906 2.3125 -1.046875 Z M 6.234375 -3.5 C 6.234375 -4.34375 6.054688 -5.003906 5.703125 -5.484375 C 5.359375 -5.960938 4.882812 -6.203125 4.28125 -6.203125 C 3.664062 -6.203125 3.179688 -5.960938 2.828125 -5.484375 C 2.484375 -5.003906 2.3125 -4.34375 2.3125 -3.5 C 2.3125 -2.644531 2.484375 -1.976562 2.828125 -1.5 C 3.179688 -1.019531 3.664062 -0.78125 4.28125 -0.78125 C 4.882812 -0.78125 5.359375 -1.019531 5.703125 -1.5 C 6.054688 -1.976562 6.234375 -2.644531 6.234375 -3.5 Z M 6.234375 -3.5 "/>
> +</symbol>
> +<symbol overflow="visible" id="glyph0-20">
> +<path style="stroke:none;" d="M 4.0625 -8.5 C 3.414062 -8.5 2.925781 -8.175781 2.59375 -7.53125 C 2.269531 -6.894531 2.109375 -5.9375 2.109375 -4.65625 C 2.109375 -3.375 2.269531 -2.410156 2.59375 -1.765625 C 2.925781 -1.128906 3.414062 -0.8125 4.0625 -0.8125 C 4.71875 -0.8125 5.207031 -1.128906 5.53125 -1.765625 C 5.863281 -2.410156 6.03125 -3.375 6.03125 -4.65625 C 6.03125 -5.9375 5.863281 -6.894531 5.53125 -7.53125 C 5.207031 -8.175781 4.71875 -8.5 4.0625 -8.5 Z M 4.0625 -9.5 C 5.113281 -9.5 5.914062 -9.082031 6.46875 -8.25 C 7.019531 -7.425781 7.296875 -6.226562 7.296875 -4.65625 C 7.296875 -3.082031 7.019531 -1.878906 6.46875 -1.046875 C 5.914062 -0.222656 5.113281 0.1875 4.0625 0.1875 C 3.019531 0.1875 2.222656 -0.222656 1.671875 -1.046875 C 1.117188 -1.878906 0.84375 -3.082031 0.84375 -4.65625 C 0.84375 -6.226562 1.117188 -7.425781 1.671875 -8.25 C 2.222656 -9.082031 3.019531 -9.5 4.0625 -9.5 Z M 4.0625 -9.5 "/>
> +</symbol>
> +</g>
> +</defs>
> +<g id="surface11910">
> +<rect x="0" y="0" width="296" height="142" style="fill:rgb(100%,100%,100%);fill-opacity:1;stroke:none;"/>
> +<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 16.35 11.5 L 25.75 11.5 L 25.75 18.4 L 16.35 18.4 Z M 16.35 11.5 " transform="matrix(20,0,0,20,-325,-228)"/>
> +<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.919141 14.75625 C 25.919141 14.99375 25.562695 14.99375 25.562695 14.75625 C 25.562695 14.518555 25.919141 14.518555 25.919141 14.75625 " transform="matrix(20,0,0,20,-325,-228)"/>
> +<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.140039 13.450586 C 18.140039 13.69707 17.770117 13.69707 17.770117 13.450586 C 17.770117 13.203906 18.140039 13.203906 18.140039 13.450586 " transform="matrix(20,0,0,20,-325,-228)"/>
> +<path style="fill-rule:evenodd;fill:rgb(100%,100%,100%);fill-opacity:1;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.116211 16.39043 C 18.116211 16.637109 17.746484 16.637109 17.746484 16.39043 C 17.746484 16.143945 18.116211 16.143945 18.116211 16.39043 " transform="matrix(20,0,0,20,-325,-228)"/>
> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-dasharray:1,1;stroke-miterlimit:10;" d="M 18.160937 16.341406 L 25.344141 14.80293 " transform="matrix(20,0,0,20,-325,-228)"/>
> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.016797 15.128711 L 25.45332 14.779687 L 24.912109 14.639844 " transform="matrix(20,0,0,20,-325,-228)"/>
> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 18.140039 13.450586 L 25.342383 14.717383 " transform="matrix(20,0,0,20,-325,-228)"/>
> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 24.916797 14.896484 L 25.452539 14.736914 L 25.003516 14.403906 " transform="matrix(20,0,0,20,-325,-228)"/>
> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 25.918945 14.75625 L 30.732813 14.715039 " transform="matrix(20,0,0,20,-325,-228)"/>
> +<path style="fill:none;stroke-width:0.1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:10;" d="M 30.34668 14.968359 L 30.844531 14.714062 L 30.342383 14.468359 " transform="matrix(20,0,0,20,-325,-228)"/>
> +<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
> +  <use xlink:href="#glyph0-1" x="34.101562" y="34.267687"/>
> +  <use xlink:href="#glyph0-2" x="37.712674" y="34.267687"/>
> +  <use xlink:href="#glyph0-3" x="50.212674" y="34.267687"/>
> +  <use xlink:href="#glyph0-4" x="57.990451" y="34.267687"/>
> +  <use xlink:href="#glyph0-5" x="66.046007" y="34.267687"/>
> +  <use xlink:href="#glyph0-6" x="73.823785" y="34.267687"/>
> +  <use xlink:href="#glyph0-7" x="77.990451" y="34.267687"/>
> +  <use xlink:href="#glyph0-3" x="86.046007" y="34.267687"/>
> +  <use xlink:href="#glyph0-8" x="93.823785" y="34.267687"/>
> +  <use xlink:href="#glyph0-3" x="98.823785" y="34.267687"/>
> +  <use xlink:href="#glyph0-6" x="106.601562" y="34.267687"/>
> +  <use xlink:href="#glyph0-9" x="110.768229" y="34.267687"/>
> +  <use xlink:href="#glyph0-10" x="115.768229" y="34.267687"/>
> +  <use xlink:href="#glyph0-11" x="123.823785" y="34.267687"/>
> +</g>
> +<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
> +  <use xlink:href="#glyph0-5" x="33.625" y="115.412218"/>
> +  <use xlink:href="#glyph0-2" x="41.402778" y="115.412218"/>
> +  <use xlink:href="#glyph0-12" x="53.902778" y="115.412218"/>
> +  <use xlink:href="#glyph0-5" x="61.958333" y="115.412218"/>
> +  <use xlink:href="#glyph0-7" x="69.736111" y="115.412218"/>
> +  <use xlink:href="#glyph0-7" x="77.791667" y="115.412218"/>
> +  <use xlink:href="#glyph0-5" x="85.847222" y="115.412218"/>
> +  <use xlink:href="#glyph0-7" x="93.625" y="115.412218"/>
> +  <use xlink:href="#glyph0-6" x="101.680556" y="115.412218"/>
> +  <use xlink:href="#glyph0-7" x="105.847222" y="115.412218"/>
> +  <use xlink:href="#glyph0-3" x="113.902778" y="115.412218"/>
> +  <use xlink:href="#glyph0-8" x="121.680556" y="115.412218"/>
> +  <use xlink:href="#glyph0-3" x="126.680556" y="115.412218"/>
> +  <use xlink:href="#glyph0-6" x="134.458333" y="115.412218"/>
> +  <use xlink:href="#glyph0-9" x="138.625" y="115.412218"/>
> +  <use xlink:href="#glyph0-13" x="143.625" y="115.412218"/>
> +  <use xlink:href="#glyph0-11" x="151.680556" y="115.412218"/>
> +</g>
> +<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
> +  <use xlink:href="#glyph0-14" x="197" y="54.505968"/>
> +  <use xlink:href="#glyph0-15" x="203.666667" y="54.505968"/>
> +  <use xlink:href="#glyph0-16" x="211.444444" y="54.505968"/>
> +  <use xlink:href="#glyph0-17" x="219.5" y="54.505968"/>
> +  <use xlink:href="#glyph0-18" x="224.5" y="54.505968"/>
> +  <use xlink:href="#glyph0-5" x="231.444444" y="54.505968"/>
> +  <use xlink:href="#glyph0-6" x="239.222222" y="54.505968"/>
> +  <use xlink:href="#glyph0-19" x="243.388889" y="54.505968"/>
> +  <use xlink:href="#glyph0-3" x="251.444444" y="54.505968"/>
> +  <use xlink:href="#glyph0-7" x="259.222222" y="54.505968"/>
> +  <use xlink:href="#glyph0-6" x="267.277778" y="54.505968"/>
> +  <use xlink:href="#glyph0-9" x="271.444444" y="54.505968"/>
> +  <use xlink:href="#glyph0-20" x="276.444444" y="54.505968"/>
> +  <use xlink:href="#glyph0-11" x="284.5" y="54.505968"/>
> +</g>
> +</g>
> +</svg>
> diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> index dcfcbd52490d..4d145bd3bd09 100644
> --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
> +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
> @@ -838,3 +838,5 @@ stream while it may be possible to enable and disable the embedded data stream.
>
>   The embedded data format does not need to be configured on the sensor's pads as
>   the format is dictated by the pixel data format in this case.
> +
> +.. include:: subdev-config-model.rst
> diff --git a/Documentation/userspace-api/media/v4l/subdev-config-model.rst b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> new file mode 100644
> index 000000000000..b0bd09772ceb
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/subdev-config-model.rst
> @@ -0,0 +1,209 @@
> +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
> +
> +.. _media_subdev_config_model:
> +
> +Sub-device configuration models
> +===============================
> +
> +A sub-device configuration model specifies in detail what the user space can
> +expect from a sub-device in terms of V4L2 sub-device interface support,
> +semantics included.
> +
> +A sub-device may implement more than one configuration model at the same
> +time. The implemented configuration models can be obtained from the sub-device's
> +``V4L2_CID_CONFIG_MODEL`` control.
> +
> +.. _media_subdev_config_model_common_raw_sensor:
> +
> +Common raw camera sensor model
> +------------------------------
> +
> +The common raw camera sensor model defines a set of enumeration and
> +configuration interfaces (formats, selections etc.) that cover the vast majority
> +of functionality of raw camera sensors. Not all of the interfaces are
> +necessarily offered by all drivers.
> +
> +A sub-device complies with the common raw sensor model if the
> +``V4L2_CONFIG_MODEL_COMMON_RAW`` bit is set in the ``V4L2_CID_CONFIG_MODEL``
> +control of the sub-device.
> +
> +The common raw camera sensor model is aligned with
> +:ref:`media_using_camera_sensor_drivers`. Please refer to that regarding aspects
> +not specified here.
> +
> +Each camera sensor implementing the common raw sensor model exposes a single
> +V4L2 sub-device. The sub-device contains a single source pad (0) and two or more
> +internal pads: an image data internal pad (1) and optionally an embedded data
> +pad (2). Additionally, further internal pads may be supported for other
> +features, in which case they are documented separately for the given device.
> +
> +This is shown in :ref:`media_subdev_config_model_common_raw_sensor_subdev`.
> +
> +.. _media_subdev_config_model_common_raw_sensor_subdev:
> +
> +.. kernel-figure:: common-raw-sensor.svg
> +    :alt:    common-raw-sensor.svg
> +    :align:  center
> +
> +    **Common raw sensor sub-device**
> +
> +Routes
> +^^^^^^
> +
> +A sub-device conforming to common raw camera sensor model implements the
> +following routes.
> +
> +.. flat-table:: Routes
> +    :header-rows: 1
> +
> +    * - Sink pad/stream
> +      - Source pad/stream
> +      - Static (X/M(aybe)/-)
> +      - Mandatory (X/-)
> +      - Synopsis
> +    * - 1/0
> +      - 0/0
> +      - X
> +      - X
> +      - Image data
> +    * - 2/0
> +      - 0/1
> +      - M
> +      - \-
> +      - Embedded data
> +
> +Support for the embedded data stream is optional. Drivers supporting the
> +embedded data stream may allow disabling and enabling the route when the
> +streaming is disabled.
> +
> +Sensor pixel array size, cropping and binning
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +The sensor's pixel array is divided into one or more areas. The areas around the
> +edge of the pixel array, usually one or more sides, may contain optical black
> +pixels, dummy pixels and other non-image pixels. The entire pixel array size is
> +conveyed by the format on (pad, stream) pair 1/0.
> +
> +A rectangle within the pixel array contains the visible pixels. Capturing the
> +non-visible pixels outside the visible pixel area may be supported by the
> +sensor. The visible pixel area corresponds to the ``V4L2_SEL_TGT_CROP_DEFAULT``
> +selection target on (pad, stream) pair 1/0.
> +
> +Sensors can perform multiple operations that affect the output image size. First
> +of these is the analogue crop. Analogue crop limits the area of the pixel array
> +which the sensor will read, affecting sensor timing as well. The granularity of
> +the analogue crop configuration varies greatly across sensors: some sensors
> +support only a few different analogue crop configurations whereas others may
> +support anything divisible by a given number of pixels. The analogue crop
> +configuration corresponds the ``V4L2_SEL_TGT_CROP`` selection target on (pad,
> +stream) pair 1/0. The default analogue crop rectangle corresponds to the visible
> +pixel area.
> +
> +In the next step, binning is performed on the image data read from camera
> +sensor's pixel array, as determined by the analogue crop configuration. Enabling
> +binning will effectively result in an image smaller than the original by given
> +binning factors horizontally and vertically. Typical values are 1/2 and 1/3 but
> +others may well be supported by the hardware as well.
> +
> +Sub-sampling follows binning. Sub-sampling, like binning, reduces the size of
> +the image by including only a subset of samples read from the sensor's pixel
> +matrix, typically every n'th pixel horizontally and vertically, taking the
> +sensor's colour pattern into account. Sub-sampling is generally configurable
> +separately horizontally and vertically.
> +
> +Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE``
> +rectangle, relative to the analogue crop rectangle, on (pad, stream) pair
> +1/0. The driver implementation determines how to configure binning and
> +sub-sampling to achieve the desired size.
> +
> +The digital crop operation takes place after binning and sub-sampling. It is
> +configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair
> +0/0. The resulting image size is further output by the sensor.
> +
> +The sensor's output mbus code is configured by setting the format on the (pad,
> +stream) pair 0/0. When setting the format, always use the same width and height
> +as for the digital crop setting.
> +
> +Drivers may only support some of even none of these configurations, in which
> +case they do not expose the corresponding selection rectangles. If any selection
> +targets are omitted, the further selection rectangle or format is instead
> +related to the previous implemented selection rectangle. For instance, if the
> +sensor supports binning but not analogue crop, then the binning configuration
> +(``V4L2_SEL_TGT_COMPOSE`` selection target) is done in relation to the visible
> +pixel area (``V4L2_SEL_TGT_CROP_DEFAULT`` selection target).
> +
> +Also refer to :ref:`Selection targets <v4l2-selection-targets-table>`.
> +
> +.. flat-table:: Selection targets on pads
> +    :header-rows: 1
> +
> +    * - Pad/Stream
> +      - Selection target/format
> +      - Mandatory (X/-)
> +      - Modifiable (X/-)
> +      - Synopsis
> +    * - 1/0
> +      - Format
> +      - X
> +      - \-
> +      - Image data format. The width and the height fields indicates the full
> +        size of the pixel array, including non-visible pixels. The media bus
> +        code of this format reflects the native pixel depth of the sensor.
> +    * - 1/0
> +      - ``V4L2_SEL_TGT_CROP_DEFAULT``
> +      - X
> +      - \
> +      - The visible pixel area. This rectangle is relative to the format on the
> +        same (pad, stream).
> +    * - 1/0
> +      - ``V4L2_SEL_TGT_CROP``
> +      - \-
> +      - X
> +      - Analogue crop. Analogue crop typically has a coarse granularity. This
> +        rectangle is relative to the format on the same (pad, stream).
> +    * - 1/0
> +      - ``V4L2_SEL_TGT_COMPOSE``
> +      - \-
> +      - X
> +      - Binning and sub-sampling. This rectangle is relative to the
> +        ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). The
> +        combination of binning and sub-sampling is configured using this
> +        selection target.
> +    * - 2/0
> +      - Format
> +      - X
> +      - \-
> +      - Embedded data format.
> +    * - 0/0
> +      - ``V4L2_SEL_TGT_CROP``
> +      - \-
> +      - X
> +      - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE``
> +        rectangle on (pad, stream) pair 1/0.
> +    * - 0/0
> +      - Format
> +      - X
> +      - X
> +      - Image data source format. Always assign the width and height fields of
> +        the format to the same values than for the ``V4L2_SEL_TGT_CROP``
> +        rectangle on (pad, stream) pair 0/0. The media bus code reflects the
> +        pixel data output of the sensor.
> +    * - 0/1
> +      - Format
> +      - X
> +      - \-
> +      - Embedded data source format.
> +
> +Embedded data
> +^^^^^^^^^^^^^
> +
> +The embedded data stream is produced by the sensor when the corresponding route
> +is enabled. The embedded data route may also be immutable or not exist at all,
> +in case the sensor (or the driver) does not support it.
> +
> +Generally the sensor embedded data width is determined by the width of the image
> +data whereas the number of lines are constant for the embedded data. The user
> +space may obtain the size of the embedded data once the image data size on the
> +source pad has been configured.
> +
> +Also see :ref:`media_using_camera_sensor_drivers_embedded_data`.
> --
> 2.39.5
>

Re: [RFC v5 07/15] media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour patterns

[Mirela Rabulea]

Hi Sakari,

sorry for my late response.

So, this control and the luma-only mbus code is expected to be used for 
all raw sensors? Or can raw sensors still use BGGR mbus codes without 
the color pattern control?

I noted this is now read-only, is the intention that the user space will 
only query the current CFA pattern? In case the user space will want to 
change the CFA, is it expected to achieve it via V4L2_CID_HFLIP or/and 
V4L2_CID_VFLIP and/or crop?

What exactly is the V4L2_CID_COLOUR_PATTERN expected to report? Is it 
the sensor's native CFA pattern, or the current CFA pattern with the 
current flip & crop setting?

One more suggestion below.

On 03.02.2025 10:58, Sakari Ailus wrote:
> Add V4L2_CID_COLOUR_PATTERN to tell the camera sensor's native colour
> pattern.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>   .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
>   drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  1 +
>   include/uapi/linux/v4l2-controls.h                     |  6 ++++++
>   3 files changed, 17 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index 71f23f131f97..fca729512b6f 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -92,3 +92,13 @@ Image Source Control IDs
>       representing a gain of exactly 1.0. For example, if this default value
>       is reported as being (say) 128, then a value of 192 would represent
>       a gain of exactly 1.5.
> +
> +``V4L2_CID_COLOUR_PATTERN (integer)``
> +    This control determines the colour components and pixel order in the
> +    sensor's CFA (Colour Filter Array) when used in conjunction with
> +    :ref:`luma-only mbus codes MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth)
> +    <v4l2-mbus-pixelcode-yuv8>` pixelformats.
> +
> +    This control may only be used on a V4L2 sub-device.
> +
> +    This is a read-only control.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 24c9c25e20d1..5b6a4a94f18f 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1155,6 +1155,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>          case V4L2_CID_TEST_PATTERN_BLUE:        return "Blue Pixel Value";
>          case V4L2_CID_TEST_PATTERN_GREENB:      return "Green (Blue) Pixel Value";
>          case V4L2_CID_NOTIFY_GAINS:             return "Notify Gains";
> +       case V4L2_CID_COLOUR_PATTERN:           return "Colour Pattern";
>
>          /* Image processing controls */
>          /* Keep the order of the 'case's the same as in v4l2-controls.h! */
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 731add75d9ee..8e761c38b995 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1214,6 +1214,12 @@ enum v4l2_jpeg_chroma_subsampling {
>   #define V4L2_CID_UNIT_CELL_SIZE                        (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
>   #define V4L2_CID_NOTIFY_GAINS                  (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
>
> +#define V4L2_CID_COLOUR_PATTERN                        (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10)
> +#define V4L2_COLOUR_PATTERN_GRBG               0
> +#define V4L2_COLOUR_PATTERN_RGGB               1
> +#define V4L2_COLOUR_PATTERN_BGGR               2
> +#define V4L2_COLOUR_PATTERN_GBRG               3
> +

I see no pattern for RGB-Ir sensors.

For Omnivision ox05b1s RGB-Ir sensor for example, it has a 4x4 pattern, 
we would need something like:

B G R G...

G Ir G Ir...

R G B G...

G Ir G Ir...

V4L2_COLOUR_PATTERN_BGRGGIGIRGBGGIGI

And also 3 other combinations from mirror and flip:

V4L2_COLOUR_PATTERN_GRGBIGIGGBGRIGIG

V4L2_COLOUR_PATTERN_GIGIRGBGGIGIBGRG

V4L2_COLOUR_PATTERN_IGIGGBGRIGIGGRGB


At which point we could have these patterns? Is it possible from the 
beginning, or should we add them at the time when there is at least a 
driver that needs them?

Also, another question, on which tree does this patch set apply?

Thanks,

Mirela

>
>   /* Image processing controls */
>
> --
> 2.39.5
>

Re: [RFC v5 10/15] media: v4l: uapi: Add a control for colour pattern flipping effect

[Mirela Rabulea]

Hi Sakari,

On 03.02.2025 10:58, Sakari Ailus wrote:
> Add a bitmask control (V4L2_CID_COLOUR_PATTERN_FLIP) to tell whether
> flipping results in a change in the sensor's colour pattern, separately
> horizontally and vertically. The information is essential for raw formats
> when using generic raw mbus codes.

Is this control really necessary? Are there cases when V4L2_CID_HFLIP 
or/and V4L2_CID_VFLIP is enabled and the CFA pattern does not change?

Could a raw sensor driver be capable of reporting the 
V4L2_CID_COLOUR_PATTERN based on current values of the HFLIP/VFLIP? I'm 
not sure I understood all the aspects of patch #7 (V4L2_CID_COLOUR_PATTERN).

Regards,

Mirela

>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>   .../media/v4l/ext-ctrls-image-source.rst          |  8 ++++++++
>   drivers/media/v4l2-core/v4l2-ctrls-defs.c         |  2 ++
>   include/uapi/linux/v4l2-controls.h                | 15 +++++++++++++++
>   3 files changed, 25 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index fca729512b6f..ecfa38c118e3 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -102,3 +102,11 @@ Image Source Control IDs
>       This control may only be used on a V4L2 sub-device.
>
>       This is a read-only control.
> +
> +``V4L2_CID_COLOUR_PATTERN_FLIP (bitmask)``
> +    Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
> +    V4L2_CID_VFLIP) have an effect on the pixel order of the output colour
> +    pattern. Macros ``V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL`` and
> +    ``V4L2_COLOUR_PATTERN_FLIP_VERTICAL`` define bitmasks for both bits. If
> +    either horizontal or vertical bit is set, the readout pattern order is that
> +    of the reversed readout.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 5b6a4a94f18f..3f0704a982b8 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1156,6 +1156,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>          case V4L2_CID_TEST_PATTERN_GREENB:      return "Green (Blue) Pixel Value";
>          case V4L2_CID_NOTIFY_GAINS:             return "Notify Gains";
>          case V4L2_CID_COLOUR_PATTERN:           return "Colour Pattern";
> +       case V4L2_CID_COLOUR_PATTERN_FLIP:      return "Colour Pattern Flip";
>
>          /* Image processing controls */
>          /* Keep the order of the 'case's the same as in v4l2-controls.h! */
> @@ -1474,6 +1475,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>                  *max = 0xffff;
>                  break;
>          case V4L2_CID_FLASH_FAULT:
> +       case V4L2_CID_COLOUR_PATTERN_FLIP:
>          case V4L2_CID_JPEG_ACTIVE_MARKER:
>          case V4L2_CID_3A_LOCK:
>          case V4L2_CID_AUTO_FOCUS_STATUS:
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 8e761c38b995..fd6465e9a743 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1215,11 +1215,26 @@ enum v4l2_jpeg_chroma_subsampling {
>   #define V4L2_CID_NOTIFY_GAINS                  (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
>
>   #define V4L2_CID_COLOUR_PATTERN                        (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10)
> +/*
> + * CFA pattern start shall be aligned with the number of patterns for colour
> + * components. Patterns shall be calculable based on flipping when it affects
> + * the pattern the following way:
> + *
> + * flipped_pattern = native_pattern ^
> + *     ((hflip ? V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL : 0) |
> + *      (vflip ? V4L2_COLOUR_PATTERN_FLIP_VERTICAL : 0))
> + *
> + * where hflip and vflip are the values of V4L2_CID_HFLIP and V4L2_CID_VFLIP
> + * controls, respectively.
> + */
>   #define V4L2_COLOUR_PATTERN_GRBG               0
>   #define V4L2_COLOUR_PATTERN_RGGB               1
>   #define V4L2_COLOUR_PATTERN_BGGR               2
>   #define V4L2_COLOUR_PATTERN_GBRG               3
>
> +#define V4L2_CID_COLOUR_PATTERN_FLIP           (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 11)
> +#define V4L2_COLOUR_PATTERN_FLIP_HORIZONTAL    (1U << 0)
> +#define V4L2_COLOUR_PATTERN_FLIP_VERTICAL      (1U << 1)
>
>   /* Image processing controls */
>
> --
> 2.39.5
>

Re: [EXT] [RFC v5 11/15] media: Documentation: Document luma-only mbus codes and CFA for cameras

[Mirela Rabulea]

Hi Sakari,

On 03.02.2025 10:58, Sakari Ailus wrote:
> Caution: This is an external email. Please take care when clicking links or opening attachments. When in doubt, report the message using the 'Report this email' button
>
>
> Document the use of luma-only mbus codes for camera sensors and how the
> V4L2_CID_COLOUR_PATTERN and V4L2_CID_COLOUR_PATTERN_FLIP controls are used
> to convey the colour filter array pattern on UAPI.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>   .../userspace-api/media/drivers/camera-sensor.rst   | 13 +++++++++++++
>   .../media/v4l/ext-ctrls-image-source.rst            |  4 ++++
>   2 files changed, 17 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index 5bc4c79d230c..91e5305458b9 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -41,6 +41,19 @@ format set on a source pad at the end of the device's internal pipeline.
>
>   Most sensor drivers are implemented this way.
>
> +V4L2_CID_COLOUR_PATTERN, luma-only mbus formats, flipping and cropping
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +For raw image data originating from camera sensors, :ref:`luma-only mbus codes
> +MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth) <v4l2-mbus-pixelcode-yuv8>`
> +are used as Colour Filter Array (CFA) agnostic raw formats. The
> +``V4L2_CID_COLOUR_PATTERN <image-source-control-colour-pattern>`` control in the
> +same sub-device defines the native colour pattern of the device. Flipping may
> +further affect the readout pattern as indicated by the
> +``V4L2_CID_COLOUR_PATTERN_FLIP <image-source-control-colour-pattern-flip>``
> +control. Further on, cropping also has an effect on the pattern if cropped
> +amount is not divisible by the size of the pattern, horizontally and vertically.
> +

Ok, so it is clear now, the V4L2_CID_COLOUR_PATTERN stands for the 
"native colour pattern", as opposed to the "readout pattern". It is good 
to make the distinction.

Maybe the V4L2_CID_COLOUR_PATTERN control description in patch #7 could 
also be made more clear with regards to native vs readout pattern?

"This control determines the colour components and pixel order in the 
sensor's CFA (Colour Filter Array)..." => "This control reports the 
colour components and native pixel order in the sensor's CFA (Colour 
Filter Array)..." ?

Thanks,

Mirela

>   Frame interval configuration
>   ----------------------------
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index ecfa38c118e3..4b98a740236a 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -93,6 +93,8 @@ Image Source Control IDs
>       is reported as being (say) 128, then a value of 192 would represent
>       a gain of exactly 1.5.
>
> +.. _image-source-control-colour-pattern:
> +
>   ``V4L2_CID_COLOUR_PATTERN (integer)``
>       This control determines the colour components and pixel order in the
>       sensor's CFA (Colour Filter Array) when used in conjunction with
> @@ -103,6 +105,8 @@ Image Source Control IDs
>
>       This is a read-only control.
>
> +.. _image-source-control-colour-pattern-flip:
> +
>   ``V4L2_CID_COLOUR_PATTERN_FLIP (bitmask)``
>       Whether the horizontal or vertical flipping controls (V4L2_CID_HFLIP and
>       V4L2_CID_VFLIP) have an effect on the pixel order of the output colour
> --
> 2.39.5
>

Re: [RFC v5 12/15] media: uapi: Documentation: Use luma formats with CFA pattern control

[Mirela Rabulea]

Hi Sakari,

Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>

Thanks,

Mirela

On 03.02.2025 10:58, Sakari Ailus wrote:
> Document that the luma-only, greyscale formats (Y10 etc.) are used also
> for non-greyscale images with the CFA pattern specified by the
> V4L2_CID_COLOUR_PATTERN control.
>
> The advantage of this approach is that there is little (or none at all)
> work needed to be able to capture such images with existing CSI-2 receiver
> drivers as many already support greyscale formats when compared to new
> CFA-agnostic pixelformats.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>   Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst | 5 +++++
>   1 file changed, 5 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
> index 74df19be91f6..ae4e431b5fd9 100644
> --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst
> @@ -9,6 +9,11 @@ Luma-Only Formats
>   This family of formats only store the luma component of a Y'CbCr image. They
>   are often referred to as greyscale formats.
>
> +These pixel formats are also used on :term:`MC-centric` devices when the CFA
> +pattern is specified by the :ref:`V4L2_CID_COLOUR_PATTERN
> +<image-source-control-colour-pattern>` control in the image source, typically a
> +camera sensor.
> +
>   .. note::
>
>      - In all the tables that follow, bit 7 is the most significant bit in a byte.
> --
> 2.39.5
>

Re: [RFC v5 07/15] media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour patterns

[Hans Verkuil]

Hi Sakari,

On 03/02/2025 09:58, Sakari Ailus wrote:
> Add V4L2_CID_COLOUR_PATTERN to tell the camera sensor's native colour
> pattern.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  1 +
>  include/uapi/linux/v4l2-controls.h                     |  6 ++++++
>  3 files changed, 17 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> index 71f23f131f97..fca729512b6f 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> @@ -92,3 +92,13 @@ Image Source Control IDs
>      representing a gain of exactly 1.0. For example, if this default value
>      is reported as being (say) 128, then a value of 192 would represent
>      a gain of exactly 1.5.
> +
> +``V4L2_CID_COLOUR_PATTERN (integer)``

Please use 'COLOR' instead of 'COLOUR' to be consistent with other color related
control names. So do a search-and-replace for this in code and documentation.

Regards,

	Hans

> +    This control determines the colour components and pixel order in the
> +    sensor's CFA (Colour Filter Array) when used in conjunction with
> +    :ref:`luma-only mbus codes MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth)
> +    <v4l2-mbus-pixelcode-yuv8>` pixelformats.
> +
> +    This control may only be used on a V4L2 sub-device.
> +
> +    This is a read-only control.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 24c9c25e20d1..5b6a4a94f18f 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1155,6 +1155,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_TEST_PATTERN_BLUE:	return "Blue Pixel Value";
>  	case V4L2_CID_TEST_PATTERN_GREENB:	return "Green (Blue) Pixel Value";
>  	case V4L2_CID_NOTIFY_GAINS:		return "Notify Gains";
> +	case V4L2_CID_COLOUR_PATTERN:		return "Colour Pattern";
>  
>  	/* Image processing controls */
>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 731add75d9ee..8e761c38b995 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1214,6 +1214,12 @@ enum v4l2_jpeg_chroma_subsampling {
>  #define V4L2_CID_UNIT_CELL_SIZE			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
>  #define V4L2_CID_NOTIFY_GAINS			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
>  
> +#define V4L2_CID_COLOUR_PATTERN			(V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10)
> +#define V4L2_COLOUR_PATTERN_GRBG		0
> +#define V4L2_COLOUR_PATTERN_RGGB		1
> +#define V4L2_COLOUR_PATTERN_BGGR		2
> +#define V4L2_COLOUR_PATTERN_GBRG		3
> +
>  
>  /* Image processing controls */
>  

Re: [RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control

[Hans Verkuil]

Hi Sakari,

On 03/02/2025 09:58, Sakari Ailus wrote:
> Add the V4L2_CID_CONFIG_MODEL control for the configuration model.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 4 ++++
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c                    | 5 +++++
>  include/uapi/linux/v4l2-controls.h                           | 3 +++
>  3 files changed, 12 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> index 27803dca8d3e..2ae17ed99729 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> @@ -55,3 +55,7 @@ Image Process Control IDs
>      control value divided by e.g. 0x100, meaning that to get no
>      digital gain the control value needs to be 0x100. The no-gain
>      configuration is also typically the default.
> +
> +``V4L2_CID_CONFIG_MODEL (bitmask)``
> +    Which configuration models the sub-device supports. Please see
> +    :ref:`media_subdev_config_model`. This is a read-only control.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 1ea52011247a..24c9c25e20d1 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_TEST_PATTERN:		return "Test Pattern";
>  	case V4L2_CID_DEINTERLACING_MODE:	return "Deinterlacing Mode";
>  	case V4L2_CID_DIGITAL_GAIN:		return "Digital Gain";
> +	case V4L2_CID_CONFIG_MODEL:		return "Sub-device configuration model";
>  
>  	/* DV controls */
>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> @@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_DV_RX_POWER_PRESENT:
>  		*type = V4L2_CTRL_TYPE_BITMASK;
>  		break;
> +	case V4L2_CID_CONFIG_MODEL:
> +		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
> +		*type = V4L2_CTRL_TYPE_BITMASK;
> +		break;
>  	case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
>  	case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 974fd254e573..731add75d9ee 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {
>  #define V4L2_CID_TEST_PATTERN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)
>  #define V4L2_CID_DEINTERLACING_MODE		(V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)
>  #define V4L2_CID_DIGITAL_GAIN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 5)
> +#define V4L2_CID_CONFIG_MODEL			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 6)
> +
> +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW	(1U << 0)

I think I mentioned this before, but what's the point of this?

You are adding a control describing a configuration model, but it has just a single
possible configuration model. I see no description anywhere about when a new model
would need to be added, or what userspace is supposed to do with this.

And as long as there is only one model anyway, I don't see the point of this control.

Is the intention that all sensor drivers will set this control? The RFC series isn't
clear about this.

The problem I see with this series is that it seems to mix seemingly unrelated changes:
adding COLOUR_PATTERN/BINNING controls doesn't seem to depend on configuration models.
Or if they do, I clearly didn't get that.

Regards,

	Hans

>  
>  /*  DV-class control IDs defined by V4L2 */
>  #define V4L2_CID_DV_CLASS_BASE			(V4L2_CTRL_CLASS_DV | 0x900)

Re: [RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control

[Sakari Ailus]

Hi Hans,

On Mon, Feb 10, 2025 at 10:09:33AM +0100, Hans Verkuil wrote:
> Hi Sakari,
> 
> On 03/02/2025 09:58, Sakari Ailus wrote:
> > Add the V4L2_CID_CONFIG_MODEL control for the configuration model.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> >  .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 4 ++++
> >  drivers/media/v4l2-core/v4l2-ctrls-defs.c                    | 5 +++++
> >  include/uapi/linux/v4l2-controls.h                           | 3 +++
> >  3 files changed, 12 insertions(+)
> > 
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> > index 27803dca8d3e..2ae17ed99729 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> > @@ -55,3 +55,7 @@ Image Process Control IDs
> >      control value divided by e.g. 0x100, meaning that to get no
> >      digital gain the control value needs to be 0x100. The no-gain
> >      configuration is also typically the default.
> > +
> > +``V4L2_CID_CONFIG_MODEL (bitmask)``
> > +    Which configuration models the sub-device supports. Please see
> > +    :ref:`media_subdev_config_model`. This is a read-only control.
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > index 1ea52011247a..24c9c25e20d1 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > @@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >  	case V4L2_CID_TEST_PATTERN:		return "Test Pattern";
> >  	case V4L2_CID_DEINTERLACING_MODE:	return "Deinterlacing Mode";
> >  	case V4L2_CID_DIGITAL_GAIN:		return "Digital Gain";
> > +	case V4L2_CID_CONFIG_MODEL:		return "Sub-device configuration model";
> >  
> >  	/* DV controls */
> >  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> > @@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> >  	case V4L2_CID_DV_RX_POWER_PRESENT:
> >  		*type = V4L2_CTRL_TYPE_BITMASK;
> >  		break;
> > +	case V4L2_CID_CONFIG_MODEL:
> > +		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
> > +		*type = V4L2_CTRL_TYPE_BITMASK;
> > +		break;
> >  	case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
> >  	case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
> >  		*type = V4L2_CTRL_TYPE_INTEGER;
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > index 974fd254e573..731add75d9ee 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {
> >  #define V4L2_CID_TEST_PATTERN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)
> >  #define V4L2_CID_DEINTERLACING_MODE		(V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)
> >  #define V4L2_CID_DIGITAL_GAIN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 5)
> > +#define V4L2_CID_CONFIG_MODEL			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 6)
> > +
> > +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW	(1U << 0)
> 
> I think I mentioned this before, but what's the point of this?

I recall Laurent was last to reply to the thread, with a good explanation
of the purpose. The message id is
<20241118024002.GJ31681@pendragon.ideasonboard.com> .

> 
> You are adding a control describing a configuration model, but it has
> just a single possible configuration model. I see no description anywhere
> about when a new model would need to be added, or what userspace is
> supposed to do with this.

At this point I'm not sure how many other configuration models might be
needed or when they would be needed.

> 
> And as long as there is only one model anyway, I don't see the point of
> this control.

I could create a control just for the common raw sensor model but 

> 
> Is the intention that all sensor drivers will set this control? The RFC
> series isn't clear about this.

I'd expect almost all new raw sensor drivers to expose this control with
the common raw bit set.

> 
> The problem I see with this series is that it seems to mix seemingly
> unrelated changes: adding COLOUR_PATTERN/BINNING controls doesn't seem to
> depend on configuration models. Or if they do, I clearly didn't get that.

These are all related to sensor API improvements. There is no direct
dependency, no, but I expect drivers implementing the common raw sensor
model would also support these controls. I can document this.

-- 
Kind regards,

Sakari Ailus

Re: [RFC v5 07/15] media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour patterns

[Sakari Ailus]

Hi Mirela,

Thank you for the review.

On Sun, Feb 09, 2025 at 07:14:37PM +0200, Mirela Rabulea wrote:
> Hi Sakari,
> 
> sorry for my late response.
> 
> So, this control and the luma-only mbus code is expected to be used for all
> raw sensors? Or can raw sensors still use BGGR mbus codes without the color
> pattern control?

Once we have these merged, I'd expect new drivers to use common raw sensor
model (as long as it's suitable for the device) and the Yxx mbus codes and
these controls. I think it's fine to keep using the old pattern specific
mbus codes in addition to that.

> 
> I noted this is now read-only, is the intention that the user space will
> only query the current CFA pattern? In case the user space will want to
> change the CFA, is it expected to achieve it via V4L2_CID_HFLIP or/and
> V4L2_CID_VFLIP and/or crop?

Correct. The CFA pattern isn't directly changeable by the user space so
this is similar to existing pattern specific mbus codes.

> 
> What exactly is the V4L2_CID_COLOUR_PATTERN expected to report? Is it the
> sensor's native CFA pattern, or the current CFA pattern with the current
> flip & crop setting?

The native pattern which isn't affected by flipping or cropping. I'll add
it to the documentation here, too.

> 
> One more suggestion below.
> 
> On 03.02.2025 10:58, Sakari Ailus wrote:
> > Add V4L2_CID_COLOUR_PATTERN to tell the camera sensor's native colour
> > pattern.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >   .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
> >   drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  1 +
> >   include/uapi/linux/v4l2-controls.h                     |  6 ++++++
> >   3 files changed, 17 insertions(+)
> > 
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > index 71f23f131f97..fca729512b6f 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > @@ -92,3 +92,13 @@ Image Source Control IDs
> >       representing a gain of exactly 1.0. For example, if this default value
> >       is reported as being (say) 128, then a value of 192 would represent
> >       a gain of exactly 1.5.
> > +
> > +``V4L2_CID_COLOUR_PATTERN (integer)``
> > +    This control determines the colour components and pixel order in the
> > +    sensor's CFA (Colour Filter Array) when used in conjunction with
> > +    :ref:`luma-only mbus codes MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth)
> > +    <v4l2-mbus-pixelcode-yuv8>` pixelformats.
> > +
> > +    This control may only be used on a V4L2 sub-device.
> > +
> > +    This is a read-only control.
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > index 24c9c25e20d1..5b6a4a94f18f 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> > @@ -1155,6 +1155,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >          case V4L2_CID_TEST_PATTERN_BLUE:        return "Blue Pixel Value";
> >          case V4L2_CID_TEST_PATTERN_GREENB:      return "Green (Blue) Pixel Value";
> >          case V4L2_CID_NOTIFY_GAINS:             return "Notify Gains";
> > +       case V4L2_CID_COLOUR_PATTERN:           return "Colour Pattern";
> > 
> >          /* Image processing controls */
> >          /* Keep the order of the 'case's the same as in v4l2-controls.h! */
> > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > index 731add75d9ee..8e761c38b995 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -1214,6 +1214,12 @@ enum v4l2_jpeg_chroma_subsampling {
> >   #define V4L2_CID_UNIT_CELL_SIZE                        (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 8)
> >   #define V4L2_CID_NOTIFY_GAINS                  (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 9)
> > 
> > +#define V4L2_CID_COLOUR_PATTERN                        (V4L2_CID_IMAGE_SOURCE_CLASS_BASE + 10)
> > +#define V4L2_COLOUR_PATTERN_GRBG               0
> > +#define V4L2_COLOUR_PATTERN_RGGB               1
> > +#define V4L2_COLOUR_PATTERN_BGGR               2
> > +#define V4L2_COLOUR_PATTERN_GBRG               3
> > +
> 
> I see no pattern for RGB-Ir sensors.

Correct. Could you write a patch for that, please? :-)

I'll add more documentation to the above four patterns, that should be a
better starting point than this version.

> 
> For Omnivision ox05b1s RGB-Ir sensor for example, it has a 4x4 pattern, we
> would need something like:
> 
> B G R G...
> 
> G Ir G Ir...
> 
> R G B G...
> 
> G Ir G Ir...
> 
> V4L2_COLOUR_PATTERN_BGRGGIGIRGBGGIGI
> 
> And also 3 other combinations from mirror and flip:
> 
> V4L2_COLOUR_PATTERN_GRGBIGIGGBGRIGIG
> 
> V4L2_COLOUR_PATTERN_GIGIRGBGGIGIBGRG
> 
> V4L2_COLOUR_PATTERN_IGIGGBGRIGIGGRGB
> 
> 
> At which point we could have these patterns? Is it possible from the
> beginning, or should we add them at the time when there is at least a driver
> that needs them?
> 
> Also, another question, on which tree does this patch set apply?

They're all here
<URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=metadata-raw>.
It seems I forgot this from the cover page.

-- 
Kind regards,

Sakari Ailus

Re: [RFC v5 00/15] Sub-device configuration models

[Sakari Ailus]

On Mon, Feb 03, 2025 at 10:58:38AM +0200, Sakari Ailus wrote:
> Hello everyone,
> 
> I've been recently working (with others) on sub-device streams support as
> well as on internal pads. The two can be used to make sub-device
> configuration more versatile.

The patches are all available here (including streams + internal pad
support)
<URL:https://git.linuxtv.org/sailus/media_tree.git/log/?h=metadata-raw>.

-- 
Sakari Ailus

Re: [RFC v5 10/15] media: v4l: uapi: Add a control for colour pattern flipping effect

[Sakari Ailus]

Hi Mirela,

On Sun, Feb 09, 2025 at 07:23:10PM +0200, Mirela Rabulea wrote:
> Hi Sakari,
> 
> On 03.02.2025 10:58, Sakari Ailus wrote:
> > Add a bitmask control (V4L2_CID_COLOUR_PATTERN_FLIP) to tell whether
> > flipping results in a change in the sensor's colour pattern, separately
> > horizontally and vertically. The information is essential for raw formats
> > when using generic raw mbus codes.
> 
> Is this control really necessary? Are there cases when V4L2_CID_HFLIP or/and
> V4L2_CID_VFLIP is enabled and the CFA pattern does not change?

Some sensors have some tricks in firmware to avoid changing the pattern (I
suppose in reality they crop one line / column) but this may not be
controllable by the driver, so this possibility should be taken into
account.

> 
> Could a raw sensor driver be capable of reporting the
> V4L2_CID_COLOUR_PATTERN based on current values of the HFLIP/VFLIP? I'm not
> sure I understood all the aspects of patch #7 (V4L2_CID_COLOUR_PATTERN).

In principle it could. The two were originally made independent also
because there was an inconvenient dependency between mbus codes and
controls. If the pattern control does not specify the native pattern but an
active pattern (whatever has an effect on it), the userspace also must
check that configuration to determine the native pattern. My understanding
is the preference on that side is the native pattern.

-- 
Regards,

Sakari Ailus

Re: [RFC v5 13/15] media: uapi: Add V4L2_CID_BINNING control for binning configuration

[Hans Verkuil]

On 03/02/2025 09:58, Sakari Ailus wrote:
> Add V4L2_CID_BINNING control for configuring binning and enumerating a
> camera sensor's binning capabilities. The control combines horizontal and
> vertical binning into a single control as the two are generally related.
> 
> New drivers should use this control to configure binning.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  .../media/drivers/camera-sensor.rst           | 10 +++++++
>  .../media/v4l/ext-ctrls-camera.rst            | 29 +++++++++++++++++++
>  drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  2 ++
>  include/uapi/linux/v4l2-controls.h            |  1 +
>  4 files changed, 42 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> index 91e5305458b9..59431ab0b923 100644
> --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> @@ -120,6 +120,16 @@ values programmed by the register sequences. The default values of these
>  controls shall be 0 (disabled). Especially these controls shall not be inverted,
>  independently of the sensor's mounting rotation.
>  
> +Binning
> +-------
> +
> +Binning has traditionally been configured using :ref:`the compose selection
> +rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING
> +<v4l2-cid-camera-sensor-binning>` is also available for binning configuration and
> +users should use it when it's available. Drivers supporting the control shall
> +also support the compose rectangle, albeit the rectangle may be read-only when
> +the control is present.
> +
>  .. _media_using_camera_sensor_drivers_embedded_data:
>  
>  Embedded data
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> index cdc515c60468..18b484ff5d75 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> @@ -672,3 +672,32 @@ enum v4l2_scene_mode -
>  
>      As modes differ for each sensor, menu items are not standardized by this
>      control and are left to the programmer.
> +
> +.. _v4l2-cid-camera-sensor-binning:
> +
> +``V4L2_CID_BINNING_FACTORS (integer menu)``
> +
> +    Horizontal and vertical binning factors. Binning combines several
> +    horizontal, vertical or both pixel values into a single pixel. It is a way
> +    to scale an image. Binning typically produces fairly good quality output.
> +
> +    Determines both horizontal and vertical binning factors for a camera
> +    sensor. The values are encoded in the following way:
> +
> +.. flat-table::
> +    :header-rows:  1
> +    :stub-columns: 0
> +
> +    * - Bits
> +      - Synopsis
> +    * - 48--63
> +      - Horizontal binning numerator.
> +    * - 32--47
> +      - Horizontal binning denominator.
> +    * - 16--31
> +      - Vertical binning numerator.
> +    * - 0--15
> +      - Vertical binning denominator.

This suggests that the vertical and horizontal binning factors are not
independent settings, is that correct?

How does this control (and the subsampling controls) relate to the ROTATE control?
Does it happen before or after the rotate?

Regards,

	Hans

> +
> +For instance, a value of ``0x0001000300020003`` indicates binning by 3
> +(horizontally) * 3/2 (vertically).
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 3f0704a982b8..6be3d4a37127 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -1087,6 +1087,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_CAMERA_ORIENTATION:	return "Camera Orientation";
>  	case V4L2_CID_CAMERA_SENSOR_ROTATION:	return "Camera Sensor Rotation";
>  	case V4L2_CID_HDR_SENSOR_MODE:		return "HDR Sensor Mode";
> +	case V4L2_CID_BINNING_FACTORS:		return "Binning Factors";
>  
>  	/* FM Radio Modulator controls */
>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> @@ -1426,6 +1427,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_HDR_SENSOR_MODE:
>  		*type = V4L2_CTRL_TYPE_MENU;
>  		break;
> +	case V4L2_CID_BINNING_FACTORS:
>  	case V4L2_CID_LINK_FREQ:
>  		*type = V4L2_CTRL_TYPE_INTEGER_MENU;
>  		break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index fd6465e9a743..ac0e38e98a38 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -1088,6 +1088,7 @@ enum v4l2_auto_focus_range {
>  #define V4L2_CID_CAMERA_SENSOR_ROTATION		(V4L2_CID_CAMERA_CLASS_BASE+35)
>  
>  #define V4L2_CID_HDR_SENSOR_MODE		(V4L2_CID_CAMERA_CLASS_BASE+36)
> +#define V4L2_CID_BINNING_FACTORS		(V4L2_CID_CAMERA_CLASS_BASE+37)
>  
>  /* FM Modulator class control IDs */
>  

Re: [RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control

[Hans Verkuil]

On 10/02/2025 14:25, Sakari Ailus wrote:
> Hi Hans,
> 
> On Mon, Feb 10, 2025 at 10:09:33AM +0100, Hans Verkuil wrote:
>> Hi Sakari,
>>
>> On 03/02/2025 09:58, Sakari Ailus wrote:
>>> Add the V4L2_CID_CONFIG_MODEL control for the configuration model.
>>>
>>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>> ---
>>>  .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 4 ++++
>>>  drivers/media/v4l2-core/v4l2-ctrls-defs.c                    | 5 +++++
>>>  include/uapi/linux/v4l2-controls.h                           | 3 +++
>>>  3 files changed, 12 insertions(+)
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
>>> index 27803dca8d3e..2ae17ed99729 100644
>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
>>> @@ -55,3 +55,7 @@ Image Process Control IDs
>>>      control value divided by e.g. 0x100, meaning that to get no
>>>      digital gain the control value needs to be 0x100. The no-gain
>>>      configuration is also typically the default.
>>> +
>>> +``V4L2_CID_CONFIG_MODEL (bitmask)``
>>> +    Which configuration models the sub-device supports. Please see
>>> +    :ref:`media_subdev_config_model`. This is a read-only control.
>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>>> index 1ea52011247a..24c9c25e20d1 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>>> @@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>>>  	case V4L2_CID_TEST_PATTERN:		return "Test Pattern";
>>>  	case V4L2_CID_DEINTERLACING_MODE:	return "Deinterlacing Mode";
>>>  	case V4L2_CID_DIGITAL_GAIN:		return "Digital Gain";
>>> +	case V4L2_CID_CONFIG_MODEL:		return "Sub-device configuration model";
>>>  
>>>  	/* DV controls */
>>>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
>>> @@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>>>  	case V4L2_CID_DV_RX_POWER_PRESENT:
>>>  		*type = V4L2_CTRL_TYPE_BITMASK;
>>>  		break;
>>> +	case V4L2_CID_CONFIG_MODEL:
>>> +		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
>>> +		*type = V4L2_CTRL_TYPE_BITMASK;
>>> +		break;
>>>  	case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
>>>  	case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
>>>  		*type = V4L2_CTRL_TYPE_INTEGER;
>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>>> index 974fd254e573..731add75d9ee 100644
>>> --- a/include/uapi/linux/v4l2-controls.h
>>> +++ b/include/uapi/linux/v4l2-controls.h
>>> @@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {
>>>  #define V4L2_CID_TEST_PATTERN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)
>>>  #define V4L2_CID_DEINTERLACING_MODE		(V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)
>>>  #define V4L2_CID_DIGITAL_GAIN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 5)
>>> +#define V4L2_CID_CONFIG_MODEL			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 6)
>>> +
>>> +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW	(1U << 0)
>>
>> I think I mentioned this before, but what's the point of this?
> 
> I recall Laurent was last to reply to the thread, with a good explanation
> of the purpose. The message id is
> <20241118024002.GJ31681@pendragon.ideasonboard.com> .

Now document that in this patch series. I double checked patch 04/15 again,
and there is no mention of that explanation from Laurent. It should be incorporated
in that patch.

Open questions:

1) If this control is not available, but it is still a camera sensor, what is
   userspace supposed to to? I guess all it can do is assume that the driver
   follows the standard rules, since there is no way to figure out if there are
   differences. So again, how does this help? In any case, this should be
   documented as well.

2) Are there compliance tests for this? Because there is no point adding this
   without having tests for it as well. Otherwise I can 100% guarantee that
   drivers will set this even if it deviates from what it should do in some
   way. Relying on code review alone is going to be a very tough job.

> 
>>
>> You are adding a control describing a configuration model, but it has
>> just a single possible configuration model. I see no description anywhere
>> about when a new model would need to be added, or what userspace is
>> supposed to do with this.
> 
> At this point I'm not sure how many other configuration models might be
> needed or when they would be needed.
> 
>>
>> And as long as there is only one model anyway, I don't see the point of
>> this control.
> 
> I could create a control just for the common raw sensor model but 
> 
>>
>> Is the intention that all sensor drivers will set this control? The RFC
>> series isn't clear about this.
> 
> I'd expect almost all new raw sensor drivers to expose this control with
> the common raw bit set.
> 
>>
>> The problem I see with this series is that it seems to mix seemingly
>> unrelated changes: adding COLOUR_PATTERN/BINNING controls doesn't seem to
>> depend on configuration models. Or if they do, I clearly didn't get that.
> 
> These are all related to sensor API improvements. There is no direct
> dependency, no, but I expect drivers implementing the common raw sensor
> model would also support these controls. I can document this.

Just split it up in two separate series. I have no objections to the sensor API
improvements, so it is much easier to get that in.

But I think that the 'config_model' part is poorly documented and I am quite
skeptical about the whole thing. So that shouldn't block the other changes
from this RFC series.

Regards,

	Hans

Re: [RFC v5 13/15] media: uapi: Add V4L2_CID_BINNING control for binning configuration

[Sakari Ailus]

Hi Hans,

Thank you for the review.

On Mon, Feb 10, 2025 at 03:07:25PM +0100, Hans Verkuil wrote:
> On 03/02/2025 09:58, Sakari Ailus wrote:
> > Add V4L2_CID_BINNING control for configuring binning and enumerating a
> > camera sensor's binning capabilities. The control combines horizontal and
> > vertical binning into a single control as the two are generally related.
> > 
> > New drivers should use this control to configure binning.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> > ---
> >  .../media/drivers/camera-sensor.rst           | 10 +++++++
> >  .../media/v4l/ext-ctrls-camera.rst            | 29 +++++++++++++++++++
> >  drivers/media/v4l2-core/v4l2-ctrls-defs.c     |  2 ++
> >  include/uapi/linux/v4l2-controls.h            |  1 +
> >  4 files changed, 42 insertions(+)
> > 
> > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > index 91e5305458b9..59431ab0b923 100644
> > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > @@ -120,6 +120,16 @@ values programmed by the register sequences. The default values of these
> >  controls shall be 0 (disabled). Especially these controls shall not be inverted,
> >  independently of the sensor's mounting rotation.
> >  
> > +Binning
> > +-------
> > +
> > +Binning has traditionally been configured using :ref:`the compose selection
> > +rectangle <v4l2-selection-targets-table>`. The :ref:`V4L2_CID_BINNING
> > +<v4l2-cid-camera-sensor-binning>` is also available for binning configuration and
> > +users should use it when it's available. Drivers supporting the control shall
> > +also support the compose rectangle, albeit the rectangle may be read-only when
> > +the control is present.
> > +
> >  .. _media_using_camera_sensor_drivers_embedded_data:
> >  
> >  Embedded data
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> > index cdc515c60468..18b484ff5d75 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-camera.rst
> > @@ -672,3 +672,32 @@ enum v4l2_scene_mode -
> >  
> >      As modes differ for each sensor, menu items are not standardized by this
> >      control and are left to the programmer.
> > +
> > +.. _v4l2-cid-camera-sensor-binning:
> > +
> > +``V4L2_CID_BINNING_FACTORS (integer menu)``
> > +
> > +    Horizontal and vertical binning factors. Binning combines several
> > +    horizontal, vertical or both pixel values into a single pixel. It is a way
> > +    to scale an image. Binning typically produces fairly good quality output.
> > +
> > +    Determines both horizontal and vertical binning factors for a camera
> > +    sensor. The values are encoded in the following way:
> > +
> > +.. flat-table::
> > +    :header-rows:  1
> > +    :stub-columns: 0
> > +
> > +    * - Bits
> > +      - Synopsis
> > +    * - 48--63
> > +      - Horizontal binning numerator.
> > +    * - 32--47
> > +      - Horizontal binning denominator.
> > +    * - 16--31
> > +      - Vertical binning numerator.
> > +    * - 0--15
> > +      - Vertical binning denominator.
> 
> This suggests that the vertical and horizontal binning factors are not
> independent settings, is that correct?

There are dependencies between the two, yes.

> 
> How does this control (and the subsampling controls) relate to the ROTATE control?
> Does it happen before or after the rotate?

In general there's no rotation support in (raw) camera sensors, just
flipping. And flipping isn't affecting binning.

-- 
Kind regards,

Sakari Ailus

Re: [EXT] [RFC v5 11/15] media: Documentation: Document luma-only mbus codes and CFA for cameras

[Sakari Ailus]

Hi Mirela,

On Sun, Feb 09, 2025 at 07:48:27PM +0200, Mirela Rabulea wrote:
> Hi Sakari,
> 
> On 03.02.2025 10:58, Sakari Ailus wrote:
> > Caution: This is an external email. Please take care when clicking links or opening attachments. When in doubt, report the message using the 'Report this email' button
> > 
> > 
> > Document the use of luma-only mbus codes for camera sensors and how the
> > V4L2_CID_COLOUR_PATTERN and V4L2_CID_COLOUR_PATTERN_FLIP controls are used
> > to convey the colour filter array pattern on UAPI.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >   .../userspace-api/media/drivers/camera-sensor.rst   | 13 +++++++++++++
> >   .../media/v4l/ext-ctrls-image-source.rst            |  4 ++++
> >   2 files changed, 17 insertions(+)
> > 
> > diff --git a/Documentation/userspace-api/media/drivers/camera-sensor.rst b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > index 5bc4c79d230c..91e5305458b9 100644
> > --- a/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > +++ b/Documentation/userspace-api/media/drivers/camera-sensor.rst
> > @@ -41,6 +41,19 @@ format set on a source pad at the end of the device's internal pipeline.
> > 
> >   Most sensor drivers are implemented this way.
> > 
> > +V4L2_CID_COLOUR_PATTERN, luma-only mbus formats, flipping and cropping
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +For raw image data originating from camera sensors, :ref:`luma-only mbus codes
> > +MEDIA_BUS_FMT_Yx_1Xx (where 'x' is the bit depth) <v4l2-mbus-pixelcode-yuv8>`
> > +are used as Colour Filter Array (CFA) agnostic raw formats. The
> > +``V4L2_CID_COLOUR_PATTERN <image-source-control-colour-pattern>`` control in the
> > +same sub-device defines the native colour pattern of the device. Flipping may
> > +further affect the readout pattern as indicated by the
> > +``V4L2_CID_COLOUR_PATTERN_FLIP <image-source-control-colour-pattern-flip>``
> > +control. Further on, cropping also has an effect on the pattern if cropped
> > +amount is not divisible by the size of the pattern, horizontally and vertically.
> > +
> 
> Ok, so it is clear now, the V4L2_CID_COLOUR_PATTERN stands for the "native
> colour pattern", as opposed to the "readout pattern". It is good to make the
> distinction.
> 
> Maybe the V4L2_CID_COLOUR_PATTERN control description in patch #7 could also
> be made more clear with regards to native vs readout pattern?
> 
> "This control determines the colour components and pixel order in the
> sensor's CFA (Colour Filter Array)..." => "This control reports the colour
> components and native pixel order in the sensor's CFA (Colour Filter
> Array)..." ?

I agree, I'll change this in v6.

-- 
Kind regards,

Sakari Ailus

Re: [RFC v5 07/15] media: uapi: Add V4L2_CID_COLOUR_PATTERN for describing colour patterns

[Sakari Ailus]

Hi Hans,

On Mon, Feb 10, 2025 at 09:57:08AM +0100, Hans Verkuil wrote:
> Hi Sakari,
> 
> On 03/02/2025 09:58, Sakari Ailus wrote:
> > Add V4L2_CID_COLOUR_PATTERN to tell the camera sensor's native colour
> > pattern.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../userspace-api/media/v4l/ext-ctrls-image-source.rst | 10 ++++++++++
> >  drivers/media/v4l2-core/v4l2-ctrls-defs.c              |  1 +
> >  include/uapi/linux/v4l2-controls.h                     |  6 ++++++
> >  3 files changed, 17 insertions(+)
> > 
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > index 71f23f131f97..fca729512b6f 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-source.rst
> > @@ -92,3 +92,13 @@ Image Source Control IDs
> >      representing a gain of exactly 1.0. For example, if this default value
> >      is reported as being (say) 128, then a value of 192 would represent
> >      a gain of exactly 1.5.
> > +
> > +``V4L2_CID_COLOUR_PATTERN (integer)``
> 
> Please use 'COLOR' instead of 'COLOUR' to be consistent with other color related
> control names. So do a search-and-replace for this in code and documentation.

Ack, I'll address this in v6.

-- 
Regards,

Sakari Ailus

Re: [RFC v5 10/15] media: v4l: uapi: Add a control for colour pattern flipping effect

[Dave Stevenson]

Hi Mirela and Sakari

On Mon, 10 Feb 2025 at 13:53, Sakari Ailus <sakari.ailus@linux.intel.com> wrote:
>
> Hi Mirela,
>
> On Sun, Feb 09, 2025 at 07:23:10PM +0200, Mirela Rabulea wrote:
> > Hi Sakari,
> >
> > On 03.02.2025 10:58, Sakari Ailus wrote:
> > > Add a bitmask control (V4L2_CID_COLOUR_PATTERN_FLIP) to tell whether
> > > flipping results in a change in the sensor's colour pattern, separately
> > > horizontally and vertically. The information is essential for raw formats
> > > when using generic raw mbus codes.
> >
> > Is this control really necessary? Are there cases when V4L2_CID_HFLIP or/and
> > V4L2_CID_VFLIP is enabled and the CFA pattern does not change?
>
> Some sensors have some tricks in firmware to avoid changing the pattern (I
> suppose in reality they crop one line / column) but this may not be
> controllable by the driver, so this possibility should be taken into
> account.

To give examples of sensors I've encountered that do this.
Sony IMX327, IMX290, IMX462, IMX415 and probably most of the Starvis range.
OnSemi AR0144 and AR0234.

Sony's datasheets do explicitly show the pixel array as being an odd
number of pixels in each direction, and that it therefore has the same
colour pixel (generally red) in each of the 4 corners. The flip
registers will be shifting readout by 1 pixel.
Readout mode restrictions generally say that the width and height must
be at least a multiple of 4, and offsets must be even, so there is no
way to read out that extra border pixel.

  Dave

> >
> > Could a raw sensor driver be capable of reporting the
> > V4L2_CID_COLOUR_PATTERN based on current values of the HFLIP/VFLIP? I'm not
> > sure I understood all the aspects of patch #7 (V4L2_CID_COLOUR_PATTERN).
>
> In principle it could. The two were originally made independent also
> because there was an inconvenient dependency between mbus codes and
> controls. If the pattern control does not specify the native pattern but an
> active pattern (whatever has an effect on it), the userspace also must
> check that configuration to determine the native pattern. My understanding
> is the preference on that side is the native pattern.
>
> --
> Regards,
>
> Sakari Ailus

Re: [RFC v5 06/15] media: uapi: Add V4L2_CID_CONFIG_MODEL control

[Sakari Ailus]

Hi Hans,

On Mon, Feb 10, 2025 at 03:07:34PM +0100, Hans Verkuil wrote:
> On 10/02/2025 14:25, Sakari Ailus wrote:
> > Hi Hans,
> > 
> > On Mon, Feb 10, 2025 at 10:09:33AM +0100, Hans Verkuil wrote:
> >> Hi Sakari,
> >>
> >> On 03/02/2025 09:58, Sakari Ailus wrote:
> >>> Add the V4L2_CID_CONFIG_MODEL control for the configuration model.
> >>>
> >>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> >>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >>> ---
> >>>  .../userspace-api/media/v4l/ext-ctrls-image-process.rst      | 4 ++++
> >>>  drivers/media/v4l2-core/v4l2-ctrls-defs.c                    | 5 +++++
> >>>  include/uapi/linux/v4l2-controls.h                           | 3 +++
> >>>  3 files changed, 12 insertions(+)
> >>>
> >>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> >>> index 27803dca8d3e..2ae17ed99729 100644
> >>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> >>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-image-process.rst
> >>> @@ -55,3 +55,7 @@ Image Process Control IDs
> >>>      control value divided by e.g. 0x100, meaning that to get no
> >>>      digital gain the control value needs to be 0x100. The no-gain
> >>>      configuration is also typically the default.
> >>> +
> >>> +``V4L2_CID_CONFIG_MODEL (bitmask)``
> >>> +    Which configuration models the sub-device supports. Please see
> >>> +    :ref:`media_subdev_config_model`. This is a read-only control.
> >>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> >>> index 1ea52011247a..24c9c25e20d1 100644
> >>> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> >>> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> >>> @@ -1164,6 +1164,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >>>  	case V4L2_CID_TEST_PATTERN:		return "Test Pattern";
> >>>  	case V4L2_CID_DEINTERLACING_MODE:	return "Deinterlacing Mode";
> >>>  	case V4L2_CID_DIGITAL_GAIN:		return "Digital Gain";
> >>> +	case V4L2_CID_CONFIG_MODEL:		return "Sub-device configuration model";
> >>>  
> >>>  	/* DV controls */
> >>>  	/* Keep the order of the 'case's the same as in v4l2-controls.h! */
> >>> @@ -1481,6 +1482,10 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> >>>  	case V4L2_CID_DV_RX_POWER_PRESENT:
> >>>  		*type = V4L2_CTRL_TYPE_BITMASK;
> >>>  		break;
> >>> +	case V4L2_CID_CONFIG_MODEL:
> >>> +		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
> >>> +		*type = V4L2_CTRL_TYPE_BITMASK;
> >>> +		break;
> >>>  	case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE:
> >>>  	case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
> >>>  		*type = V4L2_CTRL_TYPE_INTEGER;
> >>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> >>> index 974fd254e573..731add75d9ee 100644
> >>> --- a/include/uapi/linux/v4l2-controls.h
> >>> +++ b/include/uapi/linux/v4l2-controls.h
> >>> @@ -1225,6 +1225,9 @@ enum v4l2_jpeg_chroma_subsampling {
> >>>  #define V4L2_CID_TEST_PATTERN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)
> >>>  #define V4L2_CID_DEINTERLACING_MODE		(V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)
> >>>  #define V4L2_CID_DIGITAL_GAIN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 5)
> >>> +#define V4L2_CID_CONFIG_MODEL			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 6)
> >>> +
> >>> +#define V4L2_CID_CONFIG_MODEL_COMMON_RAW	(1U << 0)
> >>
> >> I think I mentioned this before, but what's the point of this?
> > 
> > I recall Laurent was last to reply to the thread, with a good explanation
> > of the purpose. The message id is
> > <20241118024002.GJ31681@pendragon.ideasonboard.com> .
> 
> Now document that in this patch series. I double checked patch 04/15 again,
> and there is no mention of that explanation from Laurent. It should be incorporated
> in that patch.
> 
> Open questions:
> 
> 1) If this control is not available, but it is still a camera sensor, what is
>    userspace supposed to to? I guess all it can do is assume that the driver
>    follows the standard rules, since there is no way to figure out if there are
>    differences. So again, how does this help? In any case, this should be
>    documented as well.

The user space cannot expect the sub-device UAPI conforms to the common raw
sensor model without this control. The purpose of the model is indeed to
allow the user space to expect certain things to be supported in the API
and if they are supported, they work in a certain way. The alternative is
to do this driver by driver and there are quite a few sensor drivers out
there, with more to come.

> 
> 2) Are there compliance tests for this? Because there is no point adding this
>    without having tests for it as well. Otherwise I can 100% guarantee that
>    drivers will set this even if it deviates from what it should do in some
>    way. Relying on code review alone is going to be a very tough job.

Compliance tests need to be added, I agree.

> 
> > 
> >>
> >> You are adding a control describing a configuration model, but it has
> >> just a single possible configuration model. I see no description anywhere
> >> about when a new model would need to be added, or what userspace is
> >> supposed to do with this.
> > 
> > At this point I'm not sure how many other configuration models might be
> > needed or when they would be needed.
> > 
> >>
> >> And as long as there is only one model anyway, I don't see the point of
> >> this control.
> > 
> > I could create a control just for the common raw sensor model but 
> > 
> >>
> >> Is the intention that all sensor drivers will set this control? The RFC
> >> series isn't clear about this.
> > 
> > I'd expect almost all new raw sensor drivers to expose this control with
> > the common raw bit set.
> > 
> >>
> >> The problem I see with this series is that it seems to mix seemingly
> >> unrelated changes: adding COLOUR_PATTERN/BINNING controls doesn't seem to
> >> depend on configuration models. Or if they do, I clearly didn't get that.
> > 
> > These are all related to sensor API improvements. There is no direct
> > dependency, no, but I expect drivers implementing the common raw sensor
> > model would also support these controls. I can document this.
> 
> Just split it up in two separate series. I have no objections to the sensor API
> improvements, so it is much easier to get that in.
> 
> But I think that the 'config_model' part is poorly documented and I am quite
> skeptical about the whole thing. So that shouldn't block the other changes
> from this RFC series.

One option could be indeed to postpone this, and continue working on the
user space one driver at a time. My initial thought really was we could
require the common raw sensor model to be used with internal pads,
otherwise it'll be harder to ensure sensor drivers using them will be using
them consistently: compliance tests would require quite a few exceptions
for instance (doing the tests only for sensors that are non-CCS for
instance).

I wonder what Laurent thinks.

-- 
Regards,

Sakari Ailus

Re: [RFC v5 04/15] media: Documentation: Add subdev configuration models, raw sensor model

[Mirela Rabulea]

Hi Sakari,

On 09.02.2025 17:42, Mirela Rabulea wrote:
> Hi Sakari,
>
> Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
>
> Regards,
>
> Mirela
>
> On 03.02.2025 10:58, Sakari Ailus wrote:
>> Sub-device configuration models define what V4L2 API elements are
>> available on a compliant sub-device and how do they behave.
>>
>> The patch also adds a model for common raw sensors.
>>
>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
>> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   .../media/drivers/camera-sensor.rst           |   4 +
>>   .../media/v4l/common-raw-sensor.dia           | 441 ++++++++++++++++++
>>   .../media/v4l/common-raw-sensor.svg           | 134 ++++++
>>   .../userspace-api/media/v4l/dev-subdev.rst    |   2 +
>>   .../media/v4l/subdev-config-model.rst         | 209 +++++++++
>>   5 files changed, 790 insertions(+)
>>   create mode 100644 
>> Documentation/userspace-api/media/v4l/common-raw-sensor.dia
>>   create mode 100644 
>> Documentation/userspace-api/media/v4l/common-raw-sensor.svg
>>   create mode 100644 
>> Documentation/userspace-api/media/v4l/subdev-config-model.rst
>>
>> ...

>> +
>> +Common raw camera sensor model
>> +------------------------------
>> +
>> +The common raw camera sensor model defines a set of enumeration and
>> +configuration interfaces (formats, selections etc.) that cover the 
>> vast majority
>> +of functionality of raw camera sensors. Not all of the interfaces are
>> +necessarily offered by all drivers.
>> +
>> +A sub-device complies with the common raw sensor model if the
>> +``V4L2_CONFIG_MODEL_COMMON_RAW`` bit is set in the 
>> ``V4L2_CID_CONFIG_MODEL``
>> +control of the sub-device.
>> +
>> +The common raw camera sensor model is aligned with
>> +:ref:`media_using_camera_sensor_drivers`. Please refer to that 
>> regarding aspects
>> +not specified here.
>> +
>> +Each camera sensor implementing the common raw sensor model exposes 
>> a single
>> +V4L2 sub-device. The sub-device contains a single source pad (0) and 
>> two or more
>> +internal pads: an image data internal pad (1) and optionally an 
>> embedded data
>> +pad (2). Additionally, further internal pads may be supported for other
>> +features, in which case they are documented separately for the given 
>> device.

One more question here, to be sure I understand correctly. So, in order 
to be compliant with the common raw camera sensor model, in case a raw 
sensor will need more than 2 internal pads, the expectation is to keep 
image on pad 0, edata on pad 1, and add more needed pads? Or is it 
acceptable that a driver completely changes the pad numbering according 
to it's needs, as long as it is documented?

The case I am targeting is os08a20 hdr sensor, with: pad1-image long 
exposure, pad2-edata, pad3-image short exposure. So far, so good, I guess.

I don't know if there will be such a case for a sensor driver which does 
not support embedded data, but will need to support other internal pads, 
and in such a case, how should pad numbering look like?


Thanks,

Mirela

Re: [RFC v5 04/15] media: Documentation: Add subdev configuration models, raw sensor model

[Sakari Ailus]

Hi Mirela,

On Wed, Feb 26, 2025 at 11:17:43AM +0200, Mirela Rabulea wrote:
> Hi Sakari,
> 
> On 09.02.2025 17:42, Mirela Rabulea wrote:
> > Hi Sakari,
> > 
> > Reviewed-by: Mirela Rabulea <mirela.rabulea@nxp.com>
> > 
> > Regards,
> > 
> > Mirela
> > 
> > On 03.02.2025 10:58, Sakari Ailus wrote:
> > > Sub-device configuration models define what V4L2 API elements are
> > > available on a compliant sub-device and how do they behave.
> > > 
> > > The patch also adds a model for common raw sensors.
> > > 
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> > > ---
> > >   .../media/drivers/camera-sensor.rst           |   4 +
> > >   .../media/v4l/common-raw-sensor.dia           | 441 ++++++++++++++++++
> > >   .../media/v4l/common-raw-sensor.svg           | 134 ++++++
> > >   .../userspace-api/media/v4l/dev-subdev.rst    |   2 +
> > >   .../media/v4l/subdev-config-model.rst         | 209 +++++++++
> > >   5 files changed, 790 insertions(+)
> > >   create mode 100644
> > > Documentation/userspace-api/media/v4l/common-raw-sensor.dia
> > >   create mode 100644
> > > Documentation/userspace-api/media/v4l/common-raw-sensor.svg
> > >   create mode 100644
> > > Documentation/userspace-api/media/v4l/subdev-config-model.rst
> > > 
> > > ...
> 
> > > +
> > > +Common raw camera sensor model
> > > +------------------------------
> > > +
> > > +The common raw camera sensor model defines a set of enumeration and
> > > +configuration interfaces (formats, selections etc.) that cover the
> > > vast majority
> > > +of functionality of raw camera sensors. Not all of the interfaces are
> > > +necessarily offered by all drivers.
> > > +
> > > +A sub-device complies with the common raw sensor model if the
> > > +``V4L2_CONFIG_MODEL_COMMON_RAW`` bit is set in the
> > > ``V4L2_CID_CONFIG_MODEL``
> > > +control of the sub-device.
> > > +
> > > +The common raw camera sensor model is aligned with
> > > +:ref:`media_using_camera_sensor_drivers`. Please refer to that
> > > regarding aspects
> > > +not specified here.
> > > +
> > > +Each camera sensor implementing the common raw sensor model exposes
> > > a single
> > > +V4L2 sub-device. The sub-device contains a single source pad (0)
> > > and two or more
> > > +internal pads: an image data internal pad (1) and optionally an
> > > embedded data
> > > +pad (2). Additionally, further internal pads may be supported for other
> > > +features, in which case they are documented separately for the
> > > given device.
> 
> One more question here, to be sure I understand correctly. So, in order to
> be compliant with the common raw camera sensor model, in case a raw sensor
> will need more than 2 internal pads, the expectation is to keep image on pad
> 0, edata on pad 1, and add more needed pads? Or is it acceptable that a
> driver completely changes the pad numbering according to it's needs, as long
> as it is documented?
> 
> The case I am targeting is os08a20 hdr sensor, with: pad1-image long
> exposure, pad2-edata, pad3-image short exposure. So far, so good, I guess.
> 
> I don't know if there will be such a case for a sensor driver which does not
> support embedded data, but will need to support other internal pads, and in
> such a case, how should pad numbering look like?

Good question.

I think we could change the common raw sensor model in such a way that the
embedded data internal pad is the first one using a metadata mbus code,
instead of using a plain number.

Laurent, any thoughts?

-- 
Kind regards,

Sakari Ailus