Linux Media Controller development
 help / color / mirror / Atom feed
From: Sakari Ailus <sakari.ailus@linux.intel.com>
To: linux-media@vger.kernel.org
Cc: hans@jjverkuil.nl, laurent.pinchart@ideasonboard.com,
	Prabhakar <prabhakar.csengg@gmail.com>,
	"Kate Hsuan" <hpa@redhat.com>,
	"Dave Stevenson" <dave.stevenson@raspberrypi.com>,
	"Tommaso Merciai" <tomm.merciai@gmail.com>,
	"Benjamin Mugnier" <benjamin.mugnier@foss.st.com>,
	"Sylvain Petinot" <sylvain.petinot@foss.st.com>,
	"Christophe JAILLET" <christophe.jaillet@wanadoo.fr>,
	"Julien Massot" <julien.massot@collabora.com>,
	"Naushir Patuck" <naush@raspberrypi.com>,
	"Yan, Dongcheng" <dongcheng.yan@intel.com>,
	"Stefan Klug" <stefan.klug@ideasonboard.com>,
	"Mirela Rabulea" <mirela.rabulea@nxp.com>,
	"André Apitzsch" <git@apitzsch.eu>,
	"Heimir Thor Sverrisson" <heimir.sverrisson@gmail.com>,
	"Kieran Bingham" <kieran.bingham@ideasonboard.com>,
	"Mehdi Djait" <mehdi.djait@linux.intel.com>,
	"Ricardo Ribalda Delgado" <ribalda@kernel.org>,
	"Hans de Goede" <hansg@kernel.org>,
	"Jacopo Mondi" <jacopo.mondi@ideasonboard.com>,
	"Tomi Valkeinen" <tomi.valkeinen@ideasonboard.com>,
	"David Plowman" <david.plowman@raspberrypi.com>,
	"Yu, Ong Hock" <ong.hock.yu@intel.com>,
	"Ng, Khai Wen" <khai.wen.ng@intel.com>,
	"Jai Luthra" <jai.luthra@ideasonboard.com>,
	"Rishikesh Donadkar" <r-donadkar@ti.com>,
	"Mattijs Korpershoek" <mkorpershoek@kernel.org>,
	"Antti Laakso" <antti.laakso@linux.intel.com>,
	"Bajpai, Manik" <manik.bajpai@intel.com>,
	"Divyamani Tripathi" <divyamani.tripathi@intel.com>
Subject: [PATCH v13 39/73] media: Documentation: Add sub-device internal pads example
Date: Wed, 18 Nov 2026 15:27:45 +0200	[thread overview]
Message-ID: <20261118132819.213473-40-sakari.ailus@linux.intel.com> (raw)
In-Reply-To: <20261118132819.213473-1-sakari.ailus@linux.intel.com>

Add an example using sub-device internal pads, with the Sony IMX219 camera
sensor.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 .../userspace-api/media/v4l/dev-subdev.rst    | 129 ++++++++++++++++++
 1 file changed, 129 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index 3472b0591d79..f6c4dfcff06d 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -726,3 +726,132 @@ To configure this pipeline, the userspace must take the following steps:
    :ref:`V4L2_CID_METADATA_LAYOUT <image_source_control_metadata_layout>`
    controls are present on the source sub-device to obtain the pixel array CFA
    pattern and metadata layout.
+
+Internal pads setup example
+---------------------------
+
+A simple example of a multiplexed stream setup might be as follows:
+
+- An IMX219 camera sensor source sub-device, with one source pad (0), one
+  internal sink pad (1) as the source of the image data and an internal sink
+  pad (2) as the source of the embedded data. There are two routes, one from the
+  internal sink pad 1 to the source pad 0 (image data) and another from the
+  internal sink pad 2 to the source pad 0 (embedded data). Both streams are
+  always active, i.e. there is no need to separately enable the embedded data
+  stream. The sensor uses the CSI-2 interface.
+
+- A CSI-2 receiver in the SoC. The receiver has a single sink pad (pad 0),
+  connected to the sensor's source pad (0), and two source pads (pads 1 and 2),
+  to two DMA engines. The receiver demultiplexes the incoming streams to the
+  source pads.
+
+- Two DMA engines in the SoC, one for each stream. Each DMA engine is connected
+  to a different source pad of the receiver.
+
+The sensor and the receiver are modelled as V4L2 sub-devices, exposed to
+userspace via /dev/v4l-subdevX device nodes. The DMA engines are modelled as
+V4L2 video devices, exposed to userspace via /dev/videoX nodes.
+
+To configure this pipeline, the userspace must take the following steps:
+
+1) Set up media links between entities: enable the links from the sensor to the
+   receiver and from the receiver to the DMA engines. This step does not differ
+   from normal non-multiplexed media controller setup.
+
+2) Configure routing
+
+.. flat-table:: Camera sensor. There are no configurable routes.
+    :header-rows: 1
+
+    * - Sink Pad/Stream
+      - Source Pad/Stream
+      - Routing Flags
+      - Comments
+    * - 1/0
+      - 0/0
+      - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_IMMUTABLE |
+        V4L2_SUBDEV_ROUTE_FL_STATIC
+      - Pixel data stream from the internal image sink pad
+    * - 2/0
+      - 0/1
+      - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_IMMUTABLE |
+        V4L2_SUBDEV_ROUTE_FL_STATIC
+      - Metadata stream from the internal embedded data sink pad
+
+While both routes are immutable for the IMX219, other camera sensors may offer
+more flexible configuration options. Routing configuration is dictated by the
+hardware capabilities: camera sensors typically always produce an image data
+stream, but some of them support enabling and disabling the embedded data
+stream.
+
+.. flat-table:: Receiver routing table. Typically both routes need to be
+		explicitly created.
+    :header-rows:  1
+
+    * - Sink Pad/Stream
+      - Source Pad/Stream
+      - Routing Flags
+      - Comments
+    * - 0/0
+      - 1/0
+      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+      - Pixel data stream from camera sensor
+    * - 0/1
+      - 2/0
+      - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+      - Metadata stream from camera sensor
+
+3) Configure formats
+
+   This example assumes that the formats are propagated from sink pad to the
+   source pad as-is. The tables contain fields of both struct v4l2_subdev_format
+   and struct v4l2_mbus_framefmt.
+
+.. flat-table:: Formats set on the sub-devices. Bold values are set, others are
+                static or propagated. The order is aligned with configured
+                routes.
+    :header-rows: 1
+    :fill-cells:
+
+    * - Sub-device
+      - Pad/Stream
+      - Width
+      - Height
+      - Code
+    * - :rspan:`3` IMX219
+      - 1/0
+      - 3296
+      - 2480
+      - MEDIA_BUS_FMT_RAW_10
+    * - 0/0
+      - 3296
+      - 2480
+      - **MEDIA_BUS_FMT_RAW_10**
+    * - 2/0
+      - 3296
+      - 2
+      - MEDIA_BUS_FMT_META_10
+    * - 0/1
+      - 3296
+      - 2
+      - MEDIA_BUS_FMT_META_10
+    * - :rspan:`3` CSI-2 receiver
+      - 0/0
+      - **3296**
+      - **2480**
+      - **MEDIA_BUS_FMT_RAW_10**
+    * - 1/0
+      - 3296
+      - 2480
+      - MEDIA_BUS_FMT_RAW_10
+    * - 0/1
+      - **3296**
+      - **2**
+      - **MEDIA_BUS_FMT_META_10**
+    * - 2/0
+      - 3296
+      - 2
+      - MEDIA_BUS_FMT_META_10
+
+The embedded data format does not need to be configured on the sensor's pads as
+the format is dictated by the pixel data format in this case.
-- 
2.47.3


  parent reply	other threads:[~2026-11-18 13:29 UTC|newest]

Thread overview: 74+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 13:27 [PATCH v13 00/72] Generic line based metadata support, internal pads Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 01/73] media: uapi: Bump the STREAMS bit a little Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 02/73] Revert "media: mc: Keep INTERNAL pad flags just for the kernel, for now" Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 03/73] media: uapi: Add generic CSI-2 raw pixelformats Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 04/73] media: uapi: Add new media bus codes for generic raw formats Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 05/73] media: uapi: Add V4L2_CID_CONFIG_MODEL control Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 06/73] media: uapi: Add V4L2_CID_CFA_PATTERN for describing color patterns Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 07/73] media: uapi: Add monochromatic CFA pattern Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 08/73] media: Documentation: Reference CFA pattern control in pixelformat docs Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 09/73] media: Documentation: Reference CFA_PATTERN control in raw mbus docs Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 10/73] media: v4l: uapi: Add a control for color pattern flipping effect Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 11/73] media: Documentation: Reference flipping controls in raw format docs Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 12/73] media: Documentation: Document raw mbus codes and CFA for cameras Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 13/73] media: uapi: Add V4L2_CID_METADATA_LAYOUT control Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 14/73] media: Documentation: Refer to metadata layout in metadata documentation Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 15/73] media: Documentation: v4l: Document internal sink pads Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 16/73] media: Documentation: Document embedded data guidelines for camera sensors Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 17/73] media: uapi: Correct generic CSI-2 metadata format 4cc Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 18/73] Revert "media: uapi: v4l: Don't expose generic metadata formats to userspace" Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 19/73] media: v4l: Add V4L2_SUBDEV_ROUTE_FL_IMMUTABLE sub-device routing flag Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 20/73] media: v4l: Add V4L2_SUBDEV_ROUTE_FL_STATIC " Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 21/73] media: Documentation: There are either immutable or mutable routes Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 22/73] media: Documentation: Document IMMUTABLE and STATIC route flags Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 23/73] media: Documentation: Add subdev configuration models, raw sensor model Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 24/73] media: v4l2-subdev: Prevent accessing internal pads without STREAMS cap Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 25/73] media: subdev: Make get_fmt on INTERNAL pads without STREAMS an error Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 26/73] media: Documentation: Add scaling and post-scaler crop for common raw Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 27/73] media: uapi: Add MIPI CCS configuration model Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 28/73] media: uapi: Add V4L2_SUBDEV_CLIENT_CAP_COMMON_RAW_SENSOR Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 29/73] media: uapi: Add V4L2_CID_BINNING_FACTORS control for binning configuration Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 30/73] media: uapi: Add controls for sub-sampling configuration Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 31/73] media: uapi: Add new controls for camera sensor FLL and LLP Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 32/73] media: v4l2-common: Add v4l2_sensor_fll_llp_set() Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 33/73] media: Documentation: Add binning and sub-sampling controls Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 34/73] media: v4l2-subdev: Set STATIC route flag if IMMUTABLE route flag is set Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 35/73] media: uapi: v4l: subdev: Enable streams API Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 36/73] media: uapi: Add binning factor helper macros Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 37/73] media: Documentation: Document frame controls for common raw sensor model Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 38/73] media: v4l2-ctrl: Improve v4l2_ctrl_g_ctrl{,int64} documentation Sakari Ailus
2026-11-18 13:27 ` Sakari Ailus [this message]
2026-11-18 13:27 ` [PATCH v13 40/73] media: Documentation: Document streaming behaviour for common raw cameras Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 41/73] media: Documentation: Add a note on Bayer raw mbus codes Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 42/73] media: uapi: Add generic raw pixelformats for 10, 12 and 14 bits Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 43/73] media: uapi: Add more media bus codes for generic raw formats Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 44/73] media: uapi: ccs: Add metadata layout for MIPI CCS embedded data Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 45/73] media: Documentation: Document non-CCS use of CCS embedded data layout Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 46/73] media: ccs: Add support for generic raw mbus codes Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 47/73] media: ccs: Add support for embedded data stream Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 48/73] media: Documentation: ccs: Document routing Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 49/73] media: ccs: Add IMMUTABLE and STATIC route flags Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 50/73] media: uapi: Add metadata layout for ov2740 embedded data Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 51/73] media: ov2740: Add support for " Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 52/73] media: ov2740: Add support for generic raw formats Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 53/73] media: ov2740: Add metadata layout control Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 54/73] media: ov2740: Add support for G_SELECTION IOCTL Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 55/73] media: ov2740: Add support for FLL and LLP controls Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 56/73] media: ov2740: Signal common raw sensor model support Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 57/73] media: ov2740: Add IMMUTABLE and STATIC route flags Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 58/73] media: imx219: Add internal image pad for Common Raw Sensor Model Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 59/73] media: imx219: Add image stream Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 60/73] media: imx219: Report internal routes to userspace Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 61/73] media: imx219: Report streams using frame descriptors Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 62/73] media: imx219: Add embedded data support Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 63/73] media: imx219: Add support for generic raw formats Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 64/73] media: imx219: Add V4L2_CID_BINNING_FACTORS Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 65/73] media: imx219: Allow configuring cropping and binning through CRSM Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 66/73] media: imx219: Support LINE_LENGTH_PIXELS and FRAME_LENGTH_LINES controls Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 67/73] media: v4l2-common: Add helper function media_bus_fmt_to_csi2_(bpp|dt)() Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 68/73] media: bcm2835-unicam: Support generic raw formats Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 69/73] media: ipu6: Add support for raw CFA-agnostic formats Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 70/73] media: ipu6: Add support for generic unpacked pixel and metadata formats Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 71/73] Documentation: media: Reword blanking control documentation Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 72/73] media: uapi: Add generic raw pixelformat for 16 bpp Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 73/73] media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp Sakari Ailus

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118132819.213473-40-sakari.ailus@linux.intel.com \
    --to=sakari.ailus@linux.intel.com \
    --cc=antti.laakso@linux.intel.com \
    --cc=benjamin.mugnier@foss.st.com \
    --cc=christophe.jaillet@wanadoo.fr \
    --cc=dave.stevenson@raspberrypi.com \
    --cc=david.plowman@raspberrypi.com \
    --cc=divyamani.tripathi@intel.com \
    --cc=dongcheng.yan@intel.com \
    --cc=git@apitzsch.eu \
    --cc=hans@jjverkuil.nl \
    --cc=hansg@kernel.org \
    --cc=heimir.sverrisson@gmail.com \
    --cc=hpa@redhat.com \
    --cc=jacopo.mondi@ideasonboard.com \
    --cc=jai.luthra@ideasonboard.com \
    --cc=julien.massot@collabora.com \
    --cc=khai.wen.ng@intel.com \
    --cc=kieran.bingham@ideasonboard.com \
    --cc=laurent.pinchart@ideasonboard.com \
    --cc=linux-media@vger.kernel.org \
    --cc=manik.bajpai@intel.com \
    --cc=mehdi.djait@linux.intel.com \
    --cc=mirela.rabulea@nxp.com \
    --cc=mkorpershoek@kernel.org \
    --cc=naush@raspberrypi.com \
    --cc=ong.hock.yu@intel.com \
    --cc=prabhakar.csengg@gmail.com \
    --cc=r-donadkar@ti.com \
    --cc=ribalda@kernel.org \
    --cc=stefan.klug@ideasonboard.com \
    --cc=sylvain.petinot@foss.st.com \
    --cc=tomi.valkeinen@ideasonboard.com \
    --cc=tomm.merciai@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox