From: Sakari Ailus <sakari.ailus@linux.intel.com>
To: linux-media@vger.kernel.org
Cc: hans@jjverkuil.nl, laurent.pinchart@ideasonboard.com,
Prabhakar <prabhakar.csengg@gmail.com>,
"Kate Hsuan" <hpa@redhat.com>,
"Dave Stevenson" <dave.stevenson@raspberrypi.com>,
"Tommaso Merciai" <tomm.merciai@gmail.com>,
"Benjamin Mugnier" <benjamin.mugnier@foss.st.com>,
"Sylvain Petinot" <sylvain.petinot@foss.st.com>,
"Christophe JAILLET" <christophe.jaillet@wanadoo.fr>,
"Julien Massot" <julien.massot@collabora.com>,
"Naushir Patuck" <naush@raspberrypi.com>,
"Yan, Dongcheng" <dongcheng.yan@intel.com>,
"Stefan Klug" <stefan.klug@ideasonboard.com>,
"Mirela Rabulea" <mirela.rabulea@nxp.com>,
"André Apitzsch" <git@apitzsch.eu>,
"Heimir Thor Sverrisson" <heimir.sverrisson@gmail.com>,
"Kieran Bingham" <kieran.bingham@ideasonboard.com>,
"Mehdi Djait" <mehdi.djait@linux.intel.com>,
"Ricardo Ribalda Delgado" <ribalda@kernel.org>,
"Hans de Goede" <hansg@kernel.org>,
"Jacopo Mondi" <jacopo.mondi@ideasonboard.com>,
"Tomi Valkeinen" <tomi.valkeinen@ideasonboard.com>,
"David Plowman" <david.plowman@raspberrypi.com>,
"Yu, Ong Hock" <ong.hock.yu@intel.com>,
"Ng, Khai Wen" <khai.wen.ng@intel.com>,
"Jai Luthra" <jai.luthra@ideasonboard.com>,
"Rishikesh Donadkar" <r-donadkar@ti.com>,
"Mattijs Korpershoek" <mkorpershoek@kernel.org>,
"Antti Laakso" <antti.laakso@linux.intel.com>,
"Bajpai, Manik" <manik.bajpai@intel.com>,
"Divyamani Tripathi" <divyamani.tripathi@intel.com>
Subject: [PATCH v13 39/73] media: Documentation: Add sub-device internal pads example
Date: Wed, 18 Nov 2026 15:27:45 +0200 [thread overview]
Message-ID: <20261118132819.213473-40-sakari.ailus@linux.intel.com> (raw)
In-Reply-To: <20261118132819.213473-1-sakari.ailus@linux.intel.com>
Add an example using sub-device internal pads, with the Sony IMX219 camera
sensor.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
.../userspace-api/media/v4l/dev-subdev.rst | 129 ++++++++++++++++++
1 file changed, 129 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst
index 3472b0591d79..f6c4dfcff06d 100644
--- a/Documentation/userspace-api/media/v4l/dev-subdev.rst
+++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst
@@ -726,3 +726,132 @@ To configure this pipeline, the userspace must take the following steps:
:ref:`V4L2_CID_METADATA_LAYOUT <image_source_control_metadata_layout>`
controls are present on the source sub-device to obtain the pixel array CFA
pattern and metadata layout.
+
+Internal pads setup example
+---------------------------
+
+A simple example of a multiplexed stream setup might be as follows:
+
+- An IMX219 camera sensor source sub-device, with one source pad (0), one
+ internal sink pad (1) as the source of the image data and an internal sink
+ pad (2) as the source of the embedded data. There are two routes, one from the
+ internal sink pad 1 to the source pad 0 (image data) and another from the
+ internal sink pad 2 to the source pad 0 (embedded data). Both streams are
+ always active, i.e. there is no need to separately enable the embedded data
+ stream. The sensor uses the CSI-2 interface.
+
+- A CSI-2 receiver in the SoC. The receiver has a single sink pad (pad 0),
+ connected to the sensor's source pad (0), and two source pads (pads 1 and 2),
+ to two DMA engines. The receiver demultiplexes the incoming streams to the
+ source pads.
+
+- Two DMA engines in the SoC, one for each stream. Each DMA engine is connected
+ to a different source pad of the receiver.
+
+The sensor and the receiver are modelled as V4L2 sub-devices, exposed to
+userspace via /dev/v4l-subdevX device nodes. The DMA engines are modelled as
+V4L2 video devices, exposed to userspace via /dev/videoX nodes.
+
+To configure this pipeline, the userspace must take the following steps:
+
+1) Set up media links between entities: enable the links from the sensor to the
+ receiver and from the receiver to the DMA engines. This step does not differ
+ from normal non-multiplexed media controller setup.
+
+2) Configure routing
+
+.. flat-table:: Camera sensor. There are no configurable routes.
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 1/0
+ - 0/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_IMMUTABLE |
+ V4L2_SUBDEV_ROUTE_FL_STATIC
+ - Pixel data stream from the internal image sink pad
+ * - 2/0
+ - 0/1
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE | V4L2_SUBDEV_ROUTE_FL_IMMUTABLE |
+ V4L2_SUBDEV_ROUTE_FL_STATIC
+ - Metadata stream from the internal embedded data sink pad
+
+While both routes are immutable for the IMX219, other camera sensors may offer
+more flexible configuration options. Routing configuration is dictated by the
+hardware capabilities: camera sensors typically always produce an image data
+stream, but some of them support enabling and disabling the embedded data
+stream.
+
+.. flat-table:: Receiver routing table. Typically both routes need to be
+ explicitly created.
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 0/0
+ - 1/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from camera sensor
+ * - 0/1
+ - 2/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from camera sensor
+
+3) Configure formats
+
+ This example assumes that the formats are propagated from sink pad to the
+ source pad as-is. The tables contain fields of both struct v4l2_subdev_format
+ and struct v4l2_mbus_framefmt.
+
+.. flat-table:: Formats set on the sub-devices. Bold values are set, others are
+ static or propagated. The order is aligned with configured
+ routes.
+ :header-rows: 1
+ :fill-cells:
+
+ * - Sub-device
+ - Pad/Stream
+ - Width
+ - Height
+ - Code
+ * - :rspan:`3` IMX219
+ - 1/0
+ - 3296
+ - 2480
+ - MEDIA_BUS_FMT_RAW_10
+ * - 0/0
+ - 3296
+ - 2480
+ - **MEDIA_BUS_FMT_RAW_10**
+ * - 2/0
+ - 3296
+ - 2
+ - MEDIA_BUS_FMT_META_10
+ * - 0/1
+ - 3296
+ - 2
+ - MEDIA_BUS_FMT_META_10
+ * - :rspan:`3` CSI-2 receiver
+ - 0/0
+ - **3296**
+ - **2480**
+ - **MEDIA_BUS_FMT_RAW_10**
+ * - 1/0
+ - 3296
+ - 2480
+ - MEDIA_BUS_FMT_RAW_10
+ * - 0/1
+ - **3296**
+ - **2**
+ - **MEDIA_BUS_FMT_META_10**
+ * - 2/0
+ - 3296
+ - 2
+ - MEDIA_BUS_FMT_META_10
+
+The embedded data format does not need to be configured on the sensor's pads as
+the format is dictated by the pixel data format in this case.
--
2.47.3
next prev parent reply other threads:[~2026-11-18 13:29 UTC|newest]
Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 13:27 [PATCH v13 00/72] Generic line based metadata support, internal pads Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 01/73] media: uapi: Bump the STREAMS bit a little Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 02/73] Revert "media: mc: Keep INTERNAL pad flags just for the kernel, for now" Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 03/73] media: uapi: Add generic CSI-2 raw pixelformats Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 04/73] media: uapi: Add new media bus codes for generic raw formats Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 05/73] media: uapi: Add V4L2_CID_CONFIG_MODEL control Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 06/73] media: uapi: Add V4L2_CID_CFA_PATTERN for describing color patterns Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 07/73] media: uapi: Add monochromatic CFA pattern Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 08/73] media: Documentation: Reference CFA pattern control in pixelformat docs Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 09/73] media: Documentation: Reference CFA_PATTERN control in raw mbus docs Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 10/73] media: v4l: uapi: Add a control for color pattern flipping effect Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 11/73] media: Documentation: Reference flipping controls in raw format docs Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 12/73] media: Documentation: Document raw mbus codes and CFA for cameras Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 13/73] media: uapi: Add V4L2_CID_METADATA_LAYOUT control Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 14/73] media: Documentation: Refer to metadata layout in metadata documentation Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 15/73] media: Documentation: v4l: Document internal sink pads Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 16/73] media: Documentation: Document embedded data guidelines for camera sensors Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 17/73] media: uapi: Correct generic CSI-2 metadata format 4cc Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 18/73] Revert "media: uapi: v4l: Don't expose generic metadata formats to userspace" Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 19/73] media: v4l: Add V4L2_SUBDEV_ROUTE_FL_IMMUTABLE sub-device routing flag Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 20/73] media: v4l: Add V4L2_SUBDEV_ROUTE_FL_STATIC " Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 21/73] media: Documentation: There are either immutable or mutable routes Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 22/73] media: Documentation: Document IMMUTABLE and STATIC route flags Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 23/73] media: Documentation: Add subdev configuration models, raw sensor model Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 24/73] media: v4l2-subdev: Prevent accessing internal pads without STREAMS cap Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 25/73] media: subdev: Make get_fmt on INTERNAL pads without STREAMS an error Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 26/73] media: Documentation: Add scaling and post-scaler crop for common raw Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 27/73] media: uapi: Add MIPI CCS configuration model Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 28/73] media: uapi: Add V4L2_SUBDEV_CLIENT_CAP_COMMON_RAW_SENSOR Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 29/73] media: uapi: Add V4L2_CID_BINNING_FACTORS control for binning configuration Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 30/73] media: uapi: Add controls for sub-sampling configuration Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 31/73] media: uapi: Add new controls for camera sensor FLL and LLP Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 32/73] media: v4l2-common: Add v4l2_sensor_fll_llp_set() Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 33/73] media: Documentation: Add binning and sub-sampling controls Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 34/73] media: v4l2-subdev: Set STATIC route flag if IMMUTABLE route flag is set Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 35/73] media: uapi: v4l: subdev: Enable streams API Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 36/73] media: uapi: Add binning factor helper macros Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 37/73] media: Documentation: Document frame controls for common raw sensor model Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 38/73] media: v4l2-ctrl: Improve v4l2_ctrl_g_ctrl{,int64} documentation Sakari Ailus
2026-11-18 13:27 ` Sakari Ailus [this message]
2026-11-18 13:27 ` [PATCH v13 40/73] media: Documentation: Document streaming behaviour for common raw cameras Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 41/73] media: Documentation: Add a note on Bayer raw mbus codes Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 42/73] media: uapi: Add generic raw pixelformats for 10, 12 and 14 bits Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 43/73] media: uapi: Add more media bus codes for generic raw formats Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 44/73] media: uapi: ccs: Add metadata layout for MIPI CCS embedded data Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 45/73] media: Documentation: Document non-CCS use of CCS embedded data layout Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 46/73] media: ccs: Add support for generic raw mbus codes Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 47/73] media: ccs: Add support for embedded data stream Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 48/73] media: Documentation: ccs: Document routing Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 49/73] media: ccs: Add IMMUTABLE and STATIC route flags Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 50/73] media: uapi: Add metadata layout for ov2740 embedded data Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 51/73] media: ov2740: Add support for " Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 52/73] media: ov2740: Add support for generic raw formats Sakari Ailus
2026-11-18 13:27 ` [PATCH v13 53/73] media: ov2740: Add metadata layout control Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 54/73] media: ov2740: Add support for G_SELECTION IOCTL Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 55/73] media: ov2740: Add support for FLL and LLP controls Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 56/73] media: ov2740: Signal common raw sensor model support Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 57/73] media: ov2740: Add IMMUTABLE and STATIC route flags Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 58/73] media: imx219: Add internal image pad for Common Raw Sensor Model Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 59/73] media: imx219: Add image stream Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 60/73] media: imx219: Report internal routes to userspace Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 61/73] media: imx219: Report streams using frame descriptors Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 62/73] media: imx219: Add embedded data support Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 63/73] media: imx219: Add support for generic raw formats Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 64/73] media: imx219: Add V4L2_CID_BINNING_FACTORS Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 65/73] media: imx219: Allow configuring cropping and binning through CRSM Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 66/73] media: imx219: Support LINE_LENGTH_PIXELS and FRAME_LENGTH_LINES controls Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 67/73] media: v4l2-common: Add helper function media_bus_fmt_to_csi2_(bpp|dt)() Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 68/73] media: bcm2835-unicam: Support generic raw formats Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 69/73] media: ipu6: Add support for raw CFA-agnostic formats Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 70/73] media: ipu6: Add support for generic unpacked pixel and metadata formats Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 71/73] Documentation: media: Reword blanking control documentation Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 72/73] media: uapi: Add generic raw pixelformat for 16 bpp Sakari Ailus
2026-11-18 13:28 ` [PATCH v13 73/73] media: uapi: Add generic CSI-2 raw pixelformats for 16, 20, 24 and 28 bpp Sakari Ailus
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118132819.213473-40-sakari.ailus@linux.intel.com \
--to=sakari.ailus@linux.intel.com \
--cc=antti.laakso@linux.intel.com \
--cc=benjamin.mugnier@foss.st.com \
--cc=christophe.jaillet@wanadoo.fr \
--cc=dave.stevenson@raspberrypi.com \
--cc=david.plowman@raspberrypi.com \
--cc=divyamani.tripathi@intel.com \
--cc=dongcheng.yan@intel.com \
--cc=git@apitzsch.eu \
--cc=hans@jjverkuil.nl \
--cc=hansg@kernel.org \
--cc=heimir.sverrisson@gmail.com \
--cc=hpa@redhat.com \
--cc=jacopo.mondi@ideasonboard.com \
--cc=jai.luthra@ideasonboard.com \
--cc=julien.massot@collabora.com \
--cc=khai.wen.ng@intel.com \
--cc=kieran.bingham@ideasonboard.com \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-media@vger.kernel.org \
--cc=manik.bajpai@intel.com \
--cc=mehdi.djait@linux.intel.com \
--cc=mirela.rabulea@nxp.com \
--cc=mkorpershoek@kernel.org \
--cc=naush@raspberrypi.com \
--cc=ong.hock.yu@intel.com \
--cc=prabhakar.csengg@gmail.com \
--cc=r-donadkar@ti.com \
--cc=ribalda@kernel.org \
--cc=stefan.klug@ideasonboard.com \
--cc=sylvain.petinot@foss.st.com \
--cc=tomi.valkeinen@ideasonboard.com \
--cc=tomm.merciai@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox