[PATCH 00/13] Streaming control for MC with metadata or streams otherwise

[PATCH 00/13] Streaming control for MC with metadata or streams otherwise

[Sakari Ailus]

Hi all,

Starting and stopping streaming is a non-trivial problem in all but most
simple cases where there's only one stream. But when multiple streams come
into play, the problem quickly gets difficult to solve properly as it
generally spans multiple hardware deveices, not only including
transmitters (e.g. camera sensors) and receivers but also devices in
between, such as CSI-2 aggregators and serdes devices.

The following document summarises the known use cases:
<URL:https://pilvi.retiisi.eu/s/44RWYjKG92R6X49>, please let me know if
you have something to add that's not included here.

What this patchset implements is a partial solution, but it's also a
solution that can be extended to cover the above cases with relatively
small amendments.

The current functionality is best documented in patch "media: v4l2-mc:
Introduce v4l2_mc_pipeline_enabled()".

This set, in some form, is also effectively required by metadata support
patchset of which I'm about to post a new version. Starting per VC (for
CSI-2) is next on my to-do list when it comes to this set.

Getting these both in for 6.18 would be great.

Comments are very welcome.

Sakari Ailus (13):
  media: ipu6: Use correct pads for xlate_streams()
  media: ipu6: Set minimum height to 1
  media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
  media: v4l2-subdev: Add a helper to figure out the pad streaming state
  media: v4l: Make media_entity_to_video_device() NULL-safe
  media: v4l2-subdev: Mark both streams of a route enabled
  media: ipu6: Set up CSI-2 receiver at correct moment
  media: v4l2-subdev: Print early in
    v4l2_subdev_{enable,disable}_streams()
  media: v4l2-subdev: Collect streams on source pads only
  media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()
  media: v4l2-subdev: Introduce v4l2_subdev_find_route()
  media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
  media: ipu6: isys: Rework stream starting and stopping

 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c |  63 +++--
 .../media/pci/intel/ipu6/ipu6-isys-queue.c    |  48 ++--
 .../media/pci/intel/ipu6/ipu6-isys-video.c    |  96 ++++---
 .../media/pci/intel/ipu6/ipu6-isys-video.h    |   9 +-
 drivers/media/pci/intel/ipu6/ipu6-isys.h      |   2 +-
 drivers/media/v4l2-core/v4l2-mc.c             | 243 ++++++++++++++++++
 drivers/media/v4l2-core/v4l2-subdev.c         | 107 +++++---
 include/media/v4l2-dev.h                      |  14 +-
 include/media/v4l2-mc.h                       |  44 ++++
 include/media/v4l2-subdev.h                   |  22 ++
 10 files changed, 539 insertions(+), 109 deletions(-)

-- 
2.39.5

[PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

[Sakari Ailus]

The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the
source pads was used as the sink pad and the source pad was a constant
(rather than the actual source pad). Fix these.

Fixes: 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
Cc: stable@vger.kernel.org
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
index da8581a37e22..6030bd23b4b9 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
@@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);
 
 	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
 	if (ret)
@@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 	struct media_pad *remote_pad;
 	u64 sink_streams;
 
-	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
-						       CSI2_PAD_SINK,
-						       &streams_mask);
+	sink_streams =
+		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
+						&streams_mask);
 
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
-- 
2.39.5

[PATCH 02/13] media: ipu6: Set minimum height to 1

[Sakari Ailus]

For image data generally 2 seems like a minimum height that surely won't
cause any issues, but some sensors have metadata the height of which is
just one line. Set the minimum height to 1.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/pci/intel/ipu6/ipu6-isys.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.h b/drivers/media/pci/intel/ipu6/ipu6-isys.h
index f488e782c26e..0e2c8b71edfc 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys.h
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys.h
@@ -40,7 +40,7 @@ struct ipu6_bus_device;
 #define IPU6_ISYS_NUM_RECV_QUEUE 1
 
 #define IPU6_ISYS_MIN_WIDTH		2U
-#define IPU6_ISYS_MIN_HEIGHT		2U
+#define IPU6_ISYS_MIN_HEIGHT		1U
 #define IPU6_ISYS_MAX_WIDTH		4672U
 #define IPU6_ISYS_MAX_HEIGHT		3416U
 
-- 
2.39.5

[PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

[Sakari Ailus]

On UAPI, streaming is started a video device at a time. The IPU6 ISYS
driver only starts streaming on the source sub-device when all relevant
video devices have been set streaming. This also needs to be reflected in
the sub-device pads, hence set them all streaming, one at a time.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 .../media/pci/intel/ipu6/ipu6-isys-video.c    | 43 ++++++++++++-------
 1 file changed, 28 insertions(+), 15 deletions(-)

diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
index 24a2ef93474c..54006b5e2ccd 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
@@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	struct device *dev = &av->isys->adev->auxdev.dev;
 	struct v4l2_subdev *sd;
 	struct media_pad *r_pad;
+	unsigned int i;
 	u32 sink_pad, sink_stream;
 	u64 r_stream;
 	u64 stream_mask = 0;
@@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		stop_streaming_firmware(av);
 
 		/* stop sub-device which connects with video */
-		dev_dbg(dev, "stream off entity %s pad:%d mask:0x%llx\n",
-			sd->name, r_pad->index, stream_mask);
-		ret = v4l2_subdev_disable_streams(sd, r_pad->index,
-						  stream_mask);
-		if (ret) {
-			dev_err(dev, "stream off %s failed with %d\n", sd->name,
-				ret);
-			return ret;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_disable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream off %s failed with %d\n",
+					sd->name, ret);
+				return ret;
+			}
 		}
 		close_streaming_firmware(av);
 	} else {
@@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		}
 
 		/* start sub-device which connects with video */
-		dev_dbg(dev, "stream on %s pad %d mask 0x%llx\n", sd->name,
-			r_pad->index, stream_mask);
-		ret = v4l2_subdev_enable_streams(sd, r_pad->index, stream_mask);
-		if (ret) {
-			dev_err(dev, "stream on %s failed with %d\n", sd->name,
-				ret);
-			goto out_media_entity_stop_streaming_firmware;
+		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			if (!media_pad_pipeline(&sd->entity.pads[i]))
+				continue;
+			ret = v4l2_subdev_enable_streams(sd, i, 1U);
+			if (ret) {
+				dev_err(dev, "stream on %s failed with %d\n",
+					sd->name, ret);
+				goto out_media_entity_stop_streaming_firmware;
+			}
 		}
 	}
 
@@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 	return 0;
 
 out_media_entity_stop_streaming_firmware:
+	while (i-- > CSI2_PAD_SRC) {
+		int ret2;
+
+		if (!media_pad_pipeline(&sd->entity.pads[i]))
+			continue;
+		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
+		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
+	}
+
 	stop_streaming_firmware(av);
 
 	return ret;
-- 
2.39.5

[PATCH 04/13] media: v4l2-subdev: Add a helper to figure out the pad streaming state

[Sakari Ailus]

Add a helper, v4l2_subdev_state_streams_enabled(), to tell which streams
are enabled on a pad. This is useful to e.g. figure out in a driver
whether a hardware configuration change is necessary to enable a given
stream.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 17 +++++++++++++++++
 include/media/v4l2-subdev.h           |  3 +++
 2 files changed, 20 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index a3074f469b15..60b8febd3339 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2453,6 +2453,23 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_disable_streams);
 
+u64 v4l2_subdev_state_streams_enabled(struct v4l2_subdev *sd,
+				      struct v4l2_subdev_state *state, u32 pad)
+{
+	u64 streams_mask = 0;
+
+	for (unsigned int i = 0; i < state->stream_configs.num_configs; i++) {
+		struct v4l2_subdev_stream_config *cfg =
+			&state->stream_configs.configs[i];
+
+		if (cfg->pad == pad && cfg->enabled)
+			streams_mask |= BIT_ULL(cfg->stream);
+	}
+
+	return streams_mask;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_state_streams_enabled);
+
 int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable)
 {
 	struct v4l2_subdev_state *state;
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 57f2bcb4eb16..deab128a4779 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1731,6 +1731,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 				u64 streams_mask);
 
+u64 v4l2_subdev_state_streams_enabled(struct v4l2_subdev *sd,
+				      struct v4l2_subdev_state *state, u32 pad);
+
 /**
  * v4l2_subdev_s_stream_helper() - Helper to implement the subdev s_stream
  *	operation using enable_streams and disable_streams
-- 
2.39.5

[PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

[Sakari Ailus]

Make media_entity_to_video_device(NULL) return NULL, instead of an invalid
pointer value.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 include/media/v4l2-dev.h | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
index 1b6222fab24e..069c2f14b473 100644
--- a/include/media/v4l2-dev.h
+++ b/include/media/v4l2-dev.h
@@ -313,10 +313,16 @@ struct video_device {
  * media_entity_to_video_device - Returns a &struct video_device from
  *	the &struct media_entity embedded on it.
  *
- * @__entity: pointer to &struct media_entity
- */
-#define media_entity_to_video_device(__entity) \
-	container_of(__entity, struct video_device, entity)
+ * @__entity: pointer to &struct media_entity, may be NULL
+ */
+#define media_entity_to_video_device(__entity)				\
+	({								\
+		typeof (__entity) __me_to_vdev_ent = __entity;		\
+									\
+		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
+						struct video_device, entity) : \
+			NULL;						\
+	})
 
 /**
  * to_video_device - Returns a &struct video_device from the
-- 
2.39.5

[PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

[Sakari Ailus]

Streams are stored in sink-source pairs in sub-device state. When a stream
was marked enabled (or disabled), only the state of one end of the stream
was modified, leaving the stream in an incoherent state. Mark both ends of
the stream enabled (or disabled).

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 60b8febd3339..5afdd9d548b5 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
 	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
 		struct v4l2_subdev_stream_config *cfg =
 			&state->stream_configs.configs[i];
+		struct v4l2_subdev_stream_config *cfg2 =
+			&state->stream_configs.configs[i ^ 1U];
 
 		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
-			cfg->enabled = enabled;
+			cfg->enabled = cfg2->enabled = enabled;
 	}
 }
 
-- 
2.39.5

[PATCH 07/13] media: ipu6: Set up CSI-2 receiver at correct moment

[Sakari Ailus]

Enable the CSI-2 receiver before the first stream is started and disable
it when the last stream is stopped. Before this patch, every time a stream
was started, the CSI-2 receiver was enabled and similarly, it was disabled
when any stream was stopped.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 25 ++++++++++++-------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
index 6030bd23b4b9..3b837e9ccffe 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
@@ -348,7 +348,7 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	struct ipu6_isys_csi2_timing timing = { };
 	struct v4l2_subdev *remote_sd;
 	struct media_pad *remote_pad;
-	u64 sink_streams;
+	u64 sink_streams, already_enabled;
 	int ret;
 
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
@@ -358,13 +358,17 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
 						&streams_mask);
 
-	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
-	if (ret)
-		return ret;
+	already_enabled = v4l2_subdev_state_streams_enabled(sd, state,
+							    CSI2_PAD_SINK);
+	if (!already_enabled) {
+		ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
+		if (ret)
+			return ret;
 
-	ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
-	if (ret)
-		return ret;
+		ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
+		if (ret)
+			return ret;
+	}
 
 	ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index,
 					 sink_streams);
@@ -382,7 +386,7 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 {
 	struct v4l2_subdev *remote_sd;
 	struct media_pad *remote_pad;
-	u64 sink_streams;
+	u64 sink_streams, still_enabled;
 
 	sink_streams =
 		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
@@ -391,7 +395,10 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
 	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
 
-	ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
+	still_enabled = v4l2_subdev_state_streams_enabled(sd, state,
+							  CSI2_PAD_SINK);
+	if (still_enabled == sink_streams)
+		ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
 
 	v4l2_subdev_disable_streams(remote_sd, remote_pad->index, sink_streams);
 
-- 
2.39.5

[PATCH 08/13] media: v4l2-subdev: Print early in v4l2_subdev_{enable,disable}_streams()

[Sakari Ailus]

Print debug messages early in v4l2_subdev_enable_streams() and
v4l2_subdev_disable_streams(), before sanity checks take place. This can
help figuring out why something goes wrong, in driver development or
otherwise.

Also print the name of the sub-device where streaming is to be enabled or
disabled.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 5afdd9d548b5..6bc855058ca6 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2273,6 +2273,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 	bool use_s_stream;
 	int ret;
 
+	dev_dbg(dev, "enable streams %s:%u/%#llx\n", sd->entity.name, pad,
+		streams_mask);
+
 	/* A few basic sanity checks first. */
 	if (pad >= sd->entity.num_pads)
 		return -EINVAL;
@@ -2320,8 +2323,6 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 		goto done;
 	}
 
-	dev_dbg(dev, "enable streams %u:%#llx\n", pad, streams_mask);
-
 	already_streaming = v4l2_subdev_is_streaming(sd);
 
 	if (!use_s_stream) {
@@ -2373,6 +2374,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 	bool use_s_stream;
 	int ret;
 
+	dev_dbg(dev, "disable streams %s:%u/%#llx\n", sd->entity.name, pad,
+		streams_mask);
+
 	/* A few basic sanity checks first. */
 	if (pad >= sd->entity.num_pads)
 		return -EINVAL;
@@ -2420,8 +2424,6 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 		goto done;
 	}
 
-	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
-
 	if (!use_s_stream) {
 		/* Call the .disable_streams() operation. */
 		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
-- 
2.39.5

[PATCH 09/13] media: v4l2-subdev: Collect streams on source pads only

[Sakari Ailus]

v4l2_subdev_collect_streams() is used to find the streams present on
source pads only. Only iterate through the streams on source pads, i.e. on
odd array indices.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 6bc855058ca6..932fca795d4a 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2225,16 +2225,17 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 	*found_streams = 0;
 	*enabled_streams = 0;
 
-	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
-		const struct v4l2_subdev_stream_config *cfg =
-			&state->stream_configs.configs[i];
+	for (unsigned int i = 0; i < state->stream_configs.num_configs; i += 2) {
+		const struct v4l2_subdev_stream_config *src_cfg =
+			&state->stream_configs.configs[i + 1];
 
-		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
+		if (src_cfg->pad != pad ||
+		    !(streams_mask & BIT_ULL(src_cfg->stream)))
 			continue;
 
-		*found_streams |= BIT_ULL(cfg->stream);
-		if (cfg->enabled)
-			*enabled_streams |= BIT_ULL(cfg->stream);
+		*found_streams |= BIT_ULL(src_cfg->stream);
+		if (src_cfg->enabled)
+			*enabled_streams |= BIT_ULL(src_cfg->stream);
 	}
 }
 
-- 
2.39.5

[PATCH 10/13] media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()

[Sakari Ailus]

Print streams found by v4l2_subdev_collect_streams() at debug level.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 932fca795d4a..c549a462dac7 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2219,6 +2219,8 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 		*found_streams = BIT_ULL(0);
 		*enabled_streams =
 			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
+		dev_dbg(sd->dev,
+			"collect_streams: sub-device does not support streams\n");
 		return;
 	}
 
@@ -2237,6 +2239,9 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 		if (src_cfg->enabled)
 			*enabled_streams |= BIT_ULL(src_cfg->stream);
 	}
+	dev_dbg(sd->dev,
+		"collect_streams: %s:%u found %#llx enabled %#llx\n",
+		sd->entity.name, pad, *found_streams, *enabled_streams);
 }
 
 static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
-- 
2.39.5

[PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()

[Sakari Ailus]

v4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(),
with the difference that it's more flexible: it can look up only active
routes and can find multiple routes, too.

v4l2_subdev_find_route() is intended to replace
v4l2_subdev_routing_find_opposite_end().

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++---------
 include/media/v4l2-subdev.h           | 19 +++++++++
 2 files changed, 56 insertions(+), 19 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index c549a462dac7..13d6e96daf3a 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
 
-int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
-					  u32 pad, u32 stream, u32 *other_pad,
-					  u32 *other_stream)
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index)
 {
 	unsigned int i;
 
 	for (i = 0; i < routing->num_routes; ++i) {
 		struct v4l2_subdev_route *route = &routing->routes[i];
 
-		if (route->source_pad == pad &&
-		    route->source_stream == stream) {
-			if (other_pad)
-				*other_pad = route->sink_pad;
-			if (other_stream)
-				*other_stream = route->sink_stream;
-			return 0;
-		}
+		if (active && !(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+			continue;
 
-		if (route->sink_pad == pad && route->sink_stream == stream) {
-			if (other_pad)
-				*other_pad = route->source_pad;
-			if (other_stream)
-				*other_stream = route->source_stream;
-			return 0;
-		}
+		if ((route->source_pad != pad ||
+		     route->source_stream != stream) &&
+		    (route->sink_pad != pad || route->sink_stream != stream))
+			continue;
+
+		if (index--)
+			continue;
+
+		return route;
 	}
 
-	return -EINVAL;
+	return ERR_PTR(-ENOENT);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_find_route);
+
+int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
+					  u32 pad, u32 stream, u32 *other_pad,
+					  u32 *other_stream)
+{
+	struct v4l2_subdev_route *route;
+
+	route = v4l2_subdev_find_route(routing, pad, stream, false, 0);
+	if (IS_ERR(route))
+		return PTR_ERR(route);
+
+	bool is_source = route->source_pad == pad;
+
+	if (other_pad)
+		*other_pad = is_source ? route->sink_pad : route->source_pad;
+	if (other_stream)
+		*other_stream = is_source ?
+			route->sink_stream : route->source_stream;
+
+	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
 
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index deab128a4779..9ed8600ba3d4 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
 				     const struct v4l2_subdev_krouting *routing,
 				     const struct v4l2_mbus_framefmt *fmt);
 
+/**
+ * v4l2_subdev_find_route() - Find routes from a (pad, stream) pair
+ * @routing: routing used to find the opposite side
+ * @pad: pad id
+ * @stream: stream id
+ * @active: set to true for looking up only active routes
+ * @index: for accessing more than one route from the pad
+ *
+ * Find a route from the routing table where one end has (pad, stream) pair
+ * matching @pad and @stream.
+ *
+ * Returns the route on success or -ENOENT if no matching route is found.
+ */
+struct v4l2_subdev_route *
+v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
+		       u32 pad, u32 stream, bool active, unsigned int index);
+
 /**
  * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
  * @routing: routing used to find the opposite side
@@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
  * @other_pad: pointer used to return the opposite pad
  * @other_stream: pointer used to return the opposite stream
  *
+ * Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end().
+ *
  * This function uses the routing table to find the pad + stream which is
  * opposite the given pad + stream.
  *
-- 
2.39.5

[PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Sakari Ailus]

v4l2_mc_pipeline_enabled() helps solving a problem known for long but
lacked any sort of general solution: with multiple streams, when streaming
is started on video nodes one by one, when should streaming be started in
the source?

v4l2_mc_pipeline_enabled() traverses the pipeline towards the source,
queries the streams generated by the source and traces them back to the
video nodes.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
 include/media/v4l2-mc.h           |  44 ++++++
 2 files changed, 287 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
index 937d358697e1..1731088ad436 100644
--- a/drivers/media/v4l2-core/v4l2-mc.c
+++ b/drivers/media/v4l2-core/v4l2-mc.c
@@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
+
+static int
+__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
+			   struct media_pad *src_pad, u64 __src_streams,
+			   struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	struct v4l2_subdev_route *route;
+	u64 src_streams = 0, sink_streams = 0;
+	bool has_sink_pad = false;
+	unsigned int sink_pad;
+
+	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
+		state->sd->entity.name, src_pad->index, __src_streams);
+	for_each_active_route(&state->routing, route) {
+		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
+			state->sd->entity.name,
+			route->sink_pad, route->sink_stream, route->source_pad,
+			route->source_stream, route->flags);
+		if (route->source_pad != src_pad->index)
+			continue;
+
+		if (!(BIT_ULL(route->source_stream) & __src_streams))
+			continue;
+
+		if (!has_sink_pad) {
+			has_sink_pad = true;
+			sink_pad = route->sink_pad;
+		}
+
+		if (route->sink_pad != sink_pad) {
+			dev_dbg(state->sd->dev,
+				"sink pads (%u vs. %u) differ\n",
+				route->sink_pad, sink_pad);
+			return -EMLINK;
+		}
+
+		sink_streams |= BIT_ULL(route->sink_stream);
+		src_streams |= BIT_ULL(route->source_stream);
+	}
+
+	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
+	*__sink_streams = sink_streams;
+
+	return 0;
+}
+
+static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
+				    unsigned int sink_stream,
+				    bool (*func)(struct video_device *vdev),
+				    struct media_pad **__sink_pad,
+				    u64 *__sink_streams)
+{
+	struct v4l2_subdev_state *state;
+	struct v4l2_subdev_route *route;
+	struct v4l2_subdev *sd;
+	struct media_pad *source_pad, *tmp_pad;
+	u32 source_stream;
+
+	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
+		return -ENXIO;
+
+	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
+	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
+		sd->entity.name);
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
+				       sink_stream, true, 0);
+	if (IS_ERR(route)) {
+		v4l2_subdev_unlock_state(state);
+		dev_dbg(sd->dev,
+			"path_enabled: can't find opposite route for %s:%u/%u",
+			sd->entity.name, sink_pad->index, sink_stream);
+		return 2;
+	}
+
+	source_pad = &sd->entity.pads[route->source_pad];
+	v4l2_subdev_unlock_state(state);
+
+	tmp_pad = sink_pad;
+	sink_pad = media_pad_remote_pad_unique(source_pad);
+	if (IS_ERR(sink_pad)) {
+		dev_dbg(sd->dev,
+			"path_enabled: can't find remote source for %s:%u\n",
+			source_pad->entity->name, source_pad->index);
+		return PTR_ERR(sink_pad);
+	}
+
+	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
+		struct video_device *vdev;
+
+		vdev = media_entity_to_video_device(sink_pad->entity);
+		if (!vdev)
+			return -ENXIO;
+
+		dev_dbg(vdev->dev_parent,
+			"path_enabled: found video device %s\n",
+			vdev->name);
+
+		if (!*__sink_pad) {
+			*__sink_pad = tmp_pad;
+			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
+				tmp_pad->index, sink_stream);
+		} else if (tmp_pad != *__sink_pad) {
+			dev_dbg(sd->dev,
+				"path_enabled: pads %s/%u and %s/%u differ\n",
+				tmp_pad->entity->name, tmp_pad->index,
+				(*__sink_pad)->entity->name,
+				(*__sink_pad)->index);
+			return -EXDEV;
+		}
+
+		*__sink_streams |= BIT_ULL(sink_stream);
+
+		return func(vdev);
+	}
+
+	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
+					__sink_pad, __sink_streams);
+}
+
+static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
+				      u64 *__streams)
+{
+	struct v4l2_mbus_frame_desc desc;
+	u64 streams = 0;
+	int ret;
+
+	if (!__streams)
+		return -EINVAL;
+
+	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
+	if (ret == -ENOIOCTLCMD) {
+		*__streams = 1ULL;
+		return 0;
+	}
+	if (ret)
+		return ret;
+
+	for (unsigned int i = 0; i < desc.num_entries; i++) {
+		if (streams & BIT_ULL(desc.entry[i].stream))
+			return -EINVAL;
+
+		streams |= BIT_ULL(desc.entry[i].stream);
+	}
+
+	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
+		*__streams, streams);
+	if (*__streams & ~streams)
+		return -EINVAL;
+
+	*__streams = streams;
+
+	return 0;
+}
+
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad, u64 *__sink_streams)
+{
+	u64 sink_streams = 1U;
+	struct media_pad *src_pad;
+	u64 src_streams;
+	struct v4l2_subdev_state *state;
+	struct media_pad *sink_pad = vdev->entity.pads;
+	struct v4l2_subdev *sd = NULL;
+	bool streaming = true;
+	struct media_pad *tmp_pad;
+	u64 tmp_streams;
+	int ret;
+
+	if (!__sink_pad)
+		__sink_pad = &tmp_pad;
+	if (!__sink_streams)
+		__sink_streams = &tmp_streams;
+	*__sink_pad = NULL;
+	*__sink_streams = 0;
+
+	do {
+		src_pad = media_pad_remote_pad_unique(sink_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd ? sd->dev : vdev->dev_parent,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		sd = media_entity_to_v4l2_subdev(src_pad->entity);
+		if (!sd) {
+			dev_dbg(sd->dev,
+				"media entity %s is not a V4L2 sub-device\n",
+				src_pad->entity->name);
+			return -ENXIO;
+		}
+
+		/* Source streams match sink. */
+		src_streams = sink_streams;
+
+		state = v4l2_subdev_lock_and_get_active_state(sd);
+		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
+						 src_streams, &sink_pad,
+						 &sink_streams);
+		v4l2_subdev_unlock_state(state);
+		if (ret)
+			return ret;
+	} while (sink_pad);
+
+	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
+	if (ret)
+		return ret;
+
+	sd = media_entity_to_v4l2_subdev(src_pad->entity);
+
+	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
+		src_pad->index, src_streams);
+
+	for (unsigned int i = __ffs(src_streams); src_streams;
+	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
+		sink_pad = media_pad_remote_pad_unique(src_pad);
+		if (IS_ERR(src_pad)) {
+			dev_dbg(sd->dev,
+				"no unique remote pad found from %s:%u\n",
+				sink_pad->entity->name, sink_pad->index);
+			return PTR_ERR(src_pad);
+		}
+
+		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
+					       __sink_streams);
+		if (ret == 2)
+			continue;
+		if (ret < 0)
+			return ret;
+		if (!ret)
+			streaming = false;
+	}
+
+	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
+		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
+		(*__sink_pad)->index, *__sink_streams);
+
+	return streaming;
+}
+EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
index 1837c9fd78cf..e72c0f62fa34 100644
--- a/include/media/v4l2-mc.h
+++ b/include/media/v4l2-mc.h
@@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
 int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
 			      unsigned int notification);
 
+/**
+ * v4l2_mc_pipeline_enabled - Tell when to start streaming
+ * @vdev: The video device
+ * @func: Caller-provided function to tell a video device's streaming state
+ * @__sink_pad: sink pad at the root of the local pipeline
+ * @__sink_streams: streams to start
+ *
+ * Use to tell whether streaming should start on a video node. @func returns
+ * true if streaming has been started on a given video node. @__sink_pad and
+ * @__sink_streams are filled with pad and streams on the sub-device closest to
+ * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
+ * v4l2_subdev_disable_streams().
+ *
+ * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
+ * a to-do list):
+ * * only unbranched streams can be supported albeit adding support for
+ *   downstream branches would be fairly trivial,
+ * * streams within a single source sub-device are considered to start at the
+ *   same time, more control could be added in two ways: 1) for sources to
+ *   determine stream starting, a control could be added to UAPI and 2) sources
+ *   could tell which streams start at the same time using a sub-device
+ *   operation,
+ * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
+ *   be implemented by letting the caller to provide a function to determine
+ *   which streams are of interest and
+ * * routes leading to nowhere are ignored, on some hardware this is a problem,
+ *   but this can also be rather trivially addressed.
+ *
+ * Return:
+ * * 0: Success, but don't start streaming yet
+ * * 1: Success, now it's time to start streaming
+ * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
+ * * -ENOTUNIQ: No unique remote pad
+ * * -ENOLINK: No remote pad found
+ * * -ENOENT: Enabled upstream route not found
+ * * -EMLINK: No unique downstream route found
+ * * -EINVAL: Stream could not be followed to source or was not produced by
+ *            the source
+ */
+int v4l2_mc_pipeline_enabled(struct video_device *vdev,
+			     bool (*func)(struct video_device *vdev),
+			     struct media_pad **__sink_pad,
+			     u64 *__sink_streams);
+
 #else /* CONFIG_MEDIA_CONTROLLER */
 
 static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
-- 
2.39.5

[PATCH 13/13] media: ipu6: isys: Rework stream starting and stopping

[Sakari Ailus]

Rework starting and stopping streams, based on determining which streams
are available on the source. Rely on v4l2_mc_pipeline_enabled() to obtain
information on when to start streaming.

This effectively both complicates and simplifies driver implementation.
There's some cleanup to be done in the driver after this patch, as
starting and stopping streaming is currently a quite complicated process.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 38 ++++++++++---
 .../media/pci/intel/ipu6/ipu6-isys-queue.c    | 48 ++++++++++-------
 .../media/pci/intel/ipu6/ipu6-isys-video.c    | 53 ++++++++++++-------
 .../media/pci/intel/ipu6/ipu6-isys-video.h    |  9 ++--
 4 files changed, 99 insertions(+), 49 deletions(-)

diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
index 3b837e9ccffe..8d57cdb6aeb0 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
@@ -346,21 +346,31 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	struct ipu6_isys_subdev *asd = to_ipu6_isys_subdev(sd);
 	struct ipu6_isys_csi2 *csi2 = to_ipu6_isys_csi2(asd);
 	struct ipu6_isys_csi2_timing timing = { };
+	struct ipu6_isys_video *iv;
+	struct ipu6_isys_stream *stream;
 	struct v4l2_subdev *remote_sd;
-	struct media_pad *remote_pad;
-	u64 sink_streams, already_enabled;
+	struct media_pad *remote_pad, *vdev_pad;
+	u64 sink_streams;
 	int ret;
 
-	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
-	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
+	vdev_pad = media_pad_remote_pad_first(&sd->entity.pads[pad]);
+	iv = container_of_const(media_entity_to_video_device(vdev_pad->entity),
+				struct ipu6_isys_video, vdev);
+	stream = iv->stream;
 
 	sink_streams =
 		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
 						&streams_mask);
 
-	already_enabled = v4l2_subdev_state_streams_enabled(sd, state,
-							    CSI2_PAD_SINK);
-	if (!already_enabled) {
+	if ((sink_streams | stream->streams_enabled) != stream->streams) {
+		stream->streams_enabled |= sink_streams;
+		return 0;
+	}
+
+	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
+	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
+
+	if (!stream->streaming) {
 		ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
 		if (ret)
 			return ret;
@@ -371,12 +381,15 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
 	}
 
 	ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index,
+					 stream->streams_enabled |
 					 sink_streams);
 	if (ret) {
 		ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
 		return ret;
 	}
 
+	stream->streams_enabled |= sink_streams;
+
 	return 0;
 }
 
@@ -402,6 +415,17 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
 
 	v4l2_subdev_disable_streams(remote_sd, remote_pad->index, sink_streams);
 
+	struct media_pad *vdev_pad;
+	struct ipu6_isys_video *iv;
+	struct ipu6_isys_stream *stream;
+
+	vdev_pad = media_pad_remote_pad_first(&sd->entity.pads[pad]);
+	iv = container_of_const(media_entity_to_video_device(vdev_pad->entity),
+				struct ipu6_isys_video, vdev);
+	stream = iv->stream;
+
+	stream->streams_enabled &= ~sink_streams;
+
 	return 0;
 }
 
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-queue.c b/drivers/media/pci/intel/ipu6/ipu6-isys-queue.c
index aa2cf7287477..89a49395ecc4 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-queue.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-queue.c
@@ -12,6 +12,7 @@
 #include <linux/types.h>
 
 #include <media/media-entity.h>
+#include <media/v4l2-mc.h>
 #include <media/v4l2-subdev.h>
 #include <media/videobuf2-dma-sg.h>
 #include <media/videobuf2-v4l2.h>
@@ -300,7 +301,7 @@ static int ipu6_isys_stream_start(struct ipu6_isys_video *av,
 	if (ret)
 		goto out_requeue;
 
-	stream->streaming = 1;
+	stream->streaming = true;
 
 	bl = &__bl;
 
@@ -380,7 +381,7 @@ static void buf_queue(struct vb2_buffer *vb)
 
 	mutex_lock(&stream->mutex);
 
-	if (stream->nr_streaming != stream->nr_queues) {
+	if (!stream->streaming) {
 		dev_dbg(dev, "not streaming yet, adding to incoming\n");
 		goto out;
 	}
@@ -410,7 +411,6 @@ static void buf_queue(struct vb2_buffer *vb)
 		ret = ipu6_isys_stream_start(av, &bl, true);
 		if (ret)
 			dev_err(dev, "stream start failed.\n");
-		goto out;
 	}
 
 	/*
@@ -536,6 +536,14 @@ static void ipu6_isys_stream_cleanup(struct ipu6_isys_video *av)
 	av->stream = NULL;
 }
 
+static bool ipu6_isys_video_streaming(struct video_device *vdev)
+{
+	struct ipu6_isys_video *iv =
+		container_of_const(vdev, struct ipu6_isys_video, vdev);
+
+	return iv->streaming;
+}
+
 static int start_streaming(struct vb2_queue *q, unsigned int count)
 {
 	struct ipu6_isys_queue *aq = vb2_queue_to_isys_queue(q);
@@ -546,13 +554,13 @@ static int start_streaming(struct vb2_queue *q, unsigned int count)
 	struct ipu6_isys_buffer_list __bl, *bl = NULL;
 	struct ipu6_isys_stream *stream;
 	struct media_entity *source_entity = NULL;
-	int nr_queues, ret;
+	int ret;
 
 	dev_dbg(dev, "stream: %s: width %u, height %u, css pixelformat %u\n",
 		av->vdev.name, ipu6_isys_get_frame_width(av),
 		ipu6_isys_get_frame_height(av), pfmt->css_pixelformat);
 
-	ret = ipu6_isys_setup_video(av, &source_entity, &nr_queues);
+	ret = ipu6_isys_setup_video(av, &source_entity);
 	if (ret < 0) {
 		dev_dbg(dev, "failed to setup video\n");
 		goto out_return_buffers;
@@ -572,23 +580,24 @@ static int start_streaming(struct vb2_queue *q, unsigned int count)
 
 	stream = av->stream;
 	mutex_lock(&stream->mutex);
-	if (!stream->nr_streaming) {
-		ret = ipu6_isys_video_prepare_stream(av, source_entity,
-						     nr_queues);
-		if (ret)
-			goto out_fw_close;
-	}
 
-	stream->nr_streaming++;
-	dev_dbg(dev, "queue %u of %u\n", stream->nr_streaming,
-		stream->nr_queues);
+	av->streaming = true;
+
+	ret = ipu6_isys_video_prepare_stream(av, source_entity);
+	if (ret)
+		goto out_fw_close;
 
 	list_add(&aq->node, &stream->queues);
 	ipu6_isys_configure_stream_watermark(av, true);
 	ipu6_isys_update_stream_watermark(av, true);
 
-	if (stream->nr_streaming != stream->nr_queues)
-		goto out;
+	if (!stream->streaming) {
+		ret = v4l2_mc_pipeline_enabled(&av->vdev,
+					       ipu6_isys_video_streaming,
+					       NULL, &stream->streams);
+		if (ret != 1)
+			goto out;
+	}
 
 	bl = &__bl;
 	ret = buffer_list_get(stream, bl);
@@ -609,7 +618,7 @@ static int start_streaming(struct vb2_queue *q, unsigned int count)
 out_stream_start:
 	ipu6_isys_update_stream_watermark(av, false);
 	list_del(&aq->node);
-	stream->nr_streaming--;
+	av->streaming = false;
 
 out_fw_close:
 	mutex_unlock(&stream->mutex);
@@ -635,11 +644,12 @@ static void stop_streaming(struct vb2_queue *q)
 	ipu6_isys_update_stream_watermark(av, false);
 
 	mutex_lock(&av->isys->stream_mutex);
-	if (stream->nr_streaming == stream->nr_queues && stream->streaming)
+	if (stream->streaming)
 		ipu6_isys_video_set_streaming(av, 0, NULL);
+
 	mutex_unlock(&av->isys->stream_mutex);
 
-	stream->nr_streaming--;
+	av->streaming = false;
 	list_del(&aq->node);
 	stream->streaming = 0;
 	mutex_unlock(&stream->mutex);
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
index 54006b5e2ccd..db00e4f3afc7 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
@@ -710,24 +710,16 @@ static void close_streaming_firmware(struct ipu6_isys_video *av)
 }
 
 int ipu6_isys_video_prepare_stream(struct ipu6_isys_video *av,
-				   struct media_entity *source_entity,
-				   int nr_queues)
+				   struct media_entity *source_entity)
 {
 	struct ipu6_isys_stream *stream = av->stream;
 	struct ipu6_isys_csi2 *csi2;
 
-	if (WARN_ON(stream->nr_streaming))
-		return -EINVAL;
-
-	stream->nr_queues = nr_queues;
 	atomic_set(&stream->sequence, 0);
 
 	stream->seq_index = 0;
 	memset(stream->seq, 0, sizeof(stream->seq));
 
-	if (WARN_ON(!list_empty(&stream->queues)))
-		return -EINVAL;
-
 	stream->stream_source = stream->asd->source;
 	csi2 = ipu6_isys_subdev_to_csi2(stream->asd);
 	csi2->receiver_errors = 0;
@@ -1020,8 +1012,24 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 
 		/* stop sub-device which connects with video */
 		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			u64 sink_stream = 1U, src_stream;
+
 			if (!media_pad_pipeline(&sd->entity.pads[i]))
 				continue;
+
+			struct v4l2_subdev_state *state =
+				v4l2_subdev_get_unlocked_active_state(sd);
+
+			src_stream =
+				v4l2_subdev_state_xlate_streams(state, i,
+								CSI2_PAD_SINK,
+								&sink_stream);
+
+			v4l2_subdev_unlock_state(state);
+
+			if (!(src_stream & stream->streams))
+				continue;
+
 			ret = v4l2_subdev_disable_streams(sd, i, 1U);
 			if (ret) {
 				dev_err(dev, "stream off %s failed with %d\n",
@@ -1039,8 +1047,24 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 
 		/* start sub-device which connects with video */
 		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
+			u64 sink_stream = 1U, src_stream;
+
 			if (!media_pad_pipeline(&sd->entity.pads[i]))
 				continue;
+
+			struct v4l2_subdev_state *state =
+				v4l2_subdev_get_unlocked_active_state(sd);
+
+			src_stream =
+				v4l2_subdev_state_xlate_streams(state, i,
+								CSI2_PAD_SINK,
+								&sink_stream);
+
+			v4l2_subdev_unlock_state(state);
+
+			if (!(src_stream & stream->streams))
+				continue;
+
 			ret = v4l2_subdev_enable_streams(sd, i, 1U);
 			if (ret) {
 				dev_err(dev, "stream on %s failed with %d\n",
@@ -1050,8 +1074,6 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 		}
 	}
 
-	av->streaming = state;
-
 	return 0;
 
 out_media_entity_stop_streaming_firmware:
@@ -1178,7 +1200,7 @@ void ipu6_isys_fw_close(struct ipu6_isys *isys)
 }
 
 int ipu6_isys_setup_video(struct ipu6_isys_video *av,
-			  struct media_entity **source_entity, int *nr_queues)
+			  struct media_entity **source_entity)
 {
 	const struct ipu6_isys_pixelformat *pfmt =
 		ipu6_isys_get_isys_format(ipu6_isys_get_format(av), 0);
@@ -1193,8 +1215,6 @@ int ipu6_isys_setup_video(struct ipu6_isys_video *av,
 	struct media_pad *source_pad, *remote_pad;
 	int ret = -EINVAL;
 
-	*nr_queues = 0;
-
 	remote_pad = media_pad_remote_pad_unique(&av->pad);
 	if (IS_ERR(remote_pad)) {
 		dev_dbg(dev, "failed to get remote pad\n");
@@ -1213,12 +1233,9 @@ int ipu6_isys_setup_video(struct ipu6_isys_video *av,
 
 	/* Find the root */
 	state = v4l2_subdev_lock_and_get_active_state(remote_sd);
-	for_each_active_route(&state->routing, r) {
-		(*nr_queues)++;
-
+	for_each_active_route(&state->routing, r)
 		if (r->source_pad == remote_pad->index)
 			route = r;
-	}
 
 	if (!route) {
 		v4l2_subdev_unlock_state(state);
diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.h b/drivers/media/pci/intel/ipu6/ipu6-isys-video.h
index 1dd36f2a077e..6f95544794cf 100644
--- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.h
+++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.h
@@ -44,6 +44,8 @@ struct sequence_info {
 struct ipu6_isys_stream {
 	struct mutex mutex;
 	struct media_entity *source_entity;
+	u64 streams;
+	u64 streams_enabled;
 	atomic_t sequence;
 	unsigned int seq_index;
 	struct sequence_info seq[IPU6_ISYS_MAX_PARALLEL_SOF];
@@ -52,8 +54,6 @@ struct ipu6_isys_stream {
 	unsigned int nr_output_pins;
 	struct ipu6_isys_subdev *asd;
 
-	int nr_queues;	/* Number of capture queues */
-	int nr_streaming;
 	int streaming;	/* Has streaming been really started? */
 	struct list_head queues;
 	struct completion stream_open_completion;
@@ -106,14 +106,13 @@ extern const struct ipu6_isys_pixelformat ipu6_isys_pfmts_packed[];
 const struct ipu6_isys_pixelformat *
 ipu6_isys_get_isys_format(u32 pixelformat, u32 code);
 int ipu6_isys_video_prepare_stream(struct ipu6_isys_video *av,
-				   struct media_entity *source_entity,
-				   int nr_queues);
+				   struct media_entity *source_entity);
 int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
 				  struct ipu6_isys_buffer_list *bl);
 int ipu6_isys_fw_open(struct ipu6_isys *isys);
 void ipu6_isys_fw_close(struct ipu6_isys *isys);
 int ipu6_isys_setup_video(struct ipu6_isys_video *av,
-			  struct media_entity **source_entity, int *nr_queues);
+			  struct media_entity **source_entity);
 int ipu6_isys_video_init(struct ipu6_isys_video *av);
 void ipu6_isys_video_cleanup(struct ipu6_isys_video *av);
 void ipu6_isys_put_stream(struct ipu6_isys_stream *stream);
-- 
2.39.5

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[kernel test robot]

Hi Sakari,

kernel test robot noticed the following build warnings:

[auto build test WARNING on linuxtv-media-pending/master]
[also build test WARNING on linus/master media-tree/master v6.16-rc2 next-20250618]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847
base:   https://git.linuxtv.org/media-ci/media-pending.git master
patch link:    https://lore.kernel.org/r/20250619081546.1582969-13-sakari.ailus%40linux.intel.com
patch subject: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
config: x86_64-buildonly-randconfig-003-20250619 (https://download.01.org/0day-ci/archive/20250619/202506191934.t9FEXlhl-lkp@intel.com/config)
compiler: clang version 20.1.2 (https://github.com/llvm/llvm-project 58df0ef89dd64126512e4ee27b4ac3fd8ddf6247)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250619/202506191934.t9FEXlhl-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202506191934.t9FEXlhl-lkp@intel.com/

All warnings (new ones prefixed by >>):

>> drivers/media/v4l2-core/v4l2-mc.c:622:6: warning: variable 'src_streams' set but not used [-Wunused-but-set-variable]
     622 |         u64 src_streams = 0, sink_streams = 0;
         |             ^
>> drivers/media/v4l2-core/v4l2-mc.c:732:44: warning: variable 'source_stream' is uninitialized when used here [-Wuninitialized]
     732 |         return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
         |                                                   ^~~~~~~~~~~~~
   drivers/media/v4l2-core/v4l2-mc.c:671:19: note: initialize the variable 'source_stream' to silence this warning
     671 |         u32 source_stream;
         |                          ^
         |                           = 0
   2 warnings generated.


vim +/src_streams +622 drivers/media/v4l2-core/v4l2-mc.c

   615	
   616	static int
   617	__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
   618				   struct media_pad *src_pad, u64 __src_streams,
   619				   struct media_pad **__sink_pad, u64 *__sink_streams)
   620	{
   621		struct v4l2_subdev_route *route;
 > 622		u64 src_streams = 0, sink_streams = 0;
   623		bool has_sink_pad = false;
   624		unsigned int sink_pad;
   625	
   626		dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
   627			state->sd->entity.name, src_pad->index, __src_streams);
   628		for_each_active_route(&state->routing, route) {
   629			dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
   630				state->sd->entity.name,
   631				route->sink_pad, route->sink_stream, route->source_pad,
   632				route->source_stream, route->flags);
   633			if (route->source_pad != src_pad->index)
   634				continue;
   635	
   636			if (!(BIT_ULL(route->source_stream) & __src_streams))
   637				continue;
   638	
   639			if (!has_sink_pad) {
   640				has_sink_pad = true;
   641				sink_pad = route->sink_pad;
   642			}
   643	
   644			if (route->sink_pad != sink_pad) {
   645				dev_dbg(state->sd->dev,
   646					"sink pads (%u vs. %u) differ\n",
   647					route->sink_pad, sink_pad);
   648				return -EMLINK;
   649			}
   650	
   651			sink_streams |= BIT_ULL(route->sink_stream);
   652			src_streams |= BIT_ULL(route->source_stream);
   653		}
   654	
   655		*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
   656		*__sink_streams = sink_streams;
   657	
   658		return 0;
   659	}
   660	
   661	static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
   662					    unsigned int sink_stream,
   663					    bool (*func)(struct video_device *vdev),
   664					    struct media_pad **__sink_pad,
   665					    u64 *__sink_streams)
   666	{
   667		struct v4l2_subdev_state *state;
   668		struct v4l2_subdev_route *route;
   669		struct v4l2_subdev *sd;
   670		struct media_pad *source_pad, *tmp_pad;
   671		u32 source_stream;
   672	
   673		if (!is_media_entity_v4l2_subdev(sink_pad->entity))
   674			return -ENXIO;
   675	
   676		sd = media_entity_to_v4l2_subdev(sink_pad->entity);
   677		dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
   678			sd->entity.name);
   679	
   680		state = v4l2_subdev_lock_and_get_active_state(sd);
   681		route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
   682					       sink_stream, true, 0);
   683		if (IS_ERR(route)) {
   684			v4l2_subdev_unlock_state(state);
   685			dev_dbg(sd->dev,
   686				"path_enabled: can't find opposite route for %s:%u/%u",
   687				sd->entity.name, sink_pad->index, sink_stream);
   688			return 2;
   689		}
   690	
   691		source_pad = &sd->entity.pads[route->source_pad];
   692		v4l2_subdev_unlock_state(state);
   693	
   694		tmp_pad = sink_pad;
   695		sink_pad = media_pad_remote_pad_unique(source_pad);
   696		if (IS_ERR(sink_pad)) {
   697			dev_dbg(sd->dev,
   698				"path_enabled: can't find remote source for %s:%u\n",
   699				source_pad->entity->name, source_pad->index);
   700			return PTR_ERR(sink_pad);
   701		}
   702	
   703		if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
   704			struct video_device *vdev;
   705	
   706			vdev = media_entity_to_video_device(sink_pad->entity);
   707			if (!vdev)
   708				return -ENXIO;
   709	
   710			dev_dbg(vdev->dev_parent,
   711				"path_enabled: found video device %s\n",
   712				vdev->name);
   713	
   714			if (!*__sink_pad) {
   715				*__sink_pad = tmp_pad;
   716				dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
   717					tmp_pad->index, sink_stream);
   718			} else if (tmp_pad != *__sink_pad) {
   719				dev_dbg(sd->dev,
   720					"path_enabled: pads %s/%u and %s/%u differ\n",
   721					tmp_pad->entity->name, tmp_pad->index,
   722					(*__sink_pad)->entity->name,
   723					(*__sink_pad)->index);
   724				return -EXDEV;
   725			}
   726	
   727			*__sink_streams |= BIT_ULL(sink_stream);
   728	
   729			return func(vdev);
   730		}
   731	
 > 732		return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
   733						__sink_pad, __sink_streams);
   734	}
   735	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

[kernel test robot]

Hi Sakari,

kernel test robot noticed the following build warnings:

[auto build test WARNING on linuxtv-media-pending/master]
[also build test WARNING on linus/master v6.16-rc2 next-20250618]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847
base:   https://git.linuxtv.org/media-ci/media-pending.git master
patch link:    https://lore.kernel.org/r/20250619081546.1582969-4-sakari.ailus%40linux.intel.com
patch subject: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad
config: i386-buildonly-randconfig-005-20250619 (https://download.01.org/0day-ci/archive/20250619/202506192043.sWSo3Ycl-lkp@intel.com/config)
compiler: clang version 20.1.2 (https://github.com/llvm/llvm-project 58df0ef89dd64126512e4ee27b4ac3fd8ddf6247)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250619/202506192043.sWSo3Ycl-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202506192043.sWSo3Ycl-lkp@intel.com/

All warnings (new ones prefixed by >>):

>> drivers/media/pci/intel/ipu6/ipu6-isys-video.c:996:6: warning: variable 'stream_mask' set but not used [-Wunused-but-set-variable]
     996 |         u64 stream_mask = 0;
         |             ^
   1 warning generated.


vim +/stream_mask +996 drivers/media/pci/intel/ipu6/ipu6-isys-video.c

3c1dfb5a69cf83 Bingbu Cao   2024-01-31   983  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   984  int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   985  				  struct ipu6_isys_buffer_list *bl)
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   986  {
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   987  	struct v4l2_subdev_krouting *routing;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   988  	struct ipu6_isys_stream *stream = av->stream;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   989  	struct v4l2_subdev_state *subdev_state;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   990  	struct device *dev = &av->isys->adev->auxdev.dev;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   991  	struct v4l2_subdev *sd;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   992  	struct media_pad *r_pad;
e232e72fbaa788 Sakari Ailus 2025-06-19   993  	unsigned int i;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   994  	u32 sink_pad, sink_stream;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   995  	u64 r_stream;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  @996  	u64 stream_mask = 0;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   997  	int ret = 0;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   998  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31   999  	dev_dbg(dev, "set stream: %d\n", state);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1000  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1001  	if (WARN(!stream->source_entity, "No source entity for stream\n"))
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1002  		return -ENODEV;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1003  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1004  	sd = &stream->asd->sd;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1005  	r_pad = media_pad_remote_pad_first(&av->pad);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1006  	r_stream = ipu6_isys_get_src_stream_by_src_pad(sd, r_pad->index);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1007  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1008  	subdev_state = v4l2_subdev_lock_and_get_active_state(sd);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1009  	routing = &subdev_state->routing;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1010  	ret = v4l2_subdev_routing_find_opposite_end(routing, r_pad->index,
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1011  						    r_stream, &sink_pad,
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1012  						    &sink_stream);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1013  	v4l2_subdev_unlock_state(subdev_state);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1014  	if (ret)
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1015  		return ret;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1016  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1017  	stream_mask = get_stream_mask_by_pipeline(av);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1018  	if (!state) {
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1019  		stop_streaming_firmware(av);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1020  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1021  		/* stop sub-device which connects with video */
e232e72fbaa788 Sakari Ailus 2025-06-19  1022  		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
e232e72fbaa788 Sakari Ailus 2025-06-19  1023  			if (!media_pad_pipeline(&sd->entity.pads[i]))
e232e72fbaa788 Sakari Ailus 2025-06-19  1024  				continue;
e232e72fbaa788 Sakari Ailus 2025-06-19  1025  			ret = v4l2_subdev_disable_streams(sd, i, 1U);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1026  			if (ret) {
e232e72fbaa788 Sakari Ailus 2025-06-19  1027  				dev_err(dev, "stream off %s failed with %d\n",
e232e72fbaa788 Sakari Ailus 2025-06-19  1028  					sd->name, ret);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1029  				return ret;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1030  			}
e232e72fbaa788 Sakari Ailus 2025-06-19  1031  		}
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1032  		close_streaming_firmware(av);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1033  	} else {
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1034  		ret = start_stream_firmware(av, bl);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1035  		if (ret) {
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1036  			dev_err(dev, "start stream of firmware failed\n");
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1037  			return ret;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1038  		}
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1039  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1040  		/* start sub-device which connects with video */
e232e72fbaa788 Sakari Ailus 2025-06-19  1041  		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
e232e72fbaa788 Sakari Ailus 2025-06-19  1042  			if (!media_pad_pipeline(&sd->entity.pads[i]))
e232e72fbaa788 Sakari Ailus 2025-06-19  1043  				continue;
e232e72fbaa788 Sakari Ailus 2025-06-19  1044  			ret = v4l2_subdev_enable_streams(sd, i, 1U);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1045  			if (ret) {
e232e72fbaa788 Sakari Ailus 2025-06-19  1046  				dev_err(dev, "stream on %s failed with %d\n",
e232e72fbaa788 Sakari Ailus 2025-06-19  1047  					sd->name, ret);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1048  				goto out_media_entity_stop_streaming_firmware;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1049  			}
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1050  		}
e232e72fbaa788 Sakari Ailus 2025-06-19  1051  	}
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1052  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1053  	av->streaming = state;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1054  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1055  	return 0;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1056  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1057  out_media_entity_stop_streaming_firmware:
e232e72fbaa788 Sakari Ailus 2025-06-19  1058  	while (i-- > CSI2_PAD_SRC) {
e232e72fbaa788 Sakari Ailus 2025-06-19  1059  		int ret2;
e232e72fbaa788 Sakari Ailus 2025-06-19  1060  
e232e72fbaa788 Sakari Ailus 2025-06-19  1061  		if (!media_pad_pipeline(&sd->entity.pads[i]))
e232e72fbaa788 Sakari Ailus 2025-06-19  1062  			continue;
e232e72fbaa788 Sakari Ailus 2025-06-19  1063  		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
e232e72fbaa788 Sakari Ailus 2025-06-19  1064  		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
e232e72fbaa788 Sakari Ailus 2025-06-19  1065  	}
e232e72fbaa788 Sakari Ailus 2025-06-19  1066  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1067  	stop_streaming_firmware(av);
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1068  
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1069  	return ret;
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1070  }
3c1dfb5a69cf83 Bingbu Cao   2024-01-31  1071  

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Thu, Jun 19, 2025 at 11:15:36AM +0300, Sakari Ailus wrote:
> On UAPI, streaming is started a video device at a time. The IPU6 ISYS
> driver only starts streaming on the source sub-device when all relevant
> video devices have been set streaming. This also needs to be reflected in
> the sub-device pads, hence set them all streaming, one at a time.

Why ? What's wrong with enabling/disabling streams on each pad as the
corresponding video device is started/stopped ?

> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  .../media/pci/intel/ipu6/ipu6-isys-video.c    | 43 ++++++++++++-------
>  1 file changed, 28 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> index 24a2ef93474c..54006b5e2ccd 100644
> --- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> @@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
>  	struct device *dev = &av->isys->adev->auxdev.dev;
>  	struct v4l2_subdev *sd;
>  	struct media_pad *r_pad;
> +	unsigned int i;
>  	u32 sink_pad, sink_stream;
>  	u64 r_stream;
>  	u64 stream_mask = 0;
> @@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
>  		stop_streaming_firmware(av);
>  
>  		/* stop sub-device which connects with video */
> -		dev_dbg(dev, "stream off entity %s pad:%d mask:0x%llx\n",
> -			sd->name, r_pad->index, stream_mask);
> -		ret = v4l2_subdev_disable_streams(sd, r_pad->index,
> -						  stream_mask);
> -		if (ret) {
> -			dev_err(dev, "stream off %s failed with %d\n", sd->name,
> -				ret);
> -			return ret;
> +		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
> +			if (!media_pad_pipeline(&sd->entity.pads[i]))
> +				continue;
> +			ret = v4l2_subdev_disable_streams(sd, i, 1U);
> +			if (ret) {
> +				dev_err(dev, "stream off %s failed with %d\n",
> +					sd->name, ret);
> +				return ret;
> +			}
>  		}
>  		close_streaming_firmware(av);
>  	} else {
> @@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
>  		}
>  
>  		/* start sub-device which connects with video */
> -		dev_dbg(dev, "stream on %s pad %d mask 0x%llx\n", sd->name,
> -			r_pad->index, stream_mask);
> -		ret = v4l2_subdev_enable_streams(sd, r_pad->index, stream_mask);
> -		if (ret) {
> -			dev_err(dev, "stream on %s failed with %d\n", sd->name,
> -				ret);
> -			goto out_media_entity_stop_streaming_firmware;
> +		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
> +			if (!media_pad_pipeline(&sd->entity.pads[i]))
> +				continue;
> +			ret = v4l2_subdev_enable_streams(sd, i, 1U);
> +			if (ret) {
> +				dev_err(dev, "stream on %s failed with %d\n",
> +					sd->name, ret);
> +				goto out_media_entity_stop_streaming_firmware;
> +			}
>  		}
>  	}
>  
> @@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
>  	return 0;
>  
>  out_media_entity_stop_streaming_firmware:
> +	while (i-- > CSI2_PAD_SRC) {
> +		int ret2;
> +
> +		if (!media_pad_pipeline(&sd->entity.pads[i]))
> +			continue;
> +		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
> +		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
> +	}
> +
>  	stop_streaming_firmware(av);
>  
>  	return ret;

-- 
Regards,

Laurent Pinchart

Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

[Sakari Ailus]

Hi Laurent,

On Thu, Jun 19, 2025 at 03:48:55PM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.
> 
> On Thu, Jun 19, 2025 at 11:15:36AM +0300, Sakari Ailus wrote:
> > On UAPI, streaming is started a video device at a time. The IPU6 ISYS
> > driver only starts streaming on the source sub-device when all relevant
> > video devices have been set streaming. This also needs to be reflected in
> > the sub-device pads, hence set them all streaming, one at a time.
> 
> Why ? What's wrong with enabling/disabling streams on each pad as the
> corresponding video device is started/stopped ?

Streaming is started once all the related video nodes are set streaming,
otherwise there's a chance of losing data.

I've squashed this with the last patch after posting the set.

> 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  .../media/pci/intel/ipu6/ipu6-isys-video.c    | 43 ++++++++++++-------
> >  1 file changed, 28 insertions(+), 15 deletions(-)
> > 
> > diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> > index 24a2ef93474c..54006b5e2ccd 100644
> > --- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> > +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> > @@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> >  	struct device *dev = &av->isys->adev->auxdev.dev;
> >  	struct v4l2_subdev *sd;
> >  	struct media_pad *r_pad;
> > +	unsigned int i;
> >  	u32 sink_pad, sink_stream;
> >  	u64 r_stream;
> >  	u64 stream_mask = 0;
> > @@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> >  		stop_streaming_firmware(av);
> >  
> >  		/* stop sub-device which connects with video */
> > -		dev_dbg(dev, "stream off entity %s pad:%d mask:0x%llx\n",
> > -			sd->name, r_pad->index, stream_mask);
> > -		ret = v4l2_subdev_disable_streams(sd, r_pad->index,
> > -						  stream_mask);
> > -		if (ret) {
> > -			dev_err(dev, "stream off %s failed with %d\n", sd->name,
> > -				ret);
> > -			return ret;
> > +		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
> > +			if (!media_pad_pipeline(&sd->entity.pads[i]))
> > +				continue;
> > +			ret = v4l2_subdev_disable_streams(sd, i, 1U);
> > +			if (ret) {
> > +				dev_err(dev, "stream off %s failed with %d\n",
> > +					sd->name, ret);
> > +				return ret;
> > +			}
> >  		}
> >  		close_streaming_firmware(av);
> >  	} else {
> > @@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> >  		}
> >  
> >  		/* start sub-device which connects with video */
> > -		dev_dbg(dev, "stream on %s pad %d mask 0x%llx\n", sd->name,
> > -			r_pad->index, stream_mask);
> > -		ret = v4l2_subdev_enable_streams(sd, r_pad->index, stream_mask);
> > -		if (ret) {
> > -			dev_err(dev, "stream on %s failed with %d\n", sd->name,
> > -				ret);
> > -			goto out_media_entity_stop_streaming_firmware;
> > +		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
> > +			if (!media_pad_pipeline(&sd->entity.pads[i]))
> > +				continue;
> > +			ret = v4l2_subdev_enable_streams(sd, i, 1U);
> > +			if (ret) {
> > +				dev_err(dev, "stream on %s failed with %d\n",
> > +					sd->name, ret);
> > +				goto out_media_entity_stop_streaming_firmware;
> > +			}
> >  		}
> >  	}
> >  
> > @@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> >  	return 0;
> >  
> >  out_media_entity_stop_streaming_firmware:
> > +	while (i-- > CSI2_PAD_SRC) {
> > +		int ret2;
> > +
> > +		if (!media_pad_pipeline(&sd->entity.pads[i]))
> > +			continue;
> > +		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
> > +		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
> > +	}
> > +
> >  	stop_streaming_firmware(av);
> >  
> >  	return ret;
> 

-- 
Regards,

Sakari Ailus

Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

[Laurent Pinchart]

On Thu, Jun 19, 2025 at 01:10:48PM +0000, Sakari Ailus wrote:
> On Thu, Jun 19, 2025 at 03:48:55PM +0300, Laurent Pinchart wrote:
> > On Thu, Jun 19, 2025 at 11:15:36AM +0300, Sakari Ailus wrote:
> > > On UAPI, streaming is started a video device at a time. The IPU6 ISYS
> > > driver only starts streaming on the source sub-device when all relevant
> > > video devices have been set streaming. This also needs to be reflected in
> > > the sub-device pads, hence set them all streaming, one at a time.
> > 
> > Why ? What's wrong with enabling/disabling streams on each pad as the
> > corresponding video device is started/stopped ?
> 
> Streaming is started once all the related video nodes are set streaming,
> otherwise there's a chance of losing data.

Yes, but that's something that can be handled in the subdev, to only
start the ISYS and its source when all the streams that are part of the
pipeline have been started.

> I've squashed this with the last patch after posting the set.

Maybe it will become clearer when I review the rest of the series.

> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  .../media/pci/intel/ipu6/ipu6-isys-video.c    | 43 ++++++++++++-------
> > >  1 file changed, 28 insertions(+), 15 deletions(-)
> > > 
> > > diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> > > index 24a2ef93474c..54006b5e2ccd 100644
> > > --- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> > > +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> > > @@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> > >  	struct device *dev = &av->isys->adev->auxdev.dev;
> > >  	struct v4l2_subdev *sd;
> > >  	struct media_pad *r_pad;
> > > +	unsigned int i;
> > >  	u32 sink_pad, sink_stream;
> > >  	u64 r_stream;
> > >  	u64 stream_mask = 0;
> > > @@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> > >  		stop_streaming_firmware(av);
> > >  
> > >  		/* stop sub-device which connects with video */
> > > -		dev_dbg(dev, "stream off entity %s pad:%d mask:0x%llx\n",
> > > -			sd->name, r_pad->index, stream_mask);
> > > -		ret = v4l2_subdev_disable_streams(sd, r_pad->index,
> > > -						  stream_mask);
> > > -		if (ret) {
> > > -			dev_err(dev, "stream off %s failed with %d\n", sd->name,
> > > -				ret);
> > > -			return ret;
> > > +		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
> > > +			if (!media_pad_pipeline(&sd->entity.pads[i]))
> > > +				continue;
> > > +			ret = v4l2_subdev_disable_streams(sd, i, 1U);
> > > +			if (ret) {
> > > +				dev_err(dev, "stream off %s failed with %d\n",
> > > +					sd->name, ret);
> > > +				return ret;
> > > +			}
> > >  		}
> > >  		close_streaming_firmware(av);
> > >  	} else {
> > > @@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> > >  		}
> > >  
> > >  		/* start sub-device which connects with video */
> > > -		dev_dbg(dev, "stream on %s pad %d mask 0x%llx\n", sd->name,
> > > -			r_pad->index, stream_mask);
> > > -		ret = v4l2_subdev_enable_streams(sd, r_pad->index, stream_mask);
> > > -		if (ret) {
> > > -			dev_err(dev, "stream on %s failed with %d\n", sd->name,
> > > -				ret);
> > > -			goto out_media_entity_stop_streaming_firmware;
> > > +		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
> > > +			if (!media_pad_pipeline(&sd->entity.pads[i]))
> > > +				continue;
> > > +			ret = v4l2_subdev_enable_streams(sd, i, 1U);
> > > +			if (ret) {
> > > +				dev_err(dev, "stream on %s failed with %d\n",
> > > +					sd->name, ret);
> > > +				goto out_media_entity_stop_streaming_firmware;
> > > +			}
> > >  		}
> > >  	}
> > >  
> > > @@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> > >  	return 0;
> > >  
> > >  out_media_entity_stop_streaming_firmware:
> > > +	while (i-- > CSI2_PAD_SRC) {
> > > +		int ret2;
> > > +
> > > +		if (!media_pad_pipeline(&sd->entity.pads[i]))
> > > +			continue;
> > > +		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
> > > +		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
> > > +	}
> > > +
> > >  	stop_streaming_firmware(av);
> > >  
> > >  	return ret;

-- 
Regards,

Laurent Pinchart

Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Thu, Jun 19, 2025 at 11:15:34AM +0300, Sakari Ailus wrote:
> The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the

s/were/are/

> source pads was used as the sink pad and the source pad was a constant

s/pads was/pad is/
s/pad was/pad is/

Are you sure though ? Unless I misread the code, you replace

pad0 = CSI2_PAD_SRC
pad1 = CSI2_PAD_SINK

with

pad0 = pad
pad1 = CSI2_PAD_SINK

This seems to be a correct fix, but I don't see where "the source pad
was used as the sink pad".

> (rather than the actual source pad). Fix these.
> 
> Fixes: 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
> Cc: stable@vger.kernel.org
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> index da8581a37e22..6030bd23b4b9 100644
> --- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> @@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
>  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
>  	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
>  
> -	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
> -						       CSI2_PAD_SINK,
> -						       &streams_mask);
> +	sink_streams =
> +		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> +						&streams_mask);

This is one of the cases where I'd write

	sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
						       &streams_mask);

even if it goes to 81 columns.

>  
>  	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
>  	if (ret)
> @@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
>  	struct media_pad *remote_pad;
>  	u64 sink_streams;
>  
> -	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
> -						       CSI2_PAD_SINK,
> -						       &streams_mask);
> +	sink_streams =
> +		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> +						&streams_mask);

Same here.

>  
>  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
>  	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);

-- 
Regards,

Laurent Pinchart

Re: [PATCH 02/13] media: ipu6: Set minimum height to 1

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Thu, Jun 19, 2025 at 11:15:35AM +0300, Sakari Ailus wrote:
> For image data generally 2 seems like a minimum height that surely won't
> cause any issues, but some sensors have metadata the height of which is
> just one line. Set the minimum height to 1.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/pci/intel/ipu6/ipu6-isys.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys.h b/drivers/media/pci/intel/ipu6/ipu6-isys.h
> index f488e782c26e..0e2c8b71edfc 100644
> --- a/drivers/media/pci/intel/ipu6/ipu6-isys.h
> +++ b/drivers/media/pci/intel/ipu6/ipu6-isys.h
> @@ -40,7 +40,7 @@ struct ipu6_bus_device;
>  #define IPU6_ISYS_NUM_RECV_QUEUE 1
>  
>  #define IPU6_ISYS_MIN_WIDTH		2U
> -#define IPU6_ISYS_MIN_HEIGHT		2U
> +#define IPU6_ISYS_MIN_HEIGHT		1U
>  #define IPU6_ISYS_MAX_WIDTH		4672U
>  #define IPU6_ISYS_MAX_HEIGHT		3416U
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 04/13] media: v4l2-subdev: Add a helper to figure out the pad streaming state

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Thu, Jun 19, 2025 at 11:15:37AM +0300, Sakari Ailus wrote:
> Add a helper, v4l2_subdev_state_streams_enabled(), to tell which streams
> are enabled on a pad. This is useful to e.g. figure out in a driver
> whether a hardware configuration change is necessary to enable a given
> stream.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 17 +++++++++++++++++
>  include/media/v4l2-subdev.h           |  3 +++
>  2 files changed, 20 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index a3074f469b15..60b8febd3339 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2453,6 +2453,23 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  }
>  EXPORT_SYMBOL_GPL(v4l2_subdev_disable_streams);
>  
> +u64 v4l2_subdev_state_streams_enabled(struct v4l2_subdev *sd,
> +				      struct v4l2_subdev_state *state, u32 pad)

You can make the state const.

> +{
> +	u64 streams_mask = 0;
> +
> +	for (unsigned int i = 0; i < state->stream_configs.num_configs; i++) {
> +		struct v4l2_subdev_stream_config *cfg =

const

> +			&state->stream_configs.configs[i];
> +
> +		if (cfg->pad == pad && cfg->enabled)
> +			streams_mask |= BIT_ULL(cfg->stream);
> +	}
> +
> +	return streams_mask;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_state_streams_enabled);
> +
>  int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable)
>  {
>  	struct v4l2_subdev_state *state;
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 57f2bcb4eb16..deab128a4779 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1731,6 +1731,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  				u64 streams_mask);
>  

Missing documentation.

The function looks fine. It's a bit costly though, so I'll see in
subsequent patches if there could be a better way to handle whatever
issue this solves.

> +u64 v4l2_subdev_state_streams_enabled(struct v4l2_subdev *sd,
> +				      struct v4l2_subdev_state *state, u32 pad);
> +
>  /**
>   * v4l2_subdev_s_stream_helper() - Helper to implement the subdev s_stream
>   *	operation using enable_streams and disable_streams

-- 
Regards,

Laurent Pinchart

Re: [PATCH 03/13] media: ipu6: Enable and disable each stream at CSI-2 subdev source pad

[Sakari Ailus]

On Thu, Jun 19, 2025 at 04:19:00PM +0300, Laurent Pinchart wrote:
> On Thu, Jun 19, 2025 at 01:10:48PM +0000, Sakari Ailus wrote:
> > On Thu, Jun 19, 2025 at 03:48:55PM +0300, Laurent Pinchart wrote:
> > > On Thu, Jun 19, 2025 at 11:15:36AM +0300, Sakari Ailus wrote:
> > > > On UAPI, streaming is started a video device at a time. The IPU6 ISYS
> > > > driver only starts streaming on the source sub-device when all relevant
> > > > video devices have been set streaming. This also needs to be reflected in
> > > > the sub-device pads, hence set them all streaming, one at a time.
> > > 
> > > Why ? What's wrong with enabling/disabling streams on each pad as the
> > > corresponding video device is started/stopped ?
> > 
> > Streaming is started once all the related video nodes are set streaming,
> > otherwise there's a chance of losing data.
> 
> Yes, but that's something that can be handled in the subdev, to only
> start the ISYS and its source when all the streams that are part of the
> pipeline have been started.

I agree, this is done in the CSI-2 receiver driver actually.

There's some work to do in cleaning up the IPU6 driver. The combination of
videobuf2 and streams isn't very driver-friendly currently. Things
generally need to happen in certain order and this may not be always veyr
convenient for drivers.

> 
> > I've squashed this with the last patch after posting the set.
> 
> Maybe it will become clearer when I review the rest of the series.
> 
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > ---
> > > >  .../media/pci/intel/ipu6/ipu6-isys-video.c    | 43 ++++++++++++-------
> > > >  1 file changed, 28 insertions(+), 15 deletions(-)
> > > > 
> > > > diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> > > > index 24a2ef93474c..54006b5e2ccd 100644
> > > > --- a/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> > > > +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-video.c
> > > > @@ -990,6 +990,7 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> > > >  	struct device *dev = &av->isys->adev->auxdev.dev;
> > > >  	struct v4l2_subdev *sd;
> > > >  	struct media_pad *r_pad;
> > > > +	unsigned int i;
> > > >  	u32 sink_pad, sink_stream;
> > > >  	u64 r_stream;
> > > >  	u64 stream_mask = 0;
> > > > @@ -1018,14 +1019,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> > > >  		stop_streaming_firmware(av);
> > > >  
> > > >  		/* stop sub-device which connects with video */
> > > > -		dev_dbg(dev, "stream off entity %s pad:%d mask:0x%llx\n",
> > > > -			sd->name, r_pad->index, stream_mask);
> > > > -		ret = v4l2_subdev_disable_streams(sd, r_pad->index,
> > > > -						  stream_mask);
> > > > -		if (ret) {
> > > > -			dev_err(dev, "stream off %s failed with %d\n", sd->name,
> > > > -				ret);
> > > > -			return ret;
> > > > +		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
> > > > +			if (!media_pad_pipeline(&sd->entity.pads[i]))
> > > > +				continue;
> > > > +			ret = v4l2_subdev_disable_streams(sd, i, 1U);
> > > > +			if (ret) {
> > > > +				dev_err(dev, "stream off %s failed with %d\n",
> > > > +					sd->name, ret);
> > > > +				return ret;
> > > > +			}
> > > >  		}
> > > >  		close_streaming_firmware(av);
> > > >  	} else {
> > > > @@ -1036,13 +1038,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> > > >  		}
> > > >  
> > > >  		/* start sub-device which connects with video */
> > > > -		dev_dbg(dev, "stream on %s pad %d mask 0x%llx\n", sd->name,
> > > > -			r_pad->index, stream_mask);
> > > > -		ret = v4l2_subdev_enable_streams(sd, r_pad->index, stream_mask);
> > > > -		if (ret) {
> > > > -			dev_err(dev, "stream on %s failed with %d\n", sd->name,
> > > > -				ret);
> > > > -			goto out_media_entity_stop_streaming_firmware;
> > > > +		for (i = CSI2_PAD_SRC; i < NR_OF_CSI2_SRC_PADS; i++) {
> > > > +			if (!media_pad_pipeline(&sd->entity.pads[i]))
> > > > +				continue;
> > > > +			ret = v4l2_subdev_enable_streams(sd, i, 1U);
> > > > +			if (ret) {
> > > > +				dev_err(dev, "stream on %s failed with %d\n",
> > > > +					sd->name, ret);
> > > > +				goto out_media_entity_stop_streaming_firmware;
> > > > +			}
> > > >  		}
> > > >  	}
> > > >  
> > > > @@ -1051,6 +1055,15 @@ int ipu6_isys_video_set_streaming(struct ipu6_isys_video *av, int state,
> > > >  	return 0;
> > > >  
> > > >  out_media_entity_stop_streaming_firmware:
> > > > +	while (i-- > CSI2_PAD_SRC) {
> > > > +		int ret2;
> > > > +
> > > > +		if (!media_pad_pipeline(&sd->entity.pads[i]))
> > > > +			continue;
> > > > +		ret2 = v4l2_subdev_disable_streams(sd, i, 1U);
> > > > +		dev_err(dev, "stream off %s failed with %d\n", sd->name, ret2);
> > > > +	}
> > > > +
> > > >  	stop_streaming_firmware(av);
> > > >  
> > > >  	return ret;
> 

-- 
Regards,

Sakari Ailus

Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

[Sakari Ailus]

Hi Laurent,

On Thu, Jun 19, 2025 at 04:27:04PM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.

Thanks for the review!

> 
> On Thu, Jun 19, 2025 at 11:15:34AM +0300, Sakari Ailus wrote:
> > The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the
> 
> s/were/are/
> 
> > source pads was used as the sink pad and the source pad was a constant
> 
> s/pads was/pad is/
> s/pad was/pad is/
> 
> Are you sure though ? Unless I misread the code, you replace
> 
> pad0 = CSI2_PAD_SRC
> pad1 = CSI2_PAD_SINK
> 
> with
> 
> pad0 = pad
> pad1 = CSI2_PAD_SINK
> 
> This seems to be a correct fix, but I don't see where "the source pad
> was used as the sink pad".

Right, I'll reword it for v2.

> 
> > (rather than the actual source pad). Fix these.
> > 
> > Fixes: 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
> >  1 file changed, 6 insertions(+), 6 deletions(-)
> > 
> > diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > index da8581a37e22..6030bd23b4b9 100644
> > --- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > @@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
> >  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
> >  	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
> >  
> > -	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
> > -						       CSI2_PAD_SINK,
> > -						       &streams_mask);
> > +	sink_streams =
> > +		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> > +						&streams_mask);
> 
> This is one of the cases where I'd write
> 
> 	sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> 						       &streams_mask);
> 
> even if it goes to 81 columns.

The limit is still 80, not 81.

Either way, a number of the new functions in the V4L2 subdev KAPI have very
long names, I'll post patches to shorten them later on.

> 
> >  
> >  	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
> >  	if (ret)
> > @@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
> >  	struct media_pad *remote_pad;
> >  	u64 sink_streams;
> >  
> > -	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
> > -						       CSI2_PAD_SINK,
> > -						       &streams_mask);
> > +	sink_streams =
> > +		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> > +						&streams_mask);
> 
> Same here.
> 
> >  
> >  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
> >  	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
> 

-- 
Regards,

Sakari Ailus

Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

[Laurent Pinchart]

On Thu, Jun 19, 2025 at 01:55:08PM +0000, Sakari Ailus wrote:
> On Thu, Jun 19, 2025 at 04:27:04PM +0300, Laurent Pinchart wrote:
> > Hi Sakari,
> > 
> > Thank you for the patch.
> 
> Thanks for the review!
> 
> > On Thu, Jun 19, 2025 at 11:15:34AM +0300, Sakari Ailus wrote:
> > > The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the
> > 
> > s/were/are/
> > 
> > > source pads was used as the sink pad and the source pad was a constant
> > 
> > s/pads was/pad is/
> > s/pad was/pad is/
> > 
> > Are you sure though ? Unless I misread the code, you replace
> > 
> > pad0 = CSI2_PAD_SRC
> > pad1 = CSI2_PAD_SINK
> > 
> > with
> > 
> > pad0 = pad
> > pad1 = CSI2_PAD_SINK
> > 
> > This seems to be a correct fix, but I don't see where "the source pad
> > was used as the sink pad".
> 
> Right, I'll reword it for v2.
> 
> > > (rather than the actual source pad). Fix these.
> > > 
> > > Fixes: 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
> > > Cc: stable@vger.kernel.org
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
> > >  1 file changed, 6 insertions(+), 6 deletions(-)
> > > 
> > > diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > > index da8581a37e22..6030bd23b4b9 100644
> > > --- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > > +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > > @@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
> > >  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
> > >  	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
> > >  
> > > -	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
> > > -						       CSI2_PAD_SINK,
> > > -						       &streams_mask);
> > > +	sink_streams =
> > > +		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> > > +						&streams_mask);
> > 
> > This is one of the cases where I'd write
> > 
> > 	sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> > 						       &streams_mask);
> > 
> > even if it goes to 81 columns.
> 
> The limit is still 80, not 81.

The global limit is 100 for the kernel. There's somehow of a consensus
in the media subsystem to keep it closer to 80, with different people
have different sensitivities. We occasionally go over 80, and that's
usually left as a driver maintainer decision.

> Either way, a number of the new functions in the V4L2 subdev KAPI have very
> long names, I'll post patches to shorten them later on.

As long as the name remains equally readable and understandable, and all
functions use the same conventions, I'm fine with that.

> > >  
> > >  	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
> > >  	if (ret)
> > > @@ -384,9 +384,9 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
> > >  	struct media_pad *remote_pad;
> > >  	u64 sink_streams;
> > >  
> > > -	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
> > > -						       CSI2_PAD_SINK,
> > > -						       &streams_mask);
> > > +	sink_streams =
> > > +		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> > > +						&streams_mask);
> > 
> > Same here.
> > 
> > >  
> > >  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
> > >  	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);

-- 
Regards,

Laurent Pinchart

Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

[Sakari Ailus]

Hi Laurent,

On Thu, Jun 19, 2025 at 05:15:35PM +0300, Laurent Pinchart wrote:
> On Thu, Jun 19, 2025 at 01:55:08PM +0000, Sakari Ailus wrote:
> > On Thu, Jun 19, 2025 at 04:27:04PM +0300, Laurent Pinchart wrote:
> > > Hi Sakari,
> > > 
> > > Thank you for the patch.
> > 
> > Thanks for the review!
> > 
> > > On Thu, Jun 19, 2025 at 11:15:34AM +0300, Sakari Ailus wrote:
> > > > The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the
> > > 
> > > s/were/are/
> > > 
> > > > source pads was used as the sink pad and the source pad was a constant
> > > 
> > > s/pads was/pad is/
> > > s/pad was/pad is/
> > > 
> > > Are you sure though ? Unless I misread the code, you replace
> > > 
> > > pad0 = CSI2_PAD_SRC
> > > pad1 = CSI2_PAD_SINK
> > > 
> > > with
> > > 
> > > pad0 = pad
> > > pad1 = CSI2_PAD_SINK
> > > 
> > > This seems to be a correct fix, but I don't see where "the source pad
> > > was used as the sink pad".
> > 
> > Right, I'll reword it for v2.
> > 
> > > > (rather than the actual source pad). Fix these.
> > > > 
> > > > Fixes: 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
> > > > Cc: stable@vger.kernel.org
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > ---
> > > >  drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
> > > >  1 file changed, 6 insertions(+), 6 deletions(-)
> > > > 
> > > > diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > > > index da8581a37e22..6030bd23b4b9 100644
> > > > --- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > > > +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > > > @@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
> > > >  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
> > > >  	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
> > > >  
> > > > -	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
> > > > -						       CSI2_PAD_SINK,
> > > > -						       &streams_mask);
> > > > +	sink_streams =
> > > > +		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> > > > +						&streams_mask);
> > > 
> > > This is one of the cases where I'd write
> > > 
> > > 	sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> > > 						       &streams_mask);
> > > 
> > > even if it goes to 81 columns.
> > 
> > The limit is still 80, not 81.
> 
> The global limit is 100 for the kernel. There's somehow of a consensus
> in the media subsystem to keep it closer to 80, with different people
> have different sensitivities. We occasionally go over 80, and that's
> usually left as a driver maintainer decision.

It's 80, not 100. The checkpatch.pl limit is higher than 80 though, as
there are still commonly cases where the code is more readable with longer
lines. See e.g. Documentation/process/coding-style.rst .

-- 
Regards,

Sakari Ailus

Re: [PATCH 01/13] media: ipu6: Use correct pads for xlate_streams()

[Laurent Pinchart]

On Thu, Jun 19, 2025 at 02:28:48PM +0000, Sakari Ailus wrote:
> On Thu, Jun 19, 2025 at 05:15:35PM +0300, Laurent Pinchart wrote:
> > On Thu, Jun 19, 2025 at 01:55:08PM +0000, Sakari Ailus wrote:
> > > On Thu, Jun 19, 2025 at 04:27:04PM +0300, Laurent Pinchart wrote:
> > > > On Thu, Jun 19, 2025 at 11:15:34AM +0300, Sakari Ailus wrote:
> > > > > The arguments to v4l2_subdev_state_xlate_streams() were incorrect, the
> > > > 
> > > > s/were/are/
> > > > 
> > > > > source pads was used as the sink pad and the source pad was a constant
> > > > 
> > > > s/pads was/pad is/
> > > > s/pad was/pad is/
> > > > 
> > > > Are you sure though ? Unless I misread the code, you replace
> > > > 
> > > > pad0 = CSI2_PAD_SRC
> > > > pad1 = CSI2_PAD_SINK
> > > > 
> > > > with
> > > > 
> > > > pad0 = pad
> > > > pad1 = CSI2_PAD_SINK
> > > > 
> > > > This seems to be a correct fix, but I don't see where "the source pad
> > > > was used as the sink pad".
> > > 
> > > Right, I'll reword it for v2.
> > > 
> > > > > (rather than the actual source pad). Fix these.
> > > > > 
> > > > > Fixes: 3a5c59ad926b ("media: ipu6: Rework CSI-2 sub-device streaming control")
> > > > > Cc: stable@vger.kernel.org
> > > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > ---
> > > > >  drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 12 ++++++------
> > > > >  1 file changed, 6 insertions(+), 6 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > > > > index da8581a37e22..6030bd23b4b9 100644
> > > > > --- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > > > > +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > > > > @@ -354,9 +354,9 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
> > > > >  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
> > > > >  	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
> > > > >  
> > > > > -	sink_streams = v4l2_subdev_state_xlate_streams(state, CSI2_PAD_SRC,
> > > > > -						       CSI2_PAD_SINK,
> > > > > -						       &streams_mask);
> > > > > +	sink_streams =
> > > > > +		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> > > > > +						&streams_mask);
> > > > 
> > > > This is one of the cases where I'd write
> > > > 
> > > > 	sink_streams = v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> > > > 						       &streams_mask);
> > > > 
> > > > even if it goes to 81 columns.
> > > 
> > > The limit is still 80, not 81.
> > 
> > The global limit is 100 for the kernel. There's somehow of a consensus
> > in the media subsystem to keep it closer to 80, with different people
> > have different sensitivities. We occasionally go over 80, and that's
> > usually left as a driver maintainer decision.
> 
> It's 80, not 100. The checkpatch.pl limit is higher than 80 though, as
> there are still commonly cases where the code is more readable with longer
> lines. See e.g. Documentation/process/coding-style.rst .

It states

----
The preferred limit on the length of a single line is 80 columns.

Statements longer than 80 columns should be broken into sensible chunks,
unless exceeding 80 columns significantly increases readability and does
not hide information.
----

In this specific case, going over 80 columns improves readability in my
opinion, and doesn't hide information. As I wrote, we treat this as a
driver maintainer preference at the moment, so I won't make a call for
the ipu6 driver.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Thu, Jun 19, 2025 at 11:15:38AM +0300, Sakari Ailus wrote:
> Make media_entity_to_video_device(NULL) return NULL, instead of an invalid
> pointer value.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  include/media/v4l2-dev.h | 14 ++++++++++----
>  1 file changed, 10 insertions(+), 4 deletions(-)
> 
> diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
> index 1b6222fab24e..069c2f14b473 100644
> --- a/include/media/v4l2-dev.h
> +++ b/include/media/v4l2-dev.h
> @@ -313,10 +313,16 @@ struct video_device {
>   * media_entity_to_video_device - Returns a &struct video_device from
>   *	the &struct media_entity embedded on it.
>   *
> - * @__entity: pointer to &struct media_entity
> - */
> -#define media_entity_to_video_device(__entity) \
> -	container_of(__entity, struct video_device, entity)
> + * @__entity: pointer to &struct media_entity, may be NULL
> + */
> +#define media_entity_to_video_device(__entity)				\
> +	({								\
> +		typeof (__entity) __me_to_vdev_ent = __entity;		\
> +									\
> +		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
> +						struct video_device, entity) : \
> +			NULL;						\
> +	})

This makes the macro safer, it's a good idea. Wouldn't it be better
implemented as a container_of_null() (name to be bikeshedded) though ? I
don't think media_entity_to_video_device() is the only macro that could
benefit from this. It could even be integrated in container_of(), but I
fear that could introduce issues.

>  
>  /**
>   * to_video_device - Returns a &struct video_device from the

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

[Sakari Ailus]

Hi Laurent,

On Thu, Jun 19, 2025 at 06:20:33PM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.
> 
> On Thu, Jun 19, 2025 at 11:15:38AM +0300, Sakari Ailus wrote:
> > Make media_entity_to_video_device(NULL) return NULL, instead of an invalid
> > pointer value.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  include/media/v4l2-dev.h | 14 ++++++++++----
> >  1 file changed, 10 insertions(+), 4 deletions(-)
> > 
> > diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
> > index 1b6222fab24e..069c2f14b473 100644
> > --- a/include/media/v4l2-dev.h
> > +++ b/include/media/v4l2-dev.h
> > @@ -313,10 +313,16 @@ struct video_device {
> >   * media_entity_to_video_device - Returns a &struct video_device from
> >   *	the &struct media_entity embedded on it.
> >   *
> > - * @__entity: pointer to &struct media_entity
> > - */
> > -#define media_entity_to_video_device(__entity) \
> > -	container_of(__entity, struct video_device, entity)
> > + * @__entity: pointer to &struct media_entity, may be NULL
> > + */
> > +#define media_entity_to_video_device(__entity)				\
> > +	({								\
> > +		typeof (__entity) __me_to_vdev_ent = __entity;		\
> > +									\
> > +		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
> > +						struct video_device, entity) : \
> > +			NULL;						\
> > +	})
> 
> This makes the macro safer, it's a good idea. Wouldn't it be better
> implemented as a container_of_null() (name to be bikeshedded) though ? I
> don't think media_entity_to_video_device() is the only macro that could
> benefit from this. It could even be integrated in container_of(), but I
> fear that could introduce issues.

That sounds like a good idea. I'll first see how this would look like with
container_of_const()...

-- 
Regards,

Sakari Ailus

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Thu, Jun 19, 2025 at 11:15:39AM +0300, Sakari Ailus wrote:
> Streams are stored in sink-source pairs in sub-device state.

Is it really this simple ? Don't we support for instance stream merging
where two streams on possibly different sink pads are routed to a single
stream on a source pad ?

> When a stream
> was marked enabled (or disabled), only the state of one end of the stream
> was modified, leaving the stream in an incoherent state. Mark both ends of
> the stream enabled (or disabled).
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 60b8febd3339..5afdd9d548b5 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>  	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
>  		struct v4l2_subdev_stream_config *cfg =
>  			&state->stream_configs.configs[i];
> +		struct v4l2_subdev_stream_config *cfg2 =
> +			&state->stream_configs.configs[i ^ 1U];
>  
>  		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
> -			cfg->enabled = enabled;
> +			cfg->enabled = cfg2->enabled = enabled;
>  	}
>  }
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 07/13] media: ipu6: Set up CSI-2 receiver at correct moment

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Thu, Jun 19, 2025 at 11:15:40AM +0300, Sakari Ailus wrote:
> Enable the CSI-2 receiver before the first stream is started and disable
> it when the last stream is stopped. Before this patch, every time a stream
> was started, the CSI-2 receiver was enabled and similarly, it was disabled
> when any stream was stopped.

You mentioned in 03/13 that we should only enable the source when all
video capture devices have been started. Shouldn't you enable the CSI-2
receiver when *all* streams are enabled, not when the first one is
enabled (and similarly when stopping) ?

> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 25 ++++++++++++-------
>  1 file changed, 16 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> index 6030bd23b4b9..3b837e9ccffe 100644
> --- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> @@ -348,7 +348,7 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
>  	struct ipu6_isys_csi2_timing timing = { };
>  	struct v4l2_subdev *remote_sd;
>  	struct media_pad *remote_pad;
> -	u64 sink_streams;
> +	u64 sink_streams, already_enabled;
>  	int ret;
>  
>  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
> @@ -358,13 +358,17 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
>  		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
>  						&streams_mask);
>  
> -	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
> -	if (ret)
> -		return ret;
> +	already_enabled = v4l2_subdev_state_streams_enabled(sd, state,
> +							    CSI2_PAD_SINK);

It would be cheaper to store a bitmask of enabled streams on the csi2
structure.

> +	if (!already_enabled) {
> +		ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
> +		if (ret)
> +			return ret;
>  
> -	ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
> -	if (ret)
> -		return ret;
> +		ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
> +		if (ret)
> +			return ret;
> +	}
>  
>  	ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index,
>  					 sink_streams);
> @@ -382,7 +386,7 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
>  {
>  	struct v4l2_subdev *remote_sd;
>  	struct media_pad *remote_pad;
> -	u64 sink_streams;
> +	u64 sink_streams, still_enabled;
>  
>  	sink_streams =
>  		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> @@ -391,7 +395,10 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
>  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
>  	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
>  
> -	ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
> +	still_enabled = v4l2_subdev_state_streams_enabled(sd, state,
> +							  CSI2_PAD_SINK);
> +	if (still_enabled == sink_streams)
> +		ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
>  
>  	v4l2_subdev_disable_streams(remote_sd, remote_pad->index, sink_streams);
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 08/13] media: v4l2-subdev: Print early in v4l2_subdev_{enable,disable}_streams()

[Laurent Pinchart]

Hi Sakari,

On Thu, Jun 19, 2025 at 11:15:41AM +0300, Sakari Ailus wrote:
> Print debug messages early in v4l2_subdev_enable_streams() and
> v4l2_subdev_disable_streams(), before sanity checks take place. This can
> help figuring out why something goes wrong, in driver development or
> otherwise.
> 
> Also print the name of the sub-device where streaming is to be enabled or
> disabled.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 5afdd9d548b5..6bc855058ca6 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2273,6 +2273,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  	bool use_s_stream;
>  	int ret;
>  
> +	dev_dbg(dev, "enable streams %s:%u/%#llx\n", sd->entity.name, pad,
> +		streams_mask);

I suppose there can be multiple subdevs per struct device, so printing
the entity name can be useful. I'd put quotes around '%s' as the entity
name can contain spaces. Apart from that,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +
>  	/* A few basic sanity checks first. */
>  	if (pad >= sd->entity.num_pads)
>  		return -EINVAL;
> @@ -2320,8 +2323,6 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  		goto done;
>  	}
>  
> -	dev_dbg(dev, "enable streams %u:%#llx\n", pad, streams_mask);
> -
>  	already_streaming = v4l2_subdev_is_streaming(sd);
>  
>  	if (!use_s_stream) {
> @@ -2373,6 +2374,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  	bool use_s_stream;
>  	int ret;
>  
> +	dev_dbg(dev, "disable streams %s:%u/%#llx\n", sd->entity.name, pad,
> +		streams_mask);
> +
>  	/* A few basic sanity checks first. */
>  	if (pad >= sd->entity.num_pads)
>  		return -EINVAL;
> @@ -2420,8 +2424,6 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  		goto done;
>  	}
>  
> -	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
> -
>  	if (!use_s_stream) {
>  		/* Call the .disable_streams() operation. */
>  		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,

-- 
Regards,

Laurent Pinchart

Re: [PATCH 09/13] media: v4l2-subdev: Collect streams on source pads only

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Thu, Jun 19, 2025 at 11:15:42AM +0300, Sakari Ailus wrote:
> v4l2_subdev_collect_streams() is used to find the streams present on
> source pads only. Only iterate through the streams on source pads, i.e. on
> odd array indices.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 15 ++++++++-------
>  1 file changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 6bc855058ca6..932fca795d4a 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2225,16 +2225,17 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
>  	*found_streams = 0;
>  	*enabled_streams = 0;
>  
> -	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
> -		const struct v4l2_subdev_stream_config *cfg =
> -			&state->stream_configs.configs[i];
> +	for (unsigned int i = 0; i < state->stream_configs.num_configs; i += 2) {
> +		const struct v4l2_subdev_stream_config *src_cfg =
> +			&state->stream_configs.configs[i + 1];

You could start at i = 1 and avoid the + 1 here.

Unless I missed it, I don't think we document anywhere that we store two
values per route in the array, and that, if multiple routes have the
same sink or source stream, multiple entries will exist for the same
stream. I'd like to see this clearly explained somewhere, instead of
relying on an implemetation detail only known by few people.

>  
> -		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
> +		if (src_cfg->pad != pad ||
> +		    !(streams_mask & BIT_ULL(src_cfg->stream)))
>  			continue;
>  
> -		*found_streams |= BIT_ULL(cfg->stream);
> -		if (cfg->enabled)
> -			*enabled_streams |= BIT_ULL(cfg->stream);
> +		*found_streams |= BIT_ULL(src_cfg->stream);
> +		if (src_cfg->enabled)
> +			*enabled_streams |= BIT_ULL(src_cfg->stream);
>  	}
>  }
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH 07/13] media: ipu6: Set up CSI-2 receiver at correct moment

[Sakari Ailus]

Hi Laurent,

On Thu, Jun 19, 2025 at 08:00:30PM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.

Thanks for the review.

> 
> On Thu, Jun 19, 2025 at 11:15:40AM +0300, Sakari Ailus wrote:
> > Enable the CSI-2 receiver before the first stream is started and disable
> > it when the last stream is stopped. Before this patch, every time a stream
> > was started, the CSI-2 receiver was enabled and similarly, it was disabled
> > when any stream was stopped.
> 
> You mentioned in 03/13 that we should only enable the source when all
> video capture devices have been started. Shouldn't you enable the CSI-2
> receiver when *all* streams are enabled, not when the first one is
> enabled (and similarly when stopping) ?

Indeed. That only comes in the last patch, but I think some of these can be
squashed, too.

> 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 25 ++++++++++++-------
> >  1 file changed, 16 insertions(+), 9 deletions(-)
> > 
> > diff --git a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > index 6030bd23b4b9..3b837e9ccffe 100644
> > --- a/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > +++ b/drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c
> > @@ -348,7 +348,7 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
> >  	struct ipu6_isys_csi2_timing timing = { };
> >  	struct v4l2_subdev *remote_sd;
> >  	struct media_pad *remote_pad;
> > -	u64 sink_streams;
> > +	u64 sink_streams, already_enabled;
> >  	int ret;
> >  
> >  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
> > @@ -358,13 +358,17 @@ static int ipu6_isys_csi2_enable_streams(struct v4l2_subdev *sd,
> >  		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> >  						&streams_mask);
> >  
> > -	ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
> > -	if (ret)
> > -		return ret;
> > +	already_enabled = v4l2_subdev_state_streams_enabled(sd, state,
> > +							    CSI2_PAD_SINK);
> 
> It would be cheaper to store a bitmask of enabled streams on the csi2
> structure.

I think I'll do just that. Indeed, this soon won't be even usable here in
any case.

> 
> > +	if (!already_enabled) {
> > +		ret = ipu6_isys_csi2_calc_timing(csi2, &timing, CSI2_ACCINV);
> > +		if (ret)
> > +			return ret;
> >  
> > -	ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
> > -	if (ret)
> > -		return ret;
> > +		ret = ipu6_isys_csi2_set_stream(sd, &timing, csi2->nlanes, true);
> > +		if (ret)
> > +			return ret;
> > +	}
> >  
> >  	ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index,
> >  					 sink_streams);
> > @@ -382,7 +386,7 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
> >  {
> >  	struct v4l2_subdev *remote_sd;
> >  	struct media_pad *remote_pad;
> > -	u64 sink_streams;
> > +	u64 sink_streams, still_enabled;
> >  
> >  	sink_streams =
> >  		v4l2_subdev_state_xlate_streams(state, pad, CSI2_PAD_SINK,
> > @@ -391,7 +395,10 @@ static int ipu6_isys_csi2_disable_streams(struct v4l2_subdev *sd,
> >  	remote_pad = media_pad_remote_pad_first(&sd->entity.pads[CSI2_PAD_SINK]);
> >  	remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
> >  
> > -	ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
> > +	still_enabled = v4l2_subdev_state_streams_enabled(sd, state,
> > +							  CSI2_PAD_SINK);
> > +	if (still_enabled == sink_streams)
> > +		ipu6_isys_csi2_set_stream(sd, NULL, 0, false);
> >  
> >  	v4l2_subdev_disable_streams(remote_sd, remote_pad->index, sink_streams);
> >  
> 

-- 
Regards,

Sakari Ailus

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

[Sakari Ailus]

Hi Laurent,

On Thu, Jun 19, 2025 at 07:56:04PM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.

Thanks for the review.

> 
> On Thu, Jun 19, 2025 at 11:15:39AM +0300, Sakari Ailus wrote:
> > Streams are stored in sink-source pairs in sub-device state.
> 
> Is it really this simple ? Don't we support for instance stream merging
> where two streams on possibly different sink pads are routed to a single
> stream on a source pad ?

As far as I'm concerned, yes. Each route has a sink and a source pad, you
may have multiple routes from a pad or to a pad, but these are all separate
routes. The UAPI as well as v4l2-subdev.c is aligned with this and I
wouldn't expect it to be otherwise.

I'd like to have Tomi's view on this, too.

> 
> > When a stream
> > was marked enabled (or disabled), only the state of one end of the stream
> > was modified, leaving the stream in an incoherent state. Mark both ends of
> > the stream enabled (or disabled).

This should discuss "route" in fact, not "stream".

-- 
Regards,

Sakari Ailus

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

[Laurent Pinchart]

On Thu, Jun 19, 2025 at 06:34:25PM +0000, Sakari Ailus wrote:
> On Thu, Jun 19, 2025 at 07:56:04PM +0300, Laurent Pinchart wrote:
> > On Thu, Jun 19, 2025 at 11:15:39AM +0300, Sakari Ailus wrote:
> > > Streams are stored in sink-source pairs in sub-device state.
> > 
> > Is it really this simple ? Don't we support for instance stream merging
> > where two streams on possibly different sink pads are routed to a single
> > stream on a source pad ?
> 
> As far as I'm concerned, yes. Each route has a sink and a source pad, you
> may have multiple routes from a pad or to a pad, but these are all separate
> routes. The UAPI as well as v4l2-subdev.c is aligned with this and I
> wouldn't expect it to be otherwise.

Should we then refactor v4l2_subdev_stream_configs to contain a list of
route configs, with each route config containing two stream configs, one
for the sink pad and one for the source pad ? We would still have
duplicated entries when multiple routes originate from or end at the
same pad/stream, but the data structures would be more explicit and we
would rely less on the fact that each route has a pair of consecutive
stream configs.

> I'd like to have Tomi's view on this, too.
> 
> > > When a stream
> > > was marked enabled (or disabled), only the state of one end of the stream
> > > was modified, leaving the stream in an incoherent state. Mark both ends of
> > > the stream enabled (or disabled).
> 
> This should discuss "route" in fact, not "stream".

That would make it a bit clearer.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 10/13] media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Thu, Jun 19, 2025 at 11:15:43AM +0300, Sakari Ailus wrote:
> Print streams found by v4l2_subdev_collect_streams() at debug level.

Explaining why you found this useful could help.

> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 932fca795d4a..c549a462dac7 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2219,6 +2219,8 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
>  		*found_streams = BIT_ULL(0);
>  		*enabled_streams =
>  			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
> +		dev_dbg(sd->dev,
> +			"collect_streams: sub-device does not support streams\n");

You print the subdev name below, I would print it here too. Maybe

			"collect_streams: '%s':%u: streams not supported\n",
			sd->entity.name, pad);

>  		return;
>  	}
>  
> @@ -2237,6 +2239,9 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
>  		if (src_cfg->enabled)
>  			*enabled_streams |= BIT_ULL(src_cfg->stream);
>  	}

Please add a blank line here.

> +	dev_dbg(sd->dev,
> +		"collect_streams: %s:%u found %#llx enabled %#llx\n",

		"collect_streams: '%s':%u: found %#llx, enabled: %#llx\n",

or possibly

		"%s: '%s':%u: found %#llx, enabled: %#llx\n", __func__,

(same above).

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +		sd->entity.name, pad, *found_streams, *enabled_streams);
>  }
>  
>  static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,

-- 
Regards,

Laurent Pinchart

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Dan Carpenter]

Hi Sakari,

kernel test robot noticed the following build warnings:

https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Sakari-Ailus/media-ipu6-Use-correct-pads-for-xlate_streams/20250619-161847
base:   https://git.linuxtv.org/media-ci/media-pending.git master
patch link:    https://lore.kernel.org/r/20250619081546.1582969-13-sakari.ailus%40linux.intel.com
patch subject: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()
config: i386-randconfig-141-20250620 (https://download.01.org/0day-ci/archive/20250620/202506201121.oifnpp7r-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
| Closes: https://lore.kernel.org/r/202506201121.oifnpp7r-lkp@intel.com/

smatch warnings:
drivers/media/v4l2-core/v4l2-mc.c:732 v4l2_mc_downpath_enabled() error: uninitialized symbol 'source_stream'.
drivers/media/v4l2-core/v4l2-mc.c:834 v4l2_mc_pipeline_enabled() warn: variable dereferenced before IS_ERR check 'src_pad' (see line 802)

vim +/source_stream +732 drivers/media/v4l2-core/v4l2-mc.c

bc83f513f8ad94 Sakari Ailus 2025-06-19  771  int v4l2_mc_pipeline_enabled(struct video_device *vdev,
bc83f513f8ad94 Sakari Ailus 2025-06-19  772  			     bool (*func)(struct video_device *vdev),
bc83f513f8ad94 Sakari Ailus 2025-06-19  773  			     struct media_pad **__sink_pad, u64 *__sink_streams)
bc83f513f8ad94 Sakari Ailus 2025-06-19  774  {
bc83f513f8ad94 Sakari Ailus 2025-06-19  775  	u64 sink_streams = 1U;
bc83f513f8ad94 Sakari Ailus 2025-06-19  776  	struct media_pad *src_pad;
bc83f513f8ad94 Sakari Ailus 2025-06-19  777  	u64 src_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19  778  	struct v4l2_subdev_state *state;
bc83f513f8ad94 Sakari Ailus 2025-06-19  779  	struct media_pad *sink_pad = vdev->entity.pads;
bc83f513f8ad94 Sakari Ailus 2025-06-19  780  	struct v4l2_subdev *sd = NULL;
bc83f513f8ad94 Sakari Ailus 2025-06-19  781  	bool streaming = true;
bc83f513f8ad94 Sakari Ailus 2025-06-19  782  	struct media_pad *tmp_pad;
bc83f513f8ad94 Sakari Ailus 2025-06-19  783  	u64 tmp_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19  784  	int ret;
bc83f513f8ad94 Sakari Ailus 2025-06-19  785  
bc83f513f8ad94 Sakari Ailus 2025-06-19  786  	if (!__sink_pad)
bc83f513f8ad94 Sakari Ailus 2025-06-19  787  		__sink_pad = &tmp_pad;
bc83f513f8ad94 Sakari Ailus 2025-06-19  788  	if (!__sink_streams)
bc83f513f8ad94 Sakari Ailus 2025-06-19  789  		__sink_streams = &tmp_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19  790  	*__sink_pad = NULL;
bc83f513f8ad94 Sakari Ailus 2025-06-19  791  	*__sink_streams = 0;
bc83f513f8ad94 Sakari Ailus 2025-06-19  792  
bc83f513f8ad94 Sakari Ailus 2025-06-19  793  	do {
bc83f513f8ad94 Sakari Ailus 2025-06-19  794  		src_pad = media_pad_remote_pad_unique(sink_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19  795  		if (IS_ERR(src_pad)) {
bc83f513f8ad94 Sakari Ailus 2025-06-19  796  			dev_dbg(sd ? sd->dev : vdev->dev_parent,
bc83f513f8ad94 Sakari Ailus 2025-06-19  797  				"no unique remote pad found from %s:%u\n",
bc83f513f8ad94 Sakari Ailus 2025-06-19  798  				sink_pad->entity->name, sink_pad->index);
bc83f513f8ad94 Sakari Ailus 2025-06-19  799  			return PTR_ERR(src_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19  800  		}
bc83f513f8ad94 Sakari Ailus 2025-06-19  801  
bc83f513f8ad94 Sakari Ailus 2025-06-19 @802  		sd = media_entity_to_v4l2_subdev(src_pad->entity);
bc83f513f8ad94 Sakari Ailus 2025-06-19  803  		if (!sd) {
bc83f513f8ad94 Sakari Ailus 2025-06-19  804  			dev_dbg(sd->dev,
bc83f513f8ad94 Sakari Ailus 2025-06-19  805  				"media entity %s is not a V4L2 sub-device\n",
bc83f513f8ad94 Sakari Ailus 2025-06-19  806  				src_pad->entity->name);
bc83f513f8ad94 Sakari Ailus 2025-06-19  807  			return -ENXIO;
bc83f513f8ad94 Sakari Ailus 2025-06-19  808  		}
bc83f513f8ad94 Sakari Ailus 2025-06-19  809  
bc83f513f8ad94 Sakari Ailus 2025-06-19  810  		/* Source streams match sink. */
bc83f513f8ad94 Sakari Ailus 2025-06-19  811  		src_streams = sink_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19  812  
bc83f513f8ad94 Sakari Ailus 2025-06-19  813  		state = v4l2_subdev_lock_and_get_active_state(sd);
bc83f513f8ad94 Sakari Ailus 2025-06-19  814  		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
bc83f513f8ad94 Sakari Ailus 2025-06-19  815  						 src_streams, &sink_pad,
bc83f513f8ad94 Sakari Ailus 2025-06-19  816  						 &sink_streams);
bc83f513f8ad94 Sakari Ailus 2025-06-19  817  		v4l2_subdev_unlock_state(state);
bc83f513f8ad94 Sakari Ailus 2025-06-19  818  		if (ret)
bc83f513f8ad94 Sakari Ailus 2025-06-19  819  			return ret;
bc83f513f8ad94 Sakari Ailus 2025-06-19  820  	} while (sink_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19  821  
bc83f513f8ad94 Sakari Ailus 2025-06-19  822  	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
bc83f513f8ad94 Sakari Ailus 2025-06-19  823  	if (ret)
bc83f513f8ad94 Sakari Ailus 2025-06-19  824  		return ret;
bc83f513f8ad94 Sakari Ailus 2025-06-19  825  
bc83f513f8ad94 Sakari Ailus 2025-06-19  826  	sd = media_entity_to_v4l2_subdev(src_pad->entity);
bc83f513f8ad94 Sakari Ailus 2025-06-19  827  
bc83f513f8ad94 Sakari Ailus 2025-06-19  828  	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
bc83f513f8ad94 Sakari Ailus 2025-06-19  829  		src_pad->index, src_streams);
bc83f513f8ad94 Sakari Ailus 2025-06-19  830  
bc83f513f8ad94 Sakari Ailus 2025-06-19  831  	for (unsigned int i = __ffs(src_streams); src_streams;
bc83f513f8ad94 Sakari Ailus 2025-06-19  832  	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
bc83f513f8ad94 Sakari Ailus 2025-06-19  833  		sink_pad = media_pad_remote_pad_unique(src_pad);
bc83f513f8ad94 Sakari Ailus 2025-06-19 @834  		if (IS_ERR(src_pad)) {
                                                                   ^^^^^^^
Copy and paste.  s/src_pad/sink_pad/.


-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()

[Jacopo Mondi]

Hi Sakari

On Thu, Jun 19, 2025 at 11:15:44AM +0300, Sakari Ailus wrote:
> v4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(),
> with the difference that it's more flexible: it can look up only active
> routes and can find multiple routes, too.
>
> v4l2_subdev_find_route() is intended to replace
> v4l2_subdev_routing_find_opposite_end().

To me this feels like v4l2_subdev_find_route() could be used to
implement more helpers like v4l2_subdev_routing_find_opposite_end()
for drivers instead of going the other way around.

let's see what the use cases are

>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++---------
>  include/media/v4l2-subdev.h           | 19 +++++++++
>  2 files changed, 56 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index c549a462dac7..13d6e96daf3a 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
>  }
>  EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
>
> -int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
> -					  u32 pad, u32 stream, u32 *other_pad,
> -					  u32 *other_stream)
> +struct v4l2_subdev_route *
> +v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
> +		       u32 pad, u32 stream, bool active, unsigned int index)
>  {
>  	unsigned int i;
>
>  	for (i = 0; i < routing->num_routes; ++i) {
>  		struct v4l2_subdev_route *route = &routing->routes[i];
>
> -		if (route->source_pad == pad &&
> -		    route->source_stream == stream) {
> -			if (other_pad)
> -				*other_pad = route->sink_pad;
> -			if (other_stream)
> -				*other_stream = route->sink_stream;
> -			return 0;
> -		}
> +		if (active && !(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> +			continue;

I know currently v4l2_subdev_routing_find_opposite_end() does return
any route that matches the provided 'pad' and 'stream' included
non-active ones, but I wonder if this is desirable. What is the use
case for enumerating a non-active route between two pads ?

(it is also my impression that all drivers that use
v4l2_subdev_routing_find_opposite_end() assume the route is active)

Also I wonder if the usage of V4L2_SUBDEV_ROUTE_FL_ACTIVE is clearly
defined, or, in other words, what is the use case for userspace to
create non-active routes, given that any new VIDIOC_SUBDEV_S_ROUTING
will anyway re-create the routing table (that's a different question,
on the ioctl definition and not on this change though)

>
> -		if (route->sink_pad == pad && route->sink_stream == stream) {
> -			if (other_pad)
> -				*other_pad = route->source_pad;
> -			if (other_stream)
> -				*other_stream = route->source_stream;
> -			return 0;
> -		}
> +		if ((route->source_pad != pad ||
> +		     route->source_stream != stream) &&
> +		    (route->sink_pad != pad || route->sink_stream != stream))
> +			continue;
> +
> +		if (index--)
> +			continue;
> +
> +		return route;
>  	}
>
> -	return -EINVAL;
> +	return ERR_PTR(-ENOENT);
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_find_route);
> +
> +int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
> +					  u32 pad, u32 stream, u32 *other_pad,
> +					  u32 *other_stream)
> +{
> +	struct v4l2_subdev_route *route;
> +
> +	route = v4l2_subdev_find_route(routing, pad, stream, false, 0);
> +	if (IS_ERR(route))
> +		return PTR_ERR(route);
> +
> +	bool is_source = route->source_pad == pad;
> +
> +	if (other_pad)
> +		*other_pad = is_source ? route->sink_pad : route->source_pad;
> +	if (other_stream)
> +		*other_stream = is_source ?
> +			route->sink_stream : route->source_stream;
> +
> +	return 0;
>  }
>  EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
>
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index deab128a4779..9ed8600ba3d4 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
>  				     const struct v4l2_subdev_krouting *routing,
>  				     const struct v4l2_mbus_framefmt *fmt);
>
> +/**
> + * v4l2_subdev_find_route() - Find routes from a (pad, stream) pair

from or for ?

> + * @routing: routing used to find the opposite side

I would not say "opposite side" but rather

      @routing: routing table used to enumerate routes

> + * @pad: pad id
> + * @stream: stream id
> + * @active: set to true for looking up only active routes
> + * @index: for accessing more than one route from the pad

I understand this but maybe

     @index: route index for enumerating multiple routes
?

> + *
> + * Find a route from the routing table where one end has (pad, stream) pair
> + * matching @pad and @stream.

    * If multiple routes in @routing match @pad and @stream, return
    * the @index one.
    *
    * Set @active to true to only enumerate active routes.

> + *
> + * Returns the route on success or -ENOENT if no matching route is found.

I see other functions documentation using

    * Return:

is this a kernel-doc thing ?

> + */
> +struct v4l2_subdev_route *
> +v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
> +		       u32 pad, u32 stream, bool active, unsigned int index);
> +
>  /**
>   * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
>   * @routing: routing used to find the opposite side
> @@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
>   * @other_pad: pointer used to return the opposite pad
>   * @other_stream: pointer used to return the opposite stream
>   *
> + * Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end().
> + *

As said, I'm not sure if that's preferred or we should rather create
more helpers using v4l2_subdev_find_route() internally. Time will tell
I guess ?

Thanks
  j


>   * This function uses the routing table to find the pad + stream which is
>   * opposite the given pad + stream.
>   *
> --
> 2.39.5
>
>

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Jacopo Mondi]

Hi Sakari

On Thu, Jun 19, 2025 at 11:15:45AM +0300, Sakari Ailus wrote:
> v4l2_mc_pipeline_enabled() helps solving a problem known for long but
> lacked any sort of general solution: with multiple streams, when streaming
> is started on video nodes one by one, when should streaming be started in
> the source?

I tried quite some time to understand this, but if I'm not mistaken,
a stream-aware subdev, which links to vdev, will always "demux"
streams to different pads and will connect to the vdev from there


     Source
     subdev
 +-----------------+
 |                (1/0) ------> vdev0
 |                 |
(0)[1,2,3]        (2/0  ------> vdev1
 |                 |
 |                (3/0) ------> vdev2
 +-----------------+

With

(0) multiplexed sink pad with 3 streams
(1) (2) and (3) source pad with a single stream

Can't we relay on the media-link state between the source pads and the
video devices with something like what Dan has proposed here ?
https://patchwork.linuxtv.org/project/linux-media/patch/20250519140403.443915-2-dan.scally@ideasonboard.com/

What am I missing ?

>
> v4l2_mc_pipeline_enabled() traverses the pipeline towards the source,
> queries the streams generated by the source and traces them back to the
> video nodes.
>
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
>  include/media/v4l2-mc.h           |  44 ++++++
>  2 files changed, 287 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
> index 937d358697e1..1731088ad436 100644
> --- a/drivers/media/v4l2-core/v4l2-mc.c
> +++ b/drivers/media/v4l2-core/v4l2-mc.c
> @@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
> +
> +static int
> +__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
> +			   struct media_pad *src_pad, u64 __src_streams,
> +			   struct media_pad **__sink_pad, u64 *__sink_streams)
> +{
> +	struct v4l2_subdev_route *route;
> +	u64 src_streams = 0, sink_streams = 0;
> +	bool has_sink_pad = false;
> +	unsigned int sink_pad;
> +
> +	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
> +		state->sd->entity.name, src_pad->index, __src_streams);
> +	for_each_active_route(&state->routing, route) {
> +		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
> +			state->sd->entity.name,
> +			route->sink_pad, route->sink_stream, route->source_pad,
> +			route->source_stream, route->flags);
> +		if (route->source_pad != src_pad->index)
> +			continue;
> +
> +		if (!(BIT_ULL(route->source_stream) & __src_streams))
> +			continue;
> +
> +		if (!has_sink_pad) {
> +			has_sink_pad = true;
> +			sink_pad = route->sink_pad;
> +		}
> +
> +		if (route->sink_pad != sink_pad) {
> +			dev_dbg(state->sd->dev,
> +				"sink pads (%u vs. %u) differ\n",
> +				route->sink_pad, sink_pad);
> +			return -EMLINK;
> +		}
> +
> +		sink_streams |= BIT_ULL(route->sink_stream);
> +		src_streams |= BIT_ULL(route->source_stream);
> +	}
> +
> +	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
> +	*__sink_streams = sink_streams;
> +
> +	return 0;
> +}
> +
> +static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
> +				    unsigned int sink_stream,
> +				    bool (*func)(struct video_device *vdev),
> +				    struct media_pad **__sink_pad,
> +				    u64 *__sink_streams)
> +{
> +	struct v4l2_subdev_state *state;
> +	struct v4l2_subdev_route *route;
> +	struct v4l2_subdev *sd;
> +	struct media_pad *source_pad, *tmp_pad;
> +	u32 source_stream;
> +
> +	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
> +		return -ENXIO;
> +
> +	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
> +	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
> +		sd->entity.name);
> +
> +	state = v4l2_subdev_lock_and_get_active_state(sd);
> +	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
> +				       sink_stream, true, 0);
> +	if (IS_ERR(route)) {
> +		v4l2_subdev_unlock_state(state);
> +		dev_dbg(sd->dev,
> +			"path_enabled: can't find opposite route for %s:%u/%u",
> +			sd->entity.name, sink_pad->index, sink_stream);
> +		return 2;
> +	}
> +
> +	source_pad = &sd->entity.pads[route->source_pad];
> +	v4l2_subdev_unlock_state(state);
> +
> +	tmp_pad = sink_pad;
> +	sink_pad = media_pad_remote_pad_unique(source_pad);
> +	if (IS_ERR(sink_pad)) {
> +		dev_dbg(sd->dev,
> +			"path_enabled: can't find remote source for %s:%u\n",
> +			source_pad->entity->name, source_pad->index);
> +		return PTR_ERR(sink_pad);
> +	}
> +
> +	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
> +		struct video_device *vdev;
> +
> +		vdev = media_entity_to_video_device(sink_pad->entity);
> +		if (!vdev)
> +			return -ENXIO;
> +
> +		dev_dbg(vdev->dev_parent,
> +			"path_enabled: found video device %s\n",
> +			vdev->name);
> +
> +		if (!*__sink_pad) {
> +			*__sink_pad = tmp_pad;
> +			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
> +				tmp_pad->index, sink_stream);
> +		} else if (tmp_pad != *__sink_pad) {
> +			dev_dbg(sd->dev,
> +				"path_enabled: pads %s/%u and %s/%u differ\n",
> +				tmp_pad->entity->name, tmp_pad->index,
> +				(*__sink_pad)->entity->name,
> +				(*__sink_pad)->index);
> +			return -EXDEV;
> +		}
> +
> +		*__sink_streams |= BIT_ULL(sink_stream);
> +
> +		return func(vdev);
> +	}
> +
> +	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
> +					__sink_pad, __sink_streams);
> +}
> +
> +static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
> +				      u64 *__streams)
> +{
> +	struct v4l2_mbus_frame_desc desc;
> +	u64 streams = 0;
> +	int ret;
> +
> +	if (!__streams)
> +		return -EINVAL;
> +
> +	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
> +	if (ret == -ENOIOCTLCMD) {
> +		*__streams = 1ULL;
> +		return 0;
> +	}
> +	if (ret)
> +		return ret;
> +
> +	for (unsigned int i = 0; i < desc.num_entries; i++) {
> +		if (streams & BIT_ULL(desc.entry[i].stream))
> +			return -EINVAL;
> +
> +		streams |= BIT_ULL(desc.entry[i].stream);
> +	}
> +
> +	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
> +		*__streams, streams);
> +	if (*__streams & ~streams)
> +		return -EINVAL;
> +
> +	*__streams = streams;
> +
> +	return 0;
> +}
> +
> +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> +			     bool (*func)(struct video_device *vdev),
> +			     struct media_pad **__sink_pad, u64 *__sink_streams)
> +{
> +	u64 sink_streams = 1U;
> +	struct media_pad *src_pad;
> +	u64 src_streams;
> +	struct v4l2_subdev_state *state;
> +	struct media_pad *sink_pad = vdev->entity.pads;
> +	struct v4l2_subdev *sd = NULL;
> +	bool streaming = true;
> +	struct media_pad *tmp_pad;
> +	u64 tmp_streams;
> +	int ret;
> +
> +	if (!__sink_pad)
> +		__sink_pad = &tmp_pad;
> +	if (!__sink_streams)
> +		__sink_streams = &tmp_streams;
> +	*__sink_pad = NULL;
> +	*__sink_streams = 0;
> +
> +	do {
> +		src_pad = media_pad_remote_pad_unique(sink_pad);
> +		if (IS_ERR(src_pad)) {
> +			dev_dbg(sd ? sd->dev : vdev->dev_parent,
> +				"no unique remote pad found from %s:%u\n",
> +				sink_pad->entity->name, sink_pad->index);
> +			return PTR_ERR(src_pad);
> +		}
> +
> +		sd = media_entity_to_v4l2_subdev(src_pad->entity);
> +		if (!sd) {
> +			dev_dbg(sd->dev,
> +				"media entity %s is not a V4L2 sub-device\n",
> +				src_pad->entity->name);
> +			return -ENXIO;
> +		}
> +
> +		/* Source streams match sink. */
> +		src_streams = sink_streams;
> +
> +		state = v4l2_subdev_lock_and_get_active_state(sd);
> +		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
> +						 src_streams, &sink_pad,
> +						 &sink_streams);
> +		v4l2_subdev_unlock_state(state);
> +		if (ret)
> +			return ret;
> +	} while (sink_pad);
> +
> +	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
> +	if (ret)
> +		return ret;
> +
> +	sd = media_entity_to_v4l2_subdev(src_pad->entity);
> +
> +	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
> +		src_pad->index, src_streams);
> +
> +	for (unsigned int i = __ffs(src_streams); src_streams;
> +	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
> +		sink_pad = media_pad_remote_pad_unique(src_pad);
> +		if (IS_ERR(src_pad)) {
> +			dev_dbg(sd->dev,
> +				"no unique remote pad found from %s:%u\n",
> +				sink_pad->entity->name, sink_pad->index);
> +			return PTR_ERR(src_pad);
> +		}
> +
> +		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
> +					       __sink_streams);
> +		if (ret == 2)
> +			continue;
> +		if (ret < 0)
> +			return ret;
> +		if (!ret)
> +			streaming = false;
> +	}
> +
> +	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
> +		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
> +		(*__sink_pad)->index, *__sink_streams);
> +
> +	return streaming;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
> diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
> index 1837c9fd78cf..e72c0f62fa34 100644
> --- a/include/media/v4l2-mc.h
> +++ b/include/media/v4l2-mc.h
> @@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
>  int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
>  			      unsigned int notification);
>
> +/**
> + * v4l2_mc_pipeline_enabled - Tell when to start streaming
> + * @vdev: The video device
> + * @func: Caller-provided function to tell a video device's streaming state
> + * @__sink_pad: sink pad at the root of the local pipeline
> + * @__sink_streams: streams to start
> + *
> + * Use to tell whether streaming should start on a video node. @func returns
> + * true if streaming has been started on a given video node. @__sink_pad and
> + * @__sink_streams are filled with pad and streams on the sub-device closest to
> + * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
> + * v4l2_subdev_disable_streams().
> + *
> + * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
> + * a to-do list):
> + * * only unbranched streams can be supported albeit adding support for
> + *   downstream branches would be fairly trivial,
> + * * streams within a single source sub-device are considered to start at the
> + *   same time, more control could be added in two ways: 1) for sources to
> + *   determine stream starting, a control could be added to UAPI and 2) sources
> + *   could tell which streams start at the same time using a sub-device
> + *   operation,
> + * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
> + *   be implemented by letting the caller to provide a function to determine
> + *   which streams are of interest and
> + * * routes leading to nowhere are ignored, on some hardware this is a problem,
> + *   but this can also be rather trivially addressed.
> + *
> + * Return:
> + * * 0: Success, but don't start streaming yet
> + * * 1: Success, now it's time to start streaming
> + * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
> + * * -ENOTUNIQ: No unique remote pad
> + * * -ENOLINK: No remote pad found
> + * * -ENOENT: Enabled upstream route not found
> + * * -EMLINK: No unique downstream route found
> + * * -EINVAL: Stream could not be followed to source or was not produced by
> + *            the source
> + */
> +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> +			     bool (*func)(struct video_device *vdev),
> +			     struct media_pad **__sink_pad,
> +			     u64 *__sink_streams);
> +
>  #else /* CONFIG_MEDIA_CONTROLLER */
>
>  static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
> --
> 2.39.5
>
>

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Sakari Ailus]

Hi Jacopo,

Thank you for the comments.

On Fri, Jun 20, 2025 at 10:53:13AM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Thu, Jun 19, 2025 at 11:15:45AM +0300, Sakari Ailus wrote:
> > v4l2_mc_pipeline_enabled() helps solving a problem known for long but
> > lacked any sort of general solution: with multiple streams, when streaming
> > is started on video nodes one by one, when should streaming be started in
> > the source?
> 
> I tried quite some time to understand this, but if I'm not mistaken,
> a stream-aware subdev, which links to vdev, will always "demux"
> streams to different pads and will connect to the vdev from there
> 
> 
>      Source
>      subdev
>  +-----------------+
>  |                (1/0) ------> vdev0
>  |                 |
> (0)[1,2,3]        (2/0  ------> vdev1
>  |                 |
>  |                (3/0) ------> vdev2
>  +-----------------+
> 
> With
> 
> (0) multiplexed sink pad with 3 streams
> (1) (2) and (3) source pad with a single stream
> 
> Can't we relay on the media-link state between the source pads and the
> video devices with something like what Dan has proposed here ?
> https://patchwork.linuxtv.org/project/linux-media/patch/20250519140403.443915-2-dan.scally@ideasonboard.com/
> 
> What am I missing ?

There are two different concepts here that are probably underdocumented at
the moment: Media entity pipeline state and then when and in which order
streaming control is applied on hardware. But... maybe the latter can be
derived from the former when it comes to video nodes. The pipeline is set
for entities that are upstream from the video device, it's not set on
downstream entities. I'll see if I could use that instead. Otherwise, I'll
check if Dan's approach could be used for the purpose.

-- 
Regards,

Sakari Ailus

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

[Sakari Ailus]

Hi Laurent,

On Fri, Jun 20, 2025 at 01:18:57AM +0300, Laurent Pinchart wrote:
> On Thu, Jun 19, 2025 at 06:34:25PM +0000, Sakari Ailus wrote:
> > On Thu, Jun 19, 2025 at 07:56:04PM +0300, Laurent Pinchart wrote:
> > > On Thu, Jun 19, 2025 at 11:15:39AM +0300, Sakari Ailus wrote:
> > > > Streams are stored in sink-source pairs in sub-device state.
> > > 
> > > Is it really this simple ? Don't we support for instance stream merging
> > > where two streams on possibly different sink pads are routed to a single
> > > stream on a source pad ?
> > 
> > As far as I'm concerned, yes. Each route has a sink and a source pad, you
> > may have multiple routes from a pad or to a pad, but these are all separate
> > routes. The UAPI as well as v4l2-subdev.c is aligned with this and I
> > wouldn't expect it to be otherwise.
> 
> Should we then refactor v4l2_subdev_stream_configs to contain a list of
> route configs, with each route config containing two stream configs, one
> for the sink pad and one for the source pad ? We would still have
> duplicated entries when multiple routes originate from or end at the
> same pad/stream, but the data structures would be more explicit and we
> would rely less on the fact that each route has a pair of consecutive
> stream configs.

Most of the code dealing with stream configurations works on either sink or
source pads and the current data structure makes that code more simple. I
think this was probably a factor why Tomi wrote it like this.

Tomi, any comment on this?

> 
> > I'd like to have Tomi's view on this, too.
> > 
> > > > When a stream
> > > > was marked enabled (or disabled), only the state of one end of the stream
> > > > was modified, leaving the stream in an incoherent state. Mark both ends of
> > > > the stream enabled (or disabled).
> > 
> > This should discuss "route" in fact, not "stream".
> 
> That would make it a bit clearer.

I'll fix this for v2.

-- 
Regards,

Sakari Ailus

Re: [PATCH 08/13] media: v4l2-subdev: Print early in v4l2_subdev_{enable,disable}_streams()

[Sakari Ailus]

Hi Laurent,

On Thu, Jun 19, 2025 at 08:03:20PM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> On Thu, Jun 19, 2025 at 11:15:41AM +0300, Sakari Ailus wrote:
> > Print debug messages early in v4l2_subdev_enable_streams() and
> > v4l2_subdev_disable_streams(), before sanity checks take place. This can
> > help figuring out why something goes wrong, in driver development or
> > otherwise.
> > 
> > Also print the name of the sub-device where streaming is to be enabled or
> > disabled.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-subdev.c | 10 ++++++----
> >  1 file changed, 6 insertions(+), 4 deletions(-)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > index 5afdd9d548b5..6bc855058ca6 100644
> > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > @@ -2273,6 +2273,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> >  	bool use_s_stream;
> >  	int ret;
> >  
> > +	dev_dbg(dev, "enable streams %s:%u/%#llx\n", sd->entity.name, pad,
> > +		streams_mask);
> 
> I suppose there can be multiple subdevs per struct device, so printing
> the entity name can be useful. I'd put quotes around '%s' as the entity
> name can contain spaces. Apart from that,

I'll use double quotes as that is how sub-device (or entity) name is
typically quoted in debug prints.

> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thank you.

-- 
Regards,

Sakari Ailus

Re: [PATCH 09/13] media: v4l2-subdev: Collect streams on source pads only

[Sakari Ailus]

Hi Laurent,

On Thu, Jun 19, 2025 at 08:07:17PM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.

Thank you for the review!

> 
> On Thu, Jun 19, 2025 at 11:15:42AM +0300, Sakari Ailus wrote:
> > v4l2_subdev_collect_streams() is used to find the streams present on
> > source pads only. Only iterate through the streams on source pads, i.e. on
> > odd array indices.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-subdev.c | 15 ++++++++-------
> >  1 file changed, 8 insertions(+), 7 deletions(-)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > index 6bc855058ca6..932fca795d4a 100644
> > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > @@ -2225,16 +2225,17 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
> >  	*found_streams = 0;
> >  	*enabled_streams = 0;
> >  
> > -	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
> > -		const struct v4l2_subdev_stream_config *cfg =
> > -			&state->stream_configs.configs[i];
> > +	for (unsigned int i = 0; i < state->stream_configs.num_configs; i += 2) {
> > +		const struct v4l2_subdev_stream_config *src_cfg =
> > +			&state->stream_configs.configs[i + 1];
> 
> You could start at i = 1 and avoid the + 1 here.
> 
> Unless I missed it, I don't think we document anywhere that we store two
> values per route in the array, and that, if multiple routes have the
> same sink or source stream, multiple entries will exist for the same
> stream. I'd like to see this clearly explained somewhere, instead of
> relying on an implemetation detail only known by few people.

I'll add this in a separate patch.

-- 
Regards,

Sakari Ailus

Re: [PATCH 10/13] media: v4l2-subdev: Add debug prints to v4l2_subdev_collect_streams()

[Sakari Ailus]

Hi Laurent,

On Fri, Jun 20, 2025 at 01:23:23AM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.
> 
> On Thu, Jun 19, 2025 at 11:15:43AM +0300, Sakari Ailus wrote:
> > Print streams found by v4l2_subdev_collect_streams() at debug level.
> 
> Explaining why you found this useful could help.
> 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-subdev.c | 5 +++++
> >  1 file changed, 5 insertions(+)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > index 932fca795d4a..c549a462dac7 100644
> > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > @@ -2219,6 +2219,8 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
> >  		*found_streams = BIT_ULL(0);
> >  		*enabled_streams =
> >  			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
> > +		dev_dbg(sd->dev,
> > +			"collect_streams: sub-device does not support streams\n");
> 
> You print the subdev name below, I would print it here too. Maybe
> 
> 			"collect_streams: '%s':%u: streams not supported\n",
> 			sd->entity.name, pad);

Makes sense.

> 
> >  		return;
> >  	}
> >  
> > @@ -2237,6 +2239,9 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
> >  		if (src_cfg->enabled)
> >  			*enabled_streams |= BIT_ULL(src_cfg->stream);
> >  	}
> 
> Please add a blank line here.
> 
> > +	dev_dbg(sd->dev,
> > +		"collect_streams: %s:%u found %#llx enabled %#llx\n",
> 
> 		"collect_streams: '%s':%u: found %#llx, enabled: %#llx\n",
> 
> or possibly
> 
> 		"%s: '%s':%u: found %#llx, enabled: %#llx\n", __func__,
> 
> (same above).

I prefer collect_streams: only as it's shorter, there's no possibility for
confusion in any case.

> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thank you!

> 
> > +		sd->entity.name, pad, *found_streams, *enabled_streams);
> >  }
> >  
> >  static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
> 

-- 
Regards,

Sakari Ailus

Re: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()

[Sakari Ailus]

Hi Jacopo,

Thanks for the review.

On Fri, Jun 20, 2025 at 10:14:58AM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Thu, Jun 19, 2025 at 11:15:44AM +0300, Sakari Ailus wrote:
> > v4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(),
> > with the difference that it's more flexible: it can look up only active
> > routes and can find multiple routes, too.
> >
> > v4l2_subdev_find_route() is intended to replace
> > v4l2_subdev_routing_find_opposite_end().
> 
> To me this feels like v4l2_subdev_find_route() could be used to
> implement more helpers like v4l2_subdev_routing_find_opposite_end()
> for drivers instead of going the other way around.
> 
> let's see what the use cases are
> 
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++---------
> >  include/media/v4l2-subdev.h           | 19 +++++++++
> >  2 files changed, 56 insertions(+), 19 deletions(-)
> >
> > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > index c549a462dac7..13d6e96daf3a 100644
> > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > @@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> >  }
> >  EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
> >
> > -int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
> > -					  u32 pad, u32 stream, u32 *other_pad,
> > -					  u32 *other_stream)
> > +struct v4l2_subdev_route *
> > +v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
> > +		       u32 pad, u32 stream, bool active, unsigned int index)
> >  {
> >  	unsigned int i;
> >
> >  	for (i = 0; i < routing->num_routes; ++i) {
> >  		struct v4l2_subdev_route *route = &routing->routes[i];
> >
> > -		if (route->source_pad == pad &&
> > -		    route->source_stream == stream) {
> > -			if (other_pad)
> > -				*other_pad = route->sink_pad;
> > -			if (other_stream)
> > -				*other_stream = route->sink_stream;
> > -			return 0;
> > -		}
> > +		if (active && !(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> > +			continue;
> 
> I know currently v4l2_subdev_routing_find_opposite_end() does return
> any route that matches the provided 'pad' and 'stream' included
> non-active ones, but I wonder if this is desirable. What is the use
> case for enumerating a non-active route between two pads ?

Good question. v4l2_subdev_routing_find_opposite_end() nevertheless returns
them. And the caller won't get the route for checking the state either.

> 
> (it is also my impression that all drivers that use
> v4l2_subdev_routing_find_opposite_end() assume the route is active)
> 
> Also I wonder if the usage of V4L2_SUBDEV_ROUTE_FL_ACTIVE is clearly
> defined, or, in other words, what is the use case for userspace to
> create non-active routes, given that any new VIDIOC_SUBDEV_S_ROUTING
> will anyway re-create the routing table (that's a different question,
> on the ioctl definition and not on this change though)

I think it is. Please review the UAPI documentation in the metadata series.
:-)

> 
> >
> > -		if (route->sink_pad == pad && route->sink_stream == stream) {
> > -			if (other_pad)
> > -				*other_pad = route->source_pad;
> > -			if (other_stream)
> > -				*other_stream = route->source_stream;
> > -			return 0;
> > -		}
> > +		if ((route->source_pad != pad ||
> > +		     route->source_stream != stream) &&
> > +		    (route->sink_pad != pad || route->sink_stream != stream))
> > +			continue;
> > +
> > +		if (index--)
> > +			continue;
> > +
> > +		return route;
> >  	}
> >
> > -	return -EINVAL;
> > +	return ERR_PTR(-ENOENT);
> > +}
> > +EXPORT_SYMBOL_GPL(v4l2_subdev_find_route);
> > +
> > +int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
> > +					  u32 pad, u32 stream, u32 *other_pad,
> > +					  u32 *other_stream)
> > +{
> > +	struct v4l2_subdev_route *route;
> > +
> > +	route = v4l2_subdev_find_route(routing, pad, stream, false, 0);
> > +	if (IS_ERR(route))
> > +		return PTR_ERR(route);
> > +
> > +	bool is_source = route->source_pad == pad;
> > +
> > +	if (other_pad)
> > +		*other_pad = is_source ? route->sink_pad : route->source_pad;
> > +	if (other_stream)
> > +		*other_stream = is_source ?
> > +			route->sink_stream : route->source_stream;
> > +
> > +	return 0;
> >  }
> >  EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
> >
> > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> > index deab128a4779..9ed8600ba3d4 100644
> > --- a/include/media/v4l2-subdev.h
> > +++ b/include/media/v4l2-subdev.h
> > @@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> >  				     const struct v4l2_subdev_krouting *routing,
> >  				     const struct v4l2_mbus_framefmt *fmt);
> >
> > +/**
> > + * v4l2_subdev_find_route() - Find routes from a (pad, stream) pair
> 
> from or for ?

From or to. I'll fix this in the next version.

> 
> > + * @routing: routing used to find the opposite side
> 
> I would not say "opposite side" but rather
> 
>       @routing: routing table used to enumerate routes

How about simply "the routing table"?

> 
> > + * @pad: pad id
> > + * @stream: stream id
> > + * @active: set to true for looking up only active routes
> > + * @index: for accessing more than one route from the pad
> 
> I understand this but maybe
> 
>      @index: route index for enumerating multiple routes
> ?

Sounds good.

> 
> > + *
> > + * Find a route from the routing table where one end has (pad, stream) pair
> > + * matching @pad and @stream.
> 
>     * If multiple routes in @routing match @pad and @stream, return
>     * the @index one.
>     *
>     * Set @active to true to only enumerate active routes.
> 
> > + *
> > + * Returns the route on success or -ENOENT if no matching route is found.
> 
> I see other functions documentation using
> 
>     * Return:
> 
> is this a kernel-doc thing ?

Yes, makes sense.

> 
> > + */
> > +struct v4l2_subdev_route *
> > +v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
> > +		       u32 pad, u32 stream, bool active, unsigned int index);
> > +
> >  /**
> >   * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
> >   * @routing: routing used to find the opposite side
> > @@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> >   * @other_pad: pointer used to return the opposite pad
> >   * @other_stream: pointer used to return the opposite stream
> >   *
> > + * Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end().
> > + *
> 
> As said, I'm not sure if that's preferred or we should rather create
> more helpers using v4l2_subdev_find_route() internally. Time will tell
> I guess ?

I agree.

The benefit of the older function was that it returns information that
doesn't need a lock for accessing it.

-- 
Regards,

Sakari Ailus

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

[Tomi Valkeinen]

Hi,

On 19/06/2025 11:15, Sakari Ailus wrote:
> Streams are stored in sink-source pairs in sub-device state. When a stream
> was marked enabled (or disabled), only the state of one end of the stream
> was modified, leaving the stream in an incoherent state. Mark both ends of
> the stream enabled (or disabled).
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 60b8febd3339..5afdd9d548b5 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>  	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
>  		struct v4l2_subdev_stream_config *cfg =
>  			&state->stream_configs.configs[i];
> +		struct v4l2_subdev_stream_config *cfg2 =
> +			&state->stream_configs.configs[i ^ 1U];

I don't think this is correct. There's no specific rule that the stream
configs would be in a particular order. Or that there even are a pair.

You should look for the other end of the stream from the routing table.

 Tomi

>  
>  		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
> -			cfg->enabled = enabled;
> +			cfg->enabled = cfg2->enabled = enabled;
>  	}
>  }
>  

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

[Tomi Valkeinen]

On 19/06/2025 21:34, Sakari Ailus wrote:
> Hi Laurent,
> 
> On Thu, Jun 19, 2025 at 07:56:04PM +0300, Laurent Pinchart wrote:
>> Hi Sakari,
>>
>> Thank you for the patch.
> 
> Thanks for the review.
> 
>>
>> On Thu, Jun 19, 2025 at 11:15:39AM +0300, Sakari Ailus wrote:
>>> Streams are stored in sink-source pairs in sub-device state.
>>
>> Is it really this simple ? Don't we support for instance stream merging
>> where two streams on possibly different sink pads are routed to a single
>> stream on a source pad ?
> 
> As far as I'm concerned, yes. Each route has a sink and a source pad, you
> may have multiple routes from a pad or to a pad, but these are all separate
> routes. The UAPI as well as v4l2-subdev.c is aligned with this and I
> wouldn't expect it to be otherwise.

But with stream merging/splitting we'll have one stream config on one
pad, and two stream configs on the other. And thus two routes from that
one config.

That said, if I recall right, I don't think merging/splitting has ever
been really explored...

 Tomi

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

[Laurent Pinchart]

On Thu, Jun 26, 2025 at 06:22:53PM +0300, Tomi Valkeinen wrote:
> On 19/06/2025 21:34, Sakari Ailus wrote:
> > On Thu, Jun 19, 2025 at 07:56:04PM +0300, Laurent Pinchart wrote:
> >> On Thu, Jun 19, 2025 at 11:15:39AM +0300, Sakari Ailus wrote:
> >>> Streams are stored in sink-source pairs in sub-device state.
> >>
> >> Is it really this simple ? Don't we support for instance stream merging
> >> where two streams on possibly different sink pads are routed to a single
> >> stream on a source pad ?
> > 
> > As far as I'm concerned, yes. Each route has a sink and a source pad, you
> > may have multiple routes from a pad or to a pad, but these are all separate
> > routes. The UAPI as well as v4l2-subdev.c is aligned with this and I
> > wouldn't expect it to be otherwise.
> 
> But with stream merging/splitting we'll have one stream config on one
> pad, and two stream configs on the other. And thus two routes from that
> one config.

With the current code, we will have duplicated stream config entries for
the same stream in that case. Only the first one will be seen by the
v4l2_subdev_state_get_format() function, but both will exist.

> That said, if I recall right, I don't think merging/splitting has ever
> been really explored...

-- 
Regards,

Laurent Pinchart

Re: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()

[Laurent Pinchart]

On Wed, Jun 25, 2025 at 04:53:54PM +0000, Sakari Ailus wrote:
> On Fri, Jun 20, 2025 at 10:14:58AM +0200, Jacopo Mondi wrote:
> > On Thu, Jun 19, 2025 at 11:15:44AM +0300, Sakari Ailus wrote:
> > > v4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(),
> > > with the difference that it's more flexible: it can look up only active
> > > routes and can find multiple routes, too.
> > >
> > > v4l2_subdev_find_route() is intended to replace
> > > v4l2_subdev_routing_find_opposite_end().
> > 
> > To me this feels like v4l2_subdev_find_route() could be used to
> > implement more helpers like v4l2_subdev_routing_find_opposite_end()
> > for drivers instead of going the other way around.
> > 
> > let's see what the use cases are
> > 
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++---------
> > >  include/media/v4l2-subdev.h           | 19 +++++++++
> > >  2 files changed, 56 insertions(+), 19 deletions(-)
> > >
> > > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > > index c549a462dac7..13d6e96daf3a 100644
> > > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > > @@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> > >  }
> > >  EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
> > >
> > > -int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
> > > -					  u32 pad, u32 stream, u32 *other_pad,
> > > -					  u32 *other_stream)
> > > +struct v4l2_subdev_route *
> > > +v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
> > > +		       u32 pad, u32 stream, bool active, unsigned int index)
> > >  {
> > >  	unsigned int i;
> > >
> > >  	for (i = 0; i < routing->num_routes; ++i) {
> > >  		struct v4l2_subdev_route *route = &routing->routes[i];
> > >
> > > -		if (route->source_pad == pad &&
> > > -		    route->source_stream == stream) {
> > > -			if (other_pad)
> > > -				*other_pad = route->sink_pad;
> > > -			if (other_stream)
> > > -				*other_stream = route->sink_stream;
> > > -			return 0;
> > > -		}
> > > +		if (active && !(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> > > +			continue;
> > 
> > I know currently v4l2_subdev_routing_find_opposite_end() does return
> > any route that matches the provided 'pad' and 'stream' included
> > non-active ones, but I wonder if this is desirable. What is the use
> > case for enumerating a non-active route between two pads ?
> 
> Good question. v4l2_subdev_routing_find_opposite_end() nevertheless returns
> them. And the caller won't get the route for checking the state either.

I'm tempted to check the callers of the function and change the
behaviour to only return active routes.

> > (it is also my impression that all drivers that use
> > v4l2_subdev_routing_find_opposite_end() assume the route is active)
> > 
> > Also I wonder if the usage of V4L2_SUBDEV_ROUTE_FL_ACTIVE is clearly
> > defined, or, in other words, what is the use case for userspace to
> > create non-active routes, given that any new VIDIOC_SUBDEV_S_ROUTING
> > will anyway re-create the routing table (that's a different question,
> > on the ioctl definition and not on this change though)
> 
> I think it is. Please review the UAPI documentation in the metadata series.
> :-)

Section "Device types and routing setup" in
Documentation/userspace-api/media/v4l/dev-subdev.rst is the best
documentation we have at the moment.

We essentially have two models for internal routing. In the first model,
which I'll nickname the crossbar switch model, a large number of routes
are possible (up to any input to any output types of scenarios). In this
case, routes are created by userspace, and all routes in the routing
table are expected to be active. Any inactive route provided by
userspace would be dropped by the driver and not be included in the
routing table.

The second model covers devices such as camera sensors, where a small
fixed set of routes are hardcoded. The routing table is fixed, an some
routes (the ones not marked with the IMMUTABLE flag) can be
enabled/disabled by userspace using the V4L2_SUBDEV_ROUTE_FL_ACTIVE.
This allows userspace to enumerate the available routes.

Documentation/userspace-api/media/v4l/dev-subdev.rst should document
more clearly that we do not allow any hybrid behaviour at the moment.

> > >
> > > -		if (route->sink_pad == pad && route->sink_stream == stream) {
> > > -			if (other_pad)
> > > -				*other_pad = route->source_pad;
> > > -			if (other_stream)
> > > -				*other_stream = route->source_stream;
> > > -			return 0;
> > > -		}
> > > +		if ((route->source_pad != pad ||
> > > +		     route->source_stream != stream) &&
> > > +		    (route->sink_pad != pad || route->sink_stream != stream))
> > > +			continue;
> > > +
> > > +		if (index--)
> > > +			continue;
> > > +
> > > +		return route;
> > >  	}
> > >
> > > -	return -EINVAL;
> > > +	return ERR_PTR(-ENOENT);
> > > +}
> > > +EXPORT_SYMBOL_GPL(v4l2_subdev_find_route);
> > > +
> > > +int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
> > > +					  u32 pad, u32 stream, u32 *other_pad,
> > > +					  u32 *other_stream)
> > > +{
> > > +	struct v4l2_subdev_route *route;
> > > +
> > > +	route = v4l2_subdev_find_route(routing, pad, stream, false, 0);
> > > +	if (IS_ERR(route))
> > > +		return PTR_ERR(route);
> > > +
> > > +	bool is_source = route->source_pad == pad;
> > > +
> > > +	if (other_pad)
> > > +		*other_pad = is_source ? route->sink_pad : route->source_pad;
> > > +	if (other_stream)
> > > +		*other_stream = is_source ?
> > > +			route->sink_stream : route->source_stream;

Having to do this is_source dance makes v4l2_subdev_find_route()
annoying to use. It may be fine when using the function to implement
other helpers, but I wouldn't like to see this being done in drivers.
Maybe we can avoid exporting v4l2_subdev_find_route() for now to ensure
that, the only user in this series is in v4l2-mc.c.

> > > +
> > > +	return 0;
> > >  }
> > >  EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
> > >
> > > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> > > index deab128a4779..9ed8600ba3d4 100644
> > > --- a/include/media/v4l2-subdev.h
> > > +++ b/include/media/v4l2-subdev.h
> > > @@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> > >  				     const struct v4l2_subdev_krouting *routing,
> > >  				     const struct v4l2_mbus_framefmt *fmt);
> > >
> > > +/**
> > > + * v4l2_subdev_find_route() - Find routes from a (pad, stream) pair
> > 
> > from or for ?
> 
> From or to. I'll fix this in the next version.
> 
> > 
> > > + * @routing: routing used to find the opposite side
> > 
> > I would not say "opposite side" but rather
> > 
> >       @routing: routing table used to enumerate routes
> 
> How about simply "the routing table"?
> 
> > 
> > > + * @pad: pad id
> > > + * @stream: stream id
> > > + * @active: set to true for looking up only active routes
> > > + * @index: for accessing more than one route from the pad
> > 
> > I understand this but maybe
> > 
> >      @index: route index for enumerating multiple routes
> > ?
> 
> Sounds good.

I'm curious to know how this parameter will be used. In the only user
(in patch 12/13), it is hardcoded to 0. I'm not sure indexing routes
will be very useful for drivers.

> > > + *
> > > + * Find a route from the routing table where one end has (pad, stream) pair
> > > + * matching @pad and @stream.
> > 
> >     * If multiple routes in @routing match @pad and @stream, return
> >     * the @index one.
> >     *
> >     * Set @active to true to only enumerate active routes.
> > 
> > > + *
> > > + * Returns the route on success or -ENOENT if no matching route is found.
> > 
> > I see other functions documentation using
> > 
> >     * Return:
> > 
> > is this a kernel-doc thing ?
> 
> Yes, makes sense.
> 
> > > + */
> > > +struct v4l2_subdev_route *
> > > +v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
> > > +		       u32 pad, u32 stream, bool active, unsigned int index);
> > > +
> > >  /**
> > >   * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
> > >   * @routing: routing used to find the opposite side
> > > @@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> > >   * @other_pad: pointer used to return the opposite pad
> > >   * @other_stream: pointer used to return the opposite stream
> > >   *
> > > + * Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end().
> > > + *
> > 
> > As said, I'm not sure if that's preferred or we should rather create
> > more helpers using v4l2_subdev_find_route() internally. Time will tell
> > I guess ?
> 
> I agree.
> 
> The benefit of the older function was that it returns information that
> doesn't need a lock for accessing it.

I'm not sure to follow you here. v4l2_subdev_find_route() has the same
locking requirements as v4l2_subdev_routing_find_opposite_end().

Given that v4l2_subdev_find_route() has a single user, and that the only
difference in that user compared to
v4l2_subdev_routing_find_opposite_end() is that only active routes are
considered, I would prefer modifying
v4l2_subdev_routing_find_opposite_end() to ignore inactive routes after
checking that no caller would break.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Laurent Pinchart]

Hi Sakari,

Thank you for the patch.

On Thu, Jun 19, 2025 at 11:15:45AM +0300, Sakari Ailus wrote:
> v4l2_mc_pipeline_enabled() helps solving a problem known for long but
> lacked any sort of general solution: with multiple streams, when streaming
> is started on video nodes one by one, when should streaming be started in
> the source?
> 
> v4l2_mc_pipeline_enabled() traverses the pipeline towards the source,
> queries the streams generated by the source and traces them back to the
> video nodes.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
>  include/media/v4l2-mc.h           |  44 ++++++
>  2 files changed, 287 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
> index 937d358697e1..1731088ad436 100644
> --- a/drivers/media/v4l2-core/v4l2-mc.c
> +++ b/drivers/media/v4l2-core/v4l2-mc.c
> @@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
> +
> +static int
> +__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
> +			   struct media_pad *src_pad, u64 __src_streams,
> +			   struct media_pad **__sink_pad, u64 *__sink_streams)
> +{
> +	struct v4l2_subdev_route *route;
> +	u64 src_streams = 0, sink_streams = 0;
> +	bool has_sink_pad = false;
> +	unsigned int sink_pad;
> +
> +	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
> +		state->sd->entity.name, src_pad->index, __src_streams);
> +	for_each_active_route(&state->routing, route) {
> +		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
> +			state->sd->entity.name,
> +			route->sink_pad, route->sink_stream, route->source_pad,
> +			route->source_stream, route->flags);
> +		if (route->source_pad != src_pad->index)
> +			continue;
> +
> +		if (!(BIT_ULL(route->source_stream) & __src_streams))
> +			continue;
> +
> +		if (!has_sink_pad) {
> +			has_sink_pad = true;
> +			sink_pad = route->sink_pad;
> +		}
> +
> +		if (route->sink_pad != sink_pad) {
> +			dev_dbg(state->sd->dev,
> +				"sink pads (%u vs. %u) differ\n",
> +				route->sink_pad, sink_pad);
> +			return -EMLINK;
> +		}
> +
> +		sink_streams |= BIT_ULL(route->sink_stream);
> +		src_streams |= BIT_ULL(route->source_stream);
> +	}
> +
> +	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
> +	*__sink_streams = sink_streams;
> +
> +	return 0;
> +}
> +
> +static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
> +				    unsigned int sink_stream,
> +				    bool (*func)(struct video_device *vdev),
> +				    struct media_pad **__sink_pad,
> +				    u64 *__sink_streams)
> +{
> +	struct v4l2_subdev_state *state;
> +	struct v4l2_subdev_route *route;
> +	struct v4l2_subdev *sd;
> +	struct media_pad *source_pad, *tmp_pad;
> +	u32 source_stream;
> +
> +	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
> +		return -ENXIO;
> +
> +	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
> +	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
> +		sd->entity.name);
> +
> +	state = v4l2_subdev_lock_and_get_active_state(sd);
> +	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
> +				       sink_stream, true, 0);
> +	if (IS_ERR(route)) {
> +		v4l2_subdev_unlock_state(state);
> +		dev_dbg(sd->dev,
> +			"path_enabled: can't find opposite route for %s:%u/%u",
> +			sd->entity.name, sink_pad->index, sink_stream);
> +		return 2;
> +	}
> +
> +	source_pad = &sd->entity.pads[route->source_pad];
> +	v4l2_subdev_unlock_state(state);
> +
> +	tmp_pad = sink_pad;
> +	sink_pad = media_pad_remote_pad_unique(source_pad);
> +	if (IS_ERR(sink_pad)) {
> +		dev_dbg(sd->dev,
> +			"path_enabled: can't find remote source for %s:%u\n",
> +			source_pad->entity->name, source_pad->index);
> +		return PTR_ERR(sink_pad);
> +	}
> +
> +	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
> +		struct video_device *vdev;
> +
> +		vdev = media_entity_to_video_device(sink_pad->entity);
> +		if (!vdev)
> +			return -ENXIO;
> +
> +		dev_dbg(vdev->dev_parent,
> +			"path_enabled: found video device %s\n",
> +			vdev->name);
> +
> +		if (!*__sink_pad) {
> +			*__sink_pad = tmp_pad;
> +			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
> +				tmp_pad->index, sink_stream);
> +		} else if (tmp_pad != *__sink_pad) {
> +			dev_dbg(sd->dev,
> +				"path_enabled: pads %s/%u and %s/%u differ\n",
> +				tmp_pad->entity->name, tmp_pad->index,
> +				(*__sink_pad)->entity->name,
> +				(*__sink_pad)->index);
> +			return -EXDEV;
> +		}
> +
> +		*__sink_streams |= BIT_ULL(sink_stream);
> +
> +		return func(vdev);
> +	}
> +
> +	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
> +					__sink_pad, __sink_streams);
> +}
> +
> +static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
> +				      u64 *__streams)
> +{
> +	struct v4l2_mbus_frame_desc desc;
> +	u64 streams = 0;
> +	int ret;
> +
> +	if (!__streams)
> +		return -EINVAL;
> +
> +	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
> +	if (ret == -ENOIOCTLCMD) {
> +		*__streams = 1ULL;
> +		return 0;
> +	}
> +	if (ret)
> +		return ret;
> +
> +	for (unsigned int i = 0; i < desc.num_entries; i++) {
> +		if (streams & BIT_ULL(desc.entry[i].stream))
> +			return -EINVAL;
> +
> +		streams |= BIT_ULL(desc.entry[i].stream);
> +	}
> +
> +	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
> +		*__streams, streams);
> +	if (*__streams & ~streams)
> +		return -EINVAL;
> +
> +	*__streams = streams;
> +
> +	return 0;
> +}
> +
> +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> +			     bool (*func)(struct video_device *vdev),
> +			     struct media_pad **__sink_pad, u64 *__sink_streams)
> +{
> +	u64 sink_streams = 1U;
> +	struct media_pad *src_pad;
> +	u64 src_streams;
> +	struct v4l2_subdev_state *state;
> +	struct media_pad *sink_pad = vdev->entity.pads;
> +	struct v4l2_subdev *sd = NULL;
> +	bool streaming = true;
> +	struct media_pad *tmp_pad;
> +	u64 tmp_streams;
> +	int ret;
> +
> +	if (!__sink_pad)
> +		__sink_pad = &tmp_pad;
> +	if (!__sink_streams)
> +		__sink_streams = &tmp_streams;
> +	*__sink_pad = NULL;
> +	*__sink_streams = 0;
> +
> +	do {
> +		src_pad = media_pad_remote_pad_unique(sink_pad);
> +		if (IS_ERR(src_pad)) {
> +			dev_dbg(sd ? sd->dev : vdev->dev_parent,
> +				"no unique remote pad found from %s:%u\n",
> +				sink_pad->entity->name, sink_pad->index);
> +			return PTR_ERR(src_pad);
> +		}
> +
> +		sd = media_entity_to_v4l2_subdev(src_pad->entity);
> +		if (!sd) {
> +			dev_dbg(sd->dev,
> +				"media entity %s is not a V4L2 sub-device\n",
> +				src_pad->entity->name);
> +			return -ENXIO;
> +		}
> +
> +		/* Source streams match sink. */
> +		src_streams = sink_streams;
> +
> +		state = v4l2_subdev_lock_and_get_active_state(sd);
> +		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
> +						 src_streams, &sink_pad,
> +						 &sink_streams);
> +		v4l2_subdev_unlock_state(state);
> +		if (ret)
> +			return ret;
> +	} while (sink_pad);
> +
> +	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
> +	if (ret)
> +		return ret;
> +
> +	sd = media_entity_to_v4l2_subdev(src_pad->entity);
> +
> +	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
> +		src_pad->index, src_streams);
> +
> +	for (unsigned int i = __ffs(src_streams); src_streams;
> +	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
> +		sink_pad = media_pad_remote_pad_unique(src_pad);
> +		if (IS_ERR(src_pad)) {
> +			dev_dbg(sd->dev,
> +				"no unique remote pad found from %s:%u\n",
> +				sink_pad->entity->name, sink_pad->index);
> +			return PTR_ERR(src_pad);
> +		}
> +
> +		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
> +					       __sink_streams);
> +		if (ret == 2)
> +			continue;
> +		if (ret < 0)
> +			return ret;
> +		if (!ret)
> +			streaming = false;
> +	}
> +
> +	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
> +		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
> +		(*__sink_pad)->index, *__sink_streams);
> +
> +	return streaming;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
> diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
> index 1837c9fd78cf..e72c0f62fa34 100644
> --- a/include/media/v4l2-mc.h
> +++ b/include/media/v4l2-mc.h
> @@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
>  int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
>  			      unsigned int notification);
>  
> +/**
> + * v4l2_mc_pipeline_enabled - Tell when to start streaming
> + * @vdev: The video device
> + * @func: Caller-provided function to tell a video device's streaming state
> + * @__sink_pad: sink pad at the root of the local pipeline
> + * @__sink_streams: streams to start

Any reason for the double underscore ?

> + *
> + * Use to tell whether streaming should start on a video node. @func returns
> + * true if streaming has been started on a given video node. @__sink_pad and
> + * @__sink_streams are filled with pad and streams on the sub-device closest to
> + * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
> + * v4l2_subdev_disable_streams().
> + *
> + * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
> + * a to-do list):
> + * * only unbranched streams can be supported albeit adding support for
> + *   downstream branches would be fairly trivial,

I can't tell from the documentation here what you mean exactly by
"unbranched streams".

> + * * streams within a single source sub-device are considered to start at the
> + *   same time, more control could be added in two ways: 1) for sources to
> + *   determine stream starting, a control could be added to UAPI and 2) sources
> + *   could tell which streams start at the same time using a sub-device
> + *   operation,
> + * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
> + *   be implemented by letting the caller to provide a function to determine
> + *   which streams are of interest and
> + * * routes leading to nowhere are ignored, on some hardware this is a problem,
> + *   but this can also be rather trivially addressed.

I'm afraid this function looks like a hack, to solve a problem that is
not even explicitly described. You don't explain the issue in the cover
letter or in Documentation/, the cover letter merely states that this is
a "partial solution". The documentation of the function doesn't explain
what criteria the decision is based on. We need a proper explanation of
the problem in Documentation/, with a description of the behaviour (or
behaviours) drivers are expected to implement.

Furthermore, on the implementation side, things are fairly inefficient.
We already traverse the whole pipeline in media_pipeline_start(), based
on links and routes, and populate the media_pipeline structure. We
shouldn't do the same here, but instead inspect media_pipeline to
extract the information we need. If you're missing information there,
let's add it.

> + *
> + * Return:
> + * * 0: Success, but don't start streaming yet
> + * * 1: Success, now it's time to start streaming
> + * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
> + * * -ENOTUNIQ: No unique remote pad
> + * * -ENOLINK: No remote pad found
> + * * -ENOENT: Enabled upstream route not found
> + * * -EMLINK: No unique downstream route found
> + * * -EINVAL: Stream could not be followed to source or was not produced by
> + *            the source
> + */
> +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> +			     bool (*func)(struct video_device *vdev),
> +			     struct media_pad **__sink_pad,
> +			     u64 *__sink_streams);
> +
>  #else /* CONFIG_MEDIA_CONTROLLER */
>  
>  static inline int v4l2_mc_create_media_graph(struct media_device *mdev)

-- 
Regards,

Laurent Pinchart

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

[Sakari Ailus]

Moi,

On Thu, Jun 26, 2025 at 06:17:13PM +0300, Tomi Valkeinen wrote:
> Hi,
> 
> On 19/06/2025 11:15, Sakari Ailus wrote:
> > Streams are stored in sink-source pairs in sub-device state. When a stream
> > was marked enabled (or disabled), only the state of one end of the stream
> > was modified, leaving the stream in an incoherent state. Mark both ends of
> > the stream enabled (or disabled).
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > index 60b8febd3339..5afdd9d548b5 100644
> > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > @@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
> >  	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
> >  		struct v4l2_subdev_stream_config *cfg =
> >  			&state->stream_configs.configs[i];
> > +		struct v4l2_subdev_stream_config *cfg2 =
> > +			&state->stream_configs.configs[i ^ 1U];
> 
> I don't think this is correct. There's no specific rule that the stream
> configs would be in a particular order. Or that there even are a pair.
> 
> You should look for the other end of the stream from the routing table.

Currently the code managing the streams stores them in this array in pairs,
the sink followed by the source for each route. As noted in the commit
message, this patch also fixes a bug.

I can change the patch to use routes to find the other end, but over time
we might want to use a different data structure to store this information
(a linked list perhaps). The number of routes and stream configurations is
presumably fairly small in most cases so right now I presume there's no
noteworthy effect on performance.

-- 
Terveisin,

Sakari Ailus

Re: [PATCH 06/13] media: v4l2-subdev: Mark both streams of a route enabled

[Laurent Pinchart]

On Fri, Jun 27, 2025 at 06:09:35AM +0000, Sakari Ailus wrote:
> On Thu, Jun 26, 2025 at 06:17:13PM +0300, Tomi Valkeinen wrote:
> > On 19/06/2025 11:15, Sakari Ailus wrote:
> > > Streams are stored in sink-source pairs in sub-device state. When a stream
> > > was marked enabled (or disabled), only the state of one end of the stream
> > > was modified, leaving the stream in an incoherent state. Mark both ends of
> > > the stream enabled (or disabled).
> > > 
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  drivers/media/v4l2-core/v4l2-subdev.c | 4 +++-
> > >  1 file changed, 3 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > > index 60b8febd3339..5afdd9d548b5 100644
> > > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > > @@ -2254,9 +2254,11 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
> > >  	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
> > >  		struct v4l2_subdev_stream_config *cfg =
> > >  			&state->stream_configs.configs[i];
> > > +		struct v4l2_subdev_stream_config *cfg2 =
> > > +			&state->stream_configs.configs[i ^ 1U];
> > 
> > I don't think this is correct. There's no specific rule that the stream
> > configs would be in a particular order. Or that there even are a pair.
> > 
> > You should look for the other end of the stream from the routing table.
> 
> Currently the code managing the streams stores them in this array in pairs,
> the sink followed by the source for each route. As noted in the commit
> message, this patch also fixes a bug.
> 
> I can change the patch to use routes to find the other end, but over time
> we might want to use a different data structure to store this information
> (a linked list perhaps). The number of routes and stream configurations is
> presumably fairly small in most cases so right now I presume there's no
> noteworthy effect on performance.

At the very least I would like to document somewhere that each route
stores a pair of stream config. We also really, *really*, *REALLY* need
to make sure that as little code as possible depends on this, and
especially that no driver accesses the stream configs directly...
aargghghhhh the ds90ub913 driver does already :'-(

I've submitted [1] to make the v4l2_subdev_stream_config structure
private.

[1] https://lore.kernel.org/r/20250630004602.23075-1-laurent.pinchart@ideasonboard.com

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

[Laurent Pinchart]

On Thu, Jun 19, 2025 at 04:14:43PM +0000, Sakari Ailus wrote:
> On Thu, Jun 19, 2025 at 06:20:33PM +0300, Laurent Pinchart wrote:
> > On Thu, Jun 19, 2025 at 11:15:38AM +0300, Sakari Ailus wrote:
> > > Make media_entity_to_video_device(NULL) return NULL, instead of an invalid
> > > pointer value.
> > > 
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  include/media/v4l2-dev.h | 14 ++++++++++----
> > >  1 file changed, 10 insertions(+), 4 deletions(-)
> > > 
> > > diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
> > > index 1b6222fab24e..069c2f14b473 100644
> > > --- a/include/media/v4l2-dev.h
> > > +++ b/include/media/v4l2-dev.h
> > > @@ -313,10 +313,16 @@ struct video_device {
> > >   * media_entity_to_video_device - Returns a &struct video_device from
> > >   *	the &struct media_entity embedded on it.
> > >   *
> > > - * @__entity: pointer to &struct media_entity
> > > - */
> > > -#define media_entity_to_video_device(__entity) \
> > > -	container_of(__entity, struct video_device, entity)
> > > + * @__entity: pointer to &struct media_entity, may be NULL
> > > + */
> > > +#define media_entity_to_video_device(__entity)				\
> > > +	({								\
> > > +		typeof (__entity) __me_to_vdev_ent = __entity;		\
> > > +									\
> > > +		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
> > > +						struct video_device, entity) : \
> > > +			NULL;						\
> > > +	})
> > 
> > This makes the macro safer, it's a good idea. Wouldn't it be better
> > implemented as a container_of_null() (name to be bikeshedded) though ? I
> > don't think media_entity_to_video_device() is the only macro that could
> > benefit from this. It could even be integrated in container_of(), but I
> > fear that could introduce issues.
> 
> That sounds like a good idea. I'll first see how this would look like with
> container_of_const()...

Thinking some more about this, I think we can move forward without
waiting for container_of_null().

Should we however add a check to ensure the entity is a video device (by
checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if
it's not ? It would make the macro even safer to use. There would be a
small additional runtime cost for call sites that guarantee the entity
is a video device.

I checked the current users of the macro, and the vast majority of them
are in the .link_validate() operation, where they know that the entity
is a video device. There are just a handful of locations where a check
precedes the media_entity_to_video_device() call. So maybe it's not
worth it ?

I also found no caller that checks for entity != NULL before calling the
macro. Is this change actually needed ?

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

[Sakari Ailus]

Hi Laurent,

On Tue, Jul 08, 2025 at 02:56:16PM +0300, Laurent Pinchart wrote:
> On Thu, Jun 19, 2025 at 04:14:43PM +0000, Sakari Ailus wrote:
> > On Thu, Jun 19, 2025 at 06:20:33PM +0300, Laurent Pinchart wrote:
> > > On Thu, Jun 19, 2025 at 11:15:38AM +0300, Sakari Ailus wrote:
> > > > Make media_entity_to_video_device(NULL) return NULL, instead of an invalid
> > > > pointer value.
> > > > 
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > ---
> > > >  include/media/v4l2-dev.h | 14 ++++++++++----
> > > >  1 file changed, 10 insertions(+), 4 deletions(-)
> > > > 
> > > > diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
> > > > index 1b6222fab24e..069c2f14b473 100644
> > > > --- a/include/media/v4l2-dev.h
> > > > +++ b/include/media/v4l2-dev.h
> > > > @@ -313,10 +313,16 @@ struct video_device {
> > > >   * media_entity_to_video_device - Returns a &struct video_device from
> > > >   *	the &struct media_entity embedded on it.
> > > >   *
> > > > - * @__entity: pointer to &struct media_entity
> > > > - */
> > > > -#define media_entity_to_video_device(__entity) \
> > > > -	container_of(__entity, struct video_device, entity)
> > > > + * @__entity: pointer to &struct media_entity, may be NULL
> > > > + */
> > > > +#define media_entity_to_video_device(__entity)				\
> > > > +	({								\
> > > > +		typeof (__entity) __me_to_vdev_ent = __entity;		\
> > > > +									\
> > > > +		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
> > > > +						struct video_device, entity) : \
> > > > +			NULL;						\
> > > > +	})
> > > 
> > > This makes the macro safer, it's a good idea. Wouldn't it be better
> > > implemented as a container_of_null() (name to be bikeshedded) though ? I
> > > don't think media_entity_to_video_device() is the only macro that could
> > > benefit from this. It could even be integrated in container_of(), but I
> > > fear that could introduce issues.
> > 
> > That sounds like a good idea. I'll first see how this would look like with
> > container_of_const()...
> 
> Thinking some more about this, I think we can move forward without
> waiting for container_of_null().
> 
> Should we however add a check to ensure the entity is a video device (by
> checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if
> it's not ? It would make the macro even safer to use. There would be a
> small additional runtime cost for call sites that guarantee the entity
> is a video device.
> 
> I checked the current users of the macro, and the vast majority of them
> are in the .link_validate() operation, where they know that the entity
> is a video device. There are just a handful of locations where a check
> precedes the media_entity_to_video_device() call. So maybe it's not
> worth it ?
> 
> I also found no caller that checks for entity != NULL before calling the
> macro. Is this change actually needed ?

We have a similar check in media_entity_to_v4l2_subdev() macro. In the
patches for streaming control there's a need for an explicit check without
this patch. That's of course fine as well if we decide so.

-- 
Regards,

Sakari Ailus

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

[Laurent Pinchart]

On Tue, Jul 08, 2025 at 12:02:29PM +0000, Sakari Ailus wrote:
> On Tue, Jul 08, 2025 at 02:56:16PM +0300, Laurent Pinchart wrote:
> > On Thu, Jun 19, 2025 at 04:14:43PM +0000, Sakari Ailus wrote:
> > > On Thu, Jun 19, 2025 at 06:20:33PM +0300, Laurent Pinchart wrote:
> > > > On Thu, Jun 19, 2025 at 11:15:38AM +0300, Sakari Ailus wrote:
> > > > > Make media_entity_to_video_device(NULL) return NULL, instead of an invalid
> > > > > pointer value.
> > > > > 
> > > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > ---
> > > > >  include/media/v4l2-dev.h | 14 ++++++++++----
> > > > >  1 file changed, 10 insertions(+), 4 deletions(-)
> > > > > 
> > > > > diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
> > > > > index 1b6222fab24e..069c2f14b473 100644
> > > > > --- a/include/media/v4l2-dev.h
> > > > > +++ b/include/media/v4l2-dev.h
> > > > > @@ -313,10 +313,16 @@ struct video_device {
> > > > >   * media_entity_to_video_device - Returns a &struct video_device from
> > > > >   *	the &struct media_entity embedded on it.
> > > > >   *
> > > > > - * @__entity: pointer to &struct media_entity
> > > > > - */
> > > > > -#define media_entity_to_video_device(__entity) \
> > > > > -	container_of(__entity, struct video_device, entity)
> > > > > + * @__entity: pointer to &struct media_entity, may be NULL
> > > > > + */
> > > > > +#define media_entity_to_video_device(__entity)				\
> > > > > +	({								\
> > > > > +		typeof (__entity) __me_to_vdev_ent = __entity;		\

This should be __me_vdev_ent to align the naming with
media_entity_to_v4l2_subdev().

> > > > > +									\
> > > > > +		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
> > > > > +						struct video_device, entity) : \

And here you should write

	__me_to_vdev_ent ?						   \
		container_of(__me_vdev_ent, struct video_device, entity) : \
		NULL;							   \

> > > > > +			NULL;						\
> > > > > +	})
> > > > 
> > > > This makes the macro safer, it's a good idea. Wouldn't it be better
> > > > implemented as a container_of_null() (name to be bikeshedded) though ? I
> > > > don't think media_entity_to_video_device() is the only macro that could
> > > > benefit from this. It could even be integrated in container_of(), but I
> > > > fear that could introduce issues.
> > > 
> > > That sounds like a good idea. I'll first see how this would look like with
> > > container_of_const()...
> > 
> > Thinking some more about this, I think we can move forward without
> > waiting for container_of_null().
> > 
> > Should we however add a check to ensure the entity is a video device (by
> > checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if
> > it's not ? It would make the macro even safer to use. There would be a
> > small additional runtime cost for call sites that guarantee the entity
> > is a video device.
> > 
> > I checked the current users of the macro, and the vast majority of them
> > are in the .link_validate() operation, where they know that the entity
> > is a video device. There are just a handful of locations where a check
> > precedes the media_entity_to_video_device() call. So maybe it's not
> > worth it ?
> > 
> > I also found no caller that checks for entity != NULL before calling the
> > macro. Is this change actually needed ?
> 
> We have a similar check in media_entity_to_v4l2_subdev() macro.

That's a good point. Different behaviours for the two macros would be
confusing I suppose, so

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

I'm tempted to convert the macro to an inline function, but that would
make it more difficult to switch to container_of_const().

> In the
> patches for streaming control there's a need for an explicit check without
> this patch. That's of course fine as well if we decide so.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

[Sakari Ailus]

Hi Laurent,

On Tue, Jul 08, 2025 at 07:17:47PM +0300, Laurent Pinchart wrote:
> On Tue, Jul 08, 2025 at 12:02:29PM +0000, Sakari Ailus wrote:
> > On Tue, Jul 08, 2025 at 02:56:16PM +0300, Laurent Pinchart wrote:
> > > On Thu, Jun 19, 2025 at 04:14:43PM +0000, Sakari Ailus wrote:
> > > > On Thu, Jun 19, 2025 at 06:20:33PM +0300, Laurent Pinchart wrote:
> > > > > On Thu, Jun 19, 2025 at 11:15:38AM +0300, Sakari Ailus wrote:
> > > > > > Make media_entity_to_video_device(NULL) return NULL, instead of an invalid
> > > > > > pointer value.
> > > > > > 
> > > > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > > ---
> > > > > >  include/media/v4l2-dev.h | 14 ++++++++++----
> > > > > >  1 file changed, 10 insertions(+), 4 deletions(-)
> > > > > > 
> > > > > > diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
> > > > > > index 1b6222fab24e..069c2f14b473 100644
> > > > > > --- a/include/media/v4l2-dev.h
> > > > > > +++ b/include/media/v4l2-dev.h
> > > > > > @@ -313,10 +313,16 @@ struct video_device {
> > > > > >   * media_entity_to_video_device - Returns a &struct video_device from
> > > > > >   *	the &struct media_entity embedded on it.
> > > > > >   *
> > > > > > - * @__entity: pointer to &struct media_entity
> > > > > > - */
> > > > > > -#define media_entity_to_video_device(__entity) \
> > > > > > -	container_of(__entity, struct video_device, entity)
> > > > > > + * @__entity: pointer to &struct media_entity, may be NULL
> > > > > > + */
> > > > > > +#define media_entity_to_video_device(__entity)				\
> > > > > > +	({								\
> > > > > > +		typeof (__entity) __me_to_vdev_ent = __entity;		\
> 
> This should be __me_vdev_ent to align the naming with
> media_entity_to_v4l2_subdev().
> 
> > > > > > +									\
> > > > > > +		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
> > > > > > +						struct video_device, entity) : \
> 
> And here you should write
> 
> 	__me_to_vdev_ent ?						   \
> 		container_of(__me_vdev_ent, struct video_device, entity) : \
> 		NULL;							   \

I'm fine wrap after '?' as well, but another wrap is needed due to
indentation in any case.

> 
> > > > > > +			NULL;						\
> > > > > > +	})
> > > > > 
> > > > > This makes the macro safer, it's a good idea. Wouldn't it be better
> > > > > implemented as a container_of_null() (name to be bikeshedded) though ? I
> > > > > don't think media_entity_to_video_device() is the only macro that could
> > > > > benefit from this. It could even be integrated in container_of(), but I
> > > > > fear that could introduce issues.
> > > > 
> > > > That sounds like a good idea. I'll first see how this would look like with
> > > > container_of_const()...
> > > 
> > > Thinking some more about this, I think we can move forward without
> > > waiting for container_of_null().
> > > 
> > > Should we however add a check to ensure the entity is a video device (by
> > > checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if
> > > it's not ? It would make the macro even safer to use. There would be a
> > > small additional runtime cost for call sites that guarantee the entity
> > > is a video device.
> > > 
> > > I checked the current users of the macro, and the vast majority of them
> > > are in the .link_validate() operation, where they know that the entity
> > > is a video device. There are just a handful of locations where a check
> > > precedes the media_entity_to_video_device() call. So maybe it's not
> > > worth it ?
> > > 
> > > I also found no caller that checks for entity != NULL before calling the
> > > macro. Is this change actually needed ?
> > 
> > We have a similar check in media_entity_to_v4l2_subdev() macro.
> 
> That's a good point. Different behaviours for the two macros would be
> confusing I suppose, so
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Thank you.

> 
> I'm tempted to convert the macro to an inline function, but that would
> make it more difficult to switch to container_of_const().

Let's not do that. Eventually container_of() should become what
container_of_const() is at the moment.

But once this is in, I can introduce container_of_null() which we could
later use. I wonder how it will look like.

-- 
Regards,

Sakari Ailus

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

[Laurent Pinchart]

On Wed, Jul 09, 2025 at 08:03:07PM +0000, Sakari Ailus wrote:
> Hi Laurent,
> 
> On Tue, Jul 08, 2025 at 07:17:47PM +0300, Laurent Pinchart wrote:
> > On Tue, Jul 08, 2025 at 12:02:29PM +0000, Sakari Ailus wrote:
> > > On Tue, Jul 08, 2025 at 02:56:16PM +0300, Laurent Pinchart wrote:
> > > > On Thu, Jun 19, 2025 at 04:14:43PM +0000, Sakari Ailus wrote:
> > > > > On Thu, Jun 19, 2025 at 06:20:33PM +0300, Laurent Pinchart wrote:
> > > > > > On Thu, Jun 19, 2025 at 11:15:38AM +0300, Sakari Ailus wrote:
> > > > > > > Make media_entity_to_video_device(NULL) return NULL, instead of an invalid
> > > > > > > pointer value.
> > > > > > > 
> > > > > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > > > ---
> > > > > > >  include/media/v4l2-dev.h | 14 ++++++++++----
> > > > > > >  1 file changed, 10 insertions(+), 4 deletions(-)
> > > > > > > 
> > > > > > > diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
> > > > > > > index 1b6222fab24e..069c2f14b473 100644
> > > > > > > --- a/include/media/v4l2-dev.h
> > > > > > > +++ b/include/media/v4l2-dev.h
> > > > > > > @@ -313,10 +313,16 @@ struct video_device {
> > > > > > >   * media_entity_to_video_device - Returns a &struct video_device from
> > > > > > >   *	the &struct media_entity embedded on it.
> > > > > > >   *
> > > > > > > - * @__entity: pointer to &struct media_entity
> > > > > > > - */
> > > > > > > -#define media_entity_to_video_device(__entity) \
> > > > > > > -	container_of(__entity, struct video_device, entity)
> > > > > > > + * @__entity: pointer to &struct media_entity, may be NULL
> > > > > > > + */
> > > > > > > +#define media_entity_to_video_device(__entity)				\
> > > > > > > +	({								\
> > > > > > > +		typeof (__entity) __me_to_vdev_ent = __entity;		\
> > 
> > This should be __me_vdev_ent to align the naming with
> > media_entity_to_v4l2_subdev().
> > 
> > > > > > > +									\
> > > > > > > +		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
> > > > > > > +						struct video_device, entity) : \
> > 
> > And here you should write
> > 
> > 	__me_to_vdev_ent ?						   \
> > 		container_of(__me_vdev_ent, struct video_device, entity) : \
> > 		NULL;							   \
> 
> I'm fine wrap after '?' as well, but another wrap is needed due to
> indentation in any case.

I didn't make it clear, my comment was about replicating the style of
media_entity_to_v4l2_subdev().

> > > > > > > +			NULL;						\
> > > > > > > +	})
> > > > > > 
> > > > > > This makes the macro safer, it's a good idea. Wouldn't it be better
> > > > > > implemented as a container_of_null() (name to be bikeshedded) though ? I
> > > > > > don't think media_entity_to_video_device() is the only macro that could
> > > > > > benefit from this. It could even be integrated in container_of(), but I
> > > > > > fear that could introduce issues.
> > > > > 
> > > > > That sounds like a good idea. I'll first see how this would look like with
> > > > > container_of_const()...
> > > > 
> > > > Thinking some more about this, I think we can move forward without
> > > > waiting for container_of_null().
> > > > 
> > > > Should we however add a check to ensure the entity is a video device (by
> > > > checking that the function is MEDIA_ENT_F_IO_V4L), and return NULL if
> > > > it's not ? It would make the macro even safer to use. There would be a
> > > > small additional runtime cost for call sites that guarantee the entity
> > > > is a video device.
> > > > 
> > > > I checked the current users of the macro, and the vast majority of them
> > > > are in the .link_validate() operation, where they know that the entity
> > > > is a video device. There are just a handful of locations where a check
> > > > precedes the media_entity_to_video_device() call. So maybe it's not
> > > > worth it ?
> > > > 
> > > > I also found no caller that checks for entity != NULL before calling the
> > > > macro. Is this change actually needed ?
> > > 
> > > We have a similar check in media_entity_to_v4l2_subdev() macro.
> > 
> > That's a good point. Different behaviours for the two macros would be
> > confusing I suppose, so
> > 
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> Thank you.
> 
> > I'm tempted to convert the macro to an inline function, but that would
> > make it more difficult to switch to container_of_const().
> 
> Let's not do that. Eventually container_of() should become what
> container_of_const() is at the moment.
> 
> But once this is in, I can introduce container_of_null() which we could
> later use. I wonder how it will look like.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 05/13] media: v4l: Make media_entity_to_video_device() NULL-safe

[Sakari Ailus]

On Wed, Jul 09, 2025 at 11:54:49PM +0300, Laurent Pinchart wrote:
> On Wed, Jul 09, 2025 at 08:03:07PM +0000, Sakari Ailus wrote:
> > Hi Laurent,
> > 
> > On Tue, Jul 08, 2025 at 07:17:47PM +0300, Laurent Pinchart wrote:
> > > On Tue, Jul 08, 2025 at 12:02:29PM +0000, Sakari Ailus wrote:
> > > > On Tue, Jul 08, 2025 at 02:56:16PM +0300, Laurent Pinchart wrote:
> > > > > On Thu, Jun 19, 2025 at 04:14:43PM +0000, Sakari Ailus wrote:
> > > > > > On Thu, Jun 19, 2025 at 06:20:33PM +0300, Laurent Pinchart wrote:
> > > > > > > On Thu, Jun 19, 2025 at 11:15:38AM +0300, Sakari Ailus wrote:
> > > > > > > > Make media_entity_to_video_device(NULL) return NULL, instead of an invalid
> > > > > > > > pointer value.
> > > > > > > > 
> > > > > > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > > > > > ---
> > > > > > > >  include/media/v4l2-dev.h | 14 ++++++++++----
> > > > > > > >  1 file changed, 10 insertions(+), 4 deletions(-)
> > > > > > > > 
> > > > > > > > diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
> > > > > > > > index 1b6222fab24e..069c2f14b473 100644
> > > > > > > > --- a/include/media/v4l2-dev.h
> > > > > > > > +++ b/include/media/v4l2-dev.h
> > > > > > > > @@ -313,10 +313,16 @@ struct video_device {
> > > > > > > >   * media_entity_to_video_device - Returns a &struct video_device from
> > > > > > > >   *	the &struct media_entity embedded on it.
> > > > > > > >   *
> > > > > > > > - * @__entity: pointer to &struct media_entity
> > > > > > > > - */
> > > > > > > > -#define media_entity_to_video_device(__entity) \
> > > > > > > > -	container_of(__entity, struct video_device, entity)
> > > > > > > > + * @__entity: pointer to &struct media_entity, may be NULL
> > > > > > > > + */
> > > > > > > > +#define media_entity_to_video_device(__entity)				\
> > > > > > > > +	({								\
> > > > > > > > +		typeof (__entity) __me_to_vdev_ent = __entity;		\
> > > 
> > > This should be __me_vdev_ent to align the naming with
> > > media_entity_to_v4l2_subdev().
> > > 
> > > > > > > > +									\
> > > > > > > > +		__me_to_vdev_ent ? container_of(__me_to_vdev_ent,	\
> > > > > > > > +						struct video_device, entity) : \
> > > 
> > > And here you should write
> > > 
> > > 	__me_to_vdev_ent ?						   \
> > > 		container_of(__me_vdev_ent, struct video_device, entity) : \
> > > 		NULL;							   \
> > 
> > I'm fine wrap after '?' as well, but another wrap is needed due to
> > indentation in any case.
> 
> I didn't make it clear, my comment was about replicating the style of
> media_entity_to_v4l2_subdev().

Ok, I'll use that in v3.

-- 
Sakari Ailus

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Sakari Ailus]

Hi Jacopo,

On Fri, Jun 20, 2025 at 10:53:13AM +0200, Jacopo Mondi wrote:
> Hi Sakari
> 
> On Thu, Jun 19, 2025 at 11:15:45AM +0300, Sakari Ailus wrote:
> > v4l2_mc_pipeline_enabled() helps solving a problem known for long but
> > lacked any sort of general solution: with multiple streams, when streaming
> > is started on video nodes one by one, when should streaming be started in
> > the source?
> 
> I tried quite some time to understand this, but if I'm not mistaken,
> a stream-aware subdev, which links to vdev, will always "demux"
> streams to different pads and will connect to the vdev from there
> 
> 
>      Source
>      subdev
>  +-----------------+
>  |                (1/0) ------> vdev0
>  |                 |
> (0)[1,2,3]        (2/0  ------> vdev1
>  |                 |
>  |                (3/0) ------> vdev2
>  +-----------------+
> 
> With
> 
> (0) multiplexed sink pad with 3 streams
> (1) (2) and (3) source pad with a single stream
> 
> Can't we relay on the media-link state between the source pads and the
> video devices with something like what Dan has proposed here ?
> https://patchwork.linuxtv.org/project/linux-media/patch/20250519140403.443915-2-dan.scally@ideasonboard.com/

This isn't entirely the same thing: pipeline is specific to the pad but not
streams, ad here we're interested in streams. Two streams may start at
different points of time even if both are part of the same pipeline.

We definitely should have just one way to figure this out.

I'll send v2 with comments and before that also see if (or how) we could
get rid of the callback.

Cc Dan.

> 
> What am I missing ?
> 
> >
> > v4l2_mc_pipeline_enabled() traverses the pipeline towards the source,
> > queries the streams generated by the source and traces them back to the
> > video nodes.
> >
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
> >  include/media/v4l2-mc.h           |  44 ++++++
> >  2 files changed, 287 insertions(+)
> >
> > diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
> > index 937d358697e1..1731088ad436 100644
> > --- a/drivers/media/v4l2-core/v4l2-mc.c
> > +++ b/drivers/media/v4l2-core/v4l2-mc.c
> > @@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> >  	return ret;
> >  }
> >  EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
> > +
> > +static int
> > +__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
> > +			   struct media_pad *src_pad, u64 __src_streams,
> > +			   struct media_pad **__sink_pad, u64 *__sink_streams)
> > +{
> > +	struct v4l2_subdev_route *route;
> > +	u64 src_streams = 0, sink_streams = 0;
> > +	bool has_sink_pad = false;
> > +	unsigned int sink_pad;
> > +
> > +	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
> > +		state->sd->entity.name, src_pad->index, __src_streams);
> > +	for_each_active_route(&state->routing, route) {
> > +		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
> > +			state->sd->entity.name,
> > +			route->sink_pad, route->sink_stream, route->source_pad,
> > +			route->source_stream, route->flags);
> > +		if (route->source_pad != src_pad->index)
> > +			continue;
> > +
> > +		if (!(BIT_ULL(route->source_stream) & __src_streams))
> > +			continue;
> > +
> > +		if (!has_sink_pad) {
> > +			has_sink_pad = true;
> > +			sink_pad = route->sink_pad;
> > +		}
> > +
> > +		if (route->sink_pad != sink_pad) {
> > +			dev_dbg(state->sd->dev,
> > +				"sink pads (%u vs. %u) differ\n",
> > +				route->sink_pad, sink_pad);
> > +			return -EMLINK;
> > +		}
> > +
> > +		sink_streams |= BIT_ULL(route->sink_stream);
> > +		src_streams |= BIT_ULL(route->source_stream);
> > +	}
> > +
> > +	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
> > +	*__sink_streams = sink_streams;
> > +
> > +	return 0;
> > +}
> > +
> > +static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
> > +				    unsigned int sink_stream,
> > +				    bool (*func)(struct video_device *vdev),
> > +				    struct media_pad **__sink_pad,
> > +				    u64 *__sink_streams)
> > +{
> > +	struct v4l2_subdev_state *state;
> > +	struct v4l2_subdev_route *route;
> > +	struct v4l2_subdev *sd;
> > +	struct media_pad *source_pad, *tmp_pad;
> > +	u32 source_stream;
> > +
> > +	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
> > +		return -ENXIO;
> > +
> > +	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
> > +	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
> > +		sd->entity.name);
> > +
> > +	state = v4l2_subdev_lock_and_get_active_state(sd);
> > +	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
> > +				       sink_stream, true, 0);
> > +	if (IS_ERR(route)) {
> > +		v4l2_subdev_unlock_state(state);
> > +		dev_dbg(sd->dev,
> > +			"path_enabled: can't find opposite route for %s:%u/%u",
> > +			sd->entity.name, sink_pad->index, sink_stream);
> > +		return 2;
> > +	}
> > +
> > +	source_pad = &sd->entity.pads[route->source_pad];
> > +	v4l2_subdev_unlock_state(state);
> > +
> > +	tmp_pad = sink_pad;
> > +	sink_pad = media_pad_remote_pad_unique(source_pad);
> > +	if (IS_ERR(sink_pad)) {
> > +		dev_dbg(sd->dev,
> > +			"path_enabled: can't find remote source for %s:%u\n",
> > +			source_pad->entity->name, source_pad->index);
> > +		return PTR_ERR(sink_pad);
> > +	}
> > +
> > +	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
> > +		struct video_device *vdev;
> > +
> > +		vdev = media_entity_to_video_device(sink_pad->entity);
> > +		if (!vdev)
> > +			return -ENXIO;
> > +
> > +		dev_dbg(vdev->dev_parent,
> > +			"path_enabled: found video device %s\n",
> > +			vdev->name);
> > +
> > +		if (!*__sink_pad) {
> > +			*__sink_pad = tmp_pad;
> > +			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
> > +				tmp_pad->index, sink_stream);
> > +		} else if (tmp_pad != *__sink_pad) {
> > +			dev_dbg(sd->dev,
> > +				"path_enabled: pads %s/%u and %s/%u differ\n",
> > +				tmp_pad->entity->name, tmp_pad->index,
> > +				(*__sink_pad)->entity->name,
> > +				(*__sink_pad)->index);
> > +			return -EXDEV;
> > +		}
> > +
> > +		*__sink_streams |= BIT_ULL(sink_stream);
> > +
> > +		return func(vdev);
> > +	}
> > +
> > +	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
> > +					__sink_pad, __sink_streams);
> > +}
> > +
> > +static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
> > +				      u64 *__streams)
> > +{
> > +	struct v4l2_mbus_frame_desc desc;
> > +	u64 streams = 0;
> > +	int ret;
> > +
> > +	if (!__streams)
> > +		return -EINVAL;
> > +
> > +	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
> > +	if (ret == -ENOIOCTLCMD) {
> > +		*__streams = 1ULL;
> > +		return 0;
> > +	}
> > +	if (ret)
> > +		return ret;
> > +
> > +	for (unsigned int i = 0; i < desc.num_entries; i++) {
> > +		if (streams & BIT_ULL(desc.entry[i].stream))
> > +			return -EINVAL;
> > +
> > +		streams |= BIT_ULL(desc.entry[i].stream);
> > +	}
> > +
> > +	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
> > +		*__streams, streams);
> > +	if (*__streams & ~streams)
> > +		return -EINVAL;
> > +
> > +	*__streams = streams;
> > +
> > +	return 0;
> > +}
> > +
> > +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> > +			     bool (*func)(struct video_device *vdev),
> > +			     struct media_pad **__sink_pad, u64 *__sink_streams)
> > +{
> > +	u64 sink_streams = 1U;
> > +	struct media_pad *src_pad;
> > +	u64 src_streams;
> > +	struct v4l2_subdev_state *state;
> > +	struct media_pad *sink_pad = vdev->entity.pads;
> > +	struct v4l2_subdev *sd = NULL;
> > +	bool streaming = true;
> > +	struct media_pad *tmp_pad;
> > +	u64 tmp_streams;
> > +	int ret;
> > +
> > +	if (!__sink_pad)
> > +		__sink_pad = &tmp_pad;
> > +	if (!__sink_streams)
> > +		__sink_streams = &tmp_streams;
> > +	*__sink_pad = NULL;
> > +	*__sink_streams = 0;
> > +
> > +	do {
> > +		src_pad = media_pad_remote_pad_unique(sink_pad);
> > +		if (IS_ERR(src_pad)) {
> > +			dev_dbg(sd ? sd->dev : vdev->dev_parent,
> > +				"no unique remote pad found from %s:%u\n",
> > +				sink_pad->entity->name, sink_pad->index);
> > +			return PTR_ERR(src_pad);
> > +		}
> > +
> > +		sd = media_entity_to_v4l2_subdev(src_pad->entity);
> > +		if (!sd) {
> > +			dev_dbg(sd->dev,
> > +				"media entity %s is not a V4L2 sub-device\n",
> > +				src_pad->entity->name);
> > +			return -ENXIO;
> > +		}
> > +
> > +		/* Source streams match sink. */
> > +		src_streams = sink_streams;
> > +
> > +		state = v4l2_subdev_lock_and_get_active_state(sd);
> > +		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
> > +						 src_streams, &sink_pad,
> > +						 &sink_streams);
> > +		v4l2_subdev_unlock_state(state);
> > +		if (ret)
> > +			return ret;
> > +	} while (sink_pad);
> > +
> > +	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
> > +	if (ret)
> > +		return ret;
> > +
> > +	sd = media_entity_to_v4l2_subdev(src_pad->entity);
> > +
> > +	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
> > +		src_pad->index, src_streams);
> > +
> > +	for (unsigned int i = __ffs(src_streams); src_streams;
> > +	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
> > +		sink_pad = media_pad_remote_pad_unique(src_pad);
> > +		if (IS_ERR(src_pad)) {
> > +			dev_dbg(sd->dev,
> > +				"no unique remote pad found from %s:%u\n",
> > +				sink_pad->entity->name, sink_pad->index);
> > +			return PTR_ERR(src_pad);
> > +		}
> > +
> > +		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
> > +					       __sink_streams);
> > +		if (ret == 2)
> > +			continue;
> > +		if (ret < 0)
> > +			return ret;
> > +		if (!ret)
> > +			streaming = false;
> > +	}
> > +
> > +	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
> > +		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
> > +		(*__sink_pad)->index, *__sink_streams);
> > +
> > +	return streaming;
> > +}
> > +EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
> > diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
> > index 1837c9fd78cf..e72c0f62fa34 100644
> > --- a/include/media/v4l2-mc.h
> > +++ b/include/media/v4l2-mc.h
> > @@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
> >  int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> >  			      unsigned int notification);
> >
> > +/**
> > + * v4l2_mc_pipeline_enabled - Tell when to start streaming
> > + * @vdev: The video device
> > + * @func: Caller-provided function to tell a video device's streaming state
> > + * @__sink_pad: sink pad at the root of the local pipeline
> > + * @__sink_streams: streams to start
> > + *
> > + * Use to tell whether streaming should start on a video node. @func returns
> > + * true if streaming has been started on a given video node. @__sink_pad and
> > + * @__sink_streams are filled with pad and streams on the sub-device closest to
> > + * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
> > + * v4l2_subdev_disable_streams().
> > + *
> > + * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
> > + * a to-do list):
> > + * * only unbranched streams can be supported albeit adding support for
> > + *   downstream branches would be fairly trivial,
> > + * * streams within a single source sub-device are considered to start at the
> > + *   same time, more control could be added in two ways: 1) for sources to
> > + *   determine stream starting, a control could be added to UAPI and 2) sources
> > + *   could tell which streams start at the same time using a sub-device
> > + *   operation,
> > + * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
> > + *   be implemented by letting the caller to provide a function to determine
> > + *   which streams are of interest and
> > + * * routes leading to nowhere are ignored, on some hardware this is a problem,
> > + *   but this can also be rather trivially addressed.
> > + *
> > + * Return:
> > + * * 0: Success, but don't start streaming yet
> > + * * 1: Success, now it's time to start streaming
> > + * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
> > + * * -ENOTUNIQ: No unique remote pad
> > + * * -ENOLINK: No remote pad found
> > + * * -ENOENT: Enabled upstream route not found
> > + * * -EMLINK: No unique downstream route found
> > + * * -EINVAL: Stream could not be followed to source or was not produced by
> > + *            the source
> > + */
> > +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> > +			     bool (*func)(struct video_device *vdev),
> > +			     struct media_pad **__sink_pad,
> > +			     u64 *__sink_streams);
> > +
> >  #else /* CONFIG_MEDIA_CONTROLLER */
> >
> >  static inline int v4l2_mc_create_media_graph(struct media_device *mdev)

-- 
Kind regards,

Sakari Ailus

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Laurent Pinchart]

On Tue, Jul 15, 2025 at 10:49:32AM +0000, Sakari Ailus wrote:
> On Fri, Jun 20, 2025 at 10:53:13AM +0200, Jacopo Mondi wrote:
> > On Thu, Jun 19, 2025 at 11:15:45AM +0300, Sakari Ailus wrote:
> > > v4l2_mc_pipeline_enabled() helps solving a problem known for long but
> > > lacked any sort of general solution: with multiple streams, when streaming
> > > is started on video nodes one by one, when should streaming be started in
> > > the source?
> > 
> > I tried quite some time to understand this, but if I'm not mistaken,
> > a stream-aware subdev, which links to vdev, will always "demux"
> > streams to different pads and will connect to the vdev from there
> > 
> > 
> >      Source
> >      subdev
> >  +-----------------+
> >  |                (1/0) ------> vdev0
> >  |                 |
> > (0)[1,2,3]        (2/0  ------> vdev1
> >  |                 |
> >  |                (3/0) ------> vdev2
> >  +-----------------+
> > 
> > With
> > 
> > (0) multiplexed sink pad with 3 streams
> > (1) (2) and (3) source pad with a single stream
> > 
> > Can't we relay on the media-link state between the source pads and the
> > video devices with something like what Dan has proposed here ?
> > https://patchwork.linuxtv.org/project/linux-media/patch/20250519140403.443915-2-dan.scally@ideasonboard.com/
> 
> This isn't entirely the same thing: pipeline is specific to the pad but not
> streams, ad here we're interested in streams. Two streams may start at
> different points of time even if both are part of the same pipeline.
> 
> We definitely should have just one way to figure this out.

But can we ? Isn't it dependent on use cases ? I can imagine an
application wanting to capture two image streams where one of them is
enabled all the time and the other one is regularly turned on and off,
in which case you would need to start the pipeline when the first stream
starts. A different application may instead want to capture the same two
streams and make sure it gets all frames on both, which possibly
requires delaying the start of capture until both video capture devices
are started.

> I'll send v2 with comments and before that also see if (or how) we could
> get rid of the callback.
> 
> Cc Dan.
> 
> > What am I missing ?
> > 
> > > v4l2_mc_pipeline_enabled() traverses the pipeline towards the source,
> > > queries the streams generated by the source and traces them back to the
> > > video nodes.
> > >
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
> > >  include/media/v4l2-mc.h           |  44 ++++++
> > >  2 files changed, 287 insertions(+)
> > >
> > > diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
> > > index 937d358697e1..1731088ad436 100644
> > > --- a/drivers/media/v4l2-core/v4l2-mc.c
> > > +++ b/drivers/media/v4l2-core/v4l2-mc.c
> > > @@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> > >  	return ret;
> > >  }
> > >  EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
> > > +
> > > +static int
> > > +__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
> > > +			   struct media_pad *src_pad, u64 __src_streams,
> > > +			   struct media_pad **__sink_pad, u64 *__sink_streams)
> > > +{
> > > +	struct v4l2_subdev_route *route;
> > > +	u64 src_streams = 0, sink_streams = 0;
> > > +	bool has_sink_pad = false;
> > > +	unsigned int sink_pad;
> > > +
> > > +	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
> > > +		state->sd->entity.name, src_pad->index, __src_streams);
> > > +	for_each_active_route(&state->routing, route) {
> > > +		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
> > > +			state->sd->entity.name,
> > > +			route->sink_pad, route->sink_stream, route->source_pad,
> > > +			route->source_stream, route->flags);
> > > +		if (route->source_pad != src_pad->index)
> > > +			continue;
> > > +
> > > +		if (!(BIT_ULL(route->source_stream) & __src_streams))
> > > +			continue;
> > > +
> > > +		if (!has_sink_pad) {
> > > +			has_sink_pad = true;
> > > +			sink_pad = route->sink_pad;
> > > +		}
> > > +
> > > +		if (route->sink_pad != sink_pad) {
> > > +			dev_dbg(state->sd->dev,
> > > +				"sink pads (%u vs. %u) differ\n",
> > > +				route->sink_pad, sink_pad);
> > > +			return -EMLINK;
> > > +		}
> > > +
> > > +		sink_streams |= BIT_ULL(route->sink_stream);
> > > +		src_streams |= BIT_ULL(route->source_stream);
> > > +	}
> > > +
> > > +	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
> > > +	*__sink_streams = sink_streams;
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
> > > +				    unsigned int sink_stream,
> > > +				    bool (*func)(struct video_device *vdev),
> > > +				    struct media_pad **__sink_pad,
> > > +				    u64 *__sink_streams)
> > > +{
> > > +	struct v4l2_subdev_state *state;
> > > +	struct v4l2_subdev_route *route;
> > > +	struct v4l2_subdev *sd;
> > > +	struct media_pad *source_pad, *tmp_pad;
> > > +	u32 source_stream;
> > > +
> > > +	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
> > > +		return -ENXIO;
> > > +
> > > +	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
> > > +	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
> > > +		sd->entity.name);
> > > +
> > > +	state = v4l2_subdev_lock_and_get_active_state(sd);
> > > +	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
> > > +				       sink_stream, true, 0);
> > > +	if (IS_ERR(route)) {
> > > +		v4l2_subdev_unlock_state(state);
> > > +		dev_dbg(sd->dev,
> > > +			"path_enabled: can't find opposite route for %s:%u/%u",
> > > +			sd->entity.name, sink_pad->index, sink_stream);
> > > +		return 2;
> > > +	}
> > > +
> > > +	source_pad = &sd->entity.pads[route->source_pad];
> > > +	v4l2_subdev_unlock_state(state);
> > > +
> > > +	tmp_pad = sink_pad;
> > > +	sink_pad = media_pad_remote_pad_unique(source_pad);
> > > +	if (IS_ERR(sink_pad)) {
> > > +		dev_dbg(sd->dev,
> > > +			"path_enabled: can't find remote source for %s:%u\n",
> > > +			source_pad->entity->name, source_pad->index);
> > > +		return PTR_ERR(sink_pad);
> > > +	}
> > > +
> > > +	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
> > > +		struct video_device *vdev;
> > > +
> > > +		vdev = media_entity_to_video_device(sink_pad->entity);
> > > +		if (!vdev)
> > > +			return -ENXIO;
> > > +
> > > +		dev_dbg(vdev->dev_parent,
> > > +			"path_enabled: found video device %s\n",
> > > +			vdev->name);
> > > +
> > > +		if (!*__sink_pad) {
> > > +			*__sink_pad = tmp_pad;
> > > +			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
> > > +				tmp_pad->index, sink_stream);
> > > +		} else if (tmp_pad != *__sink_pad) {
> > > +			dev_dbg(sd->dev,
> > > +				"path_enabled: pads %s/%u and %s/%u differ\n",
> > > +				tmp_pad->entity->name, tmp_pad->index,
> > > +				(*__sink_pad)->entity->name,
> > > +				(*__sink_pad)->index);
> > > +			return -EXDEV;
> > > +		}
> > > +
> > > +		*__sink_streams |= BIT_ULL(sink_stream);
> > > +
> > > +		return func(vdev);
> > > +	}
> > > +
> > > +	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
> > > +					__sink_pad, __sink_streams);
> > > +}
> > > +
> > > +static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
> > > +				      u64 *__streams)
> > > +{
> > > +	struct v4l2_mbus_frame_desc desc;
> > > +	u64 streams = 0;
> > > +	int ret;
> > > +
> > > +	if (!__streams)
> > > +		return -EINVAL;
> > > +
> > > +	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
> > > +	if (ret == -ENOIOCTLCMD) {
> > > +		*__streams = 1ULL;
> > > +		return 0;
> > > +	}
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	for (unsigned int i = 0; i < desc.num_entries; i++) {
> > > +		if (streams & BIT_ULL(desc.entry[i].stream))
> > > +			return -EINVAL;
> > > +
> > > +		streams |= BIT_ULL(desc.entry[i].stream);
> > > +	}
> > > +
> > > +	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
> > > +		*__streams, streams);
> > > +	if (*__streams & ~streams)
> > > +		return -EINVAL;
> > > +
> > > +	*__streams = streams;
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> > > +			     bool (*func)(struct video_device *vdev),
> > > +			     struct media_pad **__sink_pad, u64 *__sink_streams)
> > > +{
> > > +	u64 sink_streams = 1U;
> > > +	struct media_pad *src_pad;
> > > +	u64 src_streams;
> > > +	struct v4l2_subdev_state *state;
> > > +	struct media_pad *sink_pad = vdev->entity.pads;
> > > +	struct v4l2_subdev *sd = NULL;
> > > +	bool streaming = true;
> > > +	struct media_pad *tmp_pad;
> > > +	u64 tmp_streams;
> > > +	int ret;
> > > +
> > > +	if (!__sink_pad)
> > > +		__sink_pad = &tmp_pad;
> > > +	if (!__sink_streams)
> > > +		__sink_streams = &tmp_streams;
> > > +	*__sink_pad = NULL;
> > > +	*__sink_streams = 0;
> > > +
> > > +	do {
> > > +		src_pad = media_pad_remote_pad_unique(sink_pad);
> > > +		if (IS_ERR(src_pad)) {
> > > +			dev_dbg(sd ? sd->dev : vdev->dev_parent,
> > > +				"no unique remote pad found from %s:%u\n",
> > > +				sink_pad->entity->name, sink_pad->index);
> > > +			return PTR_ERR(src_pad);
> > > +		}
> > > +
> > > +		sd = media_entity_to_v4l2_subdev(src_pad->entity);
> > > +		if (!sd) {
> > > +			dev_dbg(sd->dev,
> > > +				"media entity %s is not a V4L2 sub-device\n",
> > > +				src_pad->entity->name);
> > > +			return -ENXIO;
> > > +		}
> > > +
> > > +		/* Source streams match sink. */
> > > +		src_streams = sink_streams;
> > > +
> > > +		state = v4l2_subdev_lock_and_get_active_state(sd);
> > > +		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
> > > +						 src_streams, &sink_pad,
> > > +						 &sink_streams);
> > > +		v4l2_subdev_unlock_state(state);
> > > +		if (ret)
> > > +			return ret;
> > > +	} while (sink_pad);
> > > +
> > > +	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	sd = media_entity_to_v4l2_subdev(src_pad->entity);
> > > +
> > > +	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
> > > +		src_pad->index, src_streams);
> > > +
> > > +	for (unsigned int i = __ffs(src_streams); src_streams;
> > > +	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
> > > +		sink_pad = media_pad_remote_pad_unique(src_pad);
> > > +		if (IS_ERR(src_pad)) {
> > > +			dev_dbg(sd->dev,
> > > +				"no unique remote pad found from %s:%u\n",
> > > +				sink_pad->entity->name, sink_pad->index);
> > > +			return PTR_ERR(src_pad);
> > > +		}
> > > +
> > > +		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
> > > +					       __sink_streams);
> > > +		if (ret == 2)
> > > +			continue;
> > > +		if (ret < 0)
> > > +			return ret;
> > > +		if (!ret)
> > > +			streaming = false;
> > > +	}
> > > +
> > > +	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
> > > +		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
> > > +		(*__sink_pad)->index, *__sink_streams);
> > > +
> > > +	return streaming;
> > > +}
> > > +EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
> > > diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
> > > index 1837c9fd78cf..e72c0f62fa34 100644
> > > --- a/include/media/v4l2-mc.h
> > > +++ b/include/media/v4l2-mc.h
> > > @@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
> > >  int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> > >  			      unsigned int notification);
> > >
> > > +/**
> > > + * v4l2_mc_pipeline_enabled - Tell when to start streaming
> > > + * @vdev: The video device
> > > + * @func: Caller-provided function to tell a video device's streaming state
> > > + * @__sink_pad: sink pad at the root of the local pipeline
> > > + * @__sink_streams: streams to start
> > > + *
> > > + * Use to tell whether streaming should start on a video node. @func returns
> > > + * true if streaming has been started on a given video node. @__sink_pad and
> > > + * @__sink_streams are filled with pad and streams on the sub-device closest to
> > > + * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
> > > + * v4l2_subdev_disable_streams().
> > > + *
> > > + * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
> > > + * a to-do list):
> > > + * * only unbranched streams can be supported albeit adding support for
> > > + *   downstream branches would be fairly trivial,
> > > + * * streams within a single source sub-device are considered to start at the
> > > + *   same time, more control could be added in two ways: 1) for sources to
> > > + *   determine stream starting, a control could be added to UAPI and 2) sources
> > > + *   could tell which streams start at the same time using a sub-device
> > > + *   operation,
> > > + * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
> > > + *   be implemented by letting the caller to provide a function to determine
> > > + *   which streams are of interest and
> > > + * * routes leading to nowhere are ignored, on some hardware this is a problem,
> > > + *   but this can also be rather trivially addressed.
> > > + *
> > > + * Return:
> > > + * * 0: Success, but don't start streaming yet
> > > + * * 1: Success, now it's time to start streaming
> > > + * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
> > > + * * -ENOTUNIQ: No unique remote pad
> > > + * * -ENOLINK: No remote pad found
> > > + * * -ENOENT: Enabled upstream route not found
> > > + * * -EMLINK: No unique downstream route found
> > > + * * -EINVAL: Stream could not be followed to source or was not produced by
> > > + *            the source
> > > + */
> > > +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> > > +			     bool (*func)(struct video_device *vdev),
> > > +			     struct media_pad **__sink_pad,
> > > +			     u64 *__sink_streams);
> > > +
> > >  #else /* CONFIG_MEDIA_CONTROLLER */
> > >
> > >  static inline int v4l2_mc_create_media_graph(struct media_device *mdev)
> 
> -- 
> Kind regards,
> 
> Sakari Ailus

-- 
Regards,

Laurent Pinchart

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Sakari Ailus]

Hi Laurent,

On Tue, Jul 15, 2025 at 02:25:20PM +0300, Laurent Pinchart wrote:
> On Tue, Jul 15, 2025 at 10:49:32AM +0000, Sakari Ailus wrote:
> > On Fri, Jun 20, 2025 at 10:53:13AM +0200, Jacopo Mondi wrote:
> > > On Thu, Jun 19, 2025 at 11:15:45AM +0300, Sakari Ailus wrote:
> > > > v4l2_mc_pipeline_enabled() helps solving a problem known for long but
> > > > lacked any sort of general solution: with multiple streams, when streaming
> > > > is started on video nodes one by one, when should streaming be started in
> > > > the source?
> > > 
> > > I tried quite some time to understand this, but if I'm not mistaken,
> > > a stream-aware subdev, which links to vdev, will always "demux"
> > > streams to different pads and will connect to the vdev from there
> > > 
> > > 
> > >      Source
> > >      subdev
> > >  +-----------------+
> > >  |                (1/0) ------> vdev0
> > >  |                 |
> > > (0)[1,2,3]        (2/0  ------> vdev1
> > >  |                 |
> > >  |                (3/0) ------> vdev2
> > >  +-----------------+
> > > 
> > > With
> > > 
> > > (0) multiplexed sink pad with 3 streams
> > > (1) (2) and (3) source pad with a single stream
> > > 
> > > Can't we relay on the media-link state between the source pads and the
> > > video devices with something like what Dan has proposed here ?
> > > https://patchwork.linuxtv.org/project/linux-media/patch/20250519140403.443915-2-dan.scally@ideasonboard.com/
> > 
> > This isn't entirely the same thing: pipeline is specific to the pad but not
> > streams, ad here we're interested in streams. Two streams may start at
> > different points of time even if both are part of the same pipeline.
> > 
> > We definitely should have just one way to figure this out.
> 
> But can we ? Isn't it dependent on use cases ? I can imagine an
> application wanting to capture two image streams where one of them is
> enabled all the time and the other one is regularly turned on and off,
> in which case you would need to start the pipeline when the first stream
> starts. A different application may instead want to capture the same two
> streams and make sure it gets all frames on both, which possibly
> requires delaying the start of capture until both video capture devices
> are started.

Exactly. It's indeed dependent on the use case but the driver is the same
in both cases. This is why the generic solution needs more information from
the user space but I think we can work out how to support this after
merging the metadata series.

-- 
Regards,

Sakari Ailus

Re: [PATCH 11/13] media: v4l2-subdev: Introduce v4l2_subdev_find_route()

[Sakari Ailus]

Hi Laurent,

On Fri, Jun 27, 2025 at 01:20:47AM +0300, Laurent Pinchart wrote:
> On Wed, Jun 25, 2025 at 04:53:54PM +0000, Sakari Ailus wrote:
> > On Fri, Jun 20, 2025 at 10:14:58AM +0200, Jacopo Mondi wrote:
> > > On Thu, Jun 19, 2025 at 11:15:44AM +0300, Sakari Ailus wrote:
> > > > v4l2_subdev_find_route() is like v4l2_subdev_routing_find_opposite_end(),
> > > > with the difference that it's more flexible: it can look up only active
> > > > routes and can find multiple routes, too.
> > > >
> > > > v4l2_subdev_find_route() is intended to replace
> > > > v4l2_subdev_routing_find_opposite_end().
> > > 
> > > To me this feels like v4l2_subdev_find_route() could be used to
> > > implement more helpers like v4l2_subdev_routing_find_opposite_end()
> > > for drivers instead of going the other way around.
> > > 
> > > let's see what the use cases are
> > > 
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > ---
> > > >  drivers/media/v4l2-core/v4l2-subdev.c | 56 ++++++++++++++++++---------
> > > >  include/media/v4l2-subdev.h           | 19 +++++++++
> > > >  2 files changed, 56 insertions(+), 19 deletions(-)
> > > >
> > > > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > > > index c549a462dac7..13d6e96daf3a 100644
> > > > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > > > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > > > @@ -1996,34 +1996,52 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> > > >  }
> > > >  EXPORT_SYMBOL_GPL(v4l2_subdev_set_routing_with_fmt);
> > > >
> > > > -int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
> > > > -					  u32 pad, u32 stream, u32 *other_pad,
> > > > -					  u32 *other_stream)
> > > > +struct v4l2_subdev_route *
> > > > +v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
> > > > +		       u32 pad, u32 stream, bool active, unsigned int index)
> > > >  {
> > > >  	unsigned int i;
> > > >
> > > >  	for (i = 0; i < routing->num_routes; ++i) {
> > > >  		struct v4l2_subdev_route *route = &routing->routes[i];
> > > >
> > > > -		if (route->source_pad == pad &&
> > > > -		    route->source_stream == stream) {
> > > > -			if (other_pad)
> > > > -				*other_pad = route->sink_pad;
> > > > -			if (other_stream)
> > > > -				*other_stream = route->sink_stream;
> > > > -			return 0;
> > > > -		}
> > > > +		if (active && !(route->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> > > > +			continue;
> > > 
> > > I know currently v4l2_subdev_routing_find_opposite_end() does return
> > > any route that matches the provided 'pad' and 'stream' included
> > > non-active ones, but I wonder if this is desirable. What is the use
> > > case for enumerating a non-active route between two pads ?
> > 
> > Good question. v4l2_subdev_routing_find_opposite_end() nevertheless returns
> > them. And the caller won't get the route for checking the state either.
> 
> I'm tempted to check the callers of the function and change the
> behaviour to only return active routes.

I can prepend a patch to do the change there so it won't happen here.

> 
> > > (it is also my impression that all drivers that use
> > > v4l2_subdev_routing_find_opposite_end() assume the route is active)
> > > 
> > > Also I wonder if the usage of V4L2_SUBDEV_ROUTE_FL_ACTIVE is clearly
> > > defined, or, in other words, what is the use case for userspace to
> > > create non-active routes, given that any new VIDIOC_SUBDEV_S_ROUTING
> > > will anyway re-create the routing table (that's a different question,
> > > on the ioctl definition and not on this change though)
> > 
> > I think it is. Please review the UAPI documentation in the metadata series.
> > :-)
> 
> Section "Device types and routing setup" in
> Documentation/userspace-api/media/v4l/dev-subdev.rst is the best
> documentation we have at the moment.
> 
> We essentially have two models for internal routing. In the first model,
> which I'll nickname the crossbar switch model, a large number of routes
> are possible (up to any input to any output types of scenarios). In this
> case, routes are created by userspace, and all routes in the routing
> table are expected to be active. Any inactive route provided by
> userspace would be dropped by the driver and not be included in the
> routing table.
> 
> The second model covers devices such as camera sensors, where a small
> fixed set of routes are hardcoded. The routing table is fixed, an some
> routes (the ones not marked with the IMMUTABLE flag) can be
> enabled/disabled by userspace using the V4L2_SUBDEV_ROUTE_FL_ACTIVE.
> This allows userspace to enumerate the available routes.
> 
> Documentation/userspace-api/media/v4l/dev-subdev.rst should document
> more clearly that we do not allow any hybrid behaviour at the moment.

I agree.

> 
> > > >
> > > > -		if (route->sink_pad == pad && route->sink_stream == stream) {
> > > > -			if (other_pad)
> > > > -				*other_pad = route->source_pad;
> > > > -			if (other_stream)
> > > > -				*other_stream = route->source_stream;
> > > > -			return 0;
> > > > -		}
> > > > +		if ((route->source_pad != pad ||
> > > > +		     route->source_stream != stream) &&
> > > > +		    (route->sink_pad != pad || route->sink_stream != stream))
> > > > +			continue;
> > > > +
> > > > +		if (index--)
> > > > +			continue;
> > > > +
> > > > +		return route;
> > > >  	}
> > > >
> > > > -	return -EINVAL;
> > > > +	return ERR_PTR(-ENOENT);
> > > > +}
> > > > +EXPORT_SYMBOL_GPL(v4l2_subdev_find_route);
> > > > +
> > > > +int v4l2_subdev_routing_find_opposite_end(const struct v4l2_subdev_krouting *routing,
> > > > +					  u32 pad, u32 stream, u32 *other_pad,
> > > > +					  u32 *other_stream)
> > > > +{
> > > > +	struct v4l2_subdev_route *route;
> > > > +
> > > > +	route = v4l2_subdev_find_route(routing, pad, stream, false, 0);
> > > > +	if (IS_ERR(route))
> > > > +		return PTR_ERR(route);
> > > > +
> > > > +	bool is_source = route->source_pad == pad;
> > > > +
> > > > +	if (other_pad)
> > > > +		*other_pad = is_source ? route->sink_pad : route->source_pad;
> > > > +	if (other_stream)
> > > > +		*other_stream = is_source ?
> > > > +			route->sink_stream : route->source_stream;
> 
> Having to do this is_source dance makes v4l2_subdev_find_route()
> annoying to use. It may be fine when using the function to implement
> other helpers, but I wouldn't like to see this being done in drivers.
> Maybe we can avoid exporting v4l2_subdev_find_route() for now to ensure
> that, the only user in this series is in v4l2-mc.c.

Sounds reasonable.

> 
> > > > +
> > > > +	return 0;
> > > >  }
> > > >  EXPORT_SYMBOL_GPL(v4l2_subdev_routing_find_opposite_end);
> > > >
> > > > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> > > > index deab128a4779..9ed8600ba3d4 100644
> > > > --- a/include/media/v4l2-subdev.h
> > > > +++ b/include/media/v4l2-subdev.h
> > > > @@ -1547,6 +1547,23 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> > > >  				     const struct v4l2_subdev_krouting *routing,
> > > >  				     const struct v4l2_mbus_framefmt *fmt);
> > > >
> > > > +/**
> > > > + * v4l2_subdev_find_route() - Find routes from a (pad, stream) pair
> > > 
> > > from or for ?
> > 
> > From or to. I'll fix this in the next version.
> > 
> > > 
> > > > + * @routing: routing used to find the opposite side
> > > 
> > > I would not say "opposite side" but rather
> > > 
> > >       @routing: routing table used to enumerate routes
> > 
> > How about simply "the routing table"?
> > 
> > > 
> > > > + * @pad: pad id
> > > > + * @stream: stream id
> > > > + * @active: set to true for looking up only active routes
> > > > + * @index: for accessing more than one route from the pad
> > > 
> > > I understand this but maybe
> > > 
> > >      @index: route index for enumerating multiple routes
> > > ?
> > 
> > Sounds good.
> 
> I'm curious to know how this parameter will be used. In the only user
> (in patch 12/13), it is hardcoded to 0. I'm not sure indexing routes
> will be very useful for drivers.

Looks like it'll be needed by v4l2_subdev_set_streams_enabled(), as a
result on discussion related to it.

> 
> > > > + *
> > > > + * Find a route from the routing table where one end has (pad, stream) pair
> > > > + * matching @pad and @stream.
> > > 
> > >     * If multiple routes in @routing match @pad and @stream, return
> > >     * the @index one.
> > >     *
> > >     * Set @active to true to only enumerate active routes.
> > > 
> > > > + *
> > > > + * Returns the route on success or -ENOENT if no matching route is found.
> > > 
> > > I see other functions documentation using
> > > 
> > >     * Return:
> > > 
> > > is this a kernel-doc thing ?
> > 
> > Yes, makes sense.
> > 
> > > > + */
> > > > +struct v4l2_subdev_route *
> > > > +v4l2_subdev_find_route(const struct v4l2_subdev_krouting *routing,
> > > > +		       u32 pad, u32 stream, bool active, unsigned int index);
> > > > +
> > > >  /**
> > > >   * v4l2_subdev_routing_find_opposite_end() - Find the opposite stream
> > > >   * @routing: routing used to find the opposite side
> > > > @@ -1555,6 +1572,8 @@ int v4l2_subdev_set_routing_with_fmt(struct v4l2_subdev *sd,
> > > >   * @other_pad: pointer used to return the opposite pad
> > > >   * @other_stream: pointer used to return the opposite stream
> > > >   *
> > > > + * Prefer v4l2_subdev_find_route() over v4l2_subdev_routing_find_opposite_end().
> > > > + *
> > > 
> > > As said, I'm not sure if that's preferred or we should rather create
> > > more helpers using v4l2_subdev_find_route() internally. Time will tell
> > > I guess ?
> > 
> > I agree.
> > 
> > The benefit of the older function was that it returns information that
> > doesn't need a lock for accessing it.
> 
> I'm not sure to follow you here. v4l2_subdev_find_route() has the same
> locking requirements as v4l2_subdev_routing_find_opposite_end().

The function itself does, but a lock is not required for accessing
information returned by v4l2_subdev_routing_find_opposite_end() (unlike
v4l2_subdev_find_route()).

> 
> Given that v4l2_subdev_find_route() has a single user, and that the only
> difference in that user compared to
> v4l2_subdev_routing_find_opposite_end() is that only active routes are
> considered, I would prefer modifying
> v4l2_subdev_routing_find_opposite_end() to ignore inactive routes after
> checking that no caller would break.

-- 
Regards,

Sakari Ailus

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Laurent Pinchart]

On Tue, Jul 15, 2025 at 11:32:14AM +0000, Sakari Ailus wrote:
> On Tue, Jul 15, 2025 at 02:25:20PM +0300, Laurent Pinchart wrote:
> > On Tue, Jul 15, 2025 at 10:49:32AM +0000, Sakari Ailus wrote:
> > > On Fri, Jun 20, 2025 at 10:53:13AM +0200, Jacopo Mondi wrote:
> > > > On Thu, Jun 19, 2025 at 11:15:45AM +0300, Sakari Ailus wrote:
> > > > > v4l2_mc_pipeline_enabled() helps solving a problem known for long but
> > > > > lacked any sort of general solution: with multiple streams, when streaming
> > > > > is started on video nodes one by one, when should streaming be started in
> > > > > the source?
> > > > 
> > > > I tried quite some time to understand this, but if I'm not mistaken,
> > > > a stream-aware subdev, which links to vdev, will always "demux"
> > > > streams to different pads and will connect to the vdev from there
> > > > 
> > > > 
> > > >      Source
> > > >      subdev
> > > >  +-----------------+
> > > >  |                (1/0) ------> vdev0
> > > >  |                 |
> > > > (0)[1,2,3]        (2/0  ------> vdev1
> > > >  |                 |
> > > >  |                (3/0) ------> vdev2
> > > >  +-----------------+
> > > > 
> > > > With
> > > > 
> > > > (0) multiplexed sink pad with 3 streams
> > > > (1) (2) and (3) source pad with a single stream
> > > > 
> > > > Can't we relay on the media-link state between the source pads and the
> > > > video devices with something like what Dan has proposed here ?
> > > > https://patchwork.linuxtv.org/project/linux-media/patch/20250519140403.443915-2-dan.scally@ideasonboard.com/
> > > 
> > > This isn't entirely the same thing: pipeline is specific to the pad but not
> > > streams, ad here we're interested in streams. Two streams may start at
> > > different points of time even if both are part of the same pipeline.
> > > 
> > > We definitely should have just one way to figure this out.
> > 
> > But can we ? Isn't it dependent on use cases ? I can imagine an
> > application wanting to capture two image streams where one of them is
> > enabled all the time and the other one is regularly turned on and off,
> > in which case you would need to start the pipeline when the first stream
> > starts. A different application may instead want to capture the same two
> > streams and make sure it gets all frames on both, which possibly
> > requires delaying the start of capture until both video capture devices
> > are started.
> 
> Exactly. It's indeed dependent on the use case but the driver is the same
> in both cases. This is why the generic solution needs more information from
> the user space but I think we can work out how to support this after
> merging the metadata series.

Yes, we need more information from userspace.

It's also a driver-dependent behaviour. If the streams come from
multiple cameras that all go through the same deserializer (e.g. a quad
FPD-Link or GMSL deserializer), then the decision on when to start
streaming on some of the components in the pipeline could depend on
which deserializer is used. Some will require all cameras to be started
together, while other could allow starting and stopping sources
independently. The nature of upstream elements could influence when
downstream elements will need to be started, for the same
userspace-facing behaviour.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Sakari Ailus]

Hi Laurent,

Thanks for the review.

On Fri, Jun 27, 2025 at 02:07:10AM +0300, Laurent Pinchart wrote:
> Hi Sakari,
> 
> Thank you for the patch.
> 
> On Thu, Jun 19, 2025 at 11:15:45AM +0300, Sakari Ailus wrote:
> > v4l2_mc_pipeline_enabled() helps solving a problem known for long but
> > lacked any sort of general solution: with multiple streams, when streaming
> > is started on video nodes one by one, when should streaming be started in
> > the source?
> > 
> > v4l2_mc_pipeline_enabled() traverses the pipeline towards the source,
> > queries the streams generated by the source and traces them back to the
> > video nodes.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
> >  include/media/v4l2-mc.h           |  44 ++++++
> >  2 files changed, 287 insertions(+)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
> > index 937d358697e1..1731088ad436 100644
> > --- a/drivers/media/v4l2-core/v4l2-mc.c
> > +++ b/drivers/media/v4l2-core/v4l2-mc.c
> > @@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> >  	return ret;
> >  }
> >  EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
> > +
> > +static int
> > +__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
> > +			   struct media_pad *src_pad, u64 __src_streams,
> > +			   struct media_pad **__sink_pad, u64 *__sink_streams)
> > +{
> > +	struct v4l2_subdev_route *route;
> > +	u64 src_streams = 0, sink_streams = 0;
> > +	bool has_sink_pad = false;
> > +	unsigned int sink_pad;
> > +
> > +	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
> > +		state->sd->entity.name, src_pad->index, __src_streams);
> > +	for_each_active_route(&state->routing, route) {
> > +		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
> > +			state->sd->entity.name,
> > +			route->sink_pad, route->sink_stream, route->source_pad,
> > +			route->source_stream, route->flags);
> > +		if (route->source_pad != src_pad->index)
> > +			continue;
> > +
> > +		if (!(BIT_ULL(route->source_stream) & __src_streams))
> > +			continue;
> > +
> > +		if (!has_sink_pad) {
> > +			has_sink_pad = true;
> > +			sink_pad = route->sink_pad;
> > +		}
> > +
> > +		if (route->sink_pad != sink_pad) {
> > +			dev_dbg(state->sd->dev,
> > +				"sink pads (%u vs. %u) differ\n",
> > +				route->sink_pad, sink_pad);
> > +			return -EMLINK;
> > +		}
> > +
> > +		sink_streams |= BIT_ULL(route->sink_stream);
> > +		src_streams |= BIT_ULL(route->source_stream);
> > +	}
> > +
> > +	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
> > +	*__sink_streams = sink_streams;
> > +
> > +	return 0;
> > +}
> > +
> > +static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
> > +				    unsigned int sink_stream,
> > +				    bool (*func)(struct video_device *vdev),
> > +				    struct media_pad **__sink_pad,
> > +				    u64 *__sink_streams)
> > +{
> > +	struct v4l2_subdev_state *state;
> > +	struct v4l2_subdev_route *route;
> > +	struct v4l2_subdev *sd;
> > +	struct media_pad *source_pad, *tmp_pad;
> > +	u32 source_stream;
> > +
> > +	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
> > +		return -ENXIO;
> > +
> > +	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
> > +	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
> > +		sd->entity.name);
> > +
> > +	state = v4l2_subdev_lock_and_get_active_state(sd);
> > +	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
> > +				       sink_stream, true, 0);
> > +	if (IS_ERR(route)) {
> > +		v4l2_subdev_unlock_state(state);
> > +		dev_dbg(sd->dev,
> > +			"path_enabled: can't find opposite route for %s:%u/%u",
> > +			sd->entity.name, sink_pad->index, sink_stream);
> > +		return 2;
> > +	}
> > +
> > +	source_pad = &sd->entity.pads[route->source_pad];
> > +	v4l2_subdev_unlock_state(state);
> > +
> > +	tmp_pad = sink_pad;
> > +	sink_pad = media_pad_remote_pad_unique(source_pad);
> > +	if (IS_ERR(sink_pad)) {
> > +		dev_dbg(sd->dev,
> > +			"path_enabled: can't find remote source for %s:%u\n",
> > +			source_pad->entity->name, source_pad->index);
> > +		return PTR_ERR(sink_pad);
> > +	}
> > +
> > +	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
> > +		struct video_device *vdev;
> > +
> > +		vdev = media_entity_to_video_device(sink_pad->entity);
> > +		if (!vdev)
> > +			return -ENXIO;
> > +
> > +		dev_dbg(vdev->dev_parent,
> > +			"path_enabled: found video device %s\n",
> > +			vdev->name);
> > +
> > +		if (!*__sink_pad) {
> > +			*__sink_pad = tmp_pad;
> > +			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
> > +				tmp_pad->index, sink_stream);
> > +		} else if (tmp_pad != *__sink_pad) {
> > +			dev_dbg(sd->dev,
> > +				"path_enabled: pads %s/%u and %s/%u differ\n",
> > +				tmp_pad->entity->name, tmp_pad->index,
> > +				(*__sink_pad)->entity->name,
> > +				(*__sink_pad)->index);
> > +			return -EXDEV;
> > +		}
> > +
> > +		*__sink_streams |= BIT_ULL(sink_stream);
> > +
> > +		return func(vdev);
> > +	}
> > +
> > +	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
> > +					__sink_pad, __sink_streams);
> > +}
> > +
> > +static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
> > +				      u64 *__streams)
> > +{
> > +	struct v4l2_mbus_frame_desc desc;
> > +	u64 streams = 0;
> > +	int ret;
> > +
> > +	if (!__streams)
> > +		return -EINVAL;
> > +
> > +	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
> > +	if (ret == -ENOIOCTLCMD) {
> > +		*__streams = 1ULL;
> > +		return 0;
> > +	}
> > +	if (ret)
> > +		return ret;
> > +
> > +	for (unsigned int i = 0; i < desc.num_entries; i++) {
> > +		if (streams & BIT_ULL(desc.entry[i].stream))
> > +			return -EINVAL;
> > +
> > +		streams |= BIT_ULL(desc.entry[i].stream);
> > +	}
> > +
> > +	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
> > +		*__streams, streams);
> > +	if (*__streams & ~streams)
> > +		return -EINVAL;
> > +
> > +	*__streams = streams;
> > +
> > +	return 0;
> > +}
> > +
> > +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> > +			     bool (*func)(struct video_device *vdev),
> > +			     struct media_pad **__sink_pad, u64 *__sink_streams)
> > +{
> > +	u64 sink_streams = 1U;
> > +	struct media_pad *src_pad;
> > +	u64 src_streams;
> > +	struct v4l2_subdev_state *state;
> > +	struct media_pad *sink_pad = vdev->entity.pads;
> > +	struct v4l2_subdev *sd = NULL;
> > +	bool streaming = true;
> > +	struct media_pad *tmp_pad;
> > +	u64 tmp_streams;
> > +	int ret;
> > +
> > +	if (!__sink_pad)
> > +		__sink_pad = &tmp_pad;
> > +	if (!__sink_streams)
> > +		__sink_streams = &tmp_streams;
> > +	*__sink_pad = NULL;
> > +	*__sink_streams = 0;
> > +
> > +	do {
> > +		src_pad = media_pad_remote_pad_unique(sink_pad);
> > +		if (IS_ERR(src_pad)) {
> > +			dev_dbg(sd ? sd->dev : vdev->dev_parent,
> > +				"no unique remote pad found from %s:%u\n",
> > +				sink_pad->entity->name, sink_pad->index);
> > +			return PTR_ERR(src_pad);
> > +		}
> > +
> > +		sd = media_entity_to_v4l2_subdev(src_pad->entity);
> > +		if (!sd) {
> > +			dev_dbg(sd->dev,
> > +				"media entity %s is not a V4L2 sub-device\n",
> > +				src_pad->entity->name);
> > +			return -ENXIO;
> > +		}
> > +
> > +		/* Source streams match sink. */
> > +		src_streams = sink_streams;
> > +
> > +		state = v4l2_subdev_lock_and_get_active_state(sd);
> > +		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
> > +						 src_streams, &sink_pad,
> > +						 &sink_streams);
> > +		v4l2_subdev_unlock_state(state);
> > +		if (ret)
> > +			return ret;
> > +	} while (sink_pad);
> > +
> > +	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
> > +	if (ret)
> > +		return ret;
> > +
> > +	sd = media_entity_to_v4l2_subdev(src_pad->entity);
> > +
> > +	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
> > +		src_pad->index, src_streams);
> > +
> > +	for (unsigned int i = __ffs(src_streams); src_streams;
> > +	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
> > +		sink_pad = media_pad_remote_pad_unique(src_pad);
> > +		if (IS_ERR(src_pad)) {
> > +			dev_dbg(sd->dev,
> > +				"no unique remote pad found from %s:%u\n",
> > +				sink_pad->entity->name, sink_pad->index);
> > +			return PTR_ERR(src_pad);
> > +		}
> > +
> > +		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
> > +					       __sink_streams);
> > +		if (ret == 2)
> > +			continue;
> > +		if (ret < 0)
> > +			return ret;
> > +		if (!ret)
> > +			streaming = false;
> > +	}
> > +
> > +	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
> > +		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
> > +		(*__sink_pad)->index, *__sink_streams);
> > +
> > +	return streaming;
> > +}
> > +EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
> > diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
> > index 1837c9fd78cf..e72c0f62fa34 100644
> > --- a/include/media/v4l2-mc.h
> > +++ b/include/media/v4l2-mc.h
> > @@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
> >  int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> >  			      unsigned int notification);
> >  
> > +/**
> > + * v4l2_mc_pipeline_enabled - Tell when to start streaming
> > + * @vdev: The video device
> > + * @func: Caller-provided function to tell a video device's streaming state
> > + * @__sink_pad: sink pad at the root of the local pipeline
> > + * @__sink_streams: streams to start
> 
> Any reason for the double underscore ?

The function used internally a variable without the underscores for a
different purpose.

> 
> > + *
> > + * Use to tell whether streaming should start on a video node. @func returns
> > + * true if streaming has been started on a given video node. @__sink_pad and
> > + * @__sink_streams are filled with pad and streams on the sub-device closest to
> > + * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
> > + * v4l2_subdev_disable_streams().
> > + *
> > + * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
> > + * a to-do list):
> > + * * only unbranched streams can be supported albeit adding support for
> > + *   downstream branches would be fairly trivial,
> 
> I can't tell from the documentation here what you mean exactly by
> "unbranched streams".

These may have been referred to as "linear" streams elsewhere. I'll address
this in the next version.

> 
> > + * * streams within a single source sub-device are considered to start at the
> > + *   same time, more control could be added in two ways: 1) for sources to
> > + *   determine stream starting, a control could be added to UAPI and 2) sources
> > + *   could tell which streams start at the same time using a sub-device
> > + *   operation,
> > + * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
> > + *   be implemented by letting the caller to provide a function to determine
> > + *   which streams are of interest and
> > + * * routes leading to nowhere are ignored, on some hardware this is a problem,
> > + *   but this can also be rather trivially addressed.
> 
> I'm afraid this function looks like a hack, to solve a problem that is
> not even explicitly described. You don't explain the issue in the cover
> letter or in Documentation/, the cover letter merely states that this is
> a "partial solution". The documentation of the function doesn't explain
> what criteria the decision is based on. We need a proper explanation of
> the problem in Documentation/, with a description of the behaviour (or
> behaviours) drivers are expected to implement.

I believe this is discussed in the cover letter, not in detail though. I'm
fine with adding more documentation, there isn't much as I also wanted to
get feedback on the approach itself. No alternatives have been proposed so
far either.

> 
> Furthermore, on the implementation side, things are fairly inefficient.
> We already traverse the whole pipeline in media_pipeline_start(), based
> on links and routes, and populate the media_pipeline structure. We
> shouldn't do the same here, but instead inspect media_pipeline to
> extract the information we need. If you're missing information there,
> let's add it.

The media pipeline is created and traversed, yes, but the media pipeline
does not include streams which are strictly a V4L2 concept. I agree there
is some overlap between the two but as long as MC remains separate from
V4L2, we can't reasonably access streams from the pipeline traversal.

> 
> > + *
> > + * Return:
> > + * * 0: Success, but don't start streaming yet
> > + * * 1: Success, now it's time to start streaming
> > + * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
> > + * * -ENOTUNIQ: No unique remote pad
> > + * * -ENOLINK: No remote pad found
> > + * * -ENOENT: Enabled upstream route not found
> > + * * -EMLINK: No unique downstream route found
> > + * * -EINVAL: Stream could not be followed to source or was not produced by
> > + *            the source
> > + */
> > +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> > +			     bool (*func)(struct video_device *vdev),
> > +			     struct media_pad **__sink_pad,
> > +			     u64 *__sink_streams);
> > +
> >  #else /* CONFIG_MEDIA_CONTROLLER */
> >  
> >  static inline int v4l2_mc_create_media_graph(struct media_device *mdev)

-- 
Kind regards,

Sakari Ailus

Re: [PATCH 12/13] media: v4l2-mc: Introduce v4l2_mc_pipeline_enabled()

[Laurent Pinchart]

On Mon, Aug 04, 2025 at 11:32:17AM +0000, Sakari Ailus wrote:
> On Fri, Jun 27, 2025 at 02:07:10AM +0300, Laurent Pinchart wrote:
> > On Thu, Jun 19, 2025 at 11:15:45AM +0300, Sakari Ailus wrote:
> > > v4l2_mc_pipeline_enabled() helps solving a problem known for long but
> > > lacked any sort of general solution: with multiple streams, when streaming
> > > is started on video nodes one by one, when should streaming be started in
> > > the source?
> > > 
> > > v4l2_mc_pipeline_enabled() traverses the pipeline towards the source,
> > > queries the streams generated by the source and traces them back to the
> > > video nodes.
> > > 
> > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > ---
> > >  drivers/media/v4l2-core/v4l2-mc.c | 243 ++++++++++++++++++++++++++++++
> > >  include/media/v4l2-mc.h           |  44 ++++++
> > >  2 files changed, 287 insertions(+)
> > > 
> > > diff --git a/drivers/media/v4l2-core/v4l2-mc.c b/drivers/media/v4l2-core/v4l2-mc.c
> > > index 937d358697e1..1731088ad436 100644
> > > --- a/drivers/media/v4l2-core/v4l2-mc.c
> > > +++ b/drivers/media/v4l2-core/v4l2-mc.c
> > > @@ -612,3 +612,246 @@ int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> > >  	return ret;
> > >  }
> > >  EXPORT_SYMBOL_GPL(v4l2_pipeline_link_notify);
> > > +
> > > +static int
> > > +__v4l2_mc_pipeline_enabled(struct v4l2_subdev_state *state,
> > > +			   struct media_pad *src_pad, u64 __src_streams,
> > > +			   struct media_pad **__sink_pad, u64 *__sink_streams)
> > > +{
> > > +	struct v4l2_subdev_route *route;
> > > +	u64 src_streams = 0, sink_streams = 0;
> > > +	bool has_sink_pad = false;
> > > +	unsigned int sink_pad;
> > > +
> > > +	dev_dbg(state->sd->dev, "%s: source enabled, pad/streams %u/%#llx\n",
> > > +		state->sd->entity.name, src_pad->index, __src_streams);
> > > +	for_each_active_route(&state->routing, route) {
> > > +		dev_dbg(state->sd->dev, "%s: %u/%u -> %u/%u, flags %x\n",
> > > +			state->sd->entity.name,
> > > +			route->sink_pad, route->sink_stream, route->source_pad,
> > > +			route->source_stream, route->flags);
> > > +		if (route->source_pad != src_pad->index)
> > > +			continue;
> > > +
> > > +		if (!(BIT_ULL(route->source_stream) & __src_streams))
> > > +			continue;
> > > +
> > > +		if (!has_sink_pad) {
> > > +			has_sink_pad = true;
> > > +			sink_pad = route->sink_pad;
> > > +		}
> > > +
> > > +		if (route->sink_pad != sink_pad) {
> > > +			dev_dbg(state->sd->dev,
> > > +				"sink pads (%u vs. %u) differ\n",
> > > +				route->sink_pad, sink_pad);
> > > +			return -EMLINK;
> > > +		}
> > > +
> > > +		sink_streams |= BIT_ULL(route->sink_stream);
> > > +		src_streams |= BIT_ULL(route->source_stream);
> > > +	}
> > > +
> > > +	*__sink_pad = has_sink_pad ? &state->sd->entity.pads[sink_pad] : NULL;
> > > +	*__sink_streams = sink_streams;
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static int v4l2_mc_downpath_enabled(struct media_pad *sink_pad,
> > > +				    unsigned int sink_stream,
> > > +				    bool (*func)(struct video_device *vdev),
> > > +				    struct media_pad **__sink_pad,
> > > +				    u64 *__sink_streams)
> > > +{
> > > +	struct v4l2_subdev_state *state;
> > > +	struct v4l2_subdev_route *route;
> > > +	struct v4l2_subdev *sd;
> > > +	struct media_pad *source_pad, *tmp_pad;
> > > +	u32 source_stream;
> > > +
> > > +	if (!is_media_entity_v4l2_subdev(sink_pad->entity))
> > > +		return -ENXIO;
> > > +
> > > +	sd = media_entity_to_v4l2_subdev(sink_pad->entity);
> > > +	dev_dbg(sd->dev, "path_enabled: found sub-device %s\n",
> > > +		sd->entity.name);
> > > +
> > > +	state = v4l2_subdev_lock_and_get_active_state(sd);
> > > +	route = v4l2_subdev_find_route(&state->routing, sink_pad->index,
> > > +				       sink_stream, true, 0);
> > > +	if (IS_ERR(route)) {
> > > +		v4l2_subdev_unlock_state(state);
> > > +		dev_dbg(sd->dev,
> > > +			"path_enabled: can't find opposite route for %s:%u/%u",
> > > +			sd->entity.name, sink_pad->index, sink_stream);
> > > +		return 2;
> > > +	}
> > > +
> > > +	source_pad = &sd->entity.pads[route->source_pad];
> > > +	v4l2_subdev_unlock_state(state);
> > > +
> > > +	tmp_pad = sink_pad;
> > > +	sink_pad = media_pad_remote_pad_unique(source_pad);
> > > +	if (IS_ERR(sink_pad)) {
> > > +		dev_dbg(sd->dev,
> > > +			"path_enabled: can't find remote source for %s:%u\n",
> > > +			source_pad->entity->name, source_pad->index);
> > > +		return PTR_ERR(sink_pad);
> > > +	}
> > > +
> > > +	if (is_media_entity_v4l2_video_device(sink_pad->entity)) {
> > > +		struct video_device *vdev;
> > > +
> > > +		vdev = media_entity_to_video_device(sink_pad->entity);
> > > +		if (!vdev)
> > > +			return -ENXIO;
> > > +
> > > +		dev_dbg(vdev->dev_parent,
> > > +			"path_enabled: found video device %s\n",
> > > +			vdev->name);
> > > +
> > > +		if (!*__sink_pad) {
> > > +			*__sink_pad = tmp_pad;
> > > +			dev_dbg(sd->dev, "path_enabled: sink %u/%u\n",
> > > +				tmp_pad->index, sink_stream);
> > > +		} else if (tmp_pad != *__sink_pad) {
> > > +			dev_dbg(sd->dev,
> > > +				"path_enabled: pads %s/%u and %s/%u differ\n",
> > > +				tmp_pad->entity->name, tmp_pad->index,
> > > +				(*__sink_pad)->entity->name,
> > > +				(*__sink_pad)->index);
> > > +			return -EXDEV;
> > > +		}
> > > +
> > > +		*__sink_streams |= BIT_ULL(sink_stream);
> > > +
> > > +		return func(vdev);
> > > +	}
> > > +
> > > +	return v4l2_mc_downpath_enabled(sink_pad, source_stream, func,
> > > +					__sink_pad, __sink_streams);
> > > +}
> > > +
> > > +static int v4l2_mc_source_get_streams(struct v4l2_subdev *sd, unsigned int pad,
> > > +				      u64 *__streams)
> > > +{
> > > +	struct v4l2_mbus_frame_desc desc;
> > > +	u64 streams = 0;
> > > +	int ret;
> > > +
> > > +	if (!__streams)
> > > +		return -EINVAL;
> > > +
> > > +	ret = v4l2_subdev_call(sd, pad, get_frame_desc, pad, &desc);
> > > +	if (ret == -ENOIOCTLCMD) {
> > > +		*__streams = 1ULL;
> > > +		return 0;
> > > +	}
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	for (unsigned int i = 0; i < desc.num_entries; i++) {
> > > +		if (streams & BIT_ULL(desc.entry[i].stream))
> > > +			return -EINVAL;
> > > +
> > > +		streams |= BIT_ULL(desc.entry[i].stream);
> > > +	}
> > > +
> > > +	dev_dbg(sd->dev, "found streams %#llx based on streams %#llx\n",
> > > +		*__streams, streams);
> > > +	if (*__streams & ~streams)
> > > +		return -EINVAL;
> > > +
> > > +	*__streams = streams;
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> > > +			     bool (*func)(struct video_device *vdev),
> > > +			     struct media_pad **__sink_pad, u64 *__sink_streams)
> > > +{
> > > +	u64 sink_streams = 1U;
> > > +	struct media_pad *src_pad;
> > > +	u64 src_streams;
> > > +	struct v4l2_subdev_state *state;
> > > +	struct media_pad *sink_pad = vdev->entity.pads;
> > > +	struct v4l2_subdev *sd = NULL;
> > > +	bool streaming = true;
> > > +	struct media_pad *tmp_pad;
> > > +	u64 tmp_streams;
> > > +	int ret;
> > > +
> > > +	if (!__sink_pad)
> > > +		__sink_pad = &tmp_pad;
> > > +	if (!__sink_streams)
> > > +		__sink_streams = &tmp_streams;
> > > +	*__sink_pad = NULL;
> > > +	*__sink_streams = 0;
> > > +
> > > +	do {
> > > +		src_pad = media_pad_remote_pad_unique(sink_pad);
> > > +		if (IS_ERR(src_pad)) {
> > > +			dev_dbg(sd ? sd->dev : vdev->dev_parent,
> > > +				"no unique remote pad found from %s:%u\n",
> > > +				sink_pad->entity->name, sink_pad->index);
> > > +			return PTR_ERR(src_pad);
> > > +		}
> > > +
> > > +		sd = media_entity_to_v4l2_subdev(src_pad->entity);
> > > +		if (!sd) {
> > > +			dev_dbg(sd->dev,
> > > +				"media entity %s is not a V4L2 sub-device\n",
> > > +				src_pad->entity->name);
> > > +			return -ENXIO;
> > > +		}
> > > +
> > > +		/* Source streams match sink. */
> > > +		src_streams = sink_streams;
> > > +
> > > +		state = v4l2_subdev_lock_and_get_active_state(sd);
> > > +		ret = __v4l2_mc_pipeline_enabled(state, src_pad,
> > > +						 src_streams, &sink_pad,
> > > +						 &sink_streams);
> > > +		v4l2_subdev_unlock_state(state);
> > > +		if (ret)
> > > +			return ret;
> > > +	} while (sink_pad);
> > > +
> > > +	ret = v4l2_mc_source_get_streams(sd, src_pad->index, &src_streams);
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	sd = media_entity_to_v4l2_subdev(src_pad->entity);
> > > +
> > > +	dev_dbg(sd->dev, "following %s:%u/%#llx\n", sd->entity.name,
> > > +		src_pad->index, src_streams);
> > > +
> > > +	for (unsigned int i = __ffs(src_streams); src_streams;
> > > +	     src_streams &= ~BIT_ULL(i), i = __ffs(src_streams)) {
> > > +		sink_pad = media_pad_remote_pad_unique(src_pad);
> > > +		if (IS_ERR(src_pad)) {
> > > +			dev_dbg(sd->dev,
> > > +				"no unique remote pad found from %s:%u\n",
> > > +				sink_pad->entity->name, sink_pad->index);
> > > +			return PTR_ERR(src_pad);
> > > +		}
> > > +
> > > +		ret = v4l2_mc_downpath_enabled(sink_pad, i, func, __sink_pad,
> > > +					       __sink_streams);
> > > +		if (ret == 2)
> > > +			continue;
> > > +		if (ret < 0)
> > > +			return ret;
> > > +		if (!ret)
> > > +			streaming = false;
> > > +	}
> > > +
> > > +	dev_dbg(media_entity_to_v4l2_subdev((*__sink_pad)->entity)->dev,
> > > +		"sink pad %s:%u/%#llx\n", (*__sink_pad)->entity->name,
> > > +		(*__sink_pad)->index, *__sink_streams);
> > > +
> > > +	return streaming;
> > > +}
> > > +EXPORT_SYMBOL_GPL(v4l2_mc_pipeline_enabled);
> > > diff --git a/include/media/v4l2-mc.h b/include/media/v4l2-mc.h
> > > index 1837c9fd78cf..e72c0f62fa34 100644
> > > --- a/include/media/v4l2-mc.h
> > > +++ b/include/media/v4l2-mc.h
> > > @@ -193,6 +193,50 @@ void v4l2_pipeline_pm_put(struct media_entity *entity);
> > >  int v4l2_pipeline_link_notify(struct media_link *link, u32 flags,
> > >  			      unsigned int notification);
> > >  
> > > +/**
> > > + * v4l2_mc_pipeline_enabled - Tell when to start streaming
> > > + * @vdev: The video device
> > > + * @func: Caller-provided function to tell a video device's streaming state
> > > + * @__sink_pad: sink pad at the root of the local pipeline
> > > + * @__sink_streams: streams to start
> > 
> > Any reason for the double underscore ?
> 
> The function used internally a variable without the underscores for a
> different purpose.

Then it's better to use the __ for internal variables instead of the
publicly documented API. Or rename internal variables so they don't
clash.

> > > + *
> > > + * Use to tell whether streaming should start on a video node. @func returns
> > > + * true if streaming has been started on a given video node. @__sink_pad and
> > > + * @__sink_streams are filled with pad and streams on the sub-device closest to
> > > + * the video nodes, to be used for calling v4l2_subdev_enable_streams() and
> > > + * v4l2_subdev_disable_streams().
> > > + *
> > > + * Using v4l2_mc_pipeline_enabled() has a few limitations currently (consider it
> > > + * a to-do list):
> > > + * * only unbranched streams can be supported albeit adding support for
> > > + *   downstream branches would be fairly trivial,
> > 
> > I can't tell from the documentation here what you mean exactly by
> > "unbranched streams".
> 
> These may have been referred to as "linear" streams elsewhere. I'll address
> this in the next version.
> 
> > > + * * streams within a single source sub-device are considered to start at the
> > > + *   same time, more control could be added in two ways: 1) for sources to
> > > + *   determine stream starting, a control could be added to UAPI and 2) sources
> > > + *   could tell which streams start at the same time using a sub-device
> > > + *   operation,
> > > + * * CSI-2 VC framing is ignored currently, but VC-based stream starting could
> > > + *   be implemented by letting the caller to provide a function to determine
> > > + *   which streams are of interest and
> > > + * * routes leading to nowhere are ignored, on some hardware this is a problem,
> > > + *   but this can also be rather trivially addressed.
> > 
> > I'm afraid this function looks like a hack, to solve a problem that is
> > not even explicitly described. You don't explain the issue in the cover
> > letter or in Documentation/, the cover letter merely states that this is
> > a "partial solution". The documentation of the function doesn't explain
> > what criteria the decision is based on. We need a proper explanation of
> > the problem in Documentation/, with a description of the behaviour (or
> > behaviours) drivers are expected to implement.
> 
> I believe this is discussed in the cover letter, not in detail though. I'm
> fine with adding more documentation, there isn't much as I also wanted to
> get feedback on the approach itself.

But that's my point, I have trouble giving feedback on the approach if
you don't explain what problem you're trying to solve, and why you
picked this particular solution. It may be all clear in your head, but
it isn't for me.

> No alternatives have been proposed so
> far either.
> 
> > Furthermore, on the implementation side, things are fairly inefficient.
> > We already traverse the whole pipeline in media_pipeline_start(), based
> > on links and routes, and populate the media_pipeline structure. We
> > shouldn't do the same here, but instead inspect media_pipeline to
> > extract the information we need. If you're missing information there,
> > let's add it.
> 
> The media pipeline is created and traversed, yes, but the media pipeline
> does not include streams which are strictly a V4L2 concept. I agree there
> is some overlap between the two but as long as MC remains separate from
> V4L2, we can't reasonably access streams from the pipeline traversal.
> 
> > > + *
> > > + * Return:
> > > + * * 0: Success, but don't start streaming yet
> > > + * * 1: Success, now it's time to start streaming
> > > + * * -ENXIO: Route traversal encountered a non-video device/sub-device entity
> > > + * * -ENOTUNIQ: No unique remote pad
> > > + * * -ENOLINK: No remote pad found
> > > + * * -ENOENT: Enabled upstream route not found
> > > + * * -EMLINK: No unique downstream route found
> > > + * * -EINVAL: Stream could not be followed to source or was not produced by
> > > + *            the source
> > > + */
> > > +int v4l2_mc_pipeline_enabled(struct video_device *vdev,
> > > +			     bool (*func)(struct video_device *vdev),
> > > +			     struct media_pad **__sink_pad,
> > > +			     u64 *__sink_streams);
> > > +
> > >  #else /* CONFIG_MEDIA_CONTROLLER */
> > >  
> > >  static inline int v4l2_mc_create_media_graph(struct media_device *mdev)

-- 
Regards,

Laurent Pinchart