[PATCH 0/3] media: imx-csi: cleanup media pipeline start

[PATCH 0/3] media: imx-csi: cleanup media pipeline start

[Michael Tretter]

The imx media device currently assumes that there is only a single media
pipeline. However, the media graph has multiple imx capture devices.
These may be started separately on media pipelines if they don't cause
conflicts in the media graph.

Move the media pipeline from the media device to the capture devices to
properly track and handle multiple media pipelines for the imx-csi.
Refactor the code to start the media pipeline from the driver to help
the reader.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
Michael Tretter (3):
      media: imx-csi: move media_pipeline to video device
      media: imx-csi: explicitly start media pipeline on pad 0
      media: imx-csi: use media_pad_is_streaming helper

 drivers/staging/media/imx/imx-media-capture.c |  8 ++++----
 drivers/staging/media/imx/imx-media-utils.c   | 12 ++++++++----
 drivers/staging/media/imx/imx-media.h         |  7 ++++---
 3 files changed, 16 insertions(+), 11 deletions(-)
---
base-commit: 1a31cb2782104969e48c7ce7e4fc06e403fcb5cf
change-id: 20251107-media-imx-cleanup-9022d941ae44

Best regards,
-- 
Michael Tretter <m.tretter@pengutronix.de>

[PATCH 1/3] media: imx-csi: move media_pipeline to video device

[Michael Tretter]

The imx-media driver has a single imx_media_device. Attaching the
media_pipeline to the imx_media_device prevents the execution of
multiple media pipelines on the device. This should be possible as long
as the media_pipelines don't use the same pads or pads that be
configured while the other media pipeline is streaming.

Move the media_pipeline to the imx_media_video_dev to be able to
construct media pipelines per imx capture device.

If different media pipelines in the media device conflict, the
validation will fail. Thus, the pipeline will fail to start and signal
an error to user space.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/staging/media/imx/imx-media-capture.c | 8 ++++----
 drivers/staging/media/imx/imx-media-utils.c   | 3 ++-
 drivers/staging/media/imx/imx-media.h         | 7 ++++---
 3 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/media/imx/imx-media-capture.c b/drivers/staging/media/imx/imx-media-capture.c
index e9cef7af000a..bfd71d25facc 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -722,8 +722,8 @@ static int capture_start_streaming(struct vb2_queue *vq, unsigned int count)
 		goto return_bufs;
 	}
 
-	ret = imx_media_pipeline_set_stream(priv->md, &priv->src_sd->entity,
-					    true);
+	ret = imx_media_pipeline_set_stream(priv->md, &priv->vdev,
+					    &priv->src_sd->entity, true);
 	if (ret) {
 		dev_err(priv->dev, "pipeline start failed with %d\n", ret);
 		goto return_bufs;
@@ -749,8 +749,8 @@ static void capture_stop_streaming(struct vb2_queue *vq)
 	unsigned long flags;
 	int ret;
 
-	ret = imx_media_pipeline_set_stream(priv->md, &priv->src_sd->entity,
-					    false);
+	ret = imx_media_pipeline_set_stream(priv->md, &priv->vdev,
+					    &priv->src_sd->entity, false);
 	if (ret)
 		dev_warn(priv->dev, "pipeline stop failed with %d\n", ret);
 
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 1b5af8945e6b..f520529a7cfe 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -749,6 +749,7 @@ EXPORT_SYMBOL_GPL(imx_media_pipeline_subdev);
  * Turn current pipeline streaming on/off starting from entity.
  */
 int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
+				  struct imx_media_video_dev *vdev,
 				  struct media_entity *entity,
 				  bool on)
 {
@@ -762,7 +763,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
 	mutex_lock(&imxmd->md.graph_mutex);
 
 	if (on) {
-		ret = __media_pipeline_start(entity->pads, &imxmd->pipe);
+		ret = __media_pipeline_start(entity->pads, &vdev->pipe);
 		if (ret)
 			goto out;
 		ret = v4l2_subdev_call(sd, video, s_stream, 1);
diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
index f095d9134fee..8b65f79b125f 100644
--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -104,6 +104,9 @@ struct imx_media_buffer {
 struct imx_media_video_dev {
 	struct video_device *vfd;
 
+	/* the pipeline object */
+	struct media_pipeline pipe;
+
 	/* the user format */
 	struct v4l2_pix_format fmt;
 	/* the compose rectangle */
@@ -145,9 +148,6 @@ struct imx_media_dev {
 	struct media_device md;
 	struct v4l2_device  v4l2_dev;
 
-	/* the pipeline object */
-	struct media_pipeline pipe;
-
 	struct mutex mutex; /* protect elements below */
 
 	/* master video device list */
@@ -223,6 +223,7 @@ int imx_media_alloc_dma_buf(struct device *dev,
 			    int size);
 
 int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
+				  struct imx_media_video_dev *vdev,
 				  struct media_entity *entity,
 				  bool on);
 

-- 
2.47.3

Re: [PATCH 1/3] media: imx-csi: move media_pipeline to video device

[Philipp Zabel]

On Fr, 2025-11-07 at 11:16 +0100, Michael Tretter wrote:
> The imx-media driver has a single imx_media_device. Attaching the
> media_pipeline to the imx_media_device prevents the execution of
> multiple media pipelines on the device. This should be possible as long
> as the media_pipelines don't use the same pads or pads that be
> configured while the other media pipeline is streaming.
> 
> Move the media_pipeline to the imx_media_video_dev to be able to
> construct media pipelines per imx capture device.
> 
> If different media pipelines in the media device conflict, the
> validation will fail. Thus, the pipeline will fail to start and signal
> an error to user space.
> 
> Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>

Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>

regards
Philipp

Re: [PATCH 1/3] media: imx-csi: move media_pipeline to video device

[Frank Li]

On Fri, Nov 07, 2025 at 11:16:33AM +0100, Michael Tretter wrote:
> The imx-media driver has a single imx_media_device. Attaching the
> media_pipeline to the imx_media_device prevents the execution of
> multiple media pipelines on the device. This should be possible as long
> as the media_pipelines don't use the same pads or pads that be
> configured while the other media pipeline is streaming.
>
> Move the media_pipeline to the imx_media_video_dev to be able to
> construct media pipelines per imx capture device.
>
> If different media pipelines in the media device conflict, the
> validation will fail. Thus, the pipeline will fail to start and signal
> an error to user space.
>
> Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
> ---

Reviewed-by: Frank Li <Frank.Li@nxp.com>

>  drivers/staging/media/imx/imx-media-capture.c | 8 ++++----
>  drivers/staging/media/imx/imx-media-utils.c   | 3 ++-
>  drivers/staging/media/imx/imx-media.h         | 7 ++++---
>  3 files changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/staging/media/imx/imx-media-capture.c b/drivers/staging/media/imx/imx-media-capture.c
> index e9cef7af000a..bfd71d25facc 100644
> --- a/drivers/staging/media/imx/imx-media-capture.c
> +++ b/drivers/staging/media/imx/imx-media-capture.c
> @@ -722,8 +722,8 @@ static int capture_start_streaming(struct vb2_queue *vq, unsigned int count)
>  		goto return_bufs;
>  	}
>
> -	ret = imx_media_pipeline_set_stream(priv->md, &priv->src_sd->entity,
> -					    true);
> +	ret = imx_media_pipeline_set_stream(priv->md, &priv->vdev,
> +					    &priv->src_sd->entity, true);
>  	if (ret) {
>  		dev_err(priv->dev, "pipeline start failed with %d\n", ret);
>  		goto return_bufs;
> @@ -749,8 +749,8 @@ static void capture_stop_streaming(struct vb2_queue *vq)
>  	unsigned long flags;
>  	int ret;
>
> -	ret = imx_media_pipeline_set_stream(priv->md, &priv->src_sd->entity,
> -					    false);
> +	ret = imx_media_pipeline_set_stream(priv->md, &priv->vdev,
> +					    &priv->src_sd->entity, false);
>  	if (ret)
>  		dev_warn(priv->dev, "pipeline stop failed with %d\n", ret);
>
> diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
> index 1b5af8945e6b..f520529a7cfe 100644
> --- a/drivers/staging/media/imx/imx-media-utils.c
> +++ b/drivers/staging/media/imx/imx-media-utils.c
> @@ -749,6 +749,7 @@ EXPORT_SYMBOL_GPL(imx_media_pipeline_subdev);
>   * Turn current pipeline streaming on/off starting from entity.
>   */
>  int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
> +				  struct imx_media_video_dev *vdev,
>  				  struct media_entity *entity,
>  				  bool on)
>  {
> @@ -762,7 +763,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
>  	mutex_lock(&imxmd->md.graph_mutex);
>
>  	if (on) {
> -		ret = __media_pipeline_start(entity->pads, &imxmd->pipe);
> +		ret = __media_pipeline_start(entity->pads, &vdev->pipe);
>  		if (ret)
>  			goto out;
>  		ret = v4l2_subdev_call(sd, video, s_stream, 1);
> diff --git a/drivers/staging/media/imx/imx-media.h b/drivers/staging/media/imx/imx-media.h
> index f095d9134fee..8b65f79b125f 100644
> --- a/drivers/staging/media/imx/imx-media.h
> +++ b/drivers/staging/media/imx/imx-media.h
> @@ -104,6 +104,9 @@ struct imx_media_buffer {
>  struct imx_media_video_dev {
>  	struct video_device *vfd;
>
> +	/* the pipeline object */
> +	struct media_pipeline pipe;
> +
>  	/* the user format */
>  	struct v4l2_pix_format fmt;
>  	/* the compose rectangle */
> @@ -145,9 +148,6 @@ struct imx_media_dev {
>  	struct media_device md;
>  	struct v4l2_device  v4l2_dev;
>
> -	/* the pipeline object */
> -	struct media_pipeline pipe;
> -
>  	struct mutex mutex; /* protect elements below */
>
>  	/* master video device list */
> @@ -223,6 +223,7 @@ int imx_media_alloc_dma_buf(struct device *dev,
>  			    int size);
>
>  int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
> +				  struct imx_media_video_dev *vdev,
>  				  struct media_entity *entity,
>  				  bool on);
>
>
> --
> 2.47.3
>

[PATCH 2/3] media: imx-csi: explicitly start media pipeline on pad 0

[Michael Tretter]

entity->pads is an array that contains all the pads of an entity.

Calling __media_pipeline_start() or __media_pipeline_stop() on the pads,
implicitly starts the pipeline with the first pad in this array as
origin.

Explicitly use the first pad to start the pipeline to make this more
obvious to the reader.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/staging/media/imx/imx-media-utils.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index f520529a7cfe..6d69f69c16c1 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -755,6 +755,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
 {
 	struct v4l2_subdev *sd;
 	int ret = 0;
+	struct media_pad *pad;
 
 	if (!is_media_entity_v4l2_subdev(entity))
 		return -EINVAL;
@@ -762,17 +763,19 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
 
 	mutex_lock(&imxmd->md.graph_mutex);
 
+	pad = &entity->pads[0];
+
 	if (on) {
-		ret = __media_pipeline_start(entity->pads, &vdev->pipe);
+		ret = __media_pipeline_start(pad, &vdev->pipe);
 		if (ret)
 			goto out;
 		ret = v4l2_subdev_call(sd, video, s_stream, 1);
 		if (ret)
-			__media_pipeline_stop(entity->pads);
+			__media_pipeline_stop(pad);
 	} else {
 		v4l2_subdev_call(sd, video, s_stream, 0);
-		if (media_pad_pipeline(entity->pads))
-			__media_pipeline_stop(entity->pads);
+		if (media_pad_pipeline(pad))
+			__media_pipeline_stop(pad);
 	}
 
 out:

-- 
2.47.3

Re: [PATCH 2/3] media: imx-csi: explicitly start media pipeline on pad 0

[Philipp Zabel]

On Fr, 2025-11-07 at 11:16 +0100, Michael Tretter wrote:
> entity->pads is an array that contains all the pads of an entity.
> 
> Calling __media_pipeline_start() or __media_pipeline_stop() on the pads,
> implicitly starts the pipeline with the first pad in this array as
> origin.
> 
> Explicitly use the first pad to start the pipeline to make this more
> obvious to the reader.
> 
> Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>

Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>

regards
Philipp

Re: [PATCH 2/3] media: imx-csi: explicitly start media pipeline on pad 0

[Frank Li]

On Fri, Nov 07, 2025 at 11:16:34AM +0100, Michael Tretter wrote:
> entity->pads is an array that contains all the pads of an entity.
>
> Calling __media_pipeline_start() or __media_pipeline_stop() on the pads,
> implicitly starts the pipeline with the first pad in this array as
> origin.
>
> Explicitly use the first pad to start the pipeline to make this more
> obvious to the reader.
>
> Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
> ---
>  drivers/staging/media/imx/imx-media-utils.c | 11 +++++++----
>  1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
> index f520529a7cfe..6d69f69c16c1 100644
> --- a/drivers/staging/media/imx/imx-media-utils.c
> +++ b/drivers/staging/media/imx/imx-media-utils.c
> @@ -755,6 +755,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
>  {
>  	struct v4l2_subdev *sd;
>  	int ret = 0;
> +	struct media_pad *pad;

nit: move struct media_pad *pad; before int ret = 0; for better read

Reviewed-by: Frank Li <Frank.Li@nxp.com>
>
>  	if (!is_media_entity_v4l2_subdev(entity))
>  		return -EINVAL;
> @@ -762,17 +763,19 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
>
>  	mutex_lock(&imxmd->md.graph_mutex);
>
> +	pad = &entity->pads[0];
> +
>  	if (on) {
> -		ret = __media_pipeline_start(entity->pads, &vdev->pipe);
> +		ret = __media_pipeline_start(pad, &vdev->pipe);
>  		if (ret)
>  			goto out;
>  		ret = v4l2_subdev_call(sd, video, s_stream, 1);
>  		if (ret)
> -			__media_pipeline_stop(entity->pads);
> +			__media_pipeline_stop(pad);
>  	} else {
>  		v4l2_subdev_call(sd, video, s_stream, 0);
> -		if (media_pad_pipeline(entity->pads))
> -			__media_pipeline_stop(entity->pads);
> +		if (media_pad_pipeline(pad))
> +			__media_pipeline_stop(pad);
>  	}
>
>  out:
>
> --
> 2.47.3
>

[PATCH 3/3] media: imx-csi: use media_pad_is_streaming helper

[Michael Tretter]

The media_pad_is_streaming helper is explicitly for checking, if a pad
has been started with media_pipeline_start. Use it instead of relying on
the implicit knowledge, that a pad that has a pipeline is streaming.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/staging/media/imx/imx-media-utils.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
index 6d69f69c16c1..4ea9b3fc59d0 100644
--- a/drivers/staging/media/imx/imx-media-utils.c
+++ b/drivers/staging/media/imx/imx-media-utils.c
@@ -774,7 +774,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
 			__media_pipeline_stop(pad);
 	} else {
 		v4l2_subdev_call(sd, video, s_stream, 0);
-		if (media_pad_pipeline(pad))
+		if (media_pad_is_streaming(pad))
 			__media_pipeline_stop(pad);
 	}
 

-- 
2.47.3

Re: [PATCH 3/3] media: imx-csi: use media_pad_is_streaming helper

[Philipp Zabel]

On Fr, 2025-11-07 at 11:16 +0100, Michael Tretter wrote:
> The media_pad_is_streaming helper is explicitly for checking, if a pad
> has been started with media_pipeline_start. Use it instead of relying on
> the implicit knowledge, that a pad that has a pipeline is streaming.
> 
> Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>

Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>

regards
Philipp

Re: [PATCH 3/3] media: imx-csi: use media_pad_is_streaming helper

[Frank Li]

On Fri, Nov 07, 2025 at 11:16:35AM +0100, Michael Tretter wrote:
> The media_pad_is_streaming helper is explicitly for checking, if a pad
> has been started with media_pipeline_start. Use it instead of relying on
> the implicit knowledge, that a pad that has a pipeline is streaming.

Suggested commit message:

The media_pad_is_streaming() helper is explicitly intended to check
whether a pad has been started with media_pipeline_start(). Use it
instead of relying on the implicit assumption that a pad with a
pipeline is streaming.

Frank

>
> Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
> ---
>  drivers/staging/media/imx/imx-media-utils.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c
> index 6d69f69c16c1..4ea9b3fc59d0 100644
> --- a/drivers/staging/media/imx/imx-media-utils.c
> +++ b/drivers/staging/media/imx/imx-media-utils.c
> @@ -774,7 +774,7 @@ int imx_media_pipeline_set_stream(struct imx_media_dev *imxmd,
>  			__media_pipeline_stop(pad);
>  	} else {
>  		v4l2_subdev_call(sd, video, s_stream, 0);
> -		if (media_pad_pipeline(pad))
> +		if (media_pad_is_streaming(pad))
>  			__media_pipeline_stop(pad);
>  	}
>
>
> --
> 2.47.3
>