[PATCH v4 00/15] media: rcar: Streams support

[PATCH v4 00/15] media: rcar: Streams support

[Tomi Valkeinen]

Add streams support to Renesas rcar platform driver.

The series keaps compatibility with the current upstream for a single
stream use case. However, in upstream there's a limited custom
multi-stream support implemented to the rcar driver, which will be
replaced with the upstream's Streams API.

I have tested this series on Sparrow-Hawk board, with a few different
setups:

IMX219 connected to the CSI0 connector
- The following patches applied to my test branch in addition to this
  series:
  1) The v4l2_subdev_get_frame_desc_passthrough dependency
  2) Revert of commit e7376745ad5c8548e31d9ea58adfb5a847e017a4 ("media:
     rcar-vin: Fix stride setting for RAW8 formats"), as that commit
     breaks RAW8
- Tested with a single video stream

IMX219 connected to the CSI0 connector
- Plenty of other patches applied to enable full streams support and
  embedded data support in imx219 and v4l2 framework
- Tested with video and embedded data streams
 
Arducam FPD-Link board + 4 x IMX219 connected to the CSI0 connector
- Plenty of other patches applied to enable full streams support and
  embedded data support in imx219 and v4l2 framework, and TPG support in
  ub953
- Tested with video and embedded data streams from all four cameras (so
  8 streams in total)
- Also tested with ub953's TPG, combined with video & embedded streams
  from other cameras.

I have observed one issue with the embedded data (i.e. requiring bunch
of patches not in upstream): when stopping streaming, VIN says that it
cannot stop the stream. I haven't debugged that, but a possible issue is
that the if the video stream for the imx219 is stopped first, the
embedded data stops also, and VIN does not get the frame-end it is
waiting for.

 Tomi

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
Changes in v4:
- Rebased on v6.18, with minor conflicts resolved
- Improved patch descriptions
- Re-ordered the patches a bit to move changes that could be applied
  without the full streams support to the beginning of the series
- Added "media: rcar-vin: Link VINs on Gen3 to a single channel on each
  CSI-2" which removes possibility of wrong routing config on Gen3
- Added "media: rcar-csi2: Improve FLD_FLD_EN macros" which was part of
  another patch in v3
- Addressed minor comments (constifyings, cosmetics)
- Fixed the missing stream_count checks in disable_streams ops
- Fixed a few instances in csisp and csi2 where
  v4l2_subdev_state_get_format() was called with hardcoded pad/stream,
  instead of using the data from the route
- Dropped unnecessary ISPPROCMODE_DT_REG register clears
- Squashed "media: rcar-csi2: Add more stream support to
  rcsi2_calc_mbps()" into a previous patch
- Dropped wrong use_isp check from csi2's rcsi2_set_routing()
- Link to v3: https://lore.kernel.org/r/20250530-rcar-streams-v3-0-026655df7138@ideasonboard.com

Changes in v3:
- Rebased on top of latest linux-media
- Dropped dependencies which are already in linux-media (only remaining
  dependency is v4l2_subdev_get_frame_desc_passthrough)
- Tested on white-hawk board, using the staging deser TPG
- Also tested in a WIP branch for GMSL2 (two video streams)
- Link to v2: https://lore.kernel.org/r/20250326-rcar-streams-v2-0-d0d7002c641f@ideasonboard.com

Changes in v2:
- Rebased on top of latest upstream, and updated the dependencies to
  match the latest serieses sent.
- Add new patch "media: rcar-csi2: Use the pad version of v4l2_get_link_freq()"
- Drop "media: rcar-csi2: Fix typo" (it was not a typo)
- Update the code in calc_mbps(). The previous method relied on
  V4L2_CID_LINK_FREQ, but that's not available if the link-freq is
  provided via get_mbus_config().
- Dropped dependencies to Niklas' old series which doesn't apply
  cleanly. It's needed for multi-stream, but not for the current
  upstream which only has a single stream use case.
- Link to v1: https://lore.kernel.org/r/20250219-rcar-streams-v1-0-f1b93e370aab@ideasonboard.com

---
Tomi Valkeinen (15):
      media: rcar-isp: Improve ISPPROCMODE_DT_PROC_MODE_VC
      media: rcar-csi2: Improve FLD_FLD_EN macros
      media: rcar-csi2: Move rcsi2_calc_mbps()
      media: rcar-csi2: Simplify rcsi2_calc_mbps()
      media: rcar-csi2: Optimize rcsi2_calc_mbps()
      media: rcar-vin: Link VINs on Gen3 to a single channel on each CSI-2
      media: rcar-isp: Move {enable|disable}_streams() calls
      media: rcar-csi2: Move {enable|disable}_streams() calls
      media: rcar-csi2: Switch to Streams API
      media: rcar-isp: Switch to Streams API
      media: rcar-csi2: Add .get_frame_desc op
      media: rcar-isp: Call get_frame_desc to find out VC & DT
      media: rcar-csi2: Call get_frame_desc to find out VC & DT (Gen3)
      media: rcar-csi2: Add full streams support
      media: rcar-isp: Add full streams support

 drivers/media/platform/renesas/rcar-csi2.c         | 437 +++++++++++++++------
 drivers/media/platform/renesas/rcar-isp/csisp.c    | 232 ++++++++---
 .../media/platform/renesas/rcar-vin/rcar-core.c    |  27 +-
 3 files changed, 509 insertions(+), 187 deletions(-)
---
base-commit: f7b88edb52c8dd01b7e576390d658ae6eef0e134
change-id: 20250219-rcar-streams-1fdea8860e5e
prerequisite-change-id: 20250218-frame-desc-passthrough-66805e413974:v4
prerequisite-patch-id: bce4a915a29a64f88ed1bb600c08df37d2ba20c6
prerequisite-patch-id: 69b75e7dad9ced905cb39a72f18bebbf3e8f998a
prerequisite-patch-id: 58463f6944c76acd6cf203b14a2836cdb0db2461

Best regards,
-- 
Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>

[PATCH v4 01/15] media: rcar-isp: Improve ISPPROCMODE_DT_PROC_MODE_VC

[Tomi Valkeinen]

Instead of having four macros for ISPPROCMODE_DT_PROC_MODE_VC[0123](pm),
have just one ISPPROCMODE_DT_PROC_MODE_VCn(vc, pm).

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/renesas/rcar-isp/csisp.c | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-isp/csisp.c b/drivers/media/platform/renesas/rcar-isp/csisp.c
index 1eb29a0b774a..8fb2cc3b5650 100644
--- a/drivers/media/platform/renesas/rcar-isp/csisp.c
+++ b/drivers/media/platform/renesas/rcar-isp/csisp.c
@@ -28,10 +28,7 @@
 #define ISPSTART_STOP					0x0000
 
 #define ISPPROCMODE_DT_REG(n)				(0x1100 + (0x4 * (n)))
-#define ISPPROCMODE_DT_PROC_MODE_VC3(pm)		(((pm) & 0x3f) << 24)
-#define ISPPROCMODE_DT_PROC_MODE_VC2(pm)		(((pm) & 0x3f) << 16)
-#define ISPPROCMODE_DT_PROC_MODE_VC1(pm)		(((pm) & 0x3f) << 8)
-#define ISPPROCMODE_DT_PROC_MODE_VC0(pm)		((pm) & 0x3f)
+#define ISPPROCMODE_DT_PROC_MODE_VCn(vc, pm)		(((pm) & 0x3f) << (8 * (vc)))
 
 #define ISPCS_FILTER_ID_CH_REG(n)			(0x3000 + (0x0100 * (n)))
 
@@ -263,10 +260,10 @@ static int risp_start(struct rcar_isp *isp, struct v4l2_subdev_state *state)
 
 	/* Setup processing method. */
 	risp_write_cs(isp, ISPPROCMODE_DT_REG(format->datatype),
-		      ISPPROCMODE_DT_PROC_MODE_VC3(format->procmode) |
-		      ISPPROCMODE_DT_PROC_MODE_VC2(format->procmode) |
-		      ISPPROCMODE_DT_PROC_MODE_VC1(format->procmode) |
-		      ISPPROCMODE_DT_PROC_MODE_VC0(format->procmode));
+		      ISPPROCMODE_DT_PROC_MODE_VCn(3, format->procmode) |
+		      ISPPROCMODE_DT_PROC_MODE_VCn(2, format->procmode) |
+		      ISPPROCMODE_DT_PROC_MODE_VCn(1, format->procmode) |
+		      ISPPROCMODE_DT_PROC_MODE_VCn(0, format->procmode));
 
 	/* Start ISP. */
 	risp_write_cs(isp, ISPSTART_REG, ISPSTART_START);

-- 
2.43.0

[PATCH v4 02/15] media: rcar-csi2: Improve FLD_FLD_EN macros

[Tomi Valkeinen]

Instead of having four macros for FLD_FLD_EN for different channels,
have just one FLD_FLD_EN(ch).

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-csi2.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index d1b31ab8b8c4..3cd35adb6803 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -71,10 +71,7 @@ struct rcar_csi2;
 #define FLD_REG				0x1c
 #define FLD_FLD_NUM(n)			(((n) & 0xff) << 16)
 #define FLD_DET_SEL(n)			(((n) & 0x3) << 4)
-#define FLD_FLD_EN4			BIT(3)
-#define FLD_FLD_EN3			BIT(2)
-#define FLD_FLD_EN2			BIT(1)
-#define FLD_FLD_EN			BIT(0)
+#define FLD_FLD_EN(ch)			BIT(ch)
 
 /* Automatic Standby Control */
 #define ASTBY_REG			0x20
@@ -1082,8 +1079,8 @@ static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
 	}
 
 	if (fmt->field == V4L2_FIELD_ALTERNATE)
-		fld = FLD_DET_SEL(1) | FLD_FLD_EN4 | FLD_FLD_EN3 | FLD_FLD_EN2
-			| FLD_FLD_EN;
+		fld = FLD_DET_SEL(1) | FLD_FLD_EN(3) | FLD_FLD_EN(2) |
+		      FLD_FLD_EN(1) | FLD_FLD_EN(0);
 
 	/*
 	 * Get the number of active data lanes inspecting the remote mbus

-- 
2.43.0

[PATCH v4 03/15] media: rcar-csi2: Move rcsi2_calc_mbps()

[Tomi Valkeinen]

Move the function so that it can call rcsi2_get_active_lanes() in the
following patch.

No functional change intended.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/renesas/rcar-csi2.c | 58 +++++++++++++++---------------
 1 file changed, 29 insertions(+), 29 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index 3cd35adb6803..8032fa4f7a8a 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -956,35 +956,6 @@ static int rcsi2_set_phypll(struct rcar_csi2 *priv, unsigned int mbps)
 	return 0;
 }
 
-static int rcsi2_calc_mbps(struct rcar_csi2 *priv, unsigned int bpp,
-			   unsigned int lanes)
-{
-	struct media_pad *remote_pad;
-	struct v4l2_subdev *source;
-	s64 freq;
-	u64 mbps;
-
-	if (!priv->remote)
-		return -ENODEV;
-
-	source = priv->remote;
-	remote_pad = &source->entity.pads[priv->remote_pad];
-
-	freq = v4l2_get_link_freq(remote_pad, bpp, 2 * lanes);
-	if (freq < 0) {
-		int ret = (int)freq;
-
-		dev_err(priv->dev, "failed to get link freq for %s: %d\n",
-			source->name, ret);
-
-		return ret;
-	}
-
-	mbps = div_u64(freq * 2, MEGA);
-
-	return mbps;
-}
-
 static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
 				  unsigned int *lanes)
 {
@@ -1032,6 +1003,35 @@ static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
 	return 0;
 }
 
+static int rcsi2_calc_mbps(struct rcar_csi2 *priv, unsigned int bpp,
+			   unsigned int lanes)
+{
+	struct media_pad *remote_pad;
+	struct v4l2_subdev *source;
+	s64 freq;
+	u64 mbps;
+
+	if (!priv->remote)
+		return -ENODEV;
+
+	source = priv->remote;
+	remote_pad = &source->entity.pads[priv->remote_pad];
+
+	freq = v4l2_get_link_freq(remote_pad, bpp, 2 * lanes);
+	if (freq < 0) {
+		int ret = (int)freq;
+
+		dev_err(priv->dev, "failed to get link freq for %s: %d\n",
+			source->name, ret);
+
+		return ret;
+	}
+
+	mbps = div_u64(freq * 2, MEGA);
+
+	return mbps;
+}
+
 static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
 				     struct v4l2_subdev_state *state)
 {

-- 
2.43.0

[PATCH v4 04/15] media: rcar-csi2: Simplify rcsi2_calc_mbps()

[Tomi Valkeinen]

Instead of taking the bpp and the number of lanes as parameters to
rcsi2_calc_mbps(), change the function to get those parameters inside
the function. This centralizes the code a bit and makes it easier to add
streams support.

Also, in the future when the legacy (non-link-freq) code is removed,
there will be no need to change rcsi2_calc_mbps() parameters.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
---
 drivers/media/platform/renesas/rcar-csi2.c | 45 ++++++++++++++++--------------
 1 file changed, 24 insertions(+), 21 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index 8032fa4f7a8a..a2a87c5bfd7c 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -1003,13 +1003,18 @@ static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
 	return 0;
 }
 
-static int rcsi2_calc_mbps(struct rcar_csi2 *priv, unsigned int bpp,
-			   unsigned int lanes)
+static int rcsi2_calc_mbps(struct rcar_csi2 *priv,
+			   struct v4l2_subdev_state *state)
 {
+	const struct rcar_csi2_format *format;
+	struct v4l2_mbus_framefmt *fmt;
 	struct media_pad *remote_pad;
 	struct v4l2_subdev *source;
+	unsigned int lanes;
+	unsigned int bpp;
 	s64 freq;
 	u64 mbps;
+	int ret;
 
 	if (!priv->remote)
 		return -ENODEV;
@@ -1017,6 +1022,20 @@ static int rcsi2_calc_mbps(struct rcar_csi2 *priv, unsigned int bpp,
 	source = priv->remote;
 	remote_pad = &source->entity.pads[priv->remote_pad];
 
+	ret = rcsi2_get_active_lanes(priv, &lanes);
+	if (ret)
+		return ret;
+
+	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
+	if (!fmt)
+		return -EINVAL;
+
+	format = rcsi2_code_to_fmt(fmt->code);
+	if (!format)
+		return -EINVAL;
+
+	bpp = format->bpp;
+
 	freq = v4l2_get_link_freq(remote_pad, bpp, 2 * lanes);
 	if (freq < 0) {
 		int ret = (int)freq;
@@ -1093,7 +1112,7 @@ static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
 	phycnt = PHYCNT_ENABLECLK;
 	phycnt |= (1 << lanes) - 1;
 
-	mbps = rcsi2_calc_mbps(priv, format->bpp, lanes);
+	mbps = rcsi2_calc_mbps(priv, state);
 	if (mbps < 0)
 		return mbps;
 
@@ -1475,23 +1494,15 @@ static int rcsi2_start_receiver_v4h(struct rcar_csi2 *priv,
 				    struct v4l2_subdev_state *state)
 {
 	const struct rcsi2_cphy_setting *cphy = NULL;
-	const struct rcar_csi2_format *format;
-	const struct v4l2_mbus_framefmt *fmt;
 	unsigned int lanes;
 	int mbps;
 	int ret;
 
-	/* Use the format on the sink pad to compute the receiver config. */
-	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
-	format = rcsi2_code_to_fmt(fmt->code);
-	if (!format)
-		return -EINVAL;
-
 	ret = rcsi2_get_active_lanes(priv, &lanes);
 	if (ret)
 		return ret;
 
-	mbps = rcsi2_calc_mbps(priv, format->bpp, lanes);
+	mbps = rcsi2_calc_mbps(priv, state);
 	if (mbps < 0)
 		return mbps;
 
@@ -1732,23 +1743,15 @@ static int rcsi2_init_common_v4m(struct rcar_csi2 *priv, unsigned int mbps)
 static int rcsi2_start_receiver_v4m(struct rcar_csi2 *priv,
 				    struct v4l2_subdev_state *state)
 {
-	const struct rcar_csi2_format *format;
-	const struct v4l2_mbus_framefmt *fmt;
 	unsigned int lanes;
 	int mbps;
 	int ret;
 
-	/* Calculate parameters */
-	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
-	format = rcsi2_code_to_fmt(fmt->code);
-	if (!format)
-		return -EINVAL;
-
 	ret = rcsi2_get_active_lanes(priv, &lanes);
 	if (ret)
 		return ret;
 
-	mbps = rcsi2_calc_mbps(priv, format->bpp, lanes);
+	mbps = rcsi2_calc_mbps(priv, state);
 	if (mbps < 0)
 		return mbps;
 

-- 
2.43.0

[PATCH v4 05/15] media: rcar-csi2: Optimize rcsi2_calc_mbps()

[Tomi Valkeinen]

With modern drivers supporting link-freq, we don't need to do any
calculations based on the bpp and number of lanes when figuring out the
link frequency. However, the code currently always runs code to get the
bpp and number of lanes.

Optimize the rcsi2_calc_mbps() so that we only do that when needed, i.e.
when querying the link-freq is not supported by the upstream subdevice.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-csi2.c | 50 +++++++++++++++++-------------
 1 file changed, 29 insertions(+), 21 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index a2a87c5bfd7c..7305cc4a04cb 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -1006,15 +1006,10 @@ static int rcsi2_get_active_lanes(struct rcar_csi2 *priv,
 static int rcsi2_calc_mbps(struct rcar_csi2 *priv,
 			   struct v4l2_subdev_state *state)
 {
-	const struct rcar_csi2_format *format;
-	struct v4l2_mbus_framefmt *fmt;
 	struct media_pad *remote_pad;
 	struct v4l2_subdev *source;
-	unsigned int lanes;
-	unsigned int bpp;
 	s64 freq;
 	u64 mbps;
-	int ret;
 
 	if (!priv->remote)
 		return -ENODEV;
@@ -1022,28 +1017,41 @@ static int rcsi2_calc_mbps(struct rcar_csi2 *priv,
 	source = priv->remote;
 	remote_pad = &source->entity.pads[priv->remote_pad];
 
-	ret = rcsi2_get_active_lanes(priv, &lanes);
-	if (ret)
-		return ret;
+	/*
+	 * First try to get the real link freq. If that fails, try the heuristic
+	 * method with bpp and lanes (but that only works for one route).
+	 */
+	freq = v4l2_get_link_freq(remote_pad, 0, 0);
+	if (freq < 0) {
+		const struct rcar_csi2_format *format;
+		const struct v4l2_mbus_framefmt *fmt;
+		unsigned int lanes;
+		unsigned int bpp;
+		int ret;
 
-	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
-	if (!fmt)
-		return -EINVAL;
+		ret = rcsi2_get_active_lanes(priv, &lanes);
+		if (ret)
+			return ret;
 
-	format = rcsi2_code_to_fmt(fmt->code);
-	if (!format)
-		return -EINVAL;
+		fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
+		if (!fmt)
+			return -EINVAL;
 
-	bpp = format->bpp;
+		format = rcsi2_code_to_fmt(fmt->code);
+		if (!format)
+			return -EINVAL;
 
-	freq = v4l2_get_link_freq(remote_pad, bpp, 2 * lanes);
-	if (freq < 0) {
-		int ret = (int)freq;
+		bpp = format->bpp;
 
-		dev_err(priv->dev, "failed to get link freq for %s: %d\n",
-			source->name, ret);
+		freq = v4l2_get_link_freq(remote_pad, bpp, 2 * lanes);
+		if (freq < 0) {
+			int ret = (int)freq;
 
-		return ret;
+			dev_err(priv->dev, "failed to get link freq for %s: %d\n",
+				source->name, ret);
+
+			return ret;
+		}
 	}
 
 	mbps = div_u64(freq * 2, MEGA);

-- 
2.43.0

[PATCH v4 06/15] media: rcar-vin: Link VINs on Gen3 to a single channel on each CSI-2

[Tomi Valkeinen]

The stream routing will be fully configurable when we add full streams
support to the rcar pipeline. Thus there is no need for the user to be
able to link VINs to different CSI-2 channels. In fact, allowing e.g.
VIN0 to be connected to CSI-2 channel 3 would be wrong, as the hardware
doesn't allow that.

Change rvin_csi2_create_link() so that it creates media links only
between matching VINs and CSI-2 channels (VIN0 - channel 0, VIN1 -
channel 1, etc.).

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 .../media/platform/renesas/rcar-vin/rcar-core.c    | 27 ++++++++++++----------
 1 file changed, 15 insertions(+), 12 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-vin/rcar-core.c b/drivers/media/platform/renesas/rcar-vin/rcar-core.c
index 100105b620e3..acaf22d1f450 100644
--- a/drivers/media/platform/renesas/rcar-vin/rcar-core.c
+++ b/drivers/media/platform/renesas/rcar-vin/rcar-core.c
@@ -673,23 +673,26 @@ static int rvin_csi2_create_link(struct rvin_group *group, unsigned int id,
 	struct media_entity *source = &group->remotes[route->csi].subdev->entity;
 	struct media_entity *sink = &group->vin[id]->vdev.entity;
 	struct media_pad *sink_pad = &sink->pads[0];
+	struct media_pad *source_pad;
+	unsigned int source_idx;
 	unsigned int channel;
-	int ret;
 
-	for (channel = 0; channel < 4; channel++) {
-		unsigned int source_idx = rvin_group_csi_channel_to_pad(channel);
-		struct media_pad *source_pad = &source->pads[source_idx];
+	/*
+	 * The channels from CSI-2 blocks and the VIN groups have a set of
+	 * hardcoded routing options to choose from. We only support the routing
+	 * where all VINs in a group are connected to the same CSI-2 block,
+	 * and the Nth VIN in the group is connected to the Nth CSI-2 channel.
+	 */
 
-		/* Skip if link already exists. */
-		if (media_entity_find_link(source_pad, sink_pad))
-			continue;
+	channel = id % 4;
+	source_idx = rvin_group_csi_channel_to_pad(channel);
+	source_pad = &source->pads[source_idx];
 
-		ret = media_create_pad_link(source, source_idx, sink, 0, 0);
-		if (ret)
-			return ret;
-	}
+	/* Skip if link already exists. */
+	if (media_entity_find_link(source_pad, sink_pad))
+		return 0;
 
-	return 0;
+	return media_create_pad_link(source, source_idx, sink, 0, 0);
 }
 
 static int rvin_parallel_setup_links(struct rvin_group *group)

-- 
2.43.0

[PATCH v4 07/15] media: rcar-isp: Move {enable|disable}_streams() calls

[Tomi Valkeinen]

With multiple streams the operation to enable the ISP hardware and to
call {enable|disable}_streams() on upstream subdev will need to be
handled separately.

Prepare for that by moving {enable|disable}_streams() calls out from
risp_start() and risp_stop().

On Gen4, a side effect of this change is that if the sink side devices
call .enable_streams() on rcar-isp multiple times, the second call will
fail. This is because we always use stream ID 0, so the second call
would attempt to enable the same stream again, leading to an error. In
other words, a normal single-stream setup continues to work, but trying
to use the current driver's custom VC based routing will fail.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-isp/csisp.c | 27 ++++++++++++++++---------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-isp/csisp.c b/drivers/media/platform/renesas/rcar-isp/csisp.c
index 8fb2cc3b5650..58a9a3bd9f75 100644
--- a/drivers/media/platform/renesas/rcar-isp/csisp.c
+++ b/drivers/media/platform/renesas/rcar-isp/csisp.c
@@ -268,18 +268,11 @@ static int risp_start(struct rcar_isp *isp, struct v4l2_subdev_state *state)
 	/* Start ISP. */
 	risp_write_cs(isp, ISPSTART_REG, ISPSTART_START);
 
-	ret = v4l2_subdev_enable_streams(isp->remote, isp->remote_pad,
-					 BIT_ULL(0));
-	if (ret)
-		risp_power_off(isp);
-
-	return ret;
+	return 0;
 }
 
 static void risp_stop(struct rcar_isp *isp)
 {
-	v4l2_subdev_disable_streams(isp->remote, isp->remote_pad, BIT_ULL(0));
-
 	/* Stop ISP. */
 	risp_write_cs(isp, ISPSTART_REG, ISPSTART_STOP);
 
@@ -291,7 +284,7 @@ static int risp_enable_streams(struct v4l2_subdev *sd,
 			       u64 source_streams_mask)
 {
 	struct rcar_isp *isp = sd_to_isp(sd);
-	int ret = 0;
+	int ret;
 
 	if (source_streams_mask != 1)
 		return -EINVAL;
@@ -305,9 +298,17 @@ static int risp_enable_streams(struct v4l2_subdev *sd,
 			return ret;
 	}
 
+	ret = v4l2_subdev_enable_streams(isp->remote, isp->remote_pad,
+					 BIT_ULL(0));
+	if (ret) {
+		if (isp->stream_count == 0)
+			risp_stop(isp);
+		return ret;
+	}
+
 	isp->stream_count += 1;
 
-	return ret;
+	return 0;
 }
 
 static int risp_disable_streams(struct v4l2_subdev *sd,
@@ -315,6 +316,7 @@ static int risp_disable_streams(struct v4l2_subdev *sd,
 				u64 source_streams_mask)
 {
 	struct rcar_isp *isp = sd_to_isp(sd);
+	int ret;
 
 	if (source_streams_mask != 1)
 		return -EINVAL;
@@ -322,6 +324,11 @@ static int risp_disable_streams(struct v4l2_subdev *sd,
 	if (!isp->remote)
 		return -ENODEV;
 
+	ret = v4l2_subdev_disable_streams(isp->remote, isp->remote_pad,
+					  BIT_ULL(0));
+	if (ret)
+		return ret;
+
 	if (isp->stream_count == 1)
 		risp_stop(isp);
 

-- 
2.43.0

[PATCH v4 08/15] media: rcar-csi2: Move {enable|disable}_streams() calls

[Tomi Valkeinen]

With multiple streams the operation to enable the CSI-2 hardware and to
call {enable|disable}_streams() on upstream subdev will need to be
handled separately.

Prepare for that by moving {enable|disable}_streams() calls out from
rcsi2_start() and rcsi2_stop().

On Gen3, a side effect of this change is that if the sink side devices
call .enable_streams() on rcar-csi2 multiple times, the second call will
fail. This is because we always use stream ID 0, so the second call
would attempt to enable the same stream again, leading to an error. In
other words, a normal single-stream setup continues to work, but trying
to use the current driver's custom VC based routing will fail.

On Gen4, this doesn't matter as the rcar-isp behaves in a similar way as
described above, and thus rcar-csi2 will only get a single
.enable_streams() call.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-csi2.c | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index 7305cc4a04cb..158fa447e668 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -1822,20 +1822,12 @@ static int rcsi2_start(struct rcar_csi2 *priv, struct v4l2_subdev_state *state)
 		return ret;
 	}
 
-	ret = v4l2_subdev_enable_streams(priv->remote, priv->remote_pad,
-					 BIT_ULL(0));
-	if (ret) {
-		rcsi2_enter_standby(priv);
-		return ret;
-	}
-
 	return 0;
 }
 
 static void rcsi2_stop(struct rcar_csi2 *priv)
 {
 	rcsi2_enter_standby(priv);
-	v4l2_subdev_disable_streams(priv->remote, priv->remote_pad, BIT_ULL(0));
 }
 
 static int rcsi2_enable_streams(struct v4l2_subdev *sd,
@@ -1857,6 +1849,14 @@ static int rcsi2_enable_streams(struct v4l2_subdev *sd,
 			return ret;
 	}
 
+	ret = v4l2_subdev_enable_streams(priv->remote, priv->remote_pad,
+					 BIT_ULL(0));
+	if (ret) {
+		if (priv->stream_count == 0)
+			rcsi2_stop(priv);
+		return ret;
+	}
+
 	priv->stream_count += 1;
 
 	return ret;
@@ -1867,7 +1867,7 @@ static int rcsi2_disable_streams(struct v4l2_subdev *sd,
 				 u32 source_pad, u64 source_streams_mask)
 {
 	struct rcar_csi2 *priv = sd_to_csi2(sd);
-	int ret = 0;
+	int ret;
 
 	if (source_streams_mask != 1)
 		return -EINVAL;
@@ -1878,9 +1878,14 @@ static int rcsi2_disable_streams(struct v4l2_subdev *sd,
 	if (priv->stream_count == 1)
 		rcsi2_stop(priv);
 
+	ret = v4l2_subdev_disable_streams(priv->remote, priv->remote_pad,
+					  BIT_ULL(0));
+	if (ret)
+		return ret;
+
 	priv->stream_count -= 1;
 
-	return ret;
+	return 0;
 }
 
 static int rcsi2_set_pad_format(struct v4l2_subdev *sd,

-- 
2.43.0

[PATCH v4 09/15] media: rcar-csi2: Switch to Streams API

[Tomi Valkeinen]

Switch to Streams API with a single hardcoded route.

For single-stream use case there should be no change in behavior.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-csi2.c | 64 +++++++++++++++++++++++-------
 1 file changed, 50 insertions(+), 14 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index 158fa447e668..ad62c95c8f9a 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -1023,17 +1023,24 @@ static int rcsi2_calc_mbps(struct rcar_csi2 *priv,
 	 */
 	freq = v4l2_get_link_freq(remote_pad, 0, 0);
 	if (freq < 0) {
+		const struct v4l2_subdev_route *route;
 		const struct rcar_csi2_format *format;
 		const struct v4l2_mbus_framefmt *fmt;
 		unsigned int lanes;
 		unsigned int bpp;
 		int ret;
 
+		if (state->routing.num_routes != 1)
+			return -EINVAL;
+
 		ret = rcsi2_get_active_lanes(priv, &lanes);
 		if (ret)
 			return ret;
 
-		fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
+		route = &state->routing.routes[0];
+
+		fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
+						   route->sink_stream);
 		if (!fmt)
 			return -EINVAL;
 
@@ -1062,6 +1069,7 @@ static int rcsi2_calc_mbps(struct rcar_csi2 *priv,
 static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
 				     struct v4l2_subdev_state *state)
 {
+	const struct v4l2_subdev_route *route;
 	const struct rcar_csi2_format *format;
 	u32 phycnt, vcdt = 0, vcdt2 = 0, fld = 0;
 	const struct v4l2_mbus_framefmt *fmt;
@@ -1070,7 +1078,16 @@ static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
 	int mbps, ret;
 
 	/* Use the format on the sink pad to compute the receiver config. */
-	fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK);
+
+	if (state->routing.num_routes != 1)
+		return -EINVAL;
+
+	route = &state->routing.routes[0];
+
+	fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
+					   route->sink_stream);
+	if (!fmt)
+		return -EINVAL;
 
 	dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
 		fmt->width, fmt->height,
@@ -1892,8 +1909,7 @@ static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
 				struct v4l2_subdev_state *state,
 				struct v4l2_subdev_format *format)
 {
-	struct rcar_csi2 *priv = sd_to_csi2(sd);
-	unsigned int num_pads = rcsi2_num_pads(priv);
+	struct v4l2_mbus_framefmt *fmt;
 
 	if (format->pad > RCAR_CSI2_SINK)
 		return v4l2_subdev_get_fmt(sd, state, format);
@@ -1901,11 +1917,20 @@ static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
 	if (!rcsi2_code_to_fmt(format->format.code))
 		format->format.code = rcar_csi2_formats[0].code;
 
-	*v4l2_subdev_state_get_format(state, format->pad) = format->format;
+	/* Set sink format. */
+	fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+	if (!fmt)
+		return -EINVAL;
+
+	*fmt = format->format;
+
+	/* Propagate the format to the source pad. */
+	fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
+							   format->stream);
+	if (!fmt)
+		return -EINVAL;
 
-	/* Propagate the format to the source pads. */
-	for (unsigned int i = RCAR_CSI2_SOURCE_VC0; i < num_pads; i++)
-		*v4l2_subdev_state_get_format(state, i) = format->format;
+	*fmt = format->format;
 
 	return 0;
 }
@@ -1925,8 +1950,15 @@ static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {
 static int rcsi2_init_state(struct v4l2_subdev *sd,
 			    struct v4l2_subdev_state *state)
 {
-	struct rcar_csi2 *priv = sd_to_csi2(sd);
-	unsigned int num_pads = rcsi2_num_pads(priv);
+	static struct v4l2_subdev_route routes[] = {
+		{
+			.sink_pad = RCAR_CSI2_SINK,
+			.sink_stream = 0,
+			.source_pad = RCAR_CSI2_SOURCE_VC0,
+			.source_stream = 0,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+		},
+	};
 
 	static const struct v4l2_mbus_framefmt rcar_csi2_default_fmt = {
 		.width		= 1920,
@@ -1939,10 +1971,13 @@ static int rcsi2_init_state(struct v4l2_subdev *sd,
 		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
 	};
 
-	for (unsigned int i = RCAR_CSI2_SINK; i < num_pads; i++)
-		*v4l2_subdev_state_get_format(state, i) = rcar_csi2_default_fmt;
+	static const struct v4l2_subdev_krouting routing = {
+		.num_routes = ARRAY_SIZE(routes),
+		.routes = routes,
+	};
 
-	return 0;
+	return v4l2_subdev_set_routing_with_fmt(sd, state, &routing,
+						&rcar_csi2_default_fmt);
 }
 
 static const struct v4l2_subdev_internal_ops rcar_csi2_internal_ops = {
@@ -2599,7 +2634,8 @@ static int rcsi2_probe(struct platform_device *pdev)
 	v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
 	snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s",
 		 KBUILD_MODNAME, dev_name(&pdev->dev));
-	priv->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+	priv->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE |
+			     V4L2_SUBDEV_FL_STREAMS;
 
 	priv->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
 	priv->subdev.entity.ops = &rcar_csi2_entity_ops;

-- 
2.43.0

[PATCH v4 10/15] media: rcar-isp: Switch to Streams API

[Tomi Valkeinen]

Switch to Streams API with a single hardcoded route.

For single-stream use case there should be no change in behavior.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-isp/csisp.c | 69 ++++++++++++++++++++++---
 1 file changed, 62 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-isp/csisp.c b/drivers/media/platform/renesas/rcar-isp/csisp.c
index 58a9a3bd9f75..8ac45516aa39 100644
--- a/drivers/media/platform/renesas/rcar-isp/csisp.c
+++ b/drivers/media/platform/renesas/rcar-isp/csisp.c
@@ -124,6 +124,17 @@ static const struct rcar_isp_format rcar_isp_formats[] = {
 	},
 };
 
+static const struct v4l2_mbus_framefmt risp_default_fmt = {
+	.width = 1920,
+	.height = 1080,
+	.code = MEDIA_BUS_FMT_RGB888_1X24,
+	.colorspace = V4L2_COLORSPACE_SRGB,
+	.field = V4L2_FIELD_NONE,
+	.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT,
+	.quantization = V4L2_QUANTIZATION_DEFAULT,
+	.xfer_func = V4L2_XFER_FUNC_DEFAULT,
+};
+
 static const struct rcar_isp_format *risp_code_to_fmt(unsigned int code)
 {
 	unsigned int i;
@@ -216,13 +227,20 @@ static void risp_power_off(struct rcar_isp *isp)
 
 static int risp_start(struct rcar_isp *isp, struct v4l2_subdev_state *state)
 {
+	const struct v4l2_subdev_route *route;
 	const struct v4l2_mbus_framefmt *fmt;
 	const struct rcar_isp_format *format;
 	unsigned int vc;
 	u32 sel_csi = 0;
 	int ret;
 
-	fmt = v4l2_subdev_state_get_format(state, RCAR_ISP_SINK);
+	if (state->routing.num_routes != 1)
+		return -EINVAL;
+
+	route = &state->routing.routes[0];
+
+	fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
+					   route->sink_stream);
 	if (!fmt)
 		return -EINVAL;
 
@@ -341,7 +359,7 @@ static int risp_set_pad_format(struct v4l2_subdev *sd,
 			       struct v4l2_subdev_state *state,
 			       struct v4l2_subdev_format *format)
 {
-	struct v4l2_mbus_framefmt *framefmt;
+	struct v4l2_mbus_framefmt *fmt;
 
 	if (format->pad > RCAR_ISP_SINK)
 		return v4l2_subdev_get_fmt(sd, state, format);
@@ -349,10 +367,20 @@ static int risp_set_pad_format(struct v4l2_subdev *sd,
 	if (!risp_code_to_fmt(format->format.code))
 		format->format.code = rcar_isp_formats[0].code;
 
-	for (unsigned int i = 0; i < RCAR_ISP_NUM_PADS; i++) {
-		framefmt = v4l2_subdev_state_get_format(state, i);
-		*framefmt = format->format;
-	}
+	/* Set sink format. */
+	fmt = v4l2_subdev_state_get_format(state, format->pad, format->stream);
+	if (!fmt)
+		return -EINVAL;
+
+	*fmt = format->format;
+
+	/* Propagate the format to the source pad. */
+	fmt = v4l2_subdev_state_get_opposite_stream_format(state, format->pad,
+							   format->stream);
+	if (!fmt)
+		return -EINVAL;
+
+	*fmt = format->format;
 
 	return 0;
 }
@@ -369,6 +397,32 @@ static const struct v4l2_subdev_ops rcar_isp_subdev_ops = {
 	.pad	= &risp_pad_ops,
 };
 
+static int risp_init_state(struct v4l2_subdev *sd,
+			   struct v4l2_subdev_state *state)
+{
+	static struct v4l2_subdev_route routes[] = {
+		{
+			.sink_pad = RCAR_ISP_SINK,
+			.sink_stream = 0,
+			.source_pad = RCAR_ISP_PORT0,
+			.source_stream = 0,
+			.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+		},
+	};
+
+	static const struct v4l2_subdev_krouting routing = {
+		.num_routes = ARRAY_SIZE(routes),
+		.routes = routes,
+	};
+
+	return v4l2_subdev_set_routing_with_fmt(sd, state, &routing,
+						&risp_default_fmt);
+}
+
+static const struct v4l2_subdev_internal_ops risp_internal_ops = {
+	.init_state = risp_init_state,
+};
+
 /* -----------------------------------------------------------------------------
  * Async handling and registration of subdevices and links
  */
@@ -526,11 +580,12 @@ static int risp_probe(struct platform_device *pdev)
 
 	isp->subdev.owner = THIS_MODULE;
 	isp->subdev.dev = &pdev->dev;
+	isp->subdev.internal_ops = &risp_internal_ops;
 	v4l2_subdev_init(&isp->subdev, &rcar_isp_subdev_ops);
 	v4l2_set_subdevdata(&isp->subdev, &pdev->dev);
 	snprintf(isp->subdev.name, sizeof(isp->subdev.name), "%s %s",
 		 KBUILD_MODNAME, dev_name(&pdev->dev));
-	isp->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+	isp->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_STREAMS;
 
 	isp->subdev.entity.function = MEDIA_ENT_F_VID_MUX;
 	isp->subdev.entity.ops = &risp_entity_ops;

-- 
2.43.0

[PATCH v4 11/15] media: rcar-csi2: Add .get_frame_desc op

[Tomi Valkeinen]

Add v4l2_subdev_pad_ops.get_frame_desc() implementation.

We also implement a fallback for the case where the upstream subdevice
does not implement .get_frame_desc. It assumes a single stream with VC =
0 and DT based on the configured stream mbus format.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-csi2.c | 70 ++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index ad62c95c8f9a..b8baf7c65e90 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -1935,12 +1935,82 @@ static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int rcsi2_get_frame_desc_fallback(struct v4l2_subdev *sd,
+					 unsigned int pad,
+					 struct v4l2_mbus_frame_desc *fd)
+{
+	struct v4l2_subdev_route *route;
+	const struct rcar_csi2_format *format;
+	struct v4l2_subdev_state *state;
+	struct v4l2_mbus_framefmt *fmt;
+	int ret = 0;
+
+	state = v4l2_subdev_lock_and_get_active_state(sd);
+
+	if (state->routing.num_routes != 1) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	route = &state->routing.routes[0];
+
+	if (route->source_pad != pad) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
+					   route->sink_stream);
+	if (!fmt) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	format = rcsi2_code_to_fmt(fmt->code);
+	if (!format) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	fd->num_entries = 1;
+	fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
+	fd->entry[0].stream = route->source_stream;
+	fd->entry[0].pixelcode = fmt->code;
+	fd->entry[0].bus.csi2.vc = 0;
+	fd->entry[0].bus.csi2.dt = format->datatype;
+
+out:
+	v4l2_subdev_unlock_state(state);
+
+	return ret;
+}
+
+static int rcsi2_get_frame_desc(struct v4l2_subdev *sd, unsigned int pad,
+				struct v4l2_mbus_frame_desc *fd)
+{
+	struct rcar_csi2 *priv = sd_to_csi2(sd);
+	int ret;
+
+	if (WARN_ON(!priv->info->use_isp))
+		return -ENOTTY;
+
+	if (WARN_ON(pad != RCAR_CSI2_SOURCE_VC0))
+		return -EINVAL;
+
+	ret = v4l2_subdev_get_frame_desc_passthrough(sd, pad, fd);
+	if (ret == -ENOIOCTLCMD)
+		ret = rcsi2_get_frame_desc_fallback(sd, pad, fd);
+	return ret;
+}
+
 static const struct v4l2_subdev_pad_ops rcar_csi2_pad_ops = {
 	.enable_streams = rcsi2_enable_streams,
 	.disable_streams = rcsi2_disable_streams,
 
 	.set_fmt = rcsi2_set_pad_format,
 	.get_fmt = v4l2_subdev_get_fmt,
+
+	.get_frame_desc = rcsi2_get_frame_desc,
 };
 
 static const struct v4l2_subdev_ops rcar_csi2_subdev_ops = {

-- 
2.43.0

[PATCH v4 12/15] media: rcar-isp: Call get_frame_desc to find out VC & DT

[Tomi Valkeinen]

Call get_frame_desc to find out VC & DT, instead of hardcoding the VC
routing and deducing the DT based on the mbus format.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-isp/csisp.c | 110 ++++++++++++++++--------
 1 file changed, 74 insertions(+), 36 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-isp/csisp.c b/drivers/media/platform/renesas/rcar-isp/csisp.c
index 8ac45516aa39..18d63731b6c0 100644
--- a/drivers/media/platform/renesas/rcar-isp/csisp.c
+++ b/drivers/media/platform/renesas/rcar-isp/csisp.c
@@ -42,6 +42,9 @@
 #define ISPCS_DT_CODE03_EN0				BIT(7)
 #define ISPCS_DT_CODE03_DT0(dt)				((dt) & 0x3f)
 
+/* ISP has 12 channels, of which channels 4 to 11 are connected to VINs */
+#define ISPCS_NUM_CHANNELS	12
+
 struct rcar_isp_format {
 	u32 code;
 	unsigned int datatype;
@@ -225,31 +228,82 @@ static void risp_power_off(struct rcar_isp *isp)
 	pm_runtime_put(isp->dev);
 }
 
-static int risp_start(struct rcar_isp *isp, struct v4l2_subdev_state *state)
+static int risp_configure_routing(struct rcar_isp *isp,
+				  struct v4l2_subdev_state *state)
 {
-	const struct v4l2_subdev_route *route;
-	const struct v4l2_mbus_framefmt *fmt;
-	const struct rcar_isp_format *format;
-	unsigned int vc;
-	u32 sel_csi = 0;
+	struct v4l2_mbus_frame_desc source_fd;
+	struct v4l2_subdev_route *route;
 	int ret;
 
-	if (state->routing.num_routes != 1)
-		return -EINVAL;
+	ret = v4l2_subdev_call(isp->remote, pad, get_frame_desc,
+			       isp->remote_pad, &source_fd);
+	if (ret)
+		return ret;
+
+	/* Clear the channel registers */
+	for (unsigned int ch = 0; ch < ISPCS_NUM_CHANNELS; ++ch) {
+		risp_write_cs(isp, ISPCS_FILTER_ID_CH_REG(ch), 0);
+		risp_write_cs(isp, ISPCS_DT_CODE03_CH_REG(ch), 0);
+	}
 
-	route = &state->routing.routes[0];
+	for_each_active_route(&state->routing, route) {
+		struct v4l2_mbus_frame_desc_entry *source_entry = NULL;
+		const struct rcar_isp_format *format;
+		const struct v4l2_mbus_framefmt *fmt;
+		unsigned int i;
+		u8 vc, dt, ch;
+		u32 v;
+
+		for (i = 0; i < source_fd.num_entries; i++) {
+			if (source_fd.entry[i].stream == route->sink_stream) {
+				source_entry = &source_fd.entry[i];
+				break;
+			}
+		}
+
+		if (!source_entry) {
+			dev_err(isp->dev,
+				"Failed to find source frame desc entry for stream\n");
+			return -EPIPE;
+		}
+
+		vc = source_entry->bus.csi2.vc;
+		dt = source_entry->bus.csi2.dt;
+		/* Channels 4 - 11 go to VIN */
+		ch = route->source_pad - 1 + 4;
+
+		fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
+						   route->sink_stream);
+		if (!fmt)
+			return -EINVAL;
+
+		format = risp_code_to_fmt(fmt->code);
+		if (!format) {
+			dev_err(isp->dev, "Unsupported bus format\n");
+			return -EINVAL;
+		}
+
+		/* VC Filtering */
+		risp_write_cs(isp, ISPCS_FILTER_ID_CH_REG(ch), BIT(vc));
 
-	fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
-					   route->sink_stream);
-	if (!fmt)
-		return -EINVAL;
+		/* DT Filtering */
+		risp_write_cs(isp, ISPCS_DT_CODE03_CH_REG(ch),
+			   ISPCS_DT_CODE03_EN0 | ISPCS_DT_CODE03_DT0(dt));
 
-	format = risp_code_to_fmt(fmt->code);
-	if (!format) {
-		dev_err(isp->dev, "Unsupported bus format\n");
-		return -EINVAL;
+		/* Proc mode */
+		v = risp_read_cs(isp, ISPPROCMODE_DT_REG(dt));
+		v |= ISPPROCMODE_DT_PROC_MODE_VCn(vc, format->procmode);
+		risp_write_cs(isp, ISPPROCMODE_DT_REG(dt), v);
 	}
 
+	return 0;
+}
+
+static int risp_start(struct rcar_isp *isp, struct v4l2_subdev_state *state)
+{
+	u32 sel_csi = 0;
+	int ret;
+
 	ret = risp_power_on(isp);
 	if (ret) {
 		dev_err(isp->dev, "Failed to power on ISP\n");
@@ -263,25 +317,9 @@ static int risp_start(struct rcar_isp *isp, struct v4l2_subdev_state *state)
 	risp_write_cs(isp, ISPINPUTSEL0_REG,
 		      risp_read_cs(isp, ISPINPUTSEL0_REG) | sel_csi);
 
-	/* Configure Channel Selector. */
-	for (vc = 0; vc < 4; vc++) {
-		u8 ch = vc + 4;
-		u8 dt = format->datatype;
-
-		risp_write_cs(isp, ISPCS_FILTER_ID_CH_REG(ch), BIT(vc));
-		risp_write_cs(isp, ISPCS_DT_CODE03_CH_REG(ch),
-			      ISPCS_DT_CODE03_EN3 | ISPCS_DT_CODE03_DT3(dt) |
-			      ISPCS_DT_CODE03_EN2 | ISPCS_DT_CODE03_DT2(dt) |
-			      ISPCS_DT_CODE03_EN1 | ISPCS_DT_CODE03_DT1(dt) |
-			      ISPCS_DT_CODE03_EN0 | ISPCS_DT_CODE03_DT0(dt));
-	}
-
-	/* Setup processing method. */
-	risp_write_cs(isp, ISPPROCMODE_DT_REG(format->datatype),
-		      ISPPROCMODE_DT_PROC_MODE_VCn(3, format->procmode) |
-		      ISPPROCMODE_DT_PROC_MODE_VCn(2, format->procmode) |
-		      ISPPROCMODE_DT_PROC_MODE_VCn(1, format->procmode) |
-		      ISPPROCMODE_DT_PROC_MODE_VCn(0, format->procmode));
+	ret = risp_configure_routing(isp, state);
+	if (ret)
+		return ret;
 
 	/* Start ISP. */
 	risp_write_cs(isp, ISPSTART_REG, ISPSTART_START);

-- 
2.43.0

[PATCH v4 13/15] media: rcar-csi2: Call get_frame_desc to find out VC & DT (Gen3)

[Tomi Valkeinen]

Call get_frame_desc to find out VC & DT, for Gen3 platforms, instead of
hardcoding the VC routing and deducing the DT based on the mbus format.

If the source subdevice doesn't implement .get_frame_desc, we use a
fallback case where we assume there's a single stream with VC = 0 and DT
based on the mbus format.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-csi2.c | 115 +++++++++++++++++++----------
 1 file changed, 75 insertions(+), 40 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index b8baf7c65e90..93f25fb3b82e 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -1069,62 +1069,97 @@ static int rcsi2_calc_mbps(struct rcar_csi2 *priv,
 static int rcsi2_start_receiver_gen3(struct rcar_csi2 *priv,
 				     struct v4l2_subdev_state *state)
 {
-	const struct v4l2_subdev_route *route;
-	const struct rcar_csi2_format *format;
-	u32 phycnt, vcdt = 0, vcdt2 = 0, fld = 0;
-	const struct v4l2_mbus_framefmt *fmt;
+	u32 phycnt, vcdt = 0, vcdt2 = 0;
+	u32 fld = FLD_DET_SEL(1);
+	struct v4l2_mbus_frame_desc source_fd;
+	struct v4l2_subdev_route *route;
 	unsigned int lanes;
-	unsigned int i;
 	int mbps, ret;
+	u8 ch = 0;
 
-	/* Use the format on the sink pad to compute the receiver config. */
+	ret = v4l2_subdev_call(priv->remote, pad, get_frame_desc,
+			       priv->remote_pad, &source_fd);
+	if (ret && ret != -ENOIOCTLCMD)
+		return ret;
 
-	if (state->routing.num_routes != 1)
-		return -EINVAL;
+	if (ret == -ENOIOCTLCMD) {
+		/* Create a fallback source_fd */
+		struct v4l2_mbus_frame_desc *fd = &source_fd;
+		const struct v4l2_subdev_route *route;
+		const struct rcar_csi2_format *format;
+		struct v4l2_mbus_framefmt *fmt;
 
-	route = &state->routing.routes[0];
+		if (state->routing.num_routes != 1)
+			return -EINVAL;
 
-	fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
-					   route->sink_stream);
-	if (!fmt)
-		return -EINVAL;
+		route = &state->routing.routes[0];
 
-	dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
-		fmt->width, fmt->height,
-		fmt->field == V4L2_FIELD_NONE ? 'p' : 'i');
+		fmt = v4l2_subdev_state_get_format(state, route->sink_pad,
+						   route->sink_stream);
+		if (!fmt)
+			return -EINVAL;
 
-	/* Code is validated in set_fmt. */
-	format = rcsi2_code_to_fmt(fmt->code);
-	if (!format)
-		return -EINVAL;
+		format = rcsi2_code_to_fmt(fmt->code);
+		if (!format)
+			return -EINVAL;
 
-	/*
-	 * Enable all supported CSI-2 channels with virtual channel and
-	 * data type matching.
-	 *
-	 * NOTE: It's not possible to get individual datatype for each
-	 *       source virtual channel. Once this is possible in V4L2
-	 *       it should be used here.
-	 */
-	for (i = 0; i < priv->info->num_channels; i++) {
+		memset(fd, 0, sizeof(*fd));
+
+		fd->num_entries = 1;
+		fd->type = V4L2_MBUS_FRAME_DESC_TYPE_CSI2;
+		fd->entry[0].stream = 0;
+		fd->entry[0].pixelcode = fmt->code;
+		fd->entry[0].bus.csi2.vc = 0;
+		fd->entry[0].bus.csi2.dt = format->datatype;
+	}
+
+	for_each_active_route(&state->routing, route) {
+		const struct v4l2_mbus_frame_desc_entry *source_entry = NULL;
+		const struct v4l2_mbus_framefmt *fmt;
+		const struct rcar_csi2_format *format;
+		unsigned int i;
 		u32 vcdt_part;
 
-		if (priv->channel_vc[i] < 0)
-			continue;
+		for (i = 0; i < source_fd.num_entries; i++) {
+			if (source_fd.entry[i].stream == route->sink_stream) {
+				source_entry = &source_fd.entry[i];
+				break;
+			}
+		}
 
-		vcdt_part = VCDT_SEL_VC(priv->channel_vc[i]) | VCDT_VCDTN_EN |
-			VCDT_SEL_DTN_ON | VCDT_SEL_DT(format->datatype);
+		if (!source_entry) {
+			dev_err(priv->dev,
+				"Failed to find stream from source frame desc\n");
+			return -EPIPE;
+		}
+
+		vcdt_part = VCDT_SEL_VC(source_entry->bus.csi2.vc) |
+			    VCDT_VCDTN_EN | VCDT_SEL_DTN_ON |
+			    VCDT_SEL_DT(source_entry->bus.csi2.dt);
 
 		/* Store in correct reg and offset. */
-		if (i < 2)
-			vcdt |= vcdt_part << ((i % 2) * 16);
+		if (ch < 2)
+			vcdt |= vcdt_part << ((ch % 2) * 16);
 		else
-			vcdt2 |= vcdt_part << ((i % 2) * 16);
-	}
+			vcdt2 |= vcdt_part << ((ch % 2) * 16);
+
+		fmt = v4l2_subdev_state_get_format(state, RCAR_CSI2_SINK,
+						   route->sink_stream);
+		if (!fmt)
+			return -EINVAL;
 
-	if (fmt->field == V4L2_FIELD_ALTERNATE)
-		fld = FLD_DET_SEL(1) | FLD_FLD_EN(3) | FLD_FLD_EN(2) |
-		      FLD_FLD_EN(1) | FLD_FLD_EN(0);
+		dev_dbg(priv->dev, "Input size (%ux%u%c)\n",
+			fmt->width, fmt->height,
+			fmt->field == V4L2_FIELD_NONE ? 'p' : 'i');
+
+		/* Code is validated in set_fmt. */
+		format = rcsi2_code_to_fmt(fmt->code);
+
+		if (fmt->field == V4L2_FIELD_ALTERNATE)
+			fld |= FLD_FLD_EN(ch);
+
+		ch++;
+	}
 
 	/*
 	 * Get the number of active data lanes inspecting the remote mbus

-- 
2.43.0

[PATCH v4 14/15] media: rcar-csi2: Add full streams support

[Tomi Valkeinen]

Add the missing pieces to enable full streams support:

- Add set_routing
- Drop the explicit uses of a single stream, and instead use the streams
  mask.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-csi2.c | 75 ++++++++++++++++++++++--------
 1 file changed, 55 insertions(+), 20 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-csi2.c b/drivers/media/platform/renesas/rcar-csi2.c
index 93f25fb3b82e..51d381cc647f 100644
--- a/drivers/media/platform/renesas/rcar-csi2.c
+++ b/drivers/media/platform/renesas/rcar-csi2.c
@@ -702,6 +702,17 @@ static const struct rcar_csi2_format rcar_csi2_formats[] = {
 	},
 };
 
+static const struct v4l2_mbus_framefmt rcar_csi2_default_fmt = {
+	.width		= 1920,
+	.height		= 1080,
+	.code		= MEDIA_BUS_FMT_RGB888_1X24,
+	.colorspace	= V4L2_COLORSPACE_SRGB,
+	.field		= V4L2_FIELD_NONE,
+	.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
+	.quantization	= V4L2_QUANTIZATION_DEFAULT,
+	.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
+};
+
 static const struct rcar_csi2_format *rcsi2_code_to_fmt(unsigned int code)
 {
 	unsigned int i;
@@ -1887,10 +1898,8 @@ static int rcsi2_enable_streams(struct v4l2_subdev *sd,
 				u64 source_streams_mask)
 {
 	struct rcar_csi2 *priv = sd_to_csi2(sd);
-	int ret = 0;
-
-	if (source_streams_mask != 1)
-		return -EINVAL;
+	u64 sink_streams;
+	int ret;
 
 	if (!priv->remote)
 		return -ENODEV;
@@ -1901,8 +1910,13 @@ static int rcsi2_enable_streams(struct v4l2_subdev *sd,
 			return ret;
 	}
 
+	sink_streams = v4l2_subdev_state_xlate_streams(state,
+						       RCAR_CSI2_SOURCE_VC0,
+						       RCAR_CSI2_SINK,
+						       &source_streams_mask);
+
 	ret = v4l2_subdev_enable_streams(priv->remote, priv->remote_pad,
-					 BIT_ULL(0));
+					 sink_streams);
 	if (ret) {
 		if (priv->stream_count == 0)
 			rcsi2_stop(priv);
@@ -1919,19 +1933,22 @@ static int rcsi2_disable_streams(struct v4l2_subdev *sd,
 				 u32 source_pad, u64 source_streams_mask)
 {
 	struct rcar_csi2 *priv = sd_to_csi2(sd);
+	u64 sink_streams;
 	int ret;
 
-	if (source_streams_mask != 1)
-		return -EINVAL;
-
 	if (!priv->remote)
 		return -ENODEV;
 
 	if (priv->stream_count == 1)
 		rcsi2_stop(priv);
 
+	sink_streams = v4l2_subdev_state_xlate_streams(state,
+						       RCAR_CSI2_SOURCE_VC0,
+						       RCAR_CSI2_SINK,
+						       &source_streams_mask);
+
 	ret = v4l2_subdev_disable_streams(priv->remote, priv->remote_pad,
-					  BIT_ULL(0));
+					  sink_streams);
 	if (ret)
 		return ret;
 
@@ -1970,6 +1987,34 @@ static int rcsi2_set_pad_format(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int rcsi2_set_routing(struct v4l2_subdev *sd,
+			     struct v4l2_subdev_state *state,
+			     enum v4l2_subdev_format_whence which,
+			     struct v4l2_subdev_krouting *routing)
+{
+	struct rcar_csi2 *priv = sd_to_csi2(sd);
+	int ret;
+
+	if (priv->info->use_isp) {
+		ret = v4l2_subdev_routing_validate(sd, routing,
+						   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1);
+	} else {
+		ret = v4l2_subdev_routing_validate(sd, routing,
+						   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1 |
+						   V4L2_SUBDEV_ROUTING_NO_SOURCE_MULTIPLEXING);
+	}
+
+	if (ret)
+		return ret;
+
+	ret = v4l2_subdev_set_routing_with_fmt(sd, state, routing,
+					       &rcar_csi2_default_fmt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
 static int rcsi2_get_frame_desc_fallback(struct v4l2_subdev *sd,
 					 unsigned int pad,
 					 struct v4l2_mbus_frame_desc *fd)
@@ -2045,6 +2090,7 @@ static const struct v4l2_subdev_pad_ops rcar_csi2_pad_ops = {
 	.set_fmt = rcsi2_set_pad_format,
 	.get_fmt = v4l2_subdev_get_fmt,
 
+	.set_routing = rcsi2_set_routing,
 	.get_frame_desc = rcsi2_get_frame_desc,
 };
 
@@ -2065,17 +2111,6 @@ static int rcsi2_init_state(struct v4l2_subdev *sd,
 		},
 	};
 
-	static const struct v4l2_mbus_framefmt rcar_csi2_default_fmt = {
-		.width		= 1920,
-		.height		= 1080,
-		.code		= MEDIA_BUS_FMT_RGB888_1X24,
-		.colorspace	= V4L2_COLORSPACE_SRGB,
-		.field		= V4L2_FIELD_NONE,
-		.ycbcr_enc	= V4L2_YCBCR_ENC_DEFAULT,
-		.quantization	= V4L2_QUANTIZATION_DEFAULT,
-		.xfer_func	= V4L2_XFER_FUNC_DEFAULT,
-	};
-
 	static const struct v4l2_subdev_krouting routing = {
 		.num_routes = ARRAY_SIZE(routes),
 		.routes = routes,

-- 
2.43.0

[PATCH v4 15/15] media: rcar-isp: Add full streams support

[Tomi Valkeinen]

Add the missing pieces to enable full streams support:

- Add set_routing
- Drop the explicit uses of a single stream, and instead use the streams
  mask.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/rcar-isp/csisp.c | 39 +++++++++++++++++++++++--
 1 file changed, 36 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/renesas/rcar-isp/csisp.c b/drivers/media/platform/renesas/rcar-isp/csisp.c
index 18d63731b6c0..e4b9a6204815 100644
--- a/drivers/media/platform/renesas/rcar-isp/csisp.c
+++ b/drivers/media/platform/renesas/rcar-isp/csisp.c
@@ -340,6 +340,7 @@ static int risp_enable_streams(struct v4l2_subdev *sd,
 			       u64 source_streams_mask)
 {
 	struct rcar_isp *isp = sd_to_isp(sd);
+	u64 sink_streams;
 	int ret;
 
 	if (source_streams_mask != 1)
@@ -354,8 +355,13 @@ static int risp_enable_streams(struct v4l2_subdev *sd,
 			return ret;
 	}
 
+	sink_streams = v4l2_subdev_state_xlate_streams(state,
+						       source_pad,
+						       RCAR_ISP_SINK,
+						       &source_streams_mask);
+
 	ret = v4l2_subdev_enable_streams(isp->remote, isp->remote_pad,
-					 BIT_ULL(0));
+					 sink_streams);
 	if (ret) {
 		if (isp->stream_count == 0)
 			risp_stop(isp);
@@ -372,6 +378,7 @@ static int risp_disable_streams(struct v4l2_subdev *sd,
 				u64 source_streams_mask)
 {
 	struct rcar_isp *isp = sd_to_isp(sd);
+	u64 sink_streams;
 	int ret;
 
 	if (source_streams_mask != 1)
@@ -380,8 +387,12 @@ static int risp_disable_streams(struct v4l2_subdev *sd,
 	if (!isp->remote)
 		return -ENODEV;
 
-	ret = v4l2_subdev_disable_streams(isp->remote, isp->remote_pad,
-					  BIT_ULL(0));
+	sink_streams = v4l2_subdev_state_xlate_streams(state,
+						       source_pad,
+						       RCAR_ISP_SINK,
+						       &source_streams_mask);
+
+	ret = v4l2_subdev_disable_streams(isp->remote, isp->remote_pad, sink_streams);
 	if (ret)
 		return ret;
 
@@ -423,12 +434,34 @@ static int risp_set_pad_format(struct v4l2_subdev *sd,
 	return 0;
 }
 
+static int risp_set_routing(struct v4l2_subdev *sd,
+			    struct v4l2_subdev_state *state,
+			    enum v4l2_subdev_format_whence which,
+			    struct v4l2_subdev_krouting *routing)
+{
+	int ret;
+
+	ret = v4l2_subdev_routing_validate(sd, routing,
+					   V4L2_SUBDEV_ROUTING_ONLY_1_TO_1 |
+					   V4L2_SUBDEV_ROUTING_NO_SOURCE_MULTIPLEXING);
+	if (ret)
+		return ret;
+
+	ret = v4l2_subdev_set_routing_with_fmt(sd, state, routing,
+					       &risp_default_fmt);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
 static const struct v4l2_subdev_pad_ops risp_pad_ops = {
 	.enable_streams = risp_enable_streams,
 	.disable_streams = risp_disable_streams,
 	.set_fmt = risp_set_pad_format,
 	.get_fmt = v4l2_subdev_get_fmt,
 	.link_validate = v4l2_subdev_link_validate_default,
+	.set_routing = risp_set_routing,
 };
 
 static const struct v4l2_subdev_ops rcar_isp_subdev_ops = {

-- 
2.43.0

Re: [PATCH v4 00/15] media: rcar: Streams support

[Niklas Söderlund]

Hi Tomi,

Thanks for your persistent work on this series!

On 2025-12-16 17:18:17 +0200, Tomi Valkeinen wrote:
> Add streams support to Renesas rcar platform driver.
> 
> The series keaps compatibility with the current upstream for a single
> stream use case. However, in upstream there's a limited custom
> multi-stream support implemented to the rcar driver, which will be
> replaced with the upstream's Streams API.
> 
> I have tested this series on Sparrow-Hawk board, with a few different
> setups:
> 
> IMX219 connected to the CSI0 connector
> - The following patches applied to my test branch in addition to this
>   series:
>   1) The v4l2_subdev_get_frame_desc_passthrough dependency
>   2) Revert of commit e7376745ad5c8548e31d9ea58adfb5a847e017a4 ("media:
>      rcar-vin: Fix stride setting for RAW8 formats"), as that commit
>      breaks RAW8

That is so odd, I do grab RAW8 on V4H with a IMX219. In what way is do 
you see RAW8 breaking?

> - Tested with a single video stream
> 
> IMX219 connected to the CSI0 connector
> - Plenty of other patches applied to enable full streams support and
>   embedded data support in imx219 and v4l2 framework
> - Tested with video and embedded data streams
>  
> Arducam FPD-Link board + 4 x IMX219 connected to the CSI0 connector
> - Plenty of other patches applied to enable full streams support and
>   embedded data support in imx219 and v4l2 framework, and TPG support in
>   ub953
> - Tested with video and embedded data streams from all four cameras (so
>   8 streams in total)
> - Also tested with ub953's TPG, combined with video & embedded streams
>   from other cameras.

As there are dependencies on patches that have been on the list for a 
long time that would block merging this work. Could we try and shift 
focus and get some of the nice fixups and cleanups merged first? IMHO we 
could even aim for merging the rework (reduction) of the ad-hoc VC 
support done in the graph ASAP to get it out of the way.

It would also be nice if we could sort the RAW8 issue separately to get 
it out of the way.

I have other work touching these drivers I'm holding of on to not cause 
conflicts with your nice work, and it will make my work smaller/easier 
too!

Could we start by breaking this out into:

- A series that just removes the ad-hoc VC thru media graph in the R-Car 
  VIN and CSI-2 drivers.

- And then we can follow up with the cleanup of each of the drivers as 
  separate series.

This would make it easier for everybody I think. Each series becomes 
smaller to review, we can get fixes and cleanup in now and not wait for 
all stream dependences to land first.

> 
> I have observed one issue with the embedded data (i.e. requiring bunch
> of patches not in upstream): when stopping streaming, VIN says that it
> cannot stop the stream. I haven't debugged that, but a possible issue is
> that the if the video stream for the imx219 is stopped first, the
> embedded data stops also, and VIN does not get the frame-end it is
> waiting for.

I would not be comfortable merging with this regression. I have bad 
experiences when VIN report it can't stop the stream. More often then 
not it also means it then can't start streaming again...

> 
>  Tomi
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
> ---
> Changes in v4:
> - Rebased on v6.18, with minor conflicts resolved
> - Improved patch descriptions
> - Re-ordered the patches a bit to move changes that could be applied
>   without the full streams support to the beginning of the series
> - Added "media: rcar-vin: Link VINs on Gen3 to a single channel on each
>   CSI-2" which removes possibility of wrong routing config on Gen3
> - Added "media: rcar-csi2: Improve FLD_FLD_EN macros" which was part of
>   another patch in v3
> - Addressed minor comments (constifyings, cosmetics)
> - Fixed the missing stream_count checks in disable_streams ops
> - Fixed a few instances in csisp and csi2 where
>   v4l2_subdev_state_get_format() was called with hardcoded pad/stream,
>   instead of using the data from the route
> - Dropped unnecessary ISPPROCMODE_DT_REG register clears
> - Squashed "media: rcar-csi2: Add more stream support to
>   rcsi2_calc_mbps()" into a previous patch
> - Dropped wrong use_isp check from csi2's rcsi2_set_routing()
> - Link to v3: https://lore.kernel.org/r/20250530-rcar-streams-v3-0-026655df7138@ideasonboard.com
> 
> Changes in v3:
> - Rebased on top of latest linux-media
> - Dropped dependencies which are already in linux-media (only remaining
>   dependency is v4l2_subdev_get_frame_desc_passthrough)
> - Tested on white-hawk board, using the staging deser TPG
> - Also tested in a WIP branch for GMSL2 (two video streams)
> - Link to v2: https://lore.kernel.org/r/20250326-rcar-streams-v2-0-d0d7002c641f@ideasonboard.com
> 
> Changes in v2:
> - Rebased on top of latest upstream, and updated the dependencies to
>   match the latest serieses sent.
> - Add new patch "media: rcar-csi2: Use the pad version of v4l2_get_link_freq()"
> - Drop "media: rcar-csi2: Fix typo" (it was not a typo)
> - Update the code in calc_mbps(). The previous method relied on
>   V4L2_CID_LINK_FREQ, but that's not available if the link-freq is
>   provided via get_mbus_config().
> - Dropped dependencies to Niklas' old series which doesn't apply
>   cleanly. It's needed for multi-stream, but not for the current
>   upstream which only has a single stream use case.
> - Link to v1: https://lore.kernel.org/r/20250219-rcar-streams-v1-0-f1b93e370aab@ideasonboard.com
> 
> ---
> Tomi Valkeinen (15):
>       media: rcar-isp: Improve ISPPROCMODE_DT_PROC_MODE_VC
>       media: rcar-csi2: Improve FLD_FLD_EN macros
>       media: rcar-csi2: Move rcsi2_calc_mbps()
>       media: rcar-csi2: Simplify rcsi2_calc_mbps()
>       media: rcar-csi2: Optimize rcsi2_calc_mbps()
>       media: rcar-vin: Link VINs on Gen3 to a single channel on each CSI-2
>       media: rcar-isp: Move {enable|disable}_streams() calls
>       media: rcar-csi2: Move {enable|disable}_streams() calls
>       media: rcar-csi2: Switch to Streams API
>       media: rcar-isp: Switch to Streams API
>       media: rcar-csi2: Add .get_frame_desc op
>       media: rcar-isp: Call get_frame_desc to find out VC & DT
>       media: rcar-csi2: Call get_frame_desc to find out VC & DT (Gen3)
>       media: rcar-csi2: Add full streams support
>       media: rcar-isp: Add full streams support
> 
>  drivers/media/platform/renesas/rcar-csi2.c         | 437 +++++++++++++++------
>  drivers/media/platform/renesas/rcar-isp/csisp.c    | 232 ++++++++---
>  .../media/platform/renesas/rcar-vin/rcar-core.c    |  27 +-
>  3 files changed, 509 insertions(+), 187 deletions(-)
> ---
> base-commit: f7b88edb52c8dd01b7e576390d658ae6eef0e134
> change-id: 20250219-rcar-streams-1fdea8860e5e
> prerequisite-change-id: 20250218-frame-desc-passthrough-66805e413974:v4
> prerequisite-patch-id: bce4a915a29a64f88ed1bb600c08df37d2ba20c6
> prerequisite-patch-id: 69b75e7dad9ced905cb39a72f18bebbf3e8f998a
> prerequisite-patch-id: 58463f6944c76acd6cf203b14a2836cdb0db2461
> 
> Best regards,
> -- 
> Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
> 

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH v4 00/15] media: rcar: Streams support

[Tomi Valkeinen]

Hi Niklas,

On 31/12/2025 11:57, Niklas Söderlund wrote:
> Hi Tomi,
> 
> Thanks for your persistent work on this series!
> 
> On 2025-12-16 17:18:17 +0200, Tomi Valkeinen wrote:
>> Add streams support to Renesas rcar platform driver.
>>
>> The series keaps compatibility with the current upstream for a single
>> stream use case. However, in upstream there's a limited custom
>> multi-stream support implemented to the rcar driver, which will be
>> replaced with the upstream's Streams API.
>>
>> I have tested this series on Sparrow-Hawk board, with a few different
>> setups:
>>
>> IMX219 connected to the CSI0 connector
>> - The following patches applied to my test branch in addition to this
>>   series:
>>   1) The v4l2_subdev_get_frame_desc_passthrough dependency
>>   2) Revert of commit e7376745ad5c8548e31d9ea58adfb5a847e017a4 ("media:
>>      rcar-vin: Fix stride setting for RAW8 formats"), as that commit
>>      breaks RAW8
> 
> That is so odd, I do grab RAW8 on V4H with a IMX219. In what way is do 
> you see RAW8 breaking?

I also used V4H with IMX219. Let's compare our setups and results on irc
and find out what's going on.

>> - Tested with a single video stream
>>
>> IMX219 connected to the CSI0 connector
>> - Plenty of other patches applied to enable full streams support and
>>   embedded data support in imx219 and v4l2 framework
>> - Tested with video and embedded data streams
>>  
>> Arducam FPD-Link board + 4 x IMX219 connected to the CSI0 connector
>> - Plenty of other patches applied to enable full streams support and
>>   embedded data support in imx219 and v4l2 framework, and TPG support in
>>   ub953
>> - Tested with video and embedded data streams from all four cameras (so
>>   8 streams in total)
>> - Also tested with ub953's TPG, combined with video & embedded streams
>>   from other cameras.
> 
> As there are dependencies on patches that have been on the list for a 
> long time that would block merging this work. Could we try and shift 
> focus and get some of the nice fixups and cleanups merged first? IMHO we 
> could even aim for merging the rework (reduction) of the ad-hoc VC 
> support done in the graph ASAP to get it out of the way.
> 
> It would also be nice if we could sort the RAW8 issue separately to get 
> it out of the way.

Sounds fine to me.

> I have other work touching these drivers I'm holding of on to not cause 
> conflicts with your nice work, and it will make my work smaller/easier 
> too!
> 
> Could we start by breaking this out into:
> 
> - A series that just removes the ad-hoc VC thru media graph in the R-Car 
>   VIN and CSI-2 drivers.

That's just the patch 6, "media: rcar-vin: Link VINs on Gen3 to a single
channel on each CSI-2" patch, isn't it?

> - And then we can follow up with the cleanup of each of the drivers as 
>   separate series.

How about merging 1-6 as a first step (assuming they pass reviews and
tests =)? I'm not sure if there's any benefit in sending the above VIN
patch alone, then the cleanups after that. Or perhaps patches 1-8.

> This would make it easier for everybody I think. Each series becomes 
> smaller to review, we can get fixes and cleanup in now and not wait for 
> all stream dependences to land first.

I'm fine with breaking it to smaller pieces than 1-8 if you want. I
think the split could then be 1-5, 6-8, and the rest later. But I think
1-5 are quite small and straightforward, so I'm hoping we can work with
smaller amount of patch sets.

>> I have observed one issue with the embedded data (i.e. requiring bunch
>> of patches not in upstream): when stopping streaming, VIN says that it
>> cannot stop the stream. I haven't debugged that, but a possible issue is
>> that the if the video stream for the imx219 is stopped first, the
>> embedded data stops also, and VIN does not get the frame-end it is
>> waiting for.
> 
> I would not be comfortable merging with this regression. I have bad 
> experiences when VIN report it can't stop the stream. More often then 
> not it also means it then can't start streaming again...

It's not a regression, and on the "why it doesn't matter" side is that
embedded data is not supported upstream, so the user cannot hit this
issue. Also, I did not notice any issues in restarting the streaming again.

That said, I agree that it must be sorted out.

 Tomi

Re: [PATCH v4 00/15] media: rcar: Streams support

[Niklas Söderlund]

Hello Tomi,

Thanks again for your diligent work on this series!

On 2026-01-08 15:57:10 +0200, Tomi Valkeinen wrote:
> Hi Niklas,
> 
> On 31/12/2025 11:57, Niklas Söderlund wrote:
> > Hi Tomi,
> > 
> > Thanks for your persistent work on this series!
> > 
> > On 2025-12-16 17:18:17 +0200, Tomi Valkeinen wrote:
> >> Add streams support to Renesas rcar platform driver.
> >>
> >> The series keaps compatibility with the current upstream for a single
> >> stream use case. However, in upstream there's a limited custom
> >> multi-stream support implemented to the rcar driver, which will be
> >> replaced with the upstream's Streams API.
> >>
> >> I have tested this series on Sparrow-Hawk board, with a few different
> >> setups:
> >>
> >> IMX219 connected to the CSI0 connector
> >> - The following patches applied to my test branch in addition to this
> >>   series:
> >>   1) The v4l2_subdev_get_frame_desc_passthrough dependency
> >>   2) Revert of commit e7376745ad5c8548e31d9ea58adfb5a847e017a4 ("media:
> >>      rcar-vin: Fix stride setting for RAW8 formats"), as that commit
> >>      breaks RAW8
> > 
> > That is so odd, I do grab RAW8 on V4H with a IMX219. In what way is do 
> > you see RAW8 breaking?
> 
> I also used V4H with IMX219. Let's compare our setups and results on irc
> and find out what's going on.

Sounds good.

> 
> >> - Tested with a single video stream
> >>
> >> IMX219 connected to the CSI0 connector
> >> - Plenty of other patches applied to enable full streams support and
> >>   embedded data support in imx219 and v4l2 framework
> >> - Tested with video and embedded data streams
> >>  
> >> Arducam FPD-Link board + 4 x IMX219 connected to the CSI0 connector
> >> - Plenty of other patches applied to enable full streams support and
> >>   embedded data support in imx219 and v4l2 framework, and TPG support in
> >>   ub953
> >> - Tested with video and embedded data streams from all four cameras (so
> >>   8 streams in total)
> >> - Also tested with ub953's TPG, combined with video & embedded streams
> >>   from other cameras.
> > 
> > As there are dependencies on patches that have been on the list for a 
> > long time that would block merging this work. Could we try and shift 
> > focus and get some of the nice fixups and cleanups merged first? IMHO we 
> > could even aim for merging the rework (reduction) of the ad-hoc VC 
> > support done in the graph ASAP to get it out of the way.
> > 
> > It would also be nice if we could sort the RAW8 issue separately to get 
> > it out of the way.
> 
> Sounds fine to me.
> 
> > I have other work touching these drivers I'm holding of on to not cause 
> > conflicts with your nice work, and it will make my work smaller/easier 
> > too!
> > 
> > Could we start by breaking this out into:
> > 
> > - A series that just removes the ad-hoc VC thru media graph in the R-Car 
> >   VIN and CSI-2 drivers.
> 
> That's just the patch 6, "media: rcar-vin: Link VINs on Gen3 to a single
> channel on each CSI-2" patch, isn't it?

If it do not depend on anything in 1-5, yes.

> 
> > - And then we can follow up with the cleanup of each of the drivers as 
> >   separate series.
> 
> How about merging 1-6 as a first step (assuming they pass reviews and
> tests =)? I'm not sure if there's any benefit in sending the above VIN
> patch alone, then the cleanups after that. Or perhaps patches 1-8.

See below,

> 
> > This would make it easier for everybody I think. Each series becomes 
> > smaller to review, we can get fixes and cleanup in now and not wait for 
> > all stream dependences to land first.
> 
> I'm fine with breaking it to smaller pieces than 1-8 if you want. I
> think the split could then be 1-5, 6-8, and the rest later. But I think
> 1-5 are quite small and straightforward, so I'm hoping we can work with
> smaller amount of patch sets.

I think we can go for 1-8, but I would split it into 3 series. One for 
rcar-isp, one for rcar-csi2 and one for rcar-vin. That way we can move 
forward more quickly IMHO as review and test of each in isolation will 
go quicker.

Does this sound OK to you?

> 
> >> I have observed one issue with the embedded data (i.e. requiring bunch
> >> of patches not in upstream): when stopping streaming, VIN says that it
> >> cannot stop the stream. I haven't debugged that, but a possible issue is
> >> that the if the video stream for the imx219 is stopped first, the
> >> embedded data stops also, and VIN does not get the frame-end it is
> >> waiting for.
> > 
> > I would not be comfortable merging with this regression. I have bad 
> > experiences when VIN report it can't stop the stream. More often then 
> > not it also means it then can't start streaming again...
> 
> It's not a regression, and on the "why it doesn't matter" side is that
> embedded data is not supported upstream, so the user cannot hit this
> issue. Also, I did not notice any issues in restarting the streaming again.

OK if you need more patches and the 'can't stop streaming' have only 
been observed for that it's OK. But I really don't want to merge 
anything that increases the likelyhood of this state to happen when 
stopping.

If it only happens with embedded data we should refuse to start 
streaming if embedded data is enabled and we still have not sorted this 
out.

> 
> That said, I agree that it must be sorted out.
> 
>  Tomi
> 

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH v4 00/15] media: rcar: Streams support

[Tomi Valkeinen]

Hi,

On 31/12/2025 11:57, Niklas Söderlund wrote:
> Hi Tomi,
> 
> Thanks for your persistent work on this series!
> 
> On 2025-12-16 17:18:17 +0200, Tomi Valkeinen wrote:
>> Add streams support to Renesas rcar platform driver.
>>
>> The series keaps compatibility with the current upstream for a single
>> stream use case. However, in upstream there's a limited custom
>> multi-stream support implemented to the rcar driver, which will be
>> replaced with the upstream's Streams API.
>>
>> I have tested this series on Sparrow-Hawk board, with a few different
>> setups:
>>
>> IMX219 connected to the CSI0 connector
>> - The following patches applied to my test branch in addition to this
>>   series:
>>   1) The v4l2_subdev_get_frame_desc_passthrough dependency
>>   2) Revert of commit e7376745ad5c8548e31d9ea58adfb5a847e017a4 ("media:
>>      rcar-vin: Fix stride setting for RAW8 formats"), as that commit
>>      breaks RAW8
> 
> That is so odd, I do grab RAW8 on V4H with a IMX219. In what way is do 
> you see RAW8 breaking?
I think I found it. It's broken for all formats and resolutions, based
on luck:

VNIS_REG has the lowest 4 bits always 0. From the doc: "These bits
specify the width of the transfer destination memory in 16-pixel
unit."

We do nothing to comply with that.

 Tomi

Re: [PATCH v4 00/15] media: rcar: Streams support

[Niklas Söderlund]

Hi Tomi,

Thanks for digging in this.

On 2026-01-26 16:01:36 +0200, Tomi Valkeinen wrote:
> Hi,
> 
> On 31/12/2025 11:57, Niklas Söderlund wrote:
> > Hi Tomi,
> > 
> > Thanks for your persistent work on this series!
> > 
> > On 2025-12-16 17:18:17 +0200, Tomi Valkeinen wrote:
> >> Add streams support to Renesas rcar platform driver.
> >>
> >> The series keaps compatibility with the current upstream for a single
> >> stream use case. However, in upstream there's a limited custom
> >> multi-stream support implemented to the rcar driver, which will be
> >> replaced with the upstream's Streams API.
> >>
> >> I have tested this series on Sparrow-Hawk board, with a few different
> >> setups:
> >>
> >> IMX219 connected to the CSI0 connector
> >> - The following patches applied to my test branch in addition to this
> >>   series:
> >>   1) The v4l2_subdev_get_frame_desc_passthrough dependency
> >>   2) Revert of commit e7376745ad5c8548e31d9ea58adfb5a847e017a4 ("media:
> >>      rcar-vin: Fix stride setting for RAW8 formats"), as that commit
> >>      breaks RAW8
> > 
> > That is so odd, I do grab RAW8 on V4H with a IMX219. In what way is do 
> > you see RAW8 breaking?
> I think I found it. It's broken for all formats and resolutions, based
> on luck:
> 
> VNIS_REG has the lowest 4 bits always 0. From the doc: "These bits
> specify the width of the transfer destination memory in 16-pixel
> unit."
> 
> We do nothing to comply with that.

Yes we do, but maybe not enough?

In rvin_format_bytesperline() we align for this, we even consider the 
special cases for NV12 and NV16.

The value written to VNIS_REG is vin->format.bytesperline / fmt->bpp, 
and the value writen to vin->format.bytesperline is ALIGN(pix->width, 
align) * fmt->bpp. And for all formats (not NV12 or NV16) is 0x10, so we 
do align it to the 16-pixel unit no?

Maybe their is a corner case I have missed or maybe I'm missing some 
other angle? And I agree adding and removing the fmt->bpp multiplier is 
not the best here. As we have finally moved this driver to media graph 
only there are lots of areas things can be cleaned up and improved as we 
no longer need to consider all that Gen2 special cases. I bet this can 
likely be cleaned up.

> 
>  Tomi
> 

-- 
Kind Regards,
Niklas Söderlund

Re: [PATCH v4 00/15] media: rcar: Streams support

[Tomi Valkeinen]

Hi,

On 26/01/2026 21:39, Niklas Söderlund wrote:
> Hi Tomi,
> 
> Thanks for digging in this.
> 
> On 2026-01-26 16:01:36 +0200, Tomi Valkeinen wrote:
>> Hi,
>>
>> On 31/12/2025 11:57, Niklas Söderlund wrote:
>>> Hi Tomi,
>>>
>>> Thanks for your persistent work on this series!
>>>
>>> On 2025-12-16 17:18:17 +0200, Tomi Valkeinen wrote:
>>>> Add streams support to Renesas rcar platform driver.
>>>>
>>>> The series keaps compatibility with the current upstream for a single
>>>> stream use case. However, in upstream there's a limited custom
>>>> multi-stream support implemented to the rcar driver, which will be
>>>> replaced with the upstream's Streams API.
>>>>
>>>> I have tested this series on Sparrow-Hawk board, with a few different
>>>> setups:
>>>>
>>>> IMX219 connected to the CSI0 connector
>>>> - The following patches applied to my test branch in addition to this
>>>>   series:
>>>>   1) The v4l2_subdev_get_frame_desc_passthrough dependency
>>>>   2) Revert of commit e7376745ad5c8548e31d9ea58adfb5a847e017a4 ("media:
>>>>      rcar-vin: Fix stride setting for RAW8 formats"), as that commit
>>>>      breaks RAW8
>>>
>>> That is so odd, I do grab RAW8 on V4H with a IMX219. In what way is do 
>>> you see RAW8 breaking?
>> I think I found it. It's broken for all formats and resolutions, based
>> on luck:
>>
>> VNIS_REG has the lowest 4 bits always 0. From the doc: "These bits
>> specify the width of the transfer destination memory in 16-pixel
>> unit."
>>
>> We do nothing to comply with that.
> 
> Yes we do, but maybe not enough?
> 
> In rvin_format_bytesperline() we align for this, we even consider the 
> special cases for NV12 and NV16.
> 
> The value written to VNIS_REG is vin->format.bytesperline / fmt->bpp, 
> and the value writen to vin->format.bytesperline is ALIGN(pix->width, 
> align) * fmt->bpp. And for all formats (not NV12 or NV16) is 0x10, so we 
> do align it to the 16-pixel unit no?
> 
> Maybe their is a corner case I have missed or maybe I'm missing some 
> other angle? And I agree adding and removing the fmt->bpp multiplier is 
> not the best here. As we have finally moved this driver to media graph 
> only there are lots of areas things can be cleaned up and improved as we 
> no longer need to consider all that Gen2 special cases. I bet this can 
> likely be cleaned up.
Right, but if we have the stride / 2 code in, we will get strides not
aligned to 16 bytes unless the bytesperline happened to be aligned to 32
bytes.

I sent "[PATCH] media: renesas: vin: Fix RAW8 (again)". I still don't
understand exactly how the RAW8 processing goes in the hardware, but
afaics the stride / 2 is required, I cannot get any proper images
without that. The documentation doesn't really describe it so I'm not
totally content with all this.

Anyway, please test the new patch on your end, perhaps we'll finally
have a conclusion on the RAW8 =).

 Tomi

Re: [PATCH v4 00/15] media: rcar: Streams support

[Niklas Söderlund]

Hi Tomi,

On 2026-01-27 10:59:21 +0200, Tomi Valkeinen wrote:
> Hi,
> 
> On 26/01/2026 21:39, Niklas Söderlund wrote:
> > Hi Tomi,
> > 
> > Thanks for digging in this.
> > 
> > On 2026-01-26 16:01:36 +0200, Tomi Valkeinen wrote:
> >> Hi,
> >>
> >> On 31/12/2025 11:57, Niklas Söderlund wrote:
> >>> Hi Tomi,
> >>>
> >>> Thanks for your persistent work on this series!
> >>>
> >>> On 2025-12-16 17:18:17 +0200, Tomi Valkeinen wrote:
> >>>> Add streams support to Renesas rcar platform driver.
> >>>>
> >>>> The series keaps compatibility with the current upstream for a single
> >>>> stream use case. However, in upstream there's a limited custom
> >>>> multi-stream support implemented to the rcar driver, which will be
> >>>> replaced with the upstream's Streams API.
> >>>>
> >>>> I have tested this series on Sparrow-Hawk board, with a few different
> >>>> setups:
> >>>>
> >>>> IMX219 connected to the CSI0 connector
> >>>> - The following patches applied to my test branch in addition to this
> >>>>   series:
> >>>>   1) The v4l2_subdev_get_frame_desc_passthrough dependency
> >>>>   2) Revert of commit e7376745ad5c8548e31d9ea58adfb5a847e017a4 ("media:
> >>>>      rcar-vin: Fix stride setting for RAW8 formats"), as that commit
> >>>>      breaks RAW8
> >>>
> >>> That is so odd, I do grab RAW8 on V4H with a IMX219. In what way is do 
> >>> you see RAW8 breaking?
> >> I think I found it. It's broken for all formats and resolutions, based
> >> on luck:
> >>
> >> VNIS_REG has the lowest 4 bits always 0. From the doc: "These bits
> >> specify the width of the transfer destination memory in 16-pixel
> >> unit."
> >>
> >> We do nothing to comply with that.
> > 
> > Yes we do, but maybe not enough?
> > 
> > In rvin_format_bytesperline() we align for this, we even consider the 
> > special cases for NV12 and NV16.
> > 
> > The value written to VNIS_REG is vin->format.bytesperline / fmt->bpp, 
> > and the value writen to vin->format.bytesperline is ALIGN(pix->width, 
> > align) * fmt->bpp. And for all formats (not NV12 or NV16) is 0x10, so we 
> > do align it to the 16-pixel unit no?
> > 
> > Maybe their is a corner case I have missed or maybe I'm missing some 
> > other angle? And I agree adding and removing the fmt->bpp multiplier is 
> > not the best here. As we have finally moved this driver to media graph 
> > only there are lots of areas things can be cleaned up and improved as we 
> > no longer need to consider all that Gen2 special cases. I bet this can 
> > likely be cleaned up.
> Right, but if we have the stride / 2 code in, we will get strides not
> aligned to 16 bytes unless the bytesperline happened to be aligned to 32
> bytes.
> 
> I sent "[PATCH] media: renesas: vin: Fix RAW8 (again)". I still don't
> understand exactly how the RAW8 processing goes in the hardware, but
> afaics the stride / 2 is required, I cannot get any proper images
> without that. The documentation doesn't really describe it so I'm not
> totally content with all this.
> 
> Anyway, please test the new patch on your end, perhaps we'll finally
> have a conclusion on the RAW8 =).

Reading the comments in that patch and the documentation and I now see 
the other way the docs could be interpreted :-) I agree that if it works 
(I'm sure it does) it also satisfy the documentation. I will give it a 
spin and reply in that patch. It would indeed be super to solve this 
correctly, thanks for your efforts!

> 
>  Tomi
> 

-- 
Kind Regards,
Niklas Söderlund