[PATCH v2 0/9] media: renesas: vsp1: Add colorspace support

[PATCH v2 0/9] media: renesas: vsp1: Add colorspace support

[Laurent Pinchart]

Hello,

This patch series extends the VSP1 driver with colorspace support. It
turns out that the VSP RPF and WPF entities can convert between RGB and
YUV, a feature that we have failed to test so far. The hardware support
BT.601 and BT.709, in both limited and full range. Proper configuration
of colorspace is crucial for accurate image rendering.

Patch 1/9 starts by implementing pixel format enumeration in the driver,
a feature that was surprisingly missing. Patch 2/9 then continues with
another fix, restricting the creation of the HSI and HST entities to VSP
instances that include them. Following with another fix, patch 4/9
addresses format setting on the RPF and WPF source pad to disable HSV
<-> { RGB, YUV } conversion, a feature *not* supported by the hardware.

After those initial fixes, patch 5/9 starts implementing colorspace
support by reporting the related information to userspace. The driver
currently hardcodes limited range BT.601 when programming the hardware,
so that is the value that the patch reports to userspace for YUV
formats. Patch 6/9 makes the YCbCr encoding and quantization
configurable, finalizing colorspace support in the VSP driver.

The next three patches are new in this version. They proceed with
exposing colorspace configurability in the API exposed to the DU DRM
driver in patch 8/9 (with 7/9 performing a small refactoring first).
Patch 9/9 then updates the DU driver accordingly, to create plane
colorspace properties and wire them up to the VSP.

The series has been tested with the vsp-tests suite and the kms-tests
suite. Patches that add CSC support to the vsp-tests suite have been
posted in [1], and can be found in the vsp-tests git tree in [2].
Updates to media-ctl and yavta to support setting colorspace on video
capture devices and on subdev source pads have been merged in the
respective projects, make sure to use the latest master branch if you
want to run the tests.

The series has also been tested with the latest v4l2-compliance. The bad
news is that the test flags 56 errors when run on a Renesas Salvator-X
2nd version board based on r8a77965:

Grand Total for vsp1 device /dev/media1: 757, Succeeded: 701, Failed: 56, Warnings: 0

The good news is that none of those are regressions, quite the contrary:
without this series applied, the total number of failures is 95, and the
diff shows no new error. I will therefore address those issues
separately.

I would like to get patch 1/9 to 8/9 merged in v6.16, to then merge 9/9
in v6.17.

[1] https://lore.kernel.org/linux-renesas-soc/20250409004758.11014-1-laurent.pinchart@ideasonboard.com
[2] https://git.ideasonboard.com/renesas/vsp-tests.git/log/?h=csc

Laurent Pinchart (9):
  media: renesas: vsp1: Implement pixel format enumeration
  media: renesas: vsp1: Make HSI and HST modules optional
  media: renesas: vsp1: Fix HSV format enumeration
  media: renesas: vsp1: Fix media bus code setup on RWPF source pad
  media: renesas: vsp1: Report colour space information to userspace
  media: renesas: vsp1: Allow setting encoding and quantization
  media: renesas: vsp1: Name nested structure in vsp1_drm
  media: renesas: vsp1: Expose color space through the DRM API
  drm: rcar-du: Create plane color properties

 drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c |  15 ++
 drivers/media/platform/renesas/vsp1/vsp1.h    |   1 +
 .../media/platform/renesas/vsp1/vsp1_brx.c    |   9 +-
 .../media/platform/renesas/vsp1/vsp1_drm.c    |  22 +-
 .../media/platform/renesas/vsp1/vsp1_drm.h    |   8 +-
 .../media/platform/renesas/vsp1/vsp1_drv.c    |  59 +++---
 .../media/platform/renesas/vsp1/vsp1_entity.c |  22 +-
 .../media/platform/renesas/vsp1/vsp1_entity.h |   2 +
 .../media/platform/renesas/vsp1/vsp1_hsit.c   |  11 +-
 .../media/platform/renesas/vsp1/vsp1_pipe.c   | 188 ++++++++++++++++--
 .../media/platform/renesas/vsp1/vsp1_pipe.h   |   5 +
 .../media/platform/renesas/vsp1/vsp1_rpf.c    |  29 ++-
 .../media/platform/renesas/vsp1/vsp1_rwpf.c   |  51 ++++-
 .../media/platform/renesas/vsp1/vsp1_sru.c    |   9 +-
 .../media/platform/renesas/vsp1/vsp1_uds.c    |   9 +-
 .../media/platform/renesas/vsp1/vsp1_video.c  |  50 ++++-
 .../media/platform/renesas/vsp1/vsp1_wpf.c    |  29 ++-
 include/media/vsp1.h                          |   4 +
 18 files changed, 442 insertions(+), 81 deletions(-)


base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
prerequisite-patch-id: e3d9847d1da4398f1bf0b57f5ef4a612392c7255
-- 
Regards,

Laurent Pinchart

[PATCH v2 1/9] media: renesas: vsp1: Implement pixel format enumeration

[Laurent Pinchart]

The VSP1 driver is missing the ability to enumerate pixel formats on its
video nodes, which is supposed to be supported according to the V4L2
API. Implement the enumeration to fix this issue.

As the device is media controller-centric, also implement the ability to
filter pixel formats by media bus code, and report the missing
V4L2_CAP_IO_MC capability.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 .../media/platform/renesas/vsp1/vsp1_pipe.c   | 103 ++++++++++++++----
 .../media/platform/renesas/vsp1/vsp1_pipe.h   |   3 +
 .../media/platform/renesas/vsp1/vsp1_video.c  |  24 +++-
 3 files changed, 105 insertions(+), 25 deletions(-)

diff --git a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
index bb0739f684f3..16a78a00c6c9 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
@@ -138,14 +138,6 @@ static const struct vsp1_format_info vsp1_video_formats[] = {
 	  VI6_FMT_ARGB_8888, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
 	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
 	  1, { 32, 0, 0 }, false, false, 1, 1, false },
-	{ V4L2_PIX_FMT_HSV24, MEDIA_BUS_FMT_AHSV8888_1X32,
-	  VI6_FMT_RGB_888, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
-	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
-	  1, { 24, 0, 0 }, false, false, 1, 1, false },
-	{ V4L2_PIX_FMT_HSV32, MEDIA_BUS_FMT_AHSV8888_1X32,
-	  VI6_FMT_ARGB_8888, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
-	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
-	  1, { 32, 0, 0 }, false, false, 1, 1, false },
 	{ V4L2_PIX_FMT_RGBX1010102, MEDIA_BUS_FMT_ARGB8888_1X32,
 	  VI6_FMT_RGB10_RGB10A2_A2RGB10,
 	  VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS,
@@ -162,10 +154,6 @@ static const struct vsp1_format_info vsp1_video_formats[] = {
 	  VI6_FMT_YUYV_422, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
 	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
 	  1, { 16, 0, 0 }, false, false, 2, 1, false },
-	{ V4L2_PIX_FMT_VYUY, MEDIA_BUS_FMT_AYUV8_1X32,
-	  VI6_FMT_YUYV_422, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
-	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
-	  1, { 16, 0, 0 }, false, true, 2, 1, false },
 	{ V4L2_PIX_FMT_YUYV, MEDIA_BUS_FMT_AYUV8_1X32,
 	  VI6_FMT_YUYV_422, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
 	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
@@ -222,6 +210,21 @@ static const struct vsp1_format_info vsp1_video_formats[] = {
 	  1, { 32, 0, 0 }, false, false, 2, 1, false },
 };
 
+static const struct vsp1_format_info vsp1_video_gen2_formats[] = {
+	{ V4L2_PIX_FMT_VYUY, MEDIA_BUS_FMT_AYUV8_1X32,
+	  VI6_FMT_YUYV_422, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
+	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
+	  1, { 16, 0, 0 }, false, true, 2, 1, false },
+	{ V4L2_PIX_FMT_HSV24, MEDIA_BUS_FMT_AHSV8888_1X32,
+	  VI6_FMT_RGB_888, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
+	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
+	  1, { 24, 0, 0 }, false, false, 1, 1, false },
+	{ V4L2_PIX_FMT_HSV32, MEDIA_BUS_FMT_AHSV8888_1X32,
+	  VI6_FMT_ARGB_8888, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
+	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
+	  1, { 32, 0, 0 }, false, false, 1, 1, false },
+};
+
 /**
  * vsp1_get_format_info - Retrieve format information for a 4CC
  * @vsp1: the VSP1 device
@@ -235,16 +238,6 @@ const struct vsp1_format_info *vsp1_get_format_info(struct vsp1_device *vsp1,
 {
 	unsigned int i;
 
-	/* Special case, the VYUY and HSV formats are supported on Gen2 only. */
-	if (vsp1->info->gen != 2) {
-		switch (fourcc) {
-		case V4L2_PIX_FMT_VYUY:
-		case V4L2_PIX_FMT_HSV24:
-		case V4L2_PIX_FMT_HSV32:
-			return NULL;
-		}
-	}
-
 	for (i = 0; i < ARRAY_SIZE(vsp1_video_formats); ++i) {
 		const struct vsp1_format_info *info = &vsp1_video_formats[i];
 
@@ -252,6 +245,72 @@ const struct vsp1_format_info *vsp1_get_format_info(struct vsp1_device *vsp1,
 			return info;
 	}
 
+	if (vsp1->info->gen == 2) {
+		for (i = 0; i < ARRAY_SIZE(vsp1_video_gen2_formats); ++i) {
+			const struct vsp1_format_info *info =
+				&vsp1_video_gen2_formats[i];
+
+			if (info->fourcc == fourcc)
+				return info;
+		}
+	}
+
+	return NULL;
+}
+
+/**
+ * vsp1_get_format_info_by_index - Enumerate format information
+ * @vsp1: the VSP1 device
+ * @index: the format index
+ * @code: media bus code to limit enumeration
+ *
+ * Return a pointer to the format information structure corresponding to the
+ * given index, or NULL if the index exceeds the supported formats list. If the
+ * @code parameter is not zero, only formats compatible with the media bus code
+ * will be enumerated.
+ */
+const struct vsp1_format_info *
+vsp1_get_format_info_by_index(struct vsp1_device *vsp1, unsigned int index,
+			      u32 code)
+{
+	unsigned int i;
+
+	if (!code) {
+		if (index < ARRAY_SIZE(vsp1_video_formats))
+			return &vsp1_video_formats[index];
+
+		if (vsp1->info->gen == 2) {
+			index -= ARRAY_SIZE(vsp1_video_formats);
+			if (index < ARRAY_SIZE(vsp1_video_gen2_formats))
+				return &vsp1_video_gen2_formats[index];
+		}
+
+		return NULL;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(vsp1_video_formats); ++i) {
+		const struct vsp1_format_info *info = &vsp1_video_formats[i];
+
+		if (info->mbus == code) {
+			if (!index)
+				return info;
+			index--;
+		}
+	}
+
+	if (vsp1->info->gen == 2) {
+		for (i = 0; i < ARRAY_SIZE(vsp1_video_gen2_formats); ++i) {
+			const struct vsp1_format_info *info =
+				&vsp1_video_gen2_formats[i];
+
+			if (info->mbus == code) {
+				if (!index)
+					return info;
+				index--;
+			}
+		}
+	}
+
 	return NULL;
 }
 
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_pipe.h b/drivers/media/platform/renesas/vsp1/vsp1_pipe.h
index 1ba7bdbad5a8..1d3d033af209 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_pipe.h
+++ b/drivers/media/platform/renesas/vsp1/vsp1_pipe.h
@@ -179,5 +179,8 @@ void vsp1_pipeline_calculate_partition(struct vsp1_pipeline *pipe,
 
 const struct vsp1_format_info *vsp1_get_format_info(struct vsp1_device *vsp1,
 						    u32 fourcc);
+const struct vsp1_format_info *
+vsp1_get_format_info_by_index(struct vsp1_device *vsp1, unsigned int index,
+			      u32 code);
 
 #endif /* __VSP1_PIPE_H__ */
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_video.c b/drivers/media/platform/renesas/vsp1/vsp1_video.c
index 03f4efd6b82b..da578993f472 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_video.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_video.c
@@ -888,7 +888,7 @@ vsp1_video_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
 	struct vsp1_video *video = to_vsp1_video(vfh->vdev);
 
 	cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
-			  | V4L2_CAP_VIDEO_CAPTURE_MPLANE
+			  | V4L2_CAP_IO_MC | V4L2_CAP_VIDEO_CAPTURE_MPLANE
 			  | V4L2_CAP_VIDEO_OUTPUT_MPLANE;
 
 
@@ -898,6 +898,22 @@ vsp1_video_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
 	return 0;
 }
 
+static int vsp1_video_enum_format(struct file *file, void *fh,
+				  struct v4l2_fmtdesc *f)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct vsp1_video *video = to_vsp1_video(vfh->vdev);
+	const struct vsp1_format_info *info;
+
+	info = vsp1_get_format_info_by_index(video->vsp1, f->index, f->mbus_code);
+	if (!info)
+		return -EINVAL;
+
+	f->pixelformat = info->fourcc;
+
+	return 0;
+}
+
 static int
 vsp1_video_get_format(struct file *file, void *fh, struct v4l2_format *format)
 {
@@ -1013,6 +1029,8 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 
 static const struct v4l2_ioctl_ops vsp1_video_ioctl_ops = {
 	.vidioc_querycap		= vsp1_video_querycap,
+	.vidioc_enum_fmt_vid_cap	= vsp1_video_enum_format,
+	.vidioc_enum_fmt_vid_out	= vsp1_video_enum_format,
 	.vidioc_g_fmt_vid_cap_mplane	= vsp1_video_get_format,
 	.vidioc_s_fmt_vid_cap_mplane	= vsp1_video_set_format,
 	.vidioc_try_fmt_vid_cap_mplane	= vsp1_video_try_format,
@@ -1207,14 +1225,14 @@ struct vsp1_video *vsp1_video_create(struct vsp1_device *vsp1,
 		video->pad.flags = MEDIA_PAD_FL_SOURCE;
 		video->video.vfl_dir = VFL_DIR_TX;
 		video->video.device_caps = V4L2_CAP_VIDEO_OUTPUT_MPLANE |
-					   V4L2_CAP_STREAMING;
+					   V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
 	} else {
 		direction = "output";
 		video->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
 		video->pad.flags = MEDIA_PAD_FL_SINK;
 		video->video.vfl_dir = VFL_DIR_RX;
 		video->video.device_caps = V4L2_CAP_VIDEO_CAPTURE_MPLANE |
-					   V4L2_CAP_STREAMING;
+					   V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
 	}
 
 	mutex_init(&video->lock);
-- 
Regards,

Laurent Pinchart

[PATCH v2 2/9] media: renesas: vsp1: Make HSI and HST modules optional

[Laurent Pinchart]

Not all VSP instance incorporate the HSI and HST modules. Add a
VSP1_HAS_HSIT feature flag, and create the modules only on VSP instances
that implement them.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/vsp1/vsp1.h    |  1 +
 .../media/platform/renesas/vsp1/vsp1_drv.c    | 59 ++++++++++---------
 2 files changed, 33 insertions(+), 27 deletions(-)

diff --git a/drivers/media/platform/renesas/vsp1/vsp1.h b/drivers/media/platform/renesas/vsp1/vsp1.h
index 2f6f0c6ae555..44cb991f094f 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1.h
+++ b/drivers/media/platform/renesas/vsp1/vsp1.h
@@ -56,6 +56,7 @@ struct vsp1_uif;
 #define VSP1_HAS_BRS		BIT(9)
 #define VSP1_HAS_EXT_DL		BIT(10)
 #define VSP1_HAS_NON_ZERO_LBA	BIT(11)
+#define VSP1_HAS_HSIT		BIT(12)
 
 struct vsp1_device_info {
 	u32 version;
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drv.c b/drivers/media/platform/renesas/vsp1/vsp1_drv.c
index 9fc6bf624a52..4697bf790954 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_drv.c
@@ -302,22 +302,6 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
 		list_add_tail(&vsp1->clu->entity.list_dev, &vsp1->entities);
 	}
 
-	vsp1->hsi = vsp1_hsit_create(vsp1, true);
-	if (IS_ERR(vsp1->hsi)) {
-		ret = PTR_ERR(vsp1->hsi);
-		goto done;
-	}
-
-	list_add_tail(&vsp1->hsi->entity.list_dev, &vsp1->entities);
-
-	vsp1->hst = vsp1_hsit_create(vsp1, false);
-	if (IS_ERR(vsp1->hst)) {
-		ret = PTR_ERR(vsp1->hst);
-		goto done;
-	}
-
-	list_add_tail(&vsp1->hst->entity.list_dev, &vsp1->entities);
-
 	if (vsp1_feature(vsp1, VSP1_HAS_HGO) && vsp1->info->uapi) {
 		vsp1->hgo = vsp1_hgo_create(vsp1);
 		if (IS_ERR(vsp1->hgo)) {
@@ -340,6 +324,24 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
 			      &vsp1->entities);
 	}
 
+	if (vsp1_feature(vsp1, VSP1_HAS_HSIT)) {
+		vsp1->hsi = vsp1_hsit_create(vsp1, true);
+		if (IS_ERR(vsp1->hsi)) {
+			ret = PTR_ERR(vsp1->hsi);
+			goto done;
+		}
+
+		list_add_tail(&vsp1->hsi->entity.list_dev, &vsp1->entities);
+
+		vsp1->hst = vsp1_hsit_create(vsp1, false);
+		if (IS_ERR(vsp1->hst)) {
+			ret = PTR_ERR(vsp1->hst);
+			goto done;
+		}
+
+		list_add_tail(&vsp1->hst->entity.list_dev, &vsp1->entities);
+	}
+
 	/*
 	 * The LIFs are only supported when used in conjunction with the DU, in
 	 * which case the userspace API is disabled. If the userspace API is
@@ -683,8 +685,8 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.model = "VSP1-S",
 		.gen = 2,
 		.features = VSP1_HAS_BRU | VSP1_HAS_CLU | VSP1_HAS_HGO
-			  | VSP1_HAS_HGT | VSP1_HAS_LUT | VSP1_HAS_SRU
-			  | VSP1_HAS_WPF_VFLIP,
+			  | VSP1_HAS_HGT | VSP1_HAS_HSIT | VSP1_HAS_LUT
+			  | VSP1_HAS_SRU | VSP1_HAS_WPF_VFLIP,
 		.rpf_count = 5,
 		.uds_count = 3,
 		.wpf_count = 4,
@@ -694,7 +696,8 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.version = VI6_IP_VERSION_MODEL_VSPR_H2,
 		.model = "VSP1-R",
 		.gen = 2,
-		.features = VSP1_HAS_BRU | VSP1_HAS_SRU | VSP1_HAS_WPF_VFLIP,
+		.features = VSP1_HAS_BRU | VSP1_HAS_HSIT | VSP1_HAS_SRU
+			   | VSP1_HAS_WPF_VFLIP,
 		.rpf_count = 5,
 		.uds_count = 3,
 		.wpf_count = 4,
@@ -704,7 +707,8 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.version = VI6_IP_VERSION_MODEL_VSPD_GEN2,
 		.model = "VSP1-D",
 		.gen = 2,
-		.features = VSP1_HAS_BRU | VSP1_HAS_HGO | VSP1_HAS_LUT,
+		.features = VSP1_HAS_BRU | VSP1_HAS_HGO | VSP1_HAS_HSIT
+			  | VSP1_HAS_LUT,
 		.lif_count = 1,
 		.rpf_count = 4,
 		.uds_count = 1,
@@ -716,8 +720,8 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.model = "VSP1-S",
 		.gen = 2,
 		.features = VSP1_HAS_BRU | VSP1_HAS_CLU | VSP1_HAS_HGO
-			  | VSP1_HAS_HGT | VSP1_HAS_LUT | VSP1_HAS_SRU
-			  | VSP1_HAS_WPF_VFLIP,
+			  | VSP1_HAS_HGT | VSP1_HAS_HSIT | VSP1_HAS_LUT
+			  | VSP1_HAS_SRU | VSP1_HAS_WPF_VFLIP,
 		.rpf_count = 5,
 		.uds_count = 1,
 		.wpf_count = 4,
@@ -727,8 +731,8 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.version = VI6_IP_VERSION_MODEL_VSPS_V2H,
 		.model = "VSP1V-S",
 		.gen = 2,
-		.features = VSP1_HAS_BRU | VSP1_HAS_CLU | VSP1_HAS_LUT
-			  | VSP1_HAS_SRU | VSP1_HAS_WPF_VFLIP,
+		.features = VSP1_HAS_BRU | VSP1_HAS_CLU | VSP1_HAS_HSIT
+			  | VSP1_HAS_LUT | VSP1_HAS_SRU | VSP1_HAS_WPF_VFLIP,
 		.rpf_count = 4,
 		.uds_count = 1,
 		.wpf_count = 4,
@@ -738,7 +742,8 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.version = VI6_IP_VERSION_MODEL_VSPD_V2H,
 		.model = "VSP1V-D",
 		.gen = 2,
-		.features = VSP1_HAS_BRU | VSP1_HAS_CLU | VSP1_HAS_LUT,
+		.features = VSP1_HAS_BRU | VSP1_HAS_CLU | VSP1_HAS_HSIT
+			  | VSP1_HAS_LUT,
 		.lif_count = 1,
 		.rpf_count = 4,
 		.uds_count = 1,
@@ -750,8 +755,8 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.model = "VSP2-I",
 		.gen = 3,
 		.features = VSP1_HAS_CLU | VSP1_HAS_HGO | VSP1_HAS_HGT
-			  | VSP1_HAS_LUT | VSP1_HAS_SRU | VSP1_HAS_WPF_HFLIP
-			  | VSP1_HAS_WPF_VFLIP,
+			  | VSP1_HAS_HSIT | VSP1_HAS_LUT | VSP1_HAS_SRU
+			  | VSP1_HAS_WPF_HFLIP | VSP1_HAS_WPF_VFLIP,
 		.rpf_count = 1,
 		.uds_count = 1,
 		.wpf_count = 1,
-- 
Regards,

Laurent Pinchart

[PATCH v2 3/9] media: renesas: vsp1: Fix HSV format enumeration

[Laurent Pinchart]

The HSV formats are not restricted to Gen2 platforms, but to VSP
instances that implement the HSI and HST modules. Make it conditional to
the VSP1_HAS_HSIT feature flag.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 .../media/platform/renesas/vsp1/vsp1_pipe.c   | 32 +++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
index 16a78a00c6c9..f7b133536704 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
@@ -215,6 +215,9 @@ static const struct vsp1_format_info vsp1_video_gen2_formats[] = {
 	  VI6_FMT_YUYV_422, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
 	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
 	  1, { 16, 0, 0 }, false, true, 2, 1, false },
+};
+
+static const struct vsp1_format_info vsp1_video_hsit_formats[] = {
 	{ V4L2_PIX_FMT_HSV24, MEDIA_BUS_FMT_AHSV8888_1X32,
 	  VI6_FMT_RGB_888, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
 	  VI6_RPF_DSWAP_P_WDS | VI6_RPF_DSWAP_P_BTS,
@@ -255,6 +258,16 @@ const struct vsp1_format_info *vsp1_get_format_info(struct vsp1_device *vsp1,
 		}
 	}
 
+	if (vsp1_feature(vsp1, VSP1_HAS_HSIT)) {
+		for (i = 0; i < ARRAY_SIZE(vsp1_video_hsit_formats); ++i) {
+			const struct vsp1_format_info *info =
+				&vsp1_video_hsit_formats[i];
+
+			if (info->fourcc == fourcc)
+				return info;
+		}
+	}
+
 	return NULL;
 }
 
@@ -285,6 +298,12 @@ vsp1_get_format_info_by_index(struct vsp1_device *vsp1, unsigned int index,
 				return &vsp1_video_gen2_formats[index];
 		}
 
+		if (vsp1_feature(vsp1, VSP1_HAS_HSIT)) {
+			index -= ARRAY_SIZE(vsp1_video_gen2_formats);
+			if (index < ARRAY_SIZE(vsp1_video_hsit_formats))
+				return &vsp1_video_hsit_formats[index];
+		}
+
 		return NULL;
 	}
 
@@ -311,6 +330,19 @@ vsp1_get_format_info_by_index(struct vsp1_device *vsp1, unsigned int index,
 		}
 	}
 
+	if (vsp1_feature(vsp1, VSP1_HAS_HSIT)) {
+		for (i = 0; i < ARRAY_SIZE(vsp1_video_hsit_formats); ++i) {
+			const struct vsp1_format_info *info =
+				&vsp1_video_hsit_formats[i];
+
+			if (info->mbus == code) {
+				if (!index)
+					return info;
+				index--;
+			}
+		}
+	}
+
 	return NULL;
 }
 
-- 
Regards,

Laurent Pinchart

[PATCH v2 4/9] media: renesas: vsp1: Fix media bus code setup on RWPF source pad

[Laurent Pinchart]

The RWPF source pad media bus code can only be different from the sink
pad code when enabling color space conversion, which can only convert
between RGB and YUV. If the sink pad code is HSV, no conversion is
possible. Fix the pad set format handler to reflect this hardware
limitation.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/vsp1/vsp1_rwpf.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
index 9d38203e73d0..1b4bac7b7cfa 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
@@ -76,11 +76,20 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 	format = v4l2_subdev_state_get_format(state, fmt->pad);
 
 	if (fmt->pad == RWPF_PAD_SOURCE) {
+		const struct v4l2_mbus_framefmt *sink_format =
+			v4l2_subdev_state_get_format(state, RWPF_PAD_SINK);
+
 		/*
 		 * The RWPF performs format conversion but can't scale, only the
-		 * format code can be changed on the source pad.
+		 * format code can be changed on the source pad when converting
+		 * between RGB and YUV.
 		 */
-		format->code = fmt->format.code;
+		if (sink_format->code != MEDIA_BUS_FMT_AHSV8888_1X32 &&
+		    fmt->format.code != MEDIA_BUS_FMT_AHSV8888_1X32)
+			format->code = fmt->format.code;
+		else
+			format->code = sink_format->code;
+
 		fmt->format = *format;
 		goto done;
 	}
-- 
Regards,

Laurent Pinchart

[PATCH v2 5/9] media: renesas: vsp1: Report colour space information to userspace

[Laurent Pinchart]

The vsp1 driver implements very partial colour space support: it
hardcodes the colorspace field on all video devices and subdevices to
V4L2_COLORSPACE_SRGB, regardless of the configured format. The
xfer_func, ycbcr_enc and quantization fields are not set (except for
hsv_enc for HSV formats on video devices). This doesn't match the
hardware configuration, which handles YUV data as encoding in BT.601
with limited range.

As a first step towards colour space configuration, keep the colour
space fields hardcoded, but set them based on the selected format type
(RGB, YUV or HSV).

While at it, remove an extra blank line.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
Changes since v1:

- Drop unneeded colorspace adjustements when propagating RWPF formats
---
 .../media/platform/renesas/vsp1/vsp1_brx.c    |  9 +++-
 .../media/platform/renesas/vsp1/vsp1_entity.c | 22 +++++++++-
 .../media/platform/renesas/vsp1/vsp1_entity.h |  2 +
 .../media/platform/renesas/vsp1/vsp1_hsit.c   | 11 ++++-
 .../media/platform/renesas/vsp1/vsp1_pipe.c   | 44 +++++++++++++++++++
 .../media/platform/renesas/vsp1/vsp1_pipe.h   |  2 +
 .../media/platform/renesas/vsp1/vsp1_rwpf.c   | 11 ++++-
 .../media/platform/renesas/vsp1/vsp1_sru.c    |  9 +++-
 .../media/platform/renesas/vsp1/vsp1_uds.c    |  9 +++-
 .../media/platform/renesas/vsp1/vsp1_video.c  |  7 +--
 10 files changed, 115 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/renesas/vsp1/vsp1_brx.c b/drivers/media/platform/renesas/vsp1/vsp1_brx.c
index 5dee0490c593..5fc2e5a3bb30 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_brx.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_brx.c
@@ -15,6 +15,7 @@
 #include "vsp1.h"
 #include "vsp1_brx.h"
 #include "vsp1_dl.h"
+#include "vsp1_entity.h"
 #include "vsp1_pipe.h"
 #include "vsp1_rwpf.h"
 #include "vsp1_video.h"
@@ -108,6 +109,8 @@ static void brx_try_format(struct vsp1_brx *brx,
 		if (fmt->code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
 		    fmt->code != MEDIA_BUS_FMT_AYUV8_1X32)
 			fmt->code = MEDIA_BUS_FMT_AYUV8_1X32;
+
+		vsp1_entity_adjust_color_space(fmt);
 		break;
 
 	default:
@@ -115,13 +118,17 @@ static void brx_try_format(struct vsp1_brx *brx,
 		format = v4l2_subdev_state_get_format(sd_state,
 						      BRX_PAD_SINK(0));
 		fmt->code = format->code;
+
+		fmt->colorspace = format->colorspace;
+		fmt->xfer_func = format->xfer_func;
+		fmt->ycbcr_enc = format->ycbcr_enc;
+		fmt->quantization = format->quantization;
 		break;
 	}
 
 	fmt->width = clamp(fmt->width, BRX_MIN_SIZE, BRX_MAX_SIZE);
 	fmt->height = clamp(fmt->height, BRX_MIN_SIZE, BRX_MAX_SIZE);
 	fmt->field = V4L2_FIELD_NONE;
-	fmt->colorspace = V4L2_COLORSPACE_SRGB;
 }
 
 static int brx_set_format(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_entity.c b/drivers/media/platform/renesas/vsp1/vsp1_entity.c
index 8b8945bd8f10..9f93ae86b1da 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_entity.c
@@ -99,6 +99,20 @@ void vsp1_entity_configure_partition(struct vsp1_entity *entity,
 						 dl, dlb);
 }
 
+void vsp1_entity_adjust_color_space(struct v4l2_mbus_framefmt *format)
+{
+	u8 xfer_func = format->xfer_func;
+	u8 ycbcr_enc = format->ycbcr_enc;
+	u8 quantization = format->quantization;
+
+	vsp1_adjust_color_space(format->code, &format->colorspace, &xfer_func,
+				&ycbcr_enc, &quantization);
+
+	format->xfer_func = xfer_func;
+	format->ycbcr_enc = ycbcr_enc;
+	format->quantization = quantization;
+}
+
 /* -----------------------------------------------------------------------------
  * V4L2 Subdevice Operations
  */
@@ -329,7 +343,13 @@ int vsp1_subdev_set_pad_format(struct v4l2_subdev *subdev,
 	format->height = clamp_t(unsigned int, fmt->format.height,
 				 min_height, max_height);
 	format->field = V4L2_FIELD_NONE;
-	format->colorspace = V4L2_COLORSPACE_SRGB;
+
+	format->colorspace = fmt->format.colorspace;
+	format->xfer_func = fmt->format.xfer_func;
+	format->ycbcr_enc = fmt->format.ycbcr_enc;
+	format->quantization = fmt->format.quantization;
+
+	vsp1_entity_adjust_color_space(format);
 
 	fmt->format = *format;
 
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_entity.h b/drivers/media/platform/renesas/vsp1/vsp1_entity.h
index 1bcc9e27dfdc..ce4a09610164 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/renesas/vsp1/vsp1_entity.h
@@ -170,6 +170,8 @@ void vsp1_entity_configure_partition(struct vsp1_entity *entity,
 				     struct vsp1_dl_list *dl,
 				     struct vsp1_dl_body *dlb);
 
+void vsp1_entity_adjust_color_space(struct v4l2_mbus_framefmt *format);
+
 struct media_pad *vsp1_entity_remote_pad(struct media_pad *pad);
 
 int vsp1_subdev_get_pad_format(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_hsit.c b/drivers/media/platform/renesas/vsp1/vsp1_hsit.c
index 8ba2a7c7305c..1fcd1967d3b2 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_hsit.c
@@ -14,6 +14,7 @@
 
 #include "vsp1.h"
 #include "vsp1_dl.h"
+#include "vsp1_entity.h"
 #include "vsp1_hsit.h"
 
 #define HSIT_MIN_SIZE				4U
@@ -96,7 +97,13 @@ static int hsit_set_format(struct v4l2_subdev *subdev,
 	format->height = clamp_t(unsigned int, fmt->format.height,
 				 HSIT_MIN_SIZE, HSIT_MAX_SIZE);
 	format->field = V4L2_FIELD_NONE;
-	format->colorspace = V4L2_COLORSPACE_SRGB;
+
+	format->colorspace = fmt->format.colorspace;
+	format->xfer_func = fmt->format.xfer_func;
+	format->ycbcr_enc = fmt->format.ycbcr_enc;
+	format->quantization = fmt->format.quantization;
+
+	vsp1_entity_adjust_color_space(format);
 
 	fmt->format = *format;
 
@@ -106,6 +113,8 @@ static int hsit_set_format(struct v4l2_subdev *subdev,
 	format->code = hsit->inverse ? MEDIA_BUS_FMT_ARGB8888_1X32
 		     : MEDIA_BUS_FMT_AHSV8888_1X32;
 
+	vsp1_entity_adjust_color_space(format);
+
 done:
 	mutex_unlock(&hsit->entity.lock);
 	return ret;
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
index f7b133536704..b9ab6c9c96df 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
@@ -346,6 +346,50 @@ vsp1_get_format_info_by_index(struct vsp1_device *vsp1, unsigned int index,
 	return NULL;
 }
 
+/**
+ * vsp1_adjust_color_space - Adjust color space fields in a format
+ * @code: the media bus code
+ * @colorspace: the colorspace
+ * @xfer_func: the transfer function
+ * @encoding: the encoding
+ * @quantization: the quantization
+ *
+ * This function adjusts all color space fields of a video device of subdev
+ * format structure, taking into account the requested format, requested color
+ * space and limitations of the VSP1. It should be used in the video device and
+ * subdev set format handlers.
+ *
+ * For now, simply hardcode the color space fields to the VSP1 defaults based
+ * on the media bus code.
+ */
+void vsp1_adjust_color_space(u32 code, u32 *colorspace, u8 *xfer_func,
+			     u8 *encoding, u8 *quantization)
+{
+	switch (code) {
+	case MEDIA_BUS_FMT_ARGB8888_1X32:
+	default:
+		*colorspace = V4L2_COLORSPACE_SRGB;
+		*xfer_func = V4L2_XFER_FUNC_SRGB;
+		*encoding = V4L2_YCBCR_ENC_601;
+		*quantization = V4L2_QUANTIZATION_FULL_RANGE;
+		break;
+
+	case MEDIA_BUS_FMT_AHSV8888_1X32:
+		*colorspace = V4L2_COLORSPACE_SRGB;
+		*xfer_func = V4L2_XFER_FUNC_SRGB;
+		*encoding = V4L2_HSV_ENC_256;
+		*quantization = V4L2_QUANTIZATION_FULL_RANGE;
+		break;
+
+	case MEDIA_BUS_FMT_AYUV8_1X32:
+		*colorspace = V4L2_COLORSPACE_SMPTE170M;
+		*xfer_func = V4L2_XFER_FUNC_709;
+		*encoding = V4L2_YCBCR_ENC_601;
+		*quantization = V4L2_QUANTIZATION_LIM_RANGE;
+		break;
+	}
+}
+
 /* -----------------------------------------------------------------------------
  * Pipeline Management
  */
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_pipe.h b/drivers/media/platform/renesas/vsp1/vsp1_pipe.h
index 1d3d033af209..c88a3f0d5b1e 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_pipe.h
+++ b/drivers/media/platform/renesas/vsp1/vsp1_pipe.h
@@ -182,5 +182,7 @@ const struct vsp1_format_info *vsp1_get_format_info(struct vsp1_device *vsp1,
 const struct vsp1_format_info *
 vsp1_get_format_info_by_index(struct vsp1_device *vsp1, unsigned int index,
 			      u32 code);
+void vsp1_adjust_color_space(u32 code, u32 *colorspace, u8 *xfer_func,
+			     u8 *encoding, u8 *quantization);
 
 #endif /* __VSP1_PIPE_H__ */
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
index 1b4bac7b7cfa..4e8bcf6a59ad 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
@@ -10,6 +10,7 @@
 #include <media/v4l2-subdev.h>
 
 #include "vsp1.h"
+#include "vsp1_entity.h"
 #include "vsp1_rwpf.h"
 #include "vsp1_video.h"
 
@@ -90,6 +91,8 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 		else
 			format->code = sink_format->code;
 
+		vsp1_entity_adjust_color_space(format);
+
 		fmt->format = *format;
 		goto done;
 	}
@@ -100,7 +103,13 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 	format->height = clamp_t(unsigned int, fmt->format.height,
 				 RWPF_MIN_HEIGHT, rwpf->max_height);
 	format->field = V4L2_FIELD_NONE;
-	format->colorspace = V4L2_COLORSPACE_SRGB;
+
+	format->colorspace = fmt->format.colorspace;
+	format->xfer_func = fmt->format.xfer_func;
+	format->ycbcr_enc = fmt->format.ycbcr_enc;
+	format->quantization = fmt->format.quantization;
+
+	vsp1_entity_adjust_color_space(format);
 
 	fmt->format = *format;
 
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_sru.c b/drivers/media/platform/renesas/vsp1/vsp1_sru.c
index 1759ce642e6e..bba2872afaf2 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_sru.c
@@ -14,6 +14,7 @@
 
 #include "vsp1.h"
 #include "vsp1_dl.h"
+#include "vsp1_entity.h"
 #include "vsp1_pipe.h"
 #include "vsp1_sru.h"
 
@@ -178,6 +179,8 @@ static void sru_try_format(struct vsp1_sru *sru,
 		    fmt->code != MEDIA_BUS_FMT_AYUV8_1X32)
 			fmt->code = MEDIA_BUS_FMT_AYUV8_1X32;
 
+		vsp1_entity_adjust_color_space(fmt);
+
 		fmt->width = clamp(fmt->width, SRU_MIN_SIZE, SRU_MAX_SIZE);
 		fmt->height = clamp(fmt->height, SRU_MIN_SIZE, SRU_MAX_SIZE);
 		break;
@@ -187,6 +190,11 @@ static void sru_try_format(struct vsp1_sru *sru,
 		format = v4l2_subdev_state_get_format(sd_state, SRU_PAD_SINK);
 		fmt->code = format->code;
 
+		fmt->colorspace = format->colorspace;
+		fmt->xfer_func = format->xfer_func;
+		fmt->ycbcr_enc = format->ycbcr_enc;
+		fmt->quantization = format->quantization;
+
 		/*
 		 * We can upscale by 2 in both direction, but not independently.
 		 * Compare the input and output rectangles areas (avoiding
@@ -211,7 +219,6 @@ static void sru_try_format(struct vsp1_sru *sru,
 	}
 
 	fmt->field = V4L2_FIELD_NONE;
-	fmt->colorspace = V4L2_COLORSPACE_SRGB;
 }
 
 static int sru_set_format(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_uds.c b/drivers/media/platform/renesas/vsp1/vsp1_uds.c
index c5a38478cf8c..2db473b6f83c 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_uds.c
@@ -14,6 +14,7 @@
 
 #include "vsp1.h"
 #include "vsp1_dl.h"
+#include "vsp1_entity.h"
 #include "vsp1_pipe.h"
 #include "vsp1_uds.h"
 
@@ -177,6 +178,8 @@ static void uds_try_format(struct vsp1_uds *uds,
 		    fmt->code != MEDIA_BUS_FMT_AYUV8_1X32)
 			fmt->code = MEDIA_BUS_FMT_AYUV8_1X32;
 
+		vsp1_entity_adjust_color_space(fmt);
+
 		fmt->width = clamp(fmt->width, UDS_MIN_SIZE, UDS_MAX_SIZE);
 		fmt->height = clamp(fmt->height, UDS_MIN_SIZE, UDS_MAX_SIZE);
 		break;
@@ -186,6 +189,11 @@ static void uds_try_format(struct vsp1_uds *uds,
 		format = v4l2_subdev_state_get_format(sd_state, UDS_PAD_SINK);
 		fmt->code = format->code;
 
+		fmt->colorspace = format->colorspace;
+		fmt->xfer_func = format->xfer_func;
+		fmt->ycbcr_enc = format->ycbcr_enc;
+		fmt->quantization = format->quantization;
+
 		uds_output_limits(format->width, &minimum, &maximum);
 		fmt->width = clamp(fmt->width, minimum, maximum);
 		uds_output_limits(format->height, &minimum, &maximum);
@@ -194,7 +202,6 @@ static void uds_try_format(struct vsp1_uds *uds,
 	}
 
 	fmt->field = V4L2_FIELD_NONE;
-	fmt->colorspace = V4L2_COLORSPACE_SRGB;
 }
 
 static int uds_set_format(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_video.c b/drivers/media/platform/renesas/vsp1/vsp1_video.c
index da578993f472..68d495c20a84 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_video.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_video.c
@@ -127,12 +127,10 @@ static int __vsp1_video_try_format(struct vsp1_video *video,
 		info = vsp1_get_format_info(video->vsp1, VSP1_VIDEO_DEF_FORMAT);
 
 	pix->pixelformat = info->fourcc;
-	pix->colorspace = V4L2_COLORSPACE_SRGB;
 	pix->field = V4L2_FIELD_NONE;
 
-	if (info->fourcc == V4L2_PIX_FMT_HSV24 ||
-	    info->fourcc == V4L2_PIX_FMT_HSV32)
-		pix->hsv_enc = V4L2_HSV_ENC_256;
+	vsp1_adjust_color_space(info->mbus, &pix->colorspace, &pix->xfer_func,
+				&pix->ycbcr_enc, &pix->quantization);
 
 	memset(pix->reserved, 0, sizeof(pix->reserved));
 
@@ -891,7 +889,6 @@ vsp1_video_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
 			  | V4L2_CAP_IO_MC | V4L2_CAP_VIDEO_CAPTURE_MPLANE
 			  | V4L2_CAP_VIDEO_OUTPUT_MPLANE;
 
-
 	strscpy(cap->driver, "vsp1", sizeof(cap->driver));
 	strscpy(cap->card, video->video.name, sizeof(cap->card));
 
-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 5/9] media: renesas: vsp1: Report colour space information to userspace

[Tomi Valkeinen]

Hi,

On 30/04/2025 02:29, Laurent Pinchart wrote:
> The vsp1 driver implements very partial colour space support: it
> hardcodes the colorspace field on all video devices and subdevices to
> V4L2_COLORSPACE_SRGB, regardless of the configured format. The
> xfer_func, ycbcr_enc and quantization fields are not set (except for
> hsv_enc for HSV formats on video devices). This doesn't match the
> hardware configuration, which handles YUV data as encoding in BT.601
> with limited range.
> 
> As a first step towards colour space configuration, keep the colour
> space fields hardcoded, but set them based on the selected format type
> (RGB, YUV or HSV).
> 
> While at it, remove an extra blank line.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
> Changes since v1:
> 
> - Drop unneeded colorspace adjustements when propagating RWPF formats
> ---
>   .../media/platform/renesas/vsp1/vsp1_brx.c    |  9 +++-
>   .../media/platform/renesas/vsp1/vsp1_entity.c | 22 +++++++++-
>   .../media/platform/renesas/vsp1/vsp1_entity.h |  2 +
>   .../media/platform/renesas/vsp1/vsp1_hsit.c   | 11 ++++-
>   .../media/platform/renesas/vsp1/vsp1_pipe.c   | 44 +++++++++++++++++++
>   .../media/platform/renesas/vsp1/vsp1_pipe.h   |  2 +
>   .../media/platform/renesas/vsp1/vsp1_rwpf.c   | 11 ++++-
>   .../media/platform/renesas/vsp1/vsp1_sru.c    |  9 +++-
>   .../media/platform/renesas/vsp1/vsp1_uds.c    |  9 +++-
>   .../media/platform/renesas/vsp1/vsp1_video.c  |  7 +--
>   10 files changed, 115 insertions(+), 11 deletions(-)
> 
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_brx.c b/drivers/media/platform/renesas/vsp1/vsp1_brx.c
> index 5dee0490c593..5fc2e5a3bb30 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_brx.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_brx.c
> @@ -15,6 +15,7 @@
>   #include "vsp1.h"
>   #include "vsp1_brx.h"
>   #include "vsp1_dl.h"
> +#include "vsp1_entity.h"
>   #include "vsp1_pipe.h"
>   #include "vsp1_rwpf.h"
>   #include "vsp1_video.h"
> @@ -108,6 +109,8 @@ static void brx_try_format(struct vsp1_brx *brx,
>   		if (fmt->code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
>   		    fmt->code != MEDIA_BUS_FMT_AYUV8_1X32)
>   			fmt->code = MEDIA_BUS_FMT_AYUV8_1X32;
> +
> +		vsp1_entity_adjust_color_space(fmt);
>   		break;
>   
>   	default:
> @@ -115,13 +118,17 @@ static void brx_try_format(struct vsp1_brx *brx,
>   		format = v4l2_subdev_state_get_format(sd_state,
>   						      BRX_PAD_SINK(0));
>   		fmt->code = format->code;
> +
> +		fmt->colorspace = format->colorspace;
> +		fmt->xfer_func = format->xfer_func;
> +		fmt->ycbcr_enc = format->ycbcr_enc;
> +		fmt->quantization = format->quantization;
>   		break;
>   	}
>   
>   	fmt->width = clamp(fmt->width, BRX_MIN_SIZE, BRX_MAX_SIZE);
>   	fmt->height = clamp(fmt->height, BRX_MIN_SIZE, BRX_MAX_SIZE);
>   	fmt->field = V4L2_FIELD_NONE;
> -	fmt->colorspace = V4L2_COLORSPACE_SRGB;
>   }
>   
>   static int brx_set_format(struct v4l2_subdev *subdev,
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_entity.c b/drivers/media/platform/renesas/vsp1/vsp1_entity.c
> index 8b8945bd8f10..9f93ae86b1da 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_entity.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_entity.c
> @@ -99,6 +99,20 @@ void vsp1_entity_configure_partition(struct vsp1_entity *entity,
>   						 dl, dlb);
>   }
>   
> +void vsp1_entity_adjust_color_space(struct v4l2_mbus_framefmt *format)
> +{
> +	u8 xfer_func = format->xfer_func;
> +	u8 ycbcr_enc = format->ycbcr_enc;
> +	u8 quantization = format->quantization;
> +
> +	vsp1_adjust_color_space(format->code, &format->colorspace, &xfer_func,
> +				&ycbcr_enc, &quantization);
> +
> +	format->xfer_func = xfer_func;
> +	format->ycbcr_enc = ycbcr_enc;
> +	format->quantization = quantization;
> +}
> +
>   /* -----------------------------------------------------------------------------
>    * V4L2 Subdevice Operations
>    */
> @@ -329,7 +343,13 @@ int vsp1_subdev_set_pad_format(struct v4l2_subdev *subdev,
>   	format->height = clamp_t(unsigned int, fmt->format.height,
>   				 min_height, max_height);
>   	format->field = V4L2_FIELD_NONE;
> -	format->colorspace = V4L2_COLORSPACE_SRGB;
> +
> +	format->colorspace = fmt->format.colorspace;
> +	format->xfer_func = fmt->format.xfer_func;
> +	format->ycbcr_enc = fmt->format.ycbcr_enc;
> +	format->quantization = fmt->format.quantization;
> +
> +	vsp1_entity_adjust_color_space(format);
>   
>   	fmt->format = *format;
>   
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_entity.h b/drivers/media/platform/renesas/vsp1/vsp1_entity.h
> index 1bcc9e27dfdc..ce4a09610164 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_entity.h
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_entity.h
> @@ -170,6 +170,8 @@ void vsp1_entity_configure_partition(struct vsp1_entity *entity,
>   				     struct vsp1_dl_list *dl,
>   				     struct vsp1_dl_body *dlb);
>   
> +void vsp1_entity_adjust_color_space(struct v4l2_mbus_framefmt *format);
> +
>   struct media_pad *vsp1_entity_remote_pad(struct media_pad *pad);
>   
>   int vsp1_subdev_get_pad_format(struct v4l2_subdev *subdev,
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_hsit.c b/drivers/media/platform/renesas/vsp1/vsp1_hsit.c
> index 8ba2a7c7305c..1fcd1967d3b2 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_hsit.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_hsit.c
> @@ -14,6 +14,7 @@
>   
>   #include "vsp1.h"
>   #include "vsp1_dl.h"
> +#include "vsp1_entity.h"
>   #include "vsp1_hsit.h"
>   
>   #define HSIT_MIN_SIZE				4U
> @@ -96,7 +97,13 @@ static int hsit_set_format(struct v4l2_subdev *subdev,
>   	format->height = clamp_t(unsigned int, fmt->format.height,
>   				 HSIT_MIN_SIZE, HSIT_MAX_SIZE);
>   	format->field = V4L2_FIELD_NONE;
> -	format->colorspace = V4L2_COLORSPACE_SRGB;
> +
> +	format->colorspace = fmt->format.colorspace;
> +	format->xfer_func = fmt->format.xfer_func;
> +	format->ycbcr_enc = fmt->format.ycbcr_enc;
> +	format->quantization = fmt->format.quantization;
> +
> +	vsp1_entity_adjust_color_space(format);
>   
>   	fmt->format = *format;
>   
> @@ -106,6 +113,8 @@ static int hsit_set_format(struct v4l2_subdev *subdev,
>   	format->code = hsit->inverse ? MEDIA_BUS_FMT_ARGB8888_1X32
>   		     : MEDIA_BUS_FMT_AHSV8888_1X32;
>   
> +	vsp1_entity_adjust_color_space(format);
> +
>   done:
>   	mutex_unlock(&hsit->entity.lock);
>   	return ret;
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
> index f7b133536704..b9ab6c9c96df 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
> @@ -346,6 +346,50 @@ vsp1_get_format_info_by_index(struct vsp1_device *vsp1, unsigned int index,
>   	return NULL;
>   }
>   
> +/**
> + * vsp1_adjust_color_space - Adjust color space fields in a format
> + * @code: the media bus code
> + * @colorspace: the colorspace
> + * @xfer_func: the transfer function
> + * @encoding: the encoding
> + * @quantization: the quantization
> + *
> + * This function adjusts all color space fields of a video device of subdev
> + * format structure, taking into account the requested format, requested color
> + * space and limitations of the VSP1. It should be used in the video device and
> + * subdev set format handlers.
> + *
> + * For now, simply hardcode the color space fields to the VSP1 defaults based
> + * on the media bus code.
> + */
> +void vsp1_adjust_color_space(u32 code, u32 *colorspace, u8 *xfer_func,
> +			     u8 *encoding, u8 *quantization)
> +{
> +	switch (code) {
> +	case MEDIA_BUS_FMT_ARGB8888_1X32:
> +	default:
> +		*colorspace = V4L2_COLORSPACE_SRGB;
> +		*xfer_func = V4L2_XFER_FUNC_SRGB;
> +		*encoding = V4L2_YCBCR_ENC_601;
> +		*quantization = V4L2_QUANTIZATION_FULL_RANGE;
> +		break;
> +
> +	case MEDIA_BUS_FMT_AHSV8888_1X32:
> +		*colorspace = V4L2_COLORSPACE_SRGB;
> +		*xfer_func = V4L2_XFER_FUNC_SRGB;
> +		*encoding = V4L2_HSV_ENC_256;
> +		*quantization = V4L2_QUANTIZATION_FULL_RANGE;
> +		break;
> +
> +	case MEDIA_BUS_FMT_AYUV8_1X32:
> +		*colorspace = V4L2_COLORSPACE_SMPTE170M;
> +		*xfer_func = V4L2_XFER_FUNC_709;
> +		*encoding = V4L2_YCBCR_ENC_601;
> +		*quantization = V4L2_QUANTIZATION_LIM_RANGE;
> +		break;
> +	}
> +}
> +
>   /* -----------------------------------------------------------------------------
>    * Pipeline Management
>    */
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_pipe.h b/drivers/media/platform/renesas/vsp1/vsp1_pipe.h
> index 1d3d033af209..c88a3f0d5b1e 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_pipe.h
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_pipe.h
> @@ -182,5 +182,7 @@ const struct vsp1_format_info *vsp1_get_format_info(struct vsp1_device *vsp1,
>   const struct vsp1_format_info *
>   vsp1_get_format_info_by_index(struct vsp1_device *vsp1, unsigned int index,
>   			      u32 code);
> +void vsp1_adjust_color_space(u32 code, u32 *colorspace, u8 *xfer_func,
> +			     u8 *encoding, u8 *quantization);
>   
>   #endif /* __VSP1_PIPE_H__ */
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
> index 1b4bac7b7cfa..4e8bcf6a59ad 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
> @@ -10,6 +10,7 @@
>   #include <media/v4l2-subdev.h>
>   
>   #include "vsp1.h"
> +#include "vsp1_entity.h"
>   #include "vsp1_rwpf.h"
>   #include "vsp1_video.h"
>   
> @@ -90,6 +91,8 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
>   		else
>   			format->code = sink_format->code;
>   
> +		vsp1_entity_adjust_color_space(format);
> +
>   		fmt->format = *format;
>   		goto done;
>   	}
> @@ -100,7 +103,13 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
>   	format->height = clamp_t(unsigned int, fmt->format.height,
>   				 RWPF_MIN_HEIGHT, rwpf->max_height);
>   	format->field = V4L2_FIELD_NONE;
> -	format->colorspace = V4L2_COLORSPACE_SRGB;
> +
> +	format->colorspace = fmt->format.colorspace;
> +	format->xfer_func = fmt->format.xfer_func;
> +	format->ycbcr_enc = fmt->format.ycbcr_enc;
> +	format->quantization = fmt->format.quantization;
> +
> +	vsp1_entity_adjust_color_space(format);
>   
>   	fmt->format = *format;
>   
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_sru.c b/drivers/media/platform/renesas/vsp1/vsp1_sru.c
> index 1759ce642e6e..bba2872afaf2 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_sru.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_sru.c
> @@ -14,6 +14,7 @@
>   
>   #include "vsp1.h"
>   #include "vsp1_dl.h"
> +#include "vsp1_entity.h"
>   #include "vsp1_pipe.h"
>   #include "vsp1_sru.h"
>   
> @@ -178,6 +179,8 @@ static void sru_try_format(struct vsp1_sru *sru,
>   		    fmt->code != MEDIA_BUS_FMT_AYUV8_1X32)
>   			fmt->code = MEDIA_BUS_FMT_AYUV8_1X32;
>   
> +		vsp1_entity_adjust_color_space(fmt);
> +
>   		fmt->width = clamp(fmt->width, SRU_MIN_SIZE, SRU_MAX_SIZE);
>   		fmt->height = clamp(fmt->height, SRU_MIN_SIZE, SRU_MAX_SIZE);
>   		break;
> @@ -187,6 +190,11 @@ static void sru_try_format(struct vsp1_sru *sru,
>   		format = v4l2_subdev_state_get_format(sd_state, SRU_PAD_SINK);
>   		fmt->code = format->code;
>   
> +		fmt->colorspace = format->colorspace;
> +		fmt->xfer_func = format->xfer_func;
> +		fmt->ycbcr_enc = format->ycbcr_enc;
> +		fmt->quantization = format->quantization;
> +
>   		/*
>   		 * We can upscale by 2 in both direction, but not independently.
>   		 * Compare the input and output rectangles areas (avoiding
> @@ -211,7 +219,6 @@ static void sru_try_format(struct vsp1_sru *sru,
>   	}
>   
>   	fmt->field = V4L2_FIELD_NONE;
> -	fmt->colorspace = V4L2_COLORSPACE_SRGB;
>   }
>   
>   static int sru_set_format(struct v4l2_subdev *subdev,
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_uds.c b/drivers/media/platform/renesas/vsp1/vsp1_uds.c
> index c5a38478cf8c..2db473b6f83c 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_uds.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_uds.c
> @@ -14,6 +14,7 @@
>   
>   #include "vsp1.h"
>   #include "vsp1_dl.h"
> +#include "vsp1_entity.h"
>   #include "vsp1_pipe.h"
>   #include "vsp1_uds.h"
>   
> @@ -177,6 +178,8 @@ static void uds_try_format(struct vsp1_uds *uds,
>   		    fmt->code != MEDIA_BUS_FMT_AYUV8_1X32)
>   			fmt->code = MEDIA_BUS_FMT_AYUV8_1X32;
>   
> +		vsp1_entity_adjust_color_space(fmt);
> +
>   		fmt->width = clamp(fmt->width, UDS_MIN_SIZE, UDS_MAX_SIZE);
>   		fmt->height = clamp(fmt->height, UDS_MIN_SIZE, UDS_MAX_SIZE);
>   		break;
> @@ -186,6 +189,11 @@ static void uds_try_format(struct vsp1_uds *uds,
>   		format = v4l2_subdev_state_get_format(sd_state, UDS_PAD_SINK);
>   		fmt->code = format->code;
>   
> +		fmt->colorspace = format->colorspace;
> +		fmt->xfer_func = format->xfer_func;
> +		fmt->ycbcr_enc = format->ycbcr_enc;
> +		fmt->quantization = format->quantization;
> +
>   		uds_output_limits(format->width, &minimum, &maximum);
>   		fmt->width = clamp(fmt->width, minimum, maximum);
>   		uds_output_limits(format->height, &minimum, &maximum);
> @@ -194,7 +202,6 @@ static void uds_try_format(struct vsp1_uds *uds,
>   	}
>   
>   	fmt->field = V4L2_FIELD_NONE;
> -	fmt->colorspace = V4L2_COLORSPACE_SRGB;
>   }
>   
>   static int uds_set_format(struct v4l2_subdev *subdev,
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_video.c b/drivers/media/platform/renesas/vsp1/vsp1_video.c
> index da578993f472..68d495c20a84 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_video.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_video.c
> @@ -127,12 +127,10 @@ static int __vsp1_video_try_format(struct vsp1_video *video,
>   		info = vsp1_get_format_info(video->vsp1, VSP1_VIDEO_DEF_FORMAT);
>   
>   	pix->pixelformat = info->fourcc;
> -	pix->colorspace = V4L2_COLORSPACE_SRGB;
>   	pix->field = V4L2_FIELD_NONE;
>   
> -	if (info->fourcc == V4L2_PIX_FMT_HSV24 ||
> -	    info->fourcc == V4L2_PIX_FMT_HSV32)
> -		pix->hsv_enc = V4L2_HSV_ENC_256;
> +	vsp1_adjust_color_space(info->mbus, &pix->colorspace, &pix->xfer_func,
> +				&pix->ycbcr_enc, &pix->quantization);
>   
>   	memset(pix->reserved, 0, sizeof(pix->reserved));
>   
> @@ -891,7 +889,6 @@ vsp1_video_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
>   			  | V4L2_CAP_IO_MC | V4L2_CAP_VIDEO_CAPTURE_MPLANE
>   			  | V4L2_CAP_VIDEO_OUTPUT_MPLANE;
>   
> -
>   	strscpy(cap->driver, "vsp1", sizeof(cap->driver));
>   	strscpy(cap->card, video->video.name, sizeof(cap->card));
>   

Reviewed-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>

  Tomi

[PATCH v2 6/9] media: renesas: vsp1: Allow setting encoding and quantization

[Laurent Pinchart]

The RPF and WPF support different encodings and quantizations when
converting between RGB and YUV formats. Allow setting the corresponding
format parameters from userspace, and configure the hardware
accordingly.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 .../media/platform/renesas/vsp1/vsp1_pipe.c   | 29 +++++++++++------
 .../media/platform/renesas/vsp1/vsp1_rpf.c    | 29 +++++++++++++++--
 .../media/platform/renesas/vsp1/vsp1_rwpf.c   | 31 +++++++++++++++++--
 .../media/platform/renesas/vsp1/vsp1_video.c  | 19 ++++++++++++
 .../media/platform/renesas/vsp1/vsp1_wpf.c    | 29 +++++++++++++++--
 5 files changed, 121 insertions(+), 16 deletions(-)

diff --git a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
index b9ab6c9c96df..59e26284a90f 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
@@ -359,33 +359,42 @@ vsp1_get_format_info_by_index(struct vsp1_device *vsp1, unsigned int index,
  * space and limitations of the VSP1. It should be used in the video device and
  * subdev set format handlers.
  *
- * For now, simply hardcode the color space fields to the VSP1 defaults based
- * on the media bus code.
+ * The colorspace and xfer_func fields are freely configurable, as they are out
+ * of scope for VSP processing. The encoding and quantization is hardcoded for
+ * non-YUV formats, and can be configured for YUV formats.
  */
 void vsp1_adjust_color_space(u32 code, u32 *colorspace, u8 *xfer_func,
 			     u8 *encoding, u8 *quantization)
 {
+	if (*colorspace == V4L2_COLORSPACE_DEFAULT ||
+	    *colorspace >= V4L2_COLORSPACE_LAST)
+		*colorspace = code == MEDIA_BUS_FMT_AYUV8_1X32
+			    ? V4L2_COLORSPACE_SMPTE170M
+			    : V4L2_COLORSPACE_SRGB;
+
+	if (*xfer_func == V4L2_XFER_FUNC_DEFAULT ||
+	    *xfer_func >= V4L2_XFER_FUNC_LAST)
+		*xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(*colorspace);
+
 	switch (code) {
 	case MEDIA_BUS_FMT_ARGB8888_1X32:
 	default:
-		*colorspace = V4L2_COLORSPACE_SRGB;
-		*xfer_func = V4L2_XFER_FUNC_SRGB;
 		*encoding = V4L2_YCBCR_ENC_601;
 		*quantization = V4L2_QUANTIZATION_FULL_RANGE;
 		break;
 
 	case MEDIA_BUS_FMT_AHSV8888_1X32:
-		*colorspace = V4L2_COLORSPACE_SRGB;
-		*xfer_func = V4L2_XFER_FUNC_SRGB;
 		*encoding = V4L2_HSV_ENC_256;
 		*quantization = V4L2_QUANTIZATION_FULL_RANGE;
 		break;
 
 	case MEDIA_BUS_FMT_AYUV8_1X32:
-		*colorspace = V4L2_COLORSPACE_SMPTE170M;
-		*xfer_func = V4L2_XFER_FUNC_709;
-		*encoding = V4L2_YCBCR_ENC_601;
-		*quantization = V4L2_QUANTIZATION_LIM_RANGE;
+		if (*encoding != V4L2_YCBCR_ENC_601 &&
+		    *encoding != V4L2_YCBCR_ENC_709)
+			*encoding = V4L2_YCBCR_ENC_601;
+		if (*quantization != V4L2_QUANTIZATION_FULL_RANGE &&
+		    *quantization != V4L2_QUANTIZATION_LIM_RANGE)
+			*quantization = V4L2_QUANTIZATION_LIM_RANGE;
 		break;
 	}
 }
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_rpf.c b/drivers/media/platform/renesas/vsp1/vsp1_rpf.c
index 5c8b3ba1bd3c..9f2744af54bc 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_rpf.c
@@ -92,8 +92,33 @@ static void rpf_configure_stream(struct vsp1_entity *entity,
 	if (fmtinfo->swap_uv)
 		infmt |= VI6_RPF_INFMT_SPUVS;
 
-	if (sink_format->code != source_format->code)
-		infmt |= VI6_RPF_INFMT_CSC;
+	if (sink_format->code != source_format->code) {
+		u16 ycbcr_enc;
+		u16 quantization;
+		u32 rdtm;
+
+		if (sink_format->code == MEDIA_BUS_FMT_AYUV8_1X32) {
+			ycbcr_enc = sink_format->ycbcr_enc;
+			quantization = sink_format->quantization;
+		} else {
+			ycbcr_enc = source_format->ycbcr_enc;
+			quantization = source_format->quantization;
+		}
+
+		if (ycbcr_enc == V4L2_YCBCR_ENC_601 &&
+		    quantization == V4L2_QUANTIZATION_LIM_RANGE)
+			rdtm = VI6_RPF_INFMT_RDTM_BT601;
+		else if (ycbcr_enc == V4L2_YCBCR_ENC_601 &&
+			 quantization == V4L2_QUANTIZATION_FULL_RANGE)
+			rdtm = VI6_RPF_INFMT_RDTM_BT601_EXT;
+		else if (ycbcr_enc == V4L2_YCBCR_ENC_709 &&
+			 quantization == V4L2_QUANTIZATION_LIM_RANGE)
+			rdtm = VI6_RPF_INFMT_RDTM_BT709;
+		else
+			rdtm = VI6_RPF_INFMT_RDTM_BT709_EXT;
+
+		infmt |= VI6_RPF_INFMT_CSC | rdtm;
+	}
 
 	vsp1_rpf_write(rpf, dlb, VI6_RPF_INFMT, infmt);
 	vsp1_rpf_write(rpf, dlb, VI6_RPF_DSWAP, fmtinfo->swap);
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
index 4e8bcf6a59ad..9c8085d5d306 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
@@ -36,6 +36,11 @@ static int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
 
 	code->code = codes[code->index];
 
+	if (code->pad == RWPF_PAD_SOURCE &&
+	    code->code == MEDIA_BUS_FMT_AYUV8_1X32)
+		code->flags = V4L2_SUBDEV_MBUS_CODE_CSC_YCBCR_ENC
+			    | V4L2_SUBDEV_MBUS_CODE_CSC_QUANTIZATION;
+
 	return 0;
 }
 
@@ -79,11 +84,13 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 	if (fmt->pad == RWPF_PAD_SOURCE) {
 		const struct v4l2_mbus_framefmt *sink_format =
 			v4l2_subdev_state_get_format(state, RWPF_PAD_SINK);
+		u16 flags = fmt->format.flags & V4L2_MBUS_FRAMEFMT_SET_CSC;
+		bool csc;
 
 		/*
 		 * The RWPF performs format conversion but can't scale, only the
-		 * format code can be changed on the source pad when converting
-		 * between RGB and YUV.
+		 * format code, encoding and quantization can be changed on the
+		 * source pad when converting between RGB and YUV.
 		 */
 		if (sink_format->code != MEDIA_BUS_FMT_AHSV8888_1X32 &&
 		    fmt->format.code != MEDIA_BUS_FMT_AHSV8888_1X32)
@@ -91,9 +98,29 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 		else
 			format->code = sink_format->code;
 
+		/*
+		 * Encoding and quantization can only be configured when YCbCr
+		 * <-> RGB is enabled. The V4L2 API requires userspace to set
+		 * the V4L2_MBUS_FRAMEFMT_SET_CSC flag. If either of these
+		 * conditions is not met, use the encoding and quantization
+		 * values from the sink pad.
+		 */
+		csc = (format->code == MEDIA_BUS_FMT_AYUV8_1X32) !=
+		      (sink_format->code == MEDIA_BUS_FMT_AYUV8_1X32);
+
+		if (csc && (flags & V4L2_MBUS_FRAMEFMT_SET_CSC)) {
+			format->ycbcr_enc = fmt->format.ycbcr_enc;
+			format->quantization = fmt->format.quantization;
+		} else {
+			format->ycbcr_enc = sink_format->ycbcr_enc;
+			format->quantization = sink_format->quantization;
+		}
+
 		vsp1_entity_adjust_color_space(format);
 
 		fmt->format = *format;
+		fmt->format.flags = flags;
+
 		goto done;
 	}
 
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_video.c b/drivers/media/platform/renesas/vsp1/vsp1_video.c
index 68d495c20a84..bc66fbdde3cc 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_video.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_video.c
@@ -129,6 +129,20 @@ static int __vsp1_video_try_format(struct vsp1_video *video,
 	pix->pixelformat = info->fourcc;
 	pix->field = V4L2_FIELD_NONE;
 
+	/*
+	 * Adjust the colour space fields. On capture devices, userspace needs
+	 * to set the V4L2_PIX_FMT_FLAG_SET_CSC to override the defaults. Reset
+	 * all fields to *_DEFAULT if the flag isn't set, to then handle
+	 * capture and output devices in the same way.
+	 */
+	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	    !(pix->flags & V4L2_PIX_FMT_FLAG_SET_CSC)) {
+		pix->colorspace = V4L2_COLORSPACE_DEFAULT;
+		pix->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+		pix->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+		pix->quantization = V4L2_QUANTIZATION_DEFAULT;
+	}
+
 	vsp1_adjust_color_space(info->mbus, &pix->colorspace, &pix->xfer_func,
 				&pix->ycbcr_enc, &pix->quantization);
 
@@ -908,6 +922,11 @@ static int vsp1_video_enum_format(struct file *file, void *fh,
 
 	f->pixelformat = info->fourcc;
 
+	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	    info->mbus == MEDIA_BUS_FMT_AYUV8_1X32)
+		f->flags = V4L2_FMT_FLAG_CSC_YCBCR_ENC
+			 | V4L2_FMT_FLAG_CSC_QUANTIZATION;
+
 	return 0;
 }
 
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_wpf.c b/drivers/media/platform/renesas/vsp1/vsp1_wpf.c
index f176750ccd98..49af637c8186 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_wpf.c
@@ -279,8 +279,33 @@ static void wpf_configure_stream(struct vsp1_entity *entity,
 				       (256 << VI6_WPF_ROT_CTRL_LMEM_WD_SHIFT));
 	}
 
-	if (sink_format->code != source_format->code)
-		outfmt |= VI6_WPF_OUTFMT_CSC;
+	if (sink_format->code != source_format->code) {
+		u16 ycbcr_enc;
+		u16 quantization;
+		u32 wrtm;
+
+		if (sink_format->code == MEDIA_BUS_FMT_AYUV8_1X32) {
+			ycbcr_enc = sink_format->ycbcr_enc;
+			quantization = sink_format->quantization;
+		} else {
+			ycbcr_enc = source_format->ycbcr_enc;
+			quantization = source_format->quantization;
+		}
+
+		if (ycbcr_enc == V4L2_YCBCR_ENC_601 &&
+		    quantization == V4L2_QUANTIZATION_LIM_RANGE)
+			wrtm = VI6_WPF_OUTFMT_WRTM_BT601;
+		else if (ycbcr_enc == V4L2_YCBCR_ENC_601 &&
+			 quantization == V4L2_QUANTIZATION_FULL_RANGE)
+			wrtm = VI6_WPF_OUTFMT_WRTM_BT601_EXT;
+		else if (ycbcr_enc == V4L2_YCBCR_ENC_709 &&
+			 quantization == V4L2_QUANTIZATION_LIM_RANGE)
+			wrtm = VI6_WPF_OUTFMT_WRTM_BT709;
+		else
+			wrtm = VI6_WPF_OUTFMT_WRTM_BT709_EXT;
+
+		outfmt |= VI6_WPF_OUTFMT_CSC | wrtm;
+	}
 
 	wpf->outfmt = outfmt;
 
-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 6/9] media: renesas: vsp1: Allow setting encoding and quantization

[Tomi Valkeinen]

Hi,

On 30/04/2025 02:29, Laurent Pinchart wrote:
> The RPF and WPF support different encodings and quantizations when
> converting between RGB and YUV formats. Allow setting the corresponding
> format parameters from userspace, and configure the hardware
> accordingly.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>   .../media/platform/renesas/vsp1/vsp1_pipe.c   | 29 +++++++++++------
>   .../media/platform/renesas/vsp1/vsp1_rpf.c    | 29 +++++++++++++++--
>   .../media/platform/renesas/vsp1/vsp1_rwpf.c   | 31 +++++++++++++++++--
>   .../media/platform/renesas/vsp1/vsp1_video.c  | 19 ++++++++++++
>   .../media/platform/renesas/vsp1/vsp1_wpf.c    | 29 +++++++++++++++--
>   5 files changed, 121 insertions(+), 16 deletions(-)
> 
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
> index b9ab6c9c96df..59e26284a90f 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_pipe.c
> @@ -359,33 +359,42 @@ vsp1_get_format_info_by_index(struct vsp1_device *vsp1, unsigned int index,
>    * space and limitations of the VSP1. It should be used in the video device and
>    * subdev set format handlers.
>    *
> - * For now, simply hardcode the color space fields to the VSP1 defaults based
> - * on the media bus code.
> + * The colorspace and xfer_func fields are freely configurable, as they are out
> + * of scope for VSP processing. The encoding and quantization is hardcoded for
> + * non-YUV formats, and can be configured for YUV formats.
>    */
>   void vsp1_adjust_color_space(u32 code, u32 *colorspace, u8 *xfer_func,
>   			     u8 *encoding, u8 *quantization)
>   {
> +	if (*colorspace == V4L2_COLORSPACE_DEFAULT ||
> +	    *colorspace >= V4L2_COLORSPACE_LAST)
> +		*colorspace = code == MEDIA_BUS_FMT_AYUV8_1X32
> +			    ? V4L2_COLORSPACE_SMPTE170M
> +			    : V4L2_COLORSPACE_SRGB;
> +
> +	if (*xfer_func == V4L2_XFER_FUNC_DEFAULT ||
> +	    *xfer_func >= V4L2_XFER_FUNC_LAST)
> +		*xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(*colorspace);
> +
>   	switch (code) {
>   	case MEDIA_BUS_FMT_ARGB8888_1X32:
>   	default:
> -		*colorspace = V4L2_COLORSPACE_SRGB;
> -		*xfer_func = V4L2_XFER_FUNC_SRGB;
>   		*encoding = V4L2_YCBCR_ENC_601;
>   		*quantization = V4L2_QUANTIZATION_FULL_RANGE;
>   		break;
>   
>   	case MEDIA_BUS_FMT_AHSV8888_1X32:
> -		*colorspace = V4L2_COLORSPACE_SRGB;
> -		*xfer_func = V4L2_XFER_FUNC_SRGB;
>   		*encoding = V4L2_HSV_ENC_256;
>   		*quantization = V4L2_QUANTIZATION_FULL_RANGE;
>   		break;
>   
>   	case MEDIA_BUS_FMT_AYUV8_1X32:
> -		*colorspace = V4L2_COLORSPACE_SMPTE170M;
> -		*xfer_func = V4L2_XFER_FUNC_709;
> -		*encoding = V4L2_YCBCR_ENC_601;
> -		*quantization = V4L2_QUANTIZATION_LIM_RANGE;
> +		if (*encoding != V4L2_YCBCR_ENC_601 &&
> +		    *encoding != V4L2_YCBCR_ENC_709)
> +			*encoding = V4L2_YCBCR_ENC_601;
> +		if (*quantization != V4L2_QUANTIZATION_FULL_RANGE &&
> +		    *quantization != V4L2_QUANTIZATION_LIM_RANGE)
> +			*quantization = V4L2_QUANTIZATION_LIM_RANGE;
>   		break;
>   	}
>   }
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_rpf.c b/drivers/media/platform/renesas/vsp1/vsp1_rpf.c
> index 5c8b3ba1bd3c..9f2744af54bc 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_rpf.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_rpf.c
> @@ -92,8 +92,33 @@ static void rpf_configure_stream(struct vsp1_entity *entity,
>   	if (fmtinfo->swap_uv)
>   		infmt |= VI6_RPF_INFMT_SPUVS;
>   
> -	if (sink_format->code != source_format->code)
> -		infmt |= VI6_RPF_INFMT_CSC;
> +	if (sink_format->code != source_format->code) {
> +		u16 ycbcr_enc;
> +		u16 quantization;
> +		u32 rdtm;
> +
> +		if (sink_format->code == MEDIA_BUS_FMT_AYUV8_1X32) {
> +			ycbcr_enc = sink_format->ycbcr_enc;
> +			quantization = sink_format->quantization;
> +		} else {
> +			ycbcr_enc = source_format->ycbcr_enc;
> +			quantization = source_format->quantization;
> +		}
> +
> +		if (ycbcr_enc == V4L2_YCBCR_ENC_601 &&
> +		    quantization == V4L2_QUANTIZATION_LIM_RANGE)
> +			rdtm = VI6_RPF_INFMT_RDTM_BT601;
> +		else if (ycbcr_enc == V4L2_YCBCR_ENC_601 &&
> +			 quantization == V4L2_QUANTIZATION_FULL_RANGE)
> +			rdtm = VI6_RPF_INFMT_RDTM_BT601_EXT;
> +		else if (ycbcr_enc == V4L2_YCBCR_ENC_709 &&
> +			 quantization == V4L2_QUANTIZATION_LIM_RANGE)
> +			rdtm = VI6_RPF_INFMT_RDTM_BT709;
> +		else
> +			rdtm = VI6_RPF_INFMT_RDTM_BT709_EXT;
> +
> +		infmt |= VI6_RPF_INFMT_CSC | rdtm;
> +	}
>   
>   	vsp1_rpf_write(rpf, dlb, VI6_RPF_INFMT, infmt);
>   	vsp1_rpf_write(rpf, dlb, VI6_RPF_DSWAP, fmtinfo->swap);
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
> index 4e8bcf6a59ad..9c8085d5d306 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_rwpf.c
> @@ -36,6 +36,11 @@ static int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
>   
>   	code->code = codes[code->index];
>   
> +	if (code->pad == RWPF_PAD_SOURCE &&
> +	    code->code == MEDIA_BUS_FMT_AYUV8_1X32)
> +		code->flags = V4L2_SUBDEV_MBUS_CODE_CSC_YCBCR_ENC
> +			    | V4L2_SUBDEV_MBUS_CODE_CSC_QUANTIZATION;
> +
>   	return 0;
>   }
>   
> @@ -79,11 +84,13 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
>   	if (fmt->pad == RWPF_PAD_SOURCE) {
>   		const struct v4l2_mbus_framefmt *sink_format =
>   			v4l2_subdev_state_get_format(state, RWPF_PAD_SINK);
> +		u16 flags = fmt->format.flags & V4L2_MBUS_FRAMEFMT_SET_CSC;
> +		bool csc;
>   
>   		/*
>   		 * The RWPF performs format conversion but can't scale, only the
> -		 * format code can be changed on the source pad when converting
> -		 * between RGB and YUV.
> +		 * format code, encoding and quantization can be changed on the
> +		 * source pad when converting between RGB and YUV.
>   		 */
>   		if (sink_format->code != MEDIA_BUS_FMT_AHSV8888_1X32 &&
>   		    fmt->format.code != MEDIA_BUS_FMT_AHSV8888_1X32)
> @@ -91,9 +98,29 @@ static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
>   		else
>   			format->code = sink_format->code;
>   
> +		/*
> +		 * Encoding and quantization can only be configured when YCbCr
> +		 * <-> RGB is enabled. The V4L2 API requires userspace to set
> +		 * the V4L2_MBUS_FRAMEFMT_SET_CSC flag. If either of these
> +		 * conditions is not met, use the encoding and quantization
> +		 * values from the sink pad.
> +		 */
> +		csc = (format->code == MEDIA_BUS_FMT_AYUV8_1X32) !=
> +		      (sink_format->code == MEDIA_BUS_FMT_AYUV8_1X32);
> +
> +		if (csc && (flags & V4L2_MBUS_FRAMEFMT_SET_CSC)) {
> +			format->ycbcr_enc = fmt->format.ycbcr_enc;
> +			format->quantization = fmt->format.quantization;
> +		} else {
> +			format->ycbcr_enc = sink_format->ycbcr_enc;
> +			format->quantization = sink_format->quantization;
> +		}
> +
>   		vsp1_entity_adjust_color_space(format);
>   
>   		fmt->format = *format;
> +		fmt->format.flags = flags;
> +
>   		goto done;
>   	}
>   
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_video.c b/drivers/media/platform/renesas/vsp1/vsp1_video.c
> index 68d495c20a84..bc66fbdde3cc 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_video.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_video.c
> @@ -129,6 +129,20 @@ static int __vsp1_video_try_format(struct vsp1_video *video,
>   	pix->pixelformat = info->fourcc;
>   	pix->field = V4L2_FIELD_NONE;
>   
> +	/*
> +	 * Adjust the colour space fields. On capture devices, userspace needs
> +	 * to set the V4L2_PIX_FMT_FLAG_SET_CSC to override the defaults. Reset
> +	 * all fields to *_DEFAULT if the flag isn't set, to then handle
> +	 * capture and output devices in the same way.
> +	 */
> +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> +	    !(pix->flags & V4L2_PIX_FMT_FLAG_SET_CSC)) {
> +		pix->colorspace = V4L2_COLORSPACE_DEFAULT;
> +		pix->xfer_func = V4L2_XFER_FUNC_DEFAULT;
> +		pix->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
> +		pix->quantization = V4L2_QUANTIZATION_DEFAULT;
> +	}
> +
>   	vsp1_adjust_color_space(info->mbus, &pix->colorspace, &pix->xfer_func,
>   				&pix->ycbcr_enc, &pix->quantization);
>   
> @@ -908,6 +922,11 @@ static int vsp1_video_enum_format(struct file *file, void *fh,
>   
>   	f->pixelformat = info->fourcc;
>   
> +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> +	    info->mbus == MEDIA_BUS_FMT_AYUV8_1X32)
> +		f->flags = V4L2_FMT_FLAG_CSC_YCBCR_ENC
> +			 | V4L2_FMT_FLAG_CSC_QUANTIZATION;
> +
>   	return 0;
>   }
>   
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_wpf.c b/drivers/media/platform/renesas/vsp1/vsp1_wpf.c
> index f176750ccd98..49af637c8186 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_wpf.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_wpf.c
> @@ -279,8 +279,33 @@ static void wpf_configure_stream(struct vsp1_entity *entity,
>   				       (256 << VI6_WPF_ROT_CTRL_LMEM_WD_SHIFT));
>   	}
>   
> -	if (sink_format->code != source_format->code)
> -		outfmt |= VI6_WPF_OUTFMT_CSC;
> +	if (sink_format->code != source_format->code) {
> +		u16 ycbcr_enc;
> +		u16 quantization;
> +		u32 wrtm;
> +
> +		if (sink_format->code == MEDIA_BUS_FMT_AYUV8_1X32) {
> +			ycbcr_enc = sink_format->ycbcr_enc;
> +			quantization = sink_format->quantization;
> +		} else {
> +			ycbcr_enc = source_format->ycbcr_enc;
> +			quantization = source_format->quantization;
> +		}
> +
> +		if (ycbcr_enc == V4L2_YCBCR_ENC_601 &&
> +		    quantization == V4L2_QUANTIZATION_LIM_RANGE)
> +			wrtm = VI6_WPF_OUTFMT_WRTM_BT601;
> +		else if (ycbcr_enc == V4L2_YCBCR_ENC_601 &&
> +			 quantization == V4L2_QUANTIZATION_FULL_RANGE)
> +			wrtm = VI6_WPF_OUTFMT_WRTM_BT601_EXT;
> +		else if (ycbcr_enc == V4L2_YCBCR_ENC_709 &&
> +			 quantization == V4L2_QUANTIZATION_LIM_RANGE)
> +			wrtm = VI6_WPF_OUTFMT_WRTM_BT709;
> +		else
> +			wrtm = VI6_WPF_OUTFMT_WRTM_BT709_EXT;
> +
> +		outfmt |= VI6_WPF_OUTFMT_CSC | wrtm;
> +	}
>   
>   	wpf->outfmt = outfmt;
>   

Reviewed-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>

  Tomi

[PATCH v2 7/9] media: renesas: vsp1: Name nested structure in vsp1_drm

[Laurent Pinchart]

The vsp1_drm structure defines an anonymous nested structure to store
per-input data. In preparation for extending that structure, give it a
name and is it through the driver. This improves code readability.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/vsp1/vsp1_drm.c | 18 +++++++++---------
 drivers/media/platform/renesas/vsp1/vsp1_drm.h |  2 +-
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drm.c b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
index f8a575f6188a..e5339fda5941 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
@@ -118,24 +118,22 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
 				      struct vsp1_entity *uif,
 				      unsigned int brx_input)
 {
+	const struct vsp1_drm_input *input = &vsp1->drm->inputs[rpf->entity.index];
 	struct v4l2_subdev_selection sel = {
 		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
 	};
 	struct v4l2_subdev_format format = {
 		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
 	};
-	const struct v4l2_rect *crop;
 	int ret;
 
 	/*
 	 * Configure the format on the RPF sink pad and propagate it up to the
 	 * BRx sink pad.
 	 */
-	crop = &vsp1->drm->inputs[rpf->entity.index].crop;
-
 	format.pad = RWPF_PAD_SINK;
-	format.format.width = crop->width + crop->left;
-	format.format.height = crop->height + crop->top;
+	format.format.width = input->crop.width + input->crop.left;
+	format.format.height = input->crop.height + input->crop.top;
 	format.format.code = rpf->fmtinfo->mbus;
 	format.format.field = V4L2_FIELD_NONE;
 
@@ -151,7 +149,7 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
 
 	sel.pad = RWPF_PAD_SINK;
 	sel.target = V4L2_SEL_TGT_CROP;
-	sel.r = *crop;
+	sel.r = input->crop;
 
 	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_selection, NULL,
 			       &sel);
@@ -826,12 +824,14 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int pipe_index,
 {
 	struct vsp1_device *vsp1 = dev_get_drvdata(dev);
 	struct vsp1_drm_pipeline *drm_pipe = &vsp1->drm->pipe[pipe_index];
+	struct vsp1_drm_input *input;
 	struct vsp1_rwpf *rpf;
 	int ret;
 
 	if (rpf_index >= vsp1->info->rpf_count)
 		return -EINVAL;
 
+	input = &vsp1->drm->inputs[rpf_index];
 	rpf = vsp1->rpf[rpf_index];
 
 	if (!cfg) {
@@ -873,9 +873,9 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int pipe_index,
 
 	rpf->format.flags = cfg->premult ? V4L2_PIX_FMT_FLAG_PREMUL_ALPHA : 0;
 
-	vsp1->drm->inputs[rpf_index].crop = cfg->src;
-	vsp1->drm->inputs[rpf_index].compose = cfg->dst;
-	vsp1->drm->inputs[rpf_index].zpos = cfg->zpos;
+	input->crop = cfg->src;
+	input->compose = cfg->dst;
+	input->zpos = cfg->zpos;
 
 	drm_pipe->pipe.inputs[rpf_index] = rpf;
 
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drm.h b/drivers/media/platform/renesas/vsp1/vsp1_drm.h
index 3fd95b53f27e..7234737cc464 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_drm.h
+++ b/drivers/media/platform/renesas/vsp1/vsp1_drm.h
@@ -59,7 +59,7 @@ struct vsp1_drm {
 	struct vsp1_drm_pipeline pipe[VSP1_MAX_LIF];
 	struct mutex lock;
 
-	struct {
+	struct vsp1_drm_input {
 		struct v4l2_rect crop;
 		struct v4l2_rect compose;
 		unsigned int zpos;
-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 7/9] media: renesas: vsp1: Name nested structure in vsp1_drm

[Tomi Valkeinen]

Hi,

On 30/04/2025 02:29, Laurent Pinchart wrote:
> The vsp1_drm structure defines an anonymous nested structure to store
> per-input data. In preparation for extending that structure, give it a
> name and is it through the driver. This improves code readability.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>   drivers/media/platform/renesas/vsp1/vsp1_drm.c | 18 +++++++++---------
>   drivers/media/platform/renesas/vsp1/vsp1_drm.h |  2 +-
>   2 files changed, 10 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drm.c b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
> index f8a575f6188a..e5339fda5941 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_drm.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
> @@ -118,24 +118,22 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
>   				      struct vsp1_entity *uif,
>   				      unsigned int brx_input)
>   {
> +	const struct vsp1_drm_input *input = &vsp1->drm->inputs[rpf->entity.index];
>   	struct v4l2_subdev_selection sel = {
>   		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
>   	};
>   	struct v4l2_subdev_format format = {
>   		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
>   	};
> -	const struct v4l2_rect *crop;
>   	int ret;
>   
>   	/*
>   	 * Configure the format on the RPF sink pad and propagate it up to the
>   	 * BRx sink pad.
>   	 */
> -	crop = &vsp1->drm->inputs[rpf->entity.index].crop;
> -
>   	format.pad = RWPF_PAD_SINK;
> -	format.format.width = crop->width + crop->left;
> -	format.format.height = crop->height + crop->top;
> +	format.format.width = input->crop.width + input->crop.left;
> +	format.format.height = input->crop.height + input->crop.top;
>   	format.format.code = rpf->fmtinfo->mbus;
>   	format.format.field = V4L2_FIELD_NONE;
>   
> @@ -151,7 +149,7 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
>   
>   	sel.pad = RWPF_PAD_SINK;
>   	sel.target = V4L2_SEL_TGT_CROP;
> -	sel.r = *crop;
> +	sel.r = input->crop;
>   
>   	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_selection, NULL,
>   			       &sel);
> @@ -826,12 +824,14 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int pipe_index,
>   {
>   	struct vsp1_device *vsp1 = dev_get_drvdata(dev);
>   	struct vsp1_drm_pipeline *drm_pipe = &vsp1->drm->pipe[pipe_index];
> +	struct vsp1_drm_input *input;
>   	struct vsp1_rwpf *rpf;
>   	int ret;
>   
>   	if (rpf_index >= vsp1->info->rpf_count)
>   		return -EINVAL;
>   
> +	input = &vsp1->drm->inputs[rpf_index];
>   	rpf = vsp1->rpf[rpf_index];
>   
>   	if (!cfg) {
> @@ -873,9 +873,9 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int pipe_index,
>   
>   	rpf->format.flags = cfg->premult ? V4L2_PIX_FMT_FLAG_PREMUL_ALPHA : 0;
>   
> -	vsp1->drm->inputs[rpf_index].crop = cfg->src;
> -	vsp1->drm->inputs[rpf_index].compose = cfg->dst;
> -	vsp1->drm->inputs[rpf_index].zpos = cfg->zpos;
> +	input->crop = cfg->src;
> +	input->compose = cfg->dst;
> +	input->zpos = cfg->zpos;
>   
>   	drm_pipe->pipe.inputs[rpf_index] = rpf;
>   
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drm.h b/drivers/media/platform/renesas/vsp1/vsp1_drm.h
> index 3fd95b53f27e..7234737cc464 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_drm.h
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_drm.h
> @@ -59,7 +59,7 @@ struct vsp1_drm {
>   	struct vsp1_drm_pipeline pipe[VSP1_MAX_LIF];
>   	struct mutex lock;
>   
> -	struct {
> +	struct vsp1_drm_input {
>   		struct v4l2_rect crop;
>   		struct v4l2_rect compose;
>   		unsigned int zpos;

Reviewed-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>

  Tomi

[PATCH v2 8/9] media: renesas: vsp1: Expose color space through the DRM API

[Laurent Pinchart]

Now that the VSP1 driver supports color spaces, expose them through the
API used by the DU driver. This allows configuring the YCbCr encoding
and quantization used by each plane, ensuring correct color rendering.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/renesas/vsp1/vsp1_drm.c | 4 ++++
 drivers/media/platform/renesas/vsp1/vsp1_drm.h | 6 ++++--
 include/media/vsp1.h                           | 4 ++++
 3 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drm.c b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
index e5339fda5941..fe55e8747b05 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
@@ -136,6 +136,8 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
 	format.format.height = input->crop.height + input->crop.top;
 	format.format.code = rpf->fmtinfo->mbus;
 	format.format.field = V4L2_FIELD_NONE;
+	format.format.ycbcr_enc = input->ycbcr_enc;
+	format.format.quantization = input->quantization;
 
 	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_fmt, NULL,
 			       &format);
@@ -876,6 +878,8 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int pipe_index,
 	input->crop = cfg->src;
 	input->compose = cfg->dst;
 	input->zpos = cfg->zpos;
+	input->ycbcr_enc = cfg->color_encoding;
+	input->quantization = cfg->color_range;
 
 	drm_pipe->pipe.inputs[rpf_index] = rpf;
 
diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drm.h b/drivers/media/platform/renesas/vsp1/vsp1_drm.h
index 7234737cc464..07a5d0adbd08 100644
--- a/drivers/media/platform/renesas/vsp1/vsp1_drm.h
+++ b/drivers/media/platform/renesas/vsp1/vsp1_drm.h
@@ -52,8 +52,8 @@ struct vsp1_drm_pipeline {
  * struct vsp1_drm - State for the API exposed to the DRM driver
  * @pipe: the VSP1 DRM pipeline used for display
  * @lock: protects the BRU and BRS allocation
- * @inputs: source crop rectangle, destination compose rectangle and z-order
- *	position for every input (indexed by RPF index)
+ * @inputs: source crop rectangle, destination compose rectangle, z-order
+ *	position and colorspace for every input (indexed by RPF index)
  */
 struct vsp1_drm {
 	struct vsp1_drm_pipeline pipe[VSP1_MAX_LIF];
@@ -63,6 +63,8 @@ struct vsp1_drm {
 		struct v4l2_rect crop;
 		struct v4l2_rect compose;
 		unsigned int zpos;
+		enum v4l2_ycbcr_encoding ycbcr_enc;
+		enum v4l2_quantization quantization;
 	} inputs[VSP1_MAX_RPF];
 };
 
diff --git a/include/media/vsp1.h b/include/media/vsp1.h
index 48f4a5023d81..4ea6352fd63f 100644
--- a/include/media/vsp1.h
+++ b/include/media/vsp1.h
@@ -52,6 +52,8 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int pipe_index,
  * @alpha: alpha value (0: fully transparent, 255: fully opaque)
  * @zpos: Z position of the plane (from 0 to number of planes minus 1)
  * @premult: true for premultiplied alpha
+ * @color_encoding: color encoding (valid for YUV formats only)
+ * @color_range: color range (valid for YUV formats only)
  */
 struct vsp1_du_atomic_config {
 	u32 pixelformat;
@@ -62,6 +64,8 @@ struct vsp1_du_atomic_config {
 	unsigned int alpha;
 	unsigned int zpos;
 	bool premult;
+	enum v4l2_ycbcr_encoding color_encoding;
+	enum v4l2_quantization color_range;
 };
 
 /**
-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 8/9] media: renesas: vsp1: Expose color space through the DRM API

[Tomi Valkeinen]

Hi,

On 30/04/2025 02:29, Laurent Pinchart wrote:
> Now that the VSP1 driver supports color spaces, expose them through the
> API used by the DU driver. This allows configuring the YCbCr encoding
> and quantization used by each plane, ensuring correct color rendering.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>   drivers/media/platform/renesas/vsp1/vsp1_drm.c | 4 ++++
>   drivers/media/platform/renesas/vsp1/vsp1_drm.h | 6 ++++--
>   include/media/vsp1.h                           | 4 ++++
>   3 files changed, 12 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drm.c b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
> index e5339fda5941..fe55e8747b05 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_drm.c
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_drm.c
> @@ -136,6 +136,8 @@ static int vsp1_du_pipeline_setup_rpf(struct vsp1_device *vsp1,
>   	format.format.height = input->crop.height + input->crop.top;
>   	format.format.code = rpf->fmtinfo->mbus;
>   	format.format.field = V4L2_FIELD_NONE;
> +	format.format.ycbcr_enc = input->ycbcr_enc;
> +	format.format.quantization = input->quantization;
>   
>   	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_fmt, NULL,
>   			       &format);
> @@ -876,6 +878,8 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int pipe_index,
>   	input->crop = cfg->src;
>   	input->compose = cfg->dst;
>   	input->zpos = cfg->zpos;
> +	input->ycbcr_enc = cfg->color_encoding;
> +	input->quantization = cfg->color_range;
>   
>   	drm_pipe->pipe.inputs[rpf_index] = rpf;
>   
> diff --git a/drivers/media/platform/renesas/vsp1/vsp1_drm.h b/drivers/media/platform/renesas/vsp1/vsp1_drm.h
> index 7234737cc464..07a5d0adbd08 100644
> --- a/drivers/media/platform/renesas/vsp1/vsp1_drm.h
> +++ b/drivers/media/platform/renesas/vsp1/vsp1_drm.h
> @@ -52,8 +52,8 @@ struct vsp1_drm_pipeline {
>    * struct vsp1_drm - State for the API exposed to the DRM driver
>    * @pipe: the VSP1 DRM pipeline used for display
>    * @lock: protects the BRU and BRS allocation
> - * @inputs: source crop rectangle, destination compose rectangle and z-order
> - *	position for every input (indexed by RPF index)
> + * @inputs: source crop rectangle, destination compose rectangle, z-order
> + *	position and colorspace for every input (indexed by RPF index)
>    */
>   struct vsp1_drm {
>   	struct vsp1_drm_pipeline pipe[VSP1_MAX_LIF];
> @@ -63,6 +63,8 @@ struct vsp1_drm {
>   		struct v4l2_rect crop;
>   		struct v4l2_rect compose;
>   		unsigned int zpos;
> +		enum v4l2_ycbcr_encoding ycbcr_enc;
> +		enum v4l2_quantization quantization;
>   	} inputs[VSP1_MAX_RPF];
>   };
>   
> diff --git a/include/media/vsp1.h b/include/media/vsp1.h
> index 48f4a5023d81..4ea6352fd63f 100644
> --- a/include/media/vsp1.h
> +++ b/include/media/vsp1.h
> @@ -52,6 +52,8 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int pipe_index,
>    * @alpha: alpha value (0: fully transparent, 255: fully opaque)
>    * @zpos: Z position of the plane (from 0 to number of planes minus 1)
>    * @premult: true for premultiplied alpha
> + * @color_encoding: color encoding (valid for YUV formats only)
> + * @color_range: color range (valid for YUV formats only)
>    */
>   struct vsp1_du_atomic_config {
>   	u32 pixelformat;
> @@ -62,6 +64,8 @@ struct vsp1_du_atomic_config {
>   	unsigned int alpha;
>   	unsigned int zpos;
>   	bool premult;
> +	enum v4l2_ycbcr_encoding color_encoding;
> +	enum v4l2_quantization color_range;
>   };
>   
>   /**

Reviewed-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>

  Tomi

[PATCH v2 9/9] drm: rcar-du: Create plane color properties

[Laurent Pinchart]

Now that the VSP1 driver allows setting per-plance color encoding and
color range for its DRM pipeline, create the corresponding DRM
properties in the DU driver and wire them to the VSP. This completes
support for plane color space.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c b/drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c
index 7aa0373563a4..09f9ab2111a2 100644
--- a/drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c
+++ b/drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c
@@ -10,6 +10,7 @@
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_blend.h>
+#include <drm/drm_color_mgmt.h>
 #include <drm/drm_crtc.h>
 #include <drm/drm_fb_dma_helper.h>
 #include <drm/drm_fourcc.h>
@@ -251,6 +252,12 @@ static void rcar_du_vsp_plane_setup(struct rcar_du_vsp_plane *plane)
 
 	cfg.premult = state->state.pixel_blend_mode == DRM_MODE_BLEND_PREMULTI;
 
+	cfg.color_encoding = state->state.color_encoding == DRM_COLOR_YCBCR_BT601
+			   ? V4L2_YCBCR_ENC_601 : V4L2_YCBCR_ENC_709;
+	cfg.color_range = state->state.color_range == DRM_COLOR_YCBCR_LIMITED_RANGE
+			? V4L2_QUANTIZATION_LIM_RANGE
+			: V4L2_QUANTIZATION_FULL_RANGE;
+
 	vsp1_du_atomic_update(plane->vsp->vsp, crtc->vsp_pipe,
 			      plane->index, &cfg);
 }
@@ -530,6 +537,14 @@ int rcar_du_vsp_init(struct rcar_du_vsp *vsp, struct device_node *np,
 					BIT(DRM_MODE_BLEND_PREMULTI) |
 					BIT(DRM_MODE_BLEND_COVERAGE));
 
+		drm_plane_create_color_properties(&plane->plane,
+						  BIT(DRM_COLOR_YCBCR_BT601) |
+						  BIT(DRM_COLOR_YCBCR_BT709),
+						  BIT(DRM_COLOR_YCBCR_LIMITED_RANGE) |
+						  BIT(DRM_COLOR_YCBCR_FULL_RANGE),
+						  DRM_COLOR_YCBCR_BT601,
+						  DRM_COLOR_YCBCR_LIMITED_RANGE);
+
 		vsp->num_planes++;
 	}
 
-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 9/9] drm: rcar-du: Create plane color properties

[Tomi Valkeinen]

Hi,

On 30/04/2025 02:29, Laurent Pinchart wrote:
> Now that the VSP1 driver allows setting per-plance color encoding and
> color range for its DRM pipeline, create the corresponding DRM
> properties in the DU driver and wire them to the VSP. This completes
> support for plane color space.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>   drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c | 15 +++++++++++++++
>   1 file changed, 15 insertions(+)
> 
> diff --git a/drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c b/drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c
> index 7aa0373563a4..09f9ab2111a2 100644
> --- a/drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c
> +++ b/drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c
> @@ -10,6 +10,7 @@
>   #include <drm/drm_atomic.h>
>   #include <drm/drm_atomic_helper.h>
>   #include <drm/drm_blend.h>
> +#include <drm/drm_color_mgmt.h>
>   #include <drm/drm_crtc.h>
>   #include <drm/drm_fb_dma_helper.h>
>   #include <drm/drm_fourcc.h>
> @@ -251,6 +252,12 @@ static void rcar_du_vsp_plane_setup(struct rcar_du_vsp_plane *plane)
>   
>   	cfg.premult = state->state.pixel_blend_mode == DRM_MODE_BLEND_PREMULTI;
>   
> +	cfg.color_encoding = state->state.color_encoding == DRM_COLOR_YCBCR_BT601
> +			   ? V4L2_YCBCR_ENC_601 : V4L2_YCBCR_ENC_709;
> +	cfg.color_range = state->state.color_range == DRM_COLOR_YCBCR_LIMITED_RANGE
> +			? V4L2_QUANTIZATION_LIM_RANGE
> +			: V4L2_QUANTIZATION_FULL_RANGE;
> +
>   	vsp1_du_atomic_update(plane->vsp->vsp, crtc->vsp_pipe,
>   			      plane->index, &cfg);
>   }
> @@ -530,6 +537,14 @@ int rcar_du_vsp_init(struct rcar_du_vsp *vsp, struct device_node *np,
>   					BIT(DRM_MODE_BLEND_PREMULTI) |
>   					BIT(DRM_MODE_BLEND_COVERAGE));
>   
> +		drm_plane_create_color_properties(&plane->plane,
> +						  BIT(DRM_COLOR_YCBCR_BT601) |
> +						  BIT(DRM_COLOR_YCBCR_BT709),
> +						  BIT(DRM_COLOR_YCBCR_LIMITED_RANGE) |
> +						  BIT(DRM_COLOR_YCBCR_FULL_RANGE),
> +						  DRM_COLOR_YCBCR_BT601,
> +						  DRM_COLOR_YCBCR_LIMITED_RANGE);
> +
>   		vsp->num_planes++;
>   	}
>   

Reviewed-by: Tomi Valkeinen <tomi.valkeinen+renesas@ideasonboard.com>

  Tomi

Re: [PATCH v2 0/9] media: renesas: vsp1: Add colorspace support

[Tommaso Merciai]

Hi Laurent,
Thanks for this series.

On Wed, Apr 30, 2025 at 02:28:55AM +0300, Laurent Pinchart wrote:
> Hello,
> 
> This patch series extends the VSP1 driver with colorspace support. It
> turns out that the VSP RPF and WPF entities can convert between RGB and
> YUV, a feature that we have failed to test so far. The hardware support
> BT.601 and BT.709, in both limited and full range. Proper configuration
> of colorspace is crucial for accurate image rendering.
> 
> Patch 1/9 starts by implementing pixel format enumeration in the driver,
> a feature that was surprisingly missing. Patch 2/9 then continues with
> another fix, restricting the creation of the HSI and HST entities to VSP
> instances that include them. Following with another fix, patch 4/9
> addresses format setting on the RPF and WPF source pad to disable HSV
> <-> { RGB, YUV } conversion, a feature *not* supported by the hardware.
> 
> After those initial fixes, patch 5/9 starts implementing colorspace
> support by reporting the related information to userspace. The driver
> currently hardcodes limited range BT.601 when programming the hardware,
> so that is the value that the patch reports to userspace for YUV
> formats. Patch 6/9 makes the YCbCr encoding and quantization
> configurable, finalizing colorspace support in the VSP driver.
> 
> The next three patches are new in this version. They proceed with
> exposing colorspace configurability in the API exposed to the DU DRM
> driver in patch 8/9 (with 7/9 performing a small refactoring first).
> Patch 9/9 then updates the DU driver accordingly, to create plane
> colorspace properties and wire them up to the VSP.
> 
> The series has been tested with the vsp-tests suite and the kms-tests
> suite. Patches that add CSC support to the vsp-tests suite have been
> posted in [1], and can be found in the vsp-tests git tree in [2].

I've tested this series with out of Tree patches with RZ/G3E.
Basically I've added the equivalent of:

 - drm: rcar-du: Create plane color properties

On drivers/gpu/drm/renesas/rz-du/rzg2l_du_vsp.c

Tested using modetest:

modetest -M rzg2l-du -s 48@46:800x600-60.32@YU16 -d -P 39@46:512x300+200+200@YU16

modetest -M rzg2l-du -w 32:COLOR_ENCODING:0
modetest -M rzg2l-du -w 32:COLOR_ENCODING:1

modetest -M rzg2l-du -w 32:COLOR_RANGE:0
modetest -M rzg2l-du -w 32:COLOR_RANGE:1

Where 32 is the id of the primary plane.

Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>

Thanks & Regards,
Tommaso

> Updates to media-ctl and yavta to support setting colorspace on video
> capture devices and on subdev source pads have been merged in the
> respective projects, make sure to use the latest master branch if you
> want to run the tests.
> 
> The series has also been tested with the latest v4l2-compliance. The bad
> news is that the test flags 56 errors when run on a Renesas Salvator-X
> 2nd version board based on r8a77965:
> 
> Grand Total for vsp1 device /dev/media1: 757, Succeeded: 701, Failed: 56, Warnings: 0
> 
> The good news is that none of those are regressions, quite the contrary:
> without this series applied, the total number of failures is 95, and the
> diff shows no new error. I will therefore address those issues
> separately.
> 
> I would like to get patch 1/9 to 8/9 merged in v6.16, to then merge 9/9
> in v6.17.
> 
> [1] https://lore.kernel.org/linux-renesas-soc/20250409004758.11014-1-laurent.pinchart@ideasonboard.com
> [2] https://git.ideasonboard.com/renesas/vsp-tests.git/log/?h=csc
> 
> Laurent Pinchart (9):
>   media: renesas: vsp1: Implement pixel format enumeration
>   media: renesas: vsp1: Make HSI and HST modules optional
>   media: renesas: vsp1: Fix HSV format enumeration
>   media: renesas: vsp1: Fix media bus code setup on RWPF source pad
>   media: renesas: vsp1: Report colour space information to userspace
>   media: renesas: vsp1: Allow setting encoding and quantization
>   media: renesas: vsp1: Name nested structure in vsp1_drm
>   media: renesas: vsp1: Expose color space through the DRM API
>   drm: rcar-du: Create plane color properties
> 
>  drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c |  15 ++
>  drivers/media/platform/renesas/vsp1/vsp1.h    |   1 +
>  .../media/platform/renesas/vsp1/vsp1_brx.c    |   9 +-
>  .../media/platform/renesas/vsp1/vsp1_drm.c    |  22 +-
>  .../media/platform/renesas/vsp1/vsp1_drm.h    |   8 +-
>  .../media/platform/renesas/vsp1/vsp1_drv.c    |  59 +++---
>  .../media/platform/renesas/vsp1/vsp1_entity.c |  22 +-
>  .../media/platform/renesas/vsp1/vsp1_entity.h |   2 +
>  .../media/platform/renesas/vsp1/vsp1_hsit.c   |  11 +-
>  .../media/platform/renesas/vsp1/vsp1_pipe.c   | 188 ++++++++++++++++--
>  .../media/platform/renesas/vsp1/vsp1_pipe.h   |   5 +
>  .../media/platform/renesas/vsp1/vsp1_rpf.c    |  29 ++-
>  .../media/platform/renesas/vsp1/vsp1_rwpf.c   |  51 ++++-
>  .../media/platform/renesas/vsp1/vsp1_sru.c    |   9 +-
>  .../media/platform/renesas/vsp1/vsp1_uds.c    |   9 +-
>  .../media/platform/renesas/vsp1/vsp1_video.c  |  50 ++++-
>  .../media/platform/renesas/vsp1/vsp1_wpf.c    |  29 ++-
>  include/media/vsp1.h                          |   4 +
>  18 files changed, 442 insertions(+), 81 deletions(-)
> 
> 
> base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
> prerequisite-patch-id: e3d9847d1da4398f1bf0b57f5ef4a612392c7255
> -- 
> Regards,
> 
> Laurent Pinchart
> 

Re: [PATCH v2 0/9] media: renesas: vsp1: Add colorspace support

[Laurent Pinchart]

Hi Tommaso,

On Fri, May 02, 2025 at 10:21:21AM +0200, Tommaso Merciai wrote:
> On Wed, Apr 30, 2025 at 02:28:55AM +0300, Laurent Pinchart wrote:
> > Hello,
> > 
> > This patch series extends the VSP1 driver with colorspace support. It
> > turns out that the VSP RPF and WPF entities can convert between RGB and
> > YUV, a feature that we have failed to test so far. The hardware support
> > BT.601 and BT.709, in both limited and full range. Proper configuration
> > of colorspace is crucial for accurate image rendering.
> > 
> > Patch 1/9 starts by implementing pixel format enumeration in the driver,
> > a feature that was surprisingly missing. Patch 2/9 then continues with
> > another fix, restricting the creation of the HSI and HST entities to VSP
> > instances that include them. Following with another fix, patch 4/9
> > addresses format setting on the RPF and WPF source pad to disable HSV
> > <-> { RGB, YUV } conversion, a feature *not* supported by the hardware.
> > 
> > After those initial fixes, patch 5/9 starts implementing colorspace
> > support by reporting the related information to userspace. The driver
> > currently hardcodes limited range BT.601 when programming the hardware,
> > so that is the value that the patch reports to userspace for YUV
> > formats. Patch 6/9 makes the YCbCr encoding and quantization
> > configurable, finalizing colorspace support in the VSP driver.
> > 
> > The next three patches are new in this version. They proceed with
> > exposing colorspace configurability in the API exposed to the DU DRM
> > driver in patch 8/9 (with 7/9 performing a small refactoring first).
> > Patch 9/9 then updates the DU driver accordingly, to create plane
> > colorspace properties and wire them up to the VSP.
> > 
> > The series has been tested with the vsp-tests suite and the kms-tests
> > suite. Patches that add CSC support to the vsp-tests suite have been
> > posted in [1], and can be found in the vsp-tests git tree in [2].
> 
> I've tested this series with out of Tree patches with RZ/G3E.
> Basically I've added the equivalent of:
> 
>  - drm: rcar-du: Create plane color properties
> 
> On drivers/gpu/drm/renesas/rz-du/rzg2l_du_vsp.c
> 
> Tested using modetest:
> 
> modetest -M rzg2l-du -s 48@46:800x600-60.32@YU16 -d -P 39@46:512x300+200+200@YU16
> 
> modetest -M rzg2l-du -w 32:COLOR_ENCODING:0
> modetest -M rzg2l-du -w 32:COLOR_ENCODING:1
> 
> modetest -M rzg2l-du -w 32:COLOR_RANGE:0
> modetest -M rzg2l-du -w 32:COLOR_RANGE:1
> 
> Where 32 is the id of the primary plane.
> 
> Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>

Thank you for confirming this works on RZ. Unfortunately your Tested-by
tag came after I submitted the pull request, and the patches got merged
without it. It however gives me confidence that the code is working as
intended.

> > Updates to media-ctl and yavta to support setting colorspace on video
> > capture devices and on subdev source pads have been merged in the
> > respective projects, make sure to use the latest master branch if you
> > want to run the tests.
> > 
> > The series has also been tested with the latest v4l2-compliance. The bad
> > news is that the test flags 56 errors when run on a Renesas Salvator-X
> > 2nd version board based on r8a77965:
> > 
> > Grand Total for vsp1 device /dev/media1: 757, Succeeded: 701, Failed: 56, Warnings: 0
> > 
> > The good news is that none of those are regressions, quite the contrary:
> > without this series applied, the total number of failures is 95, and the
> > diff shows no new error. I will therefore address those issues
> > separately.
> > 
> > I would like to get patch 1/9 to 8/9 merged in v6.16, to then merge 9/9
> > in v6.17.
> > 
> > [1] https://lore.kernel.org/linux-renesas-soc/20250409004758.11014-1-laurent.pinchart@ideasonboard.com
> > [2] https://git.ideasonboard.com/renesas/vsp-tests.git/log/?h=csc
> > 
> > Laurent Pinchart (9):
> >   media: renesas: vsp1: Implement pixel format enumeration
> >   media: renesas: vsp1: Make HSI and HST modules optional
> >   media: renesas: vsp1: Fix HSV format enumeration
> >   media: renesas: vsp1: Fix media bus code setup on RWPF source pad
> >   media: renesas: vsp1: Report colour space information to userspace
> >   media: renesas: vsp1: Allow setting encoding and quantization
> >   media: renesas: vsp1: Name nested structure in vsp1_drm
> >   media: renesas: vsp1: Expose color space through the DRM API
> >   drm: rcar-du: Create plane color properties
> > 
> >  drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c |  15 ++
> >  drivers/media/platform/renesas/vsp1/vsp1.h    |   1 +
> >  .../media/platform/renesas/vsp1/vsp1_brx.c    |   9 +-
> >  .../media/platform/renesas/vsp1/vsp1_drm.c    |  22 +-
> >  .../media/platform/renesas/vsp1/vsp1_drm.h    |   8 +-
> >  .../media/platform/renesas/vsp1/vsp1_drv.c    |  59 +++---
> >  .../media/platform/renesas/vsp1/vsp1_entity.c |  22 +-
> >  .../media/platform/renesas/vsp1/vsp1_entity.h |   2 +
> >  .../media/platform/renesas/vsp1/vsp1_hsit.c   |  11 +-
> >  .../media/platform/renesas/vsp1/vsp1_pipe.c   | 188 ++++++++++++++++--
> >  .../media/platform/renesas/vsp1/vsp1_pipe.h   |   5 +
> >  .../media/platform/renesas/vsp1/vsp1_rpf.c    |  29 ++-
> >  .../media/platform/renesas/vsp1/vsp1_rwpf.c   |  51 ++++-
> >  .../media/platform/renesas/vsp1/vsp1_sru.c    |   9 +-
> >  .../media/platform/renesas/vsp1/vsp1_uds.c    |   9 +-
> >  .../media/platform/renesas/vsp1/vsp1_video.c  |  50 ++++-
> >  .../media/platform/renesas/vsp1/vsp1_wpf.c    |  29 ++-
> >  include/media/vsp1.h                          |   4 +
> >  18 files changed, 442 insertions(+), 81 deletions(-)
> > 
> > 
> > base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
> > prerequisite-patch-id: e3d9847d1da4398f1bf0b57f5ef4a612392c7255

-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 0/9] media: renesas: vsp1: Add colorspace support

[Tommaso Merciai]

Hi Laurent,

On Fri, May 02, 2025 at 07:16:57PM +0300, Laurent Pinchart wrote:
> Hi Tommaso,
> 
> On Fri, May 02, 2025 at 10:21:21AM +0200, Tommaso Merciai wrote:
> > On Wed, Apr 30, 2025 at 02:28:55AM +0300, Laurent Pinchart wrote:
> > > Hello,
> > > 
> > > This patch series extends the VSP1 driver with colorspace support. It
> > > turns out that the VSP RPF and WPF entities can convert between RGB and
> > > YUV, a feature that we have failed to test so far. The hardware support
> > > BT.601 and BT.709, in both limited and full range. Proper configuration
> > > of colorspace is crucial for accurate image rendering.
> > > 
> > > Patch 1/9 starts by implementing pixel format enumeration in the driver,
> > > a feature that was surprisingly missing. Patch 2/9 then continues with
> > > another fix, restricting the creation of the HSI and HST entities to VSP
> > > instances that include them. Following with another fix, patch 4/9
> > > addresses format setting on the RPF and WPF source pad to disable HSV
> > > <-> { RGB, YUV } conversion, a feature *not* supported by the hardware.
> > > 
> > > After those initial fixes, patch 5/9 starts implementing colorspace
> > > support by reporting the related information to userspace. The driver
> > > currently hardcodes limited range BT.601 when programming the hardware,
> > > so that is the value that the patch reports to userspace for YUV
> > > formats. Patch 6/9 makes the YCbCr encoding and quantization
> > > configurable, finalizing colorspace support in the VSP driver.
> > > 
> > > The next three patches are new in this version. They proceed with
> > > exposing colorspace configurability in the API exposed to the DU DRM
> > > driver in patch 8/9 (with 7/9 performing a small refactoring first).
> > > Patch 9/9 then updates the DU driver accordingly, to create plane
> > > colorspace properties and wire them up to the VSP.
> > > 
> > > The series has been tested with the vsp-tests suite and the kms-tests
> > > suite. Patches that add CSC support to the vsp-tests suite have been
> > > posted in [1], and can be found in the vsp-tests git tree in [2].
> > 
> > I've tested this series with out of Tree patches with RZ/G3E.
> > Basically I've added the equivalent of:
> > 
> >  - drm: rcar-du: Create plane color properties
> > 
> > On drivers/gpu/drm/renesas/rz-du/rzg2l_du_vsp.c
> > 
> > Tested using modetest:
> > 
> > modetest -M rzg2l-du -s 48@46:800x600-60.32@YU16 -d -P 39@46:512x300+200+200@YU16
> > 
> > modetest -M rzg2l-du -w 32:COLOR_ENCODING:0
> > modetest -M rzg2l-du -w 32:COLOR_ENCODING:1
> > 
> > modetest -M rzg2l-du -w 32:COLOR_RANGE:0
> > modetest -M rzg2l-du -w 32:COLOR_RANGE:1
> > 
> > Where 32 is the id of the primary plane.
> > 
> > Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
> 
> Thank you for confirming this works on RZ. Unfortunately your Tested-by
> tag came after I submitted the pull request, and the patches got merged
> without it. It however gives me confidence that the code is working as
> intended.

Ouch, sorry for being late.
No worries, hope this help. :)

Thanks & Regards,
Tommaso

> 
> > > Updates to media-ctl and yavta to support setting colorspace on video
> > > capture devices and on subdev source pads have been merged in the
> > > respective projects, make sure to use the latest master branch if you
> > > want to run the tests.
> > > 
> > > The series has also been tested with the latest v4l2-compliance. The bad
> > > news is that the test flags 56 errors when run on a Renesas Salvator-X
> > > 2nd version board based on r8a77965:
> > > 
> > > Grand Total for vsp1 device /dev/media1: 757, Succeeded: 701, Failed: 56, Warnings: 0
> > > 
> > > The good news is that none of those are regressions, quite the contrary:
> > > without this series applied, the total number of failures is 95, and the
> > > diff shows no new error. I will therefore address those issues
> > > separately.
> > > 
> > > I would like to get patch 1/9 to 8/9 merged in v6.16, to then merge 9/9
> > > in v6.17.
> > > 
> > > [1] https://lore.kernel.org/linux-renesas-soc/20250409004758.11014-1-laurent.pinchart@ideasonboard.com
> > > [2] https://git.ideasonboard.com/renesas/vsp-tests.git/log/?h=csc
> > > 
> > > Laurent Pinchart (9):
> > >   media: renesas: vsp1: Implement pixel format enumeration
> > >   media: renesas: vsp1: Make HSI and HST modules optional
> > >   media: renesas: vsp1: Fix HSV format enumeration
> > >   media: renesas: vsp1: Fix media bus code setup on RWPF source pad
> > >   media: renesas: vsp1: Report colour space information to userspace
> > >   media: renesas: vsp1: Allow setting encoding and quantization
> > >   media: renesas: vsp1: Name nested structure in vsp1_drm
> > >   media: renesas: vsp1: Expose color space through the DRM API
> > >   drm: rcar-du: Create plane color properties
> > > 
> > >  drivers/gpu/drm/renesas/rcar-du/rcar_du_vsp.c |  15 ++
> > >  drivers/media/platform/renesas/vsp1/vsp1.h    |   1 +
> > >  .../media/platform/renesas/vsp1/vsp1_brx.c    |   9 +-
> > >  .../media/platform/renesas/vsp1/vsp1_drm.c    |  22 +-
> > >  .../media/platform/renesas/vsp1/vsp1_drm.h    |   8 +-
> > >  .../media/platform/renesas/vsp1/vsp1_drv.c    |  59 +++---
> > >  .../media/platform/renesas/vsp1/vsp1_entity.c |  22 +-
> > >  .../media/platform/renesas/vsp1/vsp1_entity.h |   2 +
> > >  .../media/platform/renesas/vsp1/vsp1_hsit.c   |  11 +-
> > >  .../media/platform/renesas/vsp1/vsp1_pipe.c   | 188 ++++++++++++++++--
> > >  .../media/platform/renesas/vsp1/vsp1_pipe.h   |   5 +
> > >  .../media/platform/renesas/vsp1/vsp1_rpf.c    |  29 ++-
> > >  .../media/platform/renesas/vsp1/vsp1_rwpf.c   |  51 ++++-
> > >  .../media/platform/renesas/vsp1/vsp1_sru.c    |   9 +-
> > >  .../media/platform/renesas/vsp1/vsp1_uds.c    |   9 +-
> > >  .../media/platform/renesas/vsp1/vsp1_video.c  |  50 ++++-
> > >  .../media/platform/renesas/vsp1/vsp1_wpf.c    |  29 ++-
> > >  include/media/vsp1.h                          |   4 +
> > >  18 files changed, 442 insertions(+), 81 deletions(-)
> > > 
> > > 
> > > base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa
> > > prerequisite-patch-id: e3d9847d1da4398f1bf0b57f5ef4a612392c7255
> 
> -- 
> Regards,
> 
> Laurent Pinchart
>