[PATCH v6 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

[PATCH v6 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

[Jonas Karlman]

This series add H.264 High 10 and 4:2:2 profile support to the Rockchip
Video Decoder driver.

Patch 1 add helpers for calculating plane bytesperline and sizeimage.
Patch 2 add two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.

Patch 3 change to use bytesperline and buffer height to configure strides.
Patch 4 change to use values from SPS/PPS control to configure the HW.

Patch 5-9 refactor code to support filtering of CAPUTRE formats based
on the image format returned from a get_image_fmt ops.

Patch 10 add final bits to support H.264 High 10 and 4:2:2 profiles.

Patch 11 add a fix for enumerated frame sizes returned to userspace.

Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):

  v4l2-compliance 1.28.1, 64 bits, 64-bit time_t
  ...
  Total for rkvdec device /dev/video1: 48, Succeeded: 48, Failed: 0, Warnings: 0

  Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
  ...
  Ran 65/69 tests successfully

  Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-v4l2request
  ...
  Ran 129/135 tests successfully

Before this series:

  Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
  ...
  Ran 44/69 tests successfully

Changes in v6:
- Change to use fmt_idx instead of j++ tucked inside a condition (Dan)
- Add patch to fix enumerated frame sizes returned to userspace (Alex)
- Fluster test score is same as v4 and v5, see [4] and [5]
Link to v5: https://lore.kernel.org/linux-media/20240618194647.742037-1-jonas@kwiboo.se/

Changes in v5:
- Drop Remove SPS validation at streaming start patch
- Move buffer align from rkvdec_fill_decoded_pixfmt to min/step_width
- Use correct profiles for V4L2_CID_MPEG_VIDEO_H264_PROFILE
- Collect r-b and t-b tags
- Fluster test score is same as v4, see [4] and [5]
Link to v4: https://lore.kernel.org/linux-media/20231105165521.3592037-1-jonas@kwiboo.se/

Changes in v4:
- Fix failed v4l2-compliance tests related to CAPTURE queue
- Rework CAPTURE format filter anv validate to use an image format
- Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/

Changes in v3:
- Drop merged patches
- Use bpp and bpp_div instead of prior misuse of block_w/block_h
- New patch to use values from SPS/PPS control to configure the HW
- New patch to remove an unnecessary call to validate sps at streaming start
- Reworked pixel format validation
Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/

Changes in v2:
- Collect r-b tags
- SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
- New patch to not override output buffer sizeimage
- Reworked pixel format validation
- Only align decoded buffer instead of changing frmsize step_width
Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/

To fully runtime test this series you may need FFmpeg patches from [1]
and fluster patches from [2], this series is also available at [3].

[1] https://github.com/Kwiboo/FFmpeg/commits/v4l2request-2024-v2-rkvdec/
[2] https://github.com/Kwiboo/fluster/commits/ffmpeg-v4l2request-rkvdec/
[3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v6/
[4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
[5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf

Regards,
Jonas

Alex Bee (1):
  media: rkvdec: h264: Don't hardcode SPS/PPS parameters

Jonas Karlman (10):
  media: v4l2-common: Add helpers to calculate bytesperline and
    sizeimage
  media: v4l2: Add NV15 and NV20 pixel formats
  media: rkvdec: h264: Use bytesperline and buffer height as virstride
  media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
  media: rkvdec: Move rkvdec_reset_decoded_fmt helper
  media: rkvdec: Extract decoded format enumeration into helper
  media: rkvdec: Add image format concept
  media: rkvdec: Add get_image_fmt ops
  media: rkvdec: h264: Support High 10 and 4:2:2 profiles
  media: rkvdec: Fix enumerate frame sizes

 .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++
 drivers/media/v4l2-core/v4l2-common.c         |  80 +++---
 drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
 drivers/staging/media/rkvdec/rkvdec-h264.c    |  64 +++--
 drivers/staging/media/rkvdec/rkvdec.c         | 239 +++++++++++++-----
 drivers/staging/media/rkvdec/rkvdec.h         |  18 +-
 include/uapi/linux/videodev2.h                |   2 +
 7 files changed, 410 insertions(+), 123 deletions(-)

-- 
2.46.0

[PATCH v6 01/11] media: v4l2-common: Add helpers to calculate bytesperline and sizeimage

[Jonas Karlman]

Add helper functions to calculate plane bytesperline and sizeimage,
these new helpers consider bpp div, block width and height when
calculating plane bytesperline and sizeimage.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Christopher Obbard <chris.obbard@collabora.com>
---
v6:
- No change

v5:
- Collect t-b tags

v4:
- No change

v3:
- Consider bpp_div in calculation
---
 drivers/media/v4l2-core/v4l2-common.c | 78 +++++++++++++--------------
 1 file changed, 39 insertions(+), 39 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 0a2f4f0d0a07..8ad04d48a023 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -357,6 +357,34 @@ static inline unsigned int v4l2_format_block_height(const struct v4l2_format_inf
 	return info->block_h[plane];
 }
 
+static inline unsigned int v4l2_format_plane_width(const struct v4l2_format_info *info, int plane,
+						   unsigned int width)
+{
+	unsigned int hdiv = plane ? info->hdiv : 1;
+	unsigned int aligned_width =
+		ALIGN(width, v4l2_format_block_width(info, plane));
+
+	return DIV_ROUND_UP(aligned_width, hdiv) *
+	       info->bpp[plane] / info->bpp_div[plane];
+}
+
+static inline unsigned int v4l2_format_plane_height(const struct v4l2_format_info *info, int plane,
+						    unsigned int height)
+{
+	unsigned int vdiv = plane ? info->vdiv : 1;
+	unsigned int aligned_height =
+		ALIGN(height, v4l2_format_block_height(info, plane));
+
+	return DIV_ROUND_UP(aligned_height, vdiv);
+}
+
+static inline unsigned int v4l2_format_plane_size(const struct v4l2_format_info *info, int plane,
+						  unsigned int width, unsigned int height)
+{
+	return v4l2_format_plane_width(info, plane, width) *
+	       v4l2_format_plane_height(info, plane, height);
+}
+
 void v4l2_apply_frmsize_constraints(u32 *width, u32 *height,
 				    const struct v4l2_frmsize_stepwise *frmsize)
 {
@@ -392,37 +420,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
 
 	if (info->mem_planes == 1) {
 		plane = &pixfmt->plane_fmt[0];
-		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0] / info->bpp_div[0];
+		plane->bytesperline = v4l2_format_plane_width(info, 0, width);
 		plane->sizeimage = 0;
 
-		for (i = 0; i < info->comp_planes; i++) {
-			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
-			unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
-			unsigned int aligned_width;
-			unsigned int aligned_height;
-
-			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
-			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
-
-			plane->sizeimage += info->bpp[i] *
-				DIV_ROUND_UP(aligned_width, hdiv) *
-				DIV_ROUND_UP(aligned_height, vdiv) / info->bpp_div[i];
-		}
+		for (i = 0; i < info->comp_planes; i++)
+			plane->sizeimage +=
+				v4l2_format_plane_size(info, i, width, height);
 	} else {
 		for (i = 0; i < info->comp_planes; i++) {
-			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
-			unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
-			unsigned int aligned_width;
-			unsigned int aligned_height;
-
-			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
-			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
-
 			plane = &pixfmt->plane_fmt[i];
 			plane->bytesperline =
-				info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv) / info->bpp_div[i];
-			plane->sizeimage =
-				plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv);
+				v4l2_format_plane_width(info, i, width);
+			plane->sizeimage = plane->bytesperline *
+				v4l2_format_plane_height(info, i, height);
 		}
 	}
 	return 0;
@@ -446,22 +456,12 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
 	pixfmt->width = width;
 	pixfmt->height = height;
 	pixfmt->pixelformat = pixelformat;
-	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0] / info->bpp_div[0];
+	pixfmt->bytesperline = v4l2_format_plane_width(info, 0, width);
 	pixfmt->sizeimage = 0;
 
-	for (i = 0; i < info->comp_planes; i++) {
-		unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
-		unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
-		unsigned int aligned_width;
-		unsigned int aligned_height;
-
-		aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
-		aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
-
-		pixfmt->sizeimage += info->bpp[i] *
-			DIV_ROUND_UP(aligned_width, hdiv) *
-			DIV_ROUND_UP(aligned_height, vdiv) / info->bpp_div[i];
-	}
+	for (i = 0; i < info->comp_planes; i++)
+		pixfmt->sizeimage +=
+			v4l2_format_plane_size(info, i, width, height);
 	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);
-- 
2.46.0

[PATCH v6 02/11] media: v4l2: Add NV15 and NV20 pixel formats

[Jonas Karlman]

Add NV15 and NV20 pixel formats used by the Rockchip Video Decoder for
10-bit buffers.

NV15 and NV20 is 10-bit 4:2:0/4:2:2 semi-planar YUV formats similar to
NV12 and NV16, using 10-bit components with no padding between each
component. Instead, a group of 4 luminance/chrominance samples are
stored over 5 bytes in little endian order:

YYYY = UVUV = 4 * 10 bits = 40 bits = 5 bytes

The '15' and '20' suffix refers to the optimum effective bits per pixel
which is achieved when the total number of luminance samples is a
multiple of 8 for NV15 and 4 for NV20.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Christopher Obbard <chris.obbard@collabora.com>
---
v6:
- No change

v5:
- Collect t-b tags

v4:
- No change

v3:
- Use bpp and bpp_div instead of the misuse of block_w/block_h
- Update documentation, expand to use full 4x4 sample image
---
 .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++++++++++
 drivers/media/v4l2-core/v4l2-common.c         |   2 +
 drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
 include/uapi/linux/videodev2.h                |   2 +
 4 files changed, 134 insertions(+)

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
index b788f6933855..22cad8c9726b 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
@@ -79,6 +79,13 @@ All components are stored with the same number of bits per component.
       - Cr, Cb
       - Yes
       - Linear
+    * - V4L2_PIX_FMT_NV15
+      - 'NV15'
+      - 10
+      - 4:2:0
+      - Cb, Cr
+      - Yes
+      - Linear
     * - V4L2_PIX_FMT_NV12M
       - 'NM12'
       - 8
@@ -172,6 +179,13 @@ All components are stored with the same number of bits per component.
       - Cr, Cb
       - Yes
       - Linear
+    * - V4L2_PIX_FMT_NV20
+      - 'NV20'
+      - 10
+      - 4:2:2
+      - Cb, Cr
+      - Yes
+      - Linear
     * - V4L2_PIX_FMT_NV16M
       - 'NM16'
       - 8
@@ -302,6 +316,57 @@ of the luma plane.
       - Cr\ :sub:`11`
 
 
+.. _V4L2-PIX-FMT-NV15:
+
+NV15
+----
+
+Semi-planar 10-bit YUV 4:2:0 format similar to NV12, using 10-bit components
+with no padding between each component. A group of 4 components are stored over
+5 bytes in little endian order.
+
+.. flat-table:: Sample 4x4 NV15 Image (1 byte per cell)
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - Y'\ :sub:`00[7:0]`
+      - Y'\ :sub:`01[5:0]`\ Y'\ :sub:`00[9:8]`
+      - Y'\ :sub:`02[3:0]`\ Y'\ :sub:`01[9:6]`
+      - Y'\ :sub:`03[1:0]`\ Y'\ :sub:`02[9:4]`
+      - Y'\ :sub:`03[9:2]`
+    * - start + 5:
+      - Y'\ :sub:`10[7:0]`
+      - Y'\ :sub:`11[5:0]`\ Y'\ :sub:`10[9:8]`
+      - Y'\ :sub:`12[3:0]`\ Y'\ :sub:`11[9:6]`
+      - Y'\ :sub:`13[1:0]`\ Y'\ :sub:`12[9:4]`
+      - Y'\ :sub:`13[9:2]`
+    * - start + 10:
+      - Y'\ :sub:`20[7:0]`
+      - Y'\ :sub:`21[5:0]`\ Y'\ :sub:`20[9:8]`
+      - Y'\ :sub:`22[3:0]`\ Y'\ :sub:`21[9:6]`
+      - Y'\ :sub:`23[1:0]`\ Y'\ :sub:`22[9:4]`
+      - Y'\ :sub:`23[9:2]`
+    * - start + 15:
+      - Y'\ :sub:`30[7:0]`
+      - Y'\ :sub:`31[5:0]`\ Y'\ :sub:`30[9:8]`
+      - Y'\ :sub:`32[3:0]`\ Y'\ :sub:`31[9:6]`
+      - Y'\ :sub:`33[1:0]`\ Y'\ :sub:`32[9:4]`
+      - Y'\ :sub:`33[9:2]`
+    * - start + 20:
+      - Cb\ :sub:`00[7:0]`
+      - Cr\ :sub:`00[5:0]`\ Cb\ :sub:`00[9:8]`
+      - Cb\ :sub:`01[3:0]`\ Cr\ :sub:`00[9:6]`
+      - Cr\ :sub:`01[1:0]`\ Cb\ :sub:`01[9:4]`
+      - Cr\ :sub:`01[9:2]`
+    * - start + 25:
+      - Cb\ :sub:`10[7:0]`
+      - Cr\ :sub:`10[5:0]`\ Cb\ :sub:`10[9:8]`
+      - Cb\ :sub:`11[3:0]`\ Cr\ :sub:`10[9:6]`
+      - Cr\ :sub:`11[1:0]`\ Cb\ :sub:`11[9:4]`
+      - Cr\ :sub:`11[9:2]`
+
+
 .. _V4L2-PIX-FMT-NV12MT:
 .. _V4L2-PIX-FMT-NV12MT-16X16:
 .. _V4L2-PIX-FMT-NV12-4L4:
@@ -631,6 +696,69 @@ number of lines as the luma plane.
       - Cr\ :sub:`32`
 
 
+.. _V4L2-PIX-FMT-NV20:
+
+NV20
+----
+
+Semi-planar 10-bit YUV 4:2:2 format similar to NV16, using 10-bit components
+with no padding between each component. A group of 4 components are stored over
+5 bytes in little endian order.
+
+.. flat-table:: Sample 4x4 NV20 Image (1 byte per cell)
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - start + 0:
+      - Y'\ :sub:`00[7:0]`
+      - Y'\ :sub:`01[5:0]`\ Y'\ :sub:`00[9:8]`
+      - Y'\ :sub:`02[3:0]`\ Y'\ :sub:`01[9:6]`
+      - Y'\ :sub:`03[1:0]`\ Y'\ :sub:`02[9:4]`
+      - Y'\ :sub:`03[9:2]`
+    * - start + 5:
+      - Y'\ :sub:`10[7:0]`
+      - Y'\ :sub:`11[5:0]`\ Y'\ :sub:`10[9:8]`
+      - Y'\ :sub:`12[3:0]`\ Y'\ :sub:`11[9:6]`
+      - Y'\ :sub:`13[1:0]`\ Y'\ :sub:`12[9:4]`
+      - Y'\ :sub:`13[9:2]`
+    * - start + 10:
+      - Y'\ :sub:`20[7:0]`
+      - Y'\ :sub:`21[5:0]`\ Y'\ :sub:`20[9:8]`
+      - Y'\ :sub:`22[3:0]`\ Y'\ :sub:`21[9:6]`
+      - Y'\ :sub:`23[1:0]`\ Y'\ :sub:`22[9:4]`
+      - Y'\ :sub:`23[9:2]`
+    * - start + 15:
+      - Y'\ :sub:`30[7:0]`
+      - Y'\ :sub:`31[5:0]`\ Y'\ :sub:`30[9:8]`
+      - Y'\ :sub:`32[3:0]`\ Y'\ :sub:`31[9:6]`
+      - Y'\ :sub:`33[1:0]`\ Y'\ :sub:`32[9:4]`
+      - Y'\ :sub:`33[9:2]`
+    * - start + 20:
+      - Cb\ :sub:`00[7:0]`
+      - Cr\ :sub:`00[5:0]`\ Cb\ :sub:`00[9:8]`
+      - Cb\ :sub:`01[3:0]`\ Cr\ :sub:`00[9:6]`
+      - Cr\ :sub:`01[1:0]`\ Cb\ :sub:`01[9:4]`
+      - Cr\ :sub:`01[9:2]`
+    * - start + 25:
+      - Cb\ :sub:`10[7:0]`
+      - Cr\ :sub:`10[5:0]`\ Cb\ :sub:`10[9:8]`
+      - Cb\ :sub:`11[3:0]`\ Cr\ :sub:`10[9:6]`
+      - Cr\ :sub:`11[1:0]`\ Cb\ :sub:`11[9:4]`
+      - Cr\ :sub:`11[9:2]`
+    * - start + 30:
+      - Cb\ :sub:`20[7:0]`
+      - Cr\ :sub:`20[5:0]`\ Cb\ :sub:`20[9:8]`
+      - Cb\ :sub:`21[3:0]`\ Cr\ :sub:`20[9:6]`
+      - Cr\ :sub:`21[1:0]`\ Cb\ :sub:`21[9:4]`
+      - Cr\ :sub:`21[9:2]`
+    * - start + 35:
+      - Cb\ :sub:`30[7:0]`
+      - Cr\ :sub:`30[5:0]`\ Cb\ :sub:`30[9:8]`
+      - Cb\ :sub:`31[3:0]`\ Cr\ :sub:`30[9:6]`
+      - Cr\ :sub:`31[1:0]`\ Cb\ :sub:`31[9:4]`
+      - Cr\ :sub:`31[9:2]`
+
+
 .. _V4L2-PIX-FMT-NV24:
 .. _V4L2-PIX-FMT-NV42:
 
diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 8ad04d48a023..53b6fdfcb4dd 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -277,8 +277,10 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
 		/* YUV planar formats */
 		{ .format = V4L2_PIX_FMT_NV12,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 2 },
 		{ .format = V4L2_PIX_FMT_NV21,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 2 },
+		{ .format = V4L2_PIX_FMT_NV15,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 2 },
 		{ .format = V4L2_PIX_FMT_NV16,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
 		{ .format = V4L2_PIX_FMT_NV61,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_NV20,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 1 },
 		{ .format = V4L2_PIX_FMT_NV24,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
 		{ .format = V4L2_PIX_FMT_NV42,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
 		{ .format = V4L2_PIX_FMT_P010,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index e14db67be97c..9f09f5cf3e3e 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1360,8 +1360,10 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_YUV48_12:	descr = "12-bit YUV 4:4:4 Packed"; break;
 	case V4L2_PIX_FMT_NV12:		descr = "Y/UV 4:2:0"; break;
 	case V4L2_PIX_FMT_NV21:		descr = "Y/VU 4:2:0"; break;
+	case V4L2_PIX_FMT_NV15:		descr = "10-bit Y/UV 4:2:0 (Packed)"; break;
 	case V4L2_PIX_FMT_NV16:		descr = "Y/UV 4:2:2"; break;
 	case V4L2_PIX_FMT_NV61:		descr = "Y/VU 4:2:2"; break;
+	case V4L2_PIX_FMT_NV20:		descr = "10-bit Y/UV 4:2:2 (Packed)"; break;
 	case V4L2_PIX_FMT_NV24:		descr = "Y/UV 4:4:4"; break;
 	case V4L2_PIX_FMT_NV42:		descr = "Y/VU 4:4:4"; break;
 	case V4L2_PIX_FMT_P010:		descr = "10-bit Y/UV 4:2:0"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 725e86c4bbbd..941a96b81672 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -642,8 +642,10 @@ struct v4l2_pix_format {
 /* two planes -- one Y, one Cr + Cb interleaved  */
 #define V4L2_PIX_FMT_NV12    v4l2_fourcc('N', 'V', '1', '2') /* 12  Y/CbCr 4:2:0  */
 #define V4L2_PIX_FMT_NV21    v4l2_fourcc('N', 'V', '2', '1') /* 12  Y/CrCb 4:2:0  */
+#define V4L2_PIX_FMT_NV15    v4l2_fourcc('N', 'V', '1', '5') /* 15  Y/CbCr 4:2:0 10-bit packed */
 #define V4L2_PIX_FMT_NV16    v4l2_fourcc('N', 'V', '1', '6') /* 16  Y/CbCr 4:2:2  */
 #define V4L2_PIX_FMT_NV61    v4l2_fourcc('N', 'V', '6', '1') /* 16  Y/CrCb 4:2:2  */
+#define V4L2_PIX_FMT_NV20    v4l2_fourcc('N', 'V', '2', '0') /* 20  Y/CbCr 4:2:2 10-bit packed */
 #define V4L2_PIX_FMT_NV24    v4l2_fourcc('N', 'V', '2', '4') /* 24  Y/CbCr 4:4:4  */
 #define V4L2_PIX_FMT_NV42    v4l2_fourcc('N', 'V', '4', '2') /* 24  Y/CrCb 4:4:4  */
 #define V4L2_PIX_FMT_P010    v4l2_fourcc('P', '0', '1', '0') /* 24  Y/CbCr 4:2:0 10-bit per component */
-- 
2.46.0

[PATCH v6 03/11] media: rkvdec: h264: Use bytesperline and buffer height as virstride

[Jonas Karlman]

Use bytesperline and buffer height to calculate the strides configured.

This does not really change anything other than ensuring the
bytesperline that is signaled to userspace matches what is configured
in HW.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Christopher Obbard <chris.obbard@collabora.com>
---
v6:
- No change

v5:
- Collect r-b and t-b tags

v4:
- No change

v3:
- Remove unnecessary yuv_virstride +=
---
 drivers/staging/media/rkvdec/rkvdec-h264.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 4fc167b42cf0..7a1e76d423df 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -896,9 +896,9 @@ static void config_registers(struct rkvdec_ctx *ctx,
 	dma_addr_t rlc_addr;
 	dma_addr_t refer_addr;
 	u32 rlc_len;
-	u32 hor_virstride = 0;
-	u32 ver_virstride = 0;
-	u32 y_virstride = 0;
+	u32 hor_virstride;
+	u32 ver_virstride;
+	u32 y_virstride;
 	u32 yuv_virstride = 0;
 	u32 offset;
 	dma_addr_t dst_addr;
@@ -909,16 +909,16 @@ static void config_registers(struct rkvdec_ctx *ctx,
 
 	f = &ctx->decoded_fmt;
 	dst_fmt = &f->fmt.pix_mp;
-	hor_virstride = (sps->bit_depth_luma_minus8 + 8) * dst_fmt->width / 8;
-	ver_virstride = round_up(dst_fmt->height, 16);
+	hor_virstride = dst_fmt->plane_fmt[0].bytesperline;
+	ver_virstride = dst_fmt->height;
 	y_virstride = hor_virstride * ver_virstride;
 
 	if (sps->chroma_format_idc == 0)
 		yuv_virstride = y_virstride;
 	else if (sps->chroma_format_idc == 1)
-		yuv_virstride += y_virstride + y_virstride / 2;
+		yuv_virstride = y_virstride + y_virstride / 2;
 	else if (sps->chroma_format_idc == 2)
-		yuv_virstride += 2 * y_virstride;
+		yuv_virstride = 2 * y_virstride;
 
 	reg = RKVDEC_Y_HOR_VIRSTRIDE(hor_virstride / 16) |
 	      RKVDEC_UV_HOR_VIRSTRIDE(hor_virstride / 16) |
-- 
2.46.0

[PATCH v6 04/11] media: rkvdec: h264: Don't hardcode SPS/PPS parameters

[Jonas Karlman]

From: Alex Bee <knaerzche@gmail.com>

Some SPS/PPS parameters are currently hardcoded in the driver even
though they exist in the stable uapi controls.

Use values from SPS/PPS controls instead of hardcoding them.

Signed-off-by: Alex Bee <knaerzche@gmail.com>
[jonas@kwiboo.se: constraint_set_flags condition, commit message]
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Christopher Obbard <chris.obbard@collabora.com>
---
v6:
- No change

v5:
- Collect r-b and t-b tags

v4:
- No change

v3:
- New patch
---
 drivers/staging/media/rkvdec/rkvdec-h264.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 7a1e76d423df..8bce8902b8dd 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -655,13 +655,14 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
 
 #define WRITE_PPS(value, field) set_ps_field(hw_ps->info, field, value)
 	/* write sps */
-	WRITE_PPS(0xf, SEQ_PARAMETER_SET_ID);
-	WRITE_PPS(0xff, PROFILE_IDC);
-	WRITE_PPS(1, CONSTRAINT_SET3_FLAG);
+	WRITE_PPS(sps->seq_parameter_set_id, SEQ_PARAMETER_SET_ID);
+	WRITE_PPS(sps->profile_idc, PROFILE_IDC);
+	WRITE_PPS(!!(sps->constraint_set_flags & (1 << 3)), CONSTRAINT_SET3_FLAG);
 	WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC);
 	WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA);
 	WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA);
-	WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG);
+	WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS),
+		  QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG);
 	WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4);
 	WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES);
 	WRITE_PPS(sps->pic_order_cnt_type, PIC_ORDER_CNT_TYPE);
@@ -688,8 +689,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
 		  DIRECT_8X8_INFERENCE_FLAG);
 
 	/* write pps */
-	WRITE_PPS(0xff, PIC_PARAMETER_SET_ID);
-	WRITE_PPS(0x1f, PPS_SEQ_PARAMETER_SET_ID);
+	WRITE_PPS(pps->pic_parameter_set_id, PIC_PARAMETER_SET_ID);
+	WRITE_PPS(pps->seq_parameter_set_id, PPS_SEQ_PARAMETER_SET_ID);
 	WRITE_PPS(!!(pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE),
 		  ENTROPY_CODING_MODE_FLAG);
 	WRITE_PPS(!!(pps->flags & V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT),
-- 
2.46.0

[PATCH v6 05/11] media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper

[Jonas Karlman]

Extract call to v4l2_fill_pixfmt_mp() and ajusting of sizeimage into a
helper. Replace current code with a call to the new helper.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Christopher Obbard <chris.obbard@collabora.com>
---
v6:
- No change

v5:
- Collect r-b and t-b tags

v4:
- Do not reset pix_mp->field in rkvdec_fill_decoded_pixfmt()

v3:
- No change
---
 drivers/staging/media/rkvdec/rkvdec.c | 27 ++++++++++++++-------------
 1 file changed, 14 insertions(+), 13 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index ac398b5a9736..d1087519d218 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -27,6 +27,16 @@
 #include "rkvdec.h"
 #include "rkvdec-regs.h"
 
+static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx,
+				       struct v4l2_pix_format_mplane *pix_mp)
+{
+	v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat,
+			    pix_mp->width, pix_mp->height);
+	pix_mp->plane_fmt[0].sizeimage += 128 *
+		DIV_ROUND_UP(pix_mp->width, 16) *
+		DIV_ROUND_UP(pix_mp->height, 16);
+}
+
 static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
@@ -192,13 +202,9 @@ static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
 
 	rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
 	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
-	v4l2_fill_pixfmt_mp(&f->fmt.pix_mp,
-			    ctx->coded_fmt_desc->decoded_fmts[0],
-			    ctx->coded_fmt.fmt.pix_mp.width,
-			    ctx->coded_fmt.fmt.pix_mp.height);
-	f->fmt.pix_mp.plane_fmt[0].sizeimage += 128 *
-		DIV_ROUND_UP(f->fmt.pix_mp.width, 16) *
-		DIV_ROUND_UP(f->fmt.pix_mp.height, 16);
+	f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
+	f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height;
+	rkvdec_fill_decoded_pixfmt(ctx, &f->fmt.pix_mp);
 }
 
 static int rkvdec_enum_framesizes(struct file *file, void *priv,
@@ -264,12 +270,7 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
 				       &pix_mp->height,
 				       &coded_desc->frmsize);
 
-	v4l2_fill_pixfmt_mp(pix_mp, pix_mp->pixelformat,
-			    pix_mp->width, pix_mp->height);
-	pix_mp->plane_fmt[0].sizeimage +=
-		128 *
-		DIV_ROUND_UP(pix_mp->width, 16) *
-		DIV_ROUND_UP(pix_mp->height, 16);
+	rkvdec_fill_decoded_pixfmt(ctx, pix_mp);
 	pix_mp->field = V4L2_FIELD_NONE;
 
 	return 0;
-- 
2.46.0

[PATCH v6 06/11] media: rkvdec: Move rkvdec_reset_decoded_fmt helper

[Jonas Karlman]

Move rkvdec_reset_decoded_fmt() and the called rkvdec_reset_fmt() helper
functions in preparation for adding a new caller in an upcoming patch.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Christopher Obbard <chris.obbard@collabora.com>
---
v6:
- No change

v5:
- Collect r-b and t-b tags

v4:
- No change

v3:
- New patch
---
 drivers/staging/media/rkvdec/rkvdec.c | 46 +++++++++++++--------------
 1 file changed, 23 insertions(+), 23 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index d1087519d218..7e59db3fc24d 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -37,6 +37,29 @@ static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx,
 		DIV_ROUND_UP(pix_mp->height, 16);
 }
 
+static void rkvdec_reset_fmt(struct rkvdec_ctx *ctx, struct v4l2_format *f,
+			     u32 fourcc)
+{
+	memset(f, 0, sizeof(*f));
+	f->fmt.pix_mp.pixelformat = fourcc;
+	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
+	f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+	f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
+	f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
+}
+
+static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
+{
+	struct v4l2_format *f = &ctx->decoded_fmt;
+
+	rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
+	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
+	f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height;
+	rkvdec_fill_decoded_pixfmt(ctx, &f->fmt.pix_mp);
+}
+
 static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
@@ -169,18 +192,6 @@ rkvdec_find_coded_fmt_desc(u32 fourcc)
 	return NULL;
 }
 
-static void rkvdec_reset_fmt(struct rkvdec_ctx *ctx, struct v4l2_format *f,
-			     u32 fourcc)
-{
-	memset(f, 0, sizeof(*f));
-	f->fmt.pix_mp.pixelformat = fourcc;
-	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
-	f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_REC709;
-	f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
-	f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT;
-	f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT;
-}
-
 static void rkvdec_reset_coded_fmt(struct rkvdec_ctx *ctx)
 {
 	struct v4l2_format *f = &ctx->coded_fmt;
@@ -196,17 +207,6 @@ static void rkvdec_reset_coded_fmt(struct rkvdec_ctx *ctx)
 		ctx->coded_fmt_desc->ops->adjust_fmt(ctx, f);
 }
 
-static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
-{
-	struct v4l2_format *f = &ctx->decoded_fmt;
-
-	rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
-	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
-	f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
-	f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height;
-	rkvdec_fill_decoded_pixfmt(ctx, &f->fmt.pix_mp);
-}
-
 static int rkvdec_enum_framesizes(struct file *file, void *priv,
 				  struct v4l2_frmsizeenum *fsize)
 {
-- 
2.46.0

[PATCH v6 07/11] media: rkvdec: Extract decoded format enumeration into helper

[Jonas Karlman]

Add a rkvdec_is_valid_fmt() helper that check if a fourcc is a supported
CAPTURE format, and a rkvdec_enum_decoded_fmt() helper that enumerates
valid formats.

This moves current code into helper functions in preparation for adding
CAPTURE format filtering and validation in next patch.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Christopher Obbard <chris.obbard@collabora.com>
---
v6:
- No change

v5:
- Update commit message
- Collect r-b and t-b tags

v4:
- Rename rkvdec_decoded_fmts() to rkvdec_enum_decoded_fmt()
- Rename rkvdec_valid_fmt() to rkvdec_is_valid_fmt()

v3:
- New patch
---
 drivers/staging/media/rkvdec/rkvdec.c | 49 +++++++++++++++++++--------
 1 file changed, 35 insertions(+), 14 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 7e59db3fc24d..efbf9aa578ae 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -27,6 +27,32 @@
 #include "rkvdec.h"
 #include "rkvdec-regs.h"
 
+static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index)
+{
+	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+
+	if (WARN_ON(!desc))
+		return 0;
+
+	if (index >= desc->num_decoded_fmts)
+		return 0;
+
+	return desc->decoded_fmts[index];
+}
+
+static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc)
+{
+	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+	unsigned int i;
+
+	for (i = 0; i < desc->num_decoded_fmts; i++) {
+		if (desc->decoded_fmts[i] == fourcc)
+			return true;
+	}
+
+	return false;
+}
+
 static void rkvdec_fill_decoded_pixfmt(struct rkvdec_ctx *ctx,
 				       struct v4l2_pix_format_mplane *pix_mp)
 {
@@ -52,8 +78,10 @@ static void rkvdec_reset_fmt(struct rkvdec_ctx *ctx, struct v4l2_format *f,
 static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
 {
 	struct v4l2_format *f = &ctx->decoded_fmt;
+	u32 fourcc;
 
-	rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]);
+	fourcc = rkvdec_enum_decoded_fmt(ctx, 0);
+	rkvdec_reset_fmt(ctx, f, fourcc);
 	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
 	f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
 	f->fmt.pix_mp.height = ctx->coded_fmt.fmt.pix_mp.height;
@@ -244,7 +272,6 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
 	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv);
 	const struct rkvdec_coded_fmt_desc *coded_desc;
-	unsigned int i;
 
 	/*
 	 * The codec context should point to a coded format desc, if the format
@@ -255,13 +282,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
 	if (WARN_ON(!coded_desc))
 		return -EINVAL;
 
-	for (i = 0; i < coded_desc->num_decoded_fmts; i++) {
-		if (coded_desc->decoded_fmts[i] == pix_mp->pixelformat)
-			break;
-	}
-
-	if (i == coded_desc->num_decoded_fmts)
-		pix_mp->pixelformat = coded_desc->decoded_fmts[0];
+	if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat))
+		pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0);
 
 	/* Always apply the frmsize constraint of the coded end. */
 	pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width);
@@ -425,14 +447,13 @@ static int rkvdec_enum_capture_fmt(struct file *file, void *priv,
 				   struct v4l2_fmtdesc *f)
 {
 	struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv);
+	u32 fourcc;
 
-	if (WARN_ON(!ctx->coded_fmt_desc))
-		return -EINVAL;
-
-	if (f->index >= ctx->coded_fmt_desc->num_decoded_fmts)
+	fourcc = rkvdec_enum_decoded_fmt(ctx, f->index);
+	if (!fourcc)
 		return -EINVAL;
 
-	f->pixelformat = ctx->coded_fmt_desc->decoded_fmts[f->index];
+	f->pixelformat = fourcc;
 	return 0;
 }
 
-- 
2.46.0

[PATCH v6 08/11] media: rkvdec: Add image format concept

[Jonas Karlman]

Add an enum rkvdec_image_fmt used to signal an image format, e.g.
4:2:0 8-bit, 4:2:0 10-bit or any.

Tag each supported CAPUTRE format with an image format and use this tag
to filter out unsupported CAPTURE formats.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Christopher Obbard <chris.obbard@collabora.com>
---
v6:
- Change to use fmt_idx instead of j++ tucked inside a condition (Dan)

v5:
- Collect t-b tags

v4:
- Change fmt_opaque into an image format
- Split patch into two

v3:
- New patch
---
 drivers/staging/media/rkvdec/rkvdec.c | 48 ++++++++++++++++++++-------
 drivers/staging/media/rkvdec/rkvdec.h | 13 +++++++-
 2 files changed, 48 insertions(+), 13 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index efbf9aa578ae..467fc05b347a 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -27,26 +27,45 @@
 #include "rkvdec.h"
 #include "rkvdec-regs.h"
 
-static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index)
+static inline bool rkvdec_image_fmt_match(enum rkvdec_image_fmt fmt1,
+					  enum rkvdec_image_fmt fmt2)
+{
+	return fmt1 == fmt2 || fmt2 == RKVDEC_IMG_FMT_ANY ||
+	       fmt1 == RKVDEC_IMG_FMT_ANY;
+}
+
+static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index,
+				   enum rkvdec_image_fmt image_fmt)
 {
 	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+	int fmt_idx = -1;
+	unsigned int i;
 
 	if (WARN_ON(!desc))
 		return 0;
 
-	if (index >= desc->num_decoded_fmts)
-		return 0;
+	for (i = 0; i < desc->num_decoded_fmts; i++) {
+		if (!rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
+					    image_fmt))
+			continue;
+		fmt_idx++;
+		if (index == fmt_idx)
+			return desc->decoded_fmts[i].fourcc;
+	}
 
-	return desc->decoded_fmts[index];
+	return 0;
 }
 
-static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc)
+static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc,
+				enum rkvdec_image_fmt image_fmt)
 {
 	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
 	unsigned int i;
 
 	for (i = 0; i < desc->num_decoded_fmts; i++) {
-		if (desc->decoded_fmts[i] == fourcc)
+		if (rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
+					   image_fmt) &&
+		    desc->decoded_fmts[i].fourcc == fourcc)
 			return true;
 	}
 
@@ -80,7 +99,7 @@ static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
 	struct v4l2_format *f = &ctx->decoded_fmt;
 	u32 fourcc;
 
-	fourcc = rkvdec_enum_decoded_fmt(ctx, 0);
+	fourcc = rkvdec_enum_decoded_fmt(ctx, 0, ctx->image_fmt);
 	rkvdec_reset_fmt(ctx, f, fourcc);
 	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
 	f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
@@ -149,8 +168,11 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = {
 	.num_ctrls = ARRAY_SIZE(rkvdec_h264_ctrl_descs),
 };
 
-static const u32 rkvdec_h264_vp9_decoded_fmts[] = {
-	V4L2_PIX_FMT_NV12,
+static const struct rkvdec_decoded_fmt_desc rkvdec_h264_vp9_decoded_fmts[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_NV12,
+		.image_fmt = RKVDEC_IMG_FMT_420_8BIT,
+	},
 };
 
 static const struct rkvdec_ctrl_desc rkvdec_vp9_ctrl_descs[] = {
@@ -282,8 +304,9 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
 	if (WARN_ON(!coded_desc))
 		return -EINVAL;
 
-	if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat))
-		pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0);
+	if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, ctx->image_fmt))
+		pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0,
+							      ctx->image_fmt);
 
 	/* Always apply the frmsize constraint of the coded end. */
 	pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width);
@@ -400,6 +423,7 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv,
 	 *
 	 * Note that this will propagates any size changes to the decoded format.
 	 */
+	ctx->image_fmt = RKVDEC_IMG_FMT_ANY;
 	rkvdec_reset_decoded_fmt(ctx);
 
 	/* Propagate colorspace information to capture. */
@@ -449,7 +473,7 @@ static int rkvdec_enum_capture_fmt(struct file *file, void *priv,
 	struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv);
 	u32 fourcc;
 
-	fourcc = rkvdec_enum_decoded_fmt(ctx, f->index);
+	fourcc = rkvdec_enum_decoded_fmt(ctx, f->index, ctx->image_fmt);
 	if (!fourcc)
 		return -EINVAL;
 
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 633335ebb9c4..6f8cf50c5d99 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -75,13 +75,23 @@ struct rkvdec_coded_fmt_ops {
 	int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
 };
 
+enum rkvdec_image_fmt {
+	RKVDEC_IMG_FMT_ANY = 0,
+	RKVDEC_IMG_FMT_420_8BIT,
+};
+
+struct rkvdec_decoded_fmt_desc {
+	u32 fourcc;
+	enum rkvdec_image_fmt image_fmt;
+};
+
 struct rkvdec_coded_fmt_desc {
 	u32 fourcc;
 	struct v4l2_frmsize_stepwise frmsize;
 	const struct rkvdec_ctrls *ctrls;
 	const struct rkvdec_coded_fmt_ops *ops;
 	unsigned int num_decoded_fmts;
-	const u32 *decoded_fmts;
+	const struct rkvdec_decoded_fmt_desc *decoded_fmts;
 	u32 subsystem_flags;
 };
 
@@ -104,6 +114,7 @@ struct rkvdec_ctx {
 	const struct rkvdec_coded_fmt_desc *coded_fmt_desc;
 	struct v4l2_ctrl_handler ctrl_hdl;
 	struct rkvdec_dev *dev;
+	enum rkvdec_image_fmt image_fmt;
 	void *priv;
 };
 
-- 
2.46.0

[PATCH v6 09/11] media: rkvdec: Add get_image_fmt ops

[Jonas Karlman]

Add support for a get_image_fmt() ops that return the required image
format.

The CAPTURE format is reset when required image format changes and the
buffer queue is not busy.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Christopher Obbard <chris.obbard@collabora.com>
---
v6:
- No change

v5:
- Collect t-b tags

v4:
- Change fmt_opaque into an image format
- Split patch into two

v3:
- New patch
---
 drivers/staging/media/rkvdec/rkvdec.c | 49 +++++++++++++++++++++++++--
 drivers/staging/media/rkvdec/rkvdec.h |  2 ++
 2 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 467fc05b347a..8df49ee12820 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -111,15 +111,60 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
 	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+	struct v4l2_pix_format_mplane *pix_mp = &ctx->decoded_fmt.fmt.pix_mp;
+	enum rkvdec_image_fmt image_fmt;
+	struct vb2_queue *vq;
+	int ret;
+
+	if (desc->ops->try_ctrl) {
+		ret = desc->ops->try_ctrl(ctx, ctrl);
+		if (ret)
+			return ret;
+	}
+
+	if (!desc->ops->get_image_fmt)
+		return 0;
 
-	if (desc->ops->try_ctrl)
-		return desc->ops->try_ctrl(ctx, ctrl);
+	image_fmt = desc->ops->get_image_fmt(ctx, ctrl);
+	if (ctx->image_fmt == image_fmt)
+		return 0;
+
+	if (rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, image_fmt))
+		return 0;
+
+	/* format change not allowed when queue is busy */
+	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
+			     V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+	if (vb2_is_busy(vq))
+		return -EINVAL;
+
+	return 0;
+}
+
+static int rkvdec_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
+	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+	struct v4l2_pix_format_mplane *pix_mp = &ctx->decoded_fmt.fmt.pix_mp;
+	enum rkvdec_image_fmt image_fmt;
+
+	if (!desc->ops->get_image_fmt)
+		return 0;
+
+	image_fmt = desc->ops->get_image_fmt(ctx, ctrl);
+	if (ctx->image_fmt == image_fmt)
+		return 0;
+
+	ctx->image_fmt = image_fmt;
+	if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, ctx->image_fmt))
+		rkvdec_reset_decoded_fmt(ctx);
 
 	return 0;
 }
 
 static const struct v4l2_ctrl_ops rkvdec_ctrl_ops = {
 	.try_ctrl = rkvdec_try_ctrl,
+	.s_ctrl = rkvdec_s_ctrl,
 };
 
 static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = {
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 6f8cf50c5d99..e466a2753ccf 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -73,6 +73,8 @@ struct rkvdec_coded_fmt_ops {
 		     struct vb2_v4l2_buffer *dst_buf,
 		     enum vb2_buffer_state result);
 	int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
+	enum rkvdec_image_fmt (*get_image_fmt)(struct rkvdec_ctx *ctx,
+					       struct v4l2_ctrl *ctrl);
 };
 
 enum rkvdec_image_fmt {
-- 
2.46.0

[PATCH v6 10/11] media: rkvdec: h264: Support High 10 and 4:2:2 profiles

[Jonas Karlman]

Add support and enable decoding of H264 High 10 and 4:2:2 profiles.

Decoded CAPTURE buffer width is aligned to 64 pixels to accommodate HW
requirement of 10-bit format buffers, fixes decoding of:

- Hi422FR13_SONY_A
- Hi422FR14_SONY_A
- Hi422FR15_SONY_A
- Hi422FR6_SONY_A
- Hi422FR7_SONY_A
- Hi422FR8_SONY_A
- Hi422FR9_SONY_A
- Hi422FREXT18_SONY_A

The get_image_fmt() ops is implemented to select an image format
required for the provided SPS control.

Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Christopher Obbard <chris.obbard@collabora.com>
---
v6:
- No change

v5:
- Move buffer align from rkvdec_fill_decoded_pixfmt to min/step_width
- Use correct profiles for V4L2_CID_MPEG_VIDEO_H264_PROFILE
- Collect t-b tags

v4:
- Change to use get_image_fmt() ops

v3:
- Add get_fmt_opaque ops, the expected pixelformat is used as opaque
- Add new valid_fmt ops that validate pixelformat matches opaque
- Update H264_PROFILE control max value
---
 drivers/staging/media/rkvdec/rkvdec-h264.c | 37 +++++++++++++++-----
 drivers/staging/media/rkvdec/rkvdec.c      | 40 ++++++++++++++++------
 drivers/staging/media/rkvdec/rkvdec.h      |  3 ++
 3 files changed, 61 insertions(+), 19 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 8bce8902b8dd..d14b4d173448 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -1027,24 +1027,42 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
 	return 0;
 }
 
+static enum rkvdec_image_fmt rkvdec_h264_get_image_fmt(struct rkvdec_ctx *ctx,
+						       struct v4l2_ctrl *ctrl)
+{
+	const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
+
+	if (ctrl->id != V4L2_CID_STATELESS_H264_SPS)
+		return RKVDEC_IMG_FMT_ANY;
+
+	if (sps->bit_depth_luma_minus8 == 0) {
+		if (sps->chroma_format_idc == 2)
+			return RKVDEC_IMG_FMT_422_8BIT;
+		else
+			return RKVDEC_IMG_FMT_420_8BIT;
+	} else if (sps->bit_depth_luma_minus8 == 2) {
+		if (sps->chroma_format_idc == 2)
+			return RKVDEC_IMG_FMT_422_10BIT;
+		else
+			return RKVDEC_IMG_FMT_420_10BIT;
+	}
+
+	return RKVDEC_IMG_FMT_ANY;
+}
+
 static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
 				    const struct v4l2_ctrl_h264_sps *sps)
 {
 	unsigned int width, height;
 
-	/*
-	 * TODO: The hardware supports 10-bit and 4:2:2 profiles,
-	 * but it's currently broken in the driver.
-	 * Reject them for now, until it's fixed.
-	 */
-	if (sps->chroma_format_idc > 1)
-		/* Only 4:0:0 and 4:2:0 are supported */
+	if (sps->chroma_format_idc > 2)
+		/* Only 4:0:0, 4:2:0 and 4:2:2 are supported */
 		return -EINVAL;
 	if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
 		/* Luma and chroma bit depth mismatch */
 		return -EINVAL;
-	if (sps->bit_depth_luma_minus8 != 0)
-		/* Only 8-bit is supported */
+	if (sps->bit_depth_luma_minus8 != 0 && sps->bit_depth_luma_minus8 != 2)
+		/* Only 8-bit and 10-bit is supported */
 		return -EINVAL;
 
 	width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
@@ -1190,4 +1208,5 @@ const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
 	.stop = rkvdec_h264_stop,
 	.run = rkvdec_h264_run,
 	.try_ctrl = rkvdec_h264_try_ctrl,
+	.get_image_fmt = rkvdec_h264_get_image_fmt,
 };
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 8df49ee12820..c8c14f35ac44 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -195,10 +195,11 @@ static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = {
 	},
 	{
 		.cfg.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
-		.cfg.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
-		.cfg.max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
+		.cfg.min = V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE,
+		.cfg.max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422_INTRA,
 		.cfg.menu_skip_mask =
-			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
+			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED) |
+			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_444_PREDICTIVE),
 		.cfg.def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
 	},
 	{
@@ -213,11 +214,23 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = {
 	.num_ctrls = ARRAY_SIZE(rkvdec_h264_ctrl_descs),
 };
 
-static const struct rkvdec_decoded_fmt_desc rkvdec_h264_vp9_decoded_fmts[] = {
+static const struct rkvdec_decoded_fmt_desc rkvdec_h264_decoded_fmts[] = {
 	{
 		.fourcc = V4L2_PIX_FMT_NV12,
 		.image_fmt = RKVDEC_IMG_FMT_420_8BIT,
 	},
+	{
+		.fourcc = V4L2_PIX_FMT_NV15,
+		.image_fmt = RKVDEC_IMG_FMT_420_10BIT,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_NV16,
+		.image_fmt = RKVDEC_IMG_FMT_422_8BIT,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_NV20,
+		.image_fmt = RKVDEC_IMG_FMT_422_10BIT,
+	},
 };
 
 static const struct rkvdec_ctrl_desc rkvdec_vp9_ctrl_descs[] = {
@@ -240,21 +253,28 @@ static const struct rkvdec_ctrls rkvdec_vp9_ctrls = {
 	.num_ctrls = ARRAY_SIZE(rkvdec_vp9_ctrl_descs),
 };
 
+static const struct rkvdec_decoded_fmt_desc rkvdec_vp9_decoded_fmts[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_NV12,
+		.image_fmt = RKVDEC_IMG_FMT_420_8BIT,
+	},
+};
+
 static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
 	{
 		.fourcc = V4L2_PIX_FMT_H264_SLICE,
 		.frmsize = {
-			.min_width = 48,
+			.min_width = 64,
 			.max_width = 4096,
-			.step_width = 16,
+			.step_width = 64,
 			.min_height = 48,
 			.max_height = 2560,
 			.step_height = 16,
 		},
 		.ctrls = &rkvdec_h264_ctrls,
 		.ops = &rkvdec_h264_fmt_ops,
-		.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
-		.decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
+		.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts),
+		.decoded_fmts = rkvdec_h264_decoded_fmts,
 		.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
 	},
 	{
@@ -269,8 +289,8 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
 		},
 		.ctrls = &rkvdec_vp9_ctrls,
 		.ops = &rkvdec_vp9_fmt_ops,
-		.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
-		.decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
+		.num_decoded_fmts = ARRAY_SIZE(rkvdec_vp9_decoded_fmts),
+		.decoded_fmts = rkvdec_vp9_decoded_fmts,
 	}
 };
 
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index e466a2753ccf..9a9f4fced7a1 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -80,6 +80,9 @@ struct rkvdec_coded_fmt_ops {
 enum rkvdec_image_fmt {
 	RKVDEC_IMG_FMT_ANY = 0,
 	RKVDEC_IMG_FMT_420_8BIT,
+	RKVDEC_IMG_FMT_420_10BIT,
+	RKVDEC_IMG_FMT_422_8BIT,
+	RKVDEC_IMG_FMT_422_10BIT,
 };
 
 struct rkvdec_decoded_fmt_desc {
-- 
2.46.0

[PATCH v6 11/11] media: rkvdec: Fix enumerate frame sizes

[Jonas Karlman]

The VIDIOC_ENUM_FRAMESIZES ioctl should return all frame sizes (i. e.
width and height in pixels) that the device supports for the given pixel
format.

For coded format returning the frame size used to enforce HW alignment
requirements for CAPTURE buffers does not make fully sense.

Instead, signal applications what the maximum frame size that is
supported by the HW decoder using a frame size of continuous type.

Fixes: cd33c830448b ("media: rkvdec: Add the rkvdec driver")
Suggested-by: Alex Bee <knaerzche@gmail.com>
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
---
v6:
- New patch

With this change FFmpeg V4L2 Request API hwaccels can implement a strict
check if frame size is supported by the video device:
https://ffmpeg.org/pipermail/ffmpeg-devel/2024-August/332037.html
---
 drivers/staging/media/rkvdec/rkvdec.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index c8c14f35ac44..9002eb3a59e5 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -334,8 +334,14 @@ static int rkvdec_enum_framesizes(struct file *file, void *priv,
 	if (!fmt)
 		return -EINVAL;
 
-	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
-	fsize->stepwise = fmt->frmsize;
+	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+	fsize->stepwise.min_width = 1;
+	fsize->stepwise.max_width = fmt->frmsize.max_width;
+	fsize->stepwise.step_width = 1;
+	fsize->stepwise.min_height = 1;
+	fsize->stepwise.max_height = fmt->frmsize.max_height;
+	fsize->stepwise.step_height = 1;
+
 	return 0;
 }
 
-- 
2.46.0

Re: [PATCH v6 08/11] media: rkvdec: Add image format concept

[Dan Carpenter]

On Mon, Sep 09, 2024 at 07:25:06PM +0000, Jonas Karlman wrote:
> Add an enum rkvdec_image_fmt used to signal an image format, e.g.
> 4:2:0 8-bit, 4:2:0 10-bit or any.
> 
> Tag each supported CAPUTRE format with an image format and use this tag
> to filter out unsupported CAPTURE formats.
> 
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Tested-by: Christopher Obbard <chris.obbard@collabora.com>
> ---
> v6:
> - Change to use fmt_idx instead of j++ tucked inside a condition (Dan)

Thanks!

regards,
dan carpenter

Re: [PATCH v6 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

[Diederik de Haas]

[-- Attachment #1: Type: text/plain, Size: 2490 bytes --]

On Mon Sep 9, 2024 at 9:24 PM CEST, Jonas Karlman wrote:
> This series add H.264 High 10 and 4:2:2 profile support to the Rockchip
> Video Decoder driver.
> ...
> Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
> ...
>
> Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/
>
> To fully runtime test this series you may need FFmpeg patches from [1]
> and fluster patches from [2], this series is also available at [3].

I have been using this patch set and earlier version stemming all the
way back to 2023-10-29 and with a patched ffmpeg and mpv ([1] but
earlier versions before that) I have been enjoying HW accelerated
playback on my Rock64 (rk3328) :-)

So for the series, feel free to add my

Tested-by: Diederik de Haas <didi.debian@cknow.org>

[1] https://github.com/mpv-player/mpv/pull/14690

Cheers,
  Diederik

> [1] https://github.com/Kwiboo/FFmpeg/commits/v4l2request-2024-v2-rkvdec/
> [2] https://github.com/Kwiboo/fluster/commits/ffmpeg-v4l2request-rkvdec/
> [3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v6/
> [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
> [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf
>
> Regards,
> Jonas
>
> Alex Bee (1):
>   media: rkvdec: h264: Don't hardcode SPS/PPS parameters
>
> Jonas Karlman (10):
>   media: v4l2-common: Add helpers to calculate bytesperline and
>     sizeimage
>   media: v4l2: Add NV15 and NV20 pixel formats
>   media: rkvdec: h264: Use bytesperline and buffer height as virstride
>   media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
>   media: rkvdec: Move rkvdec_reset_decoded_fmt helper
>   media: rkvdec: Extract decoded format enumeration into helper
>   media: rkvdec: Add image format concept
>   media: rkvdec: Add get_image_fmt ops
>   media: rkvdec: h264: Support High 10 and 4:2:2 profiles
>   media: rkvdec: Fix enumerate frame sizes
>
>  .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++
>  drivers/media/v4l2-core/v4l2-common.c         |  80 +++---
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
>  drivers/staging/media/rkvdec/rkvdec-h264.c    |  64 +++--
>  drivers/staging/media/rkvdec/rkvdec.c         | 239 +++++++++++++-----
>  drivers/staging/media/rkvdec/rkvdec.h         |  18 +-
>  include/uapi/linux/videodev2.h                |   2 +
>  7 files changed, 410 insertions(+), 123 deletions(-)


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v6 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

[Jonas Karlman]

Hi Sebastian,

Will you have time to look at this series any time soon?

Would like to send a v2 of the now one year old rkvdec hevc series but
this series is sort of holding that back ;-)

Regards,
Jonas

On 2024-09-09 21:24, Jonas Karlman wrote:
> This series add H.264 High 10 and 4:2:2 profile support to the Rockchip
> Video Decoder driver.
> 
> Patch 1 add helpers for calculating plane bytesperline and sizeimage.
> Patch 2 add two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.
> 
> Patch 3 change to use bytesperline and buffer height to configure strides.
> Patch 4 change to use values from SPS/PPS control to configure the HW.
> 
> Patch 5-9 refactor code to support filtering of CAPUTRE formats based
> on the image format returned from a get_image_fmt ops.
> 
> Patch 10 add final bits to support H.264 High 10 and 4:2:2 profiles.
> 
> Patch 11 add a fix for enumerated frame sizes returned to userspace.
> 
> Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
> 
>   v4l2-compliance 1.28.1, 64 bits, 64-bit time_t
>   ...
>   Total for rkvdec device /dev/video1: 48, Succeeded: 48, Failed: 0, Warnings: 0
> 
>   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
>   ...
>   Ran 65/69 tests successfully
> 
>   Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-v4l2request
>   ...
>   Ran 129/135 tests successfully
> 
> Before this series:
> 
>   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
>   ...
>   Ran 44/69 tests successfully
> 
> Changes in v6:
> - Change to use fmt_idx instead of j++ tucked inside a condition (Dan)
> - Add patch to fix enumerated frame sizes returned to userspace (Alex)
> - Fluster test score is same as v4 and v5, see [4] and [5]
> Link to v5: https://lore.kernel.org/linux-media/20240618194647.742037-1-jonas@kwiboo.se/
> 
> Changes in v5:
> - Drop Remove SPS validation at streaming start patch
> - Move buffer align from rkvdec_fill_decoded_pixfmt to min/step_width
> - Use correct profiles for V4L2_CID_MPEG_VIDEO_H264_PROFILE
> - Collect r-b and t-b tags
> - Fluster test score is same as v4, see [4] and [5]
> Link to v4: https://lore.kernel.org/linux-media/20231105165521.3592037-1-jonas@kwiboo.se/
> 
> Changes in v4:
> - Fix failed v4l2-compliance tests related to CAPTURE queue
> - Rework CAPTURE format filter anv validate to use an image format
> - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
> Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/
> 
> Changes in v3:
> - Drop merged patches
> - Use bpp and bpp_div instead of prior misuse of block_w/block_h
> - New patch to use values from SPS/PPS control to configure the HW
> - New patch to remove an unnecessary call to validate sps at streaming start
> - Reworked pixel format validation
> Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/
> 
> Changes in v2:
> - Collect r-b tags
> - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
> - New patch to not override output buffer sizeimage
> - Reworked pixel format validation
> - Only align decoded buffer instead of changing frmsize step_width
> Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/
> 
> To fully runtime test this series you may need FFmpeg patches from [1]
> and fluster patches from [2], this series is also available at [3].
> 
> [1] https://github.com/Kwiboo/FFmpeg/commits/v4l2request-2024-v2-rkvdec/
> [2] https://github.com/Kwiboo/fluster/commits/ffmpeg-v4l2request-rkvdec/
> [3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v6/
> [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
> [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf
> 
> Regards,
> Jonas
> 
> Alex Bee (1):
>   media: rkvdec: h264: Don't hardcode SPS/PPS parameters
> 
> Jonas Karlman (10):
>   media: v4l2-common: Add helpers to calculate bytesperline and
>     sizeimage
>   media: v4l2: Add NV15 and NV20 pixel formats
>   media: rkvdec: h264: Use bytesperline and buffer height as virstride
>   media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
>   media: rkvdec: Move rkvdec_reset_decoded_fmt helper
>   media: rkvdec: Extract decoded format enumeration into helper
>   media: rkvdec: Add image format concept
>   media: rkvdec: Add get_image_fmt ops
>   media: rkvdec: h264: Support High 10 and 4:2:2 profiles
>   media: rkvdec: Fix enumerate frame sizes
> 
>  .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++
>  drivers/media/v4l2-core/v4l2-common.c         |  80 +++---
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
>  drivers/staging/media/rkvdec/rkvdec-h264.c    |  64 +++--
>  drivers/staging/media/rkvdec/rkvdec.c         | 239 +++++++++++++-----
>  drivers/staging/media/rkvdec/rkvdec.h         |  18 +-
>  include/uapi/linux/videodev2.h                |   2 +
>  7 files changed, 410 insertions(+), 123 deletions(-)
> 

Re: [PATCH v6 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

[Sebastian Fricke]

Hey Jonas,

On 25.10.2024 10:20, Jonas Karlman wrote:
>Hi Sebastian,
>
>Will you have time to look at this series any time soon?
>
>Would like to send a v2 of the now one year old rkvdec hevc series but
>this series is sort of holding that back ;-)

Sorry for the delay we (maintainer & reviewer from Collabora) are
currently a bit busy, this is on top of our review list however, so
please another 1 or 2 weeks of patience :).

>
>Regards,
>Jonas

Thanks for your work and regards!
Sebastian

>
>On 2024-09-09 21:24, Jonas Karlman wrote:
>> This series add H.264 High 10 and 4:2:2 profile support to the Rockchip
>> Video Decoder driver.
>>
>> Patch 1 add helpers for calculating plane bytesperline and sizeimage.
>> Patch 2 add two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.
>>
>> Patch 3 change to use bytesperline and buffer height to configure strides.
>> Patch 4 change to use values from SPS/PPS control to configure the HW.
>>
>> Patch 5-9 refactor code to support filtering of CAPUTRE formats based
>> on the image format returned from a get_image_fmt ops.
>>
>> Patch 10 add final bits to support H.264 High 10 and 4:2:2 profiles.
>>
>> Patch 11 add a fix for enumerated frame sizes returned to userspace.
>>
>> Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
>>
>>   v4l2-compliance 1.28.1, 64 bits, 64-bit time_t
>>   ...
>>   Total for rkvdec device /dev/video1: 48, Succeeded: 48, Failed: 0, Warnings: 0
>>
>>   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
>>   ...
>>   Ran 65/69 tests successfully
>>
>>   Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-v4l2request
>>   ...
>>   Ran 129/135 tests successfully
>>
>> Before this series:
>>
>>   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
>>   ...
>>   Ran 44/69 tests successfully
>>
>> Changes in v6:
>> - Change to use fmt_idx instead of j++ tucked inside a condition (Dan)
>> - Add patch to fix enumerated frame sizes returned to userspace (Alex)
>> - Fluster test score is same as v4 and v5, see [4] and [5]
>> Link to v5: https://lore.kernel.org/linux-media/20240618194647.742037-1-jonas@kwiboo.se/
>>
>> Changes in v5:
>> - Drop Remove SPS validation at streaming start patch
>> - Move buffer align from rkvdec_fill_decoded_pixfmt to min/step_width
>> - Use correct profiles for V4L2_CID_MPEG_VIDEO_H264_PROFILE
>> - Collect r-b and t-b tags
>> - Fluster test score is same as v4, see [4] and [5]
>> Link to v4: https://lore.kernel.org/linux-media/20231105165521.3592037-1-jonas@kwiboo.se/
>>
>> Changes in v4:
>> - Fix failed v4l2-compliance tests related to CAPTURE queue
>> - Rework CAPTURE format filter anv validate to use an image format
>> - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
>> Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/
>>
>> Changes in v3:
>> - Drop merged patches
>> - Use bpp and bpp_div instead of prior misuse of block_w/block_h
>> - New patch to use values from SPS/PPS control to configure the HW
>> - New patch to remove an unnecessary call to validate sps at streaming start
>> - Reworked pixel format validation
>> Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/
>>
>> Changes in v2:
>> - Collect r-b tags
>> - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
>> - New patch to not override output buffer sizeimage
>> - Reworked pixel format validation
>> - Only align decoded buffer instead of changing frmsize step_width
>> Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/
>>
>> To fully runtime test this series you may need FFmpeg patches from [1]
>> and fluster patches from [2], this series is also available at [3].
>>
>> [1] https://github.com/Kwiboo/FFmpeg/commits/v4l2request-2024-v2-rkvdec/
>> [2] https://github.com/Kwiboo/fluster/commits/ffmpeg-v4l2request-rkvdec/
>> [3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v6/
>> [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
>> [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf
>>
>> Regards,
>> Jonas
>>
>> Alex Bee (1):
>>   media: rkvdec: h264: Don't hardcode SPS/PPS parameters
>>
>> Jonas Karlman (10):
>>   media: v4l2-common: Add helpers to calculate bytesperline and
>>     sizeimage
>>   media: v4l2: Add NV15 and NV20 pixel formats
>>   media: rkvdec: h264: Use bytesperline and buffer height as virstride
>>   media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
>>   media: rkvdec: Move rkvdec_reset_decoded_fmt helper
>>   media: rkvdec: Extract decoded format enumeration into helper
>>   media: rkvdec: Add image format concept
>>   media: rkvdec: Add get_image_fmt ops
>>   media: rkvdec: h264: Support High 10 and 4:2:2 profiles
>>   media: rkvdec: Fix enumerate frame sizes
>>
>>  .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++
>>  drivers/media/v4l2-core/v4l2-common.c         |  80 +++---
>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
>>  drivers/staging/media/rkvdec/rkvdec-h264.c    |  64 +++--
>>  drivers/staging/media/rkvdec/rkvdec.c         | 239 +++++++++++++-----
>>  drivers/staging/media/rkvdec/rkvdec.h         |  18 +-
>>  include/uapi/linux/videodev2.h                |   2 +
>>  7 files changed, 410 insertions(+), 123 deletions(-)
>>
>
>
Sebastian Fricke
Consultant Software Engineer

Collabora Ltd
Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, UK
Registered in England & Wales no 5513718.

Re: [PATCH v6 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

[Nicolas Dufresne]

Le vendredi 25 octobre 2024 à 12:30 +0200, Sebastian Fricke a écrit :
> Hey Jonas,
> 
> On 25.10.2024 10:20, Jonas Karlman wrote:
> > Hi Sebastian,
> > 
> > Will you have time to look at this series any time soon?
> > 
> > Would like to send a v2 of the now one year old rkvdec hevc series but
> > this series is sort of holding that back ;-)
> 
> Sorry for the delay we (maintainer & reviewer from Collabora) are
> currently a bit busy, this is on top of our review list however, so
> please another 1 or 2 weeks of patience :).

Actually, I'll go over it today.

regards,
Nicolas

> 
> > 
> > Regards,
> > Jonas
> 
> Thanks for your work and regards!
> Sebastian
> 
> > 
> > On 2024-09-09 21:24, Jonas Karlman wrote:
> > > This series add H.264 High 10 and 4:2:2 profile support to the Rockchip
> > > Video Decoder driver.
> > > 
> > > Patch 1 add helpers for calculating plane bytesperline and sizeimage.
> > > Patch 2 add two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.
> > > 
> > > Patch 3 change to use bytesperline and buffer height to configure strides.
> > > Patch 4 change to use values from SPS/PPS control to configure the HW.
> > > 
> > > Patch 5-9 refactor code to support filtering of CAPUTRE formats based
> > > on the image format returned from a get_image_fmt ops.
> > > 
> > > Patch 10 add final bits to support H.264 High 10 and 4:2:2 profiles.
> > > 
> > > Patch 11 add a fix for enumerated frame sizes returned to userspace.
> > > 
> > > Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
> > > 
> > >   v4l2-compliance 1.28.1, 64 bits, 64-bit time_t
> > >   ...
> > >   Total for rkvdec device /dev/video1: 48, Succeeded: 48, Failed: 0, Warnings: 0
> > > 
> > >   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
> > >   ...
> > >   Ran 65/69 tests successfully
> > > 
> > >   Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-v4l2request
> > >   ...
> > >   Ran 129/135 tests successfully
> > > 
> > > Before this series:
> > > 
> > >   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
> > >   ...
> > >   Ran 44/69 tests successfully
> > > 
> > > Changes in v6:
> > > - Change to use fmt_idx instead of j++ tucked inside a condition (Dan)
> > > - Add patch to fix enumerated frame sizes returned to userspace (Alex)
> > > - Fluster test score is same as v4 and v5, see [4] and [5]
> > > Link to v5: https://lore.kernel.org/linux-media/20240618194647.742037-1-jonas@kwiboo.se/
> > > 
> > > Changes in v5:
> > > - Drop Remove SPS validation at streaming start patch
> > > - Move buffer align from rkvdec_fill_decoded_pixfmt to min/step_width
> > > - Use correct profiles for V4L2_CID_MPEG_VIDEO_H264_PROFILE
> > > - Collect r-b and t-b tags
> > > - Fluster test score is same as v4, see [4] and [5]
> > > Link to v4: https://lore.kernel.org/linux-media/20231105165521.3592037-1-jonas@kwiboo.se/
> > > 
> > > Changes in v4:
> > > - Fix failed v4l2-compliance tests related to CAPTURE queue
> > > - Rework CAPTURE format filter anv validate to use an image format
> > > - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
> > > Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/
> > > 
> > > Changes in v3:
> > > - Drop merged patches
> > > - Use bpp and bpp_div instead of prior misuse of block_w/block_h
> > > - New patch to use values from SPS/PPS control to configure the HW
> > > - New patch to remove an unnecessary call to validate sps at streaming start
> > > - Reworked pixel format validation
> > > Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/
> > > 
> > > Changes in v2:
> > > - Collect r-b tags
> > > - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
> > > - New patch to not override output buffer sizeimage
> > > - Reworked pixel format validation
> > > - Only align decoded buffer instead of changing frmsize step_width
> > > Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/
> > > 
> > > To fully runtime test this series you may need FFmpeg patches from [1]
> > > and fluster patches from [2], this series is also available at [3].
> > > 
> > > [1] https://github.com/Kwiboo/FFmpeg/commits/v4l2request-2024-v2-rkvdec/
> > > [2] https://github.com/Kwiboo/fluster/commits/ffmpeg-v4l2request-rkvdec/
> > > [3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v6/
> > > [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
> > > [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf
> > > 
> > > Regards,
> > > Jonas
> > > 
> > > Alex Bee (1):
> > >   media: rkvdec: h264: Don't hardcode SPS/PPS parameters
> > > 
> > > Jonas Karlman (10):
> > >   media: v4l2-common: Add helpers to calculate bytesperline and
> > >     sizeimage
> > >   media: v4l2: Add NV15 and NV20 pixel formats
> > >   media: rkvdec: h264: Use bytesperline and buffer height as virstride
> > >   media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
> > >   media: rkvdec: Move rkvdec_reset_decoded_fmt helper
> > >   media: rkvdec: Extract decoded format enumeration into helper
> > >   media: rkvdec: Add image format concept
> > >   media: rkvdec: Add get_image_fmt ops
> > >   media: rkvdec: h264: Support High 10 and 4:2:2 profiles
> > >   media: rkvdec: Fix enumerate frame sizes
> > > 
> > >  .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++
> > >  drivers/media/v4l2-core/v4l2-common.c         |  80 +++---
> > >  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
> > >  drivers/staging/media/rkvdec/rkvdec-h264.c    |  64 +++--
> > >  drivers/staging/media/rkvdec/rkvdec.c         | 239 +++++++++++++-----
> > >  drivers/staging/media/rkvdec/rkvdec.h         |  18 +-
> > >  include/uapi/linux/videodev2.h                |   2 +
> > >  7 files changed, 410 insertions(+), 123 deletions(-)
> > > 
> > 
> > 
> Sebastian Fricke
> Consultant Software Engineer
> 
> Collabora Ltd
> Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, UK
> Registered in England & Wales no 5513718.

Re: [PATCH v6 01/11] media: v4l2-common: Add helpers to calculate bytesperline and sizeimage

[Nicolas Dufresne]

Le lundi 09 septembre 2024 à 19:24 +0000, Jonas Karlman a écrit :
> Add helper functions to calculate plane bytesperline and sizeimage,
> these new helpers consider bpp div, block width and height when
> calculating plane bytesperline and sizeimage.
> 
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Tested-by: Christopher Obbard <chris.obbard@collabora.com>
> ---
> v6:
> - No change
> 
> v5:
> - Collect t-b tags
> 
> v4:
> - No change
> 
> v3:
> - Consider bpp_div in calculation
> ---
>  drivers/media/v4l2-core/v4l2-common.c | 78 +++++++++++++--------------
>  1 file changed, 39 insertions(+), 39 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> index 0a2f4f0d0a07..8ad04d48a023 100644
> --- a/drivers/media/v4l2-core/v4l2-common.c
> +++ b/drivers/media/v4l2-core/v4l2-common.c
> @@ -357,6 +357,34 @@ static inline unsigned int v4l2_format_block_height(const struct v4l2_format_inf
>  	return info->block_h[plane];
>  }
>  
> +static inline unsigned int v4l2_format_plane_width(const struct v4l2_format_info *info, int plane,

nit: What would you think of naming this one v4l2_format_plane_stride() ? As the
returned value is in bytes, not in pixels. (I'm not a huge fan of
_bytesperline(), but would seem valid and consistent too).

With or without this, I'm happy to see this land:

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

> +						   unsigned int width)
> +{
> +	unsigned int hdiv = plane ? info->hdiv : 1;
> +	unsigned int aligned_width =
> +		ALIGN(width, v4l2_format_block_width(info, plane));
> +
> +	return DIV_ROUND_UP(aligned_width, hdiv) *
> +	       info->bpp[plane] / info->bpp_div[plane];
> +}
> +
> +static inline unsigned int v4l2_format_plane_height(const struct v4l2_format_info *info, int plane,
> +						    unsigned int height)
> +{
> +	unsigned int vdiv = plane ? info->vdiv : 1;
> +	unsigned int aligned_height =
> +		ALIGN(height, v4l2_format_block_height(info, plane));
> +
> +	return DIV_ROUND_UP(aligned_height, vdiv);
> +}
> +
> +static inline unsigned int v4l2_format_plane_size(const struct v4l2_format_info *info, int plane,
> +						  unsigned int width, unsigned int height)
> +{
> +	return v4l2_format_plane_width(info, plane, width) *
> +	       v4l2_format_plane_height(info, plane, height);
> +}
> +
>  void v4l2_apply_frmsize_constraints(u32 *width, u32 *height,
>  				    const struct v4l2_frmsize_stepwise *frmsize)
>  {
> @@ -392,37 +420,19 @@ int v4l2_fill_pixfmt_mp(struct v4l2_pix_format_mplane *pixfmt,
>  
>  	if (info->mem_planes == 1) {
>  		plane = &pixfmt->plane_fmt[0];
> -		plane->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0] / info->bpp_div[0];
> +		plane->bytesperline = v4l2_format_plane_width(info, 0, width);
>  		plane->sizeimage = 0;
>  
> -		for (i = 0; i < info->comp_planes; i++) {
> -			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> -			unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
> -			unsigned int aligned_width;
> -			unsigned int aligned_height;
> -
> -			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> -			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> -
> -			plane->sizeimage += info->bpp[i] *
> -				DIV_ROUND_UP(aligned_width, hdiv) *
> -				DIV_ROUND_UP(aligned_height, vdiv) / info->bpp_div[i];
> -		}
> +		for (i = 0; i < info->comp_planes; i++)
> +			plane->sizeimage +=
> +				v4l2_format_plane_size(info, i, width, height);
>  	} else {
>  		for (i = 0; i < info->comp_planes; i++) {
> -			unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> -			unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
> -			unsigned int aligned_width;
> -			unsigned int aligned_height;
> -
> -			aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> -			aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> -
>  			plane = &pixfmt->plane_fmt[i];
>  			plane->bytesperline =
> -				info->bpp[i] * DIV_ROUND_UP(aligned_width, hdiv) / info->bpp_div[i];
> -			plane->sizeimage =
> -				plane->bytesperline * DIV_ROUND_UP(aligned_height, vdiv);
> +				v4l2_format_plane_width(info, i, width);
> +			plane->sizeimage = plane->bytesperline *
> +				v4l2_format_plane_height(info, i, height);
>  		}
>  	}
>  	return 0;
> @@ -446,22 +456,12 @@ int v4l2_fill_pixfmt(struct v4l2_pix_format *pixfmt, u32 pixelformat,
>  	pixfmt->width = width;
>  	pixfmt->height = height;
>  	pixfmt->pixelformat = pixelformat;
> -	pixfmt->bytesperline = ALIGN(width, v4l2_format_block_width(info, 0)) * info->bpp[0] / info->bpp_div[0];
> +	pixfmt->bytesperline = v4l2_format_plane_width(info, 0, width);
>  	pixfmt->sizeimage = 0;
>  
> -	for (i = 0; i < info->comp_planes; i++) {
> -		unsigned int hdiv = (i == 0) ? 1 : info->hdiv;
> -		unsigned int vdiv = (i == 0) ? 1 : info->vdiv;
> -		unsigned int aligned_width;
> -		unsigned int aligned_height;
> -
> -		aligned_width = ALIGN(width, v4l2_format_block_width(info, i));
> -		aligned_height = ALIGN(height, v4l2_format_block_height(info, i));
> -
> -		pixfmt->sizeimage += info->bpp[i] *
> -			DIV_ROUND_UP(aligned_width, hdiv) *
> -			DIV_ROUND_UP(aligned_height, vdiv) / info->bpp_div[i];
> -	}
> +	for (i = 0; i < info->comp_planes; i++)
> +		pixfmt->sizeimage +=
> +			v4l2_format_plane_size(info, i, width, height);
>  	return 0;
>  }
>  EXPORT_SYMBOL_GPL(v4l2_fill_pixfmt);

Re: [PATCH v6 02/11] media: v4l2: Add NV15 and NV20 pixel formats

[Nicolas Dufresne]

Le lundi 09 septembre 2024 à 19:25 +0000, Jonas Karlman a écrit :
> Add NV15 and NV20 pixel formats used by the Rockchip Video Decoder for
> 10-bit buffers.
> 
> NV15 and NV20 is 10-bit 4:2:0/4:2:2 semi-planar YUV formats similar to
> NV12 and NV16, using 10-bit components with no padding between each
> component. Instead, a group of 4 luminance/chrominance samples are
> stored over 5 bytes in little endian order:
> 
> YYYY = UVUV = 4 * 10 bits = 40 bits = 5 bytes
> 
> The '15' and '20' suffix refers to the optimum effective bits per pixel
> which is achieved when the total number of luminance samples is a
> multiple of 8 for NV15 and 4 for NV20.
> 
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Tested-by: Christopher Obbard <chris.obbard@collabora.com>
> ---
> v6:
> - No change
> 
> v5:
> - Collect t-b tags
> 
> v4:
> - No change
> 
> v3:
> - Use bpp and bpp_div instead of the misuse of block_w/block_h
> - Update documentation, expand to use full 4x4 sample image
> ---
>  .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++++++++++
>  drivers/media/v4l2-core/v4l2-common.c         |   2 +
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
>  include/uapi/linux/videodev2.h                |   2 +
>  4 files changed, 134 insertions(+)
> 
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
> index b788f6933855..22cad8c9726b 100644
> --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst
> @@ -79,6 +79,13 @@ All components are stored with the same number of bits per component.
>        - Cr, Cb
>        - Yes
>        - Linear
> +    * - V4L2_PIX_FMT_NV15

When I look at the existing grouping, all the NV12 variants are together, but
this patch splits them appart, and leaves a distance with its tiled variant
NV15_4L4.

I think I'd prefer to see that right before NV15_4L4, it can also allow removing
redundant wordings between the two.

> +      - 'NV15'
> +      - 10
> +      - 4:2:0
> +      - Cb, Cr
> +      - Yes
> +      - Linear
>      * - V4L2_PIX_FMT_NV12M
>        - 'NM12'
>        - 8
> @@ -172,6 +179,13 @@ All components are stored with the same number of bits per component.
>        - Cr, Cb
>        - Yes
>        - Linear
> +    * - V4L2_PIX_FMT_NV20
> +      - 'NV20'
> +      - 10
> +      - 4:2:2
> +      - Cb, Cr
> +      - Yes
> +      - Linear

Again, following some logic, I'd place that after V4L2_PIX_FMT_NV61M, keeping
all 8bit 4:2:2 formats before the 10bit ones.

>      * - V4L2_PIX_FMT_NV16M
>        - 'NM16'
>        - 8
> @@ -302,6 +316,57 @@ of the luma plane.
>        - Cr\ :sub:`11`
>  
>  
> +.. _V4L2-PIX-FMT-NV15:
> +
> +NV15
> +----
> +
> +Semi-planar 10-bit YUV 4:2:0 format similar to NV12, using 10-bit components
> +with no padding between each component. A group of 4 components are stored over
> +5 bytes in little endian order.
> +
> +.. flat-table:: Sample 4x4 NV15 Image (1 byte per cell)
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - Y'\ :sub:`00[7:0]`
> +      - Y'\ :sub:`01[5:0]`\ Y'\ :sub:`00[9:8]`
> +      - Y'\ :sub:`02[3:0]`\ Y'\ :sub:`01[9:6]`
> +      - Y'\ :sub:`03[1:0]`\ Y'\ :sub:`02[9:4]`
> +      - Y'\ :sub:`03[9:2]`
> +    * - start + 5:
> +      - Y'\ :sub:`10[7:0]`
> +      - Y'\ :sub:`11[5:0]`\ Y'\ :sub:`10[9:8]`
> +      - Y'\ :sub:`12[3:0]`\ Y'\ :sub:`11[9:6]`
> +      - Y'\ :sub:`13[1:0]`\ Y'\ :sub:`12[9:4]`
> +      - Y'\ :sub:`13[9:2]`
> +    * - start + 10:
> +      - Y'\ :sub:`20[7:0]`
> +      - Y'\ :sub:`21[5:0]`\ Y'\ :sub:`20[9:8]`
> +      - Y'\ :sub:`22[3:0]`\ Y'\ :sub:`21[9:6]`
> +      - Y'\ :sub:`23[1:0]`\ Y'\ :sub:`22[9:4]`
> +      - Y'\ :sub:`23[9:2]`
> +    * - start + 15:
> +      - Y'\ :sub:`30[7:0]`
> +      - Y'\ :sub:`31[5:0]`\ Y'\ :sub:`30[9:8]`
> +      - Y'\ :sub:`32[3:0]`\ Y'\ :sub:`31[9:6]`
> +      - Y'\ :sub:`33[1:0]`\ Y'\ :sub:`32[9:4]`
> +      - Y'\ :sub:`33[9:2]`
> +    * - start + 20:
> +      - Cb\ :sub:`00[7:0]`
> +      - Cr\ :sub:`00[5:0]`\ Cb\ :sub:`00[9:8]`
> +      - Cb\ :sub:`01[3:0]`\ Cr\ :sub:`00[9:6]`
> +      - Cr\ :sub:`01[1:0]`\ Cb\ :sub:`01[9:4]`
> +      - Cr\ :sub:`01[9:2]`
> +    * - start + 25:
> +      - Cb\ :sub:`10[7:0]`
> +      - Cr\ :sub:`10[5:0]`\ Cb\ :sub:`10[9:8]`
> +      - Cb\ :sub:`11[3:0]`\ Cr\ :sub:`10[9:6]`
> +      - Cr\ :sub:`11[1:0]`\ Cb\ :sub:`11[9:4]`
> +      - Cr\ :sub:`11[9:2]`
> +
> +
>  .. _V4L2-PIX-FMT-NV12MT:
>  .. _V4L2-PIX-FMT-NV12MT-16X16:
>  .. _V4L2-PIX-FMT-NV12-4L4:
> @@ -631,6 +696,69 @@ number of lines as the luma plane.
>        - Cr\ :sub:`32`
>  
>  
> +.. _V4L2-PIX-FMT-NV20:
> +
> +NV20
> +----
> +
> +Semi-planar 10-bit YUV 4:2:2 format similar to NV16, using 10-bit components
> +with no padding between each component. A group of 4 components are stored over
> +5 bytes in little endian order.
> +
> +.. flat-table:: Sample 4x4 NV20 Image (1 byte per cell)
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * - start + 0:
> +      - Y'\ :sub:`00[7:0]`
> +      - Y'\ :sub:`01[5:0]`\ Y'\ :sub:`00[9:8]`
> +      - Y'\ :sub:`02[3:0]`\ Y'\ :sub:`01[9:6]`
> +      - Y'\ :sub:`03[1:0]`\ Y'\ :sub:`02[9:4]`
> +      - Y'\ :sub:`03[9:2]`
> +    * - start + 5:
> +      - Y'\ :sub:`10[7:0]`
> +      - Y'\ :sub:`11[5:0]`\ Y'\ :sub:`10[9:8]`
> +      - Y'\ :sub:`12[3:0]`\ Y'\ :sub:`11[9:6]`
> +      - Y'\ :sub:`13[1:0]`\ Y'\ :sub:`12[9:4]`
> +      - Y'\ :sub:`13[9:2]`
> +    * - start + 10:
> +      - Y'\ :sub:`20[7:0]`
> +      - Y'\ :sub:`21[5:0]`\ Y'\ :sub:`20[9:8]`
> +      - Y'\ :sub:`22[3:0]`\ Y'\ :sub:`21[9:6]`
> +      - Y'\ :sub:`23[1:0]`\ Y'\ :sub:`22[9:4]`
> +      - Y'\ :sub:`23[9:2]`
> +    * - start + 15:
> +      - Y'\ :sub:`30[7:0]`
> +      - Y'\ :sub:`31[5:0]`\ Y'\ :sub:`30[9:8]`
> +      - Y'\ :sub:`32[3:0]`\ Y'\ :sub:`31[9:6]`
> +      - Y'\ :sub:`33[1:0]`\ Y'\ :sub:`32[9:4]`
> +      - Y'\ :sub:`33[9:2]`
> +    * - start + 20:
> +      - Cb\ :sub:`00[7:0]`
> +      - Cr\ :sub:`00[5:0]`\ Cb\ :sub:`00[9:8]`
> +      - Cb\ :sub:`01[3:0]`\ Cr\ :sub:`00[9:6]`
> +      - Cr\ :sub:`01[1:0]`\ Cb\ :sub:`01[9:4]`
> +      - Cr\ :sub:`01[9:2]`
> +    * - start + 25:
> +      - Cb\ :sub:`10[7:0]`
> +      - Cr\ :sub:`10[5:0]`\ Cb\ :sub:`10[9:8]`
> +      - Cb\ :sub:`11[3:0]`\ Cr\ :sub:`10[9:6]`
> +      - Cr\ :sub:`11[1:0]`\ Cb\ :sub:`11[9:4]`
> +      - Cr\ :sub:`11[9:2]`
> +    * - start + 30:
> +      - Cb\ :sub:`20[7:0]`
> +      - Cr\ :sub:`20[5:0]`\ Cb\ :sub:`20[9:8]`
> +      - Cb\ :sub:`21[3:0]`\ Cr\ :sub:`20[9:6]`
> +      - Cr\ :sub:`21[1:0]`\ Cb\ :sub:`21[9:4]`
> +      - Cr\ :sub:`21[9:2]`
> +    * - start + 35:
> +      - Cb\ :sub:`30[7:0]`
> +      - Cr\ :sub:`30[5:0]`\ Cb\ :sub:`30[9:8]`
> +      - Cb\ :sub:`31[3:0]`\ Cr\ :sub:`30[9:6]`
> +      - Cr\ :sub:`31[1:0]`\ Cb\ :sub:`31[9:4]`
> +      - Cr\ :sub:`31[9:2]`
> +
> +
>  .. _V4L2-PIX-FMT-NV24:
>  .. _V4L2-PIX-FMT-NV42:
>  
> diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
> index 8ad04d48a023..53b6fdfcb4dd 100644
> --- a/drivers/media/v4l2-core/v4l2-common.c
> +++ b/drivers/media/v4l2-core/v4l2-common.c
> @@ -277,8 +277,10 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
>  		/* YUV planar formats */
>  		{ .format = V4L2_PIX_FMT_NV12,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 2 },
>  		{ .format = V4L2_PIX_FMT_NV21,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 2 },
> +		{ .format = V4L2_PIX_FMT_NV15,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 2 },
>  		{ .format = V4L2_PIX_FMT_NV16,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
>  		{ .format = V4L2_PIX_FMT_NV61,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
> +		{ .format = V4L2_PIX_FMT_NV20,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 5, 10, 0, 0 }, .bpp_div = { 4, 4, 1, 1 }, .hdiv = 2, .vdiv = 1 },
>  		{ .format = V4L2_PIX_FMT_NV24,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
>  		{ .format = V4L2_PIX_FMT_NV42,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 1, .vdiv = 1 },
>  		{ .format = V4L2_PIX_FMT_P010,    .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 2, 2, 0, 0 }, .bpp_div = { 1, 1, 1, 1 }, .hdiv = 2, .vdiv = 1 },
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index e14db67be97c..9f09f5cf3e3e 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1360,8 +1360,10 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  	case V4L2_PIX_FMT_YUV48_12:	descr = "12-bit YUV 4:4:4 Packed"; break;
>  	case V4L2_PIX_FMT_NV12:		descr = "Y/UV 4:2:0"; break;
>  	case V4L2_PIX_FMT_NV21:		descr = "Y/VU 4:2:0"; break;
> +	case V4L2_PIX_FMT_NV15:		descr = "10-bit Y/UV 4:2:0 (Packed)"; break;
>  	case V4L2_PIX_FMT_NV16:		descr = "Y/UV 4:2:2"; break;
>  	case V4L2_PIX_FMT_NV61:		descr = "Y/VU 4:2:2"; break;
> +	case V4L2_PIX_FMT_NV20:		descr = "10-bit Y/UV 4:2:2 (Packed)"; break;
>  	case V4L2_PIX_FMT_NV24:		descr = "Y/UV 4:4:4"; break;
>  	case V4L2_PIX_FMT_NV42:		descr = "Y/VU 4:4:4"; break;
>  	case V4L2_PIX_FMT_P010:		descr = "10-bit Y/UV 4:2:0"; break;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 725e86c4bbbd..941a96b81672 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -642,8 +642,10 @@ struct v4l2_pix_format {
>  /* two planes -- one Y, one Cr + Cb interleaved  */
>  #define V4L2_PIX_FMT_NV12    v4l2_fourcc('N', 'V', '1', '2') /* 12  Y/CbCr 4:2:0  */
>  #define V4L2_PIX_FMT_NV21    v4l2_fourcc('N', 'V', '2', '1') /* 12  Y/CrCb 4:2:0  */
> +#define V4L2_PIX_FMT_NV15    v4l2_fourcc('N', 'V', '1', '5') /* 15  Y/CbCr 4:2:0 10-bit packed */
>  #define V4L2_PIX_FMT_NV16    v4l2_fourcc('N', 'V', '1', '6') /* 16  Y/CbCr 4:2:2  */
>  #define V4L2_PIX_FMT_NV61    v4l2_fourcc('N', 'V', '6', '1') /* 16  Y/CrCb 4:2:2  */
> +#define V4L2_PIX_FMT_NV20    v4l2_fourcc('N', 'V', '2', '0') /* 20  Y/CbCr 4:2:2 10-bit packed */
>  #define V4L2_PIX_FMT_NV24    v4l2_fourcc('N', 'V', '2', '4') /* 24  Y/CbCr 4:4:4  */
>  #define V4L2_PIX_FMT_NV42    v4l2_fourcc('N', 'V', '4', '2') /* 24  Y/CrCb 4:4:4  */
>  #define V4L2_PIX_FMT_P010    v4l2_fourcc('P', '0', '1', '0') /* 24  Y/CbCr 4:2:0 10-bit per component */


Appart from the location, I'm happy with the implementation and the
documentation.

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

Re: [PATCH v6 08/11] media: rkvdec: Add image format concept

[Nicolas Dufresne]

Le lundi 09 septembre 2024 à 19:25 +0000, Jonas Karlman a écrit :
> Add an enum rkvdec_image_fmt used to signal an image format, e.g.
> 4:2:0 8-bit, 4:2:0 10-bit or any.
> 
> Tag each supported CAPUTRE format with an image format and use this tag
> to filter out unsupported CAPTURE formats.
> 
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Tested-by: Christopher Obbard <chris.obbard@collabora.com>
> ---
> v6:
> - Change to use fmt_idx instead of j++ tucked inside a condition (Dan)
> 
> v5:
> - Collect t-b tags
> 
> v4:
> - Change fmt_opaque into an image format
> - Split patch into two
> 
> v3:
> - New patch
> ---
>  drivers/staging/media/rkvdec/rkvdec.c | 48 ++++++++++++++++++++-------
>  drivers/staging/media/rkvdec/rkvdec.h | 13 +++++++-
>  2 files changed, 48 insertions(+), 13 deletions(-)
> 
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index efbf9aa578ae..467fc05b347a 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -27,26 +27,45 @@
>  #include "rkvdec.h"
>  #include "rkvdec-regs.h"
>  
> -static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index)
> +static inline bool rkvdec_image_fmt_match(enum rkvdec_image_fmt fmt1,
> +					  enum rkvdec_image_fmt fmt2)

nit: We usually let the compiler inline function, specially when small and
static.

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

> +{
> +	return fmt1 == fmt2 || fmt2 == RKVDEC_IMG_FMT_ANY ||
> +	       fmt1 == RKVDEC_IMG_FMT_ANY;
> +}
> +
> +static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index,
> +				   enum rkvdec_image_fmt image_fmt)
>  {
>  	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
> +	int fmt_idx = -1;
> +	unsigned int i;
>  
>  	if (WARN_ON(!desc))
>  		return 0;
>  
> -	if (index >= desc->num_decoded_fmts)
> -		return 0;
> +	for (i = 0; i < desc->num_decoded_fmts; i++) {
> +		if (!rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
> +					    image_fmt))
> +			continue;
> +		fmt_idx++;
> +		if (index == fmt_idx)
> +			return desc->decoded_fmts[i].fourcc;
> +	}
>  
> -	return desc->decoded_fmts[index];
> +	return 0;
>  }
>  
> -static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc)
> +static bool rkvdec_is_valid_fmt(struct rkvdec_ctx *ctx, u32 fourcc,
> +				enum rkvdec_image_fmt image_fmt)
>  {
>  	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
>  	unsigned int i;
>  
>  	for (i = 0; i < desc->num_decoded_fmts; i++) {
> -		if (desc->decoded_fmts[i] == fourcc)
> +		if (rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
> +					   image_fmt) &&
> +		    desc->decoded_fmts[i].fourcc == fourcc)
>  			return true;
>  	}
>  
> @@ -80,7 +99,7 @@ static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx)
>  	struct v4l2_format *f = &ctx->decoded_fmt;
>  	u32 fourcc;
>  
> -	fourcc = rkvdec_enum_decoded_fmt(ctx, 0);
> +	fourcc = rkvdec_enum_decoded_fmt(ctx, 0, ctx->image_fmt);
>  	rkvdec_reset_fmt(ctx, f, fourcc);
>  	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
>  	f->fmt.pix_mp.width = ctx->coded_fmt.fmt.pix_mp.width;
> @@ -149,8 +168,11 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = {
>  	.num_ctrls = ARRAY_SIZE(rkvdec_h264_ctrl_descs),
>  };
>  
> -static const u32 rkvdec_h264_vp9_decoded_fmts[] = {
> -	V4L2_PIX_FMT_NV12,
> +static const struct rkvdec_decoded_fmt_desc rkvdec_h264_vp9_decoded_fmts[] = {
> +	{
> +		.fourcc = V4L2_PIX_FMT_NV12,
> +		.image_fmt = RKVDEC_IMG_FMT_420_8BIT,
> +	},
>  };
>  
>  static const struct rkvdec_ctrl_desc rkvdec_vp9_ctrl_descs[] = {
> @@ -282,8 +304,9 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
>  	if (WARN_ON(!coded_desc))
>  		return -EINVAL;
>  
> -	if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat))
> -		pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0);
> +	if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, ctx->image_fmt))
> +		pix_mp->pixelformat = rkvdec_enum_decoded_fmt(ctx, 0,
> +							      ctx->image_fmt);
>  
>  	/* Always apply the frmsize constraint of the coded end. */
>  	pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width);
> @@ -400,6 +423,7 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv,
>  	 *
>  	 * Note that this will propagates any size changes to the decoded format.
>  	 */
> +	ctx->image_fmt = RKVDEC_IMG_FMT_ANY;
>  	rkvdec_reset_decoded_fmt(ctx);
>  
>  	/* Propagate colorspace information to capture. */
> @@ -449,7 +473,7 @@ static int rkvdec_enum_capture_fmt(struct file *file, void *priv,
>  	struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv);
>  	u32 fourcc;
>  
> -	fourcc = rkvdec_enum_decoded_fmt(ctx, f->index);
> +	fourcc = rkvdec_enum_decoded_fmt(ctx, f->index, ctx->image_fmt);
>  	if (!fourcc)
>  		return -EINVAL;
>  
> diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
> index 633335ebb9c4..6f8cf50c5d99 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.h
> +++ b/drivers/staging/media/rkvdec/rkvdec.h
> @@ -75,13 +75,23 @@ struct rkvdec_coded_fmt_ops {
>  	int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
>  };
>  
> +enum rkvdec_image_fmt {
> +	RKVDEC_IMG_FMT_ANY = 0,
> +	RKVDEC_IMG_FMT_420_8BIT,
> +};
> +
> +struct rkvdec_decoded_fmt_desc {
> +	u32 fourcc;
> +	enum rkvdec_image_fmt image_fmt;
> +};
> +
>  struct rkvdec_coded_fmt_desc {
>  	u32 fourcc;
>  	struct v4l2_frmsize_stepwise frmsize;
>  	const struct rkvdec_ctrls *ctrls;
>  	const struct rkvdec_coded_fmt_ops *ops;
>  	unsigned int num_decoded_fmts;
> -	const u32 *decoded_fmts;
> +	const struct rkvdec_decoded_fmt_desc *decoded_fmts;
>  	u32 subsystem_flags;
>  };
>  
> @@ -104,6 +114,7 @@ struct rkvdec_ctx {
>  	const struct rkvdec_coded_fmt_desc *coded_fmt_desc;
>  	struct v4l2_ctrl_handler ctrl_hdl;
>  	struct rkvdec_dev *dev;
> +	enum rkvdec_image_fmt image_fmt;
>  	void *priv;
>  };
>  

Re: [PATCH v6 09/11] media: rkvdec: Add get_image_fmt ops

[Nicolas Dufresne]

Le lundi 09 septembre 2024 à 19:25 +0000, Jonas Karlman a écrit :
> Add support for a get_image_fmt() ops that return the required image
> format.
> 
> The CAPTURE format is reset when required image format changes and the
> buffer queue is not busy.
> 
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Tested-by: Christopher Obbard <chris.obbard@collabora.com>

Looks good to me.

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

> ---
> v6:
> - No change
> 
> v5:
> - Collect t-b tags
> 
> v4:
> - Change fmt_opaque into an image format
> - Split patch into two
> 
> v3:
> - New patch
> ---
>  drivers/staging/media/rkvdec/rkvdec.c | 49 +++++++++++++++++++++++++--
>  drivers/staging/media/rkvdec/rkvdec.h |  2 ++
>  2 files changed, 49 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index 467fc05b347a..8df49ee12820 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -111,15 +111,60 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
>  {
>  	struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
>  	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
> +	struct v4l2_pix_format_mplane *pix_mp = &ctx->decoded_fmt.fmt.pix_mp;
> +	enum rkvdec_image_fmt image_fmt;
> +	struct vb2_queue *vq;
> +	int ret;
> +
> +	if (desc->ops->try_ctrl) {
> +		ret = desc->ops->try_ctrl(ctx, ctrl);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	if (!desc->ops->get_image_fmt)
> +		return 0;
>  
> -	if (desc->ops->try_ctrl)
> -		return desc->ops->try_ctrl(ctx, ctrl);
> +	image_fmt = desc->ops->get_image_fmt(ctx, ctrl);
> +	if (ctx->image_fmt == image_fmt)
> +		return 0;
> +
> +	if (rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, image_fmt))
> +		return 0;
> +
> +	/* format change not allowed when queue is busy */
> +	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
> +			     V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
> +	if (vb2_is_busy(vq))
> +		return -EINVAL;
> +
> +	return 0;
> +}
> +
> +static int rkvdec_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> +	struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
> +	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
> +	struct v4l2_pix_format_mplane *pix_mp = &ctx->decoded_fmt.fmt.pix_mp;
> +	enum rkvdec_image_fmt image_fmt;
> +
> +	if (!desc->ops->get_image_fmt)
> +		return 0;
> +
> +	image_fmt = desc->ops->get_image_fmt(ctx, ctrl);
> +	if (ctx->image_fmt == image_fmt)
> +		return 0;
> +
> +	ctx->image_fmt = image_fmt;
> +	if (!rkvdec_is_valid_fmt(ctx, pix_mp->pixelformat, ctx->image_fmt))
> +		rkvdec_reset_decoded_fmt(ctx);
>  
>  	return 0;
>  }
>  
>  static const struct v4l2_ctrl_ops rkvdec_ctrl_ops = {
>  	.try_ctrl = rkvdec_try_ctrl,
> +	.s_ctrl = rkvdec_s_ctrl,
>  };
>  
>  static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = {
> diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
> index 6f8cf50c5d99..e466a2753ccf 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.h
> +++ b/drivers/staging/media/rkvdec/rkvdec.h
> @@ -73,6 +73,8 @@ struct rkvdec_coded_fmt_ops {
>  		     struct vb2_v4l2_buffer *dst_buf,
>  		     enum vb2_buffer_state result);
>  	int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
> +	enum rkvdec_image_fmt (*get_image_fmt)(struct rkvdec_ctx *ctx,
> +					       struct v4l2_ctrl *ctrl);
>  };
>  
>  enum rkvdec_image_fmt {

Re: [PATCH v6 10/11] media: rkvdec: h264: Support High 10 and 4:2:2 profiles

[Nicolas Dufresne]

Le lundi 09 septembre 2024 à 19:25 +0000, Jonas Karlman a écrit :
> Add support and enable decoding of H264 High 10 and 4:2:2 profiles.
> 
> Decoded CAPTURE buffer width is aligned to 64 pixels to accommodate HW
> requirement of 10-bit format buffers, fixes decoding of:
> 
> - Hi422FR13_SONY_A
> - Hi422FR14_SONY_A
> - Hi422FR15_SONY_A
> - Hi422FR6_SONY_A
> - Hi422FR7_SONY_A
> - Hi422FR8_SONY_A
> - Hi422FR9_SONY_A
> - Hi422FREXT18_SONY_A
> 
> The get_image_fmt() ops is implemented to select an image format
> required for the provided SPS control.
> 
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Tested-by: Christopher Obbard <chris.obbard@collabora.com>
> ---
> v6:
> - No change
> 
> v5:
> - Move buffer align from rkvdec_fill_decoded_pixfmt to min/step_width
> - Use correct profiles for V4L2_CID_MPEG_VIDEO_H264_PROFILE
> - Collect t-b tags
> 
> v4:
> - Change to use get_image_fmt() ops
> 
> v3:
> - Add get_fmt_opaque ops, the expected pixelformat is used as opaque
> - Add new valid_fmt ops that validate pixelformat matches opaque
> - Update H264_PROFILE control max value
> ---
>  drivers/staging/media/rkvdec/rkvdec-h264.c | 37 +++++++++++++++-----
>  drivers/staging/media/rkvdec/rkvdec.c      | 40 ++++++++++++++++------
>  drivers/staging/media/rkvdec/rkvdec.h      |  3 ++
>  3 files changed, 61 insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 8bce8902b8dd..d14b4d173448 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -1027,24 +1027,42 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
>  	return 0;
>  }
>  
> +static enum rkvdec_image_fmt rkvdec_h264_get_image_fmt(struct rkvdec_ctx *ctx,
> +						       struct v4l2_ctrl *ctrl)
> +{
> +	const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> +
> +	if (ctrl->id != V4L2_CID_STATELESS_H264_SPS)
> +		return RKVDEC_IMG_FMT_ANY;
> +
> +	if (sps->bit_depth_luma_minus8 == 0) {
> +		if (sps->chroma_format_idc == 2)
> +			return RKVDEC_IMG_FMT_422_8BIT;
> +		else
> +			return RKVDEC_IMG_FMT_420_8BIT;
> +	} else if (sps->bit_depth_luma_minus8 == 2) {
> +		if (sps->chroma_format_idc == 2)
> +			return RKVDEC_IMG_FMT_422_10BIT;
> +		else
> +			return RKVDEC_IMG_FMT_420_10BIT;
> +	}
> +
> +	return RKVDEC_IMG_FMT_ANY;
> +}
> +
>  static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
>  				    const struct v4l2_ctrl_h264_sps *sps)
>  {
>  	unsigned int width, height;
>  
> -	/*
> -	 * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> -	 * but it's currently broken in the driver.
> -	 * Reject them for now, until it's fixed.
> -	 */
> -	if (sps->chroma_format_idc > 1)
> -		/* Only 4:0:0 and 4:2:0 are supported */
> +	if (sps->chroma_format_idc > 2)
> +		/* Only 4:0:0, 4:2:0 and 4:2:2 are supported */
>  		return -EINVAL;
>  	if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
>  		/* Luma and chroma bit depth mismatch */
>  		return -EINVAL;
> -	if (sps->bit_depth_luma_minus8 != 0)
> -		/* Only 8-bit is supported */
> +	if (sps->bit_depth_luma_minus8 != 0 && sps->bit_depth_luma_minus8 != 2)
> +		/* Only 8-bit and 10-bit is supported */
>  		return -EINVAL;
>  
>  	width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
> @@ -1190,4 +1208,5 @@ const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
>  	.stop = rkvdec_h264_stop,
>  	.run = rkvdec_h264_run,
>  	.try_ctrl = rkvdec_h264_try_ctrl,
> +	.get_image_fmt = rkvdec_h264_get_image_fmt,
>  };
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index 8df49ee12820..c8c14f35ac44 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -195,10 +195,11 @@ static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = {
>  	},
>  	{
>  		.cfg.id = V4L2_CID_MPEG_VIDEO_H264_PROFILE,
> -		.cfg.min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
> -		.cfg.max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH,
> +		.cfg.min = V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE,

The change from BASELINE to CONSTRAINED_BASELINE is correct, but it bugs me a
little to hide it into this patch. Do you mind splitting it out ?

Note that these days, since we have no software fallback for it, GStreamer, and
probably FFMPEG too, still try out contrained-baseline decoders for baseline
content, as very often the bitstream does not actually uses FMO and ASO
features. So perhaps we can just edit this one out to help with merging this
set?

With one of these two fix:

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

> +		.cfg.max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422_INTRA,
>  		.cfg.menu_skip_mask =
> -			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED),
> +			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED) |
> +			BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_444_PREDICTIVE),
>  		.cfg.def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
>  	},
>  	{
> @@ -213,11 +214,23 @@ static const struct rkvdec_ctrls rkvdec_h264_ctrls = {
>  	.num_ctrls = ARRAY_SIZE(rkvdec_h264_ctrl_descs),
>  };
>  
> -static const struct rkvdec_decoded_fmt_desc rkvdec_h264_vp9_decoded_fmts[] = {
> +static const struct rkvdec_decoded_fmt_desc rkvdec_h264_decoded_fmts[] = {
>  	{
>  		.fourcc = V4L2_PIX_FMT_NV12,
>  		.image_fmt = RKVDEC_IMG_FMT_420_8BIT,
>  	},
> +	{
> +		.fourcc = V4L2_PIX_FMT_NV15,
> +		.image_fmt = RKVDEC_IMG_FMT_420_10BIT,
> +	},
> +	{
> +		.fourcc = V4L2_PIX_FMT_NV16,
> +		.image_fmt = RKVDEC_IMG_FMT_422_8BIT,
> +	},
> +	{
> +		.fourcc = V4L2_PIX_FMT_NV20,
> +		.image_fmt = RKVDEC_IMG_FMT_422_10BIT,
> +	},
>  };
>  
>  static const struct rkvdec_ctrl_desc rkvdec_vp9_ctrl_descs[] = {
> @@ -240,21 +253,28 @@ static const struct rkvdec_ctrls rkvdec_vp9_ctrls = {
>  	.num_ctrls = ARRAY_SIZE(rkvdec_vp9_ctrl_descs),
>  };
>  
> +static const struct rkvdec_decoded_fmt_desc rkvdec_vp9_decoded_fmts[] = {
> +	{
> +		.fourcc = V4L2_PIX_FMT_NV12,
> +		.image_fmt = RKVDEC_IMG_FMT_420_8BIT,
> +	},
> +};
> +
>  static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
>  	{
>  		.fourcc = V4L2_PIX_FMT_H264_SLICE,
>  		.frmsize = {
> -			.min_width = 48,
> +			.min_width = 64,
>  			.max_width = 4096,
> -			.step_width = 16,
> +			.step_width = 64,
>  			.min_height = 48,
>  			.max_height = 2560,
>  			.step_height = 16,
>  		},
>  		.ctrls = &rkvdec_h264_ctrls,
>  		.ops = &rkvdec_h264_fmt_ops,
> -		.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
> -		.decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
> +		.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts),
> +		.decoded_fmts = rkvdec_h264_decoded_fmts,
>  		.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
>  	},
>  	{
> @@ -269,8 +289,8 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
>  		},
>  		.ctrls = &rkvdec_vp9_ctrls,
>  		.ops = &rkvdec_vp9_fmt_ops,
> -		.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
> -		.decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
> +		.num_decoded_fmts = ARRAY_SIZE(rkvdec_vp9_decoded_fmts),
> +		.decoded_fmts = rkvdec_vp9_decoded_fmts,
>  	}
>  };
>  
> diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
> index e466a2753ccf..9a9f4fced7a1 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.h
> +++ b/drivers/staging/media/rkvdec/rkvdec.h
> @@ -80,6 +80,9 @@ struct rkvdec_coded_fmt_ops {
>  enum rkvdec_image_fmt {
>  	RKVDEC_IMG_FMT_ANY = 0,
>  	RKVDEC_IMG_FMT_420_8BIT,
> +	RKVDEC_IMG_FMT_420_10BIT,
> +	RKVDEC_IMG_FMT_422_8BIT,
> +	RKVDEC_IMG_FMT_422_10BIT,
>  };
>  
>  struct rkvdec_decoded_fmt_desc {

Re: [PATCH v6 11/11] media: rkvdec: Fix enumerate frame sizes

[Nicolas Dufresne]

Le lundi 09 septembre 2024 à 19:25 +0000, Jonas Karlman a écrit :
> The VIDIOC_ENUM_FRAMESIZES ioctl should return all frame sizes (i. e.
> width and height in pixels) that the device supports for the given pixel
> format.
> 
> For coded format returning the frame size used to enforce HW alignment
> requirements for CAPTURE buffers does not make fully sense.
> 
> Instead, signal applications what the maximum frame size that is
> supported by the HW decoder using a frame size of continuous type.
> 
> Fixes: cd33c830448b ("media: rkvdec: Add the rkvdec driver")
> Suggested-by: Alex Bee <knaerzche@gmail.com>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>

Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>

> ---
> v6:
> - New patch
> 
> With this change FFmpeg V4L2 Request API hwaccels can implement a strict
> check if frame size is supported by the video device:
> https://ffmpeg.org/pipermail/ffmpeg-devel/2024-August/332037.html
> ---
>  drivers/staging/media/rkvdec/rkvdec.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index c8c14f35ac44..9002eb3a59e5 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -334,8 +334,14 @@ static int rkvdec_enum_framesizes(struct file *file, void *priv,
>  	if (!fmt)
>  		return -EINVAL;
>  
> -	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> -	fsize->stepwise = fmt->frmsize;
> +	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> +	fsize->stepwise.min_width = 1;
> +	fsize->stepwise.max_width = fmt->frmsize.max_width;
> +	fsize->stepwise.step_width = 1;
> +	fsize->stepwise.min_height = 1;
> +	fsize->stepwise.max_height = fmt->frmsize.max_height;
> +	fsize->stepwise.step_height = 1;
> +
>  	return 0;
>  }
>  

Re: [PATCH v6 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

[Nicolas Dufresne]

Le vendredi 25 octobre 2024 à 08:54 -0400, Nicolas Dufresne a écrit :
> Le vendredi 25 octobre 2024 à 12:30 +0200, Sebastian Fricke a écrit :
> > Hey Jonas,
> > 
> > On 25.10.2024 10:20, Jonas Karlman wrote:
> > > Hi Sebastian,
> > > 
> > > Will you have time to look at this series any time soon?
> > > 
> > > Would like to send a v2 of the now one year old rkvdec hevc series but
> > > this series is sort of holding that back ;-)
> > 
> > Sorry for the delay we (maintainer & reviewer from Collabora) are
> > currently a bit busy, this is on top of our review list however, so
> > please another 1 or 2 weeks of patience :).
> 
> Actually, I'll go over it today.

I'm done, there is only cosmetic comment, if you respin quickly, I'm sure we can
get this one in soon.

> 
> regards,
> Nicolas
> 
> > 
> > > 
> > > Regards,
> > > Jonas
> > 
> > Thanks for your work and regards!
> > Sebastian
> > 
> > > 
> > > On 2024-09-09 21:24, Jonas Karlman wrote:
> > > > This series add H.264 High 10 and 4:2:2 profile support to the Rockchip
> > > > Video Decoder driver.
> > > > 
> > > > Patch 1 add helpers for calculating plane bytesperline and sizeimage.
> > > > Patch 2 add two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.
> > > > 
> > > > Patch 3 change to use bytesperline and buffer height to configure strides.
> > > > Patch 4 change to use values from SPS/PPS control to configure the HW.
> > > > 
> > > > Patch 5-9 refactor code to support filtering of CAPUTRE formats based
> > > > on the image format returned from a get_image_fmt ops.
> > > > 
> > > > Patch 10 add final bits to support H.264 High 10 and 4:2:2 profiles.
> > > > 
> > > > Patch 11 add a fix for enumerated frame sizes returned to userspace.
> > > > 
> > > > Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
> > > > 
> > > >   v4l2-compliance 1.28.1, 64 bits, 64-bit time_t
> > > >   ...
> > > >   Total for rkvdec device /dev/video1: 48, Succeeded: 48, Failed: 0, Warnings: 0
> > > > 
> > > >   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
> > > >   ...
> > > >   Ran 65/69 tests successfully
> > > > 
> > > >   Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-v4l2request
> > > >   ...
> > > >   Ran 129/135 tests successfully
> > > > 
> > > > Before this series:
> > > > 
> > > >   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
> > > >   ...
> > > >   Ran 44/69 tests successfully
> > > > 
> > > > Changes in v6:
> > > > - Change to use fmt_idx instead of j++ tucked inside a condition (Dan)
> > > > - Add patch to fix enumerated frame sizes returned to userspace (Alex)
> > > > - Fluster test score is same as v4 and v5, see [4] and [5]
> > > > Link to v5: https://lore.kernel.org/linux-media/20240618194647.742037-1-jonas@kwiboo.se/
> > > > 
> > > > Changes in v5:
> > > > - Drop Remove SPS validation at streaming start patch
> > > > - Move buffer align from rkvdec_fill_decoded_pixfmt to min/step_width
> > > > - Use correct profiles for V4L2_CID_MPEG_VIDEO_H264_PROFILE
> > > > - Collect r-b and t-b tags
> > > > - Fluster test score is same as v4, see [4] and [5]
> > > > Link to v4: https://lore.kernel.org/linux-media/20231105165521.3592037-1-jonas@kwiboo.se/
> > > > 
> > > > Changes in v4:
> > > > - Fix failed v4l2-compliance tests related to CAPTURE queue
> > > > - Rework CAPTURE format filter anv validate to use an image format
> > > > - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
> > > > Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/
> > > > 
> > > > Changes in v3:
> > > > - Drop merged patches
> > > > - Use bpp and bpp_div instead of prior misuse of block_w/block_h
> > > > - New patch to use values from SPS/PPS control to configure the HW
> > > > - New patch to remove an unnecessary call to validate sps at streaming start
> > > > - Reworked pixel format validation
> > > > Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/
> > > > 
> > > > Changes in v2:
> > > > - Collect r-b tags
> > > > - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
> > > > - New patch to not override output buffer sizeimage
> > > > - Reworked pixel format validation
> > > > - Only align decoded buffer instead of changing frmsize step_width
> > > > Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/
> > > > 
> > > > To fully runtime test this series you may need FFmpeg patches from [1]
> > > > and fluster patches from [2], this series is also available at [3].
> > > > 
> > > > [1] https://github.com/Kwiboo/FFmpeg/commits/v4l2request-2024-v2-rkvdec/
> > > > [2] https://github.com/Kwiboo/fluster/commits/ffmpeg-v4l2request-rkvdec/
> > > > [3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v6/
> > > > [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
> > > > [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf
> > > > 
> > > > Regards,
> > > > Jonas
> > > > 
> > > > Alex Bee (1):
> > > >   media: rkvdec: h264: Don't hardcode SPS/PPS parameters
> > > > 
> > > > Jonas Karlman (10):
> > > >   media: v4l2-common: Add helpers to calculate bytesperline and
> > > >     sizeimage
> > > >   media: v4l2: Add NV15 and NV20 pixel formats
> > > >   media: rkvdec: h264: Use bytesperline and buffer height as virstride
> > > >   media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
> > > >   media: rkvdec: Move rkvdec_reset_decoded_fmt helper
> > > >   media: rkvdec: Extract decoded format enumeration into helper
> > > >   media: rkvdec: Add image format concept
> > > >   media: rkvdec: Add get_image_fmt ops
> > > >   media: rkvdec: h264: Support High 10 and 4:2:2 profiles
> > > >   media: rkvdec: Fix enumerate frame sizes
> > > > 
> > > >  .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++
> > > >  drivers/media/v4l2-core/v4l2-common.c         |  80 +++---
> > > >  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
> > > >  drivers/staging/media/rkvdec/rkvdec-h264.c    |  64 +++--
> > > >  drivers/staging/media/rkvdec/rkvdec.c         | 239 +++++++++++++-----
> > > >  drivers/staging/media/rkvdec/rkvdec.h         |  18 +-
> > > >  include/uapi/linux/videodev2.h                |   2 +
> > > >  7 files changed, 410 insertions(+), 123 deletions(-)
> > > > 
> > > 
> > > 
> > Sebastian Fricke
> > Consultant Software Engineer
> > 
> > Collabora Ltd
> > Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, UK
> > Registered in England & Wales no 5513718.
> 
> 

Re: [PATCH v6 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

[Jonas Karlman]

On 2024-10-25 19:47, Nicolas Dufresne wrote:
> Le vendredi 25 octobre 2024 à 08:54 -0400, Nicolas Dufresne a écrit :
>> Le vendredi 25 octobre 2024 à 12:30 +0200, Sebastian Fricke a écrit :
>>> Hey Jonas,
>>>
>>> On 25.10.2024 10:20, Jonas Karlman wrote:
>>>> Hi Sebastian,
>>>>
>>>> Will you have time to look at this series any time soon?
>>>>
>>>> Would like to send a v2 of the now one year old rkvdec hevc series but
>>>> this series is sort of holding that back ;-)
>>>
>>> Sorry for the delay we (maintainer & reviewer from Collabora) are
>>> currently a bit busy, this is on top of our review list however, so
>>> please another 1 or 2 weeks of patience :).
>>
>> Actually, I'll go over it today.
> 
> I'm done, there is only cosmetic comment, if you respin quickly, I'm sure we can
> get this one in soon.

Thanks!, I will respin a v7 with the nits fixed later this weekend :-)

Regards,
Jonas

> 
>>
>> regards,
>> Nicolas
>>
>>>
>>>>
>>>> Regards,
>>>> Jonas
>>>
>>> Thanks for your work and regards!
>>> Sebastian
>>>
>>>>
>>>> On 2024-09-09 21:24, Jonas Karlman wrote:
>>>>> This series add H.264 High 10 and 4:2:2 profile support to the Rockchip
>>>>> Video Decoder driver.
>>>>>
>>>>> Patch 1 add helpers for calculating plane bytesperline and sizeimage.
>>>>> Patch 2 add two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.
>>>>>
>>>>> Patch 3 change to use bytesperline and buffer height to configure strides.
>>>>> Patch 4 change to use values from SPS/PPS control to configure the HW.
>>>>>
>>>>> Patch 5-9 refactor code to support filtering of CAPUTRE formats based
>>>>> on the image format returned from a get_image_fmt ops.
>>>>>
>>>>> Patch 10 add final bits to support H.264 High 10 and 4:2:2 profiles.
>>>>>
>>>>> Patch 11 add a fix for enumerated frame sizes returned to userspace.
>>>>>
>>>>> Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):
>>>>>
>>>>>   v4l2-compliance 1.28.1, 64 bits, 64-bit time_t
>>>>>   ...
>>>>>   Total for rkvdec device /dev/video1: 48, Succeeded: 48, Failed: 0, Warnings: 0
>>>>>
>>>>>   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
>>>>>   ...
>>>>>   Ran 65/69 tests successfully
>>>>>
>>>>>   Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-v4l2request
>>>>>   ...
>>>>>   Ran 129/135 tests successfully
>>>>>
>>>>> Before this series:
>>>>>
>>>>>   Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-v4l2request
>>>>>   ...
>>>>>   Ran 44/69 tests successfully
>>>>>
>>>>> Changes in v6:
>>>>> - Change to use fmt_idx instead of j++ tucked inside a condition (Dan)
>>>>> - Add patch to fix enumerated frame sizes returned to userspace (Alex)
>>>>> - Fluster test score is same as v4 and v5, see [4] and [5]
>>>>> Link to v5: https://lore.kernel.org/linux-media/20240618194647.742037-1-jonas@kwiboo.se/
>>>>>
>>>>> Changes in v5:
>>>>> - Drop Remove SPS validation at streaming start patch
>>>>> - Move buffer align from rkvdec_fill_decoded_pixfmt to min/step_width
>>>>> - Use correct profiles for V4L2_CID_MPEG_VIDEO_H264_PROFILE
>>>>> - Collect r-b and t-b tags
>>>>> - Fluster test score is same as v4, see [4] and [5]
>>>>> Link to v4: https://lore.kernel.org/linux-media/20231105165521.3592037-1-jonas@kwiboo.se/
>>>>>
>>>>> Changes in v4:
>>>>> - Fix failed v4l2-compliance tests related to CAPTURE queue
>>>>> - Rework CAPTURE format filter anv validate to use an image format
>>>>> - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
>>>>> Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/
>>>>>
>>>>> Changes in v3:
>>>>> - Drop merged patches
>>>>> - Use bpp and bpp_div instead of prior misuse of block_w/block_h
>>>>> - New patch to use values from SPS/PPS control to configure the HW
>>>>> - New patch to remove an unnecessary call to validate sps at streaming start
>>>>> - Reworked pixel format validation
>>>>> Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/
>>>>>
>>>>> Changes in v2:
>>>>> - Collect r-b tags
>>>>> - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
>>>>> - New patch to not override output buffer sizeimage
>>>>> - Reworked pixel format validation
>>>>> - Only align decoded buffer instead of changing frmsize step_width
>>>>> Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/
>>>>>
>>>>> To fully runtime test this series you may need FFmpeg patches from [1]
>>>>> and fluster patches from [2], this series is also available at [3].
>>>>>
>>>>> [1] https://github.com/Kwiboo/FFmpeg/commits/v4l2request-2024-v2-rkvdec/
>>>>> [2] https://github.com/Kwiboo/fluster/commits/ffmpeg-v4l2request-rkvdec/
>>>>> [3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v6/
>>>>> [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
>>>>> [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf
>>>>>
>>>>> Regards,
>>>>> Jonas
>>>>>
>>>>> Alex Bee (1):
>>>>>   media: rkvdec: h264: Don't hardcode SPS/PPS parameters
>>>>>
>>>>> Jonas Karlman (10):
>>>>>   media: v4l2-common: Add helpers to calculate bytesperline and
>>>>>     sizeimage
>>>>>   media: v4l2: Add NV15 and NV20 pixel formats
>>>>>   media: rkvdec: h264: Use bytesperline and buffer height as virstride
>>>>>   media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
>>>>>   media: rkvdec: Move rkvdec_reset_decoded_fmt helper
>>>>>   media: rkvdec: Extract decoded format enumeration into helper
>>>>>   media: rkvdec: Add image format concept
>>>>>   media: rkvdec: Add get_image_fmt ops
>>>>>   media: rkvdec: h264: Support High 10 and 4:2:2 profiles
>>>>>   media: rkvdec: Fix enumerate frame sizes
>>>>>
>>>>>  .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++
>>>>>  drivers/media/v4l2-core/v4l2-common.c         |  80 +++---
>>>>>  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
>>>>>  drivers/staging/media/rkvdec/rkvdec-h264.c    |  64 +++--
>>>>>  drivers/staging/media/rkvdec/rkvdec.c         | 239 +++++++++++++-----
>>>>>  drivers/staging/media/rkvdec/rkvdec.h         |  18 +-
>>>>>  include/uapi/linux/videodev2.h                |   2 +
>>>>>  7 files changed, 410 insertions(+), 123 deletions(-)
>>>>>
>>>>
>>>>
>>> Sebastian Fricke
>>> Consultant Software Engineer
>>>
>>> Collabora Ltd
>>> Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, UK
>>> Registered in England & Wales no 5513718.
>>
>>
> 

Re: [PATCH v6 00/11] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

[Nicolas Dufresne]

Hi Jonas,

Le vendredi 25 octobre 2024 à 22:13 +0200, Jonas Karlman a écrit :
> On 2024-10-25 19:47, Nicolas Dufresne wrote:
> > Le vendredi 25 octobre 2024 à 08:54 -0400, Nicolas Dufresne a écrit :
> > > Le vendredi 25 octobre 2024 à 12:30 +0200, Sebastian Fricke a écrit :
> > > > Hey Jonas,
> > > > 

[...]

> > 
> > I'm done, there is only cosmetic comment, if you respin quickly, I'm sure we can
> > get this one in soon.
> 
> Thanks!, I will respin a v7 with the nits fixed later this weekend :-)

I simply wanted to check with you if everything was going well with this V7.
Feel free to ping for help.

Nicolas