[PATCH v2 0/3] media: rockchip: rkvdec: add support for the VDPU346 variant

[PATCH v2 0/3] media: rockchip: rkvdec: add support for the VDPU346 variant

[Christian Hewitt]

This series depends upon Detlev Casanova's current v7 series for VDPU381 and
VDPU383 support [0]. It adds support for the VDPU346 IP block used for H264,
HEVC and (in active work) VP9 on the RK356X boards. VDPU346 appears to be a
close relation to VDPU381 used with RK3588, except with a single core, output
limited to 4K, and minor feature differences, e.g. HEVC level 5.1 on VDPU346
vs 6.1 on VDPU381. To handle differences we declare a new compatible.

The device-tree changes are derived from prior-art in Detlev's patches for
RK3576 and RK3588, the vendor kernel [1], and some earlier patch attempts
from Piotr Oniszczuk and Diederik de Haas.

The register differences between VDPU346 and VDPU381 appear to be minimal so
we currently and intentionally reuse the VDPU381 h264, hevc, and regs files
to avoid duplicating code. There is some evidence of buffer size changes in
testing which still need to be explored. If further register analysis needs
separate files for VDPU346 the refactoring will be beyond my current n00b
coding skills and I will either need some coaching or would be happy to pass
the series over to a more experienced developer.

NB: Testing with the v1 series showed lower mbps bitrate performance. This
appears to be resolved though it's unclear to me whether this results from
kernel changes or the ongoing reworking of ffmpeg v4l2_request support [2].
However with my current Linux 6.19-rc2 test branch [3] I'm now able to play
Jellyfish H264 and HEVC test media over 100mbps.

Changes since v1:
- Drop DO-NOT-MERGE patch as changes included in Detvlev v7 series
- Adapt to variant/capability changes in Detlev's v7 series
- Add bindings ack from Rob
- Add rkvdec ack from Nicolas
- Rebase against Linux 6.19-rc2

[0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=1034794
[1] https://github.com/rockchip-linux/kernel/blob/develop-6.6/arch/arm64/boot/dts/rockchip/rk356x.dtsi#L1539
[2] https://code.ffmpeg.org/Kwiboo/FFmpeg/commits/branch/v4l2request-v3
[3] https://github.com/chewitt/linux/commits/rockchip-6.19.y

Christian Hewitt (3):
  media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings
  media: rkvdec: Add support for the VDPU346 variant
  arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356X

 .../bindings/media/rockchip,vdec.yaml         |   2 +
 arch/arm64/boot/dts/rockchip/rk356x-base.dtsi |  49 +++++++++
 .../media/platform/rockchip/rkvdec/rkvdec.c   | 103 ++++++++++++++++++
 3 files changed, 154 insertions(+)

-- 
2.34.1

[PATCH v2 1/3] media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings

[Christian Hewitt]

The video decoder in RK356X (vdpu346) is described in the same way as
the one in RK3588 (vdpu381). A new compatible is added as the decoder
capabilities are a subset of the vdpu381 capabilities.

Signed-off-by: Christian Hewitt <christianshewitt@gmail.com>
Acked-by: Rob Herring (Arm) <robh@kernel.org>
---
 Documentation/devicetree/bindings/media/rockchip,vdec.yaml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Documentation/devicetree/bindings/media/rockchip,vdec.yaml b/Documentation/devicetree/bindings/media/rockchip,vdec.yaml
index 809fda45b3bd..656ceb1f116e 100644
--- a/Documentation/devicetree/bindings/media/rockchip,vdec.yaml
+++ b/Documentation/devicetree/bindings/media/rockchip,vdec.yaml
@@ -18,6 +18,7 @@ properties:
     oneOf:
       - const: rockchip,rk3288-vdec
       - const: rockchip,rk3399-vdec
+      - const: rockchip,rk3568-vdec
       - const: rockchip,rk3576-vdec
       - const: rockchip,rk3588-vdec
       - items:
@@ -107,6 +108,7 @@ allOf:
         compatible:
           contains:
             enum:
+              - rockchip,rk3568-vdec
               - rockchip,rk3576-vdec
               - rockchip,rk3588-vdec
     then:
-- 
2.34.1

[PATCH v2 2/3] media: rkvdec: Add support for the VDPU346 variant

[Christian Hewitt]

VDPU346 is similar to VDPU381 but with a single core and limited
to 4K60 media. It is also limited to H264 L5.1 and omits AV1 and
AVS2 capabilities. VDPU346 is used with RK3566 and RK3568.

Signed-off-by: Christian Hewitt <christianshewitt@gmail.com>
Reviewed-by: Nicolas Dufresne <nicolas.collabora@collabora.com>
---
 .../media/platform/rockchip/rkvdec/rkvdec.c   | 103 ++++++++++++++++++
 1 file changed, 103 insertions(+)

diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec.c b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
index e547057dc75f..6b39e99d8a8b 100644
--- a/drivers/media/platform/rockchip/rkvdec/rkvdec.c
+++ b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
@@ -236,6 +236,62 @@ static const struct rkvdec_ctrls rkvdec_hevc_ctrls = {
 	.num_ctrls = ARRAY_SIZE(rkvdec_hevc_ctrl_descs),
 };
 
+static const struct rkvdec_ctrl_desc vdpu346_hevc_ctrl_descs[] = {
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_SPS,
+		.cfg.ops = &rkvdec_ctrl_ops,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_PPS,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
+		.cfg.min = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+		.cfg.max = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+		.cfg.def = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_START_CODE,
+		.cfg.min = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+		.cfg.def = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+		.cfg.max = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+	},
+	{
+		.cfg.id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
+		.cfg.min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+		.cfg.max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
+		.cfg.menu_skip_mask =
+			BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE),
+		.cfg.def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+	},
+	{
+		.cfg.id = V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
+		.cfg.min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
+		.cfg.max = V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1,
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS,
+		.cfg.ops = &rkvdec_ctrl_ops,
+		.cfg.dims = { 65 },
+	},
+	{
+		.cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS,
+		.cfg.ops = &rkvdec_ctrl_ops,
+		.cfg.dims = { 65 },
+	},
+};
+
+static const struct rkvdec_ctrls vdpu346_hevc_ctrls = {
+	.ctrls = vdpu346_hevc_ctrl_descs,
+	.num_ctrls = ARRAY_SIZE(vdpu346_hevc_ctrl_descs),
+};
+
 static const struct rkvdec_ctrl_desc vdpu38x_hevc_ctrl_descs[] = {
 	{
 		.cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
@@ -463,6 +519,41 @@ static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = {
 	}
 };
 
+static const struct rkvdec_coded_fmt_desc vdpu346_coded_fmts[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_HEVC_SLICE,
+		.frmsize = {
+			.min_width = 64,
+			.max_width = 65472,
+			.step_width = 64,
+			.min_height = 64,
+			.max_height = 65472,
+			.step_height = 16,
+		},
+		.ctrls = &vdpu346_hevc_ctrls,
+		.ops = &rkvdec_vdpu381_hevc_fmt_ops,
+		.num_decoded_fmts = ARRAY_SIZE(rkvdec_hevc_decoded_fmts),
+		.decoded_fmts = rkvdec_hevc_decoded_fmts,
+		.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_H264_SLICE,
+		.frmsize = {
+			.min_width = 64,
+			.max_width =  65520,
+			.step_width = 64,
+			.min_height = 64,
+			.max_height =  65520,
+			.step_height = 16,
+		},
+		.ctrls = &rkvdec_h264_ctrls,
+		.ops = &rkvdec_vdpu381_h264_fmt_ops,
+		.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts),
+		.decoded_fmts = rkvdec_h264_decoded_fmts,
+		.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
+	},
+};
+
 static const struct rkvdec_coded_fmt_desc vdpu381_coded_fmts[] = {
 	{
 		.fourcc = V4L2_PIX_FMT_HEVC_SLICE,
@@ -1643,6 +1734,14 @@ static const struct rkvdec_variant_ops vdpu381_variant_ops = {
 	.flatten_matrices = transpose_and_flatten_matrices,
 };
 
+static const struct rkvdec_variant vdpu346_variant = {
+	.coded_fmts = vdpu346_coded_fmts,
+	.num_coded_fmts = ARRAY_SIZE(vdpu346_coded_fmts),
+	.rcb_sizes = vdpu381_rcb_sizes,
+	.num_rcb_sizes = ARRAY_SIZE(vdpu381_rcb_sizes),
+	.ops = &vdpu381_variant_ops,
+};
+
 static const struct rkvdec_variant vdpu381_variant = {
 	.coded_fmts = vdpu381_coded_fmts,
 	.num_coded_fmts = ARRAY_SIZE(vdpu381_coded_fmts),
@@ -1691,6 +1790,10 @@ static const struct of_device_id of_rkvdec_match[] = {
 		.compatible = "rockchip,rk3399-vdec",
 		.data = &rk3399_rkvdec_variant,
 	},
+	{
+		.compatible = "rockchip,rk3568-vdec",
+		.data = &vdpu346_variant,
+	},
 	{
 		.compatible = "rockchip,rk3588-vdec",
 		.data = &vdpu381_variant,
-- 
2.34.1

[PATCH v2 3/3] arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356X

[Christian Hewitt]

Add the vdpu346 Video Decoders to the rk356x-base devicetree to
enable support on RK3566 and RK3568 boards. Also add the needed
sram and vdec_mmu nodes.

Suggested-by: Diederik de Haas <didi.debian@cknow.org>
Suggested-by: Piotr Oniszczuk <piotr.oniszczuk@gmail.com>
Signed-off-by: Christian Hewitt <christianshewitt@gmail.com>
---
 arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 49 +++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/arch/arm64/boot/dts/rockchip/rk356x-base.dtsi b/arch/arm64/boot/dts/rockchip/rk356x-base.dtsi
index 8893b7b6cc9f..b37eea56c221 100644
--- a/arch/arm64/boot/dts/rockchip/rk356x-base.dtsi
+++ b/arch/arm64/boot/dts/rockchip/rk356x-base.dtsi
@@ -383,6 +383,19 @@ usb2phy1_grf: syscon@fdca8000 {
 		reg = <0x0 0xfdca8000 0x0 0x8000>;
 	};
 
+	sram@fdcc0000 {
+		compatible = "mmio-sram";
+		reg = <0x0 0xfdcc0000 0x0 0xb000>;
+		ranges = <0x0 0x0 0xfdcc0000 0xb000>;
+		#address-cells = <1>;
+		#size-cells = <1>;
+
+		vdec_sram: rkvdec-sram@0 {
+			reg = <0x0 0xb000>;
+			pool;
+		};
+	};
+
 	pmucru: clock-controller@fdd00000 {
 		compatible = "rockchip,rk3568-pmucru";
 		reg = <0x0 0xfdd00000 0x0 0x1000>;
@@ -619,6 +632,42 @@ vepu_mmu: iommu@fdee0800 {
 		#iommu-cells = <0>;
 	};
 
+	vdec: video-codec@fdf80100 {
+		compatible = "rockchip,rk3568-vdec";
+		reg = <0x0 0xfdf80200 0x0 0x500>,
+		      <0x0 0xfdf80100 0x0 0x100>,
+		      <0x0 0xfdf80700 0x0 0x100>;
+		reg-names = "function", "link", "cache";
+		interrupts = <GIC_SPI 91 IRQ_TYPE_LEVEL_HIGH>;
+		clocks = <&cru ACLK_RKVDEC>, <&cru HCLK_RKVDEC>,
+			 <&cru CLK_RKVDEC_CA>, <&cru CLK_RKVDEC_CORE>,
+			 <&cru CLK_RKVDEC_HEVC_CA>;
+		clock-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
+		assigned-clocks = <&cru ACLK_RKVDEC>,
+				  <&cru CLK_RKVDEC_CORE>,
+				  <&cru CLK_RKVDEC_CA>,
+				  <&cru CLK_RKVDEC_HEVC_CA>;
+		assigned-clock-rates = <297000000>, <297000000>,
+				       <297000000>, <600000000>;
+		iommus = <&vdec_mmu>;
+		power-domains = <&power RK3568_PD_RKVDEC>;
+		resets = <&cru SRST_A_RKVDEC>, <&cru SRST_H_RKVDEC>,
+			 <&cru SRST_RKVDEC_CA>, <&cru SRST_RKVDEC_CORE>,
+			 <&cru SRST_RKVDEC_HEVC_CA>;
+		reset-names = "axi", "ahb", "cabac", "core", "hevc_cabac";
+		sram = <&vdec_sram>;
+	};
+
+	vdec_mmu: iommu@fdf80800 {
+		compatible = "rockchip,rk3568-iommu";
+		reg = <0x0 0xfdf80800 0x0 0x40>, <0x0 0xfdf80840 0x0 0x40>;
+		interrupts = <GIC_SPI 92 IRQ_TYPE_LEVEL_HIGH>;
+		clocks = <&cru ACLK_RKVDEC>, <&cru HCLK_RKVDEC>;
+		clock-names = "aclk", "iface";
+		power-domains = <&power RK3568_PD_RKVDEC>;
+		#iommu-cells = <0>;
+	};
+
 	vicap: video-capture@fdfe0000 {
 		compatible = "rockchip,rk3568-vicap";
 		reg = <0x0 0xfdfe0000 0x0 0x200>;
-- 
2.34.1

Re: [PATCH v2 2/3] media: rkvdec: Add support for the VDPU346 variant

[Diederik de Haas]

Hi Christian,

On Fri Dec 26, 2025 at 12:31 PM CET, Christian Hewitt wrote:
> VDPU346 is similar to VDPU381 but with a single core and limited
> to 4K60 media. It is also limited to H264 L5.1 and omits AV1 and
> AVS2 capabilities. VDPU346 is used with RK3566 and RK3568.
>
> Signed-off-by: Christian Hewitt <christianshewitt@gmail.com>
> Reviewed-by: Nicolas Dufresne <nicolas.collabora@collabora.com>
> ---
>  .../media/platform/rockchip/rkvdec/rkvdec.c   | 103 ++++++++++++++++++
>  1 file changed, 103 insertions(+)
>
> diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec.c b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
> index e547057dc75f..6b39e99d8a8b 100644
> --- a/drivers/media/platform/rockchip/rkvdec/rkvdec.c
> +++ b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
> @@ -236,6 +236,62 @@ static const struct rkvdec_ctrls rkvdec_hevc_ctrls = {
>  	.num_ctrls = ARRAY_SIZE(rkvdec_hevc_ctrl_descs),
>  };
>  
> +static const struct rkvdec_ctrl_desc vdpu346_hevc_ctrl_descs[] = {
> +	{
> +		.cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
> +	},
> +	{
> +		.cfg.id = V4L2_CID_STATELESS_HEVC_SPS,
> +		.cfg.ops = &rkvdec_ctrl_ops,
> +	},
> +	{
> +		.cfg.id = V4L2_CID_STATELESS_HEVC_PPS,
> +	},
> +	{
> +		.cfg.id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
> +	},
> +	{
> +		.cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
> +		.cfg.min = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> +		.cfg.max = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> +		.cfg.def = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> +	},
> +	{
> +		.cfg.id = V4L2_CID_STATELESS_HEVC_START_CODE,
> +		.cfg.min = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> +		.cfg.def = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> +		.cfg.max = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> +	},
> +	{
> +		.cfg.id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
> +		.cfg.min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> +		.cfg.max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
> +		.cfg.menu_skip_mask =
> +			BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE),
> +		.cfg.def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> +	},
> +	{
> +		.cfg.id = V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
> +		.cfg.min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
> +		.cfg.max = V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1,
> +	},
> +	{
> +		.cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS,
> +		.cfg.ops = &rkvdec_ctrl_ops,
> +		.cfg.dims = { 65 },
> +	},
> +	{
> +		.cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS,
> +		.cfg.ops = &rkvdec_ctrl_ops,
> +		.cfg.dims = { 65 },
> +	},
> +};
> +
> +static const struct rkvdec_ctrls vdpu346_hevc_ctrls = {
> +	.ctrls = vdpu346_hevc_ctrl_descs,
> +	.num_ctrls = ARRAY_SIZE(vdpu346_hevc_ctrl_descs),
> +};
> +
>  static const struct rkvdec_ctrl_desc vdpu38x_hevc_ctrl_descs[] = {
>  	{
>  		.cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
> @@ -463,6 +519,41 @@ static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = {
>  	}
>  };
>  
> +static const struct rkvdec_coded_fmt_desc vdpu346_coded_fmts[] = {
> +	{
> +		.fourcc = V4L2_PIX_FMT_HEVC_SLICE,
> +		.frmsize = {
> +			.min_width = 64,
> +			.max_width = 65472,

This should be 4096 according to page 469 of RK3568 TRM Part 2 ...

> +			.step_width = 64,
> +			.min_height = 64,
> +			.max_height = 65472,

... and this 2304.

> +			.step_height = 16,
> +		},
> +		.ctrls = &vdpu346_hevc_ctrls,
> +		.ops = &rkvdec_vdpu381_hevc_fmt_ops,
> +		.num_decoded_fmts = ARRAY_SIZE(rkvdec_hevc_decoded_fmts),
> +		.decoded_fmts = rkvdec_hevc_decoded_fmts,
> +		.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
> +	},
> +	{
> +		.fourcc = V4L2_PIX_FMT_H264_SLICE,
> +		.frmsize = {
> +			.min_width = 64,
> +			.max_width =  65520,

This too should be 4096 according to page 469 of RK3568 TRM Part 2 ...

> +			.step_width = 64,
> +			.min_height = 64,
> +			.max_height =  65520,

... and this 2304.

I guess this also explains the 'green images' Nicolas noticed.

> +			.step_height = 16,
> +		},
> +		.ctrls = &rkvdec_h264_ctrls,
> +		.ops = &rkvdec_vdpu381_h264_fmt_ops,
> +		.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts),
> +		.decoded_fmts = rkvdec_h264_decoded_fmts,
> +		.subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
> +	},

I see you've reversed the order of the blocks so that HEVC now comes
before the H264 block. While that makes it consistent with what Detlev
has in their v7 and with the existing code in the driver ... I actually
prefer having H264 before HEVC as the alphabetical sorting order is
H264 before HEVC.
In the existing code the VP9 'stuff' is listed below H264 and HEVC.

But then Detlev should do that too in their patch set ... and 'ideally'
the order of the existing code be updated to be alphabetically too.

OTOH, a consistent order works for me too.

Cheers,
  Diederik

> +};
> +
>  static const struct rkvdec_coded_fmt_desc vdpu381_coded_fmts[] = {
>  	{
>  		.fourcc = V4L2_PIX_FMT_HEVC_SLICE,
> @@ -1643,6 +1734,14 @@ static const struct rkvdec_variant_ops vdpu381_variant_ops = {
>  	.flatten_matrices = transpose_and_flatten_matrices,
>  };
>  
> +static const struct rkvdec_variant vdpu346_variant = {
> +	.coded_fmts = vdpu346_coded_fmts,
> +	.num_coded_fmts = ARRAY_SIZE(vdpu346_coded_fmts),
> +	.rcb_sizes = vdpu381_rcb_sizes,
> +	.num_rcb_sizes = ARRAY_SIZE(vdpu381_rcb_sizes),
> +	.ops = &vdpu381_variant_ops,
> +};
> +
>  static const struct rkvdec_variant vdpu381_variant = {
>  	.coded_fmts = vdpu381_coded_fmts,
>  	.num_coded_fmts = ARRAY_SIZE(vdpu381_coded_fmts),
> @@ -1691,6 +1790,10 @@ static const struct of_device_id of_rkvdec_match[] = {
>  		.compatible = "rockchip,rk3399-vdec",
>  		.data = &rk3399_rkvdec_variant,
>  	},
> +	{
> +		.compatible = "rockchip,rk3568-vdec",
> +		.data = &vdpu346_variant,
> +	},
>  	{
>  		.compatible = "rockchip,rk3588-vdec",
>  		.data = &vdpu381_variant,

Re: [PATCH v2 2/3] media: rkvdec: Add support for the VDPU346 variant

[Christian Hewitt]

> On 26 Dec 2025, at 4:00 pm, Diederik de Haas <diederik@cknow-tech.com> wrote:
> 
> Hi Christian,
> 
> On Fri Dec 26, 2025 at 12:31 PM CET, Christian Hewitt wrote:
>> VDPU346 is similar to VDPU381 but with a single core and limited
>> to 4K60 media. It is also limited to H264 L5.1 and omits AV1 and
>> AVS2 capabilities. VDPU346 is used with RK3566 and RK3568.
>> 
>> Signed-off-by: Christian Hewitt <christianshewitt@gmail.com>
>> Reviewed-by: Nicolas Dufresne <nicolas.collabora@collabora.com>
>> ---
>> .../media/platform/rockchip/rkvdec/rkvdec.c   | 103 ++++++++++++++++++
>> 1 file changed, 103 insertions(+)
>> 
>> diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec.c b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
>> index e547057dc75f..6b39e99d8a8b 100644
>> --- a/drivers/media/platform/rockchip/rkvdec/rkvdec.c
>> +++ b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
>> @@ -236,6 +236,62 @@ static const struct rkvdec_ctrls rkvdec_hevc_ctrls = {
>> .num_ctrls = ARRAY_SIZE(rkvdec_hevc_ctrl_descs),
>> };
>> 
>> +static const struct rkvdec_ctrl_desc vdpu346_hevc_ctrl_descs[] = {
>> + {
>> + .cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
>> + },
>> + {
>> + .cfg.id = V4L2_CID_STATELESS_HEVC_SPS,
>> + .cfg.ops = &rkvdec_ctrl_ops,
>> + },
>> + {
>> + .cfg.id = V4L2_CID_STATELESS_HEVC_PPS,
>> + },
>> + {
>> + .cfg.id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
>> + },
>> + {
>> + .cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
>> + .cfg.min = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
>> + .cfg.max = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
>> + .cfg.def = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
>> + },
>> + {
>> + .cfg.id = V4L2_CID_STATELESS_HEVC_START_CODE,
>> + .cfg.min = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
>> + .cfg.def = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
>> + .cfg.max = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
>> + },
>> + {
>> + .cfg.id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
>> + .cfg.min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
>> + .cfg.max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
>> + .cfg.menu_skip_mask =
>> + BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE),
>> + .cfg.def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
>> + },
>> + {
>> + .cfg.id = V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
>> + .cfg.min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
>> + .cfg.max = V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1,
>> + },
>> + {
>> + .cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS,
>> + .cfg.ops = &rkvdec_ctrl_ops,
>> + .cfg.dims = { 65 },
>> + },
>> + {
>> + .cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS,
>> + .cfg.ops = &rkvdec_ctrl_ops,
>> + .cfg.dims = { 65 },
>> + },
>> +};
>> +
>> +static const struct rkvdec_ctrls vdpu346_hevc_ctrls = {
>> + .ctrls = vdpu346_hevc_ctrl_descs,
>> + .num_ctrls = ARRAY_SIZE(vdpu346_hevc_ctrl_descs),
>> +};
>> +
>> static const struct rkvdec_ctrl_desc vdpu38x_hevc_ctrl_descs[] = {
>> {
>> .cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
>> @@ -463,6 +519,41 @@ static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = {
>> }
>> };
>> 
>> +static const struct rkvdec_coded_fmt_desc vdpu346_coded_fmts[] = {
>> + {
>> + .fourcc = V4L2_PIX_FMT_HEVC_SLICE,
>> + .frmsize = {
>> + .min_width = 64,
>> + .max_width = 65472,
> 
> This should be 4096 according to page 469 of RK3568 TRM Part 2 ...
> 
>> + .step_width = 64,
>> + .min_height = 64,
>> + .max_height = 65472,
> 
> ... and this 2304.
> 
>> + .step_height = 16,
>> + },
>> + .ctrls = &vdpu346_hevc_ctrls,
>> + .ops = &rkvdec_vdpu381_hevc_fmt_ops,
>> + .num_decoded_fmts = ARRAY_SIZE(rkvdec_hevc_decoded_fmts),
>> + .decoded_fmts = rkvdec_hevc_decoded_fmts,
>> + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
>> + },
>> + {
>> + .fourcc = V4L2_PIX_FMT_H264_SLICE,
>> + .frmsize = {
>> + .min_width = 64,
>> + .max_width =  65520,
> 
> This too should be 4096 according to page 469 of RK3568 TRM Part 2 ...
> 
>> + .step_width = 64,
>> + .min_height = 64,
>> + .max_height =  65520,
> 
> ... and this 2304.
> 
> I guess this also explains the 'green images' Nicolas noticed.

Quite probably. I’ve picked the above changes into my working tree
(for those following it) and will send a v3 series in response to
the next revision of Detlev’s patches.

> + .step_height = 16,
>> + },
>> + .ctrls = &rkvdec_h264_ctrls,
>> + .ops = &rkvdec_vdpu381_h264_fmt_ops,
>> + .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts),
>> + .decoded_fmts = rkvdec_h264_decoded_fmts,
>> + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
>> + },
> 
> I see you've reversed the order of the blocks so that HEVC now comes
> before the H264 block. While that makes it consistent with what Detlev
> has in their v7 and with the existing code in the driver ... I actually
> prefer having H264 before HEVC as the alphabetical sorting order is
> H264 before HEVC.
> In the existing code the VP9 'stuff' is listed below H264 and HEVC.
> 
> But then Detlev should do that too in their patch set ... and 'ideally'
> the order of the existing code be updated to be alphabetically too.
> 
> OTOH, a consistent order works for me too.

I believe the reorder was requested by Nic (offline from the list) so
there’s probably a reason behind it. I’ll keep things aligned to the
order in Detlev’s series (whatever that is).

Christian


> Cheers,
>  Diederik
> 
>> +};
>> +
>> static const struct rkvdec_coded_fmt_desc vdpu381_coded_fmts[] = {
>> {
>> .fourcc = V4L2_PIX_FMT_HEVC_SLICE,
>> @@ -1643,6 +1734,14 @@ static const struct rkvdec_variant_ops vdpu381_variant_ops = {
>> .flatten_matrices = transpose_and_flatten_matrices,
>> };
>> 
>> +static const struct rkvdec_variant vdpu346_variant = {
>> + .coded_fmts = vdpu346_coded_fmts,
>> + .num_coded_fmts = ARRAY_SIZE(vdpu346_coded_fmts),
>> + .rcb_sizes = vdpu381_rcb_sizes,
>> + .num_rcb_sizes = ARRAY_SIZE(vdpu381_rcb_sizes),
>> + .ops = &vdpu381_variant_ops,
>> +};
>> +
>> static const struct rkvdec_variant vdpu381_variant = {
>> .coded_fmts = vdpu381_coded_fmts,
>> .num_coded_fmts = ARRAY_SIZE(vdpu381_coded_fmts),
>> @@ -1691,6 +1790,10 @@ static const struct of_device_id of_rkvdec_match[] = {
>> .compatible = "rockchip,rk3399-vdec",
>> .data = &rk3399_rkvdec_variant,
>> },
>> + {
>> + .compatible = "rockchip,rk3568-vdec",
>> + .data = &vdpu346_variant,
>> + },
>> {
>> .compatible = "rockchip,rk3588-vdec",
>> .data = &vdpu381_variant,

Re: [PATCH v2 0/3] media: rockchip: rkvdec: add support for the VDPU346 variant

[Dang Huynh]

Hi Christian,

On Friday, December 26, 2025 11:31:37 AM Coordinated Universal Time Christian 
Hewitt wrote:
> This series depends upon Detlev Casanova's current v7 series for VDPU381 and
> VDPU383 support [0]. It adds support for the VDPU346 IP block used for
> H264, HEVC and (in active work) VP9 on the RK356X boards. VDPU346 appears
> to be a close relation to VDPU381 used with RK3588, except with a single
> core, output limited to 4K, and minor feature differences, e.g. HEVC level
> 5.1 on VDPU346 vs 6.1 on VDPU381. To handle differences we declare a new
> compatible.
> 
> The device-tree changes are derived from prior-art in Detlev's patches for
> RK3576 and RK3588, the vendor kernel [1], and some earlier patch attempts
> from Piotr Oniszczuk and Diederik de Haas.
> 
> The register differences between VDPU346 and VDPU381 appear to be minimal so
> we currently and intentionally reuse the VDPU381 h264, hevc, and regs files
> to avoid duplicating code. There is some evidence of buffer size changes in
> testing which still need to be explored. If further register analysis needs
> separate files for VDPU346 the refactoring will be beyond my current n00b
> coding skills and I will either need some coaching or would be happy to
> pass the series over to a more experienced developer.
> 
> NB: Testing with the v1 series showed lower mbps bitrate performance. This
> appears to be resolved though it's unclear to me whether this results from
> kernel changes or the ongoing reworking of ffmpeg v4l2_request support [2].
> However with my current Linux 6.19-rc2 test branch [3] I'm now able to play
> Jellyfish H264 and HEVC test media over 100mbps.
> 
> Changes since v1:
> - Drop DO-NOT-MERGE patch as changes included in Detvlev v7 series
> - Adapt to variant/capability changes in Detlev's v7 series
> - Add bindings ack from Rob
> - Add rkvdec ack from Nicolas
> - Rebase against Linux 6.19-rc2
> 
> [0] https://patchwork.kernel.org/project/linux-rockchip/list/?series=1034794
> [1]
> https://github.com/rockchip-linux/kernel/blob/develop-6.6/arch/arm64/boot/d
> ts/rockchip/rk356x.dtsi#L1539 [2]
> https://code.ffmpeg.org/Kwiboo/FFmpeg/commits/branch/v4l2request-v3 [3]
> https://github.com/chewitt/linux/commits/rockchip-6.19.y
> 
> Christian Hewitt (3):
>   media: dt-bindings: rockchip: Add RK3568 Video Decoder bindings
>   media: rkvdec: Add support for the VDPU346 variant
>   arm64: dts: rockchip: Add the vdpu346 Video Decoders on RK356X
> 
>  .../bindings/media/rockchip,vdec.yaml         |   2 +
>  arch/arm64/boot/dts/rockchip/rk356x-base.dtsi |  49 +++++++++
>  .../media/platform/rockchip/rkvdec/rkvdec.c   | 103 ++++++++++++++++++
>  3 files changed, 154 insertions(+)

I tested this patch series on my RK3566 board with my own test videos and BBB 
(1080p60 and 4k30) using mpv with [1] patchset.

The command I used was:
`mpv --vo=dmabuf-wayland --hwdec=v4l2request input_file.mp4`

Both H264 and HEVC plays with minor framedrops (which to me isn't noticeable).

You can add my Tested-by:
Tested-by: Dang Huynh <dang.huynh@mainlining.org>

[1]: https://github.com/mpv-player/mpv/pull/14690

Best regards,
Dang

Re: [PATCH v2 2/3] media: rkvdec: Add support for the VDPU346 variant

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 7306 bytes --]

Le vendredi 26 décembre 2025 à 16:25 +0400, Christian Hewitt a écrit :
> > On 26 Dec 2025, at 4:00 pm, Diederik de Haas <diederik@cknow-tech.com> wrote:
> > 
> > Hi Christian,
> > 
> > On Fri Dec 26, 2025 at 12:31 PM CET, Christian Hewitt wrote:
> > > VDPU346 is similar to VDPU381 but with a single core and limited
> > > to 4K60 media. It is also limited to H264 L5.1 and omits AV1 and
> > > AVS2 capabilities. VDPU346 is used with RK3566 and RK3568.
> > > 
> > > Signed-off-by: Christian Hewitt <christianshewitt@gmail.com>
> > > Reviewed-by: Nicolas Dufresne <nicolas.collabora@collabora.com>
> > > ---
> > > .../media/platform/rockchip/rkvdec/rkvdec.c   | 103 ++++++++++++++++++
> > > 1 file changed, 103 insertions(+)
> > > 
> > > diff --git a/drivers/media/platform/rockchip/rkvdec/rkvdec.c b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
> > > index e547057dc75f..6b39e99d8a8b 100644
> > > --- a/drivers/media/platform/rockchip/rkvdec/rkvdec.c
> > > +++ b/drivers/media/platform/rockchip/rkvdec/rkvdec.c
> > > @@ -236,6 +236,62 @@ static const struct rkvdec_ctrls rkvdec_hevc_ctrls = {
> > > .num_ctrls = ARRAY_SIZE(rkvdec_hevc_ctrl_descs),
> > > };
> > > 
> > > +static const struct rkvdec_ctrl_desc vdpu346_hevc_ctrl_descs[] = {
> > > + {
> > > + .cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
> > > + },
> > > + {
> > > + .cfg.id = V4L2_CID_STATELESS_HEVC_SPS,
> > > + .cfg.ops = &rkvdec_ctrl_ops,
> > > + },
> > > + {
> > > + .cfg.id = V4L2_CID_STATELESS_HEVC_PPS,
> > > + },
> > > + {
> > > + .cfg.id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
> > > + },
> > > + {
> > > + .cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
> > > + .cfg.min = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> > > + .cfg.max = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> > > + .cfg.def = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> > > + },
> > > + {
> > > + .cfg.id = V4L2_CID_STATELESS_HEVC_START_CODE,
> > > + .cfg.min = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> > > + .cfg.def = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> > > + .cfg.max = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> > > + },
> > > + {
> > > + .cfg.id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
> > > + .cfg.min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> > > + .cfg.max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
> > > + .cfg.menu_skip_mask =
> > > + BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE),
> > > + .cfg.def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> > > + },
> > > + {
> > > + .cfg.id = V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
> > > + .cfg.min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
> > > + .cfg.max = V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1,
> > > + },
> > > + {
> > > + .cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS,
> > > + .cfg.ops = &rkvdec_ctrl_ops,
> > > + .cfg.dims = { 65 },
> > > + },
> > > + {
> > > + .cfg.id = V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS,
> > > + .cfg.ops = &rkvdec_ctrl_ops,
> > > + .cfg.dims = { 65 },
> > > + },
> > > +};
> > > +
> > > +static const struct rkvdec_ctrls vdpu346_hevc_ctrls = {
> > > + .ctrls = vdpu346_hevc_ctrl_descs,
> > > + .num_ctrls = ARRAY_SIZE(vdpu346_hevc_ctrl_descs),
> > > +};
> > > +
> > > static const struct rkvdec_ctrl_desc vdpu38x_hevc_ctrl_descs[] = {
> > > {
> > > .cfg.id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
> > > @@ -463,6 +519,41 @@ static const struct rkvdec_coded_fmt_desc rk3288_coded_fmts[] = {
> > > }
> > > };
> > > 
> > > +static const struct rkvdec_coded_fmt_desc vdpu346_coded_fmts[] = {
> > > + {
> > > + .fourcc = V4L2_PIX_FMT_HEVC_SLICE,
> > > + .frmsize = {
> > > + .min_width = 64,
> > > + .max_width = 65472,
> > 
> > This should be 4096 according to page 469 of RK3568 TRM Part 2 ...
> > 
> > > + .step_width = 64,
> > > + .min_height = 64,
> > > + .max_height = 65472,
> > 
> > ... and this 2304.
> > 
> > > + .step_height = 16,
> > > + },
> > > + .ctrls = &vdpu346_hevc_ctrls,
> > > + .ops = &rkvdec_vdpu381_hevc_fmt_ops,
> > > + .num_decoded_fmts = ARRAY_SIZE(rkvdec_hevc_decoded_fmts),
> > > + .decoded_fmts = rkvdec_hevc_decoded_fmts,
> > > + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
> > > + },
> > > + {
> > > + .fourcc = V4L2_PIX_FMT_H264_SLICE,
> > > + .frmsize = {
> > > + .min_width = 64,
> > > + .max_width =  65520,
> > 
> > This too should be 4096 according to page 469 of RK3568 TRM Part 2 ...
> > 
> > > + .step_width = 64,
> > > + .min_height = 64,
> > > + .max_height =  65520,
> > 
> > ... and this 2304.
> > 
> > I guess this also explains the 'green images' Nicolas noticed.
> 
> Quite probably. I’ve picked the above changes into my working tree
> (for those following it) and will send a v3 series in response to
> the next revision of Detlev’s patches.
> 
> > + .step_height = 16,
> > > + },
> > > + .ctrls = &rkvdec_h264_ctrls,
> > > + .ops = &rkvdec_vdpu381_h264_fmt_ops,
> > > + .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_decoded_fmts),
> > > + .decoded_fmts = rkvdec_h264_decoded_fmts,
> > > + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
> > > + },
> > 
> > I see you've reversed the order of the blocks so that HEVC now comes
> > before the H264 block. While that makes it consistent with what Detlev
> > has in their v7 and with the existing code in the driver ... I actually
> > prefer having H264 before HEVC as the alphabetical sorting order is
> > H264 before HEVC.
> > In the existing code the VP9 'stuff' is listed below H264 and HEVC.
> > 
> > But then Detlev should do that too in their patch set ... and 'ideally'
> > the order of the existing code be updated to be alphabetically too.
> > 
> > OTOH, a consistent order works for me too.
> 
> I believe the reorder was requested by Nic (offline from the list) so
> there’s probably a reason behind it. I’ll keep things aligned to the
> order in Detlev’s series (whatever that is).

I'm taking note of this feedback, and instead of multiplying the revisions, I
will probably fix it on top once we have the series ready.

regards,
Nicolas

> 
> Christian
> 
> 
> > Cheers,
> >  Diederik
> > 
> > > +};
> > > +
> > > static const struct rkvdec_coded_fmt_desc vdpu381_coded_fmts[] = {
> > > {
> > > .fourcc = V4L2_PIX_FMT_HEVC_SLICE,
> > > @@ -1643,6 +1734,14 @@ static const struct rkvdec_variant_ops vdpu381_variant_ops = {
> > > .flatten_matrices = transpose_and_flatten_matrices,
> > > };
> > > 
> > > +static const struct rkvdec_variant vdpu346_variant = {
> > > + .coded_fmts = vdpu346_coded_fmts,
> > > + .num_coded_fmts = ARRAY_SIZE(vdpu346_coded_fmts),
> > > + .rcb_sizes = vdpu381_rcb_sizes,
> > > + .num_rcb_sizes = ARRAY_SIZE(vdpu381_rcb_sizes),
> > > + .ops = &vdpu381_variant_ops,
> > > +};
> > > +
> > > static const struct rkvdec_variant vdpu381_variant = {
> > > .coded_fmts = vdpu381_coded_fmts,
> > > .num_coded_fmts = ARRAY_SIZE(vdpu381_coded_fmts),
> > > @@ -1691,6 +1790,10 @@ static const struct of_device_id of_rkvdec_match[] = {
> > > .compatible = "rockchip,rk3399-vdec",
> > > .data = &rk3399_rkvdec_variant,
> > > },
> > > + {
> > > + .compatible = "rockchip,rk3568-vdec",
> > > + .data = &vdpu346_variant,
> > > + },
> > > {
> > > .compatible = "rockchip,rk3588-vdec",
> > > .data = &vdpu381_variant,
> 
> 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]