Re: [PATCH v6 0/6] Raspberry Pi HEVC decoder driver

[Jonas Karlman <jonas@kwiboo.se>]

Hi Dave,

On 3/4/2026 3:05 PM, Dave Stevenson wrote:
> Hi All
> 
> This has been in the pipeline for a while, but I've finally cleaned
> up our HEVC decoder driver to be in a shape to upstream.
> John Cox has done almost all of the work under contract to Raspberry
> Pi, and I'm largely just doing the process of patch curation and
> sending.
> 
> Thanks
>   Dave
> 
> v4l2-compliance 1.33.0-5448, 64 bits, 64-bit time_t
> v4l2-compliance SHA: bd0aabb96710 2026-02-09 17:38:28
> 
> Compliance test for rpi-hevc-dec device /dev/video0:
> 
> Driver Info:
> 	Driver name      : rpi-hevc-dec
> 	Card type        : rpi-hevc-dec
> 	Bus info         : platform:rpi-hevc-dec
> 	Driver version   : 6.19.0
> 	Capabilities     : 0x84204000
> 		Video Memory-to-Memory Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04204000
> 		Video Memory-to-Memory Multiplanar
> 		Streaming
> 		Extended Pix Format
> 	Detected Stateless Decoder
> Media Driver Info:
> 	Driver name      : rpi-hevc-dec
> 	Model            : rpi-hevc-dec
> 	Serial           : 
> 	Bus info         : platform:rpi-hevc-dec
> 	Media version    : 6.19.0
> 	Hardware revision: 0x00000000 (0)
> 	Driver version   : 6.19.0
> Interface Info:
> 	ID               : 0x0300000c
> 	Type             : V4L Video
> Entity Info:
> 	ID               : 0x00000001 (1)
> 	Name             : rpi-hevc-dec-source
> 	Function         : V4L2 I/O
> 	Pad 0x01000002   : 0: Source
> 	  Link 0x02000008: to remote pad 0x1000004 of entity 'rpi-hevc-dec-proc' (Video Decoder): Data, Enabled, Immutable
> 
> Required ioctls:
> 	test MC information (see 'Media Driver Info' above): OK
> 	test VIDIOC_QUERYCAP: OK
> 	test invalid ioctls: OK
> 
> Allow for multiple opens:
> 	test second /dev/video0 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 		fail: v4l2-test-controls.cpp(942): try_ext_ctrls returned an error (22)
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 3 Private Controls: 0
> 	Standard Compound Controls: 5 Private Compound Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK (Not Supported)
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> 	test CREATE_BUFS maximum buffers: OK
> 	test VIDIOC_REMOVE_BUFS: OK
> 	test VIDIOC_EXPBUF: OK
> 	test Requests: OK
> 	test blocking wait: OK
> 
> Total for rpi-hevc-dec device /dev/video0: 49, Succeeded: 48, Failed: 1, Warnings: 0
> 
> Testing is with a downstream patchset to FFmpeg. I'm told FFmpeg
> currently has no stateless decode support, but we will be reviewing
> works that have been in progress and our downstream patches to see
> whether that can be pushed onwards.

Please see FFmpeg PR20847 "Add V4L2 Request API hwaccels for MPEG2,
H.264, HEVC, VP8, VP9 and AV1" [1] for pending upstream FFmpeg work.

My 4l2request-v3-pixfmts branch [2] also contains extra commit to make
it work with downstream rpi kernel version, using non-MT pixfmts.

I will give this a spin together with the FFmpeg PR, adding support for
the MT based pixfmts should hopefully be easy.

[1] https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20847
[2] https://code.ffmpeg.org/Kwiboo/FFmpeg/commits/branch/v4l2request-v3-pixfmts

Regards,
Jonas

> Downstream tree is at
> https://github.com/jc-kynesim/rpi-ffmpeg/tree/test/7.1.2/main
> 
> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9247
> adds support for the new formats to GStreamer, and runs Fluster.
> Using the JCT-VC-HEVC_V1 test suite we get 142 passes out of 147 tests.
> PICSIZE_[ABCD]_Bossen_1 all fail due to the image sizes being greater than
> the 4096x4096 limit of this driver.
> TSUNEQBD_A_MAIN10_Technicolor_2 fails as the hardware doesn't support
> having a different bit depth for luma and chroma.
> 
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> ---
> Changes in v6:
> - Fixed up assigned by unused variable (ctrl) in hevc_d_init_ctrls
>   (reported by CI).
> - Fixed up documentation links (reported by CI)
> - Link to v5: https://lore.kernel.org/r/20260227-media-rpi-hevc-dec-v5-0-9bb3fc1816de@raspberrypi.com
> 
> Changes in v5:
> - Updated driver based on Nicolas' review comments of V4.
> - Added NV12MT_COL128 to ./drivers/media/v4l2-core/v4l2-common.c. The
>   10 bit format can't be represented with v4l2_format_info at present
>   as it assumes a power of 2 block size.
> - Switched to kzalloc_obj/kzalloc_objs as checkpatch now checks that.
> - Link to v4: https://lore.kernel.org/r/20250701-media-rpi-hevc-dec-v4-0-057cfa541177@raspberrypi.com
> 
> Changes in v4:
> - dt-bindings: Drop to just SoC compatibles and updated description (Krzysztof).
> - dts: Corrected address in node name.
> - NV12MT_COL128 format description added vertical alignment to a
>   multiple of 8 (Nicolas)
> - driver: Merged in driver updates based on Nicolas' review.
> - driver: Added debug module parameter to avoid log spam due to some bitstreams.
> - cover-letter: Added Fluster results.
> - rebased and based on the updated version of 
>   https://lore.kernel.org/linux-media/20250604-sebastianfricke-vcodec_manual_request_completion_with_state_machine-v3-0-603db4749d90@collabora.com/
> - update v4l2-compliance output
> - Link to v3: https://lore.kernel.org/r/20250423-media-rpi-hevc-dec-v3-0-8fd3fad1d6fb@raspberrypi.com
> 
> Changes in v3:
> - Updated the dtbinding with SoC specific compatible strings (Rob).
> - Reordered hevc_dec and v3d in bcm2711.dtsi to keep them in ascending
>   register order (Stefan).
> - Reordered hevc_dec in bcm2711-rpi.dtsi to keep them in alphabetical
>   order (Stefan).
> - Tested on top of Nicolas' revised version of Hans' patch set for
>   manual request completion.
>   https://lore.kernel.org/all/20250410-sebastianfricke-vcodec_manual_request_completion_with_state_machine-v2-0-5b99ec0450e6@collabora.com/
> - Link to v2: https://lore.kernel.org/r/20250206-media-rpi-hevc-dec-v2-0-69353c8805b2@raspberrypi.com
> 
> Changes in v2:
> - Rebased to use Hans' manual request completion scheme.
>   https://lore.kernel.org/linux-media/cover.1724928939.git.hverkuil-cisco@xs4all.nl/
> - Require all slices for a frame to be submitted in one request.
> - Added the missing header file.
> - Used the full macro name for pix format docs (Sakari)
> - Dropped unneeded |- from dtbinding (Rob)
> - Made reg and reg-names match in order (Rob)
> - Removed clock-names from dtbinding (Rob)
> - Driver changed to not request the clock by name
> - Dropped clock-names from DTS file
> - Minor fixes for compliance failures
>   fail: v4l2-test-formats.cpp(958): fmt_cap.g_colorspace() != col
>   fail: v4l2-test-buffers.cpp(901): q.create_bufs(node, 1, &fmt) != EINVAL
> - v4l2-compliance output added to cover letter (Nicholas)
>   I believe the "fail: v4l2-test-controls.cpp(939): try_ext_ctrls
>   returned an error (22)" is expected as it is validating the SPS.
>   Hantro and Cedrus certainly both appear to return errors in the same place
> - Link to v1: https://lore.kernel.org/r/20241220-media-rpi-hevc-dec-v1-0-0ebcc04ed42e@raspberrypi.com
> 
> ---
> Dave Stevenson (4):
>       docs: uapi: media: Document Raspberry Pi NV12 column format
>       media: ioctl: Add pixel formats NV12MT_COL128 and NV12MT_10_COL128
>       dt-bindings: media: Add the Raspberry Pi HEVC decoder
>       arm: dts: bcm2711-rpi: Add HEVC decoder node
> 
> John Cox (2):
>       docs: uapi: media: Clarify HEVC slice_param bit_size, data_byte_offset
>       media: platform: Add Raspberry Pi HEVC decoder driver
> 
>  .../bindings/media/raspberrypi,hevc-dec.yaml       |   72 +
>  .../media/v4l/ext-ctrls-codec-stateless.rst        |    6 +-
>  .../userspace-api/media/v4l/pixfmt-yuv-planar.rst  |   46 +
>  MAINTAINERS                                        |   10 +
>  arch/arm/boot/dts/broadcom/bcm2711-rpi.dtsi        |    4 +
>  arch/arm/boot/dts/broadcom/bcm2711.dtsi            |    9 +
>  drivers/media/platform/raspberrypi/Kconfig         |    1 +
>  drivers/media/platform/raspberrypi/Makefile        |    1 +
>  .../media/platform/raspberrypi/hevc_dec/Kconfig    |   17 +
>  .../media/platform/raspberrypi/hevc_dec/Makefile   |    5 +
>  .../media/platform/raspberrypi/hevc_dec/hevc_d.c   |  325 +++
>  .../media/platform/raspberrypi/hevc_dec/hevc_d.h   |  195 ++
>  .../platform/raspberrypi/hevc_dec/hevc_d_h265.c    | 2436 ++++++++++++++++++++
>  .../platform/raspberrypi/hevc_dec/hevc_d_h265.h    |   22 +
>  .../platform/raspberrypi/hevc_dec/hevc_d_hw.c      |  429 ++++
>  .../platform/raspberrypi/hevc_dec/hevc_d_hw.h      |  317 +++
>  .../platform/raspberrypi/hevc_dec/hevc_d_video.c   |  634 +++++
>  .../platform/raspberrypi/hevc_dec/hevc_d_video.h   |   38 +
>  drivers/media/v4l2-core/v4l2-common.c              |    2 +
>  drivers/media/v4l2-core/v4l2-ioctl.c               |    2 +
>  include/uapi/linux/videodev2.h                     |    4 +
>  21 files changed, 4573 insertions(+), 2 deletions(-)
> ---
> base-commit: 1512bd6430c7be7723724f31a74c3b8aaacf8a20
> change-id: 20241212-media-rpi-hevc-dec-3b5be739f3bd
> 
> Best regards,