Re: [PATCH RFC 0/7] media: qcom: iris: add support for decoding 10bit formats

[Neil Armstrong <neil.armstrong@linaro.org>]

Hi,

On 4/9/26 03:04, Nicolas Dufresne wrote:
> Hi,
> 
> Le jeudi 09 avril 2026 à 03:02 +0300, Dmitry Baryshkov a écrit :
>> On Wed, Apr 08, 2026 at 06:43:53PM +0200, Neil Armstrong wrote:
>>> This adds the plumbing to support decoding HEVC and AV1
>>> streams into 10bit pixel formats, linear and compressed.
>>>
>>> This has only been tested on SM8650 with HEVC, and was inspired by
>>> Venus and the downstream vidc driver for the buffer
>>> calculations and HFI messages.
>>>
>>> I was unable to get 10bit decoding working with Gstreamer
>>> and ffmpeg, but v4l2-ctl works with:
>>
>> Any particular errors? I assume Gstreamer needs to be taught about
>> Q10C. But P010 should (hopefully) work.
> 
> P010 should work for both Gst and FFMPEG, its probably a user error, or there is
> a hidden bug in the driver that make it fail, v4l2-ctl is very permissive as it
> simply dump to disk. You should provide an updated fluster score, so you have to
> use one of these.

I did run fluster and all main10 fails with Gstreamer and FFmpeg, I tried to manually
run the gst and ffmpeg commands with v4l2-tracer and logs but I can't explain the reason,
all returns from the driver seems valid but somehow they just error out with:

FFmpeg:
$ ffmpeg -c:v hevc_v4l2m2m -i Big_Buck_Bunny_1080_10s_30MB_main10.h265 -y -f null -
...
[hevc_v4l2m2m @ 0x55c0328aa0] Using device /dev/video-dec0
[hevc_v4l2m2m @ 0x55c0328aa0] driver 'iris_driver' on card 'iris_decoder' in mplane mode
[hevc_v4l2m2m @ 0x55c0328aa0] requesting formats: output=HEVC/none capture=NV12/yuv420p10le
...
[hevc_v4l2m2m @ 0x55c0328aa0] An invalid frame was output by a decoder. This is a bug, please report it.
[vist#0:0/hevc @ 0x55c02dc9b0] [dec:hevc_v4l2m2m @ 0x55c029d510] Decoding error: Internal bug, should not have happened

The v4l2 trace shows a normal sequence with the driver returning P010 as G_FMT after the source change event,
and the capture planes dequeued but for an unknown reason the buffer is rejected by ffmpeg.

Gst:
$ gst-launch-1.0 -v -m filesrc location=Big_Buck_Bunny_1080_10s_30MB_main10.h265 ! h265parse !  v4l2h265dec ! tee ! fakevideosink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
...
Got message #37 from element "h265parse0" (latency): no message details
ERROR: from element /GstPipeline:pipeline0/GstH265Parse:h265parse0: Internal data stream error.
Redistribute latency...
Additional debug info:
../gstreamer/subprojects/gstreamer/libs/gst/base/gstbaseparse.c(3702): gst_base_parse_loop (): /GstPipeline:pipeline0/GstH265Parse:h265parse0:
streaming stopped, reason not-negotiated (-4)
Got message #39 from pad "h265parse0:src" (property-notify): ERROR: pipeline doesn't want to preroll.
GstMessagePropertyNotify, property-name=(string)caps, property-value=(GstCaps)"video/x-h265\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ framerate\=\(fraction\)30/1\,\ chroma-format\=\(string\)4:2:0\,\ bit-depth-luma\=\(uint\)10\,\ bit-depth-chroma\=\(uint\)10\,\ parsed\=\(boolean\)true\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ profile\=\(string\)main-10\,\ tier\=\(string\)main\,\ level\=\(string\)4";
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:src: caps = video/x-h265, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, chroma-format=(string)4:2:0, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, parsed=(boolean)true, stream-format=(string)byte-stream, alignment=(string)au, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-10, tier=(string)main, level=(string)4
...

In this case OUTPUT is not STREAMON and no OUTPUT buffers are queued, so I wonder why this one fails....

My gstreamer and ffmpeg foo is bad and I probably missed something obvious...

> 
> For Q10C on GStreamer, it needs mapping [0] and you need some bugfix [1] and
> another that I will be sending tomorrow. We had never tested video compression
> with this module before.
> 
> [0] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8195
> [1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/11222
> 
> The last issue has to do with:
> https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/38965e2d9c1119674a65dc437ee7e8ec95339f31/subprojects/gst-plugins-good/sys/v4l2/gstv4l2object.c#L4378
> 
> V4L2 format gives us the number of allocation, but not really the number of
> planes, and we forgot to initialize that number for the "opaque" format case. A
> tempory fix might be to add this after S_FMT:
> 
> if (GST_VIDEO_INFO_FORMAT (&info.vinfo) == GST_VIDEO_FORMAT_DMA_DRM)
>    n_v4l_planes = format.fmt.pix_mp.num_planes;
> 
> Works for AFBC and QC at leat, since both are unambiguously single plane, and so
> cannot have mplane variants.

I'll definitely try that ! thanks for the pointers !

> 
> Let me know how far you get! Please be aware that Robert and I are making these
> patches based on feedback, we don't have access to any boards capable of
> decoding to QC compressed formats.
> 
>>
>>> v4l2-ctl --verbose --set-fmt-video-out=pixelformat=HEVC --set-fmt-video=pixelformat=P010 --stream-mmap --stream-out-mmap --stream-from-hdr Big_Buck_Bunny_1080_10s_30MB_main10.h265.hdr --stream-to out.P010
>>> v4l2-ctl --verbose --set-fmt-video-out=pixelformat=HEVC --set-fmt-video=pixelformat=Q10C --stream-mmap --stream-out-mmap --stream-from-hdr Big_Buck_Bunny_1080_10s_30MB_main10.h265.hdr --stream-to out.QC10
>>>
>>> The non-10bit decoding still works as before.
>>>
>>> With Big_Buck_Bunny_1080_10s_30MB reencoded in 10-bit profile
>>> and tranformed in v4l2 header format with [1]:
>>> ffmpeg -i Big_Buck_Bunny_1080_10s_30MB.h264 -pix_fmt yuv420p10le -c:v libx265 -crf 28 -x265-params profile=main10 Big_Buck_Bunny_1080_10s_30MB_main10.h265
>>> /path/to/mkhdr.sh Big_Buck_Bunny_1080_10s_30MB_main10.h265 raw Big_Buck_Bunny_1080_10s_30MB_main10.h265.hdr
>>>
>>> The frames correctness has been verified buy displaying them
>>> via Vulkan DMA_BUF import, including QC10C and QC08C.
> 
> In GStreamer, once the video4linux plugin issues are fixed, you should be able
> to display the frames using glimagesink. GL only allow for RGB render, which
> damages the data, so its not good enough for conformance testing with
> compression enabled, but usually just doing visual inspection is acceptable.
> 
>>>
>>> The support is probably incomplete for other platforms and
>>> I'm unsure what's required to conform to the V4L2 M2M stateless
>>
>> stateful
>>
>>> spec, especially since AFAIK the decoder doesn't support
>>> decoding 10bit streams in 8bit pixel format, thus the RFC state.
>>> Review is welcome !
> 
> Why would your decoder need to support decoding 10bit into 8bit ? This is quite
> rare and its only possible with post-processed capture buffer.

Yeah it's just a note, the Amlogic one supported outputing main10 in NV12 but
only supported 10bit in their compressed format.

> 
>>>
>>> [1] https://github.com/superna9999/pyv4l2compliance
>>>
>>> Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org>
>>> ---
>>> Neil Armstrong (7):
>>>        media: qcom: iris: add QC10C & P010 buffer size calculations
>>>        media: qcom: iris: gen2: add support for 10bit decoding
>>>        media: qcom: iris: add helpers for 8bit and 10bit formats
>>>        media: qcom: iris: vdec: update size and stride calculations for 10bit formats
>>>        media: qcom: iris: vdec: forbid g_fmt while waiting for first source change
> 
> The is suspicious, it should just send the previous state until you get the
> event. Typical use case is for application to set a format based on bitstream
> parsing, and use that as a guess to try and pre-allocate the buffers. I don't
> usually review iris code, but I'll give that series some more eyes in the coming
> days.

Yeah I wasn't sure, venus does that, same as the vendor driver so I thought it would
be good but I'm probably mistaken.

> 
> cheers,
> Nicolas

Thanks!
Neil

> 
>>>        media: qcom: iris: vdec: update find_format to handle 8bit and 10bit formats
>>>        media: qcom: iris: vdec: allow decoding into 10bit format
>>>
>>>   drivers/media/platform/qcom/iris/iris_buffer.c     | 81 +++++++++++++++++++++-
>>>   .../platform/qcom/iris/iris_hfi_gen2_command.c     | 71 ++++++++++++++++++-
>>>   .../platform/qcom/iris/iris_hfi_gen2_defines.h     |  1 +
>>>   .../platform/qcom/iris/iris_hfi_gen2_response.c    | 35 ++++++++--
>>>   drivers/media/platform/qcom/iris/iris_instance.h   |  2 +
>>>   .../platform/qcom/iris/iris_platform_common.h      |  1 +
>>>   .../media/platform/qcom/iris/iris_platform_gen2.c  |  3 +-
>>>   drivers/media/platform/qcom/iris/iris_state.c      |  6 ++
>>>   drivers/media/platform/qcom/iris/iris_state.h      |  1 +
>>>   drivers/media/platform/qcom/iris/iris_utils.c      | 16 ++++-
>>>   drivers/media/platform/qcom/iris/iris_utils.h      |  2 +
>>>   drivers/media/platform/qcom/iris/iris_vdec.c       | 70 +++++++++++++++++--
>>>   drivers/media/platform/qcom/iris/iris_vidc.c       | 14 +++-
>>>   13 files changed, 285 insertions(+), 18 deletions(-)
>>> ---
>>> base-commit: f3e6330d7fe42b204af05a2dbc68b379e0ad179e
>>> change-id: 20260408-topic-sm8x50-iris-10bit-decoding-074c3ac7975c
>>>
>>> Best regards,
>>> --
>>> Neil Armstrong <neil.armstrong@linaro.org>
>>>