Re: [PATCH v3 01/15] media: uapi: HEVC: Add v4l2_ctrl_hevc_ext_sps_[ls]t_rps controls

[Detlev Casanova <detlev.casanova@collabora.com>]

Hi Nicolas,

On 12/10/25 15:03, Nicolas Dufresne wrote:
> Hi,
>
> Le mercredi 22 octobre 2025 à 12:22 -0400, Detlev Casanova a écrit :
>> Some hardware (e.g.: Rockchip's rk3588 hevc decoder) need the
>> long and short term reference information for HEVC decoding.
> That's seems a bit short and incomplete. My understanding is that the decoder on
> this hardware needs to full parse the slice headers, and cannot be passed with
> the slice information. It also does not implement a a skip method similar to
> Hantro.
>
> So in short, I think the justification for this API is that these parameters are
> needed to fully parse HEVC slice headers. You should perhaps say that similar
> paramters ares passed in Vulkan Video API ?
Yes, I will clarify
>> Add controls to provide it as the raw data form the stream.
>>
>> Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com>
>> ---
>>   .../media/v4l/ext-ctrls-codec-stateless.rst   | 114 ++++++++++++++++++
>>   .../media/v4l/videodev2.h.rst.exceptions      |   2 +
>>   .../media/v4l/vidioc-queryctrl.rst            |  12 ++
>>   3 files changed, 128 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
>> index 0da635691fdc..bb6772cf3cee 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
>> @@ -2958,6 +2958,120 @@ This structure contains all loop filter related parameters. See sections
>>         - 0x00000004
>>         -
>>   
>> +``V4L2_CID_STATELESS_HEVC_EXT_SPS_LT_RPS (struct)``
>> +    Specifies the list of Long-Term reference sets parameters from the SPS.
>> +    These parameters are defined according to :ref:`hevc`.
>> +    They are described in section 7.4.3.2.1 "General sequence parameter set
>> +    RBSP semantics" of the specification.
>> +    This control is a dynamically sized 1-dimensional array,
>> +    V4L2_CTRL_FLAG_DYNAMIC_ARRAY flag must be set when using it.
> Can we making a bit clearer that this is a subset of the SPS (perhaps use the
> term subset). And this is user to extend another control (mention this other
> control) ?
>
>> +
>> +.. c:type:: v4l2_ctrl_hevc_ext_sps_lt_rps
>> +
>> +.. cssclass:: longtable
>> +
>> +.. flat-table:: struct v4l2_ctrl_hevc_ext_sps_lt_rps
>> +    :header-rows:  0
>> +    :stub-columns: 0
>> +    :widths:       1 1 2
>> +
>> +    * - __u16
>> +      - ``lt_ref_pic_poc_lsb_sps``
>> +      - Long term reference picture order count as described in section 7.4.3.2.1
>> +        "General sequence parameter set RBSP semantics" of the specification.
>> +    * - __u8
>> +      - ``flags``
>> +      - See :ref:`Extended Long-Term RPS Flags <hevc_ext_sps_lt_rps_flags>`
>> +
>> +.. _hevc_ext_sps_lt_rps_flags:
>> +
>> +``Extended SPS Long-Term RPS Flags``
>> +
>> +.. cssclass:: longtable
>> +
>> +.. flat-table::
>> +    :header-rows:  0
>> +    :stub-columns: 0
>> +    :widths:       1 1 2
>> +
>> +    * - ``V4L2_HEVC_EXT_SPS_LT_RPS_FLAG_USED_LT``
>> +      - 0x00000001
>> +      - Specifies if the long-term reference picture is used 7.4.3.2.1 "General sequence parameter
>> +        set RBSP semantics" of the specification.
>> +
>> +``V4L2_CID_STATELESS_HEVC_EXT_SPS_ST_RPS (struct)``
>> +    Specifies the list of Short-Term reference sets parameters from the SPS.
>> +    These parameters are defined according to :ref:`hevc`.
>> +    They are described in section 7.4.8 "Short-term reference picture set
>> +    semantics" of the specification.
>> +    This control is a dynamically sized 1-dimensional array,
>> +    V4L2_CTRL_FLAG_DYNAMIC_ARRAY flag must be set when using it.
> Déjà vue, this is a duplicate right ?

Almost, you saw the description of the Long term sets before, this is 
for the Short term.

Only the name and specification section have changed in the description.

>> +
>> +.. c:type:: v4l2_ctrl_hevc_ext_sps_st_rps
>> +
>> +.. cssclass:: longtable
>> +
>> +.. flat-table:: struct v4l2_ctrl_hevc_ext_sps_st_rps
>> +    :header-rows:  0
>> +    :stub-columns: 0
>> +    :widths:       1 1 2
>> +
>> +    * - __u8
>> +      - ``delta_idx_minus1``
>> +      - Specifies the delta compare to the index. See details in section 7.4.8 "Short-term
>> +        reference picture set semantics" of the specification.
>> +    * - __u8
>> +      - ``delta_rps_sign``
>> +      - Sign of the delta as specified in section 7.4.8 "Short-term reference picture set
>> +        semantics" of the specification.
>> +    * - __u16
>> +      - ``abs_delta_rps_minus1``
>> +      - Absolute delta RPS as specified in section 7.4.8 "Short-term reference picture set
>> +        semantics" of the specification.
>> +    * - __u8
>> +      - ``num_negative_pics``
>> +      - Number of short-term RPS entries that have picture order count values less than the
>> +        picture order count value of the current picture.
>> +    * - __u8
>> +      - ``num_positive_pics``
>> +      - Number of short-term RPS entries that have picture order count values greater than the
>> +        picture order count value of the current picture.
>> +    * - __u32
>> +      - ``used_by_curr_pic``
>> +      - Bit i specifies if short-term RPS i is used by the current picture.
>> +    * - __u32
>> +      - ``use_delta_flag``
>> +      - Bit i specifies if short-term RPS i is included in the short-term RPS entries.
>> +    * - __u16
>> +      - ``delta_poc_s0_minus1[16]``
>> +      - Specifies the negative picture order count delta for the i-th entry in the short-term RPS.
>> +        See details in section 7.4.8 "Short-term reference picture set semantics" of the
>> +        specification.
>> +    * - __u16
>> +      - ``delta_poc_s1_minus1[16]``
>> +      - Specifies the positive picture order count delta for the i-th entry in the short-term RPS.
>> +        See details in section 7.4.8 "Short-term reference picture set semantics" of the
>> +        specification.
>> +    * - __u8
>> +      - ``flags``
>> +      - See :ref:`Extended Short-Term RPS Flags <hevc_ext_sps_st_rps_flags>`
>> +
>> +.. _hevc_ext_sps_st_rps_flags:
>> +
>> +``Extended SPS Short-Term RPS Flags``
>> +
>> +.. cssclass:: longtable
>> +
>> +.. flat-table::
>> +    :header-rows:  0
>> +    :stub-columns: 0
>> +    :widths:       1 1 2
>> +
>> +    * - ``V4L2_HEVC_EXT_SPS_ST_RPS_FLAG_INTER_REF_PIC_SET_PRED``
>> +      - 0x00000001
>> +      - Specifies if the short-term RPS is predicted from another short term RPS. See details in
>> +        section 7.4.8 "Short-term reference picture set semantics" of the specification.
>> +
>>   .. _v4l2-codec-stateless-av1:
>>   
>>   ``V4L2_CID_STATELESS_AV1_SEQUENCE (struct)``
>> diff --git a/Documentation/userspace-api/media/v4l/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/v4l/videodev2.h.rst.exceptions
>> index 35d3456cc812..1663fdf8444c 100644
>> --- a/Documentation/userspace-api/media/v4l/videodev2.h.rst.exceptions
>> +++ b/Documentation/userspace-api/media/v4l/videodev2.h.rst.exceptions
>> @@ -149,6 +149,8 @@ replace symbol V4L2_CTRL_TYPE_H264_DECODE_PARAMS :c:type:`v4l2_ctrl_type`
>>   replace symbol V4L2_CTRL_TYPE_HEVC_SPS :c:type:`v4l2_ctrl_type`
>>   replace symbol V4L2_CTRL_TYPE_HEVC_PPS :c:type:`v4l2_ctrl_type`
>>   replace symbol V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS :c:type:`v4l2_ctrl_type`
>> +replace symbol V4L2_CTRL_TYPE_HEVC_EXT_SPS_ST_RPS :c:type:`v4l2_ctrl_type`
>> +replace symbol V4L2_CTRL_TYPE_HEVC_EXT_SPS_LT_RPS :c:type:`v4l2_ctrl_type`
>>   replace symbol V4L2_CTRL_TYPE_AREA :c:type:`v4l2_ctrl_type`
>>   replace symbol V4L2_CTRL_TYPE_RECT :c:type:`v4l2_ctrl_type`
>>   replace symbol V4L2_CTRL_TYPE_FWHT_PARAMS :c:type:`v4l2_ctrl_type`
>> diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
>> index 3549417c7feb..128c044d2e3c 100644
>> --- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
>> +++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
>> @@ -523,6 +523,18 @@ See also the examples in :ref:`control`.
>>         - n/a
>>         - A struct :c:type:`v4l2_ctrl_hevc_decode_params`, containing HEVC
>>   	decoding parameters for stateless video decoders.
>> +    * - ``V4L2_CTRL_TYPE_HEVC_EXT_SPS_LT_RPS``
>> +      - n/a
>> +      - n/a
>> +      - n/a
>> +      - A struct :c:type:`v4l2_ctrl_hevc_ext_sps_lt_rps`, containing HEVC
>> +	extended Long-Term RPS for stateless video decoders.
>> +    * - ``V4L2_CTRL_TYPE_HEVC_EXT_SPS_ST_RPS``
>> +      - n/a
>> +      - n/a
>> +      - n/a
>> +      - A struct :c:type:`v4l2_ctrl_hevc_ext_sps_st_rps`, containing HEVC
>> +	extended Short-Term RPS for stateless video decoders.
>>       * - ``V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR``
>>         - n/a
>>         - n/a
> cheers,
> Nicolas

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip