Re: [PATCH 1/2] media: uapi: h264: Add num_ref_idx_active_override_flag

[Paul Kocialkowski <paul.kocialkowski@bootlin.com>]

[-- Attachment #1: Type: text/plain, Size: 4558 bytes --]

Hi,

On Mon 09 Sep 19, 18:40, Tomasz Figa wrote:
> On Mon, Sep 9, 2019 at 4:30 PM Philipp Zabel <p.zabel@pengutronix.de> wrote:
> >
> > On Mon, 2019-09-09 at 16:09 +0900, Tomasz Figa wrote:
> > > On Thu, Sep 5, 2019 at 11:17 PM Nicolas Dufresne
> > > <nicolas.dufresne@collabora.com> wrote:
> > > >
> > > > Le jeudi 05 septembre 2019 à 12:39 +0200, Philipp Zabel a écrit :
> > > > > On Thu, 2019-09-05 at 19:31 +0900, Tomasz Figa wrote:
> > > > > > On Thu, Sep 5, 2019 at 7:15 PM Philipp Zabel <p.zabel@pengutronix.de> wrote:
> > > > > > > This flag tells the kernel whether the slice header contained the
> > > > > > > num_ref_idx_l[01]_active_minus1 syntax elements, or whether the
> > > > > > > num_ref_idx_l[01]_default_active_minus1 from PPS should be used
> > > > > > > instead.
> > > > > > >
> > > > > > > Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
> > > > > > > ---
> > > > > > >  Documentation/media/uapi/v4l/ext-ctrls-codec.rst | 3 +++
> > > > > > >  include/media/h264-ctrls.h                       | 1 +
> > > > > > >  2 files changed, 4 insertions(+)
> > > > > > >
> > > > > > > diff --git a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> > > > > > > index bc5dd8e76567..451a5b0f2a35 100644
> > > > > > > --- a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> > > > > > > +++ b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> > > > > > > @@ -1860,6 +1860,9 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
> > > > > > >      * - ``V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH``
> > > > > > >        - 0x00000008
> > > > > > >        -
> > > > > > > +    * - ``V4L2_H264_SLICE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE
> > > > > > > +      - 0x00000010
> > > > > > > +      - Corresponds to the num_ref_idx_active_override_flag syntax element.
> > > > > > >
> > > > > >
> > > > > > As far as I remember, the idea was for the userspace to always put the
> > > > > > right num_ref_idx in the slice_params and the drivers always use that.
> > > > > > Was there any problem with that?
> > > > >
> > > > > I don't think so, at least for currently known hardware.
> > > > >
> > > > > In that case we should drop the unused
> > > > > num_ref_idx_l[01]_default_active_minus1 fields from struct
> > > > > v4l2_ctrl_h264_pps and document that userspace should fill
> > > > > the defaults into v4l2_ctrl_h264_slice_params themselves if
> > > > > num_ref_idx_active_override_flag wasn't set.
> > > >
> > > > It might have been added in a previous effort to allow reconstructing
> > > > the bitstream from the structures.
> > >
> > > Wouldn't one still be able to reconstruct a valid (but not exact)
> > > stream without that flag, given the assumption above?
> >
> > If the Hantro G1, as appears to be the case, parses the slice header and
> > decides itself whether to use the override from the slice or the default
> > that was written to a register, it needs the
> > num_ref_idx_l[01]_default_active_minus1 field to fill the register, but
> > doesn't need either the num_ref_idx_l[01]_active_minus1 override nor the
> > flag.
> >
> > A decoder that doesn't parse the slice header can always be told to use
> > the override (thus no need to have the flag in the uapi), if userspace
> > fills the default into the override fields as a fallback. Such a decoder
> > does need the num_ref_idx_l[01]_active_minus1 override, but doesn't need
> > the num_ref_idx_l[01]_default_active_minus1 field nor the flag.
> >
> > That is my current understanding of the intention behind this interface,
> > I hope this is accurate.
> > I've tried to make the docs reflect this in ("media: uapi: h264: clarify
> > num_ref_idx_l[01]_(default_)active fields") [1].
> >
> > [1] https://lore.kernel.org/linux-media/20190905114210.9232-1-p.zabel@pengutronix.de/T/#u
> 
> Yes, makes sense for me.

Sorry for being late to the party and catching up with this just now.

It looks like the conclusion is that we need to keep both the PPS default
and slice-specific set of fields to cover both hardware that parses the slice
header and hardware that doesn't.

In H265, I had evicted the PPS default value and the flag in favor of only
providing a slice-specific value, but I should certainly revise this to have
the two sets of values and the flag. This should allow covering hardware that
does slice header parsing itself.

Cheers,

Paul

-- 
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]