H264 stateless reference frames ordering lists

H264 stateless reference frames ordering lists

[Alexandre Courbot]

Hi Boris and Ezequiel,

Patch c3adb85745ca6 has removed the ref_pic_list_* members from the
v4l2_ctrl_h264_decode_params struct. The MT8183 stateless decoder
driver I am working on still relies on these lists and I am trying (a
bit late to the game) to remove them from the Chromium OS kernel UAPI
in order to match upstream.

I have dug into the discussion that resulted in this removal and could
not really find how these are supposed to be reconstructed and on the
basis on which information. The commit log for the above-mentioned
patch mentions that "generic helpers will be provided for drivers that
need this list". I could not find any in the kernel so far, do you
have any code available at the moment? Or any idea on how these can be
reconstructed? The process seems to involve reading the DPB itself as
well as reordering information from the slice parameters, and seems to
be a bit involved to be done in the kernel, but maybe I am missing
something here.

Thanks for your help!
Alex.

Re: H264 stateless reference frames ordering lists

[Boris Brezillon]

Hi Alexandre,

On Tue, 12 Nov 2019 19:14:22 +0900
Alexandre Courbot <acourbot@google.com> wrote:

> Hi Boris and Ezequiel,
> 
> Patch c3adb85745ca6 has removed the ref_pic_list_* members from the
> v4l2_ctrl_h264_decode_params struct. The MT8183 stateless decoder
> driver I am working on still relies on these lists and I am trying (a
> bit late to the game) to remove them from the Chromium OS kernel UAPI
> in order to match upstream.
> 
> I have dug into the discussion that resulted in this removal and could
> not really find how these are supposed to be reconstructed and on the
> basis on which information. The commit log for the above-mentioned
> patch mentions that "generic helpers will be provided for drivers that
> need this list". I could not find any in the kernel so far, do you
> have any code available at the moment? Or any idea on how these can be
> reconstructed? The process seems to involve reading the DPB itself as
> well as reordering information from the slice parameters, and seems to
> be a bit involved to be done in the kernel, but maybe I am missing
> something here.

The code is here [1], and it should indeed be extracted and put in a
generic v4l2_h264 lib at some point (should happen soon, since we need
the same logic for the rkvdec driver).

Let me know if you have any questions.

Regards,

Boris

[1]https://elixir.bootlin.com/linux/v5.4-rc7/source/drivers/staging/media/hantro/hantro_h264.c#L270

Re: H264 stateless reference frames ordering lists

[Alexandre Courbot]

On Tue, Nov 12, 2019 at 7:53 PM Boris Brezillon
<boris.brezillon@collabora.com> wrote:
>
> Hi Alexandre,
>
> On Tue, 12 Nov 2019 19:14:22 +0900
> Alexandre Courbot <acourbot@google.com> wrote:
>
> > Hi Boris and Ezequiel,
> >
> > Patch c3adb85745ca6 has removed the ref_pic_list_* members from the
> > v4l2_ctrl_h264_decode_params struct. The MT8183 stateless decoder
> > driver I am working on still relies on these lists and I am trying (a
> > bit late to the game) to remove them from the Chromium OS kernel UAPI
> > in order to match upstream.
> >
> > I have dug into the discussion that resulted in this removal and could
> > not really find how these are supposed to be reconstructed and on the
> > basis on which information. The commit log for the above-mentioned
> > patch mentions that "generic helpers will be provided for drivers that
> > need this list". I could not find any in the kernel so far, do you
> > have any code available at the moment? Or any idea on how these can be
> > reconstructed? The process seems to involve reading the DPB itself as
> > well as reordering information from the slice parameters, and seems to
> > be a bit involved to be done in the kernel, but maybe I am missing
> > something here.
>
> The code is here [1], and it should indeed be extracted and put in a
> generic v4l2_h264 lib at some point (should happen soon, since we need
> the same logic for the rkvdec driver).
>
> Let me know if you have any questions.

Thanks - not sure how I missed it. I have tried adapting the code for
MT8183 and it seems to be working perfectly there as well.

Regarding the lib to make this code available to other drivers, I was
thinking about doing it on top of
https://patchwork.kernel.org/patch/11076405/ but is this patch still
being worked on?

Re: H264 stateless reference frames ordering lists

[Boris Brezillon]

Hi Alexandre,

On Wed, 13 Nov 2019 15:30:40 +0900
Alexandre Courbot <acourbot@google.com> wrote:

> On Tue, Nov 12, 2019 at 7:53 PM Boris Brezillon
> <boris.brezillon@collabora.com> wrote:
> >
> > Hi Alexandre,
> >
> > On Tue, 12 Nov 2019 19:14:22 +0900
> > Alexandre Courbot <acourbot@google.com> wrote:
> >  
> > > Hi Boris and Ezequiel,
> > >
> > > Patch c3adb85745ca6 has removed the ref_pic_list_* members from the
> > > v4l2_ctrl_h264_decode_params struct. The MT8183 stateless decoder
> > > driver I am working on still relies on these lists and I am trying (a
> > > bit late to the game) to remove them from the Chromium OS kernel UAPI
> > > in order to match upstream.
> > >
> > > I have dug into the discussion that resulted in this removal and could
> > > not really find how these are supposed to be reconstructed and on the
> > > basis on which information. The commit log for the above-mentioned
> > > patch mentions that "generic helpers will be provided for drivers that
> > > need this list". I could not find any in the kernel so far, do you
> > > have any code available at the moment? Or any idea on how these can be
> > > reconstructed? The process seems to involve reading the DPB itself as
> > > well as reordering information from the slice parameters, and seems to
> > > be a bit involved to be done in the kernel, but maybe I am missing
> > > something here.  
> >
> > The code is here [1], and it should indeed be extracted and put in a
> > generic v4l2_h264 lib at some point (should happen soon, since we need
> > the same logic for the rkvdec driver).
> >
> > Let me know if you have any questions.  
> 
> Thanks - not sure how I missed it. I have tried adapting the code for
> MT8183 and it seems to be working perfectly there as well.

That's great news! Note that we have a fix queued to media/master, so
if you copy the code, take it from there.

> 
> Regarding the lib to make this code available to other drivers, I was
> thinking about doing it on top of
> https://patchwork.kernel.org/patch/11076405/ but is this patch still
> being worked on?

Unfortunately no. Hans was not in favor of this extra codec abstraction
layer and suggested to provide a set of helpers instead. But after
looking at it, there's not much that can be automated/shared if drivers
don't all share the same base object (maybe part of the ctrls handling
logic). Also had the same feedback from Paul K (reported privately on
IRC), so I decided to give up on this approach.

Regards,

Boris