Re: Hantro H1 Encoding Upstreaming

[Nicolas Dufresne <nicolas.dufresne@collabora.com>]

Hi everyone,

despite Andrzej having left the community, we are not giving up on the encoder
work. In 2025, we aim at working more seriously on the V4L2 spec, as just
writing driver won't cut it. Each class of codecs needs a general workflow spec
similar to what we have already for stateful encoder/decoder and stateless
decoder.

- https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/dev-decoder.html
- https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/dev-encoder.html
- https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/dev-stateless-decoder.html

It is on top of this, that for each codec we have to add controls (mostly
compound) specifics and details that suites stateless accelerators.

From a community stand point, the most important focus is to write and agree on
spec and controls. Once we have that, vendors will be able to slowly move away
from their custom solution, and compete on actual hardware rather then
integration.

It is also time to start looking toward the future, since Hantro H1 is very
limited and ancient encoder. On same brand, if someone could work on VC8000E
shipped on IMX8M Plus, or Rockchip codecs, that will certainly help progress. We
can also get inspiration from many other stateless encoding APIs now, notably
VA, DXVA and Vulkan Video.

Of course, folks likes to know when this will happen, stateless decoders took 5
years from start to the first codec being merged, hopefully we don't beat that
record. I personally aim for producing work during the summer, and mostly focus
on the spec. Its obvious for me that testing on H1 with a GStreamer
implementation is the most productive, though I have strong interest in having
an ecosystem of drivers. A second userspace implementation, perhaps ffmpeg ?,
could also be useful.

If you'd like to take a bite, this is a good thread to discuss forward. Until
the summer, I planned to reach to Paul, who made this great presentation [1] at
FOSDEM last year and start moving the RFC into using these ideas. One of the
biggest discussion is rate control, it is clear to me that modern HW integrated
RC offloading, though some HW specific knobs or even firmware offloading, and
this is what Paul has been putting some thought into.

If decoders have progressed so much in quality in the last few years, it is
mostly before we have better ways to test them. It is also needed to start
thinking how do we want to test our encoders. The stateful scene is not all
green, with a very organic groth and difficult to unify set of encoders. And we
have no metric of how good or bad they are either.

regards,
Nicolas

Le lundi 13 janvier 2025 à 18:08 -0300, Daniel Almeida a écrit :
> +cc Nicolas
> 
> 
> Hey Adam,
> 
> 
> > 
> > Daniel,
> > 
> > Do you know if anyone will be picking up the H1 encoder?
> > 
> > adam
> > > 
> > > — Daniel
> > > 
> > 
> 
> I think my colleague Nicolas is the best person to answer this.
> 
> — Daniel