Re: [RFC PATCH 0/3] media: rockchip: VEPU510 H.264 encoder for RK3576

[Paul Kocialkowski <paulk@sys-base.io>]

[-- Attachment #1: Type: text/plain, Size: 4570 bytes --]

Hi,

Le Thu 23 Jul 26, 12:14, Nicolas Dufresne a écrit :
> Le jeudi 23 juillet 2026 à 12:46 +1200, Jiaxing Hu a écrit :
> > I'm not attached to the stateful interface -- I used it to get something
> > running on real silicon, not out of conviction. Before I commit to
> > reworking, I'd like to be sure I understand what "converge" means here,
> > because I see two fairly different directions in the thread: Paul's V4L2
> > stateless H.264 encoder (kernel-side reflist/rbsp/RC core), and the
> > Vulkan-Video split you describe for Detlev (thin kernel module + Mesa
> > userspace). Those differ a lot in kind and in effort. Which is the
> > intended target for the Rockchip encoders, or is that still open? Is
> > there a branch, early code or spec draft I should read beyond Paul's
> > series?

I would recommend taking a look at the slides of the talks I gave at the
Linux Media Summit last year and this year, which should give you more
context and insight about the design choices.

- https://paulk.fr/talks/2025-linux-media-summit/2025-linux-media-summit-v4l2-stateless-video-encoding-uapi.pdf
- https://paulk.fr/talks/2026-linux-media-summit/2026-linux-media-summit-v4l2-stateless-video-encoding-uapi-progress-update.pdf

Generally speaking the idea of this proposal is to stick with the same
design idea that we had for stateless decoders, including V4L2 M2M queues,
dedicated controls and media requests.

Like Nicolas explained, some hardware can be a good fit for both
approaches. I think we agreed that it wouldn't be a problem to have two
drivers (one v4l2, one drm-ish) for the same hardware, so the question is
really about what you prefer to do and get involved with.

My V4L2-based proposal is already rather usable, although it has not yet
reached its final form. I have a GStreamer implementation ready that has
been used extensively too, but it will also need to be reworked before
it can reach upstream.

The DRM-ish proposal is at an earlier stage and will likely require a
complete redesign of your driver, moving most things in userspace.

> Paul does not have this hardware protection, so I think he needs a higher level
> interface for the driver, and at the point, why not base it on existing kernel
> interface. Though, V4l2 is massively larger (probably around 100x) interface
> then what we are drafting currently for RKVENC in a drm style driver.
> 
> The common part is the rate control. Typically, stateful encoder are firmware
> based, and the rate control is in the firmware. We need an in kernel rate
> control for performance reason. The programming latency is going to be too high
> otherwise. The point of convergence, is that we can share helpers for that
> purpose, and Paul's work is really clean, and already in the shape of helpers
> not tied to v4l2.

Well there is some level of dependency to v4l2 in some places, but I'm
sure we can extract relevant parts and make them common so that a drm-ish
subsystem can reuse them easily.

> > Either way the RK3576 hardware enablement and the reference-read fix are
> > shared with RK3588, so I'd much rather contribute the VEPU510 side to a
> > common base than maintain a parallel one -- I'd just like to know which
> > base that is before rebuilding on it.
> 
> This is our experience with decoder too, its relatively easy to support both,
> differences being minor. I will ping Detlev today and see if he can help here. I
> know in his case he was trying to avoid reconstructed frame compression, and it
> only started working when he enable that compression. Though, he had mmu faults
> prior to that.

In my opinion it would be fine to have two drivers, but it's really up
to you. Since you already have a v4l2 base for it, I would encourage you
to move it to my proposal, which will probably be finalized in a shorter
time frame than the other proposal and lets you reuse the code you've
already written. Then you could also add RK3576 support to the drm-ish
Vulkan Video proposal without too much work when it is ready, reusing the
RK3588 work.

But again, it's really your decision. And it's hard to say how long
things will take to finalize. But I think it would be beneficial to have
two drivers for this hardware and not trash the work you've done.

All the best,

Paul

-- 
Paul Kocialkowski,

Independent contractor - sys-base - https://www.sys-base.io/
Free software developer - https://www.paulk.fr/

Expert in multimedia, graphics and embedded hardware support with Linux.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]