Re: [RFC PATCH 0/3] media: rockchip: VEPU510 H.264 encoder for RK3576

[Nicolas Dufresne <nicolas.dufresne@collabora.com>]

[-- Attachment #1: Type: text/plain, Size: 2981 bytes --]

Hi,

Le jeudi 23 juillet 2026 à 12:46 +1200, Jiaxing Hu a écrit :
> I'm not attached to the stateful interface -- I used it to get something
> running on real silicon, not out of conviction. Before I commit to
> reworking, I'd like to be sure I understand what "converge" means here,
> because I see two fairly different directions in the thread: Paul's V4L2
> stateless H.264 encoder (kernel-side reflist/rbsp/RC core), and the
> Vulkan-Video split you describe for Detlev (thin kernel module + Mesa
> userspace). Those differ a lot in kind and in effort. Which is the
> intended target for the Rockchip encoders, or is that still open? Is
> there a branch, early code or spec draft I should read beyond Paul's
> series?

This is a fair point, and something that will certainly bring some level of
confusion in the short term. My personal goal, and what Detlev is actively
working on, is to move all codecs drivers toward Vulkan Video, using the most
meaningful driver interface for the purpose. The drivers needs to stay safe
though, so it matters that the HW can protect memory access.

Detlev and I picked the RKVENC because there was no upstream driver for it, so a
good place for a clean transition. It has an IOMMU and can ensure per process
memory access protection. If a process miss-program the encoder and make it
corrupt on other encoding session it owns, its all right, its not different then
writing a C program with multiple thread and corrupting another thread. But
corruption other process session, or the system is not acceptable, and something
Linux driver must protect against.

Paul does not have this hardware protection, so I think he needs a higher level
interface for the driver, and at the point, why not base it on existing kernel
interface. Though, V4l2 is massively larger (probably around 100x) interface
then what we are drafting currently for RKVENC in a drm style driver.

The common part is the rate control. Typically, stateful encoder are firmware
based, and the rate control is in the firmware. We need an in kernel rate
control for performance reason. The programming latency is going to be too high
otherwise. The point of convergence, is that we can share helpers for that
purpose, and Paul's work is really clean, and already in the shape of helpers
not tied to v4l2.

> Either way the RK3576 hardware enablement and the reference-read fix are
> shared with RK3588, so I'd much rather contribute the VEPU510 side to a
> common base than maintain a parallel one -- I'd just like to know which
> base that is before rebuilding on it.

This is our experience with decoder too, its relatively easy to support both,
differences being minor. I will ping Detlev today and see if he can help here. I
know in his case he was trying to avoid reconstructed frame compression, and it
only started working when he enable that compression. Though, he had mmu faults
prior to that.

cheers,
Nicolas

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]