Re: Single- vs Multi-planar APIs for mem2mem devices

[Paul Kocialkowski <paul.kocialkowski@bootlin.com>]

Hi,

On Thu 11 Jul 19, 17:21, wens Tsai wrote:
> I noticed that recent codec driver additions, such as hantro, meson/vdec,
> mtk, used the multi-planar API (mplane) instead of the single-planar API.
> 
> Is there a preference for moving towards mplane?

Well, there is strict technical requirement for using the single-planar API
instead of the multi-planar one in cedrus.

Historically, we started with mplane because we can definitely pass
different addresses for luma and chroma to our decoder. However, we found
out that there are some internal limitations of the decoder block and the
addresses cannot be "too far apart". This means that if we allocate luma
at the begging of our pool and chroma at the end, the gap was too big
and the resulting decoded frame was not stored at the right address.

We found out about this purely by accident, where one particular sequence of
events lead to such a gap and showed the issue. One possible explanation
would be that the decoder uses an offset from the luma address for chroma
internally, on a limited number of bits.

So unfortunately, this means we're stuck with having our multi-planar
(in the YUV sense) buffers allocated in a single chunk (the single-plane
V4L2 API). I don't have any better answer than "userspace should cope with both
APIs as it reflects a hardware constraint", which is indeed what ffmpeg is
already doing.

Cheers,

Paul

> Also I noticed that v4l-utils has a plugin that seamlessly adapts mplane
> devices to work with single-planar API calls, but there isn't one the
> other way around. Would that be something worthwhile working on? To me
> it seems having one central adapter is better than having applications
> try to use both APIs, though some such as FFmpeg already do so.
> 
> My interest in this is trying to get Chromium's v4l2 video decode
> accelerator to work on the Raspberry Pi 3 with their downstream kernel,
> which has the bcm2835-v4l2 driver. It would also benefit other platforms
> that have a stateful VPU, such as Amlogic (meson) or Mediatek SoCs.
> Chromium's code uses MPLANE exclusively, and it's geared towards ChromeOS,
> not standard Linux, but still it might be interesting to get it to work.
> 
> There's also the v4l2 slice video decode accelerator which uses the
> request API which has the same issue, but lets not get ahead of ourselves.
> 
> Regards
> ChenYu

-- 
Paul Kocialkowski, Bootlin
Embedded Linux and kernel engineering
https://bootlin.com