Re: [ANN] Meeting to discuss improvements to support MC-based cameras on generic apps

[Laurent Pinchart <laurent.pinchart@ideasonboard.com>]

Hi Nicolas,

On Friday, 18 May 2018 17:22:47 EEST Nicolas Dufresne wrote:
> Le vendredi 18 mai 2018 à 11:15 +0300, Laurent Pinchart a écrit :
> >> I need to clarify a little bit on why we disabled libv4l2 in GStreamer,
> >> as it's not only for performance reason, there is couple of major issues
> >> in the libv4l2 implementation that get's in way. Just a short
> >> list:
> > 
> > Do you see any point in that list that couldn't be fixed in libv4l ?
> 
> Sure, most of it is features being added into the kernel uAPI but not
> added to the emulation layer. But appart from that, libv4l will only
> offer legacy use case, we need to think how generic userspace will be
> able to access these camera, and leverage the per request controls,
> multi-stream, etc. features. This is mostly what Android Camera HAL2
> does (and it does it well), but it won't try and ensure this stays Open
> Source in any ways. I would not mind if Android Camera HAL2 leads the
> way, and a facilitator (something that does 90% of the work if you have
> a proper Open Source driver) would lead the way in getting more OSS
> drivers submitted.

There are a few issues with the Android camera HAL that make implementations 
very painful. If we were to model a camera stack on such an API, we should at 
least fix those. The biggest issue in my opinion is that the HAL mandates that 
a request captures to a specific buffer, making implementations very complex, 
and requiring memcpy() from time to time when losing race conditions. It would 
be much simpler to instead require capture to any buffer from a given pool.

> >>    - Crash when CREATE_BUFS is being used
> 
> This is a side effect of CREATE_BUFS being passed-through, implementing
> emulation for this should be straightforward.
> 
> >>    - Crash in the jpeg decoder (when frames are corrupted)
> 
> A minimalist framing parser would detect just enough of this, and would
> fix it.
> 
> >>    - App exporting DMABuf need to be aware of emulation, otherwise the
> >>      DMABuf exported are in the orignal format
> 
> libv4l2 can return ENOTTY to expbufs calls in
> 
> >>    - RW emulation only initialize the queue on first read (causing
> >>      userspace poll() to fail)
> 
> This is not fixable, only place it would be fixed is by moving this
> emulation into VideoBuf2. That would assume someone do care about RW
> (even though it could be nicer uAPI when dealing with muxed or byte-
> stream type of data).

I personally don't care much about R/W. I have heard that the recent 
implementation of mmap'ed buffer support in DVB showed impressive performance 
improvements, so a R/W API isn't something I'd prioritize.

> > >    - Signature of v4l2_mmap does not match mmap() (minor)
> > >    - The colorimetry does not seem emulated when conversion
> 
> This one is probably tricky, special is the converter plugin API is
> considered stable. Maybe just resetting everything to DEFAULT would
> work ?

I'd actually like to rework that API. The conversion code should be moved to a 
separate library that would allow its usage without a V4L2 device.

> > >    - Sub-optimal locking (at least deadlocks were fixed)
> 
> Need more investigation really, and proper measurement.

-- 
Regards,

Laurent Pinchart