LibV4L2 and CREATE_BUFS issues

LibV4L2 and CREATE_BUFS issues

[Nicolas Dufresne]

Hi,

we recently fixed our CREATE_BUFS support in GStreamer master. It works
nicely with UVC drivers. The problem is that libv4l2 isn't aware of it,
and endup taking terribly decision the least quickly lead to crash.

I'm not sure what that right approach. It seems non-trivial to support
it, at least it would require a bit more knowledge of the converter code
and memory model. Maybe we should at least make sure that CREATE_BUF
fails if we are doing conversion ? Some input on that would be appreciated.

cheers,
Nicolas

Re: LibV4L2 and CREATE_BUFS issues

[Hans de Goede]

Hi,

On 10-12-14 17:27, Nicolas Dufresne wrote:
> Hi,
>
> we recently fixed our CREATE_BUFS support in GStreamer master. It works
> nicely with UVC drivers. The problem is that libv4l2 isn't aware of it,
> and endup taking terribly decision the least quickly lead to crash.
>
> I'm not sure what that right approach. It seems non-trivial to support
> it, at least it would require a bit more knowledge of the converter code
> and memory model. Maybe we should at least make sure that CREATE_BUF
> fails if we are doing conversion ? Some input on that would be appreciated.

I think making CREATE_BUFS fail when doing conversion is probably best,
note that gstreamer should be able to tell which formats will lead to doing
conversion, and that it can try to avoid those.

Regards,

Hans

Re: LibV4L2 and CREATE_BUFS issues

[Nicolas Dufresne]

Le 2014-12-13 05:41, Hans de Goede a écrit :
> I think making CREATE_BUFS fail when doing conversion is probably best,
> note that gstreamer should be able to tell which formats will lead to 
> doing
> conversion, and that it can try to avoid those. 

Those format indeed have a flag. The problem is for HW specific format, 
like few bayers format, which we can't avoid if we need to use such camera.

cheers,
Nicolas

Re: LibV4L2 and CREATE_BUFS issues

[Hans de Goede]

Hi,

On 13-12-14 17:15, Nicolas Dufresne wrote:
>
> Le 2014-12-13 05:41, Hans de Goede a écrit :
>> I think making CREATE_BUFS fail when doing conversion is probably best,
>> note that gstreamer should be able to tell which formats will lead to doing
>> conversion, and that it can try to avoid those.
>
> Those format indeed have a flag. The problem is for HW specific format, like few bayers format, which we can't avoid if we need to use such camera.

Ah yes I see, so I assume that if libv4l where to return a failure for
CREATE_BUFS when conversion is used, that gstreamer will then fallback to
a regular REQUEST_BUFS call ?

Then that indeed seems the best solution, can you submit patch for this ?

Regards,

Hans

Re: LibV4L2 and CREATE_BUFS issues

[Nicolas Dufresne]

Le 2014-12-14 04:49, Hans de Goede a écrit :
> Ah yes I see, so I assume that if libv4l where to return a failure for
> CREATE_BUFS when conversion is used, that gstreamer will then fallback to
> a regular REQUEST_BUFS call ?
>
> Then that indeed seems the best solution, can you submit patch for this ? 

Exactly, that should work. My concern with application side workaround 
would that the day someone implements CREATE_BUF support in v4l2 this 
application won't benefit without patching. I'll see if I can find time, 
disabling it seems faster then implementing support for it, specially 
that current experiment show that the jpeg code is really fragile. 
Current state is that libv4l2 is causing a buffer overflow, so it is 
harmful library in that sense.

This raise a concern, it would mean that USERPTR, DMABUF, CREATE_BUFS 
will now be lost (in most cases) when enabling libv4l2. This is getting 
a bit annoying. Specially that we are pushing forward having m2m 
decoders to only be usable through libv4l2 (HW specific parsers). Is 
there a long term plan or are we simply pushing the dust toward libv4l2 ?

cheers,
Nicolas

Re: LibV4L2 and CREATE_BUFS issues

[Hans de Goede]

Hi,

On 14-12-14 15:24, Nicolas Dufresne wrote:
>
> Le 2014-12-14 04:49, Hans de Goede a écrit :
>> Ah yes I see, so I assume that if libv4l where to return a failure for
>> CREATE_BUFS when conversion is used, that gstreamer will then fallback to
>> a regular REQUEST_BUFS call ?
>>
>> Then that indeed seems the best solution, can you submit patch for this ?
>
> Exactly, that should work. My concern with application side workaround would that the day someone implements CREATE_BUF support in v4l2 this application won't benefit without patching. I'll see if I can find time, disabling it seems faster then implementing support for it, specially that current experiment show that the jpeg code is really fragile. Current state is that libv4l2 is causing a buffer overflow, so it is harmful library in that sense.

Hmm, is that jpeg overflow still there with my recent (aprok 2-3 weeks ago) fix
for this?

> This raise a concern, it would mean that USERPTR, DMABUF, CREATE_BUFS will now be lost (in most cases) when enabling libv4l2.

Yes, which is not good.

> This is getting a bit annoying. Specially that we are pushing forward having m2m decoders to only be usable through libv4l2 (HW specific parsers). Is there a long term plan or are we simply pushing the dust toward libv4l2 ?

I think that trying to bold support for all of this into libv4l2 is not necessarily
a good idea. Then again if we're going to use libv4l2 plugins to do things like
media-controller pipeline setups for apps which are not media-controller aware,
maybe it is ...

libv4l2 was mostly created to get the then current generation of v4l2 apps to work
with webcams which have funky formats without pushing fmt conversion into the kernel
as several out of tree drivers were doing.

It may be better to come up with a better API for libv4lconvert, and let apps which
want to do advanced stuff deal with conversion themselves, while keeping all the
conversion code in a central place, but that does leave the media-controller issue.

Note that I've aprox. 0 time to work on libv4l now a days ...

What we really need is an active libv4l maintainer. Do not get me wrong, Gregor
has been doing a great job at maintaining it, but if we want to do some architectural
rework (or just a complete rewrite) I think we need someone who knows the v4l2 API,
media-controller, etc. a lot better.

Regards,

Hans

Re: LibV4L2 and CREATE_BUFS issues

[Nicolas Dufresne]

Le 2014-12-15 10:58, Hans de Goede a écrit :
> Hi,
>
> Hmm, is that jpeg overflow still there with my recent (aprok 2-3 weeks 
> ago) fix
> for this?
I'll need to check, might have been my fault too, since I was trying to 
reallocate the frames segment to implement support for CREATE_BUFS, 
without proper knowledge of the code. If we could have 1 allocation per 
frame, it would be trivial to implement. Could be made as a rework 
first. I wouldn't not worry too much for now, I apology for the noise.

>> This is getting a bit annoying. Specially that we are pushing forward 
>> having m2m decoders to only be usable through libv4l2 (HW specific 
>> parsers). Is there a long term plan or are we simply pushing the dust 
>> toward libv4l2 ?
>
> I think that trying to bold support for all of this into libv4l2 is 
> not necessarily
> a good idea. Then again if we're going to use libv4l2 plugins to do 
> things like
> media-controller pipeline setups for apps which are not 
> media-controller aware,
> maybe it is ...
>
> libv4l2 was mostly created to get the then current generation of v4l2 
> apps to work
> with webcams which have funky formats without pushing fmt conversion 
> into the kernel
> as several out of tree drivers were doing.
>
> It may be better to come up with a better API for libv4lconvert, and 
> let apps which
> want to do advanced stuff deal with conversion themselves, while 
> keeping all the
> conversion code in a central place, but that does leave the 
> media-controller issue.
>
> Note that I've aprox. 0 time to work on libv4l now a days ...
>
> What we really need is an active libv4l maintainer. Do not get me 
> wrong, Gregor
> has been doing a great job at maintaining it, but if we want to do 
> some architectural
> rework (or just a complete rewrite) I think we need someone who knows 
> the v4l2 API,
> media-controller, etc. a lot better.
>
Thanks for this information, and thanks for the effort so far. I didn't 
want to make this comment discouraging. I do cheer anyone taking care 
right now, and anyone that would come up next. As you mention, there has 
been plan (and currently active work) toward depending on libv4l to 
support cameras that need media-controller, decoders that need parsers, 
etc. I think these ideas are fine, but when bringing these ideas we 
should care more of how we are doing to add and test these in libv4l2, 
in a way that it's all very usable.

cheers,
Nicolas