Exact behavior of the EOS event?

Exact behavior of the EOS event?

[Hans Verkuil]

Hi Kamil, Roy,

When implementing eos support in v4l2-ctl I started wondering about the
exact timings of that.

There are two cases, polling and non-polling, and I'll explain how I do it
now in v4l2-ctl.

Polling case:

I select for both read and exceptions. When the select returns I check
for exceptions and call DQEVENT, which may return EOS.

If there is something to read then I call DQBUF to get the frame, process
it and afterwards exit the capture loop if the EOS event was seen.

This procedure assumes that setting the event and making the last frame
available to userspace happen atomically, otherwise you can get a race
condition.

Non-polling case:

I select for an exception with a timeout of 0 (i.e. returns immediately),
then I call DQBUF (which may block), process the frame and exit if EOS was
seen.

I suspect this is wrong, since when I call select the EOS may not be set
yet, but it is after the DQBUF. So in the next run through the capture loop
I capture one frame too many.


What I think is the correct sequence is to first select for a read(), but not
exceptions, then do the DQBUF, and finally do a select for exceptions with
a timeout of 0. If EOS was seen, then that was the last frame.

A potential problem with that might be when you want to select on other
events as well. Then you would select on both read and exceptions, and we
end up with a potential race condition again. The only solution I see is to
open a second filehandle to the video node and subscribe to the EOS event
only for that filehandle and use that to do the EOS polling.

It all feels rather awkward.

Kamil, Roy, any ideas/suggestions to improve this?

Regards,

	Hans

Re: Exact behavior of the EOS event?

[Tzu-Jung Lee]

Hi Hans,

On Thu, Apr 11, 2013 at 5:40 PM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
> Hi Kamil, Roy,
>
> When implementing eos support in v4l2-ctl I started wondering about the
> exact timings of that.
>
> There are two cases, polling and non-polling, and I'll explain how I do it
> now in v4l2-ctl.
>
> Polling case:
>
> I select for both read and exceptions. When the select returns I check
> for exceptions and call DQEVENT, which may return EOS.
>
> If there is something to read then I call DQBUF to get the frame, process
> it and afterwards exit the capture loop if the EOS event was seen.
>
> This procedure assumes that setting the event and making the last frame
> available to userspace happen atomically, otherwise you can get a race
> condition.
>
> Non-polling case:
>
> I select for an exception with a timeout of 0 (i.e. returns immediately),
> then I call DQBUF (which may block), process the frame and exit if EOS was
> seen.
>
> I suspect this is wrong, since when I call select the EOS may not be set
> yet, but it is after the DQBUF. So in the next run through the capture loop
> I capture one frame too many.
>
>
> What I think is the correct sequence is to first select for a read(), but not
> exceptions, then do the DQBUF, and finally do a select for exceptions with
> a timeout of 0. If EOS was seen, then that was the last frame.
>
> A potential problem with that might be when you want to select on other
> events as well. Then you would select on both read and exceptions, and we
> end up with a potential race condition again. The only solution I see is to
> open a second filehandle to the video node and subscribe to the EOS event
> only for that filehandle and use that to do the EOS polling.
>
> It all feels rather awkward.
>
> Kamil, Roy, any ideas/suggestions to improve this?

I was thinking to serialize this within the driver, which means no buffers
on capture side will be DQBUF once the EOS is generated.

I'm not sure is this generic enough, but it should make life easier
for applications

Regards,
Roy

RE: Exact behavior of the EOS event?

[Kamil Debski]

Hi Hans,

> -----Original Message-----
> From: Hans Verkuil [mailto:hverkuil@xs4all.nl]
> Sent: Thursday, April 11, 2013 11:41 AM
> To: k.debski@samsung.com
> Cc: linux-media@vger.kernel.org; Tzu-Jung Lee
> Subject: Exact behavior of the EOS event?
> 
> Hi Kamil, Roy,
> 
> When implementing eos support in v4l2-ctl I started wondering about the
> exact timings of that.
> 
> There are two cases, polling and non-polling, and I'll explain how I do
> it now in v4l2-ctl.
> 
> Polling case:
> 
> I select for both read and exceptions. When the select returns I check
> for exceptions and call DQEVENT, which may return EOS.
> 
> If there is something to read then I call DQBUF to get the frame,
> process it and afterwards exit the capture loop if the EOS event was
> seen.
> 
> This procedure assumes that setting the event and making the last frame
> available to userspace happen atomically, otherwise you can get a race
> condition.
> 
> Non-polling case:
> 
> I select for an exception with a timeout of 0 (i.e. returns
> immediately), then I call DQBUF (which may block), process the frame
> and exit if EOS was seen.
> 
> I suspect this is wrong, since when I call select the EOS may not be
> set yet, but it is after the DQBUF. So in the next run through the
> capture loop I capture one frame too many.
> 
> 
> What I think is the correct sequence is to first select for a read(),
> but not exceptions, then do the DQBUF, and finally do a select for
> exceptions with a timeout of 0. If EOS was seen, then that was the last
> frame.
> 
> A potential problem with that might be when you want to select on other
> events as well. Then you would select on both read and exceptions, and
> we end up with a potential race condition again. The only solution I
> see is to open a second filehandle to the video node and subscribe to
> the EOS event only for that filehandle and use that to do the EOS
> polling.

This would work if we have a single context only. In case of mem2mem
devices, where there is a separate context for each file this would not
work.

> 
> It all feels rather awkward.
> 
> Kamil, Roy, any ideas/suggestions to improve this?
> 
> Regards,
> 
> 	Hans

-- 
Kamil Debski
Linux Platform Group
Samsung Poland R&D Center

Re: Exact behavior of the EOS event?

[Hans Verkuil]

On Thu 11 April 2013 14:39:44 Kamil Debski wrote:
> Hi Hans,
> 
> > -----Original Message-----
> > From: Hans Verkuil [mailto:hverkuil@xs4all.nl]
> > Sent: Thursday, April 11, 2013 11:41 AM
> > To: k.debski@samsung.com
> > Cc: linux-media@vger.kernel.org; Tzu-Jung Lee
> > Subject: Exact behavior of the EOS event?
> > 
> > Hi Kamil, Roy,
> > 
> > When implementing eos support in v4l2-ctl I started wondering about the
> > exact timings of that.
> > 
> > There are two cases, polling and non-polling, and I'll explain how I do
> > it now in v4l2-ctl.
> > 
> > Polling case:
> > 
> > I select for both read and exceptions. When the select returns I check
> > for exceptions and call DQEVENT, which may return EOS.
> > 
> > If there is something to read then I call DQBUF to get the frame,
> > process it and afterwards exit the capture loop if the EOS event was
> > seen.
> > 
> > This procedure assumes that setting the event and making the last frame
> > available to userspace happen atomically, otherwise you can get a race
> > condition.
> > 
> > Non-polling case:
> > 
> > I select for an exception with a timeout of 0 (i.e. returns
> > immediately), then I call DQBUF (which may block), process the frame
> > and exit if EOS was seen.
> > 
> > I suspect this is wrong, since when I call select the EOS may not be
> > set yet, but it is after the DQBUF. So in the next run through the
> > capture loop I capture one frame too many.
> > 
> > 
> > What I think is the correct sequence is to first select for a read(),
> > but not exceptions, then do the DQBUF, and finally do a select for
> > exceptions with a timeout of 0. If EOS was seen, then that was the last
> > frame.
> > 
> > A potential problem with that might be when you want to select on other
> > events as well. Then you would select on both read and exceptions, and
> > we end up with a potential race condition again. The only solution I
> > see is to open a second filehandle to the video node and subscribe to
> > the EOS event only for that filehandle and use that to do the EOS
> > polling.
> 
> This would work if we have a single context only. In case of mem2mem
> devices, where there is a separate context for each file this would not
> work.

True.

Another idea was to set an EOS buffer flag for the last buffer, but I think
I remember that your driver won't know it is the last one until later, right?

Perhaps we should implement the EOS buffer flag idea after all. If that flag
is set, then if the buffer is empty, then that buffer should be discarded,
if it is not, then that was the last buffer.

The EOS event was originally designed for a decoder where you want to know
when the decoder finished decoding your last frame.

It's now being used for capture streams were it is not a good fit, IMHO.

Regards,

	Hans

Re: Exact behavior of the EOS event?

[Andrzej Hajda]

Hi Hans, Kamil and Roy,

On 11.04.2013 14:51, Hans Verkuil wrote:
> On Thu 11 April 2013 14:39:44 Kamil Debski wrote:
>> Hi Hans,
>>
>>> -----Original Message-----
>>> From: Hans Verkuil [mailto:hverkuil@xs4all.nl]
>>> Sent: Thursday, April 11, 2013 11:41 AM
>>> To: k.debski@samsung.com
>>> Cc: linux-media@vger.kernel.org; Tzu-Jung Lee
>>> Subject: Exact behavior of the EOS event?
>>>
>>> Hi Kamil, Roy,
>>>
>>> When implementing eos support in v4l2-ctl I started wondering about the
>>> exact timings of that.
>>>
>>> There are two cases, polling and non-polling, and I'll explain how I do
>>> it now in v4l2-ctl.
>>>
>>> Polling case:
>>>
>>> I select for both read and exceptions. When the select returns I check
>>> for exceptions and call DQEVENT, which may return EOS.
>>>
>>> If there is something to read then I call DQBUF to get the frame,
>>> process it and afterwards exit the capture loop if the EOS event was
>>> seen.
>>>
>>> This procedure assumes that setting the event and making the last frame
>>> available to userspace happen atomically, otherwise you can get a race
>>> condition.
>>>
>>> Non-polling case:
>>>
>>> I select for an exception with a timeout of 0 (i.e. returns
>>> immediately), then I call DQBUF (which may block), process the frame
>>> and exit if EOS was seen.
>>>
>>> I suspect this is wrong, since when I call select the EOS may not be
>>> set yet, but it is after the DQBUF. So in the next run through the
>>> capture loop I capture one frame too many.
>>>
>>>
>>> What I think is the correct sequence is to first select for a read(),
>>> but not exceptions, then do the DQBUF, and finally do a select for
>>> exceptions with a timeout of 0. If EOS was seen, then that was the last
>>> frame.
>>>
>>> A potential problem with that might be when you want to select on other
>>> events as well. Then you would select on both read and exceptions, and
>>> we end up with a potential race condition again. The only solution I
>>> see is to open a second filehandle to the video node and subscribe to
>>> the EOS event only for that filehandle and use that to do the EOS
>>> polling.
>>
>> This would work if we have a single context only. In case of mem2mem
>> devices, where there is a separate context for each file this would not
>> work.
>
> True.
>
> Another idea was to set an EOS buffer flag for the last buffer, but I think
> I remember that your driver won't know it is the last one until later, right?
>
> Perhaps we should implement the EOS buffer flag idea after all. If that flag
> is set, then if the buffer is empty, then that buffer should be discarded,
> if it is not, then that was the last buffer.
>
> The EOS event was originally designed for a decoder where you want to know
> when the decoder finished decoding your last frame.
>
> It's now being used for capture streams were it is not a good fit, IMHO.

After rejecting my RFC with EOS flag on buffers about year ago I have 
implemented EOS in MFC encoder using v4l2 events. In my implementation 
EOS event is sent always AFTER the last buffer of the stream was 
dequeued to the application. Additionally if there is a buffer in DQBUF, 
driver marks it done with payload 0. This way apps are able to work in 
both polling and non-polling mode.

Anyway EOS using events seems to be more difficult/error prone to both 
drivers and apps.

Thinking about it on higher level of abstraction end-of-stream as a 
concept IMO better fits to stream/queue than to asynchronous events.

>
> Regards,
>
> 	Hans
>

Regards
Andrzej