Re: epoll with ONESHOT possibly fails to deliver events

[Phil Turmel <philip@turmel.org>]

On 12/13/2012 07:08 PM, Phil Turmel wrote:
> On 12/13/2012 04:32 AM, Eric Wong wrote:
>> Andreas Voellmy <andreas.voellmy@yale.edu> wrote:
> 
> [trim /]
> 
>>>> Another thread, distinct from all of the threads serving particular
>>>> sockets, is perfoming epoll_wait calls. When sockets are returned as
>>>> being ready from an epoll_wait call, the thread signals to the
>>>> condition variable for the socket.
>>
>> Perhaps there is a bug in the way your epoll_wait thread
>> uses the condition variable to notify other threads?
> 
> Have you considered the possibility that data is arriving between
> epoll_ctl and pthread_cond_wait ?  If your monitoring thread returns
> from epoll_wait within this race window, it will call
> pthread_cond_signal while the first thread is not yet waiting for it.
> With the one-shot flag, the next iteration of epoll_wait won't see that
> socket's new data.

Let me clarify:

The read thread must perform the epoll_ctl between pthread_mutex_lock
and pthread_cond_wait, while the monitoring thread must hold the mutex
lock when signaling.

pthread_cond_signal and pthread_cond_broadcast don't require the caller
to hold the mutex in general, but your app needs it.

Phil