Re: epoll: possible bug from wakeup_source activation

[Eric Wong <normalperson@yhbt.net>]

Arve Hjønnevåg <arve@android.com> wrote:
> On Mon, Mar 11, 2013 at 5:17 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Arve Hjønnevåg <arve@android.com> wrote:
> >> On Fri, Mar 8, 2013 at 11:10 PM, Eric Wong <normalperson@yhbt.net> wrote:
> >> > Arve Hjønnevåg <arve@android.com> wrote:
> >> >> On Fri, Mar 8, 2013 at 12:49 PM, Eric Wong <normalperson@yhbt.net> wrote:
> >> >> > What happens if ep_modify calls ep_destroy_wakeup_source
> >> >> > while __pm_stay_awake is running on the same epi->ws?
> >> >>
> >> >> Yes, that looks like a problem. I think calling
> >> >> ep_destroy_wakeup_source with ep->lock held should fix that. It is not
> >> >> clear how useful changing EPOLLWAKEUP in ep_modify is, so
> >> >> alternatively we could remove that feature and instead only allow it
> >> >> to be set in ep_insert.
> >> >
> >> > ep->lock would work, but ep->lock is already a source of heavy
> >> > contention in my multithreaded+epoll webservers.
> >>
> >> This should not have any significant impact on that since you would be
> >> adding a lock to a code path that is, as far as I know, unused.
> >>
> >> > Perhaps RCU can be used?  I've no experience with RCU, but I've been
> >> > meaning to get acquainted with RCU.
> >>
> >> That adds code to the common path however. The wakeup_source is not
> >> touch without holding one of the locks so holding both locks before
> >> deleting it seems like a simpler solution.
> >
> > True.  However, I've been looking into eliminating ep->lock in more
> > places (maybe entirely)[1].
> >
> > I don't think the current overhead of RCU in epoll is significant,
> > either.
> >
> >
> > [1] I'll be testing Mathieu's wait-free concurrent queue soon:
> >     http://mid.gmane.org/20130311213602.GB9829@Krystal
> 
> OK, but is there any way you could use the same locking scheme for the
> wakeup_source and the queue?

Probably, yes.  I think I can just use ep->mtx and ignore the mutex
included with wfcq_head, need to protect the rbtree while dequeueing.