Re: [take24 3/6] kevent: poll/select() notifications.

Davide Libenzi
2006-11-09T19:42:54Z

On Thu, 9 Nov 2006, Evgeniy Polyakov wrote:

> On Thu, Nov 09, 2006 at 10:51:56AM -0800, Davide Libenzi (davidel@xmailserver.org) wrote:
> > On Thu, 9 Nov 2006, Evgeniy Polyakov wrote:
> >
> > > +static int kevent_poll_callback(struct kevent *k)
> > > +{
> > > + if (k->event.req_flags & KEVENT_REQ_LAST_CHECK) {
> > > + return 1;
> > > + } else {
> > > + struct file *file = k->st->origin;
> > > + unsigned int revents = file->f_op->poll(file, NULL);
> > > +
> > > + k->event.ret_data[0] = revents & k->event.event;
> > > +
> > > + return (revents & k->event.event);
> > > + }
> > > +}
> >
> > You need to be careful that file->f_op->poll is not called inside the
> > spin_lock_irqsave/spin_lock_irqrestore pair, since (even this came up
> > during epoll developemtn days) file->f_op->poll might do a simple
> > spin_lock_irq/spin_unlock_irq. This unfortunate constrain forced epoll to
> > have a suboptimal double O(R) loop to handle LT events.
>
> It is tricky - users call wake_up() from any context, which in turn ends
> up calling kevent_storage_ready(), which calls kevent_poll_callback() with
> KEVENT_REQ_LAST_CHECK bit set, which becomes almost empty call in fast
> path. Since callback returns 1, kevent will be queued into ready queue,
> which is processed on behalf of syscalls - in that case kevent will
> check the flag and since KEVENT_REQ_LAST_CHECK is set, will call
> callback again to check if kevent is correctly marked, but already
> without that flag (it happens in syscall context, i.e. process context
> without any locks held), so callback calls ->poll(), which can sleep,
> but it is safe. If ->poll() returns 'ready' value, kevent is transfers
> data into userspace, otherwise it is 'requeued' (just removed from
> ready queue).

Oh, mine was only a general warn. I hadn't looked at the generic code
before. But now that I poke on it, I see:

void kevent_requeue(struct kevent *k)
{
unsigned long flags;

spin_lock_irqsave(&k->st->lock, flags);
__kevent_requeue(k, 0);
spin_unlock_irqrestore(&k->st->lock, flags);
}

and then:

static int __kevent_requeue(struct kevent *k, u32 event)
{
int ret, rem;
unsigned long flags;

ret = k->callbacks.callback(k);

Isn't the k->callbacks.callback() possibly end up calling f_op->poll?

- Davide