Re: Unifying epoll,aio,futexes etc. (What I really want from epoll)

John Gardiner Myers
2002-11-01T23:09:52Z

Matthew D. Hall wrote:

> * There is a seemingly significant overhead in performing exactly one
> callback per event.

The "exactly one callback per event" semantics of aio are important for
cancellation in thread pool environments. When you're shutting down a
connection, you need to be able to get to a point where you know no
other thread is processing or will process an event for the connection,
so it is safe to free the connection state.

> * Only one queue per process or kernel thread.

Having a single thread process multiple queues is not particularly
interesting (unless you have user-space threads or coroutines). Being
able to have different threads in the same process process different
queues is interesting--it permits a library to set up its own queue,
using its own threads to process it.

> * No re-arming events. They must be manually killed.

Rearming events is a useful way to get the correct cancellation
semantics in thread pool environments.

> - Should the kernel attempt to prune the queue of "cancelled" events
> (hints later deemed irrelevant, untrue, or obsolete by newer events)?

This makes the cancellation semantics much easier to deal with in single
threaded event loops. Single threaded cancellation is difficult in the
current aio interface because in the case where the canceled operation
already has an undelivered event in the queue, the canceling code has to
defer freeing the context until it receives that event.

An additional point: In a thread pool environment, you want event wakeup
to be in LIFO order and use wake-one semantics. You also want
concurrency control: don't deliver an event to a waiting thread if that
pool does not have fewer threads in runnable state than CPUs.