Re: [Qemu-devel] [RFC 2/2] bdrv_aio_readv/writev_em

[Jamie Lokier <jamie@shareable.org>]

Andrea Arcangeli wrote:
> pthread_create() and do aio with pthreads by calling writev by hand.
> 
> Use kernel based linux aio (I think it's much better as it won't
> screwup with contiguous I/O, and it handles o_direct random writes and
> random reads by keeping the lowlevel I/O pipeline full without threads
> but by just queuing _in_order_ [in order only from the point of view
> of the I/O scheduler of course] and asynchronously the commands of
> every different direct-io aio_readv/writev in the lowlevel storage
> queue without needing any scheduler and thread synchronization
> involvement).

The good things about linux-aio are as you described.

The bad thing is that I'm told linux-aio can block sometimes.  Apart
from memory pressure (which can block the calling process anyway), an
aio implementation which can block is not so good.

Doing it in a pthread is less good for a lot of reasons.  On the other
hand, I/O in a pthread should never block the main thread.  We
discussed the "in order" part a while ago, and several people pointed
out that once there are a few I/Os in flight, the order should
stabilise automatically, but perhaps that's not so good in transient
situations.

(Ideally we'd use scheduler activations (syslets!) to avoid blocking
in any kernel I/O path no matter how complex, while keeping aio
ordering and avoiding unnecessary task switching.  I haven't heard
much about progress on this since earlier this year...)

-- Jamie