Re: [kvm-devel] [Qemu-devel] Re: [PATCH 1/3] Refactor AIO interface to allow other AIO implementations

[Jamie Lokier <jamie@shareable.org>]

Avi Kivity wrote:
> >At such a tiny difference, I'm wondering why Linux-AIO exists at all,
> >as it complicates the kernel rather a lot.  I can see the theoretical
> >appeal, but if performance is so marginal, I'm surprised it's in
> >there.
> 
> Linux aio exists, but that's all that can be said for it.  It works 
> mostly for raw disks, doesn't integrate with networking, and doesn't 
> advance at the same pace as the rest of the kernel.  I believe only 
> databases use it (and a userspace filesystem I wrote some time ago).

And video streaming on some embedded devices with no MMU!  (Due to the
page cache heuristics working poorly with no MMU, sustained reliable
streaming is managed with O_DIRECT and the app managing cache itself
(like a database), and that needs AIO to keep the request queue busy.
At least, that's the theory.)

> >I'm also surprised the Glibc implementation of AIO using ordinary
> >threads is so close to it.  
> 
> Why are you surprised?

Because I've read that Glibc AIO (which uses a thread pool) is a
relatively poor performer as AIO implementations go, and is only there
for API compatibility, not suggested for performance.

But I read that quite a while ago, perhaps it's changed.

> Actually the glibc implementation could be improved from what I've 
> heard.  My estimates are for a thread pool implementation, but there is 
> not reason why glibc couldn't achieve exactly the same performance.

Erm...  I thought you said it _does_ achieve nearly the same
performance, not that it _could_.

Do you mean it could achieve exactly the same performance by using
Linux AIO when possible?

> >And then, I'm wondering why use AIO it
> >all: it suggests QEMU would run about as fast doing synchronous I/O in
> >a few dedicated I/O threads.
> 
> Posix aio is the unix API for this, why not use it?

Because far more host platforms have threads than have POSIX AIO.  (I
suspect both options will end up supported in the end, as dedicated
I/O threads were already suggested for other things.)

> >>Also, I'd presume that those that need 10K IOPS and above will not place 
> >>their high throughput images on a filesystem; rather on a separate SAN 
> >>LUN.
> >
> >Does the separate LUN make any difference?  I thought O_DIRECT on a
> >filesystem was meant to be pretty close to block device performance.
> 
> On a good extent-based filesystem like XFS you will get good performance 
> (though more cpu overhead due to needing to go through additional 
> mapping layers.  Old clunkers like ext3 will require additional seeks or 
> a ton of cache (1 GB per 1 TB).

Hmm.  Thanks.  I may consider switching to XFS now....

-- Jamie