Re: Is user-space AIO dead?

[Phillip Susi <psusi@cfl.rr.com>]

It is not possible to use non blocking IO with O_DIRECT, because the 
kernel does not buffer the data, and once the write() call returns, the 
kernel can not touch the caller's buffer any more.  The idea of O_DIRECT 
is that the hardware can directly DMA from the caller's buffer, so if 
you want to keep the hardware busy, you need to use async IO so the 
hardware always has some work to do. 

I actually hacked up dd to use async IO ( via io_submit ) in conjunction 
with O_DIRECT and it did noticeably improve ( ~10% ish ) both throughput 
and cpu utilization.  I have an OO.o spreadsheet showing the results of 
some simple benchmarking with various parameters I did at home, which I 
will post later this evening. 

Of course, dd is a simplistic case of sequential IO.  If you have 
something like a big database that needs to concurrently handle dozens 
or hundreds of random IO requests at once, O_DIRECT async IO is 
definitely going to be a clear winner. 

Kenny Simpson wrote:
> Right, but I'm not sure O_DIRECT implies stable storage, only data sent out to the device, not
> held up in the page cache (I could be wrong).
>
> AIO is implemented for O_DIRECT according to the paper, but they observed it not having benefit.
>
> AIO being implemented to O_SYNC would be nice for my use, as it would also eliminate the extra
> alignment restrictions brought on by O_DIRECT.
>
> -Kenny
>
>