Re: performance of O_DIRECT on md/lvm

[Andi Kleen <ak@suse.de>]

Andrea Arcangeli <andrea@suse.de> writes:
> 
> if you read in chunks of a few mbytes per read syscall, the lack of
> readahead shouldn't make much difference (this is true for both raid and
> standalone device). If there's a relevant difference it's more liekly an
> issue with the blocksize.

The problem with that is that doing overlapping IO requires much more
effort (you need threads in user space). If you don't do overlapping
IO you add a latency bubble for each round trip to user space after you
read one big chunk and submitting the request for the next big chunk.
Your disk will not be constantly streaming, because of these pauses where
it doesn't have an request to process. 
The application could do it using some aio setup, but it gets rather
complicated and the kernel already knows how to do that well.

I think an optional readahead mode for O_DIRECT would be useful. 

-Andi