Re: [PATCH] readahead even for FMODE_RANDOM

[Wu Fengguang <fengguang.wu@intel.com>]

On Fri, Apr 02, 2010 at 02:59:17PM +0800, Jens Axboe wrote:
> On Fri, Apr 02 2010, Wu Fengguang wrote:
> > On Fri, Apr 02, 2010 at 02:38:30PM +0800, Jens Axboe wrote:
> > > On Fri, Apr 02 2010, Wu Fengguang wrote:
> > > > Hi Jens,
> > > > 
> > > > On Fri, Apr 02, 2010 at 02:31:51AM +0800, Jens Axboe wrote:
> > > > > Hi,
> > > > > 
> > > > > I got a problem report with fio where larger block size random reads
> > > > > where markedly slower with buffered IO than with O_DIRECT, and the
> > > > > initial thought was that perhaps this was some fio oddity. The reporter
> > > > > eventually discovered that turning off the fadvise hint made it work
> > > > > fine. So I took a look, and it seems we never do readahead for
> > > > > FMODE_RANDOM even if the request size is larger than 1 page. That seems
> > > > > like a bug, if an application is doing eg 16kb random reads, you want to
> > > > > readahead the 12kb remaining data. On devices where smaller transfer
> > > > > sizes are slower than larger ones, this can make a large difference.
> > > > > 
> > > > > This patch makes us readahead even for FMODE_RANDOM, iff we'll be
> > > > > reading more pages in that single read. I ran a quick test here, and it
> > > > > appears to fix the problem (no difference with fadvise POSIX_FADV_RANDOM
> > > > > being passed in or not).
> > > >  
> > > > I guess the application is doing (at least partial) sequential reads,
> > > > while at the same time tell kernel with POSIX_FADV_RANDOM that it's
> > > > doing random reads.
> > > > 
> > > > If so, it's mainly the application's fault.
> > > 
> > > The application is doing large random reads. It's purely random, so
> > > the POSIX_FADV_RANDOM hint is correct. However, thinking about it, it
> > 
> > How large is it? For random reads > read_ahead_kb,
> > ondemand_readahead() will break it into read_ahead_kb sized IOs, while
> > force_page_cache_readahead() won't. That may impact IO performance.
> 
> The test case was 128kb random reads. So should still be within the
> normal read_ahead_kb. I suspect the reporter would not have noticed if

Yeah. 128kb random reads won't trigger readahead.

However each 129kb random read will trigger 2*128kb readahead IOs,
if we let ondemand_readahead() handle these random reads..

> the issue size was as large as read_ahead_kb even if the request size
> was larger, the problem was that he ended up seeing 4kb ios only.
> 
> > > may be that we later hit a random "block" that has now been cached due
> > > to this read-ahead. Let me try and rule that out completely and see if
> > > there's still the difference. The original reporter observed 4kb reads
> > > hitting the driver, where 128kb was expected.
> > 
> > 4kb reads hit the disk (on POSIX_FADV_RANDOM)? That sounds like
> > behavior in pre .34 kernels that is fixed by commit 0141450f66c:
> > 
> >     readahead: introduce FMODE_RANDOM for POSIX_FADV_RANDOM
> 
> Could explain why I'm not reproducing when doing a quick test on the
> laptop. It is an older kernel. So it could be that I'm just imaging the
> issue on the current kernel, I don't have hard data to back it up on
> that version.
> 
> So disregard the patch for now, part-sequential behaviour on
> POSIX_FADV_RANDOM isn't the issue here.

OK.

Thanks,
Fengguang