Re: [PATCH 3/3] readahead: introduce context readahead algorithm

Jeff Moyer
2009-04-15T03:44:28Z

Wu Fengguang <fengguang.wu@intel.com> writes:

> Introduce page cache context based readahead algorithm.
> This is to better support concurrent read streams in general.
>
> RATIONALE
> ---------
> The current readahead algorithm detects interleaved reads in a _passive_ way.
> Given a sequence of interleaved streams 1,1001,2,1002,3,4,1003,5,1004,1005,6,...
> By checking for (offset == prev_offset + 1), it will discover the sequentialness
> between 3,4 and between 1004,1005, and start doing sequential readahead for the
> individual streams since page 4 and page 1005.
>
> The context readahead algorithm guarantees to discover the sequentialness no
> matter how the streams are interleaved. For the above example, it will start
> sequential readahead since page 2 and 1002.
>
> The trick is to poke for page @offset-1 in the page cache when it has no other
> clues on the sequentialness of request @offset: if the current requenst belongs
> to a sequential stream, that stream must have accessed page @offset-1 recently,
> and the page will still be cached now. So if page @offset-1 is there, we can
> take request @offset as a sequential access.
>
> BENEFICIARIES
> -------------
> - strictly interleaved reads i.e. 1,1001,2,1002,3,1003,...
> the current readahead will take them as silly random reads;
> the context readahead will take them as two sequential streams.
>
> - cooperative IO processes i.e. NFS and SCST
> They create a thread pool, farming off (sequential) IO requests to different
> threads which will be performing interleaved IO.
>
> It was not easy(or possible) to reliably tell from file->f_ra all those
> cooperative processes working on the same sequential stream, since they will
> have different file->f_ra instances. And NFSD's file->f_ra is particularly
> unusable, since their file objects are dynamically created for each request.
> The nfsd does have code trying to restore the f_ra bits, but not satisfactory.

Hi, Wu,

I tested out your patches. Below are some basic iozone numbers for a
single NFS client reading a file. The iozone command line is:

iozone -s 2000000 -r 64 -f /mnt/test/testfile -i 1 -w

The file system is unmounted after each run to flush the cache. The
numbers below reflect only a single run each. The file system was also
unmounted on the NFS client after each run.

KEY
---
vanilla: 2.6.30-rc1
readahead: 2.6.30-rc1 + your 10 readahead patches
context readahead: 2.6.30-rc1 + your 10 readahead patches + the 3
context readahead patches.
nfsd's: number of NFSD threads on the server

I'll note that the cfq in 2.6.30-rc1 is crippled, and that Jens has a
patch posted that makes the numbers look at least a little better, but
that's immaterial to this discussion, I think.

vanilla

nfsd's | 1 | 2 | 4 | 8
--------+---------------+-------+------
cfq | 43127 | 22354 | 20858 | 21179
deadline| 43732 | 68059 | 76659 | 83231

readahead

nfsd's | 1 | 2 | 4 | 8
--------+---------------+-------+------
cfq | 42471 | 21913 | 21252 | 20979
deadline| 42801 | 70158 | 82068 | 82406

context readahead

nfsd's | 1 | 2 | 4 | 8
--------+---------------+-------+------
cfq | 42827 | 21882 | 20678 | 21508
deadline| 43040 | 71173 | 82407 | 86583

Cheers,
Jeff