Re: Advice sought on how to lock multiple pages in ->prepare_write and ->writepage

[Sonny Rao <sonny@burdell.org>]

On Mon, Jan 31, 2005 at 03:46:15PM -0800, Bryan Henderson wrote:

> To get multi-page bios (in any natural way), you need to throw out not 
> only the generic file read/write routines, but the page cache as well.
> 
> Every time I've looked at multi-page bios, I've been unable to see any 
> reason that they would be faster than multiple single-page bios.  But I 
> haven't seen any experiments.

Well, I've certainly seen issues where certain filesystems will throw
down lots of smaller bios and rely on the io-schedulers to merge them,
but the io-schedulers don't do a perfect job of making the largest
requests possible.  This is especially true in the case of a large
fast raid array with fast writeback caching.

Here's an example output of iostat from a sequential write
(overwriting, not appending) to a 7 disk raid-0 array.

Apologies for the long lines.

Ext3 (writeback mode)

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sdc          0.00 21095.60 21.00 244.40  168.00 170723.20    84.00 85361.60 643.90    11.15   42.15   3.45  91.60

We see 21k merges per second going on, and an average request size of 
only 643 sectors where the device can handle up to 1Mb (2048 sectors).

Here is iostat from the same test w/ JFS instead:

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sdc          0.00 1110.58  0.00 97.80    0.00 201821.96     0.00 100910.98 2063.53   117.09 1054.11  10.21  99.84

So, in this case I think it is making a difference 1k merges and a big difference in
throughput, though there could be other issues. 

Sonny