readpages and network file systems

readpages and network file systems

[James Philip Roper]

Hi

Am I right in saying that the kernel has a maximum of 128K readpages?  I 
saw some discussion in the archives relating to the efficiency of reads 
that size in the context of block devices, and a lot of people were 
saying that reads shouldn't be larger than that because of various 
problems, but what about network file systems?  I'm implementing read 
ahead routines in cifs, and i think reads in the megabytes could greatly 
increase bandwidth utilisation, as you can have up to 50 cifs requests 
of 64K each on the wire at any one time.  The larger the reads, the less 
the round trip time has effect.

Please CC any responses to me as I am not on the mailing list.

James

Re: readpages and network file systems

[Sonny Rao]

On Tue, Oct 12, 2004 at 12:10:05PM +1000, James Philip Roper wrote:
> Hi
> 
> Am I right in saying that the kernel has a maximum of 128K readpages?  I 
> saw some discussion in the archives relating to the efficiency of reads 
> that size in the context of block devices, and a lot of people were 
> saying that reads shouldn't be larger than that because of various 
> problems, but what about network file systems?  I'm implementing read 
> ahead routines in cifs, and i think reads in the megabytes could greatly 
> increase bandwidth utilisation, as you can have up to 50 cifs requests 
> of 64K each on the wire at any one time.  The larger the reads, the less 
> the round trip time has effect.
> 
> Please CC any responses to me as I am not on the mailing list.

Well, 128k (256 512-byte sectors) is the default readahead in the
kernel.  It certainly isn't a hard limit on what can be done in terms
of block IO, which is more like 1MB on unmodified kernels.  Here, the
limitation comes from the max BIO-vector size.  

I haven't done a lot of work on network filesystems, so I'm not sure
one can alter the readahead values for a network filesystem, anyone?
I'm guessing that you just get the default backing store in that case,
which is 256 sector readahead.   

I have done a lot of testing with different readahead values on SCSI,
IDE, and fiber setups, and I've found that 128k can be insufficient to
reach maximum sequential hardware speed in many cases.  The filesystem
can make a significant difference as well depending on how they decide
to lay out blocks.  In a network filesystem case, that may not be a
real issue.  I can post my data somewhere if anyone is curious.

Sonny Rao
--
IBM, LTC Kernel Performance