Re: readpages & writepages

[Andrew Morton <akpm@digeo.com>]

Steven French <sfrench@us.ibm.com> wrote:
>
> Some questions about implementing readpages & writepages for network
> filesystems ...
> 
> I didn't find many examples of readpages & writepages in 2.5 yet (so far
> just the local filesystem examples that mostly use fs/mpage.c common code
> and the nfs case) as I looked through trying to implement these optional
> vfs entry points for the cifs case.

My approach there was to implement sufficient functionality for the kernel to
be able to assemble multipage BIOs for ext2 filesystems, and no more.  To
keep it simple, clean and well documented, in the expectation that when other
filesystem developers get into it, the functionality will grow on-demand.

And indeed this has happened - enhancements have been added as AFS, reiserfs3
and NFS have started to use this code.

I expect more changes will be needed to accommodate other filesystems. 
That's fine.

>   Doing larger reads, each of size 16K,
> turns out to be reasonably efficient to typical CIFS servers (larger is of
> course also possible), and would have the benefit of reducing the network
> traffic noticeably when doing readahead.    There are a few difficulties
> though - there does not appear to be an obvious mapping of the list of
> pages (passed in by to readpages/writepages), at least to something that I
> could memcopy my 16K buffer network read buffer into - so the pages have to
> be individually mapped and copied in 4K chunks, losing a small amount of
> the benefit of having a readpages/writepages in the first place

Well if you want a single 16k physically contiguous chunk of memory then yup,
there are tons of problems around that; it won't be happening in 2.6.

The overhead of mapping and unmapping 4 pages is very small, especially when
compared with the cost of the copy itself!

You'd be better off looking into avoiding that copy altogether: feed four 4k
pages down to the network stack and get them filled in direct from the
busmastering receive.

(And bear in mind that 1 single 16k chunk could perform worse than 4x4k
chunks, if the network receive and the copy are serialised.  With 4k pages,
the CPU can be copying one page _while_ the network is pulling in the next
one...)

> and the
> common routines (read_cache_pages and generic_writepages/mpage_writepages
> respectively) don't seem written for the case in which > 4K is copied in
> one call to the filesystem and for nfs the mpage_writepages call seems to
> default to calling the vfs op writepage (since a null get_block_t routine
> is passed in) which could have been done by simply not supporting the
> writepages entry point - perhaps this is just an intermediate coding step,
> a staging of the eventual function.
> 
> Has there been much discussion or much written down (other than a little
> bit in documentation/filesystems/locking and the readahead function
> comments in mm/readahead.c) on the suitability of readpages & writepages
> for network filesystems?  There obviously can be significant benefits in
> reducing the number of network roundtrips if it can be made to work
> efficiently ...

I don't understand how this affects the number of roundtrips?  You seem to be
implying that CIFS has an fs-private receive buffer, the contents of which
are copied into the VFS's pagecache?

If so, then making that buffer be 16k should work OK?

If not, then some more details on the general data flow would be needed,
please.