Caching files in nfsd was Re: [patch 12/16] fix race between writeback and unlink

Caching files in nfsd was Re: [patch 12/16] fix race between writeback and unlink

[Andi Kleen]

Linus Torvalds <torvalds@transmeta.com> writes:

> I _think_ that right now nfsd doesn't cache file opens (only inodes), so
> this could be a performance issue for nfsd, but it might be possible to
> change how nfsd acts. And it would be a _lot_ cleaner to do it at the file
> level.

Yes.

Fixing this would also help XFS (which I hope will be merged in 2.5 as
it works very well for a lot of people). It manages its extent
preallocation per file and flushes extents on closes. Currently it has
to maintain an ugly private nfs reference cache to avoid flushing an
extent after every NFS write operation (and killing write performance
this way)

Also letting nfsd know about the filemap.c readahead window information in 
struct file (that is what it currently caches in the racache) is really ugly
and a kind of layering violation...

I guess it is not too uncommon for other file systems too to hold state 
per open file.

-Andi

Re: Caching files in nfsd was Re: [patch 12/16] fix race between writeback and unlink

[Neil Brown]

On  June 4, ak@suse.de wrote:
> Linus Torvalds <torvalds@transmeta.com> writes:
> 
> > I _think_ that right now nfsd doesn't cache file opens (only inodes), so
> > this could be a performance issue for nfsd, but it might be possible to
> > change how nfsd acts. And it would be a _lot_ cleaner to do it at the file
> > level.
> 
> Yes.
> 
> Fixing this would also help XFS (which I hope will be merged in 2.5 as
> it works very well for a lot of people). It manages its extent
> preallocation per file and flushes extents on closes. Currently it has
> to maintain an ugly private nfs reference cache to avoid flushing an
> extent after every NFS write operation (and killing write performance
> this way)
> 
> Also letting nfsd know about the filemap.c readahead window information in 
> struct file (that is what it currently caches in the racache) is really ugly
> and a kind of layering violation...

I agree.  I would like to replace the racache with a "struct file"
cache (though it isn't high on my priorities).

The only issue that I can see (except for simple coding) is that as
NFS cannot be precise about closing at the *right* time we would be
changing from closing too early (and so re-opening) to closing too
late.
Would this be an issue for any filesystem?  My feeling is not, but I'm
open to opinions....

This is an issue for the user-space NFS daemon.  It caches open
filedescriptors and an open O_RDWR will imply an active
get_write_access which, for example, stops the file being executed on
the server.
kNFSd won't suffer from this as it can drop write_access without
closing the file.  Are there any other issues?

NeilBrown

Re: Caching files in nfsd was Re: [patch 12/16] fix race between writeback and unlink

[Andi Kleen]

> The only issue that I can see (except for simple coding) is that as
> NFS cannot be precise about closing at the *right* time we would be
> changing from closing too early (and so re-opening) to closing too
> late.
> Would this be an issue for any filesystem?  My feeling is not, but I'm
> open to opinions....

The only potential issue I see is that forcing a flush when the file system
fills up may be a good idea to drop preallocations (but then one would hope 
that a fs with preallocation does this already automatically, so it hopefully 
won't be needed in nfsd) 

-Andi

Re: Caching files in nfsd was Re: [patch 12/16] fix race between writeback and unlink

[Chris Mason]

On Tue, 2002-06-04 at 08:16, Andi Kleen wrote:
> > The only issue that I can see (except for simple coding) is that as
> > NFS cannot be precise about closing at the *right* time we would be
> > changing from closing too early (and so re-opening) to closing too
> > late.
> > Would this be an issue for any filesystem?  My feeling is not, but I'm
> > open to opinions....
> 
> The only potential issue I see is that forcing a flush when the file system
> fills up may be a good idea to drop preallocations (but then one would hope 
> that a fs with preallocation does this already automatically, so it hopefully 
> won't be needed in nfsd) 

reiserfs does, but I'm not sure about ext2.  reiserfs doesn't carry
preallocated blocks from one transaction to another.  So when the
transaction closes, we free preallocated blocks.

When the FS runs out of space, we stop the transaction to give everyone
a chance to free up what they can (not just preallocation), and then
start a new transaction.  So nfs closing too late won't hurt.

-chris