Re: [PATCH 1/2] [RFC] vfs: 'stat light' fstatat flags

[Jamie Lokier <jamie@shareable.org>]

Trond Myklebust wrote:
> On Tue, 2009-04-07 at 14:24 -0400, Oleg Drokin wrote:
> > Well, what the stat actually meant to do is "give me the file
> > information as it is now". By the time it returns, the data is
> > stale anyway, and the longer your path from the user app to the
> > actual file storage, the more potentially out of date the
> > information is.  NFS just takes it to an extreme case and
> > introduces an assumed validity timeout.  While I do not directly
> > oppose such a flag, I really do not see huge value in it. I wonder
> > what is the specific usecase do you have in mind?
> 
> The default behaviour of stat() on NFS is to do a revalidation of the
> cached data (by which I mean that we issue an RPC call if and only if
> the cache has timed out, or if it is known to be invalid).
> 
> The AT_STRICT would be used by the application to tell NFS that it must
> retrieve the cached data from the server. One instance where this is
> useful would be the case where you're doing a distributed compile: the
> application knows that the file may have changed on the server, and
> wants to force the kernel to check mtime.

Yes, that's exactly the sort of thing I had in mind.

The amount of time taken isn't relevant, neither is the fact that the
file may change after calling stat() before using the result.

What matters is the order of dependent events.

    program on host A does something which may affect files,
       then sends message to host B - "I've finished whatever may affect files"
    in response to message,
       program on host B calls stat() to check for changes to files

The stat() call must get attributes which are no older than when host
A modified those attributes.

This implies host A must have written the attribute changes too :-)
NFS does this on close(); other network filesystems use other mechanisms.

As Trond says, the default behaviour of stat() does not get
sufficiently recent attributes, which means things like distributed
compiles sometimes fail on NFS with default settings.

I was thinking of another application: #!/usr/bin/perlcachedcompiler
which behaves like a normal script interpreter, except it compiles the
script first (slow optimising compile...) and runs the compiled
version.  Next time it's run, it checks the persistent cache of
compiled versions against the attributes of the script file, and if
there's a match, no need to compile: just run the compiled version
quickly.

This sort of cache validation needs AT_STRICT to be reliable with NFS,
when a script is edited on one host, then run on another.

(Actually I'd like dnotify/inotify/fsnotify so I can omit the stat()
round trip entirely please on each application cache
revalidation... but only if "check for any file change events, answer
= no events" is synchronous in the same way that AT_STRICT is...)

-- Jamie