Re: [PATCH 0/7] OMFS filesystem version 3

[Jamie Lokier <jamie@shareable.org>]

Miklos Szeredi wrote:
> Most filesystems (ntfs-3g included) cannot be modified from the
> outside, so coherency isn't an issue.
> 
> Otherwise currently fuse provides rather crude settings for caching:
> 
>   - timeout for attributes (per-inode, default 1s)
>   - timeout for names (per-dentry, default 1s)
>   - page cache is bypassed completely (per-file, default off)
>   - invalidation of page cache on open (per-open, default on)

Seems sensible.  As someone who is deep in coherency protocols at the
moment (I'm writing a robust distributed database/filesystem) I don't
like the crudeness, but for FUSE's real life use it seems a fine choice :-)

> There are also plans to add some sort of cache coherency protocol,
> where the filesystem can asynchronously call back to fuse to
> invalidate data or metadata.

Great!

I suggest adding another option (as well) where the filesystem can ask
fuse to send it synchronous validation requests - some things require
that.  (In my own work, the choice of A->B async invalidation and B->A
synchronous validation is heuristic: some usage patterns benefit from
one, some from the other.)

> > Otherwise I don't see how the kernel could coherently cache file pages
> > for some kinds of FUSE filesystems.  (E.g. sshfs, for example: every
> > operation must surely invoke a user space request or involve granting
> > a caching right to the kernel, to keep accesses coherent with other
> > users of the same remote files).
> > 
> > Ergo, either its not coherent, or there is some coherency protocol,
> > which does require _some_ work in the user space implementation.
> 
> Sshfs is not coherent (but neither is NFS), it just has timeouts for
> caches and invalidation based on modification time.

Fwiw, I think NFS version 4 is coherent (it uses leases), and older
NFS should be coherent when you use fcntl file locks (it's not very
efficient though).

I have been bitten a few times by timeout based caches in the past
(NFS and SMB (pre-oplock)).  Simple things like editing a file, then
running "ssh compiler-box make" from the editor quietly building
incorrect code - and even subsequent make commands don't fix it.  Or
when I edit a file, then tell someone I've changed the file - and then
they edit the file, and my edits are lost.  Very annoying.  Nobody
should build those kind of caches into new software.  :-)

-- Jamie