[PATCH 0/8] FUSE improvements + VFS changes

[PATCH 0/8] FUSE improvements + VFS changes

[Miklos Szeredi]

There are a small number of features that are missing from the VFS
interface (and hence from the FUSE interface), that are needed to
allow a special class of filesystems to provide a full UNIX (Posix,
SUS, whatever) file semantics.

The filesystems in question are served by a process that runs

  - in userspace 
  - without special privileges

A trivial example is sshfs (sftp-server runs on remote server with
normal user privileges).

The limitations are:

  1) open("foo", O_CREAT | O_WRONLY, 0444) or similar won't work

  2) ftruncate on a file not having write permission (but file opened
     in write mode) will fail

  3) statfs() cannot return different values based on the path within
     a filesystem

1) and 2) arise from the fact, that the VFS separates the permission
checking from the actual operations while the above described
filesystem can't do that.

1) can be solved by allowing a create+open atomic operation.  The
infrastructure for this is already in -mm from the NFS tree.

2) can be solved by distinguishing ftruncate() from trunate() invoked
->setattr().  For ftruncate() the filesystem also needs to find a
handle for a file opened for writing.  The file on which ftuncate()
was performed is always a good candidate, so an obvious solution is to
pass the open file to ->setattr().

3) comes from the fact that ->statfs() is a sb operation not an inode
operation.  The only place where this is used is in sys_ustat() which
is deprecated and also useless for virtual (non disk based)
filesystems.  So introduce a i_op->statfs() operation, which if
implemented will be used instead of s_op->statfs().

Changes to VFS:

  1 - pass file pointer to ->setattr() if invoked from ftruncate()
  2 - per inode statfs (core)
  3 - per inode statfs (architectures)

Changes to FUSE:

  4 - bump interface minor version
  5 - add access call to interface
  6 - atomic open+create
  7 - pass file handle to userspace in setattr call
  8 - per inode statfs (fuse)

Miklos

Re: [PATCH 0/8] FUSE improvements + VFS changes

[Al Viro]

On Mon, Oct 24, 2005 at 06:45:05PM +0200, Miklos Szeredi wrote:
> The limitations are:
> 
>   1) open("foo", O_CREAT | O_WRONLY, 0444) or similar won't work
> 
>   2) ftruncate on a file not having write permission (but file opened
>      in write mode) will fail
> 
>   3) statfs() cannot return different values based on the path within
>      a filesystem

    4) mass of a body cannot vary depending on the way it's turned.

Horrible limitations, all of them...

Re: [PATCH 0/8] FUSE improvements + VFS changes

[Miklos Szeredi]

> > The limitations are:
> > 
> >   1) open("foo", O_CREAT | O_WRONLY, 0444) or similar won't work
> > 
> >   2) ftruncate on a file not having write permission (but file opened
> >      in write mode) will fail
> > 
> >   3) statfs() cannot return different values based on the path within
> >      a filesystem
> 
>     4) mass of a body cannot vary depending on the way it's turned.
> 
> Horrible limitations, all of them...

Troll.

We have solution for 1-3, what's your's for 4?

Miklos

Re: [PATCH 0/8] FUSE improvements + VFS changes

[Miklos Szeredi]

> > > The limitations are:
> > > 
> > >   1) open("foo", O_CREAT | O_WRONLY, 0444) or similar won't work
> > > 
> > >   2) ftruncate on a file not having write permission (but file opened
> > >      in write mode) will fail
> > > 
> > >   3) statfs() cannot return different values based on the path within
> > >      a filesystem
> > 
> >     4) mass of a body cannot vary depending on the way it's turned.
> > 
> > Horrible limitations, all of them...
> 
> Troll.
> 
> We have solution for 1-3, what's your's for 4?

And in case you're wondering, 1-2 are mandated by SUS, and obscure
programs like "cp" depend on 1) for example.  So it's not as if these
limitations were all that fair.

Miklos

Re: [PATCH 0/8] FUSE improvements + VFS changes

[Al Viro]

On Tue, Oct 25, 2005 at 09:04:21AM +0200, Miklos Szeredi wrote:
> > > > The limitations are:
> > > > 
> > > >   1) open("foo", O_CREAT | O_WRONLY, 0444) or similar won't work
> > > > 
> > > >   2) ftruncate on a file not having write permission (but file opened
> > > >      in write mode) will fail
> > > > 
> > > >   3) statfs() cannot return different values based on the path within
> > > >      a filesystem
> > > 
> > >     4) mass of a body cannot vary depending on the way it's turned.
> > > 
> > > Horrible limitations, all of them...
> > 
> > Troll.
> > 
> > We have solution for 1-3, what's your's for 4?
> 
> And in case you're wondering, 1-2 are mandated by SUS, and obscure
> programs like "cp" depend on 1) for example.  So it's not as if these
> limitations were all that fair.

(1) and (2) are real, no arguments here.  It's statfs one that is
bogus (and yes, I should've trimmed the quoted part better).

Re: [PATCH 0/8] FUSE improvements + VFS changes

[Miklos Szeredi]

> > > > > The limitations are:
> > > > > 
> > > > >   1) open("foo", O_CREAT | O_WRONLY, 0444) or similar won't work
> > > > > 
> > > > >   2) ftruncate on a file not having write permission (but file opened
> > > > >      in write mode) will fail
> > > > > 
> > > > >   3) statfs() cannot return different values based on the path within
> > > > >      a filesystem
> > > > 
> > > >     4) mass of a body cannot vary depending on the way it's turned.
> > > > 
> > > > Horrible limitations, all of them...
> > > 
> > > Troll.
> > > 
> > > We have solution for 1-3, what's your's for 4?
> > 
> > And in case you're wondering, 1-2 are mandated by SUS, and obscure
> > programs like "cp" depend on 1) for example.  So it's not as if these
> > limitations were all that fair.
> 
> (1) and (2) are real, no arguments here.  It's statfs one that is
> bogus (and yes, I should've trimmed the quoted part better).

OK, let's say it's bogus.

I still believe, the one local filesystem - many remote filesystems is
a valid model, but it's pretty low priority.  But it'll come back yet!

Miklos

Re: [PATCH 0/8] FUSE improvements + VFS changes

[Bryan Henderson]

>I still believe, the one local filesystem - many remote filesystems is
>a valid model, but it's pretty low priority.  But it'll come back yet!

I don't think distance (local/remote) is relevant -- I'd rather see one 
filesystem, many subfilesystems.  There are other ways besides remote 
filesystem access where that would apply, and I don't think at the statfs 
level Linux should care.

But even closer to clean would be one Linux filesystem image, multiple 
filesystems.  A filesystem image is the thing you create with "mount" and 
is represented by a struct super_block; a filesystem is a set of data 
structured a certain way (traditionally, the contents of a disk).

Besides the scope of statfs, I know another thing that defines the 
separation between two filesystems:  you can't link or relink across 
filesystems.  In sshfs, you probably get a failure if you try to move a 
file from a directory in one remote filesystem to one in another remote 
filesystem, even though according to the Linux view, they're both in the 
same filesystem.  (And even if it doesn't fail, it certainly doesn't do 
what you expect from rename()).

--
Bryan Henderson                     IBM Almaden Research Center
San Jose CA                         Filesystems

Re: [PATCH 0/8] FUSE improvements + VFS changes

[Miklos Szeredi]

> >I still believe, the one local filesystem - many remote filesystems is
> >a valid model, but it's pretty low priority.  But it'll come back yet!
> 
> I don't think distance (local/remote) is relevant -- I'd rather see one 
> filesystem, many subfilesystems.  There are other ways besides remote 
> filesystem access where that would apply, and I don't think at the statfs 
> level Linux should care.
> 
> But even closer to clean would be one Linux filesystem image, multiple 
> filesystems.  A filesystem image is the thing you create with "mount" and 
> is represented by a struct super_block; a filesystem is a set of data 
> structured a certain way (traditionally, the contents of a disk).

So subfilesystems are filesystems which do not have any corresponding
structure in kernel.

What have we got against subfilesystems?

Complexity?  

The patches come to a balance of +26 lines, but 20 instances of
->d_inode->i_sb moved to a common function.

I'm not forcing this change, because currently it doesn't have any
applications.  Sftp does not have a statfs() operation, though there
have been multiple requests for this.

When sftp and sshfs are extended with a statfs() I will push this
feature further.

> Besides the scope of statfs, I know another thing that defines the 
> separation between two filesystems:  you can't link or relink across 
> filesystems.  In sshfs, you probably get a failure if you try to move a 
> file from a directory in one remote filesystem to one in another remote 
> filesystem, even though according to the Linux view, they're both in the 
> same filesystem.  (And even if it doesn't fail, it certainly doesn't do 
> what you expect from rename()).

Yes, subfilesystems already exist in this sense.  User code (other
than df) simply doesn't care about filesystems.  It cares about
directories and files and the rest is the problem for the kernel.

Allowing statfs() to return different results for files within a
single mount (filesystem image), should not cause any problems.

Miklos

Re: [PATCH 0/8] FUSE improvements + VFS changes

[Jamie Lokier]

Bryan Henderson wrote:
> Besides the scope of statfs, I know another thing that defines the 
> separation between two filesystems:  you can't link or relink across 
> filesystems.  In sshfs, you probably get a failure if you try to move a 
> file from a directory in one remote filesystem to one in another remote 
> filesystem, even though according to the Linux view, they're both in the 
> same filesystem.  (And even if it doesn't fail, it certainly doesn't do 
> what you expect from rename()).

Other things that various software depends on:

   st_dev changes across a mount point (inode number space), so that
   (st_dev, st_ino) uniquely identifies each file (as long as they are
   open, on some filesystems that don't have real inode numbers).  In
   particular, identical (st_dev, st_ino) for two paths imply the
   paths are hard links.

   After finding st_nlink different paths referring to the same
   (st_dev, st_ino) pair, one may assume that's all the links to a
   file.  (That's broken by bind mounts, unfortunately).

-- Jamie

Re: [PATCH 0/8] FUSE improvements + VFS changes

[Al Viro]

On Tue, Oct 25, 2005 at 07:56:46AM +0200, Miklos Szeredi wrote:
> > > The limitations are:
> > > 
> > >   1) open("foo", O_CREAT | O_WRONLY, 0444) or similar won't work
> > > 
> > >   2) ftruncate on a file not having write permission (but file opened
> > >      in write mode) will fail
> > > 
> > >   3) statfs() cannot return different values based on the path within
> > >      a filesystem
> > 
> >     4) mass of a body cannot vary depending on the way it's turned.
> > 
> > Horrible limitations, all of them...
> 
> Troll.

Sarkasm
 
> We have solution for 1-3, what's your's for 4?

Dunno - you are the one interested in solving non-problems...