Union mounts and fchown/fchmod/utimensat

Union mounts and fchown/fchmod/utimensat

[Valerie Aurora]

In the current union mount design, files that will be modified are
copied up from the lower layer file system to the top layer at open()
time.  However, there are three cases in which metadata can be
modified through a file descriptor opened O_RDONLY: fchown(),
fchmod(), and utimensat().  Currently, we only copy up a file before
it is open, never after.  Implementing these system calls would
require replacing the read-only open file descriptor from the lower
file system with a new file descriptor from the topmost layer - in
other words, a huge nasty hack.

My question: How often do applications actually attempt fchown(),
fchmod(), or utimensat() on an O_RDONLY file descriptor?  Would it
break a lot of applications if a union mount file returned EBADF or
EPERM in this case?

-VAL

Re: Union mounts and fchown/fchmod/utimensat

[J. R. Okajima]

Valerie Aurora:
> My question: How often do applications actually attempt fchown(),
> fchmod(), or utimensat() on an O_RDONLY file descriptor?  Would it
> break a lot of applications if a union mount file returned EBADF or
> EPERM in this case?

Unfortunately I don't know how often it is.
But I am afraid it will be insufficient even if you implement the error.
Do you remember the sample case I've written before?
For instance,

	fd1 = open(path, O_RDONLY);
	fd2 = open(path, O_RDWR);
	fchmod(fd2);
	fstat(fd1, &st);

In this case, fchmod will not return the error because fd2 is opened for
write and the file would be copied-up. And the next fstat for fd1 will
return a bogus info since fd1 still refers the file on the lower layer.
Regardless fchmod() is issued by the same process or different one,
fstat() lies.
This is the same scenario I wrote before essentially (Jul 2009).

Also we should take care of the file-locking.
The copyup by UnionMount will break fcntl(F_SETLK) easily in the same
scenario. But I am not sure how many applications depend upon file lock
and UnionMount copyup affect.

Re: Union mounts and fchown/fchmod/utimensat

[Valerie Aurora]

On Sun, Mar 28, 2010 at 12:43:23AM +0900, J. R. Okajima wrote:
> 
> Valerie Aurora:
> > My question: How often do applications actually attempt fchown(),
> > fchmod(), or utimensat() on an O_RDONLY file descriptor?  Would it
> > break a lot of applications if a union mount file returned EBADF or
> > EPERM in this case?
> 
> Unfortunately I don't know how often it is.
> But I am afraid it will be insufficient even if you implement the error.
> Do you remember the sample case I've written before?
> For instance,
> 
> 	fd1 = open(path, O_RDONLY);
> 	fd2 = open(path, O_RDWR);
> 	fchmod(fd2);
> 	fstat(fd1, &st);
> 
> In this case, fchmod will not return the error because fd2 is opened for
> write and the file would be copied-up. And the next fstat for fd1 will
> return a bogus info since fd1 still refers the file on the lower layer.
> Regardless fchmod() is issued by the same process or different one,
> fstat() lies.
> This is the same scenario I wrote before essentially (Jul 2009).

I'm working on the assumption that this is okay.  Basically, an
in-kernel copyup has the same effect on an application with an
O_RDONLY fd as a userland copy-up to a temporary file followed by a
rename() to the original name.  Editors, among other applications, use
this sequence all the time.  If it breaks under union mount copyup, it
will also break if you edit the file with vi.

> Also we should take care of the file-locking.
> The copyup by UnionMount will break fcntl(F_SETLK) easily in the same
> scenario. But I am not sure how many applications depend upon file lock
> and UnionMount copyup affect.

Yeah, it isn't obvious what the expected effect would be.  I'll think
about it more.

Thanks,

-VAL

Re: Union mounts and fchown/fchmod/utimensat

[Jamie Lokier]

Valerie Aurora wrote:
> I'm working on the assumption that this is okay.  Basically, an
> in-kernel copyup has the same effect on an application with an
> O_RDONLY fd as a userland copy-up to a temporary file followed by a
> rename() to the original name.  Editors, among other applications, use
> this sequence all the time.  If it breaks under union mount copyup, it
> will also break if you edit the file with vi.

Ouch.  That's fine for rename-over style updates, but if an
application depends on in-place updates, or if it depends on file
locking to serialise something (using O_RDONLY for read locks and
O_RDWR for write locks), it will get confused.

Now, that may be acceptable compromise non-POSIX behaviour.

But if it only works with rename-over style updates.... Why have
in-kernel copyup at all?  Why not simply forbid writes to lower files?
Rename-over updates will be fine with that, and there'll be no
unexpected long delays to do copyup of a huge file.

-- Jamie

Re: Union mounts and fchown/fchmod/utimensat

[Theodore Tso]

Hmmm....   do we also need to worry about read-only fd's of directories, and interaction with the *at() system calls.   For example:

	int renameat(int olddirfd, const char *oldpath, int newdirfd, const char *newpath)

... where olddirfd and newdirfd might be r/o file descriptors for directories participating in a Union mount?

This is just one example, of course.   There's also openat(), symlinkat(), etc.

-- Ted

Re: Union mounts and fchown/fchmod/utimensat

[Valerie Aurora]

On Tue, Mar 30, 2010 at 07:16:18AM -0400, Theodore Tso wrote:
> Hmmm....   do we also need to worry about read-only fd's of directories, and interaction with the *at() system calls.   For example:
> 
> 	int renameat(int olddirfd, const char *oldpath, int newdirfd, const char *newpath)
> 
> ... where olddirfd and newdirfd might be r/o file descriptors for directories participating in a Union mount?
> 
> This is just one example, of course.   There's also openat(), symlinkat(), etc.

Directories are always copied up in this version of union mounts, so
it doesn't matter whether you open a directory O_RDONLY or not, you'll
always have the topmost version and see all the effects of link(),
linkat(), rename(), renameat(), etc.  It's only with files that you
can get a stale copy if you open it O_RDONLY and it gets modified
through some other path.

Always copying up directories solves a lot of problems.  I managed to
eliminate several hundred lines of code with that change in the
design.

-VAL

Re: Union mounts and fchown/fchmod/utimensat

[Valerie Aurora]

On Tue, Mar 30, 2010 at 12:48:43AM +0100, Jamie Lokier wrote:
> Valerie Aurora wrote:
> > I'm working on the assumption that this is okay.  Basically, an
> > in-kernel copyup has the same effect on an application with an
> > O_RDONLY fd as a userland copy-up to a temporary file followed by a
> > rename() to the original name.  Editors, among other applications, use
> > this sequence all the time.  If it breaks under union mount copyup, it
> > will also break if you edit the file with vi.
> 
> Ouch.  That's fine for rename-over style updates, but if an
> application depends on in-place updates, or if it depends on file
> locking to serialise something (using O_RDONLY for read locks and
> O_RDWR for write locks), it will get confused.
> 
> Now, that may be acceptable compromise non-POSIX behaviour.
> 
> But if it only works with rename-over style updates.... Why have
> in-kernel copyup at all?  Why not simply forbid writes to lower files?

Hm, I'm not sure we're on the same page.  In-kernel copyup will work
transparently in most cases.  An application that opens multiple fds
with O_RDWR or O_WRONLY will see completely coherent changes.  It's
only when an app opens a file O_RDONLY, and then depends on seeing
writes or metadata changes from some other agency (another app, or an
fd opened with write permission) that union mounts will break things.
The question is how common that is.

Does that make sense or I did I miss your point?

-VAL