Re: [PATCH 00/74] Union mounts version something or other

Re: [PATCH 00/74] Union mounts version something or other

[Valerie Aurora]

And I cc'd linux@vger.kernel.org on all the patches instead of
linux-kernel@vger.kernel.org - guess I'm really out of the kernel
business now!  Anyway, if you want your replies to go to lkml, you'll
have to hand edit the cc list.

-VAL

On Tue, Mar 22, 2011 at 6:58 PM, Valerie Aurora
<valerie.aurora@gmail.com> wrote:
> Hi union mounts fans(?),
>
> Here's my current union mounts patch set, against 2.6.36-rc5.  I'm
> busy with other things[1] and unlikely to put in significant work on
> union mounts in the next year.  I'm happy to answer questions from
> anyone else working on them.
>
> As always, git trees for the kernel, util-linux, and e2fsprogs, lots
> of documentation, and LWN articles describing the various problems
> unioning file systems will encounter are here:
>
> http://valerieaurora.org/union/
>
> The devkit linked to from that page includes my Usermode Linux testing
> environment, including root file system image.  The README tells you
> how to run the test suite automatically (yes, an automated test suite
> - with Makefile and version control and comments and stuff!).
>
> I took a quick look at the current overlayfs patch set, and it's
> small, clean, and easy to understand.  If it does what people need, I
> say ship it.
>
> Thanks to everyone who reviewed and submitted patches for union mounts!
>
> -VAL
>
> [1] http://adainitiative.org
>
> ---
>
> Felix Fietkau (2):
>  whiteout: jffs2 whiteout support
>  fallthru: jffs2 fallthru support
>
> Jan Blunck (9):
>  VFS: Make lookup_hash() return a struct path
>  autofs4: Save autofs trigger's vfsmount in super block info
>  whiteout/NFSD: Don't return information about whiteouts to userspace
>  whiteout: Add vfs_whiteout() and whiteout inode operation
>  whiteout: Allow removal of a directory with whiteouts
>  whiteout: tmpfs whiteout support
>  union-mount: Introduce MNT_UNION and MS_UNION flags
>  union-mount: Free union stack on removal of topmost dentry from
>    dcache
>  union-mount: Create IS_MNT_UNION()
>
> Valerie Aurora (63):
>  VFS: Comment follow_mount() and friends
>  Documentation: Fix trivial typo in filesystems/sharedsubtree.txt
>  whiteout: Define opaque inode flags and operations
>  ext2: Add ext2_dirent_in_use()
>  ext2: Split ext2_add_entry() from ext2_add_link()
>  whiteout: ext2 whiteout support
>  fallthru: Basic fallthru definitions
>  fallthru: ext2 fallthru support
>  fallthru: tmpfs fallthru support
>  VFS: Add hard read-only users count to superblock
>  VFS: Make clone_mnt()/copy_tree()/collect_mounts() return errors
>  VFS: Add CL_NO_SHARED flag to clone_mnt()/copy_tree()
>  VFS: Add CL_NO_SLAVE flag to clone_mnt()/copy_tree()
>  VFS: Add CL_MAKE_HARD_READONLY flag to clone_mnt()/copy_tree()
>  union-mount: Union mounts documentation
>  union-mount: Add CONFIG_UNION_MOUNT option
>  union-mount: Create union_stack structure
>  union-mount: Add two superblock fields for union mounts
>  union-mount: Add union_alloc()
>  union-mount: Add union_find_dir()
>  union-mount: Create d_free_unions()
>  union-mount: Create union_add_dir()
>  union-mount: Add union_create_topmost_dir()
>  union-mount: Create needs_lookup_union()
>  union-mount: Create check_topmost_union_mnt()
>  union-mount: Add clone_union_tree() and put_union_sb()
>  union-mount: Create build_root_union()
>  union-mount: Create prepare_mnt_union() and cleanup_mnt_union()
>  union-mount: Prevent improper union-related remounts
>  union-mount: Prevent topmost file system from being mounted elsewhere
>  union-mount: Prevent bind mounts of union mounts
>  union-mount: Implement union mount
>  union-mount: Temporarily disable some syscalls
>  union-mount: Basic infrastructure of __lookup_union()
>  union-mount: Process negative dentries in __lookup_union()
>  union-mount: Return files found in lower layers in __lookup_union()
>  union-mount: Build union stack in __lookup_union()
>  union-mount: Follow mount in __lookup_union()
>  union-mount: Add lookup_union()
>  union-mount: Add do_lookup_union() wrapper for __lookup_union()
>  union-mount: Call union lookup functions in lookup path
>  union-mount: Create whiteout on unlink()
>  union-mount: Create whiteout on rmdir()
>  union-mount: Set opaque flag on new directories in unioned file
>    systems
>  union-mount: Copy up directory entries on first readdir()
>  union-mount: Add generic_readdir_fallthru() helper
>  fallthru: ext2 support for lookup of d_type/d_ino in fallthrus
>  fallthru: tmpfs support for lookup of d_type/d_ino in fallthrus
>  fallthru: jffs2 support for lookup of d_type/d_ino in fallthrus
>  VFS: Split inode_permission() and create path_permission()
>  VFS: Create user_path_nd() to lookup both parent and target
>  union-mount: In-kernel file copyup routines
>  union-mount: Implement union-aware access()/faccessat()
>  union-mount: Implement union-aware link()
>  union-mount: Implement union-aware rename()
>  union-mount: Implement union-aware writable open()
>  union-mount: Implement union-aware chown()
>  union-mount: Implement union-aware truncate()
>  union-mount: Implement union-aware chmod()/fchmodat()
>  union-mount: Implement union-aware lchown()
>  union-mount: Implement union-aware utimensat()
>  union-mount: Implement union-aware setxattr()
>  union-mount: Implement union-aware lsetxattr()
>
>  Documentation/filesystems/sharedsubtree.txt |    4 +-
>  Documentation/filesystems/union-mounts.txt  |  751 +++++++++++++++++++++++++
>  Documentation/filesystems/vfs.txt           |   16 +-
>  fs/Kconfig                                  |   13 +
>  fs/Makefile                                 |    1 +
>  fs/autofs4/autofs_i.h                       |    1 +
>  fs/autofs4/init.c                           |   11 +-
>  fs/autofs4/root.c                           |    6 +
>  fs/compat.c                                 |    9 +
>  fs/dcache.c                                 |   32 +-
>  fs/ext2/dir.c                               |  116 ++++-
>  fs/ext2/ext2.h                              |    3 +
>  fs/ext2/inode.c                             |   11 +-
>  fs/ext2/namei.c                             |   85 +++-
>  fs/ext2/super.c                             |    6 +
>  fs/jffs2/dir.c                              |  117 ++++-
>  fs/jffs2/fs.c                               |    4 +
>  fs/jffs2/super.c                            |    2 +-
>  fs/libfs.c                                  |   20 +-
>  fs/namei.c                                  |  807 ++++++++++++++++++++++++---
>  fs/namespace.c                              |  394 +++++++++++--
>  fs/nfsd/nfs3xdr.c                           |    5 +
>  fs/nfsd/nfs4xdr.c                           |    5 +
>  fs/nfsd/nfsxdr.c                            |    4 +
>  fs/open.c                                   |  116 ++++-
>  fs/pnode.c                                  |    5 +-
>  fs/pnode.h                                  |    3 +
>  fs/readdir.c                                |   18 +
>  fs/super.c                                  |    9 +
>  fs/union.c                                  |  714 ++++++++++++++++++++++++
>  fs/union.h                                  |  105 ++++
>  fs/utimes.c                                 |   14 +-
>  fs/xattr.c                                  |   65 ++-
>  include/linux/dcache.h                      |   37 ++-
>  include/linux/ext2_fs.h                     |    8 +
>  include/linux/fs.h                          |   45 ++
>  include/linux/jffs2.h                       |    8 +
>  include/linux/mount.h                       |    4 +
>  include/linux/namei.h                       |    2 +
>  kernel/audit_tree.c                         |   10 +-
>  mm/shmem.c                                  |  193 ++++++-
>  41 files changed, 3551 insertions(+), 228 deletions(-)
>  create mode 100644 Documentation/filesystems/union-mounts.txt
>  create mode 100644 fs/union.c
>  create mode 100644 fs/union.h
>
>

Re: [PATCH 00/74] Union mounts version something or other

[Sedat Dilek]

On Wed, Mar 23, 2011 at 2:58 AM, Valerie Aurora
<valerie.aurora@gmail.com> wrote:
> Hi union mounts fans(?),
>
> Here's my current union mounts patch set, against 2.6.36-rc5.  I'm
> busy with other things[1] and unlikely to put in significant work on
> union mounts in the next year.  I'm happy to answer questions from
> anyone else working on them.
>
> As always, git trees for the kernel, util-linux, and e2fsprogs, lots
> of documentation, and LWN articles describing the various problems
> unioning file systems will encounter are here:
>
> http://valerieaurora.org/union/
>
> The devkit linked to from that page includes my Usermode Linux testing
> environment, including root file system image.  The README tells you
> how to run the test suite automatically (yes, an automated test suite
> - with Makefile and version control and comments and stuff!).
>
> I took a quick look at the current overlayfs patch set, and it's
> small, clean, and easy to understand.  If it does what people need, I
> say ship it.
>
> Thanks to everyone who reviewed and submitted patches for union mounts!
>
> -VAL
>
> [1] http://adainitiative.org
>
> ---
>
> Felix Fietkau (2):
>  whiteout: jffs2 whiteout support
>  fallthru: jffs2 fallthru support
>
> Jan Blunck (9):
>  VFS: Make lookup_hash() return a struct path
>  autofs4: Save autofs trigger's vfsmount in super block info
>  whiteout/NFSD: Don't return information about whiteouts to userspace
>  whiteout: Add vfs_whiteout() and whiteout inode operation
>  whiteout: Allow removal of a directory with whiteouts
>  whiteout: tmpfs whiteout support
>  union-mount: Introduce MNT_UNION and MS_UNION flags
>  union-mount: Free union stack on removal of topmost dentry from
>    dcache
>  union-mount: Create IS_MNT_UNION()
>
> Valerie Aurora (63):
>  VFS: Comment follow_mount() and friends
>  Documentation: Fix trivial typo in filesystems/sharedsubtree.txt
>  whiteout: Define opaque inode flags and operations
>  ext2: Add ext2_dirent_in_use()
>  ext2: Split ext2_add_entry() from ext2_add_link()
>  whiteout: ext2 whiteout support
>  fallthru: Basic fallthru definitions
>  fallthru: ext2 fallthru support
>  fallthru: tmpfs fallthru support
>  VFS: Add hard read-only users count to superblock
>  VFS: Make clone_mnt()/copy_tree()/collect_mounts() return errors
>  VFS: Add CL_NO_SHARED flag to clone_mnt()/copy_tree()
>  VFS: Add CL_NO_SLAVE flag to clone_mnt()/copy_tree()
>  VFS: Add CL_MAKE_HARD_READONLY flag to clone_mnt()/copy_tree()
>  union-mount: Union mounts documentation
>  union-mount: Add CONFIG_UNION_MOUNT option
>  union-mount: Create union_stack structure
>  union-mount: Add two superblock fields for union mounts
>  union-mount: Add union_alloc()
>  union-mount: Add union_find_dir()
>  union-mount: Create d_free_unions()
>  union-mount: Create union_add_dir()
>  union-mount: Add union_create_topmost_dir()
>  union-mount: Create needs_lookup_union()
>  union-mount: Create check_topmost_union_mnt()
>  union-mount: Add clone_union_tree() and put_union_sb()
>  union-mount: Create build_root_union()
>  union-mount: Create prepare_mnt_union() and cleanup_mnt_union()
>  union-mount: Prevent improper union-related remounts
>  union-mount: Prevent topmost file system from being mounted elsewhere
>  union-mount: Prevent bind mounts of union mounts
>  union-mount: Implement union mount
>  union-mount: Temporarily disable some syscalls
>  union-mount: Basic infrastructure of __lookup_union()
>  union-mount: Process negative dentries in __lookup_union()
>  union-mount: Return files found in lower layers in __lookup_union()
>  union-mount: Build union stack in __lookup_union()
>  union-mount: Follow mount in __lookup_union()
>  union-mount: Add lookup_union()
>  union-mount: Add do_lookup_union() wrapper for __lookup_union()
>  union-mount: Call union lookup functions in lookup path
>  union-mount: Create whiteout on unlink()
>  union-mount: Create whiteout on rmdir()
>  union-mount: Set opaque flag on new directories in unioned file
>    systems
>  union-mount: Copy up directory entries on first readdir()
>  union-mount: Add generic_readdir_fallthru() helper
>  fallthru: ext2 support for lookup of d_type/d_ino in fallthrus
>  fallthru: tmpfs support for lookup of d_type/d_ino in fallthrus
>  fallthru: jffs2 support for lookup of d_type/d_ino in fallthrus
>  VFS: Split inode_permission() and create path_permission()
>  VFS: Create user_path_nd() to lookup both parent and target
>  union-mount: In-kernel file copyup routines
>  union-mount: Implement union-aware access()/faccessat()
>  union-mount: Implement union-aware link()
>  union-mount: Implement union-aware rename()
>  union-mount: Implement union-aware writable open()
>  union-mount: Implement union-aware chown()
>  union-mount: Implement union-aware truncate()
>  union-mount: Implement union-aware chmod()/fchmodat()
>  union-mount: Implement union-aware lchown()
>  union-mount: Implement union-aware utimensat()
>  union-mount: Implement union-aware setxattr()
>  union-mount: Implement union-aware lsetxattr()
>

Shall I cry or laugh? I really don't know...

With your email I remembered my first steps with Linux Live-CD technologies.
As a home-project I created an own sidux live-cd and enlightenment as
window-manager to get a bit familiar with the technolgies/tools behind
it.
Still the mostly used and most effective "technology" in this area is
AUFS in combination with SquashFS compression (see for example Debian
/ GRML / ex-sidux Live-CD frameworks).

But I also remember these "famous words" [1]:

"Note: it becomes clear that "Aufs was rejected. Let's give it up."
According to Christoph Hellwig, linux rejects all union-type filesystems
but UnionMount."

Please hold the line... Please hold the line... Please hold the line???

Whuzzz up with AUFS?
Even there where massive changes to VFS and FS in 2.6.38+, there is an
adapted kernel patch around (for example see Debian's 2.6.38 linux-2.6
packages in experimental branch).

>From my POV OverlayFS is the new star at the skyline and should be
promoted as 1st choice, now.
I am definitely PRO for including it in 2.6.39!
And when looking to the code-size, OverlayFS is small while you send
70+ single patches.

Union-mounts never got out of "technology preview" status and was
stepmotherly promoted in the past.
You are leaving an outdated (unfinished?) code (IIRC I read 2.6.36-rc5
as code-base) and furthermore u-m needs hacked user-space, too.
So, u-m is for me - seen from today and having OverlayFS as an
alternative - a dead horse.
But, BKL-removal showed us... a once started job can be finished :-).

( Sorry, for the rough words. )

I do not want to end this email with some lights at the end of the
tunnel and want to quote Felix [2]:

[...]
> But I'd want Al's ack on the series. And also hear who uses it and how
> it's been tested?
We're using it in OpenWrt (an Embedded Linux distribution) for devices
with tiny amounts of flash for the entire system (e.g. 4 MB).
We're using it to provide a writable on-flash root filesystem with
squashfs for the read-only part and jffs2 for the writable overlay. This
saves some precious flash space compared to using only jffs2, and it
makes it easy for users to reset their device to defaults without having
to reflash.
With a backport of v6 of this series + my fixes that went into v7 this
is working quite well on 2.6.37 and 2.6.38 - I'm using it on a few
wireless access points at home.
[...]

OverlayFS GO GO GO!

- Sedat -

[1] http://aufs.sourceforge.net/
[2] http://lkml.org/lkml/2011/3/22/294

Union mounts comparison with overlay file system prototype?

[Ric Wheeler]

On 03/22/2011 10:12 PM, Valerie Aurora wrote:
> And I cc'd linux@vger.kernel.org on all the patches instead of
> linux-kernel@vger.kernel.org - guess I'm really out of the kernel
> business now!  Anyway, if you want your replies to go to lkml, you'll
> have to hand edit the cc list.
>
> -VAL
>
  Val, Miklos,

Can one or both of you summarize what we union mounts and overlay do better or 
worse? Do we need both or just one?

Thanks!

Ric

Re: [PATCH 00/74] Union mounts version something or other

[Ben Hutchings]

On Wed, Mar 23, 2011 at 09:38:39AM +0100, Sedat Dilek wrote:
[...]
> Whuzzz up with AUFS?
> Even there where massive changes to VFS and FS in 2.6.38+, there is an
> adapted kernel patch around (for example see Debian's 2.6.38 linux-2.6
> packages in experimental branch).

Do not cite the Debian kernel team as supporting aufs.  It is included
only because Debian Live needs some kind of union filesystem, and only
until that appears in-tree.

> From my POV OverlayFS is the new star at the skyline and should be
> promoted as 1st choice, now.
[...]

This thread is for technical review, not marketing.

Ben.

-- 
Ben Hutchings
We get into the habit of living before acquiring the habit of thinking.
                                                              - Albert Camus

Re: [PATCH 00/74] Union mounts version something or other

[Sedat Dilek]

On Thu, Mar 24, 2011 at 11:40 PM, Ben Hutchings <ben@decadent.org.uk> wrote:
> On Wed, Mar 23, 2011 at 09:38:39AM +0100, Sedat Dilek wrote:
> [...]
>> Whuzzz up with AUFS?
>> Even there where massive changes to VFS and FS in 2.6.38+, there is an
>> adapted kernel patch around (for example see Debian's 2.6.38 linux-2.6
>> packages in experimental branch).
>
> Do not cite the Debian kernel team as supporting aufs.  It is included
> only because Debian Live needs some kind of union filesystem, and only
> until that appears in-tree.
>

Debian is using AUFS - that is a fact!
That is exactly what I mean... as there is no *official* (and working!
and really used) union-filesystem from kernel-side.
Diverse distributions (and especially the embedded area) use AUFS (as
overlay) and SquashFS (for compression) as an *unofficial* working
solution for years. including Debian.
SquashsFS and hopefully SquashFS-XZ is in the kernel, but not AUFS.
So, I am interested in (a new discussion and) re-thinking what is the
number #1 choice in that area.
BTW, in the meantime Ric Wheeler asked for a comparison between
Union-mounts and OverlayFS  [1].
Let's see and read.

>> From my POV OverlayFS is the new star at the skyline and should be
>> promoted as 1st choice, now.
> [...]
>
> This thread is for technical review, not marketing.
>

My POV is clear - I already gave some technical arguments contra union-mounts.
An official and working(!) solution is needed and "promoted" from the
big five in Linux kernel (filesystem) development.
A decision what is the (next and new preferred?) standard
union-filesystem in the kernel-world.
Now, there is a new problem for union-mounts as one of its main
maintainer stopped working on it.
Even OverlayFS is young, it is already used as a working(!) solution in OpenWRT.
People do not need and want a never-ending "technical preview" for
years, they need and want a working(!) solution, that is/was mostly
AUFS chosen *unofficially* (and remember rejected into mainline).
Personally, I did not see/read union-mounts used by any distro or in
the embedded world.

- Sedat -

[1] http://www.spinics.net/lists/linux-fsdevel/msg43345.html

Re: Union mounts comparison with overlay file system prototype?

[Szeredi Miklos]

On Thu, Mar 24, 2011 at 2:43 PM, Ric Wheeler <rwheeler@redhat.com> wrote:
> Can one or both of you summarize what we union mounts and overlay do better
> or worse? Do we need both or just one?

The semantics are very similar, the differences are in the implementation.

Union mounts:

 - whiteout/opaque/fallthrough support in filesystems
 - whiteout operation is atomic
 - no dentry and inode duplication
 - copy up on lookup and readdir
 - does not support union of two read-only trees
 - merged directory stored in upper tree

Overlayfs

- whiteout/opaque as xattrs
- whiteout operation is not atomic
- dentry and inode duplication(*)
- only copy up on modification
- supports union of two read-only trees
- merged directory not cached(**)

(*) it's possible to eliminate inode duplication of non-directories
with some VFS modifications
(**) caching should be possible to do

Re: Union mounts comparison with overlay file system prototype?

[Ric Wheeler]

On 03/25/2011 07:38 AM, Szeredi Miklos wrote:
> On Thu, Mar 24, 2011 at 2:43 PM, Ric Wheeler<rwheeler@redhat.com>  wrote:
>> Can one or both of you summarize what we union mounts and overlay do better
>> or worse? Do we need both or just one?
> The semantics are very similar, the differences are in the implementation.
>
> Union mounts:
>
>   - whiteout/opaque/fallthrough support in filesystems
>   - whiteout operation is atomic
>   - no dentry and inode duplication
>   - copy up on lookup and readdir
>   - does not support union of two read-only trees
>   - merged directory stored in upper tree
>
> Overlayfs
>
> - whiteout/opaque as xattrs
> - whiteout operation is not atomic
> - dentry and inode duplication(*)
> - only copy up on modification
> - supports union of two read-only trees
> - merged directory not cached(**)
>
> (*) it's possible to eliminate inode duplication of non-directories
> with some VFS modifications
> (**) caching should be possible to do

Thanks for the high level overview!

Ric

Re: [PATCH 00/74] Union mounts version something or other

[Valerie Aurora]

On Wed, Mar 30, 2011 at 7:30 AM, David Howells <dhowells@redhat.com> wrote:
> Valerie Aurora <valerie.aurora@gmail.com> wrote:
>
>> As always, git trees for the kernel, util-linux, and e2fsprogs, lots
>> of documentation, and LWN articles describing the various problems
>> unioning file systems will encounter are here:
>>
>> http://valerieaurora.org/union/
>
> The webpage there says "linked_list" is the most recent branch, but that
> doesn't match with your patches posted here.  Is "+ext2_cleanup" the matching
> branch?

That's never up to date, but I just fixed it.  The branch you want is
"ext2_cleanup".  Any branch with "+" at the front is a mistake from
trying to do a git push of a rebased branch.  I deleted it.

-VAL

Re: [PATCH 00/74] Union mounts version something or other

[David Howells]

Valerie Aurora <valerie.aurora@gmail.com> wrote:

> That's never up to date, but I just fixed it.  The branch you want is
> "ext2_cleanup".  Any branch with "+" at the front is a mistake from
> trying to do a git push of a rebased branch.  I deleted it.

Okay, I've got that pulled up to Linus's head branch.  I've mostly got the RCU
pathwalk and managed dentry stuff correctly entangled.

However, I have a few questions:

 (1) Is it meant to be possible to unionmount over a mount tree rather than
     just a single mount?  I ask because do_lookup_union() calls
     __follow_mount().

 (2) When you open a file that exists in the lower layer but not the top
     layer, am I right in thinking that f_path points to the lower layer file?

 (3) If I'm correct in (2), I presume something must intercept fchown() and
     suchlike?

 (4) I presume IS_DIR_UNIONED() only gives true on the upper layer (the one
     that was mounted -o union)?

David

Re: [PATCH 00/74] Union mounts version something or other

[Valerie Aurora]

On Thu, Apr 21, 2011 at 6:09 AM, David Howells <dhowells@redhat.com> wrote:
> Valerie Aurora <valerie.aurora@gmail.com> wrote:
>
>> That's never up to date, but I just fixed it.  The branch you want is
>> "ext2_cleanup".  Any branch with "+" at the front is a mistake from
>> trying to do a git push of a rebased branch.  I deleted it.
>
> Okay, I've got that pulled up to Linus's head branch.  I've mostly got the RCU
> pathwalk and managed dentry stuff correctly entangled.

Awesome, thanks!

> However, I have a few questions:
>
>  (1) Is it meant to be possible to unionmount over a mount tree rather than
>     just a single mount?  I ask because do_lookup_union() calls
>     __follow_mount().

It's meant to allow mounting over a mount tree, as long as all the
mounts are read-only.   I don't recall the difference between
__follow_mount() and follow_mount() at this moment, but that code may
be left over from the time that we could only union mount a single
mount.

Think about it carefully though, and check my comments, and run the
union mounts test suite - I got this wrong a few times and added a
number of tests to make sure the mount point case is right.

>  (2) When you open a file that exists in the lower layer but not the top
>     layer, am I right in thinking that f_path points to the lower layer file?

If the file is opened read-write, then it is copied up and the f_path
points to the upper layer file.  If the file is opened read-only, then
it is not copied up and the f_path points to the lower layer file.
So, yes, f_path points to the lower layer file.

>  (3) If I'm correct in (2), I presume something must intercept fchown() and
>     suchlike?

There's a thread somewhere on this, hang on...

http://kerneltrap.org/mailarchive/linux-fsdevel/2010/3/29/6897953/thread

Basically, if you open the file read-write and do an fchown() on it,
it works fine because the file is copied up on open.  If you open the
file read-only and fchown() it (yes, that's permitted) then in union
mounts you will get EPERM or EBADF (don't recall which).  Actually
implementing this requires copy-up after open, which requires atomic
update of the struct file pointer, which is ugly and painful and what
we were trying to avoid in the first place.

I discussed this with Al and Christoph and the consensus was along the
lines of, "What?  You can do that?  POSIX is so stupid.  Yes, we don't
care if union mounts returns EPERM in this case that no one thinks
should work anyway."

If you can find a way to do this cleanly, hurray.  Otherwise the
current code works for fchown/fchmod/utimensat already in the open
read-write case.

>  (4) I presume IS_DIR_UNIONED() only gives true on the upper layer (the one
>     that was mounted -o union)?

Yes.  It maybe should be renamed...

Thanks,

-VAL