Re: [RFC 2/2] AUFS: merging/stacking several filesystems

[hooanon05@yahoo.co.jp]

> > Here are some of them and the intention of this post is to get some
> > initial feedback about its design.
	:::

I have posted some of ideas, design or approaches which are implemented
in AUFS stackable filesystem about a month before.
While I have a plan to implement some more features still, the current
AUFS status is better and used many people for years.
Since I have received requests to submit AUFS into the mainline more
than once, Now I'd ask you to include AUFS into mainline.
But the source is large (see below).
Should I send all of these files to this ML, or ask you to download them
from CVS?
If AUFS was much smaller, I would send files here without asking.


Junjiro Okajima

----------------------------------------------------------------------
$ wc -l fs/aufs25/*.[ch]
    56 fs/aufs25/aufs.h
   109 fs/aufs25/br_fuse.c
   391 fs/aufs25/br_nfs.c
    69 fs/aufs25/br_xfs.c
   932 fs/aufs25/branch.c
   345 fs/aufs25/branch.h
  1043 fs/aufs25/cpup.c
    82 fs/aufs25/cpup.h
   246 fs/aufs25/dcsub.c
    54 fs/aufs25/dcsub.h
   485 fs/aufs25/debug.c
   210 fs/aufs25/debug.h
  1020 fs/aufs25/dentry.c
   384 fs/aufs25/dentry.h
   425 fs/aufs25/dinfo.c
   573 fs/aufs25/dir.c
   146 fs/aufs25/dir.h
   113 fs/aufs25/dlgt.c
   597 fs/aufs25/export.c
   661 fs/aufs25/f_op.c
   826 fs/aufs25/file.c
   246 fs/aufs25/file.h
   185 fs/aufs25/finfo.c
   708 fs/aufs25/hin_or_dlgt.c
   188 fs/aufs25/hinode.h
  1114 fs/aufs25/hinotify.c
   844 fs/aufs25/i_op.c
   828 fs/aufs25/i_op_add.c
   582 fs/aufs25/i_op_del.c
   832 fs/aufs25/i_op_ren.c
   290 fs/aufs25/iinfo.c
   425 fs/aufs25/inode.c
   336 fs/aufs25/inode.h
   307 fs/aufs25/misc.c
   201 fs/aufs25/misc.h
   243 fs/aufs25/module.c
    78 fs/aufs25/module.h
  1489 fs/aufs25/opts.c
   245 fs/aufs25/opts.h
   349 fs/aufs25/plink.c
   111 fs/aufs25/robr.c
   268 fs/aufs25/sbinfo.c
   906 fs/aufs25/super.c
   409 fs/aufs25/super.h
   107 fs/aufs25/sysaufs.c
   150 fs/aufs25/sysaufs.h
   498 fs/aufs25/sysfs.c
   112 fs/aufs25/sysrq.c
   963 fs/aufs25/vdir.c
   653 fs/aufs25/vfsub.c
   493 fs/aufs25/vfsub.h
   693 fs/aufs25/wbr_policy.c
  1058 fs/aufs25/whout.c
   140 fs/aufs25/whout.h
   321 fs/aufs25/wkq.c
   160 fs/aufs25/wkq.h
  1304 fs/aufs25/xino.c
 26603 total
----------------------------------------------------------------------

(http://aufs.sf.net)

Aufs -- Another Unionfs
Junjiro Okajima

# $Id: README,v 1.79 2008/05/04 23:55:14 sfjro Exp $


0. Introduction
----------------------------------------
In the early days, aufs was entirely re-designed and re-implemented
Unionfs Version 1.x series. After many original ideas, approaches,
improvements and implementations, it becomes totally different from
Unionfs while keeping the basic features.
Recently, Unionfs Version 2.x series begin taking some of same
approaches to aufs's.
Unionfs is being developed by Professor Erez Zadok at Stony Brook
University and his team.
If you don't know Unionfs, I recommend you becoming familiar with it
before using aufs. Some terminology in aufs follows Unionfs's.

Bug reports (including my broken English), suggestions, comments
and donations are always welcome. Your bug report may help other users,
including future users. Especially the bug report which doesn't follow
unix/linux filesystem's semantics is important.


1. Features
----------------------------------------
- unite several directories into a single virtual filesystem. The member
  directory is called as a branch.
- you can specify the permission flags to the branch, which are 'readonly',
  'readwrite' and 'whiteout-able.'
- by upper writable branch, internal copyup and whiteout, files/dirs on
  readonly branch are modifiable logically.
- dynamic branch manipulation, add, del.
- etc... see Unionfs in detail.

Also there are many enhancements in aufs, such as:
- safer and faster
- keep inode number by external inode number table
- keep the timestamps of file/dir in internal copyup operation
- seekable directory, supporting NFS readdir.
- support mmap(2) including /proc/PID/exe symlink, without page-copy
- whiteout is hardlinked in order to reduce the consumption of inodes
  on branch
- do not copyup, nor create a whiteout when it is unnecessary
- revert a single systemcall when an error occurs in aufs
- remount interface instead of ioctl
- maintain /etc/mtab by an external shell script, /sbin/mount.aufs.
- loopback mounted filesystem as a branch
- kernel thread for removing the dir who has a plenty of whiteouts
- support copyup sparse file (a file which has a 'hole' in it)
- default permission flags for branches
- selectable permission flags for ro branch, whether whiteout can
  exist or not
- export via NFS.
- support <sysfs>/fs/aufs.
- support multiple writable branches, some policies to select one
  among multiple writable branches.
- a new semantics for link(2) and rename(2) to support multiple
  writable branches.
- a delegation of the internal branch access to support task I/O
  accounting, which also supports Linux Security Modules (LSM) mainly
  for Suse AppArmor.
- nested mount, i.e. aufs as readonly no-whiteout branch of another aufs.
- copyup-on-open or copyup-on-write
- show-whiteout mode
- show configuration even out of kernel tree
- no glibc changes are required.
- and more... see aufs manual in detail

Aufs is in still development stage, especially:
- pseudo hardlink (hardlink over branches)
- allow a direct access manually to a file on branch, e.g. bypassing aufs.
  including NFS or remote filesystem branch.
- refine xino and revalidate
- pseudo-link in NFS-exporting

(current work)
- reorder the branch index without del/re-add.
- permanent xino files

(next work)
- an option for refreshing the opened files after add/del branches
- 'move' policy for copy-up between two writable branches, after
  checking free space.
- ioctl to manipulate file between branches.
- and documentation

(just an idea)
- remount option copy/move between two branches. (unnecessary?)
- O_DIRECT (unnecessary?)
- light version, without branch manipulation. (unnecessary?)
- SMP, because I don't have such machine. But several users reported
  aufs is working fine on SMP machines.
- copyup in userspace
- inotify in userspace
- xattr, acl


2. Download
----------------------------------------
CVS tree is in aufs project of SourceForge.
Here is simple instructions to get aufs source files. It is recommended to
refer to the document about CVS on SourceForge.
	$ mkdir aufs.wcvs
	$ cd aufs.wcvs
	$ cvs -d:pserver:anonymous@aufs.cvs.sourceforge.net:/cvsroot/aufs login
	(CVS password is empty)
	$ cvs -z3 -d:pserver:anonymous@aufs.cvs.sourceforge.net:/cvsroot/aufs co aufs

In order to update files after the first checkout,
	$ cd aufs.wcvs/aufs
	$ cvs update
Do not forget -A option for 'cvs update' if you have ever 'cvs update' with
specifying a file version.

In order to see what the difference between two versions (two dates) is,
	$ cd aufs.wcvs/aufs
	$ cvs diff -D20061212 -D20061219

Usually I am updating CVS tree on every Monday.
I always try putting the stable version in CVS, so you can try CVS
instead of SourceForge File Release. And every changes are summarized
and reported to aufs-users at lists.sourceforge.net ML. I'd like to
recommend you to join this ML.

(snip)