Re: fistgen-0.1.1 released (linux-2.6 support)

[Jamie Lokier <jamie@shareable.org>]

Erez Zadok wrote:
> > Btw, does it offer the facility for lazy
> > copy-on-write trees that was discussed on linux-kernel a few months
> > ago -- useful for jails and cloned compilation trees?
> 
> Jamie, I'll have to check that thread; I don't recall it on fsdevel (sorry,
> I don't have time to read lkml often, or 2000 messages per day :-).

It wasn't on fsdevel.  About linux-kernel I concur: I've been reading
it for 10 years and stopped cold a couple of months ago due to a
technical glitch.  I've been putting off reading it again, it's a burden.

> But from what you say, it sounds like you want some sort of combo of
> caching, snapshotting, and sandboxing, right?

No.

We want an equivalent to "cp -av" that is fast and doesn't use much
disk space.  Other than timing and disk capacity, it must be
semantically equivalent to copying a tree.

Snapshotting (as in recording a current state for restoring later)
wouldn't work, because cloned trees need to evolve independently.

Think about when you have 30 versions of a common source tree, each
configured differently and built with different compilation options,
and different patches applied.  For this application, "cp -rl" is
usable but it is risky and occasionally mistakes are made.

Sandboxing is appropriate for virtual machines and daemon jails: when
you create new VMs for users or new jails, you clone a distro tree.
Then it's ok to use some kind of sandboxing, although if it's
implemented as a "unionfs" where the common underlying tree must never
be changed as all VMs and jails depend on it, that's a huge limitation.

Sandboxing isn't convenient for the multiple source trees application,
because you sometimes want to operate on several of the trees at once,
e.g. to diff between them.

> We've developed a stackable unioning file system for linux
> called... Unionfs.  It's designed as a fan-out file system, which is a way
> for a stackable file system to stack on top of more than one other f/s or
> directory.  You can see a paper on it here
> 
> 	http://www.fsl.cs.sunysb.edu/project-unionfs.html
> 
> Since we wrote the paper, we also enhanced Unionfs to support dynamic
> addition and removal of "branches" in the union, including the ability mark
> certain branches as readonly or read-write, and including the ability to
> divert f/s ops to specific branches.  All this gives us a f/s that can
> produce snapshots and sandboxes.
> 
> Maybe this is a good coincidence because we're now preparing a release of
> unionfs (testing it throughly and fixing some bugs).  If all goes well, we
> hope to release it in a couple of weeks.

It sounds like a step in a positive direction, although it also sounds
rather inconvenient to use for some purposes, in contrast to a copy
optimisation which is easy to use.

-- Jamie