Re: fistgen-0.1.1 released (linux-2.6 support)

[Jamie Lokier <jamie@shareable.org>]

Erez Zadok wrote:
> Right, it looks like you want to support version trees (branches off of
> branches, etc. ala CVS), as well as some ability to create composable views
> that mix and match version "forks."

That is a complicated way of looking at it!

> We considered supporting version trees, but frankly, couldn't come up with a
> compelling application for those that would justify the added complexity.
> Such version trees are complex and hard to manage, and the most users get
> really confused with version trees (CVS branch management is enough of a
> reason to mess up :-)
> 
> Jamie, I'd love to get a detailed example of how one envisions using such a
> versioning system, so I can see how much our current work may support it,
> and what it'd take to make it do what you want.

One uses it by typing "cp -a dir1 dir2" or "cp -r dir1 dir2".

That's all.  There are no user-visible versions, no branch
management, and no other commands.

That is what the work so far (as discussed on linux-kernel a few
months ago) began.

> Nevertheless, I agree with you that Unionfs is probably a step in the right
> direction.  It should be fairly efficient b/c it only copies-up files when
> needed (and it can be modified to copy pages instead, as our Versioning f/s
> does in "Sparse" mode; or to always copy-on-read).

Copy-on-write pages is nice if you can do it.
Do you handle copy-on-write of mmap'd pages properly?
That is one requirement.

I am not convinced that a unionfs is the right way to implement this
if the user has to do lots of "mount" commands or equivalent.  It
really is just meant to be as simple as a copy optimisation -- if you
have to remember mounts, then it's too complicated to use.

However as an implementation method, if it can be made invisible to
users, then it might work.  So simplicity is another requirement.

I suspect that you see things in terms of versioning and branching,
with all the complexities that implies for users (just like
complicated VC tools).  But the point of the COW filesystem methods is
precisely to hide that from users and offer a much simpler data model.

(Perhaps there's a nice data model in between which we haven't
explored, which is both simple to understand yet more versatile than copy.)

Another requirement is that it doesn't add much performance overhead.
It should be about as fast as the underlying filesystem, except inQA
cases where implied copying is triggered, and it must not use much
more memory e.g. by doing something silly like copying pages in RAM.
This is another reason why I'm not sure a unionfs is appropriate,
although it could be if implemented in the right way.

A filesystem filter which overlays part of an underlying filesystem
might be the right way, especially if the overlay can be made
persistent (i.e. doesn't have to be manually mounted, in effect it
becomes stuck on).  I'm not sure if this is starting to go the way of
Hurd translators but it seems there may be some similarity.

-- Jamie