Re: silent semantic changes with reiser4

[Timothy Miller <miller@techsource.com>]

Linus Torvalds wrote:

> 
> The same goes for something like a "container file". Whether you see it as
> "dir-as-file" or "file-as-dir" (and I agree with Jan that the two are
> totally equivalent), the point of having the capability in the kernel is
> not that the operations cannot be done in user space - the point is that
> they cannot be done in user space _safely_. The kernel is kind of the
> thing that guarantees that everybody follows the rules.
> 

Well, it CAN be done safely if every client has to go through the kernel 
which does all the appropriate locks/semaphors and then passes the 
request down to the daemon.  (Isn't NFS implemented this way?)

This is very micro-kernel-ish, but it's a reasonable idea to do it this 
way in cases where things are non-critical.

Say there's a way to cd into a tgz file to look around.  If the access 
methods through the kernel get routed back to a user-space process 
(which probably does some amount of caching in memory and on disk of 
uncompressed bits of the archive), it could be a bit slower than if it 
were all in-kernel.  The thing is that the processing time in the daemon 
is probably quite high compared to the overhead of the context switches, 
so it doesn't cost much.  (And if it CAN be done in userspace without 
being horribly convoluted or unsafe, then it SHOULD be done in 
userspace.)  Besides, even if it were a LOT slower to access a tgz file 
without extracting it first, I would STILL think it was wonderful AND 
use it a LOT.

Heh... now, the next thing is to be able to do some sort of union-mount 
like thing where the tgz is really read-only but if you WRITE to it, it 
gets stored somewhere else.  This way, you can compile a kernel tree 
without extracting it first.

Taking this a bit further, the union-mount could be done away with if 
the daemon were to lazily rebuild the tgz file from its cache when the 
tgz is modified.  That is, writes have little or no penalty because they 
just get stored in the daemon's cache, and the daemon rebuilds the 
archive in the background.

The VFS layer would have to recognize that a particular file has been 
accessed in such a way that it must go through the daemon for ANY access 
to it, which means that if someone tries to access the original archive 
as a file, requests have to go through the daemon.  This makes it 
coherent, although with an expected performance penalty.

One issue to consider is that of shutdowns, both intentional and 
crashes.  This is a non-issue with an underlying journaling file system. 
  As long as the tgz daemon comes up early enough, it would see its 
cache (which is on-disk anyway), recognize which archive it belongs to, 
and continue where it left off.  This way, the daemon doesn't have to go 
through a long process of rebuilding every archive you've ever modified 
on the disk on shut-down.  If, however, you needed to access the archive 
without the daemon, that could be a problem (but what you'd get is 
probably the archive before it was modified).  Also, some sort of sync 
command would be needed to cause the daemon to rebuild all archives 
immediately.

I really dig the idea of being able to access archives without 
extracting them first, regardless of any performance penalty.