Re: The argument for fs assistance in handling archives

[Hans Reiser <reiser@namesys.com>]

The reason for supporting transactions in the kernel is that there is a 
whole slew of security bugs that have their origins in fs semantics 
being non-transactional.  rename does not cut it as a transactions API.  
rename results in horribly performing apps that engage in contortions 
that cost a lot of programmer time to code them to stuff a transaction 
into a file that can be renamed.  Most of the time programmers who 
should use a transaction/rename are just too busy to do it, and they 
take the risk that the user will lose data and be less secure rather 
than write the code to do it right.  When they do do it, because it is 
complex code, they often screw it up.

That said, there is a whole slew of reasons for not supporting 
transactions in the kernel:

* the history of the literature stars transactional filesystems that 
paid too high a cost in performance

* isolation is very tricky to do without killing performance

* transactions which are kept open too long are a problem

* avoiding problems with locking shared data structures long enough to 
cause painful exclusion of things that should be able to run in parallel 
just fine

Reiser4, by being very modest in its ambitions, manages to not pay a 
performance cost.   We say, isolation and rollback are for user space to 
manage until we can find a clever implementation, if such an 
implementation can even be found.  Even if such an implementation can be 
found, there are plenty of cases where special case knowledge of what 
the app is doing can improve performance a lot, and thus for that app it 
will belong in user space.

All reiser4 does, is allow you to guarantee (using sys_reiser4) that a 
limited size set of fs operations will be fused into one atom together, 
and therefor will either all survive a crash or none will survive.  This 
does however meet a lot of apps needs.

Hans

David Masover wrote:

> Linus Torvalds wrote:
> [...]
> | It's much saner (from _any_ app standpoint) to roll their own 
> database in
> | user space - that way it just works.
>
> Then why do we have apps which use Windows' Internet Options where it
> can, and otherwise ask the user for proxy settings?
>
> |
> | In other words, nobody is really ever going to take advantage of this.
> | This is _not_ how technical advanncement happens. The way you get people
> | to take advantage of something is to have a nice gradual ramp-up, not a
> | sudden new feature that they can't realistically use.
> |
> | So before you do transactions in the filesystem, you have to be able
> to do
> | them in user space - and then make the filesystem version be 
> _compatible_
> | with that user space library. That way you can get people to use the
> | library ("hey, it works anywhere") and then you can get them to use your
> | filesystem ("hey, if you use our super-duper filesystem, you can do what
> | you are already doing five times faster").
>
> Why not:
>     do transactions in the filesystem
>     write a library to talk to that filesystem
>     have the library do emulation on other systems
>
> in that order.
>
> | See? You're starting at the wrong end. Give me a portable interface to
> use
> | _first_. Then do transactions in the filesystem.
>
> If you're going to have to do both the portable interface and the fs
> support, does it really matter which order you do them in?
>
> Again, this is curiosity, not sarcasm.  Here my reasons for wanting the
> fs support first:
>
> You don't always know exactly how the filesystem transactions will work.
> ~ You don't know until it's done whether you'll deviate from your design,
> cut certain features, replace them with others...
>
> And if you write the portable library first, and it ends up supporting
> features a, b, and c, while the kernel supports features b, c, and d,
> which obsoletes a, you have to add some features and remove some
> features from the portable library to make it sane.
>
> It'd be like writing OpenGL entirely in software, before hardware
> accelerators work, and at the last minute have to change the library to
> use triangles instead of splines.  What's more, every single app now has
> to be rewritten or use the library's emulation -- that's assuming many
> apps would consider using a 3D library that doesn't have hardware accel
> support yet.