future r4 maintenance question

future r4 maintenance question

[Pysiak Satriani]

Hello,

one of the concerns of LKML folks about r4 is that namesys after
finishing their work with r4 will "rush off" to reiser5 leaving
r4 as "stable" in a "bugfix-mode only"

I understand that way of things - you don't mess with a "stable" branch
software.

But a question arises. Will namesys keep r4 in a bugfix-only mode
(as with reiserfs v3) or will it have resources planned not only
for bugfixes but for updates too ?

I think that's a thing that needs to be very clear.

Best regards,
Maciej

Re: future r4 maintenance question

[Hans Reiser]

We are going to give changing the paradigm a try.  The difference
between 4.1-beta and 4.0 is that different plugins are the default, and
the experimental code is in the plugins you see when mounting with the
mount option 4.1-beta.   Let's see if it works in practice.....

Hans

Pysiak Satriani wrote:

>Hello,
>
>one of the concerns of LKML folks about r4 is that namesys after
>finishing their work with r4 will "rush off" to reiser5 leaving
>r4 as "stable" in a "bugfix-mode only"
>
>I understand that way of things - you don't mess with a "stable" branch
>software.
>
>But a question arises. Will namesys keep r4 in a bugfix-only mode
>(as with reiserfs v3) or will it have resources planned not only
>for bugfixes but for updates too ?
>
>I think that's a thing that needs to be very clear.
>
>Best regards,
>Maciej
>
>
>
>
>  
>

Re: future r4 maintenance question

[Maciej Sołtysiak]

Hello Hans,

Saturday, July 22, 2006, 8:03:28 PM, you wrote:

> We are going to give changing the paradigm a try.  The difference
> between 4.1-beta and 4.0 is that different plugins are the default, and
> the experimental code is in the plugins you see when mounting with the
> mount option 4.1-beta.   Let's see if it works in practice.....
I Understand. This is good news. Hm, do you think that reiser4's pluggability
is enough to have this single kernel tree (fs/reiser4) for a longer period
of time. I mean, can you predict a need of spawning something like reiser5
in the forseeable future or would fs/reiser4 + plugins be enough to do
away with the "future vision" and other "future *" stuff you've written
about ? eg. I remember reading about very granular security ACLs like
restricting a certain line in a file (like /etc/passwd)

-- 
Best regards,
Maciej

Re: future r4 maintenance question

[David Masover]

Maciej Sołtysiak wrote:

> I Understand. This is good news. Hm, do you think that reiser4's pluggability
> is enough to have this single kernel tree (fs/reiser4) for a longer period

> away with the "future vision" and other "future *" stuff you've written
> about ? eg. I remember reading about very granular security ACLs like
> restricting a certain line in a file (like /etc/passwd)

I think the granular security ACLs were always intended to be a v4 thing 
(with plugins).  I believe the Future Vision stuff is about even more 
namespace unification.  I seem to remember v5 was semantic changes and 
v6 was a semi-distributed FS, but I could have that backwards... 
Anyway, there are definitely plans for things which are non-trivially 
beyond the plugin capabilities of Reiser4, though they will almost 
certainly use Reiser4 as a starting point.

There's another note to kernel developers -- if Reiser5, 6, and 7 are 
implemented as suites of plugins on top of Reiser4, then the Reiser4 
code will be maintained for a very long time.  Kind of like ext2 vs 
ext3, only moreso -- a Reiser5 FS may well be a Reiser4 FS mounted with 
additional mount options.

There is definitely a lot that can be done to move Reiser4 (as it is 
today) closer to the Reiser4 whitepaper on the homepage.  ACLs are one 
thing, files as a directory are another.  The idea of v4 is to do away 
with many cases where a separate namespace is created for no good reason 
-- for instance, where is the data in an id3 tag?  It's inside an mp3 
file, and you can only get to it with tools written for id3 tags of mp3 
files.  The Reiser4 concept is to allow things like that to exist, but 
not require programs to know about libid3 or whatever.  Want to know 
what the artist of a particular file is?

foo.mp3/.../id3/artist

Or maybe a more generic way:

foo.mp3/.../song-info/artist

That way, you could have tools which don't even have to know if the file 
has an id3 tag, or something entirely new, or if the metadata is being 
stored outside the main file.  It'd be entirely possible to allow that 
file to be treated as a separate file entirely by the plugin, rather 
than something derived from foo.mp3.

The advantages don't seem immediately obvious until you consider that 
the program which does this doesn't have to even know that it's dealing 
with song metadata.  Consider some of the one-line shell scripts possible:


# Change the artist name for all songs in the directory:
for i in *; do echo 'Jimi Hendrix' > foo.mp3/.../song-info/artist; done

# Make a playlist of all files by Hendrix, mp3 or otherwise:
for i in `find`; do
	if [ `cat "$i/.../song-info/artist"` == 'Jimi Hendrix' ]; then
		echo "../$i" >> playlists/hendrix.m3u;
	fi;
done

# Copy all files needed by said playlist to a USB device:
cp playlist/hendrix.m3u/.../files/* /mnt/usb
cp playlist/hendrix.m3u /mnt/usb


I'm sure others can think of much more interesting examples.

All of that is planned for v4, eventually.  It's very pluggable.

Well, I think it is.  I don't work here...

Re: future r4 maintenance question

[Maciej Sołtysiak]

Hello David,

Saturday, July 22, 2006, 10:16:48 PM, you wrote:
> There's another note to kernel developers -- if Reiser5, 6, and 7 are
> implemented as suites of plugins on top of Reiser4, then the Reiser4 
> code will be maintained for a very long time.  Kind of like ext2 vs 
> ext3, only moreso -- a Reiser5 FS may well be a Reiser4 FS mounted with
> additional mount options.
This the exact information I was hoping to get. I think it's important
for people to realize it. Thanks.

> # Make a playlist of all files by Hendrix, mp3 or otherwise:
> for i in `find`; do
>         if [ `cat "$i/.../song-info/artist"` == 'Jimi Hendrix' ]; then
>                 echo "../$i" >> playlists/hendrix.m3u;
>         fi;
> done
I've read about this :-)

Are plugins for such a thing entirely/mostly kernel-space?

I mean, the power of FUSE is being in user-space...
If there was an API to write a id3 plugin it propably be more
beneficial.

Well, it's plugins. If someone writes a kernelspace id3 plugin,
someone else could write a kernelspace plugin with a userspace
supplied configuration.

-- 
Best regards,
Maciej

Re: future r4 maintenance question

[Hans Reiser]

I am going to do the enhanced semantics first, so that somebody does not
beat me to it.

David's examples are good.

>
> There's another note to kernel developers -- if Reiser5, 6, and 7 are
> implemented as suites of plugins on top of Reiser4, then the Reiser4
> code will be maintained for a very long time.  Kind of like ext2 vs
> ext3, only moreso -- a Reiser5 FS may well be a Reiser4 FS mounted
> with additional mount options.
>
> There is definitely a lot that can be done to move Reiser4 (as it is
> today) closer to the Reiser4 whitepaper on the homepage.  ACLs are one
> thing, files as a directory are another.  The idea of v4 is to do away
> with many cases where a separate namespace is created for no good
> reason -- for instance, where is the data in an id3 tag?  It's inside
> an mp3 file, and you can only get to it with tools written for id3
> tags of mp3 files.  The Reiser4 concept is to allow things like that
> to exist, but not require programs to know about libid3 or whatever. 
> Want to know what the artist of a particular file is?
>
> foo.mp3/.../id3/artist
>
> Or maybe a more generic way:
>
> foo.mp3/.../song-info/artist
>
> That way, you could have tools which don't even have to know if the
> file has an id3 tag, or something entirely new, or if the metadata is
> being stored outside the main file.  It'd be entirely possible to
> allow that file to be treated as a separate file entirely by the
> plugin, rather than something derived from foo.mp3.
>
> The advantages don't seem immediately obvious until you consider that
> the program which does this doesn't have to even know that it's
> dealing with song metadata.  Consider some of the one-line shell
> scripts possible:
>
>
> # Change the artist name for all songs in the directory:
> for i in *; do echo 'Jimi Hendrix' > foo.mp3/.../song-info/artist; done
>
> # Make a playlist of all files by Hendrix, mp3 or otherwise:
> for i in `find`; do
>     if [ `cat "$i/.../song-info/artist"` == 'Jimi Hendrix' ]; then
>         echo "../$i" >> playlists/hendrix.m3u;
>     fi;
> done
>
> # Copy all files needed by said playlist to a USB device:
> cp playlist/hendrix.m3u/.../files/* /mnt/usb
> cp playlist/hendrix.m3u /mnt/usb
>
>
> I'm sure others can think of much more interesting examples.
>
> All of that is planned for v4, eventually.  It's very pluggable.
>
> Well, I think it is.  I don't work here...
>
>

Re: future r4 maintenance question

[Hans Reiser]

Maciej Sołtysiak wrote:

>Hello Hans,
>
>Saturday, July 22, 2006, 8:03:28 PM, you wrote:
>
>  
>
>>We are going to give changing the paradigm a try.  The difference
>>between 4.1-beta and 4.0 is that different plugins are the default, and
>>the experimental code is in the plugins you see when mounting with the
>>mount option 4.1-beta.   Let's see if it works in practice.....
>>    
>>
>I Understand. This is good news. Hm, do you think that reiser4's pluggability
>is enough to have this single kernel tree (fs/reiser4) for a longer period
>of time.
>
Yes, reiser4 will have a much longer lifetime,  and improvements will
come out in small pieces rather than complete rewrites.  I think we can
add the enhanced semantics one feature at a time.  It was the storage
layer that was the big thing that had to be right before the rest could
proceed.  Now, what remains are a whole lot of incremental improvements
I hope.  If I can make enough money off the repacker or I get funding
from France or some other government and we are able to afford to keep
vs and zam working  on the storage layer, and Nate working on VFS
changes and Peter Foldiak and others at St. Andrews doing semantic
enhancements and Gorazd doing user space browsers things could get very
interesting.....  We need to get into the kernel, and money will
manifest, and programmers can get to work....

> I mean, can you predict a need of spawning something like reiser5
>in the forseeable future or would fs/reiser4 + plugins be enough to do
>away with the "future vision" and other "future *" stuff you've written
>about ? eg. I remember reading about very granular security ACLs like
>restricting a certain line in a file (like /etc/passwd)
>  
>
Yes, I still want to do that stuff....

Hans

Re: future r4 maintenance question

[Mike Benoit]

[-- Attachment #1: Type: text/plain, Size: 1807 bytes --]

On Sat, 2006-07-22 at 17:32 +0200, Pysiak Satriani wrote:
> Hello,
> 
> one of the concerns of LKML folks about r4 is that namesys after
> finishing their work with r4 will "rush off" to reiser5 leaving
> r4 as "stable" in a "bugfix-mode only"
> 
> I understand that way of things - you don't mess with a "stable" branch
> software.
> 
> But a question arises. Will namesys keep r4 in a bugfix-only mode
> (as with reiserfs v3) or will it have resources planned not only
> for bugfixes but for updates too ?
> 
> I think that's a thing that needs to be very clear.
> 

I don't understand the concerns here? Kernel v2.4 is essentially in
"bug-fix only" mode right now, yet I don't see many people complaining
that the kernel developers have abandoned it. I personally think its
crazy to expect major features be back ported to old versions. 

For as long as I've been alive programmers have released a major version
of their software, and only maintained it with minor version bug fixes
while they work on the next major version to introduce features to it. 

Hans himself has pretty much followed that same path with ReiserFS V3,
and now V4. There just happened to be a several year gap in between.

I don't know what Hans has in mind in regards to ReiserV5, but ReiserV4
was designed with plugins in mind so adding additional features (even
advanced ones) to it should be much easier than most other file systems.

Just like many other applications that support plugins/extensions/addons
( like Firefox, Thunderbird, Azureus ) I would be incredibly surprised
if Reiser4 doesn't spawn a large community of people completely
independent of Namesys that create plugins from the useless, to the
amazingly useful for years to come.

-- 
Mike Benoit <ipso@snappymail.ca>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: future r4 maintenance question

[Matthias Barremaecker]

> Just like many other applications that support plugins/extensions/addons
> ( like Firefox, Thunderbird, Azureus ) I would be incredibly surprised
> if Reiser4 doesn't spawn a large community of people completely
> independent of Namesys that create plugins from the useless, to the
> amazingly useful for years to come.
> 

if it ever gets finished ;) :)

Re: future r4 maintenance question

[David Masover]

Mike Benoit wrote:

> Just like many other applications that support plugins/extensions/addons
> ( like Firefox, Thunderbird, Azureus ) I would be incredibly surprised
> if Reiser4 doesn't spawn a large community of people completely
> independent of Namesys that create plugins from the useless, to the
> amazingly useful for years to come.

You would hope.  The problem is, most of the ideas we have for 
amazing/useless/practical plugins have already been done as specialized 
filesystems in FUSE.  What's more, FUSE doesn't usually require root 
privileges to run, causing some people to do such useless things as 
support for various filesystems already in the kernel (so you can 
loop-mount filesystems without using the loop device or having root 
privileges).

Although, there is one other thing to consider -- NTFS in the kernel has 
stagnated for something like 10 years without write support, while 
userspace tools like ntfsck and resize_ntfs, using libntfs, have 
actually gotten relatively stable at dealing with all kinds of ntfs 
oddities.  For awhile, we had captive-ntfs (also done in FUSE), but more 
recently, someone wrote a FUSE driver for libntfs, making the first 
reasonably stable fully featured read/write NTFS driver for Linux.

It's also reasonably fast, sometimes faster than ext2/3, I'm told.

The big advantage of that is, FUSE is easier to deal with than the 
kernel, and userland daemons can crash, but if something in the kernel 
crashes, there goes your OS.

Also, userland just has more options.  Some FUSE drivers are written in 
Perl, others in Ruby.

I realize there are some things that are really better done in the 
kernel, like crypto/compression support, and Reiser4 will support things 
that FUSE currently doesn't do very well.  But if we're to pull any 
filesystem hackers away from FUSE, we need to provide as good or better 
an API, both in-kernel, and for userspace daemons.

Re: future r4 maintenance question

[Sander Sweers]

On Sat, 2006-07-22 at 14:42 -0500, David Masover wrote:
> Mike Benoit wrote:
> 
[...]
> 
> You would hope.  The problem is, most of the ideas we have for 
> amazing/useless/practical plugins have already been done as specialized 
> filesystems in FUSE.  What's more, FUSE doesn't usually require root 
> privileges to run, causing some people to do such useless things as 
> support for various filesystems already in the kernel (so you can 
> loop-mount filesystems without using the loop device or having root 
> privileges).
> 
> Although, there is one other thing to consider -- NTFS in the kernel has 
> stagnated for something like 10 years without write support, while 
> userspace tools like ntfsck and resize_ntfs, using libntfs, have 
> actually gotten relatively stable at dealing with all kinds of ntfs 
> oddities.  For awhile, we had captive-ntfs (also done in FUSE), but more 
> recently, someone wrote a FUSE driver for libntfs, making the first 
> reasonably stable fully featured read/write NTFS driver for Linux.
> 
> It's also reasonably fast, sometimes faster than ext2/3, I'm told.

I am using the official libntfs fuse module but it is not all that fast
if you compare how much more cpu power is needed. And even then it is
NOT faster then ext2/3. I am copying a couple of 650mb iso's and I have
100% cpu utilisation and it still takes tens of minutes to copy just 1.

Fuse is nice for things like gmailfs or beaglefs but if you really want
performance, kernelspace imho is the only place to implement a
filesystems.

Greets
Sander

Re: future r4 maintenance question

[Maciej Sołtysiak]

Hello Mike,

Saturday, July 22, 2006, 8:53:06 PM, you wrote:
> I don't understand the concerns here? Kernel v2.4 is essentially in
> "bug-fix only" mode right now, yet I don't see many people complaining
> that the kernel developers have abandoned it. I personally think its
> crazy to expect major features be back ported to old versions. 

> For as long as I've been alive programmers have released a major version
> of their software, and only maintained it with minor version bug fixes
> while they work on the next major version to introduce features to it.

> Hans himself has pretty much followed that same path with ReiserFS V3,
> and now V4. There just happened to be a several year gap in between.
Well, that's completely clear. However, reiserfs has had a different situation:
during the bugfix mode, important functionality was beeing added to other
filesystems: extended attribute support, data=ordered, data=journaled.
Reiserfs v3 had noone but Chris Mason (outside of namesys) to do it.

The issue is that reiser4 users should never again feel abandoned like they
did with reiserfs v3. Well, actually a lot of people did not feel it or they
hadn't known about the state of things. Note that back then reiser4 was
basically vapourware if I may say so.

Concluding, I am not here to point anything bad, I am looking adoption-stoppers
for reiser4.

BTW, have Mark Shuttleworth and Hans had a talk about support/sponsorship/ubuntu
endorsement/anything? (i know it may be absolutely private stuff, sorry for asking
if this is the case)

-- 
Best regards,
Maciej

Re: future r4 maintenance question

[David Masover]

Pysiak Satriani wrote:
> Hello,
> 
> one of the concerns of LKML folks about r4 is that namesys after
> finishing their work with r4 will "rush off" to reiser5 leaving
> r4 as "stable" in a "bugfix-mode only"

> But a question arises. Will namesys keep r4 in a bugfix-only mode
> (as with reiserfs v3) or will it have resources planned not only
> for bugfixes but for updates too ?

As I understand it, r4 is currently not finished, especially comparing 
the whitepaper with reality.  So, while the features that are finished 
are mostly stable, I can't imagine them rushing off to v5 without 
finishing the features originally planned for v4.

Examples would be pseudofiles, custom plugins (per-file and 
per-directory, also), a repacker, fsync optimization, a 
crypto/compression plugin...

Ultimately, this may happen, but by then, I expect v4 would be fairly 
stable and complete.  Besides, bugfix-only does imply bugfixes, which is 
about all most other filesystems get these days...

Disclaimer:  I don't work here.  If you want a definitive answer, wait 
for someone at Namesys to confirm or deny what I've just said.