On plugins, on-disk formatting and such

On plugins, on-disk formatting and such

[mjt]

First off I'd like to thank the Namesys guys for their excellent work,
the latest snapshot has been very stable for me, so stable in fact
that my life is boring because of it ;)

I have like a ton of questions so I'll just put them in this mail and
see which points get replied to. Hopefully all ;)

1) The xattrs issue never got any replies, although I gathered that it
   would be possible to map xattr calls to the Reiser4 API.
   Is this fact or fiction? What will the Reiser4 API look like, apart
   from control files?

2) Is metadata stored anywhere? If I want to organize all my libs in one
   fibre, I make it so, I reboot and do I then have to reset the fibration
   for the directory? Is it possible to have persistency here? Wouldn't
   a wrong fibration type make the files inaccessible?
   Is this addressed in the Reiser4 API somehow?

3) How big are the chances that the on-disk layout will still change?
   On IRC I've got the impression that the chance is next to none,
   and I thank you for that!
   Still, a lot of people are hesitant to try Reiser4 because of this
   fear. Doing an mkfs is not a small thing and some assurance would
   surely bring along community testing support.

4) http://thebsh.namesys.com/snapshots/2004.03.26/READ.ME
   4.1) I have used Reiser4 as my rootfs for a long, long time.
        Why does the READ.ME say I can't do it?-) I just never mount
        it read-only, so I don't have to remount it read-write and it
        just works. Is there something amiss here?
   
   4.2) What exactly does it mean that CONFIG_REISER4_COPY_ON_CAPTURE
        is stabler than before? Is it soon safe to configure in?
        Is it possible for me to test it on only one filesystem or
        do I risk rootfs (and home directory) stability by trying it
        out? If I compile it in, what's the best way of testing it?

5) If I wanted to write an fs plugin, I gathered that I would not have
   to mkfs again. Should this not require some persistency in metadata?
   I also gathered that Hans Reiser is against loadable kernel modules
   that are in essence filesystem plugins. Is this because a stray
   rmmod might cause breakage or something else?
   One of the good things for me in Linux is that I can use honest
   modules to try something, rmmod, recompile, insmod and I don't
   think this is any different.

6) I understood there are some NFS issues still left, what other issues
   are there? Is it possible to have some TODO list on the web so
   people could guess at what's going on? (BTW, Nikita, that doesn't
   mean I won't bug you in the future ;)

7) http://namesys.com/benchmarks.html#slow.2004.03.26
   That is awesome. Could someone give a brief, or rather human-readable,
   explanation on why Reiser4 refuses to slow down under those loads?
   Not that I'd find those benchmarks hard to swollow per se, I did
   see Reiser4 perform superior for myself.
   But what is the secret? Atomic transactions?

Thanks again for the good work and I hope this brings along some discussion
more interesting than the metas/ thread!

-- 
mjt

Re: On plugins, on-disk formatting and such

[Alex Zarochentsev]

On Sat, Apr 10, 2004 at 01:07:01PM +0300, Markus Törnqvist wrote:
> First off I'd like to thank the Namesys guys for their excellent work,
> the latest snapshot has been very stable for me, so stable in fact
> that my life is boring because of it ;)
> 
> I have like a ton of questions so I'll just put them in this mail and
> see which points get replied to. Hopefully all ;)
> 
> 1) The xattrs issue never got any replies, although I gathered that it
>    would be possible to map xattr calls to the Reiser4 API.
>    Is this fact or fiction? What will the Reiser4 API look like, apart
>    from control files?

xattrs are possible but we have no plans to use them.  
2 APIs are: pseudo files and reiser4 syscall.

> 
> 2) Is metadata stored anywhere? If I want to organize all my libs in one
>    fibre, I make it so, I reboot and do I then have to reset the fibration
>    for the directory? Is it possible to have persistency here? Wouldn't
>    a wrong fibration type make the files inaccessible?
>    Is this addressed in the Reiser4 API somehow?

reiser4 is able to store plugin sets (list of plugin ids for the object) as 
a part of the object's stat data.

> 
> 3) How big are the chances that the on-disk layout will still change?
>    On IRC I've got the impression that the chance is next to none,
>    and I thank you for that!
>    Still, a lot of people are hesitant to try Reiser4 because of this
>    fear. Doing an mkfs is not a small thing and some assurance would
>    surely bring along community testing support.
> 
> 4) http://thebsh.namesys.com/snapshots/2004.03.26/READ.ME
>    4.1) I have used Reiser4 as my rootfs for a long, long time.
>         Why does the READ.ME say I can't do it?-) I just never mount
>         it read-only, so I don't have to remount it read-write and it
>         just works. Is there something amiss here?
>    
>    4.2) What exactly does it mean that CONFIG_REISER4_COPY_ON_CAPTURE
>         is stabler than before? Is it soon safe to configure in?
>         Is it possible for me to test it on only one filesystem or
>         do I risk rootfs (and home directory) stability by trying it
>         out? If I compile it in, what's the best way of testing it?

we still see fs errors if COC enable. they are rare now, but COC is not stable
yet.

> 
> 5) If I wanted to write an fs plugin, I gathered that I would not have
>    to mkfs again. Should this not require some persistency in metadata?
>    I also gathered that Hans Reiser is against loadable kernel modules
>    that are in essence filesystem plugins. Is this because a stray
>    rmmod might cause breakage or something else?
>    One of the good things for me in Linux is that I can use honest
>    modules to try something, rmmod, recompile, insmod and I don't
>    think this is any different.
> 
> 6) I understood there are some NFS issues still left, what other issues
>    are there? Is it possible to have some TODO list on the web so
>    people could guess at what's going on? (BTW, Nikita, that doesn't
>    mean I won't bug you in the future ;)
> 
> 7) http://namesys.com/benchmarks.html#slow.2004.03.26
>    That is awesome. Could someone give a brief, or rather human-readable,
>    explanation on why Reiser4 refuses to slow down under those loads?
>    Not that I'd find those benchmarks hard to swollow per se, I did
>    see Reiser4 perform superior for myself.
>    But what is the secret? Atomic transactions?

I think it is a feature of delayed block allocation. 

> 
> Thanks again for the good work and I hope this brings along some discussion
> more interesting than the metas/ thread!

> 
> -- 
> mjt
> 

-- 
Alex.

Re: On plugins, on-disk formatting and such

[mjt]

On Mon, Apr 12, 2004 at 05:30:06PM +0400, Alex Zarochentsev wrote:

>2 APIs are: pseudo files and reiser4 syscall.

And syscalls will get a new set of programs to access them?

>reiser4 is able to store plugin sets (list of plugin ids for the object) as 
>a part of the object's stat data.

Is this doable only through syscalls?
If I have ext-3 fibration on /home/mjt/movies/ and reboot, what will I
have to do so the fibration doesn't change to dot-o after the reboot?
Apart from having another default in the mount options but that's not
what I'm after ;)

I actually haven't tested this, so it might even be standard behaviour ;)

>we still see fs errors if COC enable. they are rare now, but COC is not stable
>yet.

What sort of errors, please?
Real breakage, file inaccessibility or failed writes or..?

>>    But what is the secret? Atomic transactions?
>I think it is a feature of delayed block allocation. 

I tried to check namesys.com but didn't find anything.
Is this something that could be explained a bit more clearly?

Thanks for the answers and happy cosmonautics day!

BTW, I did an apt-get source glibc on my Reiser4 root partition.
I'm waiting for dpkg-buildpackage to either complete or fail, so maybe
we'll see what's amatter here.

-- 
mjt

Re: On plugins, on-disk formatting and such

[Hans Reiser]

Alex Zarochentsev wrote:

>
>>   
>>   4.2) What exactly does it mean that CONFIG_REISER4_COPY_ON_CAPTURE
>>        is stabler than before? Is it soon safe to configure in?
>>        Is it possible for me to test it on only one filesystem or
>>        do I risk rootfs (and home directory) stability by trying it
>>        out? If I compile it in, what's the best way of testing it?
>>    
>>
>
>we still see fs errors if COC enable. they are rare now, but COC is not stable
>yet.
>
>  
>
Is Vladimir working on it today?

-- 
Hans

Re: On plugins, on-disk formatting and such

[Hans Reiser]

Markus Törnqvist wrote:

>First off I'd like to thank the Namesys guys for their excellent work,
>the latest snapshot has been very stable for me, so stable in fact
>that my life is boring because of it ;)
>  
>
oh excellent, I think we just have to fix a problem with crashing when 
mmap consumes all of memory, and we are ready to ship

>I have like a ton of questions so I'll just put them in this mail and
>see which points get replied to. Hopefully all ;)
>
>1) The xattrs issue never got any replies, although I gathered that it
>   would be possible to map xattr calls to the Reiser4 API.
>   Is this fact or fiction? What will the Reiser4 API look like, apart
>   from control files?
>  
>
I don't understand your question.  We don't use xattrs and I think they 
should never have been put into the kernel.  We will not burden our code 
with support for them.

Instead, you can access metas to do everything that xattrs do, or at 
least that is our development direction.  We will never support xattrs.

>2) Is metadata stored anywhere? If I want to organize all my libs in one
>   fibre, I make it so, I reboot and do I then have to reset the fibration
>   for the directory? Is it possible to have persistency here? Wouldn't
>   a wrong fibration type make the files inaccessible?
>   Is this addressed in the Reiser4 API somehow?
>  
>
I will let nikita answer this, hopefully by adding to our web pages and 
sending you the URL.....;-)

>3) How big are the chances that the on-disk layout will still change?
>  
>
Because plugins make it easy and painless for users, you can expect it 
to change all the time.  If you mean change without grace, we try to 
avoid that.  Unfortunately curing a recent discovery that we cannot do 
it gracefully (in regards to inheritance) required a non-graceful change 
to cure it, sorry about that.  I still need to review that code to make 
sure all issues regarding inheritance and defaults are fully graceful.

>   On IRC I've got the impression that the chance is next to none,
>   and I thank you for that!
>   Still, a lot of people are hesitant to try Reiser4 because of this
>   fear. Doing an mkfs is not a small thing and some assurance would
>   surely bring along community testing support.
>
>4) http://thebsh.namesys.com/snapshots/2004.03.26/READ.ME
>   4.1) I have used Reiser4 as my rootfs for a long, long time.
>  
>
Thanks, Vitaly fix that.

>        Why does the READ.ME say I can't do it?-) I just never mount
>        it read-only, so I don't have to remount it read-write and it
>        just works. Is there something amiss here?
>   
>   4.2) What exactly does it mean that CONFIG_REISER4_COPY_ON_CAPTURE
>        is stabler than before? Is it soon safe to configure in?
>        Is it possible for me to test it on only one filesystem or
>        do I risk rootfs (and home directory) stability by trying it
>        out? If I compile it in, what's the best way of testing it?
>  
>
vs should answer this one

>5) If I wanted to write an fs plugin, I gathered that I would not have
>   to mkfs again. Should this not require some persistency in metadata?
>   I also gathered that Hans Reiser is against loadable kernel modules
>   that are in essence filesystem plugins. Is this because a stray
>   rmmod might cause breakage or something else?
>  
>
It is because I want either code or money from those who  build on top 
of reiser4.  Allowing people to escape the GPL for free is not to my 
advantage.  Also, I don't want to find out whether it affects performance.

>   One of the good things for me in Linux is that I can use honest
>   modules to try something, rmmod, recompile, insmod and I don't
>   think this is any different.
>  
>
Hmmm, when your NVIDIA drivers lag a few releases and you don't have the 
source, fun, yes?

>6) I understood there are some NFS issues still left,
>
I think those got cured, but haven't made it into the latest snapshot.
 Elena, add these words somewhere:

This performance advantage is due to allocate on flush allowing us to 
write in huge extents at flush time even though files get written to in 
slow trickles.  This improves b

>   Not that I'd find those benchmarks hard to swollow per se, I did
>   see Reiser4 perform superior for myself.
>   But what is the secret? Atomic transactions?
>
>Thanks again for the good work and I hope this brings along some discussion
>more interesting than the metas/ thread!
>
>  
>


-- 
Hans

Re: On plugins, on-disk formatting and such

[Vladimir Saveliev]

On Mon, 2004-04-12 at 19:13, Hans Reiser wrote:
> Alex Zarochentsev wrote:
> 
> >
> >>   
> >>   4.2) What exactly does it mean that CONFIG_REISER4_COPY_ON_CAPTURE
> >>        is stabler than before? Is it soon safe to configure in?
> >>        Is it possible for me to test it on only one filesystem or
> >>        do I risk rootfs (and home directory) stability by trying it
> >>        out? If I compile it in, what's the best way of testing it?
> >>    
> >>
> >
> >we still see fs errors if COC enable. they are rare now, but COC is not stable
> >yet.
> >
> >  
> >
> Is Vladimir working on it today?

No.
I am working on reiser4 not being able to compile glibc

Re: On plugins, on-disk formatting and such

[mjt]

On Mon, Apr 12, 2004 at 08:15:04AM -0700, Hans Reiser wrote:

>oh excellent, I think we just have to fix a problem with crashing when 
>mmap consumes all of memory, and we are ready to ship

This is not something that other kernel people should look into?

>I don't understand your question.  We don't use xattrs and I think they 
>should never have been put into the kernel.  We will not burden our code 
>with support for them.

This I understood, and I may be a bit lost myself here.
I meant that if a program does a xattr call on Reiser4 it gets
transparently mapped to the Reiser4 api. I dunno, and really, I
don't even care that much. As long as it works and gets into the mainline
kernel.

[on.disk.layout.]
>Because plugins make it easy and painless for users, you can expect it 
>to change all the time.  If you mean change without grace, we try to 
>avoid that.  Unfortunately curing a recent discovery that we cannot do 
>it gracefully (in regards to inheritance) required a non-graceful change 
>to cure it, sorry about that.  I still need to review that code to make 
>sure all issues regarding inheritance and defaults are fully graceful.

What I was basically after was the real bottom-line end-user perspective.

Every chance of having to do an mkfs is a turnoff. Naturally plugins
may change the layout, but that's the point of grace :)

>>4) http://thebsh.namesys.com/snapshots/2004.03.26/READ.ME
>>  4.1) I have used Reiser4 as my rootfs for a long, long time.
>Thanks, Vitaly fix that.

Is the problem supposed to be about remounting?
I tried -o remount,ro and ,rw on my home partition, it worked, but
after doing a SysRq remount mount still said it was rw.

This freakishness may be separate from being able to use it as
the root fs.

[loadable.fs.plugins]
>It is because I want either code or money from those who  build on top 
>of reiser4.  Allowing people to escape the GPL for free is not to my 
>advantage.  Also, I don't want to find out whether it affects performance.

This is certainly a valid point.
Isn't it still possible that someone writes an fs plugin and never releases
it, though?

Maybe it's possible that it would affect performance... Although I have
no data or experience from benchmarks with average kernel modules.

>>  One of the good things for me in Linux is that I can use honest
>>  modules to try something, rmmod, recompile, insmod and I don't
>>  think this is any different.
>Hmmm, when your NVIDIA drivers lag a few releases and you don't have the 
>source, fun, yes?

I'm afraid I don't follow :)
The fear here is that I download a third-party plugin, use it, update
my kernel and get breakage because the third-party hasn't updated yet?

This is always a risk when using third-party modules, be they GPL or not.
But proprietary, closed-source, (thus infringing?), Reiser4 plugins are
imo a lot less likely.

>This performance advantage is due to allocate on flush allowing us to 
>write in huge extents at flush time even though files get written to in 
>slow trickles.  This improves b

NO CARRIER+++

Thanks a million for the answers!

-- 
mjt

Re: On plugins, on-disk formatting and such

[Hubert Chan]

>>>>> "Markus" == Markus Törnqvist <mjt@nysv.org> writes:

[...]

Markus> 2) Is metadata stored anywhere? If I want to organize all my
Markus> libs in one fibre, I make it so, I reboot and do I then have to
Markus> reset the fibration for the directory? Is it possible to have
Markus> persistency here? Wouldn't a wrong fibration type make the files
Markus> inaccessible?  Is this addressed in the Reiser4 API somehow?

I have a similar question (or possibly the same question, except from a
developer's point of view, rather than a user's):

Are plugins able to request storage space other than what is available
for the file data itself?  I would assume this would be the natural way
to do indexing: store the file contents in the regular fashion, and
keep an index on the side.

And if the answer to that question is yes, then the next question is:
are plugins able to request multiple storage streams (e.g., if it needs
to store multiple indexes)?

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: Reiser4 can't compile glibc

[Szakacsits Szabolcs]

What timestamp resolution Reiser4 has? Ext[23] can't compile gcc because
[kernel bugs] and it doesn't have on-disk high resolution timestamps
thus the dependency during compilation breaks.

But somehow I doubt this would be the case, all modernish filesystem
designed in the last 10 years has at least nanosec resolution.

	Szaka

Re: On plugins, on-disk formatting and such

[Hans Reiser]

Markus Törnqvist wrote:

>
>  
>
>>This performance advantage is due to allocate on flush allowing us to 
>>write in huge extents at flush time even though files get written to in 
>>slow trickles.  This improves 
>>    
>>
read performance dramatically.

>
>NO CARRIER+++
>
>Thanks a million for the answers!
>
>  
>

grev, please update the benchmark to say the above paragraph.

-- 
Hans

Re: Reiser4 can't compile glibc

[Vladimir Saveliev]

[-- Attachment #1: Type: text/plain, Size: 612 bytes --]

Hello

On Mon, 2004-04-12 at 20:53, Szakacsits Szabolcs wrote:
> What timestamp resolution Reiser4 has? Ext[23] can't compile gcc because
> [kernel bugs] and it doesn't have on-disk high resolution timestamps
> thus the dependency during compilation breaks.
> 
> But somehow I doubt this would be the case, all modernish filesystem
> designed in the last 10 years has at least nanosec resolution.
> 
yes, reiser4 by default has the same time resolution as ext[23] and
reiserfs.
However, I think that i found the problem already. The attached patch
for fs/reiser4/plugin/item/tail.c should help.


> 	Szaka
> 
> 

[-- Attachment #2: tail.c.diff --]
[-- Type: text/plain, Size: 914 bytes --]

--- tail.c~	2004-04-13 18:18:52.000000000 +0400
+++ tail.c	2004-04-13 18:20:14.000000000 +0400
@@ -463,12 +463,18 @@
 	}
 
 	inode = mapping->host;
-	if (get_key_offset(&f->key) > inode->i_size)
+	if (get_key_offset(&f->key) > inode->i_size) {
+		assert("vs-1649", f->user == 1);
 		INODE_SET_FIELD(inode, i_size, get_key_offset(&f->key));
-	inode->i_ctime = inode->i_mtime = CURRENT_TIME;
-	result = reiser4_update_sd(inode);
-	if (result)
-		return result;
+	}
+	if (f->user != 0) {
+		/* this was writing data from user space. Update timestamps, therefore. Othrewise, this is tail
+		   conversion where we should not update timestamps */
+		inode->i_ctime = inode->i_mtime = CURRENT_TIME;
+		result = reiser4_update_sd(inode);
+		if (result)
+			return result;
+	}
 
 	/* FIXME-VS: this is temporary: the problem is that bdp takes inodes
 	   from sb's dirty list and it looks like nobody puts there inodes of

Re: Reiser4 can't compile glibc

[Szakacsits Szabolcs]

On Tue, 13 Apr 2004, Vladimir Saveliev wrote:
>
> yes, reiser4 by default has the same time resolution as ext[23] and
> reiserfs.

Hmmm, so the kernel's nanosec timestamps can bite Reiser4 as well?

  http://groups.google.com/groups?selm=1GgcQ-4k1-17%40gated-at.bofh.it

> However, I think that i found the problem already. The attached patch
> for fs/reiser4/plugin/item/tail.c should help.

That's great :)
	
	Szaka

Re: Reiser4 can't compile glibc

[Hans Reiser]

Szakacsits Szabolcs wrote:

>On Tue, 13 Apr 2004, Vladimir Saveliev wrote:
>  
>
>>yes, reiser4 by default has the same time resolution as ext[23] and
>>reiserfs.
>>    
>>
>
>Hmmm, so the kernel's nanosec timestamps can bite Reiser4 as well?
>
>  http://groups.google.com/groups?selm=1GgcQ-4k1-17%40gated-at.bofh.it
>
>  
>
>>However, I think that i found the problem already. The attached patch
>>for fs/reiser4/plugin/item/tail.c should help.
>>    
>>
>
>That's great :)
>	
>	Szaka
>
>
>
>  
>
Interesting.  vs, describe the storage of these in the various reiserfs 
versions.

-- 
Hans

Re: Reiser4 can't compile glibc

[Grant Miner]

Thanks. I put on that patch and it fixes my make problem.

Vladimir Saveliev wrote:

>Hello
>
>On Mon, 2004-04-12 at 20:53, Szakacsits Szabolcs wrote:
>  
>
>>What timestamp resolution Reiser4 has? Ext[23] can't compile gcc because
>>[kernel bugs] and it doesn't have on-disk high resolution timestamps
>>thus the dependency during compilation breaks.
>>
>>But somehow I doubt this would be the case, all modernish filesystem
>>designed in the last 10 years has at least nanosec resolution.
>>
>>    
>>
>yes, reiser4 by default has the same time resolution as ext[23] and
>reiserfs.
>However, I think that i found the problem already. The attached patch
>for fs/reiser4/plugin/item/tail.c should help.
>
>
>  
>
>>	Szaka
>>
>>
>>    
>>
>>------------------------------------------------------------------------
>>
>>--- tail.c~	2004-04-13 18:18:52.000000000 +0400
>>+++ tail.c	2004-04-13 18:20:14.000000000 +0400
>>@@ -463,12 +463,18 @@
>> 	}
>> 
>> 	inode = mapping->host;
>>-	if (get_key_offset(&f->key) > inode->i_size)
>>+	if (get_key_offset(&f->key) > inode->i_size) {
>>+		assert("vs-1649", f->user == 1);
>> 		INODE_SET_FIELD(inode, i_size, get_key_offset(&f->key));
>>-	inode->i_ctime = inode->i_mtime = CURRENT_TIME;
>>-	result = reiser4_update_sd(inode);
>>-	if (result)
>>-		return result;
>>+	}
>>+	if (f->user != 0) {
>>+		/* this was writing data from user space. Update timestamps, therefore. Othrewise, this is tail
>>+		   conversion where we should not update timestamps */
>>+		inode->i_ctime = inode->i_mtime = CURRENT_TIME;
>>+		result = reiser4_update_sd(inode);
>>+		if (result)
>>+			return result;
>>+	}
>> 
>> 	/* FIXME-VS: this is temporary: the problem is that bdp takes inodes
>> 	   from sb's dirty list and it looks like nobody puts there inodes of
>>    
>>

Re: Reiser4 can't compile glibc

[mjt]

Grant Miner wrote:

>Thanks. I put on that patch and it fixes my make problem.

I could compile glibc as well.

However, GCC did not apparently compile. Debian Sid's GCC 3.3
was the one I tried.

It appeared to have stalled during some if its test cases, I'll
post further details, if required, when I get home and if I have
time to sit at the computer then.

Thanks you Vladimir!

-- 
mjt