Reiser4 bitmap & replication handling

Reiser4 bitmap & replication handling

[Fred -- Speed Up --]

[-- Attachment #1: Type: text/plain, Size: 1374 bytes --]

I just read this article about XFS :
http://www-106.ibm.com/developerworks/library/l-fs9.html

It says XFS uses B+Trees as Reiser4 does, and it uses this structure to store not only the keys, but also the bitmap data and replicates it storage trees to improve scalability and overall wide-disk performance. It also tries to allocate the largest contigous blocks to a file (but Reiser4 does this also), and comes close to raw performance when reading large files. The article concludes XFS is faster than Reiser (3.6 I think, as the article has been published in 2002 by Gentoo CEO Daniel Robbins) with performance tweaks at mount time. But how does Reiser4 ?

How many times does Reiser replicate and/or truncate its tree, and what structure has been chosen for bitmap data ?
Does Reiser4 divide the disk space in chunks, as XFS and ext2 do ?
Are you implementing some special features for very large disks, RAID systems and multi-processing hardware ?
Is the developement far enough to get an idea of the final performance ? I didn't remember, when do you plan to release Reiser4 ?

I'm very interested in those brand new filesystem techniques, for now I think we can consider that XFS and Reiser3 are very close in overall performance. You're saying in the doc you'd like to keep on offering the best performance in the market : are you really that far ?

Fred

Re: Reiser4 bitmap & replication handling

[Hans Reiser]

Fred -- Speed Up -- wrote:

>I just read this article about XFS :
>http://www-106.ibm.com/developerworks/library/l-fs9.html
>
>It says XFS uses B+Trees as Reiser4 does, and it uses this structure to store not only the keys, but also the bitmap data and replicates it storage trees to improve scalability and overall wide-disk performance. It also tries to allocate the largest contigous blocks to a file (but Reiser4 does this also), and comes close to raw performance when reading large files. The article concludes XFS is faster than Reiser 
>
well, his words are not quite that strong, and who knows if he is using 
the latest journaling patches but XFS is a very nice filesystem.....

>(3.6 I think, as the article has been published in 2002 by Gentoo CEO Daniel Robbins) with performance tweaks at mount time.
>
Which he does not define, so we cannot reproduce it.

> But how does Reiser4 ?
>  
>
Well, Reiser4 has much fewer seeks than other filesystems usually.  The 
current weakness of Reiser4 is CPU consumption, but our team is cutting 
that down at great speed.  It seems like each team member manages to 
reduce it by >10% a week, and there are 3 of them working actively on 
that.  This is fortunate, because it is still very high.....  I have 
this fond hope that performance will be something very nice by 
LinuxTag.....;-)

>How many times does Reiser replicate 
>
what do you mean, replicate the tree?

>and/or truncate its tree, and what structure has been chosen for bitmap data ?
>
bitmaps.;-) we have some nice little tweaks, like we count the number of 
used bits in every bitmap block so that we don't have to scan full 
blocks, and we hash around if we scan too much, but I have to say that 
bitmaps are nice and simple and work without a lot of code.

>Does Reiser4 divide the disk space in chunks, as XFS and ext2 do ?
>
No.  We experimented a little bit with that, and didn't get clear 
advantages.  My mind is still open though if someone shows me something 
compelling.

>Are you implementing some special features for very large disks,
>
how are they different

> RAID systems
>
Reiser4 will happen to work very well for them without understanding 
them (not entirely a coincidence).  I have as yet to get someone to fund 
tuning for them specifically, though it would probably yield benefits to 
do so.

> and multi-processing hardware 
>

Reiser4 made a massive code investment into fine-grained locking. 

>Is the developement far enough to get an idea of the final performance ? 
>
Reiser4 inconsistently performs very very fast.  We have no doubt that 
we can make that consistent but it is a lot of measuring and tweaking 
and measuring and tweaking.....

>I didn't remember, when do you plan to release Reiser4 ?
>
LinuxTag, July 11, if lucky, we will start our beta.

>
>I'm very interested in those brand new filesystem techniques, for now I think we can consider that XFS and Reiser3 are very close in overall performance. You're saying in the doc you'd like to keep on offering the best performance in the market : are you really that far ?
>
>Fred
>  
>


-- 
Hans

Re: Reiser4 bitmap & replication handling

[Fred -- Speed Up --]

Here are Hans's interesting answers to my previous post, thank you Hans ;)
Just a little question : when do you plan to update the kernel patch, which
is 3 month old ?

Fred

> >Maybe some really big files handling performance improvements, to near
XFS
> >performance in this domain. I'm not really concerned, but some specific
> >systems may need such features only offered by XFS. Do you plan to
release
> >different versions of the filesystem's plugins to adapt to different
systems
> >ans situations ?
>
> Large file performance is not really considered the hard thing in FS
> design (unless you do data journaling or atomicity, in which case we are
> twice as fast).

> >>Reiser4 inconsistently performs very very fast.  We have no doubt that
> >>we can make that consistent but it is a lot of measuring and tweaking
> >>and measuring and tweaking.....
> >Sorry but ... what do you mean by "unconsistently" ? Does this mean you
only
> >consider avarage performance, ignoring worse cases ?
>
> I mean we have lots of crud left to clean out of the code.  If you test
> it in a way that avoids the crud, we are fast, if you hit the crud we
> are slow.  Decrudding is in progress.;-)

> >Another question, do you want to release Reiser4 so that it will be
> >presented on LinuxTag because :
> >1- The developement team is mainly composed of german people
> >2- Suse will have a very big area on the show
> >3- The release date just accorded to the LinuxTag opening
> >4- All of you especially like LinuxTag :-D
>
> I like LinuxTag, and I am speaking there, and if we can release it then
> without rushing the code, it will be nice.  No promises, the code ships
> when the code is ready and not before....

Re: Reiser4 bitmap & replication handling

[Oleg Drokin]

Hello!

On Thu, May 22, 2003 at 07:17:45PM +0200, Fred -- Speed Up -- wrote:
> Here are Hans's interesting answers to my previous post, thank you Hans ;)
> Just a little question : when do you plan to update the kernel patch, which
> is 3 month old ?

I hope to do it soon enough. Perhaps even before the end of (this) May ;)

Bye,
    Oleg

Re: Reiser4 bitmap & replication handling

[Fred -- Speed Up --]

[-- Attachment #1: Type: text/plain, Size: 3390 bytes --]

> well, his words are not quite that strong, and who knows if he is using
> the latest journaling patches but XFS is a very nice filesystem.....
You're speaking from Reiser 3.6 right ? Are these patches included in 2.4.20
or 2.4.21 official kernel sources ?

> Well, Reiser4 has much fewer seeks than other filesystems usually.  The
> current weakness of Reiser4 is CPU consumption, but our team is cutting
> that down at great speed.  It seems like each team member manages to
> reduce it by >10% a week, and there are 3 of them working actively on
> that.  This is fortunate, because it is still very high.....  I have
> this fond hope that performance will be something very nice by
> LinuxTag.....;-)
I really hope so, performance is a big point for the Reiser filesystem ;)
You say the CPU is now the weakness ... but how do you handle with large
file read/write operations : in this case, the HDD performance level should
come first ?
Great part of the Reiser4 team is working on performance tweaking : does
this mean that Reiser has come into the final stage, that it has now proven
stable so that performance improvement is the only thing to achieve before
RC and/or release ? Did you published a release plan, with public beta and
RC or do you want to test Reiser4 internally ? I know Reiser4 is available
as a 2.5 kernel patch, but do you plan to build a dedicated structure to
report bugs found in early Reiser4 versions ?

> what do you mean, replicate the tree?
I mean, for security pruposes, does Reiser4 maintain copies of the storage
tree or does it simply trust its marvelous journalising system ?

> bitmaps.;-) we have some nice little tweaks, like we count the number of
> used bits in every bitmap block so that we don't have to scan full
> blocks, and we hash around if we scan too much, but I have to say that
> bitmaps are nice and simple and work without a lot of code.
I wad afraid Reiser4 didn't had the best bitmap-handling system over there,
now I'm reassured thank you :-D

> >Are you implementing some special features for very large disks,
> how are they different
Maybe some really big files handling performance improvements, to near XFS
performance in this domain. I'm not really concerned, but some specific
systems may need such features only offered by XFS. Do you plan to release
different versions of the filesystem's plugins to adapt to different systems
ans situations ?

> Reiser4 inconsistently performs very very fast.  We have no doubt that
> we can make that consistent but it is a lot of measuring and tweaking
> and measuring and tweaking.....
Sorry but ... what do you mean by "unconsistently" ? Does this mean you only
consider avarage performance, ignoring worse cases ?
Again, do you plan to ask some external testers to tweak Reiser4 ?

> >I didn't remember, when do you plan to release Reiser4 ?
> >
> LinuxTag, July 11, if lucky, we will start our beta.
Outstanding ... I hope you'll be sucessfull, but I'm not really scary about
that ;)
Another question, do you want to release Reiser4 so that it will be
presented on LinuxTag because :
1- The developement team is mainly composed of german people
2- Suse will have a very big area on the show
3- The release date just accorded to the LinuxTag opening
4- All of you especially like LinuxTag :-D


Thank you Hans,

Fred