Re: Reiser4. BEST FILESYSTEM EVER.

Re: Reiser4. BEST FILESYSTEM EVER.

[johnrobertbanks]

Hi Peter,

You say that the results may be accurate, but "Whether or not they're
*relevant* is a totally different ball of wax." and

"Whether or not they're relevant depends on how well they happen to
reflect your particular usage pattern."

Well, surprise, surprise,.. everyone knows that.

Have a look at the (summary) of the results: 

.-------------------------.
| FILESYSTEM | TIME |DISK |
| TYPE       |(secs)|USAGE|
.-------------------------.
|REISER4 lzo | 1938 | 278 |
|REISER4 gzip| 2295 | 213 |
|REISER4     | 3462 | 692 |
|EXT2        | 4092 | 816 |
|JFS         | 4225 | 806 |
|EXT4        | 4408 | 816 |
|EXT3        | 4421 | 816 |
|XFS         | 4625 | 779 |
|REISER3     | 6178 | 793 |
|FAT32       |12342 | 988 |
|NTFS-3g     |10414 | 772 |
.-------------------------.


for the full results see:
http://linuxhelp.150m.com/resources/fs-benchmarks.htm 

Don't you agree, that "If they are accurate,.... THEN they are obviously
very relevant."

I have set up a Reiser4 partition with gzip compression, here is the
difference in disk usage of a typical Debian installation on two 10GB
partitions, one with Reiser3 and the other with Reiser4.

debian:/# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda3             10490104   6379164   4110940  61% /3
/dev/sda7              9967960   2632488   7335472  27% /7

Partitions 3 and 7 have exactly the same data on them (the typical
Debian install).

The partitions are exactly the same size (although df records different
sizes).

Partition 3 is Reiser3 -- uses 6.4 GB.
Partition 7 is Reiser4 -- uses 2.6 GB.

So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3
6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same
info).

Don't you think this result is significant in itself?

Following your hint I have booted /dev/sda7 and all the programs seem to
work fine. They do not seem to be any faster than when using Reiser3.

The whole system seems about as responsive as always.

For fun, I ran bonnie++. Here are the results:

debian:/# ./bonnie++ -u root
Using uid:0, gid:0.
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.93c       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 
/sec %CP
debian           1G   121  99 86524  21 63297  41   920  99 187762  80 
1782 233
Latency             82484us     386ms     438ms   26758us     110ms    
398ms
Version 1.93c       ------Sequential Create------ --------Random
Create--------
debian              -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP 
              /sec %CP
                 16 +++++ +++ +++++ +++ 18509  92 17776  86 +++++ +++
                 19495  91
Latency               210us    5475us    5525us    5777us    5522us   
5839us

I particularly liked the 233%CP for Random-Seeks.

John.



On Thu, 05 Apr 2007 21:07:28 -0700, "H. Peter Anvin" <hpa@zytor.com>
said:
> johnrobertbanks@fastmail.fm wrote:
> > Hi Peter,
> > 
> > You say that the results may be accurate, but not relevant.
> > 
> 
> NO, I said that whether they're accurate is another matter.
> 
> > If they are accurate,.... THEN they are obviously very relevant.
> 
> Crap-o-la.  Whether or not they're relevant depends on how well they 
> happen to reflect your particular usage pattern.
> 
> There are NO benchmarks which are relevant to all users.  Understanding 
> whether or not a benchmark is relevant to one's particular application 
> is one of the trickiest things about benchmarks.
> 
> 	-hpa
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - Email service worth paying for. Try it for free

Re: Reiser4. BEST FILESYSTEM EVER.

[johnrobertbanks]

On Fri, 6 Apr 2007 11:21:19 -0400, "Jan Harkes" <jaharkes@cs.cmu.edu>
said:
> Do you really have to repeat the results in every email you sent?

The following benchmarks are from

http://linuxhelp.150m.com/resources/fs-benchmarks.htm or,
http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm

.-------------------------.
| FILESYSTEM | TIME |DISK |
| TYPE       |(secs)|USAGE|
.-------------------------.
|REISER4 lzo | 1938 | 278 |
|REISER4 gzip| 2295 | 213 |
|REISER4     | 3462 | 692 |
|EXT2        | 4092 | 816 |
|JFS         | 4225 | 806 |
|EXT4        | 4408 | 816 |
|EXT3        | 4421 | 816 |
|XFS         | 4625 | 779 |
|REISER3     | 6178 | 793 |
|FAT32       |12342 | 988 |
|NTFS-3g     |10414 | 772 |
.-------------------------.


Column one measures the time taken to complete the bonnie++ benchmarking
test (run with the parameters bonnie++ -n128:128k:0)

Column two, Disk Usage: measures the amount of disk used to store 655MB
of raw data (which was 3 different copies of the Linux kernel sources).

> Do you really have to repeat the results in every email you sent?

Damn, I did it again. WHY DO YOU CARE?

Look, its simple, I am (among other things) discussing these results, so
people need to see them.

> > Don't you agree, that "If they are accurate,.... THEN they are obviously
> > very relevant."
> 
> Not everyone does. I care mostly about reliability and availability
> neither of which are shown by your results.

Actually, to some extent, bonnie++ tests the reliability of the
filesystem, eg, NTFS-3g usually fails.

By the way, I have pulled the plug on my REISER4 system, a number of
times now, and it recovers without problem.

> With compression there is a pretty high probability that one corrupted
> byte or disk block will result in loss of a considerably larger amount
> of data. 

Bad blocks are NOT dealt with by the filesystem,... so your comment is
irrelevant, or just plain wrong.

If your filesystem is writing to bad blocks, then throw away your
operating system.

> > I have set up a Reiser4 partition with gzip compression, here is the
> > difference in disk usage of a typical Debian installation on two 10GB
> > partitions, one with Reiser3 and the other with Reiser4.
> > 
> > debian:/# df
> > Filesystem           1K-blocks      Used Available Use% Mounted on
> > /dev/sda3             10490104   6379164   4110940  61% /3
> > /dev/sda7              9967960   2632488   7335472  27% /7
> ...
> > Partition 3 is Reiser3 -- uses 6.4 GB.
> > Partition 7 is Reiser4 -- uses 2.6 GB.
> > 
> > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3
> > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same
> > info).
> 
> Wow, consider me totally and completely, unimpressed.
> 

Wow, consider me totally impressed by your AMAZING BIAS.

Would you like to tell me why you are SO BIASED against REISER4.

John.
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - A fast, anti-spam email service.

Re: Reiser4. BEST FILESYSTEM EVER.

[Jan Harkes]

Since you decide to publically respond to a private email, but not only
you did not 'discuss' anything I wrote and in fact cut out most of the
useful information in my reply I guess I will have to repeat my
observations.

Once I send out this email, I'll just add you to my friendly killfile
(as well as this thread's subject/msgids) so don't bother continuing you
one-sided 'discussions' on this topic.

On Fri, Apr 06, 2007 at 07:47:36PM -0700, johnrobertbanks@fastmail.fm wrote:
> > Do you really have to repeat the results in every email you sent?
> 
> Damn, I did it again. WHY DO YOU CARE?

Because I saw them the first time around. And although the performance
difference on those micro benchmarks seems quite impressive, I'm not
convinced.

> Look, its simple, I am (among other things) discussing these results, so
> people need to see them.

However, you do not discuss, you just repeat, and repeat, and repeat.
But for what reason. Do you want an actual discussion, or do you hate
the reiserfs developers so much that you want to antagonize any and all
other Linux file systems developers?

> > > Don't you agree, that "If they are accurate,.... THEN they are obviously
> > > very relevant."
> > 
> > Not everyone does. I care mostly about reliability and availability
> > neither of which are shown by your results.
> 
> Actually, to some extent, bonnie++ tests the reliability of the
> filesystem, eg, NTFS-3g usually fails.
> 
> By the way, I have pulled the plug on my REISER4 system, a number of
> times now, and it recovers without problem.

Very nice for you. I guess you have also not been hit by out of memory
conditions or failing partial writes. So I guess we can ignore the patch
that was just sent a day or two ago to the mailing list because you
succesfully pulled the plug, a number of times at that.

> > > I have set up a Reiser4 partition with gzip compression, here is the
> > > difference in disk usage of a typical Debian installation on two 10GB
> > > partitions, one with Reiser3 and the other with Reiser4.
> > > 
> > > debian:/# df
> > > Filesystem           1K-blocks      Used Available Use% Mounted on
> > > /dev/sda3             10490104   6379164   4110940  61% /3
> > > /dev/sda7              9967960   2632488   7335472  27% /7
> > ...
> > > Partition 3 is Reiser3 -- uses 6.4 GB.
> > > Partition 7 is Reiser4 -- uses 2.6 GB.
> > > 
> > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3
> > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same
> > > info).
> > 
> > Wow, consider me totally and completely, unimpressed.
> > 

Here is the part of my email that you seemed to totally ignore,

    You've just saved yourself $3.80, now go get yourself a latte.
    (see. http://tomayko.com/weblog/2006/09/11/that-dilbert-cartoon)

    Seriously, disk storage is getting less expensive all the time, you can
    already buy a 250GB SATA drive for $70. Also, compression doesn't help
    once you store already compressed data such as jpeg images, mp3 files,
    or mpeg2/4/divx video files. Not only are the savings non-existant, but
    we still end up with the disadvantage that corruption propagates to
    multiple files because of the compression in the file system.

    And if it doesn't propagate across multiple files, the compression can't
    be all that good since it can't benefit as much from similarity between
    files. So if that is the case and you really want to save diskspace you
    almost have to look at read-only compressed filesystems such as cramfs,
    squashfs, zisofs, cloop and various other variants in combination with
    a unionfs overlay to get read/write functionality.

    But in the end everything is a tradeoff. You can save diskspace, but
    increase the cost of corruption. Use a better compression algorithm, but
    that uses more CPU or which is visible in performance of the
    application. This can be offset by caching more, and being lazier with
    writebacks, but that hurts on-disk consistency, creates more memory
    pressure (more swapout/paging) and generally slows down other
    applications that aren't actually accessing the disk. Having a fast
    multi-core cpu and lots of memory helps a lot, but at some point what
    tradeoff did we just make, we saved a couple of dollars not having to
    buy a larger disk, but we spend considerably more on the more expensive
    cpu and memory.

> Wow, consider me totally impressed by your AMAZING BIAS.
> 
> Would you like to tell me why you are SO BIASED against REISER4.

And that is the reponse I get, I thought you wanted discussion, but
clearly you don't care about any meaningful discussion. Your goal seems
to be to make sure that other developers end up ignoring any thread that
has reiser in the subject. And even if they are not biased and welcome a
discussion, you will just call them out on it, because clearly if they
want to discuss something they aren't totally with it, so they have to
be totally BIASED against REISER4.

At least it looks like we agree on something, I think.

Jan

Re: Reiser4. BEST FILESYSTEM EVER.

[johnrobertbanks]

On Fri, 6 Apr 2007 23:30:49 -0400, "Jan Harkes" <jaharkes@cs.cmu.edu>
said:
> 
> Since you decide to publically respond to a private email, but not only
> you did not 'discuss' anything I wrote and in fact cut out most of the
> useful information in my reply I guess I will have to repeat my
> observations.

You are a funny guy Jan.

Here you are, once again, cutting out my most useful information, ie,
the data I was discussing, while complaining that I cut out your most
useful information. 

You know,... you cut out this bit:

-----------------------------------------------------------------

> The following benchmarks are from
> 
> http://linuxhelp.150m.com/resources/fs-benchmarks.htm or,
> http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm
> 
> .-------------------------.
> | FILESYSTEM | TIME |DISK |
> | TYPE       |(secs)|USAGE|
> .-------------------------.
> |REISER4 lzo | 1938 | 278 |
> |REISER4 gzip| 2295 | 213 |
> |REISER4     | 3462 | 692 |
> |EXT2        | 4092 | 816 |
> |JFS         | 4225 | 806 |
> |EXT4        | 4408 | 816 |
> |EXT3        | 4421 | 816 |
> |XFS         | 4625 | 779 |
> |REISER3     | 6178 | 793 |
> |FAT32       |12342 | 988 |
> |NTFS-3g     |10414 | 772 |
> .-------------------------.
> 
> Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0)
> 
> Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources).

-----------------------------------------------------------------

And this bit: 

Jan,... Here is another section, you conveniently cut out. Maybe you
should explain why you cut out this section? Was it embarrassing to you?
I mean, your statement is sort of correct,... but it shows a basic
misunderstanding of filesystems.

> > With compression there is a pretty high probability that one corrupted
> > byte or disk block will result in loss of a considerably larger amount
> > of data. 
> 
> Bad blocks are NOT dealt with by the filesystem,... so your comment is
> irrelevant, or just plain wrong.
> 
> If your filesystem is writing to bad blocks, then throw away your
> operating system.

-----------------------------------------------------------------

It is true that I considered your "most useful information," an
irrelevant section, which is why it was cut out (ignored).

I did not see my doing this, any worse than you doing it. I did not
realize that you were be so impolite.

As to your email being private, I had thought I had joined a mailing
list. I had not idea your email was meant to be private and just
considered it like all the others.

Now you mention it, I wondered why the email did not automatically list
the mailing lists, as recipients, and I had to add them. If I had
realized this I may have added the mailing lists as recipients, anyway.
It would be like me to do such. However, you should understand that I am
new to mailing lists.
 
> 
> > > Do you really have to repeat the results in every email you sent?
> > 
> > Damn, I did it again. WHY DO YOU CARE?
> 
> Because I saw them the first time around. And although the performance
> difference on those micro benchmarks seems quite impressive, I'm not
> convinced.

So, likewise, I saw your comments (you know the ones you miss so much)
the first time around, as I was not convinced of their worth.

The benchmarks measure certain data. Its fine you do not read into them,
stuff that isn't there, like reliability, for example. 

> > Look, its simple, I am (among other things) discussing these results, so
> > people need to see them.
> 
> However, you do not discuss, you just repeat, and repeat, and repeat.

I never said I wanted discussion, you just said I did.

You just repeat, and repeat, and repeat.

In reality, I quite appreciate reasonable discussion. But, I doubt I
will get much from you.

> But for what reason. Do you want an actual discussion, or do you hate
> the reiserfs developers so much that you want to antagonize any and all
> other Linux file systems developers?

Why do you think I hate reiserfs developers? That is an insane claim.
Why would I hate reiser3 developers?
Why would I hate reiser4 developers? 
Why would I even dislike them?

I think Hans Reiser is a genius. Is that what you mean by hate?

Answer this question. Why do YOU think I am antagonizing reiserfs
developers?

You must have a reason for stating what you have.

> > By the way, I have pulled the plug on my REISER4 system, a number of
> > times now, and it recovers without problem.
> 
> Very nice for you. I guess you have also not been hit by out of memory
> conditions or failing partial writes. So I guess we can ignore the patch
> that was just sent a day or two ago to the mailing list because you
> succesfully pulled the plug, a number of times at that.

Why are you attacking me with sarcasm, when I have just stated a simple
fact?

> > > > I have set up a Reiser4 partition with gzip compression, here is the
> > > > difference in disk usage of a typical Debian installation on two 10GB
> > > > partitions, one with Reiser3 and the other with Reiser4.
> > > > 
> > > > debian:/# df
> > > > Filesystem           1K-blocks      Used Available Use% Mounted on
> > > > /dev/sda3             10490104   6379164   4110940  61% /3
> > > > /dev/sda7              9967960   2632488   7335472  27% /7
> > > ...
> > > > Partition 3 is Reiser3 -- uses 6.4 GB.
> > > > Partition 7 is Reiser4 -- uses 2.6 GB.
> > > > 
> > > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3
> > > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same
> > > > info).
> > > 
> > > Wow, consider me totally and completely, unimpressed.
> > > 
> 
> Here is the part of my email that you seemed to totally ignore,

Yes, I ignored it,.... is that a crime?

> You've just saved yourself $3.80, now go get yourself a latte.
> (see. http://tomayko.com/weblog/2006/09/11/that-dilbert-cartoon)
> 
> Seriously, disk storage is getting less expensive all the time, you can
> already buy a 250GB SATA drive for $70. Also, compression doesn't help
> once you store already compressed data such as jpeg images, mp3 files,
> or mpeg2/4/divx video files. Not only are the savings non-existant, but
> we still end up with the disadvantage that corruption propagates to
> multiple files because of the compression in the file system.

I believe that compression in REISER4 is just a wrapper for gzip or lzo.
So each file is compressed and your "corruption propagation" is just in
your imagination.

> And if it doesn't propagate across multiple files, the compression can't
> be all that good 

It can be as good as gzipping every file, ie, pretty damn good.

> since it can't benefit as much from similarity between
> files. So if that is the case and you really want to save diskspace you
> almost have to look at read-only compressed filesystems such as cramfs,
> squashfs, zisofs, cloop and various other variants in combination with
> a unionfs overlay to get read/write functionality.
> 
> But in the end everything is a tradeoff. You can save diskspace, but
> increase the cost of corruption. 

You deliberately ignored the fact that bad blocks are NOT dealt with by
the filesystem,... but by the operating system. Like I said: If your
filesystem is writing to bad blocks, then throw away your operating
system.

> Use a better compression algorithm, but
> that uses more CPU or which is visible in performance of the
> application. 

Not necessarily, just look at the bonnie++ tests. The tests show that
using compression (with todays hardware) leads to read speeds that are
some FOUR times the physical disk read rate,... 

but you ignore this as you have some strange anti-REISER religion.

Think about it,... read speeds that are some FOUR times the physical
disk read rate,... impossible without the use of compression (or
something similar).

> This can be offset by caching more, and being lazier with
> writebacks, but that hurts on-disk consistency, creates more memory
> pressure (more swapout/paging) and generally slows down other
> applications that aren't actually accessing the disk. Having a fast
> multi-core cpu and lots of memory helps a lot, but at some point what
> tradeoff did we just make, we saved a couple of dollars not having to
> buy a larger disk, but we spend considerably more on the more
> expensive cpu and memory.
> 

Wow, consider me STILL totally impressed by your AMAZING BIAS.
Would you like to tell me why you are SO BIASED against REISER4?

John.
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - Choose from over 50 domains or use your own

Re: Reiser4. BEST FILESYSTEM EVER.

[Willy Tarreau]

On Fri, Apr 06, 2007 at 10:58:45PM -0700, johnrobertbanks@fastmail.fm wrote:
> You know,... you cut out this bit:
> 
> -----------------------------------------------------------------
> 
> > The following benchmarks are from
> > 
> > http://linuxhelp.150m.com/resources/fs-benchmarks.htm or,
> > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm

...

Hey John, please change your disk, it's scratched and you're repeating
yourself again and again. At first I thought "Oh cool, some good news
about reiser4", now when I see "reiserfs" in a thread, I think "oh no,
not this boring guy who escaped from the asylum again !". I hope this
thread will be cut shortly so that you stop doing bad publicity to
reiserfs and its developers, because when a product is indicated as
good by stupid people, it's really doing harm.

Also, about this part :
[Jan]
> > But in the end everything is a tradeoff. You can save diskspace, but
> > increase the cost of corruption. 

I don't 100% agree with Jan, because for some usages (temporary space),
light compression can increase speed. For instance, when processing logs,
I get better speed by compressing intermediate files with LZO on the fly.

[John]
> You deliberately ignored the fact that bad blocks are NOT dealt with by
> the filesystem,... but by the operating system. Like I said: If your
> filesystem is writing to bad blocks, then throw away your operating
> system.

But what you write here is complete crap. The filesystem relies on a
linear block device. The operating system is responsible for doing
read retries or reporting errors on bad blocks, but the FS and only
the FS can decide how not to use some known defective areas, for
instance not putting any metadata on them nor any useful data.

Now if you want to stop writing stupid things again and again, take
your bag, don't miss the bus to school, and listen to the teachers
instead of playing games on your calculator.

Willy
PS: non need to reply either, I'll kill this thread and your address here.

Re: Reiser4. BEST FILESYSTEM EVER.

[johnrobertbanks]

Hi Willy,...

> With decent CPU, you can reach higher read/write data rates than what a
> single off-the-shelf disk can achieve. For this reason, I think that
> reiser4 would be worth trying for this particular usage.

Glad to see you are willing to give Reiser4 a go.

Good man.

----------------------------------------------------------------------


On Sat, 7 Apr 2007 09:15:35 +0200, "Willy Tarreau" <w@1wt.eu> said:
> On Fri, Apr 06, 2007 at 10:58:45PM -0700, johnrobertbanks@fastmail.fm
> wrote:
> > You know,... you cut out this bit:
> > 
> > -----------------------------------------------------------------
> > 
> > > The following benchmarks are from
> > > 
> > > http://linuxhelp.150m.com/resources/fs-benchmarks.htm or,
> > > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm
> 
> ...
> 
> Hey John, please change your disk, it's scratched and you're repeating
> yourself again and again. At first I thought "Oh cool, some good news
> about reiser4", now when I see "reiserfs" in a thread, I think "oh no,
> not this boring guy who escaped from the asylum again !". I hope this
> thread will be cut shortly so that you stop doing bad publicity to
> reiserfs and its developers, because when a product is indicated as
> good by stupid people, it's really doing harm.
> 
> Also, about this part :
> [Jan]
> > > But in the end everything is a tradeoff. You can save diskspace, but
> > > increase the cost of corruption. 
> 
> I don't 100% agree with Jan, because for some usages (temporary space),
> light compression can increase speed. For instance, when processing logs,
> I get better speed by compressing intermediate files with LZO on the fly.
> 
> [John]
> > You deliberately ignored the fact that bad blocks are NOT dealt with by
> > the filesystem,... but by the operating system. Like I said: If your
> > filesystem is writing to bad blocks, then throw away your operating
> > system.
> 
> But what you write here is complete crap. The filesystem relies on a
> linear block device. The operating system is responsible for doing
> read retries or reporting errors on bad blocks, but the FS and only
> the FS can decide how not to use some known defective areas, for
> instance not putting any metadata on them nor any useful data.
> 
> Now if you want to stop writing stupid things again and again, take
> your bag, don't miss the bus to school, and listen to the teachers
> instead of playing games on your calculator.
> 
> Willy
> PS: non need to reply either, I'll kill this thread and your address
> here.
> 
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - One of many happy users:
  http://www.fastmail.fm/docs/quotes.html

Re: Reiser4. BEST FILESYSTEM EVER.

[Krzysztof Halasa]

johnrobertbanks@fastmail.fm writes:

> Why do you think I hate reiserfs developers? That is an insane claim.
> Why would I hate reiser3 developers?
> Why would I hate reiser4 developers? 
> Why would I even dislike them?
>
> I think Hans Reiser is a genius. Is that what you mean by hate?

I think they could hire a person with a bit better marketing skills,
though. People on a technical mailing list don't buy things just
because something on TV told them they have to.

> Answer this question. Why do YOU think I am antagonizing reiserfs
> developers?

That might be just a side effect.

> Think about it,... read speeds that are some FOUR times the physical
> disk read rate,... impossible without the use of compression (or
> something similar).

It's really impossible with compression only unless you're writing
only zeros or stuff alike. I don't know what bonnie uses for testing
but real life data doesn't compress 4 times. Two times, sometimes,
but then it will be typically slower than disk access (I mean read,
as write will be much slower).

You can get faster I/O (both linear speed and access times) using
multiple disks (mirrors etc). Perhaps some ZFS ideas would do us
some good?

Gzip - 3 files (zeros only, raw DV data from video camera, x86_64
kernel rpm file), 10 MB of data (10*1024*1024), done on tmpfs so no
real disk speed factor. The CPU is AMD64 with 1 MB cache per core,
2600 MHz clock (clock scaling disabled). That's my typical usage
pattern (well, not counting these zeros).

$ l -Ggh zeros dv bin
-rw-r--r-- 1 10M Apr  7 15:30 bin
-rw-r--r-- 1 10M Apr  7 15:31 dv
-rw-r--r-- 1 10M Apr  7 15:31 zeros

$ for f in zeros dv bin; do time gzip $f; done
real    0m0.112s
real    0m0.686s
real    0m0.559s

Dealing with pure zeros gzip can get almost 90 MB/s compressing, but
with DV and rpm it only does 14.5 and almost 18 MB/s respectively...

$ l -Ggh zeros.gz dv.gz bin.gz
-rw-r--r-- 1  10K Apr  7 15:31 zeros.gz
-rw-r--r-- 1 9.1M Apr  7 15:31 dv.gz
-rw-r--r-- 1 9.3M Apr  7 15:30 bin.gz

... and though the numbers may still sound impressive, space savings
are less than 10%.

$ for f in zeros dv bin; do time gunzip $f.gz; done
real    0m0.067s
real    0m0.131s
real    0m0.120s

Decompression gives 150 MB/s for zeros and ~ 80 MB/s for DV and rpm.

$ for f in zeros dv bin; do time gzip -1 $f; done
real    0m0.079s
real    0m0.572s
real    0m0.530s

Supposed to be "fastest gzip". 126 MB/s for zeros but still less than
19 MB/s for DV and rpm.

$ l -Ggh zeros.gz dv.gz bin.gz
-rw-r--r-- 1  45K Apr  7 15:31 zeros.gz
-rw-r--r-- 1 9.2M Apr  7 15:31 dv.gz
-rw-r--r-- 1 9.3M Apr  7 15:30 bin.gz

$ for f in zeros dv bin; do time gunzip $f.gz; done
real    0m0.044s
real    0m0.135s
real    0m0.120s

It seems gzip can decompress zeros with 227 MB/s rate.
I assume the "4x read speed" claim comes from something like this.

$ /sbin/hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads:  210 MB in  3.02 seconds =  69.59 MB/sec

$ echo "69.59*4" | bc
278.36

Seems you'd need a faster algorithm, faster machine or slower disk
- slower than this cheap SATA with disabled NCQ (NV SATA) at least:

$ cat /sys/block/sda/device/model
Maxtor 6V250F0

Please note that aplication-level compression usually gives way
better results - the application knows much more.
-- 
Krzysztof Halasa

Re: Reiser4. BEST FILESYSTEM EVER.

[johnrobertbanks]

Krzysztof -- Aren't you missing the point? Twice the speed would be
great,... even a 50% increase,... even a 0% increase.

I checked what bonnie++ actually writes to its test files, for you. It
is about 98-99% zeros.

Still, the results record sequential reads, of 232,729 K/sec, nearly
four times the physical disk read rate, 63,160 K/sec, of the hard drive.

The sequential writes are about three times the physical disk write
rate.

Even if the speed increase was zero, the more efficient use of disk
space means that Reiser4 is worth investigating.

People use RAID arrays to achieve speed increases. 

The people who developed RAID clearly thought that increases in speed
were worth investigating.

> 
> > Why do you think I hate reiserfs developers? That is an insane claim.
> > Why would I hate reiser3 developers?
> > Why would I hate reiser4 developers? 
> > Why would I even dislike them?
> >
> > I think Hans Reiser is a genius. Is that what you mean by hate?
> 
> I think they could hire a person with a bit better marketing skills,
> though. People on a technical mailing list don't buy things just
> because something on TV told them they have to.

I don't work for Reiser if that is what you are suggesting.

And people buy all sorts of lies because someone on TV told them it was
true.

Did you believe Iraq had WMD (weapons of mass destruction) because a
bunch of American liars told you this on TV? Millions of Americans did.

> > Answer this question. Why do YOU think I am antagonizing reiserfs
> > developers?
> 
> That might be just a side effect.
> 
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - Send your email first class

Re: Reiser4. BEST FILESYSTEM EVER.

[Pekka Enberg]

On 4/7/07, johnrobertbanks@fastmail.fm <johnrobertbanks@fastmail.fm> wrote:
> I checked what bonnie++ actually writes to its test files, for you. It
> is about 98-99% zeros.
>
> Still, the results record sequential reads, of 232,729 K/sec, nearly
> four times the physical disk read rate, 63,160 K/sec, of the hard drive.

Excellent! You've established the undeniable hard cold fact that
reiser4 beats the crap out of all other filesystems, when the files
are 98-99% filled with zeros. You've proven your point, so can we stop
this thread now?

Re: Reiser4. BEST FILESYSTEM EVER.

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1622 bytes --]

On Sat, 07 Apr 2007 16:11:46 +0200, Krzysztof Halasa said:

> > Think about it,... read speeds that are some FOUR times the physical
> > disk read rate,... impossible without the use of compression (or
> > something similar).
> 
> It's really impossible with compression only unless you're writing
> only zeros or stuff alike. I don't know what bonnie uses for testing
> but real life data doesn't compress 4 times. Two times, sometimes,

All depends on your data.  From a recent "compress the old logs" job on
our syslog server:

/logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308:       85.4% -- replaced with /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308.gz

And it wasn't a tiny file either - it's a busy mailserver, the logs run to
several hundred megabytes a day.  Syslogs *often* compress 90% or more,
meaning a 10X compression.

> but then it will be typically slower than disk access (I mean read,
> as write will be much slower).

Actually, as far back as 1998 or so, I was able to document 20% *speedups*
on an AIX system that supported compressed file systems - and that was from
when a 133mz PowerPC 604e was a *fast* machine.   Since then, CPUs have gotten
faster at a faster rate than disks have, even increasing the speedup.

The basic theory is that unless you're sitting close to 100%CPU, it is *faster*
to burn some CPU to compress/decompress a 4K chunk of data down to 2K, and then
move 2K to the disk drive, than it is to move 4K.  It's particularly noticable
for larger files - if you can apply the compression to  remove the need to move
2M of data faster than you can move 2M of data, you win.


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Reiser4. BEST FILESYSTEM EVER.

[Adrian Bunk]

On Sat, Apr 07, 2007 at 01:10:31PM -0400, Valdis.Kletnieks@vt.edu wrote:
> On Sat, 07 Apr 2007 16:11:46 +0200, Krzysztof Halasa said:
> 
> > > Think about it,... read speeds that are some FOUR times the physical
> > > disk read rate,... impossible without the use of compression (or
> > > something similar).
> > 
> > It's really impossible with compression only unless you're writing
> > only zeros or stuff alike. I don't know what bonnie uses for testing
> > but real life data doesn't compress 4 times. Two times, sometimes,
> 
> All depends on your data.  From a recent "compress the old logs" job on
> our syslog server:
> 
> /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308:       85.4% -- replaced with /logs/lennier.cc.vt.edu/2007/03/maillog-2007-0308.gz
> 
> And it wasn't a tiny file either - it's a busy mailserver, the logs run to
> several hundred megabytes a day.  Syslogs *often* compress 90% or more,
> meaning a 10X compression.
> 
> > but then it will be typically slower than disk access (I mean read,
> > as write will be much slower).
> 
> Actually, as far back as 1998 or so, I was able to document 20% *speedups*
> on an AIX system that supported compressed file systems - and that was from
> when a 133mz PowerPC 604e was a *fast* machine.   Since then, CPUs have gotten
> faster at a faster rate than disks have, even increasing the speedup.
> 
> The basic theory is that unless you're sitting close to 100%CPU, it is *faster*
> to burn some CPU to compress/decompress a 4K chunk of data down to 2K, and then
> move 2K to the disk drive, than it is to move 4K.  It's particularly noticable
> for larger files - if you can apply the compression to  remove the need to move
> 2M of data faster than you can move 2M of data, you win.

Counterpoints:
- not only CPUs have became faster, RAM has become faster, too
  a kernel tree after an allyesconfig build is at about 1 GB which is 
  less than half the size of RAM in my desktop computer
  if all disk accesses are asynchronous write accesses without any 
  pressure of being done quickly, compression can't improve performance
- today, much of the bigger data is already compressed data like mp3s
  or movies
- for cases like logfiles or databases, application specific compression
  should give best results

There might be special cases where compressed filesystems make sense, 
but my impression is that filesystem compresssion is not important and 
suited for current average systems.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Reiser4. BEST FILESYSTEM EVER.

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]

On Sat, 07 Apr 2007 16:11:46 +0200, Krzysztof Halasa said:
>
> Gzip - 3 files (zeros only, raw DV data from video camera, x86_64
> kernel rpm file), 10 MB of data (10*1024*1024),
> $ l -Ggh zeros dv bin
> -rw-r--r-- 1 10M Apr  7 15:30 bin
> -rw-r--r-- 1 10M Apr  7 15:31 dv
> -rw-r--r-- 1 10M Apr  7 15:31 zeros

> $ l -Ggh zeros.gz dv.gz bin.gz
> -rw-r--r-- 1  10K Apr  7 15:31 zeros.gz
> -rw-r--r-- 1 9.1M Apr  7 15:31 dv.gz
> -rw-r--r-- 1 9.3M Apr  7 15:30 bin.gz
> 
> ... and though the numbers may still sound impressive, space savings
> are less than 10%.

I am quite sure that the kernel RPM file is *already* compressed, at least
somewhat.  Otherwise, it's hard to explain this:

-rw-r--r--    1 529      263     17835757   Apr  5 00:19   kernel-2.6.20-1.3045.fc7.x86_64.rpm

% du -s /lib/modules/2.6.20-1.3038.fc7/
76436   /lib/modules/2.6.20-1.3038.fc7/

and it can't all be slack space at ends of files:

% find /lib/modules/2.6.20-1.3038.fc7/ -type f | wc -l
1482

Even on a 4K filesystem, the *max* wasted slack would be about 4M.

And what do you know - if you tar.gz that /lib/modules:

% tar czf /tmp/kern.tar.gz /lib/modules/2.6.20-1.3038.fc7/
tar: Removing leading `/' from member names
% ls -l /tmp/kern.tar.gz 
-rw-r--r-- 1 valdis valdis 15506359 2007-04-07 13:19 /tmp/kern.tar.gz

The *compressed* tar is about 15M (remember the .rpm contained a 2M vmlinuz
as well - that;s compressed too).  So we're right up to the 17M of the .rpm,
which indicates that the RPM is compressed at a factor close to tar.gz.

I'd not be surprised to find out that your digital-video also contains
at least some light compression - if it's mpeg or similar, that's already
had some *heavy* compression done to it....

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Reiser4. BEST FILESYSTEM EVER.

[Krzysztof Halasa]

Valdis.Kletnieks@vt.edu writes:

> I am quite sure that the kernel RPM file is *already* compressed, at least
> somewhat.

Sure - that's the point - it's better to have the tool compress
data when it makes sense.

OTOH I think Reiser4 fs is not about transparent compression, it's
rather about the plugins etc. There are other filesystems with
transparent compression, that's nothing new.
-- 
Krzysztof Halasa

Re: Reiser4. BEST FILESYSTEM EVER.

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1869 bytes --]

On Fri, 06 Apr 2007 19:47:36 PDT, johnrobertbanks@fastmail.fm said:
> On Fri, 6 Apr 2007 11:21:19 -0400, "Jan Harkes" <jaharkes@cs.cmu.edu>

> > With compression there is a pretty high probability that one corrupted
> > byte or disk block will result in loss of a considerably larger amount
> > of data. 
> 
> Bad blocks are NOT dealt with by the filesystem,... so your comment is
> irrelevant, or just plain wrong.
> 
> If your filesystem is writing to bad blocks, then throw away your
> operating system.

You know... occasionally, blocks go bad *after* you write to them.  If
you have an uncompressed filesystem, it's often possible to recover most
of the file , and just have a few 512-byte blocks of zeros, simply by
doing something like 'dd if=bad.file of=bad.file bs=512 conv=noerror'
or careful applications of 'skip=N'.  If it's compressed, you usually
can't recover the rest of a compression group if a previous block is lost.

(And for those who talk about backups - yes, taking backups is good.
However, it's the rare laptop or desktop machine that can afford the
luxury of RAID disks, and backups usually happen once a night, if that
often.  This means that if you've been working hard on something important
all day, and the disk blows chunks at 4:30PM, you *will* be suddenly very
concerned over exactly how much you can recover off the failing drive....

And yes, I'd *love* to have all my users connected to nice SAN systems that do
snapshotting and remote replication to DR sites and all that - but have you
ever *priced* a petabyte of SAN storage, the NAS gateways to serve it to users,
and upgrading several tens of thousands of network ports to Gig-E? Hint -
US$1M would get us through a pilot, and probably $5M and up to *start*
deployment. Anybody wanna buy us an EMC DMX-3? :)

http://www.emc.com/products/systems/symmetrix/DMX_series/DMX3.jsp


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Reiser4. BEST FILESYSTEM EVER.

[David H. Lynch Jr]

Valdis.Kletnieks@vt.edu wrote:
>
> (And for those who talk about backups - yes, taking backups is good.
> However, it's the rare laptop or desktop machine that can afford the
> luxury of RAID disks, and backups usually happen once a night, if that
> often.  This means that if you've been working hard on something important
> all day, and the disk blows chunks at 4:30PM, you *will* be suddenly very
> concerned over exactly how much you can recover off the failing drive....
    Been there done, that. I am not sure that RAID is physically
practical for a laptop, but it may be
    substantially more practical and economical for a desktop or home
server, than the hope that
    a home user might backup a system.

    I am sure that there are plenty of people with true stories of
backup systems saving their lives.
    But my personal history involves systems with no backup, systems
with poor backup,
    systems with wishful backup, and finally systems with failed backup.
    My first experience with tape drivers was disassembling and patching
a QIC driver (not Linux)
    because small portions of the backup tape were unreadable (the
client smoked while backing the system up)
    and the tape software thought recovery was an all or nothing
proposition.

    My first personal rule of data disaster recovery is start with the
data on the disk.
    I have only had rare instances where 99% of the data on a disk was
not readable and recoverable -
    though for some reason the most critical data is always in that 1%
that puts up a fight.
    Backup systems are what you go to after you have recovered
everything possible from the disk.


   


-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

Re: Reiser4. BEST FILESYSTEM EVER.

[Lennart Sorensen]

On Thu, Apr 05, 2007 at 09:32:11PM -0700, johnrobertbanks@fastmail.fm wrote:
> Don't you agree, that "If they are accurate,.... THEN they are obviously
> very relevant."

Nope, if they are accurate and they have something to do with your
particular usage and applications, then they are relevant.  But it
requires both to make them relevant.  Although it may be possible for a
benchmark to be relevant even if not particularly accurate.

> I have set up a Reiser4 partition with gzip compression, here is the
> difference in disk usage of a typical Debian installation on two 10GB
> partitions, one with Reiser3 and the other with Reiser4.
> 
> debian:/# df
> Filesystem           1K-blocks      Used Available Use% Mounted on
> /dev/sda3             10490104   6379164   4110940  61% /3
> /dev/sda7              9967960   2632488   7335472  27% /7
> 
> Partitions 3 and 7 have exactly the same data on them (the typical
> Debian install).
> 
> The partitions are exactly the same size (although df records different
> sizes).
> 
> Partition 3 is Reiser3 -- uses 6.4 GB.
> Partition 7 is Reiser4 -- uses 2.6 GB.
> 
> So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3
> 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same
> info).
> 
> Don't you think this result is significant in itself?

Only if you think disk space is so valuable that trading cpu time to
compress and decompress the data is a good trade off.  It is not one I
would want to make.  So you saved 3GB, what is that?  About $1 worth?
maybe $2 if you have raid.  How much extra time and cpu will it take to
access the data that way?  How much extra electricity will the cpu use?
What is your time worth?  There are so many variables.  Do you _trust_
reiserfs4 to not loose your data any more or less than some other
filesystem?

--
Len Sorensen

Re: Reiser4. BEST FILESYSTEM EVER.

[johnrobertbanks]

Lennart. Tell me again that these results from 

http://linuxhelp.150m.com/resources/fs-benchmarks.htm and
http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm

are not of interest to you. I still don't understand why you have your
head in the sand.

.-------------------------.
| FILESYSTEM | TIME |DISK |
| TYPE       |(secs)|USAGE|
.-------------------------.
|REISER4 lzo | 1938 | 278 |
|REISER4 gzip| 2295 | 213 |
|REISER4     | 3462 | 692 |
|EXT2        | 4092 | 816 |
|JFS         | 4225 | 806 |
|EXT4        | 4408 | 816 |
|EXT3        | 4421 | 816 |
|XFS         | 4625 | 779 |
|REISER3     | 6178 | 793 |
|FAT32       |12342 | 988 |
|NTFS-3g     |10414 | 772 |
.-------------------------.


Column one measures the time taken to complete the bonnie++ benchmarking
test (run with the parameters bonnie++ -n128:128k:0)

Column two, Disk Usage: measures the amount of disk used to store 655MB
of raw data (which was 3 different copies of the Linux kernel sources).

OR LOOK AT THE FULL RESULTS:

.-------------------------------------------------.
|File         |Disk |Copy |Copy |Tar  |Unzip| Del |
|System       |Usage|655MB|655MB|Gzip |UnTar| 2.5 |
|Type         | (MB)| (1) | (2) |655MB|655MB| Gig |
.-------------------------------------------------.
|REISER4 gzip | 213 | 148 |  68 |  83 |  48 |  70 |
|REISER4 lzo  | 278 | 138 |  56 |  80 |  34 |  84 |
|REISER4 tails| 673 | 148 |  63 |  78 |  33 |  65 |
|REISER4      | 692 | 148 |  55 |  67 |  25 |  56 |
|NTFS3g       | 772 |1333 |1426 | 585 | 767 | 194 |
|NTFS         | 779 | 781 | 173 |   X |   X |   X |
|REISER3      | 793 | 184 |  98 |  85 |  63 |  22 |
|XFS          | 799 | 220 | 173 | 119 |  90 | 106 |
|JFS          | 806 | 228 | 202 |  95 |  97 | 127 |
|EXT4 extents | 806 | 162 |  55 |  69 |  36 |  32 |
|EXT4 default | 816 | 174 |  70 |  74 |  42 |  50 |
|EXT3         | 816 | 182 |  74 |  73 |  43 |  51 |
|EXT2         | 816 | 201 |  82 |  73 |  39 |  67 |
|FAT32        | 988 | 253 | 158 | 118 |  81 |  95 |
.-------------------------------------------------.


Each test was preformed 5 times and the average value recorded.
Disk Usage: The amount of disk used to store the data (which was 3
different copies of the Linux kernel sources).
The raw data (without filesystem meta-data, block alignment wastage,
etc) was 655MB.
Copy 655MB (1): Copy the data over a partition boundary.
Copy 655MB (2): Copy the data within a partition.
Tar Gzip 655MB: Tar and Gzip the data.
Unzip UnTar 655MB: UnGzip and UnTar the data.
Del 2.5 Gig: Delete everything just written (about 2.5 Gig).

To get a feel for the performance increases that can be achieved by
using compression, we look at the total time (in seconds) to run the
test:

bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c)

.-------------------.
| FILESYSTEM | TIME |
.-------------------.
|REISER4 lzo |  1938|
|REISER4 gzip|  2295|
|REISER4     |  3462|
|EXT4        |  4408|
|EXT2        |  4092|
|JFS         |  4225|
|EXT3        |  4421|
|XFS         |  4625|
|REISER3     |  6178|
|FAT32       | 12342|
|NTFS-3g     |>10414|
.-------------------.
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - Access all of your messages and folders
                          wherever you are

Re: Reiser4. BEST FILESYSTEM EVER.

[Lennart Sorensen]

On Sat, Apr 07, 2007 at 05:44:57PM -0700, johnrobertbanks@fastmail.fm wrote:
> Lennart. Tell me again that these results from 
> 
> http://linuxhelp.150m.com/resources/fs-benchmarks.htm and
> http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm

Hmm, copying kernel sources around.  Not that interesting.  How does it
handle mpeg2 files (the majority of my personal data files is on a
mythtv system).  So a few large files, with mostly linear access, and
the occational file deletion.  Compression would gain nothing.

> are not of interest to you. I still don't understand why you have your
> head in the sand.

Well I find it hard to get excited about new filesystems.  I had
sufficiently nasty data loses due to reiserfs 3 back in the early 2.4
kernel days, that I no longer get excited about new filesystems.  now I
want something I trust that hasn't destroyed any of my data.  I tried
XFS for a while, ut the early 2.6 kernels had some nasty bugs in XFS too
that made that pretty much unusable.  Now I just stick with ext3.  Screw
performance, give me something that works all the time.

> .-------------------------.
> | FILESYSTEM | TIME |DISK |
> | TYPE       |(secs)|USAGE|
> .-------------------------.
> |REISER4 lzo | 1938 | 278 |
> |REISER4 gzip| 2295 | 213 |
> |REISER4     | 3462 | 692 |
> |EXT2        | 4092 | 816 |
> |JFS         | 4225 | 806 |
> |EXT4        | 4408 | 816 |
> |EXT3        | 4421 | 816 |
> |XFS         | 4625 | 779 |
> |REISER3     | 6178 | 793 |
> |FAT32       |12342 | 988 |
> |NTFS-3g     |10414 | 772 |
> .-------------------------.
> 
> 
> Column one measures the time taken to complete the bonnie++ benchmarking
> test (run with the parameters bonnie++ -n128:128k:0)

Time without cpu usage is not interesting.  If you can increase
filesystem speed by 10% by doubling cpu load, then I don't want the
increase.  It is all relative.  Wall clock time by itself just doesn't
contain enough data to be useful.

> Column two, Disk Usage: measures the amount of disk used to store 655MB
> of raw data (which was 3 different copies of the Linux kernel sources).

I remember disk compression from the DOS days.  Disk space is too cheap
to bother with that crap anymore.  I don't care if it can theoretically
turn idle cpu cycles into improved disk speed.  Sometimes I don't have
idle cpu cycles to waste on that.

> OR LOOK AT THE FULL RESULTS:
> 
> .-------------------------------------------------.
> |File         |Disk |Copy |Copy |Tar  |Unzip| Del |
> |System       |Usage|655MB|655MB|Gzip |UnTar| 2.5 |
> |Type         | (MB)| (1) | (2) |655MB|655MB| Gig |
> .-------------------------------------------------.
> |REISER4 gzip | 213 | 148 |  68 |  83 |  48 |  70 |
> |REISER4 lzo  | 278 | 138 |  56 |  80 |  34 |  84 |
> |REISER4 tails| 673 | 148 |  63 |  78 |  33 |  65 |
> |REISER4      | 692 | 148 |  55 |  67 |  25 |  56 |
> |NTFS3g       | 772 |1333 |1426 | 585 | 767 | 194 |
> |NTFS         | 779 | 781 | 173 |   X |   X |   X |
> |REISER3      | 793 | 184 |  98 |  85 |  63 |  22 |
> |XFS          | 799 | 220 | 173 | 119 |  90 | 106 |
> |JFS          | 806 | 228 | 202 |  95 |  97 | 127 |
> |EXT4 extents | 806 | 162 |  55 |  69 |  36 |  32 |
> |EXT4 default | 816 | 174 |  70 |  74 |  42 |  50 |
> |EXT3         | 816 | 182 |  74 |  73 |  43 |  51 |
> |EXT2         | 816 | 201 |  82 |  73 |  39 |  67 |
> |FAT32        | 988 | 253 | 158 | 118 |  81 |  95 |
> .-------------------------------------------------.
> 
> 
> Each test was preformed 5 times and the average value recorded.
> Disk Usage: The amount of disk used to store the data (which was 3
> different copies of the Linux kernel sources).
> The raw data (without filesystem meta-data, block alignment wastage,
> etc) was 655MB.
> Copy 655MB (1): Copy the data over a partition boundary.
> Copy 655MB (2): Copy the data within a partition.
> Tar Gzip 655MB: Tar and Gzip the data.
> Unzip UnTar 655MB: UnGzip and UnTar the data.
> Del 2.5 Gig: Delete everything just written (about 2.5 Gig).
> 
> To get a feel for the performance increases that can be achieved by
> using compression, we look at the total time (in seconds) to run the
> test:

kernel sources are some of the most compressable data files around.  Try
with some interesting data instead, like something with larger files,
mostly binary, which isn't likely to compess very much.

> bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c)
> 
> .-------------------.
> | FILESYSTEM | TIME |
> .-------------------.
> |REISER4 lzo |  1938|
> |REISER4 gzip|  2295|
> |REISER4     |  3462|
> |EXT4        |  4408|
> |EXT2        |  4092|
> |JFS         |  4225|
> |EXT3        |  4421|
> |XFS         |  4625|
> |REISER3     |  6178|
> |FAT32       | 12342|
> |NTFS-3g     |>10414|
> .-------------------.
> -- 

Well Reiser4 certainly looks impresive, but I still want to know what the
cpu load is like, what the repair tools are like, how well it handles
power failures in the middle of a write (I didn't like the way reiser3
would claim to have the filesystem back to a sane state, but the file
it had been in the middle of writing now contained garbage in many parts
of it, with no warning what so ever that this had taken place).

Performance looks impresive, but based on the history of reiser3 there
is a lot of historical damage to undo before I will even consider
testing it out.

--
Len Sorensen

Re: Reiser4. BEST FILESYSTEM EVER.

[Theodore Tso]

On Sat, Apr 07, 2007 at 05:44:57PM -0700, johnrobertbanks@fastmail.fm wrote:
> To get a feel for the performance increases that can be achieved by
> using compression, we look at the total time (in seconds) to run the
> test:

You mean the performance increases of writing a file which is mostly
all zero's?  Yawn.....

					- Ted

Re: Reiser4. BEST FILESYSTEM EVER.

[johnrobertbanks]

Teddy,

It is a pity you don't address the full set of results, when you make
your snide comments.

Now since you have them,... why don't you make reasoned comment about
them.

You can read more here:

http://linuxhelp.150m.com/resources/fs-benchmarks.htm and
http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm

.-------------------------.
| FILESYSTEM | TIME |DISK |
| TYPE       |(secs)|USAGE|
.-------------------------.
|REISER4 lzo | 1938 | 278 |
|REISER4 gzip| 2295 | 213 |
|REISER4     | 3462 | 692 |
|EXT2        | 4092 | 816 |
|JFS         | 4225 | 806 |
|EXT4        | 4408 | 816 |
|EXT3        | 4421 | 816 |
|XFS         | 4625 | 779 |
|REISER3     | 6178 | 793 |
|FAT32       |12342 | 988 |
|NTFS-3g     |10414 | 772 |
.-------------------------.


Column one measures the time taken to complete the bonnie++ benchmarking
test (run with the parameters bonnie++ -n128:128k:0)

Column two, Disk Usage: measures the amount of disk used to store 655MB
of raw data (which was 3 different copies of the Linux kernel sources).

OR LOOK AT THE FULL RESULTS:

.-------------------------------------------------.
|File         |Disk |Copy |Copy |Tar  |Unzip| Del |
|System       |Usage|655MB|655MB|Gzip |UnTar| 2.5 |
|Type         | (MB)| (1) | (2) |655MB|655MB| Gig |
.-------------------------------------------------.
|REISER4 gzip | 213 | 148 |  68 |  83 |  48 |  70 |
|REISER4 lzo  | 278 | 138 |  56 |  80 |  34 |  84 |
|REISER4 tails| 673 | 148 |  63 |  78 |  33 |  65 |
|REISER4      | 692 | 148 |  55 |  67 |  25 |  56 |
|NTFS3g       | 772 |1333 |1426 | 585 | 767 | 194 |
|NTFS         | 779 | 781 | 173 |   X |   X |   X |
|REISER3      | 793 | 184 |  98 |  85 |  63 |  22 |
|XFS          | 799 | 220 | 173 | 119 |  90 | 106 |
|JFS          | 806 | 228 | 202 |  95 |  97 | 127 |
|EXT4 extents | 806 | 162 |  55 |  69 |  36 |  32 |
|EXT4 default | 816 | 174 |  70 |  74 |  42 |  50 |
|EXT3         | 816 | 182 |  74 |  73 |  43 |  51 |
|EXT2         | 816 | 201 |  82 |  73 |  39 |  67 |
|FAT32        | 988 | 253 | 158 | 118 |  81 |  95 |
.-------------------------------------------------.


Each test was preformed 5 times and the average value recorded.
Disk Usage: The amount of disk used to store the data (which was 3
different copies of the Linux kernel sources).
The raw data (without filesystem meta-data, block alignment wastage,
etc) was 655MB.
Copy 655MB (1): Copy the data over a partition boundary.
Copy 655MB (2): Copy the data within a partition.
Tar Gzip 655MB: Tar and Gzip the data.
Unzip UnTar 655MB: UnGzip and UnTar the data.
Del 2.5 Gig: Delete everything just written (about 2.5 Gig).


To get a feel for the performance increases that can be achieved by
using compression, we look at the total time (in seconds) to run the
test:

bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c)

.-------------------.
| FILESYSTEM | TIME |
.-------------------.
|REISER4 lzo |  1938|
|REISER4 gzip|  2295|
|REISER4     |  3462|
|EXT4        |  4408|
|EXT2        |  4092|
|JFS         |  4225|
|EXT3        |  4421|
|XFS         |  4625|
|REISER3     |  6178|
|FAT32       | 12342|
|NTFS-3g     |>10414|
.-------------------.



On Sat, 7 Apr 2007 22:56:32 -0400, "Theodore Tso" <tytso@mit.edu> said:
> On Sat, Apr 07, 2007 at 05:44:57PM -0700, johnrobertbanks@fastmail.fm
> wrote:
> > To get a feel for the performance increases that can be achieved by
> > using compression, we look at the total time (in seconds) to run the
> > test:
> 
> You mean the performance increases of writing a file which is mostly
> all zero's?  Yawn.....
> 
> 					- Ted
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - IMAP accessible web-mail

Re: Reiser4. BEST FILESYSTEM EVER.

[Jose Celestino]

Words by johnrobertbanks@fastmail.fm [Sat, Apr 07, 2007 at 09:13:32PM -0700]:
> Teddy,
> 
> It is a pity you don't address the full set of results, when you make
> your snide comments.
> 
> Now since you have them,... why don't you make reasoned comment about
> them.
> 
> You can read more here:
> 

John,

it is not because you keep posting the same numbers over and over again
(or is this your new signature?) that they will be more substantiated
(hint: cpu usage).
Just more annoying each time.

I'll remember to use reiser4 for my
90-percent-zero-files-no-need-for-proven-robustness-and-plenty-of-cpu-to-spare
boxes. Thank you.

> http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm
> 
> .-------------------------.
> | FILESYSTEM | TIME |DISK |
> | TYPE       |(secs)|USAGE|
> .-------------------------.
> |REISER4 lzo | 1938 | 278 |
> |REISER4 gzip| 2295 | 213 |
> |REISER4     | 3462 | 692 |
> |EXT2        | 4092 | 816 |
> |JFS         | 4225 | 806 |
> |EXT4        | 4408 | 816 |
> |EXT3        | 4421 | 816 |
> |XFS         | 4625 | 779 |
> |REISER3     | 6178 | 793 |
> |FAT32       |12342 | 988 |
> |NTFS-3g     |10414 | 772 |
> .-------------------------.
> 
> 
> Column one measures the time taken to complete the bonnie++ benchmarking
> test (run with the parameters bonnie++ -n128:128k:0)
> 
> Column two, Disk Usage: measures the amount of disk used to store 655MB
> of raw data (which was 3 different copies of the Linux kernel sources).
> 
> OR LOOK AT THE FULL RESULTS:
> 
> .-------------------------------------------------.
> |File         |Disk |Copy |Copy |Tar  |Unzip| Del |
> |System       |Usage|655MB|655MB|Gzip |UnTar| 2.5 |
> |Type         | (MB)| (1) | (2) |655MB|655MB| Gig |
> .-------------------------------------------------.
> |REISER4 gzip | 213 | 148 |  68 |  83 |  48 |  70 |
> |REISER4 lzo  | 278 | 138 |  56 |  80 |  34 |  84 |
> |REISER4 tails| 673 | 148 |  63 |  78 |  33 |  65 |
> |REISER4      | 692 | 148 |  55 |  67 |  25 |  56 |
> |NTFS3g       | 772 |1333 |1426 | 585 | 767 | 194 |
> |NTFS         | 779 | 781 | 173 |   X |   X |   X |
> |REISER3      | 793 | 184 |  98 |  85 |  63 |  22 |
> |XFS          | 799 | 220 | 173 | 119 |  90 | 106 |
> |JFS          | 806 | 228 | 202 |  95 |  97 | 127 |
> |EXT4 extents | 806 | 162 |  55 |  69 |  36 |  32 |
> |EXT4 default | 816 | 174 |  70 |  74 |  42 |  50 |
> |EXT3         | 816 | 182 |  74 |  73 |  43 |  51 |
> |EXT2         | 816 | 201 |  82 |  73 |  39 |  67 |
> |FAT32        | 988 | 253 | 158 | 118 |  81 |  95 |
> .-------------------------------------------------.
> 
> 
> Each test was preformed 5 times and the average value recorded.
> Disk Usage: The amount of disk used to store the data (which was 3
> different copies of the Linux kernel sources).
> The raw data (without filesystem meta-data, block alignment wastage,
> etc) was 655MB.
> Copy 655MB (1): Copy the data over a partition boundary.
> Copy 655MB (2): Copy the data within a partition.
> Tar Gzip 655MB: Tar and Gzip the data.
> Unzip UnTar 655MB: UnGzip and UnTar the data.
> Del 2.5 Gig: Delete everything just written (about 2.5 Gig).
> 
> 
> To get a feel for the performance increases that can be achieved by
> using compression, we look at the total time (in seconds) to run the
> test:
> 
> bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c)
> 
> .-------------------.
> | FILESYSTEM | TIME |
> .-------------------.
> |REISER4 lzo |  1938|
> |REISER4 gzip|  2295|
> |REISER4     |  3462|
> |EXT4        |  4408|
> |EXT2        |  4092|
> |JFS         |  4225|
> |EXT3        |  4421|
> |XFS         |  4625|
> |REISER3     |  6178|
> |FAT32       | 12342|
> |NTFS-3g     |>10414|
> .-------------------.
> 
> 
> 
> On Sat, 7 Apr 2007 22:56:32 -0400, "Theodore Tso" <tytso@mit.edu> said:
> > On Sat, Apr 07, 2007 at 05:44:57PM -0700, johnrobertbanks@fastmail.fm
> > wrote:
> > > To get a feel for the performance increases that can be achieved by
> > > using compression, we look at the total time (in seconds) to run the
> > > test:
> > 
> > You mean the performance increases of writing a file which is mostly
> > all zero's?  Yawn.....
> > 
> > 					- Ted
> -- 
>   
>   johnrobertbanks@fastmail.fm
> 
> -- 
> http://www.fastmail.fm - IMAP accessible web-mail
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Jose Celestino
----------------------------------------------------------------
http://www.msversus.org/     ; http://techp.org/petition/show/1
http://www.vinc17.org/noswpat.en.html
----------------------------------------------------------------
"And on the trillionth day, Man created Gods." -- Thomas D. Pate

Re: Reiser4. BEST FILESYSTEM EVER.

[johnrobertbanks]

Jose,
  since you clearly have nothing useful to say. Why don't you let Teddy
  talk for himself.


On Sun, 8 Apr 2007 13:48:11 +0100, "Jose Celestino" <japc@co.sapo.pt>
said:
> Words by johnrobertbanks@fastmail.fm [Sat, Apr 07, 2007 at 09:13:32PM
> -0700]:
> > Teddy,
> > 
> > It is a pity you don't address the full set of results, when you make
> > your snide comments.
> > 
> > Now since you have them,... why don't you make reasoned comment about
> > them.
> > 
> > You can read more here:
> > 
> 
> John,
> 
> it is not because you keep posting the same numbers over and over again
> (or is this your new signature?) that they will be more substantiated
> (hint: cpu usage).
> Just more annoying each time.
> 
> I'll remember to use reiser4 for my
> 90-percent-zero-files-no-need-for-proven-robustness-and-plenty-of-cpu-to-spare
> boxes. Thank you.
> 
> > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm
> > 
> > .-------------------------.
> > | FILESYSTEM | TIME |DISK |
> > | TYPE       |(secs)|USAGE|
> > .-------------------------.
> > |REISER4 lzo | 1938 | 278 |
> > |REISER4 gzip| 2295 | 213 |
> > |REISER4     | 3462 | 692 |
> > |EXT2        | 4092 | 816 |
> > |JFS         | 4225 | 806 |
> > |EXT4        | 4408 | 816 |
> > |EXT3        | 4421 | 816 |
> > |XFS         | 4625 | 779 |
> > |REISER3     | 6178 | 793 |
> > |FAT32       |12342 | 988 |
> > |NTFS-3g     |10414 | 772 |
> > .-------------------------.
> > 
> > 
> > Column one measures the time taken to complete the bonnie++ benchmarking
> > test (run with the parameters bonnie++ -n128:128k:0)
> > 
> > Column two, Disk Usage: measures the amount of disk used to store 655MB
> > of raw data (which was 3 different copies of the Linux kernel sources).
> > 
> > OR LOOK AT THE FULL RESULTS:
> > 
> > .-------------------------------------------------.
> > |File         |Disk |Copy |Copy |Tar  |Unzip| Del |
> > |System       |Usage|655MB|655MB|Gzip |UnTar| 2.5 |
> > |Type         | (MB)| (1) | (2) |655MB|655MB| Gig |
> > .-------------------------------------------------.
> > |REISER4 gzip | 213 | 148 |  68 |  83 |  48 |  70 |
> > |REISER4 lzo  | 278 | 138 |  56 |  80 |  34 |  84 |
> > |REISER4 tails| 673 | 148 |  63 |  78 |  33 |  65 |
> > |REISER4      | 692 | 148 |  55 |  67 |  25 |  56 |
> > |NTFS3g       | 772 |1333 |1426 | 585 | 767 | 194 |
> > |NTFS         | 779 | 781 | 173 |   X |   X |   X |
> > |REISER3      | 793 | 184 |  98 |  85 |  63 |  22 |
> > |XFS          | 799 | 220 | 173 | 119 |  90 | 106 |
> > |JFS          | 806 | 228 | 202 |  95 |  97 | 127 |
> > |EXT4 extents | 806 | 162 |  55 |  69 |  36 |  32 |
> > |EXT4 default | 816 | 174 |  70 |  74 |  42 |  50 |
> > |EXT3         | 816 | 182 |  74 |  73 |  43 |  51 |
> > |EXT2         | 816 | 201 |  82 |  73 |  39 |  67 |
> > |FAT32        | 988 | 253 | 158 | 118 |  81 |  95 |
> > .-------------------------------------------------.
> > 
> > 
> > Each test was preformed 5 times and the average value recorded.
> > Disk Usage: The amount of disk used to store the data (which was 3
> > different copies of the Linux kernel sources).
> > The raw data (without filesystem meta-data, block alignment wastage,
> > etc) was 655MB.
> > Copy 655MB (1): Copy the data over a partition boundary.
> > Copy 655MB (2): Copy the data within a partition.
> > Tar Gzip 655MB: Tar and Gzip the data.
> > Unzip UnTar 655MB: UnGzip and UnTar the data.
> > Del 2.5 Gig: Delete everything just written (about 2.5 Gig).
> > 
> > 
> > To get a feel for the performance increases that can be achieved by
> > using compression, we look at the total time (in seconds) to run the
> > test:
> > 
> > bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c)
> > 
> > .-------------------.
> > | FILESYSTEM | TIME |
> > .-------------------.
> > |REISER4 lzo |  1938|
> > |REISER4 gzip|  2295|
> > |REISER4     |  3462|
> > |EXT4        |  4408|
> > |EXT2        |  4092|
> > |JFS         |  4225|
> > |EXT3        |  4421|
> > |XFS         |  4625|
> > |REISER3     |  6178|
> > |FAT32       | 12342|
> > |NTFS-3g     |>10414|
> > .-------------------.
> > 
> > 
> > 
> > On Sat, 7 Apr 2007 22:56:32 -0400, "Theodore Tso" <tytso@mit.edu> said:
> > > On Sat, Apr 07, 2007 at 05:44:57PM -0700, johnrobertbanks@fastmail.fm
> > > wrote:
> > > > To get a feel for the performance increases that can be achieved by
> > > > using compression, we look at the total time (in seconds) to run the
> > > > test:
> > > 
> > > You mean the performance increases of writing a file which is mostly
> > > all zero's?  Yawn.....
> > > 
> > > 					- Ted
> > -- 
> >   
> >   johnrobertbanks@fastmail.fm
> > 
> > -- 
> > http://www.fastmail.fm - IMAP accessible web-mail
> > 
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> -- 
> Jose Celestino
> ----------------------------------------------------------------
> http://www.msversus.org/     ; http://techp.org/petition/show/1
> http://www.vinc17.org/noswpat.en.html
> ----------------------------------------------------------------
> "And on the trillionth day, Man created Gods." -- Thomas D. Pate
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - Does exactly what it says on the tin

Re: Reiser4. BEST FILESYSTEM EVER.

[Willy Tarreau]

On Sun, Apr 08, 2007 at 06:21:29AM -0700, johnrobertbanks@fastmail.fm wrote:
> Jose,
>   since you clearly have nothing useful to say. Why don't you let Teddy
>   talk for himself.

John,

You should first apply your own advice to yourself. Annoying everyone
with the exact same mail 10 times a day is really disserving to the
cause you pretend to defend. Please stop tainting reiser4's reputation,
because I suspect that it can do far more things than what you make it
look like. Its developers certainly need useful reports instead of a
mentally deficient's rant.

Now please call the nurse for your injection and go back to bed.

Thank you in advance,
Willy

Re: Reiser4. BEST FILESYSTEM EVER.

[Theodore Tso]

The reason why I ignore the tar+gzip tests is that in the past Hans
has rigged the test by using a tar ball which was generated by
unpacking a set of kernel sources on a reiser4 filesystem, and then
repacking them using tar+gzip.  The result was a tar file whose files
were optimally laid out so that reiser4 could insert them into the
filesystem b-tree without doing any extra work.

I can't say for sure whether or not this set of benchmarks has done
this (there's not enough information describing the benchmark setup),
but the sad fact of the matter is that people trying to pitch Reiser4
have generated for themselves a reputation for using rigged
benchmarks.  Hans's used of a carefully stacked and ordered tar file
(which is the same as stacking a deck of cards), and your repeated use
of the bonnee++ benchmarks despite being told that it is a meaningless
result given the fact that well, zero's compress very well and most
people are interested in storing a file of all zeros, has caused me to
look at any benchmarks cited by Reiser4 partisans with a very
jaundiced and skeptical eye.

Fortunately for you, it's not up to me whether or not Reiser4 makes it
into the kernel.  And if it works for you, hey, go wild.  You can
always patch it into your own kernel and encourage others to do the
same with respect to getting it tested and adopted.  My personal take
on it is that Reiser3, Reiser4 and JFS suffer the same problems, which
is to say they have a very small and limited development community,
and this was referenced in Novell's decision to drop Reiser3:

http://linux.wordpress.com/2006/09/27/suse-102-ditching-reiserfs-as-it-default-fs/

SuSE has deprecated Reiser3 *and* JFS, and I believe quite strongly it
is the failure of the organizations to attract a diverse development
community is ultimately what doomed them in the long term, both in
terms of support as the kernel migrated and new feature support.  It
is for that reason that Hans' personality traits that tend to drive
away those developers who would help them, beyond those that he hires,
is what has been so self-destructive to Reiser4.  Read the
announcement Jeff Mahoney from SUSE Labs again; he pointed out was
that reiser3 was getting dropped even though it performs better than
ext3 in some scenarios.  There are many other considerations, such as
a filesystem's robustness in case on-disk corruption, long term
maintenance as the kernel maintains, availability of developers to
provide bug fixes, how well the system performs on systems with
multiple cores/CPU's, etc.

						- Ted

Re: Reiser4. BEST FILESYSTEM EVER.

[Jeff Mahoney]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Theodore Tso wrote:
> The reason why I ignore the tar+gzip tests is that in the past Hans
> has rigged the test by using a tar ball which was generated by
> unpacking a set of kernel sources on a reiser4 filesystem, and then
> repacking them using tar+gzip.  The result was a tar file whose files
> were optimally laid out so that reiser4 could insert them into the
> filesystem b-tree without doing any extra work.
> 
> I can't say for sure whether or not this set of benchmarks has done
> this (there's not enough information describing the benchmark setup),
> but the sad fact of the matter is that people trying to pitch Reiser4
> have generated for themselves a reputation for using rigged
> benchmarks.  Hans's used of a carefully stacked and ordered tar file
> (which is the same as stacking a deck of cards), and your repeated use
> of the bonnee++ benchmarks despite being told that it is a meaningless
> result given the fact that well, zero's compress very well and most
> people are interested in storing a file of all zeros, has caused me to
> look at any benchmarks cited by Reiser4 partisans with a very
> jaundiced and skeptical eye.
> 
> Fortunately for you, it's not up to me whether or not Reiser4 makes it
> into the kernel.  And if it works for you, hey, go wild.  You can
> always patch it into your own kernel and encourage others to do the
> same with respect to getting it tested and adopted.  My personal take
> on it is that Reiser3, Reiser4 and JFS suffer the same problems, which
> is to say they have a very small and limited development community,
> and this was referenced in Novell's decision to drop Reiser3:
> 
> http://linux.wordpress.com/2006/09/27/suse-102-ditching-reiserfs-as-it-default-fs/
> 
> SuSE has deprecated Reiser3 *and* JFS, and I believe quite strongly it
> is the failure of the organizations to attract a diverse development
> community is ultimately what doomed them in the long term, both in
> terms of support as the kernel migrated and new feature support.  It
> is for that reason that Hans' personality traits that tend to drive
> away those developers who would help them, beyond those that he hires,
> is what has been so self-destructive to Reiser4.  Read the
> announcement Jeff Mahoney from SUSE Labs again; he pointed out was
> that reiser3 was getting dropped even though it performs better than
> ext3 in some scenarios.  There are many other considerations, such as
> a filesystem's robustness in case on-disk corruption, long term
> maintenance as the kernel maintains, availability of developers to
> provide bug fixes, how well the system performs on systems with
> multiple cores/CPU's, etc.

Those are all arguments I've made and still stand by, but I should
address one point that has been repeated fairly often. Novell _isn't_
dropping support for Reiser3 in any of our products. The change only
refers to the choice of a default file system. Most users don't care
about which file system they use, and those that do are still free to
choose reiser3 if they want it. We'll support it and I still have
patches under development to improve it.

- -Jeff

- --
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFGGTHYLPWxlyuTD7IRAj0SAJ4txD5NoStOA4GFgkzcXDdE/Xf9ngCZATNL
QtyNTGbi6YFbNF71T5C9hTA=
=Emwr
-----END PGP SIGNATURE-----

Re: Reiser4. BEST FILESYSTEM EVER.

[Christer Weinigel]

johnrobertbanks@fastmail.fm writes:

> Lennart. Tell me again that these results from 
> 
> http://linuxhelp.150m.com/resources/fs-benchmarks.htm and
> http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm
> 
> are not of interest to you. I still don't understand why you have your
> head in the sand.

Oh, for fucks sake, stop sounding like a broken record.  You have
repeated the same totally meaningless statistics more times than I
care to count.  Please shut the fuck up.

As you discovered yourself (even though you seem to fail to understand
the significance of your discovery), bonnie writes files that consist
of mostly zeroes.  If your normal use cases consist of creating a
bunch of files containing zeroes, reiser4 with compression will do
great.  Just lovely.  Except that nobody sane would store a lot of
files containing zeroes, except an an excercize in mental
masturbation.  So the two bonnie benchmarks with lzo and gzip are
totally meaningless for any real life usages.

As for the amount of disk needed to store three kernel trees, the
figures you quote show that Reiser4 does tail combining where the tail
of multiple files are stored in one disk block.  A nice trick that
seems save you about 15% disk space compared to ext3.  Now you have to
realise what that means, it means that if the disk block containing
those tails (or any metadata pointing at that block) gets corrupted,
instead of just losing one disk block for one file, you will have lost
the tail for all the files sharing that disk block.  Depending on your
personal prioritites, saving 15% of the space may be worth the risk to
you, or maybe not.  Personally, for the only disk I'm short on space
on, I mostly store flac encoded images of my CD collection, and saving
2kByte out of every 300MByte disk simply doesn't make any difference,
and I much prefer a stable file system that I can trust not to lose my
data.  You might make different choices.

The same goes for just about every feature that you tout, it has its
advantages, and it has its disadvantages.  Doing compression on data
is great if the data you store is compressible, and sucks if it isn't.
Doing compression on each disk block and then packing multiple
compressed blocks into each physical disk block will probably save
some space if the data is compressible, but at the same time it means
that you will spend a lot of CPU (and cache footprint) compressing and
uncompressing that data.  On a single user system where the CPU is
mostly idle it might not make much of a difference, on a heavily
loaded multiuser system it might do.

Logs can be compressed quite well using a block based compression
scheme, but the logs can be compressed even better by doing
compression on the whole file with gzip.  So what's the best choice,
to do transparent compression on the fly giving ok compression or
teaching the userspace tools to do compression of old logs and get
really good compression?  Or maybe disk space really isn't that
important anyway and the best thing is to just leave the logs
uncompressed.

Another example: one of the things Reiser3 is supposed to be really
good for is storing an INN news spool, doing tail merging of lots of
individual files containing articles gives a great space saving, and
since it's just a news spool, reliability in face of a system crashes
really don't matter all that much.  On the other hand, INN's Cyclic
News File System running on top of ext2 is probably an even better
choice in that case.  What do you want to use?

What I want to get at is that you can troll the mailing lists (and
crossposting stupid inflammatory material with an inane subject to a
bunch of mailing lists the way you have done definitely is trolling)
trying to say that whatever you're trying to sell is the best, but at
the end, if a file system is better or not is a lot more complex than
quoting just one benchmark (which, once again, is meaningless,
compressing a lot of zeroes is simple and really does not tell you
anything about real world usages).  And there are other considerations
too, even if Reiser4 would be the best thing since sliced breadd, can
I trust Hans Reiser to support Reiser4 for the next five years?  Or
will he drop support for Reiser4 the same way he dropped support for
the old Reiser3 when Reiser4 came along?  Or will he drop Reiser4 when
the grant to do Reiser 4 development expires?

  /Christer

-- 
"Just how much can I get away with and still go to heaven?"

Christer Weinigel <christer@weinigel.se>  http://www.weinigel.se

Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel

[johnrobertbanks]

Christer Weinigel: Until YOU, have actually used the REISER4 filesystem
yourself, I think YOU OWE IT to the people on the linux-kernel mailing
list, to, AS YOU SAY, shut the fuck up. 

Even reading up on the REISER4 filesystem would help. 

Applying a little intelligence would undoubtedly help too.

> johnrobertbanks@fastmail.fm writes:
> 
> > Lennart. Tell me again that these results from 
> > 
> > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and
> > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm
> > 
> > are not of interest to you. I still don't understand why you
> > have your head in the sand.
> 
> Oh, for fucks sake, stop sounding like a broken record.  

Oh, for fucks sake, would you, and your religious anti-REISER cohorts,
stop sounding like a broken record.

> You have repeated the same totally meaningless statistics more 
> times than I care to count.  Please shut the fuck up.

You, and your religious anti-REISER cohorts, have indeed repeated the
same broken arguments (if you can call them such) more times than I care
to count.

NO statistics, NO real facts, just selective MANIPULATION of facts.

> Please shut the fuck up.

Yes, why don't you politely, shut the fuck up.

Until YOU, have actually used the REISER4 filesystem yourself, I think
YOU OWE IT to the people on the linux-kernel mailing list, to shut the
fuck up, as YOU say.

I guess, the fact that you are a TOTAL HYPOCRITE, has completely escaped
you.

By the way: Did I thank you "delightful" people for the "pleasant"
welcome to the linux-kernel mailing list?

-----------------------------------------------------------------

> So the two bonnie benchmarks with lzo and gzip are
> totally meaningless for any real life usages.

YOU (yes, the one with no experience and next to NO knowledge on the
subject) claim that because bonnie++ writes files that are mostly zeros,
the results are meaningless. It should be mentioned that bonnie++ writes
files that are mostly zero for all the filesystems compared. So the
results are meaningful, contrary to would you claim.

And hopefully all will notice that you just ignore these tests:

.-------------------------------------------------.
|File         |Disk |Copy |Copy |Tar  |Unzip| Del |
|System       |Usage|655MB|655MB|Gzip |UnTar| 2.5 |
|Type         | (MB)| (1) | (2) |655MB|655MB| Gig |
.-------------------------------------------------.
|REISER4 gzip | 213 | 148 |  68 |  83 |  48 |  70 |
|REISER4 lzo  | 278 | 138 |  56 |  80 |  34 |  84 |
|REISER4 tails| 673 | 148 |  63 |  78 |  33 |  65 |
|REISER4      | 692 | 148 |  55 |  67 |  25 |  56 |
|NTFS3g       | 772 |1333 |1426 | 585 | 767 | 194 |
|NTFS         | 779 | 781 | 173 |   X |   X |   X |
|REISER3      | 793 | 184 |  98 |  85 |  63 |  22 |
|XFS          | 799 | 220 | 173 | 119 |  90 | 106 |
|JFS          | 806 | 228 | 202 |  95 |  97 | 127 |
|EXT4 extents | 806 | 162 |  55 |  69 |  36 |  32 |
|EXT4 default | 816 | 174 |  70 |  74 |  42 |  50 |
|EXT3         | 816 | 182 |  74 |  73 |  43 |  51 |
|EXT2         | 816 | 201 |  82 |  73 |  39 |  67 |
|FAT32        | 988 | 253 | 158 | 118 |  81 |  95 |
.-------------------------------------------------.


where the files are definitely NOT mostly zeros. 

Your negligence has to be deliberate,... but why?

Are you manipulating the facts just to try and win an argument?

Most sane people will realize, that what you say is simply wrong.

ALSO YOU IGNORE examples offered by others, on lkml, which contradict
your assertion: FOR EXAMPLE:

> I see the same thing with my nightly scripts that do syslog analysis, last year 
> I trimmed 2 hours from the nightly run by processing compressed files instead of 
> uncompressed ones (after I did this I configured it to compress the files as 
> they are rolled, but rolling every 5 min the compression takes <20 seconds, so 
> the compression is < 30 min)

>From David Lang http://lkml.org/lkml/2007/4/7/196

Willy Tarreau also mentions this situation in a couple of articles.

Let me spoon feed you:

David has said that compressing the logs takes

24 x 12 x 20 secs = 5,760 secs = 1.6 hours of CPU time (over the day)

but he saves 2 hours of CPU time on the daily syslog analysis.

For a total (minimum) saving of 24 minutes.

The actual saving is probably much greater. It depends on the CPU
utilization when not compressing, ie, whether you are using ide CPU
cycles or not. I guess it also depends on whether you can go home one
and a half hours earlier by using compression, or if your boss makes you
stick around anyway.

NOTE THAT THE FILES IN THIS EXAMPLE ARE ALSO NOT MAINLY ZEROS.

MAYBE you just lacked the knowledge to understand what David was saying,
or maybe your desire to denigrate REISER4 is so strong, that you simply
don't care what other people say about similar circumstances.

I am not sure why you have to be spoon feed on these matters, or why you
adamantly refuse to find the facts of the matter, for yourself.

-----------------------------------------------------------------
-----------------------------------------------------------------

On Fri, 6 Apr 2007 23:30:49 -0400, "Jan Harkes" <jaharkes@cs.cmu.edu>
said:
> 
> Since you decide to publically respond to a private email, but not only
> you did not 'discuss' anything I wrote and in fact cut out most of the
> useful information in my reply I guess I will have to repeat my
> observations.

You are a funny guy Jan.

Here you are, once again, cutting out my most useful information, ie,
the data I was discussing, while complaining that I cut out your most
useful information. 

You know,... you cut out this bit:

-----------------------------------------------------------------

> The following benchmarks are from
> 
> http://linuxhelp.150m.com/resources/fs-benchmarks.htm or,
> http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm
> 
> .-------------------------.
> | FILESYSTEM | TIME |DISK |
> | TYPE       |(secs)|USAGE|
> .-------------------------.
> |REISER4 lzo | 1938 | 278 |
> |REISER4 gzip| 2295 | 213 |
> |REISER4     | 3462 | 692 |
> |EXT2        | 4092 | 816 |
> |JFS         | 4225 | 806 |
> |EXT4        | 4408 | 816 |
> |EXT3        | 4421 | 816 |
> |XFS         | 4625 | 779 |
> |REISER3     | 6178 | 793 |
> |FAT32       |12342 | 988 |
> |NTFS-3g     |10414 | 772 |
> .-------------------------.
> 
> Column one measures the time taken to complete the bonnie++ benchmarking test (run with the parameters bonnie++ -n128:128k:0)
> 
> Column two, Disk Usage: measures the amount of disk used to store 655MB of raw data (which was 3 different copies of the Linux kernel sources).

-----------------------------------------------------------------

And this bit: 

Jan,... Here is another section, you conveniently cut out. Maybe you
should explain why you cut out this section? Was it embarrassing to you?
I mean, your statement is sort of correct,... but it shows a basic
misunderstanding of filesystems.

> > With compression there is a pretty high probability that one corrupted
> > byte or disk block will result in loss of a considerably larger amount
> > of data. 
> 
> Bad blocks are NOT dealt with by the filesystem,... so your comment is
> irrelevant, or just plain wrong.
> 
> If your filesystem is writing to bad blocks, then throw away your
> operating system.

-----------------------------------------------------------------

It is true that I considered your "most useful information," an
irrelevant section, which is why it was cut out (ignored).

I did not see my doing this, any worse than you doing it. I did not
realize that you were being so impolite.

As to your email being private, I had thought I had joined a mailing
list. I had no idea your email was meant to be private and just
considered it like all the others.

Now you mention it, I wondered why the email did not automatically list
the mailing lists, as recipients, and I had to add them. If I had
realized this I may have added the mailing lists as recipients, anyway.
It would be like me to do such. However, you should understand that I am
new to mailing lists.
 
> 
> > > Do you really have to repeat the results in every email you sent?
> > 
> > Damn, I did it again. WHY DO YOU CARE?
> 
> Because I saw them the first time around. And although the performance
> difference on those micro benchmarks seems quite impressive, I'm not
> convinced.

So, likewise, I saw your comments (you know the ones you miss so much)
the first time around, and I ignored them, as I was not convinced of
their worth.

The benchmarks measure certain data. Its fine you do not read into them,
stuff that isn't there, like reliability, for example. 

> > Look, its simple, I am (among other things) discussing these results, so
> > people need to see them.
> 
> However, you do not discuss, you just repeat, and repeat, and repeat.

I never said I wanted discussion, you just said I did.

You just repeat, and repeat, and repeat.

In reality, I quite appreciate reasonable discussion. But, I doubt I
will get much from you.

> But for what reason. Do you want an actual discussion, or do you hate
> the reiserfs developers so much that you want to antagonize any and all
> other Linux file systems developers?

Why do you think I hate reiserfs developers? That is an insane claim.
Why would I hate reiser3 developers?
Why would I hate reiser4 developers? 
Why would I even dislike them?

I think Hans Reiser is a genius. Is that what you mean by hate?

Answer this question. Why do YOU think I am antagonizing reiserfs
developers?

You must have a reason for stating what you have.

> > By the way, I have pulled the plug on my REISER4 system, a number of
> > times now, and it recovers without problem.
> 
> Very nice for you. I guess you have also not been hit by out of memory
> conditions or failing partial writes. So I guess we can ignore the patch
> that was just sent a day or two ago to the mailing list because you
> succesfully pulled the plug, a number of times at that.

Why are you attacking me with sarcasm, when I have just stated a simple
fact?

> > > > I have set up a Reiser4 partition with gzip compression, here is the
> > > > difference in disk usage of a typical Debian installation on two 10GB
> > > > partitions, one with Reiser3 and the other with Reiser4.
> > > > 
> > > > debian:/# df
> > > > Filesystem           1K-blocks      Used Available Use% Mounted on
> > > > /dev/sda3             10490104   6379164   4110940  61% /3
> > > > /dev/sda7              9967960   2632488   7335472  27% /7
> > > ...
> > > > Partition 3 is Reiser3 -- uses 6.4 GB.
> > > > Partition 7 is Reiser4 -- uses 2.6 GB.
> > > > 
> > > > So Reiser4 uses 2.6 GB to store the (typical) data that it takes Reiser3
> > > > 6.4 GB to store (note it would take ext2/3/4 some 7 GB to store the same
> > > > info).
> > > 
> > > Wow, consider me totally and completely, unimpressed.
> > > 
> 
> Here is the part of my email that you seemed to totally ignore,

Yes, I ignored it,.... is that a crime?

> You've just saved yourself $3.80, now go get yourself a latte.
> (see. http://tomayko.com/weblog/2006/09/11/that-dilbert-cartoon)
> 
> Seriously, disk storage is getting less expensive all the time, you can
> already buy a 250GB SATA drive for $70. Also, compression doesn't help
> once you store already compressed data such as jpeg images, mp3 files,
> or mpeg2/4/divx video files. Not only are the savings non-existant, but
> we still end up with the disadvantage that corruption propagates to
> multiple files because of the compression in the file system.

I believe that compression in REISER4 is just a wrapper for gzip or lzo.
So each file is compressed and your "corruption propagation" is just in
your imagination.

> And if it doesn't propagate across multiple files, the compression can't
> be all that good 

It can be as good as gzipping every file, ie, pretty damn good.

> since it can't benefit as much from similarity between
> files. So if that is the case and you really want to save diskspace you
> almost have to look at read-only compressed filesystems such as cramfs,
> squashfs, zisofs, cloop and various other variants in combination with
> a unionfs overlay to get read/write functionality.
> 
> But in the end everything is a tradeoff. You can save diskspace, but
> increase the cost of corruption. 

You deliberately ignored the fact that bad blocks are NOT dealt with by
the filesystem,... but by the operating system. Like I said: If your
filesystem is writing to bad blocks, then throw away your operating
system.

> Use a better compression algorithm, but
> that uses more CPU or which is visible in performance of the
> application. 

Not necessarily, just look at the bonnie++ tests. The tests show that
using compression (with todays hardware) leads to read speeds that are
some FOUR times the physical disk read rate,... 

but you ignore this as you have some strange anti-REISER religion.

Think about it,... read speeds that are some FOUR times the physical
disk read rate,... impossible without the use of compression (or
something similar).

> This can be offset by caching more, and being lazier with
> writebacks, but that hurts on-disk consistency, creates more memory
> pressure (more swapout/paging) and generally slows down other
> applications that aren't actually accessing the disk. Having a fast
> multi-core cpu and lots of memory helps a lot, but at some point what
> tradeoff did we just make, we saved a couple of dollars not having to
> buy a larger disk, but we spend considerably more on the more
> expensive cpu and memory.
> 

Wow, consider me STILL totally impressed by your AMAZING BIAS.
Would you like to tell me why you are SO BIASED against REISER4?

John.
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - The professional email service

Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel

[Richard Knutsson]

Wow, I'm impressed. Think you got the record on how many mails you 
referenced to in a reply... But dude, please calm down, the caps-lock is 
not the answer. You have got some rude answers and you have called them 
back on it + you have repeated the same statement several times, that is 
not the best way of convincing people.

I believe you picked up the "anti-Reiser religion"-phrase from previous 
rant-wars (otherwise, why does that "religion"-phrase always come up, 
and (almost) only when dealing with Reiser-fs), and yes, there has been 
some clashes caused by both sides, so please be careful when dealing 
with this matter.

Would you be willing to benchmark Reiser4 with some compressed 
binary-blob and show the time as well as the CPU-usage? And document how 
it is set up so it can be reproduced. After all, Windows is suppose to 
be more stable, maintained and cost-efficient then Linux, but they don't 
tell us how ;)

>> since it can't benefit as much from similarity between
>> files. So if that is the case and you really want to save diskspace you
>> almost have to look at read-only compressed filesystems such as cramfs,
>> squashfs, zisofs, cloop and various other variants in combination with
>> a unionfs overlay to get read/write functionality.
>>
>> But in the end everything is a tradeoff. You can save diskspace, but
>> increase the cost of corruption. 
>>     
>
> You deliberately ignored the fact that bad blocks are NOT dealt with by
> the filesystem,... but by the operating system. Like I said: If your
> filesystem is writing to bad blocks, then throw away your operating
> system.
>   
I may have missed something, but if my room-mate took my harddrive, 
screwed it open, wrote a love-letter on the disk with a pencil and then 
returned it (ok, there may be some more plausible reasons for 
corruption), is the OS really suppose to handle it? Yes, it should not 
assign any new data to those blocks but should it not also fall into the 
file-systems domain to be able to restore some/all data?


Just my 2c to the pond
Richard Knutsson

Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel

[johnrobertbanks]

On Mon, 09 Apr 2007 00:58:53 +0200, "Richard Knutsson"
<ricknu-0@student.ltu.se> said:
> Wow, I'm impressed. Think you got the record on how many mails you 
> referenced to in a reply... 

TWO actually. I guess you are easily impressed.

A simple cut and paste error.

> You have got some rude answers and you have called them back on it 

Yeah, I (fairly closely) mimicked their behavior to make a point.

> + you have repeated the same statement several times, that is 
> not the best way of convincing people.

I know you DON'T believe that, as you are about the tenth person to
repeat that "repeating stuff has no effect."

> I believe you picked up the "anti-Reiser religion"-phrase from previous 
> rant-wars (otherwise, why does that "religion"-phrase always come up, 
> and (almost) only when dealing with Reiser-fs), and yes, there has been 
> some clashes caused by both sides, so please be careful when dealing 
> with this matter.

NO. You people simply come across as zealots who work together, against
Reiser4.

Hence the term "anti-Reiser religion."

> Would you be willing to benchmark Reiser4 with some compressed 
> binary-blob and show the time as well as the CPU-usage? 

I might be. I don't really know how to set it all up.

Perhaps if you guided me through it.

> >
> > You deliberately ignored the fact that bad blocks are NOT dealt with by
> > the filesystem,... but by the operating system. Like I said: If your
> > filesystem is writing to bad blocks, then throw away your operating
> > system.
> >   

> I may have missed something, but if my room-mate took my harddrive, 
> screwed it open, wrote a love-letter on the disk with a pencil and then 
> returned it (ok, there may be some more plausible reasons for 
> corruption), is the OS really suppose to handle it? 

Yeah, I can't see how the OS could read the love-letter either.

But one thing is for sure. The FS ain't responsible for reading it.

> Yes, it should not 
> assign any new data to those blocks but should it not also fall into the 
> file-systems domain to be able to restore some/all data?

It's a tough ask of any FS. 

Microsoft's filesystem checker totally roasted all my data on an XP-box
last night. 

I had used ntfsresize to reduce the partition size and had a power
outage. 

Later, Windows booted, ran the filesystem checker, seemed OK. 

Next time I boot, all I get is Input/Output error.

> 
> Just my 2c to the pond
> Richard Knutsson
> 
Addin my 2c
John.
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - A no graphics, no pop-ups email service

Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel

[Willy Tarreau]

On Sun, Apr 08, 2007 at 10:14:18PM -0700, johnrobertbanks@fastmail.fm wrote:
> NO. You people simply come across as zealots who work together, against
> Reiser4.

Poor guy ! People are not against Reiser4, they are against the stupid and
irritating person who pollutes the lists always sending the same results
without any comment because he doesn't even understand the results. Just
like the kid on the beach "Look Ma, I found a soft shell!". "Leave it
overthere, it's a jellyfish !".  "I don't know what a jellyfish is, I will
take all those soft shells with me".

You keep saying people do not want to read you, but there is nothing to read.
In fact, you hope that people will comment on your results so that you will
finally understand them, but people keep saying there is nothing to read there.

When will you stop annoying people with your noisy toys ?

Oh, and please send your conspiracy claims somewhere else.

Willy

Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel

[Richard Knutsson]

johnrobertbanks@fastmail.fm wrote:
> On Mon, 09 Apr 2007 00:58:53 +0200, "Richard Knutsson"
> <ricknu-0@student.ltu.se> said:
>   
>> Wow, I'm impressed. Think you got the record on how many mails you 
>> referenced to in a reply... 
>>     
>
> TWO actually. I guess you are easily impressed.
>   
Oh, took it to be from 5-6 sources...
>> + you have repeated the same statement several times, that is 
>> not the best way of convincing people.
>>     
>
> I know you DON'T believe that, as you are about the tenth person to
> repeat that "repeating stuff has no effect."
>   
Why should we change our response to the same error? The only solution 
to this loop is when people stops answering you and you "lose".
>> I believe you picked up the "anti-Reiser religion"-phrase from previous 
>> rant-wars (otherwise, why does that "religion"-phrase always come up, 
>> and (almost) only when dealing with Reiser-fs), and yes, there has been 
>> some clashes caused by both sides, so please be careful when dealing 
>> with this matter.
>>     
>
> NO. You people simply come across as zealots who work together, against
> Reiser4.
>
> Hence the term "anti-Reiser religion."
>   
Please, don't address someone you meet for the first time as "you people"!
Yes, we do _work_ together, it is a community and as a community you 
have to follow the social rules agreed upon. Without all those 
pro-Reiser peoples who knew how to work with the rest, there would not 
be a ResierFS/Reiser3 in the kernel. Unfortunately, Hans is in this case 
his own worst enemy and has ruffed quite a few feathers over the time. I 
don't think you would like someone who tells you "if you do it my way, 
then you are doing it wrong"...

But personally, even if I find Hans a bit too strong-headed, he got some 
interesting design-ideas and the Reiser-filesystem is something I think 
many find interesting as a concept but not yet trust-worthy for their 
own machines.
>> Would you be willing to benchmark Reiser4 with some compressed 
>> binary-blob and show the time as well as the CPU-usage? 
>>     
>
> I might be. I don't really know how to set it all up.
>
> Perhaps if you guided me through it.
>   
Am not sure how much help I would be but from the responses to your 
benchmark-list, there seems to be many who could help you. But first I 
think you should set up a system to test on, and then after some tests 
and made the result public, there will (most likely) be people who ask 
you to test it in some specific way.
>> I may have missed something, but if my room-mate took my harddrive, 
>> screwed it open, wrote a love-letter on the disk with a pencil and then 
>> returned it (ok, there may be some more plausible reasons for 
>> corruption), is the OS really suppose to handle it? 
>>     
>
> Yeah, I can't see how the OS could read the love-letter either.
>
> But one thing is for sure. The FS ain't responsible for reading it.
>   
And no-one has asked the file-system to _read_ the disk, but to be 
designed to help restore the file-structure. This I have found to be the 
main-point people complains about.
It is like arguing against air-bags in a car. Of course the car should 
not be responsible for preventing accidents, but they are designed so 
_if_ it happens, you should not be totally screwed.
>> Yes, it should not 
>> assign any new data to those blocks but should it not also fall into the 
>> file-systems domain to be able to restore some/all data?
>>     
>
> It's a tough ask of any FS. 
>
> Microsoft's filesystem checker totally roasted all my data on an XP-box
> last night. 
>   
Sorry to hear that, but two wrongs does not make it right.

Richard Knutsson

Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel

[Christer Weinigel]

[linux-kernel trimmed from the Cc list]

johnrobertbanks@fastmail.fm writes:

> [lots of drivel with lots of capital letters elided]
> [a totally confused mess of responses to at least three different mails]

Wow.  You are absolutely amazing.  But, oh, well.

> By the way: Did I thank you "delightful" people for the "pleasant"
> welcome to the linux-kernel mailing list?

You are so welcome.  *grin*

> > So the two bonnie benchmarks with lzo and gzip are
> > totally meaningless for any real life usages.
> 
> YOU (yes, the one with no experience and next to NO knowledge on the
> subject) claim that because bonnie++ writes files that are mostly zeros,
> the results are meaningless. It should be mentioned that bonnie++ writes
> files that are mostly zero for all the filesystems compared. So the
> results are meaningful, contrary to would you claim.

Ok, lets take this really slowly so that you may understand.

Compression in the file system can be useful, there is no doubt about
that, but you have to be aware of the tradeoffs.

First of all, very few people have any use for storing files
consisting of just zeroes.  Trying to make any decision based on a
file systems ability to compress zeroes is just plain dumb.

Bonnie++ assumes that the data it writes will end up being written to
disk and not be compressed.  Right now it allocates a buffer which is
filled with zeroes and half a dozen bytes that the beginning of the
buffer are filled in with some random data.  So to make the bonnie
runs have any meaning on any compressed file systems you really want
to be able to choose what data it writes to disk.  If you modified
bonnie to do multiple test runs, one with zeroes, another one with
some easily compressed data such as a syslog, some not so easily
compressed data such as the contents of /bin/bash and some
uncompressible data such as from /dev/urandom that would be a lot
better benchmark.  

Then you have to be aware of the cost of that compression.  First of
all, it is going to use some CPU, so measuring the CPU load during the
benchmark is a good start.  Another thing with compression is that it
requires you to keep both some compressed and the uncompressed data in
RAM at the same time, so the memory pressure will increase.  This is
harder to measure and quantify.  Finally, since the CPU has to get
involved at compressing and decompressing the data doing this will
pull both the uncompressed and compressed data into the CPU cache and
may evict data from the cache that other processes would have found
useful.  This cache pollution is even harder to measure.  And none of
these costs make any difference for benchmarks run on a lightly loaded
system, but may make a difference in real life on any system that
tries to do something useful at the same time.

Then you have to consider your use cases.  As I said in my previous
mail, for my only space constrained disk, I store a lot of large flac
encoded CD images.  That data is basically uncompressible, so
compression buys me nothing, it just costs me a lot of extra CPU to
try to compress uncompressible data.  In addition, each CD image is a
quite large too, about 300MByte for a full size album, so whatever
savings I can get though the tail merging that Reiser4 (and Reiser3)
does is marginal for my use case.

Other use cases might have a lot to gain from compression.

> ALSO YOU IGNORE examples offered by others, on lkml, which contradict
> your assertion: FOR EXAMPLE:
> 
> > I see the same thing with my nightly scripts that do syslog
> > analysis, last year I trimmed 2 hours from the nightly run by
> > processing compressed files instead of uncompressed ones (after I
> > did this I configured it to compress the files as they are rolled,
> > but rolling every 5 min the compression takes <20 seconds, so the
> > compression is < 30 min)
> 
> David has said that compressing the logs takes
> 
> 24 x 12 x 20 secs = 5,760 secs = 1.6 hours of CPU time (over the day)
> 
> but he saves 2 hours of CPU time on the daily syslog analysis.
> 
> For a total (minimum) saving of 24 minutes.

So lets look at the syslog case then.  First of all, lets compress my
syslog with gzip:

    gzip -c /var/log/messages >whole.gz

    du -h /var/log/messages whole.gz
    532K    messages
    64K     whole.gz

Unfortunately, this compressed format isn't very efficient for some
use cases, lets say that I want to read the last 10 lines of the
syslog.  On a normal uncompressed file system I can just seek to the
end of the file, read the last block and get those 10 lines (or if the
last block didn't have 10 lines, I can try the block before that).
But with a compressed file, I have to uncompress the whole file and
throw away 531 kBytes at the begining of the file to do that.  So a
file system that wants to give the user efficient random access to a
files can't compress the whole file as done above.  It has to make
some tradeoffs to make random access practically usable.  Most
compressing file systems do that by splitting the file into fixed
chunks which are compressed independently of each other.  So lets
simulate that by splitting the file into 4k chunks and compressing
those separately and then combining them together:

    split -b 4096 /var/log/messages chunks
    gzip chunks*
    cat chunks*.gz >combined.gz

    du -h combined.gz
    120K    combined.gz

The reason for the loss of compression is that the chunk based
compression can't reuse knowledge across chunks.  It's possible to
mitigate this by increasing the chunks size:

    84K     combined-16368-chunk-size.gz
    72K     combined-65536-chunk-size.gz

once again that has a downside, the bigger the chunks, the more data
will be uncompressed unneccesarily when doing random accesses in the
file.

So the syslog example you are quoting above does not tell you how well
reiser4 will do on that specific use case.  A lot of the benefit in
David's example comes from knowing that he wants to process the file
as a whole and doesn't need random access.  So having application
specific knowledge and doing the compression outside of the file
system is what gives him that gain.

Of course, it may also be that the convenience of having transparent
compression in the file system is more worth more than the 50% benefit
in size from having to compress the syslog manually.  That depends.

> but he saves 2 hours of CPU time on the daily syslog analysis.

And no, he spends 1.6 hours of CPU time (maybe, some IO wait is
probaby included in that number) to save 2 hours of runtime (mostly IO
wait I assume).  So it seems that the disk is the bottle neck in his
case.  On a slightly different system CPU might be the bottle neck
because the same machine has to do a lot of processing at the same
time, so that it's better to skip the compression.  Once again, it
depends.

What I'm trying to get at here is that yes, compression can be useful,
but it is usese dependent, and it's impossible to catch all the
nuances of all use cases in one single number, especially an extremely
artificial number such as a bonnie++ run with files mostly consisting
of zeroes.

You can foam at the mouth and post the same meaningless, benchmark
figures over and over again and yell even louder, but that still don't
make them relevant.  It's not reiser4 that is the problem, but the way
you try to present reiser4 as the best thing since sliced bread.

To misquote Scott Adams: I'm not anti-Reiser4, I'm anti-idiot.  

  /Christer

-- 
"Just how much can I get away with and still go to heaven?"

Christer Weinigel <christer@weinigel.se>  http://www.weinigel.se

Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel

[David H. Lynch Jr]

Christer Weinigel wrote:
>
> To misquote Scott Adams: I'm not anti-Reiser4, I'm anti-idiot.  
>
>   /Christer
>   
    So can we ignore the benchmarks issue entirely and deal with the
issue of what might it actually take now that Hans is not pissing all
over everyone to get Reiser4 submitted with some hope of being accepted ?

    Your analysis was excellent. It make the case that just is there
will be a few cases where Reiser4 is absolutely compelling and a few
cases where it makes no sense at all that there are alot of cases in the
middle.

    I can keep coming up with examples where the attributes unique to
Reiser4 are an excellent idea, and you can just as easily come up with
cases where it is a bad idea.
   
    If the standard is that it must be all things to all people, then we
need to quit letting any filesystems in.
   
    Let's say John takes your advice and tweaks bonnie or builds
whatever benchmark test bed you want, and proves Reiser4 outperforms
everything in all cases (highly unlikely)
    does that mean it should get it ?
    What about if it is 10% slower in most normal cases ? Is that a good
enough reason for excluding it ?
    Is anyone ready to claim that there are no cases (beyond the
extremely rare case of compressing zero's) where Reiser4 may prove to be
the best choice ?
   
    Compression itself is almost a religious war subject. for some
people it is ALWAYS a bad idea, to others it is ALWAYS a good idea.
    In the real world it varies. Application level compression is
generally superior to filesystem compression - there is more knowledge
about the data leading to better choice of compression algorithms.
    and fortunately Linux provides one of the best environments for
incorporating application level compression.
    But generally is not the same as ALWAYS.
    While I raised a specific case I was aware of where my understanding
was compressed data could result in a net overall gain in performance -
network servers where compression decompression were handled at the
client, several other instances have been raised.
    Depending on the speed of the CPU, size of memory, size and speed of
cache, speed of disk, type of data, ....
    sometimes compressed data will not only save space but improve
performance - and sometimes it will be costly to performance.
    That does not make it a bad idea. We have several scheduling
algorithms, because one size does not fit all.

    Besides I doubt given the complexity of the issue that John or
anyone else can put together a benchmark that I can't poke holes in.
    There are so many cases that there is no such thing as the general
case. Even performance compressing zeros starts to almost look like a
rational measure when you start to calculate all the
    permutations in the data compression matrix.
   
   



-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel

[David H. Lynch Jr]

johnrobertbanks@fastmail.fm wrote:
> Christer Weinigel: Until YOU, have actually used the REISER4 filesystem
> yourself, I think YOU OWE IT to the people on the linux-kernel mailing
> list, to, AS YOU SAY, shut the fuck up. 
>
> Even reading up on the REISER4 filesystem would help. 
>
> Applying a little intelligence would undoubtedly help too.
>   
    John;
       I would like to see Reiser4 get into the Kernel too. But angry
rhetoric ... is not going to do it.
       If it was enough Hans did a much better job of insulting the
naysayers without resulting to vulgarity,
       And it got him pretty much nowhere.

       It is my personal oppinion that the reason Reiser4 did not reach
the kernel long ago
       was because Hans pissed all over way to many people.
   
       Making this about people will serve no one. We can all call each
other assholes, go home feeling vindicated,
       but Reiser4 will be no closer to the kernel, and any potential
that anyone might have benefited from that will just be
       pissed away.

    After that the statistics you are reporting are not TOTALLY meaningless.
    But they are also not compelling.

    It is not wise to be arguing benchmarks anyway.
    I hope that few on this list are not willing to conceed that Reiser4
is fast.
    Whether it is 10% slower than the best or 3 times as fast is
probably not relevant to whether it gets included.

    All the worst case scenarious of Reiser4 performance and behavior
should all still be sufficient to justify
    letting it in as Experimental.

    You can argue your benchmarks forever.
    Your benchmark has narrow applicability - pretty much every
benchmark will.
    And all you do is provoke a firestorm of debate over whether
compressing zeros means anything,
    whether compression is even a good idea, whether, ......
    The answer to ALL of these questions will be "it depends"
   
    I have dealt with huge data sets that were mostly zero's, a file
system that had a factor of 3 space savings and or performance benefit
     might have mattered alot there - but they are not the norm.
    I have dealt with instances where compression increased performance
rather than decreased it.
    Every "feature" of Reiser4 is an asset in some environments and a
liability in others.
    Whether it is suitable for most environments is nto going to be
established by a benchmark.

   
   
   
   

-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

Re: Reiser4. BEST FILESYSTEM EVER - Christer Weinigel

[johnrobertbanks]

On Sun, 08 Apr 2007 23:16:18 -0400, "David H. Lynch Jr"
<dhlii@comcast.net> said:
> johnrobertbanks@fastmail.fm wrote:
> > Christer Weinigel: Until YOU, have actually used the REISER4 filesystem
> > yourself, I think YOU OWE IT to the people on the linux-kernel mailing
> > list, to, AS YOU SAY, shut the fuck up. 
> >
> > Even reading up on the REISER4 filesystem would help. 
> >
> > Applying a little intelligence would undoubtedly help too.
> >   
>     John;
>        I would like to see Reiser4 get into the Kernel too. But angry
> rhetoric ... is not going to do it.
>

I WAS JUST POINTING OUT THE VERY ANGRY RHETORIC COMING FROM CERTAIN FOLK
ON THE MAILING LIST.

I think it really needs to be pointed out as some of those here are
extremely rude, bordering on unpleasant.

I only use angry rhetoric to make a point.

HAS THAT POINT BEEN MADE?
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - Same, same, but different…

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Reiser4. BEST FILESYSTEM EVER.

[Nate Diller]

On 08 Apr 2007 06:32:26 +0200, Christer Weinigel <christer@weinigel.se> wrote:
> johnrobertbanks@fastmail.fm writes:
>
> > Lennart. Tell me again that these results from
> >
> > http://linuxhelp.150m.com/resources/fs-benchmarks.htm and
> > http://m.domaindlx.com/LinuxHelp/resources/fs-benchmarks.htm
> >
> > are not of interest to you. I still don't understand why you have your
> > head in the sand.
>
> Oh, for fucks sake, stop sounding like a broken record.  You have
> repeated the same totally meaningless statistics more times than I
> care to count.  Please shut the fuck up.

wow, it's really amazing how reiser4 can still inspire flamewars so
easily when Hans isn't even around to antagonize people and escalate
things

> As you discovered yourself (even though you seem to fail to understand
> the significance of your discovery), bonnie writes files that consist
> of mostly zeroes.  If your normal use cases consist of creating a
> bunch of files containing zeroes, reiser4 with compression will do
> great.  Just lovely.  Except that nobody sane would store a lot of
> files containing zeroes, except an an excercize in mental
> masturbation.  So the two bonnie benchmarks with lzo and gzip are
> totally meaningless for any real life usages.

yeah, i sure wish Grev was still around running the benchmarks and
regression testing, cause I thought she came up with a good, QA
oriented mix of real benchmarks.  aside from a number of streaming
video benchmarks i did, those were the only results i actually trusted
to compare reiser4 with other systems.  I know Ted doesn't like the
Mongo suite, cause it focuses on small files and shows the common
weakness of block-aligned storage ... personally i thought it was
great for its primary purpose, making sure reiser4 was optimized for
its target workload.  i also recall that the distribution of small
files to large ones in mongo was pulled from some paper out of CMU,
but i can't find the reference to that study right now.

> As for the amount of disk needed to store three kernel trees, the
> figures you quote show that Reiser4 does tail combining where the tail
> of multiple files are stored in one disk block.  A nice trick that
> seems save you about 15% disk space compared to ext3.  Now you have to
> realise what that means, it means that if the disk block containing
> those tails (or any metadata pointing at that block) gets corrupted,
> instead of just losing one disk block for one file, you will have lost
> the tail for all the files sharing that disk block.  Depending on your
> personal prioritites, saving 15% of the space may be worth the risk to
> you, or maybe not.  Personally, for the only disk I'm short on space
> on, I mostly store flac encoded images of my CD collection, and saving
> 2kByte out of every 300MByte disk simply doesn't make any difference,
> and I much prefer a stable file system that I can trust not to lose my
> data.  You might make different choices.

well, it turns out that reiser4 does things a little differently,
since tail packing has bad performance effects (i always turn it off
on my reiserfs partitions).  Reiser4 guarantees a file will be stored
contiguously if it is below a certain size (20K?), and instead stores
the whole file unaligned, so that many files can be packed together
without slack space.  this gives the best of both worlds
performance-wise, at the expense of some complicated flush code to
pack everything together in the tree before it gets written.  that
combined with the fine-grained locking scheme (per-node -- reiserfs
just has a global lock) is the primary reason the code is so
convoluted ... not poor coding.

> The same goes for just about every feature that you tout, it has its
> advantages, and it has its disadvantages.  Doing compression on data
> is great if the data you store is compressible, and sucks if it isn't.
> Doing compression on each disk block and then packing multiple
> compressed blocks into each physical disk block will probably save
> some space if the data is compressible, but at the same time it means
> that you will spend a lot of CPU (and cache footprint) compressing and
> uncompressing that data.  On a single user system where the CPU is
> mostly idle it might not make much of a difference, on a heavily
> loaded multiuser system it might do.

my understanding of the code is that it uses a heuristic to decide if
a file is already compressed, so that the system doesn't waste time on
them and simply writes them out directly.  there may also be a way to
turn it off for certain classes of files, this would be most useful
for executables and the like that are frequently mmap()ed and we care
more about page-alignment than read bandwidth or data density.
edward?

> Logs can be compressed quite well using a block based compression
> scheme, but the logs can be compressed even better by doing
> compression on the whole file with gzip.  So what's the best choice,
> to do transparent compression on the fly giving ok compression or
> teaching the userspace tools to do compression of old logs and get
> really good compression?  Or maybe disk space really isn't that
> important anyway and the best thing is to just leave the logs
> uncompressed.

i guess the idea with reiser4's compression (encryption and
compression, actually) is that you can get the feature for files you
care about without having to use *extra* CPU time, it only does the
work at flush time so that you can take advantage of cache effects for
files that see lots of modifications.  ATM i doubt this works well
though, cause you'd have to manually increase dirty_background_ratio
to keep things from continually flushing and using background CPU

NATE

Re: Reiser4. BEST FILESYSTEM EVER.

[David H. Lynch Jr]

Lennart Sorensen wrote:
> Nope, if they are accurate and they have something to do with your
> particular usage and applications, then they are relevant.  But it
> requires both to make them relevant.  Although it may be possible for a
> benchmark to be relevant even if not particularly accurate.
    I do not care about the benchmarks or the compression.
    They are pointless anyway, Reiser4 is not and can not be all things
to all people.
    I do not care how many angels can dance on the head of a pin.
    Whether it is most suitable for broad use or only narrow cases
remains to seen.
   
    Hans Reiser is now in jail awaiting trial for the murder of his wife
- I doubt he is going to be posting and stirring things up anytime soon
    Those who need to assurance of their moral superiority can take that
as proof if they need.

    I do care about getting Reiser4 into the kernel so that it can
actually get a real test,
    and frankly do not see any compelling reason that should not happen.

    Their may be better filesystems currently in the kernel - there
certainly are crappier ones.
      
    I have used ReiserFs extensively, and on the whole been extremely
happy with it.
    I have been slightly singed - a long time ago, but it is certainly
not the only file system that has caused me grief.
    I can not think of an existing Linux (or other) filesystem that is
the be all or end all.

    I have no time for either the zealots that think Reiser4 is the next
best thing to sliced bread
    nor those who are going to find some case to demonstrate it is total
crap - no matter what.

    Most people are wise not to use Reiser4 - or any other new
filesystem until it has been
    rigorously tested. I have no bone to pick with that.
   
    I will try it once it is available in standard kernel sources,
    but I am juggling enough patches as it is, and I am not the patch
wizard that some of the rest of you are.
    If I get burned - so be it. It won't be the first time.
   
    This game has gone on sufficiently long that Reiser4 has zero hope
of becoming more than a footnote
    used primarily by oddballs and misfits.

    But there is still some hope that a slightly wider audience might
result in numerous positive benefits
    to other filesystems.

    I have read many of the critiques - but even if they are valid, I
thought Linux was a place where
    competing ideas were tested in use not rhetoric.
   





   






-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

Re: Reiser4. BEST FILESYSTEM EVER.

[Jeff Garzik]

David H. Lynch Jr wrote:
>     I do care about getting Reiser4 into the kernel so that it can
> actually get a real test,
>     and frankly do not see any compelling reason that should not happen.


If the compelling reason is that it needs a test, I'd say its not ready.

	Jeff

Re: Reiser4. BEST FILESYSTEM EVER.

[David H. Lynch Jr]

Jeff Garzik wrote:
>
> If the compelling reason is that it needs a test, I'd say its not ready.

    Can you please elaborate ? I am not sure I understand what you are
arguing ?

    Despite his substantially less than polite rhetoric, I have read
Hans's post from months if not years ago.
    Aside from the pissing contests - which where not entirely one
sided, I actually beleive that Hans made a reasonable case
    that Reiser4 had gone about as far as it could reasonably go with
regard to testing, robustness, ... without the broader base of
    use that even an experimental filesystem in distribution tree would get.

    I for one would atleast play with it if it were in the distribution
tree.
    As far as I could tell pretty much everything else that was demanded
Hans eventually caved and provided - albeit with much pissing and moaning,
    and holy than thou rhetoric.

    The argument that anything that needs testing can't get into the
distribution tree's is specious. There is alot of poorly tested crap in
the distribution trees.
    But separately, there is the issue of scale. Namesys claims that
they have no currently know bugs, faults ... - with their base of
internal and external users.
    I would fully expect new failures to crop up with any filesystem,
driver, ... moving  up an order of magnitude in users.
 
    Are you going to subject all filesystems and drivers to the same
high standards you are placing on Reiser4 ? If so then we need to strip
the distribution tree now.
   
    I am not looking to defend Hans - he is likely to be in jail and no
longer a factor for a long time. Nor am I looking to make or support
claims for Reiser4.
    But I am asking - why we can not get past the bad blood, rhetoric,
and zealotry -which to my eyes has not been all one sided.
    I am NOT looking for a technical explanation of all the relative
merits and demerits of Reiser4.
    I do not care for arguments about whether it compresses 0's well, or
that tail combining is a bad thing. They may have merit, but there is
not a filesystem
    that is going to be all things to all people. Whether Reiser4 is a
small niche filesystem or a significant general use one, is a decision
that should be reached
    by its performance in practice, not it rhetoric. Regardless, even as
a niche filesystem, I beleive at this point it merits inclusion.
   
   




-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

Re: Reiser4. BEST FILESYSTEM EVER.

[Jeff Garzik]

David H. Lynch Jr wrote:
> Jeff Garzik wrote:
>> If the compelling reason is that it needs a test, I'd say its not ready.
> 
>     Can you please elaborate ? I am not sure I understand what you are
> arguing ?
> 
>     Despite his substantially less than polite rhetoric, I have read
> Hans's post from months if not years ago.
>     Aside from the pissing contests - which where not entirely one
> sided, I actually beleive that Hans made a reasonable case
>     that Reiser4 had gone about as far as it could reasonably go with
> regard to testing, robustness, ... without the broader base of
>     use that even an experimental filesystem in distribution tree would get.
> 
>     I for one would atleast play with it if it were in the distribution
> tree.
>     As far as I could tell pretty much everything else that was demanded
> Hans eventually caved and provided - albeit with much pissing and moaning,
>     and holy than thou rhetoric.
> 
>     The argument that anything that needs testing can't get into the
> distribution tree's is specious. There is alot of poorly tested crap in
> the distribution trees.

I'm arguing against circular logic:  the claim that one cannot determine 
reiser4's true usefulness unless its in the tree.

The better method is to get a distro to add reiser4, _then_ if it proves 
worthy add it to the kernel tree.

Not the other way around.

	Jeff

Re: Reiser4. BEST FILESYSTEM EVER.

[David H. Lynch Jr]

Jeff Garzik wrote:
> David H. Lynch Jr wrote:
>
> I'm arguing against circular logic:  the claim that one cannot
> determine reiser4's true usefulness unless its in the tree.
>
> The better method is to get a distro to add reiser4, _then_ if it
> proves worthy add it to the kernel tree.
>
> Not the other way around. 
    And is that how other filesystems made it into the tree ?
    I am actually trying to be rational about this.
    But if my logic is circular then the above is even more so.
    I am aware of distributions that would incorporate Reiser4 if it was
in the tree, but not really aware of much that works the other way around.
    Besides after eliminating say the top 4 distributions, how is it
that getting included as a burried alternate in an obscure distribution
is going to change
    the total number of existing Reiser4 users to a meaningful extent.
It would be my guess that Reiser4 already has more users than any
filesystem has ever had prior
    to inclusion in the kernel.
    Aside from a few instances where distributors are also contributors,
and a few instances where specific drivers such as gaming video drivers
are in
    extremely high demand, how frequently do drivers go into
distributions before they go into the tree ?
    Is that supposed to be the standard for all drivers ?
    I regularly see drivers with very little in the way of testing go
straight nearly straight into the tree - without even getting tagged as
experimental.
   
    And what is CONFIG_EXERIMENTAL supposed to be for ?

      All I am asking is that now that Hans is not pissing all over
people that the Reiser4 Filesystem get exactly the same treatment that
    pretty much every other filesystem already in or slated for
inclusion in the kernel has received.

    I am sure that their are others who may have a more accurate
perception of history. But my recollection is that the only filesystem
driver that went through a distribution
    prior to getting into the Kernel was ReiserFS.

    I am not even asking that it get accepted exactly asis - though it
is my perception that despite Hans's pissing and moaning in the end he
made most every change that was asked.
    Regardless, now that you don't have Hans to deal with anymore.
    Give Namesys an honest and reasonable set of requirements to get a
decent review - the same shot any other filesystem would get.
   




   
   
   





-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

Re: Reiser4. BEST FILESYSTEM EVER.

[Jeff Garzik]

David H. Lynch Jr wrote:
> Jeff Garzik wrote:
>> David H. Lynch Jr wrote:
>>
>> I'm arguing against circular logic:  the claim that one cannot
>> determine reiser4's true usefulness unless its in the tree.
>>
>> The better method is to get a distro to add reiser4, _then_ if it
>> proves worthy add it to the kernel tree.
>>
>> Not the other way around. 

>     And is that how other filesystems made it into the tree ?

In the case of most major filesystems, yes.  Distros are a proving 
ground for new stuff, not the upstream kernel.


>     I regularly see drivers with very little in the way of testing go
> straight nearly straight into the tree - without even getting tagged as
> experimental.

Hardware drivers are vastly different from filesystem drivers.

	Jeff

Re: COMPILING AND CONFIGURING A NEW KERNEL.

[johnrobertbanks]

Just correcting some errors and typos. 

Wouldn't want you to say that the linux kernel mailing list gave you
incorrect info.

COMPILING AND CONFIGURING A NEW KERNEL.

Download a recent kernel from http://www.kernel.org/
I will use the kernel linux-2.6.20.tar.bz2

You will have to change details of the following to suit your purposes.

Save it in /usr/src/
# mv linux-2.6.20.tar.bz2 /usr/src/

Unzip the kernel package
# tar -jxf linux-2.6.20.tar.bz2

Copy the original kernel configuration file (that came with your distro)
to .config
# cp /boot/config-2.6.13-15-default /usr/src/linux-2.6.20/.config

Change to the new kernel source directory
# cd /usr/src/linux-2.6.20/

Look at the available kernel building options
# make help

Run oldconfig to update the original kernel configuration to a current
configuration
# make oldconfig

Use menuconfig (or xconfig or gconfig) to make any further changes
# make menuconfig

YOU SHOULD compile all the drivers necessary to boot your system, into
the kernel (ie, such drivers should not be built as modules).

This way you will NOT need an initrd file.

Use rpm-pkg to create a Red Hat RPM kernel package.
# make rpm-pkg

When built, the RPM package is put in
/usr/src/packages/RPMS/*your*architecture*

# cd /usr/src/packages/RPMS/x86_64

Install the package (you may have to un-install previous installs)
# rpm -i kernel-2.6.20-1.x86_64.rpm

Use deb-pkg to create a Debian .deb kernel package.
# make deb-pkg

When built, the .deb package is put in /usr/src/
# cd /usr/src/

Install the package (you may have to un-install previous installs)
# dpkg --install linux-2.6.20_2.6.20_amd64.deb

If you were unable to determine which drivers you need (to boot), then
you will need an initrd file. To build it use the command
# mkinitrd -o /boot/initrd-2.6.20

IF YOU ARE CUSTOMIZING YOUR KERNEL, YOU SHOULD PUT IN THE EFFORT TO
BUILD A KERNEL THAT DOES NOT NEED AN INITRD FILE.

It is possible that deb-pkg and rpm-pkg take care of creating the initrd
automatically.

I have always compiled in the important drivers, so I do not know.

Does any caring person here know the answer to this question?

------------------------------------------------------

Now you need to configure your kernel. Using GRUB you need to change the
menu.lst file.

# emacs /boot/grub/menu.lst &

The grub entry that you presently boot with, will look something like:

###Don't change this comment - YaST2 identifier: Original name: linux###
title SUSE LINUX 10.0
    root (hd0,2)
    kernel /boot/vmlinuz-2.6.13-15-default root=/dev/hda3
    resume=/dev/hda5 vga=0x317 video=vesafb:nomtrr splash=silent
    initrd /boot/initrd

Do NOT delete the old boot entry, so you can boot it, if things go wrong
with the new kernel.

Cut a copy of it and paste it above the original. Then adjust the copy
for the new kernel.

###Don't change this comment - YaST2 identifier: Original name: linux###
title MY NEW KERNEL
    root (hd0,2)
    kernel /boot/linux-2.6.20 root=/dev/hda3 resume=/dev/hda5 vga=0x317
    video=vesafb:nomtrr splash=silent

Of course, you don't need a initrd entry as you have compiled in all the
vital drivers,... right?
If you could not determine the vital drivers and needed to build an
initrd file, then you need an entry, like

    initrd /boot/initrd-2.6.20

------------------------------------------------------

If your new kernel is destined to have the same name as the old one, you
need to do something about it (unless you do not mind the old one being
overwritten).

Use your favorite text editor to change the top level Makefile
# emacs /usr/src/linux-2.6.20/Makefile &

change the line 
EXTRAVERSION
to 
EXTRAVERSION = something

This will change the name of the new kernel to linux-2.6.20-something

Your /boot/grub/menu.lst entry will now look something like:

###Don't change this comment - YaST2 identifier: Original name: linux###
title MY NEW KERNEL
    root (hd0,2)
    kernel /boot/linux-2.6.20-something root=/dev/hda3 resume=/dev/hda5
    vga=0x317 video=vesafb:nomtrr splash=silent

and perhaps an entry

    initrd /boot/initrd-2.6.20-something

------------------------------------------------------

Now reboot and choose the "MY NEW KERNEL" entry from the GRUB boot menu,
and see how you went.

-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - Send your email first class

Re: COMPILING AND CONFIGURING A NEW KERNEL.

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

On Sat, 07 Apr 2007 00:45:32 PDT, johnrobertbanks@fastmail.fm said:

> Use rpm-pkg to create a Red Hat RPM kernel package.
> # make rpm-pkg
> 
> When built, the RPM package is put in
> /usr/src/packages/RPMS/*your*architecture*
> 
> # cd /usr/src/packages/RPMS/x86_64
> 
> Install the package (you may have to un-install previous installs)
> # rpm -i kernel-2.6.20-1.x86_64.rpm

It is *highly* recommended that you change the kernel identifier at
least slightly, so that you can install '2.6.20-1.local' without overlaying
the vendor-supplied 2.6.20-1 kernel.  Among other things, this lets you
boot back to the equivalent code level in the vendor kernel, so you can figure
out if it's your .config file that's broken, or if you hit a bug upggrading
from 2.6.19-10 to 2.6.20-1.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: COMPILING AND CONFIGURING A NEW KERNEL.

[johnrobertbanks]

> It is *highly* recommended that you change the kernel identifier at
> least slightly, so that you can install '2.6.20-1.local' without
> overlaying
> the vendor-supplied 2.6.20-1 kernel.  Among other things, this lets you
> boot back to the equivalent code level in the vendor kernel, 
> so you can figure
> out if it's your .config file that's broken, or if you hit a bug
> upggrading from 2.6.19-10 to 2.6.20-1.

I agree. I think your advice is *highly* recommended.

I had this problem once after forcing an upgrade, which removed the
working kernel. I just booted the kernel and stuff, from another
partition.

John.
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - The way an email service should be

Re: Reiser4. BEST FILESYSTEM EVER.

[Dale Amon]

Jan does have a point about bad blocks. A couple years ago
I had a relatively new disk start to go bad on random blocks.
I detected it fairly quickly but did have some data loss.

All the compressed archives which were hit were near
total losses; most other files were at least partially
recoverable.

It is not a matter of your operating system writing
to bad blocks. It is a matter of what happens when the
blocks on which your data sit go bad underneath you.

This issue has also been discussed by people working
with revision control system. If you are archiving
data, how do you know you if your data is still good
unless you actually need it? If you do not know it
is bad, you may well get rid of good copies thinking
you do not need the extras... it does happen.

I would be quite hesitant to go with on disk compression
unless damage was limited to only the bad bits or blocks
and did not propagate through the rest of the file.

Perhaps if everyone used hardware RAID and the RAID
automatically detected a difference due to trashed
data on one disk and flagged the admin with a warning...

BTW: I'm a CMU Alum, so who are you working with Jan?

Re: Reiser4. BEST FILESYSTEM EVER.

[johnrobertbanks]

On Sat, 7 Apr 2007 13:59:14 +0100, "Dale Amon" <amon@vnl.com> said:
> Jan does have a point about bad blocks. A couple years ago
> I had a relatively new disk start to go bad on random blocks.
> I detected it fairly quickly but did have some data loss.
> 
> All the compressed archives which were hit were near
> total losses; most other files were at least partially
> recoverable.

As you know, there is not substitute for backups. What if the disk had
totally crashed and scratched GBs of your data.

And did you ever trust those (non-compressed) executables that you saved
after recovering them from corruption?

Of course not. No one would. The fact that they were not compressed did
not save them.

You are really arguing for backups, not for one filesystem or another.

Besides, Jan claimed that corruption due to bad blocks propagates to
MULTIPLE files because of the compression in the file system. You are
arguing something different.

> It is not a matter of your operating system writing
> to bad blocks. It is a matter of what happens when the
> blocks on which your data sit go bad underneath you.
>
> This issue has also been discussed by people working
> with revision control system. If you are archiving
> data, how do you know you if your data is still good
> unless you actually need it? If you do not know it
> is bad, you may well get rid of good copies thinking
> you do not need the extras... it does happen.
> 
> I would be quite hesitant to go with on disk compression
> unless damage was limited to only the bad bits or blocks
> and did not propagate through the rest of the file.

You don't really mean that. Most backup uses compression (which
propagates errors through the rest of the file).

> Perhaps if everyone used hardware RAID and the RAID
> automatically detected a difference due to trashed
> data on one disk and flagged the admin with a warning...
> 
> BTW: I'm a CMU Alum, so who are you working with John?

I retired quite young.
-- 
  
  johnrobertbanks@fastmail.fm

-- 
http://www.fastmail.fm - Send your email first class