* Re: reiser4 plugins
2004-08-26 13:40 ` Christoph Hellwig
2004-08-26 13:58 ` Christophe Saout
@ 2004-08-26 23:54 ` Hans Reiser
1 sibling, 0 replies; 18+ messages in thread
From: Hans Reiser @ 2004-08-26 23:54 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Christophe Saout, Andrew Morton, linux-fsdevel, linux-kernel, flx,
torvalds, reiserfs-list
Christoph Hellwig wrote:
>
> The problem is not that it
>doesn't work but that it's really hard to maintain.
>
You really like to talk about what you know nothing of, yes?
Reiser4 is FAR easier to maintain than V3. Ask anyone on reiserfs-dev.
It was designed to reduce our software maintenance costs.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-26 13:58 ` Christophe Saout
@ 2004-08-26 23:55 ` Hans Reiser
2004-08-27 12:04 ` Nikita Danilov
0 siblings, 1 reply; 18+ messages in thread
From: Hans Reiser @ 2004-08-26 23:55 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Christophe Saout, Andrew Morton, linux-fsdevel, linux-kernel, flx,
torvalds, reiserfs-list
Christophe Saout wrote:
>
>
>I don't know, ask Hans. How could the VFS know it a filesystem wants to
>do something specific with a file that is completely transparent to the
>VFS?
>
>
>
To know what method to use, you must determine the pluginid, and then
find the method within that plugin for that vfs operation.
As for overhead, well, who eats whose dust in the benchmarks....?
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-26 23:55 ` reiser4 plugins Hans Reiser
@ 2004-08-27 12:04 ` Nikita Danilov
2004-08-27 18:15 ` Hans Reiser
0 siblings, 1 reply; 18+ messages in thread
From: Nikita Danilov @ 2004-08-27 12:04 UTC (permalink / raw)
To: Hans Reiser
Cc: Christoph Hellwig, Christophe Saout, Andrew Morton, linux-fsdevel,
linux-kernel, flx, torvalds, reiserfs-list
Hans Reiser writes:
> Christophe Saout wrote:
>
> >
> >
> >I don't know, ask Hans. How could the VFS know it a filesystem wants to
> >do something specific with a file that is completely transparent to the
> >VFS?
> >
> >
> >
> To know what method to use, you must determine the pluginid, and then
> find the method within that plugin for that vfs operation.
>
> As for overhead, well, who eats whose dust in the benchmarks....?
Whoever sponsors the benchmark usually wins. Had you forgotten that
mongo setup used by http://www.namesys.com/benchmarks.html was specially
`tuned' to reach peak reiser4 performance? Remember why you decided to
turn OVERWRITE and MODIFY phases off? People on #reiser4 report 90
_second_ stalls with reiser4 under high io loads (large atom is
obviously being flushed and everyone waits on it...). In my opinion, it
is such things that are of utmost importance for real reiser4
acceptance, not how to name `metas' sub-directory.
Nikita.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-27 12:04 ` Nikita Danilov
@ 2004-08-27 18:15 ` Hans Reiser
2004-08-27 18:55 ` Nikita Danilov
2004-08-27 22:29 ` Steve Bergman
0 siblings, 2 replies; 18+ messages in thread
From: Hans Reiser @ 2004-08-27 18:15 UTC (permalink / raw)
To: Nikita Danilov
Cc: Christoph Hellwig, Christophe Saout, Andrew Morton, linux-fsdevel,
linux-kernel, flx, torvalds, reiserfs-list
Nikita Danilov wrote:
>Hans Reiser writes:
> > Christophe Saout wrote:
> >
> > >
> > >
> > >I don't know, ask Hans. How could the VFS know it a filesystem wants to
> > >do something specific with a file that is completely transparent to the
> > >VFS?
> > >
> > >
> > >
> > To know what method to use, you must determine the pluginid, and then
> > find the method within that plugin for that vfs operation.
> >
> > As for overhead, well, who eats whose dust in the benchmarks....?
>
>Whoever sponsors the benchmark usually wins. Had you forgotten that
>mongo setup used by http://www.namesys.com/benchmarks.html was specially
>`tuned' to reach peak reiser4 performance? Remember why you decided to
>turn OVERWRITE and MODIFY phases off? People on #reiser4 report 90
>_second_ stalls with reiser4 under high io loads (large atom is
>obviously being flushed and everyone waits on it...). In my opinion, it
>is such things that are of utmost importance for real reiser4
>acceptance, not how to name `metas' sub-directory.
>
>Nikita.
>
>
>
>
If you ask real users, they say that reiser4 is fast, and their
experience matches our benchmark. You can criticize the benchmark if
you want, but then you should run your own and publish it.
There are still plenty of things needing fixing though, and you are
right that they are more important than the pretext being attempted for
keeping us out of the kernel. Probably you could supply them with many
superior pretexts.;-)
The 90 second stalls, those should be fixed by debugging copy on capture
and turning it on, yes? You can hardly claim I failed to push for the
completion of copy on capture.... and of course the reason it is not
done is simply that there is too much to do.... I (sincerely) hope vs
enjoys his vacation, the whole team is working too hard. I am doing 7
day weeks now to try to make payroll for us and keep darpa happy at the
same time.
fsync performance needs fixing badly, and Chris is right in his
diagnosis of what we can do to fix it.
Hans
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-27 18:15 ` Hans Reiser
@ 2004-08-27 18:55 ` Nikita Danilov
2004-08-28 9:53 ` Hans Reiser
2004-08-29 11:17 ` Alex Zarochentsev
2004-08-27 22:29 ` Steve Bergman
1 sibling, 2 replies; 18+ messages in thread
From: Nikita Danilov @ 2004-08-27 18:55 UTC (permalink / raw)
To: Hans Reiser
Cc: Christoph Hellwig, Christophe Saout, Andrew Morton, linux-fsdevel,
linux-kernel, flx, torvalds, reiserfs-list
Hans Reiser writes:
> Nikita Danilov wrote:
>
> >Hans Reiser writes:
> > > Christophe Saout wrote:
> > >
> > > >
> > > >
> > > >I don't know, ask Hans. How could the VFS know it a filesystem wants to
> > > >do something specific with a file that is completely transparent to the
> > > >VFS?
> > > >
> > > >
> > > >
> > > To know what method to use, you must determine the pluginid, and then
> > > find the method within that plugin for that vfs operation.
> > >
> > > As for overhead, well, who eats whose dust in the benchmarks....?
> >
> >Whoever sponsors the benchmark usually wins. Had you forgotten that
> >mongo setup used by http://www.namesys.com/benchmarks.html was specially
> >`tuned' to reach peak reiser4 performance? Remember why you decided to
> >turn OVERWRITE and MODIFY phases off? People on #reiser4 report 90
> >_second_ stalls with reiser4 under high io loads (large atom is
> >obviously being flushed and everyone waits on it...). In my opinion, it
> >is such things that are of utmost importance for real reiser4
> >acceptance, not how to name `metas' sub-directory.
> >
> >Nikita.
> >
> >
> >
> >
> If you ask real users, they say that reiser4 is fast, and their
> experience matches our benchmark. You can criticize the benchmark if
They experience 90 second stalls. And please, do not tell me how fast
reiser4 is, I spent a lot of time working with it, and I know very well
when it's fast, and when it's deadly slow.
> you want, but then you should run your own and publish it.
I did, after which you told me to turn OVERWRITE and MODIFY phases off,
beause performance was horrible.
I not criticizing mongo benchmark per se. I think that it is
fundamentally wrong to use results that were deliberatly manipulated to
get best appearance to reiser4 (by omitting work-loads where it performs
poorly) as an argument. It's not clear who will, according to your
colorful expression, `eat dust' as a result of this. Or do you think
that users never overwrite of modify files in real life?
> The 90 second stalls, those should be fixed by debugging copy on capture
> and turning it on, yes? You can hardly claim I failed to push for the
No. I described so many times to you, why COC is ineffectual here.
> Hans
Nikita.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-27 18:15 ` Hans Reiser
2004-08-27 18:55 ` Nikita Danilov
@ 2004-08-27 22:29 ` Steve Bergman
2004-08-28 6:54 ` Hans Reiser
2004-08-29 11:42 ` Alex Zarochentsev
1 sibling, 2 replies; 18+ messages in thread
From: Steve Bergman @ 2004-08-27 22:29 UTC (permalink / raw)
To: Hans Reiser
Cc: Nikita Danilov, Christoph Hellwig, Christophe Saout,
Andrew Morton, linux-fsdevel, linux-kernel, flx, torvalds,
reiserfs
On Fri, 2004-08-27 at 11:15 -0700, Hans Reiser wrote:
> >
> If you ask real users, they say that reiser4 is fast, and their
> experience matches our benchmark. You can criticize the benchmark if
> you want, but then you should run your own and publish it.
>
As a "real" desktop user who just converted all his partitions from ext3
to reiser4, I have not, as yet, noticed any startling performance
increase. Being slightly slightly irked to hear that the benchmark
numbers that have been paraded around on Slashdot and the internet in
general, at ext3's expense, have had reiser4's "bad" results surgically
extracted, I am running my own benchmarks to get the real story on
reiser4/ext3 mongo performance on my, rather average, desktop hardware.
I am using the latest Mongo on FC/rawhide and the 2.6.8.1-mm4 kernel.
Unfortunately, I get an error from mongo.pl that "Done" is not a numeric
argument at line 439.
I have done this to fix it:
--- mongo.pl 2004-08-27 17:07:01.681723313 -0500
+++ mongo_fixed.pl 2004-08-27 17:06:51.369306735 -0500
@@ -429,8 +429,8 @@
if ( -e ${ERR_FILE}) {
&DIE ("\nEXITED WITH FAIL\n");
}
- my $real = (split ' ', $time_output[0])[1];
- my $cpu = (split ' ', $time_output[2])[1];
+ my $real = (split ' ', $time_output[1])[1];
+ my $cpu = (split ' ', $time_output[3])[1];
unless ( $real =~ /\s*\d+/ && $cpu =~ /\s*\d+/) {
LOG "@time_output";
What it gets me is the "real" line of the "time" output for "STAT
REAL_TIME" and the "sys" line of the "time" output for "STAT CPU_TIME".
i.e. only system time is counted. I believe this was the intent of the
original code, but want to verify before continuing.
Thanks,
Steve Bergman
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-27 22:29 ` Steve Bergman
@ 2004-08-28 6:54 ` Hans Reiser
2004-08-29 11:42 ` Alex Zarochentsev
1 sibling, 0 replies; 18+ messages in thread
From: Hans Reiser @ 2004-08-28 6:54 UTC (permalink / raw)
To: Steve Bergman
Cc: Nikita Danilov, Christoph Hellwig, Christophe Saout,
Andrew Morton, linux-fsdevel, linux-kernel, flx, torvalds,
reiserfs, Alexander Zarochentcev
Steve Bergman wrote:
>On Fri, 2004-08-27 at 11:15 -0700, Hans Reiser wrote:
>
>
>>If you ask real users, they say that reiser4 is fast, and their
>>experience matches our benchmark. You can criticize the benchmark if
>>you want, but then you should run your own and publish it.
>>
>>
>>
>
>
>As a "real" desktop user who just converted all his partitions from ext3
>to reiser4, I have not, as yet, noticed any startling performance
>increase. Being slightly slightly irked to hear that the benchmark
>numbers that have been paraded around on Slashdot and the internet in
>general, at ext3's expense, have had reiser4's "bad" results surgically
>extracted, I am running my own benchmarks to get the real story on
>reiser4/ext3 mongo performance on my, rather average, desktop hardware.
>
>I am using the latest Mongo on FC/rawhide and the 2.6.8.1-mm4 kernel.
>
>Unfortunately, I get an error from mongo.pl that "Done" is not a numeric
>argument at line 439.
>
>I have done this to fix it:
>
>
>--- mongo.pl 2004-08-27 17:07:01.681723313 -0500
>+++ mongo_fixed.pl 2004-08-27 17:06:51.369306735 -0500
>@@ -429,8 +429,8 @@
> if ( -e ${ERR_FILE}) {
> &DIE ("\nEXITED WITH FAIL\n");
> }
>- my $real = (split ' ', $time_output[0])[1];
>- my $cpu = (split ' ', $time_output[2])[1];
>+ my $real = (split ' ', $time_output[1])[1];
>+ my $cpu = (split ' ', $time_output[3])[1];
>
> unless ( $real =~ /\s*\d+/ && $cpu =~ /\s*\d+/) {
> LOG "@time_output";
>
>
>What it gets me is the "real" line of the "time" output for "STAT
>REAL_TIME" and the "sys" line of the "time" output for "STAT CPU_TIME".
>i.e. only system time is counted. I believe this was the intent of the
>original code, but want to verify before continuing.
>
>Thanks,
>Steve Bergman
>
>
>
>
>
I didn't write this (more precisely, it only vaguely resembles what I
wrote in 1996). Are you saying that it reports system time as real
time? If yes, then it is an error, we need to go remove a bunch of
numbers from our benchmarks, and thanks for finding it.
Zam, please comment.
Hans
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-27 18:55 ` Nikita Danilov
@ 2004-08-28 9:53 ` Hans Reiser
2004-08-28 13:47 ` Nikita Danilov
2004-08-29 11:17 ` Alex Zarochentsev
1 sibling, 1 reply; 18+ messages in thread
From: Hans Reiser @ 2004-08-28 9:53 UTC (permalink / raw)
To: Nikita Danilov
Cc: Christoph Hellwig, Christophe Saout, Andrew Morton, linux-fsdevel,
linux-kernel, flx, torvalds, reiserfs-list,
Alexander Zarochentcev
Nikita Danilov wrote:
> > >Whoever sponsors the benchmark usually wins. Had you forgotten that
> > >mongo setup used by http://www.namesys.com/benchmarks.html was specially
> > >`tuned' to reach peak reiser4 performance? Remember why you decided to
> > >turn OVERWRITE and MODIFY phases off?
>
What I should have done was what I did with fsync performance. With
fsync performance I told people that we had not yet tuned for it, please
wait a bit and we will tune for it, for now it sucks.
Instead what I did was discuss with Zam at the time how it could be
fixed, leave it off the website until Zam was given a chance to fix it,
and then I managed to forget about it. After release one remembers
what all the things that should have been fixed before release were, sigh.
Zam and I both know what is needed to fix performance in these phases,
and I just spoke with him on the phone. I imagine it is 2 weeks of work
for him, but it could be up to 6 weeks. He will comment later.
The fix should consist of the following:
1) tweak the relocation policies (zam will say more about this, as he is
the maintainer of them)
2) optimize overwrite set so that in the modify phase we behave like a
write twice journaling filesystem, which means implement a reserved for
the journal only area that exists as long as plenty of disk space is free.
3) finish the repacker (but we won't do that this month....)
The modify phase, which picks a random block to modify, is a worst case
for a wandering log without a repacker. We could actually do very very
well at that kind of activity with a repacker, probably better than a
fixed journal, but for now we should try acting like a write twice
journal for atoms that small.
So, having realized my error thanks to the gracious kindness of how you
pointed it out, we will modify the web site to say that we are not yet
tuned for and currently suck at modifying random blocks in the middle of
files, and appending small amounts to the end of them, and overwriting
small files, but that we think we know what is needed to fix those things.
I think your characterization of my reasons was unkind and also unfair.
I will pass on saying similarly unkind things about you as you are
mostly a good person. You never did seem to understand me. You didn't
understand the reasons for my technical designs (when they worked it
made no sense to you that they did), and you didn't understand me as a
person.
I wish you well in your new job, and thank you for the hard work you did
for me.
Hans
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-28 9:53 ` Hans Reiser
@ 2004-08-28 13:47 ` Nikita Danilov
2004-08-28 23:45 ` Hans Reiser
0 siblings, 1 reply; 18+ messages in thread
From: Nikita Danilov @ 2004-08-28 13:47 UTC (permalink / raw)
To: Hans Reiser
Cc: Christoph Hellwig, Christophe Saout, Andrew Morton, linux-fsdevel,
linux-kernel, flx, torvalds, reiserfs-list,
Alexander Zarochentcev
Hans Reiser writes:
> Nikita Danilov wrote:
>
> > > >Whoever sponsors the benchmark usually wins. Had you forgotten that
> > > >mongo setup used by http://www.namesys.com/benchmarks.html was specially
> > > >`tuned' to reach peak reiser4 performance? Remember why you decided to
> > > >turn OVERWRITE and MODIFY phases off?
> >
> What I should have done was what I did with fsync performance. With
> fsync performance I told people that we had not yet tuned for it, please
> wait a bit and we will tune for it, for now it sucks.
Right, and instead of this you are now claiming that these `benchmarks'
show superior reiser4 performance. Moreover, you are doing this
aggressively and proposing other people to `eat the dust' over what
happened to only exist due to your failing to remember something. As
some other notable LKML poster put it `inform yourself before
posting'. :-)
>
> Instead what I did was discuss with Zam at the time how it could be
> fixed, leave it off the website until Zam was given a chance to fix it,
It wasn't Zam. It was me to begin with. OVERWRITE and MODIFY phases were
turned off after switching to large keys.
> and then I managed to forget about it. After release one remembers
> what all the things that should have been fixed before release were, sigh.
>
[...]
>
> I think your characterization of my reasons was unkind and also unfair.
What fairness and kindness one expects after styling others as `puppies'
in public, and commenting on their hair style in purportedly technical
argument?
[...]
>
> Hans
Nikita.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-28 19:09 ` Christoph Hellwig
@ 2004-08-28 21:41 ` Hans Reiser
2004-08-30 16:02 ` Herbert Poetzl
0 siblings, 1 reply; 18+ messages in thread
From: Hans Reiser @ 2004-08-28 21:41 UTC (permalink / raw)
To: Linus Torvalds
Cc: Christoph Hellwig, flx, Christophe Saout, Andrew Morton,
linux-fsdevel, linux-kernel, flx, reiserfs-list
I think it is reasonable to make the -nopseudos (turns off the metafiles
) mount option mandatory, until the bugs are resolved.
Our testing did not find these metafile/VFS bugs because of the reason
for all our bugs, we screwed up.
There is a distinct difference between some persons and I, which is that
some think all of reiser4 should be excluded until metafiles are
implemented by VFS some long time from now, and I, in that I merely
think buggy optional features should be turned off until they are
fixed. I, being renowned for my paranoia and asininity as I am, think
these persons find it convenient as an excuse to keep us from competing,
and I think that if we were slower there would be less hassle every time
we try to get into the kernel.
While reiser4 has some significant roughnesses remaining in its
performance, I think the average user would find it performs better than
other filesystems, and is stable enough for, say, a laptop, and I
predict that by the time we have it stable enough for mission critical
servers, all the roughness in various important corner cases will be
gone.
Persons benchmarking it with tarballs, please be sure to use tarballs
created on reiser4, not ext2 tarballs, readdir order matters a lot for
sorted directory filesystems.
Hans
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-28 19:23 ` Alexander Lyamin
@ 2004-08-28 22:36 ` Hans Reiser
0 siblings, 0 replies; 18+ messages in thread
From: Hans Reiser @ 2004-08-28 22:36 UTC (permalink / raw)
To: flx
Cc: Christoph Hellwig, Christophe Saout, Andrew Morton, linux-fsdevel,
linux-kernel, flx, torvalds, reiserfs-list
Alexander Lyamin wrote:
>
>
> o metafiles - AFAIK it was product of Nikita Danilov just playing and fooling.
>
>
No, it was not, it was an idea I had for so long I cannot remember when,
maybe 1984, and I proposed it to darpa, and then nikita got told about
it probably from being told to read the web page on our darpa project,
and he took on the task of coding it.
Hans
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-28 13:47 ` Nikita Danilov
@ 2004-08-28 23:45 ` Hans Reiser
2004-08-29 9:35 ` Nikita Danilov
0 siblings, 1 reply; 18+ messages in thread
From: Hans Reiser @ 2004-08-28 23:45 UTC (permalink / raw)
To: Nikita Danilov
Cc: Christoph Hellwig, Christophe Saout, Andrew Morton, linux-fsdevel,
linux-kernel, flx, torvalds, reiserfs-list,
Alexander Zarochentcev
Nikita Danilov wrote:
> >
> > Instead what I did was discuss with Zam at the time how it could be
> > fixed, leave it off the website until Zam was given a chance to fix it,
>
>It wasn't Zam. It was me to begin with. OVERWRITE and MODIFY phases were
>turned off after switching to large keys.
>
>
>
The online repacker is the best technical fix for these problems. Tight
packing is generally more damaged in its layout by small changes than
loose packing, and V4 is the ultimate tight packer. It is not clear to
me that making V4 pack more loosely is necessarily a good idea. These
phases disturb the original packing at flush time, and the other phases
don't. One solution to try is packing less tightly. Maybe once every
megabyte skip forward 1 megabyte, and once every 64 megabytes skip to a
random disk location, when allocating large new atoms. I think I prefer
using a repacker, but we should try it and see. The other solutions of
the previous email should also help a lot. The issues of layout in this
are similar to the ones improved by the fibration plugin.
I remember talking with not just you but zam about how we could fix it,
and there was too much in the queue of work, and everyone was
complaining to me that we should be debugging not optimizing, and you
were the only one who thought it was a big deal. I guess you still
think it is a big issue and I still think things are good enough to use
without it. I still think Zam understood the issues better than you did
(allocation is his code not yours).
There are some layout optimizations that a repacker can do best. Still,
there are some rough spots in the current allocation policy, and we
should look at it.
Probably you will have reason to howl if we setup a benchmark which
disturbs things with these phases, runs the repacker (I hope to have one
in 6 months), and then measures our read performance compared to other
filesystems without a repacker....;-)
Hans
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-28 23:45 ` Hans Reiser
@ 2004-08-29 9:35 ` Nikita Danilov
0 siblings, 0 replies; 18+ messages in thread
From: Nikita Danilov @ 2004-08-29 9:35 UTC (permalink / raw)
To: Hans Reiser
Cc: Christoph Hellwig, Christophe Saout, Andrew Morton, linux-fsdevel,
linux-kernel, flx, torvalds, reiserfs-list,
Alexander Zarochentcev
Hans Reiser writes:
[...]
>
> I remember talking with not just you but zam about how we could fix it,
> and there was too much in the queue of work, and everyone was
> complaining to me that we should be debugging not optimizing, and you
> were the only one who thought it was a big deal. I guess you still
You are trying to evade the point. And the point is not how performance
in these phases (and their impact on other phases) can be improved, or
what excuses did you have for not `doing' it, but the fact that after it
was found that large keys behave badly, these phases were turned off,
others were switched to lexicographical operation, _and_ resulting
benchmark is used as a basis to call other people names.
> think it is a big issue and I still think things are good enough to use
> without it. I still think Zam understood the issues better than you did
> (allocation is his code not yours).
>
> There are some layout optimizations that a repacker can do best. Still,
> there are some rough spots in the current allocation policy, and we
> should look at it.
>
> Probably you will have reason to howl if we setup a benchmark which
Surely, as a `big dog' what other sounds can I produce? :)
> disturbs things with these phases, runs the repacker (I hope to have one
> in 6 months), and then measures our read performance compared to other
> filesystems without a repacker....;-)
That's ok with me, but it would make sense to remove unfair benchmarks
from the site until then.
>
> Hans
Nikita.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-27 18:55 ` Nikita Danilov
2004-08-28 9:53 ` Hans Reiser
@ 2004-08-29 11:17 ` Alex Zarochentsev
1 sibling, 0 replies; 18+ messages in thread
From: Alex Zarochentsev @ 2004-08-29 11:17 UTC (permalink / raw)
To: Nikita Danilov
Cc: Hans Reiser, Christoph Hellwig, Christophe Saout, Andrew Morton,
linux-fsdevel, linux-kernel, flx, torvalds, reiserfs-list
On Fri, Aug 27, 2004 at 10:55:50PM +0400, Nikita Danilov wrote:
> Hans Reiser writes:
> > Nikita Danilov wrote:
> >
> > >Hans Reiser writes:
> > > > Christophe Saout wrote:
> > > >
> > > > >
> > > > >
> > > > >I don't know, ask Hans. How could the VFS know it a filesystem wants to
> > > > >do something specific with a file that is completely transparent to the
> > > > >VFS?
> > > > >
> > > > >
> > > > >
> > > > To know what method to use, you must determine the pluginid, and then
> > > > find the method within that plugin for that vfs operation.
> > > >
> > > > As for overhead, well, who eats whose dust in the benchmarks....?
> > >
> > >Whoever sponsors the benchmark usually wins. Had you forgotten that
> > >mongo setup used by http://www.namesys.com/benchmarks.html was specially
> > >`tuned' to reach peak reiser4 performance? Remember why you decided to
> > >turn OVERWRITE and MODIFY phases off? People on #reiser4 report 90
> > >_second_ stalls with reiser4 under high io loads (large atom is
> > >obviously being flushed and everyone waits on it...). In my opinion, it
> > >is such things that are of utmost importance for real reiser4
> > >acceptance, not how to name `metas' sub-directory.
> > >
> > >Nikita.
> > >
> > >
> > >
> > >
> > If you ask real users, they say that reiser4 is fast, and their
> > experience matches our benchmark. You can criticize the benchmark if
>
> They experience 90 second stalls. And please, do not tell me how fast
> reiser4 is, I spent a lot of time working with it, and I know very well
> when it's fast, and when it's deadly slow.
>
> > you want, but then you should run your own and publish it.
>
> I did, after which you told me to turn OVERWRITE and MODIFY phases off,
> beause performance was horrible.
Hmm, I think not the modify/overwrite phases performance were the problem. The
modify and overwrite mongo phases fragment the filesystem too much, it had a
bad effect visible in the read phase and even in delete phase.
We measured/optimized read performance in assumption that fs is not
fragmented, why we should measure bad effect of overwrite instead?
other fs do not relocate already allocated blocks, negative effect of
file set overwrite is zero for them. We could do the same for reiser4
by tuning flush algorithm and journal blocks allocation.
If users want to see read and delete performance as they are, overwrite phase
should be excluded, at least until we teach flush algorithm to not break good
block allocation.
If one wants to measure overwite performance, it is easy to enable overwrite
phase in mongo, but read and delete speed will be affected (it was so when I
tried to optimize delete).
> I not criticizing mongo benchmark per se. I think that it is
> fundamentally wrong to use results that were deliberatly manipulated to
> get best appearance to reiser4 (by omitting work-loads where it performs
> poorly) as an argument. It's not clear who will, according to your
> colorful expression, `eat dust' as a result of this.
> Or do you think
> that users never overwrite of modify files in real life?
not whole file set. I mean it is unusual to overwrite all files in the fs.
> > The 90 second stalls, those should be fixed by debugging copy on capture
> > and turning it on, yes? You can hardly claim I failed to push for the
>
> No. I described so many times to you, why COC is ineffectual here.
There are other ideas :) but, it seems to me, if we will focus on benchmarks,
we never complete plugins, metafiles and other useful things. What I want
from reiser4 is a possiblity to encrypt and compress ~/Mail/ :).
>
> > Hans
>
> Nikita.
--
Alex.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-27 22:29 ` Steve Bergman
2004-08-28 6:54 ` Hans Reiser
@ 2004-08-29 11:42 ` Alex Zarochentsev
1 sibling, 0 replies; 18+ messages in thread
From: Alex Zarochentsev @ 2004-08-29 11:42 UTC (permalink / raw)
To: Steve Bergman
Cc: Hans Reiser, Nikita Danilov, Christoph Hellwig, Christophe Saout,
Andrew Morton, linux-fsdevel, linux-kernel, flx, torvalds,
reiserfs
On Fri, Aug 27, 2004 at 05:29:07PM -0500, Steve Bergman wrote:
> On Fri, 2004-08-27 at 11:15 -0700, Hans Reiser wrote:
> > >
> > If you ask real users, they say that reiser4 is fast, and their
> > experience matches our benchmark. You can criticize the benchmark if
> > you want, but then you should run your own and publish it.
> >
>
>
> As a "real" desktop user who just converted all his partitions from ext3
> to reiser4, I have not, as yet, noticed any startling performance
> increase. Being slightly slightly irked to hear that the benchmark
> numbers that have been paraded around on Slashdot and the internet in
> general, at ext3's expense, have had reiser4's "bad" results surgically
> extracted, I am running my own benchmarks to get the real story on
> reiser4/ext3 mongo performance on my, rather average, desktop hardware.
>
> I am using the latest Mongo on FC/rawhide and the 2.6.8.1-mm4 kernel.
>
> Unfortunately, I get an error from mongo.pl that "Done" is not a numeric
> argument at line 439.
>
> I have done this to fix it:
>
>
> --- mongo.pl 2004-08-27 17:07:01.681723313 -0500
> +++ mongo_fixed.pl 2004-08-27 17:06:51.369306735 -0500
> @@ -429,8 +429,8 @@
> if ( -e ${ERR_FILE}) {
> &DIE ("\nEXITED WITH FAIL\n");
> }
> - my $real = (split ' ', $time_output[0])[1];
> - my $cpu = (split ' ', $time_output[2])[1];
> + my $real = (split ' ', $time_output[1])[1];
> + my $cpu = (split ' ', $time_output[3])[1];
>
> unless ( $real =~ /\s*\d+/ && $cpu =~ /\s*\d+/) {
> LOG "@time_output";
>
>
> What it gets me is the "real" line of the "time" output for "STAT
> REAL_TIME" and the "sys" line of the "time" output for "STAT CPU_TIME".
> i.e. only system time is counted. I believe this was the intent of the
> original code, but want to verify before continuing.
it works on our test servers which are SuSE9.0/9.1.
I think there is a dependency on system utilities, "Done" is not printed by
mongo.pl or any other program from the mongo package.
It would be nice to tell us what linux distro you are using and what mongo
phase fails.
> Thanks,
> Steve Bergman
Thanks.
--
Alex.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-28 21:41 ` reiser4 plugins Hans Reiser
@ 2004-08-30 16:02 ` Herbert Poetzl
2004-08-30 18:55 ` Hans Reiser
0 siblings, 1 reply; 18+ messages in thread
From: Herbert Poetzl @ 2004-08-30 16:02 UTC (permalink / raw)
To: Hans Reiser
Cc: Linus Torvalds, Christoph Hellwig, flx, Christophe Saout,
Andrew Morton, linux-fsdevel, linux-kernel, flx, reiserfs-list
On Sat, Aug 28, 2004 at 02:41:12PM -0700, Hans Reiser wrote:
> I think it is reasonable to make the -nopseudos (turns off the metafiles
> ) mount option mandatory, until the bugs are resolved.
>
> Our testing did not find these metafile/VFS bugs because of the reason
> for all our bugs, we screwed up.
>
> There is a distinct difference between some persons and I, which is that
> some think all of reiser4 should be excluded until metafiles are
> implemented by VFS some long time from now, and I, in that I merely
> think buggy optional features should be turned off until they are
> fixed. I, being renowned for my paranoia and asininity as I am, think
> these persons find it convenient as an excuse to keep us from competing,
> and I think that if we were slower there would be less hassle every time
> we try to get into the kernel.
>
> While reiser4 has some significant roughnesses remaining in its
> performance, I think the average user would find it performs better than
> other filesystems, and is stable enough for, say, a laptop, and I
> predict that by the time we have it stable enough for mission critical
> servers, all the roughness in various important corner cases will be
> gone.
>
> Persons benchmarking it with tarballs, please be sure to use tarballs
> created on reiser4, not ext2 tarballs, readdir order matters a lot for
> sorted directory filesystems.
hmm, so probably we have to wait until all
tar packagers moved to reiser4, so that the
available tar files are 'sorted properly' ...
best,
Herbert
> Hans
> -
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: reiser4 plugins
2004-08-30 16:02 ` Herbert Poetzl
@ 2004-08-30 18:55 ` Hans Reiser
0 siblings, 0 replies; 18+ messages in thread
From: Hans Reiser @ 2004-08-30 18:55 UTC (permalink / raw)
To: Herbert Poetzl
Cc: Linus Torvalds, Christoph Hellwig, flx, Christophe Saout,
Andrew Morton, linux-fsdevel, linux-kernel, flx, reiserfs-list
Herbert Poetzl wrote:
>
>hmm, so probably we have to wait until all
>tar packagers moved to reiser4, so that the
>available tar files are 'sorted properly' ...
>
>best,
>Herbert
>
>
>
Or wait for a repacker, which will probably happen sooner. Distros that
use reiser4 will probably tar in reiser4 order.
^ permalink raw reply [flat|nested] 18+ messages in thread
* RE: reiser4 plugins
[not found] <42C05F16.5000804@xfs.org>
@ 2005-06-28 6:37 ` Al Boldi
0 siblings, 0 replies; 18+ messages in thread
From: Al Boldi @ 2005-06-28 6:37 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-fsdevel
Hans Reiser wrote:
> Steve, there is a remark about XFS below which you are going to be
> more expert on.
>
> Theodore Ts'o wrote:
>
>>
>>XFS has similar issues where it assumes that hardware has powerfail
>>interrupts, and that the OS can use said powerfail interrupt to stop
>>DMA's in its tracks on an power failure, so that you don't have
>>garbage written to key filesystem data structures when the memory
>>starts suffering from the dropping voltage on the power bus faster
>>than the DMA engine or the disk drives. So XFS is a great filesystem
>>--- but you'd better be running it on a UPS, or on a system which has
>>power fail interrupts and an OS that knows what to do. Ext3, because
>>it does physical block journalling, does not suffer from this problem.
>>(Yes, Resierfs uses logical journalling as well, so it suffers from
>>the same problem.)
>>
True now, not so around 2.4.20 when XFS was rock-solid. I think they tried
to improve on performance and broke something. I wish they would fix that
because it forced me back to ext3, as in consistency over performance any
time.
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2005-06-28 6:37 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <42C05F16.5000804@xfs.org>
2005-06-28 6:37 ` reiser4 plugins Al Boldi
2004-08-25 22:28 silent semantic changes with reiser4 Andrew Morton
2004-08-26 8:31 ` Hans Reiser
2004-08-26 8:45 ` Andrew Morton
2004-08-26 12:18 ` Christophe Saout
2004-08-26 12:49 ` Christoph Hellwig
2004-08-26 13:00 ` Christophe Saout
2004-08-26 13:07 ` Christoph Hellwig
2004-08-26 13:17 ` reiser4 plugins (was: silent semantic changes with reiser4) Christophe Saout
2004-08-26 13:24 ` Christoph Hellwig
2004-08-26 13:35 ` Christophe Saout
2004-08-26 13:40 ` Christoph Hellwig
2004-08-26 13:58 ` Christophe Saout
2004-08-26 23:55 ` reiser4 plugins Hans Reiser
2004-08-27 12:04 ` Nikita Danilov
2004-08-27 18:15 ` Hans Reiser
2004-08-27 18:55 ` Nikita Danilov
2004-08-28 9:53 ` Hans Reiser
2004-08-28 13:47 ` Nikita Danilov
2004-08-28 23:45 ` Hans Reiser
2004-08-29 9:35 ` Nikita Danilov
2004-08-29 11:17 ` Alex Zarochentsev
2004-08-27 22:29 ` Steve Bergman
2004-08-28 6:54 ` Hans Reiser
2004-08-29 11:42 ` Alex Zarochentsev
2004-08-26 23:54 ` Hans Reiser
2004-08-28 10:59 ` reiser4 plugins (was: silent semantic changes with reiser4) Alexander Lyamin
2004-08-28 11:12 ` Christoph Hellwig
2004-08-28 12:05 ` Alexander Lyamin
2004-08-28 13:56 ` Christoph Hellwig
2004-08-28 19:23 ` Alexander Lyamin
2004-08-28 22:36 ` reiser4 plugins Hans Reiser
2004-08-28 17:18 ` reiser4 plugins (was: silent semantic changes with reiser4) Linus Torvalds
2004-08-28 19:03 ` Alexander Lyamin
2004-08-28 19:09 ` Christoph Hellwig
2004-08-28 21:41 ` reiser4 plugins Hans Reiser
2004-08-30 16:02 ` Herbert Poetzl
2004-08-30 18:55 ` Hans Reiser
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).