Re: [STUPID TESTCASE] ext3 htree vs. reiserfs on 2.5.40-mm1

Re: [STUPID TESTCASE] ext3 htree vs. reiserfs on 2.5.40-mm1

[Dieter Nützel]

Am Mittwoch, 2. Oktober 2002 10:48 schrieb Paul P Komkoff Jr:
> Replying to Andreas Dilger:
> > Why do you think data=writeback is better than data=journal?  If the
> > files have no data then it should not make a difference.
>
> It is better than default data=ordered I think :)
>
> Thanks for detailed explanation - it saved much time for me and
> accortind to yours directions I have recalculated my test. Now ext3 is
> better :)
>
> e3
> create          2m49.545s       0m4.162s        2m20.766s
> delete          2m8.155s        0m3.614s        1m34.945s
>
> reiser
> create          3m13.577s       0m4.338s        2m54.026s
> delete          4m39.249s       0m3.968s        4m16.297s
>
> e3
> create          2m50.766s       0m4.024s        2m21.197s
> delete          2m8.755s        0m3.501s        1m35.737s
>
> reiser
> create          3m13.015s       0m4.432s        2m53.412s
> delete          4m41.011s       0m3.893s        4m16.845s
>
>
> this is two typical runs. Now I creating ext3 with
> mke2fs -j -O dir_index -J size=192 -T news /dev/sda4
>
> as you can see, this improves performance by 1/4
>
> Unfortunately, there still one issue in ext3. It called "inode limit".
> Initially I wanted to run this test on 1000000 files but ... I hit
> inode limit and don't want to increase it artificially yet.
>
> Reiserfs worked fine because it don't have such kind of limit ...

Orange and apples, again?

Try with latest ReiserFS data-logging and see what we get?
Chris do you have something for 2.5.40+ ready?

2.5.40 is GREAT!

Regards,
	Dieter
-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel at hamburg.de (replace at with @)

2.5.40 (was Re: ext3 htree vs. reiserfs on 2.5.40-mm1

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 550 bytes --]

(OK, a bit off-topic for the list, but.. ;)

On Fri, 04 Oct 2002 22:54:23 +0200, Dieter =?iso-8859-15?q?N=FCtzel?= said:

> 2.5.40 is GREAT!

2.5.40 would be even greater if somebody could point me at a suitable
replacement for the LVM code I'm currently using - the Sistina people
have LVM 1.0.5 out for the 2.4 series, but apparently stopped supporting
2.5 a while ago.  Is anybody using the LVM2 stuff, or is there some other
similar things for 2.5?
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.5.40 (was Re: ext3 htree vs. reiserfs on 2.5.40-mm1

[Ed Tomlinson]

On October 4, 2002 05:28 pm, Valdis.Kletnieks@vt.edu wrote:
> (OK, a bit off-topic for the list, but.. ;)
>
> On Fri, 04 Oct 2002 22:54:23 +0200, Dieter =?iso-8859-15?q?N=FCtzel?= said:
> > 2.5.40 is GREAT!
>
> 2.5.40 would be even greater if somebody could point me at a suitable
> replacement for the LVM code I'm currently using - the Sistina people
> have LVM 1.0.5 out for the 2.4 series, but apparently stopped supporting
> 2.5 a while ago.  Is anybody using the LVM2 stuff, or is there some other
> similar things for 2.5?

Look up EVMS.  It can support your lvm1 volumes and more.  They are pushing for
inclusion in 2.5 and have code for 2.5.40.  Having used both lvm1 and evms I perfer 
evms.

Ed

Re: 2.5.40 (was Re: ext3 htree vs. reiserfs on 2.5.40-mm1

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 407 bytes --]

On Fri, 04 Oct 2002 17:54:15 EDT, Ed Tomlinson said:

> Look up EVMS.  It can support your lvm1 volumes and more.  They are pushing for
> inclusion in 2.5 and have code for 2.5.40.  Having used both lvm1 and evms I perfer 
> evms.

I owe you a beer if you're ever in this corner of Virginia - that was the only
thing keeping me from testing 2.5.N ;)

/Valdis (who now has enough pieces to go test 2.5.40 ;)

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.5.40 (was Re: ext3 htree vs. reiserfs on 2.5.40-mm1

[Andreas Dilger]

On Oct 04, 2002  17:28 -0400, Valdis.Kletnieks@vt.edu wrote:
> 2.5.40 would be even greater if somebody could point me at a suitable
> replacement for the LVM code I'm currently using - the Sistina people
> have LVM 1.0.5 out for the 2.4 series, but apparently stopped supporting
> 2.5 a while ago.  Is anybody using the LVM2 stuff, or is there some other
> similar things for 2.5?

EVMS has a 2.5 release (and BK tree) to test.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: Re: [STUPID TESTCASE] ext3 htree vs. reiserfs on 2.5.40-mm1

[Andreas Dilger]

On Oct 04, 2002  22:54 +0200, Dieter N?tzel wrote:
> Am Mittwoch, 2. Oktober 2002 10:48 schrieb Paul P Komkoff Jr:
> > Replying to Andreas Dilger:
> > > Why do you think data=writeback is better than data=journal?  If the
> > > files have no data then it should not make a difference.
> >
> > It is better than default data=ordered I think :)
> > Reiserfs worked fine because it don't have such kind of limit ...
> 
> Orange and apples, again?
> 
> Try with latest ReiserFS data-logging and see what we get?
> Chris do you have something for 2.5.40+ ready?

I don't think it will make much difference - the files being created
are zero-length files so the journaling mode is irrelevant.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: Re: [STUPID TESTCASE] ext3 htree vs. reiserfs on 2.5.40-mm1

[Oleg Drokin]

Hello!
On Fri, Oct 04, 2002 at 04:01:26PM -0600, Andreas Dilger wrote:

> > Try with latest ReiserFS data-logging and see what we get?
> > Chris do you have something for 2.5.40+ ready?
> I don't think it will make much difference - the files being created
> are zero-length files so the journaling mode is irrelevant.

The mode is unerlevend indeed, but there are other fixes in hose pathces that
prevent too frequent updates to disk and similar stuff, and that might help.

Bye,
    Oleg

Re: Re: [STUPID TESTCASE] ext3 htree vs. reiserfs on 2.5.40-mm1

[Chris Mason]

On Fri, 2002-10-04 at 18:01, Andreas Dilger wrote:
> On Oct 04, 2002  22:54 +0200, Dieter N?tzel wrote:
> > Am Mittwoch, 2. Oktober 2002 10:48 schrieb Paul P Komkoff Jr:
> > > Replying to Andreas Dilger:
> > > > Why do you think data=writeback is better than data=journal?  If the
> > > > files have no data then it should not make a difference.
> > >
> > > It is better than default data=ordered I think :)
> > > Reiserfs worked fine because it don't have such kind of limit ...
> > 
> > Orange and apples, again?
> > 
> > Try with latest ReiserFS data-logging and see what we get?
> > Chris do you have something for 2.5.40+ ready?
> 
> I don't think it will make much difference - the files being created
> are zero-length files so the journaling mode is irrelevant.

The data logging patches have a bunch of other metadata writeback
optimizations.  It helps most for small transactions, but improves
things for all metadata workloads.  Updated 2.4.20-preX patches should
finally go out today, and then I'll start on 2.5

-chris

Re: Re: [STUPID TESTCASE] ext3 htree vs. reiserfs on 2.5.40-mm1

[Manuel Krause]

On 10/07/2002 03:12 PM, Chris Mason wrote:
> On Fri, 2002-10-04 at 18:01, Andreas Dilger wrote:
> 
>>On Oct 04, 2002  22:54 +0200, Dieter N?tzel wrote:
>>
>>>Am Mittwoch, 2. Oktober 2002 10:48 schrieb Paul P Komkoff Jr:
>>>
[...]
> 
> The data logging patches have a bunch of other metadata writeback
> optimizations.  It helps most for small transactions, but improves
> things for all metadata workloads.  Updated 2.4.20-preX patches should
> finally go out today, and then I'll start on 2.5
                  ^^^^^                                   ?

Sorry, Chris & list!

Somehow, I don't like these kinds of announcements<->reality relation 
@reiserfs-list until now. These patches have been announced 2 weeks ago 
already, IIRC.

Mmh. If I didn't follow my announced timelines, I would keep getting 
serious problems here... ;-)  How Do _you_ solve this? Any hints?!


But, best wishes for your "hidden" work, Chris, I still believe...

Manuel

Re: Re: [STUPID TESTCASE] ext3 htree vs. reiserfs on 2.5.40-mm1

[Hans Reiser]

Dieter Nützel wrote:

>Am Mittwoch, 2. Oktober 2002 10:48 schrieb Paul P Komkoff Jr:
>  
>
>>Replying to Andreas Dilger:
>>    
>>
>>>Why do you think data=writeback is better than data=journal?  If the
>>>files have no data then it should not make a difference.
>>>      
>>>
>>It is better than default data=ordered I think :)
>>
>>Thanks for detailed explanation - it saved much time for me and
>>accortind to yours directions I have recalculated my test. Now ext3 is
>>better :)
>>
>>e3
>>create          2m49.545s       0m4.162s        2m20.766s
>>delete          2m8.155s        0m3.614s        1m34.945s
>>
>>reiser
>>create          3m13.577s       0m4.338s        2m54.026s
>>delete          4m39.249s       0m3.968s        4m16.297s
>>
>>e3
>>create          2m50.766s       0m4.024s        2m21.197s
>>delete          2m8.755s        0m3.501s        1m35.737s
>>
>>reiser
>>create          3m13.015s       0m4.432s        2m53.412s
>>delete          4m41.011s       0m3.893s        4m16.845s
>>
>>
>>this is two typical runs. Now I creating ext3 with
>>mke2fs -j -O dir_index -J size=192 -T news /dev/sda4
>>
>>as you can see, this improves performance by 1/4
>>
>>Unfortunately, there still one issue in ext3. It called "inode limit".
>>Initially I wanted to run this test on 1000000 files but ... I hit
>>inode limit and don't want to increase it artificially yet.
>>
>>Reiserfs worked fine because it don't have such kind of limit ...
>>    
>>
>
>Orange and apples, again?
>
>Try with latest ReiserFS data-logging and see what we get?
>Chris do you have something for 2.5.40+ ready?
>
>2.5.40 is GREAT!
>
>Regards,
>	Dieter
>  
>
So, I think that if you create enough files in your test to stress the 
IO subsystem, then it will be a much more interesting test. 
 Unfortunately, you may need to increase the inode allocation to test 
this for ext3.  Once you start to get above a million files, you may 
start to see reiser4 doing substantially better than ReiserFS because 
reiser4 has better layout and better hashes based on things we learned.

I would be curious to see your benchmark done for 3 million files.  I 
would be curious to see what happens if you umount in the middle of it. 
 I would not be surprised if interesting VFS/dcache performance issues 
arise as you increase the number of files.

Hans