Results of "bk clone" tests with 2.6.2, reiser4, reiserfs, ext3, xfs and jfs

Results of "bk clone" tests with 2.6.2, reiser4, reiserfs, ext3, xfs and jfs

[Steven Cole]

I tested ext3, reiser4, reiserfs, xfs and jfs on the
same disk partition, using kernel 2.6.2.  The system
is a dual 700Mhz PIII, SMP kernel, scsi disk.

Reiser4 came in second fastest for this particular test,
and was the most efficient at storing cloned bk trees.

I used the "-o key=key_short" option when creating the
reiser4 file system due to getting this message:
"kernel: WARNING: Key format mismatch. Only small keys are supported."

Other file systems were created with no special options.

The command given for the first of each run was:
$ /usr/bin/time bk clone -q testing-2.6 clone_1-2.6

The local "testing-2.6" tree was cloned from an ext3 file system
to the target file system under test, e.g.
$ bk clone -q testing-2.6 /e3_testing/testing-2.6

Data from the first of five runs on each file system:

ext3		57.66user 18.29system 1:50.42elapsed 68%CPU
reiser4		58.98user 37.25system 2:04.20elapsed 77%CPU
reiserfs	57.87user 21.79system 2:07.07elapsed 62%CPU
xfs		57.85user 20.58system 2:18.80elapsed 56%CPU
jfs		57.18user 15.24system 2:13.77elapsed 54%CPU

In each case, the first of five runs gave the best time.

The full results for all five runs for all five file systems
follows this summary.

Results of df -T for /dev/sda9 with 6 clones of 2.6 bk tree for
five file systems.

Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda9     ext3     4071744   1449120   2415792  38% /e3_testing
/dev/sda9  reiser4     3930680   1203224   2727456  31% /r4_testing
/dev/sda9 reiserfs     4136572   1301644   2834928  32% /r3_testing
/dev/sda9      xfs     4126464   1403412   2723052  35% /xfs_testing
/dev/sda9      jfs     4119988   1455152   2664836  36% /jfs_testing

Even though the same six trees were created, the reported Used space
varies quite a bit.  Reiser4 wins here. 

Here are the data from all the runs:

[steven@spc2 e3_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_1-2.6
57.66user 18.29system 1:50.42elapsed 68%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (21343major+154944minor)pagefaults 0swaps
[steven@spc2 e3_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_2-2.6
57.57user 17.72system 2:02.91elapsed 61%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (12639major+163648minor)pagefaults 0swaps
[steven@spc2 e3_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_3-2.6
57.29user 17.83system 2:02.27elapsed 61%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (10706major+165581minor)pagefaults 0swaps
[steven@spc2 e3_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_4-2.6
57.70user 17.80system 2:06.23elapsed 59%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (10668major+165619minor)pagefaults 0swaps
[steven@spc2 e3_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_5-2.6
57.31user 17.66system 2:03.02elapsed 60%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (10384major+165903minor)pagefaults 0swaps
[steven@spc2 e3_testing]$ df -T
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext3      798476    155092    602824  21% /
/dev/sda8     ext3     7147260   3547248   3236948  53% /home
none         tmpfs      257336         0    257336   0% /dev/shm
/dev/sda6     ext3     4024188   1871100   1948664  49% /usr
/dev/sda7     ext3      806368     60396    705008   8% /var
/dev/sda9     ext3     4071744   1449120   2415792  38% /e3_testing


[steven@spc2 r4_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_1-2.6
58.98user 37.25system 2:04.20elapsed 77%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (2major+176402minor)pagefaults 0swaps
[steven@spc2 r4_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_2-2.6
58.74user 38.41system 2:14.33elapsed 72%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+176404minor)pagefaults 0swaps
[steven@spc2 r4_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_3-2.6
58.92user 38.54system 2:12.14elapsed 73%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (2major+176402minor)pagefaults 0swaps
[steven@spc2 r4_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_4-2.6
58.79user 38.18system 2:13.41elapsed 72%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (2major+176402minor)pagefaults 0swaps
[steven@spc2 r4_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_5-2.6
58.77user 38.08system 2:13.04elapsed 72%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (3major+176401minor)pagefaults 0swaps
[steven@spc2 r4_testing]$ df -T
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext3      798476    155092    602824  21% /
/dev/sda8     ext3     7147260   3547740   3236456  53% /home
none         tmpfs      257336         0    257336   0% /dev/shm
/dev/sda6     ext3     4024188   1871100   1948664  49% /usr
/dev/sda7     ext3      806368     60396    705008   8% /var
/dev/sda9  reiser4     3930680   1203224   2727456  31% /r4_testing


[steven@spc2 r3_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_1-2.6
57.87user 21.79system 2:07.07elapsed 62%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (14582major+162660minor)pagefaults 0swaps
[steven@spc2 r3_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_2-2.6
58.11user 20.89system 2:41.70elapsed 48%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (12873major+164369minor)pagefaults 0swaps
[steven@spc2 r3_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_3-2.6
58.05user 20.95system 2:33.79elapsed 51%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (12721major+164521minor)pagefaults 0swaps
[steven@spc2 r3_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_4-2.6
58.45user 21.51system 2:33.44elapsed 52%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (12597major+164645minor)pagefaults 0swaps
[steven@spc2 r3_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_5-2.6
58.01user 21.57system 2:34.61elapsed 51%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (12416major+164826minor)pagefaults 0swaps
[steven@spc2 r3_testing]$ df -T
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext3      798476    155096    602820  21% /
/dev/sda8     ext3     7147260   3548168   3236028  53% /home
none         tmpfs      257336         0    257336   0% /dev/shm
/dev/sda6     ext3     4024188   1871100   1948664  49% /usr
/dev/sda7     ext3      806368     60396    705008   8% /var
/dev/sda9 reiserfs     4136572   1301644   2834928  32% /r3_testing

[steven@spc2 xfs_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_1-2.6
57.85user 20.58system 2:18.80elapsed 56%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (19309major+156978minor)pagefaults 0swaps
[steven@spc2 xfs_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_2-2.6
57.18user 19.62system 2:44.10elapsed 46%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (11460major+164827minor)pagefaults 0swaps
[steven@spc2 xfs_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_3-2.6
57.75user 19.72system 2:28.18elapsed 52%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (9789major+166498minor)pagefaults 0swaps
[steven@spc2 xfs_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_4-2.6
57.45user 20.04system 2:30.13elapsed 51%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (10556major+165731minor)pagefaults 0swaps
[steven@spc2 xfs_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_5-2.6
57.75user 19.64system 2:28.46elapsed 52%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (10348major+165939minor)pagefaults 0swaps
[steven@spc2 xfs_testing]$ df -T
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext3      798476    155100    602816  21% /
/dev/sda8     ext3     7147260   3548168   3236028  53% /home
none         tmpfs      257336         0    257336   0% /dev/shm
/dev/sda6     ext3     4024188   1871100   1948664  49% /usr
/dev/sda7     ext3      806368     60396    705008   8% /var
/dev/sda9      xfs     4126464   1403412   2723052  35% /xfs_testing

[steven@spc2 jfs_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_1-2.6
57.18user 15.24system 2:13.77elapsed 54%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (18052major+158235minor)pagefaults 0swaps
[steven@spc2 jfs_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_2-2.6
56.83user 13.95system 2:29.60elapsed 47%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (11788major+164499minor)pagefaults 0swaps
[steven@spc2 jfs_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_3-2.6
57.37user 14.06system 2:24.45elapsed 49%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (10592major+165695minor)pagefaults 0swaps
[steven@spc2 jfs_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_4-2.6
57.20user 14.32system 2:24.41elapsed 49%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (11455major+164832minor)pagefaults 0swaps
[steven@spc2 jfs_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_5-2.6
57.13user 14.18system 2:27.03elapsed 48%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (10952major+165335minor)pagefaults 0swaps
[steven@spc2 jfs_testing]$ df -T
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext3      798476    155104    602812  21% /
/dev/sda8     ext3     7147260   3548168   3236028  53% /home
none         tmpfs      257336         0    257336   0% /dev/shm
/dev/sda6     ext3     4024188   1871100   1948664  49% /usr
/dev/sda7     ext3      806368     60396    705008   8% /var
/dev/sda9      jfs     4119988   1455152   2664836  36% /jfs_testing

Steven

Re: Results of "bk clone" tests with 2.6.2, reiser4, reiserfs, ext3, xfs and jfs

[Hans Reiser]

Steven Cole wrote:

>I used the "-o key=key_short" option when creating the
>reiser4 file system due to getting this message:
>"kernel: WARNING: Key format mismatch. Only small keys are supported."
>  
>
This might have hurt our performance substantially.  Did you 
intentionally choose small keys?  If not, Nikita, why did this happen?

Re: Results of "bk clone" tests with 2.6.2, reiser4, reiserfs, ext3, xfs and jfs

[Steven Cole]

On Wednesday 11 February 2004 09:12 am, Hans Reiser wrote:
> Steven Cole wrote:
> >I used the "-o key=key_short" option when creating the
> >reiser4 file system due to getting this message:
> >"kernel: WARNING: Key format mismatch. Only small keys are supported."
>
> This might have hurt our performance substantially.  Did you
> intentionally choose small keys?  If not, Nikita, why did this happen?

Well, intentionally _only_ because it seemed the reasonable thing to do
in the face of the above warning message.  Tomorrow, I can recompile
the reiser4-enabled kernel with the large keys option and mkfs.reiser4
again without that key_short option.  I'll let you know if there are any interesting
results.

Steven

Re: Results of "bk clone" tests with 2.6.2, reiser4, reiserfs, ext3, xfs and jfs

[Steven Cole]

On Wed, 2004-02-11 at 02:32, Steven Cole wrote:
> On Wednesday 11 February 2004 09:12 am, Hans Reiser wrote:
> > Steven Cole wrote:
> > >I used the "-o key=key_short" option when creating the
> > >reiser4 file system due to getting this message:
> > >"kernel: WARNING: Key format mismatch. Only small keys are supported."
> >
> > This might have hurt our performance substantially.  Did you
> > intentionally choose small keys?  If not, Nikita, why did this happen?
> 
> Well, intentionally _only_ because it seemed the reasonable thing to do
> in the face of the above warning message.  Tomorrow, I can recompile
> the reiser4-enabled kernel with the large keys option and mkfs.reiser4
> again without that key_short option.  I'll let you know if there are any interesting
> results.
> 
> Steven

OK, tomorrow came a day early this week.

I recompiled the 2.6.2 reiser4 kernel with CONFIG_REISER4_LARGE_KEY=y,
rebooted, and created a reiser4 fs without the key_small option as before.

I repeated the tests using the same bk tree and got very similar results
to using short keys.

After that, I umounted and did a fsck.reiser4 and got many errors,
"12463 fixable corrections".  I repeated fsck.reiser4 with --fix, and
the errors were fixed. (No corruption found)

I then mounted the fs again, created another clone, ran "bk -r check -acv"
on the new clone, and that test passed with no errors.  I then ran fsck again, and
it reported 2080 fixable corrections, which were fixed with a following
fsck.reiser4 --fix /dev/sda9.

[root@spc2 steven]# mount -t reiser4 /dev/sda9 /r4_testing
[root@spc2 steven]# df -T
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext3      798476    155108    602808  21% /
/dev/sda8     ext3     7147260   3548360   3235836  53% /home
none         tmpfs      257332         0    257332   0% /dev/shm
/dev/sda6     ext3     4024188   1871100   1948664  49% /usr
/dev/sda7     ext3      806368     61504    703900   9% /var
/dev/sda9  reiser4     3930680       288   3930392   1% /r4_testing


[steven@spc2 r4_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_1-2.6
58.97user 37.55system 2:05.34elapsed 77%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1major+176404minor)pagefaults 0swaps
[steven@spc2 r4_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_2-2.6
59.29user 38.55system 2:15.31elapsed 72%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1major+176404minor)pagefaults 0swaps
[steven@spc2 r4_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_3-2.6
59.27user 38.29system 2:13.40elapsed 73%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1major+176404minor)pagefaults 0swaps
[steven@spc2 r4_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_4-2.6
59.22user 38.08system 2:14.81elapsed 72%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (2major+176403minor)pagefaults 0swaps
[steven@spc2 r4_testing]$ /usr/bin/time bk clone -q testing-2.6 clone_5-2.6
59.15user 38.42system 2:14.20elapsed 72%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (3major+176402minor)pagefaults 0swaps
[steven@spc2 r4_testing]$ df -T
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext3      798476    155104    602812  21% /
/dev/sda8     ext3     7147260   3548360   3235836  53% /home
none         tmpfs      257332         0    257332   0% /dev/shm
/dev/sda6     ext3     4024188   1871100   1948664  49% /usr
/dev/sda7     ext3      806368     61504    703900   9% /var
/dev/sda9  reiser4     3930680   1208876   2721804  31% /r4_testing

I then umounted the reiser4 partition and ran fsck.reiser4 on it,
and got lots of the following:

Error: Node (50123), item (11): StatData of the file [14d9e:732e7573627175:14da6] has the wrong bytes (35872),
Should be (36864). Plugin (stat40).
        Time interval: Wed Feb 11 11:08:35 2004 - Wed Feb 11 11:10:43 2004
***** fsck.reiser4 finished at Wed Feb 11 11:10:43 2004
Closing fs...done

12463 fixable corruptions were detected in the FileSystem. Run with --fix option to fix them.

[root@spc2 /]# /usr/local/sbin/fsck.reiser4 --fix /dev/sda9
*******************************************************************
This is an EXPERIMENTAL version of fsck.reiser4. Read REAMDE first.
*******************************************************************

Fscking the /dev/sda9 block device.
Will fix minor corruptions of the Reiser4 SuperBblock.
Will fix minor corruptions of the Reiser4 FileSystem.
Continue?
(Yes/No): yes
***** Openning the fs.
Reiser4 fs was detected on the /dev/sda9.
Master super block (16):
magic:          ReIsEr4
blksize:        4096
format:         0x0 (format40)
label:          <none>

Format super block (17):
plugin:         format40
description:    Disk-format for reiser4, ver. 0.5.0
magic:          ReIsEr40FoRmAt
flushes:        0
mkfs id:        0x4b822079
blocks:         1034176
free blocks:    731967
root block:     293783
tail policy:    0x4 (smart)
next oid:       0x2d287
file count:     119341
tree height:    4
key policy:     LARGE

***** Tree Traverse Pass: scanning the reiser4 internal tree.

After many messages, fsck reported "No corruption found".

Steven

Re: Results of "bk clone" tests with 2.6.2, reiser4, reiserfs, ext3, xfs and jfs

[Hans Reiser]

Steven Cole wrote:

>On Wed, 2004-02-11 at 02:32, Steven Cole wrote:
>  
>
>>On Wednesday 11 February 2004 09:12 am, Hans Reiser wrote:
>>    
>>
>>>Steven Cole wrote:
>>>      
>>>
>>>>I used the "-o key=key_short" option when creating the
>>>>reiser4 file system due to getting this message:
>>>>"kernel: WARNING: Key format mismatch. Only small keys are supported."
>>>>        
>>>>
>>>This might have hurt our performance substantially.  Did you
>>>intentionally choose small keys?  If not, Nikita, why did this happen?
>>>      
>>>
>>Well, intentionally _only_ because it seemed the reasonable thing to do
>>in the face of the above warning message.  Tomorrow, I can recompile
>>the reiser4-enabled kernel with the large keys option and mkfs.reiser4
>>again without that key_short option.  I'll let you know if there are any interesting
>>results.
>>
>>Steven
>>    
>>
>
>OK, tomorrow came a day early this week.
>
>I recompiled the 2.6.2 reiser4 kernel with CONFIG_REISER4_LARGE_KEY=y,
>rebooted, and created a reiser4 fs without the key_small option as before.
>
>I repeated the tests using the same bk tree and got very similar results
>to using short keys.
>  
>
Hmmm, we need to look into that.  Thanks Steven.  Zam, please analyze 
just what fs operations this benchmark performs, report it, and then 
analyze why we are slow.

>After that, I umounted and did a fsck.reiser4 and got many errors,
>"12463 fixable corrections". 
>
Well it seems you found a reason for us to delay our release.  Nikita 
will go fix it and then get back to you.

Re: Results of "bk clone" tests with 2.6.2, reiser4, reiserfs, ext3, xfs and jfs

[Hans Reiser]

Hans Reiser wrote:

>
> Hmmm, we need to look into that.  Thanks Steven.  Zam, please analyze 
> just what fs operations this benchmark performs, report it, and then 
> analyze why we are slow.

Zam, ignore this, I forgot that we already figured out that it was due 
to it being an sync intensive workload, and that bk would need to use 
sys_reiser4 instead of fsync if it wants to go fast with reiser4.

Re: Results of "bk clone" tests with 2.6.2, reiser4, reiserfs, ext3, xfs and jfs

[Steven Cole]

On Thursday 12 February 2004 12:12 am, Hans Reiser wrote:
> Hans Reiser wrote:
> > Hmmm, we need to look into that.  Thanks Steven.  Zam, please analyze
> > just what fs operations this benchmark performs, report it, and then
> > analyze why we are slow.
>
> Zam, ignore this, I forgot that we already figured out that it was due
> to it being an sync intensive workload, and that bk would need to use
> sys_reiser4 instead of fsync if it wants to go fast with reiser4.

Thanks Hans.  When you get to where sys_reiser4 is usable, it might be
a good idea to add some notes regarding when it would be preferable
to the fs/Kconfig help section, or a pointer to file in Documentation.  Also,
on the -mm vs linus tree issue, the value of sys_reiser4 would be currently
about 277 in the -mm tree (vs 274 in linus), so that would be an additional
complication to folks trying to test in both.

Other bk-related loads to look at are:

bk -r co						# recursive bk checkout
bk export -tplain ../plain_text_tree	# export to a plain text tree

Steven