We need help benchmarking and debugging reiser4

We need help benchmarking and debugging reiser4

[Hans Reiser]

Can some of you help us by doing such things as replicating our 
benchmarks, and helping us debug it as we enter the last stretch before 
Halloween?

Nikita and Oleg will describe the details of what to do to replicate the 
benchmarks, please be sure to use reiser4 readdir order for writes to 
reiser4 (that means don't use tarballs made from ext2 (Remember that 
writes determine subsequent read performance.)), and to use the latest 
hard drives and fast processors with udma 5 turned on.  We are quite 
sensitive to transfer speed since we do a good job of avoiding seeks.  
We are sensitive to readdir order because we sort directory entries 
(which is necessary for having efficient large directory lookups).   In 
reiser4.1 we will ship a repacker, and then it won't matter what order 
you do writes in so long as the repacker gets a chance to run at night.  

-- 
Hans

Re: We need help benchmarking and debugging reiser4

[Cliff White]

> Can some of you help us by doing such things as replicating our 
> benchmarks, and helping us debug it as we enter the last stretch before 
> Halloween?
> 

We are interested in helping, but i haven't seen the follow-up mail 
mentioned below - if you could send us some more specifics, we'd be
glad to join the fun.
cliffw
OSDL

> Nikita and Oleg will describe the details of what to do to replicate the 
> benchmarks, please be sure to use reiser4 readdir order for writes to 
> reiser4 (that means don't use tarballs made from ext2 (Remember that 
> writes determine subsequent read performance.)), and to use the latest 
> hard drives and fast processors with udma 5 turned on.  We are quite 
> sensitive to transfer speed since we do a good job of avoiding seeks.  
> We are sensitive to readdir order because we sort directory entries 
> (which is necessary for having efficient large directory lookups).   In 
> reiser4.1 we will ship a repacker, and then it won't matter what order 
> you do writes in so long as the repacker gets a chance to run at night.  
> 
> -- 
> Hans
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: We need help benchmarking and debugging reiser4

[reiser]

Cliff White wrote:

>>Can some of you help us by doing such things as replicating our 
>>benchmarks, and helping us debug it as we enter the last stretch before 
>>Halloween?
>>
>>    
>>
>
>We are interested in helping, but i haven't seen the follow-up mail 
>mentioned below - if you could send us some more specifics, we'd be
>glad to join the fun.
>cliffw
>OSDL
>
>  
>
>>Nikita and Oleg will describe the details of what to do to replicate the 
>>benchmarks, please be sure to use reiser4 readdir order for writes to 
>>reiser4 (that means don't use tarballs made from ext2 (Remember that 
>>writes determine subsequent read performance.)), and to use the latest 
>>hard drives and fast processors with udma 5 turned on.  We are quite 
>>sensitive to transfer speed since we do a good job of avoiding seeks.  
>>We are sensitive to readdir order because we sort directory entries 
>>(which is necessary for having efficient large directory lookups).   In 
>>reiser4.1 we will ship a repacker, and then it won't matter what order 
>>you do writes in so long as the repacker gets a chance to run at night.  
>>
>>-- 
>>Hans
>>
>>
>>-
>>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>Please read the FAQ at  http://www.tux.org/lkml/
>>
>>    
>>
>
>
>
>
>  
>
Cliff, has there been a russian timezone working day since you requested 
it?  (I don't have access to your original email at the moment.) If 
there has been, then Nikita, please explain why the request was lost, 
and assure me that next time it goes into someone's todo list, and gets 
responded to on the same day.

Hans

Re: We need help benchmarking and debugging reiser4

[Nikita Danilov]

Cliff White writes:
 > > Can some of you help us by doing such things as replicating our 
 > > benchmarks, and helping us debug it as we enter the last stretch before 
 > > Halloween?
 > > 
 > 
 > We are interested in helping, but i haven't seen the follow-up mail 
 > mentioned below - if you could send us some more specifics, we'd be
 > glad to join the fun.

Sorry for delay.

Benchmarks Hans is talking about consist of reading some directory tree
from given file system (foofs, foo={reiser{4,fs},ext{2,3}}). Reading is
done by copying data to the ramfs. One can do the same with tar, but be
warned that "tar cf /dev/null ." is special cased in gnu tar: no actual
reads take place if output is /dev/null.

Another (very important, as it turned out) point is that directory tree
has to be in read optimal order for foofs: files were created in the
order of their directory entries.

Probably example is the best way to describe this: to create
read-optimal directory tree in foofs do:

$ cd /mnt/foofs-mount-point
$ cp -a /somewhere/directory-tree .
$ tar cf directory-tree /somewhere/directory-tree-foofs-read-optimal.tar
$ cd /
$ umount /mnt/foofs-mount-point
$ /sbin/mkfs.foofs /dev/device
$ mount /dev/device /mnt/foofs-mount-point
$ cd /mnt/foofs-mount-point
$ tar xf /somewhere/directory-tree-foofs-read-optimal.tar

Now, /mnt/foofs-mount-point/directory-tree contains read-optimal
directory tree for foofs.

We are also generally interested in results of any standard file system
benchmarks/stress tests.

Reiser4 has known rough corners with running out of disk space.

And of course, SMP scalability measurements on OSDL hardware would be
very interesting. Just simple kernel compilation would pass as rough
test, I think.

 > cliffw
 > OSDL
 > 
 > > Nikita and Oleg will describe the details of what to do to replicate the 
 > > benchmarks, please be sure to use reiser4 readdir order for writes to 
 > > reiser4 (that means don't use tarballs made from ext2 (Remember that 
 > > writes determine subsequent read performance.)), and to use the latest 
 > > hard drives and fast processors with udma 5 turned on.  We are quite 
 > > sensitive to transfer speed since we do a good job of avoiding seeks.  
 > > We are sensitive to readdir order because we sort directory entries 
 > > (which is necessary for having efficient large directory lookups).   In 
 > > reiser4.1 we will ship a repacker, and then it won't matter what order 
 > > you do writes in so long as the repacker gets a chance to run at night.  
 > > 
 > > -- 
 > > Hans
 > > 

Nikita.