naive but spectacular ext3 HTREE+Orlov benchmark

naive but spectacular ext3 HTREE+Orlov benchmark

[bert hubert]

On a 192 megabyte 1.1GHz laptop with boring disk, 13G well worn partition -
this is not a stylized benchmark! Result is repeatable though.

Summary of HTREE ext3 Orlov vs non-Orlov, in real minute:seconds

                                2.5.45	2.5.46
----------------------------------------------
unpacking kernel tar.bz2:       1:26	1:16
cold traversal:                 1:01.5  0:42.9
hot traversal:                  0:51.0  0:34.5
delete                          0:05.3  <0:02

Congratulations everybody, this is a major result! You can in fact *hear*
the difference. With the Orlov allocator, seeks are much more higher pitched
as if they are generally over shorter distances - which they probably are.

The cold traversal boils down to 4.47 megabytes/second over 13035 files,
close to 303 files/second which is comfortably more than the number of
seeks/second I expect this disk to be able to do. Magic is being performed
here.

traverse script:
#!/bin/sh
find . -type f | xargs -n 500 cat > /dev/null

On Linux 2.5.45, ext3+HTREE:

# mount /dev/hda3 /mnt
$ cd mnt
$ time tar xjf linux-2.5.45.tar.bz2
real    1m26.640s
user    0m48.256s
sys     0m4.592s

*reboot*

# mount /dev/hda3 /mnt
$ cd mnt/linux-2.5.45
$ time ../traverse
real    1m1.518s
user    0m0.159s
sys     0m1.267s
$ time ../traverse
real    0m51.007s
user    0m0.143s
sys     0m1.236s

$ cd .. ; time rm -rf linux-2.5.45
real    0m5.248s
user    0m0.020s
sys     0m0.440s

Same on Linux 2.5.46, ext3+HTREE+Orlov:
# mount /dev/hda3 /mnt
$ cd mnt
$ time tar xjf linux-2.5.45.tar.bz2
real    1m16.071s
user    0m48.291s
sys     0m4.918s

*reboot*

# mount /dev/hda3 /mnt
$ cd mnt/linux-2.5.45
$ time ../traverse
real    0m42.869s
user    0m0.148s
sys     0m1.323s
$ time ../traverse
real    0m34.468s
user    0m0.151s
sys     0m1.358s
$ cd .. ; rm -rf linux-2.5.45
$ 

This last delete wasn't measured but it appeared to be <2 seconds.


-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO

Re: naive but spectacular ext3 HTREE+Orlov benchmark

[bert hubert]

On Tue, Nov 05, 2002 at 04:10:04PM +0000, Alan Cox wrote:
> On Tue, 2002-11-05 at 15:17, bert hubert wrote:
> > Congratulations everybody, this is a major result! You can in fact *hear*
> > the difference. With the Orlov allocator, seeks are much more higher pitched
> > as if they are generally over shorter distances - which they probably are.
> 
> How does the Orlov allocator do if you continually randomly use and
> reuse the file space for a long period of time - the current allocator
> is pretty stable, does Orlov behave the same or degenerate ?

This fs is in daily use, I'll keep an eye on it and rerun the benchmark
above every once in a while. Luckily 2.5 is stable enough for me to run on
my main computer.

Although all my important stuff lives in cvs anyhow.

Regards,

bert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO

Re: naive but spectacular ext3 HTREE+Orlov benchmark

[Alan Cox]

On Tue, 2002-11-05 at 15:17, bert hubert wrote:
> Congratulations everybody, this is a major result! You can in fact *hear*
> the difference. With the Orlov allocator, seeks are much more higher pitched
> as if they are generally over shorter distances - which they probably are.

How does the Orlov allocator do if you continually randomly use and
reuse the file space for a long period of time - the current allocator
is pretty stable, does Orlov behave the same or degenerate ?

Re: naive but spectacular ext3 HTREE+Orlov benchmark

[Andreas Dilger]

On Nov 05, 2002  16:10 +0000, Alan Cox wrote:
> On Tue, 2002-11-05 at 15:17, bert hubert wrote:
> > Congratulations everybody, this is a major result! You can in fact *hear*
> > the difference. With the Orlov allocator, seeks are much more higher pitched
> > as if they are generally over shorter distances - which they probably are.
> 
> How does the Orlov allocator do if you continually randomly use and
> reuse the file space for a long period of time - the current allocator
> is pretty stable, does Orlov behave the same or degenerate ?

AKPM did some simulated testing of this and it wasn't too bad, but there
is of course a tradeoff.  If you pack your files more closely to improve
short term performance, you can cause additional fragmentation in the
future, if some of those files are randomly deleted.

However, I don't think the orlov allocator is "FAT-like" and just fills up
everything sequentially.

What would be an interesting test, let's say, would be lcloning the very
base 2.4.0 BK repository, and then applying all of the changesets in
sequence (or the equivalent with tarballs and patches), and timing
"make" between each run (if there was a way to flush the page cache for
that filesystem it would be very helpful).  This will easily simulate
a long-life directory tree in a very reproducable and quantitative way.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/