Simple preformance test

Simple preformance test

[Ivan Ivanov]

After reading some messages about performance tests for ReiserFS
I decided to make my oun.
The test is simple:

1. Create 10 directories with 10000 files of 10240 bites in each dir
    "dd if=/dev/zero of=xxxx bs=1024 count=10"
2. Touch each file in each dir
3. Delete each file in each dir

Test shows that there is NO reasonable preformance degradation when the
amount of used space grows up.
Funy thing is that between 90% and 100% used space there is a performence
boost.

Test machine is P166 with 80MB RAM
kernel 2.4.18 with Reiserfs quota patches
test partition 1.2GB reiserfs
slackware 8.0

------------------------------------------------
Here are the results:

1. Create test
================================================
Dir No.		10000 files create time [ sec ]
------------------------------------------------
0			170
1 			171
2 			173
3 			172
4 			173
5 			174
6 			173
7 			174
8 			176
9 			155
================================================
test partition is 100% full
df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda7              1172672   1170676      1996 100% /mnt/test

2. Touch test
================================================
Dir No.		10000 files touch time [ sec ]
------------------------------------------------
0 			132
1 			134
2 			133
3 			133
4 			132
5 			133
6 			133
7 			133
8 			133
9 			56
================================================

2. Delete test
================================================
Dir No.		10000 files delete time [ sec ]
------------------------------------------------
0 			142
1 			143
2 			144
3 			142
4 			143
5 			141
6 			142
7 			141
8 			141
9 			132
================================================

And here are the test scripts:

test.create
-----------
#!/bin/sh
block_size=1024
blocks_count=10
for d1 in 0 1 2 3 4 5 6 7 8 9; do
    echo -n "$d1 "
    begin_time=`date +%s`
    mkdir -p testdir/$d1
    for f1 in 0 1 2 3 4 5 6 7 8 9; do
        for f2 in 0 1 2 3 4 5 6 7 8 9; do
	    for f3 in 0 1 2 3 4 5 6 7 8 9; do
		for f4 in 0 1 2 3 4 5 6 7 8 9; do
		    dd if=/dev/zero of=testdir/$d1/$f1$f2$f3$f4 \
		        bs=$block_size count=$blocks_count \
		        >/dev/null 2>&1
		done
	    done
	done
    done
    end_time=`date +%s`
    echo $(($end_time - $begin_time))
done

test.touch
-----------
#!/bin/sh
for d1 in 0 1 2 3 4 5 6 7 8 9; do
    if [ -d testdir/$d1 ]; then
	echo -n "$d1 "
        begin_time=`date +%s`
	for f1 in 0 1 2 3 4 5 6 7 8 9; do
	    for f2 in 0 1 2 3 4 5 6 7 8 9; do
		for f3 in 0 1 2 3 4 5 6 7 8 9; do
		    for f4 in 0 1 2 3 4 5 6 7 8 9; do
			[ -f testdir/$d1/$f1$f2$f3$f4 ] && \
		    	    touch testdir/$d1/$f1$f2$f3$f4
		    done
	        done
	    done
        done
        touch testdir/$d1
        end_time=`date +%s`
        echo $(($end_time - $begin_time))
    fi
done

test.delete
-----------
#!/bin/sh
for d1 in 0 1 2 3 4 5 6 7 8 9; do
    echo -n "$d1 "
    begin_time=`date +%s`
    for f1 in 0 1 2 3 4 5 6 7 8 9; do
	for f2 in 0 1 2 3 4 5 6 7 8 9; do
	    for f3 in 0 1 2 3 4 5 6 7 8 9; do
		for f4 in 0 1 2 3 4 5 6 7 8 9; do
		    rm -f testdir/$d1/$f1$f2$f3$f4
	        done
	    done
	done
    done
    rm -rf testdir/$d1
    end_time=`date +%s`
    echo $(($end_time - $begin_time))
done

Re: Simple preformance test

[Oleg Drokin]

Hello!

On Thu, Apr 11, 2002 at 03:14:03PM +0300, Ivan Ivanov wrote:
> 		    dd if=/dev/zero of=testdir/$d1/$f1$f2$f3$f4 \
> 		        bs=$block_size count=$blocks_count \
> 		        >/dev/null 2>&1
> 		    rm -f testdir/$d1/$f1$f2$f3$f4

This is the explanation of your speedup.
You try to write to a file, but there is not enough space and file is not
created, but you did not notice.
And then you delete nonexistent files, and no doubts this is very fast
operation ;)

BTW, if you turn off tails, your numbers might get faster, but space usage
will get worse.

Bye,
    Oleg

Re: Simple preformance test

[Ivan Ivanov]

Oleg Drokin wrote:

> Hello!
>
> On Thu, Apr 11, 2002 at 03:14:03PM +0300, Ivan Ivanov wrote:
> >                   dd if=/dev/zero of=testdir/$d1/$f1$f2$f3$f4 \
> >                       bs=$block_size count=$blocks_count \
> >                       >/dev/null 2>&1
> >                   rm -f testdir/$d1/$f1$f2$f3$f4
>
> This is the explanation of your speedup.
> You try to write to a file, but there is not enough space and file is not
> created, but you did not notice.
> And then you delete nonexistent files, and no doubts this is very fast
> operation ;)
>
> BTW, if you turn off tails, your numbers might get faster, but space usage
> will get worse.
>
> Bye,
>     Oleg

Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda7              1172672   1170676      1996 100% /mnt/test

I know this but  df  shows 1996 blocks free

------
Ivan

Re: Simple preformance test

[Oleg Drokin]

Hello!

On Thu, Apr 11, 2002 at 05:07:31PM +0300, Ivan Ivanov wrote:

> Filesystem           1k-blocks      Used Available Use% Mounted on
> /dev/hda7              1172672   1170676      1996 100% /mnt/test
> I know this but  df  shows 1996 blocks free

These blocks cannot be used.
They are reserved for some purpose, and there is even a code to grab
these blocks, but the code is disabled, so each reiserfs FS have 2 Mb of data
that cannot be allocated.


Bye,
    Oleg

Re: Simple preformance test

[Yura Umanets]

Ivan Ivanov wrote:

>Oleg Drokin wrote:
>
>>Hello!
>>
>>On Thu, Apr 11, 2002 at 03:14:03PM +0300, Ivan Ivanov wrote:
>>
>>>                  dd if=/dev/zero of=testdir/$d1/$f1$f2$f3$f4 \
>>>                      bs=$block_size count=$blocks_count \
>>>                      >/dev/null 2>&1
>>>                  rm -f testdir/$d1/$f1$f2$f3$f4
>>>
>>This is the explanation of your speedup.
>>You try to write to a file, but there is not enough space and file is not
>>created, but you did not notice.
>>And then you delete nonexistent files, and no doubts this is very fast
>>operation ;)
>>
>>BTW, if you turn off tails, your numbers might get faster, but space usage
>>will get worse.
>>
>>Bye,
>>    Oleg
>>
>
>Filesystem           1k-blocks      Used Available Use% Mounted on
>/dev/hda7              1172672   1170676      1996 100% /mnt/test
>
>I know this but  df  shows 1996 blocks free
>
>------
>Ivan
>
Besides, I think you didn't use "smart" block alocation technics in your 
tests and the all ppl know, that performance degradates because of 
bitmap when used space grows up.

Oleg, is this correct?

-- 
Yury Umanets
IT Engineer of Priocom Corp.
ICQ: 55494590

Re: Simple preformance test

[Oleg Drokin]

Hello!

On Thu, Apr 11, 2002 at 05:27:31PM +0300, Yura Umanets wrote:

> Besides, I think you didn't use "smart" block alocation technics in your 
> tests and the all ppl know, that performance degradates because of 
> bitmap when used space grows up.
> Oleg, is this correct?
No, For this kind of test this is not true in fact.
What is done is we continuosly allocate blocks until we can
Then we free blocks.
There is no situation where we free some blocks, then allocate again and so on.
So no fragmentation, free space is not fragmented either.
So no slowdown on nearly full disk.
But there is no speedups either. Adn what we see in published numers is simply
script/benchmark error that should be ignored at least.

Bye,
    Oleg

Re: Simple preformance test

[Hans Reiser]

Oleg Drokin wrote:

>Hello!
>
>On Thu, Apr 11, 2002 at 05:07:31PM +0300, Ivan Ivanov wrote:
>
>>Filesystem           1k-blocks      Used Available Use% Mounted on
>>/dev/hda7              1172672   1170676      1996 100% /mnt/test
>>I know this but  df  shows 1996 blocks free
>>
>
>These blocks cannot be used.
>They are reserved for some purpose, and there is even a code to grab
>these blocks, but the code is disabled, so each reiserfs FS have 2 Mb of data
>that cannot be allocated.
>
>
>Bye,
>    Oleg
>
>
This is a bug.  These blocks should not show up as available.  Fix it.

Hans

Re: Simple preformance test

[Ivan Ivanov]

Yura Umanets wrote:

> Ivan Ivanov wrote:
>
> >Oleg Drokin wrote:
> >
> >>Hello!
> >>
> >>On Thu, Apr 11, 2002 at 03:14:03PM +0300, Ivan Ivanov wrote:
> >>
> >>>                  dd if=/dev/zero of=testdir/$d1/$f1$f2$f3$f4 \
> >>>                      bs=$block_size count=$blocks_count \
> >>>                      >/dev/null 2>&1
> >>>                  rm -f testdir/$d1/$f1$f2$f3$f4
> >>>
> >>This is the explanation of your speedup.
> >>You try to write to a file, but there is not enough space and file is not
> >>created, but you did not notice.
> >>And then you delete nonexistent files, and no doubts this is very fast
> >>operation ;)
> >>
> >>BTW, if you turn off tails, your numbers might get faster, but space usage
> >>will get worse.
> >>
> >>Bye,
> >>    Oleg
> >>
> >
> >Filesystem           1k-blocks      Used Available Use% Mounted on
> >/dev/hda7              1172672   1170676      1996 100% /mnt/test
> >
> >I know this but  df  shows 1996 blocks free
> >
> >------
> >Ivan
> >
> Besides, I think you didn't use "smart" block alocation technics in your
> tests and the all ppl know, that performance degradates because of
> bitmap when used space grows up.
>
> Oleg, is this correct?
>
> --
> Yury Umanets
> IT Engineer of Priocom Corp.
> ICQ: 55494590

As it seen this was, a very simple test, so it does't test performance when the
file system is fragmented.
There was a big discussion about fragmentation/defragmentation a few days ago.
There was a message about performance degradation, so I made this just for
check.

-----
Ivan

Re: Simple preformance test

[Ivan Ivanov]

Oleg Drokin wrote:

> Hello!
>
> On Thu, Apr 11, 2002 at 05:07:31PM +0300, Ivan Ivanov wrote:
>
> > Filesystem           1k-blocks      Used Available Use% Mounted on
> > /dev/hda7              1172672   1170676      1996 100% /mnt/test
> > I know this but  df  shows 1996 blocks free
>
> These blocks cannot be used.
> They are reserved for some purpose, and there is even a code to grab
> these blocks, but the code is disabled, so each reiserfs FS have 2 Mb of data
> that cannot be allocated.
>
> Bye,
>     Oleg

When the reiserfs partition is filled up to 100% there is a bunch of messages in
logs.
I have no such messages. That was my criteria for choosing partition size - to
be allmost 100% full.

May be the performance difference is my mistake - this was just a simple test.

------
Ivan

Re: Simple preformance test

[Ivan Ivanov]

By the way, I patched my kernel with speedup patch.
Netscape loads about 30% faster - great !

Ivan

Re: Simple preformance test

[Oleg Drokin]

Hello!


> When the reiserfs partition is filled up to 100% there is a bunch of messages in
> logs.

Not anymore.
These confusing messages were eliminated.

Bye,
    Oleg

Re: Simple preformance test

[Oleg Drokin]

Hello!

   Finaly!!! At least some good news ;)

Bye,
    Oleg
On Thu, Apr 11, 2002 at 08:22:55PM +0300, Ivan Ivanov wrote:
> By the way, I patched my kernel with speedup patch.
> Netscape loads about 30% faster - great !
> 
> Ivan
> 
>