mongo benchmark results

mongo benchmark results

[David Dabbs]

At http://dabbs.net/reiser4/mongo.html I've posted some benchmarks from my
aging but available test box. All numbers were generated with the Namesys
mm7 snapshot. The "A.INFO_R4=new" are results from a reiser4 with modified
comparison functions as well as znode_contains_key_strict(). So, here are my
questions:

Do these results appear consistent with others' recent benchmarking? 
Should I be using particular mount options during benchmarking?
I would think running mongo would be necessary to ensuring that reiser4 mods
are 'safe,' but it is sufficient?
Viz the prior question, is there a recommended test regimen or regression
suite?


I started to dig into mongo a bit and noticed that it does not appear to
vary the file & dir _names_ it generates. Most appear to be ~7 characters
long with a pattern of 'f' followed by some number. Isn't this a) atypical
of file/dir name distribution and b) favorable to/biased towards the "short
name" code? IOW, if R4_LARGE_KEYS is the default and all generated test file
names' lengths < 15 characters, then the large file name code is not being
covered or benchmarked. Perhaps I'm mixing apples and oranges (code coverage
and benchmarking). 


Just curious,

David

Re: mongo benchmark results

[Hans Reiser]

David Dabbs wrote:

>At http://dabbs.net/reiser4/mongo.html I've posted some benchmarks from my
>aging but available test box. All numbers were generated with the Namesys
>mm7 snapshot. The "A.INFO_R4=new" are results from a reiser4 with modified
>comparison functions as well as znode_contains_key_strict(). So, here are my
>questions:
>
>Do these results appear consistent with others' recent benchmarking? 
>  
>
Increase file_size to 8k, and then we can compare.  This will reduce 
reiser4's space savings from 19% to 9%, but it will probably increase 
performance.

Your work had a remarkable effect on the create phase.  Very good work.  
I don't know why the impact was not larger for the stats phase.

Elena, please reproduce his results on our hardware.

>Should I be using particular mount options during benchmarking?
>I would think running mongo would be necessary to ensuring that reiser4 mods
>are 'safe,' but it is sufficient?
>Viz the prior question, is there a recommended test regimen or regression
>suite?
>
>
>I started to dig into mongo a bit and noticed that it does not appear to
>vary the file & dir _names_ it generates. Most appear to be ~7 characters
>long with a pattern of 'f' followed by some number. Isn't this a) atypical
>of file/dir name distribution and b) favorable to/biased towards the "short
>name" code? IOW, if R4_LARGE_KEYS is the default and all generated test file
>names' lengths < 15 characters,
>
You are encouraged to survey filesystems and supply a realistic filename 
generator.

In practice, most filenames are less than 15 characters. 

Another thing we need to do is add a file data generator that is based 
on slicing and dicing a linux kernel tarball, because we won't be able 
to do a good benchmark of the compression code until we do so.

> then the large file name code is not being
>covered or benchmarked. Perhaps I'm mixing apples and oranges (code coverage
>and benchmarking). 
>
>
>Just curious,
>
>David
>
>
>
>
>
>  
>

Re: mongo benchmark results

[Nikita Danilov]

Hans Reiser writes:
 > David Dabbs wrote:
 > 
 > >At http://dabbs.net/reiser4/mongo.html I've posted some benchmarks from my
 > >aging but available test box. All numbers were generated with the Namesys
 > >mm7 snapshot. The "A.INFO_R4=new" are results from a reiser4 with modified
 > >comparison functions as well as znode_contains_key_strict(). So, here are my
 > >questions:
 > >
 > >Do these results appear consistent with others' recent benchmarking? 
 > >  
 > >
 > Increase file_size to 8k, and then we can compare.  This will reduce 
 > reiser4's space savings from 19% to 9%, but it will probably increase 
 > performance.
 > 
 > Your work had a remarkable effect on the create phase.  Very good work.  
 > I don't know why the impact was not larger for the stats phase.

Isn't this strange that changes to the key comparison function
improved _real_ time more than _cpu_ time?

I don't understand this.

Nikita.

Re: mongo benchmark results

[Alex Zarochentsev]

On Mon, Jul 26, 2004 at 12:24:50PM +0400, Nikita Danilov wrote:
> Hans Reiser writes:
>  > David Dabbs wrote:
>  > 
>  > >At http://dabbs.net/reiser4/mongo.html I've posted some benchmarks from my
>  > >aging but available test box. All numbers were generated with the Namesys
>  > >mm7 snapshot. The "A.INFO_R4=new" are results from a reiser4 with modified
>  > >comparison functions as well as znode_contains_key_strict(). So, here are my
>  > >questions:
>  > >
>  > >Do these results appear consistent with others' recent benchmarking? 
>  > >  
>  > >
>  > Increase file_size to 8k, and then we can compare.  This will reduce 
>  > reiser4's space savings from 19% to 9%, but it will probably increase 
>  > performance.
>  > 
>  > Your work had a remarkable effect on the create phase.  Very good work.  
>  > I don't know why the impact was not larger for the stats phase.
> 
> Isn't this strange that changes to the key comparison function
> improved _real_ time more than _cpu_ time?

cpu time is hidden :) A foreground process may just wait, when pdflush
does all work.  We can't measure it by time(1).

> 
> I don't understand this.
> 
> Nikita.
> 

-- 
Alex.

Re: mongo benchmark results

[Nikita Danilov]

Alex Zarochentsev writes:
 > On Mon, Jul 26, 2004 at 12:24:50PM +0400, Nikita Danilov wrote:
 > > Hans Reiser writes:
 > >  > David Dabbs wrote:
 > >  > 
 > >  > >At http://dabbs.net/reiser4/mongo.html I've posted some benchmarks from my
 > >  > >aging but available test box. All numbers were generated with the Namesys
 > >  > >mm7 snapshot. The "A.INFO_R4=new" are results from a reiser4 with modified
 > >  > >comparison functions as well as znode_contains_key_strict(). So, here are my
 > >  > >questions:
 > >  > >
 > >  > >Do these results appear consistent with others' recent benchmarking? 
 > >  > >  
 > >  > >
 > >  > Increase file_size to 8k, and then we can compare.  This will reduce 
 > >  > reiser4's space savings from 19% to 9%, but it will probably increase 
 > >  > performance.
 > >  > 
 > >  > Your work had a remarkable effect on the create phase.  Very good work.  
 > >  > I don't know why the impact was not larger for the stats phase.
 > > 
 > > Isn't this strange that changes to the key comparison function
 > > improved _real_ time more than _cpu_ time?
 > 
 > cpu time is hidden :) A foreground process may just wait, when pdflush
 > does all work.  We can't measure it by time(1).

In this case real time is less than twice larger than cpu time, but
speedup is 9 times larger. Even if all (real - cpu) time were spent by
mongo process waiting for pdflush that was only doing CPU work, such
difference couldn't be explained.

 > 
 > > 
 > > I don't understand this.
 > > 

Nikita.

 > > 
 > 
 > -- 
 > Alex.

Re: mongo benchmark results

[Alex Zarochentsev]

On Sun, Jul 25, 2004 at 12:21:37PM +0400, Nikita Danilov wrote:
> Alex Zarochentsev writes:
>  > On Mon, Jul 26, 2004 at 12:24:50PM +0400, Nikita Danilov wrote:
>  > > Hans Reiser writes:
>  > >  > David Dabbs wrote:
>  > >  > 
>  > >  > >At http://dabbs.net/reiser4/mongo.html I've posted some benchmarks from my
>  > >  > >aging but available test box. All numbers were generated with the Namesys
>  > >  > >mm7 snapshot. The "A.INFO_R4=new" are results from a reiser4 with modified
>  > >  > >comparison functions as well as znode_contains_key_strict(). So, here are my
>  > >  > >questions:
>  > >  > >
>  > >  > >Do these results appear consistent with others' recent benchmarking? 
>  > >  > >  
>  > >  > >
>  > >  > Increase file_size to 8k, and then we can compare.  This will reduce 
>  > >  > reiser4's space savings from 19% to 9%, but it will probably increase 
>  > >  > performance.
>  > >  > 
>  > >  > Your work had a remarkable effect on the create phase.  Very good work.  
>  > >  > I don't know why the impact was not larger for the stats phase.
>  > > 
>  > > Isn't this strange that changes to the key comparison function
>  > > improved _real_ time more than _cpu_ time?
>  > 
>  > cpu time is hidden :) A foreground process may just wait, when pdflush
>  > does all work.  We can't measure it by time(1).
> 
> In this case real time is less than twice larger than cpu time, but
> speedup is 9 times larger. Even if all (real - cpu) time were spent by
> mongo process waiting for pdflush that was only doing CPU work, such
> difference couldn't be explained.

Who said that new code improves CPU time for backgound pdflush/flush not more
than it improves foreground reiser_fract_tree?  It seems that the flush code
contains more key comparion ops than reiser4_write() which is expected to
spent most CPU time in copy_from_user().

-- 
Alex.

Re: mongo benchmark results

[Alex Zarochentsev]

On Mon, Jul 26, 2004 at 12:43:15PM +0400, Alex Zarochentcev wrote:
> On Mon, Jul 26, 2004 at 12:24:50PM +0400, Nikita Danilov wrote:
> > Hans Reiser writes:
> >  > David Dabbs wrote:
> >  > 
> >  > >At http://dabbs.net/reiser4/mongo.html I've posted some benchmarks from my
> >  > >aging but available test box. All numbers were generated with the Namesys
> >  > >mm7 snapshot. The "A.INFO_R4=new" are results from a reiser4 with modified
> >  > >comparison functions as well as znode_contains_key_strict(). So, here are my
> >  > >questions:
> >  > >
> >  > >Do these results appear consistent with others' recent benchmarking? 
> >  > >  
> >  > >
> >  > Increase file_size to 8k, and then we can compare.  This will reduce 
> >  > reiser4's space savings from 19% to 9%, but it will probably increase 
> >  > performance.
> >  > 
> >  > Your work had a remarkable effect on the create phase.  Very good work.  
> >  > I don't know why the impact was not larger for the stats phase.
> > 
> > Isn't this strange that changes to the key comparison function
> > improved _real_ time more than _cpu_ time?
> 
> cpu time is hidden :) A foreground process may just wait, when pdflush
> does all work.  We can't measure it by time(1).

I think we should calculate cpu utilization from /proc/stat values
same way as vmstat(8) does.

-- 
Alex.

Re: mongo benchmark results

[Alex Zarochentsev]

On Sun, Jul 25, 2004 at 11:48:19PM -0700, Hans Reiser wrote:
> David Dabbs wrote:
> 
> >At http://dabbs.net/reiser4/mongo.html I've posted some benchmarks from my
> >aging but available test box. All numbers were generated with the Namesys
> >mm7 snapshot. The "A.INFO_R4=new" are results from a reiser4 with modified
> >comparison functions as well as znode_contains_key_strict(). So, here are 
> >my
> >questions:
> >
> >Do these results appear consistent with others' recent benchmarking? 
> > 
> >
> Increase file_size to 8k, and then we can compare.  This will reduce 
> reiser4's space savings from 19% to 9%, but it will probably increase 
> performance.
> 
> Your work had a remarkable effect on the create phase.  Very good work.  
> I don't know why the impact was not larger for the stats phase.
> 
> Elena, please reproduce his results on our hardware.

2 Lena:

I have added a cpu utilization stat (ratio of system time for all cpus to total
time, in percents) to mongo.pl.

Please take new mongo before repeating the benchmark.

-- 
Alex.

RE: mongo benchmark results

[David Dabbs]

> -----Original Message-----
> From: Alex Zarochentsev [mailto:zam@namesys.com]
> Sent: Monday, July 26, 2004 12:09 PM
>
> 2 Lena:
> 
> I have added a cpu utilization stat (ratio of system time for all cpus to
> total time, in percents) to mongo.pl.
> 
> Please take new mongo before repeating the benchmark.
> 
> --
> Alex.

For what it's worth, I reproduced the mongo runs with 8k files as Hans
suggested. Results are at 

http://dabbs.net/reiser4/mongo8kfiles.html

The CREATE performance increase seems to have evaporated, though perhaps
that was what Hans was looking to test with the updated config. I was in a
hurry this morning and I may have varied the test slightly. I'll use Zam's
new mongo when I get home this evening. I will as well test copying and
compiling the kernel. Unless there's another recommended way to do this, I
I'll substitute the output from "time -c" into a mongo log file so
mongo_parser can compare the results.

Zam/Elena where should I look for the new mongo tgz? I don't see it at
http://thebsh.namesys.com/benchmarks/.

Thanks for the updated mongo.

david

 

Re: mongo benchmark results

[Hans Reiser]

David Dabbs wrote:

>>-----Original Message-----
>>From: Alex Zarochentsev [mailto:zam@namesys.com]
>>Sent: Monday, July 26, 2004 12:09 PM
>>
>>2 Lena:
>>
>>I have added a cpu utilization stat (ratio of system time for all cpus to
>>total time, in percents) to mongo.pl.
>>
>>Please take new mongo before repeating the benchmark.
>>
>>--
>>Alex.
>>    
>>
>
>For what it's worth, I reproduced the mongo runs with 8k files as Hans
>suggested. Results are at 
>
>http://dabbs.net/reiser4/mongo8kfiles.html
>
>The CREATE performance increase seems to have evaporated, though perhaps
>that was what Hans was looking to test with the updated config. I was in a
>hurry this morning and I may have varied the test slightly. I'll use Zam's
>new mongo when I get home this evening. I will as well test copying and
>compiling the kernel. Unless there's another recommended way to do this, I
>I'll substitute the output from "time -c" into a mongo log file so
>mongo_parser can compare the results.
>
>Zam/Elena where should I look for the new mongo tgz? I don't see it at
>http://thebsh.namesys.com/benchmarks/.
>
>Thanks for the updated mongo.
>
>david
>
> 
>
>
>
>  
>
The 4k results are also valid/important results.  I just tend to choose 
8k for the website for various valid reasons.

We probably should try 500 byte file sizes and 50mb file sizes, with 
gamma of 0, just for fun.

Hans

RE: mongo benchmark results

[David Dabbs]

> -----Original Message-----
> From: Hans Reiser 
> Sent: Monday, July 26, 2004 2:12 PM
> Subject: Re: mongo benchmark results
> 
>
> The 4k results are also valid/important results.  I just tend to choose
> 8k for the website for various valid reasons.
> 
> We probably should try 500 byte file sizes and 50mb file sizes, with
> gamma of 0, just for fun.
> 
> Hans

BTW, I mentioned this in a prior post, but will recap. I get the following
error when attempting to run mongo with OVERWRITE, MODIFY or APPEND = 'on'

...
Use of uninitialized value in concatenation (.) or string at ./mongo.pl line
562.
PHASE OVERWRITE
...

I modified mongo to LOG the command it was attempting to execute. Though I
don't have an example (I'm at work), it appeared that it wasn't correctly
building the command to time, hence the error parsing the time results. I
looked over the parameter checking and didn't see evidence that one should
set these phases to anything but on/off. Since I had limited time and wanted
to get through at least a partial run I punted and simply set these phases
'off.'

david

Re: mongo benchmark results

[Hans Reiser]

David Dabbs wrote:

>>-----Original Message-----
>>From: Alex Zarochentsev [mailto:zam@namesys.com]
>>Sent: Monday, July 26, 2004 12:09 PM
>>
>>2 Lena:
>>
>>I have added a cpu utilization stat (ratio of system time for all cpus to
>>total time, in percents) to mongo.pl.
>>
>>Please take new mongo before repeating the benchmark.
>>
>>--
>>Alex.
>>    
>>
>
>For what it's worth, I reproduced the mongo runs with 8k files as Hans
>suggested. Results are at 
>
>http://dabbs.net/reiser4/mongo8kfiles.html
>
>The CREATE performance increase seems to have evaporated, though perhaps
>that was what Hans was looking to test with the updated config. I was in a
>hurry this morning and I may have varied the test slightly. I'll use Zam's
>new mongo when I get home this evening. I will as well test copying and
>compiling the kernel. Unless there's another recommended way to do this, I
>I'll substitute the output from "time -c" into a mongo log file so
>mongo_parser can compare the results.
>
>Zam/Elena where should I look for the new mongo tgz? I don't see it at
>http://thebsh.namesys.com/benchmarks/.
>
>Thanks for the updated mongo.
>
>david
>
> 
>
>
>
>  
>
Ignore what I said about 500mb mongo, the dd gives the info desired.

I don't understand these results.  If they are correct, we have lost a 
lot of our ext3 performance advantage, and I want to know where it 
went.  Elena, Zam, please comment.

Reiser4 slowed down, Elena has found the changesets responsible, we will fix it.....

[Hans Reiser]

I thought the list would be interested to know.

Zam, this is your task, except for the debugging code, which is vs's.

vs, I want you to ensure that it is not possible to leave debugging code 
on without being warned of it.

Good work elena.

Hans

E. Gryaznova wrote:

>
>>>
>> So, is reiser4 slower or is ext3 faster....?
>
>
> Hello.
>
> I tried to find what reiser4 changes cause the reiser4 CREATE 
> performance degradation.
>
> There is a "reiser4, CREATE" chart :
>
> http://www.namesys.com/intbenchmarks/mongo/04.07.26/256MB.RAM/reiser4-performance-degradation/charts/CREATE.png 
>
>
> (while reading this chart, please notice that some tests were 
> performed twice or more times)
>
> My conclusion is :
> 1) the first degradation was invited by 1.1614.1.1 snapshot :
> ChangeSet@1.1614.1.1, 2004-06-25 16:33:37+04:00, zam@crimson.namesys.com
>  flush_some_atom: force atom to commit if jnode_flush() does not 
> submit nodes to disk even if it
>  has progress in processing nodes (in case of large fragmented 
> overwrite set, for example).
>
> 2) for the second one the "vs debugging code" is responsible. It is 
> started from
> ChangeSet@1.1623, 2004-07-20 12:08:38+04:00, 
> reiser4@tribesman.namesys.com
>  debugging code
>
> Thanks,
> Lena.
>
>
>>
>> Zam, please comment.
>>
>> Hans
>>
>>
>
>
>
>

Reiser4 performance is almost completely restored now

[Hans Reiser]

If you look at the benchmarks cited below, we have our performance back, 
and the slight difference in performance remaining might be within 
normal benchmark variation, we have to run it again and check.

Hans

E. Gryaznova wrote:

> Alex Zarochentsev wrote:
>
>> On Thu, Jul 29, 2004 at 03:17:13AM +0400, Elena Gryaznova wrote:
>>  
>>
>>> Hans Reiser wrote:
>>>
>>>   
>>>
>>>> E. Gryaznova wrote:
>>>>
>>>>     
>>>>
>>>>> Hans Reiser wrote:
>>>>>
>>>>>       
>>>>
> [skipped]
>
>>> I tried to find what reiser4 changes cause the reiser4 CREATE 
>>> performance degradation.
>>>
>>> There is a "reiser4, CREATE" chart :
>>>
>>> http://www.namesys.com/intbenchmarks/mongo/04.07.26/256MB.RAM/reiser4-performance-degradation/charts/CREATE.png 
>>>
>>>
>>> (while reading this chart, please notice that some tests were 
>>> performed twice or more times)
>>>
>>> My conclusion is :
>>> 1) the first degradation was invited by 1.1614.1.1 snapshot :
>>> ChangeSet@1.1614.1.1, 2004-06-25 16:33:37+04:00, 
>>> zam@crimson.namesys.com
>>> flush_some_atom: force atom to commit if jnode_flush() does not 
>>> submit nodes to disk even if it
>>> has progress in processing nodes (in case of large fragmented 
>>> overwrite set, for example).
>>>   
>>
>>
>> Yes. I know it might be too strong commit policy.
>> I have ideas how to fix the fix -- we can repeat jnode_flush() until 
>> we submit
>> something to disk -- if one slum goes into overwrite set completely 
>> second one
>> may have something sorted to relocate set. Current code commits atom 
>> after first
>> jnode_flush which has submitted to disk nothing.
>>
>>  
>>
>
> Performance is restored :
>
> http://thebsh.namesys.com/intbenchmarks/mongo/04.07.26/256MB.RAM/reiser4-performance-degradation/charts/CREATE-performance-back.png 
>
>
> http://thebsh.namesys.com/intbenchmarks/mongo/04.07.26/256MB.RAM/reiser4-performance-degradation/r4-create-performance-back.txt 
>
>
> Thanks,
> Lena
>
>>  
>>
>>> 2) for the second one the "vs debugging code" is responsible. It is 
>>> started from
>>> ChangeSet@1.1623, 2004-07-20 12:08:38+04:00, 
>>> reiser4@tribesman.namesys.com
>>> debugging code
>>>
>>> Thanks,
>>> Lena.
>>>
>>>   
>>
>>
>>  
>>
>
>
>
>

Re: mongo benchmark results

[E. Gryaznova]

Hello,

David Dabbs wrote:

>>-----Original Message-----
>>From: Alex Zarochentsev [mailto:zam@namesys.com]
>>Sent: Monday, July 26, 2004 12:09 PM
>>
>>2 Lena:
>>
>>I have added a cpu utilization stat (ratio of system time for all cpus to
>>total time, in percents) to mongo.pl.
>>
>>Please take new mongo before repeating the benchmark.
>>
>>--
>>Alex.
>>    
>>
>
>For what it's worth, I reproduced the mongo runs with 8k files as Hans
>suggested. Results are at 
>
>http://dabbs.net/reiser4/mongo8kfiles.html
>  
>
I will let you know when my current benchmarks set will be finished.
BTW, was reiserfs mounted default, w/o notail? If yes, would you please 
try notail too?

>The CREATE performance increase seems to have evaporated, though perhaps
>that was what Hans was looking to test with the updated config. I was in a
>hurry this morning and I may have varied the test slightly. I'll use Zam's
>new mongo when I get home this evening. I will as well test copying and
>compiling the kernel. Unless there's another recommended way to do this, I
>I'll substitute the output from "time -c" into a mongo log file so
>mongo_parser can compare the results.
>  
>
would you please reboot before the each  test?
 

>Zam/Elena where should I look for the new mongo tgz? I don't see it at
>http://thebsh.namesys.com/benchmarks/.
>  
>
is updated, try please :
http://thebsh.namesys.com/benchmarks/mongo-2004.07.26.tar.gz

Thanks,
Lena

>dated mongo.
>
>david
>
> 
>
>
>
>  
>

RE: mongo benchmark results

[David Dabbs]

> -----Original Message-----
> From: E. Gryaznova [mailto:grev@namesys.com]
> Sent: Monday, July 26, 2004 2:41 PM

Okay, here's a recap of the mongo suggestions that I've received:

Hans:
...change file size to 8000
...try 500 byte file sizes, with gamma of 0, just for fun.

Elena:
...run reiser4 with and without 'notail'
...would you please reboot before each test?

Since I have limited time [mother-in-law incoming ;-)] I will run

FILE_SIZE=500, 4000, 8000 	<in that order>
GAMMA=0, .2				<in that order>
R4_INFO=notail & default	<in that order>

For the notail runs I assume that notail should also be specified with
reiserfs as well. Any corresponding option for ext3 or use its default as I
have been doing?

Regarding rebooting, are you requesting that I reboot before each
_individual_ run or between 'multi-run' batches 
(e.g. #sh:/ <options> RUN <options> RUN...?)

david

Re: mongo benchmark results

[E. Gryaznova]

David Dabbs wrote:

>>-----Original Message-----
>>From: E. Gryaznova [mailto:grev@namesys.com]
>>Sent: Monday, July 26, 2004 2:41 PM
>>    
>>
>
>Okay, here's a recap of the mongo suggestions that I've received:
>
>Hans:
>...change file size to 8000
>...try 500 byte file sizes, with gamma of 0, just for fun.
>
>Elena:
>...run reiser4 with and without 'notail'
>  
>
not reiser4, but reiserfs notail :)
reiser4 default is enough.

>...would you please reboot before each test?
>
>Since I have limited time [mother-in-law incoming ;-)] I will run
>
>FILE_SIZE=500, 4000, 8000 	<in that order>
>
it does not matter 4k or 8k, I think.  So,  you can try to repeat 4k 
benchmark just to be sure that your previous results were correct (if 
you want, of course).

>GAMMA=0, .2				<in that order>
>R4_INFO=notail & default	<in that order>
>  
>

>For the notail runs I assume that notail should also be specified with
>reiserfs as well. Any corresponding option for ext3 or use its default as I
>have been doing?
>  
>
ext3 default is ok.

Thanks,
Lena

>Regarding rebooting, are you requesting that I reboot before each
>_individual_ run or between 'multi-run' batches 
>(e.g. #sh:/ <options> RUN <options> RUN...?)
>
>david
>
>
>
>
>
>  
>

RE: mongo benchmark results

[David Dabbs]

> >
> >Elena:
> >...run reiser4 with and without 'notail'
> >
> >...would you please reboot before each test?
> >
> >
> it does not matter 4k or 8k, I think.  So, you can try to repeat 4k
> benchmark just to be sure that your previous results were correct (if
> you want, of course).
> 
> >
> ext3 default is ok.
> 
> Thanks,
> Lena
>

Elena,

I've reproduced my original mongo benchmarks using the 'cpu util' mongo you
posted. The results are at http://dabbs.net/reiser4/mongo_updated.html. I
was careful to use the same kernel for all the runs except for one (the
reiser4 "mm7" run), rebooting before each test. For some reason I could not
get the CREATE phase to finish with GAMMA=0 using ext3, so I didn't move
forward with that. Will investigate more when I get the time to do so.


Best,

david

Re: mongo benchmark results

[E. Gryaznova]

David Dabbs wrote:

>>>Elena:
>>>...run reiser4 with and without 'notail'
>>>
>>>...would you please reboot before each test?
>>>
>>>
>>>      
>>>
>>it does not matter 4k or 8k, I think.  So, you can try to repeat 4k
>>benchmark just to be sure that your previous results were correct (if
>>you want, of course).
>>
>>    
>>
>>ext3 default is ok.
>>
>>Thanks,
>>Lena
>>
>>    
>>
>
>Elena,
>
>I've reproduced my original mongo benchmarks using the 'cpu util' mongo you
>posted. The results are at http://dabbs.net/reiser4/mongo_updated.html. I
>was careful to use the same kernel for all the runs except for one (the
>reiser4 "mm7" run), rebooting before each test. For some reason I could not
>get the CREATE phase to finish with GAMMA=0 using ext3, so I didn't move
>forward with that. Will investigate more when I get the time to do so.
>
>  
>
Hello, David.

what are PHASE_MKFILES and PHASE_MKDIRS in your benchmark? Did you use 
not original mongo? Would you please send  us
the modified one?

Thanks,
Lena
 

>Best,
>
>david
>
>
>
>
>
>  
>

RE: mongo benchmark results

[David Dabbs]

> >
> Hello, David.
> 
> what are PHASE_MKFILES and PHASE_MKDIRS in your benchmark? Did you use
> not original mongo? Would you please send  us
> the modified one?
> 
> Thanks,
> Lena
> 

Hi Lena, 

PHASE_MKDIRS and PHASE_MKFILES were attempts on my part to inject some
realistic filesystem noise into the mongo tests. I simply listed all the
directories / files on my computer into files. I created two new programs,
mongo_mkfiles and mongo_mkdirs that read a list from stdin and create each
directory or file (zero length). I disabled these phases in the runs I
published as my test disk space is limited and I didn't want to muddy the
waters with non-standard mongo settings. I'll be happy to send you the
sources later today. Since the phases were disabled, essentially the results
were generated by 'standard' mongo (well, the cpu_util version) sources.

By the way, great work on finding the patch that had been slowing things
down.


David

RE: Reiser4 performance is almost completely restored now

[David Dabbs]

>If you look at the benchmarks cited below, we have our performance back, 
>and the slight difference in performance remaining might be within 
>normal benchmark variation, we have to run it again and check.
>
>Hans

Are the patches that restore r4 performance included in the latest
auto-snapshot (7-29) or is an updated patch forthcoming? I plan to run some
mongo tests and want to ensure that I include the fixes.

david

Re: Reiser4 performance is almost completely restored now

[Alex Zarochentsev]

On Fri, Jul 30, 2004 at 01:57:39PM -0500, David Dabbs wrote:
> 
> >If you look at the benchmarks cited below, we have our performance back, 
> >and the slight difference in performance remaining might be within 
> >normal benchmark variation, we have to run it again and check.
> >
> >Hans
> 
> Are the patches that restore r4 performance included in the latest
> auto-snapshot (7-29) or is an updated patch forthcoming? I plan to run some

7-30 will include the fixes.

> mongo tests and want to ensure that I include the fixes.
> 
> david

-- 
Alex.

Unattended mongo benchmark?

[David Dabbs]

I was wondering about a solution to running mongo benchmarks unattended.
If one does not exist, how about the following as a 'straw man' approach?

Create a script (say mongo_auto.sh) that takes one numeric parameter. The
parameter would indicate which action/mongo script to perform next. The
parameters and their associated actions might be something like the
following:

Parameter: N=0
- Make a copy of /etc/inittab
- Add a 'sysinit' entry in /etc/inittab that runs "/path/mongo_auto.sh 1"
- Reboot


Parameter: N
- Run the Nth mongo run (specified in a case statement in mongo_auto.sh)0.
- If (ran_without_error)
     N = N + 1
     if ( N <= Last_Run_Number)
        - Update /etc/inittab entry to run "/path/mongo_auto.sh N"
        - Reboot
     else
        - Clean up/restore /etc/inittab
        - Run /path/to/mongo_parse.sh 1.log 2.log... > comparison.html


Or, to limit inittab mods to the start/end, the script could read/write a
.mongo_auto file that specifies the next run number to take.


David

Re: Unattended mongo benchmark?

[Hans Reiser]

David Dabbs wrote:

>I was wondering about a solution to running mongo benchmarks unattended.
>If one does not exist, how about the following as a 'straw man' approach?
>
>Create a script (say mongo_auto.sh) that takes one numeric parameter. The
>parameter would indicate which action/mongo script to perform next. The
>parameters and their associated actions might be something like the
>following:
>
>Parameter: N=0
>- Make a copy of /etc/inittab
>- Add a 'sysinit' entry in /etc/inittab that runs "/path/mongo_auto.sh 1"
>- Reboot
>
>
>Parameter: N
>- Run the Nth mongo run (specified in a case statement in mongo_auto.sh)0.
>- If (ran_without_error)
>     N = N + 1
>     if ( N <= Last_Run_Number)
>        - Update /etc/inittab entry to run "/path/mongo_auto.sh N"
>        - Reboot
>     else
>        - Clean up/restore /etc/inittab
>        - Run /path/to/mongo_parse.sh 1.log 2.log... > comparison.html
>
>
>Or, to limit inittab mods to the start/end, the script could read/write a
>.mongo_auto file that specifies the next run number to take.
>
>
>David
>
>
>
>
>
>  
>
sounds good.

Re: Unattended mongo benchmark?

[Nikita Danilov]

Hans Reiser writes:
 > David Dabbs wrote:
 > 
 > >I was wondering about a solution to running mongo benchmarks unattended.
 > >If one does not exist, how about the following as a 'straw man' approach?
 > >
 > >Create a script (say mongo_auto.sh) that takes one numeric parameter. The
 > >parameter would indicate which action/mongo script to perform next. The
 > >parameters and their associated actions might be something like the
 > >following:
 > >
 > >Parameter: N=0
 > >- Make a copy of /etc/inittab
 > >- Add a 'sysinit' entry in /etc/inittab that runs "/path/mongo_auto.sh 1"
 > >- Reboot
 > >
 > >
 > >Parameter: N
 > >- Run the Nth mongo run (specified in a case statement in mongo_auto.sh)0.
 > >- If (ran_without_error)
 > >     N = N + 1
 > >     if ( N <= Last_Run_Number)
 > >        - Update /etc/inittab entry to run "/path/mongo_auto.sh N"
 > >        - Reboot
 > >     else
 > >        - Clean up/restore /etc/inittab
 > >        - Run /path/to/mongo_parse.sh 1.log 2.log... > comparison.html
 > >
 > >
 > >Or, to limit inittab mods to the start/end, the script could read/write a
 > >.mongo_auto file that specifies the next run number to take.
 > >
 > >
 > >David
 > >
 > >
 > >
 > >
 > >
 > >  
 > >
 > sounds good.

Yes, Namesys has scripts to run arbitrary sequence of benchmarks with
reboots in between. These scripts plug themselves into
/etc/rc.d/boot.local rather than in /etc/inittab though.

 > 

Nikita.

RE: Unattended mongo benchmark?

[David Dabbs]

Nikita,

would you mind including them in the mongo distribution or forwarding them
to me? 

David

> -----Original Message-----
> From: Nikita Danilov [mailto:Nikita@Clusterfs.COM]
> Sent: Saturday, July 31, 2004 6:01 AM
> To: Hans Reiser
> Cc: David Dabbs; reiserfs-list@namesys.com
> Subject: Re: Unattended mongo benchmark?
> 

>  > sounds good.
> 
> Yes, Namesys has scripts to run arbitrary sequence of benchmarks with
> reboots in between. These scripts plug themselves into
> /etc/rc.d/boot.local rather than in /etc/inittab though.
> 
>  >
> 
> Nikita.

Re: Unattended mongo benchmark?

[Nikita Danilov]

David Dabbs writes:
 > Nikita,
 > 
 > would you mind including them in the mongo distribution or forwarding them
 > to me? 

You should ask Namesys folk about this.

 > 
 > David
 > 
 > > -----Original Message-----
 > > From: Nikita Danilov [mailto:Nikita@Clusterfs.COM]
 > > Sent: Saturday, July 31, 2004 6:01 AM
 > > To: Hans Reiser
 > > Cc: David Dabbs; reiserfs-list@namesys.com
 > > Subject: Re: Unattended mongo benchmark?
 > > 
 > 
 > >  > sounds good.
 > > 
 > > Yes, Namesys has scripts to run arbitrary sequence of benchmarks with
 > > reboots in between. These scripts plug themselves into
 > > /etc/rc.d/boot.local rather than in /etc/inittab though.
 > > 
 > >  >
 > > 

Nikita.

Re: Unattended mongo benchmark?

[Vladimir V. Saveliev]

[-- Attachment #1: Type: text/plain, Size: 646 bytes --]

David Dabbs wrote:
> Nikita,
> 
> would you mind including them in the mongo distribution or forwarding them
> to me? 
> 
I hope this is a right thing

> David
> 
> 
>>-----Original Message-----
>>From: Nikita Danilov [mailto:Nikita@Clusterfs.COM]
>>Sent: Saturday, July 31, 2004 6:01 AM
>>To: Hans Reiser
>>Cc: David Dabbs; reiserfs-list@namesys.com
>>Subject: Re: Unattended mongo benchmark?
>>
> 
> 
>> > sounds good.
>>
>>Yes, Namesys has scripts to run arbitrary sequence of benchmarks with
>>reboots in between. These scripts plug themselves into
>>/etc/rc.d/boot.local rather than in /etc/inittab though.
>>
>> >
>>
>>Nikita.
> 
> 
> 
> 


[-- Attachment #2: bench.tar.gz --]
[-- Type: application/x-gzip, Size: 2829 bytes --]

Re: Unattended mongo benchmark?

[Nikita Danilov]

Vladimir V. Saveliev writes:
 > David Dabbs wrote:
 > > Nikita,
 > > 
 > > would you mind including them in the mongo distribution or forwarding them
 > > to me? 
 > > 
 > I hope this is a right thing

Yes, it was.

 > 
 > > David
 > > 
 > > 
 > >>-----Original Message-----
 > >>From: Nikita Danilov [mailto:Nikita@Clusterfs.COM]
 > >>Sent: Saturday, July 31, 2004 6:01 AM
 > >>To: Hans Reiser
 > >>Cc: David Dabbs; reiserfs-list@namesys.com
 > >>Subject: Re: Unattended mongo benchmark?
 > >>
 > > 
 > > 
 > >> > sounds good.
 > >>
 > >>Yes, Namesys has scripts to run arbitrary sequence of benchmarks with
 > >>reboots in between. These scripts plug themselves into
 > >>/etc/rc.d/boot.local rather than in /etc/inittab though.
 > >>
 > >> >
 > >>

Nikita.

 > > 
 > > 
 > > 
 > > 
 > 

RE: Unattended mongo benchmark?

[David Dabbs]

> -----Original Message-----
> From: Nikita Danilov [mailto:nikita@clusterfs.com]
> Sent: Friday, July 30, 2004 4:59 PM
> To: Vladimir V. Saveliev
> Cc: David Dabbs; reiserfs-list@namesys.com
> Subject: Re: Unattended mongo benchmark?
> 
> Vladimir V. Saveliev writes:
>  > David Dabbs wrote:
>  > > Nikita,
>  > >
>  > > would you mind including them in the mongo distribution or forwarding
>  > > to me?
>  > >
>  > I hope this is a right thing
> 
> Yes, it was.
> 

Spasiba to you both.

David

Re: Unattended mongo benchmark?

[Vladimir V. Saveliev]

Hello

David Dabbs wrote:
>>-----Original Message-----
>>From: Nikita Danilov [mailto:nikita@clusterfs.com]
>>Sent: Friday, July 30, 2004 4:59 PM
>>To: Vladimir V. Saveliev
>>Cc: David Dabbs; reiserfs-list@namesys.com
>>Subject: Re: Unattended mongo benchmark?
>>
>>Vladimir V. Saveliev writes:
>> > David Dabbs wrote:
>> > > Nikita,
>> > >
>> > > would you mind including them in the mongo distribution or forwarding
>> > > to me?
>> > >
>> > I hope this is a right thing
>>
>>Yes, it was.
>>
> 
> 
> Spasiba to you both.
> 

It spells: spasibo

> David
> 
> 
> 
> 
> 

Re: mongo benchmark results

[Hans Reiser]

It looks like either ext3 gained performance or we lost some recently.   
Elena is still testing to narrow that down some, and will email her 
results when they are less preliminary.

Hans