File system shootout v2.0...

File system shootout v2.0...

[Mike Benoit]

Since the last shootout was so popular, I was able to get my hands on
some better hardware to run even more tests on. As well the generated
results format was improved slightly. 

Bonnie++ and IOZone results on a AMD Opteron 240 system are now posted
for your viewing pleasure at: http://fsbench.netnation.com/

-- 
Best Regards,
 
Mike Benoit
NetNation Communications Inc.
Systems Engineer
Tel: 604-684-6892 or 888-983-6600
 ---------------------------------------
 
 Disclaimer: Opinions expressed here are my own and not 
 necessarily those of my employer

Re: File system shootout v2.0...

[Hans Reiser]

Mike Benoit wrote:

>Since the last shootout was so popular, I was able to get my hands on
>some better hardware to run even more tests on. As well the generated
>results format was improved slightly. 
>
>Bonnie++ and IOZone results on a AMD Opteron 240 system are now posted
>for your viewing pleasure at: http://fsbench.netnation.com/
>
>  
>
We have a new snapshot on the way that does much better at this 
benchmark (we fixed a bug thanks to spotting a performance anomaly while 
replicating your settings, and the snapshot you are using is actually 
quite old thanks to our neglect of posting snapshots).

I will be interested in seeing whether once you and us are using the 
same snapshot we get the same results.;-) 

It is a bit odd that reiser4 has both the lowest run time and the worst 
rating.  We use middle of the road 32 bit CPUs (we'd like to use the 
latest ones but....;-) ).  For them our CPU consumption seems reasonable.

-- 
Hans

Re: File system shootout v2.0...

[Ookhoi]

Mike Benoit wrote (ao):
> Since the last shootout was so popular, I was able to get my hands on
> some better hardware to run even more tests on. As well the generated
> results format was improved slightly. 
> 
> Bonnie++ and IOZone results on a AMD Opteron 240 system are now posted
> for your viewing pleasure at: http://fsbench.netnation.com/

What kernel is used? I can't find it, but then again, I'm still quite
sleepy.
Nice AMD system btw. Would love to play with one.

Re: File system shootout v2.0...

[Mike Benoit]

On Thu, 2003-10-16 at 23:46, Ookhoi wrote:
> Mike Benoit wrote (ao):
> > Since the last shootout was so popular, I was able to get my hands on
> > some better hardware to run even more tests on. As well the generated
> > results format was improved slightly. 
> > 
> > Bonnie++ and IOZone results on a AMD Opteron 240 system are now posted
> > for your viewing pleasure at: http://fsbench.netnation.com/
> 
> What kernel is used? I can't find it, but then again, I'm still quite
> sleepy.
> Nice AMD system btw. Would love to play with one.

If you scroll over to the far right, it shows the kernel version on each
row. Currently its 2.6.0-test5.

-- 
Best Regards,
 
Mike Benoit
NetNation Communications Inc.
Systems Engineer
Tel: 604-684-6892 or 888-983-6600
 ---------------------------------------
 
 Disclaimer: Opinions expressed here are my own and not 
 necessarily those of my employer

Re: File system shootout v2.0...

[Mike Benoit]

On Thu, 2003-10-16 at 23:03, Hans Reiser wrote:
> Mike Benoit wrote:
> 
> >Since the last shootout was so popular, I was able to get my hands on
> >some better hardware to run even more tests on. As well the generated
> >results format was improved slightly. 
> >
> >Bonnie++ and IOZone results on a AMD Opteron 240 system are now posted
> >for your viewing pleasure at: http://fsbench.netnation.com/
> >
> >  
> >
> We have a new snapshot on the way that does much better at this 
> benchmark (we fixed a bug thanks to spotting a performance anomaly while 
> replicating your settings, and the snapshot you are using is actually 
> quite old thanks to our neglect of posting snapshots).
> 
> I will be interested in seeing whether once you and us are using the 
> same snapshot we get the same results.;-) 
> 
> It is a bit odd that reiser4 has both the lowest run time and the worst 
> rating.  We use middle of the road 32 bit CPUs (we'd like to use the 
> latest ones but....;-) ).  For them our CPU consumption seems reasonable.

I assume your referring to the "overall rating" column? The only reason
of course is because Reiser4 uses the most CPU of any other FS I've
tested. The thing I find interesting is that Reiser4 doesn't seem to
perform near as well with IOZone, as it does with Bonnie++, Infact
Reiser3 does slightly better in every IOZone test. 

I look forward to taking the new snapshot out for a test drive. ;) 

PS. Even though I'm running an opteron, I'm have not compiled the kernel
or any other app with a 64bit compiler.

-- 
Best Regards,
 
Mike Benoit
NetNation Communications Inc.
Systems Engineer
Tel: 604-684-6892 or 888-983-6600
 ---------------------------------------
 
 Disclaimer: Opinions expressed here are my own and not 
 necessarily those of my employer

Re: File system shootout v2.0...

[Hans Reiser]

Mike Benoit wrote:

>On Thu, 2003-10-16 at 23:03, Hans Reiser wrote:
>  
>
>
> The thing I find interesting is that Reiser4 doesn't seem to
>perform near as well with IOZone, as it does with Bonnie++, Infact
>Reiser3 does slightly better in every IOZone test. 
>
>
>  
>
To do well at IOzone, we need to carefully code to optimize for doing 
small amounts of work with every system call, and make the code very 
lightweight when doing so.  We would probably need to make the code 
uglier and less well abstracted to do this. 

I think I would rather add lots of plugins instead.  We probably will 
make a modest effort to improve at IOzone, but not a big one.

If I can produce a filesystem that has more plugins and more features, 
and wins at performance for disk IO intensive workloads, that is 
probably more valuable than winning at both disk IO intensive and cache 
intensive work loads.  At least that is my thinking.  Right now I think 
that our performance is good enough that we should concentrate mostly on 
debugging (fixing a few performance anomalies as we find them though), 
ship what we have so users can use, and then focus on creating the 
semantics that will make LongHorn's SQL look quite cumbersome in 
comparison when operating on semi-structured data.

Performance is not my goal.  Performance is just a way point that I need 
to reach so that I can resupply with cash for payroll purposes, get 
enough market share that users will adopt semantic changes I introduce, 
and then I can go LongHorn hunting.

-- 
Hans

Re: File system shootout v2.0...

[Mike Benoit]

On Fri, 2003-10-17 at 11:31, Hans Reiser wrote:
> Mike Benoit wrote:
> 
> >On Thu, 2003-10-16 at 23:03, Hans Reiser wrote:
> >  
> >
> >
> > The thing I find interesting is that Reiser4 doesn't seem to
> >perform near as well with IOZone, as it does with Bonnie++, Infact
> >Reiser3 does slightly better in every IOZone test. 
> >
> >
> >  
> >
> To do well at IOzone, we need to carefully code to optimize for doing 
> small amounts of work with every system call, and make the code very 
> lightweight when doing so.  We would probably need to make the code 
> uglier and less well abstracted to do this. 
> 

Does this have anyhing to do with IOZone's "Recond Length" option by
chance? Though, that wouldn't seem to make sense, as the larger the
reclen I tried, the slower Reiser4 performed in comparison to the other
file systems. 

http://fsbench.netnation.com/new_hardware/2.6.0-test5/iozone.html

Nikita released a new snapshot today, so I'll try running some more
tests with a much wider range of benchmark options in the coming days.


So far though, I think the test5 and test7 kernels are making more of a
difference in the benchmark results then the new Reiser4 snapshot is. 

http://fsbench.netnation.com/new_hardware/combined/bonnie.html

As all the file system results have improved marginally, with the
exception of Reiser3, which seems to have improved almost 10%. Reiser4
in Extents mode is showing some fairly different numbers, mainly in
regard to CPU usage.

> I think I would rather add lots of plugins instead.  We probably will 
> make a modest effort to improve at IOzone, but not a big one.
> 
> If I can produce a filesystem that has more plugins and more features, 
> and wins at performance for disk IO intensive workloads, that is 
> probably more valuable than winning at both disk IO intensive and cache 
> intensive work loads.  At least that is my thinking.  Right now I think 
> that our performance is good enough that we should concentrate mostly on 
> debugging (fixing a few performance anomalies as we find them though), 
> ship what we have so users can use, and then focus on creating the 
> semantics that will make LongHorn's SQL look quite cumbersome in 
> comparison when operating on semi-structured data.
> 
> Performance is not my goal.  Performance is just a way point that I need 
> to reach so that I can resupply with cash for payroll purposes, get 
> enough market share that users will adopt semantic changes I introduce, 
> and then I can go LongHorn hunting.

I'm definitely looking forward to all the different plugins people will
create for Reiser4, it should prove to be quite interesting. 


-- 
Best Regards,
 
Mike Benoit
NetNation Communications Inc.
Systems Engineer
Tel: 604-684-6892 or 888-983-6600
 ---------------------------------------
 
 Disclaimer: Opinions expressed here are my own and not 
 necessarily those of my employer

Re: File system shootout v2.0...

[Hans Reiser]

Mike Benoit wrote:

>
>>    
>>
>
>Does this have anyhing to do with IOZone's "Recond Length" option by
>chance? Though, that wouldn't seem to make sense, as the larger the
>reclen I tried, the slower Reiser4 performed in comparison to the other
>file systems.
>
One the other hand, maybe I just don't know what is going on and need to 
figure out exactly what is being done by IOzone.  Thanks Mike, we'll 
examine it in more detail on Monday.



-- 
Hans