Quick update on latest Linux kernel performance

Quick update on latest Linux kernel performance

[Chen, Kenneth W]

New performance result are posted on http://kernel-perf.sourceforge.net
with latest data collected on kernel 2.6.13-git9.

Kernel-build bench are fairly stable over the last 14 kernel versions
or so.  It was consistently 3-5% better on x86_64 over baseline 2.6.9
kernel.  It showed a lot smaller gain on ia64 though.

Java business benchmark showed very little change in performance on all
kernel versions.

Volanomark took some heavy performance hit during 2.6.12-rc* period, but
come back in 2.6.13 on x86_64 configuration.  Though latest 2.6.13-git9
showed a little bit perf. regression.

Netperf is showing wildly result, especially the 1-byte request/response
component.  Overall, UDP portion Of the netperf are showing nice improvement
over baseline 2.6.9 kernel.

Industry standard transaction processing database workload still suffering
13% performance regression with 2.6.13. (data will be posted in a separate mail)

Take a look at the performance data.  Comments and suggestions are always
welcome and please post them to LKML.

Re: Quick update on latest Linux kernel performance

[Chris White]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]

On Wednesday 14 September 2005 06:32, Chen, Kenneth W wrote:
> New performance result are posted on http://kernel-perf.sourceforge.net
> with latest data collected on kernel 2.6.13-git9.

[snip]

> Take a look at the performance data.  Comments and suggestions are always
> welcome and please post them to LKML.

The benchmarks here have a slight flaw in that the main hardware components 
tested are not given.  About the only thing I can see regarding these tests 
is what processor they run on.  Displaying network performance tests without 
showing the network card or io tests without showing the disk controller 
seems rather odd.  I guess it comes down to requesting a full hardware 
rundown.  If this is displayed someplace on the site or elsewhere please 
provide the link.

Chris White

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Quick update on latest Linux kernel performance

[Bill Davidsen]

Chris White wrote:
> On Wednesday 14 September 2005 06:32, Chen, Kenneth W wrote:
> 
>>New performance result are posted on http://kernel-perf.sourceforge.net
>>with latest data collected on kernel 2.6.13-git9.
> 
> 
> [snip]
> 
> 
>>Take a look at the performance data.  Comments and suggestions are always
>>welcome and please post them to LKML.
> 
> 
> The benchmarks here have a slight flaw in that the main hardware components 
> tested are not given.  About the only thing I can see regarding these tests 
> is what processor they run on.  Displaying network performance tests without 
> showing the network card or io tests without showing the disk controller 
> seems rather odd.  I guess it comes down to requesting a full hardware 
> rundown.  If this is displayed someplace on the site or elsewhere please 
> provide the link.

Unless the hardware was changed, this is not particularly relevant. It's 
good testing to change only one thing, so you know that's what caused 
the change in results.

I think the config was posted at least once, you can probably find it in 
the archives if not on the site.

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: Quick update on latest Linux kernel performance

[Helge Hafting]

Bill Davidsen wrote:

> Chris White wrote:

[...]

>>
>> The benchmarks here have a slight flaw in that the main hardware 
>> components tested are not given.  About the only thing I can see 
>> regarding these tests is what processor they run on.  Displaying 
>> network performance tests without showing the network card or io 
>> tests without showing the disk controller seems rather odd.  I guess 
>> it comes down to requesting a full hardware rundown.  If this is 
>> displayed someplace on the site or elsewhere please provide the link.
>
>
> Unless the hardware was changed, this is not particularly relevant. 
> It's good testing to change only one thing, so you know that's what 
> caused the change in results.

The benchmarks surely says something about the kernel regardless of
wether they specify hardware.  But if you want performance regressions
fixed, then the hardware list is necessary.  It is interesting to know
wether the test machine used SCSI or IDE for IO for example, for those
systems get different patches.  One may regress while another improves.
Similiar for all the different network adapter drivers and so on.

Helge Hafting

Re: Quick update on latest Linux kernel performance

[Danial Thom]

--- "Chen, Kenneth W" <kenneth.w.chen@intel.com>
wrote:

> New performance result are posted on
> http://kernel-perf.sourceforge.net
> with latest data collected on kernel
> 2.6.13-git9.
> 
> Kernel-build bench are fairly stable over the
> last 14 kernel versions
> or so.  It was consistently 3-5% better on
> x86_64 over baseline 2.6.9
> kernel.  It showed a lot smaller gain on ia64
> though.
> 
> Java business benchmark showed very little
> change in performance on all
> kernel versions.
> 
> Volanomark took some heavy performance hit
> during 2.6.12-rc* period, but
> come back in 2.6.13 on x86_64 configuration. 
> Though latest 2.6.13-git9
> showed a little bit perf. regression.
> 
> Netperf is showing wildly result, especially
> the 1-byte request/response
> component.  Overall, UDP portion Of the netperf
> are showing nice improvement
> over baseline 2.6.9 kernel.
> 
> Industry standard transaction processing
> database workload still suffering
> 13% performance regression with 2.6.13. (data
> will be posted in a separate mail)
> 
> Take a look at the performance data.  Comments
> and suggestions are always
> welcome and please post them to LKML.

Does it still drop packets when only running at
15% utilization?

Did you run data streams through the box while
doing these tests? There's a difference between
gaining performance in a benchmark and just
shifting performance from one activity to
another. If the latter is the case, then there's
been no progress at all. I keep seeing all these
great benchmark results, and linux keeps dropping
more and more packets as the versions increase.
Its become practically unusable as a specialized
networking appliance.

Danial




		
__________________________________ 
Yahoo! Mail - PC Magazine Editors' Choice 2005 
http://mail.yahoo.com

Re: Quick update on latest Linux kernel performance

[Felix Oxley]

I think this is a fantastic and useful project.
However it is unfortunate that currently results are only available  
for >=2.6.9.
(If only somebody had been brilliant enough to start this in 2000!!!)

How long does the test suite take to run?
Are there plans to test previous versions?

I suppose that a baseline of 2.4.xx would be interesting, and also  
all releases back to 2.6.0

Also, is there any benefit in expanding the range of systems?
ie low end kit, different processor architectures etc.

Or is the primary purpose served well by using a small, modern group  
of test machines?