Linux Kernel Performance Testing

Linux Kernel Performance Testing

[Grecko OSCP]

Hey All,

I noticed yesterday a news article on Linux.org about more kernel
performance testing being called for, and I decided it would be a nice
project to try. I have 10 completely identical systems that can be
used for this, and would like to get started while I know I have them
for a while.

However, I wanted to make sure I didn't waste time. My plan was to do
all testing, prerelease, and release kernels from the 2.4, 2.5, and
2.6 trees, with both lmbench and the Linux Testing Project (LTP)
benchmark suite. Will this help out? Is there anything else a person
should do? With those two benchmarks, and all the kernels I mentioned,
I could be done in about 25 days, at one kernel a machine a day. I
assume it wouldn't matter what distribution was used, so long as its
the same for all tests?

Thanks!
Grecko

RE: Linux Kernel Performance Testing

[Chen, Kenneth W]

Grecko OSCP wrote on Friday, April 01, 2005 10:22 AM
> I noticed yesterday a news article on Linux.org about more kernel
> performance testing being called for, and I decided it would be a nice
> project to try. I have 10 completely identical systems that can be
> used for this, and would like to get started while I know I have them
> for a while.
>
> However, I wanted to make sure I didn't waste time. My plan was to do
> all testing, prerelease, and release kernels from the 2.4, 2.5, and
> 2.6 trees, with both lmbench and the Linux Testing Project (LTP)
> benchmark suite. Will this help out? Is there anything else a person
> should do? With those two benchmarks, and all the kernels I mentioned,
> I could be done in about 25 days, at one kernel a machine a day. I
> assume it wouldn't matter what distribution was used, so long as its
> the same for all tests?

The 10 machines for running benchmarks is not a bad infrastructure to
start with.  However, it may not be sufficient to identify performance
regression.  The benchmarks that show regression in Linux kernel requires
huge infrastructure - lots of memory, disks, network and clients.  The
simple benchmarks sometime do not show regression and are usually well
covered by the community and OSDL.

As mentioned in another thread, we (as Intel) will take on the challenge
to do performance testing on a regular basis.  We have fairly extensive
hardware mix and infrastructure (large smp/numa box with lots of memory,
disk farm, network etc) to really stress the kernel, performance wise.