YABM (Yet another benchmark)

YABM (Yet another benchmark)

[Jeroen Vreeken]

[-- Attachment #1: Type: text/plain, Size: 1901 bytes --]

Hi,

This benchmark was made in response to a recent post here on lkm were 
Linus indicated he would welcom pretty much any benchmark.
Since there are already several database benchmarks, 3d benchmarks I 
opted for a more down to earth approach.
As such I am pleased to announce the 'linux kernel hacker benchmark', a 
benchmark designed to simulate the activities of the average linux 
kernel hacker.
With this benchmark it should be possible to measure the performance off 
the kernel for its most important user group, the kernel hacker.

This workload turns out to be relativly simple to simulate as can be 
seen in the attached benchmark program 'lkh-bm.c'.
It is compiled with 'gcc -Wall lkh-bm.c -o lkh-bm'.

This test has been run on all 2.6 releases and several older kernels 
dating some years back. Unfortunatly 1.1 and lower kernels aren't able 
to complete the test.
The compiler used was gcc 3.2.3, the cpu a Celeron @ 2.4GHz.
I plan to run this test daily on all releases, bk, mm and ac snapshots 
and maybe more trees on kernel.org asuming nobody objects to me doing a 
recursive web-suck with wget.

At the end of this post you will find the already done benchmarks.
As Linus seems to dig pretty pictures a graph has been attached 
(lkh-bm.gif) with the same results.
Surprisingly the number seems to be constant during the last years. This 
could either indicate that the kernel hasn't regressed for years in this 
respect (which would mean somebody is doing a fine job indeed) or it 
could mean that the average kernel hacker simply doesn't do much usufull 
anyway....

Regards,
Jeroen


Benchmark results:
1.1.0 0
1.1.20 0
1.1.40 0
1.1.60 0
1.2.0 603
2.0.0 604
2.0.10 605
2.0.20 600
2.0.30 601
2.2.0 602
2.2.10 603
2.2.20 604
2.4.0 605
2.4.10 600
2.4.20 601
2.6.0 602
2.6.1 603
2.6.2 604
2.6.3 605
2.6.4 600
2.6.5 601
2.6.6 602
2.6.7 603
2.6.8 604
2.6.9 605
2.6.10 600
2.6.11 601


[-- Attachment #2: lkh-bm.gif --]
[-- Type: image/gif, Size: 18561 bytes --]

[-- Attachment #3: lkh-bm.c --]
[-- Type: text/x-c, Size: 894 bytes --]

#include <stdio.h>
#include <time.h>

#define MEASUREMENT_TIME	60
#define LINUS_CONSTANT		6

/*
 * my_integer_pi()
 *
 * This function calculates the value of PI, and returns
 * 3. It does so by adding "1" in a loop three times.
 */
int my_integer_pi(void)
{
	int i, pi;
	
	/*
	 * This is the main loop.
	 */
	pi = 0;
	for (i = 0; i < 3; i++)
		pi++;
	
	/* Ok, return it */
	return pi;
}

int main(int argc, char **argv)
{
	time_t timer, start, prev;
	int completed = 0;
	int calc;

	start = time(NULL);
	timer = start;
	prev = start;
	while ( timer - start <= MEASUREMENT_TIME ) {
		/* do some typical kernel hacker stuff... */
		calc = my_integer_pi();
		timer = time(NULL);
		if ((timer - prev) == LINUS_CONSTANT ) {
			completed++;
			prev = timer;
		}
	}
	printf("endless LKH loops per hour: %ld\n",
	    completed * 3600 / MEASUREMENT_TIME + (time(NULL) % LINUS_CONSTANT));

	return 0;
}

Re: YABM (Yet another benchmark)

[Matthias-Christian Ott]

Jeroen Vreeken schrieb:

> Hi,
>
> This benchmark was made in response to a recent post here on lkm were 
> Linus indicated he would welcom pretty much any benchmark.
> Since there are already several database benchmarks, 3d benchmarks I 
> opted for a more down to earth approach.
> As such I am pleased to announce the 'linux kernel hacker benchmark', 
> a benchmark designed to simulate the activities of the average linux 
> kernel hacker.
> With this benchmark it should be possible to measure the performance 
> off the kernel for its most important user group, the kernel hacker.
>
> This workload turns out to be relativly simple to simulate as can be 
> seen in the attached benchmark program 'lkh-bm.c'.
> It is compiled with 'gcc -Wall lkh-bm.c -o lkh-bm'.
>
> This test has been run on all 2.6 releases and several older kernels 
> dating some years back. Unfortunatly 1.1 and lower kernels aren't able 
> to complete the test.
> The compiler used was gcc 3.2.3, the cpu a Celeron @ 2.4GHz.
> I plan to run this test daily on all releases, bk, mm and ac snapshots 
> and maybe more trees on kernel.org asuming nobody objects to me doing 
> a recursive web-suck with wget.
>
> At the end of this post you will find the already done benchmarks.
> As Linus seems to dig pretty pictures a graph has been attached 
> (lkh-bm.gif) with the same results.
> Surprisingly the number seems to be constant during the last years. 
> This could either indicate that the kernel hasn't regressed for years 
> in this respect (which would mean somebody is doing a fine job indeed) 
> or it could mean that the average kernel hacker simply doesn't do much 
> usufull anyway....
>
> Regards,
> Jeroen
>
>
> Benchmark results:
> 1.1.0 0
> 1.1.20 0
> 1.1.40 0
> 1.1.60 0
> 1.2.0 603
> 2.0.0 604
> 2.0.10 605
> 2.0.20 600
> 2.0.30 601
> 2.2.0 602
> 2.2.10 603
> 2.2.20 604
> 2.4.0 605
> 2.4.10 600
> 2.4.20 601
> 2.6.0 602
> 2.6.1 603
> 2.6.2 604
> 2.6.3 605
> 2.6.4 600
> 2.6.5 601
> 2.6.6 602
> 2.6.7 603
> 2.6.8 604
> 2.6.9 605
> 2.6.10 600
> 2.6.11 601
>
>
> ------------------------------------------------------------------------
>
>------------------------------------------------------------------------
>
>#include <stdio.h>
>#include <time.h>
>
>#define MEASUREMENT_TIME	60
>#define LINUS_CONSTANT		6
>
>/*
> * my_integer_pi()
> *
> * This function calculates the value of PI, and returns
> * 3. It does so by adding "1" in a loop three times.
> */
>int my_integer_pi(void)
>{
>	int i, pi;
>	
>	/*
>	 * This is the main loop.
>	 */
>	pi = 0;
>	for (i = 0; i < 3; i++)
>		pi++;
>	
>	/* Ok, return it */
>	return pi;
>}
>
>int main(int argc, char **argv)
>{
>	time_t timer, start, prev;
>	int completed = 0;
>	int calc;
>
>	start = time(NULL);
>	timer = start;
>	prev = start;
>	while ( timer - start <= MEASUREMENT_TIME ) {
>		/* do some typical kernel hacker stuff... */
>		calc = my_integer_pi();
>		timer = time(NULL);
>		if ((timer - prev) == LINUS_CONSTANT ) {
>			completed++;
>			prev = timer;
>		}
>	}
>	printf("endless LKH loops per hour: %ld\n",
>	    completed * 3600 / MEASUREMENT_TIME + (time(NULL) % LINUS_CONSTANT));
>
>	return 0;
>}
>  
>
This is _not_ serious a benchmark! It's just a counter! A _real_ 
benchmark would test threads, memory management, the schedule, . . . I 
guess on a NetBSD or Windows machine with the same Hardware you would 
get the same result.

Matthias-Christian Ott

Re: YABM (Yet another benchmark)

[Paul Rolland]

> This is _not_ serious a benchmark! It's just a counter! A _real_ 
> benchmark would test threads, memory management, the 
> schedule, . . . I 
> guess on a NetBSD or Windows machine with the same Hardware you would 
> get the same result.


We are April, 1st ;-)

Paul