collecting simple benchmark scripts?

collecting simple benchmark scripts?

[Kees Cook]

Hi,

In a separate thread, some folks were looking for some simple
benchmarks for evaluating various changes to kernel internals (as
opposed to the much more focused things like xfstests). For me, this
has been an area of lore and passed-around scripts, and it seems like
maybe we should have a subdirectory of tools/testing/benchmarks/ or
something to collect these?

(Or maybe this already exists and I've totally missed it?)

I've got at least one micro-benchmark in
tools/testing/selftests/seccomp/seccomp_benchmark.c, and searches show
tools/testing/selftests/vm/gup_benchmark.c too, but I was thinking of
either more generali things more like the famous "kernel build
benchmark" or a wrapper for running hackbench to get some statistics
out of it, etc.

Or, I guess, at least collecting all the micro-benchmarks in some
single place, as they're a bit scattered.

I'm sure I'm not remotely the first person to bring this up, but my
attempts at searches for this have failed.

Thoughts?

-Kees

-- 
Kees Cook
Pixel Security

Re: collecting simple benchmark scripts?

[Tycho Andersen]

On Wed, Feb 14, 2018 at 02:50:37PM -0800, Kees Cook wrote:
> I'm sure I'm not remotely the first person to bring this up, but my
> attempts at searches for this have failed.
> 
> Thoughts?

There is mmtests: https://github.com/gormanm/mmtests which looks like
it has kernel build and a bunch of other stuff. Are you thinking of
bringing stuff like that in tree?

Cheers,

Tycho

Re: collecting simple benchmark scripts?

[Laura Abbott]

On 02/14/2018 02:50 PM, Kees Cook wrote:
> Hi,
> 
> In a separate thread, some folks were looking for some simple
> benchmarks for evaluating various changes to kernel internals (as
> opposed to the much more focused things like xfstests). For me, this
> has been an area of lore and passed-around scripts, and it seems like
> maybe we should have a subdirectory of tools/testing/benchmarks/ or
> something to collect these?
> 
> (Or maybe this already exists and I've totally missed it?)
> 
> I've got at least one micro-benchmark in
> tools/testing/selftests/seccomp/seccomp_benchmark.c, and searches show
> tools/testing/selftests/vm/gup_benchmark.c too, but I was thinking of
> either more generali things more like the famous "kernel build
> benchmark" or a wrapper for running hackbench to get some statistics
> out of it, etc.
> 
> Or, I guess, at least collecting all the micro-benchmarks in some
> single place, as they're a bit scattered.
> 
> I'm sure I'm not remotely the first person to bring this up, but my
> attempts at searches for this have failed.
> 
> Thoughts?
> 
> -Kees
> 

This is the script I've been using for hackbench since it can
be noisy. I expect someone to tell me the math is wrong but
I'd be happy to throw this in a repo if others are interested

#!/bin/sh

CNT=100
mean=0.0
M2=0.0
for i in $(seq 1 $CNT); do
	echo $i
	r=`hackbench -g 20 -l 1000 | grep Time | cut -d ' ' -f 2`
	d_calc="$r-$mean"
         d=`echo $d_calc | bc -l`
	mean_calc="$mean+($d/$i)"
	mean=`echo $mean_calc | bc -l`
	M2_calc="$M2+($d*($r-$mean))"
	M2=`echo  $M2_calc| bc -l`
done

echo "mean $mean"
V_calc="$M2/$(($CNT-1))"
V=`echo $V_calc | bc -l`
DEV_calc="sqrt($V)"
DEV=`echo $DEV_calc | bc -l`
echo "variance $V"
echo "stdev $DEV"

Re: collecting simple benchmark scripts?

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]

Hi!

> In a separate thread, some folks were looking for some simple
> benchmarks for evaluating various changes to kernel internals (as
> opposed to the much more focused things like xfstests). For me, this
> has been an area of lore and passed-around scripts, and it seems like
> maybe we should have a subdirectory of tools/testing/benchmarks/ or
> something to collect these?
> 
> (Or maybe this already exists and I've totally missed it?)
> 
> I've got at least one micro-benchmark in
> tools/testing/selftests/seccomp/seccomp_benchmark.c, and searches show
> tools/testing/selftests/vm/gup_benchmark.c too, but I was thinking of
> either more generali things more like the famous "kernel build
> benchmark" or a wrapper for running hackbench to get some statistics
> out of it, etc.

Ok, I guess my micro-benchmark is too micro for you, but I like to use

"time cat /dev/urandom | head -c 10000000 |  bzip2 -9 - | wc -c"

I have a page with various results online somewhere, probably at

http://atrey.karlin.mff.cuni.cz/~pavel/quickbench.html


									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]