Re: LMbench as gcc performance regression test?

[Dan Kegel <dank@kegel.com>]

Larry McVoy wrote:
> Here is some background, pick a benchmark and play with it and see if
> you can convince yourself of anything.  The basic idea is to run the
> benchmark TRIES times for $ENOUGH milliseconds.  TRIES is set to an odd
> number in bench.h because we sort the results and take the midpoint and
> print that as the result. 

It seems lat_pipe never does any median smoothing; it always sets TRIES to 1.
However, at least on the fairly quiet embedded system I'm testing on,
smoothing samples taken within a single run wouldn't make
a huge difference.  Any smoothing you get with that would be swamped by
the fact that lat_pipe's result has a bimodal distribution only one of whose
peaks shows up in any one run.
This sure sounds like the kind of thing page coloring is
supposed to solve; has anyone observed page coloring improving
the repeatability of the lat_pipe benchmark?

(There's no median smoothing in lat_pipe.c, I think, because it passes
a value >= 1000000 as the 2nd arg of BENCH:
                 BENCH(doit(p2[0], p1[1]), SHORT);
BENCH computes the number of samples to take the median of as
         __N = (get_enough(1000000) <= 100000) ? TRIES : 1;
get_enough() will always return at least what it is passed,
thus __N will always be 1.  It sure was whenever I printed it out, too.
This seems to be the case for the following tests:
bw_pipe  bw_tcp  bw_unix
lat_fcntl lat_fifo lat_pipe lat_rpc lat_tcp lat_udp lat_unix)
- Dan

-- 
Dan Kegel
http://www.kegel.com
http://counter.li.org/cgi-bin/runscript/display-person.cgi?user=78045