Re: LMbench as gcc performance regression test?

Dan Kegel
2003-08-31T22:34:06Z

Larry McVoy wrote:
> Here is some background, pick a benchmark and play with it and see if
> you can convince yourself of anything. The basic idea is to run the
> benchmark TRIES times for $ENOUGH milliseconds. TRIES is set to an odd
> number in bench.h because we sort the results and take the midpoint and
> print that as the result.

It seems lat_pipe never does any median smoothing; it always sets TRIES to 1.
However, at least on the fairly quiet embedded system I'm testing on,
smoothing samples taken within a single run wouldn't make
a huge difference. Any smoothing you get with that would be swamped by
the fact that lat_pipe's result has a bimodal distribution only one of whose
peaks shows up in any one run.
This sure sounds like the kind of thing page coloring is
supposed to solve; has anyone observed page coloring improving
the repeatability of the lat_pipe benchmark?

(There's no median smoothing in lat_pipe.c, I think, because it passes
a value >= 1000000 as the 2nd arg of BENCH:
BENCH(doit(p2[0], p1[1]), SHORT);
BENCH computes the number of samples to take the median of as
__N = (get_enough(1000000) <= 100000) ? TRIES : 1;
get_enough() will always return at least what it is passed,
thus __N will always be 1. It sure was whenever I printed it out, too.
This seems to be the case for the following tests:
bw_pipe bw_tcp bw_unix
lat_fcntl lat_fifo lat_pipe lat_rpc lat_tcp lat_udp lat_unix)
- Dan

--
Dan Kegel