[PATCH v1 0/2] t/perf: bisect performance regressions

[Christian Couder <christian.couder@gmail.com>]

This is a small patch series on top of cc/perf-aggregate-sort, which
is next, to add scripts that make it much easier to bisect performance
regressions.

For example if you ran perf test using the perf.conf config file that
contains a "with libpcre", then using:

$ ./aggregate.perl --subsection "with libpcre" --sort-by=regression v2.15.1 v2.16.2 v2.17.0 

you can get a line in the output like:

+8.2% p5302-pack-index.6 14.33(44.50+0.77) 15.50(43.83+0.94) v2.16.2 v2.17.0

and you can now use this output line to bisect where the regression
comes from using:

echo "+8.2% p5302-pack-index.6 14.33(44.50+0.77) 15.50(43.83+0.94) v2.16.2 v2.17.0" | ./bisect_regression --config perf.conf --subsection "with libpcre"

Caveats:

You must make sure that the times are consistent between runs and that
there is a significant time difference between the "good" runs and the
"bad" runs.

For example the following is not easily bisectable:

+100.0% p0000-perf-lib-sanity.2 0.01(0.01+0.01) 0.02(0.02+0.00) v2.15.1 v2.16.2

as the rtime went from 0.01 s to 0.02 s and we only have 0.01 s precision.

Christian Couder (2):
  perf/run: add --subsection option
  t/perf: add scripts to bisect performance regressions

 t/perf/bisect_regression | 73 ++++++++++++++++++++++++++++++++++++++++
 t/perf/bisect_run_script | 47 ++++++++++++++++++++++++++
 t/perf/run               | 56 ++++++++++++++++++++++++------
 3 files changed, 166 insertions(+), 10 deletions(-)
 create mode 100755 t/perf/bisect_regression
 create mode 100755 t/perf/bisect_run_script

-- 
2.17.0.rc1.36.g098d832c9.dirty