Re: [PATCH v1 0/4] Teach 'run' perf script to read config files

[Junio C Hamano <gitster@pobox.com>]

Jeff King <peff@peff.net> writes:

> Because perf-lib is built on test-lib, it already reads
> GIT-BUILD-OPTIONS. And the Makefile copies several perf-related values
> into it, including GIT_PERF_MAKE_OPTS and GIT_PERF_REPEAT_COUNT. So you
> can already do:
> ...
> But right now the perf suite is not useful at
> all for comparing two builds of the same tree.
> For that, I think it
> would be more useful if we could define a tuple of parameters for a run.
> One of which could be the tree we're testing. Build opts are another.
> Tested repository is another. And then we'd fill in a table of results
> and let you slice up the table by any column (e.g., compare times for
> runs against a single tree but with differing build options).

Yeah, I think we saw this discussed in not-so-distant past, for
which we want a good solution, and it might be the case that such a
solution can be made easier to use with a separate configuration
file (which this topic may or may not be used as a building block).

> So then I think your config file primarily becomes about defining the
> properties of each run. I'm not sure if it would look like what you're
> starting on here or not.

Yeah, I suspect that the final shape that defines the matrix might
have to become quite a bit different.