Re: [PATCH] perf: accommodate for MacOSX

[Lars Schneider <larsxschneider@gmail.com>]

> On 21 Jun 2016, at 13:55, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> 
>> ...
>> I think we definitively should take the "perf-lib.sh" part of the patch
>> as this makes the perf test run on OSX and therefore is a strict
>> improvement.
> 
> Yes, it was meant as the starting point to get more things to run on
> MacOSX.
> 
>> If we don't run any perf tests by default on Travis CI then I wouldn't
>> take the ".travis.yml" part of the patch just to keep our Travis CI
>> setup as lean as possible.
> 
> Maybe commented-out, so that people like me have a chance to use Travis
> for MacOSX perf testing?
> 
>> Running perf tests on Travis CI is probably bogus anyways because we
>> never know on what hardware our jobs run and what other jobs run in
>> parallel on that hardware.
> 
> While I agree that the absolute timings cannot be trusted, I have to point
> out that the relative timings on Linux at least are consistent with what I
> could test locally.
> 
> Could you let me know whether a commented-out
> 
> 	# Uncomment this if you want to run perf tests:
> 	# brew install gnu-time
> 
> would be acceptable? I will reroll the patch accordingly.

Commented-out would be fine with me!

Independent of your patch:
Given that the relative timings are consistent for you. Maybe there is
value to run the performance tests (e.g. only on the master branch)
in a separate Travis job. Then we could chart the timings over releases.
I dunno.

- Lars