Re: [PATCH] t1050-large: replace dd by test-genrandom

[Jeff King <peff@peff.net>]

On Tue, Jan 13, 2015 at 06:36:27PM +0100, Johannes Sixt wrote:

> For some unknown reason, the dd on my Windows box segfaults every now
> and than, but since recently, it does so much more often than it used
> to, which makes running the test suite burdensome.
> 
> Get rid of four invocations of dd and use test-genrandom instead.

There are a dozen other uses of dd in the test suite. Do they all need
to go?

> The new code does change some properties of the generated files:
> 
>  - They are a bit smaller.
>  - They are not sparse anymore.
>  - They do not compress well anymore.

This is unfortunate, as it means other platforms will be slower. I
measured a best-of-five on running t1050 going from 0.780s to 1.750s.
That's on an SSD. Doing it on a RAM disk the numbers are 0.600s and
1.394s. Better, but not great.

One second on the test suite probably isn't breaking the bank, but these
sorts of things do add up. I wonder if we can shrink the test size. We
use 2000k files with a 200k core.bigfilethreshold, and a 1500k
GIT_ALLOC_LIMIT.  Skimming through the history, the sizes seem fairly
arbitrary. We can't go _too_ low, or GIT_ALLOC_LIMIT will prevent us
from even allocating heap memory for non-objects.

I tried dropping it by a factor of 10, but sadly that hits several
cases. The commit-slab code wants 512k chunks (which seems like rather a
lot to me), and pack-objects starts at just over 150k for the set of
objects. It would be nice to have a finer-grained tool than
GIT_ALLOC_LIMIT that applied only to objects, but I guess then we would
not be as sure of catching stray code paths (each caller would have to
annotate "this is for an object").

-Peff