Re: [PATCH v2 2/2] convert: Stream from fd to required clean filter instead of mmap

[Jeff King <peff@peff.net>]

On Sat, Aug 16, 2014 at 06:26:08PM +0200, Steffen Prohaska wrote:

> > Is the 15MB limit supposed to be imposed somewhere or is it just a guide
> > of how much memory we expect Git to use in this scenario?
> 
> The test should confirm that the the file that is added is not mmapped
> to memory.  The process size should be relatively small independently
> of the size of the file that is added.  I wanted to keep the file size
> small.  The chosen sizes worked for me on Mac and Linux.

Measuring memory usage seems inherently a bit flaky for the test suite.
It's also a little out of place, as the test suite is generally about
correctness and outcomes, and this is a performance issue.

Would it make more sense to construct a t/perf test that shows off the
change? I suppose the run-time change may not be that impressive, but it
would be cool if t/perf could measure max memory use, too. Then we can
just compare results between versions, which is enough to detect
regressions.

There's some prior art in the jk/pack-bitmap-reuse-deltas series (which
is not merged), where I taught it to measure output sizes of commands.
That should provide the necessary refactoring base, I think.

-Peff