Re: [PATCH] pack-objects: only throw away data during memory pressure

[bdowning@lavos.net (Brian Downing)]

On Tue, Feb 12, 2008 at 09:26:25AM -0500, Nicolas Pitre wrote:
> I think your use case has merits, but the previous behavior had 
> semantics problems.  We always had constant window size with dynamic 
> memory usage, and now we have constant window size with bounded memory 
> usage.
> 
> If what you want is really to have a dynamic window size using a 
> constant memory usage then it needs a different and coherent way to be 
> specified.

Sometimes I want a bounded window size with bounded memory usage; i.e. a
maximum of 50 entries OR 256 megs worth.  That's for everyday repacking
of my troublesome repository; without the window going down to less than
10 or so for the large files, it still takes way too long, but doing the
whole thing at 10 makes for very poor packing.

So that gives four options:

1. No memory limit, constant entry depth.  Original Git behavior.
2. The above with an additional memory-based depth limit.  This is 
   what was added with --memory-limit.
3. Constant entry depth with a memory-usage limit.  This is what the
   proposed patch does.
4. Dynamic entry depth, with a memory-based limit.  This is I believe
   what you are proposing above, and what I emulate by setting
   --window=$bignum --window-memory=x.

I'm willing to try and make all of those work. (Though frankly I don't
care much about #3; setting the window entry size to something "large
enough" seems a simple enough work-around for me, and it prevents what's
probably some truly ridiculous behavior if you have a gigantic number of
tiny, say, tree objects.  Having a cap on window depth stops that case
from taking a truly inordinate amount of time.)

However, I can't figure out what sensible command-line and/or config
parameters would be for the cases above.  Any ideas?

-bcd