Re: [PATCH v2] Bump core.deltaBaseCacheLimit to 128MiB

[Junio C Hamano <gitster@pobox.com>]

Duy Nguyen <pclouds@gmail.com> writes:

> On Thu, Mar 20, 2014 at 5:11 AM, Junio C Hamano <gitster@pobox.com> wrote:
> ...
>> I know that the 512MiB default for the bitFileThreashold (aka
>> "forget about delta compression") came out of thin air.  It was just
>> "1GB is always too huge for anybody, so let's cut it in half and
>> declare that value the initial version of a sane threashold",
>> nothing more.
>>
>> So it might be that the problem is 512MiB is still too big, relative
>> to the 16MiB of delta base cache, and the former may be what needs
>> to be tweaked.  If a blob close to but below 512MiB is a problem for
>> 16MiB delta base cache, it would still be too big to cause the same
>> problem for 128MiB delta base cache---it would evict all the other
>> objects and then end up not being able to fit in the limit itself,
>> busting the limit immediately, no?
>>
>> I would understand if the change were to update the definition of
>> deltaBaseCacheLimit and link it to the value of bigFileThreashold,
>> for example.  With the presented discussion, I am still not sure if
>> we can say that bumping deltaBaseCacheLimit is the right solution to
>> the "description with the current setting is clearly wrong" (which
>> is a real issue).
>
> I vote make big_file_threshold smaller. 512MB is already unfriendly
> for many smaller machines. I'm thinking somewhere around 32MB-64MB
> (and maybe increase delta cache base limit to match).

These numbers match my gut feeling (e.g. 4k*4k*32-bit uncompressed
would be 64MB); delta cash base that is sized to the same as (or
perhaps twice as big as) that limit may be a good default.

> The only
> downside I see is large blobs will be packed  undeltified, which could
> increase pack size if you have lots of them.

I think that is something that can be tweaked, unless the user tells
us otherwise via command line override, when running the improved
"gc --aggressive" ;-)