Re: Warnings in gc.log can prevent gc --auto from running

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Mon, Jul 29 2019, Jeff King wrote:

> On Thu, Jul 25, 2019 at 07:18:57PM -0700, Gregory Szorc wrote:
>
>> I think I've found some undesirable behavior with regards to the
>> behavior of `git gc --auto`. The tl;dr is that a warning message written
>> to gc.log can result in `git gc --auto` effectively disabling itself for
>> gc.logExpiry. The problem is easier to trigger in 2.22 as a result of
>> enabling bitmap indices for bare repositories by default and the
>> behavior can easily result in performance degradation, especially on
>> servers.
>
> Yuck, thanks for reporting this.
>
> As you note, this is a special case of a much larger problem. The other
> common case is the "oops, you still have a lot of loose objects after
> repacking" warning. There's more discussion and some patches here:
>
>   https://public-inbox.org/git/20180716172717.237373-1-jonathantanmy@google.com/
>
> though I don't think any of the work that came out of that fundamentally
> solves the issue.

To add to that Gregory probably finds these two old reports of mine
interesting. The former is pretty much his report (but for a different
root cause, the loose object issue):
https://public-inbox.org/git/87inc89j38.fsf@evledraar.gmail.com/ &
https://public-inbox.org/git/87fu6bmr0j.fsf@evledraar.gmail.com/

>> I don't prescribe to know the best way to solve this problem. I just
>> know it is a footgun sitting in the default Git configuration. And the
>> footgun became a lot easier to fire with the introduction of warning
>> messages related to bitmap indices and again when bitmap indices were
>> enabled by default for bare repositories in Git 2.22.
>
> IMHO one way to mitigate this is to simply warn less. In particular, if
> we are auto-enabling bitmaps, then it doesn't necessarily make sense for
> us to warn about them being disabled.
>
> In the case of .keep files, we've already got 7328482253 (repack:
> disable bitmaps-by-default if .keep files exist, 2019-06-29), which
> should be in the next released version of Git. But I suspect that's
> racy with respect to somebody creating .keep files, and as you note
> there are other config options that might prevent us from generating
> bitmaps.
>
> Instead, it may make sense to turn the --write-bitmap-index option of
> pack-objects into a tri-state: true/false/auto. Then pack-objects would
> know that we are in best-effort mode, and would avoid warning in that
> case. That would also let git-repack express its intentions better to
> git-pack-objects, so we could replace 7328482253, and keep more of the
> logic in pack-objects, which is ultimately what has to make the decision
> about whether it can generate bitmaps.

Sounds like pentastate to me :) (penta = 5, had to look it up). I.e. in
most cases of "auto" we pick a true/false at the outset, whereas this is
true/true-but-dont-care-much/false/false-but-dont-care-much with "auto"
picking the "-but-dont-care-much" versions of a "soft" true/false.

On this general topic a *soft* poke about relying to
https://public-inbox.org/git/8736lnxlig.fsf@evledraar.gmail.com/ if you
have time. I think a "loose pack" might be a way forward for the loose
object proliferation, but maybe I'm wrong.

More generally we're really straining the gc.log pass-along-a-message
facility.