Re: [Internet]Re: reachability-bitmap makes push performance worse ?

[Derrick Stolee <derrickstolee@github.com>]

On 6/14/2022 7:00 AM, kylezhao(赵柯宇) wrote:
>> This is a known issue, I think you've found the same problem discussed in these past threads:
>>
>> https://lore.kernel.org/git/38b99459158a45b1bea09037f3dd092d@exmbdft7.ad.twosigma.com/
>> https://lore.kernel.org/git/87zhoz8b9o.fsf@evledraar.gmail.com/
> 
> Thanks.
> 
>> The latter one in particular has a lot of extra details. The former also
>> has the suggestion of a per-push bitmap configuration as a workaround.
>>
>> As your numbers show it's still an issue today, but those threads should
>> help you if you're looking to dig further into the root cause.
>>
>> Aside from the underlying root causes it would be very nice to fix the
>> progress code in that area, i.e. we "stall" on "Enumerating objects",
>> which is just a matter of us not having a separate progress bar for the
>> very expensive bitmap work we're doing.
> 
> It looks like optimizing the bitmap to solve the problem will be a long
> process. This requires developers to have a deep understanding of the
> algorithm.
> 
> A per-push bitmap configuration as a workaround can't completely solve the
> problem, but it works for me. After all, bitmap was not designed to optimize
> git push. Most of time, git push is not been called as frequently as git fetch.

I think the issue is that bitmaps are designed to support servers, which don't
exactly use "git push" but instead use "git upload-pack" with a very different
type of data (a lot of branches simultaneously from a large variety of bases).

In general, clients that use "git push" don't generally have bitmaps, so this
has not been a priority. For clients, it is faster to do a more focused object
walk. See these commits and blog post for more details:

* d5d2e93577 (revision: implement sparse algorithm, 2019-01-16)
* 3d036eb0d2 (pack-objects: create pack.useSparse setting, 2019-01-16)
* https://devblogs.microsoft.com/devops/exploring-new-frontiers-for-git-push-performance/

Hopefully that gives enough context as to why one would want to disable bitmaps
for most "git push" operations.

> The problem has been around for 3 years, has the community considered providing
> a config like "push.useBitmap" to prevent git push using bitmap? It would be
> appreciated if there is such a config, which can quickly solve my problem and
> doesn't seem like a lot of work.

I think this config would be a good idea, and I would even argue that we might
want to set it to "false" by default.

> If no one is interested in it, I can also try to submit a patch (although it
> may be a bit slow since all I am new to the git community).

We would welcome the contribution! I look forward to seeing it when you're
ready.

Thanks,
-Stolee