Re: Git Scaling: What factors most affect Git performance for a large repo?

[Junio C Hamano <gitster@pobox.com>]

David Turner <dturner@twopensource.com> writes:

> On Fri, 2015-02-20 at 06:38 +0700, Duy Nguyen wrote:
>> >    * 'git push'?
>> 
>> This one is not affected by how deep your repo's history is, or how
>> wide your tree is, so should be quick..
>> 
>> Ah the number of refs may affect both git-push and git-pull. I think
>> Stefan knows better than I in this area.
>
> I can tell you that this is a bit of a problem for us at Twitter.  We
> have over 100k refs, which adds ~20MiB of downstream traffic to every
> push.
>
> I added a hack to improve this locally inside Twitter: The client sends
> a bloom filter of shas that it believes that the server knows about; the
> server sends only the sha of master and any refs that are not in the
> bloom filter.  The client  uses its local version of the servers' refs
> as if they had just been sent....

Interesting.

Care to extend the discussion to improve the protocol exchange,
which starts at $gmane/263932 [*1*], where I list the known issues
around the current protocol (and a possible way to correct them in
footnotes)?


[Footnote]

*1* http://thread.gmane.org/gmane.comp.version-control.git/263898/focus=263932