Git push performance problems with ~100K refs

[Martin Fick <mfick@codeaurora.org>]

Hello,

I am back to talk about git performance with lots of refs: 
~100K.  This time I am investigating pushes to a repo with 
about ~100K ref, a gerrit mirror which includes the gerrit 
changes.  When I push just a simple one line change to a 
file, the push takes about ~43s.  If I delete the changes
from the destination repo, this push takes about 6s.  This 
seems rather excessive to me, in fact given that the repo 
with 100K refs has more data in it and is more likely to 
have the objects I am pushing in it, if things are done 
right, it should be a faster push (in theory).

So, upon early investigation, I noticed that the time to 
push seems mostly determined by the receiving end which is 
processing all out for 100% on a CPU.  During this time 
period, the receiving end git commands look like this:

  git-receive-pack path/to/repo.git

and:

  git rev-list --objects --stdin --not --all

The latter of these two commands is the one burning CPU.

Does anyone have any hints as to what might be wrong with 
the receiving end algorithm that would cause a small change 
to use so much CPU?  Is there anything that can be done 
about it?  I noticed that the --all option will effectively 
feed  all the 100K refs to rev-list, is this really 
necessary?  Are there any tests that I can perform to help 
debug this?

I am using git 1.7.8.3 and I also tried 1.7.10.rc3, same 
results.

Thanks,

-Martin


-- 
Employee of Qualcomm Innovation Center, Inc. which is a 
member of Code Aurora Forum