Re: Git Scaling: What factors most affect Git performance for a large repo?

[David Turner <dturner@twopensource.com>]

On Fri, 2015-02-20 at 13:37 -0700, Martin Fick wrote:
> On Friday, February 20, 2015 01:29:12 PM David Turner wrote:
> >...
> > For a more general solution, perhaps a log of ref updates
> > could be used. Every time a ref is updated on the server,
> > that ref would be written into an append-only log.  Every
> > time a client pulls, their pull data includes an index
> > into that log.  Then on push, the client could say, "I
> > have refs as-of $index", and the server could read the
> > log (or do something more-optimized) and send only refs
> > updated since that index.
> 
> Interesting idea, I like it.
> 
> How would you make this reliable?  It relies on updates 
> being reliably recorded which would mean that you would have 
> to ensure that any tool which touches the repo follows this 
> convention.  That is unfortunately a tough thing to enforce 
> for most people.

I think it only truly relies on the server reliably updating its state
on ref updates. Which of course the server will do because why would you
let arbitrary processes write to your central git repo?  (That is, most
people use git in a roughly-centralized way, and if you turn on this
config option, you promise to only do ref updates that write to the
log).

If the client fails to update its state (on a fetch), it will send
larger-than-necessary packs but not otherwise fail.  And this situation
is sometimes be detectable on the client side -- if
mtime(.git/refs/remotes/$remote ) > mtime
(.git/server-ref-log-index/$remote), then we know our
server-ref-log-index is out-of-date.