Re: git-svn with big subversion repository

[Jason Miller <jason@milr.com>]

On Wed, 2 Mar 2011 02:43:23 +0000
John Kristian <jkristian@linkedin.com> wrote:

> How do you recommend using git to work with branches of a large, busy
> subversion repository? In general, how can small teams use git for
> their tasks, and use subversion to coordinate with a larger
> organization?

I don't know if this is the same problem that you are having, but...

I had to clone a repository with 200k revisions and 12000
branches+tags.  It was going to take weeks and weeks with a local
svnsync mirror on a high-end workstation with fast disks.

I've never touched perl-code previous to this, but a friend pointed me
at a good perl profiler, and I found pretty quickly the offending line
of code in git-svn:

3515         return unless ::verify_ref($self->refname.'^0');

This was basically doing a
  system("git rev-parse --verify some-reference^0")

several times per revision fetched per branch.  When you have 12000
branches, that really, really adds up.  I made a change that seems to
speed it up by a factor of about 10-20x on my repository, but I'm still
digging around in git to see if I'm doing it correctly.

My basic logic is that if the above one-liner returns true then either
one of the following files will exist:
    $ENV{GIT_DIR}/refs/remotes/$refname
    $ENV{GIT_DIR}/refs/$refname
    $ENV{GIT_DIR}/refs/heads/$refname
or there will be an entry for the reference in
    $ENV{GIT_DIR}/packed-refs

Furthermore since packed-refs changes infrequently, you can cache its
contents.

I'm still digging around in the plumbing to see if this is
assumption is true or not.  If I find it is true, I'll likely submit a
patch.  Now that it "works on my machine" I've backburnered it a bit
since git is more a tool I use than a project I hack on.

-Jason