git-svn taking a long time

git-svn taking a long time

[David Kågedal]

I compiled the perl bindings from subversion 1.6.x, and git git version
1.7.0.rc0.52.g64ba.

I have imported a largish svn repo using "git svn fetch", which takes
quite a long time (at least a day).

But my problems is that if I rerun "git svn fetch" again right after
fetching, it will chew for hours before figuring out there are just a
few new revisions to import. And the same if I try again.

I have another import from the same repo (on another machine, using
slightly different versions of stuff) that doesn't have this problem.

So, does anyone have any suggestions for what could be done about this?

This is what the entry i .git/config looks like (url edited to protect
the innocent)

[svn-remote "svn"]
        url = svn://svn.server
        branches = foo/branches/*:refs/remotes/svn/*
        tags = foo/tags/*:refs/remotes/svn/tags/*

-- 
David Kågedal

Re: git-svn taking a long time

[Andrew Myrick]

Give 1.7.0-rc2 a try.  It includes commit 8bff7c5383ed833bd1df9c8d85c00a27af3e5b02, which attempts to persistently cache a lot of the processing that git-svn has to do on subversion's merge tickets, which has improved my fetch times significantly.  

Note 1: git did not support subversion merge tickets before git-1.6.6, so there would be no slowdown if you use prior versions.

Note 2: you shouldn't need to clone your subversion repository from scratch.  git will create the caches the first time you fetch after upgrading (so the first fetch will be slow), and it will continue to use and update the caches thereafter.

-Andrew

On Feb 10, 2010, at 4:41 AM, David Kågedal wrote:

> I compiled the perl bindings from subversion 1.6.x, and git git version
> 1.7.0.rc0.52.g64ba.
> 
> I have imported a largish svn repo using "git svn fetch", which takes
> quite a long time (at least a day).
> 
> But my problems is that if I rerun "git svn fetch" again right after
> fetching, it will chew for hours before figuring out there are just a
> few new revisions to import. And the same if I try again.
> 
> I have another import from the same repo (on another machine, using
> slightly different versions of stuff) that doesn't have this problem.
> 
> So, does anyone have any suggestions for what could be done about this?
> 
> This is what the entry i .git/config looks like (url edited to protect
> the innocent)
> 
> [svn-remote "svn"]
>        url = svn://svn.server
>        branches = foo/branches/*:refs/remotes/svn/*
>        tags = foo/tags/*:refs/remotes/svn/tags/*
> 
> -- 
> David Kågedal
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git-svn taking a long time

[David Kågedal]

Andrew Myrick <amyrick@apple.com> writes:

> Give 1.7.0-rc2 a try.  It includes commit 8bff7c5383ed833bd1df9c8d85c00a27af3e5b02, which attempts to persistently cache a lot of the processing that git-svn has to do on subversion's merge tickets, which has improved my fetch times significantly.  

By "merge tickets", are you talking about the merge functionality that
appeared in subversion 1.5? We don't use that.

But I had another idea. I pecularity of our subversion repo is that we
no longer use the foo/trunk branch, but only foo/branches/*. But we did
once upon a time have a foo/trunk. And since I didn't include a "fetch =
foo/trunk:refs/remotes/svn/trunk" in my config, it might need to refetch
that information every time. For instance, the first revision is on
trunk.

I'm rerunning the fetch now with the trunk added, so see if it helps.

And another note is that "git svn fetch --parent" was always quick.

-- 
David Kågedal

Re: git-svn taking a long time

[Andrew Myrick]

On Feb 10, 2010, at 9:29 AM, David Kågedal wrote:

> Andrew Myrick <amyrick@apple.com> writes:
> 
>> Give 1.7.0-rc2 a try.  It includes commit 8bff7c5383ed833bd1df9c8d85c00a27af3e5b02, which attempts to persistently cache a lot of the processing that git-svn has to do on subversion's merge tickets, which has improved my fetch times significantly.  
> 
> By "merge tickets", are you talking about the merge functionality that
> appeared in subversion 1.5? We don't use that.

I do mean that.  If you don't use them, then I'm stumped.

> But I had another idea. I pecularity of our subversion repo is that we
> no longer use the foo/trunk branch, but only foo/branches/*. But we did
> once upon a time have a foo/trunk. And since I didn't include a "fetch =
> foo/trunk:refs/remotes/svn/trunk" in my config, it might need to refetch
> that information every time. For instance, the first revision is on
> trunk.
> 
> I'm rerunning the fetch now with the trunk added, so see if it helps.
> 
> And another note is that "git svn fetch --parent" was always quick.

Sounds reasonable.  Good luck!

-Andrew

Re: git-svn taking a long time

[David Kågedal]

David Kågedal <davidk@lysator.liu.se> writes:

> Andrew Myrick <amyrick@apple.com> writes:
>
>> Give 1.7.0-rc2 a try.  It includes commit 8bff7c5383ed833bd1df9c8d85c00a27af3e5b02, which attempts to persistently cache a lot of the processing that git-svn has to do on subversion's merge tickets, which has improved my fetch times significantly.  
>
> By "merge tickets", are you talking about the merge functionality that
> appeared in subversion 1.5? We don't use that.
>
> But I had another idea. I pecularity of our subversion repo is that we
> no longer use the foo/trunk branch, but only foo/branches/*. But we did
> once upon a time have a foo/trunk. And since I didn't include a "fetch =
> foo/trunk:refs/remotes/svn/trunk" in my config, it might need to refetch
> that information every time. For instance, the first revision is on
> trunk.
>
> I'm rerunning the fetch now with the trunk added, so see if it helps.

That seems to have done the trick. Now it's quick when there is nothing
to do.

Perhaps there is something in the git-svn caching that could still be
improved?

-- 
David Kågedal