speeding up git-svn when directories are copied?

speeding up git-svn when directories are copied?

[Karl Hasselström]

When importing a whole repository, git-svn currently takes a _lot_ of
time. Almost all of it seems to be spent getting the full text of each
and every file when a tag or new branch is created.

If I remember correctly (but it was quite a while since I looked at
it), git-svnimport didn't ask the Subversion server for all the files;
it just noticed when the server said that a new directory was a copy
of an old directory, which makes them just as fast as any other commit
that doesn't change a gazillion files. (I've had an import running for
a few hours now, and have checked on it about a dozen times; every
single time, it was importing a new tag or branch. This suggests that
making these go as fast as regular commits would speed up the import
by more than a factor 10.)

Have you looked at doing this with git-svn? If it's not on your to-do
list, I might take a stab at it at some point. But I only speak the
copy-n-paste dialect of perl, so I'll need to find a largish block of
uninterrupted time first. :-)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: speeding up git-svn when directories are copied?

[Eric Wong]

Karl Hasselström <kha@treskal.com> wrote:
> When importing a whole repository, git-svn currently takes a _lot_ of
> time. Almost all of it seems to be spent getting the full text of each
> and every file when a tag or new branch is created.

Try SVN 1.4.3 with my patch[1] to get do_switch() working.
trunk should work without the patch.

[1] - http://svn.haxx.se/dev/archive-2007-01/0936.shtml

do_switch() is broken otherwise, and do_update() is extremely
inefficient.

-- 
Eric Wong

Re: speeding up git-svn when directories are copied?

[Karl Hasselström]

On 2007-04-23 18:54:05 -0700, Eric Wong wrote:

> Karl Hasselström <kha@treskal.com> wrote:
>
> > When importing a whole repository, git-svn currently takes a _lot_
> > of time. Almost all of it seems to be spent getting the full text
> > of each and every file when a tag or new branch is created.
>
> Try SVN 1.4.3 with my patch[1] to get do_switch() working. trunk
> should work without the patch.
>
> [1] - http://svn.haxx.se/dev/archive-2007-01/0936.shtml
>
> do_switch() is broken otherwise, and do_update() is extremely
> inefficient.

Thanks for the help. I applied your patch to a clean 1.4.3 tree, and
built subversion itself plus the perl bindings. But git-svn is still
slow, and the testcase you gave in that thread still fails.

(I'm pretty sure that I'm actually using the bindings I've built; I
install them to a nonstandard location, so if I don't set PERL5LIB
they aren't found at all, and perl complains. When I do set it to my
patched bindings, perl accepts it but your testcase fails.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: speeding up git-svn when directories are copied?

[Eric Wong]

Karl Hasselström <kha@treskal.com> wrote:
> On 2007-04-23 18:54:05 -0700, Eric Wong wrote:
> 
> > Karl Hasselström <kha@treskal.com> wrote:
> >
> > > When importing a whole repository, git-svn currently takes a _lot_
> > > of time. Almost all of it seems to be spent getting the full text
> > > of each and every file when a tag or new branch is created.
> >
> > Try SVN 1.4.3 with my patch[1] to get do_switch() working. trunk
> > should work without the patch.
> >
> > [1] - http://svn.haxx.se/dev/archive-2007-01/0936.shtml
> >
> > do_switch() is broken otherwise, and do_update() is extremely
> > inefficient.
> 
> Thanks for the help. I applied your patch to a clean 1.4.3 tree, and
> built subversion itself plus the perl bindings. But git-svn is still
> slow, and the testcase you gave in that thread still fails.
> 
> (I'm pretty sure that I'm actually using the bindings I've built; I
> install them to a nonstandard location, so if I don't set PERL5LIB
> they aren't found at all, and perl complains. When I do set it to my
> patched bindings, perl accepts it but your testcase fails.)

Hmm... I've been using 1.4.3 + patch since January on at least
one of my machines and do_switch() has worked correctly every time.

git-svn will display a message telling you that it's using do_switch()
when following a branch.  Otherwise, it'll say it's using do_update().

-- 
Eric Wong