Speed of conversion from Subversion repository to Git

Speed of conversion from Subversion repository to Git

[Markus Elfring]

Hello,

I am trying to convert the YaST Subversion repository into a Git variant on my
system. Unfortunately, a command like "svn2git
http://svn.opensuse.org/svn/yast/" does not finish this attempt after 1,5 days
(> 715:13.24 CPU time) here. I would appreciate suggestions for improvements of
the execution speed.
(Renicing of the process "git-svn" does not help to increase its processor
utilisation in noticeable ways.)

Regards,
Markus

Re: Speed of conversion from Subversion repository to Git

[Jacob Helwig]

On Mon, Feb 1, 2010 at 10:05, Markus Elfring <Markus.Elfring@web.de> wrote:
> Hello,
>
> I am trying to convert the YaST Subversion repository into a Git variant on my
> system. Unfortunately, a command like "svn2git
> http://svn.opensuse.org/svn/yast/" does not finish this attempt after 1,5 days
> (> 715:13.24 CPU time) here. I would appreciate suggestions for improvements of
> the execution speed.
> (Renicing of the process "git-svn" does not help to increase its processor
> utilisation in noticeable ways.)
>
> Regards,
> Markus
>

We're in the process of converting to Git at work, and I've found that
svn-all-fast-export was the best option for us.  git-svn's behavior of
a branch's entire history every time it found a new branch (even
though it has already seen 3/4 of the revisions involved) caused it to
be far too slow for our needs.  Admittedly, this was ~1 year ago that
I tried git-svn to do a test conversion, and things might have
improved since then.

svn-all-fast-export does require access to the raw repository, instead
of just (http|svn|svn+ssh):// access, but it does allow you to split
out one SVN "repository" into multiple Git repositories in one go.

-Jacob

Re: Speed of conversion from Subversion repository to Git

[Markus Elfring]

Hello,

The execution of the command "svn2git http://svn.opensuse.org/svn/yast/" was finished today somewhere during the night. I used the version 1.3.1 together with Git 1.6.6.1 on my openSUSE 11.2 system. Now I hope that my conversion try was really successful.

elfring@Sonne:~/Projekte/YaST/lokal> du -hs .git . && git branch
518M    .git
653M    .
  Moblin
  REFACTORING-10_3
  SuSE-Code-11-Branch
  SuSE-Code-11-SP1-Branch
  SuSE-Linux-10_0-Branch
  SuSE-Linux-10_1-Branch
  SuSE-Linux-10_1-Branch@30668
  SuSE-Linux-10_2-Branch
  SuSE-Linux-10_3-Branch
  SuSE-Linux-11_0-Branch
  SuSE-Linux-11_2-Branch
  SuSE-Linux-9_3-Branch
  SuSE-SLE-10-Branch
  SuSE-SLE-10-SP1-Branch
  SuSE-SLE-10-SP1-Features-Branch
  SuSE-SLE-10-SP2-Branch
  SuSE-SLE-10-SP3-Branch
  YaST2-X11-2-13-SuSE-10-1
  YaST2-X11-2-13-SuSE-10-1@29086
  YaST2-X11-2-14-SuSE-10-2
  YaST2-X11-2-14-SuSE-10-2@34577
* master
  moblin
  research
  selection_widgets_cleanup-core
  selection_widgets_cleanup-core@22138
  selection_widgets_cleanup-qt
  selection_widgets_cleanup-qt@22138
  tmp
  unsupported

The network activity was mostly low during this lengthy process. Are more software developers interested to speed this procedure up?
Do know ways to improve the processor utilisation so that such a data import will be completed faster?

Regards,
Markus

Re: Speed of conversion from Subversion repository to Git

[Markus Elfring]

Hello,

Are there any chances to increase the processor utilisation in noticeable ways by multi-threaded programming?
http://git.kernel.org/?p=git/git.git;a=blob;f=git-svn.perl;h=265852f4596bfe5aeca12be06f78631320b8ebb4;hb=HEAD

Do any conversion tasks exist which would benefit from parallel processing?

Regards,
Markus