Getting correct tree layout when importing svn repo into git

Getting correct tree layout when importing svn repo into git

[Michael Kerrisk]

Hello,

I'm currently trying to import an svn repository, along with its tags,
into git, and everything seems okay except that after the import I
expect to have the following structure to my checked out repository:

    [root-dir]
        .git
        <checked-out-files>

But instead I end up with

    [root-dir]
        .git
        man-pages               <-- name of my svn project
            <checked-out-files>

I've tried out a few different command-line flag settings but so far I
haven't managed to get the desired layout.  I guess that I'm missing
something trivial, but I haven't worked out what it is so far.

The commands I'm using to do the import are:

$ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
$ git svn fetch

The svn tags are getting imported okay, since:

git branch -a | head
  tags/man-pages-2.00
  tags/man-pages-2.01
  ...
  tags/man-pages-3.08
  tags/man-pages-3.09
  trunk

The following commands show the layout of my svn repo, which is pretty
much standard:

$ svn list file:///home/mtk/man-pages-rep
branches/
tags/
trunk/
$ svn list file:///home/mtk/man-pages-rep/trunk
man-pages/
$ svn list file:///home/mtk/man-pages-rep/trunk/man-pages
Changes
Changes.old
Makefile
README
man-pages-3.09.Announce
...
man7/
man8/
scripts/
$ svn list file:///home/mtk/man-pages-rep/tags
man-pages-2.00/
man-pages-2.01/
...
man-pages-3.08/
man-pages-3.09/

What 'git svn init' command do I need to get the layout that I want
for my imported git repo?

Cheers,

Michael

Re: Getting correct tree layout when importing svn repo into git

[Dmitry Potapov]

On Wed, Sep 17, 2008 at 2:05 AM, Michael Kerrisk
<mtk.manpages@googlemail.com> wrote:
>
> I'm currently trying to import an svn repository, along with its tags,
> into git, and everything seems okay except that after the import I
> expect to have the following structure to my checked out repository:
>
>    [root-dir]
>        .git
>        <checked-out-files>
>
> But instead I end up with
>
>    [root-dir]
>        .git
>        man-pages               <-- name of my svn project
>            <checked-out-files>
>
> I've tried out a few different command-line flag settings but so far I
> haven't managed to get the desired layout.  I guess that I'm missing
> something trivial, but I haven't worked out what it is so far.
>
> The commands I'm using to do the import are:
>
> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches

Probably, you want to run:

$ git svn init file:///home/mtk/man-pages-rep/ -t tags -T
trunk/man-pages -b branches

Dmitry

Re: Getting correct tree layout when importing svn repo into git

[Michael Kerrisk]

Hello Dmitry,

On Wed, Sep 17, 2008 at 1:20 AM, Dmitry Potapov <dpotapov@gmail.com> wrote:
> On Wed, Sep 17, 2008 at 2:05 AM, Michael Kerrisk
> <mtk.manpages@googlemail.com> wrote:
>>
>> I'm currently trying to import an svn repository, along with its tags,
>> into git, and everything seems okay except that after the import I
>> expect to have the following structure to my checked out repository:
>>
>>    [root-dir]
>>        .git
>>        <checked-out-files>
>>
>> But instead I end up with
>>
>>    [root-dir]
>>        .git
>>        man-pages               <-- name of my svn project
>>            <checked-out-files>
>>
>> I've tried out a few different command-line flag settings but so far I
>> haven't managed to get the desired layout.  I guess that I'm missing
>> something trivial, but I haven't worked out what it is so far.
>>
>> The commands I'm using to do the import are:
>>
>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
>
> Probably, you want to run:
>
> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T
> trunk/man-pages -b branches

I had wondered about this.  And I haven't yet had a chance to verify
the result.  The reason is that whereas the command:

$ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches

takes about half an hour to run, the other command (which I already
started yesterday) seems to be taking (far) more than a day!
Therefore, so far, I have not had a chance to run the command to
completion to see if it gives the desired result.  The greatly
increased tun time also made me suspicious about whether the command
was going to do the right thing.  And, I end up with a lot of strange
looking tags in the (as yet incompletely) imported tree:

$ git branch -a
  tags/man-pages-2.00
  tags/man-pages-2.00@117
  tags/man-pages-2.01
  tags/man-pages-2.01@145
  tags/man-pages-2.02
  tags/man-pages-2.02@184
  tags/man-pages-2.03
  tags/man-pages-2.03@232
  tags/man-pages-2.04
  tags/man-pages-2.04@283

What are the @nnn tags about?

By tomorrow, probably the command will complete and I can see whether
it produces the desired result.  But why does it take so much longer
to run this version of the command?

Cheers,

Michael

Re: Getting correct tree layout when importing svn repo into git

[Dmitry Potapov]

On Wed, Sep 17, 2008 at 01:40:36AM +0200, Michael Kerrisk wrote:
> 
> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
> 
> takes about half an hour to run, the other command (which I already
> started yesterday) seems to be taking (far) more than a day!

What version of Git do you use? If you use a version earlier than 1.5.6
than you can notice *considerable* speed up by upgrading to the latest
version of Git. Also, git-svn does not work well on Windows. Cygwin
version is slow due to fork(), and MSYS Git has some other issues with
Perl. So, if you want to convert a huge SVN repo, it is better to do on
Linux and using the latest version of Git. Perhaps, running on tmpfs may
speed up the process even more.

> Therefore, so far, I have not had a chance to run the command to
> completion to see if it gives the desired result.  The greatly
> increased tun time also made me suspicious about whether the command
> was going to do the right thing.  And, I end up with a lot of strange
> looking tags in the (as yet incompletely) imported tree:
> 
> $ git branch -a
>   tags/man-pages-2.00
>   tags/man-pages-2.00@117
>   tags/man-pages-2.01
>   tags/man-pages-2.01@145
>   tags/man-pages-2.02
>   tags/man-pages-2.02@184
>   tags/man-pages-2.03
>   tags/man-pages-2.03@232
>   tags/man-pages-2.04
>   tags/man-pages-2.04@283
> 
> What are the @nnn tags about?

I have never encounted them. Are you sure that you import into a clean
Git repo?

Dmitry

Re: Getting correct tree layout when importing svn repo into git

[Michael Kerrisk]

Hi Dimitry,

On Tue, Sep 16, 2008 at 6:18 PM, Dmitry Potapov <dpotapov@gmail.com> wrote:
> On Wed, Sep 17, 2008 at 01:40:36AM +0200, Michael Kerrisk wrote:
>>
>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
>>
>> takes about half an hour to run, the other command (which I already
>> started yesterday) seems to be taking (far) more than a day!
>
> What version of Git do you use?

1.5.4.5, on Linux.

> If you use a version earlier than 1.5.6
> than you can notice *considerable* speed up by upgrading to the latest

How much is "considerable"?  My import is *still* running.

> version of Git. Also, git-svn does not work well on Windows. Cygwin
> version is slow due to fork(), and MSYS Git has some other issues with
> Perl. So, if you want to convert a huge SVN repo,

My svn repo is not huge.  5000 commits, 176MB.

> it is better to do on
> Linux and using the latest version of Git. Perhaps, running on tmpfs may
> speed up the process even more.
>
>> Therefore, so far, I have not had a chance to run the command to
>> completion to see if it gives the desired result.  The greatly
>> increased tun time also made me suspicious about whether the command
>> was going to do the right thing.  And, I end up with a lot of strange
>> looking tags in the (as yet incompletely) imported tree:
>>
>> $ git branch -a
>>   tags/man-pages-2.00
>>   tags/man-pages-2.00@117
>>   tags/man-pages-2.01
>>   tags/man-pages-2.01@145
>>   tags/man-pages-2.02
>>   tags/man-pages-2.02@184
>>   tags/man-pages-2.03
>>   tags/man-pages-2.03@232
>>   tags/man-pages-2.04
>>   tags/man-pages-2.04@283
>>
>> What are the @nnn tags about?
>
> I have never encounted them. Are you sure that you import into a clean
> Git repo?

Yes.  It's a clean repo in a new directory.

Cheers,

Michael

Re: Getting correct tree layout when importing svn repo into git

[Andreas Ericsson]

Michael Kerrisk wrote:
> Hi Dimitry,
> 
> On Tue, Sep 16, 2008 at 6:18 PM, Dmitry Potapov <dpotapov@gmail.com> wrote:
>> On Wed, Sep 17, 2008 at 01:40:36AM +0200, Michael Kerrisk wrote:
>>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
>>>
>>> takes about half an hour to run, the other command (which I already
>>> started yesterday) seems to be taking (far) more than a day!
>> What version of Git do you use?
> 
> 1.5.4.5, on Linux.
> 

An upgrade would do you good.

>> If you use a version earlier than 1.5.6
>> than you can notice *considerable* speed up by upgrading to the latest
> 
> How much is "considerable"?  My import is *still* running.
> 

The greatest factor will usually be network round trip time, so if
this is a one-shot conversion, rsync the repository to somewhere
close and run the import over a local network. I've had some "fun"
trying to import sourceforge svn repositories that were large-ish
but not huge (same seat you're in, basically), but since I get up
to 800ms round trip time from sweden, conversion took forever.

>>
>>> Therefore, so far, I have not had a chance to run the command to
>>> completion to see if it gives the desired result.  The greatly
>>> increased tun time also made me suspicious about whether the command
>>> was going to do the right thing.  And, I end up with a lot of strange
>>> looking tags in the (as yet incompletely) imported tree:
>>>
>>> $ git branch -a
>>>   tags/man-pages-2.00
>>>   tags/man-pages-2.00@117
>>>   tags/man-pages-2.01
>>>   tags/man-pages-2.01@145
>>>   tags/man-pages-2.02
>>>   tags/man-pages-2.02@184
>>>   tags/man-pages-2.03
>>>   tags/man-pages-2.03@232
>>>   tags/man-pages-2.04
>>>   tags/man-pages-2.04@283
>>>
>>> What are the @nnn tags about?
>> I have never encounted them. Are you sure that you import into a clean
>> Git repo?
> 
> Yes.  It's a clean repo in a new directory.
> 

I've seen such tags before. They were added by an automatic build bot
as a marker to say "hey, I just built this".

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Getting correct tree layout when importing svn repo into git

[Michael J Gruber]

Michael Kerrisk venit, vidit, dixit 17.09.2008 00:05:
> Hello,
> 
> I'm currently trying to import an svn repository, along with its tags,
> into git, and everything seems okay except that after the import I
> expect to have the following structure to my checked out repository:
> 
>     [root-dir]
>         .git
>         <checked-out-files>
> 
> But instead I end up with
> 
>     [root-dir]
>         .git
>         man-pages               <-- name of my svn project
>             <checked-out-files>
> 
> I've tried out a few different command-line flag settings but so far I
> haven't managed to get the desired layout.  I guess that I'm missing
> something trivial, but I haven't worked out what it is so far.
> 
> The commands I'm using to do the import are:
> 
> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
> $ git svn fetch
> 
> The svn tags are getting imported okay, since:
> 
> git branch -a | head
>   tags/man-pages-2.00
>   tags/man-pages-2.01
>   ...
>   tags/man-pages-3.08
>   tags/man-pages-3.09
>   trunk
> 
> The following commands show the layout of my svn repo, which is pretty
> much standard:
> 
> $ svn list file:///home/mtk/man-pages-rep
> branches/
> tags/
> trunk/
> $ svn list file:///home/mtk/man-pages-rep/trunk
> man-pages/

That's the part that says that your layout is non-standard. What do your
 branches and tags look like? Do they have that superfluous "man-pages"
part as well?

> $ svn list file:///home/mtk/man-pages-rep/trunk/man-pages
> Changes
> Changes.old
> Makefile
> README
> man-pages-3.09.Announce
> ...
> man7/
> man8/
> scripts/
> $ svn list file:///home/mtk/man-pages-rep/tags
> man-pages-2.00/
> man-pages-2.01/
> ...
> man-pages-3.08/
> man-pages-3.09/
> 
> What 'git svn init' command do I need to get the layout that I want
> for my imported git repo?

It depends on the structure of your branches and tags. My guess would be
that you have something like tags/man-pages-2.00/man-pages/ etc., in
which case you would need a special configuration if you want to get rid
of the "man-pages" part. Tells us your structure, we'll tell you the
config ;)

Also, updating your git is strongly advised, but unrelated to the
problem at hand.

Michael

Re: Getting correct tree layout when importing svn repo into git

[Michael Kerrisk]

Hi Andreas,

On Tue, Sep 16, 2008 at 11:56 PM, Andreas Ericsson <ae@op5.se> wrote:
> Michael Kerrisk wrote:
>>
>> Hi Dimitry,
>>
>> On Tue, Sep 16, 2008 at 6:18 PM, Dmitry Potapov <dpotapov@gmail.com>
>> wrote:
>>>
>>> On Wed, Sep 17, 2008 at 01:40:36AM +0200, Michael Kerrisk wrote:
>>>>
>>>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b
>>>> branches
>>>>
>>>> takes about half an hour to run, the other command (which I already
>>>> started yesterday) seems to be taking (far) more than a day!
>>>
>>> What version of Git do you use?
>>
>> 1.5.4.5, on Linux.
>>
>
> An upgrade would do you good.

But, is it going to make this much difference to the run time?  By
now, the import using

$ git svn init file:///home/mtk/man-pages-rep/ -t tags -T
trunk/man-pages -b branches

has been running for over 2 days (and I still don't know if it will
give the layout I want), and seems to be slowing down exponentially as
it gets further along in the import process, so at this rate it looks
like it would take several more days to complete, whereas

$ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches

which doesn't give the layout I want, takes less than an hour.  We're
talking about a factor of at least 100 x in the speed difference for
the two imports.

>>> If you use a version earlier than 1.5.6
>>> than you can notice *considerable* speed up by upgrading to the latest
>>
>> How much is "considerable"?  My import is *still* running.
>>
>
> The greatest factor will usually be network round trip time, so if
> this is a one-shot conversion, rsync the repository to somewhere
> close and run the import over a local network. I've had some "fun"
> trying to import sourceforge svn repositories that were large-ish
> but not huge (same seat you're in, basically), but since I get up
> to 800ms round trip time from sweden, conversion took forever.

Perhaps I should have stated earleir, that the import is from a local
svn repo (note the 'file:///' in the 'git svn init'...).  176 MB, land
less than 5000 commits.

Cheers,

Michael

>>>> Therefore, so far, I have not had a chance to run the command to
>>>> completion to see if it gives the desired result.  The greatly
>>>> increased tun time also made me suspicious about whether the command
>>>> was going to do the right thing.  And, I end up with a lot of strange
>>>> looking tags in the (as yet incompletely) imported tree:
>>>>
>>>> $ git branch -a
>>>>  tags/man-pages-2.00
>>>>  tags/man-pages-2.00@117
>>>>  tags/man-pages-2.01
>>>>  tags/man-pages-2.01@145
>>>>  tags/man-pages-2.02
>>>>  tags/man-pages-2.02@184
>>>>  tags/man-pages-2.03
>>>>  tags/man-pages-2.03@232
>>>>  tags/man-pages-2.04
>>>>  tags/man-pages-2.04@283
>>>>
>>>> What are the @nnn tags about?
>>>
>>> I have never encounted them. Are you sure that you import into a clean
>>> Git repo?
>>
>> Yes.  It's a clean repo in a new directory.
>>
>
> I've seen such tags before. They were added by an automatic build bot
> as a marker to say "hey, I just built this".
>
> --
> Andreas Ericsson                   andreas.ericsson@op5.se
> OP5 AB                             www.op5.se
> Tel: +46 8-230225                  Fax: +46 8-230231
>



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html

Re: Getting correct tree layout when importing svn repo into git

[Michael Kerrisk]

Hello Michael,

On Wed, Sep 17, 2008 at 1:25 AM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> Michael Kerrisk venit, vidit, dixit 17.09.2008 00:05:
>> Hello,
>>
>> I'm currently trying to import an svn repository, along with its tags,
>> into git, and everything seems okay except that after the import I
>> expect to have the following structure to my checked out repository:
>>
>>     [root-dir]
>>         .git
>>         <checked-out-files>
>>
>> But instead I end up with
>>
>>     [root-dir]
>>         .git
>>         man-pages               <-- name of my svn project
>>             <checked-out-files>
>>
>> I've tried out a few different command-line flag settings but so far I
>> haven't managed to get the desired layout.  I guess that I'm missing
>> something trivial, but I haven't worked out what it is so far.
>>
>> The commands I'm using to do the import are:
>>
>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
>> $ git svn fetch
>>
>> The svn tags are getting imported okay, since:
>>
>> git branch -a | head
>>   tags/man-pages-2.00
>>   tags/man-pages-2.01
>>   ...
>>   tags/man-pages-3.08
>>   tags/man-pages-3.09
>>   trunk
>>
>> The following commands show the layout of my svn repo, which is pretty
>> much standard:
>>
>> $ svn list file:///home/mtk/man-pages-rep
>> branches/
>> tags/
>> trunk/
>> $ svn list file:///home/mtk/man-pages-rep/trunk
>> man-pages/
>
> That's the part that says that your layout is non-standard. What do your
>  branches and tags look like? Do they have that superfluous "man-pages"
> part as well?

$ svn list file:///home/mtk/man-pages-rep/tags
man-pages-2.00
man-pages-2.01
man-pages-2.02
man-pages-2.03
...
$ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.00
man-pages
$ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.01
man-pages
[and so on]
$ svn list file:///home/mtk/man-pages-rep/branches
$
(i.e., no branches, since this has been a linear svn repo.)

>> $ svn list file:///home/mtk/man-pages-rep/trunk/man-pages
>> Changes
>> Changes.old
>> Makefile
>> README
>> man-pages-3.09.Announce
>> ...
>> man7/
>> man8/
>> scripts/
>> $ svn list file:///home/mtk/man-pages-rep/tags
>> man-pages-2.00/
>> man-pages-2.01/
>> ...
>> man-pages-3.08/
>> man-pages-3.09/
>>
>> What 'git svn init' command do I need to get the layout that I want
>> for my imported git repo?
>
> It depends on the structure of your branches and tags. My guess would be
> that you have something like tags/man-pages-2.00/man-pages/ etc., in
> which case you would need a special configuration if you want to get rid
> of the "man-pages" part. Tells us your structure, we'll tell you the
> config ;)

Do you now have enough info above?

Thanks,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html

Re: Getting correct tree layout when importing svn repo into git

[Michael Gebetsroither]

* Michael Kerrisk <mtk.manpages@googlemail.com> wrote:

> But, is it going to make this much difference to the run time?  By
> now, the import using
>
> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T
> trunk/man-pages -b branches
>
> has been running for over 2 days (and I still don't know if it will
> give the layout I want), and seems to be slowing down exponentially as
> it gets further along in the import process, so at this rate it looks
> like it would take several more days to complete, whereas

git-svn conversions are _much_ faster here if done on tmpfs with
occasional git gc runs (in parallel).

May be the fault of the xfs.

cu,
michael
-- 
It's already too late!

Re: Getting correct tree layout when importing svn repo into git

[Björn Steinbrink]

On 2008.09.17 03:28:46 -0700, Michael Kerrisk wrote:
> Hi Andreas,
> 
> On Tue, Sep 16, 2008 at 11:56 PM, Andreas Ericsson <ae@op5.se> wrote:
> > Michael Kerrisk wrote:
> >>
> >> Hi Dimitry,
> >>
> >> On Tue, Sep 16, 2008 at 6:18 PM, Dmitry Potapov <dpotapov@gmail.com>
> >> wrote:
> >>>
> >>> On Wed, Sep 17, 2008 at 01:40:36AM +0200, Michael Kerrisk wrote:
> >>>>
> >>>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b
> >>>> branches
> >>>>
> >>>> takes about half an hour to run, the other command (which I already
> >>>> started yesterday) seems to be taking (far) more than a day!
> >>>
> >>> What version of Git do you use?
> >>
> >> 1.5.4.5, on Linux.
> >>
> >
> > An upgrade would do you good.
> 
> But, is it going to make this much difference to the run time?  By
> now, the import using

It should.

> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T
> trunk/man-pages -b branches
> 
> has been running for over 2 days (and I still don't know if it will
> give the layout I want), and seems to be slowing down exponentially as
> it gets further along in the import process, so at this rate it looks
> like it would take several more days to complete, whereas
> 
> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
> 
> which doesn't give the layout I want, takes less than an hour.  We're
> talking about a factor of at least 100 x in the speed difference for
> the two imports.

I guess your old import doesn't see any relations between the trunk and
the tags/branches, right? Then the huge increase in runtime would be, at
least in part, explained by git-svn searching for ancestry. That you see
those tags/1.2.3@123 things also means that the 1.2.3 tags got deleted
and recreated, or at least git-svn thinks so. Maybe the ancestry
detection goes nuts, hard to tell.

Is that svn repo available anywhere?

Björn

Re: Getting correct tree layout when importing svn repo into git

[Björn Steinbrink]

On 2008.09.17 03:38:12 -0700, Michael Kerrisk wrote:
> $ svn list file:///home/mtk/man-pages-rep/tags
> man-pages-2.00
> man-pages-2.01
> man-pages-2.02
> man-pages-2.03
> ...
> $ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.00
> man-pages
> $ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.01
> man-pages
> [and so on]
> $ svn list file:///home/mtk/man-pages-rep/branches
> $
> (i.e., no branches, since this has been a linear svn repo.)

That means that you also need to pass the "man-pages" suffix for the
tags and branches.

git svn clone file:///path/to/repo \
	-T trunk/man-pages \
	-b branches/*/man-pages \
	-t tags/*/man-pages

Might require a more recent git-svn version, not sure about that.

Björn

Re: Getting correct tree layout when importing svn repo into git

[Michael J Gruber]

Michael Kerrisk venit, vidit, dixit 17.09.2008 12:38:
> Hello Michael,
> 
> On Wed, Sep 17, 2008 at 1:25 AM, Michael J Gruber
> <git@drmicha.warpmail.net> wrote:
>> Michael Kerrisk venit, vidit, dixit 17.09.2008 00:05:
>>> Hello,
>>>
>>> I'm currently trying to import an svn repository, along with its tags,
>>> into git, and everything seems okay except that after the import I
>>> expect to have the following structure to my checked out repository:
>>>
>>>     [root-dir]
>>>         .git
>>>         <checked-out-files>
>>>
>>> But instead I end up with
>>>
>>>     [root-dir]
>>>         .git
>>>         man-pages               <-- name of my svn project
>>>             <checked-out-files>
>>>
>>> I've tried out a few different command-line flag settings but so far I
>>> haven't managed to get the desired layout.  I guess that I'm missing
>>> something trivial, but I haven't worked out what it is so far.
>>>
>>> The commands I'm using to do the import are:
>>>
>>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
>>> $ git svn fetch
>>>
>>> The svn tags are getting imported okay, since:
>>>
>>> git branch -a | head
>>>   tags/man-pages-2.00
>>>   tags/man-pages-2.01
>>>   ...
>>>   tags/man-pages-3.08
>>>   tags/man-pages-3.09
>>>   trunk
>>>
>>> The following commands show the layout of my svn repo, which is pretty
>>> much standard:
>>>
>>> $ svn list file:///home/mtk/man-pages-rep
>>> branches/
>>> tags/
>>> trunk/
>>> $ svn list file:///home/mtk/man-pages-rep/trunk
>>> man-pages/
>> That's the part that says that your layout is non-standard. What do your
>>  branches and tags look like? Do they have that superfluous "man-pages"
>> part as well?
> 
> $ svn list file:///home/mtk/man-pages-rep/tags
> man-pages-2.00
> man-pages-2.01
> man-pages-2.02
> man-pages-2.03
> ...
> $ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.00
> man-pages
> $ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.01
> man-pages
> [and so on]

Bingo!

> $ svn list file:///home/mtk/man-pages-rep/branches
> $
> (i.e., no branches, since this has been a linear svn repo.)

git-svn converts svn tags into git branches (because they can change).
So you will get a git repo with lots of branches which are subsets (HEAD
commit is in master/trunk) of master/trunk.

>>> $ svn list file:///home/mtk/man-pages-rep/trunk/man-pages
>>> Changes
>>> Changes.old
>>> Makefile
>>> README
>>> man-pages-3.09.Announce
>>> ...
>>> man7/
>>> man8/
>>> scripts/
>>> $ svn list file:///home/mtk/man-pages-rep/tags
>>> man-pages-2.00/
>>> man-pages-2.01/
>>> ...
>>> man-pages-3.08/
>>> man-pages-3.09/
>>>
>>> What 'git svn init' command do I need to get the layout that I want
>>> for my imported git repo?
>> It depends on the structure of your branches and tags. My guess would be
>> that you have something like tags/man-pages-2.00/man-pages/ etc., in
>> which case you would need a special configuration if you want to get rid
>> of the "man-pages" part. Tells us your structure, we'll tell you the
>> config ;)
> 
> Do you now have enough info above?

Yes, thanks. If you use git-svn init -s (which is equivalent to -T trunk
-t tags -b branches) you get an "exact" clone of your svn repo in
reasonable time, as you already noticed.

In order to get rid of the "man-pages" path component you can either:

- remove it after the fact using git filter-branch

or

- remove it during the conversion process

For the latter, do the following:

git svn init -s file:///home/mtk/man-pages-rep/

as usual. Then, before the first fetch, make sure your .git/config svn
section looks as follows:

[svn-remote "svn"]
        url = file:///home/mtk/man-pages-rep/
        fetch = trunk/man-pages:refs/remotes/trunk
        branches = branches/*/man-pages:refs/remotes/*
        tags = tags/*/man-pages:refs/remotes/tags/*

That is, if you used "git svn init -s" you only have to add "/man-pages"
three times on the left hand side of the refspecs. You can also do this
using "git config", of course, if you watch out for the shell quoting.

Cheers,
Michael

Re: Getting correct tree layout when importing svn repo into git

[Michael Kerrisk]

Björn,

On Wed, Sep 17, 2008 at 3:56 AM, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:
> On 2008.09.17 03:28:46 -0700, Michael Kerrisk wrote:
>> Hi Andreas,
>>
>> On Tue, Sep 16, 2008 at 11:56 PM, Andreas Ericsson <ae@op5.se> wrote:
>> > Michael Kerrisk wrote:
>> >>
>> >> Hi Dimitry,
>> >>
>> >> On Tue, Sep 16, 2008 at 6:18 PM, Dmitry Potapov <dpotapov@gmail.com>
>> >> wrote:
>> >>>
>> >>> On Wed, Sep 17, 2008 at 01:40:36AM +0200, Michael Kerrisk wrote:
>> >>>>
>> >>>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b
>> >>>> branches
>> >>>>
>> >>>> takes about half an hour to run, the other command (which I already
>> >>>> started yesterday) seems to be taking (far) more than a day!
>> >>>
>> >>> What version of Git do you use?
>> >>
>> >> 1.5.4.5, on Linux.
>> >>
>> >
>> > An upgrade would do you good.
>>
>> But, is it going to make this much difference to the run time?  By
>> now, the import using
>
> It should.
>
>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T
>> trunk/man-pages -b branches
>>
>> has been running for over 2 days (and I still don't know if it will
>> give the layout I want), and seems to be slowing down exponentially as
>> it gets further along in the import process, so at this rate it looks
>> like it would take several more days to complete, whereas
>>
>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
>>
>> which doesn't give the layout I want, takes less than an hour.  We're
>> talking about a factor of at least 100 x in the speed difference for
>> the two imports.
>
> I guess your old import doesn't see any relations between the trunk and
> the tags/branches, right? Then the huge increase in runtime would be, at
> least in part, explained by git-svn searching for ancestry. That you see
> those tags/1.2.3@123 things also means that the 1.2.3 tags got deleted
> and recreated, or at least git-svn thinks so. Maybe the ancestry
> detection goes nuts, hard to tell.

The above sounds very plausible as an explanation of what I'm seeing.

> Is that svn repo available anywhere?

It's local.  If someone felt the need to look at it to see if there is
something that could be improved in git (and maybe already is by the
current release), then I could put a copy somewhere.

Anyway, it looks like I have a fix that works from someone else in the
thread -- see my reply there.

Thanks!

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html

Re: Getting correct tree layout when importing svn repo into git

[Michael Kerrisk]

Hi Björn,

D'oh -- the fix was from you too ;-).  See below.

On Wed, Sep 17, 2008 at 4:05 AM, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:
> On 2008.09.17 03:38:12 -0700, Michael Kerrisk wrote:
>> $ svn list file:///home/mtk/man-pages-rep/tags
>> man-pages-2.00
>> man-pages-2.01
>> man-pages-2.02
>> man-pages-2.03
>> ...
>> $ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.00
>> man-pages
>> $ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.01
>> man-pages
>> [and so on]
>> $ svn list file:///home/mtk/man-pages-rep/branches
>> $
>> (i.e., no branches, since this has been a linear svn repo.)
>
> That means that you also need to pass the "man-pages" suffix for the
> tags and branches.
>
> git svn clone file:///path/to/repo \
>        -T trunk/man-pages \
>        -b branches/*/man-pages \
>        -t tags/*/man-pages
>
> Might require a more recent git-svn version, not sure about that.

Using my current git version, this seems to produce the right result.
Thank you!  (I guess Michael's solution is essentially equivalent, but
I tried yours first.)

Thanks everyone!

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html

Re: Getting correct tree layout when importing svn repo into git

[Michael Kerrisk]

Michael,

On Wed, Sep 17, 2008 at 4:14 AM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> Michael Kerrisk venit, vidit, dixit 17.09.2008 12:38:
>> Hello Michael,
>>
>> On Wed, Sep 17, 2008 at 1:25 AM, Michael J Gruber
>> <git@drmicha.warpmail.net> wrote:
>>> Michael Kerrisk venit, vidit, dixit 17.09.2008 00:05:
>>>> Hello,
>>>>
>>>> I'm currently trying to import an svn repository, along with its tags,
>>>> into git, and everything seems okay except that after the import I
>>>> expect to have the following structure to my checked out repository:
>>>>
>>>>     [root-dir]
>>>>         .git
>>>>         <checked-out-files>
>>>>
>>>> But instead I end up with
>>>>
>>>>     [root-dir]
>>>>         .git
>>>>         man-pages               <-- name of my svn project
>>>>             <checked-out-files>
>>>>
>>>> I've tried out a few different command-line flag settings but so far I
>>>> haven't managed to get the desired layout.  I guess that I'm missing
>>>> something trivial, but I haven't worked out what it is so far.
>>>>
>>>> The commands I'm using to do the import are:
>>>>
>>>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
>>>> $ git svn fetch
>>>>
>>>> The svn tags are getting imported okay, since:
>>>>
>>>> git branch -a | head
>>>>   tags/man-pages-2.00
>>>>   tags/man-pages-2.01
>>>>   ...
>>>>   tags/man-pages-3.08
>>>>   tags/man-pages-3.09
>>>>   trunk
>>>>
>>>> The following commands show the layout of my svn repo, which is pretty
>>>> much standard:
>>>>
>>>> $ svn list file:///home/mtk/man-pages-rep
>>>> branches/
>>>> tags/
>>>> trunk/
>>>> $ svn list file:///home/mtk/man-pages-rep/trunk
>>>> man-pages/
>>> That's the part that says that your layout is non-standard. What do your
>>>  branches and tags look like? Do they have that superfluous "man-pages"
>>> part as well?
>>
>> $ svn list file:///home/mtk/man-pages-rep/tags
>> man-pages-2.00
>> man-pages-2.01
>> man-pages-2.02
>> man-pages-2.03
>> ...
>> $ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.00
>> man-pages
>> $ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.01
>> man-pages
>> [and so on]
>
> Bingo!
>
>> $ svn list file:///home/mtk/man-pages-rep/branches
>> $
>> (i.e., no branches, since this has been a linear svn repo.)
>
> git-svn converts svn tags into git branches (because they can change).
> So you will get a git repo with lots of branches which are subsets (HEAD
> commit is in master/trunk) of master/trunk.
>
>>>> $ svn list file:///home/mtk/man-pages-rep/trunk/man-pages
>>>> Changes
>>>> Changes.old
>>>> Makefile
>>>> README
>>>> man-pages-3.09.Announce
>>>> ...
>>>> man7/
>>>> man8/
>>>> scripts/
>>>> $ svn list file:///home/mtk/man-pages-rep/tags
>>>> man-pages-2.00/
>>>> man-pages-2.01/
>>>> ...
>>>> man-pages-3.08/
>>>> man-pages-3.09/
>>>>
>>>> What 'git svn init' command do I need to get the layout that I want
>>>> for my imported git repo?
>>> It depends on the structure of your branches and tags. My guess would be
>>> that you have something like tags/man-pages-2.00/man-pages/ etc., in
>>> which case you would need a special configuration if you want to get rid
>>> of the "man-pages" part. Tells us your structure, we'll tell you the
>>> config ;)
>>
>> Do you now have enough info above?
>
> Yes, thanks. If you use git-svn init -s (which is equivalent to -T trunk
> -t tags -b branches) you get an "exact" clone of your svn repo in
> reasonable time, as you already noticed.
>
> In order to get rid of the "man-pages" path component you can either:
>
> - remove it after the fact using git filter-branch
>
> or
>
> - remove it during the conversion process
>
> For the latter, do the following:
>
> git svn init -s file:///home/mtk/man-pages-rep/
>
> as usual. Then, before the first fetch, make sure your .git/config svn
> section looks as follows:
>
> [svn-remote "svn"]
>        url = file:///home/mtk/man-pages-rep/
>        fetch = trunk/man-pages:refs/remotes/trunk
>        branches = branches/*/man-pages:refs/remotes/*
>        tags = tags/*/man-pages:refs/remotes/tags/*

I guess your solution here was equivalent to Björn's, since after I
tried his suggestion, I ended up with an [svn-remote "svn"] that looks
just as you describe.

Thanks,

Michael

> That is, if you used "git svn init -s" you only have to add "/man-pages"
> three times on the left hand side of the refspecs. You can also do this
> using "git config", of course, if you watch out for the shell quoting.
>
> Cheers,
> Michael
>



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html

Re: Getting correct tree layout when importing svn repo into git

[Michael J Gruber]

Michael Kerrisk venit, vidit, dixit 17.09.2008 15:00:
> Michael,
> 
> On Wed, Sep 17, 2008 at 4:14 AM, Michael J Gruber
> <git@drmicha.warpmail.net> wrote:
>> Michael Kerrisk venit, vidit, dixit 17.09.2008 12:38:
>>> Hello Michael,
>>>
>>> On Wed, Sep 17, 2008 at 1:25 AM, Michael J Gruber
>>> <git@drmicha.warpmail.net> wrote:
>>>> Michael Kerrisk venit, vidit, dixit 17.09.2008 00:05:
>>>>> Hello,
>>>>>
>>>>> I'm currently trying to import an svn repository, along with its tags,
>>>>> into git, and everything seems okay except that after the import I
>>>>> expect to have the following structure to my checked out repository:
>>>>>
>>>>>     [root-dir]
>>>>>         .git
>>>>>         <checked-out-files>
>>>>>
>>>>> But instead I end up with
>>>>>
>>>>>     [root-dir]
>>>>>         .git
>>>>>         man-pages               <-- name of my svn project
>>>>>             <checked-out-files>
>>>>>
>>>>> I've tried out a few different command-line flag settings but so far I
>>>>> haven't managed to get the desired layout.  I guess that I'm missing
>>>>> something trivial, but I haven't worked out what it is so far.
>>>>>
>>>>> The commands I'm using to do the import are:
>>>>>
>>>>> $ git svn init file:///home/mtk/man-pages-rep/ -t tags -T trunk -b branches
>>>>> $ git svn fetch
>>>>>
>>>>> The svn tags are getting imported okay, since:
>>>>>
>>>>> git branch -a | head
>>>>>   tags/man-pages-2.00
>>>>>   tags/man-pages-2.01
>>>>>   ...
>>>>>   tags/man-pages-3.08
>>>>>   tags/man-pages-3.09
>>>>>   trunk
>>>>>
>>>>> The following commands show the layout of my svn repo, which is pretty
>>>>> much standard:
>>>>>
>>>>> $ svn list file:///home/mtk/man-pages-rep
>>>>> branches/
>>>>> tags/
>>>>> trunk/
>>>>> $ svn list file:///home/mtk/man-pages-rep/trunk
>>>>> man-pages/
>>>> That's the part that says that your layout is non-standard. What do your
>>>>  branches and tags look like? Do they have that superfluous "man-pages"
>>>> part as well?
>>> $ svn list file:///home/mtk/man-pages-rep/tags
>>> man-pages-2.00
>>> man-pages-2.01
>>> man-pages-2.02
>>> man-pages-2.03
>>> ...
>>> $ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.00
>>> man-pages
>>> $ svn list file:///home/mtk/man-pages-rep/tags/man-pages-2.01
>>> man-pages
>>> [and so on]
>> Bingo!
>>
>>> $ svn list file:///home/mtk/man-pages-rep/branches
>>> $
>>> (i.e., no branches, since this has been a linear svn repo.)
>> git-svn converts svn tags into git branches (because they can change).
>> So you will get a git repo with lots of branches which are subsets (HEAD
>> commit is in master/trunk) of master/trunk.
>>
>>>>> $ svn list file:///home/mtk/man-pages-rep/trunk/man-pages
>>>>> Changes
>>>>> Changes.old
>>>>> Makefile
>>>>> README
>>>>> man-pages-3.09.Announce
>>>>> ...
>>>>> man7/
>>>>> man8/
>>>>> scripts/
>>>>> $ svn list file:///home/mtk/man-pages-rep/tags
>>>>> man-pages-2.00/
>>>>> man-pages-2.01/
>>>>> ...
>>>>> man-pages-3.08/
>>>>> man-pages-3.09/
>>>>>
>>>>> What 'git svn init' command do I need to get the layout that I want
>>>>> for my imported git repo?
>>>> It depends on the structure of your branches and tags. My guess would be
>>>> that you have something like tags/man-pages-2.00/man-pages/ etc., in
>>>> which case you would need a special configuration if you want to get rid
>>>> of the "man-pages" part. Tells us your structure, we'll tell you the
>>>> config ;)
>>> Do you now have enough info above?
>> Yes, thanks. If you use git-svn init -s (which is equivalent to -T trunk
>> -t tags -b branches) you get an "exact" clone of your svn repo in
>> reasonable time, as you already noticed.
>>
>> In order to get rid of the "man-pages" path component you can either:
>>
>> - remove it after the fact using git filter-branch
>>
>> or
>>
>> - remove it during the conversion process
>>
>> For the latter, do the following:
>>
>> git svn init -s file:///home/mtk/man-pages-rep/
>>
>> as usual. Then, before the first fetch, make sure your .git/config svn
>> section looks as follows:
>>
>> [svn-remote "svn"]
>>        url = file:///home/mtk/man-pages-rep/
>>        fetch = trunk/man-pages:refs/remotes/trunk
>>        branches = branches/*/man-pages:refs/remotes/*
>>        tags = tags/*/man-pages:refs/remotes/tags/*
> 
> I guess your solution here was equivalent to Björn's, since after I
> tried his suggestion, I ended up with an [svn-remote "svn"] that looks
> just as you describe.
> 
> Thanks,
> 
> Michael

Yes. You mentioned you used an old version. Newer ones accept the
parameters in the way Björn described. Older ones accepted the same
config, just not as parameters.

That kind of config is well-suited for splitting large svn-repos which
consist of well-separated subprojects.

So, welcome among the happy svn->git converts ;)

Michael

>> That is, if you used "git svn init -s" you only have to add "/man-pages"
>> three times on the left hand side of the refspecs. You can also do this
>> using "git config", of course, if you watch out for the shell quoting.
>>
>> Cheers,
>> Michael
>>
> 
> 
>