Errors importing Apache Synapse SVN using Git

Errors importing Apache Synapse SVN using Git

[Asankha C. Perera]

Hi All

I am an Apache Synapse developer, and want to import the Synapse SVN 
repo into Git, so that Ohloh can properly get the Synapse history 
(http://www.ohloh.net/topics/1326?page=1#post_6287)

However, when I try the command: "git svn clone --trunk=trunk 
--tags=tags --branches=branches http://svn.apache.org/repos/asf/synapse" 
it seems to take forever, (or at least until the next network glitch), 
and keeps filling up a file with just plain zeros ("0") : 
./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68

Can someone try the above command on the Synapse repo and tell me what I 
can do to import from the SVN?

many thanks
asankha

Re: Errors importing Apache Synapse SVN using Git

[Michael J Gruber]

Asankha C. Perera venit, vidit, dixit 10.07.2008 14:01:
> Hi All
> 
> I am an Apache Synapse developer, and want to import the Synapse SVN 
> repo into Git, so that Ohloh can properly get the Synapse history 
> (http://www.ohloh.net/topics/1326?page=1#post_6287)
> 
> However, when I try the command: "git svn clone --trunk=trunk 
> --tags=tags --branches=branches http://svn.apache.org/repos/asf/synapse" 
> it seems to take forever, (or at least until the next network glitch), 
> and keeps filling up a file with just plain zeros ("0") : 
> ./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68
> 
> Can someone try the above command on the Synapse repo and tell me what I 
> can do to import from the SVN?

"svn log" takes forever on that repo, too. Current rev seems to be 
675546, and the synapse path does not exist in early revisions. Knowing 
the initial revision would help, then you could save "git svn" from 
having to comb through (supposedly) tens of thousands of irrelevant revs.

I just checked out trunk using svn 1.4.6, "svn log ." takes forever in 
the root dir. So the svn repo seems to be largely unusable, at least 
when accessed from svn 1.4.6 clients (the server is 1.5.0, I see).

Okay, I bisected it and got r234477 as the beginning of time for 
synapse. "svn log -r 234477:HEAD" is still painful.

You may want to fetch 1000 revs each or so from there each time.

Michael

Re: Errors importing Apache Synapse SVN using Git

[Björn Steinbrink]

On 2008.07.10 14:59:11 +0200, Michael J Gruber wrote:
> Asankha C. Perera venit, vidit, dixit 10.07.2008 14:01:
>> Hi All
>>
>> I am an Apache Synapse developer, and want to import the Synapse SVN  
>> repo into Git, so that Ohloh can properly get the Synapse history  
>> (http://www.ohloh.net/topics/1326?page=1#post_6287)
>>
>> However, when I try the command: "git svn clone --trunk=trunk  
>> --tags=tags --branches=branches 
>> http://svn.apache.org/repos/asf/synapse" it seems to take forever, (or 
>> at least until the next network glitch), and keeps filling up a file 
>> with just plain zeros ("0") :  
>> ./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68
>>
>> Can someone try the above command on the Synapse repo and tell me what 
>> I can do to import from the SVN?
>
> "svn log" takes forever on that repo, too. Current rev seems to be  
> 675546, and the synapse path does not exist in early revisions. Knowing  
> the initial revision would help, then you could save "git svn" from  
> having to comb through (supposedly) tens of thousands of irrelevant revs.
>
> I just checked out trunk using svn 1.4.6, "svn log ." takes forever in  
> the root dir. So the svn repo seems to be largely unusable, at least  
> when accessed from svn 1.4.6 clients (the server is 1.5.0, I see).
>
> Okay, I bisected it and got r234477 as the beginning of time for  
> synapse. "svn log -r 234477:HEAD" is still painful.
>
> You may want to fetch 1000 revs each or so from there each time.

Also, upgrading git to 1.5.6.2 might be a good idea. It doesn't use the
.revdb file format anymore, but a more efficient .revmap file. And it
has quite a few performance improvements (although they won't help
against the primary issue with that svn server). Don't use
1.5.6/1.5.6.1, they have a git-svn bug that can lead to corrupted clones
(just in case that your distro has packages for those versions but not
yet got 1.5.6.2).

Björn

Re: Errors importing Apache Synapse SVN using Git

[Michael J Gruber]

Asankha C. Perera venit, vidit, dixit 10.07.2008 14:01:
> Hi All
> 
> I am an Apache Synapse developer, and want to import the Synapse SVN 
> repo into Git, so that Ohloh can properly get the Synapse history 
> (http://www.ohloh.net/topics/1326?page=1#post_6287)
> 
> However, when I try the command: "git svn clone --trunk=trunk 
> --tags=tags --branches=branches http://svn.apache.org/repos/asf/synapse" 
> it seems to take forever, (or at least until the next network glitch), 
> and keeps filling up a file with just plain zeros ("0") : 
> ./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68

Okay, besides the zero offset, there seems to be big problem with the 
svn repo:

svn list -r 234478 http://svn.apache.org/repos/asf/synapse
branches/
site/
tags/
trunk/

svn list -r 234478 http://svn.apache.org/repos/asf|grep synapse

Right, nothing returned.

git-svn insists on using http://svn.apache.org/repos/asf as the base 
URL, which is basically correct, but doesn't work for this repo.

Even editing .git/config and .git/svn/metadata (after git svn init, 
befire the first fetch) to change the url doesn't help. git svn still 
uses .../asf. Is this as intended? cc'ing the git svn author.

Michael

Re: Errors importing Apache Synapse SVN using Git

[Eric Wong]

Michael J Gruber <michaeljgruber+gmane@fastmail.fm> wrote:
> Asankha C. Perera venit, vidit, dixit 10.07.2008 14:01:
> >Hi All
> >
> >I am an Apache Synapse developer, and want to import the Synapse SVN 
> >repo into Git, so that Ohloh can properly get the Synapse history 
> >(http://www.ohloh.net/topics/1326?page=1#post_6287)
> >
> >However, when I try the command: "git svn clone --trunk=trunk 
> >--tags=tags --branches=branches http://svn.apache.org/repos/asf/synapse" 
> >it seems to take forever, (or at least until the next network glitch), 
> >and keeps filling up a file with just plain zeros ("0") : 
> >./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68

The latest versions of git-svn should be vastly more space efficient
with .rev_map files.
 
> Okay, besides the zero offset, there seems to be big problem with the 
> svn repo:
> 
> svn list -r 234478 http://svn.apache.org/repos/asf/synapse
> branches/
> site/
> tags/
> trunk/
> 
> svn list -r 234478 http://svn.apache.org/repos/asf|grep synapse

Ah, that's because it started off in the /incubator directory, not /synapse:

$ svn log -v -r 234478 http://svn.apache.org/repos/asf
------------------------------------------------------------------------
r234478 | dims | 2005-08-22 05:39:51 -0700 (Mon, 22 Aug 2005) | 3 lines
Changed paths:
   A /incubator/synapse/branches
   A /incubator/synapse/site
   A /incubator/synapse/tags
   A /incubator/synapse/trunk

init synapse SVN


------------------------------------------------------------------------

> Right, nothing returned.
> 
> git-svn insists on using http://svn.apache.org/repos/asf as the base 
> URL, which is basically correct, but doesn't work for this repo.
> 
> Even editing .git/config and .git/svn/metadata (after git svn init, 
> befire the first fetch) to change the url doesn't help. git svn still 
> uses .../asf. Is this as intended? cc'ing the git svn author.

Can you try using the latest git-svn and also --no-follow-parent?

--no-follow-parent won't traverse copy history so it may be faster,
but you'll get an incomplete history.

On the other hand, it looks like the asf repo is hopelessly slow
for history retrieval.

I'm pulling the asf log (svn log -v http://svn.apache.org/repos/asf >
tmp.log) but it's taking forever...  git-svn uses basically the same
operation as 'svn log -v' and I've yet to see an SVN server take this
long to show history.

I've gotten 70,000 revisions after over one hour with the above svn log
command.  I'm going to restart this from a server with a better
connection since I need to power off this one when I sleep.

-- 
Eric Wong

Re: Errors importing Apache Synapse SVN using Git

[Michael J Gruber]

Eric Wong venit, vidit, dixit 14.07.2008 10:16:
> Michael J Gruber <michaeljgruber+gmane@fastmail.fm> wrote:
>> Asankha C. Perera venit, vidit, dixit 10.07.2008 14:01:
>>> Hi All
>>>
>>> I am an Apache Synapse developer, and want to import the Synapse SVN 
>>> repo into Git, so that Ohloh can properly get the Synapse history 
>>> (http://www.ohloh.net/topics/1326?page=1#post_6287)
>>>
>>> However, when I try the command: "git svn clone --trunk=trunk 
>>> --tags=tags --branches=branches http://svn.apache.org/repos/asf/synapse" 
>>> it seems to take forever, (or at least until the next network glitch), 
>>> and keeps filling up a file with just plain zeros ("0") : 
>>> ./.git/svn/trunk/.rev_db.13f79535-47bb-0310-9956-ffa450edef68
> 
> The latest versions of git-svn should be vastly more space efficient
> with .rev_map files.
>  
>> Okay, besides the zero offset, there seems to be big problem with the 
>> svn repo:
>>
>> svn list -r 234478 http://svn.apache.org/repos/asf/synapse
>> branches/
>> site/
>> tags/
>> trunk/
>>
>> svn list -r 234478 http://svn.apache.org/repos/asf|grep synapse
> 
> Ah, that's because it started off in the /incubator directory, not /synapse:
> 
> $ svn log -v -r 234478 http://svn.apache.org/repos/asf
> ------------------------------------------------------------------------
> r234478 | dims | 2005-08-22 05:39:51 -0700 (Mon, 22 Aug 2005) | 3 lines
> Changed paths:
>    A /incubator/synapse/branches
>    A /incubator/synapse/site
>    A /incubator/synapse/tags
>    A /incubator/synapse/trunk
> 
> init synapse SVN

Yes, and later on there was some more moving around to 
"webservices/synapse/synapse", ".../syaps2", ".../synapse" (i.e. 
renaming "synapse/synapse" to "synapse") and then to the final destination.

Some svn client command know how to follow moves, other don't. I can't 
make them return results by using peg revisions, but I don't know how 
git-svn handles them.


> ------------------------------------------------------------------------
> 
>> Right, nothing returned.
>>
>> git-svn insists on using http://svn.apache.org/repos/asf as the base 
>> URL, which is basically correct, but doesn't work for this repo.
>>
>> Even editing .git/config and .git/svn/metadata (after git svn init, 
>> befire the first fetch) to change the url doesn't help. git svn still 
>> uses .../asf. Is this as intended? cc'ing the git svn author.
> 
> Can you try using the latest git-svn and also --no-follow-parent?
> 
> --no-follow-parent won't traverse copy history so it may be faster,
> but you'll get an incomplete history.
> 
> On the other hand, it looks like the asf repo is hopelessly slow
> for history retrieval.
> 
> I'm pulling the asf log (svn log -v http://svn.apache.org/repos/asf >
> tmp.log) but it's taking forever...  git-svn uses basically the same
> operation as 'svn log -v' and I've yet to see an SVN server take this
> long to show history.
> 
> I've gotten 70,000 revisions after over one hour with the above svn log
> command.  I'm going to restart this from a server with a better
> connection since I need to power off this one when I sleep.

Note that I'm not the OP, I'm just a git-svn user who got curious about 
that post. I think the asf repo shows exactly those mistakes which svn 
drives its users into (and which I've made also, on a much smaller 
scale, of course). As far as I understand, the synapse repo there is 
historical, and they need to convert to git in order to feed it into a 
CMS. I don't know if OP is still interested.

Michael