git svn clone a non-standard repository

git svn clone a non-standard repository

[John Locke]

Hi,

I'm trying to create a git repo of the Dojo Toolkit, which has a quite 
non-standard layout. There are 4 modules to the current Dojo project 
that I care about (and some others in the repo that I'm not interested 
in). The problem is, the trunk code is split out into 
<modulename>/trunk, while tags are in tags/<tagname>/<modulename> and 
branches are in branches/<branchname>/<modulename>. e.g:

moduleA/trunk <- contains trunk development of moduleA
moduleA/tags   <- empty
moduleA/branches <- empty
moduleB/trunk
moduleB/tags
moduleB/branches
moduleC/trunk
moduleC/tags
moduleC/branches
moduleD/trunk
moduleD/tags
moduleD/branches
moduleE/trunk <- I don't care about this one...
trunk/   <- contains ancient version, not actual trunk
tags/1.0.0/moduleA <- contains tagged version of moduleA
tags/1.0.0/moduleB <- contains tagged version of moduleB
tags/1.0.0/moduleC <- contains tagged version of moduleC
tags/1.0.0/moduleD <- contains tagged version of moduleD
tags/1.0.1/moduleA
tags/1.0.1/moduleB
...

So I'd like to set up a git repo that tracks this SVN repository, and 
allows me to see:
moduleA/
moduleB/
moduleC/
moduleD/
... in my checkout, whether I'm on trunk or a tag.

What's the best way to set this up?

I've started with "git svn clone http://path/to/svn -T moduleA/trunk -t 
tags -b branches", and it's been sucking down branches for a couple days 
now, still not done. Can I set up moduleB/moduleC/moduleD as additional 
remotes in this same repository, and end up with the desired result? Was 
thinking I would add additional svn sections to .git/config, and then 
git svn fetch -- will this work, or is there a better way?


Thanks,

-- 
John Locke
"Open Source Solutions for Small Business Problems"
published by Charles River Media, June 2004
http://www.freelock.com

Re: git svn clone a non-standard repository

[Michael J Gruber]

John Locke venit, vidit, dixit 24.06.2008 20:32:
> Hi,
> 
> I'm trying to create a git repo of the Dojo Toolkit, which has a quite 
> non-standard layout. There are 4 modules to the current Dojo project 
> that I care about (and some others in the repo that I'm not interested 
> in). The problem is, the trunk code is split out into 
> <modulename>/trunk, while tags are in tags/<tagname>/<modulename> and 
> branches are in branches/<branchname>/<modulename>. e.g:
> 
> moduleA/trunk <- contains trunk development of moduleA
> moduleA/tags   <- empty
> moduleA/branches <- empty
> moduleB/trunk
> moduleB/tags
> moduleB/branches
> moduleC/trunk
> moduleC/tags
> moduleC/branches
> moduleD/trunk
> moduleD/tags
> moduleD/branches

All of those module?/{tags,branches} are empty, right? I assume branches 
follow the pattern below for tags.

> moduleE/trunk <- I don't care about this one...
> trunk/   <- contains ancient version, not actual trunk
> tags/1.0.0/moduleA <- contains tagged version of moduleA
> tags/1.0.0/moduleB <- contains tagged version of moduleB
> tags/1.0.0/moduleC <- contains tagged version of moduleC
> tags/1.0.0/moduleD <- contains tagged version of moduleD
> tags/1.0.1/moduleA
> tags/1.0.1/moduleB
> ...
> 
> So I'd like to set up a git repo that tracks this SVN repository, and 
> allows me to see:
> moduleA/
> moduleB/
> moduleC/
> moduleD/
> ... in my checkout, whether I'm on trunk or a tag.

So you want one git repo, reflecting the modules simply by different 
subdirs? I'm afraid that raises similar problems like those I asked 
about in my (yet unanswered) post

http://permalink.gmane.org/gmane.comp.version-control.git/85861

> I've started with "git svn clone http://path/to/svn -T moduleA/trunk -t 
> tags -b branches", and it's been sucking down branches for a couple days 

This should give you all "modules" (as subsdirs) for all branches and 
tags already. svn branch "branchname" becomes git remote branch 
"branchname", svn tag "tagname" becomes git remote branch "tags/tagname".
Note that your new git branch trunk will contain the contents of 
"moduleA" only.

> now, still not done. Can I set up moduleB/moduleC/moduleD as additional 
> remotes in this same repository, and end up with the desired result? Was 
> thinking I would add additional svn sections to .git/config, and then 
> git svn fetch -- will this work, or is there a better way?

If I understand your layout (and "desires") correctly then trunk will be 
your only problem: you want to map "module?/trunk" to subdir "module?" 
of branch "trunk". This is more or less the problem I'm facing. Regular 
expressions (other than the ones for branches and tags) in git svn 
config would solve this.

Michael

Re: git svn clone a non-standard repository

[John Locke]

Hi, Michael,

Michael J Gruber wrote:
>>
>> moduleA/trunk <- contains trunk development of moduleA
>> moduleA/tags   <- empty
>> moduleA/branches <- empty
>> moduleB/trunk
>> moduleB/tags
>> moduleB/branches
>> moduleC/trunk
>> moduleC/tags
>> moduleC/branches
>> moduleD/trunk
>> moduleD/tags
>> moduleD/branches
>
> All of those module?/{tags,branches} are empty, right? I assume 
> branches follow the pattern below for tags.
Yup.

>
>> moduleE/trunk <- I don't care about this one...
>> trunk/   <- contains ancient version, not actual trunk
>> tags/1.0.0/moduleA <- contains tagged version of moduleA
>> tags/1.0.0/moduleB <- contains tagged version of moduleB
>> tags/1.0.0/moduleC <- contains tagged version of moduleC
>> tags/1.0.0/moduleD <- contains tagged version of moduleD
>> tags/1.0.1/moduleA
>> tags/1.0.1/moduleB
>> ...
>>
>> So I'd like to set up a git repo that tracks this SVN repository, and 
>> allows me to see:
>> moduleA/
>> moduleB/
>> moduleC/
>> moduleD/
>> ... in my checkout, whether I'm on trunk or a tag.
>
> So you want one git repo, reflecting the modules simply by different 
> subdirs? I'm afraid that raises similar problems like those I asked 
> about in my (yet unanswered) post
>
> http://permalink.gmane.org/gmane.comp.version-control.git/85861

I saw your post, and flagged it to see if anybody answered it, yes it 
does seem similar.
>
>> I've started with "git svn clone http://path/to/svn -T moduleA/trunk 
>> -t tags -b branches", and it's been sucking down branches for a 
>> couple days 
>
> This should give you all "modules" (as subsdirs) for all branches and 
> tags already. svn branch "branchname" becomes git remote branch 
> "branchname", svn tag "tagname" becomes git remote branch "tags/tagname".
> Note that your new git branch trunk will contain the contents of 
> "moduleA" only.

That's exactly what happened.
>
>> now, still not done. Can I set up moduleB/moduleC/moduleD as 
>> additional remotes in this same repository, and end up with the 
>> desired result? Was thinking I would add additional svn sections to 
>> .git/config, and then git svn fetch -- will this work, or is there a 
>> better way?
>
> If I understand your layout (and "desires") correctly then trunk will 
> be your only problem: you want to map "module?/trunk" to subdir 
> "module?" of branch "trunk". This is more or less the problem I'm 
> facing. Regular expressions (other than the ones for branches and 
> tags) in git svn config would solve this.
>

I found in the docs a section on subtrees, and with a bit of trickery, I 
think I'm getting what I want, though it's not simple to update. What 
I've done so far is change .git/config to:

[core]
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[svn-remote "svn"]
    url = http://svn.dojotoolkit.org/src
    branches = branches/*:refs/remotes/*
    tags = tags/*:refs/remotes/tags/*
[svn-remote "dojo"]
    url = http://svn.dojotoolkit.org/src
    fetch = dojo/trunk:refs/remotes/dojo-trunk
[svn-remote "dijit"]
    url = http://svn.dojotoolkit.org/src
    fetch = dijit/trunk:refs/remotes/dijit-trunk
[svn-remote "dojox"]
    url = http://svn.dojotoolkit.org/src
    fetch = dojox/trunk:refs/remotes/dojox-trunk
[svn-remote "util"]
    url = http://svn.dojotoolkit.org/src
    fetch = util/trunk:refs/remotes/util-trunk


... dojo, dijit, dojox, and util correspond to moduleA-D in my previous 
example. So I have "svn" for the branches, and separate SVN sections for 
the trunk of each module. Then I used git svn fetch 
[dojo|dijit|dojox|util] to suck down those as individual branches.

Now the tricky part: I checked out a new "work" branch, and deleted 
everything in the working copy. Then, 4 steps out of the howto, adjusted 
to pull the particular branch from the current repository (instead of an 
outside one):

git remote add -t dojo -f dojo ./
git merge -s ours --no-commit dojo-trunk
git read-tree --prefix=dojo/ -u dojo-trunk
git commit -m "merge dojo into subdirectory"

... repeat for the other modules, and I ended up with my "work" branch 
containing the full trunk for all 4 modules.

Now for tracking the original project. According to the howto, you use 
this syntax to pull down changes:

 git pull -s subtree Bproject master
... when I try git pull -s subtree ./ dojo, I get "Already up to date", 
even though I know there are new changes. Still need to mess around with 
git svn rebase, I guess, to get the updates working correctly.

Cheers,

-- 
John Locke
"Open Source Solutions for Small Business Problems"
published by Charles River Media, June 2004
http://www.freelock.com

Re: git svn clone a non-standard repository

[John Locke]

Sorry to respond to my own post, but this section looks wrong:

John Locke wrote:
>
>
> Now the tricky part: I checked out a new "work" branch, and deleted 
> everything in the working copy. Then, 4 steps out of the howto, 
> adjusted to pull the particular branch from the current repository 
> (instead of an outside one):
>
> git remote add -t dojo -f dojo ./
> git merge -s ours --no-commit dojo-trunk
> git read-tree --prefix=dojo/ -u dojo-trunk
> git commit -m "merge dojo into subdirectory"

... since I added the remotes as svn-remote sections directly in 
git-config, I skipped that first line.

I'm going to write this all up on my blog when I get it working correctly...

Cheers,

-- 
John Locke
"Open Source Solutions for Small Business Problems"
published by Charles River Media, June 2004
http://www.freelock.com

Re: git svn clone a non-standard repository

[John Locke]

Okay. I think I've closed the loop, and have this updating successfully, 
if not necessarily easily.

John Locke wrote:
> Sorry to respond to my own post, but this section looks wrong:
>
> John Locke wrote:
>>
>>
>> Now the tricky part: I checked out a new "work" branch, and deleted 
>> everything in the working copy. Then, 4 steps out of the howto, 
>> adjusted to pull the particular branch from the current repository 
>> (instead of an outside one):
>>
>> git remote add -t dojo -f dojo ./
>> git merge -s ours --no-commit dojo-trunk
>> git read-tree --prefix=dojo/ -u dojo-trunk
>> git commit -m "merge dojo into subdirectory"
>
> ... since I added the remotes as svn-remote sections directly in 
> git-config, I skipped that first line.
>
>
To update, we need a local branch for each submodule:

git checkout -b dojo dojo-trunk <- create local branch and switch to it 
(only create it the first time through)

git svn rebase dojo-trunk  <- do this to update each local branch from 
the remote svn repository version--must have corresponding branch 
checked out.

... now that we've merged the remote changes to a local branch, we can 
update our work branch from each module's branch:

git checkout work
git pull -s subtree ./ dojo
git pull -s subtree ./ dijit
....




-- 
John Locke
"Open Source Solutions for Small Business Problems"
published by Charles River Media, June 2004
http://www.freelock.com