git svn clone, a non-standard layout question

git svn clone, a non-standard layout question

[Christopher Marshall]

I would like to use git svn to clone an svn repo with a non-standard
branches layout roughly like this:
trunk/
tags/
branches/
   b1
   b2
   ...
   bdir/
      b3
      b4
      ...

That is, every directory under branches is a branch except bdir, and
every directory under bdir is a branch.

One thing I have attempted is this:
   git svn clone \
                      --trunk=trunk \
                      --branches=branches/bdir \
                      --branches=branches \
                      --tags=tags \
                      --prefix=svn/ $SVN_REPO \
                      git.svn

That properly creates the remote tracking branches b1,b2,b3,b4 but
also creates the remote tracking branch bdir, which I am trying to
exclude.  If I were to settle for this, the bdir branch would have
enormous trees committed to it (when I ran against the real svn repo I
am targetting).

I get can exactly the branch mapping I want by editing .git/config like this:
[svn-remote "svn"]
        url = file:///home/chris/programs/svn/repo
        fetch = trunk:refs/remotes/svn/trunk
        tags = tags/*:refs/remotes/svn/tags/*
        branches = branches/{b1,b2}:refs/remotes/svn/*
        branches = branches/bdir/{b3,b4}:refs/remotes/svn/*

but then I would have to manually add branches before every git svn
fetch, or risk not importing new branches that other developers have
created since I last fetched.

Chris Marshall

p.s.  Here is the bash script I am using to experiment with this type
of svn layout.  It creates a svn repo with the structure described
above, then applies my second (non-wildcard) solution.  The "bash"
line right before the cleanup lines at the end is to allow you to look
around before everything disappears.

#!/bin/bash
# file_summary: svn non-standard layout.  this works, and excludes
bdir, but by not using wild cards it requires that branches be
added manually before a fetch.

export SVN_EDITOR=vi
CWD=$(pwd)
export URL=file://$(pwd)/repo

svnadmin create repo

# create top level directories.
svn checkout $URL proj
cd proj
svn mkdir branches tags trunk
svn commit -m "created top level dirs"
svn mkdir branches/bdir
svn commit -m "created non-standard branches dir"
cd $CWD

# trunk
svn checkout ${URL}/trunk proj-t
cd proj-t
echo -e "1\n2\n3" > f1
svn add f1
svn commit -m "added f1: 1,2,3"
cd $CWD

# create branches b1,b2,b3,b4
svn copy ${URL}/trunk ${URL}/branches/b1 -m "created branch b1"
svn copy ${URL}/trunk ${URL}/branches/b2 -m "created branch b2"
svn copy ${URL}/trunk ${URL}/branches/bdir/b3 -m "created branch b3"
svn copy ${URL}/trunk ${URL}/branches/bdir/b4 -m "created branch b4"
cd $CWD

# create a b1 commit
svn checkout ${URL}/branches/b1 proj-b1
cd proj-b1
echo -e "b1" >> f1; svn commit -m "b1 line"
cd $CWD

# create a b2 commit
svn checkout ${URL}/branches/b2 proj-b2
cd proj-b2
echo -e "b2" >> f1; svn commit -m "b2 line"
cd $CWD

# create a b3 commit
svn checkout ${URL}/branches/bdir/b3 proj-b3
cd proj-b3
echo -e "b3" >> f1; svn commit -m "b3 line"
cd $CWD

# create a b4 commit
svn checkout ${URL}/branches/bdir/b4 proj-b4
cd proj-b4
echo -e "b4" >> f1; svn commit -m "b4 line"
cd $CWD

git svn clone --trunk=trunk --branches=branches --tags=tags
--prefix=svn/ $URL -r 1:1 git.svn
cd git.svn
grep -v branches .git/config > .git/config2
mv .git/config2 .git/config
echo "branches = branches/{b1,b2}:refs/remotes/svn/*" >> .git/config
echo "branches = branches/bdir/{b3,b4}:refs/remotes/svn/*" >> .git/config
rm .git/svn/.metadata
git svn fetch

bash
cd $CWD
rm -rf repo proj proj-{t,b{1,2,3,4}} git.svn

Re: git svn clone, a non-standard layout question

[Peter Baumann]

On Sun, Aug 12, 2012 at 11:23:16PM -0400, Christopher Marshall wrote:
> I would like to use git svn to clone an svn repo with a non-standard
> branches layout roughly like this:
> trunk/
> tags/
> branches/
>    b1
>    b2
>    ...
>    bdir/
>       b3
>       b4
>       ...
> 
> That is, every directory under branches is a branch except bdir, and
> every directory under bdir is a branch.
> 
> One thing I have attempted is this:
>    git svn clone \
>                       --trunk=trunk \
>                       --branches=branches/bdir \
>                       --branches=branches \
>                       --tags=tags \
>                       --prefix=svn/ $SVN_REPO \
>                       git.svn
> 
> That properly creates the remote tracking branches b1,b2,b3,b4 but
> also creates the remote tracking branch bdir, which I am trying to
> exclude.  If I were to settle for this, the bdir branch would have
> enormous trees committed to it (when I ran against the real svn repo I
> am targetting).
> 
> I get can exactly the branch mapping I want by editing .git/config like this:
> [svn-remote "svn"]
>         url = file:///home/chris/programs/svn/repo
>         fetch = trunk:refs/remotes/svn/trunk
>         tags = tags/*:refs/remotes/svn/tags/*
>         branches = branches/{b1,b2}:refs/remotes/svn/*
>         branches = branches/bdir/{b3,b4}:refs/remotes/svn/*
> 
> but then I would have to manually add branches before every git svn
> fetch, or risk not importing new branches that other developers have
> created since I last fetched.
> 

I had a similar problem, but I solved it using "ignore-paths" and "ignore-refs".
If I remember correctly, you need to set both to ignore bdir directly without
ignoring b3, b4,...

For ignore-refs, pls see cdb51a13c3cf4830d499d1138160eacdd2b8aa46, as it is currently
undocumented.

So I would try experimenting with the following settings:

[svn-remote "svn"]
	url = file:///home/chris/programs/svn/repo
	fetch = trunk:refs/remotes/svn/trunk
	tags = tags/*:refs/remotes/svn/tags/*
	branches = branches/{b1,b2}:refs/remotes/svn/*
	branches = branches/bdir/{b3,b4}:refs/remotes/svn/*

	# Operates on the imported git branches
	ignore-refs  = ^refs/remotes/bdir$

	# Operates on the SVN branches; you might try it first without this statement
	ignore-paths = ^branches/bdir$

Re: git svn clone, a non-standard layout question

[Christopher Marshall]

>
> I had a similar problem, but I solved it using "ignore-paths" and "ignore-refs".
> If I remember correctly, you need to set both to ignore bdir directly without
> ignoring b3, b4,...
>
> For ignore-refs, pls see cdb51a13c3cf4830d499d1138160eacdd2b8aa46, as it is currently
> undocumented.
>
> So I would try experimenting with the following settings:
>
> [svn-remote "svn"]
>         url = file:///home/chris/programs/svn/repo
>         fetch = trunk:refs/remotes/svn/trunk
>         tags = tags/*:refs/remotes/svn/tags/*
>         branches = branches/{b1,b2}:refs/remotes/svn/*
>         branches = branches/bdir/{b3,b4}:refs/remotes/svn/*
>
>         # Operates on the imported git branches
>         ignore-refs  = ^refs/remotes/bdir$
>
>         # Operates on the SVN branches; you might try it first without this statement
>         ignore-paths = ^branches/bdir$
> --

Peter:

Thanks for the advice.  I tried this:

[svn-remote "svn"]
        url = file:///home/chris/programs/svn/repo
        fetch = trunk:refs/remotes/svn/trunk
        branches = branches/*:refs/remotes/svn/*
        tags = tags/*:refs/remotes/svn/tags/*
        branches = branches/bdir/*:refs/remotes/svn/bdir2/*
        ignore-paths  = ^branches/bdir$
        ignore-refs  = ^refs/remotes/bdir$

It doesn't seem to change anything.

Chris

Re: git svn clone, a non-standard layout question

[Peter Baumann]

On Mon, Aug 13, 2012 at 09:29:53AM -0400, Christopher Marshall wrote:
> >
> > I had a similar problem, but I solved it using "ignore-paths" and "ignore-refs".
> > If I remember correctly, you need to set both to ignore bdir directly without
> > ignoring b3, b4,...
> >
> > For ignore-refs, pls see cdb51a13c3cf4830d499d1138160eacdd2b8aa46, as it is currently
> > undocumented.
> >
> > So I would try experimenting with the following settings:
> >
> > [svn-remote "svn"]
> >         url = file:///home/chris/programs/svn/repo
> >         fetch = trunk:refs/remotes/svn/trunk
> >         tags = tags/*:refs/remotes/svn/tags/*
> >         branches = branches/{b1,b2}:refs/remotes/svn/*
> >         branches = branches/bdir/{b3,b4}:refs/remotes/svn/*
> >
> >         # Operates on the imported git branches
> >         ignore-refs  = ^refs/remotes/bdir$
> >
> >         # Operates on the SVN branches; you might try it first without this statement
> >         ignore-paths = ^branches/bdir$
> > --
> 
> Peter:
> 
> Thanks for the advice.  I tried this:
> 
> [svn-remote "svn"]
>         url = file:///home/chris/programs/svn/repo
>         fetch = trunk:refs/remotes/svn/trunk
>         branches = branches/*:refs/remotes/svn/*
>         tags = tags/*:refs/remotes/svn/tags/*
>         branches = branches/bdir/*:refs/remotes/svn/bdir2/*
>         ignore-paths  = ^branches/bdir$
>         ignore-refs  = ^refs/remotes/bdir$
> 
> It doesn't seem to change anything.
> 

You need a git version new enough to include cdb51a13c3cf4830d499d1138160eacdd2b8aa46, otherwise
it won't have any effect and will be silently ignored.

> Chris
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: git svn clone, a non-standard layout question

[Christopher Marshall]

>> [svn-remote "svn"]
>>         url = file:///home/chris/programs/svn/repo
>>         fetch = trunk:refs/remotes/svn/trunk
>>         branches = branches/*:refs/remotes/svn/*
>>         tags = tags/*:refs/remotes/svn/tags/*
>>         branches = branches/bdir/*:refs/remotes/svn/bdir2/*
>>         ignore-paths  = ^branches/bdir$
>>         ignore-refs  = ^refs/remotes/bdir$
>>
>> It doesn't seem to change anything.
>>
>
> You need a git version new enough to include cdb51a13c3cf4830d499d1138160eacdd2b8aa46, otherwise
> it won't have any effect and will be silently ignored.
>
>> Chris
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>

Peter:

I see what you mean.  I will download a version of git that includes
that commit and try again.

Thanks for all your help,

Chris