git-svn set-tree bug

git-svn set-tree bug

[Joakim Tjernlund]

trying to do git-svn set-tree remotes/trunk..svn
in my new git-svn repo I get:
config --get svn-remote.svn.fetch :refs/remotes/git-svn$: command returned error: 1

git version 1.5.2.1

Re: git-svn set-tree bug

[Eric Wong]

Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> trying to do git-svn set-tree remotes/trunk..svn
> in my new git-svn repo I get:
> config --get svn-remote.svn.fetch :refs/remotes/git-svn$: command returned error: 1

You need to specify "-i trunk" in the command-line

git-svn set-tree -i trunk remotes/trunk..svn

-- 
Eric Wong

Re: git-svn set-tree bug

[Joakim Tjernlund]

On Sat, 2007-06-09 at 18:47 -0700, Eric Wong wrote:
> Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> > trying to do git-svn set-tree remotes/trunk..svn
> > in my new git-svn repo I get:
> > config --get svn-remote.svn.fetch :refs/remotes/git-svn$: command returned error: 1
> 
> You need to specify "-i trunk" in the command-line
> 
> git-svn set-tree -i trunk remotes/trunk..svn
> 

Thanks

I have found a bug or two. Run this script and
see what happens ant the end.
rm -rf mygitsvn
rm -rf mysvnrepo
rm -rf mysvnwork
mkdir mysvnrepo
cd mysvnrepo
svnadmin create .
cd ..
svn checkout file:///$PWD/mysvnrepo mysvnwork
mkdir -p mysvnwork/trunk
cd mysvnwork/
cat << EOF  > trunk/README
#
# (C) Copyright 2000 - 2005
# Wolfgang Denk, DENX Software Engineering, wd@denx.de.
#
# See file CREDITS for list of people who contributed to this
# project.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston,
# MA 02111-1307 USA
#

EOF

svn add trunk
svn ci -m "first commit" trunk
cd ..
git-svn clone  file:///$PWD/mysvnrepo -t tags -T trunk -b branches
mygitsvn
cd mygitsvn

git checkout --track -b svn remotes/trunk
git checkout -b merge
echo new file > new_file
git add new_file
git commit -a -m "New file"

echo hello >> README
git commit -a  -m "hello"

echo add some stuff >> new_file
git commit -a -m "add some stuff"

git checkout svn
mv -f README tmp
echo friend > README
cat tmp >> README
git commit -a -m "friend"
git pull . merge
git svn dcommit  # this fails 
git svn rebase  
# this fails too, mismerges the last commit and remove the merge commit

Re: git-svn set-tree bug

[Joakim Tjernlund]

On Sun, 2007-06-10 at 19:21 +0200, Joakim Tjernlund wrote:
> On Sat, 2007-06-09 at 18:47 -0700, Eric Wong wrote:
> > Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> > > trying to do git-svn set-tree remotes/trunk..svn
> > > in my new git-svn repo I get:
> > > config --get svn-remote.svn.fetch :refs/remotes/git-svn$: command returned error: 1
> > 
> > You need to specify "-i trunk" in the command-line
> > 
> > git-svn set-tree -i trunk remotes/trunk..svn
> > 
> 
> Thanks

[SNIP script]

forgot:
 git version 1.5.2.1
 subversion version 1.4.3

 Jocke

Re: git-svn set-tree bug

[Eric Wong]

Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> On Sat, 2007-06-09 at 18:47 -0700, Eric Wong wrote:
> > Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> > > trying to do git-svn set-tree remotes/trunk..svn
> > > in my new git-svn repo I get:
> > > config --get svn-remote.svn.fetch :refs/remotes/git-svn$: command returned error: 1
> > 
> > You need to specify "-i trunk" in the command-line
> > 
> > git-svn set-tree -i trunk remotes/trunk..svn
> > 
> 
> Thanks
> 
> I have found a bug or two. Run this script and
> see what happens ant the end.

<snip>

> git pull . merge

This is a non-fast-forward merge, giving you non-linear history.  git
understands non-linear history without problems, but svn does not.

> git svn dcommit  # this fails 

If you have non-linear history, don't use dcommit, use set-tree.  Linear
history is cleaner and easier to manage, which is why I recommend
format-patch/am/dcommit/rebase, and avoid using pull/merge unless it's
fast-forward.

-- 
Eric Wong

RE: git-svn set-tree bug

[Joakim Tjernlund]

 

> -----Original Message-----
> From: Eric Wong [mailto:normalperson@yhbt.net] 
> Sent: den 10 juni 2007 23:33
> To: Joakim Tjernlund
> Cc: git
> Subject: Re: git-svn set-tree bug
> 
> Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> > On Sat, 2007-06-09 at 18:47 -0700, Eric Wong wrote:
> > > Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> > > > trying to do git-svn set-tree remotes/trunk..svn
> > > > in my new git-svn repo I get:
> > > > config --get svn-remote.svn.fetch 
> :refs/remotes/git-svn$: command returned error: 1
> > > 
> > > You need to specify "-i trunk" in the command-line
> > > 
> > > git-svn set-tree -i trunk remotes/trunk..svn
> > > 
> > 
> > Thanks
> > 
> > I have found a bug or two. Run this script and
> > see what happens ant the end.
> 
> <snip>
> 
> > git pull . merge
> 
> This is a non-fast-forward merge, giving you non-linear history.  git
> understands non-linear history without problems, but svn does not.
> 
> > git svn dcommit  # this fails 
> 
> If you have non-linear history, don't use dcommit, use 
> set-tree.  Linear
> history is cleaner and easier to manage, which is why I recommend
> format-patch/am/dcommit/rebase, and avoid using pull/merge unless it's
> fast-forward.

I see, I figured git-svn could work around that.
So I should do a git svn set-tree -i trunk remotes/trunk..svn
and then git svn rebase? That makes the git history hard to
follow.
hmm, is it wise to mix set-tree and dcommit in one repo?
Is there a way to tell set-tree to commit the whole "merge" branch
as one svn commit?
If I merge the latest kernel into my tree there will
be a lot of commits that I don't want in svn.

 Jocke


   Jocke

Re: git-svn set-tree bug

[Steven Grimm]

Joakim Tjernlund wrote:
> Is there a way to tell set-tree to commit the whole "merge" branch
> as one svn commit?
> If I merge the latest kernel into my tree there will
> be a lot of commits that I don't want in svn.
>   

You want a "squash" merge. Something like this:

git checkout -b tempbranch origin/svn-branch-to-commit-merge-to
git merge --squash branch-with-commits-you-want-to-merge
git commit
git svn dcommit

The "merge" command will merge in the changes but will not commit 
anything; when you do the explicit "commit" command afterwards, you get 
the contents of the merge but from git's point of view it's just a 
regular commit so git-svn doesn't get confused.

After you do git svn dcommit, you may want to edit .git/info/grafts to 
tell git after the fact that this commit was a merge. It won't hurt 
git-svn at that point and it will mean you can do another merge later 
without git getting confused about what has already been merged.

Take a look at the script I posted a while back, which does something 
similar:

http://www.spinics.net/lists/git/msg29119.html

-Steve

RE: git-svn set-tree bug

[Joakim Tjernlund]

 

> -----Original Message-----
> From: Steven Grimm [mailto:koreth@midwinter.com] 
> Sent: den 11 juni 2007 01:37
> To: Joakim Tjernlund
> Cc: 'Eric Wong'; 'git'
> Subject: Re: git-svn set-tree bug
> 
> Joakim Tjernlund wrote:
> > Is there a way to tell set-tree to commit the whole "merge" branch
> > as one svn commit?
> > If I merge the latest kernel into my tree there will
> > be a lot of commits that I don't want in svn.
> >   
> 
> You want a "squash" merge. Something like this:
> 
> git checkout -b tempbranch origin/svn-branch-to-commit-merge-to
> git merge --squash branch-with-commits-you-want-to-merge
> git commit
> git svn dcommit
> 
> The "merge" command will merge in the changes but will not commit 
> anything; when you do the explicit "commit" command 
> afterwards, you get 
> the contents of the merge but from git's point of view it's just a 
> regular commit so git-svn doesn't get confused.
> 
> After you do git svn dcommit, you may want to edit 
> .git/info/grafts to 
> tell git after the fact that this commit was a merge. It won't hurt 
> git-svn at that point and it will mean you can do another merge later 
> without git getting confused about what has already been merged.
> 
> Take a look at the script I posted a while back, which does something 
> similar:
> 
> http://www.spinics.net/lists/git/msg29119.html
> 

Hi Steven

That looks promising, especially Junos comment about making git-svn
able to deal with merges. Eric, do you feel this is doable?

 Jocke 

Re: git-svn set-tree bug

[Eric Wong]

Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> > -----Original Message-----
> > From: Steven Grimm [mailto:koreth@midwinter.com] 
> > Sent: den 11 juni 2007 01:37
> > To: Joakim Tjernlund
> > Cc: 'Eric Wong'; 'git'
> > Subject: Re: git-svn set-tree bug
> > 
> > Joakim Tjernlund wrote:
> > > Is there a way to tell set-tree to commit the whole "merge" branch
> > > as one svn commit?
> > > If I merge the latest kernel into my tree there will
> > > be a lot of commits that I don't want in svn.
> > >   
> > 
> > You want a "squash" merge. Something like this:
> > 
> > git checkout -b tempbranch origin/svn-branch-to-commit-merge-to
> > git merge --squash branch-with-commits-you-want-to-merge
> > git commit
> > git svn dcommit
> > 
> > The "merge" command will merge in the changes but will not commit 
> > anything; when you do the explicit "commit" command 
> > afterwards, you get 
> > the contents of the merge but from git's point of view it's just a 
> > regular commit so git-svn doesn't get confused.
> > 
> > After you do git svn dcommit, you may want to edit 
> > .git/info/grafts to 
> > tell git after the fact that this commit was a merge. It won't hurt 
> > git-svn at that point and it will mean you can do another merge later 
> > without git getting confused about what has already been merged.
> > 
> > Take a look at the script I posted a while back, which does something 
> > similar:
> > 
> > http://www.spinics.net/lists/git/msg29119.html

I must have missed this message the first time around.

> Hi Steven
> 
> That looks promising, especially Junos comment about making git-svn
> able to deal with merges. Eric, do you feel this is doable?

Doable?  Yes.  However, I think using grafts is quite hackish and
unreliable[1].  I'd rather just have users using set-tree if
they want to deal with non-linear history in the first place.

I'd personally avoid any sort of non-linear history when interacting
with SVN repositories, however.

[1] - as far as I know, graft files have no verification/protection
      against corruption.  They don't get cloned, either.

-- 
Eric Wong

Re: git-svn set-tree bug

[Junio C Hamano]

Eric Wong <normalperson@yhbt.net> writes:

> Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
>> > -----Original Message-----
>> > From: Steven Grimm [mailto:koreth@midwinter.com] 
>> > Sent: den 11 juni 2007 01:37
>> > To: Joakim Tjernlund
>> > Cc: 'Eric Wong'; 'git'
>> > Subject: Re: git-svn set-tree bug
>> > 
>> > Joakim Tjernlund wrote:
>> > > Is there a way to tell set-tree to commit the whole "merge" branch
>> > > as one svn commit?
>> > > If I merge the latest kernel into my tree there will
>> > > be a lot of commits that I don't want in svn.
>> > >   
>> > 
>> > You want a "squash" merge. Something like this:
>> > 
>> > git checkout -b tempbranch origin/svn-branch-to-commit-merge-to
>> > git merge --squash branch-with-commits-you-want-to-merge
>> > git commit
>> > git svn dcommit
>> > 
>> > The "merge" command will merge in the changes but will not commit 
>> > anything; when you do the explicit "commit" command 
>> > afterwards, you get 
>> > the contents of the merge but from git's point of view it's just a 
>> > regular commit so git-svn doesn't get confused.
>> > 
>> > After you do git svn dcommit, you may want to edit 
>> > .git/info/grafts to 
>> > tell git after the fact that this commit was a merge. It won't hurt 
>> > git-svn at that point and it will mean you can do another merge later 
>> > without git getting confused about what has already been merged.
>> > 
>> > Take a look at the script I posted a while back, which does something 
>> > similar:
>> > 
>> > http://www.spinics.net/lists/git/msg29119.html
>
> I must have missed this message the first time around.
>
>> Hi Steven
>> 
>> That looks promising, especially Junos comment about making git-svn
>> able to deal with merges. Eric, do you feel this is doable?
>
> Doable?  Yes.  However, I think using grafts is quite hackish and
> unreliable[1].  I'd rather just have users using set-tree if
> they want to deal with non-linear history in the first place.
>
> I'd personally avoid any sort of non-linear history when interacting
> with SVN repositories, however.

I've been wondering if you can do a moral equilvalent of the
graft trick but without using graft inside dcommit.  Perform a
merge --squash of the other branch (call the tip commit $B),
then dcommit on the git side as usual, and call it commit $C.
Steven's procedure would do a graft trick here, but instead of
doing that, rewrite $C to have the two parents.  Using the tree
object of $C, create a new git commit $D that is a merge between
the parent of $C (i.e. $C^) and the squashed branch tip $B.
Replace the tip of the current branch (which is $C) with $D.
Finally, replace the mapping between svn commit and git side
recorded in the revdb (which currently says $C on the git side
corresponds to the HEAD of SVN side) with this new commit $D.

Wouldn't that let the git side know what was merged into the
branch, so that later merges on the git side would go smoothly?

Or am I grossly misunderstanding how dcommit, tracking of svn vs
git commit mappings and the graft trick work?

Re: git-svn set-tree bug

[Eric Wong]

Junio C Hamano <junkio@cox.net> wrote:
> Eric Wong <normalperson@yhbt.net> writes:
> 
> > Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> >> > -----Original Message-----
> >> > From: Steven Grimm [mailto:koreth@midwinter.com] 
> >> > Sent: den 11 juni 2007 01:37
> >> > To: Joakim Tjernlund
> >> > Cc: 'Eric Wong'; 'git'
> >> > Subject: Re: git-svn set-tree bug
> >> > 
> >> > Joakim Tjernlund wrote:
> >> > > Is there a way to tell set-tree to commit the whole "merge" branch
> >> > > as one svn commit?
> >> > > If I merge the latest kernel into my tree there will
> >> > > be a lot of commits that I don't want in svn.
> >> > >   
> >> > 
> >> > You want a "squash" merge. Something like this:
> >> > 
> >> > git checkout -b tempbranch origin/svn-branch-to-commit-merge-to
> >> > git merge --squash branch-with-commits-you-want-to-merge
> >> > git commit
> >> > git svn dcommit
> >> > 
> >> > The "merge" command will merge in the changes but will not commit 
> >> > anything; when you do the explicit "commit" command 
> >> > afterwards, you get 
> >> > the contents of the merge but from git's point of view it's just a 
> >> > regular commit so git-svn doesn't get confused.
> >> > 
> >> > After you do git svn dcommit, you may want to edit 
> >> > .git/info/grafts to 
> >> > tell git after the fact that this commit was a merge. It won't hurt 
> >> > git-svn at that point and it will mean you can do another merge later 
> >> > without git getting confused about what has already been merged.
> >> > 
> >> > Take a look at the script I posted a while back, which does something 
> >> > similar:
> >> > 
> >> > http://www.spinics.net/lists/git/msg29119.html
> >
> > I must have missed this message the first time around.
> >
> >> Hi Steven
> >> 
> >> That looks promising, especially Junos comment about making git-svn
> >> able to deal with merges. Eric, do you feel this is doable?
> >
> > Doable?  Yes.  However, I think using grafts is quite hackish and
> > unreliable[1].  I'd rather just have users using set-tree if
> > they want to deal with non-linear history in the first place.
> >
> > I'd personally avoid any sort of non-linear history when interacting
> > with SVN repositories, however.
> 
> I've been wondering if you can do a moral equilvalent of the
> graft trick but without using graft inside dcommit.  Perform a
> merge --squash of the other branch (call the tip commit $B),
> then dcommit on the git side as usual, and call it commit $C.
> Steven's procedure would do a graft trick here, but instead of
> doing that, rewrite $C to have the two parents.  Using the tree
> object of $C, create a new git commit $D that is a merge between
> the parent of $C (i.e. $C^) and the squashed branch tip $B.
> Replace the tip of the current branch (which is $C) with $D.
> Finally, replace the mapping between svn commit and git side
> recorded in the revdb (which currently says $C on the git side
> corresponds to the HEAD of SVN side) with this new commit $D.
> 
> Wouldn't that let the git side know what was merged into the
> branch, so that later merges on the git side would go smoothly?
> 
> Or am I grossly misunderstanding how dcommit, tracking of svn vs
> git commit mappings and the graft trick work?

Ok, it took me a few reads, but I think that'll work...

If dcommit detects a merge commit when doing rev-list When looking at
commit objects, is it safe to assume that the first parent is always the
"mainline" and that parents after it are the ones to merge from?

So if I saw:

commit $X
parent $A
parent $B

I'd basically do:
  reset --hard $A
  merge --squash $B

And resulting in $C which would have the same tree as $X,
then, when dcommit-ting, $D would be created with two parents:
  $D~1 (svn), $B (git), but not $A

Rewritten history:
  $A         =>  $D~1
  $X         =>  $D (HEAD revision in SVN)

$X and $A are now discarded and gc-able.


Of course, since I already have the result of "merge --squash $B" in $X,
I could just rewrite $X with a single parent (call it $X'), dcommit, and
then give $D ($D~1 and $B) as parents.  Avoiding the nastiness of
set-tree

-- 
Eric Wong

Re: git-svn set-tree bug

[Junio C Hamano]

Eric Wong <normalperson@yhbt.net> writes:

> If dcommit detects a merge commit when doing rev-list When looking at
> commit objects, is it safe to assume that the first parent is always the
> "mainline" and that parents after it are the ones to merge from?
>
> So if I saw:
>
> commit $X
> parent $A
> parent $B
>
> I'd basically do:
>   reset --hard $A
>   merge --squash $B
>
> And resulting in $C which would have the same tree as $X,
> then, when dcommit-ting, $D would be created with two parents:
>   $D~1 (svn), $B (git), but not $A

I am not sure what you mean by "mainline", but I assume that you
mean "SVN is the main and we are tracking it while taking
advantage of more efficient and merge-capable git in guerrilla
fashion".  Because the tip of the current branch is what the
user is pushing back to SVN via dcommit, I would say it is safe
to assume that the first parent of such a merge is the line that
corresponds to the SVN branch you are keeping track.

Re: git-svn set-tree bug

[Eric Wong]

Junio C Hamano <gitster@pobox.com> wrote:
> Eric Wong <normalperson@yhbt.net> writes:
> 
> > If dcommit detects a merge commit when doing rev-list When looking at
> > commit objects, is it safe to assume that the first parent is always the
> > "mainline" and that parents after it are the ones to merge from?
> >
> > So if I saw:
> >
> > commit $X
> > parent $A
> > parent $B
> >
> > I'd basically do:
> >   reset --hard $A
> >   merge --squash $B
> >
> > And resulting in $C which would have the same tree as $X,
> > then, when dcommit-ting, $D would be created with two parents:
> >   $D~1 (svn), $B (git), but not $A
> 
> I am not sure what you mean by "mainline", but I assume that you
> mean "SVN is the main and we are tracking it while taking
> advantage of more efficient and merge-capable git in guerrilla
> fashion".  Because the tip of the current branch is what the
> user is pushing back to SVN via dcommit, I would say it is safe
> to assume that the first parent of such a merge is the line that
> corresponds to the SVN branch you are keeping track.

Yes, "mainline" meaning the history that would be committed to SVN if
history were linear.

I've gotten the following patch working for Joakim's second test script
(with dcommit before merge).  However, without the dcommit before merge
in the first test script, git-svn has trouble figuring out which history
to follow.  It'll take more work to figure out what to do in this
situation, and how to deal with more complex history...

Subject: git-svn: Allow dcommit to handle certain single-parent merge commits

This only works if a merge is the first commit to be committed
in a chain of commits.
---
 git-svn.perl |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index 0ae8d70..6b3e021 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -403,6 +403,9 @@ sub cmd_dcommit {
 			                svn_path => '');
 			if (!SVN::Git::Editor->new(\%ed_opts)->apply_diff) {
 				print "No changes\n$d~1 == $d\n";
+			} elsif (my $merge_parent = verify_ref("$d^2")) {
+				$gs->{inject_parents}->{$last_rev} =
+				                                 $merge_parent;
 			}
 		}
 	}
-- 
Eric Wong

Re: git-svn set-tree bug

[Joakim Tjernlund]

On Tue, 2007-06-12 at 01:39 -0700, Eric Wong wrote:
> Junio C Hamano <gitster@pobox.com> wrote:
> > Eric Wong <normalperson@yhbt.net> writes:
> > 
> > > If dcommit detects a merge commit when doing rev-list When looking at
> > > commit objects, is it safe to assume that the first parent is always the
> > > "mainline" and that parents after it are the ones to merge from?
> > >
> > > So if I saw:
> > >
> > > commit $X
> > > parent $A
> > > parent $B
> > >
> > > I'd basically do:
> > >   reset --hard $A
> > >   merge --squash $B
> > >
> > > And resulting in $C which would have the same tree as $X,
> > > then, when dcommit-ting, $D would be created with two parents:
> > >   $D~1 (svn), $B (git), but not $A
> > 
> > I am not sure what you mean by "mainline", but I assume that you
> > mean "SVN is the main and we are tracking it while taking
> > advantage of more efficient and merge-capable git in guerrilla
> > fashion".  Because the tip of the current branch is what the
> > user is pushing back to SVN via dcommit, I would say it is safe
> > to assume that the first parent of such a merge is the line that
> > corresponds to the SVN branch you are keeping track.
> 
> Yes, "mainline" meaning the history that would be committed to SVN if
> history were linear.
> 
> I've gotten the following patch working for Joakim's second test script
> (with dcommit before merge).  However, without the dcommit before merge
> in the first test script, git-svn has trouble figuring out which history
> to follow.  It'll take more work to figure out what to do in this
> situation, and how to deal with more complex history...
> 
> Subject: git-svn: Allow dcommit to handle certain single-parent merge commits
> 
> This only works if a merge is the first commit to be committed
> in a chain of commits.
[SNIP patch]

Nice!, now I get to keep the merge between the "svn" and the "merge" branch. The parents are swapped though:
before last dcommit:
  Parent: b31cef1d3c6655441854ea8649359f0fc27f3e87 (friend)
  Parent: ed95b698c2e3336d387fed3763b213b3b90ebf4e (add some stuff)
  Branch: svn
  Follows: 
  Precedes: 

    Merge branch 'merge' into svn

after dcommit:

Parent: ed95b698c2e3336d387fed3763b213b3b90ebf4e (add some stuff)
Parent: b31cef1d3c6655441854ea8649359f0fc27f3e87 (friend)
Branches: svn, remotes/trunk
Follows: 
Precedes: 

    Merge branch 'merge' into svn
    
    
    git-svn-id: file:////usr/local/src/tst-git-svn/mysvnrepo/trunk@3 1585b9b0-b13 ....


Will this also work for merging stuff from latest u-boot?

I am doing dev. on my own u-boot branch and from time to time I want
to merge in the latest from WD tree, then dcommit that merge. Later
I want repeat that cycle.

I have a SVN repo with my changes in it and I have grafted the beginning of
that tree into a clone of WDs tree.

 Jocke

Re: git-svn set-tree bug

[Steven Grimm]

Eric Wong wrote:
> Yes, "mainline" meaning the history that would be committed to SVN if
> history were linear.
>   

I think the first parent is always the right one to follow. The only 
time you won't hit a git-svn revision is if the user is trying to commit 
a branch that is not originally derived from a git-svn branch, and IMO 
that's something that git-svn is perfectly justified in refusing to do. 
Also, the default "git log" output will follow the first parent; users 
who run that are going to have a natural expectation that the subsequent 
commits will be merged into the most recent git-svn revision as shown by 
that tool.

> This only works if a merge is the first commit to be committed
> in a chain of commits.
>   

As for the more complex case of a chain of commits with a merge in the 
middle, IMO walking along the chain of first parents until you hit a 
git-svn revision, then proceeding forward in time from there rewriting 
parents as you've described, is always going to be the right thing to 
do. Or at least, all the use cases I can think of seem to be correctly 
covered by that approach. Can someone come up with counterexamples?

This is great -- I'm looking forward to ditching my hackish merge script!

At the risk of getting ahead of myself, here's one more thought: in the 
case where a merge's parents are all git-svn revisions -- that is, where 
the user is using git to merge svn branches -- I wonder if it makes 
sense to optionally record that merge somehow in the commit comment on 
the svn side. I think that could be made relatively human-readable so as 
not to be too obnoxious for people browsing the svn history. That way 
someone pulling down a fresh git-svn clone of the svn repo could get a 
nice clean history with the merges represented properly in the git 
revision history.

That's justifiable in another way too: the autogenerated comments on git 
merge commits won't really make much sense over on the svn side, where 
merges are thought of in terms of revision ranges. So replacing the 
git-specific merge message with an svn-specific one doesn't seem 
unreasonable to me. (Again, optionally.) And in cases where the user has 
supplied his own merge comment on the git side, annotating it with the 
additional git-svn metadata seems reasonable to me. We are already fine 
with the git-side comments having a line of git-svn metadata, after all.

Most of the svn-side merge comments in my company's repo look like 
either "svn merge -r12345:67890 mybranch" (where the developer wants to 
make the merge's inputs very explicit to avoid any confusion) or "Merge 
revisions 12345 through 67890 from mybranch", occasionally surrounded by 
some explanatory text. If git-svn replaced the canned git merge message 
with a canned message like one of those, people wouldn't be able to tell 
I'd used git-svn instead of svn to do the merge.

-Steve

[PATCH] git-svn: allow dcommit to retain local merge information

[Eric Wong]

dcommit will still rewrite the HEAD commit and the history of the first
parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.

However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
with dcommit.  Commits written to SVN will still not have any merge
information besides anything in the commit message.

Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
for explanations, feedback, examples and test case.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
---

 This is a better patch that replaces the previous one.

 Junio:
   This one is a big change and should probably sit in pu or next
   for a bit.  Double-checking the logic in linearize_history()
   would be greatly appreciated, too.
   
   I don't think there are any regressions for the
   already-linear-history case besides slightly reduced performance for
   new calls to cat-file.

 Joakim/Steven:
   Any further testing and test cases would be appreciated.  Be very
   careful with real-world repositories, and run dcommit with the
   '-n' flag before actually committing to verify the diffs are sane.

  Thanks

 git-svn.perl                     |   72 +++++++++++++++++++++++++++----
 t/t9114-git-svn-dcommit-merge.sh |   89 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 152 insertions(+), 9 deletions(-)
 create mode 100755 t/t9114-git-svn-dcommit-merge.sh

diff --git a/git-svn.perl b/git-svn.perl
index 0ae8d70..4290676 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -372,16 +372,9 @@ sub cmd_dcommit {
 		die "Unable to determine upstream SVN information from ",
 		    "$head history\n";
 	}
-	my $c = $refs[-1];
 	my $last_rev;
-	foreach my $d (@refs) {
-		if (!verify_ref("$d~1")) {
-			fatal "Commit $d\n",
-			      "has no parent commit, and therefore ",
-			      "nothing to diff against.\n",
-			      "You should be working from a repository ",
-			      "originally created by git-svn\n";
-		}
+	my ($linear_refs, $parents) = linearize_history($gs, \@refs);
+	foreach my $d (@$linear_refs) {
 		unless (defined $last_rev) {
 			(undef, $last_rev, undef) = cmt_metadata("$d~1");
 			unless (defined $last_rev) {
@@ -403,6 +396,9 @@ sub cmd_dcommit {
 			                svn_path => '');
 			if (!SVN::Git::Editor->new(\%ed_opts)->apply_diff) {
 				print "No changes\n$d~1 == $d\n";
+			} elsif ($parents->{$d} && @{$parents->{$d}}) {
+				$gs->{inject_parents_dcommit}->{$last_rev} =
+				                               $parents->{$d};
 			}
 		}
 	}
@@ -821,6 +817,59 @@ sub working_head_info {
 	(undef, undef, undef, undef);
 }
 
+sub read_commit_parents {
+	my ($parents, $c) = @_;
+	my ($fh, $ctx) = command_output_pipe(qw/cat-file commit/, $c);
+	while (<$fh>) {
+		chomp;
+		last if '';
+		/^parent ($sha1)/ or next;
+		push @{$parents->{$c}}, $1;
+	}
+	close $fh; # break the pipe
+}
+
+sub linearize_history {
+	my ($gs, $refs) = @_;
+	my %parents;
+	foreach my $c (@$refs) {
+		read_commit_parents(\%parents, $c);
+	}
+
+	my @linear_refs;
+	my %skip = ();
+	my $last_svn_commit = $gs->last_commit;
+	foreach my $c (reverse @$refs) {
+		next if $c eq $last_svn_commit;
+		last if $skip{$c};
+
+		unshift @linear_refs, $c;
+		$skip{$c} = 1;
+
+		# we only want the first parent to diff against for linear
+		# history, we save the rest to inject when we finalize the
+		# svn commit
+		my $fp_a = verify_ref("$c~1");
+		my $fp_b = shift @{$parents{$c}} if $parents{$c};
+		if (!$fp_a || !$fp_b) {
+			die "Commit $c\n",
+			    "has no parent commit, and therefore ",
+			    "nothing to diff against.\n",
+			    "You should be working from a repository ",
+			    "originally created by git-svn\n";
+		}
+		if ($fp_a ne $fp_b) {
+			die "$c~1 = $fp_a, however parsing commit $c ",
+			    "revealed that:\n$c~1 = $fp_b\nBUG!\n";
+		}
+
+		foreach my $p (@{$parents{$c}}) {
+			$skip{$p} = 1;
+		}
+	}
+	(\@linear_refs, \%parents);
+}
+
 package Git::SVN;
 use strict;
 use warnings;
@@ -1541,6 +1590,11 @@ sub get_commit_parents {
 	if (my $cur = ::verify_ref($self->refname.'^0')) {
 		push @tmp, $cur;
 	}
+	if (my $ipd = $self->{inject_parents_dcommit}) {
+		if (my $commit = delete $ipd->{$log_entry->{revision}}) {
+			push @tmp, @$commit;
+		}
+	}
 	push @tmp, $_ foreach (@{$log_entry->{parents}}, @tmp);
 	while (my $p = shift @tmp) {
 		next if $seen{$p};
diff --git a/t/t9114-git-svn-dcommit-merge.sh b/t/t9114-git-svn-dcommit-merge.sh
new file mode 100755
index 0000000..d6ca955
--- /dev/null
+++ b/t/t9114-git-svn-dcommit-merge.sh
@@ -0,0 +1,89 @@
+#!/bin/sh
+#
+# Copyright (c) 2007 Eric Wong
+# Based on a script by Joakim Tjernlund <joakim.tjernlund@transmode.se>
+
+test_description='git-svn dcommit handles merges'
+
+. ./lib-git-svn.sh
+
+big_text_block () {
+cat << EOF
+#
+# (C) Copyright 2000 - 2005
+# Wolfgang Denk, DENX Software Engineering, wd@denx.de.
+#
+# See file CREDITS for list of people who contributed to this
+# project.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation; either version 2 of
+# the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston,
+# MA 02111-1307 USA
+#
+EOF
+}
+
+test_expect_success 'setup svn repository' "
+	svn co $svnrepo mysvnwork &&
+	mkdir -p mysvnwork/trunk &&
+	cd mysvnwork &&
+		big_text_block >> trunk/README &&
+		svn add trunk &&
+		svn ci -m 'first commit' trunk &&
+		cd ..
+	"
+
+test_expect_success 'setup git mirror and merge' "
+	git svn init $svnrepo -t tags -T trunk -b branches &&
+	git svn fetch &&
+	git checkout --track -b svn remotes/trunk &&
+	git checkout -b merge &&
+	echo new file > new_file &&
+	git add new_file &&
+	git commit -a -m 'New file' &&
+	echo hello >> README &&
+	git commit -a -m 'hello' &&
+	echo add some stuff >> new_file &&
+	git commit -a -m 'add some stuff' &&
+	git checkout svn &&
+	mv -f README tmp &&
+	echo friend > README &&
+	cat tmp >> README &&
+	git commit -a -m 'friend' &&
+	git pull . merge
+	"
+
+test_debug 'gitk --all & sleep 1'
+
+test_expect_success 'verify pre-merge ancestry' "
+	test x\`git rev-parse --verify refs/heads/svn^2\` = \
+	     x\`git rev-parse --verify refs/heads/merge\` &&
+	git cat-file commit refs/heads/svn^ | grep '^friend$'
+	"
+
+test_expect_success 'git svn dcommit merges' "
+	git svn dcommit
+	"
+
+test_debug 'gitk --all & sleep 1'
+
+test_expect_success 'verify post-merge ancestry' "
+	test x\`git rev-parse --verify refs/heads/svn\` = \
+	     x\`git rev-parse --verify refs/remotes/trunk \` &&
+	test x\`git rev-parse --verify refs/heads/svn^2\` = \
+	     x\`git rev-parse --verify refs/heads/merge\` &&
+	git cat-file commit refs/heads/svn^ | grep '^friend$'
+	"
+
+test_done
-- 
Eric Wong

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Joakim Tjernlund]

On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:
> dcommit will still rewrite the HEAD commit and the history of the first
> parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.
> 
> However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
> preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
> with dcommit.  Commits written to SVN will still not have any merge
> information besides anything in the commit message.
> 
> Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
> for explanations, feedback, examples and test case.
> 
> Signed-off-by: Eric Wong <normalperson@yhbt.net>
> ---
> 
>  This is a better patch that replaces the previous one.
> 
>  Junio:
>    This one is a big change and should probably sit in pu or next
>    for a bit.  Double-checking the logic in linearize_history()
>    would be greatly appreciated, too.
>    
>    I don't think there are any regressions for the
>    already-linear-history case besides slightly reduced performance for
>    new calls to cat-file.
> 
>  Joakim/Steven:
>    Any further testing and test cases would be appreciated.  Be very
>    careful with real-world repositories, and run dcommit with the
>    '-n' flag before actually committing to verify the diffs are sane.
> 
>   Thanks
> 

Did a little testing and so far it looks good :)

Sidenote:
Doing this 
  git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
  git-svn fetch --quiet
makes git svn fetch stop for rather long periods in do_update:
  Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
  Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
  Following parent with do_update
  Successfully followed parent

Is it possible to speed up do_update?


Lastly, when adding the above u-boot svn repo into a fresh u-boot clone from WD,
can I attach the svn tree to git u-boot tree without using a graft?

I want to be able to recreate my own git repo by cloning the orginal u-boot
repo and the svn repo.

 Jocke

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Joakim Tjernlund]

On Wed, 2007-06-13 at 19:13 +0200, Joakim Tjernlund wrote:
> On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:
> > dcommit will still rewrite the HEAD commit and the history of the first
> > parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.
> > 
> > However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
> > preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
> > with dcommit.  Commits written to SVN will still not have any merge
> > information besides anything in the commit message.
> > 
> > Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
> > for explanations, feedback, examples and test case.
> > 
> > Signed-off-by: Eric Wong <normalperson@yhbt.net>
> > ---
> > 
> >  This is a better patch that replaces the previous one.
> > 
> >  Junio:
> >    This one is a big change and should probably sit in pu or next
> >    for a bit.  Double-checking the logic in linearize_history()
> >    would be greatly appreciated, too.
> >    
> >    I don't think there are any regressions for the
> >    already-linear-history case besides slightly reduced performance for
> >    new calls to cat-file.
> > 
> >  Joakim/Steven:
> >    Any further testing and test cases would be appreciated.  Be very
> >    careful with real-world repositories, and run dcommit with the
> >    '-n' flag before actually committing to verify the diffs are sane.
> > 
> >   Thanks
> > 
> 
> Did a little testing and so far it looks good :)
> 
> Sidenote:
> Doing this 
>   git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
>   git-svn fetch --quiet
> makes git svn fetch stop for rather long periods in do_update:
>   Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
>   Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
>   Following parent with do_update
>   Successfully followed parent
> 
> Is it possible to speed up do_update?
> 
> 
> Lastly, when adding the above u-boot svn repo into a fresh u-boot clone from WD,
> can I attach the svn tree to git u-boot tree without using a graft?
> 
> I want to be able to recreate my own git repo by cloning the orginal u-boot
> repo and the svn repo.
> 
>  Jocke

Tried using --no-metadata(git svn clone --no-metadata) in my little test
script I sent earlier and got
  "Unable to determine upstream SVN information from HEAD history"
when dcommiting, -i trunk didn't help either.

It is not entierly clear to me what --no-metadata means to me.
Does git-svn still rewrite commits?
I can't rebuild rev_db file, if lost, but I guess I could still
do a new git-svn clone and restore my repo? I guess I lose something
if I do that but what?
Also don't really understand why git-svn log doesn't work, can't it get
that info from the svn repo?

 Jocke

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Eric Wong]

Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> On Wed, 2007-06-13 at 19:13 +0200, Joakim Tjernlund wrote:
> > On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:
> > > dcommit will still rewrite the HEAD commit and the history of the first
> > > parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.
> > > 
> > > However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
> > > preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
> > > with dcommit.  Commits written to SVN will still not have any merge
> > > information besides anything in the commit message.
> > > 
> > > Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
> > > for explanations, feedback, examples and test case.
> > > 
> > > Signed-off-by: Eric Wong <normalperson@yhbt.net>
> > > ---
> > > 
> > >  This is a better patch that replaces the previous one.
> > > 
> > >  Junio:
> > >    This one is a big change and should probably sit in pu or next
> > >    for a bit.  Double-checking the logic in linearize_history()
> > >    would be greatly appreciated, too.
> > >    
> > >    I don't think there are any regressions for the
> > >    already-linear-history case besides slightly reduced performance for
> > >    new calls to cat-file.
> > > 
> > >  Joakim/Steven:
> > >    Any further testing and test cases would be appreciated.  Be very
> > >    careful with real-world repositories, and run dcommit with the
> > >    '-n' flag before actually committing to verify the diffs are sane.
> > > 
> > >   Thanks
> > > 
> > 
> > Did a little testing and so far it looks good :)
> > 
> > Sidenote:
> > Doing this 
> >   git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
> >   git-svn fetch --quiet
> > makes git svn fetch stop for rather long periods in do_update:
> >   Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
> >   Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
> >   Following parent with do_update
> >   Successfully followed parent
> > 
> > Is it possible to speed up do_update?
> > 
> > 
> > Lastly, when adding the above u-boot svn repo into a fresh u-boot clone from WD,
> > can I attach the svn tree to git u-boot tree without using a graft?
> > 
> > I want to be able to recreate my own git repo by cloning the orginal u-boot
> > repo and the svn repo.
> > 
> >  Jocke
> 
> Tried using --no-metadata(git svn clone --no-metadata) in my little test
> script I sent earlier and got
>   "Unable to determine upstream SVN information from HEAD history"
> when dcommiting, -i trunk didn't help either.
> 
> It is not entierly clear to me what --no-metadata means to me.
> Does git-svn still rewrite commits?

--no-metadata is really only useful for people doing one-shot imports
and abandoning SVN.  It leaves out the git-svn-id: lines at the bottom
of commit messages, but still sets the committer/author names/email/date
to what is in the SVN repository.

> I can't rebuild rev_db file, if lost, but I guess I could still
> do a new git-svn clone and restore my repo? I guess I lose something
> if I do that but what?

If you lose your rev_db file with no-metadata, you'll have to redo the
git-svn clone

> Also don't really understand why git-svn log doesn't work, can't it get
> that info from the svn repo?

Getting git-svn log working with --no-metadata would require radically
different code.  dcommit would be very different, too.  So yes, they
don't work because I'm lazy.

-- 
Eric Wong

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Eric Wong]

Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:
> > dcommit will still rewrite the HEAD commit and the history of the first
> > parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.
> > 
> > However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
> > preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
> > with dcommit.  Commits written to SVN will still not have any merge
> > information besides anything in the commit message.
> > 
> > Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
> > for explanations, feedback, examples and test case.
> > 
> > Signed-off-by: Eric Wong <normalperson@yhbt.net>
> > ---
> > 
> >  This is a better patch that replaces the previous one.
> > 
> >  Junio:
> >    This one is a big change and should probably sit in pu or next
> >    for a bit.  Double-checking the logic in linearize_history()
> >    would be greatly appreciated, too.
> >    
> >    I don't think there are any regressions for the
> >    already-linear-history case besides slightly reduced performance for
> >    new calls to cat-file.
> > 
> >  Joakim/Steven:
> >    Any further testing and test cases would be appreciated.  Be very
> >    careful with real-world repositories, and run dcommit with the
> >    '-n' flag before actually committing to verify the diffs are sane.
> > 
> >   Thanks
> > 
> 
> Did a little testing and so far it looks good :)
> 
> Sidenote:
> Doing this 
>   git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
>   git-svn fetch --quiet
> makes git svn fetch stop for rather long periods in do_update:
>   Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
>   Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
>   Following parent with do_update
>   Successfully followed parent
> 
> Is it possible to speed up do_update?

Use a do_switch()-enabled SVN to avoid do_update().  do_update will
redownload everything.  I have patched 1.4.3 debian packages with source
and a diff here: http://git-svn.bogomips.org/svn.  SVN 1.4.4 claims to
have fixed the bindings, but 1.4.3 claimed the same thing, too...
Confirmation of it working in SVN 1.4.4 would be nice.

> Lastly, when adding the above u-boot svn repo into a fresh u-boot
> clone from WD, can I attach the svn tree to git u-boot tree without
> using a graft?

Not with the current version.  The 1.5.0 (or previous, I forget) allowed
forced-parenting with: "git-svn fetch <rev>=<commit>" but I figured
nobody was using it, and it would be difficult to get working since
fetch can now works on multiple trees and the same revision numbers can
appear in multiple trees.

> I want to be able to recreate my own git repo by cloning the orginal
> u-boot repo and the svn repo.

-- 
Eric Wong

RE: [PATCH] git-svn: allow dcommit to retain local merge information

[Joakim Tjernlund]

 

> -----Original Message-----
> From: Eric Wong [mailto:normalperson@yhbt.net] 
> Sent: den 20 juni 2007 08:56
> To: Joakim Tjernlund
> Cc: Junio C Hamano; Steven Grimm; git@vger.kernel.org
> Subject: Re: [PATCH] git-svn: allow dcommit to retain local 
> merge information
> 
> Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> > On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:
> > > dcommit will still rewrite the HEAD commit and the 
> history of the first
> > > parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.
> > > 
> > > However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
> > > preserved when the new HEAD and HEAD~[0-9]+ commits are 
> rewritten to SVN
> > > with dcommit.  Commits written to SVN will still not have 
> any merge
> > > information besides anything in the commit message.
> > > 
> > > Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
> > > for explanations, feedback, examples and test case.
> > > 
> > > Signed-off-by: Eric Wong <normalperson@yhbt.net>
> > > ---
> > > 
> > >  This is a better patch that replaces the previous one.
> > > 
> > >  Junio:
> > >    This one is a big change and should probably sit in pu or next
> > >    for a bit.  Double-checking the logic in linearize_history()
> > >    would be greatly appreciated, too.
> > >    
> > >    I don't think there are any regressions for the
> > >    already-linear-history case besides slightly reduced 
> performance for
> > >    new calls to cat-file.
> > > 
> > >  Joakim/Steven:
> > >    Any further testing and test cases would be 
> appreciated.  Be very
> > >    careful with real-world repositories, and run dcommit with the
> > >    '-n' flag before actually committing to verify the 
> diffs are sane.
> > > 
> > >   Thanks
> > > 
> > 
> > Did a little testing and so far it looks good :)
> > 
> > Sidenote:
> > Doing this 
> >   git-svn init -t tags -T trunk -b branches  
> file:///usr/local/src/tst-git-svn/svn-uboot-repo
> >   git-svn fetch --quiet
> > makes git svn fetch stop for rather long periods in do_update:
> >   Found possible branch point: 
> file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => 
> file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-ubo
> ot-1.1.4, 2
> >   Found branch parent: (tags/snap-uboot-1.1.4) 
> 81eef14963597cc99ba375f52e6d0b3bc09e25f8
> >   Following parent with do_update
> >   Successfully followed parent
> > 
> > Is it possible to speed up do_update?
> 
> Use a do_switch()-enabled SVN to avoid do_update().  do_update will
> redownload everything.  I have patched 1.4.3 debian packages 
> with source
> and a diff here: http://git-svn.bogomips.org/svn.  SVN 1.4.4 claims to
> have fixed the bindings, but 1.4.3 claimed the same thing, too...
> Confirmation of it working in SVN 1.4.4 would be nice.

Confirmed as requested, I installed 1.4.4(Gentoo) an ran the
same test case. Now I see "Following parent with do_switch"
instead and it is almost instant. It felt though that
git-svn was somewhat slower importing large diffs.

 Jocke

> 
> > Lastly, when adding the above u-boot svn repo into a fresh u-boot
> > clone from WD, can I attach the svn tree to git u-boot tree without
> > using a graft?
> 
> Not with the current version.  The 1.5.0 (or previous, I 
> forget) allowed
> forced-parenting with: "git-svn fetch <rev>=<commit>" but I figured
> nobody was using it, and it would be difficult to get working since
> fetch can now works on multiple trees and the same revision 
> numbers can
> appear in multiple trees.

If you reconsider this, please let me know.

 Jocke

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Joakim Tjernlund]

> > Sidenote:
> > Doing this 
> >   git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
> >   git-svn fetch --quiet
> > makes git svn fetch stop for rather long periods in do_update:
> >   Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
> >   Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
> >   Following parent with do_update
> >   Successfully followed parent
> > 
> > Is it possible to speed up do_update?
> 
> Use a do_switch()-enabled SVN to avoid do_update().  do_update will
> redownload everything.  I have patched 1.4.3 debian packages with source
> and a diff here: http://git-svn.bogomips.org/svn.  SVN 1.4.4 claims to
> have fixed the bindings, but 1.4.3 claimed the same thing, too...
> Confirmation of it working in SVN 1.4.4 would be nice.

I upgraded to svn 1.4.4 and it seemed to work OK, but then i noticed
some problems, 
 git clone -t tags -T trunk -b branches svn+ssh://devsrv/svn/TM-uboot
failed(error msg was something with "Malformed data ...")

I decided to go back to 1.4.3 with you patch, switch-editor-perl.diff
applied. Now the error was gone, but the clone operation did use
do_update insted of do_switch. Did I miss someting?

 Jocke

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Steven Grimm]

Eric Wong wrote:
>  Joakim/Steven:
>    Any further testing and test cases would be appreciated.  Be very
>    careful with real-world repositories, and run dcommit with the
>    '-n' flag before actually committing to verify the diffs are sane.
>   

I poked at this some tonight with an eye toward the use case of using 
git to merge svn branches. I ran into one inconvenience and one bug. 
I'll try playing with the "use lots of git branches to develop on one 
svn branch" use case some too, but for now, here are the notes I took, 
along with the commands if anyone wants to reproduce what I did. 
Hopefully this won't be too annoying to read. The bug is near the bottom.

svn repo with a trunk and a branch, each with changes (no conflicts at 
first, keep it simple to start)

$ svnadmin create svnrepo
$ svn co file://`pwd`/svnrepo svnclient
$ cd svnclient
$ mkdir trunk tags branches
$ echo test file number 1 > trunk/testfile1
$ echo test file number 2 > trunk/testfile2
$ svn add *
$ svn commit -m "initial commit"
$ echo trunk change 1 >> trunk/testfile1
$ svn commit -m "trunk change 1"
$ echo trunk change 2 >> trunk/testfile2
$ svn commit -m "trunk change 2"
$ svn cp trunk branches/mybranch
$ svn commit -m "make a branch"
$ echo trunk change 3 >> trunk/testfile1
$ svn commit -m "post-branch change in trunk"
$ echo branch change 1 >> branches/mybranch/testfile2
$ svn commit -m "change in branch"

git-svn clone of this dinky repo

$ cd ..
$ git-svn clone --trunk=trunk --branches=branches --tags=tags 
file://`pwd`/svnrepo gitclone
$ cd gitclone

Try to merge trunk change into branch using git

$ git reset --hard mybranch
$ git merge trunk

Conflicts! what's going on?

$ gitk --all

Aha, looks like git-svn guessed wrong about where I made the branch; it 
thinks the branch comes from the initial rev. Easy enough to hack 
around, but might be nice to be able to do this using git-svn's history 
rewriting rather than a grafts file.

$ echo `git-svn find-rev r4` `git-svn find-rev r3 trunk` > .git/info/grafts
$ git reset --hard mybranch
$ git merge trunk

No conflicts now. Let's see what git-svn thinks it should do

$ git-svn dcommit -n

Looks like the right diff

$ git-svn dcommit

Refresh gitk display. Looks good, the new revision is a merge with the 
right parents. Let's check it out in svn land

$ cd ../svnclient
$ svn up
$ cat branches/mybranch/testfile1

Yep, the trunk change is there, nice! Now for a couple more revs with a 
conflict.

$ echo post-merge trunk change >> trunk/testfile1
$ svn commit -m "trunk change after merge"
$ echo post-merge conflicting change >> trunk/testfile2
$ svn commit -m "trunk change with conflict"
$ cd ../gitclone
$ git-svn fetch
$ git merge -m "change with conflict" trunk

Conflict, as expected

$ vi testfile2
$ git add testfile2
$ git commit
$ git-svn dcommit
Transaction is out of date: Out of date: '/trunk/testfile1' in 
transaction '9-1' at /Users/koreth/git/git-svn line 398

Hmm, this merge was in mybranch, not in trunk

$ git log --first-parent

Yes, the most recent commit with a git-svn-id line has a mybranch URL. 
So why is it complaining about a trunk file being out of date?

My experimentation pretty much ended there (I tried a few things to 
clear the error up, but none of them helped.)

This machine is an OS X laptop. Subversion is 1.4.3 (r23084) from 
MacPorts. I used the git-svn from the "pu" branch since it had this 
patch and all the recent fixes.

Let me know if you need more details. Hope this is helpful.

-Steve

RE: [PATCH] git-svn: allow dcommit to retain local merge information

[Joakim Tjernlund]

> -----Original Message-----
> From: Steven Grimm [mailto:koreth@midwinter.com] 
> Sent: den 14 juni 2007 08:31
> To: Eric Wong
> Cc: Junio C Hamano; Joakim Tjernlund; git@vger.kernel.org
> Subject: Re: [PATCH] git-svn: allow dcommit to retain local 
> merge information
> 
> Eric Wong wrote:
> >  Joakim/Steven:
> >    Any further testing and test cases would be appreciated.  Be very
> >    careful with real-world repositories, and run dcommit with the
> >    '-n' flag before actually committing to verify the diffs 
> are sane.
> >   
> 
> I poked at this some tonight with an eye toward the use case of using 
> git to merge svn branches. I ran into one inconvenience and one bug. 
> I'll try playing with the "use lots of git branches to develop on one 
> svn branch" use case some too, but for now, here are the 
> notes I took, 
> along with the commands if anyone wants to reproduce what I did. 
> Hopefully this won't be too annoying to read. The bug is near 
> the bottom.
> 
> svn repo with a trunk and a branch, each with changes (no 
> conflicts at 
> first, keep it simple to start)
> 
> $ svnadmin create svnrepo
> $ svn co file://`pwd`/svnrepo svnclient
> $ cd svnclient
> $ mkdir trunk tags branches
> $ echo test file number 1 > trunk/testfile1
> $ echo test file number 2 > trunk/testfile2
> $ svn add *
> $ svn commit -m "initial commit"
> $ echo trunk change 1 >> trunk/testfile1
> $ svn commit -m "trunk change 1"
> $ echo trunk change 2 >> trunk/testfile2
> $ svn commit -m "trunk change 2"
> $ svn cp trunk branches/mybranch
> $ svn commit -m "make a branch"
> $ echo trunk change 3 >> trunk/testfile1
> $ svn commit -m "post-branch change in trunk"
> $ echo branch change 1 >> branches/mybranch/testfile2
> $ svn commit -m "change in branch"
> 
> git-svn clone of this dinky repo
> 
> $ cd ..
> $ git-svn clone --trunk=trunk --branches=branches --tags=tags 
> file://`pwd`/svnrepo gitclone
> $ cd gitclone
> 
> Try to merge trunk change into branch using git
> 
> $ git reset --hard mybranch
> $ git merge trunk
> 
> Conflicts! what's going on?
> 
> $ gitk --all
> 
> Aha, looks like git-svn guessed wrong about where I made the 
> branch; it 
> thinks the branch comes from the initial rev. Easy enough to hack 
> around, but might be nice to be able to do this using 
> git-svn's history 
> rewriting rather than a grafts file.

Yes, that would be nice indeed.

> 
> $ echo `git-svn find-rev r4` `git-svn find-rev r3 trunk` > 
> .git/info/grafts
> $ git reset --hard mybranch
> $ git merge trunk
> 
> No conflicts now. Let's see what git-svn thinks it should do
> 
> $ git-svn dcommit -n
> 
> Looks like the right diff
> 
> $ git-svn dcommit
> 
> Refresh gitk display. Looks good, the new revision is a merge 
> with the 
> right parents. Let's check it out in svn land
> 
> $ cd ../svnclient
> $ svn up
> $ cat branches/mybranch/testfile1
> 
> Yep, the trunk change is there, nice! Now for a couple more 
> revs with a 
> conflict.
> 
> $ echo post-merge trunk change >> trunk/testfile1
> $ svn commit -m "trunk change after merge"
> $ echo post-merge conflicting change >> trunk/testfile2
> $ svn commit -m "trunk change with conflict"
> $ cd ../gitclone
> $ git-svn fetch
> $ git merge -m "change with conflict" trunk
> 
> Conflict, as expected
> 
> $ vi testfile2
> $ git add testfile2
> $ git commit
> $ git-svn dcommit
> Transaction is out of date: Out of date: '/trunk/testfile1' in 
> transaction '9-1' at /Users/koreth/git/git-svn line 398

Maybe this can help?
http://svn.haxx.se/subusers/archive-2005-02/0096.shtml
http://subclipse.tigris.org/faq.html#out-of-date

> 
> Hmm, this merge was in mybranch, not in trunk
> 
> $ git log --first-parent
> 
> Yes, the most recent commit with a git-svn-id line has a 
> mybranch URL. 
> So why is it complaining about a trunk file being out of date?
> 
> My experimentation pretty much ended there (I tried a few things to 
> clear the error up, but none of them helped.)
> 
> This machine is an OS X laptop. Subversion is 1.4.3 (r23084) from 
> MacPorts. I used the git-svn from the "pu" branch since it had this 
> patch and all the recent fixes.
> 
> Let me know if you need more details. Hope this is helpful.
> 
> -Steve
> 
> 

Re: git-svn set-tree bug

[Lars Hjemli]

On 6/12/07, Eric Wong <normalperson@yhbt.net> wrote:
> If dcommit detects a merge commit when doing rev-list When looking at
> commit objects, is it safe to assume that the first parent is always the
> "mainline" and that parents after it are the ones to merge from?
>
> So if I saw:
>
> commit $X
> parent $A
> parent $B
>
> I'd basically do:
>   reset --hard $A
>   merge --squash $B
>
> And resulting in $C which would have the same tree as $X,
> then, when dcommit-ting, $D would be created with two parents:
>   $D~1 (svn), $B (git), but not $A
>
> Rewritten history:
>   $A         =>  $D~1
>   $X         =>  $D (HEAD revision in SVN)
>
> $X and $A are now discarded and gc-able.
>
>
> Of course, since I already have the result of "merge --squash $B" in $X,
> I could just rewrite $X with a single parent (call it $X'), dcommit, and
> then give $D ($D~1 and $B) as parents.  Avoiding the nastiness of
> set-tree
>

Would it be possible to keep the 'local commit' $X and change the
mapping in .rev_db to point at $X instead of $D? This would of course
require a matching TREE-ID + noMetadata=1. I've been tempted to try to
implement this, but my perl-skills are sadly non-existent.

If this is possible it would make my day-job interaction with svn a
much more pleasant experience: push/pull between git repos would 'just
work' (without -f).

And if "follow left parent" also works out, I (and our
'releasemaster') can finally do all merging in git (without --squash)
and preserve the DAG. Ahh, that would be great...

--
larsh

Re: git-svn set-tree bug

[Steven Grimm]

Eric Wong wrote:
> Doable?  Yes.  However, I think using grafts is quite hackish and
> unreliable[1].  I'd rather just have users using set-tree if
> they want to deal with non-linear history in the first place.
>   

Agreed about grafts being hackish and unreliable. But they were what I 
had to work with, given that I know little enough about git-svn's 
internals to be able to implement Junio's more robust idea.

IMO set-tree is not much of an option. In my environment it is 
unacceptable for there to be any possibility of accidentally and 
silently overwriting some other change that just happened to hit the svn 
repo right before I committed my change, which (unless it has changed 
since I last tried it) set-tree will happily do. I can get away with 
doing that maybe once before my company's release manager will, quite 
justifiably, require me to stop using git and switch back to the 
standard svn client.

> I'd personally avoid any sort of non-linear history when interacting
> with SVN repositories, however.
>   

Which is a shame since git loses a lot of its utility without nonlinear 
history. For example, the script I posted uses git to do merges between 
svn branches. It works wonderfully even if, as you and Junio point out, 
its use of grafts to record svn merges scales poorly and is potentially 
susceptible to corruption. Thanks to the ability to record the fact that 
my merges between svn branches were actually merges, my git clone has a 
more complete picture of what's in my svn repository than the svn 
repository itself does!

-Steve

Re: git-svn set-tree bug

[Joakim Tjernlund]

On Sun, 2007-06-10 at 23:58 -0700, Steven Grimm wrote:
> Eric Wong wrote:
> > Doable?  Yes.  However, I think using grafts is quite hackish and
> > unreliable[1].  I'd rather just have users using set-tree if
> > they want to deal with non-linear history in the first place.
> >   
> 
> Agreed about grafts being hackish and unreliable. But they were what I 
> had to work with, given that I know little enough about git-svn's 
> internals to be able to implement Junio's more robust idea.
> 
> IMO set-tree is not much of an option. In my environment it is 
> unacceptable for there to be any possibility of accidentally and 
> silently overwriting some other change that just happened to hit the svn 
> repo right before I committed my change, which (unless it has changed 
> since I last tried it) set-tree will happily do. I can get away with 
> doing that maybe once before my company's release manager will, quite 
> justifiably, require me to stop using git and switch back to the 
> standard svn client.
> 
> > I'd personally avoid any sort of non-linear history when interacting
> > with SVN repositories, however.
> >   
> 
> Which is a shame since git loses a lot of its utility without nonlinear 
> history. For example, the script I posted uses git to do merges between 
> svn branches. It works wonderfully even if, as you and Junio point out, 
> its use of grafts to record svn merges scales poorly and is potentially 
> susceptible to corruption. Thanks to the ability to record the fact that 
> my merges between svn branches were actually merges, my git clone has a 
> more complete picture of what's in my svn repository than the svn 
> repository itself does!

Exactly, I too think that this is a much needed feature for git-svn.

I have also noted that this modified(added a git svn dcommit) script 
works:
rm -rf mygitsvn
rm -rf mysvnrepo
rm -rf mysvnwork
mkdir mysvnrepo
cd mysvnrepo
svnadmin create .
cd ..
svn checkout file:///$PWD/mysvnrepo mysvnwork
mkdir -p mysvnwork/trunk
cd mysvnwork/
cat << EOF  > trunk/README
#
# (C) Copyright 2000 - 2005
# Wolfgang Denk, DENX Software Engineering, wd@denx.de.
#
# See file CREDITS for list of people who contributed to this
# project.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2 of
# the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston,
# MA 02111-1307 USA
#

EOF

svn add trunk
svn ci -m "first commit" trunk
cd ..
git-svn clone  file:///$PWD/mysvnrepo -t tags -T trunk -b branches  mygitsvn
cd mygitsvn

git checkout --track -b svn remotes/trunk
git checkout -b merge
echo new file > new_file
git add new_file
git commit -a -m "New file"

echo hello >> README
git commit -a  -m "hello"

echo add some stuff >> new_file
git commit -a -m "add some stuff"

git checkout svn
mv -f README tmp
echo friend > README
cat tmp >> README
git commit -a -m "friend"
git svn dcommit
git pull . merge
git svn dcommit

but, as expected, then merge commit is still gone.