[PATCH] git-svn: allow dcommit to retain local merge information

[PATCH] git-svn: allow dcommit to retain local merge information

[Eric Wong]

dcommit will still rewrite the HEAD commit and the history of the first
parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.

However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
with dcommit.  Commits written to SVN will still not have any merge
information besides anything in the commit message.

Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
for explanations, feedback, examples and test case.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
---

 This is a better patch that replaces the previous one.

 Junio:
   This one is a big change and should probably sit in pu or next
   for a bit.  Double-checking the logic in linearize_history()
   would be greatly appreciated, too.
   
   I don't think there are any regressions for the
   already-linear-history case besides slightly reduced performance for
   new calls to cat-file.

 Joakim/Steven:
   Any further testing and test cases would be appreciated.  Be very
   careful with real-world repositories, and run dcommit with the
   '-n' flag before actually committing to verify the diffs are sane.

  Thanks

 git-svn.perl                     |   72 +++++++++++++++++++++++++++----
 t/t9114-git-svn-dcommit-merge.sh |   89 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 152 insertions(+), 9 deletions(-)
 create mode 100755 t/t9114-git-svn-dcommit-merge.sh

diff --git a/git-svn.perl b/git-svn.perl
index 0ae8d70..4290676 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -372,16 +372,9 @@ sub cmd_dcommit {
 		die "Unable to determine upstream SVN information from ",
 		    "$head history\n";
 	}
-	my $c = $refs[-1];
 	my $last_rev;
-	foreach my $d (@refs) {
-		if (!verify_ref("$d~1")) {
-			fatal "Commit $d\n",
-			      "has no parent commit, and therefore ",
-			      "nothing to diff against.\n",
-			      "You should be working from a repository ",
-			      "originally created by git-svn\n";
-		}
+	my ($linear_refs, $parents) = linearize_history($gs, \@refs);
+	foreach my $d (@$linear_refs) {
 		unless (defined $last_rev) {
 			(undef, $last_rev, undef) = cmt_metadata("$d~1");
 			unless (defined $last_rev) {
@@ -403,6 +396,9 @@ sub cmd_dcommit {
 			                svn_path => '');
 			if (!SVN::Git::Editor->new(\%ed_opts)->apply_diff) {
 				print "No changes\n$d~1 == $d\n";
+			} elsif ($parents->{$d} && @{$parents->{$d}}) {
+				$gs->{inject_parents_dcommit}->{$last_rev} =
+				                               $parents->{$d};
 			}
 		}
 	}
@@ -821,6 +817,59 @@ sub working_head_info {
 	(undef, undef, undef, undef);
 }
 
+sub read_commit_parents {
+	my ($parents, $c) = @_;
+	my ($fh, $ctx) = command_output_pipe(qw/cat-file commit/, $c);
+	while (<$fh>) {
+		chomp;
+		last if '';
+		/^parent ($sha1)/ or next;
+		push @{$parents->{$c}}, $1;
+	}
+	close $fh; # break the pipe
+}
+
+sub linearize_history {
+	my ($gs, $refs) = @_;
+	my %parents;
+	foreach my $c (@$refs) {
+		read_commit_parents(\%parents, $c);
+	}
+
+	my @linear_refs;
+	my %skip = ();
+	my $last_svn_commit = $gs->last_commit;
+	foreach my $c (reverse @$refs) {
+		next if $c eq $last_svn_commit;
+		last if $skip{$c};
+
+		unshift @linear_refs, $c;
+		$skip{$c} = 1;
+
+		# we only want the first parent to diff against for linear
+		# history, we save the rest to inject when we finalize the
+		# svn commit
+		my $fp_a = verify_ref("$c~1");
+		my $fp_b = shift @{$parents{$c}} if $parents{$c};
+		if (!$fp_a || !$fp_b) {
+			die "Commit $c\n",
+			    "has no parent commit, and therefore ",
+			    "nothing to diff against.\n",
+			    "You should be working from a repository ",
+			    "originally created by git-svn\n";
+		}
+		if ($fp_a ne $fp_b) {
+			die "$c~1 = $fp_a, however parsing commit $c ",
+			    "revealed that:\n$c~1 = $fp_b\nBUG!\n";
+		}
+
+		foreach my $p (@{$parents{$c}}) {
+			$skip{$p} = 1;
+		}
+	}
+	(\@linear_refs, \%parents);
+}
+
 package Git::SVN;
 use strict;
 use warnings;
@@ -1541,6 +1590,11 @@ sub get_commit_parents {
 	if (my $cur = ::verify_ref($self->refname.'^0')) {
 		push @tmp, $cur;
 	}
+	if (my $ipd = $self->{inject_parents_dcommit}) {
+		if (my $commit = delete $ipd->{$log_entry->{revision}}) {
+			push @tmp, @$commit;
+		}
+	}
 	push @tmp, $_ foreach (@{$log_entry->{parents}}, @tmp);
 	while (my $p = shift @tmp) {
 		next if $seen{$p};
diff --git a/t/t9114-git-svn-dcommit-merge.sh b/t/t9114-git-svn-dcommit-merge.sh
new file mode 100755
index 0000000..d6ca955
--- /dev/null
+++ b/t/t9114-git-svn-dcommit-merge.sh
@@ -0,0 +1,89 @@
+#!/bin/sh
+#
+# Copyright (c) 2007 Eric Wong
+# Based on a script by Joakim Tjernlund <joakim.tjernlund@transmode.se>
+
+test_description='git-svn dcommit handles merges'
+
+. ./lib-git-svn.sh
+
+big_text_block () {
+cat << EOF
+#
+# (C) Copyright 2000 - 2005
+# Wolfgang Denk, DENX Software Engineering, wd@denx.de.
+#
+# See file CREDITS for list of people who contributed to this
+# project.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation; either version 2 of
+# the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.	See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston,
+# MA 02111-1307 USA
+#
+EOF
+}
+
+test_expect_success 'setup svn repository' "
+	svn co $svnrepo mysvnwork &&
+	mkdir -p mysvnwork/trunk &&
+	cd mysvnwork &&
+		big_text_block >> trunk/README &&
+		svn add trunk &&
+		svn ci -m 'first commit' trunk &&
+		cd ..
+	"
+
+test_expect_success 'setup git mirror and merge' "
+	git svn init $svnrepo -t tags -T trunk -b branches &&
+	git svn fetch &&
+	git checkout --track -b svn remotes/trunk &&
+	git checkout -b merge &&
+	echo new file > new_file &&
+	git add new_file &&
+	git commit -a -m 'New file' &&
+	echo hello >> README &&
+	git commit -a -m 'hello' &&
+	echo add some stuff >> new_file &&
+	git commit -a -m 'add some stuff' &&
+	git checkout svn &&
+	mv -f README tmp &&
+	echo friend > README &&
+	cat tmp >> README &&
+	git commit -a -m 'friend' &&
+	git pull . merge
+	"
+
+test_debug 'gitk --all & sleep 1'
+
+test_expect_success 'verify pre-merge ancestry' "
+	test x\`git rev-parse --verify refs/heads/svn^2\` = \
+	     x\`git rev-parse --verify refs/heads/merge\` &&
+	git cat-file commit refs/heads/svn^ | grep '^friend$'
+	"
+
+test_expect_success 'git svn dcommit merges' "
+	git svn dcommit
+	"
+
+test_debug 'gitk --all & sleep 1'
+
+test_expect_success 'verify post-merge ancestry' "
+	test x\`git rev-parse --verify refs/heads/svn\` = \
+	     x\`git rev-parse --verify refs/remotes/trunk \` &&
+	test x\`git rev-parse --verify refs/heads/svn^2\` = \
+	     x\`git rev-parse --verify refs/heads/merge\` &&
+	git cat-file commit refs/heads/svn^ | grep '^friend$'
+	"
+
+test_done
-- 
Eric Wong

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Joakim Tjernlund]

On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:
> dcommit will still rewrite the HEAD commit and the history of the first
> parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.
> 
> However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
> preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
> with dcommit.  Commits written to SVN will still not have any merge
> information besides anything in the commit message.
> 
> Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
> for explanations, feedback, examples and test case.
> 
> Signed-off-by: Eric Wong <normalperson@yhbt.net>
> ---
> 
>  This is a better patch that replaces the previous one.
> 
>  Junio:
>    This one is a big change and should probably sit in pu or next
>    for a bit.  Double-checking the logic in linearize_history()
>    would be greatly appreciated, too.
>    
>    I don't think there are any regressions for the
>    already-linear-history case besides slightly reduced performance for
>    new calls to cat-file.
> 
>  Joakim/Steven:
>    Any further testing and test cases would be appreciated.  Be very
>    careful with real-world repositories, and run dcommit with the
>    '-n' flag before actually committing to verify the diffs are sane.
> 
>   Thanks
> 

Did a little testing and so far it looks good :)

Sidenote:
Doing this 
  git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
  git-svn fetch --quiet
makes git svn fetch stop for rather long periods in do_update:
  Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
  Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
  Following parent with do_update
  Successfully followed parent

Is it possible to speed up do_update?


Lastly, when adding the above u-boot svn repo into a fresh u-boot clone from WD,
can I attach the svn tree to git u-boot tree without using a graft?

I want to be able to recreate my own git repo by cloning the orginal u-boot
repo and the svn repo.

 Jocke

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Joakim Tjernlund]

On Wed, 2007-06-13 at 19:13 +0200, Joakim Tjernlund wrote:
> On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:
> > dcommit will still rewrite the HEAD commit and the history of the first
> > parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.
> > 
> > However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
> > preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
> > with dcommit.  Commits written to SVN will still not have any merge
> > information besides anything in the commit message.
> > 
> > Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
> > for explanations, feedback, examples and test case.
> > 
> > Signed-off-by: Eric Wong <normalperson@yhbt.net>
> > ---
> > 
> >  This is a better patch that replaces the previous one.
> > 
> >  Junio:
> >    This one is a big change and should probably sit in pu or next
> >    for a bit.  Double-checking the logic in linearize_history()
> >    would be greatly appreciated, too.
> >    
> >    I don't think there are any regressions for the
> >    already-linear-history case besides slightly reduced performance for
> >    new calls to cat-file.
> > 
> >  Joakim/Steven:
> >    Any further testing and test cases would be appreciated.  Be very
> >    careful with real-world repositories, and run dcommit with the
> >    '-n' flag before actually committing to verify the diffs are sane.
> > 
> >   Thanks
> > 
> 
> Did a little testing and so far it looks good :)
> 
> Sidenote:
> Doing this 
>   git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
>   git-svn fetch --quiet
> makes git svn fetch stop for rather long periods in do_update:
>   Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
>   Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
>   Following parent with do_update
>   Successfully followed parent
> 
> Is it possible to speed up do_update?
> 
> 
> Lastly, when adding the above u-boot svn repo into a fresh u-boot clone from WD,
> can I attach the svn tree to git u-boot tree without using a graft?
> 
> I want to be able to recreate my own git repo by cloning the orginal u-boot
> repo and the svn repo.
> 
>  Jocke

Tried using --no-metadata(git svn clone --no-metadata) in my little test
script I sent earlier and got
  "Unable to determine upstream SVN information from HEAD history"
when dcommiting, -i trunk didn't help either.

It is not entierly clear to me what --no-metadata means to me.
Does git-svn still rewrite commits?
I can't rebuild rev_db file, if lost, but I guess I could still
do a new git-svn clone and restore my repo? I guess I lose something
if I do that but what?
Also don't really understand why git-svn log doesn't work, can't it get
that info from the svn repo?

 Jocke

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Steven Grimm]

Eric Wong wrote:
>  Joakim/Steven:
>    Any further testing and test cases would be appreciated.  Be very
>    careful with real-world repositories, and run dcommit with the
>    '-n' flag before actually committing to verify the diffs are sane.
>   

I poked at this some tonight with an eye toward the use case of using 
git to merge svn branches. I ran into one inconvenience and one bug. 
I'll try playing with the "use lots of git branches to develop on one 
svn branch" use case some too, but for now, here are the notes I took, 
along with the commands if anyone wants to reproduce what I did. 
Hopefully this won't be too annoying to read. The bug is near the bottom.

svn repo with a trunk and a branch, each with changes (no conflicts at 
first, keep it simple to start)

$ svnadmin create svnrepo
$ svn co file://`pwd`/svnrepo svnclient
$ cd svnclient
$ mkdir trunk tags branches
$ echo test file number 1 > trunk/testfile1
$ echo test file number 2 > trunk/testfile2
$ svn add *
$ svn commit -m "initial commit"
$ echo trunk change 1 >> trunk/testfile1
$ svn commit -m "trunk change 1"
$ echo trunk change 2 >> trunk/testfile2
$ svn commit -m "trunk change 2"
$ svn cp trunk branches/mybranch
$ svn commit -m "make a branch"
$ echo trunk change 3 >> trunk/testfile1
$ svn commit -m "post-branch change in trunk"
$ echo branch change 1 >> branches/mybranch/testfile2
$ svn commit -m "change in branch"

git-svn clone of this dinky repo

$ cd ..
$ git-svn clone --trunk=trunk --branches=branches --tags=tags 
file://`pwd`/svnrepo gitclone
$ cd gitclone

Try to merge trunk change into branch using git

$ git reset --hard mybranch
$ git merge trunk

Conflicts! what's going on?

$ gitk --all

Aha, looks like git-svn guessed wrong about where I made the branch; it 
thinks the branch comes from the initial rev. Easy enough to hack 
around, but might be nice to be able to do this using git-svn's history 
rewriting rather than a grafts file.

$ echo `git-svn find-rev r4` `git-svn find-rev r3 trunk` > .git/info/grafts
$ git reset --hard mybranch
$ git merge trunk

No conflicts now. Let's see what git-svn thinks it should do

$ git-svn dcommit -n

Looks like the right diff

$ git-svn dcommit

Refresh gitk display. Looks good, the new revision is a merge with the 
right parents. Let's check it out in svn land

$ cd ../svnclient
$ svn up
$ cat branches/mybranch/testfile1

Yep, the trunk change is there, nice! Now for a couple more revs with a 
conflict.

$ echo post-merge trunk change >> trunk/testfile1
$ svn commit -m "trunk change after merge"
$ echo post-merge conflicting change >> trunk/testfile2
$ svn commit -m "trunk change with conflict"
$ cd ../gitclone
$ git-svn fetch
$ git merge -m "change with conflict" trunk

Conflict, as expected

$ vi testfile2
$ git add testfile2
$ git commit
$ git-svn dcommit
Transaction is out of date: Out of date: '/trunk/testfile1' in 
transaction '9-1' at /Users/koreth/git/git-svn line 398

Hmm, this merge was in mybranch, not in trunk

$ git log --first-parent

Yes, the most recent commit with a git-svn-id line has a mybranch URL. 
So why is it complaining about a trunk file being out of date?

My experimentation pretty much ended there (I tried a few things to 
clear the error up, but none of them helped.)

This machine is an OS X laptop. Subversion is 1.4.3 (r23084) from 
MacPorts. I used the git-svn from the "pu" branch since it had this 
patch and all the recent fixes.

Let me know if you need more details. Hope this is helpful.

-Steve

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Eric Wong]

Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:
> > dcommit will still rewrite the HEAD commit and the history of the first
> > parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.
> > 
> > However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
> > preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
> > with dcommit.  Commits written to SVN will still not have any merge
> > information besides anything in the commit message.
> > 
> > Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
> > for explanations, feedback, examples and test case.
> > 
> > Signed-off-by: Eric Wong <normalperson@yhbt.net>
> > ---
> > 
> >  This is a better patch that replaces the previous one.
> > 
> >  Junio:
> >    This one is a big change and should probably sit in pu or next
> >    for a bit.  Double-checking the logic in linearize_history()
> >    would be greatly appreciated, too.
> >    
> >    I don't think there are any regressions for the
> >    already-linear-history case besides slightly reduced performance for
> >    new calls to cat-file.
> > 
> >  Joakim/Steven:
> >    Any further testing and test cases would be appreciated.  Be very
> >    careful with real-world repositories, and run dcommit with the
> >    '-n' flag before actually committing to verify the diffs are sane.
> > 
> >   Thanks
> > 
> 
> Did a little testing and so far it looks good :)
> 
> Sidenote:
> Doing this 
>   git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
>   git-svn fetch --quiet
> makes git svn fetch stop for rather long periods in do_update:
>   Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
>   Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
>   Following parent with do_update
>   Successfully followed parent
> 
> Is it possible to speed up do_update?

Use a do_switch()-enabled SVN to avoid do_update().  do_update will
redownload everything.  I have patched 1.4.3 debian packages with source
and a diff here: http://git-svn.bogomips.org/svn.  SVN 1.4.4 claims to
have fixed the bindings, but 1.4.3 claimed the same thing, too...
Confirmation of it working in SVN 1.4.4 would be nice.

> Lastly, when adding the above u-boot svn repo into a fresh u-boot
> clone from WD, can I attach the svn tree to git u-boot tree without
> using a graft?

Not with the current version.  The 1.5.0 (or previous, I forget) allowed
forced-parenting with: "git-svn fetch <rev>=<commit>" but I figured
nobody was using it, and it would be difficult to get working since
fetch can now works on multiple trees and the same revision numbers can
appear in multiple trees.

> I want to be able to recreate my own git repo by cloning the orginal
> u-boot repo and the svn repo.

-- 
Eric Wong

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Eric Wong]

Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> On Wed, 2007-06-13 at 19:13 +0200, Joakim Tjernlund wrote:
> > On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:
> > > dcommit will still rewrite the HEAD commit and the history of the first
> > > parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.
> > > 
> > > However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
> > > preserved when the new HEAD and HEAD~[0-9]+ commits are rewritten to SVN
> > > with dcommit.  Commits written to SVN will still not have any merge
> > > information besides anything in the commit message.
> > > 
> > > Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
> > > for explanations, feedback, examples and test case.
> > > 
> > > Signed-off-by: Eric Wong <normalperson@yhbt.net>
> > > ---
> > > 
> > >  This is a better patch that replaces the previous one.
> > > 
> > >  Junio:
> > >    This one is a big change and should probably sit in pu or next
> > >    for a bit.  Double-checking the logic in linearize_history()
> > >    would be greatly appreciated, too.
> > >    
> > >    I don't think there are any regressions for the
> > >    already-linear-history case besides slightly reduced performance for
> > >    new calls to cat-file.
> > > 
> > >  Joakim/Steven:
> > >    Any further testing and test cases would be appreciated.  Be very
> > >    careful with real-world repositories, and run dcommit with the
> > >    '-n' flag before actually committing to verify the diffs are sane.
> > > 
> > >   Thanks
> > > 
> > 
> > Did a little testing and so far it looks good :)
> > 
> > Sidenote:
> > Doing this 
> >   git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
> >   git-svn fetch --quiet
> > makes git svn fetch stop for rather long periods in do_update:
> >   Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
> >   Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
> >   Following parent with do_update
> >   Successfully followed parent
> > 
> > Is it possible to speed up do_update?
> > 
> > 
> > Lastly, when adding the above u-boot svn repo into a fresh u-boot clone from WD,
> > can I attach the svn tree to git u-boot tree without using a graft?
> > 
> > I want to be able to recreate my own git repo by cloning the orginal u-boot
> > repo and the svn repo.
> > 
> >  Jocke
> 
> Tried using --no-metadata(git svn clone --no-metadata) in my little test
> script I sent earlier and got
>   "Unable to determine upstream SVN information from HEAD history"
> when dcommiting, -i trunk didn't help either.
> 
> It is not entierly clear to me what --no-metadata means to me.
> Does git-svn still rewrite commits?

--no-metadata is really only useful for people doing one-shot imports
and abandoning SVN.  It leaves out the git-svn-id: lines at the bottom
of commit messages, but still sets the committer/author names/email/date
to what is in the SVN repository.

> I can't rebuild rev_db file, if lost, but I guess I could still
> do a new git-svn clone and restore my repo? I guess I lose something
> if I do that but what?

If you lose your rev_db file with no-metadata, you'll have to redo the
git-svn clone

> Also don't really understand why git-svn log doesn't work, can't it get
> that info from the svn repo?

Getting git-svn log working with --no-metadata would require radically
different code.  dcommit would be very different, too.  So yes, they
don't work because I'm lazy.

-- 
Eric Wong

RE: [PATCH] git-svn: allow dcommit to retain local merge information

[Joakim Tjernlund]

 

> -----Original Message-----
> From: Eric Wong [mailto:normalperson@yhbt.net] 
> Sent: den 20 juni 2007 08:56
> To: Joakim Tjernlund
> Cc: Junio C Hamano; Steven Grimm; git@vger.kernel.org
> Subject: Re: [PATCH] git-svn: allow dcommit to retain local 
> merge information
> 
> Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> > On Wed, 2007-06-13 at 02:23 -0700, Eric Wong wrote:
> > > dcommit will still rewrite the HEAD commit and the 
> history of the first
> > > parents of each HEAD~1, HEAD~2, HEAD~3 as it always has.
> > > 
> > > However, any merge parents (HEAD^2, HEAD^^2, HEAD~2^2) will now be
> > > preserved when the new HEAD and HEAD~[0-9]+ commits are 
> rewritten to SVN
> > > with dcommit.  Commits written to SVN will still not have 
> any merge
> > > information besides anything in the commit message.
> > > 
> > > Thanks to Joakim Tjernlund, Junio C Hamano and Steven Grimm
> > > for explanations, feedback, examples and test case.
> > > 
> > > Signed-off-by: Eric Wong <normalperson@yhbt.net>
> > > ---
> > > 
> > >  This is a better patch that replaces the previous one.
> > > 
> > >  Junio:
> > >    This one is a big change and should probably sit in pu or next
> > >    for a bit.  Double-checking the logic in linearize_history()
> > >    would be greatly appreciated, too.
> > >    
> > >    I don't think there are any regressions for the
> > >    already-linear-history case besides slightly reduced 
> performance for
> > >    new calls to cat-file.
> > > 
> > >  Joakim/Steven:
> > >    Any further testing and test cases would be 
> appreciated.  Be very
> > >    careful with real-world repositories, and run dcommit with the
> > >    '-n' flag before actually committing to verify the 
> diffs are sane.
> > > 
> > >   Thanks
> > > 
> > 
> > Did a little testing and so far it looks good :)
> > 
> > Sidenote:
> > Doing this 
> >   git-svn init -t tags -T trunk -b branches  
> file:///usr/local/src/tst-git-svn/svn-uboot-repo
> >   git-svn fetch --quiet
> > makes git svn fetch stop for rather long periods in do_update:
> >   Found possible branch point: 
> file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => 
> file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-ubo
> ot-1.1.4, 2
> >   Found branch parent: (tags/snap-uboot-1.1.4) 
> 81eef14963597cc99ba375f52e6d0b3bc09e25f8
> >   Following parent with do_update
> >   Successfully followed parent
> > 
> > Is it possible to speed up do_update?
> 
> Use a do_switch()-enabled SVN to avoid do_update().  do_update will
> redownload everything.  I have patched 1.4.3 debian packages 
> with source
> and a diff here: http://git-svn.bogomips.org/svn.  SVN 1.4.4 claims to
> have fixed the bindings, but 1.4.3 claimed the same thing, too...
> Confirmation of it working in SVN 1.4.4 would be nice.

Confirmed as requested, I installed 1.4.4(Gentoo) an ran the
same test case. Now I see "Following parent with do_switch"
instead and it is almost instant. It felt though that
git-svn was somewhat slower importing large diffs.

 Jocke

> 
> > Lastly, when adding the above u-boot svn repo into a fresh u-boot
> > clone from WD, can I attach the svn tree to git u-boot tree without
> > using a graft?
> 
> Not with the current version.  The 1.5.0 (or previous, I 
> forget) allowed
> forced-parenting with: "git-svn fetch <rev>=<commit>" but I figured
> nobody was using it, and it would be difficult to get working since
> fetch can now works on multiple trees and the same revision 
> numbers can
> appear in multiple trees.

If you reconsider this, please let me know.

 Jocke

RE: [PATCH] git-svn: allow dcommit to retain local merge information

[Joakim Tjernlund]

> -----Original Message-----
> From: Steven Grimm [mailto:koreth@midwinter.com] 
> Sent: den 14 juni 2007 08:31
> To: Eric Wong
> Cc: Junio C Hamano; Joakim Tjernlund; git@vger.kernel.org
> Subject: Re: [PATCH] git-svn: allow dcommit to retain local 
> merge information
> 
> Eric Wong wrote:
> >  Joakim/Steven:
> >    Any further testing and test cases would be appreciated.  Be very
> >    careful with real-world repositories, and run dcommit with the
> >    '-n' flag before actually committing to verify the diffs 
> are sane.
> >   
> 
> I poked at this some tonight with an eye toward the use case of using 
> git to merge svn branches. I ran into one inconvenience and one bug. 
> I'll try playing with the "use lots of git branches to develop on one 
> svn branch" use case some too, but for now, here are the 
> notes I took, 
> along with the commands if anyone wants to reproduce what I did. 
> Hopefully this won't be too annoying to read. The bug is near 
> the bottom.
> 
> svn repo with a trunk and a branch, each with changes (no 
> conflicts at 
> first, keep it simple to start)
> 
> $ svnadmin create svnrepo
> $ svn co file://`pwd`/svnrepo svnclient
> $ cd svnclient
> $ mkdir trunk tags branches
> $ echo test file number 1 > trunk/testfile1
> $ echo test file number 2 > trunk/testfile2
> $ svn add *
> $ svn commit -m "initial commit"
> $ echo trunk change 1 >> trunk/testfile1
> $ svn commit -m "trunk change 1"
> $ echo trunk change 2 >> trunk/testfile2
> $ svn commit -m "trunk change 2"
> $ svn cp trunk branches/mybranch
> $ svn commit -m "make a branch"
> $ echo trunk change 3 >> trunk/testfile1
> $ svn commit -m "post-branch change in trunk"
> $ echo branch change 1 >> branches/mybranch/testfile2
> $ svn commit -m "change in branch"
> 
> git-svn clone of this dinky repo
> 
> $ cd ..
> $ git-svn clone --trunk=trunk --branches=branches --tags=tags 
> file://`pwd`/svnrepo gitclone
> $ cd gitclone
> 
> Try to merge trunk change into branch using git
> 
> $ git reset --hard mybranch
> $ git merge trunk
> 
> Conflicts! what's going on?
> 
> $ gitk --all
> 
> Aha, looks like git-svn guessed wrong about where I made the 
> branch; it 
> thinks the branch comes from the initial rev. Easy enough to hack 
> around, but might be nice to be able to do this using 
> git-svn's history 
> rewriting rather than a grafts file.

Yes, that would be nice indeed.

> 
> $ echo `git-svn find-rev r4` `git-svn find-rev r3 trunk` > 
> .git/info/grafts
> $ git reset --hard mybranch
> $ git merge trunk
> 
> No conflicts now. Let's see what git-svn thinks it should do
> 
> $ git-svn dcommit -n
> 
> Looks like the right diff
> 
> $ git-svn dcommit
> 
> Refresh gitk display. Looks good, the new revision is a merge 
> with the 
> right parents. Let's check it out in svn land
> 
> $ cd ../svnclient
> $ svn up
> $ cat branches/mybranch/testfile1
> 
> Yep, the trunk change is there, nice! Now for a couple more 
> revs with a 
> conflict.
> 
> $ echo post-merge trunk change >> trunk/testfile1
> $ svn commit -m "trunk change after merge"
> $ echo post-merge conflicting change >> trunk/testfile2
> $ svn commit -m "trunk change with conflict"
> $ cd ../gitclone
> $ git-svn fetch
> $ git merge -m "change with conflict" trunk
> 
> Conflict, as expected
> 
> $ vi testfile2
> $ git add testfile2
> $ git commit
> $ git-svn dcommit
> Transaction is out of date: Out of date: '/trunk/testfile1' in 
> transaction '9-1' at /Users/koreth/git/git-svn line 398

Maybe this can help?
http://svn.haxx.se/subusers/archive-2005-02/0096.shtml
http://subclipse.tigris.org/faq.html#out-of-date

> 
> Hmm, this merge was in mybranch, not in trunk
> 
> $ git log --first-parent
> 
> Yes, the most recent commit with a git-svn-id line has a 
> mybranch URL. 
> So why is it complaining about a trunk file being out of date?
> 
> My experimentation pretty much ended there (I tried a few things to 
> clear the error up, but none of them helped.)
> 
> This machine is an OS X laptop. Subversion is 1.4.3 (r23084) from 
> MacPorts. I used the git-svn from the "pu" branch since it had this 
> patch and all the recent fixes.
> 
> Let me know if you need more details. Hope this is helpful.
> 
> -Steve
> 
> 

RE: [PATCH] git-svn: allow dcommit to retain local merge information

[Tjernlund]

> > $ echo post-merge trunk change >> trunk/testfile1
> > $ svn commit -m "trunk change after merge"
> > $ echo post-merge conflicting change >> trunk/testfile2
> > $ svn commit -m "trunk change with conflict"
> > $ cd ../gitclone
> > $ git-svn fetch
> > $ git merge -m "change with conflict" trunk
> > 
> > Conflict, as expected
> > 
> > $ vi testfile2
> > $ git add testfile2
> > $ git commit
> > $ git-svn dcommit
> > Transaction is out of date: Out of date: '/trunk/testfile1' in 
> > transaction '9-1' at /Users/koreth/git/git-svn line 398
> 
> Maybe this can help?
> http://svn.haxx.se/subusers/archive-2005-02/0096.shtml
> http://subclipse.tigris.org/faq.html#out-of-date

Could this be related to do_update vs. do_switch?
Latest subversion appears to have a working
do_switch function.   


Jocke

Re: [PATCH] git-svn: allow dcommit to retain local merge information

[Joakim Tjernlund]

> > Sidenote:
> > Doing this 
> >   git-svn init -t tags -T trunk -b branches  file:///usr/local/src/tst-git-svn/svn-uboot-repo
> >   git-svn fetch --quiet
> > makes git svn fetch stop for rather long periods in do_update:
> >   Found possible branch point: file:///usr/local/src/tst-git-svn/svn-uboot-repo/trunk => file:///usr/local/src/tst-git-svn/svn-uboot-repo/tags/snap-uboot-1.1.4, 2
> >   Found branch parent: (tags/snap-uboot-1.1.4) 81eef14963597cc99ba375f52e6d0b3bc09e25f8
> >   Following parent with do_update
> >   Successfully followed parent
> > 
> > Is it possible to speed up do_update?
> 
> Use a do_switch()-enabled SVN to avoid do_update().  do_update will
> redownload everything.  I have patched 1.4.3 debian packages with source
> and a diff here: http://git-svn.bogomips.org/svn.  SVN 1.4.4 claims to
> have fixed the bindings, but 1.4.3 claimed the same thing, too...
> Confirmation of it working in SVN 1.4.4 would be nice.

I upgraded to svn 1.4.4 and it seemed to work OK, but then i noticed
some problems, 
 git clone -t tags -T trunk -b branches svn+ssh://devsrv/svn/TM-uboot
failed(error msg was something with "Malformed data ...")

I decided to go back to 1.4.3 with you patch, switch-editor-perl.diff
applied. Now the error was gone, but the clone operation did use
do_update insted of do_switch. Did I miss someting?

 Jocke