Re: Trying to sync two svn repositories with git-svn (repost)

[Avery Pennarun <apenwarr@gmail.com>]

On Wed, May 13, 2009 at 6:22 PM, Josef Wolf <jw@raven.inka.de> wrote:
> Thanks for your patience, Avery!  I would be completely lost here without
> your help..

Okay, well, I think I've been making things worse instead of better :(

Fundamentally, my claim that merging symmetrically between two svn
branches "ought to be easy" was incorrect.  The problem is that when
git does a merge from branch A to B and then back to A, it really
*really* wants the two branches to end up identical.  All of git's
merge machinery works on the assumption that this is what you want.

Now, you can still bypass this by using various clever tricks.  The
solution we were working with turned out to be *almost* right, and I
did get it working in my test environment, but it got so convoluted
that I couldn't even explain to myself why it was correct.  That's
usually a bad sign.  So I threw that one away.

By far the sanest thing you could possibly do is to create a central
"public" branch that contains all the common commits, then merge from
that public branch to the site-specific branches, but never merge in
the opposite direction.  In case you happen to make some changes on
the site-specific branches that you want to share, you can just
cherry-pick them; the resulting conflicts when merging back are likely
to be fairly minor.  This would be entirely consistent with git's
normal operations, and would be easy:

   git checkout public
   git cherry-pick stuff   # as rarely as possible; do the work
directly on public if you can

   git checkout svn-1
   git merge --no-ff public
   git svn dcommit

   git checkout svn-2
   git merge --no-ff public
   git svn dcommit

No criss-cross merges, no insanity, no question about whether it's correct.

More as an academic exercise than anything, I did find a way that will
let you do criss-cross merging of all changes on A and B.  I still
don't *really* recommend you use it, because it's extremely error
prone, and there are lots of places where you could get merge
conflicts and then end up in trouble.  (The above simple method, in
contrast, might get conflicts sometimes, but you can just fix them as
you encounter them and be done with it.)

The script below demonstrates how to take branches remote-ab and
remote-ac, and auto-pick their changes (as they happen) into a new
(automatically managed) branch public.  Then it merges public back
into each branch, while avoiding conflicts.  The magic itself happens
in extract() and crossmerge().

If nothing else, this method makes the gitk output far more sane than
the original method.  This is because it doesn't include the history
of 'public' in the site-specific branches.  That was the fundamental
flaw in the method I had identified originally.  You can trick that
original method into working too, but it's stunningly complex.  This
is much more sane, albeit still not really sane.

Enjoy!

Have fun,

Avery

P.S. Sorry for the mess.  I suppose I should have broken down and
written (or asked for :)) a minimal test case earlier, as it quickly
revealed the problem.


#!/bin/bash -x
set -e
rm -rf tt
mkdir tt
cd tt
git init

count=100
newfile()
{
	count=$(($count + 1))
	echo $count >$1
	git add $1
	git commit -m "$1"
}

newfile .gitignore
git checkout -b public
newfile a
git checkout -b remote-ab public
newfile b
git checkout -b remote-ac public
newfile c

# We've simulated two remote branches (perhaps svn repositories), remote-ab
# and remote-ac.  They contain one identical file (a) and one different file
# (b vs. c).  We've arranged for the common part to end up in 'public'.

git tag -f remote-ab-lastmerge remote-ab
git tag -f remote-ac-lastmerge remote-ac

extract()
{
	last_public="$(git merge-base $1-lastmerge public)"
	if false; then
		# use this if you want each patch separately
		git branch -f $1-public $1
		git checkout $1-public
		git rebase --onto "$last_public" $1-lastmerge
	else
		# use this if you want changes squashed into one patch
		git branch -f $1-public "$last_public"
		git checkout $1-public
		git diff --binary $1-lastmerge $1 -- | git apply --index
		(
			echo "merged from $1"
			echo
			git log $1-lastmerge..$1
		) | git commit -F -
	fi
	
	git checkout $1
	git merge -s ours -m 'no-op' $1-public
}

crossmerge()
{
	branches="remote-ab remote-ac"
	
	for b in $branches; do
		# extract the most recent changes from $b-lastmerge..$b
		# The changes can be found as public..$b-public
		extract $b
		
		# Merge those completed changes into public
		git checkout public
		git merge $b-public
		git branch -d $b-public
		git tag -f $b-lastmerge $b  # to reduce problems if this script dies
halfway through
	done
	
	# merge changes from public back into each branch.
	# changes that originated in each branch won't be re-merged, because
	# we already merged back $b-public into each $b.
	for b in $branches; do
		git checkout $b
		git merge --no-ff public
		git tag -f $b-lastmerge $b
	done
}

iterate()
{
	# Some changes have arrived in the remote repos:
	#git svn fetch ab
	#git svn fetch ac
	git checkout remote-ab
	newfile x$1
	git checkout remote-ac
	newfile y$1

	crossmerge
}

iterate 1
git tag remote-ab-1 remote-ab
git tag remote-ac-1 remote-ac

iterate 2
git tag remote-ab-2 remote-ab
git tag remote-ac-2 remote-ac

iterate 3

gitk --all