Help breaking up a large merge.

Help breaking up a large merge.

[David Brown]

Say we have a tree that we've been working on for a few months.  An
outside vendor has also been working on the same tree during this
time, and we need to merge with their work.

The difficulty I'm having is that there are a lot of conflicts
resulting from the merge (expected), and it would be nice to somehow
be able to work on a smaller set of these conflicts at a time.

Some of the conflicts are caused by a single change in the other tree.
This is easy to cherry-pick into my tree, resolve, and then test those
changes independently.

But other conflicts are caused by groups of commits that are
interleaved with others.

Ideally, I'd like to be able to divide this conflict resolution work
up among a small group of people, have everybody work on part, test
their part, and then I could bring all of the resolved versions
together, test that, and make that the actual merge commit in our
repo.  I've had a few ideas, but none really seem to work all that
well.

   - Do "git merge -s ours their-tree".  Then, in another branch try
     the merge, and resolve a group of files that go together.  Put
     these into the 'ours' tree and ammend these changes to the merge
     commit.  The problem here is that this misses all of the merges
     that would happen automatically.

   - "git branch -b tmp $(git merge-base our-head their-tree)", and
     then: "git checkout their-tree filenames" for a small group of
     conflicting files.  Commit this, then in my regular tree, cherry
     pick this change and resolve the conflict.  Then, when finally
     doing the merge with their tree, these files can be resolved by
     just using our version.  This still requires doing some tracking.

Any other ideas?

In the future, we're going to try and work more closely with the
vendor, but we have to get to the point of having something common to
start that.

Thanks,
David

Re: Help breaking up a large merge.

[Santi Béjar]

On Thu, Sep 18, 2008 at 5:21 PM, David Brown <git@davidb.org> wrote:
> Say we have a tree that we've been working on for a few months.  An
> outside vendor has also been working on the same tree during this
> time, and we need to merge with their work.
>
> The difficulty I'm having is that there are a lot of conflicts
> resulting from the merge (expected), and it would be nice to somehow
> be able to work on a smaller set of these conflicts at a time.

If the two (or at least one) branches have sufficient isolated commits
you can recreate the merges that could have happened, as is explained
(for monotone) in:

http://www.venge.net/mtn-wiki/ZipperMerge

Another option is to rebase one branch onto the other.

Even another option is to merge the two branches but use the rebased
tree to resolve the conflicts.

They are not for parallel merge resolution, but at least you can do
them incrementally.

HTH,
Santi

Re: Help breaking up a large merge.

[David Brown]

On Thu, Sep 18, 2008 at 05:55:34PM +0200, Santi Béjar wrote:

>If the two (or at least one) branches have sufficient isolated commits
>you can recreate the merges that could have happened, as is explained
>(for monotone) in:
>
>http://www.venge.net/mtn-wiki/ZipperMerge
>
>Another option is to rebase one branch onto the other.

Either of these is likely to result in more work.  Both branches have
intermediate results, and there has been some communication between
the developers of each branch, so some things are closer at the two
tips than they would be at intermediate stages.

I started with this approach, and started realizing that I was
resolving similar conflicts repeatedly (not the same, or rerere could
have helped).

Basically, I'm looking for a way to break the merge effort up by
groups of files/directories.

Thanks,
David

Re: Help breaking up a large merge.

[Johannes Sixt]

David Brown schrieb:
> Say we have a tree that we've been working on for a few months.  An
> outside vendor has also been working on the same tree during this
> time, and we need to merge with their work.
> 
> The difficulty I'm having is that there are a lot of conflicts
> resulting from the merge (expected), and it would be nice to somehow
> be able to work on a smaller set of these conflicts at a time.
> 
> Some of the conflicts are caused by a single change in the other tree.
> This is easy to cherry-pick into my tree, resolve, and then test those
> changes independently.
> 
> But other conflicts are caused by groups of commits that are
> interleaved with others.

In a similar situation I was thinking about this approach:

1. Do the merge.
2. Resolve conflicts in an area that can be tested in isolation.
3. Undo all other changes that the merge brought in.
4. Commit.
5. Install a graft that removes the second parent of the merge commit.
6. Rinse and repeat.
7. Finally, remove the grafts, and perhaps collapse the merge commits.

I didn't test this, yet.

Hmm, thinking a bit more about this, 1 and 5 can probably be replaced by a
mere 'git merge --squash'.

-- Hannes

Re: Help breaking up a large merge.

[Avery Pennarun]

On Thu, Sep 18, 2008 at 11:21 AM, David Brown <git@davidb.org> wrote:
> The difficulty I'm having is that there are a lot of conflicts
> resulting from the merge (expected), and it would be nice to somehow
> be able to work on a smaller set of these conflicts at a time.
>
> Some of the conflicts are caused by a single change in the other tree.
> This is easy to cherry-pick into my tree, resolve, and then test those
> changes independently.

I've had the same sort of problem at work and I've gone through
several iterations trying to solve a problem.  The short version is
that all the solutions proposed here so far, plus some other ones I
thought of, don't seem to cut down the work for me :)

But here's one that has helped quite a bit, assuming that breaking up
the merge by *files* makes sense.

...

Let's say you have two branches, A and B, derived from a base X.  We
want to merge the branch with fewer changes on top of the branch with
more changes, because it'll be less work to divide up :)  Let's assume
the branch with fewer changes is A.

1) Generate a giant patch file using 'git diff X..A'.  Call the patch P.

2) Using an editor, divide up the patch by files/subdirs based on how
you want to subdivide the work.  Or alternatively, do that with 'git
diff' itself, but beware of accidentally forgetting to ask for some
files if you do it that way.  Call these n individual patches P1..Pn.

3) Create new branches called A1..An, copied directly from X.

4) Apply each patch P1..Pn to each branch A1..An.  (They will all
apply cleanly, because they are all patches against X in the first
place.)

5) Create new branches called B1..Bn, copied directly from *B*.

6) 'git merge' A1 into B1, A2 into B2, and so on.  Resolve the conflicts.

7) Create a new branch, TEST, copied from B.  Do an octopus merge from
B1..Bn.  There will be no conflicts, because those branches all make
mutually exclusive changes, and they're all based on B.  You now have
a combined branch containing A+B, but the history from A is missing.

8) Create a new patch, PFINAL, using 'git diff B..TEST'.  This is the
complete set of changes to turn B into A+B.

9) Checkout B.  'git merge A'; there will be conflicts.  'git checkout
HEAD .' to go back to B's files. Patch in PFINAL, and commit.

Now you have a single merge commit with all the changes, and the
history will be correct.

10) Optional: 'git merge B1..Bn' so that you don't lose the history of
the individual sub-merges (you might want to look at them later or use
them for blame purposes).  There shouldn't be any conflicts, as the
changes in those branches are identical to the changes you just
committed, and git will discard them in the extra merge.

10b) Optional advanced-only trick: amend the main commit so that it
looks like an octopus merge of B, A, and B1..Bn, instead of having a
separate fake merge commit.

Note that this also helps a lot (for me) even if it's just a single
person doing the merge: I like to do the library changes first, get
all the library unit tests passing, then move up to bigger and bigger
components.

Hope this helps.

Have fun,

Avery

Re: Help breaking up a large merge.

[David Brown]

On Thu, Sep 18, 2008 at 01:40:55PM -0400, Avery Pennarun wrote:

>1) Generate a giant patch file using 'git diff X..A'.  Call the patch P.
>
>2) Using an editor, divide up the patch by files/subdirs based on how
>you want to subdivide the work.  Or alternatively, do that with 'git
>diff' itself, but beware of accidentally forgetting to ask for some
>files if you do it that way.  Call these n individual patches P1..Pn.

The is fairly close to the approach I've been taking, but somehow I
didn't think of using the output of diff itself and hacking that up.
I like this because it makes it easier to not forget some files.

It is likely there will be makefiles to cleanup and such, but
otherwise I like this idea.

Thanks,
David