What is the best way to backport a feature?

What is the best way to backport a feature?

[Peter Weseloh]

Hi,

Suppose I have the following situation:

  o--o--o                    Release_1.0
 /    \  \                  
o-o-o--o--o-o-o-o-o-o---o--o Mainline
     \       \       \ /    
      F1--F2--M1--F3--M2     Feature_A

Now I want to backport "Feature_A" to the "Release_1.0" branch so that it gets
included into the next minor release, i.e. I want to apply the commits F1, F2
and F3 onto the "Release_1.0" branch.
I cannot just merge "Feature_A" into "Release_1.0" because that would also bring
in the merges M1 and M2 so a lot of other stuff from the Mainline.

I played with cherry-pick but that means I have to manually find the commits F1,
F2 and F3 (which in reality could be many more if Feature_A is big) which is not
very nice.

I also tried 'rebase -i' but that means I have to manually delete all the lines
for changesets from the mainline. Also not very nice.

Is there a better way? To me this scenario sounds not unusual but I could not
find a solution.

Thanks,
Peter

Re: What is the best way to backport a feature?

[Björn Steinbrink]

On 2009.11.29 16:28:17 +0000, Peter Weseloh wrote:
> Suppose I have the following situation:
>
>   o--o--o                    Release_1.0
>  /    \  \
> o-o-o--o--o-o-o-o-o-o---o--o Mainline
>      \       \       \ /
>       F1--F2--M1--F3--M2     Feature_A
>
> Now I want to backport "Feature_A" to the "Release_1.0" branch so that
> it gets included into the next minor release, i.e. I want to apply the
> commits F1, F2 and F3 onto the "Release_1.0" branch.

> Is there a better way? To me this scenario sounds not unusual but I
> could not find a solution.

What's unusual there is that you merged from Mainline to Feature_A.
Usually, the history would look like this:

   o--o--o                    Release_1.0
  /    \  \
 o-o-o--o--o-o-o-o-o-o---o--o Mainline
      \                 /
       F1-----F2------F3      Feature_A

And then you could easily use rebase to get the job done.

Had you known beforehand that Feature_A is a candidate for backporting,
you would have even branch from an older commit like this:

   o--o--o                    Release_1.0
  /    \  \
 o-o-o--o--o-o-o-o-o-o---o--o Mainline
  \                     /
   F1--------F2-------F3      Feature_A

Then you could easily merge Feature_A to Release_1.0 as well, without
merging anything unrelated.

But that's just for the future...

Given you current history, you could use format-patch + am like this:

git format-patch --stdout --first-parent Mainline..Feature_A > fa.mbox
git checkout Release_1.0
git am -3 fa.mbox

The --first-parent options make it follow the first parent of the merge
commits only, so the whole stuff on the Mainline branch is ignored. And
you just get F1, F2 and F3 in fa.mbox, which you then apply using am.

A long time ago, I hacked the --first-parent thing into rebase, but (of
course) the first iteration of the patch wasn't quite perfect and as
I've not been scratching my own itch there, I never got around to
actually polish the patch so it could get into git.git. Maybe you want
to pick it up?

http://thread.gmane.org/gmane.comp.version-control.git/62782

Björn

Re: What is the best way to backport a feature?

[Pascal Obry]

Le 29/11/2009 17:47, Björn Steinbrink a écrit :
> What's unusual there is that you merged from Mainline to Feature_A.
> Usually, the history would look like this:

Right, I missed that! It is indeed very unusual at the point that I 
missed to read it properly :) So my reply is wrong.

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Fwd: What is the best way to backport a feature?

[Peter Weseloh]

Hi Björn,

First of all thank you very much for your quick reply (actually my
thanks go to all that have replied so far).
Note that at the moment it is just a brain exercise. We are currently
using CVS (yes, I know) and want to switch to something else and I'm
trying to push for git. During our discussions this scenario came up
and I could not give a simple answer. That's why I thought I'd better
ask the experts.


> What's unusual there is that you merged from Mainline to Feature_A.
> Usually, the history would look like this:
>
>   o--o--o                    Release_1.0
>  /    \  \
>  o-o-o--o--o-o-o-o-o-o---o--o Mainline
>      \                 /
>       F1-----F2------F3      Feature_A
>
> And then you could easily use rebase to get the job done.

But on the other hand the intermediate merges from the Mainline make
for much simpler merges, right?.
If merging is done only when Feature_A is ready it might become a real
pain. It might take several month to complete it and the mainline
might have changed a lot.

>
> Had you known beforehand that Feature_A is a candidate for backporting,
> you would have even branch from an older commit like this:
>
>   o--o--o                    Release_1.0
>  /    \  \
>  o-o-o--o--o-o-o-o-o-o---o--o Mainline
>  \                     /
>   F1--------F2-------F3      Feature_A
>
>
> Then you could easily merge Feature_A to Release_1.0 as well, without
> merging anything unrelated.
>
> But that's just for the future...
>

Yes, sure. If I would know the future already today I would not need
to do any coding anymore :-) But seriously our policy is clear:
Bugfixes (and small enhancements) go to the release branch to end up
in the next minor release. The release branch gets merged with the
mainline so that it is always a superset. Big new features are
developed in seperated branches and are finaly merged to the mainline
to end up in the next major release. But every now and then the
managment is so excited about such a new feature that they want it in
the next minor release. That's life.

>
> Given you current history, you could use format-patch + am like this:
>
> git format-patch --stdout --first-parent Mainline..Feature_A > fa.mbox
> git checkout Release_1.0
> git am -3 fa.mbox
>
> The --first-parent options make it follow the first parent of the merge
> commits only, so the whole stuff on the Mainline branch is ignored. And
> you just get F1, F2 and F3 in fa.mbox, which you then apply using am.
>

Ah, great! I played with format-patch + am but missed the
'--first-parent' option. I will give it a try. Thanks a lot!

>
> A long time ago, I hacked the --first-parent thing into rebase, but (of
> course) the first iteration of the patch wasn't quite perfect and as
> I've not been scratching my own itch there, I never got around to
> actually polish the patch so it could get into git.git. Maybe you want
> to pick it up?
>
> http://thread.gmane.org/gmane.comp.version-control.git/62782

In case we go for git this might very well be the case.
>
> Björn

Peter

Re: Fwd: What is the best way to backport a feature?

[Johannes Sixt]

[please keep the Cc list]

On Sonntag, 29. November 2009, Peter Weseloh wrote:
> But on the other hand the intermediate merges from the Mainline make
> for much simpler merges, right?.
> If merging is done only when Feature_A is ready it might become a real
> pain. It might take several month to complete it and the mainline
> might have changed a lot.

Incidentally, Junio has blogged about this just the other day:

http://gitster.livejournal.com/42247.html

Basically, as soon as you merge Mainline into Feature_A, you change the topic 
of Feature_A from "Feature for Release_1.0" to "Feature for this Mainline". 
Clearly, this topic is not suitable for Release_1.0 anymore.

There is a way around this that doesn't sacrifice the topic-oriented nature of 
the branch: You keep developing Feature_A as planned for Release_1.0 and when 
you notice that merging this feature to Mainline will become increasingly 
complex, you fork off a new branch Feature_A_for_Release_2.0 from Mainline 
and merge Feature_A into this new branch:

   o--o--o                    Release_1.0
  /    \  \
 o-o-o--o--o-o-o-o-X-o---o--o Mainline
      \             \
       F1            o--o     Feature_A_for_Release_2.0   
        \           /  /
         F2--------F3-F4      Feature_A

The fork point X must be in Release_2.0.

-- Hannes

Re: Fwd: What is the best way to backport a feature?

[Peter Weseloh]

2009/11/29 Johannes Sixt <j6t@kdbg.org>:
> [please keep the Cc list]
Sorry!
>
> http://gitster.livejournal.com/42247.html
>
> Basically, as soon as you merge Mainline into Feature_A, you change the topic
> of Feature_A from "Feature for Release_1.0" to "Feature for this Mainline".
> Clearly, this topic is not suitable for Release_1.0 anymore.
>
> There is a way around this that doesn't sacrifice the topic-oriented nature of
> the branch: You keep developing Feature_A as planned for Release_1.0 and when
> you notice that merging this feature to Mainline will become increasingly
> complex, you fork off a new branch Feature_A_for_Release_2.0 from Mainline
> and merge Feature_A into this new branch:
>
>   o--o--o                    Release_1.0
>  /    \  \
>  o-o-o--o--o-o-o-o-X-o---o--o Mainline
>      \             \
>       F1            o--o     Feature_A_for_Release_2.0
>        \           /  /
>         F2--------F3-F4      Feature_A
>
> The fork point X must be in Release_2.0.

That makes perfect sense. I will discuss your suggestion with my
colleagues and will send them the link you mentioned. It's just that
branching and especially merging with CVS is so painful that they
might get scared :-). With git that's completly different, of course.

Thanks a lot,
Peter

Re: What is the best way to backport a feature?

[Björn Steinbrink]

On 2009.11.29 18:41:35 +0100, Peter Weseloh wrote:
> >  What's unusual there is that you merged from Mainline to Feature_A.
> > Usually, the history would look like this:
> >
> >   o--o--o                    Release_1.0
> >  /    \  \
> >  o-o-o--o--o-o-o-o-o-o---o--o Mainline
> >      \                 /
> >       F1-----F2------F3      Feature_A
> >
> > And then you could easily use rebase to get the job done.
> 
> But on the other hand the intermediate merges from the Mainline make for
> much simpler merges, right?.
> If merging is done only when Feature_A is ready it might become a real pain.

That's usually more often true with CVS or SVN than with git, but ...

> It might take several month to complete it and the mainline might have
> changed a lot.

... over such a long timeframe, yes, things might become ugly. OTOH such
a long timeframe might also mean that the topic branch actually does too
much. Splitting such a large thing into more manageable pieces would
help there, as you could merge completed topic branch to your mainline
branch earlier and more often.

> > Had you known beforehand that Feature_A is a candidate for backporting,
> > you would have even branch from an older commit like this:
> >
> >   o--o--o                    Release_1.0
> >  /    \  \
> >  o-o-o--o--o-o-o-o-o-o---o--o Mainline
> >  \                     /
> >   F1--------F2-------F3      Feature_A
> >
> > Then you could easily merge Feature_A to Release_1.0 as well, without
> > merging anything unrelated.
> >
> > But that's just for the future...
> >
> Yes, sure. If I would know the future already today I would not need to do
> any coding anymore :-)

I meant something like "I just said that, so you can avoid problems in
the future" ;-) But yeah, knowing beforehand that things should go into
a maintenance branch isn't common, unless it's about a bugfix.

> > Given you current history, you could use format-patch + am like this:
> >
> > git format-patch --stdout --first-parent Mainline..Feature_A > fa.mbox
> > git checkout Release_1.0
> > git am -3 fa.mbox
> >
> > The --first-parent options make it follow the first parent of the merge
> > commits only, so the whole stuff on the Mainline branch is ignored. And
> > you just get F1, F2 and F3 in fa.mbox, which you then apply using am.
> >
> >
> Ah, great! I played with format-patch + am but missed the '--first-parent'
> option. I will give it a try. Thanks a lot!

Well, it's a rev-list option, which might work by accident. Junio
recently said that the fact that format-patch accepts path limiting is
by accident, might be true for --first-parent as well... No clue. Junio?

Björn

Re: What is the best way to backport a feature?

[Pascal Obry]

Le 29/11/2009 17:28, Peter Weseloh a écrit :
> Hi,
>
> Suppose I have the following situation:
>
>    o--o--o                    Release_1.0
>   /    \  \
> o-o-o--o--o-o-o-o-o-o---o--o Mainline
>       \       \       \ /
>        F1--F2--M1--F3--M2     Feature_A
>
> Now I want to backport "Feature_A" to the "Release_1.0" branch so that it gets
> included into the next minor release, i.e. I want to apply the commits F1, F2
> and F3 onto the "Release_1.0" branch.
> I cannot just merge "Feature_A" into "Release_1.0" because that would also bring
> in the merges M1 and M2 so a lot of other stuff from the Mainline.
>
> I played with cherry-pick but that means I have to manually find the commits F1,
> F2 and F3 (which in reality could be many more if Feature_A is big) which is not
> very nice.
>
> I also tried 'rebase -i' but that means I have to manually delete all the lines
> for changesets from the mainline. Also not very nice.
>
> Is there a better way? To me this scenario sounds not unusual but I could not
> find a solution.

In such a case I would use a rebase onto:

    $ git co Feature_A
    $ git rebase --onto Release_1.0 F1 F3

Then

    $ git co Release_1.0
    $ git merge Feature_A

Pascal.

-- 

--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Re: What is the best way to backport a feature?

[Michael J Gruber]

Peter Weseloh venit, vidit, dixit 29.11.2009 17:28:
> Hi,
> 
> Suppose I have the following situation:
> 
>   o--o--o                    Release_1.0
>  /    \  \                  
> o-o-o--o--o-o-o-o-o-o---o--o Mainline
>      \       \       \ /    
>       F1--F2--M1--F3--M2     Feature_A
> 
> Now I want to backport "Feature_A" to the "Release_1.0" branch so that it gets
> included into the next minor release, i.e. I want to apply the commits F1, F2
> and F3 onto the "Release_1.0" branch.
> I cannot just merge "Feature_A" into "Release_1.0" because that would also bring
> in the merges M1 and M2 so a lot of other stuff from the Mainline.
> 
> I played with cherry-pick but that means I have to manually find the commits F1,
> F2 and F3 (which in reality could be many more if Feature_A is big) which is not
> very nice.
> 
> I also tried 'rebase -i' but that means I have to manually delete all the lines
> for changesets from the mainline. Also not very nice.
> 
> Is there a better way? To me this scenario sounds not unusual but I could not
> find a solution.

The problem is that you've been a bad boy to begin with ;)

Seriously, I suggest reading up on "topic branches". Feature_A should
have been based off the common merge base of Mainline and Release_1.0,
and, even more importantly, there should not have been any merges from
Mainline into Feature_A. So, that branch is not at all what one would
call a feature branch/topic branch. Hopefully, this scenario is very
uncommon :)

I assume you have to deal with the given structure anyhow, and merge
will not help. The only solution is to try and replay your Feature_A
commits on top of the release branch. (Since you have merged Feature_A
into Mainline already, you probabably don't want to redo that branch and
merge.)

I you have many commits to deal with I suggest finding a good
semi-automated way to list the commits you are after, such as git
rev-list --no-merges sha1..Feature_A (with sha1 being the fork point). A
good way to find out could be git log --no-merges sha1..Feature_A.

Then, try and cherry-pick those onto the release branch. Alternatively,
you can use format-patch/am, or in fact try with rebase (I thought it
would ignore merges), which basically does what cherry-pick does.

Cheers,
Michael

Re: What is the best way to backport a feature?

[Greg A. Woods]

[-- Attachment #1: Type: text/plain, Size: 3457 bytes --]

Hmmm.... this topic seems in part to be very close to my thread
	 "git merge" merges too much!

At Sun, 29 Nov 2009 18:02:32 +0100, Michael J Gruber <git@drmicha.warpmail.net> wrote:
Subject: Re: What is the best way to backport a feature?
> 
> Seriously, I suggest reading up on "topic branches". Feature_A should
> have been based off the common merge base of Mainline and Release_1.0,
> and, even more importantly, there should not have been any merges from
> Mainline into Feature_A. So, that branch is not at all what one would
> call a feature branch/topic branch. Hopefully, this scenario is very
> uncommon :)

IFF I understand the original post correctly then actually in my
experience this scenario is VERY common in some environments!

This is exactly how large projects which use the likes of CVS (and SVN?)
manage longer-running development branches for important new features.

One excellent example is NetBSD (and perhaps the other BSDs too).

A developer creates a "working" branch from the trunk, then begins to
make changes and commits to that branch.  Periodically the (entire)
trunk is merged again to the working branch.  I think this is somewhat
equivalent to using "git rebase" to re-apply the feature branch to a new
fork point from the trunk.  However the actual branch base point remains
at the original point -- it is only the delta between the last merge
from trunk and the current head of the trunk which is merged onto the
feature working branch.  I think this is what people in the CVS world
mean when they say they want the tool to remember the point on the
source branch from where they did the last merge.  They've got their
work-flow "backwards", but this is the best they can do with CVS.

These periodic merges from the trunk mean that once the feature is
finished the delta between the trunk and the new feature branch is going
to be just the new feature, and so merging that delta alone to the trunk
as one commit adds the new feature to the trunk with few or no
conflicts, and the feature working branch can finally be "closed".

I'm guessing that people moving to Git from CVS may choose to stick with
this pattern where they periodically merge-from-master to keep
long-running feature branches as close to in-sync with the master branch
as possible (to avoid final merge conflicts).  Ideally, IIUC, perhaps
they should use rebase instead.

Perhaps this "mess" can indeed be cleaned up using "git rebase -i" so
that the final version of the feature branch can be back-ported more
easily (though one will still need to use git-cherry-pick or git-am to
do the back-port to the previous release branch).  The result of the
cleanup, before the merge of Feature_A to 1.0 might look more like this:

  o--o--o                                 Release_1.0
 /    \  \
o-o-o--o--o-o-o-o-o-o-o-o---------------o Mainline
                         \             /
                          F1'--F2'--F3'   Feature_A

and then after the merge of Feature_A to Release_1.0:

  o--o--o--F1''--F2''--F3''               Release_1.0
 /    \  \
o-o-o--o--o-o-o-o-o-o-o-o---------------o Mainline
                         \             /
                          F1'--F2'--F3'   Feature_A

-- 
						Greg A. Woods

+1 416 218-0098                VE3TCP          RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>      Secrets of the Weird <woods@weird.com>

[-- Attachment #2: Type: application/pgp-signature, Size: 186 bytes --]