Maintaining two branches.

Maintaining two branches.

[David Brown]

Looking for some advice/ideas on a git workflow:

We have three branches of the code:

   - upstream - The upstream release versions, tracks outside git repo.
   - external - Other external patches not included in the main git repo.
   - local - Our local development.

For release reasons, we need to keep our local branch separate, but normal
development needs to be done on a merge of 'external' and 'local' (the tree
needs the merge of both just to build).  Developers will generate patches,
and maintainers will apply these patches to 'local'.

I've tried creating a 'next' branch where I merge each change from local
and/or external, and this seems to work, but the history gets very
cluttered with merge commits.  For some reason, history simplification
doesn't seem to eliminate any of these merges.

We also have an automated build/test system that checks out versions of
'next', and when the build and test moves 'master' forward to that version.
Because of this, it would be nice to keep 'next' fairly up-to-date, which
implies having a lot of merges.

Does anyone have any better ideas on how to maintain this process?  The
'next' and 'master' branches won't be exported too far, so could be
replaced with simpler history occasionally, but there is a good number of
developers who will be using them, so it would be best to not do this too
often.

Thanks,
David Brown

Re: Maintaining two branches.

[Stephan Beyer]

Hi,

> I've tried creating a 'next' branch where I merge each change from local
> and/or external, and this seems to work, but the history gets very
> cluttered with merge commits.

You could cherry-pick commits from local and/or external instead of
merging. See git-cherry-pick(1).

Regards,
  Stephan

-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F

Re: Maintaining two branches.

[Pieter de Bie]

On 3 jun 2008, at 18:34, David Brown wrote:

> We have three branches of the code:
>
>  - upstream - The upstream release versions, tracks outside git repo.
>  - external - Other external patches not included in the main git  
> repo.
>  - local - Our local development.
>
> For release reasons, we need to keep our local branch separate, but  
> normal
> development needs to be done on a merge of 'external' and  
> 'local' (the tree
> needs the merge of both just to build).  Developers will generate  
> patches,
> and maintainers will apply these patches to 'local'.

You might do the same workflow that Git has with stable / master / next

If there is a new upstream release, merge it into external. If you  
have patches you want to show to the outside, apply those patches to  
external. Then you can merge external into local. The trick is to  
never merge local into external.

By going only one way (upstream --> external --> local), you'll never  
have to worry about having to separate the different patches. Your  
history will be displayed much nicer too.

- Pieter

Re: Maintaining two branches.

[David Brown]

On Tue, Jun 03, 2008 at 08:02:13PM +0200, Stephan Beyer wrote:
>Hi,
>
>> I've tried creating a 'next' branch where I merge each change from local
>> and/or external, and this seems to work, but the history gets very
>> cluttered with merge commits.
>
>You could cherry-pick commits from local and/or external instead of
>merging. See git-cherry-pick(1).

The problem is that this makes future merges more difficult, since there
isn't a history between the trees.  Our 'local' branch really is a merge of
the two.

Once I cherry pick anything, that branch has divergent history from the
other branches, and I have to cherry pick from then on (or merge from then
on).

David

Re: Maintaining two branches.

[David Brown]

On Tue, Jun 03, 2008 at 08:08:09PM +0200, Pieter de Bie wrote:

> You might do the same workflow that Git has with stable / master / next
>
> If there is a new upstream release, merge it into external. If you have 
> patches you want to show to the outside, apply those patches to external. 
> Then you can merge external into local. The trick is to never merge local 
> into external.
>
> By going only one way (upstream --> external --> local), you'll never have 
> to worry about having to separate the different patches. Your history will 
> be displayed much nicer too.

I guess I didn't explain our dilema very well.  We _have_ to separate the
different patches, for legal reasons.  Perhaps 'external' isn't a good name
for the branch, maybe it should be 'other'.  Basically, the 'upstream'
branch is the real upstream tree.  The 'external' or 'other' branch
contains patches from outside our company.  We are forbidden from
redistributing these changes, and will be having our customers get them
from the same source that we do.  Then our 'local' branch is where we do
our development.

But, we can only build the system when the 'external/other' branch is
combined with 'local'.  It's just that the history has to be easily
separable so that we can generate the patches that are on the 'local'
branch along with the descriptions of the external branch so the customers
can get the same changes.

Thanks,
David

Re: Maintaining two branches.

[Dirk Süsserott]

David Brown schrieb:
> On Tue, Jun 03, 2008 at 08:08:09PM +0200, Pieter de Bie wrote:
>
>> You might do the same workflow that Git has with stable / master / next
>>
>> If there is a new upstream release, merge it into external. If you 
>> have patches you want to show to the outside, apply those patches to 
>> external. Then you can merge external into local. The trick is to 
>> never merge local into external.
>>
>> By going only one way (upstream --> external --> local), you'll never 
>> have to worry about having to separate the different patches. Your 
>> history will be displayed much nicer too.
>
> I guess I didn't explain our dilema very well.  We _have_ to separate the
> different patches, for legal reasons.  Perhaps 'external' isn't a good 
> name
> for the branch, maybe it should be 'other'.  Basically, the 'upstream'
> branch is the real upstream tree.  The 'external' or 'other' branch
> contains patches from outside our company.  We are forbidden from
> redistributing these changes, and will be having our customers get them
> from the same source that we do.  Then our 'local' branch is where we do
> our development.
For my understanding: does your 'other' branch contain only a 3rd party
library that you (or the 3rd party people) did patch for your purpose
and you're not allowed to distribute that patched 3rd party lib? What
about using a git submodule for the 3rd party thing. The submodule
contains the patched library (for your development) and the customer
is then told to supply the genuine library (from the internet) and then
apply your patches to it. Would that help you? Probably I missed
some legal issues here...

     -- Dirk

Re: Maintaining two branches.

[Junio C Hamano]

David Brown <git@davidb.org> writes:

> Looking for some advice/ideas on a git workflow:
>
> We have three branches of the code:
>
>   - upstream - The upstream release versions, tracks outside git repo.
>   - external - Other external patches not included in the main git repo.
>   - local - Our local development.
>
> For release reasons, we need to keep our local branch separate, but normal
> development needs to be done on a merge of 'external' and 'local' (the tree
> needs the merge of both just to build).  Developers will generate patches,
> and maintainers will apply these patches to 'local'.
>
> I've tried creating a 'next' branch where I merge each change from local
> and/or external, and this seems to work, but the history gets very
> cluttered with merge commits.  For some reason, history simplification
> doesn't seem to eliminate any of these merges.
>
> We also have an automated build/test system that checks out versions of
> 'next', and when the build and test moves 'master' forward to that version.
> Because of this, it would be nice to keep 'next' fairly up-to-date, which
> implies having a lot of merges.

You do not have to, and very likely do not want to, merge 'next' directly
into 'master'.

Instead of having a single local integration branch 'local', it is wiser
to keep separate topic branches to house your local development.  Each of
them will start its life in an imperfect shape, and will need multiple
rounds of fixups, which is normal, and you will merge them regularly into
'next' to make sure everything will work together, and 'next' will have
tons of merges for this exact reason.

However, what you will merge to 'master' is not 'next' as a whole.
Instead, you will merge each individual topic, only it is fully cooked and
proved to be Ok while it has been in 'next', to 'master'.  That way, if a
work to enhance an appliation area took four attempts to get it right over
four 'next' cycles, 'next' branch may have four merges from that topic
branch, you will see only one merge from that topic to 'master', that
merges the enhancement in completed state.  At that point, 'next' will
have other topics in an earlier stage of polishing, and these imperfect
ones will not in 'master' yet.

IOW, just like you keep separate things separate by having distinction
between 'external' and 'local', by keeping separate development tracks
among what you currently use single basket 'local' to hold, your history
will become much more meaningful and understandable.