interactive rebase results across shared histories

interactive rebase results across shared histories

[Seb]

Hello,

I've recently learnt how to consolidate and clean up the master branch's
commit history.  I've squashed/fixuped many commits thinking these would
propagate to the children branches with whom it shares the earlier parts
of the its history.  However, this is not the case; switching to the
child branch still shows the non-rebased (dirty) commit history from
master.  Am I misunderstanding something with this?

Thanks,

-- 
Seb

Re: interactive rebase results across shared histories

[Moritz Neeb]

Hi Seb,

On 02/20/2016 11:58 PM, Seb wrote:
> Hello,
> 
> I've recently learnt how to consolidate and clean up the master branch's
> commit history.  I've squashed/fixuped many commits thinking these would
> propagate to the children branches with whom it shares the earlier parts
> of the its history.  However, this is not the case; switching to the
> child branch still shows the non-rebased (dirty) commit history from
> master.  Am I misunderstanding something with this?

I am not sure what you meand by "child branch". If I understand
corretly, you have something like:

            A---B---C topic
           /
      D---E---F---G master

then you merge the topic:

            A---B---C topic
           /         \
      D---E---F---G---H master

and then you do something like "git rebase -i E" to linearize history
and maybe squash some commits, to result in something like:

      D---E---F---G---AB'---C' master

Where AB' is a squashed commit containing the changes from A and B.

Now, your misunderstanding may be in the fact of "what happened to the
topic branch?". Because looking at the whole graph, you have something
like this:

          A---B---C topic
         /
    D---E---F---G---AB'---C' master

where it is important to note, that the topic still points to C. Which
is totally correct, because you did not say anything about topic after
the merge. If you wanted to continue working on the topic branch, then
maybe a non-interactive rebasing, like described in the rebase manpage
would be something you might want to do before rebasing. E.g., from the
start doing "git rebase master topic" leads to:

                     A'--B'--C' topic
                    /
       D---E---F---G master

and then you could squash your commits as you like with "git rebase -i G":

	      AB'--C' topic
             /
D---E---F---G master

and maybe fast-forward merging master with "git merge master", then you
have both branches pointing to C':

    D---E---F---G---AB'--C' topic,master

The same could've been reached in one step via "git rebase -i master topic".

Maybe, to get a better understanding, you could use visualization tool
like "tig" or "gitk" to observe what happens to your commits (hashes)
and branches (labels) and just play around with some of these operations.

Re: interactive rebase results across shared histories

[Seb]

On Sun, 21 Feb 2016 03:12:49 +0100,
Moritz Neeb <lists@moritzneeb.de> wrote:

> Hi Seb,
> On 02/20/2016 11:58 PM, Seb wrote:
>> Hello,

>> I've recently learnt how to consolidate and clean up the master
>> branch's commit history.  I've squashed/fixuped many commits thinking
>> these would propagate to the children branches with whom it shares
>> the earlier parts of the its history.  However, this is not the case;
>> switching to the child branch still shows the non-rebased (dirty)
>> commit history from master.  Am I misunderstanding something with
>> this?

> I am not sure what you meand by "child branch". If I understand
> corretly, you have something like:

>             A---B---C topic
>            /
>       D---E---F---G master

Thanks Moritz and sorry for not adequately describing the situation.
The scenario is much simpler; imagine master has a longer history behind
the point where the topic branch started:

                A---B---C topic
               /
  *---D---E---F---G master

And we want to keep both branches separate (no desire to merge them for
now), but we realize that, say, commits D and E should be
squashed/fixup, so we do an interactive rebase.  Now, the problem is
that if I do that from the topic branch, the results are not reflected
in the master branch, even though these commits are certainly shared
with master.  It seems counterintuitive that a part of history that is
shared among branches can be independently manipulated/rewritten with
rebase.  I must be missing something...


-- 
Seb

Re: interactive rebase results across shared histories

[Eric Sunshine]

On Sun, Feb 21, 2016 at 12:25 PM, Seb <spluque@gmail.com> wrote:
> The scenario is much simpler; imagine master has a longer history behind
> the point where the topic branch started:
>
>                 A---B---C topic
>                /
>   *---D---E---F---G master
>
> And we want to keep both branches separate (no desire to merge them for
> now), but we realize that, say, commits D and E should be
> squashed/fixup, so we do an interactive rebase.  Now, the problem is
> that if I do that from the topic branch, the results are not reflected
> in the master branch, even though these commits are certainly shared
> with master.  It seems counterintuitive that a part of history that is
> shared among branches can be independently manipulated/rewritten with
> rebase.  I must be missing something...

What you're probably missing is that you can't actually edit commits
in Git. Instead, what you think of as "editing" actually creates a new
commit with its own commit-ID, and the original commit still exists
with its own commit-ID. Since Git commits are chained together by
their commit-ID's, any commits pointing at the original commit-ID
continue to point to that commit, and only commits rebased atop the
new commit-ID of the "edited" commit point at it.

In your example, you're "editing" D and E, which creates new commits
D' and E', so your resulting graph looks like this:

    D'---E'---A---B---C topic
   /
  *---D---E---F---G master

So, "master" and "topic" really are not sharing D and E (or D' and
E'). You could "fix" this to match your intuition by rebasing F...G
onto E' (see git-rebase --onto, for instance), which would give you
this:

                  A---B---C topic
                 /
  *---D'---E'---F---G master

and then "master" and "topic" would really be sharing D' and E' as
common history. (Of course, rebasing "master" or any branch may not be
desirable if you've published it, so applicable warnings about
rebasing apply.)

By the way, the problem isn't restricted to when you rebase "topic"
(as your problem description implies). You'd see the same behavior if
you'd rebased D and E in "master" to become D' and E'. "topic" would
still have old D and E in its history, and not D' and E'.

Re: interactive rebase results across shared histories

[Seb]

On Sun, 21 Feb 2016 14:08:44 -0500,
Eric Sunshine <sunshine@sunshineco.com> wrote:

[...]

> What you're probably missing is that you can't actually edit commits
> in Git. Instead, what you think of as "editing" actually creates a new
> commit with its own commit-ID, and the original commit still exists
> with its own commit-ID. Since Git commits are chained together by
> their commit-ID's, any commits pointing at the original commit-ID
> continue to point to that commit, and only commits rebased atop the
> new commit-ID of the "edited" commit point at it.

> In your example, you're "editing" D and E, which creates new commits
> D' and E', so your resulting graph looks like this:

>     D'---E'---A---B---C topic / *---D---E---F---G master

> So, "master" and "topic" really are not sharing D and E (or D' and
> E'). You could "fix" this to match your intuition by rebasing F...G
> onto E' (see git-rebase --onto, for instance), which would give you
> this:

>                   A---B---C topic / *---D'---E'---F---G master

> and then "master" and "topic" would really be sharing D' and E' as
> common history. (Of course, rebasing "master" or any branch may not be
> desirable if you've published it, so applicable warnings about
> rebasing apply.)

> By the way, the problem isn't restricted to when you rebase "topic"
> (as your problem description implies). You'd see the same behavior if
> you'd rebased D and E in "master" to become D' and E'. "topic" would
> still have old D and E in its history, and not D' and E'.

Thank you for this excellent explanation.  I will look into git rebase
--onto to have both branches share the history that should be shared
after the interactive rebase.  Fortunately, this is a private repository
for now.

-- 
Seb

Re: interactive rebase results across shared histories

[David]

On 21 February 2016 at 13:12, Moritz Neeb <lists@moritzneeb.de> wrote:
>
> On 02/20/2016 11:58 PM, Seb wrote:
>>
>> I've recently learnt how to consolidate and clean up the master branch's
>> commit history.  I've squashed/fixuped many commits thinking these would
>> propagate to the children branches with whom it shares the earlier parts
>> of the its history.  However, this is not the case; switching to the
>> child branch still shows the non-rebased (dirty) commit history from
>> master.  Am I misunderstanding something with this?
>
> [snip]
>
> Maybe, to get a better understanding, you could use visualization tool
> like "tig" or "gitk" to observe what happens to your commits (hashes)
> and branches (labels) and just play around with some of these operations.

 Seb,

If you have a gui environment, the command
  gitk --all
will show you diagrams that will complement the explanations
others have given you here. You must specify --all to see more
than one branch.

And if you give a branch two names before rebasing one of those
names, then you will easily be able to compare before and after.

Re: interactive rebase results across shared histories

[Seb]

On Sun, 21 Feb 2016 03:12:49 +0100,
Moritz Neeb <lists@moritzneeb.de> wrote:

> Hi Seb,
> On 02/20/2016 11:58 PM, Seb wrote:
>> Hello,

>> I've recently learnt how to consolidate and clean up the master
>> branch's commit history.  I've squashed/fixuped many commits thinking
>> these would propagate to the children branches with whom it shares
>> the earlier parts of the its history.  However, this is not the case;
>> switching to the child branch still shows the non-rebased (dirty)
>> commit history from master.  Am I misunderstanding something with
>> this?

> I am not sure what you meand by "child branch". If I understand
> corretly, you have something like:

[...]

> Maybe, to get a better understanding, you could use visualization tool
> like "tig" or "gitk" to observe what happens to your commits (hashes)
> and branches (labels) and just play around with some of these
> operations.

OK, I've followed this advice and looked at the dependency graphs in
gitk before and after rebasing, I've managed to obtain what I was
after.  The repository now has two branches: master and topic.  However,
Gitk reveals a problem with a string of commits that are not part of any
branch:

A---B---H---I                   (master)
     \
      C---D---E                 (loose string of commits)
       \
        D'---E'---F---G         (topic)

How do I remove these loose commits (C, D, E)?

Thanks for your feedback,

-- 
Seb

Re: interactive rebase results across shared histories

[Moritz Neeb]

On 02/23/2016 06:39 PM, Seb wrote:
> On Sun, 21 Feb 2016 03:12:49 +0100,
> Moritz Neeb <lists@moritzneeb.de> wrote:
> 
>> Hi Seb,
>> On 02/20/2016 11:58 PM, Seb wrote:
>>> Hello,
> 
>>> I've recently learnt how to consolidate and clean up the master
>>> branch's commit history.  I've squashed/fixuped many commits thinking
>>> these would propagate to the children branches with whom it shares
>>> the earlier parts of the its history.  However, this is not the case;
>>> switching to the child branch still shows the non-rebased (dirty)
>>> commit history from master.  Am I misunderstanding something with
>>> this?
> 
>> I am not sure what you meand by "child branch". If I understand
>> corretly, you have something like:
> 
> [...]
> 
>> Maybe, to get a better understanding, you could use visualization tool
>> like "tig" or "gitk" to observe what happens to your commits (hashes)
>> and branches (labels) and just play around with some of these
>> operations.
> 
> OK, I've followed this advice and looked at the dependency graphs in
> gitk before and after rebasing, I've managed to obtain what I was
> after.  The repository now has two branches: master and topic.  However,
> Gitk reveals a problem with a string of commits that are not part of any
> branch:
> 
> A---B---H---I                   (master)
>      \
>       C---D---E                 (loose string of commits)
>        \
>         D'---E'---F---G         (topic)
> 
> How do I remove these loose commits (C, D, E)?
>

what you might be after is "git gc". But I never used it, it was not
neccesary for me. I would let the automatic garbage collection drop my
dangling commits. It's safer - who knows when you will still want to
restore your recent "loose string of commits".

How exactly are the loose commits causing trouble?

-Moritz

Re: interactive rebase results across shared histories

[Kevin Daudt]

On Tue, Feb 23, 2016 at 11:57:06PM +0100, Moritz Neeb wrote:
> On 02/23/2016 06:39 PM, Seb wrote:
> > On Sun, 21 Feb 2016 03:12:49 +0100,
> > Moritz Neeb <lists@moritzneeb.de> wrote:
> > 
> >> Hi Seb,
> >> On 02/20/2016 11:58 PM, Seb wrote:
> >>> Hello,
> > 
> >>> I've recently learnt how to consolidate and clean up the master
> >>> branch's commit history.  I've squashed/fixuped many commits thinking
> >>> these would propagate to the children branches with whom it shares
> >>> the earlier parts of the its history.  However, this is not the case;
> >>> switching to the child branch still shows the non-rebased (dirty)
> >>> commit history from master.  Am I misunderstanding something with
> >>> this?
> > 
> >> I am not sure what you meand by "child branch". If I understand
> >> corretly, you have something like:
> > 
> > [...]
> > 
> >> Maybe, to get a better understanding, you could use visualization tool
> >> like "tig" or "gitk" to observe what happens to your commits (hashes)
> >> and branches (labels) and just play around with some of these
> >> operations.
> > 
> > OK, I've followed this advice and looked at the dependency graphs in
> > gitk before and after rebasing, I've managed to obtain what I was
> > after.  The repository now has two branches: master and topic.  However,
> > Gitk reveals a problem with a string of commits that are not part of any
> > branch:
> > 
> > A---B---H---I                   (master)
> >      \
> >       C---D---E                 (loose string of commits)
> >        \
> >         D'---E'---F---G         (topic)
> > 
> > How do I remove these loose commits (C, D, E)?
> >
> 
> what you might be after is "git gc". But I never used it, it was not
> neccesary for me. I would let the automatic garbage collection drop my
> dangling commits. It's safer - who knows when you will still want to
> restore your recent "loose string of commits".
> 
> How exactly are the loose commits causing trouble?
> 

Right, you should normally not care about these commits. In any case,
the reflog is still keeping a reference to these commits, and objects
not older then 14 days are also kept.

But git is regularly running its gc to clean up older objects, and they
will not be shared when pushing.

Re: interactive rebase results across shared histories

[Seb]

On Tue, 23 Feb 2016 23:57:06 +0100,
Moritz Neeb <lists@moritzneeb.de> wrote:

[...]

>> OK, I've followed this advice and looked at the dependency graphs in
>> gitk before and after rebasing, I've managed to obtain what I was
>> after.  The repository now has two branches: master and topic.
>> However, Gitk reveals a problem with a string of commits that are not
>> part of any branch:

>> A---B---H---I (master) \ C---D---E (loose string of commits) \
>> D'---E'---F---G (topic)

>> How do I remove these loose commits (C, D, E)?


> what you might be after is "git gc". But I never used it, it was not
> neccesary for me. I would let the automatic garbage collection drop my
> dangling commits. It's safer - who knows when you will still want to
> restore your recent "loose string of commits".

> How exactly are the loose commits causing trouble?

Sure enough, these dangling commits were removed automatically without
any intervention.  All is good.

Thanks!

-- 
Seb

Re: interactive rebase results across shared histories

[David]

On 24 February 2016 at 10:05, Seb <spluque@gmail.com> wrote:
> On Tue, 23 Feb 2016 23:57:06 +0100,
> Moritz Neeb <lists@moritzneeb.de> wrote:
>
> [...]
>
>>> OK, I've followed this advice and looked at the dependency graphs in
>>> gitk before and after rebasing, I've managed to obtain what I was
>>> after.  The repository now has two branches: master and topic.
>>> However, Gitk reveals a problem with a string of commits that are not
>>> part of any branch:
>
>>> A---B---H---I (master) \ C---D---E (loose string of commits) \
>>> D'---E'---F---G (topic)
>
>>> How do I remove these loose commits (C, D, E)?
>
>
>> what you might be after is "git gc". But I never used it, it was not
>> neccesary for me. I would let the automatic garbage collection drop my
>> dangling commits. It's safer - who knows when you will still want to
>> restore your recent "loose string of commits".
>
>> How exactly are the loose commits causing trouble?
>
> Sure enough, these dangling commits were removed automatically without
> any intervention.  All is good.

This discussion could end there without problem. But if you want to
understand a little more thoroughly, read on ...

First, for basic git use, please stop being concerned about when or whether
dangling commits are removed or not. For basic git use, it is unimportant
(except for advanced use like disaster recovery).

By default, git waits a while (couple of weeks I think?) and then removes
dangling commits silently.

Why remove them? Because dangling commits are commits that git has
been instructed to forget about and discard, because no reference
(like a branch or tag) points to them.

And because they are unimportant, gitk chooses not to show them;
whenever you refresh or restart gitk, it won't show any dangling commits.

Why does git wait before discarding them entirely? Because advanced
users might want to do something clever with them.

So your dangling commits (after rebase) are probably not removed for a
couple of weeks. If you were to somehow recreate a reference to them,
then gitk will show them again. But otherwise it treats them as unworthy
of being shown.

So how did you ever see the dangling commits? I think it is because gitk
was aware of them because it had a reference to them before the rebase.
So if you keep gitk open it still shows them afterwards even though the
reference is gone. And if I recall correctly, if you refresh or restart gitk,
they will disappear from its display.

Re: interactive rebase results across shared histories

[Seb]

On Fri, 26 Feb 2016 23:38:38 +1100,
David <bouncingcats@gmail.com> wrote:

> On 24 February 2016 at 10:05, Seb <spluque@gmail.com> wrote:
>> On Tue, 23 Feb 2016 23:57:06 +0100,
>> Moritz Neeb <lists@moritzneeb.de> wrote:

>> [...]

>>>> OK, I've followed this advice and looked at the dependency graphs
>>>> in gitk before and after rebasing, I've managed to obtain what I
>>>> was after.  The repository now has two branches: master and topic.
>>>> However, Gitk reveals a problem with a string of commits that are
>>>> not part of any branch:

>>>> A---B---H---I (master) \ C---D---E (loose string of commits) \
>>>> D'---E'---F---G (topic)

>>>> How do I remove these loose commits (C, D, E)?


>>> what you might be after is "git gc". But I never used it, it was not
>>> neccesary for me. I would let the automatic garbage collection drop
>>> my dangling commits. It's safer - who knows when you will still want
>>> to restore your recent "loose string of commits".

>>> How exactly are the loose commits causing trouble?

>> Sure enough, these dangling commits were removed automatically
>> without any intervention.  All is good.

> This discussion could end there without problem. But if you want to
> understand a little more thoroughly, read on ...

Thanks David, I appreciate the insight.  Indeed, I've learnt a lot over
the last few days with help in this thread as I confronted a lurking
problem after many years neglecting it.  Briefly, long ago I was
developing a project in RCS, then on CVS and SVN, until some years ago I
imported it into git via cvs2svn.  I had turned a blind eye to a bit of
mess up to the very early releases, likely due to my inexperience but
also differences between VCS.

After cleaning up all the mess, I've ended up with a long master branch,
and a series of earlier commits that are not reachable from master.
Fortunately, the tags have kept them alive. This is the scenario
simplified:

A---C---D(tag2)                 loose commits (not on any branch)
 \
  B(tag1)

E---F---G---H---*               (master)

I could put the "loose" (but tagged) commits on a branch at "tag2", but
I hate that "tag1" shows as a twig there...  It would be nice to have
all the history reachable from master.  So two questions I'm working on
right now: 1) how to bring "tag1" into the "tag2" chain of commits, and
then 2) how to tie it all together into master so that it reads
linearly.

-- 
Seb

Re: interactive rebase results across shared histories

[Stepan Kasal]

Hello Seb,

let me add a few things, perhaps they'll clean some misunderstandings.

On Fri, Feb 26, 2016 at 03:12:46PM -0600, Seb wrote:
> After cleaning up all the mess, I've ended up with a long master branch,
> and a series of earlier commits that are not reachable from master.
> Fortunately, the tags have kept them alive. This is the scenario
> simplified:
> 
> A---C---D(tag2)                 loose commits (not on any branch)
>  \
>   B(tag1)
> 
> E---F---G---H---*               (master)

I noticed that these are tags.  Actually, tags in git are meant to mark
certain fixed points, and they are not meant to change in the future.

They are used to mark releases in the project, and they can even have
a cryptographic signature attached to it, so that no one can forge them.

I guess that you used them as a temporary mark for yourself when you
reaced a point in development, or to mark a stable point as a base for
future development.  If that was the case, you could have used a branch
as well.

I guess that people subconsciously still think that branch is ecpensive,
based on the experiences from other VCS.  It's not true: branch is
a variable that holds a value: the hex id of the top commit.
As you do not care how many variables you need to declare when writing
code, you should not hesitate to create as many branches as you need
to mark the points in the history.

Perhaps you just delte the tags.  That would be true if the code
in these old branches has already been copied (by rebase) to the new
master.

If you would like to mark the corresponding points in the rewritten
history, you can create these marks again.  (As branches, perhaps.)

Then you say that you want to put everything to one branch master,
to create a linear history of development.  (Well, not true history,
but a made up history, you see.)

For that goal, does master already contain all the code?
If yes, then you have it.  You can perhaps just delete all the old
branches that were used to develop individual parts...

I'm working on a project, where we decided to have linear history
on the main server.
So my own local git clone of my work contains lots of branches named
after features and their combination, like:
    counting
    counting-3counters
    pairs
    counting-with-pairs
    counting-3c-pairs

I use rebase heavily to move each of the feature (=series of commits)
to another branch ot to master.

For example, when i wanted to enhance branch "counting" with the feature
"pairs," I did it like this:

git branch counting-with-pairs pairs
git rebase -i counting counting-with-pairs

That will take all the commits that are in pairs, but not in counting.
(This is usually all the commits in "pairs", staring from the point
where in branched from master.) And all these get replayed on the tip of
"counting".

Likewise, I can move the topic to the latest master, then move master
(git checkout master; git merge --ff-only some-branch)
and then push.

BTW, another thing:
when you decided to clean up the history -- are there any other
people working with the repository you are working with?

If yes, then each of them has to be prepared to follow your cleanup.
All their work is currently base on the old history; if they push
their work all the old hostory will be back.

They could throw away their current local clones and start anew,
if they had no work there.  But that's probaably not the case.

So they have to fetch your new history of master and then rebase
all of their features --onto your new master.

I hope some of my talking will be useful.

Stepan