git rebase behaviour changed?

git rebase behaviour changed?

[Mike McCormack]

Hi,

git-rebase was a very useful tool for me to help organize my patches.

Recently, it seems the behaviour of git-rebase changed.  It used to take 
the commits I'd made to my "master" branch and reapply them to a new 
"master" branch on top of "origin".  Where rebase from git-0.99.x used 
to work, git-1.1.3 now does nothing and gives me the following message:

git-rebase origin
-> Current branch refs/heads/master is up to date.

However, I can do the "rebase" manually with:

git branch master-20060117
git reset --hard origin
git-format-patch -k --stdout --full-index origin master-20060117 | \
	git am --binary -3 -k

Is this broken, or am I meant to be doing something different now?

thanks,

Mike

Re: git rebase behaviour changed?

[Junio C Hamano]

Mike McCormack <mike@codeweavers.com> writes:

> git-rebase origin
> -> Current branch refs/heads/master is up to date.
>
> However, I can do the "rebase" manually with:
>
> git branch master-20060117
> git reset --hard origin
> git-format-patch -k --stdout --full-index origin master-20060117 | \
> 	git am --binary -3 -k
>
> Is this broken, or am I meant to be doing something different now?

What does "git-merge-base master-20060117 origin" give you?  If
it is the same as "origin", then the master-20060117 has been
merged with origin, and rebase does not run in this case.

Here is the simplest example:

                  1---2---3---4 master
                 /
        origin  0

Of course, you _could_ extract patches #1, #2, #3, and #4
between origin and master, and apply them on top of #0 to
reconstruct "master" as you found out, but there is not much
point doing so.

Rebase changes the "master" branch when the development track
between you (master) and upstream (origin) have forked:

                  1---2---3---4 master
                 /
        origin' 0---5---6 origin

In this case, things are rearranged by rebase:

                        1'--2'--3'--4' master
                       /
        origin' 0--5--6 origin


End of on-topic answers.


BTW, what this means is that it would not rearrange something
like this:

                    2---3
                   /     \
                  1---4---5---6 master
                 / 
        origin  0

But a structure like this could be rebased:

                    2---3
                   /     \
                  1---4---5---6 master
                 / 
        origin' 0---7---8 origin

to produce something like this:

                          1'--2'--3'--4'--6' master
                         / 
        origin' 0---7---8 origin

The ordering of patches may turn out to be wrong; #4 might
conflict with already applied #2 and #3.  In general, rebasing
such a merged structure is highly discouraged.  I think there
was a discussion on this topic on the list recently, and a short
summary was: "if you do a merge, do not rebase; if you are going
to rebase, do not merge".  The thread is this one:

	http://thread.gmane.org/gmane.comp.version-control.git/14308

Especially please look at a couple of message from Linus:

	http://article.gmane.org/gmane.linux.kernel/365410
        http://article.gmane.org/gmane.linux.kernel/365409
        http://article.gmane.org/gmane.linux.kernel/365501

I guess we could decompose the commit ancestry chain in such a
case, and reproduce something like this:

                            2'--3'
                           /     \
                          1'--4'--5'--6' master
                         / 
        origin' 0---7---8 origin

Rebase has never done this, though.  It is left as an exercise
for the reader ;-).

Re: git rebase behaviour changed?

[Mike McCormack]

Junio C Hamano wrote:

> Rebase changes the "master" branch when the development track
> between you (master) and upstream (origin) have forked:
> 
>                   1---2---3---4 master
>                  /
>         origin' 0---5---6 origin

Well, I thought I was in the above situation, but it seems that "origin" 
has been merged into "master" :/

The "pull, rebase, commit, commit, send patches, pull, ..." strategy 
used to work for me, but now it doesn't.

 > summary was: "if you do a merge, do not rebase; if you are going
 > to rebase, do not merge".  The thread is this one:

I want to do rebases.  So is it that behaviour of "git pull" that has 
been changed to do merges, and I should be using "fetch" instead of 
"pull" or something similar?

Mike


btw. I'm not the only person having this problem.  Others using the same 
commands, and upgrading GIT have run into it too, so something has 
changed...

Re: git rebase behaviour changed?

[Junio C Hamano]

Mike McCormack <mike@codeweavers.com> writes:

> btw. I'm not the only person having this problem.  Others using the
> same commands, and upgrading GIT have run into it too, so something
> has changed...

Sorry, I accidentally removed the part from my message where I
said "Yes it was changed on Nov 28 after discussion on the list
regarding rebase".

	$ git whatchanged -S'is up to date' git-rebase.sh

would show that commit.

Re: git rebase behaviour changed?

[Martin Langhoff]

On 1/17/06, Mike McCormack <mike@codeweavers.com> wrote:
>  > summary was: "if you do a merge, do not rebase; if you are going
>  > to rebase, do not merge".  The thread is this one:
>
> I want to do rebases.  So is it that behaviour of "git pull" that has
> been changed to do merges, and I should be using "fetch" instead of
> "pull" or something similar?

Now, I have realised that a simple mistake (merging from origin in you
scenario) would lead git-rebase to discard earlier patches during the
rebase. If you had a single commit *after* the merge, git-rebase would
have rebased that single patch, and dropped earlier patches.

git-rebase should refuse to run in the above scenario. Is there a
straightforward way to ask if the merge base is "shared"?

<thinking>
If the commit following right after the merge base on "our" side is a
merge commit, there's a good chance we're about to fuck up. To make
double sure, we can walk up that commit to the other parent (the one
that is not the merge base for the current merge) and get what merge
base between that commit and the current merge base. If it returns
anything interesting, we bail out if we are conservative -- or walk up
the history again if we take a more adventurous approach.

If it returns empty, it's a splice merge and get outta here -- you are
not supposed to rebase a splice merge. And if the merge after our
original merge base was an octopus, die too. Those are not rebasable
either.
</thinking>

So, the conservative (and easy) approach would be to make rebase bail
out when it finds any merge commit.

cheers,


martin

Re: git rebase behaviour changed?

[Junio C Hamano]

Martin Langhoff <martin.langhoff@gmail.com> writes:

> Now, I have realised that a simple mistake (merging from origin in you
> scenario) would lead git-rebase to discard earlier patches during the
> rebase. If you had a single commit *after* the merge, git-rebase would
> have rebased that single patch, and dropped earlier patches.

It may not necessarily be a mistake.

> git-rebase should refuse to run in the above scenario. Is there a
> straightforward way to ask if the merge base is "shared"?
>
> <thinking>
>...
> </thinking>

Sorry I am always slow but I am a bit slower than I usually am
tonight, and do not understand this part without an
illustration:

        master    1---2---3---4---5---A
                 /           /
	origin  0---6---7---B


                A = master head
                B = origin head == merge base

	rev-list B..A = 1 2 3 4 5
        rev-list A..B = 6 7

The first rev-list is "what we have but they do not".  They are
the candidates to be fed upstream.  The latter is "what they
have but we do not".  Potentially some of them are commit that
represent patches we submitted earlier upstream.

Among the first list of commit, there is #4 which is a merge.
So we reject.  Is that what you meant?  Which makes sense in
this picture (but I am a bit tired and maybe this may not apply
in a different picture).


By the way, the longer I think about this, the more I agree with
the conclusion of the earlier thread: "if you rebase, do not
merge; if you merge, do not rebase".  It is really about picking
the right workflow.

Let's say you submitted #1, #2, #3 earlier, and #3 was accepted
upstream and came back as #7, and let's further assume that we
are lucky enough that patch-id gives the same answer for
"diff-tree #2" and "diff-tree #7".  So the set of commits left
are #1, #3, and #5 (#4 is just a merge so we will not re-apply).

Now, what is the shape of the final "rebased" ancestry graph we
would want?


        master                1'--3'--5'--A'
                             /
	origin  0---6---7---B

If this is what we want, why did we make #4 merge in the first
place, I wonder.  If the workflow is based on rebase [*1*],
instead of making a merge at #4, the developer *should* have
done fetch and rebase, not merge:

        master    1---2---3
                 /
	origin  0---6---7---B

        ==>

        master                1'--3'
                             /
	origin  0---6---7---B

This would have been easier to manage at the point we discovered
#6, #7, and #B, than creating #4 merge only to discard it later.

And #5 and #A can be built on top of #3'.

        master                1'--3'--5---A
                             /
	origin  0---6---7---B


[Footnote]

*1* That is certainly easier to manage for an individual
developer than a merge based workflow.  I know it because I
operated that way for a long time back when Linus was managing
git)

Re: git rebase behaviour changed?

[Martin Langhoff]

On 1/17/06, Junio C Hamano <junkio@cox.net> wrote:
> Sorry I am always slow but I am a bit slower than I usually am
> tonight, and do not understand this part without an
> illustration:


My fault. I did have a few bits of paper here on my lap, but gmail's
textbox sucks at ascii art...

>
>         master    1---2---3---4---5---A
>                  /           /
>         origin  0---6---7---B
>
>
>                 A = master head
>                 B = origin head == merge base
>
>         rev-list B..A = 1 2 3 4 5
>         rev-list A..B = 6 7

Yep, exacly the example I was thinking about.

(...)
> Among the first list of commit, there is #4 which is a merge.
> So we reject.  Is that what you meant?

Exactly. We refuse to reset the head and begin the rebase operation,
because it looks like operator error.

> By the way, the longer I think about this, the more I agree with
> the conclusion of the earlier thread: "if you rebase, do not
> merge; if you merge, do not rebase".  It is really about picking
> the right workflow.

Definitely. But errors and misunderstandings are frequent, and people
who haven't thought the process through or aren't that familiar with
the internals are very likely to try it.

Refusing to rebase, with a good error msg gives the user a chance to
evaluate what to do with the commits. Right now, if I read the
situation right, there's a good chance commits 1,2 and 3 will be
"lost" once the rebase is complete.

GIT won't literally lose them,  someone could run git-fsck and fish
out the dangling heads from the repo, and after a bit of spelunking
recover them, but it's suddenly a really tricky operation.

cheers,


martin

Re: git rebase behaviour changed?

[Junio C Hamano]

Martin Langhoff <martin.langhoff@gmail.com> writes:

> GIT won't literally lose them,  someone could run git-fsck and fish
> out the dangling heads from the repo, and after a bit of spelunking
> recover them, but it's suddenly a really tricky operation.

"git-lost-found".

You are right.  We will lose #1 and #2, (although the "already
up to date" might catch some cases) and this _is_ dangerous.  I
need to do something about this soon.

Thanks for the discussion.


[Footnote]

*1* ... or #1 and #3 --- sorry, my "one of them picked up by
upstream" scenario description was inconsistent in the previous
message.

Re: git rebase behaviour changed?

[Junio C Hamano]

Junio C Hamano <junkio@cox.net> writes:

> You are right.  We will lose #1 and #2, (although the "already
> up to date" might catch some cases) and this _is_ dangerous.  I
> need to do something about this soon.

Actually, I think we are OK; I do not think we would lose any
commits.  git-format-patch (actually, git-cherry called from
there) does the right thing.  It does not use the merge base
done in git-rebase in any way.

In any case, we _do_ need an explanation and error-out upon
finding a merge, as we discussed.  If somebody really wants to
rebase a merge, he can do that by hand, as Mike easily
demonstrated.