[BUG] rebase no longer omits local commits

[BUG] rebase no longer omits local commits

[Ted Felix]

[-- Attachment #1: Type: text/plain, Size: 825 bytes --]

Starting with git 1.9.0, rebase no longer omits local commits that 
appear in both the upstream and local branches.

I've bisected this down to commit bb3f458: "rebase: fix fork-point with 
zero arguments".  The attached script reproduces the problem.  Reverting 
the aforementioned commit fixes the problem.

A failed run of this script will result in conflicts.  A successful run 
against master with bb3f458 reverted ends as follows:

 From /tmp/rebase-issue/maint
    fe401cd..955af04  master     -> origin/master
fatal: Not a valid object name: ''
First, rewinding head to replay your work on top of it...
Applying: Third change

(I'm not sure if that "fatal: Not a valid object name: ''" is of any 
concern.  It started appearing for me at some point during the bisect.)

Let me know if there's more I can do to help.


[-- Attachment #2: rebase-issue.sh --]
[-- Type: text/x-shellscript, Size: 1665 bytes --]

#!/bin/bash

# git-rebase is supposed to drop commits that it finds in both the 
# local and upstream branches.  As of 1.9.0, this isn't happening.
# This script reproduces the problem.

# I've bisected the issue down to commit bb3f458.  Reverting that commit
# solves the problem.

# Run this in a directory where you have create privs.
# At the end, if there are conflicts, then the test has failed.

# Create a repo.
mkdir rebase-issue
cd rebase-issue
mkdir maint
cd maint
git init

# Create a README file and put some text in it
echo "Hi there!" >README
git add README
git commit -a -m "Initial commit"

# Clone the repo for "dev"
cd ..
git clone maint dev

# Dev makes *two* changes to the *same* area.
cd dev
# edit something, make some typos
echo "Freekwently Mispeled Werdz" >README
git commit -a -m "First change"
# edit same thing, fix those typos
echo "Frequently Misspelled Words" >README
git commit -a -m "Second change"

# Create patches to send to maintainer...
git format-patch -M origin/master
mv *.patch ../maint

# Add a third change that should make it through for completeness.
echo "Frequently Misspelled Words version 2" >README
git commit -a -m "Third change"

# We have to sleep (to make sure the times do not match?).
# If we don't, this script will succeed on fast machines.
# This can probably be reduced to 2 which should guarantee that
# the seconds will turn over on the clock.
echo
echo "Waiting 5 seconds to make sure apply time is different from patch time..."
sleep 5

echo
echo "Maint applies patches..."
cd ../maint
git am -3 *.patch

echo
echo "Dev does the fetch/rebase..."
cd ../dev
git fetch
git rebase

echo
git --version

Re: [BUG] rebase no longer omits local commits

[John Keeping]

On Thu, Jul 03, 2014 at 11:14:26AM -0400, Ted Felix wrote:
> Starting with git 1.9.0, rebase no longer omits local commits that 
> appear in both the upstream and local branches.
> 
> I've bisected this down to commit bb3f458: "rebase: fix fork-point with 
> zero arguments".  The attached script reproduces the problem.  Reverting 
> the aforementioned commit fixes the problem.
> 
> A failed run of this script will result in conflicts.  A successful run 
> against master with bb3f458 reverted ends as follows:
> 
>  From /tmp/rebase-issue/maint
>     fe401cd..955af04  master     -> origin/master
> fatal: Not a valid object name: ''
> First, rewinding head to replay your work on top of it...
> Applying: Third change
> 
> (I'm not sure if that "fatal: Not a valid object name: ''" is of any 
> concern.  It started appearing for me at some point during the bisect.)

It is the problem that bb3f458 fixes.  The change in behaviour is
actually introduced by ad8261d (rebase: use reflog to find common base
with upstream).

In your example, I think this is working as designed.  You can restore
the previous behaviour either with `git rebase --no-fork-point` or with
`git rebase @{u}`.

The change is designed to help users recover from an upstream rebase, as
described in the "DISCUSSION ON FORK-POINT MODE" section of
git-merge-base(1) and makes `git rebase` match the behaviour of
`git pull --rebase` so that:

	git fetch &&
	git rebase

really is equivalent to:

	git pull --rebase

Re: [BUG] rebase no longer omits local commits

[John Keeping]

On Thu, Jul 03, 2014 at 08:09:17PM +0100, John Keeping wrote:
> On Thu, Jul 03, 2014 at 11:14:26AM -0400, Ted Felix wrote:
> > Starting with git 1.9.0, rebase no longer omits local commits that 
> > appear in both the upstream and local branches.
> 
> It is the problem that bb3f458 fixes.  The change in behaviour is
> actually introduced by ad8261d (rebase: use reflog to find common base
> with upstream).
> 
> In your example, I think this is working as designed.  You can restore
> the previous behaviour either with `git rebase --no-fork-point` or with
> `git rebase @{u}`.
> 
> The change is designed to help users recover from an upstream rebase, as
> described in the "DISCUSSION ON FORK-POINT MODE" section of
> git-merge-base(1).

Having thought about this a bit more, I think the case you've identified
is an unexpected side effect of that commit.

Perhaps we shuld do something like this (which passes the test suite):

-- >8 --
diff --git a/git-rebase.sh b/git-rebase.sh
index 06c810b..0c6c5d3 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -544,7 +544,8 @@ if test "$fork_point" = t
 then
 	new_upstream=$(git merge-base --fork-point "$upstream_name" \
 			"${switch_to:-HEAD}")
-	if test -n "$new_upstream"
+	if test -n "$new_upstream" &&
+	   ! git merge-base --is-ancestor "$new_upstream" "$upstream_name"
 	then
 		upstream=$new_upstream
 	fi
-- 8< --

Since the intent of `--fork-point` is to find the best starting point
for the "$upstream...$orig_head" range, if the fork point is behind the
new location of the upstream then should we leave the upstream as it
was?

I haven't thought through this completely, but it seems like we should
be doing a check like the above, at least when we're in
"$fork_point=auto" mode.

Re: [BUG] rebase no longer omits local commits

[Junio C Hamano]

John Keeping <john@keeping.me.uk> writes:

> Perhaps we shuld do something like this (which passes the test suite):
>
> -- >8 --
> diff --git a/git-rebase.sh b/git-rebase.sh
> index 06c810b..0c6c5d3 100755
> --- a/git-rebase.sh
> +++ b/git-rebase.sh
> @@ -544,7 +544,8 @@ if test "$fork_point" = t
>  then
>  	new_upstream=$(git merge-base --fork-point "$upstream_name" \
>  			"${switch_to:-HEAD}")
> -	if test -n "$new_upstream"
> +	if test -n "$new_upstream" &&
> +	   ! git merge-base --is-ancestor "$new_upstream" "$upstream_name"
>  	then
>  		upstream=$new_upstream
>  	fi
> -- 8< --
>
> Since the intent of `--fork-point` is to find the best starting point
> for the "$upstream...$orig_head" range, if the fork point is behind the
> new location of the upstream then should we leave the upstream as it
> was?

Probably; but the check to avoid giving worse fork-point should be
in the implementation of "merge-base --fork-point" itself, so that
we do not have to do the above to both "rebase" and "pull --rebase",
no?

> I haven't thought through this completely, but it seems like we should
> be doing a check like the above, at least when we're in
> "$fork_point=auto" mode.

Re: [BUG] rebase no longer omits local commits

[John Keeping]

On Mon, Jul 07, 2014 at 10:56:23AM -0700, Junio C Hamano wrote:
> John Keeping <john@keeping.me.uk> writes:
> 
> > Perhaps we shuld do something like this (which passes the test suite):
> >
> > -- >8 --
> > diff --git a/git-rebase.sh b/git-rebase.sh
> > index 06c810b..0c6c5d3 100755
> > --- a/git-rebase.sh
> > +++ b/git-rebase.sh
> > @@ -544,7 +544,8 @@ if test "$fork_point" = t
> >  then
> >  	new_upstream=$(git merge-base --fork-point "$upstream_name" \
> >  			"${switch_to:-HEAD}")
> > -	if test -n "$new_upstream"
> > +	if test -n "$new_upstream" &&
> > +	   ! git merge-base --is-ancestor "$new_upstream" "$upstream_name"
> >  	then
> >  		upstream=$new_upstream
> >  	fi
> > -- 8< --
> >
> > Since the intent of `--fork-point` is to find the best starting point
> > for the "$upstream...$orig_head" range, if the fork point is behind the
> > new location of the upstream then should we leave the upstream as it
> > was?
> 
> Probably; but the check to avoid giving worse fork-point should be
> in the implementation of "merge-base --fork-point" itself, so that
> we do not have to do the above to both "rebase" and "pull --rebase",
> no?

I don't think so, since in that case we're not actually finding the fork
point as defined in the documentation, we're finding the upstream rebase
wants.

Having played with this a bit, I think we shouldn't be replacing the
upstream with the fork point but should instead add the fork point as an
additional negative ref:

	$upstream...$orig_head ^$fork_point

Here's a script that creates a repository showing this:

-- >8 --
#!/bin/sh
git init rebase-test &&
cd rebase-test &&
echo one >file &&
git add file &&
git commit -m one &&
echo frist >file2 &&
git add file2 &&
git commit -m first &&
git branch --track dev &&
echo first >file2 &&
git commit -a --amend --no-edit &&
echo two >file &&
git commit -a -m two &&
echo three >file &&
git commit -a -m three &&
echo second >file2 &&
git commit -a -m second &&
git checkout dev &&
git cherry-pick -2 master &&
echo four >file &&
git commit -a -m four &&
printf '\nWithout fork point (old behaviour)\n' &&
git rev-list --oneline --cherry @{u}... &&
printf '\nFork point as upstream (current behaviour)\n' &&
git rev-list --oneline --cherry $(git merge-base --fork-point master HEAD)... &&
printf '\nWith fork point\n' &&
git rev-list --oneline --cherry @{u}... ^$(git merge-base --fork-point master HEAD)
-- 8< --

In this case the rebase should be clean since the only applicable patch
changes "three" to "four" in "file", but the current rebase code fails
both with `--fork-point` and with `--no-fork-point`.

With `--fork-point` we try to apply "two" and "three" which have already
been cherry-picked across (as Ted originally reported) and with
`--no-fork-point`, we try to apply "first" which conflicts because we
have the version prior to it being fixed up on master.

I hacked up git-rebase to test this and the change to use the fork point
as in the last line of the script above does indeed make the rebase go
through cleanly, but I have not yet looked at how to cleanly patch in
that behaviour.

I haven't tested git-pull, but it looks like it has always (since 2009)
behaved in the way `git rebase --fork-point` does now, failing to detect
cherry-picked commits that are now in the upstream.

[PATCH 1/2] rebase--am: use --cherry-pick instead of --ignore-if-in-upstream

[John Keeping]

When using `git format-patch --ignore-if-in-upstream` we are only
allowed to give a single revision range.  In the next commit we will
want to add an additional exclusion revision in order to handle fork
points correctly, so convert `git-rebase--am` to use a symmetric
difference with `--cherry-pick --right-only`.

This does not change the result of the format-patch invocation, just how
we spell the arguments.

Signed-off-by: John Keeping <john@keeping.me.uk>
---
 git-rebase--am.sh | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/git-rebase--am.sh b/git-rebase--am.sh
index ca20e1e..902bf2d 100644
--- a/git-rebase--am.sh
+++ b/git-rebase--am.sh
@@ -29,7 +29,13 @@ skip)
 	;;
 esac
 
-test -n "$rebase_root" && root_flag=--root
+if test -z "$rebase_root"
+	# this is now equivalent to ! -z "$upstream"
+then
+	revisions=$upstream...$orig_head
+else
+	revisions=$onto...$orig_head
+fi
 
 ret=0
 if test -n "$keep_empty"
@@ -38,14 +44,15 @@ then
 	# empty commits and even if it didn't the format doesn't really lend
 	# itself well to recording empty patches.  fortunately, cherry-pick
 	# makes this easy
-	git cherry-pick ${gpg_sign_opt:+"$gpg_sign_opt"} --allow-empty "$revisions"
+	git cherry-pick ${gpg_sign_opt:+"$gpg_sign_opt"} --allow-empty \
+		--right-only "$revisions"
 	ret=$?
 else
 	rm -f "$GIT_DIR/rebased-patches"
 
-	git format-patch -k --stdout --full-index --ignore-if-in-upstream \
+	git format-patch -k --stdout --full-index --cherry-pick --right-only \
 		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
-		$root_flag "$revisions" >"$GIT_DIR/rebased-patches"
+		"$revisions" >"$GIT_DIR/rebased-patches"
 	ret=$?
 
 	if test 0 != $ret
-- 
2.0.1.472.g6f92e5f.dirty

[PATCH 2/2] rebase: omit patch-identical commits with --fork-point

[John Keeping]

When the `--fork-point` argument was added to `git rebase`, we changed
the value of $upstream to be the fork point instead of the point from
which we want to rebase.  When $orig_head..$upstream is empty this does
not change the behaviour, but when there are new changes in the upstream
we are no longer checking if any of them are patch-identical with
changes in $upstream..$orig_head.

Fix this by introducing a new variable to hold the fork point and using
this to restrict the range as an extra (negative) revision argument so
that the set of desired revisions becomes (in fork-point mode):

	git rev-list --cherry-pick --right-only \
		$upstream...$orig_head ^$fork_point

This allows us to correctly handle the scenario where we have the
following topology:

	    C --- D --- E  <- dev
	   /
	  B  <- master@{1}
	 /
	o --- B' --- C* --- D*  <- master

where:
- B' is a fixed-up version of B that is not patch-identical with B;
- C* and D* are patch-identical to C and D respectively and conflict
  textually if applied in the wrong order;
- E depends textually on D.

The correct result of `git rebase master dev` is that B is identified as
the fork-point of dev and master, so that C, D, E are the commits that
need to be replayed onto master; but C and D are patch-identical with C*
and D* and so can be dropped, so that the end result is:

	o --- B' --- C* --- D* --- E  <- dev

If the fork-point is not identified, then picking B onto a branch
containing B' results in a conflict and if the patch-identical commits
are not correctly identified then picking C onto a branch containing D
(or equivalently D*) results in a conflict.

This change allows us to handle both of these cases, where previously we
either identified the fork-point (with `--fork-point`) but not the
patch-identical commits *or* (with `--no-fork-point`) identified the
patch-identical commits but not the fact that master had been rewritten.

Reported-by: Ted Felix <ted@tedfelix.com>
Signed-off-by: John Keeping <john@keeping.me.uk>
---
 git-rebase--am.sh          | 6 ++++--
 git-rebase--interactive.sh | 2 +-
 git-rebase.sh              | 7 ++++---
 3 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/git-rebase--am.sh b/git-rebase--am.sh
index 902bf2d..f923732 100644
--- a/git-rebase--am.sh
+++ b/git-rebase--am.sh
@@ -45,14 +45,16 @@ then
 	# itself well to recording empty patches.  fortunately, cherry-pick
 	# makes this easy
 	git cherry-pick ${gpg_sign_opt:+"$gpg_sign_opt"} --allow-empty \
-		--right-only "$revisions"
+		--right-only "$revisions" \
+		${restrict_revision+^$restrict_revision}
 	ret=$?
 else
 	rm -f "$GIT_DIR/rebased-patches"
 
 	git format-patch -k --stdout --full-index --cherry-pick --right-only \
 		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
-		"$revisions" >"$GIT_DIR/rebased-patches"
+		"$revisions" ${restrict_revision+^$restrict_revision} \
+		>"$GIT_DIR/rebased-patches"
 	ret=$?
 
 	if test 0 != $ret
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 7e1eda0..b64dd28 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -963,7 +963,7 @@ else
 fi
 git rev-list $merges_option --pretty=oneline --abbrev-commit \
 	--abbrev=7 --reverse --left-right --topo-order \
-	$revisions | \
+	$revisions ${restrict_revision+^$restrict_revision} | \
 	sed -n "s/^>//p" |
 while read -r shortsha1 rest
 do
diff --git a/git-rebase.sh b/git-rebase.sh
index 06c810b..55da9db 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -59,6 +59,7 @@ If you prefer to skip this patch, run "git rebase --skip" instead.
 To check out the original branch and stop rebasing, run "git rebase --abort".')
 "
 unset onto
+unset restrict_revision
 cmd=
 strategy=
 strategy_opts=
@@ -546,7 +547,7 @@ then
 			"${switch_to:-HEAD}")
 	if test -n "$new_upstream"
 	then
-		upstream=$new_upstream
+		restrict_revision=$new_upstream
 	fi
 fi
 
@@ -572,7 +573,7 @@ require_clean_work_tree "rebase" "$(gettext "Please commit or stash them.")"
 # and if this is not an interactive rebase.
 mb=$(git merge-base "$onto" "$orig_head")
 if test "$type" != interactive && test "$upstream" = "$onto" &&
-	test "$mb" = "$onto" &&
+	test "$mb" = "$onto" && test -z "$restrict_revision" &&
 	# linear history?
 	! (git rev-list --parents "$onto".."$orig_head" | sane_grep " .* ") > /dev/null
 then
@@ -626,7 +627,7 @@ if test -n "$rebase_root"
 then
 	revisions="$onto..$orig_head"
 else
-	revisions="$upstream..$orig_head"
+	revisions="${restrict_revision-$upstream}..$orig_head"
 fi
 
 run_specific_rebase
-- 
2.0.1.472.g6f92e5f.dirty

Re: [PATCH 2/2] rebase: omit patch-identical commits with --fork-point

[Ted Felix]

   Thanks, John.  My test script is working fine for me now with these 
patches.

Re: [PATCH 2/2] rebase: omit patch-identical commits with --fork-point

[Junio C Hamano]

John Keeping <john@keeping.me.uk> writes:

> When the `--fork-point` argument was added to `git rebase`, we changed
> the value of $upstream to be the fork point instead of the point from
> which we want to rebase.  When $orig_head..$upstream is empty this does
> not change the behaviour, but when there are new changes in the upstream
> we are no longer checking if any of them are patch-identical with
> changes in $upstream..$orig_head.
>
> Fix this by introducing a new variable to hold the fork point and using
> this to restrict the range as an extra (negative) revision argument so
> that the set of desired revisions becomes (in fork-point mode):
>
> 	git rev-list --cherry-pick --right-only \
> 		$upstream...$orig_head ^$fork_point
>
> This allows us to correctly handle the scenario where we have the
> following topology:
>
> 	    C --- D --- E  <- dev
> 	   /
> 	  B  <- master@{1}
> 	 /
> 	o --- B' --- C* --- D*  <- master
>
> where:
> - B' is a fixed-up version of B that is not patch-identical with B;
> - C* and D* are patch-identical to C and D respectively and conflict
>   textually if applied in the wrong order;
> - E depends textually on D.
>
> The correct result of `git rebase master dev` is that B is identified as
> the fork-point of dev and master, so that C, D, E are the commits that
> need to be replayed onto master; but C and D are patch-identical with C*
> and D* and so can be dropped, so that the end result is:
>
> 	o --- B' --- C* --- D* --- E  <- dev
>
> If the fork-point is not identified, then picking B onto a branch
> containing B' results in a conflict and if the patch-identical commits
> are not correctly identified then picking C onto a branch containing D
> (or equivalently D*) results in a conflict.
>
> This change allows us to handle both of these cases, where previously we
> either identified the fork-point (with `--fork-point`) but not the
> patch-identical commits *or* (with `--no-fork-point`) identified the
> patch-identical commits but not the fact that master had been rewritten.
>
> Reported-by: Ted Felix <ted@tedfelix.com>
> Signed-off-by: John Keeping <john@keeping.me.uk>
> ---
>  git-rebase--am.sh          | 6 ++++--
>  git-rebase--interactive.sh | 2 +-
>  git-rebase.sh              | 7 ++++---
>  3 files changed, 9 insertions(+), 6 deletions(-)

Can we have some tests, too?

The changes look correct from a cursory reading.

Thanks.

[PATCH v2 1/2] rebase--am: use --cherry-pick instead of --ignore-if-in-upstream

[John Keeping]

When using `git format-patch --ignore-if-in-upstream` we are only
allowed to give a single revision range.  In the next commit we will
want to add an additional exclusion revision in order to handle fork
points correctly, so convert `git-rebase--am` to use a symmetric
difference with `--cherry-pick --right-only`.

This does not change the result of the format-patch invocation, just how
we spell the arguments.

Signed-off-by: John Keeping <john@keeping.me.uk>
---

Unchanged from v1.

 git-rebase--am.sh | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/git-rebase--am.sh b/git-rebase--am.sh
index ca20e1e..902bf2d 100644
--- a/git-rebase--am.sh
+++ b/git-rebase--am.sh
@@ -29,7 +29,13 @@ skip)
 	;;
 esac
 
-test -n "$rebase_root" && root_flag=--root
+if test -z "$rebase_root"
+	# this is now equivalent to ! -z "$upstream"
+then
+	revisions=$upstream...$orig_head
+else
+	revisions=$onto...$orig_head
+fi
 
 ret=0
 if test -n "$keep_empty"
@@ -38,14 +44,15 @@ then
 	# empty commits and even if it didn't the format doesn't really lend
 	# itself well to recording empty patches.  fortunately, cherry-pick
 	# makes this easy
-	git cherry-pick ${gpg_sign_opt:+"$gpg_sign_opt"} --allow-empty "$revisions"
+	git cherry-pick ${gpg_sign_opt:+"$gpg_sign_opt"} --allow-empty \
+		--right-only "$revisions"
 	ret=$?
 else
 	rm -f "$GIT_DIR/rebased-patches"
 
-	git format-patch -k --stdout --full-index --ignore-if-in-upstream \
+	git format-patch -k --stdout --full-index --cherry-pick --right-only \
 		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
-		$root_flag "$revisions" >"$GIT_DIR/rebased-patches"
+		"$revisions" >"$GIT_DIR/rebased-patches"
 	ret=$?
 
 	if test 0 != $ret
-- 
2.0.1.472.g6f92e5f.dirty

[PATCH v2 2/2] rebase: omit patch-identical commits with --fork-point

[John Keeping]

When the `--fork-point` argument was added to `git rebase`, we changed
the value of $upstream to be the fork point instead of the point from
which we want to rebase.  When $orig_head..$upstream is empty this does
not change the behaviour, but when there are new changes in the upstream
we are no longer checking if any of them are patch-identical with
changes in $upstream..$orig_head.

Fix this by introducing a new variable to hold the fork point and using
this to restrict the range as an extra (negative) revision argument so
that the set of desired revisions becomes (in fork-point mode):

	git rev-list --cherry-pick --right-only \
		$upstream...$orig_head ^$fork_point

This allows us to correctly handle the scenario where we have the
following topology:

	    C --- D --- E  <- dev
	   /
	  B  <- master@{1}
	 /
	o --- B' --- C* --- D*  <- master

where:
- B' is a fixed-up version of B that is not patch-identical with B;
- C* and D* are patch-identical to C and D respectively and conflict
  textually if applied in the wrong order;
- E depends textually on D.

The correct result of `git rebase master dev` is that B is identified as
the fork-point of dev and master, so that C, D, E are the commits that
need to be replayed onto master; but C and D are patch-identical with C*
and D* and so can be dropped, so that the end result is:

	o --- B' --- C* --- D* --- E  <- dev

If the fork-point is not identified, then picking B onto a branch
containing B' results in a conflict and if the patch-identical commits
are not correctly identified then picking C onto a branch containing D
(or equivalently D*) results in a conflict.

This change allows us to handle both of these cases, where previously we
either identified the fork-point (with `--fork-point`) but not the
patch-identical commits *or* (with `--no-fork-point`) identified the
patch-identical commits but not the fact that master had been rewritten.

Reported-by: Ted Felix <ted@tedfelix.com>
Signed-off-by: John Keeping <john@keeping.me.uk>
---

Change from v1:
    - add a test case

 git-rebase--am.sh          |  6 ++++--
 git-rebase--interactive.sh |  2 +-
 git-rebase.sh              |  7 ++++---
 t/t3400-rebase.sh          | 23 +++++++++++++++++++++++
 4 files changed, 32 insertions(+), 6 deletions(-)

diff --git a/git-rebase--am.sh b/git-rebase--am.sh
index 902bf2d..f923732 100644
--- a/git-rebase--am.sh
+++ b/git-rebase--am.sh
@@ -45,14 +45,16 @@ then
 	# itself well to recording empty patches.  fortunately, cherry-pick
 	# makes this easy
 	git cherry-pick ${gpg_sign_opt:+"$gpg_sign_opt"} --allow-empty \
-		--right-only "$revisions"
+		--right-only "$revisions" \
+		${restrict_revision+^$restrict_revision}
 	ret=$?
 else
 	rm -f "$GIT_DIR/rebased-patches"
 
 	git format-patch -k --stdout --full-index --cherry-pick --right-only \
 		--src-prefix=a/ --dst-prefix=b/ --no-renames --no-cover-letter \
-		"$revisions" >"$GIT_DIR/rebased-patches"
+		"$revisions" ${restrict_revision+^$restrict_revision} \
+		>"$GIT_DIR/rebased-patches"
 	ret=$?
 
 	if test 0 != $ret
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 7e1eda0..b64dd28 100644
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -963,7 +963,7 @@ else
 fi
 git rev-list $merges_option --pretty=oneline --abbrev-commit \
 	--abbrev=7 --reverse --left-right --topo-order \
-	$revisions | \
+	$revisions ${restrict_revision+^$restrict_revision} | \
 	sed -n "s/^>//p" |
 while read -r shortsha1 rest
 do
diff --git a/git-rebase.sh b/git-rebase.sh
index 06c810b..55da9db 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -59,6 +59,7 @@ If you prefer to skip this patch, run "git rebase --skip" instead.
 To check out the original branch and stop rebasing, run "git rebase --abort".')
 "
 unset onto
+unset restrict_revision
 cmd=
 strategy=
 strategy_opts=
@@ -546,7 +547,7 @@ then
 			"${switch_to:-HEAD}")
 	if test -n "$new_upstream"
 	then
-		upstream=$new_upstream
+		restrict_revision=$new_upstream
 	fi
 fi
 
@@ -572,7 +573,7 @@ require_clean_work_tree "rebase" "$(gettext "Please commit or stash them.")"
 # and if this is not an interactive rebase.
 mb=$(git merge-base "$onto" "$orig_head")
 if test "$type" != interactive && test "$upstream" = "$onto" &&
-	test "$mb" = "$onto" &&
+	test "$mb" = "$onto" && test -z "$restrict_revision" &&
 	# linear history?
 	! (git rev-list --parents "$onto".."$orig_head" | sane_grep " .* ") > /dev/null
 then
@@ -626,7 +627,7 @@ if test -n "$rebase_root"
 then
 	revisions="$onto..$orig_head"
 else
-	revisions="$upstream..$orig_head"
+	revisions="${restrict_revision-$upstream}..$orig_head"
 fi
 
 run_specific_rebase
diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
index 80e0a95..47b5682 100755
--- a/t/t3400-rebase.sh
+++ b/t/t3400-rebase.sh
@@ -169,6 +169,29 @@ test_expect_success 'default to common base in @{upstream}s reflog if no upstrea
 	test_cmp expect actual
 '
 
+test_expect_success 'cherry-picked commits and fork-point work together' '
+	git checkout default-base &&
+	echo Amended >A &&
+	git commit -a --no-edit --amend &&
+	test_commit B B &&
+	test_commit new_B B "New B" &&
+	test_commit C C &&
+	git checkout default &&
+	git reset --hard default-base@{4} &&
+	test_commit D D &&
+	git cherry-pick -2 default-base^ &&
+	test_commit final_B B "Final B" &&
+	git rebase &&
+	echo Amended >expect &&
+	test_cmp A expect &&
+	echo "Final B" >expect &&
+	test_cmp B expect &&
+	echo C >expect &&
+	test_cmp C expect &&
+	echo D >expect &&
+	test_cmp D expect
+'
+
 test_expect_success 'rebase -q is quiet' '
 	git checkout -b quiet topic &&
 	git rebase -q master >output.out 2>&1 &&
-- 
2.0.1.472.g6f92e5f.dirty

Re: [PATCH v2 2/2] rebase: omit patch-identical commits with --fork-point

[Junio C Hamano]

John Keeping <john@keeping.me.uk> writes:

> When the `--fork-point` argument was added to `git rebase`, we changed
> the value of $upstream to be the fork point instead of the point from
> which we want to rebase.  When $orig_head..$upstream is empty this does
> not change the behaviour, but when there are new changes in the upstream
> we are no longer checking if any of them are patch-identical with
> changes in $upstream..$orig_head.
>
> Fix this by introducing a new variable to hold the fork point and using
> this to restrict the range as an extra (negative) revision argument so
> that the set of desired revisions becomes (in fork-point mode):
>
> 	git rev-list --cherry-pick --right-only \
> 		$upstream...$orig_head ^$fork_point
>
> This allows us to correctly handle the scenario where we have the
> following topology:
>
> 	    C --- D --- E  <- dev
> 	   /
> 	  B  <- master@{1}
> 	 /
> 	o --- B' --- C* --- D*  <- master
>
> where:
> - B' is a fixed-up version of B that is not patch-identical with B;
> - C* and D* are patch-identical to C and D respectively and conflict
>   textually if applied in the wrong order;
> - E depends textually on D.
>
> The correct result of `git rebase master dev` is that B is identified as
> the fork-point of dev and master, so that C, D, E are the commits that
> need to be replayed onto master; but C and D are patch-identical with C*
> and D* and so can be dropped, so that the end result is:
>
> 	o --- B' --- C* --- D* --- E  <- dev
>
> If the fork-point is not identified, then picking B onto a branch
> containing B' results in a conflict and if the patch-identical commits
> are not correctly identified then picking C onto a branch containing D
> (or equivalently D*) results in a conflict.
>
> This change allows us to handle both of these cases, where previously we
> either identified the fork-point (with `--fork-point`) but not the
> patch-identical commits *or* (with `--no-fork-point`) identified the
> patch-identical commits but not the fact that master had been rewritten.
>
> Reported-by: Ted Felix <ted@tedfelix.com>
> Signed-off-by: John Keeping <john@keeping.me.uk>
> ---
>
> Change from v1:
>     - add a test case

> diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
> index 80e0a95..47b5682 100755
> --- a/t/t3400-rebase.sh
> +++ b/t/t3400-rebase.sh
> @@ -169,6 +169,29 @@ test_expect_success 'default to common base in @{upstream}s reflog if no upstrea
>  	test_cmp expect actual
>  '
>  
> +test_expect_success 'cherry-picked commits and fork-point work together' '
> +	git checkout default-base &&
> +	echo Amended >A &&
> +	git commit -a --no-edit --amend &&
> +	test_commit B B &&
> +	test_commit new_B B "New B" &&
> +	test_commit C C &&
> +	git checkout default &&
> +	git reset --hard default-base@{4} &&
> +	test_commit D D &&
> +	git cherry-pick -2 default-base^ &&
> +	test_commit final_B B "Final B" &&
> +	git rebase &&

mental note: this rebases default (i.e. the current branch) on
default-base; it depends on branch.default.{remote,merge} being set
by the previous test piece.

> +	echo Amended >expect &&
> +	test_cmp A expect &&
> +	echo "Final B" >expect &&
> +	test_cmp B expect &&
> +	echo C >expect &&
> +	test_cmp C expect &&
> +	echo D >expect &&
> +	test_cmp D expect
> +'

Thanks.  Do these labels on the commits have any relation to the
topology illustrated in the log message?

It looks like the above produces this:

      a'---D---B'--new_B'---final_B (default)
     /
    o----a---B---new_B---C (default-base)
                          \
                           D''---final_B''

where 'a' is "Modify A." from the original set-up and B and new_B
are the cherry-picks to be filtered out during the rebase.  Am I
reading the test correctly?

>  test_expect_success 'rebase -q is quiet' '
>  	git checkout -b quiet topic &&
>  	git rebase -q master >output.out 2>&1 &&

Re: [PATCH v2 2/2] rebase: omit patch-identical commits with --fork-point

[John Keeping]

On Wed, Jul 16, 2014 at 01:26:32PM -0700, Junio C Hamano wrote:
> John Keeping <john@keeping.me.uk> writes:
> 
> > When the `--fork-point` argument was added to `git rebase`, we changed
> > the value of $upstream to be the fork point instead of the point from
> > which we want to rebase.  When $orig_head..$upstream is empty this does
> > not change the behaviour, but when there are new changes in the upstream
> > we are no longer checking if any of them are patch-identical with
> > changes in $upstream..$orig_head.
> >
> > Fix this by introducing a new variable to hold the fork point and using
> > this to restrict the range as an extra (negative) revision argument so
> > that the set of desired revisions becomes (in fork-point mode):
> >
> > 	git rev-list --cherry-pick --right-only \
> > 		$upstream...$orig_head ^$fork_point
> >
> > This allows us to correctly handle the scenario where we have the
> > following topology:
> >
> > 	    C --- D --- E  <- dev
> > 	   /
> > 	  B  <- master@{1}
> > 	 /
> > 	o --- B' --- C* --- D*  <- master
> >
> > where:
> > - B' is a fixed-up version of B that is not patch-identical with B;
> > - C* and D* are patch-identical to C and D respectively and conflict
> >   textually if applied in the wrong order;
> > - E depends textually on D.
> >
> > The correct result of `git rebase master dev` is that B is identified as
> > the fork-point of dev and master, so that C, D, E are the commits that
> > need to be replayed onto master; but C and D are patch-identical with C*
> > and D* and so can be dropped, so that the end result is:
> >
> > 	o --- B' --- C* --- D* --- E  <- dev
> >
> > If the fork-point is not identified, then picking B onto a branch
> > containing B' results in a conflict and if the patch-identical commits
> > are not correctly identified then picking C onto a branch containing D
> > (or equivalently D*) results in a conflict.
> >
> > This change allows us to handle both of these cases, where previously we
> > either identified the fork-point (with `--fork-point`) but not the
> > patch-identical commits *or* (with `--no-fork-point`) identified the
> > patch-identical commits but not the fact that master had been rewritten.
> >
> > Reported-by: Ted Felix <ted@tedfelix.com>
> > Signed-off-by: John Keeping <john@keeping.me.uk>
> > ---
> >
> > Change from v1:
> >     - add a test case
> 
> > diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
> > index 80e0a95..47b5682 100755
> > --- a/t/t3400-rebase.sh
> > +++ b/t/t3400-rebase.sh
> > @@ -169,6 +169,29 @@ test_expect_success 'default to common base in @{upstream}s reflog if no upstrea
> >  	test_cmp expect actual
> >  '
> >  
> > +test_expect_success 'cherry-picked commits and fork-point work together' '
> > +	git checkout default-base &&
> > +	echo Amended >A &&
> > +	git commit -a --no-edit --amend &&
> > +	test_commit B B &&
> > +	test_commit new_B B "New B" &&
> > +	test_commit C C &&
> > +	git checkout default &&
> > +	git reset --hard default-base@{4} &&
> > +	test_commit D D &&
> > +	git cherry-pick -2 default-base^ &&
> > +	test_commit final_B B "Final B" &&
> > +	git rebase &&
> 
> mental note: this rebases default (i.e. the current branch) on
> default-base; it depends on branch.default.{remote,merge} being set
> by the previous test piece.
> 
> > +	echo Amended >expect &&
> > +	test_cmp A expect &&
> > +	echo "Final B" >expect &&
> > +	test_cmp B expect &&
> > +	echo C >expect &&
> > +	test_cmp C expect &&
> > +	echo D >expect &&
> > +	test_cmp D expect
> > +'
> 
> Thanks.  Do these labels on the commits have any relation to the
> topology illustrated in the log message?

No, I didn't consider the log message when adding the test; it simply
uses the next letters that are unused in the test repository.

> It looks like the above produces this:
> 
>       a'---D---B'--new_B'---final_B (default)
>      /
>     o----a---B---new_B---C (default-base)
>                           \
>                            D''---final_B''
> 
> where 'a' is "Modify A." from the original set-up and B and new_B
> are the cherry-picks to be filtered out during the rebase.  Am I
> reading the test correctly?

Yes, that's right.  The idea is to make sure that we skip the
cherry-picked commits, which requires two commits that will conflict if
we try to apply the first on top of the second and to make sure that we
don't lose the new commit on the upstream (C).

> >  test_expect_success 'rebase -q is quiet' '
> >  	git checkout -b quiet topic &&
> >  	git rebase -q master >output.out 2>&1 &&

Re: [PATCH v2 2/2] rebase: omit patch-identical commits with --fork-point

[Ted Felix]

On 07/16/2014 03:23 PM, John Keeping wrote:
> Change from v1:
>      - add a test case

Test case is working fine for me.  It passes with the patch and fails 
without.  However, it does seem to cause all the rest of the test cases 
to fail if it fails.  Is there some cleanup missing?

Ted.

Re: [PATCH v2 2/2] rebase: omit patch-identical commits with --fork-point

[John Keeping]

On Wed, Jul 16, 2014 at 05:36:03PM -0400, Ted Felix wrote:
> On 07/16/2014 03:23 PM, John Keeping wrote:
> > Change from v1:
> >      - add a test case
> 
> Test case is working fine for me.  It passes with the patch and fails 
> without.  However, it does seem to cause all the rest of the test cases 
> to fail if it fails.  Is there some cleanup missing?

The individual test cases in the script don't run in isolation, so I
don't think it's surprising the the remainder fail if this one ends up
stuck in rebase-in-progress state.  I think the same will happen with
most of the other test cases in this script.