[PATCH v3 0/9] History traversal refinements

[PATCH v3 0/9] History traversal refinements

[Kevin Bracey]

New version - nothing much changed since v2.2, except the new test
set to illustrate and prove the changes. Not sure about the t6111
numbering - there wasn't space where I really wanted to put it.
And maybe it should be appended to one of the existing tests.

You will note that I'm floundering for the name for the commits I care
about in part 9. Currently at "priority", but that's horrible, not least
because it isn't an adjective. I think the word I really want is
"interesting", but that's already taken... "Relevant", "important"?

Junio C Hamano (1):
  t6012: update test for tweaked full-history traversal

Kevin Bracey (8):
  decorate.c: compact table when growing
  t6019: test file dropped in -s ours merge
  t6111: new TREESAME test set
  rev-list-options.txt: correct TREESAME for P
  revision.c: Make --full-history consider more merges
  simplify-merges: never remove all TREESAME parents
  simplify-merges: drop merge from irrelevant side branch
  revision.c: discount side branches when computing TREESAME

 Documentation/rev-list-options.txt |  38 +--
 decorate.c                         |   2 +-
 revision.c                         | 511 +++++++++++++++++++++++++++++++++----
 revision.h                         |   4 +-
 t/t6012-rev-list-simplify.sh       |  31 ++-
 t/t6019-rev-list-ancestry-path.sh  |  29 ++-
 t/t6111-rev-list-treesame.sh       | 180 +++++++++++++
 7 files changed, 723 insertions(+), 72 deletions(-)
 create mode 100755 t/t6111-rev-list-treesame.sh

-- 
1.8.3.rc0.28.g682c2d9

[PATCH v3 1/9] decorate.c: compact table when growing

[Kevin Bracey]

When growing the table, take the opportunity to "compact" it by removing
entries with NULL decoration.

Users may have "removed" decorations by passing NULL to
insert_decoration. An object's table entry can't actually be removed
during normal operation, as it would break the linear hash collision
search. But we can remove NULL decoration entries when rebuilding the
table.

Signed-off-by: Kevin Bracey <kevin@bracey.fi>
---
 decorate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/decorate.c b/decorate.c
index 2f8a63e..7cb5d29 100644
--- a/decorate.c
+++ b/decorate.c
@@ -49,7 +49,7 @@ static void grow_decoration(struct decoration *n)
 		const struct object *base = old_hash[i].base;
 		void *decoration = old_hash[i].decoration;
 
-		if (!base)
+		if (!decoration)
 			continue;
 		insert_decoration(n, base, decoration);
 	}
-- 
1.8.3.rc0.28.g682c2d9

[PATCH v3 2/9] t6019: test file dropped in -s ours merge

[Kevin Bracey]

In preparation for upcoming TREESAME work, check the result for G.t,
which is dropped in "-s ours" merge L. The default rev-list is empty, as
expected - it follows the first parent path where it never existed.

Unfortunately, --ancestry-path is also empty. Merges H J and L are all
TREESAME to 1 parent, so are treated as TREESAME and not shown. This is
clearly undesirable in the case of merge L, which dropped our G.t by
taking the non-ancestry-path version. Document this as a known failure,
and expect "H J L", the 3 merges along the path that had to chose G.t
versions.

Signed-off-by: Kevin Bracey <kevin@bracey.fi>
---
 t/t6019-rev-list-ancestry-path.sh | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/t/t6019-rev-list-ancestry-path.sh b/t/t6019-rev-list-ancestry-path.sh
index 39b4cb0..86ef908 100755
--- a/t/t6019-rev-list-ancestry-path.sh
+++ b/t/t6019-rev-list-ancestry-path.sh
@@ -13,6 +13,9 @@ test_description='--ancestry-path'
 #
 #  D..M -- M.t                 == M
 #  --ancestry-path D..M -- M.t == M
+#
+#  G..M -- G.t                 == [nothing - was dropped in "-s ours" merge L]
+#  --ancestry-path G..M -- G.t == H J L
 
 . ./test-lib.sh
 
@@ -63,13 +66,29 @@ test_expect_success 'rev-list D..M -- M.t' '
 	test_cmp expect actual
 '
 
-test_expect_success 'rev-list --ancestry-patch D..M -- M.t' '
+test_expect_success 'rev-list --ancestry-path D..M -- M.t' '
 	echo M >expect &&
 	git rev-list --ancestry-path --format=%s D..M -- M.t |
 	sed -e "/^commit /d" >actual &&
 	test_cmp expect actual
 '
 
+# G.t is dropped in an "-s ours" merge
+test_expect_success 'rev-list G..M -- G.t' '
+	>expect &&
+	git rev-list --format=%s G..M -- G.t |
+	sed -e "/^commit /d" >actual &&
+	test_cmp expect actual
+'
+
+test_expect_failure 'rev-list --ancestry-path G..M -- G.t' '
+	for c in H J L; do echo $c; done >expect &&
+	git rev-list --ancestry-path --format=%s G..M -- G.t |
+	sed -e "/^commit /d" |
+	sort >actual &&
+	test_cmp expect actual
+'
+
 #   b---bc
 #  / \ /
 # a   X
-- 
1.8.3.rc0.28.g682c2d9

[PATCH v3 3/9] t6111: new TREESAME test set

[Kevin Bracey]

Some side branching and odd merging to illustrate various flaws in
revision list scans, particularly when limiting the list.

Many expected failures, which will be gone by the end of the "history
traversal refinements" series.

Signed-off-by: Kevin Bracey <kevin@bracey.fi>
---
 t/t6111-rev-list-treesame.sh | 180 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 180 insertions(+)
 create mode 100755 t/t6111-rev-list-treesame.sh

diff --git a/t/t6111-rev-list-treesame.sh b/t/t6111-rev-list-treesame.sh
new file mode 100755
index 0000000..602c02d
--- /dev/null
+++ b/t/t6111-rev-list-treesame.sh
@@ -0,0 +1,180 @@
+#!/bin/sh
+#
+#        ,---E--.   *H----------.             * marks !TREESAME parent paths
+#       /        \ /             \*
+# *A--*B---D--*F-*G---------K-*L-*M
+#   \     /*       \       /
+#    `-C-'          `-*I-*J
+#
+# A creates "file", B and F change it.
+# Odd merge G takes the old version from B.
+# I changes it, but J reverts it.
+# H and L both change it, and M merges those changes.
+
+test_description='TREESAME and limiting'
+
+. ./test-lib.sh
+
+note () {
+	git tag "$1"
+}
+
+unnote () {
+	git name-rev --tags --stdin | sed -e "s|$_x40 (tags/\([^)]*\)) |\1 |g"
+}
+
+test_expect_success setup '
+	test_commit "Initial file" file "Hi there" A &&
+	git branch other-branch &&
+
+	test_commit "file=Hello" file "Hello" B &&
+	git branch third-branch &&
+
+	git checkout other-branch &&
+	test_commit "Added other" other "Hello" C &&
+
+	git checkout master &&
+	test_merge D other-branch &&
+
+	git checkout third-branch &&
+	test_commit "Third file" third "Nothing" E &&
+
+	git checkout master &&
+	test_commit "file=Blah" file "Blah" F &&
+
+	test_tick && git merge --no-commit third-branch &&
+	git checkout third-branch file &&
+	git commit &&
+	note G &&
+	git branch fiddler-branch &&
+
+	git checkout -b part2-branch &&
+	test_commit "file=Part 2" file "Part 2" H &&
+
+	git checkout fiddler-branch &&
+	test_commit "Bad commit" file "Silly" I &&
+
+	test_tick && git revert I && note J &&
+
+	git checkout master &&
+	test_tick && git merge --no-ff fiddler-branch &&
+	note K
+
+	test_commit "file=Part 1" file "Part 1" L &&
+
+	test_tick && test_must_fail git merge part2-branch &&
+	test_commit M file "Parts 1+2"
+'
+
+FMT='tformat:%P 	%H | %s'
+
+# could we soup this up to optionally check parents? So "(BA)C" would check
+# that C is shown and has parents B A.
+check_outcome () {
+	outcome=$1
+	shift
+	for c in $1
+	do
+		echo "$c"
+	done >expect &&
+	shift &&
+	param="$*" &&
+	test_expect_$outcome "log $param" '
+		git log --format="$FMT" $param |
+		unnote >actual &&
+		sed -e "s/^.*	\([^ ]*\) .*/\1/" >check <actual &&
+		test_cmp expect check || {
+			cat actual
+			false
+		}
+	'
+}
+
+check_result () {
+	check_outcome success "$@"
+}
+
+# Odd merge G drops a change in F. Important that G is listed in all
+# except the most basic list. Achieving this means normal merge D will also be
+# shown in normal full-history, as we can't distinguish unless we do a
+# simplification pass. After simplification, D is dropped but G remains.
+check_result 'M L K J I H G F E D C B A'
+check_result 'M H L K J I G E F D C B A' --topo-order
+check_result 'M L H B A' -- file
+check_result 'M L H B A' --parents -- file
+check_outcome failure 'M L J I H G F D B A' --full-history -- file # drops G
+check_result 'M L K J I H G F D B A' --full-history --parents -- file
+check_outcome failure 'M H L J I G F B A' --simplify-merges -- file # drops G
+check_result 'M L K G F D B A' --first-parent
+check_result 'M L G F B A' --first-parent -- file
+
+# Check that odd merge G remains shown when F is the bottom.
+check_result 'M L K J I H G E' F..M
+check_result 'M H L K J I G E' F..M --topo-order
+check_result 'M L H' F..M -- file
+check_result 'M L H' F..M --parents -- file # L+H's parents rewritten to B, so more useful than it may seem
+check_outcome failure 'M L J I H G' F..M --full-history -- file # drops G
+check_result 'M L K J I H G' F..M --full-history --parents -- file
+check_outcome failure 'M H L J I G' F..M --simplify-merges -- file # drops G
+check_result 'M L K J I H G' F..M --ancestry-path
+check_outcome failure 'M L J I H G' F..M --ancestry-path -- file # drops G
+check_result 'M L K J I H G' F..M --ancestry-path --parents -- file
+check_result 'M H L J I G' F..M --ancestry-path --simplify-merges -- file
+check_result 'M L K G' F..M --first-parent
+check_result 'M L G' F..M --first-parent -- file
+
+# Note that G is pruned when E is the bottom, even if it's the same commit list
+# If we want history since E, then we're quite happy to ignore G that took E.
+check_result 'M L K J I H G' E..M --ancestry-path
+check_result 'M L J I H' E..M --ancestry-path -- file
+check_outcome failure 'M L K J I H' E..M --ancestry-path --parents -- file
+check_outcome failure 'M H L J I' E..M --ancestry-path --simplify-merges -- file # includes G
+
+# Should still be able to ignore I-J branch in simple log, despite limiting
+# to G.
+check_result 'M L K J I H' G..M
+check_result 'M H L K J I' G..M --topo-order
+check_outcome failure 'M L H' G..M -- file # includes J I
+check_outcome failure 'M L H' G..M --parents -- file # includes J I
+check_result 'M L J I H' G..M --full-history -- file
+check_result 'M L K J I H' G..M --full-history --parents -- file
+check_result 'M H L J I' G..M --simplify-merges -- file
+check_result 'M L K J I H' G..M --ancestry-path
+check_result 'M L J I H' G..M --ancestry-path -- file
+check_result 'M L K J I H' G..M --ancestry-path --parents -- file
+check_result 'M H L J I' G..M --ancestry-path --simplify-merges -- file
+
+# B..F should be able to simplify the merge D from irrelevant side branch C.
+# Default log should also be free to follow B-D, and ignore C.
+# But --full-history shouldn't drop D on its own - without simplification,
+# we can't decide if the merge from INTERESTING commit C was sensible.
+check_result 'F D C' B..F
+check_result 'F' B..F -- file
+check_outcome failure 'F' B..F --parents -- file # includes D
+check_outcome failure 'F D' B..F --full-history -- file # drops D prematurely
+check_result 'F D' B..F --full-history --parents -- file
+check_result 'F' B..F --simplify-merges -- file
+check_result 'F D' B..F --ancestry-path
+check_result 'F' B..F --ancestry-path -- file
+check_outcome failure 'F' B..F --ancestry-path --parents -- file # includes D
+check_outcome failure 'F' B..F --ancestry-path --simplify-merges -- file # includes D
+check_result 'F D' B..F --first-parent
+check_result 'F' B..F --first-parent -- file
+
+# Any sort of full history of C..F should show D, as it's the connection to C,
+# and it differs from it.
+check_result 'F D B' C..F
+check_result 'F B' C..F -- file
+check_result 'F B' C..F --parents -- file
+check_outcome failure 'F D B' C..F --full-history -- file # drops D
+check_result 'F D B' C..F --full-history --parents -- file
+check_result 'F D B' C..F --simplify-merges -- file
+check_result 'F D' C..F --ancestry-path
+check_outcome failure 'F D' C..F --ancestry-path -- file # drops D
+check_result 'F D' C..F --ancestry-path --parents -- file
+check_result 'F D' C..F --ancestry-path --simplify-merges -- file
+check_result 'F D B' C..F --first-parent
+check_result 'F B' C..F --first-parent -- file
+
+
+test_done
-- 
1.8.3.rc0.28.g682c2d9

[PATCH v3 4/9] rev-list-options.txt: correct TREESAME for P

[Kevin Bracey]

In the example given, P is not TREESAME to E. This doesn't affect the
current result, but it will matter when we change behaviour.

Signed-off-by: Kevin Bracey <kevin@bracey.fi>
---
 Documentation/rev-list-options.txt | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt
index 3bdbf5e..50bbff7 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -367,8 +367,7 @@ each merge.  The commits are:
   `N` and `D` to "foobarbaz"; i.e., it is not TREESAME to any parent.
 
 * `E` changes `quux` to "xyzzy", and its merge `P` combines the
-  strings to "quux xyzzy".  Despite appearing interesting, `P` is
-  TREESAME to all parents.
+  strings to "quux xyzzy".  `P` is TREESAME to `O`, but not to `E`.
 
 'rev-list' walks backwards through history, including or excluding
 commits based on whether '\--full-history' and/or parent rewriting
-- 
1.8.3.rc0.28.g682c2d9

[PATCH v3 5/9] revision.c: Make --full-history consider more merges

[Kevin Bracey]

History simplification previously always treated merges as TREESAME
if they were TREESAME to any parent.

While this was consistent with the default behaviour, this could be
extremely unhelpful when searching detailed history, and could not be
overridden. For example, if a merge had ignored a change, as if by "-s
ours", then:

  git log -m -p --full-history -Schange file

would successfully locate "change"'s addition but would not locate the
merge that resolved against it.

Futher, simplify_merges could drop the actual parent that a commit
was TREESAME to, leaving it as a normal commit marked TREESAME that
isn't actually TREESAME to its remaining parent.

Now redefine a commit's TREESAME flag to be true only if a commit is
TREESAME to _all_ of its parents. This doesn't affect either the default
simplify_history behaviour (because partially TREESAME merges are turned
into normal commits), or full-history with parent rewriting (because all
merges are output). But it does affect other modes. The clearest
difference is that --full-history will show more merges - sufficient to
ensure that -m -p --full-history log searches can really explain every
change to the file, including those changes' ultimate fate in merges.

Also modify simplify_merges to recalculate TREESAME after removing
a parent. This is achieved by storing per-parent TREESAME flags on the
initial scan, so the combined flag can be easily recomputed.

This fixes some t6111 failures, but creates a couple of new ones -
we are now showing some merges that don't need to be shown.

Signed-off-by: Kevin Bracey <kevin@bracey.fi>
---
 Documentation/rev-list-options.txt |   6 +-
 revision.c                         | 241 ++++++++++++++++++++++++++++++++-----
 revision.h                         |   1 +
 t/t6019-rev-list-ancestry-path.sh  |   2 +-
 t/t6111-rev-list-treesame.sh       |  20 +--
 5 files changed, 226 insertions(+), 44 deletions(-)

diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt
index 50bbff7..1dad341 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -409,10 +409,10 @@ parent lines.
 	the example, we get
 +
 -----------------------------------------------------------------------
-	I  A  B  N  D  O
+	I  A  B  N  D  O  P
 -----------------------------------------------------------------------
 +
-`P` and `M` were excluded because they are TREESAME to a parent.  `E`,
+`M` was excluded because it is TREESAME to both parents.  `E`,
 `C` and `B` were all walked, but only `B` was !TREESAME, so the others
 do not appear.
 +
@@ -440,7 +440,7 @@ themselves.  This results in
 Compare to '\--full-history' without rewriting above.  Note that `E`
 was pruned away because it is TREESAME, but the parent list of P was
 rewritten to contain `E`'s parent `I`.  The same happened for `C` and
-`N`.  Note also that `P` was included despite being TREESAME.
+`N`.
 
 In addition to the above settings, you can change whether TREESAME
 affects inclusion:
diff --git a/revision.c b/revision.c
index a67b615..c88ded8 100644
--- a/revision.c
+++ b/revision.c
@@ -429,10 +429,100 @@ static int rev_same_tree_as_empty(struct rev_info *revs, struct commit *commit)
 	return retval >= 0 && (tree_difference == REV_TREE_SAME);
 }
 
+struct treesame_state {
+	unsigned int nparents;
+	unsigned char treesame[FLEX_ARRAY];
+};
+
+static struct treesame_state *initialise_treesame(struct rev_info *revs, struct commit *commit)
+{
+	unsigned n = commit_list_count(commit->parents);
+	struct treesame_state *st = xcalloc(1, sizeof(*st) + n);
+	st->nparents = n;
+	add_decoration(&revs->treesame, &commit->object, st);
+	return st;
+}
+
+/*
+ * Must be called immediately after removing the nth_parent from a commit's
+ * parent list, if we are maintaining the per-parent treesame[] decoration.
+ * This does not recalculate the master TREESAME flag - update_treesame()
+ * should be called to update it after a sequence of treesame[] modifications
+ * that may have affected it.
+ */
+static int compact_treesame(struct rev_info *revs, struct commit *commit, unsigned nth_parent)
+{
+	struct treesame_state *st;
+	int old_same;
+
+	if (!commit->parents) {
+		/*
+		 * Have just removed the only parent from a non-merge.
+		 * Different handling, as we lack decoration.
+		 */
+		if (nth_parent != 0)
+			die("compact_treesame %u", nth_parent);
+		old_same = !!(commit->object.flags & TREESAME);
+		if (rev_same_tree_as_empty(revs, commit))
+			commit->object.flags |= TREESAME;
+		else
+			commit->object.flags &= ~TREESAME;
+		return old_same;
+	}
+
+	st = lookup_decoration(&revs->treesame, &commit->object);
+	if (!st || nth_parent >= st->nparents)
+		die("compact_treesame %u", nth_parent);
+
+	old_same = st->treesame[nth_parent];
+	memmove(st->treesame + nth_parent,
+		st->treesame + nth_parent + 1,
+		st->nparents - nth_parent - 1);
+
+	/*
+	 * If we've just become a non-merge commit, update TREESAME
+	 * immediately, and remove the no-longer-needed decoration.
+	 * If still a merge, defer update until update_treesame().
+	 */
+	if (--st->nparents == 1) {
+		if (commit->parents->next)
+			die("compact_treesame parents mismatch");
+		if (st->treesame[0] && revs->dense)
+			commit->object.flags |= TREESAME;
+		else
+			commit->object.flags &= ~TREESAME;
+		free(add_decoration(&revs->treesame, &commit->object, NULL));
+	}
+
+	return old_same;
+}
+
+static unsigned update_treesame(struct rev_info *revs, struct commit *commit)
+{
+	if (commit->parents && commit->parents->next) {
+		unsigned n;
+		struct treesame_state *st;
+
+		st = lookup_decoration(&revs->treesame, &commit->object);
+		if (!st)
+			die("update_treesame %s", sha1_to_hex(commit->object.sha1));
+		commit->object.flags |= TREESAME;
+		for (n = 0; n < st->nparents; n++) {
+			if (!st->treesame[n]) {
+				commit->object.flags &= ~TREESAME;
+				break;
+			}
+		}
+	}
+
+	return commit->object.flags & TREESAME;
+}
+
 static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 {
 	struct commit_list **pp, *parent;
-	int tree_changed = 0, tree_same = 0, nth_parent = 0;
+	struct treesame_state *ts = NULL;
+	int tree_changed = 0, nth_parent;
 
 	/*
 	 * If we don't do pruning, everything is interesting
@@ -456,25 +546,43 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 	if (!revs->dense && !commit->parents->next)
 		return;
 
-	pp = &commit->parents;
-	while ((parent = *pp) != NULL) {
+	for (pp = &commit->parents, nth_parent = 0;
+	     (parent = *pp) != NULL;
+	     pp = &parent->next, nth_parent++) {
 		struct commit *p = parent->item;
 
-		/*
-		 * Do not compare with later parents when we care only about
-		 * the first parent chain, in order to avoid derailing the
-		 * traversal to follow a side branch that brought everything
-		 * in the path we are limited to by the pathspec.
-		 */
-		if (revs->first_parent_only && nth_parent++)
-			break;
+		if (nth_parent == 1) {
+			/*
+			 * This our second loop iteration - so we now know
+			 * we're dealing with a merge.
+			 *
+			 * Do not compare with later parents when we care only about
+			 * the first parent chain, in order to avoid derailing the
+			 * traversal to follow a side branch that brought everything
+			 * in the path we are limited to by the pathspec.
+			 */
+			if (revs->first_parent_only)
+				break;
+			/*
+			 * If this will remain a potentially-simplifiable
+			 * merge, remember per-parent treesame if needed.
+			 * Initialise the array with the comparison from our
+			 * first iteration.
+			 */
+			if (revs->treesame.name &&
+			    !revs->simplify_history &&
+			    !(commit->object.flags & UNINTERESTING)) {
+				ts = initialise_treesame(revs, commit);
+				if (!tree_changed)
+					ts->treesame[0] = 1;
+			}
+		}
 		if (parse_commit(p) < 0)
 			die("cannot simplify commit %s (because of %s)",
 			    sha1_to_hex(commit->object.sha1),
 			    sha1_to_hex(p->object.sha1));
 		switch (rev_compare_tree(revs, p, commit)) {
 		case REV_TREE_SAME:
-			tree_same = 1;
 			if (!revs->simplify_history || (p->object.flags & UNINTERESTING)) {
 				/* Even if a merge with an uninteresting
 				 * side branch brought the entire change
@@ -482,7 +590,8 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 				 * to lose the other branches of this
 				 * merge, so we just keep going.
 				 */
-				pp = &parent->next;
+				if (ts)
+					ts->treesame[nth_parent] = 1;
 				continue;
 			}
 			parent->next = NULL;
@@ -511,12 +620,11 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 		case REV_TREE_OLD:
 		case REV_TREE_DIFFERENT:
 			tree_changed = 1;
-			pp = &parent->next;
 			continue;
 		}
 		die("bad tree compare for commit %s", sha1_to_hex(commit->object.sha1));
 	}
-	if (tree_changed && !tree_same)
+	if (tree_changed)
 		return;
 	commit->object.flags |= TREESAME;
 }
@@ -1930,28 +2038,32 @@ static void add_child(struct rev_info *revs, struct commit *parent, struct commi
 	l->next = add_decoration(&revs->children, &parent->object, l);
 }
 
-static int remove_duplicate_parents(struct commit *commit)
+static int remove_duplicate_parents(struct rev_info *revs, struct commit *commit)
 {
+	struct treesame_state *ts = lookup_decoration(&revs->treesame, &commit->object);
 	struct commit_list **pp, *p;
 	int surviving_parents;
 
 	/* Examine existing parents while marking ones we have seen... */
 	pp = &commit->parents;
+	surviving_parents = 0;
 	while ((p = *pp) != NULL) {
 		struct commit *parent = p->item;
 		if (parent->object.flags & TMP_MARK) {
 			*pp = p->next;
+			if (ts)
+				compact_treesame(revs, commit, surviving_parents);
 			continue;
 		}
 		parent->object.flags |= TMP_MARK;
+		surviving_parents++;
 		pp = &p->next;
 	}
-	/* count them while clearing the temporary mark */
-	surviving_parents = 0;
+	/* clear the temporary mark */
 	for (p = commit->parents; p; p = p->next) {
 		p->item->object.flags &= ~TMP_MARK;
-		surviving_parents++;
 	}
+	/* no update_treesame() - removing duplicates can't affect TREESAME */
 	return surviving_parents;
 }
 
@@ -1971,6 +2083,70 @@ static struct merge_simplify_state *locate_simplify_state(struct rev_info *revs,
 	return st;
 }
 
+static int mark_redundant_parents(struct rev_info *revs, struct commit *commit)
+{
+	struct commit_list *h = reduce_heads(commit->parents);
+	int i = 0, marked = 0;
+	struct commit_list *po, *pn;
+
+	/* Want these for sanity-checking only */
+	int orig_cnt = commit_list_count(commit->parents);
+	int cnt = commit_list_count(h);
+
+	/*
+	 * Not ready to remove items yet, just mark them for now, based
+	 * on the output of reduce_heads(). reduce_heads outputs the reduced
+	 * set in its original order, so this isn't too hard.
+	 */
+	po = commit->parents;
+	pn = h;
+	while (po) {
+		if (pn && po->item == pn->item) {
+			pn = pn->next;
+			i++;
+		} else {
+			po->item->object.flags |= TMP_MARK;
+			marked++;
+		}
+		po=po->next;
+	}
+
+	if (i != cnt || cnt+marked != orig_cnt)
+		die("mark_redundant_parents %d %d %d %d", orig_cnt, cnt, i, marked);
+
+	free_commit_list(h);
+
+	return marked;
+}
+
+static int remove_marked_parents(struct rev_info *revs, struct commit *commit)
+{
+	struct commit_list **pp, *p;
+	int nth_parent, removed = 0;
+
+	pp = &commit->parents;
+	nth_parent = 0;
+	while ((p = *pp) != NULL) {
+		struct commit *parent = p->item;
+		if (parent->object.flags & TMP_MARK) {
+			parent->object.flags &= ~TMP_MARK;
+			*pp = p->next;
+			free(p);
+			removed++;
+			compact_treesame(revs, commit, nth_parent);
+			continue;
+		}
+		pp = &p->next;
+		nth_parent++;
+	}
+
+	/* Removing parents can only increase TREESAMEness */
+	if (removed && !(commit->object.flags & TREESAME))
+		update_treesame(revs, commit);
+
+	return nth_parent;
+}
+
 static struct commit_list **simplify_one(struct rev_info *revs, struct commit *commit, struct commit_list **tail)
 {
 	struct commit_list *p;
@@ -2015,7 +2191,9 @@ static struct commit_list **simplify_one(struct rev_info *revs, struct commit *c
 	}
 
 	/*
-	 * Rewrite our list of parents.
+	 * Rewrite our list of parents. Note that this cannot
+	 * affect our TREESAME flags in any way - a commit is
+	 * always TREESAME to its simplification.
 	 */
 	for (p = commit->parents; p; p = p->next) {
 		pst = locate_simplify_state(revs, p->item);
@@ -2027,31 +2205,30 @@ static struct commit_list **simplify_one(struct rev_info *revs, struct commit *c
 	if (revs->first_parent_only)
 		cnt = 1;
 	else
-		cnt = remove_duplicate_parents(commit);
+		cnt = remove_duplicate_parents(revs, commit);
 
 	/*
 	 * It is possible that we are a merge and one side branch
 	 * does not have any commit that touches the given paths;
-	 * in such a case, the immediate parents will be rewritten
-	 * to different commits.
+	 * in such a case, the immediate parent from that branch
+	 * will be rewritten to be the merge base.
 	 *
 	 *      o----X		X: the commit we are looking at;
 	 *     /    /		o: a commit that touches the paths;
 	 * ---o----'
 	 *
-	 * Further reduce the parents by removing redundant parents.
+	 * Detect and simplify this case.
 	 */
 	if (1 < cnt) {
-		struct commit_list *h = reduce_heads(commit->parents);
-		cnt = commit_list_count(h);
-		free_commit_list(commit->parents);
-		commit->parents = h;
+		int marked = mark_redundant_parents(revs, commit);
+		if (marked)
+			cnt = remove_marked_parents(revs, commit);
 	}
 
 	/*
 	 * A commit simplifies to itself if it is a root, if it is
 	 * UNINTERESTING, if it touches the given paths, or if it is a
-	 * merge and its parents simplifies to more than one commits
+	 * merge and its parents simplify to more than one commit
 	 * (the first two cases are already handled at the beginning of
 	 * this function).
 	 *
@@ -2159,6 +2336,10 @@ int prepare_revision_walk(struct rev_info *revs)
 	if (!revs->leak_pending)
 		free(list);
 
+	/* Signal whether we need per-parent treesame decoration */
+	if (revs->simplify_merges)
+		revs->treesame.name = "treesame";
+
 	if (revs->no_walk != REVISION_WALK_NO_WALK_UNSORTED)
 		commit_list_sort_by_date(&revs->commits);
 	if (revs->no_walk)
@@ -2218,7 +2399,7 @@ static int rewrite_parents(struct rev_info *revs, struct commit *commit)
 		}
 		pp = &parent->next;
 	}
-	remove_duplicate_parents(commit);
+	remove_duplicate_parents(revs, commit);
 	return 0;
 }
 
diff --git a/revision.h b/revision.h
index 01bd2b7..1abe57b 100644
--- a/revision.h
+++ b/revision.h
@@ -167,6 +167,7 @@ struct rev_info {
 	struct reflog_walk_info *reflog_info;
 	struct decoration children;
 	struct decoration merge_simplification;
+	struct decoration treesame;
 
 	/* notes-specific options: which refs to show */
 	struct display_notes_opt notes_opt;
diff --git a/t/t6019-rev-list-ancestry-path.sh b/t/t6019-rev-list-ancestry-path.sh
index 86ef908..ebe79ac 100755
--- a/t/t6019-rev-list-ancestry-path.sh
+++ b/t/t6019-rev-list-ancestry-path.sh
@@ -81,7 +81,7 @@ test_expect_success 'rev-list G..M -- G.t' '
 	test_cmp expect actual
 '
 
-test_expect_failure 'rev-list --ancestry-path G..M -- G.t' '
+test_expect_success 'rev-list --ancestry-path G..M -- G.t' '
 	for c in H J L; do echo $c; done >expect &&
 	git rev-list --ancestry-path --format=%s G..M -- G.t |
 	sed -e "/^commit /d" |
diff --git a/t/t6111-rev-list-treesame.sh b/t/t6111-rev-list-treesame.sh
index 602c02d..efc2442 100755
--- a/t/t6111-rev-list-treesame.sh
+++ b/t/t6111-rev-list-treesame.sh
@@ -102,9 +102,9 @@ check_result 'M L K J I H G F E D C B A'
 check_result 'M H L K J I G E F D C B A' --topo-order
 check_result 'M L H B A' -- file
 check_result 'M L H B A' --parents -- file
-check_outcome failure 'M L J I H G F D B A' --full-history -- file # drops G
+check_result 'M L J I H G F D B A' --full-history -- file
 check_result 'M L K J I H G F D B A' --full-history --parents -- file
-check_outcome failure 'M H L J I G F B A' --simplify-merges -- file # drops G
+check_result 'M H L J I G F B A' --simplify-merges -- file
 check_result 'M L K G F D B A' --first-parent
 check_result 'M L G F B A' --first-parent -- file
 
@@ -113,11 +113,11 @@ check_result 'M L K J I H G E' F..M
 check_result 'M H L K J I G E' F..M --topo-order
 check_result 'M L H' F..M -- file
 check_result 'M L H' F..M --parents -- file # L+H's parents rewritten to B, so more useful than it may seem
-check_outcome failure 'M L J I H G' F..M --full-history -- file # drops G
+check_result 'M L J I H G' F..M --full-history -- file
 check_result 'M L K J I H G' F..M --full-history --parents -- file
-check_outcome failure 'M H L J I G' F..M --simplify-merges -- file # drops G
+check_result 'M H L J I G' F..M --simplify-merges -- file
 check_result 'M L K J I H G' F..M --ancestry-path
-check_outcome failure 'M L J I H G' F..M --ancestry-path -- file # drops G
+check_result 'M L J I H G' F..M --ancestry-path -- file
 check_result 'M L K J I H G' F..M --ancestry-path --parents -- file
 check_result 'M H L J I G' F..M --ancestry-path --simplify-merges -- file
 check_result 'M L K G' F..M --first-parent
@@ -126,7 +126,7 @@ check_result 'M L G' F..M --first-parent -- file
 # Note that G is pruned when E is the bottom, even if it's the same commit list
 # If we want history since E, then we're quite happy to ignore G that took E.
 check_result 'M L K J I H G' E..M --ancestry-path
-check_result 'M L J I H' E..M --ancestry-path -- file
+check_outcome failure 'M L J I H' E..M --ancestry-path -- file # includes G
 check_outcome failure 'M L K J I H' E..M --ancestry-path --parents -- file
 check_outcome failure 'M H L J I' E..M --ancestry-path --simplify-merges -- file # includes G
 
@@ -149,13 +149,13 @@ check_result 'M H L J I' G..M --ancestry-path --simplify-merges -- file
 # But --full-history shouldn't drop D on its own - without simplification,
 # we can't decide if the merge from INTERESTING commit C was sensible.
 check_result 'F D C' B..F
-check_result 'F' B..F -- file
+check_outcome failure 'F' B..F -- file # includes D
 check_outcome failure 'F' B..F --parents -- file # includes D
-check_outcome failure 'F D' B..F --full-history -- file # drops D prematurely
+check_result 'F D' B..F --full-history -- file
 check_result 'F D' B..F --full-history --parents -- file
 check_result 'F' B..F --simplify-merges -- file
 check_result 'F D' B..F --ancestry-path
-check_result 'F' B..F --ancestry-path -- file
+check_outcome failure 'F' B..F --ancestry-path -- file # includes D
 check_outcome failure 'F' B..F --ancestry-path --parents -- file # includes D
 check_outcome failure 'F' B..F --ancestry-path --simplify-merges -- file # includes D
 check_result 'F D' B..F --first-parent
@@ -170,7 +170,7 @@ check_outcome failure 'F D B' C..F --full-history -- file # drops D
 check_result 'F D B' C..F --full-history --parents -- file
 check_result 'F D B' C..F --simplify-merges -- file
 check_result 'F D' C..F --ancestry-path
-check_outcome failure 'F D' C..F --ancestry-path -- file # drops D
+check_result 'F D' C..F --ancestry-path -- file
 check_result 'F D' C..F --ancestry-path --parents -- file
 check_result 'F D' C..F --ancestry-path --simplify-merges -- file
 check_result 'F D B' C..F --first-parent
-- 
1.8.3.rc0.28.g682c2d9

[PATCH v3 6/9] t6012: update test for tweaked full-history traversal

[Kevin Bracey]

From: Junio C Hamano <gitster@pobox.com>

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t6012-rev-list-simplify.sh | 29 +++++++++++++++++++++++------
 1 file changed, 23 insertions(+), 6 deletions(-)

diff --git a/t/t6012-rev-list-simplify.sh b/t/t6012-rev-list-simplify.sh
index dd6dc84..4e55872 100755
--- a/t/t6012-rev-list-simplify.sh
+++ b/t/t6012-rev-list-simplify.sh
@@ -14,21 +14,24 @@ unnote () {
 
 test_expect_success setup '
 	echo "Hi there" >file &&
-	git add file &&
-	test_tick && git commit -m "Initial file" &&
+	echo "initial" >lost &&
+	git add file lost &&
+	test_tick && git commit -m "Initial file and lost" &&
 	note A &&
 
 	git branch other-branch &&
 
 	echo "Hello" >file &&
-	git add file &&
-	test_tick && git commit -m "Modified file" &&
+	echo "second" >lost &&
+	git add file lost &&
+	test_tick && git commit -m "Modified file and lost" &&
 	note B &&
 
 	git checkout other-branch &&
 
 	echo "Hello" >file &&
-	git add file &&
+	>lost &&
+	git add file lost &&
 	test_tick && git commit -m "Modified the file identically" &&
 	note C &&
 
@@ -37,7 +40,9 @@ test_expect_success setup '
 	test_tick && git commit -m "Add another file" &&
 	note D &&
 
-	test_tick && git merge -m "merge" master &&
+	test_tick &&
+	test_must_fail git merge -m "merge" master &&
+	>lost && git commit -a -m "merge" &&
 	note E &&
 
 	echo "Yet another" >elif &&
@@ -110,4 +115,16 @@ check_result 'I B A' -- file
 check_result 'I B A' --topo-order -- file
 check_result 'H' --first-parent -- another-file
 
+check_result 'E C B A' --full-history E -- lost
+test_expect_success 'full history simplification without parent' '
+	printf "%s\n" E C B A >expect &&
+	git log --pretty="$FMT" --full-history E -- lost |
+	unnote >actual &&
+	sed -e "s/^.*	\([^ ]*\) .*/\1/" >check <actual &&
+	test_cmp expect check || {
+		cat actual
+		false
+	}
+'
+
 test_done
-- 
1.8.3.rc0.28.g682c2d9

[PATCH v3 7/9] simplify-merges: never remove all TREESAME parents

[Kevin Bracey]

When simplifying an odd merge, such as one that used "-s ours", we may
find ourselves TREESAME to apparently redundant parents. Prevent
simplify_merges() from removing every TREESAME parent; if this would
happen reinstate the first TREESAME parent - the one that the default
log would have followed.

This avoids producing a totally disjoint history from the default log
when the default log is a better explanation of the end result, and aids
visualisation of odd merges.

Signed-off-by: Kevin Bracey <kevin@bracey.fi>
---
 Documentation/rev-list-options.txt |  3 +-
 revision.c                         | 69 ++++++++++++++++++++++++++++++++++++++
 t/t6111-rev-list-treesame.sh       |  2 +-
 3 files changed, 72 insertions(+), 2 deletions(-)

diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt
index 1dad341..e23bdb0 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -471,7 +471,8 @@ history according to the following rules:
 +
 * Replace each parent `P` of `C'` with its simplification `P'`.  In
   the process, drop parents that are ancestors of other parents, and
-  remove duplicates.
+  remove duplicates, but take care to never drop all parents that
+  we are TREESAME to.
 +
 * If after this parent rewriting, `C'` is a root or merge commit (has
   zero or >1 parents), a boundary commit, or !TREESAME, it remains.
diff --git a/revision.c b/revision.c
index c88ded8..7535757 100644
--- a/revision.c
+++ b/revision.c
@@ -2119,6 +2119,73 @@ static int mark_redundant_parents(struct rev_info *revs, struct commit *commit)
 	return marked;
 }
 
+/*
+ * Awkward naming - this means one parent we are TREESAME to.
+ * cf mark_treesame_root_parents: root parents that are TREESAME (to an
+ * empty tree). Better name suggestions?
+ */
+static int leave_one_treesame_to_parent(struct rev_info *revs, struct commit *commit)
+{
+	struct treesame_state *ts = lookup_decoration(&revs->treesame, &commit->object);
+	struct commit *unmarked = NULL, *marked = NULL;
+	struct commit_list *p;
+	unsigned n;
+
+	for (p = commit->parents, n = 0; p; p = p->next, n++) {
+		if (ts->treesame[n]) {
+			if (p->item->object.flags & TMP_MARK) {
+				if (!marked)
+					marked = p->item;
+			} else {
+				if (!unmarked) {
+					unmarked = p->item;
+					break;
+				}
+			}
+		}
+	}
+
+	/*
+	 * If we are TREESAME to a marked-for-deletion parent, but not to any
+	 * unmarked parents, unmark the first TREESAME parent. This is the
+	 * parent that the default simplify_history==1 scan would have followed,
+	 * and it doesn't make sense to omit that path when asking for a
+	 * simplified full history. Retaining it improves the chances of
+	 * understanding odd missed merges that took an old version of a file.
+	 *
+	 * Example:
+	 *
+	 *   I---------X       A modified the file, but mainline merge X used
+	 *    \       /        "-s ours", so took the version from I. X is
+	 *     `--A--'         TREESAME to I and !TREESAME to A.
+	 *
+	 * Default log from X would produce "I". Without this check,
+	 * --full-history --simplify-merges would produce "I-A-X", showing
+	 * the merge commit X and that it changed A, but not making clear that
+	 * it had just taken the I version. With this check, the topology above
+	 * is retained.
+	 *
+	 * Note that it is possible that the simplification chooses a different
+	 * TREESAME parent from the default, in which case this test doesn't
+	 * activate, and we _do_ drop the default parent. Example:
+	 *
+	 *   I------X         A modified the file, but it was reverted in B,
+	 *    \    /          meaning mainline merge X is TREESAME to both
+	 *     A--B           parents.
+	 *
+	 * Default log would produce "I" by following the first parent;
+	 * --full-history --simplify-merges will produce "I-A-B". But this is a
+	 * reasonable result - it presents a logical full history leading from
+	 * I to X, and X is not an important merge.
+	 */
+	if (!unmarked && marked) {
+		marked->object.flags &= ~TMP_MARK;
+		return 1;
+	}
+
+	return 0;
+}
+
 static int remove_marked_parents(struct rev_info *revs, struct commit *commit)
 {
 	struct commit_list **pp, *p;
@@ -2222,6 +2289,8 @@ static struct commit_list **simplify_one(struct rev_info *revs, struct commit *c
 	if (1 < cnt) {
 		int marked = mark_redundant_parents(revs, commit);
 		if (marked)
+			marked -= leave_one_treesame_to_parent(revs, commit);
+		if (marked)
 			cnt = remove_marked_parents(revs, commit);
 	}
 
diff --git a/t/t6111-rev-list-treesame.sh b/t/t6111-rev-list-treesame.sh
index efc2442..5cc0c34 100755
--- a/t/t6111-rev-list-treesame.sh
+++ b/t/t6111-rev-list-treesame.sh
@@ -104,7 +104,7 @@ check_result 'M L H B A' -- file
 check_result 'M L H B A' --parents -- file
 check_result 'M L J I H G F D B A' --full-history -- file
 check_result 'M L K J I H G F D B A' --full-history --parents -- file
-check_result 'M H L J I G F B A' --simplify-merges -- file
+check_result 'M H L J I G F B A' --simplify-merges -- file # should check that G has parent B
 check_result 'M L K G F D B A' --first-parent
 check_result 'M L G F B A' --first-parent -- file
 
-- 
1.8.3.rc0.28.g682c2d9

[PATCH v3 8/9] simplify-merges: drop merge from irrelevant side branch

[Kevin Bracey]

Reimplement commit 4b7f53da on top of the new simplify-merges
infrastructure, tightening the condition to only consider root parents;
the original version incorrectly dropped parents that were TREESAME to
anything.

Original log message follows.

The merge simplification rule stated in 6546b59 (revision traversal:
show full history with merge simplification, 2008-07-31) still
treated merge commits too specially.  Namely, in a history with this
shape:

	---o---o---M
	          /
         x---x---x

where three 'x' were on a history completely unrelated to the main
history 'o' and do not touch any of the paths we are following, we
still said that after simplifying all of the parents of M, 'x'
(which is the leftmost 'x' that rightmost 'x simplifies down to) and
'o' (which would be the last commit on the main history that touches
the paths we are following) are independent from each other, and
both need to be kept.

That is incorrect; when the side branch 'x' never touches the paths,
it should be removed to allow M to simplify down to the last commit
on the main history that touches the paths.

Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Kevin Bracey <kevin@bracey.fi>
---
 Documentation/rev-list-options.txt | 34 +++++++++++++++++++++-------------
 revision.c                         | 26 +++++++++++++++++++++++++-
 t/t6012-rev-list-simplify.sh       |  2 +-
 3 files changed, 47 insertions(+), 15 deletions(-)

diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt
index e23bdb0..b7fbc80 100644
--- a/Documentation/rev-list-options.txt
+++ b/Documentation/rev-list-options.txt
@@ -342,13 +342,13 @@ In the following, we will always refer to the same example history to
 illustrate the differences between simplification settings.  We assume
 that you are filtering for a file `foo` in this commit graph:
 -----------------------------------------------------------------------
-	  .-A---M---N---O---P
-	 /     /   /   /   /
-	I     B   C   D   E
-	 \   /   /   /   /
-	  `-------------'
+	  .-A---M---N---O---P---Q
+	 /     /   /   /   /   /
+	I     B   C   D   E   Y
+	 \   /   /   /   /   /
+	  `-------------'   X
 -----------------------------------------------------------------------
-The horizontal line of history A---P is taken to be the first parent of
+The horizontal line of history A---Q is taken to be the first parent of
 each merge.  The commits are:
 
 * `I` is the initial commit, in which `foo` exists with contents
@@ -369,6 +369,10 @@ each merge.  The commits are:
 * `E` changes `quux` to "xyzzy", and its merge `P` combines the
   strings to "quux xyzzy".  `P` is TREESAME to `O`, but not to `E`.
 
+* `X` is an indpendent root commit that added a new file `side`, and `Y`
+  modified it. `Y` is TREESAME to `X`. Its merge `Q` added `side` to `P`, and
+  `Q` is TREESAME to `P`, but not to `Y`.
+
 'rev-list' walks backwards through history, including or excluding
 commits based on whether '\--full-history' and/or parent rewriting
 (via '\--parents' or '\--children') are used.  The following settings
@@ -409,7 +413,7 @@ parent lines.
 	the example, we get
 +
 -----------------------------------------------------------------------
-	I  A  B  N  D  O  P
+	I  A  B  N  D  O  P  Q
 -----------------------------------------------------------------------
 +
 `M` was excluded because it is TREESAME to both parents.  `E`,
@@ -430,7 +434,7 @@ Along each parent, prune away commits that are not included
 themselves.  This results in
 +
 -----------------------------------------------------------------------
-	  .-A---M---N---O---P
+	  .-A---M---N---O---P---Q
 	 /     /   /   /   /
 	I     B   /   D   /
 	 \   /   /   /   /
@@ -440,7 +444,7 @@ themselves.  This results in
 Compare to '\--full-history' without rewriting above.  Note that `E`
 was pruned away because it is TREESAME, but the parent list of P was
 rewritten to contain `E`'s parent `I`.  The same happened for `C` and
-`N`.
+`N`, and `X`, `Y` and `Q`.
 
 In addition to the above settings, you can change whether TREESAME
 affects inclusion:
@@ -470,9 +474,9 @@ history according to the following rules:
 * Set `C'` to `C`.
 +
 * Replace each parent `P` of `C'` with its simplification `P'`.  In
-  the process, drop parents that are ancestors of other parents, and
-  remove duplicates, but take care to never drop all parents that
-  we are TREESAME to.
+  the process, drop parents that are ancestors of other parents or that are
+  root commits TREESAME to an empty tree, and remove duplicates, but take care
+  to never drop all parents that we are TREESAME to.
 +
 * If after this parent rewriting, `C'` is a root or merge commit (has
   zero or >1 parents), a boundary commit, or !TREESAME, it remains.
@@ -490,7 +494,7 @@ The effect of this is best shown by way of comparing to
 	  `---------'
 -----------------------------------------------------------------------
 +
-Note the major differences in `N` and `P` over '--full-history':
+Note the major differences in `N`, `P` and `Q` over '--full-history':
 +
 --
 * `N`'s parent list had `I` removed, because it is an ancestor of the
@@ -498,6 +502,10 @@ Note the major differences in `N` and `P` over '--full-history':
 +
 * `P`'s parent list similarly had `I` removed.  `P` was then
   removed completely, because it had one parent and is TREESAME.
++
+* `Q`'s parent list had `Y` simplified to `X`. `X` was then removed, because it
+  was a TREESAME root. `Q` was then removed completely, because it had one
+  parent and is TREESAME.
 --
 
 Finally, there is a fifth simplification mode available:
diff --git a/revision.c b/revision.c
index 7535757..20c7058 100644
--- a/revision.c
+++ b/revision.c
@@ -2119,6 +2119,22 @@ static int mark_redundant_parents(struct rev_info *revs, struct commit *commit)
 	return marked;
 }
 
+static int mark_treesame_root_parents(struct rev_info *revs, struct commit *commit)
+{
+	struct commit_list *p;
+	int marked = 0;
+
+	for (p = commit->parents; p; p = p->next) {
+		struct commit *parent = p->item;
+		if (!parent->parents && (parent->object.flags & TREESAME)) {
+			parent->object.flags |= TMP_MARK;
+			marked++;
+		}
+	}
+
+	return marked;
+}
+
 /*
  * Awkward naming - this means one parent we are TREESAME to.
  * cf mark_treesame_root_parents: root parents that are TREESAME (to an
@@ -2284,10 +2300,18 @@ static struct commit_list **simplify_one(struct rev_info *revs, struct commit *c
 	 *     /    /		o: a commit that touches the paths;
 	 * ---o----'
 	 *
-	 * Detect and simplify this case.
+	 * Further, a merge of an independent branch that doesn't
+	 * touch the path will reduce to a treesame root parent:
+	 *
+	 *  ----o----X		X: the commit we are looking at;
+	 *          /		o: a commit that touches the paths;
+	 *         r		r: a root commit not touching the paths
+	 *
+	 * Detect and simplify both cases.
 	 */
 	if (1 < cnt) {
 		int marked = mark_redundant_parents(revs, commit);
+		marked += mark_treesame_root_parents(revs, commit);
 		if (marked)
 			marked -= leave_one_treesame_to_parent(revs, commit);
 		if (marked)
diff --git a/t/t6012-rev-list-simplify.sh b/t/t6012-rev-list-simplify.sh
index 4e55872..57ce239 100755
--- a/t/t6012-rev-list-simplify.sh
+++ b/t/t6012-rev-list-simplify.sh
@@ -110,7 +110,7 @@ check_result 'L K J I H G F E D C B A' --full-history
 check_result 'K I H E C B A' --full-history -- file
 check_result 'K I H E C B A' --full-history --topo-order -- file
 check_result 'K I H E C B A' --full-history --date-order -- file
-check_outcome failure 'I E C B A' --simplify-merges -- file
+check_result 'I E C B A' --simplify-merges -- file
 check_result 'I B A' -- file
 check_result 'I B A' --topo-order -- file
 check_result 'H' --first-parent -- another-file
-- 
1.8.3.rc0.28.g682c2d9

[PATCH v3 9/9] revision.c: discount side branches when computing TREESAME

[Kevin Bracey]

Add a BOTTOM flag to commit objects, and use it to define priority for
pruning. Priority commits are those that are !UNINTERESTING or BOTTOM,
and this allows us to identify irrelevant side branches (UNINTERESTING
&& !BOTTOM).

If a merge has priority parents, and it is TREESAME to them, then do
not let non-priority parents cause the merge to be treated as
!TREESAME.

When considering simplification, don't always include all merges -
merges with exactly 1 priority parent can be simplified, if TREESAME
according to the above rule.

These two changes greatly increase simplification in limited revision
lists - irrelevant merges from unlisted commits can be omitted.

Signed-off-by: Kevin Bracey <kevin@bracey.fi>
---
 revision.c                        | 201 ++++++++++++++++++++++++++++++++------
 revision.h                        |   3 +-
 t/t6019-rev-list-ancestry-path.sh |  12 ++-
 t/t6111-rev-list-treesame.sh      |  22 ++---
 4 files changed, 195 insertions(+), 43 deletions(-)

diff --git a/revision.c b/revision.c
index 20c7058..99a3432 100644
--- a/revision.c
+++ b/revision.c
@@ -333,6 +333,76 @@ static int everybody_uninteresting(struct commit_list *orig)
 }
 
 /*
+ * A definition of "priority" commit that we can use to simplify limited graphs
+ * by eliminating side branches.
+ *
+ * A "priority" commit is one that is !UNINTERESTING (ie we are including it
+ * in our list), or that is a specified BOTTOM commit. Then after computing
+ * a limited list, during processing we can generally ignore boundary merges
+ * coming from outside the graph, (ie from non-priority parents), and treat
+ * those merges as if they were single-parent. TREESAME is defined to consider
+ * only priority parents, if any. If we are TREESAME to our on-graph parents,
+ * we don't care if we were !TREESAME to non-graph parents.
+ *
+ * Including bottom commits in "priority" ensures that a limited graph's
+ * connection to the actual bottom commit is not viewed as an uninteresting
+ * side branch, but treated as part of the graph. For example:
+ *
+ *   ....Z...A---X---o---o---B
+ *        .     /
+ *         W---Y
+ *
+ * When computing "A..B", the A-X connection is at least as important as
+ * Y-X, despite A being flagged UNINTERESTING.
+ *
+ * And when computing --ancestry-path "A..B", the A-X connection is more
+ * important than Y-X, despite both A and Y being flagged UNINTERESTING.
+ */
+static inline int priority_commit(struct commit *commit)
+{
+	return (commit->object.flags & (UNINTERESTING | BOTTOM)) != UNINTERESTING;
+}
+
+/*
+ * Return a single priority commit from a parent list. If we are a TREESAME
+ * commit, and this selects one of our parents, then we can safely simplify to
+ * that parent.
+ *
+ * For 1-parent commits, or if first-parent-only, then this returns the only
+ * parent. TREESAME will have been set purely on that parent.
+ *
+ * For multi-parent commits, identify a sole priority parent, if any.
+ * If we have only one priority parent, then TREESAME will be set purely
+ * with regard to that parent, and we can choose simplification appropriately.
+ *
+ * If we have more than one priority parent, or no priority parents
+ * (and multiple non-priority ones), then we can't select a parent here and
+ * return NULL.
+ */
+static struct commit *priority_select_parent(struct rev_info *revs, struct commit_list *orig)
+{
+	struct commit_list *list = orig;
+	struct commit *priority = NULL;
+
+	if (!orig)
+		return NULL;
+
+	if (revs->first_parent_only || !orig->next)
+		return orig->item;
+
+	while (list) {
+		struct commit *commit = list->item;
+		list = list->next;
+		if (priority_commit(commit)) {
+			if (priority)
+				return NULL;
+			priority = commit;
+		}
+	}
+	return priority;
+}
+
+/*
  * The goal is to get REV_TREE_NEW as the result only if the
  * diff consists of all '+' (and no other changes), REV_TREE_OLD
  * if the whole diff is removal of old data, and otherwise
@@ -502,27 +572,52 @@ static unsigned update_treesame(struct rev_info *revs, struct commit *commit)
 	if (commit->parents && commit->parents->next) {
 		unsigned n;
 		struct treesame_state *st;
+		struct commit_list *p;
+		unsigned priority_parents;
+		unsigned priority_change, nonpriority_change;
 
 		st = lookup_decoration(&revs->treesame, &commit->object);
 		if (!st)
 			die("update_treesame %s", sha1_to_hex(commit->object.sha1));
-		commit->object.flags |= TREESAME;
-		for (n = 0; n < st->nparents; n++) {
-			if (!st->treesame[n]) {
-				commit->object.flags &= ~TREESAME;
-				break;
-			}
+		priority_parents = 0;
+		priority_change = nonpriority_change = 0;
+		for (p = commit->parents, n = 0; p; n++, p = p->next) {
+			if (priority_commit(p->item)) {
+				priority_change |= !st->treesame[n];
+				priority_parents++;
+			} else
+				nonpriority_change |= !st->treesame[n];
 		}
+		if (priority_parents ? priority_change : nonpriority_change)
+			commit->object.flags &= ~TREESAME;
+		else
+			commit->object.flags |= TREESAME;
 	}
 
 	return commit->object.flags & TREESAME;
 }
 
+static inline int limiting_can_increase_treesame(const struct rev_info *revs)
+{
+	/*
+	 * TREESAME is irrelevant unless prune && dense;
+	 * if simplify_history is set, we can't have a mixture of TREESAME and
+	 *    !TREESAME INTERESTING parents (and we don't have treesame[]
+	 *    decoration anyway);
+	 * if first_parent_only is set, then the TREESAME flag is locked
+	 *    against the first parent (and again we lack treesame[] decoration).
+	 */
+	return revs->prune && revs->dense &&
+	       !revs->simplify_history &&
+	       !revs->first_parent_only;
+}
+
 static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 {
 	struct commit_list **pp, *parent;
 	struct treesame_state *ts = NULL;
-	int tree_changed = 0, nth_parent;
+	int priority_change = 0, nonpriority_change = 0;
+	int priority_parents, nth_parent;
 
 	/*
 	 * If we don't do pruning, everything is interesting
@@ -546,10 +641,12 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 	if (!revs->dense && !commit->parents->next)
 		return;
 
-	for (pp = &commit->parents, nth_parent = 0;
+	for (pp = &commit->parents, nth_parent = 0, priority_parents = 0;
 	     (parent = *pp) != NULL;
 	     pp = &parent->next, nth_parent++) {
 		struct commit *p = parent->item;
+		if (priority_commit(p))
+			priority_parents++;
 
 		if (nth_parent == 1) {
 			/*
@@ -573,7 +670,7 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 			    !revs->simplify_history &&
 			    !(commit->object.flags & UNINTERESTING)) {
 				ts = initialise_treesame(revs, commit);
-				if (!tree_changed)
+				if (!(nonpriority_change || priority_change))
 					ts->treesame[0] = 1;
 			}
 		}
@@ -583,7 +680,7 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 			    sha1_to_hex(p->object.sha1));
 		switch (rev_compare_tree(revs, p, commit)) {
 		case REV_TREE_SAME:
-			if (!revs->simplify_history || (p->object.flags & UNINTERESTING)) {
+			if (!revs->simplify_history || !priority_commit(p)) {
 				/* Even if a merge with an uninteresting
 				 * side branch brought the entire change
 				 * we are interested in, we do not want
@@ -619,14 +716,27 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 		/* fallthrough */
 		case REV_TREE_OLD:
 		case REV_TREE_DIFFERENT:
-			tree_changed = 1;
+			if (priority_commit(p))
+				priority_change = 1;
+			else
+				nonpriority_change = 1;
 			continue;
 		}
 		die("bad tree compare for commit %s", sha1_to_hex(commit->object.sha1));
 	}
-	if (tree_changed)
-		return;
-	commit->object.flags |= TREESAME;
+
+	/*
+	 * TREESAME is straightforward for single-parent commits. For merge
+	 * commits, it is most useful to define it so that "non-priority"
+	 * parents cannot make us !TREESAME - if we have any priority
+	 * parents, then we only consider TREESAMEness with respect to them,
+	 * allowing irrelevant merges from uninteresting branches to be
+	 * simplified away. Only if we have only nonpriority parents do we
+	 * base TREESAME on them. Note that this logic is replicated in
+	 * update_treesame, which should be kept in sync.
+	 */
+	if (priority_parents ? !priority_change : !nonpriority_change)
+		commit->object.flags |= TREESAME;
 }
 
 static void commit_list_insert_by_date_cached(struct commit *p, struct commit_list **head,
@@ -1002,6 +1112,18 @@ static int limit_list(struct rev_info *revs)
 		free_commit_list(bottom);
 	}
 
+	/*
+	 * Check if any commits have become TREESAME by some of their parents
+	 * becoming UNINTERESTING.
+	 */
+	if (limiting_can_increase_treesame(revs))
+		for (list = newlist; list; list = list->next) {
+			struct commit *c = list->item;
+			if (c->object.flags & (UNINTERESTING | TREESAME))
+				continue;
+			update_treesame(revs, c);
+		}
+
 	revs->commits = newlist;
 	return 0;
 }
@@ -1015,6 +1137,16 @@ static void add_rev_cmdline(struct rev_info *revs,
 	struct rev_cmdline_info *info = &revs->cmdline;
 	int nr = info->nr;
 
+	/*
+	 * Total hack, but at least doing it here means we're consistent
+	 * with collect_bottom_commits(). But if we just flagged BOTTOM
+	 * commits before this, could we ditch this entire function?
+	 */
+	if (item->type == OBJ_COMMIT && (flags & UNINTERESTING)) {
+		item->flags |= BOTTOM;
+		flags |= BOTTOM;
+	}
+
 	ALLOC_GROW(info->rev, nr + 1, info->alloc);
 	info->rev[nr].item = item;
 	info->rev[nr].name = name;
@@ -2233,6 +2365,7 @@ static int remove_marked_parents(struct rev_info *revs, struct commit *commit)
 static struct commit_list **simplify_one(struct rev_info *revs, struct commit *commit, struct commit_list **tail)
 {
 	struct commit_list *p;
+	struct commit *parent;
 	struct merge_simplify_state *st, *pst;
 	int cnt;
 
@@ -2321,19 +2454,20 @@ static struct commit_list **simplify_one(struct rev_info *revs, struct commit *c
 	/*
 	 * A commit simplifies to itself if it is a root, if it is
 	 * UNINTERESTING, if it touches the given paths, or if it is a
-	 * merge and its parents simplify to more than one commit
+	 * merge and its parents don't simplify to one priority commit
 	 * (the first two cases are already handled at the beginning of
 	 * this function).
 	 *
-	 * Otherwise, it simplifies to what its sole parent simplifies to.
+	 * Otherwise, it simplifies to what its sole priority parent
+	 * simplifies to.
 	 */
 	if (!cnt ||
 	    (commit->object.flags & UNINTERESTING) ||
 	    !(commit->object.flags & TREESAME) ||
-	    (1 < cnt))
+	    (parent = priority_select_parent(revs, commit->parents)) == NULL)
 		st->simplified = commit;
 	else {
-		pst = locate_simplify_state(revs, commit->parents->item);
+		pst = locate_simplify_state(revs, parent);
 		st->simplified = pst->simplified;
 	}
 	return tail;
@@ -2430,7 +2564,8 @@ int prepare_revision_walk(struct rev_info *revs)
 		free(list);
 
 	/* Signal whether we need per-parent treesame decoration */
-	if (revs->simplify_merges)
+	if (revs->simplify_merges ||
+	    (revs->limited && limiting_can_increase_treesame(revs)))
 		revs->treesame.name = "treesame";
 
 	if (revs->no_walk != REVISION_WALK_NO_WALK_UNSORTED)
@@ -2464,15 +2599,15 @@ static enum rewrite_result rewrite_one(struct rev_info *revs, struct commit **pp
 		if (!revs->limited)
 			if (add_parents_to_list(revs, p, &revs->commits, &cache) < 0)
 				return rewrite_one_error;
-		if (p->parents && p->parents->next)
-			return rewrite_one_ok;
 		if (p->object.flags & UNINTERESTING)
 			return rewrite_one_ok;
 		if (!(p->object.flags & TREESAME))
 			return rewrite_one_ok;
 		if (!p->parents)
 			return rewrite_one_noparents;
-		*pp = p->parents->item;
+		if ((p = priority_select_parent(revs, p->parents)) == NULL)
+			return rewrite_one_ok;
+		*pp = p;
 	}
 }
 
@@ -2616,10 +2751,7 @@ enum commit_action get_commit_action(struct rev_info *revs, struct commit *commi
 	if (revs->min_age != -1 && (commit->date > revs->min_age))
 		return commit_ignore;
 	if (revs->min_parents || (revs->max_parents >= 0)) {
-		int n = 0;
-		struct commit_list *p;
-		for (p = commit->parents; p; p = p->next)
-			n++;
+		int n = commit_list_count(commit->parents);
 		if ((n < revs->min_parents) ||
 		    ((revs->max_parents >= 0) && (n > revs->max_parents)))
 			return commit_ignore;
@@ -2629,12 +2761,23 @@ enum commit_action get_commit_action(struct rev_info *revs, struct commit *commi
 	if (revs->prune && revs->dense) {
 		/* Commit without changes? */
 		if (commit->object.flags & TREESAME) {
+			int n;
+			struct commit_list *p;
 			/* drop merges unless we want parenthood */
 			if (!want_ancestry(revs))
 				return commit_ignore;
-			/* non-merge - always ignore it */
-			if (!commit->parents || !commit->parents->next)
-				return commit_ignore;
+			/*
+			 * If we want ancestry, then need to keep any merges
+			 * between priority commits to tie together topology.
+			 * For consistency with TREESAME and simplification.
+			 * use "priority" here rather than just INTERESTING,
+			 * to treat bottom commit(s) as part of the topology.
+			 */
+			for (n = 0, p = commit->parents; p; p = p->next)
+				if (priority_commit(p->item))
+					if (++n >= 2)
+						return commit_show;
+			return commit_ignore;
 		}
 	}
 	return commit_show;
diff --git a/revision.h b/revision.h
index 1abe57b..765630a 100644
--- a/revision.h
+++ b/revision.h
@@ -15,7 +15,8 @@
 #define ADDED		(1u<<7)	/* Parents already parsed and added? */
 #define SYMMETRIC_LEFT	(1u<<8)
 #define PATCHSAME	(1u<<9)
-#define ALL_REV_FLAGS	((1u<<10)-1)
+#define BOTTOM		(1u<<10)
+#define ALL_REV_FLAGS	((1u<<11)-1)
 
 #define DECORATE_SHORT_REFS	1
 #define DECORATE_FULL_REFS	2
diff --git a/t/t6019-rev-list-ancestry-path.sh b/t/t6019-rev-list-ancestry-path.sh
index ebe79ac..c5a412c 100755
--- a/t/t6019-rev-list-ancestry-path.sh
+++ b/t/t6019-rev-list-ancestry-path.sh
@@ -15,7 +15,8 @@ test_description='--ancestry-path'
 #  --ancestry-path D..M -- M.t == M
 #
 #  G..M -- G.t                 == [nothing - was dropped in "-s ours" merge L]
-#  --ancestry-path G..M -- G.t == H J L
+#  --ancestry-path G..M -- G.t == L
+#  --ancestry-path --simplify-merges G^..M -- G.t == G L
 
 . ./test-lib.sh
 
@@ -82,8 +83,15 @@ test_expect_success 'rev-list G..M -- G.t' '
 '
 
 test_expect_success 'rev-list --ancestry-path G..M -- G.t' '
-	for c in H J L; do echo $c; done >expect &&
+	echo L >expect &&
 	git rev-list --ancestry-path --format=%s G..M -- G.t |
+	sed -e "/^commit /d" >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'rev-list --ancestry-path --simplify-merges G^..M -- G.t' '
+	for c in G L; do echo $c; done >expect &&
+	git rev-list --ancestry-path --simplify-merges --format=%s G^..M -- G.t |
 	sed -e "/^commit /d" |
 	sort >actual &&
 	test_cmp expect actual
diff --git a/t/t6111-rev-list-treesame.sh b/t/t6111-rev-list-treesame.sh
index 5cc0c34..689d357 100755
--- a/t/t6111-rev-list-treesame.sh
+++ b/t/t6111-rev-list-treesame.sh
@@ -126,16 +126,16 @@ check_result 'M L G' F..M --first-parent -- file
 # Note that G is pruned when E is the bottom, even if it's the same commit list
 # If we want history since E, then we're quite happy to ignore G that took E.
 check_result 'M L K J I H G' E..M --ancestry-path
-check_outcome failure 'M L J I H' E..M --ancestry-path -- file # includes G
-check_outcome failure 'M L K J I H' E..M --ancestry-path --parents -- file
-check_outcome failure 'M H L J I' E..M --ancestry-path --simplify-merges -- file # includes G
+check_result 'M L J I H' E..M --ancestry-path -- file
+check_result 'M L K J I H' E..M --ancestry-path --parents -- file
+check_result 'M H L J I' E..M --ancestry-path --simplify-merges -- file
 
 # Should still be able to ignore I-J branch in simple log, despite limiting
 # to G.
 check_result 'M L K J I H' G..M
 check_result 'M H L K J I' G..M --topo-order
-check_outcome failure 'M L H' G..M -- file # includes J I
-check_outcome failure 'M L H' G..M --parents -- file # includes J I
+check_result 'M L H' G..M -- file
+check_result 'M L H' G..M --parents -- file
 check_result 'M L J I H' G..M --full-history -- file
 check_result 'M L K J I H' G..M --full-history --parents -- file
 check_result 'M H L J I' G..M --simplify-merges -- file
@@ -149,15 +149,15 @@ check_result 'M H L J I' G..M --ancestry-path --simplify-merges -- file
 # But --full-history shouldn't drop D on its own - without simplification,
 # we can't decide if the merge from INTERESTING commit C was sensible.
 check_result 'F D C' B..F
-check_outcome failure 'F' B..F -- file # includes D
-check_outcome failure 'F' B..F --parents -- file # includes D
+check_result 'F' B..F -- file
+check_result 'F' B..F --parents -- file
 check_result 'F D' B..F --full-history -- file
 check_result 'F D' B..F --full-history --parents -- file
 check_result 'F' B..F --simplify-merges -- file
 check_result 'F D' B..F --ancestry-path
-check_outcome failure 'F' B..F --ancestry-path -- file # includes D
-check_outcome failure 'F' B..F --ancestry-path --parents -- file # includes D
-check_outcome failure 'F' B..F --ancestry-path --simplify-merges -- file # includes D
+check_result 'F' B..F --ancestry-path -- file
+check_result 'F' B..F --ancestry-path --parents -- file
+check_result 'F' B..F --ancestry-path --simplify-merges -- file
 check_result 'F D' B..F --first-parent
 check_result 'F' B..F --first-parent -- file
 
@@ -166,7 +166,7 @@ check_result 'F' B..F --first-parent -- file
 check_result 'F D B' C..F
 check_result 'F B' C..F -- file
 check_result 'F B' C..F --parents -- file
-check_outcome failure 'F D B' C..F --full-history -- file # drops D
+check_result 'F D B' C..F --full-history -- file
 check_result 'F D B' C..F --full-history --parents -- file
 check_result 'F D B' C..F --simplify-merges -- file
 check_result 'F D' C..F --ancestry-path
-- 
1.8.3.rc0.28.g682c2d9

[PATCH v3 10/9] revision.c: treat A...B merge bases as if manually specified

[Kevin Bracey]

The documentation assures users that "A...B" is defined as 'r1 r2 --not
$(git merge-base --all r1 r2)'. This isn't in fact quite true, because
the calculated merge bases are not sent to add_rev_cmdline().

Previously, the effect was pretty minor - all that I can think of is
that "git rev-list --ancestry-path A B ^AB_base" works, but "git
rev-list --ancestry-path A...B" fails with a "no bottom commits" error.

But now that all history walking cares about bottom commits, this
failure to note the merge bases as bottoms can lead to worse results for
"A...B" compared to "A B ^AB_base".

So ensure that the calculated merge bases are sent to add_rev_cmdline(),
flagged as 'REV_CMD_MERGE_BASE'.

Signed-off-by: Kevin Bracey <kevin@bracey.fi>
---
 revision.c                   | 9 +++++++--
 revision.h                   | 2 ++
 t/t6111-rev-list-treesame.sh | 4 ++++
 3 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/revision.c b/revision.c
index 99a3432..aad16a6 100644
--- a/revision.c
+++ b/revision.c
@@ -1305,12 +1305,16 @@ void init_revisions(struct rev_info *revs, const char *prefix)
 
 static void add_pending_commit_list(struct rev_info *revs,
                                     struct commit_list *commit_list,
+                                    int whence,
                                     unsigned int flags)
 {
 	while (commit_list) {
 		struct object *object = &commit_list->item->object;
+		const char *sha1 = sha1_to_hex(object->sha1);
 		object->flags |= flags;
-		add_pending_object(revs, object, sha1_to_hex(object->sha1));
+		if (whence != REV_CMD_NONE)
+			add_rev_cmdline(revs, object, sha1, whence, flags);
+		add_pending_object(revs, object, sha1);
 		commit_list = commit_list->next;
 	}
 }
@@ -1332,7 +1336,7 @@ static void prepare_show_merge(struct rev_info *revs)
 	add_pending_object(revs, &head->object, "HEAD");
 	add_pending_object(revs, &other->object, "MERGE_HEAD");
 	bases = get_merge_bases(head, other, 1);
-	add_pending_commit_list(revs, bases, UNINTERESTING);
+	add_pending_commit_list(revs, bases, REV_CMD_MERGE_BASE, UNINTERESTING);
 	free_commit_list(bases);
 	head->object.flags |= SYMMETRIC_LEFT;
 
@@ -1420,6 +1424,7 @@ int handle_revision_arg(const char *arg_, struct rev_info *revs, int flags, unsi
 			if (symmetric) {
 				exclude = get_merge_bases(a, b, 1);
 				add_pending_commit_list(revs, exclude,
+							REV_CMD_MERGE_BASE,
 							flags_exclude);
 				free_commit_list(exclude);
 				a_flags = flags | SYMMETRIC_LEFT;
diff --git a/revision.h b/revision.h
index 765630a..b2ac5a3 100644
--- a/revision.h
+++ b/revision.h
@@ -32,10 +32,12 @@ struct rev_cmdline_info {
 		struct object *item;
 		const char *name;
 		enum {
+			REV_CMD_NONE,
 			REV_CMD_REF,
 			REV_CMD_PARENTS_ONLY,
 			REV_CMD_LEFT,
 			REV_CMD_RIGHT,
+			REV_CMD_MERGE_BASE,
 			REV_CMD_REV
 		} whence;
 		unsigned flags;
diff --git a/t/t6111-rev-list-treesame.sh b/t/t6111-rev-list-treesame.sh
index 689d357..ded0b1a 100755
--- a/t/t6111-rev-list-treesame.sh
+++ b/t/t6111-rev-list-treesame.sh
@@ -161,6 +161,10 @@ check_result 'F' B..F --ancestry-path --simplify-merges -- file
 check_result 'F D' B..F --first-parent
 check_result 'F' B..F --first-parent -- file
 
+# E...F should be equivalent to E F ^B, and be able to drop D as above.
+check_result 'F' E F ^B -- file
+check_result 'F' E...F -- file
+
 # Any sort of full history of C..F should show D, as it's the connection to C,
 # and it differs from it.
 check_result 'F D B' C..F
-- 
1.8.3.rc0.28.g4b02ef5

Re: [PATCH v3 3/9] t6111: new TREESAME test set

[Junio C Hamano]

Kevin Bracey <kevin@bracey.fi> writes:

> Some side branching and odd merging to illustrate various flaws in
> revision list scans, particularly when limiting the list.
>
> Many expected failures, which will be gone by the end of the "history
> traversal refinements" series.
>
> Signed-off-by: Kevin Bracey <kevin@bracey.fi>
> ---
>  t/t6111-rev-list-treesame.sh | 180 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 180 insertions(+)
>  create mode 100755 t/t6111-rev-list-treesame.sh
>
> diff --git a/t/t6111-rev-list-treesame.sh b/t/t6111-rev-list-treesame.sh
> new file mode 100755
> index 0000000..602c02d
> --- /dev/null
> +++ b/t/t6111-rev-list-treesame.sh
> @@ -0,0 +1,180 @@
> +#!/bin/sh
> +#
> +#        ,---E--.   *H----------.             * marks !TREESAME parent paths
> +#       /        \ /             \*
> +# *A--*B---D--*F-*G---------K-*L-*M
> +#   \     /*       \       /
> +#    `-C-'          `-*I-*J
> +#
> +# A creates "file", B and F change it.
> +# Odd merge G takes the old version from B.
> +# I changes it, but J reverts it.
> +# H and L both change it, and M merges those changes.

... and because J is a revert of I, K ends up merging the same and
being the same with both G and J.

> +test_description='TREESAME and limiting'
> +
> +. ./test-lib.sh
> +
> +note () {
> +	git tag "$1"
> +}
> +
> +unnote () {
> +	git name-rev --tags --stdin | sed -e "s|$_x40 (tags/\([^)]*\)) |\1 |g"
> +}
> +
> +test_expect_success setup '
> +	test_commit "Initial file" file "Hi there" A &&
> +	git branch other-branch &&
> +
> +	test_commit "file=Hello" file "Hello" B &&
> +	git branch third-branch &&
> +
> +	git checkout other-branch &&
> +	test_commit "Added other" other "Hello" C &&
> +
> +	git checkout master &&
> +	test_merge D other-branch &&
> +
> +	git checkout third-branch &&
> +	test_commit "Third file" third "Nothing" E &&
> +
> +	git checkout master &&
> +	test_commit "file=Blah" file "Blah" F &&
> +
> +	test_tick && git merge --no-commit third-branch &&
> +	git checkout third-branch file &&
> +	git commit &&
> +	note G &&
> +	git branch fiddler-branch &&
> +
> +	git checkout -b part2-branch &&
> +	test_commit "file=Part 2" file "Part 2" H &&
> +
> +	git checkout fiddler-branch &&
> +	test_commit "Bad commit" file "Silly" I &&
> +
> +	test_tick && git revert I && note J &&
> +
> +	git checkout master &&
> +	test_tick && git merge --no-ff fiddler-branch &&
> +	note K
> +
> +	test_commit "file=Part 1" file "Part 1" L &&
> +
> +	test_tick && test_must_fail git merge part2-branch &&
> +	test_commit M file "Parts 1+2"
> +'
> +
> +FMT='tformat:%P 	%H | %s'
> +
> +# could we soup this up to optionally check parents? So "(BA)C" would check
> +# that C is shown and has parents B A.

Sounds like a good thing to do, especially given that some of the
earlier issues you brought up triggered only when --parents is not
used, so...

Perhaps something like the attached.

> +check_outcome () {
> +	outcome=$1
> +	shift
> +	for c in $1
> +	do
> +		echo "$c"
> +	done >expect &&

Maybe

	printf "%s\n" $1 >expect

is more fashionable these days?

> +# Odd merge G drops a change in F. Important that G is listed in all
> +# except the most basic list. Achieving this means normal merge D will also be
> +# shown in normal full-history, as we can't distinguish unless we do a
> +# simplification pass. After simplification, D is dropped but G remains.
> +check_result 'M L K J I H G F E D C B A'

I'll read the expectation and comment on them in a separate message.

 t/t6111-rev-list-treesame.sh | 30 +++++++++++++++++++++---------
 1 file changed, 21 insertions(+), 9 deletions(-)

diff --git a/t/t6111-rev-list-treesame.sh b/t/t6111-rev-list-treesame.sh
index 602c02d..f4c9cac 100755
--- a/t/t6111-rev-list-treesame.sh
+++ b/t/t6111-rev-list-treesame.sh
@@ -20,7 +20,7 @@ note () {
 }
 
 unnote () {
-	git name-rev --tags --stdin | sed -e "s|$_x40 (tags/\([^)]*\)) |\1 |g"
+	git name-rev --tags --stdin | sed -e "s|$_x40 (tags/\([^)]*\))\([ 	]\)|\1\2|g"
 }
 
 test_expect_success setup '
@@ -66,23 +66,34 @@ test_expect_success setup '
 	test_commit M file "Parts 1+2"
 '
 
-FMT='tformat:%P 	%H | %s'
-
 # could we soup this up to optionally check parents? So "(BA)C" would check
 # that C is shown and has parents B A.
 check_outcome () {
 	outcome=$1
 	shift
-	for c in $1
-	do
-		echo "$c"
-	done >expect &&
-	shift &&
+
+	case "$1" in
+	*"("*)
+		FMT="%P	%H | %s"
+		munge_actual="
+			s/^\([^	]*\)	\([^ ]*\) .*/(\1)\2/
+			s/ //g
+			s/()//
+		"
+		;;
+	*)
+		FMT="%H | %s"
+		munge_actual="s/^\([^ ]*\) .*/\1/"
+		;;
+	esac &&
+	printf "%s\n" $1 >expect &&
+	shift
+
 	param="$*" &&
 	test_expect_$outcome "log $param" '
 		git log --format="$FMT" $param |
 		unnote >actual &&
-		sed -e "s/^.*	\([^ ]*\) .*/\1/" >check <actual &&
+		sed -e "$munge_actual" <actual >check &&
 		test_cmp expect check || {
 			cat actual
 			false
@@ -99,6 +110,7 @@ check_result () {
 # shown in normal full-history, as we can't distinguish unless we do a
 # simplification pass. After simplification, D is dropped but G remains.
 check_result 'M L K J I H G F E D C B A'
+check_result '(LH)M (K)L (GJ)K (I)J (G)I (G)H (FE)G (D)F (B)E (BC)D (A)C (A)B A'
 check_result 'M H L K J I G E F D C B A' --topo-order
 check_result 'M L H B A' -- file
 check_result 'M L H B A' --parents -- file

Re: [PATCH v3 3/9] t6111: new TREESAME test set

[Junio C Hamano]

Kevin Bracey <kevin@bracey.fi> writes:

> +#        ,---E--.   *H----------.             * marks !TREESAME parent paths
> +#       /        \ /             \*
> +# *A--*B---D--*F-*G---------K-*L-*M
> +#   \     /*       \       /
> +#    `-C-'          `-*I-*J
> +#
> +# A creates "file", B and F change it.
> +# Odd merge G takes the old version from B.
> +# I changes it, but J reverts it.
> +# H and L both change it, and M merges those changes.
> + ...
> +# Odd merge G drops a change in F. Important that G is listed in all
> +# except the most basic list. Achieving this means normal merge D will also be
> +# shown in normal full-history, as we can't distinguish unless we do a
> +# simplification pass. After simplification, D is dropped but G remains.
> +check_result 'M L K J I H G F E D C B A'

OK.

> +check_result 'M H L K J I G E F D C B A' --topo-order

OK.

> +check_result 'M L H B A' -- file

OK.

> +check_result 'M L H B A' --parents -- file

OK.

> +check_outcome failure 'M L J I H G F D B A' --full-history -- file # drops G

This breaks the promise that full history at least gives a connected
graph.

> +check_result 'M L K J I H G F D B A' --full-history --parents -- file

And it is saved by --parents; the "(parents)commit" notation may
make it easier to read the above two tests.

> +check_outcome failure 'M H L J I G F B A' --simplify-merges -- file # drops G

The hsitory simplify merges gives ought to be a full history that
keeps any commit that locally touches the specified path, so keeping
I and J even though they do not contribute to the end result is
correct.  The path in question is the same between G and B, but
again, F should be shown because it should be in the full history.

Even though E simplifies down to B, making G a merge between B and F
(the former is an ancestor of the latter), G is kept, because...?

Because the path at B and F are different?

I have to wonder what should happen if D makes path different from B,
and then F makes path the same as B.  B and F should still be kept.

So "because the path at B and F are different" is not the reason why
we keep G.  How do we decide why G must be kept?

> +check_result 'M L K G F D B A' --first-parent
> +check_result 'M L G F B A' --first-parent -- file

OK.

> +# Check that odd merge G remains shown when F is the bottom.

I did not have a chance to say this when you responded in the
previous round with that "bottom" thing before you sent this new
round, but I am not sure if it is a good idea to pay special
attention to the "bottom"-ness.  Very often, instead of running "git
log -- path" and stopping when seeing enough by killing the pager,
people run "git log HEAD~200.. -- path" to pick "recent enough"
commits, but the number 200 is very much arbitrary.

> +check_result 'M L K J I H G E' F..M

OK.

> +check_result 'M H L K J I G E' F..M --topo-order

OK.  But a note about style: options should come before revision
ranges.

> +check_result 'M L H' F..M -- file

OK.  It is still one possible history to explain what we have in the
path at the end.

> +check_result 'M L H' F..M --parents -- file # L+H's parents rewritten to B, so more useful than it may seem

OK.

> +check_outcome failure 'M L J I H G' F..M --full-history -- file # drops G

G is same as E which is in the graph (F and B are the boundaries of
the history, and F is no special than B in that regard).  You want
to show a merge that is different from at least one parent (as
opposed to showing only when it is different from all parents), so
you want to show G.  OK.

> +check_result 'M L K J I H G' F..M --full-history --parents -- file

OK.

It almost makes me suspect that the --full-history option should
flip revs->rewrite_parents on (but not revs->print_parents, unless
the --parents option asks us to) when the option is parsed.

> +check_outcome failure 'M H L J I G' F..M --simplify-merges -- file # drops G

The same as the "--full-history without --parents" before.

> +check_result 'M L K J I H G' F..M --ancestry-path
> +check_outcome failure 'M L J I H G' F..M --ancestry-path -- file # drops G

With --ancestry-path, what G does relative to E becomes irrelevant,
because E is not in the F..M graph.  G changes path relative to its
remaining parent F, so it should be shown.  OK.

> +check_result 'M L K J I H G' F..M --ancestry-path --parents -- file

OK.  Again this makes me suspect --ancestry-path should flip
revs->rewrite_parents on.

> +check_result 'M H L J I G' F..M --ancestry-path --simplify-merges -- file

> +check_result 'M L K G' F..M --first-parent
> +check_result 'M L G' F..M --first-parent -- file
> +
> +# Note that G is pruned when E is the bottom, even if it's the same commit list
> +# If we want history since E, then we're quite happy to ignore G that took E.
> +check_result 'M L K J I H G' E..M --ancestry-path

OK.

> +check_result 'M L J I H' E..M --ancestry-path -- file

Because F is outside our graph with --ancestry-path, G does not
bring anything new to path relative to its remaining parent E and
should not be shown.  OK.

> +check_outcome failure 'M L K J I H' E..M --ancestry-path --parents -- file

This fails in what way?  G is shown, because the !treesame with now
irrelevant F comes into play?

> +check_outcome failure 'M H L J I' E..M --ancestry-path --simplify-merges -- file # includes G

Makes me wonder how "--ancestry-path --full-history" would work with
this range and with/without the pathspec.

> +# Should still be able to ignore I-J branch in simple log, despite limiting
> +# to G.
> +check_result 'M L K J I H' G..M
> +check_result 'M H L K J I' G..M --topo-order

OK.

> +check_outcome failure 'M L H' G..M -- file # includes J I
> +check_outcome failure 'M L H' G..M --parents -- file # includes J I

I am not sure if it should be a failure or your expectation is
wrong.  G is outside the graph, so as far as the remainder of the
graph is concerned, J is the sole remaining parent of K and I and J
did change the path in question.  

What makes you think I and J should be excluded in these cases?

Shouldn't the same logic as "--ancestry-path [EF]..M" we saw earlier
apply here?  That is, "-a-p E..M" makes F the sole remaining parent
of G and G does change the path from F so it should be shown, while
"-a-p F..M" makes E the sole parent of G, and G does not change the
path from E, so it should not be shown.

> +check_result 'M L J I H' G..M --full-history -- file
> +check_result 'M L K J I H' G..M --full-history --parents -- file
> +check_result 'M H L J I' G..M --simplify-merges -- file
> +check_result 'M L K J I H' G..M --ancestry-path
> +check_result 'M L J I H' G..M --ancestry-path -- file
> +check_result 'M L K J I H' G..M --ancestry-path --parents -- file
> +check_result 'M H L J I' G..M --ancestry-path --simplify-merges -- file
> +
> +# B..F should be able to simplify the merge D from irrelevant side branch C.
> +# Default log should also be free to follow B-D, and ignore C.
> +# But --full-history shouldn't drop D on its own - without simplification,
> +# we can't decide if the merge from INTERESTING commit C was sensible.
> +check_result 'F D C' B..F
> +check_result 'F' B..F -- file
> +check_outcome failure 'F' B..F --parents -- file # includes D

OK.  --parents should not affect the outcome.

> +check_outcome failure 'F D' B..F --full-history -- file # drops D prematurely

OK.

> +check_result 'F D' B..F --full-history --parents -- file

Again, OK.

> +check_result 'F' B..F --simplify-merges -- file
> +check_result 'F D' B..F --ancestry-path
> +check_result 'F' B..F --ancestry-path -- file

OK.

> +check_outcome failure 'F' B..F --ancestry-path --parents -- file # includes D
> +check_outcome failure 'F' B..F --ancestry-path --simplify-merges -- file # includes D

OK.  "-a-p B..F" makes C outside our graph, and D does not change
the path relative to its sole remaining parent B, so it should not
appear in the result.

> +check_result 'F D' B..F --first-parent
> +check_result 'F' B..F --first-parent -- file

OK.

> +# Any sort of full history of C..F should show D, as it's the connection to C,
> +# and it differs from it.
> +check_result 'F D B' C..F
> +check_result 'F B' C..F -- file

OK, as A is a boundary and its child B changes the path from it.

> +check_result 'F B' C..F --parents -- file

OK.

> +check_outcome failure 'F D B' C..F --full-history -- file # drops D

OK.

> +check_result 'F D B' C..F --full-history --parents -- file
> +check_result 'F D B' C..F --simplify-merges -- file
> +check_result 'F D' C..F --ancestry-path
> +check_outcome failure 'F D' C..F --ancestry-path -- file # drops D

OK. D should not be dropped merely because it is the same with B,
which is now outside our graph "-a-p C..F".

> +check_result 'F D' C..F --ancestry-path --parents -- file
> +check_result 'F D' C..F --ancestry-path --simplify-merges -- file

OK.

> +check_result 'F D B' C..F --first-parent
> +check_result 'F B' C..F --first-parent -- file

OK.  "-f-p C..F" is the same as "-f-p A..F", and C is outside our
graph, so these expectations look sensible.

Re: [PATCH v3 5/9] revision.c: Make --full-history consider more merges

[Junio C Hamano]

Kevin Bracey <kevin@bracey.fi> writes:

> diff --git a/revision.c b/revision.c
> index a67b615..c88ded8 100644
> --- a/revision.c
> +++ b/revision.c
> @@ -429,10 +429,100 @@ static int rev_same_tree_as_empty(struct rev_info *revs, struct commit *commit)
>  	return retval >= 0 && (tree_difference == REV_TREE_SAME);
>  }
>  
> +struct treesame_state {
> +	unsigned int nparents;
> +	unsigned char treesame[FLEX_ARRAY];
> +};

I have been wondering if we want to do one-bit (not one-byte) per
parent but no biggie ;-)

> @@ -1971,6 +2083,70 @@ static struct merge_simplify_state *locate_simplify_state(struct rev_info *revs,
>  	return st;
>  }
>  
> +static int mark_redundant_parents(struct rev_info *revs, struct commit *commit)
> +{
> +...
> +		po=po->next;

	po = po->next;

> +	}

This seems to be identical (modulo tests) from the previous round,
which I found a reasonable thing to do.

Re: [PATCH v3 10/9] revision.c: treat A...B merge bases as if manually specified

[Junio C Hamano]

Kevin Bracey <kevin@bracey.fi> writes:

> The documentation assures users that "A...B" is defined as 'r1 r2 --not
> $(git merge-base --all r1 r2)'. This isn't in fact quite true, because
> the calculated merge bases are not sent to add_rev_cmdline().

We want the commands to be able to tell which ones in revs->pending
and revs->commits were specified by the end user and how.  While I
think it makes sense to mark these negative merge bases with "These
came from the command line with A...B syntax", I am not sure if it
is the best way to do so in add_pending_commit_list().

By the way, why does this have anything to do with the history
traversal series in the first place?

When there is anythning marked UNINTERESTING on the rev->pending
before calling prepare_revision_walk(), you have a history with some
bottom boundaries, and when there isn't, your bottom boundaries are
root commits.  If you want to behave differently depending on how
the user gave us the revision range from the command line, e.g.
acting differently between "A ^B" and "B..A", cmdline is a good
place to learn the exact form, but at the history traversal level, I
do not think you should even care.  Why does the code even look at
the cmdline, and not rev->pending?

Re: [PATCH v3 5/9] revision.c: Make --full-history consider more merges

[Kevin Bracey]

On 06/05/2013 23:45, Junio C Hamano wrote:
> Kevin Bracey <kevin@bracey.fi> writes:
>
>> +struct treesame_state {
>> +	unsigned int nparents;
>> +	unsigned char treesame[FLEX_ARRAY];
>> +};
> I have been wondering if we want to do one-bit (not one-byte) per
> parent but no biggie ;-)

I did start down that path, because I felt bad about bloat.

But then I realised how much I would be complicating and slowing the 
code down to only save a few bytes each time we walk a merge with at 
least 5 parents, and I came to my senses. :)

Kevin

Re: [PATCH v3 3/9] t6111: new TREESAME test set

[Kevin Bracey]

On 06/05/2013 23:36, Junio C Hamano wrote:
> Kevin Bracey <kevin@bracey.fi> writes:
>
>> +#        ,---E--.   *H----------.             * marks !TREESAME parent paths
>> +#       /        \ /             \*
>> +# *A--*B---D--*F-*G---------K-*L-*M
>> +#   \     /*       \       /
>> +#    `-C-'          `-*I-*J
>> +#
>> +# A creates "file", B and F change it.
>> +# Odd merge G takes the old version from B.
>> +# I changes it, but J reverts it.
>> +# H and L both change it, and M merges those changes.
>> + ...
>>
>> +check_outcome failure 'M H L J I G F B A' --simplify-merges -- file # drops G
> Even though E simplifies down to B, making G a merge between B and F (the former is an ancestor of the latter), G is kept, because...?
>
> Because the path at B and F are different?

G's simplified parent B is marked for deletion by the redundant parent scan; it would be redundant in any normal merge scenario. But here, because of the odd merge, it gets a reprieve from the "don't drop all TREESAME parents" rule. So merge G remains with 2 parents: B (its "content" parent) and F (its "topological" parent).

> I have to wonder what should happen if D makes path different from B, and then F makes path the same as B.  B and F should still be kept.

That would happen. G would be TREESAME to F, so the "keep 1 TREESAME" rule wouldn't need to kick in, parent B would be dropped from G, and G would simplify to F.


> +# Check that odd merge G remains shown when F is the bottom.
> I did not have a chance to say this when you responded in the
> previous round with that "bottom" thing before you sent this new round,
> but I am not sure if it is a good idea to pay special attention to the
> "bottom"-ness.  Very often, instead of running "git log -- path" and
> stopping when seeing enough by killing the pager, people run
> "git log HEAD~200.. -- path" to pick "recent enough"
> commits, but the number 200 is very much arbitrary.

True, the bottom can be arbitrary. But that actually turns out to be the key argument in favour of what I'm doing. The reason for treating the bottom "specially" is actually to prevent side branches that are immediately above the bottom from being treated specially. In early tests of the side branch logic, it became clear that whether a side branch was displayed or not depended on whether it was exactly adjacent to the bottom. The behaviour was arbitrary and annoying.

Considering your arbitrary bottom example, observe what happens if there's an irrelevant topic merge coming in at HEAD~199, and you try various bottoms around that point, without treating the bottom "specially". You get a transient simplification failure as the bottom passes:

* Bottom=HEAD~300 - below the topic branch point:

Normal simplification will work fine for the topic branch; it is unaffected by limiting, so the entire topic branch and merge is simplified away. This worked fine before this patch series (assuming a normal merge).

Now we move forward to put the bottom between the topic branch point and merge.

* Bottom=HEAD~201:                (B=bottom, o=other UNINTERESTING, x=INTERESTING, *=!TREESAME)

         B--*x---x--*x    simplifies to      B--*x---x--*x
                /*                                  /*
       o---x---x                                   o

   New rules reduce to      B--*x--*x      (1 TREESAME INTERESTING parent,
                                           so ignore UNINTERESTING side)

* Move bottom up one to HEAD~200:

             B---x--*x    simplifies to          B---x--*x
                /*                                  /*
       o---x---x                                   o

                                           (Merge can't be dropped -
                                            2 UNINTERESTING parents that differ)

If I can't treat bottoms specially, then I can't distinguish between the merge's two UNINTERESTING parents, so I have to show the merge.

* Move bottom up one again to HEAD~199, and the branch is excluded by the path-spec anyway:

                 B--*x

So the net effect there is that we have a side branch merge that is shown for _exactly_ one choice of bottom commit on the mainline. It's not shown for 'HEAD~5000..' to 'HEAD~201..', and it's not shown for 'HEAD~199..' to 'HEAD..'. But it pops up in 'HEAD~200..'.  Something's wrong with that picture, and treating the bottom as interesting removes the effect. Fuller discussion below.

>> +check_result 'M L H' F..M -- file
> OK.  It is still one possible history to explain what we have in the path at the end.

Yes, but I don't really like it much. It's not clear that we came from an ancestor of F, not F itself. But there's nothing you can do about that in this most basic of commands, and at least adding any option does reveal the truth that we're not based on F. Even just "--parents" (implied in gitk) reveals that H came from B.

>> +check_result 'M L K J I H G' F..M --full-history --parents -- file
> OK.
>
> It almost makes me suspect that the --full-history option should flip
> revs->rewrite_parents on (but not revs->print_parents, unless the --parents
> option asks us to) when the option is parsed.

I don't think so. Why do we want to show K unless we have to? It would just be a lot of noise in a normal "git log" - this tree isn't typical. In a normal tree, most merges will be more like K - so when requesting a log of a specific file, the mainline won't have modified the relevant file during the life of most irrelevant topic branches, so those branches' merges will be fully TREESAME and can be omitted.

I assume --simplify-merges had to be invented to get rid of the noise of all the pointless fully-TREESAME merges from "--full-history --parents", which keeps all merges. But without --parents, a file-specific --full-history isn't currently that bad.


>> +check_result 'M L J I H' E..M --ancestry-path -- file
> Because F is outside our graph with --ancestry-path, G does not bring
> anything new to path relative to its remaining parent E and should not be shown.  OK.

Yep. And this works because we treat bottom E as a parent of G for TREESAME, despite it being UNINTERESTING. If we didn't, then E and F would be "equals", and G would have to be shown.

>> +check_outcome failure 'M L K J I H' E..M --ancestry-path --parents --
>> +file
> This fails in what way?  G is shown, because the !treesame with now irrelevant F comes into play?

Yes, that's right. Sorry, missing comment.

>> +check_outcome failure 'M H L J I' E..M --ancestry-path
>> +--simplify-merges -- file # includes G
> Makes me wonder how "--ancestry-path --full-history" would work with this range and with/without the pathspec.

--ancestry-path implies full-history, as far as I can see.

>> +check_outcome failure 'M L H' G..M -- file # includes J I
>> +check_outcome failure 'M L H' G..M --parents -- file # includes J I
> I am not sure if it should be a failure or your expectation is wrong.
> G is outside the graph, so as far as the remainder of the graph is concerned,
> J is the sole remaining parent of K and I and J did change the path in question.
>
> What makes you think I and J should be excluded in these cases?

Because it's the simplest answer to the question "what happened in M since G", which is what "G..M" is supposed to mean. And because I J _would_ have been excluded and we would have got 'M L H' if we added an extra null commit G1 between G and K. That suggests an instability in our logic. Why should adding a totally irrelevant commit affect whether I-J is hidden? I J is hidden in every case apart from when you specify K^ as bottom. Why is K^ a special bottom?

Graph theory time!

This all comes about because the formal graph definition doesn't match the user interface. The question "B..C" currently generates a graph of all commits in C since B, and the connections between those commits. It turns out to be problematic that the graph doesn't include the connection to B itself. It would be fine if only worrying about nodes in the graph. But it's not fine when you start doing graph operations that care about edge connections to parents.

Thinking linearly, consider:

A..C:   o--o--B--o--o--C

The two queries "A..B" and "B..C" should, between them, cover "A..C", surely? But with the current logic, they don't:

A..B:   o--o--B
B..C:            o--o--C

Wherever you place B, the following edge gets "deleted" from consideration. Not a problem for A..B, because we never consider graph commits' children. But it's a problem for B..C - it affects the graph operations on the first commit of the graph. Which is what triggers this G..M problem here. In all normal circumstances, the default graph following would always choose to take K's first TREESAME parent, and ignore IJ. But when the bottom is specifically K^, it breaks, because that mainline TREESAME edge is removed from consideration. K is handled differently because it's still on the graph, but it's lost its mainline parent.

Looking at the graph above, with my proposed addition of G1:

K~3..M:

        *G---G1--<-K-*L-*M
          \       /
           `-*I-*J

K~2..M:

             G1--<-K-*L-*M
                  /
             *I-*J

K^..M:

                   K-*L-*M    - whoops. Get diverted onto IJ
                  /
             *I-*J

K..M:

                      L-*M

So for exactly one position of bottom along the mainline, we "forget" the path of our mainline, and get diverted, because we're not including the bottom in the graph. This is just like the HEAD~200 example - different graph optimisation, but same symptom and root cause. Include the bottom in the graph, but unprinted, and it works:

K~3..M:

  (F)---*G---G1--<-K-*L-*M
          \       /
           `-*I-*J

K~2..M:

       (G)---G1--<-K-*L-*M
          \       /
           `-*I-*J
K^..M:

            (G1)-<-K-*L-*M
                  /
             *I-*J

K..M:

                      L-*M

What I'm effectively doing is extending the graph to actually include the unshown bottom. I think it just makes more sense. We connect up the linear history:

A..B:  (A)--o--o--B
B..C:            (B)--o--o--C

The two are now connected, with every commit and edge considered once between them. The A..B and B..C graphs are now really connected at B. In the second range, we don't print B, but do treat it as if it was part of the graph for graph-edge-based operations that want to distinguish between relevant and irrelevant parents. Which then means that the choice of split point B on the mainline between A and C doesn't perturb the side branch logic, because we always consider the same edges, regardless of where B is. B doesn't "delete" its following edge. And "B..C" now really is "how did we get from B to C" - it includes that first step.

(In a non-linear graph, in this scheme, A..B and B..C aren't fully connected - all other commit boundaries between the sets are disjoint, and they only connect at B. But that's what makes the "side branch" logic work out - we can tell which path the user cares about at the bottom boundary, and hence which paths are the sides. Anything UNINTERESTING && !BOTTOM will be an ancestor of a bottom, and thus can be candidate for side-branch simplification, which is what we want to happen with 'HEAD~200..' above).

> Shouldn't the same logic as "--ancestry-path [EF]..M" we saw earlier apply here?
> That is, "-a-p E..M" makes F the sole remaining parent of G and G does change the
> path from F so it should be shown, while "-a-p F..M" makes E the sole parent of G,
> and G does not change the path from E, so it should not be shown.

That's muddled. I assume you mean:

> That is, "-a-p F..M" makes F the sole remaining parent of G and G does change the
> path from F so it should be shown, while "-a-p E..M" makes E the sole parent of G,
> and G does not change the path from E, so it should not be shown.

Which is the way the logic works - we treat F and E as interesting/priority parents when they're specified as a bottom in each case. Without doing that, G would have 2 differing and equally (un)important parents in each case, and thus would be shown in both cases.

In this case, the same logic says that G is treated as an interesting parent of K because it is the specified bottom. Which then enables the default following to follow that path direct to G, rather than having to go down the IJ path (which leads to G anyway).

Kevin

Re: [PATCH v3 5/9] revision.c: Make --full-history consider more merges

[Junio C Hamano]

Kevin Bracey <kevin@bracey.fi> writes:

> On 06/05/2013 23:45, Junio C Hamano wrote:
>> Kevin Bracey <kevin@bracey.fi> writes:
>>
>>> +struct treesame_state {
>>> +	unsigned int nparents;
>>> +	unsigned char treesame[FLEX_ARRAY];
>>> +};
>> I have been wondering if we want to do one-bit (not one-byte) per
>> parent but no biggie ;-)
>
> I did start down that path, because I felt bad about bloat.
>
> But then I realised how much I would be complicating and slowing the
> code down to only save a few bytes each time we walk a merge with at
> least 5 parents, and I came to my senses. :)

;-)

Re: [PATCH v3 10/9] revision.c: treat A...B merge bases as if manually specified

[Kevin Bracey]

On 07/05/2013 00:24, Junio C Hamano wrote:
> Kevin Bracey <kevin@bracey.fi> writes:
>
>> The documentation assures users that "A...B" is defined as 'r1 r2 --not
>> $(git merge-base --all r1 r2)'. This isn't in fact quite true, because
>> the calculated merge bases are not sent to add_rev_cmdline().
> We want the commands to be able to tell which ones in revs->pending
> and revs->commits were specified by the end user and how.  While I
> think it makes sense to mark these negative merge bases with "These
> came from the command line with A...B syntax", I am not sure if it
> is the best way to do so in add_pending_commit_list().
Layering violation? The callers of add_pending_commit_list() should be 
doing it themselves? Yes, I suppose it was a bit of a convenience hack, 
having add_pending_commit_list do it.
>
> By the way, why does this have anything to do with the history
> traversal series in the first place?

The answer's in the test, but it's not that clear in this series 
addendum, with the test newly appearing and passing. The next version of 
the series will have the test in initially as a failure.

Without this patch, "git log E...F file" will unnecessarily show D, as 
it has 2 differing non-priority parents B and C.

Whereas "git log E F ^B file" doesn't show D. So we have a behaviour 
difference between two allegedly equivalent commands.

When querying E...F, C is a side branch between the merge base B and F, 
so D should be removable, just as for "B..F" or "E F ^B". So we need to 
give B the BOTTOM marker to make that work, as if it had been specified 
"E F ^B".


>
> When there is anythning marked UNINTERESTING on the rev->pending
> before calling prepare_revision_walk(), you have a history with some
> bottom boundaries, and when there isn't, your bottom boundaries are
> root commits.  If you want to behave differently depending on how
> the user gave us the revision range from the command line, e.g.
> acting differently between "A ^B" and "B..A", cmdline is a good
> place to learn the exact form, but at the history traversal level, I
> do not think you should even care.  Why does the code even look at
> the cmdline, and not rev->pending?
>

Well, on this first pass I wanted to be sure I was using the same 
definition of "bottom" as ancestry-path, so I hacked the flag setting in 
there, and I believe the answer to "why not just look at UNINTERESTING" 
lies in commit 281eee4. If I understand that correctly, it goes wrong 
when you have multiple negative specifications - some initial walking is 
done, meaning we can't distinguish between specified bottoms and other 
walk results.

Now, the current structure of the code is clearly silly, and my "hack" 
comment acknowledged that - if we have the BOTTOM flag, we could just 
set that immediately during command parsing, and neither this nor 
ancestry-path would need to look back at the command line to identify 
what the bottom commits were. But I didn't want to start work on that 
without further discussion about the merits of the BOTTOM flag.

(Actually, no, I'm not looking back at the command line, I just know 
that ancestry-path does, so I make sure I arrange it so that my 
inspection of rev->pending shows the same bottom commits as its 
inspection of the command line).

Kevin

Re: [PATCH v3 3/9] t6111: new TREESAME test set

[Junio C Hamano]

Kevin Bracey <kevin@bracey.fi> writes:

> On 06/05/2013 23:36, Junio C Hamano wrote:
>> Kevin Bracey <kevin@bracey.fi> writes:
>>
>>> +#        ,---E--.   *H----------.             * marks !TREESAME parent paths
>>> +#       /        \ /             \*
>>> +# *A--*B---D--*F-*G---------K-*L-*M
>>> +#   \     /*       \       /
>>> +#    `-C-'          `-*I-*J
>>> +#
>>> +# A creates "file", B and F change it.
>>> +# Odd merge G takes the old version from B.
>>> +# I changes it, but J reverts it.
>>> +# H and L both change it, and M merges those changes.
>>> + ...
> ...
>>> +check_outcome failure 'M L H' G..M -- file # includes J I
>>> +check_outcome failure 'M L H' G..M --parents -- file # includes J I
>> I am not sure if it should be a failure or your expectation is wrong.
>> G is outside the graph, so as far as the remainder of the graph is concerned,
>> J is the sole remaining parent of K and I and J did change the path in question.
>>
>> What makes you think I and J should be excluded in these cases?
>
> Because it's the simplest answer to the question "what happened in
> M since G", which is what "G..M" is supposed to mean. ...
> 
> This all comes about because the formal graph definition doesn't
> match the user interface. The question "B..C" currently generates
> a graph of all commits in C since B, and the connections between
> those commits. It turns out to be problematic that the graph
> doesn't include the connection to B itself. It would be fine if
> only worrying about nodes in the graph. But it's not fine when you
> start doing graph operations that care about edge connections to
> parents.

OK, that makes sense.

> ...
> What I'm effectively doing is extending the graph to actually
> include the unshown bottom. I think it just makes more sense.

Yup, and this is a good summary.


> ... I assume you mean:
>
>> That is, "-a-p F..M" makes F the sole remaining parent of G and G does change the
>> path from F so it should be shown, while "-a-p E..M" makes E the sole parent of G,
>> and G does not change the path from E, so it should not be shown.

Yes.

> Which is the way the logic works - we treat F and E as
> interesting/priority parents when they're specified as a bottom in
> each case. Without doing that, G would have 2 differing and
> equally (un)important parents in each case, and thus would be
> shown in both cases.
>
> In this case, the same logic says that G is treated as an
> interesting parent of K because it is the specified bottom. Which
> then enables the default following to follow that path direct to
> G, rather than having to go down the IJ path (which leads to G
> anyway).

OK.