[PATCH v10 22/36] merge-recursive: when comparing files, don't include trees

[Elijah Newren <newren@gmail.com>]

get_renames() would look up stage data that already existed (populated
in get_unmerged(), taken from whatever unpack_trees() created), and if
it didn't exist, would call insert_stage_data() to create the necessary
entry for the given file.  The insert_stage_data() fallback becomes
much more important for directory rename detection, because that creates
a mechanism to have a file in the resulting merge that didn't exist on
either side of history.  However, insert_stage_data(), due to calling
get_tree_entry() loaded up trees as readily as files.  We aren't
interested in comparing trees to files; the D/F conflict handling is
done elsewhere.  This code is just concerned with what entries existed
for a given path on the different sides of the merge, so create a
get_tree_entry_if_blob() helper function and use it.

Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 merge-recursive.c | 27 +++++++++++++++++++++------
 1 file changed, 21 insertions(+), 6 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index ecead3df4b..d569e3e893 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -421,6 +421,21 @@ static void get_files_dirs(struct merge_options *o, struct tree *tree)
 	read_tree_recursive(tree, "", 0, 0, &match_all, save_files_dirs, o);
 }
 
+static int get_tree_entry_if_blob(const struct object_id *tree,
+				  const char *path,
+				  struct object_id *hashy,
+				  unsigned int *mode_o)
+{
+	int ret;
+
+	ret = get_tree_entry(tree, path, hashy, mode_o);
+	if (S_ISDIR(*mode_o)) {
+		oidcpy(hashy, &null_oid);
+		*mode_o = 0;
+	}
+	return ret;
+}
+
 /*
  * Returns an index_entry instance which doesn't have to correspond to
  * a real cache entry in Git's index.
@@ -431,12 +446,12 @@ static struct stage_data *insert_stage_data(const char *path,
 {
 	struct string_list_item *item;
 	struct stage_data *e = xcalloc(1, sizeof(struct stage_data));
-	get_tree_entry(&o->object.oid, path,
-			&e->stages[1].oid, &e->stages[1].mode);
-	get_tree_entry(&a->object.oid, path,
-			&e->stages[2].oid, &e->stages[2].mode);
-	get_tree_entry(&b->object.oid, path,
-			&e->stages[3].oid, &e->stages[3].mode);
+	get_tree_entry_if_blob(&o->object.oid, path,
+			       &e->stages[1].oid, &e->stages[1].mode);
+	get_tree_entry_if_blob(&a->object.oid, path,
+			       &e->stages[2].oid, &e->stages[2].mode);
+	get_tree_entry_if_blob(&b->object.oid, path,
+			       &e->stages[3].oid, &e->stages[3].mode);
 	item = string_list_insert(entries, path);
 	item->util = e;
 	return e;
-- 
2.17.0.290.ge988e9ce2a