problem in unpack-trees.c

problem in unpack-trees.c

[Roman Zippel]

Hi,

Git currently fails to succeed the tests as you can see here:
http://buildd.debian.org/fetch.cgi?&pkg=git-core&ver=1%3A1.4.4-2&arch=m68k&stamp=1164303729&file=log

I looked into it and the problem is during the "git-read-tree --reset" 
step and it seems that the local df_conflict_entry variable of 
unpack_trees() survives past that function. If you check in 
add_cache_entry() it's called with this variable and only because 
verify_path() fails it's not added to the tree on the other archs, but on 
m68k the data on the stack is a bit different and thus verify_path() 
succeeds and the stack variable is added to the tree and later saved.

Using the patch below, you can simulate what's happing on m68k and now I 
need some help fixing this properly, as I'm not that familiar with the 
internals.
Thanks.

bye, Roman

diff -ur git-core-1.4.4.1/unpack-trees.c git-core-1.4.4.1/unpack-trees.c
--- git-core-1.4.4.1/unpack-trees.c	2006-11-23 03:38:07.000000000 +0100
+++ git-core-1.4.4.1/unpack-trees.c	2006-12-03 01:54:04.000000000 +0100
@@ -370,7 +370,10 @@
 	int i;
 	struct object_list *posn = trees;
 	struct tree_entry_list df_conflict_list;
-	struct cache_entry df_conflict_entry;
+	static struct {
+		struct cache_entry entry;
+		char name[4];
+	} df_conflict_entry;
 
 	memset(&df_conflict_list, 0, sizeof(df_conflict_list));
 	df_conflict_list.next = &df_conflict_list;
@@ -382,7 +385,8 @@
 
 	o->merge_size = len;
 	memset(&df_conflict_entry, 0, sizeof(df_conflict_entry));
-	o->df_conflict_entry = &df_conflict_entry;
+	o->df_conflict_entry = &df_conflict_entry.entry;
+	df_conflict_entry.entry.name[0] = 'x';
 
 	if (len) {
 		posns = xmalloc(len * sizeof(struct tree_entry_list *));
@@ -399,6 +403,7 @@
 		die("Merge requires file-level merging");
 
 	check_updates(active_cache, active_nr, o);
+	memset(&df_conflict_entry, 0x11, sizeof(df_conflict_entry));
 	return 0;
 }

Re: problem in unpack-trees.c

[Junio C Hamano]

Roman Zippel <zippel@linux-m68k.org> writes:

> I looked into it and the problem is during the "git-read-tree --reset" 
> step and it seems that the local df_conflict_entry variable of 
> unpack_trees() survives past that function. If you check in 
> add_cache_entry() it's called with this variable and only because 
> verify_path() fails it's not added to the tree on the other archs, but on 
> m68k the data on the stack is a bit different and thus verify_path() 
> succeeds and the stack variable is added to the tree and later saved.

I am very puzzled about this.

You are correct that the address of the df_conflict_entry is
assigned to "struct unpack_trees_options *o" in unpack_trees(),
and add_cache_entry() are called from many places in the call
chain that starts from that function.  And these call sites do
rely on the conflict_entry to have a NUL name to prevent
add_cache_entry from adding the entry to the index.  Which feels
like a hack, but it should get the job done while it is running.

But I do not see the o->df_conflict_entry used after unpack_trees()
returns in git-read-tree.

> Using the patch below, you can simulate what's happing on m68k...

The patch does not compile for me (the static definition part),
but it appears that the memset() in question is not clearing the
name field.

Ah....

Oops....

Isn't "struct cache_entry" a structure with a flexible array
member at the end?  I wonder why we have it on the stack to
begin with.

On my x86-64 box with gcc 4 (i.e. "#define FLEX_ARRAY /* empty */"
is used,

        #include "cache.h"

        int
        main(int ac, char **av)
        {
                printf("sz %zu\n", sizeof(struct cache_entry));
                printf("of %zu\n", offsetof(struct cache_entry, name));
                memset(&dfc, 0, sizeof(dfc));
        }

size of "struct cache_entry" is 64 while the offset of name
member is 62, so I am luckily getting two bytes of room for
memset to fill and cause name[] to be properly NUL terminated.
If the alignment requirement of the platform is smaller, we may
be overstepping the struct when we access its name[] member.

Re: problem in unpack-trees.c

[Junio C Hamano]

Junio C Hamano <junkio@cox.net> writes:

> If the alignment requirement of the platform is smaller, we may
> be overstepping the struct when we access its name[] member.

Would this help?

I do not think we need a separate identity for df_conflict_entry
per invocation of unpack_trees(), so this allocates one such
entry with NUL name at the first call to the function and reuses
it from the variable.

diff --git a/unpack-trees.c b/unpack-trees.c
index 7cfd628..47aa804 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -370,7 +370,7 @@ int unpack_trees(struct object_list *trees, struct unpack_trees_options *o)
 	int i;
 	struct object_list *posn = trees;
 	struct tree_entry_list df_conflict_list;
-	struct cache_entry df_conflict_entry;
+	static struct cache_entry *dfc;
 
 	memset(&df_conflict_list, 0, sizeof(df_conflict_list));
 	df_conflict_list.next = &df_conflict_list;
@@ -381,8 +381,10 @@ int unpack_trees(struct object_list *trees, struct unpack_trees_options *o)
 	state.refresh_cache = 1;
 
 	o->merge_size = len;
-	memset(&df_conflict_entry, 0, sizeof(df_conflict_entry));
-	o->df_conflict_entry = &df_conflict_entry;
+
+	if (!dfc)
+		dfc = xcalloc(1, sizeof(struct cache_entry) + 1);
+	o->df_conflict_entry = dfc;
 
 	if (len) {
 		posns = xmalloc(len * sizeof(struct tree_entry_list *));

Re: problem in unpack-trees.c

[Roman Zippel]

Hi,

On Mon, 4 Dec 2006, Junio C Hamano wrote:

> Roman Zippel <zippel@linux-m68k.org> writes:
> 
> > I looked into it and the problem is during the "git-read-tree --reset" 
> > step and it seems that the local df_conflict_entry variable of 
> > unpack_trees() survives past that function. If you check in 
> > add_cache_entry() it's called with this variable and only because 
> > verify_path() fails it's not added to the tree on the other archs, but on 
> > m68k the data on the stack is a bit different and thus verify_path() 
> > succeeds and the stack variable is added to the tree and later saved.
> 
> I am very puzzled about this.
> 
> You are correct that the address of the df_conflict_entry is
> assigned to "struct unpack_trees_options *o" in unpack_trees(),
> and add_cache_entry() are called from many places in the call
> chain that starts from that function.  And these call sites do
> rely on the conflict_entry to have a NUL name to prevent
> add_cache_entry from adding the entry to the index.  Which feels
> like a hack, but it should get the job done while it is running.

Ok, I see, I wasn't sure that this part was really intentional.

> On my x86-64 box with gcc 4 (i.e. "#define FLEX_ARRAY /* empty */"
> is used,
> 
>         #include "cache.h"
> 
>         int
>         main(int ac, char **av)
>         {
>                 printf("sz %zu\n", sizeof(struct cache_entry));
>                 printf("of %zu\n", offsetof(struct cache_entry, name));
>                 memset(&dfc, 0, sizeof(dfc));
>         }
> 
> size of "struct cache_entry" is 64 while the offset of name
> member is 62, so I am luckily getting two bytes of room for
> memset to fill and cause name[] to be properly NUL terminated.
> If the alignment requirement of the platform is smaller, we may
> be overstepping the struct when we access its name[] member.

Yes, on m68k both values are the same and thus name is not initialized.
Your patch should do the trick, I'll give it a try.