mktree: multiple same-named objects

mktree: multiple same-named objects

[David Turner]

git mktree seems to allow the creation of a tree object with multiple
objects of the same name but different SHAs.  This leads to weird
behavior later, unsurprisingly.  For instance, if there are two tree
objects with the same name but different SHAs, the checked out tree will
be the union of them (reasonably), but if you do git add $name, some or
all unmodified files under $name will show up in git status as modified
-- since they differ from one of the parent trees, presumably.

And if different git implementations treat this case differently, then
it might be possible to make a repo that appears to contain one thing
when viewed with one implementation, but contains a different thing for
a different implementation.

Summary: git mktree ought to forbid this, and possibly there ought to be
other checks (for instance, when unpacking) to prevent this.

Re: mktree: multiple same-named objects

[Duy Nguyen]

On Wed, Aug 27, 2014 at 11:41 AM, David Turner <dturner@twopensource.com> wrote:
> Summary: git mktree ought to forbid this, and possibly there ought to be
> other checks (for instance, when unpacking) to prevent this.

Does fsck detect this (because we ought to fix fsck first if it does not)?
-- 
Duy

Re: mktree: multiple same-named objects

[David Turner]

On Wed, 2014-08-27 at 12:01 +0700, Duy Nguyen wrote:
> On Wed, Aug 27, 2014 at 11:41 AM, David Turner <dturner@twopensource.com> wrote:
> > Summary: git mktree ought to forbid this, and possibly there ought to be
> > other checks (for instance, when unpacking) to prevent this.
> 
> Does fsck detect this (because we ought to fix fsck first if it does not)?

Yes, it does.

Re: mktree: multiple same-named objects

[Jeff King]

On Wed, Aug 27, 2014 at 12:41:57AM -0400, David Turner wrote:

> git mktree seems to allow the creation of a tree object with multiple
> objects of the same name but different SHAs.

Yeah, I don't think we do many quality checks there. Ditto for "git
hash-object".

The latter goes through index_mem, which at least checks that the
resulting tree is parseable. It does not look like mktree even checks
that.

> Summary: git mktree ought to forbid this, and possibly there ought to be
> other checks (for instance, when unpacking) to prevent this.

The checks in git-fsck will notice your problem (and many others). I
think we should be running them anytime we create an object based on
arbitrary data (including mktree and hash-object). Code paths like "git
write-tree" and "git commit-tree" are probably OK, as their code should
follow the standard (it would not hurt to double-check their output,
though there may be a performance implication).

Dscho (cc'd) has been looking into this approach; I don't know how far
he has gotten.

-Peff

Re: mktree: multiple same-named objects

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Wed, Aug 27, 2014 at 12:41:57AM -0400, David Turner wrote:
>
>> git mktree seems to allow the creation of a tree object with multiple
>> objects of the same name but different SHAs.
>
> Yeah, I don't think we do many quality checks there. Ditto for "git
> hash-object".

I am somewhat against outright removing the capability to write out
invalid objects deliberately from these low level tools, because we
would need a way to easily reproduce bugs in end-user facing tools
by other people who claim to produce Git objects, but I would agree
that by default that should be forbidden.

In other words, two things must happen; improve checks when these
low level debugging aid tools are creating objects, and allow
bypassing these additional checks with "--experiment" option or
something.

Re: mktree: multiple same-named objects

[Jeff King]

On Wed, Aug 27, 2014 at 08:17:15AM -0700, Junio C Hamano wrote:

> I am somewhat against outright removing the capability to write out
> invalid objects deliberately from these low level tools, because we
> would need a way to easily reproduce bugs in end-user facing tools
> by other people who claim to produce Git objects, but I would agree
> that by default that should be forbidden.
> 
> In other words, two things must happen; improve checks when these
> low level debugging aid tools are creating objects, and allow
> bypassing these additional checks with "--experiment" option or
> something.

Yeah, definitely. I had imagined it as "--strict" and "--no-strict",
with flipping the default to "--strict" at some point (I do not see a
reason anybody would not want it in normal use, but if we are worried,
we can even go slowly on flipping the default).

-Peff