In tree object, Must the

In tree object, Must the

[Yi, EungJun]

Hello,

I'm implementing Git using node.js, and I have a question while I
write some code to store tree object.

Tree object looks a table consists of three fields: blob's mode, name
and id, as below.

e.g.)
$ git cat-file -p 45799547
100644 blob cd242b1e5bb403500feb49a1aa656c21c6c0be69	Makefile
100644 blob bf382321749577d52bd2fbf2281df0510b4bad31	README.md
100644 blob 5441bb48428611a3cb140d8192d39484fcf3b742	fsutil.js
100644 blob 0af680a5c0dd4482b09aa7f8e837234bed0b7cfa	package.json
040000 tree 39a4d45669addfb1e8f0a499deebc5b97b4edfa0	test

It seems that the table is stored in order by blob's name.

If it is true, what happens if it is not ordered?

Does that cause any troubles to users to use a git repository created
and managed by my Git implementation?

Re: In tree object, Must the

[Carlos Martín Nieto]

[-- Attachment #1: Type: text/plain, Size: 1816 bytes --]

On Sun, 2012-04-08 at 12:43 +0900, Yi, EungJun wrote:
> Hello,
> 
> I'm implementing Git using node.js, and I have a question while I
> write some code to store tree object.
> 
> Tree object looks a table consists of three fields: blob's mode, name
> and id, as below.
> 
> e.g.)
> $ git cat-file -p 45799547
> 100644 blob cd242b1e5bb403500feb49a1aa656c21c6c0be69	Makefile
> 100644 blob bf382321749577d52bd2fbf2281df0510b4bad31	README.md
> 100644 blob 5441bb48428611a3cb140d8192d39484fcf3b742	fsutil.js
> 100644 blob 0af680a5c0dd4482b09aa7f8e837234bed0b7cfa	package.json
> 040000 tree 39a4d45669addfb1e8f0a499deebc5b97b4edfa0	test
> 
> It seems that the table is stored in order by blob's name.

Yes, the entries in the tree are alpha-sorted. The exception are trees,
where you have to pretend that there is a trailing slash. In other
words, the order is the same as you'd see in the index (as there, the
test/ directory in your example would be stored with a slash and the
name of the subdirs and files in it.

> 
> If it is true, what happens if it is not ordered?

fsck complains for one.

> 
> Does that cause any troubles to users to use a git repository created
> and managed by my Git implementation?

How does your implementation store things? You haven't said (maybe
hinted that you may be writing trees with the wrong order). Depending on
the particular implementation of whatever is reading the git repository,
it might not be able to find an entry in your tree, as it's wrongly
sorted, but that depends on the exact implementation and possibly luck.

Do you need to write this in pure js? There are some bindings for
node.js[0] already for libgit2 so you don't need to redo the low-level
work.

Cheers,
   cmn

[0] https://github.com/libgit2/node-gitteh


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: In tree object, Must the

[Yi, EungJun]

You are right. I tried to clone a repository which has a unordered
tree object and all you said happened, as below.

$ git ls-tree HEAD
100644 blob a6ccb8dec6b21e9d3e0b716d3a0915262a8c4474	b
100644 blob a3df8f3749f855a2d905d92f11590a9c18a1bcbb	a
100644 blob 424907d3fe9f1fa4b2e3a0bb65ab99780d9acfba	c
$ git show -- b
$ git show -- a
$ git fsck
error in tree 3f5a56e992a01f2e500bed57360f3961323b9d6e: not properly sorted
error in tree a94e7dc2820eb4d0ef308fac1a32b3715e82dc76: not properly sorted

I want to allow users to clone the git repository created by my implementation,
but before I have to sort tree objects. Thanks to you, I know that now.

And your suggestion to use node-gitteh is also thankful,
but I want pure js implementation without libgit2.

2012년 4월 8일 오후 10:20, Carlos Martín Nieto <cmn@elego.de>님의 말:
> On Sun, 2012-04-08 at 12:43 +0900, Yi, EungJun wrote:
>> Hello,
>>
>> I'm implementing Git using node.js, and I have a question while I
>> write some code to store tree object.
>>
>> Tree object looks a table consists of three fields: blob's mode, name
>> and id, as below.
>>
>> e.g.)
>> $ git cat-file -p 45799547
>> 100644 blob cd242b1e5bb403500feb49a1aa656c21c6c0be69  Makefile
>> 100644 blob bf382321749577d52bd2fbf2281df0510b4bad31  README.md
>> 100644 blob 5441bb48428611a3cb140d8192d39484fcf3b742  fsutil.js
>> 100644 blob 0af680a5c0dd4482b09aa7f8e837234bed0b7cfa  package.json
>> 040000 tree 39a4d45669addfb1e8f0a499deebc5b97b4edfa0  test
>>
>> It seems that the table is stored in order by blob's name.
>
> Yes, the entries in the tree are alpha-sorted. The exception are trees,
> where you have to pretend that there is a trailing slash. In other
> words, the order is the same as you'd see in the index (as there, the
> test/ directory in your example would be stored with a slash and the
> name of the subdirs and files in it.
>
>>
>> If it is true, what happens if it is not ordered?
>
> fsck complains for one.
>
>>
>> Does that cause any troubles to users to use a git repository created
>> and managed by my Git implementation?
>
> How does your implementation store things? You haven't said (maybe
> hinted that you may be writing trees with the wrong order). Depending on
> the particular implementation of whatever is reading the git repository,
> it might not be able to find an entry in your tree, as it's wrongly
> sorted, but that depends on the exact implementation and possibly luck.
>
> Do you need to write this in pure js? There are some bindings for
> node.js[0] already for libgit2 so you don't need to redo the low-level
> work.
>
> Cheers,
>   cmn
>
> [0] https://github.com/libgit2/node-gitteh
>

Re: In tree object, Must the

[Junio C Hamano]

"Yi, EungJun" <semtlenori@gmail.com> writes:

> If it is true, what happens if it is not ordered?

"git diff-tree" and "git diff-index" (hence "git diff" frontend when it
walks the tree for comparison) will fail.

Re: In tree object, Must the

[Shawn Pearce]

On Mon, Apr 9, 2012 at 11:25, Junio C Hamano <gitster@pobox.com> wrote:
> "Yi, EungJun" <semtlenori@gmail.com> writes:
>
>> If it is true, what happens if it is not ordered?
>
> "git diff-tree" and "git diff-index" (hence "git diff" frontend when it
> walks the tree for comparison) will fail.

And JGit and any JGit derived tool (e.g. EGit, Gerrit Code Review)
will fail horribly and refuse to process that tree, or worse, will
produce incorrect results when fed garbage like an unsorted or
incorrectly sorted tree.

Tree ordering matters. And that niggly bit about subtrees sorting as
though their names end in '/' even though they don't really matters.

Re: In tree object, Must the

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> ... And that niggly bit about subtrees sorting as
> though their names end in '/' even though they don't really matters.

Yeah, in older and buggier unpack-trees it might have made the code
simpler while walking the index and the tree in parallel, but when we have
to deal with D/F conflicts, we have to find a entry with matching name
anyway, so it turned out not to be a win at all.

Re: In tree object, Must the

[Shawn Pearce]

On Mon, Apr 9, 2012 at 13:05, Junio C Hamano <gitster@pobox.com> wrote:
> Shawn Pearce <spearce@spearce.org> writes:
>
>> ... And that niggly bit about subtrees sorting as
>> though their names end in '/' even though they don't really matters.
>
> Yeah, in older and buggier unpack-trees it might have made the code
> simpler while walking the index and the tree in parallel, but when we have
> to deal with D/F conflicts, we have to find a entry with matching name
> anyway, so it turned out not to be a win at all.

JGit still waks the trees in parallel. We handle the D/F conflict
stuff using a small amount of look-ahead and sometimes rescan back
from the start. In practice that works out really well, it catches
most forms of D/F conflict with little overhead, and almost no
overhead when there is no D/F conflict, which is common.