Re: RFC: Flat directory for notes, or fan-out? Both!

[Sam Vilain <sam@vilain.net>]

Linus Torvalds wrote:
>> The only case where it hurts is when you want to merge. Nothing else
>> should care. So, if a merge of these note trees sees two different trie
>> sizes then it can convert the shorter one to the longer length first,
>> and then try the merge again. So you get the pain, but only once. And
>> when a project decides that its split is too small, it can split then
>> and it should "silently" spread out to others.
>>     
>
> But what's the advantage of the added complexity?
>
> The non-fixed trie only helps for the case that doesn't matter - just a 
> few annotations. If you have a thousand annotations or less, you _really_ 
> don't care. Whatever you do will be fine.
>
> So the whole thing only matters once you have tens of thousands of 
> entries, and then you do want to have fan-out. No?
>   

Yeah. I see your point and you may be right, that a 12/28 split hurts
no-one, if we take this to the benchmarks. There's certainly savings in
terms of total object count for the small users by using a smaller split.

I just already wrote the code to handle an arbitrary split for the
features written so far[1]. If *I* can write it, in C, it means it must
not be that complicated ;-)

So it comes down to how complicated things are when merging happens. If
12 is fixed in stone this is simple, because there are no chances for
discrepancies. But refs/notes/commits still needs special treatment to
be fetched, because it is not under refs/heads/* and you wouldn't
normally have a working tree to resolve conflicts.

So I think probably the most productive thing to do is for me to write
the code to handle the merge as I described above, once the code to
handle pulling in - and merging - notes at 'git fetch' time is written.
Then we can see whether it's that much of a complication.

To bench this we need the current builtin-log implementation to be
re-written to be lazy. Which means we can't put it in the next release
unless someone writes that. However my proposal means that we can
release as we are and not care, and let some code - which I hope I have
shown isn't *that* complicated, really - deal with it in a later
release, and not break backwards compatibility.

Sam.

1. see message <1233455960.17688.122.camel@maia.lan>