Re: [PATCH 2/6] chunkd: change the prefix length of object pathname from 4 to 3

[Jeff Garzik <jeff@garzik.org>]

On 01/12/2010 03:28 AM, Akinobu Mita wrote:
> 2010/1/12 Jeff Garzik<jeff@garzik.org>:
>> On 01/11/2010 07:25 AM, Akinobu Mita wrote:
>>>
>>> The volume directory may have 2^16 (=65536) directories because
>>> prefix length of object pathname is 4 bytes. But ext3 filesystem can
>>> only have EXT_LINK_MAX (32000) directories.
>>>
>>> This changes the prefix length from 4 bytes to 3 bytes and breaks
>>> the volume data format compatibility with older version.
>>>
>>> Now the maximum count of links in volume directory is 2^12 (=4096)
>>> and it doesn't exceed ext3's upper limit of max links.
>>>
>>> Signed-off-by: Akinobu Mita<akinobu.mita@gmail.com>
>>> ---
>>>   server/be-fs.c |    2 +-
>>>   1 files changed, 1 insertions(+), 1 deletions(-)
>>
>> hey, welcome, new contributor!
>>
>> This patch makes sense, but it also raises the question of whether or not we
>> should move to a two-level directory scheme, eg.
>>
>>         123/456/7890ABCDEF
>>     rather than
>>         123/4567890ABCDEF
>>
>> to limit the size of the top-level directories.  It really depends on the
>> object counts a typical chunkd node will be seeing.  As with the other
>> patch, I will give this some thought after sleep.
>
> Two-level directory scheme looks good.
>
> I will do it unless someone thinks 536,870,912,000(=4096*4096*32000)
> objects in one table is not enough :)

FWIW, 32000 is only the limit on directories-with-a-directory.  You can 
easily have millions of regular files in a single ext3 directory.  So it 
is really 4096*4096*millions.

	Jeff