Re: Severe VFS Performance Issues 2.6 with > 95000 directory entries

Jeff V. Merkey
2005-11-13T22:03:51Z

Douglas McNaught wrote:

>jmerkey <jmerkey@soleranetworks.com> writes:
>
>
>
>>Nikita Danilov wrote:
>>
>>
>>
>>>Jeff V. Merkey writes:
>>>
>>>
>>>>>The subject line speaks for itself. This is using standard VFS
>>>>>
>>>>>
>>>readdir > and lookup calls through the VFSwith ftp. Very poor.
>>>
>>>Reiser4 works fine with 100M entries in a directory, so VFS is not a
>>>bottleneck here.
>>>
>>>
>>>
>>>
>>how about with ftp running on top? Try running FTP in directory with
>>100M entries. See how long it takes to return the data to
>>the remote client for a dir listing.
>>
>>
>
>What filesystem are you using? If it's ext3 without dirindex turned
>on, that would definitely explain it.
>
>
>
I just noticed the I_NEW flag for iget which prevents multiple calls to
refresh the inode. There's another code section where I update the
filesize field
after I call iget from lookup. This does not explain it either since I
use math here to hash and post into the inode. I am still convinced that
either in userspace
or in the kernel VFS, there's still a case where readdit goes linear and
starts to exhibit (O)(Log 2(N)) behavior as the directory gets large
(above 50,000 entries).

Jeff