Re: Severe VFS Performance Issues 2.6 with > 95000 directory entries

["Jeff V. Merkey" <jmerkey@wolfmountaingroup.com>]

Douglas McNaught wrote:

>jmerkey <jmerkey@soleranetworks.com> writes:
>
>  
>
>>Nikita Danilov wrote:
>>
>>    
>>
>>>Jeff V. Merkey writes:
>>>      
>>>
>>>>>The subject line speaks for itself.   This is using standard VFS
>>>>>          
>>>>>
>>>readdir > and lookup calls through the VFSwith ftp.  Very poor. 
>>>
>>>Reiser4 works fine with 100M entries in a directory, so VFS is not a
>>>bottleneck here.
>>> 
>>>
>>>      
>>>
>>how about with ftp running on top? Try running FTP in directory with
>>100M entries. See how long it takes to return the data to
>>the remote client for a dir listing.
>>    
>>
>
>What filesystem are you using?  If it's ext3 without dirindex turned
>on, that would definitely explain it.
>
>  
>
I just noticed the I_NEW flag for iget which prevents multiple calls to 
refresh the inode. There's another code section where I update the 
filesize field
after I call iget from lookup. This does not explain it either since I 
use math here to hash and post into the inode. I am still convinced that 
either in userspace
or in the kernel VFS, there's still a case where readdit goes linear and 
starts to exhibit (O)(Log 2(N)) behavior as the directory gets large 
(above 50,000 entries).

Jeff