Re: readdir & bonnie++

[Tomas Hruby <thruby@few.vu.nl>]

>> Bonnie++ creates a directory with 16384 files. Then it calls readdir in a loop
>> to read all the entries and calls stat for each. The test fails since only 16383
>> entries were returned (one is missing). According to my logs our readdir calls
>> filldir on the last item as well. Also strace says that all entries were
>> returned. But the last one is missing.
>
>Possibly unrelated, but I saw something similar in my filesystem.  A
>simple loop calling readdir would fail, but explicit calls to getdents
>worked properly (and whatever ls did also worked).  In my case, I was
>playing games with d_off (filp->fpos) that caused d_off to overflow 32
>bits.  Then glibc would detect this and be "smart" -- seek back to
>some point and start over to read up to the overflow.  Eliminating the

Thanks for a good hint, I checked that but there is no overflow. 

> Are you getting this problem *after* bonnie has deleted some entries
> from the directory?  I had some problems with bonnie because it tries
> to delete entries from a directory in a while(readdir) loop, which
> happens to test a somewhat obscure and underspecified behaviour of
> directories.  I discussed this on the ML a while back.  Maybe this
> thread might help you:

I check the dicsussion on the suggested thread. It describes all issues I was
already facing :-) since we use similar tree structure as ext3.
I return hash (32bit) instead of a real position, but it works fine with ls 
rm -r so far.

My problem with bonnie happens in a test which counts how many files where
created. It just call readdir in loop and counts how many entries were
returned. As far as I understand the bonnie code, there were no files deleted
yet.

I checked the glibc code (i hope that the correct one) and it seems that the
loop inside readdir is returned only if there are no more data in the buffer
and so the getdents is called again. My problem is that there SHOULD be the
last entry still in the buffer.

		Tomas