readdir() loses entries on ramfs and tmpfs

readdir() loses entries on ramfs and tmpfs

[Pavel Roskin]

Hello!

I got a report that GNU Midnight Commander fails to erase some directories 
on tmpfs from the first attempt.  However, it succeeds the next time.

I could reproduce the problem on 2.4.18-pre1 compiled with gcc-2.96 from
RedHat 7.2.

I have reduced the problem to a simple test.  This script creates 
directories dir, dir/0, ... dir/168

=========================
#!/bin/sh
rm -rf dir
mkdir dir
i=0
while test $i != 169; do
  mkdir dir/$i
  i=$(($i+1))
done
=========================

And this C program tries to remove all subdirectories under "dir":

=========================
#include <stdio.h>
#include <unistd.h>
#include <dirent.h>
int main()
{
    DIR *dir;
    struct dirent *d;
    if (chdir("dir") != 0)
	return 1;
    dir = opendir(".");
    if (!dir)
	return 2;
    while ((d = readdir(dir)) != NULL) {
	printf("%s\n", d->d_name);
	rmdir(d->d_name);
    }
    closedir(dir);
    return 0;
}
=========================

This program succeeds on vfat and ext3 but it fails to remove "dir/0" on 
ramfs and tmpfs.

169 is not a random number.  It's the minimal number required to reproduce 
the problem.  Maybe other systems need another value.

Basically, removing a subdirectory in a directory open with opendir() 
causes an entry (file or directory) 168 entries later to be skipped by 
readdir().

I'm sorry, I cannot elaborate more, but the issue seems to be very 
serious.

-- 
Regards,
Pavel Roskin

Re: readdir() loses entries on ramfs and tmpfs

[Edgar Toernig]

Pavel Roskin wrote:
> 
>     while ((d = readdir(dir)) != NULL) {
>         printf("%s\n", d->d_name);
>         rmdir(d->d_name);
>     }
>[...]
> Basically, removing a subdirectory in a directory open with opendir()
> causes an entry (file or directory) 168 entries later to be skipped by
> readdir().
> 
> I'm sorry, I cannot elaborate more, but the issue seems to be very
> serious.

Not nice but not serious.  Modifying the directory you scan is
never save and programs relying on this are broken.  You can
build a list of items to work on but even then another process
may add new entries or remove some and you have to handle that.
One could "fix" your "bug" but the program will still be broken.

Ciao, ET.

Re: readdir() loses entries on ramfs and tmpfs

[Legacy Fishtank]

On Wed, Dec 26, 2001 at 07:50:11PM -0500, Pavel Roskin wrote:
> I got a report that GNU Midnight Commander fails to erase some directories 
> on tmpfs from the first attempt.  However, it succeeds the next time.
[...]
>     while ((d = readdir(dir)) != NULL) {
> 	printf("%s\n", d->d_name);
> 	rmdir(d->d_name);
>     }
[...]
> I'm sorry, I cannot elaborate more, but the issue seems to be very 
> serious.

If Midnight Commander does similar to the above it's pretty silly...

In Perl I normally do something like this in a loop (with a max-loops
limiter):

	opendir
	@dirs = readdir(FOO);
	closedir
	last unless @dirs;
	&remove_the_dirs(@dirs);

Clearly that's slack since you could have a million files in a
directory, but you see the point.  readdir(2) and getdents(2)
are inherently racy.  If your code does not assume such and take
appropriate action, it's broken.

	Jeff