readlink(), the pause that refreshes

[Andre Majorel <aym-xunil@teaser.fr>]

Kernel 2.6.22.14. Create a tree of directories containing tens or
hundreds of thousands of symlinks and run

  find linkfarm | program-that-readlinks-the-file-names-in-its-stdin

Memory usage will grow and grow until the system starts to swap,
after which you are SOL. The combined effects of thrashing and the
OOM killer will render your system catatonic for a long time and
if it finally recovers, you probably won't be able to log in. With
1 GB of RAM, the tipping point is somewhere above 700,000
symlinks.

So you kill the pipe before it starts to swap. Memory usage does
not go down. Try "sync" or "umount /linkfarm". They will block for
minutes, during which memory usage goes down slowly, on the order
of 1 MB per second. All the while, iostat indicates several
hundred transactions per second on the disk where the link farm
resides.

All this crap appears to be related to the updating of the atimes
after a readlink. I understand that the problem could be
alleviated by mounting the file system noatime or relatime, but
this not just a performance problem. You should not be able to
make your system unusable with just one process making readlink()
calls.

As a stop gap, is there an equivalent of O_NOATIME for readlink() ?

-- 
André Majorel <URL:http://www.teaser.fr/~amajorel/>
Do not use this account for regular correspondence.
See the URL above for contact information.