/proc reliability & performance

[Albert Cahalan <albert@users.sf.net>]

I created a process with 360 thousand threads,
went into the /proc/*/task directory, and did
a simple /bin/ls. It took over 9 minutes on a
nice fast Opteron. (it's the same at top-level
with processes, but I wasn't about to mess up
my system that much)

OK, that's a bit extreme, but it does show a
scalability problem. There is an O(n*n)
algorithm in there. Here is a proposed fix:

Tie directory readers to a task_struct (or to
some of the PID tracking structs), so that
a directory reader is on a list. When a task
exits, move the list of directory readers on
to a neighboring task.

That is O(1) on task exit, and generally O(n)
for the whole /proc or /proc/42/task read.
It's O(1) per step of the read, excepting
where multiple directory readers wind up at
the same location.

Another benefit is that it is reliable as
long as tasks don't move around on the lists.
Each task will appear at most once, and will
appear exactly once if it doesn't start or
exit during the directory scan.