Re: [BENCHMARK] nproc: Look Ma, No get_tgid_list!

[William Lee Irwin III <wli@holomorphy.com>]

On Tue, Aug 31, 2004 at 05:34:32PM +0200, Roger Luethi wrote:
> This posting demonstrates a new method of monitoring all processes in
> a large system.
> You may remember what a /proc based tool does when monitoring some
> 10^5 processes -- it spends its time in the kernel hanging on to a
> read task_list_lock:
> ==> 10000 processes: top -d 0 -b > /dev/null <==
> CPU: CPU with timer interrupt, speed 0 MHz (estimated)
> Profiling through timer interrupt
> samples  %        image name               symbol name
> 35855    36.0707  vmlinux                  get_tgid_list
> 9366      9.4223  vmlinux                  pid_alive
> 7077      7.1196  libc-2.3.3.so            _IO_vfscanf_internal
> 5386      5.4184  vmlinux                  number
> 3664      3.6860  vmlinux                  proc_pid_stat
[...]

The most crucial issue for larger systems is removing the rather easily
triggerable rwlock starvation. Perhaps dipankar's /proc/ -only tasklist
RCU patch can resolve that.


On Tue, Aug 31, 2004 at 05:34:32PM +0200, Roger Luethi wrote:
> Here's a profile for an nproc based tool monitoring the same set
> of processes:
> ==> 10000 processes: nprocbench <==
> CPU: CPU with timer interrupt, speed 0 MHz (estimated)
> Profiling through timer interrupt
> samples  %        app name                 symbol name
> 8641     24.8626  vmlinux                  __task_mem
> 2778      7.9931  vmlinux                  find_pid
> 2536      7.2968  vmlinux                  finish_task_switch
> 1872      5.3863  vmlinux                  netlink_recvmsg
> 1637      4.7101  vmlinux                  nproc_pid_fields
[...]
> Resource usage is now dominated by field computation, rather than by
> delivery overhead. By now it should be clear that nproc is not only a
> cleaner interface with lower overhead for tools, it also scales a lot
> better than /proc.

With this in hand we can probably ignore the /proc/ -related efficiency
issues in favor of any method preventing the rwlock starvation, e.g.
dipankar's /proc/ -only tasklist RCU patch.


-- wli