Re: [Patch][RFC] Disabling per-tgid stats on task exit in taskstats

[Andrew Morton <akpm@osdl.org>]

Shailabh Nagar <nagar@watson.ibm.com> wrote:
>
> Based on previous discussions, the above solutions can be expanded/modified to:
> 
> a) allow userspace to listen to a group of cpus instead of all. Multiple
> collection daemons can distribute the load as you pointed out. Doing collection
> by cpu groups rather than individual cpus reduces the aggregation burden on
> userspace (and scales better with NR_CPUS)
> 
> b) do flow control on the kernel send side. This can involve buffering and sending
> later (to handle bursty case) or dropping (to handle sustained load) as pointed out
> by you, Jamal in other threads.
> 
> c) increase receiver's socket buffer. This can and should always be done but no
> involvement needed.
> 
> 
> With regards to taskstats changes to handle the problem and its impact on userspace
> visible changes,
> 
> a) will change userspace
> b) will be transparent.
> c) is immaterial going forward (except perhaps as a change in Documentation)
> 
> 
> I'm sending a patch that demonstrates how a) can be done quite simply
> and a patch for b) is in progress.
> 
> If the approach suggested in patch a) is acceptable (and I'll provide the testing, stability
> results once comments on it are largely over), could taskstats acceptance in 2.6.18 go ahead
> and patch b) be added later (solution outline has already been provided and a prelim patch should
> be out by eod)

Throwing more CPUs at the problem makes heaps of sense.

It's not necessarily a userspace-incompatible change.  As long as userspace
sets nl_pid to 0x00000000, future kernel revisions can treat that as "all
CPUs".  Or userspace can be forward-compatible by setting nl_pid to
0xffff0000, or whatever.