Re: cpusets not cpu hotplug aware

[Nathan Lynch <ntl@pobox.com>]

Paul Jackson wrote:
> Nathan wrote:
> > How about this? 
> 
> The code likely works, and the locking seems ok at first blush.
> And this patch seems to match just what I asked for ;).
> 
> But the more I think about this, the less I like this direction.
> 
> Your patch, and what I initially asked for, impose a policy and create
> a side affect.  When you bring a cpu online, the top cpuset changes as
> a side affect, in order to impose a policy that the top cpuset tracks
> what is online.
> 
> The kernel should avoid such side affects and avoid imposing policy.

If you want to call that a policy that's fine, but it hardly seems to
be a non-intuitive behavior to me, especially in the case that the
administrator has not explicitly configured any cpusets.


> It should be user code that imposes the policy that the top cpuset
> tracks what is online.
> 
> The kernel gets things going with reasonable basic defaults at system
> boot, then adapts to whatever user space mandates from then on.
> 
> Kernels should provide generic, orthogonal mechanisms.

Maybe the cpuset code should just stay out of the way unless the admin
has instantiated one?


> Let user space figure out what it wants to do with them.

The user who initially reported this probably has no idea what cpusets
are or how to deal with the situation.


> It is not a kernel bug that the top cpuset doesn't track what is
> online.

It's a bug that one's ability to bind a task to a new cpu is entirely
dependent on whether you know your way around cpusets.  Doesn't sound
orthogonal to me.

Try looking at it this way.  You have an application that works on
distro x, where cpu hotplug is supported but cpusets is not enabled in
the kernel config.  That application is cpu hotplug-aware, and binding
a task to a new cpu just works.  Now, with distro x+1, cpusets is
enabled and that same scenario doesn't work anymore.  That's generally
viewed as a regression.

And no, I don't like pushing the responsibility for fixing up the
top-level cpuset out to userspace -- that would force apps to
busy-wait (and for how long?) for the new cpu to be added to the
top-level cpuset by the hotplug helper.  It means something as simple
as this:

# echo 1 > /sys/devices/system/cpu/cpu3/online
# taskset 0x8 foo

has a race condition, depending on your kernel's configuration.