Re: cpusets not cpu hotplug aware

[Nathan Lynch <ntl@pobox.com>]

Andrew Morton wrote:
> On Mon, 21 Aug 2006 14:01:48 -0700
> Paul Jackson <pj@sgi.com> wrote:
> > Anton wrote:
> > > If cpuset_cpus_allowed doesnt return the current online mask and we want
> > > to schedule on a cpu that has been added since boot it looks like we
> > > will fail.
> > 
> > In general, on systems actually using cpusets, that -is- what should
> > happen.  Just because a cpu was brought online doesn't mean it was
> > intended to be allowed in any given tasks current cpuset.
> > 
> > Granted, I would guess users of systems not using cpusets (but
> > still have cpusets configured in their kernel, as is common in some
> > distro kernels), would expect the behaviour you expected - bringing
> > a cpu (or memory node) on or offline would make it available (or
> > not) for something like a sched_setaffinity (or mbind/set_mempolicy)
> > immediately, without having to invoke some magic cpuset voodoo.
> > 
> > Offhand, this sounds to me like a choice of two modes of operation.
> > 
> >     If you aren't actually using cpusets (so every task is in the
> >     original top_cpuset) then you'd expect that cpuset to "get out
> >     of the way", meaning top_cpuset (the only cpuset, in this case)
> >     tracked whatever cpus and nodes were online at the moment.
> > 
> >     If instead you start messing with cpusets (creating more than one
> >     of them and moving tasks between them) then you'd expect cpusets
> >     to be enforced, without automatically adding newly added cpus or
> >     memory nodes to existing cpusets.  Only the user knows which
> >     cpusets should get the added cpus or memory nodes in this case.
> > 
> > I don't jump for joy over yet another modal state flag.  But I don't see
> > a better alternative -- do you?
> > 
> 
> If the kernel provider (ie: distro) has enabled cpusets then it would be
> appropriate that they also provide a hotplug script which detects whether their
> user is actually using cpusets and if not, to take some sensible default action. 
> ie: add the newly-added CPU to the system's single cpuset, no?

I think it would be more sensible for the default (i.e. user hasn't
explicitly configured any cpusets) behavior on a CONFIG_CPUSETS=y
kernel to match the behavior with a CONFIG_CPUSETS=n kernel.  Right
now we don't have that.  Binding a task to a newly added cpu just
fails if CONFIG_CPUSETS=y, but it works if CONFIG_CPUSETS=n.