Re: [PATCH V2] cpuset: Add knob to make allowed masks hotplug invariant on legacy hierarchy

[Zefan Li <lizefan-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>]

On 2015/4/16 19:57, Preeti U Murthy wrote:
> On 04/15/2015 10:06 PM, Serge E. Hallyn wrote:
>> On Wed, Apr 15, 2015 at 12:18:11PM -0400, Tejun Heo wrote:
>>> On Wed, Apr 15, 2015 at 11:15:35AM -0500, Serge E. Hallyn wrote:
>>>> The reason would be because it breaks "legacy" software.  So that
>>>> would only matter if Preeti needs to run such software.
>>>
>>> Sure, I get that argument but this is changing how the contorller
>>> behaves in a major way.
>>
>> It is.  My main counter to that would be that it is how cpusets
>> should always have worked :)
>>
>>>   There are specifics which may make this
>>> particular case more justifiable but overall the combination of
>>> arguments is pretty weird.
>>
>> And becomes harder to reason about and review/maintain.  I agree
>> there.
>>
>> From userspace, I suppose one approach (though note it is racy) to
>> solving this would be to have udev rules which
>>
>> . On cpu unplug, record all cgroups which were using that cpu
>> . on cpu plug, re-add the cpu to all recorded cgroups for that
>> cpu (if any), as well as to any cgroups marked (in some /etc
>> file) as using "all" or a percentage of all cpus.
> 
> Yes this is the best way to move forward. But the stacks in userspace
> that manage cgroups need to each have such a daemon managing the cpusets
> like is pointed above. It would have been fair to expect them to do this
> except that this is one issue that userspace is expecting the kernel to
> handle. The side effects of this behavior of the legacy hierarchy is
> showing up in weird ways.
> 
> Another approach as Tejun pointed is to switch over to the unified
> hierarchy. I can test this out (How do we mount specific controllers in
> the unified hierarchy Tejun? It does not allow any mount options today),

You just can't. All controllers are bound to unified hierarchy, except those
already mounted in the old way. That said, in the unified hierarchy you're able
to use cpuset only, so other controllers can still be mounted seperated.

Here is an example:

  # use blkio cgroup in legacy hierachy
  mount -t cgroup -o blkio xxx old/

  # now mount the unified hierarchy
  mount -t cgroup -o __DEVEL__sane_behavior xxx unified/

  # use cpuset only
  mkdir unified/cpuset
  echo +cpuset > cgroup.subtree_control 

  # use memory cgroup in another legacy hierarchy
  mount -t cgroup -o memory xxx old2/

You may refer to Documentation/cgroups/unified-hierarchy.txt

> but I don't think I can exhaustively test this out by myself across
> distros. So I am not confident of this approach.
> 
> The last approach would be as Peter pointed, mimick the entire unified
> hierarchy through a mount option, similar to the SANE_BEHAVIOR option
> that was present earlier and was removed. Tejun ? Can we bring back that
> mount option ?
> 

I don't think it's an option. We want users move to unified hierarchy.