Re: [RFC][PATCH] CGroups: Add a per-subsystem hierarchy lock

[Max Krasnyansky <maxk@qualcomm.com>]

Paul Menage wrote:
> On Wed, Jul 2, 2008 at 9:46 PM, Max Krasnyansky <maxk@qualcomm.com> wrote:
>> mkdir /dev/cpuset
>> mount -t cgroup -o cpuset cpuset /dev/cpuset
>> mkdir /dev/cpuset/0
>> mkdir /dev/cpuset/1
>> echo 0-2 > /dev/cpuset/0/cpuset.cpus
>> echo 3   > /dev/cpuset/1/cpuset.cpus
>> echo 0 > /dev/cpuset/cpuset.sched_load_balance
>> echo 0 > /sys/devices/system/cpu/cpu3/online
>>
> 
> OK, I still can't reproduce this, on a 2-cpu system using one cpu for
> each cpuset.
> 
> But the basic problem seemns to be that we have cpu_hotplug.lock taken
> at the outer level (when offlining a CPU) and at the inner level (via
> get_online_cpus() called from the guts of partition_sched_domains(),
> if we didn't already take it at the outer level.
Yes. I'm looking at this stuff right now.

> While looking at the code trying to figure out a nice way around this,
> it struck me that we have the call path
> 
> cpuset_track_online_nodes() ->
>   common_cpu_mem_hotplug_unplug() ->
>     scan_for_empty_cpusets() ->
>       access cpu_online_map with no calls to get_online_cpus()
> 
> Is that allowed? Maybe we need separate versions of
> scan_for_empty_cpusets() that look at memory and cpus?
Yes. This also came up in the other discussions. ie That we need to have this
split up somehow. Looks like my fix for domain handling exposed all kinds of
problems that we punted on before :).

> I think that we're going to want eventually a solution such pushing
> the locking of cpuset_subsys.hierarchy_mutex down into the first part
> of partition_sched_domains, that actually walks the cpuset tree
Right now I'm trying to get away with not using new locks. Since we need some
solution for 2.6.26 and new cgroup locking that you added is probably a .27
material.

Max