Re: sched_mc_power_savings broken with CGROUPS+CPUSETS

[Max Krasnyansky <maxk@qualcomm.com>]

Vaidyanathan Srinivasan wrote:
> Hi,
> 
> sched_mc_power_savings seems to be broken with CGROUPS+CPUSETS.
> When CONFIG_CPUSETS=y the attached BUG_ON() is being hit.
> 
> I added a BUG_ON to check if SD_POWERSAVINGS_BALANCE is set at
> SD_LV_CPU whenever sched_mc_power_savings is set.
> 
> This BUG is hit when config CONFIG_CPUSETS (depends on CONFIG_CGROUPS)
> is just compiled in while this is never hit when they are compiled
> out.  The fact that SD_POWERSAVINGS_BALANCE being cleared even when
> sched_mc_power_savings = 1 completely breaks the
> sched_mc_power_savings heuristics.
> 
> To recreate the problem,
> Have sched_mc power savings enabled CONFIG_SCHED_MC=y
> Add this BUG_ON()
> 
> echo 1 > /sys/devices/system/cpu/sched_mc_power_savings
> 
> Try these these on a multi core x86 box.
> 
> sched_mc_power_savings seems to be broken from 2.6.26-rc1, but
> I do not have a confirmation that the root cause is same in all
> successive versions. sched_mc_power_savings works perfect in
> 2.6.25.
> 
> Please help me root cause the issue.  Please point me to changes that
> may potential cause this bug.

That's my fault. I redid domain rebuild/hotplug handling awhile ago and missed
the fact that partition_sched_domains() is trying to avoid unnecessary domain
rebuilds. Primary issue at the time was circular locking issues and the the
testing that I was doing then was exercising a bunch of different scenarios
(ie cpu up/down, cpuset create/destroy, mcpowersave on/off) and I did not
notice that mcpowersave did not actually trigger domain rebuilds.

Anyway, I sent you guys a patch that should fix this issue.
Please confirm.

Max