Re: [PATCH v2] sched: keep quiescent cpu out of idle balance loop

[Mike Galbraith <bitbucket@online.de>]

On Fri, 2014-02-21 at 15:28 +0800, Lei Wen wrote:

> Actually, what I have experiment is as:
> 1. set top cpuset as disable load balance
> 2. set 0-2 cpus to "system", and enable its load balance
> 3. set 3 cpu to "rt" and disable load balance.

Exactly as I do, pertinent part of my cheezy script being...

        # ...and fire up the shield
        cset shield --userset=rtcpus --cpu=${START_CPU}-${END_CPU} --kthread=on

        # If cpuset wasn't previously mounted (no obnoxious systemd),
        # we just mounted it.  Find the mount point.
        if [ -z $CPUSET_ROOT ]; then
                CPUSET_ROOT=$(grep cpuset /proc/mounts|cut -d ' ' -f2)
                if [ -z $CPUSET_ROOT ]; then
                        # If it's not mounted now, bail.
                        echo EEK, cupset is not mounted!
                        exit
                else
                        # ok, check for cgroup mount
                        if [ -f ${CPUSET_ROOT}/cpuset.cpus ]; then
                                CPUSET_PREFIX=cpuset.
                        fi
                fi
        fi

        echo 0 > ${CPUSET_ROOT}/${CPUSET_PREFIX}sched_load_balance
        echo 1 > ${CPUSET_ROOT}/system/${CPUSET_PREFIX}sched_load_balance
        echo 0 > ${CPUSET_ROOT}/rtcpus/${CPUSET_PREFIX}sched_load_balance
        echo 0 > ${CPUSET_ROOT}/rtcpus/${CPUSET_PREFIX}sched_relax_domain_level

> While by this way, root span always covering [0-2] which is seen
> by cpu 0-2, as you also mentioned.
> And it is true that if I disable load balance, I would see span mask
> get them merged.
> 
> So how about below change?
> +       if (!this_rq()->sd)
> +               return;
> Suppose isolated cpu would lose its sd, could you help
> confirm it from crash too?

Yeah, isolated CPUs have no sd connectivity.  I sent Peter a patchlet
offline showing what I do to keep nohz at bay.
> Or, you think it is wrong to do merge job when system group disable
> the load balance?

I think the construction stuff works fine, and !->sd is the perfect cue
to tell various things to keep their grubby mitts off of a CPU.

-Mike