Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE

[Peter Zijlstra <peterz@infradead.org>]

On Wed, May 27, 2015 at 05:22:16PM -0400, Josef Bacik wrote:
> 
> SD_BALANCE_WAKE is supposed to find us an idle cpu to run on, however

sd->flags's SD_BALANCE_WAKE, not sd_flags.

And note that SD_BALANCE_WAKE is not set on domains by default.

> it is just looking for an idle sibling, preferring affinity over all
> else.

Not sure what you're saying here, what affinity?

> This is not helpful in all cases, and SD_BALANCE_WAKE's job is
> to find us an idle cpu, not garuntee affinity.

Your argument is going backwards, SD_BALANCE_WAKE is not actually set.

> Fix this by first
> trying to find an idle sibling, and then if the cpu is not idle fall
> through to the logic to find an idle cpu.  With this patch we get
> slightly better performance than with our forward port of
> SD_WAKE_IDLE. 

This is broken. You most certainly do not want to go do that whole load
balance pass on wakeups. It should be controlled by sd->flags. It is far
too expensive to consider turning that on by default.

In fact, select_idle_sibling() is already too expensive on current
server hardware (far too damn many cpus in a LLC domain).