Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE

[Josef Bacik <jbacik@fb.com>]

On 06/17/2015 05:55 PM, Mike Galbraith wrote:
> On Wed, 2015-06-17 at 11:06 -0700, Josef Bacik wrote:
>> On 06/11/2015 10:35 PM, Mike Galbraith wrote:
>>> On Thu, 2015-05-28 at 13:05 +0200, Peter Zijlstra wrote:
>
>>> If sd == NULL, we fall through and try to pull wakee despite nacked-by
>>> tsk_cpus_allowed() or wake_affine().
>>>
>>
>> So maybe add a check in the if (sd_flag & SD_BALANCE_WAKE) for something
>> like this
>>
>> if (tmp >= 0) {
>> 	new_cpu = tmp;
>> 	goto unlock;
>> } else if (!want_affine) {
>> 	new_cpu = prev_cpu;
>> }
>>
>> so we can make sure we're not being pushed onto a cpu that we aren't
>> allowed on?  Thanks,
>
> The buglet is a messenger methinks.  You saying the patch helped without
> SD_BALANCE_WAKE being set is why I looked.  The buglet would seem to say
> that preferring cache is not harming your load after all.  It now sounds
> as though wake_wide() may be what you're squabbling with.
>
> Things aren't adding up all that well.

Yeah I'm horribly confused.  The other thing is I had to switch clusters 
(I know, I know, I'm changing the parameters of the test).  So these new 
boxes are haswell boxes, but basically the same otherwise, 2 socket 12 
core with HT, just newer/faster CPUs.  I'll re-run everything again and 
give the numbers so we're all on the same page again, but as it stands 
now I think we have this

3.10 with wake_idle forward ported - good
4.0 stock - 20% perf drop
4.0 w/ Peter's patch - good
4.0 w/ Peter's patch + SD_BALANCE_WAKE - 5% perf drop

I can do all these iterations again to verify, is there any other 
permutation you'd like to see?  Thanks,

Josef