Re: netperf ~50% regression with 2.6.33-rc1, bisect to 1b9508f

[Mike Galbraith <efault@gmx.de>]

On Mon, 2010-01-25 at 18:03 +0800, Lin Ming wrote:

> With above commit, the idle balance was rate limited, so CPU 15(server,
> waiting data from client) is idle at most time.
> 
> CPU0(client) executes as below,
> 
> try_to_wake_up
>    check_preempt_curr_idle
>       resched_task
>          smp_send_reschedule
> 
> This causes a lot of rescheduling IPI.
> 
> This commit can't be reverted due to conflict, so I just add below code
> to disable "Rate-limit newidle" and the performance was recovered.
> 
> diff --git a/kernel/sched.c b/kernel/sched.c
> index 18cceee..588fdef 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -4421,9 +4421,6 @@ static void idle_balance(int this_cpu, struct rq *this_rq)
>  
>  	this_rq->idle_stamp = this_rq->clock;
>  
> -	if (this_rq->avg_idle < sysctl_sched_migration_cost)
> -		return;
> -
>  	for_each_domain(this_cpu, sd) {
>  		unsigned long interval;
>  

Heh, so you should see the same thing with newidle disabled, as it was
in .31 and many kernels prior.  Do you?

So.  Rummaging around doing absolutely _nothing_ useful, there being
zero movable tasks in this load, prevents us switching to the idle
thread before the scheduling task is requeued.  Oh joy.

Hm.... <imagines Peter's reaction to a busy wait> :)  OTOH, that's what
it's _doing_ with above patch.

	-Mike