Re: [Patch] Idle balancer: cache align nohz structure to improve idle load balancing scalability

[Peter Zijlstra <peterz@infradead.org>]

On Tue, 2011-11-01 at 16:52 -0700, Suresh Siddha wrote:
> @@ -3317,6 +3317,7 @@ static void update_cpu_power(struct sched_domain *sd, int cpu)
>  
>         cpu_rq(cpu)->cpu_power = power;
>         sdg->sgp->power = power;
> +       atomic_set(&sdg->sgp->nr_busy_cpus, sdg->group_weight);
>  }
>  
>  static void update_group_power(struct sched_domain *sd, int cpu)
> @@ -3339,6 +3340,7 @@ static void update_group_power(struct sched_domain *sd, int cpu)
>         } while (group != child->groups);
>  
>         sdg->sgp->power = power;
> +       atomic_set(&sdg->sgp->nr_busy_cpus, sdg->group_weight);
>  } 

So we run this rather frequently, and it will trample all over:

> +                */
> +               for_each_domain(cpu, sd)
> +                       atomic_dec(&sd->groups->sgp->nr_busy_cpus);

because I cannot see any serialization between those sites.

Also, isn't it rather weird to just assume all cpus are busy in
update_group_power()? If you would actually set the right value in
update_cpu_power() you could use a straight sum in update_group_power()
and get a more or less accurate number out.