Re: [Lse-tech] Re: NUMA scheduler (was: 2.5 merge candidate list 1.5)

[Michael Hohnbaum <hohnbaum@us.ibm.com>]

> I'm trying Michael's balance_exec on top of your patch 1 at the 
> moment, but I'm somewhat confused by his code for sched_best_cpu.
> 
> +static int sched_best_cpu(struct task_struct *p)
> +{
> +       int i, minload, best_cpu, cur_cpu, node;
> +       best_cpu = task_cpu(p);
> +       if (cpu_rq(best_cpu)->nr_running <= 2)
> +               return best_cpu;
> +
> +       node = __cpu_to_node(__get_cpu_var(last_exec_cpu));
> +       if (++node >= numnodes)
> +               node = 0;
> +       
> +       cur_cpu = __node_to_first_cpu(node);
> +       minload = cpu_rq(best_cpu)->nr_running;
> +
> +       for (i = 0; i < NR_CPUS; i++) {
> +               if (!cpu_online(cur_cpu))
> +                       continue;
> +
> +               if (minload > cpu_rq(cur_cpu)->nr_running) {
> +                       minload = cpu_rq(cur_cpu)->nr_running;
> +                       best_cpu = cur_cpu;
> +               }
> +               if (++cur_cpu >= NR_CPUS)
> +                       cur_cpu = 0;
> +       }
> +       __get_cpu_var(last_exec_cpu) = best_cpu;
> +       return best_cpu;
> +}
> 
> Michael, the way I read the NR_CPUS loop, you walk every cpu
> in the system, and take the best from all of them. In which case
> what's the point of the last_exec_cpu stuff? On the other hand, 
> I changed your NR_CPUS to 4 (ie just walk the cpus in that node), 
> and it got worse. So perhaps I'm just misreading your code ...
> and it does seem significantly cheaper to execute than Erich's.
> 
You are reading it correct.  The only thing that the last_exec_cpu
does is to help spread the load across nodes.  Without that what was
happening is that node 0 would get completely loaded, then node 1,
etc.  With it, in cases where one or more runqueues have the same
length, the one chosen tends to get spread out a bit.  Not the 
greatest solution, but it helps.
> 
-- 
Michael Hohnbaum            503-578-5486
hohnbaum@us.ibm.com         T/L 775-5486