Re: [PATCH v2 1/3] sched: introduce distinct per-cpu load average

[Andrea Righi <andrea@betterlinux.com>]

On Mon, Oct 22, 2012 at 01:10:40PM +0200, Peter Zijlstra wrote:
> On Sat, 2012-10-20 at 21:06 +0200, Andrea Righi wrote:
> > @@ -383,13 +383,7 @@ struct rq {
> >         struct list_head leaf_rt_rq_list;
> >  #endif
> >  
> 
> > +       unsigned long __percpu *nr_uninterruptible; 
> 
> This is O(nr_cpus^2) memory..
> 

Correct, this doesn't add too much overhead to the wakeup/sleep path,
but it's bad both in terms of memory and performance overhead in the
other parts of the code for large SMP systems.

> 
> > +unsigned long nr_uninterruptible_cpu(int cpu)
> > +{
> > +       struct rq *this = cpu_rq(cpu);
> > +       unsigned long val = 0;
> > +       int i;
> > +
> > +       for_each_online_cpu(i)
> > +               val += per_cpu(*this->nr_uninterruptible, i);
> > +
> > +       return val;
> > +}
> > 
> > 
> I suspect you've got an accounting leak here on hot-plug.

And I think you're right about the accounting leak with cpu hotplug.
I'll do more tests with this part, until I come up with a better idea
in general for the nr_uninterruptible accounting.

Thanks!
-Andrea

> > 
> >  unsigned long nr_uninterruptible(void)
> >  {
> >         unsigned long i, sum = 0;
> >  
> >         for_each_possible_cpu(i)
> > -               sum += cpu_rq(i)->nr_uninterruptible;
> > +               sum += nr_uninterruptible_cpu(i);
> >  
> >         /*
> >          * Since we read the counters lockless, it might be slightly
> 
> And this makes O(n^2) runtime!
>