Re: [patch 00/41] cpu alloc / cpu ops v3: Optimize per cpu access

[Peter Zijlstra <peterz@infradead.org>]

On Sat, 2008-05-31 at 20:25 -0700, Christoph Lameter wrote:
> On Fri, 30 May 2008, Peter Zijlstra wrote:
> 
> > Remember, horribly long on -rt is in the order of ~30us.
> > 
> > See for instance the path:
> > 
> >  flush_slab()
> >    slab_lock() <-- bit_spinlock
> >    deactive_slab()
> >      loop over objects
> 
> looping over all objects is rarely ever done.
> 
> >      unfreeze_slab()
> >        add_partial()
> >          spin_lock(&n->list_lock) <-- another lock
> > 
> > all of that runs with preempt disabled, and worse it can spin for some
> > indefinite amount of time.
> > 
> > That is totally unacceptable on -rt, so we need to make it all
> > preemptible and use sleeping locks (IRQ inversion can't happen, for the
> > only thing that runs in hardirq is try_to_wake_up()).
> 
> The way I thought about -rt was that it would convert the 
> spinlocks to semaphores. Isnt it possible to sleep on a bit spinlock in 
> -rt? Aside from that I would have thought that the simple disabling of 
> rescheduling on another processor in some context would gets us there. 
> 
> Never thought about the details though.

s/semaphores/PI-mutexes/

yes you can do a bit-sleep-lock ala lock_page.

But for any sleeping lock we must not have preemption disabled, coming
right back to the initial problem.