Re: [PATCH 2/2] smp_call_function: use rwlocks on queues rather than rcu

[Andi Kleen <andi@firstfloor.org>]

On Fri, Aug 22, 2008 at 11:35:46AM -0700, Jeremy Fitzhardinge wrote:
> Andi Kleen wrote:
> > Right now my impression is that it is not well understood why
> > the kmalloc makes the IPI that much slower. In theory a kmalloc
> > shouldn't be all that slow, it's essentially just a 
> > "disable interrupts; unlink object from cpu cache; enable interrupts"
> > with some window dressing. kfree() is similar.
> >
> > Does it bounce a cache line on freeing perhaps?
> 
> I think it's just an assumption that it would be slower.  Has anyone
> measured it?

It's likely slower than no kmalloc because
there will be more instructions executed, the question is just how much.

> 
> (Note: The measurements I posted do not cover this path, because it was
> on a two cpu system, and it was always using the call-single path.)

Ah so it was already 25% slower even without kmalloc? I thought
that was with already. That doesn't sound good. Any idea where that slowdown 
comes from?

-Andi