Re: [PATCH] 2.4.0-prerelease: preemptive kernel.

Nigel Gamble
2001-01-04T22:29:26Z

On Thu, 4 Jan 2001, Andi Kleen wrote:
> On Thu, Jan 04, 2001 at 01:39:57PM -0800, Nigel Gamble wrote:
> > Experience has shown that adaptive spinlocks are not worth the extra
> > overhead (if you mean the type that spin for a short time
> > and then decide to sleep). It is better to use spin_lock_irqsave()
> > (which, by definition, disables kernel preemption without the need
> > to set a no-preempt flag) to protect regions where the lock is held
> > for a maximum of around 100us, and to use a sleeping mutex lock for
> > longer regions. This is what I'm working towards.
>
> What experience ? Only real-time latency testing or SMP scalability
> testing?

Both. We spent a lot of time on this when I was at SGI working on IRIX.
I think we ended up with excellent SMP scalability and good real-time
latency. There is also some academic research that suggests that
the extra overhead of a dynamic adaptive spinlock usually outweighs
any possible gains.

> The case I was thinking about is a heavily contended lock like the
> inode semaphore of a file that is used by several threads on several
> CPUs in parallel or the mm semaphore of a often faulted shared mm.
>
> It's not an option to convert them to a spinlock, but often the delays
> are short enough that a short spin could make sense.

I think the first order performance problem of a heavily contended lock
is not how it is implemented, but the fact that it is heavily contended.
In IRIX we spent a lot of time looking for these bottlenecks and
re-architecting to avoid them. (This would mean minimizing the shared
accesses in your examples.)

Nigel Gamble nigel@nrg.org
Mountain View, CA, USA. http://www.nrg.org/