Re: CONFIG_PREEMPT and server workloads

[Jamie Lokier <jamie@shareable.org>]

Robert Love wrote:
> This is because of work Dave Miller and Ingo did - irq count, softirq
> count, and lock count (when PREEMPT=y) are unified into preempt_count. 

(x86 only) Have you considerd unifying the "interrupts disabled" bit
into it as well, for spin_lock_irqsave/spin_unlock_irqrestore?

The principle is to use a memory bit or counter instead of "cli" and
"sti", because it is cheaper or even free when it's unified in
preempt_count.

The very first instructions of the interrupt handlers check
preempt_count, and if interrupts are logically disabled they modify
preempt_count to indicate that an irq is pending, save the interrupt
vector number, and return keeping interrupts disabled at the CPU level
(i.e. not using "iret").  Only one vector can be pending in this way,
because we keep CPU interrupts disabled after the first one.

In spin_unlock_irqrestore, it checks preempt_count (as it already
does), and in the slow path if there's an irq pending, calls the
handler for that irq as if it were invoked by the CPU.

That should effectively eliminate the "cli" and "sti" isnructions
from spin_lock_irqsave and spin_unlock_irqrestore, saving a few cycles.

-- Jamie