[patch] big IRQ lock removal, 2.5.27-G0

[patch] big IRQ lock removal, 2.5.27-G0

[Ingo Molnar]

the -G0 irqlock patch can be found at:

   http://redhat.com/~mingo/remove-irqlock-patches/remove-irqlock-2.5.27-G0

Changes in -G0:

 - fix buggy in_softirq(). Fortunately the bug made the test broader,
   which didnt result in algorithmical breakage, just suboptimal
   performance.

 - move do_softirq() processing into irq_exit() => this also fixes the
   softirq processing bugs present in apic.c IRQ handlers that did not
   test for softirqs after irq_exit().

 - simplify local_bh_enable().

Changes in -F9:

 - replace all instances of:

	local_save_flags(flags);
	local_irq_disable();

   with the shorter form of:

	local_irq_save(flags);

  about 30 files are affected by this change.

Changes in -F8:

 - preempt/hardirq/softirq count separation, cleanups.

 - skbuff.c fix.

 - use irq_count() in scheduler_tick()

Changes in -F3:

 - the entry.S cleanups/speedups by Oleg Nesterov.

 - a rather critical synchronize_irq() bugfix: if a driver frees an 
   interrupt that is still being probed then synchronize_irq() locks up.
   This bug has caused a spurious boot-lockup on one of my testsystems,
   ifconfig would lock up trying to close eth0.

 - remove duplicate definitions from asm-i386/system.h, this fixes 
   compiler warnings.

	Ingo

Re: [patch] big IRQ lock removal, 2.5.27-G0

[Oleg Nesterov]

Hello.

I can't understand the preempt_count() check in irq_exit()
and local_bh_enable(). Did you meant irq_count() instead?
I beleive preempt_disable() should not disable soft interrupts.

In local_bh_enable() it is probably unneeded at all, because
nested local_bh_disable() is rare (i think), and do_softirq()
checks in_interrupt().

Now suppose preempt_count() & PREEMPT_MASK == 0.

Then local_bh_enable() has a small preemptible window between
__local_bh_enable() and do_softirq()->local_irq_save(flags).
It is only latency problem.

But in irq_exit() case interrupt context may be preempted
while doing wakeup_softirqd(cpu) after __local_bh_enable()
in do_softirq().

So i suggest something like this pseudo code:

__preempt_hack(offset)
{
        barrier();
        preempt_count() -= offset
#ifdef  CONFIG_PREEMPT
                        - 1
#endif
        ;
}

irq_exit()
{
        __preempt_hack(HARDIRQ_OFFSET);
        if (unlikely(!irq_count() &&
                        softirq_pending(smp_processor_id())))
                do_softirq();
        preempt_enable_no_resched();
}

local_bh_enable()
{
        __preempt_hack(SOFTIRQ_OFFSET);
        if (unlikely(!irq_count() &&
                        softirq_pending(smp_processor_id())))
                do_softirq();
        preempt_enable();
}

Or just add extra preempt_disable() in both functions to kill
terrible __preempt_hack().

Sorry, i have no remove-irqlock patches applied, so can't
suggest diff.

Oleg.

Re: [patch] big IRQ lock removal, 2.5.27-G0

[Ingo Molnar]

On Wed, 24 Jul 2002, Oleg Nesterov wrote:

> I can't understand the preempt_count() check in irq_exit() and
> local_bh_enable(). Did you meant irq_count() instead? I beleive
> preempt_disable() should not disable soft interrupts.

yes, you are right, and i fixed them in -G3.

doh, i *thought* i fixed both places in -G3, but i only fixed irq_exit().  
the local_bh_enable() fix is in -G4:

  http://redhat.com/~mingo/remove-irqlock-patches/remove-irqlock-2.5.27-G4

> In local_bh_enable() it is probably unneeded at all, because nested
> local_bh_disable() is rare (i think), and do_softirq() checks
> in_interrupt().

but still we dont want to call do_softirq() all the time. local_bh_enable
is used in quite performance-sensitive networking code.

> Now suppose preempt_count() & PREEMPT_MASK == 0.
> 
> Then local_bh_enable() has a small preemptible window between
> __local_bh_enable() and do_softirq()->local_irq_save(flags).
> It is only latency problem.

i dont think getting a preemption before softirqs are processed is a big
problem. Such type of preemption comes in form of an interrupt, which 
handles softirqs in irq_exit() anyway, so there's no delay.

> But in irq_exit() case interrupt context may be preempted
> while doing wakeup_softirqd(cpu) after __local_bh_enable()
> in do_softirq().

okay - i've fixed irq_exit() once more in -G4, could you double-check it?

	Ingo

Re: [patch] big IRQ lock removal, 2.5.27-G0

[Ingo Molnar]

On Wed, 24 Jul 2002, Ingo Molnar wrote:

> > local_bh_disable() is rare (i think), and do_softirq() checks
> > in_interrupt().
> 
> but still we dont want to call do_softirq() all the time. local_bh_enable
> is used in quite performance-sensitive networking code.

in fact the in_interrupt() check in do_softirq() should never trigger with
the latest patch applied - i'll put a debugging printk there and we can
remove it after some time. This will speed things up a bit.

	Ingo

Re: [patch] big IRQ lock removal, 2.5.27-G0

[Oleg Nesterov]

Hello.

> > Then local_bh_enable() has a small preemptible window between
>
> i dont think getting a preemption before softirqs are processed is a big
> problem. Such type of preemption comes in form of an interrupt, which

Ah, yes...

> > But in irq_exit() case interrupt context may be preempted
>
> okay - i've fixed irq_exit() once more in -G4

found G5, your forgot to add preempt_disable() in irq_exit()

 #define irq_exit()
 do {
+               preempt_disable();
                preempt_count() -= IRQ_EXIT_OFFSET;
                if (!in_interrupt() && softirq_pending(smp_processor_id()))

Oleg.

Re: [patch] big IRQ lock removal, 2.5.27-G0

[Oleg Nesterov]

Hello.

> > Then local_bh_enable() has a small preemptible window between
> > __local_bh_enable() and do_softirq()->local_irq_save(flags).
> > It is only latency problem.
> 
> i dont think getting a preemption before softirqs are processed is a big
> problem. Such type of preemption comes in form of an interrupt, which
> handles softirqs in irq_exit() anyway, so there's no delay.

Well, no. Not all smp_xxx_interrupt() use irq_enter/exit().
Reschedule interrupt, for example, do not. But indeed, it is not
big problem.

Oleg.

Re: [patch] big IRQ lock removal, 2.5.27-G0

[Ingo Molnar]

On Wed, 24 Jul 2002, Oleg Nesterov wrote:

> > okay - i've fixed irq_exit() once more in -G4
> 
> found G5, your forgot to add preempt_disable() in irq_exit()
> 
>  #define irq_exit()
>  do {
> +               preempt_disable();
>                 preempt_count() -= IRQ_EXIT_OFFSET;
>                 if (!in_interrupt() && softirq_pending(smp_processor_id()))
> 
> Oleg.

nope, it's tricky, check out the define of IRQ_EXIT_OFFSET, it has the
plus 1 count added already.

using preempt_disable() has the problem of putting a barrier() between the
++ and the -IRQ_OFFSET, causing one more instruction to be generated.

but there was another (not too critical) bug here - irq_enter() needs to
have a barrier() after manipulating the preemption count, irq_exit() needs
to have a barrier() before. Fixed in my tree.

	Ingo