Re: [announce, patch, RFC] "big IRQ lock" removal, IRQ cleanups.

Re: [announce, patch, RFC] "big IRQ lock" removal, IRQ cleanups.

[Oleg Nesterov]

Hello.

> > [...] To do this from irq.c means that it must exit with interrupts off
> > and the the low level code needs to keep them off till the irtn. [...]

> yes, we are very careful to keep irqs disabled in do_IRQ(), both before
> and after calling the handler.

Note that smp_xxx_interrupt() functions must be carefull
with preemt_{disable,enable} brackets.

For example, smp_invalidate_interrupt() may be preempted
after put_cpu(). Probably not big deal (it is return path),
but it is better to use preempt_enable_no_resched() here -
let ret_from_intr: do its job.

smp_{error,spurious,thermal}_interrupt() - all of them
use printk() without bumping preemt_count and have problem
after spin_unlock_irqrestore(&logbuf_lock, flags).

If these problems worth fixing, then preempt_stop (cli)
can be killed in entry.S:ret_from_intr(), yes? If i understand
correctly none of the irq handlers should return to low level
code with irq enabled.

P.S.

May I suggest somebody with good english fix
Documentation/preempt-locking.txt?
It states, that disabled interrupts prevents preemption.
Yes, but only in a sense, that the delivery of reschedule
interrupt is suppressed.

Process with irqs disabled and current->preempt_count == 0 can
be preempted (with interrupts enabled) after spin_lock/unlock etc.
Even in UP case preemption can happen while calling wake_up_...().

Oleg.

[patch] big IRQ lock removal, 2.5.27-D9

[Ingo Molnar]

On Mon, 22 Jul 2002, Oleg Nesterov wrote:

> Note that smp_xxx_interrupt() functions must be carefull
> with preemt_{disable,enable} brackets.
> 
> For example, smp_invalidate_interrupt() may be preempted
> after put_cpu(). Probably not big deal (it is return path),
> but it is better to use preempt_enable_no_resched() here -
> let ret_from_intr: do its job.

i solved it slightly differently: added a new put_cpu_no_resched() macro.

> smp_{error,spurious,thermal}_interrupt() - all of them
> use printk() without bumping preemt_count and have problem
> after spin_unlock_irqrestore(&logbuf_lock, flags).

fixed this - all these IRQ vector paths must use irq_enter()/irq_exit()  
pairs.

> If these problems worth fixing, then preempt_stop (cli) can be killed in
> entry.S:ret_from_intr(), yes? If i understand correctly none of the irq
> handlers should return to low level code with irq enabled.

yes, it can be removed, and i did this.

> May I suggest somebody with good english fix
> Documentation/preempt-locking.txt?
> It states, that disabled interrupts prevents preemption.
> Yes, but only in a sense, that the delivery of reschedule
> interrupt is suppressed.
> 
> Process with irqs disabled and current->preempt_count == 0 can
> be preempted (with interrupts enabled) after spin_lock/unlock etc.
> Even in UP case preemption can happen while calling wake_up_...().

added such a section to preempt-locking.txt.

this and the other changes can be found at:

   http://redhat.com/~mingo/remove-irqlock-patches/remove-irqlock-2.5.27-D9

is anything else missing?

	Ingo

Re: [patch] big IRQ lock removal, 2.5.27-D9

[Oleg Nesterov]

Hello.

> > then preempt_stop (cli) can be killed in entry.S:ret_from_intr()
>
> yes, it can be removed, and i did this.

Sorry, forgot to say that GET_THREAD_INFO(%ebx) can be shifted
from common_interrupt: and BUILD_INTERRUPT() to ret_from_intr:
in entry.s (sorry again, can't make diff).

Oleg.

Re: [patch] big IRQ lock removal, 2.5.27-D9

[Oleg Nesterov]

Hello.

George Anzinger wrote:
>
> > then preempt_stop (cli) can be killed in entry.S:ret_from_intr()
>
> Also at least some of the trap code returns with interrupts enabled.

Then linux already have a bug - ret_from_exception: bypasses
ret_from_intr:. But as far as i can see, it does not - all users of
ret_from_exception: do preempt_stop. And so it can be shifted to
ret_from_exception.

I beleive none of the traps need current_thread_info() in regs->ebx,
so GET_THREAD_INFO(%ebx) can be killed in error_code:,
common_interrupt:,
and BUILD_INTERRUPT().

Not sure it is right time to such minor cleanups, but...
Patch on top of remove-irqlock-2.5.27-E0:

--- arch/i386/kernel/entry.S.orig	Mon Jul 22 23:44:41 2002
+++ arch/i386/kernel/entry.S	Tue Jul 23 00:04:02 2002
@@ -185,8 +185,10 @@
 
 	# userspace resumption stub bypassing syscall exit tracing
 	ALIGN
-ret_from_intr:
 ret_from_exception:
+	preempt_stop
+ret_from_intr:
+	GET_THREAD_INFO(%ebx)
 	movl EFLAGS(%esp), %eax		# mix EFLAGS and CS
 	movb CS(%esp), %al
 	testl $(VM_MASK | 3), %eax
@@ -333,14 +335,12 @@
 common_interrupt:
 	SAVE_ALL
 	call do_IRQ
-	GET_THREAD_INFO(%ebx)
 	jmp ret_from_intr
 
 #define BUILD_INTERRUPT(name, nr)	\
 ENTRY(name)				\
 	pushl $nr-256;			\
 	SAVE_ALL			\
-	GET_THREAD_INFO(%ebx);		\
 	call smp_/**/name;	\
 	jmp ret_from_intr;
 
@@ -400,10 +400,8 @@
 	movl $(__KERNEL_DS), %edx
 	movl %edx, %ds
 	movl %edx, %es
-	GET_THREAD_INFO(%ebx)
 	call *%edi
 	addl $8, %esp
-	preempt_stop
 	jmp ret_from_exception
 
 ENTRY(coprocessor_error)
@@ -430,7 +428,6 @@
 	pushl $0			# temporary storage for ORIG_EIP
 	call math_emulate
 	addl $4, %esp
-	preempt_stop
 	jmp ret_from_exception
 
 ENTRY(debug)

Oleg.

Re: [patch] big IRQ lock removal, 2.5.27-D9

[Ingo Molnar]

On Tue, 23 Jul 2002, Oleg Nesterov wrote:

> Not sure it is right time to such minor cleanups, but...

i've applied your patch to my tree - all of the irqlock patches are
cleanups to begin with.

	Ingo

Re: [announce, patch, RFC] "big IRQ lock" removal, IRQ cleanups.

[george anzinger]

Oleg Nesterov wrote:
> 
> Hello.
> 
> > > [...] To do this from irq.c means that it must exit with interrupts off
> > > and the the low level code needs to keep them off till the irtn. [...]
> 
> > yes, we are very careful to keep irqs disabled in do_IRQ(), both before
> > and after calling the handler.
> 
> Note that smp_xxx_interrupt() functions must be carefull
> with preemt_{disable,enable} brackets.
> 
> For example, smp_invalidate_interrupt() may be preempted
> after put_cpu(). Probably not big deal (it is return path),
> but it is better to use preempt_enable_no_resched() here -
> let ret_from_intr: do its job.
> 
> smp_{error,spurious,thermal}_interrupt() - all of them
> use printk() without bumping preemt_count and have problem
> after spin_unlock_irqrestore(&logbuf_lock, flags).
> 
> If these problems worth fixing, then preempt_stop (cli)
> can be killed in entry.S:ret_from_intr(), yes? 

I REALLY have a problem with this abstraction for cli.  I
think it just makes the code hard to read...
> If i understand
> correctly none of the irq handlers should return to low level
> code with irq enabled.

But schedule and signal code does return with interrupts
enabled, so a cli is still needed here.  Also at least some
of the trap code returns with interrupts enabled.
> 
> P.S.
> 
> May I suggest somebody with good english fix
> Documentation/preempt-locking.txt?
> It states, that disabled interrupts prevents preemption.
> Yes, but only in a sense, that the delivery of reschedule
> interrupt is suppressed.
> 
> Process with irqs disabled and current->preempt_count == 0 can
> be preempted (with interrupts enabled) after spin_lock/unlock etc.
> Even in UP case preemption can happen while calling wake_up_...().

This is really a bug and a fix is on the way.  Turning
interrupts off MUST disable preemption, but trying to
preempt from this state is so rare that the test will be in
preempt_schedule() rather than inline or an attempt to put
disable/enable code along with each cli/sti.

-- 
George Anzinger   george@mvista.com
High-res-timers: 
http://sourceforge.net/projects/high-res-timers/
Real time sched:  http://sourceforge.net/projects/rtsched/
Preemption patch:
http://www.kernel.org/pub/linux/kernel/people/rml

Re: [announce, patch, RFC] "big IRQ lock" removal, IRQ cleanups.

[Ingo Molnar]

On Mon, 22 Jul 2002, george anzinger wrote:

> But schedule and signal code does return with interrupts enabled, so a
> cli is still needed here.  Also at least some of the trap code returns
> with interrupts enabled.

the change only affects the ret_from_intr path, which is IRQ-only. The
signal and schedule path is still disabling interrupts.

	Ingo