[patch] 2.6.5 BUG if smp_call_function is called with interrupts disabled

[patch] 2.6.5 BUG if smp_call_function is called with interrupts disabled

[Keith Owens]

There is a nasty race if smp_call_function() is called with interrupts
disabled.

CPU A                               CPU B
Disable interrupts
                                    smp_call_function()
                                    Take call_lock
                                    Send IPIs
                                    Wait for all cpus to acknowledge IPI
                                    CPU A has not responded, spin waiting
                                    for cpu A to respond, holding call_lock
smp_call_function()
Spin waiting for call_lock
Deadlock                            Deadlock

This bug is hard to reproduce and even harder to diagnose.  Since the
comments at the start of smp_call_function() say it should never be
entered with interrupts disabled, make it so.

Index: linux/arch/ia64/kernel/smp.c
=================================--- linux.orig/arch/ia64/kernel/smp.c	Mon Apr 26 15:20:17 2004
+++ linux/arch/ia64/kernel/smp.c	Tue Apr 27 09:56:55 2004
@@ -319,10 +319,14 @@
 {
 	struct call_data_struct data;
 	int cpus = num_online_cpus()-1;
+	unsigned long psr;
 
 	if (!cpus)
 		return 0;
 
+	__asm__ __volatile__ ("mov %0=psr;; " : "=r" (psr));
+	BUG_ON(!(psr & IA64_PSR_I));
+
 	data.func = func;
 	data.info = info;
 	atomic_set(&data.started, 0);

Re: [patch] 2.6.5 BUG if smp_call_function is called with interrupts disabled

[David Mosberger]

>>>>> On Tue, 27 Apr 2004 10:09:39 +1000, Keith Owens <kaos@sgi.com> said:

  Keith> + __asm__ __volatile__ ("mov %0=psr;; " : "=r" (psr))

Please no inline-asm.  Especially when there is a perfectly suitable
macro already (local_save_flags).

	--david

Re: [patch] 2.6.5 BUG if smp_call_function is called with interrupts disabled

[Keith Owens]

On Mon, 26 Apr 2004 17:31:32 -0700, 
David Mosberger <davidm@napali.hpl.hp.com> wrote:
>>>>>> On Tue, 27 Apr 2004 10:09:39 +1000, Keith Owens <kaos@sgi.com> said:
>
>  Keith> + __asm__ __volatile__ ("mov %0=psr;; " : "=r" (psr))
>
>Please no inline-asm.  Especially when there is a perfectly suitable
>macro already (local_save_flags).

Sorry, ported from an older 2.4 kernel.  Redo.

There is a nasty race if smp_call_function() is called with interrupts
disabled.

CPU A                               CPU B
Disable interrupts
                                    smp_call_function()
                                    Take call_lock
                                    Send IPIs
                                    Wait for all cpus to acknowledge IPI
                                    CPU A has not responded, spin waiting
                                    for cpu A to respond, holding call_lock
smp_call_function()
Spin waiting for call_lock
Deadlock                            Deadlock

This bug is hard to reproduce and even harder to diagnose.  Since the
comments at the start of smp_call_function() say it should never be
entered with interrupts disabled, make it so.

Index: linux/arch/ia64/kernel/smp.c
=================================--- linux.orig/arch/ia64/kernel/smp.c	Mon Apr 26 15:20:17 2004
+++ linux/arch/ia64/kernel/smp.c	Tue Apr 27 10:36:58 2004
@@ -319,10 +319,14 @@
 {
 	struct call_data_struct data;
 	int cpus = num_online_cpus()-1;
+	unsigned long psr;
 
 	if (!cpus)
 		return 0;
 
+	local_save_flags(psr);
+	BUG_ON(!(psr & IA64_PSR_I));
+
 	data.func = func;
 	data.info = info;
 	atomic_set(&data.started, 0);

Re: [patch] 2.6.5 BUG if smp_call_function is called with interrupts disabled

[David Mosberger]

>>>>> On Tue, 27 Apr 2004 10:43:39 +1000, Keith Owens <kaos@sgi.com> said:

  Keith> Sorry, ported from an older 2.4 kernel.  Redo.

Sorry for doing this in an incremental fashion, but upon re-reading
your patch, I vaguely remembered that we had a macro to check whether
irqs are disabled.  Sure enough, there is:

	irqs_disabled()

  --david

Re: [patch] 2.6.5 BUG if smp_call_function is called with interrupts disabled

[Keith Owens]

Take 3.

There is a nasty race if smp_call_function() is called with interrupts
disabled.

CPU A                               CPU B
Disable interrupts
                                    smp_call_function()
                                    Take call_lock
                                    Send IPIs
                                    Wait for all cpus to acknowledge IPI
                                    CPU A has not responded, spin waiting
                                    for cpu A to respond, holding call_lock
smp_call_function()
Spin waiting for call_lock
Deadlock                            Deadlock

This bug is hard to reproduce and even harder to diagnose.  Since the
comments at the start of smp_call_function() say it should never be
entered with interrupts disabled, make it so.

Index: linux/arch/ia64/kernel/smp.c
=================================--- linux.orig/arch/ia64/kernel/smp.c	Mon Apr 26 15:20:17 2004
+++ linux/arch/ia64/kernel/smp.c	Tue Apr 27 11:30:48 2004
@@ -323,6 +323,8 @@
 	if (!cpus)
 		return 0;
 
+	BUG_ON(irqs_disabled());
+
 	data.func = func;
 	data.info = info;
 	atomic_set(&data.started, 0);