* [PATCH 2/3] x86: Avoid calling arch_trigger_all_cpu_backtrace() at the same time
2011-01-05 3:38 [PATCH 1/3] x86: Only call smp_processor_id in non-preempt cases Don Zickus
@ 2011-01-05 3:38 ` Don Zickus
2011-01-05 14:07 ` [tip:perf/core] " tip-bot for Dongdong Deng
2011-01-05 3:38 ` [PATCH 3/3] x86, NMI: Add touch_nmi_watchdog to io_check_error delay Don Zickus
2011-01-05 14:07 ` [tip:perf/core] x86: Only call smp_processor_id in non-preempt cases tip-bot for Don Zickus
2 siblings, 1 reply; 6+ messages in thread
From: Don Zickus @ 2011-01-05 3:38 UTC (permalink / raw)
To: x86
Cc: LKML, Dongdong Deng, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
fweisbec, Don Zickus
From: Dongdong Deng <dongdong.deng@windriver.com>
The spin_lock_debug/rcu_cpu_stall detector uses
trigger_all_cpu_backtrace() to dump cpu backtrace.
Therefore it is possible that trigger_all_cpu_backtrace()
could be called at the same time on different CPUs, which
triggers and 'unknown reason NMI' warning. The following case
illustrates the problem:
CPU1 CPU2 ... CPU N
trigger_all_cpu_backtrace()
set "backtrace_mask" to cpu mask
|
generate NMI interrupts generate NMI interrupts ...
\ | /
\ | /
The "backtrace_mask" will be cleaned by the first NMI interrupt
at nmi_watchdog_tick(), then the following NMI interrupts generated
by other cpus's arch_trigger_all_cpu_backtrace() will be taken as
unknown reason NMI interrupts.
This patch uses a test_and_set to avoid the problem, and stop the
arch_trigger_all_cpu_backtrace() from calling to avoid dumping a
double cpu backtrace info when there is already a
trigger_all_cpu_backtrace() in progress.
Signed-off-by: Dongdong Deng <dongdong.deng@windriver.com>
Reviewed-by: Bruce Ashfield <bruce.ashfield@windriver.com>
CC: Thomas Gleixner <tglx@linutronix.de>
CC: Ingo Molnar <mingo@redhat.com>
CC: "H. Peter Anvin" <hpa@zytor.com>
CC: x86@kernel.org
CC: linux-kernel@vger.kernel.org
CC: fweisbec@gmail.com
Signed-off-by: Don Zickus <dzickus@redhat.com>
---
arch/x86/kernel/apic/hw_nmi.c | 13 +++++++++++++
1 files changed, 13 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c
index 2b40a60..72ec29e 100644
--- a/arch/x86/kernel/apic/hw_nmi.c
+++ b/arch/x86/kernel/apic/hw_nmi.c
@@ -28,10 +28,20 @@ u64 hw_nmi_get_sample_period(void)
/* For reliability, we're prepared to waste bits here. */
static DECLARE_BITMAP(backtrace_mask, NR_CPUS) __read_mostly;
+/* "in progress" flag of arch_trigger_all_cpu_backtrace */
+static unsigned long backtrace_flag;
+
void arch_trigger_all_cpu_backtrace(void)
{
int i;
+ if (test_and_set_bit(0, &backtrace_flag))
+ /*
+ * If there is already a trigger_all_cpu_backtrace() in progress
+ * (backtrace_flag == 1), don't output double cpu dump infos.
+ */
+ return;
+
cpumask_copy(to_cpumask(backtrace_mask), cpu_online_mask);
printk(KERN_INFO "sending NMI to all CPUs:\n");
@@ -43,6 +53,9 @@ void arch_trigger_all_cpu_backtrace(void)
break;
mdelay(1);
}
+
+ clear_bit(0, &backtrace_flag);
+ smp_mb__after_clear_bit();
}
static int __kprobes
--
1.7.3.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* [tip:perf/core] x86: Avoid calling arch_trigger_all_cpu_backtrace() at the same time
2011-01-05 3:38 ` [PATCH 2/3] x86: Avoid calling arch_trigger_all_cpu_backtrace() at the same time Don Zickus
@ 2011-01-05 14:07 ` tip-bot for Dongdong Deng
0 siblings, 0 replies; 6+ messages in thread
From: tip-bot for Dongdong Deng @ 2011-01-05 14:07 UTC (permalink / raw)
To: linux-tip-commits
Cc: linux-kernel, hpa, mingo, bruce.ashfield, dongdong.deng, tglx,
mingo, dzickus
Commit-ID: 554ec063982752e9a569ab9189eeffa3d96731b2
Gitweb: http://git.kernel.org/tip/554ec063982752e9a569ab9189eeffa3d96731b2
Author: Dongdong Deng <dongdong.deng@windriver.com>
AuthorDate: Tue, 4 Jan 2011 22:38:08 -0500
Committer: Ingo Molnar <mingo@elte.hu>
CommitDate: Wed, 5 Jan 2011 14:22:57 +0100
x86: Avoid calling arch_trigger_all_cpu_backtrace() at the same time
The spin_lock_debug/rcu_cpu_stall detector uses
trigger_all_cpu_backtrace() to dump cpu backtrace.
Therefore it is possible that trigger_all_cpu_backtrace()
could be called at the same time on different CPUs, which
triggers and 'unknown reason NMI' warning. The following case
illustrates the problem:
CPU1 CPU2 ... CPU N
trigger_all_cpu_backtrace()
set "backtrace_mask" to cpu mask
|
generate NMI interrupts generate NMI interrupts ...
\ | /
\ | /
The "backtrace_mask" will be cleaned by the first NMI interrupt
at nmi_watchdog_tick(), then the following NMI interrupts
generated by other cpus's arch_trigger_all_cpu_backtrace() will
be taken as unknown reason NMI interrupts.
This patch uses a test_and_set to avoid the problem, and stop
the arch_trigger_all_cpu_backtrace() from calling to avoid
dumping a double cpu backtrace info when there is already a
trigger_all_cpu_backtrace() in progress.
Signed-off-by: Dongdong Deng <dongdong.deng@windriver.com>
Reviewed-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Cc: fweisbec@gmail.com
LKML-Reference: <1294198689-15447-2-git-send-email-dzickus@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Don Zickus <dzickus@redhat.com>
---
arch/x86/kernel/apic/hw_nmi.c | 13 +++++++++++++
1 files changed, 13 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c
index 2b40a60..72ec29e 100644
--- a/arch/x86/kernel/apic/hw_nmi.c
+++ b/arch/x86/kernel/apic/hw_nmi.c
@@ -28,10 +28,20 @@ u64 hw_nmi_get_sample_period(void)
/* For reliability, we're prepared to waste bits here. */
static DECLARE_BITMAP(backtrace_mask, NR_CPUS) __read_mostly;
+/* "in progress" flag of arch_trigger_all_cpu_backtrace */
+static unsigned long backtrace_flag;
+
void arch_trigger_all_cpu_backtrace(void)
{
int i;
+ if (test_and_set_bit(0, &backtrace_flag))
+ /*
+ * If there is already a trigger_all_cpu_backtrace() in progress
+ * (backtrace_flag == 1), don't output double cpu dump infos.
+ */
+ return;
+
cpumask_copy(to_cpumask(backtrace_mask), cpu_online_mask);
printk(KERN_INFO "sending NMI to all CPUs:\n");
@@ -43,6 +53,9 @@ void arch_trigger_all_cpu_backtrace(void)
break;
mdelay(1);
}
+
+ clear_bit(0, &backtrace_flag);
+ smp_mb__after_clear_bit();
}
static int __kprobes
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 3/3] x86, NMI: Add touch_nmi_watchdog to io_check_error delay
2011-01-05 3:38 [PATCH 1/3] x86: Only call smp_processor_id in non-preempt cases Don Zickus
2011-01-05 3:38 ` [PATCH 2/3] x86: Avoid calling arch_trigger_all_cpu_backtrace() at the same time Don Zickus
@ 2011-01-05 3:38 ` Don Zickus
2011-01-05 14:08 ` [tip:perf/core] " tip-bot for Huang Ying
2011-01-05 14:07 ` [tip:perf/core] x86: Only call smp_processor_id in non-preempt cases tip-bot for Don Zickus
2 siblings, 1 reply; 6+ messages in thread
From: Don Zickus @ 2011-01-05 3:38 UTC (permalink / raw)
To: x86; +Cc: LKML, Huang Ying, Don Zickus
From: Huang Ying <ying.huang@intel.com>
Prevent the long delay in io_check_error making NMI watchdog timeout.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Don Zickus <dzickus@redhat.com>
---
arch/x86/kernel/traps.c | 8 +++++---
1 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index bb6f041..c76aaca 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -351,9 +351,11 @@ io_check_error(unsigned char reason, struct pt_regs *regs)
reason = (reason & 0xf) | 8;
outb(reason, 0x61);
- i = 2000;
- while (--i)
- udelay(1000);
+ i = 20000;
+ while (--i) {
+ touch_nmi_watchdog();
+ udelay(100);
+ }
reason &= ~8;
outb(reason, 0x61);
--
1.7.3.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* [tip:perf/core] x86, NMI: Add touch_nmi_watchdog to io_check_error delay
2011-01-05 3:38 ` [PATCH 3/3] x86, NMI: Add touch_nmi_watchdog to io_check_error delay Don Zickus
@ 2011-01-05 14:08 ` tip-bot for Huang Ying
0 siblings, 0 replies; 6+ messages in thread
From: tip-bot for Huang Ying @ 2011-01-05 14:08 UTC (permalink / raw)
To: linux-tip-commits
Cc: linux-kernel, ying.huang, hpa, mingo, tglx, mingo, dzickus
Commit-ID: 74d91e3c6a66359bb754fb5d8a5b54fb6ba2f9a6
Gitweb: http://git.kernel.org/tip/74d91e3c6a66359bb754fb5d8a5b54fb6ba2f9a6
Author: Huang Ying <ying.huang@intel.com>
AuthorDate: Tue, 4 Jan 2011 22:38:09 -0500
Committer: Ingo Molnar <mingo@elte.hu>
CommitDate: Wed, 5 Jan 2011 14:22:58 +0100
x86, NMI: Add touch_nmi_watchdog to io_check_error delay
Prevent the long delay in io_check_error making NMI watchdog
timeout.
Signed-off-by: Huang Ying <ying.huang@intel.com>
Signed-off-by: Don Zickus <dzickus@redhat.com>
LKML-Reference: <1294198689-15447-3-git-send-email-dzickus@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
arch/x86/kernel/traps.c | 8 +++++---
1 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index bb6f041..c76aaca 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -351,9 +351,11 @@ io_check_error(unsigned char reason, struct pt_regs *regs)
reason = (reason & 0xf) | 8;
outb(reason, 0x61);
- i = 2000;
- while (--i)
- udelay(1000);
+ i = 20000;
+ while (--i) {
+ touch_nmi_watchdog();
+ udelay(100);
+ }
reason &= ~8;
outb(reason, 0x61);
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [tip:perf/core] x86: Only call smp_processor_id in non-preempt cases
2011-01-05 3:38 [PATCH 1/3] x86: Only call smp_processor_id in non-preempt cases Don Zickus
2011-01-05 3:38 ` [PATCH 2/3] x86: Avoid calling arch_trigger_all_cpu_backtrace() at the same time Don Zickus
2011-01-05 3:38 ` [PATCH 3/3] x86, NMI: Add touch_nmi_watchdog to io_check_error delay Don Zickus
@ 2011-01-05 14:07 ` tip-bot for Don Zickus
2 siblings, 0 replies; 6+ messages in thread
From: tip-bot for Don Zickus @ 2011-01-05 14:07 UTC (permalink / raw)
To: linux-tip-commits
Cc: linux-kernel, hpa, mingo, jan.kiszka, tglx, mingo, dzickus
Commit-ID: 9ab181fa9ff73a38fccd0a4f1c40a38dfe62b535
Gitweb: http://git.kernel.org/tip/9ab181fa9ff73a38fccd0a4f1c40a38dfe62b535
Author: Don Zickus <dzickus@redhat.com>
AuthorDate: Tue, 4 Jan 2011 22:38:07 -0500
Committer: Ingo Molnar <mingo@elte.hu>
CommitDate: Wed, 5 Jan 2011 14:22:57 +0100
x86: Only call smp_processor_id in non-preempt cases
There are some paths that walk the die_chain with preemption on.
Make sure we are in an NMI call before we start doing anything.
This was triggered by do_general_protection calling notify_die
with DIE_GPF.
Reported-by: Jan Kiszka <jan.kiszka@web.de>
Signed-off-by: Don Zickus <dzickus@redhat.com>
LKML-Reference: <1294198689-15447-1-git-send-email-dzickus@redhat.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
arch/x86/kernel/apic/hw_nmi.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c
index c57d0b5..2b40a60 100644
--- a/arch/x86/kernel/apic/hw_nmi.c
+++ b/arch/x86/kernel/apic/hw_nmi.c
@@ -51,7 +51,7 @@ arch_trigger_all_cpu_backtrace_handler(struct notifier_block *self,
{
struct die_args *args = __args;
struct pt_regs *regs;
- int cpu = smp_processor_id();
+ int cpu;
switch (cmd) {
case DIE_NMI:
@@ -63,6 +63,7 @@ arch_trigger_all_cpu_backtrace_handler(struct notifier_block *self,
}
regs = args->regs;
+ cpu = smp_processor_id();
if (cpumask_test_cpu(cpu, to_cpumask(backtrace_mask))) {
static arch_spinlock_t lock = __ARCH_SPIN_LOCK_UNLOCKED;
^ permalink raw reply related [flat|nested] 6+ messages in thread