linuxppc-dev.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
From: Anton Blanchard <anton@samba.org>
To: benh@kernel.crashing.org, paulus@samba.org, mpe@ellerman.id.au,
	paulmck@linux.vnet.ibm.com
Cc: linuxppc-dev@lists.ozlabs.org
Subject: [PATCH 2/2] powerpc: Add ppc64 hard lockup detector support
Date: Tue, 5 Aug 2014 14:56:21 +1000	[thread overview]
Message-ID: <20140805145621.2fa2a372@kryten> (raw)
In-Reply-To: <20140805145500.773004e9@kryten>

The hard lockup detector uses a PMU event as a periodic NMI to
detect if we are stuck (where stuck means no timer interrupts have
occurred).

Ben's rework of the ppc64 soft disable code has made ppc64 PMU
exceptions a partial NMI. They can get disabled if an external interrupt
comes in, but otherwise PMU interrupts will fire in interrupt disabled
regions.

I wrote a kernel module to test this patch and noticed we sometimes
missed hard lockup warnings. The RCU code detected the stall first and
issued an IPI to backtrace all CPUs. Unfortunately an IPI is an external
interrupt and that will hard disable interrupts, preventing the hard
lockup detector from going off.

If I reduced the hard lockup threshold to 5 seconds:

echo 5 > /proc/sys/kernel/watchdog_thresh

Then it would beat the RCU code in detecting a stall and get a
correct backtrace out.

Another downside is that our PMCs can only count to 2^31, so even when
we ask for 10 seconds of processor cycles, we end up taking a couple
of PMU exceptions a second.

Signed-off-by: Anton Blanchard <anton@samba.org>
---

Index: b/arch/powerpc/Kconfig
===================================================================
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -145,6 +145,7 @@ config PPC
 	select HAVE_IRQ_EXIT_ON_IRQ_STACK
 	select ARCH_USE_CMPXCHG_LOCKREF if PPC64
 	select HAVE_ARCH_AUDITSYSCALL
+	select HAVE_PERF_EVENTS_NMI if PPC64
 
 config GENERIC_CSUM
 	def_bool CPU_LITTLE_ENDIAN
Index: b/arch/powerpc/include/asm/nmi.h
===================================================================
--- /dev/null
+++ b/arch/powerpc/include/asm/nmi.h
@@ -0,0 +1,4 @@
+#ifndef _ASM_NMI_H
+#define _ASM_NMI_H
+
+#endif /* _ASM_NMI_H */
Index: b/arch/powerpc/kernel/setup_64.c
===================================================================
--- a/arch/powerpc/kernel/setup_64.c
+++ b/arch/powerpc/kernel/setup_64.c
@@ -796,3 +796,10 @@ unsigned long memory_block_size_bytes(vo
 struct ppc_pci_io ppc_pci_io;
 EXPORT_SYMBOL(ppc_pci_io);
 #endif
+
+#ifdef CONFIG_HARDLOCKUP_DETECTOR
+u64 hw_nmi_get_sample_period(int watchdog_thresh)
+{
+	return ppc_proc_freq * watchdog_thresh;
+}
+#endif

  reply	other threads:[~2014-08-05  4:56 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-08-05  4:55 [PATCH 1/2] powerpc: Hard disable interrupts in xmon Anton Blanchard
2014-08-05  4:56 ` Anton Blanchard [this message]
2014-08-11 23:31   ` [PATCH 2/2] powerpc: Add ppc64 hard lockup detector support Anton Blanchard
2014-08-11 23:42     ` Paul E. McKenney
  -- strict thread matches above, loose matches on Subject: below --
2015-01-21  3:46 [PATCH 1/2] oprofile: Add HAVE_OPROFILE_NMI_TIMER Anton Blanchard
2015-01-21  3:46 ` [PATCH 2/2] powerpc: Add ppc64 hard lockup detector support Anton Blanchard
2015-04-09  2:52 [PATCH 1/2] oprofile: Disable oprofile NMI timer on ppc64 Anton Blanchard
2015-04-09  2:52 ` [PATCH 2/2] powerpc: Add ppc64 hard lockup detector support Anton Blanchard
2015-04-09 23:48   ` Scott Wood

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20140805145621.2fa2a372@kryten \
    --to=anton@samba.org \
    --cc=benh@kernel.crashing.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=mpe@ellerman.id.au \
    --cc=paulmck@linux.vnet.ibm.com \
    --cc=paulus@samba.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).