public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Dave Hansen <dave@sr71.net>
To: a.p.zijlstra@chello.nl
Cc: mingo@redhat.com, paulus@samba.org, acme@ghostprotocols.net,
	tglx@linutronix.de, x86@kernel.org, linux-kernel@vger.kernel.org,
	Dave Hansen <dave@sr71.net>
Subject: [v3][PATCH 0/4] Work around perf NMI-induced hangs
Date: Wed, 29 May 2013 15:27:56 -0700	[thread overview]
Message-ID: <20130529222756.25535229@viggo.jf.intel.com> (raw)

Changes from v2:

2/4:
 * Only warn on the longest NMIs.  Don't print when over
   a threshhold.
 * Output in ms as opposed to ns
4/4:
 * Add some Documentation/ for the tracepoint
 * keep tracepoint delta in a s64 instead of an int, and
   vall it 'delta_ns' instead of 'len'

Changes from v1:

 * keep a running average instead of taking a single value
   for determining NMI lengths.
 * Fixed some of the math converting from ns to/from
   percentages (it was backwards)
 * Included nmi length tracepoint at end of series 

--

If root or an unprivileged user runs 'perf top', my system hangs.
If I'm lucky, I get a warning out to dmesg, along these lines:

	hrtimer: interrupt took 13915457 ns cpu: 132

or a hard-lockup message on occasion.

The proxmiate cause of this is that perf_event_nmi_handler() has
been observed to take tens of ms on occasion.  That needs to get
fixed, and I'm working on tracking the root cause down.

But, These patches make the situation better: perf can no longer
simply wedge the box, and we have a safe, controlled exit path
when things go wrong.

             reply	other threads:[~2013-05-29 22:28 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-05-29 22:27 Dave Hansen [this message]
2013-05-29 22:27 ` [v3][PATCH 1/4] perf/x86: only print PMU state when also WARN()'ing Dave Hansen
2013-05-29 22:27 ` [v3][PATCH 2/4] x86: warn when NMI handlers take large amounts of time Dave Hansen
2013-05-30  8:33   ` Ingo Molnar
2013-05-30  9:37   ` Peter Zijlstra
2013-05-29 22:28 ` [v3][PATCH 3/4] perf: drop sample rate when sampling is too slow Dave Hansen
2013-05-29 22:28 ` [v3][PATCH 4/4] x86: nmi length tracepoints Dave Hansen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130529222756.25535229@viggo.jf.intel.com \
    --to=dave@sr71.net \
    --cc=a.p.zijlstra@chello.nl \
    --cc=acme@ghostprotocols.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=paulus@samba.org \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox