From: Calvin Owens <calvin@wbinvd.org>
To: linux-kernel@vger.kernel.org
Cc: linux-perf-users@vger.kernel.org, x86@kernel.org,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
James Clark <james.clark@linaro.org>,
Thomas Gleixner <tglx@kernel.org>, Borislav Petkov <bp@alien8.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
"H. Peter Anvin" <hpa@zytor.com>
Subject: [PATCH 0/2] Two semi-related perf throttling fixes
Date: Tue, 31 Mar 2026 08:25:48 -0700 [thread overview]
Message-ID: <cover.1774969692.git.calvin@wbinvd.org> (raw)
Hi all,
In the course of investigating [1], I set out to understand why this
sequence of messages is printed every boot, even when nobody is using
perf at all:
perf: interrupt took too long (2516 > 2500), lowering kernel.perf_event_max_sample_rate to 79000
perf: interrupt took too long (3156 > 3145), lowering kernel.perf_event_max_sample_rate to 63000
perf: interrupt took too long (4014 > 3945), lowering kernel.perf_event_max_sample_rate to 49000
perf: interrupt took too long (5035 > 5017), lowering kernel.perf_event_max_sample_rate to 39000
perf: interrupt took too long (6302 > 6293), lowering kernel.perf_event_max_sample_rate to 31000
perf: interrupt took too long (7879 > 7877), lowering kernel.perf_event_max_sample_rate to 25000
perf: interrupt took too long (9852 > 9848), lowering kernel.perf_event_max_sample_rate to 20000
It turns out this happens because of how the dynamic sample rate
throttling interacts with the perf hardware watchdog. Patch [2/2] is my
attempt to prevent the dynamic throttling logic from acting solely based
on the latency of the watchdog NMI.
Intel CPUs were happy with that. But AMD CPUs still printed the messages!
That happens because AMD CPUs have a second PMU facility with its own
NMI handler, and both NMI handlers average in their latency, even when
they don't actually handle the NMI.
Patch [1/2] fixes that, which is a correctness issue entirely
independent of patch [2/2]. But it also happens to be required for patch
[2/2] to achieve its goal on AMD CPUs, so I sent them together.
Thanks,
Calvin
[1] https://lore.kernel.org/all/acMe-QZUel-bBYUh@mozart.vkv.me/
Calvin Owens (2):
perf/x86: Avoid double accounting of PMU NMI latencies
perf: Don't throttle based on NMI watchdog events
arch/x86/events/amd/ibs.c | 6 +++---
arch/x86/events/core.c | 3 ++-
kernel/events/core.c | 14 ++++++++++++++
3 files changed, 19 insertions(+), 4 deletions(-)
--
2.47.3
next reply other threads:[~2026-03-31 15:26 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-03-31 15:25 Calvin Owens [this message]
2026-03-31 15:25 ` [PATCH 1/2] perf/x86: Avoid double accounting of PMU NMI latencies Calvin Owens
2026-03-31 15:25 ` [PATCH 2/2] perf: Don't throttle based on NMI watchdog events Calvin Owens
2026-03-31 17:22 ` Calvin Owens
2026-03-31 17:43 ` Calvin Owens
2026-03-31 18:10 ` Calvin Owens
2026-03-31 21:07 ` Calvin Owens
2026-04-01 8:01 ` [PATCH 0/2] Two semi-related perf throttling fixes Andi Kleen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1774969692.git.calvin@wbinvd.org \
--to=calvin@wbinvd.org \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=hpa@zytor.com \
--cc=irogers@google.com \
--cc=james.clark@linaro.org \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox