Linux Perf Users
 help / color / mirror / Atom feed
* Subject: [BUG] perf/core: frequency-mode period overshoots during ramp-up
@ 2026-11-12 19:17 Gregg Leventhal
  2026-11-17 20:23 ` [PATCH] perf/core: Use the elapsed period for frequency-mode period adjustment Gregg Leventhal
  0 siblings, 1 reply; 3+ messages in thread
From: Gregg Leventhal @ 2026-11-12 19:17 UTC (permalink / raw)
  To: linux-perf-users
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Mark Barnett,
	linux-kernel, Jay Hilton, Andrew Delgadillo, Eric Hagberg

Hi,

In frequency mode (attr.freq = 1, e.g. perf record -F 11), the sample
period overshoots its target while it ramps up from its initial value.
The sampled thread then goes unsampled for roughly 2-6x longer than
1/freq, and when the kernel later cuts the period back, events go
missing from the profile. We see this on vanilla v7.3, 6.18.56 and
6.12.112. The code involved is unchanged in v7.4-rc2.

Tested on (all vanilla kernel.org kernels):

- AMD EPYC 9655P (Zen5), bare metal, 7.3.0: cycles 50/50,
  instructions 50/50, raw cycles 50/50 runs (>2x in 17, 25 and 16 of
  those), up to 5.9x. IBS (ibs_op, cycles:P): >2x in 28/30, up to 7.0x.
- Intel Xeon Platinum 8468, bare metal, 7.3.0: raw cycles (0x3c)
  49/50, up to 6.8x; cycles 0/50.
- AMD EPYC Turin (Zen5), KVM guest, 7.3.0: 3/3 runs.
- AMD EPYC Genoa (Zen4), KVM guest, 6.18.56: 3/3 runs.
- AMD EPYC Genoa (Zen4), KVM guest, 6.12.112: 3/3 runs.

"N/50" counts runs in which the ramp-up produced either a period over
2x the target or a sample whose period doesn't match the cycles
counted. The 9655P uses PerfMonV2 (amd_pmu_v2_handle_irq()); the
guests don't expose it and use x86_pmu_handle_irq().

IBS (perf_ibs_handle_irq()) is affected the same way. It starts at its
minimum period of 144 and, for example, goes 144 -> 318,671 ->
2,021,072,009 (4.9x the target, a 450 ms gap). IBS runs included
kernel mode, since IBS rejects exclude_kernel without its software
filter.

Reproducer
----------

The program at the end samples its own user-mode cycles at 11 Hz while
spinning for 3 seconds. For each sample it prints the reported period
(PERF_SAMPLE_PERIOD), the cycles counted since the previous sample
(PERF_SAMPLE_READ), and the time since the previous sample. On v7.3,
bare-metal EPYC 9655P (target period ~411M cycles, ~91 ms):

        period      counted    dt(ms)
             1         1790     0.001
          2426         1863     0.001
      21864282     21864341     4.852
    2313570064   2458402940   543.880  <- != counted  <- > 2x target
     411062043    411061989    90.958
     410683637    410683588    90.945
  target period 410934292, max period 2313570064 (5.6x): BUG

After a 21.9M-cycle interval, the period jumps to 2.3G (5.6x the target)
and the next sample comes 544 ms later instead of 91 ms. 2.46G cycles
elapsed, so 145M are not attributed to any sample (see below for why).

On the Turin guest (v7.3, target ~403M), a run with a smaller overshoot:

        period      counted    dt(ms)
             1         1706     0.000
             1         2133     0.015
             1         1179     0.009
          1210         1398     0.011
       1019729      1020007     0.241
     998467121    998466679   225.390  <- > 2x target
     402677477    402677426    90.819
     402960793    402960844    90.921
  target period 403056550, max period 998467121 (2.5x): BUG

After a 1.02M-cycle interval, the counter is armed for 998M cycles,
2.5x the target, and period == counted: nothing was cut back, and the
next sample comes 225 ms later instead of 91 ms. When the overshoot is
larger, a later tick cuts it back, which shows up as a sample whose
period is smaller than the cycles that elapsed:

       2036681      2037233     0.458
     464053892    711277272   159.223  <- != counted
     406964862    406964897    90.959

Here 247M cycles (~55 ms of CPU) are not attributed to any sample.
6.18.56 behaves the same way (955M armed against a 329M target, or
379.6M reported for 573.4M counted).

Cause
-----

__perf_event_account_interrupt() adjusts the period with

        if (delta > 0 && delta < 2*TICK_NSEC)
                perf_adjust_period(event, delta, hwc->last_period,
                                   true);

but the PMU drivers re-arm the counter before calling
perf_event_overflow() (x86_pmu_handle_irq(), amd_pmu_v2_handle_irq(),
handle_pmi_common(), perf_ibs_handle_irq(), ...), and re-arming sets
hwc->last_period to the period of the *next* interval. The estimator
therefore divides the next period by the time the previous one took.
Once the period is stable the two are about equal, but during ramp-up
each period is much larger than the one before it, so the rate is
overestimated by that ratio.

The Turin guest run above matches this. At the overflow that ended
the 1,210-cycle interval (about 11 us), the counter was re-armed for
1,019,729 cycles, so perf_adjust_period() got count = 1,019,729 and
nsec ~ 11 us:

  period        = 1019729 * 1e9 / (11e3 * 11)       ~ 8.4e9
  sample_period = 1019729 + (8.4e9 - 1019729) / 8   ~ 1.05e9

which is the ~998M period armed one overflow later. With the elapsed
count (1,210) the same calculation gives a period of ~10M and a next
sample_period of ~2.1M. The low-pass filter can only move sample_period
1/8 of the way toward the estimate, so a correctly fed estimator cannot
overshoot the target. Reaching 2.5x the target needs an estimated rate
around 20x what the CPU can do.

When the overshoot exceeds 8x, perf_adjust_period() on the next tick
sees period_left > 8 * sample_period, zeroes period_left and restarts
the counter. Restarting sets hwc->last_period to the new, smaller
period, which is why the reported period then understates the cycles
that elapsed (the 464M/711M sample above).

Since 1734d98fbcce ("perf/arch: Record sample last_period before
updating on the x86 and PowerPC platforms"), the elapsed period is in
data->period when perf_event_overflow() is called, so that looks like
the right input for the estimator, but we haven't tested that change
on mainline.

Kernels without 1734d98fbcce (e.g. 6.12) also report each sample's
period one sample late. Combined with this bug, the oversized period is
attached to a sample that covered only ~1M cycles. For example, on
6.12.112:

       1037581         1464     0.012  <- != counted
    1002666013      1037874     0.293  <- != counted  <- > 2x target

That sample claims ~1000x the cycles it represents.

Impact
------

This matters most at low frequencies and for threads that run in short
bursts, since they spend a large share of their samples in ramp-up.
It isn't limited to low frequencies, though: on the bare-metal 9655P,
perf's default 4000 Hz overshot in 10/10 runs, by up to 4x (in the KVM
guests it was ~1.03x).

We found it because continuous per-thread profiles at 11 Hz (on
6.12-based kernels) attributed ~12% of a CPU-bound program's time to a
function that only runs at startup, and heavily overweighted a mostly
idle runtime ticker thread. On kernels with 1734d98fbcce, the symptoms
are sampling gaps and events missing from the profile, rather than
inflated weights.

On Intel (Xeon 8468, bare metal, 7.3), the default cycles and
instructions events are not affected: intel_pmu_freq_start_period()
starts them at 500M/freq (45,454,545 at 11 Hz), close to the target, so
there is no ramp-up to overshoot. A raw cycles event (0x3c) starts much
lower (23,272) and overshoots in 49 of 50 runs, by up to 6.8x. In a
typical run (target ~344M), the period jumps to 1.94G (5.7x) and 61M
cycles are not attributed to any sample:

         23272        23255     0.007
      25310078     25310067     6.737
    1943552252   2005048959   530.205  <- != counted  <- > 2x target
     343751272    343751292    90.879

So the problem is in the generic code; Intel's start-period estimate
only hides it for some events. We haven't looked at or tested other
architectures.

Thanks to Jay Hilton <jahilton@janestreet.com> and Andrew Delgadillo
<adelgadillo@janestreet.com>, who found and analyzed this.

Reproducer:

// Minimal reproducer for the perf frequency-mode ramp-up bug.
// Samples this thread's user-mode cycles at 11 Hz while burning CPU
// for 3s, then prints each sample's period, the cycles actually
// counted since the previous sample, and the time since the previous
// sample.
//
// On a correct kernel, period == counted (up to skid) and no period
// exceeds 2x the steady-state target (cycles per second / samples
// per second). Exits 1 if either check fails.
//
//   cc -O2 -o repro-min repro-min.c && ./repro-min
#define _GNU_SOURCE
#include <linux/perf_event.h>
#include <stdint.h>
#include <stdio.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/syscall.h>
#include <time.h>
#include <unistd.h>

#define FREQ 11
#define PAGES 64
#define SKID 100000

static double now(void) {
  struct timespec ts;
  clock_gettime(CLOCK_MONOTONIC, &ts);
  return ts.tv_sec + ts.tv_nsec / 1e9;
}

int main(void) {
  struct perf_event_attr attr = {
      .size = sizeof(attr),
      .type = PERF_TYPE_HARDWARE,
      .config = PERF_COUNT_HW_CPU_CYCLES,
      .freq = 1,
      .sample_freq = FREQ,
      .sample_type =
          PERF_SAMPLE_TIME | PERF_SAMPLE_PERIOD | PERF_SAMPLE_READ,
      .exclude_kernel = 1,
      .disabled = 1,
  };
  int fd = syscall(SYS_perf_event_open, &attr, 0, -1, -1, 0);
  if (fd < 0) return perror("perf_event_open"), 2;
  long pg = sysconf(_SC_PAGESIZE);
  struct perf_event_mmap_page *meta =
      mmap(NULL, (PAGES + 1) * pg, PROT_READ | PROT_WRITE,
           MAP_SHARED, fd, 0);
  if (meta == MAP_FAILED) return perror("mmap"), 2;

  ioctl(fd, PERF_EVENT_IOC_ENABLE, 0);
  double start = now();
  while (now() < start + 3) {
  }
  ioctl(fd, PERF_EVENT_IOC_DISABLE, 0);
  double elapsed = now() - start;
  uint64_t total;
  if (read(fd, &total, sizeof total) != sizeof total)
    return perror("read"), 2;
  double target = total / elapsed / FREQ;

  // 3s at 11 Hz fits in the buffer, so records never wrap.
  char *data = (char *)meta + pg;
  uint64_t head = __atomic_load_n(&meta->data_head, __ATOMIC_ACQUIRE);
  uint64_t prev_count = 0, prev_time = 0, max = 0;
  int bad = 0;
  printf("%12s %12s %9s\n", "period", "counted", "dt(ms)");
  for (uint64_t pos = 0; pos < head;) {
    struct perf_event_header *h = (void *)(data + pos);
    if (h->type == PERF_RECORD_SAMPLE) {
      // v[0] = time, v[1] = period, v[2] = counter value
      uint64_t *v = (void *)(h + 1);
      uint64_t counted = v[2] - prev_count;
      long long err = (long long)(v[1] - counted);
      int mismatch = err > SKID || err < -SKID;
      int over = v[1] > 2 * target;
      double dt = prev_time ? (v[0] - prev_time) / 1e6 : 0;
      printf("%12llu %12llu %9.3f%s%s\n",
             (unsigned long long)v[1], (unsigned long long)counted,
             dt, mismatch ? "  <- != counted" : "",
             over ? "  <- > 2x target" : "");
      bad |= mismatch || over;
      if (v[1] > max) max = v[1];
      prev_count = v[2];
      prev_time = v[0];
    }
    pos += h->size;
  }
  printf("target period %.0f, max period %llu (%.1fx): %s\n", target,
         (unsigned long long)max, max / target, bad ? "BUG" : "OK");
  return bad;
}

Thanks,
Gregg Leventhal

^ permalink raw reply	[flat|nested] 3+ messages in thread

* [PATCH] perf/core: Use the elapsed period for frequency-mode period adjustment
  2026-11-12 19:17 Subject: [BUG] perf/core: frequency-mode period overshoots during ramp-up Gregg Leventhal
@ 2026-11-17 20:23 ` Gregg Leventhal
  2026-11-17 20:37   ` sashiko-bot
  0 siblings, 1 reply; 3+ messages in thread
From: Gregg Leventhal @ 2026-11-17 20:23 UTC (permalink / raw)
  To: linux-perf-users
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Namhyung Kim, Mark Rutland, Alexander Shishkin, Jiri Olsa,
	Ian Rogers, Adrian Hunter, James Clark, Mark Barnett,
	linux-kernel, Jay Hilton, Andrew Delgadillo, Eric Hagberg

From: Jay Hilton <jahilton@janestreet.com>

In frequency mode, __perf_event_account_interrupt() estimates the event
rate from hwc->last_period and the time since the previous overflow,
and passes it to perf_adjust_period().

The PMU drivers re-arm the counter before they call
perf_event_overflow(), and re-arming sets hwc->last_period to the
period of the next interval. The estimator therefore divides the next
period by the time the previous one took. While the period ramps up
from its initial value, each period is much larger than the previous
one, so the rate is overestimated by that ratio and the period
overshoots its target.

On an AMD EPYC 9655P running v7.3, sampling user-mode cycles at 11 Hz,
the period jumped from 21.9M to 2.3G cycles, 5.6x the target, and the
next sample arrived after 544 ms instead of 91 ms. When the overshoot
exceeds 8x, the next tick's perf_adjust_period() resets period_left,
and the cycles counted before the reset are not attributed to any
sample (145M in that run). Intel and AMD IBS show the same overshoot.
On Intel, intel_pmu_freq_start_period() hides it for the default
hardware events, but not for raw events.

Since commit 1734d98fbcce ("perf/arch: Record sample last_period before
updating on the x86 and PowerPC platforms"), the PMU drivers record the
elapsed period in data->period before re-arming. Pass that to the
estimator instead of hwc->last_period. perf_event_account_interrupt(),
which has no sample data, keeps using hwc->last_period.

Fixes: abd50713944c ("perf: Reimplement frequency driven sampling")
Closes: https://lore.kernel.org/all/CAFN_u7G94CC_RGL1La66jX+jRKHSJdAB5PKkTRSWrUvqhM48oQ@mail.gmail.com/
Assisted-by: LLM
Signed-off-by: Jay Hilton <jahilton@janestreet.com>
Signed-off-by: Gregg Leventhal <gleventhal@janestreet.com>
---
Notes (not part of the commit message):

This fixes the overshoot reported in the Closes: thread.

Tested on v7.3 with and without this patch, using the reproducer from
the report and repeated runs (user mode, 11 Hz unless noted). Runs
with the bug, i.e. a period over 2x the target or a sample whose period
doesn't match the cycles counted:

  AMD EPYC 9655P (Zen5), bare metal   v7.3         v7.3 + patch
    cycles                            50/50        0/50
    instructions                      50/50        0/50
    raw cycles                        50/50        0/50
    cycles, 4000 Hz                   10/10        0/10
    IBS (ibs_op, incl. kernel)        20/20        0/20
    max period / target               up to 7.0x   1.00x

  Intel Xeon Platinum 8468, bare metal v7.3         v7.3 + patch
    raw cycles (0x3c)                 49/50        0/50
    cycles                            0/50         0/50
    instructions                      0/50         0/50
    max period / target               up to 6.8x   1.02x

(On Intel, intel_pmu_freq_start_period() already keeps the default
events close to the target, so only the raw event shows the bug.)

With the patch, the period approaches the target from below, as the
low-pass filter intends (9655P, target ~411M cycles):

        period      counted    dt(ms)
          2371         1791     0.001
         11854        11880     0.004
      28341794     28341765     6.284
     251213029    251213027    55.579
     411087485    411087548    90.895

Base: v7.3. The code this touches is unchanged in v7.4-rc2. Builds
cleanly with W=1.

All perf_event_overflow() callers under arch/ and drivers/ in v7.3
initialize the sample data with the period recorded before re-arming,
so data->period is the elapsed period on every path we checked.

perf_event_account_interrupt() still passes hwc->last_period, and this
patch doesn't change its callers. Two of them run after the counter
has been re-armed, so they still feed the estimator the next period:
perf_ibs_handle_irq() when it discards a sample (user/kernel mode
filtering, the load-latency and fetch-latency filters, and the
zero-RIP erratum workaround), and powerpc's record_and_restart() for
an invalid SIAR. The Intel PEBS error path runs before re-arming and
is fine. We haven't measured the IBS discard case; if it's worth
fixing, those callers could pass the elapsed period too, in a
follow-up.

On kernels without 1734d98fbcce, x86_pmu_handle_irq() and
handle_pmi_common() still put the next period in data->period, so a
backport there also needs 1734d98fbcce.

The original change was written by an AI coding assistant in Jay's
session. This version, the changelog and these notes were prepared
with an AI coding assistant; Jay and I reviewed them, and I ran the
tests above.

 kernel/events/core.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index c4217c996..55b72a044 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -10592,8 +10592,15 @@ void perf_report_aux_output_id(struct perf_event *event, u64 hw_id)
 }
 EXPORT_SYMBOL_GPL(perf_report_aux_output_id);
 
+/*
+ * @period is the period of the interval that just ended. PMU drivers
+ * re-arm the counter before calling perf_event_overflow(), which sets
+ * hwc->last_period to the period of the next interval, so the overflow
+ * path passes the sample's period instead.
+ */
 static int
-__perf_event_account_interrupt(struct perf_event *event, int throttle)
+__perf_event_account_interrupt(struct perf_event *event, int throttle,
+			       u64 period)
 {
 	struct hw_perf_event *hwc = &event->hw;
 	int ret = 0;
@@ -10621,7 +10628,7 @@ __perf_event_account_interrupt(struct perf_event *event, int throttle)
 		hwc->freq_time_stamp = now;
 
 		if (delta > 0 && delta < 2*TICK_NSEC)
-			perf_adjust_period(event, delta, hwc->last_period, true);
+			perf_adjust_period(event, delta, period, true);
 	}
 
 	return ret;
@@ -10629,7 +10636,7 @@ __perf_event_account_interrupt(struct perf_event *event, int throttle)
 
 int perf_event_account_interrupt(struct perf_event *event)
 {
-	return __perf_event_account_interrupt(event, 1);
+	return __perf_event_account_interrupt(event, 1, event->hw.last_period);
 }
 
 static inline bool sample_is_allowed(struct perf_event *event, struct pt_regs *regs)
@@ -10757,7 +10764,7 @@ static int __perf_event_overflow(struct perf_event *event,
 	if (unlikely(!is_sampling_event(event)))
 		return 0;
 
-	ret = __perf_event_account_interrupt(event, throttle);
+	ret = __perf_event_account_interrupt(event, throttle, data->period);
 
 	if (event->attr.aux_pause)
 		perf_event_aux_pause(event->aux_event, true);
-- 
2.43.7


^ permalink raw reply related	[flat|nested] 3+ messages in thread

* Re: [PATCH] perf/core: Use the elapsed period for frequency-mode period adjustment
  2026-11-17 20:23 ` [PATCH] perf/core: Use the elapsed period for frequency-mode period adjustment Gregg Leventhal
@ 2026-11-17 20:37   ` sashiko-bot
  0 siblings, 0 replies; 3+ messages in thread
From: sashiko-bot @ 2026-11-17 20:37 UTC (permalink / raw)
  To: Gregg Leventhal, linux-perf-users

Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 1 related pre-existing issue it does not
introduce:
- [Critical] perf: data race in __perf_event_overflow() [1]
--

commit 3821f44b0f2d83639af791a4e85517a4c30f74ba
Author: Jay Hilton <jahilton@janestreet.com>

perf/core: Use the elapsed period for frequency-mode period adjustment

This commit passes the sample's elapsed period (data->period) instead of
hwc->last_period from __perf_event_overflow() to
__perf_event_account_interrupt() so frequency-mode period adjustment uses
the period of the interval that just completed rather than the newly
re-armed period.

> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index 614f9f0e65005..5ce85180db7c6 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c

[ ... ]

> @@ -10912,7 +10919,7 @@ static int __perf_event_overflow(struct perf_event *event,
>  	if (unlikely(!is_sampling_event(event)))
>  		return 0;
>  
> -	ret = __perf_event_account_interrupt(event, throttle);
> +	ret = __perf_event_account_interrupt(event, throttle, data->period);
>  
>  	if (event->attr.aux_pause)
>  		perf_event_aux_pause(event->aux_event, true);

[Severity: Critical]
This problem wasn't introduced by this patch, but can
__perf_event_overflow() hit a data race [1] when reading
event->aux_event for aux_pause or aux_resume?

When a PMI fires in NMI context, __perf_event_overflow() loads
event->aux_event with a plain read instead of smp_load_acquire(), racing
with concurrent updates to event->aux_event.  Passing a stale or
uninitialized pointer to perf_event_aux_pause() -> ring_buffer_get() in
NMI context can dereference invalid memory and panic the kernel.

[1] https://sashiko.dev/#/bug/linux-3357f385-bcd3-40bc-a0de-f4f4d883aff5

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261117152351.1294050-1-gleventhal@janestreet.com?part=1

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2026-11-17 20:37 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-12 19:17 Subject: [BUG] perf/core: frequency-mode period overshoots during ramp-up Gregg Leventhal
2026-11-17 20:23 ` [PATCH] perf/core: Use the elapsed period for frequency-mode period adjustment Gregg Leventhal
2026-11-17 20:37   ` sashiko-bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox