[PATCH] perf/core: Use AUX pause/resume for throttling

[PATCH] perf/core: Use AUX pause/resume for throttling

[Leo Yan]

Since commit 9734e25fbf5a ("perf: Fix the throttle logic for a group"),
perf throttles events in a group with pmu->stop(event, 0) and
unthrottles them with pmu->start(event, 0).

That is suitable for ordinary counter PMUs, where the zero flags mean
stop/start without updating the count. For AUX trace PMUs, however,
these two callbacks also manage AUX output state. A throttle/unthrottle
cycle can therefore stop tracing without ending AUX output, then try to
begin AUX output again when unthrottling.

On Arm CoreSight this can trigger:

  perf record -a -e cs_etm/aux-action=start-paused/k \
	-e cycles/aux-action=pause,period=3000/ \
	-e cycles/aux-action=resume,period=1500/ -- sleep 10

  [  181.161897] ------------[ cut here ]------------
  [  181.161906] WARNING: kernel/events/ring_buffer.c:415 at perf_aux_output_begin+0x1dc/0x1f0, CPU#0:2
  [  181.234120] arm-scmi arm-scmi.1.auto: timed out in resp(caller: do_xfer+0x1a0/0x558)
  [  181.234223] cpufreq: __target_index: Failed to change cpu frequency: -110
  [  181.286675] Modules linked in: panfrost tda9950 hdlcd tda998x drm_shmem_helper drm_client_lib cect
  [  181.314139] CPU: 0 UID: 0 PID: 352 Comm: perf Not tainted 7.1.0-rc1-00058-gdca922e019dd-dirty #11
  [  181.323915] Hardware name: ARM LTD ARM Juno Development Platform/ARM Juno Development Platform, B5
  [  181.334732] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
  [  181.341721] pc : perf_aux_output_begin+0x1dc/0x1f0
  [  181.346547] lr : perf_aux_output_begin+0x9c/0x1f0
  [  181.351280] sp : ffff800080003be0
  [  181.354606] x29: ffff800080003be0 x28: ffff0008023fc000 x27: ffff4b45d287f000
  [  181.361789] x26: ffff0009764787a8 x25: 0000000000000000 x24: ffff00080dc99710
  [  181.368970] x23: ffffb4c334553000 x22: 0000000000000001 x21: ffff00080dc99710
  [  181.376151] x20: ffff0009764787a8 x19: ffff000801cc6000 x18: 00000000ffffffff
  [  181.383332] x17: 0000000000000000 x16: ffffb4c3a118c998 x15: 0000000000000000
  [  181.390512] x14: 00003d0900000000 x13: 00000001ffff006e x12: 696765625f747570
  [  181.397692] x11: ffffb4c3a52758d8 x10: 0000000000000108 x9 : ffffb4c3a117f620
  [  181.404873] x8 : ffff800080003a58 x7 : ffff000808de4000 x6 : 00000000000fffff
  [  181.412053] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000000
  [  181.419232] x2 : 0000000000000007 x1 : 0000000000000000 x0 : 0000000000000007
  [  181.426411] Call trace:
  [  181.428868]  perf_aux_output_begin+0x1dc/0x1f0 (P)
  [  181.433692]  etm_event_start+0xdc/0x290 [coresight]
  [  181.438665]  perf_event_unthrottle+0x70/0xa8
  [  181.442964]  perf_event_unthrottle_group+0x40/0x170
  [  181.447871]  perf_adjust_freq_unthr_events+0x178/0x1a8
  [  181.453038]  perf_adjust_freq_unthr_context+0x74/0xf8
  [  181.458120]  perf_event_task_tick+0xa8/0x290
  [  181.462418]  sched_tick+0x144/0x2c8
  [  181.465937]  update_process_times+0xc0/0x198

Use perf_event_aux_pause() for throttling and unthrottling when the PMU
has PERF_PMU_CAP_AUX_PAUSE capability. This uses the pause/resume path
for fast AUX enable and disable, and uses the existing serialization
against pause/resume recursion.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 kernel/events/core.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index 7935d5663944ee1cbaf38cf8018c3347635e8d31..33d049b9cccfa32d4c6f5c8bd41b866c9ee33e65 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -2756,6 +2756,7 @@ void perf_event_disable_inatomic(struct perf_event *event)
 
 static void perf_log_throttle(struct perf_event *event, int enable);
 static void perf_log_itrace_start(struct perf_event *event);
+static void perf_event_aux_pause(struct perf_event *event, bool pause);
 
 static void perf_event_unthrottle(struct perf_event *event, bool start)
 {
@@ -2763,8 +2764,14 @@ static void perf_event_unthrottle(struct perf_event *event, bool start)
 		return;
 
 	event->hw.interrupts = 0;
-	if (start)
-		event->pmu->start(event, 0);
+
+	if (start) {
+		if (event->pmu->capabilities & PERF_PMU_CAP_AUX_PAUSE)
+			perf_event_aux_pause(event, false);
+		else
+			event->pmu->start(event, 0);
+	}
+
 	if (event == event->group_leader)
 		perf_log_throttle(event, 1);
 }
@@ -2775,7 +2782,12 @@ static void perf_event_throttle(struct perf_event *event)
 		return;
 
 	event->hw.interrupts = MAX_INTERRUPTS;
-	event->pmu->stop(event, 0);
+
+	if (event->pmu->capabilities & PERF_PMU_CAP_AUX_PAUSE)
+		perf_event_aux_pause(event, true);
+	else
+		event->pmu->stop(event, 0);
+
 	if (event == event->group_leader)
 		perf_log_throttle(event, 0);
 }

---
base-commit: e1914add2799225a87502051415fc5c32aeb02ae
change-id: 20260514-aux-throttle-fix-53cce43f3e27

Best regards,
-- 
Leo Yan <leo.yan@arm.com>

Re: [PATCH] perf/core: Use AUX pause/resume for throttling

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [High] The event->hw.aux_paused flag is permanently leaked when an event is unthrottled during a context switch, leaving the event paused forever.
- [High] Throttling overwrites and breaks the user-configured aux-action=pause state, overriding user intent.
- [High] Throttle requests from NMIs can be silently dropped due to the recursion guard in perf_event_aux_pause().
- [High] Throttling silently fails to stop the PMU if the event does not have a mapped ring buffer.
- [High] The commit restricts its fix to PMUs with PERF_PMU_CAP_AUX_PAUSE, leaving other AUX PMUs broken and vulnerable to the same WARN_ON_ONCE(nest).
--

commit 1ca7736295ce051e131ce2cca7c64a9e4d034b68
Author: Leo Yan <leo.yan@arm.com>

perf/core: Use AUX pause/resume for throttling

Since commit 9734e25fbf5a ("perf: Fix the throttle logic for a group"), perf
throttles events in a group with pmu->stop(event, 0) and unthrottles them
with pmu->start(event, 0). For AUX trace PMUs, these callbacks also manage
AUX output state, which can trigger a kernel warning. This commit uses
perf_event_aux_pause() for throttling when the PMU supports it.

> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index 7935d5663944e..33d049b9cccfa 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c

[ ... ]

> @@ -2763,8 +2764,14 @@ static void perf_event_unthrottle(struct perf_event *event, bool start)
>  		return;
>  
>  	event->hw.interrupts = 0;
> -	if (start)
> -		event->pmu->start(event, 0);
> +
> +	if (start) {
> +		if (event->pmu->capabilities & PERF_PMU_CAP_AUX_PAUSE)
> +			perf_event_aux_pause(event, false);

If a task is scheduled out, the throttled event is stopped. When scheduled
back in, event_sched_in() detects the throttle state and calls
perf_event_unthrottle(event, false). Because start is false, the call to
perf_event_aux_pause(event, false) is skipped, meaning event->hw.aux_paused
is never cleared.

When event_sched_in() subsequently calls event->pmu->add(event, PERF_EF_START),
PMU drivers check the leaked event->hw.aux_paused flag. Seeing it as 1, does
this cause a regression where the hardware starts in a paused state and never
resumes tracing?

Additionally, if a user or BPF program explicitly pauses an AUX event (e.g.,
via aux-action=pause), event->hw.aux_paused is set to 1. If a sibling event
overflows, the kernel throttles the group.

When the group is subsequently unthrottled, this calls
perf_event_aux_pause(event, false) which clears hw.aux_paused and resumes the
PMU. Could this unconditionally override the user's explicit pause request,
starting tracing when it was intentionally halted?

> +		else
> +			event->pmu->start(event, 0);

This isn't a bug introduced by this patch, but doesn't restricting this fix to
PMUs with PERF_PMU_CAP_AUX_PAUSE leave other AUX PMUs broken?

They still fall back to the original event->pmu->stop(event, 0) logic and will
suffer from the exact same unmatched perf_aux_output_begin() cycle and
subsequent kernel warning when unthrottled.

> +	}
> +
>  	if (event == event->group_leader)
>  		perf_log_throttle(event, 1);
>  }

[ ... ]

> @@ -2775,7 +2782,12 @@ static void perf_event_throttle(struct perf_event *event)
>  		return;
>  
>  	event->hw.interrupts = MAX_INTERRUPTS;
> -	event->pmu->stop(event, 0);
> +
> +	if (event->pmu->capabilities & PERF_PMU_CAP_AUX_PAUSE)
> +		perf_event_aux_pause(event, true);

If an event group is unthrottled in timer context (e.g., sched_tick()) and is
executing perf_event_aux_pause(), an NMI from a sibling event overflow can
interrupt it.

The NMI handler attempts to throttle the group and calls
perf_event_aux_pause(). Looking at perf_event_aux_pause(), it has a recursion
guard:

    if (READ_ONCE(rb->aux_in_pause_resume))
        break;

Since this breaks out early without pausing the hardware, and the fallback to
pmu->stop() was removed here, does the hardware event continue running despite
hw.interrupts == MAX_INTERRUPTS? Could this defeat the throttling mechanism
and cause NMI storms?

Furthermore, if an AUX event is active but has no mapped ring buffer
(event->rb == NULL), perf_event_aux_pause() immediately returns early.

Since the unconditional call to event->pmu->stop(event, 0) was removed, does
this silently fail to stop the PMU if there's no mapped ring buffer?

> +	else
> +		event->pmu->stop(event, 0);
> +
>  	if (event == event->group_leader)
>  		perf_log_throttle(event, 0);
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260514-aux-throttle-fix-v1-1-92bdc84f5922@arm.com?part=1