[PATCH] perf/core: Do not resurrect cpuctx->cgrp after last cgroup event exit

[PATCH] perf/core: Do not resurrect cpuctx->cgrp after last cgroup event exit

[Phat Dinh]

perf_cgroup_switch() relies on:

    /*
     * cpuctx->cgrp is set when the first cgroup event enabled,
     * and is cleared when the last cgroup event disabled.
     */

It computes the next task's cgroup, schedules out the CPU context and
then assigns cpuctx->cgrp unconditionally:

    ctx_sched_out(&cpuctx->ctx, NULL, EVENT_ALL|EVENT_CGROUP);
    /*
     * must not be done before ctxswout due
     * to update_cgrp_time_from_cpuctx() in
     * ctx_sched_out()
     */
    cpuctx->cgrp = cgrp;

However, ctx_sched_out() can disable events itself. If a cgroup event
has a pending disable from perf_event_disable_inatomic() (e.g. its
PERF_EVENT_IOC_REFRESH limit ran out) that the irq_work has not
handled yet, event_sched_out() consumes it inline:

    if (event->pending_disable) {
        event->pending_disable = 0;
        perf_cgroup_event_disable(event, ctx);
        state = PERF_EVENT_STATE_OFF;
    }

If that was the last cgroup event, perf_cgroup_event_disable() clears
cpuctx->cgrp:

    if (--ctx->nr_cgroups)
        return;

    cpuctx->cgrp = NULL;

and perf_cgroup_switch() then sets it again. cpuctx->cgrp is left
non-NULL with ctx.nr_cgroups == 0, so the NULL check in
perf_cgroup_switch() no longer catches it and the next switch trips:

    WARN_ON_ONCE(cpuctx->ctx.nr_cgroups == 0);

cpuctx->cgrp does not hold a reference on the cgroup. Once that cgroup
is removed, the next ctx_sched_in() dereferences the stale pointer:

  BUG: KASAN: slab-use-after-free in ctx_sched_in+0xb1f/0xfa0
  Read of size 8 at addr ffff8881008972d0 by task swapper/3/0

  Tainted: [W]=WARN
  Call Trace:
   <IRQ>
   ctx_sched_in+0xb1f/0xfa0
   perf_event_sched_in+0x1a/0x70
   ctx_resched+0x32b/0xac0
   __perf_install_in_context+0x475/0xab0
   remote_function+0x105/0x200
   __flush_smp_call_function_queue+0x179/0x8d0
   __sysvec_call_function_single+0x1b/0x1c0
   sysvec_call_function_single+0x66/0x80
   </IRQ>
   <TASK>
   asm_sysvec_call_function_single+0x1a/0x20

Only assign cpuctx->cgrp when cgroup events remain after the context
has been scheduled out, so that it stays NULL as the invariant
requires.

Fixes: e19cd0b6fa59 ("perf/core: Always set cpuctx cgrp when enable cgroup event")
Cc: stable@vger.kernel.org
Signed-off-by: Phat Dinh <phatdt1116@gmail.com>
---
 kernel/events/core.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index f36a3bbaa33c..54b87584c752 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -1072,8 +1072,13 @@ static void perf_cgroup_switch(struct task_struct *task)
 	 * must not be done before ctxswout due
 	 * to update_cgrp_time_from_cpuctx() in
 	 * ctx_sched_out()
+	 *
+	 * ctx_sched_out() can consume a pending disable and drop the last cgroup
+	 * event, which clear the cpuctx->cgrp. Cgroup tracking follow active cgroup
+	 * event, so do not resurrect it
 	 */
-	cpuctx->cgrp = cgrp;
+	if (cpuctx->ctx.nr_cgroups)
+		cpuctx->cgrp = cgrp;
 	/*
 	 * set cgrp before ctxsw in to allow
 	 * perf_cgroup_set_timestamp() in ctx_sched_in()

base-commit: d095321349222961813333277881b3c74f10b17b
-- 
2.56.0

Re: [PATCH] perf/core: Do not resurrect cpuctx->cgrp after last cgroup event exit

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [High] perf/core: perf_cgroup_switch() skips pmu_ctx after last cgroup
  event disable
--

commit 16552da1b5b468a49aa24ce4ee62d6e477839323
Author: Phat Dinh <phatdt1116@gmail.com>

perf/core: Do not resurrect cpuctx->cgrp after last cgroup event exit

This commit avoids setting cpuctx->cgrp in perf_cgroup_switch() if
ctx_sched_out() disabled the last cgroup event on the CPU context.

> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index f36a3bbaa33cf..54b87584c752e 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -1072,8 +1072,13 @@ static void perf_cgroup_switch(struct task_struct *task)
>  	 * must not be done before ctxswout due
>  	 * to update_cgrp_time_from_cpuctx() in
>  	 * ctx_sched_out()
> +	 *
> +	 * ctx_sched_out() can consume a pending disable and drop the last cgroup
> +	 * event, which clear the cpuctx->cgrp. Cgroup tracking follow active cgroup
> +	 * event, so do not resurrect it
>  	 */
> -	cpuctx->cgrp = cgrp;
> +	if (cpuctx->ctx.nr_cgroups)
> +		cpuctx->cgrp = cgrp;
>  	/*
>  	 * set cgrp before ctxsw in to allow
>  	 * perf_cgroup_set_timestamp() in ctx_sched_in()

[Severity: High]
Does perf_cgroup_switch() still leak pmu_disable_count and leave
non-cgroup per-CPU events unscheduled when ctx_sched_out() consumes a
pending_disable on the last cgroup event of a pmu_ctx?

Before ctx_sched_out(), perf_ctx_disable(&cpuctx->ctx, EVENT_CGROUP)
calls perf_pmu_disable(pmu_ctx->pmu) for each pmu_ctx with
pmu_ctx->nr_cgroups > 0, and ctx_sched_out() schedules out all events on
that pmu_ctx.  When event_sched_out() consumes event->pending_disable on
the last cgroup event of that pmu_ctx, perf_cgroup_event_disable()
decrements event->pmu_ctx->nr_cgroups to 0 alongside ctx->nr_cgroups.

The subsequent ctx_sched_in() and perf_ctx_enable() calls pass
EVENT_CGROUP to for_each_epc(), which skips the pmu_ctx:

kernel/events/core.c:perf_skip_pmu_ctx() {
	if ((event_type & EVENT_CGROUP) && !pmu_ctx->nr_cgroups)
		return true;
	...
}

As a result, perf_ctx_enable() never calls perf_pmu_enable(pmu_ctx->pmu),
leaving pmu_disable_count elevated and the PMU permanently disabled on
that CPU.  ctx_sched_in() also skips __pmu_ctx_sched_in() for that
pmu_ctx while setting cpuctx->ctx.is_active back to EVENT_ALL |
EVENT_TIME, leaving any non-cgroup per-CPU events on that PMU stuck in
PERF_EVENT_STATE_INACTIVE.  On x86 hardware PMUs, because x86_pmu_enable()
no longer runs to assign event->hw.idx, scheduling in and then scheduling
out a later hardware event calls x86_pmu_del() with event->hw.idx == -1,
writing out of bounds to cpuc->dirty and cpuc->events[-1].

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106101642.494674-1-phatdt1116@gmail.com?part=1

Re: [PATCH] perf/core: Do not resurrect cpuctx->cgrp after last cgroup event exit

[Peter Zijlstra]

On Fri, Nov 06, 2026 at 05:16:42PM +0700, Phat Dinh wrote:
> perf_cgroup_switch() relies on:
> 
>     /*
>      * cpuctx->cgrp is set when the first cgroup event enabled,
>      * and is cleared when the last cgroup event disabled.
>      */
> 
> It computes the next task's cgroup, schedules out the CPU context and
> then assigns cpuctx->cgrp unconditionally:
> 
>     ctx_sched_out(&cpuctx->ctx, NULL, EVENT_ALL|EVENT_CGROUP);
>     /*
>      * must not be done before ctxswout due
>      * to update_cgrp_time_from_cpuctx() in
>      * ctx_sched_out()
>      */
>     cpuctx->cgrp = cgrp;
> 
> However, ctx_sched_out() can disable events itself. If a cgroup event
> has a pending disable from perf_event_disable_inatomic() (e.g. its
> PERF_EVENT_IOC_REFRESH limit ran out) that the irq_work has not
> handled yet, event_sched_out() consumes it inline:
> 
>     if (event->pending_disable) {
>         event->pending_disable = 0;
>         perf_cgroup_event_disable(event, ctx);
>         state = PERF_EVENT_STATE_OFF;
>     }
> 
> If that was the last cgroup event, perf_cgroup_event_disable() clears
> cpuctx->cgrp:
> 
>     if (--ctx->nr_cgroups)
>         return;
> 
>     cpuctx->cgrp = NULL;
> 
> and perf_cgroup_switch() then sets it again. cpuctx->cgrp is left
> non-NULL with ctx.nr_cgroups == 0, so the NULL check in
> perf_cgroup_switch() no longer catches it and the next switch trips:
> 
>     WARN_ON_ONCE(cpuctx->ctx.nr_cgroups == 0);
> 
> cpuctx->cgrp does not hold a reference on the cgroup. Once that cgroup
> is removed, the next ctx_sched_in() dereferences the stale pointer:
> 
>   BUG: KASAN: slab-use-after-free in ctx_sched_in+0xb1f/0xfa0
>   Read of size 8 at addr ffff8881008972d0 by task swapper/3/0
> 
>   Tainted: [W]=WARN
>   Call Trace:
>    <IRQ>
>    ctx_sched_in+0xb1f/0xfa0
>    perf_event_sched_in+0x1a/0x70
>    ctx_resched+0x32b/0xac0
>    __perf_install_in_context+0x475/0xab0
>    remote_function+0x105/0x200
>    __flush_smp_call_function_queue+0x179/0x8d0
>    __sysvec_call_function_single+0x1b/0x1c0
>    sysvec_call_function_single+0x66/0x80
>    </IRQ>
>    <TASK>
>    asm_sysvec_call_function_single+0x1a/0x20
> 
> Only assign cpuctx->cgrp when cgroup events remain after the context
> has been scheduled out, so that it stays NULL as the invariant
> requires.
> 
> Fixes: e19cd0b6fa59 ("perf/core: Always set cpuctx cgrp when enable cgroup event")
> Cc: stable@vger.kernel.org
> Signed-off-by: Phat Dinh <phatdt1116@gmail.com>

Indeed, thanks!

> ---
>  kernel/events/core.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index f36a3bbaa33c..54b87584c752 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -1072,8 +1072,13 @@ static void perf_cgroup_switch(struct task_struct *task)
>  	 * must not be done before ctxswout due
>  	 * to update_cgrp_time_from_cpuctx() in
>  	 * ctx_sched_out()
> +	 *
> +	 * ctx_sched_out() can consume a pending disable and drop the last cgroup
> +	 * event, which clear the cpuctx->cgrp. Cgroup tracking follow active cgroup
> +	 * event, so do not resurrect it
>  	 */
> -	cpuctx->cgrp = cgrp;
> +	if (cpuctx->ctx.nr_cgroups)
> +		cpuctx->cgrp = cgrp;
>  	/*
>  	 * set cgrp before ctxsw in to allow
>  	 * perf_cgroup_set_timestamp() in ctx_sched_in()
> 
> base-commit: d095321349222961813333277881b3c74f10b17b
> -- 
> 2.56.0
> 

[tip: perf/urgent] perf/core: Do not resurrect cpuctx->cgrp after last cgroup event exit

[tip-bot2 for Phat Dinh]

The following commit has been merged into the perf/urgent branch of tip:

Commit-ID:     59f1398140869432c7171b9a2a279ca315467c4e
Gitweb:        https://git.kernel.org/tip/59f1398140869432c7171b9a2a279ca315467c4e
Author:        Phat Dinh <phatdt1116@gmail.com>
AuthorDate:    Fri, 06 Nov 2026 17:16:42 +07:00
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Fri, 06 Nov 2026 11:45:15 +01:00

perf/core: Do not resurrect cpuctx->cgrp after last cgroup event exit

perf_cgroup_switch() relies on:

    /*
     * cpuctx->cgrp is set when the first cgroup event enabled,
     * and is cleared when the last cgroup event disabled.
     */

It computes the next task's cgroup, schedules out the CPU context and
then assigns cpuctx->cgrp unconditionally:

    ctx_sched_out(&cpuctx->ctx, NULL, EVENT_ALL|EVENT_CGROUP);
    /*
     * must not be done before ctxswout due
     * to update_cgrp_time_from_cpuctx() in
     * ctx_sched_out()
     */
    cpuctx->cgrp = cgrp;

However, ctx_sched_out() can disable events itself. If a cgroup event
has a pending disable from perf_event_disable_inatomic() (e.g. its
PERF_EVENT_IOC_REFRESH limit ran out) that the irq_work has not
handled yet, event_sched_out() consumes it inline:

    if (event->pending_disable) {
        event->pending_disable = 0;
        perf_cgroup_event_disable(event, ctx);
        state = PERF_EVENT_STATE_OFF;
    }

If that was the last cgroup event, perf_cgroup_event_disable() clears
cpuctx->cgrp:

    if (--ctx->nr_cgroups)
        return;

    cpuctx->cgrp = NULL;

and perf_cgroup_switch() then sets it again. cpuctx->cgrp is left
non-NULL with ctx.nr_cgroups == 0, so the NULL check in
perf_cgroup_switch() no longer catches it and the next switch trips:

    WARN_ON_ONCE(cpuctx->ctx.nr_cgroups == 0);

cpuctx->cgrp does not hold a reference on the cgroup. Once that cgroup
is removed, the next ctx_sched_in() dereferences the stale pointer:

  BUG: KASAN: slab-use-after-free in ctx_sched_in+0xb1f/0xfa0
  Read of size 8 at addr ffff8881008972d0 by task swapper/3/0

  Tainted: [W]=WARN
  Call Trace:
   <IRQ>
   ctx_sched_in+0xb1f/0xfa0
   perf_event_sched_in+0x1a/0x70
   ctx_resched+0x32b/0xac0
   __perf_install_in_context+0x475/0xab0
   remote_function+0x105/0x200
   __flush_smp_call_function_queue+0x179/0x8d0
   __sysvec_call_function_single+0x1b/0x1c0
   sysvec_call_function_single+0x66/0x80
   </IRQ>
   <TASK>
   asm_sysvec_call_function_single+0x1a/0x20

Only assign cpuctx->cgrp when cgroup events remain after the context
has been scheduled out, so that it stays NULL as the invariant
requires.

Fixes: e19cd0b6fa59 ("perf/core: Always set cpuctx cgrp when enable cgroup event")
Signed-off-by: Phat Dinh <phatdt1116@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20261106101642.494674-1-phatdt1116@gmail.com
---
 kernel/events/core.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index 1404dab..e870f12 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -1072,8 +1072,13 @@ static void perf_cgroup_switch(struct task_struct *task)
 	 * must not be done before ctxswout due
 	 * to update_cgrp_time_from_cpuctx() in
 	 * ctx_sched_out()
+	 *
+	 * ctx_sched_out() can consume a pending disable and drop the last cgroup
+	 * event, which clear the cpuctx->cgrp. Cgroup tracking follow active cgroup
+	 * event, so do not resurrect it
 	 */
-	cpuctx->cgrp = cgrp;
+	if (cpuctx->ctx.nr_cgroups)
+		cpuctx->cgrp = cgrp;
 	/*
 	 * set cgrp before ctxsw in to allow
 	 * perf_cgroup_set_timestamp() in ctx_sched_in()