[PATCH sched_ext/for-7.4] sched_ext: Keep proxy donors local through core-sched retries

[PATCH sched_ext/for-7.4] sched_ext: Keep proxy donors local through core-sched retries

[Andrea Righi]

An abandoned blocked-donor pick can leave SCX_RQ_PROXY_PENDING set if
the next core-sched pick selects a higher-priority class without calling
pick_task_scx(). A cookie mismatch can then force the CPU idle, while
put_prev_task_scx() mistakes the stale flag for a proxy bookkeeping put.

A valid proxy put of an IMMED donor also schedules a local reenqueue
scan. If an SCX task wakes before the scan runs, rq->next_class becomes
ext again and rq_is_open() reports the CPU closed. The scan then sends
the parked donor back to BPF even though it is first on the local DSQ
and the CPU is switching to idle.

Require the donor's cookie to match before treating a put as a proxy
put. Mark donors parked by a proxy put in dsq_flags, clear the mark
when they leave the DSQ, and let rq_is_open() recognize a matching
marked donor at the head of the local DSQ while the rq donor is idle.

Fixes: 7ab553756463 ("sched_ext: Keep proxy donors with slice left on the local DSQ")
Reported-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/r/atCQB_nF5di8pgUN@gpd4
Signed-off-by: Andrea Righi <arighi@nvidia.com>
---
 include/linux/sched/ext.h |  1 +
 kernel/sched/ext/ext.c    | 28 +++++++++++++++++++++++++++-
 2 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/include/linux/sched/ext.h b/include/linux/sched/ext.h
index ae696d11dac9a..4267304823cf4 100644
--- a/include/linux/sched/ext.h
+++ b/include/linux/sched/ext.h
@@ -157,6 +157,7 @@ enum scx_ent_flags {
 /* scx_entity.dsq_flags */
 enum scx_ent_dsq_flags {
 	SCX_TASK_DSQ_ON_PRIQ	= 1 << 0, /* task is queued on the priority queue of a dsq */
+	SCX_TASK_DSQ_PROXY_IDLE	= 1 << 1, /* proxy donor parked through idle */
 };
 
 enum scx_dsq_lnode_flags {
diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c
index 4fa294f810f4b..2465e792db2fb 100644
--- a/kernel/sched/ext/ext.c
+++ b/kernel/sched/ext/ext.c
@@ -437,6 +437,23 @@ static bool rq_is_open(struct rq *rq, u64 enq_flags)
 	if (sched_class_above(&ext_sched_class, rq->next_class))
 		return true;
 
+	/*
+	 * proxy_resched_idle() can put a blocked donor at the head of the local
+	 * DSQ and switch @rq's donor to idle. An SCX wakeup then raises
+	 * next_class to ext before the pending IMMED reenq scan runs. If the
+	 * first task was parked by that proxy put and its cookie can run here,
+	 * leave it local for the next pick instead of sending it back to BPF.
+	 */
+	if (rq->donor == rq->idle) {
+		struct task_struct *first;
+
+		first = list_first_entry_or_null(&rq->scx.local_dsq.list,
+						 struct task_struct, scx.dsq_list.node);
+		if (first && (first->scx.dsq_flags & SCX_TASK_DSQ_PROXY_IDLE) &&
+		    sched_cpu_cookie_match(rq, first))
+			return true;
+	}
+
 	/*
 	 * @rq is either picking, in transition to, or running an SCX task.
 	 */
@@ -1608,6 +1625,7 @@ static void dsq_dec_nr(struct scx_dispatch_q *dsq, struct task_struct *p)
 {
 	/* see dsq_inc_nr() */
 	WRITE_ONCE(dsq->nr, dsq->nr - 1);
+	p->scx.dsq_flags &= ~SCX_TASK_DSQ_PROXY_IDLE;
 
 	if (p->scx.flags & SCX_TASK_IMMED) {
 		struct rq *rq = container_of(dsq, struct rq, scx.local_dsq);
@@ -3521,12 +3539,18 @@ static void switch_class(struct rq *rq, struct task_struct *next)
 	}
 }
 
+/*
+ * A proxy retry can bypass pick_task_scx() and leave PROXY_PENDING set.
+ * A cookie mismatch means core scheduling forced this CPU idle, not that
+ * proxy resolution put a selected donor through idle.
+ */
 static void put_prev_task_scx(struct rq *rq, struct task_struct *p,
 			      struct task_struct *next)
 {
 	struct scx_sched *sch = scx_task_sched(p);
 	bool proxy_put = p->is_blocked && next == rq->idle &&
-			 (rq->scx.flags & SCX_RQ_PROXY_PENDING);
+			 (rq->scx.flags & SCX_RQ_PROXY_PENDING) &&
+			 sched_cpu_cookie_match(rq, p);
 	bool proxy_block = rq->scx.flags & SCX_RQ_PROXY_BLOCKING;
 	bool rescue_keep = false;
 
@@ -3612,6 +3636,8 @@ static void put_prev_task_scx(struct rq *rq, struct task_struct *p,
 						enq_flags |= SCX_ENQ_IMMED;
 				}
 
+				if (proxy_put)
+					p->scx.dsq_flags |= SCX_TASK_DSQ_PROXY_IDLE;
 				scx_dispatch_enqueue(sch, rq, &rq->scx.local_dsq, p, 0, 0,
 						     enq_flags);
 			}
-- 
2.56.0

Re: [PATCH sched_ext/for-7.4] sched_ext: Keep proxy donors local through core-sched retries

[Tejun Heo]

Hello, Andrea.

The following is a Claude-generated review.

Everything below is from reading the code and nothing has been
reproduced.

On Thu, Oct 15, 2026 at 12:19:35PM +0200, Andrea Righi wrote:
> Fixes: 7ab553756463 ("sched_ext: Keep proxy donors with slice left on the local DSQ")

This commit doesn't exist. The parent landed in sched_ext/for-7.4 as
67d0d8d018f3.

> +	if (rq->donor == rq->idle) {
> +		struct task_struct *first;
> +
> +		first = list_first_entry_or_null(&rq->scx.local_dsq.list,
> +						 struct task_struct, scx.dsq_list.node);
> +		if (first && (first->scx.dsq_flags & SCX_TASK_DSQ_PROXY_IDLE) &&
> +		    sched_cpu_cookie_match(rq, first))
> +			return true;
> +	}

Does this need the mark? With rq->donor == rq->idle and next_class at
ext, the next pick takes the local DSQ head no matter how it got there.
Non-proxy IMMED tasks hit the same spurious reenq today. For example,
two IMMED wakeups land on an idle CPU. The first insert sees the rq open
and rq_owned_post_enq() raises next_class to ext. The second makes nr 2
and schedules a reenq, which task_woken_scx() runs. reenq_local() then
sees the rq closed and sends the first task back to BPF even though it's
at the head.

Testing just the head's cookie while the donor is idle would cover both,
and SCX_TASK_DSQ_PROXY_IDLE with its set and clear could go away.

Also, first_local_task() is the same lookup if moved up. As the block
only runs with next_class at ext, maybe it belongs below the "@rq is
either picking, in transition to, or running an SCX task." comment?

> +/*
> + * A proxy retry can bypass pick_task_scx() and leave PROXY_PENDING set.
> + * A cookie mismatch means core scheduling forced this CPU idle, not that
> + * proxy resolution put a selected donor through idle.
> + */
>  static void put_prev_task_scx(struct rq *rq, struct task_struct *p,
>  			      struct task_struct *next)
>  {
>  	struct scx_sched *sch = scx_task_sched(p);
>  	bool proxy_put = p->is_blocked && next == rq->idle &&
> -			 (rq->scx.flags & SCX_RQ_PROXY_PENDING);
> +			 (rq->scx.flags & SCX_RQ_PROXY_PENDING) &&
> +			 sched_cpu_cookie_match(rq, p);

A forced idle can also happen with a matching cookie. If the core-wide
max is untagged, core_cookie is 0 and pick_next_task() forces a
mismatching rq idle without trying sched_core_find(), so an untagged
donor still passes sched_cpu_cookie_match(). For example, with CFS
bandwidth control:

1. A tagged fair task H on this rq is throttled. A sibling's core-wide
   pick selects the untagged blocked donor D for this rq under an
   untagged max.

2. H is unthrottled. The unthrottle re-enqueues H with
   enqueue_task_fair(), which doesn't bump core_task_seq, so this rq
   still takes the core_pick fastpath and set_next_task_scx() sets
   SCX_RQ_PROXY_PENDING for D.

3. find_proxy_task() abandons the pick without a put and __schedule()
   goes to pick_again.

4. H wins pick_task() on this rq, so pick_task_scx() doesn't run. The
   sibling's untagged task is still the max, core_cookie is 0 and this
   rq is forced idle.

5. put_prev_task_scx() sees the stale flag and a matching cookie and
   parks D as a proxy put.

Maybe clear SCX_RQ_PROXY_PENDING where find_proxy_task() abandons the
pick instead? None of the existing hooks fit the pick_again site, so
it'd need a small new one, but the flag then can't go stale and the
cookie test isn't needed.

And maybe retitle? The reproduced bug is an SCX wakeup during the proxy
idle rather than a core-sched retry, and "retries" reads like
SCX_RQ_PROXY_RETRY.

Thanks.

-- 
tejun