From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADCD137F8D3 for ; Wed, 4 Mar 2026 06:38:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.202 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772606307; cv=none; b=C+peAKTJiUyssab89RE5ploLzZlyyRdC7GuJHF1BnhTRZlPni1LcdisA0onaI2ZLnA8ZCweGPBan+rS5HiphQ48FmpiMN56pycmjkj2nSHRkHeEAnVB0+6FvOTa2FiytbwNEYawPVCZozUGhj09pnSV7x4YO/IobLaiO1uAe+OE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772606307; c=relaxed/simple; bh=+yy5NUrajz1605BSsFfS34I8KO+4KSKY7c8hrB837+U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DBzECjKxVORoDLHVu40E4HFJOFsY8omhhVDdcqIdouR2LzHYe7X0PAb123Tv17bH21eEL3qmgyLUtHl8D63aBGqVcEgasf4b/d2tsEAAE0FQKibnfidkwQTYvecHPp7NMhhZJQ4PCpr4o60x/xQhWPf3BXHHRwMFnwPO0cllki0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jstultz.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=T3gzEidc; arc=none smtp.client-ip=209.85.215.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jstultz.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="T3gzEidc" Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-c70eb09daefso3667132a12.0 for ; Tue, 03 Mar 2026 22:38:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1772606305; x=1773211105; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=F3DxcTs2/nxeJzEczoWxtKja1sVoACzOL6w2KTC0xHU=; b=T3gzEidcuBxG21vm+MzET8UoYJWaDdMAgCmDirXpoOjg9AngQaQ9XNYKszQF2iyg/r QbCxLO3wxPVYzaM94XyT4xQHUEBHUAEKCb9LE6o/Rg+31NsM8xGeHIc4jTyN0c/dZBOB NLbYD/huJERLMa/gsKxbbrv5rn5xuRfHpV8jOuxeth6mYFHaTVSTvljLgi/DR802RlDJ OrNK99LeJ/G0LfSH0etR/nmPBAuPBH7VwECa6kZmQSSrS7vEo2FeWFv9LWM2r69CpPTa j5cAHs2i1J4NrBNIHxGVokIPBasyf32uB+HX1KIRH40o3f+dPmtJAjk49ud3+rZbQwwS NwSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772606305; x=1773211105; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=F3DxcTs2/nxeJzEczoWxtKja1sVoACzOL6w2KTC0xHU=; b=bkYsUuBr9sMHRxHymizB+O4b2nzBRYz/NYzSUKOjTfrjTrEOUIW3/7YgaOSwp15CMu Rty5uIcIicENEw5mgoUcn86D5IqPpqeTCpY7fevmWSs/wNvGqobtwe/NHdzCtRiJJbQ4 2B8pe0QnUn+ITS+s6/XF8xCURLVlhGYdaBDxyk2DfADF00PeD0t+wAHn2ND4Z3Mh3vQm Dd+HGn54FxRT3bBiegfVjzHQHewP0EJQvIoPrFi8lCw2go19YdebtBprEPigvMTSTlh/ vKHe8WWWwm0XKjru1GPDG92qGg7lUyWp8m6rrW3Hdmg2gBvZ3rM2XUSFed8lPfG8YU2X QT7A== X-Gm-Message-State: AOJu0YwZQss5dxu9aAou4tDvhESB1J+/8XJZJWLoUNgTXzQAauhd/ZDi ID+qqIzD3dG5KtxQdDLYVwlhIxF8tiXmmnYM665HHLisLYQ7eqkiV1ImgyknBSlKuwQISrQvFJ9 ZjgJKu+gz2dCMzuqSakHrjC8NvW3hFo4K0TsndPAKpOhzQjKQaFrCrF39KmWLBqbJHFfD/QkjlV Rn/lvMlRjyklRQrTdkeuYqEHhRRiWj4ijEVK+hzzhSsGhf2BSb X-Received: from pge15.prod.google.com ([2002:a05:6a02:2d0f:b0:bac:6acd:8182]) (user=jstultz job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:68f:b0:33b:5a3f:3cbe with SMTP id adf61e73a8af0-3982e1f479bmr991490637.54.1772606304640; Tue, 03 Mar 2026 22:38:24 -0800 (PST) Date: Wed, 4 Mar 2026 06:38:10 +0000 In-Reply-To: <20260304063817.796180-1-jstultz@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260304063817.796180-1-jstultz@google.com> X-Mailer: git-send-email 2.53.0.473.g4a7958ca14-goog Message-ID: <20260304063817.796180-2-jstultz@google.com> Subject: [RFC][PATCH 2/2] sched: proxy-exec: Add allow/prevent_migration hooks in the sched classes for proxy_tag_curr From: John Stultz To: LKML Cc: John Stultz , zhidao su , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Valentin Schneider , Johannes Weiner , Steven Rostedt , Ben Segall , Mel Gorman , Joel Fernandes , Qais Yousef , Xuewen Yan , K Prateek Nayak , Suleiman Souhlal , kuyo chang , hupu , soolaugust@gmail.com, kernel-team@android.com Content-Type: text/plain; charset="UTF-8" Currently proxy_tag_curr() calls task dequeue and task enqueue functions (on lock owners we want to run on behalf of a waiting donor) in order to force the owner task we are going to run to be removed from any sched_class pushable lists. This avoids crashes that could happen if the task being run ended up being migrated to another cpu. The dequeue/enqueue pair is sort of an ugly hack though, so replace these with more focused prevent_migration and allow_migration function pointers to prevent and then to later allow it to be migratable after the blocked donor has finished running it on its behalf. This patch was inspired from discussion around a similar RFC patch by: zhidao su Which highlighted the inefficiency of the dequeue/enqueue pair: https://lore.kernel.org/lkml/20260303115718.278608-1-soolaugust@gmail.com/ Reported-by: zhidao su Closes: https://lore.kernel.org/lkml/20260303115718.278608-1-soolaugust@gmail.com/ Signed-off-by: John Stultz --- Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Juri Lelli Cc: Vincent Guittot Cc: Dietmar Eggemann Cc: Valentin Schneider Cc: Johannes Weiner Cc: Steven Rostedt Cc: Ben Segall Cc: Mel Gorman Cc: Joel Fernandes Cc: Qais Yousef Cc: Xuewen Yan Cc: K Prateek Nayak Cc: Suleiman Souhlal Cc: kuyo chang Cc: hupu Cc: zhidao su Cc: soolaugust@gmail.com Cc: kernel-team@android.com --- kernel/sched/core.c | 18 +++++++++++++----- kernel/sched/deadline.c | 34 ++++++++++++++++++++++++++-------- kernel/sched/rt.c | 28 +++++++++++++++++++++++----- kernel/sched/sched.h | 3 +++ 4 files changed, 65 insertions(+), 18 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 55bafb1585eca..174a3177a3a6b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6712,11 +6712,19 @@ static inline void proxy_tag_curr(struct rq *rq, struct task_struct *owner) * However, the chosen/donor task *and* the mutex owner form an * atomic pair wrt push/pull. * - * Make sure owner we run is not pushable. Unfortunately we can - * only deal with that by means of a dequeue/enqueue cycle. :-/ + * Make sure owner we run is not pushable. */ - dequeue_task(rq, owner, DEQUEUE_NOCLOCK | DEQUEUE_SAVE); - enqueue_task(rq, owner, ENQUEUE_NOCLOCK | ENQUEUE_RESTORE); + if (owner->sched_class->prevent_migration) + owner->sched_class->prevent_migration(rq, owner); +} + +static inline void proxy_untag_prev(struct rq *rq, struct task_struct *prev) +{ + if (!sched_proxy_exec()) + return; + + if (prev->sched_class->allow_migration) + prev->sched_class->allow_migration(rq, prev); } /* @@ -6874,7 +6882,7 @@ static void __sched notrace __schedule(int sched_mode) if (!task_current_donor(rq, next)) proxy_tag_curr(rq, next); if (!(!preempt && prev_state) && prev != prev_donor) - proxy_tag_curr(rq, prev); + proxy_untag_prev(rq, prev); /* * The membarrier system call requires each architecture diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index d08b004293234..6bd6f0682e6c6 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2289,6 +2289,28 @@ static void dequeue_dl_entity(struct sched_dl_entity *dl_se, int flags) task_non_contending(dl_se, true); } +static inline void __allow_migration_dl(struct rq *rq, struct task_struct *p) +{ + if (dl_server(&p->dl)) + return; + + if (task_is_blocked(p)) + return; + + if (!task_current(rq, p) && !p->dl.dl_throttled && p->nr_cpus_allowed > 1) + enqueue_pushable_dl_task(rq, p); +} + +static void allow_migration_dl(struct rq *rq, struct task_struct *p) +{ + __allow_migration_dl(rq, p); +} + +static void prevent_migration_dl(struct rq *rq, struct task_struct *p) +{ + dequeue_pushable_dl_task(rq, p); +} + static void enqueue_task_dl(struct rq *rq, struct task_struct *p, int flags) { if (is_dl_boosted(&p->dl)) { @@ -2339,14 +2361,7 @@ static void enqueue_task_dl(struct rq *rq, struct task_struct *p, int flags) enqueue_dl_entity(&p->dl, flags); - if (dl_server(&p->dl)) - return; - - if (task_is_blocked(p)) - return; - - if (!task_current(rq, p) && !p->dl.dl_throttled && p->nr_cpus_allowed > 1) - enqueue_pushable_dl_task(rq, p); + __allow_migration_dl(rq, p); } static bool dequeue_task_dl(struct rq *rq, struct task_struct *p, int flags) @@ -3408,6 +3423,9 @@ DEFINE_SCHED_CLASS(dl) = { .dequeue_task = dequeue_task_dl, .yield_task = yield_task_dl, + .allow_migration = allow_migration_dl, + .prevent_migration = prevent_migration_dl, + .wakeup_preempt = wakeup_preempt_dl, .pick_task = pick_task_dl, diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index f69e1f16d9238..90f1c62e1f827 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1424,6 +1424,25 @@ static void dequeue_rt_entity(struct sched_rt_entity *rt_se, unsigned int flags) enqueue_top_rt_rq(&rq->rt); } +static void __allow_migration_rt(struct rq *rq, struct task_struct *p) +{ + if (task_is_blocked(p)) + return; + + if (!task_current(rq, p) && p->nr_cpus_allowed > 1) + enqueue_pushable_task(rq, p); +} + +static void allow_migration_rt(struct rq *rq, struct task_struct *p) +{ + __allow_migration_rt(rq, p); +} + +static void prevent_migration_rt(struct rq *rq, struct task_struct *p) +{ + dequeue_pushable_task(rq, p); +} + /* * Adding/removing a task to/from a priority array: */ @@ -1440,11 +1459,7 @@ enqueue_task_rt(struct rq *rq, struct task_struct *p, int flags) enqueue_rt_entity(rt_se, flags); - if (task_is_blocked(p)) - return; - - if (!task_current(rq, p) && p->nr_cpus_allowed > 1) - enqueue_pushable_task(rq, p); + __allow_migration_rt(rq, p); } static bool dequeue_task_rt(struct rq *rq, struct task_struct *p, int flags) @@ -2583,6 +2598,9 @@ DEFINE_SCHED_CLASS(rt) = { .dequeue_task = dequeue_task_rt, .yield_task = yield_task_rt, + .allow_migration = allow_migration_rt, + .prevent_migration = prevent_migration_rt, + .wakeup_preempt = wakeup_preempt_rt, .pick_task = pick_task_rt, diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 43bbf0693cca4..5c3eb8b28ebd3 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2575,6 +2575,9 @@ struct sched_class { */ void (*migrate_task_rq)(struct task_struct *p, int new_cpu); + void (*allow_migration)(struct rq *rq, struct task_struct *p); + void (*prevent_migration)(struct rq *rq, struct task_struct *p); + /* * ttwu_do_activate: rq->lock * wake_up_new_task: task_rq_lock -- 2.53.0.473.g4a7958ca14-goog