From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from DM1PR04CU001.outbound.protection.outlook.com (mail-centralusazon11010069.outbound.protection.outlook.com [52.101.61.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CBC41427A for ; Mon, 4 May 2026 05:37:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.61.69 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777873046; cv=fail; b=EOjqwskfwDlvph8YoZce+j7kVaDZ/yG5tnQCRCd2XjbZE9wpzIhc7JxbcVBRP+8EkatNvzv/UDo1cj2h2NlaN+N2M/b87G8P3b1R2db83K/qN4NIMakSbyeNEk6EoyfSrF5/d7U61ILRBYrZp+/sNiXxKUG2Izv3LJzIuIhY0UU= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777873046; c=relaxed/simple; bh=kQkLJzayDhE/oAPg/7PefJKHx343M1r2De+iY+2YSWY=; h=Message-ID:Date:MIME-Version:Subject:From:To:CC:References: In-Reply-To:Content-Type; b=u6THXBUUrf8XNnK+bbROGXyUs7F8EjURUoORnAGRoqxaFKKPxqp8ZvN6U9BvQPQ1HFpW1Bq2umQC4M53dXa9BZMuwPijYolwHzjul+DTjbrbrWs+UvaZ4I026GMrdL2B56vALCAC65y/PeArFlfMpYPvhAldq5XHER/hYMypuXI= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=rZBovTV9; arc=fail smtp.client-ip=52.101.61.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="rZBovTV9" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CZ2eXwb1zQWxzFuaf0oGRrzk7DGN35eP4XyiOrJXgQ4JKHzjU99kLldIIZOqOMr6/cfySay34FSdSww+HA3lGv/D7lwReXr4c2/fBp5gKgcMK/nO/Sk1zMIIZ/2QnYp/vassCuXc+wWPnj+n4JNHQLE+TMEiy/YtEsaCi3530E9NB32zIzRPa1Nkw9eVZ09ifQyPmrrA6XFnB5boePWdz6bvMea3IHbhLZujIwCLkK2b4Pr0SL/zmldqPoXN2f1kuY2YWQbOH54HxLCy8ylltHoHSBXnZiX/cEJXZdhEDMmXfAhJRGablBUwi+W/0Z4g4UU7kvoi4qkkgEYGSMqpJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aIm2lClB+FP5yMTkdeH5qcDRzxMoOdK8IDvkRvQw4WM=; b=bDuxLHF7nMpDvEqqn6SsVOmT2OaIuFYXRO5UJ3TKkU4UTzGMS00JjB/ca1nYg/Cx6rKo5iSYy8gez8G7edqhttUal1mkdwdu07Hi5dMDGFwgeiJX7DfcX6aWpvUWnc/YI2UNhuBxkdzCABX7uZqGuVHYNc9469hv3N+hHlNRJl5i7NrciHOPcdz0Dj0lqfKzogBCU/5LawLW7lI9ppjHRS1VNBrZkASAS7f7nE4nl/uP9q61xMbIpWVYdEZ8dD0hyJ537HMlCfYX7y1avKHTDto+9kaeHwQNE/JgBmhlO4gJWY85XC5WueCeFPA/3QrrFvuX0ZIeVV6kG/RnTtuxoA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=google.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aIm2lClB+FP5yMTkdeH5qcDRzxMoOdK8IDvkRvQw4WM=; b=rZBovTV9Y4YSUbwZjWG4Wi7ntelJxJUnnmXP1oCC0uyqORda0qE6QuLhq6nes5K6A/lp85PPedqGebTCtYoaIoVpfDgNwK4V4boEFYXEcE7rpSo3rqI1G7z6bAzEZGadH/bWL/nKY/jYmbX5D70UoKpqCrmw16yJkvSmxNhGuPs= Received: from MW4PR04CA0098.namprd04.prod.outlook.com (2603:10b6:303:83::13) by CH1PPFF5B95D789.namprd12.prod.outlook.com (2603:10b6:61f:fc00::62a) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.26; Mon, 4 May 2026 05:37:16 +0000 Received: from CO1PEPF000066ED.namprd05.prod.outlook.com (2603:10b6:303:83:cafe::2d) by MW4PR04CA0098.outlook.office365.com (2603:10b6:303:83::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9870.25 via Frontend Transport; Mon, 4 May 2026 05:37:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by CO1PEPF000066ED.mail.protection.outlook.com (10.167.249.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.9 via Frontend Transport; Mon, 4 May 2026 05:37:15 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.2562.17; Mon, 4 May 2026 00:37:13 -0500 Received: from satlexmb08.amd.com (10.181.42.217) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 4 May 2026 00:37:12 -0500 Received: from [172.31.184.125] (10.180.168.240) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Mon, 4 May 2026 00:37:06 -0500 Message-ID: <46ce422c-e796-4280-8165-b7c163928c68@amd.com> Date: Mon, 4 May 2026 11:07:05 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 1/2] sched: proxy-exec: Close race causing workqueue work being delayed From: K Prateek Nayak To: John Stultz , Peter Zijlstra CC: LKML , Vineeth Pillai , Sonam Sanju , "Sean Christopherson" , Kunwu Chan , "Tejun Heo" , Joel Fernandes , Qais Yousef , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Valentin Schneider , Steven Rostedt , Will Deacon , Waiman Long , Boqun Feng , "Paul E. McKenney" , Metin Kaya , Xuewen Yan , Thomas Gleixner , Daniel Lezcano , "Suleiman Souhlal" , kuyo chang , hupu , References: <20260430215103.2978955-1-jstultz@google.com> <20260430215103.2978955-2-jstultz@google.com> <20260501132143.GC1026330@noisy.programming.kicks-ass.net> <63c830c3-fe6d-4822-81db-9fdd1597282e@amd.com> <20260501185900.GF1026330@noisy.programming.kicks-ass.net> Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Received-SPF: None (SATLEXMB04.amd.com: kprateek.nayak@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000066ED:EE_|CH1PPFF5B95D789:EE_ X-MS-Office365-Filtering-Correlation-Id: b4ca9a9a-c50e-4c17-1542-08dea99f32fd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|82310400026|36860700016|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: p8mEeCyYx3S08fdlS4YQB1smXPmXQzKDWijp5U1fizIewN2AWT2LHZ8UrGjX2Axq2mujdkUM30Gx18dEDrUso6oLPu5j4LHzpjGfzY5DNhNhXfo9+6mE6qc8xjv9TvE6xC0Ssh7N9XZcCTjBxSXilFVu0ZBoR+/SSNIifgMeLe9l5m+3sdwCyWCSc9wZWQWfvAU6AQBu5vVMrslmOele0L0vHav8TkS59NWlRIUiD6CSem8pyyepi0KI9jmpapz3Kv5bVuGjPE4Sr7eQe3WkI+CNImxw1HF+J+jTkllQAcWQFVeBRuweQwgIRseyWLDbwg1szY/ct6cnJUewwKyTkm+aC6/CLWeE+Qx1wiU/x/PYkvGCwn2oaju7HIj88qIKZhCUrExV5hEN6SQf0S8Iss6K9PxEAY6szNgGfCNVnOVEdU/Fa5s7p0yYKd7AdkhaQ8UjGErAeO1bOd3zEwnAYl33qqs12HH4ODBQjJoMluAZGPTVp6yUFfaub68fpj8FZo4GpSwzB+cy7IyppZr3uSrYcoAzAK7WB/G6Fh5VqPCCga/3IFy/xJ00YPtCtXVsvsQsV0oGkM1GfH+ZOWBdzmCumtTg7re7Z/4u2iKHfKx9qe5nbupHFibr9iGmnVW68v1s07ir3/dFcrXxVFQjXi+fqOFS3lnhn7T5+q8ivqHs9ZViJrC4RIs8XFx5zUTMN96AMFDQgF9a0l2xtJ7q3ZS5uJRrw0Qt49k0oPIg6+o= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(82310400026)(36860700016)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ylUf7qIjg7yUH2FFtPPyTyLZimHEhkXjJ4uCs8TZxsJNWD3IYiNhtDNlGUzyE3a7V/Tn6ucm4z8ZHs1pgwLL+wiLcq5iAWv8EsuPFQ4D73oGGMYCYFQbmmuGeN45YES+n6Bf7/wSA0ozsoWCoxG/3s3malcQDR9x9Xh9NLCla5xcrng73+NQVwcmGFEJbUiE1XRO0ambDA9/8gQXKX87bzyiOqX1vUFbCwzgnnrXgsfeK29CkPn69SLtsmepEA9jRoAkoqpRqyuhYDs6M+x/U/oD/tuyadeANHsiMSkQBiu4bURBtVL/3d8bH2i51Heok1PYfcnbBYfDHhWlt2oWx5rcICSLQpy/3Me1yLIP5a0xdgbdBgT46XhWDZ2hH4Xpwx5CmQSAR01A2HE467wabC3ThCM4CR7Rg5NPLwHMY8fCTjShKRrmuH+y9bsayhGv X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 May 2026 05:37:15.2417 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b4ca9a9a-c50e-4c17-1542-08dea99f32fd X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000066ED.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH1PPFF5B95D789 On 5/4/2026 12:12 AM, K Prateek Nayak wrote: > So when looking at all of this, I realized we probably don't need > PROXY_WAKING anymore if we have the "is_blocked" state in task_struct. > The owner can simply clear the blocked_on and move along and the > waiter's "is_blocked" state will handle the sched bits. > > (p->is_blocked && !p->blocked_on) can then be interpreted as > PROXY_WAKING and that task should explore return migration in > find_proxy_task(). > > Would something like below be more amenable from a backport standpoint > instead of marking the config broken? > > (Lightly tested; Based on tip:sched/core) ... and I missed this hunk for try_to_block_task(): diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 49cd5d2171613..ee89d751b9594 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -7026,13 +7074,13 @@ static void __sched notrace __schedule(int sched_mode) } } else if (!preempt && prev_state) { /* - * We pass task_is_blocked() as the should_block arg + * We pass task_should_block() as the should_block arg * in order to keep mutex-blocked tasks on the runqueue * for slection with proxy-exec (without proxy-exec - * task_is_blocked() will always be false). + * task_should_block() will always be true). */ try_to_block_task(rq, prev, &prev_state, - !task_is_blocked(prev)); + task_should_block(prev)); switch_count = &prev->nvcsw; } --- Sorry about that and sorry for the noise! Final diffstat looks like: include/linux/sched.h | 56 ++++--------------------- kernel/locking/mutex.c | 2 +- kernel/locking/ww_mutex.h | 14 +++---- kernel/sched/core.c | 72 +++++++++++++++++++++++++++------ kernel/sched/sched.h | 2 +- 5 files changed, 75 insertions(+), 71 deletions(-) It mostly relocates the PROXY_WAKING bits from linux/sched.h to internal "is_blocked" helper in core.c. Pasting the full diff again with some cleanups for convenience: (Lightly tested with test-ww_mutex and sched-messaging) diff --git a/include/linux/sched.h b/include/linux/sched.h index 8ec3b6d7d718b..7be5e1faf56a1 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -846,7 +846,11 @@ struct task_struct { struct alloc_tag *alloc_tag; #endif - int on_cpu; + u8 on_cpu; + u8 on_rq; + u8 is_blocked; + u8 __pad; + struct __call_single_node wake_entry; unsigned int wakee_flips; unsigned long wakee_flip_decay_ts; @@ -861,7 +865,6 @@ struct task_struct { */ int recent_used_cpu; int wake_cpu; - int on_rq; int prio; int static_prio; @@ -2181,19 +2184,10 @@ extern int __cond_resched_rwlock_write(rwlock_t *lock) __must_hold(lock); #ifndef CONFIG_PREEMPT_RT -/* - * With proxy exec, if a task has been proxy-migrated, it may be a donor - * on a cpu that it can't actually run on. Thus we need a special state - * to denote that the task is being woken, but that it needs to be - * evaluated for return-migration before it is run. So if the task is - * blocked_on PROXY_WAKING, return migrate it before running it. - */ -#define PROXY_WAKING ((struct mutex *)(-1L)) - static inline struct mutex *__get_task_blocked_on(struct task_struct *p) { lockdep_assert_held_once(&p->blocked_lock); - return p->blocked_on == PROXY_WAKING ? NULL : p->blocked_on; + return p->blocked_on; } static inline void __set_task_blocked_on(struct task_struct *p, struct mutex *m) @@ -2221,7 +2215,7 @@ static inline void __clear_task_blocked_on(struct task_struct *p, struct mutex * * blocked_on relationships, but make sure we are not * clearing the relationship with a different lock. */ - WARN_ON_ONCE(m && p->blocked_on && p->blocked_on != m && p->blocked_on != PROXY_WAKING); + WARN_ON_ONCE(m && p->blocked_on && p->blocked_on != m); p->blocked_on = NULL; } @@ -2231,34 +2225,6 @@ static inline void clear_task_blocked_on(struct task_struct *p, struct mutex *m) __clear_task_blocked_on(p, m); } -static inline void __set_task_blocked_on_waking(struct task_struct *p, struct mutex *m) -{ - /* Currently we serialize blocked_on under the task::blocked_lock */ - lockdep_assert_held_once(&p->blocked_lock); - - if (!sched_proxy_exec()) { - __clear_task_blocked_on(p, m); - return; - } - - /* Don't set PROXY_WAKING if blocked_on was already cleared */ - if (!p->blocked_on) - return; - /* - * There may be cases where we set PROXY_WAKING on tasks that were - * already set to waking, but make sure we are not changing - * the relationship with a different lock. - */ - WARN_ON_ONCE(m && p->blocked_on != m && p->blocked_on != PROXY_WAKING); - p->blocked_on = PROXY_WAKING; -} - -static inline void set_task_blocked_on_waking(struct task_struct *p, struct mutex *m) -{ - guard(raw_spinlock_irqsave)(&p->blocked_lock); - __set_task_blocked_on_waking(p, m); -} - #else static inline void __clear_task_blocked_on(struct task_struct *p, struct rt_mutex *m) { @@ -2267,14 +2233,6 @@ static inline void __clear_task_blocked_on(struct task_struct *p, struct rt_mute static inline void clear_task_blocked_on(struct task_struct *p, struct rt_mutex *m) { } - -static inline void __set_task_blocked_on_waking(struct task_struct *p, struct rt_mutex *m) -{ -} - -static inline void set_task_blocked_on_waking(struct task_struct *p, struct rt_mutex *m) -{ -} #endif /* !CONFIG_PREEMPT_RT */ static __always_inline bool need_resched(void) diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index 7d359647156df..4aa79bcab08c7 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -983,7 +983,7 @@ static noinline void __sched __mutex_unlock_slowpath(struct mutex *lock, unsigne next = waiter->task; debug_mutex_wake_waiter(lock, waiter); - set_task_blocked_on_waking(next, lock); + clear_task_blocked_on(next, lock); wake_q_add(&wake_q, next); } diff --git a/kernel/locking/ww_mutex.h b/kernel/locking/ww_mutex.h index 5cd9dfa4b31e6..522fe045eb1b2 100644 --- a/kernel/locking/ww_mutex.h +++ b/kernel/locking/ww_mutex.h @@ -285,11 +285,11 @@ __ww_mutex_die(struct MUTEX *lock, struct MUTEX_WAITER *waiter, debug_mutex_wake_waiter(lock, waiter); #endif /* - * When waking up the task to die, be sure to set the - * blocked_on to PROXY_WAKING. Otherwise we can see - * circular blocked_on relationships that can't resolve. + * When waking up the task to die, be sure to clear the + * blocked_on. Otherwise we can see circular blocked_on + * relationships that can't resolve. */ - set_task_blocked_on_waking(waiter->task, lock); + clear_task_blocked_on(waiter->task, lock); wake_q_add(wake_q, waiter->task); } @@ -340,14 +340,14 @@ static bool __ww_mutex_wound(struct MUTEX *lock, if (owner != current) { /* * When waking up the task to wound, be sure to set the - * blocked_on to PROXY_WAKING. Otherwise we can see - * circular blocked_on relationships that can't resolve. + * clear blocked_on. Otherwise we can see circular + * blocked_on relationships that can't resolve. * * NOTE: We pass NULL here instead of lock, because we * are waking the mutex owner, who may be currently * blocked on a different mutex. */ - set_task_blocked_on_waking(owner, NULL); + clear_task_blocked_on(owner, NULL); wake_q_add(wake_q, owner); } return true; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 49cd5d2171613..30672390e6f99 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6495,6 +6495,8 @@ pick_next_task(struct rq *rq, struct task_struct *prev, struct rq_flags *rf) #endif /* !CONFIG_SCHED_CORE */ +static inline void sched_set_task_is_blocked(struct task_struct *p); + /* * Constants for the sched_mode argument of __schedule(). * @@ -6523,7 +6525,18 @@ static bool try_to_block_task(struct rq *rq, struct task_struct *p, if (signal_pending_state(task_state, p)) { WRITE_ONCE(p->__state, TASK_RUNNING); *task_state_p = TASK_RUNNING; - set_task_blocked_on_waking(p, NULL); + + /* + * Clear blocked_on relation if we were planning to + * retain the task as proxy donor since it is runnable + * again as a result of pending signal. + * + * Since only the running task can set the blocked_on + * relation for itself, do not unnecessarily grab the + * blocked_lock if blocked_on is not set. + */ + if (!should_block) + clear_task_blocked_on(p, NULL); return false; } @@ -6535,8 +6548,10 @@ static bool try_to_block_task(struct rq *rq, struct task_struct *p, * blocked on a mutex, and we want to keep it on the runqueue * to be selectable for proxy-execution. */ - if (!should_block) + if (!should_block) { + sched_set_task_is_blocked(p); return false; + } p->sched_contributes_to_load = (task_state & TASK_UNINTERRUPTIBLE) && @@ -6562,6 +6577,27 @@ static bool try_to_block_task(struct rq *rq, struct task_struct *p, } #ifdef CONFIG_SCHED_PROXY_EXEC +static inline void sched_set_task_is_blocked(struct task_struct *p) +{ + if (!sched_proxy_exec()) + return; + + p->is_blocked = 1; +} + +static inline void sched_clear_task_is_blocked(struct task_struct *p) +{ + p->is_blocked = 0; +} + +static inline bool task_should_block(struct task_struct *p) +{ + if (!sched_proxy_exec()) + return true; + + return !p->blocked_on; +} + static inline void proxy_set_task_cpu(struct task_struct *p, int cpu) { unsigned int wake_cpu; @@ -6602,6 +6638,7 @@ static bool proxy_deactivate(struct rq *rq, struct task_struct *donor) * need to be changed from next *before* we deactivate. */ proxy_resched_idle(rq); + sched_clear_task_is_blocked(donor); return try_to_block_task(rq, donor, &state, true); } @@ -6732,7 +6769,7 @@ static void proxy_force_return(struct rq *rq, struct rq_flags *rf, cpu = select_task_rq(p, p->wake_cpu, &wake_flag); set_task_cpu(p, cpu); target_rq = cpu_rq(cpu); - clear_task_blocked_on(p, NULL); + sched_clear_task_is_blocked(p); } if (target_rq) @@ -6765,15 +6802,16 @@ find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf) bool curr_in_chain = false; int this_cpu = cpu_of(rq); struct task_struct *p; - struct mutex *mutex; int owner_cpu; /* Follow blocked_on chain. */ - for (p = donor; (mutex = p->blocked_on); p = owner) { - /* if its PROXY_WAKING, do return migration or run if current */ - if (mutex == PROXY_WAKING) { + for (p = donor; task_is_blocked(p); p = owner) { + struct mutex *mutex = p->blocked_on; + + /* If task is no longer blocked, do return migration or run if current */ + if (!mutex) { if (task_current(rq, p)) { - clear_task_blocked_on(p, PROXY_WAKING); + sched_clear_task_is_blocked(p); return p; } goto force_return; @@ -6807,8 +6845,9 @@ find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf) * and return p (if it is current and safe to * just run on this rq), or return-migrate the task. */ + __clear_task_blocked_on(p, mutex); if (task_current(rq, p)) { - __clear_task_blocked_on(p, NULL); + sched_clear_task_is_blocked(p); return p; } goto force_return; @@ -6902,6 +6941,13 @@ find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf) return NULL; } #else /* SCHED_PROXY_EXEC */ +static inline void sched_set_task_is_blocked(struct task_struct *p) {} + +static inline bool task_should_block(struct task_struct *p) +{ + return true; +} + static struct task_struct * find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf) { @@ -7026,13 +7072,13 @@ static void __sched notrace __schedule(int sched_mode) } } else if (!preempt && prev_state) { /* - * We pass task_is_blocked() as the should_block arg + * We pass task_should_block() as the should_block arg * in order to keep mutex-blocked tasks on the runqueue * for slection with proxy-exec (without proxy-exec - * task_is_blocked() will always be false). + * task_should_block() will always be true). */ try_to_block_task(rq, prev, &prev_state, - !task_is_blocked(prev)); + task_should_block(prev)); switch_count = &prev->nvcsw; } @@ -7044,7 +7090,7 @@ static void __sched notrace __schedule(int sched_mode) struct task_struct *prev_donor = rq->donor; rq_set_donor(rq, next); - if (unlikely(next->blocked_on)) { + if (unlikely(task_is_blocked(next))) { next = find_proxy_task(rq, next, &rf); if (!next) { zap_balance_callbacks(rq); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index c95584191d58f..5c1085f260ad4 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2390,7 +2390,7 @@ static inline bool task_is_blocked(struct task_struct *p) if (!sched_proxy_exec()) return false; - return !!p->blocked_on; + return !!p->is_blocked; } static inline int task_on_cpu(struct rq *rq, struct task_struct *p) -- Thanks and Regards, Prateek