[PATCH] sched: Clarify wake_up_q()'s write to task->wake_q.next

[PATCH] sched: Clarify wake_up_q()'s write to task->wake_q.next

[Jann Horn]

Clarify that wake_up_q() does an atomic write to task->wake_q.next, after
which a concurrent __wake_q_add() can immediately overwrite
task->wake_q.next again.

Signed-off-by: Jann Horn <jannh@google.com>
---
a minor change; I figured we should at least be using WRITE_ONCE() here,
and I might as well change the comments a little bit while I'm touching
this
---
 kernel/sched/core.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 9142a0394d46605e96e10cef97cce02d741f6c93..ce64652858703826dca510479f563a28c2fb2405 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1063,9 +1063,10 @@ void wake_up_q(struct wake_q_head *head)
 		struct task_struct *task;
 
 		task = container_of(node, struct task_struct, wake_q);
-		/* Task can safely be re-inserted now: */
 		node = node->next;
-		task->wake_q.next = NULL;
+		/* pairs with cmpxchg_relaxed() in __wake_q_add() */
+		WRITE_ONCE(task->wake_q.next, NULL);
+		/* Task can safely be re-inserted now. */
 
 		/*
 		 * wake_up_process() executes a full barrier, which pairs with

---
base-commit: 05dbaf8dd8bf537d4b4eb3115ab42a5fb40ff1f5
change-id: 20250129-sched-wakeup-prettier-c28762d8deee

-- 
Jann Horn <jannh@google.com>

Re: [PATCH] sched: Clarify wake_up_q()'s write to task->wake_q.next

[Peter Zijlstra]

On Wed, Jan 29, 2025 at 08:53:03PM +0100, Jann Horn wrote:
> Clarify that wake_up_q() does an atomic write to task->wake_q.next, after
> which a concurrent __wake_q_add() can immediately overwrite
> task->wake_q.next again.
> 
> Signed-off-by: Jann Horn <jannh@google.com>
> ---
> a minor change; I figured we should at least be using WRITE_ONCE() here,
> and I might as well change the comments a little bit while I'm touching
> this
> ---
>  kernel/sched/core.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 9142a0394d46605e96e10cef97cce02d741f6c93..ce64652858703826dca510479f563a28c2fb2405 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -1063,9 +1063,10 @@ void wake_up_q(struct wake_q_head *head)
>  		struct task_struct *task;
>  
>  		task = container_of(node, struct task_struct, wake_q);
> -		/* Task can safely be re-inserted now: */
>  		node = node->next;
> -		task->wake_q.next = NULL;
> +		/* pairs with cmpxchg_relaxed() in __wake_q_add() */
> +		WRITE_ONCE(task->wake_q.next, NULL);
> +		/* Task can safely be re-inserted now. */

Right, so even if the store is shattered, the cmpxchg won't proceed
until all bits land -- eg. NULL becomes 'complete'.

That said, your patch makes sense, so let me go stick that in a queue
somewhere.

Thanks!

[tip: sched/urgent] sched: Clarify wake_up_q()'s write to task->wake_q.next

[tip-bot2 for Jann Horn]

The following commit has been merged into the sched/urgent branch of tip:

Commit-ID:     bcc6244e13b4d4903511a1ea84368abf925031c0
Gitweb:        https://git.kernel.org/tip/bcc6244e13b4d4903511a1ea84368abf925031c0
Author:        Jann Horn <jannh@google.com>
AuthorDate:    Wed, 29 Jan 2025 20:53:03 +01:00
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Sat, 08 Feb 2025 15:43:12 +01:00

sched: Clarify wake_up_q()'s write to task->wake_q.next

Clarify that wake_up_q() does an atomic write to task->wake_q.next, after
which a concurrent __wake_q_add() can immediately overwrite
task->wake_q.next again.

Signed-off-by: Jann Horn <jannh@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20250129-sched-wakeup-prettier-v1-1-2f51f5f663fa@google.com
---
 kernel/sched/core.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 3e5a6bf..8931d9b 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1055,9 +1055,10 @@ void wake_up_q(struct wake_q_head *head)
 		struct task_struct *task;
 
 		task = container_of(node, struct task_struct, wake_q);
-		/* Task can safely be re-inserted now: */
 		node = node->next;
-		task->wake_q.next = NULL;
+		/* pairs with cmpxchg_relaxed() in __wake_q_add() */
+		WRITE_ONCE(task->wake_q.next, NULL);
+		/* Task can safely be re-inserted now. */
 
 		/*
 		 * wake_up_process() executes a full barrier, which pairs with