From: Oleg Nesterov <oleg@redhat.com>
To: Linus Torvalds <torvalds@linux-foundation.org>,
Mani Tofigh via B4 Relay <devnull+mt4022.columbia.edu@kernel.org>
Cc: Breno Leitao <leitao@debian.org>,
Christian Brauner <brauner@kernel.org>,
Mateusz Guzik <mjguzik@gmail.com>,
Alexander Viro <viro@zeniv.linux.org.uk>, Jan Kara <jack@suse.cz>,
linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
tz2294@columbia.edu, stable@vger.kernel.org,
Mani Tofigh <mt4022@columbia.edu>
Subject: Re: [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers
Date: Sun, 22 Nov 2026 18:09:24 +0100 [thread overview]
Message-ID: <awMhxEuMHULvk8SA@redhat.com> (raw)
In-Reply-To: <av3SmRMGDLVAwMzK@redhat.com>
On 11/18, Oleg Nesterov wrote:
>
> On 11/18, Linus Torvalds wrote:
> >
> > Some duplication - with explicit naming of the locking status for the
> > special cases that are called with the lock already held - is much
> > preferable to dynamically doing the locking
>
> OK, this was my plan B ;) I'll try to make a patch this week.
Damn, I am stipid.
So I wrote the patches, but just before sending them, I realized I was wrong.
Just for the record, see the diff I was going to send at the end.
Yes, __wait_event_interruptible_locked() is buggy, but only in that do_wait_intr*()
use list_add_tail() regardless of WQ_FLAG_EXCLUSIVE.
I missed that fact that do_wait_intr*() always return 0 if they drop wq->lock,
and in this case __wait_event_interruptible_locked() will always recheck the
condition with this lock held. So an exclusive waiter can't miss miss a wakeup.
So we only need to change do_wait_intr*() to check WQ_FLAG_EXCLUSIVE and use
__add_wait_queue() if it is set. It doesn't make sense to refactor this code.
Mani, can you resend your patch? perhaps it also makes sense to turn the
"exclusive" arg into wq_flags like the Linus's commit cea5f91daa3ac does.
Oleg.
---
diff --git a/include/linux/wait.h b/include/linux/wait.h
index b3e94c71cd3e..c30e6b2ee231 100644
--- a/include/linux/wait.h
+++ b/include/linux/wait.h
@@ -298,7 +298,6 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags);
* on purpose; we use long where we can return timeout values and int
* otherwise.
*/
-
#define ___wait_event(wq_head, condition, state, wq_flag, ret, cmd) \
({ \
__label__ __out; \
@@ -326,6 +325,35 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags);
__out: __ret; \
})
+/*
+ * Same as ___wait_event(), but called with wq_head->lock held.
+ * @cmd must release and reacquire this lock around schedule().
+ */
+#define ___wait_event_locked(wq_head, condition, state, wq_flag, ret, cmd) \
+({ \
+ __label__ __out; \
+ struct wait_queue_entry __wq_entry; \
+ long __ret = ret; /* explicit shadow */ \
+ \
+ init_wait_entry(&__wq_entry, wq_flag); \
+ for (;;) { \
+ long __int = prepare_to_wait_event_locked(&wq_head, \
+ &__wq_entry, state); \
+ if (condition) \
+ break; \
+ \
+ if (___wait_is_interruptible(state) && __int) { \
+ __ret = __int; \
+ goto __out; \
+ } \
+ \
+ cmd; \
+ } \
+ __set_current_state(TASK_RUNNING); \
+ __remove_wait_queue(&wq_head, &__wq_entry); \
+__out: __ret; \
+})
+
#define __wait_event(wq_head, condition) \
(void)__wait_event_state(wq_head, condition, TASK_UNINTERRUPTIBLE)
@@ -767,26 +795,16 @@ do { \
__ret; \
})
-extern int do_wait_intr(wait_queue_head_t *, wait_queue_entry_t *);
-extern int do_wait_intr_irq(wait_queue_head_t *, wait_queue_entry_t *);
-
-#define __wait_event_interruptible_locked(wq, condition, exclusive, fn) \
-({ \
- int __ret; \
- DEFINE_WAIT(__wait); \
- if (exclusive) \
- __wait.flags |= WQ_FLAG_EXCLUSIVE; \
- do { \
- __ret = fn(&(wq), &__wait); \
- if (__ret) \
- break; \
- } while (!(condition)); \
- __remove_wait_queue(&(wq), &__wait); \
- __set_current_state(TASK_RUNNING); \
- __ret; \
-})
+extern void do_wait_intr(wait_queue_head_t *);
+extern void do_wait_intr_irq(wait_queue_head_t *);
+#define __wait_event_interruptible_locked(wq_head, condition, cmd) \
+ ___wait_event_locked(wq_head, condition, TASK_INTERRUPTIBLE, \
+ 0, 0, cmd(&wq_head))
+#define __wait_event_interruptible_exclusive_locked(wq_head, condition, cmd) \
+ ___wait_event_locked(wq_head, condition, TASK_INTERRUPTIBLE, \
+ WQ_FLAG_EXCLUSIVE, 0, cmd(&wq_head))
/**
* wait_event_interruptible_locked - sleep until a condition gets true
* @wq: the waitqueue to wait on
@@ -810,9 +828,14 @@ extern int do_wait_intr_irq(wait_queue_head_t *, wait_queue_entry_t *);
* The function will return -ERESTARTSYS if it was interrupted by a
* signal and 0 if @condition evaluated to true.
*/
-#define wait_event_interruptible_locked(wq, condition) \
- ((condition) \
- ? 0 : __wait_event_interruptible_locked(wq, condition, 0, do_wait_intr))
+#define wait_event_interruptible_locked(wq_head, condition) \
+({ \
+ int __ret = 0; \
+ if (!(condition)) \
+ __ret = __wait_event_interruptible_locked(wq_head, condition, \
+ do_wait_intr); \
+ __ret; \
+})
/**
* wait_event_interruptible_locked_irq - sleep until a condition gets true
@@ -837,9 +860,14 @@ extern int do_wait_intr_irq(wait_queue_head_t *, wait_queue_entry_t *);
* The function will return -ERESTARTSYS if it was interrupted by a
* signal and 0 if @condition evaluated to true.
*/
-#define wait_event_interruptible_locked_irq(wq, condition) \
- ((condition) \
- ? 0 : __wait_event_interruptible_locked(wq, condition, 0, do_wait_intr_irq))
+#define wait_event_interruptible_locked_irq(wq_head, condition) \
+({ \
+ int __ret = 0; \
+ if (!(condition)) \
+ __ret = __wait_event_interruptible_locked(wq_head, condition, \
+ do_wait_intr_irq); \
+ __ret; \
+})
/**
* wait_event_interruptible_exclusive_locked - sleep exclusively until a condition gets true
@@ -868,9 +896,14 @@ extern int do_wait_intr_irq(wait_queue_head_t *, wait_queue_entry_t *);
* The function will return -ERESTARTSYS if it was interrupted by a
* signal and 0 if @condition evaluated to true.
*/
-#define wait_event_interruptible_exclusive_locked(wq, condition) \
- ((condition) \
- ? 0 : __wait_event_interruptible_locked(wq, condition, 1, do_wait_intr))
+#define wait_event_interruptible_exclusive_locked(wq_head, condition) \
+({ \
+ int __ret = 0; \
+ if (!(condition)) \
+ __ret = __wait_event_interruptible_exclusive_locked(wq_head, condition, \
+ do_wait_intr); \
+ __ret; \
+})
/**
* wait_event_interruptible_exclusive_locked_irq - sleep until a condition gets true
@@ -899,10 +932,14 @@ extern int do_wait_intr_irq(wait_queue_head_t *, wait_queue_entry_t *);
* The function will return -ERESTARTSYS if it was interrupted by a
* signal and 0 if @condition evaluated to true.
*/
-#define wait_event_interruptible_exclusive_locked_irq(wq, condition) \
- ((condition) \
- ? 0 : __wait_event_interruptible_locked(wq, condition, 1, do_wait_intr_irq))
-
+#define wait_event_interruptible_exclusive_locked_irq(wq_head, condition) \
+({ \
+ int __ret = 0; \
+ if (!(condition)) \
+ __ret = __wait_event_interruptible_exclusive_locked(wq_head, condition, \
+ do_wait_intr_irq); \
+ __ret; \
+})
#define __wait_event_killable(wq, condition) \
__wait_event_state(wq, condition, TASK_KILLABLE)
@@ -1218,6 +1255,7 @@ do { \
*/
void prepare_to_wait(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry, int state);
bool prepare_to_wait_exclusive(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry, int state);
+long prepare_to_wait_event_locked(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry, int state);
long prepare_to_wait_event(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry, int state);
void finish_wait(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry);
long wait_woken(struct wait_queue_entry *wq_entry, unsigned mode, long timeout);
diff --git a/kernel/sched/wait.c b/kernel/sched/wait.c
index d0521b54eeb3..b7d03ceb80cd 100644
--- a/kernel/sched/wait.c
+++ b/kernel/sched/wait.c
@@ -269,12 +269,13 @@ void init_wait_entry(struct wait_queue_entry *wq_entry, int flags)
}
EXPORT_SYMBOL(init_wait_entry);
-long prepare_to_wait_event(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry, int state)
+long prepare_to_wait_event_locked(struct wait_queue_head *wq_head,
+ struct wait_queue_entry *wq_entry, int state)
{
- unsigned long flags;
long ret = 0;
- spin_lock_irqsave(&wq_head->lock, flags);
+ lockdep_assert_held(&wq_head->lock);
+
if (signal_pending_state(state, current)) {
/*
* Exclusive waiter must not fail if it was selected by wakeup,
@@ -299,50 +300,37 @@ long prepare_to_wait_event(struct wait_queue_head *wq_head, struct wait_queue_en
}
set_current_state(state);
}
- spin_unlock_irqrestore(&wq_head->lock, flags);
return ret;
}
+EXPORT_SYMBOL(prepare_to_wait_event_locked);
+
+long prepare_to_wait_event(struct wait_queue_head *wq_head,
+ struct wait_queue_entry *wq_entry, int state)
+{
+ guard(spinlock_irqsave)(&wq_head->lock);
+
+ return prepare_to_wait_event_locked(wq_head, wq_entry, state);
+}
EXPORT_SYMBOL(prepare_to_wait_event);
/*
- * Note! These two wait functions are entered with the
- * wait-queue lock held (and interrupts off in the _irq
- * case), so there is no race with testing the wakeup
- * condition in the caller before they add the wait
- * entry to the wake queue.
+ * Note! These two wait functions are entered with the wait-queue
+ * lock held (and interrupts off in the _irq case).
*/
-int do_wait_intr(wait_queue_head_t *wq, wait_queue_entry_t *wait)
+void do_wait_intr(wait_queue_head_t *wq)
{
- if (likely(list_empty(&wait->entry)))
- __add_wait_queue_entry_tail(wq, wait);
-
- set_current_state(TASK_INTERRUPTIBLE);
- if (signal_pending(current))
- return -ERESTARTSYS;
-
spin_unlock(&wq->lock);
schedule();
spin_lock(&wq->lock);
-
- return 0;
}
EXPORT_SYMBOL(do_wait_intr);
-int do_wait_intr_irq(wait_queue_head_t *wq, wait_queue_entry_t *wait)
+void do_wait_intr_irq(wait_queue_head_t *wq)
{
- if (likely(list_empty(&wait->entry)))
- __add_wait_queue_entry_tail(wq, wait);
-
- set_current_state(TASK_INTERRUPTIBLE);
- if (signal_pending(current))
- return -ERESTARTSYS;
-
spin_unlock_irq(&wq->lock);
schedule();
spin_lock_irq(&wq->lock);
-
- return 0;
}
EXPORT_SYMBOL(do_wait_intr_irq);
prev parent reply other threads:[~2026-11-22 17:09 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-11 21:02 [PATCH] pipe: keep per-write wakeups for EPOLLEXCLUSIVE consumers Mani Tofigh via B4 Relay
2026-11-11 23:34 ` Oleg Nesterov
2026-11-12 10:26 ` Oleg Nesterov
2026-11-12 12:56 ` Oleg Nesterov
2026-11-17 13:27 ` Oleg Nesterov
2026-11-17 16:22 ` Linus Torvalds
2026-11-17 17:25 ` Oleg Nesterov
2026-11-17 17:35 ` Linus Torvalds
2026-11-17 17:58 ` Oleg Nesterov
2026-11-17 19:12 ` Linus Torvalds
2026-11-17 19:48 ` Linus Torvalds
2026-11-17 20:41 ` Oleg Nesterov
2026-11-18 13:10 ` Oleg Nesterov
2026-11-18 16:01 ` Linus Torvalds
2026-11-18 16:30 ` Oleg Nesterov
2026-11-18 16:39 ` Linus Torvalds
2026-11-22 17:09 ` Oleg Nesterov [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=awMhxEuMHULvk8SA@redhat.com \
--to=oleg@redhat.com \
--cc=brauner@kernel.org \
--cc=devnull+mt4022.columbia.edu@kernel.org \
--cc=jack@suse.cz \
--cc=leitao@debian.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mjguzik@gmail.com \
--cc=mt4022@columbia.edu \
--cc=stable@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=tz2294@columbia.edu \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox