* [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
2026-11-17 0:14 [PATCH v3 0/5] rust: sync: add WaitQueue infrastructure Danilo Krummrich
@ 2026-11-17 0:14 ` Danilo Krummrich
2026-11-17 0:45 ` FUJITA Tomonori
2026-11-17 14:16 ` Alice Ryhl
2026-11-17 0:14 ` [PATCH v3 2/5] rust: sync: add WaitQueue infrastructure Danilo Krummrich
` (3 subsequent siblings)
4 siblings, 2 replies; 17+ messages in thread
From: Danilo Krummrich @ 2026-11-17 0:14 UTC (permalink / raw)
To: boqun, gary, aliceryhl, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, dakr, tamird, acourbot,
viro, brauner, jack
Cc: linux-kernel, rust-for-linux, linux-fsdevel
Add a safe wrapper around schedule_timeout() that takes and returns
Delta<Jiffy>, treating negative durations as zero.
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
rust/kernel/task.rs | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/rust/kernel/task.rs b/rust/kernel/task.rs
index 3336df493dec..2fc0607ba9b7 100644
--- a/rust/kernel/task.rs
+++ b/rust/kernel/task.rs
@@ -10,6 +10,10 @@
pid_namespace::PidNamespace,
prelude::*,
sync::aref::ARef,
+ time::{
+ Delta,
+ Jiffy, //
+ },
types::{NotThreadSafe, Opaque},
};
use core::{
@@ -20,6 +24,25 @@
/// A sentinel value used for infinite timeouts.
pub const MAX_SCHEDULE_TIMEOUT: c_long = c_long::MAX;
+/// Sleeps for the given timeout, or until woken.
+///
+/// Returns the remaining time, or zero if the timeout expired. The task state should be set before
+/// calling this.
+///
+/// A negative timeout is treated as zero.
+///
+/// A timeout of [`MAX_SCHEDULE_TIMEOUT`] jiffies means an indefinite wait. In that case, the
+/// returned duration is also [`MAX_SCHEDULE_TIMEOUT`] jiffies.
+#[inline]
+pub fn schedule_timeout(timeout: Delta<Jiffy>) -> Delta<Jiffy> {
+ let timeout = timeout.as_jiffies().max(0);
+
+ // SAFETY: `schedule_timeout()` is always safe to call.
+ let remaining = unsafe { bindings::schedule_timeout(timeout) };
+
+ Delta::from_jiffies(remaining)
+}
+
/// Bitmask for tasks that are sleeping in an interruptible state.
pub const TASK_INTERRUPTIBLE: c_int = bindings::TASK_INTERRUPTIBLE as c_int;
/// Bitmask for tasks that are sleeping in an uninterruptible state.
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* Re: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
2026-11-17 0:14 ` [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper Danilo Krummrich
@ 2026-11-17 0:45 ` FUJITA Tomonori
2026-11-17 11:40 ` Gary Guo
2026-11-17 14:16 ` Alice Ryhl
1 sibling, 1 reply; 17+ messages in thread
From: FUJITA Tomonori @ 2026-11-17 0:45 UTC (permalink / raw)
To: dakr
Cc: boqun, gary, aliceryhl, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, tamird, acourbot, viro,
brauner, jack, linux-kernel, rust-for-linux, linux-fsdevel
On Tue, 17 Nov 2026 01:14:42 +0100
Danilo Krummrich <dakr@kernel.org> wrote:
> +pub fn schedule_timeout(timeout: Delta<Jiffy>) -> Delta<Jiffy> {
> + let timeout = timeout.as_jiffies().max(0);
In earlier reviews of my patches, Gary asked me to use the function
syntax or clamp() for this instead of the method syntax [1][2]:
let timeout = isize::max(timeout.as_jiffies(), 0);
[1] https://lore.kernel.org/rust-for-linux/DLMME4OBH3P8.3Q6UK95T2WGI@garyguo.net/
[2] https://lore.kernel.org/rust-for-linux/DLUDG2TH77RI.ZQ0T5MH7Q573@garyguo.net/
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
2026-11-17 0:45 ` FUJITA Tomonori
@ 2026-11-17 11:40 ` Gary Guo
2026-11-17 11:56 ` Danilo Krummrich
0 siblings, 1 reply; 17+ messages in thread
From: Gary Guo @ 2026-11-17 11:40 UTC (permalink / raw)
To: FUJITA Tomonori, dakr
Cc: boqun, gary, aliceryhl, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, tamird, acourbot, viro,
brauner, jack, linux-kernel, rust-for-linux, linux-fsdevel
On Tue Nov 17, 2026 at 12:45 AM GMT, FUJITA Tomonori wrote:
> On Tue, 17 Nov 2026 01:14:42 +0100
> Danilo Krummrich <dakr@kernel.org> wrote:
>
>> +pub fn schedule_timeout(timeout: Delta<Jiffy>) -> Delta<Jiffy> {
>> + let timeout = timeout.as_jiffies().max(0);
>
> In earlier reviews of my patches, Gary asked me to use the function
> syntax or clamp() for this instead of the method syntax [1][2]:
>
> let timeout = isize::max(timeout.as_jiffies(), 0);
Yes, please do. I'm okay with either `isize::max` or `cmp::max`.
Best,
Gary
>
> [1] https://lore.kernel.org/rust-for-linux/DLMME4OBH3P8.3Q6UK95T2WGI@garyguo.net/
> [2] https://lore.kernel.org/rust-for-linux/DLUDG2TH77RI.ZQ0T5MH7Q573@garyguo.net/
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
2026-11-17 11:40 ` Gary Guo
@ 2026-11-17 11:56 ` Danilo Krummrich
2026-11-17 12:02 ` Gary Guo
0 siblings, 1 reply; 17+ messages in thread
From: Danilo Krummrich @ 2026-11-17 11:56 UTC (permalink / raw)
To: Gary Guo
Cc: FUJITA Tomonori, boqun, aliceryhl, lyude, daniel.almeida, work,
ojeda, bjorn3_gh, lossin, a.hindborg, tmgross, tamird, acourbot,
viro, brauner, jack, linux-kernel, rust-for-linux, linux-fsdevel
On Tue Nov 17, 2026 at 12:40 PM CET, Gary Guo wrote:
> On Tue Nov 17, 2026 at 12:45 AM GMT, FUJITA Tomonori wrote:
>> On Tue, 17 Nov 2026 01:14:42 +0100
>> Danilo Krummrich <dakr@kernel.org> wrote:
>>
>>> +pub fn schedule_timeout(timeout: Delta<Jiffy>) -> Delta<Jiffy> {
>>> + let timeout = timeout.as_jiffies().max(0);
>>
>> In earlier reviews of my patches, Gary asked me to use the function
>> syntax or clamp() for this instead of the method syntax [1][2]:
>>
>> let timeout = isize::max(timeout.as_jiffies(), 0);
>
> Yes, please do. I'm okay with either `isize::max` or `cmp::max`.
The examples in [1][2] both seemed to be able to avoid a cast with this, which
isn't the case for this patch. There also doesn't seem to be any ambiguity in
the extression used in this patch. So I don't see why changing it would be
better.
Thanks,
Danilo
>> [1] https://lore.kernel.org/rust-for-linux/DLMME4OBH3P8.3Q6UK95T2WGI@garyguo.net/
>> [2] https://lore.kernel.org/rust-for-linux/DLUDG2TH77RI.ZQ0T5MH7Q573@garyguo.net/
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
2026-11-17 11:56 ` Danilo Krummrich
@ 2026-11-17 12:02 ` Gary Guo
2026-11-17 13:14 ` Danilo Krummrich
0 siblings, 1 reply; 17+ messages in thread
From: Gary Guo @ 2026-11-17 12:02 UTC (permalink / raw)
To: Danilo Krummrich, Gary Guo
Cc: FUJITA Tomonori, boqun, aliceryhl, lyude, daniel.almeida, work,
ojeda, bjorn3_gh, lossin, a.hindborg, tmgross, tamird, acourbot,
viro, brauner, jack, linux-kernel, rust-for-linux, linux-fsdevel
On Tue Nov 17, 2026 at 11:56 AM GMT, Danilo Krummrich wrote:
> On Tue Nov 17, 2026 at 12:40 PM CET, Gary Guo wrote:
>> On Tue Nov 17, 2026 at 12:45 AM GMT, FUJITA Tomonori wrote:
>>> On Tue, 17 Nov 2026 01:14:42 +0100
>>> Danilo Krummrich <dakr@kernel.org> wrote:
>>>
>>>> +pub fn schedule_timeout(timeout: Delta<Jiffy>) -> Delta<Jiffy> {
>>>> + let timeout = timeout.as_jiffies().max(0);
>>>
>>> In earlier reviews of my patches, Gary asked me to use the function
>>> syntax or clamp() for this instead of the method syntax [1][2]:
>>>
>>> let timeout = isize::max(timeout.as_jiffies(), 0);
>>
>> Yes, please do. I'm okay with either `isize::max` or `cmp::max`.
>
> The examples in [1][2] both seemed to be able to avoid a cast with this, which
> isn't the case for this patch. There also doesn't seem to be any ambiguity in
> the extression used in this patch. So I don't see why changing it would be
> better.
The ambiguity is "the maximum value returned will be 0" and "the minimum value
returned will be 0". The answer is always the latter but I'd avoid the mental
gymnastics.
Best,
Gary
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
2026-11-17 12:02 ` Gary Guo
@ 2026-11-17 13:14 ` Danilo Krummrich
0 siblings, 0 replies; 17+ messages in thread
From: Danilo Krummrich @ 2026-11-17 13:14 UTC (permalink / raw)
To: Gary Guo
Cc: FUJITA Tomonori, boqun, aliceryhl, lyude, daniel.almeida, work,
ojeda, bjorn3_gh, lossin, a.hindborg, tmgross, tamird, acourbot,
viro, brauner, jack, linux-kernel, rust-for-linux, linux-fsdevel
On Tue Nov 17, 2026 at 1:02 PM CET, Gary Guo wrote:
> On Tue Nov 17, 2026 at 11:56 AM GMT, Danilo Krummrich wrote:
>> On Tue Nov 17, 2026 at 12:40 PM CET, Gary Guo wrote:
>>> On Tue Nov 17, 2026 at 12:45 AM GMT, FUJITA Tomonori wrote:
>>>> On Tue, 17 Nov 2026 01:14:42 +0100
>>>> Danilo Krummrich <dakr@kernel.org> wrote:
>>>>
>>>>> +pub fn schedule_timeout(timeout: Delta<Jiffy>) -> Delta<Jiffy> {
>>>>> + let timeout = timeout.as_jiffies().max(0);
>>>>
>>>> In earlier reviews of my patches, Gary asked me to use the function
>>>> syntax or clamp() for this instead of the method syntax [1][2]:
>>>>
>>>> let timeout = isize::max(timeout.as_jiffies(), 0);
>>>
>>> Yes, please do. I'm okay with either `isize::max` or `cmp::max`.
>>
>> The examples in [1][2] both seemed to be able to avoid a cast with this, which
>> isn't the case for this patch. There also doesn't seem to be any ambiguity in
>> the extression used in this patch. So I don't see why changing it would be
>> better.
>
> The ambiguity is "the maximum value returned will be 0" and "the minimum value
> returned will be 0". The answer is always the latter but I'd avoid the mental
> gymnastics.
To me clearly both
a.max(b) (1)
max(a, b) (2)
read as the maximum of a and b.
IIUC, what you are saying is that people may read (1) as
a.at_max(b)
which would indeed be equivalent to min(a, b).
If this is indeed commonly the case I'd suggest to raise this to become a common
rule (ideally with a lint), such that we avoid this to be discussed in reviews.
Personally, I prefer (1) for the expression above, but I don't care too much, so
I'm fine to change it. (In case I don't need to respin I'll change it on apply.)
Thanks,
Danilo
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
2026-11-17 0:14 ` [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper Danilo Krummrich
2026-11-17 0:45 ` FUJITA Tomonori
@ 2026-11-17 14:16 ` Alice Ryhl
1 sibling, 0 replies; 17+ messages in thread
From: Alice Ryhl @ 2026-11-17 14:16 UTC (permalink / raw)
To: Danilo Krummrich
Cc: Alice Ryhl, boqun, gary, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, tamird, acourbot, viro,
brauner, jack, linux-kernel, rust-for-linux, linux-fsdevel
On Tue, Nov 17, 2026 at 01:14:42AM +0100, Danilo Krummrich wrote:
> Add a safe wrapper around schedule_timeout() that takes and returns
> Delta<Jiffy>, treating negative durations as zero.
>
> Acked-by: Miguel Ojeda <ojeda@kernel.org>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v3 2/5] rust: sync: add WaitQueue infrastructure
2026-11-17 0:14 [PATCH v3 0/5] rust: sync: add WaitQueue infrastructure Danilo Krummrich
2026-11-17 0:14 ` [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper Danilo Krummrich
@ 2026-11-17 0:14 ` Danilo Krummrich
2026-11-17 11:59 ` Gary Guo
2026-11-17 14:16 ` Alice Ryhl
2026-11-17 0:14 ` [PATCH v3 3/5] rust: sync: convert CondVar and PollCondVar to use WaitQueue Danilo Krummrich
` (2 subsequent siblings)
4 siblings, 2 replies; 17+ messages in thread
From: Danilo Krummrich @ 2026-11-17 0:14 UTC (permalink / raw)
To: boqun, gary, aliceryhl, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, dakr, tamird, acourbot,
viro, brauner, jack
Cc: linux-kernel, rust-for-linux, linux-fsdevel
Implement a wait queue wrapping the kernel's struct wait_queue_head,
with wait_event()-style methods that take a condition closure directly.
The API mirrors the C wait_event() family:
- wait_event()
- wait_event_interruptible()
- wait_event_timeout()
- wait_event_interruptible_timeout()
- wake_up() / wake_up_all() / wake_up_sync()
The interruptible variant returns Result<(), Interrupted>, the timeout
variant Result<(), TimedOut>, and the combined variant Result<(),
WaitError>. These errors map to ERESTARTSYS (signal) or ETIMEDOUT
(timeout).
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
rust/kernel/Kconfig.test | 10 +
rust/kernel/sync.rs | 8 +
rust/kernel/sync/wait.rs | 549 +++++++++++++++++++++++++++++++++++++++
3 files changed, 567 insertions(+)
create mode 100644 rust/kernel/sync/wait.rs
diff --git a/rust/kernel/Kconfig.test b/rust/kernel/Kconfig.test
index 3bf4194f28c7..7a1a464b6b29 100644
--- a/rust/kernel/Kconfig.test
+++ b/rust/kernel/Kconfig.test
@@ -104,4 +104,14 @@ config RUST_TIME_KUNIT_TEST
If unsure, say N.
+config RUST_WAITQUEUE_KUNIT_TEST
+ bool "KUnit tests for the Rust wait queue API" if !KUNIT_ALL_TESTS
+ default KUNIT_ALL_TESTS
+ help
+ This option enables KUnit tests for the Rust wait queue API.
+ These are only for development and testing, not for regular
+ kernel use cases.
+
+ If unsure, say N.
+
endif
diff --git a/rust/kernel/sync.rs b/rust/kernel/sync.rs
index ee372c048a8a..5f7ffd828259 100644
--- a/rust/kernel/sync.rs
+++ b/rust/kernel/sync.rs
@@ -19,6 +19,7 @@
mod refcount;
mod set_once;
pub mod srcu;
+mod wait;
pub use arc::{Arc, ArcBorrow, UniqueArc};
pub use completion::Completion;
@@ -41,3 +42,10 @@
pub use refcount::Refcount;
pub use set_once::SetOnce;
pub use srcu::Srcu;
+pub use wait::{
+ new_waitqueue,
+ Interrupted,
+ TimedOut,
+ WaitError,
+ WaitQueue, //
+};
diff --git a/rust/kernel/sync/wait.rs b/rust/kernel/sync/wait.rs
new file mode 100644
index 000000000000..abbd5466ff44
--- /dev/null
+++ b/rust/kernel/sync/wait.rs
@@ -0,0 +1,549 @@
+// SPDX-License-Identifier: GPL-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+
+//! Wait queue.
+//!
+//! C header: [`include/linux/wait.h`](srctree/include/linux/wait.h)
+
+use super::LockClassKey;
+use crate::{
+ prelude::*,
+ str::CStr,
+ task,
+ time::{
+ Delta,
+ Jiffy, //
+ },
+ types::Opaque, //
+};
+
+use core::{
+ pin::Pin,
+ ptr, //
+};
+
+/// Creates a [`WaitQueue`] initialiser with the given name and a newly-created lock class.
+#[macro_export]
+macro_rules! new_waitqueue {
+ ($($name:literal)?) => {
+ $crate::sync::WaitQueue::new(
+ $crate::optional_name!($($name)?),
+ $crate::static_lock_class!(),
+ )
+ };
+}
+pub use new_waitqueue;
+
+/// Exposes the kernel's [`struct wait_queue_head`] as a Rust wait queue.
+///
+/// A `WaitQueue` allows a thread to sleep until a caller-supplied condition becomes true,
+/// re-checking the condition on each wake-up. This matches the C `wait_event()` family of macros.
+///
+/// For waiting with a lock guard (the condition variable pattern), use [`CondVar`](super::CondVar)
+/// instead.
+///
+/// Instances of `WaitQueue` need a lock class and to be pinned. The recommended way to create such
+/// instances is with the [`pin_init!`] and [`new_waitqueue!`] macros.
+///
+/// # Examples
+///
+/// ```
+/// use kernel::sync::{
+/// atomic::{
+/// Atomic,
+/// Relaxed,
+/// },
+/// new_waitqueue,
+/// WaitQueue,
+/// };
+///
+/// #[pin_data]
+/// pub struct Example {
+/// value: Atomic<i32>,
+/// #[pin]
+/// queue: WaitQueue,
+/// }
+///
+/// fn wait_for_value(e: &Example, v: i32) {
+/// e.queue.wait_event(|| e.value.load(Relaxed) == v);
+/// }
+///
+/// fn set_value(e: &Example, v: i32) {
+/// e.value.store(v, Relaxed);
+/// e.queue.wake_up();
+/// }
+/// ```
+///
+/// [`struct wait_queue_head`]: srctree/include/linux/wait.h
+#[pin_data]
+pub struct WaitQueue {
+ #[pin]
+ wait_queue_head: Opaque<bindings::wait_queue_head>,
+}
+
+// SAFETY: `WaitQueue` only uses a `struct wait_queue_head`, which is safe to use on any thread.
+unsafe impl Send for WaitQueue {}
+
+// SAFETY: `WaitQueue` only uses a `struct wait_queue_head`, which is safe to use on multiple
+// threads concurrently.
+unsafe impl Sync for WaitQueue {}
+
+impl WaitQueue {
+ /// Constructs a new wait queue initialiser.
+ pub fn new(name: &'static CStr, key: Pin<&'static LockClassKey>) -> impl PinInit<Self> {
+ pin_init!(Self {
+ // SAFETY: `slot` is valid while the closure is called and both `name` and `key` have
+ // static lifetimes so they live indefinitely.
+ wait_queue_head <- Opaque::ffi_init(|slot| unsafe {
+ bindings::__init_waitqueue_head(slot, name.as_char_ptr(), key.as_ptr())
+ }),
+ })
+ }
+
+ /// Returns a raw pointer to the underlying `wait_queue_head`.
+ #[inline]
+ pub fn as_raw(&self) -> *mut bindings::wait_queue_head {
+ self.wait_queue_head.get()
+ }
+
+ /// Sleeps until the condition returns `true`.
+ ///
+ /// The condition is checked before each sleep and after each wake-up. The wait is
+ /// uninterruptible.
+ #[track_caller]
+ #[inline]
+ pub fn wait_event<F: FnMut() -> bool>(&self, condition: F) {
+ // An uninterruptible wait without a timeout cannot fail.
+ let _ = self.wait_event_timeout_internal(
+ task::TASK_UNINTERRUPTIBLE,
+ condition,
+ Delta::from_jiffies(task::MAX_SCHEDULE_TIMEOUT),
+ );
+ }
+
+ /// Sleeps until the condition returns `true` or a signal is received.
+ ///
+ /// Returns `Ok(())` when the condition is met, or [`Interrupted`] if interrupted by a signal.
+ #[track_caller]
+ #[inline]
+ pub fn wait_event_interruptible<F: FnMut() -> bool>(
+ &self,
+ condition: F,
+ ) -> Result<(), Interrupted> {
+ // Without a timeout, interruption is the only possible error.
+ self.wait_event_timeout_internal(
+ task::TASK_INTERRUPTIBLE,
+ condition,
+ Delta::from_jiffies(task::MAX_SCHEDULE_TIMEOUT),
+ )
+ .map_err(|_| Interrupted)
+ }
+
+ /// Sleeps until the condition returns `true` or the timeout expires.
+ ///
+ /// Returns `Ok(())` when the condition is met, or [`TimedOut`] if the timeout elapsed first.
+ /// A negative timeout is treated as zero.
+ #[track_caller]
+ #[inline]
+ pub fn wait_event_timeout<F: FnMut() -> bool>(
+ &self,
+ condition: F,
+ timeout: Delta<Jiffy>,
+ ) -> Result<(), TimedOut> {
+ // An uninterruptible wait can only fail by timing out.
+ self.wait_event_timeout_internal(task::TASK_UNINTERRUPTIBLE, condition, timeout)
+ .map_err(|_| TimedOut)
+ }
+
+ /// Sleeps until the condition returns `true`, a signal is received, or the timeout expires.
+ ///
+ /// Returns `Ok(())` when the condition is met, or [`WaitError`] on signal or timeout.
+ /// A negative timeout is treated as zero.
+ #[track_caller]
+ #[inline]
+ pub fn wait_event_interruptible_timeout<F: FnMut() -> bool>(
+ &self,
+ condition: F,
+ timeout: Delta<Jiffy>,
+ ) -> Result<(), WaitError> {
+ self.wait_event_timeout_internal(task::TASK_INTERRUPTIBLE, condition, timeout)
+ }
+
+ #[track_caller]
+ fn wait_event_timeout_internal<F: FnMut() -> bool>(
+ &self,
+ wait_state: c_int,
+ mut condition: F,
+ timeout: Delta<Jiffy>,
+ ) -> Result<(), WaitError> {
+ task::might_sleep();
+
+ if condition() {
+ return Ok(());
+ }
+
+ if timeout <= Delta::ZERO {
+ return Err(TimedOut.into());
+ }
+
+ let wait = Opaque::<bindings::wait_queue_entry>::uninit();
+
+ // SAFETY: `wait` points to valid memory.
+ unsafe { bindings::init_wait(wait.get()) };
+
+ let mut remaining = timeout;
+
+ let result = loop {
+ // SAFETY: Both pointers are valid, and `wait` was initialised by `init_wait()` above.
+ let ret = unsafe {
+ bindings::prepare_to_wait_event(self.wait_queue_head.get(), wait.get(), wait_state)
+ };
+
+ if condition() {
+ break Ok(());
+ }
+
+ if ret != 0 {
+ break Err(Interrupted.into());
+ }
+
+ remaining = task::schedule_timeout(remaining);
+
+ if condition() {
+ break Ok(());
+ }
+
+ if remaining == Delta::ZERO {
+ break Err(TimedOut.into());
+ }
+ };
+
+ // SAFETY: Both pointers are valid, and `wait` is no longer used after this call.
+ unsafe { bindings::finish_wait(self.wait_queue_head.get(), wait.get()) };
+
+ result
+ }
+
+ /// Performs a single non-exclusive prepare-to-wait / finish-wait cycle, calling `schedule_fn`
+ /// in between. Non-exclusive waiters are all woken by [`wake_up()`](Self::wake_up).
+ #[inline]
+ #[expect(unused)]
+ pub(super) fn wait_once<F, R>(&self, wait_state: c_int, schedule_fn: F) -> R
+ where
+ F: FnOnce() -> R,
+ {
+ self.wait_once_internal(wait_state, false, schedule_fn)
+ }
+
+ /// Performs a single exclusive prepare-to-wait / finish-wait cycle, calling `schedule_fn`
+ /// in between.
+ #[inline]
+ #[expect(unused)]
+ pub(super) fn wait_once_exclusive<F, R>(&self, wait_state: c_int, schedule_fn: F) -> R
+ where
+ F: FnOnce() -> R,
+ {
+ self.wait_once_internal(wait_state, true, schedule_fn)
+ }
+
+ fn wait_once_internal<F, R>(&self, wait_state: c_int, exclusive: bool, schedule_fn: F) -> R
+ where
+ F: FnOnce() -> R,
+ {
+ let wait = Opaque::<bindings::wait_queue_entry>::uninit();
+
+ // SAFETY: `wait` points to valid memory.
+ unsafe { bindings::init_wait(wait.get()) };
+
+ // SAFETY: Both `wait` and `wait_queue_head` point to valid memory.
+ unsafe {
+ if exclusive {
+ bindings::prepare_to_wait_exclusive(
+ self.wait_queue_head.get(),
+ wait.get(),
+ wait_state,
+ );
+ } else {
+ bindings::prepare_to_wait(self.wait_queue_head.get(), wait.get(), wait_state);
+ }
+ };
+
+ let ret = schedule_fn();
+
+ // SAFETY: Both `wait` and `wait_queue_head` point to valid memory.
+ unsafe { bindings::finish_wait(self.wait_queue_head.get(), wait.get()) };
+
+ ret
+ }
+
+ /// Wakes up waiters.
+ ///
+ /// Wakes all non-exclusive waiters and one exclusive waiter, if any. Matches C's `wake_up()`.
+ #[inline]
+ pub fn wake_up(&self) {
+ // SAFETY: `wait_queue_head` points to valid memory.
+ unsafe {
+ bindings::__wake_up(
+ self.wait_queue_head.get(),
+ bindings::TASK_NORMAL,
+ 1,
+ ptr::null_mut(),
+ )
+ };
+ }
+
+ /// Wakes up all waiters.
+ ///
+ /// Wakes all non-exclusive and all exclusive waiters, if any.
+ /// Matches C's `wake_up_all()`.
+ #[inline]
+ pub fn wake_up_all(&self) {
+ // SAFETY: `wait_queue_head` points to valid memory.
+ unsafe {
+ bindings::__wake_up(
+ self.wait_queue_head.get(),
+ bindings::TASK_NORMAL,
+ 0,
+ ptr::null_mut(),
+ )
+ };
+ }
+
+ /// Like [`wake_up()`](Self::wake_up), but hints to the scheduler that the current task
+ /// expects to sleep soon. Matches C's `wake_up_sync()`.
+ #[inline]
+ pub fn wake_up_sync(&self) {
+ // SAFETY: `wait_queue_head` points to valid memory.
+ unsafe { bindings::__wake_up_sync(self.wait_queue_head.get(), bindings::TASK_NORMAL) };
+ }
+
+ /// Wakes up all waiters and clears poll registrations.
+ ///
+ /// Used when a wait queue is about to be freed, to ensure epoll items are properly removed.
+ /// Matches C's `wake_up_pollfree()`.
+ #[inline]
+ #[expect(unused)]
+ pub(super) fn wake_up_pollfree(&self) {
+ // SAFETY: `wait_queue_head` points to valid memory.
+ unsafe { bindings::__wake_up_pollfree(self.wait_queue_head.get()) };
+ }
+}
+
+/// A wait was interrupted by a signal.
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub struct Interrupted;
+
+/// A wait timed out before its condition was met.
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub struct TimedOut;
+
+impl From<Interrupted> for Error {
+ #[inline]
+ fn from(_: Interrupted) -> Error {
+ ERESTARTSYS
+ }
+}
+
+impl From<TimedOut> for Error {
+ #[inline]
+ fn from(_: TimedOut) -> Error {
+ ETIMEDOUT
+ }
+}
+
+/// Error returned by [`WaitQueue`] wait functions.
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub enum WaitError {
+ /// Interrupted by a signal.
+ Signal,
+ /// The timeout elapsed without the condition being met.
+ Timeout,
+}
+
+impl From<Interrupted> for WaitError {
+ #[inline]
+ fn from(_: Interrupted) -> Self {
+ Self::Signal
+ }
+}
+
+impl From<TimedOut> for WaitError {
+ #[inline]
+ fn from(_: TimedOut) -> Self {
+ Self::Timeout
+ }
+}
+
+impl From<WaitError> for Error {
+ #[inline]
+ fn from(e: WaitError) -> Error {
+ match e {
+ WaitError::Signal => ERESTARTSYS,
+ WaitError::Timeout => ETIMEDOUT,
+ }
+ }
+}
+
+#[cfg(CONFIG_RUST_WAITQUEUE_KUNIT_TEST)]
+#[macros::kunit_tests(rust_waitqueue)]
+mod tests {
+ use super::*;
+ use crate::{
+ sync::{
+ atomic::{
+ Atomic,
+ Relaxed, //
+ },
+ Arc,
+ },
+ time::delay::fsleep,
+ workqueue,
+ };
+
+ #[pin_data]
+ struct State {
+ value: Atomic<i32>,
+ #[pin]
+ wq: WaitQueue,
+ }
+
+ impl State {
+ fn new() -> Result<Arc<Self>> {
+ Arc::pin_init(
+ pin_init!(Self {
+ value: Atomic::new(0),
+ wq <- new_waitqueue!(),
+ }),
+ GFP_KERNEL,
+ )
+ }
+ }
+
+ #[test]
+ fn wait_event_from_work() {
+ let s = State::new().unwrap();
+
+ let s2 = s.clone();
+ workqueue::system_dfl()
+ .try_spawn(GFP_KERNEL, move || {
+ s2.value.store(1, Relaxed);
+ s2.wq.wake_up();
+ })
+ .unwrap();
+
+ s.wq.wait_event(|| s.value.load(Relaxed) == 1);
+ assert_eq!(s.value.load(Relaxed), 1);
+ }
+
+ #[test]
+ fn wait_event_condition_already_true() {
+ let s = State::new().unwrap();
+ let mut calls = 0;
+
+ s.wq.wait_event(|| {
+ calls += 1;
+ true
+ });
+
+ assert_eq!(calls, 1);
+ }
+
+ #[test]
+ fn wait_event_condition_not_yet_met() {
+ let s = State::new().unwrap();
+
+ let s2 = s.clone();
+ workqueue::system_dfl()
+ .try_spawn(GFP_KERNEL, move || {
+ s2.value.store(1, Relaxed);
+ s2.wq.wake_up_all();
+
+ fsleep(Delta::from_millis(50));
+
+ s2.value.store(2, Relaxed);
+ s2.wq.wake_up_all();
+ })
+ .unwrap();
+
+ s.wq.wait_event(|| s.value.load(Relaxed) == 2);
+ assert_eq!(s.value.load(Relaxed), 2);
+ }
+
+ #[test]
+ fn wait_event_timeout_expires() {
+ let s = State::new().unwrap();
+
+ let ret =
+ s.wq.wait_event_timeout(|| s.value.load(Relaxed) == 1, Delta::from_jiffies(1));
+ assert_eq!(ret, Err(TimedOut));
+ }
+
+ #[test]
+ fn wait_event_does_not_recheck_success() {
+ enum WaitKind {
+ Uninterruptible,
+ Interruptible,
+ Timeout,
+ InterruptibleTimeout,
+ }
+
+ let s = State::new().unwrap();
+
+ for kind in [
+ WaitKind::Uninterruptible,
+ WaitKind::Interruptible,
+ WaitKind::Timeout,
+ WaitKind::InterruptibleTimeout,
+ ] {
+ let mut calls = 0;
+ let condition = || {
+ // Bypass the initial fast path to exercise the wait loop.
+ calls += 1;
+ calls > 1
+ };
+
+ let ret = match kind {
+ WaitKind::Uninterruptible => {
+ s.wq.wait_event(condition);
+ Ok(())
+ }
+ WaitKind::Interruptible => {
+ s.wq.wait_event_interruptible(condition).map_err(Into::into)
+ }
+ WaitKind::Timeout => {
+ s.wq.wait_event_timeout(condition, Delta::from_jiffies(1))
+ .map_err(Into::into)
+ }
+ WaitKind::InterruptibleTimeout => {
+ s.wq.wait_event_interruptible_timeout(condition, Delta::from_jiffies(1))
+ }
+ };
+
+ assert_eq!(ret, Ok(()));
+ assert_eq!(calls, 2);
+ }
+ }
+
+ #[test]
+ fn wait_event_zero_timeout() {
+ let s = State::new().unwrap();
+
+ for interruptible in [false, true] {
+ let mut calls = 0;
+ let condition = || {
+ calls += 1;
+ false
+ };
+
+ let ret = if interruptible {
+ s.wq.wait_event_interruptible_timeout(condition, Delta::ZERO)
+ } else {
+ s.wq.wait_event_timeout(condition, Delta::ZERO)
+ .map_err(Into::into)
+ };
+
+ assert_eq!(ret, Err(WaitError::Timeout));
+ assert_eq!(calls, 1);
+ }
+ }
+}
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* Re: [PATCH v3 2/5] rust: sync: add WaitQueue infrastructure
2026-11-17 0:14 ` [PATCH v3 2/5] rust: sync: add WaitQueue infrastructure Danilo Krummrich
@ 2026-11-17 11:59 ` Gary Guo
2026-11-17 14:16 ` Alice Ryhl
1 sibling, 0 replies; 17+ messages in thread
From: Gary Guo @ 2026-11-17 11:59 UTC (permalink / raw)
To: Danilo Krummrich, boqun, gary, aliceryhl, lyude, daniel.almeida,
work, ojeda, bjorn3_gh, lossin, a.hindborg, tmgross, tamird,
acourbot, viro, brauner, jack
Cc: linux-kernel, rust-for-linux, linux-fsdevel
On Tue Nov 17, 2026 at 12:14 AM GMT, Danilo Krummrich wrote:
> Implement a wait queue wrapping the kernel's struct wait_queue_head,
> with wait_event()-style methods that take a condition closure directly.
>
> The API mirrors the C wait_event() family:
>
> - wait_event()
> - wait_event_interruptible()
> - wait_event_timeout()
> - wait_event_interruptible_timeout()
> - wake_up() / wake_up_all() / wake_up_sync()
>
> The interruptible variant returns Result<(), Interrupted>, the timeout
> variant Result<(), TimedOut>, and the combined variant Result<(),
> WaitError>. These errors map to ERESTARTSYS (signal) or ETIMEDOUT
> (timeout).
>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
> rust/kernel/Kconfig.test | 10 +
> rust/kernel/sync.rs | 8 +
> rust/kernel/sync/wait.rs | 549 +++++++++++++++++++++++++++++++++++++++
> 3 files changed, 567 insertions(+)
> create mode 100644 rust/kernel/sync/wait.rs
>
> diff --git a/rust/kernel/Kconfig.test b/rust/kernel/Kconfig.test
> index 3bf4194f28c7..7a1a464b6b29 100644
> --- a/rust/kernel/Kconfig.test
> +++ b/rust/kernel/Kconfig.test
> @@ -104,4 +104,14 @@ config RUST_TIME_KUNIT_TEST
>
> If unsure, say N.
>
> +config RUST_WAITQUEUE_KUNIT_TEST
> + bool "KUnit tests for the Rust wait queue API" if !KUNIT_ALL_TESTS
> + default KUNIT_ALL_TESTS
> + help
> + This option enables KUnit tests for the Rust wait queue API.
> + These are only for development and testing, not for regular
> + kernel use cases.
> +
> + If unsure, say N.
> +
> endif
> diff --git a/rust/kernel/sync.rs b/rust/kernel/sync.rs
> index ee372c048a8a..5f7ffd828259 100644
> --- a/rust/kernel/sync.rs
> +++ b/rust/kernel/sync.rs
> @@ -19,6 +19,7 @@
> mod refcount;
> mod set_once;
> pub mod srcu;
> +mod wait;
>
> pub use arc::{Arc, ArcBorrow, UniqueArc};
> pub use completion::Completion;
> @@ -41,3 +42,10 @@
> pub use refcount::Refcount;
> pub use set_once::SetOnce;
> pub use srcu::Srcu;
> +pub use wait::{
> + new_waitqueue,
> + Interrupted,
> + TimedOut,
> + WaitError,
> + WaitQueue, //
> +};
> diff --git a/rust/kernel/sync/wait.rs b/rust/kernel/sync/wait.rs
> new file mode 100644
> index 000000000000..abbd5466ff44
> --- /dev/null
> +++ b/rust/kernel/sync/wait.rs
> @@ -0,0 +1,549 @@
> +// SPDX-License-Identifier: GPL-2.0
> +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
> +
> +//! Wait queue.
> +//!
> +//! C header: [`include/linux/wait.h`](srctree/include/linux/wait.h)
> +
> +use super::LockClassKey;
> +use crate::{
> + prelude::*,
> + str::CStr,
> + task,
> + time::{
> + Delta,
> + Jiffy, //
> + },
> + types::Opaque, //
> +};
> +
> +use core::{
> + pin::Pin,
> + ptr, //
> +};
> +
> +/// Creates a [`WaitQueue`] initialiser with the given name and a newly-created lock class.
> +#[macro_export]
> +macro_rules! new_waitqueue {
> + ($($name:literal)?) => {
> + $crate::sync::WaitQueue::new(
> + $crate::optional_name!($($name)?),
> + $crate::static_lock_class!(),
> + )
> + };
> +}
> +pub use new_waitqueue;
> +
> +/// Exposes the kernel's [`struct wait_queue_head`] as a Rust wait queue.
The first paragraph is rendered as rustdoc short item description so please
have something more descriptive and leave the C equivalence to other pragraphs.
Best,
Gary
> +///
> +/// A `WaitQueue` allows a thread to sleep until a caller-supplied condition becomes true,
> +/// re-checking the condition on each wake-up. This matches the C `wait_event()` family of macros.
> +///
> +/// For waiting with a lock guard (the condition variable pattern), use [`CondVar`](super::CondVar)
> +/// instead.
> +///
> +/// Instances of `WaitQueue` need a lock class and to be pinned. The recommended way to create such
> +/// instances is with the [`pin_init!`] and [`new_waitqueue!`] macros.
> +///
> +/// # Examples
> +///
> +/// ```
> +/// use kernel::sync::{
> +/// atomic::{
> +/// Atomic,
> +/// Relaxed,
> +/// },
> +/// new_waitqueue,
> +/// WaitQueue,
> +/// };
> +///
> +/// #[pin_data]
> +/// pub struct Example {
> +/// value: Atomic<i32>,
> +/// #[pin]
> +/// queue: WaitQueue,
> +/// }
> +///
> +/// fn wait_for_value(e: &Example, v: i32) {
> +/// e.queue.wait_event(|| e.value.load(Relaxed) == v);
> +/// }
> +///
> +/// fn set_value(e: &Example, v: i32) {
> +/// e.value.store(v, Relaxed);
> +/// e.queue.wake_up();
> +/// }
> +/// ```
> +///
> +/// [`struct wait_queue_head`]: srctree/include/linux/wait.h
> +#[pin_data]
> +pub struct WaitQueue {
> + #[pin]
> + wait_queue_head: Opaque<bindings::wait_queue_head>,
> +}
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [PATCH v3 2/5] rust: sync: add WaitQueue infrastructure
2026-11-17 0:14 ` [PATCH v3 2/5] rust: sync: add WaitQueue infrastructure Danilo Krummrich
2026-11-17 11:59 ` Gary Guo
@ 2026-11-17 14:16 ` Alice Ryhl
1 sibling, 0 replies; 17+ messages in thread
From: Alice Ryhl @ 2026-11-17 14:16 UTC (permalink / raw)
To: Danilo Krummrich
Cc: Alice Ryhl, boqun, gary, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, tamird, acourbot, viro,
brauner, jack, linux-kernel, rust-for-linux, linux-fsdevel
On Tue, Nov 17, 2026 at 01:14:43AM +0100, Danilo Krummrich wrote:
> Implement a wait queue wrapping the kernel's struct wait_queue_head,
> with wait_event()-style methods that take a condition closure directly.
>
> The API mirrors the C wait_event() family:
>
> - wait_event()
> - wait_event_interruptible()
> - wait_event_timeout()
> - wait_event_interruptible_timeout()
> - wake_up() / wake_up_all() / wake_up_sync()
>
> The interruptible variant returns Result<(), Interrupted>, the timeout
> variant Result<(), TimedOut>, and the combined variant Result<(),
> WaitError>. These errors map to ERESTARTSYS (signal) or ETIMEDOUT
> (timeout).
>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Thanks for addressing the feedback from v1!
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Alice
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v3 3/5] rust: sync: convert CondVar and PollCondVar to use WaitQueue
2026-11-17 0:14 [PATCH v3 0/5] rust: sync: add WaitQueue infrastructure Danilo Krummrich
2026-11-17 0:14 ` [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper Danilo Krummrich
2026-11-17 0:14 ` [PATCH v3 2/5] rust: sync: add WaitQueue infrastructure Danilo Krummrich
@ 2026-11-17 0:14 ` Danilo Krummrich
2026-11-17 14:16 ` Alice Ryhl
2026-11-17 0:14 ` [PATCH v3 4/5] rust: sync: condvar: use task::schedule_timeout() Danilo Krummrich
2026-11-17 0:14 ` [PATCH v3 5/5] rust: sync: condvar: add non-exclusive freezable waiting Danilo Krummrich
4 siblings, 1 reply; 17+ messages in thread
From: Danilo Krummrich @ 2026-11-17 0:14 UTC (permalink / raw)
To: boqun, gary, aliceryhl, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, dakr, tamird, acourbot,
viro, brauner, jack
Cc: linux-kernel, rust-for-linux, linux-fsdevel
CondVar currently wraps a raw wait_queue_head directly. Now that
WaitQueue provides the same primitive, convert CondVar to use it
instead. This removes duplicate init, pinning, Send/Sync, and wake-up
code from CondVar.
Keep the wait logic in CondVar since it needs to release and reacquire a
lock guard around the sleep, which WaitQueue's condition-based
wait_event() methods do not handle. Use wait_once_exclusive() to perform
a single prepare_to_wait_exclusive / finish_wait cycle with a
caller-provided schedule function to implement the lock-aware sleep.
Adapt PollCondVar accordingly.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
rust/kernel/sync/condvar.rs | 89 +++++++++++--------------------------
rust/kernel/sync/poll.rs | 19 ++++----
rust/kernel/sync/wait.rs | 2 -
3 files changed, 33 insertions(+), 77 deletions(-)
diff --git a/rust/kernel/sync/condvar.rs b/rust/kernel/sync/condvar.rs
index 4be2c3bf8802..2deeaf2038e4 100644
--- a/rust/kernel/sync/condvar.rs
+++ b/rust/kernel/sync/condvar.rs
@@ -5,21 +5,29 @@
//! This module allows Rust code to use the kernel's [`struct wait_queue_head`] as a condition
//! variable.
-use super::{lock::Backend, lock::Guard, LockClassKey};
+use super::{
+ lock::Backend,
+ lock::Guard,
+ LockClassKey,
+ WaitQueue, //
+};
+
use crate::{
- ffi::{c_int, c_long},
- str::{CStr, CStrExt as _},
+ prelude::*,
+ str::CStr,
task::{
- MAX_SCHEDULE_TIMEOUT, TASK_FREEZABLE, TASK_INTERRUPTIBLE, TASK_NORMAL, TASK_UNINTERRUPTIBLE,
+ MAX_SCHEDULE_TIMEOUT,
+ TASK_FREEZABLE,
+ TASK_INTERRUPTIBLE,
+ TASK_UNINTERRUPTIBLE, //
},
time::{
Delta,
Jiffy, //
},
- types::Opaque,
};
-use core::{marker::PhantomPinned, pin::Pin, ptr};
-use pin_init::{pin_data, pin_init, PinInit};
+
+use core::pin::Pin;
/// Creates a [`CondVar`] initialiser with the given name and a newly-created lock class.
#[macro_export]
@@ -84,33 +92,14 @@ macro_rules! new_condvar {
#[pin_data]
pub struct CondVar {
#[pin]
- pub(crate) wait_queue_head: Opaque<bindings::wait_queue_head>,
-
- /// A condvar needs to be pinned because it contains a [`struct list_head`] that is
- /// self-referential, so it cannot be safely moved once it is initialised.
- ///
- /// [`struct list_head`]: srctree/include/linux/types.h
- #[pin]
- _pin: PhantomPinned,
+ pub(crate) wq: WaitQueue,
}
-// SAFETY: `CondVar` only uses a `struct wait_queue_head`, which is safe to use on any thread.
-unsafe impl Send for CondVar {}
-
-// SAFETY: `CondVar` only uses a `struct wait_queue_head`, which is safe to use on multiple threads
-// concurrently.
-unsafe impl Sync for CondVar {}
-
impl CondVar {
/// Constructs a new condvar initialiser.
pub fn new(name: &'static CStr, key: Pin<&'static LockClassKey>) -> impl PinInit<Self> {
pin_init!(Self {
- _pin: PhantomPinned,
- // SAFETY: `slot` is valid while the closure is called and both `name` and `key` have
- // static lifetimes so they live indefinitely.
- wait_queue_head <- Opaque::ffi_init(|slot| unsafe {
- bindings::__init_waitqueue_head(slot, name.as_char_ptr(), key.as_ptr())
- }),
+ wq <- WaitQueue::new(name, key),
})
}
@@ -120,23 +109,10 @@ fn wait_internal<T: ?Sized, B: Backend>(
guard: &mut Guard<'_, T, B>,
timeout_in_jiffies: c_long,
) -> c_long {
- let wait = Opaque::<bindings::wait_queue_entry>::uninit();
-
- // SAFETY: `wait` points to valid memory.
- unsafe { bindings::init_wait(wait.get()) };
-
- // SAFETY: Both `wait` and `wait_queue_head` point to valid memory.
- unsafe {
- bindings::prepare_to_wait_exclusive(self.wait_queue_head.get(), wait.get(), wait_state)
- };
-
- // SAFETY: Switches to another thread. The timeout can be any number.
- let ret = guard.do_unlocked(|| unsafe { bindings::schedule_timeout(timeout_in_jiffies) });
-
- // SAFETY: Both `wait` and `wait_queue_head` point to valid memory.
- unsafe { bindings::finish_wait(self.wait_queue_head.get(), wait.get()) };
-
- ret
+ self.wq.wait_once_exclusive(wait_state, || {
+ // SAFETY: Switches to another thread. The timeout can be any number.
+ guard.do_unlocked(|| unsafe { bindings::schedule_timeout(timeout_in_jiffies) })
+ })
}
/// Releases the lock and waits for a notification in uninterruptible mode.
@@ -211,28 +187,13 @@ pub fn wait_interruptible_timeout<T: ?Sized, B: Backend>(
}
}
- /// Calls the kernel function to notify the appropriate number of threads.
- fn notify(&self, count: c_int) {
- // SAFETY: `wait_queue_head` points to valid memory.
- unsafe {
- bindings::__wake_up(
- self.wait_queue_head.get(),
- TASK_NORMAL,
- count,
- ptr::null_mut(),
- )
- };
- }
-
/// Calls the kernel function to notify one thread synchronously.
///
/// This method behaves like `notify_one`, except that it hints to the scheduler that the
- /// current thread is about to go to sleep, so it should schedule the target thread on the same
- /// CPU.
+ /// current thread expects to sleep soon.
#[inline]
pub fn notify_sync(&self) {
- // SAFETY: `wait_queue_head` points to valid memory.
- unsafe { bindings::__wake_up_sync(self.wait_queue_head.get(), TASK_NORMAL) };
+ self.wq.wake_up_sync();
}
/// Wakes a single waiter up, if any.
@@ -241,7 +202,7 @@ pub fn notify_sync(&self) {
/// completely (as opposed to automatically waking up the next waiter).
#[inline]
pub fn notify_one(&self) {
- self.notify(1);
+ self.wq.wake_up();
}
/// Wakes all waiters up, if any.
@@ -250,7 +211,7 @@ pub fn notify_one(&self) {
/// completely (as opposed to automatically waking up the next waiter).
#[inline]
pub fn notify_all(&self) {
- self.notify(0);
+ self.wq.wake_up_all();
}
}
diff --git a/rust/kernel/sync/poll.rs b/rust/kernel/sync/poll.rs
index dc40bfaa57e6..fbd754a0bc54 100644
--- a/rust/kernel/sync/poll.rs
+++ b/rust/kernel/sync/poll.rs
@@ -64,11 +64,11 @@ pub fn register_wait(&self, file: &File, cv: &PollCondVar) {
// * `file.as_ptr()` references a valid file for the duration of this call.
// * `self.table` is null or references a valid poll_table for the duration of this call.
// * Since `PollCondVar` is pinned, its destructor is guaranteed to run before the memory
- // containing `cv.wait_queue_head` is invalidated. Since the destructor clears all
- // waiters and then waits for an rcu grace period, it's guaranteed that
- // `cv.wait_queue_head` remains valid for at least an rcu grace period after the removal
- // of the last waiter.
- unsafe { bindings::poll_wait(file.as_ptr(), cv.wait_queue_head.get(), self.table) }
+ // containing the wait queue head is invalidated. Since the destructor clears all
+ // waiters and then waits for an rcu grace period, it's guaranteed that the wait queue
+ // head remains valid for at least an rcu grace period after the removal of the last
+ // waiter.
+ unsafe { bindings::poll_wait(file.as_ptr(), cv.wq.as_raw(), self.table) }
}
}
@@ -105,9 +105,7 @@ impl PinnedDrop for PollCondVar {
#[inline]
fn drop(self: Pin<&mut Self>) {
// Clear anything registered using `register_wait`.
- //
- // SAFETY: The pointer points at a valid `wait_queue_head`.
- unsafe { bindings::__wake_up_pollfree(self.inner.wait_queue_head.get()) };
+ self.inner.wq.wake_up_pollfree();
// Wait for epoll items to be properly removed.
synchronize_rcu();
@@ -165,14 +163,13 @@ fn drop(&mut self) {
// SAFETY: ManuallyDrop::take ok because not already taken.
let boxed = unsafe { ManuallyDrop::take(&mut self.inner) };
+ boxed.inner.inner.wq.wake_up_pollfree();
+
// SAFETY: The code below frees the box without calling the actual destructor of the type,
// but it's okay because it re-implements the destructor using `kfree_rcu()` in place of
// `synchronize_rcu()`.
let ptr = KBox::into_raw(unsafe { Pin::into_inner_unchecked(boxed) });
- // SAFETY: The pointer points at a valid `wait_queue_head`.
- unsafe { bindings::__wake_up_pollfree((*ptr).inner.inner.wait_queue_head.get()) };
-
// SAFETY: This was allocated using `KBox::pin_init`, so it can be freed with `kvfree`.
unsafe { bindings::kvfree_call_rcu((*ptr).rcu.get(), ptr.cast::<ffi::c_void>()) };
}
diff --git a/rust/kernel/sync/wait.rs b/rust/kernel/sync/wait.rs
index abbd5466ff44..819dbd0eb3a0 100644
--- a/rust/kernel/sync/wait.rs
+++ b/rust/kernel/sync/wait.rs
@@ -238,7 +238,6 @@ pub(super) fn wait_once<F, R>(&self, wait_state: c_int, schedule_fn: F) -> R
/// Performs a single exclusive prepare-to-wait / finish-wait cycle, calling `schedule_fn`
/// in between.
#[inline]
- #[expect(unused)]
pub(super) fn wait_once_exclusive<F, R>(&self, wait_state: c_int, schedule_fn: F) -> R
where
F: FnOnce() -> R,
@@ -322,7 +321,6 @@ pub fn wake_up_sync(&self) {
/// Used when a wait queue is about to be freed, to ensure epoll items are properly removed.
/// Matches C's `wake_up_pollfree()`.
#[inline]
- #[expect(unused)]
pub(super) fn wake_up_pollfree(&self) {
// SAFETY: `wait_queue_head` points to valid memory.
unsafe { bindings::__wake_up_pollfree(self.wait_queue_head.get()) };
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* Re: [PATCH v3 3/5] rust: sync: convert CondVar and PollCondVar to use WaitQueue
2026-11-17 0:14 ` [PATCH v3 3/5] rust: sync: convert CondVar and PollCondVar to use WaitQueue Danilo Krummrich
@ 2026-11-17 14:16 ` Alice Ryhl
0 siblings, 0 replies; 17+ messages in thread
From: Alice Ryhl @ 2026-11-17 14:16 UTC (permalink / raw)
To: Danilo Krummrich
Cc: Alice Ryhl, boqun, gary, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, tamird, acourbot, viro,
brauner, jack, linux-kernel, rust-for-linux, linux-fsdevel
On Tue, Nov 17, 2026 at 01:14:44AM +0100, Danilo Krummrich wrote:
> CondVar currently wraps a raw wait_queue_head directly. Now that
> WaitQueue provides the same primitive, convert CondVar to use it
> instead. This removes duplicate init, pinning, Send/Sync, and wake-up
> code from CondVar.
>
> Keep the wait logic in CondVar since it needs to release and reacquire a
> lock guard around the sleep, which WaitQueue's condition-based
> wait_event() methods do not handle. Use wait_once_exclusive() to perform
> a single prepare_to_wait_exclusive / finish_wait cycle with a
> caller-provided schedule function to implement the lock-aware sleep.
>
> Adapt PollCondVar accordingly.
>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v3 4/5] rust: sync: condvar: use task::schedule_timeout()
2026-11-17 0:14 [PATCH v3 0/5] rust: sync: add WaitQueue infrastructure Danilo Krummrich
` (2 preceding siblings ...)
2026-11-17 0:14 ` [PATCH v3 3/5] rust: sync: convert CondVar and PollCondVar to use WaitQueue Danilo Krummrich
@ 2026-11-17 0:14 ` Danilo Krummrich
2026-11-17 14:16 ` Alice Ryhl
2026-11-17 0:14 ` [PATCH v3 5/5] rust: sync: condvar: add non-exclusive freezable waiting Danilo Krummrich
4 siblings, 1 reply; 17+ messages in thread
From: Danilo Krummrich @ 2026-11-17 0:14 UTC (permalink / raw)
To: boqun, gary, aliceryhl, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, dakr, tamird, acourbot,
viro, brauner, jack
Cc: linux-kernel, rust-for-linux, linux-fsdevel
Convert CondVar::wait_internal() from calling
bindings::schedule_timeout() directly to using the safe
task::schedule_timeout() wrapper. This removes the unsafe call from
CondVar.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
rust/kernel/sync/condvar.rs | 42 ++++++++++++++++++-------------------
1 file changed, 20 insertions(+), 22 deletions(-)
diff --git a/rust/kernel/sync/condvar.rs b/rust/kernel/sync/condvar.rs
index 2deeaf2038e4..dae07dc140c5 100644
--- a/rust/kernel/sync/condvar.rs
+++ b/rust/kernel/sync/condvar.rs
@@ -16,7 +16,7 @@
prelude::*,
str::CStr,
task::{
- MAX_SCHEDULE_TIMEOUT,
+ self,
TASK_FREEZABLE,
TASK_INTERRUPTIBLE,
TASK_UNINTERRUPTIBLE, //
@@ -107,11 +107,10 @@ fn wait_internal<T: ?Sized, B: Backend>(
&self,
wait_state: c_int,
guard: &mut Guard<'_, T, B>,
- timeout_in_jiffies: c_long,
- ) -> c_long {
+ timeout: Delta<Jiffy>,
+ ) -> Delta<Jiffy> {
self.wq.wait_once_exclusive(wait_state, || {
- // SAFETY: Switches to another thread. The timeout can be any number.
- guard.do_unlocked(|| unsafe { bindings::schedule_timeout(timeout_in_jiffies) })
+ guard.do_unlocked(|| task::schedule_timeout(timeout))
})
}
@@ -122,7 +121,11 @@ fn wait_internal<T: ?Sized, B: Backend>(
/// [`CondVar::notify_one`] or [`CondVar::notify_all`]. Note that it may also wake up
/// spuriously.
pub fn wait<T: ?Sized, B: Backend>(&self, guard: &mut Guard<'_, T, B>) {
- self.wait_internal(TASK_UNINTERRUPTIBLE, guard, MAX_SCHEDULE_TIMEOUT);
+ self.wait_internal(
+ TASK_UNINTERRUPTIBLE,
+ guard,
+ Delta::from_jiffies(task::MAX_SCHEDULE_TIMEOUT),
+ );
}
/// Releases the lock and waits for a notification in interruptible mode.
@@ -133,7 +136,11 @@ pub fn wait<T: ?Sized, B: Backend>(&self, guard: &mut Guard<'_, T, B>) {
/// Returns whether there is a signal pending.
#[must_use = "wait_interruptible returns if a signal is pending, so the caller must check the return value"]
pub fn wait_interruptible<T: ?Sized, B: Backend>(&self, guard: &mut Guard<'_, T, B>) -> bool {
- self.wait_internal(TASK_INTERRUPTIBLE, guard, MAX_SCHEDULE_TIMEOUT);
+ self.wait_internal(
+ TASK_INTERRUPTIBLE,
+ guard,
+ Delta::from_jiffies(task::MAX_SCHEDULE_TIMEOUT),
+ );
crate::current!().signal_pending()
}
@@ -151,7 +158,7 @@ pub fn wait_interruptible_freezable<T: ?Sized, B: Backend>(
self.wait_internal(
TASK_INTERRUPTIBLE | TASK_FREEZABLE,
guard,
- MAX_SCHEDULE_TIMEOUT,
+ Delta::from_jiffies(task::MAX_SCHEDULE_TIMEOUT),
);
crate::current!().signal_pending()
}
@@ -169,21 +176,12 @@ pub fn wait_interruptible_timeout<T: ?Sized, B: Backend>(
guard: &mut Guard<'_, T, B>,
timeout: Delta<Jiffy>,
) -> CondVarTimeoutResult {
- let jiffies = timeout.as_jiffies();
- let res = self.wait_internal(
- TASK_INTERRUPTIBLE,
- guard,
- jiffies.clamp(0, MAX_SCHEDULE_TIMEOUT),
- );
+ let remaining = self.wait_internal(TASK_INTERRUPTIBLE, guard, timeout);
- match (res, crate::current!().signal_pending()) {
- (remainder, true) => CondVarTimeoutResult::Signal {
- remainder: Delta::from_jiffies(remainder),
- },
- (0, false) => CondVarTimeoutResult::Timeout,
- (remainder, false) => CondVarTimeoutResult::Woken {
- remainder: Delta::from_jiffies(remainder),
- },
+ match (remaining, current!().signal_pending()) {
+ (remainder, true) => CondVarTimeoutResult::Signal { remainder },
+ (Delta::ZERO, false) => CondVarTimeoutResult::Timeout,
+ (remainder, false) => CondVarTimeoutResult::Woken { remainder },
}
}
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* Re: [PATCH v3 4/5] rust: sync: condvar: use task::schedule_timeout()
2026-11-17 0:14 ` [PATCH v3 4/5] rust: sync: condvar: use task::schedule_timeout() Danilo Krummrich
@ 2026-11-17 14:16 ` Alice Ryhl
0 siblings, 0 replies; 17+ messages in thread
From: Alice Ryhl @ 2026-11-17 14:16 UTC (permalink / raw)
To: Danilo Krummrich
Cc: Alice Ryhl, boqun, gary, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, tamird, acourbot, viro,
brauner, jack, linux-kernel, rust-for-linux, linux-fsdevel
On Tue, Nov 17, 2026 at 01:14:45AM +0100, Danilo Krummrich wrote:
> @@ -122,7 +121,11 @@ fn wait_internal<T: ?Sized, B: Backend>(
> /// [`CondVar::notify_one`] or [`CondVar::notify_all`]. Note that it may also wake up
> /// spuriously.
> pub fn wait<T: ?Sized, B: Backend>(&self, guard: &mut Guard<'_, T, B>) {
> - self.wait_internal(TASK_UNINTERRUPTIBLE, guard, MAX_SCHEDULE_TIMEOUT);
> + self.wait_internal(
> + TASK_UNINTERRUPTIBLE,
> + guard,
> + Delta::from_jiffies(task::MAX_SCHEDULE_TIMEOUT),
> + );
> }
Now that `CondVar` uses `task::schedule_timeout()`, all remaining uses of
`task::MAX_SCHEDULE_TIMEOUT` wrap it in `Delta::from_jiffies(...)`, so we
could also change `MAX_SCHEDULE_TIMEOUT` itself to `Delta<Jiffy>` (either
if you spin a v4 or as a follow-up). Either way:
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Alice
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v3 5/5] rust: sync: condvar: add non-exclusive freezable waiting
2026-11-17 0:14 [PATCH v3 0/5] rust: sync: add WaitQueue infrastructure Danilo Krummrich
` (3 preceding siblings ...)
2026-11-17 0:14 ` [PATCH v3 4/5] rust: sync: condvar: use task::schedule_timeout() Danilo Krummrich
@ 2026-11-17 0:14 ` Danilo Krummrich
2026-11-17 14:16 ` Alice Ryhl
4 siblings, 1 reply; 17+ messages in thread
From: Danilo Krummrich @ 2026-11-17 0:14 UTC (permalink / raw)
To: boqun, gary, aliceryhl, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, dakr, tamird, acourbot,
viro, brauner, jack
Cc: linux-kernel, rust-for-linux, linux-fsdevel
CondVar currently registers all sleeping tasks as exclusive waiters.
Add wait_interruptible_freezable_nonexclusive() for callers that need
non-exclusive waits, using WaitQueue::wait_once().
Document that notify_one() and notify_sync() also wake all non-exclusive
waiters.
This provides non-exclusive waiting support for a subsequent Binder
conversion.
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
rust/kernel/sync/condvar.rs | 29 +++++++++++++++++++++++++++--
rust/kernel/sync/wait.rs | 1 -
2 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/rust/kernel/sync/condvar.rs b/rust/kernel/sync/condvar.rs
index dae07dc140c5..89f1a31afb89 100644
--- a/rust/kernel/sync/condvar.rs
+++ b/rust/kernel/sync/condvar.rs
@@ -163,6 +163,31 @@ pub fn wait_interruptible_freezable<T: ?Sized, B: Backend>(
crate::current!().signal_pending()
}
+ /// Releases the lock and waits non-exclusively in interruptible and freezable mode.
+ ///
+ /// Like [`CondVar::wait_interruptible_freezable`], but registers the current task as a
+ /// non-exclusive waiter.
+ ///
+ /// The lock is reacquired before returning. The wait may also end due to a signal or a
+ /// spurious wakeup.
+ ///
+ /// No lock other than the one released through `guard` should be held, since the task may
+ /// be frozen while sleeping.
+ ///
+ /// Returns whether there is a signal pending.
+ #[must_use = "the caller must check whether a signal is pending"]
+ pub fn wait_interruptible_freezable_nonexclusive<T: ?Sized, B: Backend>(
+ &self,
+ guard: &mut Guard<'_, T, B>,
+ ) -> bool {
+ self.wq.wait_once(TASK_INTERRUPTIBLE | TASK_FREEZABLE, || {
+ guard.do_unlocked(|| {
+ task::schedule_timeout(Delta::from_jiffies(task::MAX_SCHEDULE_TIMEOUT))
+ })
+ });
+ crate::current!().signal_pending()
+ }
+
/// Releases the lock and waits for a notification in interruptible mode.
///
/// Atomically releases the given lock (whose ownership is proven by the guard) and puts the
@@ -185,7 +210,7 @@ pub fn wait_interruptible_timeout<T: ?Sized, B: Backend>(
}
}
- /// Calls the kernel function to notify one thread synchronously.
+ /// Wakes all non-exclusive waiters and one exclusive waiter synchronously.
///
/// This method behaves like `notify_one`, except that it hints to the scheduler that the
/// current thread expects to sleep soon.
@@ -194,7 +219,7 @@ pub fn notify_sync(&self) {
self.wq.wake_up_sync();
}
- /// Wakes a single waiter up, if any.
+ /// Wakes all non-exclusive waiters and one exclusive waiter, if any.
///
/// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost
/// completely (as opposed to automatically waking up the next waiter).
diff --git a/rust/kernel/sync/wait.rs b/rust/kernel/sync/wait.rs
index 819dbd0eb3a0..8de7aed89e0c 100644
--- a/rust/kernel/sync/wait.rs
+++ b/rust/kernel/sync/wait.rs
@@ -227,7 +227,6 @@ fn wait_event_timeout_internal<F: FnMut() -> bool>(
/// Performs a single non-exclusive prepare-to-wait / finish-wait cycle, calling `schedule_fn`
/// in between. Non-exclusive waiters are all woken by [`wake_up()`](Self::wake_up).
#[inline]
- #[expect(unused)]
pub(super) fn wait_once<F, R>(&self, wait_state: c_int, schedule_fn: F) -> R
where
F: FnOnce() -> R,
--
2.56.1
^ permalink raw reply related [flat|nested] 17+ messages in thread* Re: [PATCH v3 5/5] rust: sync: condvar: add non-exclusive freezable waiting
2026-11-17 0:14 ` [PATCH v3 5/5] rust: sync: condvar: add non-exclusive freezable waiting Danilo Krummrich
@ 2026-11-17 14:16 ` Alice Ryhl
0 siblings, 0 replies; 17+ messages in thread
From: Alice Ryhl @ 2026-11-17 14:16 UTC (permalink / raw)
To: Danilo Krummrich
Cc: Alice Ryhl, boqun, gary, lyude, daniel.almeida, work, ojeda,
bjorn3_gh, lossin, a.hindborg, tmgross, tamird, acourbot, viro,
brauner, jack, linux-kernel, rust-for-linux, linux-fsdevel
On Tue, Nov 17, 2026 at 01:14:46AM +0100, Danilo Krummrich wrote:
> CondVar currently registers all sleeping tasks as exclusive waiters.
> Add wait_interruptible_freezable_nonexclusive() for callers that need
> non-exclusive waits, using WaitQueue::wait_once().
>
> Document that notify_one() and notify_sync() also wake all non-exclusive
> waiters.
>
> This provides non-exclusive waiting support for a subsequent Binder
> conversion.
>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
^ permalink raw reply [flat|nested] 17+ messages in thread