From: Andreas Hindborg <a.hindborg@kernel.org>
To: "Andrew Morton" <akpm@linux-foundation.org>,
"Vlastimil Babka" <vbabka@kernel.org>,
"Lorenzo Stoakes" <ljs@kernel.org>,
"Liam R. Howlett" <liam@infradead.org>,
"Uladzislau Rezki" <urezki@gmail.com>,
"David Hildenbrand" <david@kernel.org>,
"Suren Baghdasaryan" <surenb@google.com>,
"Michal Hocko" <mhocko@suse.com>,
"Brendan Jackman" <brendan.jackman@linux.dev>,
"Johannes Weiner" <hannes@cmpxchg.org>, "Zi Yan" <ziy@nvidia.com>,
"Mike Rapoport" <rppt@kernel.org>,
"Joshua Hahn" <joshua.hahnjy@gmail.com>,
"Rakie Kim" <rakie.kim@sk.com>,
"Gregory Price" <gourry@gourry.net>,
"Ying Huang" <ying.huang@linux.alibaba.com>,
"Matthew Wilcox" <willy@infradead.org>,
"Danilo Krummrich" <dakr@kernel.org>,
"Miguel Ojeda" <ojeda@kernel.org>,
"Boqun Feng" <boqun@kernel.org>, "Gary Guo" <gary@garyguo.net>,
"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
"Benno Lossin" <lossin@kernel.org>,
"Andreas Hindborg" <a.hindborg@kernel.org>,
"Alice Ryhl" <aliceryhl@google.com>,
"Trevor Gross" <tmgross@umich.edu>,
"Daniel Almeida" <daniel.almeida@collabora.com>,
"Tamir Duberstein" <tamird@kernel.org>,
"Alexandre Courbot" <acourbot@nvidia.com>,
"Onur Özkan" <work@onurozkan.dev>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
"Arve Hjønnevåg" <arve@android.com>,
"Todd Kjos" <tkjos@android.com>,
"Christian Brauner" <brauner@kernel.org>,
"Carlos Llamas" <cmllamas@google.com>
Cc: Igor B <igor.b@beldev.am>, Vitaly Wool <vitaly.wool@konsulko.se>,
linux-mm@kvack.org, rust-for-linux@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: [PATCH v22 4/6] rust: page: add `Exclusive` page kind for race-free page access
Date: Wed, 18 Nov 2026 17:18:20 +0100 [thread overview]
Message-ID: <20261118-page-ownable-v22-4-d63ff73f4075@kernel.org> (raw)
In-Reply-To: <20261118-page-ownable-v22-0-d63ff73f4075@kernel.org>
Add the page kind `Exclusive`. A `Page<Exclusive>` is a page that is
uniquely owned through an `Owned<Page<Exclusive>>`, with the added
invariant that the page data area does not incur data races. This means a
`Page<Exclusive>` cannot be mapped to user space or shared with devices,
and it becomes simpler to directly reference the contents of the page.
Back `Page<Exclusive>` by a frozen page, allocated with
`alloc_frozen_pages()` and freed with `free_frozen_pages()`. The reference
count of a frozen page is zero and references to it cannot be taken, so the
exclusive ownership is upheld by mm and not only by the Rust type system.
Add `Page::alloc_exclusive()` to allocate a `Page<Exclusive>`. The function
is not named `alloc_page()`, because inherent functions of the same name on
two page kinds would make the call `Page::alloc_page()` ambiguous.
`Page<Exclusive>` dereferences to `Page<Base>`. This does not allow
safe code to break the aliasing invariant of `Page<Exclusive>`:
`Page<Base>` does not implement `AlwaysRefCounted`, so no `ARef` to the
page can be obtained, and the raw data accessors are unsafe and already
require their callers to rule out races with all other accesses to the
page.
Suggested-by: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Andreas Hindborg <a.hindborg@kernel.org>
---
rust/helpers/page.c | 6 ++++
rust/kernel/page.rs | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++--
2 files changed, 100 insertions(+), 2 deletions(-)
diff --git a/rust/helpers/page.c b/rust/helpers/page.c
index 906eb4b18f02..8f9f624ea141 100644
--- a/rust/helpers/page.c
+++ b/rust/helpers/page.c
@@ -10,6 +10,12 @@ __rust_helper struct page *rust_helper_alloc_pages(gfp_t gfp_mask,
return alloc_pages(gfp_mask, order);
}
+__rust_helper struct page *rust_helper_alloc_frozen_pages(gfp_t gfp_mask,
+ unsigned int order)
+{
+ return alloc_frozen_pages(gfp_mask, order);
+}
+
__rust_helper void rust_helper_get_page(struct page *page)
{
get_page(page);
diff --git a/rust/kernel/page.rs b/rust/kernel/page.rs
index 2ceeac0237d5..c885bcf4f886 100644
--- a/rust/kernel/page.rs
+++ b/rust/kernel/page.rs
@@ -17,7 +17,11 @@
ARef,
AlwaysRefCounted, //
},
- types::Opaque,
+ types::{
+ Opaque,
+ Ownable,
+ Owned, //
+ },
uaccess::UserSliceReader, //
};
use core::{
@@ -89,7 +93,7 @@ pub trait Sealed {}
/// Marker trait for the kinds of a [`Page`].
///
/// A page kind describes what is known about a `struct page`, and thereby what a [`Page`] of that
-/// kind can be used for. See [`Base`] and [`RefCounted`].
+/// kind can be used for. See [`Base`], [`RefCounted`] and [`Exclusive`].
///
/// A [`Page`] of any other kind dereferences to a [`Page`] of the base kind [`Base`]. The
/// purpose of the dereference is coercion: a reference to a page of any kind can be passed to code
@@ -134,6 +138,7 @@ impl PageKind for RefCounted {}
///
/// - `Page<Base>` is a page for which nothing is known beyond its validity.
/// - `Page<RefCounted>`, or just `Page`, is a page that is managed by the page reference count.
+/// - `Page<Exclusive>` is a uniquely owned page whose data follows standard Rust aliasing rules.
///
/// Operations that are valid for any page are available for all page kinds. All other page kinds
/// dereference to [`Page<Base>`], so that a reference to a page of any kind coerces to a
@@ -429,3 +434,90 @@ unsafe fn dec_ref(obj: NonNull<Self>) {
// SAFETY: All instances of `Page` are reference counted, so a `&Page` proves that the refcount is
// non-zero and it is okay to obtain an `ARef<Page>` from it.
unsafe impl AlwaysRefCounted for Page {}
+
+/// A [`PageKind`] for a page whose data area follows standard Rust aliasing rules.
+///
+/// A [`Page<Exclusive>`] is accessed through an owning [`Owned<Page<Exclusive>>`]. It has the same
+/// usage constraints as other Rust types. Thus, it cannot be mapped to user space or shared with
+/// devices. This makes it safe to reference the contents of the page while the page is mapped in
+/// kernel space.
+///
+/// A [`Page<Exclusive>`] is backed by a frozen page. The reference count of a frozen page is zero,
+/// and references to it cannot be taken. Thus, the exclusive ownership of the page is upheld by
+/// the kernel, and not only by the Rust type system.
+///
+/// # Invariants
+///
+/// For a [`Page<Exclusive>`]:
+///
+/// - The page is an order 0 frozen page: it was allocated with `alloc_frozen_pages()`, and its
+/// reference count is zero for as long as the [`Page<Exclusive>`] exists.
+/// - The page is owned by a unique [`Owned<Page<Exclusive>>`]. No references to the page exist
+/// together with the [`Owned<Page<Exclusive>>`], other than the ones borrowed from it.
+/// - The data of the page is accessed only through references to the [`Page<Exclusive>`],
+/// including the [`Page<Base>`] references it dereferences to.
+/// - Safe code reads the data of the page only through a shared reference, and writes it only
+/// through a mutable reference. The unsafe raw accessors of [`Page`] can write through a shared
+/// reference, their callers are required to rule out races with all other accesses.
+pub struct Exclusive;
+
+impl private::Sealed for Exclusive {}
+impl PageKind for Exclusive {}
+
+impl Deref for Page<Exclusive> {
+ type Target = Page<Base>;
+
+ #[inline]
+ fn deref(&self) -> &Self::Target {
+ // SAFETY: `Page` is transparent over `struct page` for all page kinds, so the pointer cast
+ // is valid. `Base` has no invariants in addition to the ones of `Page`, which hold for
+ // `self`.
+ unsafe { &*ptr::from_ref(self).cast() }
+ }
+}
+
+impl Page<Exclusive> {
+ /// Allocates a new exclusive page.
+ ///
+ /// # Examples
+ ///
+ /// ```
+ /// use kernel::page::Page;
+ ///
+ /// let page = Page::alloc_exclusive(GFP_KERNEL | __GFP_ZERO)?;
+ /// # Ok::<(), kernel::alloc::AllocError>(())
+ /// ```
+ #[inline]
+ pub fn alloc_exclusive(flags: Flags) -> Result<Owned<Self>, AllocError> {
+ // SAFETY: Depending on the value of `gfp_flags`, this call may sleep. Other than that, it
+ // is always safe to call this method.
+ let page = unsafe { bindings::alloc_frozen_pages(flags.as_raw(), 0) };
+ let page = NonNull::new(page).ok_or(AllocError)?;
+
+ // INVARIANT: The page was just allocated with `alloc_frozen_pages()`, so it is frozen, we
+ // own it, and its data is only accessible through the returned `Owned<Page<Exclusive>>`.
+ // SAFETY:
+ // - We just successfully allocated the page, so `page` points to a valid `struct page`.
+ // Since `Page` is transparent over `struct page`, we can cast the pointer directly.
+ // - The page was just allocated, so no other `Owned` exists for it.
+ // - A `struct page` is never moved, so the page is pinned.
+ // - `Page<Exclusive>` allows no other references to exist together with the `Owned`, and
+ // none exist, as the page is newly allocated and references cannot be taken to a frozen
+ // page. Thus, it is sound to hand out `&Page<Exclusive>` and
+ // `Pin<&mut Page<Exclusive>>`.
+ // - Pages have no logic that relies on them staying on a given thread or on them not being
+ // accessed concurrently.
+ Ok(unsafe { Owned::from_raw(page.cast()) })
+ }
+}
+
+impl Ownable for Page<Exclusive> {
+ #[inline]
+ unsafe fn release(this: NonNull<Self>) {
+ // SAFETY: By the function safety requirements, we have exclusive ownership of the page, and
+ // by the type invariant the page is an order 0 frozen page, so we can free it with
+ // `free_frozen_pages()`. Since `Page` is transparent over `struct page`, we can cast the
+ // pointer directly.
+ unsafe { bindings::free_frozen_pages(this.cast().as_ptr(), 0) };
+ }
+}
--
2.54.0
next prev parent reply other threads:[~2026-11-18 16:20 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 16:18 [PATCH v22 0/6] rust: page: add page kinds Andreas Hindborg
2026-11-18 16:18 ` [PATCH v22 1/6] rust: page: add page kinds and make `Page` reference counted Andreas Hindborg
2026-11-18 16:18 ` [PATCH v22 2/6] rust: page: add `from_raw()` Andreas Hindborg
2026-11-18 16:18 ` [PATCH v22 3/6] mm/page_alloc: make alloc_frozen_pages() available outside of mm Andreas Hindborg
2026-11-18 16:18 ` Andreas Hindborg [this message]
2026-11-18 16:18 ` [PATCH v22 5/6] rust: page: add `Frozen` page kind Andreas Hindborg
2026-11-18 16:18 ` [PATCH v22 6/6] rust: page: add slice access to `Page<Exclusive>` Andreas Hindborg
2026-11-18 17:02 ` [PATCH v22 0/6] rust: page: add page kinds Zi Yan
2026-11-18 19:34 ` Andreas Hindborg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261118-page-ownable-v22-4-d63ff73f4075@kernel.org \
--to=a.hindborg@kernel.org \
--cc=acourbot@nvidia.com \
--cc=akpm@linux-foundation.org \
--cc=aliceryhl@google.com \
--cc=arve@android.com \
--cc=bjorn3_gh@protonmail.com \
--cc=boqun@kernel.org \
--cc=brauner@kernel.org \
--cc=brendan.jackman@linux.dev \
--cc=cmllamas@google.com \
--cc=dakr@kernel.org \
--cc=daniel.almeida@collabora.com \
--cc=david@kernel.org \
--cc=gary@garyguo.net \
--cc=gourry@gourry.net \
--cc=gregkh@linuxfoundation.org \
--cc=hannes@cmpxchg.org \
--cc=igor.b@beldev.am \
--cc=joshua.hahnjy@gmail.com \
--cc=liam@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=lossin@kernel.org \
--cc=mhocko@suse.com \
--cc=ojeda@kernel.org \
--cc=rakie.kim@sk.com \
--cc=rppt@kernel.org \
--cc=rust-for-linux@vger.kernel.org \
--cc=surenb@google.com \
--cc=tamird@kernel.org \
--cc=tkjos@android.com \
--cc=tmgross@umich.edu \
--cc=urezki@gmail.com \
--cc=vbabka@kernel.org \
--cc=vitaly.wool@konsulko.se \
--cc=willy@infradead.org \
--cc=work@onurozkan.dev \
--cc=ying.huang@linux.alibaba.com \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox