From: Andreas Hindborg <a.hindborg@kernel.org>
To: "Alice Ryhl" <aliceryhl@google.com>,
"Lorenzo Stoakes" <ljs@kernel.org>,
"Liam R. Howlett" <liam@infradead.org>,
"Miguel Ojeda" <ojeda@kernel.org>,
"Boqun Feng" <boqun@kernel.org>, "Gary Guo" <gary@garyguo.net>,
"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
"Benno Lossin" <lossin@kernel.org>,
"Trevor Gross" <tmgross@umich.edu>,
"Danilo Krummrich" <dakr@kernel.org>,
"Daniel Almeida" <daniel.almeida@collabora.com>,
"Tamir Duberstein" <tamird@kernel.org>,
"Alexandre Courbot" <acourbot@nvidia.com>,
"Onur Özkan" <work@onurozkan.dev>,
"Will Deacon" <will@kernel.org>,
"Peter Zijlstra" <peterz@infradead.org>,
"Mark Rutland" <mark.rutland@arm.com>,
"Lyude Paul" <lyude@redhat.com>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
"Arve Hjønnevåg" <arve@android.com>,
"Todd Kjos" <tkjos@android.com>,
"Christian Brauner" <brauner@kernel.org>,
"Carlos Llamas" <cmllamas@google.com>, "Zi Yan" <ziy@nvidia.com>
Cc: linux-mm@kvack.org, rust-for-linux@vger.kernel.org,
linux-kernel@vger.kernel.org, driver-core@lists.linux.dev,
Andreas Hindborg <a.hindborg@kernel.org>
Subject: [PATCH 04/11] rust: page: use byte-wise atomic operations in raw accessors
Date: Fri, 20 Nov 2026 11:27:03 +0100 [thread overview]
Message-ID: <20261120-page-shared-v1-4-66f8c4c62822@kernel.org> (raw)
In-Reply-To: <20261120-page-shared-v1-0-66f8c4c62822@kernel.org>
The raw accessors of `Page` are used to access pages that may be mapped to
user space or shared with devices. A plain access to such a page is a data
race when the other party writes to the page.
Add the private method `Page::with_page_as_sysmem()`, which maps a range
of a page and passes it to a closure as a `SysMem` view. Access the data
of the page through this view in `Page::read_raw()`, `Page::write_raw()`
and `Page::fill_zero_raw()`, so that all accesses are byte-wise atomic.
Zero the data with `SysMem::fill_zero()`.
Narrow the safety requirements of these methods, and of
`Page::read_sysmem()` and `Page::write_sysmem()`, accordingly: a call must
not race with a plain access to the same data, but it is allowed to race
with other non-plain accesses.
`Page::copy_from_user_slice_raw()` still writes to the page with plain
accesses. State that in the safety requirements.
Signed-off-by: Andreas Hindborg <a.hindborg@kernel.org>
---
rust/kernel/page.rs | 153 ++++++++++++++++++++++++++++++++++++----------------
1 file changed, 107 insertions(+), 46 deletions(-)
diff --git a/rust/kernel/page.rs b/rust/kernel/page.rs
index 3c48ecde4d14..ff191bfcd3f9 100644
--- a/rust/kernel/page.rs
+++ b/rust/kernel/page.rs
@@ -334,28 +334,74 @@ fn with_pointer_into_page<T>(
}
}
+ /// Runs a piece of code with a range of this page mapped as a [`SysMem`] view.
+ ///
+ /// If `f` is called, then it is called with a view of the `len` bytes of the page that start
+ /// at `offset`. Accesses through the view are volatile or byte-wise atomic, so the view is
+ /// suitable for a page whose data is subject to concurrent writes by hardware or user space.
+ /// The view is valid for the duration of the call to `f`.
+ ///
+ /// If `offset` and `len` refer to a region outside of this page, then this method returns
+ /// [`EINVAL`] and does not call `f`.
+ ///
+ /// # Safety
+ ///
+ /// * Callers must ensure that no plain access to the mapped range races with an access
+ /// performed through the view, see [`SysMem`].
+ /// * Callers must ensure that no mutable reference to the data of the page that overlaps with
+ /// the mapped range exists during this call. If `f` writes through the view, no reference to
+ /// that data may exist during this call. Such references can be obtained from a
+ /// [`Page<Exclusive>`], see [`Page::as_slice`].
+ /// * If the page is a [`Page<Exclusive>`], `f` must only write initialized bytes through the
+ /// view.
+ /// * Callers must ensure that the view is only used on the current task, as this method may
+ /// use a local mapping.
+ unsafe fn with_page_as_sysmem<T>(
+ &self,
+ offset: usize,
+ len: usize,
+ f: impl FnOnce(SysMem<'_, [u8]>) -> Result<T>,
+ ) -> Result<T> {
+ self.with_pointer_into_page(offset, len, move |ptr| {
+ // SAFETY:
+ // - If `with_pointer_into_page` calls into this closure, then it has performed a
+ // bounds check and guarantees that `ptr` is valid for `len` bytes for the duration
+ // of the closure. A byte slice has an alignment of 1.
+ // - By the safety requirements of this function, no plain access to the range races
+ // with an access through the view.
+ f(unsafe { SysMem::new(ptr::slice_from_raw_parts_mut(ptr, len)) })
+ })
+ }
+
/// Maps the page and reads from it into the given buffer.
///
/// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
/// outside of the page, then this call returns [`EINVAL`].
///
+ /// The copy is byte-wise atomic, so it is well-defined even if the data of the page is
+ /// concurrently written by hardware, by user space, or by other kernel code using atomic or
+ /// volatile accesses. In that case, the data that is read may be arbitrary.
+ ///
/// # Safety
///
- /// * Callers must ensure that `dst` is valid for writing `len` bytes.
- /// * Callers must ensure that this call does not race with a write to the same page that
- /// overlaps with this read.
+ /// * Callers must ensure that `dst` is valid for writing `len` bytes, and that this memory
+ /// does not overlap with the data of the page.
+ /// * Callers must ensure that this call does not race with a plain write to the same page that
+ /// overlaps with this read, see [`SysMem`].
/// * Callers must ensure that no mutable reference to the data of the page that overlaps with
/// this read exists during this call.
pub unsafe fn read_raw(&self, dst: *mut u8, offset: usize, len: usize) -> Result {
- self.with_pointer_into_page(offset, len, move |src| {
- // SAFETY: If `with_pointer_into_page` calls into this closure, then
- // it has performed a bounds check and guarantees that `src` is
- // valid for `len` bytes.
- //
- // There caller guarantees that there is no data race.
- unsafe { ptr::copy_nonoverlapping(src, dst, len) };
+ let copy = |src: SysMem<'_, [u8]>| {
+ // SAFETY: By the safety requirements of this function, `dst` is valid for writing
+ // `len` bytes, which is equal to `src.size()`, and it does not overlap with `src`.
+ unsafe { SysMemBackend::copy_from_io(src, dst) };
Ok(())
- })
+ };
+
+ // SAFETY: By the safety requirements of this function, no plain write races with the read
+ // through the view, and no mutable reference to the data that is read exists. `copy` does
+ // not write through the view, and it only uses the view on the current task.
+ unsafe { self.with_page_as_sysmem(offset, len, copy) }
}
/// Maps the page and reads from it into the given system memory region using byte-wise atomic
@@ -366,14 +412,14 @@ pub unsafe fn read_raw(&self, dst: *mut u8, offset: usize, len: usize) -> Result
///
/// The writes to `dst` are byte-wise atomic, so they are well-defined even if `dst` is
/// concurrently accessed by hardware, by user space, or by other kernel code using atomic or
- /// volatile accesses. The reads from the page are plain accesses, so the page must not be
- /// written to concurrently.
+ /// volatile accesses. The reads from the page are byte-wise atomic as well.
///
/// # Safety
///
/// Callers must ensure that:
///
- /// - This call does not race with a write to the source page that overlaps with this read.
+ /// - This call does not race with a plain write to the source page that overlaps with this
+ /// read, see [`SysMem`].
/// - No mutable reference to the data of the page that overlaps with this read exists during
/// this call.
/// - The memory region of `dst` does not overlap with the memory of the page.
@@ -393,8 +439,8 @@ pub unsafe fn read_raw(&self, dst: *mut u8, offset: usize, len: usize) -> Result
/// // way while `dst` is alive.
/// let dst = unsafe { SysMem::new(&raw mut buf as *mut [u8]) };
///
- /// // SAFETY: `page` is not written to during this call, no reference to the data of `page`
- /// // exists, and `buf` is not part of the page.
+ /// // SAFETY: No plain write to `page` happens during this call, no reference to the data of
+ /// // `page` exists, and `buf` is not part of the page.
/// unsafe { page.read_sysmem(dst, 0)? };
/// assert_eq!(buf, [0u8; 16]);
/// # Ok::<(), Error>(())
@@ -405,8 +451,8 @@ pub unsafe fn read_sysmem(&self, dst: SysMem<'_, [u8]>, offset: usize) -> Result
// - If `with_pointer_into_page` calls into this closure, then it has performed a
// bounds check and guarantees that `src` is valid for `dst.len()` bytes, which is
// equal to `dst.size()`.
- // - By the safety requirements of this function, the page is not written to during
- // this call, so `src` is valid for volatile reads.
+ // - By the safety requirements of this function, no plain write to the page races
+ // with this call, so `src` is valid for volatile reads.
// - By the safety requirements of this function, `src` does not overlap with `dst`.
unsafe { SysMemBackend::copy_to_io(dst, src) };
Ok(())
@@ -418,25 +464,34 @@ pub unsafe fn read_sysmem(&self, dst: SysMem<'_, [u8]>, offset: usize) -> Result
/// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
/// outside of the page, then this call returns [`EINVAL`].
///
+ /// The copy is byte-wise atomic, so it is well-defined even if the data of the page is
+ /// concurrently accessed by hardware, by user space, or by other kernel code using atomic or
+ /// volatile accesses.
+ ///
/// # Safety
///
- /// * Callers must ensure that `src` is valid for reading `len` bytes.
- /// * Callers must ensure that this call does not race with a read or write to the same page
- /// that overlaps with this write.
+ /// * Callers must ensure that `src` is valid for reading `len` bytes, and that this memory
+ /// does not overlap with the data of the page.
+ /// * Callers must ensure that this call does not race with a plain read or write to the same
+ /// page that overlaps with this write, see [`SysMem`].
/// * Callers must ensure that no reference to the data of the page that overlaps with this
/// write exists during this call. Such references can be obtained from a
/// [`Page<Exclusive>`], see [`Page::as_slice`].
/// * If the page is a [`Page<Exclusive>`], callers must ensure that the `len` bytes at `src`
/// are initialized.
pub unsafe fn write_raw(&self, src: *const u8, offset: usize, len: usize) -> Result {
- self.with_pointer_into_page(offset, len, move |dst| {
- // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
- // bounds check and guarantees that `dst` is valid for `len` bytes.
- //
- // There caller guarantees that there is no data race.
- unsafe { ptr::copy_nonoverlapping(src, dst, len) };
+ let copy = |dst: SysMem<'_, [u8]>| {
+ // SAFETY: By the safety requirements of this function, `src` is valid for reading
+ // `len` bytes, which is equal to `dst.size()`, and it does not overlap with `dst`.
+ unsafe { SysMemBackend::copy_to_io(dst, src) };
Ok(())
- })
+ };
+
+ // SAFETY: By the safety requirements of this function, no plain access races with the
+ // write through the view, no reference to the data that is written exists, and the bytes
+ // written are initialized if the page requires it. `copy` only uses the view on the
+ // current task.
+ unsafe { self.with_page_as_sysmem(offset, len, copy) }
}
/// Maps the page and writes into it from the given system memory region using byte-wise atomic
@@ -447,15 +502,14 @@ pub unsafe fn write_raw(&self, src: *const u8, offset: usize, len: usize) -> Res
///
/// The reads from `src` are byte-wise atomic, so they are well-defined even if `src` is
/// concurrently accessed by hardware, by user space, or by other kernel code using atomic or
- /// volatile accesses. The writes to the page are plain accesses, so the page must not be read
- /// from or written to concurrently.
+ /// volatile accesses. The writes to the page are byte-wise atomic as well.
///
/// # Safety
///
/// Callers must ensure that:
///
- /// - This call does not race with a read or write to the destination page that overlaps with
- /// this write.
+ /// - This call does not race with a plain read or write to the destination page that overlaps
+ /// with this write, see [`SysMem`].
/// - No reference to the data of the page that overlaps with this write exists during this
/// call. Such references can be obtained from a [`Page<Exclusive>`], see
/// [`Page::as_slice`].
@@ -483,8 +537,9 @@ pub unsafe fn write_raw(&self, src: *const u8, offset: usize, len: usize) -> Res
/// )
/// };
///
- /// // SAFETY: `page` is not accessed in any other way during these calls, no reference to the
- /// // data of `page` exists, and neither `data` nor `buf` is part of the page.
+ /// // SAFETY: No plain access to `page` happens during these calls, no reference to the data
+ /// // of `page` exists, `data` is initialized, and neither `data` nor `buf` is part of the
+ /// // page.
/// unsafe {
/// page.write_sysmem(src, 0)?;
/// page.read_sysmem(dst, 0)?;
@@ -498,8 +553,8 @@ pub unsafe fn write_sysmem(&self, src: SysMem<'_, [u8]>, offset: usize) -> Resul
// - If `with_pointer_into_page` calls into this closure, then it has performed a
// bounds check and guarantees that `dst` is valid for `src.len()` bytes, which is
// equal to `src.size()`.
- // - By the safety requirements of this function, the page is not read from or written
- // to during this call, so `dst` is valid for volatile writes.
+ // - By the safety requirements of this function, no plain access to the page races
+ // with this call, so `dst` is valid for volatile writes.
// - By the safety requirements of this function, `dst` does not overlap with `src`.
unsafe { SysMemBackend::copy_from_io(src, dst) };
Ok(())
@@ -511,22 +566,27 @@ pub unsafe fn write_sysmem(&self, src: SysMem<'_, [u8]>, offset: usize) -> Resul
/// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
/// outside of the page, then this call returns [`EINVAL`].
///
+ /// The writes are byte-wise atomic, so they are well-defined even if the data of the page is
+ /// concurrently accessed by hardware, by user space, or by other kernel code using atomic or
+ /// volatile accesses.
+ ///
/// # Safety
///
- /// * Callers must ensure that this call does not race with a read or write to the same page
- /// that overlaps with this write.
+ /// * Callers must ensure that this call does not race with a plain read or write to the same
+ /// page that overlaps with this write, see [`SysMem`].
/// * Callers must ensure that no reference to the data of the page that overlaps with this
/// write exists during this call. Such references can be obtained from a
/// [`Page<Exclusive>`], see [`Page::as_slice`].
pub unsafe fn fill_zero_raw(&self, offset: usize, len: usize) -> Result {
- self.with_pointer_into_page(offset, len, move |dst| {
- // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
- // bounds check and guarantees that `dst` is valid for `len` bytes.
- //
- // There caller guarantees that there is no data race.
- unsafe { ptr::write_bytes(dst, 0u8, len) };
+ let fill = |dst: SysMem<'_, [u8]>| {
+ dst.fill_zero();
Ok(())
- })
+ };
+
+ // SAFETY: By the safety requirements of this function, no plain access races with the
+ // write through the view, and no reference to the data that is written exists. The bytes
+ // written are initialized. `fill` only uses the view on the current task.
+ unsafe { self.with_page_as_sysmem(offset, len, fill) }
}
/// Copies data from userspace into this page.
@@ -540,7 +600,8 @@ pub unsafe fn fill_zero_raw(&self, offset: usize, len: usize) -> Result {
/// # Safety
///
/// * Callers must ensure that this call does not race with a read or write to the same page
- /// that overlaps with this write.
+ /// that overlaps with this write. This includes byte-wise atomic accesses, as this method
+ /// writes to the page with plain accesses.
/// * Callers must ensure that no reference to the data of the page that overlaps with this
/// write exists during this call. Such references can be obtained from a
/// [`Page<Exclusive>`], see [`Page::as_slice`].
--
2.54.0
next prev parent reply other threads:[~2026-11-20 10:28 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-20 10:26 [PATCH 00/11] rust: page: improve access methods for racy memory regions Andreas Hindborg
2026-11-20 10:27 ` [PATCH 01/11] rust: page: tighten safety requirements of `SysMem` copy methods Andreas Hindborg
2026-11-20 10:27 ` [PATCH 02/11] rust: sync: atomic: add memset Andreas Hindborg
2026-11-20 10:27 ` [PATCH 03/11] rust: io: add `SysMem::fill_zero` Andreas Hindborg
2026-11-20 10:27 ` Andreas Hindborg [this message]
2026-11-20 10:27 ` [PATCH 05/11] rust: uaccess: take a raw pointer in `UserSliceReader::read_raw` Andreas Hindborg
2026-11-20 10:27 ` [PATCH 06/11] rust: uaccess: add `UserSliceReader::read_sysmem` Andreas Hindborg
2026-11-20 10:27 ` [PATCH 07/11] rust: page: add `Shared` page kind Andreas Hindborg
2026-11-20 10:27 ` [PATCH 08/11] rust: io: add `SysMemLocal` Andreas Hindborg
2026-11-20 10:27 ` [PATCH 09/11] rust: page: add `Page::with_page_as_sysmem_local` Andreas Hindborg
2026-11-20 10:27 ` [PATCH 10/11] rust_binder: access page data through `SysMemLocal` Andreas Hindborg
2026-11-20 10:27 ` [PATCH 11/11] rust: page: remove `Page::copy_from_user_slice_raw` Andreas Hindborg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120-page-shared-v1-4-66f8c4c62822@kernel.org \
--to=a.hindborg@kernel.org \
--cc=acourbot@nvidia.com \
--cc=aliceryhl@google.com \
--cc=arve@android.com \
--cc=bjorn3_gh@protonmail.com \
--cc=boqun@kernel.org \
--cc=brauner@kernel.org \
--cc=cmllamas@google.com \
--cc=dakr@kernel.org \
--cc=daniel.almeida@collabora.com \
--cc=driver-core@lists.linux.dev \
--cc=gary@garyguo.net \
--cc=gregkh@linuxfoundation.org \
--cc=liam@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=lossin@kernel.org \
--cc=lyude@redhat.com \
--cc=mark.rutland@arm.com \
--cc=ojeda@kernel.org \
--cc=peterz@infradead.org \
--cc=rust-for-linux@vger.kernel.org \
--cc=tamird@kernel.org \
--cc=tkjos@android.com \
--cc=tmgross@umich.edu \
--cc=will@kernel.org \
--cc=work@onurozkan.dev \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox