Rust for Linux List
 help / color / mirror / Atom feed
From: Andreas Hindborg <a.hindborg@kernel.org>
To: "Alice Ryhl" <aliceryhl@google.com>,
	"Lorenzo Stoakes" <ljs@kernel.org>,
	"Liam R. Howlett" <liam@infradead.org>,
	"Miguel Ojeda" <ojeda@kernel.org>,
	"Boqun Feng" <boqun@kernel.org>, "Gary Guo" <gary@garyguo.net>,
	"Björn Roy Baron" <bjorn3_gh@protonmail.com>,
	"Benno Lossin" <lossin@kernel.org>,
	"Trevor Gross" <tmgross@umich.edu>,
	"Danilo Krummrich" <dakr@kernel.org>,
	"Daniel Almeida" <daniel.almeida@collabora.com>,
	"Tamir Duberstein" <tamird@kernel.org>,
	"Alexandre Courbot" <acourbot@nvidia.com>,
	"Onur Özkan" <work@onurozkan.dev>,
	"Will Deacon" <will@kernel.org>,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Mark Rutland" <mark.rutland@arm.com>,
	"Lyude Paul" <lyude@redhat.com>,
	"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
	"Arve Hjønnevåg" <arve@android.com>,
	"Todd Kjos" <tkjos@android.com>,
	"Christian Brauner" <brauner@kernel.org>,
	"Carlos Llamas" <cmllamas@google.com>, "Zi Yan" <ziy@nvidia.com>
Cc: linux-mm@kvack.org, rust-for-linux@vger.kernel.org,
	 linux-kernel@vger.kernel.org, driver-core@lists.linux.dev,
	 Andreas Hindborg <a.hindborg@kernel.org>
Subject: [PATCH 04/11] rust: page: use byte-wise atomic operations in raw accessors
Date: Fri, 20 Nov 2026 11:27:03 +0100	[thread overview]
Message-ID: <20261120-page-shared-v1-4-66f8c4c62822@kernel.org> (raw)
In-Reply-To: <20261120-page-shared-v1-0-66f8c4c62822@kernel.org>

The raw accessors of `Page` are used to access pages that may be mapped to
user space or shared with devices. A plain access to such a page is a data
race when the other party writes to the page.

Add the private method `Page::with_page_as_sysmem()`, which maps a range
of a page and passes it to a closure as a `SysMem` view. Access the data
of the page through this view in `Page::read_raw()`, `Page::write_raw()`
and `Page::fill_zero_raw()`, so that all accesses are byte-wise atomic.
Zero the data with `SysMem::fill_zero()`.

Narrow the safety requirements of these methods, and of
`Page::read_sysmem()` and `Page::write_sysmem()`, accordingly: a call must
not race with a plain access to the same data, but it is allowed to race
with other non-plain accesses.

`Page::copy_from_user_slice_raw()` still writes to the page with plain
accesses. State that in the safety requirements.

Signed-off-by: Andreas Hindborg <a.hindborg@kernel.org>
---
 rust/kernel/page.rs | 153 ++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 107 insertions(+), 46 deletions(-)

diff --git a/rust/kernel/page.rs b/rust/kernel/page.rs
index 3c48ecde4d14..ff191bfcd3f9 100644
--- a/rust/kernel/page.rs
+++ b/rust/kernel/page.rs
@@ -334,28 +334,74 @@ fn with_pointer_into_page<T>(
         }
     }
 
+    /// Runs a piece of code with a range of this page mapped as a [`SysMem`] view.
+    ///
+    /// If `f` is called, then it is called with a view of the `len` bytes of the page that start
+    /// at `offset`. Accesses through the view are volatile or byte-wise atomic, so the view is
+    /// suitable for a page whose data is subject to concurrent writes by hardware or user space.
+    /// The view is valid for the duration of the call to `f`.
+    ///
+    /// If `offset` and `len` refer to a region outside of this page, then this method returns
+    /// [`EINVAL`] and does not call `f`.
+    ///
+    /// # Safety
+    ///
+    /// * Callers must ensure that no plain access to the mapped range races with an access
+    ///   performed through the view, see [`SysMem`].
+    /// * Callers must ensure that no mutable reference to the data of the page that overlaps with
+    ///   the mapped range exists during this call. If `f` writes through the view, no reference to
+    ///   that data may exist during this call. Such references can be obtained from a
+    ///   [`Page<Exclusive>`], see [`Page::as_slice`].
+    /// * If the page is a [`Page<Exclusive>`], `f` must only write initialized bytes through the
+    ///   view.
+    /// * Callers must ensure that the view is only used on the current task, as this method may
+    ///   use a local mapping.
+    unsafe fn with_page_as_sysmem<T>(
+        &self,
+        offset: usize,
+        len: usize,
+        f: impl FnOnce(SysMem<'_, [u8]>) -> Result<T>,
+    ) -> Result<T> {
+        self.with_pointer_into_page(offset, len, move |ptr| {
+            // SAFETY:
+            // - If `with_pointer_into_page` calls into this closure, then it has performed a
+            //   bounds check and guarantees that `ptr` is valid for `len` bytes for the duration
+            //   of the closure. A byte slice has an alignment of 1.
+            // - By the safety requirements of this function, no plain access to the range races
+            //   with an access through the view.
+            f(unsafe { SysMem::new(ptr::slice_from_raw_parts_mut(ptr, len)) })
+        })
+    }
+
     /// Maps the page and reads from it into the given buffer.
     ///
     /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
     /// outside of the page, then this call returns [`EINVAL`].
     ///
+    /// The copy is byte-wise atomic, so it is well-defined even if the data of the page is
+    /// concurrently written by hardware, by user space, or by other kernel code using atomic or
+    /// volatile accesses. In that case, the data that is read may be arbitrary.
+    ///
     /// # Safety
     ///
-    /// * Callers must ensure that `dst` is valid for writing `len` bytes.
-    /// * Callers must ensure that this call does not race with a write to the same page that
-    ///   overlaps with this read.
+    /// * Callers must ensure that `dst` is valid for writing `len` bytes, and that this memory
+    ///   does not overlap with the data of the page.
+    /// * Callers must ensure that this call does not race with a plain write to the same page that
+    ///   overlaps with this read, see [`SysMem`].
     /// * Callers must ensure that no mutable reference to the data of the page that overlaps with
     ///   this read exists during this call.
     pub unsafe fn read_raw(&self, dst: *mut u8, offset: usize, len: usize) -> Result {
-        self.with_pointer_into_page(offset, len, move |src| {
-            // SAFETY: If `with_pointer_into_page` calls into this closure, then
-            // it has performed a bounds check and guarantees that `src` is
-            // valid for `len` bytes.
-            //
-            // There caller guarantees that there is no data race.
-            unsafe { ptr::copy_nonoverlapping(src, dst, len) };
+        let copy = |src: SysMem<'_, [u8]>| {
+            // SAFETY: By the safety requirements of this function, `dst` is valid for writing
+            // `len` bytes, which is equal to `src.size()`, and it does not overlap with `src`.
+            unsafe { SysMemBackend::copy_from_io(src, dst) };
             Ok(())
-        })
+        };
+
+        // SAFETY: By the safety requirements of this function, no plain write races with the read
+        // through the view, and no mutable reference to the data that is read exists. `copy` does
+        // not write through the view, and it only uses the view on the current task.
+        unsafe { self.with_page_as_sysmem(offset, len, copy) }
     }
 
     /// Maps the page and reads from it into the given system memory region using byte-wise atomic
@@ -366,14 +412,14 @@ pub unsafe fn read_raw(&self, dst: *mut u8, offset: usize, len: usize) -> Result
     ///
     /// The writes to `dst` are byte-wise atomic, so they are well-defined even if `dst` is
     /// concurrently accessed by hardware, by user space, or by other kernel code using atomic or
-    /// volatile accesses. The reads from the page are plain accesses, so the page must not be
-    /// written to concurrently.
+    /// volatile accesses. The reads from the page are byte-wise atomic as well.
     ///
     /// # Safety
     ///
     /// Callers must ensure that:
     ///
-    /// - This call does not race with a write to the source page that overlaps with this read.
+    /// - This call does not race with a plain write to the source page that overlaps with this
+    ///   read, see [`SysMem`].
     /// - No mutable reference to the data of the page that overlaps with this read exists during
     ///   this call.
     /// - The memory region of `dst` does not overlap with the memory of the page.
@@ -393,8 +439,8 @@ pub unsafe fn read_raw(&self, dst: *mut u8, offset: usize, len: usize) -> Result
     /// // way while `dst` is alive.
     /// let dst = unsafe { SysMem::new(&raw mut buf as *mut [u8]) };
     ///
-    /// // SAFETY: `page` is not written to during this call, no reference to the data of `page`
-    /// // exists, and `buf` is not part of the page.
+    /// // SAFETY: No plain write to `page` happens during this call, no reference to the data of
+    /// // `page` exists, and `buf` is not part of the page.
     /// unsafe { page.read_sysmem(dst, 0)? };
     /// assert_eq!(buf, [0u8; 16]);
     /// # Ok::<(), Error>(())
@@ -405,8 +451,8 @@ pub unsafe fn read_sysmem(&self, dst: SysMem<'_, [u8]>, offset: usize) -> Result
             // - If `with_pointer_into_page` calls into this closure, then it has performed a
             //   bounds check and guarantees that `src` is valid for `dst.len()` bytes, which is
             //   equal to `dst.size()`.
-            // - By the safety requirements of this function, the page is not written to during
-            //   this call, so `src` is valid for volatile reads.
+            // - By the safety requirements of this function, no plain write to the page races
+            //   with this call, so `src` is valid for volatile reads.
             // - By the safety requirements of this function, `src` does not overlap with `dst`.
             unsafe { SysMemBackend::copy_to_io(dst, src) };
             Ok(())
@@ -418,25 +464,34 @@ pub unsafe fn read_sysmem(&self, dst: SysMem<'_, [u8]>, offset: usize) -> Result
     /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
     /// outside of the page, then this call returns [`EINVAL`].
     ///
+    /// The copy is byte-wise atomic, so it is well-defined even if the data of the page is
+    /// concurrently accessed by hardware, by user space, or by other kernel code using atomic or
+    /// volatile accesses.
+    ///
     /// # Safety
     ///
-    /// * Callers must ensure that `src` is valid for reading `len` bytes.
-    /// * Callers must ensure that this call does not race with a read or write to the same page
-    ///   that overlaps with this write.
+    /// * Callers must ensure that `src` is valid for reading `len` bytes, and that this memory
+    ///   does not overlap with the data of the page.
+    /// * Callers must ensure that this call does not race with a plain read or write to the same
+    ///   page that overlaps with this write, see [`SysMem`].
     /// * Callers must ensure that no reference to the data of the page that overlaps with this
     ///   write exists during this call. Such references can be obtained from a
     ///   [`Page<Exclusive>`], see [`Page::as_slice`].
     /// * If the page is a [`Page<Exclusive>`], callers must ensure that the `len` bytes at `src`
     ///   are initialized.
     pub unsafe fn write_raw(&self, src: *const u8, offset: usize, len: usize) -> Result {
-        self.with_pointer_into_page(offset, len, move |dst| {
-            // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
-            // bounds check and guarantees that `dst` is valid for `len` bytes.
-            //
-            // There caller guarantees that there is no data race.
-            unsafe { ptr::copy_nonoverlapping(src, dst, len) };
+        let copy = |dst: SysMem<'_, [u8]>| {
+            // SAFETY: By the safety requirements of this function, `src` is valid for reading
+            // `len` bytes, which is equal to `dst.size()`, and it does not overlap with `dst`.
+            unsafe { SysMemBackend::copy_to_io(dst, src) };
             Ok(())
-        })
+        };
+
+        // SAFETY: By the safety requirements of this function, no plain access races with the
+        // write through the view, no reference to the data that is written exists, and the bytes
+        // written are initialized if the page requires it. `copy` only uses the view on the
+        // current task.
+        unsafe { self.with_page_as_sysmem(offset, len, copy) }
     }
 
     /// Maps the page and writes into it from the given system memory region using byte-wise atomic
@@ -447,15 +502,14 @@ pub unsafe fn write_raw(&self, src: *const u8, offset: usize, len: usize) -> Res
     ///
     /// The reads from `src` are byte-wise atomic, so they are well-defined even if `src` is
     /// concurrently accessed by hardware, by user space, or by other kernel code using atomic or
-    /// volatile accesses. The writes to the page are plain accesses, so the page must not be read
-    /// from or written to concurrently.
+    /// volatile accesses. The writes to the page are byte-wise atomic as well.
     ///
     /// # Safety
     ///
     /// Callers must ensure that:
     ///
-    /// - This call does not race with a read or write to the destination page that overlaps with
-    ///   this write.
+    /// - This call does not race with a plain read or write to the destination page that overlaps
+    ///   with this write, see [`SysMem`].
     /// - No reference to the data of the page that overlaps with this write exists during this
     ///   call. Such references can be obtained from a [`Page<Exclusive>`], see
     ///   [`Page::as_slice`].
@@ -483,8 +537,9 @@ pub unsafe fn write_raw(&self, src: *const u8, offset: usize, len: usize) -> Res
     ///     )
     /// };
     ///
-    /// // SAFETY: `page` is not accessed in any other way during these calls, no reference to the
-    /// // data of `page` exists, and neither `data` nor `buf` is part of the page.
+    /// // SAFETY: No plain access to `page` happens during these calls, no reference to the data
+    /// // of `page` exists, `data` is initialized, and neither `data` nor `buf` is part of the
+    /// // page.
     /// unsafe {
     ///     page.write_sysmem(src, 0)?;
     ///     page.read_sysmem(dst, 0)?;
@@ -498,8 +553,8 @@ pub unsafe fn write_sysmem(&self, src: SysMem<'_, [u8]>, offset: usize) -> Resul
             // - If `with_pointer_into_page` calls into this closure, then it has performed a
             //   bounds check and guarantees that `dst` is valid for `src.len()` bytes, which is
             //   equal to `src.size()`.
-            // - By the safety requirements of this function, the page is not read from or written
-            //   to during this call, so `dst` is valid for volatile writes.
+            // - By the safety requirements of this function, no plain access to the page races
+            //   with this call, so `dst` is valid for volatile writes.
             // - By the safety requirements of this function, `dst` does not overlap with `src`.
             unsafe { SysMemBackend::copy_from_io(src, dst) };
             Ok(())
@@ -511,22 +566,27 @@ pub unsafe fn write_sysmem(&self, src: SysMem<'_, [u8]>, offset: usize) -> Resul
     /// This method will perform bounds checks on the page offset. If `offset .. offset+len` goes
     /// outside of the page, then this call returns [`EINVAL`].
     ///
+    /// The writes are byte-wise atomic, so they are well-defined even if the data of the page is
+    /// concurrently accessed by hardware, by user space, or by other kernel code using atomic or
+    /// volatile accesses.
+    ///
     /// # Safety
     ///
-    /// * Callers must ensure that this call does not race with a read or write to the same page
-    ///   that overlaps with this write.
+    /// * Callers must ensure that this call does not race with a plain read or write to the same
+    ///   page that overlaps with this write, see [`SysMem`].
     /// * Callers must ensure that no reference to the data of the page that overlaps with this
     ///   write exists during this call. Such references can be obtained from a
     ///   [`Page<Exclusive>`], see [`Page::as_slice`].
     pub unsafe fn fill_zero_raw(&self, offset: usize, len: usize) -> Result {
-        self.with_pointer_into_page(offset, len, move |dst| {
-            // SAFETY: If `with_pointer_into_page` calls into this closure, then it has performed a
-            // bounds check and guarantees that `dst` is valid for `len` bytes.
-            //
-            // There caller guarantees that there is no data race.
-            unsafe { ptr::write_bytes(dst, 0u8, len) };
+        let fill = |dst: SysMem<'_, [u8]>| {
+            dst.fill_zero();
             Ok(())
-        })
+        };
+
+        // SAFETY: By the safety requirements of this function, no plain access races with the
+        // write through the view, and no reference to the data that is written exists. The bytes
+        // written are initialized. `fill` only uses the view on the current task.
+        unsafe { self.with_page_as_sysmem(offset, len, fill) }
     }
 
     /// Copies data from userspace into this page.
@@ -540,7 +600,8 @@ pub unsafe fn fill_zero_raw(&self, offset: usize, len: usize) -> Result {
     /// # Safety
     ///
     /// * Callers must ensure that this call does not race with a read or write to the same page
-    ///   that overlaps with this write.
+    ///   that overlaps with this write. This includes byte-wise atomic accesses, as this method
+    ///   writes to the page with plain accesses.
     /// * Callers must ensure that no reference to the data of the page that overlaps with this
     ///   write exists during this call. Such references can be obtained from a
     ///   [`Page<Exclusive>`], see [`Page::as_slice`].

-- 
2.54.0



  parent reply	other threads:[~2026-11-20 10:28 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-20 10:26 [PATCH 00/11] rust: page: improve access methods for racy memory regions Andreas Hindborg
2026-11-20 10:27 ` [PATCH 01/11] rust: page: tighten safety requirements of `SysMem` copy methods Andreas Hindborg
2026-11-20 10:27 ` [PATCH 02/11] rust: sync: atomic: add memset Andreas Hindborg
2026-11-20 10:27 ` [PATCH 03/11] rust: io: add `SysMem::fill_zero` Andreas Hindborg
2026-11-20 10:27 ` Andreas Hindborg [this message]
2026-11-20 10:27 ` [PATCH 05/11] rust: uaccess: take a raw pointer in `UserSliceReader::read_raw` Andreas Hindborg
2026-11-20 10:27 ` [PATCH 06/11] rust: uaccess: add `UserSliceReader::read_sysmem` Andreas Hindborg
2026-11-20 10:27 ` [PATCH 07/11] rust: page: add `Shared` page kind Andreas Hindborg
2026-11-20 10:27 ` [PATCH 08/11] rust: io: add `SysMemLocal` Andreas Hindborg
2026-11-20 10:27 ` [PATCH 09/11] rust: page: add `Page::with_page_as_sysmem_local` Andreas Hindborg
2026-11-20 10:27 ` [PATCH 10/11] rust_binder: access page data through `SysMemLocal` Andreas Hindborg
2026-11-20 10:27 ` [PATCH 11/11] rust: page: remove `Page::copy_from_user_slice_raw` Andreas Hindborg

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261120-page-shared-v1-4-66f8c4c62822@kernel.org \
    --to=a.hindborg@kernel.org \
    --cc=acourbot@nvidia.com \
    --cc=aliceryhl@google.com \
    --cc=arve@android.com \
    --cc=bjorn3_gh@protonmail.com \
    --cc=boqun@kernel.org \
    --cc=brauner@kernel.org \
    --cc=cmllamas@google.com \
    --cc=dakr@kernel.org \
    --cc=daniel.almeida@collabora.com \
    --cc=driver-core@lists.linux.dev \
    --cc=gary@garyguo.net \
    --cc=gregkh@linuxfoundation.org \
    --cc=liam@infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=ljs@kernel.org \
    --cc=lossin@kernel.org \
    --cc=lyude@redhat.com \
    --cc=mark.rutland@arm.com \
    --cc=ojeda@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rust-for-linux@vger.kernel.org \
    --cc=tamird@kernel.org \
    --cc=tkjos@android.com \
    --cc=tmgross@umich.edu \
    --cc=will@kernel.org \
    --cc=work@onurozkan.dev \
    --cc=ziy@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox