* Re: [PATCH v9 3/5] rust: add bitmap API. @ 2025-05-29 19:42 Pekka Ristola 2025-05-29 20:07 ` Boqun Feng 0 siblings, 1 reply; 5+ messages in thread From: Pekka Ristola @ 2025-05-29 19:42 UTC (permalink / raw) To: Burak Emir Cc: Yury Norov, Kees Cook, Rasmus Villemoes, Viresh Kumar, Miguel Ojeda, Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross, Gustavo A . R . Silva, rust-for-linux, linux-kernel, linux-hardening, Pekka Ristola On Mon, 26 May 2025 15:01:32 +0000, Burak Emir wrote: > Provides an abstraction for C bitmap API and bitops operations. > > This commit enables a Rust implementation of an Android Binder > data structure from commit 15d9da3f818c ("binder: use bitmap for faster > descriptor lookup"), which can be found in drivers/android/dbitmap.h. > It is a step towards upstreaming the Rust port of Android Binder driver. > > We follow the C Bitmap API closely in naming and semantics, with > a few differences that take advantage of Rust language facilities > and idioms: > > * We leverage Rust type system guarantees as follows: > > * all (non-atomic) mutating operations require a &mut reference which > amounts to exclusive access. > > * the Bitmap type implements Send. This enables transferring > ownership between threads and is needed for Binder. > > * the Bitmap type implements Sync, which enables passing shared > references &Bitmap between threads. Atomic operations can be > used to safely modify from multiple threads (interior > mutability), though without ordering guarantees. > > * The Rust API uses `{set,clear}_bit` vs `{set,clear}_bit_atomic` as > names, which differs from the C naming convention which uses > set_bit for atomic vs __set_bit for non-atomic. > > * we include enough operations for the API to be useful, but not all > operations are exposed yet in order to avoid dead code. The missing > ones can be added later. > > * We follow the C API closely with a fine-grained approach to safety: > > * Low-level bit-ops get a safe API with bounds checks. Calling with > an out-of-bounds arguments to {set,clear}_bit becomes a no-op and > get logged as errors. > > * We introduce a RUST_BITMAP_HARDENED config, which > causes invocations with out-of-bounds arguments to panic. > > * methods correspond to find_* C methods tolerate out-of-bounds > since the C implementation does. Also here, we log out-of-bounds > arguments as errors and panic in RUST_BITMAP_HARDENED mode. > > * We add a way to "borrow" bitmaps from C in Rust, to make C bitmaps > that were allocated in C directly usable in Rust code (`CBitmap`). > > * the Rust API is optimized to represent the bitmap inline if it would > fit into a pointer. This saves allocations which is > relevant in the Binder use case. > > The underlying C bitmap is*not* exposed, and must never be exposed > (except in tests). Exposing the representation of the owned bitmap would > lose static guarantees. > > An alternative route of vendoring an existing Rust bitmap package was > considered but suboptimal overall. Reusing the C implementation is > preferable for a basic data structure like bitmaps. It enables Rust > code to be a lot more similar and predictable with respect to C code > that uses the same data structures and enables the use of code that > has been tried-and-tested in the kernel, with the same performance > characteristics whenever possible. > > We use the `usize` type for sizes and indices into the bitmap, > because Rust generally always uses that type for indices and lengths > and it will be more convenient if the API accepts that type. This means > that we need to perform some casts to/from u32 and usize, since the C > headers use unsigned int instead of size_t/unsigned long for these > numbers in some places. > > Adds new MAINTAINERS section BITMAP API [RUST]. > > Suggested-by: Alice Ryhl <aliceryhl@google.com> > Suggested-by: Yury Norov <yury.norov@gmail.com> > Signed-off-by: Burak Emir <bqe@google.com> > --- > MAINTAINERS | 7 + > rust/kernel/bitmap.rs | 554 +++++++++++++++++++++++++++++++++++++ > rust/kernel/lib.rs | 1 + > security/Kconfig.hardening | 10 + > 4 files changed, 572 insertions(+) > create mode 100644 rust/kernel/bitmap.rs > > diff --git a/MAINTAINERS b/MAINTAINERS > index 04d6727e944c..565eaa015d9e 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -4127,6 +4127,13 @@ S: Maintained > F: rust/helpers/bitmap.c > F: rust/helpers/cpumask.c > > +BITMAP API [RUST] > +M: Alice Ryhl <aliceryhl@google.com> > +M: Burak Emir <bqe@google.com> > +R: Yury Norov <yury.norov@gmail.com> > +S: Maintained > +F: rust/kernel/bitmap.rs > + > BITOPS API > M: Yury Norov <yury.norov@gmail.com> > R: Rasmus Villemoes <linux@rasmusvillemoes.dk> > diff --git a/rust/kernel/bitmap.rs b/rust/kernel/bitmap.rs > new file mode 100644 > index 000000000000..a6edd4889518 > --- /dev/null > +++ b/rust/kernel/bitmap.rs > @@ -0,0 +1,554 @@ > +// SPDX-License-Identifier: GPL-2.0 > + > +// Copyright (C) 2025 Google LLC. > + > +//! Rust API for bitmap. > +//! > +//! C headers: [`include/linux/bitmap.h`](srctree/include/linux/bitmap.h). > + > +use crate::alloc::{AllocError, Flags}; > +use crate::bindings; > +use crate::pr_err; > +use core::ptr::NonNull; > + > +/// Represents a C bitmap. Wraps underlying C bitmap API. > +/// > +/// # Invariants > +/// > +/// Must reference a `[c_ulong]` long enough to fit `data.len()` bits. > +pub struct CBitmap { > + _align: [usize; 0], > + data: [()], Interestingly, this zero sized slice layout seems to be fine under Tree Borrows but UB under Stacked Borrows. This slightly modified version[0] that runs in userspace triggers Miri with Stacked Borrows. Though I guess it's fine since Miri doesn't complain with Tree Borrows. [0] https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=0ccf6fd892c044db9b644a77ad9bd1e9 > +} > + > +impl CBitmap { > + /// Borrows a C bitmap. > + /// > + /// # Safety > + /// > + /// * `ptr` holds a non-null address of an initialized array of `unsigned long` > + /// that is large enough to hold `nbits` bits. > + /// * the array must not be freed for the lifetime of this [`CBitmap`] > + /// * concurrent access only happens through atomic operations > + pub unsafe fn from_raw<'a>(ptr: *const usize, nbits: usize) -> &'a CBitmap { > + let data: *const [()] = core::ptr::slice_from_raw_parts(ptr.cast(), nbits); > + unsafe { &*(data as *const CBitmap) } Safety comment is missing here. Running Clippy with `make LLVM=1 CLIPPY=1` finds all the missing safety comments and some other issues as well. [...] > +/// Holds either a pointer to array of `unsigned long` or a small bitmap. > +#[repr(C)] > +union BitmapRepr { > + bitmap: usize, > + ptr: NonNull<usize>, > +} > + > +macro_rules! bitmap_assert { > + ($cond:expr, $($arg:tt)+) => { > + #[cfg(RUST_BITMAP_HARDENED)] > + assert!($e, $($arg)*); Should be $cond instead of $e. > + } > +} > + > +macro_rules! bitmap_assert_return { > + ($cond:expr, $($arg:tt)+) => { > + #[cfg(RUST_BITMAP_HARDENED)] > + assert!($e, $($arg)*); Same here. > + > + #[cfg(not(RUST_BITMAP_HARDENED))] > + if !($cond) { > + pr_err!($($arg)*); > + return > + } > + } > +} > + > +/// Represents an owned bitmap. > +/// > +/// Wraps underlying C bitmap API. See [`CBitmap`] for available > +/// methods. > +/// > +/// # Examples > +/// > +/// Basic usage > +/// > +/// ``` > +/// use kernel::alloc::flags::GFP_KERNEL; > +/// use kernel::bitmap::Bitmap; > +/// > +/// let mut b = Bitmap::new(16, GFP_KERNEL)?; > +/// > +/// assert_eq!(16, b.len()); > +/// for i in 0..16 { > +/// if i % 4 == 0 { > +/// b.set_bit(i); > +/// } > +/// } > +/// assert_eq!(Some(0), b.next_bit(0)); > +/// assert_eq!(Some(1), b.next_zero_bit(0)); > +/// assert_eq!(Some(4), b.next_bit(1)); > +/// assert_eq!(Some(5), b.next_zero_bit(4)); > +/// assert_eq!(Some(12), b.last_bit()); > +/// # Ok::<(), Error>(()) > +/// ``` > +/// > +/// # Invariants > +/// > +/// * `inner.nbits` is `<= i32::MAX` and never changes. > +/// * if `inner.nbits <= bindings::BITS_PER_LONG`, then `inner.repr` is > +/// a `usize`. > +/// * otherwise, `inner.repr` holds a non-null pointer to an initialized > +/// array of `unsigned long` that is large enough to hold `nbits` bits. There is no `inner` in the struct. > +pub struct Bitmap { > + /// Representation of bitmap. > + repr: BitmapRepr, > + /// Length of this bitmap. Must be `<= i32::MAX`. > + nbits: usize, > +} > + > +impl core::ops::Deref for Bitmap { > + type Target = CBitmap; > + > + fn deref(&self) -> &CBitmap { > + let ptr = if self.nbits <= bindings::BITS_PER_LONG as _ { > + // SAFETY: Bitmap is represented inline. > + unsafe { core::ptr::addr_of!(self.repr.bitmap) } This can use the raw ref syntax, `&raw const self.repr.bitmap`. > + } else { > + // SAFETY: Bitmap is represented as array of `unsigned long`. > + unsafe { self.repr.ptr.as_ptr() } > + }; > + > + // SAFETY: We got the right pointer and invariants of [`Bitmap`] hold. > + // An inline bitmap is treated like an array with single element. > + unsafe { CBitmap::from_raw(ptr, self.nbits) } > + } > +} > + > +impl core::ops::DerefMut for Bitmap { > + fn deref_mut(&mut self) -> &mut CBitmap { > + let ptr = if self.nbits <= bindings::BITS_PER_LONG as _ { > + // SAFETY: Bitmap is represented inline. > + unsafe { core::ptr::addr_of_mut!(self.repr.bitmap) } Same here, `&raw mut self.repr.bitmap`. [...] > + /// Set bit with index `index`, atomically. > + /// > + /// This is a relaxed atomic operation (no implied memory barriers). > + /// > + /// ATTENTION: The naming convention differs from C, where the corresponding > + /// function is called `set_bit`. > + /// > + /// If RUST_BITMAP_HARDENED is not enabled and `index` is greater than > + /// or equal to `self.len()`, does nothing. > + /// > + /// # Panics > + /// > + /// Panics if RUST_BITMAP_HARDENED is enabled and `index` is greater than > + /// or equal to `self.len()`. > + #[inline] > + pub fn set_bit_atomic(&self, index: usize) { > + bitmap_assert_return!( > + index < self.len(), > + "Bit `index` must be < {}, was {}", > + self.len(), > + index > + ); > + // SAFETY: `index` is within bounds and the caller has ensured that > + // there is no mix of non-atomic and atomic operations. > + unsafe { bindings::set_bit(index as u32, self.as_ptr() as *mut usize) }; Maybe use `.cast_mut()` instead of `as *mut usize`? [...] > + /// Clear `index` bit, atomically. > + /// > + /// This is a relaxed atomic operation (no implied memory barriers). > + /// > + /// ATTENTION: The naming convention differs from C, where the corresponding > + /// function is called `clear_bit`. > + /// > + /// If RUST_BITMAP_HARDENED is not enabled and `index` is greater than > + /// or equal to `self.len()`, does nothing. > + /// > + /// # Panics > + /// > + /// Panics if RUST_BITMAP_HARDENED is enabled and `index` is greater than > + /// or equal to `self.len()`. > + #[inline] > + pub fn clear_bit_atomic(&self, index: usize) { > + bitmap_assert_return!( > + index < self.len(), > + "Bit `index` must be < {}, was {}", > + self.len(), > + index > + ); > + // SAFETY: `index` is within bounds and the caller has ensured that > + // there is no mix of non-atomic and atomic operations. > + unsafe { bindings::clear_bit(index as u32, self.as_ptr() as *mut usize) }; Same here. `cast_mut` won't silently change the type unlike `as` casts so it's a bit safer. > + } > + > + /// Copy `src` into this [`Bitmap`] and set any remaining bits to zero. > + /// > + /// # Examples > + /// > + /// ``` > + /// use kernel::alloc::{AllocError, flags::GFP_KERNEL}; > + /// use kernel::bitmap::Bitmap; > + /// > + /// let mut long_bitmap = Bitmap::new(256, GFP_KERNEL)?; > + // > + /// assert_eq!(None, long_bitmap.last_bit()); > + // > + /// let mut short_bitmap = Bitmap::new(16, GFP_KERNEL)?; > + // > + /// short_bitmap.set_bit(7); > + /// long_bitmap.copy_and_extend(&short_bitmap); > + /// assert_eq!(Some(7), long_bitmap.last_bit()); > + /// > + /// # Ok::<(), AllocError>(()) > + /// ``` > + #[inline] > + pub fn copy_and_extend(&mut self, src: &Bitmap) { > + let len = core::cmp::min(src.nbits, self.len()); > + // SAFETY: access to `self` and `src` is within bounds. > + unsafe { > + bindings::bitmap_copy_and_extend( > + self.as_mut_ptr(), > + src.as_ptr(), > + len as u32, > + self.len() as u32, > + ) Would this cause a data race if `src` is concurrently (atomically) modified? The C function seems to use a plain `memcpy` which is not atomic. > + }; > + } > + > + /// Finds last set bit. > + /// > + /// # Examples > + /// > + /// ``` > + /// use kernel::alloc::{AllocError, flags::GFP_KERNEL}; > + /// use kernel::bitmap::Bitmap; > + /// > + /// let bitmap = Bitmap::new(64, GFP_KERNEL)?; > + /// > + /// match bitmap.last_bit() { > + /// Some(idx) => { > + /// pr_info!("The last bit has index {idx}.\n"); > + /// } > + /// None => { > + /// pr_info!("All bits in this bitmap are 0.\n"); > + /// } > + /// } > + /// # Ok::<(), AllocError>(()) > + /// ``` > + #[inline] > + pub fn last_bit(&self) -> Option<usize> { > + // SAFETY: `_find_next_bit` access is within bounds due to invariant. > + let index = unsafe { bindings::_find_last_bit(self.as_ptr(), self.len()) }; The C function uses non-atomic reads, so this might cause data races too. > + if index >= self.len() { > + None > + } else { > + Some(index) > + } > + } Pekka ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v9 3/5] rust: add bitmap API. 2025-05-29 19:42 [PATCH v9 3/5] rust: add bitmap API Pekka Ristola @ 2025-05-29 20:07 ` Boqun Feng 2025-05-29 20:15 ` Pekka Ristola 0 siblings, 1 reply; 5+ messages in thread From: Boqun Feng @ 2025-05-29 20:07 UTC (permalink / raw) To: Pekka Ristola Cc: Burak Emir, Yury Norov, Kees Cook, Rasmus Villemoes, Viresh Kumar, Miguel Ojeda, Alex Gaynor, Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross, Gustavo A . R . Silva, rust-for-linux, linux-kernel, linux-hardening On Thu, May 29, 2025 at 07:42:09PM +0000, Pekka Ristola wrote: [...] > > + } > > + > > + /// Copy `src` into this [`Bitmap`] and set any remaining bits to zero. > > + /// > > + /// # Examples > > + /// > > + /// ``` > > + /// use kernel::alloc::{AllocError, flags::GFP_KERNEL}; > > + /// use kernel::bitmap::Bitmap; > > + /// > > + /// let mut long_bitmap = Bitmap::new(256, GFP_KERNEL)?; > > + // > > + /// assert_eq!(None, long_bitmap.last_bit()); > > + // > > + /// let mut short_bitmap = Bitmap::new(16, GFP_KERNEL)?; > > + // > > + /// short_bitmap.set_bit(7); > > + /// long_bitmap.copy_and_extend(&short_bitmap); > > + /// assert_eq!(Some(7), long_bitmap.last_bit()); > > + /// > > + /// # Ok::<(), AllocError>(()) > > + /// ``` > > + #[inline] > > + pub fn copy_and_extend(&mut self, src: &Bitmap) { > > + let len = core::cmp::min(src.nbits, self.len()); > > + // SAFETY: access to `self` and `src` is within bounds. > > + unsafe { > > + bindings::bitmap_copy_and_extend( > > + self.as_mut_ptr(), > > + src.as_ptr(), > > + len as u32, > > + self.len() as u32, > > + ) > > Would this cause a data race if `src` is concurrently (atomically) > modified? The C function seems to use a plain `memcpy` which is not atomic. > We need some better documentation on the effect of kernel C's `memcpy()`-like functions regarding data races, but in general a kernel C's `memcpy` can be treated as a volatile one (or per-byte atomic), so it won't cause data race. > > + }; > > + } > > + > > + /// Finds last set bit. > > + /// > > + /// # Examples > > + /// > > + /// ``` > > + /// use kernel::alloc::{AllocError, flags::GFP_KERNEL}; > > + /// use kernel::bitmap::Bitmap; > > + /// > > + /// let bitmap = Bitmap::new(64, GFP_KERNEL)?; > > + /// > > + /// match bitmap.last_bit() { > > + /// Some(idx) => { > > + /// pr_info!("The last bit has index {idx}.\n"); > > + /// } > > + /// None => { > > + /// pr_info!("All bits in this bitmap are 0.\n"); > > + /// } > > + /// } > > + /// # Ok::<(), AllocError>(()) > > + /// ``` > > + #[inline] > > + pub fn last_bit(&self) -> Option<usize> { > > + // SAFETY: `_find_next_bit` access is within bounds due to invariant. > > + let index = unsafe { bindings::_find_last_bit(self.as_ptr(), self.len()) }; > > The C function uses non-atomic reads, so this might cause data races too. > Similar here. Regards, Boqun > > + if index >= self.len() { > > + None > > + } else { > > + Some(index) > > + } > > + } > > Pekka > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v9 3/5] rust: add bitmap API. 2025-05-29 20:07 ` Boqun Feng @ 2025-05-29 20:15 ` Pekka Ristola 0 siblings, 0 replies; 5+ messages in thread From: Pekka Ristola @ 2025-05-29 20:15 UTC (permalink / raw) To: Boqun Feng Cc: Burak Emir, Yury Norov, Kees Cook, Rasmus Villemoes, Viresh Kumar, Miguel Ojeda, Alex Gaynor, Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross, Gustavo A . R . Silva, rust-for-linux, linux-kernel, linux-hardening On Thursday, May 29th, 2025 at 23.07, Boqun Feng <boqun.feng@gmail.com> wrote: > On Thu, May 29, 2025 at 07:42:09PM +0000, Pekka Ristola wrote: > [...] > > > > + } > > > + > > > + /// Copy `src` into this [`Bitmap`] and set any remaining bits to zero. > > > + /// > > > + /// # Examples > > > + /// > > > + /// `+ /// use kernel::alloc::{AllocError, flags::GFP_KERNEL}; + /// use kernel::bitmap::Bitmap; + /// + /// let mut long_bitmap = Bitmap::new(256, GFP_KERNEL)?; + // + /// assert_eq!(None, long_bitmap.last_bit()); + // + /// let mut short_bitmap = Bitmap::new(16, GFP_KERNEL)?; + // + /// short_bitmap.set_bit(7); + /// long_bitmap.copy_and_extend(&short_bitmap); + /// assert_eq!(Some(7), long_bitmap.last_bit()); + /// + /// # Ok::<(), AllocError>(()) + ///` > > > + #[inline] > > > + pub fn copy_and_extend(&mut self, src: &Bitmap) { > > > + let len = core::cmp::min(src.nbits, self.len()); > > > + // SAFETY: access to `self` and `src` is within bounds. > > > + unsafe { > > > + bindings::bitmap_copy_and_extend( > > > + self.as_mut_ptr(), > > > + src.as_ptr(), > > > + len as u32, > > > + self.len() as u32, > > > + ) > > > > Would this cause a data race if `src` is concurrently (atomically) > > modified? The C function seems to use a plain `memcpy` which is not atomic. > > > We need some better documentation on the effect of kernel C's > `memcpy()`-like functions regarding data races, but in general a kernel > C's `memcpy` can be treated as a volatile one (or per-byte atomic), so > it won't cause data race. Ah, thanks for the clarification. Pekka ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v9 0/5] rust: adds Bitmap API, ID pool and bindings
@ 2025-05-26 15:01 Burak Emir
2025-05-26 15:01 ` [PATCH v9 3/5] rust: add bitmap API Burak Emir
0 siblings, 1 reply; 5+ messages in thread
From: Burak Emir @ 2025-05-26 15:01 UTC (permalink / raw)
To: Yury Norov, Kees Cook
Cc: Burak Emir, Rasmus Villemoes, Viresh Kumar, Miguel Ojeda,
Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Gustavo A . R . Silva, rust-for-linux, linux-kernel,
linux-hardening
This series adds a Rust bitmap API for porting the approach from
commit 15d9da3f818c ("binder: use bitmap for faster descriptor lookup")
to Rust. The functionality in dbitmap.h makes use of bitmap and bitops.
The Rust bitmap API provides a safe abstraction to underlying bitmap
and bitops operations. For now, only includes method necessary for
dbitmap.h, more can be added later. We perform bounds checks for
hardening, violations are programmer errors that result in panics.
We include set_bit_atomic and clear_bit_atomic operations. One has
to avoid races with non-atomic operations, which is ensure by the
Rust type system: either callers have shared references &bitmap in
which case the mutations are atomic operations. Or there is a
exclusive reference &mut bitmap, in which case there is no concurrent
access.
This version includes an optimization to represent the bitmap inline,
as suggested by Yury.
We ran a simple benchmark (sample size 32), with RUST_BITMAP_HARDENED=n
on a x86_64 production machine that was not running other processes.
Results for random-filled bitmap:
+------------+------+-----------+--------------+-----------+-----------+
| Benchmark | Code | Mean (ns) | Std Dev (ns) | 95% CI Lo | 95% CI Hi |
+------------+------+-----------+--------------+-----------+-----------+
| find_bit/ | C | 5.18 | 0.32 | 5.17 | 5.18 |
| next_bit | Rust | 5.30 | 0.37 | 5.30 | 5.31 |
+------------+------+-----------+--------------+-----------+-----------+
| find_zero/ | C | 5.66 | 0.33 | 5.66 | 5.67 |
| next_zero | Rust | 5.88 | 0.32 | 5.88 | 5.89 |
+------------+------+-----------+--------------+-----------+-----------+
Results for sparse bitmap:
+------------+------+-----------+--------------+-----------+-----------+
| Benchmark | Code | Mean (ns) | Std Dev (ns) | 95% CI Lo | 95% CI Hi |
+------------+------+-----------+--------------+-----------+-----------+
| find_bit/ | C | 22.51 | 12.34 | 22.38 | 22.65 |
| next_bit | Rust | 30.53 | 20.44 | 30.30 | 30.75 |
+------------+------+-----------+--------------+-----------+-----------+
| find_zero/ | C | 5.69 | 0.22 | 5.68 | 5.69 |
| next_zero | Rust | 5.68 | 0.29 | 5.68 | 5.68 |
+------------+------+-----------+--------------+-----------+-----------+
We introduce a Rust API that would replace (dbitmap.h) in file
id_pool.rs. This data structure is tightly coupled to the bitmap API.
Includes an example of usage that requires releasing a spinlock, as
expected in Binder driver.
This is v9 of a patch introducing Rust bitmap API [v7]. Thanks
for all the helpful comments, this series has improved significantly
as a result of your work.
Changes v8 --> v9:
- added a new type `CBitmap` that makes any C bitmap accessible with
the same API, and add Deref so both Bitmap and
CBitmap can share the same implementation. Full credit for this
goes to Alice who suggested idea and code.
- added config dependency on CONFIG_RUST that was missing from
CONIG_FIND_BIT_BENCHMARK_RUST.
- implemented Send for Bitmap, it is actually needed by Binder.
- reworded commit msg for clarity.
- removed unsafe for atomic ops.
- renamed `bitmap_hardening_assert` to `bitmap_assert` and make operations
do nothing and log error when RUST_BITMAP_HARDENED is off.
- update author information in find_bit_benchmark_rust.rs
- Various improvements to id_pool, better names and comments.
Changes v7 --> v8:
- added Acked-by for bindings patches
- add RUST_BITMAP_HARDENED config, making the extra bound-checks configurable.
This is added to security/Kconfig.hardening
- changed checks of `index` return value to >=
- removed change to FIND_BIT_BENCHMARK
Changes v6 --> v7:
- Added separate unit tests in bitmap.rs and benchmark module,
following the example in find_bit_benchmark.c
- Added discussion about using vendored bitset to commit message.
- Refined warning about naming convention
Changes v5 --> v6:
- Added SAFETY comment for atomic operations.
- Added missing volatile to bitops set_bit and clear_bit bindings.
- Fixed condition on `nbits` to be <= i32::MAX, update SAFETY comments.
- Readability improvements.
- Updated doc comments wording and indentation.
Changes v4 --> v5: (suggested by Yury and Alice)
- rebased on next-20250318
- split MAINTAINERS changes
- no dependencies on [1] and [2] anymore - Viresh,
please do add a separate section if you want to maintain cpumask.rs
separately.
- imports atomic and non-atomic variants, introduces a naming convention
set_bit and set_bit_atomic on the Rust side.
- changed naming and comments. Keeping `new`.
- change dynamic_id_pool to id_pool
- represent bitmap inline when possible
- add some more tests
- add myself to M: line for the Rust abstractions
Changes v3 --> v4:
- Rebased on Viresh's v3 [2].
- split into multiple patches, separate Rust and bindings. (Yury)
- adds dynamic_id_pool.rs to show the Binder use case. (Yury)
- include example usage that requires release of spinlock (Alice)
- changed bounds checks to `assert!`, shorter (Yury)
- fix param names in binding helpers. (Miguel)
- proper rustdoc formatting, and use examples as kunit tests. (Miguel)
- reduce number of Bitmap methods, and simplify API through
use Option<usize> to handle the "not found" case.
- make Bitmap pointer accessors private, so Rust Bitmap API
provides an actual abstraction boundary (Tamir)
- we still return `AllocError` in `Bitmap::new` in case client code
asks for a size that is too large. Intentionally
different from other bounds checks because it is not about
access but allocation, and we expect that client code need
never handle AllocError and nbits > u32::MAX situations
differently.
Changes v2 --> v3:
- change `bitmap_copy` to `copy_from_bitmap_and_extend` which
zeroes out extra bits. This enables dbitmap shrink and grow use
cases while offering a consistent and understandable Rust API for
other uses (Alice)
Changes v1 --> v2:
- Rebased on Yury's v2 [1] and Viresh's v3 [2] changes related to
bitmap.
- Removed import of `bindings::*`, keeping only prefix (Miguel)
- Renamed panic methods to make more explicit (Miguel)
- use markdown in doc comments and added example/kunit test (Miguel)
- Added maintainer section for BITOPS API BINDINGS [RUST] (Yury)
- Added M: entry for bitmap.rs which goes to Alice (Viresh, Alice)
- Changed calls from find_* to _find_*, removed helpers (Yury)
- Use non-atomic __set_bit and __clear_bit from Bitmap Rust API (Yury)
Link [1] https://lore.kernel.org/all/20250224233938.3158-1-yury.norov@gmail.com/
Link [2] https://lore.kernel.org/rust-for-linux/cover.1742296835.git.viresh.kumar@linaro.org/
Link [v8] https://lore.kernel.org/rust-for-linux/20250519161712.2609395-1-bqe@google.com/
Burak Emir (6):
rust: add bindings for bitmap.h
rust: add bindings for bitops.h
rust: add bitmap API.
bitmap: fix find_bit_benchmark module name and config description
rust: add find_bit_benchmark_rust module.
rust: add dynamic ID pool abstraction for bitmap
MAINTAINERS | 15 ++
lib/Kconfig.debug | 18 +-
lib/Makefile | 1 +
lib/find_bit_benchmark_rust.rs | 94 +++++++
rust/bindings/bindings_helper.h | 2 +
rust/helpers/bitmap.c | 9 +
rust/helpers/bitops.c | 23 ++
rust/helpers/helpers.c | 2 +
rust/kernel/bitmap.rs | 429 ++++++++++++++++++++++++++++++++
rust/kernel/id_pool.rs | 201 +++++++++++++++
rust/kernel/lib.rs | 2 +
security/Kconfig.hardening | 11 +
12 files changed, 805 insertions(+), 2 deletions(-)
create mode 100644 lib/find_bit_benchmark_rust.rs
create mode 100644 rust/helpers/bitmap.c
create mode 100644 rust/helpers/bitops.c
create mode 100644 rust/kernel/bitmap.rs
create mode 100644 rust/kernel/id_pool.rs
--
2.49.0.1101.gccaa498523-goog
Burak Emir (5):
rust: add bindings for bitmap.h
rust: add bindings for bitops.h
rust: add bitmap API.
rust: add find_bit_benchmark_rust module.
rust: add dynamic ID pool abstraction for bitmap
MAINTAINERS | 15 +
lib/Kconfig.debug | 13 +
lib/Makefile | 1 +
lib/find_bit_benchmark_rust.rs | 94 ++++++
rust/bindings/bindings_helper.h | 2 +
rust/helpers/bitmap.c | 9 +
rust/helpers/bitops.c | 23 ++
rust/helpers/helpers.c | 2 +
rust/kernel/bitmap.rs | 568 ++++++++++++++++++++++++++++++++
rust/kernel/id_pool.rs | 222 +++++++++++++
rust/kernel/lib.rs | 2 +
security/Kconfig.hardening | 10 +
12 files changed, 961 insertions(+)
create mode 100644 lib/find_bit_benchmark_rust.rs
create mode 100644 rust/helpers/bitmap.c
create mode 100644 rust/helpers/bitops.c
create mode 100644 rust/kernel/bitmap.rs
create mode 100644 rust/kernel/id_pool.rs
--
2.49.0.1151.ga128411c76-goog
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH v9 3/5] rust: add bitmap API. 2025-05-26 15:01 [PATCH v9 0/5] rust: adds Bitmap API, ID pool and bindings Burak Emir @ 2025-05-26 15:01 ` Burak Emir 2025-05-27 16:35 ` Boqun Feng 0 siblings, 1 reply; 5+ messages in thread From: Burak Emir @ 2025-05-26 15:01 UTC (permalink / raw) To: Yury Norov, Kees Cook Cc: Burak Emir, Rasmus Villemoes, Viresh Kumar, Miguel Ojeda, Alex Gaynor, Boqun Feng, Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross, Gustavo A . R . Silva, rust-for-linux, linux-kernel, linux-hardening Provides an abstraction for C bitmap API and bitops operations. This commit enables a Rust implementation of an Android Binder data structure from commit 15d9da3f818c ("binder: use bitmap for faster descriptor lookup"), which can be found in drivers/android/dbitmap.h. It is a step towards upstreaming the Rust port of Android Binder driver. We follow the C Bitmap API closely in naming and semantics, with a few differences that take advantage of Rust language facilities and idioms: * We leverage Rust type system guarantees as follows: * all (non-atomic) mutating operations require a &mut reference which amounts to exclusive access. * the Bitmap type implements Send. This enables transferring ownership between threads and is needed for Binder. * the Bitmap type implements Sync, which enables passing shared references &Bitmap between threads. Atomic operations can be used to safely modify from multiple threads (interior mutability), though without ordering guarantees. * The Rust API uses `{set,clear}_bit` vs `{set,clear}_bit_atomic` as names, which differs from the C naming convention which uses set_bit for atomic vs __set_bit for non-atomic. * we include enough operations for the API to be useful, but not all operations are exposed yet in order to avoid dead code. The missing ones can be added later. * We follow the C API closely with a fine-grained approach to safety: * Low-level bit-ops get a safe API with bounds checks. Calling with an out-of-bounds arguments to {set,clear}_bit becomes a no-op and get logged as errors. * We introduce a RUST_BITMAP_HARDENED config, which causes invocations with out-of-bounds arguments to panic. * methods correspond to find_* C methods tolerate out-of-bounds since the C implementation does. Also here, we log out-of-bounds arguments as errors and panic in RUST_BITMAP_HARDENED mode. * We add a way to "borrow" bitmaps from C in Rust, to make C bitmaps that were allocated in C directly usable in Rust code (`CBitmap`). * the Rust API is optimized to represent the bitmap inline if it would fit into a pointer. This saves allocations which is relevant in the Binder use case. The underlying C bitmap is *not* exposed, and must never be exposed (except in tests). Exposing the representation of the owned bitmap would lose static guarantees. An alternative route of vendoring an existing Rust bitmap package was considered but suboptimal overall. Reusing the C implementation is preferable for a basic data structure like bitmaps. It enables Rust code to be a lot more similar and predictable with respect to C code that uses the same data structures and enables the use of code that has been tried-and-tested in the kernel, with the same performance characteristics whenever possible. We use the `usize` type for sizes and indices into the bitmap, because Rust generally always uses that type for indices and lengths and it will be more convenient if the API accepts that type. This means that we need to perform some casts to/from u32 and usize, since the C headers use unsigned int instead of size_t/unsigned long for these numbers in some places. Adds new MAINTAINERS section BITMAP API [RUST]. Suggested-by: Alice Ryhl <aliceryhl@google.com> Suggested-by: Yury Norov <yury.norov@gmail.com> Signed-off-by: Burak Emir <bqe@google.com> --- MAINTAINERS | 7 + rust/kernel/bitmap.rs | 554 +++++++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 1 + security/Kconfig.hardening | 10 + 4 files changed, 572 insertions(+) create mode 100644 rust/kernel/bitmap.rs diff --git a/MAINTAINERS b/MAINTAINERS index 04d6727e944c..565eaa015d9e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4127,6 +4127,13 @@ S: Maintained F: rust/helpers/bitmap.c F: rust/helpers/cpumask.c +BITMAP API [RUST] +M: Alice Ryhl <aliceryhl@google.com> +M: Burak Emir <bqe@google.com> +R: Yury Norov <yury.norov@gmail.com> +S: Maintained +F: rust/kernel/bitmap.rs + BITOPS API M: Yury Norov <yury.norov@gmail.com> R: Rasmus Villemoes <linux@rasmusvillemoes.dk> diff --git a/rust/kernel/bitmap.rs b/rust/kernel/bitmap.rs new file mode 100644 index 000000000000..a6edd4889518 --- /dev/null +++ b/rust/kernel/bitmap.rs @@ -0,0 +1,554 @@ +// SPDX-License-Identifier: GPL-2.0 + +// Copyright (C) 2025 Google LLC. + +//! Rust API for bitmap. +//! +//! C headers: [`include/linux/bitmap.h`](srctree/include/linux/bitmap.h). + +use crate::alloc::{AllocError, Flags}; +use crate::bindings; +use crate::pr_err; +use core::ptr::NonNull; + +/// Represents a C bitmap. Wraps underlying C bitmap API. +/// +/// # Invariants +/// +/// Must reference a `[c_ulong]` long enough to fit `data.len()` bits. +pub struct CBitmap { + _align: [usize; 0], + data: [()], +} + +impl CBitmap { + /// Borrows a C bitmap. + /// + /// # Safety + /// + /// * `ptr` holds a non-null address of an initialized array of `unsigned long` + /// that is large enough to hold `nbits` bits. + /// * the array must not be freed for the lifetime of this [`CBitmap`] + /// * concurrent access only happens through atomic operations + pub unsafe fn from_raw<'a>(ptr: *const usize, nbits: usize) -> &'a CBitmap { + let data: *const [()] = core::ptr::slice_from_raw_parts(ptr.cast(), nbits); + unsafe { &*(data as *const CBitmap) } + } + + /// Borrows a C bitmap exclusively. + /// + /// # Safety + /// + /// * `ptr` holds a non-null address of an initialized array of `unsigned long` + /// that is large enough to hold `nbits` bits. + /// * the array must not be freed for the lifetime of this [`CBitmap`] + /// * no concurrent access may happen. + pub unsafe fn from_raw_mut<'a>(ptr: *mut usize, nbits: usize) -> &'a mut CBitmap { + let data: *mut [()] = core::ptr::slice_from_raw_parts_mut(ptr.cast(), nbits); + unsafe { &mut *(data as *mut CBitmap) } + } + + /// Returns a raw pointer to the backing [`Bitmap`]. + pub fn as_ptr(&self) -> *const usize { + self as *const CBitmap as *const usize + } + + /// Returns a mutable raw pointer to the backing [`Bitmap`]. + pub fn as_mut_ptr(&mut self) -> *mut usize { + self as *mut CBitmap as *mut usize + } + + /// Returns length of this [`CBitmap`]. + pub fn len(&self) -> usize { + self.data.len() + } +} + +/// Holds either a pointer to array of `unsigned long` or a small bitmap. +#[repr(C)] +union BitmapRepr { + bitmap: usize, + ptr: NonNull<usize>, +} + +macro_rules! bitmap_assert { + ($cond:expr, $($arg:tt)+) => { + #[cfg(RUST_BITMAP_HARDENED)] + assert!($e, $($arg)*); + } +} + +macro_rules! bitmap_assert_return { + ($cond:expr, $($arg:tt)+) => { + #[cfg(RUST_BITMAP_HARDENED)] + assert!($e, $($arg)*); + + #[cfg(not(RUST_BITMAP_HARDENED))] + if !($cond) { + pr_err!($($arg)*); + return + } + } +} + +/// Represents an owned bitmap. +/// +/// Wraps underlying C bitmap API. See [`CBitmap`] for available +/// methods. +/// +/// # Examples +/// +/// Basic usage +/// +/// ``` +/// use kernel::alloc::flags::GFP_KERNEL; +/// use kernel::bitmap::Bitmap; +/// +/// let mut b = Bitmap::new(16, GFP_KERNEL)?; +/// +/// assert_eq!(16, b.len()); +/// for i in 0..16 { +/// if i % 4 == 0 { +/// b.set_bit(i); +/// } +/// } +/// assert_eq!(Some(0), b.next_bit(0)); +/// assert_eq!(Some(1), b.next_zero_bit(0)); +/// assert_eq!(Some(4), b.next_bit(1)); +/// assert_eq!(Some(5), b.next_zero_bit(4)); +/// assert_eq!(Some(12), b.last_bit()); +/// # Ok::<(), Error>(()) +/// ``` +/// +/// # Invariants +/// +/// * `inner.nbits` is `<= i32::MAX` and never changes. +/// * if `inner.nbits <= bindings::BITS_PER_LONG`, then `inner.repr` is +/// a `usize`. +/// * otherwise, `inner.repr` holds a non-null pointer to an initialized +/// array of `unsigned long` that is large enough to hold `nbits` bits. +pub struct Bitmap { + /// Representation of bitmap. + repr: BitmapRepr, + /// Length of this bitmap. Must be `<= i32::MAX`. + nbits: usize, +} + +impl core::ops::Deref for Bitmap { + type Target = CBitmap; + + fn deref(&self) -> &CBitmap { + let ptr = if self.nbits <= bindings::BITS_PER_LONG as _ { + // SAFETY: Bitmap is represented inline. + unsafe { core::ptr::addr_of!(self.repr.bitmap) } + } else { + // SAFETY: Bitmap is represented as array of `unsigned long`. + unsafe { self.repr.ptr.as_ptr() } + }; + + // SAFETY: We got the right pointer and invariants of [`Bitmap`] hold. + // An inline bitmap is treated like an array with single element. + unsafe { CBitmap::from_raw(ptr, self.nbits) } + } +} + +impl core::ops::DerefMut for Bitmap { + fn deref_mut(&mut self) -> &mut CBitmap { + let ptr = if self.nbits <= bindings::BITS_PER_LONG as _ { + // SAFETY: Bitmap is represented inline. + unsafe { core::ptr::addr_of_mut!(self.repr.bitmap) } + } else { + // SAFETY: Bitmap is represented as array of `unsigned long`. + unsafe { self.repr.ptr.as_mut() } + }; + + // SAFETY: We got the right pointer and invariants of [`Bitmap`] hold. + // An inline bitmap is treated like an array with single element. + unsafe { CBitmap::from_raw_mut(ptr, self.nbits) } + } +} + +/// Enable ownership transfer to other threads. +/// +/// # Safety +/// +/// We own the underlying bitmap representation. +unsafe impl Send for Bitmap {} + +/// Enable unsynchronized concurrent access to [`Bitmap`] through shared references. +/// +/// # Safety +/// +/// * When no thread performs any mutations, concurrent access is safe. +/// * Mutations are permitted through atomic operations and interior mutability. +/// All such methods are marked unsafe, to account for the lack of ordering +/// guarantees. Callers must acknowledge that updates may be observed in any +/// order. +unsafe impl Sync for Bitmap {} + +impl Drop for Bitmap { + fn drop(&mut self) { + if self.nbits <= bindings::BITS_PER_LONG as _ { + return; + } + // SAFETY: `self.ptr` was returned by the C `bitmap_zalloc`. + // + // INVARIANT: there is no other use of the `self.ptr` after this + // call and the value is being dropped so the broken invariant is + // not observable on function exit. + unsafe { bindings::bitmap_free(self.repr.ptr.as_ptr()) }; + } +} + +impl Bitmap { + /// Constructs a new [`Bitmap`]. + /// + /// Fails with [`AllocError`] when the [`Bitmap`] could not be allocated. This + /// includes the case when `nbits` is greater than `i32::MAX`. + #[inline] + pub fn new(nbits: usize, flags: Flags) -> Result<Self, AllocError> { + if nbits <= bindings::BITS_PER_LONG as _ { + return Ok(Bitmap { + repr: BitmapRepr { bitmap: 0 }, + nbits, + }); + } + if nbits > i32::MAX.try_into().unwrap() { + return Err(AllocError); + } + let nbits_u32 = u32::try_from(nbits).unwrap(); + // SAFETY: `bindings::BITS_PER_LONG < nbits` and `nbits <= i32::MAX`. + let ptr = unsafe { bindings::bitmap_zalloc(nbits_u32, flags.as_raw()) }; + let ptr = NonNull::new(ptr).ok_or(AllocError)?; + // INVARIANT: `ptr` returned by C `bitmap_zalloc` and `nbits` checked. + return Ok(Bitmap { + repr: BitmapRepr { ptr }, + nbits, + }); + } + + /// Returns length of this [`CBitmap`]. + #[inline] + pub fn len(&self) -> usize { + self.nbits + } +} + +impl CBitmap { + /// Set bit with index `index`. + /// + /// ATTENTION: `set_bit` is non-atomic, which differs from the naming + /// convention in C code. The corresponding C function is `__set_bit`. + /// + /// If RUST_BITMAP_HARDENED is not enabled and `index` is greater than + /// or equal to `self.nbits`, does nothing. + /// + /// # Panics + /// + /// Panics if RUST_BITMAP_HARDENED is enabled and `index` is greater than + /// or equal to `self.nbits`. + #[inline] + pub fn set_bit(&mut self, index: usize) { + bitmap_assert_return!( + index < self.len(), + "Bit `index` must be < {}, was {}", + self.len(), + index + ); + // SAFETY: Bit `index` is within bounds. + unsafe { bindings::__set_bit(index as u32, self.as_mut_ptr()) }; + } + + /// Set bit with index `index`, atomically. + /// + /// This is a relaxed atomic operation (no implied memory barriers). + /// + /// ATTENTION: The naming convention differs from C, where the corresponding + /// function is called `set_bit`. + /// + /// If RUST_BITMAP_HARDENED is not enabled and `index` is greater than + /// or equal to `self.len()`, does nothing. + /// + /// # Panics + /// + /// Panics if RUST_BITMAP_HARDENED is enabled and `index` is greater than + /// or equal to `self.len()`. + #[inline] + pub fn set_bit_atomic(&self, index: usize) { + bitmap_assert_return!( + index < self.len(), + "Bit `index` must be < {}, was {}", + self.len(), + index + ); + // SAFETY: `index` is within bounds and the caller has ensured that + // there is no mix of non-atomic and atomic operations. + unsafe { bindings::set_bit(index as u32, self.as_ptr() as *mut usize) }; + } + + /// Clear `index` bit. + /// + /// ATTENTION: `clear_bit` is non-atomic, which differs from the naming + /// convention in C code. The corresponding C function is `__clear_bit`. + /// + /// If RUST_BITMAP_HARDENED is not enabled and `index` is greater than + /// or equal to `self.len()`, does nothing. + /// + /// # Panics + /// + /// Panics if RUST_BITMAP_HARDENED is enabled and `index` is greater than + /// or equal to `self.len()`. + #[inline] + pub fn clear_bit(&mut self, index: usize) { + bitmap_assert_return!( + index < self.len(), + "Bit `index` must be < {}, was {}", + self.len(), + index + ); + // SAFETY: `index` is within bounds. + unsafe { bindings::__clear_bit(index as u32, self.as_mut_ptr()) }; + } + + /// Clear `index` bit, atomically. + /// + /// This is a relaxed atomic operation (no implied memory barriers). + /// + /// ATTENTION: The naming convention differs from C, where the corresponding + /// function is called `clear_bit`. + /// + /// If RUST_BITMAP_HARDENED is not enabled and `index` is greater than + /// or equal to `self.len()`, does nothing. + /// + /// # Panics + /// + /// Panics if RUST_BITMAP_HARDENED is enabled and `index` is greater than + /// or equal to `self.len()`. + #[inline] + pub fn clear_bit_atomic(&self, index: usize) { + bitmap_assert_return!( + index < self.len(), + "Bit `index` must be < {}, was {}", + self.len(), + index + ); + // SAFETY: `index` is within bounds and the caller has ensured that + // there is no mix of non-atomic and atomic operations. + unsafe { bindings::clear_bit(index as u32, self.as_ptr() as *mut usize) }; + } + + /// Copy `src` into this [`Bitmap`] and set any remaining bits to zero. + /// + /// # Examples + /// + /// ``` + /// use kernel::alloc::{AllocError, flags::GFP_KERNEL}; + /// use kernel::bitmap::Bitmap; + /// + /// let mut long_bitmap = Bitmap::new(256, GFP_KERNEL)?; + // + /// assert_eq!(None, long_bitmap.last_bit()); + // + /// let mut short_bitmap = Bitmap::new(16, GFP_KERNEL)?; + // + /// short_bitmap.set_bit(7); + /// long_bitmap.copy_and_extend(&short_bitmap); + /// assert_eq!(Some(7), long_bitmap.last_bit()); + /// + /// # Ok::<(), AllocError>(()) + /// ``` + #[inline] + pub fn copy_and_extend(&mut self, src: &Bitmap) { + let len = core::cmp::min(src.nbits, self.len()); + // SAFETY: access to `self` and `src` is within bounds. + unsafe { + bindings::bitmap_copy_and_extend( + self.as_mut_ptr(), + src.as_ptr(), + len as u32, + self.len() as u32, + ) + }; + } + + /// Finds last set bit. + /// + /// # Examples + /// + /// ``` + /// use kernel::alloc::{AllocError, flags::GFP_KERNEL}; + /// use kernel::bitmap::Bitmap; + /// + /// let bitmap = Bitmap::new(64, GFP_KERNEL)?; + /// + /// match bitmap.last_bit() { + /// Some(idx) => { + /// pr_info!("The last bit has index {idx}.\n"); + /// } + /// None => { + /// pr_info!("All bits in this bitmap are 0.\n"); + /// } + /// } + /// # Ok::<(), AllocError>(()) + /// ``` + #[inline] + pub fn last_bit(&self) -> Option<usize> { + // SAFETY: `_find_next_bit` access is within bounds due to invariant. + let index = unsafe { bindings::_find_last_bit(self.as_ptr(), self.len()) }; + if index >= self.len() { + None + } else { + Some(index) + } + } + + /// Finds next set bit, starting from `start`. + /// Returns `None` if `start` is greater of equal than `self.nbits`. + #[inline] + pub fn next_bit(&self, start: usize) -> Option<usize> { + bitmap_assert!( + start < self.len(), + "`start` must be < {} was {}", + self.len(), + start + ); + // SAFETY: `_find_next_bit` tolerates out-of-bounds arguments and returns a + // value larger than or equal to `self.len()` in that case. + let index = unsafe { bindings::_find_next_bit(self.as_ptr(), self.len(), start) }; + if index >= self.len() { + None + } else { + Some(index) + } + } + + /// Finds next zero bit, starting from `start`. + /// Returns `None` if `start` is greater than or equal to `self.len()`. + #[inline] + pub fn next_zero_bit(&self, start: usize) -> Option<usize> { + bitmap_assert!( + start < self.len(), + "`start` must be < {} was {}", + self.len(), + start + ); + // SAFETY: `_find_next_zero_bit` tolerates out-of-bounds arguments and returns a + // value larger than or equal to `self.len()` in that case. + let index = unsafe { bindings::_find_next_zero_bit(self.as_ptr(), self.len(), start) }; + if index >= self.len() { + None + } else { + Some(index) + } + } +} + +use macros::kunit_tests; + +#[kunit_tests(rust_kernel_bitmap)] +mod tests { + use super::*; + use kernel::alloc::flags::GFP_KERNEL; + + #[test] + fn cbitmap_borrow() { + let fake_c_bitmap: [usize; 2] = [0, 0]; + // SAFETY: `fake_c_bitmap` is an array of expected length. + let b = unsafe { + CBitmap::from_raw( + core::ptr::addr_of!(fake_c_bitmap) as *const usize, + 2 * bindings::BITS_PER_LONG as usize, + ) + }; + assert_eq!(2 * bindings::BITS_PER_LONG as usize, b.len()); + assert_eq!(None, b.next_bit(0)); + } + + #[test] + fn cbitmap_copy() { + let fake_c_bitmap: usize = 0xFF; + // SAFETY: `fake_c_bitmap` can be used as one-element array of expected length. + let b = unsafe { CBitmap::from_raw(core::ptr::addr_of!(fake_c_bitmap), 8) }; + assert_eq!(8, b.len()); + assert_eq!(None, b.next_zero_bit(0)); + } + + #[test] + fn bitmap_new() { + let b = Bitmap::new(0, GFP_KERNEL).unwrap(); + assert_eq!(0, b.len()); + + let b = Bitmap::new(3, GFP_KERNEL).unwrap(); + assert_eq!(3, b.len()); + + let b = Bitmap::new(1024, GFP_KERNEL).unwrap(); + assert_eq!(1024, b.len()); + + // Requesting too large values results in [`AllocError`]. + let b = Bitmap::new(1 << 31, GFP_KERNEL); + assert!(b.is_err()); + } + + #[test] + fn bitmap_set_clear_find() { + let mut b = Bitmap::new(128, GFP_KERNEL).unwrap(); + + // Zero-initialized + assert_eq!(None, b.next_bit(0)); + assert_eq!(Some(0), b.next_zero_bit(0)); + assert_eq!(None, b.last_bit()); + + b.set_bit(17); + + assert_eq!(Some(17), b.next_bit(0)); + assert_eq!(Some(17), b.next_bit(17)); + assert_eq!(None, b.next_bit(18)); + assert_eq!(Some(17), b.last_bit()); + + b.set_bit(107); + + assert_eq!(Some(17), b.next_bit(0)); + assert_eq!(Some(17), b.next_bit(17)); + assert_eq!(Some(107), b.next_bit(18)); + assert_eq!(Some(107), b.last_bit()); + + b.clear_bit(17); + + assert_eq!(Some(107), b.next_bit(0)); + assert_eq!(Some(107), b.last_bit()); + } + + #[cfg(not(RUST_BITMAP_HARDENED))] + #[test] + fn bitmap_out_of_bounds() { + let mut b = Bitmap::new(128, GFP_KERNEL).unwrap(); + + b.set_bit(2048); + b.set_bit_atomic(2048); + b.clear_bit(2048); + b.clear_bit_atomic(2048); + assert_eq!(None, b.next_bit(2048)); + assert_eq!(None, b.next_zero_bit(2048)); + assert_eq!(None, b.last_bit()); + } + + // TODO: add test RUST_BITMAP_HARDENED when KUnit supports this type of config. + + #[test] + fn bitmap_copy_and_extend() { + let mut long_bitmap = Bitmap::new(256, GFP_KERNEL).unwrap(); + + long_bitmap.set_bit(3); + long_bitmap.set_bit(200); + + let mut short_bitmap = Bitmap::new(32, GFP_KERNEL).unwrap(); + + short_bitmap.set_bit(17); + + long_bitmap.copy_and_extend(&short_bitmap); + + // Previous bits have been cleared. + assert_eq!(Some(17), long_bitmap.next_bit(0)); + assert_eq!(Some(17), long_bitmap.last_bit()); + } +} diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index de07aadd1ff5..8c4161cd82ac 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -38,6 +38,7 @@ pub use ffi; pub mod alloc; +pub mod bitmap; #[cfg(CONFIG_BLOCK)] pub mod block; #[doc(hidden)] diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index 3fe9d7b945c4..3ca3c7dc4381 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -324,6 +324,16 @@ config LIST_HARDENED If unsure, say N. +config RUST_BITMAP_HARDENED + bool "Check integrity of linked list manipulation" + depends on CONFIG_RUST + help + Enables additional assertions in the Rust Bitmap API to catch + arguments that are not guaranteed to result in an immediate access + fault. + + If unsure, say N. + config BUG_ON_DATA_CORRUPTION bool "Trigger a BUG when data corruption is detected" select LIST_HARDENED -- 2.49.0.1151.ga128411c76-goog ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH v9 3/5] rust: add bitmap API. 2025-05-26 15:01 ` [PATCH v9 3/5] rust: add bitmap API Burak Emir @ 2025-05-27 16:35 ` Boqun Feng 0 siblings, 0 replies; 5+ messages in thread From: Boqun Feng @ 2025-05-27 16:35 UTC (permalink / raw) To: Burak Emir Cc: Yury Norov, Kees Cook, Rasmus Villemoes, Viresh Kumar, Miguel Ojeda, Alex Gaynor, Gary Guo, Björn Roy Baron, Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross, Gustavo A . R . Silva, rust-for-linux, linux-kernel, linux-hardening On Mon, May 26, 2025 at 03:01:32PM +0000, Burak Emir wrote: > Provides an abstraction for C bitmap API and bitops operations. > > This commit enables a Rust implementation of an Android Binder > data structure from commit 15d9da3f818c ("binder: use bitmap for faster > descriptor lookup"), which can be found in drivers/android/dbitmap.h. > It is a step towards upstreaming the Rust port of Android Binder driver. > > We follow the C Bitmap API closely in naming and semantics, with > a few differences that take advantage of Rust language facilities > and idioms: > > * We leverage Rust type system guarantees as follows: > > * all (non-atomic) mutating operations require a &mut reference which > amounts to exclusive access. > > * the Bitmap type implements Send. This enables transferring > ownership between threads and is needed for Binder. > > * the Bitmap type implements Sync, which enables passing shared > references &Bitmap between threads. Atomic operations can be > used to safely modify from multiple threads (interior > mutability), though without ordering guarantees. > > * The Rust API uses `{set,clear}_bit` vs `{set,clear}_bit_atomic` as > names, which differs from the C naming convention which uses > set_bit for atomic vs __set_bit for non-atomic. > > * we include enough operations for the API to be useful, but not all > operations are exposed yet in order to avoid dead code. The missing > ones can be added later. > > * We follow the C API closely with a fine-grained approach to safety: > > * Low-level bit-ops get a safe API with bounds checks. Calling with > an out-of-bounds arguments to {set,clear}_bit becomes a no-op and > get logged as errors. > > * We introduce a RUST_BITMAP_HARDENED config, which > causes invocations with out-of-bounds arguments to panic. > > * methods correspond to find_* C methods tolerate out-of-bounds > since the C implementation does. Also here, we log out-of-bounds > arguments as errors and panic in RUST_BITMAP_HARDENED mode. > > * We add a way to "borrow" bitmaps from C in Rust, to make C bitmaps > that were allocated in C directly usable in Rust code (`CBitmap`). > > * the Rust API is optimized to represent the bitmap inline if it would > fit into a pointer. This saves allocations which is > relevant in the Binder use case. > > The underlying C bitmap is *not* exposed, and must never be exposed > (except in tests). Exposing the representation of the owned bitmap would > lose static guarantees. > > An alternative route of vendoring an existing Rust bitmap package was > considered but suboptimal overall. Reusing the C implementation is > preferable for a basic data structure like bitmaps. It enables Rust > code to be a lot more similar and predictable with respect to C code > that uses the same data structures and enables the use of code that > has been tried-and-tested in the kernel, with the same performance > characteristics whenever possible. > > We use the `usize` type for sizes and indices into the bitmap, > because Rust generally always uses that type for indices and lengths > and it will be more convenient if the API accepts that type. This means > that we need to perform some casts to/from u32 and usize, since the C > headers use unsigned int instead of size_t/unsigned long for these > numbers in some places. > > Adds new MAINTAINERS section BITMAP API [RUST]. > > Suggested-by: Alice Ryhl <aliceryhl@google.com> > Suggested-by: Yury Norov <yury.norov@gmail.com> > Signed-off-by: Burak Emir <bqe@google.com> > --- > MAINTAINERS | 7 + > rust/kernel/bitmap.rs | 554 +++++++++++++++++++++++++++++++++++++ > rust/kernel/lib.rs | 1 + > security/Kconfig.hardening | 10 + > 4 files changed, 572 insertions(+) > create mode 100644 rust/kernel/bitmap.rs > > diff --git a/MAINTAINERS b/MAINTAINERS > index 04d6727e944c..565eaa015d9e 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -4127,6 +4127,13 @@ S: Maintained > F: rust/helpers/bitmap.c > F: rust/helpers/cpumask.c > > +BITMAP API [RUST] > +M: Alice Ryhl <aliceryhl@google.com> > +M: Burak Emir <bqe@google.com> > +R: Yury Norov <yury.norov@gmail.com> > +S: Maintained > +F: rust/kernel/bitmap.rs > + > BITOPS API > M: Yury Norov <yury.norov@gmail.com> > R: Rasmus Villemoes <linux@rasmusvillemoes.dk> > diff --git a/rust/kernel/bitmap.rs b/rust/kernel/bitmap.rs > new file mode 100644 > index 000000000000..a6edd4889518 > --- /dev/null > +++ b/rust/kernel/bitmap.rs > @@ -0,0 +1,554 @@ > +// SPDX-License-Identifier: GPL-2.0 > + > +// Copyright (C) 2025 Google LLC. > + > +//! Rust API for bitmap. > +//! > +//! C headers: [`include/linux/bitmap.h`](srctree/include/linux/bitmap.h). > + > +use crate::alloc::{AllocError, Flags}; > +use crate::bindings; > +use crate::pr_err; > +use core::ptr::NonNull; > + > +/// Represents a C bitmap. Wraps underlying C bitmap API. > +/// > +/// # Invariants > +/// > +/// Must reference a `[c_ulong]` long enough to fit `data.len()` bits. > +pub struct CBitmap { > + _align: [usize; 0], This is the same as #[repr(align(8)] on 64bit architecture? If so, maybe we should do: #[cfg_attr(CONFIG_64BIT, repr(align(8)))] #[cfg_attr(not(CONFIG_64BIT), repr(align(4)))] pub struct CBitmap { data: [()], } ? I find it slightly better because it's explicit on the targeted alignment. > + data: [()], > +} > + > +impl CBitmap { > + /// Borrows a C bitmap. > + /// > + /// # Safety > + /// > + /// * `ptr` holds a non-null address of an initialized array of `unsigned long` > + /// that is large enough to hold `nbits` bits. > + /// * the array must not be freed for the lifetime of this [`CBitmap`] > + /// * concurrent access only happens through atomic operations > + pub unsafe fn from_raw<'a>(ptr: *const usize, nbits: usize) -> &'a CBitmap { > + let data: *const [()] = core::ptr::slice_from_raw_parts(ptr.cast(), nbits); Need "// INVARIANT:" here and in from_raw_mut() to explain why the references are valid. > + unsafe { &*(data as *const CBitmap) } > + } > + > + /// Borrows a C bitmap exclusively. > + /// > + /// # Safety > + /// > + /// * `ptr` holds a non-null address of an initialized array of `unsigned long` > + /// that is large enough to hold `nbits` bits. > + /// * the array must not be freed for the lifetime of this [`CBitmap`] > + /// * no concurrent access may happen. > + pub unsafe fn from_raw_mut<'a>(ptr: *mut usize, nbits: usize) -> &'a mut CBitmap { > + let data: *mut [()] = core::ptr::slice_from_raw_parts_mut(ptr.cast(), nbits); > + unsafe { &mut *(data as *mut CBitmap) } > + } > + > + /// Returns a raw pointer to the backing [`Bitmap`]. > + pub fn as_ptr(&self) -> *const usize { > + self as *const CBitmap as *const usize > + } > + > + /// Returns a mutable raw pointer to the backing [`Bitmap`]. > + pub fn as_mut_ptr(&mut self) -> *mut usize { > + self as *mut CBitmap as *mut usize > + } > + > + /// Returns length of this [`CBitmap`]. > + pub fn len(&self) -> usize { > + self.data.len() > + } > +} > + [...] > +/// Enable ownership transfer to other threads. > +/// > +/// # Safety > +/// > +/// We own the underlying bitmap representation. Usually the comment here should be a: // SAFETY: <explain why Bitmap is Send> "# Safety" is for describing the safety requirement. > +unsafe impl Send for Bitmap {} > + > +/// Enable unsynchronized concurrent access to [`Bitmap`] through shared references. > +/// > +/// # Safety > +/// > +/// * When no thread performs any mutations, concurrent access is safe. > +/// * Mutations are permitted through atomic operations and interior mutability. > +/// All such methods are marked unsafe, to account for the lack of ordering > +/// guarantees. Callers must acknowledge that updates may be observed in any > +/// order. I think this comment is out-of-date. What makes `Bitmap` is that all methods will immutable references are either atomic or read-only (more accurately, `Bitmap`'s deref() will return a reference to a `CBitmap` which is `Sync`). > +unsafe impl Sync for Bitmap {} > + > +impl Drop for Bitmap { > + fn drop(&mut self) { > + if self.nbits <= bindings::BITS_PER_LONG as _ { > + return; > + } > + // SAFETY: `self.ptr` was returned by the C `bitmap_zalloc`. > + // > + // INVARIANT: there is no other use of the `self.ptr` after this > + // call and the value is being dropped so the broken invariant is > + // not observable on function exit. > + unsafe { bindings::bitmap_free(self.repr.ptr.as_ptr()) }; > + } > +} > + > +impl Bitmap { > + /// Constructs a new [`Bitmap`]. > + /// > + /// Fails with [`AllocError`] when the [`Bitmap`] could not be allocated. This > + /// includes the case when `nbits` is greater than `i32::MAX`. > + #[inline] > + pub fn new(nbits: usize, flags: Flags) -> Result<Self, AllocError> { > + if nbits <= bindings::BITS_PER_LONG as _ { > + return Ok(Bitmap { > + repr: BitmapRepr { bitmap: 0 }, > + nbits, > + }); > + } > + if nbits > i32::MAX.try_into().unwrap() { > + return Err(AllocError); > + } > + let nbits_u32 = u32::try_from(nbits).unwrap(); > + // SAFETY: `bindings::BITS_PER_LONG < nbits` and `nbits <= i32::MAX`. > + let ptr = unsafe { bindings::bitmap_zalloc(nbits_u32, flags.as_raw()) }; > + let ptr = NonNull::new(ptr).ok_or(AllocError)?; > + // INVARIANT: `ptr` returned by C `bitmap_zalloc` and `nbits` checked. > + return Ok(Bitmap { > + repr: BitmapRepr { ptr }, > + nbits, > + }); > + } > + > + /// Returns length of this [`CBitmap`]. Copy-pasta? It should be "[`Bitmap`]". > + #[inline] > + pub fn len(&self) -> usize { > + self.nbits > + } > +} > + Probably need: unsafe impl Sync for CBitmap { } > +impl CBitmap { > + /// Set bit with index `index`. > + /// > + /// ATTENTION: `set_bit` is non-atomic, which differs from the naming > + /// convention in C code. The corresponding C function is `__set_bit`. > + /// > + /// If RUST_BITMAP_HARDENED is not enabled and `index` is greater than > + /// or equal to `self.nbits`, does nothing. > + /// > + /// # Panics > + /// > + /// Panics if RUST_BITMAP_HARDENED is enabled and `index` is greater than > + /// or equal to `self.nbits`. > + #[inline] > + pub fn set_bit(&mut self, index: usize) { > + bitmap_assert_return!( > + index < self.len(), > + "Bit `index` must be < {}, was {}", > + self.len(), > + index > + ); > + // SAFETY: Bit `index` is within bounds. > + unsafe { bindings::__set_bit(index as u32, self.as_mut_ptr()) }; > + } > + [...] Regards, Boqun ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2025-05-29 20:15 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-05-29 19:42 [PATCH v9 3/5] rust: add bitmap API Pekka Ristola 2025-05-29 20:07 ` Boqun Feng 2025-05-29 20:15 ` Pekka Ristola -- strict thread matches above, loose matches on Subject: below -- 2025-05-26 15:01 [PATCH v9 0/5] rust: adds Bitmap API, ID pool and bindings Burak Emir 2025-05-26 15:01 ` [PATCH v9 3/5] rust: add bitmap API Burak Emir 2025-05-27 16:35 ` Boqun Feng
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox