[PATCH v6 00/26] Generic `Allocator` support for Rust

[PATCH v6 00/26] Generic `Allocator` support for Rust

[Danilo Krummrich]

Hi,

This patch series adds generic kernel allocator support for Rust, which so far
is limited to `kmalloc` allocations.

In order to abstain from (re-)adding unstable Rust features to the kernel, this
patch series does not extend the `Allocator` trait from Rust's `alloc` crate,
nor does it extend the `BoxExt` and `VecExt` extensions.

Instead, this series introduces a kernel specific `Allocator` trait, which is
implemented by the `Kmalloc`, `Vmalloc` and `KVmalloc` allocators, also
implemented in the context of this series.

As a consequence we need our own kernel `Box<T, A>` and `Vec<T, A>` types.
Additionally, this series adds the following type aliases:

```
pub type KBox<T> = Box<T, Kmalloc>;
pub type VBox<T> = Box<T, Vmalloc>;
pub type KVBox<T> = Box<T, KVmalloc>;


pub type KVec<T> = Vec<T, Kmalloc>;
pub type VVec<T> = Vec<T, Vmalloc>;
pub type KVVec<T> = Vec<T, KVmalloc>;
```

With that, we can start using the kernel `Box` and `Vec` types throughout the
tree and remove the now obolete extensions `BoxExt` and `VecExt`.

For a final cleanup, this series removes the last minor dependencies to Rust's
`alloc` crate and removes it from the entire kernel build.

The series ensures not to break the `rusttest` make target by implementing the
`allocator_test` module providing a stub implementation for all kernel
`Allocator`s.

This patch series passes all KUnit tests, including the ones added by this
series. Additionally, the tests were run with `kmemleak` and `KASAN` enabled,
without any issues.

This series is based on [1], which hit -mm/mm-unstable, and is also available
in [2].

[1] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=mm/krealloc
[2] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=rust/mm

Changes in v6:
 - rebase onto rust-dev
 - keep compiler annotations for {k,v,kv}realloc()
 - documentation changes suggested by Alice, Benno
 - remove `Box::into_pin`
 - fix typo in `Send` and `Sync` for `Box` and `Vec`
 - `kvec!` changes suggested by Alice
 - free `src` after copy in `Cmalloc`
 - handle `n == 0` in `Vec::extend_with`

Changes in v5:
 - (safety) comment / documentation fixes suggested by Alice, Benno and Gary
 - remove `Unique<T>` and implement `Send` and `Sync` for `Box` and `Vec`
 - use `KMALLOC_MAX_SIZE` for `KVmalloc` test and add a `Kmalloc` test that
   expects to fail for `KMALLOC_MAX_SIZE`
 - create use constants `KREALLOC`, `VREALLOC` and `KVREALLOC` for
   `ReallocFuncs`
 - drop `Box::drop_contents` for now, will add it again, once I actually rebase
   on the original patch that introduces it
 - improve usage of `size_of_val` in `Box`
 - move `InPlaceInit` and `ForeignOwnable` impls into kbox.rs
 - fix missing `Box` conversions in rnull.rs
 - reworked `Cmalloc` to keep track of the size of memory allocations itself
 - remove `GlobalAlloc` together with the `alloc` crate to avoid a linker error
 - remove `alloc` from scripts/generate_rust_analyzer.py

Changes in v4:
 - (safety) comment fixes suggested by Alice and Boqun
 - remove `Box::from_raw_alloc` and `Box::into_raw_alloc`, we don't need them
 - in `Box::drop` call `size_of_val` before `drop_in_place`
 - implement ForeignOwnable for Pin<Box<T>> as suggested by Alice
 - in `Vec::extend_with`, iterate over `n` instead of `spare.len()`
 - for `Vmalloc` and `KVmalloc` fail allocation for alignments larger than
   PAGE_SIZE for now (will add support for larger alignments in a separate
   series)
 - implement `Cmalloc` in `allocator_test` and type alias all kernel allocator
   types to it, such that we can use the kernel's `Box` and `Vec` types in
   userspace tests (rusttest)
   - this makes patch "rust: str: test: replace `alloc::format`" rather trivial

Changes in v3:
 - Box:
   - minor documentation fixes
   - removed unnecessary imports in doc tests
   - dropeed `self` argument from some remaining `Box` methods
   - implement `InPlaceInit` for Box<T, A> rather than specifically for `KBox<T>`
 - Vec:
   - minor documentation fixes
   - removed useless `Vec::allocator` method
   - in `Vec::extend_with` use `Vec::spare_capacity_mut` instead of raw pointer operations
   - added a few missing safety comments
   - pass GFP flags to `Vec::collect`
 - fixed a rustdoc warning in alloc.rs
 - fixed the allocator_test module to implement the `Allocator` trait correctly
 - rebased to rust-next

Changes in v2:
  - preserve `impl GlobalAlloc for Kmalloc` and remove it at the end (Benno)
  - remove `&self` parameter from all `Allocator` functions (Benno)
  - various documentation fixes for `Allocator` (Benno)
  - use `NonNull<u8>` for `Allocator::free` and `Option<NonNull<u8>>` for
    `Allocator::realloc` (Benno)
  - fix leak of `IntoIter` in `Vec::collect` (Boqun)
  - always realloc (try to shrink) in `Vec::collect`, it's up the the
    `Allocator` to provide a heuristic whether it makes sense to actually shrink
  - rename `KBox<T, A>` -> `Box<T, A>` and `KVec<T, A>` -> `Vec<T, A>` and
    provide type aliases `KBox<T>`, `VBox<T>`, `KVBox<T>`, etc.
    - This allows for much cleaner code and, in combination with removing
      `&self` parameters from `Allocator`s, gets us rid of the need for
      `Box::new` and `Box::new_alloc` and all other "_alloc" postfixed
      functions.
    - Before: `KBox::new_alloc(foo, Vmalloc)?`
    - After:  `VBox::new(foo)?`, which resolves to
              `Box::<Foo,  Vmalloc>::new(foo)?;


Danilo Krummrich (26):
  rust: alloc: add `Allocator` trait
  rust: alloc: separate `aligned_size` from `krealloc_aligned`
  rust: alloc: rename `KernelAllocator` to `Kmalloc`
  rust: alloc: implement `Allocator` for `Kmalloc`
  rust: alloc: add module `allocator_test`
  rust: alloc: implement `Vmalloc` allocator
  rust: alloc: implement `KVmalloc` allocator
  rust: alloc: add __GFP_NOWARN to `Flags`
  rust: alloc: implement kernel `Box`
  rust: treewide: switch to our kernel `Box` type
  rust: alloc: remove `BoxExt` extension
  rust: alloc: add `Box` to prelude
  rust: alloc: implement kernel `Vec` type
  rust: alloc: implement `IntoIterator` for `Vec`
  rust: alloc: implement `collect` for `IntoIter`
  rust: treewide: switch to the kernel `Vec` type
  rust: alloc: remove `VecExt` extension
  rust: alloc: add `Vec` to prelude
  rust: error: use `core::alloc::LayoutError`
  rust: error: check for config `test` in `Error::name`
  rust: alloc: implement `contains` for `Flags`
  rust: alloc: implement `Cmalloc` in module allocator_test
  rust: str: test: replace `alloc::format`
  rust: alloc: update module comment of alloc.rs
  kbuild: rust: remove the `alloc` crate and `GlobalAlloc`
  MAINTAINERS: add entry for the Rust `alloc` module

 MAINTAINERS                         |   7 +
 drivers/block/rnull.rs              |   4 +-
 rust/Makefile                       |  44 +-
 rust/bindings/bindings_helper.h     |   1 +
 rust/exports.c                      |   1 -
 rust/helpers.c                      |  15 +
 rust/kernel/alloc.rs                | 142 ++++-
 rust/kernel/alloc/allocator.rs      | 173 ++++--
 rust/kernel/alloc/allocator_test.rs | 185 ++++++
 rust/kernel/alloc/box_ext.rs        |  80 ---
 rust/kernel/alloc/kbox.rs           | 480 +++++++++++++++
 rust/kernel/alloc/kvec.rs           | 891 ++++++++++++++++++++++++++++
 rust/kernel/alloc/vec_ext.rs        | 185 ------
 rust/kernel/error.rs                |   6 +-
 rust/kernel/init.rs                 |  93 +--
 rust/kernel/init/__internal.rs      |   2 +-
 rust/kernel/lib.rs                  |   1 -
 rust/kernel/prelude.rs              |   5 +-
 rust/kernel/rbtree.rs               |  34 +-
 rust/kernel/str.rs                  |  35 +-
 rust/kernel/sync/arc.rs             |  17 +-
 rust/kernel/sync/condvar.rs         |   4 +-
 rust/kernel/sync/lock/mutex.rs      |   2 +-
 rust/kernel/sync/lock/spinlock.rs   |   2 +-
 rust/kernel/sync/locked_by.rs       |   2 +-
 rust/kernel/types.rs                |  30 +-
 rust/kernel/uaccess.rs              |  17 +-
 rust/kernel/workqueue.rs            |  20 +-
 rust/macros/lib.rs                  |  12 +-
 samples/rust/rust_minimal.rs        |   4 +-
 scripts/Makefile.build              |   7 +-
 scripts/generate_rust_analyzer.py   |  11 +-
 32 files changed, 1973 insertions(+), 539 deletions(-)
 create mode 100644 rust/kernel/alloc/allocator_test.rs
 delete mode 100644 rust/kernel/alloc/box_ext.rs
 create mode 100644 rust/kernel/alloc/kbox.rs
 create mode 100644 rust/kernel/alloc/kvec.rs
 delete mode 100644 rust/kernel/alloc/vec_ext.rs


base-commit: f005c686d6c1a2e66f2f9d21179d6b6bd45b20e2
-- 
2.46.0

[PATCH v6 01/26] rust: alloc: add `Allocator` trait

[Danilo Krummrich]

Add a kernel specific `Allocator` trait, that in contrast to the one in
Rust's core library doesn't require unstable features and supports GFP
flags.

Subsequent patches add the following trait implementors: `Kmalloc`,
`Vmalloc` and `KVmalloc`.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc.rs | 102 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 102 insertions(+)

diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index 1966bd407017..9932f21b0539 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -11,6 +11,7 @@
 /// Indicates an allocation error.
 #[derive(Copy, Clone, PartialEq, Eq, Debug)]
 pub struct AllocError;
+use core::{alloc::Layout, ptr::NonNull};
 
 /// Flags to be used when allocating memory.
 ///
@@ -86,3 +87,104 @@ pub mod flags {
     /// small allocations.
     pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT);
 }
+
+/// The kernel's [`Allocator`] trait.
+///
+/// An implementation of [`Allocator`] can allocate, re-allocate and free memory buffer described
+/// via [`Layout`].
+///
+/// [`Allocator`] is designed to be implemented as a ZST; [`Allocator`] functions do not operate on
+/// an object instance.
+///
+/// In order to be able to support `#[derive(SmartPointer)]` later on, we need to avoid a design
+/// that requires an `Allocator` to be instantiated, hence its functions must not contain any kind
+/// of `self` parameter.
+///
+/// # Safety
+///
+/// A memory allocation returned from an allocator must remain valid until it is explicitly freed.
+///
+/// Any pointer to a valid memory allocation must be valid to be passed to any other [`Allocator`]
+/// function of the same type.
+///
+/// Implementers must ensure that all trait functions abide by the guarantees documented in the
+/// `# Guarantees` sections.
+pub unsafe trait Allocator {
+    /// Allocate memory based on `layout` and `flags`.
+    ///
+    /// On success, returns a buffer represented as `NonNull<[u8]>` that satisfies the layout
+    /// constraints (i.e. minimum size and alignment as specified by `layout`).
+    ///
+    /// This function is equivalent to `realloc` when called with `None`.
+    ///
+    /// # Guarantees
+    ///
+    /// When the return value is `Ok(ptr)`, then `ptr` is
+    /// - valid for reads and writes for `layout.size()` bytes, until it is passed to
+    ///   [`Allocator::free`] or [`Allocator::realloc`],
+    /// - aligned to `layout.align()`,
+    ///
+    /// Additionally, `Flags` are honored as documented in
+    /// <https://docs.kernel.org/core-api/mm-api.html#mm-api-gfp-flags>.
+    fn alloc(layout: Layout, flags: Flags) -> Result<NonNull<[u8]>, AllocError> {
+        // SAFETY: Passing `None` to `realloc` is valid by it's safety requirements and asks for a
+        // new memory allocation.
+        unsafe { Self::realloc(None, layout, flags) }
+    }
+
+    /// Re-allocate an existing memory allocation to satisfy the requested `layout`.
+    ///
+    /// If the requested size is zero, `realloc` behaves equivalent to `free`.
+    ///
+    /// If the requested size is larger than the size of the existing allocation, a successful call
+    /// to `realloc` guarantees that the new or grown buffer has at least `Layout::size` bytes, but
+    /// may also be larger.
+    ///
+    /// If the requested size is smaller than the size of the existing allocation, `realloc` may or
+    /// may not shrink the buffer; this is implementation specific to the allocator.
+    ///
+    /// On allocation failure, the existing buffer, if any, remains valid.
+    ///
+    /// The buffer is represented as `NonNull<[u8]>`.
+    ///
+    /// # Safety
+    ///
+    /// If `ptr == Some(p)`, then `p` must point to an existing and valid memory allocation created
+    /// by this allocator. The alignment encoded in `layout` must be smaller than or equal to the
+    /// alignment requested in the previous `alloc` or `realloc` call of the same allocation.
+    ///
+    /// Additionally, `ptr` is allowed to be `None`; in this case a new memory allocation is
+    /// created.
+    ///
+    /// # Guarantees
+    ///
+    /// This function has the same guarantees as [`Allocator::alloc`]. When `ptr == Some(p)`, then
+    /// it additionally guarantees that:
+    /// - the contents of the memory pointed to by `p` are preserved up to the lesser of the new
+    ///   and old size,
+    ///   and old size, i.e.
+    ///   `ret_ptr[0..min(layout.size(), old_size)] == p[0..min(layout.size(), old_size)]`, where
+    ///   `old_size` is the size of the allocation that `p` points at.
+
+    /// - when the return value is `Err(AllocError)`, then `p` is still valid.
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError>;
+
+    /// Free an existing memory allocation.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an existing and valid memory allocation created by this `Allocator` and
+    /// must not be a dangling pointer.
+    ///
+    /// The memory allocation at `ptr` must never again be read from or written to.
+    unsafe fn free(ptr: NonNull<u8>) {
+        // SAFETY: The caller guarantees that `ptr` points at a valid allocation created by this
+        // allocator. We are passing a `Layout` with the smallest possible alignment, so it is
+        // smaller than or equal to the alignment previously used with this allocation.
+        let _ = unsafe { Self::realloc(Some(ptr), Layout::new::<()>(), Flags(0)) };
+    }
+}
-- 
2.46.0

Re: [PATCH v6 01/26] rust: alloc: add `Allocator` trait

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> Add a kernel specific `Allocator` trait, that in contrast to the one in
> Rust's core library doesn't require unstable features and supports GFP
> flags.
> 
> Subsequent patches add the following trait implementors: `Kmalloc`,
> `Vmalloc` and `KVmalloc`.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

We discussed this in our weekly meeting (I think ~one week ago?). If you
give me a draft version of the comment that you plan to add regarding
the `old_layout` parameter, I can see if I am happy with it. If I am, I
would give you my RB.

---
Cheers,
Benno

Re: [PATCH v6 01/26] rust: alloc: add `Allocator` trait

[Danilo Krummrich]

On Thu, Aug 29, 2024 at 06:19:09PM +0000, Benno Lossin wrote:
> On 16.08.24 02:10, Danilo Krummrich wrote:
> > Add a kernel specific `Allocator` trait, that in contrast to the one in
> > Rust's core library doesn't require unstable features and supports GFP
> > flags.
> > 
> > Subsequent patches add the following trait implementors: `Kmalloc`,
> > `Vmalloc` and `KVmalloc`.
> > 
> > Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> 
> We discussed this in our weekly meeting (I think ~one week ago?). If you
> give me a draft version of the comment that you plan to add regarding
> the `old_layout` parameter, I can see if I am happy with it. If I am, I
> would give you my RB.

May I propose you let me know what you would like to see covered, rather than
me trying to guess it. :-)

> 
> ---
> Cheers,
> Benno
> 

Re: [PATCH v6 01/26] rust: alloc: add `Allocator` trait

[Benno Lossin]

On 29.08.24 23:56, Danilo Krummrich wrote:
> On Thu, Aug 29, 2024 at 06:19:09PM +0000, Benno Lossin wrote:
>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>> Add a kernel specific `Allocator` trait, that in contrast to the one in
>>> Rust's core library doesn't require unstable features and supports GFP
>>> flags.
>>>
>>> Subsequent patches add the following trait implementors: `Kmalloc`,
>>> `Vmalloc` and `KVmalloc`.
>>>
>>> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
>>> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
>>
>> We discussed this in our weekly meeting (I think ~one week ago?). If you
>> give me a draft version of the comment that you plan to add regarding
>> the `old_layout` parameter, I can see if I am happy with it. If I am, I
>> would give you my RB.
> 
> May I propose you let me know what you would like to see covered, rather than
> me trying to guess it. :-)

I was hoping that we put that in our meeting notes, but I failed to find
them... I would put this in a normal comment, so it doesn't show up in the
documentation. Preface it like implementation decision/detail:
- Why do `Allocator::{realloc,free}` not have an `old_layout` parameter
  like in the stdlib? (the reasons you had for that decision, like we
  don't need it etc.)
- Then something along the lines of "Note that no technical reason is
  listed above, so if you need/want to implement an allocator taking
  advantage of that, you can change it"

I don't think we need a lot here. Additionally it would be very useful
to also put this in an issue to not lose track of it.

---
Cheers,
Benno

Re: [PATCH v6 01/26] rust: alloc: add `Allocator` trait

[Danilo Krummrich]

On Fri, Aug 30, 2024 at 01:06:00PM +0000, Benno Lossin wrote:
> On 29.08.24 23:56, Danilo Krummrich wrote:
> > On Thu, Aug 29, 2024 at 06:19:09PM +0000, Benno Lossin wrote:
> >> On 16.08.24 02:10, Danilo Krummrich wrote:
> >>> Add a kernel specific `Allocator` trait, that in contrast to the one in
> >>> Rust's core library doesn't require unstable features and supports GFP
> >>> flags.
> >>>
> >>> Subsequent patches add the following trait implementors: `Kmalloc`,
> >>> `Vmalloc` and `KVmalloc`.
> >>>
> >>> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> >>> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> >>
> >> We discussed this in our weekly meeting (I think ~one week ago?). If you
> >> give me a draft version of the comment that you plan to add regarding
> >> the `old_layout` parameter, I can see if I am happy with it. If I am, I
> >> would give you my RB.
> > 
> > May I propose you let me know what you would like to see covered, rather than
> > me trying to guess it. :-)
> 
> I was hoping that we put that in our meeting notes, but I failed to find
> them... I would put this in a normal comment, so it doesn't show up in the
> documentation. Preface it like implementation decision/detail:
> - Why do `Allocator::{realloc,free}` not have an `old_layout` parameter
>   like in the stdlib? (the reasons you had for that decision, like we
>   don't need it etc.)

Ok.

> - Then something along the lines of "Note that no technical reason is
>   listed above, so if you need/want to implement an allocator taking
>   advantage of that, you can change it"

I don't really want to set the conditions for this to change in the
documentation. It really depends on whether it's actually needed or the
advantage of having it is huge enough to leave the core kernel allocators with
unused arguments.

This can really only be properly evaluated case by case in a discussion.

> 
> I don't think we need a lot here. Additionally it would be very useful
> to also put this in an issue to not lose track of it.
> 
> ---
> Cheers,
> Benno
> 

Re: [PATCH v6 01/26] rust: alloc: add `Allocator` trait

[Benno Lossin]

On 03.09.24 13:56, Danilo Krummrich wrote:
> On Fri, Aug 30, 2024 at 01:06:00PM +0000, Benno Lossin wrote:
>> On 29.08.24 23:56, Danilo Krummrich wrote:
>>> On Thu, Aug 29, 2024 at 06:19:09PM +0000, Benno Lossin wrote:
>>>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>>>> Add a kernel specific `Allocator` trait, that in contrast to the one in
>>>>> Rust's core library doesn't require unstable features and supports GFP
>>>>> flags.
>>>>>
>>>>> Subsequent patches add the following trait implementors: `Kmalloc`,
>>>>> `Vmalloc` and `KVmalloc`.
>>>>>
>>>>> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
>>>>> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
>>>>
>>>> We discussed this in our weekly meeting (I think ~one week ago?). If you
>>>> give me a draft version of the comment that you plan to add regarding
>>>> the `old_layout` parameter, I can see if I am happy with it. If I am, I
>>>> would give you my RB.
>>>
>>> May I propose you let me know what you would like to see covered, rather than
>>> me trying to guess it. :-)
>>
>> I was hoping that we put that in our meeting notes, but I failed to find
>> them... I would put this in a normal comment, so it doesn't show up in the
>> documentation. Preface it like implementation decision/detail:
>> - Why do `Allocator::{realloc,free}` not have an `old_layout` parameter
>>   like in the stdlib? (the reasons you had for that decision, like we
>>   don't need it etc.)
> 
> Ok.
> 
>> - Then something along the lines of "Note that no technical reason is
>>   listed above, so if you need/want to implement an allocator taking
>>   advantage of that, you can change it"
> 
> I don't really want to set the conditions for this to change in the
> documentation. It really depends on whether it's actually needed or the
> advantage of having it is huge enough to leave the core kernel allocators with
> unused arguments.
> 
> This can really only be properly evaluated case by case in a discussion.

Agreed, but I don't want people to think that we have a reason against
doing it in the future. Do you have an idea how to convey this?

---
Cheers,
Benno

Re: [PATCH v6 01/26] rust: alloc: add `Allocator` trait

[Danilo Krummrich]

On Tue, Sep 10, 2024 at 01:03:48PM +0000, Benno Lossin wrote:
> On 03.09.24 13:56, Danilo Krummrich wrote:
> > On Fri, Aug 30, 2024 at 01:06:00PM +0000, Benno Lossin wrote:
> >> On 29.08.24 23:56, Danilo Krummrich wrote:
> >>> On Thu, Aug 29, 2024 at 06:19:09PM +0000, Benno Lossin wrote:
> >>>> On 16.08.24 02:10, Danilo Krummrich wrote:
> >>>>> Add a kernel specific `Allocator` trait, that in contrast to the one in
> >>>>> Rust's core library doesn't require unstable features and supports GFP
> >>>>> flags.
> >>>>>
> >>>>> Subsequent patches add the following trait implementors: `Kmalloc`,
> >>>>> `Vmalloc` and `KVmalloc`.
> >>>>>
> >>>>> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> >>>>> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> >>>>
> >>>> We discussed this in our weekly meeting (I think ~one week ago?). If you
> >>>> give me a draft version of the comment that you plan to add regarding
> >>>> the `old_layout` parameter, I can see if I am happy with it. If I am, I
> >>>> would give you my RB.
> >>>
> >>> May I propose you let me know what you would like to see covered, rather than
> >>> me trying to guess it. :-)
> >>
> >> I was hoping that we put that in our meeting notes, but I failed to find
> >> them... I would put this in a normal comment, so it doesn't show up in the
> >> documentation. Preface it like implementation decision/detail:
> >> - Why do `Allocator::{realloc,free}` not have an `old_layout` parameter
> >>   like in the stdlib? (the reasons you had for that decision, like we
> >>   don't need it etc.)
> > 
> > Ok.
> > 
> >> - Then something along the lines of "Note that no technical reason is
> >>   listed above, so if you need/want to implement an allocator taking
> >>   advantage of that, you can change it"
> > 
> > I don't really want to set the conditions for this to change in the
> > documentation. It really depends on whether it's actually needed or the
> > advantage of having it is huge enough to leave the core kernel allocators with
> > unused arguments.
> > 
> > This can really only be properly evaluated case by case in a discussion.
> 
> Agreed, but I don't want people to think that we have a reason against
> doing it in the future. Do you have an idea how to convey this?

I understand (and agree with) your intention. But I don't think it's necessary
to document, because, ideally, this is already true for the whole kernel.

Generally, I think it's valid to assume that people are willing to change the
code if the advantages outweigh the disadvantages.

So, we could write something like "This may be changed if the advantages
outweigh the disadvantages.", but it'd be a bit random, since we could probably
sprinkle this everywhere.

> 
> ---
> Cheers,
> Benno
> 

Re: [PATCH v6 01/26] rust: alloc: add `Allocator` trait

[Benno Lossin]

On 10.09.24 15:23, Danilo Krummrich wrote:
> On Tue, Sep 10, 2024 at 01:03:48PM +0000, Benno Lossin wrote:
>> On 03.09.24 13:56, Danilo Krummrich wrote:
>>> On Fri, Aug 30, 2024 at 01:06:00PM +0000, Benno Lossin wrote:
>>>> On 29.08.24 23:56, Danilo Krummrich wrote:
>>>>> On Thu, Aug 29, 2024 at 06:19:09PM +0000, Benno Lossin wrote:
>>>>>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>>>>>> Add a kernel specific `Allocator` trait, that in contrast to the one in
>>>>>>> Rust's core library doesn't require unstable features and supports GFP
>>>>>>> flags.
>>>>>>>
>>>>>>> Subsequent patches add the following trait implementors: `Kmalloc`,
>>>>>>> `Vmalloc` and `KVmalloc`.
>>>>>>>
>>>>>>> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
>>>>>>> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
>>>>>>
>>>>>> We discussed this in our weekly meeting (I think ~one week ago?). If you
>>>>>> give me a draft version of the comment that you plan to add regarding
>>>>>> the `old_layout` parameter, I can see if I am happy with it. If I am, I
>>>>>> would give you my RB.
>>>>>
>>>>> May I propose you let me know what you would like to see covered, rather than
>>>>> me trying to guess it. :-)
>>>>
>>>> I was hoping that we put that in our meeting notes, but I failed to find
>>>> them... I would put this in a normal comment, so it doesn't show up in the
>>>> documentation. Preface it like implementation decision/detail:
>>>> - Why do `Allocator::{realloc,free}` not have an `old_layout` parameter
>>>>   like in the stdlib? (the reasons you had for that decision, like we
>>>>   don't need it etc.)
>>>
>>> Ok.
>>>
>>>> - Then something along the lines of "Note that no technical reason is
>>>>   listed above, so if you need/want to implement an allocator taking
>>>>   advantage of that, you can change it"
>>>
>>> I don't really want to set the conditions for this to change in the
>>> documentation. It really depends on whether it's actually needed or the
>>> advantage of having it is huge enough to leave the core kernel allocators with
>>> unused arguments.
>>>
>>> This can really only be properly evaluated case by case in a discussion.
>>
>> Agreed, but I don't want people to think that we have a reason against
>> doing it in the future. Do you have an idea how to convey this?
> 
> I understand (and agree with) your intention. But I don't think it's necessary
> to document, because, ideally, this is already true for the whole kernel.
> 
> Generally, I think it's valid to assume that people are willing to change the
> code if the advantages outweigh the disadvantages.

Sure, but for certain areas you might need a very, very good reason to
do so.
In this case, I fear that people might believe that this is the case,
even though it isn't (one of course still needs a good reason).

> So, we could write something like "This may be changed if the advantages
> outweigh the disadvantages.", but it'd be a bit random, since we could probably
> sprinkle this everywhere.

Sure, that works for me. I don't think that we can sprinkle this
everywhere (but of course a in lot of places).

---
Cheers,
Benno

Re: [PATCH v6 01/26] rust: alloc: add `Allocator` trait

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> +/// The kernel's [`Allocator`] trait.
> +///
> +/// An implementation of [`Allocator`] can allocate, re-allocate and free memory buffer described
> +/// via [`Layout`].
> +///
> +/// [`Allocator`] is designed to be implemented as a ZST; [`Allocator`] functions do not operate on
> +/// an object instance.
> +///
> +/// In order to be able to support `#[derive(SmartPointer)]` later on, we need to avoid a design
> +/// that requires an `Allocator` to be instantiated, hence its functions must not contain any kind
> +/// of `self` parameter.
> +///
> +/// # Safety
> +///
> +/// A memory allocation returned from an allocator must remain valid until it is explicitly freed.
> +///
> +/// Any pointer to a valid memory allocation must be valid to be passed to any other [`Allocator`]
> +/// function of the same type.
> +///
> +/// Implementers must ensure that all trait functions abide by the guarantees documented in the
> +/// `# Guarantees` sections.

Can you make a bullet point list out of these three paragraphs?

---
Cheers,
Benno

Re: [PATCH v6 01/26] rust: alloc: add `Allocator` trait

[Gary Guo]

On Fri, 16 Aug 2024 02:10:43 +0200
Danilo Krummrich <dakr@kernel.org> wrote:

> Add a kernel specific `Allocator` trait, that in contrast to the one in
> Rust's core library doesn't require unstable features and supports GFP
> flags.
> 
> Subsequent patches add the following trait implementors: `Kmalloc`,
> `Vmalloc` and `KVmalloc`.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
>  rust/kernel/alloc.rs | 102 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 102 insertions(+)
> 
> diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> index 1966bd407017..9932f21b0539 100644
> --- a/rust/kernel/alloc.rs
> +++ b/rust/kernel/alloc.rs
> @@ -11,6 +11,7 @@
>  /// Indicates an allocation error.
>  #[derive(Copy, Clone, PartialEq, Eq, Debug)]
>  pub struct AllocError;
> +use core::{alloc::Layout, ptr::NonNull};
>  
>  /// Flags to be used when allocating memory.
>  ///
> @@ -86,3 +87,104 @@ pub mod flags {
>      /// small allocations.
>      pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT);
>  }
> +
> +/// The kernel's [`Allocator`] trait.
> +///
> +/// An implementation of [`Allocator`] can allocate, re-allocate and free memory buffer described
> +/// via [`Layout`].
> +///
> +/// [`Allocator`] is designed to be implemented as a ZST; [`Allocator`] functions do not operate on
> +/// an object instance.
> +///
> +/// In order to be able to support `#[derive(SmartPointer)]` later on, we need to avoid a design
> +/// that requires an `Allocator` to be instantiated, hence its functions must not contain any kind
> +/// of `self` parameter.
> +///
> +/// # Safety
> +///
> +/// A memory allocation returned from an allocator must remain valid until it is explicitly freed.
> +///
> +/// Any pointer to a valid memory allocation must be valid to be passed to any other [`Allocator`]
> +/// function of the same type.
> +///
> +/// Implementers must ensure that all trait functions abide by the guarantees documented in the
> +/// `# Guarantees` sections.
> +pub unsafe trait Allocator {
> +    /// Allocate memory based on `layout` and `flags`.
> +    ///
> +    /// On success, returns a buffer represented as `NonNull<[u8]>` that satisfies the layout
> +    /// constraints (i.e. minimum size and alignment as specified by `layout`).
> +    ///
> +    /// This function is equivalent to `realloc` when called with `None`.
> +    ///
> +    /// # Guarantees
> +    ///
> +    /// When the return value is `Ok(ptr)`, then `ptr` is
> +    /// - valid for reads and writes for `layout.size()` bytes, until it is passed to
> +    ///   [`Allocator::free`] or [`Allocator::realloc`],
> +    /// - aligned to `layout.align()`,
> +    ///
> +    /// Additionally, `Flags` are honored as documented in
> +    /// <https://docs.kernel.org/core-api/mm-api.html#mm-api-gfp-flags>.
> +    fn alloc(layout: Layout, flags: Flags) -> Result<NonNull<[u8]>, AllocError> {
> +        // SAFETY: Passing `None` to `realloc` is valid by it's safety requirements and asks for a
> +        // new memory allocation.
> +        unsafe { Self::realloc(None, layout, flags) }
> +    }
> +
> +    /// Re-allocate an existing memory allocation to satisfy the requested `layout`.
> +    ///
> +    /// If the requested size is zero, `realloc` behaves equivalent to `free`.
> +    ///
> +    /// If the requested size is larger than the size of the existing allocation, a successful call
> +    /// to `realloc` guarantees that the new or grown buffer has at least `Layout::size` bytes, but
> +    /// may also be larger.
> +    ///
> +    /// If the requested size is smaller than the size of the existing allocation, `realloc` may or
> +    /// may not shrink the buffer; this is implementation specific to the allocator.
> +    ///
> +    /// On allocation failure, the existing buffer, if any, remains valid.
> +    ///
> +    /// The buffer is represented as `NonNull<[u8]>`.
> +    ///
> +    /// # Safety
> +    ///
> +    /// If `ptr == Some(p)`, then `p` must point to an existing and valid memory allocation created
> +    /// by this allocator. The alignment encoded in `layout` must be smaller than or equal to the
> +    /// alignment requested in the previous `alloc` or `realloc` call of the same allocation.
> +    ///
> +    /// Additionally, `ptr` is allowed to be `None`; in this case a new memory allocation is
> +    /// created.
> +    ///
> +    /// # Guarantees
> +    ///
> +    /// This function has the same guarantees as [`Allocator::alloc`]. When `ptr == Some(p)`, then
> +    /// it additionally guarantees that:
> +    /// - the contents of the memory pointed to by `p` are preserved up to the lesser of the new
> +    ///   and old size,
> +    ///   and old size, i.e.
> +    ///   `ret_ptr[0..min(layout.size(), old_size)] == p[0..min(layout.size(), old_size)]`, where
> +    ///   `old_size` is the size of the allocation that `p` points at.
> +

This line seems to be missing `///`?

> +    /// - when the return value is `Err(AllocError)`, then `p` is still valid.
> +    unsafe fn realloc(
> +        ptr: Option<NonNull<u8>>,
> +        layout: Layout,
> +        flags: Flags,
> +    ) -> Result<NonNull<[u8]>, AllocError>;
> +
> +    /// Free an existing memory allocation.
> +    ///
> +    /// # Safety
> +    ///
> +    /// `ptr` must point to an existing and valid memory allocation created by this `Allocator` and
> +    /// must not be a dangling pointer.
> +    ///
> +    /// The memory allocation at `ptr` must never again be read from or written to.
> +    unsafe fn free(ptr: NonNull<u8>) {
> +        // SAFETY: The caller guarantees that `ptr` points at a valid allocation created by this
> +        // allocator. We are passing a `Layout` with the smallest possible alignment, so it is
> +        // smaller than or equal to the alignment previously used with this allocation.
> +        let _ = unsafe { Self::realloc(Some(ptr), Layout::new::<()>(), Flags(0)) };
> +    }
> +}

[PATCH v6 02/26] rust: alloc: separate `aligned_size` from `krealloc_aligned`

[Danilo Krummrich]

Separate `aligned_size` from `krealloc_aligned`.

Subsequent patches implement `Allocator` derivates, such as `Kmalloc`,
that require `aligned_size` and replace the original `krealloc_aligned`.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc/allocator.rs | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
index e6ea601f38c6..c83b6dff896d 100644
--- a/rust/kernel/alloc/allocator.rs
+++ b/rust/kernel/alloc/allocator.rs
@@ -8,6 +8,17 @@
 
 struct KernelAllocator;
 
+/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
+fn aligned_size(new_layout: Layout) -> usize {
+    // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.
+    let layout = new_layout.pad_to_align();
+
+    // Note that `layout.size()` (after padding) is guaranteed to be a multiple of `layout.align()`
+    // which together with the slab guarantees means the `krealloc` will return a properly aligned
+    // object (see comments in `kmalloc()` for more information).
+    layout.size()
+}
+
 /// Calls `krealloc` with a proper size to alloc a new object aligned to `new_layout`'s alignment.
 ///
 /// # Safety
@@ -15,13 +26,7 @@
 /// - `ptr` can be either null or a pointer which has been allocated by this allocator.
 /// - `new_layout` must have a non-zero size.
 pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: Flags) -> *mut u8 {
-    // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.
-    let layout = new_layout.pad_to_align();
-
-    // Note that `layout.size()` (after padding) is guaranteed to be a multiple of `layout.align()`
-    // which together with the slab guarantees means the `krealloc` will return a properly aligned
-    // object (see comments in `kmalloc()` for more information).
-    let size = layout.size();
+    let size = aligned_size(new_layout);
 
     // SAFETY:
     // - `ptr` is either null or a pointer returned from a previous `k{re}alloc()` by the
-- 
2.46.0

Re: [PATCH v6 02/26] rust: alloc: separate `aligned_size` from `krealloc_aligned`

[Gary Guo]

On Fri, 16 Aug 2024 02:10:44 +0200
Danilo Krummrich <dakr@kernel.org> wrote:

> Separate `aligned_size` from `krealloc_aligned`.
> 
> Subsequent patches implement `Allocator` derivates, such as `Kmalloc`,
> that require `aligned_size` and replace the original `krealloc_aligned`.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Reviewed-by: Benno Lossin <benno.lossin@proton.me>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

Reviewed-by: Gary Guo <gary@garyguo.net>

> ---
>  rust/kernel/alloc/allocator.rs | 19 ++++++++++++-------
>  1 file changed, 12 insertions(+), 7 deletions(-)
> 
> diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
> index e6ea601f38c6..c83b6dff896d 100644
> --- a/rust/kernel/alloc/allocator.rs
> +++ b/rust/kernel/alloc/allocator.rs
> @@ -8,6 +8,17 @@
>  
>  struct KernelAllocator;
>  
> +/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
> +fn aligned_size(new_layout: Layout) -> usize {
> +    // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.
> +    let layout = new_layout.pad_to_align();
> +
> +    // Note that `layout.size()` (after padding) is guaranteed to be a multiple of `layout.align()`
> +    // which together with the slab guarantees means the `krealloc` will return a properly aligned
> +    // object (see comments in `kmalloc()` for more information).
> +    layout.size()
> +}
> +
>  /// Calls `krealloc` with a proper size to alloc a new object aligned to `new_layout`'s alignment.
>  ///
>  /// # Safety
> @@ -15,13 +26,7 @@
>  /// - `ptr` can be either null or a pointer which has been allocated by this allocator.
>  /// - `new_layout` must have a non-zero size.
>  pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: Flags) -> *mut u8 {
> -    // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.
> -    let layout = new_layout.pad_to_align();
> -
> -    // Note that `layout.size()` (after padding) is guaranteed to be a multiple of `layout.align()`
> -    // which together with the slab guarantees means the `krealloc` will return a properly aligned
> -    // object (see comments in `kmalloc()` for more information).
> -    let size = layout.size();
> +    let size = aligned_size(new_layout);
>  
>      // SAFETY:
>      // - `ptr` is either null or a pointer returned from a previous `k{re}alloc()` by the

[PATCH v6 03/26] rust: alloc: rename `KernelAllocator` to `Kmalloc`

[Danilo Krummrich]

Subsequent patches implement `Vmalloc` and `KVmalloc` allocators, hence
align `KernelAllocator` to this naming scheme.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
Reviewed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc/allocator.rs | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
index c83b6dff896d..e32182f91167 100644
--- a/rust/kernel/alloc/allocator.rs
+++ b/rust/kernel/alloc/allocator.rs
@@ -6,7 +6,7 @@
 use core::alloc::{GlobalAlloc, Layout};
 use core::ptr;
 
-struct KernelAllocator;
+struct Kmalloc;
 
 /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
 fn aligned_size(new_layout: Layout) -> usize {
@@ -36,7 +36,7 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F
     unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 }
 }
 
-unsafe impl GlobalAlloc for KernelAllocator {
+unsafe impl GlobalAlloc for Kmalloc {
     unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
         // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
         // requirement.
@@ -72,7 +72,7 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
 }
 
 #[global_allocator]
-static ALLOCATOR: KernelAllocator = KernelAllocator;
+static ALLOCATOR: Kmalloc = Kmalloc;
 
 // See <https://github.com/rust-lang/rust/pull/86844>.
 #[no_mangle]
-- 
2.46.0

[PATCH v6 04/26] rust: alloc: implement `Allocator` for `Kmalloc`

[Danilo Krummrich]

Implement `Allocator` for `Kmalloc`, the kernel's default allocator,
typically used for objects smaller than page size.

All memory allocations made with `Kmalloc` end up in `krealloc()`.

It serves as allocator for the subsequently introduced types `KBox` and
`KVec`.

Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc.rs           |  2 +-
 rust/kernel/alloc/allocator.rs | 72 +++++++++++++++++++++++++++++++++-
 2 files changed, 72 insertions(+), 2 deletions(-)

diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index 9932f21b0539..477dbe3c5a2f 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -4,7 +4,7 @@
 
 #[cfg(not(test))]
 #[cfg(not(testlib))]
-mod allocator;
+pub mod allocator;
 pub mod box_ext;
 pub mod vec_ext;
 
diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
index e32182f91167..78e7d5488843 100644
--- a/rust/kernel/alloc/allocator.rs
+++ b/rust/kernel/alloc/allocator.rs
@@ -1,12 +1,28 @@
 // SPDX-License-Identifier: GPL-2.0
 
 //! Allocator support.
+//!
+//! Documentation for the kernel's memory allocators can found in the "Memory Allocation Guide"
+//! linked below. For instance, this includes the concept of "get free page" (GFP) flags and the
+//! typical application of the different kernel allocators.
+//!
+//! Reference: <https://docs.kernel.org/core-api/memory-allocation.html>
 
 use super::{flags::*, Flags};
 use core::alloc::{GlobalAlloc, Layout};
 use core::ptr;
+use core::ptr::NonNull;
 
-struct Kmalloc;
+use crate::alloc::{AllocError, Allocator};
+use crate::bindings;
+
+/// The contiguous kernel allocator.
+///
+/// `Kmalloc` is typically used for physically contiguous allocations up to page size, but also
+/// supports larger allocations up to `bindings::KMALLOC_MAX_SIZE`, which is hardware specific.
+///
+/// For more details see [self].
+pub struct Kmalloc;
 
 /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
 fn aligned_size(new_layout: Layout) -> usize {
@@ -36,6 +52,60 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F
     unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 }
 }
 
+/// # Invariants
+///
+/// One of the following `krealloc`, `vrealloc`, `kvrealloc`.
+struct ReallocFunc(
+    unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void,
+);
+
+impl ReallocFunc {
+    // INVARIANT: `krealloc` satisfies the type invariants.
+    const KREALLOC: Self = Self(bindings::krealloc);
+
+    /// # Safety
+    ///
+    /// This method has the same safety requirements as [`Allocator::realloc`].
+    unsafe fn call(
+        &self,
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let size = aligned_size(layout);
+        let ptr = match ptr {
+            Some(ptr) => ptr.as_ptr(),
+            None => ptr::null(),
+        };
+
+        // SAFETY: `ptr` is either NULL or valid by the safety requirements of this function.
+        let raw_ptr = unsafe {
+            // If `size == 0` and `ptr != NULL` the memory behind the pointer is freed.
+            self.0(ptr.cast(), size, flags.0).cast()
+        };
+
+        let ptr = if size == 0 {
+            NonNull::dangling()
+        } else {
+            NonNull::new(raw_ptr).ok_or(AllocError)?
+        };
+
+        Ok(NonNull::slice_from_raw_parts(ptr, size))
+    }
+}
+
+unsafe impl Allocator for Kmalloc {
+    #[inline]
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        // SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`.
+        unsafe { ReallocFunc::KREALLOC.call(ptr, layout, flags) }
+    }
+}
+
 unsafe impl GlobalAlloc for Kmalloc {
     unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
         // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
-- 
2.46.0

Re: [PATCH v6 04/26] rust: alloc: implement `Allocator` for `Kmalloc`

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> Implement `Allocator` for `Kmalloc`, the kernel's default allocator,
> typically used for objects smaller than page size.
> 
> All memory allocations made with `Kmalloc` end up in `krealloc()`.
> 
> It serves as allocator for the subsequently introduced types `KBox` and
> `KVec`.
> 
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
>  rust/kernel/alloc.rs           |  2 +-
>  rust/kernel/alloc/allocator.rs | 72 +++++++++++++++++++++++++++++++++-
>  2 files changed, 72 insertions(+), 2 deletions(-)
> 
> diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> index 9932f21b0539..477dbe3c5a2f 100644
> --- a/rust/kernel/alloc.rs
> +++ b/rust/kernel/alloc.rs
> @@ -4,7 +4,7 @@
> 
>  #[cfg(not(test))]
>  #[cfg(not(testlib))]
> -mod allocator;
> +pub mod allocator;
>  pub mod box_ext;
>  pub mod vec_ext;
> 
> diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
> index e32182f91167..78e7d5488843 100644
> --- a/rust/kernel/alloc/allocator.rs
> +++ b/rust/kernel/alloc/allocator.rs
> @@ -1,12 +1,28 @@
>  // SPDX-License-Identifier: GPL-2.0
> 
>  //! Allocator support.
> +//!
> +//! Documentation for the kernel's memory allocators can found in the "Memory Allocation Guide"
> +//! linked below. For instance, this includes the concept of "get free page" (GFP) flags and the
> +//! typical application of the different kernel allocators.
> +//!
> +//! Reference: <https://docs.kernel.org/core-api/memory-allocation.html>

Thanks, this nice.

> 
>  use super::{flags::*, Flags};
>  use core::alloc::{GlobalAlloc, Layout};
>  use core::ptr;
> +use core::ptr::NonNull;
> 
> -struct Kmalloc;
> +use crate::alloc::{AllocError, Allocator};
> +use crate::bindings;
> +
> +/// The contiguous kernel allocator.
> +///
> +/// `Kmalloc` is typically used for physically contiguous allocations up to page size, but also
> +/// supports larger allocations up to `bindings::KMALLOC_MAX_SIZE`, which is hardware specific.

Does putting a link here work? (I guess we don't yet export the bindings
documentation, so it will probably fail... When we decide to enable it,
we should create an issue to add missing links)

> +///
> +/// For more details see [self].
> +pub struct Kmalloc;
> 
>  /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
>  fn aligned_size(new_layout: Layout) -> usize {
> @@ -36,6 +52,60 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F
>      unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 }
>  }
> 
> +/// # Invariants
> +///
> +/// One of the following `krealloc`, `vrealloc`, `kvrealloc`.
> +struct ReallocFunc(
> +    unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void,
> +);
> +
> +impl ReallocFunc {
> +    // INVARIANT: `krealloc` satisfies the type invariants.
> +    const KREALLOC: Self = Self(bindings::krealloc);
> +
> +    /// # Safety
> +    ///
> +    /// This method has the same safety requirements as [`Allocator::realloc`].
> +    unsafe fn call(
> +        &self,
> +        ptr: Option<NonNull<u8>>,
> +        layout: Layout,
> +        flags: Flags,
> +    ) -> Result<NonNull<[u8]>, AllocError> {
> +        let size = aligned_size(layout);
> +        let ptr = match ptr {
> +            Some(ptr) => ptr.as_ptr(),
> +            None => ptr::null(),
> +        };
> +
> +        // SAFETY: `ptr` is either NULL or valid by the safety requirements of this function.

You need some justification as to why calling the three allowed
functions here.

> +        let raw_ptr = unsafe {
> +            // If `size == 0` and `ptr != NULL` the memory behind the pointer is freed.
> +            self.0(ptr.cast(), size, flags.0).cast()
> +        };
> +
> +        let ptr = if size == 0 {
> +            NonNull::dangling()
> +        } else {
> +            NonNull::new(raw_ptr).ok_or(AllocError)?
> +        };
> +
> +        Ok(NonNull::slice_from_raw_parts(ptr, size))
> +    }
> +}
> +
> +unsafe impl Allocator for Kmalloc {

Missing SAFETY comment.

---
Cheers,
Benno

> +    #[inline]
> +    unsafe fn realloc(
> +        ptr: Option<NonNull<u8>>,
> +        layout: Layout,
> +        flags: Flags,
> +    ) -> Result<NonNull<[u8]>, AllocError> {
> +        // SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`.
> +        unsafe { ReallocFunc::KREALLOC.call(ptr, layout, flags) }
> +    }
> +}
> +
>  unsafe impl GlobalAlloc for Kmalloc {
>      unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
>          // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
> --
> 2.46.0
> 

Re: [PATCH v6 04/26] rust: alloc: implement `Allocator` for `Kmalloc`

[Danilo Krummrich]

On Thu, Aug 29, 2024 at 06:32:42PM +0000, Benno Lossin wrote:
> On 16.08.24 02:10, Danilo Krummrich wrote:
> > Implement `Allocator` for `Kmalloc`, the kernel's default allocator,
> > typically used for objects smaller than page size.
> > 
> > All memory allocations made with `Kmalloc` end up in `krealloc()`.
> > 
> > It serves as allocator for the subsequently introduced types `KBox` and
> > `KVec`.
> > 
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > ---
> >  rust/kernel/alloc.rs           |  2 +-
> >  rust/kernel/alloc/allocator.rs | 72 +++++++++++++++++++++++++++++++++-
> >  2 files changed, 72 insertions(+), 2 deletions(-)
> > 
> > diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> > index 9932f21b0539..477dbe3c5a2f 100644
> > --- a/rust/kernel/alloc.rs
> > +++ b/rust/kernel/alloc.rs
> > @@ -4,7 +4,7 @@
> > 
> >  #[cfg(not(test))]
> >  #[cfg(not(testlib))]
> > -mod allocator;
> > +pub mod allocator;
> >  pub mod box_ext;
> >  pub mod vec_ext;
> > 
> > diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
> > index e32182f91167..78e7d5488843 100644
> > --- a/rust/kernel/alloc/allocator.rs
> > +++ b/rust/kernel/alloc/allocator.rs
> > @@ -1,12 +1,28 @@
> >  // SPDX-License-Identifier: GPL-2.0
> > 
> >  //! Allocator support.
> > +//!
> > +//! Documentation for the kernel's memory allocators can found in the "Memory Allocation Guide"
> > +//! linked below. For instance, this includes the concept of "get free page" (GFP) flags and the
> > +//! typical application of the different kernel allocators.
> > +//!
> > +//! Reference: <https://docs.kernel.org/core-api/memory-allocation.html>
> 
> Thanks, this nice.
> 
> > 
> >  use super::{flags::*, Flags};
> >  use core::alloc::{GlobalAlloc, Layout};
> >  use core::ptr;
> > +use core::ptr::NonNull;
> > 
> > -struct Kmalloc;
> > +use crate::alloc::{AllocError, Allocator};
> > +use crate::bindings;
> > +
> > +/// The contiguous kernel allocator.
> > +///
> > +/// `Kmalloc` is typically used for physically contiguous allocations up to page size, but also
> > +/// supports larger allocations up to `bindings::KMALLOC_MAX_SIZE`, which is hardware specific.
> 
> Does putting a link here work? (I guess we don't yet export the bindings
> documentation, so it will probably fail... When we decide to enable it,
> we should create an issue to add missing links)
> 
> > +///
> > +/// For more details see [self].
> > +pub struct Kmalloc;
> > 
> >  /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
> >  fn aligned_size(new_layout: Layout) -> usize {
> > @@ -36,6 +52,60 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F
> >      unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 }
> >  }
> > 
> > +/// # Invariants
> > +///
> > +/// One of the following `krealloc`, `vrealloc`, `kvrealloc`.
> > +struct ReallocFunc(
> > +    unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void,
> > +);
> > +
> > +impl ReallocFunc {
> > +    // INVARIANT: `krealloc` satisfies the type invariants.
> > +    const KREALLOC: Self = Self(bindings::krealloc);
> > +
> > +    /// # Safety
> > +    ///
> > +    /// This method has the same safety requirements as [`Allocator::realloc`].
> > +    unsafe fn call(
> > +        &self,
> > +        ptr: Option<NonNull<u8>>,
> > +        layout: Layout,
> > +        flags: Flags,
> > +    ) -> Result<NonNull<[u8]>, AllocError> {
> > +        let size = aligned_size(layout);
> > +        let ptr = match ptr {
> > +            Some(ptr) => ptr.as_ptr(),
> > +            None => ptr::null(),
> > +        };
> > +
> > +        // SAFETY: `ptr` is either NULL or valid by the safety requirements of this function.
> 
> You need some justification as to why calling the three allowed
> functions here.

What kind of justification do I need? Can you please share some more details on
what you think is missing here?

> 
> > +        let raw_ptr = unsafe {
> > +            // If `size == 0` and `ptr != NULL` the memory behind the pointer is freed.
> > +            self.0(ptr.cast(), size, flags.0).cast()
> > +        };
> > +
> > +        let ptr = if size == 0 {
> > +            NonNull::dangling()
> > +        } else {
> > +            NonNull::new(raw_ptr).ok_or(AllocError)?
> > +        };
> > +
> > +        Ok(NonNull::slice_from_raw_parts(ptr, size))
> > +    }
> > +}
> > +
> > +unsafe impl Allocator for Kmalloc {
> 
> Missing SAFETY comment.

Yeah, I think we came across this in an earlier version of the series. I asked
you about the content and usefulness of a comment here, since I'd just end up
re-iterating what the `Allocator` trait documentation says.

IIRC, you replied that you want to think of something that'd make sense to add
here.

What do you think should be written here?

> 
> ---
> Cheers,
> Benno
> 
> > +    #[inline]
> > +    unsafe fn realloc(
> > +        ptr: Option<NonNull<u8>>,
> > +        layout: Layout,
> > +        flags: Flags,
> > +    ) -> Result<NonNull<[u8]>, AllocError> {
> > +        // SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`.
> > +        unsafe { ReallocFunc::KREALLOC.call(ptr, layout, flags) }
> > +    }
> > +}
> > +
> >  unsafe impl GlobalAlloc for Kmalloc {
> >      unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
> >          // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
> > --
> > 2.46.0
> > 
> 

Re: [PATCH v6 04/26] rust: alloc: implement `Allocator` for `Kmalloc`

[Benno Lossin]

On 30.08.24 00:04, Danilo Krummrich wrote:
> On Thu, Aug 29, 2024 at 06:32:42PM +0000, Benno Lossin wrote:
>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>> Implement `Allocator` for `Kmalloc`, the kernel's default allocator,
>>> typically used for objects smaller than page size.
>>>
>>> All memory allocations made with `Kmalloc` end up in `krealloc()`.
>>>
>>> It serves as allocator for the subsequently introduced types `KBox` and
>>> `KVec`.
>>>
>>> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
>>> ---
>>>  rust/kernel/alloc.rs           |  2 +-
>>>  rust/kernel/alloc/allocator.rs | 72 +++++++++++++++++++++++++++++++++-
>>>  2 files changed, 72 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
>>> index 9932f21b0539..477dbe3c5a2f 100644
>>> --- a/rust/kernel/alloc.rs
>>> +++ b/rust/kernel/alloc.rs
>>> @@ -4,7 +4,7 @@
>>>
>>>  #[cfg(not(test))]
>>>  #[cfg(not(testlib))]
>>> -mod allocator;
>>> +pub mod allocator;
>>>  pub mod box_ext;
>>>  pub mod vec_ext;
>>>
>>> diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
>>> index e32182f91167..78e7d5488843 100644
>>> --- a/rust/kernel/alloc/allocator.rs
>>> +++ b/rust/kernel/alloc/allocator.rs
>>> @@ -1,12 +1,28 @@
>>>  // SPDX-License-Identifier: GPL-2.0
>>>
>>>  //! Allocator support.
>>> +//!
>>> +//! Documentation for the kernel's memory allocators can found in the "Memory Allocation Guide"
>>> +//! linked below. For instance, this includes the concept of "get free page" (GFP) flags and the
>>> +//! typical application of the different kernel allocators.
>>> +//!
>>> +//! Reference: <https://docs.kernel.org/core-api/memory-allocation.html>
>>
>> Thanks, this nice.
>>
>>>
>>>  use super::{flags::*, Flags};
>>>  use core::alloc::{GlobalAlloc, Layout};
>>>  use core::ptr;
>>> +use core::ptr::NonNull;
>>>
>>> -struct Kmalloc;
>>> +use crate::alloc::{AllocError, Allocator};
>>> +use crate::bindings;
>>> +
>>> +/// The contiguous kernel allocator.
>>> +///
>>> +/// `Kmalloc` is typically used for physically contiguous allocations up to page size, but also
>>> +/// supports larger allocations up to `bindings::KMALLOC_MAX_SIZE`, which is hardware specific.
>>
>> Does putting a link here work? (I guess we don't yet export the bindings
>> documentation, so it will probably fail... When we decide to enable it,
>> we should create an issue to add missing links)
>>
>>> +///
>>> +/// For more details see [self].
>>> +pub struct Kmalloc;
>>>
>>>  /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
>>>  fn aligned_size(new_layout: Layout) -> usize {
>>> @@ -36,6 +52,60 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F
>>>      unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 }
>>>  }
>>>
>>> +/// # Invariants
>>> +///
>>> +/// One of the following `krealloc`, `vrealloc`, `kvrealloc`.
>>> +struct ReallocFunc(
>>> +    unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void,
>>> +);
>>> +
>>> +impl ReallocFunc {
>>> +    // INVARIANT: `krealloc` satisfies the type invariants.
>>> +    const KREALLOC: Self = Self(bindings::krealloc);
>>> +
>>> +    /// # Safety
>>> +    ///
>>> +    /// This method has the same safety requirements as [`Allocator::realloc`].
>>> +    unsafe fn call(
>>> +        &self,
>>> +        ptr: Option<NonNull<u8>>,
>>> +        layout: Layout,
>>> +        flags: Flags,
>>> +    ) -> Result<NonNull<[u8]>, AllocError> {
>>> +        let size = aligned_size(layout);
>>> +        let ptr = match ptr {
>>> +            Some(ptr) => ptr.as_ptr(),
>>> +            None => ptr::null(),
>>> +        };
>>> +
>>> +        // SAFETY: `ptr` is either NULL or valid by the safety requirements of this function.
>>
>> You need some justification as to why calling the three allowed
>> functions here.
> 
> What kind of justification do I need? Can you please share some more details on
> what you think is missing here?

So, you are calling a function pointer to an `unsafe` function. This
means that through some invariant you have to know what the safety
requirements are (otherwise how can you guarantee that this is OK?). You
have the invariant that the pointer points at one of the three functions
mentioned above. What are the safety requirements of those functions? I
would assume that the only one is that `ptr` is valid. So you can use:

    // SAFETY:
    // - `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc` and thus only requires that `ptr` is
    //   NULL or valid.
    // - `ptr` is either NULL or valid by the safety requirements of this function.

>>> +        let raw_ptr = unsafe {
>>> +            // If `size == 0` and `ptr != NULL` the memory behind the pointer is freed.
>>> +            self.0(ptr.cast(), size, flags.0).cast()
>>> +        };
>>> +
>>> +        let ptr = if size == 0 {
>>> +            NonNull::dangling()
>>> +        } else {
>>> +            NonNull::new(raw_ptr).ok_or(AllocError)?
>>> +        };
>>> +
>>> +        Ok(NonNull::slice_from_raw_parts(ptr, size))
>>> +    }
>>> +}
>>> +
>>> +unsafe impl Allocator for Kmalloc {
>>
>> Missing SAFETY comment.
> 
> Yeah, I think we came across this in an earlier version of the series. I asked
> you about the content and usefulness of a comment here, since I'd just end up
> re-iterating what the `Allocator` trait documentation says.
> 
> IIRC, you replied that you want to think of something that'd make sense to add
> here.

Oh yeah, sorry I forgot about that.
 
> What do you think should be written here?

I think the best way to do it, would be to push this question down into
`ReallocFunc::call`. So we would put this on the trait:

    // SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that
    // - memory remains valid until it is explicitly freed,
    // - passing a pointer to a vaild memory allocation is OK,
    // - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same.

We then need to put this on `ReallocFunc::call`:

    /// # Guarantees
    ///
    /// This method has the same guarantees as `Allocator::realloc`. Additionally
    /// - it accepts any pointer to a valid memory allocation allocated by this function.
    /// - memory allocated by this function remains valid until it is passed to this function.

Finally, we need a `GUARANTEE` comment (just above the return [^1]
value) that establishes these guarantees:

    // GUARANTEE: Since we called `self.0` with `size` above and by the type invariants of `Self`,
    // `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc`. Those functions provide the guarantees of
    // this function.

I am not really happy with the last sentence, but I also don't think
that there is value in listing out all the guarantees, only to then say
"all of this is guaranteed by us calling one of these three functions.


[^1]: I am not sure that there is the right place. If you have any
      suggestions, feel free to share them.


>>> +    #[inline]
>>> +    unsafe fn realloc(
>>> +        ptr: Option<NonNull<u8>>,
>>> +        layout: Layout,
>>> +        flags: Flags,
>>> +    ) -> Result<NonNull<[u8]>, AllocError> {
>>> +        // SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`.
>>> +        unsafe { ReallocFunc::KREALLOC.call(ptr, layout, flags) }
>>> +    }
>>> +}

Oh one more thing, I know that you already have a lot of patches in this
series, but could you split this one into two? So the first one should
introduce `ReallocFunc` and the second one add the impl for `Kmalloc`?
I managed to confuse me twice because of that :) 

---
Cheers,
Benno

>>> +
>>>  unsafe impl GlobalAlloc for Kmalloc {
>>>      unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
>>>          // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
>>> --
>>> 2.46.0
>>>
>>

Re: [PATCH v6 04/26] rust: alloc: implement `Allocator` for `Kmalloc`

[Danilo Krummrich]

On Fri, Aug 30, 2024 at 02:45:35PM +0000, Benno Lossin wrote:
> On 30.08.24 00:04, Danilo Krummrich wrote:
> > On Thu, Aug 29, 2024 at 06:32:42PM +0000, Benno Lossin wrote:
> >> On 16.08.24 02:10, Danilo Krummrich wrote:
> >>> Implement `Allocator` for `Kmalloc`, the kernel's default allocator,
> >>> typically used for objects smaller than page size.
> >>>
> >>> All memory allocations made with `Kmalloc` end up in `krealloc()`.
> >>>
> >>> It serves as allocator for the subsequently introduced types `KBox` and
> >>> `KVec`.
> >>>
> >>> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> >>> ---
> >>>  rust/kernel/alloc.rs           |  2 +-
> >>>  rust/kernel/alloc/allocator.rs | 72 +++++++++++++++++++++++++++++++++-
> >>>  2 files changed, 72 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> >>> index 9932f21b0539..477dbe3c5a2f 100644
> >>> --- a/rust/kernel/alloc.rs
> >>> +++ b/rust/kernel/alloc.rs
> >>> @@ -4,7 +4,7 @@
> >>>
> >>>  #[cfg(not(test))]
> >>>  #[cfg(not(testlib))]
> >>> -mod allocator;
> >>> +pub mod allocator;
> >>>  pub mod box_ext;
> >>>  pub mod vec_ext;
> >>>
> >>> diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
> >>> index e32182f91167..78e7d5488843 100644
> >>> --- a/rust/kernel/alloc/allocator.rs
> >>> +++ b/rust/kernel/alloc/allocator.rs
> >>> @@ -1,12 +1,28 @@
> >>>  // SPDX-License-Identifier: GPL-2.0
> >>>
> >>>  //! Allocator support.
> >>> +//!
> >>> +//! Documentation for the kernel's memory allocators can found in the "Memory Allocation Guide"
> >>> +//! linked below. For instance, this includes the concept of "get free page" (GFP) flags and the
> >>> +//! typical application of the different kernel allocators.
> >>> +//!
> >>> +//! Reference: <https://docs.kernel.org/core-api/memory-allocation.html>
> >>
> >> Thanks, this nice.
> >>
> >>>
> >>>  use super::{flags::*, Flags};
> >>>  use core::alloc::{GlobalAlloc, Layout};
> >>>  use core::ptr;
> >>> +use core::ptr::NonNull;
> >>>
> >>> -struct Kmalloc;
> >>> +use crate::alloc::{AllocError, Allocator};
> >>> +use crate::bindings;
> >>> +
> >>> +/// The contiguous kernel allocator.
> >>> +///
> >>> +/// `Kmalloc` is typically used for physically contiguous allocations up to page size, but also
> >>> +/// supports larger allocations up to `bindings::KMALLOC_MAX_SIZE`, which is hardware specific.
> >>
> >> Does putting a link here work? (I guess we don't yet export the bindings
> >> documentation, so it will probably fail... When we decide to enable it,
> >> we should create an issue to add missing links)
> >>
> >>> +///
> >>> +/// For more details see [self].
> >>> +pub struct Kmalloc;
> >>>
> >>>  /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
> >>>  fn aligned_size(new_layout: Layout) -> usize {
> >>> @@ -36,6 +52,60 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F
> >>>      unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 }
> >>>  }
> >>>
> >>> +/// # Invariants
> >>> +///
> >>> +/// One of the following `krealloc`, `vrealloc`, `kvrealloc`.
> >>> +struct ReallocFunc(
> >>> +    unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void,
> >>> +);
> >>> +
> >>> +impl ReallocFunc {
> >>> +    // INVARIANT: `krealloc` satisfies the type invariants.
> >>> +    const KREALLOC: Self = Self(bindings::krealloc);
> >>> +
> >>> +    /// # Safety
> >>> +    ///
> >>> +    /// This method has the same safety requirements as [`Allocator::realloc`].
> >>> +    unsafe fn call(
> >>> +        &self,
> >>> +        ptr: Option<NonNull<u8>>,
> >>> +        layout: Layout,
> >>> +        flags: Flags,
> >>> +    ) -> Result<NonNull<[u8]>, AllocError> {
> >>> +        let size = aligned_size(layout);
> >>> +        let ptr = match ptr {
> >>> +            Some(ptr) => ptr.as_ptr(),
> >>> +            None => ptr::null(),
> >>> +        };
> >>> +
> >>> +        // SAFETY: `ptr` is either NULL or valid by the safety requirements of this function.
> >>
> >> You need some justification as to why calling the three allowed
> >> functions here.
> > 
> > What kind of justification do I need? Can you please share some more details on
> > what you think is missing here?
> 
> So, you are calling a function pointer to an `unsafe` function. This
> means that through some invariant you have to know what the safety
> requirements are (otherwise how can you guarantee that this is OK?). You
> have the invariant that the pointer points at one of the three functions
> mentioned above. What are the safety requirements of those functions? I
> would assume that the only one is that `ptr` is valid. So you can use:
> 
>     // SAFETY:
>     // - `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc` and thus only requires that `ptr` is
>     //   NULL or valid.

I'm fine adding it, but I'd like to understand why you think it's required in
the safety comment here? Isn't this implicit by being the type invariant?

>     // - `ptr` is either NULL or valid by the safety requirements of this function.

This is the part I already have.

> 
> >>> +        let raw_ptr = unsafe {
> >>> +            // If `size == 0` and `ptr != NULL` the memory behind the pointer is freed.
> >>> +            self.0(ptr.cast(), size, flags.0).cast()
> >>> +        };
> >>> +
> >>> +        let ptr = if size == 0 {
> >>> +            NonNull::dangling()
> >>> +        } else {
> >>> +            NonNull::new(raw_ptr).ok_or(AllocError)?
> >>> +        };
> >>> +
> >>> +        Ok(NonNull::slice_from_raw_parts(ptr, size))
> >>> +    }
> >>> +}
> >>> +
> >>> +unsafe impl Allocator for Kmalloc {
> >>
> >> Missing SAFETY comment.
> > 
> > Yeah, I think we came across this in an earlier version of the series. I asked
> > you about the content and usefulness of a comment here, since I'd just end up
> > re-iterating what the `Allocator` trait documentation says.
> > 
> > IIRC, you replied that you want to think of something that'd make sense to add
> > here.
> 
> Oh yeah, sorry I forgot about that.
>  
> > What do you think should be written here?
> 
> I think the best way to do it, would be to push this question down into
> `ReallocFunc::call`. So we would put this on the trait:
> 
>     // SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that
>     // - memory remains valid until it is explicitly freed,
>     // - passing a pointer to a vaild memory allocation is OK,
>     // - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same.

So, we'd also need the same for:
  - `unsafe impl Allocator for Vmalloc`
  - `unsafe impl Allocator for KVmalloc`

> 
> We then need to put this on `ReallocFunc::call`:
> 
>     /// # Guarantees
>     ///
>     /// This method has the same guarantees as `Allocator::realloc`. Additionally
>     /// - it accepts any pointer to a valid memory allocation allocated by this function.

You propose this, since for `Allocator::realloc` memory allocated with
`Allocator::alloc` would be fine too I guess.

But if e.g. `Kmalloc` wouldn't use the default `Allocator::alloc`, this would be
valid too.

We could instead write something like:

"it accepts any pointer to a valid memory allocation allocated with the same
kernel allocator."

>     /// - memory allocated by this function remains valid until it is passed to this function.

Same here, `Kmalloc` could implement its own `Allocator::free`.

Maybe just "...until it is explicitly freed.".

Anyway, I'm fine with both, since non of the kernel allocators uses anything
else than `ReallocFunc::call` to allocate and free memory.

> 
> Finally, we need a `GUARANTEE` comment (just above the return [^1]
> value) that establishes these guarantees:
> 
>     // GUARANTEE: Since we called `self.0` with `size` above and by the type invariants of `Self`,
>     // `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc`. Those functions provide the guarantees of
>     // this function.
> 
> I am not really happy with the last sentence, but I also don't think
> that there is value in listing out all the guarantees, only to then say
> "all of this is guaranteed by us calling one of these three functions.
> 
> 
> [^1]: I am not sure that there is the right place. If you have any
>       suggestions, feel free to share them.

Either way, I'm fine with this proposal.

> 
> 
> >>> +    #[inline]
> >>> +    unsafe fn realloc(
> >>> +        ptr: Option<NonNull<u8>>,
> >>> +        layout: Layout,
> >>> +        flags: Flags,
> >>> +    ) -> Result<NonNull<[u8]>, AllocError> {
> >>> +        // SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`.
> >>> +        unsafe { ReallocFunc::KREALLOC.call(ptr, layout, flags) }
> >>> +    }
> >>> +}
> 
> Oh one more thing, I know that you already have a lot of patches in this
> series, but could you split this one into two? So the first one should
> introduce `ReallocFunc` and the second one add the impl for `Kmalloc`?
> I managed to confuse me twice because of that :) 

Generally, I'm fine with that, but I'm not sure if I can avoid an intermediate
compiler warning about unused code doing that.

> 
> ---
> Cheers,
> Benno
> 
> >>> +
> >>>  unsafe impl GlobalAlloc for Kmalloc {
> >>>      unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
> >>>          // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
> >>> --
> >>> 2.46.0
> >>>
> >>
> 

Re: [PATCH v6 04/26] rust: alloc: implement `Allocator` for `Kmalloc`

[Benno Lossin]

On 03.09.24 13:48, Danilo Krummrich wrote:
> On Fri, Aug 30, 2024 at 02:45:35PM +0000, Benno Lossin wrote:
>> On 30.08.24 00:04, Danilo Krummrich wrote:
>>> On Thu, Aug 29, 2024 at 06:32:42PM +0000, Benno Lossin wrote:
>>>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>>>> +///
>>>>> +/// For more details see [self].
>>>>> +pub struct Kmalloc;
>>>>>
>>>>>  /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
>>>>>  fn aligned_size(new_layout: Layout) -> usize {
>>>>> @@ -36,6 +52,60 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F
>>>>>      unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 }
>>>>>  }
>>>>>
>>>>> +/// # Invariants
>>>>> +///
>>>>> +/// One of the following `krealloc`, `vrealloc`, `kvrealloc`.
>>>>> +struct ReallocFunc(
>>>>> +    unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void,
>>>>> +);
>>>>> +
>>>>> +impl ReallocFunc {
>>>>> +    // INVARIANT: `krealloc` satisfies the type invariants.
>>>>> +    const KREALLOC: Self = Self(bindings::krealloc);
>>>>> +
>>>>> +    /// # Safety
>>>>> +    ///
>>>>> +    /// This method has the same safety requirements as [`Allocator::realloc`].
>>>>> +    unsafe fn call(
>>>>> +        &self,
>>>>> +        ptr: Option<NonNull<u8>>,
>>>>> +        layout: Layout,
>>>>> +        flags: Flags,
>>>>> +    ) -> Result<NonNull<[u8]>, AllocError> {
>>>>> +        let size = aligned_size(layout);
>>>>> +        let ptr = match ptr {
>>>>> +            Some(ptr) => ptr.as_ptr(),
>>>>> +            None => ptr::null(),
>>>>> +        };
>>>>> +
>>>>> +        // SAFETY: `ptr` is either NULL or valid by the safety requirements of this function.
>>>>
>>>> You need some justification as to why calling the three allowed
>>>> functions here.
>>>
>>> What kind of justification do I need? Can you please share some more details on
>>> what you think is missing here?
>>
>> So, you are calling a function pointer to an `unsafe` function. This
>> means that through some invariant you have to know what the safety
>> requirements are (otherwise how can you guarantee that this is OK?). You
>> have the invariant that the pointer points at one of the three functions
>> mentioned above. What are the safety requirements of those functions? I
>> would assume that the only one is that `ptr` is valid. So you can use:
>>
>>     // SAFETY:
>>     // - `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc` and thus only requires that `ptr` is
>>     //   NULL or valid.
> 
> I'm fine adding it, but I'd like to understand why you think it's required in
> the safety comment here? Isn't this implicit by being the type invariant?

You are calling a function pointer to an `unsafe` function that takes a
raw pointer. Without this comment it is not clear what the function
pointer's safety requirements are for the raw pointer parameter.

>>     // - `ptr` is either NULL or valid by the safety requirements of this function.
> 
> This is the part I already have.

I kept it to ensure that you also keep it.

>>>>> +        let raw_ptr = unsafe {
>>>>> +            // If `size == 0` and `ptr != NULL` the memory behind the pointer is freed.
>>>>> +            self.0(ptr.cast(), size, flags.0).cast()
>>>>> +        };
>>>>> +
>>>>> +        let ptr = if size == 0 {
>>>>> +            NonNull::dangling()
>>>>> +        } else {
>>>>> +            NonNull::new(raw_ptr).ok_or(AllocError)?
>>>>> +        };
>>>>> +
>>>>> +        Ok(NonNull::slice_from_raw_parts(ptr, size))
>>>>> +    }
>>>>> +}
>>>>> +
>>>>> +unsafe impl Allocator for Kmalloc {
>>>>
>>>> Missing SAFETY comment.
>>>
>>> Yeah, I think we came across this in an earlier version of the series. I asked
>>> you about the content and usefulness of a comment here, since I'd just end up
>>> re-iterating what the `Allocator` trait documentation says.
>>>
>>> IIRC, you replied that you want to think of something that'd make sense to add
>>> here.
>>
>> Oh yeah, sorry I forgot about that.
>>
>>> What do you think should be written here?
>>
>> I think the best way to do it, would be to push this question down into
>> `ReallocFunc::call`. So we would put this on the trait:
>>
>>     // SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that
>>     // - memory remains valid until it is explicitly freed,
>>     // - passing a pointer to a vaild memory allocation is OK,
>>     // - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same.
> 
> So, we'd also need the same for:
>   - `unsafe impl Allocator for Vmalloc`
>   - `unsafe impl Allocator for KVmalloc`

Yes.

>> We then need to put this on `ReallocFunc::call`:
>>
>>     /// # Guarantees
>>     ///
>>     /// This method has the same guarantees as `Allocator::realloc`. Additionally
>>     /// - it accepts any pointer to a valid memory allocation allocated by this function.
> 
> You propose this, since for `Allocator::realloc` memory allocated with
> `Allocator::alloc` would be fine too I guess.
> 
> But if e.g. `Kmalloc` wouldn't use the default `Allocator::alloc`, this would be
> valid too.

So if `Kmalloc` were to implement `alloc` by not calling
`ReallocFun::call`, then we couldn't use this comment. Do you think that
such a change might be required at some point?

> We could instead write something like:
> 
> "it accepts any pointer to a valid memory allocation allocated with the same
> kernel allocator."

It would be better, if we can keep it simpler (ie only `realloc` is
implemented).

>>     /// - memory allocated by this function remains valid until it is passed to this function.
> 
> Same here, `Kmalloc` could implement its own `Allocator::free`.
> 
> Maybe just "...until it is explicitly freed.".

I don't really like that, since by that any other function could be
meant. Do you need to override the `free` function? If not then it would
be better.

> Anyway, I'm fine with both, since non of the kernel allocators uses anything
> else than `ReallocFunc::call` to allocate and free memory.
> 
>>
>> Finally, we need a `GUARANTEE` comment (just above the return [^1]
>> value) that establishes these guarantees:
>>
>>     // GUARANTEE: Since we called `self.0` with `size` above and by the type invariants of `Self`,
>>     // `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc`. Those functions provide the guarantees of
>>     // this function.
>>
>> I am not really happy with the last sentence, but I also don't think
>> that there is value in listing out all the guarantees, only to then say
>> "all of this is guaranteed by us calling one of these three functions.
>>
>>
>> [^1]: I am not sure that there is the right place. If you have any
>>       suggestions, feel free to share them.
> 
> Either way, I'm fine with this proposal.
> 
>>
>>
>>>>> +    #[inline]
>>>>> +    unsafe fn realloc(
>>>>> +        ptr: Option<NonNull<u8>>,
>>>>> +        layout: Layout,
>>>>> +        flags: Flags,
>>>>> +    ) -> Result<NonNull<[u8]>, AllocError> {
>>>>> +        // SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`.
>>>>> +        unsafe { ReallocFunc::KREALLOC.call(ptr, layout, flags) }
>>>>> +    }
>>>>> +}
>>
>> Oh one more thing, I know that you already have a lot of patches in this
>> series, but could you split this one into two? So the first one should
>> introduce `ReallocFunc` and the second one add the impl for `Kmalloc`?
>> I managed to confuse me twice because of that :)
> 
> Generally, I'm fine with that, but I'm not sure if I can avoid an intermediate
> compiler warning about unused code doing that.

You can just use `#[expect(dead_code)]` for that in the intermediate
patches.

---
Cheers,
Benno

Re: [PATCH v6 04/26] rust: alloc: implement `Allocator` for `Kmalloc`

[Danilo Krummrich]

On Tue, Sep 10, 2024 at 01:11:35PM +0000, Benno Lossin wrote:
> On 03.09.24 13:48, Danilo Krummrich wrote:
> > On Fri, Aug 30, 2024 at 02:45:35PM +0000, Benno Lossin wrote:
> >> On 30.08.24 00:04, Danilo Krummrich wrote:
> >>> On Thu, Aug 29, 2024 at 06:32:42PM +0000, Benno Lossin wrote:
> >>>> On 16.08.24 02:10, Danilo Krummrich wrote:
> >>>>> +///
> >>>>> +/// For more details see [self].
> >>>>> +pub struct Kmalloc;
> >>>>>
> >>>>>  /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
> >>>>>  fn aligned_size(new_layout: Layout) -> usize {
> >>>>> @@ -36,6 +52,60 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F
> >>>>>      unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 }
> >>>>>  }
> >>>>>
> >>>>> +/// # Invariants
> >>>>> +///
> >>>>> +/// One of the following `krealloc`, `vrealloc`, `kvrealloc`.
> >>>>> +struct ReallocFunc(
> >>>>> +    unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void,
> >>>>> +);
> >>>>> +
> >>>>> +impl ReallocFunc {
> >>>>> +    // INVARIANT: `krealloc` satisfies the type invariants.
> >>>>> +    const KREALLOC: Self = Self(bindings::krealloc);
> >>>>> +
> >>>>> +    /// # Safety
> >>>>> +    ///
> >>>>> +    /// This method has the same safety requirements as [`Allocator::realloc`].
> >>>>> +    unsafe fn call(
> >>>>> +        &self,
> >>>>> +        ptr: Option<NonNull<u8>>,
> >>>>> +        layout: Layout,
> >>>>> +        flags: Flags,
> >>>>> +    ) -> Result<NonNull<[u8]>, AllocError> {
> >>>>> +        let size = aligned_size(layout);
> >>>>> +        let ptr = match ptr {
> >>>>> +            Some(ptr) => ptr.as_ptr(),
> >>>>> +            None => ptr::null(),
> >>>>> +        };
> >>>>> +
> >>>>> +        // SAFETY: `ptr` is either NULL or valid by the safety requirements of this function.
> >>>>
> >>>> You need some justification as to why calling the three allowed
> >>>> functions here.
> >>>
> >>> What kind of justification do I need? Can you please share some more details on
> >>> what you think is missing here?
> >>
> >> So, you are calling a function pointer to an `unsafe` function. This
> >> means that through some invariant you have to know what the safety
> >> requirements are (otherwise how can you guarantee that this is OK?). You
> >> have the invariant that the pointer points at one of the three functions
> >> mentioned above. What are the safety requirements of those functions? I
> >> would assume that the only one is that `ptr` is valid. So you can use:
> >>
> >>     // SAFETY:
> >>     // - `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc` and thus only requires that `ptr` is
> >>     //   NULL or valid.
> > 
> > I'm fine adding it, but I'd like to understand why you think it's required in
> > the safety comment here? Isn't this implicit by being the type invariant?
> 
> You are calling a function pointer to an `unsafe` function that takes a
> raw pointer. Without this comment it is not clear what the function
> pointer's safety requirements are for the raw pointer parameter.

That's my point, isn't this implicitly clear by the type invariant? If needed,
shouldn't it be:

// INVARIANT:
// - `self.0` is one of [...]
//
// SAFETY:
// - `ptr` is either NULL or [...]

> 
> >>     // - `ptr` is either NULL or valid by the safety requirements of this function.
> > 
> > This is the part I already have.
> 
> I kept it to ensure that you also keep it.
> 
> >>>>> +        let raw_ptr = unsafe {
> >>>>> +            // If `size == 0` and `ptr != NULL` the memory behind the pointer is freed.
> >>>>> +            self.0(ptr.cast(), size, flags.0).cast()
> >>>>> +        };
> >>>>> +
> >>>>> +        let ptr = if size == 0 {
> >>>>> +            NonNull::dangling()
> >>>>> +        } else {
> >>>>> +            NonNull::new(raw_ptr).ok_or(AllocError)?
> >>>>> +        };
> >>>>> +
> >>>>> +        Ok(NonNull::slice_from_raw_parts(ptr, size))
> >>>>> +    }
> >>>>> +}
> >>>>> +
> >>>>> +unsafe impl Allocator for Kmalloc {
> >>>>
> >>>> Missing SAFETY comment.
> >>>
> >>> Yeah, I think we came across this in an earlier version of the series. I asked
> >>> you about the content and usefulness of a comment here, since I'd just end up
> >>> re-iterating what the `Allocator` trait documentation says.
> >>>
> >>> IIRC, you replied that you want to think of something that'd make sense to add
> >>> here.
> >>
> >> Oh yeah, sorry I forgot about that.
> >>
> >>> What do you think should be written here?
> >>
> >> I think the best way to do it, would be to push this question down into
> >> `ReallocFunc::call`. So we would put this on the trait:
> >>
> >>     // SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that
> >>     // - memory remains valid until it is explicitly freed,
> >>     // - passing a pointer to a vaild memory allocation is OK,
> >>     // - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same.
> > 
> > So, we'd also need the same for:
> >   - `unsafe impl Allocator for Vmalloc`
> >   - `unsafe impl Allocator for KVmalloc`
> 
> Yes.
> 
> >> We then need to put this on `ReallocFunc::call`:
> >>
> >>     /// # Guarantees
> >>     ///
> >>     /// This method has the same guarantees as `Allocator::realloc`. Additionally
> >>     /// - it accepts any pointer to a valid memory allocation allocated by this function.
> > 
> > You propose this, since for `Allocator::realloc` memory allocated with
> > `Allocator::alloc` would be fine too I guess.
> > 
> > But if e.g. `Kmalloc` wouldn't use the default `Allocator::alloc`, this would be
> > valid too.
> 
> So if `Kmalloc` were to implement `alloc` by not calling
> `ReallocFun::call`, then we couldn't use this comment. Do you think that
> such a change might be required at some point?

I don't think so, this was purely hypothetical. Let's stick to your proposal.

> 
> > We could instead write something like:
> > 
> > "it accepts any pointer to a valid memory allocation allocated with the same
> > kernel allocator."
> 
> It would be better, if we can keep it simpler (ie only `realloc` is
> implemented).
> 
> >>     /// - memory allocated by this function remains valid until it is passed to this function.
> > 
> > Same here, `Kmalloc` could implement its own `Allocator::free`.
> > 
> > Maybe just "...until it is explicitly freed.".
> 
> I don't really like that, since by that any other function could be
> meant. Do you need to override the `free` function? If not then it would
> be better.
> 
> > Anyway, I'm fine with both, since non of the kernel allocators uses anything
> > else than `ReallocFunc::call` to allocate and free memory.
> > 
> >>
> >> Finally, we need a `GUARANTEE` comment (just above the return [^1]
> >> value) that establishes these guarantees:
> >>
> >>     // GUARANTEE: Since we called `self.0` with `size` above and by the type invariants of `Self`,
> >>     // `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc`. Those functions provide the guarantees of
> >>     // this function.
> >>
> >> I am not really happy with the last sentence, but I also don't think
> >> that there is value in listing out all the guarantees, only to then say
> >> "all of this is guaranteed by us calling one of these three functions.
> >>
> >>
> >> [^1]: I am not sure that there is the right place. If you have any
> >>       suggestions, feel free to share them.
> > 
> > Either way, I'm fine with this proposal.
> > 
> >>
> >>
> >>>>> +    #[inline]
> >>>>> +    unsafe fn realloc(
> >>>>> +        ptr: Option<NonNull<u8>>,
> >>>>> +        layout: Layout,
> >>>>> +        flags: Flags,
> >>>>> +    ) -> Result<NonNull<[u8]>, AllocError> {
> >>>>> +        // SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`.
> >>>>> +        unsafe { ReallocFunc::KREALLOC.call(ptr, layout, flags) }
> >>>>> +    }
> >>>>> +}
> >>
> >> Oh one more thing, I know that you already have a lot of patches in this
> >> series, but could you split this one into two? So the first one should
> >> introduce `ReallocFunc` and the second one add the impl for `Kmalloc`?
> >> I managed to confuse me twice because of that :)
> > 
> > Generally, I'm fine with that, but I'm not sure if I can avoid an intermediate
> > compiler warning about unused code doing that.
> 
> You can just use `#[expect(dead_code)]` for that in the intermediate
> patches.

I usually try to avoid that, because it can be misleading when bisecting things.

If the temporarily unused code contains a bug, your bisection doesn't end up at
this patch, but some other patch that starts using it.

> 
> ---
> Cheers,
> Benno
> 

Re: [PATCH v6 04/26] rust: alloc: implement `Allocator` for `Kmalloc`

[Benno Lossin]

On 10.09.24 15:37, Danilo Krummrich wrote:
> On Tue, Sep 10, 2024 at 01:11:35PM +0000, Benno Lossin wrote:
>> On 03.09.24 13:48, Danilo Krummrich wrote:
>>> On Fri, Aug 30, 2024 at 02:45:35PM +0000, Benno Lossin wrote:
>>>> On 30.08.24 00:04, Danilo Krummrich wrote:
>>>>> On Thu, Aug 29, 2024 at 06:32:42PM +0000, Benno Lossin wrote:
>>>>>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>>>>>> +///
>>>>>>> +/// For more details see [self].
>>>>>>> +pub struct Kmalloc;
>>>>>>>
>>>>>>>  /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
>>>>>>>  fn aligned_size(new_layout: Layout) -> usize {
>>>>>>> @@ -36,6 +52,60 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F
>>>>>>>      unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 }
>>>>>>>  }
>>>>>>>
>>>>>>> +/// # Invariants
>>>>>>> +///
>>>>>>> +/// One of the following `krealloc`, `vrealloc`, `kvrealloc`.
>>>>>>> +struct ReallocFunc(
>>>>>>> +    unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void,
>>>>>>> +);
>>>>>>> +
>>>>>>> +impl ReallocFunc {
>>>>>>> +    // INVARIANT: `krealloc` satisfies the type invariants.
>>>>>>> +    const KREALLOC: Self = Self(bindings::krealloc);
>>>>>>> +
>>>>>>> +    /// # Safety
>>>>>>> +    ///
>>>>>>> +    /// This method has the same safety requirements as [`Allocator::realloc`].
>>>>>>> +    unsafe fn call(
>>>>>>> +        &self,
>>>>>>> +        ptr: Option<NonNull<u8>>,
>>>>>>> +        layout: Layout,
>>>>>>> +        flags: Flags,
>>>>>>> +    ) -> Result<NonNull<[u8]>, AllocError> {
>>>>>>> +        let size = aligned_size(layout);
>>>>>>> +        let ptr = match ptr {
>>>>>>> +            Some(ptr) => ptr.as_ptr(),
>>>>>>> +            None => ptr::null(),
>>>>>>> +        };
>>>>>>> +
>>>>>>> +        // SAFETY: `ptr` is either NULL or valid by the safety requirements of this function.
>>>>>>
>>>>>> You need some justification as to why calling the three allowed
>>>>>> functions here.
>>>>>
>>>>> What kind of justification do I need? Can you please share some more details on
>>>>> what you think is missing here?
>>>>
>>>> So, you are calling a function pointer to an `unsafe` function. This
>>>> means that through some invariant you have to know what the safety
>>>> requirements are (otherwise how can you guarantee that this is OK?). You
>>>> have the invariant that the pointer points at one of the three functions
>>>> mentioned above. What are the safety requirements of those functions? I
>>>> would assume that the only one is that `ptr` is valid. So you can use:
>>>>
>>>>     // SAFETY:
>>>>     // - `self.0` is one of `krealloc`, `vrealloc`, `kvrealloc` and thus only requires that `ptr` is
>>>>     //   NULL or valid.
>>>
>>> I'm fine adding it, but I'd like to understand why you think it's required in
>>> the safety comment here? Isn't this implicit by being the type invariant?
>>
>> You are calling a function pointer to an `unsafe` function that takes a
>> raw pointer. Without this comment it is not clear what the function
>> pointer's safety requirements are for the raw pointer parameter.
> 
> That's my point, isn't this implicitly clear by the type invariant? If needed,
> shouldn't it be:

I would argue that it is not implicitly clear, since to the reader of
just that unsafe block it's totally unclear that `self.0` has such an
invariant. They would have to read the type definition.

> // INVARIANT:
> // - `self.0` is one of [...]
> //
> // SAFETY:
> // - `ptr` is either NULL or [...]
> 
>>
>>>>     // - `ptr` is either NULL or valid by the safety requirements of this function.
>>>
>>> This is the part I already have.
>>
>> I kept it to ensure that you also keep it.

[...]

>>>>>>> +    #[inline]
>>>>>>> +    unsafe fn realloc(
>>>>>>> +        ptr: Option<NonNull<u8>>,
>>>>>>> +        layout: Layout,
>>>>>>> +        flags: Flags,
>>>>>>> +    ) -> Result<NonNull<[u8]>, AllocError> {
>>>>>>> +        // SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`.
>>>>>>> +        unsafe { ReallocFunc::KREALLOC.call(ptr, layout, flags) }
>>>>>>> +    }
>>>>>>> +}
>>>>
>>>> Oh one more thing, I know that you already have a lot of patches in this
>>>> series, but could you split this one into two? So the first one should
>>>> introduce `ReallocFunc` and the second one add the impl for `Kmalloc`?
>>>> I managed to confuse me twice because of that :)
>>>
>>> Generally, I'm fine with that, but I'm not sure if I can avoid an intermediate
>>> compiler warning about unused code doing that.
>>
>> You can just use `#[expect(dead_code)]` for that in the intermediate
>> patches.
> 
> I usually try to avoid that, because it can be misleading when bisecting things.
> 
> If the temporarily unused code contains a bug, your bisection doesn't end up at
> this patch, but some other patch that starts using it.

I don't think it's a problem in this case, since the two patches are
directly next to each other and you're not changing existing code, just
splitting up the addition of new code.

---
Cheers,
Benno

[PATCH v6 05/26] rust: alloc: add module `allocator_test`

[Danilo Krummrich]

`Allocator`s, such as `Kmalloc`, will be used by e.g. `Box` and `Vec` in
subsequent patches, and hence this dependency propagates throughout the
whole kernel.

Add the `allocator_test` module that provides an empty implementation
for all `Allocator`s in the kernel, such that we don't break the
`rusttest` make target in subsequent patches.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc.rs                |  9 +++++++--
 rust/kernel/alloc/allocator_test.rs | 19 +++++++++++++++++++
 2 files changed, 26 insertions(+), 2 deletions(-)
 create mode 100644 rust/kernel/alloc/allocator_test.rs

diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index 477dbe3c5a2f..91a367a58f36 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -2,12 +2,17 @@
 
 //! Extensions to the [`alloc`] crate.
 
-#[cfg(not(test))]
-#[cfg(not(testlib))]
+#[cfg(not(any(test, testlib)))]
 pub mod allocator;
 pub mod box_ext;
 pub mod vec_ext;
 
+#[cfg(any(test, testlib))]
+pub mod allocator_test;
+
+#[cfg(any(test, testlib))]
+pub use self::allocator_test as allocator;
+
 /// Indicates an allocation error.
 #[derive(Copy, Clone, PartialEq, Eq, Debug)]
 pub struct AllocError;
diff --git a/rust/kernel/alloc/allocator_test.rs b/rust/kernel/alloc/allocator_test.rs
new file mode 100644
index 000000000000..4785efc474a7
--- /dev/null
+++ b/rust/kernel/alloc/allocator_test.rs
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#![allow(missing_docs)]
+
+use super::{AllocError, Allocator, Flags};
+use core::alloc::Layout;
+use core::ptr::NonNull;
+
+pub struct Kmalloc;
+
+unsafe impl Allocator for Kmalloc {
+    unsafe fn realloc(
+        _ptr: Option<NonNull<u8>>,
+        _layout: Layout,
+        _flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        panic!();
+    }
+}
-- 
2.46.0

Re: [PATCH v6 05/26] rust: alloc: add module `allocator_test`

[Gary Guo]

On Fri, 16 Aug 2024 02:10:47 +0200
Danilo Krummrich <dakr@kernel.org> wrote:

> `Allocator`s, such as `Kmalloc`, will be used by e.g. `Box` and `Vec` in
> subsequent patches, and hence this dependency propagates throughout the
> whole kernel.
> 
> Add the `allocator_test` module that provides an empty implementation
> for all `Allocator`s in the kernel, such that we don't break the
> `rusttest` make target in subsequent patches.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Reviewed-by: Benno Lossin <benno.lossin@proton.me>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

Reviewed-by: Gary Guo <gary@garyguo.net>

> ---
>  rust/kernel/alloc.rs                |  9 +++++++--
>  rust/kernel/alloc/allocator_test.rs | 19 +++++++++++++++++++
>  2 files changed, 26 insertions(+), 2 deletions(-)
>  create mode 100644 rust/kernel/alloc/allocator_test.rs
> 
> diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> index 477dbe3c5a2f..91a367a58f36 100644
> --- a/rust/kernel/alloc.rs
> +++ b/rust/kernel/alloc.rs
> @@ -2,12 +2,17 @@
>  
>  //! Extensions to the [`alloc`] crate.
>  
> -#[cfg(not(test))]
> -#[cfg(not(testlib))]
> +#[cfg(not(any(test, testlib)))]
>  pub mod allocator;
>  pub mod box_ext;
>  pub mod vec_ext;
>  
> +#[cfg(any(test, testlib))]
> +pub mod allocator_test;
> +
> +#[cfg(any(test, testlib))]
> +pub use self::allocator_test as allocator;
> +
>  /// Indicates an allocation error.
>  #[derive(Copy, Clone, PartialEq, Eq, Debug)]
>  pub struct AllocError;
> diff --git a/rust/kernel/alloc/allocator_test.rs b/rust/kernel/alloc/allocator_test.rs
> new file mode 100644
> index 000000000000..4785efc474a7
> --- /dev/null
> +++ b/rust/kernel/alloc/allocator_test.rs
> @@ -0,0 +1,19 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +#![allow(missing_docs)]
> +
> +use super::{AllocError, Allocator, Flags};
> +use core::alloc::Layout;
> +use core::ptr::NonNull;
> +
> +pub struct Kmalloc;
> +
> +unsafe impl Allocator for Kmalloc {
> +    unsafe fn realloc(
> +        _ptr: Option<NonNull<u8>>,
> +        _layout: Layout,
> +        _flags: Flags,
> +    ) -> Result<NonNull<[u8]>, AllocError> {
> +        panic!();
> +    }
> +}

[PATCH v6 06/26] rust: alloc: implement `Vmalloc` allocator

[Danilo Krummrich]

Implement `Allocator` for `Vmalloc`, the kernel's virtually contiguous
allocator, typically used for larger objects, (much) larger than page
size.

All memory allocations made with `Vmalloc` end up in `vrealloc()`.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/helpers.c                      |  8 ++++++++
 rust/kernel/alloc/allocator.rs      | 32 +++++++++++++++++++++++++++++
 rust/kernel/alloc/allocator_test.rs |  1 +
 3 files changed, 41 insertions(+)

diff --git a/rust/helpers.c b/rust/helpers.c
index 41df5b1a6de6..96f8b37a668b 100644
--- a/rust/helpers.c
+++ b/rust/helpers.c
@@ -36,6 +36,7 @@
 #include <linux/security.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
+#include <linux/vmalloc.h>
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 
@@ -248,6 +249,13 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags)
 }
 EXPORT_SYMBOL_GPL(rust_helper_krealloc);
 
+void * __must_check __realloc_size(2)
+rust_helper_vrealloc(const void *p, size_t size, gfp_t flags)
+{
+	return vrealloc(p, size, flags);
+}
+EXPORT_SYMBOL_GPL(rust_helper_vrealloc);
+
 void rust_helper_rb_link_node(struct rb_node *node, struct rb_node *parent,
 			      struct rb_node **rb_link)
 {
diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
index 78e7d5488843..09a322e9168d 100644
--- a/rust/kernel/alloc/allocator.rs
+++ b/rust/kernel/alloc/allocator.rs
@@ -15,6 +15,7 @@
 
 use crate::alloc::{AllocError, Allocator};
 use crate::bindings;
+use crate::pr_warn;
 
 /// The contiguous kernel allocator.
 ///
@@ -24,6 +25,15 @@
 /// For more details see [self].
 pub struct Kmalloc;
 
+/// The virtually contiguous kernel allocator.
+///
+/// `Vmalloc` allocates pages from the page level allocator and maps them into the contiguous kernel
+/// virtual space. It is typically used for large allocations. The memory allocated with this
+/// allocator is not physically contiguous.
+///
+/// For more details see [self].
+pub struct Vmalloc;
+
 /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
 fn aligned_size(new_layout: Layout) -> usize {
     // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.
@@ -63,6 +73,9 @@ impl ReallocFunc {
     // INVARIANT: `krealloc` satisfies the type invariants.
     const KREALLOC: Self = Self(bindings::krealloc);
 
+    // INVARIANT: `vrealloc` satisfies the type invariants.
+    const VREALLOC: Self = Self(bindings::vrealloc);
+
     /// # Safety
     ///
     /// This method has the same safety requirements as [`Allocator::realloc`].
@@ -141,6 +154,25 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
     }
 }
 
+unsafe impl Allocator for Vmalloc {
+    #[inline]
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        // TODO: Support alignments larger than PAGE_SIZE.
+        if layout.align() > bindings::PAGE_SIZE {
+            pr_warn!("Vmalloc does not support alignments larger than PAGE_SIZE yet.\n");
+            return Err(AllocError);
+        }
+
+        // SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously
+        // allocated with this `Allocator`.
+        unsafe { ReallocFunc::VREALLOC.call(ptr, layout, flags) }
+    }
+}
+
 #[global_allocator]
 static ALLOCATOR: Kmalloc = Kmalloc;
 
diff --git a/rust/kernel/alloc/allocator_test.rs b/rust/kernel/alloc/allocator_test.rs
index 4785efc474a7..e7bf2982f68f 100644
--- a/rust/kernel/alloc/allocator_test.rs
+++ b/rust/kernel/alloc/allocator_test.rs
@@ -7,6 +7,7 @@
 use core::ptr::NonNull;
 
 pub struct Kmalloc;
+pub type Vmalloc = Kmalloc;
 
 unsafe impl Allocator for Kmalloc {
     unsafe fn realloc(
-- 
2.46.0

Re: [PATCH v6 06/26] rust: alloc: implement `Vmalloc` allocator

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> @@ -63,6 +73,9 @@ impl ReallocFunc {
>      // INVARIANT: `krealloc` satisfies the type invariants.
>      const KREALLOC: Self = Self(bindings::krealloc);
> 
> +    // INVARIANT: `vrealloc` satisfies the type invariants.
> +    const VREALLOC: Self = Self(bindings::vrealloc);
> +
>      /// # Safety
>      ///
>      /// This method has the same safety requirements as [`Allocator::realloc`].
> @@ -141,6 +154,25 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
>      }
>  }
> 
> +unsafe impl Allocator for Vmalloc {

The same safety comment as with `Kmalloc` should fit here (and with
`KVmalloc` too).

---
Cheers,
Benno

> +    #[inline]
> +    unsafe fn realloc(
> +        ptr: Option<NonNull<u8>>,
> +        layout: Layout,
> +        flags: Flags,
> +    ) -> Result<NonNull<[u8]>, AllocError> {
> +        // TODO: Support alignments larger than PAGE_SIZE.
> +        if layout.align() > bindings::PAGE_SIZE {
> +            pr_warn!("Vmalloc does not support alignments larger than PAGE_SIZE yet.\n");
> +            return Err(AllocError);
> +        }
> +
> +        // SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously
> +        // allocated with this `Allocator`.
> +        unsafe { ReallocFunc::VREALLOC.call(ptr, layout, flags) }
> +    }
> +}
> +
>  #[global_allocator]
>  static ALLOCATOR: Kmalloc = Kmalloc;
> 
> diff --git a/rust/kernel/alloc/allocator_test.rs b/rust/kernel/alloc/allocator_test.rs
> index 4785efc474a7..e7bf2982f68f 100644
> --- a/rust/kernel/alloc/allocator_test.rs
> +++ b/rust/kernel/alloc/allocator_test.rs
> @@ -7,6 +7,7 @@
>  use core::ptr::NonNull;
> 
>  pub struct Kmalloc;
> +pub type Vmalloc = Kmalloc;
> 
>  unsafe impl Allocator for Kmalloc {
>      unsafe fn realloc(
> --
> 2.46.0
> 

[PATCH v6 07/26] rust: alloc: implement `KVmalloc` allocator

[Danilo Krummrich]

Implement `Allocator` for `KVmalloc`, an `Allocator` that tries to
allocate memory wth `kmalloc` first and, on failure, falls back to
`vmalloc`.

All memory allocations made with `KVmalloc` end up in
`kvrealloc_noprof()`; all frees in `kvfree()`.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/helpers.c                      |  7 +++++++
 rust/kernel/alloc/allocator.rs      | 31 +++++++++++++++++++++++++++++
 rust/kernel/alloc/allocator_test.rs |  1 +
 3 files changed, 39 insertions(+)

diff --git a/rust/helpers.c b/rust/helpers.c
index 96f8b37a668b..9a7604e3a749 100644
--- a/rust/helpers.c
+++ b/rust/helpers.c
@@ -256,6 +256,13 @@ rust_helper_vrealloc(const void *p, size_t size, gfp_t flags)
 }
 EXPORT_SYMBOL_GPL(rust_helper_vrealloc);
 
+void * __must_check __realloc_size(2)
+rust_helper_kvrealloc(const void *p, size_t size, gfp_t flags)
+{
+	return kvrealloc(p, size, flags);
+}
+EXPORT_SYMBOL_GPL(rust_helper_kvrealloc);
+
 void rust_helper_rb_link_node(struct rb_node *node, struct rb_node *parent,
 			      struct rb_node **rb_link)
 {
diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
index 09a322e9168d..0183fbac1ada 100644
--- a/rust/kernel/alloc/allocator.rs
+++ b/rust/kernel/alloc/allocator.rs
@@ -34,6 +34,15 @@
 /// For more details see [self].
 pub struct Vmalloc;
 
+/// The kvmalloc kernel allocator.
+///
+/// `KVmalloc` attempts to allocate memory with `Kmalloc` first, but falls back to `Vmalloc` upon
+/// failure. This allocator is typically used when the size for the requsted allocation is not
+/// known and may exceed the capabilities of `Kmalloc`.
+///
+/// For more details see [self].
+pub struct KVmalloc;
+
 /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
 fn aligned_size(new_layout: Layout) -> usize {
     // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.
@@ -76,6 +85,9 @@ impl ReallocFunc {
     // INVARIANT: `vrealloc` satisfies the type invariants.
     const VREALLOC: Self = Self(bindings::vrealloc);
 
+    // INVARIANT: `kvrealloc` satisfies the type invariants.
+    const KVREALLOC: Self = Self(bindings::kvrealloc);
+
     /// # Safety
     ///
     /// This method has the same safety requirements as [`Allocator::realloc`].
@@ -173,6 +185,25 @@ unsafe fn realloc(
     }
 }
 
+unsafe impl Allocator for KVmalloc {
+    #[inline]
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        // TODO: Support alignments larger than PAGE_SIZE.
+        if layout.align() > bindings::PAGE_SIZE {
+            pr_warn!("KVmalloc does not support alignments larger than PAGE_SIZE yet.\n");
+            return Err(AllocError);
+        }
+
+        // SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously
+        // allocated with this `Allocator`.
+        unsafe { ReallocFunc::KVREALLOC.call(ptr, layout, flags) }
+    }
+}
+
 #[global_allocator]
 static ALLOCATOR: Kmalloc = Kmalloc;
 
diff --git a/rust/kernel/alloc/allocator_test.rs b/rust/kernel/alloc/allocator_test.rs
index e7bf2982f68f..1b2642c547ec 100644
--- a/rust/kernel/alloc/allocator_test.rs
+++ b/rust/kernel/alloc/allocator_test.rs
@@ -8,6 +8,7 @@
 
 pub struct Kmalloc;
 pub type Vmalloc = Kmalloc;
+pub type KVmalloc = Kmalloc;
 
 unsafe impl Allocator for Kmalloc {
     unsafe fn realloc(
-- 
2.46.0

[PATCH v6 08/26] rust: alloc: add __GFP_NOWARN to `Flags`

[Danilo Krummrich]

Some test cases in subsequent patches provoke allocation failures. Add
`__GFP_NOWARN` to enable test cases to silence unpleasant warnings.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/bindings/bindings_helper.h | 1 +
 rust/kernel/alloc.rs            | 5 +++++
 2 files changed, 6 insertions(+)

diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h
index b5793142904d..90493123bdf1 100644
--- a/rust/bindings/bindings_helper.h
+++ b/rust/bindings/bindings_helper.h
@@ -37,4 +37,5 @@ const gfp_t RUST_CONST_HELPER_GFP_KERNEL_ACCOUNT = GFP_KERNEL_ACCOUNT;
 const gfp_t RUST_CONST_HELPER_GFP_NOWAIT = GFP_NOWAIT;
 const gfp_t RUST_CONST_HELPER___GFP_ZERO = __GFP_ZERO;
 const gfp_t RUST_CONST_HELPER___GFP_HIGHMEM = ___GFP_HIGHMEM;
+const gfp_t RUST_CONST_HELPER___GFP_NOWARN = ___GFP_NOWARN;
 const blk_features_t RUST_CONST_HELPER_BLK_FEAT_ROTATIONAL = BLK_FEAT_ROTATIONAL;
diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index 91a367a58f36..53a93617a9f6 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -91,6 +91,11 @@ pub mod flags {
     /// use any filesystem callback.  It is very likely to fail to allocate memory, even for very
     /// small allocations.
     pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT);
+
+    /// Suppresses allocation failure reports.
+    ///
+    /// This is normally or'd with other flags.
+    pub const __GFP_NOWARN: Flags = Flags(bindings::__GFP_NOWARN);
 }
 
 /// The kernel's [`Allocator`] trait.
-- 
2.46.0

[PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Danilo Krummrich]

`Box` provides the simplest way to allocate memory for a generic type
with one of the kernel's allocators, e.g. `Kmalloc`, `Vmalloc` or
`KVmalloc`.

In contrast to Rust's `Box` type, the kernel `Box` type considers the
kernel's GFP flags for all appropriate functions, always reports
allocation failures through `Result<_, AllocError>` and remains
independent from unstable features.

Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc.rs      |   6 +
 rust/kernel/alloc/kbox.rs | 480 ++++++++++++++++++++++++++++++++++++++
 rust/kernel/prelude.rs    |   2 +-
 3 files changed, 487 insertions(+), 1 deletion(-)
 create mode 100644 rust/kernel/alloc/kbox.rs

diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index 53a93617a9f6..0c9bb60250af 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -5,6 +5,7 @@
 #[cfg(not(any(test, testlib)))]
 pub mod allocator;
 pub mod box_ext;
+pub mod kbox;
 pub mod vec_ext;
 
 #[cfg(any(test, testlib))]
@@ -13,6 +14,11 @@
 #[cfg(any(test, testlib))]
 pub use self::allocator_test as allocator;
 
+pub use self::kbox::Box;
+pub use self::kbox::KBox;
+pub use self::kbox::KVBox;
+pub use self::kbox::VBox;
+
 /// Indicates an allocation error.
 #[derive(Copy, Clone, PartialEq, Eq, Debug)]
 pub struct AllocError;
diff --git a/rust/kernel/alloc/kbox.rs b/rust/kernel/alloc/kbox.rs
new file mode 100644
index 000000000000..93b1ab9de6e8
--- /dev/null
+++ b/rust/kernel/alloc/kbox.rs
@@ -0,0 +1,480 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Implementation of [`Box`].
+
+#[allow(unused_imports)] // Used in doc comments.
+use super::allocator::{KVmalloc, Kmalloc, Vmalloc};
+use super::{AllocError, Allocator, Flags};
+use core::fmt;
+use core::marker::PhantomData;
+use core::mem::ManuallyDrop;
+use core::mem::MaybeUninit;
+use core::ops::{Deref, DerefMut};
+use core::pin::Pin;
+use core::ptr::NonNull;
+use core::result::Result;
+
+use crate::init::{InPlaceInit, InPlaceWrite, Init, PinInit};
+use crate::types::ForeignOwnable;
+
+/// The kernel's [`Box`] type - a heap allocation for a single value of type `T`.
+///
+/// This is the kernel's version of the Rust stdlib's `Box`. There are several of differences,
+/// for example no `noalias` attribute is emitted and partially moving out of a `Box` is not
+/// supported. There are also several API differences, e.g. `Box` always requires an [`Allocator`]
+/// implementation to be passed as generic, page [`Flags`] when allocating memory and all functions
+/// that may allocate memory are failable.
+///
+/// `Box` works with any of the kernel's allocators, e.g. [`Kmalloc`], [`Vmalloc`] or [`KVmalloc`].
+/// There are aliases for `Box` with these allocators ([`KBox`], [`VBox`], [`KVBox`]).
+///
+/// When dropping a [`Box`], the value is also dropped and the heap memory is automatically freed.
+///
+/// # Examples
+///
+/// ```
+/// let b = KBox::<u64>::new(24_u64, GFP_KERNEL)?;
+///
+/// assert_eq!(*b, 24_u64);
+/// # Ok::<(), Error>(())
+/// ```
+///
+/// ```
+/// # use kernel::bindings;
+/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
+/// struct Huge([u8; SIZE]);
+///
+/// assert!(KBox::<Huge>::new_uninit(GFP_KERNEL | __GFP_NOWARN).is_err());
+/// ```
+///
+/// ```
+/// # use kernel::bindings;
+/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
+/// struct Huge([u8; SIZE]);
+///
+/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
+/// ```
+///
+/// # Invariants
+///
+/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`
+/// or, for zero-sized types, is a dangling pointer.
+#[repr(transparent)]
+pub struct Box<T: ?Sized, A: Allocator>(NonNull<T>, PhantomData<A>);
+
+/// Type alias for `Box` with a [`Kmalloc`] allocator.
+///
+/// # Examples
+///
+/// ```
+/// let b = KBox::new(24_u64, GFP_KERNEL)?;
+///
+/// assert_eq!(*b, 24_u64);
+/// # Ok::<(), Error>(())
+/// ```
+pub type KBox<T> = Box<T, super::allocator::Kmalloc>;
+
+/// Type alias for `Box` with a [`Vmalloc`] allocator.
+///
+/// # Examples
+///
+/// ```
+/// let b = VBox::new(24_u64, GFP_KERNEL)?;
+///
+/// assert_eq!(*b, 24_u64);
+/// # Ok::<(), Error>(())
+/// ```
+pub type VBox<T> = Box<T, super::allocator::Vmalloc>;
+
+/// Type alias for `Box` with a [`KVmalloc`] allocator.
+///
+/// # Examples
+///
+/// ```
+/// let b = KVBox::new(24_u64, GFP_KERNEL)?;
+///
+/// assert_eq!(*b, 24_u64);
+/// # Ok::<(), Error>(())
+/// ```
+pub type KVBox<T> = Box<T, super::allocator::KVmalloc>;
+
+// SAFETY: `Box` is `Send` if `T` is `Send` because the `Box` owns a `T`.
+unsafe impl<T, A> Send for Box<T, A>
+where
+    T: Send + ?Sized,
+    A: Allocator,
+{
+}
+
+// SAFETY: `Box` is `Sync` if `T` is `Sync` because the `Box` owns a `T`.
+unsafe impl<T, A> Sync for Box<T, A>
+where
+    T: Sync + ?Sized,
+    A: Allocator,
+{
+}
+
+impl<T, A> Box<T, A>
+where
+    T: ?Sized,
+    A: Allocator,
+{
+    /// Creates a new `Box<T, A>` from a raw pointer.
+    ///
+    /// # Safety
+    ///
+    /// For non-ZSTs, `raw` must point at an allocation allocated with `A`that is sufficiently
+    /// aligned for and holds a valid `T`. The caller passes ownership of the allocation to the
+    /// `Box`.
+    #[inline]
+    pub const unsafe fn from_raw(raw: *mut T) -> Self {
+        // INVARIANT: Validity of `raw` is guaranteed by the safety preconditions of this function.
+        // SAFETY: By the safety preconditions of this function, `raw` is not a NULL pointer.
+        Self(unsafe { NonNull::new_unchecked(raw) }, PhantomData::<A>)
+    }
+
+    /// Consumes the `Box<T, A>` and returns a raw pointer.
+    ///
+    /// This will not run the destructor of `T` and for non-ZSTs the allocation will stay alive
+    /// indefinitely. Use [`Box::from_raw`] to recover the [`Box`], drop the value and free the
+    /// allocation, if any.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let x = KBox::new(24, GFP_KERNEL)?;
+    /// let ptr = KBox::into_raw(x);
+    /// let x = unsafe { KBox::from_raw(ptr) };
+    ///
+    /// assert_eq!(*x, 24);
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    pub fn into_raw(b: Self) -> *mut T {
+        let b = ManuallyDrop::new(b);
+
+        b.0.as_ptr()
+    }
+
+    /// Consumes and leaks the `Box<T, A>` and returns a mutable reference.
+    ///
+    /// See [Box::into_raw] for more details.
+    #[inline]
+    pub fn leak<'a>(b: Self) -> &'a mut T {
+        // SAFETY: `Box::into_raw` always returns a properly aligned and dereferenceable pointer
+        // which points to an initialized instance of `T`.
+        unsafe { &mut *Box::into_raw(b) }
+    }
+}
+
+impl<T, A> Box<MaybeUninit<T>, A>
+where
+    A: Allocator,
+{
+    /// Converts a `Box<MaybeUninit<T>, A>` to a `Box<T, A>`.
+    ///
+    /// It is undefined behavior to call this function while the value inside of `b` is not yet
+    /// fully initialized.
+    ///
+    /// # Safety
+    ///
+    /// Callers must ensure that the value inside of `b` is in an initialized state.
+    pub unsafe fn assume_init(b: Self) -> Box<T, A> {
+        let raw = Self::into_raw(b);
+
+        // SAFETY: `raw` comes from a previous call to `Box::into_raw`. By the safety requirements
+        // of this function, the value inside the `Box` is in an initialized state. Hence, it is
+        // safe to reconstruct the `Box` as `Box<T, A>`.
+        unsafe { Box::from_raw(raw as *mut T) }
+    }
+
+    /// Writes the value and converts to `Box<T, A>`.
+    pub fn write(mut b: Self, value: T) -> Box<T, A> {
+        (*b).write(value);
+        // SAFETY: We've just initialized `boxed`'s value.
+        unsafe { Self::assume_init(b) }
+    }
+}
+
+impl<T, A> Box<T, A>
+where
+    A: Allocator,
+{
+    fn is_zst() -> bool {
+        core::mem::size_of::<T>() == 0
+    }
+
+    /// Creates a new `Box<T, A>` and initializes its contents with `x`.
+    ///
+    /// New memory is allocated with `A`. The allocation may fail, in which case an error is
+    /// returned. For ZSTs no memory is allocated.
+    pub fn new(x: T, flags: Flags) -> Result<Self, AllocError> {
+        let b = Self::new_uninit(flags)?;
+        Ok(Box::write(b, x))
+    }
+
+    /// Creates a new `Box<T, A>` with uninitialized contents.
+    ///
+    /// New memory is allocated with `A`. The allocation may fail, in which case an error is
+    /// returned. For ZSTs no memory is allocated.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let b = KBox::<u64>::new_uninit(GFP_KERNEL)?;
+    /// let b = KBox::write(b, 24);
+    ///
+    /// assert_eq!(*b, 24_u64);
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub fn new_uninit(flags: Flags) -> Result<Box<MaybeUninit<T>, A>, AllocError> {
+        let ptr = if Self::is_zst() {
+            NonNull::dangling()
+        } else {
+            let layout = core::alloc::Layout::new::<MaybeUninit<T>>();
+            let ptr = A::alloc(layout, flags)?;
+
+            ptr.cast()
+        };
+
+        // INVARIANT: `ptr` is either a dangling pointer or points to memory allocated with `A`,
+        // which is sufficient in size and alignment for storing a `T`.
+        Ok(Box(ptr, PhantomData::<A>))
+    }
+
+    /// Constructs a new `Pin<Box<T, A>>`. If `T` does not implement [`Unpin`], then `x` will be
+    /// pinned in memory and can't be moved.
+    #[inline]
+    pub fn pin(x: T, flags: Flags) -> Result<Pin<Box<T, A>>, AllocError>
+    where
+        A: 'static,
+    {
+        Ok(Self::new(x, flags)?.into())
+    }
+
+    /// Forgets the contents (does not run the destructor), but keeps the allocation.
+    fn forget_contents(this: Self) -> Box<MaybeUninit<T>, A> {
+        let ptr = Self::into_raw(this);
+
+        // SAFETY: `ptr` is valid, because it came from `Box::into_raw`.
+        unsafe { Box::from_raw(ptr.cast()) }
+    }
+
+    /// Drops the contents, but keeps the allocation.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let value = KBox::new([0; 32], GFP_KERNEL)?;
+    /// assert_eq!(*value, [0; 32]);
+    /// let value = KBox::drop_contents(value);
+    /// // Now we can re-use `value`:
+    /// let value = KBox::write(value, [1; 32]);
+    /// assert_eq!(*value, [1; 32]);
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub fn drop_contents(this: Self) -> Box<MaybeUninit<T>, A> {
+        let ptr = this.0.as_ptr();
+
+        // SAFETY: `ptr` is valid, because it came from `this`. After this call we never access the
+        // value stored in `this` again.
+        unsafe { core::ptr::drop_in_place(ptr) };
+
+        Self::forget_contents(this)
+    }
+
+    /// Moves the `Box`' value out of the `Box` and consumes the `Box`.
+    pub fn into_inner(b: Self) -> T {
+        // SAFETY: By the type invariant `&*b` is valid for `read`.
+        let value = unsafe { core::ptr::read(&*b) };
+        let _ = Self::forget_contents(b);
+        value
+    }
+}
+
+impl<T, A> From<Box<T, A>> for Pin<Box<T, A>>
+where
+    T: ?Sized,
+    A: Allocator,
+{
+    /// Converts a `Box<T, A>` into a `Pin<Box<T, A>>`. If `T` does not implement [`Unpin`], then
+    /// `*b` will be pinned in memory and can't be moved.
+    ///
+    /// This moves `b` into `Pin` without moving `*b` or allocating and copying any memory.
+    fn from(b: Box<T, A>) -> Self {
+        // SAFETY: The value wrapped inside a `Pin<Box<T, A>>` cannot be moved or replaced as long
+        // as `T` does not implement `Unpin`.
+        unsafe { Pin::new_unchecked(b) }
+    }
+}
+
+impl<T, A> InPlaceWrite<T> for Box<MaybeUninit<T>, A>
+where
+    A: Allocator + 'static,
+{
+    type Initialized = Box<T, A>;
+
+    fn write_init<E>(mut self, init: impl Init<T, E>) -> Result<Self::Initialized, E> {
+        let slot = self.as_mut_ptr();
+        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
+        // slot is valid.
+        unsafe { init.__init(slot)? };
+        // SAFETY: All fields have been initialized.
+        Ok(unsafe { Box::assume_init(self) })
+    }
+
+    fn write_pin_init<E>(mut self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E> {
+        let slot = self.as_mut_ptr();
+        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
+        // slot is valid and will not be moved, because we pin it later.
+        unsafe { init.__pinned_init(slot)? };
+        // SAFETY: All fields have been initialized.
+        Ok(unsafe { Box::assume_init(self) }.into())
+    }
+}
+
+impl<T, A> InPlaceInit<T> for Box<T, A>
+where
+    A: Allocator + 'static,
+{
+    #[inline]
+    fn try_pin_init<E>(init: impl PinInit<T, E>, flags: Flags) -> Result<Pin<Self>, E>
+    where
+        E: From<AllocError>,
+    {
+        Box::<_, A>::new_uninit(flags)?.write_pin_init(init)
+    }
+
+    #[inline]
+    fn try_init<E>(init: impl Init<T, E>, flags: Flags) -> Result<Self, E>
+    where
+        E: From<AllocError>,
+    {
+        Box::<_, A>::new_uninit(flags)?.write_init(init)
+    }
+}
+
+impl<T: 'static, A> ForeignOwnable for Box<T, A>
+where
+    A: Allocator,
+{
+    type Borrowed<'a> = &'a T;
+    type BorrowedMut<'a> = &'a mut T;
+
+    fn into_foreign(self) -> *const core::ffi::c_void {
+        Box::into_raw(self) as _
+    }
+
+    unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self {
+        // SAFETY: The safety requirements of this function ensure that `ptr` comes from a previous
+        // call to `Self::into_foreign`.
+        unsafe { Box::from_raw(ptr as _) }
+    }
+
+    unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> &'a T {
+        // SAFETY: The safety requirements of this method ensure that the object remains alive and
+        // immutable for the duration of 'a.
+        unsafe { &*ptr.cast() }
+    }
+
+    unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> &'a mut T {
+        // SAFETY: The safety requirements of this method ensure that the pointer is valid and that
+        // nothing else will access the value for the duration of 'a.
+        unsafe { &mut *ptr.cast_mut().cast() }
+    }
+}
+
+impl<T: 'static, A> ForeignOwnable for Pin<Box<T, A>>
+where
+    A: Allocator,
+{
+    type Borrowed<'a> = Pin<&'a T>;
+    type BorrowedMut<'a> = Pin<&'a mut T>;
+
+    fn into_foreign(self) -> *const core::ffi::c_void {
+        // SAFETY: We are still treating the box as pinned.
+        Box::into_raw(unsafe { Pin::into_inner_unchecked(self) }) as _
+    }
+
+    unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self {
+        // SAFETY: The safety requirements of this function ensure that `ptr` comes from a previous
+        // call to `Self::into_foreign`.
+        unsafe { Pin::new_unchecked(Box::from_raw(ptr as _)) }
+    }
+
+    unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> Pin<&'a T> {
+        // SAFETY: The safety requirements for this function ensure that the object is still alive,
+        // so it is safe to dereference the raw pointer.
+        // The safety requirements of `from_foreign` also ensure that the object remains alive for
+        // the lifetime of the returned value.
+        let r = unsafe { &*ptr.cast() };
+
+        // SAFETY: This pointer originates from a `Pin<Box<T>>`.
+        unsafe { Pin::new_unchecked(r) }
+    }
+
+    unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> Pin<&'a mut T> {
+        // SAFETY: The safety requirements for this function ensure that the object is still alive,
+        // so it is safe to dereference the raw pointer.
+        // The safety requirements of `from_foreign` also ensure that the object remains alive for
+        // the lifetime of the returned value.
+        let r = unsafe { &mut *ptr.cast_mut().cast() };
+
+        // SAFETY: This pointer originates from a `Pin<Box<T>>`.
+        unsafe { Pin::new_unchecked(r) }
+    }
+}
+
+impl<T, A> Deref for Box<T, A>
+where
+    T: ?Sized,
+    A: Allocator,
+{
+    type Target = T;
+
+    fn deref(&self) -> &T {
+        // SAFETY: `self.0` is always properly aligned, dereferenceable and points to an initialized
+        // instance of `T`.
+        unsafe { self.0.as_ref() }
+    }
+}
+
+impl<T, A> DerefMut for Box<T, A>
+where
+    T: ?Sized,
+    A: Allocator,
+{
+    fn deref_mut(&mut self) -> &mut T {
+        // SAFETY: `self.0` is always properly aligned, dereferenceable and points to an initialized
+        // instance of `T`.
+        unsafe { self.0.as_mut() }
+    }
+}
+
+impl<T, A> fmt::Debug for Box<T, A>
+where
+    T: ?Sized + fmt::Debug,
+    A: Allocator,
+{
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        fmt::Debug::fmt(&**self, f)
+    }
+}
+
+impl<T, A> Drop for Box<T, A>
+where
+    T: ?Sized,
+    A: Allocator,
+{
+    fn drop(&mut self) {
+        let size = core::mem::size_of_val::<T>(self);
+
+        // SAFETY: We need to drop `self.0` in place, before we free the backing memory.
+        unsafe { core::ptr::drop_in_place(self.0.as_ptr()) };
+
+        if size != 0 {
+            // SAFETY: `ptr` was previously allocated with `A`.
+            unsafe { A::free(self.0.cast()) };
+        }
+    }
+}
diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
index 4571daec0961..a9210634a8c3 100644
--- a/rust/kernel/prelude.rs
+++ b/rust/kernel/prelude.rs
@@ -14,7 +14,7 @@
 #[doc(no_inline)]
 pub use core::pin::Pin;
 
-pub use crate::alloc::{box_ext::BoxExt, flags::*, vec_ext::VecExt};
+pub use crate::alloc::{box_ext::BoxExt, flags::*, vec_ext::VecExt, KBox, KVBox, VBox};
 
 #[doc(no_inline)]
 pub use alloc::{boxed::Box, vec::Vec};
-- 
2.46.0

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Alice Ryhl]

On Fri, Aug 16, 2024 at 2:13 AM Danilo Krummrich <dakr@kernel.org> wrote:
>
> `Box` provides the simplest way to allocate memory for a generic type
> with one of the kernel's allocators, e.g. `Kmalloc`, `Vmalloc` or
> `KVmalloc`.
>
> In contrast to Rust's `Box` type, the kernel `Box` type considers the
> kernel's GFP flags for all appropriate functions, always reports
> allocation failures through `Result<_, AllocError>` and remains
> independent from unstable features.
>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

Overall looks good to me, but I have a question:

> +impl<T: 'static, A> ForeignOwnable for Box<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Borrowed<'a> = &'a T;
> +    type BorrowedMut<'a> = &'a mut T;
> [..]
> +    unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> &'a mut T {
> +        // SAFETY: The safety requirements of this method ensure that the pointer is valid and that
> +        // nothing else will access the value for the duration of 'a.
> +        unsafe { &mut *ptr.cast_mut().cast() }
> +    }

Where does this come from? It looks like you've based the series on
top of [1], but I dropped that patch a long time ago, and I don't see
it in rust-dev anymore.

Alice

[1]: https://lore.kernel.org/all/20230710074642.683831-1-aliceryhl@google.com/

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Danilo Krummrich]

On Tue, Aug 20, 2024 at 11:47:45AM +0200, Alice Ryhl wrote:
> On Fri, Aug 16, 2024 at 2:13 AM Danilo Krummrich <dakr@kernel.org> wrote:
> >
> > `Box` provides the simplest way to allocate memory for a generic type
> > with one of the kernel's allocators, e.g. `Kmalloc`, `Vmalloc` or
> > `KVmalloc`.
> >
> > In contrast to Rust's `Box` type, the kernel `Box` type considers the
> > kernel's GFP flags for all appropriate functions, always reports
> > allocation failures through `Result<_, AllocError>` and remains
> > independent from unstable features.
> >
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> 
> Overall looks good to me, but I have a question:
> 
> > +impl<T: 'static, A> ForeignOwnable for Box<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    type Borrowed<'a> = &'a T;
> > +    type BorrowedMut<'a> = &'a mut T;
> > [..]
> > +    unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> &'a mut T {
> > +        // SAFETY: The safety requirements of this method ensure that the pointer is valid and that
> > +        // nothing else will access the value for the duration of 'a.
> > +        unsafe { &mut *ptr.cast_mut().cast() }
> > +    }
> 
> Where does this come from? It looks like you've based the series on
> top of [1], but I dropped that patch a long time ago, and I don't see
> it in rust-dev anymore.

I comes from me rebasing onto rust-dev. When Boqun asked me to resolve the merge
conflicts a few days ago, this patch was in rust-dev. I think it disappeared two
days ago or so.

@Bonqun: Need to me to rebase again?

- Danilo

> 
> Alice
> 
> [1]: https://lore.kernel.org/all/20230710074642.683831-1-aliceryhl@google.com/
> 

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Boqun Feng]

On Tue, Aug 20, 2024 at 05:26:22PM +0200, Danilo Krummrich wrote:
> On Tue, Aug 20, 2024 at 11:47:45AM +0200, Alice Ryhl wrote:
> > On Fri, Aug 16, 2024 at 2:13 AM Danilo Krummrich <dakr@kernel.org> wrote:
> > >
> > > `Box` provides the simplest way to allocate memory for a generic type
> > > with one of the kernel's allocators, e.g. `Kmalloc`, `Vmalloc` or
> > > `KVmalloc`.
> > >
> > > In contrast to Rust's `Box` type, the kernel `Box` type considers the
> > > kernel's GFP flags for all appropriate functions, always reports
> > > allocation failures through `Result<_, AllocError>` and remains
> > > independent from unstable features.
> > >
> > > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > 
> > Overall looks good to me, but I have a question:
> > 
> > > +impl<T: 'static, A> ForeignOwnable for Box<T, A>
> > > +where
> > > +    A: Allocator,
> > > +{
> > > +    type Borrowed<'a> = &'a T;
> > > +    type BorrowedMut<'a> = &'a mut T;
> > > [..]
> > > +    unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> &'a mut T {
> > > +        // SAFETY: The safety requirements of this method ensure that the pointer is valid and that
> > > +        // nothing else will access the value for the duration of 'a.
> > > +        unsafe { &mut *ptr.cast_mut().cast() }
> > > +    }
> > 
> > Where does this come from? It looks like you've based the series on
> > top of [1], but I dropped that patch a long time ago, and I don't see
> > it in rust-dev anymore.
> 
> I comes from me rebasing onto rust-dev. When Boqun asked me to resolve the merge
> conflicts a few days ago, this patch was in rust-dev. I think it disappeared two
> days ago or so.
> 
> @Bonqun: Need to me to rebase again?
> 

No need to rebase for this. I've already applied your series on
rust-dev. The background of carrying `borrow_mut` is because it was
simple to do, and so I carried it as the best effort, now with more
`ForeignOwnable` ipmls, I think it's better if we just drop it, and
re-introduce if needed.

Sorry for a bit out-of-sync on this one.

Regards,
Boqun

> - Danilo
> 
> > 
> > Alice
> > 
> > [1]: https://lore.kernel.org/all/20230710074642.683831-1-aliceryhl@google.com/
> > 

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> diff --git a/rust/kernel/alloc/kbox.rs b/rust/kernel/alloc/kbox.rs
> new file mode 100644
> index 000000000000..93b1ab9de6e8
> --- /dev/null
> +++ b/rust/kernel/alloc/kbox.rs
> @@ -0,0 +1,480 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! Implementation of [`Box`].
> +
> +#[allow(unused_imports)] // Used in doc comments.
> +use super::allocator::{KVmalloc, Kmalloc, Vmalloc};
> +use super::{AllocError, Allocator, Flags};
> +use core::fmt;
> +use core::marker::PhantomData;
> +use core::mem::ManuallyDrop;
> +use core::mem::MaybeUninit;
> +use core::ops::{Deref, DerefMut};
> +use core::pin::Pin;
> +use core::ptr::NonNull;
> +use core::result::Result;
> +
> +use crate::init::{InPlaceInit, InPlaceWrite, Init, PinInit};
> +use crate::types::ForeignOwnable;
> +
> +/// The kernel's [`Box`] type - a heap allocation for a single value of type `T`.

A single `-` doesn't really render nicely in markdown, instead use a
double or triple dash (`--` or `---`).

> +///
> +/// This is the kernel's version of the Rust stdlib's `Box`. There are several of differences,
> +/// for example no `noalias` attribute is emitted and partially moving out of a `Box` is not
> +/// supported. There are also several API differences, e.g. `Box` always requires an [`Allocator`]
> +/// implementation to be passed as generic, page [`Flags`] when allocating memory and all functions
> +/// that may allocate memory are failable.

Do you mean fallible?

> +///
> +/// `Box` works with any of the kernel's allocators, e.g. [`Kmalloc`], [`Vmalloc`] or [`KVmalloc`].
> +/// There are aliases for `Box` with these allocators ([`KBox`], [`VBox`], [`KVBox`]).
> +///
> +/// When dropping a [`Box`], the value is also dropped and the heap memory is automatically freed.
> +///
> +/// # Examples
> +///
> +/// ```
> +/// let b = KBox::<u64>::new(24_u64, GFP_KERNEL)?;
> +///
> +/// assert_eq!(*b, 24_u64);
> +/// # Ok::<(), Error>(())
> +/// ```
> +///
> +/// ```
> +/// # use kernel::bindings;
> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
> +/// struct Huge([u8; SIZE]);
> +///
> +/// assert!(KBox::<Huge>::new_uninit(GFP_KERNEL | __GFP_NOWARN).is_err());
> +/// ```

It would be nice if you could add something like "KBox can't handle big
allocations:" above this example, so that people aren't confused why
this example expects an error.

> +///
> +/// ```
> +/// # use kernel::bindings;
> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
> +/// struct Huge([u8; SIZE]);
> +///
> +/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
> +/// ```

Similarly, you could then say above this one "Instead use either `VBox`
or `KVBox`:"

> +///
> +/// # Invariants
> +///
> +/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`

Please use `self.0` instead of "[`Box`]'".

> +/// or, for zero-sized types, is a dangling pointer.

Probably "dangling, well aligned pointer.".

> +#[repr(transparent)]
> +pub struct Box<T: ?Sized, A: Allocator>(NonNull<T>, PhantomData<A>);
> +
> +/// Type alias for `Box` with a [`Kmalloc`] allocator.

Can you make these `Box` references links?

> +///
> +/// # Examples
> +///
> +/// ```
> +/// let b = KBox::new(24_u64, GFP_KERNEL)?;
> +///
> +/// assert_eq!(*b, 24_u64);
> +/// # Ok::<(), Error>(())
> +/// ```
> +pub type KBox<T> = Box<T, super::allocator::Kmalloc>;
> +
> +/// Type alias for `Box` with a [`Vmalloc`] allocator.
> +///
> +/// # Examples
> +///
> +/// ```
> +/// let b = VBox::new(24_u64, GFP_KERNEL)?;
> +///
> +/// assert_eq!(*b, 24_u64);
> +/// # Ok::<(), Error>(())
> +/// ```
> +pub type VBox<T> = Box<T, super::allocator::Vmalloc>;
> +
> +/// Type alias for `Box` with a [`KVmalloc`] allocator.
> +///
> +/// # Examples
> +///
> +/// ```
> +/// let b = KVBox::new(24_u64, GFP_KERNEL)?;
> +///
> +/// assert_eq!(*b, 24_u64);
> +/// # Ok::<(), Error>(())
> +/// ```
> +pub type KVBox<T> = Box<T, super::allocator::KVmalloc>;
> +
> +// SAFETY: `Box` is `Send` if `T` is `Send` because the `Box` owns a `T`.
> +unsafe impl<T, A> Send for Box<T, A>
> +where
> +    T: Send + ?Sized,
> +    A: Allocator,
> +{
> +}
> +
> +// SAFETY: `Box` is `Sync` if `T` is `Sync` because the `Box` owns a `T`.
> +unsafe impl<T, A> Sync for Box<T, A>
> +where
> +    T: Sync + ?Sized,
> +    A: Allocator,
> +{
> +}
> +
> +impl<T, A> Box<T, A>
> +where
> +    T: ?Sized,
> +    A: Allocator,
> +{
> +    /// Creates a new `Box<T, A>` from a raw pointer.
> +    ///
> +    /// # Safety
> +    ///
> +    /// For non-ZSTs, `raw` must point at an allocation allocated with `A`that is sufficiently
> +    /// aligned for and holds a valid `T`. The caller passes ownership of the allocation to the
> +    /// `Box`.

You don't say what must happen for ZSTs.

> +    #[inline]
> +    pub const unsafe fn from_raw(raw: *mut T) -> Self {
> +        // INVARIANT: Validity of `raw` is guaranteed by the safety preconditions of this function.
> +        // SAFETY: By the safety preconditions of this function, `raw` is not a NULL pointer.
> +        Self(unsafe { NonNull::new_unchecked(raw) }, PhantomData::<A>)
> +    }
> +
> +    /// Consumes the `Box<T, A>` and returns a raw pointer.
> +    ///
> +    /// This will not run the destructor of `T` and for non-ZSTs the allocation will stay alive
> +    /// indefinitely. Use [`Box::from_raw`] to recover the [`Box`], drop the value and free the
> +    /// allocation, if any.
> +    ///
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let x = KBox::new(24, GFP_KERNEL)?;
> +    /// let ptr = KBox::into_raw(x);
> +    /// let x = unsafe { KBox::from_raw(ptr) };
> +    ///
> +    /// assert_eq!(*x, 24);
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    #[inline]
> +    pub fn into_raw(b: Self) -> *mut T {
> +        let b = ManuallyDrop::new(b);
> +
> +        b.0.as_ptr()
> +    }
> +
> +    /// Consumes and leaks the `Box<T, A>` and returns a mutable reference.
> +    ///
> +    /// See [Box::into_raw] for more details.
> +    #[inline]
> +    pub fn leak<'a>(b: Self) -> &'a mut T {
> +        // SAFETY: `Box::into_raw` always returns a properly aligned and dereferenceable pointer
> +        // which points to an initialized instance of `T`.
> +        unsafe { &mut *Box::into_raw(b) }
> +    }
> +}
> +
> +impl<T, A> Box<MaybeUninit<T>, A>
> +where
> +    A: Allocator,
> +{
> +    /// Converts a `Box<MaybeUninit<T>, A>` to a `Box<T, A>`.
> +    ///
> +    /// It is undefined behavior to call this function while the value inside of `b` is not yet
> +    /// fully initialized.
> +    ///
> +    /// # Safety
> +    ///
> +    /// Callers must ensure that the value inside of `b` is in an initialized state.
> +    pub unsafe fn assume_init(b: Self) -> Box<T, A> {
> +        let raw = Self::into_raw(b);
> +
> +        // SAFETY: `raw` comes from a previous call to `Box::into_raw`. By the safety requirements
> +        // of this function, the value inside the `Box` is in an initialized state. Hence, it is
> +        // safe to reconstruct the `Box` as `Box<T, A>`.
> +        unsafe { Box::from_raw(raw as *mut T) }

You should be able to use `.cast()` instead.

> +    }
> +
> +    /// Writes the value and converts to `Box<T, A>`.
> +    pub fn write(mut b: Self, value: T) -> Box<T, A> {
> +        (*b).write(value);
> +        // SAFETY: We've just initialized `boxed`'s value.

The variable is called `b`.

> +        unsafe { Self::assume_init(b) }
> +    }
> +}

[...]

> +impl<T, A> Drop for Box<T, A>
> +where
> +    T: ?Sized,
> +    A: Allocator,
> +{
> +    fn drop(&mut self) {
> +        let size = core::mem::size_of_val::<T>(self);
> +
> +        // SAFETY: We need to drop `self.0` in place, before we free the backing memory.

This is the reason you are calling this function, not the justification
why it is OK to do so. (the pointer is valid)

> +        unsafe { core::ptr::drop_in_place(self.0.as_ptr()) };

Instead of using the raw pointer directly, you can also just use
`deref_mut`.

> +
> +        if size != 0 {
> +            // SAFETY: `ptr` was previously allocated with `A`.

There is no variable `ptr`, this is guaranteed by the type invariant of
`Self`.

---
Cheers,
Benno

> +            unsafe { A::free(self.0.cast()) };
> +        }
> +    }
> +}
> diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
> index 4571daec0961..a9210634a8c3 100644
> --- a/rust/kernel/prelude.rs
> +++ b/rust/kernel/prelude.rs
> @@ -14,7 +14,7 @@
>  #[doc(no_inline)]
>  pub use core::pin::Pin;
> 
> -pub use crate::alloc::{box_ext::BoxExt, flags::*, vec_ext::VecExt};
> +pub use crate::alloc::{box_ext::BoxExt, flags::*, vec_ext::VecExt, KBox, KVBox, VBox};
> 
>  #[doc(no_inline)]
>  pub use alloc::{boxed::Box, vec::Vec};
> --
> 2.46.0
> 

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Danilo Krummrich]

On Sat, Aug 31, 2024 at 05:39:07AM +0000, Benno Lossin wrote:
> On 16.08.24 02:10, Danilo Krummrich wrote:
> > diff --git a/rust/kernel/alloc/kbox.rs b/rust/kernel/alloc/kbox.rs
> > new file mode 100644
> > index 000000000000..93b1ab9de6e8
> > --- /dev/null
> > +++ b/rust/kernel/alloc/kbox.rs
> > @@ -0,0 +1,480 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +//! Implementation of [`Box`].
> > +
> > +#[allow(unused_imports)] // Used in doc comments.
> > +use super::allocator::{KVmalloc, Kmalloc, Vmalloc};
> > +use super::{AllocError, Allocator, Flags};
> > +use core::fmt;
> > +use core::marker::PhantomData;
> > +use core::mem::ManuallyDrop;
> > +use core::mem::MaybeUninit;
> > +use core::ops::{Deref, DerefMut};
> > +use core::pin::Pin;
> > +use core::ptr::NonNull;
> > +use core::result::Result;
> > +
> > +use crate::init::{InPlaceInit, InPlaceWrite, Init, PinInit};
> > +use crate::types::ForeignOwnable;
> > +
> > +/// The kernel's [`Box`] type - a heap allocation for a single value of type `T`.
> 
> A single `-` doesn't really render nicely in markdown, instead use a
> double or triple dash (`--` or `---`).
> 
> > +///
> > +/// This is the kernel's version of the Rust stdlib's `Box`. There are several of differences,
> > +/// for example no `noalias` attribute is emitted and partially moving out of a `Box` is not
> > +/// supported. There are also several API differences, e.g. `Box` always requires an [`Allocator`]
> > +/// implementation to be passed as generic, page [`Flags`] when allocating memory and all functions
> > +/// that may allocate memory are failable.
> 
> Do you mean fallible?
> 
> > +///
> > +/// `Box` works with any of the kernel's allocators, e.g. [`Kmalloc`], [`Vmalloc`] or [`KVmalloc`].
> > +/// There are aliases for `Box` with these allocators ([`KBox`], [`VBox`], [`KVBox`]).
> > +///
> > +/// When dropping a [`Box`], the value is also dropped and the heap memory is automatically freed.
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// let b = KBox::<u64>::new(24_u64, GFP_KERNEL)?;
> > +///
> > +/// assert_eq!(*b, 24_u64);
> > +/// # Ok::<(), Error>(())
> > +/// ```
> > +///
> > +/// ```
> > +/// # use kernel::bindings;
> > +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
> > +/// struct Huge([u8; SIZE]);
> > +///
> > +/// assert!(KBox::<Huge>::new_uninit(GFP_KERNEL | __GFP_NOWARN).is_err());
> > +/// ```
> 
> It would be nice if you could add something like "KBox can't handle big
> allocations:" above this example, so that people aren't confused why
> this example expects an error.

I don't think that's needed, it's implied by
`SIZE == bindings::KMALLOC_MAX_SIZE + 1`.

Surely, we could add it nevertheless, but it's not very precise to just say "big
allocations". And I think this isn't the place for lengthy explanations of
`Kmalloc` behavior.

> 
> > +///
> > +/// ```
> > +/// # use kernel::bindings;
> > +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
> > +/// struct Huge([u8; SIZE]);
> > +///
> > +/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
> > +/// ```
> 
> Similarly, you could then say above this one "Instead use either `VBox`
> or `KVBox`:"
> 
> > +///
> > +/// # Invariants
> > +///
> > +/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`
> 
> Please use `self.0` instead of "[`Box`]'".
> 
> > +/// or, for zero-sized types, is a dangling pointer.
> 
> Probably "dangling, well aligned pointer.".

Does this add any value? For ZSTs everything is "well aligned", isn't it?

> 
> > +#[repr(transparent)]
> > +pub struct Box<T: ?Sized, A: Allocator>(NonNull<T>, PhantomData<A>);
> > +
> > +/// Type alias for `Box` with a [`Kmalloc`] allocator.
> 
> Can you make these `Box` references links?
> 
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// let b = KBox::new(24_u64, GFP_KERNEL)?;
> > +///
> > +/// assert_eq!(*b, 24_u64);
> > +/// # Ok::<(), Error>(())
> > +/// ```
> > +pub type KBox<T> = Box<T, super::allocator::Kmalloc>;
> > +
> > +/// Type alias for `Box` with a [`Vmalloc`] allocator.
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// let b = VBox::new(24_u64, GFP_KERNEL)?;
> > +///
> > +/// assert_eq!(*b, 24_u64);
> > +/// # Ok::<(), Error>(())
> > +/// ```
> > +pub type VBox<T> = Box<T, super::allocator::Vmalloc>;
> > +
> > +/// Type alias for `Box` with a [`KVmalloc`] allocator.
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// let b = KVBox::new(24_u64, GFP_KERNEL)?;
> > +///
> > +/// assert_eq!(*b, 24_u64);
> > +/// # Ok::<(), Error>(())
> > +/// ```
> > +pub type KVBox<T> = Box<T, super::allocator::KVmalloc>;
> > +
> > +// SAFETY: `Box` is `Send` if `T` is `Send` because the `Box` owns a `T`.
> > +unsafe impl<T, A> Send for Box<T, A>
> > +where
> > +    T: Send + ?Sized,
> > +    A: Allocator,
> > +{
> > +}
> > +
> > +// SAFETY: `Box` is `Sync` if `T` is `Sync` because the `Box` owns a `T`.
> > +unsafe impl<T, A> Sync for Box<T, A>
> > +where
> > +    T: Sync + ?Sized,
> > +    A: Allocator,
> > +{
> > +}
> > +
> > +impl<T, A> Box<T, A>
> > +where
> > +    T: ?Sized,
> > +    A: Allocator,
> > +{
> > +    /// Creates a new `Box<T, A>` from a raw pointer.
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// For non-ZSTs, `raw` must point at an allocation allocated with `A`that is sufficiently
> > +    /// aligned for and holds a valid `T`. The caller passes ownership of the allocation to the
> > +    /// `Box`.
> 
> You don't say what must happen for ZSTs.

Because we don't require anything for a ZST, do we?

> 
> > +    #[inline]
> > +    pub const unsafe fn from_raw(raw: *mut T) -> Self {
> > +        // INVARIANT: Validity of `raw` is guaranteed by the safety preconditions of this function.
> > +        // SAFETY: By the safety preconditions of this function, `raw` is not a NULL pointer.
> > +        Self(unsafe { NonNull::new_unchecked(raw) }, PhantomData::<A>)
> > +    }
> > +
> > +    /// Consumes the `Box<T, A>` and returns a raw pointer.
> > +    ///
> > +    /// This will not run the destructor of `T` and for non-ZSTs the allocation will stay alive
> > +    /// indefinitely. Use [`Box::from_raw`] to recover the [`Box`], drop the value and free the
> > +    /// allocation, if any.
> > +    ///
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let x = KBox::new(24, GFP_KERNEL)?;
> > +    /// let ptr = KBox::into_raw(x);
> > +    /// let x = unsafe { KBox::from_raw(ptr) };
> > +    ///
> > +    /// assert_eq!(*x, 24);
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> > +    #[inline]
> > +    pub fn into_raw(b: Self) -> *mut T {
> > +        let b = ManuallyDrop::new(b);
> > +
> > +        b.0.as_ptr()
> > +    }
> > +
> > +    /// Consumes and leaks the `Box<T, A>` and returns a mutable reference.
> > +    ///
> > +    /// See [Box::into_raw] for more details.
> > +    #[inline]
> > +    pub fn leak<'a>(b: Self) -> &'a mut T {
> > +        // SAFETY: `Box::into_raw` always returns a properly aligned and dereferenceable pointer
> > +        // which points to an initialized instance of `T`.
> > +        unsafe { &mut *Box::into_raw(b) }
> > +    }
> > +}
> > +
> > +impl<T, A> Box<MaybeUninit<T>, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    /// Converts a `Box<MaybeUninit<T>, A>` to a `Box<T, A>`.
> > +    ///
> > +    /// It is undefined behavior to call this function while the value inside of `b` is not yet
> > +    /// fully initialized.
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// Callers must ensure that the value inside of `b` is in an initialized state.
> > +    pub unsafe fn assume_init(b: Self) -> Box<T, A> {
> > +        let raw = Self::into_raw(b);
> > +
> > +        // SAFETY: `raw` comes from a previous call to `Box::into_raw`. By the safety requirements
> > +        // of this function, the value inside the `Box` is in an initialized state. Hence, it is
> > +        // safe to reconstruct the `Box` as `Box<T, A>`.
> > +        unsafe { Box::from_raw(raw as *mut T) }
> 
> You should be able to use `.cast()` instead.
> 
> > +    }
> > +
> > +    /// Writes the value and converts to `Box<T, A>`.
> > +    pub fn write(mut b: Self, value: T) -> Box<T, A> {
> > +        (*b).write(value);
> > +        // SAFETY: We've just initialized `boxed`'s value.
> 
> The variable is called `b`.
> 
> > +        unsafe { Self::assume_init(b) }
> > +    }
> > +}
> 
> [...]
> 
> > +impl<T, A> Drop for Box<T, A>
> > +where
> > +    T: ?Sized,
> > +    A: Allocator,
> > +{
> > +    fn drop(&mut self) {
> > +        let size = core::mem::size_of_val::<T>(self);
> > +
> > +        // SAFETY: We need to drop `self.0` in place, before we free the backing memory.
> 
> This is the reason you are calling this function, not the justification
> why it is OK to do so. (the pointer is valid)
> 
> > +        unsafe { core::ptr::drop_in_place(self.0.as_ptr()) };
> 
> Instead of using the raw pointer directly, you can also just use
> `deref_mut`.
> 
> > +
> > +        if size != 0 {
> > +            // SAFETY: `ptr` was previously allocated with `A`.
> 
> There is no variable `ptr`, this is guaranteed by the type invariant of
> `Self`.
> 
> ---
> Cheers,
> Benno
> 
> > +            unsafe { A::free(self.0.cast()) };
> > +        }
> > +    }
> > +}
> > diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
> > index 4571daec0961..a9210634a8c3 100644
> > --- a/rust/kernel/prelude.rs
> > +++ b/rust/kernel/prelude.rs
> > @@ -14,7 +14,7 @@
> >  #[doc(no_inline)]
> >  pub use core::pin::Pin;
> > 
> > -pub use crate::alloc::{box_ext::BoxExt, flags::*, vec_ext::VecExt};
> > +pub use crate::alloc::{box_ext::BoxExt, flags::*, vec_ext::VecExt, KBox, KVBox, VBox};
> > 
> >  #[doc(no_inline)]
> >  pub use alloc::{boxed::Box, vec::Vec};
> > --
> > 2.46.0
> > 
> 

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Benno Lossin]

On 10.09.24 19:40, Danilo Krummrich wrote:
> On Sat, Aug 31, 2024 at 05:39:07AM +0000, Benno Lossin wrote:
>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>> +/// # Examples
>>> +///
>>> +/// ```
>>> +/// let b = KBox::<u64>::new(24_u64, GFP_KERNEL)?;
>>> +///
>>> +/// assert_eq!(*b, 24_u64);
>>> +/// # Ok::<(), Error>(())
>>> +/// ```
>>> +///
>>> +/// ```
>>> +/// # use kernel::bindings;
>>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
>>> +/// struct Huge([u8; SIZE]);
>>> +///
>>> +/// assert!(KBox::<Huge>::new_uninit(GFP_KERNEL | __GFP_NOWARN).is_err());
>>> +/// ```
>>
>> It would be nice if you could add something like "KBox can't handle big
>> allocations:" above this example, so that people aren't confused why
>> this example expects an error.
> 
> I don't think that's needed, it's implied by
> `SIZE == bindings::KMALLOC_MAX_SIZE + 1`.
> 
> Surely, we could add it nevertheless, but it's not very precise to just say "big
> allocations". And I think this isn't the place for lengthy explanations of
> `Kmalloc` behavior.

Fair point, nevertheless I find examples a bit more useful, when the
intention behind them is not only given as code.

>>> +///
>>> +/// ```
>>> +/// # use kernel::bindings;
>>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
>>> +/// struct Huge([u8; SIZE]);
>>> +///
>>> +/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
>>> +/// ```
>>
>> Similarly, you could then say above this one "Instead use either `VBox`
>> or `KVBox`:"
>>
>>> +///
>>> +/// # Invariants
>>> +///
>>> +/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`
>>
>> Please use `self.0` instead of "[`Box`]'".
>>
>>> +/// or, for zero-sized types, is a dangling pointer.
>>
>> Probably "dangling, well aligned pointer.".
> 
> Does this add any value? For ZSTs everything is "well aligned", isn't it?

ZSTs can have alignment and then unaligned pointers do exist for them
(and dereferencing them is UB!):

    #[repr(align(64))]
    struct Token;

    fn main() {
        let t = 64 as *mut Token;
        let t = unsafe { t.read() }; // this is fine.
        let t = 4 as *mut Token;
        let t = unsafe { t.read() }; // this is UB, see below for miri's output
    }

Miri complains:

    error: Undefined Behavior: accessing memory based on pointer with alignment 4, but alignment 64 is required
     --> src/main.rs:8:22
      |
    8 |     let t = unsafe { t.read() }; // this is UB, see below for miri's output
      |                      ^^^^^^^^ accessing memory based on pointer with alignment 4, but alignment 64 is required
      |
      = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
      = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
      = note: BACKTRACE:
      = note: inside `main` at src/main.rs:8:22: 8:30

>>> +#[repr(transparent)]
>>> +pub struct Box<T: ?Sized, A: Allocator>(NonNull<T>, PhantomData<A>);


>>> +impl<T, A> Box<T, A>
>>> +where
>>> +    T: ?Sized,
>>> +    A: Allocator,
>>> +{
>>> +    /// Creates a new `Box<T, A>` from a raw pointer.
>>> +    ///
>>> +    /// # Safety
>>> +    ///
>>> +    /// For non-ZSTs, `raw` must point at an allocation allocated with `A`that is sufficiently
>>> +    /// aligned for and holds a valid `T`. The caller passes ownership of the allocation to the
>>> +    /// `Box`.
>>
>> You don't say what must happen for ZSTs.
> 
> Because we don't require anything for a ZST, do we?

We require a non-null, well aligned pointer (see above).

---
Cheers,
Benno

>>> +    #[inline]
>>> +    pub const unsafe fn from_raw(raw: *mut T) -> Self {
>>> +        // INVARIANT: Validity of `raw` is guaranteed by the safety preconditions of this function.
>>> +        // SAFETY: By the safety preconditions of this function, `raw` is not a NULL pointer.
>>> +        Self(unsafe { NonNull::new_unchecked(raw) }, PhantomData::<A>)
>>> +    }

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Danilo Krummrich]

On Tue, Sep 10, 2024 at 07:49:42PM +0000, Benno Lossin wrote:
> On 10.09.24 19:40, Danilo Krummrich wrote:
> > On Sat, Aug 31, 2024 at 05:39:07AM +0000, Benno Lossin wrote:
> >> On 16.08.24 02:10, Danilo Krummrich wrote:
> >>> +/// # Examples
> >>> +///
> >>> +/// ```
> >>> +/// let b = KBox::<u64>::new(24_u64, GFP_KERNEL)?;
> >>> +///
> >>> +/// assert_eq!(*b, 24_u64);
> >>> +/// # Ok::<(), Error>(())
> >>> +/// ```
> >>> +///
> >>> +/// ```
> >>> +/// # use kernel::bindings;
> >>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
> >>> +/// struct Huge([u8; SIZE]);
> >>> +///
> >>> +/// assert!(KBox::<Huge>::new_uninit(GFP_KERNEL | __GFP_NOWARN).is_err());
> >>> +/// ```
> >>
> >> It would be nice if you could add something like "KBox can't handle big
> >> allocations:" above this example, so that people aren't confused why
> >> this example expects an error.
> > 
> > I don't think that's needed, it's implied by
> > `SIZE == bindings::KMALLOC_MAX_SIZE + 1`.
> > 
> > Surely, we could add it nevertheless, but it's not very precise to just say "big
> > allocations". And I think this isn't the place for lengthy explanations of
> > `Kmalloc` behavior.
> 
> Fair point, nevertheless I find examples a bit more useful, when the
> intention behind them is not only given as code.
> 
> >>> +///
> >>> +/// ```
> >>> +/// # use kernel::bindings;
> >>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
> >>> +/// struct Huge([u8; SIZE]);
> >>> +///
> >>> +/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
> >>> +/// ```
> >>
> >> Similarly, you could then say above this one "Instead use either `VBox`
> >> or `KVBox`:"
> >>
> >>> +///
> >>> +/// # Invariants
> >>> +///
> >>> +/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`
> >>
> >> Please use `self.0` instead of "[`Box`]'".
> >>
> >>> +/// or, for zero-sized types, is a dangling pointer.
> >>
> >> Probably "dangling, well aligned pointer.".
> > 
> > Does this add any value? For ZSTs everything is "well aligned", isn't it?
> 
> ZSTs can have alignment and then unaligned pointers do exist for them
> (and dereferencing them is UB!):

Where is this documented? The documentation says:

"For operations of size zero, *every* pointer is valid, including the null
pointer. The following points are only concerned with non-zero-sized accesses."
[1]

[1] https://doc.rust-lang.org/std/ptr/index.html

> 
>     #[repr(align(64))]
>     struct Token;
> 
>     fn main() {
>         let t = 64 as *mut Token;
>         let t = unsafe { t.read() }; // this is fine.
>         let t = 4 as *mut Token;
>         let t = unsafe { t.read() }; // this is UB, see below for miri's output
>     }
> 
> Miri complains:
> 
>     error: Undefined Behavior: accessing memory based on pointer with alignment 4, but alignment 64 is required
>      --> src/main.rs:8:22
>       |
>     8 |     let t = unsafe { t.read() }; // this is UB, see below for miri's output
>       |                      ^^^^^^^^ accessing memory based on pointer with alignment 4, but alignment 64 is required
>       |
>       = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
>       = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
>       = note: BACKTRACE:
>       = note: inside `main` at src/main.rs:8:22: 8:30

`read` explicitly asks for non-null and properly aligned even if `T` has size
zero.

> 
> >>> +#[repr(transparent)]
> >>> +pub struct Box<T: ?Sized, A: Allocator>(NonNull<T>, PhantomData<A>);
> 
> 
> >>> +impl<T, A> Box<T, A>
> >>> +where
> >>> +    T: ?Sized,
> >>> +    A: Allocator,
> >>> +{
> >>> +    /// Creates a new `Box<T, A>` from a raw pointer.
> >>> +    ///
> >>> +    /// # Safety
> >>> +    ///
> >>> +    /// For non-ZSTs, `raw` must point at an allocation allocated with `A`that is sufficiently
> >>> +    /// aligned for and holds a valid `T`. The caller passes ownership of the allocation to the
> >>> +    /// `Box`.
> >>
> >> You don't say what must happen for ZSTs.
> > 
> > Because we don't require anything for a ZST, do we?
> 
> We require a non-null, well aligned pointer (see above).

We indeed do, gonna add it.

> 
> ---
> Cheers,
> Benno
> 
> >>> +    #[inline]
> >>> +    pub const unsafe fn from_raw(raw: *mut T) -> Self {
> >>> +        // INVARIANT: Validity of `raw` is guaranteed by the safety preconditions of this function.
> >>> +        // SAFETY: By the safety preconditions of this function, `raw` is not a NULL pointer.
> >>> +        Self(unsafe { NonNull::new_unchecked(raw) }, PhantomData::<A>)
> >>> +    }
> 

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Benno Lossin]

On 11.09.24 01:25, Danilo Krummrich wrote:
> On Tue, Sep 10, 2024 at 07:49:42PM +0000, Benno Lossin wrote:
>> On 10.09.24 19:40, Danilo Krummrich wrote:
>>> On Sat, Aug 31, 2024 at 05:39:07AM +0000, Benno Lossin wrote:
>>>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>>>> +/// # Examples
>>>>> +///
>>>>> +/// ```
>>>>> +/// let b = KBox::<u64>::new(24_u64, GFP_KERNEL)?;
>>>>> +///
>>>>> +/// assert_eq!(*b, 24_u64);
>>>>> +/// # Ok::<(), Error>(())
>>>>> +/// ```
>>>>> +///
>>>>> +/// ```
>>>>> +/// # use kernel::bindings;
>>>>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
>>>>> +/// struct Huge([u8; SIZE]);
>>>>> +///
>>>>> +/// assert!(KBox::<Huge>::new_uninit(GFP_KERNEL | __GFP_NOWARN).is_err());
>>>>> +/// ```
>>>>
>>>> It would be nice if you could add something like "KBox can't handle big
>>>> allocations:" above this example, so that people aren't confused why
>>>> this example expects an error.
>>>
>>> I don't think that's needed, it's implied by
>>> `SIZE == bindings::KMALLOC_MAX_SIZE + 1`.
>>>
>>> Surely, we could add it nevertheless, but it's not very precise to just say "big
>>> allocations". And I think this isn't the place for lengthy explanations of
>>> `Kmalloc` behavior.
>>
>> Fair point, nevertheless I find examples a bit more useful, when the
>> intention behind them is not only given as code.
>>
>>>>> +///
>>>>> +/// ```
>>>>> +/// # use kernel::bindings;
>>>>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
>>>>> +/// struct Huge([u8; SIZE]);
>>>>> +///
>>>>> +/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
>>>>> +/// ```
>>>>
>>>> Similarly, you could then say above this one "Instead use either `VBox`
>>>> or `KVBox`:"
>>>>
>>>>> +///
>>>>> +/// # Invariants
>>>>> +///
>>>>> +/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`
>>>>
>>>> Please use `self.0` instead of "[`Box`]'".
>>>>
>>>>> +/// or, for zero-sized types, is a dangling pointer.
>>>>
>>>> Probably "dangling, well aligned pointer.".
>>>
>>> Does this add any value? For ZSTs everything is "well aligned", isn't it?
>>
>> ZSTs can have alignment and then unaligned pointers do exist for them
>> (and dereferencing them is UB!):
> 
> Where is this documented? The documentation says:
> 
> "For operations of size zero, *every* pointer is valid, including the null
> pointer. The following points are only concerned with non-zero-sized accesses."
> [1]

That's a good point, the documentation looks a bit outdated. I found
this page in the nomicon: https://doc.rust-lang.org/nomicon/vec/vec-zsts.html
The first iterator implementation has an alignment issue. (Nevertheless,
that chapter of the nomicon is probably useful to you, since it goes
over implementing `Vec`, but maybe you already saw it)

> [1] https://doc.rust-lang.org/std/ptr/index.html

Might be a good idea to improve/complain about this at the rust project.

>>     #[repr(align(64))]
>>     struct Token;
>>
>>     fn main() {
>>         let t = 64 as *mut Token;
>>         let t = unsafe { t.read() }; // this is fine.
>>         let t = 4 as *mut Token;
>>         let t = unsafe { t.read() }; // this is UB, see below for miri's output
>>     }
>>
>> Miri complains:
>>
>>     error: Undefined Behavior: accessing memory based on pointer with alignment 4, but alignment 64 is required
>>      --> src/main.rs:8:22
>>       |
>>     8 |     let t = unsafe { t.read() }; // this is UB, see below for miri's output
>>       |                      ^^^^^^^^ accessing memory based on pointer with alignment 4, but alignment 64 is required
>>       |
>>       = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
>>       = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
>>       = note: BACKTRACE:
>>       = note: inside `main` at src/main.rs:8:22: 8:30
> 
> `read` explicitly asks for non-null and properly aligned even if `T` has size
> zero.

Dereferencing (ie `*t`) also requires that (I just didn't do it, since
then the `Token` must implement `Copy`).

---
Cheers,
Benno

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Danilo Krummrich]

On Wed, Sep 11, 2024 at 08:36:38AM +0000, Benno Lossin wrote:
> On 11.09.24 01:25, Danilo Krummrich wrote:
> > On Tue, Sep 10, 2024 at 07:49:42PM +0000, Benno Lossin wrote:
> >> On 10.09.24 19:40, Danilo Krummrich wrote:
> >>> On Sat, Aug 31, 2024 at 05:39:07AM +0000, Benno Lossin wrote:
> >>>> On 16.08.24 02:10, Danilo Krummrich wrote:
> >>>>> +/// # Examples
> >>>>> +///
> >>>>> +/// ```
> >>>>> +/// let b = KBox::<u64>::new(24_u64, GFP_KERNEL)?;
> >>>>> +///
> >>>>> +/// assert_eq!(*b, 24_u64);
> >>>>> +/// # Ok::<(), Error>(())
> >>>>> +/// ```
> >>>>> +///
> >>>>> +/// ```
> >>>>> +/// # use kernel::bindings;
> >>>>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
> >>>>> +/// struct Huge([u8; SIZE]);
> >>>>> +///
> >>>>> +/// assert!(KBox::<Huge>::new_uninit(GFP_KERNEL | __GFP_NOWARN).is_err());
> >>>>> +/// ```
> >>>>
> >>>> It would be nice if you could add something like "KBox can't handle big
> >>>> allocations:" above this example, so that people aren't confused why
> >>>> this example expects an error.
> >>>
> >>> I don't think that's needed, it's implied by
> >>> `SIZE == bindings::KMALLOC_MAX_SIZE + 1`.
> >>>
> >>> Surely, we could add it nevertheless, but it's not very precise to just say "big
> >>> allocations". And I think this isn't the place for lengthy explanations of
> >>> `Kmalloc` behavior.
> >>
> >> Fair point, nevertheless I find examples a bit more useful, when the
> >> intention behind them is not only given as code.
> >>
> >>>>> +///
> >>>>> +/// ```
> >>>>> +/// # use kernel::bindings;
> >>>>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
> >>>>> +/// struct Huge([u8; SIZE]);
> >>>>> +///
> >>>>> +/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
> >>>>> +/// ```
> >>>>
> >>>> Similarly, you could then say above this one "Instead use either `VBox`
> >>>> or `KVBox`:"
> >>>>
> >>>>> +///
> >>>>> +/// # Invariants
> >>>>> +///
> >>>>> +/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`
> >>>>
> >>>> Please use `self.0` instead of "[`Box`]'".
> >>>>
> >>>>> +/// or, for zero-sized types, is a dangling pointer.
> >>>>
> >>>> Probably "dangling, well aligned pointer.".
> >>>
> >>> Does this add any value? For ZSTs everything is "well aligned", isn't it?
> >>
> >> ZSTs can have alignment and then unaligned pointers do exist for them
> >> (and dereferencing them is UB!):
> > 
> > Where is this documented? The documentation says:
> > 
> > "For operations of size zero, *every* pointer is valid, including the null
> > pointer. The following points are only concerned with non-zero-sized accesses."
> > [1]
> 
> That's a good point, the documentation looks a bit outdated. I found
> this page in the nomicon: https://doc.rust-lang.org/nomicon/vec/vec-zsts.html
> The first iterator implementation has an alignment issue. (Nevertheless,
> that chapter of the nomicon is probably useful to you, since it goes
> over implementing `Vec`, but maybe you already saw it)
> 
> > [1] https://doc.rust-lang.org/std/ptr/index.html
> 
> Might be a good idea to improve/complain about this at the rust project.

Well, my point is how do we know? There's no language specification and the
documentation is (at least) ambiguous.

> 
> >>     #[repr(align(64))]
> >>     struct Token;
> >>
> >>     fn main() {
> >>         let t = 64 as *mut Token;
> >>         let t = unsafe { t.read() }; // this is fine.
> >>         let t = 4 as *mut Token;
> >>         let t = unsafe { t.read() }; // this is UB, see below for miri's output
> >>     }
> >>
> >> Miri complains:
> >>
> >>     error: Undefined Behavior: accessing memory based on pointer with alignment 4, but alignment 64 is required
> >>      --> src/main.rs:8:22
> >>       |
> >>     8 |     let t = unsafe { t.read() }; // this is UB, see below for miri's output
> >>       |                      ^^^^^^^^ accessing memory based on pointer with alignment 4, but alignment 64 is required
> >>       |
> >>       = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
> >>       = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
> >>       = note: BACKTRACE:
> >>       = note: inside `main` at src/main.rs:8:22: 8:30
> > 
> > `read` explicitly asks for non-null and properly aligned even if `T` has size
> > zero.

I mentioned this because for `read` it's explicitly documented.

However, the nomicon also says "This is possibly needless pedantry because
ptr::read is a noop for a ZST, [...]".

> 
> Dereferencing (ie `*t`) also requires that (I just didn't do it, since
> then the `Token` must implement `Copy`).

Again, how do you know? The documentation isn't clear about it.

> 
> ---
> Cheers,
> Benno
> 

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Benno Lossin]

On 11.09.24 13:02, Danilo Krummrich wrote:
> On Wed, Sep 11, 2024 at 08:36:38AM +0000, Benno Lossin wrote:
>> On 11.09.24 01:25, Danilo Krummrich wrote:
>>> On Tue, Sep 10, 2024 at 07:49:42PM +0000, Benno Lossin wrote:
>>>> On 10.09.24 19:40, Danilo Krummrich wrote:
>>>>> On Sat, Aug 31, 2024 at 05:39:07AM +0000, Benno Lossin wrote:
>>>>>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>>>>>> +///
>>>>>>> +/// ```
>>>>>>> +/// # use kernel::bindings;
>>>>>>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
>>>>>>> +/// struct Huge([u8; SIZE]);
>>>>>>> +///
>>>>>>> +/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
>>>>>>> +/// ```
>>>>>>
>>>>>> Similarly, you could then say above this one "Instead use either `VBox`
>>>>>> or `KVBox`:"
>>>>>>
>>>>>>> +///
>>>>>>> +/// # Invariants
>>>>>>> +///
>>>>>>> +/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`
>>>>>>
>>>>>> Please use `self.0` instead of "[`Box`]'".
>>>>>>
>>>>>>> +/// or, for zero-sized types, is a dangling pointer.
>>>>>>
>>>>>> Probably "dangling, well aligned pointer.".
>>>>>
>>>>> Does this add any value? For ZSTs everything is "well aligned", isn't it?
>>>>
>>>> ZSTs can have alignment and then unaligned pointers do exist for them
>>>> (and dereferencing them is UB!):
>>>
>>> Where is this documented? The documentation says:
>>>
>>> "For operations of size zero, *every* pointer is valid, including the null
>>> pointer. The following points are only concerned with non-zero-sized accesses."
>>> [1]
>>
>> That's a good point, the documentation looks a bit outdated. I found
>> this page in the nomicon: https://doc.rust-lang.org/nomicon/vec/vec-zsts.html
>> The first iterator implementation has an alignment issue. (Nevertheless,
>> that chapter of the nomicon is probably useful to you, since it goes
>> over implementing `Vec`, but maybe you already saw it)
>>
>>> [1] https://doc.rust-lang.org/std/ptr/index.html
>>
>> Might be a good idea to improve/complain about this at the rust project.
> 
> Well, my point is how do we know? There's no language specification and the
> documentation is (at least) ambiguous.

So I went through the unsafe-coding-guidelines issues list and only
found this one: https://github.com/rust-lang/unsafe-code-guidelines/issues/93
Maybe I missed something. You could also try to ask at the rust zulip in
the t-opsem channel for further clarification.

I think we should just be on the safe side and assume that ZSTs require
alignment. But if you get a convincing answer and if they say that they
will document it, then I don't mind removing the alignment requirement.

---
Cheers,
Benno

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Alice Ryhl]

On Wed, Sep 11, 2024 at 3:26 PM Benno Lossin <benno.lossin@proton.me> wrote:
>
> On 11.09.24 13:02, Danilo Krummrich wrote:
> > On Wed, Sep 11, 2024 at 08:36:38AM +0000, Benno Lossin wrote:
> >> On 11.09.24 01:25, Danilo Krummrich wrote:
> >>> On Tue, Sep 10, 2024 at 07:49:42PM +0000, Benno Lossin wrote:
> >>>> On 10.09.24 19:40, Danilo Krummrich wrote:
> >>>>> On Sat, Aug 31, 2024 at 05:39:07AM +0000, Benno Lossin wrote:
> >>>>>> On 16.08.24 02:10, Danilo Krummrich wrote:
> >>>>>>> +///
> >>>>>>> +/// ```
> >>>>>>> +/// # use kernel::bindings;
> >>>>>>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
> >>>>>>> +/// struct Huge([u8; SIZE]);
> >>>>>>> +///
> >>>>>>> +/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
> >>>>>>> +/// ```
> >>>>>>
> >>>>>> Similarly, you could then say above this one "Instead use either `VBox`
> >>>>>> or `KVBox`:"
> >>>>>>
> >>>>>>> +///
> >>>>>>> +/// # Invariants
> >>>>>>> +///
> >>>>>>> +/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`
> >>>>>>
> >>>>>> Please use `self.0` instead of "[`Box`]'".
> >>>>>>
> >>>>>>> +/// or, for zero-sized types, is a dangling pointer.
> >>>>>>
> >>>>>> Probably "dangling, well aligned pointer.".
> >>>>>
> >>>>> Does this add any value? For ZSTs everything is "well aligned", isn't it?
> >>>>
> >>>> ZSTs can have alignment and then unaligned pointers do exist for them
> >>>> (and dereferencing them is UB!):
> >>>
> >>> Where is this documented? The documentation says:
> >>>
> >>> "For operations of size zero, *every* pointer is valid, including the null
> >>> pointer. The following points are only concerned with non-zero-sized accesses."
> >>> [1]
> >>
> >> That's a good point, the documentation looks a bit outdated. I found
> >> this page in the nomicon: https://doc.rust-lang.org/nomicon/vec/vec-zsts.html
> >> The first iterator implementation has an alignment issue. (Nevertheless,
> >> that chapter of the nomicon is probably useful to you, since it goes
> >> over implementing `Vec`, but maybe you already saw it)
> >>
> >>> [1] https://doc.rust-lang.org/std/ptr/index.html
> >>
> >> Might be a good idea to improve/complain about this at the rust project.
> >
> > Well, my point is how do we know? There's no language specification and the
> > documentation is (at least) ambiguous.
>
> So I went through the unsafe-coding-guidelines issues list and only
> found this one: https://github.com/rust-lang/unsafe-code-guidelines/issues/93
> Maybe I missed something. You could also try to ask at the rust zulip in
> the t-opsem channel for further clarification.
>
> I think we should just be on the safe side and assume that ZSTs require
> alignment. But if you get a convincing answer and if they say that they
> will document it, then I don't mind removing the alignment requirement.

Please see the section on alignment in the same page. Just because a
pointer is valid does not mean that it is properly aligned.

From the page:

Valid raw pointers as defined above are not necessarily properly
aligned (where “proper” alignment is defined by the pointee type,
i.e., *const T must be aligned to mem::align_of::<T>()). However, most
functions require their arguments to be properly aligned, and will
explicitly state this requirement in their documentation. Notable
exceptions to this are read_unaligned and write_unaligned.

When a function requires proper alignment, it does so even if the
access has size 0, i.e., even if memory is not actually touched.
Consider using NonNull::dangling in such cases.

Alice

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Danilo Krummrich]

On Wed, Sep 11, 2024 at 03:27:57PM +0200, Alice Ryhl wrote:
> On Wed, Sep 11, 2024 at 3:26 PM Benno Lossin <benno.lossin@proton.me> wrote:
> >
> > On 11.09.24 13:02, Danilo Krummrich wrote:
> > > On Wed, Sep 11, 2024 at 08:36:38AM +0000, Benno Lossin wrote:
> > >> On 11.09.24 01:25, Danilo Krummrich wrote:
> > >>> On Tue, Sep 10, 2024 at 07:49:42PM +0000, Benno Lossin wrote:
> > >>>> On 10.09.24 19:40, Danilo Krummrich wrote:
> > >>>>> On Sat, Aug 31, 2024 at 05:39:07AM +0000, Benno Lossin wrote:
> > >>>>>> On 16.08.24 02:10, Danilo Krummrich wrote:
> > >>>>>>> +///
> > >>>>>>> +/// ```
> > >>>>>>> +/// # use kernel::bindings;
> > >>>>>>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
> > >>>>>>> +/// struct Huge([u8; SIZE]);
> > >>>>>>> +///
> > >>>>>>> +/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
> > >>>>>>> +/// ```
> > >>>>>>
> > >>>>>> Similarly, you could then say above this one "Instead use either `VBox`
> > >>>>>> or `KVBox`:"
> > >>>>>>
> > >>>>>>> +///
> > >>>>>>> +/// # Invariants
> > >>>>>>> +///
> > >>>>>>> +/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`
> > >>>>>>
> > >>>>>> Please use `self.0` instead of "[`Box`]'".
> > >>>>>>
> > >>>>>>> +/// or, for zero-sized types, is a dangling pointer.
> > >>>>>>
> > >>>>>> Probably "dangling, well aligned pointer.".
> > >>>>>
> > >>>>> Does this add any value? For ZSTs everything is "well aligned", isn't it?
> > >>>>
> > >>>> ZSTs can have alignment and then unaligned pointers do exist for them
> > >>>> (and dereferencing them is UB!):
> > >>>
> > >>> Where is this documented? The documentation says:
> > >>>
> > >>> "For operations of size zero, *every* pointer is valid, including the null
> > >>> pointer. The following points are only concerned with non-zero-sized accesses."
> > >>> [1]
> > >>
> > >> That's a good point, the documentation looks a bit outdated. I found
> > >> this page in the nomicon: https://doc.rust-lang.org/nomicon/vec/vec-zsts.html
> > >> The first iterator implementation has an alignment issue. (Nevertheless,
> > >> that chapter of the nomicon is probably useful to you, since it goes
> > >> over implementing `Vec`, but maybe you already saw it)
> > >>
> > >>> [1] https://doc.rust-lang.org/std/ptr/index.html
> > >>
> > >> Might be a good idea to improve/complain about this at the rust project.
> > >
> > > Well, my point is how do we know? There's no language specification and the
> > > documentation is (at least) ambiguous.
> >
> > So I went through the unsafe-coding-guidelines issues list and only
> > found this one: https://github.com/rust-lang/unsafe-code-guidelines/issues/93
> > Maybe I missed something. You could also try to ask at the rust zulip in
> > the t-opsem channel for further clarification.
> >
> > I think we should just be on the safe side and assume that ZSTs require
> > alignment. But if you get a convincing answer and if they say that they
> > will document it, then I don't mind removing the alignment requirement.

I agree -- I also wrote this in a previous mail.

I was just wondering why you are so sure about it, since the documentation
doesn't seem to be clear about it.

> 
> Please see the section on alignment in the same page. Just because a
> pointer is valid does not mean that it is properly aligned.
> 
> From the page:
> 
> Valid raw pointers as defined above are not necessarily properly
> aligned (where “proper” alignment is defined by the pointee type,
> i.e., *const T must be aligned to mem::align_of::<T>()). However, most
> functions require their arguments to be properly aligned, and will
> explicitly state this requirement in their documentation. Notable
> exceptions to this are read_unaligned and write_unaligned.
> 
> When a function requires proper alignment, it does so even if the
> access has size 0, i.e., even if memory is not actually touched.
> Consider using NonNull::dangling in such cases.

Good point.

It still sounds like it's only required for functions that explicitly state so.

And as cited from nomicon "This is possibly needless pedantry because ptr::read
is a noop for a ZST, [...]". But, no question, of course we have to honor it
anyways.

> 
> Alice
> 

Re: [PATCH v6 09/26] rust: alloc: implement kernel `Box`

[Benno Lossin]

On 11.09.24 16:50, Danilo Krummrich wrote:
> On Wed, Sep 11, 2024 at 03:27:57PM +0200, Alice Ryhl wrote:
>> On Wed, Sep 11, 2024 at 3:26 PM Benno Lossin <benno.lossin@proton.me> wrote:
>>> On 11.09.24 13:02, Danilo Krummrich wrote:
>>>> On Wed, Sep 11, 2024 at 08:36:38AM +0000, Benno Lossin wrote:
>>>>> On 11.09.24 01:25, Danilo Krummrich wrote:
>>>>>> On Tue, Sep 10, 2024 at 07:49:42PM +0000, Benno Lossin wrote:
>>>>>>> On 10.09.24 19:40, Danilo Krummrich wrote:
>>>>>>>> On Sat, Aug 31, 2024 at 05:39:07AM +0000, Benno Lossin wrote:
>>>>>>>>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>>>>>>>>> +///
>>>>>>>>>> +/// ```
>>>>>>>>>> +/// # use kernel::bindings;
>>>>>>>>>> +/// const SIZE: usize = bindings::KMALLOC_MAX_SIZE as usize + 1;
>>>>>>>>>> +/// struct Huge([u8; SIZE]);
>>>>>>>>>> +///
>>>>>>>>>> +/// assert!(KVBox::<Huge>::new_uninit(GFP_KERNEL).is_ok());
>>>>>>>>>> +/// ```
>>>>>>>>>
>>>>>>>>> Similarly, you could then say above this one "Instead use either `VBox`
>>>>>>>>> or `KVBox`:"
>>>>>>>>>
>>>>>>>>>> +///
>>>>>>>>>> +/// # Invariants
>>>>>>>>>> +///
>>>>>>>>>> +/// The [`Box`]' pointer is always properly aligned and either points to memory allocated with `A`
>>>>>>>>>
>>>>>>>>> Please use `self.0` instead of "[`Box`]'".
>>>>>>>>>
>>>>>>>>>> +/// or, for zero-sized types, is a dangling pointer.
>>>>>>>>>
>>>>>>>>> Probably "dangling, well aligned pointer.".
>>>>>>>>
>>>>>>>> Does this add any value? For ZSTs everything is "well aligned", isn't it?
>>>>>>>
>>>>>>> ZSTs can have alignment and then unaligned pointers do exist for them
>>>>>>> (and dereferencing them is UB!):
>>>>>>
>>>>>> Where is this documented? The documentation says:
>>>>>>
>>>>>> "For operations of size zero, *every* pointer is valid, including the null
>>>>>> pointer. The following points are only concerned with non-zero-sized accesses."
>>>>>> [1]
>>>>>
>>>>> That's a good point, the documentation looks a bit outdated. I found
>>>>> this page in the nomicon: https://doc.rust-lang.org/nomicon/vec/vec-zsts.html
>>>>> The first iterator implementation has an alignment issue. (Nevertheless,
>>>>> that chapter of the nomicon is probably useful to you, since it goes
>>>>> over implementing `Vec`, but maybe you already saw it)
>>>>>
>>>>>> [1] https://doc.rust-lang.org/std/ptr/index.html
>>>>>
>>>>> Might be a good idea to improve/complain about this at the rust project.
>>>>
>>>> Well, my point is how do we know? There's no language specification and the
>>>> documentation is (at least) ambiguous.
>>>
>>> So I went through the unsafe-coding-guidelines issues list and only
>>> found this one: https://github.com/rust-lang/unsafe-code-guidelines/issues/93
>>> Maybe I missed something. You could also try to ask at the rust zulip in
>>> the t-opsem channel for further clarification.
>>>
>>> I think we should just be on the safe side and assume that ZSTs require
>>> alignment. But if you get a convincing answer and if they say that they
>>> will document it, then I don't mind removing the alignment requirement.
> 
> I agree -- I also wrote this in a previous mail.
> 
> I was just wondering why you are so sure about it, since the documentation
> doesn't seem to be clear about it.

As Alice found below, the documentation is actually clear about this. (I
think I read it at some point, but forgot exactly where it was)

Maybe it could be better documented that dereferencing has the same
requirements as `read` (or whatever they are).

>> Please see the section on alignment in the same page. Just because a
>> pointer is valid does not mean that it is properly aligned.
>>
>> From the page:
>>
>> Valid raw pointers as defined above are not necessarily properly
>> aligned (where “proper” alignment is defined by the pointee type,
>> i.e., *const T must be aligned to mem::align_of::<T>()). However, most
>> functions require their arguments to be properly aligned, and will
>> explicitly state this requirement in their documentation. Notable
>> exceptions to this are read_unaligned and write_unaligned.
>>
>> When a function requires proper alignment, it does so even if the
>> access has size 0, i.e., even if memory is not actually touched.
>> Consider using NonNull::dangling in such cases.
> 
> Good point.
> 
> It still sounds like it's only required for functions that explicitly state so.
> 
> And as cited from nomicon "This is possibly needless pedantry because ptr::read
> is a noop for a ZST, [...]". But, no question, of course we have to honor it
> anyways.

This sounds to me like an implementation detail note, not something that
a caller should consider. But that's my interpretation.

---
Cheers,
Benno

[PATCH v6 10/26] rust: treewide: switch to our kernel `Box` type

[Danilo Krummrich]

Now that we got the kernel `Box` type in place, convert all existing
`Box` users to make use of it.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 drivers/block/rnull.rs            |  4 +--
 rust/kernel/init.rs               | 51 ++++++++++++++++---------------
 rust/kernel/init/__internal.rs    |  2 +-
 rust/kernel/rbtree.rs             | 34 ++++++++++++---------
 rust/kernel/sync/arc.rs           | 17 +++++------
 rust/kernel/sync/condvar.rs       |  4 +--
 rust/kernel/sync/lock/mutex.rs    |  2 +-
 rust/kernel/sync/lock/spinlock.rs |  2 +-
 rust/kernel/workqueue.rs          | 20 ++++++------
 rust/macros/lib.rs                |  6 ++--
 10 files changed, 73 insertions(+), 69 deletions(-)

diff --git a/drivers/block/rnull.rs b/drivers/block/rnull.rs
index b0227cf9ddd3..5de7223beb4d 100644
--- a/drivers/block/rnull.rs
+++ b/drivers/block/rnull.rs
@@ -32,7 +32,7 @@
 }
 
 struct NullBlkModule {
-    _disk: Pin<Box<Mutex<GenDisk<NullBlkDevice>>>>,
+    _disk: Pin<KBox<Mutex<GenDisk<NullBlkDevice>>>>,
 }
 
 impl kernel::Module for NullBlkModule {
@@ -47,7 +47,7 @@ fn init(_module: &'static ThisModule) -> Result<Self> {
             .rotational(false)
             .build(format_args!("rnullb{}", 0), tagset)?;
 
-        let disk = Box::pin_init(new_mutex!(disk, "nullb:disk"), flags::GFP_KERNEL)?;
+        let disk = KBox::pin_init(new_mutex!(disk, "nullb:disk"), flags::GFP_KERNEL)?;
 
         Ok(Self { _disk: disk })
     }
diff --git a/rust/kernel/init.rs b/rust/kernel/init.rs
index 1c3cfad89165..0646546d9356 100644
--- a/rust/kernel/init.rs
+++ b/rust/kernel/init.rs
@@ -13,7 +13,7 @@
 //! To initialize a `struct` with an in-place constructor you will need two things:
 //! - an in-place constructor,
 //! - a memory location that can hold your `struct` (this can be the [stack], an [`Arc<T>`],
-//!   [`UniqueArc<T>`], [`Box<T>`] or any other smart pointer that implements [`InPlaceInit`]).
+//!   [`UniqueArc<T>`], [`KBox<T>`] or any other smart pointer that implements [`InPlaceInit`]).
 //!
 //! To get an in-place constructor there are generally three options:
 //! - directly creating an in-place constructor using the [`pin_init!`] macro,
@@ -68,7 +68,7 @@
 //! #     a <- new_mutex!(42, "Foo::a"),
 //! #     b: 24,
 //! # });
-//! let foo: Result<Pin<Box<Foo>>> = Box::pin_init(foo, GFP_KERNEL);
+//! let foo: Result<Pin<KBox<Foo>>> = KBox::pin_init(foo, GFP_KERNEL);
 //! ```
 //!
 //! For more information see the [`pin_init!`] macro.
@@ -93,14 +93,14 @@
 //! struct DriverData {
 //!     #[pin]
 //!     status: Mutex<i32>,
-//!     buffer: Box<[u8; 1_000_000]>,
+//!     buffer: KBox<[u8; 1_000_000]>,
 //! }
 //!
 //! impl DriverData {
 //!     fn new() -> impl PinInit<Self, Error> {
 //!         try_pin_init!(Self {
 //!             status <- new_mutex!(0, "DriverData::status"),
-//!             buffer: Box::init(kernel::init::zeroed(), GFP_KERNEL)?,
+//!             buffer: KBox::init(kernel::init::zeroed(), GFP_KERNEL)?,
 //!         })
 //!     }
 //! }
@@ -211,7 +211,7 @@
 //! [`pin_init!`]: crate::pin_init!
 
 use crate::{
-    alloc::{box_ext::BoxExt, AllocError, Flags},
+    alloc::{box_ext::BoxExt, AllocError, Flags, KBox},
     error::{self, Error},
     sync::UniqueArc,
     types::{Opaque, ScopeGuard},
@@ -297,7 +297,7 @@ macro_rules! stack_pin_init {
 /// struct Foo {
 ///     #[pin]
 ///     a: Mutex<usize>,
-///     b: Box<Bar>,
+///     b: KBox<Bar>,
 /// }
 ///
 /// struct Bar {
@@ -306,7 +306,7 @@ macro_rules! stack_pin_init {
 ///
 /// stack_try_pin_init!(let foo: Result<Pin<&mut Foo>, AllocError> = pin_init!(Foo {
 ///     a <- new_mutex!(42),
-///     b: Box::new(Bar {
+///     b: KBox::new(Bar {
 ///         x: 64,
 ///     }, GFP_KERNEL)?,
 /// }));
@@ -323,7 +323,7 @@ macro_rules! stack_pin_init {
 /// struct Foo {
 ///     #[pin]
 ///     a: Mutex<usize>,
-///     b: Box<Bar>,
+///     b: KBox<Bar>,
 /// }
 ///
 /// struct Bar {
@@ -332,7 +332,7 @@ macro_rules! stack_pin_init {
 ///
 /// stack_try_pin_init!(let foo: Pin<&mut Foo> =? pin_init!(Foo {
 ///     a <- new_mutex!(42),
-///     b: Box::new(Bar {
+///     b: KBox::new(Bar {
 ///         x: 64,
 ///     }, GFP_KERNEL)?,
 /// }));
@@ -391,7 +391,7 @@ macro_rules! stack_try_pin_init {
 ///     },
 /// });
 /// # initializer }
-/// # Box::pin_init(demo(), GFP_KERNEL).unwrap();
+/// # KBox::pin_init(demo(), GFP_KERNEL).unwrap();
 /// ```
 ///
 /// Arbitrary Rust expressions can be used to set the value of a variable.
@@ -461,7 +461,7 @@ macro_rules! stack_try_pin_init {
 /// #         })
 /// #     }
 /// # }
-/// let foo = Box::pin_init(Foo::new(), GFP_KERNEL);
+/// let foo = KBox::pin_init(Foo::new(), GFP_KERNEL);
 /// ```
 ///
 /// They can also easily embed it into their own `struct`s:
@@ -593,7 +593,7 @@ macro_rules! pin_init {
 /// use kernel::{init::{self, PinInit}, error::Error};
 /// #[pin_data]
 /// struct BigBuf {
-///     big: Box<[u8; 1024 * 1024 * 1024]>,
+///     big: KBox<[u8; 1024 * 1024 * 1024]>,
 ///     small: [u8; 1024 * 1024],
 ///     ptr: *mut u8,
 /// }
@@ -601,7 +601,7 @@ macro_rules! pin_init {
 /// impl BigBuf {
 ///     fn new() -> impl PinInit<Self, Error> {
 ///         try_pin_init!(Self {
-///             big: Box::init(init::zeroed(), GFP_KERNEL)?,
+///             big: KBox::init(init::zeroed(), GFP_KERNEL)?,
 ///             small: [0; 1024 * 1024],
 ///             ptr: core::ptr::null_mut(),
 ///         }? Error)
@@ -693,16 +693,16 @@ macro_rules! init {
 /// # Examples
 ///
 /// ```rust
-/// use kernel::{init::{PinInit, zeroed}, error::Error};
+/// use kernel::{alloc::KBox, init::{PinInit, zeroed}, error::Error};
 /// struct BigBuf {
-///     big: Box<[u8; 1024 * 1024 * 1024]>,
+///     big: KBox<[u8; 1024 * 1024 * 1024]>,
 ///     small: [u8; 1024 * 1024],
 /// }
 ///
 /// impl BigBuf {
 ///     fn new() -> impl Init<Self, Error> {
 ///         try_init!(Self {
-///             big: Box::init(zeroed(), GFP_KERNEL)?,
+///             big: KBox::init(zeroed(), GFP_KERNEL)?,
 ///             small: [0; 1024 * 1024],
 ///         }? Error)
 ///     }
@@ -745,8 +745,8 @@ macro_rules! try_init {
 /// A pin-initializer for the type `T`.
 ///
 /// To use this initializer, you will need a suitable memory location that can hold a `T`. This can
-/// be [`Box<T>`], [`Arc<T>`], [`UniqueArc<T>`] or even the stack (see [`stack_pin_init!`]). Use the
-/// [`InPlaceInit::pin_init`] function of a smart pointer like [`Arc<T>`] on this.
+/// be [`KBox<T>`], [`Arc<T>`], [`UniqueArc<T>`] or even the stack (see [`stack_pin_init!`]). Use
+/// the [`InPlaceInit::pin_init`] function of a smart pointer like [`Arc<T>`] on this.
 ///
 /// Also see the [module description](self).
 ///
@@ -825,7 +825,7 @@ fn pin_chain<F>(self, f: F) -> ChainPinInit<Self, F, T, E>
 }
 
 /// An initializer returned by [`PinInit::pin_chain`].
-pub struct ChainPinInit<I, F, T: ?Sized, E>(I, F, __internal::Invariant<(E, Box<T>)>);
+pub struct ChainPinInit<I, F, T: ?Sized, E>(I, F, __internal::Invariant<(E, KBox<T>)>);
 
 // SAFETY: The `__pinned_init` function is implemented such that it
 // - returns `Ok(())` on successful initialization,
@@ -851,8 +851,8 @@ unsafe fn __pinned_init(self, slot: *mut T) -> Result<(), E> {
 /// An initializer for `T`.
 ///
 /// To use this initializer, you will need a suitable memory location that can hold a `T`. This can
-/// be [`Box<T>`], [`Arc<T>`], [`UniqueArc<T>`] or even the stack (see [`stack_pin_init!`]). Use the
-/// [`InPlaceInit::init`] function of a smart pointer like [`Arc<T>`] on this. Because
+/// be [`KBox<T>`], [`Arc<T>`], [`UniqueArc<T>`] or even the stack (see [`stack_pin_init!`]). Use
+/// the [`InPlaceInit::init`] function of a smart pointer like [`Arc<T>`] on this. Because
 /// [`PinInit<T, E>`] is a super trait, you can use every function that takes it as well.
 ///
 /// Also see the [module description](self).
@@ -924,7 +924,7 @@ fn chain<F>(self, f: F) -> ChainInit<Self, F, T, E>
 }
 
 /// An initializer returned by [`Init::chain`].
-pub struct ChainInit<I, F, T: ?Sized, E>(I, F, __internal::Invariant<(E, Box<T>)>);
+pub struct ChainInit<I, F, T: ?Sized, E>(I, F, __internal::Invariant<(E, KBox<T>)>);
 
 // SAFETY: The `__init` function is implemented such that it
 // - returns `Ok(())` on successful initialization,
@@ -1008,8 +1008,9 @@ pub fn uninit<T, E>() -> impl Init<MaybeUninit<T>, E> {
 /// # Examples
 ///
 /// ```rust
-/// use kernel::{error::Error, init::init_array_from_fn};
-/// let array: Box<[usize; 1_000]> = Box::init::<Error>(init_array_from_fn(|i| i), GFP_KERNEL).unwrap();
+/// use kernel::{alloc::KBox, error::Error, init::init_array_from_fn};
+/// let array: KBox<[usize; 1_000]> =
+///     KBox::init::<Error>(init_array_from_fn(|i| i), GFP_KERNEL).unwrap();
 /// assert_eq!(array.len(), 1_000);
 /// ```
 pub fn init_array_from_fn<I, const N: usize, T, E>(
@@ -1352,7 +1353,7 @@ macro_rules! impl_zeroable {
     //
     // In this case we are allowed to use `T: ?Sized`, since all zeros is the `None` variant.
     {<T: ?Sized>} Option<NonNull<T>>,
-    {<T: ?Sized>} Option<Box<T>>,
+    {<T: ?Sized>} Option<KBox<T>>,
 
     // SAFETY: `null` pointer is valid.
     //
diff --git a/rust/kernel/init/__internal.rs b/rust/kernel/init/__internal.rs
index db3372619ecd..dfb2204918c7 100644
--- a/rust/kernel/init/__internal.rs
+++ b/rust/kernel/init/__internal.rs
@@ -102,7 +102,7 @@ fn make_closure<F, O, E>(self, f: F) -> F
     }
 }
 
-pub struct AllData<T: ?Sized>(PhantomData<fn(Box<T>) -> Box<T>>);
+pub struct AllData<T: ?Sized>(PhantomData<fn(KBox<T>) -> KBox<T>>);
 
 impl<T: ?Sized> Clone for AllData<T> {
     fn clone(&self) -> Self {
diff --git a/rust/kernel/rbtree.rs b/rust/kernel/rbtree.rs
index 428f8be8f3a2..e42fe222826c 100644
--- a/rust/kernel/rbtree.rs
+++ b/rust/kernel/rbtree.rs
@@ -7,7 +7,6 @@
 //! Reference: <https://www.kernel.org/doc/html/latest/core-api/rbtree.html>
 
 use crate::{alloc::Flags, bindings, container_of, error::Result, prelude::*};
-use alloc::boxed::Box;
 use core::{
     cmp::{Ord, Ordering},
     marker::PhantomData,
@@ -507,7 +506,7 @@ fn drop(&mut self) {
             // but it is not observable. The loop invariant is still maintained.
 
             // SAFETY: `this` is valid per the loop invariant.
-            unsafe { drop(Box::from_raw(this.cast_mut())) };
+            unsafe { drop(KBox::from_raw(this.cast_mut())) };
         }
     }
 }
@@ -771,7 +770,7 @@ pub fn remove_current(self) -> (Option<Self>, RBTreeNode<K, V>) {
         // point to the links field of `Node<K, V>` objects.
         let this = unsafe { container_of!(self.current.as_ptr(), Node<K, V>, links) }.cast_mut();
         // SAFETY: `this` is valid by the type invariants as described above.
-        let node = unsafe { Box::from_raw(this) };
+        let node = unsafe { KBox::from_raw(this) };
         let node = RBTreeNode { node };
         // SAFETY: The reference to the tree used to create the cursor outlives the cursor, so
         // the tree cannot change. By the tree invariant, all nodes are valid.
@@ -819,7 +818,7 @@ fn remove_neighbor(&mut self, direction: Direction) -> Option<RBTreeNode<K, V>>
             // point to the links field of `Node<K, V>` objects.
             let this = unsafe { container_of!(neighbor, Node<K, V>, links) }.cast_mut();
             // SAFETY: `this` is valid by the type invariants as described above.
-            let node = unsafe { Box::from_raw(this) };
+            let node = unsafe { KBox::from_raw(this) };
             return Some(RBTreeNode { node });
         }
         None
@@ -1047,7 +1046,7 @@ fn next(&mut self) -> Option<Self::Item> {
 /// It contains the memory needed to hold a node that can be inserted into a red-black tree. One
 /// can be obtained by directly allocating it ([`RBTreeNodeReservation::new`]).
 pub struct RBTreeNodeReservation<K, V> {
-    node: Box<MaybeUninit<Node<K, V>>>,
+    node: KBox<MaybeUninit<Node<K, V>>>,
 }
 
 impl<K, V> RBTreeNodeReservation<K, V> {
@@ -1055,7 +1054,7 @@ impl<K, V> RBTreeNodeReservation<K, V> {
     /// call to [`RBTree::insert`].
     pub fn new(flags: Flags) -> Result<RBTreeNodeReservation<K, V>> {
         Ok(RBTreeNodeReservation {
-            node: Box::new_uninit(flags)?,
+            node: KBox::new_uninit(flags)?,
         })
     }
 }
@@ -1072,7 +1071,7 @@ impl<K, V> RBTreeNodeReservation<K, V> {
     ///
     /// It then becomes an [`RBTreeNode`] that can be inserted into a tree.
     pub fn into_node(self, key: K, value: V) -> RBTreeNode<K, V> {
-        let node = Box::write(
+        let node = KBox::write(
             self.node,
             Node {
                 key,
@@ -1089,7 +1088,7 @@ pub fn into_node(self, key: K, value: V) -> RBTreeNode<K, V> {
 /// The node is fully initialised (with key and value) and can be inserted into a tree without any
 /// extra allocations or failure paths.
 pub struct RBTreeNode<K, V> {
-    node: Box<Node<K, V>>,
+    node: KBox<Node<K, V>>,
 }
 
 impl<K, V> RBTreeNode<K, V> {
@@ -1101,7 +1100,9 @@ pub fn new(key: K, value: V, flags: Flags) -> Result<RBTreeNode<K, V>> {
 
     /// Get the key and value from inside the node.
     pub fn to_key_value(self) -> (K, V) {
-        (self.node.key, self.node.value)
+        let node = KBox::into_inner(self.node);
+
+        (node.key, node.value)
     }
 }
 
@@ -1123,7 +1124,7 @@ impl<K, V> RBTreeNode<K, V> {
     /// may be freed (but only for the key/value; memory for the node itself is kept for reuse).
     pub fn into_reservation(self) -> RBTreeNodeReservation<K, V> {
         RBTreeNodeReservation {
-            node: Box::drop_contents(self.node),
+            node: KBox::drop_contents(self.node),
         }
     }
 }
@@ -1174,7 +1175,7 @@ impl<'a, K, V> RawVacantEntry<'a, K, V> {
     /// The `node` must have a key such that inserting it here does not break the ordering of this
     /// [`RBTree`].
     fn insert(self, node: RBTreeNode<K, V>) -> &'a mut V {
-        let node = Box::into_raw(node.node);
+        let node = KBox::into_raw(node.node);
 
         // SAFETY: `node` is valid at least until we call `Box::from_raw`, which only happens when
         // the node is removed or replaced.
@@ -1247,20 +1248,23 @@ pub fn remove_node(self) -> RBTreeNode<K, V> {
         RBTreeNode {
             // SAFETY: The node was a node in the tree, but we removed it, so we can convert it
             // back into a box.
-            node: unsafe { Box::from_raw(self.node_ptr()) },
+            node: unsafe { KBox::from_raw(self.node_ptr()) },
         }
     }
 
     /// Takes the value of the entry out of the map, and returns it.
     pub fn remove(self) -> V {
-        self.remove_node().node.value
+        let rb_node = self.remove_node();
+        let node = KBox::into_inner(rb_node.node);
+
+        node.value
     }
 
     /// Swap the current node for the provided node.
     ///
     /// The key of both nodes must be equal.
     fn replace(self, node: RBTreeNode<K, V>) -> RBTreeNode<K, V> {
-        let node = Box::into_raw(node.node);
+        let node = KBox::into_raw(node.node);
 
         // SAFETY: `node` is valid at least until we call `Box::from_raw`, which only happens when
         // the node is removed or replaced.
@@ -1275,7 +1279,7 @@ fn replace(self, node: RBTreeNode<K, V>) -> RBTreeNode<K, V> {
         // SAFETY:
         // - `self.node_ptr` produces a valid pointer to a node in the tree.
         // - Now that we removed this entry from the tree, we can convert the node to a box.
-        let old_node = unsafe { Box::from_raw(self.node_ptr()) };
+        let old_node = unsafe { KBox::from_raw(self.node_ptr()) };
 
         RBTreeNode { node: old_node }
     }
diff --git a/rust/kernel/sync/arc.rs b/rust/kernel/sync/arc.rs
index 9023479281f0..5a868ce013f7 100644
--- a/rust/kernel/sync/arc.rs
+++ b/rust/kernel/sync/arc.rs
@@ -16,13 +16,12 @@
 //! [`Arc`]: https://doc.rust-lang.org/std/sync/struct.Arc.html
 
 use crate::{
-    alloc::{box_ext::BoxExt, AllocError, Flags},
+    alloc::{AllocError, Flags, KBox},
     error::{self, Error},
     init::{self, InPlaceInit, Init, PinInit},
     try_init,
     types::{ForeignOwnable, Opaque},
 };
-use alloc::boxed::Box;
 use core::{
     alloc::Layout,
     fmt,
@@ -203,11 +202,11 @@ pub fn new(contents: T, flags: Flags) -> Result<Self, AllocError> {
             data: contents,
         };
 
-        let inner = <Box<_> as BoxExt<_>>::new(value, flags)?;
+        let inner = KBox::new(value, flags)?;
 
         // SAFETY: We just created `inner` with a reference count of 1, which is owned by the new
         // `Arc` object.
-        Ok(unsafe { Self::from_inner(Box::leak(inner).into()) })
+        Ok(unsafe { Self::from_inner(KBox::leak(inner).into()) })
     }
 
     /// Use the given initializer to in-place initialize a `T`.
@@ -431,8 +430,8 @@ fn drop(&mut self) {
         if is_zero {
             // The count reached zero, we must free the memory.
             //
-            // SAFETY: The pointer was initialised from the result of `Box::leak`.
-            unsafe { drop(Box::from_raw(self.ptr.as_ptr())) };
+            // SAFETY: The pointer was initialised from the result of `KBox::leak`.
+            unsafe { drop(KBox::from_raw(self.ptr.as_ptr())) };
         }
     }
 }
@@ -677,7 +676,7 @@ pub fn new(value: T, flags: Flags) -> Result<Self, AllocError> {
     /// Tries to allocate a new [`UniqueArc`] instance whose contents are not initialised yet.
     pub fn new_uninit(flags: Flags) -> Result<UniqueArc<MaybeUninit<T>>, AllocError> {
         // INVARIANT: The refcount is initialised to a non-zero value.
-        let inner = Box::try_init::<AllocError>(
+        let inner = KBox::try_init::<AllocError>(
             try_init!(ArcInner {
                 // SAFETY: There are no safety requirements for this FFI call.
                 refcount: Opaque::new(unsafe { bindings::REFCOUNT_INIT(1) }),
@@ -687,8 +686,8 @@ pub fn new_uninit(flags: Flags) -> Result<UniqueArc<MaybeUninit<T>>, AllocError>
         )?;
         Ok(UniqueArc {
             // INVARIANT: The newly-created object has a refcount of 1.
-            // SAFETY: The pointer from the `Box` is valid.
-            inner: unsafe { Arc::from_inner(Box::leak(inner).into()) },
+            // SAFETY: The pointer from the `KBox` is valid.
+            inner: unsafe { Arc::from_inner(KBox::leak(inner).into()) },
         })
     }
 }
diff --git a/rust/kernel/sync/condvar.rs b/rust/kernel/sync/condvar.rs
index 2b306afbe56d..2081932bb4b9 100644
--- a/rust/kernel/sync/condvar.rs
+++ b/rust/kernel/sync/condvar.rs
@@ -70,8 +70,8 @@ macro_rules! new_condvar {
 /// }
 ///
 /// /// Allocates a new boxed `Example`.
-/// fn new_example() -> Result<Pin<Box<Example>>> {
-///     Box::pin_init(pin_init!(Example {
+/// fn new_example() -> Result<Pin<KBox<Example>>> {
+///     KBox::pin_init(pin_init!(Example {
 ///         value <- new_mutex!(0),
 ///         value_changed <- new_condvar!(),
 ///     }), GFP_KERNEL)
diff --git a/rust/kernel/sync/lock/mutex.rs b/rust/kernel/sync/lock/mutex.rs
index 30632070ee67..f8f6d530db7d 100644
--- a/rust/kernel/sync/lock/mutex.rs
+++ b/rust/kernel/sync/lock/mutex.rs
@@ -58,7 +58,7 @@ macro_rules! new_mutex {
 /// }
 ///
 /// // Allocate a boxed `Example`.
-/// let e = Box::pin_init(Example::new(), GFP_KERNEL)?;
+/// let e = KBox::pin_init(Example::new(), GFP_KERNEL)?;
 /// assert_eq!(e.c, 10);
 /// assert_eq!(e.d.lock().a, 20);
 /// assert_eq!(e.d.lock().b, 30);
diff --git a/rust/kernel/sync/lock/spinlock.rs b/rust/kernel/sync/lock/spinlock.rs
index ea5c5bc1ce12..a9096a4dc42a 100644
--- a/rust/kernel/sync/lock/spinlock.rs
+++ b/rust/kernel/sync/lock/spinlock.rs
@@ -56,7 +56,7 @@ macro_rules! new_spinlock {
 /// }
 ///
 /// // Allocate a boxed `Example`.
-/// let e = Box::pin_init(Example::new(), GFP_KERNEL)?;
+/// let e = KBox::pin_init(Example::new(), GFP_KERNEL)?;
 /// assert_eq!(e.c, 10);
 /// assert_eq!(e.d.lock().a, 20);
 /// assert_eq!(e.d.lock().b, 30);
diff --git a/rust/kernel/workqueue.rs b/rust/kernel/workqueue.rs
index 553a5cba2adc..94318472507f 100644
--- a/rust/kernel/workqueue.rs
+++ b/rust/kernel/workqueue.rs
@@ -216,7 +216,7 @@ pub fn try_spawn<T: 'static + Send + FnOnce()>(
             func: Some(func),
         });
 
-        self.enqueue(Box::pin_init(init, flags).map_err(|_| AllocError)?);
+        self.enqueue(KBox::pin_init(init, flags).map_err(|_| AllocError)?);
         Ok(())
     }
 }
@@ -239,9 +239,9 @@ fn project(self: Pin<&mut Self>) -> &mut Option<T> {
 }
 
 impl<T: FnOnce()> WorkItem for ClosureWork<T> {
-    type Pointer = Pin<Box<Self>>;
+    type Pointer = Pin<KBox<Self>>;
 
-    fn run(mut this: Pin<Box<Self>>) {
+    fn run(mut this: Pin<KBox<Self>>) {
         if let Some(func) = this.as_mut().project().take() {
             (func)()
         }
@@ -297,7 +297,7 @@ unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput
 
 /// Defines the method that should be called directly when a work item is executed.
 ///
-/// This trait is implemented by `Pin<Box<T>>` and [`Arc<T>`], and is mainly intended to be
+/// This trait is implemented by `Pin<KBox<T>>` and [`Arc<T>`], and is mainly intended to be
 /// implemented for smart pointer types. For your own structs, you would implement [`WorkItem`]
 /// instead. The [`run`] method on this trait will usually just perform the appropriate
 /// `container_of` translation and then call into the [`run`][WorkItem::run] method from the
@@ -329,7 +329,7 @@ pub unsafe trait WorkItemPointer<const ID: u64>: RawWorkItem<ID> {
 /// This trait is used when the `work_struct` field is defined using the [`Work`] helper.
 pub trait WorkItem<const ID: u64 = 0> {
     /// The pointer type that this struct is wrapped in. This will typically be `Arc<Self>` or
-    /// `Pin<Box<Self>>`.
+    /// `Pin<KBox<Self>>`.
     type Pointer: WorkItemPointer<ID>;
 
     /// The method that should be called when this work item is executed.
@@ -565,7 +565,7 @@ unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput
     }
 }
 
-unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Pin<Box<T>>
+unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Pin<KBox<T>>
 where
     T: WorkItem<ID, Pointer = Self>,
     T: HasWork<T, ID>,
@@ -576,7 +576,7 @@ unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Pin<Box<T>>
         // SAFETY: This computes the pointer that `__enqueue` got from `Arc::into_raw`.
         let ptr = unsafe { T::work_container_of(ptr) };
         // SAFETY: This pointer comes from `Arc::into_raw` and we've been given back ownership.
-        let boxed = unsafe { Box::from_raw(ptr) };
+        let boxed = unsafe { KBox::from_raw(ptr) };
         // SAFETY: The box was already pinned when it was enqueued.
         let pinned = unsafe { Pin::new_unchecked(boxed) };
 
@@ -584,7 +584,7 @@ unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Pin<Box<T>>
     }
 }
 
-unsafe impl<T, const ID: u64> RawWorkItem<ID> for Pin<Box<T>>
+unsafe impl<T, const ID: u64> RawWorkItem<ID> for Pin<KBox<T>>
 where
     T: WorkItem<ID, Pointer = Self>,
     T: HasWork<T, ID>,
@@ -598,9 +598,9 @@ unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput
         // SAFETY: We're not going to move `self` or any of its fields, so its okay to temporarily
         // remove the `Pin` wrapper.
         let boxed = unsafe { Pin::into_inner_unchecked(self) };
-        let ptr = Box::into_raw(boxed);
+        let ptr = KBox::into_raw(boxed);
 
-        // SAFETY: Pointers into a `Box` point at a valid value.
+        // SAFETY: Pointers into a `KBox` point at a valid value.
         let work_ptr = unsafe { T::raw_get_work(ptr) };
         // SAFETY: `raw_get_work` returns a pointer to a valid value.
         let work_ptr = unsafe { Work::raw_get(work_ptr) };
diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
index 159e75292970..c327fe0617c8 100644
--- a/rust/macros/lib.rs
+++ b/rust/macros/lib.rs
@@ -239,7 +239,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {
 /// struct DriverData {
 ///     #[pin]
 ///     queue: Mutex<Vec<Command>>,
-///     buf: Box<[u8; 1024 * 1024]>,
+///     buf: KBox<[u8; 1024 * 1024]>,
 /// }
 /// ```
 ///
@@ -248,7 +248,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {
 /// struct DriverData {
 ///     #[pin]
 ///     queue: Mutex<Vec<Command>>,
-///     buf: Box<[u8; 1024 * 1024]>,
+///     buf: KBox<[u8; 1024 * 1024]>,
 ///     raw_info: *mut Info,
 /// }
 ///
@@ -278,7 +278,7 @@ pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream {
 /// struct DriverData {
 ///     #[pin]
 ///     queue: Mutex<Vec<Command>>,
-///     buf: Box<[u8; 1024 * 1024]>,
+///     buf: KBox<[u8; 1024 * 1024]>,
 ///     raw_info: *mut Info,
 /// }
 ///
-- 
2.46.0

Re: [PATCH v6 10/26] rust: treewide: switch to our kernel `Box` type

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> Now that we got the kernel `Box` type in place, convert all existing
> `Box` users to make use of it.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

Reviewed-by: Benno Lossin <benno.lossin@proton.me>

---
Cheers,
Benno

> ---
>  drivers/block/rnull.rs            |  4 +--
>  rust/kernel/init.rs               | 51 ++++++++++++++++---------------
>  rust/kernel/init/__internal.rs    |  2 +-
>  rust/kernel/rbtree.rs             | 34 ++++++++++++---------
>  rust/kernel/sync/arc.rs           | 17 +++++------
>  rust/kernel/sync/condvar.rs       |  4 +--
>  rust/kernel/sync/lock/mutex.rs    |  2 +-
>  rust/kernel/sync/lock/spinlock.rs |  2 +-
>  rust/kernel/workqueue.rs          | 20 ++++++------
>  rust/macros/lib.rs                |  6 ++--
>  10 files changed, 73 insertions(+), 69 deletions(-)

[PATCH v6 11/26] rust: alloc: remove `BoxExt` extension

[Danilo Krummrich]

Now that all existing `Box` users were moved to the kernel `Box` type,
remove the `BoxExt` extension and all other related extensions.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc.rs         |  1 -
 rust/kernel/alloc/box_ext.rs | 80 ------------------------------------
 rust/kernel/init.rs          | 44 +-------------------
 rust/kernel/lib.rs           |  1 -
 rust/kernel/prelude.rs       |  4 +-
 rust/kernel/types.rs         | 28 -------------
 6 files changed, 3 insertions(+), 155 deletions(-)
 delete mode 100644 rust/kernel/alloc/box_ext.rs

diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index 0c9bb60250af..d248390345ec 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -4,7 +4,6 @@
 
 #[cfg(not(any(test, testlib)))]
 pub mod allocator;
-pub mod box_ext;
 pub mod kbox;
 pub mod vec_ext;
 
diff --git a/rust/kernel/alloc/box_ext.rs b/rust/kernel/alloc/box_ext.rs
deleted file mode 100644
index 076d5de5f47d..000000000000
--- a/rust/kernel/alloc/box_ext.rs
+++ /dev/null
@@ -1,80 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-//! Extensions to [`Box`] for fallible allocations.
-
-use super::{AllocError, Flags};
-use alloc::boxed::Box;
-use core::{mem::MaybeUninit, ptr, result::Result};
-
-/// Extensions to [`Box`].
-pub trait BoxExt<T>: Sized {
-    /// Allocates a new box.
-    ///
-    /// The allocation may fail, in which case an error is returned.
-    fn new(x: T, flags: Flags) -> Result<Self, AllocError>;
-
-    /// Allocates a new uninitialised box.
-    ///
-    /// The allocation may fail, in which case an error is returned.
-    fn new_uninit(flags: Flags) -> Result<Box<MaybeUninit<T>>, AllocError>;
-
-    /// Drops the contents, but keeps the allocation.
-    ///
-    /// # Examples
-    ///
-    /// ```
-    /// use kernel::alloc::{flags, box_ext::BoxExt};
-    /// let value = Box::new([0; 32], flags::GFP_KERNEL)?;
-    /// assert_eq!(*value, [0; 32]);
-    /// let value = Box::drop_contents(value);
-    /// // Now we can re-use `value`:
-    /// let value = Box::write(value, [1; 32]);
-    /// assert_eq!(*value, [1; 32]);
-    /// # Ok::<(), Error>(())
-    /// ```
-    fn drop_contents(this: Self) -> Box<MaybeUninit<T>>;
-}
-
-impl<T> BoxExt<T> for Box<T> {
-    fn new(x: T, flags: Flags) -> Result<Self, AllocError> {
-        let b = <Self as BoxExt<_>>::new_uninit(flags)?;
-        Ok(Box::write(b, x))
-    }
-
-    #[cfg(any(test, testlib))]
-    fn new_uninit(_flags: Flags) -> Result<Box<MaybeUninit<T>>, AllocError> {
-        Ok(Box::new_uninit())
-    }
-
-    #[cfg(not(any(test, testlib)))]
-    fn new_uninit(flags: Flags) -> Result<Box<MaybeUninit<T>>, AllocError> {
-        let ptr = if core::mem::size_of::<MaybeUninit<T>>() == 0 {
-            core::ptr::NonNull::<_>::dangling().as_ptr()
-        } else {
-            let layout = core::alloc::Layout::new::<MaybeUninit<T>>();
-
-            // SAFETY: Memory is being allocated (first arg is null). The only other source of
-            // safety issues is sleeping on atomic context, which is addressed by klint. Lastly,
-            // the type is not a SZT (checked above).
-            let ptr =
-                unsafe { super::allocator::krealloc_aligned(core::ptr::null_mut(), layout, flags) };
-            if ptr.is_null() {
-                return Err(AllocError);
-            }
-
-            ptr.cast::<MaybeUninit<T>>()
-        };
-
-        // SAFETY: For non-zero-sized types, we allocate above using the global allocator. For
-        // zero-sized types, we use `NonNull::dangling`.
-        Ok(unsafe { Box::from_raw(ptr) })
-    }
-
-    fn drop_contents(this: Self) -> Box<MaybeUninit<T>> {
-        let ptr = Box::into_raw(this);
-        // SAFETY: `ptr` is valid, because it came from `Box::into_raw`.
-        unsafe { ptr::drop_in_place(ptr) };
-        // SAFETY: `ptr` is valid, because it came from `Box::into_raw`.
-        unsafe { Box::from_raw(ptr.cast()) }
-    }
-}
diff --git a/rust/kernel/init.rs b/rust/kernel/init.rs
index 0646546d9356..72452ddc0606 100644
--- a/rust/kernel/init.rs
+++ b/rust/kernel/init.rs
@@ -211,12 +211,11 @@
 //! [`pin_init!`]: crate::pin_init!
 
 use crate::{
-    alloc::{box_ext::BoxExt, AllocError, Flags, KBox},
+    alloc::{AllocError, Flags, KBox},
     error::{self, Error},
     sync::UniqueArc,
     types::{Opaque, ScopeGuard},
 };
-use alloc::boxed::Box;
 use core::{
     cell::UnsafeCell,
     convert::Infallible,
@@ -589,7 +588,6 @@ macro_rules! pin_init {
 /// # Examples
 ///
 /// ```rust
-/// # #![feature(new_uninit)]
 /// use kernel::{init::{self, PinInit}, error::Error};
 /// #[pin_data]
 /// struct BigBuf {
@@ -1149,24 +1147,6 @@ fn init<E>(init: impl Init<T, E>, flags: Flags) -> error::Result<Self>
     }
 }
 
-impl<T> InPlaceInit<T> for Box<T> {
-    #[inline]
-    fn try_pin_init<E>(init: impl PinInit<T, E>, flags: Flags) -> Result<Pin<Self>, E>
-    where
-        E: From<AllocError>,
-    {
-        <Box<_> as BoxExt<_>>::new_uninit(flags)?.write_pin_init(init)
-    }
-
-    #[inline]
-    fn try_init<E>(init: impl Init<T, E>, flags: Flags) -> Result<Self, E>
-    where
-        E: From<AllocError>,
-    {
-        <Box<_> as BoxExt<_>>::new_uninit(flags)?.write_init(init)
-    }
-}
-
 impl<T> InPlaceInit<T> for UniqueArc<T> {
     #[inline]
     fn try_pin_init<E>(init: impl PinInit<T, E>, flags: Flags) -> Result<Pin<Self>, E>
@@ -1201,28 +1181,6 @@ pub trait InPlaceWrite<T> {
     fn write_pin_init<E>(self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E>;
 }
 
-impl<T> InPlaceWrite<T> for Box<MaybeUninit<T>> {
-    type Initialized = Box<T>;
-
-    fn write_init<E>(mut self, init: impl Init<T, E>) -> Result<Self::Initialized, E> {
-        let slot = self.as_mut_ptr();
-        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
-        // slot is valid.
-        unsafe { init.__init(slot)? };
-        // SAFETY: All fields have been initialized.
-        Ok(unsafe { self.assume_init() })
-    }
-
-    fn write_pin_init<E>(mut self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E> {
-        let slot = self.as_mut_ptr();
-        // SAFETY: When init errors/panics, slot will get deallocated but not dropped,
-        // slot is valid and will not be moved, because we pin it later.
-        unsafe { init.__pinned_init(slot)? };
-        // SAFETY: All fields have been initialized.
-        Ok(unsafe { self.assume_init() }.into())
-    }
-}
-
 impl<T> InPlaceWrite<T> for UniqueArc<MaybeUninit<T>> {
     type Initialized = UniqueArc<T>;
 
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index a10d0e140ab1..4599bd06b1e6 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -14,7 +14,6 @@
 #![no_std]
 #![feature(coerce_unsized)]
 #![feature(dispatch_from_dyn)]
-#![feature(new_uninit)]
 #![feature(receiver_trait)]
 #![feature(unsize)]
 
diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
index a9210634a8c3..c1f8e5c832e2 100644
--- a/rust/kernel/prelude.rs
+++ b/rust/kernel/prelude.rs
@@ -14,10 +14,10 @@
 #[doc(no_inline)]
 pub use core::pin::Pin;
 
-pub use crate::alloc::{box_ext::BoxExt, flags::*, vec_ext::VecExt, KBox, KVBox, VBox};
+pub use crate::alloc::{flags::*, vec_ext::VecExt, KBox, KVBox, VBox};
 
 #[doc(no_inline)]
-pub use alloc::{boxed::Box, vec::Vec};
+pub use alloc::vec::Vec;
 
 #[doc(no_inline)]
 pub use macros::{module, pin_data, pinned_drop, vtable, Zeroable};
diff --git a/rust/kernel/types.rs b/rust/kernel/types.rs
index 9b9e9b913d1c..234b615662a2 100644
--- a/rust/kernel/types.rs
+++ b/rust/kernel/types.rs
@@ -3,7 +3,6 @@
 //! Kernel types.
 
 use crate::init::{self, PinInit};
-use alloc::boxed::Box;
 use core::{
     cell::UnsafeCell,
     marker::{PhantomData, PhantomPinned},
@@ -105,33 +104,6 @@ unsafe fn try_from_foreign(ptr: *const core::ffi::c_void) -> Option<Self> {
     unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> Self::BorrowedMut<'a>;
 }
 
-impl<T: 'static> ForeignOwnable for Box<T> {
-    type Borrowed<'a> = &'a T;
-    type BorrowedMut<'a> = &'a mut T;
-
-    fn into_foreign(self) -> *const core::ffi::c_void {
-        Box::into_raw(self) as _
-    }
-
-    unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self {
-        // SAFETY: The safety requirements of this function ensure that `ptr` comes from a previous
-        // call to `Self::into_foreign`.
-        unsafe { Box::from_raw(ptr as _) }
-    }
-
-    unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> &'a T {
-        // SAFETY: The safety requirements of this method ensure that the object remains alive and
-        // immutable for the duration of 'a.
-        unsafe { &*ptr.cast() }
-    }
-
-    unsafe fn borrow_mut<'a>(ptr: *const core::ffi::c_void) -> &'a mut T {
-        // SAFETY: The safety requirements of this method ensure that the pointer is valid and that
-        // nothing else will access the value for the duration of 'a.
-        unsafe { &mut *ptr.cast_mut().cast() }
-    }
-}
-
 impl ForeignOwnable for () {
     type Borrowed<'a> = ();
     type BorrowedMut<'a> = ();
-- 
2.46.0

Re: [PATCH v6 11/26] rust: alloc: remove `BoxExt` extension

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> Now that all existing `Box` users were moved to the kernel `Box` type,
> remove the `BoxExt` extension and all other related extensions.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

This is just a minor nit, but could you please change the commit title
to "rust: alloc: remove extension of std's `Box`"?
With that:

Reviewed-by: Benno Lossin <benno.lossin@proton.me>

---
Cheers,
Benno

> ---
>  rust/kernel/alloc.rs         |  1 -
>  rust/kernel/alloc/box_ext.rs | 80 ------------------------------------
>  rust/kernel/init.rs          | 44 +-------------------
>  rust/kernel/lib.rs           |  1 -
>  rust/kernel/prelude.rs       |  4 +-
>  rust/kernel/types.rs         | 28 -------------
>  6 files changed, 3 insertions(+), 155 deletions(-)
>  delete mode 100644 rust/kernel/alloc/box_ext.rs

[PATCH v6 12/26] rust: alloc: add `Box` to prelude

[Danilo Krummrich]

Now that we removed `BoxExt` and the corresponding includes in
prelude.rs, add the new kernel `Box` type instead.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/prelude.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
index c1f8e5c832e2..d5f2fe42d093 100644
--- a/rust/kernel/prelude.rs
+++ b/rust/kernel/prelude.rs
@@ -14,7 +14,7 @@
 #[doc(no_inline)]
 pub use core::pin::Pin;
 
-pub use crate::alloc::{flags::*, vec_ext::VecExt, KBox, KVBox, VBox};
+pub use crate::alloc::{flags::*, vec_ext::VecExt, Box, KBox, KVBox, VBox};
 
 #[doc(no_inline)]
 pub use alloc::vec::Vec;
-- 
2.46.0

[PATCH v6 13/26] rust: alloc: implement kernel `Vec` type

[Danilo Krummrich]

`Vec` provides a contiguous growable array type (such as `Vec`) with
contents allocated with the kernel's allocators (e.g. `Kmalloc`,
`Vmalloc` or `KVmalloc`).

In contrast to Rust's `Vec` type, the kernel `Vec` type considers the
kernel's GFP flags for all appropriate functions, always reports
allocation failures through `Result<_, AllocError>` and remains
independent from unstable features.

Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc.rs      |   6 +
 rust/kernel/alloc/kvec.rs | 629 ++++++++++++++++++++++++++++++++++++++
 rust/kernel/prelude.rs    |   2 +-
 3 files changed, 636 insertions(+), 1 deletion(-)
 create mode 100644 rust/kernel/alloc/kvec.rs

diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index d248390345ec..e88c7e10ee9b 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -5,6 +5,7 @@
 #[cfg(not(any(test, testlib)))]
 pub mod allocator;
 pub mod kbox;
+pub mod kvec;
 pub mod vec_ext;
 
 #[cfg(any(test, testlib))]
@@ -18,6 +19,11 @@
 pub use self::kbox::KVBox;
 pub use self::kbox::VBox;
 
+pub use self::kvec::KVVec;
+pub use self::kvec::KVec;
+pub use self::kvec::VVec;
+pub use self::kvec::Vec;
+
 /// Indicates an allocation error.
 #[derive(Copy, Clone, PartialEq, Eq, Debug)]
 pub struct AllocError;
diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
new file mode 100644
index 000000000000..89afc0f25bd4
--- /dev/null
+++ b/rust/kernel/alloc/kvec.rs
@@ -0,0 +1,629 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Implementation of [`Vec`].
+
+use super::{AllocError, Allocator, Box, Flags};
+use core::{
+    fmt,
+    marker::PhantomData,
+    mem::{ManuallyDrop, MaybeUninit},
+    ops::Deref,
+    ops::DerefMut,
+    ops::Index,
+    ops::IndexMut,
+    ptr::NonNull,
+    slice,
+    slice::SliceIndex,
+};
+
+/// Create a [`Vec`] containing the arguments.
+///
+/// # Examples
+///
+/// ```
+/// let mut v = kernel::kvec![];
+/// v.push(1, GFP_KERNEL)?;
+/// assert_eq!(v, [1]);
+///
+/// let mut v = kernel::kvec![1; 3]?;
+/// v.push(4, GFP_KERNEL)?;
+/// assert_eq!(v, [1, 1, 1, 4]);
+///
+/// let mut v = kernel::kvec![1, 2, 3]?;
+/// v.push(4, GFP_KERNEL)?;
+/// assert_eq!(v, [1, 2, 3, 4]);
+///
+/// # Ok::<(), Error>(())
+/// ```
+#[macro_export]
+macro_rules! kvec {
+    () => (
+        {
+            $crate::alloc::KVec::new()
+        }
+    );
+    ($elem:expr; $n:expr) => (
+        {
+            $crate::alloc::KVec::from_elem($elem, $n, GFP_KERNEL)
+        }
+    );
+    ($($x:expr),+ $(,)?) => (
+        {
+            match $crate::alloc::KBox::new_uninit(GFP_KERNEL) {
+                Ok(b) => Ok($crate::alloc::KVec::from($crate::alloc::KBox::write(b, [$($x),+]))),
+                Err(e) => Err(e),
+            }
+        }
+    );
+}
+
+/// The kernel's [`Vec`] type.
+///
+/// A contiguous growable array type with contents allocated with the kernel's allocators (e.g.
+/// `Kmalloc`, `Vmalloc` or `KVmalloc`), written `Vec<T, A>`.
+///
+/// For non-zero-sized values, a [`Vec`] will use the given allocator `A` for its allocation. For
+/// the most common allocators the type aliases `KVec`, `VVec` and `KVVec` exist.
+///
+/// For zero-sized types the [`Vec`]'s pointer must be `dangling_mut::<T>`; no memory is allocated.
+///
+/// Generally, [`Vec`] consists of a pointer that represents the vector's backing buffer, the
+/// capacity of the vector (the number of elements that currently fit into the vector), it's length
+/// (the number of elements that are currently stored in the vector) and the `Allocator` type used
+/// to allocate (and free) the backing buffer.
+///
+/// A [`Vec`] can be deconstructed into and (re-)constructed from it's previously named raw parts
+/// and manually modified.
+///
+/// [`Vec`]'s backing buffer gets, if required, automatically increased (re-allocated) when elements
+/// are added to the vector.
+///
+/// # Invariants
+///
+/// The [`Vec`] backing buffer's pointer is always properly aligned and either points to memory
+/// allocated with `A` or, for zero-sized types, is a dangling pointer.
+///
+/// The length of the vector always represents the exact number of elements stored in the vector.
+///
+/// The capacity of the vector always represents the absolute number of elements that can be stored
+/// within the vector without re-allocation. However, it is legal for the backing buffer to be
+/// larger than `size_of<T>` times the capacity.
+///
+/// The `Allocator` type `A` of the vector is the exact same `Allocator` type the backing buffer was
+/// allocated with (and must be freed with).
+pub struct Vec<T, A: Allocator> {
+    ptr: NonNull<T>,
+    /// Represents the actual buffer size as `cap` times `size_of::<T>` bytes.
+    ///
+    /// Note: This isn't quite the same as `Self::capacity`, which in contrast returns the number of
+    /// elements we can still store without reallocating.
+    ///
+    /// # Invariants
+    ///
+    /// `cap` must be in the `0..=isize::MAX` range.
+    cap: usize,
+    len: usize,
+    _p: PhantomData<A>,
+}
+
+/// Type alias for `Vec` with a `Kmalloc` allocator.
+///
+/// # Examples
+///
+/// ```
+/// let mut v = KVec::new();
+/// v.push(1, GFP_KERNEL)?;
+/// assert_eq!(&v, &[1]);
+///
+/// # Ok::<(), Error>(())
+/// ```
+pub type KVec<T> = Vec<T, super::allocator::Kmalloc>;
+
+/// Type alias for `Vec` with a `Vmalloc` allocator.
+///
+/// # Examples
+///
+/// ```
+/// let mut v = VVec::new();
+/// v.push(1, GFP_KERNEL)?;
+/// assert_eq!(&v, &[1]);
+///
+/// # Ok::<(), Error>(())
+/// ```
+pub type VVec<T> = Vec<T, super::allocator::Vmalloc>;
+
+/// Type alias for `Vec` with a `KVmalloc` allocator.
+///
+/// # Examples
+///
+/// ```
+/// let mut v = KVVec::new();
+/// v.push(1, GFP_KERNEL)?;
+/// assert_eq!(&v, &[1]);
+///
+/// # Ok::<(), Error>(())
+/// ```
+pub type KVVec<T> = Vec<T, super::allocator::KVmalloc>;
+
+// SAFETY: `Vec` is `Send` if `T` is `Send` because `Vec` owns its elements.
+unsafe impl<T, A> Send for Vec<T, A>
+where
+    T: Send,
+    A: Allocator,
+{
+}
+
+// SAFETY: `Vec` is `Sync` if `T` is `Sync` because `Vec` owns its elements.
+unsafe impl<T, A> Sync for Vec<T, A>
+where
+    T: Sync,
+    A: Allocator,
+{
+}
+
+impl<T, A> Vec<T, A>
+where
+    A: Allocator,
+{
+    #[inline]
+    fn is_zst() -> bool {
+        core::mem::size_of::<T>() == 0
+    }
+
+    /// Returns the number of elements that can be stored within the vector without allocating
+    /// additional memory.
+    pub fn capacity(&self) -> usize {
+        if Self::is_zst() {
+            usize::MAX
+        } else {
+            self.cap
+        }
+    }
+
+    /// Returns the number of elements stored within the vector.
+    #[inline]
+    pub fn len(&self) -> usize {
+        self.len
+    }
+
+    /// Forcefully sets `self.len` to `new_len`.
+    ///
+    /// # Safety
+    ///
+    /// - `new_len` must be less than or equal to [`Self::capacity`].
+    /// - If `new_len` is greater than `self.len`, all elements within the interval
+    ///   [`self.len`,`new_len`] must be initialized.
+    #[inline]
+    pub unsafe fn set_len(&mut self, new_len: usize) {
+        self.len = new_len;
+    }
+
+    /// Returns a slice of the entire vector.
+    ///
+    /// Equivalent to `&s[..]`.
+    #[inline]
+    pub fn as_slice(&self) -> &[T] {
+        self
+    }
+
+    /// Returns a mutable slice of the entire vector.
+    ///
+    /// Equivalent to `&mut s[..]`.
+    #[inline]
+    pub fn as_mut_slice(&mut self) -> &mut [T] {
+        self
+    }
+
+    /// Returns a mutable raw pointer to the vector's backing buffer, or, if `T` is a ZST, a
+    /// dangling raw pointer.
+    #[inline]
+    pub fn as_mut_ptr(&self) -> *mut T {
+        self.ptr.as_ptr()
+    }
+
+    /// Returns a raw pointer to the vector's backing buffer, or, if `T` is a ZST, a dangling raw
+    /// pointer.
+    #[inline]
+    pub fn as_ptr(&self) -> *const T {
+        self.as_mut_ptr()
+    }
+
+    /// Returns `true` if the vector contains no elements, `false` otherwise.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let mut v = KVec::new();
+    /// assert!(v.is_empty());
+    ///
+    /// v.push(1, GFP_KERNEL);
+    /// assert!(!v.is_empty());
+    /// ```
+    #[inline]
+    pub fn is_empty(&self) -> bool {
+        self.len() == 0
+    }
+
+    /// Creates a new, empty Vec<T, A>.
+    ///
+    /// This method does not allocate by itself.
+    #[inline]
+    pub const fn new() -> Self {
+        Self {
+            ptr: NonNull::dangling(),
+            cap: 0,
+            len: 0,
+            _p: PhantomData::<A>,
+        }
+    }
+
+    /// Returns a slice of `MaybeUninit<T>` for the remaining spare capacity of the vector.
+    pub fn spare_capacity_mut(&mut self) -> &mut [MaybeUninit<T>] {
+        // SAFETY: The memory between `self.len` and `self.capacity` is guaranteed to be allocated
+        // and valid, but uninitialized.
+        unsafe {
+            slice::from_raw_parts_mut(
+                self.as_mut_ptr().add(self.len) as *mut MaybeUninit<T>,
+                self.capacity() - self.len,
+            )
+        }
+    }
+
+    /// Appends an element to the back of the [`Vec`] instance.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let mut v = KVec::new();
+    /// v.push(1, GFP_KERNEL)?;
+    /// assert_eq!(&v, &[1]);
+    ///
+    /// v.push(2, GFP_KERNEL)?;
+    /// assert_eq!(&v, &[1, 2]);
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub fn push(&mut self, v: T, flags: Flags) -> Result<(), AllocError> {
+        Vec::reserve(self, 1, flags)?;
+        let s = self.spare_capacity_mut();
+        s[0].write(v);
+
+        // SAFETY: We just initialised the first spare entry, so it is safe to increase the length
+        // by 1. We also know that the new length is <= capacity because of the previous call to
+        // `reserve` above.
+        unsafe { self.set_len(self.len() + 1) };
+        Ok(())
+    }
+
+    /// Creates a new [`Vec`] instance with at least the given capacity.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let v = KVec::<u32>::with_capacity(20, GFP_KERNEL)?;
+    ///
+    /// assert!(v.capacity() >= 20);
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub fn with_capacity(capacity: usize, flags: Flags) -> Result<Self, AllocError> {
+        let mut v = Vec::new();
+
+        Self::reserve(&mut v, capacity, flags)?;
+
+        Ok(v)
+    }
+
+    /// Pushes clones of the elements of slice into the [`Vec`] instance.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let mut v = KVec::new();
+    /// v.push(1, GFP_KERNEL)?;
+    ///
+    /// v.extend_from_slice(&[20, 30, 40], GFP_KERNEL)?;
+    /// assert_eq!(&v, &[1, 20, 30, 40]);
+    ///
+    /// v.extend_from_slice(&[50, 60], GFP_KERNEL)?;
+    /// assert_eq!(&v, &[1, 20, 30, 40, 50, 60]);
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub fn extend_from_slice(&mut self, other: &[T], flags: Flags) -> Result<(), AllocError>
+    where
+        T: Clone,
+    {
+        self.reserve(other.len(), flags)?;
+        for (slot, item) in core::iter::zip(self.spare_capacity_mut(), other) {
+            slot.write(item.clone());
+        }
+
+        // SAFETY: We just initialised the `other.len()` spare entries, so it is safe to increase
+        // the length by the same amount. We also know that the new length is <= capacity because
+        // of the previous call to `reserve` above.
+        unsafe { self.set_len(self.len() + other.len()) };
+        Ok(())
+    }
+
+    /// Creates a Vec<T, A> from a pointer, a length and a capacity using the allocator `A`.
+    ///
+    /// # Safety
+    ///
+    /// If `T` is a ZST:
+    ///
+    /// - `ptr` must be a dangling pointer.
+    /// - `capacity` must be zero.
+    /// - `length` must be smaller than or equal to `usize::MAX`.
+    ///
+    /// Otherwise:
+    ///
+    /// - `ptr` must have been allocated with the allocator `A`.
+    /// - `ptr` must satisfy or exceed the alignment requirements of `T`.
+    /// - `ptr` must point to memory with a size of at least `size_of::<T>` times the `capacity`
+    ///    bytes.
+    /// - The allocated size in bytes must not be larger than `isize::MAX`.
+    /// - `length` must be less than or equal to `capacity`.
+    /// - The first `length` elements must be initialized values of type `T`.
+    ///
+    /// It is also valid to create an empty `Vec` passing a dangling pointer for `ptr` and zero for
+    /// `cap` and `len`.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let mut v = kernel::kvec![1, 2, 3]?;
+    /// v.reserve(1, GFP_KERNEL)?;
+    ///
+    /// let (mut ptr, mut len, cap) = v.into_raw_parts();
+    ///
+    /// // SAFETY: We've just reserved memory for another element.
+    /// unsafe { ptr.add(len).write(4) };
+    /// len += 1;
+    ///
+    /// // SAFETY: We only wrote an additional element at the end of the `KVec`'s buffer and
+    /// // correspondingly increased the length of the `KVec` by one. Otherwise, we construct it
+    /// // from the exact same raw parts.
+    /// let v = unsafe { KVec::from_raw_parts(ptr, len, cap) };
+    ///
+    /// assert_eq!(v, [1, 2, 3, 4]);
+    ///
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub unsafe fn from_raw_parts(ptr: *mut T, length: usize, capacity: usize) -> Self {
+        let cap = if Self::is_zst() { 0 } else { capacity };
+
+        Self {
+            // SAFETY: By the safety requirements, `ptr` is either dangling or pointing to a valid
+            // memory allocation, allocated with `A`.
+            ptr: unsafe { NonNull::new_unchecked(ptr) },
+            cap,
+            len: length,
+            _p: PhantomData::<A>,
+        }
+    }
+
+    /// Consumes the `Vec<T, A>` and returns its raw components `pointer`, `length` and `capacity`.
+    ///
+    /// This will not run the destructor of the contained elements and for non-ZSTs the allocation
+    /// will stay alive indefinitely. Use [`Vec::from_raw_parts`] to recover the [`Vec`], drop the
+    /// elements and free the allocation, if any.
+    pub fn into_raw_parts(self) -> (*mut T, usize, usize) {
+        let me = ManuallyDrop::new(self);
+        let len = me.len();
+        let capacity = me.capacity();
+        let ptr = me.as_mut_ptr();
+        (ptr, len, capacity)
+    }
+
+    /// Ensures that the capacity exceeds the length by at least `additional`
+    /// elements.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let mut v = KVec::new();
+    /// v.push(1, GFP_KERNEL)?;
+    ///
+    /// v.reserve(10, GFP_KERNEL)?;
+    /// let cap = v.capacity();
+    /// assert!(cap >= 10);
+    ///
+    /// v.reserve(10, GFP_KERNEL)?;
+    /// let new_cap = v.capacity();
+    /// assert_eq!(new_cap, cap);
+    ///
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub fn reserve(&mut self, additional: usize, flags: Flags) -> Result<(), AllocError> {
+        let len = self.len();
+        let cap = self.capacity();
+
+        if cap - len >= additional {
+            return Ok(());
+        }
+
+        if Self::is_zst() {
+            // The capacity is already `usize::MAX` for SZTs, we can't go higher.
+            return Err(AllocError);
+        }
+
+        // We know `cap` is <= `isize::MAX` because of it's type invariant. So the multiplication by
+        // two won't overflow.
+        let new_cap = core::cmp::max(cap * 2, len.checked_add(additional).ok_or(AllocError)?);
+        let layout = core::alloc::Layout::array::<T>(new_cap).map_err(|_| AllocError)?;
+
+        // We need to make sure that `ptr` is either NULL or comes from a previous call to
+        // `realloc_flags`. A `Vec<T, A>`'s `ptr` value is not guaranteed to be NULL and might be
+        // dangling after being created with `Vec::new`. Instead, we can rely on `Vec<T, A>`'s
+        // capacity to be zero if no memory has been allocated yet.
+        let ptr = if cap == 0 {
+            None
+        } else {
+            Some(self.ptr.cast())
+        };
+
+        // SAFETY: `ptr` is valid because it's either `None` or comes from a previous call to
+        // `A::realloc`. We also verified that the type is not a ZST.
+        let ptr = unsafe { A::realloc(ptr, layout, flags)? };
+
+        self.ptr = ptr.cast();
+
+        // INVARIANT: `Layout::array` fails if the resulting byte size is greater than `isize::MAX`.
+        self.cap = new_cap;
+
+        Ok(())
+    }
+}
+
+impl<T: Clone, A: Allocator> Vec<T, A> {
+    /// Extend the vector by `n` clones of value.
+    pub fn extend_with(&mut self, n: usize, value: T, flags: Flags) -> Result<(), AllocError> {
+        if n == 0 {
+            return Ok(());
+        }
+
+        self.reserve(n, flags)?;
+
+        let spare = self.spare_capacity_mut();
+
+        for item in spare.iter_mut().take(n - 1) {
+            item.write(value.clone());
+        }
+
+        // We can write the last element directly without cloning needlessly.
+        spare[n - 1].write(value);
+
+        // SAFETY: `self.reserve` not bailing out with an error guarantees that we're not
+        // exceeding the capacity of this `Vec`.
+        unsafe { self.set_len(self.len() + n) };
+
+        Ok(())
+    }
+
+    /// Create a new `Vec<T, A> and extend it by `n` clones of `value`.
+    pub fn from_elem(value: T, n: usize, flags: Flags) -> Result<Self, AllocError> {
+        let mut v = Self::with_capacity(n, flags)?;
+
+        v.extend_with(n, value, flags)?;
+
+        Ok(v)
+    }
+}
+
+impl<T, A> Drop for Vec<T, A>
+where
+    A: Allocator,
+{
+    fn drop(&mut self) {
+        // SAFETY: We need to drop the vector's elements in place, before we free the backing
+        // memory.
+        unsafe {
+            core::ptr::drop_in_place(core::ptr::slice_from_raw_parts_mut(
+                self.as_mut_ptr(),
+                self.len,
+            ))
+        };
+
+        // If `cap == 0` we never allocated any memory in the first place.
+        if self.cap != 0 {
+            // SAFETY: `self.ptr` was previously allocated with `A`.
+            unsafe { A::free(self.ptr.cast()) };
+        }
+    }
+}
+
+impl<T, A, const N: usize> From<Box<[T; N], A>> for Vec<T, A>
+where
+    A: Allocator,
+{
+    fn from(b: Box<[T; N], A>) -> Vec<T, A> {
+        let len = b.len();
+        let ptr = Box::into_raw(b);
+
+        unsafe { Vec::from_raw_parts(ptr as _, len, len) }
+    }
+}
+
+impl<T> Default for KVec<T> {
+    #[inline]
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl<T: fmt::Debug, A: Allocator> fmt::Debug for Vec<T, A> {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        fmt::Debug::fmt(&**self, f)
+    }
+}
+
+impl<T, A> Deref for Vec<T, A>
+where
+    A: Allocator,
+{
+    type Target = [T];
+
+    #[inline]
+    fn deref(&self) -> &[T] {
+        // SAFETY: The memory behind `self.as_ptr()` is guaranteed to contain `self.len`
+        // initialized elements of type `T`.
+        unsafe { slice::from_raw_parts(self.as_ptr(), self.len) }
+    }
+}
+
+impl<T, A> DerefMut for Vec<T, A>
+where
+    A: Allocator,
+{
+    #[inline]
+    fn deref_mut(&mut self) -> &mut [T] {
+        // SAFETY: The memory behind `self.as_ptr()` is guaranteed to contain `self.len`
+        // initialized elements of type `T`.
+        unsafe { slice::from_raw_parts_mut(self.as_mut_ptr(), self.len) }
+    }
+}
+
+impl<T: Eq, A> Eq for Vec<T, A> where A: Allocator {}
+
+impl<T, I: SliceIndex<[T]>, A> Index<I> for Vec<T, A>
+where
+    A: Allocator,
+{
+    type Output = I::Output;
+
+    #[inline]
+    fn index(&self, index: I) -> &Self::Output {
+        Index::index(&**self, index)
+    }
+}
+
+impl<T, I: SliceIndex<[T]>, A> IndexMut<I> for Vec<T, A>
+where
+    A: Allocator,
+{
+    #[inline]
+    fn index_mut(&mut self, index: I) -> &mut Self::Output {
+        IndexMut::index_mut(&mut **self, index)
+    }
+}
+
+macro_rules! __impl_slice_eq {
+    ([$($vars:tt)*] $lhs:ty, $rhs:ty $(where $ty:ty: $bound:ident)?) => {
+        impl<T, U, $($vars)*> PartialEq<$rhs> for $lhs
+        where
+            T: PartialEq<U>,
+            $($ty: $bound)?
+        {
+            #[inline]
+            fn eq(&self, other: &$rhs) -> bool { self[..] == other[..] }
+        }
+    }
+}
+
+__impl_slice_eq! { [A1: Allocator, A2: Allocator] Vec<T, A1>, Vec<U, A2> }
+__impl_slice_eq! { [A: Allocator] Vec<T, A>, &[U] }
+__impl_slice_eq! { [A: Allocator] Vec<T, A>, &mut [U] }
+__impl_slice_eq! { [A: Allocator] &[T], Vec<U, A> }
+__impl_slice_eq! { [A: Allocator] &mut [T], Vec<U, A> }
+__impl_slice_eq! { [A: Allocator] Vec<T, A>, [U] }
+__impl_slice_eq! { [A: Allocator] [T], Vec<U, A> }
+__impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, [U; N] }
+__impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, &[U; N] }
diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
index d5f2fe42d093..80223cdaa485 100644
--- a/rust/kernel/prelude.rs
+++ b/rust/kernel/prelude.rs
@@ -14,7 +14,7 @@
 #[doc(no_inline)]
 pub use core::pin::Pin;
 
-pub use crate::alloc::{flags::*, vec_ext::VecExt, Box, KBox, KVBox, VBox};
+pub use crate::alloc::{flags::*, vec_ext::VecExt, Box, KBox, KVBox, KVVec, KVec, VBox, VVec};
 
 #[doc(no_inline)]
 pub use alloc::vec::Vec;
-- 
2.46.0

Re: [PATCH v6 13/26] rust: alloc: implement kernel `Vec` type

[Boqun Feng]

On Fri, Aug 16, 2024 at 02:10:55AM +0200, Danilo Krummrich wrote:
> `Vec` provides a contiguous growable array type (such as `Vec`) with
> contents allocated with the kernel's allocators (e.g. `Kmalloc`,
> `Vmalloc` or `KVmalloc`).
> 
> In contrast to Rust's `Vec` type, the kernel `Vec` type considers the
> kernel's GFP flags for all appropriate functions, always reports
> allocation failures through `Result<_, AllocError>` and remains
> independent from unstable features.
> 
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
[...]
> +
> +impl<T, A> Vec<T, A>
> +where
> +    A: Allocator,
> +{
[...]
> +    /// Forcefully sets `self.len` to `new_len`.
> +    ///
> +    /// # Safety
> +    ///
> +    /// - `new_len` must be less than or equal to [`Self::capacity`].
> +    /// - If `new_len` is greater than `self.len`, all elements within the interval
> +    ///   [`self.len`,`new_len`] must be initialized.

Maybe use "[`self.len`, `new_len`)" to indicate `new_len` side is open?
Also `self.len` may confuse people whether it's the old length or new
length, could you use `old_len` and add note saying "`old_len` is the
length before `set_len()`?

> +    #[inline]
> +    pub unsafe fn set_len(&mut self, new_len: usize) {
> +        self.len = new_len;
> +    }
> +
> +    /// Returns a slice of the entire vector.
> +    ///
> +    /// Equivalent to `&s[..]`.

"s" is used here without pre-definition, it's a bit confusing. Either we
use an example here, or we can just avoid mentioning it at all?

> +    #[inline]
> +    pub fn as_slice(&self) -> &[T] {
> +        self
> +    }
> +
> +    /// Returns a mutable slice of the entire vector.
> +    ///
> +    /// Equivalent to `&mut s[..]`.

Ditto.

> +    #[inline]
> +    pub fn as_mut_slice(&mut self) -> &mut [T] {
> +        self
> +    }
> +
> +    /// Returns a mutable raw pointer to the vector's backing buffer, or, if `T` is a ZST, a
> +    /// dangling raw pointer.
> +    #[inline]
> +    pub fn as_mut_ptr(&self) -> *mut T {

I think this function needs to take a `&mut self` to mirror the
`as_mut_ptr` of a slice.

Regards,
Boqun

> +        self.ptr.as_ptr()
> +    }
> +
> +    /// Returns a raw pointer to the vector's backing buffer, or, if `T` is a ZST, a dangling raw
> +    /// pointer.
> +    #[inline]
> +    pub fn as_ptr(&self) -> *const T {
> +        self.as_mut_ptr()
> +    }
> +
[...]

Re: [PATCH v6 13/26] rust: alloc: implement kernel `Vec` type

[Danilo Krummrich]

On Tue, Sep 03, 2024 at 12:08:08PM -0700, Boqun Feng wrote:
> On Fri, Aug 16, 2024 at 02:10:55AM +0200, Danilo Krummrich wrote:
> > `Vec` provides a contiguous growable array type (such as `Vec`) with
> > contents allocated with the kernel's allocators (e.g. `Kmalloc`,
> > `Vmalloc` or `KVmalloc`).
> > 
> > In contrast to Rust's `Vec` type, the kernel `Vec` type considers the
> > kernel's GFP flags for all appropriate functions, always reports
> > allocation failures through `Result<_, AllocError>` and remains
> > independent from unstable features.
> > 
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > ---
> [...]
> > +
> > +impl<T, A> Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> [...]
> > +    /// Forcefully sets `self.len` to `new_len`.
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// - `new_len` must be less than or equal to [`Self::capacity`].
> > +    /// - If `new_len` is greater than `self.len`, all elements within the interval
> > +    ///   [`self.len`,`new_len`] must be initialized.
> 
> Maybe use "[`self.len`, `new_len`)" to indicate `new_len` side is open?

Agreed.

> Also `self.len` may confuse people whether it's the old length or new
> length, could you use `old_len` and add note saying "`old_len` is the
> length before `set_len()`?

What about:

   /// - If `new_len` is greater than `self.len` (from before calling this function), all elements
   ///   within the interval [`self.len`,`new_len`] must be initialized.

Re: [PATCH v6 13/26] rust: alloc: implement kernel `Vec` type

[Benno Lossin]

On 10.09.24 20:26, Danilo Krummrich wrote:
> On Tue, Sep 03, 2024 at 12:08:08PM -0700, Boqun Feng wrote:
>> On Fri, Aug 16, 2024 at 02:10:55AM +0200, Danilo Krummrich wrote:
>>> `Vec` provides a contiguous growable array type (such as `Vec`) with
>>> contents allocated with the kernel's allocators (e.g. `Kmalloc`,
>>> `Vmalloc` or `KVmalloc`).
>>>
>>> In contrast to Rust's `Vec` type, the kernel `Vec` type considers the
>>> kernel's GFP flags for all appropriate functions, always reports
>>> allocation failures through `Result<_, AllocError>` and remains
>>> independent from unstable features.
>>>
>>> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
>>> ---
>> [...]
>>> +
>>> +impl<T, A> Vec<T, A>
>>> +where
>>> +    A: Allocator,
>>> +{
>> [...]
>>> +    /// Forcefully sets `self.len` to `new_len`.
>>> +    ///
>>> +    /// # Safety
>>> +    ///
>>> +    /// - `new_len` must be less than or equal to [`Self::capacity`].
>>> +    /// - If `new_len` is greater than `self.len`, all elements within the interval
>>> +    ///   [`self.len`,`new_len`] must be initialized.
>>
>> Maybe use "[`self.len`, `new_len`)" to indicate `new_len` side is open?
> 
> Agreed.

Alternatively just use the rust range operator `self.len..new_len`, it
is exclusive by default (for an inclusive range you can use `..=`).

>> Also `self.len` may confuse people whether it's the old length or new
>> length, could you use `old_len` and add note saying "`old_len` is the
>> length before `set_len()`?

I personally think this is not a big issue, since safety requirements
are preconditions to calling a function (so the function couldn't have
been called yet).

> What about:
> 
>    /// - If `new_len` is greater than `self.len` (from before calling this function), all elements
>    ///   within the interval [`self.len`,`new_len`] must be initialized.

You will still apply the range fix, right?

---
Cheers,
Benno

Re: [PATCH v6 13/26] rust: alloc: implement kernel `Vec` type

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> `Vec` provides a contiguous growable array type (such as `Vec`) with

"`Vec` [...] such as `Vec`"? What do you want to say? I think without
the parenthesis it works better.

> contents allocated with the kernel's allocators (e.g. `Kmalloc`,
> `Vmalloc` or `KVmalloc`).
> 
> In contrast to Rust's `Vec` type, the kernel `Vec` type considers the

I would add "stdlib" after "Rust's".

> kernel's GFP flags for all appropriate functions, always reports
> allocation failures through `Result<_, AllocError>` and remains
> independent from unstable features.
> 
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
>  rust/kernel/alloc.rs      |   6 +
>  rust/kernel/alloc/kvec.rs | 629 ++++++++++++++++++++++++++++++++++++++
>  rust/kernel/prelude.rs    |   2 +-
>  3 files changed, 636 insertions(+), 1 deletion(-)
>  create mode 100644 rust/kernel/alloc/kvec.rs
> 
> diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> index d248390345ec..e88c7e10ee9b 100644
> --- a/rust/kernel/alloc.rs
> +++ b/rust/kernel/alloc.rs
> @@ -5,6 +5,7 @@
>  #[cfg(not(any(test, testlib)))]
>  pub mod allocator;
>  pub mod kbox;
> +pub mod kvec;
>  pub mod vec_ext;
> 
>  #[cfg(any(test, testlib))]
> @@ -18,6 +19,11 @@
>  pub use self::kbox::KVBox;
>  pub use self::kbox::VBox;
> 
> +pub use self::kvec::KVVec;
> +pub use self::kvec::KVec;
> +pub use self::kvec::VVec;
> +pub use self::kvec::Vec;
> +
>  /// Indicates an allocation error.
>  #[derive(Copy, Clone, PartialEq, Eq, Debug)]
>  pub struct AllocError;
> diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
> new file mode 100644
> index 000000000000..89afc0f25bd4
> --- /dev/null
> +++ b/rust/kernel/alloc/kvec.rs
> @@ -0,0 +1,629 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! Implementation of [`Vec`].
> +
> +use super::{AllocError, Allocator, Box, Flags};
> +use core::{
> +    fmt,
> +    marker::PhantomData,
> +    mem::{ManuallyDrop, MaybeUninit},
> +    ops::Deref,
> +    ops::DerefMut,
> +    ops::Index,
> +    ops::IndexMut,
> +    ptr::NonNull,
> +    slice,
> +    slice::SliceIndex,
> +};
> +
> +/// Create a [`Vec`] containing the arguments.
> +///
> +/// # Examples
> +///
> +/// ```
> +/// let mut v = kernel::kvec![];
> +/// v.push(1, GFP_KERNEL)?;
> +/// assert_eq!(v, [1]);
> +///
> +/// let mut v = kernel::kvec![1; 3]?;
> +/// v.push(4, GFP_KERNEL)?;
> +/// assert_eq!(v, [1, 1, 1, 4]);
> +///
> +/// let mut v = kernel::kvec![1, 2, 3]?;
> +/// v.push(4, GFP_KERNEL)?;
> +/// assert_eq!(v, [1, 2, 3, 4]);
> +///
> +/// # Ok::<(), Error>(())
> +/// ```
> +#[macro_export]
> +macro_rules! kvec {

Do we only want to support creating `KVec` via a macro?

> +    () => (
> +        {
> +            $crate::alloc::KVec::new()
> +        }
> +    );

Why do you have parenthesis and braces? IIRC just braces/parenthesis or
brackets should be enough.

> +    ($elem:expr; $n:expr) => (
> +        {
> +            $crate::alloc::KVec::from_elem($elem, $n, GFP_KERNEL)

Hmm, could it be that one might want to use `kvec!` without
`GFP_KERNEL`?
Or add additional flags, eg __GFP_ZERO?

If you think that supporting this is not immediately necessary or if
this is too much for this patchset, then we can also postpone it (maybe
it could be a good-first-issue). Do you keep a list of future
improvements for the new allocator API somewhere? If not, then I think
we should create one (best place would be the issue tracker on GH).

> +        }
> +    );
> +    ($($x:expr),+ $(,)?) => (
> +        {
> +            match $crate::alloc::KBox::new_uninit(GFP_KERNEL) {
> +                Ok(b) => Ok($crate::alloc::KVec::from($crate::alloc::KBox::write(b, [$($x),+]))),
> +                Err(e) => Err(e),
> +            }
> +        }
> +    );
> +}
> +
> +/// The kernel's [`Vec`] type.
> +///
> +/// A contiguous growable array type with contents allocated with the kernel's allocators (e.g.
> +/// `Kmalloc`, `Vmalloc` or `KVmalloc`), written `Vec<T, A>`.

New folks might get confused as to which allocator they should choose.
Do we have a sensible default for `Vec`?
If yes, then we at least should document that here. We might also want
to make it the default value for the generic parameter.
This is also a good idea for `Box`.

> +///
> +/// For non-zero-sized values, a [`Vec`] will use the given allocator `A` for its allocation. For
> +/// the most common allocators the type aliases `KVec`, `VVec` and `KVVec` exist.
> +///
> +/// For zero-sized types the [`Vec`]'s pointer must be `dangling_mut::<T>`; no memory is allocated.
> +///
> +/// Generally, [`Vec`] consists of a pointer that represents the vector's backing buffer, the
> +/// capacity of the vector (the number of elements that currently fit into the vector), it's length
> +/// (the number of elements that are currently stored in the vector) and the `Allocator` type used
> +/// to allocate (and free) the backing buffer.
> +///
> +/// A [`Vec`] can be deconstructed into and (re-)constructed from it's previously named raw parts
> +/// and manually modified.
> +///
> +/// [`Vec`]'s backing buffer gets, if required, automatically increased (re-allocated) when elements
> +/// are added to the vector.
> +///
> +/// # Invariants
> +///
> +/// The [`Vec`] backing buffer's pointer is always properly aligned and either points to memory
> +/// allocated with `A` or, for zero-sized types, is a dangling pointer.

Just use `self.ptr` instead of "backing buffer".

> +///
> +/// The length of the vector always represents the exact number of elements stored in the vector.

Same here, `self.len`.

> +///
> +/// The capacity of the vector always represents the absolute number of elements that can be stored

Ditto.

> +/// within the vector without re-allocation. However, it is legal for the backing buffer to be
> +/// larger than `size_of<T>` times the capacity.
> +///
> +/// The `Allocator` type `A` of the vector is the exact same `Allocator` type the backing buffer was
> +/// allocated with (and must be freed with).

Please turn this into a bullet-point list.

> +pub struct Vec<T, A: Allocator> {
> +    ptr: NonNull<T>,
> +    /// Represents the actual buffer size as `cap` times `size_of::<T>` bytes.
> +    ///
> +    /// Note: This isn't quite the same as `Self::capacity`, which in contrast returns the number of
> +    /// elements we can still store without reallocating.
> +    ///
> +    /// # Invariants
> +    ///
> +    /// `cap` must be in the `0..=isize::MAX` range.
> +    cap: usize,
> +    len: usize,
> +    _p: PhantomData<A>,
> +}
> +
> +/// Type alias for `Vec` with a `Kmalloc` allocator.

Can you turn these into links?

> +///
> +/// # Examples
> +///
> +/// ```
> +/// let mut v = KVec::new();
> +/// v.push(1, GFP_KERNEL)?;
> +/// assert_eq!(&v, &[1]);
> +///
> +/// # Ok::<(), Error>(())
> +/// ```
> +pub type KVec<T> = Vec<T, super::allocator::Kmalloc>;
> +
> +/// Type alias for `Vec` with a `Vmalloc` allocator.
> +///
> +/// # Examples
> +///
> +/// ```
> +/// let mut v = VVec::new();
> +/// v.push(1, GFP_KERNEL)?;
> +/// assert_eq!(&v, &[1]);
> +///
> +/// # Ok::<(), Error>(())
> +/// ```
> +pub type VVec<T> = Vec<T, super::allocator::Vmalloc>;
> +
> +/// Type alias for `Vec` with a `KVmalloc` allocator.
> +///
> +/// # Examples
> +///
> +/// ```
> +/// let mut v = KVVec::new();
> +/// v.push(1, GFP_KERNEL)?;
> +/// assert_eq!(&v, &[1]);
> +///
> +/// # Ok::<(), Error>(())
> +/// ```
> +pub type KVVec<T> = Vec<T, super::allocator::KVmalloc>;
> +
> +// SAFETY: `Vec` is `Send` if `T` is `Send` because `Vec` owns its elements.
> +unsafe impl<T, A> Send for Vec<T, A>
> +where
> +    T: Send,
> +    A: Allocator,
> +{
> +}
> +
> +// SAFETY: `Vec` is `Sync` if `T` is `Sync` because `Vec` owns its elements.
> +unsafe impl<T, A> Sync for Vec<T, A>
> +where
> +    T: Sync,
> +    A: Allocator,
> +{
> +}
> +
> +impl<T, A> Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    #[inline]
> +    fn is_zst() -> bool {
> +        core::mem::size_of::<T>() == 0
> +    }
> +
> +    /// Returns the number of elements that can be stored within the vector without allocating
> +    /// additional memory.
> +    pub fn capacity(&self) -> usize {
> +        if Self::is_zst() {
> +            usize::MAX
> +        } else {
> +            self.cap
> +        }
> +    }
> +
> +    /// Returns the number of elements stored within the vector.
> +    #[inline]
> +    pub fn len(&self) -> usize {
> +        self.len
> +    }
> +
> +    /// Forcefully sets `self.len` to `new_len`.
> +    ///
> +    /// # Safety
> +    ///
> +    /// - `new_len` must be less than or equal to [`Self::capacity`].
> +    /// - If `new_len` is greater than `self.len`, all elements within the interval
> +    ///   [`self.len`,`new_len`] must be initialized.
> +    #[inline]
> +    pub unsafe fn set_len(&mut self, new_len: usize) {

Can you add a `debug_assert!(new_len < self.cap)`

> +        self.len = new_len;
> +    }
> +
> +    /// Returns a slice of the entire vector.
> +    ///
> +    /// Equivalent to `&s[..]`.
> +    #[inline]
> +    pub fn as_slice(&self) -> &[T] {
> +        self
> +    }
> +
> +    /// Returns a mutable slice of the entire vector.
> +    ///
> +    /// Equivalent to `&mut s[..]`.
> +    #[inline]
> +    pub fn as_mut_slice(&mut self) -> &mut [T] {
> +        self
> +    }
> +
> +    /// Returns a mutable raw pointer to the vector's backing buffer, or, if `T` is a ZST, a
> +    /// dangling raw pointer.
> +    #[inline]
> +    pub fn as_mut_ptr(&self) -> *mut T {
> +        self.ptr.as_ptr()
> +    }
> +
> +    /// Returns a raw pointer to the vector's backing buffer, or, if `T` is a ZST, a dangling raw
> +    /// pointer.
> +    #[inline]
> +    pub fn as_ptr(&self) -> *const T {
> +        self.as_mut_ptr()
> +    }
> +
> +    /// Returns `true` if the vector contains no elements, `false` otherwise.
> +    ///
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let mut v = KVec::new();
> +    /// assert!(v.is_empty());
> +    ///
> +    /// v.push(1, GFP_KERNEL);
> +    /// assert!(!v.is_empty());
> +    /// ```
> +    #[inline]
> +    pub fn is_empty(&self) -> bool {
> +        self.len() == 0
> +    }
> +
> +    /// Creates a new, empty Vec<T, A>.
> +    ///
> +    /// This method does not allocate by itself.
> +    #[inline]
> +    pub const fn new() -> Self {
> +        Self {
> +            ptr: NonNull::dangling(),
> +            cap: 0,
> +            len: 0,
> +            _p: PhantomData::<A>,
> +        }
> +    }
> +
> +    /// Returns a slice of `MaybeUninit<T>` for the remaining spare capacity of the vector.

Returns 

> +    pub fn spare_capacity_mut(&mut self) -> &mut [MaybeUninit<T>] {
> +        // SAFETY: The memory between `self.len` and `self.capacity` is guaranteed to be allocated
> +        // and valid, but uninitialized.
> +        unsafe {
> +            slice::from_raw_parts_mut(
> +                self.as_mut_ptr().add(self.len) as *mut MaybeUninit<T>,
> +                self.capacity() - self.len,
> +            )
> +        }

There are two `unsafe` operations in this block, please split it into
two blocks.

> +    }
> +
> +    /// Appends an element to the back of the [`Vec`] instance.
> +    ///
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let mut v = KVec::new();
> +    /// v.push(1, GFP_KERNEL)?;
> +    /// assert_eq!(&v, &[1]);
> +    ///
> +    /// v.push(2, GFP_KERNEL)?;
> +    /// assert_eq!(&v, &[1, 2]);
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    pub fn push(&mut self, v: T, flags: Flags) -> Result<(), AllocError> {
> +        Vec::reserve(self, 1, flags)?;
> +        let s = self.spare_capacity_mut();
> +        s[0].write(v);

The slice access will check the size, I don't know if it will be
optimized out, if not, I would use a raw pointer here instead.

> +
> +        // SAFETY: We just initialised the first spare entry, so it is safe to increase the length
> +        // by 1. We also know that the new length is <= capacity because of the previous call to
> +        // `reserve` above.
> +        unsafe { self.set_len(self.len() + 1) };
> +        Ok(())
> +    }
> +
> +    /// Creates a new [`Vec`] instance with at least the given capacity.
> +    ///
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let v = KVec::<u32>::with_capacity(20, GFP_KERNEL)?;
> +    ///
> +    /// assert!(v.capacity() >= 20);
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    pub fn with_capacity(capacity: usize, flags: Flags) -> Result<Self, AllocError> {
> +        let mut v = Vec::new();
> +
> +        Self::reserve(&mut v, capacity, flags)?;
> +
> +        Ok(v)
> +    }
> +
> +    /// Pushes clones of the elements of slice into the [`Vec`] instance.
> +    ///
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let mut v = KVec::new();
> +    /// v.push(1, GFP_KERNEL)?;
> +    ///
> +    /// v.extend_from_slice(&[20, 30, 40], GFP_KERNEL)?;
> +    /// assert_eq!(&v, &[1, 20, 30, 40]);
> +    ///
> +    /// v.extend_from_slice(&[50, 60], GFP_KERNEL)?;
> +    /// assert_eq!(&v, &[1, 20, 30, 40, 50, 60]);
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    pub fn extend_from_slice(&mut self, other: &[T], flags: Flags) -> Result<(), AllocError>
> +    where
> +        T: Clone,
> +    {
> +        self.reserve(other.len(), flags)?;
> +        for (slot, item) in core::iter::zip(self.spare_capacity_mut(), other) {
> +            slot.write(item.clone());
> +        }
> +
> +        // SAFETY: We just initialised the `other.len()` spare entries, so it is safe to increase
> +        // the length by the same amount. We also know that the new length is <= capacity because
> +        // of the previous call to `reserve` above.

Please use a bullet point list and avoid natural language filler words.

> +        unsafe { self.set_len(self.len() + other.len()) };
> +        Ok(())
> +    }
> +
> +    /// Creates a Vec<T, A> from a pointer, a length and a capacity using the allocator `A`.
> +    ///
> +    /// # Safety
> +    ///
> +    /// If `T` is a ZST:
> +    ///
> +    /// - `ptr` must be a dangling pointer.
> +    /// - `capacity` must be zero.

When `T` is a ZST, `into_raw_parts` returns `usize::MAX` for `capacity`,
so one can't just pipe its output to this function? That seems strange.

Also below you discard the value of `capacity`, when `T` is a ZST, so I
think you can just remove this line.

> +    /// - `length` must be smaller than or equal to `usize::MAX`.

This is always true.

> +    ///
> +    /// Otherwise:
> +    ///
> +    /// - `ptr` must have been allocated with the allocator `A`.
> +    /// - `ptr` must satisfy or exceed the alignment requirements of `T`.
> +    /// - `ptr` must point to memory with a size of at least `size_of::<T>` times the `capacity`
> +    ///    bytes.

Just write "`size_of::<T>() * capacity` bytes".

> +    /// - The allocated size in bytes must not be larger than `isize::MAX`.

Should we make this a safety requirement of `Allocator`? I think this is
generally the maximum size other allocated things can have anyways.

> +    /// - `length` must be less than or equal to `capacity`.
> +    /// - The first `length` elements must be initialized values of type `T`.
> +    ///
> +    /// It is also valid to create an empty `Vec` passing a dangling pointer for `ptr` and zero for
> +    /// `cap` and `len`.

Can you make this last sentence part of the `if` chain that you have
above (ie the one started with "If `T` is a ZST:").


Just to experiment with the suggestion from Kangrejos to use Rust as the
language for safety documentation, here is what it could look like (we
should discuss it more before we start using it):

    /// # Safety
    ///
    /// ```ignore
    /// assert!(ptr.is_aligned_to(align_of::<T>()));
    /// assert!(!ptr.is_null());
    /// assert!(length <= capacity);
    /// if capacity != 0 && size_of::<T>() != 0 {
    ///     assert!(A::did_allocate(ptr));
    ///     assert!(size_of::<T>() * capacity <= A::layout_of(ptr).len());
    ///     assert!(is_initialized(ptr::slice_from_raw_parts(ptr, length)));
    /// }
    /// ```

I really like how this looks! We might want to add labels/names to each
of the conditions and then one could use those in the justifications. A
tool could then read those and match them to the requirements of the
unsafe operation.

> +    ///
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let mut v = kernel::kvec![1, 2, 3]?;
> +    /// v.reserve(1, GFP_KERNEL)?;
> +    ///
> +    /// let (mut ptr, mut len, cap) = v.into_raw_parts();
> +    ///
> +    /// // SAFETY: We've just reserved memory for another element.
> +    /// unsafe { ptr.add(len).write(4) };
> +    /// len += 1;
> +    ///
> +    /// // SAFETY: We only wrote an additional element at the end of the `KVec`'s buffer and
> +    /// // correspondingly increased the length of the `KVec` by one. Otherwise, we construct it
> +    /// // from the exact same raw parts.
> +    /// let v = unsafe { KVec::from_raw_parts(ptr, len, cap) };
> +    ///
> +    /// assert_eq!(v, [1, 2, 3, 4]);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```

Please reorder the sections to be `# Examples` followed by `# Safety`.

> +    pub unsafe fn from_raw_parts(ptr: *mut T, length: usize, capacity: usize) -> Self {
> +        let cap = if Self::is_zst() { 0 } else { capacity };
> +
> +        Self {
> +            // SAFETY: By the safety requirements, `ptr` is either dangling or pointing to a valid
> +            // memory allocation, allocated with `A`.
> +            ptr: unsafe { NonNull::new_unchecked(ptr) },
> +            cap,
> +            len: length,
> +            _p: PhantomData::<A>,
> +        }
> +    }
> +
> +    /// Consumes the `Vec<T, A>` and returns its raw components `pointer`, `length` and `capacity`.
> +    ///
> +    /// This will not run the destructor of the contained elements and for non-ZSTs the allocation
> +    /// will stay alive indefinitely. Use [`Vec::from_raw_parts`] to recover the [`Vec`], drop the
> +    /// elements and free the allocation, if any.
> +    pub fn into_raw_parts(self) -> (*mut T, usize, usize) {
> +        let me = ManuallyDrop::new(self);
> +        let len = me.len();
> +        let capacity = me.capacity();
> +        let ptr = me.as_mut_ptr();
> +        (ptr, len, capacity)
> +    }
> +
> +    /// Ensures that the capacity exceeds the length by at least `additional`
> +    /// elements.
> +    ///
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let mut v = KVec::new();
> +    /// v.push(1, GFP_KERNEL)?;
> +    ///
> +    /// v.reserve(10, GFP_KERNEL)?;
> +    /// let cap = v.capacity();
> +    /// assert!(cap >= 10);
> +    ///
> +    /// v.reserve(10, GFP_KERNEL)?;
> +    /// let new_cap = v.capacity();
> +    /// assert_eq!(new_cap, cap);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    pub fn reserve(&mut self, additional: usize, flags: Flags) -> Result<(), AllocError> {
> +        let len = self.len();
> +        let cap = self.capacity();
> +
> +        if cap - len >= additional {
> +            return Ok(());
> +        }
> +
> +        if Self::is_zst() {
> +            // The capacity is already `usize::MAX` for SZTs, we can't go higher.

Typo: SZTs -> ZSTs

> +            return Err(AllocError);
> +        }
> +
> +        // We know `cap` is <= `isize::MAX` because of it's type invariant. So the multiplication by

"it's type invariant" -> "type invariants of Self"

> +        // two won't overflow.
> +        let new_cap = core::cmp::max(cap * 2, len.checked_add(additional).ok_or(AllocError)?);
> +        let layout = core::alloc::Layout::array::<T>(new_cap).map_err(|_| AllocError)?;
> +
> +        // We need to make sure that `ptr` is either NULL or comes from a previous call to
> +        // `realloc_flags`. A `Vec<T, A>`'s `ptr` value is not guaranteed to be NULL and might be
> +        // dangling after being created with `Vec::new`. Instead, we can rely on `Vec<T, A>`'s
> +        // capacity to be zero if no memory has been allocated yet.
> +        let ptr = if cap == 0 {
> +            None
> +        } else {
> +            Some(self.ptr.cast())
> +        };
> +
> +        // SAFETY: `ptr` is valid because it's either `None` or comes from a previous call to
> +        // `A::realloc`. We also verified that the type is not a ZST.
> +        let ptr = unsafe { A::realloc(ptr, layout, flags)? };
> +
> +        self.ptr = ptr.cast();
> +
> +        // INVARIANT: `Layout::array` fails if the resulting byte size is greater than `isize::MAX`.
> +        self.cap = new_cap;
> +
> +        Ok(())
> +    }
> +}
> +
> +impl<T: Clone, A: Allocator> Vec<T, A> {
> +    /// Extend the vector by `n` clones of value.

value -> `value`

> +    pub fn extend_with(&mut self, n: usize, value: T, flags: Flags) -> Result<(), AllocError> {
> +        if n == 0 {
> +            return Ok(());
> +        }
> +
> +        self.reserve(n, flags)?;
> +
> +        let spare = self.spare_capacity_mut();
> +
> +        for item in spare.iter_mut().take(n - 1) {
> +            item.write(value.clone());
> +        }
> +
> +        // We can write the last element directly without cloning needlessly.
> +        spare[n - 1].write(value);
> +
> +        // SAFETY: `self.reserve` not bailing out with an error guarantees that we're not
> +        // exceeding the capacity of this `Vec`.

This safety comment needs to be improved, how about using bullet points:
- `self.len() + n < capacity` due to the call to reserve above,
- the loop and the line above initialized the next `n` elements.

> +        unsafe { self.set_len(self.len() + n) };
> +
> +        Ok(())
> +    }
> +
> +    /// Create a new `Vec<T, A> and extend it by `n` clones of `value`.
> +    pub fn from_elem(value: T, n: usize, flags: Flags) -> Result<Self, AllocError> {
> +        let mut v = Self::with_capacity(n, flags)?;
> +
> +        v.extend_with(n, value, flags)?;
> +
> +        Ok(v)
> +    }
> +}
> +
> +impl<T, A> Drop for Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    fn drop(&mut self) {
> +        // SAFETY: We need to drop the vector's elements in place, before we free the backing
> +        // memory.
> +        unsafe {
> +            core::ptr::drop_in_place(core::ptr::slice_from_raw_parts_mut(

You are using `core::ptr` a bunch, consider using it, then you don't
need to write the prefix `core::`.

> +                self.as_mut_ptr(),
> +                self.len,
> +            ))
> +        };
> +
> +        // If `cap == 0` we never allocated any memory in the first place.
> +        if self.cap != 0 {
> +            // SAFETY: `self.ptr` was previously allocated with `A`.
> +            unsafe { A::free(self.ptr.cast()) };
> +        }
> +    }
> +}
> +
> +impl<T, A, const N: usize> From<Box<[T; N], A>> for Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    fn from(b: Box<[T; N], A>) -> Vec<T, A> {
> +        let len = b.len();
> +        let ptr = Box::into_raw(b);
> +
> +        unsafe { Vec::from_raw_parts(ptr as _, len, len) }

Missing SAFETY comment.

> +    }
> +}
> +
> +impl<T> Default for KVec<T> {
> +    #[inline]
> +    fn default() -> Self {
> +        Self::new()
> +    }
> +}
> +
> +impl<T: fmt::Debug, A: Allocator> fmt::Debug for Vec<T, A> {
> +    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
> +        fmt::Debug::fmt(&**self, f)
> +    }
> +}
> +
> +impl<T, A> Deref for Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Target = [T];
> +
> +    #[inline]
> +    fn deref(&self) -> &[T] {
> +        // SAFETY: The memory behind `self.as_ptr()` is guaranteed to contain `self.len`
> +        // initialized elements of type `T`.
> +        unsafe { slice::from_raw_parts(self.as_ptr(), self.len) }
> +    }
> +}
> +
> +impl<T, A> DerefMut for Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    #[inline]
> +    fn deref_mut(&mut self) -> &mut [T] {
> +        // SAFETY: The memory behind `self.as_ptr()` is guaranteed to contain `self.len`
> +        // initialized elements of type `T`.
> +        unsafe { slice::from_raw_parts_mut(self.as_mut_ptr(), self.len) }
> +    }
> +}
> +
> +impl<T: Eq, A> Eq for Vec<T, A> where A: Allocator {}
> +
> +impl<T, I: SliceIndex<[T]>, A> Index<I> for Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Output = I::Output;
> +
> +    #[inline]
> +    fn index(&self, index: I) -> &Self::Output {
> +        Index::index(&**self, index)
> +    }
> +}
> +
> +impl<T, I: SliceIndex<[T]>, A> IndexMut<I> for Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    #[inline]
> +    fn index_mut(&mut self, index: I) -> &mut Self::Output {
> +        IndexMut::index_mut(&mut **self, index)
> +    }
> +}
> +
> +macro_rules! __impl_slice_eq {

I wouldn't begin this macro with `__`, it is already private.

> +    ([$($vars:tt)*] $lhs:ty, $rhs:ty $(where $ty:ty: $bound:ident)?) => {

You don't use the optional `where` part in the code below, if you don't
use it later, then remove it (if you do use it later, I would move it
into that patch).

---
Cheers,
Benno

> +        impl<T, U, $($vars)*> PartialEq<$rhs> for $lhs
> +        where
> +            T: PartialEq<U>,
> +            $($ty: $bound)?
> +        {
> +            #[inline]
> +            fn eq(&self, other: &$rhs) -> bool { self[..] == other[..] }
> +        }
> +    }
> +}
> +
> +__impl_slice_eq! { [A1: Allocator, A2: Allocator] Vec<T, A1>, Vec<U, A2> }
> +__impl_slice_eq! { [A: Allocator] Vec<T, A>, &[U] }
> +__impl_slice_eq! { [A: Allocator] Vec<T, A>, &mut [U] }
> +__impl_slice_eq! { [A: Allocator] &[T], Vec<U, A> }
> +__impl_slice_eq! { [A: Allocator] &mut [T], Vec<U, A> }
> +__impl_slice_eq! { [A: Allocator] Vec<T, A>, [U] }
> +__impl_slice_eq! { [A: Allocator] [T], Vec<U, A> }
> +__impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, [U; N] }
> +__impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, &[U; N] }
> diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
> index d5f2fe42d093..80223cdaa485 100644
> --- a/rust/kernel/prelude.rs
> +++ b/rust/kernel/prelude.rs
> @@ -14,7 +14,7 @@
>  #[doc(no_inline)]
>  pub use core::pin::Pin;
> 
> -pub use crate::alloc::{flags::*, vec_ext::VecExt, Box, KBox, KVBox, VBox};
> +pub use crate::alloc::{flags::*, vec_ext::VecExt, Box, KBox, KVBox, KVVec, KVec, VBox, VVec};
> 
>  #[doc(no_inline)]
>  pub use alloc::vec::Vec;
> --
> 2.46.0
> 

Re: [PATCH v6 13/26] rust: alloc: implement kernel `Vec` type

[Danilo Krummrich]

On Tue, Sep 10, 2024 at 07:32:06PM +0000, Benno Lossin wrote:
> On 16.08.24 02:10, Danilo Krummrich wrote:
> > `Vec` provides a contiguous growable array type (such as `Vec`) with
> 
> "`Vec` [...] such as `Vec`"? What do you want to say? I think without
> the parenthesis it works better.
> 
> > contents allocated with the kernel's allocators (e.g. `Kmalloc`,
> > `Vmalloc` or `KVmalloc`).
> > 
> > In contrast to Rust's `Vec` type, the kernel `Vec` type considers the
> 
> I would add "stdlib" after "Rust's".
> 
> > kernel's GFP flags for all appropriate functions, always reports
> > allocation failures through `Result<_, AllocError>` and remains
> > independent from unstable features.
> > 
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > ---
> >  rust/kernel/alloc.rs      |   6 +
> >  rust/kernel/alloc/kvec.rs | 629 ++++++++++++++++++++++++++++++++++++++
> >  rust/kernel/prelude.rs    |   2 +-
> >  3 files changed, 636 insertions(+), 1 deletion(-)
> >  create mode 100644 rust/kernel/alloc/kvec.rs
> > 
> > diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> > index d248390345ec..e88c7e10ee9b 100644
> > --- a/rust/kernel/alloc.rs
> > +++ b/rust/kernel/alloc.rs
> > @@ -5,6 +5,7 @@
> >  #[cfg(not(any(test, testlib)))]
> >  pub mod allocator;
> >  pub mod kbox;
> > +pub mod kvec;
> >  pub mod vec_ext;
> > 
> >  #[cfg(any(test, testlib))]
> > @@ -18,6 +19,11 @@
> >  pub use self::kbox::KVBox;
> >  pub use self::kbox::VBox;
> > 
> > +pub use self::kvec::KVVec;
> > +pub use self::kvec::KVec;
> > +pub use self::kvec::VVec;
> > +pub use self::kvec::Vec;
> > +
> >  /// Indicates an allocation error.
> >  #[derive(Copy, Clone, PartialEq, Eq, Debug)]
> >  pub struct AllocError;
> > diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
> > new file mode 100644
> > index 000000000000..89afc0f25bd4
> > --- /dev/null
> > +++ b/rust/kernel/alloc/kvec.rs
> > @@ -0,0 +1,629 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +//! Implementation of [`Vec`].
> > +
> > +use super::{AllocError, Allocator, Box, Flags};
> > +use core::{
> > +    fmt,
> > +    marker::PhantomData,
> > +    mem::{ManuallyDrop, MaybeUninit},
> > +    ops::Deref,
> > +    ops::DerefMut,
> > +    ops::Index,
> > +    ops::IndexMut,
> > +    ptr::NonNull,
> > +    slice,
> > +    slice::SliceIndex,
> > +};
> > +
> > +/// Create a [`Vec`] containing the arguments.
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// let mut v = kernel::kvec![];
> > +/// v.push(1, GFP_KERNEL)?;
> > +/// assert_eq!(v, [1]);
> > +///
> > +/// let mut v = kernel::kvec![1; 3]?;
> > +/// v.push(4, GFP_KERNEL)?;
> > +/// assert_eq!(v, [1, 1, 1, 4]);
> > +///
> > +/// let mut v = kernel::kvec![1, 2, 3]?;
> > +/// v.push(4, GFP_KERNEL)?;
> > +/// assert_eq!(v, [1, 2, 3, 4]);
> > +///
> > +/// # Ok::<(), Error>(())
> > +/// ```
> > +#[macro_export]
> > +macro_rules! kvec {
> 
> Do we only want to support creating `KVec` via a macro?

Yes, needing such a macro for any other allocator is rather unlikely.

> 
> > +    () => (
> > +        {
> > +            $crate::alloc::KVec::new()
> > +        }
> > +    );
> 
> Why do you have parenthesis and braces? IIRC just braces/parenthesis or
> brackets should be enough.
> 
> > +    ($elem:expr; $n:expr) => (
> > +        {
> > +            $crate::alloc::KVec::from_elem($elem, $n, GFP_KERNEL)
> 
> Hmm, could it be that one might want to use `kvec!` without
> `GFP_KERNEL`?
> Or add additional flags, eg __GFP_ZERO?

Pretty unlikely, I'd say. __GFP_ZERO in particular wouldn't make much sense,
since the macro enforces initialization anyways.

Maybe something like GFP_ATOMIC, but I expect this to be uncommon enough to not
consider this for this macro for now.

> 
> If you think that supporting this is not immediately necessary or if
> this is too much for this patchset, then we can also postpone it (maybe
> it could be a good-first-issue). Do you keep a list of future
> improvements for the new allocator API somewhere? If not, then I think
> we should create one (best place would be the issue tracker on GH).

I would only add it if it turns out to be a common need. As mentioned, I don't
expect it to be.

I'd rather keep issues in the source tree. For instance, DRM has them in
'/Documentation/gpu/todo.rst'.

> 
> > +        }
> > +    );
> > +    ($($x:expr),+ $(,)?) => (
> > +        {
> > +            match $crate::alloc::KBox::new_uninit(GFP_KERNEL) {
> > +                Ok(b) => Ok($crate::alloc::KVec::from($crate::alloc::KBox::write(b, [$($x),+]))),
> > +                Err(e) => Err(e),
> > +            }
> > +        }
> > +    );
> > +}
> > +
> > +/// The kernel's [`Vec`] type.
> > +///
> > +/// A contiguous growable array type with contents allocated with the kernel's allocators (e.g.
> > +/// `Kmalloc`, `Vmalloc` or `KVmalloc`), written `Vec<T, A>`.
> 
> New folks might get confused as to which allocator they should choose.
> Do we have a sensible default for `Vec`?

Then they should read the documentation about the kernel's allocators. We
already link them in rust/kernel/alloc/allocator.rs.

> If yes, then we at least should document that here. We might also want
> to make it the default value for the generic parameter.
> This is also a good idea for `Box`.

If we really want a default it should be `Kmalloc`, but I really think we should
force people to make an explicit decision and think about it and don't just go
with whatever default.

It makes it also easier to grep for things. :)

> 
> > +///
> > +/// For non-zero-sized values, a [`Vec`] will use the given allocator `A` for its allocation. For
> > +/// the most common allocators the type aliases `KVec`, `VVec` and `KVVec` exist.
> > +///
> > +/// For zero-sized types the [`Vec`]'s pointer must be `dangling_mut::<T>`; no memory is allocated.
> > +///
> > +/// Generally, [`Vec`] consists of a pointer that represents the vector's backing buffer, the
> > +/// capacity of the vector (the number of elements that currently fit into the vector), it's length
> > +/// (the number of elements that are currently stored in the vector) and the `Allocator` type used
> > +/// to allocate (and free) the backing buffer.
> > +///
> > +/// A [`Vec`] can be deconstructed into and (re-)constructed from it's previously named raw parts
> > +/// and manually modified.
> > +///
> > +/// [`Vec`]'s backing buffer gets, if required, automatically increased (re-allocated) when elements
> > +/// are added to the vector.
> > +///
> > +/// # Invariants
> > +///
> > +/// The [`Vec`] backing buffer's pointer is always properly aligned and either points to memory
> > +/// allocated with `A` or, for zero-sized types, is a dangling pointer.
> 
> Just use `self.ptr` instead of "backing buffer".
> 
> > +///
> > +/// The length of the vector always represents the exact number of elements stored in the vector.
> 
> Same here, `self.len`.
> 
> > +///
> > +/// The capacity of the vector always represents the absolute number of elements that can be stored
> 
> Ditto.
> 
> > +/// within the vector without re-allocation. However, it is legal for the backing buffer to be
> > +/// larger than `size_of<T>` times the capacity.
> > +///
> > +/// The `Allocator` type `A` of the vector is the exact same `Allocator` type the backing buffer was
> > +/// allocated with (and must be freed with).
> 
> Please turn this into a bullet-point list.

Is this a rule? Do we want to make it one?

> 
> > +pub struct Vec<T, A: Allocator> {
> > +    ptr: NonNull<T>,
> > +    /// Represents the actual buffer size as `cap` times `size_of::<T>` bytes.
> > +    ///
> > +    /// Note: This isn't quite the same as `Self::capacity`, which in contrast returns the number of
> > +    /// elements we can still store without reallocating.
> > +    ///
> > +    /// # Invariants
> > +    ///
> > +    /// `cap` must be in the `0..=isize::MAX` range.
> > +    cap: usize,
> > +    len: usize,
> > +    _p: PhantomData<A>,
> > +}
> > +
> > +/// Type alias for `Vec` with a `Kmalloc` allocator.
> 
> Can you turn these into links?
> 
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// let mut v = KVec::new();
> > +/// v.push(1, GFP_KERNEL)?;
> > +/// assert_eq!(&v, &[1]);
> > +///
> > +/// # Ok::<(), Error>(())
> > +/// ```
> > +pub type KVec<T> = Vec<T, super::allocator::Kmalloc>;
> > +
> > +/// Type alias for `Vec` with a `Vmalloc` allocator.
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// let mut v = VVec::new();
> > +/// v.push(1, GFP_KERNEL)?;
> > +/// assert_eq!(&v, &[1]);
> > +///
> > +/// # Ok::<(), Error>(())
> > +/// ```
> > +pub type VVec<T> = Vec<T, super::allocator::Vmalloc>;
> > +
> > +/// Type alias for `Vec` with a `KVmalloc` allocator.
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// let mut v = KVVec::new();
> > +/// v.push(1, GFP_KERNEL)?;
> > +/// assert_eq!(&v, &[1]);
> > +///
> > +/// # Ok::<(), Error>(())
> > +/// ```
> > +pub type KVVec<T> = Vec<T, super::allocator::KVmalloc>;
> > +
> > +// SAFETY: `Vec` is `Send` if `T` is `Send` because `Vec` owns its elements.
> > +unsafe impl<T, A> Send for Vec<T, A>
> > +where
> > +    T: Send,
> > +    A: Allocator,
> > +{
> > +}
> > +
> > +// SAFETY: `Vec` is `Sync` if `T` is `Sync` because `Vec` owns its elements.
> > +unsafe impl<T, A> Sync for Vec<T, A>
> > +where
> > +    T: Sync,
> > +    A: Allocator,
> > +{
> > +}
> > +
> > +impl<T, A> Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    #[inline]
> > +    fn is_zst() -> bool {
> > +        core::mem::size_of::<T>() == 0
> > +    }
> > +
> > +    /// Returns the number of elements that can be stored within the vector without allocating
> > +    /// additional memory.
> > +    pub fn capacity(&self) -> usize {
> > +        if Self::is_zst() {
> > +            usize::MAX
> > +        } else {
> > +            self.cap
> > +        }
> > +    }
> > +
> > +    /// Returns the number of elements stored within the vector.
> > +    #[inline]
> > +    pub fn len(&self) -> usize {
> > +        self.len
> > +    }
> > +
> > +    /// Forcefully sets `self.len` to `new_len`.
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// - `new_len` must be less than or equal to [`Self::capacity`].
> > +    /// - If `new_len` is greater than `self.len`, all elements within the interval
> > +    ///   [`self.len`,`new_len`] must be initialized.
> > +    #[inline]
> > +    pub unsafe fn set_len(&mut self, new_len: usize) {
> 
> Can you add a `debug_assert!(new_len < self.cap)`
> 
> > +        self.len = new_len;
> > +    }
> > +
> > +    /// Returns a slice of the entire vector.
> > +    ///
> > +    /// Equivalent to `&s[..]`.
> > +    #[inline]
> > +    pub fn as_slice(&self) -> &[T] {
> > +        self
> > +    }
> > +
> > +    /// Returns a mutable slice of the entire vector.
> > +    ///
> > +    /// Equivalent to `&mut s[..]`.
> > +    #[inline]
> > +    pub fn as_mut_slice(&mut self) -> &mut [T] {
> > +        self
> > +    }
> > +
> > +    /// Returns a mutable raw pointer to the vector's backing buffer, or, if `T` is a ZST, a
> > +    /// dangling raw pointer.
> > +    #[inline]
> > +    pub fn as_mut_ptr(&self) -> *mut T {
> > +        self.ptr.as_ptr()
> > +    }
> > +
> > +    /// Returns a raw pointer to the vector's backing buffer, or, if `T` is a ZST, a dangling raw
> > +    /// pointer.
> > +    #[inline]
> > +    pub fn as_ptr(&self) -> *const T {
> > +        self.as_mut_ptr()
> > +    }
> > +
> > +    /// Returns `true` if the vector contains no elements, `false` otherwise.
> > +    ///
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let mut v = KVec::new();
> > +    /// assert!(v.is_empty());
> > +    ///
> > +    /// v.push(1, GFP_KERNEL);
> > +    /// assert!(!v.is_empty());
> > +    /// ```
> > +    #[inline]
> > +    pub fn is_empty(&self) -> bool {
> > +        self.len() == 0
> > +    }
> > +
> > +    /// Creates a new, empty Vec<T, A>.
> > +    ///
> > +    /// This method does not allocate by itself.
> > +    #[inline]
> > +    pub const fn new() -> Self {
> > +        Self {
> > +            ptr: NonNull::dangling(),
> > +            cap: 0,
> > +            len: 0,
> > +            _p: PhantomData::<A>,
> > +        }
> > +    }
> > +
> > +    /// Returns a slice of `MaybeUninit<T>` for the remaining spare capacity of the vector.
> 
> Returns 

Hm?

> 
> > +    pub fn spare_capacity_mut(&mut self) -> &mut [MaybeUninit<T>] {
> > +        // SAFETY: The memory between `self.len` and `self.capacity` is guaranteed to be allocated
> > +        // and valid, but uninitialized.
> > +        unsafe {
> > +            slice::from_raw_parts_mut(
> > +                self.as_mut_ptr().add(self.len) as *mut MaybeUninit<T>,
> > +                self.capacity() - self.len,
> > +            )
> > +        }
> 
> There are two `unsafe` operations in this block, please split it into
> two blocks.
> 
> > +    }
> > +
> > +    /// Appends an element to the back of the [`Vec`] instance.
> > +    ///
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let mut v = KVec::new();
> > +    /// v.push(1, GFP_KERNEL)?;
> > +    /// assert_eq!(&v, &[1]);
> > +    ///
> > +    /// v.push(2, GFP_KERNEL)?;
> > +    /// assert_eq!(&v, &[1, 2]);
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> > +    pub fn push(&mut self, v: T, flags: Flags) -> Result<(), AllocError> {
> > +        Vec::reserve(self, 1, flags)?;
> > +        let s = self.spare_capacity_mut();
> > +        s[0].write(v);
> 
> The slice access will check the size, I don't know if it will be
> optimized out, if not, I would use a raw pointer here instead.
> 
> > +
> > +        // SAFETY: We just initialised the first spare entry, so it is safe to increase the length
> > +        // by 1. We also know that the new length is <= capacity because of the previous call to
> > +        // `reserve` above.
> > +        unsafe { self.set_len(self.len() + 1) };
> > +        Ok(())
> > +    }
> > +
> > +    /// Creates a new [`Vec`] instance with at least the given capacity.
> > +    ///
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let v = KVec::<u32>::with_capacity(20, GFP_KERNEL)?;
> > +    ///
> > +    /// assert!(v.capacity() >= 20);
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> > +    pub fn with_capacity(capacity: usize, flags: Flags) -> Result<Self, AllocError> {
> > +        let mut v = Vec::new();
> > +
> > +        Self::reserve(&mut v, capacity, flags)?;
> > +
> > +        Ok(v)
> > +    }
> > +
> > +    /// Pushes clones of the elements of slice into the [`Vec`] instance.
> > +    ///
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let mut v = KVec::new();
> > +    /// v.push(1, GFP_KERNEL)?;
> > +    ///
> > +    /// v.extend_from_slice(&[20, 30, 40], GFP_KERNEL)?;
> > +    /// assert_eq!(&v, &[1, 20, 30, 40]);
> > +    ///
> > +    /// v.extend_from_slice(&[50, 60], GFP_KERNEL)?;
> > +    /// assert_eq!(&v, &[1, 20, 30, 40, 50, 60]);
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> > +    pub fn extend_from_slice(&mut self, other: &[T], flags: Flags) -> Result<(), AllocError>
> > +    where
> > +        T: Clone,
> > +    {
> > +        self.reserve(other.len(), flags)?;
> > +        for (slot, item) in core::iter::zip(self.spare_capacity_mut(), other) {
> > +            slot.write(item.clone());
> > +        }
> > +
> > +        // SAFETY: We just initialised the `other.len()` spare entries, so it is safe to increase
> > +        // the length by the same amount. We also know that the new length is <= capacity because
> > +        // of the previous call to `reserve` above.
> 
> Please use a bullet point list and avoid natural language filler words.
> 
> > +        unsafe { self.set_len(self.len() + other.len()) };
> > +        Ok(())
> > +    }
> > +
> > +    /// Creates a Vec<T, A> from a pointer, a length and a capacity using the allocator `A`.
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// If `T` is a ZST:
> > +    ///
> > +    /// - `ptr` must be a dangling pointer.
> > +    /// - `capacity` must be zero.
> 
> When `T` is a ZST, `into_raw_parts` returns `usize::MAX` for `capacity`,
> so one can't just pipe its output to this function? That seems strange.
> 
> Also below you discard the value of `capacity`, when `T` is a ZST, so I
> think you can just remove this line.
> 
> > +    /// - `length` must be smaller than or equal to `usize::MAX`.
> 
> This is always true.
> 
> > +    ///
> > +    /// Otherwise:
> > +    ///
> > +    /// - `ptr` must have been allocated with the allocator `A`.
> > +    /// - `ptr` must satisfy or exceed the alignment requirements of `T`.
> > +    /// - `ptr` must point to memory with a size of at least `size_of::<T>` times the `capacity`
> > +    ///    bytes.
> 
> Just write "`size_of::<T>() * capacity` bytes".
> 
> > +    /// - The allocated size in bytes must not be larger than `isize::MAX`.
> 
> Should we make this a safety requirement of `Allocator`? I think this is
> generally the maximum size other allocated things can have anyways.



> 
> > +    /// - `length` must be less than or equal to `capacity`.
> > +    /// - The first `length` elements must be initialized values of type `T`.
> > +    ///
> > +    /// It is also valid to create an empty `Vec` passing a dangling pointer for `ptr` and zero for
> > +    /// `cap` and `len`.
> 
> Can you make this last sentence part of the `if` chain that you have
> above (ie the one started with "If `T` is a ZST:").

But `T` isn't necessarily a ZST, but it may be.

> 
> 
> Just to experiment with the suggestion from Kangrejos to use Rust as the
> language for safety documentation, here is what it could look like (we
> should discuss it more before we start using it):
> 
>     /// # Safety
>     ///
>     /// ```ignore
>     /// assert!(ptr.is_aligned_to(align_of::<T>()));
>     /// assert!(!ptr.is_null());
>     /// assert!(length <= capacity);
>     /// if capacity != 0 && size_of::<T>() != 0 {
>     ///     assert!(A::did_allocate(ptr));
>     ///     assert!(size_of::<T>() * capacity <= A::layout_of(ptr).len());
>     ///     assert!(is_initialized(ptr::slice_from_raw_parts(ptr, length)));
>     /// }
>     /// ```
> 
> I really like how this looks! We might want to add labels/names to each
> of the conditions and then one could use those in the justifications. A
> tool could then read those and match them to the requirements of the
> unsafe operation.

I need to think about this a bit more, but at a first glance I think I like it.

The tool would ideally be the compiler itself...

> 
> > +    ///
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let mut v = kernel::kvec![1, 2, 3]?;
> > +    /// v.reserve(1, GFP_KERNEL)?;
> > +    ///
> > +    /// let (mut ptr, mut len, cap) = v.into_raw_parts();
> > +    ///
> > +    /// // SAFETY: We've just reserved memory for another element.
> > +    /// unsafe { ptr.add(len).write(4) };
> > +    /// len += 1;
> > +    ///
> > +    /// // SAFETY: We only wrote an additional element at the end of the `KVec`'s buffer and
> > +    /// // correspondingly increased the length of the `KVec` by one. Otherwise, we construct it
> > +    /// // from the exact same raw parts.
> > +    /// let v = unsafe { KVec::from_raw_parts(ptr, len, cap) };
> > +    ///
> > +    /// assert_eq!(v, [1, 2, 3, 4]);
> > +    ///
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> 
> Please reorder the sections to be `# Examples` followed by `# Safety`.
> 
> > +    pub unsafe fn from_raw_parts(ptr: *mut T, length: usize, capacity: usize) -> Self {
> > +        let cap = if Self::is_zst() { 0 } else { capacity };
> > +
> > +        Self {
> > +            // SAFETY: By the safety requirements, `ptr` is either dangling or pointing to a valid
> > +            // memory allocation, allocated with `A`.
> > +            ptr: unsafe { NonNull::new_unchecked(ptr) },
> > +            cap,
> > +            len: length,
> > +            _p: PhantomData::<A>,
> > +        }
> > +    }
> > +
> > +    /// Consumes the `Vec<T, A>` and returns its raw components `pointer`, `length` and `capacity`.
> > +    ///
> > +    /// This will not run the destructor of the contained elements and for non-ZSTs the allocation
> > +    /// will stay alive indefinitely. Use [`Vec::from_raw_parts`] to recover the [`Vec`], drop the
> > +    /// elements and free the allocation, if any.
> > +    pub fn into_raw_parts(self) -> (*mut T, usize, usize) {
> > +        let me = ManuallyDrop::new(self);
> > +        let len = me.len();
> > +        let capacity = me.capacity();
> > +        let ptr = me.as_mut_ptr();
> > +        (ptr, len, capacity)
> > +    }
> > +
> > +    /// Ensures that the capacity exceeds the length by at least `additional`
> > +    /// elements.
> > +    ///
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let mut v = KVec::new();
> > +    /// v.push(1, GFP_KERNEL)?;
> > +    ///
> > +    /// v.reserve(10, GFP_KERNEL)?;
> > +    /// let cap = v.capacity();
> > +    /// assert!(cap >= 10);
> > +    ///
> > +    /// v.reserve(10, GFP_KERNEL)?;
> > +    /// let new_cap = v.capacity();
> > +    /// assert_eq!(new_cap, cap);
> > +    ///
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> > +    pub fn reserve(&mut self, additional: usize, flags: Flags) -> Result<(), AllocError> {
> > +        let len = self.len();
> > +        let cap = self.capacity();
> > +
> > +        if cap - len >= additional {
> > +            return Ok(());
> > +        }
> > +
> > +        if Self::is_zst() {
> > +            // The capacity is already `usize::MAX` for SZTs, we can't go higher.
> 
> Typo: SZTs -> ZSTs
> 
> > +            return Err(AllocError);
> > +        }
> > +
> > +        // We know `cap` is <= `isize::MAX` because of it's type invariant. So the multiplication by
> 
> "it's type invariant" -> "type invariants of Self"
> 
> > +        // two won't overflow.
> > +        let new_cap = core::cmp::max(cap * 2, len.checked_add(additional).ok_or(AllocError)?);
> > +        let layout = core::alloc::Layout::array::<T>(new_cap).map_err(|_| AllocError)?;
> > +
> > +        // We need to make sure that `ptr` is either NULL or comes from a previous call to
> > +        // `realloc_flags`. A `Vec<T, A>`'s `ptr` value is not guaranteed to be NULL and might be
> > +        // dangling after being created with `Vec::new`. Instead, we can rely on `Vec<T, A>`'s
> > +        // capacity to be zero if no memory has been allocated yet.
> > +        let ptr = if cap == 0 {
> > +            None
> > +        } else {
> > +            Some(self.ptr.cast())
> > +        };
> > +
> > +        // SAFETY: `ptr` is valid because it's either `None` or comes from a previous call to
> > +        // `A::realloc`. We also verified that the type is not a ZST.
> > +        let ptr = unsafe { A::realloc(ptr, layout, flags)? };
> > +
> > +        self.ptr = ptr.cast();
> > +
> > +        // INVARIANT: `Layout::array` fails if the resulting byte size is greater than `isize::MAX`.
> > +        self.cap = new_cap;
> > +
> > +        Ok(())
> > +    }
> > +}
> > +
> > +impl<T: Clone, A: Allocator> Vec<T, A> {
> > +    /// Extend the vector by `n` clones of value.
> 
> value -> `value`
> 
> > +    pub fn extend_with(&mut self, n: usize, value: T, flags: Flags) -> Result<(), AllocError> {
> > +        if n == 0 {
> > +            return Ok(());
> > +        }
> > +
> > +        self.reserve(n, flags)?;
> > +
> > +        let spare = self.spare_capacity_mut();
> > +
> > +        for item in spare.iter_mut().take(n - 1) {
> > +            item.write(value.clone());
> > +        }
> > +
> > +        // We can write the last element directly without cloning needlessly.
> > +        spare[n - 1].write(value);
> > +
> > +        // SAFETY: `self.reserve` not bailing out with an error guarantees that we're not
> > +        // exceeding the capacity of this `Vec`.
> 
> This safety comment needs to be improved, how about using bullet points:
> - `self.len() + n < capacity` due to the call to reserve above,
> - the loop and the line above initialized the next `n` elements.
> 
> > +        unsafe { self.set_len(self.len() + n) };
> > +
> > +        Ok(())
> > +    }
> > +
> > +    /// Create a new `Vec<T, A> and extend it by `n` clones of `value`.
> > +    pub fn from_elem(value: T, n: usize, flags: Flags) -> Result<Self, AllocError> {
> > +        let mut v = Self::with_capacity(n, flags)?;
> > +
> > +        v.extend_with(n, value, flags)?;
> > +
> > +        Ok(v)
> > +    }
> > +}
> > +
> > +impl<T, A> Drop for Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    fn drop(&mut self) {
> > +        // SAFETY: We need to drop the vector's elements in place, before we free the backing
> > +        // memory.
> > +        unsafe {
> > +            core::ptr::drop_in_place(core::ptr::slice_from_raw_parts_mut(
> 
> You are using `core::ptr` a bunch, consider using it, then you don't
> need to write the prefix `core::`.
> 
> > +                self.as_mut_ptr(),
> > +                self.len,
> > +            ))
> > +        };
> > +
> > +        // If `cap == 0` we never allocated any memory in the first place.
> > +        if self.cap != 0 {
> > +            // SAFETY: `self.ptr` was previously allocated with `A`.
> > +            unsafe { A::free(self.ptr.cast()) };
> > +        }
> > +    }
> > +}
> > +
> > +impl<T, A, const N: usize> From<Box<[T; N], A>> for Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    fn from(b: Box<[T; N], A>) -> Vec<T, A> {
> > +        let len = b.len();
> > +        let ptr = Box::into_raw(b);
> > +
> > +        unsafe { Vec::from_raw_parts(ptr as _, len, len) }
> 
> Missing SAFETY comment.
> 
> > +    }
> > +}
> > +
> > +impl<T> Default for KVec<T> {
> > +    #[inline]
> > +    fn default() -> Self {
> > +        Self::new()
> > +    }
> > +}
> > +
> > +impl<T: fmt::Debug, A: Allocator> fmt::Debug for Vec<T, A> {
> > +    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
> > +        fmt::Debug::fmt(&**self, f)
> > +    }
> > +}
> > +
> > +impl<T, A> Deref for Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    type Target = [T];
> > +
> > +    #[inline]
> > +    fn deref(&self) -> &[T] {
> > +        // SAFETY: The memory behind `self.as_ptr()` is guaranteed to contain `self.len`
> > +        // initialized elements of type `T`.
> > +        unsafe { slice::from_raw_parts(self.as_ptr(), self.len) }
> > +    }
> > +}
> > +
> > +impl<T, A> DerefMut for Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    #[inline]
> > +    fn deref_mut(&mut self) -> &mut [T] {
> > +        // SAFETY: The memory behind `self.as_ptr()` is guaranteed to contain `self.len`
> > +        // initialized elements of type `T`.
> > +        unsafe { slice::from_raw_parts_mut(self.as_mut_ptr(), self.len) }
> > +    }
> > +}
> > +
> > +impl<T: Eq, A> Eq for Vec<T, A> where A: Allocator {}
> > +
> > +impl<T, I: SliceIndex<[T]>, A> Index<I> for Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    type Output = I::Output;
> > +
> > +    #[inline]
> > +    fn index(&self, index: I) -> &Self::Output {
> > +        Index::index(&**self, index)
> > +    }
> > +}
> > +
> > +impl<T, I: SliceIndex<[T]>, A> IndexMut<I> for Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    #[inline]
> > +    fn index_mut(&mut self, index: I) -> &mut Self::Output {
> > +        IndexMut::index_mut(&mut **self, index)
> > +    }
> > +}
> > +
> > +macro_rules! __impl_slice_eq {
> 
> I wouldn't begin this macro with `__`, it is already private.
> 
> > +    ([$($vars:tt)*] $lhs:ty, $rhs:ty $(where $ty:ty: $bound:ident)?) => {
> 
> You don't use the optional `where` part in the code below, if you don't
> use it later, then remove it (if you do use it later, I would move it
> into that patch).
> 
> ---
> Cheers,
> Benno
> 
> > +        impl<T, U, $($vars)*> PartialEq<$rhs> for $lhs
> > +        where
> > +            T: PartialEq<U>,
> > +            $($ty: $bound)?
> > +        {
> > +            #[inline]
> > +            fn eq(&self, other: &$rhs) -> bool { self[..] == other[..] }
> > +        }
> > +    }
> > +}
> > +
> > +__impl_slice_eq! { [A1: Allocator, A2: Allocator] Vec<T, A1>, Vec<U, A2> }
> > +__impl_slice_eq! { [A: Allocator] Vec<T, A>, &[U] }
> > +__impl_slice_eq! { [A: Allocator] Vec<T, A>, &mut [U] }
> > +__impl_slice_eq! { [A: Allocator] &[T], Vec<U, A> }
> > +__impl_slice_eq! { [A: Allocator] &mut [T], Vec<U, A> }
> > +__impl_slice_eq! { [A: Allocator] Vec<T, A>, [U] }
> > +__impl_slice_eq! { [A: Allocator] [T], Vec<U, A> }
> > +__impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, [U; N] }
> > +__impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, &[U; N] }
> > diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
> > index d5f2fe42d093..80223cdaa485 100644
> > --- a/rust/kernel/prelude.rs
> > +++ b/rust/kernel/prelude.rs
> > @@ -14,7 +14,7 @@
> >  #[doc(no_inline)]
> >  pub use core::pin::Pin;
> > 
> > -pub use crate::alloc::{flags::*, vec_ext::VecExt, Box, KBox, KVBox, VBox};
> > +pub use crate::alloc::{flags::*, vec_ext::VecExt, Box, KBox, KVBox, KVVec, KVec, VBox, VVec};
> > 
> >  #[doc(no_inline)]
> >  pub use alloc::vec::Vec;
> > --
> > 2.46.0
> > 
> 

Re: [PATCH v6 13/26] rust: alloc: implement kernel `Vec` type

[Benno Lossin]

On 11.09.24 02:18, Danilo Krummrich wrote:
> On Tue, Sep 10, 2024 at 07:32:06PM +0000, Benno Lossin wrote:
>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>> +    ($elem:expr; $n:expr) => (
>>> +        {
>>> +            $crate::alloc::KVec::from_elem($elem, $n, GFP_KERNEL)
>>
>> Hmm, could it be that one might want to use `kvec!` without
>> `GFP_KERNEL`?
>> Or add additional flags, eg __GFP_ZERO?
> 
> Pretty unlikely, I'd say. __GFP_ZERO in particular wouldn't make much sense,
> since the macro enforces initialization anyways.
> 
> Maybe something like GFP_ATOMIC, but I expect this to be uncommon enough to not
> consider this for this macro for now.

SGTM

>> If you think that supporting this is not immediately necessary or if
>> this is too much for this patchset, then we can also postpone it (maybe
>> it could be a good-first-issue). Do you keep a list of future
>> improvements for the new allocator API somewhere? If not, then I think
>> we should create one (best place would be the issue tracker on GH).
> 
> I would only add it if it turns out to be a common need. As mentioned, I don't
> expect it to be.
> 
> I'd rather keep issues in the source tree. For instance, DRM has them in
> '/Documentation/gpu/todo.rst'.

But then you need to submit patches to change them... That sounds like a
hassle and we already have the precedent to use the github issue
tracker. It is also much better for discoverability for people outside
of the kernel.

>>> +        }
>>> +    );
>>> +    ($($x:expr),+ $(,)?) => (
>>> +        {
>>> +            match $crate::alloc::KBox::new_uninit(GFP_KERNEL) {
>>> +                Ok(b) => Ok($crate::alloc::KVec::from($crate::alloc::KBox::write(b, [$($x),+]))),
>>> +                Err(e) => Err(e),
>>> +            }
>>> +        }
>>> +    );
>>> +}
>>> +
>>> +/// The kernel's [`Vec`] type.
>>> +///
>>> +/// A contiguous growable array type with contents allocated with the kernel's allocators (e.g.
>>> +/// `Kmalloc`, `Vmalloc` or `KVmalloc`), written `Vec<T, A>`.
>>
>> New folks might get confused as to which allocator they should choose.
>> Do we have a sensible default for `Vec`?
> 
> Then they should read the documentation about the kernel's allocators. We
> already link them in rust/kernel/alloc/allocator.rs.
> 
>> If yes, then we at least should document that here. We might also want
>> to make it the default value for the generic parameter.
>> This is also a good idea for `Box`.
> 
> If we really want a default it should be `Kmalloc`, but I really think we should
> force people to make an explicit decision and think about it and don't just go
> with whatever default.
> 
> It makes it also easier to grep for things. :)

SGTM

>>> +///
>>> +/// For non-zero-sized values, a [`Vec`] will use the given allocator `A` for its allocation. For
>>> +/// the most common allocators the type aliases `KVec`, `VVec` and `KVVec` exist.
>>> +///
>>> +/// For zero-sized types the [`Vec`]'s pointer must be `dangling_mut::<T>`; no memory is allocated.
>>> +///
>>> +/// Generally, [`Vec`] consists of a pointer that represents the vector's backing buffer, the
>>> +/// capacity of the vector (the number of elements that currently fit into the vector), it's length
>>> +/// (the number of elements that are currently stored in the vector) and the `Allocator` type used
>>> +/// to allocate (and free) the backing buffer.
>>> +///
>>> +/// A [`Vec`] can be deconstructed into and (re-)constructed from it's previously named raw parts
>>> +/// and manually modified.
>>> +///
>>> +/// [`Vec`]'s backing buffer gets, if required, automatically increased (re-allocated) when elements
>>> +/// are added to the vector.
>>> +///
>>> +/// # Invariants
>>> +///
>>> +/// The [`Vec`] backing buffer's pointer is always properly aligned and either points to memory
>>> +/// allocated with `A` or, for zero-sized types, is a dangling pointer.
>>
>> Just use `self.ptr` instead of "backing buffer".
>>
>>> +///
>>> +/// The length of the vector always represents the exact number of elements stored in the vector.
>>
>> Same here, `self.len`.
>>
>>> +///
>>> +/// The capacity of the vector always represents the absolute number of elements that can be stored
>>
>> Ditto.
>>
>>> +/// within the vector without re-allocation. However, it is legal for the backing buffer to be
>>> +/// larger than `size_of<T>` times the capacity.
>>> +///
>>> +/// The `Allocator` type `A` of the vector is the exact same `Allocator` type the backing buffer was
>>> +/// allocated with (and must be freed with).
>>
>> Please turn this into a bullet-point list.
> 
> Is this a rule? Do we want to make it one?

I am trying to make it one with my safety standard.

>>> +pub struct Vec<T, A: Allocator> {
>>> +    ptr: NonNull<T>,
>>> +    /// Represents the actual buffer size as `cap` times `size_of::<T>` bytes.
>>> +    ///
>>> +    /// Note: This isn't quite the same as `Self::capacity`, which in contrast returns the number of
>>> +    /// elements we can still store without reallocating.
>>> +    ///
>>> +    /// # Invariants
>>> +    ///
>>> +    /// `cap` must be in the `0..=isize::MAX` range.
>>> +    cap: usize,
>>> +    len: usize,
>>> +    _p: PhantomData<A>,
>>> +}



>>> +    /// Creates a new, empty Vec<T, A>.
>>> +    ///
>>> +    /// This method does not allocate by itself.
>>> +    #[inline]
>>> +    pub const fn new() -> Self {
>>> +        Self {
>>> +            ptr: NonNull::dangling(),
>>> +            cap: 0,
>>> +            len: 0,
>>> +            _p: PhantomData::<A>,
>>> +        }
>>> +    }
>>> +
>>> +    /// Returns a slice of `MaybeUninit<T>` for the remaining spare capacity of the vector.
>>
>> Returns
> 
> Hm?

Forgot to finish that sentence, since I couldn't really pinpoint what
exactly I wanted to change. Just ignore it.

>>> +    pub fn spare_capacity_mut(&mut self) -> &mut [MaybeUninit<T>] {
>>> +        // SAFETY: The memory between `self.len` and `self.capacity` is guaranteed to be allocated
>>> +        // and valid, but uninitialized.
>>> +        unsafe {
>>> +            slice::from_raw_parts_mut(
>>> +                self.as_mut_ptr().add(self.len) as *mut MaybeUninit<T>,
>>> +                self.capacity() - self.len,
>>> +            )
>>> +        }



>>> +    ///
>>> +    /// Otherwise:
>>> +    ///
>>> +    /// - `ptr` must have been allocated with the allocator `A`.
>>> +    /// - `ptr` must satisfy or exceed the alignment requirements of `T`.
>>> +    /// - `ptr` must point to memory with a size of at least `size_of::<T>` times the `capacity`
>>> +    ///    bytes.
>>
>> Just write "`size_of::<T>() * capacity` bytes".
>>
>>> +    /// - The allocated size in bytes must not be larger than `isize::MAX`.
>>
>> Should we make this a safety requirement of `Allocator`? I think this is
>> generally the maximum size other allocated things can have anyways.
> 
> 
> 
>>
>>> +    /// - `length` must be less than or equal to `capacity`.
>>> +    /// - The first `length` elements must be initialized values of type `T`.
>>> +    ///
>>> +    /// It is also valid to create an empty `Vec` passing a dangling pointer for `ptr` and zero for
>>> +    /// `cap` and `len`.
>>
>> Can you make this last sentence part of the `if` chain that you have
>> above (ie the one started with "If `T` is a ZST:").
> 
> But `T` isn't necessarily a ZST, but it may be.

I originally meant adding an "else if" part that checks for empty
capacity. But you could just add that to the if at the top ie "If
`capacity == 0` or `T` is a ZST".

>> Just to experiment with the suggestion from Kangrejos to use Rust as the
>> language for safety documentation, here is what it could look like (we
>> should discuss it more before we start using it):
>>
>>     /// # Safety
>>     ///
>>     /// ```ignore
>>     /// assert!(ptr.is_aligned_to(align_of::<T>()));
>>     /// assert!(!ptr.is_null());
>>     /// assert!(length <= capacity);
>>     /// if capacity != 0 && size_of::<T>() != 0 {
>>     ///     assert!(A::did_allocate(ptr));
>>     ///     assert!(size_of::<T>() * capacity <= A::layout_of(ptr).len());
>>     ///     assert!(is_initialized(ptr::slice_from_raw_parts(ptr, length)));
>>     /// }
>>     /// ```
>>
>> I really like how this looks! We might want to add labels/names to each
>> of the conditions and then one could use those in the justifications. A
>> tool could then read those and match them to the requirements of the
>> unsafe operation.
> 
> I need to think about this a bit more, but at a first glance I think I like it.
> 
> The tool would ideally be the compiler itself...

Yes! There is the contracts draft PR that might add support for
this: https://github.com/rust-lang/rust/pull/128045

---
Cheers,
Benno

Re: [PATCH v6 13/26] rust: alloc: implement kernel `Vec` type

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> `Vec` provides a contiguous growable array type (such as `Vec`) with
> contents allocated with the kernel's allocators (e.g. `Kmalloc`,
> `Vmalloc` or `KVmalloc`).
> 
> In contrast to Rust's `Vec` type, the kernel `Vec` type considers the
> kernel's GFP flags for all appropriate functions, always reports
> allocation failures through `Result<_, AllocError>` and remains
> independent from unstable features.
> 
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
>  rust/kernel/alloc.rs      |   6 +
>  rust/kernel/alloc/kvec.rs | 629 ++++++++++++++++++++++++++++++++++++++
>  rust/kernel/prelude.rs    |   2 +-
>  3 files changed, 636 insertions(+), 1 deletion(-)
>  create mode 100644 rust/kernel/alloc/kvec.rs

I noticed that you don't have a `pop` or `remove` function implemented,
I think it would be weird to have all the other functions but not those.

---
Cheers,
Benno

Re: [PATCH v6 13/26] rust: alloc: implement kernel `Vec` type

[Danilo Krummrich]

On Tue, Sep 10, 2024 at 08:07:40PM +0000, Benno Lossin wrote:
> On 16.08.24 02:10, Danilo Krummrich wrote:
> > `Vec` provides a contiguous growable array type (such as `Vec`) with
> > contents allocated with the kernel's allocators (e.g. `Kmalloc`,
> > `Vmalloc` or `KVmalloc`).
> > 
> > In contrast to Rust's `Vec` type, the kernel `Vec` type considers the
> > kernel's GFP flags for all appropriate functions, always reports
> > allocation failures through `Result<_, AllocError>` and remains
> > independent from unstable features.
> > 
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > ---
> >  rust/kernel/alloc.rs      |   6 +
> >  rust/kernel/alloc/kvec.rs | 629 ++++++++++++++++++++++++++++++++++++++
> >  rust/kernel/prelude.rs    |   2 +-
> >  3 files changed, 636 insertions(+), 1 deletion(-)
> >  create mode 100644 rust/kernel/alloc/kvec.rs
> 
> I noticed that you don't have a `pop` or `remove` function implemented,
> I think it would be weird to have all the other functions but not those.

I'd rather not add them for this series anymore. We can add them in subsequent
patches, or wait until they're actually needed.

I also already have some subsequent patches in the queue. For instance, in Nova
we'll need `Vec::resize` and `Vec::truncate`.

> 
> ---
> Cheers,
> Benno
> 

Re: [PATCH v6 13/26] rust: alloc: implement kernel `Vec` type

[Alice Ryhl]

On Wed, Sep 11, 2024 at 11:59 PM Danilo Krummrich <dakr@kernel.org> wrote:
>
> On Tue, Sep 10, 2024 at 08:07:40PM +0000, Benno Lossin wrote:
> > On 16.08.24 02:10, Danilo Krummrich wrote:
> > > `Vec` provides a contiguous growable array type (such as `Vec`) with
> > > contents allocated with the kernel's allocators (e.g. `Kmalloc`,
> > > `Vmalloc` or `KVmalloc`).
> > >
> > > In contrast to Rust's `Vec` type, the kernel `Vec` type considers the
> > > kernel's GFP flags for all appropriate functions, always reports
> > > allocation failures through `Result<_, AllocError>` and remains
> > > independent from unstable features.
> > >
> > > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > > ---
> > >  rust/kernel/alloc.rs      |   6 +
> > >  rust/kernel/alloc/kvec.rs | 629 ++++++++++++++++++++++++++++++++++++++
> > >  rust/kernel/prelude.rs    |   2 +-
> > >  3 files changed, 636 insertions(+), 1 deletion(-)
> > >  create mode 100644 rust/kernel/alloc/kvec.rs
> >
> > I noticed that you don't have a `pop` or `remove` function implemented,
> > I think it would be weird to have all the other functions but not those.
>
> I'd rather not add them for this series anymore. We can add them in subsequent
> patches, or wait until they're actually needed.
>
> I also already have some subsequent patches in the queue. For instance, in Nova
> we'll need `Vec::resize` and `Vec::truncate`.

When this lands in rust-next, we should open good-first-issues for
these on the Rust-for-Linux bug tracker.

Alice

[PATCH v6 14/26] rust: alloc: implement `IntoIterator` for `Vec`

[Danilo Krummrich]

Implement `IntoIterator` for `Vec`, `Vec`'s `IntoIter` type, as well as
`Iterator` for `IntoIter`.

`Vec::into_iter` disassembles the `Vec` into its raw parts; additionally,
`IntoIter` keeps track of a separate pointer, which is incremented
correspondingsly as the iterator advances, while the length, or the count
of elements, is decremented.

This also means that `IntoIter` takes the ownership of the backing
buffer and is responsible to drop the remaining elements and free the
backing buffer, if it's dropped.

Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc.rs      |   1 +
 rust/kernel/alloc/kvec.rs | 184 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 185 insertions(+)

diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index e88c7e10ee9b..4ff4df4597a3 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -19,6 +19,7 @@
 pub use self::kbox::KVBox;
 pub use self::kbox::VBox;
 
+pub use self::kvec::IntoIter;
 pub use self::kvec::KVVec;
 pub use self::kvec::KVec;
 pub use self::kvec::VVec;
diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
index 89afc0f25bd4..3b79f977b65e 100644
--- a/rust/kernel/alloc/kvec.rs
+++ b/rust/kernel/alloc/kvec.rs
@@ -11,6 +11,7 @@
     ops::DerefMut,
     ops::Index,
     ops::IndexMut,
+    ptr,
     ptr::NonNull,
     slice,
     slice::SliceIndex,
@@ -627,3 +628,186 @@ fn eq(&self, other: &$rhs) -> bool { self[..] == other[..] }
 __impl_slice_eq! { [A: Allocator] [T], Vec<U, A> }
 __impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, [U; N] }
 __impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, &[U; N] }
+
+impl<'a, T, A> IntoIterator for &'a Vec<T, A>
+where
+    A: Allocator,
+{
+    type Item = &'a T;
+    type IntoIter = slice::Iter<'a, T>;
+
+    fn into_iter(self) -> Self::IntoIter {
+        self.iter()
+    }
+}
+
+impl<'a, T, A: Allocator> IntoIterator for &'a mut Vec<T, A>
+where
+    A: Allocator,
+{
+    type Item = &'a mut T;
+    type IntoIter = slice::IterMut<'a, T>;
+
+    fn into_iter(self) -> Self::IntoIter {
+        self.iter_mut()
+    }
+}
+
+/// An `Iterator` implementation for `Vec<T,A>` that moves elements out of a vector.
+///
+/// This structure is created by the `Vec::into_iter` method on [`Vec`] (provided by the
+/// [`IntoIterator`] trait).
+///
+/// # Examples
+///
+/// ```
+/// let v = kernel::kvec![0, 1, 2]?;
+/// let iter = v.into_iter();
+///
+/// # Ok::<(), Error>(())
+/// ```
+pub struct IntoIter<T, A: Allocator> {
+    ptr: *mut T,
+    buf: NonNull<T>,
+    len: usize,
+    cap: usize,
+    _p: PhantomData<A>,
+}
+
+impl<T, A> IntoIter<T, A>
+where
+    A: Allocator,
+{
+    fn as_raw_mut_slice(&mut self) -> *mut [T] {
+        ptr::slice_from_raw_parts_mut(self.ptr, self.len)
+    }
+}
+
+impl<T, A> Iterator for IntoIter<T, A>
+where
+    A: Allocator,
+{
+    type Item = T;
+
+    /// # Examples
+    ///
+    /// ```
+    /// let v = kernel::kvec![1, 2, 3]?;
+    /// let mut it = v.into_iter();
+    ///
+    /// assert_eq!(it.next(), Some(1));
+    /// assert_eq!(it.next(), Some(2));
+    /// assert_eq!(it.next(), Some(3));
+    /// assert_eq!(it.next(), None);
+    ///
+    /// # Ok::<(), Error>(())
+    /// ```
+    fn next(&mut self) -> Option<T> {
+        if self.len == 0 {
+            return None;
+        }
+
+        let ptr = self.ptr;
+        if !Vec::<T, A>::is_zst() {
+            // SAFETY: We can't overflow; `end` is guaranteed to mark the end of the buffer.
+            unsafe { self.ptr = self.ptr.add(1) };
+        } else {
+            // For ZST `ptr` has to stay where it is to remain aligned, so we just reduce `self.len`
+            // by 1.
+        }
+        self.len -= 1;
+
+        // SAFETY: `ptr` is guaranteed to point at a valid element within the buffer.
+        Some(unsafe { ptr.read() })
+    }
+
+    /// # Examples
+    ///
+    /// ```
+    /// let v: KVec<u32> = kernel::kvec![1, 2, 3]?;
+    /// let mut iter = v.into_iter();
+    /// let size = iter.size_hint().0;
+    ///
+    /// iter.next();
+    /// assert_eq!(iter.size_hint().0, size - 1);
+    ///
+    /// iter.next();
+    /// assert_eq!(iter.size_hint().0, size - 2);
+    ///
+    /// iter.next();
+    /// assert_eq!(iter.size_hint().0, size - 3);
+    ///
+    /// # Ok::<(), Error>(())
+    /// ```
+    fn size_hint(&self) -> (usize, Option<usize>) {
+        (self.len, Some(self.len))
+    }
+}
+
+impl<T, A> Drop for IntoIter<T, A>
+where
+    A: Allocator,
+{
+    fn drop(&mut self) {
+        // SAFETY: Drop the remaining vector's elements in place, before we free the backing
+        // memory.
+        unsafe { ptr::drop_in_place(self.as_raw_mut_slice()) };
+
+        // If `cap == 0` we never allocated any memory in the first place.
+        if self.cap != 0 {
+            // SAFETY: `self.buf` was previously allocated with `A`.
+            unsafe { A::free(self.buf.cast()) };
+        }
+    }
+}
+
+impl<T, A> IntoIterator for Vec<T, A>
+where
+    A: Allocator,
+{
+    type Item = T;
+    type IntoIter = IntoIter<T, A>;
+
+    /// Consumes the `Vec<T, A>` and creates an `Iterator`, which moves each value out of the
+    /// vector (from start to end).
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let v = kernel::kvec![1, 2]?;
+    /// let mut v_iter = v.into_iter();
+    ///
+    /// let first_element: Option<u32> = v_iter.next();
+    ///
+    /// assert_eq!(first_element, Some(1));
+    /// assert_eq!(v_iter.next(), Some(2));
+    /// assert_eq!(v_iter.next(), None);
+    ///
+    /// # Ok::<(), Error>(())
+    /// ```
+    ///
+    /// ```
+    /// let v = kernel::kvec![];
+    /// let mut v_iter = v.into_iter();
+    ///
+    /// let first_element: Option<u32> = v_iter.next();
+    ///
+    /// assert_eq!(first_element, None);
+    ///
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    fn into_iter(self) -> Self::IntoIter {
+        let (ptr, len, cap) = self.into_raw_parts();
+
+        IntoIter {
+            ptr,
+            // SAFETY: `ptr` is either a dangling pointer or a pointer to a valid memory
+            // allocation, allocated with `A`.
+            buf: unsafe { NonNull::new_unchecked(ptr) },
+            len,
+            cap,
+            _p: PhantomData::<A>,
+        }
+    }
+}
-- 
2.46.0

Re: [PATCH v6 14/26] rust: alloc: implement `IntoIterator` for `Vec`

[Alice Ryhl]

On Fri, Aug 16, 2024 at 2:13 AM Danilo Krummrich <dakr@kernel.org> wrote:
>
> Implement `IntoIterator` for `Vec`, `Vec`'s `IntoIter` type, as well as
> `Iterator` for `IntoIter`.
>
> `Vec::into_iter` disassembles the `Vec` into its raw parts; additionally,
> `IntoIter` keeps track of a separate pointer, which is incremented
> correspondingsly as the iterator advances, while the length, or the count
> of elements, is decremented.
>
> This also means that `IntoIter` takes the ownership of the backing
> buffer and is responsible to drop the remaining elements and free the
> backing buffer, if it's dropped.
>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

This looks ok to me. One nit below, though. Either way:

Reviewed-by: Alice Ryhl <aliceryhl@google.com>

>  rust/kernel/alloc.rs      |   1 +
>  rust/kernel/alloc/kvec.rs | 184 ++++++++++++++++++++++++++++++++++++++
>  2 files changed, 185 insertions(+)
>
> diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> index e88c7e10ee9b..4ff4df4597a3 100644
> --- a/rust/kernel/alloc.rs
> +++ b/rust/kernel/alloc.rs
> @@ -19,6 +19,7 @@
>  pub use self::kbox::KVBox;
>  pub use self::kbox::VBox;
>
> +pub use self::kvec::IntoIter;
>  pub use self::kvec::KVVec;
>  pub use self::kvec::KVec;
>  pub use self::kvec::VVec;
> diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
> index 89afc0f25bd4..3b79f977b65e 100644
> --- a/rust/kernel/alloc/kvec.rs
> +++ b/rust/kernel/alloc/kvec.rs
> @@ -11,6 +11,7 @@
>      ops::DerefMut,
>      ops::Index,
>      ops::IndexMut,
> +    ptr,
>      ptr::NonNull,
>      slice,
>      slice::SliceIndex,
> @@ -627,3 +628,186 @@ fn eq(&self, other: &$rhs) -> bool { self[..] == other[..] }
>  __impl_slice_eq! { [A: Allocator] [T], Vec<U, A> }
>  __impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, [U; N] }
>  __impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, &[U; N] }
> +
> +impl<'a, T, A> IntoIterator for &'a Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Item = &'a T;
> +    type IntoIter = slice::Iter<'a, T>;
> +
> +    fn into_iter(self) -> Self::IntoIter {
> +        self.iter()
> +    }
> +}
> +
> +impl<'a, T, A: Allocator> IntoIterator for &'a mut Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Item = &'a mut T;
> +    type IntoIter = slice::IterMut<'a, T>;
> +
> +    fn into_iter(self) -> Self::IntoIter {
> +        self.iter_mut()
> +    }
> +}
> +
> +/// An `Iterator` implementation for `Vec<T,A>` that moves elements out of a vector.
> +///
> +/// This structure is created by the `Vec::into_iter` method on [`Vec`] (provided by the
> +/// [`IntoIterator`] trait).
> +///
> +/// # Examples
> +///
> +/// ```
> +/// let v = kernel::kvec![0, 1, 2]?;
> +/// let iter = v.into_iter();
> +///
> +/// # Ok::<(), Error>(())
> +/// ```
> +pub struct IntoIter<T, A: Allocator> {
> +    ptr: *mut T,
> +    buf: NonNull<T>,
> +    len: usize,
> +    cap: usize,
> +    _p: PhantomData<A>,
> +}
> +
> +impl<T, A> IntoIter<T, A>
> +where
> +    A: Allocator,
> +{
> +    fn as_raw_mut_slice(&mut self) -> *mut [T] {
> +        ptr::slice_from_raw_parts_mut(self.ptr, self.len)
> +    }
> +}
> +
> +impl<T, A> Iterator for IntoIter<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Item = T;
> +
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let v = kernel::kvec![1, 2, 3]?;
> +    /// let mut it = v.into_iter();
> +    ///
> +    /// assert_eq!(it.next(), Some(1));
> +    /// assert_eq!(it.next(), Some(2));
> +    /// assert_eq!(it.next(), Some(3));
> +    /// assert_eq!(it.next(), None);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    fn next(&mut self) -> Option<T> {
> +        if self.len == 0 {
> +            return None;
> +        }
> +
> +        let ptr = self.ptr;

Nit: It would probably be slightly clearer to rename this variable to `current`.

> +        if !Vec::<T, A>::is_zst() {
> +            // SAFETY: We can't overflow; `end` is guaranteed to mark the end of the buffer.
> +            unsafe { self.ptr = self.ptr.add(1) };
> +        } else {
> +            // For ZST `ptr` has to stay where it is to remain aligned, so we just reduce `self.len`
> +            // by 1.
> +        }
> +        self.len -= 1;
> +
> +        // SAFETY: `ptr` is guaranteed to point at a valid element within the buffer.
> +        Some(unsafe { ptr.read() })
> +    }
> +
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let v: KVec<u32> = kernel::kvec![1, 2, 3]?;
> +    /// let mut iter = v.into_iter();
> +    /// let size = iter.size_hint().0;
> +    ///
> +    /// iter.next();
> +    /// assert_eq!(iter.size_hint().0, size - 1);
> +    ///
> +    /// iter.next();
> +    /// assert_eq!(iter.size_hint().0, size - 2);
> +    ///
> +    /// iter.next();
> +    /// assert_eq!(iter.size_hint().0, size - 3);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    fn size_hint(&self) -> (usize, Option<usize>) {
> +        (self.len, Some(self.len))
> +    }
> +}
> +
> +impl<T, A> Drop for IntoIter<T, A>
> +where
> +    A: Allocator,
> +{
> +    fn drop(&mut self) {
> +        // SAFETY: Drop the remaining vector's elements in place, before we free the backing
> +        // memory.
> +        unsafe { ptr::drop_in_place(self.as_raw_mut_slice()) };
> +
> +        // If `cap == 0` we never allocated any memory in the first place.
> +        if self.cap != 0 {
> +            // SAFETY: `self.buf` was previously allocated with `A`.
> +            unsafe { A::free(self.buf.cast()) };
> +        }
> +    }
> +}
> +
> +impl<T, A> IntoIterator for Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Item = T;
> +    type IntoIter = IntoIter<T, A>;
> +
> +    /// Consumes the `Vec<T, A>` and creates an `Iterator`, which moves each value out of the
> +    /// vector (from start to end).
> +    ///
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let v = kernel::kvec![1, 2]?;
> +    /// let mut v_iter = v.into_iter();
> +    ///
> +    /// let first_element: Option<u32> = v_iter.next();
> +    ///
> +    /// assert_eq!(first_element, Some(1));
> +    /// assert_eq!(v_iter.next(), Some(2));
> +    /// assert_eq!(v_iter.next(), None);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    ///
> +    /// ```
> +    /// let v = kernel::kvec![];
> +    /// let mut v_iter = v.into_iter();
> +    ///
> +    /// let first_element: Option<u32> = v_iter.next();
> +    ///
> +    /// assert_eq!(first_element, None);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    #[inline]
> +    fn into_iter(self) -> Self::IntoIter {
> +        let (ptr, len, cap) = self.into_raw_parts();
> +
> +        IntoIter {
> +            ptr,
> +            // SAFETY: `ptr` is either a dangling pointer or a pointer to a valid memory
> +            // allocation, allocated with `A`.
> +            buf: unsafe { NonNull::new_unchecked(ptr) },
> +            len,
> +            cap,
> +            _p: PhantomData::<A>,
> +        }
> +    }
> +}
> --
> 2.46.0
>

Re: [PATCH v6 14/26] rust: alloc: implement `IntoIterator` for `Vec`

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> Implement `IntoIterator` for `Vec`, `Vec`'s `IntoIter` type, as well as
> `Iterator` for `IntoIter`.
> 
> `Vec::into_iter` disassembles the `Vec` into its raw parts; additionally,
> `IntoIter` keeps track of a separate pointer, which is incremented
> correspondingsly as the iterator advances, while the length, or the count
> of elements, is decremented.
> 
> This also means that `IntoIter` takes the ownership of the backing
> buffer and is responsible to drop the remaining elements and free the
> backing buffer, if it's dropped.
> 
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
>  rust/kernel/alloc.rs      |   1 +
>  rust/kernel/alloc/kvec.rs | 184 ++++++++++++++++++++++++++++++++++++++
>  2 files changed, 185 insertions(+)
> 
> diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> index e88c7e10ee9b..4ff4df4597a3 100644
> --- a/rust/kernel/alloc.rs
> +++ b/rust/kernel/alloc.rs
> @@ -19,6 +19,7 @@
>  pub use self::kbox::KVBox;
>  pub use self::kbox::VBox;
> 
> +pub use self::kvec::IntoIter;
>  pub use self::kvec::KVVec;
>  pub use self::kvec::KVec;
>  pub use self::kvec::VVec;
> diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
> index 89afc0f25bd4..3b79f977b65e 100644
> --- a/rust/kernel/alloc/kvec.rs
> +++ b/rust/kernel/alloc/kvec.rs
> @@ -11,6 +11,7 @@
>      ops::DerefMut,
>      ops::Index,
>      ops::IndexMut,
> +    ptr,
>      ptr::NonNull,
>      slice,
>      slice::SliceIndex,
> @@ -627,3 +628,186 @@ fn eq(&self, other: &$rhs) -> bool { self[..] == other[..] }
>  __impl_slice_eq! { [A: Allocator] [T], Vec<U, A> }
>  __impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, [U; N] }
>  __impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, &[U; N] }
> +
> +impl<'a, T, A> IntoIterator for &'a Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Item = &'a T;
> +    type IntoIter = slice::Iter<'a, T>;
> +
> +    fn into_iter(self) -> Self::IntoIter {
> +        self.iter()
> +    }
> +}
> +
> +impl<'a, T, A: Allocator> IntoIterator for &'a mut Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Item = &'a mut T;
> +    type IntoIter = slice::IterMut<'a, T>;
> +
> +    fn into_iter(self) -> Self::IntoIter {
> +        self.iter_mut()
> +    }
> +}
> +
> +/// An `Iterator` implementation for `Vec<T,A>` that moves elements out of a vector.

Please make both links.

> +///
> +/// This structure is created by the `Vec::into_iter` method on [`Vec`] (provided by the

Ditto.

> +/// [`IntoIterator`] trait).
> +///
> +/// # Examples
> +///
> +/// ```
> +/// let v = kernel::kvec![0, 1, 2]?;
> +/// let iter = v.into_iter();
> +///
> +/// # Ok::<(), Error>(())
> +/// ```
> +pub struct IntoIter<T, A: Allocator> {
> +    ptr: *mut T,
> +    buf: NonNull<T>,

No invariants for these two fields?

> +    len: usize,
> +    cap: usize,
> +    _p: PhantomData<A>,
> +}
> +
> +impl<T, A> IntoIter<T, A>
> +where
> +    A: Allocator,
> +{
> +    fn as_raw_mut_slice(&mut self) -> *mut [T] {
> +        ptr::slice_from_raw_parts_mut(self.ptr, self.len)
> +    }
> +}
> +
> +impl<T, A> Iterator for IntoIter<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Item = T;
> +
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let v = kernel::kvec![1, 2, 3]?;
> +    /// let mut it = v.into_iter();
> +    ///
> +    /// assert_eq!(it.next(), Some(1));
> +    /// assert_eq!(it.next(), Some(2));
> +    /// assert_eq!(it.next(), Some(3));
> +    /// assert_eq!(it.next(), None);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```

AFAIK documentation on functions in trait implementations won't show up
in rustdoc (I just checked this). So I would remove it.

> +    fn next(&mut self) -> Option<T> {
> +        if self.len == 0 {
> +            return None;
> +        }
> +
> +        let ptr = self.ptr;
> +        if !Vec::<T, A>::is_zst() {
> +            // SAFETY: We can't overflow; `end` is guaranteed to mark the end of the buffer.
> +            unsafe { self.ptr = self.ptr.add(1) };
> +        } else {
> +            // For ZST `ptr` has to stay where it is to remain aligned, so we just reduce `self.len`
> +            // by 1.

Note that `<*mut T>::add` advances the pointer by `size_of::<T>()`
bytes. So in the case that `T` is a ZST, it won't be advanced.
So you could remove this `if`.

> +        }
> +        self.len -= 1;
> +
> +        // SAFETY: `ptr` is guaranteed to point at a valid element within the buffer.
> +        Some(unsafe { ptr.read() })
> +    }
> +
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let v: KVec<u32> = kernel::kvec![1, 2, 3]?;
> +    /// let mut iter = v.into_iter();
> +    /// let size = iter.size_hint().0;
> +    ///
> +    /// iter.next();
> +    /// assert_eq!(iter.size_hint().0, size - 1);
> +    ///
> +    /// iter.next();
> +    /// assert_eq!(iter.size_hint().0, size - 2);
> +    ///
> +    /// iter.next();
> +    /// assert_eq!(iter.size_hint().0, size - 3);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    fn size_hint(&self) -> (usize, Option<usize>) {
> +        (self.len, Some(self.len))
> +    }
> +}
> +
> +impl<T, A> Drop for IntoIter<T, A>
> +where
> +    A: Allocator,
> +{
> +    fn drop(&mut self) {
> +        // SAFETY: Drop the remaining vector's elements in place, before we free the backing
> +        // memory.

This comment explains why you are doing it, not why it's ok to do it.

> +        unsafe { ptr::drop_in_place(self.as_raw_mut_slice()) };
> +
> +        // If `cap == 0` we never allocated any memory in the first place.
> +        if self.cap != 0 {
> +            // SAFETY: `self.buf` was previously allocated with `A`.
> +            unsafe { A::free(self.buf.cast()) };
> +        }
> +    }
> +}
> +
> +impl<T, A> IntoIterator for Vec<T, A>
> +where
> +    A: Allocator,
> +{
> +    type Item = T;
> +    type IntoIter = IntoIter<T, A>;
> +
> +    /// Consumes the `Vec<T, A>` and creates an `Iterator`, which moves each value out of the
> +    /// vector (from start to end).
> +    ///
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let v = kernel::kvec![1, 2]?;
> +    /// let mut v_iter = v.into_iter();
> +    ///
> +    /// let first_element: Option<u32> = v_iter.next();
> +    ///
> +    /// assert_eq!(first_element, Some(1));
> +    /// assert_eq!(v_iter.next(), Some(2));
> +    /// assert_eq!(v_iter.next(), None);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    ///
> +    /// ```
> +    /// let v = kernel::kvec![];
> +    /// let mut v_iter = v.into_iter();
> +    ///
> +    /// let first_element: Option<u32> = v_iter.next();
> +    ///
> +    /// assert_eq!(first_element, None);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```

I feel a bit bad that you wrote all of this nice documentation for
functions that receive their documentation from the trait...

---
Cheers,
Benno

> +    #[inline]
> +    fn into_iter(self) -> Self::IntoIter {
> +        let (ptr, len, cap) = self.into_raw_parts();
> +
> +        IntoIter {
> +            ptr,
> +            // SAFETY: `ptr` is either a dangling pointer or a pointer to a valid memory
> +            // allocation, allocated with `A`.
> +            buf: unsafe { NonNull::new_unchecked(ptr) },
> +            len,
> +            cap,
> +            _p: PhantomData::<A>,
> +        }
> +    }
> +}
> --
> 2.46.0
> 

Re: [PATCH v6 14/26] rust: alloc: implement `IntoIterator` for `Vec`

[Danilo Krummrich]

On Tue, Sep 10, 2024 at 08:04:27PM +0000, Benno Lossin wrote:
> On 16.08.24 02:10, Danilo Krummrich wrote:
> > Implement `IntoIterator` for `Vec`, `Vec`'s `IntoIter` type, as well as
> > `Iterator` for `IntoIter`.
> > 
> > `Vec::into_iter` disassembles the `Vec` into its raw parts; additionally,
> > `IntoIter` keeps track of a separate pointer, which is incremented
> > correspondingsly as the iterator advances, while the length, or the count
> > of elements, is decremented.
> > 
> > This also means that `IntoIter` takes the ownership of the backing
> > buffer and is responsible to drop the remaining elements and free the
> > backing buffer, if it's dropped.
> > 
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > ---
> >  rust/kernel/alloc.rs      |   1 +
> >  rust/kernel/alloc/kvec.rs | 184 ++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 185 insertions(+)
> > 
> > diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> > index e88c7e10ee9b..4ff4df4597a3 100644
> > --- a/rust/kernel/alloc.rs
> > +++ b/rust/kernel/alloc.rs
> > @@ -19,6 +19,7 @@
> >  pub use self::kbox::KVBox;
> >  pub use self::kbox::VBox;
> > 
> > +pub use self::kvec::IntoIter;
> >  pub use self::kvec::KVVec;
> >  pub use self::kvec::KVec;
> >  pub use self::kvec::VVec;
> > diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
> > index 89afc0f25bd4..3b79f977b65e 100644
> > --- a/rust/kernel/alloc/kvec.rs
> > +++ b/rust/kernel/alloc/kvec.rs
> > @@ -11,6 +11,7 @@
> >      ops::DerefMut,
> >      ops::Index,
> >      ops::IndexMut,
> > +    ptr,
> >      ptr::NonNull,
> >      slice,
> >      slice::SliceIndex,
> > @@ -627,3 +628,186 @@ fn eq(&self, other: &$rhs) -> bool { self[..] == other[..] }
> >  __impl_slice_eq! { [A: Allocator] [T], Vec<U, A> }
> >  __impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, [U; N] }
> >  __impl_slice_eq! { [A: Allocator, const N: usize] Vec<T, A>, &[U; N] }
> > +
> > +impl<'a, T, A> IntoIterator for &'a Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    type Item = &'a T;
> > +    type IntoIter = slice::Iter<'a, T>;
> > +
> > +    fn into_iter(self) -> Self::IntoIter {
> > +        self.iter()
> > +    }
> > +}
> > +
> > +impl<'a, T, A: Allocator> IntoIterator for &'a mut Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    type Item = &'a mut T;
> > +    type IntoIter = slice::IterMut<'a, T>;
> > +
> > +    fn into_iter(self) -> Self::IntoIter {
> > +        self.iter_mut()
> > +    }
> > +}
> > +
> > +/// An `Iterator` implementation for `Vec<T,A>` that moves elements out of a vector.
> 
> Please make both links.
> 
> > +///
> > +/// This structure is created by the `Vec::into_iter` method on [`Vec`] (provided by the
> 
> Ditto.
> 
> > +/// [`IntoIterator`] trait).
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// let v = kernel::kvec![0, 1, 2]?;
> > +/// let iter = v.into_iter();
> > +///
> > +/// # Ok::<(), Error>(())
> > +/// ```
> > +pub struct IntoIter<T, A: Allocator> {
> > +    ptr: *mut T,
> > +    buf: NonNull<T>,
> 
> No invariants for these two fields?

Suggestions?

> 
> > +    len: usize,
> > +    cap: usize,
> > +    _p: PhantomData<A>,
> > +}
> > +
> > +impl<T, A> IntoIter<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    fn as_raw_mut_slice(&mut self) -> *mut [T] {
> > +        ptr::slice_from_raw_parts_mut(self.ptr, self.len)
> > +    }
> > +}
> > +
> > +impl<T, A> Iterator for IntoIter<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    type Item = T;
> > +
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let v = kernel::kvec![1, 2, 3]?;
> > +    /// let mut it = v.into_iter();
> > +    ///
> > +    /// assert_eq!(it.next(), Some(1));
> > +    /// assert_eq!(it.next(), Some(2));
> > +    /// assert_eq!(it.next(), Some(3));
> > +    /// assert_eq!(it.next(), None);
> > +    ///
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> 
> AFAIK documentation on functions in trait implementations won't show up
> in rustdoc (I just checked this). So I would remove it.

They don't, but the KUnit tests are still executed. :)

> 
> > +    fn next(&mut self) -> Option<T> {
> > +        if self.len == 0 {
> > +            return None;
> > +        }
> > +
> > +        let ptr = self.ptr;
> > +        if !Vec::<T, A>::is_zst() {
> > +            // SAFETY: We can't overflow; `end` is guaranteed to mark the end of the buffer.
> > +            unsafe { self.ptr = self.ptr.add(1) };
> > +        } else {
> > +            // For ZST `ptr` has to stay where it is to remain aligned, so we just reduce `self.len`
> > +            // by 1.
> 
> Note that `<*mut T>::add` advances the pointer by `size_of::<T>()`
> bytes. So in the case that `T` is a ZST, it won't be advanced.
> So you could remove this `if`.
> 
> > +        }
> > +        self.len -= 1;
> > +
> > +        // SAFETY: `ptr` is guaranteed to point at a valid element within the buffer.
> > +        Some(unsafe { ptr.read() })
> > +    }
> > +
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let v: KVec<u32> = kernel::kvec![1, 2, 3]?;
> > +    /// let mut iter = v.into_iter();
> > +    /// let size = iter.size_hint().0;
> > +    ///
> > +    /// iter.next();
> > +    /// assert_eq!(iter.size_hint().0, size - 1);
> > +    ///
> > +    /// iter.next();
> > +    /// assert_eq!(iter.size_hint().0, size - 2);
> > +    ///
> > +    /// iter.next();
> > +    /// assert_eq!(iter.size_hint().0, size - 3);
> > +    ///
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> > +    fn size_hint(&self) -> (usize, Option<usize>) {
> > +        (self.len, Some(self.len))
> > +    }
> > +}
> > +
> > +impl<T, A> Drop for IntoIter<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    fn drop(&mut self) {
> > +        // SAFETY: Drop the remaining vector's elements in place, before we free the backing
> > +        // memory.
> 
> This comment explains why you are doing it, not why it's ok to do it.
> 
> > +        unsafe { ptr::drop_in_place(self.as_raw_mut_slice()) };
> > +
> > +        // If `cap == 0` we never allocated any memory in the first place.
> > +        if self.cap != 0 {
> > +            // SAFETY: `self.buf` was previously allocated with `A`.
> > +            unsafe { A::free(self.buf.cast()) };
> > +        }
> > +    }
> > +}
> > +
> > +impl<T, A> IntoIterator for Vec<T, A>
> > +where
> > +    A: Allocator,
> > +{
> > +    type Item = T;
> > +    type IntoIter = IntoIter<T, A>;
> > +
> > +    /// Consumes the `Vec<T, A>` and creates an `Iterator`, which moves each value out of the
> > +    /// vector (from start to end).
> > +    ///
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let v = kernel::kvec![1, 2]?;
> > +    /// let mut v_iter = v.into_iter();
> > +    ///
> > +    /// let first_element: Option<u32> = v_iter.next();
> > +    ///
> > +    /// assert_eq!(first_element, Some(1));
> > +    /// assert_eq!(v_iter.next(), Some(2));
> > +    /// assert_eq!(v_iter.next(), None);
> > +    ///
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> > +    ///
> > +    /// ```
> > +    /// let v = kernel::kvec![];
> > +    /// let mut v_iter = v.into_iter();
> > +    ///
> > +    /// let first_element: Option<u32> = v_iter.next();
> > +    ///
> > +    /// assert_eq!(first_element, None);
> > +    ///
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> 
> I feel a bit bad that you wrote all of this nice documentation for
> functions that receive their documentation from the trait...

No worries, I really only added them for the KUnit tests.

> 
> ---
> Cheers,
> Benno
> 
> > +    #[inline]
> > +    fn into_iter(self) -> Self::IntoIter {
> > +        let (ptr, len, cap) = self.into_raw_parts();
> > +
> > +        IntoIter {
> > +            ptr,
> > +            // SAFETY: `ptr` is either a dangling pointer or a pointer to a valid memory
> > +            // allocation, allocated with `A`.
> > +            buf: unsafe { NonNull::new_unchecked(ptr) },
> > +            len,
> > +            cap,
> > +            _p: PhantomData::<A>,
> > +        }
> > +    }
> > +}
> > --
> > 2.46.0
> > 
> 

Re: [PATCH v6 14/26] rust: alloc: implement `IntoIterator` for `Vec`

[Benno Lossin]

On 11.09.24 01:39, Danilo Krummrich wrote:
> On Tue, Sep 10, 2024 at 08:04:27PM +0000, Benno Lossin wrote:
>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>> +/// [`IntoIterator`] trait).
>>> +///
>>> +/// # Examples
>>> +///
>>> +/// ```
>>> +/// let v = kernel::kvec![0, 1, 2]?;
>>> +/// let iter = v.into_iter();
>>> +///
>>> +/// # Ok::<(), Error>(())
>>> +/// ```
>>> +pub struct IntoIter<T, A: Allocator> {
>>> +    ptr: *mut T,
>>> +    buf: NonNull<T>,
>>
>> No invariants for these two fields?
> 
> Suggestions?

When determining the invariants, I look at the places where you would
want to use them, ie the `SAFETY` comments that use these fields:
- for `buf` you only use it to free the backing allocation, so you only
  need that it has been allocated by `A` if `cap != 0`.
- for `ptr` you need that it is valid for reads for `size_of::<T>() *
  length` bytes.

So I would put those two things into invariants.

>>> +    len: usize,
>>> +    cap: usize,
>>> +    _p: PhantomData<A>,
>>> +}
>>> +
>>> +impl<T, A> IntoIter<T, A>
>>> +where
>>> +    A: Allocator,
>>> +{
>>> +    fn as_raw_mut_slice(&mut self) -> *mut [T] {
>>> +        ptr::slice_from_raw_parts_mut(self.ptr, self.len)
>>> +    }
>>> +}
>>> +
>>> +impl<T, A> Iterator for IntoIter<T, A>
>>> +where
>>> +    A: Allocator,
>>> +{
>>> +    type Item = T;
>>> +
>>> +    /// # Examples
>>> +    ///
>>> +    /// ```
>>> +    /// let v = kernel::kvec![1, 2, 3]?;
>>> +    /// let mut it = v.into_iter();
>>> +    ///
>>> +    /// assert_eq!(it.next(), Some(1));
>>> +    /// assert_eq!(it.next(), Some(2));
>>> +    /// assert_eq!(it.next(), Some(3));
>>> +    /// assert_eq!(it.next(), None);
>>> +    ///
>>> +    /// # Ok::<(), Error>(())
>>> +    /// ```
>>
>> AFAIK documentation on functions in trait implementations won't show up
>> in rustdoc (I just checked this). So I would remove it.
> 
> They don't, but the KUnit tests are still executed. :)

Oh I see, then may I suggest moving them to the module documentation or
put them onto `Vec`, that way people can also read them :)

---
Cheers,
Benno

Re: [PATCH v6 14/26] rust: alloc: implement `IntoIterator` for `Vec`

[Danilo Krummrich]

On Wed, Sep 11, 2024 at 08:52:03AM +0000, Benno Lossin wrote:
> On 11.09.24 01:39, Danilo Krummrich wrote:
> > On Tue, Sep 10, 2024 at 08:04:27PM +0000, Benno Lossin wrote:
> >> On 16.08.24 02:10, Danilo Krummrich wrote:
> >>> +/// [`IntoIterator`] trait).
> >>> +///
> >>> +/// # Examples
> >>> +///
> >>> +/// ```
> >>> +/// let v = kernel::kvec![0, 1, 2]?;
> >>> +/// let iter = v.into_iter();
> >>> +///
> >>> +/// # Ok::<(), Error>(())
> >>> +/// ```
> >>> +pub struct IntoIter<T, A: Allocator> {
> >>> +    ptr: *mut T,
> >>> +    buf: NonNull<T>,
> >>
> >> No invariants for these two fields?
> > 
> > Suggestions?
> 
> When determining the invariants, I look at the places where you would
> want to use them, ie the `SAFETY` comments that use these fields:
> - for `buf` you only use it to free the backing allocation, so you only
>   need that it has been allocated by `A` if `cap != 0`.
> - for `ptr` you need that it is valid for reads for `size_of::<T>() *
>   length` bytes.
> 
> So I would put those two things into invariants.
> 
> >>> +    len: usize,
> >>> +    cap: usize,
> >>> +    _p: PhantomData<A>,
> >>> +}
> >>> +
> >>> +impl<T, A> IntoIter<T, A>
> >>> +where
> >>> +    A: Allocator,
> >>> +{
> >>> +    fn as_raw_mut_slice(&mut self) -> *mut [T] {
> >>> +        ptr::slice_from_raw_parts_mut(self.ptr, self.len)
> >>> +    }
> >>> +}
> >>> +
> >>> +impl<T, A> Iterator for IntoIter<T, A>
> >>> +where
> >>> +    A: Allocator,
> >>> +{
> >>> +    type Item = T;
> >>> +
> >>> +    /// # Examples
> >>> +    ///
> >>> +    /// ```
> >>> +    /// let v = kernel::kvec![1, 2, 3]?;
> >>> +    /// let mut it = v.into_iter();
> >>> +    ///
> >>> +    /// assert_eq!(it.next(), Some(1));
> >>> +    /// assert_eq!(it.next(), Some(2));
> >>> +    /// assert_eq!(it.next(), Some(3));
> >>> +    /// assert_eq!(it.next(), None);
> >>> +    ///
> >>> +    /// # Ok::<(), Error>(())
> >>> +    /// ```
> >>
> >> AFAIK documentation on functions in trait implementations won't show up
> >> in rustdoc (I just checked this). So I would remove it.
> > 
> > They don't, but the KUnit tests are still executed. :)
> 
> Oh I see, then may I suggest moving them to the module documentation or
> put them onto `Vec`, that way people can also read them :)

Hm, I'd rather keep them close on the functions they're testing. Those examples
probably don't have a huge documentation purpose. As you've said, the trait is
documented already and has examples.

> 
> ---
> Cheers,
> Benno
> 
> 

[PATCH v6 15/26] rust: alloc: implement `collect` for `IntoIter`

[Danilo Krummrich]

Currently, we can't implement `FromIterator`. There are a couple of
issues with this trait in the kernel, namely:

  - Rust's specialization feature is unstable. This prevents us to
    optimze for the special case where `I::IntoIter` equals `Vec`'s
    `IntoIter` type.
  - We also can't use `I::IntoIter`'s type ID either to work around this,
    since `FromIterator` doesn't require this type to be `'static`.
  - `FromIterator::from_iter` does return `Self` instead of
    `Result<Self, AllocError>`, hence we can't properly handle allocation
    failures.
  - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle
    additional allocation flags.

Instead, provide `IntoIter::collect`, such that we can at least convert
`IntoIter` into a `Vec` again.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc/kvec.rs | 78 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 78 insertions(+)

diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
index 3b79f977b65e..ad96f4c3af9e 100644
--- a/rust/kernel/alloc/kvec.rs
+++ b/rust/kernel/alloc/kvec.rs
@@ -681,6 +681,84 @@ impl<T, A> IntoIter<T, A>
     fn as_raw_mut_slice(&mut self) -> *mut [T] {
         ptr::slice_from_raw_parts_mut(self.ptr, self.len)
     }
+
+    fn into_raw_parts(self) -> (*mut T, NonNull<T>, usize, usize) {
+        let me = ManuallyDrop::new(self);
+        let ptr = me.ptr;
+        let buf = me.buf;
+        let len = me.len;
+        let cap = me.cap;
+        (ptr, buf, len, cap)
+    }
+
+    /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`.
+    ///
+    /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait
+    /// in the kernel, namely:
+    ///
+    /// - Rust's specialization feature is unstable. This prevents us to optimze for the special
+    ///   case where `I::IntoIter` equals `Vec`'s `IntoIter` type.
+    /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator`
+    ///   doesn't require this type to be `'static`.
+    /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence
+    ///   we can't properly handle allocation failures.
+    /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation
+    ///   flags.
+    ///
+    /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a
+    /// `Vec` again.
+    ///
+    /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing
+    /// buffer. However, this backing buffer may be shrunk to the actual count of elements.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let v = kernel::kvec![1, 2, 3]?;
+    /// let mut it = v.into_iter();
+    ///
+    /// assert_eq!(it.next(), Some(1));
+    ///
+    /// let v = it.collect(GFP_KERNEL);
+    /// assert_eq!(v, [2, 3]);
+    ///
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub fn collect(self, flags: Flags) -> Vec<T, A> {
+        let (mut ptr, buf, len, mut cap) = self.into_raw_parts();
+        let has_advanced = ptr != buf.as_ptr();
+
+        if has_advanced {
+            // SAFETY: Copy the contents we have advanced to at the beginning of the buffer.
+            // `ptr` is guaranteed to be between `buf` and `buf.add(cap)` and `ptr.add(len)` is
+            // guaranteed to be smaller than `buf.add(cap)`.
+            unsafe { ptr::copy(ptr, buf.as_ptr(), len) };
+            ptr = buf.as_ptr();
+        }
+
+        // This can never fail, `len` is guaranteed to be smaller than `cap`.
+        let layout = core::alloc::Layout::array::<T>(len).unwrap();
+
+        // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be
+        // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves
+        // it as it is.
+        ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } {
+            // If we fail to shrink, which likely can't even happen, continue with the existing
+            // buffer.
+            Err(_) => ptr,
+            Ok(ptr) => {
+                cap = len;
+                ptr.as_ptr().cast()
+            }
+        };
+
+        // SAFETY: If the iterator has been advanced, the advanced elements have been copied to
+        // the beginning of the buffer and `len` has been adjusted accordingly. `ptr` is guaranteed
+        // to point to the start of the backing buffer. `cap` is either the original capacity or,
+        // after shrinking the buffer, equal to `len`. `alloc` is guaranteed to be unchanged since
+        // `into_iter` has been called on the original `Vec`.
+        unsafe { Vec::from_raw_parts(ptr, len, cap) }
+    }
 }
 
 impl<T, A> Iterator for IntoIter<T, A>
-- 
2.46.0

Re: [PATCH v6 15/26] rust: alloc: implement `collect` for `IntoIter`

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> Currently, we can't implement `FromIterator`. There are a couple of
> issues with this trait in the kernel, namely:
> 
>   - Rust's specialization feature is unstable. This prevents us to
>     optimze for the special case where `I::IntoIter` equals `Vec`'s
>     `IntoIter` type.
>   - We also can't use `I::IntoIter`'s type ID either to work around this,
>     since `FromIterator` doesn't require this type to be `'static`.
>   - `FromIterator::from_iter` does return `Self` instead of
>     `Result<Self, AllocError>`, hence we can't properly handle allocation
>     failures.
>   - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle
>     additional allocation flags.
> 
> Instead, provide `IntoIter::collect`, such that we can at least convert
> `IntoIter` into a `Vec` again.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
>  rust/kernel/alloc/kvec.rs | 78 +++++++++++++++++++++++++++++++++++++++
>  1 file changed, 78 insertions(+)
> 
> diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
> index 3b79f977b65e..ad96f4c3af9e 100644
> --- a/rust/kernel/alloc/kvec.rs
> +++ b/rust/kernel/alloc/kvec.rs
> @@ -681,6 +681,84 @@ impl<T, A> IntoIter<T, A>
>      fn as_raw_mut_slice(&mut self) -> *mut [T] {
>          ptr::slice_from_raw_parts_mut(self.ptr, self.len)
>      }
> +
> +    fn into_raw_parts(self) -> (*mut T, NonNull<T>, usize, usize) {
> +        let me = ManuallyDrop::new(self);
> +        let ptr = me.ptr;
> +        let buf = me.buf;
> +        let len = me.len;
> +        let cap = me.cap;
> +        (ptr, buf, len, cap)
> +    }
> +
> +    /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`.
> +    ///
> +    /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait
> +    /// in the kernel, namely:
> +    ///
> +    /// - Rust's specialization feature is unstable. This prevents us to optimze for the special
> +    ///   case where `I::IntoIter` equals `Vec`'s `IntoIter` type.
> +    /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator`
> +    ///   doesn't require this type to be `'static`.
> +    /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence
> +    ///   we can't properly handle allocation failures.
> +    /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation
> +    ///   flags.
> +    ///
> +    /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a
> +    /// `Vec` again.

I think it's great that you include this in the code, but I don't think
that it should be visible in the documentation, can you move it under
the `Examples` section and turn it into normal comments?

> +    ///
> +    /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing
> +    /// buffer. However, this backing buffer may be shrunk to the actual count of elements.
> +    ///
> +    /// # Examples
> +    ///
> +    /// ```
> +    /// let v = kernel::kvec![1, 2, 3]?;
> +    /// let mut it = v.into_iter();
> +    ///
> +    /// assert_eq!(it.next(), Some(1));
> +    ///
> +    /// let v = it.collect(GFP_KERNEL);
> +    /// assert_eq!(v, [2, 3]);
> +    ///
> +    /// # Ok::<(), Error>(())
> +    /// ```
> +    pub fn collect(self, flags: Flags) -> Vec<T, A> {
> +        let (mut ptr, buf, len, mut cap) = self.into_raw_parts();
> +        let has_advanced = ptr != buf.as_ptr();
> +
> +        if has_advanced {
> +            // SAFETY: Copy the contents we have advanced to at the beginning of the buffer.

This first sentence should not be part of the SAFETY comment.

> +            // `ptr` is guaranteed to be between `buf` and `buf.add(cap)` and `ptr.add(len)` is
> +            // guaranteed to be smaller than `buf.add(cap)`.

This doesn't justify all the requirements documented in [1].

[1]: https://doc.rust-lang.org/core/ptr/fn.copy.html#safety

> +            unsafe { ptr::copy(ptr, buf.as_ptr(), len) };
> +            ptr = buf.as_ptr();
> +        }
> +
> +        // This can never fail, `len` is guaranteed to be smaller than `cap`.
> +        let layout = core::alloc::Layout::array::<T>(len).unwrap();
> +
> +        // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be
> +        // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves
> +        // it as it is.
> +        ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } {
> +            // If we fail to shrink, which likely can't even happen, continue with the existing
> +            // buffer.
> +            Err(_) => ptr,
> +            Ok(ptr) => {
> +                cap = len;
> +                ptr.as_ptr().cast()
> +            }
> +        };
> +
> +        // SAFETY: If the iterator has been advanced, the advanced elements have been copied to
> +        // the beginning of the buffer and `len` has been adjusted accordingly. `ptr` is guaranteed
> +        // to point to the start of the backing buffer. `cap` is either the original capacity or,
> +        // after shrinking the buffer, equal to `len`. `alloc` is guaranteed to be unchanged since
> +        // `into_iter` has been called on the original `Vec`.

Turn this into bullet points please.

---
Cheers,
Benno

> +        unsafe { Vec::from_raw_parts(ptr, len, cap) }
> +    }
>  }
> 
>  impl<T, A> Iterator for IntoIter<T, A>
> --
> 2.46.0
> 

Re: [PATCH v6 15/26] rust: alloc: implement `collect` for `IntoIter`

[Danilo Krummrich]

On Tue, Sep 10, 2024 at 08:12:24PM +0000, Benno Lossin wrote:
> On 16.08.24 02:10, Danilo Krummrich wrote:
> > Currently, we can't implement `FromIterator`. There are a couple of
> > issues with this trait in the kernel, namely:
> > 
> >   - Rust's specialization feature is unstable. This prevents us to
> >     optimze for the special case where `I::IntoIter` equals `Vec`'s
> >     `IntoIter` type.
> >   - We also can't use `I::IntoIter`'s type ID either to work around this,
> >     since `FromIterator` doesn't require this type to be `'static`.
> >   - `FromIterator::from_iter` does return `Self` instead of
> >     `Result<Self, AllocError>`, hence we can't properly handle allocation
> >     failures.
> >   - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle
> >     additional allocation flags.
> > 
> > Instead, provide `IntoIter::collect`, such that we can at least convert
> > `IntoIter` into a `Vec` again.
> > 
> > Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > ---
> >  rust/kernel/alloc/kvec.rs | 78 +++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 78 insertions(+)
> > 
> > diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
> > index 3b79f977b65e..ad96f4c3af9e 100644
> > --- a/rust/kernel/alloc/kvec.rs
> > +++ b/rust/kernel/alloc/kvec.rs
> > @@ -681,6 +681,84 @@ impl<T, A> IntoIter<T, A>
> >      fn as_raw_mut_slice(&mut self) -> *mut [T] {
> >          ptr::slice_from_raw_parts_mut(self.ptr, self.len)
> >      }
> > +
> > +    fn into_raw_parts(self) -> (*mut T, NonNull<T>, usize, usize) {
> > +        let me = ManuallyDrop::new(self);
> > +        let ptr = me.ptr;
> > +        let buf = me.buf;
> > +        let len = me.len;
> > +        let cap = me.cap;
> > +        (ptr, buf, len, cap)
> > +    }
> > +
> > +    /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`.
> > +    ///
> > +    /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait
> > +    /// in the kernel, namely:
> > +    ///
> > +    /// - Rust's specialization feature is unstable. This prevents us to optimze for the special
> > +    ///   case where `I::IntoIter` equals `Vec`'s `IntoIter` type.
> > +    /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator`
> > +    ///   doesn't require this type to be `'static`.
> > +    /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence
> > +    ///   we can't properly handle allocation failures.
> > +    /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation
> > +    ///   flags.
> > +    ///
> > +    /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a
> > +    /// `Vec` again.
> 
> I think it's great that you include this in the code, but I don't think
> that it should be visible in the documentation,

Why not? I think this information is valuable for users of this API.

> can you move it under
> the `Examples` section and turn it into normal comments?
> 
> > +    ///
> > +    /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing
> > +    /// buffer. However, this backing buffer may be shrunk to the actual count of elements.
> > +    ///
> > +    /// # Examples
> > +    ///
> > +    /// ```
> > +    /// let v = kernel::kvec![1, 2, 3]?;
> > +    /// let mut it = v.into_iter();
> > +    ///
> > +    /// assert_eq!(it.next(), Some(1));
> > +    ///
> > +    /// let v = it.collect(GFP_KERNEL);
> > +    /// assert_eq!(v, [2, 3]);
> > +    ///
> > +    /// # Ok::<(), Error>(())
> > +    /// ```
> > +    pub fn collect(self, flags: Flags) -> Vec<T, A> {
> > +        let (mut ptr, buf, len, mut cap) = self.into_raw_parts();
> > +        let has_advanced = ptr != buf.as_ptr();
> > +
> > +        if has_advanced {
> > +            // SAFETY: Copy the contents we have advanced to at the beginning of the buffer.
> 
> This first sentence should not be part of the SAFETY comment.
> 
> > +            // `ptr` is guaranteed to be between `buf` and `buf.add(cap)` and `ptr.add(len)` is
> > +            // guaranteed to be smaller than `buf.add(cap)`.
> 
> This doesn't justify all the requirements documented in [1].
> 
> [1]: https://doc.rust-lang.org/core/ptr/fn.copy.html#safety
> 
> > +            unsafe { ptr::copy(ptr, buf.as_ptr(), len) };
> > +            ptr = buf.as_ptr();
> > +        }
> > +
> > +        // This can never fail, `len` is guaranteed to be smaller than `cap`.
> > +        let layout = core::alloc::Layout::array::<T>(len).unwrap();
> > +
> > +        // SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be
> > +        // smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves
> > +        // it as it is.
> > +        ptr = match unsafe { A::realloc(Some(buf.cast()), layout, flags) } {
> > +            // If we fail to shrink, which likely can't even happen, continue with the existing
> > +            // buffer.
> > +            Err(_) => ptr,
> > +            Ok(ptr) => {
> > +                cap = len;
> > +                ptr.as_ptr().cast()
> > +            }
> > +        };
> > +
> > +        // SAFETY: If the iterator has been advanced, the advanced elements have been copied to
> > +        // the beginning of the buffer and `len` has been adjusted accordingly. `ptr` is guaranteed
> > +        // to point to the start of the backing buffer. `cap` is either the original capacity or,
> > +        // after shrinking the buffer, equal to `len`. `alloc` is guaranteed to be unchanged since
> > +        // `into_iter` has been called on the original `Vec`.
> 
> Turn this into bullet points please.
> 
> ---
> Cheers,
> Benno
> 
> > +        unsafe { Vec::from_raw_parts(ptr, len, cap) }
> > +    }
> >  }
> > 
> >  impl<T, A> Iterator for IntoIter<T, A>
> > --
> > 2.46.0
> > 
> 

Re: [PATCH v6 15/26] rust: alloc: implement `collect` for `IntoIter`

[Benno Lossin]

On 11.09.24 02:22, Danilo Krummrich wrote:
> On Tue, Sep 10, 2024 at 08:12:24PM +0000, Benno Lossin wrote:
>> On 16.08.24 02:10, Danilo Krummrich wrote:
>>> +    /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`.
>>> +    ///
>>> +    /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait
>>> +    /// in the kernel, namely:
>>> +    ///
>>> +    /// - Rust's specialization feature is unstable. This prevents us to optimze for the special
>>> +    ///   case where `I::IntoIter` equals `Vec`'s `IntoIter` type.
>>> +    /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator`
>>> +    ///   doesn't require this type to be `'static`.
>>> +    /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence
>>> +    ///   we can't properly handle allocation failures.
>>> +    /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation
>>> +    ///   flags.
>>> +    ///
>>> +    /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a
>>> +    /// `Vec` again.
>>
>> I think it's great that you include this in the code, but I don't think
>> that it should be visible in the documentation,
> 
> Why not? I think this information is valuable for users of this API.

If you want to keep it, then I don't mind, but I would still move it
underneath `Examples` and add a section header `# Implementation
Details` or similar.

---
Cheers,
Benno

>> can you move it under
>> the `Examples` section and turn it into normal comments?
>>
>>> +    ///
>>> +    /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing
>>> +    /// buffer. However, this backing buffer may be shrunk to the actual count of elements.
>>> +    ///
>>> +    /// # Examples
>>> +    ///
>>> +    /// ```
>>> +    /// let v = kernel::kvec![1, 2, 3]?;
>>> +    /// let mut it = v.into_iter();
>>> +    ///
>>> +    /// assert_eq!(it.next(), Some(1));
>>> +    ///
>>> +    /// let v = it.collect(GFP_KERNEL);
>>> +    /// assert_eq!(v, [2, 3]);
>>> +    ///
>>> +    /// # Ok::<(), Error>(())
>>> +    /// ```
>>> +    pub fn collect(self, flags: Flags) -> Vec<T, A> {

Re: [PATCH v6 15/26] rust: alloc: implement `collect` for `IntoIter`

[Danilo Krummrich]

On Wed, Sep 11, 2024 at 08:53:24AM +0000, Benno Lossin wrote:
> On 11.09.24 02:22, Danilo Krummrich wrote:
> > On Tue, Sep 10, 2024 at 08:12:24PM +0000, Benno Lossin wrote:
> >> On 16.08.24 02:10, Danilo Krummrich wrote:
> >>> +    /// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`.
> >>> +    ///
> >>> +    /// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait
> >>> +    /// in the kernel, namely:
> >>> +    ///
> >>> +    /// - Rust's specialization feature is unstable. This prevents us to optimze for the special
> >>> +    ///   case where `I::IntoIter` equals `Vec`'s `IntoIter` type.
> >>> +    /// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator`
> >>> +    ///   doesn't require this type to be `'static`.
> >>> +    /// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence
> >>> +    ///   we can't properly handle allocation failures.
> >>> +    /// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation
> >>> +    ///   flags.
> >>> +    ///
> >>> +    /// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a
> >>> +    /// `Vec` again.
> >>
> >> I think it's great that you include this in the code, but I don't think
> >> that it should be visible in the documentation,
> > 
> > Why not? I think this information is valuable for users of this API.
> 
> If you want to keep it, then I don't mind, but I would still move it
> underneath `Examples` and add a section header `# Implementation
> Details` or similar.

Sure, we can do that.

> 
> ---
> Cheers,
> Benno
> 
> >> can you move it under
> >> the `Examples` section and turn it into normal comments?
> >>
> >>> +    ///
> >>> +    /// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing
> >>> +    /// buffer. However, this backing buffer may be shrunk to the actual count of elements.
> >>> +    ///
> >>> +    /// # Examples
> >>> +    ///
> >>> +    /// ```
> >>> +    /// let v = kernel::kvec![1, 2, 3]?;
> >>> +    /// let mut it = v.into_iter();
> >>> +    ///
> >>> +    /// assert_eq!(it.next(), Some(1));
> >>> +    ///
> >>> +    /// let v = it.collect(GFP_KERNEL);
> >>> +    /// assert_eq!(v, [2, 3]);
> >>> +    ///
> >>> +    /// # Ok::<(), Error>(())
> >>> +    /// ```
> >>> +    pub fn collect(self, flags: Flags) -> Vec<T, A> {
> 

[PATCH v6 16/26] rust: treewide: switch to the kernel `Vec` type

[Danilo Krummrich]

Now that we got the kernel `Vec` in place, convert all existing `Vec`
users to make use of it.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/str.rs            | 12 +++++-------
 rust/kernel/sync/locked_by.rs |  2 +-
 rust/kernel/types.rs          |  2 +-
 rust/kernel/uaccess.rs        | 17 +++++++----------
 rust/macros/lib.rs            |  6 +++---
 samples/rust/rust_minimal.rs  |  4 ++--
 6 files changed, 19 insertions(+), 24 deletions(-)

diff --git a/rust/kernel/str.rs b/rust/kernel/str.rs
index bb8d4f41475b..209ad1bb1a49 100644
--- a/rust/kernel/str.rs
+++ b/rust/kernel/str.rs
@@ -2,8 +2,7 @@
 
 //! String representations.
 
-use crate::alloc::{flags::*, vec_ext::VecExt, AllocError};
-use alloc::vec::Vec;
+use crate::alloc::{flags::*, AllocError, KVec};
 use core::fmt::{self, Write};
 use core::ops::{self, Deref, DerefMut, Index};
 
@@ -790,7 +789,7 @@ fn write_str(&mut self, s: &str) -> fmt::Result {
 /// assert_eq!(s.is_ok(), false);
 /// ```
 pub struct CString {
-    buf: Vec<u8>,
+    buf: KVec<u8>,
 }
 
 impl CString {
@@ -803,7 +802,7 @@ pub fn try_from_fmt(args: fmt::Arguments<'_>) -> Result<Self, Error> {
         let size = f.bytes_written();
 
         // Allocate a vector with the required number of bytes, and write to it.
-        let mut buf = <Vec<_> as VecExt<_>>::with_capacity(size, GFP_KERNEL)?;
+        let mut buf = KVec::with_capacity(size, GFP_KERNEL)?;
         // SAFETY: The buffer stored in `buf` is at least of size `size` and is valid for writes.
         let mut f = unsafe { Formatter::from_buffer(buf.as_mut_ptr(), size) };
         f.write_fmt(args)?;
@@ -850,10 +849,9 @@ impl<'a> TryFrom<&'a CStr> for CString {
     type Error = AllocError;
 
     fn try_from(cstr: &'a CStr) -> Result<CString, AllocError> {
-        let mut buf = Vec::new();
+        let mut buf = KVec::new();
 
-        <Vec<_> as VecExt<_>>::extend_from_slice(&mut buf, cstr.as_bytes_with_nul(), GFP_KERNEL)
-            .map_err(|_| AllocError)?;
+        buf.extend_from_slice(cstr.as_bytes_with_nul(), GFP_KERNEL)?;
 
         // INVARIANT: The `CStr` and `CString` types have the same invariants for
         // the string data, and we copied it over without changes.
diff --git a/rust/kernel/sync/locked_by.rs b/rust/kernel/sync/locked_by.rs
index babc731bd5f6..b94517231fcc 100644
--- a/rust/kernel/sync/locked_by.rs
+++ b/rust/kernel/sync/locked_by.rs
@@ -43,7 +43,7 @@
 /// struct InnerDirectory {
 ///     /// The sum of the bytes used by all files.
 ///     bytes_used: u64,
-///     _files: Vec<File>,
+///     _files: KVec<File>,
 /// }
 ///
 /// struct Directory {
diff --git a/rust/kernel/types.rs b/rust/kernel/types.rs
index 234b615662a2..43f6efbe31c6 100644
--- a/rust/kernel/types.rs
+++ b/rust/kernel/types.rs
@@ -172,7 +172,7 @@ unsafe fn borrow_mut<'a>(_: *const core::ffi::c_void) -> Self::BorrowedMut<'a> {
 /// # use kernel::types::ScopeGuard;
 /// fn example3(arg: bool) -> Result {
 ///     let mut vec =
-///         ScopeGuard::new_with_data(Vec::new(), |v| pr_info!("vec had {} elements\n", v.len()));
+///         ScopeGuard::new_with_data(KVec::new(), |v| pr_info!("vec had {} elements\n", v.len()));
 ///
 ///     vec.push(10u8, GFP_KERNEL)?;
 ///     if arg {
diff --git a/rust/kernel/uaccess.rs b/rust/kernel/uaccess.rs
index e9347cff99ab..bc011061de45 100644
--- a/rust/kernel/uaccess.rs
+++ b/rust/kernel/uaccess.rs
@@ -11,7 +11,6 @@
     prelude::*,
     types::{AsBytes, FromBytes},
 };
-use alloc::vec::Vec;
 use core::ffi::{c_ulong, c_void};
 use core::mem::{size_of, MaybeUninit};
 
@@ -46,7 +45,6 @@
 /// every byte in the region.
 ///
 /// ```no_run
-/// use alloc::vec::Vec;
 /// use core::ffi::c_void;
 /// use kernel::error::Result;
 /// use kernel::uaccess::{UserPtr, UserSlice};
@@ -54,7 +52,7 @@
 /// fn bytes_add_one(uptr: UserPtr, len: usize) -> Result<()> {
 ///     let (read, mut write) = UserSlice::new(uptr, len).reader_writer();
 ///
-///     let mut buf = Vec::new();
+///     let mut buf = KVec::new();
 ///     read.read_all(&mut buf, GFP_KERNEL)?;
 ///
 ///     for b in &mut buf {
@@ -69,7 +67,6 @@
 /// Example illustrating a TOCTOU (time-of-check to time-of-use) bug.
 ///
 /// ```no_run
-/// use alloc::vec::Vec;
 /// use core::ffi::c_void;
 /// use kernel::error::{code::EINVAL, Result};
 /// use kernel::uaccess::{UserPtr, UserSlice};
@@ -78,21 +75,21 @@
 /// fn is_valid(uptr: UserPtr, len: usize) -> Result<bool> {
 ///     let read = UserSlice::new(uptr, len).reader();
 ///
-///     let mut buf = Vec::new();
+///     let mut buf = KVec::new();
 ///     read.read_all(&mut buf, GFP_KERNEL)?;
 ///
 ///     todo!()
 /// }
 ///
 /// /// Returns the bytes behind this user pointer if they are valid.
-/// fn get_bytes_if_valid(uptr: UserPtr, len: usize) -> Result<Vec<u8>> {
+/// fn get_bytes_if_valid(uptr: UserPtr, len: usize) -> Result<KVec<u8>> {
 ///     if !is_valid(uptr, len)? {
 ///         return Err(EINVAL);
 ///     }
 ///
 ///     let read = UserSlice::new(uptr, len).reader();
 ///
-///     let mut buf = Vec::new();
+///     let mut buf = KVec::new();
 ///     read.read_all(&mut buf, GFP_KERNEL)?;
 ///
 ///     // THIS IS A BUG! The bytes could have changed since we checked them.
@@ -130,7 +127,7 @@ pub fn new(ptr: UserPtr, length: usize) -> Self {
     /// Reads the entirety of the user slice, appending it to the end of the provided buffer.
     ///
     /// Fails with [`EFAULT`] if the read happens on a bad address.
-    pub fn read_all(self, buf: &mut Vec<u8>, flags: Flags) -> Result {
+    pub fn read_all(self, buf: &mut KVec<u8>, flags: Flags) -> Result {
         self.reader().read_all(buf, flags)
     }
 
@@ -291,9 +288,9 @@ pub fn read<T: FromBytes>(&mut self) -> Result<T> {
     /// Reads the entirety of the user slice, appending it to the end of the provided buffer.
     ///
     /// Fails with [`EFAULT`] if the read happens on a bad address.
-    pub fn read_all(mut self, buf: &mut Vec<u8>, flags: Flags) -> Result {
+    pub fn read_all(mut self, buf: &mut KVec<u8>, flags: Flags) -> Result {
         let len = self.length;
-        VecExt::<u8>::reserve(buf, len, flags)?;
+        buf.reserve(len, flags)?;
 
         // The call to `try_reserve` was successful, so the spare capacity is at least `len` bytes
         // long.
diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
index c327fe0617c8..a733168d40f9 100644
--- a/rust/macros/lib.rs
+++ b/rust/macros/lib.rs
@@ -238,7 +238,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {
 /// #[pin_data]
 /// struct DriverData {
 ///     #[pin]
-///     queue: Mutex<Vec<Command>>,
+///     queue: Mutex<KVec<Command>>,
 ///     buf: KBox<[u8; 1024 * 1024]>,
 /// }
 /// ```
@@ -247,7 +247,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {
 /// #[pin_data(PinnedDrop)]
 /// struct DriverData {
 ///     #[pin]
-///     queue: Mutex<Vec<Command>>,
+///     queue: Mutex<KVec<Command>>,
 ///     buf: KBox<[u8; 1024 * 1024]>,
 ///     raw_info: *mut Info,
 /// }
@@ -277,7 +277,7 @@ pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream {
 /// #[pin_data(PinnedDrop)]
 /// struct DriverData {
 ///     #[pin]
-///     queue: Mutex<Vec<Command>>,
+///     queue: Mutex<KVec<Command>>,
 ///     buf: KBox<[u8; 1024 * 1024]>,
 ///     raw_info: *mut Info,
 /// }
diff --git a/samples/rust/rust_minimal.rs b/samples/rust/rust_minimal.rs
index 2a9eaab62d1c..4aaf117bf8e3 100644
--- a/samples/rust/rust_minimal.rs
+++ b/samples/rust/rust_minimal.rs
@@ -13,7 +13,7 @@
 }
 
 struct RustMinimal {
-    numbers: Vec<i32>,
+    numbers: KVec<i32>,
 }
 
 impl kernel::Module for RustMinimal {
@@ -21,7 +21,7 @@ fn init(_module: &'static ThisModule) -> Result<Self> {
         pr_info!("Rust minimal sample (init)\n");
         pr_info!("Am I built-in? {}\n", !cfg!(MODULE));
 
-        let mut numbers = Vec::new();
+        let mut numbers = KVec::new();
         numbers.push(72, GFP_KERNEL)?;
         numbers.push(108, GFP_KERNEL)?;
         numbers.push(200, GFP_KERNEL)?;
-- 
2.46.0

Re: [PATCH v6 16/26] rust: treewide: switch to the kernel `Vec` type

[Benno Lossin]

On 16.08.24 02:10, Danilo Krummrich wrote:
> Now that we got the kernel `Vec` in place, convert all existing `Vec`
> users to make use of it.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

Reviewed-by: Benno Lossin <benno.lossin@proton.me>

---
Cheers,
Benno

> ---
>  rust/kernel/str.rs            | 12 +++++-------
>  rust/kernel/sync/locked_by.rs |  2 +-
>  rust/kernel/types.rs          |  2 +-
>  rust/kernel/uaccess.rs        | 17 +++++++----------
>  rust/macros/lib.rs            |  6 +++---
>  samples/rust/rust_minimal.rs  |  4 ++--
>  6 files changed, 19 insertions(+), 24 deletions(-)

[PATCH v6 17/26] rust: alloc: remove `VecExt` extension

[Danilo Krummrich]

Now that all existing `Vec` users were moved to the kernel `Vec` type,
remove the `VecExt` extension.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc.rs         |   1 -
 rust/kernel/alloc/vec_ext.rs | 185 -----------------------------------
 rust/kernel/prelude.rs       |   5 +-
 3 files changed, 1 insertion(+), 190 deletions(-)
 delete mode 100644 rust/kernel/alloc/vec_ext.rs

diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index 4ff4df4597a3..1feabc817d00 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -6,7 +6,6 @@
 pub mod allocator;
 pub mod kbox;
 pub mod kvec;
-pub mod vec_ext;
 
 #[cfg(any(test, testlib))]
 pub mod allocator_test;
diff --git a/rust/kernel/alloc/vec_ext.rs b/rust/kernel/alloc/vec_ext.rs
deleted file mode 100644
index 1297a4be32e8..000000000000
--- a/rust/kernel/alloc/vec_ext.rs
+++ /dev/null
@@ -1,185 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-
-//! Extensions to [`Vec`] for fallible allocations.
-
-use super::{AllocError, Flags};
-use alloc::vec::Vec;
-
-/// Extensions to [`Vec`].
-pub trait VecExt<T>: Sized {
-    /// Creates a new [`Vec`] instance with at least the given capacity.
-    ///
-    /// # Examples
-    ///
-    /// ```
-    /// let v = Vec::<u32>::with_capacity(20, GFP_KERNEL)?;
-    ///
-    /// assert!(v.capacity() >= 20);
-    /// # Ok::<(), Error>(())
-    /// ```
-    fn with_capacity(capacity: usize, flags: Flags) -> Result<Self, AllocError>;
-
-    /// Appends an element to the back of the [`Vec`] instance.
-    ///
-    /// # Examples
-    ///
-    /// ```
-    /// let mut v = Vec::new();
-    /// v.push(1, GFP_KERNEL)?;
-    /// assert_eq!(&v, &[1]);
-    ///
-    /// v.push(2, GFP_KERNEL)?;
-    /// assert_eq!(&v, &[1, 2]);
-    /// # Ok::<(), Error>(())
-    /// ```
-    fn push(&mut self, v: T, flags: Flags) -> Result<(), AllocError>;
-
-    /// Pushes clones of the elements of slice into the [`Vec`] instance.
-    ///
-    /// # Examples
-    ///
-    /// ```
-    /// let mut v = Vec::new();
-    /// v.push(1, GFP_KERNEL)?;
-    ///
-    /// v.extend_from_slice(&[20, 30, 40], GFP_KERNEL)?;
-    /// assert_eq!(&v, &[1, 20, 30, 40]);
-    ///
-    /// v.extend_from_slice(&[50, 60], GFP_KERNEL)?;
-    /// assert_eq!(&v, &[1, 20, 30, 40, 50, 60]);
-    /// # Ok::<(), Error>(())
-    /// ```
-    fn extend_from_slice(&mut self, other: &[T], flags: Flags) -> Result<(), AllocError>
-    where
-        T: Clone;
-
-    /// Ensures that the capacity exceeds the length by at least `additional` elements.
-    ///
-    /// # Examples
-    ///
-    /// ```
-    /// let mut v = Vec::new();
-    /// v.push(1, GFP_KERNEL)?;
-    ///
-    /// v.reserve(10, GFP_KERNEL)?;
-    /// let cap = v.capacity();
-    /// assert!(cap >= 10);
-    ///
-    /// v.reserve(10, GFP_KERNEL)?;
-    /// let new_cap = v.capacity();
-    /// assert_eq!(new_cap, cap);
-    ///
-    /// # Ok::<(), Error>(())
-    /// ```
-    fn reserve(&mut self, additional: usize, flags: Flags) -> Result<(), AllocError>;
-}
-
-impl<T> VecExt<T> for Vec<T> {
-    fn with_capacity(capacity: usize, flags: Flags) -> Result<Self, AllocError> {
-        let mut v = Vec::new();
-        <Self as VecExt<_>>::reserve(&mut v, capacity, flags)?;
-        Ok(v)
-    }
-
-    fn push(&mut self, v: T, flags: Flags) -> Result<(), AllocError> {
-        <Self as VecExt<_>>::reserve(self, 1, flags)?;
-        let s = self.spare_capacity_mut();
-        s[0].write(v);
-
-        // SAFETY: We just initialised the first spare entry, so it is safe to increase the length
-        // by 1. We also know that the new length is <= capacity because of the previous call to
-        // `reserve` above.
-        unsafe { self.set_len(self.len() + 1) };
-        Ok(())
-    }
-
-    fn extend_from_slice(&mut self, other: &[T], flags: Flags) -> Result<(), AllocError>
-    where
-        T: Clone,
-    {
-        <Self as VecExt<_>>::reserve(self, other.len(), flags)?;
-        for (slot, item) in core::iter::zip(self.spare_capacity_mut(), other) {
-            slot.write(item.clone());
-        }
-
-        // SAFETY: We just initialised the `other.len()` spare entries, so it is safe to increase
-        // the length by the same amount. We also know that the new length is <= capacity because
-        // of the previous call to `reserve` above.
-        unsafe { self.set_len(self.len() + other.len()) };
-        Ok(())
-    }
-
-    #[cfg(any(test, testlib))]
-    fn reserve(&mut self, additional: usize, _flags: Flags) -> Result<(), AllocError> {
-        Vec::reserve(self, additional);
-        Ok(())
-    }
-
-    #[cfg(not(any(test, testlib)))]
-    fn reserve(&mut self, additional: usize, flags: Flags) -> Result<(), AllocError> {
-        let len = self.len();
-        let cap = self.capacity();
-
-        if cap - len >= additional {
-            return Ok(());
-        }
-
-        if core::mem::size_of::<T>() == 0 {
-            // The capacity is already `usize::MAX` for SZTs, we can't go higher.
-            return Err(AllocError);
-        }
-
-        // We know cap is <= `isize::MAX` because `Layout::array` fails if the resulting byte size
-        // is greater than `isize::MAX`. So the multiplication by two won't overflow.
-        let new_cap = core::cmp::max(cap * 2, len.checked_add(additional).ok_or(AllocError)?);
-        let layout = core::alloc::Layout::array::<T>(new_cap).map_err(|_| AllocError)?;
-
-        let (old_ptr, len, cap) = destructure(self);
-
-        // We need to make sure that `ptr` is either NULL or comes from a previous call to
-        // `krealloc_aligned`. A `Vec<T>`'s `ptr` value is not guaranteed to be NULL and might be
-        // dangling after being created with `Vec::new`. Instead, we can rely on `Vec<T>`'s capacity
-        // to be zero if no memory has been allocated yet.
-        let ptr = if cap == 0 {
-            core::ptr::null_mut()
-        } else {
-            old_ptr
-        };
-
-        // SAFETY: `ptr` is valid because it's either NULL or comes from a previous call to
-        // `krealloc_aligned`. We also verified that the type is not a ZST.
-        let new_ptr = unsafe { super::allocator::krealloc_aligned(ptr.cast(), layout, flags) };
-        if new_ptr.is_null() {
-            // SAFETY: We are just rebuilding the existing `Vec` with no changes.
-            unsafe { rebuild(self, old_ptr, len, cap) };
-            Err(AllocError)
-        } else {
-            // SAFETY: `ptr` has been reallocated with the layout for `new_cap` elements. New cap
-            // is greater than `cap`, so it continues to be >= `len`.
-            unsafe { rebuild(self, new_ptr.cast::<T>(), len, new_cap) };
-            Ok(())
-        }
-    }
-}
-
-#[cfg(not(any(test, testlib)))]
-fn destructure<T>(v: &mut Vec<T>) -> (*mut T, usize, usize) {
-    let mut tmp = Vec::new();
-    core::mem::swap(&mut tmp, v);
-    let mut tmp = core::mem::ManuallyDrop::new(tmp);
-    let len = tmp.len();
-    let cap = tmp.capacity();
-    (tmp.as_mut_ptr(), len, cap)
-}
-
-/// Rebuilds a `Vec` from a pointer, length, and capacity.
-///
-/// # Safety
-///
-/// The same as [`Vec::from_raw_parts`].
-#[cfg(not(any(test, testlib)))]
-unsafe fn rebuild<T>(v: &mut Vec<T>, ptr: *mut T, len: usize, cap: usize) {
-    // SAFETY: The safety requirements from this function satisfy those of `from_raw_parts`.
-    let mut tmp = unsafe { Vec::from_raw_parts(ptr, len, cap) };
-    core::mem::swap(&mut tmp, v);
-}
diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
index 80223cdaa485..07daccf6ca8e 100644
--- a/rust/kernel/prelude.rs
+++ b/rust/kernel/prelude.rs
@@ -14,10 +14,7 @@
 #[doc(no_inline)]
 pub use core::pin::Pin;
 
-pub use crate::alloc::{flags::*, vec_ext::VecExt, Box, KBox, KVBox, KVVec, KVec, VBox, VVec};
-
-#[doc(no_inline)]
-pub use alloc::vec::Vec;
+pub use crate::alloc::{flags::*, Box, KBox, KVBox, KVVec, KVec, VBox, VVec};
 
 #[doc(no_inline)]
 pub use macros::{module, pin_data, pinned_drop, vtable, Zeroable};
-- 
2.46.0

[PATCH v6 18/26] rust: alloc: add `Vec` to prelude

[Danilo Krummrich]

Now that we removed `VecExt` and the corresponding includes in
prelude.rs, add the new kernel `Vec` type instead.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/prelude.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
index 07daccf6ca8e..8bdab9aa0d16 100644
--- a/rust/kernel/prelude.rs
+++ b/rust/kernel/prelude.rs
@@ -14,7 +14,7 @@
 #[doc(no_inline)]
 pub use core::pin::Pin;
 
-pub use crate::alloc::{flags::*, Box, KBox, KVBox, KVVec, KVec, VBox, VVec};
+pub use crate::alloc::{flags::*, Box, KBox, KVBox, KVVec, KVec, VBox, VVec, Vec};
 
 #[doc(no_inline)]
 pub use macros::{module, pin_data, pinned_drop, vtable, Zeroable};
-- 
2.46.0

[PATCH v6 19/26] rust: error: use `core::alloc::LayoutError`

[Danilo Krummrich]

Use `core::alloc::LayoutError` instead of `alloc::alloc::LayoutError` in
preparation to get rid of Rust's alloc crate.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/error.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/rust/kernel/error.rs b/rust/kernel/error.rs
index 145f5c397009..2d012cc3881a 100644
--- a/rust/kernel/error.rs
+++ b/rust/kernel/error.rs
@@ -6,7 +6,7 @@
 
 use crate::{alloc::AllocError, str::CStr};
 
-use alloc::alloc::LayoutError;
+use core::alloc::LayoutError;
 
 use core::fmt;
 use core::num::TryFromIntError;
-- 
2.46.0

[PATCH v6 20/26] rust: error: check for config `test` in `Error::name`

[Danilo Krummrich]

Additional to `testlib` also check for `test` in `Error::name`. This is
required by a subsequent patch that (indirectly) uses `Error` in test
cases.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/error.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/rust/kernel/error.rs b/rust/kernel/error.rs
index 2d012cc3881a..7adf889b9526 100644
--- a/rust/kernel/error.rs
+++ b/rust/kernel/error.rs
@@ -140,7 +140,7 @@ pub(crate) fn to_ptr<T>(self) -> *mut T {
     }
 
     /// Returns a string representing the error, if one exists.
-    #[cfg(not(testlib))]
+    #[cfg(not(any(test, testlib)))]
     pub fn name(&self) -> Option<&'static CStr> {
         // SAFETY: Just an FFI call, there are no extra safety requirements.
         let ptr = unsafe { bindings::errname(-self.0) };
@@ -157,7 +157,7 @@ pub fn name(&self) -> Option<&'static CStr> {
     /// When `testlib` is configured, this always returns `None` to avoid the dependency on a
     /// kernel function so that tests that use this (e.g., by calling [`Result::unwrap`]) can still
     /// run in userspace.
-    #[cfg(testlib)]
+    #[cfg(any(test, testlib))]
     pub fn name(&self) -> Option<&'static CStr> {
         None
     }
-- 
2.46.0

Re: [PATCH v6 20/26] rust: error: check for config `test` in `Error::name`

[Benno Lossin]

On 16.08.24 02:11, Danilo Krummrich wrote:
> Additional to `testlib` also check for `test` in `Error::name`. This is
> required by a subsequent patch that (indirectly) uses `Error` in test
> cases.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

Reviewed-by: Benno Lossin <benno.lossin@proton.me>

---
Cheers,
Benno

> ---
>  rust/kernel/error.rs | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/rust/kernel/error.rs b/rust/kernel/error.rs
> index 2d012cc3881a..7adf889b9526 100644
> --- a/rust/kernel/error.rs
> +++ b/rust/kernel/error.rs
> @@ -140,7 +140,7 @@ pub(crate) fn to_ptr<T>(self) -> *mut T {
>      }
> 
>      /// Returns a string representing the error, if one exists.
> -    #[cfg(not(testlib))]
> +    #[cfg(not(any(test, testlib)))]
>      pub fn name(&self) -> Option<&'static CStr> {
>          // SAFETY: Just an FFI call, there are no extra safety requirements.
>          let ptr = unsafe { bindings::errname(-self.0) };
> @@ -157,7 +157,7 @@ pub fn name(&self) -> Option<&'static CStr> {
>      /// When `testlib` is configured, this always returns `None` to avoid the dependency on a
>      /// kernel function so that tests that use this (e.g., by calling [`Result::unwrap`]) can still
>      /// run in userspace.
> -    #[cfg(testlib)]
> +    #[cfg(any(test, testlib))]
>      pub fn name(&self) -> Option<&'static CStr> {
>          None
>      }
> --
> 2.46.0
> 

[PATCH v6 21/26] rust: alloc: implement `contains` for `Flags`

[Danilo Krummrich]

Provide a simple helper function to check whether given flags do
contain one or multiple other flags.

This is used by a subsequent patch implementing the Cmalloc `Allocator`
to check for __GFP_ZERO.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc.rs | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index 1feabc817d00..5c66229a7542 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -34,7 +34,7 @@
 /// They can be combined with the operators `|`, `&`, and `!`.
 ///
 /// Values can be used from the [`flags`] module.
-#[derive(Clone, Copy)]
+#[derive(Clone, Copy, PartialEq)]
 pub struct Flags(u32);
 
 impl Flags {
@@ -42,6 +42,11 @@ impl Flags {
     pub(crate) fn as_raw(self) -> u32 {
         self.0
     }
+
+    /// Check whether `flags` is contained in `self`.
+    pub fn contains(self, flags: Flags) -> bool {
+        (self & flags) == flags
+    }
 }
 
 impl core::ops::BitOr for Flags {
-- 
2.46.0

Re: [PATCH v6 21/26] rust: alloc: implement `contains` for `Flags`

[Benno Lossin]

On 16.08.24 02:11, Danilo Krummrich wrote:
> Provide a simple helper function to check whether given flags do
> contain one or multiple other flags.
> 
> This is used by a subsequent patch implementing the Cmalloc `Allocator`
> to check for __GFP_ZERO.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

Reviewed-by: Benno Lossin <benno.lossin@proton.me>

---
Cheers,
Benno

> ---
>  rust/kernel/alloc.rs | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
> index 1feabc817d00..5c66229a7542 100644
> --- a/rust/kernel/alloc.rs
> +++ b/rust/kernel/alloc.rs
> @@ -34,7 +34,7 @@
>  /// They can be combined with the operators `|`, `&`, and `!`.
>  ///
>  /// Values can be used from the [`flags`] module.
> -#[derive(Clone, Copy)]
> +#[derive(Clone, Copy, PartialEq)]
>  pub struct Flags(u32);
> 
>  impl Flags {
> @@ -42,6 +42,11 @@ impl Flags {
>      pub(crate) fn as_raw(self) -> u32 {
>          self.0
>      }
> +
> +    /// Check whether `flags` is contained in `self`.
> +    pub fn contains(self, flags: Flags) -> bool {
> +        (self & flags) == flags
> +    }
>  }
> 
>  impl core::ops::BitOr for Flags {
> --
> 2.46.0
> 

[PATCH v6 22/26] rust: alloc: implement `Cmalloc` in module allocator_test

[Danilo Krummrich]

So far the kernel's `Box` and `Vec` types can't be used by userspace
test cases, since all users of those types (e.g. `CString`) use kernel
allocators for instantiation.

In order to allow userspace test cases to make use of such types as
well, implement the `Cmalloc` allocator within the allocator_test module
and type alias all kernel allocators to `Cmalloc`. The `Cmalloc`
allocator uses libc's realloc() function as allocator backend.

Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc/allocator_test.rs | 178 ++++++++++++++++++++++++++--
 1 file changed, 171 insertions(+), 7 deletions(-)

diff --git a/rust/kernel/alloc/allocator_test.rs b/rust/kernel/alloc/allocator_test.rs
index 1b2642c547ec..7fff308d02dc 100644
--- a/rust/kernel/alloc/allocator_test.rs
+++ b/rust/kernel/alloc/allocator_test.rs
@@ -2,20 +2,184 @@
 
 #![allow(missing_docs)]
 
-use super::{AllocError, Allocator, Flags};
+use super::{flags::*, AllocError, Allocator, Flags};
 use core::alloc::Layout;
+use core::cmp;
+use core::mem;
+use core::ptr;
 use core::ptr::NonNull;
 
-pub struct Kmalloc;
+pub struct Cmalloc;
+pub type Kmalloc = Cmalloc;
 pub type Vmalloc = Kmalloc;
 pub type KVmalloc = Kmalloc;
 
-unsafe impl Allocator for Kmalloc {
+extern "C" {
+    #[link_name = "aligned_alloc"]
+    fn libc_aligned_alloc(align: usize, size: usize) -> *mut core::ffi::c_void;
+
+    #[link_name = "free"]
+    fn libc_free(ptr: *mut core::ffi::c_void);
+}
+
+struct CmallocData {
+    // The actual size as requested through `Cmalloc::alloc` or `Cmalloc::realloc`.
+    size: usize,
+    // The offset from the pointer returned to the caller of `Cmalloc::alloc` or `Cmalloc::realloc`
+    // to the actual base address of the allocation.
+    offset: usize,
+}
+
+impl Cmalloc {
+    /// Adjust the size and alignment such that we can additionally store `CmallocData` right
+    /// before the actual data described by `layout`.
+    ///
+    /// Example:
+    ///
+    /// For `CmallocData` assume an alignment of 8 and a size of 16.
+    /// For `layout` assume and alignment of 16 and a size of 64.
+    ///
+    /// 0                16               32                                               96
+    /// |----------------|----------------|------------------------------------------------|
+    ///        empty         CmallocData                         data
+    ///
+    /// For this example the returned `Layout` has an alignment of 32 and a size of 96.
+    fn layout_adjust(layout: Layout) -> Result<Layout, AllocError> {
+        let layout = layout.pad_to_align();
+
+        // Ensure that `CmallocData` fits into half the alignment. Additionally, this guarantees
+        // that advancing a pointer aligned to `align` by `align / 2` we still satisfy or exceed
+        // the alignment requested through `layout`.
+        let align = cmp::max(
+            layout.align(),
+            mem::size_of::<CmallocData>().next_power_of_two(),
+        ) * 2;
+
+        // Add the additional space required for `CmallocData`.
+        let size = layout.size() + mem::size_of::<CmallocData>();
+
+        Ok(Layout::from_size_align(size, align)
+            .map_err(|_| AllocError)?
+            .pad_to_align())
+    }
+
+    fn alloc_store_data(layout: Layout) -> Result<NonNull<u8>, AllocError> {
+        let requested_size = layout.size();
+
+        let layout = Self::layout_adjust(layout)?;
+        let min_align = layout.align() / 2;
+
+        // SAFETY: Returns either NULL or a pointer to a memory allocation that satisfies or
+        // exceeds the given size and alignment requirements.
+        let raw_ptr = unsafe { libc_aligned_alloc(layout.align(), layout.size()) } as *mut u8;
+
+        let priv_ptr = NonNull::new(raw_ptr).ok_or(AllocError)?;
+
+        // SAFETY: Advance the pointer by `min_align`. The adjustments from `Self::layout_adjust`
+        // ensure that after this operation the original size and alignment requirements are still
+        // satisfied or exceeded.
+        let ptr = unsafe { priv_ptr.as_ptr().add(min_align) };
+
+        // SAFETY: `min_align` is greater than or equal to the size of `CmallocData`, hence we
+        // don't exceed the allocation boundaries.
+        let data_ptr: *mut CmallocData = unsafe { ptr.sub(mem::size_of::<CmallocData>()) }.cast();
+
+        let data = CmallocData {
+            size: requested_size,
+            offset: min_align,
+        };
+
+        // SAFETY: `data_ptr` is properly aligned and within the allocation boundaries reserved for
+        // `CmallocData`.
+        unsafe { data_ptr.write(data) };
+
+        NonNull::new(ptr).ok_or(AllocError)
+    }
+
+    /// # Safety
+    ///
+    /// `ptr` must have been previously allocated with `Self::alloc_store_data`.
+    unsafe fn data<'a>(ptr: NonNull<u8>) -> &'a CmallocData {
+        // SAFETY: `Self::alloc_store_data` stores the `CmallocData` right before the address
+        // returned to callers of `Self::alloc_store_data`.
+        let data_ptr: *mut CmallocData =
+            unsafe { ptr.as_ptr().sub(mem::size_of::<CmallocData>()) }.cast();
+
+        // SAFETY: The `CmallocData` has been previously stored at this offset with
+        // `Self::alloc_store_data`.
+        unsafe { &*data_ptr }
+    }
+
+    /// # Safety
+    ///
+    /// This function must not be called more than once for the same allocation.
+    ///
+    /// `ptr` must have been previously allocated with `Self::alloc_store_data`.
+    unsafe fn free_read_data(ptr: NonNull<u8>) {
+        // SAFETY: `ptr` has been created by `Self::alloc_store_data`.
+        let data = unsafe { Self::data(ptr) };
+
+        // SAFETY: `ptr` has been created by `Self::alloc_store_data`.
+        let priv_ptr = unsafe { ptr.as_ptr().sub(data.offset) };
+
+        // SAFETY: `priv_ptr` has previously been allocatored with this `Allocator`.
+        unsafe { libc_free(priv_ptr.cast()) };
+    }
+}
+
+unsafe impl Allocator for Cmalloc {
+    fn alloc(layout: Layout, flags: Flags) -> Result<NonNull<[u8]>, AllocError> {
+        if layout.size() == 0 {
+            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
+        }
+
+        let ptr = Self::alloc_store_data(layout)?;
+
+        if flags.contains(__GFP_ZERO) {
+            // SAFETY: `Self::alloc_store_data` guarantees that `ptr` points to memory of at least
+            // `layout.size()` bytes.
+            unsafe { ptr.as_ptr().write_bytes(0, layout.size()) };
+        }
+
+        Ok(NonNull::slice_from_raw_parts(ptr, layout.size()))
+    }
+
     unsafe fn realloc(
-        _ptr: Option<NonNull<u8>>,
-        _layout: Layout,
-        _flags: Flags,
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
     ) -> Result<NonNull<[u8]>, AllocError> {
-        panic!();
+        let src: NonNull<u8> = if let Some(src) = ptr {
+            src.cast()
+        } else {
+            return Self::alloc(layout, flags);
+        };
+
+        if layout.size() == 0 {
+            // SAFETY: `src` has been created by `Self::alloc_store_data`.
+            unsafe { Self::free_read_data(src) };
+
+            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
+        }
+
+        let dst = Self::alloc(layout, flags)?;
+
+        // SAFETY: `src` has been created by `Self::alloc_store_data`.
+        let data = unsafe { Self::data(src) };
+
+        // SAFETY: `src` has previously been allocated with this `Allocator`; `dst` has just been
+        // newly allocated. Copy up to the smaller of both sizes.
+        unsafe {
+            ptr::copy_nonoverlapping(
+                src.as_ptr(),
+                dst.as_ptr().cast(),
+                cmp::min(layout.size(), data.size),
+            )
+        };
+
+        // SAFETY: `src` has been created by `Self::alloc_store_data`.
+        unsafe { Self::free_read_data(src) };
+
+        Ok(dst)
     }
 }
-- 
2.46.0

Re: [PATCH v6 22/26] rust: alloc: implement `Cmalloc` in module allocator_test

[Benno Lossin]

On 16.08.24 02:11, Danilo Krummrich wrote:
> So far the kernel's `Box` and `Vec` types can't be used by userspace
> test cases, since all users of those types (e.g. `CString`) use kernel
> allocators for instantiation.
> 
> In order to allow userspace test cases to make use of such types as
> well, implement the `Cmalloc` allocator within the allocator_test module
> and type alias all kernel allocators to `Cmalloc`. The `Cmalloc`
> allocator uses libc's realloc() function as allocator backend.
> 
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
>  rust/kernel/alloc/allocator_test.rs | 178 ++++++++++++++++++++++++++--
>  1 file changed, 171 insertions(+), 7 deletions(-)
> 
> diff --git a/rust/kernel/alloc/allocator_test.rs b/rust/kernel/alloc/allocator_test.rs
> index 1b2642c547ec..7fff308d02dc 100644
> --- a/rust/kernel/alloc/allocator_test.rs
> +++ b/rust/kernel/alloc/allocator_test.rs
> @@ -2,20 +2,184 @@
> 

Could add a short paragraph as the module description why this module
exists? Would probably be enough to paste the commit message.

>  #![allow(missing_docs)]
> 
> -use super::{AllocError, Allocator, Flags};
> +use super::{flags::*, AllocError, Allocator, Flags};
>  use core::alloc::Layout;
> +use core::cmp;
> +use core::mem;
> +use core::ptr;
>  use core::ptr::NonNull;
> 
> -pub struct Kmalloc;
> +pub struct Cmalloc;
> +pub type Kmalloc = Cmalloc;
>  pub type Vmalloc = Kmalloc;
>  pub type KVmalloc = Kmalloc;
> 
> -unsafe impl Allocator for Kmalloc {
> +extern "C" {
> +    #[link_name = "aligned_alloc"]
> +    fn libc_aligned_alloc(align: usize, size: usize) -> *mut core::ffi::c_void;
> +
> +    #[link_name = "free"]
> +    fn libc_free(ptr: *mut core::ffi::c_void);
> +}
> +
> +struct CmallocData {
> +    // The actual size as requested through `Cmalloc::alloc` or `Cmalloc::realloc`.
> +    size: usize,
> +    // The offset from the pointer returned to the caller of `Cmalloc::alloc` or `Cmalloc::realloc`
> +    // to the actual base address of the allocation.
> +    offset: usize,
> +}
> +
> +impl Cmalloc {
> +    /// Adjust the size and alignment such that we can additionally store `CmallocData` right
> +    /// before the actual data described by `layout`.
> +    ///
> +    /// Example:
> +    ///
> +    /// For `CmallocData` assume an alignment of 8 and a size of 16.
> +    /// For `layout` assume and alignment of 16 and a size of 64.

This looks like you want it rendered as bulletpoints (but it won't).

> +    ///
> +    /// 0                16               32                                               96
> +    /// |----------------|----------------|------------------------------------------------|
> +    ///        empty         CmallocData                         data

Can you put this inside of '```'? Then it will render nicely in markdown
(don't forget to specify the type 'text')

> +    ///
> +    /// For this example the returned `Layout` has an alignment of 32 and a size of 96.
> +    fn layout_adjust(layout: Layout) -> Result<Layout, AllocError> {
> +        let layout = layout.pad_to_align();
> +
> +        // Ensure that `CmallocData` fits into half the alignment. Additionally, this guarantees
> +        // that advancing a pointer aligned to `align` by `align / 2` we still satisfy or exceed
> +        // the alignment requested through `layout`.
> +        let align = cmp::max(
> +            layout.align(),
> +            mem::size_of::<CmallocData>().next_power_of_two(),
> +        ) * 2;
> +
> +        // Add the additional space required for `CmallocData`.
> +        let size = layout.size() + mem::size_of::<CmallocData>();
> +
> +        Ok(Layout::from_size_align(size, align)
> +            .map_err(|_| AllocError)?
> +            .pad_to_align())
> +    }
> +
> +    fn alloc_store_data(layout: Layout) -> Result<NonNull<u8>, AllocError> {
> +        let requested_size = layout.size();
> +
> +        let layout = Self::layout_adjust(layout)?;
> +        let min_align = layout.align() / 2;
> +
> +        // SAFETY: Returns either NULL or a pointer to a memory allocation that satisfies or
> +        // exceeds the given size and alignment requirements.
> +        let raw_ptr = unsafe { libc_aligned_alloc(layout.align(), layout.size()) } as *mut u8;
> +
> +        let priv_ptr = NonNull::new(raw_ptr).ok_or(AllocError)?;
> +
> +        // SAFETY: Advance the pointer by `min_align`. The adjustments from `Self::layout_adjust`
> +        // ensure that after this operation the original size and alignment requirements are still
> +        // satisfied or exceeded.

This SAFETY comment should address why it's OK to call `add`. You
justify something different, namely why the allocation still satisfies
the requirements of `layout`. That is something that this function
should probably guarantee.

> +        let ptr = unsafe { priv_ptr.as_ptr().add(min_align) };
> +
> +        // SAFETY: `min_align` is greater than or equal to the size of `CmallocData`, hence we
> +        // don't exceed the allocation boundaries.
> +        let data_ptr: *mut CmallocData = unsafe { ptr.sub(mem::size_of::<CmallocData>()) }.cast();
> +
> +        let data = CmallocData {
> +            size: requested_size,
> +            offset: min_align,
> +        };
> +
> +        // SAFETY: `data_ptr` is properly aligned and within the allocation boundaries reserved for
> +        // `CmallocData`.
> +        unsafe { data_ptr.write(data) };
> +
> +        NonNull::new(ptr).ok_or(AllocError)
> +    }
> +
> +    /// # Safety
> +    ///
> +    /// `ptr` must have been previously allocated with `Self::alloc_store_data`.

You additionally need that you have shared access to the pointee.

> +    unsafe fn data<'a>(ptr: NonNull<u8>) -> &'a CmallocData {
> +        // SAFETY: `Self::alloc_store_data` stores the `CmallocData` right before the address
> +        // returned to callers of `Self::alloc_store_data`.
> +        let data_ptr: *mut CmallocData =
> +            unsafe { ptr.as_ptr().sub(mem::size_of::<CmallocData>()) }.cast();
> +
> +        // SAFETY: The `CmallocData` has been previously stored at this offset with
> +        // `Self::alloc_store_data`.
> +        unsafe { &*data_ptr }
> +    }
> +
> +    /// # Safety
> +    ///
> +    /// This function must not be called more than once for the same allocation.
> +    ///
> +    /// `ptr` must have been previously allocated with `Self::alloc_store_data`.

You additionally need that you have exclusive access to the pointee.

> +    unsafe fn free_read_data(ptr: NonNull<u8>) {
> +        // SAFETY: `ptr` has been created by `Self::alloc_store_data`.
> +        let data = unsafe { Self::data(ptr) };
> +
> +        // SAFETY: `ptr` has been created by `Self::alloc_store_data`.
> +        let priv_ptr = unsafe { ptr.as_ptr().sub(data.offset) };
> +
> +        // SAFETY: `priv_ptr` has previously been allocatored with this `Allocator`.
> +        unsafe { libc_free(priv_ptr.cast()) };
> +    }
> +}
> +
> +unsafe impl Allocator for Cmalloc {
> +    fn alloc(layout: Layout, flags: Flags) -> Result<NonNull<[u8]>, AllocError> {
> +        if layout.size() == 0 {
> +            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
> +        }
> +
> +        let ptr = Self::alloc_store_data(layout)?;
> +
> +        if flags.contains(__GFP_ZERO) {
> +            // SAFETY: `Self::alloc_store_data` guarantees that `ptr` points to memory of at least
> +            // `layout.size()` bytes.
> +            unsafe { ptr.as_ptr().write_bytes(0, layout.size()) };
> +        }

This makes me wonder, what other flags should we handle for this
allocator?

> +
> +        Ok(NonNull::slice_from_raw_parts(ptr, layout.size()))
> +    }
> +
>      unsafe fn realloc(
> -        _ptr: Option<NonNull<u8>>,
> -        _layout: Layout,
> -        _flags: Flags,
> +        ptr: Option<NonNull<u8>>,
> +        layout: Layout,
> +        flags: Flags,
>      ) -> Result<NonNull<[u8]>, AllocError> {
> -        panic!();
> +        let src: NonNull<u8> = if let Some(src) = ptr {
> +            src.cast()

Why the cast?

> +        } else {
> +            return Self::alloc(layout, flags);
> +        };

You should be able to write this instead:

    let Some(src) = ptr else {
        return Self::alloc(layout, flags);
    };

> +
> +        if layout.size() == 0 {
> +            // SAFETY: `src` has been created by `Self::alloc_store_data`.

This is not true, consider:

    let ptr = alloc(size = 0);
    free(ptr)

Alloc will return a dangling pointer due to the first if statement and
then this function will pass it to `free_read_data`, even though it
wasn't created by `alloc_store_data`.
This isn't forbidden by the `Allocator` trait function's safety
requirements.

> +            unsafe { Self::free_read_data(src) };
> +
> +            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
> +        }
> +
> +        let dst = Self::alloc(layout, flags)?;
> +
> +        // SAFETY: `src` has been created by `Self::alloc_store_data`.
> +        let data = unsafe { Self::data(src) };

Same issue here, if the allocation passed in is zero size. I think you
have no other choice than to allocate even for zero size requests...
Otherwise how would you know that they are zero-sized.

---
Cheers,
Benno

> +
> +        // SAFETY: `src` has previously been allocated with this `Allocator`; `dst` has just been
> +        // newly allocated. Copy up to the smaller of both sizes.
> +        unsafe {
> +            ptr::copy_nonoverlapping(
> +                src.as_ptr(),
> +                dst.as_ptr().cast(),
> +                cmp::min(layout.size(), data.size),
> +            )
> +        };
> +
> +        // SAFETY: `src` has been created by `Self::alloc_store_data`.
> +        unsafe { Self::free_read_data(src) };
> +
> +        Ok(dst)
>      }
>  }
> --
> 2.46.0
> 

Re: [PATCH v6 22/26] rust: alloc: implement `Cmalloc` in module allocator_test

[Danilo Krummrich]

On Thu, Aug 29, 2024 at 07:14:18PM +0000, Benno Lossin wrote:
> On 16.08.24 02:11, Danilo Krummrich wrote:
> > So far the kernel's `Box` and `Vec` types can't be used by userspace
> > test cases, since all users of those types (e.g. `CString`) use kernel
> > allocators for instantiation.
> > 
> > In order to allow userspace test cases to make use of such types as
> > well, implement the `Cmalloc` allocator within the allocator_test module
> > and type alias all kernel allocators to `Cmalloc`. The `Cmalloc`
> > allocator uses libc's realloc() function as allocator backend.
> > 
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > ---
> >  rust/kernel/alloc/allocator_test.rs | 178 ++++++++++++++++++++++++++--
> >  1 file changed, 171 insertions(+), 7 deletions(-)
> > 
> > diff --git a/rust/kernel/alloc/allocator_test.rs b/rust/kernel/alloc/allocator_test.rs
> > index 1b2642c547ec..7fff308d02dc 100644
> > --- a/rust/kernel/alloc/allocator_test.rs
> > +++ b/rust/kernel/alloc/allocator_test.rs
> > @@ -2,20 +2,184 @@
> > 
> 
> Could add a short paragraph as the module description why this module
> exists? Would probably be enough to paste the commit message.

Yes, sounds good.

> 
> >  #![allow(missing_docs)]
> > 
> > -use super::{AllocError, Allocator, Flags};
> > +use super::{flags::*, AllocError, Allocator, Flags};
> >  use core::alloc::Layout;
> > +use core::cmp;
> > +use core::mem;
> > +use core::ptr;
> >  use core::ptr::NonNull;
> > 
> > -pub struct Kmalloc;
> > +pub struct Cmalloc;
> > +pub type Kmalloc = Cmalloc;
> >  pub type Vmalloc = Kmalloc;
> >  pub type KVmalloc = Kmalloc;
> > 
> > -unsafe impl Allocator for Kmalloc {
> > +extern "C" {
> > +    #[link_name = "aligned_alloc"]
> > +    fn libc_aligned_alloc(align: usize, size: usize) -> *mut core::ffi::c_void;
> > +
> > +    #[link_name = "free"]
> > +    fn libc_free(ptr: *mut core::ffi::c_void);
> > +}
> > +
> > +struct CmallocData {
> > +    // The actual size as requested through `Cmalloc::alloc` or `Cmalloc::realloc`.
> > +    size: usize,
> > +    // The offset from the pointer returned to the caller of `Cmalloc::alloc` or `Cmalloc::realloc`
> > +    // to the actual base address of the allocation.
> > +    offset: usize,
> > +}
> > +
> > +impl Cmalloc {
> > +    /// Adjust the size and alignment such that we can additionally store `CmallocData` right
> > +    /// before the actual data described by `layout`.
> > +    ///
> > +    /// Example:
> > +    ///
> > +    /// For `CmallocData` assume an alignment of 8 and a size of 16.
> > +    /// For `layout` assume and alignment of 16 and a size of 64.
> 
> This looks like you want it rendered as bulletpoints (but it won't).

Actually, that wasn't my intention, but I'm fine changing that.

> 
> > +    ///
> > +    /// 0                16               32                                               96
> > +    /// |----------------|----------------|------------------------------------------------|
> > +    ///        empty         CmallocData                         data
> 
> Can you put this inside of '```'? Then it will render nicely in markdown
> (don't forget to specify the type 'text')

Sure.

> 
> > +    ///
> > +    /// For this example the returned `Layout` has an alignment of 32 and a size of 96.
> > +    fn layout_adjust(layout: Layout) -> Result<Layout, AllocError> {
> > +        let layout = layout.pad_to_align();
> > +
> > +        // Ensure that `CmallocData` fits into half the alignment. Additionally, this guarantees
> > +        // that advancing a pointer aligned to `align` by `align / 2` we still satisfy or exceed
> > +        // the alignment requested through `layout`.
> > +        let align = cmp::max(
> > +            layout.align(),
> > +            mem::size_of::<CmallocData>().next_power_of_two(),
> > +        ) * 2;
> > +
> > +        // Add the additional space required for `CmallocData`.
> > +        let size = layout.size() + mem::size_of::<CmallocData>();
> > +
> > +        Ok(Layout::from_size_align(size, align)
> > +            .map_err(|_| AllocError)?
> > +            .pad_to_align())
> > +    }
> > +
> > +    fn alloc_store_data(layout: Layout) -> Result<NonNull<u8>, AllocError> {
> > +        let requested_size = layout.size();
> > +
> > +        let layout = Self::layout_adjust(layout)?;
> > +        let min_align = layout.align() / 2;
> > +
> > +        // SAFETY: Returns either NULL or a pointer to a memory allocation that satisfies or
> > +        // exceeds the given size and alignment requirements.
> > +        let raw_ptr = unsafe { libc_aligned_alloc(layout.align(), layout.size()) } as *mut u8;
> > +
> > +        let priv_ptr = NonNull::new(raw_ptr).ok_or(AllocError)?;
> > +
> > +        // SAFETY: Advance the pointer by `min_align`. The adjustments from `Self::layout_adjust`
> > +        // ensure that after this operation the original size and alignment requirements are still
> > +        // satisfied or exceeded.
> 
> This SAFETY comment should address why it's OK to call `add`. You
> justify something different, namely why the allocation still satisfies
> the requirements of `layout`. That is something that this function
> should probably guarantee.

So, I guess you're arguing that instead I should say that, we're still within
the bounds of the same allocated object and don't exceed `isize`?

> 
> > +        let ptr = unsafe { priv_ptr.as_ptr().add(min_align) };
> > +
> > +        // SAFETY: `min_align` is greater than or equal to the size of `CmallocData`, hence we
> > +        // don't exceed the allocation boundaries.
> > +        let data_ptr: *mut CmallocData = unsafe { ptr.sub(mem::size_of::<CmallocData>()) }.cast();
> > +
> > +        let data = CmallocData {
> > +            size: requested_size,
> > +            offset: min_align,
> > +        };
> > +
> > +        // SAFETY: `data_ptr` is properly aligned and within the allocation boundaries reserved for
> > +        // `CmallocData`.
> > +        unsafe { data_ptr.write(data) };
> > +
> > +        NonNull::new(ptr).ok_or(AllocError)
> > +    }
> > +
> > +    /// # Safety
> > +    ///
> > +    /// `ptr` must have been previously allocated with `Self::alloc_store_data`.
> 
> You additionally need that you have shared access to the pointee.
> 
> > +    unsafe fn data<'a>(ptr: NonNull<u8>) -> &'a CmallocData {
> > +        // SAFETY: `Self::alloc_store_data` stores the `CmallocData` right before the address
> > +        // returned to callers of `Self::alloc_store_data`.
> > +        let data_ptr: *mut CmallocData =
> > +            unsafe { ptr.as_ptr().sub(mem::size_of::<CmallocData>()) }.cast();
> > +
> > +        // SAFETY: The `CmallocData` has been previously stored at this offset with
> > +        // `Self::alloc_store_data`.
> > +        unsafe { &*data_ptr }
> > +    }
> > +
> > +    /// # Safety
> > +    ///
> > +    /// This function must not be called more than once for the same allocation.
> > +    ///
> > +    /// `ptr` must have been previously allocated with `Self::alloc_store_data`.
> 
> You additionally need that you have exclusive access to the pointee.
> 
> > +    unsafe fn free_read_data(ptr: NonNull<u8>) {
> > +        // SAFETY: `ptr` has been created by `Self::alloc_store_data`.
> > +        let data = unsafe { Self::data(ptr) };
> > +
> > +        // SAFETY: `ptr` has been created by `Self::alloc_store_data`.
> > +        let priv_ptr = unsafe { ptr.as_ptr().sub(data.offset) };
> > +
> > +        // SAFETY: `priv_ptr` has previously been allocatored with this `Allocator`.
> > +        unsafe { libc_free(priv_ptr.cast()) };
> > +    }
> > +}
> > +
> > +unsafe impl Allocator for Cmalloc {
> > +    fn alloc(layout: Layout, flags: Flags) -> Result<NonNull<[u8]>, AllocError> {
> > +        if layout.size() == 0 {
> > +            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
> > +        }
> > +
> > +        let ptr = Self::alloc_store_data(layout)?;
> > +
> > +        if flags.contains(__GFP_ZERO) {
> > +            // SAFETY: `Self::alloc_store_data` guarantees that `ptr` points to memory of at least
> > +            // `layout.size()` bytes.
> > +            unsafe { ptr.as_ptr().write_bytes(0, layout.size()) };
> > +        }
> 
> This makes me wonder, what other flags should we handle for this
> allocator?

I don't think there are any other flags that we can handle. The only other one
that'd make sense is __GFP_NOFAIL, but we can't guarantee that.

If any specific gfp flags are needed, I think it's simply not a candidate for a
userspace test.

If we really want to do something here, we could whitelist the flags we ignore,
since they do not matter (such as __GFP_NOWARN) and panic() for everything else.

But I don't think that's really needed.

> 
> > +
> > +        Ok(NonNull::slice_from_raw_parts(ptr, layout.size()))
> > +    }
> > +
> >      unsafe fn realloc(
> > -        _ptr: Option<NonNull<u8>>,
> > -        _layout: Layout,
> > -        _flags: Flags,
> > +        ptr: Option<NonNull<u8>>,
> > +        layout: Layout,
> > +        flags: Flags,
> >      ) -> Result<NonNull<[u8]>, AllocError> {
> > -        panic!();
> > +        let src: NonNull<u8> = if let Some(src) = ptr {
> > +            src.cast()
> 
> Why the cast?

Probably a copy-paste mistake.

> 
> > +        } else {
> > +            return Self::alloc(layout, flags);
> > +        };
> 
> You should be able to write this instead:
> 
>     let Some(src) = ptr else {
>         return Self::alloc(layout, flags);
>     };

Yes, indeed.

> 
> > +
> > +        if layout.size() == 0 {
> > +            // SAFETY: `src` has been created by `Self::alloc_store_data`.
> 
> This is not true, consider:
> 
>     let ptr = alloc(size = 0);
>     free(ptr)
> 
> Alloc will return a dangling pointer due to the first if statement and
> then this function will pass it to `free_read_data`, even though it
> wasn't created by `alloc_store_data`.
> This isn't forbidden by the `Allocator` trait function's safety
> requirements.
> 
> > +            unsafe { Self::free_read_data(src) };
> > +
> > +            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
> > +        }
> > +
> > +        let dst = Self::alloc(layout, flags)?;
> > +
> > +        // SAFETY: `src` has been created by `Self::alloc_store_data`.
> > +        let data = unsafe { Self::data(src) };
> 
> Same issue here, if the allocation passed in is zero size. I think you
> have no other choice than to allocate even for zero size requests...
> Otherwise how would you know that they are zero-sized.

Good catch - gonna fix it.

> 
> ---
> Cheers,
> Benno
> 
> > +
> > +        // SAFETY: `src` has previously been allocated with this `Allocator`; `dst` has just been
> > +        // newly allocated. Copy up to the smaller of both sizes.
> > +        unsafe {
> > +            ptr::copy_nonoverlapping(
> > +                src.as_ptr(),
> > +                dst.as_ptr().cast(),
> > +                cmp::min(layout.size(), data.size),
> > +            )
> > +        };
> > +
> > +        // SAFETY: `src` has been created by `Self::alloc_store_data`.
> > +        unsafe { Self::free_read_data(src) };
> > +
> > +        Ok(dst)
> >      }
> >  }
> > --
> > 2.46.0
> > 
> 

Re: [PATCH v6 22/26] rust: alloc: implement `Cmalloc` in module allocator_test

[Benno Lossin]

On 30.08.24 00:25, Danilo Krummrich wrote:
> On Thu, Aug 29, 2024 at 07:14:18PM +0000, Benno Lossin wrote:
>> On 16.08.24 02:11, Danilo Krummrich wrote:
>>> +impl Cmalloc {
>>> +    /// Adjust the size and alignment such that we can additionally store `CmallocData` right
>>> +    /// before the actual data described by `layout`.
>>> +    ///
>>> +    /// Example:
>>> +    ///
>>> +    /// For `CmallocData` assume an alignment of 8 and a size of 16.
>>> +    /// For `layout` assume and alignment of 16 and a size of 64.
>>
>> This looks like you want it rendered as bulletpoints (but it won't).
> 
> Actually, that wasn't my intention, but I'm fine changing that.

I see, in that case not putting a newline there is also fine with me.
But I think bulletpoints are probably easier to read.

>>> +    fn alloc_store_data(layout: Layout) -> Result<NonNull<u8>, AllocError> {
>>> +        let requested_size = layout.size();
>>> +
>>> +        let layout = Self::layout_adjust(layout)?;
>>> +        let min_align = layout.align() / 2;
>>> +
>>> +        // SAFETY: Returns either NULL or a pointer to a memory allocation that satisfies or
>>> +        // exceeds the given size and alignment requirements.
>>> +        let raw_ptr = unsafe { libc_aligned_alloc(layout.align(), layout.size()) } as *mut u8;
>>> +
>>> +        let priv_ptr = NonNull::new(raw_ptr).ok_or(AllocError)?;
>>> +
>>> +        // SAFETY: Advance the pointer by `min_align`. The adjustments from `Self::layout_adjust`
>>> +        // ensure that after this operation the original size and alignment requirements are still
>>> +        // satisfied or exceeded.
>>
>> This SAFETY comment should address why it's OK to call `add`. You
>> justify something different, namely why the allocation still satisfies
>> the requirements of `layout`. That is something that this function
>> should probably guarantee.
> 
> So, I guess you're arguing that instead I should say that, we're still within
> the bounds of the same allocated object and don't exceed `isize`?

Yes.

>>> +    unsafe fn free_read_data(ptr: NonNull<u8>) {
>>> +        // SAFETY: `ptr` has been created by `Self::alloc_store_data`.
>>> +        let data = unsafe { Self::data(ptr) };
>>> +
>>> +        // SAFETY: `ptr` has been created by `Self::alloc_store_data`.
>>> +        let priv_ptr = unsafe { ptr.as_ptr().sub(data.offset) };
>>> +
>>> +        // SAFETY: `priv_ptr` has previously been allocatored with this `Allocator`.
>>> +        unsafe { libc_free(priv_ptr.cast()) };
>>> +    }
>>> +}
>>> +
>>> +unsafe impl Allocator for Cmalloc {
>>> +    fn alloc(layout: Layout, flags: Flags) -> Result<NonNull<[u8]>, AllocError> {
>>> +        if layout.size() == 0 {
>>> +            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
>>> +        }
>>> +
>>> +        let ptr = Self::alloc_store_data(layout)?;
>>> +
>>> +        if flags.contains(__GFP_ZERO) {
>>> +            // SAFETY: `Self::alloc_store_data` guarantees that `ptr` points to memory of at least
>>> +            // `layout.size()` bytes.
>>> +            unsafe { ptr.as_ptr().write_bytes(0, layout.size()) };
>>> +        }
>>
>> This makes me wonder, what other flags should we handle for this
>> allocator?
> 
> I don't think there are any other flags that we can handle. The only other one
> that'd make sense is __GFP_NOFAIL, but we can't guarantee that.
> 
> If any specific gfp flags are needed, I think it's simply not a candidate for a
> userspace test.
> 
> If we really want to do something here, we could whitelist the flags we ignore,
> since they do not matter (such as __GFP_NOWARN) and panic() for everything else.
> 
> But I don't think that's really needed.

Makes sense, just wanted to check that this has been accounted for.

---
Cheers,
Benno

Re: [PATCH v6 22/26] rust: alloc: implement `Cmalloc` in module allocator_test

[Danilo Krummrich]

On Fri, Aug 30, 2024 at 12:25:27AM +0200, Danilo Krummrich wrote:
> On Thu, Aug 29, 2024 at 07:14:18PM +0000, Benno Lossin wrote:
> > On 16.08.24 02:11, Danilo Krummrich wrote:
> > 
> > > +
> > > +        if layout.size() == 0 {
> > > +            // SAFETY: `src` has been created by `Self::alloc_store_data`.
> > 
> > This is not true, consider:
> > 
> >     let ptr = alloc(size = 0);
> >     free(ptr)
> > 
> > Alloc will return a dangling pointer due to the first if statement and
> > then this function will pass it to `free_read_data`, even though it
> > wasn't created by `alloc_store_data`.
> > This isn't forbidden by the `Allocator` trait function's safety
> > requirements.
> > 
> > > +            unsafe { Self::free_read_data(src) };
> > > +
> > > +            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
> > > +        }
> > > +
> > > +        let dst = Self::alloc(layout, flags)?;
> > > +
> > > +        // SAFETY: `src` has been created by `Self::alloc_store_data`.
> > > +        let data = unsafe { Self::data(src) };
> > 
> > Same issue here, if the allocation passed in is zero size. I think you
> > have no other choice than to allocate even for zero size requests...
> > Otherwise how would you know that they are zero-sized.
> 
> Good catch - gonna fix it.

Almost got me. :) I think the code is fine, callers are not allowed to pass
pointers to `realloc` and `free`, which haven't been allocated with the same
corresponding allocator or are dangling.

> 
> > 
> > ---
> > Cheers,
> > Benno
> > 
> > > +
> > > +        // SAFETY: `src` has previously been allocated with this `Allocator`; `dst` has just been
> > > +        // newly allocated. Copy up to the smaller of both sizes.
> > > +        unsafe {
> > > +            ptr::copy_nonoverlapping(
> > > +                src.as_ptr(),
> > > +                dst.as_ptr().cast(),
> > > +                cmp::min(layout.size(), data.size),
> > > +            )
> > > +        };
> > > +
> > > +        // SAFETY: `src` has been created by `Self::alloc_store_data`.
> > > +        unsafe { Self::free_read_data(src) };
> > > +
> > > +        Ok(dst)
> > >      }
> > >  }
> > > --
> > > 2.46.0
> > > 
> > 

Re: [PATCH v6 22/26] rust: alloc: implement `Cmalloc` in module allocator_test

[Benno Lossin]

On 11.09.24 14:31, Danilo Krummrich wrote:
> On Fri, Aug 30, 2024 at 12:25:27AM +0200, Danilo Krummrich wrote:
>> On Thu, Aug 29, 2024 at 07:14:18PM +0000, Benno Lossin wrote:
>>> On 16.08.24 02:11, Danilo Krummrich wrote:
>>>> +
>>>> +        if layout.size() == 0 {
>>>> +            // SAFETY: `src` has been created by `Self::alloc_store_data`.
>>>
>>> This is not true, consider:
>>>
>>>     let ptr = alloc(size = 0);
>>>     free(ptr)
>>>
>>> Alloc will return a dangling pointer due to the first if statement and
>>> then this function will pass it to `free_read_data`, even though it
>>> wasn't created by `alloc_store_data`.
>>> This isn't forbidden by the `Allocator` trait function's safety
>>> requirements.
>>>
>>>> +            unsafe { Self::free_read_data(src) };
>>>> +
>>>> +            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
>>>> +        }
>>>> +
>>>> +        let dst = Self::alloc(layout, flags)?;
>>>> +
>>>> +        // SAFETY: `src` has been created by `Self::alloc_store_data`.
>>>> +        let data = unsafe { Self::data(src) };
>>>
>>> Same issue here, if the allocation passed in is zero size. I think you
>>> have no other choice than to allocate even for zero size requests...
>>> Otherwise how would you know that they are zero-sized.
>>
>> Good catch - gonna fix it.
> 
> Almost got me. :) I think the code is fine, callers are not allowed to pass
> pointers to `realloc` and `free`, which haven't been allocated with the same
> corresponding allocator or are dangling.

But what about the example above (ie the `alloc(size = 0)` and then
`free`)? I guess this all depends on how one interprets the term
"existing, valid memory allocation". To me that describes anything an
`Allocator` returns via `alloc` and `realloc`, including zero-sized
allocations.
But if you argue that those are not valid allocations from that
allocator, then that is not properly documented in the safety
requirements of `Allocator`. 

---
Cheers,
Benno

Re: [PATCH v6 22/26] rust: alloc: implement `Cmalloc` in module allocator_test

[Danilo Krummrich]

On Wed, Sep 11, 2024 at 01:32:31PM +0000, Benno Lossin wrote:
> On 11.09.24 14:31, Danilo Krummrich wrote:
> > On Fri, Aug 30, 2024 at 12:25:27AM +0200, Danilo Krummrich wrote:
> >> On Thu, Aug 29, 2024 at 07:14:18PM +0000, Benno Lossin wrote:
> >>> On 16.08.24 02:11, Danilo Krummrich wrote:
> >>>> +
> >>>> +        if layout.size() == 0 {
> >>>> +            // SAFETY: `src` has been created by `Self::alloc_store_data`.
> >>>
> >>> This is not true, consider:
> >>>
> >>>     let ptr = alloc(size = 0);
> >>>     free(ptr)
> >>>
> >>> Alloc will return a dangling pointer due to the first if statement and
> >>> then this function will pass it to `free_read_data`, even though it
> >>> wasn't created by `alloc_store_data`.
> >>> This isn't forbidden by the `Allocator` trait function's safety
> >>> requirements.
> >>>
> >>>> +            unsafe { Self::free_read_data(src) };
> >>>> +
> >>>> +            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
> >>>> +        }
> >>>> +
> >>>> +        let dst = Self::alloc(layout, flags)?;
> >>>> +
> >>>> +        // SAFETY: `src` has been created by `Self::alloc_store_data`.
> >>>> +        let data = unsafe { Self::data(src) };
> >>>
> >>> Same issue here, if the allocation passed in is zero size. I think you
> >>> have no other choice than to allocate even for zero size requests...
> >>> Otherwise how would you know that they are zero-sized.
> >>
> >> Good catch - gonna fix it.
> > 
> > Almost got me. :) I think the code is fine, callers are not allowed to pass
> > pointers to `realloc` and `free`, which haven't been allocated with the same
> > corresponding allocator or are dangling.
> 
> But what about the example above (ie the `alloc(size = 0)` and then
> `free`)?

This never has been valid for the `Allocator` trait. Look at `Kmalloc`,
`Vmalloc` and `KVmalloc`, they don't allow this either.

We've discussed this already in previous versions of this series, where for this
purpose, you asked for `old_layout` for `free`. Such that `free` can check if
the `size` was zero and therefore return without doing anything.

> I guess this all depends on how one interprets the term
> "existing, valid memory allocation". To me that describes anything an
> `Allocator` returns via `alloc` and `realloc`, including zero-sized
> allocations.

I argue that the dangling pointer returned for `size == 0` does not point to any
allocation in the sense of those allocators. It's just a dangling `[u8]`
pointer.

> But if you argue that those are not valid allocations from that
> allocator, then that is not properly documented in the safety
> requirements of `Allocator`. 

The safety requirements of `Allocator` where proposed by you and I thought they
consider this aspect?

`realloc` has:

"If `ptr == Some(p)`, then `p` must point to an existing and valid memory
allocation created by this allocator."

`free` has:

"`ptr` must point to an existing and valid memory allocation created by this
`Allocator` and must not be a dangling pointer."

We can add the part about the dangling pointer to `realloc` if you want.

> 
> ---
> Cheers,
> Benno
> 

Re: [PATCH v6 22/26] rust: alloc: implement `Cmalloc` in module allocator_test

[Benno Lossin]

On 11.09.24 16:37, Danilo Krummrich wrote:
> On Wed, Sep 11, 2024 at 01:32:31PM +0000, Benno Lossin wrote:
>> On 11.09.24 14:31, Danilo Krummrich wrote:
>>> On Fri, Aug 30, 2024 at 12:25:27AM +0200, Danilo Krummrich wrote:
>>>> On Thu, Aug 29, 2024 at 07:14:18PM +0000, Benno Lossin wrote:
>>>>> On 16.08.24 02:11, Danilo Krummrich wrote:
>>>>>> +
>>>>>> +        if layout.size() == 0 {
>>>>>> +            // SAFETY: `src` has been created by `Self::alloc_store_data`.
>>>>>
>>>>> This is not true, consider:
>>>>>
>>>>>     let ptr = alloc(size = 0);
>>>>>     free(ptr)
>>>>>
>>>>> Alloc will return a dangling pointer due to the first if statement and
>>>>> then this function will pass it to `free_read_data`, even though it
>>>>> wasn't created by `alloc_store_data`.
>>>>> This isn't forbidden by the `Allocator` trait function's safety
>>>>> requirements.
>>>>>
>>>>>> +            unsafe { Self::free_read_data(src) };
>>>>>> +
>>>>>> +            return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
>>>>>> +        }
>>>>>> +
>>>>>> +        let dst = Self::alloc(layout, flags)?;
>>>>>> +
>>>>>> +        // SAFETY: `src` has been created by `Self::alloc_store_data`.
>>>>>> +        let data = unsafe { Self::data(src) };
>>>>>
>>>>> Same issue here, if the allocation passed in is zero size. I think you
>>>>> have no other choice than to allocate even for zero size requests...
>>>>> Otherwise how would you know that they are zero-sized.
>>>>
>>>> Good catch - gonna fix it.
>>>
>>> Almost got me. :) I think the code is fine, callers are not allowed to pass
>>> pointers to `realloc` and `free`, which haven't been allocated with the same
>>> corresponding allocator or are dangling.
>>
>> But what about the example above (ie the `alloc(size = 0)` and then
>> `free`)?
> 
> This never has been valid for the `Allocator` trait. Look at `Kmalloc`,
> `Vmalloc` and `KVmalloc`, they don't allow this either.

That is true.

> We've discussed this already in previous versions of this series, where for this
> purpose, you asked for `old_layout` for `free`. Such that `free` can check if
> the `size` was zero and therefore return without doing anything.

Yes, but that was only about the old_layout parameter (at least that's
what I thought).

>> I guess this all depends on how one interprets the term
>> "existing, valid memory allocation". To me that describes anything an
>> `Allocator` returns via `alloc` and `realloc`, including zero-sized
>> allocations.
> 
> I argue that the dangling pointer returned for `size == 0` does not point to any
> allocation in the sense of those allocators. It's just a dangling `[u8]`
> pointer.

Sure, but to me the concept of zero-sized allocations does exist.

>> But if you argue that those are not valid allocations from that
>> allocator, then that is not properly documented in the safety
>> requirements of `Allocator`.
> 
> The safety requirements of `Allocator` where proposed by you and I thought they
> consider this aspect?

No, they did not consider this aspect. I was under the impression, that
we would still allow zero-sized allocations (in retrospect, this is
stupid, since dangling pointers shouldn't be passed to `krealloc` etc.).

> `realloc` has:
> 
> "If `ptr == Some(p)`, then `p` must point to an existing and valid memory
> allocation created by this allocator."
> 
> `free` has:
> 
> "`ptr` must point to an existing and valid memory allocation created by this
> `Allocator` and must not be a dangling pointer."
> 
> We can add the part about the dangling pointer to `realloc` if you want.

So I think we should do the following: 
(1) Add a paragraph to the `Allocator` trait that explains that
    zero-sized allocations are not supported.
(2) Add a check to `realloc` for zero-sized allocations + null pointer
    (ie a new allocation request) that prints a warning and returns an
    error
(3) Instead of writing "existing and valid memory allocation created by
    this allocator", I think "valid non-zero-sized memory allocation
    created by this allocator" fits better.

---
Cheers,
Benno

[PATCH v6 23/26] rust: str: test: replace `alloc::format`

[Danilo Krummrich]

The current implementation of tests in str.rs use `format!` to format
strings for comparison, which, internally, creates a new `String`.

In order to prepare for getting rid of Rust's alloc crate, we have to
cut this dependency. Instead, implement `format!` for `CString`.

Note that for userspace tests, `Kmalloc`, which is backing `CString`'s
memory, is just a type alias to `Cmalloc`.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/str.rs | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/rust/kernel/str.rs b/rust/kernel/str.rs
index 209ad1bb1a49..726329530338 100644
--- a/rust/kernel/str.rs
+++ b/rust/kernel/str.rs
@@ -523,7 +523,28 @@ macro_rules! c_str {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use alloc::format;
+
+    struct String(CString);
+
+    impl String {
+        fn from_fmt(args: fmt::Arguments<'_>) -> Self {
+            String(CString::try_from_fmt(args).unwrap())
+        }
+    }
+
+    impl Deref for String {
+        type Target = str;
+
+        fn deref(&self) -> &str {
+            self.0.to_str().unwrap()
+        }
+    }
+
+    macro_rules! format {
+        ($($f:tt)*) => ({
+            &*String::from_fmt(kernel::fmt!($($f)*))
+        })
+    }
 
     const ALL_ASCII_CHARS: &'static str =
         "\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08\\x09\\x0a\\x0b\\x0c\\x0d\\x0e\\x0f\
-- 
2.46.0

Re: [PATCH v6 23/26] rust: str: test: replace `alloc::format`

[Benno Lossin]

On 16.08.24 02:11, Danilo Krummrich wrote:
> The current implementation of tests in str.rs use `format!` to format
> strings for comparison, which, internally, creates a new `String`.
> 
> In order to prepare for getting rid of Rust's alloc crate, we have to
> cut this dependency. Instead, implement `format!` for `CString`.
> 
> Note that for userspace tests, `Kmalloc`, which is backing `CString`'s
> memory, is just a type alias to `Cmalloc`.
> 
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>

Reviewed-by: Benno Lossin <benno.lossin@proton.me>

---
Cheers,
Benno

> ---
>  rust/kernel/str.rs | 23 ++++++++++++++++++++++-
>  1 file changed, 22 insertions(+), 1 deletion(-)
> 
> diff --git a/rust/kernel/str.rs b/rust/kernel/str.rs
> index 209ad1bb1a49..726329530338 100644
> --- a/rust/kernel/str.rs
> +++ b/rust/kernel/str.rs
> @@ -523,7 +523,28 @@ macro_rules! c_str {
>  #[cfg(test)]
>  mod tests {
>      use super::*;
> -    use alloc::format;
> +
> +    struct String(CString);
> +
> +    impl String {
> +        fn from_fmt(args: fmt::Arguments<'_>) -> Self {
> +            String(CString::try_from_fmt(args).unwrap())
> +        }
> +    }
> +
> +    impl Deref for String {
> +        type Target = str;
> +
> +        fn deref(&self) -> &str {
> +            self.0.to_str().unwrap()
> +        }
> +    }
> +
> +    macro_rules! format {
> +        ($($f:tt)*) => ({
> +            &*String::from_fmt(kernel::fmt!($($f)*))
> +        })
> +    }
> 
>      const ALL_ASCII_CHARS: &'static str =
>          "\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08\\x09\\x0a\\x0b\\x0c\\x0d\\x0e\\x0f\
> --
> 2.46.0
> 

[PATCH v6 24/26] rust: alloc: update module comment of alloc.rs

[Danilo Krummrich]

Before we remove Rust's alloc crate, rewrite the module comment in
alloc.rs to avoid a rustdoc warning.

Besides that, the module comment in alloc.rs isn't correct anymore,
we're no longer extending Rust's alloc crate.

Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/kernel/alloc.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/rust/kernel/alloc.rs b/rust/kernel/alloc.rs
index 5c66229a7542..282afd184957 100644
--- a/rust/kernel/alloc.rs
+++ b/rust/kernel/alloc.rs
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 
-//! Extensions to the [`alloc`] crate.
+//! Implementation of the kernel's memory allocation infrastructure.
 
 #[cfg(not(any(test, testlib)))]
 pub mod allocator;
-- 
2.46.0

[PATCH v6 25/26] kbuild: rust: remove the `alloc` crate and `GlobalAlloc`

[Danilo Krummrich]

Now that we have our own `Allocator`, `Box` and `Vec` types we can remove
Rust's `alloc` crate and the `new_uninit` unstable feature.

Also remove `Kmalloc`'s `GlobalAlloc` implementation -- we can't remove
this in a separate patch, since the `alloc` crate requires a
`#[global_allocator]` to set, that implements `GlobalAlloc`.

Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 rust/Makefile                     | 44 ++++++-----------------
 rust/exports.c                    |  1 -
 rust/kernel/alloc/allocator.rs    | 59 ++-----------------------------
 scripts/Makefile.build            |  7 +---
 scripts/generate_rust_analyzer.py | 11 ++----
 5 files changed, 15 insertions(+), 107 deletions(-)

diff --git a/rust/Makefile b/rust/Makefile
index 77836388377d..8963f53c5d6c 100644
--- a/rust/Makefile
+++ b/rust/Makefile
@@ -15,9 +15,8 @@ always-$(CONFIG_RUST) += libmacros.so
 no-clean-files += libmacros.so
 
 always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs
-obj-$(CONFIG_RUST) += alloc.o bindings.o kernel.o
-always-$(CONFIG_RUST) += exports_alloc_generated.h exports_bindings_generated.h \
-    exports_kernel_generated.h
+obj-$(CONFIG_RUST) += bindings.o kernel.o
+always-$(CONFIG_RUST) += exports_bindings_generated.h exports_kernel_generated.h
 
 always-$(CONFIG_RUST) += uapi/uapi_generated.rs
 obj-$(CONFIG_RUST) += uapi.o
@@ -53,11 +52,6 @@ endif
 core-cfgs = \
     --cfg no_fp_fmt_parse
 
-alloc-cfgs = \
-    --cfg no_global_oom_handling \
-    --cfg no_rc \
-    --cfg no_sync
-
 quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
       cmd_rustdoc = \
 	OBJTREE=$(abspath $(objtree)) \
@@ -80,7 +74,7 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
 # command-like flags to solve the issue. Meanwhile, we use the non-custom case
 # and then retouch the generated files.
 rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \
-    rustdoc-alloc rustdoc-kernel
+    rustdoc-kernel
 	$(Q)cp $(srctree)/Documentation/images/logo.svg $(rustdoc_output)/static.files/
 	$(Q)cp $(srctree)/Documentation/images/COPYING-logo $(rustdoc_output)/static.files/
 	$(Q)find $(rustdoc_output) -name '*.html' -type f -print0 | xargs -0 sed -Ei \
@@ -104,20 +98,11 @@ rustdoc-core: $(RUST_LIB_SRC)/core/src/lib.rs FORCE
 rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE
 	+$(call if_changed,rustdoc)
 
-# We need to allow `rustdoc::broken_intra_doc_links` because some
-# `no_global_oom_handling` functions refer to non-`no_global_oom_handling`
-# functions. Ideally `rustdoc` would have a way to distinguish broken links
-# due to things that are "configured out" vs. entirely non-existing ones.
-rustdoc-alloc: private rustc_target_flags = $(alloc-cfgs) \
-    -Arustdoc::broken_intra_doc_links
-rustdoc-alloc: $(RUST_LIB_SRC)/alloc/src/lib.rs rustdoc-core rustdoc-compiler_builtins FORCE
-	+$(call if_changed,rustdoc)
-
-rustdoc-kernel: private rustc_target_flags = --extern alloc \
+rustdoc-kernel: private rustc_target_flags = \
     --extern build_error --extern macros=$(objtree)/$(obj)/libmacros.so \
     --extern bindings --extern uapi
 rustdoc-kernel: $(src)/kernel/lib.rs rustdoc-core rustdoc-macros \
-    rustdoc-compiler_builtins rustdoc-alloc $(obj)/libmacros.so \
+    rustdoc-compiler_builtins $(obj)/libmacros.so \
     $(obj)/bindings.o FORCE
 	+$(call if_changed,rustdoc)
 
@@ -161,7 +146,7 @@ quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $<
 	mkdir -p $(objtree)/$(obj)/test/doctests/kernel; \
 	OBJTREE=$(abspath $(objtree)) \
 	$(RUSTDOC) --test $(rust_flags) \
-		-L$(objtree)/$(obj) --extern alloc --extern kernel \
+		-L$(objtree)/$(obj) --extern kernel \
 		--extern build_error --extern macros \
 		--extern bindings --extern uapi \
 		--no-run --crate-name kernel -Zunstable-options \
@@ -197,7 +182,7 @@ rusttest-macros: $(src)/macros/lib.rs FORCE
 	+$(call if_changed,rustc_test)
 	+$(call if_changed,rustdoc_test)
 
-rusttest-kernel: private rustc_target_flags = --extern alloc \
+rusttest-kernel: private rustc_target_flags = \
     --extern build_error --extern macros --extern bindings --extern uapi
 rusttest-kernel: $(src)/kernel/lib.rs \
     rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \
@@ -310,9 +295,6 @@ quiet_cmd_exports = EXPORTS $@
 $(obj)/exports_core_generated.h: $(obj)/core.o FORCE
 	$(call if_changed,exports)
 
-$(obj)/exports_alloc_generated.h: $(obj)/alloc.o FORCE
-	$(call if_changed,exports)
-
 $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
 	$(call if_changed,exports)
 
@@ -348,7 +330,7 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L
 
 rust-analyzer:
 	$(Q)$(srctree)/scripts/generate_rust_analyzer.py \
-		--cfgs='core=$(core-cfgs)' --cfgs='alloc=$(alloc-cfgs)' \
+		--cfgs='core=$(core-cfgs)' \
 		$(realpath $(srctree)) $(realpath $(objtree)) \
 		$(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \
 		$(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json
@@ -380,12 +362,6 @@ $(obj)/compiler_builtins.o: private rustc_objcopy = -w -W '__*'
 $(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE
 	+$(call if_changed_dep,rustc_library)
 
-$(obj)/alloc.o: private skip_clippy = 1
-$(obj)/alloc.o: private skip_flags = -Wunreachable_pub
-$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs)
-$(obj)/alloc.o: $(RUST_LIB_SRC)/alloc/src/lib.rs $(obj)/compiler_builtins.o FORCE
-	+$(call if_changed_dep,rustc_library)
-
 $(obj)/build_error.o: $(src)/build_error.rs $(obj)/compiler_builtins.o FORCE
 	+$(call if_changed_dep,rustc_library)
 
@@ -400,9 +376,9 @@ $(obj)/uapi.o: $(src)/uapi/lib.rs \
     $(obj)/uapi/uapi_generated.rs FORCE
 	+$(call if_changed_dep,rustc_library)
 
-$(obj)/kernel.o: private rustc_target_flags = --extern alloc \
+$(obj)/kernel.o: private rustc_target_flags = \
     --extern build_error --extern macros --extern bindings --extern uapi
-$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \
+$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/build_error.o \
     $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o FORCE
 	+$(call if_changed_dep,rustc_library)
 
diff --git a/rust/exports.c b/rust/exports.c
index 3803c21d1403..1b870e8e83ea 100644
--- a/rust/exports.c
+++ b/rust/exports.c
@@ -16,7 +16,6 @@
 #define EXPORT_SYMBOL_RUST_GPL(sym) extern int sym; EXPORT_SYMBOL_GPL(sym)
 
 #include "exports_core_generated.h"
-#include "exports_alloc_generated.h"
 #include "exports_bindings_generated.h"
 #include "exports_kernel_generated.h"
 
diff --git a/rust/kernel/alloc/allocator.rs b/rust/kernel/alloc/allocator.rs
index 0183fbac1ada..44eccc2b1819 100644
--- a/rust/kernel/alloc/allocator.rs
+++ b/rust/kernel/alloc/allocator.rs
@@ -8,8 +8,8 @@
 //!
 //! Reference: <https://docs.kernel.org/core-api/memory-allocation.html>
 
-use super::{flags::*, Flags};
-use core::alloc::{GlobalAlloc, Layout};
+use super::Flags;
+use core::alloc::Layout;
 use core::ptr;
 use core::ptr::NonNull;
 
@@ -54,23 +54,6 @@ fn aligned_size(new_layout: Layout) -> usize {
     layout.size()
 }
 
-/// Calls `krealloc` with a proper size to alloc a new object aligned to `new_layout`'s alignment.
-///
-/// # Safety
-///
-/// - `ptr` can be either null or a pointer which has been allocated by this allocator.
-/// - `new_layout` must have a non-zero size.
-pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: Flags) -> *mut u8 {
-    let size = aligned_size(new_layout);
-
-    // SAFETY:
-    // - `ptr` is either null or a pointer returned from a previous `k{re}alloc()` by the
-    //   function safety requirement.
-    // - `size` is greater than 0 since it's from `layout.size()` (which cannot be zero according
-    //   to the function safety requirement)
-    unsafe { bindings::krealloc(ptr as *const core::ffi::c_void, size, flags.0) as *mut u8 }
-}
-
 /// # Invariants
 ///
 /// One of the following `krealloc`, `vrealloc`, `kvrealloc`.
@@ -131,41 +114,6 @@ unsafe fn realloc(
     }
 }
 
-unsafe impl GlobalAlloc for Kmalloc {
-    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
-        // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
-        // requirement.
-        unsafe { krealloc_aligned(ptr::null_mut(), layout, GFP_KERNEL) }
-    }
-
-    unsafe fn dealloc(&self, ptr: *mut u8, _layout: Layout) {
-        unsafe {
-            bindings::kfree(ptr as *const core::ffi::c_void);
-        }
-    }
-
-    unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
-        // SAFETY:
-        // - `new_size`, when rounded up to the nearest multiple of `layout.align()`, will not
-        //   overflow `isize` by the function safety requirement.
-        // - `layout.align()` is a proper alignment (i.e. not zero and must be a power of two).
-        let layout = unsafe { Layout::from_size_align_unchecked(new_size, layout.align()) };
-
-        // SAFETY:
-        // - `ptr` is either null or a pointer allocated by this allocator by the function safety
-        //   requirement.
-        // - the size of `layout` is not zero because `new_size` is not zero by the function safety
-        //   requirement.
-        unsafe { krealloc_aligned(ptr, layout, GFP_KERNEL) }
-    }
-
-    unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
-        // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
-        // requirement.
-        unsafe { krealloc_aligned(ptr::null_mut(), layout, GFP_KERNEL | __GFP_ZERO) }
-    }
-}
-
 unsafe impl Allocator for Vmalloc {
     #[inline]
     unsafe fn realloc(
@@ -204,9 +152,6 @@ unsafe fn realloc(
     }
 }
 
-#[global_allocator]
-static ALLOCATOR: Kmalloc = Kmalloc;
-
 // See <https://github.com/rust-lang/rust/pull/86844>.
 #[no_mangle]
 static __rust_no_alloc_shim_is_unstable: u8 = 0;
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index efacca63c897..7e7b6b3d5bb9 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -262,18 +262,13 @@ $(obj)/%.lst: $(obj)/%.c FORCE
 
 # Compile Rust sources (.rs)
 # ---------------------------------------------------------------------------
-
-rust_allowed_features := new_uninit
-
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 rust_common_cmd = \
 	RUST_MODFILE=$(modfile) $(RUSTC_OR_CLIPPY) $(rust_flags) \
-	-Zallow-features=$(rust_allowed_features) \
 	-Zcrate-attr=no_std \
-	-Zcrate-attr='feature($(rust_allowed_features))' \
-	-Zunstable-options --extern force:alloc --extern kernel \
+	-Zunstable-options --extern kernel \
 	--crate-type rlib -L $(objtree)/rust/ \
 	--crate-name $(basename $(notdir $@)) \
 	--sysroot=/dev/null \
diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
index f270c7b0cf34..3421a9ea3604 100755
--- a/scripts/generate_rust_analyzer.py
+++ b/scripts/generate_rust_analyzer.py
@@ -64,13 +64,6 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs):
         [],
     )
 
-    append_crate(
-        "alloc",
-        sysroot_src / "alloc" / "src" / "lib.rs",
-        ["core", "compiler_builtins"],
-        cfg=crates_cfgs.get("alloc", []),
-    )
-
     append_crate(
         "macros",
         srctree / "rust" / "macros" / "lib.rs",
@@ -96,7 +89,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs):
     append_crate(
         "kernel",
         srctree / "rust" / "kernel" / "lib.rs",
-        ["core", "alloc", "macros", "build_error", "bindings"],
+        ["core", "macros", "build_error", "bindings"],
         cfg=cfg,
     )
     crates[-1]["source"] = {
@@ -133,7 +126,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs):
             append_crate(
                 name,
                 path,
-                ["core", "alloc", "kernel"],
+                ["core", "kernel"],
                 cfg=cfg,
             )
 
-- 
2.46.0

Re: [PATCH v6 25/26] kbuild: rust: remove the `alloc` crate and `GlobalAlloc`

[Benno Lossin]

On 16.08.24 02:11, Danilo Krummrich wrote:
> @@ -204,9 +152,6 @@ unsafe fn realloc(
>      }
>  }
> 
> -#[global_allocator]
> -static ALLOCATOR: Kmalloc = Kmalloc;
> -
>  // See <https://github.com/rust-lang/rust/pull/86844>.
>  #[no_mangle]
>  static __rust_no_alloc_shim_is_unstable: u8 = 0;

I think Bjorn already mentioned this, but you can also remove this
static, since it was also something that alloc needed.

---
Cheers,
Benno

[PATCH v6 26/26] MAINTAINERS: add entry for the Rust `alloc` module

[Danilo Krummrich]

Add maintainers entry for the Rust `alloc` module.

Currently, this includes the `Allocator` API itself, `Allocator`
implementations, such as `Kmalloc` or `Vmalloc`, as well as the kernel's
implementation of the primary memory allocation data structures, `Box`
and `Vec`.

Signed-off-by: Danilo Krummrich <dakr@kernel.org>
---
 MAINTAINERS | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 42decde38320..560516b3aaf4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -19925,6 +19925,13 @@ F:	scripts/*rust*
 F:	tools/testing/selftests/rust/
 K:	\b(?i:rust)\b
 
+RUST [ALLOC]
+M:	Danilo Krummrich <dakr@kernel.org>
+L:	rust-for-linux@vger.kernel.org
+S:	Maintained
+F:	rust/kernel/alloc.rs
+F:	rust/kernel/alloc/
+
 RXRPC SOCKETS (AF_RXRPC)
 M:	David Howells <dhowells@redhat.com>
 M:	Marc Dionne <marc.dionne@auristor.com>
-- 
2.46.0

Re: [PATCH v6 26/26] MAINTAINERS: add entry for the Rust `alloc` module

[Gary Guo]

On Fri, 16 Aug 2024 02:11:08 +0200
Danilo Krummrich <dakr@kernel.org> wrote:

> Add maintainers entry for the Rust `alloc` module.
> 
> Currently, this includes the `Allocator` API itself, `Allocator`
> implementations, such as `Kmalloc` or `Vmalloc`, as well as the kernel's
> implementation of the primary memory allocation data structures, `Box`
> and `Vec`.
> 
> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> ---
>  MAINTAINERS | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 42decde38320..560516b3aaf4 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -19925,6 +19925,13 @@ F:	scripts/*rust*
>  F:	tools/testing/selftests/rust/
>  K:	\b(?i:rust)\b
>  
> +RUST [ALLOC]
> +M:	Danilo Krummrich <dakr@kernel.org>
> +L:	rust-for-linux@vger.kernel.org
> +S:	Maintained
> +F:	rust/kernel/alloc.rs
> +F:	rust/kernel/alloc/

It feels like we should use `mod.rs`.

> +
>  RXRPC SOCKETS (AF_RXRPC)
>  M:	David Howells <dhowells@redhat.com>
>  M:	Marc Dionne <marc.dionne@auristor.com>

Re: [PATCH v6 26/26] MAINTAINERS: add entry for the Rust `alloc` module

[Danilo Krummrich]

On Sat, Aug 31, 2024 at 01:57:12PM +0100, Gary Guo wrote:
> On Fri, 16 Aug 2024 02:11:08 +0200
> Danilo Krummrich <dakr@kernel.org> wrote:
> 
> > Add maintainers entry for the Rust `alloc` module.
> > 
> > Currently, this includes the `Allocator` API itself, `Allocator`
> > implementations, such as `Kmalloc` or `Vmalloc`, as well as the kernel's
> > implementation of the primary memory allocation data structures, `Box`
> > and `Vec`.
> > 
> > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > ---
> >  MAINTAINERS | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 42decde38320..560516b3aaf4 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -19925,6 +19925,13 @@ F:	scripts/*rust*
> >  F:	tools/testing/selftests/rust/
> >  K:	\b(?i:rust)\b
> >  
> > +RUST [ALLOC]
> > +M:	Danilo Krummrich <dakr@kernel.org>
> > +L:	rust-for-linux@vger.kernel.org
> > +S:	Maintained
> > +F:	rust/kernel/alloc.rs
> > +F:	rust/kernel/alloc/
> 
> It feels like we should use `mod.rs`.

The same would be true for:

- rust/kernel/sync.rs
- rust/kernel/net.rs
- rust/kernel/init.rs
- rust/kernel/fs.rs
- ...

Do you propose to change it for all of them?

> 
> > +
> >  RXRPC SOCKETS (AF_RXRPC)
> >  M:	David Howells <dhowells@redhat.com>
> >  M:	Marc Dionne <marc.dionne@auristor.com>
> 

Re: [PATCH v6 26/26] MAINTAINERS: add entry for the Rust `alloc` module

[Alice Ryhl]

On Tue, Sep 3, 2024 at 2:03 PM Danilo Krummrich <dakr@kernel.org> wrote:
>
> On Sat, Aug 31, 2024 at 01:57:12PM +0100, Gary Guo wrote:
> > On Fri, 16 Aug 2024 02:11:08 +0200
> > Danilo Krummrich <dakr@kernel.org> wrote:
> >
> > > Add maintainers entry for the Rust `alloc` module.
> > >
> > > Currently, this includes the `Allocator` API itself, `Allocator`
> > > implementations, such as `Kmalloc` or `Vmalloc`, as well as the kernel's
> > > implementation of the primary memory allocation data structures, `Box`
> > > and `Vec`.
> > >
> > > Signed-off-by: Danilo Krummrich <dakr@kernel.org>
> > > ---
> > >  MAINTAINERS | 7 +++++++
> > >  1 file changed, 7 insertions(+)
> > >
> > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > index 42decde38320..560516b3aaf4 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -19925,6 +19925,13 @@ F: scripts/*rust*
> > >  F: tools/testing/selftests/rust/
> > >  K: \b(?i:rust)\b
> > >
> > > +RUST [ALLOC]
> > > +M: Danilo Krummrich <dakr@kernel.org>
> > > +L: rust-for-linux@vger.kernel.org
> > > +S: Maintained
> > > +F: rust/kernel/alloc.rs
> > > +F: rust/kernel/alloc/
> >
> > It feels like we should use `mod.rs`.
>
> The same would be true for:
>
> - rust/kernel/sync.rs
> - rust/kernel/net.rs
> - rust/kernel/init.rs
> - rust/kernel/fs.rs
> - ...
>
> Do you propose to change it for all of them?

I do actually think `mod.rs` is superior in general, but it's probably
not worth changing it right now.

Alice

Re: [PATCH v6 26/26] MAINTAINERS: add entry for the Rust `alloc` module

[Benno Lossin]

On 04.09.24 12:15, Alice Ryhl wrote:
> On Tue, Sep 3, 2024 at 2:03 PM Danilo Krummrich <dakr@kernel.org> wrote:
>>
>> On Sat, Aug 31, 2024 at 01:57:12PM +0100, Gary Guo wrote:
>>> On Fri, 16 Aug 2024 02:11:08 +0200
>>> Danilo Krummrich <dakr@kernel.org> wrote:
>>>
>>>> Add maintainers entry for the Rust `alloc` module.
>>>>
>>>> Currently, this includes the `Allocator` API itself, `Allocator`
>>>> implementations, such as `Kmalloc` or `Vmalloc`, as well as the kernel's
>>>> implementation of the primary memory allocation data structures, `Box`
>>>> and `Vec`.
>>>>
>>>> Signed-off-by: Danilo Krummrich <dakr@kernel.org>
>>>> ---
>>>>  MAINTAINERS | 7 +++++++
>>>>  1 file changed, 7 insertions(+)
>>>>
>>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>>> index 42decde38320..560516b3aaf4 100644
>>>> --- a/MAINTAINERS
>>>> +++ b/MAINTAINERS
>>>> @@ -19925,6 +19925,13 @@ F: scripts/*rust*
>>>>  F: tools/testing/selftests/rust/
>>>>  K: \b(?i:rust)\b
>>>>
>>>> +RUST [ALLOC]
>>>> +M: Danilo Krummrich <dakr@kernel.org>
>>>> +L: rust-for-linux@vger.kernel.org
>>>> +S: Maintained
>>>> +F: rust/kernel/alloc.rs
>>>> +F: rust/kernel/alloc/
>>>
>>> It feels like we should use `mod.rs`.
>>
>> The same would be true for:
>>
>> - rust/kernel/sync.rs
>> - rust/kernel/net.rs
>> - rust/kernel/init.rs
>> - rust/kernel/fs.rs
>> - ...
>>
>> Do you propose to change it for all of them?
> 
> I do actually think `mod.rs` is superior in general, but it's probably
> not worth changing it right now.

I forgot the reason for not using it, does anyone remember?

---
Cheers,
Benno

Re: [PATCH v6 26/26] MAINTAINERS: add entry for the Rust `alloc` module

[Miguel Ojeda]

On Wed, Sep 4, 2024 at 2:51 PM Benno Lossin <benno.lossin@proton.me> wrote:
>
> I forgot the reason for not using it, does anyone remember?

One of the reasons argued was that `mod.rs` is the same name
everywhere, and thus it is hard to notice the difference in some
editors and may be harder to "jump into" in history/tabs/etc.

Cheers,
Miguel

Re: [PATCH v6 26/26] MAINTAINERS: add entry for the Rust `alloc` module

[Benno Lossin]

On 04.09.24 14:57, Miguel Ojeda wrote:
> On Wed, Sep 4, 2024 at 2:51 PM Benno Lossin <benno.lossin@proton.me> wrote:
>>
>> I forgot the reason for not using it, does anyone remember?
> 
> One of the reasons argued was that `mod.rs` is the same name
> everywhere, and thus it is hard to notice the difference in some
> editors and may be harder to "jump into" in history/tabs/etc.

I don't usually open more than 2-4 files anyways, so it's not an issue
for me. But of course people's workflow is different, does anyone have a
problem with switching to `mod.rs`?

I have another reason that it might be nicer, when I want to edit eg
`init.rs` and I decide to directly open it with the editor from my
shell, not using my fuzzy finder, then I need to press tab to select
between `rust/kernel/init` and `rust/kernel/init.rs`.

---
Cheers,
Benno

Re: [PATCH v6 26/26] MAINTAINERS: add entry for the Rust `alloc` module

[Danilo Krummrich]

On Tue, Sep 10, 2024 at 01:26:34PM +0000, Benno Lossin wrote:
> On 04.09.24 14:57, Miguel Ojeda wrote:
> > On Wed, Sep 4, 2024 at 2:51 PM Benno Lossin <benno.lossin@proton.me> wrote:
> >>
> >> I forgot the reason for not using it, does anyone remember?
> > 
> > One of the reasons argued was that `mod.rs` is the same name
> > everywhere, and thus it is hard to notice the difference in some
> > editors and may be harder to "jump into" in history/tabs/etc.
> 
> I don't usually open more than 2-4 files anyways, so it's not an issue
> for me. But of course people's workflow is different, does anyone have a
> problem with switching to `mod.rs`?

I'm also not against it. I'd appreciate if a potential change doesn't interfere
with this series too much though. :-)

> 
> I have another reason that it might be nicer, when I want to edit eg
> `init.rs` and I decide to directly open it with the editor from my
> shell, not using my fuzzy finder, then I need to press tab to select
> between `rust/kernel/init` and `rust/kernel/init.rs`.
> 
> ---
> Cheers,
> Benno
> 

Re: [PATCH v6 26/26] MAINTAINERS: add entry for the Rust `alloc` module

[Benno Lossin]

On 10.09.24 15:42, Danilo Krummrich wrote:
> On Tue, Sep 10, 2024 at 01:26:34PM +0000, Benno Lossin wrote:
>> On 04.09.24 14:57, Miguel Ojeda wrote:
>>> On Wed, Sep 4, 2024 at 2:51 PM Benno Lossin <benno.lossin@proton.me> wrote:
>>>>
>>>> I forgot the reason for not using it, does anyone remember?
>>>
>>> One of the reasons argued was that `mod.rs` is the same name
>>> everywhere, and thus it is hard to notice the difference in some
>>> editors and may be harder to "jump into" in history/tabs/etc.
>>
>> I don't usually open more than 2-4 files anyways, so it's not an issue
>> for me. But of course people's workflow is different, does anyone have a
>> problem with switching to `mod.rs`?
> 
> I'm also not against it. I'd appreciate if a potential change doesn't interfere
> with this series too much though. :-)

Sure, we can do it after the series.

---
Cheers,
Benno

Re: [PATCH v6 00/26] Generic `Allocator` support for Rust

[Boqun Feng]

On Fri, Aug 16, 2024 at 02:10:42AM +0200, Danilo Krummrich wrote:
> Hi,
> 
> This patch series adds generic kernel allocator support for Rust, which so far
> is limited to `kmalloc` allocations.
> 
> In order to abstain from (re-)adding unstable Rust features to the kernel, this
> patch series does not extend the `Allocator` trait from Rust's `alloc` crate,
> nor does it extend the `BoxExt` and `VecExt` extensions.
> 
> Instead, this series introduces a kernel specific `Allocator` trait, which is
> implemented by the `Kmalloc`, `Vmalloc` and `KVmalloc` allocators, also
> implemented in the context of this series.
> 
> As a consequence we need our own kernel `Box<T, A>` and `Vec<T, A>` types.
> Additionally, this series adds the following type aliases:
> 
> ```
> pub type KBox<T> = Box<T, Kmalloc>;
> pub type VBox<T> = Box<T, Vmalloc>;
> pub type KVBox<T> = Box<T, KVmalloc>;
> 
> 
> pub type KVec<T> = Vec<T, Kmalloc>;
> pub type VVec<T> = Vec<T, Vmalloc>;
> pub type KVVec<T> = Vec<T, KVmalloc>;
> ```
> 
> With that, we can start using the kernel `Box` and `Vec` types throughout the
> tree and remove the now obolete extensions `BoxExt` and `VecExt`.
> 
> For a final cleanup, this series removes the last minor dependencies to Rust's
> `alloc` crate and removes it from the entire kernel build.
> 
> The series ensures not to break the `rusttest` make target by implementing the
> `allocator_test` module providing a stub implementation for all kernel
> `Allocator`s.
> 
> This patch series passes all KUnit tests, including the ones added by this
> series. Additionally, the tests were run with `kmemleak` and `KASAN` enabled,
> without any issues.
> 
> This series is based on [1], which hit -mm/mm-unstable, and is also available
> in [2].
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=mm/krealloc
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=rust/mm
> 

Finally managed to apply this on rust-dev ;-) A few things I resolved,
JFYI:

*	Resolve the conflicts between patch 6 & 7 and the helper split,
	note that the vmalloc() helper temporarily lives in a file
	called `slab.c`.

*	Adjust patch 9 with the introduction of InPlaceInit::PinnedSelf,
	also drop the ForeignOwnable::borrow_mut() implemenation.

*	Resolve the conflict between patch 10 and the latest rbtree
	patchset (v12).

*	Adjust patch 11 with the introduction of
	InPlaceInit::PinnedSelf.

*	Resolve the conflicts between patch 25 and the helper split.

(very likely I could make mistakes during these conflict resolution,
just let me know)

I've put this on the rust-dev based on today's rust-next (+ your mm
changes) for testing, will continue to review the series, thanks!

Regards,
Boqun

> Changes in v6:
>  - rebase onto rust-dev
>  - keep compiler annotations for {k,v,kv}realloc()
>  - documentation changes suggested by Alice, Benno
>  - remove `Box::into_pin`
>  - fix typo in `Send` and `Sync` for `Box` and `Vec`
>  - `kvec!` changes suggested by Alice
>  - free `src` after copy in `Cmalloc`
>  - handle `n == 0` in `Vec::extend_with`
> 
> Changes in v5:
>  - (safety) comment / documentation fixes suggested by Alice, Benno and Gary
>  - remove `Unique<T>` and implement `Send` and `Sync` for `Box` and `Vec`
>  - use `KMALLOC_MAX_SIZE` for `KVmalloc` test and add a `Kmalloc` test that
>    expects to fail for `KMALLOC_MAX_SIZE`
>  - create use constants `KREALLOC`, `VREALLOC` and `KVREALLOC` for
>    `ReallocFuncs`
>  - drop `Box::drop_contents` for now, will add it again, once I actually rebase
>    on the original patch that introduces it
>  - improve usage of `size_of_val` in `Box`
>  - move `InPlaceInit` and `ForeignOwnable` impls into kbox.rs
>  - fix missing `Box` conversions in rnull.rs
>  - reworked `Cmalloc` to keep track of the size of memory allocations itself
>  - remove `GlobalAlloc` together with the `alloc` crate to avoid a linker error
>  - remove `alloc` from scripts/generate_rust_analyzer.py
> 
> Changes in v4:
>  - (safety) comment fixes suggested by Alice and Boqun
>  - remove `Box::from_raw_alloc` and `Box::into_raw_alloc`, we don't need them
>  - in `Box::drop` call `size_of_val` before `drop_in_place`
>  - implement ForeignOwnable for Pin<Box<T>> as suggested by Alice
>  - in `Vec::extend_with`, iterate over `n` instead of `spare.len()`
>  - for `Vmalloc` and `KVmalloc` fail allocation for alignments larger than
>    PAGE_SIZE for now (will add support for larger alignments in a separate
>    series)
>  - implement `Cmalloc` in `allocator_test` and type alias all kernel allocator
>    types to it, such that we can use the kernel's `Box` and `Vec` types in
>    userspace tests (rusttest)
>    - this makes patch "rust: str: test: replace `alloc::format`" rather trivial
> 
> Changes in v3:
>  - Box:
>    - minor documentation fixes
>    - removed unnecessary imports in doc tests
>    - dropeed `self` argument from some remaining `Box` methods
>    - implement `InPlaceInit` for Box<T, A> rather than specifically for `KBox<T>`
>  - Vec:
>    - minor documentation fixes
>    - removed useless `Vec::allocator` method
>    - in `Vec::extend_with` use `Vec::spare_capacity_mut` instead of raw pointer operations
>    - added a few missing safety comments
>    - pass GFP flags to `Vec::collect`
>  - fixed a rustdoc warning in alloc.rs
>  - fixed the allocator_test module to implement the `Allocator` trait correctly
>  - rebased to rust-next
> 
> Changes in v2:
>   - preserve `impl GlobalAlloc for Kmalloc` and remove it at the end (Benno)
>   - remove `&self` parameter from all `Allocator` functions (Benno)
>   - various documentation fixes for `Allocator` (Benno)
>   - use `NonNull<u8>` for `Allocator::free` and `Option<NonNull<u8>>` for
>     `Allocator::realloc` (Benno)
>   - fix leak of `IntoIter` in `Vec::collect` (Boqun)
>   - always realloc (try to shrink) in `Vec::collect`, it's up the the
>     `Allocator` to provide a heuristic whether it makes sense to actually shrink
>   - rename `KBox<T, A>` -> `Box<T, A>` and `KVec<T, A>` -> `Vec<T, A>` and
>     provide type aliases `KBox<T>`, `VBox<T>`, `KVBox<T>`, etc.
>     - This allows for much cleaner code and, in combination with removing
>       `&self` parameters from `Allocator`s, gets us rid of the need for
>       `Box::new` and `Box::new_alloc` and all other "_alloc" postfixed
>       functions.
>     - Before: `KBox::new_alloc(foo, Vmalloc)?`
>     - After:  `VBox::new(foo)?`, which resolves to
>               `Box::<Foo,  Vmalloc>::new(foo)?;
> 
> 
> Danilo Krummrich (26):
>   rust: alloc: add `Allocator` trait
>   rust: alloc: separate `aligned_size` from `krealloc_aligned`
>   rust: alloc: rename `KernelAllocator` to `Kmalloc`
>   rust: alloc: implement `Allocator` for `Kmalloc`
>   rust: alloc: add module `allocator_test`
>   rust: alloc: implement `Vmalloc` allocator
>   rust: alloc: implement `KVmalloc` allocator
>   rust: alloc: add __GFP_NOWARN to `Flags`
>   rust: alloc: implement kernel `Box`
>   rust: treewide: switch to our kernel `Box` type
>   rust: alloc: remove `BoxExt` extension
>   rust: alloc: add `Box` to prelude
>   rust: alloc: implement kernel `Vec` type
>   rust: alloc: implement `IntoIterator` for `Vec`
>   rust: alloc: implement `collect` for `IntoIter`
>   rust: treewide: switch to the kernel `Vec` type
>   rust: alloc: remove `VecExt` extension
>   rust: alloc: add `Vec` to prelude
>   rust: error: use `core::alloc::LayoutError`
>   rust: error: check for config `test` in `Error::name`
>   rust: alloc: implement `contains` for `Flags`
>   rust: alloc: implement `Cmalloc` in module allocator_test
>   rust: str: test: replace `alloc::format`
>   rust: alloc: update module comment of alloc.rs
>   kbuild: rust: remove the `alloc` crate and `GlobalAlloc`
>   MAINTAINERS: add entry for the Rust `alloc` module
> 
>  MAINTAINERS                         |   7 +
>  drivers/block/rnull.rs              |   4 +-
>  rust/Makefile                       |  44 +-
>  rust/bindings/bindings_helper.h     |   1 +
>  rust/exports.c                      |   1 -
>  rust/helpers.c                      |  15 +
>  rust/kernel/alloc.rs                | 142 ++++-
>  rust/kernel/alloc/allocator.rs      | 173 ++++--
>  rust/kernel/alloc/allocator_test.rs | 185 ++++++
>  rust/kernel/alloc/box_ext.rs        |  80 ---
>  rust/kernel/alloc/kbox.rs           | 480 +++++++++++++++
>  rust/kernel/alloc/kvec.rs           | 891 ++++++++++++++++++++++++++++
>  rust/kernel/alloc/vec_ext.rs        | 185 ------
>  rust/kernel/error.rs                |   6 +-
>  rust/kernel/init.rs                 |  93 +--
>  rust/kernel/init/__internal.rs      |   2 +-
>  rust/kernel/lib.rs                  |   1 -
>  rust/kernel/prelude.rs              |   5 +-
>  rust/kernel/rbtree.rs               |  34 +-
>  rust/kernel/str.rs                  |  35 +-
>  rust/kernel/sync/arc.rs             |  17 +-
>  rust/kernel/sync/condvar.rs         |   4 +-
>  rust/kernel/sync/lock/mutex.rs      |   2 +-
>  rust/kernel/sync/lock/spinlock.rs   |   2 +-
>  rust/kernel/sync/locked_by.rs       |   2 +-
>  rust/kernel/types.rs                |  30 +-
>  rust/kernel/uaccess.rs              |  17 +-
>  rust/kernel/workqueue.rs            |  20 +-
>  rust/macros/lib.rs                  |  12 +-
>  samples/rust/rust_minimal.rs        |   4 +-
>  scripts/Makefile.build              |   7 +-
>  scripts/generate_rust_analyzer.py   |  11 +-
>  32 files changed, 1973 insertions(+), 539 deletions(-)
>  create mode 100644 rust/kernel/alloc/allocator_test.rs
>  delete mode 100644 rust/kernel/alloc/box_ext.rs
>  create mode 100644 rust/kernel/alloc/kbox.rs
>  create mode 100644 rust/kernel/alloc/kvec.rs
>  delete mode 100644 rust/kernel/alloc/vec_ext.rs
> 
> 
> base-commit: f005c686d6c1a2e66f2f9d21179d6b6bd45b20e2
> -- 
> 2.46.0
>