[RFC PATCH 00/11] [RFC] Device / Driver and PCI Rust abstractions

[RFC PATCH 00/11] [RFC] Device / Driver and PCI Rust abstractions

[Danilo Krummrich]

This patch sereis implements basic generic device / driver Rust abstractions,
as well as some basic PCI abstractions.

This patch series is sent in the context of [1], and the corresponding patch
series [2], which contains some basic DRM Rust abstractions and a stub
implementation of the Nova GPU driver.

Nova is intended to be developed upstream, starting out with just a stub driver
to lift some initial required infrastructure upstream. A more detailed
explanation can be found in [1].

Some patches, which implement the generic device / driver Rust abstractions have
been sent a couple of weeks ago already [3]. For those patches the following
changes have been made since then:

- remove RawDevice::name()
- remove rust helper for dev_name() and dev_get_drvdata()
- use AlwaysRefCounted for struct Device
- drop trait RawDevice entirely in favor of AsRef and provide
  Device::from_raw(), Device::as_raw() and Device::as_ref() instead
- implement RevocableGuard
- device::Data, remove resources and replace it with a Devres abstraction
- implement Devres abstraction for resources

As mentioned above, a driver serving as example on how these abstractions are
used within a (DRM) driver can be found in [2].

Additionally, the device / driver bits can also be found in [3], all
abstractions required for Nova in [4] and Nova in [5].

[1] https://lore.kernel.org/dri-devel/Zfsj0_tb-0-tNrJy@cassiopeiae/T/#u
[2] https://lore.kernel.org/dri-devel/20240520172059.181256-1-dakr@redhat.com/
[3] https://github.com/Rust-for-Linux/linux/tree/staging/rust-device
[4] https://github.com/Rust-for-Linux/linux/tree/staging/dev
[5] https://gitlab.freedesktop.org/drm/nova/-/tree/nova-next

Danilo Krummrich (2):
  rust: add abstraction for struct device
  rust: add devres abstraction

FUJITA Tomonori (1):
  rust: add basic PCI driver abstractions

Philipp Stanner (2):
  rust: add basic abstractions for iomem operations
  rust: PCI: add BAR request and ioremap

Wedson Almeida Filho (6):
  rust: add driver abstraction
  rust: add rcu abstraction
  rust: add revocable mutex
  rust: add revocable objects
  rust: add device::Data
  rust: add `dev_*` print macros.

 rust/bindings/bindings_helper.h |   1 +
 rust/helpers.c                  | 145 ++++++++++
 rust/kernel/device.rs           | 498 ++++++++++++++++++++++++++++++++
 rust/kernel/devres.rs           | 151 ++++++++++
 rust/kernel/driver.rs           | 492 +++++++++++++++++++++++++++++++
 rust/kernel/iomem.rs            | 135 +++++++++
 rust/kernel/lib.rs              |  10 +-
 rust/kernel/pci.rs              | 449 ++++++++++++++++++++++++++++
 rust/kernel/prelude.rs          |   2 +
 rust/kernel/revocable.rs        | 441 ++++++++++++++++++++++++++++
 rust/kernel/sync.rs             |   3 +
 rust/kernel/sync/rcu.rs         |  52 ++++
 rust/kernel/sync/revocable.rs   | 148 ++++++++++
 rust/macros/module.rs           |   2 +-
 samples/rust/rust_minimal.rs    |   2 +-
 samples/rust/rust_print.rs      |   2 +-
 16 files changed, 2529 insertions(+), 4 deletions(-)
 create mode 100644 rust/kernel/device.rs
 create mode 100644 rust/kernel/devres.rs
 create mode 100644 rust/kernel/driver.rs
 create mode 100644 rust/kernel/iomem.rs
 create mode 100644 rust/kernel/pci.rs
 create mode 100644 rust/kernel/revocable.rs
 create mode 100644 rust/kernel/sync/rcu.rs
 create mode 100644 rust/kernel/sync/revocable.rs


base-commit: 97ab3e8eec0ce79d9e265e6c9e4c480492180409
-- 
2.45.1

[RFC PATCH 01/11] rust: add abstraction for struct device

[Danilo Krummrich]

Add an (always) reference counted abstraction for a generic struct
device. This abstraction encapsulates existing struct device instances
and manages its reference count.

Subsystems may use this abstraction as a base to abstract subsystem
specific device instances based on a generic struct device.

Co-developed-by: Wedson Almeida Filho <wedsonaf@gmail.com>
Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/helpers.c        |  1 +
 rust/kernel/device.rs | 76 +++++++++++++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs    |  1 +
 3 files changed, 78 insertions(+)
 create mode 100644 rust/kernel/device.rs

diff --git a/rust/helpers.c b/rust/helpers.c
index 4c8b7b92a4f4..f9d2db1d1f33 100644
--- a/rust/helpers.c
+++ b/rust/helpers.c
@@ -23,6 +23,7 @@
 #include <kunit/test-bug.h>
 #include <linux/bug.h>
 #include <linux/build_bug.h>
+#include <linux/device.h>
 #include <linux/err.h>
 #include <linux/errname.h>
 #include <linux/mutex.h>
diff --git a/rust/kernel/device.rs b/rust/kernel/device.rs
new file mode 100644
index 000000000000..fafec70effb6
--- /dev/null
+++ b/rust/kernel/device.rs
@@ -0,0 +1,76 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Generic devices that are part of the kernel's driver model.
+//!
+//! C header: [`include/linux/device.h`](../../../../include/linux/device.h)
+
+use crate::{
+    bindings,
+    types::{ARef, Opaque},
+};
+use core::ptr;
+
+/// A ref-counted device.
+///
+/// # Invariants
+///
+/// The pointer stored in `Self` is non-null and valid for the lifetime of the ARef instance. In
+/// particular, the ARef instance owns an increment on underlying object’s reference count.
+#[repr(transparent)]
+pub struct Device(Opaque<bindings::device>);
+
+impl Device {
+    /// Creates a new ref-counted instance of an existing device pointer.
+    ///
+    /// # Safety
+    ///
+    /// Callers must ensure that `ptr` is valid, non-null, and has a non-zero reference count.
+    pub unsafe fn from_raw(ptr: *mut bindings::device) -> ARef<Self> {
+        // SAFETY: By the safety requirements, ptr is valid.
+        // Initially increase the reference count by one to compensate for the final decrement once
+        // this newly created `ARef<Device>` instance is dropped.
+        unsafe { bindings::get_device(ptr) };
+
+        // CAST: `Self` is a `repr(transparent)` wrapper around `bindings::device`.
+        let ptr = ptr.cast::<Self>();
+
+        // SAFETY: By the safety requirements, ptr is valid.
+        unsafe { ARef::from_raw(ptr::NonNull::new_unchecked(ptr)) }
+    }
+
+    /// Obtain the raw `struct device *`.
+    pub(crate) fn as_raw(&self) -> *mut bindings::device {
+        self.0.get()
+    }
+
+    /// Convert a raw `struct device` pointer to a `&Device`.
+    ///
+    /// # Safety
+    ///
+    /// Callers must ensure that `ptr` is valid, non-null, and has a non-zero reference count for
+    /// the entire duration when the returned reference exists.
+    pub unsafe fn as_ref<'a>(ptr: *mut bindings::device) -> &'a Self {
+        // SAFETY: Guaranteed by the safety requirements of the function.
+        unsafe { &*ptr.cast() }
+    }
+}
+
+// SAFETY: Instances of `Device` are always ref-counted.
+unsafe impl crate::types::AlwaysRefCounted for Device {
+    fn inc_ref(&self) {
+        // SAFETY: The existence of a shared reference guarantees that the refcount is nonzero.
+        unsafe { bindings::get_device(self.as_raw()) };
+    }
+
+    unsafe fn dec_ref(obj: ptr::NonNull<Self>) {
+        // SAFETY: The safety requirements guarantee that the refcount is nonzero.
+        unsafe { bindings::put_device(obj.cast().as_ptr()) }
+    }
+}
+
+// SAFETY: `Device` only holds a pointer to a C device, which is safe to be used from any thread.
+unsafe impl Send for Device {}
+
+// SAFETY: `Device` only holds a pointer to a C device, references to which are safe to be used
+// from any thread.
+unsafe impl Sync for Device {}
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index 9a943d99c71a..4ba3d4a49e9c 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -28,6 +28,7 @@
 
 pub mod alloc;
 mod build_assert;
+pub mod device;
 pub mod error;
 pub mod init;
 pub mod ioctl;
-- 
2.45.1

[RFC PATCH 02/11] rust: add driver abstraction

[Danilo Krummrich]

From: Wedson Almeida Filho <wedsonaf@gmail.com>

This defines general functionality related to registering drivers with
their respective subsystems, and registering modules that implement
drivers.

Co-developed-by: Asahi Lina <lina@asahilina.net>
Signed-off-by: Asahi Lina <lina@asahilina.net>
Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs           |   4 +-
 rust/macros/module.rs        |   2 +-
 samples/rust/rust_minimal.rs |   2 +-
 samples/rust/rust_print.rs   |   2 +-
 5 files changed, 498 insertions(+), 4 deletions(-)
 create mode 100644 rust/kernel/driver.rs

diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
new file mode 100644
index 000000000000..e0cfc36d47ff
--- /dev/null
+++ b/rust/kernel/driver.rs
@@ -0,0 +1,492 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
+//!
+//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
+//! using the [`Registration`] class.
+
+use crate::{
+    alloc::{box_ext::BoxExt, flags::*},
+    error::code::*,
+    error::Result,
+    str::CStr,
+    sync::Arc,
+    ThisModule,
+};
+use alloc::boxed::Box;
+use core::{cell::UnsafeCell, marker::PhantomData, ops::Deref, pin::Pin};
+
+/// A subsystem (e.g., PCI, Platform, Amba, etc.) that allows drivers to be written for it.
+pub trait DriverOps {
+    /// The type that holds information about the registration. This is typically a struct defined
+    /// by the C portion of the kernel.
+    type RegType: Default;
+
+    /// Registers a driver.
+    ///
+    /// # Safety
+    ///
+    /// `reg` must point to valid, initialised, and writable memory. It may be modified by this
+    /// function to hold registration state.
+    ///
+    /// On success, `reg` must remain pinned and valid until the matching call to
+    /// [`DriverOps::unregister`].
+    unsafe fn register(
+        reg: *mut Self::RegType,
+        name: &'static CStr,
+        module: &'static ThisModule,
+    ) -> Result;
+
+    /// Unregisters a driver previously registered with [`DriverOps::register`].
+    ///
+    /// # Safety
+    ///
+    /// `reg` must point to valid writable memory, initialised by a previous successful call to
+    /// [`DriverOps::register`].
+    unsafe fn unregister(reg: *mut Self::RegType);
+}
+
+/// The registration of a driver.
+pub struct Registration<T: DriverOps> {
+    is_registered: bool,
+    concrete_reg: UnsafeCell<T::RegType>,
+}
+
+// SAFETY: `Registration` has no fields or methods accessible via `&Registration`, so it is safe to
+// share references to it with multiple threads as nothing can be done.
+unsafe impl<T: DriverOps> Sync for Registration<T> {}
+
+impl<T: DriverOps> Registration<T> {
+    /// Creates a new instance of the registration object.
+    pub fn new() -> Self {
+        Self {
+            is_registered: false,
+            concrete_reg: UnsafeCell::new(T::RegType::default()),
+        }
+    }
+
+    /// Allocates a pinned registration object and registers it.
+    ///
+    /// Returns a pinned heap-allocated representation of the registration.
+    pub fn new_pinned(name: &'static CStr, module: &'static ThisModule) -> Result<Pin<Box<Self>>> {
+        let mut reg = Pin::from(Box::new(Self::new(), GFP_KERNEL)?);
+        reg.as_mut().register(name, module)?;
+        Ok(reg)
+    }
+
+    /// Registers a driver with its subsystem.
+    ///
+    /// It must be pinned because the memory block that represents the registration is potentially
+    /// self-referential.
+    pub fn register(
+        self: Pin<&mut Self>,
+        name: &'static CStr,
+        module: &'static ThisModule,
+    ) -> Result {
+        // SAFETY: We never move out of `this`.
+        let this = unsafe { self.get_unchecked_mut() };
+        if this.is_registered {
+            // Already registered.
+            return Err(EINVAL);
+        }
+
+        // SAFETY: `concrete_reg` was initialised via its default constructor. It is only freed
+        // after `Self::drop` is called, which first calls `T::unregister`.
+        unsafe { T::register(this.concrete_reg.get(), name, module) }?;
+
+        this.is_registered = true;
+        Ok(())
+    }
+}
+
+impl<T: DriverOps> Default for Registration<T> {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl<T: DriverOps> Drop for Registration<T> {
+    fn drop(&mut self) {
+        if self.is_registered {
+            // SAFETY: This path only runs if a previous call to `T::register` completed
+            // successfully.
+            unsafe { T::unregister(self.concrete_reg.get()) };
+        }
+    }
+}
+
+/// Conversion from a device id to a raw device id.
+///
+/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
+/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
+///
+/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
+/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
+/// concrete types (which can still have const associated functions) instead of a trait.
+///
+/// # Safety
+///
+/// Implementers must ensure that:
+///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
+///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
+///     that buses can recover the pointer to the data.
+pub unsafe trait RawDeviceId {
+    /// The raw type that holds the device id.
+    ///
+    /// Id tables created from [`Self`] are going to hold this type in its zero-terminated array.
+    type RawType: Copy;
+
+    /// A zeroed-out representation of the raw device id.
+    ///
+    /// Id tables created from [`Self`] use [`Self::ZERO`] as the sentinel to indicate the end of
+    /// the table.
+    const ZERO: Self::RawType;
+}
+
+/// A zero-terminated device id array, followed by context data.
+#[repr(C)]
+pub struct IdArray<T: RawDeviceId, U, const N: usize> {
+    ids: [T::RawType; N],
+    sentinel: T::RawType,
+    id_infos: [Option<U>; N],
+}
+
+impl<T: RawDeviceId, U, const N: usize> IdArray<T, U, N> {
+    const U_NONE: Option<U> = None;
+
+    /// Returns an `IdTable` backed by `self`.
+    ///
+    /// This is used to essentially erase the array size.
+    pub const fn as_table(&self) -> IdTable<'_, T, U> {
+        IdTable {
+            first: &self.ids[0],
+            _p: PhantomData,
+        }
+    }
+
+    /// Creates a new instance of the array.
+    ///
+    /// The contents are derived from the given identifiers and context information.
+    #[doc(hidden)]
+    pub const unsafe fn new(raw_ids: [T::RawType; N], infos: [Option<U>; N]) -> Self
+    where
+        T: RawDeviceId + Copy,
+        T::RawType: Copy + Clone,
+    {
+        Self {
+            ids: raw_ids,
+            sentinel: T::ZERO,
+            id_infos: infos,
+        }
+    }
+
+    #[doc(hidden)]
+    pub const fn get_offset(idx: usize) -> isize
+    where
+        T: RawDeviceId + Copy,
+        T::RawType: Copy + Clone,
+    {
+        // SAFETY: We are only using this dummy value to get offsets.
+        let array = unsafe { Self::new([T::ZERO; N], [Self::U_NONE; N]) };
+        // SAFETY: Both pointers are within `array` (or one byte beyond), consequently they are
+        // derived from the same allocated object. We are using a `u8` pointer, whose size 1,
+        // so the pointers are necessarily 1-byte aligned.
+        let ret = unsafe {
+            (&array.id_infos[idx] as *const _ as *const u8)
+                .offset_from(&array.ids[idx] as *const _ as _)
+        };
+        core::mem::forget(array);
+        ret
+    }
+}
+
+// Creates a new ID array. This is a macro so it can take as a parameter the concrete ID type in
+// order to call to_rawid() on it, and still remain const. This is necessary until a new
+// const_trait_impl implementation lands, since the existing implementation was removed in Rust
+// 1.73.
+#[macro_export]
+#[doc(hidden)]
+macro_rules! _new_id_array {
+    (($($args:tt)*), $id_type:ty) => {{
+        /// Creates a new instance of the array.
+        ///
+        /// The contents are derived from the given identifiers and context information.
+        const fn new< U, const N: usize>(ids: [$id_type; N], infos: [Option<U>; N])
+            -> $crate::driver::IdArray<$id_type, U, N>
+        where
+            $id_type: $crate::driver::RawDeviceId + Copy,
+            <$id_type as $crate::driver::RawDeviceId>::RawType: Copy + Clone,
+        {
+            let mut raw_ids =
+                [<$id_type as $crate::driver::RawDeviceId>::ZERO; N];
+            let mut i = 0usize;
+            while i < N {
+                let offset: isize = $crate::driver::IdArray::<$id_type, U, N>::get_offset(i);
+                raw_ids[i] = ids[i].to_rawid(offset);
+                i += 1;
+            }
+
+            // SAFETY: We are passing valid arguments computed with the correct offsets.
+            unsafe {
+                $crate::driver::IdArray::<$id_type, U, N>::new(raw_ids, infos)
+            }
+       }
+
+        new($($args)*)
+    }}
+}
+
+/// A device id table.
+///
+/// The table is guaranteed to be zero-terminated and to be followed by an array of context data of
+/// type `Option<U>`.
+pub struct IdTable<'a, T: RawDeviceId, U> {
+    first: &'a T::RawType,
+    _p: PhantomData<&'a U>,
+}
+
+impl<T: RawDeviceId, U> AsRef<T::RawType> for IdTable<'_, T, U> {
+    fn as_ref(&self) -> &T::RawType {
+        self.first
+    }
+}
+
+/// Counts the number of parenthesis-delimited, comma-separated items.
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::count_paren_items;
+///
+/// assert_eq!(0, count_paren_items!());
+/// assert_eq!(1, count_paren_items!((A)));
+/// assert_eq!(1, count_paren_items!((A),));
+/// assert_eq!(2, count_paren_items!((A), (B)));
+/// assert_eq!(2, count_paren_items!((A), (B),));
+/// assert_eq!(3, count_paren_items!((A), (B), (C)));
+/// assert_eq!(3, count_paren_items!((A), (B), (C),));
+/// ```
+#[macro_export]
+macro_rules! count_paren_items {
+    (($($item:tt)*), $($remaining:tt)*) => { 1 + $crate::count_paren_items!($($remaining)*) };
+    (($($item:tt)*)) => { 1 };
+    () => { 0 };
+}
+
+/// Converts a comma-separated list of pairs into an array with the first element. That is, it
+/// discards the second element of the pair.
+///
+/// Additionally, it automatically introduces a type if the first element is warpped in curly
+/// braces, for example, if it's `{v: 10}`, it becomes `X { v: 10 }`; this is to avoid repeating
+/// the type.
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::first_item;
+///
+/// #[derive(PartialEq, Debug)]
+/// struct X {
+///     v: u32,
+/// }
+///
+/// assert_eq!([] as [X; 0], first_item!(X, ));
+/// assert_eq!([X { v: 10 }], first_item!(X, ({ v: 10 }, Y)));
+/// assert_eq!([X { v: 10 }], first_item!(X, ({ v: 10 }, Y),));
+/// assert_eq!([X { v: 10 }], first_item!(X, (X { v: 10 }, Y)));
+/// assert_eq!([X { v: 10 }], first_item!(X, (X { v: 10 }, Y),));
+/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y)));
+/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y),));
+/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y)));
+/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y),));
+/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
+///            first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y), ({v: 30}, Y)));
+/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
+///            first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y), ({v: 30}, Y),));
+/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
+///            first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y), (X {v: 30}, Y)));
+/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
+///            first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y), (X {v: 30}, Y),));
+/// ```
+#[macro_export]
+macro_rules! first_item {
+    ($id_type:ty, $(({$($first:tt)*}, $second:expr)),* $(,)?) => {
+        {
+            type IdType = $id_type;
+            [$(IdType{$($first)*},)*]
+        }
+    };
+    ($id_type:ty, $(($first:expr, $second:expr)),* $(,)?) => { [$($first,)*] };
+}
+
+/// Converts a comma-separated list of pairs into an array with the second element. That is, it
+/// discards the first element of the pair.
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::second_item;
+///
+/// assert_eq!([] as [u32; 0], second_item!());
+/// assert_eq!([10u32], second_item!((X, 10u32)));
+/// assert_eq!([10u32], second_item!((X, 10u32),));
+/// assert_eq!([10u32], second_item!(({ X }, 10u32)));
+/// assert_eq!([10u32], second_item!(({ X }, 10u32),));
+/// assert_eq!([10u32, 20], second_item!((X, 10u32), (X, 20)));
+/// assert_eq!([10u32, 20], second_item!((X, 10u32), (X, 20),));
+/// assert_eq!([10u32, 20], second_item!(({ X }, 10u32), ({ X }, 20)));
+/// assert_eq!([10u32, 20], second_item!(({ X }, 10u32), ({ X }, 20),));
+/// assert_eq!([10u32, 20, 30], second_item!((X, 10u32), (X, 20), (X, 30)));
+/// assert_eq!([10u32, 20, 30], second_item!((X, 10u32), (X, 20), (X, 30),));
+/// assert_eq!([10u32, 20, 30], second_item!(({ X }, 10u32), ({ X }, 20), ({ X }, 30)));
+/// assert_eq!([10u32, 20, 30], second_item!(({ X }, 10u32), ({ X }, 20), ({ X }, 30),));
+/// ```
+#[macro_export]
+macro_rules! second_item {
+    ($(({$($first:tt)*}, $second:expr)),* $(,)?) => { [$($second,)*] };
+    ($(($first:expr, $second:expr)),* $(,)?) => { [$($second,)*] };
+}
+
+/// Defines a new constant [`IdArray`] with a concise syntax.
+///
+/// It is meant to be used by buses and subsystems to create a similar macro with their device id
+/// type already specified, i.e., with fewer parameters to the end user.
+///
+/// # Examples
+///
+// TODO: Exported but not usable by kernel modules (requires `const_trait_impl`).
+/// ```ignore
+/// #![feature(const_trait_impl)]
+/// # use kernel::{define_id_array, driver::RawDeviceId};
+///
+/// #[derive(Copy, Clone)]
+/// struct Id(u32);
+///
+/// // SAFETY: `ZERO` is all zeroes and `to_rawid` stores `offset` as the second element of the raw
+/// // device id pair.
+/// unsafe impl const RawDeviceId for Id {
+///     type RawType = (u64, isize);
+///     const ZERO: Self::RawType = (0, 0);
+///     fn to_rawid(&self, offset: isize) -> Self::RawType {
+///         (self.0 as u64 + 1, offset)
+///     }
+/// }
+///
+/// define_id_array!(A1, Id, (), []);
+/// define_id_array!(A2, Id, &'static [u8], [(Id(10), None)]);
+/// define_id_array!(A3, Id, &'static [u8], [(Id(10), Some(b"id1")), ]);
+/// define_id_array!(A4, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2"))]);
+/// define_id_array!(A5, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2")), ]);
+/// define_id_array!(A6, Id, &'static [u8], [(Id(10), None), (Id(20), Some(b"id2")), ]);
+/// define_id_array!(A7, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), None), ]);
+/// define_id_array!(A8, Id, &'static [u8], [(Id(10), None), (Id(20), None), ]);
+/// ```
+#[macro_export]
+macro_rules! define_id_array {
+    ($table_name:ident, $id_type:ty, $data_type:ty, [ $($t:tt)* ]) => {
+        const $table_name:
+            $crate::driver::IdArray<$id_type, $data_type, { $crate::count_paren_items!($($t)*) }> =
+                $crate::_new_id_array!((
+                    $crate::first_item!($id_type, $($t)*), $crate::second_item!($($t)*)), $id_type);
+    };
+}
+
+/// Defines a new constant [`IdTable`] with a concise syntax.
+///
+/// It is meant to be used by buses and subsystems to create a similar macro with their device id
+/// type already specified, i.e., with fewer parameters to the end user.
+///
+/// # Examples
+///
+// TODO: Exported but not usable by kernel modules (requires `const_trait_impl`).
+/// ```ignore
+/// #![feature(const_trait_impl)]
+/// # use kernel::{define_id_table, driver::RawDeviceId};
+///
+/// #[derive(Copy, Clone)]
+/// struct Id(u32);
+///
+/// // SAFETY: `ZERO` is all zeroes and `to_rawid` stores `offset` as the second element of the raw
+/// // device id pair.
+/// unsafe impl const RawDeviceId for Id {
+///     type RawType = (u64, isize);
+///     const ZERO: Self::RawType = (0, 0);
+///     fn to_rawid(&self, offset: isize) -> Self::RawType {
+///         (self.0 as u64 + 1, offset)
+///     }
+/// }
+///
+/// define_id_table!(T1, Id, &'static [u8], [(Id(10), None)]);
+/// define_id_table!(T2, Id, &'static [u8], [(Id(10), Some(b"id1")), ]);
+/// define_id_table!(T3, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2"))]);
+/// define_id_table!(T4, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2")), ]);
+/// define_id_table!(T5, Id, &'static [u8], [(Id(10), None), (Id(20), Some(b"id2")), ]);
+/// define_id_table!(T6, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), None), ]);
+/// define_id_table!(T7, Id, &'static [u8], [(Id(10), None), (Id(20), None), ]);
+/// ```
+#[macro_export]
+macro_rules! define_id_table {
+    ($table_name:ident, $id_type:ty, $data_type:ty, [ $($t:tt)* ]) => {
+        const $table_name: Option<$crate::driver::IdTable<'static, $id_type, $data_type>> = {
+            $crate::define_id_array!(ARRAY, $id_type, $data_type, [ $($t)* ]);
+            Some(ARRAY.as_table())
+        };
+    };
+}
+
+/// Custom code within device removal.
+pub trait DeviceRemoval {
+    /// Cleans resources up when the device is removed.
+    ///
+    /// This is called when a device is removed and offers implementers the chance to run some code
+    /// that cleans state up.
+    fn device_remove(&self);
+}
+
+impl DeviceRemoval for () {
+    fn device_remove(&self) {}
+}
+
+impl<T: DeviceRemoval> DeviceRemoval for Arc<T> {
+    fn device_remove(&self) {
+        self.deref().device_remove();
+    }
+}
+
+impl<T: DeviceRemoval> DeviceRemoval for Box<T> {
+    fn device_remove(&self) {
+        self.deref().device_remove();
+    }
+}
+
+/// A kernel module that only registers the given driver on init.
+///
+/// This is a helper struct to make it easier to define single-functionality modules, in this case,
+/// modules that offer a single driver.
+pub struct Module<T: DriverOps> {
+    _driver: Pin<Box<Registration<T>>>,
+}
+
+impl<T: DriverOps> crate::Module for Module<T> {
+    fn init(name: &'static CStr, module: &'static ThisModule) -> Result<Self> {
+        Ok(Self {
+            _driver: Registration::new_pinned(name, module)?,
+        })
+    }
+}
+
+/// Declares a kernel module that exposes a single driver.
+///
+/// It is meant to be used as a helper by other subsystems so they can more easily expose their own
+/// macros.
+#[macro_export]
+macro_rules! module_driver {
+    (<$gen_type:ident>, $driver_ops:ty, { type: $type:ty, $($f:tt)* }) => {
+        type Ops<$gen_type> = $driver_ops;
+        type ModuleType = $crate::driver::Module<Ops<$type>>;
+        $crate::prelude::module! {
+            type: ModuleType,
+            $($f)*
+        }
+    }
+}
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index 4ba3d4a49e9c..698121c925f3 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -13,6 +13,7 @@
 
 #![no_std]
 #![feature(coerce_unsized)]
+#![feature(const_refs_to_cell)]
 #![feature(dispatch_from_dyn)]
 #![feature(new_uninit)]
 #![feature(receiver_trait)]
@@ -29,6 +30,7 @@
 pub mod alloc;
 mod build_assert;
 pub mod device;
+pub mod driver;
 pub mod error;
 pub mod init;
 pub mod ioctl;
@@ -69,7 +71,7 @@ pub trait Module: Sized + Sync {
     /// should do.
     ///
     /// Equivalent to the `module_init` macro in the C API.
-    fn init(module: &'static ThisModule) -> error::Result<Self>;
+    fn init(name: &'static str::CStr, module: &'static ThisModule) -> error::Result<Self>;
 }
 
 /// Equivalent to `THIS_MODULE` in the C API.
diff --git a/rust/macros/module.rs b/rust/macros/module.rs
index 27979e582e4b..3e7a6a8560f5 100644
--- a/rust/macros/module.rs
+++ b/rust/macros/module.rs
@@ -275,7 +275,7 @@ pub(crate) fn module(ts: TokenStream) -> TokenStream {
             }}
 
             fn __init() -> core::ffi::c_int {{
-                match <{type_} as kernel::Module>::init(&THIS_MODULE) {{
+                match <{type_} as kernel::Module>::init(kernel::c_str!(\"{name}\"), &THIS_MODULE) {{
                     Ok(m) => {{
                         unsafe {{
                             __MOD = Some(m);
diff --git a/samples/rust/rust_minimal.rs b/samples/rust/rust_minimal.rs
index 2a9eaab62d1c..3b918ff5eebb 100644
--- a/samples/rust/rust_minimal.rs
+++ b/samples/rust/rust_minimal.rs
@@ -17,7 +17,7 @@ struct RustMinimal {
 }
 
 impl kernel::Module for RustMinimal {
-    fn init(_module: &'static ThisModule) -> Result<Self> {
+    fn init(_name: &'static CStr, _module: &'static ThisModule) -> Result<Self> {
         pr_info!("Rust minimal sample (init)\n");
         pr_info!("Am I built-in? {}\n", !cfg!(MODULE));
 
diff --git a/samples/rust/rust_print.rs b/samples/rust/rust_print.rs
index 6eabb0d79ea3..722275a735f1 100644
--- a/samples/rust/rust_print.rs
+++ b/samples/rust/rust_print.rs
@@ -40,7 +40,7 @@ fn arc_print() -> Result {
 }
 
 impl kernel::Module for RustPrint {
-    fn init(_module: &'static ThisModule) -> Result<Self> {
+    fn init(_name: &'static CStr, _module: &'static ThisModule) -> Result<Self> {
         pr_info!("Rust printing macros sample (init)\n");
 
         pr_emerg!("Emergency message (level 0) without args\n");
-- 
2.45.1

[RFC PATCH 03/11] rust: add rcu abstraction

[Danilo Krummrich]

From: Wedson Almeida Filho <wedsonaf@gmail.com>

Add a simple abstraction to guard critical code sections with an rcu
read lock.

Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/helpers.c          | 15 ++++++++++++
 rust/kernel/sync.rs     |  1 +
 rust/kernel/sync/rcu.rs | 52 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 68 insertions(+)
 create mode 100644 rust/kernel/sync/rcu.rs

diff --git a/rust/helpers.c b/rust/helpers.c
index f9d2db1d1f33..1d3e800140fc 100644
--- a/rust/helpers.c
+++ b/rust/helpers.c
@@ -27,6 +27,7 @@
 #include <linux/err.h>
 #include <linux/errname.h>
 #include <linux/mutex.h>
+#include <linux/rcupdate.h>
 #include <linux/refcount.h>
 #include <linux/sched/signal.h>
 #include <linux/spinlock.h>
@@ -158,6 +159,20 @@ void rust_helper_init_work_with_key(struct work_struct *work, work_func_t func,
 }
 EXPORT_SYMBOL_GPL(rust_helper_init_work_with_key);
 
+/* rcu */
+void rust_helper_rcu_read_lock(void)
+{
+	rcu_read_lock();
+}
+EXPORT_SYMBOL_GPL(rust_helper_rcu_read_lock);
+
+void rust_helper_rcu_read_unlock(void)
+{
+	rcu_read_unlock();
+}
+EXPORT_SYMBOL_GPL(rust_helper_rcu_read_unlock);
+/* end rcu */
+
 /*
  * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can
  * use it in contexts where Rust expects a `usize` like slice (array) indices.
diff --git a/rust/kernel/sync.rs b/rust/kernel/sync.rs
index 0ab20975a3b5..1806767359fe 100644
--- a/rust/kernel/sync.rs
+++ b/rust/kernel/sync.rs
@@ -11,6 +11,7 @@
 mod condvar;
 pub mod lock;
 mod locked_by;
+pub mod rcu;
 
 pub use arc::{Arc, ArcBorrow, UniqueArc};
 pub use condvar::{new_condvar, CondVar, CondVarTimeoutResult};
diff --git a/rust/kernel/sync/rcu.rs b/rust/kernel/sync/rcu.rs
new file mode 100644
index 000000000000..1a1c8ea49359
--- /dev/null
+++ b/rust/kernel/sync/rcu.rs
@@ -0,0 +1,52 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! RCU support.
+//!
+//! C header: [`include/linux/rcupdate.h`](../../../../include/linux/rcupdate.h)
+
+use crate::bindings;
+use core::marker::PhantomData;
+
+/// Evidence that the RCU read side lock is held on the current thread/CPU.
+///
+/// The type is explicitly not `Send` because this property is per-thread/CPU.
+///
+/// # Invariants
+///
+/// The RCU read side lock is actually held while instances of this guard exist.
+pub struct Guard {
+    _not_send: PhantomData<*mut ()>,
+}
+
+impl Guard {
+    /// Acquires the RCU read side lock and returns a guard.
+    pub fn new() -> Self {
+        // SAFETY: An FFI call with no additional requirements.
+        unsafe { bindings::rcu_read_lock() };
+        // INVARIANT: The RCU read side lock was just acquired above.
+        Self {
+            _not_send: PhantomData,
+        }
+    }
+
+    /// Explicitly releases the RCU read side lock.
+    pub fn unlock(self) {}
+}
+
+impl Default for Guard {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl Drop for Guard {
+    fn drop(&mut self) {
+        // SAFETY: By the type invariants, the rcu read side is locked, so it is ok to unlock it.
+        unsafe { bindings::rcu_read_unlock() };
+    }
+}
+
+/// Acquires the RCU read side lock.
+pub fn read_lock() -> Guard {
+    Guard::new()
+}
-- 
2.45.1

[RFC PATCH 04/11] rust: add revocable mutex

[Danilo Krummrich]

From: Wedson Almeida Filho <wedsonaf@gmail.com>

This is a mutex where access to its contents can be revoked at runtime.

This is different from `Revocable` in a few ways:
  1. The caller may sleep while holding a guard.
  2. Accessors are all serialised.
  3. Accessing is not as cheap (because it involves acquiring the mutex).
  4. The size of the object is larger (because it involves a mutex +
     flag).

An example of where this a good fit to be used in device state that
holds registrations to other subsystems.

Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
Co-developed-by: Danilo Krummrich <dakr@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/kernel/sync.rs           |   2 +
 rust/kernel/sync/revocable.rs | 148 ++++++++++++++++++++++++++++++++++
 2 files changed, 150 insertions(+)
 create mode 100644 rust/kernel/sync/revocable.rs

diff --git a/rust/kernel/sync.rs b/rust/kernel/sync.rs
index 1806767359fe..13257a4bcff6 100644
--- a/rust/kernel/sync.rs
+++ b/rust/kernel/sync.rs
@@ -12,12 +12,14 @@
 pub mod lock;
 mod locked_by;
 pub mod rcu;
+mod revocable;
 
 pub use arc::{Arc, ArcBorrow, UniqueArc};
 pub use condvar::{new_condvar, CondVar, CondVarTimeoutResult};
 pub use lock::mutex::{new_mutex, Mutex};
 pub use lock::spinlock::{new_spinlock, SpinLock};
 pub use locked_by::LockedBy;
+pub use revocable::{RevocableMutex, RevocableMutexGuard};
 
 /// Represents a lockdep class. It's a wrapper around C's `lock_class_key`.
 #[repr(transparent)]
diff --git a/rust/kernel/sync/revocable.rs b/rust/kernel/sync/revocable.rs
new file mode 100644
index 000000000000..4632bb838180
--- /dev/null
+++ b/rust/kernel/sync/revocable.rs
@@ -0,0 +1,148 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Synchronisation primitives where access to their contents can be revoked at runtime.
+
+use macros::pin_data;
+
+use crate::{
+    init::PinInit,
+    pin_init,
+    str::CStr,
+    sync::{lock, lock::Lock, LockClassKey},
+};
+use core::{
+    mem::MaybeUninit,
+    ops::{Deref, DerefMut},
+    pin::Pin,
+};
+
+use super::lock::Guard;
+
+/// The state within the revocable synchronisation primitive.
+///
+/// We don't use simply `Option<T>` because we need to drop in-place because the contents are
+/// implicitly pinned.
+///
+/// # Invariants
+///
+/// The `is_available` field determines if `data` is initialised.
+struct Inner<T> {
+    is_available: bool,
+    data: MaybeUninit<T>,
+}
+
+impl<T> Inner<T> {
+    fn new(data: T) -> Self {
+        // INVARIANT: `data` is initialised and `is_available` is `true`, so the state matches.
+        Self {
+            is_available: true,
+            data: MaybeUninit::new(data),
+        }
+    }
+
+    fn drop_in_place(&mut self) {
+        if !self.is_available {
+            // Already dropped.
+            return;
+        }
+
+        // INVARIANT: `data` is being dropped and `is_available` is set to `false`, so the state
+        // matches.
+        self.is_available = false;
+
+        // SAFETY: By the type invariants, `data` is valid because `is_available` was true.
+        unsafe { self.data.assume_init_drop() };
+    }
+}
+
+impl<T> Drop for Inner<T> {
+    fn drop(&mut self) {
+        self.drop_in_place();
+    }
+}
+
+#[pin_data]
+pub struct Revocable<T, B: lock::Backend> {
+    #[pin]
+    inner: Lock<Inner<T>, B>,
+}
+
+/// Safely initialises a [`Revocable`] instance with the given name, generating a new lock class.
+// #[macro_export]
+// macro_rules! revocable_init {
+//     ($mutex:expr, $name:literal) => {
+//         $crate::init_with_lockdep!($mutex, $name)
+//     };
+// }
+
+impl<T, B> Revocable<T, B>
+where
+    B: lock::Backend,
+{
+    /// Creates a new revocable instance of the given lock.
+    pub fn new(data: T, name: &'static CStr, key: &'static LockClassKey) -> impl PinInit<Self> {
+        pin_init!(Self {
+            inner <- Lock::new(Inner::new(data), name, key) ,
+        })
+    }
+
+    /// Revokes access to and drops the wrapped object.
+    ///
+    /// Revocation and dropping happen after ongoing accessors complete.
+    pub fn revoke(&self) {
+        self.lock().drop_in_place();
+    }
+
+    pub fn try_write(&self) -> Option<RevocableGuard<'_, T, B>> {
+        let inner = self.lock();
+
+        if !inner.is_available {
+            return None;
+        }
+
+        Some(RevocableGuard::new(inner))
+    }
+
+    fn lock(&self) -> Guard<'_, Inner<T>, B> {
+        self.inner.lock()
+    }
+}
+
+pub struct RevocableGuard<'a, T, B>
+where
+    B: lock::Backend,
+{
+    guard: Guard<'a, Inner<T>, B>,
+}
+
+impl<'a, T, B: lock::Backend> RevocableGuard<'a, T, B> {
+    fn new(guard: Guard<'a, Inner<T>, B>) -> Self {
+        Self { guard }
+    }
+}
+
+impl<T, B: lock::Backend> RevocableGuard<'_, T, B> {
+    pub fn as_pinned_mut(&mut self) -> Pin<&mut T> {
+        unsafe { Pin::new_unchecked(&mut *self) }
+    }
+}
+
+impl<T, B: lock::Backend> Deref for RevocableGuard<'_, T, B> {
+    type Target = T;
+
+    fn deref(&self) -> &Self::Target {
+        unsafe { &*self.guard.data.as_ptr() }
+    }
+}
+
+impl<T, B: lock::Backend> DerefMut for RevocableGuard<'_, T, B> {
+    fn deref_mut(&mut self) -> &mut Self::Target {
+        unsafe { &mut *self.guard.data.as_mut_ptr() }
+    }
+}
+
+/// Type alias for a `Revocable` with a `MutexBackend`.
+pub type RevocableMutex<T> = Revocable<T, super::lock::mutex::MutexBackend>;
+
+/// Type alias for a `RevocableGuard` with a `MutexBackend`.
+pub type RevocableMutexGuard<'a, T> = RevocableGuard<'a, T, super::lock::mutex::MutexBackend>;
-- 
2.45.1

[RFC PATCH 05/11] rust: add revocable objects

[Danilo Krummrich]

From: Wedson Almeida Filho <wedsonaf@gmail.com>

This implements the Revocable and AsyncRevocable types.

Revocable allows access to objects to be safely revoked at run time.

This is useful, for example, for resources allocated during device probe;
when the device is removed, the driver should stop accessing the device
resources even if other state is kept in memory due to existing
references (i.e., device context data is ref-counted and has a non-zero
refcount after removal of the device).

AsyncRevocable allows access to objects to be revoked without having to
wait for existing users to complete. This will be used to drop futures
in tasks when executors are being torn down.

Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/kernel/lib.rs       |   1 +
 rust/kernel/revocable.rs | 441 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 442 insertions(+)
 create mode 100644 rust/kernel/revocable.rs

diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index 698121c925f3..d7d415429517 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -40,6 +40,7 @@
 pub mod net;
 pub mod prelude;
 pub mod print;
+pub mod revocable;
 mod static_assert;
 #[doc(hidden)]
 pub mod std_vendor;
diff --git a/rust/kernel/revocable.rs b/rust/kernel/revocable.rs
new file mode 100644
index 000000000000..71408039a117
--- /dev/null
+++ b/rust/kernel/revocable.rs
@@ -0,0 +1,441 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Revocable objects.
+//!
+//! The [`Revocable`] type wraps other types and allows access to them to be revoked. The existence
+//! of a [`RevocableGuard`] ensures that objects remain valid.
+
+use crate::{
+    bindings,
+    init::{self},
+    prelude::*,
+    sync::rcu,
+};
+use core::{
+    cell::UnsafeCell,
+    marker::PhantomData,
+    mem::MaybeUninit,
+    ops::Deref,
+    ptr::drop_in_place,
+    sync::atomic::{fence, AtomicBool, AtomicU32, Ordering},
+};
+
+/// An object that can become inaccessible at runtime.
+///
+/// Once access is revoked and all concurrent users complete (i.e., all existing instances of
+/// [`RevocableGuard`] are dropped), the wrapped object is also dropped.
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::revocable::Revocable;
+///
+/// struct Example {
+///     a: u32,
+///     b: u32,
+/// }
+///
+/// fn add_two(v: &Revocable<Example>) -> Option<u32> {
+///     let guard = v.try_access()?;
+///     Some(guard.a + guard.b)
+/// }
+///
+/// let v = Revocable::new(Example { a: 10, b: 20 });
+/// assert_eq!(add_two(&v), Some(30));
+/// v.revoke();
+/// assert_eq!(add_two(&v), None);
+/// ```
+///
+/// Sample example as above, but explicitly using the rcu read side lock.
+///
+/// ```
+/// # use kernel::revocable::Revocable;
+/// use kernel::sync::rcu;
+///
+/// struct Example {
+///     a: u32,
+///     b: u32,
+/// }
+///
+/// fn add_two(v: &Revocable<Example>) -> Option<u32> {
+///     let guard = rcu::read_lock();
+///     let e = v.try_access_with_guard(&guard)?;
+///     Some(e.a + e.b)
+/// }
+///
+/// let v = Revocable::new(Example { a: 10, b: 20 });
+/// assert_eq!(add_two(&v), Some(30));
+/// v.revoke();
+/// assert_eq!(add_two(&v), None);
+/// ```
+#[pin_data(PinnedDrop)]
+pub struct Revocable<T> {
+    is_available: AtomicBool,
+    #[pin]
+    data: MaybeUninit<UnsafeCell<T>>,
+}
+
+// SAFETY: `Revocable` is `Send` if the wrapped object is also `Send`. This is because while the
+// functionality exposed by `Revocable` can be accessed from any thread/CPU, it is possible that
+// this isn't supported by the wrapped object.
+unsafe impl<T: Send> Send for Revocable<T> {}
+
+// SAFETY: `Revocable` is `Sync` if the wrapped object is both `Send` and `Sync`. We require `Send`
+// from the wrapped object as well because  of `Revocable::revoke`, which can trigger the `Drop`
+// implementation of the wrapped object from an arbitrary thread.
+unsafe impl<T: Sync + Send> Sync for Revocable<T> {}
+
+impl<T> Revocable<T> {
+    /// Creates a new revocable instance of the given data.
+    pub fn new(data: impl PinInit<T>) -> impl PinInit<Self> {
+        pin_init!(Self {
+            is_available: AtomicBool::new(true),
+            data <- unsafe {
+                init::pin_init_from_closure(move |slot: *mut MaybeUninit<UnsafeCell<T>>| {
+                    init::PinInit::<T, core::convert::Infallible>::__pinned_init(data,
+                                                                                 slot as *mut T)?;
+                    Ok::<(), core::convert::Infallible>(())
+                })
+            },
+        })
+    }
+
+    /// Tries to access the \[revocable\] wrapped object.
+    ///
+    /// Returns `None` if the object has been revoked and is therefore no longer accessible.
+    ///
+    /// Returns a guard that gives access to the object otherwise; the object is guaranteed to
+    /// remain accessible while the guard is alive. In such cases, callers are not allowed to sleep
+    /// because another CPU may be waiting to complete the revocation of this object.
+    pub fn try_access(&self) -> Option<RevocableGuard<'_, T>> {
+        let guard = rcu::read_lock();
+        if self.is_available.load(Ordering::Relaxed) {
+            // SAFETY: Since `self.is_available` is true, data is initialised and has to remain
+            // valid because the RCU read side lock prevents it from being dropped.
+            Some(unsafe { RevocableGuard::new(self.data.assume_init_ref().get(), guard) })
+        } else {
+            None
+        }
+    }
+
+    /// Tries to access the \[revocable\] wrapped object.
+    ///
+    /// Returns `None` if the object has been revoked and is therefore no longer accessible.
+    ///
+    /// Returns a shared reference to the object otherwise; the object is guaranteed to
+    /// remain accessible while the rcu read side guard is alive. In such cases, callers are not
+    /// allowed to sleep because another CPU may be waiting to complete the revocation of this
+    /// object.
+    pub fn try_access_with_guard<'a>(&'a self, _guard: &'a rcu::Guard) -> Option<&'a T> {
+        if self.is_available.load(Ordering::Relaxed) {
+            // SAFETY: Since `self.is_available` is true, data is initialised and has to remain
+            // valid because the RCU read side lock prevents it from being dropped.
+            Some(unsafe { &*self.data.assume_init_ref().get() })
+        } else {
+            None
+        }
+    }
+
+    /// Revokes access to and drops the wrapped object.
+    ///
+    /// Access to the object is revoked immediately to new callers of [`Revocable::try_access`]. If
+    /// there are concurrent users of the object (i.e., ones that called [`Revocable::try_access`]
+    /// beforehand and still haven't dropped the returned guard), this function waits for the
+    /// concurrent access to complete before dropping the wrapped object.
+    pub fn revoke(&self) {
+        if self
+            .is_available
+            .compare_exchange(true, false, Ordering::Relaxed, Ordering::Relaxed)
+            .is_ok()
+        {
+            // SAFETY: Just an FFI call, there are no further requirements.
+            unsafe { bindings::synchronize_rcu() };
+
+            // SAFETY: We know `self.data` is valid because only one CPU can succeed the
+            // `compare_exchange` above that takes `is_available` from `true` to `false`.
+            unsafe { drop_in_place(self.data.assume_init_ref().get()) };
+        }
+    }
+}
+
+#[pinned_drop]
+impl<T> PinnedDrop for Revocable<T> {
+    fn drop(self: Pin<&mut Self>) {
+        // Drop only if the data hasn't been revoked yet (in which case it has already been
+        // dropped).
+        // SAFETY: We are not moving out of `p`, only dropping in place
+        let p = unsafe { self.get_unchecked_mut() };
+        if *p.is_available.get_mut() {
+            // SAFETY: We know `self.data` is valid because no other CPU has changed
+            // `is_available` to `false` yet, and no other CPU can do it anymore because this CPU
+            // holds the only reference (mutable) to `self` now.
+            unsafe { drop_in_place(p.data.assume_init_ref().get()) };
+        }
+    }
+}
+
+/// A guard that allows access to a revocable object and keeps it alive.
+///
+/// CPUs may not sleep while holding on to [`RevocableGuard`] because it's in atomic context
+/// holding the RCU read-side lock.
+///
+/// # Invariants
+///
+/// The RCU read-side lock is held while the guard is alive.
+pub struct RevocableGuard<'a, T> {
+    data_ref: *const T,
+    _rcu_guard: rcu::Guard,
+    _p: PhantomData<&'a ()>,
+}
+
+impl<T> RevocableGuard<'_, T> {
+    fn new(data_ref: *const T, rcu_guard: rcu::Guard) -> Self {
+        Self {
+            data_ref,
+            _rcu_guard: rcu_guard,
+            _p: PhantomData,
+        }
+    }
+}
+
+impl<T> Deref for RevocableGuard<'_, T> {
+    type Target = T;
+
+    fn deref(&self) -> &Self::Target {
+        // SAFETY: By the type invariants, we hold the rcu read-side lock, so the object is
+        // guaranteed to remain valid.
+        unsafe { &*self.data_ref }
+    }
+}
+
+/// An object that can become inaccessible at runtime.
+///
+/// Once access is revoked and all concurrent users complete (i.e., all existing instances of
+/// [`AsyncRevocableGuard`] are dropped), the wrapped object is also dropped.
+///
+/// Unlike [`Revocable`], [`AsyncRevocable`] does not wait for concurrent users of the wrapped
+/// object to finish before [`AsyncRevocable::revoke`] completes -- thus the async qualifier. This
+/// has the advantage of not requiring RCU locks or waits of any kind.
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::revocable::AsyncRevocable;
+///
+/// struct Example {
+///     a: u32,
+///     b: u32,
+/// }
+///
+/// fn add_two(v: &AsyncRevocable<Example>) -> Option<u32> {
+///     let guard = v.try_access()?;
+///     Some(guard.a + guard.b)
+/// }
+///
+/// let v = AsyncRevocable::new(Example { a: 10, b: 20 });
+/// assert_eq!(add_two(&v), Some(30));
+/// v.revoke();
+/// assert_eq!(add_two(&v), None);
+/// ```
+///
+/// Example where revocation happens while there is a user:
+///
+/// ```
+/// # use kernel::revocable::AsyncRevocable;
+/// use core::sync::atomic::{AtomicBool, Ordering};
+///
+/// struct Example {
+///     a: u32,
+///     b: u32,
+/// }
+///
+/// static DROPPED: AtomicBool = AtomicBool::new(false);
+///
+/// impl Drop for Example {
+///     fn drop(&mut self) {
+///         DROPPED.store(true, Ordering::Relaxed);
+///     }
+/// }
+///
+/// fn add_two(v: &AsyncRevocable<Example>) -> Option<u32> {
+///     let guard = v.try_access()?;
+///     Some(guard.a + guard.b)
+/// }
+///
+/// let v = AsyncRevocable::new(Example { a: 10, b: 20 });
+/// assert_eq!(add_two(&v), Some(30));
+///
+/// let guard = v.try_access().unwrap();
+/// assert!(!v.is_revoked());
+/// assert!(!DROPPED.load(Ordering::Relaxed));
+/// v.revoke();
+/// assert!(!DROPPED.load(Ordering::Relaxed));
+/// assert!(v.is_revoked());
+/// assert!(v.try_access().is_none());
+/// assert_eq!(guard.a + guard.b, 30);
+/// drop(guard);
+/// assert!(DROPPED.load(Ordering::Relaxed));
+/// ```
+pub struct AsyncRevocable<T> {
+    usage_count: AtomicU32,
+    data: MaybeUninit<UnsafeCell<T>>,
+}
+
+// SAFETY: `AsyncRevocable` is `Send` if the wrapped object is also `Send`. This is because while
+// the functionality exposed by `AsyncRevocable` can be accessed from any thread/CPU, it is
+// possible that this isn't supported by the wrapped object.
+unsafe impl<T: Send> Send for AsyncRevocable<T> {}
+
+// SAFETY: `AsyncRevocable` is `Sync` if the wrapped object is both `Send` and `Sync`. We require
+// `Send` from the wrapped object as well because  of `AsyncRevocable::revoke`, which can trigger
+// the `Drop` implementation of the wrapped object from an arbitrary thread.
+unsafe impl<T: Sync + Send> Sync for AsyncRevocable<T> {}
+
+const REVOKED: u32 = 0x80000000;
+const COUNT_MASK: u32 = !REVOKED;
+const SATURATED_COUNT: u32 = REVOKED - 1;
+
+impl<T> AsyncRevocable<T> {
+    /// Creates a new asynchronously revocable instance of the given data.
+    pub fn new(data: T) -> Self {
+        Self {
+            usage_count: AtomicU32::new(0),
+            data: MaybeUninit::new(UnsafeCell::new(data)),
+        }
+    }
+
+    /// Tries to access the \[revocable\] wrapped object.
+    ///
+    /// Returns `None` if the object has been revoked and is therefore no longer accessible.
+    ///
+    /// Returns a guard that gives access to the object otherwise; the object is guaranteed to
+    /// remain accessible while the guard is alive.
+    pub fn try_access(&self) -> Option<AsyncRevocableGuard<'_, T>> {
+        loop {
+            let count = self.usage_count.load(Ordering::Relaxed);
+
+            // Fail attempt to access if the object is already revoked.
+            if count & REVOKED != 0 {
+                return None;
+            }
+
+            // No need to increment if the count is saturated.
+            if count == SATURATED_COUNT
+                || self
+                    .usage_count
+                    .compare_exchange(count, count + 1, Ordering::Relaxed, Ordering::Relaxed)
+                    .is_ok()
+            {
+                return Some(AsyncRevocableGuard { revocable: self });
+            }
+        }
+    }
+
+    /// Revokes access to the protected object.
+    ///
+    /// Returns `true` if access has been revoked, or `false` when the object has already been
+    /// revoked by a previous call to [`AsyncRevocable::revoke`].
+    ///
+    /// This call is non-blocking, that is, no new users of the revocable object will be allowed,
+    /// but potential current users are able to continue to use it and the thread won't wait for
+    /// them to finish. In such cases, the object will be dropped when the last user completes.
+    pub fn revoke(&self) -> bool {
+        // Set the `REVOKED` bit.
+        //
+        // The acquire barrier matches up with the release when decrementing the usage count.
+        let prev = self.usage_count.fetch_or(REVOKED, Ordering::Acquire);
+        if prev & REVOKED != 0 {
+            // Another thread already revoked this object.
+            return false;
+        }
+
+        if prev == 0 {
+            // SAFETY: This thread just revoked the object and the usage count is zero, so the
+            // object is valid and there will be no future users.
+            unsafe { drop_in_place(UnsafeCell::raw_get(self.data.as_ptr())) };
+        }
+
+        true
+    }
+
+    /// Returns whether access to the object has been revoked.
+    pub fn is_revoked(&self) -> bool {
+        self.usage_count.load(Ordering::Relaxed) & REVOKED != 0
+    }
+}
+
+impl<T> Drop for AsyncRevocable<T> {
+    fn drop(&mut self) {
+        let count = *self.usage_count.get_mut();
+        if count != REVOKED {
+            // The object hasn't been dropped yet, so we do it now.
+
+            // This matches with the release when decrementing the usage count.
+            fence(Ordering::Acquire);
+
+            // SAFETY: Since `count` is does not indicate a count of 0 and the REVOKED bit set, the
+            // object is still valid.
+            unsafe { drop_in_place(UnsafeCell::raw_get(self.data.as_ptr())) };
+        }
+    }
+}
+
+/// A guard that allows access to a revocable object and keeps it alive.
+///
+/// # Invariants
+///
+/// The owner owns an increment on the usage count (which may have saturated it), which keeps the
+/// revocable object alive.
+pub struct AsyncRevocableGuard<'a, T> {
+    revocable: &'a AsyncRevocable<T>,
+}
+
+impl<T> Deref for AsyncRevocableGuard<'_, T> {
+    type Target = T;
+
+    fn deref(&self) -> &Self::Target {
+        // SAFETY: The type invariants guarantee that the caller owns an increment.
+        unsafe { &*self.revocable.data.assume_init_ref().get() }
+    }
+}
+
+impl<T> Drop for AsyncRevocableGuard<'_, T> {
+    fn drop(&mut self) {
+        loop {
+            let count = self.revocable.usage_count.load(Ordering::Relaxed);
+            let actual_count = count & COUNT_MASK;
+            if actual_count == SATURATED_COUNT {
+                // The count is saturated, so we won't decrement (nor do we drop the object).
+                return;
+            }
+
+            if actual_count == 0 {
+                // Trying to underflow the count.
+                panic!("actual_count is zero");
+            }
+
+            // On success, we use release ordering, which matches with the acquire in one of the
+            // places where we drop the object, namely: below, in `AsyncRevocable::revoke`, or in
+            // `AsyncRevocable::drop`.
+            if self
+                .revocable
+                .usage_count
+                .compare_exchange(count, count - 1, Ordering::Release, Ordering::Relaxed)
+                .is_ok()
+            {
+                if count == 1 | REVOKED {
+                    // `count`  is now zero and it is revoked, so free it now.
+
+                    // This matches with the release above (which may have happened in other
+                    // threads concurrently).
+                    fence(Ordering::Acquire);
+
+                    // SAFETY: Since `count` was 1, the object is still alive.
+                    unsafe { drop_in_place(UnsafeCell::raw_get(self.revocable.data.as_ptr())) };
+                }
+
+                return;
+            }
+        }
+    }
+}
-- 
2.45.1

[RFC PATCH 06/11] rust: add device::Data

[Danilo Krummrich]

From: Wedson Almeida Filho <wedsonaf@gmail.com>

Add a generic type `device::Data` to represent driver specific data bound
to a device.

`device::Data` also stores and allows access to registrations, which are
revoked automatically when the corresponding device is unbound, even if
the `device::Data`'s reference count is non-zero.

Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
Co-developed-by: Danilo Krummrich <dakr@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/kernel/device.rs | 103 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 102 insertions(+), 1 deletion(-)

diff --git a/rust/kernel/device.rs b/rust/kernel/device.rs
index fafec70effb6..b1c3f7a0d623 100644
--- a/rust/kernel/device.rs
+++ b/rust/kernel/device.rs
@@ -4,11 +4,24 @@
 //!
 //! C header: [`include/linux/device.h`](../../../../include/linux/device.h)
 
+use macros::pin_data;
+
 use crate::{
+    alloc::flags::*,
     bindings,
+    error::Result,
+    init::InPlaceInit,
+    init::PinInit,
+    pin_init,
+    str::CStr,
+    sync::{LockClassKey, RevocableMutex, RevocableMutexGuard, UniqueArc},
     types::{ARef, Opaque},
 };
-use core::ptr;
+use core::{
+    ops::{Deref, DerefMut},
+    pin::Pin,
+    ptr,
+};
 
 /// A ref-counted device.
 ///
@@ -74,3 +87,91 @@ unsafe impl Send for Device {}
 // SAFETY: `Device` only holds a pointer to a C device, references to which are safe to be used
 // from any thread.
 unsafe impl Sync for Device {}
+
+/// Device data.
+///
+/// When a device is unbound (for whatever reason, for example, because the device was unplugged or
+/// because the user decided to unbind the driver), the driver is given a chance to clean up its
+/// state.
+///
+/// The device data is reference-counted because other subsystems may hold pointers to it; some
+/// device state must be freed and not used anymore, while others must remain accessible.
+///
+/// This struct separates the device data into two categories:
+///   1. Registrations: are destroyed when the device is removed.
+///   2. General data: remain available as long as the reference count is nonzero.
+///
+/// This struct implements the `DeviceRemoval` trait such that `registrations` can be revoked when
+/// the device is unbound.
+#[pin_data]
+pub struct Data<T, U> {
+    #[pin]
+    registrations: RevocableMutex<T>,
+    #[pin]
+    general: U,
+}
+
+/// Safely creates an new reference-counted instance of [`Data`].
+#[doc(hidden)]
+#[macro_export]
+macro_rules! new_device_data {
+    ($reg:expr, $gen:expr, $name:literal) => {{
+        static CLASS1: $crate::sync::LockClassKey = $crate::sync::LockClassKey::new();
+        let regs = $reg;
+        let gen = $gen;
+        let name = $crate::c_str!($name);
+        $crate::device::Data::try_new(regs, gen, name, &CLASS1)
+    }};
+}
+
+impl<T, U> Data<T, U> {
+    /// Creates a new instance of `Data`.
+    ///
+    /// It is recommended that the [`new_device_data`] macro be used as it automatically creates
+    /// the lock classes.
+    pub fn try_new(
+        registrations: T,
+        general: impl PinInit<U>,
+        name: &'static CStr,
+        key1: &'static LockClassKey,
+    ) -> Result<Pin<UniqueArc<Self>>> {
+        let ret = UniqueArc::pin_init(
+            pin_init!(Self {
+                registrations <- RevocableMutex::new(
+                    registrations,
+                    name,
+                    key1,
+                ),
+                general <- general,
+            }),
+            GFP_KERNEL,
+        )?;
+
+        Ok(ret)
+    }
+
+    /// Returns the locked registrations if they're still available.
+    pub fn registrations(&self) -> Option<RevocableMutexGuard<'_, T>> {
+        self.registrations.try_write()
+    }
+}
+
+impl<T, U> crate::driver::DeviceRemoval for Data<T, U> {
+    fn device_remove(&self) {
+        self.registrations.revoke();
+    }
+}
+
+impl<T, U> Deref for Data<T, U> {
+    type Target = U;
+
+    fn deref(&self) -> &U {
+        &self.general
+    }
+}
+
+impl<T, U> DerefMut for Data<T, U> {
+    fn deref_mut(&mut self) -> &mut U {
+        &mut self.general
+    }
+}
-- 
2.45.1

[RFC PATCH 07/11] rust: add `dev_*` print macros.

[Danilo Krummrich]

From: Wedson Almeida Filho <wedsonaf@google.com>

Implement `dev_*` print macros for `device::Device`.

They behave like the macros with the same names in C, i.e., they print
messages to the kernel ring buffer with the given level, prefixing the
messages with corresponding device information.

Signed-off-by: Wedson Almeida Filho <wedsonaf@google.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/kernel/device.rs  | 321 +++++++++++++++++++++++++++++++++++++++++
 rust/kernel/prelude.rs |   2 +
 2 files changed, 323 insertions(+)

diff --git a/rust/kernel/device.rs b/rust/kernel/device.rs
index b1c3f7a0d623..2988aeb4e040 100644
--- a/rust/kernel/device.rs
+++ b/rust/kernel/device.rs
@@ -18,11 +18,15 @@
     types::{ARef, Opaque},
 };
 use core::{
+    fmt,
     ops::{Deref, DerefMut},
     pin::Pin,
     ptr,
 };
 
+#[cfg(CONFIG_PRINTK)]
+use crate::c_str;
+
 /// A ref-counted device.
 ///
 /// # Invariants
@@ -66,6 +70,110 @@ pub unsafe fn as_ref<'a>(ptr: *mut bindings::device) -> &'a Self {
         // SAFETY: Guaranteed by the safety requirements of the function.
         unsafe { &*ptr.cast() }
     }
+
+    /// Prints an emergency-level message (level 0) prefixed with device information.
+    ///
+    /// More details are available from [`dev_emerg`].
+    ///
+    /// [`dev_emerg`]: crate::dev_emerg
+    pub fn pr_emerg(&self, args: fmt::Arguments<'_>) {
+        // SAFETY: `klevel` is null-terminated, uses one of the kernel constants.
+        unsafe { self.printk(bindings::KERN_EMERG, args) };
+    }
+
+    /// Prints an alert-level message (level 1) prefixed with device information.
+    ///
+    /// More details are available from [`dev_alert`].
+    ///
+    /// [`dev_alert`]: crate::dev_alert
+    pub fn pr_alert(&self, args: fmt::Arguments<'_>) {
+        // SAFETY: `klevel` is null-terminated, uses one of the kernel constants.
+        unsafe { self.printk(bindings::KERN_ALERT, args) };
+    }
+
+    /// Prints a critical-level message (level 2) prefixed with device information.
+    ///
+    /// More details are available from [`dev_crit`].
+    ///
+    /// [`dev_crit`]: crate::dev_crit
+    pub fn pr_crit(&self, args: fmt::Arguments<'_>) {
+        // SAFETY: `klevel` is null-terminated, uses one of the kernel constants.
+        unsafe { self.printk(bindings::KERN_CRIT, args) };
+    }
+
+    /// Prints an error-level message (level 3) prefixed with device information.
+    ///
+    /// More details are available from [`dev_err`].
+    ///
+    /// [`dev_err`]: crate::dev_err
+    pub fn pr_err(&self, args: fmt::Arguments<'_>) {
+        // SAFETY: `klevel` is null-terminated, uses one of the kernel constants.
+        unsafe { self.printk(bindings::KERN_ERR, args) };
+    }
+
+    /// Prints a warning-level message (level 4) prefixed with device information.
+    ///
+    /// More details are available from [`dev_warn`].
+    ///
+    /// [`dev_warn`]: crate::dev_warn
+    pub fn pr_warn(&self, args: fmt::Arguments<'_>) {
+        // SAFETY: `klevel` is null-terminated, uses one of the kernel constants.
+        unsafe { self.printk(bindings::KERN_WARNING, args) };
+    }
+
+    /// Prints a notice-level message (level 5) prefixed with device information.
+    ///
+    /// More details are available from [`dev_notice`].
+    ///
+    /// [`dev_notice`]: crate::dev_notice
+    pub fn pr_notice(&self, args: fmt::Arguments<'_>) {
+        // SAFETY: `klevel` is null-terminated, uses one of the kernel constants.
+        unsafe { self.printk(bindings::KERN_NOTICE, args) };
+    }
+
+    /// Prints an info-level message (level 6) prefixed with device information.
+    ///
+    /// More details are available from [`dev_info`].
+    ///
+    /// [`dev_info`]: crate::dev_info
+    pub fn pr_info(&self, args: fmt::Arguments<'_>) {
+        // SAFETY: `klevel` is null-terminated, uses one of the kernel constants.
+        unsafe { self.printk(bindings::KERN_INFO, args) };
+    }
+
+    /// Prints a debug-level message (level 7) prefixed with device information.
+    ///
+    /// More details are available from [`dev_dbg`].
+    ///
+    /// [`dev_dbg`]: crate::dev_dbg
+    pub fn pr_dbg(&self, args: fmt::Arguments<'_>) {
+        if cfg!(debug_assertions) {
+            // SAFETY: `klevel` is null-terminated, uses one of the kernel constants.
+            unsafe { self.printk(bindings::KERN_DEBUG, args) };
+        }
+    }
+
+    /// Prints the provided message to the console.
+    ///
+    /// # Safety
+    ///
+    /// Callers must ensure that `klevel` is null-terminated; in particular, one of the
+    /// `KERN_*`constants, for example, `KERN_CRIT`, `KERN_ALERT`, etc.
+    #[cfg_attr(not(CONFIG_PRINTK), allow(unused_variables))]
+    unsafe fn printk(&self, klevel: &[u8], msg: fmt::Arguments<'_>) {
+        // SAFETY: `klevel` is null-terminated and one of the kernel constants. `self.as_raw`
+        // is valid because `self` is valid. The "%pA" format string expects a pointer to
+        // `fmt::Arguments`, which is what we're passing as the last argument.
+        #[cfg(CONFIG_PRINTK)]
+        unsafe {
+            bindings::_dev_printk(
+                klevel as *const _ as *const core::ffi::c_char,
+                self.as_raw(),
+                c_str!("%pA").as_char_ptr(),
+                &msg as *const _ as *const core::ffi::c_void,
+            )
+        };
+    }
 }
 
 // SAFETY: Instances of `Device` are always ref-counted.
@@ -175,3 +283,216 @@ fn deref_mut(&mut self) -> &mut U {
         &mut self.general
     }
 }
+
+#[doc(hidden)]
+#[macro_export]
+macro_rules! dev_printk {
+    ($method:ident, $dev:expr, $($f:tt)*) => {
+        {
+            // We have an explicity `use` statement here so that callers of this macro are not
+            // required to explicitly use the `RawDevice` trait to use its functions.
+            use $crate::device::Device;
+            ($dev).$method(core::format_args!($($f)*));
+        }
+    }
+}
+
+/// Prints an emergency-level message (level 0) prefixed with device information.
+///
+/// This level should be used if the system is unusable.
+///
+/// Equivalent to the kernel's `dev_emerg` macro.
+///
+/// Mimics the interface of [`std::print!`]. More information about the syntax is available from
+/// [`core::fmt`] and [`alloc::format!`].
+///
+/// [`std::print!`]: https://doc.rust-lang.org/std/macro.print.html
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::device::Device;
+///
+/// fn example(dev: &Device) {
+///     dev_emerg!(dev, "hello {}\n", "there");
+/// }
+/// ```
+#[macro_export]
+macro_rules! dev_emerg {
+    ($($f:tt)*) => { $crate::dev_printk!(pr_emerg, $($f)*); }
+}
+
+/// Prints an alert-level message (level 1) prefixed with device information.
+///
+/// This level should be used if action must be taken immediately.
+///
+/// Equivalent to the kernel's `dev_alert` macro.
+///
+/// Mimics the interface of [`std::print!`]. More information about the syntax is available from
+/// [`core::fmt`] and [`alloc::format!`].
+///
+/// [`std::print!`]: https://doc.rust-lang.org/std/macro.print.html
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::device::Device;
+///
+/// fn example(dev: &Device) {
+///     dev_alert!(dev, "hello {}\n", "there");
+/// }
+/// ```
+#[macro_export]
+macro_rules! dev_alert {
+    ($($f:tt)*) => { $crate::dev_printk!(pr_alert, $($f)*); }
+}
+
+/// Prints a critical-level message (level 2) prefixed with device information.
+///
+/// This level should be used in critical conditions.
+///
+/// Equivalent to the kernel's `dev_crit` macro.
+///
+/// Mimics the interface of [`std::print!`]. More information about the syntax is available from
+/// [`core::fmt`] and [`alloc::format!`].
+///
+/// [`std::print!`]: https://doc.rust-lang.org/std/macro.print.html
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::device::Device;
+///
+/// fn example(dev: &Device) {
+///     dev_crit!(dev, "hello {}\n", "there");
+/// }
+/// ```
+#[macro_export]
+macro_rules! dev_crit {
+    ($($f:tt)*) => { $crate::dev_printk!(pr_crit, $($f)*); }
+}
+
+/// Prints an error-level message (level 3) prefixed with device information.
+///
+/// This level should be used in error conditions.
+///
+/// Equivalent to the kernel's `dev_err` macro.
+///
+/// Mimics the interface of [`std::print!`]. More information about the syntax is available from
+/// [`core::fmt`] and [`alloc::format!`].
+///
+/// [`std::print!`]: https://doc.rust-lang.org/std/macro.print.html
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::device::Device;
+///
+/// fn example(dev: &Device) {
+///     dev_err!(dev, "hello {}\n", "there");
+/// }
+/// ```
+#[macro_export]
+macro_rules! dev_err {
+    ($($f:tt)*) => { $crate::dev_printk!(pr_err, $($f)*); }
+}
+
+/// Prints a warning-level message (level 4) prefixed with device information.
+///
+/// This level should be used in warning conditions.
+///
+/// Equivalent to the kernel's `dev_warn` macro.
+///
+/// Mimics the interface of [`std::print!`]. More information about the syntax is available from
+/// [`core::fmt`] and [`alloc::format!`].
+///
+/// [`std::print!`]: https://doc.rust-lang.org/std/macro.print.html
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::device::Device;
+///
+/// fn example(dev: &Device) {
+///     dev_warn!(dev, "hello {}\n", "there");
+/// }
+/// ```
+#[macro_export]
+macro_rules! dev_warn {
+    ($($f:tt)*) => { $crate::dev_printk!(pr_warn, $($f)*); }
+}
+
+/// Prints a notice-level message (level 5) prefixed with device information.
+///
+/// This level should be used in normal but significant conditions.
+///
+/// Equivalent to the kernel's `dev_notice` macro.
+///
+/// Mimics the interface of [`std::print!`]. More information about the syntax is available from
+/// [`core::fmt`] and [`alloc::format!`].
+///
+/// [`std::print!`]: https://doc.rust-lang.org/std/macro.print.html
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::device::Device;
+///
+/// fn example(dev: &Device) {
+///     dev_notice!(dev, "hello {}\n", "there");
+/// }
+/// ```
+#[macro_export]
+macro_rules! dev_notice {
+    ($($f:tt)*) => { $crate::dev_printk!(pr_notice, $($f)*); }
+}
+
+/// Prints an info-level message (level 6) prefixed with device information.
+///
+/// This level should be used for informational messages.
+///
+/// Equivalent to the kernel's `dev_info` macro.
+///
+/// Mimics the interface of [`std::print!`]. More information about the syntax is available from
+/// [`core::fmt`] and [`alloc::format!`].
+///
+/// [`std::print!`]: https://doc.rust-lang.org/std/macro.print.html
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::device::Device;
+///
+/// fn example(dev: &Device) {
+///     dev_info!(dev, "hello {}\n", "there");
+/// }
+/// ```
+#[macro_export]
+macro_rules! dev_info {
+    ($($f:tt)*) => { $crate::dev_printk!(pr_info, $($f)*); }
+}
+
+/// Prints a debug-level message (level 7) prefixed with device information.
+///
+/// This level should be used for debug messages.
+///
+/// Equivalent to the kernel's `dev_dbg` macro, except that it doesn't support dynamic debug yet.
+///
+/// Mimics the interface of [`std::print!`]. More information about the syntax is available from
+/// [`core::fmt`] and [`alloc::format!`].
+///
+/// [`std::print!`]: https://doc.rust-lang.org/std/macro.print.html
+///
+/// # Examples
+///
+/// ```
+/// # use kernel::device::Device;
+///
+/// fn example(dev: &Device) {
+///     dev_dbg!(dev, "hello {}\n", "there");
+/// }
+/// ```
+#[macro_export]
+macro_rules! dev_dbg {
+    ($($f:tt)*) => { $crate::dev_printk!(pr_dbg, $($f)*); }
+}
diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs
index b37a0b3180fb..c5765ab863d6 100644
--- a/rust/kernel/prelude.rs
+++ b/rust/kernel/prelude.rs
@@ -27,6 +27,8 @@
 // `super::std_vendor` is hidden, which makes the macro inline for some reason.
 #[doc(no_inline)]
 pub use super::dbg;
+pub use super::fmt;
+pub use super::{dev_alert, dev_crit, dev_dbg, dev_emerg, dev_err, dev_info, dev_notice, dev_warn};
 pub use super::{pr_alert, pr_crit, pr_debug, pr_emerg, pr_err, pr_info, pr_notice, pr_warn};
 
 pub use super::{init, pin_init, try_init, try_pin_init};
-- 
2.45.1

[RFC PATCH 08/11] rust: add devres abstraction

[Danilo Krummrich]

Add a Rust abstraction for the kernel's devres (device resource
management) implementation.

The Devres type acts as a container to manage the lifetime and
accessibility of device bound resources. Therefore it registers a
devres callback and revokes access to the resource on invocation.

Users of the Devres abstraction can simply free the corresponding
resources in their Drop implementation, which is invoked when either the
Devres instance goes out of scope or the devres callback leads to the
resource being revoked, which implies a call to drop_in_place().

Co-developed-by: Philipp Stanner <pstanner@redhat.com>
Signed-off-by: Philipp Stanner <pstanner@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/helpers.c        |   5 ++
 rust/kernel/devres.rs | 151 ++++++++++++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs    |   1 +
 3 files changed, 157 insertions(+)
 create mode 100644 rust/kernel/devres.rs

diff --git a/rust/helpers.c b/rust/helpers.c
index 1d3e800140fc..34061eca05a0 100644
--- a/rust/helpers.c
+++ b/rust/helpers.c
@@ -173,6 +173,11 @@ void rust_helper_rcu_read_unlock(void)
 EXPORT_SYMBOL_GPL(rust_helper_rcu_read_unlock);
 /* end rcu */
 
+int rust_helper_devm_add_action(struct device *dev, void (*action)(void *), void *data)
+{
+	return devm_add_action(dev, action, data);
+}
+
 /*
  * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can
  * use it in contexts where Rust expects a `usize` like slice (array) indices.
diff --git a/rust/kernel/devres.rs b/rust/kernel/devres.rs
new file mode 100644
index 000000000000..bf7bd304cd9b
--- /dev/null
+++ b/rust/kernel/devres.rs
@@ -0,0 +1,151 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Devres abstraction
+//!
+//! [`Devres`] represents an abstraction for the kernel devres (device resource management)
+//! implementation.
+
+use crate::{
+    alloc::Flags,
+    bindings,
+    device::Device,
+    error::{Error, Result},
+    prelude::*,
+    revocable::Revocable,
+    types::ARef,
+};
+
+use core::ffi::c_void;
+use core::ops::Deref;
+
+#[pin_data]
+struct DevresInner<T> {
+    dev: ARef<Device>,
+    #[pin]
+    data: Revocable<T>,
+}
+
+/// This abstraction is meant to be used by subsystems to containerize [`Device`] bound resources to
+/// manage their lifetime.
+///
+/// [`Device`] bound resources should be freed when either the resource goes out of scope or the
+/// [`Device`] is unbound respectively, depending on what happens first.
+///
+/// To achieve that [`Devres`] registers a devres callback on creation, which is called once the
+/// [`Device`] is unbound, revoking access to the encapsulated resource (see also [`Revocable`]).
+///
+/// After the [`Devres`] has been unbound it is not possible to access the encapsulated resource
+/// anymore.
+///
+/// [`Devres`] users should make sure to simply free the corresponding backing resource in `T`'s
+/// [`Drop`] implementation.
+///
+/// # Example
+///
+/// ```
+/// use kernel::devres::Devres;
+///
+/// // See also [`pci::Bar`] for a real example.
+/// struct IoRemap(IoMem);
+///
+/// impl IoRemap {
+///     fn new(usize paddr, usize len) -> Result<Self>{
+///         // assert success
+///         let addr = unsafe { bindings::ioremap(paddr as _); };
+///         let iomem = IoMem::new(addr, len)?;
+///
+///         Ok(IoRemap(iomem))
+///     }
+/// }
+///
+/// impl Drop for IoRemap {
+///     fn drop(&mut self) {
+///         unsafe { bindings::iounmap(self.0.ioptr as _); };
+///     }
+/// }
+///
+/// impl Deref for IoRemap {
+///    type Target = IoMem;
+///
+///    fn deref(&self) -> &Self::Target {
+///        &self.0
+///    }
+/// }
+///
+/// let devres = Devres::new(dev, IoRemap::new(0xBAAAAAAD, 0x4)?, GFP_KERNEL)?;
+///
+/// let res = devres.try_access().ok_or(ENXIO)?;
+/// res.writel(0xBAD);
+/// ```
+///
+pub struct Devres<T> {
+    inner: Pin<Box<DevresInner<T>>>,
+    callback: unsafe extern "C" fn(*mut c_void),
+}
+
+impl<T> DevresInner<T> {
+    fn as_ptr(&self) -> *const DevresInner<T> {
+        self as *const DevresInner<T>
+    }
+
+    fn as_cptr(&self) -> *mut c_void {
+        self.as_ptr() as *mut c_void
+    }
+}
+
+unsafe extern "C" fn devres_callback<T>(inner: *mut c_void) {
+    let inner = inner as *const DevresInner<T>;
+    let inner = unsafe { &*inner };
+
+    inner.data.revoke();
+}
+
+impl<T> Devres<T> {
+    /// Creates a new [`Devres`] instance of the give data.
+    pub fn new(dev: ARef<Device>, data: T, flags: Flags) -> Result<Self> {
+        let callback = devres_callback::<T>;
+
+        let inner = Box::pin_init(
+            pin_init!( DevresInner {
+                dev: dev,
+                data <- Revocable::new(data),
+            }),
+            flags,
+        )?;
+
+        let ret = unsafe {
+            bindings::devm_add_action(inner.dev.as_raw(), Some(callback), inner.as_cptr())
+        };
+
+        if ret != 0 {
+            return Err(Error::from_errno(ret));
+        }
+
+        // We have to store the exact callback function pointer used with
+        // `bindings::devm_add_action` for `bindings::devm_remove_action`. There compiler might put
+        // multiple definitions of `devres_callback<T>` for the same `T` in both the kernel itself
+        // and modules. Hence, we might see different pointer values depending on whether we look
+        // at `devres_callback<T>`'s address from `Devres::new` or `Devres::drop`.
+        Ok(Devres { inner, callback })
+    }
+}
+
+impl<T> Deref for Devres<T> {
+    type Target = Revocable<T>;
+
+    fn deref(&self) -> &Self::Target {
+        &self.inner.data
+    }
+}
+
+impl<T> Drop for Devres<T> {
+    fn drop(&mut self) {
+        unsafe {
+            bindings::devm_remove_action(
+                self.inner.dev.as_raw(),
+                Some(self.callback),
+                self.inner.as_cptr(),
+            )
+        }
+    }
+}
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index d7d415429517..11645060b444 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -30,6 +30,7 @@
 pub mod alloc;
 mod build_assert;
 pub mod device;
+pub mod devres;
 pub mod driver;
 pub mod error;
 pub mod init;
-- 
2.45.1

[RFC PATCH 09/11] rust: add basic PCI driver abstractions

[Danilo Krummrich]

From: FUJITA Tomonori <fujita.tomonori@gmail.com>

This commit implements the abstractions necessary to bind a most basic
PCI driver to a PCI device. It also serves as a ground layer for further
PCI functionality.

Specifically, a basic PCI driver has to provide register() and
unregister() methods, a PCI device structure for Rust, and probe() and
remove() callbacks for the C side.

A PCI driver shall be able to register itself for the desired devices,
recognized by their device ID. Another basic necessity is the ability to
store driver data, i.e., through pci_set_drvdata().

In congruency with the C implementation of pci_dev, a Rust PCI device
holds a basic device (device::Device) which is always reference counted
to ensure it cannot disappear as long as there are still users.

Holding a basic device allows for both using interfaces that require a
device, as well as such that demand a pci_dev, which can be obtained
through as_raw(), using the established container_of() macro.

Implement a basic driver model with probe() and remove() callbacks,
implementing the corresponding traits from the 'driver' crate.

Implement PCI device IDs.

Implement pci::Device with basic methods, holding an always reference
counted device::Device.

Signed-off-by: FUJITA Tomonori <fujita.tomonori@gmail.com>
Co-developed-by: Philipp Stanner <pstanner@redhat.com>
Signed-off-by: Philipp Stanner <pstanner@redhat.com>
Co-developed-by: Danilo Krummrich <dakr@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/bindings/bindings_helper.h |   1 +
 rust/helpers.c                  |  18 ++
 rust/kernel/lib.rs              |   2 +
 rust/kernel/pci.rs              | 328 ++++++++++++++++++++++++++++++++
 4 files changed, 349 insertions(+)
 create mode 100644 rust/kernel/pci.rs

diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h
index ddb5644d4fd9..32221de16e57 100644
--- a/rust/bindings/bindings_helper.h
+++ b/rust/bindings/bindings_helper.h
@@ -11,6 +11,7 @@
 #include <linux/ethtool.h>
 #include <linux/jiffies.h>
 #include <linux/mdio.h>
+#include <linux/pci.h>
 #include <linux/phy.h>
 #include <linux/refcount.h>
 #include <linux/sched.h>
diff --git a/rust/helpers.c b/rust/helpers.c
index 34061eca05a0..c3d80301185c 100644
--- a/rust/helpers.c
+++ b/rust/helpers.c
@@ -33,6 +33,7 @@
 #include <linux/spinlock.h>
 #include <linux/wait.h>
 #include <linux/workqueue.h>
+#include <linux/pci.h>
 
 __noreturn void rust_helper_BUG(void)
 {
@@ -178,6 +179,23 @@ int rust_helper_devm_add_action(struct device *dev, void (*action)(void *), void
 	return devm_add_action(dev, action, data);
 }
 
+void rust_helper_pci_set_drvdata(struct pci_dev *pdev, void *data)
+{
+	pci_set_drvdata(pdev, data);
+}
+EXPORT_SYMBOL_GPL(rust_helper_pci_set_drvdata);
+
+void *rust_helper_pci_get_drvdata(struct pci_dev *pdev)
+{
+	return pci_get_drvdata(pdev);
+}
+EXPORT_SYMBOL_GPL(rust_helper_pci_get_drvdata);
+
+u64 rust_helper_pci_resource_len(struct pci_dev *pdev, int barnr)
+{
+	return pci_resource_len(pdev, barnr);
+}
+
 /*
  * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can
  * use it in contexts where Rust expects a `usize` like slice (array) indices.
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index 11645060b444..606391cbff83 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -55,6 +55,8 @@
 #[doc(hidden)]
 pub use bindings;
 pub use macros;
+#[cfg(all(CONFIG_PCI, CONFIG_PCI_MSI))]
+pub mod pci;
 pub use uapi;
 
 #[doc(hidden)]
diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
new file mode 100644
index 000000000000..323aea565d84
--- /dev/null
+++ b/rust/kernel/pci.rs
@@ -0,0 +1,328 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Wrappers for the PCI subsystem
+//!
+//! C header: [`include/linux/pci.h`](../../../../include/linux/pci.h)
+
+use crate::{
+    bindings, container_of, device, driver,
+    error::{to_result, Result},
+    str::CStr,
+    types::{ARef, ForeignOwnable},
+    ThisModule,
+};
+use kernel::prelude::*; // for pinned_drop
+
+/// An adapter for the registration of PCI drivers.
+///
+/// # Example
+///
+///```
+/// use kernel::pci;
+///
+/// impl pci::Driver for MyDriver {
+///     type Data = Arc<MyDeviceData>;
+///
+///     define_pci_id_table! {
+///         (),
+///         [ (pci::DeviceId::new(bindings::PCI_VENDOR_ID_MY_VENDOR,
+///                               bindings::PCI_ANY_ID as u32),
+///            None)
+///         ]
+///     }
+///
+///     fn probe(
+///         pdev: &mut pci::Device,
+///         id_info: Option<&Self::IdInfo>
+///     ) -> Result<Arc<Self::Data>> {
+///         ...
+///     }
+///
+///     fn remove(data: &Self::Data) {
+///         ...
+///     }
+/// }
+///
+/// struct MyModule {
+///     _registration: Pin<Box<driver::Registration<pci::Adapter<MyDriver>>>>,
+/// }
+///
+/// impl kernel::Module for MyModule {
+///     fn init(_name: &'static CStr, module: &'static ThisModule) -> Result<Self> {
+///         let registration = driver::Registration::new_pinned(c_str!("MyDriver"), module)?;
+///
+///         Ok(Self {
+///             _registration: registration,
+///         })
+///     }
+/// }
+///```
+pub struct Adapter<T: Driver>(T);
+
+impl<T: Driver> driver::DriverOps for Adapter<T> {
+    type RegType = bindings::pci_driver;
+
+    // SAFETY: The caller must ensure that `reg` is valid and unequal NULL.
+    unsafe fn register(
+        reg: *mut bindings::pci_driver,
+        name: &'static CStr,
+        module: &'static ThisModule,
+    ) -> Result {
+        // SAFETY: Guaranteed by the safety requirements of this function.
+        let pdrv: &mut bindings::pci_driver = unsafe { &mut *reg };
+
+        pdrv.name = name.as_char_ptr();
+        pdrv.probe = Some(Self::probe_callback);
+        pdrv.remove = Some(Self::remove_callback);
+        pdrv.id_table = T::ID_TABLE.as_ref();
+        // SAFETY: Guaranteed by the safety requirements of this function.
+        to_result(unsafe { bindings::__pci_register_driver(reg, module.0, name.as_char_ptr()) })
+    }
+
+    // SAFETY: The caller must ensure that `reg` is valid and unequal NULL.
+    unsafe fn unregister(reg: *mut bindings::pci_driver) {
+        // SAFETY: Guaranteed by the safety requirements of this function.
+        unsafe { bindings::pci_unregister_driver(reg) }
+    }
+}
+
+impl<T: Driver> Adapter<T> {
+    extern "C" fn probe_callback(
+        pdev: *mut bindings::pci_dev,
+        id: *const bindings::pci_device_id,
+    ) -> core::ffi::c_int {
+        // SAFETY: Safe because the core kernel only ever calls the probe callback with a valid
+        // `pdev`.
+        let dev = unsafe { device::Device::from_raw(&mut (*pdev).dev) };
+        // SAFETY: Guaranteed by the rules described above.
+        let mut pdev = unsafe { Device::from_dev(dev) };
+
+        // SAFETY: `id` is a pointer within the static table, so it's always valid.
+        let offset = unsafe { (*id).driver_data };
+        let info = {
+            // SAFETY: The offset comes from a previous call to `offset_from` in `IdArray::new`,
+            // which guarantees that the resulting pointer is within the table.
+            let ptr = unsafe {
+                id.cast::<u8>()
+                    .offset(offset as _)
+                    .cast::<Option<T::IdInfo>>()
+            };
+            // SAFETY: Guaranteed by the preceding safety requirement.
+            unsafe { (*ptr).as_ref() }
+        };
+        match T::probe(&mut pdev, info) {
+            Ok(data) => {
+                // SAFETY:
+                // A valid `pdev` is always passed to this function. `data` is always valid since
+                // it's created in Rust.
+                unsafe { bindings::pci_set_drvdata(pdev.as_raw(), data.into_foreign() as _) };
+            }
+            Err(err) => return Error::to_errno(err),
+        }
+
+        0
+    }
+
+    extern "C" fn remove_callback(pdev: *mut bindings::pci_dev) {
+        // SAFETY: This function is called by the C side and always with a valid `pdev`.
+        let ptr = unsafe { bindings::pci_get_drvdata(pdev) };
+        // SAFETY: Guaranteed by the preceding safety requirement.
+        let data = unsafe { T::Data::from_foreign(ptr) };
+        T::remove(&data);
+        <T::Data as driver::DeviceRemoval>::device_remove(&data);
+    }
+}
+
+/// Abstraction for bindings::pci_device_id.
+#[derive(Clone, Copy)]
+pub struct DeviceId {
+    /// Vendor ID
+    pub vendor: u32,
+    /// Device ID
+    pub device: u32,
+    /// Subsystem vendor ID
+    pub subvendor: u32,
+    /// Subsystem device ID
+    pub subdevice: u32,
+    /// Device class and subclass
+    pub class: u32,
+    /// Limit which sub-fields of the class
+    pub class_mask: u32,
+}
+
+impl DeviceId {
+    const PCI_ANY_ID: u32 = !0;
+
+    /// PCI_DEVICE macro.
+    pub const fn new(vendor: u32, device: u32) -> Self {
+        Self {
+            vendor,
+            device,
+            subvendor: DeviceId::PCI_ANY_ID,
+            subdevice: DeviceId::PCI_ANY_ID,
+            class: 0,
+            class_mask: 0,
+        }
+    }
+
+    /// PCI_DEVICE_CLASS macro.
+    pub const fn with_class(class: u32, class_mask: u32) -> Self {
+        Self {
+            vendor: DeviceId::PCI_ANY_ID,
+            device: DeviceId::PCI_ANY_ID,
+            subvendor: DeviceId::PCI_ANY_ID,
+            subdevice: DeviceId::PCI_ANY_ID,
+            class,
+            class_mask,
+        }
+    }
+
+    /// PCI_DEVICE_ID macro.
+    pub const fn to_rawid(&self, offset: isize) -> bindings::pci_device_id {
+        bindings::pci_device_id {
+            vendor: self.vendor,
+            device: self.device,
+            subvendor: self.subvendor,
+            subdevice: self.subdevice,
+            class: self.class,
+            class_mask: self.class_mask,
+            driver_data: offset as _,
+            override_only: 0,
+        }
+    }
+}
+
+// SAFETY: `ZERO` is all zeroed-out and `to_rawid` stores `offset` in `pci_device_id::driver_data`.
+unsafe impl driver::RawDeviceId for DeviceId {
+    type RawType = bindings::pci_device_id;
+
+    const ZERO: Self::RawType = bindings::pci_device_id {
+        vendor: 0,
+        device: 0,
+        subvendor: 0,
+        subdevice: 0,
+        class: 0,
+        class_mask: 0,
+        driver_data: 0,
+        override_only: 0,
+    };
+}
+
+/// Define a const pci device id table
+///
+/// # Examples
+///
+/// ```ignore
+/// # use kernel::{pci, define_pci_id_table};
+/// #
+/// struct MyDriver;
+/// impl pci::Driver for MyDriver {
+///     // [...]
+/// #   fn probe(_dev: &mut pci::Device, _id_info: Option<&Self::IdInfo>) -> Result {
+/// #       Ok(())
+/// #   }
+/// #   define_pci_id_table! {u32, [
+/// #       (pci::DeviceId::new(0x010800, 0xffffff), None),
+/// #       (pci::DeviceId::with_class(0x010802, 0xfffff), Some(0x10)),
+/// #   ]}
+/// }
+/// ```
+#[macro_export]
+macro_rules! define_pci_id_table {
+    ($data_type:ty, $($t:tt)*) => {
+        type IdInfo = $data_type;
+        const ID_TABLE: $crate::driver::IdTable<'static, $crate::pci::DeviceId, $data_type> = {
+            $crate::define_id_array!(ARRAY, $crate::pci::DeviceId, $data_type, $($t)* );
+            ARRAY.as_table()
+        };
+    };
+}
+pub use define_pci_id_table;
+
+/// The PCI driver trait.
+///
+/// Drivers must implement this trait in order to get a PCI driver registered. Please refer to the
+/// `Adapter` documentation for an example.
+pub trait Driver {
+    /// Data stored on device by driver.
+    ///
+    /// Corresponds to the data set or retrieved via the kernel's
+    /// `pci_{set,get}_drvdata()` functions.
+    ///
+    /// Require that `Data` implements `ForeignOwnable`. We guarantee to
+    /// never move the underlying wrapped data structure.
+    ///
+    /// TODO: Use associated_type_defaults once stabilized:
+    ///
+    /// `type Data: ForeignOwnable + driver::DeviceRemoval = ();`
+    type Data: ForeignOwnable + driver::DeviceRemoval;
+
+    /// The type holding information about each device id supported by the driver.
+    ///
+    /// TODO: Use associated_type_defaults once stabilized:
+    ///
+    /// type IdInfo: 'static = ();
+    type IdInfo: 'static;
+
+    /// The table of device ids supported by the driver.
+    const ID_TABLE: driver::IdTable<'static, DeviceId, Self::IdInfo>;
+
+    /// PCI driver probe.
+    ///
+    /// Called when a new platform device is added or discovered.
+    /// Implementers should attempt to initialize the device here.
+    fn probe(dev: &mut Device, id: Option<&Self::IdInfo>) -> Result<Self::Data>;
+
+    /// PCI driver remove.
+    ///
+    /// Called when a platform device is removed.
+    /// Implementers should prepare the device for complete removal here.
+    fn remove(_data: &Self::Data);
+}
+
+/// The PCI device representation.
+///
+/// A PCI device is based on an always reference counted `device:Device` instance. Cloning a PCI
+/// device, hence, also increments the base device' reference count.
+#[derive(Clone)]
+pub struct Device(ARef<device::Device>);
+
+impl Device {
+    /// Create a PCI Device instance from an existing `device::Device`.
+    ///
+    /// # Safety
+    ///
+    /// `dev` must be an `ARef<device::Device>` whose underlying `bindings::device` is a member of
+    /// a `bindings::pci_dev`.
+    pub unsafe fn from_dev(dev: ARef<device::Device>) -> Self {
+        Self(dev)
+    }
+
+    fn as_raw(&self) -> *mut bindings::pci_dev {
+        // SAFETY: Guaranteed by the requirements described in pci::Device::new().
+        unsafe { container_of!(self.0.as_raw(), bindings::pci_dev, dev) as _ }
+    }
+
+    /// Enable the Device's memory.
+    pub fn enable_device_mem(&self) -> Result {
+        // SAFETY: By the type invariants, we know that `self.ptr` is non-null and valid.
+        let ret = unsafe { bindings::pci_enable_device_mem(self.as_raw()) };
+        if ret != 0 {
+            Err(Error::from_errno(ret))
+        } else {
+            Ok(())
+        }
+    }
+
+    /// Set the Device's master.
+    pub fn set_master(&self) {
+        // SAFETY: By the type invariants, we know that `self.ptr` is non-null and valid.
+        unsafe { bindings::pci_set_master(self.as_raw()) };
+    }
+}
+
+impl AsRef<device::Device> for Device {
+    fn as_ref(&self) -> &device::Device {
+        &self.0
+    }
+}
-- 
2.45.1

[RFC PATCH 10/11] rust: add basic abstractions for iomem operations

[Danilo Krummrich]

From: Philipp Stanner <pstanner@redhat.com>

Access to the kernel's IO-functions (e.g., readb()) is needed by almost
all drivers. Currently, there are no abstractions for those functions.

Since iomem is so widely used, it's necessary to provide a generic
interface that all subsystems providing IO memory can use. The existing
C implementations of such subsystems typically provide their own
wrappers around functions like ioremap() which take care of respecting
resource boundaries etc. It is, therefore, desirable to use these
wrappers because using ioremap() and iounmap() directly would
effectively result in parts of those subsystems being reimplemented in
Rust.

As most if not all device drivers should be fully satisfied regarding
their iomem demands by the existing subsystem interfaces, Rust
abstractions as congruent as possible with the existing infrastructure
shall use the existing subsystem (e.g., PCI) interfaces for creating
IO mappings, while simultaneously wrapping those mappings in Rust
containers whose Drop() traits ensure that the resources are released
again.

The process for mapping iomem would consequently look as follows:

  1. The subsystem abstraction (e.g., PCI) requests and ioremaps the
     memory through the corresponding C functions.
  2. The subsystem uses resources obtained in step #1 to create a Rust
     IoMem data structure that implements the IO functionality such as
     readb() for the iomem.
  3. The subsystem code wrapps IoMem into additional containers that
     ensure, e.g., thread safety, prevent UAF etc. Additionally, the
     subsystem ensures that access to IoMem is revoked latest when the
     driver's remove() callback is invoked.

Hereby, the subsystem data structure obtains ownership over the iomem.
Release of the iomem and, possibly, other subsystem associated data is
then handled through the Drop() trait of the subsystem data structure.

IO memory can become invalid during runtime (for example because the
driver's remove() callback was invoked, revoking access to the driver's
resources). However, in parallel executing routines might still be
active. Consequently, the subsytem should also guard the iomem in some
way.

One way to do this is the Devres implementation, which provides a
container that is capable of revoking access to its payload when
the driver's remove() callback is invoked.

The figure illustrates what usage of IoMem through subsystems might look
like:
               Devres
  *------------------------------*
  |                              |
  |   subsystem data structure   |
  |   *----------------------*   |
  |   |        IoMem         |   |
  |   | *------------------* |   |
  |   | |  io_addr = 0x42, | |   |
  |   | |  io_len = 9001,  | |   |
  |   | |                  | |   |
  |   | |  readb(),        | |   |
  |   | |  writeb(),       | |   |
  |   | |  ...             | |   |
  |   | *------------------* |   |
  |   |   deref(),           |   |
  |   |   drop(),            |   |
  |   |   ...                |   |
  |   *----------------------*   |
  |    deref(),                  |
  |    drop(),                   |
  *------------------------------*

For additional convenience, subsystem abstractions can implement the
Deref() trait for their data structures so that access he iomem can be
fully transparent.

In summary, IoMem would just serve as a container providing the IO
functions, and the subsystem, which knows about the memory layout,
request mechanisms etc, shall create and guard IoMem and ensure that its
resources are released latest at remove() (through Devres) or earlier
through its Drop() implementation.

Add 'IoMem', a struct holding an IO-Pointer and a length parameter,
which both shall be initialized validly by the subsystem.

Add Rust abstractions for basic IO memory operations on IoMem.

Signed-off-by: Philipp Stanner <pstanner@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/helpers.c       | 106 +++++++++++++++++++++++++++++++++
 rust/kernel/iomem.rs | 135 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 241 insertions(+)
 create mode 100644 rust/kernel/iomem.rs

diff --git a/rust/helpers.c b/rust/helpers.c
index c3d80301185c..dc2405772b1a 100644
--- a/rust/helpers.c
+++ b/rust/helpers.c
@@ -34,6 +34,7 @@
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 #include <linux/pci.h>
+#include <linux/io.h>
 
 __noreturn void rust_helper_BUG(void)
 {
@@ -179,6 +180,111 @@ int rust_helper_devm_add_action(struct device *dev, void (*action)(void *), void
 	return devm_add_action(dev, action, data);
 }
 
+/* io.h */
+u8 rust_helper_readb(const volatile void __iomem *addr)
+{
+	return readb(addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_readb);
+
+u16 rust_helper_readw(const volatile void __iomem *addr)
+{
+	return readw(addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_readw);
+
+u32 rust_helper_readl(const volatile void __iomem *addr)
+{
+	return readl(addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_readl);
+
+#ifdef CONFIG_64BIT
+u64 rust_helper_readq(const volatile void __iomem *addr)
+{
+	return readq(addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_readq);
+#endif
+
+void rust_helper_writeb(u8 value, volatile void __iomem *addr)
+{
+	writeb(value, addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_writeb);
+
+void rust_helper_writew(u16 value, volatile void __iomem *addr)
+{
+	writew(value, addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_writew);
+
+void rust_helper_writel(u32 value, volatile void __iomem *addr)
+{
+	writel(value, addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_writel);
+
+#ifdef CONFIG_64BIT
+void rust_helper_writeq(u64 value, volatile void __iomem *addr)
+{
+	writeq(value, addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_writeq);
+#endif
+
+u8 rust_helper_readb_relaxed(const volatile void __iomem *addr)
+{
+	return readb_relaxed(addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_readb_relaxed);
+
+u16 rust_helper_readw_relaxed(const volatile void __iomem *addr)
+{
+	return readw_relaxed(addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_readw_relaxed);
+
+u32 rust_helper_readl_relaxed(const volatile void __iomem *addr)
+{
+	return readl_relaxed(addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_readl_relaxed);
+
+#ifdef CONFIG_64BIT
+u64 rust_helper_readq_relaxed(const volatile void __iomem *addr)
+{
+	return readq_relaxed(addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_readq_relaxed);
+#endif
+
+void rust_helper_writeb_relaxed(u8 value, volatile void __iomem *addr)
+{
+	writeb_relaxed(value, addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_writeb_relaxed);
+
+void rust_helper_writew_relaxed(u16 value, volatile void __iomem *addr)
+{
+	writew_relaxed(value, addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_writew_relaxed);
+
+void rust_helper_writel_relaxed(u32 value, volatile void __iomem *addr)
+{
+	writel_relaxed(value, addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_writel_relaxed);
+
+#ifdef CONFIG_64BIT
+void rust_helper_writeq_relaxed(u64 value, volatile void __iomem *addr)
+{
+	writeq_relaxed(value, addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_writeq_relaxed);
+#endif
+
 void rust_helper_pci_set_drvdata(struct pci_dev *pdev, void *data)
 {
 	pci_set_drvdata(pdev, data);
diff --git a/rust/kernel/iomem.rs b/rust/kernel/iomem.rs
new file mode 100644
index 000000000000..efb6cd0829b4
--- /dev/null
+++ b/rust/kernel/iomem.rs
@@ -0,0 +1,135 @@
+// SPDX-License-Identifier: GPL-2.0
+
+use crate::bindings;
+use crate::error::{code::EINVAL, Result};
+
+/// IO-mapped memory, starting at the base pointer @ioptr and spanning @malxen bytes.
+///
+/// The creator (usually a subsystem such as PCI) is responsible for creating the
+/// mapping, performing an additional region request etc.
+pub struct IoMem {
+    pub ioptr: usize,
+    maxlen: usize,
+}
+
+impl IoMem {
+    pub(crate) fn new(ioptr: usize, maxlen: usize) -> Result<Self> {
+        if ioptr == 0 || maxlen == 0 {
+            return Err(EINVAL);
+        }
+
+        Ok(Self { ioptr, maxlen })
+    }
+
+    fn get_io_addr(&self, offset: usize, len: usize) -> Result<usize> {
+        if offset + len > self.maxlen {
+            return Err(EINVAL);
+        }
+
+        Ok(self.ioptr + offset)
+    }
+
+    pub fn readb(&self, offset: usize) -> Result<u8> {
+        let ioptr: usize = self.get_io_addr(offset, 1)?;
+
+        Ok(unsafe { bindings::readb(ioptr as _) })
+    }
+
+    pub fn readw(&self, offset: usize) -> Result<u16> {
+        let ioptr: usize = self.get_io_addr(offset, 2)?;
+
+        Ok(unsafe { bindings::readw(ioptr as _) })
+    }
+
+    pub fn readl(&self, offset: usize) -> Result<u32> {
+        let ioptr: usize = self.get_io_addr(offset, 4)?;
+
+        Ok(unsafe { bindings::readl(ioptr as _) })
+    }
+
+    pub fn readq(&self, offset: usize) -> Result<u64> {
+        let ioptr: usize = self.get_io_addr(offset, 8)?;
+
+        Ok(unsafe { bindings::readq(ioptr as _) })
+    }
+
+    pub fn readb_relaxed(&self, offset: usize) -> Result<u8> {
+        let ioptr: usize = self.get_io_addr(offset, 1)?;
+
+        Ok(unsafe { bindings::readb_relaxed(ioptr as _) })
+    }
+
+    pub fn readw_relaxed(&self, offset: usize) -> Result<u16> {
+        let ioptr: usize = self.get_io_addr(offset, 2)?;
+
+        Ok(unsafe { bindings::readw_relaxed(ioptr as _) })
+    }
+
+    pub fn readl_relaxed(&self, offset: usize) -> Result<u32> {
+        let ioptr: usize = self.get_io_addr(offset, 4)?;
+
+        Ok(unsafe { bindings::readl_relaxed(ioptr as _) })
+    }
+
+    pub fn readq_relaxed(&self, offset: usize) -> Result<u64> {
+        let ioptr: usize = self.get_io_addr(offset, 8)?;
+
+        Ok(unsafe { bindings::readq_relaxed(ioptr as _) })
+    }
+
+    pub fn writeb(&self, byte: u8, offset: usize) -> Result {
+        let ioptr: usize = self.get_io_addr(offset, 1)?;
+
+        unsafe { bindings::writeb(byte, ioptr as _) }
+        Ok(())
+    }
+
+    pub fn writew(&self, word: u16, offset: usize) -> Result {
+        let ioptr: usize = self.get_io_addr(offset, 2)?;
+
+        unsafe { bindings::writew(word, ioptr as _) }
+        Ok(())
+    }
+
+    pub fn writel(&self, lword: u32, offset: usize) -> Result {
+        let ioptr: usize = self.get_io_addr(offset, 4)?;
+
+        unsafe { bindings::writel(lword, ioptr as _) }
+        Ok(())
+    }
+
+    pub fn writeq(&self, qword: u64, offset: usize) -> Result {
+        let ioptr: usize = self.get_io_addr(offset, 8)?;
+
+        unsafe { bindings::writeq(qword, ioptr as _) }
+        Ok(())
+    }
+
+    pub fn writeb_relaxed(&self, byte: u8, offset: usize) -> Result {
+        let ioptr: usize = self.get_io_addr(offset, 1)?;
+
+        unsafe { bindings::writeb_relaxed(byte, ioptr as _) }
+        Ok(())
+    }
+
+    pub fn writew_relaxed(&self, word: u16, offset: usize) -> Result {
+        let ioptr: usize = self.get_io_addr(offset, 2)?;
+
+        unsafe { bindings::writew_relaxed(word, ioptr as _) }
+        Ok(())
+    }
+
+    pub fn writel_relaxed(&self, lword: u32, offset: usize) -> Result {
+        let ioptr: usize = self.get_io_addr(offset, 4)?;
+
+        unsafe { bindings::writel_relaxed(lword, ioptr as _) }
+        Ok(())
+    }
+
+    pub fn writeq_relaxed(&self, qword: u64, offset: usize) -> Result {
+        let ioptr: usize = self.get_io_addr(offset, 8)?;
+
+        unsafe { bindings::writeq_relaxed(qword, ioptr as _) }
+        Ok(())
+    }
+}
-- 
2.45.1

[RFC PATCH 11/11] rust: PCI: add BAR request and ioremap

[Danilo Krummrich]

From: Philipp Stanner <pstanner@redhat.com>

This commit implements a basic mechanism for requesting and IO-mapping
PCI BARs.

To perform IO on PCI devices it is necessary to have memory mapped PCI
BARs. Before mapping those, a region request should be performed so that
collisions with other drivers can be avoided.

Since all the logic necessary to obtain the aforementioned resources are
already implemented in C, Rust abstractions should use these interfaces.
Hereby, the Rust implementation has to ensure that all resources are
released again latest when the driver's remove() callback gets invoked,
or earlier if the driver drop()s the PCI resource.

This can be achieved through the Devres container, which uses devres
callbacks combined with Revocable to block access to the resource - in
this case, the PCI BAR and its IoMem.

A pci::Bar's Drop() trait shall deregister the memory region request and
iounmap() the mapping. In case remove() is invoked before such a Bar is
drop()ed, the Devres container shall ensure that access to the Bar is
revoke()d (through Revocable) so that no UAFs can occur.

Implement 'Bar', a container for requested and ioremapped PCI BARs.

Implement the Drop() trait such that the memory request and IO-mapping
get freed if Bar goes out of scope.

Implement Deref() so that the container is transparent.

Implement iomap_region() to create a Bar and have the result returned
through a Devres container, ensuring that the resources are latest freed
in the driver's remove() callback and access to the Bar is revoke()d for
outstanding users.

Signed-off-by: Philipp Stanner <pstanner@redhat.com>
Co-developed-by: Danilo Krummrich <dakr@redhat.com>
Signed-off-by: Danilo Krummrich <dakr@redhat.com>
---
 rust/kernel/lib.rs |   1 +
 rust/kernel/pci.rs | 123 ++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 123 insertions(+), 1 deletion(-)

diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index 606391cbff83..15730deca822 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -54,6 +54,7 @@
 
 #[doc(hidden)]
 pub use bindings;
+mod iomem;
 pub use macros;
 #[cfg(all(CONFIG_PCI, CONFIG_PCI_MSI))]
 pub mod pci;
diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs
index 323aea565d84..403a1f53eb25 100644
--- a/rust/kernel/pci.rs
+++ b/rust/kernel/pci.rs
@@ -5,12 +5,17 @@
 //! C header: [`include/linux/pci.h`](../../../../include/linux/pci.h)
 
 use crate::{
-    bindings, container_of, device, driver,
+    alloc::flags::*,
+    bindings, container_of, device,
+    devres::Devres,
+    driver,
     error::{to_result, Result},
+    iomem::IoMem,
     str::CStr,
     types::{ARef, ForeignOwnable},
     ThisModule,
 };
+use core::ops::Deref;
 use kernel::prelude::*; // for pinned_drop
 
 /// An adapter for the registration of PCI drivers.
@@ -287,6 +292,104 @@ pub trait Driver {
 #[derive(Clone)]
 pub struct Device(ARef<device::Device>);
 
+/// A PCI BAR to perform IO-Operations on.
+pub struct Bar {
+    pdev: Device,
+    iomem: IoMem,
+    num: u8,
+}
+
+impl Bar {
+    fn new(pdev: Device, num: u8, name: &CStr) -> Result<Self> {
+        let barnr = num as i32;
+
+        let barlen = pdev.resource_len(num)?;
+        if barlen == 0 {
+            return Err(ENOMEM);
+        }
+
+        // SAFETY:
+        // `pdev` is always valid.
+        // `barnr` is checked for validity at the top of the function.
+        // `name` is always valid.
+        let ret = unsafe { bindings::pci_request_region(pdev.as_raw(), barnr, name.as_char_ptr()) };
+        if ret != 0 {
+            return Err(EBUSY);
+        }
+
+        // SAFETY:
+        // `pdev` is always valid.
+        // `barnr` is checked for validity at the top of the function.
+        // `name` is always valid.
+        let ioptr: usize = unsafe { bindings::pci_iomap(pdev.as_raw(), barnr, 0) } as usize;
+        if ioptr == 0 {
+            // SAFETY:
+            // `pdev` is always valid.
+            // `barnr` is checked for validity at the top of the function.
+            unsafe { bindings::pci_release_region(pdev.as_raw(), barnr) };
+            return Err(ENOMEM);
+        }
+
+        let iomem = match IoMem::new(ioptr, barlen as usize) {
+            Ok(iomem) => iomem,
+            Err(err) => {
+                // SAFETY:
+                // `pdev` is always valid.
+                // `ioptr` was created above, and `num` was checked at the top of the function.
+                unsafe { Self::do_release(&pdev, ioptr, num) };
+                return Err(err);
+            }
+        };
+
+        Ok(Bar { pdev, iomem, num })
+    }
+
+    fn index_is_valid(i: u8) -> bool {
+        // A pci_dev on the C side owns an array of resources with at most
+        // PCI_NUM_RESOURCES entries.
+        if i as i32 >= bindings::PCI_NUM_RESOURCES as i32 {
+            return false;
+        }
+
+        true
+    }
+
+    // SAFETY: The caller should ensure that `ioptr` is valid.
+    unsafe fn do_release(pdev: &Device, ioptr: usize, num: u8) {
+        // SAFETY:
+        // `pdev` is Rust data and guaranteed to be valid.
+        // A valid `ioptr` should be provided by the caller, but an invalid one
+        // does not cause faults on the C side.
+        // `num` is checked for validity above.
+        unsafe {
+            bindings::pci_iounmap(pdev.as_raw(), ioptr as _);
+            bindings::pci_release_region(pdev.as_raw(), num as i32);
+        }
+    }
+
+    fn release(&self) {
+        // SAFETY:
+        // Safe because `self` always contains a refcounted device that belongs
+        // to a pci::Device.
+        // `ioptr` and `num` are always valid because the Bar was created successfully.
+        unsafe { Self::do_release(&self.pdev, self.iomem.ioptr, self.num) };
+    }
+}
+
+impl Drop for Bar {
+    fn drop(&mut self) {
+        self.release();
+    }
+}
+
+impl Deref for Bar {
+    type Target = IoMem;
+
+    fn deref(&self) -> &Self::Target {
+        &self.iomem
+    }
+}
+
 impl Device {
     /// Create a PCI Device instance from an existing `device::Device`.
     ///
@@ -319,6 +422,24 @@ pub fn set_master(&self) {
         // SAFETY: By the type invariants, we know that `self.ptr` is non-null and valid.
         unsafe { bindings::pci_set_master(self.as_raw()) };
     }
+
+    /// Returns the size of the given PCI bar resource.
+    pub fn resource_len(&self, bar: u8) -> Result<bindings::resource_size_t> {
+        if !Bar::index_is_valid(bar) {
+            return Err(EINVAL);
+        }
+
+        // SAFETY: Safe as by the type invariant.
+        Ok(unsafe { bindings::pci_resource_len(self.as_raw(), bar.into()) })
+    }
+
+    /// Mapps an entire PCI-BAR after performing a region-request on it.
+    pub fn iomap_region(&mut self, barnr: u8, name: &CStr) -> Result<Devres<Bar>> {
+        let bar = Bar::new(self.clone(), barnr, name)?;
+        let devres = Devres::new(self.0.clone(), bar, GFP_KERNEL)?;
+
+        Ok(devres)
+    }
 }
 
 impl AsRef<device::Device> for Device {
-- 
2.45.1

Re: [RFC PATCH 01/11] rust: add abstraction for struct device

[Greg KH]

On Mon, May 20, 2024 at 07:25:38PM +0200, Danilo Krummrich wrote:
> Add an (always) reference counted abstraction for a generic struct
> device. This abstraction encapsulates existing struct device instances
> and manages its reference count.
> 
> Subsystems may use this abstraction as a base to abstract subsystem
> specific device instances based on a generic struct device.
> 
> Co-developed-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> ---
>  rust/helpers.c        |  1 +
>  rust/kernel/device.rs | 76 +++++++++++++++++++++++++++++++++++++++++++

What's the status of moving .rs files next to their respective .c files
in the build system?  Keeping them separate like this just isn't going
to work, sorry.

> --- /dev/null
> +++ b/rust/kernel/device.rs
> @@ -0,0 +1,76 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! Generic devices that are part of the kernel's driver model.
> +//!
> +//! C header: [`include/linux/device.h`](../../../../include/linux/device.h)

relative paths for a common "include <linux/device.h" type of thing?
Rust can't handle include paths from directories?

> +
> +use crate::{
> +    bindings,
> +    types::{ARef, Opaque},
> +};
> +use core::ptr;
> +
> +/// A ref-counted device.
> +///
> +/// # Invariants
> +///
> +/// The pointer stored in `Self` is non-null and valid for the lifetime of the ARef instance. In
> +/// particular, the ARef instance owns an increment on underlying object’s reference count.
> +#[repr(transparent)]
> +pub struct Device(Opaque<bindings::device>);
> +
> +impl Device {
> +    /// Creates a new ref-counted instance of an existing device pointer.
> +    ///
> +    /// # Safety
> +    ///
> +    /// Callers must ensure that `ptr` is valid, non-null, and has a non-zero reference count.

Callers NEVER care about the reference count of a struct device, anyone
poking in that is asking for trouble.

And why non-NULL?  Can't you check for that here?  Shouldn't you check
for that here?  Many driver core functions can handle a NULL pointer
just fine (i.e. get/put_device() can), why should Rust code assume that
a pointer passed to it from the C layer is going to have stricter rules
than the C layer can provide?

> +    pub unsafe fn from_raw(ptr: *mut bindings::device) -> ARef<Self> {
> +        // SAFETY: By the safety requirements, ptr is valid.
> +        // Initially increase the reference count by one to compensate for the final decrement once
> +        // this newly created `ARef<Device>` instance is dropped.
> +        unsafe { bindings::get_device(ptr) };
> +
> +        // CAST: `Self` is a `repr(transparent)` wrapper around `bindings::device`.
> +        let ptr = ptr.cast::<Self>();
> +
> +        // SAFETY: By the safety requirements, ptr is valid.
> +        unsafe { ARef::from_raw(ptr::NonNull::new_unchecked(ptr)) }
> +    }
> +
> +    /// Obtain the raw `struct device *`.
> +    pub(crate) fn as_raw(&self) -> *mut bindings::device {
> +        self.0.get()
> +    }
> +
> +    /// Convert a raw `struct device` pointer to a `&Device`.
> +    ///
> +    /// # Safety
> +    ///
> +    /// Callers must ensure that `ptr` is valid, non-null, and has a non-zero reference count for
> +    /// the entire duration when the returned reference exists.

Again, non-NULL might not be true, and reference counts are never
tracked by any user EXCEPT to increment/decrement it, you never know if
it is 0 or not, all you know is that if a pointer is given to you by the
driver core to a 'struct device' for a function that it is a valid
reference at that point in time, or maybe NULL, until your function
returns.  Anything after that can not be counted on.

thanks,

greg k-h

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Greg KH]

On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> From: Wedson Almeida Filho <wedsonaf@gmail.com>
> 
> This defines general functionality related to registering drivers with
> their respective subsystems, and registering modules that implement
> drivers.
> 
> Co-developed-by: Asahi Lina <lina@asahilina.net>
> Signed-off-by: Asahi Lina <lina@asahilina.net>
> Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> ---
>  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
>  rust/kernel/lib.rs           |   4 +-
>  rust/macros/module.rs        |   2 +-
>  samples/rust/rust_minimal.rs |   2 +-
>  samples/rust/rust_print.rs   |   2 +-
>  5 files changed, 498 insertions(+), 4 deletions(-)
>  create mode 100644 rust/kernel/driver.rs
> 
> diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> new file mode 100644
> index 000000000000..e0cfc36d47ff
> --- /dev/null
> +++ b/rust/kernel/driver.rs
> @@ -0,0 +1,492 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> +//!
> +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> +//! using the [`Registration`] class.

Why are you creating new "names" here?  "DriverOps" is part of a 'struct
device_driver' why are you separating it out here?  And what is
'Registration'?  That's a bus/class thing, not a driver thing.

And be very careful of the use of the word 'class' here, remember, there
is 'struct class' as part of the driver model :)

> +use crate::{
> +    alloc::{box_ext::BoxExt, flags::*},
> +    error::code::*,
> +    error::Result,
> +    str::CStr,
> +    sync::Arc,
> +    ThisModule,
> +};
> +use alloc::boxed::Box;
> +use core::{cell::UnsafeCell, marker::PhantomData, ops::Deref, pin::Pin};
> +
> +/// A subsystem (e.g., PCI, Platform, Amba, etc.) that allows drivers to be written for it.
> +pub trait DriverOps {

Again, why is this not called DeviceDriver?

> +    /// The type that holds information about the registration. This is typically a struct defined
> +    /// by the C portion of the kernel.
> +    type RegType: Default;
> +
> +    /// Registers a driver.
> +    ///
> +    /// # Safety
> +    ///
> +    /// `reg` must point to valid, initialised, and writable memory. It may be modified by this
> +    /// function to hold registration state.
> +    ///
> +    /// On success, `reg` must remain pinned and valid until the matching call to
> +    /// [`DriverOps::unregister`].
> +    unsafe fn register(
> +        reg: *mut Self::RegType,
> +        name: &'static CStr,
> +        module: &'static ThisModule,
> +    ) -> Result;
> +
> +    /// Unregisters a driver previously registered with [`DriverOps::register`].
> +    ///
> +    /// # Safety
> +    ///
> +    /// `reg` must point to valid writable memory, initialised by a previous successful call to
> +    /// [`DriverOps::register`].
> +    unsafe fn unregister(reg: *mut Self::RegType);
> +}
> +
> +/// The registration of a driver.
> +pub struct Registration<T: DriverOps> {
> +    is_registered: bool,

Why does a driver need to know if it is registered or not?  Only the
driver core cares about that, please do not expose that, it's racy and
should not be relied on.

> +    concrete_reg: UnsafeCell<T::RegType>,
> +}
> +
> +// SAFETY: `Registration` has no fields or methods accessible via `&Registration`, so it is safe to
> +// share references to it with multiple threads as nothing can be done.
> +unsafe impl<T: DriverOps> Sync for Registration<T> {}
> +
> +impl<T: DriverOps> Registration<T> {
> +    /// Creates a new instance of the registration object.
> +    pub fn new() -> Self {
> +        Self {
> +            is_registered: false,
> +            concrete_reg: UnsafeCell::new(T::RegType::default()),
> +        }
> +    }
> +
> +    /// Allocates a pinned registration object and registers it.
> +    ///
> +    /// Returns a pinned heap-allocated representation of the registration.
> +    pub fn new_pinned(name: &'static CStr, module: &'static ThisModule) -> Result<Pin<Box<Self>>> {
> +        let mut reg = Pin::from(Box::new(Self::new(), GFP_KERNEL)?);
> +        reg.as_mut().register(name, module)?;
> +        Ok(reg)
> +    }
> +
> +    /// Registers a driver with its subsystem.
> +    ///
> +    /// It must be pinned because the memory block that represents the registration is potentially
> +    /// self-referential.
> +    pub fn register(
> +        self: Pin<&mut Self>,
> +        name: &'static CStr,
> +        module: &'static ThisModule,
> +    ) -> Result {
> +        // SAFETY: We never move out of `this`.
> +        let this = unsafe { self.get_unchecked_mut() };
> +        if this.is_registered {
> +            // Already registered.
> +            return Err(EINVAL);
> +        }
> +
> +        // SAFETY: `concrete_reg` was initialised via its default constructor. It is only freed
> +        // after `Self::drop` is called, which first calls `T::unregister`.
> +        unsafe { T::register(this.concrete_reg.get(), name, module) }?;
> +
> +        this.is_registered = true;

Again, the driver core knows if it is registered or not, that's all that
matters, the rust side should never care.

> +        Ok(())
> +    }
> +}
> +
> +impl<T: DriverOps> Default for Registration<T> {
> +    fn default() -> Self {
> +        Self::new()
> +    }
> +}
> +
> +impl<T: DriverOps> Drop for Registration<T> {
> +    fn drop(&mut self) {
> +        if self.is_registered {
> +            // SAFETY: This path only runs if a previous call to `T::register` completed
> +            // successfully.
> +            unsafe { T::unregister(self.concrete_reg.get()) };

Can't the rust code ensure that this isn't run if register didn't
succeed?  Having a boolean feels really wrong here (can't that race?)

> +        }
> +    }
> +}
> +
> +/// Conversion from a device id to a raw device id.
> +///
> +/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
> +/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
> +///
> +/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
> +/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
> +/// concrete types (which can still have const associated functions) instead of a trait.
> +///
> +/// # Safety
> +///
> +/// Implementers must ensure that:
> +///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
> +///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
> +///     that buses can recover the pointer to the data.
> +pub unsafe trait RawDeviceId {
> +    /// The raw type that holds the device id.
> +    ///
> +    /// Id tables created from [`Self`] are going to hold this type in its zero-terminated array.
> +    type RawType: Copy;
> +
> +    /// A zeroed-out representation of the raw device id.
> +    ///
> +    /// Id tables created from [`Self`] use [`Self::ZERO`] as the sentinel to indicate the end of
> +    /// the table.
> +    const ZERO: Self::RawType;

All busses have their own way of creating "ids" and that is limited to
the bus code itself, why is any of this in the rust side?  What needs
this?  A bus will create the id for the devices it manages, and can use
it as part of the name it gives the device (but not required), so all of
this belongs to the bus, NOT to a driver, or a device.

> +}
> +
> +/// A zero-terminated device id array, followed by context data.

Why?  What is this for?

> +#[repr(C)]
> +pub struct IdArray<T: RawDeviceId, U, const N: usize> {
> +    ids: [T::RawType; N],
> +    sentinel: T::RawType,
> +    id_infos: [Option<U>; N],
> +}
> +
> +impl<T: RawDeviceId, U, const N: usize> IdArray<T, U, N> {
> +    const U_NONE: Option<U> = None;
> +
> +    /// Returns an `IdTable` backed by `self`.
> +    ///
> +    /// This is used to essentially erase the array size.
> +    pub const fn as_table(&self) -> IdTable<'_, T, U> {
> +        IdTable {
> +            first: &self.ids[0],
> +            _p: PhantomData,
> +        }
> +    }
> +
> +    /// Creates a new instance of the array.
> +    ///
> +    /// The contents are derived from the given identifiers and context information.
> +    #[doc(hidden)]
> +    pub const unsafe fn new(raw_ids: [T::RawType; N], infos: [Option<U>; N]) -> Self
> +    where
> +        T: RawDeviceId + Copy,
> +        T::RawType: Copy + Clone,
> +    {
> +        Self {
> +            ids: raw_ids,
> +            sentinel: T::ZERO,
> +            id_infos: infos,
> +        }
> +    }
> +
> +    #[doc(hidden)]
> +    pub const fn get_offset(idx: usize) -> isize
> +    where
> +        T: RawDeviceId + Copy,
> +        T::RawType: Copy + Clone,
> +    {
> +        // SAFETY: We are only using this dummy value to get offsets.
> +        let array = unsafe { Self::new([T::ZERO; N], [Self::U_NONE; N]) };
> +        // SAFETY: Both pointers are within `array` (or one byte beyond), consequently they are
> +        // derived from the same allocated object. We are using a `u8` pointer, whose size 1,
> +        // so the pointers are necessarily 1-byte aligned.
> +        let ret = unsafe {
> +            (&array.id_infos[idx] as *const _ as *const u8)
> +                .offset_from(&array.ids[idx] as *const _ as _)
> +        };
> +        core::mem::forget(array);
> +        ret
> +    }
> +}
> +
> +// Creates a new ID array. This is a macro so it can take as a parameter the concrete ID type in
> +// order to call to_rawid() on it, and still remain const. This is necessary until a new
> +// const_trait_impl implementation lands, since the existing implementation was removed in Rust
> +// 1.73.

Again, what are these id lists for?  Busses work on individual ids that
they create dynamically.

You aren't thinking this is an id that could be used to match devices to
drivers, are you?  That's VERY bus specific, and also specified already
in .c code for those busses and passed to userspace.  That doesn't
belong here.

If this isn't that list, what exactly is this?

> +#[macro_export]
> +#[doc(hidden)]
> +macro_rules! _new_id_array {
> +    (($($args:tt)*), $id_type:ty) => {{
> +        /// Creates a new instance of the array.
> +        ///
> +        /// The contents are derived from the given identifiers and context information.
> +        const fn new< U, const N: usize>(ids: [$id_type; N], infos: [Option<U>; N])
> +            -> $crate::driver::IdArray<$id_type, U, N>
> +        where
> +            $id_type: $crate::driver::RawDeviceId + Copy,
> +            <$id_type as $crate::driver::RawDeviceId>::RawType: Copy + Clone,
> +        {
> +            let mut raw_ids =
> +                [<$id_type as $crate::driver::RawDeviceId>::ZERO; N];
> +            let mut i = 0usize;
> +            while i < N {
> +                let offset: isize = $crate::driver::IdArray::<$id_type, U, N>::get_offset(i);
> +                raw_ids[i] = ids[i].to_rawid(offset);
> +                i += 1;
> +            }
> +
> +            // SAFETY: We are passing valid arguments computed with the correct offsets.
> +            unsafe {
> +                $crate::driver::IdArray::<$id_type, U, N>::new(raw_ids, infos)
> +            }
> +       }
> +
> +        new($($args)*)
> +    }}
> +}
> +
> +/// A device id table.
> +///
> +/// The table is guaranteed to be zero-terminated and to be followed by an array of context data of
> +/// type `Option<U>`.
> +pub struct IdTable<'a, T: RawDeviceId, U> {
> +    first: &'a T::RawType,
> +    _p: PhantomData<&'a U>,
> +}

All busses have different ways of matching drivers to devices, and they
might be called a "device id table" and it might not.  The driver core
doesn't care, and neither should this rust code.  That's all
bus-specific and unique to each and every one of them.  None of this
should be needed here at all.

> +
> +impl<T: RawDeviceId, U> AsRef<T::RawType> for IdTable<'_, T, U> {
> +    fn as_ref(&self) -> &T::RawType {
> +        self.first
> +    }
> +}
> +
> +/// Counts the number of parenthesis-delimited, comma-separated items.
> +///
> +/// # Examples
> +///
> +/// ```
> +/// # use kernel::count_paren_items;
> +///
> +/// assert_eq!(0, count_paren_items!());
> +/// assert_eq!(1, count_paren_items!((A)));
> +/// assert_eq!(1, count_paren_items!((A),));
> +/// assert_eq!(2, count_paren_items!((A), (B)));
> +/// assert_eq!(2, count_paren_items!((A), (B),));
> +/// assert_eq!(3, count_paren_items!((A), (B), (C)));
> +/// assert_eq!(3, count_paren_items!((A), (B), (C),));
> +/// ```
> +#[macro_export]
> +macro_rules! count_paren_items {
> +    (($($item:tt)*), $($remaining:tt)*) => { 1 + $crate::count_paren_items!($($remaining)*) };
> +    (($($item:tt)*)) => { 1 };
> +    () => { 0 };
> +}

Shouldn't this go in some common header somewhere?  Why is this here?

> +
> +/// Converts a comma-separated list of pairs into an array with the first element. That is, it
> +/// discards the second element of the pair.
> +///
> +/// Additionally, it automatically introduces a type if the first element is warpped in curly
> +/// braces, for example, if it's `{v: 10}`, it becomes `X { v: 10 }`; this is to avoid repeating
> +/// the type.
> +///
> +/// # Examples
> +///
> +/// ```
> +/// # use kernel::first_item;
> +///
> +/// #[derive(PartialEq, Debug)]
> +/// struct X {
> +///     v: u32,
> +/// }
> +///
> +/// assert_eq!([] as [X; 0], first_item!(X, ));
> +/// assert_eq!([X { v: 10 }], first_item!(X, ({ v: 10 }, Y)));
> +/// assert_eq!([X { v: 10 }], first_item!(X, ({ v: 10 }, Y),));
> +/// assert_eq!([X { v: 10 }], first_item!(X, (X { v: 10 }, Y)));
> +/// assert_eq!([X { v: 10 }], first_item!(X, (X { v: 10 }, Y),));
> +/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y)));
> +/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y),));
> +/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y)));
> +/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y),));
> +/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
> +///            first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y), ({v: 30}, Y)));
> +/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
> +///            first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y), ({v: 30}, Y),));
> +/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
> +///            first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y), (X {v: 30}, Y)));
> +/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
> +///            first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y), (X {v: 30}, Y),));
> +/// ```
> +#[macro_export]
> +macro_rules! first_item {
> +    ($id_type:ty, $(({$($first:tt)*}, $second:expr)),* $(,)?) => {
> +        {
> +            type IdType = $id_type;
> +            [$(IdType{$($first)*},)*]
> +        }
> +    };
> +    ($id_type:ty, $(($first:expr, $second:expr)),* $(,)?) => { [$($first,)*] };
> +}

Again, why here?

> +
> +/// Converts a comma-separated list of pairs into an array with the second element. That is, it
> +/// discards the first element of the pair.
> +///
> +/// # Examples
> +///
> +/// ```
> +/// # use kernel::second_item;
> +///
> +/// assert_eq!([] as [u32; 0], second_item!());
> +/// assert_eq!([10u32], second_item!((X, 10u32)));
> +/// assert_eq!([10u32], second_item!((X, 10u32),));
> +/// assert_eq!([10u32], second_item!(({ X }, 10u32)));
> +/// assert_eq!([10u32], second_item!(({ X }, 10u32),));
> +/// assert_eq!([10u32, 20], second_item!((X, 10u32), (X, 20)));
> +/// assert_eq!([10u32, 20], second_item!((X, 10u32), (X, 20),));
> +/// assert_eq!([10u32, 20], second_item!(({ X }, 10u32), ({ X }, 20)));
> +/// assert_eq!([10u32, 20], second_item!(({ X }, 10u32), ({ X }, 20),));
> +/// assert_eq!([10u32, 20, 30], second_item!((X, 10u32), (X, 20), (X, 30)));
> +/// assert_eq!([10u32, 20, 30], second_item!((X, 10u32), (X, 20), (X, 30),));
> +/// assert_eq!([10u32, 20, 30], second_item!(({ X }, 10u32), ({ X }, 20), ({ X }, 30)));
> +/// assert_eq!([10u32, 20, 30], second_item!(({ X }, 10u32), ({ X }, 20), ({ X }, 30),));
> +/// ```
> +#[macro_export]
> +macro_rules! second_item {
> +    ($(({$($first:tt)*}, $second:expr)),* $(,)?) => { [$($second,)*] };
> +    ($(($first:expr, $second:expr)),* $(,)?) => { [$($second,)*] };
> +}

Again, why here?

> +
> +/// Defines a new constant [`IdArray`] with a concise syntax.
> +///
> +/// It is meant to be used by buses and subsystems to create a similar macro with their device id
> +/// type already specified, i.e., with fewer parameters to the end user.
> +///
> +/// # Examples
> +///
> +// TODO: Exported but not usable by kernel modules (requires `const_trait_impl`).
> +/// ```ignore
> +/// #![feature(const_trait_impl)]
> +/// # use kernel::{define_id_array, driver::RawDeviceId};
> +///
> +/// #[derive(Copy, Clone)]
> +/// struct Id(u32);
> +///
> +/// // SAFETY: `ZERO` is all zeroes and `to_rawid` stores `offset` as the second element of the raw
> +/// // device id pair.
> +/// unsafe impl const RawDeviceId for Id {
> +///     type RawType = (u64, isize);
> +///     const ZERO: Self::RawType = (0, 0);
> +///     fn to_rawid(&self, offset: isize) -> Self::RawType {
> +///         (self.0 as u64 + 1, offset)
> +///     }
> +/// }
> +///
> +/// define_id_array!(A1, Id, (), []);
> +/// define_id_array!(A2, Id, &'static [u8], [(Id(10), None)]);
> +/// define_id_array!(A3, Id, &'static [u8], [(Id(10), Some(b"id1")), ]);
> +/// define_id_array!(A4, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2"))]);
> +/// define_id_array!(A5, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2")), ]);
> +/// define_id_array!(A6, Id, &'static [u8], [(Id(10), None), (Id(20), Some(b"id2")), ]);
> +/// define_id_array!(A7, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), None), ]);
> +/// define_id_array!(A8, Id, &'static [u8], [(Id(10), None), (Id(20), None), ]);
> +/// ```

Again, busses define this, put this in the bus-specific code that you
wish to define/match with, it does not belong here as every bus does it
differently.  Not all the world is PCI :)

> +#[macro_export]
> +macro_rules! define_id_array {
> +    ($table_name:ident, $id_type:ty, $data_type:ty, [ $($t:tt)* ]) => {
> +        const $table_name:
> +            $crate::driver::IdArray<$id_type, $data_type, { $crate::count_paren_items!($($t)*) }> =
> +                $crate::_new_id_array!((
> +                    $crate::first_item!($id_type, $($t)*), $crate::second_item!($($t)*)), $id_type);
> +    };
> +}
> +
> +/// Defines a new constant [`IdTable`] with a concise syntax.
> +///
> +/// It is meant to be used by buses and subsystems to create a similar macro with their device id
> +/// type already specified, i.e., with fewer parameters to the end user.
> +///
> +/// # Examples
> +///
> +// TODO: Exported but not usable by kernel modules (requires `const_trait_impl`).
> +/// ```ignore
> +/// #![feature(const_trait_impl)]
> +/// # use kernel::{define_id_table, driver::RawDeviceId};
> +///
> +/// #[derive(Copy, Clone)]
> +/// struct Id(u32);
> +///
> +/// // SAFETY: `ZERO` is all zeroes and `to_rawid` stores `offset` as the second element of the raw
> +/// // device id pair.
> +/// unsafe impl const RawDeviceId for Id {
> +///     type RawType = (u64, isize);
> +///     const ZERO: Self::RawType = (0, 0);
> +///     fn to_rawid(&self, offset: isize) -> Self::RawType {
> +///         (self.0 as u64 + 1, offset)
> +///     }
> +/// }
> +///
> +/// define_id_table!(T1, Id, &'static [u8], [(Id(10), None)]);
> +/// define_id_table!(T2, Id, &'static [u8], [(Id(10), Some(b"id1")), ]);
> +/// define_id_table!(T3, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2"))]);
> +/// define_id_table!(T4, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2")), ]);
> +/// define_id_table!(T5, Id, &'static [u8], [(Id(10), None), (Id(20), Some(b"id2")), ]);
> +/// define_id_table!(T6, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), None), ]);
> +/// define_id_table!(T7, Id, &'static [u8], [(Id(10), None), (Id(20), None), ]);
> +/// ```
> +#[macro_export]
> +macro_rules! define_id_table {
> +    ($table_name:ident, $id_type:ty, $data_type:ty, [ $($t:tt)* ]) => {
> +        const $table_name: Option<$crate::driver::IdTable<'static, $id_type, $data_type>> = {
> +            $crate::define_id_array!(ARRAY, $id_type, $data_type, [ $($t)* ]);
> +            Some(ARRAY.as_table())
> +        };
> +    };
> +}

Again, see above, does not belong here.

> +
> +/// Custom code within device removal.

You better define the heck out of "device removal" as specified last
time this all came up.  From what I can see here, this is totally wrong
and confusing and will be a mess.

Do it right, name it properly.

I'm not reviewingn beyond here, sorry.  It's the merge window and I
shouldn't have even looked at this until next week anyway.

But I was hoping that the whole long rant I gave last time would be
addressed at least a little bit.  I don't see that it has :(


> +pub trait DeviceRemoval {
> +    /// Cleans resources up when the device is removed.
> +    ///
> +    /// This is called when a device is removed and offers implementers the chance to run some code
> +    /// that cleans state up.
> +    fn device_remove(&self);
> +}
> +
> +impl DeviceRemoval for () {
> +    fn device_remove(&self) {}
> +}
> +
> +impl<T: DeviceRemoval> DeviceRemoval for Arc<T> {
> +    fn device_remove(&self) {
> +        self.deref().device_remove();
> +    }
> +}
> +
> +impl<T: DeviceRemoval> DeviceRemoval for Box<T> {
> +    fn device_remove(&self) {
> +        self.deref().device_remove();
> +    }
> +}
> +
> +/// A kernel module that only registers the given driver on init.
> +///
> +/// This is a helper struct to make it easier to define single-functionality modules, in this case,
> +/// modules that offer a single driver.
> +pub struct Module<T: DriverOps> {
> +    _driver: Pin<Box<Registration<T>>>,
> +}
> +
> +impl<T: DriverOps> crate::Module for Module<T> {
> +    fn init(name: &'static CStr, module: &'static ThisModule) -> Result<Self> {
> +        Ok(Self {
> +            _driver: Registration::new_pinned(name, module)?,
> +        })
> +    }
> +}
> +
> +/// Declares a kernel module that exposes a single driver.

There is no requirement that a kernel module only exposes a single
driver, so what exactly is this used for?  Is this some attempt to make
the module_driver() macro in rust?  Why?  Only add something like that
if you really need it (i.e. you have a bus in rust and you want the
benifit of having that macro to save you some lines of code.)  I don't
see that happening here...

greg k-h

Re: [RFC PATCH 00/11] [RFC] Device / Driver and PCI Rust abstractions

[Greg KH]

On Mon, May 20, 2024 at 07:25:37PM +0200, Danilo Krummrich wrote:
> This patch sereis implements basic generic device / driver Rust abstractions,
> as well as some basic PCI abstractions.
> 
> This patch series is sent in the context of [1], and the corresponding patch
> series [2], which contains some basic DRM Rust abstractions and a stub
> implementation of the Nova GPU driver.
> 
> Nova is intended to be developed upstream, starting out with just a stub driver
> to lift some initial required infrastructure upstream. A more detailed
> explanation can be found in [1].
> 
> Some patches, which implement the generic device / driver Rust abstractions have
> been sent a couple of weeks ago already [3]. For those patches the following
> changes have been made since then:
> 
> - remove RawDevice::name()
> - remove rust helper for dev_name() and dev_get_drvdata()
> - use AlwaysRefCounted for struct Device
> - drop trait RawDevice entirely in favor of AsRef and provide
>   Device::from_raw(), Device::as_raw() and Device::as_ref() instead
> - implement RevocableGuard
> - device::Data, remove resources and replace it with a Devres abstraction
> - implement Devres abstraction for resources
> 
> As mentioned above, a driver serving as example on how these abstractions are
> used within a (DRM) driver can be found in [2].
> 
> Additionally, the device / driver bits can also be found in [3], all
> abstractions required for Nova in [4] and Nova in [5].
> 
> [1] https://lore.kernel.org/dri-devel/Zfsj0_tb-0-tNrJy@cassiopeiae/T/#u
> [2] https://lore.kernel.org/dri-devel/20240520172059.181256-1-dakr@redhat.com/
> [3] https://github.com/Rust-for-Linux/linux/tree/staging/rust-device
> [4] https://github.com/Rust-for-Linux/linux/tree/staging/dev
> [5] https://gitlab.freedesktop.org/drm/nova/-/tree/nova-next
> 
> Danilo Krummrich (2):
>   rust: add abstraction for struct device
>   rust: add devres abstraction
> 
> FUJITA Tomonori (1):
>   rust: add basic PCI driver abstractions
> 
> Philipp Stanner (2):
>   rust: add basic abstractions for iomem operations
>   rust: PCI: add BAR request and ioremap
> 
> Wedson Almeida Filho (6):
>   rust: add driver abstraction
>   rust: add rcu abstraction
>   rust: add revocable mutex
>   rust: add revocable objects
>   rust: add device::Data
>   rust: add `dev_*` print macros.

No list of the changes made since the last time this was submitted?

No versioning of this submission?

Why not?

thanks,

greg k-h

Re: [RFC PATCH 00/11] [RFC] Device / Driver and PCI Rust abstractions

[Greg KH]

On Mon, May 20, 2024 at 08:14:57PM +0200, Greg KH wrote:
> On Mon, May 20, 2024 at 07:25:37PM +0200, Danilo Krummrich wrote:
> > This patch sereis implements basic generic device / driver Rust abstractions,
> > as well as some basic PCI abstractions.
> > 
> > This patch series is sent in the context of [1], and the corresponding patch
> > series [2], which contains some basic DRM Rust abstractions and a stub
> > implementation of the Nova GPU driver.
> > 
> > Nova is intended to be developed upstream, starting out with just a stub driver
> > to lift some initial required infrastructure upstream. A more detailed
> > explanation can be found in [1].
> > 
> > Some patches, which implement the generic device / driver Rust abstractions have
> > been sent a couple of weeks ago already [3]. For those patches the following
> > changes have been made since then:
> > 
> > - remove RawDevice::name()
> > - remove rust helper for dev_name() and dev_get_drvdata()
> > - use AlwaysRefCounted for struct Device
> > - drop trait RawDevice entirely in favor of AsRef and provide
> >   Device::from_raw(), Device::as_raw() and Device::as_ref() instead
> > - implement RevocableGuard
> > - device::Data, remove resources and replace it with a Devres abstraction
> > - implement Devres abstraction for resources

Ah, here's the difference from the last time, sorry, it wasn't obvious.

Still nothing about proper handling and use of 'remove' in the context
of all of this, that's something you really really really need to get
right if you want to attempt to have a driver in rust interact with the
driver core properly.

thanks,

greg k-h

Re: [RFC PATCH 01/11] rust: add abstraction for struct device

[Miguel Ojeda]

On Mon, May 20, 2024 at 8:00 PM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> What's the status of moving .rs files next to their respective .c files
> in the build system?  Keeping them separate like this just isn't going
> to work, sorry.

In progress.

> > +//! Generic devices that are part of the kernel's driver model.
> > +//!
> > +//! C header: [`include/linux/device.h`](../../../../include/linux/device.h)
>
> relative paths for a common "include <linux/device.h" type of thing?
> Rust can't handle include paths from directories?

We have the custom `srctree/` notation for that (which the patch should use).

And eventually we should have support for easy-to-use links to C
structs and so on (i.e. to the C docs) -- today I happened to be
talking to the `rustdoc` maintainers about this old idea of ours.

Cheers,
Miguel

Re: [RFC PATCH 00/11] [RFC] Device / Driver and PCI Rust abstractions

[Danilo Krummrich]

On Mon, May 20, 2024 at 08:16:19PM +0200, Greg KH wrote:
> On Mon, May 20, 2024 at 08:14:57PM +0200, Greg KH wrote:
> > On Mon, May 20, 2024 at 07:25:37PM +0200, Danilo Krummrich wrote:
> > > This patch sereis implements basic generic device / driver Rust abstractions,
> > > as well as some basic PCI abstractions.
> > > 
> > > This patch series is sent in the context of [1], and the corresponding patch
> > > series [2], which contains some basic DRM Rust abstractions and a stub
> > > implementation of the Nova GPU driver.
> > > 
> > > Nova is intended to be developed upstream, starting out with just a stub driver
> > > to lift some initial required infrastructure upstream. A more detailed
> > > explanation can be found in [1].
> > > 
> > > Some patches, which implement the generic device / driver Rust abstractions have
> > > been sent a couple of weeks ago already [3]. For those patches the following
> > > changes have been made since then:
> > > 
> > > - remove RawDevice::name()
> > > - remove rust helper for dev_name() and dev_get_drvdata()
> > > - use AlwaysRefCounted for struct Device
> > > - drop trait RawDevice entirely in favor of AsRef and provide
> > >   Device::from_raw(), Device::as_raw() and Device::as_ref() instead
> > > - implement RevocableGuard
> > > - device::Data, remove resources and replace it with a Devres abstraction
> > > - implement Devres abstraction for resources
> 
> Ah, here's the difference from the last time, sorry, it wasn't obvious.
> 
> Still nothing about proper handling and use of 'remove' in the context
> of all of this, that's something you really really really need to get
> right if you want to attempt to have a driver in rust interact with the
> driver core properly.

We were right in the middle of discussing about the correct wording when I sent
those patches the first time. There were some replies from my side, e.g. [1] and
another reply from Wedson [2] about this, which you did not want to reply to any
more.

I'm not saying I insist on not changing those comments up, but first we have to
agree on how we want them to be rephrased, especially since from the
discussions so far I got the impression that we might talk a bit past each
other.

Hence, I'd propose to just continue the discussion, where we need to.

[1] https://lore.kernel.org/rust-for-linux/ZgNCo4Hvs-EjDNGT@pollux/
[2] https://lore.kernel.org/rust-for-linux/CANeycqrdBVJhAiGZdv2NSnz2SUCuDC=04D_JiTF6vfs8iLvjrA@mail.gmail.com/

> 
> thanks,
> 
> greg k-h
> 

Re: [RFC PATCH 01/11] rust: add abstraction for struct device

[Danilo Krummrich]

On Mon, May 20, 2024 at 08:00:23PM +0200, Greg KH wrote:
> On Mon, May 20, 2024 at 07:25:38PM +0200, Danilo Krummrich wrote:
> > Add an (always) reference counted abstraction for a generic struct
> > device. This abstraction encapsulates existing struct device instances
> > and manages its reference count.
> > 
> > Subsystems may use this abstraction as a base to abstract subsystem
> > specific device instances based on a generic struct device.
> > 
> > Co-developed-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > ---
> >  rust/helpers.c        |  1 +
> >  rust/kernel/device.rs | 76 +++++++++++++++++++++++++++++++++++++++++++
> 
> What's the status of moving .rs files next to their respective .c files
> in the build system?  Keeping them separate like this just isn't going
> to work, sorry.
> 
> > --- /dev/null
> > +++ b/rust/kernel/device.rs
> > @@ -0,0 +1,76 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +//! Generic devices that are part of the kernel's driver model.
> > +//!
> > +//! C header: [`include/linux/device.h`](../../../../include/linux/device.h)
> 
> relative paths for a common "include <linux/device.h" type of thing?
> Rust can't handle include paths from directories?

Going to change this to `srctree/` as proposed by Miguel.

> 
> > +
> > +use crate::{
> > +    bindings,
> > +    types::{ARef, Opaque},
> > +};
> > +use core::ptr;
> > +
> > +/// A ref-counted device.
> > +///
> > +/// # Invariants
> > +///
> > +/// The pointer stored in `Self` is non-null and valid for the lifetime of the ARef instance. In
> > +/// particular, the ARef instance owns an increment on underlying object’s reference count.
> > +#[repr(transparent)]
> > +pub struct Device(Opaque<bindings::device>);
> > +
> > +impl Device {
> > +    /// Creates a new ref-counted instance of an existing device pointer.
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// Callers must ensure that `ptr` is valid, non-null, and has a non-zero reference count.
> 
> Callers NEVER care about the reference count of a struct device, anyone
> poking in that is asking for trouble.

That's confusing, if not the caller who's passing the device pointer somewhere,
who else?

Who takes care that a device' reference count is non-zero when a driver's probe
function is called?

It's the same here. The PCI code calls Device::from_raw() from its
probe_callback() function, which is called from the C side. For instance:

extern "C" fn probe_callback(
   pdev: *mut bindings::pci_dev,
   id: *const bindings::pci_device_id,
) -> core::ffi::c_int {
   // SAFETY: This is safe, since the C side guarantees that pdev is a valid,
   // non-null pointer to a struct pci_dev with a non-zero reference count.
   let dev = unsafe { device::Device::from_raw(&mut (*pdev).dev) };

   [...]
}

> 
> And why non-NULL?  Can't you check for that here?  Shouldn't you check
> for that here?  Many driver core functions can handle a NULL pointer
> just fine (i.e. get/put_device() can), why should Rust code assume that
> a pointer passed to it from the C layer is going to have stricter rules
> than the C layer can provide?

We could check for NULL here, but I think it'd be pointless. Even if the pointer
is not NULL, it can still be an invalid one. There is no check we can do to
guarantee safety, hence the function is and remains unsafe and has safety
requirements instead that the caller must guarantee to fulfil.

Like in the example above, probe_callback() can give those guarantees instead.

> 
> > +    pub unsafe fn from_raw(ptr: *mut bindings::device) -> ARef<Self> {
> > +        // SAFETY: By the safety requirements, ptr is valid.
> > +        // Initially increase the reference count by one to compensate for the final decrement once
> > +        // this newly created `ARef<Device>` instance is dropped.
> > +        unsafe { bindings::get_device(ptr) };
> > +
> > +        // CAST: `Self` is a `repr(transparent)` wrapper around `bindings::device`.
> > +        let ptr = ptr.cast::<Self>();
> > +
> > +        // SAFETY: By the safety requirements, ptr is valid.
> > +        unsafe { ARef::from_raw(ptr::NonNull::new_unchecked(ptr)) }
> > +    }
> > +
> > +    /// Obtain the raw `struct device *`.
> > +    pub(crate) fn as_raw(&self) -> *mut bindings::device {
> > +        self.0.get()
> > +    }
> > +
> > +    /// Convert a raw `struct device` pointer to a `&Device`.
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// Callers must ensure that `ptr` is valid, non-null, and has a non-zero reference count for
> > +    /// the entire duration when the returned reference exists.
> 

This is the doc comment of pub unsafe fn as_ref<'a>(ptr: *mut bindings::device)
-> &'a Self. Let's keep this context, it's confusing for other readers
otherwise.

> Again, non-NULL might not be true, and reference counts are never

Like above, it's just the safety precondition. Checking for NULL does not
improve the situation, we still need to rely on the pointer being a valid one.

> tracked by any user EXCEPT to increment/decrement it, you never know if

That's the whole point, if one takes an increment on the reference count they
can guarantee it's non-zero.

> it is 0 or not, all you know is that if a pointer is given to you by the
> driver core to a 'struct device' for a function that it is a valid
> reference at that point in time, or maybe NULL, until your function
> returns.  Anything after that can not be counted on.

That's not contradictive to those safety comments. When Device::from_raw()
returns it has increased the reference count of the device by one. And when
Device::as_ref() returns the returned reference' lifetime is bound to the
lifetime of the pointer passed to it.

> 
> thanks,
> 
> greg k-h
> 

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Danilo Krummrich]

On Mon, May 20, 2024 at 08:14:18PM +0200, Greg KH wrote:
> On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> > From: Wedson Almeida Filho <wedsonaf@gmail.com>
> > 
> > This defines general functionality related to registering drivers with
> > their respective subsystems, and registering modules that implement
> > drivers.
> > 
> > Co-developed-by: Asahi Lina <lina@asahilina.net>
> > Signed-off-by: Asahi Lina <lina@asahilina.net>
> > Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> > Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > ---
> >  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
> >  rust/kernel/lib.rs           |   4 +-
> >  rust/macros/module.rs        |   2 +-
> >  samples/rust/rust_minimal.rs |   2 +-
> >  samples/rust/rust_print.rs   |   2 +-
> >  5 files changed, 498 insertions(+), 4 deletions(-)
> >  create mode 100644 rust/kernel/driver.rs
> > 
> > diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> > new file mode 100644
> > index 000000000000..e0cfc36d47ff
> > --- /dev/null
> > +++ b/rust/kernel/driver.rs
> > @@ -0,0 +1,492 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> > +//!
> > +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> > +//! using the [`Registration`] class.
> 
> Why are you creating new "names" here?  "DriverOps" is part of a 'struct
> device_driver' why are you separating it out here?  And what is

DriverOps is a trait which abstracts a subsystems register() and unregister()
functions to (un)register drivers. It exists such that a generic Registration
implementation calls the correct one for the subsystem.

For instance, PCI would implement DriverOps::register() by calling into
bindings::__pci_register_driver().

We can discuss whether DriverOps is a good name for the trait, but it's not a
(different) name for something that already exists and already has a name.

> 'Registration'?  That's a bus/class thing, not a driver thing.

A Registration is an object representation of the driver's registered state.
Having an object representation for that is basically the Rust way to manage the
lifetime of this state. Once the Registration is dropped, the driver is
unregistered. For instance, a PCI driver Registration can be bound to a module,
such that the PCI driver is unregistered when the module is unloaded (the
Registration is dropped in the module_exit() callback).

> 
> And be very careful of the use of the word 'class' here, remember, there
> is 'struct class' as part of the driver model :)

I think in this context "class" might be meant as something like "struct with
methods", not sure what would be the best term to describe this.

> 
> > +use crate::{
> > +    alloc::{box_ext::BoxExt, flags::*},
> > +    error::code::*,
> > +    error::Result,
> > +    str::CStr,
> > +    sync::Arc,
> > +    ThisModule,
> > +};
> > +use alloc::boxed::Box;
> > +use core::{cell::UnsafeCell, marker::PhantomData, ops::Deref, pin::Pin};
> > +
> > +/// A subsystem (e.g., PCI, Platform, Amba, etc.) that allows drivers to be written for it.
> > +pub trait DriverOps {
> 
> Again, why is this not called DeviceDriver?

See above.

> 
> > +    /// The type that holds information about the registration. This is typically a struct defined
> > +    /// by the C portion of the kernel.
> > +    type RegType: Default;
> > +
> > +    /// Registers a driver.
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// `reg` must point to valid, initialised, and writable memory. It may be modified by this
> > +    /// function to hold registration state.
> > +    ///
> > +    /// On success, `reg` must remain pinned and valid until the matching call to
> > +    /// [`DriverOps::unregister`].
> > +    unsafe fn register(
> > +        reg: *mut Self::RegType,
> > +        name: &'static CStr,
> > +        module: &'static ThisModule,
> > +    ) -> Result;
> > +
> > +    /// Unregisters a driver previously registered with [`DriverOps::register`].
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// `reg` must point to valid writable memory, initialised by a previous successful call to
> > +    /// [`DriverOps::register`].
> > +    unsafe fn unregister(reg: *mut Self::RegType);
> > +}
> > +
> > +/// The registration of a driver.
> > +pub struct Registration<T: DriverOps> {
> > +    is_registered: bool,
> 
> Why does a driver need to know if it is registered or not?  Only the
> driver core cares about that, please do not expose that, it's racy and
> should not be relied on.

We need it to ensure we do not try to register the same thing twice, some
subsystems might just catch fire otherwise.

Another reason is the one you mention below.

> 
> > +    concrete_reg: UnsafeCell<T::RegType>,
> > +}
> > +
> > +// SAFETY: `Registration` has no fields or methods accessible via `&Registration`, so it is safe to
> > +// share references to it with multiple threads as nothing can be done.
> > +unsafe impl<T: DriverOps> Sync for Registration<T> {}
> > +
> > +impl<T: DriverOps> Registration<T> {
> > +    /// Creates a new instance of the registration object.
> > +    pub fn new() -> Self {
> > +        Self {
> > +            is_registered: false,
> > +            concrete_reg: UnsafeCell::new(T::RegType::default()),
> > +        }
> > +    }
> > +
> > +    /// Allocates a pinned registration object and registers it.
> > +    ///
> > +    /// Returns a pinned heap-allocated representation of the registration.
> > +    pub fn new_pinned(name: &'static CStr, module: &'static ThisModule) -> Result<Pin<Box<Self>>> {
> > +        let mut reg = Pin::from(Box::new(Self::new(), GFP_KERNEL)?);
> > +        reg.as_mut().register(name, module)?;
> > +        Ok(reg)
> > +    }
> > +
> > +    /// Registers a driver with its subsystem.
> > +    ///
> > +    /// It must be pinned because the memory block that represents the registration is potentially
> > +    /// self-referential.
> > +    pub fn register(
> > +        self: Pin<&mut Self>,
> > +        name: &'static CStr,
> > +        module: &'static ThisModule,
> > +    ) -> Result {
> > +        // SAFETY: We never move out of `this`.
> > +        let this = unsafe { self.get_unchecked_mut() };
> > +        if this.is_registered {
> > +            // Already registered.
> > +            return Err(EINVAL);
> > +        }
> > +
> > +        // SAFETY: `concrete_reg` was initialised via its default constructor. It is only freed
> > +        // after `Self::drop` is called, which first calls `T::unregister`.
> > +        unsafe { T::register(this.concrete_reg.get(), name, module) }?;
> > +
> > +        this.is_registered = true;
> 
> Again, the driver core knows if it is registered or not, that's all that
> matters, the rust side should never care.
> 
> > +        Ok(())
> > +    }
> > +}
> > +
> > +impl<T: DriverOps> Default for Registration<T> {
> > +    fn default() -> Self {
> > +        Self::new()
> > +    }
> > +}
> > +
> > +impl<T: DriverOps> Drop for Registration<T> {
> > +    fn drop(&mut self) {
> > +        if self.is_registered {
> > +            // SAFETY: This path only runs if a previous call to `T::register` completed
> > +            // successfully.
> > +            unsafe { T::unregister(self.concrete_reg.get()) };
> 
> Can't the rust code ensure that this isn't run if register didn't
> succeed?  Having a boolean feels really wrong here (can't that race?)

No, we want to automatically unregister once this object is destroyed, but not
if it was never registered in the first place.

This shouldn't be racy, we only ever (un)register things in places like probe()
/ remove() or module_init() / module_exit().

> 
> > +        }
> > +    }
> > +}
> > +
> > +/// Conversion from a device id to a raw device id.
> > +///
> > +/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
> > +/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
> > +///
> > +/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
> > +/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
> > +/// concrete types (which can still have const associated functions) instead of a trait.
> > +///
> > +/// # Safety
> > +///
> > +/// Implementers must ensure that:
> > +///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
> > +///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
> > +///     that buses can recover the pointer to the data.
> > +pub unsafe trait RawDeviceId {
> > +    /// The raw type that holds the device id.
> > +    ///
> > +    /// Id tables created from [`Self`] are going to hold this type in its zero-terminated array.
> > +    type RawType: Copy;
> > +
> > +    /// A zeroed-out representation of the raw device id.
> > +    ///
> > +    /// Id tables created from [`Self`] use [`Self::ZERO`] as the sentinel to indicate the end of
> > +    /// the table.
> > +    const ZERO: Self::RawType;
> 
> All busses have their own way of creating "ids" and that is limited to
> the bus code itself, why is any of this in the rust side?  What needs
> this?  A bus will create the id for the devices it manages, and can use
> it as part of the name it gives the device (but not required), so all of
> this belongs to the bus, NOT to a driver, or a device.

This is just a generalized interface which can be used by subsystems to
implement the subsystem / bus specific ID type.

> 
> > +}
> > +
> > +/// A zero-terminated device id array, followed by context data.
> 
> Why?  What is this for?

An array of generic ID types implementing the RawDeviceId interface.

It's used as generic implementation for the pattern of having some (generic) ID
type that ends up in an array with some sentinel marking the end of it.

> > +#[repr(C)]
> > +pub struct IdArray<T: RawDeviceId, U, const N: usize> {
> > +    ids: [T::RawType; N],
> > +    sentinel: T::RawType,
> > +    id_infos: [Option<U>; N],
> > +}
> > +
> > +impl<T: RawDeviceId, U, const N: usize> IdArray<T, U, N> {
> > +    const U_NONE: Option<U> = None;
> > +
> > +    /// Returns an `IdTable` backed by `self`.
> > +    ///
> > +    /// This is used to essentially erase the array size.
> > +    pub const fn as_table(&self) -> IdTable<'_, T, U> {
> > +        IdTable {
> > +            first: &self.ids[0],
> > +            _p: PhantomData,
> > +        }
> > +    }
> > +
> > +    /// Creates a new instance of the array.
> > +    ///
> > +    /// The contents are derived from the given identifiers and context information.
> > +    #[doc(hidden)]
> > +    pub const unsafe fn new(raw_ids: [T::RawType; N], infos: [Option<U>; N]) -> Self
> > +    where
> > +        T: RawDeviceId + Copy,
> > +        T::RawType: Copy + Clone,
> > +    {
> > +        Self {
> > +            ids: raw_ids,
> > +            sentinel: T::ZERO,
> > +            id_infos: infos,
> > +        }
> > +    }
> > +
> > +    #[doc(hidden)]
> > +    pub const fn get_offset(idx: usize) -> isize
> > +    where
> > +        T: RawDeviceId + Copy,
> > +        T::RawType: Copy + Clone,
> > +    {
> > +        // SAFETY: We are only using this dummy value to get offsets.
> > +        let array = unsafe { Self::new([T::ZERO; N], [Self::U_NONE; N]) };
> > +        // SAFETY: Both pointers are within `array` (or one byte beyond), consequently they are
> > +        // derived from the same allocated object. We are using a `u8` pointer, whose size 1,
> > +        // so the pointers are necessarily 1-byte aligned.
> > +        let ret = unsafe {
> > +            (&array.id_infos[idx] as *const _ as *const u8)
> > +                .offset_from(&array.ids[idx] as *const _ as _)
> > +        };
> > +        core::mem::forget(array);
> > +        ret
> > +    }
> > +}
> > +
> > +// Creates a new ID array. This is a macro so it can take as a parameter the concrete ID type in
> > +// order to call to_rawid() on it, and still remain const. This is necessary until a new
> > +// const_trait_impl implementation lands, since the existing implementation was removed in Rust
> > +// 1.73.
> 
> Again, what are these id lists for?  Busses work on individual ids that
> they create dynamically.
> 
> You aren't thinking this is an id that could be used to match devices to
> drivers, are you?  That's VERY bus specific, and also specified already
> in .c code for those busses and passed to userspace.  That doesn't
> belong here.

It is indeed a *generic* representation of IDs to match drivers to a device.
It's up the the corresponding subsystem to fill it with an ID type that matches
the subsystem.

The reason we need this in Rust is that also Rust drivers need to provide those
IDs on registration with the subsystem. This is just the abstraction for that.

In this patch series this is implemented for PCI [1], but, for instance, it'd also
work for OF [1] with 'compatible' strings.

[1] https://lore.kernel.org/rust-for-linux/20240520172554.182094-10-dakr@redhat.com/
[2] https://github.com/Rust-for-Linux/linux/blob/rust/rust/kernel/of.rs

> 
> If this isn't that list, what exactly is this?
> 
> > +#[macro_export]
> > +#[doc(hidden)]
> > +macro_rules! _new_id_array {
> > +    (($($args:tt)*), $id_type:ty) => {{
> > +        /// Creates a new instance of the array.
> > +        ///
> > +        /// The contents are derived from the given identifiers and context information.
> > +        const fn new< U, const N: usize>(ids: [$id_type; N], infos: [Option<U>; N])
> > +            -> $crate::driver::IdArray<$id_type, U, N>
> > +        where
> > +            $id_type: $crate::driver::RawDeviceId + Copy,
> > +            <$id_type as $crate::driver::RawDeviceId>::RawType: Copy + Clone,
> > +        {
> > +            let mut raw_ids =
> > +                [<$id_type as $crate::driver::RawDeviceId>::ZERO; N];
> > +            let mut i = 0usize;
> > +            while i < N {
> > +                let offset: isize = $crate::driver::IdArray::<$id_type, U, N>::get_offset(i);
> > +                raw_ids[i] = ids[i].to_rawid(offset);
> > +                i += 1;
> > +            }
> > +
> > +            // SAFETY: We are passing valid arguments computed with the correct offsets.
> > +            unsafe {
> > +                $crate::driver::IdArray::<$id_type, U, N>::new(raw_ids, infos)
> > +            }
> > +       }
> > +
> > +        new($($args)*)
> > +    }}
> > +}
> > +
> > +/// A device id table.
> > +///
> > +/// The table is guaranteed to be zero-terminated and to be followed by an array of context data of
> > +/// type `Option<U>`.
> > +pub struct IdTable<'a, T: RawDeviceId, U> {
> > +    first: &'a T::RawType,
> > +    _p: PhantomData<&'a U>,
> > +}
> 
> All busses have different ways of matching drivers to devices, and they
> might be called a "device id table" and it might not.  The driver core
> doesn't care, and neither should this rust code.  That's all
> bus-specific and unique to each and every one of them.  None of this
> should be needed here at all.

Again, this is just a generic abstraction that needs to be filled with
subsystem / bus specific ID types by the subsystem code. It's also not matching
anything on the Rust side. It's just a helper to let drivers provide IDs codes /
types that can be passed to the C side subsystem to do the match.

> 
> > +
> > +impl<T: RawDeviceId, U> AsRef<T::RawType> for IdTable<'_, T, U> {
> > +    fn as_ref(&self) -> &T::RawType {
> > +        self.first
> > +    }
> > +}
> > +
> > +/// Counts the number of parenthesis-delimited, comma-separated items.
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// # use kernel::count_paren_items;
> > +///
> > +/// assert_eq!(0, count_paren_items!());
> > +/// assert_eq!(1, count_paren_items!((A)));
> > +/// assert_eq!(1, count_paren_items!((A),));
> > +/// assert_eq!(2, count_paren_items!((A), (B)));
> > +/// assert_eq!(2, count_paren_items!((A), (B),));
> > +/// assert_eq!(3, count_paren_items!((A), (B), (C)));
> > +/// assert_eq!(3, count_paren_items!((A), (B), (C),));
> > +/// ```
> > +#[macro_export]
> > +macro_rules! count_paren_items {
> > +    (($($item:tt)*), $($remaining:tt)*) => { 1 + $crate::count_paren_items!($($remaining)*) };
> > +    (($($item:tt)*)) => { 1 };
> > +    () => { 0 };
> > +}
> 
> Shouldn't this go in some common header somewhere?  Why is this here?

There are no headers in Rust. This seems to be the correct place for this macro
(and the ones below).

> 
> > +
> > +/// Converts a comma-separated list of pairs into an array with the first element. That is, it
> > +/// discards the second element of the pair.
> > +///
> > +/// Additionally, it automatically introduces a type if the first element is warpped in curly
> > +/// braces, for example, if it's `{v: 10}`, it becomes `X { v: 10 }`; this is to avoid repeating
> > +/// the type.
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// # use kernel::first_item;
> > +///
> > +/// #[derive(PartialEq, Debug)]
> > +/// struct X {
> > +///     v: u32,
> > +/// }
> > +///
> > +/// assert_eq!([] as [X; 0], first_item!(X, ));
> > +/// assert_eq!([X { v: 10 }], first_item!(X, ({ v: 10 }, Y)));
> > +/// assert_eq!([X { v: 10 }], first_item!(X, ({ v: 10 }, Y),));
> > +/// assert_eq!([X { v: 10 }], first_item!(X, (X { v: 10 }, Y)));
> > +/// assert_eq!([X { v: 10 }], first_item!(X, (X { v: 10 }, Y),));
> > +/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y)));
> > +/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y),));
> > +/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y)));
> > +/// assert_eq!([X { v: 10 }, X { v: 20 }], first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y),));
> > +/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
> > +///            first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y), ({v: 30}, Y)));
> > +/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
> > +///            first_item!(X, ({ v: 10 }, Y), ({ v: 20 }, Y), ({v: 30}, Y),));
> > +/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
> > +///            first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y), (X {v: 30}, Y)));
> > +/// assert_eq!([X { v: 10 }, X { v: 20 }, X { v: 30 }],
> > +///            first_item!(X, (X { v: 10 }, Y), (X { v: 20 }, Y), (X {v: 30}, Y),));
> > +/// ```
> > +#[macro_export]
> > +macro_rules! first_item {
> > +    ($id_type:ty, $(({$($first:tt)*}, $second:expr)),* $(,)?) => {
> > +        {
> > +            type IdType = $id_type;
> > +            [$(IdType{$($first)*},)*]
> > +        }
> > +    };
> > +    ($id_type:ty, $(($first:expr, $second:expr)),* $(,)?) => { [$($first,)*] };
> > +}
> 
> Again, why here?
> 
> > +
> > +/// Converts a comma-separated list of pairs into an array with the second element. That is, it
> > +/// discards the first element of the pair.
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// # use kernel::second_item;
> > +///
> > +/// assert_eq!([] as [u32; 0], second_item!());
> > +/// assert_eq!([10u32], second_item!((X, 10u32)));
> > +/// assert_eq!([10u32], second_item!((X, 10u32),));
> > +/// assert_eq!([10u32], second_item!(({ X }, 10u32)));
> > +/// assert_eq!([10u32], second_item!(({ X }, 10u32),));
> > +/// assert_eq!([10u32, 20], second_item!((X, 10u32), (X, 20)));
> > +/// assert_eq!([10u32, 20], second_item!((X, 10u32), (X, 20),));
> > +/// assert_eq!([10u32, 20], second_item!(({ X }, 10u32), ({ X }, 20)));
> > +/// assert_eq!([10u32, 20], second_item!(({ X }, 10u32), ({ X }, 20),));
> > +/// assert_eq!([10u32, 20, 30], second_item!((X, 10u32), (X, 20), (X, 30)));
> > +/// assert_eq!([10u32, 20, 30], second_item!((X, 10u32), (X, 20), (X, 30),));
> > +/// assert_eq!([10u32, 20, 30], second_item!(({ X }, 10u32), ({ X }, 20), ({ X }, 30)));
> > +/// assert_eq!([10u32, 20, 30], second_item!(({ X }, 10u32), ({ X }, 20), ({ X }, 30),));
> > +/// ```
> > +#[macro_export]
> > +macro_rules! second_item {
> > +    ($(({$($first:tt)*}, $second:expr)),* $(,)?) => { [$($second,)*] };
> > +    ($(($first:expr, $second:expr)),* $(,)?) => { [$($second,)*] };
> > +}
> 
> Again, why here?
> 
> > +
> > +/// Defines a new constant [`IdArray`] with a concise syntax.
> > +///
> > +/// It is meant to be used by buses and subsystems to create a similar macro with their device id
> > +/// type already specified, i.e., with fewer parameters to the end user.
> > +///
> > +/// # Examples
> > +///
> > +// TODO: Exported but not usable by kernel modules (requires `const_trait_impl`).
> > +/// ```ignore
> > +/// #![feature(const_trait_impl)]
> > +/// # use kernel::{define_id_array, driver::RawDeviceId};
> > +///
> > +/// #[derive(Copy, Clone)]
> > +/// struct Id(u32);
> > +///
> > +/// // SAFETY: `ZERO` is all zeroes and `to_rawid` stores `offset` as the second element of the raw
> > +/// // device id pair.
> > +/// unsafe impl const RawDeviceId for Id {
> > +///     type RawType = (u64, isize);
> > +///     const ZERO: Self::RawType = (0, 0);
> > +///     fn to_rawid(&self, offset: isize) -> Self::RawType {
> > +///         (self.0 as u64 + 1, offset)
> > +///     }
> > +/// }
> > +///
> > +/// define_id_array!(A1, Id, (), []);
> > +/// define_id_array!(A2, Id, &'static [u8], [(Id(10), None)]);
> > +/// define_id_array!(A3, Id, &'static [u8], [(Id(10), Some(b"id1")), ]);
> > +/// define_id_array!(A4, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2"))]);
> > +/// define_id_array!(A5, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2")), ]);
> > +/// define_id_array!(A6, Id, &'static [u8], [(Id(10), None), (Id(20), Some(b"id2")), ]);
> > +/// define_id_array!(A7, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), None), ]);
> > +/// define_id_array!(A8, Id, &'static [u8], [(Id(10), None), (Id(20), None), ]);
> > +/// ```
> 
> Again, busses define this, put this in the bus-specific code that you
> wish to define/match with, it does not belong here as every bus does it
> differently.  Not all the world is PCI :)

The macro below is just a helper macro for the subsystem / bus specific code to
define its own macro that can be used by drivers to provide the ID table to
match.

In [1] PCI uses this macro to define its own one (define_pci_id_table() and in
[2] Nova uses define_pci_id_table() to define the matching PCI devices.

[1] https://lore.kernel.org/rust-for-linux/20240520172554.182094-10-dakr@redhat.com/
[2] https://lore.kernel.org/rust-for-linux/20240520172422.181763-5-dakr@redhat.com/

> 
> > +#[macro_export]
> > +macro_rules! define_id_array {
> > +    ($table_name:ident, $id_type:ty, $data_type:ty, [ $($t:tt)* ]) => {
> > +        const $table_name:
> > +            $crate::driver::IdArray<$id_type, $data_type, { $crate::count_paren_items!($($t)*) }> =
> > +                $crate::_new_id_array!((
> > +                    $crate::first_item!($id_type, $($t)*), $crate::second_item!($($t)*)), $id_type);
> > +    };
> > +}
> > +
> > +/// Defines a new constant [`IdTable`] with a concise syntax.
> > +///
> > +/// It is meant to be used by buses and subsystems to create a similar macro with their device id
> > +/// type already specified, i.e., with fewer parameters to the end user.
> > +///
> > +/// # Examples
> > +///
> > +// TODO: Exported but not usable by kernel modules (requires `const_trait_impl`).
> > +/// ```ignore
> > +/// #![feature(const_trait_impl)]
> > +/// # use kernel::{define_id_table, driver::RawDeviceId};
> > +///
> > +/// #[derive(Copy, Clone)]
> > +/// struct Id(u32);
> > +///
> > +/// // SAFETY: `ZERO` is all zeroes and `to_rawid` stores `offset` as the second element of the raw
> > +/// // device id pair.
> > +/// unsafe impl const RawDeviceId for Id {
> > +///     type RawType = (u64, isize);
> > +///     const ZERO: Self::RawType = (0, 0);
> > +///     fn to_rawid(&self, offset: isize) -> Self::RawType {
> > +///         (self.0 as u64 + 1, offset)
> > +///     }
> > +/// }
> > +///
> > +/// define_id_table!(T1, Id, &'static [u8], [(Id(10), None)]);
> > +/// define_id_table!(T2, Id, &'static [u8], [(Id(10), Some(b"id1")), ]);
> > +/// define_id_table!(T3, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2"))]);
> > +/// define_id_table!(T4, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), Some(b"id2")), ]);
> > +/// define_id_table!(T5, Id, &'static [u8], [(Id(10), None), (Id(20), Some(b"id2")), ]);
> > +/// define_id_table!(T6, Id, &'static [u8], [(Id(10), Some(b"id1")), (Id(20), None), ]);
> > +/// define_id_table!(T7, Id, &'static [u8], [(Id(10), None), (Id(20), None), ]);
> > +/// ```
> > +#[macro_export]
> > +macro_rules! define_id_table {
> > +    ($table_name:ident, $id_type:ty, $data_type:ty, [ $($t:tt)* ]) => {
> > +        const $table_name: Option<$crate::driver::IdTable<'static, $id_type, $data_type>> = {
> > +            $crate::define_id_array!(ARRAY, $id_type, $data_type, [ $($t)* ]);
> > +            Some(ARRAY.as_table())
> > +        };
> > +    };
> > +}
> 
> Again, see above, does not belong here.
> 
> > +
> > +/// Custom code within device removal.
> 
> You better define the heck out of "device removal" as specified last
> time this all came up.  From what I can see here, this is totally wrong
> and confusing and will be a mess.
> 
> Do it right, name it properly.

Please also see my reply [1] on your reply to another patch of this series.

Otherwise, what's wrong with this name? device_remove() is supposed to be called
once the driver's remove() callback is called.

It doesn't seem uncommon to use this terminology. For instance, in the
documentation of struct pci_driver [2] it says: "The remove() function gets
called whenever a device being handled by this driver is removed [...]".

In the documentation of struct device_driver [3] it says: "Called when the
device is removed from the system to unbind a device from this driver.".

Don't get me wrong, I'm happy to change it, but I think you should make me
understand why this terminology is widely used in the kernel, but here it's
totally wrong.

[1] https://lore.kernel.org/rust-for-linux/ZkupaTM+xjCiBbb4@pollux.localdomain/
[2] https://elixir.bootlin.com/linux/latest/source/include/linux/pci.h#L905
[3] https://elixir.bootlin.com/linux/latest/source/include/linux/device/driver.h#L71

> 
> I'm not reviewingn beyond here, sorry.  It's the merge window and I
> shouldn't have even looked at this until next week anyway.
> 
> But I was hoping that the whole long rant I gave last time would be
> addressed at least a little bit.  I don't see that it has :(

I think it's not really fair to expect this documentation to be rewritten when
you just back out of the discussion before getting to an agreement. Especially
when the situation at least looks contradictive as presented above.

I take your concerns serious and spend a lot of time in answering your questions
and giving explanations - let's keep the discussion up!

> 
> 
> > +pub trait DeviceRemoval {
> > +    /// Cleans resources up when the device is removed.
> > +    ///
> > +    /// This is called when a device is removed and offers implementers the chance to run some code
> > +    /// that cleans state up.
> > +    fn device_remove(&self);
> > +}
> > +
> > +impl DeviceRemoval for () {
> > +    fn device_remove(&self) {}
> > +}
> > +
> > +impl<T: DeviceRemoval> DeviceRemoval for Arc<T> {
> > +    fn device_remove(&self) {
> > +        self.deref().device_remove();
> > +    }
> > +}
> > +
> > +impl<T: DeviceRemoval> DeviceRemoval for Box<T> {
> > +    fn device_remove(&self) {
> > +        self.deref().device_remove();
> > +    }
> > +}
> > +
> > +/// A kernel module that only registers the given driver on init.
> > +///
> > +/// This is a helper struct to make it easier to define single-functionality modules, in this case,
> > +/// modules that offer a single driver.
> > +pub struct Module<T: DriverOps> {
> > +    _driver: Pin<Box<Registration<T>>>,
> > +}
> > +
> > +impl<T: DriverOps> crate::Module for Module<T> {
> > +    fn init(name: &'static CStr, module: &'static ThisModule) -> Result<Self> {
> > +        Ok(Self {
> > +            _driver: Registration::new_pinned(name, module)?,
> > +        })
> > +    }
> > +}
> > +
> > +/// Declares a kernel module that exposes a single driver.
> 
> There is no requirement that a kernel module only exposes a single
> driver, so what exactly is this used for?  Is this some attempt to make
> the module_driver() macro in rust?  Why?  Only add something like that
> if you really need it (i.e. you have a bus in rust and you want the
> benifit of having that macro to save you some lines of code.)  I don't
> see that happening here...

It's just a macro helper for subsystems to cover the common case of a single
driver in a module. Kinda the generic Rust version of e.g. module_pci_driver()
or module_usb_driver().

There is no general limitation of a single driver per module.

> 
> greg k-h
> 

Re: [RFC PATCH 10/11] rust: add basic abstractions for iomem operations

[Miguel Ojeda]

On Mon, May 20, 2024 at 7:27 PM Danilo Krummrich <dakr@redhat.com> wrote:
>
> through its Drop() implementation.

Nit: `Drop`, `Deref` and so on are traits -- what do the `()` mean
here? I guess you may be referring to their method, but those are
lowercase.

> +/// IO-mapped memory, starting at the base pointer @ioptr and spanning @malxen bytes.

Please use Markdown code spans instead (and intra-doc links where
possible) -- we don't use the `@` notation. There is a typo on the
variable name too.

> +pub struct IoMem {
> +    pub ioptr: usize,

This field is public, which raises some questions...

> +    pub fn readb(&self, offset: usize) -> Result<u8> {
> +        let ioptr: usize = self.get_io_addr(offset, 1)?;
> +
> +        Ok(unsafe { bindings::readb(ioptr as _) })
> +    }

These methods are unsound, since `ioptr` may end up being anything
here, given `self.ioptr` it is controlled by the caller. One could
also trigger an overflow in `get_io_addr`.

Wedson wrote a similar abstraction in the past
(`rust/kernel/io_mem.rs` in the old `rust` branch), with a
compile-time `SIZE` -- it is probably worth taking a look.

Also, there are missing `// SAFETY:` comments here. Documentation and
examples would also be nice to have.

Thanks!

Cheers,
Miguel

Re: [RFC PATCH 11/11] rust: PCI: add BAR request and ioremap

[Miguel Ojeda]

Hi Philipp,

A few quick nits I noticed for this one...

On Mon, May 20, 2024 at 7:27 PM Danilo Krummrich <dakr@redhat.com> wrote:
>
> +/// A PCI BAR to perform IO-Operations on.

Some more documentation, examples, and/or references would be nice.

> +pub struct Bar {
> +    pdev: Device,
> +    iomem: IoMem,
> +    num: u8,
> +}
> +
> +impl Bar {
> +    fn new(pdev: Device, num: u8, name: &CStr) -> Result<Self> {
> +        let barnr = num as i32;

Would it make sense to use newtypes for `num`/`barnr`, perhaps?

> +        let barlen = pdev.resource_len(num)?;
> +        if barlen == 0 {
> +            return Err(ENOMEM);
> +        }
> +
> +        // SAFETY:
> +        // `pdev` is always valid.

Please explain why it is always valid -- the point of a `SAFETY`
comment is not to say something is OK, but why it is so.

> +        // `barnr` is checked for validity at the top of the function.

Where was it checked? I may be missing something, but I only see a
widening cast.

> +        // SAFETY:
> +        // `pdev` is always valid.
> +        // `barnr` is checked for validity at the top of the function.
> +        // `name` is always valid.

Please use the convention we have elsewhere for this kind of lists,
i.e. use a list with the `-` bullet list marker.

> +        let ioptr: usize = unsafe { bindings::pci_iomap(pdev.as_raw(), barnr, 0) } as usize;

Is the type needed, since there is an explicit cast?

> +        if ioptr == 0 {
> +            // SAFETY:
> +            // `pdev` is always valid.
> +            // `barnr` is checked for validity at the top of the function.
> +            unsafe { bindings::pci_release_region(pdev.as_raw(), barnr) };
> +            return Err(ENOMEM);
> +        }

Should the region be managed in a RAII type instead?

> +    fn index_is_valid(i: u8) -> bool {
> +        // A pci_dev on the C side owns an array of resources with at most
> +        // PCI_NUM_RESOURCES entries.

Missing Markdown. There are other instances as well.

> +        if i as i32 >= bindings::PCI_NUM_RESOURCES as i32 {
> +            return false;
> +        }
> +
> +        true

The body of the function could just be `... < ...`, i.e. no `if` needed.

> +    // SAFETY: The caller should ensure that `ioptr` is valid.
> +    unsafe fn do_release(pdev: &Device, ioptr: usize, num: u8) {

This should not be a comment but documentation, and it should be a `#
Safety` section, not a `// SAFETY:` comment.

> +    /// Returns the size of the given PCI bar resource.
> +    pub fn resource_len(&self, bar: u8) -> Result<bindings::resource_size_t> {

Sometimes `bindings::` in signatures of public methods may be
justified -- is it the case here? Or should a newtype or similar be
provided instead? I only see this function being called inside the
module, anyway.

> +    /// Mapps an entire PCI-BAR after performing a region-request on it.

Typo.

Cheers,
Miguel

Re: [RFC PATCH 10/11] rust: add basic abstractions for iomem operations

[Dave Airlie]

>
> Wedson wrote a similar abstraction in the past
> (`rust/kernel/io_mem.rs` in the old `rust` branch), with a
> compile-time `SIZE` -- it is probably worth taking a look.
>

Just on this point, we can't know in advance what size the IO BARs are
at compile time.

The old method just isn't useful for real devices with runtime IO BAR sizes.

Dave.

Re: [RFC PATCH 10/11] rust: add basic abstractions for iomem operations

[Danilo Krummrich]

(quick reply from my phone)

> On 21. May 2024, at 00:32, Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> wrote:
> 
> On Mon, May 20, 2024 at 7:27 PM Danilo Krummrich <dakr@redhat.com> wrote:
>> 
>> through its Drop() implementation.
> 
> Nit: `Drop`, `Deref` and so on are traits -- what do the `()` mean
> here? I guess you may be referring to their method, but those are
> lowercase.
> 
>> +/// IO-mapped memory, starting at the base pointer @ioptr and spanning @malxen bytes.
> 
> Please use Markdown code spans instead (and intra-doc links where
> possible) -- we don't use the `@` notation. There is a typo on the
> variable name too.
> 
>> +pub struct IoMem {
>> +    pub ioptr: usize,
> 
> This field is public, which raises some questions...
> 
>> +    pub fn readb(&self, offset: usize) -> Result<u8> {
>> +        let ioptr: usize = self.get_io_addr(offset, 1)?;
>> +
>> +        Ok(unsafe { bindings::readb(ioptr as _) })
>> +    }
> 
> These methods are unsound, since `ioptr` may end up being anything
> here, given `self.ioptr` it is controlled by the caller. One could
> also trigger an overflow in `get_io_addr`.
> 

I think the only thing we really want from IoMem is a generic implementation of the read*() and write*() functions.

Everything else should be up the the resource itself, see e.g. pci::Bar. Hence I think we should just make IoMem a trait instead and just let the resource implement a getter for the MMIO pointer, etc.

> Wedson wrote a similar abstraction in the past
> (`rust/kernel/io_mem.rs` in the old `rust` branch), with a
> compile-time `SIZE` -- it is probably worth taking a look.
> 
> Also, there are missing `// SAFETY:` comments here. Documentation and
> examples would also be nice to have.
> 
> Thanks!
> 
> Cheers,
> Miguel
> 

Re: [RFC PATCH 10/11] rust: add basic abstractions for iomem operations

[Wedson Almeida Filho]

On Mon, 20 May 2024 at 23:07, Dave Airlie <airlied@gmail.com> wrote:
>
> >
> > Wedson wrote a similar abstraction in the past
> > (`rust/kernel/io_mem.rs` in the old `rust` branch), with a
> > compile-time `SIZE` -- it is probably worth taking a look.
> >
>
> Just on this point, we can't know in advance what size the IO BARs are
> at compile time.
>
> The old method just isn't useful for real devices with runtime IO BAR sizes.

The compile-time `SIZE` in my implementation is a minimum size.

Attempts to read/write with constants within that size (offset + size)
were checked at compile time, that is, they would have zero additional
runtime cost when compared to C. Reads/writes beyond the minimum would
have to be checked at runtime.

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Dave Airlie]

On Tue, 21 May 2024 at 04:14, Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> > From: Wedson Almeida Filho <wedsonaf@gmail.com>
> >
> > This defines general functionality related to registering drivers with
> > their respective subsystems, and registering modules that implement
> > drivers.
> >
> > Co-developed-by: Asahi Lina <lina@asahilina.net>
> > Signed-off-by: Asahi Lina <lina@asahilina.net>
> > Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> > Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > ---
> >  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
> >  rust/kernel/lib.rs           |   4 +-
> >  rust/macros/module.rs        |   2 +-
> >  samples/rust/rust_minimal.rs |   2 +-
> >  samples/rust/rust_print.rs   |   2 +-
> >  5 files changed, 498 insertions(+), 4 deletions(-)
> >  create mode 100644 rust/kernel/driver.rs
> >
> > diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> > new file mode 100644
> > index 000000000000..e0cfc36d47ff
> > --- /dev/null
> > +++ b/rust/kernel/driver.rs
> > @@ -0,0 +1,492 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> > +//!
> > +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> > +//! using the [`Registration`] class.
>
> Why are you creating new "names" here?  "DriverOps" is part of a 'struct
> device_driver' why are you separating it out here?  And what is
> 'Registration'?  That's a bus/class thing, not a driver thing.
>
> And be very careful of the use of the word 'class' here, remember, there
> is 'struct class' as part of the driver model :)
>
> > +use crate::{
> > +    alloc::{box_ext::BoxExt, flags::*},
> > +    error::code::*,
> > +    error::Result,
> > +    str::CStr,
> > +    sync::Arc,
> > +    ThisModule,
> > +};
> > +use alloc::boxed::Box;
> > +use core::{cell::UnsafeCell, marker::PhantomData, ops::Deref, pin::Pin};
> > +
> > +/// A subsystem (e.g., PCI, Platform, Amba, etc.) that allows drivers to be written for it.
> > +pub trait DriverOps {
>
> Again, why is this not called DeviceDriver?

This is not the same as the C device_driver, it might mildly align in
concept with it, but I'm not sure it shares enough to align it name
wise with the C one.

> > +    /// The type that holds information about the registration. This is typically a struct defined
> > +    /// by the C portion of the kernel.
> > +    type RegType: Default;
> > +
> > +    /// Registers a driver.
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// `reg` must point to valid, initialised, and writable memory. It may be modified by this
> > +    /// function to hold registration state.
> > +    ///
> > +    /// On success, `reg` must remain pinned and valid until the matching call to
> > +    /// [`DriverOps::unregister`].
> > +    unsafe fn register(
> > +        reg: *mut Self::RegType,
> > +        name: &'static CStr,
> > +        module: &'static ThisModule,
> > +    ) -> Result;
> > +
> > +    /// Unregisters a driver previously registered with [`DriverOps::register`].
> > +    ///
> > +    /// # Safety
> > +    ///
> > +    /// `reg` must point to valid writable memory, initialised by a previous successful call to
> > +    /// [`DriverOps::register`].
> > +    unsafe fn unregister(reg: *mut Self::RegType);
> > +}
> > +
> > +/// The registration of a driver.
> > +pub struct Registration<T: DriverOps> {
> > +    is_registered: bool,
>
> Why does a driver need to know if it is registered or not?  Only the
> driver core cares about that, please do not expose that, it's racy and
> should not be relied on.

From the C side this does look unusual because on the C side something
like struct pci_driver is statically allocated everywhere.
In this rust abstraction, these are allocated dynamically, so instead
of just it always being safe to just call register/unregister
with static memory, a flag is kept around to say if the unregister
should happen at all, as the memory may have
been allocated but never registered. This is all the Registration is
for, it's tracking the bus _driver structure allocation, and
whether the bus register/unregister have been called since the object
was allocated.

I'm not sure it makes sense (or if you can at all), have a static like
pci_driver object here to match how C does things.

> > +impl<T: DriverOps> Drop for Registration<T> {
> > +    fn drop(&mut self) {
> > +        if self.is_registered {
> > +            // SAFETY: This path only runs if a previous call to `T::register` completed
> > +            // successfully.
> > +            unsafe { T::unregister(self.concrete_reg.get()) };
>
> Can't the rust code ensure that this isn't run if register didn't
> succeed?  Having a boolean feels really wrong here (can't that race?)

There might be a way of using Option<> here but I don't think it adds
anything over and above using an explicit bool.

> > +///
> > +/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
> > +/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
> > +///
> > +/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
> > +/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
> > +/// concrete types (which can still have const associated functions) instead of a trait.
> > +///
> > +/// # Safety
> > +///
> > +/// Implementers must ensure that:
> > +///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
> > +///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
> > +///     that buses can recover the pointer to the data.
> > +pub unsafe trait RawDeviceId {
> > +    /// The raw type that holds the device id.
> > +    ///
> > +    /// Id tables created from [`Self`] are going to hold this type in its zero-terminated array.
> > +    type RawType: Copy;
> > +
> > +    /// A zeroed-out representation of the raw device id.
> > +    ///
> > +    /// Id tables created from [`Self`] use [`Self::ZERO`] as the sentinel to indicate the end of
> > +    /// the table.
> > +    const ZERO: Self::RawType;
>
> All busses have their own way of creating "ids" and that is limited to
> the bus code itself, why is any of this in the rust side?  What needs
> this?  A bus will create the id for the devices it manages, and can use
> it as part of the name it gives the device (but not required), so all of
> this belongs to the bus, NOT to a driver, or a device.

Consider this a base class (Trait) for bus specific IDs.

> > +}
> > +
> > +// Creates a new ID array. This is a macro so it can take as a parameter the concrete ID type in
> > +// order to call to_rawid() on it, and still remain const. This is necessary until a new
> > +// const_trait_impl implementation lands, since the existing implementation was removed in Rust
> > +// 1.73.
>
> Again, what are these id lists for?  Busses work on individual ids that
> they create dynamically.
>
> You aren't thinking this is an id that could be used to match devices to
> drivers, are you?  That's VERY bus specific, and also specified already
> in .c code for those busses and passed to userspace.  That doesn't
> belong here.
>
> If this isn't that list, what exactly is this?

Rust traits have to be specified somewhere, a RawDeviceId is the basis
for other DeviceId Traits to be implemented on top off, those then
talk to the C side ones.

All the pci and nova code is posted as well for you to work this out,
like you asked for. The use cases are all there to be reviewed, and
should help answer these sorts of questions.


>
> > +#[macro_export]
> > +#[doc(hidden)]
> > +macro_rules! _new_id_array {
> > +    (($($args:tt)*), $id_type:ty) => {{
> > +        /// Creates a new instance of the array.
> > +        ///
> > +        /// The contents are derived from the given identifiers and context information.
> > +        const fn new< U, const N: usize>(ids: [$id_type; N], infos: [Option<U>; N])
> > +            -> $crate::driver::IdArray<$id_type, U, N>
> > +        where
> > +            $id_type: $crate::driver::RawDeviceId + Copy,
> > +            <$id_type as $crate::driver::RawDeviceId>::RawType: Copy + Clone,
> > +        {
> > +            let mut raw_ids =
> > +                [<$id_type as $crate::driver::RawDeviceId>::ZERO; N];
> > +            let mut i = 0usize;
> > +            while i < N {
> > +                let offset: isize = $crate::driver::IdArray::<$id_type, U, N>::get_offset(i);
> > +                raw_ids[i] = ids[i].to_rawid(offset);
> > +                i += 1;
> > +            }
> > +
> > +            // SAFETY: We are passing valid arguments computed with the correct offsets.
> > +            unsafe {
> > +                $crate::driver::IdArray::<$id_type, U, N>::new(raw_ids, infos)
> > +            }
> > +       }
> > +
> > +        new($($args)*)
> > +    }}
> > +}
> > +
> > +/// A device id table.
> > +///
> > +/// The table is guaranteed to be zero-terminated and to be followed by an array of context data of
> > +/// type `Option<U>`.
> > +pub struct IdTable<'a, T: RawDeviceId, U> {
> > +    first: &'a T::RawType,
> > +    _p: PhantomData<&'a U>,
> > +}
>
> All busses have different ways of matching drivers to devices, and they
> might be called a "device id table" and it might not.  The driver core
> doesn't care, and neither should this rust code.  That's all
> bus-specific and unique to each and every one of them.  None of this
> should be needed here at all.

This is just a base Trait, for the bus specifics to implement.
Consider it a bunch of C macros or a bunch of C++ vfuncs, whatever
mental model helps more.

> Shouldn't this go in some common header somewhere?  Why is this here?

Again to reiterate what Danilo said, rust has no header files. Things
that in C we would put in header files (macros, struct definitions,
inlines) go into rust code and other rust code references it.

Now there might be some sense in splitting things further into
separate rust files if this concept is going to confuse people, and
what is "core" device code, and what is "template" device code. But
I'm not sure how much sense that makes really.

Maybe the device table stuff should be in a separate file?

> > +/// Custom code within device removal.
>
> You better define the heck out of "device removal" as specified last
> time this all came up.  From what I can see here, this is totally wrong
> and confusing and will be a mess.
>
> Do it right, name it properly.
>
> I'm not reviewingn beyond here, sorry.  It's the merge window and I
> shouldn't have even looked at this until next week anyway.
>
> But I was hoping that the whole long rant I gave last time would be
> addressed at least a little bit.  I don't see that it has :(

I won't comment too much on the specifics here, but you've twice got
to this stage, said something is wrong, change it, and given no
actionable feedback on what is wrong, or what to change.

I've looked at this code for a few hours today with the device docs
and core code and can't spot what you are seeing that is wrong here,
which means I don't expect anyone else is going to unless you can help
educate us more.

Dave.

Re: [RFC PATCH 10/11] rust: add basic abstractions for iomem operations

[Philipp Stanner]

On Tue, 2024-05-21 at 00:32 +0200, Miguel Ojeda wrote:
> On Mon, May 20, 2024 at 7:27 PM Danilo Krummrich <dakr@redhat.com>
> wrote:
> > 
> > through its Drop() implementation.
> 
> Nit: `Drop`, `Deref` and so on are traits -- what do the `()` mean
> here? I guess you may be referring to their method, but those are
> lowercase.

ACK

> 
> > +/// IO-mapped memory, starting at the base pointer @ioptr and
> > spanning @malxen bytes.
> 
> Please use Markdown code spans instead (and intra-doc links where
> possible) -- we don't use the `@` notation. There is a typo on the
> variable name too.
> 
> > +pub struct IoMem {
> > +    pub ioptr: usize,
> 
> This field is public, which raises some questions...

Justified questions – it is public because the Drop implementation for
pci::Bar requires the ioptr to pass it to pci_iounmap().

The alternative would be to give pci::Bar a copy of ioptr (it's just an
integer after all), but that would also not be exactly beautiful.

The subsystem (as PCI does here) shall not make an instance of IoMem
mutable, so the driver programmer couldn't modify ioptr.

I'm very open for ideas for alternatives, though. See also the other
mail where Danilo brainstorms about making IoMem a trait.

> 
> > +    pub fn readb(&self, offset: usize) -> Result<u8> {
> > +        let ioptr: usize = self.get_io_addr(offset, 1)?;
> > +
> > +        Ok(unsafe { bindings::readb(ioptr as _) })
> > +    }
> 
> These methods are unsound, since `ioptr` may end up being anything
> here, given `self.ioptr` it is controlled by the caller. 

Only if IoMem is mutable, correct?

The commit message states (btw this file would get more extensive
comments soonish) that with this design its the subsystem that is
responsible for creating IoMem validly, because the subsystem is the
one who knows about the memory regions and lengths and stuff.

The driver should only ever take an IoMem through a subsystem, so that
would be safe.

> One could
> also trigger an overflow in `get_io_addr`.

Yes, if the addition violates the capacity of a usize. But that would
then be a bug we really want to notice, wouldn't we?

Only alternative I can think of would be to do a wrapping_add(), but
that would be even worse UB.

Ideas?

> 
> Wedson wrote a similar abstraction in the past
> (`rust/kernel/io_mem.rs` in the old `rust` branch), with a
> compile-time `SIZE` -- it is probably worth taking a look.

Yes, we're aware of that one. We also did some experiments with it.
Will discuss it in the other thread where Dave and Wedson mention it.

> 
> Also, there are missing `// SAFETY:` comments here. Documentation and
> examples would also be nice to have.

Oh yes, ACK, will do


Thx for the review!


> 
> Thanks!
> 
> Cheers,
> Miguel
> 

Re: [RFC PATCH 10/11] rust: add basic abstractions for iomem operations

[Philipp Stanner]

On Tue, 2024-05-21 at 00:01 -0300, Wedson Almeida Filho wrote:
> On Mon, 20 May 2024 at 23:07, Dave Airlie <airlied@gmail.com> wrote:
> > 
> > > 
> > > Wedson wrote a similar abstraction in the past
> > > (`rust/kernel/io_mem.rs` in the old `rust` branch), with a
> > > compile-time `SIZE` -- it is probably worth taking a look.
> > > 
> > 
> > Just on this point, we can't know in advance what size the IO BARs
> > are
> > at compile time.
> > 
> > The old method just isn't useful for real devices with runtime IO
> > BAR sizes.
> 
> The compile-time `SIZE` in my implementation is a minimum size.
> 
> Attempts to read/write with constants within that size (offset +
> size)
> were checked at compile time, that is, they would have zero
> additional
> runtime cost when compared to C. Reads/writes beyond the minimum
> would
> have to be checked at runtime.
> 

We looked at this implementation

Its disadvantage is that it moves the responsibility for setting that
minimum size to the driver programmer. Andreas Hindborg is using that
currently for rnvme [1].

I believe that the driver programmer in Rust should not be responsible
for controlling such sensitive parameters (one could far more easily
provide invalid values), but the subsystem (e.g. PCI) should do it,
because it knows about the exact resource lengths.

The only way to set the actual, real value is through subsystem code.
But when we (i.e., currently, the driver programmer) have to use that
anyways, we can just use it from the very beginning and have the exact
valid parameters.

So driver programmers would always have correct IoMem and would have a
harder time breaking stuff, and wouldn't have to think about minimum
lengths and things like that.


P.

[1] https://github.com/metaspace/linux/blob/rnvme/drivers/block/rnvme.rs#L580

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Greg KH]

On Tue, May 21, 2024 at 03:42:50PM +1000, Dave Airlie wrote:
> On Tue, 21 May 2024 at 04:14, Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> > On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> > > From: Wedson Almeida Filho <wedsonaf@gmail.com>
> > >
> > > This defines general functionality related to registering drivers with
> > > their respective subsystems, and registering modules that implement
> > > drivers.
> > >
> > > Co-developed-by: Asahi Lina <lina@asahilina.net>
> > > Signed-off-by: Asahi Lina <lina@asahilina.net>
> > > Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > > ---
> > >  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
> > >  rust/kernel/lib.rs           |   4 +-
> > >  rust/macros/module.rs        |   2 +-
> > >  samples/rust/rust_minimal.rs |   2 +-
> > >  samples/rust/rust_print.rs   |   2 +-
> > >  5 files changed, 498 insertions(+), 4 deletions(-)
> > >  create mode 100644 rust/kernel/driver.rs
> > >
> > > diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> > > new file mode 100644
> > > index 000000000000..e0cfc36d47ff
> > > --- /dev/null
> > > +++ b/rust/kernel/driver.rs
> > > @@ -0,0 +1,492 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +
> > > +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> > > +//!
> > > +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> > > +//! using the [`Registration`] class.
> >
> > Why are you creating new "names" here?  "DriverOps" is part of a 'struct
> > device_driver' why are you separating it out here?  And what is
> > 'Registration'?  That's a bus/class thing, not a driver thing.
> >
> > And be very careful of the use of the word 'class' here, remember, there
> > is 'struct class' as part of the driver model :)
> >
> > > +use crate::{
> > > +    alloc::{box_ext::BoxExt, flags::*},
> > > +    error::code::*,
> > > +    error::Result,
> > > +    str::CStr,
> > > +    sync::Arc,
> > > +    ThisModule,
> > > +};
> > > +use alloc::boxed::Box;
> > > +use core::{cell::UnsafeCell, marker::PhantomData, ops::Deref, pin::Pin};
> > > +
> > > +/// A subsystem (e.g., PCI, Platform, Amba, etc.) that allows drivers to be written for it.
> > > +pub trait DriverOps {
> >
> > Again, why is this not called DeviceDriver?
> 
> This is not the same as the C device_driver, it might mildly align in
> concept with it, but I'm not sure it shares enough to align it name
> wise with the C one.

Why not use the same terms and design decisions that the C code has?  To
differ needs to have a good reason, otherwise it's just going to cause
us all confusion as we have to learn two different terms for the same
thing.

> > > +    /// The type that holds information about the registration. This is typically a struct defined
> > > +    /// by the C portion of the kernel.
> > > +    type RegType: Default;
> > > +
> > > +    /// Registers a driver.
> > > +    ///
> > > +    /// # Safety
> > > +    ///
> > > +    /// `reg` must point to valid, initialised, and writable memory. It may be modified by this
> > > +    /// function to hold registration state.
> > > +    ///
> > > +    /// On success, `reg` must remain pinned and valid until the matching call to
> > > +    /// [`DriverOps::unregister`].
> > > +    unsafe fn register(
> > > +        reg: *mut Self::RegType,
> > > +        name: &'static CStr,
> > > +        module: &'static ThisModule,
> > > +    ) -> Result;
> > > +
> > > +    /// Unregisters a driver previously registered with [`DriverOps::register`].
> > > +    ///
> > > +    /// # Safety
> > > +    ///
> > > +    /// `reg` must point to valid writable memory, initialised by a previous successful call to
> > > +    /// [`DriverOps::register`].
> > > +    unsafe fn unregister(reg: *mut Self::RegType);
> > > +}
> > > +
> > > +/// The registration of a driver.
> > > +pub struct Registration<T: DriverOps> {
> > > +    is_registered: bool,
> >
> > Why does a driver need to know if it is registered or not?  Only the
> > driver core cares about that, please do not expose that, it's racy and
> > should not be relied on.
> 
> >From the C side this does look unusual because on the C side something
> like struct pci_driver is statically allocated everywhere.
> In this rust abstraction, these are allocated dynamically, so instead
> of just it always being safe to just call register/unregister
> with static memory, a flag is kept around to say if the unregister
> should happen at all, as the memory may have
> been allocated but never registered. This is all the Registration is
> for, it's tracking the bus _driver structure allocation, and
> whether the bus register/unregister have been called since the object
> was allocated.
> 
> I'm not sure it makes sense (or if you can at all), have a static like
> pci_driver object here to match how C does things.

Wait, why can't you have a static "rust_driver" type thing?  Having it
be in ram feels like a waste of memory.  We are working hard to move
more and more of these driver model structures into read-only memory for
good reasons (security, reduce bugs, etc.), moving backwards to having
them all be dynamically created/copied around feels wrong.

We are one stage away from being able to mark all 'driver_*()' calls as
using a const * to struct driver, please don't make that work pointless
if you want to write a driver in rust instead.

And again, a driver should never know/care/wonder if it has been
registered or not, that is up to the driver core to handle, not the rust
code, as it is the only thing that can know this, and the only thing
that should need to know it.  A driver structure should not have any
dynamic memory associated with it to require knowing its "state" in the
system, so let's not move backwards here to require that just because we
are using Rust.

> > > +impl<T: DriverOps> Drop for Registration<T> {
> > > +    fn drop(&mut self) {
> > > +        if self.is_registered {
> > > +            // SAFETY: This path only runs if a previous call to `T::register` completed
> > > +            // successfully.
> > > +            unsafe { T::unregister(self.concrete_reg.get()) };
> >
> > Can't the rust code ensure that this isn't run if register didn't
> > succeed?  Having a boolean feels really wrong here (can't that race?)
> 
> There might be a way of using Option<> here but I don't think it adds
> anything over and above using an explicit bool.

Again, this should not be needed.  If so, something is designed
incorrectly with the bindings.

> > > +///
> > > +/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
> > > +/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
> > > +///
> > > +/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
> > > +/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
> > > +/// concrete types (which can still have const associated functions) instead of a trait.
> > > +///
> > > +/// # Safety
> > > +///
> > > +/// Implementers must ensure that:
> > > +///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
> > > +///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
> > > +///     that buses can recover the pointer to the data.
> > > +pub unsafe trait RawDeviceId {
> > > +    /// The raw type that holds the device id.
> > > +    ///
> > > +    /// Id tables created from [`Self`] are going to hold this type in its zero-terminated array.
> > > +    type RawType: Copy;
> > > +
> > > +    /// A zeroed-out representation of the raw device id.
> > > +    ///
> > > +    /// Id tables created from [`Self`] use [`Self::ZERO`] as the sentinel to indicate the end of
> > > +    /// the table.
> > > +    const ZERO: Self::RawType;
> >
> > All busses have their own way of creating "ids" and that is limited to
> > the bus code itself, why is any of this in the rust side?  What needs
> > this?  A bus will create the id for the devices it manages, and can use
> > it as part of the name it gives the device (but not required), so all of
> > this belongs to the bus, NOT to a driver, or a device.
> 
> Consider this a base class (Trait) for bus specific IDs.

Then all of that should be in a separate file as they belong to a "bus"
not a driver, as each and every bus will have a different way of
expressing the list of devices a driver can bind to.

And again, the core "struct device_driver" does not have a list of ids,
and neither should the rust bindings, that belongs to the bus logic.

> > > +/// Custom code within device removal.
> >
> > You better define the heck out of "device removal" as specified last
> > time this all came up.  From what I can see here, this is totally wrong
> > and confusing and will be a mess.
> >
> > Do it right, name it properly.
> >
> > I'm not reviewingn beyond here, sorry.  It's the merge window and I
> > shouldn't have even looked at this until next week anyway.
> >
> > But I was hoping that the whole long rant I gave last time would be
> > addressed at least a little bit.  I don't see that it has :(
> 
> I won't comment too much on the specifics here, but you've twice got
> to this stage, said something is wrong, change it, and given no
> actionable feedback on what is wrong, or what to change.
> 
> I've looked at this code for a few hours today with the device docs
> and core code and can't spot what you are seeing that is wrong here,
> which means I don't expect anyone else is going to unless you can help
> educate us more.

A lifecycle of a device is:
	device create by the bus
	device register with driver core
	device bind to a driver (i.e. probe() callback in the driver)
	device unbind from a driver (i.e. remove() callback in the driver, can be triggered many ways)
	device destroy by the bus

"remove" can happen for a driver where it needs to clean up everything
it has done for a specific device, but that does not mean the device is
actually gone from the system, that's up to the bus to decide, as it
owns the device lifecycle and gets to decide when it thinks it is really
gone.  And then, when the bus tells the driver core that it wants to
mark the device as "destroyed", it's up to the driver core to eventually
call back into the bus to really clean that device up from the system at
some later point in time.

Try splitting this file out into driver and bus and device logic, like
the driver core has, and see if that helps in explaining and making all
of this more understandable, and to keep with the names/design of the
model we currently have.

Note, all of this is my biggest worry about writing a driver in rust,
getting the lifecycle of the driver core and it's logic of how it
handles memory manged in C, to align up with how rust is going to access
it with its different rules and requirements, is not going to be simple
as you have two different models intersecting at the same place.  I've
been working over the past year to make the rust side happen easier by
marking more and more of the C structures as "not mutable", i.e. const
*, so that the Rust side doesn't have to worry that the driver core
really will change things it passes to the C side, but there is more to
be done (see above about making struct device_driver * const).

I want to see this happen, but it's going to be a hard slog due to the
different expectations of these two systems.  Keeping naming identical
where possible is one way to make it simpler for everyone involved, as
would splitting the logic out the same way and not mixing it all up into
one big hairy file that combines different things into a single chunk.

thanks,

greg k-h

Re: [RFC PATCH 10/11] rust: add basic abstractions for iomem operations

[Miguel Ojeda]

On Tue, May 21, 2024 at 9:36 AM Philipp Stanner <pstanner@redhat.com> wrote:
>
> Justified questions – it is public because the Drop implementation for
> pci::Bar requires the ioptr to pass it to pci_iounmap().
>
> The alternative would be to give pci::Bar a copy of ioptr (it's just an
> integer after all), but that would also not be exactly beautiful.

If by copy you mean keeping an actual copy elsewhere, then you could
provide an access method instead.

If you meant the access method, it may not be "beautiful" (Why? What
do you mean?), but it is way more important to be sound.

> The commit message states (btw this file would get more extensive
> comments soonish) that with this design its the subsystem that is
> responsible for creating IoMem validly, because the subsystem is the
> one who knows about the memory regions and lengths and stuff.
>
> The driver should only ever take an IoMem through a subsystem, so that
> would be safe.

The Rust safety boundary is normally the module -- you want that
subsystems cannot make mistakes either when using the `iomem` module,
not just drivers when using the subsystem APIs.

So you can't rely on a user, even if it is a subsystem, to "validly
create" objects and also hope that they do not modify the fields later
etc. If you need to ask subsystems for that, then you need to require
`unsafe` somewhere, e.g. the constructor (and make the field private,
and add an invariant to the type, and add `INVARIANT:` comments).

Think about it this way: if we were to write all the code like that
(e.g. with all structs using public fields), then essentially we would
be back at C, since we would be trusting everybody not to touch what
they shouldn't, and not to put values that could later lead something
else into UB, and we would not even have the documentation/invariants
to verify those either, etc.

> Yes, if the addition violates the capacity of a usize. But that would
> then be a bug we really want to notice, wouldn't we?

Definitely, but what I wanted is that you consider whether it is
reasonable to have the panic possibility there, since it depends on a
value that others control. For instance, would it make sense to make
it a fallible operation instead? Should the panic be documented
otherwise? Could it be prevented somehow? etc.

Please check Wedson's `io_mem` in the old `rust` branch for some ideas too.

Cheers,
Miguel

Re: [RFC PATCH 00/11] [RFC] Device / Driver and PCI Rust abstractions

[Greg KH]

On Mon, May 20, 2024 at 09:50:01PM +0200, Danilo Krummrich wrote:
> On Mon, May 20, 2024 at 08:16:19PM +0200, Greg KH wrote:
> > On Mon, May 20, 2024 at 08:14:57PM +0200, Greg KH wrote:
> > > On Mon, May 20, 2024 at 07:25:37PM +0200, Danilo Krummrich wrote:
> > > > This patch sereis implements basic generic device / driver Rust abstractions,
> > > > as well as some basic PCI abstractions.
> > > > 
> > > > This patch series is sent in the context of [1], and the corresponding patch
> > > > series [2], which contains some basic DRM Rust abstractions and a stub
> > > > implementation of the Nova GPU driver.
> > > > 
> > > > Nova is intended to be developed upstream, starting out with just a stub driver
> > > > to lift some initial required infrastructure upstream. A more detailed
> > > > explanation can be found in [1].
> > > > 
> > > > Some patches, which implement the generic device / driver Rust abstractions have
> > > > been sent a couple of weeks ago already [3]. For those patches the following
> > > > changes have been made since then:
> > > > 
> > > > - remove RawDevice::name()
> > > > - remove rust helper for dev_name() and dev_get_drvdata()
> > > > - use AlwaysRefCounted for struct Device
> > > > - drop trait RawDevice entirely in favor of AsRef and provide
> > > >   Device::from_raw(), Device::as_raw() and Device::as_ref() instead
> > > > - implement RevocableGuard
> > > > - device::Data, remove resources and replace it with a Devres abstraction
> > > > - implement Devres abstraction for resources
> > 
> > Ah, here's the difference from the last time, sorry, it wasn't obvious.
> > 
> > Still nothing about proper handling and use of 'remove' in the context
> > of all of this, that's something you really really really need to get
> > right if you want to attempt to have a driver in rust interact with the
> > driver core properly.
> 
> We were right in the middle of discussing about the correct wording when I sent
> those patches the first time. There were some replies from my side, e.g. [1] and
> another reply from Wedson [2] about this, which you did not want to reply to any
> more.
> 
> I'm not saying I insist on not changing those comments up, but first we have to
> agree on how we want them to be rephrased, especially since from the
> discussions so far I got the impression that we might talk a bit past each
> other.
> 
> Hence, I'd propose to just continue the discussion, where we need to.

See my responses in this thread, let's continue this there.

thanks,

greg k-h

Re: [RFC PATCH 01/11] rust: add abstraction for struct device

[Greg KH]

On Mon, May 20, 2024 at 10:22:22PM +0200, Danilo Krummrich wrote:
> On Mon, May 20, 2024 at 08:00:23PM +0200, Greg KH wrote:
> > On Mon, May 20, 2024 at 07:25:38PM +0200, Danilo Krummrich wrote:
> > > Add an (always) reference counted abstraction for a generic struct
> > > device. This abstraction encapsulates existing struct device instances
> > > and manages its reference count.
> > > 
> > > Subsystems may use this abstraction as a base to abstract subsystem
> > > specific device instances based on a generic struct device.
> > > 
> > > Co-developed-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > > ---
> > >  rust/helpers.c        |  1 +
> > >  rust/kernel/device.rs | 76 +++++++++++++++++++++++++++++++++++++++++++
> > 
> > What's the status of moving .rs files next to their respective .c files
> > in the build system?  Keeping them separate like this just isn't going
> > to work, sorry.
> > 
> > > --- /dev/null
> > > +++ b/rust/kernel/device.rs
> > > @@ -0,0 +1,76 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +
> > > +//! Generic devices that are part of the kernel's driver model.
> > > +//!
> > > +//! C header: [`include/linux/device.h`](../../../../include/linux/device.h)
> > 
> > relative paths for a common "include <linux/device.h" type of thing?
> > Rust can't handle include paths from directories?
> 
> Going to change this to `srctree/` as proposed by Miguel.
> 
> > 
> > > +
> > > +use crate::{
> > > +    bindings,
> > > +    types::{ARef, Opaque},
> > > +};
> > > +use core::ptr;
> > > +
> > > +/// A ref-counted device.
> > > +///
> > > +/// # Invariants
> > > +///
> > > +/// The pointer stored in `Self` is non-null and valid for the lifetime of the ARef instance. In
> > > +/// particular, the ARef instance owns an increment on underlying object’s reference count.
> > > +#[repr(transparent)]
> > > +pub struct Device(Opaque<bindings::device>);
> > > +
> > > +impl Device {
> > > +    /// Creates a new ref-counted instance of an existing device pointer.
> > > +    ///
> > > +    /// # Safety
> > > +    ///
> > > +    /// Callers must ensure that `ptr` is valid, non-null, and has a non-zero reference count.
> > 
> > Callers NEVER care about the reference count of a struct device, anyone
> > poking in that is asking for trouble.
> 
> That's confusing, if not the caller who's passing the device pointer somewhere,
> who else?
> 
> Who takes care that a device' reference count is non-zero when a driver's probe
> function is called?

A device's reference count will be non-zero, I'm saying that sometimes,
some driver core functions are called with a 'struct device' that is
NULL, and it can handle it just fine.  Hopefully no callbacks to the
rust code will happen that way, but why aren't you checking just "to be
sure!" otherwise you could have a bug here, and it costs nothing to
verify it, right?

> It's the same here. The PCI code calls Device::from_raw() from its
> probe_callback() function, which is called from the C side. For instance:
> 
> extern "C" fn probe_callback(
>    pdev: *mut bindings::pci_dev,
>    id: *const bindings::pci_device_id,
> ) -> core::ffi::c_int {
>    // SAFETY: This is safe, since the C side guarantees that pdev is a valid,
>    // non-null pointer to a struct pci_dev with a non-zero reference count.
>    let dev = unsafe { device::Device::from_raw(&mut (*pdev).dev) };

Yes, that's fine, if you are calling this from a probe callback, but
again, the driver core has lots of functions in it :)

>    [...]
> }
> 
> > 
> > And why non-NULL?  Can't you check for that here?  Shouldn't you check
> > for that here?  Many driver core functions can handle a NULL pointer
> > just fine (i.e. get/put_device() can), why should Rust code assume that
> > a pointer passed to it from the C layer is going to have stricter rules
> > than the C layer can provide?
> 
> We could check for NULL here, but I think it'd be pointless. Even if the pointer
> is not NULL, it can still be an invalid one. There is no check we can do to
> guarantee safety, hence the function is and remains unsafe and has safety
> requirements instead that the caller must guarantee to fulfil.
> 
> Like in the example above, probe_callback() can give those guarantees instead.

Ok, if you say so, should we bookmark this thread for when this does
happen?  :)

What will the rust code do if it is passed in a NULL pointer?  Will it
crash like C code does?  Or something else?

thanks,

greg k-h

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Greg KH]

On Tue, May 21, 2024 at 12:30:37AM +0200, Danilo Krummrich wrote:
> On Mon, May 20, 2024 at 08:14:18PM +0200, Greg KH wrote:
> > On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> > > From: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > 
> > > This defines general functionality related to registering drivers with
> > > their respective subsystems, and registering modules that implement
> > > drivers.
> > > 
> > > Co-developed-by: Asahi Lina <lina@asahilina.net>
> > > Signed-off-by: Asahi Lina <lina@asahilina.net>
> > > Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > > ---
> > >  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
> > >  rust/kernel/lib.rs           |   4 +-
> > >  rust/macros/module.rs        |   2 +-
> > >  samples/rust/rust_minimal.rs |   2 +-
> > >  samples/rust/rust_print.rs   |   2 +-
> > >  5 files changed, 498 insertions(+), 4 deletions(-)
> > >  create mode 100644 rust/kernel/driver.rs
> > > 
> > > diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> > > new file mode 100644
> > > index 000000000000..e0cfc36d47ff
> > > --- /dev/null
> > > +++ b/rust/kernel/driver.rs
> > > @@ -0,0 +1,492 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +
> > > +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> > > +//!
> > > +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> > > +//! using the [`Registration`] class.
> > 
> > Why are you creating new "names" here?  "DriverOps" is part of a 'struct
> > device_driver' why are you separating it out here?  And what is
> 
> DriverOps is a trait which abstracts a subsystems register() and unregister()
> functions to (un)register drivers. It exists such that a generic Registration
> implementation calls the correct one for the subsystem.
> 
> For instance, PCI would implement DriverOps::register() by calling into
> bindings::__pci_register_driver().
> 
> We can discuss whether DriverOps is a good name for the trait, but it's not a
> (different) name for something that already exists and already has a name.

It's a name we don't have in the C code as the design of the driver core
does not need or provide it.  It's just the section of 'struct
device_driver' that provides function callbacks, why does it need to be
separate at all?

> > 'Registration'?  That's a bus/class thing, not a driver thing.
> 
> A Registration is an object representation of the driver's registered state.

And that representation should not ever need to be tracked by the
driver, that's up to the driver core to track that.

> Having an object representation for that is basically the Rust way to manage the
> lifetime of this state.

This all should be a static chunk of read-only memory that should never
have a lifetime, why does it need to be managed at all?

> Once the Registration is dropped, the driver is
> unregistered. For instance, a PCI driver Registration can be bound to a module,
> such that the PCI driver is unregistered when the module is unloaded (the
> Registration is dropped in the module_exit() callback).

Ok, that's fine, but note that your module_exit() function will never be
called if your module_init() callback fails, so why do you need to track
this state?  Again, C drivers never need to track this state, why is
rust requiring more logic here for something that is NOT a dynamic chunk
of memory (or should not be a dynamic chunk of memory, let's get it
correct from the start and not require us to change things later on to
make it more secure).

> > And be very careful of the use of the word 'class' here, remember, there
> > is 'struct class' as part of the driver model :)
> 
> I think in this context "class" might be meant as something like "struct with
> methods", not sure what would be the best term to describe this.

"struct with methods" is nice :)

Again, 'class' means something different here in the driver model, so be
careful with terms, language matters, especially when many of our
developers do not have English as their native language.

> > > +/// The registration of a driver.
> > > +pub struct Registration<T: DriverOps> {
> > > +    is_registered: bool,
> > 
> > Why does a driver need to know if it is registered or not?  Only the
> > driver core cares about that, please do not expose that, it's racy and
> > should not be relied on.
> 
> We need it to ensure we do not try to register the same thing twice

Your logic in your code is wrong if you attempt to register it twice,
AND the driver core will return an error if you do so, so a driver
should not need to care about this.

> , some subsystems might just catch fire otherwise.

Which ones?

> > > +impl<T: DriverOps> Drop for Registration<T> {
> > > +    fn drop(&mut self) {
> > > +        if self.is_registered {
> > > +            // SAFETY: This path only runs if a previous call to `T::register` completed
> > > +            // successfully.
> > > +            unsafe { T::unregister(self.concrete_reg.get()) };
> > 
> > Can't the rust code ensure that this isn't run if register didn't
> > succeed?  Having a boolean feels really wrong here (can't that race?)
> 
> No, we want to automatically unregister once this object is destroyed, but not
> if it was never registered in the first place.
> 
> This shouldn't be racy, we only ever (un)register things in places like probe()
> / remove() or module_init() / module_exit().

probe/remove never calls driver_register/unregister on itself, so that's
not an issue.  module_init/exit() does not race with itself and again,
module_exit() is not called if module_init() fails.

Again, you are adding logic here that is not needed.  Or if it really is
needed, please explain why the C code does not need this today and let's
work to fix that.

> > > +        }
> > > +    }
> > > +}
> > > +
> > > +/// Conversion from a device id to a raw device id.
> > > +///
> > > +/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
> > > +/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
> > > +///
> > > +/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
> > > +/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
> > > +/// concrete types (which can still have const associated functions) instead of a trait.
> > > +///
> > > +/// # Safety
> > > +///
> > > +/// Implementers must ensure that:
> > > +///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
> > > +///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
> > > +///     that buses can recover the pointer to the data.
> > > +pub unsafe trait RawDeviceId {
> > > +    /// The raw type that holds the device id.
> > > +    ///
> > > +    /// Id tables created from [`Self`] are going to hold this type in its zero-terminated array.
> > > +    type RawType: Copy;
> > > +
> > > +    /// A zeroed-out representation of the raw device id.
> > > +    ///
> > > +    /// Id tables created from [`Self`] use [`Self::ZERO`] as the sentinel to indicate the end of
> > > +    /// the table.
> > > +    const ZERO: Self::RawType;
> > 
> > All busses have their own way of creating "ids" and that is limited to
> > the bus code itself, why is any of this in the rust side?  What needs
> > this?  A bus will create the id for the devices it manages, and can use
> > it as part of the name it gives the device (but not required), so all of
> > this belongs to the bus, NOT to a driver, or a device.
> 
> This is just a generalized interface which can be used by subsystems to
> implement the subsystem / bus specific ID type.

Please move this all to a different file as it has nothing to do with
the driver core bindings with the include/device/driver.h api.

Let's keep it simple and obvious please, and separate out things into
logical chunks, hopefully in the same way that the C apis are separated
out into.  That way we can properly review, understand, and most
importantly for all of us, maintain the code over the next 40+ years.

thanks,

greg k-h

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Greg KH]

On Tue, May 21, 2024 at 11:35:43AM +0200, Greg KH wrote:
> On Tue, May 21, 2024 at 12:30:37AM +0200, Danilo Krummrich wrote:
> > > > +impl<T: DriverOps> Drop for Registration<T> {
> > > > +    fn drop(&mut self) {
> > > > +        if self.is_registered {
> > > > +            // SAFETY: This path only runs if a previous call to `T::register` completed
> > > > +            // successfully.
> > > > +            unsafe { T::unregister(self.concrete_reg.get()) };
> > > 
> > > Can't the rust code ensure that this isn't run if register didn't
> > > succeed?  Having a boolean feels really wrong here (can't that race?)
> > 
> > No, we want to automatically unregister once this object is destroyed, but not
> > if it was never registered in the first place.
> > 
> > This shouldn't be racy, we only ever (un)register things in places like probe()
> > / remove() or module_init() / module_exit().
> 
> probe/remove never calls driver_register/unregister on itself, so that's
> not an issue.  module_init/exit() does not race with itself and again,
> module_exit() is not called if module_init() fails.
> 
> Again, you are adding logic here that is not needed.  Or if it really is
> needed, please explain why the C code does not need this today and let's
> work to fix that.

And, to be more explicit, a driver should not have any state of its own,
any "internal state" should always be bound to the device that it is
controlling, NOT generic to the driver itself, as a driver can, and
should, be able to control multiple devices all at the same time without
ever knowing anything is going on.  A driver is just code, not data or
state.

Yes, we have bad examples in the kernel where drivers do keep
independent state, but those are the exceptions, never the rule, and I
would argue, should be fixed to not do that.  Most were due to being
created before the driver model existed, or programmers being lazy :)

thanks,

greg k-h

Re: [RFC PATCH 11/11] rust: PCI: add BAR request and ioremap

[Danilo Krummrich]

On 5/21/24 01:27, Miguel Ojeda wrote:
> On Mon, May 20, 2024 at 7:27 PM Danilo Krummrich <dakr@redhat.com> wrote:
>> +        let barlen = pdev.resource_len(num)?;
>> +        if barlen == 0 {
>> +            return Err(ENOMEM);
>> +        }
>> +
>> +        // SAFETY:
>> +        // `pdev` is always valid.
> 
> Please explain why it is always valid -- the point of a `SAFETY`
> comment is not to say something is OK, but why it is so.
> 
>> +        // `barnr` is checked for validity at the top of the function.

I added pci::Device::resource_len(), since I needed to get the VRAM bar size in Nova.

pci::Device::resource_len() also needs to check for a valid  bar index and is used
above, hence the check is implicit. I just forgot to change this comment accordingly.

>> +    /// Returns the size of the given PCI bar resource.
>> +    pub fn resource_len(&self, bar: u8) -> Result<bindings::resource_size_t> {
> 
> Sometimes `bindings::` in signatures of public methods may be
> justified -- is it the case here? Or should a newtype or similar be
> provided instead? I only see this function being called inside the
> module, anyway.

I agree, I just added this function real quick to let Nova report the VRAM bar size
and forgot to clean this up.

> 
>> +    /// Mapps an entire PCI-BAR after performing a region-request on it.
> 
> Typo.
> 
> Cheers,
> Miguel
> 

Re: [RFC PATCH 10/11] rust: add basic abstractions for iomem operations

[Danilo Krummrich]

On Tue, May 21, 2024 at 11:18:04AM +0200, Miguel Ojeda wrote:
> On Tue, May 21, 2024 at 9:36 AM Philipp Stanner <pstanner@redhat.com> wrote:
> >
> > Justified questions – it is public because the Drop implementation for
> > pci::Bar requires the ioptr to pass it to pci_iounmap().
> >
> > The alternative would be to give pci::Bar a copy of ioptr (it's just an
> > integer after all), but that would also not be exactly beautiful.
> 
> If by copy you mean keeping an actual copy elsewhere, then you could
> provide an access method instead.

As mentioned earlier, given the context how we use IoMem, I think IoMem should
just be a trait. And given that, maybe we'd want to name this trait differently
then, something like `trait IoOps` maybe?

pub trait IoOps {
   // INVARIANT: The implementation must ensure that the returned value is
   // either an error code or a non-null and valid address suitable for  I/O
   // operations of the given offset and length.
   fn io_addr(&self, offset: usize, len: usize) -> Result<usize>;

   fn readb(&self, offset: usize) -> Result<u8> {
      let addr = self.io_addr(offset, 1)?;

      // SAFETY: `addr` is guaranteed to be valid as by the invariant required
      // by `io_addr`.
      Ok(unsafe { bindings::readb(addr as _) })
   }

   [...]
}

We can let the resource type (e.g. `pci::Bar`) track the base address and limit
instead and just let pci::Bar implement `IoMem::io_addr`.

As for the compile time size, this would be up the the actual resource then.
`pci::Bar` can't make use of this optimization, while others might be able to.

Does that sound reasonable?

- Danilo

Re: [RFC PATCH 01/11] rust: add abstraction for struct device

[Danilo Krummrich]

On Tue, May 21, 2024 at 11:24:38AM +0200, Greg KH wrote:
> On Mon, May 20, 2024 at 10:22:22PM +0200, Danilo Krummrich wrote:
> > On Mon, May 20, 2024 at 08:00:23PM +0200, Greg KH wrote:
> > > On Mon, May 20, 2024 at 07:25:38PM +0200, Danilo Krummrich wrote:
> > > > Add an (always) reference counted abstraction for a generic struct
> > > > device. This abstraction encapsulates existing struct device instances
> > > > and manages its reference count.
> > > > 
> > > > Subsystems may use this abstraction as a base to abstract subsystem
> > > > specific device instances based on a generic struct device.
> > > > 
> > > > Co-developed-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > > > ---
> > > >  rust/helpers.c        |  1 +
> > > >  rust/kernel/device.rs | 76 +++++++++++++++++++++++++++++++++++++++++++
> > > 
> > > What's the status of moving .rs files next to their respective .c files
> > > in the build system?  Keeping them separate like this just isn't going
> > > to work, sorry.
> > > 
> > > > --- /dev/null
> > > > +++ b/rust/kernel/device.rs
> > > > @@ -0,0 +1,76 @@
> > > > +// SPDX-License-Identifier: GPL-2.0
> > > > +
> > > > +//! Generic devices that are part of the kernel's driver model.
> > > > +//!
> > > > +//! C header: [`include/linux/device.h`](../../../../include/linux/device.h)
> > > 
> > > relative paths for a common "include <linux/device.h" type of thing?
> > > Rust can't handle include paths from directories?
> > 
> > Going to change this to `srctree/` as proposed by Miguel.
> > 
> > > 
> > > > +
> > > > +use crate::{
> > > > +    bindings,
> > > > +    types::{ARef, Opaque},
> > > > +};
> > > > +use core::ptr;
> > > > +
> > > > +/// A ref-counted device.
> > > > +///
> > > > +/// # Invariants
> > > > +///
> > > > +/// The pointer stored in `Self` is non-null and valid for the lifetime of the ARef instance. In
> > > > +/// particular, the ARef instance owns an increment on underlying object’s reference count.
> > > > +#[repr(transparent)]
> > > > +pub struct Device(Opaque<bindings::device>);
> > > > +
> > > > +impl Device {
> > > > +    /// Creates a new ref-counted instance of an existing device pointer.
> > > > +    ///
> > > > +    /// # Safety
> > > > +    ///
> > > > +    /// Callers must ensure that `ptr` is valid, non-null, and has a non-zero reference count.
> > > 
> > > Callers NEVER care about the reference count of a struct device, anyone
> > > poking in that is asking for trouble.
> > 
> > That's confusing, if not the caller who's passing the device pointer somewhere,
> > who else?
> > 
> > Who takes care that a device' reference count is non-zero when a driver's probe
> > function is called?
> 
> A device's reference count will be non-zero, I'm saying that sometimes,
> some driver core functions are called with a 'struct device' that is
> NULL, and it can handle it just fine.  Hopefully no callbacks to the
> rust code will happen that way, but why aren't you checking just "to be
> sure!" otherwise you could have a bug here, and it costs nothing to
> verify it, right?

I get your point on that one. But let me explain a bit more why I think that
check is not overly helpful here.

In Rust we have the concept of marking functions as 'unsafe'. Unsafe functions
need to document their safety preconsitions, i.e. the conditions the caller of
the function must guarantee. The caller of an unsafe function needs an unsafe
block for it to compile and every unsafe block needs an explanation why it is
safe to call this function with the corresponding arguments.

(Ideally, we want to avoid having them in the first place, but for C abstractions
we have to deal with raw pointers we receive from the C side and dereferencing a
raw pointer is unsafe by definition.)

In this case we have a function that constructs the Rust `Device` structure from
a raw (device) pointer we potentially received from the C side. Now we have to
decide whether this function is going to be unsafe or safe.

In order for this function to be safe we would need to be able to guarantee that
this is a valid, non-null pointer with a non-zero reference count, which
unfortunately we can't. Hence, it's going to be an unsafe function.

A NULL pointer check would not make it a safe function either, since the pointer
could still be an invalid one, or a pointer to a device it's not guaranteed that
the reference count is held up for the duration of the function call.

Given that, we could add the NULL check and change the safety precondition to
"valid pointer to a device with non-zero reference count OR NULL", but I don't
see how this improves the situation for the caller, plus we'd need to return
`Result<Device>` instead and let the caller handle that the `Device` was not
created.

> 
> > It's the same here. The PCI code calls Device::from_raw() from its
> > probe_callback() function, which is called from the C side. For instance:
> > 
> > extern "C" fn probe_callback(
> >    pdev: *mut bindings::pci_dev,
> >    id: *const bindings::pci_device_id,
> > ) -> core::ffi::c_int {
> >    // SAFETY: This is safe, since the C side guarantees that pdev is a valid,
> >    // non-null pointer to a struct pci_dev with a non-zero reference count.
> >    let dev = unsafe { device::Device::from_raw(&mut (*pdev).dev) };
> 
> Yes, that's fine, if you are calling this from a probe callback, but
> again, the driver core has lots of functions in it :)

See explanation above.

> 
> >    [...]
> > }
> > 
> > > 
> > > And why non-NULL?  Can't you check for that here?  Shouldn't you check
> > > for that here?  Many driver core functions can handle a NULL pointer
> > > just fine (i.e. get/put_device() can), why should Rust code assume that
> > > a pointer passed to it from the C layer is going to have stricter rules
> > > than the C layer can provide?
> > 
> > We could check for NULL here, but I think it'd be pointless. Even if the pointer
> > is not NULL, it can still be an invalid one. There is no check we can do to
> > guarantee safety, hence the function is and remains unsafe and has safety
> > requirements instead that the caller must guarantee to fulfil.
> > 
> > Like in the example above, probe_callback() can give those guarantees instead.
> 
> Ok, if you say so, should we bookmark this thread for when this does
> happen?  :)

I'm just saying the caller has to validate that or provide a rationale why this
is safe anyways, hence it'd be just a duplicate check.

> 
> What will the rust code do if it is passed in a NULL pointer?  Will it
> crash like C code does?  Or something else?

It mostly calls into C functions with this pointer, depends on what they do.

Checking a few random places, e.g. [1], it seems to crash in most cases.

[1] https://elixir.free-electrons.com/linux/latest/source/drivers/base/core.c#L3863

> 
> thanks,
> 
> greg k-h
> 

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Danilo Krummrich]

On Tue, May 21, 2024 at 11:35:43AM +0200, Greg KH wrote:
> On Tue, May 21, 2024 at 12:30:37AM +0200, Danilo Krummrich wrote:
> > On Mon, May 20, 2024 at 08:14:18PM +0200, Greg KH wrote:
> > > On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> > > > From: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > 
> > > > This defines general functionality related to registering drivers with
> > > > their respective subsystems, and registering modules that implement
> > > > drivers.
> > > > 
> > > > Co-developed-by: Asahi Lina <lina@asahilina.net>
> > > > Signed-off-by: Asahi Lina <lina@asahilina.net>
> > > > Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > > > ---
> > > >  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
> > > >  rust/kernel/lib.rs           |   4 +-
> > > >  rust/macros/module.rs        |   2 +-
> > > >  samples/rust/rust_minimal.rs |   2 +-
> > > >  samples/rust/rust_print.rs   |   2 +-
> > > >  5 files changed, 498 insertions(+), 4 deletions(-)
> > > >  create mode 100644 rust/kernel/driver.rs
> > > > 
> > > > diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> > > > new file mode 100644
> > > > index 000000000000..e0cfc36d47ff
> > > > --- /dev/null
> > > > +++ b/rust/kernel/driver.rs
> > > > @@ -0,0 +1,492 @@
> > > > +// SPDX-License-Identifier: GPL-2.0
> > > > +
> > > > +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> > > > +//!
> > > > +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> > > > +//! using the [`Registration`] class.
> > > 
> > > Why are you creating new "names" here?  "DriverOps" is part of a 'struct
> > > device_driver' why are you separating it out here?  And what is
> > 
> > DriverOps is a trait which abstracts a subsystems register() and unregister()
> > functions to (un)register drivers. It exists such that a generic Registration
> > implementation calls the correct one for the subsystem.
> > 
> > For instance, PCI would implement DriverOps::register() by calling into
> > bindings::__pci_register_driver().
> > 
> > We can discuss whether DriverOps is a good name for the trait, but it's not a
> > (different) name for something that already exists and already has a name.
> 
> It's a name we don't have in the C code as the design of the driver core
> does not need or provide it.  It's just the section of 'struct
> device_driver' that provides function callbacks, why does it need to be
> separate at all?

I'm confused by the relationship to `struct device_driver` you seem to imply.
How is it related?

Again, this is just a trait for subsystems to provide their corresponding
register and unregister implementation, e.g. pci_register_driver() and
pci_unregister_driver(), such that they can be called from the generic
Registration code below.

See [1] for an example implementation in PCI.

Please also consider that some structures might be a 1:1 representation of C
structures, some C structures are not required at the Rust side at all, and
then there might be additional structures and traits that abstract things C has
no data structure for.

This is due to the fact that 1. we're not replicating the functionality on the C
side, but only make use of it, and 2. we're trying to abstract existing code to
make it work with a conceptually different language. Which means, if you find
something that's not on the C side, it doesn't (necessarily) mean we're making
something up that should either not exist or be on the C side as well.

I'm not saying don't question it, I appreciate that, and it's even the whole
reason for sending this RFC, but I ask you to consider this fact. :)

[1] https://lore.kernel.org/rust-for-linux/20240520172554.182094-10-dakr@redhat.com/

> 
> > > 'Registration'?  That's a bus/class thing, not a driver thing.
> > 
> > A Registration is an object representation of the driver's registered state.
> 
> And that representation should not ever need to be tracked by the
> driver, that's up to the driver core to track that.

The driver doesn't need it, the Registration abstraction does need it. Please
see my comments below.

> 
> > Having an object representation for that is basically the Rust way to manage the
> > lifetime of this state.
> 
> This all should be a static chunk of read-only memory that should never
> have a lifetime, why does it need to be managed at all?

What I meant here is that if a driver was registered, we need to make sure it's
going to be unregistered eventually, e.g. when the module is removed or when
something fails after registration and we need to unwind.

When the Registration structure goes out of scope, which would happen in both
the cases above, it will automatically unregister the driver, due to the
automatic call to `drop()`.

> 
> > Once the Registration is dropped, the driver is
> > unregistered. For instance, a PCI driver Registration can be bound to a module,
> > such that the PCI driver is unregistered when the module is unloaded (the
> > Registration is dropped in the module_exit() callback).
> 
> Ok, that's fine, but note that your module_exit() function will never be
> called if your module_init() callback fails, so why do you need to track
> this state?  Again, C drivers never need to track this state, why is
> rust requiring more logic here for something that is NOT a dynamic chunk
> of memory (or should not be a dynamic chunk of memory, let's get it
> correct from the start and not require us to change things later on to
> make it more secure).

That's fine, if module_init() would fail, the Registration would be dropped as
well.

As for why doesn't C need this is a good example of what I wrote above. Because
it is useful for Rust, but not for C.

In Rust we get drop() automatically called when a structure is destroyed. This
means that if we let drivers put the Registration structure (e.g. representing
that a PCI driver was registered) into its `Module` representation structure
(already upstream) then this Registration is automatically destroyed once the
module representation is destroyed (which happens on module_exit()). This leads
to `drop()` of the `Registration` structure being called, which unregisteres the
(e.g. PCI) driver.

This way the driver does not need to take care of unregistering the PCI driver
explicitly. The driver can also place multiple registrations into the `Module`
structure. All of them would be unregistered automatically in module_exit().

> 
> > > And be very careful of the use of the word 'class' here, remember, there
> > > is 'struct class' as part of the driver model :)
> > 
> > I think in this context "class" might be meant as something like "struct with
> > methods", not sure what would be the best term to describe this.
> 
> "struct with methods" is nice :)

Great, I will change that.

> 
> Again, 'class' means something different here in the driver model, so be
> careful with terms, language matters, especially when many of our
> developers do not have English as their native language.
> 
> > > > +/// The registration of a driver.
> > > > +pub struct Registration<T: DriverOps> {
> > > > +    is_registered: bool,
> > > 
> > > Why does a driver need to know if it is registered or not?  Only the
> > > driver core cares about that, please do not expose that, it's racy and
> > > should not be relied on.
> > 
> > We need it to ensure we do not try to register the same thing twice
> 
> Your logic in your code is wrong if you attempt to register it twice,
> AND the driver core will return an error if you do so, so a driver
> should not need to care about this.

We want it to be safe, if the driver logic is wrong and registers it twice, we
don't want it to blow up.

The driver core takes care, but I think there are subsystems that do
initializations that could make things blow up when registering the driver
twice.

> 
> > , some subsystems might just catch fire otherwise.
> 
> Which ones?

Let's take the one we provide abstractons for, PCI.

In __pci_register_driver() we call spin_lock_init() and INIT_LIST_HEAD() before
driver_register() could bail out [1].

What if this driver is already registered and in use and we're randomly altering
the list pointers or call spin_lock_init() on a spin lock that's currently being
held?

[1] https://elixir.bootlin.com/linux/latest/source/drivers/pci/pci-driver.c#L1447

> 
> > > > +impl<T: DriverOps> Drop for Registration<T> {
> > > > +    fn drop(&mut self) {
> > > > +        if self.is_registered {
> > > > +            // SAFETY: This path only runs if a previous call to `T::register` completed
> > > > +            // successfully.
> > > > +            unsafe { T::unregister(self.concrete_reg.get()) };
> > > 
> > > Can't the rust code ensure that this isn't run if register didn't
> > > succeed?  Having a boolean feels really wrong here (can't that race?)
> > 
> > No, we want to automatically unregister once this object is destroyed, but not
> > if it was never registered in the first place.
> > 
> > This shouldn't be racy, we only ever (un)register things in places like probe()
> > / remove() or module_init() / module_exit().
> 
> probe/remove never calls driver_register/unregister on itself, so that's
> not an issue.  module_init/exit() does not race with itself and again,
> module_exit() is not called if module_init() fails.

That's clear, I should mention that we can use the Registration abstraction also
for things that are typically registered in probe(), a DRM device for instance.

As mentioned above I'm aware that module_exit() is not called when module_init()
fails, in this case the Registration structure is dropped in module_init(),
hence it's fine.

> 
> Again, you are adding logic here that is not needed.  Or if it really is
> needed, please explain why the C code does not need this today and let's
> work to fix that.

Please see above, where I start with "As for why doesn't C need this..".

> 
> > > > +        }
> > > > +    }
> > > > +}
> > > > +
> > > > +/// Conversion from a device id to a raw device id.
> > > > +///
> > > > +/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
> > > > +/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
> > > > +///
> > > > +/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
> > > > +/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
> > > > +/// concrete types (which can still have const associated functions) instead of a trait.
> > > > +///
> > > > +/// # Safety
> > > > +///
> > > > +/// Implementers must ensure that:
> > > > +///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
> > > > +///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
> > > > +///     that buses can recover the pointer to the data.
> > > > +pub unsafe trait RawDeviceId {
> > > > +    /// The raw type that holds the device id.
> > > > +    ///
> > > > +    /// Id tables created from [`Self`] are going to hold this type in its zero-terminated array.
> > > > +    type RawType: Copy;
> > > > +
> > > > +    /// A zeroed-out representation of the raw device id.
> > > > +    ///
> > > > +    /// Id tables created from [`Self`] use [`Self::ZERO`] as the sentinel to indicate the end of
> > > > +    /// the table.
> > > > +    const ZERO: Self::RawType;
> > > 
> > > All busses have their own way of creating "ids" and that is limited to
> > > the bus code itself, why is any of this in the rust side?  What needs
> > > this?  A bus will create the id for the devices it manages, and can use
> > > it as part of the name it gives the device (but not required), so all of
> > > this belongs to the bus, NOT to a driver, or a device.
> > 
> > This is just a generalized interface which can be used by subsystems to
> > implement the subsystem / bus specific ID type.
> 
> Please move this all to a different file as it has nothing to do with
> the driver core bindings with the include/device/driver.h api.

I don't think driver.rs in an unreasonable place for a generic device ID
representation that is required by every driver structure.

But I'm also not overly resistant to move it out. What do you think would be a
good name?

Please consider that this name will also be the namespace for this trait.
Currently you can reference it with `kernel::driver::RawDeviceId`. If you move
it to foo.rs, it'd be `kernel::foo::RawDeviceId`.

> 
> Let's keep it simple and obvious please, and separate out things into
> logical chunks, hopefully in the same way that the C apis are separated
> out into.  That way we can properly review, understand, and most
> importantly for all of us, maintain the code over the next 40+ years.
> 
> thanks,
> 
> greg k-h
> 

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Danilo Krummrich]

On Tue, May 21, 2024 at 10:04:02AM +0200, Greg KH wrote:
> On Tue, May 21, 2024 at 03:42:50PM +1000, Dave Airlie wrote:
> > On Tue, 21 May 2024 at 04:14, Greg KH <gregkh@linuxfoundation.org> wrote:
> > >
> > > On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> > > > From: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > >
> > > > This defines general functionality related to registering drivers with
> > > > their respective subsystems, and registering modules that implement
> > > > drivers.
> > > >
> > > > Co-developed-by: Asahi Lina <lina@asahilina.net>
> > > > Signed-off-by: Asahi Lina <lina@asahilina.net>
> > > > Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > > > ---
> > > >  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
> > > >  rust/kernel/lib.rs           |   4 +-
> > > >  rust/macros/module.rs        |   2 +-
> > > >  samples/rust/rust_minimal.rs |   2 +-
> > > >  samples/rust/rust_print.rs   |   2 +-
> > > >  5 files changed, 498 insertions(+), 4 deletions(-)
> > > >  create mode 100644 rust/kernel/driver.rs
> > > >
> > > > diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> > > > new file mode 100644
> > > > index 000000000000..e0cfc36d47ff
> > > > --- /dev/null
> > > > +++ b/rust/kernel/driver.rs
> > > > @@ -0,0 +1,492 @@
> > > > +// SPDX-License-Identifier: GPL-2.0
> > > > +
> > > > +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> > > > +//!
> > > > +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> > > > +//! using the [`Registration`] class.
> > >
> > > Why are you creating new "names" here?  "DriverOps" is part of a 'struct
> > > device_driver' why are you separating it out here?  And what is
> > > 'Registration'?  That's a bus/class thing, not a driver thing.
> > >
> > > And be very careful of the use of the word 'class' here, remember, there
> > > is 'struct class' as part of the driver model :)
> > >
> > > > +use crate::{
> > > > +    alloc::{box_ext::BoxExt, flags::*},
> > > > +    error::code::*,
> > > > +    error::Result,
> > > > +    str::CStr,
> > > > +    sync::Arc,
> > > > +    ThisModule,
> > > > +};
> > > > +use alloc::boxed::Box;
> > > > +use core::{cell::UnsafeCell, marker::PhantomData, ops::Deref, pin::Pin};
> > > > +
> > > > +/// A subsystem (e.g., PCI, Platform, Amba, etc.) that allows drivers to be written for it.
> > > > +pub trait DriverOps {
> > >
> > > Again, why is this not called DeviceDriver?
> > 
> > This is not the same as the C device_driver, it might mildly align in
> > concept with it, but I'm not sure it shares enough to align it name
> > wise with the C one.
> 
> Why not use the same terms and design decisions that the C code has?  To
> differ needs to have a good reason, otherwise it's just going to cause
> us all confusion as we have to learn two different terms for the same
> thing.

Please see my reply [1] for that one. Let's continue this in the other thread.

[1] https://lore.kernel.org/rust-for-linux/Zk0egew_AxvNpUG-@pollux/

> 
> > > > +    /// The type that holds information about the registration. This is typically a struct defined
> > > > +    /// by the C portion of the kernel.
> > > > +    type RegType: Default;
> > > > +
> > > > +    /// Registers a driver.
> > > > +    ///
> > > > +    /// # Safety
> > > > +    ///
> > > > +    /// `reg` must point to valid, initialised, and writable memory. It may be modified by this
> > > > +    /// function to hold registration state.
> > > > +    ///
> > > > +    /// On success, `reg` must remain pinned and valid until the matching call to
> > > > +    /// [`DriverOps::unregister`].
> > > > +    unsafe fn register(
> > > > +        reg: *mut Self::RegType,
> > > > +        name: &'static CStr,
> > > > +        module: &'static ThisModule,
> > > > +    ) -> Result;
> > > > +
> > > > +    /// Unregisters a driver previously registered with [`DriverOps::register`].
> > > > +    ///
> > > > +    /// # Safety
> > > > +    ///
> > > > +    /// `reg` must point to valid writable memory, initialised by a previous successful call to
> > > > +    /// [`DriverOps::register`].
> > > > +    unsafe fn unregister(reg: *mut Self::RegType);
> > > > +}
> > > > +
> > > > +/// The registration of a driver.
> > > > +pub struct Registration<T: DriverOps> {
> > > > +    is_registered: bool,
> > >
> > > Why does a driver need to know if it is registered or not?  Only the
> > > driver core cares about that, please do not expose that, it's racy and
> > > should not be relied on.
> > 
> > >From the C side this does look unusual because on the C side something
> > like struct pci_driver is statically allocated everywhere.
> > In this rust abstraction, these are allocated dynamically, so instead
> > of just it always being safe to just call register/unregister
> > with static memory, a flag is kept around to say if the unregister
> > should happen at all, as the memory may have
> > been allocated but never registered. This is all the Registration is
> > for, it's tracking the bus _driver structure allocation, and
> > whether the bus register/unregister have been called since the object
> > was allocated.
> > 
> > I'm not sure it makes sense (or if you can at all), have a static like
> > pci_driver object here to match how C does things.
> 
> Wait, why can't you have a static "rust_driver" type thing?  Having it
> be in ram feels like a waste of memory.  We are working hard to move
> more and more of these driver model structures into read-only memory for
> good reasons (security, reduce bugs, etc.), moving backwards to having
> them all be dynamically created/copied around feels wrong.

I think this would be possible, though it might cause some inconvenience and a
and maybe a few trade offs to do that on the Rust side.

However, I agree that we should explore how to do it in a static way. I will dig
in a bit more on the options we have.

> 
> We are one stage away from being able to mark all 'driver_*()' calls as
> using a const * to struct driver, please don't make that work pointless
> if you want to write a driver in rust instead.
> 
> And again, a driver should never know/care/wonder if it has been
> registered or not, that is up to the driver core to handle, not the rust
> code, as it is the only thing that can know this, and the only thing
> that should need to know it.  A driver structure should not have any
> dynamic memory associated with it to require knowing its "state" in the
> system, so let's not move backwards here to require that just because we
> are using Rust.

I answered on this in [1], let's continue there. This also accounts for all
points below I don't address. :)

[1] https://lore.kernel.org/rust-for-linux/Zk0egew_AxvNpUG-@pollux/

> 
> > > > +impl<T: DriverOps> Drop for Registration<T> {
> > > > +    fn drop(&mut self) {
> > > > +        if self.is_registered {
> > > > +            // SAFETY: This path only runs if a previous call to `T::register` completed
> > > > +            // successfully.
> > > > +            unsafe { T::unregister(self.concrete_reg.get()) };
> > >
> > > Can't the rust code ensure that this isn't run if register didn't
> > > succeed?  Having a boolean feels really wrong here (can't that race?)
> > 
> > There might be a way of using Option<> here but I don't think it adds
> > anything over and above using an explicit bool.
> 
> Again, this should not be needed.  If so, something is designed
> incorrectly with the bindings.
> 
> > > > +///
> > > > +/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
> > > > +/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
> > > > +///
> > > > +/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
> > > > +/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
> > > > +/// concrete types (which can still have const associated functions) instead of a trait.
> > > > +///
> > > > +/// # Safety
> > > > +///
> > > > +/// Implementers must ensure that:
> > > > +///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
> > > > +///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
> > > > +///     that buses can recover the pointer to the data.
> > > > +pub unsafe trait RawDeviceId {
> > > > +    /// The raw type that holds the device id.
> > > > +    ///
> > > > +    /// Id tables created from [`Self`] are going to hold this type in its zero-terminated array.
> > > > +    type RawType: Copy;
> > > > +
> > > > +    /// A zeroed-out representation of the raw device id.
> > > > +    ///
> > > > +    /// Id tables created from [`Self`] use [`Self::ZERO`] as the sentinel to indicate the end of
> > > > +    /// the table.
> > > > +    const ZERO: Self::RawType;
> > >
> > > All busses have their own way of creating "ids" and that is limited to
> > > the bus code itself, why is any of this in the rust side?  What needs
> > > this?  A bus will create the id for the devices it manages, and can use
> > > it as part of the name it gives the device (but not required), so all of
> > > this belongs to the bus, NOT to a driver, or a device.
> > 
> > Consider this a base class (Trait) for bus specific IDs.
> 
> Then all of that should be in a separate file as they belong to a "bus"
> not a driver, as each and every bus will have a different way of
> expressing the list of devices a driver can bind to.
> 
> And again, the core "struct device_driver" does not have a list of ids,
> and neither should the rust bindings, that belongs to the bus logic.
> 
> > > > +/// Custom code within device removal.
> > >
> > > You better define the heck out of "device removal" as specified last
> > > time this all came up.  From what I can see here, this is totally wrong
> > > and confusing and will be a mess.
> > >
> > > Do it right, name it properly.
> > >
> > > I'm not reviewingn beyond here, sorry.  It's the merge window and I
> > > shouldn't have even looked at this until next week anyway.
> > >
> > > But I was hoping that the whole long rant I gave last time would be
> > > addressed at least a little bit.  I don't see that it has :(
> > 
> > I won't comment too much on the specifics here, but you've twice got
> > to this stage, said something is wrong, change it, and given no
> > actionable feedback on what is wrong, or what to change.
> > 
> > I've looked at this code for a few hours today with the device docs
> > and core code and can't spot what you are seeing that is wrong here,
> > which means I don't expect anyone else is going to unless you can help
> > educate us more.
> 
> A lifecycle of a device is:
> 	device create by the bus
> 	device register with driver core
> 	device bind to a driver (i.e. probe() callback in the driver)
> 	device unbind from a driver (i.e. remove() callback in the driver, can be triggered many ways)
> 	device destroy by the bus
> 
> "remove" can happen for a driver where it needs to clean up everything
> it has done for a specific device, but that does not mean the device is
> actually gone from the system, that's up to the bus to decide, as it
> owns the device lifecycle and gets to decide when it thinks it is really
> gone.  And then, when the bus tells the driver core that it wants to
> mark the device as "destroyed", it's up to the driver core to eventually
> call back into the bus to really clean that device up from the system at
> some later point in time.

That matches at least my understanding and I don't see how the name of the
`DeviceRemoval` trait for instance is contradictive to that.

Again, the C side uses the exact same terminology in struct device_driver and
struct pci_driver, e.g. [1][2].

If it's wrong there as well, how would you phrase it differently? I'm happy to
fix those places as well.

If it's correct there, why is it wrong here? Where is the difference? May I ask
you for a proposal on how to name the `DeviceRemoval` trait instead? Maybe
something like `DeviceUnbind`?

Again, I'm not generally resisting to change it, but I think we should clarify
the above.

[1] https://elixir.bootlin.com/linux/latest/source/include/linux/device/driver.h#L71
[2] https://elixir.bootlin.com/linux/latest/source/include/linux/pci.h#L905

> 
> Try splitting this file out into driver and bus and device logic, like
> the driver core has, and see if that helps in explaining and making all
> of this more understandable, and to keep with the names/design of the
> model we currently have.
> 
> Note, all of this is my biggest worry about writing a driver in rust,
> getting the lifecycle of the driver core and it's logic of how it
> handles memory manged in C, to align up with how rust is going to access
> it with its different rules and requirements, is not going to be simple
> as you have two different models intersecting at the same place.  I've
> been working over the past year to make the rust side happen easier by
> marking more and more of the C structures as "not mutable", i.e. const
> *, so that the Rust side doesn't have to worry that the driver core
> really will change things it passes to the C side, but there is more to
> be done (see above about making struct device_driver * const).
> 
> I want to see this happen, but it's going to be a hard slog due to the
> different expectations of these two systems.  Keeping naming identical
> where possible is one way to make it simpler for everyone involved, as
> would splitting the logic out the same way and not mixing it all up into
> one big hairy file that combines different things into a single chunk.
> 
> thanks,
> 
> greg k-h
> 

Re: [RFC PATCH 10/11] rust: add basic abstractions for iomem operations

[Wedson Almeida Filho]

On Tue, 21 May 2024 at 05:03, Philipp Stanner <pstanner@redhat.com> wrote:
>
> On Tue, 2024-05-21 at 00:01 -0300, Wedson Almeida Filho wrote:
> > On Mon, 20 May 2024 at 23:07, Dave Airlie <airlied@gmail.com> wrote:
> > >
> > > >
> > > > Wedson wrote a similar abstraction in the past
> > > > (`rust/kernel/io_mem.rs` in the old `rust` branch), with a
> > > > compile-time `SIZE` -- it is probably worth taking a look.
> > > >
> > >
> > > Just on this point, we can't know in advance what size the IO BARs
> > > are
> > > at compile time.
> > >
> > > The old method just isn't useful for real devices with runtime IO
> > > BAR sizes.
> >
> > The compile-time `SIZE` in my implementation is a minimum size.
> >
> > Attempts to read/write with constants within that size (offset +
> > size)
> > were checked at compile time, that is, they would have zero
> > additional
> > runtime cost when compared to C. Reads/writes beyond the minimum
> > would
> > have to be checked at runtime.
> >
>
> We looked at this implementation
>
> Its disadvantage is that it moves the responsibility for setting that
> minimum size to the driver programmer. Andreas Hindborg is using that
> currently for rnvme [1].
>
> I believe that the driver programmer in Rust should not be responsible
> for controlling such sensitive parameters (one could far more easily
> provide invalid values), but the subsystem (e.g. PCI) should do it,
> because it knows about the exact resource lengths.

There is no responsibility being moved. The bus is still that one that
knows about the resources attached to the device.

The driver, however, can say for example: I need at least 4 registers
of 32 bits starting at offset X, which results in a minimum size of X
+ 16. If at runtime a device compatible with this driver appears and
has an io mem of at least that size, then the driver can drive it
without any additional runtime checks. I did this in the gpio driver
here: https://lwn.net/Articles/863459/

Note that in addition to not having to check offset at runtime, the
reads/writes are also infallible because all failures are caught at
compile time.

Obviously not all drivers can benefit from this, but it is
considerable simplification for the ones that can.

> The only way to set the actual, real value is through subsystem code.
> But when we (i.e., currently, the driver programmer) have to use that
> anyways, we can just use it from the very beginning and have the exact
> valid parameters.

Yes, only the bus knows. But to reiterate: if the driver declares and
checks a minimum size at attach time, it obviates the needs to check
again throughout the lifetime of the driver, which is more performant
and eliminates error paths.

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Dirk Behme]

On 20.05.2024 19:25, Danilo Krummrich wrote:
> From: Wedson Almeida Filho <wedsonaf@gmail.com>
> 
> This defines general functionality related to registering drivers with
> their respective subsystems, and registering modules that implement
> drivers.
> 
> Co-developed-by: Asahi Lina <lina@asahilina.net>
> Signed-off-by: Asahi Lina <lina@asahilina.net>
> Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> ---
>   rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
>   rust/kernel/lib.rs           |   4 +-
>   rust/macros/module.rs        |   2 +-
>   samples/rust/rust_minimal.rs |   2 +-
>   samples/rust/rust_print.rs   |   2 +-
>   5 files changed, 498 insertions(+), 4 deletions(-)
>   create mode 100644 rust/kernel/driver.rs
> 
> diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> new file mode 100644
> index 000000000000..e0cfc36d47ff
> --- /dev/null
> +++ b/rust/kernel/driver.rs
> @@ -0,0 +1,492 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> +//!
> +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> +//! using the [`Registration`] class.
> +
> +use crate::{
> +    alloc::{box_ext::BoxExt, flags::*},
> +    error::code::*,
> +    error::Result,
> +    str::CStr,
> +    sync::Arc,
> +    ThisModule,
> +};
> +use alloc::boxed::Box;
> +use core::{cell::UnsafeCell, marker::PhantomData, ops::Deref, pin::Pin};
> +
> +/// A subsystem (e.g., PCI, Platform, Amba, etc.) that allows drivers to be written for it.
> +pub trait DriverOps {
> +    /// The type that holds information about the registration. This is typically a struct defined
> +    /// by the C portion of the kernel.
> +    type RegType: Default;
> +
> +    /// Registers a driver.
> +    ///
> +    /// # Safety
> +    ///
> +    /// `reg` must point to valid, initialised, and writable memory. It may be modified by this
> +    /// function to hold registration state.
> +    ///
> +    /// On success, `reg` must remain pinned and valid until the matching call to
> +    /// [`DriverOps::unregister`].
> +    unsafe fn register(
> +        reg: *mut Self::RegType,
> +        name: &'static CStr,
> +        module: &'static ThisModule,
> +    ) -> Result;
> +
> +    /// Unregisters a driver previously registered with [`DriverOps::register`].
> +    ///
> +    /// # Safety
> +    ///
> +    /// `reg` must point to valid writable memory, initialised by a previous successful call to
> +    /// [`DriverOps::register`].
> +    unsafe fn unregister(reg: *mut Self::RegType);
> +}
> +
> +/// The registration of a driver.
> +pub struct Registration<T: DriverOps> {
> +    is_registered: bool,
> +    concrete_reg: UnsafeCell<T::RegType>,
> +}
> +
> +// SAFETY: `Registration` has no fields or methods accessible via `&Registration`, so it is safe to
> +// share references to it with multiple threads as nothing can be done.
> +unsafe impl<T: DriverOps> Sync for Registration<T> {}


You might want to check if we additionally need 'Send' due to

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=323617f649c0966ad5e741e47e27e06d3a680d8f

here?

+ unsafe impl<T: DriverOps> Send for Registration<T> {}

This was found re-basing

https://github.com/Rust-for-Linux/linux/commits/staging/rust-device/

to v6.10-rc1.

Sorry if I missed anything ;)

Best regards

Dirk

Re: [RFC PATCH 08/11] rust: add devres abstraction

[Dirk Behme]

On 20.05.2024 19:25, Danilo Krummrich wrote:
> Add a Rust abstraction for the kernel's devres (device resource
> management) implementation.
> 
> The Devres type acts as a container to manage the lifetime and
> accessibility of device bound resources. Therefore it registers a
> devres callback and revokes access to the resource on invocation.
> 
> Users of the Devres abstraction can simply free the corresponding
> resources in their Drop implementation, which is invoked when either the
> Devres instance goes out of scope or the devres callback leads to the
> resource being revoked, which implies a call to drop_in_place().
> 
> Co-developed-by: Philipp Stanner <pstanner@redhat.com>
> Signed-off-by: Philipp Stanner <pstanner@redhat.com>
> Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> ---
>   rust/helpers.c        |   5 ++
>   rust/kernel/devres.rs | 151 ++++++++++++++++++++++++++++++++++++++++++
>   rust/kernel/lib.rs    |   1 +
>   3 files changed, 157 insertions(+)
>   create mode 100644 rust/kernel/devres.rs
> 
> diff --git a/rust/helpers.c b/rust/helpers.c
> index 1d3e800140fc..34061eca05a0 100644
> --- a/rust/helpers.c
> +++ b/rust/helpers.c
> @@ -173,6 +173,11 @@ void rust_helper_rcu_read_unlock(void)
>   EXPORT_SYMBOL_GPL(rust_helper_rcu_read_unlock);
>   /* end rcu */
>   
> +int rust_helper_devm_add_action(struct device *dev, void (*action)(void *), void *data)
> +{
> +	return devm_add_action(dev, action, data);
> +}
> +
>   /*

Is it intended to have no EXPORT_SYMBOL_GPL() for 
rust_helper_devm_add_action() here?

Best regards

Dirk

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Dirk Behme]

On 20.05.2024 19:25, Danilo Krummrich wrote:
> From: Wedson Almeida Filho <wedsonaf@gmail.com>
> 
> This defines general functionality related to registering drivers with
> their respective subsystems, and registering modules that implement
> drivers.
> 
> Co-developed-by: Asahi Lina <lina@asahilina.net>
> Signed-off-by: Asahi Lina <lina@asahilina.net>
> Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> ---
>   rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
>   rust/kernel/lib.rs           |   4 +-
>   rust/macros/module.rs        |   2 +-
>   samples/rust/rust_minimal.rs |   2 +-
>   samples/rust/rust_print.rs   |   2 +-
>   5 files changed, 498 insertions(+), 4 deletions(-)
>   create mode 100644 rust/kernel/driver.rs
> 
> diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> new file mode 100644
> index 000000000000..e0cfc36d47ff
> --- /dev/null
> +++ b/rust/kernel/driver.rs
> @@ -0,0 +1,492 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> +//!
> +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> +//! using the [`Registration`] class.
> +
> +use crate::{
> +    alloc::{box_ext::BoxExt, flags::*},
> +    error::code::*,
> +    error::Result,
> +    str::CStr,
> +    sync::Arc,
> +    ThisModule,
> +};
> +use alloc::boxed::Box;
> +use core::{cell::UnsafeCell, marker::PhantomData, ops::Deref, pin::Pin};
> +
> +/// A subsystem (e.g., PCI, Platform, Amba, etc.) that allows drivers to be written for it.
> +pub trait DriverOps {
> +    /// The type that holds information about the registration. This is typically a struct defined
> +    /// by the C portion of the kernel.
> +    type RegType: Default;
> +
> +    /// Registers a driver.
> +    ///
> +    /// # Safety
> +    ///
> +    /// `reg` must point to valid, initialised, and writable memory. It may be modified by this
> +    /// function to hold registration state.
> +    ///
> +    /// On success, `reg` must remain pinned and valid until the matching call to
> +    /// [`DriverOps::unregister`].
> +    unsafe fn register(
> +        reg: *mut Self::RegType,
> +        name: &'static CStr,
> +        module: &'static ThisModule,
> +    ) -> Result;
> +
> +    /// Unregisters a driver previously registered with [`DriverOps::register`].
> +    ///
> +    /// # Safety
> +    ///
> +    /// `reg` must point to valid writable memory, initialised by a previous successful call to
> +    /// [`DriverOps::register`].
> +    unsafe fn unregister(reg: *mut Self::RegType);
> +}
> +
> +/// The registration of a driver.
> +pub struct Registration<T: DriverOps> {
> +    is_registered: bool,
> +    concrete_reg: UnsafeCell<T::RegType>,
> +}
> +
> +// SAFETY: `Registration` has no fields or methods accessible via `&Registration`, so it is safe to
> +// share references to it with multiple threads as nothing can be done.
> +unsafe impl<T: DriverOps> Sync for Registration<T> {}
> +
> +impl<T: DriverOps> Registration<T> {
> +    /// Creates a new instance of the registration object.
> +    pub fn new() -> Self {
> +        Self {
> +            is_registered: false,
> +            concrete_reg: UnsafeCell::new(T::RegType::default()),
> +        }
> +    }
> +
> +    /// Allocates a pinned registration object and registers it.
> +    ///
> +    /// Returns a pinned heap-allocated representation of the registration.
> +    pub fn new_pinned(name: &'static CStr, module: &'static ThisModule) -> Result<Pin<Box<Self>>> {
> +        let mut reg = Pin::from(Box::new(Self::new(), GFP_KERNEL)?);
> +        reg.as_mut().register(name, module)?;
> +        Ok(reg)
> +    }
> +
> +    /// Registers a driver with its subsystem.
> +    ///
> +    /// It must be pinned because the memory block that represents the registration is potentially
> +    /// self-referential.
> +    pub fn register(
> +        self: Pin<&mut Self>,
> +        name: &'static CStr,
> +        module: &'static ThisModule,
> +    ) -> Result {
> +        // SAFETY: We never move out of `this`.
> +        let this = unsafe { self.get_unchecked_mut() };
> +        if this.is_registered {
> +            // Already registered.
> +            return Err(EINVAL);
> +        }
> +
> +        // SAFETY: `concrete_reg` was initialised via its default constructor. It is only freed
> +        // after `Self::drop` is called, which first calls `T::unregister`.
> +        unsafe { T::register(this.concrete_reg.get(), name, module) }?;
> +
> +        this.is_registered = true;
> +        Ok(())
> +    }
> +}
> +
> +impl<T: DriverOps> Default for Registration<T> {
> +    fn default() -> Self {
> +        Self::new()
> +    }
> +}
> +
> +impl<T: DriverOps> Drop for Registration<T> {
> +    fn drop(&mut self) {
> +        if self.is_registered {
> +            // SAFETY: This path only runs if a previous call to `T::register` completed
> +            // successfully.
> +            unsafe { T::unregister(self.concrete_reg.get()) };
> +        }
> +    }
> +}
> +
> +/// Conversion from a device id to a raw device id.
> +///
> +/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
> +/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
> +///
> +/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
> +/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
> +/// concrete types (which can still have const associated functions) instead of a trait.
> +///
> +/// # Safety
> +///
> +/// Implementers must ensure that:
> +///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
> +///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
> +///     that buses can recover the pointer to the data.

In his I2C branch Fabien has a patch [1] [2] to remove the 
RawDeviceId::to_rawid part above. Maybe it could be aligned that that 
patch isn't required any more?

Best regards

Dirk

[1] 
https://github.com/Fabo/linux/commit/4b65b8d7ffe07057672b8eb89d376759d67bf060

[2]

 From 4b65b8d7ffe07057672b8eb89d376759d67bf060 Mon Sep 17 00:00:00 2001
From: Fabien Parent <fabien.parent@linaro.org>
Date: Sun, 28 Apr 2024 11:12:46 -0700
Subject: [PATCH] fixup! rust: add driver abstraction

RawDeviceId::to_rawid is not part of the RawDeviceId trait. Nonetheless
this function must be defined by the type that will implement
RawDeviceId, but to keep `rustdoc` from throwing a warning, let's just
remove it from the docs.

	warning: unresolved link to `RawDeviceId::to_rawid`
	   --> rust/kernel/driver.rs:120:11
	    |
	120 | ///   - [`RawDeviceId::to_rawid`] stores `offset` in the 
context/data field of the raw device id so
	    |           ^^^^^^^^^^^^^^^^^^^^^ the trait `RawDeviceId` has no 
associated item named `to_rawid`
	    |
	    = note: `#[warn(rustdoc::broken_intra_doc_links)]` on by default

	warning: 1 warning emitted
---
  rust/kernel/driver.rs | 2 --
  1 file changed, 2 deletions(-)

diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
index c1258ce0d041af..d141e23224d3db 100644
--- a/rust/kernel/driver.rs
+++ b/rust/kernel/driver.rs
@@ -128,8 +128,6 @@ impl<T: DriverOps> Drop for Registration<T> {
  ///
  /// Implementers must ensure that:
  ///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the 
raw device id.
-///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data 
field of the raw device id so
-///     that buses can recover the pointer to the data.
  pub unsafe trait RawDeviceId {
      /// The raw type that holds the device id.
      ///

Re: [RFC PATCH 05/11] rust: add revocable objects

[Dirk Behme]

On 20.05.2024 19:25, Danilo Krummrich wrote:
> From: Wedson Almeida Filho <wedsonaf@gmail.com>
> 
> This implements the Revocable and AsyncRevocable types.
> 
> Revocable allows access to objects to be safely revoked at run time.
> 
> This is useful, for example, for resources allocated during device probe;
> when the device is removed, the driver should stop accessing the device
> resources even if other state is kept in memory due to existing
> references (i.e., device context data is ref-counted and has a non-zero
> refcount after removal of the device).
> 
> AsyncRevocable allows access to objects to be revoked without having to
> wait for existing users to complete. This will be used to drop futures
> in tasks when executors are being torn down.
> 
> Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> ---
>   rust/kernel/lib.rs       |   1 +
>   rust/kernel/revocable.rs | 441 +++++++++++++++++++++++++++++++++++++++
>   2 files changed, 442 insertions(+)
>   create mode 100644 rust/kernel/revocable.rs
> 
> diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
> index 698121c925f3..d7d415429517 100644
> --- a/rust/kernel/lib.rs
> +++ b/rust/kernel/lib.rs
> @@ -40,6 +40,7 @@
>   pub mod net;
>   pub mod prelude;
>   pub mod print;
> +pub mod revocable;
>   mod static_assert;
>   #[doc(hidden)]
>   pub mod std_vendor;
> diff --git a/rust/kernel/revocable.rs b/rust/kernel/revocable.rs
> new file mode 100644
> index 000000000000..71408039a117
> --- /dev/null
> +++ b/rust/kernel/revocable.rs
> @@ -0,0 +1,441 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +//! Revocable objects.
> +//!
> +//! The [`Revocable`] type wraps other types and allows access to them to be revoked. The existence
> +//! of a [`RevocableGuard`] ensures that objects remain valid.
> +
> +use crate::{
> +    bindings,
> +    init::{self},
> +    prelude::*,
> +    sync::rcu,
> +};
> +use core::{
> +    cell::UnsafeCell,
> +    marker::PhantomData,
> +    mem::MaybeUninit,
> +    ops::Deref,
> +    ptr::drop_in_place,
> +    sync::atomic::{fence, AtomicBool, AtomicU32, Ordering},
> +};
> +
> +/// An object that can become inaccessible at runtime.
> +///
> +/// Once access is revoked and all concurrent users complete (i.e., all existing instances of
> +/// [`RevocableGuard`] are dropped), the wrapped object is also dropped.
> +///
> +/// # Examples

You might want to enable the doctest and check if the Examples are at 
least compiling ;) Here and in devres as well.

Best regards

Dirk

Re: [RFC PATCH 08/11] rust: add devres abstraction

[Dirk Behme]

On 20.05.2024 19:25, Danilo Krummrich wrote:
> Add a Rust abstraction for the kernel's devres (device resource
> management) implementation.
> 
> The Devres type acts as a container to manage the lifetime and
> accessibility of device bound resources. Therefore it registers a
> devres callback and revokes access to the resource on invocation.
> 
> Users of the Devres abstraction can simply free the corresponding
> resources in their Drop implementation, which is invoked when either the
> Devres instance goes out of scope or the devres callback leads to the
> resource being revoked, which implies a call to drop_in_place().
> 
> Co-developed-by: Philipp Stanner <pstanner@redhat.com>
> Signed-off-by: Philipp Stanner <pstanner@redhat.com>
> Signed-off-by: Danilo Krummrich <dakr@redhat.com>

...

> +impl<T> Devres<T> {
> +    /// Creates a new [`Devres`] instance of the give data.

Typo? give -> given

> +    pub fn new(dev: ARef<Device>, data: T, flags: Flags) -> Result<Self> {
> +        let callback = devres_callback::<T>;
> +
> +        let inner = Box::pin_init(
> +            pin_init!( DevresInner {
> +                dev: dev,
> +                data <- Revocable::new(data),
> +            }),
> +            flags,
> +        )?;
> +
> +        let ret = unsafe {
> +            bindings::devm_add_action(inner.dev.as_raw(), Some(callback), inner.as_cptr())
> +        };
> +
> +        if ret != 0 {
> +            return Err(Error::from_errno(ret));
> +        }
> +
> +        // We have to store the exact callback function pointer used with
> +        // `bindings::devm_add_action` for `bindings::devm_remove_action`. There compiler might put

Missing 'the'? "There *the* compiler might put ..."

Best regards

Dirk

Re: [RFC PATCH 01/11] rust: add abstraction for struct device

[Greg KH]

On Tue, May 21, 2024 at 10:42:03PM +0200, Danilo Krummrich wrote:
> On Tue, May 21, 2024 at 11:24:38AM +0200, Greg KH wrote:
> > On Mon, May 20, 2024 at 10:22:22PM +0200, Danilo Krummrich wrote:
> > > > > +impl Device {
> > > > > +    /// Creates a new ref-counted instance of an existing device pointer.
> > > > > +    ///
> > > > > +    /// # Safety
> > > > > +    ///
> > > > > +    /// Callers must ensure that `ptr` is valid, non-null, and has a non-zero reference count.
> > > > 
> > > > Callers NEVER care about the reference count of a struct device, anyone
> > > > poking in that is asking for trouble.
> > > 
> > > That's confusing, if not the caller who's passing the device pointer somewhere,
> > > who else?
> > > 
> > > Who takes care that a device' reference count is non-zero when a driver's probe
> > > function is called?
> > 
> > A device's reference count will be non-zero, I'm saying that sometimes,
> > some driver core functions are called with a 'struct device' that is
> > NULL, and it can handle it just fine.  Hopefully no callbacks to the
> > rust code will happen that way, but why aren't you checking just "to be
> > sure!" otherwise you could have a bug here, and it costs nothing to
> > verify it, right?
> 
> I get your point on that one. But let me explain a bit more why I think that
> check is not overly helpful here.
> 
> In Rust we have the concept of marking functions as 'unsafe'. Unsafe functions
> need to document their safety preconsitions, i.e. the conditions the caller of
> the function must guarantee. The caller of an unsafe function needs an unsafe
> block for it to compile and every unsafe block needs an explanation why it is
> safe to call this function with the corresponding arguments.
> 
> (Ideally, we want to avoid having them in the first place, but for C abstractions
> we have to deal with raw pointers we receive from the C side and dereferencing a
> raw pointer is unsafe by definition.)
> 
> In this case we have a function that constructs the Rust `Device` structure from
> a raw (device) pointer we potentially received from the C side. Now we have to
> decide whether this function is going to be unsafe or safe.
> 
> In order for this function to be safe we would need to be able to guarantee that
> this is a valid, non-null pointer with a non-zero reference count, which
> unfortunately we can't. Hence, it's going to be an unsafe function.

But you can verify it is non-null, so why not?

> A NULL pointer check would not make it a safe function either, since the pointer
> could still be an invalid one, or a pointer to a device it's not guaranteed that
> the reference count is held up for the duration of the function call.

True, but you just took one huge swatch of "potential crashes" off the
table.  To ignore that feels odd.

> Given that, we could add the NULL check and change the safety precondition to
> "valid pointer to a device with non-zero reference count OR NULL", but I don't
> see how this improves the situation for the caller, plus we'd need to return
> `Result<Device>` instead and let the caller handle that the `Device` was not
> created.

It better be able to handle if `Device` was not created, as you could
have been out of memory and nothing would have been allocated.  To not
check feels very broken.

> > Ok, if you say so, should we bookmark this thread for when this does
> > happen?  :)
> 
> I'm just saying the caller has to validate that or provide a rationale why this
> is safe anyways, hence it'd be just a duplicate check.
> 
> > 
> > What will the rust code do if it is passed in a NULL pointer?  Will it
> > crash like C code does?  Or something else?
> 
> It mostly calls into C functions with this pointer, depends on what they do.
> 
> Checking a few random places, e.g. [1], it seems to crash in most cases.
> 
> [1] https://elixir.free-electrons.com/linux/latest/source/drivers/base/core.c#L3863

Great, then you should check :)

thanks,

greg k-h

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Greg KH]

On Wed, May 22, 2024 at 12:21:53AM +0200, Danilo Krummrich wrote:
> On Tue, May 21, 2024 at 11:35:43AM +0200, Greg KH wrote:
> > On Tue, May 21, 2024 at 12:30:37AM +0200, Danilo Krummrich wrote:
> > > On Mon, May 20, 2024 at 08:14:18PM +0200, Greg KH wrote:
> > > > On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> > > > > From: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > > 
> > > > > This defines general functionality related to registering drivers with
> > > > > their respective subsystems, and registering modules that implement
> > > > > drivers.
> > > > > 
> > > > > Co-developed-by: Asahi Lina <lina@asahilina.net>
> > > > > Signed-off-by: Asahi Lina <lina@asahilina.net>
> > > > > Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > > Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > > > > ---
> > > > >  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
> > > > >  rust/kernel/lib.rs           |   4 +-
> > > > >  rust/macros/module.rs        |   2 +-
> > > > >  samples/rust/rust_minimal.rs |   2 +-
> > > > >  samples/rust/rust_print.rs   |   2 +-
> > > > >  5 files changed, 498 insertions(+), 4 deletions(-)
> > > > >  create mode 100644 rust/kernel/driver.rs
> > > > > 
> > > > > diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> > > > > new file mode 100644
> > > > > index 000000000000..e0cfc36d47ff
> > > > > --- /dev/null
> > > > > +++ b/rust/kernel/driver.rs
> > > > > @@ -0,0 +1,492 @@
> > > > > +// SPDX-License-Identifier: GPL-2.0
> > > > > +
> > > > > +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> > > > > +//!
> > > > > +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> > > > > +//! using the [`Registration`] class.
> > > > 
> > > > Why are you creating new "names" here?  "DriverOps" is part of a 'struct
> > > > device_driver' why are you separating it out here?  And what is
> > > 
> > > DriverOps is a trait which abstracts a subsystems register() and unregister()
> > > functions to (un)register drivers. It exists such that a generic Registration
> > > implementation calls the correct one for the subsystem.
> > > 
> > > For instance, PCI would implement DriverOps::register() by calling into
> > > bindings::__pci_register_driver().
> > > 
> > > We can discuss whether DriverOps is a good name for the trait, but it's not a
> > > (different) name for something that already exists and already has a name.
> > 
> > It's a name we don't have in the C code as the design of the driver core
> > does not need or provide it.  It's just the section of 'struct
> > device_driver' that provides function callbacks, why does it need to be
> > separate at all?
> 
> I'm confused by the relationship to `struct device_driver` you seem to imply.
> How is it related?
> 
> Again, this is just a trait for subsystems to provide their corresponding
> register and unregister implementation, e.g. pci_register_driver() and
> pci_unregister_driver(), such that they can be called from the generic
> Registration code below.
> 
> See [1] for an example implementation in PCI.

registering and unregistering drivers belongs in the bus code, NOT in
the driver code.

I think lots of the objections I had here will be fixed up when you move
the bus logic out to it's own file, it does not belong here in a driver
file (device ids, etc.)

> Please also consider that some structures might be a 1:1 representation of C
> structures, some C structures are not required at the Rust side at all, and
> then there might be additional structures and traits that abstract things C has
> no data structure for.

That's fine, but let's keep the separate of what we have today at the
very least and not try to lump it all into one file, that makes it
harder to review and maintain over time.

> > > > 'Registration'?  That's a bus/class thing, not a driver thing.
> > > 
> > > A Registration is an object representation of the driver's registered state.
> > 
> > And that representation should not ever need to be tracked by the
> > driver, that's up to the driver core to track that.
> 
> The driver doesn't need it, the Registration abstraction does need it. Please
> see my comments below.

Great, put it elsewhere please, it does not belong in driver.rs.

> > > Having an object representation for that is basically the Rust way to manage the
> > > lifetime of this state.
> > 
> > This all should be a static chunk of read-only memory that should never
> > have a lifetime, why does it need to be managed at all?
> 
> What I meant here is that if a driver was registered, we need to make sure it's
> going to be unregistered eventually, e.g. when the module is removed or when
> something fails after registration and we need to unwind.
> 
> When the Registration structure goes out of scope, which would happen in both
> the cases above, it will automatically unregister the driver, due to the
> automatic call to `drop()`.

That's fine, but again, this all should just be static code, not
dynamic.

> > > Once the Registration is dropped, the driver is
> > > unregistered. For instance, a PCI driver Registration can be bound to a module,
> > > such that the PCI driver is unregistered when the module is unloaded (the
> > > Registration is dropped in the module_exit() callback).
> > 
> > Ok, that's fine, but note that your module_exit() function will never be
> > called if your module_init() callback fails, so why do you need to track
> > this state?  Again, C drivers never need to track this state, why is
> > rust requiring more logic here for something that is NOT a dynamic chunk
> > of memory (or should not be a dynamic chunk of memory, let's get it
> > correct from the start and not require us to change things later on to
> > make it more secure).
> 
> That's fine, if module_init() would fail, the Registration would be dropped as
> well.
> 
> As for why doesn't C need this is a good example of what I wrote above. Because
> it is useful for Rust, but not for C.
> 
> In Rust we get drop() automatically called when a structure is destroyed. This
> means that if we let drivers put the Registration structure (e.g. representing
> that a PCI driver was registered) into its `Module` representation structure
> (already upstream) then this Registration is automatically destroyed once the
> module representation is destroyed (which happens on module_exit()). This leads
> to `drop()` of the `Registration` structure being called, which unregisteres the
> (e.g. PCI) driver.
> 
> This way the driver does not need to take care of unregistering the PCI driver
> explicitly. The driver can also place multiple registrations into the `Module`
> structure. All of them would be unregistered automatically in module_exit().

Ok, I think we are agreeing here, except that you do not need a "am I
registered" flag, as the existance of the "object" defines if it is
registered or not (i.e. if it exists and the "destructor" is called,
it's been registered, otherwise it hasn't been and the check is
pointless.)

> > Again, 'class' means something different here in the driver model, so be
> > careful with terms, language matters, especially when many of our
> > developers do not have English as their native language.
> > 
> > > > > +/// The registration of a driver.
> > > > > +pub struct Registration<T: DriverOps> {
> > > > > +    is_registered: bool,
> > > > 
> > > > Why does a driver need to know if it is registered or not?  Only the
> > > > driver core cares about that, please do not expose that, it's racy and
> > > > should not be relied on.
> > > 
> > > We need it to ensure we do not try to register the same thing twice
> > 
> > Your logic in your code is wrong if you attempt to register it twice,
> > AND the driver core will return an error if you do so, so a driver
> > should not need to care about this.
> 
> We want it to be safe, if the driver logic is wrong and registers it twice, we
> don't want it to blow up.

How could that happen?

> The driver core takes care, but I think there are subsystems that do
> initializations that could make things blow up when registering the driver
> twice.

Nope, should not be needed, see above.  Rust should make this _easier_
not harder, than C code here :)

> > > , some subsystems might just catch fire otherwise.
> > 
> > Which ones?
> 
> Let's take the one we provide abstractons for, PCI.
> 
> In __pci_register_driver() we call spin_lock_init() and INIT_LIST_HEAD() before
> driver_register() could bail out [1].
> 
> What if this driver is already registered and in use and we're randomly altering
> the list pointers or call spin_lock_init() on a spin lock that's currently being
> held?

I don't understand, why would you ever call "register driver" BEFORE the
driver was properly set up to actually be registered?

PCI works properly here, you don't register unless everything is set up.
Which is why it doesn't have a "hey, am I registered or not?" type flag,
it's not needed.

> > 
> > > > > +        }
> > > > > +    }
> > > > > +}
> > > > > +
> > > > > +/// Conversion from a device id to a raw device id.
> > > > > +///
> > > > > +/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
> > > > > +/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
> > > > > +///
> > > > > +/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
> > > > > +/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
> > > > > +/// concrete types (which can still have const associated functions) instead of a trait.
> > > > > +///
> > > > > +/// # Safety
> > > > > +///
> > > > > +/// Implementers must ensure that:
> > > > > +///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
> > > > > +///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
> > > > > +///     that buses can recover the pointer to the data.
> > > > > +pub unsafe trait RawDeviceId {
> > > > > +    /// The raw type that holds the device id.
> > > > > +    ///
> > > > > +    /// Id tables created from [`Self`] are going to hold this type in its zero-terminated array.
> > > > > +    type RawType: Copy;
> > > > > +
> > > > > +    /// A zeroed-out representation of the raw device id.
> > > > > +    ///
> > > > > +    /// Id tables created from [`Self`] use [`Self::ZERO`] as the sentinel to indicate the end of
> > > > > +    /// the table.
> > > > > +    const ZERO: Self::RawType;
> > > > 
> > > > All busses have their own way of creating "ids" and that is limited to
> > > > the bus code itself, why is any of this in the rust side?  What needs
> > > > this?  A bus will create the id for the devices it manages, and can use
> > > > it as part of the name it gives the device (but not required), so all of
> > > > this belongs to the bus, NOT to a driver, or a device.
> > > 
> > > This is just a generalized interface which can be used by subsystems to
> > > implement the subsystem / bus specific ID type.
> > 
> > Please move this all to a different file as it has nothing to do with
> > the driver core bindings with the include/device/driver.h api.
> 
> I don't think driver.rs in an unreasonable place for a generic device ID
> representation that is required by every driver structure.

It has nothing to do with drivers on their own, it's a bus attribute,
please put that in bus.rs at the least.  Or it's own file if you need
lots of code for simple arrays like this :)

> But I'm also not overly resistant to move it out. What do you think would be a
> good name?
> 
> Please consider that this name will also be the namespace for this trait.
> Currently you can reference it with `kernel::driver::RawDeviceId`. If you move
> it to foo.rs, it'd be `kernel::foo::RawDeviceId`.

I don't see why this isn't just unique to each bus type, it is not a
driver attribute, but a bus-specific-driver type.  Please put it there
and don't attempt to make it "generic".

thanks,

greg k-h

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Danilo Krummrich]

On Tue, Jun 04, 2024 at 04:27:31PM +0200, Greg KH wrote:
> On Wed, May 22, 2024 at 12:21:53AM +0200, Danilo Krummrich wrote:
> > On Tue, May 21, 2024 at 11:35:43AM +0200, Greg KH wrote:
> > > On Tue, May 21, 2024 at 12:30:37AM +0200, Danilo Krummrich wrote:
> > > > On Mon, May 20, 2024 at 08:14:18PM +0200, Greg KH wrote:
> > > > > On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> > > > > > From: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > > > 
> > > > > > This defines general functionality related to registering drivers with
> > > > > > their respective subsystems, and registering modules that implement
> > > > > > drivers.
> > > > > > 
> > > > > > Co-developed-by: Asahi Lina <lina@asahilina.net>
> > > > > > Signed-off-by: Asahi Lina <lina@asahilina.net>
> > > > > > Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > > > Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > > > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > > > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > > > > > ---
> > > > > >  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
> > > > > >  rust/kernel/lib.rs           |   4 +-
> > > > > >  rust/macros/module.rs        |   2 +-
> > > > > >  samples/rust/rust_minimal.rs |   2 +-
> > > > > >  samples/rust/rust_print.rs   |   2 +-
> > > > > >  5 files changed, 498 insertions(+), 4 deletions(-)
> > > > > >  create mode 100644 rust/kernel/driver.rs
> > > > > > 
> > > > > > diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> > > > > > new file mode 100644
> > > > > > index 000000000000..e0cfc36d47ff
> > > > > > --- /dev/null
> > > > > > +++ b/rust/kernel/driver.rs
> > > > > > @@ -0,0 +1,492 @@
> > > > > > +// SPDX-License-Identifier: GPL-2.0
> > > > > > +
> > > > > > +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> > > > > > +//!
> > > > > > +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> > > > > > +//! using the [`Registration`] class.
> > > > > 
> > > > > Why are you creating new "names" here?  "DriverOps" is part of a 'struct
> > > > > device_driver' why are you separating it out here?  And what is
> > > > 
> > > > DriverOps is a trait which abstracts a subsystems register() and unregister()
> > > > functions to (un)register drivers. It exists such that a generic Registration
> > > > implementation calls the correct one for the subsystem.
> > > > 
> > > > For instance, PCI would implement DriverOps::register() by calling into
> > > > bindings::__pci_register_driver().
> > > > 
> > > > We can discuss whether DriverOps is a good name for the trait, but it's not a
> > > > (different) name for something that already exists and already has a name.
> > > 
> > > It's a name we don't have in the C code as the design of the driver core
> > > does not need or provide it.  It's just the section of 'struct
> > > device_driver' that provides function callbacks, why does it need to be
> > > separate at all?
> > 
> > I'm confused by the relationship to `struct device_driver` you seem to imply.
> > How is it related?
> > 
> > Again, this is just a trait for subsystems to provide their corresponding
> > register and unregister implementation, e.g. pci_register_driver() and
> > pci_unregister_driver(), such that they can be called from the generic
> > Registration code below.
> > 
> > See [1] for an example implementation in PCI.
> 
> registering and unregistering drivers belongs in the bus code, NOT in
> the driver code.

Why? We're not (re-)implementing a bus here. Again, those are just abstractions
to call the C functions to register a driver. The corresponding C functions are
e.g. driver_register() or __pci_register_driver(). Those are defined in
drivers/base/driver.c and drivers/pci/pci-driver.c respectively.

Why wouldn't we follow the same scheme in Rust abstractions?

> 
> I think lots of the objections I had here will be fixed up when you move
> the bus logic out to it's own file, it does not belong here in a driver
> file (device ids, etc.)
> 
> > Please also consider that some structures might be a 1:1 representation of C
> > structures, some C structures are not required at the Rust side at all, and
> > then there might be additional structures and traits that abstract things C has
> > no data structure for.
> 
> That's fine, but let's keep the separate of what we have today at the
> very least and not try to lump it all into one file, that makes it
> harder to review and maintain over time.
> 
> > > > > 'Registration'?  That's a bus/class thing, not a driver thing.
> > > > 
> > > > A Registration is an object representation of the driver's registered state.
> > > 
> > > And that representation should not ever need to be tracked by the
> > > driver, that's up to the driver core to track that.
> > 
> > The driver doesn't need it, the Registration abstraction does need it. Please
> > see my comments below.
> 
> Great, put it elsewhere please, it does not belong in driver.rs.

This `Registration` structure is a generic abstraction to call some
$SUBSYSTEM_driver_register() function (e.g. pci_register_driver()). Why would it
belong somewhere else? Again, the corresponding C functions are in some driver.c
file as well.

> 
> > > > Having an object representation for that is basically the Rust way to manage the
> > > > lifetime of this state.
> > > 
> > > This all should be a static chunk of read-only memory that should never
> > > have a lifetime, why does it need to be managed at all?
> > 
> > What I meant here is that if a driver was registered, we need to make sure it's
> > going to be unregistered eventually, e.g. when the module is removed or when
> > something fails after registration and we need to unwind.
> > 
> > When the Registration structure goes out of scope, which would happen in both
> > the cases above, it will automatically unregister the driver, due to the
> > automatic call to `drop()`.
> 
> That's fine, but again, this all should just be static code, not
> dynamic.

I agree. As already mentioned in another thread, this will be static in v2.

I got the code for that in place already. :)

> 
> > > > Once the Registration is dropped, the driver is
> > > > unregistered. For instance, a PCI driver Registration can be bound to a module,
> > > > such that the PCI driver is unregistered when the module is unloaded (the
> > > > Registration is dropped in the module_exit() callback).
> > > 
> > > Ok, that's fine, but note that your module_exit() function will never be
> > > called if your module_init() callback fails, so why do you need to track
> > > this state?  Again, C drivers never need to track this state, why is
> > > rust requiring more logic here for something that is NOT a dynamic chunk
> > > of memory (or should not be a dynamic chunk of memory, let's get it
> > > correct from the start and not require us to change things later on to
> > > make it more secure).
> > 
> > That's fine, if module_init() would fail, the Registration would be dropped as
> > well.
> > 
> > As for why doesn't C need this is a good example of what I wrote above. Because
> > it is useful for Rust, but not for C.
> > 
> > In Rust we get drop() automatically called when a structure is destroyed. This
> > means that if we let drivers put the Registration structure (e.g. representing
> > that a PCI driver was registered) into its `Module` representation structure
> > (already upstream) then this Registration is automatically destroyed once the
> > module representation is destroyed (which happens on module_exit()). This leads
> > to `drop()` of the `Registration` structure being called, which unregisteres the
> > (e.g. PCI) driver.
> > 
> > This way the driver does not need to take care of unregistering the PCI driver
> > explicitly. The driver can also place multiple registrations into the `Module`
> > structure. All of them would be unregistered automatically in module_exit().
> 
> Ok, I think we are agreeing here, except that you do not need a "am I
> registered" flag, as the existance of the "object" defines if it is
> registered or not (i.e. if it exists and the "destructor" is called,
> it's been registered, otherwise it hasn't been and the check is
> pointless.)

The static implementation does not need this anymore, since there is no separate
register() function anymore that we need to protect.

> 
> > > Again, 'class' means something different here in the driver model, so be
> > > careful with terms, language matters, especially when many of our
> > > developers do not have English as their native language.
> > > 
> > > > > > +/// The registration of a driver.
> > > > > > +pub struct Registration<T: DriverOps> {
> > > > > > +    is_registered: bool,
> > > > > 
> > > > > Why does a driver need to know if it is registered or not?  Only the
> > > > > driver core cares about that, please do not expose that, it's racy and
> > > > > should not be relied on.
> > > > 
> > > > We need it to ensure we do not try to register the same thing twice
> > > 
> > > Your logic in your code is wrong if you attempt to register it twice,
> > > AND the driver core will return an error if you do so, so a driver
> > > should not need to care about this.
> > 
> > We want it to be safe, if the driver logic is wrong and registers it twice, we
> > don't want it to blow up.
> 
> How could that happen?
> 
> > The driver core takes care, but I think there are subsystems that do
> > initializations that could make things blow up when registering the driver
> > twice.
> 
> Nope, should not be needed, see above.  Rust should make this _easier_
> not harder, than C code here :)

Agree, and as mentioned above, with v2 Rust drivers can't register the same
thing twice anymore.

> 
> > > > , some subsystems might just catch fire otherwise.
> > > 
> > > Which ones?
> > 
> > Let's take the one we provide abstractons for, PCI.
> > 
> > In __pci_register_driver() we call spin_lock_init() and INIT_LIST_HEAD() before
> > driver_register() could bail out [1].
> > 
> > What if this driver is already registered and in use and we're randomly altering
> > the list pointers or call spin_lock_init() on a spin lock that's currently being
> > held?
> 
> I don't understand, why would you ever call "register driver" BEFORE the
> driver was properly set up to actually be registered?
> 
> PCI works properly here, you don't register unless everything is set up.
> Which is why it doesn't have a "hey, am I registered or not?" type flag,
> it's not needed.

That's not what I meant, but I think we can drop this specific part of the
discussion anyways, since with v2 we can't hit this anymore. :)

> 
> > > 
> > > > > > +        }
> > > > > > +    }
> > > > > > +}
> > > > > > +
> > > > > > +/// Conversion from a device id to a raw device id.
> > > > > > +///
> > > > > > +/// This is meant to be implemented by buses/subsystems so that they can use [`IdTable`] to
> > > > > > +/// guarantee (at compile-time) zero-termination of device id tables provided by drivers.
> > > > > > +///
> > > > > > +/// Originally, RawDeviceId was implemented as a const trait. However, this unstable feature is
> > > > > > +/// broken/gone in 1.73. To work around this, turn IdArray::new() into a macro such that it can use
> > > > > > +/// concrete types (which can still have const associated functions) instead of a trait.
> > > > > > +///
> > > > > > +/// # Safety
> > > > > > +///
> > > > > > +/// Implementers must ensure that:
> > > > > > +///   - [`RawDeviceId::ZERO`] is actually a zeroed-out version of the raw device id.
> > > > > > +///   - [`RawDeviceId::to_rawid`] stores `offset` in the context/data field of the raw device id so
> > > > > > +///     that buses can recover the pointer to the data.
> > > > > > +pub unsafe trait RawDeviceId {
> > > > > > +    /// The raw type that holds the device id.
> > > > > > +    ///
> > > > > > +    /// Id tables created from [`Self`] are going to hold this type in its zero-terminated array.
> > > > > > +    type RawType: Copy;
> > > > > > +
> > > > > > +    /// A zeroed-out representation of the raw device id.
> > > > > > +    ///
> > > > > > +    /// Id tables created from [`Self`] use [`Self::ZERO`] as the sentinel to indicate the end of
> > > > > > +    /// the table.
> > > > > > +    const ZERO: Self::RawType;
> > > > > 
> > > > > All busses have their own way of creating "ids" and that is limited to
> > > > > the bus code itself, why is any of this in the rust side?  What needs
> > > > > this?  A bus will create the id for the devices it manages, and can use
> > > > > it as part of the name it gives the device (but not required), so all of
> > > > > this belongs to the bus, NOT to a driver, or a device.
> > > > 
> > > > This is just a generalized interface which can be used by subsystems to
> > > > implement the subsystem / bus specific ID type.
> > > 
> > > Please move this all to a different file as it has nothing to do with
> > > the driver core bindings with the include/device/driver.h api.
> > 
> > I don't think driver.rs in an unreasonable place for a generic device ID
> > representation that is required by every driver structure.
> 
> It has nothing to do with drivers on their own, it's a bus attribute,
> please put that in bus.rs at the least.  Or it's own file if you need
> lots of code for simple arrays like this :)

It's not a lot of code actually, probably less than 100 lines, there is a lot of
documentation / examples though. :)

The corresponding C structure definitions are in
include/linux/mod_devicetable.h. Maybe we can move it to a separte device_id.rs
if you prefer that?

> 
> > But I'm also not overly resistant to move it out. What do you think would be a
> > good name?
> > 
> > Please consider that this name will also be the namespace for this trait.
> > Currently you can reference it with `kernel::driver::RawDeviceId`. If you move
> > it to foo.rs, it'd be `kernel::foo::RawDeviceId`.
> 
> I don't see why this isn't just unique to each bus type, it is not a
> driver attribute, but a bus-specific-driver type.  Please put it there
> and don't attempt to make it "generic".

If we don't make it generic we end up with a lot of duplicate code in every
subsystem that has some kind of device ID, e.g. OF, PCI, etc. Why would we want
to do that? I think moving the generic abstraction into a separate device_id.rs
seems to be the better option.

> 
> thanks,
> 
> greg k-h
> 

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Greg KH]

On Tue, Jun 04, 2024 at 05:41:21PM +0200, Danilo Krummrich wrote:
> On Tue, Jun 04, 2024 at 04:27:31PM +0200, Greg KH wrote:
> > On Wed, May 22, 2024 at 12:21:53AM +0200, Danilo Krummrich wrote:
> > > On Tue, May 21, 2024 at 11:35:43AM +0200, Greg KH wrote:
> > > > On Tue, May 21, 2024 at 12:30:37AM +0200, Danilo Krummrich wrote:
> > > > > On Mon, May 20, 2024 at 08:14:18PM +0200, Greg KH wrote:
> > > > > > On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> > > > > > > From: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > > > > 
> > > > > > > This defines general functionality related to registering drivers with
> > > > > > > their respective subsystems, and registering modules that implement
> > > > > > > drivers.
> > > > > > > 
> > > > > > > Co-developed-by: Asahi Lina <lina@asahilina.net>
> > > > > > > Signed-off-by: Asahi Lina <lina@asahilina.net>
> > > > > > > Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > > > > Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > > > > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > > > > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > > > > > > ---
> > > > > > >  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
> > > > > > >  rust/kernel/lib.rs           |   4 +-
> > > > > > >  rust/macros/module.rs        |   2 +-
> > > > > > >  samples/rust/rust_minimal.rs |   2 +-
> > > > > > >  samples/rust/rust_print.rs   |   2 +-
> > > > > > >  5 files changed, 498 insertions(+), 4 deletions(-)
> > > > > > >  create mode 100644 rust/kernel/driver.rs
> > > > > > > 
> > > > > > > diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> > > > > > > new file mode 100644
> > > > > > > index 000000000000..e0cfc36d47ff
> > > > > > > --- /dev/null
> > > > > > > +++ b/rust/kernel/driver.rs
> > > > > > > @@ -0,0 +1,492 @@
> > > > > > > +// SPDX-License-Identifier: GPL-2.0
> > > > > > > +
> > > > > > > +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> > > > > > > +//!
> > > > > > > +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> > > > > > > +//! using the [`Registration`] class.
> > > > > > 
> > > > > > Why are you creating new "names" here?  "DriverOps" is part of a 'struct
> > > > > > device_driver' why are you separating it out here?  And what is
> > > > > 
> > > > > DriverOps is a trait which abstracts a subsystems register() and unregister()
> > > > > functions to (un)register drivers. It exists such that a generic Registration
> > > > > implementation calls the correct one for the subsystem.
> > > > > 
> > > > > For instance, PCI would implement DriverOps::register() by calling into
> > > > > bindings::__pci_register_driver().
> > > > > 
> > > > > We can discuss whether DriverOps is a good name for the trait, but it's not a
> > > > > (different) name for something that already exists and already has a name.
> > > > 
> > > > It's a name we don't have in the C code as the design of the driver core
> > > > does not need or provide it.  It's just the section of 'struct
> > > > device_driver' that provides function callbacks, why does it need to be
> > > > separate at all?
> > > 
> > > I'm confused by the relationship to `struct device_driver` you seem to imply.
> > > How is it related?
> > > 
> > > Again, this is just a trait for subsystems to provide their corresponding
> > > register and unregister implementation, e.g. pci_register_driver() and
> > > pci_unregister_driver(), such that they can be called from the generic
> > > Registration code below.
> > > 
> > > See [1] for an example implementation in PCI.
> > 
> > registering and unregistering drivers belongs in the bus code, NOT in
> > the driver code.
> 
> Why? We're not (re-)implementing a bus here. Again, those are just abstractions
> to call the C functions to register a driver. The corresponding C functions are
> e.g. driver_register() or __pci_register_driver(). Those are defined in
> drivers/base/driver.c and drivers/pci/pci-driver.c respectively.
> 
> Why wouldn't we follow the same scheme in Rust abstractions?

It's the bus that does the registering, so yeah, don't put it here at
all as it's not going to be needed (i.e. unless you write a bus in rust
you will never call driver_register())  So this can just be a wrapper
for the pci bus logic, keeping it simpler.

So you might be able to delete a lot of code here, only deal with a
"dumb" struct device wrapper to handle reference counts, and then do the
rest for the specific bus bindings?  Or is that too much to dream?

You aren't writing a "raw" driver here, no one does that, it's a bus
that handles that logic for you, and you should not have to expose any
"raw" driver attributes.

Yes, for some busses, they like to force a driver to set the "raw"
driver attribute, but I don't think that's a good idea and for the pci
driver layer, that shouldn't be necessary now, right?  If not, what
fields are you wanting to get direct access to?

thanks,

greg k-h

Re: [RFC PATCH 01/11] rust: add abstraction for struct device

[Danilo Krummrich]

On Tue, Jun 04, 2024 at 04:17:29PM +0200, Greg KH wrote:
> On Tue, May 21, 2024 at 10:42:03PM +0200, Danilo Krummrich wrote:
> > On Tue, May 21, 2024 at 11:24:38AM +0200, Greg KH wrote:
> > > On Mon, May 20, 2024 at 10:22:22PM +0200, Danilo Krummrich wrote:
> > > > > > +impl Device {
> > > > > > +    /// Creates a new ref-counted instance of an existing device pointer.
> > > > > > +    ///
> > > > > > +    /// # Safety
> > > > > > +    ///
> > > > > > +    /// Callers must ensure that `ptr` is valid, non-null, and has a non-zero reference count.
> > > > > 
> > > > > Callers NEVER care about the reference count of a struct device, anyone
> > > > > poking in that is asking for trouble.
> > > > 
> > > > That's confusing, if not the caller who's passing the device pointer somewhere,
> > > > who else?
> > > > 
> > > > Who takes care that a device' reference count is non-zero when a driver's probe
> > > > function is called?
> > > 
> > > A device's reference count will be non-zero, I'm saying that sometimes,
> > > some driver core functions are called with a 'struct device' that is
> > > NULL, and it can handle it just fine.  Hopefully no callbacks to the
> > > rust code will happen that way, but why aren't you checking just "to be
> > > sure!" otherwise you could have a bug here, and it costs nothing to
> > > verify it, right?
> > 
> > I get your point on that one. But let me explain a bit more why I think that
> > check is not overly helpful here.
> > 
> > In Rust we have the concept of marking functions as 'unsafe'. Unsafe functions
> > need to document their safety preconsitions, i.e. the conditions the caller of
> > the function must guarantee. The caller of an unsafe function needs an unsafe
> > block for it to compile and every unsafe block needs an explanation why it is
> > safe to call this function with the corresponding arguments.
> > 
> > (Ideally, we want to avoid having them in the first place, but for C abstractions
> > we have to deal with raw pointers we receive from the C side and dereferencing a
> > raw pointer is unsafe by definition.)
> > 
> > In this case we have a function that constructs the Rust `Device` structure from
> > a raw (device) pointer we potentially received from the C side. Now we have to
> > decide whether this function is going to be unsafe or safe.
> > 
> > In order for this function to be safe we would need to be able to guarantee that
> > this is a valid, non-null pointer with a non-zero reference count, which
> > unfortunately we can't. Hence, it's going to be an unsafe function.
> 
> But you can verify it is non-null, so why not?

I suggest to check out the code making use of this.

From the PCI abstractions:

    extern "C" fn probe_callback(
        pdev: *mut bindings::pci_dev,
        id: *const bindings::pci_device_id,
    ) -> core::ffi::c_int {
        // SAFETY: Safe because the core kernel only ever calls the probe callback with a valid
        // `pdev`.
        let dev = unsafe { device::Device::from_raw(&mut (*pdev).dev) };

        [...]
    }

Doing the NULL check would turn this into something like:

    extern "C" fn probe_callback(
        pdev: *mut bindings::pci_dev,
        id: *const bindings::pci_device_id,
    ) -> core::ffi::c_int {
        // SAFETY: Safe because the core kernel only ever calls the probe callback with a valid
        // `pdev`, but we still have to handle `Device::from_raw`'s NULL check.
        let dev = match unsafe { device::Device::from_raw(&mut (*pdev).dev) } {
           Ok(dev) => dev,
           Err(err) => return Error::to_errno(err),
        }
    }

This would be super odd. If `Device::from_raw` reports "Ok" it actually wouldn't
mean everything is well. It would *only* mean that the pointer that was passed
is not NULL. This is counter intuitive; IMHO unsafe functions shouldn't return
any type of result, because it just isn't meaningful.

> 
> > A NULL pointer check would not make it a safe function either, since the pointer
> > could still be an invalid one, or a pointer to a device it's not guaranteed that
> > the reference count is held up for the duration of the function call.
> 
> True, but you just took one huge swatch of "potential crashes" off the
> table.  To ignore that feels odd.
> 
> > Given that, we could add the NULL check and change the safety precondition to
> > "valid pointer to a device with non-zero reference count OR NULL", but I don't
> > see how this improves the situation for the caller, plus we'd need to return
> > `Result<Device>` instead and let the caller handle that the `Device` was not
> > created.
> 
> It better be able to handle if `Device` was not created, as you could
> have been out of memory and nothing would have been allocated.  To not
> check feels very broken.

The abstraction is not allocating a new C struct device, it's just abstracting a
pointer to an existing struct device. There is no OOM case to handle, the
abstraction holding the pointer lives on the stack.

> 
> > > Ok, if you say so, should we bookmark this thread for when this does
> > > happen?  :)
> > 
> > I'm just saying the caller has to validate that or provide a rationale why this
> > is safe anyways, hence it'd be just a duplicate check.
> > 
> > > 
> > > What will the rust code do if it is passed in a NULL pointer?  Will it
> > > crash like C code does?  Or something else?
> > 
> > It mostly calls into C functions with this pointer, depends on what they do.
> > 
> > Checking a few random places, e.g. [1], it seems to crash in most cases.
> > 
> > [1] https://elixir.free-electrons.com/linux/latest/source/drivers/base/core.c#L3863
> 
> Great, then you should check :)

Why isn't the conclusion that the C code should check (as well)? :) Would you
want to add a NULL check at the beginning of device_del()?

In Rust we have a clear separation between safe and unsafe functions with, for
the latter, documented requirements on what's actually allowed to pass in and
which preconditions must be guaranteed. The check happens, just not within the
unsafe function.

> 
> thanks,
> 
> greg k-h
> 

Re: [RFC PATCH 02/11] rust: add driver abstraction

[Danilo Krummrich]

On Tue, Jun 04, 2024 at 06:00:11PM +0200, Greg KH wrote:
> On Tue, Jun 04, 2024 at 05:41:21PM +0200, Danilo Krummrich wrote:
> > On Tue, Jun 04, 2024 at 04:27:31PM +0200, Greg KH wrote:
> > > On Wed, May 22, 2024 at 12:21:53AM +0200, Danilo Krummrich wrote:
> > > > On Tue, May 21, 2024 at 11:35:43AM +0200, Greg KH wrote:
> > > > > On Tue, May 21, 2024 at 12:30:37AM +0200, Danilo Krummrich wrote:
> > > > > > On Mon, May 20, 2024 at 08:14:18PM +0200, Greg KH wrote:
> > > > > > > On Mon, May 20, 2024 at 07:25:39PM +0200, Danilo Krummrich wrote:
> > > > > > > > From: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > > > > > 
> > > > > > > > This defines general functionality related to registering drivers with
> > > > > > > > their respective subsystems, and registering modules that implement
> > > > > > > > drivers.
> > > > > > > > 
> > > > > > > > Co-developed-by: Asahi Lina <lina@asahilina.net>
> > > > > > > > Signed-off-by: Asahi Lina <lina@asahilina.net>
> > > > > > > > Co-developed-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > > > > > Signed-off-by: Andreas Hindborg <a.hindborg@samsung.com>
> > > > > > > > Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com>
> > > > > > > > Signed-off-by: Danilo Krummrich <dakr@redhat.com>
> > > > > > > > ---
> > > > > > > >  rust/kernel/driver.rs        | 492 +++++++++++++++++++++++++++++++++++
> > > > > > > >  rust/kernel/lib.rs           |   4 +-
> > > > > > > >  rust/macros/module.rs        |   2 +-
> > > > > > > >  samples/rust/rust_minimal.rs |   2 +-
> > > > > > > >  samples/rust/rust_print.rs   |   2 +-
> > > > > > > >  5 files changed, 498 insertions(+), 4 deletions(-)
> > > > > > > >  create mode 100644 rust/kernel/driver.rs
> > > > > > > > 
> > > > > > > > diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs
> > > > > > > > new file mode 100644
> > > > > > > > index 000000000000..e0cfc36d47ff
> > > > > > > > --- /dev/null
> > > > > > > > +++ b/rust/kernel/driver.rs
> > > > > > > > @@ -0,0 +1,492 @@
> > > > > > > > +// SPDX-License-Identifier: GPL-2.0
> > > > > > > > +
> > > > > > > > +//! Generic support for drivers of different buses (e.g., PCI, Platform, Amba, etc.).
> > > > > > > > +//!
> > > > > > > > +//! Each bus/subsystem is expected to implement [`DriverOps`], which allows drivers to register
> > > > > > > > +//! using the [`Registration`] class.
> > > > > > > 
> > > > > > > Why are you creating new "names" here?  "DriverOps" is part of a 'struct
> > > > > > > device_driver' why are you separating it out here?  And what is
> > > > > > 
> > > > > > DriverOps is a trait which abstracts a subsystems register() and unregister()
> > > > > > functions to (un)register drivers. It exists such that a generic Registration
> > > > > > implementation calls the correct one for the subsystem.
> > > > > > 
> > > > > > For instance, PCI would implement DriverOps::register() by calling into
> > > > > > bindings::__pci_register_driver().
> > > > > > 
> > > > > > We can discuss whether DriverOps is a good name for the trait, but it's not a
> > > > > > (different) name for something that already exists and already has a name.
> > > > > 
> > > > > It's a name we don't have in the C code as the design of the driver core
> > > > > does not need or provide it.  It's just the section of 'struct
> > > > > device_driver' that provides function callbacks, why does it need to be
> > > > > separate at all?
> > > > 
> > > > I'm confused by the relationship to `struct device_driver` you seem to imply.
> > > > How is it related?
> > > > 
> > > > Again, this is just a trait for subsystems to provide their corresponding
> > > > register and unregister implementation, e.g. pci_register_driver() and
> > > > pci_unregister_driver(), such that they can be called from the generic
> > > > Registration code below.
> > > > 
> > > > See [1] for an example implementation in PCI.
> > > 
> > > registering and unregistering drivers belongs in the bus code, NOT in
> > > the driver code.
> > 
> > Why? We're not (re-)implementing a bus here. Again, those are just abstractions
> > to call the C functions to register a driver. The corresponding C functions are
> > e.g. driver_register() or __pci_register_driver(). Those are defined in
> > drivers/base/driver.c and drivers/pci/pci-driver.c respectively.
> > 
> > Why wouldn't we follow the same scheme in Rust abstractions?
> 
> It's the bus that does the registering, so yeah, don't put it here at
> all as it's not going to be needed (i.e. unless you write a bus in rust
> you will never call driver_register())  So this can just be a wrapper
> for the pci bus logic, keeping it simpler.

We never call driver_register() of course, I gave this example for another
reason above. Sorry if that was confusing.

> 
> So you might be able to delete a lot of code here, only deal with a
> "dumb" struct device wrapper to handle reference counts, and then do the
> rest for the specific bus bindings?  Or is that too much to dream?

Again, this is a generalization such that we do not have to replicate code for
every subsystem / bus. Please see the full explanation below.

> 
> You aren't writing a "raw" driver here, no one does that, it's a bus
> that handles that logic for you, and you should not have to expose any
> "raw" driver attributes.

Indeed - we're not doing that here.

> 
> Yes, for some busses, they like to force a driver to set the "raw"
> driver attribute, but I don't think that's a good idea and for the pci
> driver layer, that shouldn't be necessary now, right?  If not, what
> fields are you wanting to get direct access to?

Honestly, this all reads as if you did not (carefully) read the code we're
discussing about in the first place, did you?

It reads more as if you take assumptions based on my previous explanations, and
since communication is difficult, it looks like we're talking past each other.
Maybe also my explanations were just not good enough. :(

Either way, I suggest to focus more on the actual code. In particular let's have
a look at the `Registration` and `DriverOps` struct from this patch and how
they're used in the PCI code and in an actual driver.

Please have a look at how the PCI code implements the `DriverOps` trait (or
interface as many other languages would call it) [1]. In `DriverOps::register`
and `DriverOps::unregister` the PCI code simply calls the C functions
__pci_register_driver() and pci_unregister_driver().

A driver can use the `module_pci_driver!` macro [2] to declare a kernel module
that registers a single PCI driver. This is equivalent to C's
module_pci_driver() macro.

The `module_pci_driver!` macro calls the generic `module_driver!` macro [3] and
passes the `pci::Adapter` [1] (the PCI thing that actually calls the C
pci_{un}register_driver() functions).

The `module_driver!` macro creates a new `Module` structure [4] that holds the
`Registration` structure. This `Registration` structure has a generic argument
which is the `pci::Adapter` [1]. Which means that once the `Registration` is
created C's __pci_register_driver() is called and once it is destroyed C's
pci_unregister_driver() is called. The `Registration` is created in
module_init() and destroyed in module_exit() accordingly.

So, as you can see a `Registration` is really just the parts generalized that
otherwise every subsystem would need to implement and `DriverOps` is the glue
between a driver `Registration` and the subsystem (e.g. PCI) defining which
function to call on driver register() or driver unregister().

My argument above is simply that since  all this is just the abstraction to
declare a driver structure and register it, it belongs into driver.rs. At least
if we want to go along with where the C side places the correspong functions on
the C side.

(The links already point to the new code that allocates the driver statically,
but this should not matter, since conceptually it's the same.)

[1] https://gitlab.freedesktop.org/drm/nova/-/blob/989338f129146af9952304c2cc6b33fbd90e8909/rust/kernel/pci.rs#L24
[2] https://gitlab.freedesktop.org/drm/nova/-/blob/989338f129146af9952304c2cc6b33fbd90e8909/rust/kernel/pci.rs#L135
[3] https://gitlab.freedesktop.org/drm/nova/-/blob/989338f129146af9952304c2cc6b33fbd90e8909/rust/kernel/driver.rs#L453
[4] https://gitlab.freedesktop.org/drm/nova/-/blob/989338f129146af9952304c2cc6b33fbd90e8909/rust/kernel/driver.rs#L435

> 
> thanks,
> 
> greg k-h
>