* [PATCH v2 0/3] Support more safe `AsBytes`/`FromBytes` usage
@ 2025-12-16 0:44 Matthew Maurer
2025-12-16 0:44 ` [PATCH v2 1/3] rust: transmute: Support transmuting slices of AsBytes/FromBytes types Matthew Maurer
` (2 more replies)
0 siblings, 3 replies; 18+ messages in thread
From: Matthew Maurer @ 2025-12-16 0:44 UTC (permalink / raw)
To: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich
Cc: rust-for-linux, linux-kernel, Matthew Maurer
Currently:
* Slices of `AsBytes`/`FromBytes` types cannot be synthesized from
bytes slices (without unsafe).
* Users must use `unsafe impl` to assert that structs are `AsBytes` or
`FromBytes` and write appropriate justifications.
* Bindgen-generated types cannot implement `AsBytes` or `FromBytes`,
meaning that casting them to or from bytes involves assumptions in the
`unsafe impl` that could easily go out of sync if the underlying
header is edited or an assumption is invalid on a platform the author
did not consider.
This series seeks to address all there of these by:
1. Adding slice cast functions to `FromBytes`
2. Adding a derive for `AsBytes` and `FromBytes`, for now restricted to
the simple case of structs.
3. Refactoring the crate structure to allow the derives added in 2 to be
used on bindgen definitions.
1 or 2 can be taken independently, 3 requires 2.
Signed-off-by: Matthew Maurer <mmaurer@google.com>
---
Changes in v2:
- Reworked to put `transmute` in `ffi` rather than creating a new crate,
per Alice's comment on Zulip.
- Switched to new kernel import style.
- Link to v1: https://lore.kernel.org/r/20251212-transmute-v1-0-9b28e06c6508@google.com
---
Matthew Maurer (3):
rust: transmute: Support transmuting slices of AsBytes/FromBytes types
rust: Add support for deriving `AsBytes` and `FromBytes`
rust: Support deriving `AsBytes`/`FromBytes` on bindgen types
rust/Makefile | 14 ++++---
rust/bindgen_parameters | 8 ++++
rust/bindings/lib.rs | 4 ++
rust/{ffi.rs => ffi/lib.rs} | 5 +++
rust/{kernel => ffi}/transmute.rs | 72 +++++++++++++++++++++++++++++++++
rust/kernel/lib.rs | 2 +-
rust/macros/lib.rs | 83 +++++++++++++++++++++++++++++++++++++++
rust/macros/transmute.rs | 60 ++++++++++++++++++++++++++++
rust/uapi/lib.rs | 4 ++
scripts/generate_rust_analyzer.py | 2 +-
10 files changed, 247 insertions(+), 7 deletions(-)
---
base-commit: 008d3547aae5bc86fac3eda317489169c3fda112
change-id: 20251212-transmute-8ab6076700a8
Best regards,
--
Matthew Maurer <mmaurer@google.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v2 1/3] rust: transmute: Support transmuting slices of AsBytes/FromBytes types
2025-12-16 0:44 [PATCH v2 0/3] Support more safe `AsBytes`/`FromBytes` usage Matthew Maurer
@ 2025-12-16 0:44 ` Matthew Maurer
2025-12-17 16:51 ` Daniel Almeida
2025-12-16 0:44 ` [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes` Matthew Maurer
2025-12-16 0:44 ` [PATCH v2 3/3] rust: Support deriving `AsBytes`/`FromBytes` on bindgen types Matthew Maurer
2 siblings, 1 reply; 18+ messages in thread
From: Matthew Maurer @ 2025-12-16 0:44 UTC (permalink / raw)
To: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich
Cc: rust-for-linux, linux-kernel, Matthew Maurer
Currently, we support either transmuting a byte slice of the exact size
of the target type or a single element from a prefix of a byte slice.
This adds support for transmuting from a byte slice to a slice of
elements, assuming appropriate traits are implemented.
Signed-off-by: Matthew Maurer <mmaurer@google.com>
---
rust/kernel/transmute.rs | 72 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 72 insertions(+)
diff --git a/rust/kernel/transmute.rs b/rust/kernel/transmute.rs
index be5dbf3829e25c92f85083cc0f4940f35bb7baec..5cc5f4fde4c31cea3c51ab078b4d68f3a0a2caa1 100644
--- a/rust/kernel/transmute.rs
+++ b/rust/kernel/transmute.rs
@@ -122,6 +122,78 @@ fn from_bytes_mut_prefix(bytes: &mut [u8]) -> Option<(&mut Self, &mut [u8])>
}
}
+ /// Converts a slice of bytes to a slice of `Self`.
+ ///
+ /// Succeeds if the reference is properly aligned, and the size of `bytes` is a multiple of that
+ /// of `Self`.
+ ///
+ /// Otherwise, returns [`None`].
+ fn from_bytes_to_slice(bytes: &[u8]) -> Option<&[Self]>
+ where
+ Self: Sized,
+ {
+ let size = size_of::<Self>();
+ if size == 0 {
+ return None;
+ }
+ if bytes.len() % size != 0 {
+ return None;
+ }
+ let len = bytes.len() / size;
+ let ptr = bytes.as_ptr().cast::<Self>();
+
+ #[allow(clippy::incompatible_msrv)]
+ if ptr.is_aligned() {
+ // SAFETY:
+ // - `ptr` is valid for reads of `bytes.len()` bytes, which is
+ // `len * size_of::<Self>()`.
+ // - `ptr` is aligned for `Self`.
+ // - `ptr` points to `len` consecutive properly initialized values of type `Self`
+ // because `Self` implements `FromBytes` (any bit pattern is valid).
+ // - The lifetime of the returned slice is bound to the input `bytes`.
+ unsafe { Some(core::slice::from_raw_parts(ptr, len)) }
+ } else {
+ None
+ }
+ }
+
+ /// Converts a mutable slice of bytes to a mutable slice of `Self`.
+ ///
+ /// Succeeds if the reference is properly aligned, and the size of `bytes` is a multiple of that
+ /// of `Self`.
+ ///
+ /// Otherwise, returns [`None`].
+ fn from_bytes_to_mut_slice(bytes: &mut [u8]) -> Option<&mut [Self]>
+ where
+ Self: AsBytes + Sized,
+ {
+ let size = size_of::<Self>();
+ if size == 0 {
+ return None;
+ }
+ if bytes.len() % size != 0 {
+ return None;
+ }
+ let len = bytes.len() / size;
+ let ptr = bytes.as_mut_ptr().cast::<Self>();
+
+ #[allow(clippy::incompatible_msrv)]
+ if ptr.is_aligned() {
+ // SAFETY:
+ // - `ptr` is valid for reads and writes of `bytes.len()` bytes, which is
+ // `len * size_of::<Self>()`.
+ // - `ptr` is aligned for `Self`.
+ // - `ptr` points to `len` consecutive properly initialized values of type `Self`
+ // because `Self` implements `FromBytes`.
+ // - `AsBytes` guarantees that writing a new `Self` to the resulting slice can safely
+ // be reflected as bytes in the original slice.
+ // - The lifetime of the returned slice is bound to the input `bytes`.
+ unsafe { Some(core::slice::from_raw_parts_mut(ptr, len)) }
+ } else {
+ None
+ }
+ }
+
/// Creates an owned instance of `Self` by copying `bytes`.
///
/// Unlike [`FromBytes::from_bytes`], which requires aligned input, this method can be used on
--
2.52.0.305.g3fc767764a-goog
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes`
2025-12-16 0:44 [PATCH v2 0/3] Support more safe `AsBytes`/`FromBytes` usage Matthew Maurer
2025-12-16 0:44 ` [PATCH v2 1/3] rust: transmute: Support transmuting slices of AsBytes/FromBytes types Matthew Maurer
@ 2025-12-16 0:44 ` Matthew Maurer
2025-12-17 3:12 ` Alexandre Courbot
2025-12-17 17:35 ` Daniel Almeida
2025-12-16 0:44 ` [PATCH v2 3/3] rust: Support deriving `AsBytes`/`FromBytes` on bindgen types Matthew Maurer
2 siblings, 2 replies; 18+ messages in thread
From: Matthew Maurer @ 2025-12-16 0:44 UTC (permalink / raw)
To: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich
Cc: rust-for-linux, linux-kernel, Matthew Maurer
This provides a derive macro for `AsBytes` and `FromBytes` for structs
only. For both, it checks the respective trait on every underlying
field. For `AsBytes`, it emits a const-time padding check that will fail
the compilation if derived on a type with padding.
Signed-off-by: Matthew Maurer <mmaurer@google.com>
---
rust/macros/lib.rs | 63 ++++++++++++++++++++++++++++++++++++++++++++++++
rust/macros/transmute.rs | 58 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 121 insertions(+)
diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
index b38002151871a33f6b4efea70be2deb6ddad38e2..d66397942529f67697f74a908e257cacc4201d84 100644
--- a/rust/macros/lib.rs
+++ b/rust/macros/lib.rs
@@ -20,9 +20,14 @@
mod kunit;
mod module;
mod paste;
+mod transmute;
mod vtable;
use proc_macro::TokenStream;
+use syn::{
+ parse_macro_input,
+ DeriveInput, //
+};
/// Declares a kernel module.
///
@@ -475,3 +480,61 @@ pub fn paste(input: TokenStream) -> TokenStream {
pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
kunit::kunit_tests(attr, ts)
}
+
+/// Implements `FromBytes` for a struct.
+///
+/// It will fail compilation if the struct you are deriving on cannot be determined to implement
+/// `FromBytes` safely. It may still fail for some types which would be safe to implement
+/// `FromBytes` for, in which case you will need to write the implementation and justification
+/// yourself.
+///
+/// Main reasons your type may be rejected:
+/// * Not a `struct`
+/// * One of the fields is not `FromBytes`
+///
+/// # Examples
+///
+/// ```
+/// #[derive(FromBytes)]
+/// #[repr(C)]
+/// struct Foo {
+/// x: u32,
+/// y: u16,
+/// z: u16,
+/// }
+/// ```
+#[proc_macro_derive(FromBytes)]
+pub fn derive_from_bytes(tokens: TokenStream) -> TokenStream {
+ let input = parse_macro_input!(tokens as DeriveInput);
+ transmute::from_bytes(input).into()
+}
+
+/// Implements `AsBytes` for a struct.
+///
+/// It will fail compilation if the struct you are deriving on cannot be determined to implement
+/// `AsBytes` safely. It may still fail for some structures which would be safe to implement
+/// `AsBytes`, in which case you will need to write the implementation and justification
+/// yourself.
+///
+/// Main reasons your type may be rejected:
+/// * Not a `struct`
+/// * One of the fields is not `AsBytes`
+/// * Your struct has generic parameters
+/// * There is padding somewhere in your struct
+///
+/// # Examples
+///
+/// ```
+/// #[derive(AsBytes)]
+/// #[repr(C)]
+/// struct Foo {
+/// x: u32,
+/// y: u16,
+/// z: u16,
+/// }
+/// ```
+#[proc_macro_derive(AsBytes)]
+pub fn derive_as_bytes(tokens: TokenStream) -> TokenStream {
+ let input = parse_macro_input!(tokens as DeriveInput);
+ transmute::as_bytes(input).into()
+}
diff --git a/rust/macros/transmute.rs b/rust/macros/transmute.rs
new file mode 100644
index 0000000000000000000000000000000000000000..43cf36a1334f1fed23c0e777026392f987f78d8d
--- /dev/null
+++ b/rust/macros/transmute.rs
@@ -0,0 +1,58 @@
+// SPDX-License-Identifier: GPL-2.0
+
+use proc_macro2::TokenStream;
+use syn::{parse_quote, DeriveInput, Fields, Ident, ItemConst, Path, WhereClause};
+
+fn all_fields_impl(fields: &Fields, trait_: &Path) -> WhereClause {
+ let tys = fields.iter().map(|field| &field.ty);
+ parse_quote! {
+ where #(for<'a> #tys: #trait_),*
+ }
+}
+
+fn struct_padding_check(fields: &Fields, name: &Ident) -> ItemConst {
+ let tys = fields.iter().map(|field| &field.ty);
+ parse_quote! {
+ const _: () = {
+ assert!(#(core::mem::size_of::<#tys>())+* == core::mem::size_of::<#name>());
+ };
+ }
+}
+
+pub(crate) fn as_bytes(input: DeriveInput) -> TokenStream {
+ if !input.generics.params.is_empty() {
+ return quote::quote! { compile_error!("#[derive(AsBytes)] does not support generics") };
+ }
+ let syn::Data::Struct(ref ds) = &input.data else {
+ return quote::quote! { compile_error!("#[derive(AsBytes)] only supports structs") };
+ };
+ let name = input.ident;
+ let trait_ = parse_quote! { ::kernel::transmute::AsBytes };
+ let where_clause = all_fields_impl(&ds.fields, &trait_);
+ let padding_check = struct_padding_check(&ds.fields, &name);
+ quote::quote! {
+ #padding_check
+ // SAFETY: #name has no padding and all of its fields implement `AsBytes`
+ unsafe impl #trait_ for #name #where_clause {}
+ }
+}
+
+pub(crate) fn from_bytes(input: DeriveInput) -> TokenStream {
+ let syn::Data::Struct(ref ds) = &input.data else {
+ return quote::quote! { compile_error!("#[derive(FromBytes)] only supports structs") };
+ };
+ let (impl_generics, ty_generics, base_where_clause) = input.generics.split_for_impl();
+ let name = input.ident;
+ let trait_ = parse_quote! { ::kernel::transmute::FromBytes };
+ let mut where_clause = all_fields_impl(&ds.fields, &trait_);
+ if let Some(base_clause) = base_where_clause {
+ where_clause
+ .predicates
+ .extend(base_clause.predicates.clone())
+ };
+ quote::quote! {
+ // SAFETY: All fields of #name implement `FromBytes` and it is a struct, so there is no
+ // implicit discriminator.
+ unsafe impl #impl_generics #trait_ for #name #ty_generics #where_clause {}
+ }
+}
--
2.52.0.305.g3fc767764a-goog
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v2 3/3] rust: Support deriving `AsBytes`/`FromBytes` on bindgen types
2025-12-16 0:44 [PATCH v2 0/3] Support more safe `AsBytes`/`FromBytes` usage Matthew Maurer
2025-12-16 0:44 ` [PATCH v2 1/3] rust: transmute: Support transmuting slices of AsBytes/FromBytes types Matthew Maurer
2025-12-16 0:44 ` [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes` Matthew Maurer
@ 2025-12-16 0:44 ` Matthew Maurer
2025-12-17 3:15 ` Alexandre Courbot
2025-12-17 19:26 ` Daniel Almeida
2 siblings, 2 replies; 18+ messages in thread
From: Matthew Maurer @ 2025-12-16 0:44 UTC (permalink / raw)
To: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich
Cc: rust-for-linux, linux-kernel, Matthew Maurer
To support this, we need to move the `transmute` module into a separate
crate to allow the `bindings` crate to depend on it. Most user code is
still expected to address the module as `kernel::transmute`, which is a
re-export. `ffi::transmute` is now available for use in `bindings`.
Signed-off-by: Matthew Maurer <mmaurer@google.com>
---
rust/Makefile | 14 +++++++++-----
rust/bindgen_parameters | 8 ++++++++
rust/bindings/lib.rs | 4 ++++
rust/{ffi.rs => ffi/lib.rs} | 5 +++++
rust/{kernel => ffi}/transmute.rs | 0
rust/kernel/lib.rs | 2 +-
rust/macros/lib.rs | 24 ++++++++++++++++++++++--
rust/macros/transmute.rs | 12 +++++++-----
rust/uapi/lib.rs | 4 ++++
scripts/generate_rust_analyzer.py | 2 +-
10 files changed, 61 insertions(+), 14 deletions(-)
diff --git a/rust/Makefile b/rust/Makefile
index 5d357dce1704d15e43effc528be8f5a4d74d3d8d..178aa3036c4acba1c4c266196912da2d60fe0d5f 100644
--- a/rust/Makefile
+++ b/rust/Makefile
@@ -207,7 +207,7 @@ rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE
+$(call if_changed,rustdoc)
rustdoc-ffi: private is-kernel-object := y
-rustdoc-ffi: $(src)/ffi.rs rustdoc-core FORCE
+rustdoc-ffi: $(src)/ffi/lib.rs rustdoc-core FORCE
+$(call if_changed,rustdoc)
rustdoc-pin_init_internal: private rustdoc_host = yes
@@ -249,7 +249,7 @@ quiet_cmd_rustc_test_library = $(RUSTC_OR_CLIPPY_QUIET) TL $<
rusttestlib-build_error: $(src)/build_error.rs FORCE
+$(call if_changed,rustc_test_library)
-rusttestlib-ffi: $(src)/ffi.rs FORCE
+rusttestlib-ffi: $(src)/ffi/lib.rs FORCE
+$(call if_changed,rustc_test_library)
rusttestlib-proc_macro2: private rustc_target_flags = $(proc_macro2-flags)
@@ -657,22 +657,26 @@ $(obj)/build_error.o: $(src)/build_error.rs $(obj)/compiler_builtins.o FORCE
+$(call if_changed_rule,rustc_library)
$(obj)/ffi.o: private skip_gendwarfksyms = 1
-$(obj)/ffi.o: $(src)/ffi.rs $(obj)/compiler_builtins.o FORCE
+$(obj)/ffi.o: $(src)/ffi/lib.rs $(obj)/compiler_builtins.o FORCE
+$(call if_changed_rule,rustc_library)
-$(obj)/bindings.o: private rustc_target_flags = --extern ffi --extern pin_init
+$(obj)/bindings.o: private rustc_target_flags = --extern ffi --extern pin_init \
+ --extern macros
$(obj)/bindings.o: $(src)/bindings/lib.rs \
$(obj)/ffi.o \
$(obj)/pin_init.o \
+ $(obj)/$(libmacros_name) \
$(obj)/bindings/bindings_generated.rs \
$(obj)/bindings/bindings_helpers_generated.rs FORCE
+$(call if_changed_rule,rustc_library)
-$(obj)/uapi.o: private rustc_target_flags = --extern ffi --extern pin_init
+$(obj)/uapi.o: private rustc_target_flags = --extern ffi --extern pin_init \
+ --extern macros
$(obj)/uapi.o: private skip_gendwarfksyms = 1
$(obj)/uapi.o: $(src)/uapi/lib.rs \
$(obj)/ffi.o \
$(obj)/pin_init.o \
+ $(obj)/$(libmacros_name) \
$(obj)/uapi/uapi_generated.rs FORCE
+$(call if_changed_rule,rustc_library)
diff --git a/rust/bindgen_parameters b/rust/bindgen_parameters
index fd2fd1c3cb9a51ea46fcd721907783b457aa1378..d56343ca03979e345f8adb7eb8fd7f2b9d4be6ee 100644
--- a/rust/bindgen_parameters
+++ b/rust/bindgen_parameters
@@ -64,3 +64,11 @@
# Structs should implement `Zeroable` when all of their fields do.
--with-derive-custom-struct .*=MaybeZeroable
--with-derive-custom-union .*=MaybeZeroable
+
+# Every C struct can try to derive FromBytes. If they have unmet requirements,
+# the impl will just be a no-op due to the where clause.
+--with-derive-custom-struct .*=FromBytesFfi
+
+# We can't auto-derive AsBytes, as we need a const-time check to see if there
+# is padding involved. Add it explicitly when you expect no padding.
+--with-derive-custom-struct cpumask=AsBytesFfi
diff --git a/rust/bindings/lib.rs b/rust/bindings/lib.rs
index 0c57cf9b4004f176997c59ecc58a9a9ac76163d9..c29312fca1b01b707bb11b05d446d44480a4b81f 100644
--- a/rust/bindings/lib.rs
+++ b/rust/bindings/lib.rs
@@ -31,6 +31,10 @@
#[allow(clippy::undocumented_unsafe_blocks)]
#[cfg_attr(CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES, allow(unnecessary_transmutes))]
mod bindings_raw {
+ use macros::{
+ AsBytesFfi,
+ FromBytesFfi, //
+ };
use pin_init::{MaybeZeroable, Zeroable};
// Manual definition for blocklisted types.
diff --git a/rust/ffi.rs b/rust/ffi/lib.rs
similarity index 87%
rename from rust/ffi.rs
rename to rust/ffi/lib.rs
index f961e9728f590fd2c52d4c03a1f715d654051d04..14052362f091a609bc505fe6eca77fe998fe2321 100644
--- a/rust/ffi.rs
+++ b/rust/ffi/lib.rs
@@ -10,6 +10,11 @@
#![no_std]
+#[doc(hidden)]
+// This lives here to make it accessible to `bindings`, similar to the other `ffi` types.
+// User code should access it through `kernel::transmute`.
+pub mod transmute;
+
macro_rules! alias {
($($name:ident = $ty:ty;)*) => {$(
#[allow(non_camel_case_types, missing_docs)]
diff --git a/rust/kernel/transmute.rs b/rust/ffi/transmute.rs
similarity index 100%
rename from rust/kernel/transmute.rs
rename to rust/ffi/transmute.rs
diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
index f812cf12004286962985a068665443dc22c389a2..4aa54dd83319ef16bd4baa1964114f1e6549942b 100644
--- a/rust/kernel/lib.rs
+++ b/rust/kernel/lib.rs
@@ -63,6 +63,7 @@
extern crate self as kernel;
pub use ffi;
+pub use ffi::transmute;
pub mod acpi;
pub mod alloc;
@@ -146,7 +147,6 @@
pub mod task;
pub mod time;
pub mod tracepoint;
-pub mod transmute;
pub mod types;
pub mod uaccess;
#[cfg(CONFIG_USB = "y")]
diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
index d66397942529f67697f74a908e257cacc4201d84..bde94c2a8ddf708872bcd56a4713784b5ccdf04e 100644
--- a/rust/macros/lib.rs
+++ b/rust/macros/lib.rs
@@ -506,7 +506,7 @@ pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
#[proc_macro_derive(FromBytes)]
pub fn derive_from_bytes(tokens: TokenStream) -> TokenStream {
let input = parse_macro_input!(tokens as DeriveInput);
- transmute::from_bytes(input).into()
+ transmute::from_bytes("kernel", input).into()
}
/// Implements `AsBytes` for a struct.
@@ -536,5 +536,25 @@ pub fn derive_from_bytes(tokens: TokenStream) -> TokenStream {
#[proc_macro_derive(AsBytes)]
pub fn derive_as_bytes(tokens: TokenStream) -> TokenStream {
let input = parse_macro_input!(tokens as DeriveInput);
- transmute::as_bytes(input).into()
+ transmute::as_bytes("kernel", input).into()
+}
+
+#[doc(hidden)]
+#[proc_macro_derive(FromBytesFfi)]
+/// This is equivalent to `FromBytes`, but uses the `ffi` crate as the trait definition site
+/// instead of `kernel`. This is intended for use inside `bindings`. Everyone else can refer to the
+/// trait through `kernel`.
+pub fn derive_from_bytes_trait(tokens: TokenStream) -> TokenStream {
+ let input = parse_macro_input!(tokens as DeriveInput);
+ transmute::from_bytes("ffi", input).into()
+}
+
+#[doc(hidden)]
+#[proc_macro_derive(AsBytesFfi)]
+/// This is equivalent to `AsBytes`, but uses the `ffi` crate as the trait definition site
+/// instead of `kernel`. This is intended for use inside `bindings`. Everyone else can refer to the
+/// trait through `kernel`.
+pub fn derive_as_bytes_trait(tokens: TokenStream) -> TokenStream {
+ let input = parse_macro_input!(tokens as DeriveInput);
+ transmute::as_bytes("ffi", input).into()
}
diff --git a/rust/macros/transmute.rs b/rust/macros/transmute.rs
index 43cf36a1334f1fed23c0e777026392f987f78d8d..9bd6d279675fb1d58cdceff1f4dde0dcf33fbf99 100644
--- a/rust/macros/transmute.rs
+++ b/rust/macros/transmute.rs
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0
-use proc_macro2::TokenStream;
+use proc_macro2::{Span, TokenStream};
use syn::{parse_quote, DeriveInput, Fields, Ident, ItemConst, Path, WhereClause};
fn all_fields_impl(fields: &Fields, trait_: &Path) -> WhereClause {
@@ -19,7 +19,8 @@ fn struct_padding_check(fields: &Fields, name: &Ident) -> ItemConst {
}
}
-pub(crate) fn as_bytes(input: DeriveInput) -> TokenStream {
+pub(crate) fn as_bytes(crate_: &str, input: DeriveInput) -> TokenStream {
+ let crate_ = Ident::new(crate_, Span::call_site());
if !input.generics.params.is_empty() {
return quote::quote! { compile_error!("#[derive(AsBytes)] does not support generics") };
}
@@ -27,7 +28,7 @@ pub(crate) fn as_bytes(input: DeriveInput) -> TokenStream {
return quote::quote! { compile_error!("#[derive(AsBytes)] only supports structs") };
};
let name = input.ident;
- let trait_ = parse_quote! { ::kernel::transmute::AsBytes };
+ let trait_ = parse_quote! { ::#crate_::transmute::AsBytes };
let where_clause = all_fields_impl(&ds.fields, &trait_);
let padding_check = struct_padding_check(&ds.fields, &name);
quote::quote! {
@@ -37,13 +38,14 @@ unsafe impl #trait_ for #name #where_clause {}
}
}
-pub(crate) fn from_bytes(input: DeriveInput) -> TokenStream {
+pub(crate) fn from_bytes(crate_: &str, input: DeriveInput) -> TokenStream {
+ let crate_ = Ident::new(crate_, Span::call_site());
let syn::Data::Struct(ref ds) = &input.data else {
return quote::quote! { compile_error!("#[derive(FromBytes)] only supports structs") };
};
let (impl_generics, ty_generics, base_where_clause) = input.generics.split_for_impl();
let name = input.ident;
- let trait_ = parse_quote! { ::kernel::transmute::FromBytes };
+ let trait_ = parse_quote! { ::#crate_::transmute::FromBytes };
let mut where_clause = all_fields_impl(&ds.fields, &trait_);
if let Some(base_clause) = base_where_clause {
where_clause
diff --git a/rust/uapi/lib.rs b/rust/uapi/lib.rs
index 1d5fd9efb93e9db97fec84fca2bae37b500c20c5..2033b7125558d7ccfae67b94777f5ce59593a528 100644
--- a/rust/uapi/lib.rs
+++ b/rust/uapi/lib.rs
@@ -34,6 +34,10 @@
type __kernel_ssize_t = isize;
type __kernel_ptrdiff_t = isize;
+use macros::{
+ AsBytesFfi,
+ FromBytesFfi, //
+};
use pin_init::MaybeZeroable;
include!(concat!(env!("OBJTREE"), "/rust/uapi/uapi_generated.rs"));
diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
index 147d0cc940681426771db865bc2462e7029a6d7d..843d081eacaca8edeeac5978bd8107a498008186 100755
--- a/scripts/generate_rust_analyzer.py
+++ b/scripts/generate_rust_analyzer.py
@@ -137,7 +137,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs, core_edit
append_crate(
"ffi",
- srctree / "rust" / "ffi.rs",
+ srctree / "rust" / "ffi" / "lib.rs",
["core", "compiler_builtins"],
)
--
2.52.0.305.g3fc767764a-goog
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes`
2025-12-16 0:44 ` [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes` Matthew Maurer
@ 2025-12-17 3:12 ` Alexandre Courbot
2025-12-17 18:01 ` Matthew Maurer
2025-12-17 17:35 ` Daniel Almeida
1 sibling, 1 reply; 18+ messages in thread
From: Alexandre Courbot @ 2025-12-17 3:12 UTC (permalink / raw)
To: Matthew Maurer, Miguel Ojeda, Boqun Feng, Gary Guo,
Björn Roy Baron, Benno Lossin, Andreas Hindborg, Alice Ryhl,
Trevor Gross, Danilo Krummrich
Cc: rust-for-linux, linux-kernel
On Tue Dec 16, 2025 at 9:44 AM JST, Matthew Maurer wrote:
> This provides a derive macro for `AsBytes` and `FromBytes` for structs
> only. For both, it checks the respective trait on every underlying
> field. For `AsBytes`, it emits a const-time padding check that will fail
> the compilation if derived on a type with padding.
>
> Signed-off-by: Matthew Maurer <mmaurer@google.com>
I like this a lot. We have a bunch of unsafe impls in Nova that this
could help us get rid of.
Amazed that this even seems to work on tuple structs!
> ---
> rust/macros/lib.rs | 63 ++++++++++++++++++++++++++++++++++++++++++++++++
> rust/macros/transmute.rs | 58 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 121 insertions(+)
>
> diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
> index b38002151871a33f6b4efea70be2deb6ddad38e2..d66397942529f67697f74a908e257cacc4201d84 100644
> --- a/rust/macros/lib.rs
> +++ b/rust/macros/lib.rs
> @@ -20,9 +20,14 @@
> mod kunit;
> mod module;
> mod paste;
> +mod transmute;
> mod vtable;
>
> use proc_macro::TokenStream;
> +use syn::{
> + parse_macro_input,
> + DeriveInput, //
> +};
>
> /// Declares a kernel module.
> ///
> @@ -475,3 +480,61 @@ pub fn paste(input: TokenStream) -> TokenStream {
> pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
> kunit::kunit_tests(attr, ts)
> }
> +
> +/// Implements `FromBytes` for a struct.
> +///
> +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
> +/// `FromBytes` safely. It may still fail for some types which would be safe to implement
> +/// `FromBytes` for, in which case you will need to write the implementation and justification
> +/// yourself.
> +///
> +/// Main reasons your type may be rejected:
> +/// * Not a `struct`
> +/// * One of the fields is not `FromBytes`
> +///
> +/// # Examples
> +///
> +/// ```
> +/// #[derive(FromBytes)]
> +/// #[repr(C)]
> +/// struct Foo {
> +/// x: u32,
> +/// y: u16,
> +/// z: u16,
> +/// }
> +/// ```
One thing I have noticed is that I could sucessfully derive `FromBytes`
on a struct that is not `repr(C)`... Is that something we want to
disallow?
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 3/3] rust: Support deriving `AsBytes`/`FromBytes` on bindgen types
2025-12-16 0:44 ` [PATCH v2 3/3] rust: Support deriving `AsBytes`/`FromBytes` on bindgen types Matthew Maurer
@ 2025-12-17 3:15 ` Alexandre Courbot
2025-12-17 18:26 ` Matthew Maurer
2025-12-17 19:26 ` Daniel Almeida
1 sibling, 1 reply; 18+ messages in thread
From: Alexandre Courbot @ 2025-12-17 3:15 UTC (permalink / raw)
To: Matthew Maurer, Miguel Ojeda, Boqun Feng, Gary Guo,
Björn Roy Baron, Benno Lossin, Andreas Hindborg, Alice Ryhl,
Trevor Gross, Danilo Krummrich
Cc: rust-for-linux, linux-kernel
On Tue Dec 16, 2025 at 9:44 AM JST, Matthew Maurer wrote:
> To support this, we need to move the `transmute` module into a separate
> crate to allow the `bindings` crate to depend on it. Most user code is
> still expected to address the module as `kernel::transmute`, which is a
> re-export. `ffi::transmute` is now available for use in `bindings`.
>
> Signed-off-by: Matthew Maurer <mmaurer@google.com>
Maybe this should be two commits, one for the new crate, another one to
introduce the ability to use on bindgen types.
I have tried this with the Nova bindings, and somehow could not get past
this error:
error[E0433]: failed to resolve: could not find `ffi` in the list of imported crates
--> ../drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs:321:54
|
321 | #[derive(Debug, Default, Copy, Clone, MaybeZeroable, FromBytesFfi)]
| ^^^^^^^^^^^^ could not find `ffi` in the list of imported crates
|
= note: this error originates in the derive macro `FromBytesFfi` (in Nightly builds, run with -Z macro-backtrace for more info)
I do have `kernel::ffi` imported in the bindings module though, so I am
not quite sure what this is about. Any idea?
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 1/3] rust: transmute: Support transmuting slices of AsBytes/FromBytes types
2025-12-16 0:44 ` [PATCH v2 1/3] rust: transmute: Support transmuting slices of AsBytes/FromBytes types Matthew Maurer
@ 2025-12-17 16:51 ` Daniel Almeida
2025-12-26 20:27 ` Matthew Maurer
0 siblings, 1 reply; 18+ messages in thread
From: Daniel Almeida @ 2025-12-17 16:51 UTC (permalink / raw)
To: Matthew Maurer
Cc: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich, rust-for-linux, linux-kernel
Hi Matthew,
> On 15 Dec 2025, at 21:44, Matthew Maurer <mmaurer@google.com> wrote:
>
> Currently, we support either transmuting a byte slice of the exact size
> of the target type or a single element from a prefix of a byte slice.
>
> This adds support for transmuting from a byte slice to a slice of
> elements, assuming appropriate traits are implemented.
>
> Signed-off-by: Matthew Maurer <mmaurer@google.com>
> ---
> rust/kernel/transmute.rs | 72 ++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 72 insertions(+)
>
> diff --git a/rust/kernel/transmute.rs b/rust/kernel/transmute.rs
> index be5dbf3829e25c92f85083cc0f4940f35bb7baec..5cc5f4fde4c31cea3c51ab078b4d68f3a0a2caa1 100644
> --- a/rust/kernel/transmute.rs
> +++ b/rust/kernel/transmute.rs
> @@ -122,6 +122,78 @@ fn from_bytes_mut_prefix(bytes: &mut [u8]) -> Option<(&mut Self, &mut [u8])>
> }
> }
>
> + /// Converts a slice of bytes to a slice of `Self`.
> + ///
> + /// Succeeds if the reference is properly aligned, and the size of `bytes` is a multiple of that
> + /// of `Self`.
> + ///
> + /// Otherwise, returns [`None`].
> + fn from_bytes_to_slice(bytes: &[u8]) -> Option<&[Self]>
> + where
> + Self: Sized,
> + {
> + let size = size_of::<Self>();
> + if size == 0 {
> + return None;
> + }
> + if bytes.len() % size != 0 {
> + return None;
> + }
> + let len = bytes.len() / size;
> + let ptr = bytes.as_ptr().cast::<Self>();
> +
> + #[allow(clippy::incompatible_msrv)]
> + if ptr.is_aligned() {
> + // SAFETY:
> + // - `ptr` is valid for reads of `bytes.len()` bytes, which is
> + // `len * size_of::<Self>()`.
> + // - `ptr` is aligned for `Self`.
> + // - `ptr` points to `len` consecutive properly initialized values of type `Self`
> + // because `Self` implements `FromBytes` (any bit pattern is valid).
> + // - The lifetime of the returned slice is bound to the input `bytes`.
> + unsafe { Some(core::slice::from_raw_parts(ptr, len)) }
> + } else {
> + None
> + }
Nit: I’d rewrite this as
if !ptr.is_aligned() {
return None;
}
unsafe { Some(core::slice::from_raw_parts(ptr, len)) }
This saves one indentation level.
> + }
> +
> + /// Converts a mutable slice of bytes to a mutable slice of `Self`.
> + ///
> + /// Succeeds if the reference is properly aligned, and the size of `bytes` is a multiple of that
> + /// of `Self`.
> + ///
> + /// Otherwise, returns [`None`].
> + fn from_bytes_to_mut_slice(bytes: &mut [u8]) -> Option<&mut [Self]>
> + where
> + Self: AsBytes + Sized,
> + {
> + let size = size_of::<Self>();
> + if size == 0 {
> + return None;
> + }
> + if bytes.len() % size != 0 {
> + return None;
> + }
> + let len = bytes.len() / size;
> + let ptr = bytes.as_mut_ptr().cast::<Self>();
> +
> + #[allow(clippy::incompatible_msrv)]
> + if ptr.is_aligned() {
> + // SAFETY:
> + // - `ptr` is valid for reads and writes of `bytes.len()` bytes, which is
> + // `len * size_of::<Self>()`.
> + // - `ptr` is aligned for `Self`.
> + // - `ptr` points to `len` consecutive properly initialized values of type `Self`
> + // because `Self` implements `FromBytes`.
> + // - `AsBytes` guarantees that writing a new `Self` to the resulting slice can safely
> + // be reflected as bytes in the original slice.
> + // - The lifetime of the returned slice is bound to the input `bytes`.
> + unsafe { Some(core::slice::from_raw_parts_mut(ptr, len)) }
> + } else {
> + None
> + }
Same here.
> + }
> +
> /// Creates an owned instance of `Self` by copying `bytes`.
> ///
> /// Unlike [`FromBytes::from_bytes`], which requires aligned input, this method can be used on
>
> --
> 2.52.0.305.g3fc767764a-goog
>
>
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes`
2025-12-16 0:44 ` [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes` Matthew Maurer
2025-12-17 3:12 ` Alexandre Courbot
@ 2025-12-17 17:35 ` Daniel Almeida
2025-12-17 17:57 ` Matthew Maurer
1 sibling, 1 reply; 18+ messages in thread
From: Daniel Almeida @ 2025-12-17 17:35 UTC (permalink / raw)
To: Matthew Maurer
Cc: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich, rust-for-linux, linux-kernel
Matthew,
> On 15 Dec 2025, at 21:44, Matthew Maurer <mmaurer@google.com> wrote:
>
> This provides a derive macro for `AsBytes` and `FromBytes` for structs
> only. For both, it checks the respective trait on every underlying
> field. For `AsBytes`, it emits a const-time padding check that will fail
> the compilation if derived on a type with padding.
>
> Signed-off-by: Matthew Maurer <mmaurer@google.com>
> ---
> rust/macros/lib.rs | 63 ++++++++++++++++++++++++++++++++++++++++++++++++
> rust/macros/transmute.rs | 58 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 121 insertions(+)
>
> diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
> index b38002151871a33f6b4efea70be2deb6ddad38e2..d66397942529f67697f74a908e257cacc4201d84 100644
> --- a/rust/macros/lib.rs
> +++ b/rust/macros/lib.rs
> @@ -20,9 +20,14 @@
> mod kunit;
> mod module;
> mod paste;
> +mod transmute;
> mod vtable;
>
> use proc_macro::TokenStream;
> +use syn::{
> + parse_macro_input,
> + DeriveInput, //
> +};
>
> /// Declares a kernel module.
> ///
> @@ -475,3 +480,61 @@ pub fn paste(input: TokenStream) -> TokenStream {
> pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
> kunit::kunit_tests(attr, ts)
> }
> +
> +/// Implements `FromBytes` for a struct.
> +///
> +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
> +/// `FromBytes` safely. It may still fail for some types which would be safe to implement
> +/// `FromBytes` for, in which case you will need to write the implementation and justification
> +/// yourself.
> +///
> +/// Main reasons your type may be rejected:
> +/// * Not a `struct`
> +/// * One of the fields is not `FromBytes`
> +///
> +/// # Examples
> +///
> +/// ```
> +/// #[derive(FromBytes)]
> +/// #[repr(C)]
> +/// struct Foo {
> +/// x: u32,
> +/// y: u16,
> +/// z: u16,
> +/// }
> +/// ```
> +#[proc_macro_derive(FromBytes)]
> +pub fn derive_from_bytes(tokens: TokenStream) -> TokenStream {
> + let input = parse_macro_input!(tokens as DeriveInput);
> + transmute::from_bytes(input).into()
> +}
> +
> +/// Implements `AsBytes` for a struct.
> +///
> +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
> +/// `AsBytes` safely. It may still fail for some structures which would be safe to implement
> +/// `AsBytes`, in which case you will need to write the implementation and justification
> +/// yourself.
> +///
> +/// Main reasons your type may be rejected:
> +/// * Not a `struct`
> +/// * One of the fields is not `AsBytes`
> +/// * Your struct has generic parameters
> +/// * There is padding somewhere in your struct
Why is padding relevant here but not in FromBytes?
> +///
> +/// # Examples
> +///
> +/// ```
> +/// #[derive(AsBytes)]
> +/// #[repr(C)]
> +/// struct Foo {
> +/// x: u32,
> +/// y: u16,
> +/// z: u16,
> +/// }
> +/// ```
> +#[proc_macro_derive(AsBytes)]
> +pub fn derive_as_bytes(tokens: TokenStream) -> TokenStream {
> + let input = parse_macro_input!(tokens as DeriveInput);
> + transmute::as_bytes(input).into()
> +}
> diff --git a/rust/macros/transmute.rs b/rust/macros/transmute.rs
> new file mode 100644
> index 0000000000000000000000000000000000000000..43cf36a1334f1fed23c0e777026392f987f78d8d
> --- /dev/null
> +++ b/rust/macros/transmute.rs
> @@ -0,0 +1,58 @@
> +// SPDX-License-Identifier: GPL-2.0
> +
> +use proc_macro2::TokenStream;
> +use syn::{parse_quote, DeriveInput, Fields, Ident, ItemConst, Path, WhereClause};
> +
> +fn all_fields_impl(fields: &Fields, trait_: &Path) -> WhereClause {
> + let tys = fields.iter().map(|field| &field.ty);
> + parse_quote! {
> + where #(for<'a> #tys: #trait_),*
Why do we need this hrtb here?
> + }
> +}
> +
> +fn struct_padding_check(fields: &Fields, name: &Ident) -> ItemConst {
> + let tys = fields.iter().map(|field| &field.ty);
> + parse_quote! {
> + const _: () = {
> + assert!(#(core::mem::size_of::<#tys>())+* == core::mem::size_of::<#name>());
> + };
> + }
> +}
> +
> +pub(crate) fn as_bytes(input: DeriveInput) -> TokenStream {
> + if !input.generics.params.is_empty() {
> + return quote::quote! { compile_error!("#[derive(AsBytes)] does not support generics") };
> + }
> + let syn::Data::Struct(ref ds) = &input.data else {
> + return quote::quote! { compile_error!("#[derive(AsBytes)] only supports structs") };
> + };
> + let name = input.ident;
> + let trait_ = parse_quote! { ::kernel::transmute::AsBytes };
> + let where_clause = all_fields_impl(&ds.fields, &trait_);
> + let padding_check = struct_padding_check(&ds.fields, &name);
> + quote::quote! {
> + #padding_check
> + // SAFETY: #name has no padding and all of its fields implement `AsBytes`
> + unsafe impl #trait_ for #name #where_clause {}
> + }
In general I’d add blanks.
> +}
> +
> +pub(crate) fn from_bytes(input: DeriveInput) -> TokenStream {
> + let syn::Data::Struct(ref ds) = &input.data else {
> + return quote::quote! { compile_error!("#[derive(FromBytes)] only supports structs") };
> + };
> + let (impl_generics, ty_generics, base_where_clause) = input.generics.split_for_impl();
> + let name = input.ident;
> + let trait_ = parse_quote! { ::kernel::transmute::FromBytes };
> + let mut where_clause = all_fields_impl(&ds.fields, &trait_);
> + if let Some(base_clause) = base_where_clause {
> + where_clause
> + .predicates
> + .extend(base_clause.predicates.clone())
> + };
> + quote::quote! {
> + // SAFETY: All fields of #name implement `FromBytes` and it is a struct, so there is no
> + // implicit discriminator.
> + unsafe impl #impl_generics #trait_ for #name #ty_generics #where_clause {}
> + }
> +}
>
> --
> 2.52.0.305.g3fc767764a-goog
>
>
Overall looks good. Please chime in on the two questions above.
— Daniel
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes`
2025-12-17 17:35 ` Daniel Almeida
@ 2025-12-17 17:57 ` Matthew Maurer
2025-12-17 19:11 ` Daniel Almeida
0 siblings, 1 reply; 18+ messages in thread
From: Matthew Maurer @ 2025-12-17 17:57 UTC (permalink / raw)
To: Daniel Almeida
Cc: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich, rust-for-linux, linux-kernel
On Wed, Dec 17, 2025 at 9:36 AM Daniel Almeida
<daniel.almeida@collabora.com> wrote:
>
> Matthew,
>
> > On 15 Dec 2025, at 21:44, Matthew Maurer <mmaurer@google.com> wrote:
> >
> > This provides a derive macro for `AsBytes` and `FromBytes` for structs
> > only. For both, it checks the respective trait on every underlying
> > field. For `AsBytes`, it emits a const-time padding check that will fail
> > the compilation if derived on a type with padding.
> >
> > Signed-off-by: Matthew Maurer <mmaurer@google.com>
> > ---
> > rust/macros/lib.rs | 63 ++++++++++++++++++++++++++++++++++++++++++++++++
> > rust/macros/transmute.rs | 58 ++++++++++++++++++++++++++++++++++++++++++++
> > 2 files changed, 121 insertions(+)
> >
> > diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
> > index b38002151871a33f6b4efea70be2deb6ddad38e2..d66397942529f67697f74a908e257cacc4201d84 100644
> > --- a/rust/macros/lib.rs
> > +++ b/rust/macros/lib.rs
> > @@ -20,9 +20,14 @@
> > mod kunit;
> > mod module;
> > mod paste;
> > +mod transmute;
> > mod vtable;
> >
> > use proc_macro::TokenStream;
> > +use syn::{
> > + parse_macro_input,
> > + DeriveInput, //
> > +};
> >
> > /// Declares a kernel module.
> > ///
> > @@ -475,3 +480,61 @@ pub fn paste(input: TokenStream) -> TokenStream {
> > pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
> > kunit::kunit_tests(attr, ts)
> > }
> > +
> > +/// Implements `FromBytes` for a struct.
> > +///
> > +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
> > +/// `FromBytes` safely. It may still fail for some types which would be safe to implement
> > +/// `FromBytes` for, in which case you will need to write the implementation and justification
> > +/// yourself.
> > +///
> > +/// Main reasons your type may be rejected:
> > +/// * Not a `struct`
> > +/// * One of the fields is not `FromBytes`
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// #[derive(FromBytes)]
> > +/// #[repr(C)]
> > +/// struct Foo {
> > +/// x: u32,
> > +/// y: u16,
> > +/// z: u16,
> > +/// }
> > +/// ```
> > +#[proc_macro_derive(FromBytes)]
> > +pub fn derive_from_bytes(tokens: TokenStream) -> TokenStream {
> > + let input = parse_macro_input!(tokens as DeriveInput);
> > + transmute::from_bytes(input).into()
> > +}
> > +
> > +/// Implements `AsBytes` for a struct.
> > +///
> > +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
> > +/// `AsBytes` safely. It may still fail for some structures which would be safe to implement
> > +/// `AsBytes`, in which case you will need to write the implementation and justification
> > +/// yourself.
> > +///
> > +/// Main reasons your type may be rejected:
> > +/// * Not a `struct`
> > +/// * One of the fields is not `AsBytes`
> > +/// * Your struct has generic parameters
> > +/// * There is padding somewhere in your struct
>
> Why is padding relevant here but not in FromBytes?
Padding bytes can be initialized to any value, but it is UB to observe
their contents because they may be implicitly uninitialized[1]. This
also matches the approach of `zerocopy`[2], which is the standard for
these sorts of transmutations outside the kernel - any byte can go
into the padding, but you cannot *read* any byte out of the padding.
[1]: https://doc.rust-lang.org/reference/behavior-considered-undefined.html#r-undefined.validity.undef
[2]: https://docs.rs/zerocopy/latest/zerocopy/trait.FromBytes.html#warning-padding-bytes
>
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// #[derive(AsBytes)]
> > +/// #[repr(C)]
> > +/// struct Foo {
> > +/// x: u32,
> > +/// y: u16,
> > +/// z: u16,
> > +/// }
> > +/// ```
> > +#[proc_macro_derive(AsBytes)]
> > +pub fn derive_as_bytes(tokens: TokenStream) -> TokenStream {
> > + let input = parse_macro_input!(tokens as DeriveInput);
> > + transmute::as_bytes(input).into()
> > +}
> > diff --git a/rust/macros/transmute.rs b/rust/macros/transmute.rs
> > new file mode 100644
> > index 0000000000000000000000000000000000000000..43cf36a1334f1fed23c0e777026392f987f78d8d
> > --- /dev/null
> > +++ b/rust/macros/transmute.rs
> > @@ -0,0 +1,58 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +
> > +use proc_macro2::TokenStream;
> > +use syn::{parse_quote, DeriveInput, Fields, Ident, ItemConst, Path, WhereClause};
> > +
> > +fn all_fields_impl(fields: &Fields, trait_: &Path) -> WhereClause {
> > + let tys = fields.iter().map(|field| &field.ty);
> > + parse_quote! {
> > + where #(for<'a> #tys: #trait_),*
>
> Why do we need this hrtb here?
It's a workaround to avoid needing `#![feature(trivial_bounds)]`.
Without it, generated code for, say, `where *const T: FromBytes` will
immediately cause a compilation error. With it, it will simply cause
the trait's requirements to be unfulfilled.
>
> > + }
> > +}
> > +
> > +fn struct_padding_check(fields: &Fields, name: &Ident) -> ItemConst {
> > + let tys = fields.iter().map(|field| &field.ty);
> > + parse_quote! {
> > + const _: () = {
> > + assert!(#(core::mem::size_of::<#tys>())+* == core::mem::size_of::<#name>());
> > + };
> > + }
> > +}
> > +
> > +pub(crate) fn as_bytes(input: DeriveInput) -> TokenStream {
> > + if !input.generics.params.is_empty() {
> > + return quote::quote! { compile_error!("#[derive(AsBytes)] does not support generics") };
> > + }
> > + let syn::Data::Struct(ref ds) = &input.data else {
> > + return quote::quote! { compile_error!("#[derive(AsBytes)] only supports structs") };
> > + };
> > + let name = input.ident;
> > + let trait_ = parse_quote! { ::kernel::transmute::AsBytes };
> > + let where_clause = all_fields_impl(&ds.fields, &trait_);
> > + let padding_check = struct_padding_check(&ds.fields, &name);
> > + quote::quote! {
> > + #padding_check
> > + // SAFETY: #name has no padding and all of its fields implement `AsBytes`
> > + unsafe impl #trait_ for #name #where_clause {}
> > + }
>
> In general I’d add blanks.
I'm not sure what you're suggesting here.
>
> > +}
> > +
> > +pub(crate) fn from_bytes(input: DeriveInput) -> TokenStream {
> > + let syn::Data::Struct(ref ds) = &input.data else {
> > + return quote::quote! { compile_error!("#[derive(FromBytes)] only supports structs") };
> > + };
> > + let (impl_generics, ty_generics, base_where_clause) = input.generics.split_for_impl();
> > + let name = input.ident;
> > + let trait_ = parse_quote! { ::kernel::transmute::FromBytes };
> > + let mut where_clause = all_fields_impl(&ds.fields, &trait_);
> > + if let Some(base_clause) = base_where_clause {
> > + where_clause
> > + .predicates
> > + .extend(base_clause.predicates.clone())
> > + };
> > + quote::quote! {
> > + // SAFETY: All fields of #name implement `FromBytes` and it is a struct, so there is no
> > + // implicit discriminator.
> > + unsafe impl #impl_generics #trait_ for #name #ty_generics #where_clause {}
> > + }
> > +}
> >
> > --
> > 2.52.0.305.g3fc767764a-goog
> >
> >
>
> Overall looks good. Please chime in on the two questions above.
>
> — Daniel
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes`
2025-12-17 3:12 ` Alexandre Courbot
@ 2025-12-17 18:01 ` Matthew Maurer
2025-12-17 19:14 ` Daniel Almeida
2025-12-18 7:23 ` Alexandre Courbot
0 siblings, 2 replies; 18+ messages in thread
From: Matthew Maurer @ 2025-12-17 18:01 UTC (permalink / raw)
To: Alexandre Courbot
Cc: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich, rust-for-linux, linux-kernel
On Tue, Dec 16, 2025 at 7:12 PM Alexandre Courbot <acourbot@nvidia.com> wrote:
>
> On Tue Dec 16, 2025 at 9:44 AM JST, Matthew Maurer wrote:
> > This provides a derive macro for `AsBytes` and `FromBytes` for structs
> > only. For both, it checks the respective trait on every underlying
> > field. For `AsBytes`, it emits a const-time padding check that will fail
> > the compilation if derived on a type with padding.
> >
> > Signed-off-by: Matthew Maurer <mmaurer@google.com>
>
> I like this a lot. We have a bunch of unsafe impls in Nova that this
> could help us get rid of.
>
> Amazed that this even seems to work on tuple structs!
>
> > ---
> > rust/macros/lib.rs | 63 ++++++++++++++++++++++++++++++++++++++++++++++++
> > rust/macros/transmute.rs | 58 ++++++++++++++++++++++++++++++++++++++++++++
> > 2 files changed, 121 insertions(+)
> >
> > diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
> > index b38002151871a33f6b4efea70be2deb6ddad38e2..d66397942529f67697f74a908e257cacc4201d84 100644
> > --- a/rust/macros/lib.rs
> > +++ b/rust/macros/lib.rs
> > @@ -20,9 +20,14 @@
> > mod kunit;
> > mod module;
> > mod paste;
> > +mod transmute;
> > mod vtable;
> >
> > use proc_macro::TokenStream;
> > +use syn::{
> > + parse_macro_input,
> > + DeriveInput, //
> > +};
> >
> > /// Declares a kernel module.
> > ///
> > @@ -475,3 +480,61 @@ pub fn paste(input: TokenStream) -> TokenStream {
> > pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
> > kunit::kunit_tests(attr, ts)
> > }
> > +
> > +/// Implements `FromBytes` for a struct.
> > +///
> > +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
> > +/// `FromBytes` safely. It may still fail for some types which would be safe to implement
> > +/// `FromBytes` for, in which case you will need to write the implementation and justification
> > +/// yourself.
> > +///
> > +/// Main reasons your type may be rejected:
> > +/// * Not a `struct`
> > +/// * One of the fields is not `FromBytes`
> > +///
> > +/// # Examples
> > +///
> > +/// ```
> > +/// #[derive(FromBytes)]
> > +/// #[repr(C)]
> > +/// struct Foo {
> > +/// x: u32,
> > +/// y: u16,
> > +/// z: u16,
> > +/// }
> > +/// ```
>
> One thing I have noticed is that I could sucessfully derive `FromBytes`
> on a struct that is not `repr(C)`... Is that something we want to
> disallow?
>
Why should we disallow this? I can enforce it very easily if we want
it, but the only difference between `#[repr(C)]` and `#[repr(Rust)]`
is whether we can statically predict their layout. In theory you can
use this to elide the padding check for `#[repr(C)]` structs (and
`zerocopy` does this), but it's significantly more complicated.
The only argument I see in favor of disallowing `#[repr(Rust)]` here
is that if it's not a struct that also supports `AsBytes`, there's a
question about where you're getting the bytes to load from.
I will point out that we probably don't *just* want to restrict to
`#[repr(C)]` because `#[repr(transparent)]` and `#[repr(packed)]` are
also great use cases.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 3/3] rust: Support deriving `AsBytes`/`FromBytes` on bindgen types
2025-12-17 3:15 ` Alexandre Courbot
@ 2025-12-17 18:26 ` Matthew Maurer
0 siblings, 0 replies; 18+ messages in thread
From: Matthew Maurer @ 2025-12-17 18:26 UTC (permalink / raw)
To: Alexandre Courbot
Cc: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich, rust-for-linux, linux-kernel
On Tue, Dec 16, 2025 at 7:16 PM Alexandre Courbot <acourbot@nvidia.com> wrote:
>
> On Tue Dec 16, 2025 at 9:44 AM JST, Matthew Maurer wrote:
> > To support this, we need to move the `transmute` module into a separate
> > crate to allow the `bindings` crate to depend on it. Most user code is
> > still expected to address the module as `kernel::transmute`, which is a
> > re-export. `ffi::transmute` is now available for use in `bindings`.
> >
> > Signed-off-by: Matthew Maurer <mmaurer@google.com>
>
> Maybe this should be two commits, one for the new crate, another one to
> introduce the ability to use on bindgen types.
I could do this if people think it'd be helpful, but moving it into
the `ffi` crate has no point without using on the main set of bindgen
types.
>
> I have tried this with the Nova bindings, and somehow could not get past
> this error:
>
> error[E0433]: failed to resolve: could not find `ffi` in the list of imported crates
> --> ../drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs:321:54
> |
> 321 | #[derive(Debug, Default, Copy, Clone, MaybeZeroable, FromBytesFfi)]
> | ^^^^^^^^^^^^ could not find `ffi` in the list of imported crates
> |
> = note: this error originates in the derive macro `FromBytesFfi` (in Nightly builds, run with -Z macro-backtrace for more info)
>
> I do have `kernel::ffi` imported in the bindings module though, so I am
> not quite sure what this is about. Any idea?
If you have the kernel module available, please set the bindgen flag
to apply `FromBytes` and `AsBytes` rather than `FromBytesFfi` and
`AsBytesFfi`. Those are only for the crates that are used by the
kernel crate, and so do not have access to the `kernel` crate.
The macro itself will generate `::kernel::transmute::FromBytes` when
you use `FromBytes` and `::ffi::transmute::FromBytes` when you use
`FromBytesFfi`, specifically to avoid weirdness around namespaces and
imports.
If for some reason you really wanted to use `FromBytesFfi` (you
shouldn't, but just so you understand the mechanism), you'd need to
add a direct dependency edge on the `ffi` crate, i.e. `--extern crate
ffi`
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes`
2025-12-17 17:57 ` Matthew Maurer
@ 2025-12-17 19:11 ` Daniel Almeida
0 siblings, 0 replies; 18+ messages in thread
From: Daniel Almeida @ 2025-12-17 19:11 UTC (permalink / raw)
To: Matthew Maurer
Cc: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich, rust-for-linux, linux-kernel
> On 17 Dec 2025, at 14:57, Matthew Maurer <mmaurer@google.com> wrote:
>
> On Wed, Dec 17, 2025 at 9:36 AM Daniel Almeida
> <daniel.almeida@collabora.com> wrote:
>>
>> Matthew,
>>
>>> On 15 Dec 2025, at 21:44, Matthew Maurer <mmaurer@google.com> wrote:
>>>
>>> This provides a derive macro for `AsBytes` and `FromBytes` for structs
>>> only. For both, it checks the respective trait on every underlying
>>> field. For `AsBytes`, it emits a const-time padding check that will fail
>>> the compilation if derived on a type with padding.
>>>
>>> Signed-off-by: Matthew Maurer <mmaurer@google.com>
>>> ---
>>> rust/macros/lib.rs | 63 ++++++++++++++++++++++++++++++++++++++++++++++++
>>> rust/macros/transmute.rs | 58 ++++++++++++++++++++++++++++++++++++++++++++
>>> 2 files changed, 121 insertions(+)
>>>
>>> diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
>>> index b38002151871a33f6b4efea70be2deb6ddad38e2..d66397942529f67697f74a908e257cacc4201d84 100644
>>> --- a/rust/macros/lib.rs
>>> +++ b/rust/macros/lib.rs
>>> @@ -20,9 +20,14 @@
>>> mod kunit;
>>> mod module;
>>> mod paste;
>>> +mod transmute;
>>> mod vtable;
>>>
>>> use proc_macro::TokenStream;
>>> +use syn::{
>>> + parse_macro_input,
>>> + DeriveInput, //
>>> +};
>>>
>>> /// Declares a kernel module.
>>> ///
>>> @@ -475,3 +480,61 @@ pub fn paste(input: TokenStream) -> TokenStream {
>>> pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
>>> kunit::kunit_tests(attr, ts)
>>> }
>>> +
>>> +/// Implements `FromBytes` for a struct.
>>> +///
>>> +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
>>> +/// `FromBytes` safely. It may still fail for some types which would be safe to implement
>>> +/// `FromBytes` for, in which case you will need to write the implementation and justification
>>> +/// yourself.
>>> +///
>>> +/// Main reasons your type may be rejected:
>>> +/// * Not a `struct`
>>> +/// * One of the fields is not `FromBytes`
>>> +///
>>> +/// # Examples
>>> +///
>>> +/// ```
>>> +/// #[derive(FromBytes)]
>>> +/// #[repr(C)]
>>> +/// struct Foo {
>>> +/// x: u32,
>>> +/// y: u16,
>>> +/// z: u16,
>>> +/// }
>>> +/// ```
>>> +#[proc_macro_derive(FromBytes)]
>>> +pub fn derive_from_bytes(tokens: TokenStream) -> TokenStream {
>>> + let input = parse_macro_input!(tokens as DeriveInput);
>>> + transmute::from_bytes(input).into()
>>> +}
>>> +
>>> +/// Implements `AsBytes` for a struct.
>>> +///
>>> +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
>>> +/// `AsBytes` safely. It may still fail for some structures which would be safe to implement
>>> +/// `AsBytes`, in which case you will need to write the implementation and justification
>>> +/// yourself.
>>> +///
>>> +/// Main reasons your type may be rejected:
>>> +/// * Not a `struct`
>>> +/// * One of the fields is not `AsBytes`
>>> +/// * Your struct has generic parameters
>>> +/// * There is padding somewhere in your struct
>>
>> Why is padding relevant here but not in FromBytes?
>
> Padding bytes can be initialized to any value, but it is UB to observe
> their contents because they may be implicitly uninitialized[1]. This
> also matches the approach of `zerocopy`[2], which is the standard for
> these sorts of transmutations outside the kernel - any byte can go
> into the padding, but you cannot *read* any byte out of the padding.
>
> [1]: https://doc.rust-lang.org/reference/behavior-considered-undefined.html#r-undefined.validity.undef
> [2]: https://docs.rs/zerocopy/latest/zerocopy/trait.FromBytes.html#warning-padding-bytes
Ack
>
>>
>>> +///
>>> +/// # Examples
>>> +///
>>> +/// ```
>>> +/// #[derive(AsBytes)]
>>> +/// #[repr(C)]
>>> +/// struct Foo {
>>> +/// x: u32,
>>> +/// y: u16,
>>> +/// z: u16,
>>> +/// }
>>> +/// ```
>>> +#[proc_macro_derive(AsBytes)]
>>> +pub fn derive_as_bytes(tokens: TokenStream) -> TokenStream {
>>> + let input = parse_macro_input!(tokens as DeriveInput);
>>> + transmute::as_bytes(input).into()
>>> +}
>>> diff --git a/rust/macros/transmute.rs b/rust/macros/transmute.rs
>>> new file mode 100644
>>> index 0000000000000000000000000000000000000000..43cf36a1334f1fed23c0e777026392f987f78d8d
>>> --- /dev/null
>>> +++ b/rust/macros/transmute.rs
>>> @@ -0,0 +1,58 @@
>>> +// SPDX-License-Identifier: GPL-2.0
>>> +
>>> +use proc_macro2::TokenStream;
>>> +use syn::{parse_quote, DeriveInput, Fields, Ident, ItemConst, Path, WhereClause};
>>> +
>>> +fn all_fields_impl(fields: &Fields, trait_: &Path) -> WhereClause {
>>> + let tys = fields.iter().map(|field| &field.ty);
>>> + parse_quote! {
>>> + where #(for<'a> #tys: #trait_),*
>>
>> Why do we need this hrtb here?
>
> It's a workaround to avoid needing `#![feature(trivial_bounds)]`.
> Without it, generated code for, say, `where *const T: FromBytes` will
> immediately cause a compilation error. With it, it will simply cause
> the trait's requirements to be unfulfilled.
Perhaps add this as a comment. It’s not clear at all that this is a workaround at first.
>
>>
>>> + }
>>> +}
>>> +
>>> +fn struct_padding_check(fields: &Fields, name: &Ident) -> ItemConst {
>>> + let tys = fields.iter().map(|field| &field.ty);
>>> + parse_quote! {
>>> + const _: () = {
>>> + assert!(#(core::mem::size_of::<#tys>())+* == core::mem::size_of::<#name>());
>>> + };
>>> + }
>>> +}
>>> +
>>> +pub(crate) fn as_bytes(input: DeriveInput) -> TokenStream {
>>> + if !input.generics.params.is_empty() {
>>> + return quote::quote! { compile_error!("#[derive(AsBytes)] does not support generics") };
>>> + }
>>> + let syn::Data::Struct(ref ds) = &input.data else {
>>> + return quote::quote! { compile_error!("#[derive(AsBytes)] only supports structs") };
>>> + };
>>> + let name = input.ident;
>>> + let trait_ = parse_quote! { ::kernel::transmute::AsBytes };
>>> + let where_clause = all_fields_impl(&ds.fields, &trait_);
>>> + let padding_check = struct_padding_check(&ds.fields, &name);
>>> + quote::quote! {
>>> + #padding_check
>>> + // SAFETY: #name has no padding and all of its fields implement `AsBytes`
>>> + unsafe impl #trait_ for #name #where_clause {}
>>> + }
>>
>> In general I’d add blanks.
>
> I'm not sure what you're suggesting here.
Add blank lines to break this thing up for readability. This entire function
has 0 blank lines, it’s a bit hard to read. Anyways, this is just a nit.
>
>>
>>> +}
>>> +
>>> +pub(crate) fn from_bytes(input: DeriveInput) -> TokenStream {
>>> + let syn::Data::Struct(ref ds) = &input.data else {
>>> + return quote::quote! { compile_error!("#[derive(FromBytes)] only supports structs") };
>>> + };
>>> + let (impl_generics, ty_generics, base_where_clause) = input.generics.split_for_impl();
>>> + let name = input.ident;
>>> + let trait_ = parse_quote! { ::kernel::transmute::FromBytes };
>>> + let mut where_clause = all_fields_impl(&ds.fields, &trait_);
>>> + if let Some(base_clause) = base_where_clause {
>>> + where_clause
>>> + .predicates
>>> + .extend(base_clause.predicates.clone())
>>> + };
>>> + quote::quote! {
>>> + // SAFETY: All fields of #name implement `FromBytes` and it is a struct, so there is no
>>> + // implicit discriminator.
>>> + unsafe impl #impl_generics #trait_ for #name #ty_generics #where_clause {}
>>> + }
>>> +}
>>>
>>> --
>>> 2.52.0.305.g3fc767764a-goog
>>>
>>>
>>
>> Overall looks good. Please chime in on the two questions above.
>>
>> — Daniel
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes`
2025-12-17 18:01 ` Matthew Maurer
@ 2025-12-17 19:14 ` Daniel Almeida
2025-12-18 7:23 ` Alexandre Courbot
1 sibling, 0 replies; 18+ messages in thread
From: Daniel Almeida @ 2025-12-17 19:14 UTC (permalink / raw)
To: Matthew Maurer
Cc: Alexandre Courbot, Miguel Ojeda, Boqun Feng, Gary Guo,
Björn Roy Baron, Benno Lossin, Andreas Hindborg, Alice Ryhl,
Trevor Gross, Danilo Krummrich, rust-for-linux, linux-kernel
> On 17 Dec 2025, at 15:01, Matthew Maurer <mmaurer@google.com> wrote:
>
> On Tue, Dec 16, 2025 at 7:12 PM Alexandre Courbot <acourbot@nvidia.com> wrote:
>>
>> On Tue Dec 16, 2025 at 9:44 AM JST, Matthew Maurer wrote:
>>> This provides a derive macro for `AsBytes` and `FromBytes` for structs
>>> only. For both, it checks the respective trait on every underlying
>>> field. For `AsBytes`, it emits a const-time padding check that will fail
>>> the compilation if derived on a type with padding.
>>>
>>> Signed-off-by: Matthew Maurer <mmaurer@google.com>
>>
>> I like this a lot. We have a bunch of unsafe impls in Nova that this
>> could help us get rid of.
>>
>> Amazed that this even seems to work on tuple structs!
>>
>>> ---
>>> rust/macros/lib.rs | 63 ++++++++++++++++++++++++++++++++++++++++++++++++
>>> rust/macros/transmute.rs | 58 ++++++++++++++++++++++++++++++++++++++++++++
>>> 2 files changed, 121 insertions(+)
>>>
>>> diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
>>> index b38002151871a33f6b4efea70be2deb6ddad38e2..d66397942529f67697f74a908e257cacc4201d84 100644
>>> --- a/rust/macros/lib.rs
>>> +++ b/rust/macros/lib.rs
>>> @@ -20,9 +20,14 @@
>>> mod kunit;
>>> mod module;
>>> mod paste;
>>> +mod transmute;
>>> mod vtable;
>>>
>>> use proc_macro::TokenStream;
>>> +use syn::{
>>> + parse_macro_input,
>>> + DeriveInput, //
>>> +};
>>>
>>> /// Declares a kernel module.
>>> ///
>>> @@ -475,3 +480,61 @@ pub fn paste(input: TokenStream) -> TokenStream {
>>> pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
>>> kunit::kunit_tests(attr, ts)
>>> }
>>> +
>>> +/// Implements `FromBytes` for a struct.
>>> +///
>>> +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
>>> +/// `FromBytes` safely. It may still fail for some types which would be safe to implement
>>> +/// `FromBytes` for, in which case you will need to write the implementation and justification
>>> +/// yourself.
>>> +///
>>> +/// Main reasons your type may be rejected:
>>> +/// * Not a `struct`
>>> +/// * One of the fields is not `FromBytes`
>>> +///
>>> +/// # Examples
>>> +///
>>> +/// ```
>>> +/// #[derive(FromBytes)]
>>> +/// #[repr(C)]
>>> +/// struct Foo {
>>> +/// x: u32,
>>> +/// y: u16,
>>> +/// z: u16,
>>> +/// }
>>> +/// ```
>>
>> One thing I have noticed is that I could sucessfully derive `FromBytes`
>> on a struct that is not `repr(C)`... Is that something we want to
>> disallow?
>>
>
> Why should we disallow this? I can enforce it very easily if we want
> it, but the only difference between `#[repr(C)]` and `#[repr(Rust)]`
> is whether we can statically predict their layout. In theory you can
> use this to elide the padding check for `#[repr(C)]` structs (and
> `zerocopy` does this), but it's significantly more complicated.
>
> The only argument I see in favor of disallowing `#[repr(Rust)]` here
> is that if it's not a struct that also supports `AsBytes`, there's a
> question about where you're getting the bytes to load from.
>
> I will point out that we probably don't *just* want to restrict to
> `#[repr(C)]` because `#[repr(transparent)]` and `#[repr(packed)]` are
> also great use cases.
>
I don’t see the point of disallowing other reprs. You can currently derive
FromBytes/AsBytes for any repr anyways. It’s up to the caller to make sure
that deriving these traits make sense.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 3/3] rust: Support deriving `AsBytes`/`FromBytes` on bindgen types
2025-12-16 0:44 ` [PATCH v2 3/3] rust: Support deriving `AsBytes`/`FromBytes` on bindgen types Matthew Maurer
2025-12-17 3:15 ` Alexandre Courbot
@ 2025-12-17 19:26 ` Daniel Almeida
2025-12-17 19:33 ` Matthew Maurer
1 sibling, 1 reply; 18+ messages in thread
From: Daniel Almeida @ 2025-12-17 19:26 UTC (permalink / raw)
To: Matthew Maurer
Cc: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich, rust-for-linux, linux-kernel
Matthew,
> On 15 Dec 2025, at 21:44, Matthew Maurer <mmaurer@google.com> wrote:
>
> To support this, we need to move the `transmute` module into a separate
> crate to allow the `bindings` crate to depend on it. Most user code is
> still expected to address the module as `kernel::transmute`, which is a
> re-export. `ffi::transmute` is now available for use in `bindings`.
Thanks, that’s really useful for Tyr.
>
> Signed-off-by: Matthew Maurer <mmaurer@google.com>
> ---
> rust/Makefile | 14 +++++++++-----
> rust/bindgen_parameters | 8 ++++++++
> rust/bindings/lib.rs | 4 ++++
> rust/{ffi.rs => ffi/lib.rs} | 5 +++++
> rust/{kernel => ffi}/transmute.rs | 0
> rust/kernel/lib.rs | 2 +-
> rust/macros/lib.rs | 24 ++++++++++++++++++++++--
> rust/macros/transmute.rs | 12 +++++++-----
> rust/uapi/lib.rs | 4 ++++
> scripts/generate_rust_analyzer.py | 2 +-
> 10 files changed, 61 insertions(+), 14 deletions(-)
>
> diff --git a/rust/Makefile b/rust/Makefile
> index 5d357dce1704d15e43effc528be8f5a4d74d3d8d..178aa3036c4acba1c4c266196912da2d60fe0d5f 100644
> --- a/rust/Makefile
> +++ b/rust/Makefile
> @@ -207,7 +207,7 @@ rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE
> +$(call if_changed,rustdoc)
>
> rustdoc-ffi: private is-kernel-object := y
> -rustdoc-ffi: $(src)/ffi.rs rustdoc-core FORCE
> +rustdoc-ffi: $(src)/ffi/lib.rs rustdoc-core FORCE
> +$(call if_changed,rustdoc)
>
> rustdoc-pin_init_internal: private rustdoc_host = yes
> @@ -249,7 +249,7 @@ quiet_cmd_rustc_test_library = $(RUSTC_OR_CLIPPY_QUIET) TL $<
> rusttestlib-build_error: $(src)/build_error.rs FORCE
> +$(call if_changed,rustc_test_library)
>
> -rusttestlib-ffi: $(src)/ffi.rs FORCE
> +rusttestlib-ffi: $(src)/ffi/lib.rs FORCE
> +$(call if_changed,rustc_test_library)
>
> rusttestlib-proc_macro2: private rustc_target_flags = $(proc_macro2-flags)
> @@ -657,22 +657,26 @@ $(obj)/build_error.o: $(src)/build_error.rs $(obj)/compiler_builtins.o FORCE
> +$(call if_changed_rule,rustc_library)
>
> $(obj)/ffi.o: private skip_gendwarfksyms = 1
> -$(obj)/ffi.o: $(src)/ffi.rs $(obj)/compiler_builtins.o FORCE
> +$(obj)/ffi.o: $(src)/ffi/lib.rs $(obj)/compiler_builtins.o FORCE
> +$(call if_changed_rule,rustc_library)
>
> -$(obj)/bindings.o: private rustc_target_flags = --extern ffi --extern pin_init
> +$(obj)/bindings.o: private rustc_target_flags = --extern ffi --extern pin_init \
> + --extern macros
> $(obj)/bindings.o: $(src)/bindings/lib.rs \
> $(obj)/ffi.o \
> $(obj)/pin_init.o \
> + $(obj)/$(libmacros_name) \
> $(obj)/bindings/bindings_generated.rs \
> $(obj)/bindings/bindings_helpers_generated.rs FORCE
> +$(call if_changed_rule,rustc_library)
>
> -$(obj)/uapi.o: private rustc_target_flags = --extern ffi --extern pin_init
> +$(obj)/uapi.o: private rustc_target_flags = --extern ffi --extern pin_init \
> + --extern macros
> $(obj)/uapi.o: private skip_gendwarfksyms = 1
> $(obj)/uapi.o: $(src)/uapi/lib.rs \
> $(obj)/ffi.o \
> $(obj)/pin_init.o \
> + $(obj)/$(libmacros_name) \
> $(obj)/uapi/uapi_generated.rs FORCE
> +$(call if_changed_rule,rustc_library)
+1 on splitting the above into a separate commit
>
> diff --git a/rust/bindgen_parameters b/rust/bindgen_parameters
> index fd2fd1c3cb9a51ea46fcd721907783b457aa1378..d56343ca03979e345f8adb7eb8fd7f2b9d4be6ee 100644
> --- a/rust/bindgen_parameters
> +++ b/rust/bindgen_parameters
> @@ -64,3 +64,11 @@
> # Structs should implement `Zeroable` when all of their fields do.
> --with-derive-custom-struct .*=MaybeZeroable
> --with-derive-custom-union .*=MaybeZeroable
> +
> +# Every C struct can try to derive FromBytes. If they have unmet requirements,
> +# the impl will just be a no-op due to the where clause.
> +--with-derive-custom-struct .*=FromBytesFfi
> +
> +# We can't auto-derive AsBytes, as we need a const-time check to see if there
> +# is padding involved. Add it explicitly when you expect no padding.
> +--with-derive-custom-struct cpumask=AsBytesFfi
> diff --git a/rust/bindings/lib.rs b/rust/bindings/lib.rs
> index 0c57cf9b4004f176997c59ecc58a9a9ac76163d9..c29312fca1b01b707bb11b05d446d44480a4b81f 100644
> --- a/rust/bindings/lib.rs
> +++ b/rust/bindings/lib.rs
> @@ -31,6 +31,10 @@
> #[allow(clippy::undocumented_unsafe_blocks)]
> #[cfg_attr(CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES, allow(unnecessary_transmutes))]
> mod bindings_raw {
> + use macros::{
> + AsBytesFfi,
> + FromBytesFfi, //
> + };
> use pin_init::{MaybeZeroable, Zeroable};
>
> // Manual definition for blocklisted types.
> diff --git a/rust/ffi.rs b/rust/ffi/lib.rs
> similarity index 87%
> rename from rust/ffi.rs
> rename to rust/ffi/lib.rs
> index f961e9728f590fd2c52d4c03a1f715d654051d04..14052362f091a609bc505fe6eca77fe998fe2321 100644
> --- a/rust/ffi.rs
> +++ b/rust/ffi/lib.rs
> @@ -10,6 +10,11 @@
>
> #![no_std]
>
> +#[doc(hidden)]
> +// This lives here to make it accessible to `bindings`, similar to the other `ffi` types.
> +// User code should access it through `kernel::transmute`.
> +pub mod transmute;
> +
> macro_rules! alias {
> ($($name:ident = $ty:ty;)*) => {$(
> #[allow(non_camel_case_types, missing_docs)]
> diff --git a/rust/kernel/transmute.rs b/rust/ffi/transmute.rs
> similarity index 100%
> rename from rust/kernel/transmute.rs
> rename to rust/ffi/transmute.rs
> diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
> index f812cf12004286962985a068665443dc22c389a2..4aa54dd83319ef16bd4baa1964114f1e6549942b 100644
> --- a/rust/kernel/lib.rs
> +++ b/rust/kernel/lib.rs
> @@ -63,6 +63,7 @@
> extern crate self as kernel;
>
> pub use ffi;
> +pub use ffi::transmute;
>
> pub mod acpi;
> pub mod alloc;
> @@ -146,7 +147,6 @@
> pub mod task;
> pub mod time;
> pub mod tracepoint;
> -pub mod transmute;
> pub mod types;
> pub mod uaccess;
> #[cfg(CONFIG_USB = "y")]
> diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
> index d66397942529f67697f74a908e257cacc4201d84..bde94c2a8ddf708872bcd56a4713784b5ccdf04e 100644
> --- a/rust/macros/lib.rs
> +++ b/rust/macros/lib.rs
> @@ -506,7 +506,7 @@ pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
> #[proc_macro_derive(FromBytes)]
> pub fn derive_from_bytes(tokens: TokenStream) -> TokenStream {
> let input = parse_macro_input!(tokens as DeriveInput);
> - transmute::from_bytes(input).into()
> + transmute::from_bytes("kernel", input).into()
> }
>
> /// Implements `AsBytes` for a struct.
> @@ -536,5 +536,25 @@ pub fn derive_from_bytes(tokens: TokenStream) -> TokenStream {
> #[proc_macro_derive(AsBytes)]
> pub fn derive_as_bytes(tokens: TokenStream) -> TokenStream {
> let input = parse_macro_input!(tokens as DeriveInput);
> - transmute::as_bytes(input).into()
> + transmute::as_bytes("kernel", input).into()
> +}
> +
> +#[doc(hidden)]
> +#[proc_macro_derive(FromBytesFfi)]
Let’s perhaps capitalize this as FromBytesFFI?
> +/// This is equivalent to `FromBytes`, but uses the `ffi` crate as the trait definition site
> +/// instead of `kernel`. This is intended for use inside `bindings`. Everyone else can refer to the
> +/// trait through `kernel`.
> +pub fn derive_from_bytes_trait(tokens: TokenStream) -> TokenStream {
> + let input = parse_macro_input!(tokens as DeriveInput);
> + transmute::from_bytes("ffi", input).into()
> +}
> +
> +#[doc(hidden)]
> +#[proc_macro_derive(AsBytesFfi)]
Same here?
> +/// This is equivalent to `AsBytes`, but uses the `ffi` crate as the trait definition site
> +/// instead of `kernel`. This is intended for use inside `bindings`. Everyone else can refer to the
> +/// trait through `kernel`.
> +pub fn derive_as_bytes_trait(tokens: TokenStream) -> TokenStream {
> + let input = parse_macro_input!(tokens as DeriveInput);
> + transmute::as_bytes("ffi", input).into()
> }
> diff --git a/rust/macros/transmute.rs b/rust/macros/transmute.rs
> index 43cf36a1334f1fed23c0e777026392f987f78d8d..9bd6d279675fb1d58cdceff1f4dde0dcf33fbf99 100644
> --- a/rust/macros/transmute.rs
> +++ b/rust/macros/transmute.rs
> @@ -1,6 +1,6 @@
> // SPDX-License-Identifier: GPL-2.0
>
> -use proc_macro2::TokenStream;
> +use proc_macro2::{Span, TokenStream};
> use syn::{parse_quote, DeriveInput, Fields, Ident, ItemConst, Path, WhereClause};
>
> fn all_fields_impl(fields: &Fields, trait_: &Path) -> WhereClause {
> @@ -19,7 +19,8 @@ fn struct_padding_check(fields: &Fields, name: &Ident) -> ItemConst {
> }
> }
>
> -pub(crate) fn as_bytes(input: DeriveInput) -> TokenStream {
> +pub(crate) fn as_bytes(crate_: &str, input: DeriveInput) -> TokenStream {
> + let crate_ = Ident::new(crate_, Span::call_site());
> if !input.generics.params.is_empty() {
> return quote::quote! { compile_error!("#[derive(AsBytes)] does not support generics") };
> }
> @@ -27,7 +28,7 @@ pub(crate) fn as_bytes(input: DeriveInput) -> TokenStream {
> return quote::quote! { compile_error!("#[derive(AsBytes)] only supports structs") };
> };
> let name = input.ident;
> - let trait_ = parse_quote! { ::kernel::transmute::AsBytes };
> + let trait_ = parse_quote! { ::#crate_::transmute::AsBytes };
> let where_clause = all_fields_impl(&ds.fields, &trait_);
> let padding_check = struct_padding_check(&ds.fields, &name);
> quote::quote! {
> @@ -37,13 +38,14 @@ unsafe impl #trait_ for #name #where_clause {}
> }
> }
>
> -pub(crate) fn from_bytes(input: DeriveInput) -> TokenStream {
> +pub(crate) fn from_bytes(crate_: &str, input: DeriveInput) -> TokenStream {
> + let crate_ = Ident::new(crate_, Span::call_site());
> let syn::Data::Struct(ref ds) = &input.data else {
> return quote::quote! { compile_error!("#[derive(FromBytes)] only supports structs") };
> };
> let (impl_generics, ty_generics, base_where_clause) = input.generics.split_for_impl();
> let name = input.ident;
> - let trait_ = parse_quote! { ::kernel::transmute::FromBytes };
> + let trait_ = parse_quote! { ::#crate_::transmute::FromBytes };
> let mut where_clause = all_fields_impl(&ds.fields, &trait_);
> if let Some(base_clause) = base_where_clause {
> where_clause
> diff --git a/rust/uapi/lib.rs b/rust/uapi/lib.rs
> index 1d5fd9efb93e9db97fec84fca2bae37b500c20c5..2033b7125558d7ccfae67b94777f5ce59593a528 100644
> --- a/rust/uapi/lib.rs
> +++ b/rust/uapi/lib.rs
> @@ -34,6 +34,10 @@
> type __kernel_ssize_t = isize;
> type __kernel_ptrdiff_t = isize;
>
> +use macros::{
> + AsBytesFfi,
> + FromBytesFfi, //
Why is this “//" here? Is this related to the import style change that took place recently?
> +};
> use pin_init::MaybeZeroable;
>
> include!(concat!(env!("OBJTREE"), "/rust/uapi/uapi_generated.rs"));
> diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
> index 147d0cc940681426771db865bc2462e7029a6d7d..843d081eacaca8edeeac5978bd8107a498008186 100755
> --- a/scripts/generate_rust_analyzer.py
> +++ b/scripts/generate_rust_analyzer.py
> @@ -137,7 +137,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs, core_edit
>
> append_crate(
> "ffi",
> - srctree / "rust" / "ffi.rs",
> + srctree / "rust" / "ffi" / "lib.rs",
> ["core", "compiler_builtins"],
> )
>
>
> --
> 2.52.0.305.g3fc767764a-goog
>
>
^ IMHO, this should also be in separate commit.
Looks good to me overall.
— Daniel
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 3/3] rust: Support deriving `AsBytes`/`FromBytes` on bindgen types
2025-12-17 19:26 ` Daniel Almeida
@ 2025-12-17 19:33 ` Matthew Maurer
0 siblings, 0 replies; 18+ messages in thread
From: Matthew Maurer @ 2025-12-17 19:33 UTC (permalink / raw)
To: Daniel Almeida
Cc: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich, rust-for-linux, linux-kernel
On Wed, Dec 17, 2025 at 11:26 AM Daniel Almeida
<daniel.almeida@collabora.com> wrote:
>
> Matthew,
>
> > On 15 Dec 2025, at 21:44, Matthew Maurer <mmaurer@google.com> wrote:
> >
> > To support this, we need to move the `transmute` module into a separate
> > crate to allow the `bindings` crate to depend on it. Most user code is
> > still expected to address the module as `kernel::transmute`, which is a
> > re-export. `ffi::transmute` is now available for use in `bindings`.
>
> Thanks, that’s really useful for Tyr.
>
> >
> > Signed-off-by: Matthew Maurer <mmaurer@google.com>
> > ---
> > rust/Makefile | 14 +++++++++-----
> > rust/bindgen_parameters | 8 ++++++++
> > rust/bindings/lib.rs | 4 ++++
> > rust/{ffi.rs => ffi/lib.rs} | 5 +++++
> > rust/{kernel => ffi}/transmute.rs | 0
> > rust/kernel/lib.rs | 2 +-
> > rust/macros/lib.rs | 24 ++++++++++++++++++++++--
> > rust/macros/transmute.rs | 12 +++++++-----
> > rust/uapi/lib.rs | 4 ++++
> > scripts/generate_rust_analyzer.py | 2 +-
> > 10 files changed, 61 insertions(+), 14 deletions(-)
> >
> > diff --git a/rust/Makefile b/rust/Makefile
> > index 5d357dce1704d15e43effc528be8f5a4d74d3d8d..178aa3036c4acba1c4c266196912da2d60fe0d5f 100644
> > --- a/rust/Makefile
> > +++ b/rust/Makefile
> > @@ -207,7 +207,7 @@ rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE
> > +$(call if_changed,rustdoc)
> >
> > rustdoc-ffi: private is-kernel-object := y
> > -rustdoc-ffi: $(src)/ffi.rs rustdoc-core FORCE
> > +rustdoc-ffi: $(src)/ffi/lib.rs rustdoc-core FORCE
> > +$(call if_changed,rustdoc)
> >
> > rustdoc-pin_init_internal: private rustdoc_host = yes
> > @@ -249,7 +249,7 @@ quiet_cmd_rustc_test_library = $(RUSTC_OR_CLIPPY_QUIET) TL $<
> > rusttestlib-build_error: $(src)/build_error.rs FORCE
> > +$(call if_changed,rustc_test_library)
> >
> > -rusttestlib-ffi: $(src)/ffi.rs FORCE
> > +rusttestlib-ffi: $(src)/ffi/lib.rs FORCE
> > +$(call if_changed,rustc_test_library)
> >
> > rusttestlib-proc_macro2: private rustc_target_flags = $(proc_macro2-flags)
> > @@ -657,22 +657,26 @@ $(obj)/build_error.o: $(src)/build_error.rs $(obj)/compiler_builtins.o FORCE
> > +$(call if_changed_rule,rustc_library)
> >
> > $(obj)/ffi.o: private skip_gendwarfksyms = 1
> > -$(obj)/ffi.o: $(src)/ffi.rs $(obj)/compiler_builtins.o FORCE
> > +$(obj)/ffi.o: $(src)/ffi/lib.rs $(obj)/compiler_builtins.o FORCE
> > +$(call if_changed_rule,rustc_library)
> >
> > -$(obj)/bindings.o: private rustc_target_flags = --extern ffi --extern pin_init
> > +$(obj)/bindings.o: private rustc_target_flags = --extern ffi --extern pin_init \
> > + --extern macros
> > $(obj)/bindings.o: $(src)/bindings/lib.rs \
> > $(obj)/ffi.o \
> > $(obj)/pin_init.o \
> > + $(obj)/$(libmacros_name) \
> > $(obj)/bindings/bindings_generated.rs \
> > $(obj)/bindings/bindings_helpers_generated.rs FORCE
> > +$(call if_changed_rule,rustc_library)
> >
> > -$(obj)/uapi.o: private rustc_target_flags = --extern ffi --extern pin_init
> > +$(obj)/uapi.o: private rustc_target_flags = --extern ffi --extern pin_init \
> > + --extern macros
> > $(obj)/uapi.o: private skip_gendwarfksyms = 1
> > $(obj)/uapi.o: $(src)/uapi/lib.rs \
> > $(obj)/ffi.o \
> > $(obj)/pin_init.o \
> > + $(obj)/$(libmacros_name) \
> > $(obj)/uapi/uapi_generated.rs FORCE
> > +$(call if_changed_rule,rustc_library)
>
> +1 on splitting the above into a separate commit
Sure, I'll do that in the next rev. Two people seems like enough of a signal :)
>
> >
> > diff --git a/rust/bindgen_parameters b/rust/bindgen_parameters
> > index fd2fd1c3cb9a51ea46fcd721907783b457aa1378..d56343ca03979e345f8adb7eb8fd7f2b9d4be6ee 100644
> > --- a/rust/bindgen_parameters
> > +++ b/rust/bindgen_parameters
> > @@ -64,3 +64,11 @@
> > # Structs should implement `Zeroable` when all of their fields do.
> > --with-derive-custom-struct .*=MaybeZeroable
> > --with-derive-custom-union .*=MaybeZeroable
> > +
> > +# Every C struct can try to derive FromBytes. If they have unmet requirements,
> > +# the impl will just be a no-op due to the where clause.
> > +--with-derive-custom-struct .*=FromBytesFfi
> > +
> > +# We can't auto-derive AsBytes, as we need a const-time check to see if there
> > +# is padding involved. Add it explicitly when you expect no padding.
> > +--with-derive-custom-struct cpumask=AsBytesFfi
> > diff --git a/rust/bindings/lib.rs b/rust/bindings/lib.rs
> > index 0c57cf9b4004f176997c59ecc58a9a9ac76163d9..c29312fca1b01b707bb11b05d446d44480a4b81f 100644
> > --- a/rust/bindings/lib.rs
> > +++ b/rust/bindings/lib.rs
> > @@ -31,6 +31,10 @@
> > #[allow(clippy::undocumented_unsafe_blocks)]
> > #[cfg_attr(CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES, allow(unnecessary_transmutes))]
> > mod bindings_raw {
> > + use macros::{
> > + AsBytesFfi,
> > + FromBytesFfi, //
> > + };
> > use pin_init::{MaybeZeroable, Zeroable};
> >
> > // Manual definition for blocklisted types.
> > diff --git a/rust/ffi.rs b/rust/ffi/lib.rs
> > similarity index 87%
> > rename from rust/ffi.rs
> > rename to rust/ffi/lib.rs
> > index f961e9728f590fd2c52d4c03a1f715d654051d04..14052362f091a609bc505fe6eca77fe998fe2321 100644
> > --- a/rust/ffi.rs
> > +++ b/rust/ffi/lib.rs
> > @@ -10,6 +10,11 @@
> >
> > #![no_std]
> >
> > +#[doc(hidden)]
> > +// This lives here to make it accessible to `bindings`, similar to the other `ffi` types.
> > +// User code should access it through `kernel::transmute`.
> > +pub mod transmute;
> > +
> > macro_rules! alias {
> > ($($name:ident = $ty:ty;)*) => {$(
> > #[allow(non_camel_case_types, missing_docs)]
> > diff --git a/rust/kernel/transmute.rs b/rust/ffi/transmute.rs
> > similarity index 100%
> > rename from rust/kernel/transmute.rs
> > rename to rust/ffi/transmute.rs
> > diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs
> > index f812cf12004286962985a068665443dc22c389a2..4aa54dd83319ef16bd4baa1964114f1e6549942b 100644
> > --- a/rust/kernel/lib.rs
> > +++ b/rust/kernel/lib.rs
> > @@ -63,6 +63,7 @@
> > extern crate self as kernel;
> >
> > pub use ffi;
> > +pub use ffi::transmute;
> >
> > pub mod acpi;
> > pub mod alloc;
> > @@ -146,7 +147,6 @@
> > pub mod task;
> > pub mod time;
> > pub mod tracepoint;
> > -pub mod transmute;
> > pub mod types;
> > pub mod uaccess;
> > #[cfg(CONFIG_USB = "y")]
> > diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
> > index d66397942529f67697f74a908e257cacc4201d84..bde94c2a8ddf708872bcd56a4713784b5ccdf04e 100644
> > --- a/rust/macros/lib.rs
> > +++ b/rust/macros/lib.rs
> > @@ -506,7 +506,7 @@ pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
> > #[proc_macro_derive(FromBytes)]
> > pub fn derive_from_bytes(tokens: TokenStream) -> TokenStream {
> > let input = parse_macro_input!(tokens as DeriveInput);
> > - transmute::from_bytes(input).into()
> > + transmute::from_bytes("kernel", input).into()
> > }
> >
> > /// Implements `AsBytes` for a struct.
> > @@ -536,5 +536,25 @@ pub fn derive_from_bytes(tokens: TokenStream) -> TokenStream {
> > #[proc_macro_derive(AsBytes)]
> > pub fn derive_as_bytes(tokens: TokenStream) -> TokenStream {
> > let input = parse_macro_input!(tokens as DeriveInput);
> > - transmute::as_bytes(input).into()
> > + transmute::as_bytes("kernel", input).into()
> > +}
> > +
> > +#[doc(hidden)]
> > +#[proc_macro_derive(FromBytesFfi)]
>
> Let’s perhaps capitalize this as FromBytesFFI?
Our Rust coding guidelines[1] do not specify a kernel-specific
practice here, and say to defer to standard ecosystem guidance[2].
Derive macros take the position of traits, which should be in
UpperCamelCase. There is an explicit section which reads: "In
UpperCamelCase, acronyms and contractions of compound words count as
one word: use Uuid rather than UUID". While we are free to define our
own naming standards in the kernel, I do not see anything in the
current Rust coding guidelines[2] to overrule standard guidance here.
[1]: https://docs.kernel.org/rust/coding-guidelines.html
[2]: https://rust-lang.github.io/api-guidelines/naming.html
>
> > +/// This is equivalent to `FromBytes`, but uses the `ffi` crate as the trait definition site
> > +/// instead of `kernel`. This is intended for use inside `bindings`. Everyone else can refer to the
> > +/// trait through `kernel`.
> > +pub fn derive_from_bytes_trait(tokens: TokenStream) -> TokenStream {
> > + let input = parse_macro_input!(tokens as DeriveInput);
> > + transmute::from_bytes("ffi", input).into()
> > +}
> > +
> > +#[doc(hidden)]
> > +#[proc_macro_derive(AsBytesFfi)]
>
> Same here?
>
> > +/// This is equivalent to `AsBytes`, but uses the `ffi` crate as the trait definition site
> > +/// instead of `kernel`. This is intended for use inside `bindings`. Everyone else can refer to the
> > +/// trait through `kernel`.
> > +pub fn derive_as_bytes_trait(tokens: TokenStream) -> TokenStream {
> > + let input = parse_macro_input!(tokens as DeriveInput);
> > + transmute::as_bytes("ffi", input).into()
> > }
> > diff --git a/rust/macros/transmute.rs b/rust/macros/transmute.rs
> > index 43cf36a1334f1fed23c0e777026392f987f78d8d..9bd6d279675fb1d58cdceff1f4dde0dcf33fbf99 100644
> > --- a/rust/macros/transmute.rs
> > +++ b/rust/macros/transmute.rs
> > @@ -1,6 +1,6 @@
> > // SPDX-License-Identifier: GPL-2.0
> >
> > -use proc_macro2::TokenStream;
> > +use proc_macro2::{Span, TokenStream};
> > use syn::{parse_quote, DeriveInput, Fields, Ident, ItemConst, Path, WhereClause};
> >
> > fn all_fields_impl(fields: &Fields, trait_: &Path) -> WhereClause {
> > @@ -19,7 +19,8 @@ fn struct_padding_check(fields: &Fields, name: &Ident) -> ItemConst {
> > }
> > }
> >
> > -pub(crate) fn as_bytes(input: DeriveInput) -> TokenStream {
> > +pub(crate) fn as_bytes(crate_: &str, input: DeriveInput) -> TokenStream {
> > + let crate_ = Ident::new(crate_, Span::call_site());
> > if !input.generics.params.is_empty() {
> > return quote::quote! { compile_error!("#[derive(AsBytes)] does not support generics") };
> > }
> > @@ -27,7 +28,7 @@ pub(crate) fn as_bytes(input: DeriveInput) -> TokenStream {
> > return quote::quote! { compile_error!("#[derive(AsBytes)] only supports structs") };
> > };
> > let name = input.ident;
> > - let trait_ = parse_quote! { ::kernel::transmute::AsBytes };
> > + let trait_ = parse_quote! { ::#crate_::transmute::AsBytes };
> > let where_clause = all_fields_impl(&ds.fields, &trait_);
> > let padding_check = struct_padding_check(&ds.fields, &name);
> > quote::quote! {
> > @@ -37,13 +38,14 @@ unsafe impl #trait_ for #name #where_clause {}
> > }
> > }
> >
> > -pub(crate) fn from_bytes(input: DeriveInput) -> TokenStream {
> > +pub(crate) fn from_bytes(crate_: &str, input: DeriveInput) -> TokenStream {
> > + let crate_ = Ident::new(crate_, Span::call_site());
> > let syn::Data::Struct(ref ds) = &input.data else {
> > return quote::quote! { compile_error!("#[derive(FromBytes)] only supports structs") };
> > };
> > let (impl_generics, ty_generics, base_where_clause) = input.generics.split_for_impl();
> > let name = input.ident;
> > - let trait_ = parse_quote! { ::kernel::transmute::FromBytes };
> > + let trait_ = parse_quote! { ::#crate_::transmute::FromBytes };
> > let mut where_clause = all_fields_impl(&ds.fields, &trait_);
> > if let Some(base_clause) = base_where_clause {
> > where_clause
> > diff --git a/rust/uapi/lib.rs b/rust/uapi/lib.rs
> > index 1d5fd9efb93e9db97fec84fca2bae37b500c20c5..2033b7125558d7ccfae67b94777f5ce59593a528 100644
> > --- a/rust/uapi/lib.rs
> > +++ b/rust/uapi/lib.rs
> > @@ -34,6 +34,10 @@
> > type __kernel_ssize_t = isize;
> > type __kernel_ptrdiff_t = isize;
> >
> > +use macros::{
> > + AsBytesFfi,
> > + FromBytesFfi, //
>
> Why is this “//" here? Is this related to the import style change that took place recently?
>
> > +};
> > use pin_init::MaybeZeroable;
> >
> > include!(concat!(env!("OBJTREE"), "/rust/uapi/uapi_generated.rs"));
> > diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
> > index 147d0cc940681426771db865bc2462e7029a6d7d..843d081eacaca8edeeac5978bd8107a498008186 100755
> > --- a/scripts/generate_rust_analyzer.py
> > +++ b/scripts/generate_rust_analyzer.py
> > @@ -137,7 +137,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs, core_edit
> >
> > append_crate(
> > "ffi",
> > - srctree / "rust" / "ffi.rs",
> > + srctree / "rust" / "ffi" / "lib.rs",
> > ["core", "compiler_builtins"],
> > )
> >
> >
> > --
> > 2.52.0.305.g3fc767764a-goog
> >
> >
>
> ^ IMHO, this should also be in separate commit.
>
> Looks good to me overall.
>
> — Daniel
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes`
2025-12-17 18:01 ` Matthew Maurer
2025-12-17 19:14 ` Daniel Almeida
@ 2025-12-18 7:23 ` Alexandre Courbot
2025-12-18 8:26 ` Alice Ryhl
1 sibling, 1 reply; 18+ messages in thread
From: Alexandre Courbot @ 2025-12-18 7:23 UTC (permalink / raw)
To: Matthew Maurer, Alexandre Courbot
Cc: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich, rust-for-linux, linux-kernel
On Thu Dec 18, 2025 at 3:01 AM JST, Matthew Maurer wrote:
> On Tue, Dec 16, 2025 at 7:12 PM Alexandre Courbot <acourbot@nvidia.com> wrote:
>>
>> On Tue Dec 16, 2025 at 9:44 AM JST, Matthew Maurer wrote:
>> > This provides a derive macro for `AsBytes` and `FromBytes` for structs
>> > only. For both, it checks the respective trait on every underlying
>> > field. For `AsBytes`, it emits a const-time padding check that will fail
>> > the compilation if derived on a type with padding.
>> >
>> > Signed-off-by: Matthew Maurer <mmaurer@google.com>
>>
>> I like this a lot. We have a bunch of unsafe impls in Nova that this
>> could help us get rid of.
>>
>> Amazed that this even seems to work on tuple structs!
>>
>> > ---
>> > rust/macros/lib.rs | 63 ++++++++++++++++++++++++++++++++++++++++++++++++
>> > rust/macros/transmute.rs | 58 ++++++++++++++++++++++++++++++++++++++++++++
>> > 2 files changed, 121 insertions(+)
>> >
>> > diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs
>> > index b38002151871a33f6b4efea70be2deb6ddad38e2..d66397942529f67697f74a908e257cacc4201d84 100644
>> > --- a/rust/macros/lib.rs
>> > +++ b/rust/macros/lib.rs
>> > @@ -20,9 +20,14 @@
>> > mod kunit;
>> > mod module;
>> > mod paste;
>> > +mod transmute;
>> > mod vtable;
>> >
>> > use proc_macro::TokenStream;
>> > +use syn::{
>> > + parse_macro_input,
>> > + DeriveInput, //
>> > +};
>> >
>> > /// Declares a kernel module.
>> > ///
>> > @@ -475,3 +480,61 @@ pub fn paste(input: TokenStream) -> TokenStream {
>> > pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
>> > kunit::kunit_tests(attr, ts)
>> > }
>> > +
>> > +/// Implements `FromBytes` for a struct.
>> > +///
>> > +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
>> > +/// `FromBytes` safely. It may still fail for some types which would be safe to implement
>> > +/// `FromBytes` for, in which case you will need to write the implementation and justification
>> > +/// yourself.
>> > +///
>> > +/// Main reasons your type may be rejected:
>> > +/// * Not a `struct`
>> > +/// * One of the fields is not `FromBytes`
>> > +///
>> > +/// # Examples
>> > +///
>> > +/// ```
>> > +/// #[derive(FromBytes)]
>> > +/// #[repr(C)]
>> > +/// struct Foo {
>> > +/// x: u32,
>> > +/// y: u16,
>> > +/// z: u16,
>> > +/// }
>> > +/// ```
>>
>> One thing I have noticed is that I could sucessfully derive `FromBytes`
>> on a struct that is not `repr(C)`... Is that something we want to
>> disallow?
>>
>
> Why should we disallow this? I can enforce it very easily if we want
> it, but the only difference between `#[repr(C)]` and `#[repr(Rust)]`
> is whether we can statically predict their layout. In theory you can
> use this to elide the padding check for `#[repr(C)]` structs (and
> `zerocopy` does this), but it's significantly more complicated.
>
> The only argument I see in favor of disallowing `#[repr(Rust)]` here
> is that if it's not a struct that also supports `AsBytes`, there's a
> question about where you're getting the bytes to load from.
>
> I will point out that we probably don't *just* want to restrict to
> `#[repr(C)]` because `#[repr(transparent)]` and `#[repr(packed)]` are
> also great use cases.
Yeah it's probably correct as it is. I am not sure why we would want to
use it on types without a predictable layout, but also cannot say this
is fundamentally broken.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes`
2025-12-18 7:23 ` Alexandre Courbot
@ 2025-12-18 8:26 ` Alice Ryhl
0 siblings, 0 replies; 18+ messages in thread
From: Alice Ryhl @ 2025-12-18 8:26 UTC (permalink / raw)
To: Alexandre Courbot
Cc: Matthew Maurer, Miguel Ojeda, Boqun Feng, Gary Guo,
Björn Roy Baron, Benno Lossin, Andreas Hindborg,
Trevor Gross, Danilo Krummrich, rust-for-linux, linux-kernel
On Thu, Dec 18, 2025 at 04:23:43PM +0900, Alexandre Courbot wrote:
> On Thu Dec 18, 2025 at 3:01 AM JST, Matthew Maurer wrote:
> > On Tue, Dec 16, 2025 at 7:12 PM Alexandre Courbot <acourbot@nvidia.com> wrote:
> >> > +/// Implements `FromBytes` for a struct.
> >> > +///
> >> > +/// It will fail compilation if the struct you are deriving on cannot be determined to implement
> >> > +/// `FromBytes` safely. It may still fail for some types which would be safe to implement
> >> > +/// `FromBytes` for, in which case you will need to write the implementation and justification
> >> > +/// yourself.
> >> > +///
> >> > +/// Main reasons your type may be rejected:
> >> > +/// * Not a `struct`
> >> > +/// * One of the fields is not `FromBytes`
> >> > +///
> >> > +/// # Examples
> >> > +///
> >> > +/// ```
> >> > +/// #[derive(FromBytes)]
> >> > +/// #[repr(C)]
> >> > +/// struct Foo {
> >> > +/// x: u32,
> >> > +/// y: u16,
> >> > +/// z: u16,
> >> > +/// }
> >> > +/// ```
> >>
> >> One thing I have noticed is that I could sucessfully derive `FromBytes`
> >> on a struct that is not `repr(C)`... Is that something we want to
> >> disallow?
> >>
> >
> > Why should we disallow this? I can enforce it very easily if we want
> > it, but the only difference between `#[repr(C)]` and `#[repr(Rust)]`
> > is whether we can statically predict their layout. In theory you can
> > use this to elide the padding check for `#[repr(C)]` structs (and
> > `zerocopy` does this), but it's significantly more complicated.
> >
> > The only argument I see in favor of disallowing `#[repr(Rust)]` here
> > is that if it's not a struct that also supports `AsBytes`, there's a
> > question about where you're getting the bytes to load from.
> >
> > I will point out that we probably don't *just* want to restrict to
> > `#[repr(C)]` because `#[repr(transparent)]` and `#[repr(packed)]` are
> > also great use cases.
>
> Yeah it's probably correct as it is. I am not sure why we would want to
> use it on types without a predictable layout, but also cannot say this
> is fundamentally broken.
At the very least such types can roundtrip to/from byte arrays. Or you
could pass them to the randomness pool:
https://lore.kernel.org/all/20251216-add-entropy-v2-1-4d866f251474@google.com/
Alice
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v2 1/3] rust: transmute: Support transmuting slices of AsBytes/FromBytes types
2025-12-17 16:51 ` Daniel Almeida
@ 2025-12-26 20:27 ` Matthew Maurer
0 siblings, 0 replies; 18+ messages in thread
From: Matthew Maurer @ 2025-12-26 20:27 UTC (permalink / raw)
To: Daniel Almeida
Cc: Miguel Ojeda, Boqun Feng, Gary Guo, Björn Roy Baron,
Benno Lossin, Andreas Hindborg, Alice Ryhl, Trevor Gross,
Danilo Krummrich, rust-for-linux, linux-kernel
On Wed, Dec 17, 2025 at 8:52 AM Daniel Almeida
<daniel.almeida@collabora.com> wrote:
>
> Hi Matthew,
>
> > On 15 Dec 2025, at 21:44, Matthew Maurer <mmaurer@google.com> wrote:
> >
> > Currently, we support either transmuting a byte slice of the exact size
> > of the target type or a single element from a prefix of a byte slice.
> >
> > This adds support for transmuting from a byte slice to a slice of
> > elements, assuming appropriate traits are implemented.
> >
> > Signed-off-by: Matthew Maurer <mmaurer@google.com>
> > ---
> > rust/kernel/transmute.rs | 72 ++++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 72 insertions(+)
> >
> > diff --git a/rust/kernel/transmute.rs b/rust/kernel/transmute.rs
> > index be5dbf3829e25c92f85083cc0f4940f35bb7baec..5cc5f4fde4c31cea3c51ab078b4d68f3a0a2caa1 100644
> > --- a/rust/kernel/transmute.rs
> > +++ b/rust/kernel/transmute.rs
> > @@ -122,6 +122,78 @@ fn from_bytes_mut_prefix(bytes: &mut [u8]) -> Option<(&mut Self, &mut [u8])>
> > }
> > }
> >
> > + /// Converts a slice of bytes to a slice of `Self`.
> > + ///
> > + /// Succeeds if the reference is properly aligned, and the size of `bytes` is a multiple of that
> > + /// of `Self`.
> > + ///
> > + /// Otherwise, returns [`None`].
> > + fn from_bytes_to_slice(bytes: &[u8]) -> Option<&[Self]>
> > + where
> > + Self: Sized,
> > + {
> > + let size = size_of::<Self>();
> > + if size == 0 {
> > + return None;
> > + }
> > + if bytes.len() % size != 0 {
> > + return None;
> > + }
> > + let len = bytes.len() / size;
> > + let ptr = bytes.as_ptr().cast::<Self>();
> > +
> > + #[allow(clippy::incompatible_msrv)]
> > + if ptr.is_aligned() {
> > + // SAFETY:
> > + // - `ptr` is valid for reads of `bytes.len()` bytes, which is
> > + // `len * size_of::<Self>()`.
> > + // - `ptr` is aligned for `Self`.
> > + // - `ptr` points to `len` consecutive properly initialized values of type `Self`
> > + // because `Self` implements `FromBytes` (any bit pattern is valid).
> > + // - The lifetime of the returned slice is bound to the input `bytes`.
> > + unsafe { Some(core::slice::from_raw_parts(ptr, len)) }
> > + } else {
> > + None
> > + }
>
> Nit: I’d rewrite this as
>
> if !ptr.is_aligned() {
> return None;
> }
>
> unsafe { Some(core::slice::from_raw_parts(ptr, len)) }
>
> This saves one indentation level.
>
If this code were sizable, or if there were no unsafe, I'd agree with
you. However:
1. It makes the path condition for the `unsafe` block clearer - the if
condition directly guards the unsafe block, which makes it locally
obvious why it can assume "`ptr` is aligned for `Self`." With the
"check error and early return" strategy, you need to reason less
locally about the reachability of the statement.
2. The guarded code is one line long. If it were significant, then I
can see the point of using an early-return style in order to avoid
indentation, especially if the block *itself* had indentation. This is
one line.
3. This is coherent with `from_bytes_mut` and `from_bytes` in this
file, which also use this style.
> > + }
> > +
> > + /// Converts a mutable slice of bytes to a mutable slice of `Self`.
> > + ///
> > + /// Succeeds if the reference is properly aligned, and the size of `bytes` is a multiple of that
> > + /// of `Self`.
> > + ///
> > + /// Otherwise, returns [`None`].
> > + fn from_bytes_to_mut_slice(bytes: &mut [u8]) -> Option<&mut [Self]>
> > + where
> > + Self: AsBytes + Sized,
> > + {
> > + let size = size_of::<Self>();
> > + if size == 0 {
> > + return None;
> > + }
> > + if bytes.len() % size != 0 {
> > + return None;
> > + }
> > + let len = bytes.len() / size;
> > + let ptr = bytes.as_mut_ptr().cast::<Self>();
> > +
> > + #[allow(clippy::incompatible_msrv)]
> > + if ptr.is_aligned() {
> > + // SAFETY:
> > + // - `ptr` is valid for reads and writes of `bytes.len()` bytes, which is
> > + // `len * size_of::<Self>()`.
> > + // - `ptr` is aligned for `Self`.
> > + // - `ptr` points to `len` consecutive properly initialized values of type `Self`
> > + // because `Self` implements `FromBytes`.
> > + // - `AsBytes` guarantees that writing a new `Self` to the resulting slice can safely
> > + // be reflected as bytes in the original slice.
> > + // - The lifetime of the returned slice is bound to the input `bytes`.
> > + unsafe { Some(core::slice::from_raw_parts_mut(ptr, len)) }
> > + } else {
> > + None
> > + }
>
> Same here.
Same here :)
>
> > + }
> > +
> > /// Creates an owned instance of `Self` by copying `bytes`.
> > ///
> > /// Unlike [`FromBytes::from_bytes`], which requires aligned input, this method can be used on
> >
> > --
> > 2.52.0.305.g3fc767764a-goog
> >
> >
>
> Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2025-12-26 20:27 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-12-16 0:44 [PATCH v2 0/3] Support more safe `AsBytes`/`FromBytes` usage Matthew Maurer
2025-12-16 0:44 ` [PATCH v2 1/3] rust: transmute: Support transmuting slices of AsBytes/FromBytes types Matthew Maurer
2025-12-17 16:51 ` Daniel Almeida
2025-12-26 20:27 ` Matthew Maurer
2025-12-16 0:44 ` [PATCH v2 2/3] rust: Add support for deriving `AsBytes` and `FromBytes` Matthew Maurer
2025-12-17 3:12 ` Alexandre Courbot
2025-12-17 18:01 ` Matthew Maurer
2025-12-17 19:14 ` Daniel Almeida
2025-12-18 7:23 ` Alexandre Courbot
2025-12-18 8:26 ` Alice Ryhl
2025-12-17 17:35 ` Daniel Almeida
2025-12-17 17:57 ` Matthew Maurer
2025-12-17 19:11 ` Daniel Almeida
2025-12-16 0:44 ` [PATCH v2 3/3] rust: Support deriving `AsBytes`/`FromBytes` on bindgen types Matthew Maurer
2025-12-17 3:15 ` Alexandre Courbot
2025-12-17 18:26 ` Matthew Maurer
2025-12-17 19:26 ` Daniel Almeida
2025-12-17 19:33 ` Matthew Maurer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox