* [PATCH v6 00/23] Rust support
@ 2022-05-07 5:23 Miguel Ojeda
2022-05-07 5:24 ` [PATCH v6 22/23] [RFC] drivers: gpio: PrimeCell PL061 in Rust Miguel Ojeda
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Miguel Ojeda @ 2022-05-07 5:23 UTC (permalink / raw)
To: Linus Torvalds, Greg Kroah-Hartman
Cc: rust-for-linux, linux-kernel, Jarkko Sakkinen, Miguel Ojeda,
kunit-dev, linux-arm-kernel, linux-doc, linux-gpio, linux-kbuild,
linux-kselftest, linux-perf-users, linuxppc-dev, linux-riscv,
live-patching
Rust support
This is the patch series (v6) to add support for Rust as a second
language to the Linux kernel.
If you are interested in following this effort, please join us in
the mailing list at:
rust-for-linux@vger.kernel.org
and take a look at the project itself at:
https://github.com/Rust-for-Linux
As usual, special thanks go to ISRG (Internet Security Research
Group) and Google for their financial support on this endeavor.
Cheers,
Miguel
--
# Rust support
This cover letter explains the major changes and updates done since
the previous ones. For those, please see:
RFC: https://lore.kernel.org/lkml/20210414184604.23473-1-ojeda@kernel.org/
v1: https://lore.kernel.org/lkml/20210704202756.29107-1-ojeda@kernel.org/
v2: https://lore.kernel.org/lkml/20211206140313.5653-1-ojeda@kernel.org/
v3: https://lore.kernel.org/lkml/20220117053349.6804-1-ojeda@kernel.org/
v4: https://lore.kernel.org/lkml/20220212130410.6901-1-ojeda@kernel.org/
v5: https://lore.kernel.org/lkml/20220317181032.15436-1-ojeda@kernel.org/
## Infrastructure updates
There have been several improvements to the overall Rust support:
- The toolchain and `alloc` have been upgraded to Rust 1.60.0.
This version stabilized `feature(maybe_uninit_extra)` that we
are using.
- Support running documentation tests in-kernel, based on KUnit.
Rust documentation tests are typically examples of usage of any
item (e.g. function, struct, module...). They are very convenient
because they are just written alongside the documentation, e.g.:
/// Sums two numbers.
///
/// # Examples
///
/// ```
/// assert_eq!(mymod::f(10, 20), 30);
/// ```
pub fn f(a: i32, b: i32) -> i32 {
a + b
}
So far, we were compiling and running them in the host as any
other Rust documentation test. However, that meant we could not
run tests that used kernel APIs (though we were compile-testing
them, which was already useful to keep the documentation in sync
with the code).
Now, the documentation tests for the `kernel` crate are
transformed into a KUnit test suite during compilation and run
within the kernel at boot time, if enabled. This means now we can
run the tests that use kernel APIs.
They look like this (their name is generated by `rustdoc`, based
on the file and line):
[ 0.581961] TAP version 14
[ 0.582092] 1..1
[ 0.582267] # Subtest: rust_kernel_doctests
[ 0.582358] 1..70
[ 0.583626] ok 1 - rust_kernel_doctest_build_assert_rs_12_0
[ 0.584579] ok 2 - rust_kernel_doctest_build_assert_rs_55_0
[ 0.587357] ok 3 - rust_kernel_doctest_device_rs_361_0
[ 0.588037] ok 4 - rust_kernel_doctest_device_rs_386_0
...
[ 0.659249] ok 69 - rust_kernel_doctest_types_rs_445_0
[ 0.660451] ok 70 - rust_kernel_doctest_types_rs_509_0
[ 0.660680] # rust_kernel_doctests: pass:70 fail:0 skip:0 total:70
[ 0.660894] # Totals: pass:70 fail:0 skip:0 total:70
[ 0.661135] ok 1 - rust_kernel_doctests
There are other benefits from this, such as being able to remove
unneeded wrapper functions (that were used to avoid running
some tests) as well as ensuring test code would actually compile
within the kernel (e.g. `alloc` used different `cfg`s).
- Tests are now (and are enforced to be) Clippy-clean, like the rest
of the Rust kernel code (i.e. according to the same rules).
- Other cleanups, fixes and improvements.
## Abstractions and driver updates
Some of the improvements to the abstractions and example drivers are:
- The start of networking support (`net` module), with types like:
+ `Namespace` (based on `struct net`).
+ `SkBuff` (based on `struct sk_buff`).
+ `Ipv4Addr` (based on `struct in_addr`), and its v6 equivalent.
+ `SocketAddrV4` (based on `struct sockaddr_in`), and its v6
equivalent.
+ `TcpListener` and `TcpStream` (based on `struct socket`).
- The beginning of `async` support (`kasync` module).
Rust provides support for asynchronous programming in a way that
can be used in constrained environments, including the kernel.
For instance, this allows us to write asynchronous TCP socket code
within the kernel such as:
async fn echo_server(stream: TcpStream) -> Result {
let mut buf = [0u8; 1024];
loop {
let n = stream.read(&mut buf).await?;
if n == 0 {
return Ok(());
}
stream.write_all(&buf[..n]).await?;
}
}
This code looks very close to a synchronous version, yet it
supports being driven to completion "step by step" by an executor.
The `read()`/`write_all()` calls above, instead of blocking the
current thread, return a future which can be polled. The `.await`
points poll the future and, if the result is not ready, suspend
the state such that execution resumes there later on (the state
machine needed for this gets implemented by the compiler). This
allows an executor to drive multiple futures to completion
concurrently on the same thread.
An executor is not included yet, but `kasync` includes async
versions of `TcpListener` and `TcpStream` (based on the non-async
ones) which employ `SocketFuture` (which in turn uses a
`struct wait_queue_entry`).
- Support for network packet filters (`net::filter` module) and its
related `rust_netfilter.rs` sample.
- Added `smutex::Mutex`: a simple mutex that does not require
pinning, so that the ergonomics are much improved, though the
implementation is not as feature-rich as the C-based one.
- New `NoWaitLock`: one that never waits, that is, if it is owned
by another thread/CPU, then attempts to acquire it will fail
(instead of, for example, blocking the caller).
- Added `RawSpinLock` (backed by `raw_spinlock_t`), used when code
sections cannot sleep even in real-time variants of the kernel.
- Added `ARef`, an owned reference to an always-refcounted object,
meant to simplify how we define wrappers to types defined on the
C side of the source code.
- Other cleanups, fixes and improvements.
## Patch series status
The Rust support is still to be considered experimental. However,
support is good enough that kernel developers can start working on the
Rust abstractions for subsystems and write drivers and other modules.
The current series has just arrived in `linux-next`, as usual.
Similarly, the preview docs for this series can be seen at:
https://rust-for-linux.github.io/docs/kernel/
As usual, please see the following link for the live list of unstable
Rust features we are using:
https://github.com/Rust-for-Linux/linux/issues/2
## Conferences, meetings and liaisons
We would like to announce the Rust MC (microconference) in
the upcoming LPC 2022 (Linux Plumbers Conference):
https://lpc.events/event/16/contributions/1159/
The Rust MC intends to cover talks and discussions on both Rust for
Linux as well as other non-kernel Rust topics. The Call for Proposals
is open!
Furthermore, we would like to thank you the venues we were invited to:
- Rust Linz 2022
- Linux Foundation Live Mentorship Series
## Related news
`rustc_codegen_gcc` (the GCC backend for `rustc`) can now bootstrap
`rustc`! In addition, GCC 12.1 (just released) carries some of the
patches that were needed by the project in upstream `libgccjit`; and
the project is looking into getting distributed with `rustup`.
`gccrs` (the Rust frontend for GCC) has got a second full time
developer working on it, Arthur Cohen, as well as a lot of technical
progress too, such as a new testing project, slice generation support
and macro-related work.
## Acknowledgements
The signatures in the main commits correspond to the people that
wrote code that has ended up in them at the present time. For details
on contributions to code and discussions, please see our repository:
https://github.com/Rust-for-Linux/linux
However, we would like to give credit to everyone that has contributed
in one way or another to the Rust for Linux project. Since the
previous cover letter:
- Andy Shevchenko, Petr Mladek, Sergey Senozhatsky for their review
of the `vsprintf` patch.
- Arnaldo Carvalho de Melo and Andrii Nakryiko for their input on
`pahole` and BTF, Arnaldo for adding support `pahole` for `--lang`
and `--lang_exclude` (useful to skip Rust CUs) and Martin Reboredo
for reporting the `CONFIG_DEBUG_INFO_BTF` issue.
- Daniel Latypov, David Gow and Brendan Higgins for their input
on KUnit and their reviews on a prerequisite Rust patch on it.
- Kees Cook for reviewing the kallsyms prerequisite patches.
- Greg Kroah-Hartman for his suggestions on the `alloc` patch.
- Daniel Paoliello for his ongoing work on adding more `try_*`
methods to `Vec` in the standard library. Currently, we have some
similar methods in our custom `alloc` that we could drop once
equivalents arrive upstream. Also John Ericson for his reviews.
- bjorn3 for reviewing many PRs and the input around potential UB
in doctests.
- As usual, bjorn3 and Gary Guo for all the input on Rust compiler
details and suggestions.
- Adam Bratschi-Kaye for working on `seq_file` and `debugfs`
abstractions.
- Maciej Falkowski for continuing his work on the Samsung Exynos
TRNG driver and the required abstractions around it, such as
adding `delay`, `ktime` and `iopoll` abstractions, new methods
to `platform::Device` and run-time power management abstractions.
- Daniel Xu for working on adding a Rust allocator based on the
`kvmalloc` family of functions.
- Hongyu Li for working on Rust iterators as the equivalent of
`cpumask`'s `for_each_*_cpu`.
- Andreas Hindborg for adding support to `kernel::Pages` methods to
allow read/write of multiple pages.
- Sergio González Collado for working on adding `#[cold]` attributes
for error-related items and GitHub CI problem matchers.
- Sean Nash for updating the out-of-tree-module example due to a
change in the main repository.
- Michael Ellerman, Nicholas Piggin, Paul E. McKenney and Zhouyi
Zhou for debugging the `CONFIG_HIGH_RES_TIMERS=n` stall issue
in PowerPC that we triggered in our CI.
- Jonathan Corbet for writing an LWN article on the crates
discussion that took place in the Rust for Linux mailing list.
- Wei Liu for taking the time to answer questions from newcomers
in Zulip.
- Philip Li, Yujie Liu et al. for continuing their work on adding
Rust support to the Intel 0DAY/LKP kernel test robot.
- Philip Herron and Arthur Cohen (and his supporters Open Source
Security and Embecosm) et al. for their ongoing work on GCC Rust.
- Antoni Boucher (and his supporters) et al. for their ongoing
work on `rustc_codegen_gcc`.
- Mats Larsen, Marc Poulhiès et al. for their ongoing work on
improving Rust support in Compiler Explorer.
- Many folks that have reported issues, tested the project,
helped spread the word, joined discussions and contributed in
other ways!
Please see also the acknowledgements on the previous cover letters.
Boqun Feng (1):
kallsyms: avoid hardcoding the buffer size
Gary Guo (2):
rust: add `build_error` crate
vsprintf: add new `%pA` format specifier
Miguel Ojeda (16):
kallsyms: support "big" kernel symbols
kallsyms: increase maximum kernel symbol length to 512
kunit: take `kunit_assert` as `const`
rust: add C helpers
rust: add `compiler_builtins` crate
rust: import upstream `alloc` crate
rust: adapt `alloc` crate to the kernel
rust: add `macros` crate
rust: export generated symbols
scripts: add `rustdoc_test_{builder,gen}.py` scripts
scripts: add `generate_rust_analyzer.py` scripts
scripts: decode_stacktrace: demangle Rust symbols
docs: add Rust documentation
Kbuild: add Rust support
samples: add Rust examples
MAINTAINERS: Rust
Wedson Almeida Filho (4):
rust: add `kernel` crate's `sync` module
rust: add `kernel` crate
[RFC] drivers: gpio: PrimeCell PL061 in Rust
[RFC] drivers: android: Binder IPC in Rust
.gitignore | 5 +
.rustfmt.toml | 12 +
Documentation/doc-guide/kernel-doc.rst | 3 +
Documentation/index.rst | 1 +
Documentation/kbuild/kbuild.rst | 17 +
Documentation/kbuild/makefiles.rst | 50 +-
Documentation/process/changes.rst | 41 +
Documentation/rust/arch-support.rst | 34 +
Documentation/rust/coding-guidelines.rst | 214 ++
Documentation/rust/general-information.rst | 77 +
Documentation/rust/index.rst | 20 +
Documentation/rust/logo.svg | 357 ++
Documentation/rust/quick-start.rst | 230 ++
MAINTAINERS | 15 +
Makefile | 175 +-
arch/Kconfig | 6 +
arch/arm/Kconfig | 1 +
arch/arm64/Kconfig | 1 +
arch/powerpc/Kconfig | 1 +
arch/riscv/Kconfig | 1 +
arch/riscv/Makefile | 5 +
arch/x86/Kconfig | 1 +
arch/x86/Makefile | 14 +
drivers/android/Kconfig | 6 +
drivers/android/Makefile | 2 +
drivers/android/allocation.rs | 266 ++
drivers/android/context.rs | 80 +
drivers/android/defs.rs | 99 +
drivers/android/node.rs | 476 +++
drivers/android/process.rs | 960 +++++
drivers/android/range_alloc.rs | 189 +
drivers/android/rust_binder.rs | 111 +
drivers/android/thread.rs | 870 +++++
drivers/android/transaction.rs | 326 ++
drivers/gpio/Kconfig | 8 +
drivers/gpio/Makefile | 1 +
drivers/gpio/gpio_pl061_rust.rs | 370 ++
include/kunit/test.h | 2 +-
include/linux/kallsyms.h | 2 +-
include/linux/spinlock.h | 25 +-
include/uapi/linux/android/binder.h | 28 +-
init/Kconfig | 45 +-
kernel/kallsyms.c | 26 +-
kernel/livepatch/core.c | 4 +-
lib/Kconfig.debug | 155 +
lib/kunit/test.c | 4 +-
lib/vsprintf.c | 13 +
rust/.gitignore | 10 +
rust/Makefile | 397 +++
rust/alloc/README.md | 32 +
rust/alloc/alloc.rs | 438 +++
rust/alloc/borrow.rs | 498 +++
rust/alloc/boxed.rs | 2007 +++++++++++
rust/alloc/collections/mod.rs | 156 +
rust/alloc/fmt.rs | 601 ++++
rust/alloc/lib.rs | 226 ++
rust/alloc/macros.rs | 127 +
rust/alloc/raw_vec.rs | 567 +++
rust/alloc/slice.rs | 1282 +++++++
rust/alloc/str.rs | 632 ++++
rust/alloc/string.rs | 2869 +++++++++++++++
rust/alloc/vec/drain.rs | 186 +
rust/alloc/vec/drain_filter.rs | 145 +
rust/alloc/vec/into_iter.rs | 356 ++
rust/alloc/vec/is_zero.rs | 106 +
rust/alloc/vec/mod.rs | 3362 ++++++++++++++++++
rust/alloc/vec/partial_eq.rs | 49 +
rust/alloc/vec/set_len_on_drop.rs | 30 +
rust/alloc/vec/spec_extend.rs | 174 +
rust/bindgen_parameters | 17 +
rust/build_error.rs | 29 +
rust/compiler_builtins.rs | 57 +
rust/exports.c | 20 +
rust/helpers.c | 639 ++++
rust/kernel/allocator.rs | 65 +
rust/kernel/amba.rs | 257 ++
rust/kernel/bindings.rs | 47 +
rust/kernel/bindings_helper.h | 46 +
rust/kernel/build_assert.rs | 82 +
rust/kernel/c_types.rs | 119 +
rust/kernel/chrdev.rs | 207 ++
rust/kernel/clk.rs | 79 +
rust/kernel/cred.rs | 46 +
rust/kernel/device.rs | 546 +++
rust/kernel/driver.rs | 442 +++
rust/kernel/error.rs | 565 +++
rust/kernel/file.rs | 860 +++++
rust/kernel/gpio.rs | 478 +++
rust/kernel/hwrng.rs | 242 ++
rust/kernel/io_buffer.rs | 153 +
rust/kernel/io_mem.rs | 275 ++
rust/kernel/iov_iter.rs | 81 +
rust/kernel/irq.rs | 409 +++
rust/kernel/kasync.rs | 6 +
rust/kernel/kasync/net.rs | 322 ++
rust/kernel/kunit.rs | 91 +
rust/kernel/lib.rs | 260 ++
rust/kernel/linked_list.rs | 247 ++
rust/kernel/miscdev.rs | 291 ++
rust/kernel/mm.rs | 149 +
rust/kernel/module_param.rs | 498 +++
rust/kernel/net.rs | 392 ++
rust/kernel/net/filter.rs | 447 +++
rust/kernel/of.rs | 63 +
rust/kernel/pages.rs | 144 +
rust/kernel/platform.rs | 223 ++
rust/kernel/power.rs | 118 +
rust/kernel/prelude.rs | 36 +
rust/kernel/print.rs | 405 +++
rust/kernel/random.rs | 42 +
rust/kernel/raw_list.rs | 361 ++
rust/kernel/rbtree.rs | 563 +++
rust/kernel/revocable.rs | 161 +
rust/kernel/security.rs | 38 +
rust/kernel/static_assert.rs | 38 +
rust/kernel/std_vendor.rs | 160 +
rust/kernel/str.rs | 597 ++++
rust/kernel/sync.rs | 161 +
rust/kernel/sync/arc.rs | 503 +++
rust/kernel/sync/condvar.rs | 138 +
rust/kernel/sync/guard.rs | 169 +
rust/kernel/sync/locked_by.rs | 111 +
rust/kernel/sync/mutex.rs | 153 +
rust/kernel/sync/nowait.rs | 188 +
rust/kernel/sync/revocable.rs | 250 ++
rust/kernel/sync/rwsem.rs | 197 +
rust/kernel/sync/seqlock.rs | 202 ++
rust/kernel/sync/smutex.rs | 295 ++
rust/kernel/sync/spinlock.rs | 360 ++
rust/kernel/sysctl.rs | 199 ++
rust/kernel/task.rs | 175 +
rust/kernel/types.rs | 679 ++++
rust/kernel/user_ptr.rs | 175 +
rust/macros/helpers.rs | 79 +
rust/macros/lib.rs | 94 +
rust/macros/module.rs | 631 ++++
samples/Kconfig | 2 +
samples/Makefile | 1 +
samples/rust/Kconfig | 140 +
samples/rust/Makefile | 16 +
samples/rust/hostprogs/.gitignore | 3 +
samples/rust/hostprogs/Makefile | 5 +
samples/rust/hostprogs/a.rs | 7 +
samples/rust/hostprogs/b.rs | 5 +
samples/rust/hostprogs/single.rs | 12 +
samples/rust/rust_chrdev.rs | 50 +
samples/rust/rust_minimal.rs | 35 +
samples/rust/rust_miscdev.rs | 143 +
samples/rust/rust_module_parameters.rs | 69 +
samples/rust/rust_netfilter.rs | 54 +
samples/rust/rust_platform.rs | 22 +
samples/rust/rust_print.rs | 54 +
samples/rust/rust_random.rs | 60 +
samples/rust/rust_semaphore.rs | 171 +
samples/rust/rust_semaphore_c.c | 212 ++
samples/rust/rust_stack_probing.rs | 36 +
samples/rust/rust_sync.rs | 93 +
scripts/.gitignore | 1 +
scripts/Kconfig.include | 6 +-
scripts/Makefile | 3 +
scripts/Makefile.build | 60 +
scripts/Makefile.debug | 10 +
scripts/Makefile.host | 34 +-
scripts/Makefile.lib | 12 +
scripts/Makefile.modfinal | 8 +-
scripts/cc-version.sh | 12 +-
scripts/decode_stacktrace.sh | 14 +
scripts/generate_rust_analyzer.py | 134 +
scripts/generate_rust_target.rs | 227 ++
scripts/is_rust_module.sh | 13 +
scripts/kallsyms.c | 47 +-
scripts/kconfig/confdata.c | 75 +
scripts/min-tool-version.sh | 6 +
scripts/rust-is-available-bindgen-libclang.h | 2 +
scripts/rust-is-available.sh | 158 +
scripts/rustdoc_test_builder.py | 59 +
scripts/rustdoc_test_gen.py | 164 +
tools/include/linux/kallsyms.h | 2 +-
tools/lib/perf/include/perf/event.h | 2 +-
tools/lib/symbol/kallsyms.h | 2 +-
180 files changed, 37945 insertions(+), 67 deletions(-)
create mode 100644 .rustfmt.toml
create mode 100644 Documentation/rust/arch-support.rst
create mode 100644 Documentation/rust/coding-guidelines.rst
create mode 100644 Documentation/rust/general-information.rst
create mode 100644 Documentation/rust/index.rst
create mode 100644 Documentation/rust/logo.svg
create mode 100644 Documentation/rust/quick-start.rst
create mode 100644 drivers/android/allocation.rs
create mode 100644 drivers/android/context.rs
create mode 100644 drivers/android/defs.rs
create mode 100644 drivers/android/node.rs
create mode 100644 drivers/android/process.rs
create mode 100644 drivers/android/range_alloc.rs
create mode 100644 drivers/android/rust_binder.rs
create mode 100644 drivers/android/thread.rs
create mode 100644 drivers/android/transaction.rs
create mode 100644 drivers/gpio/gpio_pl061_rust.rs
create mode 100644 rust/.gitignore
create mode 100644 rust/Makefile
create mode 100644 rust/alloc/README.md
create mode 100644 rust/alloc/alloc.rs
create mode 100644 rust/alloc/borrow.rs
create mode 100644 rust/alloc/boxed.rs
create mode 100644 rust/alloc/collections/mod.rs
create mode 100644 rust/alloc/fmt.rs
create mode 100644 rust/alloc/lib.rs
create mode 100644 rust/alloc/macros.rs
create mode 100644 rust/alloc/raw_vec.rs
create mode 100644 rust/alloc/slice.rs
create mode 100644 rust/alloc/str.rs
create mode 100644 rust/alloc/string.rs
create mode 100644 rust/alloc/vec/drain.rs
create mode 100644 rust/alloc/vec/drain_filter.rs
create mode 100644 rust/alloc/vec/into_iter.rs
create mode 100644 rust/alloc/vec/is_zero.rs
create mode 100644 rust/alloc/vec/mod.rs
create mode 100644 rust/alloc/vec/partial_eq.rs
create mode 100644 rust/alloc/vec/set_len_on_drop.rs
create mode 100644 rust/alloc/vec/spec_extend.rs
create mode 100644 rust/bindgen_parameters
create mode 100644 rust/build_error.rs
create mode 100644 rust/compiler_builtins.rs
create mode 100644 rust/exports.c
create mode 100644 rust/helpers.c
create mode 100644 rust/kernel/allocator.rs
create mode 100644 rust/kernel/amba.rs
create mode 100644 rust/kernel/bindings.rs
create mode 100644 rust/kernel/bindings_helper.h
create mode 100644 rust/kernel/build_assert.rs
create mode 100644 rust/kernel/c_types.rs
create mode 100644 rust/kernel/chrdev.rs
create mode 100644 rust/kernel/clk.rs
create mode 100644 rust/kernel/cred.rs
create mode 100644 rust/kernel/device.rs
create mode 100644 rust/kernel/driver.rs
create mode 100644 rust/kernel/error.rs
create mode 100644 rust/kernel/file.rs
create mode 100644 rust/kernel/gpio.rs
create mode 100644 rust/kernel/hwrng.rs
create mode 100644 rust/kernel/io_buffer.rs
create mode 100644 rust/kernel/io_mem.rs
create mode 100644 rust/kernel/iov_iter.rs
create mode 100644 rust/kernel/irq.rs
create mode 100644 rust/kernel/kasync.rs
create mode 100644 rust/kernel/kasync/net.rs
create mode 100644 rust/kernel/kunit.rs
create mode 100644 rust/kernel/lib.rs
create mode 100644 rust/kernel/linked_list.rs
create mode 100644 rust/kernel/miscdev.rs
create mode 100644 rust/kernel/mm.rs
create mode 100644 rust/kernel/module_param.rs
create mode 100644 rust/kernel/net.rs
create mode 100644 rust/kernel/net/filter.rs
create mode 100644 rust/kernel/of.rs
create mode 100644 rust/kernel/pages.rs
create mode 100644 rust/kernel/platform.rs
create mode 100644 rust/kernel/power.rs
create mode 100644 rust/kernel/prelude.rs
create mode 100644 rust/kernel/print.rs
create mode 100644 rust/kernel/random.rs
create mode 100644 rust/kernel/raw_list.rs
create mode 100644 rust/kernel/rbtree.rs
create mode 100644 rust/kernel/revocable.rs
create mode 100644 rust/kernel/security.rs
create mode 100644 rust/kernel/static_assert.rs
create mode 100644 rust/kernel/std_vendor.rs
create mode 100644 rust/kernel/str.rs
create mode 100644 rust/kernel/sync.rs
create mode 100644 rust/kernel/sync/arc.rs
create mode 100644 rust/kernel/sync/condvar.rs
create mode 100644 rust/kernel/sync/guard.rs
create mode 100644 rust/kernel/sync/locked_by.rs
create mode 100644 rust/kernel/sync/mutex.rs
create mode 100644 rust/kernel/sync/nowait.rs
create mode 100644 rust/kernel/sync/revocable.rs
create mode 100644 rust/kernel/sync/rwsem.rs
create mode 100644 rust/kernel/sync/seqlock.rs
create mode 100644 rust/kernel/sync/smutex.rs
create mode 100644 rust/kernel/sync/spinlock.rs
create mode 100644 rust/kernel/sysctl.rs
create mode 100644 rust/kernel/task.rs
create mode 100644 rust/kernel/types.rs
create mode 100644 rust/kernel/user_ptr.rs
create mode 100644 rust/macros/helpers.rs
create mode 100644 rust/macros/lib.rs
create mode 100644 rust/macros/module.rs
create mode 100644 samples/rust/Kconfig
create mode 100644 samples/rust/Makefile
create mode 100644 samples/rust/hostprogs/.gitignore
create mode 100644 samples/rust/hostprogs/Makefile
create mode 100644 samples/rust/hostprogs/a.rs
create mode 100644 samples/rust/hostprogs/b.rs
create mode 100644 samples/rust/hostprogs/single.rs
create mode 100644 samples/rust/rust_chrdev.rs
create mode 100644 samples/rust/rust_minimal.rs
create mode 100644 samples/rust/rust_miscdev.rs
create mode 100644 samples/rust/rust_module_parameters.rs
create mode 100644 samples/rust/rust_netfilter.rs
create mode 100644 samples/rust/rust_platform.rs
create mode 100644 samples/rust/rust_print.rs
create mode 100644 samples/rust/rust_random.rs
create mode 100644 samples/rust/rust_semaphore.rs
create mode 100644 samples/rust/rust_semaphore_c.c
create mode 100644 samples/rust/rust_stack_probing.rs
create mode 100644 samples/rust/rust_sync.rs
create mode 100755 scripts/generate_rust_analyzer.py
create mode 100644 scripts/generate_rust_target.rs
create mode 100755 scripts/is_rust_module.sh
create mode 100644 scripts/rust-is-available-bindgen-libclang.h
create mode 100755 scripts/rust-is-available.sh
create mode 100755 scripts/rustdoc_test_builder.py
create mode 100755 scripts/rustdoc_test_gen.py
base-commit: 672c0c5173427e6b3e2a9bbb7be51ceeec78093a
--
2.35.3
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH v6 22/23] [RFC] drivers: gpio: PrimeCell PL061 in Rust 2022-05-07 5:23 [PATCH v6 00/23] Rust support Miguel Ojeda @ 2022-05-07 5:24 ` Miguel Ojeda 2022-05-07 8:06 ` [PATCH v6 00/23] Rust support Kees Cook 2022-05-07 9:29 ` David Gow 2 siblings, 0 replies; 9+ messages in thread From: Miguel Ojeda @ 2022-05-07 5:24 UTC (permalink / raw) To: Linus Torvalds, Greg Kroah-Hartman Cc: rust-for-linux, linux-kernel, Jarkko Sakkinen, Miguel Ojeda, Wedson Almeida Filho, Linus Walleij, Bartosz Golaszewski, linux-gpio From: Wedson Almeida Filho <wedsonaf@google.com> A port to Rust of the PrimeCell PL061 GPIO driver. This module is a work in progress and will be sent for review later on, as well as separately from the Rust support. However, it is included to show how an actual working module written in Rust may look like. Signed-off-by: Wedson Almeida Filho <wedsonaf@google.com> Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- drivers/gpio/Kconfig | 8 + drivers/gpio/Makefile | 1 + drivers/gpio/gpio_pl061_rust.rs | 370 ++++++++++++++++++++++++++++++++ 3 files changed, 379 insertions(+) create mode 100644 drivers/gpio/gpio_pl061_rust.rs diff --git a/drivers/gpio/Kconfig b/drivers/gpio/Kconfig index 45764ec3b2eb..ad99b96f6d79 100644 --- a/drivers/gpio/Kconfig +++ b/drivers/gpio/Kconfig @@ -481,6 +481,14 @@ config GPIO_PL061 help Say yes here to support the PrimeCell PL061 GPIO device. +config GPIO_PL061_RUST + tristate "PrimeCell PL061 GPIO support written in Rust" + depends on ARM_AMBA && RUST + select IRQ_DOMAIN + select GPIOLIB_IRQCHIP + help + Say yes here to support the PrimeCell PL061 GPIO device + config GPIO_PMIC_EIC_SPRD tristate "Spreadtrum PMIC EIC support" depends on MFD_SC27XX_PMIC || COMPILE_TEST diff --git a/drivers/gpio/Makefile b/drivers/gpio/Makefile index 14352f6dfe8e..30141fec12be 100644 --- a/drivers/gpio/Makefile +++ b/drivers/gpio/Makefile @@ -118,6 +118,7 @@ obj-$(CONFIG_GPIO_PCIE_IDIO_24) += gpio-pcie-idio-24.o obj-$(CONFIG_GPIO_PCI_IDIO_16) += gpio-pci-idio-16.o obj-$(CONFIG_GPIO_PISOSR) += gpio-pisosr.o obj-$(CONFIG_GPIO_PL061) += gpio-pl061.o +obj-$(CONFIG_GPIO_PL061_RUST) += gpio_pl061_rust.o obj-$(CONFIG_GPIO_PMIC_EIC_SPRD) += gpio-pmic-eic-sprd.o obj-$(CONFIG_GPIO_PXA) += gpio-pxa.o obj-$(CONFIG_GPIO_RASPBERRYPI_EXP) += gpio-raspberrypi-exp.o diff --git a/drivers/gpio/gpio_pl061_rust.rs b/drivers/gpio/gpio_pl061_rust.rs new file mode 100644 index 000000000000..13c8c3eb3e4f --- /dev/null +++ b/drivers/gpio/gpio_pl061_rust.rs @@ -0,0 +1,370 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Driver for the ARM PrimeCell(tm) General Purpose Input/Output (PL061). +//! +//! Based on the C driver written by Baruch Siach <baruch@tkos.co.il>. + +use kernel::{ + amba, bit, bits_iter, define_amba_id_table, device, gpio, + io_mem::IoMem, + irq::{self, ExtraResult, IrqData, LockedIrqData}, + power, + prelude::*, + sync::{RawSpinLock, Ref, RefBorrow}, +}; + +const GPIODIR: usize = 0x400; +const GPIOIS: usize = 0x404; +const GPIOIBE: usize = 0x408; +const GPIOIEV: usize = 0x40C; +const GPIOIE: usize = 0x410; +const GPIOMIS: usize = 0x418; +const GPIOIC: usize = 0x41C; +const GPIO_SIZE: usize = 0x1000; + +const PL061_GPIO_NR: u16 = 8; + +#[derive(Default)] +struct ContextSaveRegs { + gpio_data: u8, + gpio_dir: u8, + gpio_is: u8, + gpio_ibe: u8, + gpio_iev: u8, + gpio_ie: u8, +} + +#[derive(Default)] +struct PL061DataInner { + csave_regs: ContextSaveRegs, +} + +struct PL061Data { + dev: device::Device, + inner: RawSpinLock<PL061DataInner>, +} + +struct PL061Resources { + base: IoMem<GPIO_SIZE>, + parent_irq: u32, +} + +type PL061Registrations = gpio::RegistrationWithIrqChip<PL061Device>; + +type DeviceData = device::Data<PL061Registrations, PL061Resources, PL061Data>; + +struct PL061Device; + +impl gpio::Chip for PL061Device { + type Data = Ref<DeviceData>; + + kernel::declare_gpio_chip_operations!( + get_direction, + direction_input, + direction_output, + get, + set + ); + + fn get_direction(data: RefBorrow<'_, DeviceData>, offset: u32) -> Result<gpio::LineDirection> { + let pl061 = data.resources().ok_or(ENXIO)?; + Ok(if pl061.base.readb(GPIODIR) & bit(offset) != 0 { + gpio::LineDirection::Out + } else { + gpio::LineDirection::In + }) + } + + fn direction_input(data: RefBorrow<'_, DeviceData>, offset: u32) -> Result { + let _guard = data.inner.lock_irqdisable(); + let pl061 = data.resources().ok_or(ENXIO)?; + let mut gpiodir = pl061.base.readb(GPIODIR); + gpiodir &= !bit(offset); + pl061.base.writeb(gpiodir, GPIODIR); + Ok(()) + } + + fn direction_output(data: RefBorrow<'_, DeviceData>, offset: u32, value: bool) -> Result { + let woffset = bit(offset + 2).into(); + let _guard = data.inner.lock_irqdisable(); + let pl061 = data.resources().ok_or(ENXIO)?; + pl061.base.try_writeb((value as u8) << offset, woffset)?; + let mut gpiodir = pl061.base.readb(GPIODIR); + gpiodir |= bit(offset); + pl061.base.writeb(gpiodir, GPIODIR); + + // gpio value is set again, because pl061 doesn't allow to set value of a gpio pin before + // configuring it in OUT mode. + pl061.base.try_writeb((value as u8) << offset, woffset)?; + Ok(()) + } + + fn get(data: RefBorrow<'_, DeviceData>, offset: u32) -> Result<bool> { + let pl061 = data.resources().ok_or(ENXIO)?; + Ok(pl061.base.try_readb(bit(offset + 2).into())? != 0) + } + + fn set(data: RefBorrow<'_, DeviceData>, offset: u32, value: bool) { + if let Some(pl061) = data.resources() { + let woffset = bit(offset + 2).into(); + let _ = pl061.base.try_writeb((value as u8) << offset, woffset); + } + } +} + +impl gpio::ChipWithIrqChip for PL061Device { + fn handle_irq_flow( + data: RefBorrow<'_, DeviceData>, + desc: &irq::Descriptor, + domain: &irq::Domain, + ) { + let chained = desc.enter_chained(); + + if let Some(pl061) = data.resources() { + let pending = pl061.base.readb(GPIOMIS); + for offset in bits_iter(pending) { + domain.generic_handle_chained(offset, &chained); + } + } + } +} + +impl irq::Chip for PL061Device { + type Data = Ref<DeviceData>; + + kernel::declare_irq_chip_operations!(set_type, set_wake); + + fn set_type( + data: RefBorrow<'_, DeviceData>, + irq_data: &mut LockedIrqData, + trigger: u32, + ) -> Result<ExtraResult> { + let offset = irq_data.hwirq(); + let bit = bit(offset); + + if offset >= PL061_GPIO_NR.into() { + return Err(EINVAL); + } + + if trigger & (irq::Type::LEVEL_HIGH | irq::Type::LEVEL_LOW) != 0 + && trigger & (irq::Type::EDGE_RISING | irq::Type::EDGE_FALLING) != 0 + { + dev_err!( + data.dev, + "trying to configure line {} for both level and edge detection, choose one!\n", + offset + ); + return Err(EINVAL); + } + + let _guard = data.inner.lock_irqdisable(); + let pl061 = data.resources().ok_or(ENXIO)?; + + let mut gpioiev = pl061.base.readb(GPIOIEV); + let mut gpiois = pl061.base.readb(GPIOIS); + let mut gpioibe = pl061.base.readb(GPIOIBE); + + if trigger & (irq::Type::LEVEL_HIGH | irq::Type::LEVEL_LOW) != 0 { + let polarity = trigger & irq::Type::LEVEL_HIGH != 0; + + // Disable edge detection. + gpioibe &= !bit; + // Enable level detection. + gpiois |= bit; + // Select polarity. + if polarity { + gpioiev |= bit; + } else { + gpioiev &= !bit; + } + irq_data.set_level_handler(); + dev_dbg!( + data.dev, + "line {}: IRQ on {} level\n", + offset, + if polarity { "HIGH" } else { "LOW" } + ); + } else if (trigger & irq::Type::EDGE_BOTH) == irq::Type::EDGE_BOTH { + // Disable level detection. + gpiois &= !bit; + // Select both edges, settings this makes GPIOEV be ignored. + gpioibe |= bit; + irq_data.set_edge_handler(); + dev_dbg!(data.dev, "line {}: IRQ on both edges\n", offset); + } else if trigger & (irq::Type::EDGE_RISING | irq::Type::EDGE_FALLING) != 0 { + let rising = trigger & irq::Type::EDGE_RISING != 0; + + // Disable level detection. + gpiois &= !bit; + // Clear detection on both edges. + gpioibe &= !bit; + // Select edge. + if rising { + gpioiev |= bit; + } else { + gpioiev &= !bit; + } + irq_data.set_edge_handler(); + dev_dbg!( + data.dev, + "line {}: IRQ on {} edge\n", + offset, + if rising { "RISING" } else { "FALLING}" } + ); + } else { + // No trigger: disable everything. + gpiois &= !bit; + gpioibe &= !bit; + gpioiev &= !bit; + irq_data.set_bad_handler(); + dev_warn!(data.dev, "no trigger selected for line {}\n", offset); + } + + pl061.base.writeb(gpiois, GPIOIS); + pl061.base.writeb(gpioibe, GPIOIBE); + pl061.base.writeb(gpioiev, GPIOIEV); + + Ok(ExtraResult::None) + } + + fn mask(data: RefBorrow<'_, DeviceData>, irq_data: &IrqData) { + let mask = bit(irq_data.hwirq() % irq::HwNumber::from(PL061_GPIO_NR)); + let _guard = data.inner.lock(); + if let Some(pl061) = data.resources() { + let gpioie = pl061.base.readb(GPIOIE) & !mask; + pl061.base.writeb(gpioie, GPIOIE); + } + } + + fn unmask(data: RefBorrow<'_, DeviceData>, irq_data: &IrqData) { + let mask = bit(irq_data.hwirq() % irq::HwNumber::from(PL061_GPIO_NR)); + let _guard = data.inner.lock(); + if let Some(pl061) = data.resources() { + let gpioie = pl061.base.readb(GPIOIE) | mask; + pl061.base.writeb(gpioie, GPIOIE); + } + } + + // This gets called from the edge IRQ handler to ACK the edge IRQ in the GPIOIC + // (interrupt-clear) register. For level IRQs this is not needed: these go away when the level + // signal goes away. + fn ack(data: RefBorrow<'_, DeviceData>, irq_data: &IrqData) { + let mask = bit(irq_data.hwirq() % irq::HwNumber::from(PL061_GPIO_NR)); + let _guard = data.inner.lock(); + if let Some(pl061) = data.resources() { + pl061.base.writeb(mask.into(), GPIOIC); + } + } + + fn set_wake(data: RefBorrow<'_, DeviceData>, _irq_data: &IrqData, on: bool) -> Result { + let pl061 = data.resources().ok_or(ENXIO)?; + irq::set_wake(pl061.parent_irq, on) + } +} + +impl amba::Driver for PL061Device { + type Data = Ref<DeviceData>; + type PowerOps = Self; + + define_amba_id_table! {(), [ + ({id: 0x00041061, mask: 0x000fffff}, None), + ]} + + fn probe(dev: &mut amba::Device, _data: Option<&Self::IdInfo>) -> Result<Ref<DeviceData>> { + let res = dev.take_resource().ok_or(ENXIO)?; + let irq = dev.irq(0).ok_or(ENXIO)?; + + let mut data = kernel::new_device_data!( + gpio::RegistrationWithIrqChip::new(), + PL061Resources { + // SAFETY: This device doesn't support DMA. + base: unsafe { IoMem::try_new(res)? }, + parent_irq: irq, + }, + PL061Data { + dev: device::Device::from_dev(dev), + // SAFETY: We call `rawspinlock_init` below. + inner: unsafe { RawSpinLock::new(PL061DataInner::default()) }, + }, + "PL061::Registrations" + )?; + + // SAFETY: General part of the data is pinned when `data` is. + let gen_inner = unsafe { data.as_mut().map_unchecked_mut(|d| &mut (**d).inner) }; + kernel::rawspinlock_init!(gen_inner, "PL061Data::inner"); + + let data = Ref::<DeviceData>::from(data); + + data.resources().ok_or(ENXIO)?.base.writeb(0, GPIOIE); // disable irqs + + data.registrations() + .ok_or(ENXIO)? + .as_pinned_mut() + .register::<Self>(PL061_GPIO_NR, None, dev, data.clone(), irq)?; + + dev_info!(data.dev, "PL061 GPIO chip registered\n"); + + Ok(data) + } +} + +impl power::Operations for PL061Device { + type Data = Ref<DeviceData>; + + fn suspend(data: RefBorrow<'_, DeviceData>) -> Result { + let mut inner = data.inner.lock(); + let pl061 = data.resources().ok_or(ENXIO)?; + inner.csave_regs.gpio_data = 0; + inner.csave_regs.gpio_dir = pl061.base.readb(GPIODIR); + inner.csave_regs.gpio_is = pl061.base.readb(GPIOIS); + inner.csave_regs.gpio_ibe = pl061.base.readb(GPIOIBE); + inner.csave_regs.gpio_iev = pl061.base.readb(GPIOIEV); + inner.csave_regs.gpio_ie = pl061.base.readb(GPIOIE); + + for offset in 0..PL061_GPIO_NR { + if inner.csave_regs.gpio_dir & bit(offset) != 0 { + if let Ok(v) = <Self as gpio::Chip>::get(data, offset.into()) { + inner.csave_regs.gpio_data |= (v as u8) << offset; + } + } + } + + Ok(()) + } + + fn resume(data: RefBorrow<'_, DeviceData>) -> Result { + let inner = data.inner.lock(); + let pl061 = data.resources().ok_or(ENXIO)?; + + for offset in 0..PL061_GPIO_NR { + if inner.csave_regs.gpio_dir & bit(offset) != 0 { + let value = inner.csave_regs.gpio_data & bit(offset) != 0; + let _ = <Self as gpio::Chip>::direction_output(data, offset.into(), value); + } else { + let _ = <Self as gpio::Chip>::direction_input(data, offset.into()); + } + } + + pl061.base.writeb(inner.csave_regs.gpio_is, GPIOIS); + pl061.base.writeb(inner.csave_regs.gpio_ibe, GPIOIBE); + pl061.base.writeb(inner.csave_regs.gpio_iev, GPIOIEV); + pl061.base.writeb(inner.csave_regs.gpio_ie, GPIOIE); + + Ok(()) + } + + fn freeze(data: RefBorrow<'_, DeviceData>) -> Result { + Self::suspend(data) + } + + fn restore(data: RefBorrow<'_, DeviceData>) -> Result { + Self::resume(data) + } +} + +module_amba_driver! { + type: PL061Device, + name: b"pl061_gpio", + author: b"Wedson Almeida Filho", + license: b"GPL v2", +} -- 2.35.3 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v6 00/23] Rust support 2022-05-07 5:23 [PATCH v6 00/23] Rust support Miguel Ojeda 2022-05-07 5:24 ` [PATCH v6 22/23] [RFC] drivers: gpio: PrimeCell PL061 in Rust Miguel Ojeda @ 2022-05-07 8:06 ` Kees Cook 2022-05-08 18:06 ` Matthew Wilcox 2022-05-09 9:39 ` Wei Liu 2022-05-07 9:29 ` David Gow 2 siblings, 2 replies; 9+ messages in thread From: Kees Cook @ 2022-05-07 8:06 UTC (permalink / raw) To: Miguel Ojeda Cc: Linus Torvalds, Greg Kroah-Hartman, rust-for-linux, linux-kernel, Jarkko Sakkinen, kunit-dev, linux-arm-kernel, linux-doc, linux-gpio, linux-kbuild, linux-kselftest, linux-perf-users, linuxppc-dev, linux-riscv, live-patching On Sat, May 07, 2022 at 07:23:58AM +0200, Miguel Ojeda wrote: > ## Patch series status > > The Rust support is still to be considered experimental. However, > support is good enough that kernel developers can start working on the > Rust abstractions for subsystems and write drivers and other modules. I'd really like to see this landed for a few reasons: - It's under active development, and I'd rather review the changes "normally", incrementally, etc. Right now it can be hard to re-review some of the "mostly the same each version" patches in the series. - I'd like to break the catch-22 of "ask for a new driver to be written in rust but the rust support isn't landed" vs "the rust support isn't landed because there aren't enough drivers". It really feels like "release early, release often" is needed here; it's hard to develop against -next. :) Should we give it a try for this coming merge window? -- Kees Cook ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v6 00/23] Rust support 2022-05-07 8:06 ` [PATCH v6 00/23] Rust support Kees Cook @ 2022-05-08 18:06 ` Matthew Wilcox 2022-05-09 9:39 ` Wei Liu 1 sibling, 0 replies; 9+ messages in thread From: Matthew Wilcox @ 2022-05-08 18:06 UTC (permalink / raw) To: Kees Cook Cc: Miguel Ojeda, Linus Torvalds, Greg Kroah-Hartman, rust-for-linux, linux-kernel, Jarkko Sakkinen, kunit-dev, linux-arm-kernel, linux-doc, linux-gpio, linux-kbuild, linux-kselftest, linux-perf-users, linuxppc-dev, linux-riscv, live-patching On Sat, May 07, 2022 at 01:06:18AM -0700, Kees Cook wrote: > On Sat, May 07, 2022 at 07:23:58AM +0200, Miguel Ojeda wrote: > > ## Patch series status > > > > The Rust support is still to be considered experimental. However, > > support is good enough that kernel developers can start working on the > > Rust abstractions for subsystems and write drivers and other modules. > > I'd really like to see this landed for a few reasons: > > - It's under active development, and I'd rather review the changes > "normally", incrementally, etc. Right now it can be hard to re-review > some of the "mostly the same each version" patches in the series. > > - I'd like to break the catch-22 of "ask for a new driver to be > written in rust but the rust support isn't landed" vs "the rust > support isn't landed because there aren't enough drivers". It > really feels like "release early, release often" is needed here; > it's hard to develop against -next. :) > > Should we give it a try for this coming merge window? I'm broadly in favour of that. It's just code, we can always drop it again or fix it. There's sufficient development community around it that it's hardly going to become abandonware. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v6 00/23] Rust support 2022-05-07 8:06 ` [PATCH v6 00/23] Rust support Kees Cook 2022-05-08 18:06 ` Matthew Wilcox @ 2022-05-09 9:39 ` Wei Liu 1 sibling, 0 replies; 9+ messages in thread From: Wei Liu @ 2022-05-09 9:39 UTC (permalink / raw) To: Kees Cook Cc: Miguel Ojeda, Linus Torvalds, Greg Kroah-Hartman, rust-for-linux, linux-kernel, Jarkko Sakkinen, kunit-dev, linux-arm-kernel, linux-doc, linux-gpio, linux-kbuild, linux-kselftest, linux-perf-users, linuxppc-dev, linux-riscv, live-patching, Wei Liu On Sat, May 07, 2022 at 01:06:18AM -0700, Kees Cook wrote: > On Sat, May 07, 2022 at 07:23:58AM +0200, Miguel Ojeda wrote: > > ## Patch series status > > > > The Rust support is still to be considered experimental. However, > > support is good enough that kernel developers can start working on the > > Rust abstractions for subsystems and write drivers and other modules. > > I'd really like to see this landed for a few reasons: > > - It's under active development, and I'd rather review the changes > "normally", incrementally, etc. Right now it can be hard to re-review > some of the "mostly the same each version" patches in the series. > > - I'd like to break the catch-22 of "ask for a new driver to be > written in rust but the rust support isn't landed" vs "the rust > support isn't landed because there aren't enough drivers". It > really feels like "release early, release often" is needed here; > it's hard to develop against -next. :) +1 to both points. :-) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v6 00/23] Rust support 2022-05-07 5:23 [PATCH v6 00/23] Rust support Miguel Ojeda 2022-05-07 5:24 ` [PATCH v6 22/23] [RFC] drivers: gpio: PrimeCell PL061 in Rust Miguel Ojeda 2022-05-07 8:06 ` [PATCH v6 00/23] Rust support Kees Cook @ 2022-05-07 9:29 ` David Gow 2022-05-07 15:03 ` Miguel Ojeda 2 siblings, 1 reply; 9+ messages in thread From: David Gow @ 2022-05-07 9:29 UTC (permalink / raw) To: Miguel Ojeda Cc: Linus Torvalds, Greg Kroah-Hartman, rust-for-linux, Linux Kernel Mailing List, Jarkko Sakkinen, KUnit Development, Linux ARM, open list:DOCUMENTATION, linux-gpio, linux-kbuild, open list:KERNEL SELFTEST FRAMEWORK, linux-perf-users, linuxppc-dev, linux-riscv, live-patching On Sat, May 7, 2022 at 1:25 PM Miguel Ojeda <ojeda@kernel.org> wrote: > > Rust support > <...> > - Support running documentation tests in-kernel, based on KUnit. > > Rust documentation tests are typically examples of usage of any > item (e.g. function, struct, module...). They are very convenient > because they are just written alongside the documentation, e.g.: > > /// Sums two numbers. > /// > /// # Examples > /// > /// ``` > /// assert_eq!(mymod::f(10, 20), 30); > /// ``` > pub fn f(a: i32, b: i32) -> i32 { > a + b > } > > So far, we were compiling and running them in the host as any > other Rust documentation test. However, that meant we could not > run tests that used kernel APIs (though we were compile-testing > them, which was already useful to keep the documentation in sync > with the code). > > Now, the documentation tests for the `kernel` crate are > transformed into a KUnit test suite during compilation and run > within the kernel at boot time, if enabled. This means now we can > run the tests that use kernel APIs. > > They look like this (their name is generated by `rustdoc`, based > on the file and line): > > [ 0.581961] TAP version 14 > [ 0.582092] 1..1 > [ 0.582267] # Subtest: rust_kernel_doctests > [ 0.582358] 1..70 > [ 0.583626] ok 1 - rust_kernel_doctest_build_assert_rs_12_0 > [ 0.584579] ok 2 - rust_kernel_doctest_build_assert_rs_55_0 > [ 0.587357] ok 3 - rust_kernel_doctest_device_rs_361_0 > [ 0.588037] ok 4 - rust_kernel_doctest_device_rs_386_0 > > ... > > [ 0.659249] ok 69 - rust_kernel_doctest_types_rs_445_0 > [ 0.660451] ok 70 - rust_kernel_doctest_types_rs_509_0 > [ 0.660680] # rust_kernel_doctests: pass:70 fail:0 skip:0 total:70 > [ 0.660894] # Totals: pass:70 fail:0 skip:0 total:70 > [ 0.661135] ok 1 - rust_kernel_doctests > > There are other benefits from this, such as being able to remove > unneeded wrapper functions (that were used to avoid running > some tests) as well as ensuring test code would actually compile > within the kernel (e.g. `alloc` used different `cfg`s). It's great to see some KUnit support here! It's also possible to run these tests using the KUnit wrapper tool with: $ ./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_RUST=y --make_options LLVM=1 --arch x86_64 'rust_kernel_doctests' That also nicely formats the results. (It obviously doesn't run under UML yet, though I did get it to work after indiscriminately hacking out everything that wasn't supported. Assuming we can hide the irq and iomem stuff behind the appropriate config options, and rework some of the architecture detection to either support SUBARCH or check for X86_64 instead of X86, it should be pretty easy to get going.) That all being said, I can't say I'm thrilled with the test names here: none of them are particularly descriptive, and they'll probably not be static (which would make it difficult to track results / regressions / etc between kernel versions). Neither of those are necessarily deal breakers, though it might make sense to hide them behind a kernel option (like all other KUnit tests) so that they can easily be excluded where they would otherwise clutter up results. (And if there's a way to properly name them, or maybe even split them into per-file or per-module suites, that would make them a bit easier to deal.) Additionally, there are some plans to taint the kernel[1] when KUnit tests run, so having a way to turn them off would be very useful. Regardless, this is very neat, and I'm looking forward to taking a closer look at it. Cheers, -- David [1]: https://lore.kernel.org/linux-kselftest/20220429043913.626647-1-davidgow@google.com/ ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v6 00/23] Rust support 2022-05-07 9:29 ` David Gow @ 2022-05-07 15:03 ` Miguel Ojeda 2022-05-10 4:44 ` David Gow 0 siblings, 1 reply; 9+ messages in thread From: Miguel Ojeda @ 2022-05-07 15:03 UTC (permalink / raw) To: David Gow Cc: Miguel Ojeda, Linus Torvalds, Greg Kroah-Hartman, rust-for-linux, Linux Kernel Mailing List, Jarkko Sakkinen, KUnit Development, Linux ARM, open list:DOCUMENTATION, open list:GPIO SUBSYSTEM, Linux Kbuild mailing list, open list:KERNEL SELFTEST FRAMEWORK, linux-perf-users, linuxppc-dev, linux-riscv, live-patching Hi David, On Sat, May 7, 2022 at 11:29 AM David Gow <davidgow@google.com> wrote: > > It's great to see some KUnit support here! Thanks! > It's also possible to run these tests using the KUnit wrapper tool with: > $ ./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_RUST=y > --make_options LLVM=1 --arch x86_64 'rust_kernel_doctests' > > That also nicely formats the results. Indeed! [16:55:52] ============ rust_kernel_doctests (70 subtests) ============ [16:55:52] [PASSED] rust_kernel_doctest_build_assert_rs_12_0 [16:55:52] [PASSED] rust_kernel_doctest_build_assert_rs_55_0 ... [16:55:52] [PASSED] rust_kernel_doctest_types_rs_445_0 [16:55:52] [PASSED] rust_kernel_doctest_types_rs_509_0 [16:55:52] ============== [PASSED] rust_kernel_doctests =============== [16:55:52] ============================================================ [16:55:52] Testing complete. Passed: 70, Failed: 0, Crashed: 0, Skipped: 0, Errors: 0 > That all being said, I can't say I'm thrilled with the test names > here: none of them are particularly descriptive, and they'll probably > not be static (which would make it difficult to track results / > regressions / etc between kernel versions). Neither of those are Yeah, the names are not great and would change from time to time across kernel versions. We could ask example writers to give each example a name, but that would make them fairly less convenient. For instance, sometimes they may be very small snippets interleaved with docs' prose (where giving a name may feel a bit of a burden, and people may end writing `foo_example1`, `foo_example2` etc. for each small "step" of an explanation). In other cases they may be very long, testing a wide API surface (e.g. when describing a module or type), where it is also hard to give non-generic names like `rbtree_doctest`. In those kind of cases, I think we would end up with not much better names than automatically generated ones. The other aspect is that, given they are part of the documentation, the prose or how things are explained/split may change, thus the doctests as well. For instance, one may need to split a very long `rbtree_doctest` in pieces, and then the name would need to change anyway. So I think we should avoid asking documentation writers to add a manual name, even if that means a bit ugly test names. Also this way they are consistently named. What do you think? One idea could be giving them a name based on the hash of the content and avoiding the line number, so that there is a higher chance for the name to stay the same even when the file gets modified for other reasons. > necessarily deal breakers, though it might make sense to hide them > behind a kernel option (like all other KUnit tests) so that they can > easily be excluded where they would otherwise clutter up results. (And Currently they are under `CONFIG_RUST_KERNEL_KUNIT_TEST` -- or do you mean something else? > if there's a way to properly name them, or maybe even split them into > per-file or per-module suites, that would make them a bit easier to > deal.) Additionally, there are some plans to taint the kernel[1] when Yeah, splitting them further is definitely possible. We are also likely splitting the `kernel` crate into several, which would also make the suites smaller etc. so perhaps further splits may not be needed. > Regardless, this is very neat, and I'm looking forward to taking a > closer look at it. Thanks again for taking a look and playing with it, I am glad you liked it! (even if it is just a first approximation, and only supports the `kernel` crate, etc.). Cheers, Miguel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v6 00/23] Rust support 2022-05-07 15:03 ` Miguel Ojeda @ 2022-05-10 4:44 ` David Gow 2022-05-10 11:36 ` Miguel Ojeda 0 siblings, 1 reply; 9+ messages in thread From: David Gow @ 2022-05-10 4:44 UTC (permalink / raw) To: Miguel Ojeda Cc: Miguel Ojeda, Linus Torvalds, Greg Kroah-Hartman, rust-for-linux, Linux Kernel Mailing List, Jarkko Sakkinen, KUnit Development, Linux ARM, open list:DOCUMENTATION, open list:GPIO SUBSYSTEM, Linux Kbuild mailing list, open list:KERNEL SELFTEST FRAMEWORK, linux-perf-users, linuxppc-dev, linux-riscv, live-patching On Sat, May 7, 2022 at 11:03 PM Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> wrote: > > Hi David, > > On Sat, May 7, 2022 at 11:29 AM David Gow <davidgow@google.com> wrote: > > > > It's great to see some KUnit support here! > > Thanks! > > > It's also possible to run these tests using the KUnit wrapper tool with: > > $ ./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_RUST=y > > --make_options LLVM=1 --arch x86_64 'rust_kernel_doctests' > > > > That also nicely formats the results. > > Indeed! > > [16:55:52] ============ rust_kernel_doctests (70 subtests) ============ > [16:55:52] [PASSED] rust_kernel_doctest_build_assert_rs_12_0 > [16:55:52] [PASSED] rust_kernel_doctest_build_assert_rs_55_0 > ... > [16:55:52] [PASSED] rust_kernel_doctest_types_rs_445_0 > [16:55:52] [PASSED] rust_kernel_doctest_types_rs_509_0 > [16:55:52] ============== [PASSED] rust_kernel_doctests =============== > [16:55:52] ============================================================ > [16:55:52] Testing complete. Passed: 70, Failed: 0, Crashed: 0, > Skipped: 0, Errors: 0 > I've just sent out a pull request to get this working under UML as well, which would simplify running these further: https://github.com/Rust-for-Linux/linux/pull/766 > > That all being said, I can't say I'm thrilled with the test names > > here: none of them are particularly descriptive, and they'll probably > > not be static (which would make it difficult to track results / > > regressions / etc between kernel versions). Neither of those are > > Yeah, the names are not great and would change from time to time > across kernel versions. > > We could ask example writers to give each example a name, but that > would make them fairly less convenient. For instance, sometimes they > may be very small snippets interleaved with docs' prose (where giving > a name may feel a bit of a burden, and people may end writing > `foo_example1`, `foo_example2` etc. for each small "step" of an > explanation). In other cases they may be very long, testing a wide API > surface (e.g. when describing a module or type), where it is also hard > to give non-generic names like `rbtree_doctest`. In those kind of > cases, I think we would end up with not much better names than > automatically generated ones. > > The other aspect is that, given they are part of the documentation, > the prose or how things are explained/split may change, thus the > doctests as well. For instance, one may need to split a very long > `rbtree_doctest` in pieces, and then the name would need to change > anyway. > > So I think we should avoid asking documentation writers to add a > manual name, even if that means a bit ugly test names. Also this way > they are consistently named. What do you think? Yeah, these are all fair points: particularly for small doctests. Maybe having an optional name, which more significant tests could use to override the file:line names? That could be useful for a few of the larger, more often referenced tests. > One idea could be giving them a name based on the hash of the content > and avoiding the line number, so that there is a higher chance for the > name to stay the same even when the file gets modified for other > reasons. Ugh: it's a bit ugly either way. I suspect that file:line is still probably better, if only because we need some way of looking up the test in the code if it fails. I'd hate for people to be randomly hashing bits of just to find out what test is failing. > > necessarily deal breakers, though it might make sense to hide them > > behind a kernel option (like all other KUnit tests) so that they can > > easily be excluded where they would otherwise clutter up results. (And > > Currently they are under `CONFIG_RUST_KERNEL_KUNIT_TEST` -- or do you > mean something else? > Oops: I missed that (one of the issues with testing this on a different machine which had a rust toolchain). Looks good to me. > > if there's a way to properly name them, or maybe even split them into > > per-file or per-module suites, that would make them a bit easier to > > deal.) Additionally, there are some plans to taint the kernel[1] when > > Yeah, splitting them further is definitely possible. We are also > likely splitting the `kernel` crate into several, which would also > make the suites smaller etc. so perhaps further splits may not be > needed. Ah: I didn't realise the plan was always to have crate-specific suites, and possibly to split things up. The KTAP output specification does actually support arbitrary nesting (though KUnit itself doesn't at the moment), which would potentially be an option if (e.g.) providing the complete module nesting made sense. I'm not convinced that'd make things easier to read, though. > > Regardless, this is very neat, and I'm looking forward to taking a > > closer look at it. > > Thanks again for taking a look and playing with it, I am glad you > liked it! (even if it is just a first approximation, and only supports > the `kernel` crate, etc.). > > Cheers, > Miguel Thanks, -- David ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v6 00/23] Rust support 2022-05-10 4:44 ` David Gow @ 2022-05-10 11:36 ` Miguel Ojeda 0 siblings, 0 replies; 9+ messages in thread From: Miguel Ojeda @ 2022-05-10 11:36 UTC (permalink / raw) To: David Gow Cc: Miguel Ojeda, Linus Torvalds, Greg Kroah-Hartman, rust-for-linux, Linux Kernel Mailing List, Jarkko Sakkinen, KUnit Development, Linux ARM, open list:DOCUMENTATION, open list:GPIO SUBSYSTEM, Linux Kbuild mailing list, open list:KERNEL SELFTEST FRAMEWORK, linux-perf-users, linuxppc-dev, linux-riscv, live-patching Hi David, On Tue, May 10, 2022 at 6:45 AM David Gow <davidgow@google.com> wrote: > > I've just sent out a pull request to get this working under UML as > well, which would simplify running these further: > https://github.com/Rust-for-Linux/linux/pull/766 Thanks a lot! > Yeah, these are all fair points: particularly for small doctests. > > Maybe having an optional name, which more significant tests could use > to override the file:line names? That could be useful for a few of the > larger, more often referenced tests. Sounds reasonable. I can add support for that. > Ugh: it's a bit ugly either way. I suspect that file:line is still > probably better, if only because we need some way of looking up the > test in the code if it fails. I'd hate for people to be randomly > hashing bits of just to find out what test is failing. One redeeming quality is that the assertion prints the line/file number in the generated file, so it would still be possible to check where it came from: [13:13:43] # rust_kernel_doctest_str_rs_somehash: ASSERTION FAILED at rust/doctests_kernel_generated.rs:2209 [13:13:43] Expected 2 > 3 to be true, but is false [13:13:43] not ok 43 - rust_kernel_doctest_str_rs_somehash [13:13:43] [FAILED] rust_kernel_doctest_str_rs_somehash Another alternative is to keep the file:line information around without embedding it into the test name, e.g. in a TAP comment or a mapping file (which `kunit.py` could read). But, yeah, before doing hashes or things like that, I would just go for simplicity and keep things as they are unless some use case really needs doctests to be stable. > Oops: I missed that (one of the issues with testing this on a > different machine which had a rust toolchain). Looks good to me. > > Ah: I didn't realise the plan was always to have crate-specific > suites, and possibly to split things up. > > The KTAP output specification does actually support arbitrary nesting > (though KUnit itself doesn't at the moment), which would potentially > be an option if (e.g.) providing the complete module nesting made > sense. I'm not convinced that'd make things easier to read, though. That is useful to know in case we need it, thanks! Cheers, Miguel ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2022-05-10 11:36 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-05-07 5:23 [PATCH v6 00/23] Rust support Miguel Ojeda 2022-05-07 5:24 ` [PATCH v6 22/23] [RFC] drivers: gpio: PrimeCell PL061 in Rust Miguel Ojeda 2022-05-07 8:06 ` [PATCH v6 00/23] Rust support Kees Cook 2022-05-08 18:06 ` Matthew Wilcox 2022-05-09 9:39 ` Wei Liu 2022-05-07 9:29 ` David Gow 2022-05-07 15:03 ` Miguel Ojeda 2022-05-10 4:44 ` David Gow 2022-05-10 11:36 ` Miguel Ojeda
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).