[PATCH v6 00/23] Rust support

[PATCH v6 00/23] Rust support

[Miguel Ojeda]

Rust support

This is the patch series (v6) to add support for Rust as a second
language to the Linux kernel.

If you are interested in following this effort, please join us in
the mailing list at:

    rust-for-linux@vger.kernel.org

and take a look at the project itself at:

    https://github.com/Rust-for-Linux

As usual, special thanks go to ISRG (Internet Security Research
Group) and Google for their financial support on this endeavor.

Cheers,
Miguel

--

# Rust support

This cover letter explains the major changes and updates done since
the previous ones. For those, please see:

    RFC: https://lore.kernel.org/lkml/20210414184604.23473-1-ojeda@kernel.org/
    v1:  https://lore.kernel.org/lkml/20210704202756.29107-1-ojeda@kernel.org/
    v2:  https://lore.kernel.org/lkml/20211206140313.5653-1-ojeda@kernel.org/
    v3:  https://lore.kernel.org/lkml/20220117053349.6804-1-ojeda@kernel.org/
    v4:  https://lore.kernel.org/lkml/20220212130410.6901-1-ojeda@kernel.org/
    v5:  https://lore.kernel.org/lkml/20220317181032.15436-1-ojeda@kernel.org/


## Infrastructure updates

There have been several improvements to the overall Rust support:

  - The toolchain and `alloc` have been upgraded to Rust 1.60.0.
    This version stabilized `feature(maybe_uninit_extra)` that we
    are using.

  - Support running documentation tests in-kernel, based on KUnit.

    Rust documentation tests are typically examples of usage of any
    item (e.g. function, struct, module...). They are very convenient
    because they are just written alongside the documentation, e.g.:

        /// Sums two numbers.
        ///
        /// # Examples
        ///
        /// ```
        /// assert_eq!(mymod::f(10, 20), 30);
        /// ```
        pub fn f(a: i32, b: i32) -> i32 {
            a + b
        }

    So far, we were compiling and running them in the host as any
    other Rust documentation test. However, that meant we could not
    run tests that used kernel APIs (though we were compile-testing
    them, which was already useful to keep the documentation in sync
    with the code).

    Now, the documentation tests for the `kernel` crate are
    transformed into a KUnit test suite during compilation and run
    within the kernel at boot time, if enabled. This means now we can
    run the tests that use kernel APIs.

    They look like this (their name is generated by `rustdoc`, based
    on the file and line):

        [    0.581961] TAP version 14
        [    0.582092] 1..1
        [    0.582267]     # Subtest: rust_kernel_doctests
        [    0.582358]     1..70
        [    0.583626]     ok 1 - rust_kernel_doctest_build_assert_rs_12_0
        [    0.584579]     ok 2 - rust_kernel_doctest_build_assert_rs_55_0
        [    0.587357]     ok 3 - rust_kernel_doctest_device_rs_361_0
        [    0.588037]     ok 4 - rust_kernel_doctest_device_rs_386_0

        ...

        [    0.659249]     ok 69 - rust_kernel_doctest_types_rs_445_0
        [    0.660451]     ok 70 - rust_kernel_doctest_types_rs_509_0
        [    0.660680] # rust_kernel_doctests: pass:70 fail:0 skip:0 total:70
        [    0.660894] # Totals: pass:70 fail:0 skip:0 total:70
        [    0.661135] ok 1 - rust_kernel_doctests

    There are other benefits from this, such as being able to remove
    unneeded wrapper functions (that were used to avoid running
    some tests) as well as ensuring test code would actually compile
    within the kernel (e.g. `alloc` used different `cfg`s).

  - Tests are now (and are enforced to be) Clippy-clean, like the rest
    of the Rust kernel code (i.e. according to the same rules).

  - Other cleanups, fixes and improvements.


## Abstractions and driver updates

Some of the improvements to the abstractions and example drivers are:

  - The start of networking support (`net` module), with types like:

    + `Namespace` (based on `struct net`).
    + `SkBuff` (based on `struct sk_buff`).
    + `Ipv4Addr` (based on `struct in_addr`), and its v6 equivalent.
    + `SocketAddrV4` (based on `struct sockaddr_in`), and its v6
      equivalent.
    + `TcpListener` and `TcpStream` (based on `struct socket`).

  - The beginning of `async` support (`kasync` module).

    Rust provides support for asynchronous programming in a way that
    can be used in constrained environments, including the kernel.

    For instance, this allows us to write asynchronous TCP socket code
    within the kernel such as:

        async fn echo_server(stream: TcpStream) -> Result {
            let mut buf = [0u8; 1024];
            loop {
                let n = stream.read(&mut buf).await?;
                if n == 0 {
                    return Ok(());
                }
                stream.write_all(&buf[..n]).await?;
           }
        }

    This code looks very close to a synchronous version, yet it
    supports being driven to completion "step by step" by an executor.
    The `read()`/`write_all()` calls above, instead of blocking the
    current thread, return a future which can be polled. The `.await`
    points poll the future and, if the result is not ready, suspend
    the state such that execution resumes there later on (the state
    machine needed for this gets implemented by the compiler). This
    allows an executor to drive multiple futures to completion
    concurrently on the same thread.

    An executor is not included yet, but `kasync` includes async
    versions of `TcpListener` and `TcpStream` (based on the non-async
    ones) which employ `SocketFuture` (which in turn uses a
    `struct wait_queue_entry`).

  - Support for network packet filters (`net::filter` module) and its
    related `rust_netfilter.rs` sample.

  - Added `smutex::Mutex`: a simple mutex that does not require
    pinning, so that the ergonomics are much improved, though the
    implementation is not as feature-rich as the C-based one.

  - New `NoWaitLock`: one that never waits, that is, if it is owned
    by another thread/CPU, then attempts to acquire it will fail
    (instead of, for example, blocking the caller).

  - Added `RawSpinLock` (backed by `raw_spinlock_t`), used when code
    sections cannot sleep even in real-time variants of the kernel.

  - Added `ARef`, an owned reference to an always-refcounted object,
    meant to simplify how we define wrappers to types defined on the
    C side of the source code.

  - Other cleanups, fixes and improvements.


## Patch series status

The Rust support is still to be considered experimental. However,
support is good enough that kernel developers can start working on the
Rust abstractions for subsystems and write drivers and other modules.

The current series has just arrived in `linux-next`, as usual.
Similarly, the preview docs for this series can be seen at:

    https://rust-for-linux.github.io/docs/kernel/

As usual, please see the following link for the live list of unstable
Rust features we are using:

    https://github.com/Rust-for-Linux/linux/issues/2


## Conferences, meetings and liaisons

We would like to announce the Rust MC (microconference) in
the upcoming LPC 2022 (Linux Plumbers Conference):

    https://lpc.events/event/16/contributions/1159/

The Rust MC intends to cover talks and discussions on both Rust for
Linux as well as other non-kernel Rust topics. The Call for Proposals
is open!

Furthermore, we would like to thank you the venues we were invited to:

  - Rust Linz 2022
  - Linux Foundation Live Mentorship Series


## Related news

`rustc_codegen_gcc` (the GCC backend for `rustc`) can now bootstrap
`rustc`! In addition, GCC 12.1 (just released) carries some of the
patches that were needed by the project in upstream `libgccjit`; and
the project is looking into getting distributed with `rustup`.

`gccrs` (the Rust frontend for GCC) has got a second full time
developer working on it, Arthur Cohen, as well as a lot of technical
progress too, such as a new testing project, slice generation support
and macro-related work.


## Acknowledgements

The signatures in the main commits correspond to the people that
wrote code that has ended up in them at the present time. For details
on contributions to code and discussions, please see our repository:

    https://github.com/Rust-for-Linux/linux

However, we would like to give credit to everyone that has contributed
in one way or another to the Rust for Linux project. Since the
previous cover letter:

  - Andy Shevchenko, Petr Mladek, Sergey Senozhatsky for their review
    of the `vsprintf` patch.

  - Arnaldo Carvalho de Melo and Andrii Nakryiko for their input on
    `pahole` and BTF, Arnaldo for adding support `pahole` for `--lang`
    and `--lang_exclude` (useful to skip Rust CUs) and Martin Reboredo
    for reporting the `CONFIG_DEBUG_INFO_BTF` issue.

  - Daniel Latypov, David Gow and Brendan Higgins for their input
    on KUnit and their reviews on a prerequisite Rust patch on it.

  - Kees Cook for reviewing the kallsyms prerequisite patches.

  - Greg Kroah-Hartman for his suggestions on the `alloc` patch.

  - Daniel Paoliello for his ongoing work on adding more `try_*`
    methods to `Vec` in the standard library. Currently, we have some
    similar methods in our custom `alloc` that we could drop once
    equivalents arrive upstream. Also John Ericson for his reviews.

  - bjorn3 for reviewing many PRs and the input around potential UB
    in doctests.

  - As usual, bjorn3 and Gary Guo for all the input on Rust compiler
    details and suggestions.

  - Adam Bratschi-Kaye for working on `seq_file` and `debugfs`
    abstractions.

  - Maciej Falkowski for continuing his work on the Samsung Exynos
    TRNG driver and the required abstractions around it, such as
    adding `delay`, `ktime` and `iopoll` abstractions, new methods
    to `platform::Device` and run-time power management abstractions.

  - Daniel Xu for working on adding a Rust allocator based on the
    `kvmalloc` family of functions.

  - Hongyu Li for working on Rust iterators as the equivalent of
    `cpumask`'s `for_each_*_cpu`.

  - Andreas Hindborg for adding support to `kernel::Pages` methods to
    allow read/write of multiple pages.

  - Sergio González Collado for working on adding `#[cold]` attributes
    for error-related items and GitHub CI problem matchers.

  - Sean Nash for updating the out-of-tree-module example due to a
    change in the main repository.

  - Michael Ellerman, Nicholas Piggin, Paul E. McKenney and Zhouyi
    Zhou for debugging the `CONFIG_HIGH_RES_TIMERS=n` stall issue
    in PowerPC that we triggered in our CI.

  - Jonathan Corbet for writing an LWN article on the crates
    discussion that took place in the Rust for Linux mailing list.

  - Wei Liu for taking the time to answer questions from newcomers
    in Zulip.

  - Philip Li, Yujie Liu et al. for continuing their work on adding
    Rust support to the Intel 0DAY/LKP kernel test robot.

  - Philip Herron and Arthur Cohen (and his supporters Open Source
    Security and Embecosm) et al. for their ongoing work on GCC Rust.

  - Antoni Boucher (and his supporters) et al. for their ongoing
    work on `rustc_codegen_gcc`.

  - Mats Larsen, Marc Poulhiès et al. for their ongoing work on
    improving Rust support in Compiler Explorer.

  - Many folks that have reported issues, tested the project,
    helped spread the word, joined discussions and contributed in
    other ways!

Please see also the acknowledgements on the previous cover letters.


Boqun Feng (1):
  kallsyms: avoid hardcoding the buffer size

Gary Guo (2):
  rust: add `build_error` crate
  vsprintf: add new `%pA` format specifier

Miguel Ojeda (16):
  kallsyms: support "big" kernel symbols
  kallsyms: increase maximum kernel symbol length to 512
  kunit: take `kunit_assert` as `const`
  rust: add C helpers
  rust: add `compiler_builtins` crate
  rust: import upstream `alloc` crate
  rust: adapt `alloc` crate to the kernel
  rust: add `macros` crate
  rust: export generated symbols
  scripts: add `rustdoc_test_{builder,gen}.py` scripts
  scripts: add `generate_rust_analyzer.py` scripts
  scripts: decode_stacktrace: demangle Rust symbols
  docs: add Rust documentation
  Kbuild: add Rust support
  samples: add Rust examples
  MAINTAINERS: Rust

Wedson Almeida Filho (4):
  rust: add `kernel` crate's `sync` module
  rust: add `kernel` crate
  [RFC] drivers: gpio: PrimeCell PL061 in Rust
  [RFC] drivers: android: Binder IPC in Rust

 .gitignore                                   |    5 +
 .rustfmt.toml                                |   12 +
 Documentation/doc-guide/kernel-doc.rst       |    3 +
 Documentation/index.rst                      |    1 +
 Documentation/kbuild/kbuild.rst              |   17 +
 Documentation/kbuild/makefiles.rst           |   50 +-
 Documentation/process/changes.rst            |   41 +
 Documentation/rust/arch-support.rst          |   34 +
 Documentation/rust/coding-guidelines.rst     |  214 ++
 Documentation/rust/general-information.rst   |   77 +
 Documentation/rust/index.rst                 |   20 +
 Documentation/rust/logo.svg                  |  357 ++
 Documentation/rust/quick-start.rst           |  230 ++
 MAINTAINERS                                  |   15 +
 Makefile                                     |  175 +-
 arch/Kconfig                                 |    6 +
 arch/arm/Kconfig                             |    1 +
 arch/arm64/Kconfig                           |    1 +
 arch/powerpc/Kconfig                         |    1 +
 arch/riscv/Kconfig                           |    1 +
 arch/riscv/Makefile                          |    5 +
 arch/x86/Kconfig                             |    1 +
 arch/x86/Makefile                            |   14 +
 drivers/android/Kconfig                      |    6 +
 drivers/android/Makefile                     |    2 +
 drivers/android/allocation.rs                |  266 ++
 drivers/android/context.rs                   |   80 +
 drivers/android/defs.rs                      |   99 +
 drivers/android/node.rs                      |  476 +++
 drivers/android/process.rs                   |  960 +++++
 drivers/android/range_alloc.rs               |  189 +
 drivers/android/rust_binder.rs               |  111 +
 drivers/android/thread.rs                    |  870 +++++
 drivers/android/transaction.rs               |  326 ++
 drivers/gpio/Kconfig                         |    8 +
 drivers/gpio/Makefile                        |    1 +
 drivers/gpio/gpio_pl061_rust.rs              |  370 ++
 include/kunit/test.h                         |    2 +-
 include/linux/kallsyms.h                     |    2 +-
 include/linux/spinlock.h                     |   25 +-
 include/uapi/linux/android/binder.h          |   28 +-
 init/Kconfig                                 |   45 +-
 kernel/kallsyms.c                            |   26 +-
 kernel/livepatch/core.c                      |    4 +-
 lib/Kconfig.debug                            |  155 +
 lib/kunit/test.c                             |    4 +-
 lib/vsprintf.c                               |   13 +
 rust/.gitignore                              |   10 +
 rust/Makefile                                |  397 +++
 rust/alloc/README.md                         |   32 +
 rust/alloc/alloc.rs                          |  438 +++
 rust/alloc/borrow.rs                         |  498 +++
 rust/alloc/boxed.rs                          | 2007 +++++++++++
 rust/alloc/collections/mod.rs                |  156 +
 rust/alloc/fmt.rs                            |  601 ++++
 rust/alloc/lib.rs                            |  226 ++
 rust/alloc/macros.rs                         |  127 +
 rust/alloc/raw_vec.rs                        |  567 +++
 rust/alloc/slice.rs                          | 1282 +++++++
 rust/alloc/str.rs                            |  632 ++++
 rust/alloc/string.rs                         | 2869 +++++++++++++++
 rust/alloc/vec/drain.rs                      |  186 +
 rust/alloc/vec/drain_filter.rs               |  145 +
 rust/alloc/vec/into_iter.rs                  |  356 ++
 rust/alloc/vec/is_zero.rs                    |  106 +
 rust/alloc/vec/mod.rs                        | 3362 ++++++++++++++++++
 rust/alloc/vec/partial_eq.rs                 |   49 +
 rust/alloc/vec/set_len_on_drop.rs            |   30 +
 rust/alloc/vec/spec_extend.rs                |  174 +
 rust/bindgen_parameters                      |   17 +
 rust/build_error.rs                          |   29 +
 rust/compiler_builtins.rs                    |   57 +
 rust/exports.c                               |   20 +
 rust/helpers.c                               |  639 ++++
 rust/kernel/allocator.rs                     |   65 +
 rust/kernel/amba.rs                          |  257 ++
 rust/kernel/bindings.rs                      |   47 +
 rust/kernel/bindings_helper.h                |   46 +
 rust/kernel/build_assert.rs                  |   82 +
 rust/kernel/c_types.rs                       |  119 +
 rust/kernel/chrdev.rs                        |  207 ++
 rust/kernel/clk.rs                           |   79 +
 rust/kernel/cred.rs                          |   46 +
 rust/kernel/device.rs                        |  546 +++
 rust/kernel/driver.rs                        |  442 +++
 rust/kernel/error.rs                         |  565 +++
 rust/kernel/file.rs                          |  860 +++++
 rust/kernel/gpio.rs                          |  478 +++
 rust/kernel/hwrng.rs                         |  242 ++
 rust/kernel/io_buffer.rs                     |  153 +
 rust/kernel/io_mem.rs                        |  275 ++
 rust/kernel/iov_iter.rs                      |   81 +
 rust/kernel/irq.rs                           |  409 +++
 rust/kernel/kasync.rs                        |    6 +
 rust/kernel/kasync/net.rs                    |  322 ++
 rust/kernel/kunit.rs                         |   91 +
 rust/kernel/lib.rs                           |  260 ++
 rust/kernel/linked_list.rs                   |  247 ++
 rust/kernel/miscdev.rs                       |  291 ++
 rust/kernel/mm.rs                            |  149 +
 rust/kernel/module_param.rs                  |  498 +++
 rust/kernel/net.rs                           |  392 ++
 rust/kernel/net/filter.rs                    |  447 +++
 rust/kernel/of.rs                            |   63 +
 rust/kernel/pages.rs                         |  144 +
 rust/kernel/platform.rs                      |  223 ++
 rust/kernel/power.rs                         |  118 +
 rust/kernel/prelude.rs                       |   36 +
 rust/kernel/print.rs                         |  405 +++
 rust/kernel/random.rs                        |   42 +
 rust/kernel/raw_list.rs                      |  361 ++
 rust/kernel/rbtree.rs                        |  563 +++
 rust/kernel/revocable.rs                     |  161 +
 rust/kernel/security.rs                      |   38 +
 rust/kernel/static_assert.rs                 |   38 +
 rust/kernel/std_vendor.rs                    |  160 +
 rust/kernel/str.rs                           |  597 ++++
 rust/kernel/sync.rs                          |  161 +
 rust/kernel/sync/arc.rs                      |  503 +++
 rust/kernel/sync/condvar.rs                  |  138 +
 rust/kernel/sync/guard.rs                    |  169 +
 rust/kernel/sync/locked_by.rs                |  111 +
 rust/kernel/sync/mutex.rs                    |  153 +
 rust/kernel/sync/nowait.rs                   |  188 +
 rust/kernel/sync/revocable.rs                |  250 ++
 rust/kernel/sync/rwsem.rs                    |  197 +
 rust/kernel/sync/seqlock.rs                  |  202 ++
 rust/kernel/sync/smutex.rs                   |  295 ++
 rust/kernel/sync/spinlock.rs                 |  360 ++
 rust/kernel/sysctl.rs                        |  199 ++
 rust/kernel/task.rs                          |  175 +
 rust/kernel/types.rs                         |  679 ++++
 rust/kernel/user_ptr.rs                      |  175 +
 rust/macros/helpers.rs                       |   79 +
 rust/macros/lib.rs                           |   94 +
 rust/macros/module.rs                        |  631 ++++
 samples/Kconfig                              |    2 +
 samples/Makefile                             |    1 +
 samples/rust/Kconfig                         |  140 +
 samples/rust/Makefile                        |   16 +
 samples/rust/hostprogs/.gitignore            |    3 +
 samples/rust/hostprogs/Makefile              |    5 +
 samples/rust/hostprogs/a.rs                  |    7 +
 samples/rust/hostprogs/b.rs                  |    5 +
 samples/rust/hostprogs/single.rs             |   12 +
 samples/rust/rust_chrdev.rs                  |   50 +
 samples/rust/rust_minimal.rs                 |   35 +
 samples/rust/rust_miscdev.rs                 |  143 +
 samples/rust/rust_module_parameters.rs       |   69 +
 samples/rust/rust_netfilter.rs               |   54 +
 samples/rust/rust_platform.rs                |   22 +
 samples/rust/rust_print.rs                   |   54 +
 samples/rust/rust_random.rs                  |   60 +
 samples/rust/rust_semaphore.rs               |  171 +
 samples/rust/rust_semaphore_c.c              |  212 ++
 samples/rust/rust_stack_probing.rs           |   36 +
 samples/rust/rust_sync.rs                    |   93 +
 scripts/.gitignore                           |    1 +
 scripts/Kconfig.include                      |    6 +-
 scripts/Makefile                             |    3 +
 scripts/Makefile.build                       |   60 +
 scripts/Makefile.debug                       |   10 +
 scripts/Makefile.host                        |   34 +-
 scripts/Makefile.lib                         |   12 +
 scripts/Makefile.modfinal                    |    8 +-
 scripts/cc-version.sh                        |   12 +-
 scripts/decode_stacktrace.sh                 |   14 +
 scripts/generate_rust_analyzer.py            |  134 +
 scripts/generate_rust_target.rs              |  227 ++
 scripts/is_rust_module.sh                    |   13 +
 scripts/kallsyms.c                           |   47 +-
 scripts/kconfig/confdata.c                   |   75 +
 scripts/min-tool-version.sh                  |    6 +
 scripts/rust-is-available-bindgen-libclang.h |    2 +
 scripts/rust-is-available.sh                 |  158 +
 scripts/rustdoc_test_builder.py              |   59 +
 scripts/rustdoc_test_gen.py                  |  164 +
 tools/include/linux/kallsyms.h               |    2 +-
 tools/lib/perf/include/perf/event.h          |    2 +-
 tools/lib/symbol/kallsyms.h                  |    2 +-
 180 files changed, 37945 insertions(+), 67 deletions(-)
 create mode 100644 .rustfmt.toml
 create mode 100644 Documentation/rust/arch-support.rst
 create mode 100644 Documentation/rust/coding-guidelines.rst
 create mode 100644 Documentation/rust/general-information.rst
 create mode 100644 Documentation/rust/index.rst
 create mode 100644 Documentation/rust/logo.svg
 create mode 100644 Documentation/rust/quick-start.rst
 create mode 100644 drivers/android/allocation.rs
 create mode 100644 drivers/android/context.rs
 create mode 100644 drivers/android/defs.rs
 create mode 100644 drivers/android/node.rs
 create mode 100644 drivers/android/process.rs
 create mode 100644 drivers/android/range_alloc.rs
 create mode 100644 drivers/android/rust_binder.rs
 create mode 100644 drivers/android/thread.rs
 create mode 100644 drivers/android/transaction.rs
 create mode 100644 drivers/gpio/gpio_pl061_rust.rs
 create mode 100644 rust/.gitignore
 create mode 100644 rust/Makefile
 create mode 100644 rust/alloc/README.md
 create mode 100644 rust/alloc/alloc.rs
 create mode 100644 rust/alloc/borrow.rs
 create mode 100644 rust/alloc/boxed.rs
 create mode 100644 rust/alloc/collections/mod.rs
 create mode 100644 rust/alloc/fmt.rs
 create mode 100644 rust/alloc/lib.rs
 create mode 100644 rust/alloc/macros.rs
 create mode 100644 rust/alloc/raw_vec.rs
 create mode 100644 rust/alloc/slice.rs
 create mode 100644 rust/alloc/str.rs
 create mode 100644 rust/alloc/string.rs
 create mode 100644 rust/alloc/vec/drain.rs
 create mode 100644 rust/alloc/vec/drain_filter.rs
 create mode 100644 rust/alloc/vec/into_iter.rs
 create mode 100644 rust/alloc/vec/is_zero.rs
 create mode 100644 rust/alloc/vec/mod.rs
 create mode 100644 rust/alloc/vec/partial_eq.rs
 create mode 100644 rust/alloc/vec/set_len_on_drop.rs
 create mode 100644 rust/alloc/vec/spec_extend.rs
 create mode 100644 rust/bindgen_parameters
 create mode 100644 rust/build_error.rs
 create mode 100644 rust/compiler_builtins.rs
 create mode 100644 rust/exports.c
 create mode 100644 rust/helpers.c
 create mode 100644 rust/kernel/allocator.rs
 create mode 100644 rust/kernel/amba.rs
 create mode 100644 rust/kernel/bindings.rs
 create mode 100644 rust/kernel/bindings_helper.h
 create mode 100644 rust/kernel/build_assert.rs
 create mode 100644 rust/kernel/c_types.rs
 create mode 100644 rust/kernel/chrdev.rs
 create mode 100644 rust/kernel/clk.rs
 create mode 100644 rust/kernel/cred.rs
 create mode 100644 rust/kernel/device.rs
 create mode 100644 rust/kernel/driver.rs
 create mode 100644 rust/kernel/error.rs
 create mode 100644 rust/kernel/file.rs
 create mode 100644 rust/kernel/gpio.rs
 create mode 100644 rust/kernel/hwrng.rs
 create mode 100644 rust/kernel/io_buffer.rs
 create mode 100644 rust/kernel/io_mem.rs
 create mode 100644 rust/kernel/iov_iter.rs
 create mode 100644 rust/kernel/irq.rs
 create mode 100644 rust/kernel/kasync.rs
 create mode 100644 rust/kernel/kasync/net.rs
 create mode 100644 rust/kernel/kunit.rs
 create mode 100644 rust/kernel/lib.rs
 create mode 100644 rust/kernel/linked_list.rs
 create mode 100644 rust/kernel/miscdev.rs
 create mode 100644 rust/kernel/mm.rs
 create mode 100644 rust/kernel/module_param.rs
 create mode 100644 rust/kernel/net.rs
 create mode 100644 rust/kernel/net/filter.rs
 create mode 100644 rust/kernel/of.rs
 create mode 100644 rust/kernel/pages.rs
 create mode 100644 rust/kernel/platform.rs
 create mode 100644 rust/kernel/power.rs
 create mode 100644 rust/kernel/prelude.rs
 create mode 100644 rust/kernel/print.rs
 create mode 100644 rust/kernel/random.rs
 create mode 100644 rust/kernel/raw_list.rs
 create mode 100644 rust/kernel/rbtree.rs
 create mode 100644 rust/kernel/revocable.rs
 create mode 100644 rust/kernel/security.rs
 create mode 100644 rust/kernel/static_assert.rs
 create mode 100644 rust/kernel/std_vendor.rs
 create mode 100644 rust/kernel/str.rs
 create mode 100644 rust/kernel/sync.rs
 create mode 100644 rust/kernel/sync/arc.rs
 create mode 100644 rust/kernel/sync/condvar.rs
 create mode 100644 rust/kernel/sync/guard.rs
 create mode 100644 rust/kernel/sync/locked_by.rs
 create mode 100644 rust/kernel/sync/mutex.rs
 create mode 100644 rust/kernel/sync/nowait.rs
 create mode 100644 rust/kernel/sync/revocable.rs
 create mode 100644 rust/kernel/sync/rwsem.rs
 create mode 100644 rust/kernel/sync/seqlock.rs
 create mode 100644 rust/kernel/sync/smutex.rs
 create mode 100644 rust/kernel/sync/spinlock.rs
 create mode 100644 rust/kernel/sysctl.rs
 create mode 100644 rust/kernel/task.rs
 create mode 100644 rust/kernel/types.rs
 create mode 100644 rust/kernel/user_ptr.rs
 create mode 100644 rust/macros/helpers.rs
 create mode 100644 rust/macros/lib.rs
 create mode 100644 rust/macros/module.rs
 create mode 100644 samples/rust/Kconfig
 create mode 100644 samples/rust/Makefile
 create mode 100644 samples/rust/hostprogs/.gitignore
 create mode 100644 samples/rust/hostprogs/Makefile
 create mode 100644 samples/rust/hostprogs/a.rs
 create mode 100644 samples/rust/hostprogs/b.rs
 create mode 100644 samples/rust/hostprogs/single.rs
 create mode 100644 samples/rust/rust_chrdev.rs
 create mode 100644 samples/rust/rust_minimal.rs
 create mode 100644 samples/rust/rust_miscdev.rs
 create mode 100644 samples/rust/rust_module_parameters.rs
 create mode 100644 samples/rust/rust_netfilter.rs
 create mode 100644 samples/rust/rust_platform.rs
 create mode 100644 samples/rust/rust_print.rs
 create mode 100644 samples/rust/rust_random.rs
 create mode 100644 samples/rust/rust_semaphore.rs
 create mode 100644 samples/rust/rust_semaphore_c.c
 create mode 100644 samples/rust/rust_stack_probing.rs
 create mode 100644 samples/rust/rust_sync.rs
 create mode 100755 scripts/generate_rust_analyzer.py
 create mode 100644 scripts/generate_rust_target.rs
 create mode 100755 scripts/is_rust_module.sh
 create mode 100644 scripts/rust-is-available-bindgen-libclang.h
 create mode 100755 scripts/rust-is-available.sh
 create mode 100755 scripts/rustdoc_test_builder.py
 create mode 100755 scripts/rustdoc_test_gen.py


base-commit: 672c0c5173427e6b3e2a9bbb7be51ceeec78093a
-- 
2.35.3

Re: [PATCH v6 00/23] Rust support

[Kees Cook]

On Sat, May 07, 2022 at 07:23:58AM +0200, Miguel Ojeda wrote:
> ## Patch series status
> 
> The Rust support is still to be considered experimental. However,
> support is good enough that kernel developers can start working on the
> Rust abstractions for subsystems and write drivers and other modules.

I'd really like to see this landed for a few reasons:

- It's under active development, and I'd rather review the changes
  "normally", incrementally, etc. Right now it can be hard to re-review
  some of the "mostly the same each version" patches in the series.

- I'd like to break the catch-22 of "ask for a new driver to be
  written in rust but the rust support isn't landed" vs "the rust
  support isn't landed because there aren't enough drivers". It
  really feels like "release early, release often" is needed here;
  it's hard to develop against -next. :)

Should we give it a try for this coming merge window?

-- 
Kees Cook

Re: [PATCH v6 18/23] docs: add Rust documentation

[Kees Cook]

On Sat, May 07, 2022 at 07:24:16AM +0200, Miguel Ojeda wrote:
> Most of the documentation for Rust is written within the source code
> itself, as it is idiomatic for Rust projects. This applies to both
> the shared infrastructure at `rust/` as well as any other Rust module
> (e.g. drivers) written across the kernel.
> 
> However, these documents contain general information that does not
> fit particularly well in the source code, like the Quick Start guide.
> 
> It also contains an asset (SVG logo) used for the `rustdoc` target
> and a few other small changes elsewhere in the documentation folder.

I like the docs! :) It'll be interesting to see how we can grow
cross-linking between rustdoc and kerndoc.

Reviewed-by: Kees Cook <keescook@chromium.org>

-- 
Kees Cook

Re: [PATCH v6 18/23] docs: add Rust documentation

[Miguel Ojeda]

On Sat, May 7, 2022 at 10:15 AM Kees Cook <keescook@chromium.org> wrote:
>
> I like the docs! :) It'll be interesting to see how we can grow
> cross-linking between rustdoc and kerndoc.

Thanks!

For cross-referencing, my plan is to start on the Rust side by
allowing to link C files and items (either their generated docs or the
source file), since we do that often in the Rust docs. The goal is
that one can just write an "intra-doc link" in Rust docs just like any
other one, e.g.:

    /// Wraps the kernel's [`struct sk_buff`].
    #[repr(transparent)]
    pub struct SkBuff(UnsafeCell<bindings::sk_buff>);

so that it is as easy as possible to add them.

Cheers,
Miguel

Re: [PATCH v6 00/23] Rust support

[David Gow]

On Sat, May 7, 2022 at 1:25 PM Miguel Ojeda <ojeda@kernel.org> wrote:
>
> Rust support
>

<...>

>   - Support running documentation tests in-kernel, based on KUnit.
>
>     Rust documentation tests are typically examples of usage of any
>     item (e.g. function, struct, module...). They are very convenient
>     because they are just written alongside the documentation, e.g.:
>
>         /// Sums two numbers.
>         ///
>         /// # Examples
>         ///
>         /// ```
>         /// assert_eq!(mymod::f(10, 20), 30);
>         /// ```
>         pub fn f(a: i32, b: i32) -> i32 {
>             a + b
>         }
>
>     So far, we were compiling and running them in the host as any
>     other Rust documentation test. However, that meant we could not
>     run tests that used kernel APIs (though we were compile-testing
>     them, which was already useful to keep the documentation in sync
>     with the code).
>
>     Now, the documentation tests for the `kernel` crate are
>     transformed into a KUnit test suite during compilation and run
>     within the kernel at boot time, if enabled. This means now we can
>     run the tests that use kernel APIs.
>
>     They look like this (their name is generated by `rustdoc`, based
>     on the file and line):
>
>         [    0.581961] TAP version 14
>         [    0.582092] 1..1
>         [    0.582267]     # Subtest: rust_kernel_doctests
>         [    0.582358]     1..70
>         [    0.583626]     ok 1 - rust_kernel_doctest_build_assert_rs_12_0
>         [    0.584579]     ok 2 - rust_kernel_doctest_build_assert_rs_55_0
>         [    0.587357]     ok 3 - rust_kernel_doctest_device_rs_361_0
>         [    0.588037]     ok 4 - rust_kernel_doctest_device_rs_386_0
>
>         ...
>
>         [    0.659249]     ok 69 - rust_kernel_doctest_types_rs_445_0
>         [    0.660451]     ok 70 - rust_kernel_doctest_types_rs_509_0
>         [    0.660680] # rust_kernel_doctests: pass:70 fail:0 skip:0 total:70
>         [    0.660894] # Totals: pass:70 fail:0 skip:0 total:70
>         [    0.661135] ok 1 - rust_kernel_doctests
>
>     There are other benefits from this, such as being able to remove
>     unneeded wrapper functions (that were used to avoid running
>     some tests) as well as ensuring test code would actually compile
>     within the kernel (e.g. `alloc` used different `cfg`s).

It's great to see some KUnit support here!

It's also possible to run these tests using the KUnit wrapper tool with:
$ ./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_RUST=y
--make_options LLVM=1 --arch x86_64 'rust_kernel_doctests'

That also nicely formats the results.

(It obviously doesn't run under UML yet, though I did get it to work
after indiscriminately hacking out everything that wasn't supported.
Assuming we can hide the irq and iomem stuff behind the appropriate
config options, and rework some of the architecture detection to
either support SUBARCH or check for X86_64 instead of X86, it should
be pretty easy to get going.)

That all being said, I can't say I'm thrilled with the test names
here: none of them are particularly descriptive, and they'll probably
not be static (which would make it difficult to track results /
regressions / etc between kernel versions). Neither of those are
necessarily deal breakers, though it might make sense to hide them
behind a kernel option (like all other KUnit tests) so that they can
easily be excluded where they would otherwise clutter up results. (And
if there's a way to properly name them, or maybe even split them into
per-file or per-module suites, that would make them a bit easier to
deal.) Additionally, there are some plans to taint the kernel[1] when
KUnit tests run, so having a way to turn them off would be very
useful.

Regardless, this is very neat, and I'm looking forward to taking a
closer look at it.

Cheers,
-- David

[1]: https://lore.kernel.org/linux-kselftest/20220429043913.626647-1-davidgow@google.com/

Re: [PATCH v6 00/23] Rust support

[Miguel Ojeda]

Hi David,

On Sat, May 7, 2022 at 11:29 AM David Gow <davidgow@google.com> wrote:
>
> It's great to see some KUnit support here!

Thanks!

> It's also possible to run these tests using the KUnit wrapper tool with:
> $ ./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_RUST=y
> --make_options LLVM=1 --arch x86_64 'rust_kernel_doctests'
>
> That also nicely formats the results.

Indeed!

    [16:55:52] ============ rust_kernel_doctests (70 subtests) ============
    [16:55:52] [PASSED] rust_kernel_doctest_build_assert_rs_12_0
    [16:55:52] [PASSED] rust_kernel_doctest_build_assert_rs_55_0
    ...
    [16:55:52] [PASSED] rust_kernel_doctest_types_rs_445_0
    [16:55:52] [PASSED] rust_kernel_doctest_types_rs_509_0
    [16:55:52] ============== [PASSED] rust_kernel_doctests ===============
    [16:55:52] ============================================================
    [16:55:52] Testing complete. Passed: 70, Failed: 0, Crashed: 0,
    Skipped: 0, Errors: 0

> That all being said, I can't say I'm thrilled with the test names
> here: none of them are particularly descriptive, and they'll probably
> not be static (which would make it difficult to track results /
> regressions / etc between kernel versions). Neither of those are

Yeah, the names are not great and would change from time to time
across kernel versions.

We could ask example writers to give each example a name, but that
would make them fairly less convenient. For instance, sometimes they
may be very small snippets interleaved with docs' prose (where giving
a name may feel a bit of a burden, and people may end writing
`foo_example1`, `foo_example2` etc. for each small "step" of an
explanation). In other cases they may be very long, testing a wide API
surface (e.g. when describing a module or type), where it is also hard
to give non-generic names like `rbtree_doctest`. In those kind of
cases, I think we would end up with not much better names than
automatically generated ones.

The other aspect is that, given they are part of the documentation,
the prose or how things are explained/split may change, thus the
doctests as well. For instance, one may need to split a very long
`rbtree_doctest` in pieces, and then the name would need to change
anyway.

So I think we should avoid asking documentation writers to add a
manual name, even if that means a bit ugly test names. Also this way
they are consistently named. What do you think?

One idea could be giving them a name based on the hash of the content
and avoiding the line number, so that there is a higher chance for the
name to stay the same even when the file gets modified for other
reasons.

> necessarily deal breakers, though it might make sense to hide them
> behind a kernel option (like all other KUnit tests) so that they can
> easily be excluded where they would otherwise clutter up results. (And

Currently they are under `CONFIG_RUST_KERNEL_KUNIT_TEST` -- or do you
mean something else?

> if there's a way to properly name them, or maybe even split them into
> per-file or per-module suites, that would make them a bit easier to
> deal.) Additionally, there are some plans to taint the kernel[1] when

Yeah, splitting them further is definitely possible. We are also
likely splitting the `kernel` crate into several, which would also
make the suites smaller etc. so perhaps further splits may not be
needed.

> Regardless, this is very neat, and I'm looking forward to taking a
> closer look at it.

Thanks again for taking a look and playing with it, I am glad you
liked it! (even if it is just a first approximation, and only supports
the `kernel` crate, etc.).

Cheers,
Miguel

Re: [PATCH v6 00/23] Rust support

[Matthew Wilcox]

On Sat, May 07, 2022 at 01:06:18AM -0700, Kees Cook wrote:
> On Sat, May 07, 2022 at 07:23:58AM +0200, Miguel Ojeda wrote:
> > ## Patch series status
> > 
> > The Rust support is still to be considered experimental. However,
> > support is good enough that kernel developers can start working on the
> > Rust abstractions for subsystems and write drivers and other modules.
> 
> I'd really like to see this landed for a few reasons:
> 
> - It's under active development, and I'd rather review the changes
>   "normally", incrementally, etc. Right now it can be hard to re-review
>   some of the "mostly the same each version" patches in the series.
> 
> - I'd like to break the catch-22 of "ask for a new driver to be
>   written in rust but the rust support isn't landed" vs "the rust
>   support isn't landed because there aren't enough drivers". It
>   really feels like "release early, release often" is needed here;
>   it's hard to develop against -next. :)
> 
> Should we give it a try for this coming merge window?

I'm broadly in favour of that.  It's just code, we can always drop it
again or fix it.  There's sufficient development community around it
that it's hardly going to become abandonware.

Re: [PATCH v6 18/23] docs: add Rust documentation

[Akira Yokosawa]

Hi Miguel,

On Sat,  7 May 2022 07:24:16 +0200,
Miguel Ojeda wrote:
> Most of the documentation for Rust is written within the source code
> itself, as it is idiomatic for Rust projects. This applies to both
> the shared infrastructure at `rust/` as well as any other Rust module
> (e.g. drivers) written across the kernel.
> 
> However, these documents contain general information that does not
> fit particularly well in the source code, like the Quick Start guide.
> 
> It also contains an asset (SVG logo) used for the `rustdoc` target
> and a few other small changes elsewhere in the documentation folder.
> 
> Co-developed-by: Alex Gaynor <alex.gaynor@gmail.com>
> Signed-off-by: Alex Gaynor <alex.gaynor@gmail.com>
> Co-developed-by: Finn Behrens <me@kloenk.de>
> Signed-off-by: Finn Behrens <me@kloenk.de>
> Co-developed-by: Adam Bratschi-Kaye <ark.email@gmail.com>
> Signed-off-by: Adam Bratschi-Kaye <ark.email@gmail.com>
> Co-developed-by: Wedson Almeida Filho <wedsonaf@google.com>
> Signed-off-by: Wedson Almeida Filho <wedsonaf@google.com>
> Co-developed-by: Michael Ellerman <mpe@ellerman.id.au>
> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
> Co-developed-by: Sven Van Asbroeck <thesven73@gmail.com>
> Signed-off-by: Sven Van Asbroeck <thesven73@gmail.com>
> Co-developed-by: Wu XiangCheng <bobwxc@email.cn>
> Signed-off-by: Wu XiangCheng <bobwxc@email.cn>
> Co-developed-by: Gary Guo <gary@garyguo.net>
> Signed-off-by: Gary Guo <gary@garyguo.net>
> Co-developed-by: Boris-Chengbiao Zhou <bobo1239@web.de>
> Signed-off-by: Boris-Chengbiao Zhou <bobo1239@web.de>
> Co-developed-by: Yuki Okushi <jtitor@2k36.org>
> Signed-off-by: Yuki Okushi <jtitor@2k36.org>
> Co-developed-by: Wei Liu <wei.liu@kernel.org>
> Signed-off-by: Wei Liu <wei.liu@kernel.org>
> Co-developed-by: Daniel Xu <dxu@dxuuu.xyz>
> Signed-off-by: Daniel Xu <dxu@dxuuu.xyz>
> Co-developed-by: Julian Merkle <me@jvmerkle.de>
> Signed-off-by: Julian Merkle <me@jvmerkle.de>
> Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
> ---
>  Documentation/doc-guide/kernel-doc.rst     |   3 +
>  Documentation/index.rst                    |   1 +
>  Documentation/kbuild/kbuild.rst            |  17 +
>  Documentation/kbuild/makefiles.rst         |  50 ++-
>  Documentation/process/changes.rst          |  41 +++
>  Documentation/rust/arch-support.rst        |  34 ++
>  Documentation/rust/coding-guidelines.rst   | 214 ++++++++++++
>  Documentation/rust/general-information.rst |  77 +++++
>  Documentation/rust/index.rst               |  20 ++
>  Documentation/rust/logo.svg                | 357 +++++++++++++++++++++
I think you agreed splitting SVG part into its own patch with
a proper copying info, etc.  Let me see...  So, here is the link:

https://lore.kernel.org/lkml/CANiq72mLtvWJ5peSTpYQ8AeLEskga6Pda8Q7Daysv2pfycnyxA@mail.gmail.com/

I might have missed v5 of this patch series.
That might be because v5's 15/20 was not accepted by linux-doc's
lore archive (maybe) due to its size despite it had Cc: linux-doc.
v6's 18/23 was also rejected.

>  Documentation/rust/quick-start.rst         | 230 +++++++++++++
>  11 files changed, 1040 insertions(+), 4 deletions(-)
>  create mode 100644 Documentation/rust/arch-support.rst
>  create mode 100644 Documentation/rust/coding-guidelines.rst
>  create mode 100644 Documentation/rust/general-information.rst
>  create mode 100644 Documentation/rust/index.rst
>  create mode 100644 Documentation/rust/logo.svg
>  create mode 100644 Documentation/rust/quick-start.rst

I have some alternative ideas for table formatting in ReST.

> diff --git a/Documentation/rust/arch-support.rst b/Documentation/rust/arch-support.rst
> new file mode 100644
> index 000000000000..482757a1f3d0
> --- /dev/null
> +++ b/Documentation/rust/arch-support.rst
> @@ -0,0 +1,34 @@
> +Arch Support
> +============
> +
> +Currently, the Rust compiler (``rustc``) uses LLVM for code generation,
> +which limits the supported architectures that can be targeted. In addition,
> +support for building the kernel with LLVM/Clang varies (please see
> +Documentation/kbuild/llvm.rst). This support is needed for ``bindgen``
> +which uses ``libclang``.
> +
> +Below is a general summary of architectures that currently work. Level of
> +support corresponds to ``S`` values in the ``MAINTAINERS`` file.
> +
> +.. list-table::
> +   :widths: 10 10 10
> +   :header-rows: 1
> +
> +   * - Architecture
> +     - Level of support
> +     - Constraints
> +   * - ``arm``
> +     - Maintained
> +     - ``armv6`` and compatible only, ``RUST_OPT_LEVEL >= 2``
> +   * - ``arm64``
> +     - Maintained
> +     - None
> +   * - ``powerpc``
> +     - Maintained
> +     - ``ppc64le`` only, ``RUST_OPT_LEVEL < 2`` requires ``CONFIG_THREAD_SHIFT=15``
> +   * - ``riscv``
> +     - Maintained
> +     - ``riscv64`` only
> +   * - ``x86``
> +     - Maintained
> +     - ``x86_64`` only

Excerpt from Section "list tables" in
Documentation/doc-guide/sphinx.rst:

> The list-table formats can be useful for tables that are not easily laid
> out in the usual Sphinx ASCII-art formats.  These formats are nearly
> impossible for readers of the plain-text documents to understand, though,
> and should be avoided in the absence of a strong justification for their
> use.

So here are a couple of alternative ways to represent the table

* ASCII-art format:

============ ================ ==========================================
Architecture Level of support Constraints
============ ================ ==========================================
``arm``      Maintained       ``armv6`` and compatible only,
                              ``RUST_OPT_LEVEL >= 2``
``arm64``    Maintained       None
``powerpc``  Maintained       ``ppc64le`` only, ``RUST_OPT_LEVEL < 2``
                              requires ``CONFIG_THREAD_SHIFT=15``
``riscv``    Maintained       ``riscv64`` only
``x86``      Maintained       ``x86_64`` only
============ ================ ==========================================

* Literal block format:

::

 Architecture  Level of support  Constraints
 ------------  ----------------  -------------------------------------
 arm           Maintained        armv6 and compatible only,
                                 RUST_OPT_LEVEL >= 2
 arm64         Maintained        None
 powerpc       Maintained        ppc64le only, RUST_OPT_LEVEL < 2
                                 requires CONFIG_THREAD_SHIFT=15
 riscv         Maintained        riscv64 only
 x86           Maintained        x86_64 only


"::" above the table marks the start of a literal block.
Indents are important for la iteral block to work.
A literal block ends at a line which has the same indent as
the preceding paragraph, in this case with no indent, or at
the end of file.

As you see, those inline-literal markers of ``xxxx``, which are
distracting when the .rst file is read as plain-text, are not
necessary in the literal-block approach.  And you can directly
tweak line breaks in the Constraints column in the final HTML
and PDF docs.

In my opinion, the literal-block approach should be the most
reasonable choice here. Of course its your call which one
to choose.

        Thanks, Akira

Re: [PATCH v6 00/23] Rust support

[Wei Liu]

On Sat, May 07, 2022 at 01:06:18AM -0700, Kees Cook wrote:
> On Sat, May 07, 2022 at 07:23:58AM +0200, Miguel Ojeda wrote:
> > ## Patch series status
> > 
> > The Rust support is still to be considered experimental. However,
> > support is good enough that kernel developers can start working on the
> > Rust abstractions for subsystems and write drivers and other modules.
> 
> I'd really like to see this landed for a few reasons:
> 
> - It's under active development, and I'd rather review the changes
>   "normally", incrementally, etc. Right now it can be hard to re-review
>   some of the "mostly the same each version" patches in the series.
> 
> - I'd like to break the catch-22 of "ask for a new driver to be
>   written in rust but the rust support isn't landed" vs "the rust
>   support isn't landed because there aren't enough drivers". It
>   really feels like "release early, release often" is needed here;
>   it's hard to develop against -next. :)

+1 to both points. :-)

Re: [PATCH v6 18/23] docs: add Rust documentation

[Miguel Ojeda]

Hi Akira,

On Mon, May 9, 2022 at 6:02 AM Akira Yokosawa <akiyks@gmail.com> wrote:
>
> I think you agreed splitting SVG part into its own patch with
> a proper copying info, etc.  Let me see...  So, here is the link:

Yes, sorry, will do (in fact, it should have been there in v5 too).

By the way, the Linux SVG logo (used to make the one here) is pending
in the linux-doc ML.

> I might have missed v5 of this patch series.
> That might be because v5's 15/20 was not accepted by linux-doc's
> lore archive (maybe) due to its size despite it had Cc: linux-doc.
> v6's 18/23 was also rejected.

Yes, a few patches get rejected in several lists. We were told this
was fine as long as LKML gets them (the cover letter has the lists in
Cc).

> I have some alternative ideas for table formatting in ReST.

I was following the LLVM one, but it makes sense to use the other ones
where possible. I can send a patch for that one too.

> So here are a couple of alternative ways to represent the table
>
> * ASCII-art format:
> * Literal block format:

Thanks for taking the time to format the examples, it is useful :)

> As you see, those inline-literal markers of ``xxxx``, which are
> distracting when the .rst file is read as plain-text, are not
> necessary in the literal-block approach.  And you can directly

I agree, it can be better (it is one reason I find Markdown a bit more
readable since it uses a single backquote for that instead of two).

> In my opinion, the literal-block approach should be the most
> reasonable choice here. Of course its your call which one
> to choose.

Yeah, that sounds reasonable. I will take a look.

Thanks for the review!

Cheers,
Miguel

Re: [PATCH v6 18/23] docs: add Rust documentation

[Akira Yokosawa]

[+To: Jon]

On Mon, 9 May 2022 12:41:28 +0200,
Miguel Ojeda wrote:
> Hi Akira,
> 
> On Mon, May 9, 2022 at 6:02 AM Akira Yokosawa <akiyks@gmail.com> wrote:
>>
>> I think you agreed splitting SVG part into its own patch with
>> a proper copying info, etc.  Let me see...  So, here is the link:
> 
> Yes, sorry, will do (in fact, it should have been there in v5 too).
> 
> By the way, the Linux SVG logo (used to make the one here) is pending
> in the linux-doc ML.

So you mean the following post:

    https://lore.kernel.org/lkml/20220207014418.GA28724@kernel.org/

I'm not sure why Jon has not responded.

Jon, was there any issue on this patch?

> 
>> I might have missed v5 of this patch series.
>> That might be because v5's 15/20 was not accepted by linux-doc's
>> lore archive (maybe) due to its size despite it had Cc: linux-doc.
>> v6's 18/23 was also rejected.
> 
> Yes, a few patches get rejected in several lists. We were told this
> was fine as long as LKML gets them (the cover letter has the lists in
> Cc).
> 
>> I have some alternative ideas for table formatting in ReST.
> 
> I was following the LLVM one, but it makes sense to use the other ones
> where possible. I can send a patch for that one too.
> 
>> So here are a couple of alternative ways to represent the table
>>
>> * ASCII-art format:
>> * Literal block format:
> 
> Thanks for taking the time to format the examples, it is useful :)
Glad you like it.  ;-)

        Thanks, Akira

> 
>> As you see, those inline-literal markers of ``xxxx``, which are
>> distracting when the .rst file is read as plain-text, are not
>> necessary in the literal-block approach.  And you can directly
> 
> I agree, it can be better (it is one reason I find Markdown a bit more
> readable since it uses a single backquote for that instead of two).
> 
>> In my opinion, the literal-block approach should be the most
>> reasonable choice here. Of course its your call which one
>> to choose.
> 
> Yeah, that sounds reasonable. I will take a look.
> 
> Thanks for the review!
> 
> Cheers,
> Miguel

Re: [PATCH v6 18/23] docs: add Rust documentation

[Jonathan Corbet]

Miguel Ojeda <ojeda@kernel.org> writes:

> Most of the documentation for Rust is written within the source code
> itself, as it is idiomatic for Rust projects. This applies to both
> the shared infrastructure at `rust/` as well as any other Rust module
> (e.g. drivers) written across the kernel.
>
> However, these documents contain general information that does not
> fit particularly well in the source code, like the Quick Start guide.
>
> It also contains an asset (SVG logo) used for the `rustdoc` target
> and a few other small changes elsewhere in the documentation folder.

Trying to take a closer look this time...

I foresee merge conflicts, but so it goes.  Trying to split this apart
would not make a lot of sense.

[...]

> --- /dev/null
> +++ b/Documentation/rust/arch-support.rst
> @@ -0,0 +1,34 @@
> +Arch Support
> +============
> +
> +Currently, the Rust compiler (``rustc``) uses LLVM for code generation,
> +which limits the supported architectures that can be targeted. In addition,
> +support for building the kernel with LLVM/Clang varies (please see
> +Documentation/kbuild/llvm.rst). This support is needed for ``bindgen``
> +which uses ``libclang``.
> +
> +Below is a general summary of architectures that currently work. Level of
> +support corresponds to ``S`` values in the ``MAINTAINERS`` file.
> +
> +.. list-table::
> +   :widths: 10 10 10
> +   :header-rows: 1

Please use normal tables rather than list-table; this kind of thing is
really unreadable in the source form.

> +   * - Architecture
> +     - Level of support
> +     - Constraints
> +   * - ``arm``
> +     - Maintained
> +     - ``armv6`` and compatible only, ``RUST_OPT_LEVEL >= 2``
> +   * - ``arm64``
> +     - Maintained
> +     - None
> +   * - ``powerpc``
> +     - Maintained
> +     - ``ppc64le`` only, ``RUST_OPT_LEVEL < 2`` requires ``CONFIG_THREAD_SHIFT=15``
> +   * - ``riscv``
> +     - Maintained
> +     - ``riscv64`` only
> +   * - ``x86``
> +     - Maintained
> +     - ``x86_64`` only
> diff --git a/Documentation/rust/coding-guidelines.rst b/Documentation/rust/coding-guidelines.rst
> new file mode 100644
> index 000000000000..2a71fd68a06d
> --- /dev/null
> +++ b/Documentation/rust/coding-guidelines.rst
> @@ -0,0 +1,214 @@
> +Coding Guidelines
> +=================
> +
> +This document describes how to write Rust code in the kernel.
> +
> +
> +Style & formatting
> +------------------
> +
> +The code should be formatted using ``rustfmt``. In this way, a person
> +contributing from time to time to the kernel does not need to learn and
> +remember one more style guide. More importantly, reviewers and maintainers
> +do not need to spend time pointing out style issues anymore, and thus
> +less patch roundtrips may be needed to land a change.

I foresee disagreements over coding style conventions in the
future... I don't plan to be part of that conversation :)

> +.. note:: Conventions on comments and documentation are not checked by
> +  ``rustfmt``. Thus those are still needed to be taken care of.
> +
> +The default settings of ``rustfmt`` are used. This means the idiomatic Rust
> +style is followed. For instance, 4 spaces are used for indentation rather
> +than tabs.
> +
> +It is convenient to instruct editors/IDEs to format while typing,
> +when saving or at commit time. However, if for some reason reformatting
> +the entire kernel Rust sources is needed at some point, the following can be
> +run::
> +
> +	make LLVM=1 rustfmt

I will ask whether we want this, though.  Why would anybody want to
mass-reformat the entire body of kernel code?  This seems like something
that would generate an endless stream of "helpful" patches and a lot of
churn.

Thanks,

jon

Re: [PATCH v6 18/23] docs: add Rust documentation

[Jonathan Corbet]

Akira Yokosawa <akiyks@gmail.com> writes:

> [+To: Jon]
>
> On Mon, 9 May 2022 12:41:28 +0200,
> Miguel Ojeda wrote:
>> Hi Akira,
>> 
>> On Mon, May 9, 2022 at 6:02 AM Akira Yokosawa <akiyks@gmail.com> wrote:
>>>
>>> I think you agreed splitting SVG part into its own patch with
>>> a proper copying info, etc.  Let me see...  So, here is the link:
>> 
>> Yes, sorry, will do (in fact, it should have been there in v5 too).
>> 
>> By the way, the Linux SVG logo (used to make the one here) is pending
>> in the linux-doc ML.
>
> So you mean the following post:
>
>     https://lore.kernel.org/lkml/20220207014418.GA28724@kernel.org/
>
> I'm not sure why Jon has not responded.
>
> Jon, was there any issue on this patch?

Yeah, but the issues are all with me :)  Please accept my apologies for
letting it slip through the cracks.

Looking at it now, though, I hesitate to add the logo (and another
COPYING file) in the top-level Documentation directory - I'd really
rather people not have to pick through a bunch of unrelated stuff to
find the actual docs they want.  I'd recommend we make a
Documentation/images (or .../assets or whatever) and put things like
logos there.

Disagree?  If not, could I get a version of the patch that does that?  I
promise not to set on it for three months this time...

Thanks,

jon

Re: [PATCH v6 18/23] docs: add Rust documentation

[Gaelan Steele]

> On May 9, 2022, at 3:32 PM, Jonathan Corbet <corbet@lwn.net> wrote:
> 
>> +It is convenient to instruct editors/IDEs to format while typing,
>> +when saving or at commit time. However, if for some reason reformatting
>> +the entire kernel Rust sources is needed at some point, the following can be
>> +run::
>> +
>> +	make LLVM=1 rustfmt
> 
> I will ask whether we want this, though. Why would anybody want to
> mass-reformat the entire body of kernel code? This seems like something
> that would generate an endless stream of "helpful" patches and a lot of
> churn.

That would only happen if the code diverged from rustfmt’s output in the
first place. Generally, in Rust projects, the source tree is always kept
formatted with rustfmt - so running `make LLVM=1 rustfmt` would only
ever touch code that you’d just changed. 

Gaelan

Re: [PATCH v6 00/23] Rust support

[David Gow]

On Sat, May 7, 2022 at 11:03 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> Hi David,
>
> On Sat, May 7, 2022 at 11:29 AM David Gow <davidgow@google.com> wrote:
> >
> > It's great to see some KUnit support here!
>
> Thanks!
>
> > It's also possible to run these tests using the KUnit wrapper tool with:
> > $ ./tools/testing/kunit/kunit.py run --kconfig_add CONFIG_RUST=y
> > --make_options LLVM=1 --arch x86_64 'rust_kernel_doctests'
> >
> > That also nicely formats the results.
>
> Indeed!
>
>     [16:55:52] ============ rust_kernel_doctests (70 subtests) ============
>     [16:55:52] [PASSED] rust_kernel_doctest_build_assert_rs_12_0
>     [16:55:52] [PASSED] rust_kernel_doctest_build_assert_rs_55_0
>     ...
>     [16:55:52] [PASSED] rust_kernel_doctest_types_rs_445_0
>     [16:55:52] [PASSED] rust_kernel_doctest_types_rs_509_0
>     [16:55:52] ============== [PASSED] rust_kernel_doctests ===============
>     [16:55:52] ============================================================
>     [16:55:52] Testing complete. Passed: 70, Failed: 0, Crashed: 0,
>     Skipped: 0, Errors: 0
>

I've just sent out a pull request to get this working under UML as
well, which would simplify running these further:
https://github.com/Rust-for-Linux/linux/pull/766

> > That all being said, I can't say I'm thrilled with the test names
> > here: none of them are particularly descriptive, and they'll probably
> > not be static (which would make it difficult to track results /
> > regressions / etc between kernel versions). Neither of those are
>
> Yeah, the names are not great and would change from time to time
> across kernel versions.
>
> We could ask example writers to give each example a name, but that
> would make them fairly less convenient. For instance, sometimes they
> may be very small snippets interleaved with docs' prose (where giving
> a name may feel a bit of a burden, and people may end writing
> `foo_example1`, `foo_example2` etc. for each small "step" of an
> explanation). In other cases they may be very long, testing a wide API
> surface (e.g. when describing a module or type), where it is also hard
> to give non-generic names like `rbtree_doctest`. In those kind of
> cases, I think we would end up with not much better names than
> automatically generated ones.
>
> The other aspect is that, given they are part of the documentation,
> the prose or how things are explained/split may change, thus the
> doctests as well. For instance, one may need to split a very long
> `rbtree_doctest` in pieces, and then the name would need to change
> anyway.
>
> So I think we should avoid asking documentation writers to add a
> manual name, even if that means a bit ugly test names. Also this way
> they are consistently named. What do you think?

Yeah, these are all fair points: particularly for small doctests.

Maybe having an optional name, which more significant tests could use
to override the file:line names? That could be useful for a few of the
larger, more often referenced tests.

> One idea could be giving them a name based on the hash of the content
> and avoiding the line number, so that there is a higher chance for the
> name to stay the same even when the file gets modified for other
> reasons.

Ugh: it's a bit ugly either way. I suspect that file:line is still
probably better, if only because we need some way of looking up the
test in the code if it fails. I'd hate for people to be randomly
hashing bits of just to find out what test is failing.

> > necessarily deal breakers, though it might make sense to hide them
> > behind a kernel option (like all other KUnit tests) so that they can
> > easily be excluded where they would otherwise clutter up results. (And
>
> Currently they are under `CONFIG_RUST_KERNEL_KUNIT_TEST` -- or do you
> mean something else?
>

Oops: I missed that (one of the issues with testing this on a
different machine which had a rust toolchain). Looks good to me.

> > if there's a way to properly name them, or maybe even split them into
> > per-file or per-module suites, that would make them a bit easier to
> > deal.) Additionally, there are some plans to taint the kernel[1] when
>
> Yeah, splitting them further is definitely possible. We are also
> likely splitting the `kernel` crate into several, which would also
> make the suites smaller etc. so perhaps further splits may not be
> needed.

Ah: I didn't realise the plan was always to have crate-specific
suites, and possibly to split things up.

The KTAP output specification does actually support arbitrary nesting
(though KUnit itself doesn't at the moment), which would potentially
be an option if (e.g.) providing the complete module nesting made
sense. I'm not convinced that'd make things easier to read, though.

> > Regardless, this is very neat, and I'm looking forward to taking a
> > closer look at it.
>
> Thanks again for taking a look and playing with it, I am glad you
> liked it! (even if it is just a first approximation, and only supports
> the `kernel` crate, etc.).
>
> Cheers,
> Miguel

Thanks,
-- David

Re: [PATCH v6 18/23] docs: add Rust documentation

[Josh Triplett]

On Mon, May 09, 2022 at 08:14:54PM -0700, Gaelan Steele wrote:
> 
> 
> > On May 9, 2022, at 3:32 PM, Jonathan Corbet <corbet@lwn.net> wrote:
> > 
> >> +It is convenient to instruct editors/IDEs to format while typing,
> >> +when saving or at commit time. However, if for some reason reformatting
> >> +the entire kernel Rust sources is needed at some point, the following can be
> >> +run::
> >> +
> >> +	make LLVM=1 rustfmt
> > 
> > I will ask whether we want this, though. Why would anybody want to
> > mass-reformat the entire body of kernel code? This seems like something
> > that would generate an endless stream of "helpful" patches and a lot of
> > churn.
> 
> That would only happen if the code diverged from rustfmt’s output in the
> first place. Generally, in Rust projects, the source tree is always kept
> formatted with rustfmt - so running `make LLVM=1 rustfmt` would only
> ever touch code that you’d just changed. 

Exactly. This is convenient for the same reason doing a project-wide
`cargo fmt` is useful in Rust projects: you can do all your editing,
then format your code before committing.

Re: [PATCH v6 00/23] Rust support

[Miguel Ojeda]

Hi David,

On Tue, May 10, 2022 at 6:45 AM David Gow <davidgow@google.com> wrote:
>
> I've just sent out a pull request to get this working under UML as
> well, which would simplify running these further:
> https://github.com/Rust-for-Linux/linux/pull/766

Thanks a lot!

> Yeah, these are all fair points: particularly for small doctests.
>
> Maybe having an optional name, which more significant tests could use
> to override the file:line names? That could be useful for a few of the
> larger, more often referenced tests.

Sounds reasonable. I can add support for that.

> Ugh: it's a bit ugly either way. I suspect that file:line is still
> probably better, if only because we need some way of looking up the
> test in the code if it fails. I'd hate for people to be randomly
> hashing bits of just to find out what test is failing.

One redeeming quality is that the assertion prints the line/file
number in the generated file, so it would still be possible to check
where it came from:

    [13:13:43] # rust_kernel_doctest_str_rs_somehash: ASSERTION FAILED
at rust/doctests_kernel_generated.rs:2209
    [13:13:43] Expected 2 > 3 to be true, but is false
    [13:13:43] not ok 43 - rust_kernel_doctest_str_rs_somehash
    [13:13:43] [FAILED] rust_kernel_doctest_str_rs_somehash

Another alternative is to keep the file:line information around
without embedding it into the test name, e.g. in a TAP comment or a
mapping file (which `kunit.py` could read).

But, yeah, before doing hashes or things like that, I would just go
for simplicity and keep things as they are unless some use case really
needs doctests to be stable.

> Oops: I missed that (one of the issues with testing this on a
> different machine which had a rust toolchain). Looks good to me.
>
> Ah: I didn't realise the plan was always to have crate-specific
> suites, and possibly to split things up.
>
> The KTAP output specification does actually support arbitrary nesting
> (though KUnit itself doesn't at the moment), which would potentially
> be an option if (e.g.) providing the complete module nesting made
> sense. I'm not convinced that'd make things easier to read, though.

That is useful to know in case we need it, thanks!

Cheers,
Miguel

Re: [PATCH v6 18/23] docs: add Rust documentation

[Miguel Ojeda]

On Tue, May 10, 2022 at 12:37 AM Jonathan Corbet <corbet@lwn.net> wrote:
>
> Yeah, but the issues are all with me :)  Please accept my apologies for
> letting it slip through the cracks.

No apologies needed! It is not an important patch -- I just thought it
would be better to have the "base logo" merged before the "derived
one".

> Looking at it now, though, I hesitate to add the logo (and another
> COPYING file) in the top-level Documentation directory - I'd really
> rather people not have to pick through a bunch of unrelated stuff to
> find the actual docs they want.  I'd recommend we make a
> Documentation/images (or .../assets or whatever) and put things like
> logos there.

Yeah, makes sense to avoid dumping things in the top-level directory.

> Disagree?  If not, could I get a version of the patch that does that?  I
> promise not to set on it for three months this time...

+1, I will send it.

Thanks Jon!

Cheers,
Miguel

Re: [PATCH v6 18/23] docs: add Rust documentation

[Miguel Ojeda]

On Tue, May 10, 2022 at 12:32 AM Jonathan Corbet <corbet@lwn.net> wrote:
>
> Trying to take a closer look this time...
>
> I foresee merge conflicts, but so it goes.  Trying to split this apart
> would not make a lot of sense.

Is there a big change coming to docs? (there are not conflicts in
linux-next within the docs). Or what do you mean?

> Please use normal tables rather than list-table; this kind of thing is
> really unreadable in the source form.

Will do!

> I foresee disagreements over coding style conventions in the
> future... I don't plan to be part of that conversation :)

Do you mean with the tool settings? I guess we may get some proposals
about tweaking them, yeah, but one reason to stick to the defaults is
to avoid that! :)

If you mean enforcing `rustfmt`, please see below.

> I will ask whether we want this, though.  Why would anybody want to
> mass-reformat the entire body of kernel code?  This seems like something
> that would generate an endless stream of "helpful" patches and a lot of
> churn.

So the idea is that, since everything is already formatted, the output
of this is empty (in mainline), like Gaelan/Josh mentioned. Thus
nobody should be sending any formatting patches since there is nothing
to change.

Now, for those developing and not running `rustfmt` automatically in
some way (e.g. in their editors), it can be useful to run it before
submitting the patches: the output should only show changes to
whatever you changed since everything else should be already
formatted.

Of course, as soon as others start submitting patches independently,
mistakes may slip through, but we are enforcing this in our CI (and it
could be done more centrally), so we should notice quickly.

There could be, of course, bugs in the tool; and there are a few
situations where the tool may not guarantee formatting stability, but
hopefully those are rare and small enough so that we can keep
enforcing this. I think it is worth trying.

Cheers,
Miguel