[PATCH v4 1/2] docs: rust: Move testing to a separate page

[PATCH v4 1/2] docs: rust: Move testing to a separate page

[Dirk Behme]

To be able to add more testing documentation move the testing
section to it's own page.

No change on the documentation itself.

Suggested-by: Trevor Gross <tmgross@umich.edu>
Suggested-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: Trevor Gross <tmgross@umich.edu>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
---
Changes in v4: Add some Reviewed-by

 Documentation/rust/general-information.rst | 24 ----------------------
 Documentation/rust/index.rst               |  1 +
 Documentation/rust/testing.rst             | 24 ++++++++++++++++++++++
 3 files changed, 25 insertions(+), 24 deletions(-)
 create mode 100644 Documentation/rust/testing.rst

diff --git a/Documentation/rust/general-information.rst b/Documentation/rust/general-information.rst
index 236c6dd3c647f..081397827a7ea 100644
--- a/Documentation/rust/general-information.rst
+++ b/Documentation/rust/general-information.rst
@@ -77,27 +77,3 @@ configuration:
 	#[cfg(CONFIG_X="y")]   // Enabled as a built-in (`y`)
 	#[cfg(CONFIG_X="m")]   // Enabled as a module   (`m`)
 	#[cfg(not(CONFIG_X))]  // Disabled
-
-
-Testing
--------
-
-There are the tests that come from the examples in the Rust documentation
-and get transformed into KUnit tests. These can be run via KUnit. For example
-via ``kunit_tool`` (``kunit.py``) on the command line::
-
-	./tools/testing/kunit/kunit.py run --make_options LLVM=1 --arch x86_64 --kconfig_add CONFIG_RUST=y
-
-Alternatively, KUnit can run them as kernel built-in at boot. Refer to
-Documentation/dev-tools/kunit/index.rst for the general KUnit documentation
-and Documentation/dev-tools/kunit/architecture.rst for the details of kernel
-built-in vs. command line testing.
-
-Additionally, there are the ``#[test]`` tests. These can be run using
-the ``rusttest`` Make target::
-
-	make LLVM=1 rusttest
-
-This requires the kernel ``.config`` and downloads external repositories.
-It runs the ``#[test]`` tests on the host (currently) and thus is fairly
-limited in what these tests can test.
diff --git a/Documentation/rust/index.rst b/Documentation/rust/index.rst
index 965f2db529e0f..46d35bd395cf5 100644
--- a/Documentation/rust/index.rst
+++ b/Documentation/rust/index.rst
@@ -40,6 +40,7 @@ configurations.
     general-information
     coding-guidelines
     arch-support
+    testing
 
 .. only::  subproject and html
 
diff --git a/Documentation/rust/testing.rst b/Documentation/rust/testing.rst
new file mode 100644
index 0000000000000..ba8a01015abad
--- /dev/null
+++ b/Documentation/rust/testing.rst
@@ -0,0 +1,24 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Testing
+=======
+
+There are the tests that come from the examples in the Rust documentation
+and get transformed into KUnit tests. These can be run via KUnit. For example
+via ``kunit_tool`` (``kunit.py``) on the command line::
+
+	./tools/testing/kunit/kunit.py run --make_options LLVM=1 --arch x86_64 --kconfig_add CONFIG_RUST=y
+
+Alternatively, KUnit can run them as kernel built-in at boot. Refer to
+Documentation/dev-tools/kunit/index.rst for the general KUnit documentation
+and Documentation/dev-tools/kunit/architecture.rst for the details of kernel
+built-in vs. command line testing.
+
+Additionally, there are the ``#[test]`` tests. These can be run using
+the ``rusttest`` Make target::
+
+	make LLVM=1 rusttest
+
+This requires the kernel ``.config`` and downloads external repositories.
+It runs the ``#[test]`` tests on the host (currently) and thus is fairly
+limited in what these tests can test.
-- 
2.28.0

[PATCH v4 2/2] docs: rust: Add description of Rust documentation test as KUnit ones

[Dirk Behme]

Rust documentation tests are automatically converted into KUnit
tests. The commit adding this feature

commit a66d733da801 ("rust: support running Rust documentation tests as KUnit ones")

from Miguel has a very nice commit message with a lot details
for this. To not 'hide' that just in a commit message, pick the main
parts of it and add it to the documentation. And add a short info
how to enable this. While adding this, improve the structure of
the sections.

Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Co-developed-by: Miguel Ojeda <ojeda@kernel.org>
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
---
Changes in v4:
 * Add Miguel's Signed-off-by. Thanks! :)
 * Fix a typo and minor rewording as suggested by Miguel.
 * Add some Reviewed-by

 Documentation/rust/testing.rst | 120 +++++++++++++++++++++++++++++++--
 1 file changed, 116 insertions(+), 4 deletions(-)

diff --git a/Documentation/rust/testing.rst b/Documentation/rust/testing.rst
index ba8a01015abad..a6ae4ae46b5b9 100644
--- a/Documentation/rust/testing.rst
+++ b/Documentation/rust/testing.rst
@@ -1,11 +1,25 @@
 .. SPDX-License-Identifier: GPL-2.0
 
 Testing
-=======
++++++++
 
-There are the tests that come from the examples in the Rust documentation
-and get transformed into KUnit tests. These can be run via KUnit. For example
-via ``kunit_tool`` (``kunit.py``) on the command line::
+This document contains useful information how to test the Rust code in the kernel.
+
+There are two sorts of tests:
+
+ * The KUnit tests
+ * The ``#[test]`` tests
+
+The KUnit tests
+===============
+
+These are the tests that come from the examples in the Rust documentation.
+They get transformed into KUnit tests.
+
+Usage
+~~~~~
+
+These tests can be run via KUnit. For example via ``kunit_tool`` (``kunit.py``) on the command line::
 
 	./tools/testing/kunit/kunit.py run --make_options LLVM=1 --arch x86_64 --kconfig_add CONFIG_RUST=y
 
@@ -14,6 +28,104 @@ Documentation/dev-tools/kunit/index.rst for the general KUnit documentation
 and Documentation/dev-tools/kunit/architecture.rst for the details of kernel
 built-in vs. command line testing.
 
+To use these KUnit doctests, the following must be enabled::
+
+	CONFIG_KUNIT
+	   Kernel hacking -> Kernel Testing and Coverage -> KUnit - Enable support for unit tests
+	CONFIG_RUST_KERNEL_DOCTESTS
+	   Kernel hacking -> Rust hacking -> Doctests for the `kernel` crate
+
+in the kernel config system.
+
+KUnit tests are documentation tests
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+These documentation tests are typically examples of
+usage of any item (e.g. function, struct, module...).
+
+They are very convenient because they are just written
+alongside the documentation. For instance:
+
+.. code-block:: rust
+
+	/// Sums two numbers.
+	///
+	/// ```
+	/// assert_eq!(mymod::f(10, 20), 30);
+	/// ```
+	pub fn f(a: i32, b: i32) -> i32 {
+	    a + b
+	}
+
+In userspace, the tests are collected and run via ``rustdoc``.
+Using the tool as-is would be useful already, since it allows
+verifying that examples compile (thus enforcing they are kept
+in sync with the code they document) and as well as running
+those that do not depend on in-kernel APIs.
+
+For the kernel, however, these tests get transformed into KUnit
+test suites. This means that doctests get compiled as Rust
+kernel objects, allowing them to run against a built kernel.
+
+A benefit of this KUnit integration is that Rust doctests get
+to reuse existing testing facilities. For instance, the kernel
+log would look like::
+
+	KTAP version 1
+	1..1
+	    KTAP version 1
+	    # Subtest: rust_doctests_kernel
+	    1..59
+	    # rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13
+	    ok 1 rust_doctest_kernel_build_assert_rs_0
+	    # rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56
+	    ok 2 rust_doctest_kernel_build_assert_rs_1
+	    # rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122
+	    ok 3 rust_doctest_kernel_init_rs_0
+	    ...
+	    # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
+	    ok 59 rust_doctest_kernel_types_rs_2
+	# rust_doctests_kernel: pass:59 fail:0 skip:0 total:59
+	# Totals: pass:59 fail:0 skip:0 total:59
+	ok 1 rust_doctests_kernel
+
+Tests using the `? <https://doc.rust-lang.org/reference/expressions/operator-expr.html#the-question-mark-operator>`_ operator are also supported as usual, e.g.:
+
+.. code-block:: rust
+
+	/// ```
+	/// # use kernel::{spawn_work_item, workqueue};
+	/// spawn_work_item!(workqueue::system(), || pr_info!("x"))?;
+	/// # Ok::<(), Error>(())
+	/// ```
+
+The tests are also compiled with Clippy under ``CLIPPY=1``, just
+like normal code, thus also benefitting from extra linting.
+
+In order for developers to easily see which line of doctest code
+caused a failure, a KTAP diagnostic line is printed to the log.
+This contains the location (file and line) of the original test
+(i.e. instead of the location in the generated Rust file)::
+
+	# rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150
+
+Rust tests appear to assert using the usual ``assert!`` and
+``assert_eq!`` macros from the Rust standard library (``core``).
+We provide a custom version that forwards the call to KUnit instead.
+Importantly, these macros do not require passing context,
+unlike those for KUnit testing (i.e. ``struct kunit *``). This makes
+them easier to use, and readers of the documentation do not need
+to care about which testing framework is used. In addition, it
+may allow us to test third-party code more easily in the future.
+
+A current limitation is that KUnit does not support assertions
+in other tasks. Thus, we presently simply print an error to the
+kernel log if an assertion actually failed. Additionally,
+doctests are not run for nonpublic functions.
+
+The ``#[test]`` tests
+=====================
+
 Additionally, there are the ``#[test]`` tests. These can be run using
 the ``rusttest`` Make target::
 
-- 
2.28.0

Re: [PATCH v4 2/2] docs: rust: Add description of Rust documentation test as KUnit ones

[Trevor Gross]

On Tue, Jan 30, 2024 at 2:51 AM Dirk Behme <dirk.behme@de.bosch.com> wrote:
>
> Rust documentation tests are automatically converted into KUnit
> tests. The commit adding this feature
>
> commit a66d733da801 ("rust: support running Rust documentation tests as KUnit ones")
>
> from Miguel has a very nice commit message with a lot details
> for this. To not 'hide' that just in a commit message, pick the main
> parts of it and add it to the documentation. And add a short info
> how to enable this. While adding this, improve the structure of
> the sections.
>
> Reviewed-by: David Gow <davidgow@google.com>
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Co-developed-by: Miguel Ojeda <ojeda@kernel.org>
> Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>
> ---

This looks good to me, thanks for the improvement.

Reviewed-by: Trevor Gross <tmgross@umich.edu>

Re: [PATCH v4 1/2] docs: rust: Move testing to a separate page

[Miguel Ojeda]

On Tue, Jan 30, 2024 at 8:51 AM Dirk Behme <dirk.behme@de.bosch.com> wrote:
>
> To be able to add more testing documentation move the testing
> section to it's own page.
>
> No change on the documentation itself.
>
> Suggested-by: Trevor Gross <tmgross@umich.edu>
> Suggested-by: Miguel Ojeda <ojeda@kernel.org>
> Reviewed-by: Trevor Gross <tmgross@umich.edu>
> Reviewed-by: David Gow <davidgow@google.com>
> Reviewed-by: Alice Ryhl <aliceryhl@google.com>
> Signed-off-by: Dirk Behme <dirk.behme@de.bosch.com>

Applied to `rust-next` with fixed unordered list rendering, rewrapped
text and headers made consistent with the other documents in `rust/`
-- thanks everyone!

(I also created a "good first issue" about another bad rendering case
of an unordered list:
https://github.com/Rust-for-Linux/linux/issues/1063)

Cheers,
Miguel