Re: [PATCH v1 4/4] Documentation: kunit: document support for QEMU in kunit_tool

[David Gow <davidgow@google.com>]

[-- Attachment #1: Type: text/plain, Size: 4077 bytes --]

On Sat, May 8, 2021 at 5:31 AM Brendan Higgins
<brendanhiggins@google.com> wrote:
>
> Document QEMU support, what it does, and how to use it in kunit_tool.
>
> Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
> ---

This is a good start, and probably meets the minimum requirements, but
I do have a number of comments and suggestions below.

Cheers,
-- David

>  Documentation/dev-tools/kunit/usage.rst | 37 +++++++++++++++++++++----
>  1 file changed, 31 insertions(+), 6 deletions(-)
>
> diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
> index 650f99590df57..b74bd7c87cc20 100644
> --- a/Documentation/dev-tools/kunit/usage.rst
> +++ b/Documentation/dev-tools/kunit/usage.rst
> @@ -612,14 +612,39 @@ only things to be aware of when doing so.
>  The biggest impediment will likely be that certain KUnit features and
>  infrastructure may not support your target environment. For example, at this
>  time the KUnit Wrapper (``tools/testing/kunit/kunit.py``) does not work outside
> -of UML. Unfortunately, there is no way around this. Using UML (or even just a
> -particular architecture) allows us to make a lot of assumptions that make it
> -possible to do things which might otherwise be impossible.
> +of UML and QEMU. Unfortunately, there is no way around this. Using UML and QEMU
> +(or even just a particular architecture) allows us to make a lot of assumptions
> +that make it possible to do things which might otherwise be impossible.

This is a bit more awkward now, and I don't think gives quite the
right impression. Particularly the "Unfortunately, there is no way
around this." bit: there's no fundamental reason that someone couldn't
implement support for some other emulator (or even a setup which
pushed to real hardware and read results over serial), it'd just take
a bit of work to implement (like this patch series has done for qemu).

Personally, I think it'd be easiest to simplify this section and say
that kunit_tool currently only fully supports some architectures, via
UML and QEMU.

>
>  Nevertheless, all core KUnit framework features are fully supported on all
> -architectures, and using them is straightforward: all you need to do is to take
> -your kunitconfig, your Kconfig options for the tests you would like to run, and
> -merge them into whatever config your are using for your platform. That's it!
> +architectures, and using them is straightforward: Most popular architectures
> +are supported directly in the KUnit Wrapper via QEMU.  Currently, supported
> +architectures on QEMU include:
> +
> +*   i386
> +*   x86_64
> +*   arm
> +*   arm64
> +*   alpha
> +*   powerpc
> +*   riscv
> +*   s390
> +*   sparc
> +
> +In order to run KUnit tests on one of these architectures via QEMU with the
> +KUnit wrapper, all you need to do is specify the flags ``--arch`` and
> +``--cross_compile`` when invoking the KUnit Wrapper. For example, we could run
> +the default KUnit tests on ARM in the following manner (assuming we have an ARM
> +toolchain installed):
> +
> +.. code-block:: bash
> +
> +       tools/testing/kunit/kunit.py run --timeout=60 --jobs=12 --arch=arm --cross_compile=arm-linux-gnueabihf-
> +

Is it worth also documenting the --qemu_config option here?
(Particularly given the restriction on its path?) Or is that something
best added to the kunit_tool page?

That being said, changes to the kunit_tool page are probably worth
adding as a section in the updated page:
https://patchwork.kernel.org/project/linux-kselftest/patch/20210417034553.1048895-1-davidgow@google.com/

At the very least, it'd be nice to have the new QEMU-related options
documented there.


> +Alternatively, if you want to run your tests on real hardware or in some other
> +emulation environment, all you need to do is to take your kunitconfig, your
> +Kconfig options for the tests you would like to run, and merge them into
> +whatever config your are using for your platform. That's it!
>
>  For example, let's say you have the following kunitconfig:
>
> --
> 2.31.1.607.g51e8a6a459-goog
>

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4000 bytes --]