[kvm-unit-tests PATCH v4 00/13] arm/arm64: Add kvmtool to the runner script

* [kvm-unit-tests PATCH v4 00/13] arm/arm64: Add kvmtool to the runner script
@ 2025-06-25 15:48 Alexandru Elisei
  2025-06-25 15:48 ` [kvm-unit-tests PATCH v4 01/13] run_tests.sh: Document --probe-maxsmp argument Alexandru Elisei
                   ` (15 more replies)
  0 siblings, 16 replies; 26+ messages in thread
From: Alexandru Elisei @ 2025-06-25 15:48 UTC (permalink / raw)
  To: andrew.jones, eric.auger, lvivier, thuth, frankja, imbrenda, nrb,
	david, pbonzini
  Cc: kvm, kvmarm, linuxppc-dev, kvm-riscv, linux-s390, will,
	julien.thierry.kdev, maz, oliver.upton, suzuki.poulose, yuzenghui,
	joey.gouly, andre.przywara, shahuang

v3 can be found here [1]. Based on top of the series that add qemu_params and
test_args [2].

To goal is to allow the user to do:

$ ./configure --target=kvmtool
$ make clean && make
$ ./run_tests.sh

to run all the tests automatically with kvmtool.

Reasons to use kvmtool:

* kvmtool is smaller and a lot easier to modify compared to qemu, which
means developers may prefer it when adding or prototyping new features to
KVM, and being able to run all the tests reliably and automatically is very
useful.

* kvmtool is faster to run the tests (a couple of times faster on
my rockpro64), making for a quick turnaround. But do keep in mind that not
all tests work on kvmtool because of missing features compared to qemu.

* kvmtool does things differently than qemu: different memory layout,
different uart, PMU emulation is disabled by default, etc. This makes it a
good testing vehicule for kvm-unit-tests itself.

Changes v3->v4
--------------

Overview of the changes:

* Gathered Reviewed-by tags - thanks for the review!

* Sent patches #1 ("scripts: unittests.cfg: Rename 'extra_params' to
'qemu_params'") and #2 ("scripts: Add 'test_args' test definition parameter")
as a separate series.

* Fixed the typos reported during the review.

* Ran shellcheck on the patches, this resulted in minor changes.

* Dropped patch "configure: Export TARGET unconditionally" - now the functions
in vmm.bash will check if TARGET is set, instead of having the other scripts use
$TARGET to directly index the vmm_opts array.

* Direct reads of $TARGET have been replaced with vmm_get_target(), to account
for the fact that most architectures don't configure $TARGET (only arm and
arm64 do that).

* Renamed check_vmm_supported() to vmm_check_supported() to match the
function names introduced in subsequent patches.

* Renamed vmm_opts->vmm_optname to match the new function names.

* Reordered the key-value pairs from vmm_optname in alphabetical order.

* Use the "," separator for the composite keys of the associative array instead
of ":" (don't remember why I originally settled on ":", but it was a really poor
choice).

* Dropped the Reviewed-by tags from Drew and Shaoqin Huang from patch #6
("scripts: Use an associative array for qemu argument names") - the review is
much appreciated, but the way the vmm_opts array (now renamed to vmm_optname) is
created, and used, has changed, and since the patch is about introducing the
associative array, I thought it would be useful to have another round of review.

* Use functions instead of indexing vmm_opts (now vmm_optname) directly.

* Fixed standalone test generation by removing 'source vmm.bash' from
scripts/arch-run.bash, $arch/run and scripts/runtime, and having
scripts/mkstandalone.sh::generate_test() copy it directly in the final test
script. Didn't catch that during the previous iterations because I was
running the standalone tests from the top level source directory, and
"source scripts/vmm.bash" happened to work.

More details in the changelog for the modified patches.

[1] https://lore.kernel.org/kvm/20250507151256.167769-1-alexandru.elisei@arm.com/
[2] https://lore.kernel.org/kvm/20250625154354.27015-1-alexandru.elisei@arm.com/

Alexandru Elisei (13):
  run_tests.sh: Document --probe-maxsmp argument
  scripts: Document environment variables
  scripts: Refuse to run the tests if not configured for qemu
  scripts: Use an associative array for qemu argument names
  scripts: Add 'kvmtool_params' to test definition
  scripts: Add support for kvmtool
  scripts: Add default arguments for kvmtool
  scripts: Add KVMTOOL environment variable for kvmtool binary path
  scripts: Detect kvmtool failure in premature_failure()
  scripts: Do not probe for maximum number of VCPUs when using kvmtool
  scripts/mkstandalone: Export $TARGET
  scripts: Add 'disabled_if' test definition parameter for kvmtool to
    use
  scripts: Enable kvmtool

 README.md               |  18 +++-
 arm/efi/run             |   8 ++
 arm/run                 | 161 ++++++++++++++++-----------
 arm/unittests.cfg       |  31 ++++++
 configure               |   1 -
 docs/unittests.txt      |  26 ++++-
 powerpc/run             |   5 +-
 riscv/run               |   5 +-
 run_tests.sh            |  35 +++---
 s390x/run               |   3 +-
 scripts/arch-run.bash   | 112 +++++++------------
 scripts/common.bash     |  30 ++++--
 scripts/mkstandalone.sh |   8 +-
 scripts/runtime.bash    |  35 ++----
 scripts/vmm.bash        | 234 ++++++++++++++++++++++++++++++++++++++++
 x86/run                 |   5 +-
 16 files changed, 525 insertions(+), 192 deletions(-)
 create mode 100644 scripts/vmm.bash

-- 
2.50.0

-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

^ permalink raw reply	[flat|nested] 26+ messages in thread