Re: [PATCH v3 9/9] KVM: selftests: Provide README.rst for KVM selftests runner

Vipin Sharma
2025-10-01T17:32:31Z

On 2025-10-01 09:44:22, Marc Zyngier wrote:
> On Tue, 30 Sep 2025 17:36:35 +0100,
> Vipin Sharma <vipinsh@google.com> wrote:
> >
> > +KVM selftest runner is highly configurable test executor that allows to run
> > +tests with different configurations (not just the default), parallely, save
>
> s/parallely/in parallel/
>

Thanks, I will fix it.

> > +output to disk hierarchically, control what gets printed on console, provide
> > +execution status.
> > +
> > +To generate default tests use::
> > +
> > + # make tests_install
> > +
> > +This will create ``testcases_default_gen`` directory which will have testcases
>
> I don't think using the future tense is correct here. I'd rather see
> something written in the present tense, possibly imperative. For
> example:
>
> "Create 'blah' directory containing 'foo' files, one per test-case.
>

Thanks, I will fix it.

> > +in `default.test` files. Each KVM selftest will have a directory in which
> > +`default.test` file will be created with executable path relative to KVM
> > +selftest root directory i.e. `/tools/testing/selftests/kvm`.
>
> Shouldn't this honor the existing build output directives? If it
> actually does, then you want to call this out.
>

To generate default test files in a specific directory one can use
"OUTPUT" in the make command

make OUTPUT="~/test/directory/path" tests_install

This allows to generate testcases_default_gen in the given output
directory. default.test files will still have test binary path relative
kvm selftest root directory.

$OUTPUT
└── testcases_default_gen
├── access_tracking_perf_test
│ └── default.test
├── arch_timer
│ └── default.test
├── arm64
│ ├── aarch32_id_regs
│ │ └── default.test
│ ├── arch_timer_edge_cases
│ │ └── default.test
│ ├── debug-exceptions
│ │ └── default.test
│ ├── external_aborts
│ │ └── default.test
│ │ └── default.test
│ └── ...
├── coalesced_io_test
│ └── default.test
├── demand_paging_test
│ └── default.test
├── ...

So, arm64/aarch32_id_regs/default.test will have 'arm64/aarch32_id_regs'

User can then supply -p/--path with the path of build output directory
to runner.

python3 runner -p ~/path/to/selftest/binaries -d $OUTPUT/testcases_default_gen

If -p not given then current directory is considered for test
executables.

> > For example, the
> > +`dirty_log_perf_test` will have::
> > +
> > + # cat testcase_default_gen/dirty_log_perf_test/default.test
> > + dirty_log_perf_test
> > +
> > +Runner will execute `dirty_log_perf_test`. Testcases files can also provide
> > +extra arguments to the test::
> > +
> > + # cat tests/dirty_log_perf_test/2slot_5vcpu_10iter.test
> > + dirty_log_perf_test -x 2 -v 5 -i 10
> > +
> > +In this case runner will execute the `dirty_log_perf_test` with the options.
> > +
>
> The beginning of the text talks about "non-default' configurations,
> but you only seem to talk about the default stuff. How does one deals
> with a non-default config?
>

In the patch 1, I created two sample tests files,
2slot_5vcpu_10iter.test and no_dirty_log_protect.test, in the directory
tools/testing/selftests/kvm/tests/dirty_log_perf_test.

Contents of those files provide non-default arguments to test, for example,
2slot_5vcpu10iter.test has the command:

dirty_log_perf_test -x 2 -v 5 -i 10

One can run these non-default tests as (assuming current directory is
kvm selftests):

python3 runner -d ./tests

Over the time we will add more of these non-default interesting
testcases. One can then run:

python3 runner -d ./tests ./testcases_default_gen