Re: [PATCH v3 9/9] KVM: selftests: Provide README.rst for KVM selftests runner

[Vipin Sharma <vipinsh@google.com>]

On 2025-10-01 09:44:22, Marc Zyngier wrote:
> On Tue, 30 Sep 2025 17:36:35 +0100,
> Vipin Sharma <vipinsh@google.com> wrote:
> > 
> > +KVM selftest runner is highly configurable test executor that allows to run
> > +tests with different configurations (not just the default), parallely, save
> 
> s/parallely/in parallel/
> 

Thanks, I will fix it.

> > +output to disk hierarchically, control what gets printed on console, provide
> > +execution status.
> > +
> > +To generate default tests use::
> > +
> > +  # make tests_install
> > +
> > +This will create ``testcases_default_gen`` directory which will have testcases
> 
> I don't think using the future tense is correct here. I'd rather see
> something written in the present tense, possibly imperative. For
> example:
> 
> "Create 'blah' directory containing 'foo' files, one per test-case.
> 

Thanks, I will fix it.

> > +in `default.test` files. Each KVM selftest will have a directory in  which
> > +`default.test` file will be created with executable path relative to KVM
> > +selftest root directory i.e. `/tools/testing/selftests/kvm`.
> 
> Shouldn't this honor the existing build output directives? If it
> actually does, then you want to call this out.
> 

To generate default test files in a specific directory one can use
"OUTPUT" in the make command

  make OUTPUT="~/test/directory/path" tests_install

This allows to generate testcases_default_gen in the given output
directory. default.test files will still have test binary path relative
kvm selftest root directory. 

$OUTPUT
└── testcases_default_gen
    ├── access_tracking_perf_test
    │   └── default.test
    ├── arch_timer
    │   └── default.test
    ├── arm64
    │   ├── aarch32_id_regs
    │   │   └── default.test
    │   ├── arch_timer_edge_cases
    │   │   └── default.test
    │   ├── debug-exceptions
    │   │   └── default.test
    │   ├── external_aborts
    │   │   └── default.test
    │   │   └── default.test
    │   └── ...
    ├── coalesced_io_test
    │   └── default.test
    ├── demand_paging_test
    │   └── default.test
    ├── ...

So, arm64/aarch32_id_regs/default.test will have 'arm64/aarch32_id_regs'

User can then supply -p/--path with the path of build output directory
to runner. 

  python3 runner -p ~/path/to/selftest/binaries -d $OUTPUT/testcases_default_gen

If -p not given then current directory is considered for test
executables.

> > For example, the
> > +`dirty_log_perf_test` will have::
> > +
> > +  # cat testcase_default_gen/dirty_log_perf_test/default.test
> > +  dirty_log_perf_test
> > +
> > +Runner will execute `dirty_log_perf_test`. Testcases files can also provide
> > +extra arguments to the test::
> > +
> > +  # cat tests/dirty_log_perf_test/2slot_5vcpu_10iter.test
> > +  dirty_log_perf_test -x 2 -v 5 -i 10
> > +
> > +In this case runner will execute the `dirty_log_perf_test` with the options.
> > +
> 
> The beginning of the text talks about "non-default' configurations,
> but you only seem to talk about the default stuff. How does one deals
> with a non-default config?
> 

In the patch 1, I created two sample tests files,
2slot_5vcpu_10iter.test and no_dirty_log_protect.test, in the directory
tools/testing/selftests/kvm/tests/dirty_log_perf_test.

Contents of those files provide non-default arguments to test, for example,
2slot_5vcpu10iter.test has the command:

  dirty_log_perf_test -x 2 -v 5  -i 10

One can run these non-default tests as (assuming current directory is
kvm selftests):

  python3 runner -d ./tests

Over the time we will add more of these non-default interesting
testcases. One can then run:

  python3 runner -d ./tests ./testcases_default_gen