Re: [RFC V2 PATCH 0/8] selftests: KVM: selftests for fd-based approach of supporting private memory

[Sean Christopherson <seanjc@google.com>]

On Wed, May 11, 2022, Vishal Annapurve wrote:
>  tools/testing/selftests/kvm/priv_memfd_test.c | 1359 +++++++++++++++++

Please don't create a megatest.  We have megatests for nVMX and nSVM in KVM-Unit-Test
and IMO they are a mistake.  E.g. to run a single test instead of the entire suite,
the KUT tests provide a funky wildcard/filter syntax.  But the names of the tests
aren't discoverable, so inevitably I have to look at the source code to figure out
the exact name of the test I want to run.  And don't get me started on sub-tests
within sub-tests...

AFAICT, what you've proposed here doesn't provide any such filter mechanism.  And
I would rather we NOT start adding those to selftests, because we'd effectively be
reinventing the wheel _and_ dealing with strings in C is a pain.  Writing a script
to find and run all tests is trivial, e.g. grep the .gitignore to find tests for
the target arch.  Or when the system under test is different than the build system,
copy the binaries to a dedicated directory and run every binary in that directory.

Discovering and running a single test is similarly trivial.  For KUT, it's less
trivial because running a test involves invoking a VMM command line, and some of
the tests need specific command line parameters.  But for selftests, except for the
NX huge page test, they're all standalone and don't need additional setup.

And unlike KUT's nVMX and nSVM tests, which involve running hundreds of little
sub-tests with only minor differences in setup, these tests are largely independent,
i.e. you're not really getting much code reuse.

And if you split the tests up, then all of the inter-test namespacing goes away,
e.g. there is zero chance I will ever remember what "PSPAHCT" stands for.

+#define PSPAHCT_GUEST_STARTED                          0ULL
+#define PSPAHCT_GUEST_PRIVATE_MEM_UPDATED              1ULL
+#define PSPAHCT_GUEST_SHARED_MEM_UPDATED               2ULL
+#define PSPAHCT_GUEST_PRIVATE_MEM_UPDATED2             3ULL

If you find yourself doing a lot of copy+paste, then we should enhance the APIs
provided by the core infrastructure.