* Re: [kvm-unit-tests PATCH] x86: vmx: Break the 'vmx' monolith into multiple tests
2022-05-02 16:40 [kvm-unit-tests PATCH] x86: vmx: Break the 'vmx' monolith into multiple tests Aaron Lewis
@ 2022-05-02 23:28 ` Sean Christopherson
0 siblings, 0 replies; 2+ messages in thread
From: Sean Christopherson @ 2022-05-02 23:28 UTC (permalink / raw)
To: Aaron Lewis; +Cc: kvm, pbonzini, drjones, jmattson
On Mon, May 02, 2022, Aaron Lewis wrote:
> Break 'vmx' into multiple tests to reduce the number of tests run at
> once. This will help get a better signal up front on failure with them
> broken up into categories. This will also do a better job of not
> disguising new failures if one of the tests is already failing.
>
> One side effect of breaking this test up is any new test added will
> have to be manually added to a categorized test. It will not
> automatically be picked up by 'vmx'.
>
> Signed-off-by: Aaron Lewis <aaronlewis@google.com>
> ---
> x86/unittests.cfg | 42 +++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 39 insertions(+), 3 deletions(-)
>
> diff --git a/x86/unittests.cfg b/x86/unittests.cfg
> index b48c98b..0d90413 100644
> --- a/x86/unittests.cfg
> +++ b/x86/unittests.cfg
> @@ -254,13 +254,49 @@ extra_params = -cpu qemu64,+umip
> file = la57.flat
> arch = i386
>
> -[vmx]
> +[vmx_legacy]
> file = vmx.flat
> -extra_params = -cpu host,+vmx -append "-exit_monitor_from_l2_test -ept_access* -vmx_smp* -vmx_vmcs_shadow_test -atomic_switch_overflow_msrs_test -vmx_init_signal_test -vmx_apic_passthrough_tpr_threshold_test"
> +extra_params = -cpu max,+vmx -append "null vmenter preemption_timer control_field_PAT control_field_EFER CR_shadowing I/O_bitmap instruction_intercept EPT_A/D_disabled EPT_A/D_enabled PML interrupt nmi_hlt debug_controls MSR_switch vmmcall disable_RDTSCP int3 into invalid_msr"
> arch = x86_64
> groups = vmx
>
> -[ept]
> +[vmx_basic]
> +file = vmx.flat
> +extra_params = -cpu max,+vmx -append "v2_null_test v2_multiple_entries_test fixture_test_case1 fixture_test_case2"
> +arch = x86_64
> +groups = vmx
> +
> +[vmx_vm_entry]
> +file = vmx.flat
> +extra_params = -cpu max,+vmx -append "vmx_controls_test vmx_host_state_area_test vmx_guest_state_area_test vmentry_movss_shadow_test vmentry_unrestricted_guest_test"
> +arch = x86_64
> +groups = vmx
> +
> +[vmx_apic]
> +file = vmx.flat
> +extra_params = -cpu max,+vmx -append "vmx_eoi_bitmap_ioapic_scan_test vmx_hlt_with_rvi_test vmx_apic_passthrough_test vmx_apic_passthrough_thread_test vmx_sipi_signal_test"
> +arch = x86_64
> +groups = vmx
> +
> +[vmx_regression]
> +file = vmx.flat
> +extra_params = -cpu max,+vmx -append "vmx_ldtr_test vmx_cr_load_test vmx_cr4_osxsave_test vmx_nm_test vmx_db_test vmx_nmi_window_test vmx_intr_window_test vmx_pending_event_test vmx_pending_event_hlt_test vmx_store_tsc_test"
> +arch = x86_64
> +groups = vmx
> +
> +[vmx_preemption_timer]
> +file = vmx.flat
> +extra_params = -cpu max,+vmx -append "vmx_preemption_timer_zero_test vmx_preemption_timer_tf_test vmx_preemption_timer_expiry_test"
> +arch = x86_64
> +groups = vmx
> +
> +[vmx_misc]
> +file = vmx.flat
> +extra_params = -cpu max,+vmx -append "invvpid_test atomic_switch_max_msrs_test rdtsc_vmexit_diff_test vmx_mtf_test vmx_mtf_pdpte_test vmx_exception_test"
> +arch = x86_64
> +groups = vmx
> +
> +[vmx_ept]
> file = vmx.flat
> extra_params = -cpu host,host-phys-bits,+vmx -m 2560 -append "ept_access*"
> arch = x86_64
hmm, I don't like that there's no common criteria for the buckets. E.g.
"regression" is a bucket for how a test came to exist, whereas "apic" is a bucket
for what a test exercises. And the granularity ends up being odd too, e.g. the
preemption timer gets its own bucket, but "regression" has a massive variety of
tests.
My preference would be to separate tests by what they cover, which I think will be
most useful. E.g. if I'm making changes to EPT/paging related stuff, it would be
nice to be able to run just the paging tests early on. And when a bucket fails,
it'd also give the user a hint as to want might have gone wrong. I'm sure there
are tests that straddle multiple buckets, so it won't be perfect, but I think we
can get something useful.
vmx_cpu (don't like the name, want it to be for tests that verify CPU arch behavior)
vmx_consistency_checks
vmx_events (non-vAPIC APIC tests, timers, NMIs, IRQs, preemption timer, etc...)
vmx_paging (ept A/D, PML, invvpid, vmx_pf_exception_test, etc...)
vmx_vapic (vAPIC, VID, PI, etc...)
ept_access (because they probably needs their own config, but they aren't the only EPT tests)
^ permalink raw reply [flat|nested] 2+ messages in thread