* [PATCH 0/2] Add KVM selftest runner
@ 2025-02-22 0:59 Vipin Sharma
2025-02-22 0:59 ` [PATCH 1/2] KVM: selftests: Add default testfiles for KVM selftests runner Vipin Sharma
2025-02-22 0:59 ` [PATCH 2/2] KVM: selftests: Create KVM selftest runner Vipin Sharma
0 siblings, 2 replies; 9+ messages in thread
From: Vipin Sharma @ 2025-02-22 0:59 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, Vipin Sharma
Hello,
This series in continuation of the selftest runner discussion we had
some months ago.
https://lore.kernel.org/kvm/20240821223012.3757828-1-vipinsh@google.com/
Based on the discussion, this is phase 1 implementation of KVM selftest
runner. Patch 1 is providing the testcase files and patch 2 is runner
implementation which uses testcase files.
This version has following features:
- Parallel test execution.
- Dumping separate output for each test.
- Timeout for test execution
- Specify single test or a test directory.
Vipin Sharma (2):
KVM: selftests: Add default testfiles for KVM selftests runner
KVM: selftests: Create KVM selftest runner
tools/testing/selftests/kvm/.gitignore | 4 +-
.../testing/selftests/kvm/runner/__main__.py | 96 +++++++++++++++++++
tools/testing/selftests/kvm/runner/command.py | 42 ++++++++
.../testing/selftests/kvm/runner/selftest.py | 49 ++++++++++
.../selftests/kvm/runner/test_runner.py | 40 ++++++++
.../aarch64/aarch32_id_regs/default.test | 1 +
.../testcases/aarch64/arch_timer/default.test | 1 +
.../arch_timer_edge_cases/default.test | 1 +
.../aarch64/debug-exceptions/default.test | 1 +
.../aarch64/get-reg-list/default.test | 1 +
.../testcases/aarch64/hypercalls/default.test | 1 +
.../testcases/aarch64/no-vgic-v3/default.test | 1 +
.../aarch64/page_fault_test/default.test | 1 +
.../testcases/aarch64/psci_test/default.test | 1 +
.../aarch64/set_id_regs/default.test | 1 +
.../aarch64/smccc_filter/default.test | 1 +
.../aarch64/vcpu_width_config/default.test | 1 +
.../testcases/aarch64/vgic_init/default.test | 1 +
.../testcases/aarch64/vgic_irq/default.test | 1 +
.../aarch64/vgic_lpi_stress/default.test | 1 +
.../aarch64/vpmu_counter_access/default.test | 1 +
.../access_tracking_perf_test/default.test | 1 +
.../testcases/coalesced_io_test/default.test | 1 +
.../testcases/demand_paging_test/default.test | 1 +
.../dirty_log_perf_test/default.test | 1 +
.../kvm/testcases/dirty_log_test/default.test | 1 +
.../testcases/guest_memfd_test/default.test | 1 +
.../testcases/guest_print_test/default.test | 1 +
.../hardware_disable_test/default.test | 1 +
.../kvm_binary_stats_test/default.test | 1 +
.../kvm_create_max_vcpus/default.test | 1 +
.../kvm_page_table_test/default.test | 1 +
.../max_guest_memory_test/default.test | 1 +
.../default.test | 1 +
.../testcases/memslot_perf_test/default.test | 1 +
.../pre_fault_memory_test/default.test | 1 +
.../testcases/riscv/arch_timer/default.test | 1 +
.../testcases/riscv/ebreak_test/default.test | 1 +
.../testcases/riscv/get-reg-list/default.test | 1 +
.../testcases/riscv/sbi_pmu_test/default.test | 1 +
.../kvm/testcases/rseq_test/default.test | 1 +
.../testcases/s390x/cmma_test/default.test | 1 +
.../testcases/s390x/debug_test/default.test | 1 +
.../kvm/testcases/s390x/memop/default.test | 1 +
.../kvm/testcases/s390x/resets/default.test | 1 +
.../s390x/shared_zeropage_test/default.test | 1 +
.../s390x/sync_regs_test/default.test | 1 +
.../kvm/testcases/s390x/tprot/default.test | 1 +
.../s390x/ucontrol_test/default.test | 1 +
.../set_memory_region_test/default.test | 1 +
.../kvm/testcases/steal_time/default.test | 1 +
.../system_counter_offset_test/default.test | 1 +
.../testcases/x86_64/amx_test/default.test | 1 +
.../x86_64/apic_bus_clock_test/default.test | 1 +
.../testcases/x86_64/cpuid_test/default.test | 1 +
.../x86_64/cr4_cpuid_sync_test/default.test | 1 +
.../testcases/x86_64/debug_regs/default.test | 1 +
.../default.test | 1 +
.../default.test | 1 +
.../x86_64/feature_msrs_test/default.test | 1 +
.../x86_64/fix_hypercall_test/default.test | 1 +
.../x86_64/hwcr_msr_test/default.test | 1 +
.../x86_64/hyperv_clock/default.test | 1 +
.../x86_64/hyperv_cpuid/default.test | 1 +
.../x86_64/hyperv_evmcs/default.test | 1 +
.../hyperv_extended_hypercalls/default.test | 1 +
.../x86_64/hyperv_features/default.test | 1 +
.../testcases/x86_64/hyperv_ipi/default.test | 1 +
.../x86_64/hyperv_svm_test/default.test | 1 +
.../x86_64/hyperv_tlb_flush/default.test | 1 +
.../x86_64/kvm_clock_test/default.test | 1 +
.../testcases/x86_64/kvm_pv_test/default.test | 1 +
.../x86_64/max_vcpuid_cap_test/default.test | 1 +
.../x86_64/monitor_mwait_test/default.test | 1 +
.../nested_exceptions_test/default.test | 1 +
.../x86_64/nx_huge_pages_test/default.test | 1 +
.../x86_64/platform_info_test/default.test | 1 +
.../x86_64/pmu_counters_test/default.test | 1 +
.../x86_64/pmu_event_filter_test/default.test | 1 +
.../private_mem_conversions_test/default.test | 1 +
.../private_mem_kvm_exits_test/default.test | 1 +
.../x86_64/recalc_apic_map_test/default.test | 1 +
.../x86_64/set_boot_cpu_id/default.test | 1 +
.../x86_64/set_sregs_test/default.test | 1 +
.../x86_64/sev_init2_tests/default.test | 1 +
.../x86_64/sev_migrate_tests/default.test | 1 +
.../x86_64/sev_smoke_test/default.test | 1 +
.../default.test | 1 +
.../testcases/x86_64/smm_test/default.test | 1 +
.../testcases/x86_64/state_test/default.test | 1 +
.../x86_64/svm_int_ctl_test/default.test | 1 +
.../svm_nested_shutdown_test/default.test | 1 +
.../svm_nested_soft_inject_test/default.test | 1 +
.../x86_64/svm_vmcall_test/default.test | 1 +
.../x86_64/sync_regs_test/default.test | 1 +
.../triple_fault_event_test/default.test | 1 +
.../x86_64/tsc_msrs_test/default.test | 1 +
.../x86_64/tsc_scaling_sync/default.test | 1 +
.../x86_64/ucna_injection_test/default.test | 1 +
.../x86_64/userspace_io_test/default.test | 1 +
.../userspace_msr_exit_test/default.test | 1 +
.../x86_64/vmx_apic_access_test/default.test | 1 +
.../vmx_close_while_nested_test/default.test | 1 +
.../x86_64/vmx_dirty_log_test/default.test | 1 +
.../default.test | 1 +
.../default.test | 1 +
.../x86_64/vmx_msrs_test/default.test | 1 +
.../vmx_nested_tsc_scaling_test/default.test | 1 +
.../x86_64/vmx_pmu_caps_test/default.test | 1 +
.../vmx_preemption_timer_test/default.test | 1 +
.../vmx_set_nested_state_test/default.test | 1 +
.../x86_64/vmx_tsc_adjust_test/default.test | 1 +
.../x86_64/xapic_ipi_test/default.test | 1 +
.../x86_64/xapic_state_test/default.test | 1 +
.../x86_64/xcr0_cpuid_test/default.test | 1 +
.../x86_64/xen_shinfo_test/default.test | 1 +
.../x86_64/xen_vmcall_test/default.test | 1 +
.../x86_64/xss_msr_test/default.test | 1 +
118 files changed, 343 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/kvm/runner/__main__.py
create mode 100644 tools/testing/selftests/kvm/runner/command.py
create mode 100644 tools/testing/selftests/kvm/runner/selftest.py
create mode 100644 tools/testing/selftests/kvm/runner/test_runner.py
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/arch_timer/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/arch_timer_edge_cases/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/debug-exceptions/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/get-reg-list/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/hypercalls/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/no-vgic-v3/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/page_fault_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/psci_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/set_id_regs/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/smccc_filter/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/vcpu_width_config/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/vgic_init/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/vgic_irq/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/vgic_lpi_stress/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/vpmu_counter_access/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/access_tracking_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/coalesced_io_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/demand_paging_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/dirty_log_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/dirty_log_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/guest_memfd_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/guest_print_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/hardware_disable_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/kvm_binary_stats_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/kvm_create_max_vcpus/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/kvm_page_table_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/max_guest_memory_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/memslot_modification_stress_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/memslot_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/pre_fault_memory_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/riscv/arch_timer/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/riscv/ebreak_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/riscv/get-reg-list/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/riscv/sbi_pmu_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/rseq_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/cmma_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/debug_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/memop/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/resets/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/shared_zeropage_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/sync_regs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/tprot/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/ucontrol_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/set_memory_region_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/steal_time/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/system_counter_offset_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/amx_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/apic_bus_clock_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/cpuid_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/cr4_cpuid_sync_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/debug_regs/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/dirty_log_page_splitting_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/exit_on_emulation_failure_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/feature_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/fix_hypercall_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hwcr_msr_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_clock/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_cpuid/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_evmcs/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_extended_hypercalls/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_features/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_ipi/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_svm_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_tlb_flush/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/kvm_clock_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/kvm_pv_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/max_vcpuid_cap_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/monitor_mwait_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/nested_exceptions_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/nx_huge_pages_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/platform_info_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/pmu_counters_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/pmu_event_filter_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/private_mem_conversions_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/private_mem_kvm_exits_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/recalc_apic_map_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/set_boot_cpu_id/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/set_sregs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/sev_init2_tests/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/sev_migrate_tests/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/sev_smoke_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/smaller_maxphyaddr_emulation_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/smm_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/state_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/svm_int_ctl_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/svm_nested_shutdown_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/svm_nested_soft_inject_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/svm_vmcall_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/sync_regs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/triple_fault_event_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/tsc_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/tsc_scaling_sync/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/ucna_injection_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/userspace_io_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/userspace_msr_exit_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_apic_access_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_close_while_nested_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_dirty_log_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_exception_with_invalid_guest_state/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_invalid_nested_guest_state/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_nested_tsc_scaling_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_pmu_caps_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_preemption_timer_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_set_nested_state_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_tsc_adjust_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xapic_ipi_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xapic_state_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xcr0_cpuid_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xen_shinfo_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xen_vmcall_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xss_msr_test/default.test
base-commit: fed48e2967f402f561d80075a20c5c9e16866e53
--
2.48.1.601.g30ceb7b040-goog
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/2] KVM: selftests: Add default testfiles for KVM selftests runner
2025-02-22 0:59 [PATCH 0/2] Add KVM selftest runner Vipin Sharma
@ 2025-02-22 0:59 ` Vipin Sharma
2025-04-30 17:01 ` Sean Christopherson
2025-02-22 0:59 ` [PATCH 2/2] KVM: selftests: Create KVM selftest runner Vipin Sharma
1 sibling, 1 reply; 9+ messages in thread
From: Vipin Sharma @ 2025-02-22 0:59 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, Vipin Sharma
Create a root "testcases" folder in KVM selftests directory. Add test
files for all of the KVM selftests across all of the supported
platforms. Write only default test execution command in the test files.
Commands written in the test files will be ran by KVM selftest runner.
Test files are organized based following rules:
1. All test files resides in "testcases" directory.
2. Test files should have .test extension. This is needed so that
git doesn't ignore the test file changes.
3. Each KVM selftest resides in a folder in "testcases" directory.
It follows the path of KVM selftests directory. For example,
kvm/x86_64/vmx_msrs_test.c will be in
kvm/testcases/x86_64/vmx_msrs_tests directory.
4. default.test name is reserved for the default command to execute the
test.
5. Different configuration of the tests should reside in their own test
files under the same test directory. For example dirty_log_perf_test
can have:
- testcases/dirty_log_perf_test/default.test
- testcases/dirty_log_perf_test/hugetlb1g.test
- testcases/dirty_log_perf_test/disable_dirty_log_manual.test
6. If there is an arch specific option of a common test then it should
be specified under an arch name directory in the test directory. This
will avoid test runner to execute the common test or its option on
unsupported machine. For example:
testcases/memslot_modification_stress_test/x86_64/disable_slot_zap_quirk.test
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
tools/testing/selftests/kvm/.gitignore | 3 ++-
.../kvm/testcases/aarch64/aarch32_id_regs/default.test | 1 +
.../selftests/kvm/testcases/aarch64/arch_timer/default.test | 1 +
.../kvm/testcases/aarch64/arch_timer_edge_cases/default.test | 1 +
.../kvm/testcases/aarch64/debug-exceptions/default.test | 1 +
.../selftests/kvm/testcases/aarch64/get-reg-list/default.test | 1 +
.../selftests/kvm/testcases/aarch64/hypercalls/default.test | 1 +
.../selftests/kvm/testcases/aarch64/no-vgic-v3/default.test | 1 +
.../kvm/testcases/aarch64/page_fault_test/default.test | 1 +
.../selftests/kvm/testcases/aarch64/psci_test/default.test | 1 +
.../selftests/kvm/testcases/aarch64/set_id_regs/default.test | 1 +
.../selftests/kvm/testcases/aarch64/smccc_filter/default.test | 1 +
.../kvm/testcases/aarch64/vcpu_width_config/default.test | 1 +
.../selftests/kvm/testcases/aarch64/vgic_init/default.test | 1 +
.../selftests/kvm/testcases/aarch64/vgic_irq/default.test | 1 +
.../kvm/testcases/aarch64/vgic_lpi_stress/default.test | 1 +
.../kvm/testcases/aarch64/vpmu_counter_access/default.test | 1 +
.../kvm/testcases/access_tracking_perf_test/default.test | 1 +
.../selftests/kvm/testcases/coalesced_io_test/default.test | 1 +
.../selftests/kvm/testcases/demand_paging_test/default.test | 1 +
.../selftests/kvm/testcases/dirty_log_perf_test/default.test | 1 +
.../selftests/kvm/testcases/dirty_log_test/default.test | 1 +
.../selftests/kvm/testcases/guest_memfd_test/default.test | 1 +
.../selftests/kvm/testcases/guest_print_test/default.test | 1 +
.../selftests/kvm/testcases/hardware_disable_test/default.test | 1 +
.../selftests/kvm/testcases/kvm_binary_stats_test/default.test | 1 +
.../selftests/kvm/testcases/kvm_create_max_vcpus/default.test | 1 +
.../selftests/kvm/testcases/kvm_page_table_test/default.test | 1 +
.../selftests/kvm/testcases/max_guest_memory_test/default.test | 1 +
.../testcases/memslot_modification_stress_test/default.test | 1 +
.../selftests/kvm/testcases/memslot_perf_test/default.test | 1 +
.../selftests/kvm/testcases/pre_fault_memory_test/default.test | 1 +
.../selftests/kvm/testcases/riscv/arch_timer/default.test | 1 +
.../selftests/kvm/testcases/riscv/ebreak_test/default.test | 1 +
.../selftests/kvm/testcases/riscv/get-reg-list/default.test | 1 +
.../selftests/kvm/testcases/riscv/sbi_pmu_test/default.test | 1 +
tools/testing/selftests/kvm/testcases/rseq_test/default.test | 1 +
.../selftests/kvm/testcases/s390x/cmma_test/default.test | 1 +
.../selftests/kvm/testcases/s390x/debug_test/default.test | 1 +
tools/testing/selftests/kvm/testcases/s390x/memop/default.test | 1 +
.../testing/selftests/kvm/testcases/s390x/resets/default.test | 1 +
.../kvm/testcases/s390x/shared_zeropage_test/default.test | 1 +
.../selftests/kvm/testcases/s390x/sync_regs_test/default.test | 1 +
tools/testing/selftests/kvm/testcases/s390x/tprot/default.test | 1 +
.../selftests/kvm/testcases/s390x/ucontrol_test/default.test | 1 +
.../kvm/testcases/set_memory_region_test/default.test | 1 +
tools/testing/selftests/kvm/testcases/steal_time/default.test | 1 +
.../kvm/testcases/system_counter_offset_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/amx_test/default.test | 1 +
.../kvm/testcases/x86_64/apic_bus_clock_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/cpuid_test/default.test | 1 +
.../kvm/testcases/x86_64/cr4_cpuid_sync_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/debug_regs/default.test | 1 +
.../x86_64/dirty_log_page_splitting_test/default.test | 1 +
.../x86_64/exit_on_emulation_failure_test/default.test | 1 +
.../kvm/testcases/x86_64/feature_msrs_test/default.test | 1 +
.../kvm/testcases/x86_64/fix_hypercall_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/hwcr_msr_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/hyperv_clock/default.test | 1 +
.../selftests/kvm/testcases/x86_64/hyperv_cpuid/default.test | 1 +
.../selftests/kvm/testcases/x86_64/hyperv_evmcs/default.test | 1 +
.../testcases/x86_64/hyperv_extended_hypercalls/default.test | 1 +
.../kvm/testcases/x86_64/hyperv_features/default.test | 1 +
.../selftests/kvm/testcases/x86_64/hyperv_ipi/default.test | 1 +
.../kvm/testcases/x86_64/hyperv_svm_test/default.test | 1 +
.../kvm/testcases/x86_64/hyperv_tlb_flush/default.test | 1 +
.../selftests/kvm/testcases/x86_64/kvm_clock_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/kvm_pv_test/default.test | 1 +
.../kvm/testcases/x86_64/max_vcpuid_cap_test/default.test | 1 +
.../kvm/testcases/x86_64/monitor_mwait_test/default.test | 1 +
.../kvm/testcases/x86_64/nested_exceptions_test/default.test | 1 +
.../kvm/testcases/x86_64/nx_huge_pages_test/default.test | 1 +
.../kvm/testcases/x86_64/platform_info_test/default.test | 1 +
.../kvm/testcases/x86_64/pmu_counters_test/default.test | 1 +
.../kvm/testcases/x86_64/pmu_event_filter_test/default.test | 1 +
.../testcases/x86_64/private_mem_conversions_test/default.test | 1 +
.../testcases/x86_64/private_mem_kvm_exits_test/default.test | 1 +
.../kvm/testcases/x86_64/recalc_apic_map_test/default.test | 1 +
.../kvm/testcases/x86_64/set_boot_cpu_id/default.test | 1 +
.../selftests/kvm/testcases/x86_64/set_sregs_test/default.test | 1 +
.../kvm/testcases/x86_64/sev_init2_tests/default.test | 1 +
.../kvm/testcases/x86_64/sev_migrate_tests/default.test | 1 +
.../selftests/kvm/testcases/x86_64/sev_smoke_test/default.test | 1 +
.../x86_64/smaller_maxphyaddr_emulation_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/smm_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/state_test/default.test | 1 +
.../kvm/testcases/x86_64/svm_int_ctl_test/default.test | 1 +
.../kvm/testcases/x86_64/svm_nested_shutdown_test/default.test | 1 +
.../testcases/x86_64/svm_nested_soft_inject_test/default.test | 1 +
.../kvm/testcases/x86_64/svm_vmcall_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/sync_regs_test/default.test | 1 +
.../kvm/testcases/x86_64/triple_fault_event_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/tsc_msrs_test/default.test | 1 +
.../kvm/testcases/x86_64/tsc_scaling_sync/default.test | 1 +
.../kvm/testcases/x86_64/ucna_injection_test/default.test | 1 +
.../kvm/testcases/x86_64/userspace_io_test/default.test | 1 +
.../kvm/testcases/x86_64/userspace_msr_exit_test/default.test | 1 +
.../kvm/testcases/x86_64/vmx_apic_access_test/default.test | 1 +
.../testcases/x86_64/vmx_close_while_nested_test/default.test | 1 +
.../kvm/testcases/x86_64/vmx_dirty_log_test/default.test | 1 +
.../x86_64/vmx_exception_with_invalid_guest_state/default.test | 1 +
.../x86_64/vmx_invalid_nested_guest_state/default.test | 1 +
.../selftests/kvm/testcases/x86_64/vmx_msrs_test/default.test | 1 +
.../testcases/x86_64/vmx_nested_tsc_scaling_test/default.test | 1 +
.../kvm/testcases/x86_64/vmx_pmu_caps_test/default.test | 1 +
.../testcases/x86_64/vmx_preemption_timer_test/default.test | 1 +
.../testcases/x86_64/vmx_set_nested_state_test/default.test | 1 +
.../kvm/testcases/x86_64/vmx_tsc_adjust_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/xapic_ipi_test/default.test | 1 +
.../kvm/testcases/x86_64/xapic_state_test/default.test | 1 +
.../kvm/testcases/x86_64/xcr0_cpuid_test/default.test | 1 +
.../kvm/testcases/x86_64/xen_shinfo_test/default.test | 1 +
.../kvm/testcases/x86_64/xen_vmcall_test/default.test | 1 +
.../selftests/kvm/testcases/x86_64/xss_msr_test/default.test | 1 +
114 files changed, 115 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/arch_timer/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/arch_timer_edge_cases/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/debug-exceptions/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/get-reg-list/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/hypercalls/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/no-vgic-v3/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/page_fault_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/psci_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/set_id_regs/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/smccc_filter/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/vcpu_width_config/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/vgic_init/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/vgic_irq/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/vgic_lpi_stress/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/aarch64/vpmu_counter_access/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/access_tracking_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/coalesced_io_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/demand_paging_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/dirty_log_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/dirty_log_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/guest_memfd_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/guest_print_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/hardware_disable_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/kvm_binary_stats_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/kvm_create_max_vcpus/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/kvm_page_table_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/max_guest_memory_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/memslot_modification_stress_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/memslot_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/pre_fault_memory_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/riscv/arch_timer/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/riscv/ebreak_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/riscv/get-reg-list/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/riscv/sbi_pmu_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/rseq_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/cmma_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/debug_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/memop/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/resets/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/shared_zeropage_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/sync_regs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/tprot/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/s390x/ucontrol_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/set_memory_region_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/steal_time/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/system_counter_offset_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/amx_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/apic_bus_clock_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/cpuid_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/cr4_cpuid_sync_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/debug_regs/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/dirty_log_page_splitting_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/exit_on_emulation_failure_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/feature_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/fix_hypercall_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hwcr_msr_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_clock/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_cpuid/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_evmcs/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_extended_hypercalls/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_features/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_ipi/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_svm_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/hyperv_tlb_flush/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/kvm_clock_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/kvm_pv_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/max_vcpuid_cap_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/monitor_mwait_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/nested_exceptions_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/nx_huge_pages_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/platform_info_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/pmu_counters_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/pmu_event_filter_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/private_mem_conversions_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/private_mem_kvm_exits_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/recalc_apic_map_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/set_boot_cpu_id/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/set_sregs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/sev_init2_tests/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/sev_migrate_tests/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/sev_smoke_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/smaller_maxphyaddr_emulation_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/smm_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/state_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/svm_int_ctl_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/svm_nested_shutdown_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/svm_nested_soft_inject_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/svm_vmcall_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/sync_regs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/triple_fault_event_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/tsc_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/tsc_scaling_sync/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/ucna_injection_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/userspace_io_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/userspace_msr_exit_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_apic_access_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_close_while_nested_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_dirty_log_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_exception_with_invalid_guest_state/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_invalid_nested_guest_state/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_nested_tsc_scaling_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_pmu_caps_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_preemption_timer_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_set_nested_state_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/vmx_tsc_adjust_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xapic_ipi_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xapic_state_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xcr0_cpuid_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xen_shinfo_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xen_vmcall_test/default.test
create mode 100644 tools/testing/selftests/kvm/testcases/x86_64/xss_msr_test/default.test
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 1d41a046a7bf..550b7c2b4a0c 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -9,4 +9,5 @@
!config
!settings
!Makefile
-!Makefile.kvm
\ No newline at end of file
+!Makefile.kvm
+!*.test
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test b/tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test
new file mode 100644
index 000000000000..5db8723f554f
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test
@@ -0,0 +1 @@
+./aarch64/aarch32_id_regs
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/arch_timer/default.test b/tools/testing/selftests/kvm/testcases/aarch64/arch_timer/default.test
new file mode 100644
index 000000000000..4eabd25b1c88
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/arch_timer/default.test
@@ -0,0 +1 @@
+./aarch64/arch_timer
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/arch_timer_edge_cases/default.test b/tools/testing/selftests/kvm/testcases/aarch64/arch_timer_edge_cases/default.test
new file mode 100644
index 000000000000..c2c17884d6ff
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/arch_timer_edge_cases/default.test
@@ -0,0 +1 @@
+./aarch64/arch_timer_edge_cases
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/debug-exceptions/default.test b/tools/testing/selftests/kvm/testcases/aarch64/debug-exceptions/default.test
new file mode 100644
index 000000000000..3dd50672ea2a
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/debug-exceptions/default.test
@@ -0,0 +1 @@
+./aarch64/debug-exceptions
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/get-reg-list/default.test b/tools/testing/selftests/kvm/testcases/aarch64/get-reg-list/default.test
new file mode 100644
index 000000000000..a7656ab23faa
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/get-reg-list/default.test
@@ -0,0 +1 @@
+./aarch64/get-reg-list
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/hypercalls/default.test b/tools/testing/selftests/kvm/testcases/aarch64/hypercalls/default.test
new file mode 100644
index 000000000000..c206440ad0e4
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/hypercalls/default.test
@@ -0,0 +1 @@
+./aarch64/hypercalls
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/no-vgic-v3/default.test b/tools/testing/selftests/kvm/testcases/aarch64/no-vgic-v3/default.test
new file mode 100644
index 000000000000..2dc6002ec63b
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/no-vgic-v3/default.test
@@ -0,0 +1 @@
+./aarch64/no-vgic-v3
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/page_fault_test/default.test b/tools/testing/selftests/kvm/testcases/aarch64/page_fault_test/default.test
new file mode 100644
index 000000000000..90d59bf94b53
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/page_fault_test/default.test
@@ -0,0 +1 @@
+./aarch64/page_fault_test
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/psci_test/default.test b/tools/testing/selftests/kvm/testcases/aarch64/psci_test/default.test
new file mode 100644
index 000000000000..55342f569e9e
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/psci_test/default.test
@@ -0,0 +1 @@
+./aarch64/psci_test
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/set_id_regs/default.test b/tools/testing/selftests/kvm/testcases/aarch64/set_id_regs/default.test
new file mode 100644
index 000000000000..ffabfceae569
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/set_id_regs/default.test
@@ -0,0 +1 @@
+./aarch64/set_id_regs
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/smccc_filter/default.test b/tools/testing/selftests/kvm/testcases/aarch64/smccc_filter/default.test
new file mode 100644
index 000000000000..e4ae3145f15e
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/smccc_filter/default.test
@@ -0,0 +1 @@
+./aarch64/smccc_filter
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/vcpu_width_config/default.test b/tools/testing/selftests/kvm/testcases/aarch64/vcpu_width_config/default.test
new file mode 100644
index 000000000000..489c20dc6cf0
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/vcpu_width_config/default.test
@@ -0,0 +1 @@
+./aarch64/vcpu_width_config
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/vgic_init/default.test b/tools/testing/selftests/kvm/testcases/aarch64/vgic_init/default.test
new file mode 100644
index 000000000000..b8e1a8a7cfb6
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/vgic_init/default.test
@@ -0,0 +1 @@
+./aarch64/vgic_init
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/vgic_irq/default.test b/tools/testing/selftests/kvm/testcases/aarch64/vgic_irq/default.test
new file mode 100644
index 000000000000..308c12aa5f13
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/vgic_irq/default.test
@@ -0,0 +1 @@
+./aarch64/vgic_irq
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/vgic_lpi_stress/default.test b/tools/testing/selftests/kvm/testcases/aarch64/vgic_lpi_stress/default.test
new file mode 100644
index 000000000000..b33c45fb8998
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/vgic_lpi_stress/default.test
@@ -0,0 +1 @@
+./aarch64/vgic_lpi_stress
diff --git a/tools/testing/selftests/kvm/testcases/aarch64/vpmu_counter_access/default.test b/tools/testing/selftests/kvm/testcases/aarch64/vpmu_counter_access/default.test
new file mode 100644
index 000000000000..cd3ef4c21274
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/aarch64/vpmu_counter_access/default.test
@@ -0,0 +1 @@
+./aarch64/vpmu_counter_access
diff --git a/tools/testing/selftests/kvm/testcases/access_tracking_perf_test/default.test b/tools/testing/selftests/kvm/testcases/access_tracking_perf_test/default.test
new file mode 100644
index 000000000000..e940543be193
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/access_tracking_perf_test/default.test
@@ -0,0 +1 @@
+./access_tracking_perf_test
diff --git a/tools/testing/selftests/kvm/testcases/coalesced_io_test/default.test b/tools/testing/selftests/kvm/testcases/coalesced_io_test/default.test
new file mode 100644
index 000000000000..08dfcbb4fcd0
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/coalesced_io_test/default.test
@@ -0,0 +1 @@
+./coalesced_io_test
diff --git a/tools/testing/selftests/kvm/testcases/demand_paging_test/default.test b/tools/testing/selftests/kvm/testcases/demand_paging_test/default.test
new file mode 100644
index 000000000000..26043696d095
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/demand_paging_test/default.test
@@ -0,0 +1 @@
+./demand_paging_test
diff --git a/tools/testing/selftests/kvm/testcases/dirty_log_perf_test/default.test b/tools/testing/selftests/kvm/testcases/dirty_log_perf_test/default.test
new file mode 100644
index 000000000000..8968bf6eb881
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/dirty_log_perf_test/default.test
@@ -0,0 +1 @@
+./dirty_log_perf_test
diff --git a/tools/testing/selftests/kvm/testcases/dirty_log_test/default.test b/tools/testing/selftests/kvm/testcases/dirty_log_test/default.test
new file mode 100644
index 000000000000..87bc10443ff9
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/dirty_log_test/default.test
@@ -0,0 +1 @@
+./dirty_log_test
diff --git a/tools/testing/selftests/kvm/testcases/guest_memfd_test/default.test b/tools/testing/selftests/kvm/testcases/guest_memfd_test/default.test
new file mode 100644
index 000000000000..4bba43fcca8d
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/guest_memfd_test/default.test
@@ -0,0 +1 @@
+./guest_memfd_test
diff --git a/tools/testing/selftests/kvm/testcases/guest_print_test/default.test b/tools/testing/selftests/kvm/testcases/guest_print_test/default.test
new file mode 100644
index 000000000000..2b7d376d6b09
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/guest_print_test/default.test
@@ -0,0 +1 @@
+./guest_print_test
diff --git a/tools/testing/selftests/kvm/testcases/hardware_disable_test/default.test b/tools/testing/selftests/kvm/testcases/hardware_disable_test/default.test
new file mode 100644
index 000000000000..e960b290d00a
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/hardware_disable_test/default.test
@@ -0,0 +1 @@
+./hardware_disable_test
diff --git a/tools/testing/selftests/kvm/testcases/kvm_binary_stats_test/default.test b/tools/testing/selftests/kvm/testcases/kvm_binary_stats_test/default.test
new file mode 100644
index 000000000000..a8c0240251ce
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/kvm_binary_stats_test/default.test
@@ -0,0 +1 @@
+./kvm_binary_stats_test
diff --git a/tools/testing/selftests/kvm/testcases/kvm_create_max_vcpus/default.test b/tools/testing/selftests/kvm/testcases/kvm_create_max_vcpus/default.test
new file mode 100644
index 000000000000..a23226454cdc
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/kvm_create_max_vcpus/default.test
@@ -0,0 +1 @@
+./kvm_create_max_vcpus
diff --git a/tools/testing/selftests/kvm/testcases/kvm_page_table_test/default.test b/tools/testing/selftests/kvm/testcases/kvm_page_table_test/default.test
new file mode 100644
index 000000000000..0efb9a150562
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/kvm_page_table_test/default.test
@@ -0,0 +1 @@
+./kvm_page_table_test
diff --git a/tools/testing/selftests/kvm/testcases/max_guest_memory_test/default.test b/tools/testing/selftests/kvm/testcases/max_guest_memory_test/default.test
new file mode 100644
index 000000000000..85c43b0a0f74
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/max_guest_memory_test/default.test
@@ -0,0 +1 @@
+./max_guest_memory_test
diff --git a/tools/testing/selftests/kvm/testcases/memslot_modification_stress_test/default.test b/tools/testing/selftests/kvm/testcases/memslot_modification_stress_test/default.test
new file mode 100644
index 000000000000..4a4bb75db039
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/memslot_modification_stress_test/default.test
@@ -0,0 +1 @@
+./memslot_modification_stress_test
diff --git a/tools/testing/selftests/kvm/testcases/memslot_perf_test/default.test b/tools/testing/selftests/kvm/testcases/memslot_perf_test/default.test
new file mode 100644
index 000000000000..70889328cea3
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/memslot_perf_test/default.test
@@ -0,0 +1 @@
+./memslot_perf_test
diff --git a/tools/testing/selftests/kvm/testcases/pre_fault_memory_test/default.test b/tools/testing/selftests/kvm/testcases/pre_fault_memory_test/default.test
new file mode 100644
index 000000000000..2b7c896d7c54
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/pre_fault_memory_test/default.test
@@ -0,0 +1 @@
+./pre_fault_memory_test
diff --git a/tools/testing/selftests/kvm/testcases/riscv/arch_timer/default.test b/tools/testing/selftests/kvm/testcases/riscv/arch_timer/default.test
new file mode 100644
index 000000000000..c16ee2797869
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/riscv/arch_timer/default.test
@@ -0,0 +1 @@
+./riscv/arch_timer
diff --git a/tools/testing/selftests/kvm/testcases/riscv/ebreak_test/default.test b/tools/testing/selftests/kvm/testcases/riscv/ebreak_test/default.test
new file mode 100644
index 000000000000..8415f42d391a
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/riscv/ebreak_test/default.test
@@ -0,0 +1 @@
+./riscv/ebreak_test
diff --git a/tools/testing/selftests/kvm/testcases/riscv/get-reg-list/default.test b/tools/testing/selftests/kvm/testcases/riscv/get-reg-list/default.test
new file mode 100644
index 000000000000..0238b91deecd
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/riscv/get-reg-list/default.test
@@ -0,0 +1 @@
+./riscv/get-reg-list
diff --git a/tools/testing/selftests/kvm/testcases/riscv/sbi_pmu_test/default.test b/tools/testing/selftests/kvm/testcases/riscv/sbi_pmu_test/default.test
new file mode 100644
index 000000000000..efa41caabe3e
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/riscv/sbi_pmu_test/default.test
@@ -0,0 +1 @@
+./riscv/sbi_pmu_test
diff --git a/tools/testing/selftests/kvm/testcases/rseq_test/default.test b/tools/testing/selftests/kvm/testcases/rseq_test/default.test
new file mode 100644
index 000000000000..6098cd71bd56
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/rseq_test/default.test
@@ -0,0 +1 @@
+./rseq_test
diff --git a/tools/testing/selftests/kvm/testcases/s390x/cmma_test/default.test b/tools/testing/selftests/kvm/testcases/s390x/cmma_test/default.test
new file mode 100644
index 000000000000..b36736b053dc
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/s390x/cmma_test/default.test
@@ -0,0 +1 @@
+./s390x/cmma_test
diff --git a/tools/testing/selftests/kvm/testcases/s390x/debug_test/default.test b/tools/testing/selftests/kvm/testcases/s390x/debug_test/default.test
new file mode 100644
index 000000000000..56eadb53ff96
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/s390x/debug_test/default.test
@@ -0,0 +1 @@
+./s390x/debug_test
diff --git a/tools/testing/selftests/kvm/testcases/s390x/memop/default.test b/tools/testing/selftests/kvm/testcases/s390x/memop/default.test
new file mode 100644
index 000000000000..b80932431521
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/s390x/memop/default.test
@@ -0,0 +1 @@
+./s390x/memop
diff --git a/tools/testing/selftests/kvm/testcases/s390x/resets/default.test b/tools/testing/selftests/kvm/testcases/s390x/resets/default.test
new file mode 100644
index 000000000000..7e116efaf6e3
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/s390x/resets/default.test
@@ -0,0 +1 @@
+./s390x/resets
diff --git a/tools/testing/selftests/kvm/testcases/s390x/shared_zeropage_test/default.test b/tools/testing/selftests/kvm/testcases/s390x/shared_zeropage_test/default.test
new file mode 100644
index 000000000000..d7ed954a5d87
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/s390x/shared_zeropage_test/default.test
@@ -0,0 +1 @@
+./s390x/shared_zeropage_test
diff --git a/tools/testing/selftests/kvm/testcases/s390x/sync_regs_test/default.test b/tools/testing/selftests/kvm/testcases/s390x/sync_regs_test/default.test
new file mode 100644
index 000000000000..fb5a97dc1a60
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/s390x/sync_regs_test/default.test
@@ -0,0 +1 @@
+./s390x/sync_regs_test
diff --git a/tools/testing/selftests/kvm/testcases/s390x/tprot/default.test b/tools/testing/selftests/kvm/testcases/s390x/tprot/default.test
new file mode 100644
index 000000000000..16476e6f7fe7
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/s390x/tprot/default.test
@@ -0,0 +1 @@
+./s390x/tprot
diff --git a/tools/testing/selftests/kvm/testcases/s390x/ucontrol_test/default.test b/tools/testing/selftests/kvm/testcases/s390x/ucontrol_test/default.test
new file mode 100644
index 000000000000..1a9a32db0d99
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/s390x/ucontrol_test/default.test
@@ -0,0 +1 @@
+./s390x/ucontrol_test
diff --git a/tools/testing/selftests/kvm/testcases/set_memory_region_test/default.test b/tools/testing/selftests/kvm/testcases/set_memory_region_test/default.test
new file mode 100644
index 000000000000..356c31e88471
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/set_memory_region_test/default.test
@@ -0,0 +1 @@
+./set_memory_region_test
diff --git a/tools/testing/selftests/kvm/testcases/steal_time/default.test b/tools/testing/selftests/kvm/testcases/steal_time/default.test
new file mode 100644
index 000000000000..10a490246d2d
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/steal_time/default.test
@@ -0,0 +1 @@
+./steal_time
diff --git a/tools/testing/selftests/kvm/testcases/system_counter_offset_test/default.test b/tools/testing/selftests/kvm/testcases/system_counter_offset_test/default.test
new file mode 100644
index 000000000000..a35557a85f79
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/system_counter_offset_test/default.test
@@ -0,0 +1 @@
+./system_counter_offset_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/amx_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/amx_test/default.test
new file mode 100644
index 000000000000..9328a4997849
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/amx_test/default.test
@@ -0,0 +1 @@
+./x86_64/amx_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/apic_bus_clock_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/apic_bus_clock_test/default.test
new file mode 100644
index 000000000000..71dc5be7a4de
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/apic_bus_clock_test/default.test
@@ -0,0 +1 @@
+./x86_64/apic_bus_clock_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/cpuid_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/cpuid_test/default.test
new file mode 100644
index 000000000000..f9f518c76732
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/cpuid_test/default.test
@@ -0,0 +1 @@
+./x86_64/cpuid_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/cr4_cpuid_sync_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/cr4_cpuid_sync_test/default.test
new file mode 100644
index 000000000000..a7ebd6d37523
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/cr4_cpuid_sync_test/default.test
@@ -0,0 +1 @@
+./x86_64/cr4_cpuid_sync_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/debug_regs/default.test b/tools/testing/selftests/kvm/testcases/x86_64/debug_regs/default.test
new file mode 100644
index 000000000000..972e16d8ae80
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/debug_regs/default.test
@@ -0,0 +1 @@
+./x86_64/debug_regs
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/dirty_log_page_splitting_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/dirty_log_page_splitting_test/default.test
new file mode 100644
index 000000000000..5e5eedb125fb
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/dirty_log_page_splitting_test/default.test
@@ -0,0 +1 @@
+./x86_64/dirty_log_page_splitting_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/exit_on_emulation_failure_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/exit_on_emulation_failure_test/default.test
new file mode 100644
index 000000000000..5e8d932a985a
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/exit_on_emulation_failure_test/default.test
@@ -0,0 +1 @@
+./x86_64/exit_on_emulation_failure_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/feature_msrs_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/feature_msrs_test/default.test
new file mode 100644
index 000000000000..a7f2bbf4bf56
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/feature_msrs_test/default.test
@@ -0,0 +1 @@
+./x86_64/feature_msrs_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/fix_hypercall_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/fix_hypercall_test/default.test
new file mode 100644
index 000000000000..e14c80fa8f45
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/fix_hypercall_test/default.test
@@ -0,0 +1 @@
+./x86_64/fix_hypercall_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/hwcr_msr_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/hwcr_msr_test/default.test
new file mode 100644
index 000000000000..62edde1e311d
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/hwcr_msr_test/default.test
@@ -0,0 +1 @@
+./x86_64/hwcr_msr_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/hyperv_clock/default.test b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_clock/default.test
new file mode 100644
index 000000000000..f636d01c1c04
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_clock/default.test
@@ -0,0 +1 @@
+./x86_64/hyperv_clock
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/hyperv_cpuid/default.test b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_cpuid/default.test
new file mode 100644
index 000000000000..eafad0b1579e
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_cpuid/default.test
@@ -0,0 +1 @@
+./x86_64/hyperv_cpuid
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/hyperv_evmcs/default.test b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_evmcs/default.test
new file mode 100644
index 000000000000..851add3c6e81
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_evmcs/default.test
@@ -0,0 +1 @@
+./x86_64/hyperv_evmcs
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/hyperv_extended_hypercalls/default.test b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_extended_hypercalls/default.test
new file mode 100644
index 000000000000..f66c5faa362b
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_extended_hypercalls/default.test
@@ -0,0 +1 @@
+./x86_64/hyperv_extended_hypercalls
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/hyperv_features/default.test b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_features/default.test
new file mode 100644
index 000000000000..fbf677334f30
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_features/default.test
@@ -0,0 +1 @@
+./x86_64/hyperv_features
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/hyperv_ipi/default.test b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_ipi/default.test
new file mode 100644
index 000000000000..5e080d57c89a
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_ipi/default.test
@@ -0,0 +1 @@
+./x86_64/hyperv_ipi
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/hyperv_svm_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_svm_test/default.test
new file mode 100644
index 000000000000..4d10b12513cb
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_svm_test/default.test
@@ -0,0 +1 @@
+./x86_64/hyperv_svm_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/hyperv_tlb_flush/default.test b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_tlb_flush/default.test
new file mode 100644
index 000000000000..0eb679f846e7
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/hyperv_tlb_flush/default.test
@@ -0,0 +1 @@
+./x86_64/hyperv_tlb_flush
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/kvm_clock_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/kvm_clock_test/default.test
new file mode 100644
index 000000000000..7d448d089a65
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/kvm_clock_test/default.test
@@ -0,0 +1 @@
+./x86_64/kvm_clock_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/kvm_pv_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/kvm_pv_test/default.test
new file mode 100644
index 000000000000..fb71a5fd8544
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/kvm_pv_test/default.test
@@ -0,0 +1 @@
+./x86_64/kvm_pv_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/max_vcpuid_cap_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/max_vcpuid_cap_test/default.test
new file mode 100644
index 000000000000..cc83f9b0224a
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/max_vcpuid_cap_test/default.test
@@ -0,0 +1 @@
+./x86_64/max_vcpuid_cap_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/monitor_mwait_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/monitor_mwait_test/default.test
new file mode 100644
index 000000000000..b863c909f6c6
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/monitor_mwait_test/default.test
@@ -0,0 +1 @@
+./x86_64/monitor_mwait_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/nested_exceptions_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/nested_exceptions_test/default.test
new file mode 100644
index 000000000000..bc326f0a4607
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/nested_exceptions_test/default.test
@@ -0,0 +1 @@
+./x86_64/nested_exceptions_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/nx_huge_pages_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/nx_huge_pages_test/default.test
new file mode 100644
index 000000000000..7195b59fe8a5
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/nx_huge_pages_test/default.test
@@ -0,0 +1 @@
+./x86_64/nx_huge_pages_test.sh
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/platform_info_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/platform_info_test/default.test
new file mode 100644
index 000000000000..d8e9803516ae
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/platform_info_test/default.test
@@ -0,0 +1 @@
+./x86_64/platform_info_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/pmu_counters_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/pmu_counters_test/default.test
new file mode 100644
index 000000000000..08f30fffe000
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/pmu_counters_test/default.test
@@ -0,0 +1 @@
+./x86_64/pmu_counters_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/pmu_event_filter_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/pmu_event_filter_test/default.test
new file mode 100644
index 000000000000..58f0afe0af50
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/pmu_event_filter_test/default.test
@@ -0,0 +1 @@
+./x86_64/pmu_event_filter_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/private_mem_conversions_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/private_mem_conversions_test/default.test
new file mode 100644
index 000000000000..8bb591b63967
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/private_mem_conversions_test/default.test
@@ -0,0 +1 @@
+./x86_64/private_mem_conversions_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/private_mem_kvm_exits_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/private_mem_kvm_exits_test/default.test
new file mode 100644
index 000000000000..9c624949de95
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/private_mem_kvm_exits_test/default.test
@@ -0,0 +1 @@
+./x86_64/private_mem_kvm_exits_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/recalc_apic_map_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/recalc_apic_map_test/default.test
new file mode 100644
index 000000000000..441e3bea8b71
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/recalc_apic_map_test/default.test
@@ -0,0 +1 @@
+./x86_64/recalc_apic_map_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/set_boot_cpu_id/default.test b/tools/testing/selftests/kvm/testcases/x86_64/set_boot_cpu_id/default.test
new file mode 100644
index 000000000000..80d5156f0623
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/set_boot_cpu_id/default.test
@@ -0,0 +1 @@
+./x86_64/set_boot_cpu_id
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/set_sregs_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/set_sregs_test/default.test
new file mode 100644
index 000000000000..83531913d381
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/set_sregs_test/default.test
@@ -0,0 +1 @@
+./x86_64/set_sregs_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/sev_init2_tests/default.test b/tools/testing/selftests/kvm/testcases/x86_64/sev_init2_tests/default.test
new file mode 100644
index 000000000000..9c356b1d7717
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/sev_init2_tests/default.test
@@ -0,0 +1 @@
+./x86_64/sev_init2_tests
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/sev_migrate_tests/default.test b/tools/testing/selftests/kvm/testcases/x86_64/sev_migrate_tests/default.test
new file mode 100644
index 000000000000..f0e2742cf6b4
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/sev_migrate_tests/default.test
@@ -0,0 +1 @@
+./x86_64/sev_migrate_tests
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/sev_smoke_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/sev_smoke_test/default.test
new file mode 100644
index 000000000000..56733d3d47d5
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/sev_smoke_test/default.test
@@ -0,0 +1 @@
+./x86_64/sev_smoke_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/smaller_maxphyaddr_emulation_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/smaller_maxphyaddr_emulation_test/default.test
new file mode 100644
index 000000000000..52f6dcebe18e
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/smaller_maxphyaddr_emulation_test/default.test
@@ -0,0 +1 @@
+./x86_64/smaller_maxphyaddr_emulation_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/smm_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/smm_test/default.test
new file mode 100644
index 000000000000..bd50d1fe0e39
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/smm_test/default.test
@@ -0,0 +1 @@
+./x86_64/smm_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/state_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/state_test/default.test
new file mode 100644
index 000000000000..e2f261ecb141
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/state_test/default.test
@@ -0,0 +1 @@
+./x86_64/state_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/svm_int_ctl_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/svm_int_ctl_test/default.test
new file mode 100644
index 000000000000..51c2b1a4d7ea
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/svm_int_ctl_test/default.test
@@ -0,0 +1 @@
+./x86_64/svm_int_ctl_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/svm_nested_shutdown_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/svm_nested_shutdown_test/default.test
new file mode 100644
index 000000000000..aa1320afb235
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/svm_nested_shutdown_test/default.test
@@ -0,0 +1 @@
+./x86_64/svm_nested_shutdown_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/svm_nested_soft_inject_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/svm_nested_soft_inject_test/default.test
new file mode 100644
index 000000000000..75ffe38b0246
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/svm_nested_soft_inject_test/default.test
@@ -0,0 +1 @@
+./x86_64/svm_nested_soft_inject_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/svm_vmcall_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/svm_vmcall_test/default.test
new file mode 100644
index 000000000000..415f8112e873
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/svm_vmcall_test/default.test
@@ -0,0 +1 @@
+./x86_64/svm_vmcall_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/sync_regs_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/sync_regs_test/default.test
new file mode 100644
index 000000000000..26f1e3c00208
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/sync_regs_test/default.test
@@ -0,0 +1 @@
+./x86_64/sync_regs_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/triple_fault_event_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/triple_fault_event_test/default.test
new file mode 100644
index 000000000000..86709ec1af57
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/triple_fault_event_test/default.test
@@ -0,0 +1 @@
+./x86_64/triple_fault_event_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/tsc_msrs_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/tsc_msrs_test/default.test
new file mode 100644
index 000000000000..e7eca5c5425b
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/tsc_msrs_test/default.test
@@ -0,0 +1 @@
+./x86_64/tsc_msrs_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/tsc_scaling_sync/default.test b/tools/testing/selftests/kvm/testcases/x86_64/tsc_scaling_sync/default.test
new file mode 100644
index 000000000000..75cec395980e
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/tsc_scaling_sync/default.test
@@ -0,0 +1 @@
+./x86_64/tsc_scaling_sync
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/ucna_injection_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/ucna_injection_test/default.test
new file mode 100644
index 000000000000..991ab4b35376
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/ucna_injection_test/default.test
@@ -0,0 +1 @@
+./x86_64/ucna_injection_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/userspace_io_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/userspace_io_test/default.test
new file mode 100644
index 000000000000..3401aa68ec4f
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/userspace_io_test/default.test
@@ -0,0 +1 @@
+./x86_64/userspace_io_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/userspace_msr_exit_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/userspace_msr_exit_test/default.test
new file mode 100644
index 000000000000..25d34dbb542d
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/userspace_msr_exit_test/default.test
@@ -0,0 +1 @@
+./x86_64/userspace_msr_exit_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_apic_access_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_apic_access_test/default.test
new file mode 100644
index 000000000000..970ce4a87b69
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_apic_access_test/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_apic_access_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_close_while_nested_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_close_while_nested_test/default.test
new file mode 100644
index 000000000000..97b5c9bc2385
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_close_while_nested_test/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_close_while_nested_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_dirty_log_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_dirty_log_test/default.test
new file mode 100644
index 000000000000..dbbc6771830d
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_dirty_log_test/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_dirty_log_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_exception_with_invalid_guest_state/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_exception_with_invalid_guest_state/default.test
new file mode 100644
index 000000000000..d9cb4e720d2b
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_exception_with_invalid_guest_state/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_exception_with_invalid_guest_state
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_invalid_nested_guest_state/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_invalid_nested_guest_state/default.test
new file mode 100644
index 000000000000..396253417646
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_invalid_nested_guest_state/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_invalid_nested_guest_state
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_msrs_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_msrs_test/default.test
new file mode 100644
index 000000000000..5eb96a67bc3a
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_msrs_test/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_msrs_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_nested_tsc_scaling_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_nested_tsc_scaling_test/default.test
new file mode 100644
index 000000000000..9ffd06d52cce
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_nested_tsc_scaling_test/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_nested_tsc_scaling_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_pmu_caps_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_pmu_caps_test/default.test
new file mode 100644
index 000000000000..e7bbc34d0d98
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_pmu_caps_test/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_pmu_caps_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_preemption_timer_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_preemption_timer_test/default.test
new file mode 100644
index 000000000000..08432fd3071a
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_preemption_timer_test/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_preemption_timer_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_set_nested_state_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_set_nested_state_test/default.test
new file mode 100644
index 000000000000..58b2bc65f767
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_set_nested_state_test/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_set_nested_state_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/vmx_tsc_adjust_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/vmx_tsc_adjust_test/default.test
new file mode 100644
index 000000000000..3ec7560cab11
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/vmx_tsc_adjust_test/default.test
@@ -0,0 +1 @@
+./x86_64/vmx_tsc_adjust_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/xapic_ipi_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/xapic_ipi_test/default.test
new file mode 100644
index 000000000000..2cbd6b12f6e3
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/xapic_ipi_test/default.test
@@ -0,0 +1 @@
+./x86_64/xapic_ipi_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/xapic_state_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/xapic_state_test/default.test
new file mode 100644
index 000000000000..649e0268d6fd
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/xapic_state_test/default.test
@@ -0,0 +1 @@
+./x86_64/xapic_state_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/xcr0_cpuid_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/xcr0_cpuid_test/default.test
new file mode 100644
index 000000000000..d5ea6a4e2b20
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/xcr0_cpuid_test/default.test
@@ -0,0 +1 @@
+./x86_64/xcr0_cpuid_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/xen_shinfo_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/xen_shinfo_test/default.test
new file mode 100644
index 000000000000..79423b5a11e1
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/xen_shinfo_test/default.test
@@ -0,0 +1 @@
+./x86_64/xen_shinfo_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/xen_vmcall_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/xen_vmcall_test/default.test
new file mode 100644
index 000000000000..32ba7eb9d7e9
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/xen_vmcall_test/default.test
@@ -0,0 +1 @@
+./x86_64/xen_vmcall_test
diff --git a/tools/testing/selftests/kvm/testcases/x86_64/xss_msr_test/default.test b/tools/testing/selftests/kvm/testcases/x86_64/xss_msr_test/default.test
new file mode 100644
index 000000000000..c61ac639825b
--- /dev/null
+++ b/tools/testing/selftests/kvm/testcases/x86_64/xss_msr_test/default.test
@@ -0,0 +1 @@
+./x86_64/xss_msr_test
--
2.48.1.601.g30ceb7b040-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 2/2] KVM: selftests: Create KVM selftest runner
2025-02-22 0:59 [PATCH 0/2] Add KVM selftest runner Vipin Sharma
2025-02-22 0:59 ` [PATCH 1/2] KVM: selftests: Add default testfiles for KVM selftests runner Vipin Sharma
@ 2025-02-22 0:59 ` Vipin Sharma
2025-04-30 21:31 ` Sean Christopherson
1 sibling, 1 reply; 9+ messages in thread
From: Vipin Sharma @ 2025-02-22 0:59 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, Vipin Sharma
Create KVM selftest runner to run selftests and provide various options
for execution.
Provide following features in the runner:
1. --timeout/-t: Max time each test should finish in before killing it.
2. --jobs/-j: Run these many tests in parallel.
3. --tests: Provide space separated path of tests to execute.
4. --test_dirs: Directories to search for test files and run them.
5. --output/-o: Create the folder with given name and dump output of
each test in a hierarchical way.
6. Add summary at the end.
Runner needs testcase files which are provided in the previous patch.
Following are the examples to start the runner (cwd is
tools/testing/selftests/kvm)
- Basic run:
python3 runner --test_dirs testcases
- Run specific test
python3 runner --tests ./testcases/dirty_log_perf_test/default.test
- Run tests parallel
python3 runner --test_dirs testcases -j 10
- Run 5 tests parallely at a time, with the timeout of 10 seconds and
dump output in "result" directory
python3 runner --test_dirs testcases -j 5 -t 10 --output result
Sample output from the above command:
python3_binary runner --test_dirs testcases -j 5 -t 10 --output result
2025-02-21 16:45:46,774 | 16809 | INFO | [Passed] testcases/guest_print_test/default.test
2025-02-21 16:45:47,040 | 16809 | INFO | [Passed] testcases/kvm_create_max_vcpus/default.test
2025-02-21 16:45:49,244 | 16809 | INFO | [Passed] testcases/dirty_log_perf_test/default.test
...
2025-02-21 16:46:07,225 | 16809 | INFO | [Passed] testcases/x86_64/pmu_event_filter_test/default.test
2025-02-21 16:46:08,020 | 16809 | INFO | [Passed] testcases/x86_64/vmx_preemption_timer_test/default.test
2025-02-21 16:46:09,734 | 16809 | INFO | [Timed out] testcases/x86_64/pmu_counters_test/default.test
2025-02-21 16:46:10,202 | 16809 | INFO | [Passed] testcases/hardware_disable_test/default.test
2025-02-21 16:46:10,203 | 16809 | INFO | Tests ran: 85 tests
2025-02-21 16:46:10,204 | 16809 | INFO | Passed: 61
2025-02-21 16:46:10,204 | 16809 | INFO | Failed: 4
2025-02-21 16:46:10,204 | 16809 | INFO | Skipped: 17
2025-02-21 16:46:10,204 | 16809 | INFO | Timed out: 3
2025-02-21 16:46:10,204 | 16809 | INFO | No run: 0
Output dumped in result directory
$ tree result/
result/
├── log
└── testcases
├── access_tracking_perf_test
│ └── default.test
│ ├── stderr
│ └── stdout
├── coalesced_io_test
│ └── default.test
│ ├── stderr
│ └── stdout
...
results/log file will have the status of each test like the one printed
on console. Each stderr and stdout will have data based on the
execution.
Runner is implemented in python and needs at least 3.6 version.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
tools/testing/selftests/kvm/.gitignore | 1 +
.../testing/selftests/kvm/runner/__main__.py | 96 +++++++++++++++++++
tools/testing/selftests/kvm/runner/command.py | 42 ++++++++
.../testing/selftests/kvm/runner/selftest.py | 49 ++++++++++
.../selftests/kvm/runner/test_runner.py | 40 ++++++++
5 files changed, 228 insertions(+)
create mode 100644 tools/testing/selftests/kvm/runner/__main__.py
create mode 100644 tools/testing/selftests/kvm/runner/command.py
create mode 100644 tools/testing/selftests/kvm/runner/selftest.py
create mode 100644 tools/testing/selftests/kvm/runner/test_runner.py
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 550b7c2b4a0c..a23fd4b2cb5f 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -11,3 +11,4 @@
!Makefile
!Makefile.kvm
!*.test
+!*.py
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
new file mode 100644
index 000000000000..008d862757f2
--- /dev/null
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -0,0 +1,96 @@
+# SPDX-License-Identifier: GPL-2.0
+import pathlib
+import argparse
+import platform
+import logging
+import os
+import enum
+import test_runner
+
+
+def cli():
+ parser = argparse.ArgumentParser(
+ prog="KVM Selftests Runner",
+ description="Run KVM selftests with different configurations",
+ formatter_class=argparse.RawTextHelpFormatter
+ )
+
+ parser.add_argument("--tests",
+ nargs="*",
+ default=[],
+ help="Test cases to run. Provide the space separated test case file paths")
+
+ parser.add_argument("--test_dirs",
+ nargs="*",
+ default=[],
+ help="Run tests in the given directory and all its sub directories. Provide the space separated paths to add multiple directories.")
+
+ parser.add_argument("-j",
+ "--jobs",
+ default=1,
+ type=int,
+ help="Number of parallel test runners to start")
+
+ parser.add_argument("-t",
+ "--timeout",
+ default=120,
+ type=int,
+ help="How long to wait for a single test to finish before killing it")
+
+ parser.add_argument("-o",
+ "--output",
+ nargs='?',
+ help="Output directory for test results.")
+
+ return parser.parse_args()
+
+
+def setup_logging(args):
+ output = args.output
+ if output == None:
+ logging.basicConfig(level=logging.INFO,
+ format="%(asctime)s | %(process)d | %(levelname)8s | %(message)s")
+ else:
+ logging_file = os.path.join(output, "log")
+ pathlib.Path(output).mkdir(parents=True, exist_ok=True)
+ logging.basicConfig(level=logging.INFO,
+ format="%(asctime)s | %(process)d | %(levelname)8s | %(message)s",
+ handlers=[
+ logging.FileHandler(logging_file, mode='w'),
+ logging.StreamHandler()
+ ])
+
+
+def fetch_tests_from_dirs(scan_dirs, exclude_dirs):
+ test_files = []
+ for scan_dir in scan_dirs:
+ for root, dirs, files in os.walk(scan_dir):
+ dirs[:] = [dir for dir in dirs if dir not in exclude_dirs]
+ for file in files:
+ test_files.append(os.path.join(root, file))
+ return test_files
+
+
+def fetch_test_files(args):
+ exclude_dirs = ["aarch64", "x86_64", "riscv", "s390x"]
+ # Don't exclude tests of the current platform
+ exclude_dirs.remove(platform.machine())
+
+ test_files = args.tests
+ test_files.extend(fetch_tests_from_dirs(args.test_dirs, exclude_dirs))
+ # Remove duplicates
+ test_files = list(dict.fromkeys(test_files))
+ return test_files
+
+
+def main():
+ args = cli()
+ setup_logging(args)
+ test_files = fetch_test_files(args)
+ tr = test_runner.TestRunner(
+ test_files, args.output, args.timeout, args.jobs)
+ tr.start()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/testing/selftests/kvm/runner/command.py b/tools/testing/selftests/kvm/runner/command.py
new file mode 100644
index 000000000000..a58f16fe4542
--- /dev/null
+++ b/tools/testing/selftests/kvm/runner/command.py
@@ -0,0 +1,42 @@
+# SPDX-License-Identifier: GPL-2.0
+import contextlib
+import subprocess
+import os
+import pathlib
+
+
+class Command:
+ """Executes a command
+
+ Just execute a command. Dump output to the directory if provided.
+
+ Returns the exit code of the command.
+ """
+
+ def __init__(self, command, timeout=None, output_dir=None):
+ self.command = command
+ self.timeout = timeout
+ self.output_dir = output_dir
+
+ def __run(self, output=None, error=None):
+ proc = subprocess.run(self.command, stdout=output,
+ stderr=error, universal_newlines=True,
+ shell=True, timeout=self.timeout)
+ return proc.returncode
+
+ def run(self):
+ if self.output_dir is not None:
+ pathlib.Path(self.output_dir).mkdir(parents=True, exist_ok=True)
+
+ output = None
+ error = None
+ with contextlib.ExitStack() as stack:
+ if self.output_dir is not None:
+ output_path = os.path.join(self.output_dir, "stdout")
+ output = stack.enter_context(
+ open(output_path, encoding="utf-8", mode="w"))
+
+ error_path = os.path.join(self.output_dir, "stderr")
+ error = stack.enter_context(
+ open(error_path, encoding="utf-8", mode="w"))
+ return self.__run(output, error)
diff --git a/tools/testing/selftests/kvm/runner/selftest.py b/tools/testing/selftests/kvm/runner/selftest.py
new file mode 100644
index 000000000000..cdf5d1085c08
--- /dev/null
+++ b/tools/testing/selftests/kvm/runner/selftest.py
@@ -0,0 +1,49 @@
+# SPDX-License-Identifier: GPL-2.0
+import subprocess
+import command
+import pathlib
+import enum
+import os
+import logging
+
+
+class SelftestStatus(str, enum.Enum):
+ PASSED = "Passed"
+ FAILED = "Failed"
+ SKIPPED = "Skipped"
+ TIMED_OUT = "Timed out"
+ NO_RUN = "No run"
+
+ def __str__(self):
+ return str.__str__(self)
+
+
+class Selftest:
+ """A single test.
+
+ A test which can be run on its own.
+ """
+
+ def __init__(self, test_path, output_dir=None, timeout=None,):
+ test_command = pathlib.Path(test_path).read_text().strip()
+ if not test_command:
+ raise ValueError("Empty test command in " + test_path)
+
+ if output_dir is not None:
+ output_dir = os.path.join(output_dir, test_path)
+ self.test_path = test_path
+ self.command = command.Command(test_command, timeout, output_dir)
+ self.status = SelftestStatus.NO_RUN
+
+ def run(self):
+ try:
+ ret = self.command.run()
+ if ret == 0:
+ self.status = SelftestStatus.PASSED
+ elif ret == 4:
+ self.status = SelftestStatus.SKIPPED
+ else:
+ self.status = SelftestStatus.FAILED
+ except subprocess.TimeoutExpired as e:
+ # logging.error(type(e).__name__ + str(e))
+ self.status = SelftestStatus.TIMED_OUT
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
new file mode 100644
index 000000000000..b9d34c20bf88
--- /dev/null
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -0,0 +1,40 @@
+# SPDX-License-Identifier: GPL-2.0
+import queue
+import concurrent.futures
+import logging
+import time
+import selftest
+
+
+class TestRunner:
+ def __init__(self, test_files, output_dir, timeout, parallelism):
+ self.parallelism = parallelism
+ self.tests = []
+
+ for test_file in test_files:
+ self.tests.append(selftest.Selftest(
+ test_file, output_dir, timeout))
+
+ def _run(self, test):
+ test.run()
+ return test
+
+ def start(self):
+
+ status = {x: 0 for x in selftest.SelftestStatus}
+ count = 0
+ with concurrent.futures.ProcessPoolExecutor(max_workers=self.parallelism) as executor:
+ all_futures = []
+ for test in self.tests:
+ future = executor.submit(self._run, test)
+ all_futures.append(future)
+
+ for future in concurrent.futures.as_completed(all_futures):
+ test = future.result()
+ logging.info(f"[{test.status}] {test.test_path}")
+ status[test.status] += 1
+ count += 1
+
+ logging.info(f"Tests ran: {count} tests")
+ for result, count in status.items():
+ logging.info(f"{result}: {count}")
--
2.48.1.601.g30ceb7b040-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] KVM: selftests: Add default testfiles for KVM selftests runner
2025-02-22 0:59 ` [PATCH 1/2] KVM: selftests: Add default testfiles for KVM selftests runner Vipin Sharma
@ 2025-04-30 17:01 ` Sean Christopherson
2025-05-05 19:05 ` Vipin Sharma
0 siblings, 1 reply; 9+ messages in thread
From: Sean Christopherson @ 2025-04-30 17:01 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton
On Fri, Feb 21, 2025, Vipin Sharma wrote:
> Create a root "testcases" folder in KVM selftests directory. Add test
> files for all of the KVM selftests across all of the supported
> platforms. Write only default test execution command in the test files.
> Commands written in the test files will be ran by KVM selftest runner.
>
> Test files are organized based following rules:
> 1. All test files resides in "testcases" directory.
> 2. Test files should have .test extension. This is needed so that
> git doesn't ignore the test file changes.
> 3. Each KVM selftest resides in a folder in "testcases" directory.
> It follows the path of KVM selftests directory. For example,
> kvm/x86_64/vmx_msrs_test.c will be in
> kvm/testcases/x86_64/vmx_msrs_tests directory.
> 4. default.test name is reserved for the default command to execute the
> test.
> 5. Different configuration of the tests should reside in their own test
> files under the same test directory. For example dirty_log_perf_test
> can have:
> - testcases/dirty_log_perf_test/default.test
> - testcases/dirty_log_perf_test/hugetlb1g.test
> - testcases/dirty_log_perf_test/disable_dirty_log_manual.test
> 6. If there is an arch specific option of a common test then it should
> be specified under an arch name directory in the test directory. This
> will avoid test runner to execute the common test or its option on
> unsupported machine. For example:
> testcases/memslot_modification_stress_test/x86_64/disable_slot_zap_quirk.test
>
> Signed-off-by: Vipin Sharma <vipinsh@google.com>
> ---
> tools/testing/selftests/kvm/.gitignore | 3 ++-
After spending a comical amount of time fiddling with the default testcases, I
think I have a final opinion.
We definitely auto-generate the default testcases. Relying on the user to remember
to add a testcase, and on the maintainer to remember to check for that, isn't a
winning strategy.
But, I do think we should commit the default.test files to the repository. If
they're ephemeral, then several problems arise:
1. For out-of-tree builds, the default.test files should arguably be placed in
the OUTPUT directory. But if/when we add curated testcases/, then we'll either
end up with multiple testcases/ directories (source and output), or we'll have
to copy testcases/ from the source to the output on a normal build, which is
rather gross. Or we'd need e.g. "make testcases", which is also gross, e.g.
I don't want to have to run yet more commands just to execute tests.
2. Generating default.test could overwrite a user-defined file. That's firmly
a user error, but at least if they default.test files are commited, the user
will get a hint or three that they're doing things wrong.
3. If the files aren't committed, then they probably should removed on "clean",
which isn't the end of the world since they're trivially easy to generate,
but it's kinda funky.
So, what if we add this to auto-generate the files? It's obviously wasteful since
the files will exist 99.9999999% of the time, but the overhead is completely
negligible. The only concern I have is if this will do the wrong thing for some
build environments, i.e. shove the files in the wrong location.
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index f62b0a5aba35..d94bb8330ad1 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -198,6 +198,13 @@ TEST_GEN_PROGS += $(TEST_GEN_PROGS_$(ARCH))
TEST_GEN_PROGS_EXTENDED += $(TEST_GEN_PROGS_EXTENDED_$(ARCH))
LIBKVM += $(LIBKVM_$(ARCH))
+$(foreach tc, $(TEST_PROGS), $(shell mkdir -p testcases/$(patsubst %.sh,%,$(tc))))
+$(foreach tc, $(TEST_GEN_PROGS), $(shell mkdir -p testcases/$(tc)))
+$(foreach tc, $(TEST_PROGS), \
+ $(shell echo $(tc) > $(patsubst %.sh,testcases/%/default.test,$(tc))))
+$(foreach tc, $(TEST_GEN_PROGS), \
+ $(shell echo $(tc) > $(patsubst %,testcases/%/default.test,$(tc))))
+
OVERRIDE_TARGETS = 1
# lib.mak defines $(OUTPUT), prepends $(OUTPUT)/ to $(TEST_GEN_PROGS), and most
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index 1d41a046a7bf..550b7c2b4a0c 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -9,4 +9,5 @@
> !config
> !settings
> !Makefile
> -!Makefile.kvm
> \ No newline at end of file
> +!Makefile.kvm
> +!*.test
Let's keep the extension wildcards sorted alphabetically, i.e.:
# SPDX-License-Identifier: GPL-2.0-only
*
!/**/
!*.c
!*.h
!*.py
!*.S
!*.sh
!*.test
!.gitignore
!config
!settings
!Makefile
!Makefile.kvm
> diff --git a/tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test b/tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test
> new file mode 100644
> index 000000000000..5db8723f554f
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test
> @@ -0,0 +1 @@
> +./aarch64/aarch32_id_regs
I don't see any reason to make the paths directly consumable with a leading "./".
Spoiler alert from the next patch: the location of the test executables needs to
explicitly resolved by the test runner.
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 2/2] KVM: selftests: Create KVM selftest runner
2025-02-22 0:59 ` [PATCH 2/2] KVM: selftests: Create KVM selftest runner Vipin Sharma
@ 2025-04-30 21:31 ` Sean Christopherson
2025-05-05 19:48 ` Vipin Sharma
0 siblings, 1 reply; 9+ messages in thread
From: Sean Christopherson @ 2025-04-30 21:31 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton
This is awesome!
I have lots of idea, but not all of them need to be address for the initial
commit. Some of them I would consider blockers, but I also don't want to end up
with massive scope creep that causes this to stall out for (more) months on end.
On Fri, Feb 21, 2025, Vipin Sharma wrote:
> Create KVM selftest runner to run selftests and provide various options
> for execution.
>
> Provide following features in the runner:
> 1. --timeout/-t: Max time each test should finish in before killing it.
The help for this needs to specify the units. I assume it's seconds?
> 2. --jobs/-j: Run these many tests in parallel.
> 3. --tests: Provide space separated path of tests to execute.
> 4. --test_dirs: Directories to search for test files and run them.
> 5. --output/-o: Create the folder with given name and dump output of
> each test in a hierarchical way.
> 6. Add summary at the end.
>
> Runner needs testcase files which are provided in the previous patch.
> Following are the examples to start the runner (cwd is
> tools/testing/selftests/kvm)
The runner definitely needs a command line option to specify the path to the
test executables. Defaulting to in-tree builds is totally fine, but we should
also support out-of-tree builds (or copying to a remote host, etc.).
The default testcases will have relative paths, e.g. $arch/$test, so the user
will still need to maintain the same directory structure as in-tree builds, but
IMO that's totally fine.
> - Basic run:
> python3 runner --test_dirs testcases
>
> - Run specific test
> python3 runner --tests ./testcases/dirty_log_perf_test/default.test
>
> - Run tests parallel
> python3 runner --test_dirs testcases -j 10
>
> - Run 5 tests parallely at a time, with the timeout of 10 seconds and
> dump output in "result" directory
> python3 runner --test_dirs testcases -j 5 -t 10 --output result
>
> Sample output from the above command:
>
> python3_binary runner --test_dirs testcases -j 5 -t 10 --output result
>
> 2025-02-21 16:45:46,774 | 16809 | INFO | [Passed] testcases/guest_print_test/default.test
> 2025-02-21 16:45:47,040 | 16809 | INFO | [Passed] testcases/kvm_create_max_vcpus/default.test
> 2025-02-21 16:45:49,244 | 16809 | INFO | [Passed] testcases/dirty_log_perf_test/default.test
Printing the timestamps to the console isn't terrible interesting, and IMO isn't
at all worth the noise.
The PID is nice, but it needs to be printed _before_ the test finishes, and it
needs to track the PID of the test. If getting that working is non-trivial,
definitely punt it for the initial commit.
And presumably INFO is the level of logging. That needs to go.
> ...
> 2025-02-21 16:46:07,225 | 16809 | INFO | [Passed] testcases/x86_64/pmu_event_filter_test/default.test
> 2025-02-21 16:46:08,020 | 16809 | INFO | [Passed] testcases/x86_64/vmx_preemption_timer_test/default.test
> 2025-02-21 16:46:09,734 | 16809 | INFO | [Timed out] testcases/x86_64/pmu_counters_test/default.test
I would really like to have terminal colored+bolded output for tests that fail
(or timeout) or are skipped.
I think we should also provide controls for the verbosity of the output. E.g. to
skip printing tests that pass entirely. My vote would be for a collection of
boolean knobs, i.e. not a log_level or whatever, because inevitably we'll end up
with output that isn't strictly "increasing".
Adding a param to disable printing of passed tests is presumably trivial, so maybe
do that for the initial commit, and then we can work on the fancier stuff?
> 2025-02-21 16:46:10,202 | 16809 | INFO | [Passed] testcases/hardware_disable_test/default.test
> 2025-02-21 16:46:10,203 | 16809 | INFO | Tests ran: 85 tests
It would be very nice to have a summary of things printed out periodically. E.g.
if my normal run has a few failures, but the current run has already failed a
decent number of tests, then I'd probably kill the run and start debugging.
Alternatively, and maybe even better, would be to make the runner mildly interactive,
i.e. to accept rudimentary commands while tests are running. Then the user can
query the state of things while the runner is still doing its thing. E.g. bind
a few keys to print the various statuses.
> 2025-02-21 16:46:10,204 | 16809 | INFO | Passed: 61
> 2025-02-21 16:46:10,204 | 16809 | INFO | Failed: 4
> 2025-02-21 16:46:10,204 | 16809 | INFO | Skipped: 17
> 2025-02-21 16:46:10,204 | 16809 | INFO | Timed out: 3
> 2025-02-21 16:46:10,204 | 16809 | INFO | No run: 0
A not-quite-mandatory, but very-nice-to-have feature would be the ability to
display which tests Passed/Failed/Skipped/Timed Out/Incomplete, with command line
knobs for each. My vote is for everything but Passed on-by-default, though it's
easy enough to put a light wrapper around this (which I'll do no matter what), so
my preference for the default doesn't matter all that much.
That could tie into the above idea of grabbing keys to print such information on-demand.
Also CTRL-C handling needs much more graceful output. Ideally, no stack traces
whatsover, and instead a summary like the above, but with information about which
tests didn't complete.
> Output dumped in result directory
>
> $ tree result/
> result/
The runner should have an (on-by-default?) option to abort if the output directory
already exists, e.g. so that users don't clobber previous runs. And/or an option
to append a timestamp, e.g. $result.yyyy.mm.dd.MM.SS, so that all users don't end
up writing the same wrapper to generate a timestamp.
Having a no-timestamp + overwrite mode is also useful, e.g. when I'm running in
a more "interactive" mode where I'm doing initial testing of something, and I
don't care about
> ├── log
> └── testcases
> ├── access_tracking_perf_test
> │ └── default.test
> │ ├── stderr
> │ └── stdout
> ├── coalesced_io_test
> │ └── default.test
> │ ├── stderr
> │ └── stdout
> ...
>
> results/log file will have the status of each test like the one printed
> on console. Each stderr and stdout will have data based on the
> execution.
>
> Runner is implemented in python and needs at least 3.6 version.
>
> Signed-off-by: Vipin Sharma <vipinsh@google.com>
> ---
...
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index 550b7c2b4a0c..a23fd4b2cb5f 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -11,3 +11,4 @@
> !Makefile
> !Makefile.kvm
> !*.test
> +!*.py
Sort this alphabetically as well.
> +def fetch_test_files(args):
> + exclude_dirs = ["aarch64", "x86_64", "riscv", "s390x"]
These are now:
arm64, x86, riscv, s390
> + def __init__(self, test_path, output_dir=None, timeout=None,):
> + test_command = pathlib.Path(test_path).read_text().strip()
> + if not test_command:
> + raise ValueError("Empty test command in " + test_path)
> +
> + if output_dir is not None:
> + output_dir = os.path.join(output_dir, test_path)
This doesn't do the right thing if test_path is absolute (or maybe it's if the
output_dir is in a completely different hierarchy?)
I was able to fudge around this with
diff --git a/tools/testing/selftests/kvm/runner/selftest.py b/tools/testing/selftests/kvm/runner/selftest.py
index cdf5d1085c08..3bce023693cb 100644
--- a/tools/testing/selftests/kvm/runner/selftest.py
+++ b/tools/testing/selftests/kvm/runner/selftest.py
@@ -30,7 +30,8 @@ class Selftest:
raise ValueError("Empty test command in " + test_path)
if output_dir is not None:
- output_dir = os.path.join(output_dir, test_path)
+ dirpath, filename = os.path.split(test_path)
+ output_dir = os.path.join(output_dir, os.path.basename(dirpath), filename)
self.test_path = test_path
self.command = command.Command(test_command, timeout, output_dir)
self.status = SelftestStatus.NO_RUN
Lastly, please don't wrap agressively (off-list, you mentioned it was due to a
formatter?). E.g.
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
index b9d34c20bf88..5a568e155477 100644
--- a/tools/testing/selftests/kvm/runner/test_runner.py
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -12,8 +12,7 @@ class TestRunner:
self.tests = []
for test_file in test_files:
- self.tests.append(selftest.Selftest(
- test_file, output_dir, timeout))
+ self.tests.append(selftest.Selftest(test_file, output_dir, timeout))
def _run(self, test):
test.run()
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] KVM: selftests: Add default testfiles for KVM selftests runner
2025-04-30 17:01 ` Sean Christopherson
@ 2025-05-05 19:05 ` Vipin Sharma
2025-05-05 22:57 ` Sean Christopherson
0 siblings, 1 reply; 9+ messages in thread
From: Vipin Sharma @ 2025-05-05 19:05 UTC (permalink / raw)
To: Sean Christopherson
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton
On 2025-04-30 10:01:29, Sean Christopherson wrote:
> On Fri, Feb 21, 2025, Vipin Sharma wrote:
> We definitely auto-generate the default testcases. Relying on the user to remember
> to add a testcase, and on the maintainer to remember to check for that, isn't a
> winning strategy.
Good idea, this will help in at least automatically run the default tests.
Also, if tests are in the selftest kvm source directory then it will
give an hint to developer/maintainer that they missed writing a default
test file for a newly introduced test when they run git status.
>
> But, I do think we should commit the default.test files to the repository. If
> they're ephemeral, then several problems arise:
>
> 1. For out-of-tree builds, the default.test files should arguably be placed in
> the OUTPUT directory. But if/when we add curated testcases/, then we'll either
> end up with multiple testcases/ directories (source and output), or we'll have
> to copy testcases/ from the source to the output on a normal build, which is
> rather gross. Or we'd need e.g. "make testcases", which is also gross, e.g.
> I don't want to have to run yet more commands just to execute tests.
>
> 2. Generating default.test could overwrite a user-defined file. That's firmly
> a user error, but at least if they default.test files are commited, the user
> will get a hint or three that they're doing things wrong.
>
> 3. If the files aren't committed, then they probably should removed on "clean",
> which isn't the end of the world since they're trivially easy to generate,
> but it's kinda funky.
>
> So, what if we add this to auto-generate the files? It's obviously wasteful since
> the files will exist 99.9999999% of the time, but the overhead is completely
> negligible. The only concern I have is if this will do the wrong thing for some
> build environments, i.e. shove the files in the wrong location.
We can get the current path of the Makefile.kvm by writing this at the top
of the Makefile.kvm:
MAKEFILE_DIR := $(dir $(realpath $(lastword $(MAKEFILE_LIST))))
Then MAKEFILE_DIR will have the source directory of Makfile.kvm and
testcase will be in the same directory.
With this we can modify the below foreach you wrote by prefixing
MAKEFILE_DIR to "testcases".
Does this alleviate concern regaring build environment?
>
> diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
> index f62b0a5aba35..d94bb8330ad1 100644
> --- a/tools/testing/selftests/kvm/Makefile.kvm
> +++ b/tools/testing/selftests/kvm/Makefile.kvm
> @@ -198,6 +198,13 @@ TEST_GEN_PROGS += $(TEST_GEN_PROGS_$(ARCH))
> TEST_GEN_PROGS_EXTENDED += $(TEST_GEN_PROGS_EXTENDED_$(ARCH))
> LIBKVM += $(LIBKVM_$(ARCH))
>
> +$(foreach tc, $(TEST_PROGS), $(shell mkdir -p testcases/$(patsubst %.sh,%,$(tc))))
> +$(foreach tc, $(TEST_GEN_PROGS), $(shell mkdir -p testcases/$(tc)))
> +$(foreach tc, $(TEST_PROGS), \
> + $(shell echo $(tc) > $(patsubst %.sh,testcases/%/default.test,$(tc))))
> +$(foreach tc, $(TEST_GEN_PROGS), \
> + $(shell echo $(tc) > $(patsubst %,testcases/%/default.test,$(tc))))
> +
This looks good.
> > diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> > index 1d41a046a7bf..550b7c2b4a0c 100644
> > --- a/tools/testing/selftests/kvm/.gitignore
> > +++ b/tools/testing/selftests/kvm/.gitignore
> > @@ -9,4 +9,5 @@
> > !config
> > !settings
> > !Makefile
> > -!Makefile.kvm
> > \ No newline at end of file
> > +!Makefile.kvm
> > +!*.test
>
> Let's keep the extension wildcards sorted alphabetically, i.e.:
>
> # SPDX-License-Identifier: GPL-2.0-only
> *
> !/**/
> !*.c
> !*.h
> !*.py
> !*.S
> !*.sh
> !*.test
> !.gitignore
> !config
> !settings
> !Makefile
> !Makefile.kvm
>
Okay.
> > diff --git a/tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test b/tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test
> > new file mode 100644
> > index 000000000000..5db8723f554f
> > --- /dev/null
> > +++ b/tools/testing/selftests/kvm/testcases/aarch64/aarch32_id_regs/default.test
> > @@ -0,0 +1 @@
> > +./aarch64/aarch32_id_regs
>
> I don't see any reason to make the paths directly consumable with a leading "./".
> Spoiler alert from the next patch: the location of the test executables needs to
> explicitly resolved by the test runner.
I will remove this.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 2/2] KVM: selftests: Create KVM selftest runner
2025-04-30 21:31 ` Sean Christopherson
@ 2025-05-05 19:48 ` Vipin Sharma
2025-05-05 23:26 ` Sean Christopherson
0 siblings, 1 reply; 9+ messages in thread
From: Vipin Sharma @ 2025-05-05 19:48 UTC (permalink / raw)
To: Sean Christopherson
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton
On 2025-04-30 14:31:05, Sean Christopherson wrote:
> This is awesome!
>
> I have lots of idea, but not all of them need to be address for the initial
> commit. Some of them I would consider blockers, but I also don't want to end up
> with massive scope creep that causes this to stall out for (more) months on end.
>
> On Fri, Feb 21, 2025, Vipin Sharma wrote:
> > Create KVM selftest runner to run selftests and provide various options
> > for execution.
> >
> > Provide following features in the runner:
> > 1. --timeout/-t: Max time each test should finish in before killing it.
>
> The help for this needs to specify the units. I assume it's seconds?
Yup, it is in seconds. I will update the help text.
> > Runner needs testcase files which are provided in the previous patch.
> > Following are the examples to start the runner (cwd is
> > tools/testing/selftests/kvm)
>
> The runner definitely needs a command line option to specify the path to the
> test executables. Defaulting to in-tree builds is totally fine, but we should
> also support out-of-tree builds (or copying to a remote host, etc.).
>
> The default testcases will have relative paths, e.g. $arch/$test, so the user
> will still need to maintain the same directory structure as in-tree builds, but
> IMO that's totally fine.
>
I didn't think about out-of-tree builds. Thanks for catching it.
I will add a command line option to pass the path of kvm selftests
executable, with the assumption of same directory structure as in-tree
builds. Default will be assuming in-tree builds.
> > - Basic run:
> > python3 runner --test_dirs testcases
> >
> > - Run specific test
> > python3 runner --tests ./testcases/dirty_log_perf_test/default.test
> >
> > - Run tests parallel
> > python3 runner --test_dirs testcases -j 10
> >
> > - Run 5 tests parallely at a time, with the timeout of 10 seconds and
> > dump output in "result" directory
> > python3 runner --test_dirs testcases -j 5 -t 10 --output result
> >
> > Sample output from the above command:
> >
> > python3_binary runner --test_dirs testcases -j 5 -t 10 --output result
> >
> > 2025-02-21 16:45:46,774 | 16809 | INFO | [Passed] testcases/guest_print_test/default.test
> > 2025-02-21 16:45:47,040 | 16809 | INFO | [Passed] testcases/kvm_create_max_vcpus/default.test
> > 2025-02-21 16:45:49,244 | 16809 | INFO | [Passed] testcases/dirty_log_perf_test/default.test
>
>
> Printing the timestamps to the console isn't terrible interesting, and IMO isn't
> at all worth the noise.
>
> The PID is nice, but it needs to be printed _before_ the test finishes, and it
> needs to track the PID of the test. If getting that working is non-trivial,
> definitely punt it for the initial commit.
>
> And presumably INFO is the level of logging. That needs to go.
>
Instead of removing timestamp, I can just print HH:MM:SS, I think it
provides value in seeing how fast runner and tests are executing.
I will modify PID to print each test pid.
INFO will go away.
> > ...
> > 2025-02-21 16:46:07,225 | 16809 | INFO | [Passed] testcases/x86_64/pmu_event_filter_test/default.test
> > 2025-02-21 16:46:08,020 | 16809 | INFO | [Passed] testcases/x86_64/vmx_preemption_timer_test/default.test
> > 2025-02-21 16:46:09,734 | 16809 | INFO | [Timed out] testcases/x86_64/pmu_counters_test/default.test
>
> I would really like to have terminal colored+bolded output for tests that fail
> (or timeout) or are skipped.
>
I will add color+bold for test which fails, timeouts, or skips.
> I think we should also provide controls for the verbosity of the output. E.g. to
> skip printing tests that pass entirely. My vote would be for a collection of
> boolean knobs, i.e. not a log_level or whatever, because inevitably we'll end up
> with output that isn't strictly "increasing".
>
> Adding a param to disable printing of passed tests is presumably trivial, so maybe
> do that for the initial commit, and then we can work on the fancier stuff?
You mean some command line options like:
testrunner --print-passed --print-failed
testrunner --print-skipped
testrunner --print-timeouts
testrunner --quiet
I can provide few options in the first commit, and then later we can
extend it based on usages.
>
> > 2025-02-21 16:46:10,202 | 16809 | INFO | [Passed] testcases/hardware_disable_test/default.test
> > 2025-02-21 16:46:10,203 | 16809 | INFO | Tests ran: 85 tests
>
> It would be very nice to have a summary of things printed out periodically. E.g.
> if my normal run has a few failures, but the current run has already failed a
> decent number of tests, then I'd probably kill the run and start debugging.
I can add a sticky bottom line which prints the current state of the
total, passed, failed, skipped and timeouts.
>
> Alternatively, and maybe even better, would be to make the runner mildly interactive,
> i.e. to accept rudimentary commands while tests are running. Then the user can
> query the state of things while the runner is still doing its thing. E.g. bind
> a few keys to print the various statuses.
>
> > 2025-02-21 16:46:10,204 | 16809 | INFO | Passed: 61
> > 2025-02-21 16:46:10,204 | 16809 | INFO | Failed: 4
> > 2025-02-21 16:46:10,204 | 16809 | INFO | Skipped: 17
> > 2025-02-21 16:46:10,204 | 16809 | INFO | Timed out: 3
> > 2025-02-21 16:46:10,204 | 16809 | INFO | No run: 0
>
> A not-quite-mandatory, but very-nice-to-have feature would be the ability to
> display which tests Passed/Failed/Skipped/Timed Out/Incomplete, with command line
> knobs for each. My vote is for everything but Passed on-by-default, though it's
> easy enough to put a light wrapper around this (which I'll do no matter what), so
> my preference for the default doesn't matter all that much.
>
> That could tie into the above idea of grabbing keys to print such information on-demand.
>
This will be very involved feature, lets punt it to a later versions, if
needed.
> Also CTRL-C handling needs much more graceful output. Ideally, no stack traces
> whatsover, and instead a summary like the above, but with information about which
> tests didn't complete.
>
Should be doable.
> > Output dumped in result directory
> >
> > $ tree result/
> > result/
>
> The runner should have an (on-by-default?) option to abort if the output directory
> already exists, e.g. so that users don't clobber previous runs. And/or an option
> to append a timestamp, e.g. $result.yyyy.mm.dd.MM.SS, so that all users don't end
> up writing the same wrapper to generate a timestamp.
>
> Having a no-timestamp + overwrite mode is also useful, e.g. when I'm running in
> a more "interactive" mode where I'm doing initial testing of something, and I
> don't care about
>
We can provide user an option like:
testrunner --output result_TIME
then internally runner will replace TIME with the current time?
If user doesn't provide _TIME then we can overwrite the directory
provided.
This sounds reasonable to me, what do you think?
> > +++ b/tools/testing/selftests/kvm/.gitignore
> > @@ -11,3 +11,4 @@
> > !Makefile
> > !Makefile.kvm
> > !*.test
> > +!*.py
>
> Sort this alphabetically as well.
Okay.
>
> > +def fetch_test_files(args):
> > + exclude_dirs = ["aarch64", "x86_64", "riscv", "s390x"]
>
> These are now:
>
> arm64, x86, riscv, s390
>
Yeah, I will make the change.
> > + def __init__(self, test_path, output_dir=None, timeout=None,):
> > + test_command = pathlib.Path(test_path).read_text().strip()
> > + if not test_command:
> > + raise ValueError("Empty test command in " + test_path)
> > +
> > + if output_dir is not None:
> > + output_dir = os.path.join(output_dir, test_path)
>
> This doesn't do the right thing if test_path is absolute (or maybe it's if the
> output_dir is in a completely different hierarchy?)
>
> I was able to fudge around this with
>
> diff --git a/tools/testing/selftests/kvm/runner/selftest.py b/tools/testing/selftests/kvm/runner/selftest.py
> index cdf5d1085c08..3bce023693cb 100644
> --- a/tools/testing/selftests/kvm/runner/selftest.py
> +++ b/tools/testing/selftests/kvm/runner/selftest.py
> @@ -30,7 +30,8 @@ class Selftest:
> raise ValueError("Empty test command in " + test_path)
>
> if output_dir is not None:
> - output_dir = os.path.join(output_dir, test_path)
> + dirpath, filename = os.path.split(test_path)
> + output_dir = os.path.join(output_dir, os.path.basename(dirpath), filename)
> self.test_path = test_path
> self.command = command.Command(test_command, timeout, output_dir)
> self.status = SelftestStatus.NO_RUN
>
I will fix it in next version. Thanks.
> Lastly, please don't wrap agressively (off-list, you mentioned it was due to a
> formatter?). E.g.
>
> diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
> index b9d34c20bf88..5a568e155477 100644
> --- a/tools/testing/selftests/kvm/runner/test_runner.py
> +++ b/tools/testing/selftests/kvm/runner/test_runner.py
> @@ -12,8 +12,7 @@ class TestRunner:
> self.tests = []
>
> for test_file in test_files:
> - self.tests.append(selftest.Selftest(
> - test_file, output_dir, timeout))
> + self.tests.append(selftest.Selftest(test_file, output_dir, timeout))
>
> def _run(self, test):
> test.run()
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/2] KVM: selftests: Add default testfiles for KVM selftests runner
2025-05-05 19:05 ` Vipin Sharma
@ 2025-05-05 22:57 ` Sean Christopherson
0 siblings, 0 replies; 9+ messages in thread
From: Sean Christopherson @ 2025-05-05 22:57 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton
On Mon, May 05, 2025, Vipin Sharma wrote:
> On 2025-04-30 10:01:29, Sean Christopherson wrote:
> > But, I do think we should commit the default.test files to the repository. If
> > they're ephemeral, then several problems arise:
> >
> > 1. For out-of-tree builds, the default.test files should arguably be placed in
> > the OUTPUT directory. But if/when we add curated testcases/, then we'll either
> > end up with multiple testcases/ directories (source and output), or we'll have
> > to copy testcases/ from the source to the output on a normal build, which is
> > rather gross. Or we'd need e.g. "make testcases", which is also gross, e.g.
> > I don't want to have to run yet more commands just to execute tests.
> >
> > 2. Generating default.test could overwrite a user-defined file. That's firmly
> > a user error, but at least if they default.test files are commited, the user
> > will get a hint or three that they're doing things wrong.
> >
> > 3. If the files aren't committed, then they probably should removed on "clean",
> > which isn't the end of the world since they're trivially easy to generate,
> > but it's kinda funky.
> >
> > So, what if we add this to auto-generate the files? It's obviously wasteful since
> > the files will exist 99.9999999% of the time, but the overhead is completely
> > negligible. The only concern I have is if this will do the wrong thing for some
> > build environments, i.e. shove the files in the wrong location.
>
> We can get the current path of the Makefile.kvm by writing this at the top
> of the Makefile.kvm:
> MAKEFILE_DIR := $(dir $(realpath $(lastword $(MAKEFILE_LIST))))
>
> Then MAKEFILE_DIR will have the source directory of Makfile.kvm and
> testcase will be in the same directory.
>
> With this we can modify the below foreach you wrote by prefixing
> MAKEFILE_DIR to "testcases".
>
> Does this alleviate concern regaring build environment?
Yeah, I think so. FWIW, "concern" probably isn't the right word, more like "the
only thing I haven't thought much about".
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 2/2] KVM: selftests: Create KVM selftest runner
2025-05-05 19:48 ` Vipin Sharma
@ 2025-05-05 23:26 ` Sean Christopherson
0 siblings, 0 replies; 9+ messages in thread
From: Sean Christopherson @ 2025-05-05 23:26 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton
On Mon, May 05, 2025, Vipin Sharma wrote:
> On 2025-04-30 14:31:05, Sean Christopherson wrote:
> > Printing the timestamps to the console isn't terrible interesting, and IMO isn't
> > at all worth the noise.
> >
> > The PID is nice, but it needs to be printed _before_ the test finishes, and it
> > needs to track the PID of the test. If getting that working is non-trivial,
> > definitely punt it for the initial commit.
> >
> > And presumably INFO is the level of logging. That needs to go.
> >
>
> Instead of removing timestamp, I can just print HH:MM:SS, I think it
> provides value in seeing how fast runner and tests are executing.
I can probably live with that. :-)
> > I think we should also provide controls for the verbosity of the output. E.g. to
> > skip printing tests that pass entirely. My vote would be for a collection of
> > boolean knobs, i.e. not a log_level or whatever, because inevitably we'll end up
> > with output that isn't strictly "increasing".
> >
> > Adding a param to disable printing of passed tests is presumably trivial, so maybe
> > do that for the initial commit, and then we can work on the fancier stuff?
>
> You mean some command line options like:
> testrunner --print-passed --print-failed
> testrunner --print-skipped
> testrunner --print-timeouts
> testrunner --quiet
Ya, something like that.
> I can provide few options in the first commit, and then later we can
> extend it based on usages.
+1
> > A not-quite-mandatory, but very-nice-to-have feature would be the ability to
> > display which tests Passed/Failed/Skipped/Timed Out/Incomplete, with command line
> > knobs for each. My vote is for everything but Passed on-by-default, though it's
> > easy enough to put a light wrapper around this (which I'll do no matter what), so
> > my preference for the default doesn't matter all that much.
> >
> > That could tie into the above idea of grabbing keys to print such information on-demand.
> >
>
> This will be very involved feature, lets punt it to a later versions, if
> needed.
Sounds good.
> > The runner should have an (on-by-default?) option to abort if the output directory
> > already exists, e.g. so that users don't clobber previous runs. And/or an option
> > to append a timestamp, e.g. $result.yyyy.mm.dd.MM.SS, so that all users don't end
> > up writing the same wrapper to generate a timestamp.
> >
> > Having a no-timestamp + overwrite mode is also useful, e.g. when I'm running in
> > a more "interactive" mode where I'm doing initial testing of something, and I
> > don't care about
> >
>
> We can provide user an option like:
> testrunner --output result_TIME
>
> then internally runner will replace TIME with the current time?
Why overload --output and then have to do more parsing? I assume adding options
is easy, so presumably --append-timestamp would be just as easy to add.
> If user doesn't provide _TIME then we can overwrite the directory
> provided.
I don't see any reason to tie those two together. Again, on the assumption that
adding options is mechaincally easy, I'd much rather have --overwrite or whatever.
In general, so long as it doesn't meaningfully increase complexity, make the
interface as flexible as possible so that the runner has a decent chance of being
able to handle whatever use cases people come up with, without needing constant
tweaking and churn.
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2025-05-06 4:59 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-02-22 0:59 [PATCH 0/2] Add KVM selftest runner Vipin Sharma
2025-02-22 0:59 ` [PATCH 1/2] KVM: selftests: Add default testfiles for KVM selftests runner Vipin Sharma
2025-04-30 17:01 ` Sean Christopherson
2025-05-05 19:05 ` Vipin Sharma
2025-05-05 22:57 ` Sean Christopherson
2025-02-22 0:59 ` [PATCH 2/2] KVM: selftests: Create KVM selftest runner Vipin Sharma
2025-04-30 21:31 ` Sean Christopherson
2025-05-05 19:48 ` Vipin Sharma
2025-05-05 23:26 ` Sean Christopherson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).