* [PATCH v2 00/15] Add KVM Selftests runner
@ 2025-06-06 23:56 Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 01/15] KVM: selftest: Create KVM selftest runner Vipin Sharma
` (15 more replies)
0 siblings, 16 replies; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Hello,
Create KVM Selftest Runner which allows running KVM selftests with added
features not present in default selftest runner provided by selftests
framework.
This Runner has two broad goals:
1. Make it easier for contributors and maintainers to run various
configuration of tests with features like preserving output,
controlling output verbosity, parallelism, different combinations of
command line arguments.
2. Provide common place to write interesting and useful combinations of
tests command line arguments to improve KVM test coverage. Default
selftests runner provide little to no control over this.
Patch 1 adds a very basic sefltest runner.
Patches 2-10 add various features to runner.
Patch 11 provides a way to auto generate test files.
Patch 12-15 commits the generated files for each architecture supported
by KVM selftests.
This version doesn't have all of the features envisioned in RFC and v1.
Future patches will add features like:
- Print process id of the test in execution.
- Skip other platform tests. Currently, runner expects test binary will
not be available and it marks that test as it didn't run in the
output. This can be handled by either hardcoding paths in runner or
extending *.test schema. I will work on it in the next version.
- CTRL+C currently spits out lots of warning (depending on --job value).
This will be fixed in the next version.
- Add more tests configurations.
- Provide a way to set the environment in which runner will start tests. For
example, setting huge pages, stress testing based on resources
available on host.
v2:
- Automatic default test generation.
- Command line flag to provide executables location
- Dump output to filesystem with timestamp
- Accept absolute path of *.test files/directory location
- Sticky status at bottom for the current state of runner.
- Knobs to control output verbosity
- Colored output for terminals.
v1: https://lore.kernel.org/kvm/20250222005943.3348627-1-vipinsh@google.com/
- Parallel test execution.
- Dumping separate output for each test.
- Timeout for test execution
- Specify single test or a test directory.
RFC: https://lore.kernel.org/kvm/20240821223012.3757828-1-vipinsh@google.com/
Vipin Sharma (15):
KVM: selftest: Create KVM selftest runner
KVM: selftests: Enable selftests runner to find executables in
different path
KVM: selftests: Add timeout option in selftests runner
KVM: selftests: Add option to save selftest runner output to a
directory
KVM: selftests: Run tests concurrently in KVM selftests runner
KVM: selftests: Add a flag to print only test status in KVM Selftests
run
KVM: selftests: Add various print flags to KVM Selftest Runner
KVM: selftests: Print sticky KVM Selftests Runner status at bottom
KVM: selftests: Add a flag to print only sticky summary in the
selftests runner
KVM: selftests: Add flag to suppress all output from Selftest KVM
Runner
KVM: selftests: Auto generate default tests for KVM Selftests Runner
KVM: selftests: Add x86 auto generated test files for KVM Selftests
Runner
KVM: selftests: Add arm64 auto generated test files for KVM Selftests
Runner
KVM: selftests: Add s390 auto generated test files for KVM Selftests
Runner
KVM: selftests: Add riscv auto generated test files for KVM Selftests
Runner
tools/testing/selftests/kvm/.gitignore | 4 +-
tools/testing/selftests/kvm/Makefile.kvm | 8 +
.../testing/selftests/kvm/runner/__main__.py | 271 ++++++++++++++++++
tools/testing/selftests/kvm/runner/command.py | 53 ++++
.../testing/selftests/kvm/runner/selftest.py | 66 +++++
.../selftests/kvm/runner/test_runner.py | 88 ++++++
.../access_tracking_perf_test/default.test | 1 +
.../kvm/tests/arch_timer/default.test | 1 +
.../tests/arm64/aarch32_id_regs/default.test | 1 +
.../arm64/arch_timer_edge_cases/default.test | 1 +
.../tests/arm64/debug-exceptions/default.test | 1 +
.../kvm/tests/arm64/hypercalls/default.test | 1 +
.../kvm/tests/arm64/mmio_abort/default.test | 1 +
.../kvm/tests/arm64/no-vgic-v3/default.test | 1 +
.../tests/arm64/page_fault_test/default.test | 1 +
.../kvm/tests/arm64/psci_test/default.test | 1 +
.../kvm/tests/arm64/set_id_regs/default.test | 1 +
.../kvm/tests/arm64/smccc_filter/default.test | 1 +
.../arm64/vcpu_width_config/default.test | 1 +
.../kvm/tests/arm64/vgic_init/default.test | 1 +
.../kvm/tests/arm64/vgic_irq/default.test | 1 +
.../tests/arm64/vgic_lpi_stress/default.test | 1 +
.../arm64/vpmu_counter_access/default.test | 1 +
.../kvm/tests/coalesced_io_test/default.test | 1 +
.../kvm/tests/demand_paging_test/default.test | 1 +
.../2slot_5vcpu_10iter.test | 1 +
.../tests/dirty_log_perf_test/default.test | 1 +
.../no_dirty_log_protect.test | 1 +
.../kvm/tests/dirty_log_test/default.test | 1 +
.../kvm/tests/get-reg-list/default.test | 1 +
.../kvm/tests/guest_memfd_test/default.test | 1 +
.../kvm/tests/guest_print_test/default.test | 1 +
.../tests/hardware_disable_test/default.test | 1 +
.../tests/kvm_binary_stats_test/default.test | 1 +
.../tests/kvm_create_max_vcpus/default.test | 1 +
.../tests/kvm_page_table_test/default.test | 1 +
.../default.test | 1 +
.../kvm/tests/memslot_perf_test/default.test | 1 +
.../kvm/tests/mmu_stress_test/default.test | 1 +
.../tests/pre_fault_memory_test/default.test | 1 +
.../kvm/tests/riscv/ebreak_test/default.test | 1 +
.../kvm/tests/riscv/sbi_pmu_test/default.test | 1 +
.../kvm/tests/rseq_test/default.test | 1 +
.../kvm/tests/s390/cmma_test/default.test | 1 +
.../s390/cpumodel_subfuncs_test/default.test | 1 +
.../kvm/tests/s390/debug_test/default.test | 1 +
.../kvm/tests/s390/memop/default.test | 1 +
.../kvm/tests/s390/resets/default.test | 1 +
.../s390/shared_zeropage_test/default.test | 1 +
.../tests/s390/sync_regs_test/default.test | 1 +
.../kvm/tests/s390/tprot/default.test | 1 +
.../kvm/tests/s390/ucontrol_test/default.test | 1 +
.../tests/set_memory_region_test/default.test | 1 +
.../kvm/tests/steal_time/default.test | 1 +
.../system_counter_offset_test/default.test | 1 +
.../kvm/tests/x86/amx_test/default.test | 1 +
.../x86/apic_bus_clock_test/default.test | 1 +
.../kvm/tests/x86/cpuid_test/default.test | 1 +
.../x86/cr4_cpuid_sync_test/default.test | 1 +
.../kvm/tests/x86/debug_regs/default.test | 1 +
.../default.test | 1 +
.../default.test | 1 +
.../kvm/tests/x86/fastops_test/default.test | 1 +
.../tests/x86/feature_msrs_test/default.test | 1 +
.../tests/x86/fix_hypercall_test/default.test | 1 +
.../kvm/tests/x86/hwcr_msr_test/default.test | 1 +
.../kvm/tests/x86/hyperv_clock/default.test | 1 +
.../kvm/tests/x86/hyperv_cpuid/default.test | 1 +
.../kvm/tests/x86/hyperv_evmcs/default.test | 1 +
.../hyperv_extended_hypercalls/default.test | 1 +
.../tests/x86/hyperv_features/default.test | 1 +
.../kvm/tests/x86/hyperv_ipi/default.test | 1 +
.../tests/x86/hyperv_svm_test/default.test | 1 +
.../tests/x86/hyperv_tlb_flush/default.test | 1 +
.../tests/x86/kvm_buslock_test/default.test | 1 +
.../kvm/tests/x86/kvm_clock_test/default.test | 1 +
.../kvm/tests/x86/kvm_pv_test/default.test | 1 +
.../x86/max_vcpuid_cap_test/default.test | 1 +
.../tests/x86/monitor_mwait_test/default.test | 1 +
.../x86/nested_emulation_test/default.test | 1 +
.../x86/nested_exceptions_test/default.test | 1 +
.../tests/x86/nx_huge_pages_test/default.test | 1 +
.../tests/x86/platform_info_test/default.test | 1 +
.../tests/x86/pmu_counters_test/default.test | 1 +
.../x86/pmu_event_filter_test/default.test | 1 +
.../private_mem_conversions_test/default.test | 1 +
.../private_mem_kvm_exits_test/default.test | 1 +
.../x86/recalc_apic_map_test/default.test | 1 +
.../tests/x86/set_boot_cpu_id/default.test | 1 +
.../kvm/tests/x86/set_sregs_test/default.test | 1 +
.../tests/x86/sev_init2_tests/default.test | 1 +
.../tests/x86/sev_migrate_tests/default.test | 1 +
.../kvm/tests/x86/sev_smoke_test/default.test | 1 +
.../default.test | 1 +
.../kvm/tests/x86/smm_test/default.test | 1 +
.../kvm/tests/x86/state_test/default.test | 1 +
.../tests/x86/svm_int_ctl_test/default.test | 1 +
.../x86/svm_nested_shutdown_test/default.test | 1 +
.../svm_nested_soft_inject_test/default.test | 1 +
.../tests/x86/svm_vmcall_test/default.test | 1 +
.../kvm/tests/x86/sync_regs_test/default.test | 1 +
.../x86/triple_fault_event_test/default.test | 1 +
.../kvm/tests/x86/tsc_msrs_test/default.test | 1 +
.../tests/x86/tsc_scaling_sync/default.test | 1 +
.../x86/ucna_injection_test/default.test | 1 +
.../tests/x86/userspace_io_test/default.test | 1 +
.../x86/userspace_msr_exit_test/default.test | 1 +
.../x86/vmx_apic_access_test/default.test | 1 +
.../vmx_close_while_nested_test/default.test | 1 +
.../tests/x86/vmx_dirty_log_test/default.test | 1 +
.../default.test | 1 +
.../default.test | 1 +
.../kvm/tests/x86/vmx_msrs_test/default.test | 1 +
.../vmx_nested_tsc_scaling_test/default.test | 1 +
.../tests/x86/vmx_pmu_caps_test/default.test | 1 +
.../vmx_preemption_timer_test/default.test | 1 +
.../vmx_set_nested_state_test/default.test | 1 +
.../x86/vmx_tsc_adjust_test/default.test | 1 +
.../kvm/tests/x86/xapic_ipi_test/default.test | 1 +
.../tests/x86/xapic_state_test/default.test | 1 +
.../tests/x86/xcr0_cpuid_test/default.test | 1 +
.../tests/x86/xen_shinfo_test/default.test | 1 +
.../tests/x86/xen_vmcall_test/default.test | 1 +
.../kvm/tests/x86/xss_msr_test/default.test | 1 +
124 files changed, 607 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/kvm/runner/__main__.py
create mode 100644 tools/testing/selftests/kvm/runner/command.py
create mode 100644 tools/testing/selftests/kvm/runner/selftest.py
create mode 100644 tools/testing/selftests/kvm/runner/test_runner.py
create mode 100644 tools/testing/selftests/kvm/tests/access_tracking_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arch_timer/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/aarch32_id_regs/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/arch_timer_edge_cases/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/debug-exceptions/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/hypercalls/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/mmio_abort/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/no-vgic-v3/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/page_fault_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/psci_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/set_id_regs/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/smccc_filter/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/vcpu_width_config/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/vgic_init/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/vgic_irq/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/vgic_lpi_stress/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/vpmu_counter_access/default.test
create mode 100644 tools/testing/selftests/kvm/tests/coalesced_io_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/demand_paging_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/dirty_log_perf_test/2slot_5vcpu_10iter.test
create mode 100644 tools/testing/selftests/kvm/tests/dirty_log_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/dirty_log_perf_test/no_dirty_log_protect.test
create mode 100644 tools/testing/selftests/kvm/tests/dirty_log_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/get-reg-list/default.test
create mode 100644 tools/testing/selftests/kvm/tests/guest_memfd_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/guest_print_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/hardware_disable_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/kvm_binary_stats_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/kvm_create_max_vcpus/default.test
create mode 100644 tools/testing/selftests/kvm/tests/kvm_page_table_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/memslot_modification_stress_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/memslot_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/mmu_stress_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/pre_fault_memory_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/rseq_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/cmma_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/cpumodel_subfuncs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/debug_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/memop/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/resets/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/shared_zeropage_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/sync_regs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/tprot/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/ucontrol_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/set_memory_region_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/steal_time/default.test
create mode 100644 tools/testing/selftests/kvm/tests/system_counter_offset_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/amx_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/apic_bus_clock_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/cpuid_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/cr4_cpuid_sync_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/debug_regs/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/dirty_log_page_splitting_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/exit_on_emulation_failure_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/fastops_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/feature_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/fix_hypercall_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hwcr_msr_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_clock/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_cpuid/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_evmcs/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_extended_hypercalls/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_features/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_ipi/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_svm_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_tlb_flush/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/kvm_buslock_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/kvm_clock_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/kvm_pv_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/max_vcpuid_cap_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/monitor_mwait_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/nested_emulation_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/nested_exceptions_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/nx_huge_pages_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/platform_info_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/pmu_counters_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/pmu_event_filter_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/private_mem_conversions_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/private_mem_kvm_exits_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/recalc_apic_map_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/set_boot_cpu_id/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/set_sregs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/sev_init2_tests/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/sev_migrate_tests/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/sev_smoke_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/smaller_maxphyaddr_emulation_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/smm_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/state_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/svm_int_ctl_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/svm_nested_shutdown_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/svm_nested_soft_inject_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/svm_vmcall_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/sync_regs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/triple_fault_event_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/tsc_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/tsc_scaling_sync/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/ucna_injection_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/userspace_io_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/userspace_msr_exit_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_apic_access_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_close_while_nested_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_dirty_log_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_exception_with_invalid_guest_state/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_invalid_nested_guest_state/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_nested_tsc_scaling_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_pmu_caps_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_preemption_timer_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_set_nested_state_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_tsc_adjust_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xapic_ipi_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xapic_state_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xcr0_cpuid_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xen_shinfo_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xen_vmcall_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xss_msr_test/default.test
base-commit: 3f7b307757ecffc1c18ede9ee3cf9ce8101f3cc9
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v2 01/15] KVM: selftest: Create KVM selftest runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-07-10 0:20 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 02/15] KVM: selftests: Enable selftests runner to find executables in different path Vipin Sharma
` (14 subsequent siblings)
15 siblings, 1 reply; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Implement a basic KVM selftest runner in Python to run selftests based
on the passed test configuration. Add command line options to select
individual test configuration file or a directory containing test
configuration files.
After selecting the tests to run, start their execution and print their
final execution status (passed, failed, skipped, no run), stdout and
stderr on terminal.
Print execution status in colors on the terminals where it is supported
to easily distinguish statuses from the stdout/stderr of the tests
execution.
If a test fails or times out, then return with a non-zero exit code
after all of the tests execution have completed. If none of the tests
fails or times out then exit with status 0
Provide some sample test configuration files to demonstrate the
execution of the runner.
Runner can be started from tools/testing/selftests/kvm directory as:
python3 runner --test-dirs tests
OR
python3 runner --test-files \
tests/dirty_log_perf_test/no_dirty_log_protect.test
This is a very basic implementation of the runner. Next patches will
enhance the runner by adding more features like parallelization, dumping
output to file system, time limit, out-of-tree builds run, etc.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
tools/testing/selftests/kvm/.gitignore | 4 +-
.../testing/selftests/kvm/runner/__main__.py | 92 +++++++++++++++++++
tools/testing/selftests/kvm/runner/command.py | 26 ++++++
.../testing/selftests/kvm/runner/selftest.py | 57 ++++++++++++
.../selftests/kvm/runner/test_runner.py | 41 +++++++++
.../2slot_5vcpu_10iter.test | 1 +
.../tests/dirty_log_perf_test/default.test | 1 +
.../no_dirty_log_protect.test | 1 +
8 files changed, 222 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/kvm/runner/__main__.py
create mode 100644 tools/testing/selftests/kvm/runner/command.py
create mode 100644 tools/testing/selftests/kvm/runner/selftest.py
create mode 100644 tools/testing/selftests/kvm/runner/test_runner.py
create mode 100644 tools/testing/selftests/kvm/tests/dirty_log_perf_test/2slot_5vcpu_10iter.test
create mode 100644 tools/testing/selftests/kvm/tests/dirty_log_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/dirty_log_perf_test/no_dirty_log_protect.test
diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 1d41a046a7bf..95af97b1ff9e 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -3,10 +3,12 @@
!/**/
!*.c
!*.h
+!*.py
!*.S
!*.sh
+!*.test
!.gitignore
!config
!settings
!Makefile
-!Makefile.kvm
\ No newline at end of file
+!Makefile.kvm
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
new file mode 100644
index 000000000000..b2c85606c516
--- /dev/null
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -0,0 +1,92 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright 2025 Google LLC
+#
+# Author: vipinsh@google.com (Vipin Sharma)
+
+import argparse
+import logging
+import os
+import sys
+
+from test_runner import TestRunner
+from selftest import SelftestStatus
+
+
+def cli():
+ parser = argparse.ArgumentParser(
+ prog="KVM Selftests Runner",
+ formatter_class=argparse.RawTextHelpFormatter,
+ allow_abbrev=False
+ )
+
+ parser.add_argument("--test-files",
+ nargs="*",
+ default=[],
+ help="Test files to run. Provide the space separated test file paths")
+
+ parser.add_argument("--test-dirs",
+ nargs="*",
+ default=[],
+ help="Run tests in the given directory and all of its sub directories. Provide the space separated paths to add multiple directories.")
+
+ return parser.parse_args()
+
+
+def setup_logging(args):
+ class TerminalColorFormatter(logging.Formatter):
+ reset = "\033[0m"
+ red_bold = "\033[31;1m"
+ green = "\033[32m"
+ yellow = "\033[33m"
+ blue = "\033[34m"
+
+ COLORS = {
+ SelftestStatus.PASSED: green,
+ SelftestStatus.NO_RUN: blue,
+ SelftestStatus.SKIPPED: yellow,
+ SelftestStatus.FAILED: red_bold
+ }
+
+ def __init__(self, fmt=None, datefmt=None):
+ super().__init__(fmt, datefmt)
+
+ def format(self, record):
+ return (self.COLORS.get(record.levelno, "") +
+ super().format(record) + self.reset)
+
+ logger = logging.getLogger("runner")
+ logger.setLevel(logging.INFO)
+
+ ch = logging.StreamHandler()
+ ch_formatter = TerminalColorFormatter(fmt="%(asctime)s | %(message)s",
+ datefmt="%H:%M:%S")
+ ch.setFormatter(ch_formatter)
+ logger.addHandler(ch)
+
+
+def fetch_tests_from_dirs(scan_dirs):
+ test_files = []
+ for scan_dir in scan_dirs:
+ for root, dirs, files in os.walk(scan_dir):
+ for file in files:
+ test_files.append(os.path.join(root, file))
+ return test_files
+
+
+def fetch_test_files(args):
+ test_files = args.test_files
+ test_files.extend(fetch_tests_from_dirs(args.test_dirs))
+ # Remove duplicates
+ test_files = list(dict.fromkeys(test_files))
+ return test_files
+
+
+def main():
+ args = cli()
+ setup_logging(args)
+ test_files = fetch_test_files(args)
+ return TestRunner(test_files).start()
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/tools/testing/selftests/kvm/runner/command.py b/tools/testing/selftests/kvm/runner/command.py
new file mode 100644
index 000000000000..a63ff53a92b3
--- /dev/null
+++ b/tools/testing/selftests/kvm/runner/command.py
@@ -0,0 +1,26 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright 2025 Google LLC
+#
+# Author: vipinsh@google.com (Vipin Sharma)
+
+import subprocess
+
+
+class Command:
+ """Executes a command in shell.
+
+ Returns the exit code, std output and std error of the command.
+ """
+
+ def __init__(self, command):
+ self.command = command
+
+ def run(self):
+ run_args = {
+ "universal_newlines": True,
+ "shell": True,
+ "capture_output": True,
+ }
+
+ proc = subprocess.run(self.command, **run_args)
+ return proc.returncode, proc.stdout, proc.stderr
diff --git a/tools/testing/selftests/kvm/runner/selftest.py b/tools/testing/selftests/kvm/runner/selftest.py
new file mode 100644
index 000000000000..cc56c45b1c93
--- /dev/null
+++ b/tools/testing/selftests/kvm/runner/selftest.py
@@ -0,0 +1,57 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright 2025 Google LLC
+#
+# Author: vipinsh@google.com (Vipin Sharma)
+
+import command
+import pathlib
+import enum
+import os
+
+
+class SelftestStatus(enum.IntEnum):
+ """
+ Selftest Status. Integer values are just +1 to the logging.INFO level.
+ """
+
+ PASSED = 21
+ NO_RUN = 22
+ SKIPPED = 23
+ FAILED = 24
+
+ def __str__(self):
+ return str.__str__(self.name)
+
+
+class Selftest:
+ """
+ Represents a single selftest.
+
+ Extract the test execution command from test file and executes it.
+ """
+
+ def __init__(self, test_path):
+ test_command = pathlib.Path(test_path).read_text().strip()
+ if not test_command:
+ raise ValueError("Empty test command in " + test_path)
+
+ test_command = os.path.join(".", test_command)
+ self.exists = os.path.isfile(test_command.split(maxsplit=1)[0])
+ self.test_path = test_path
+ self.command = command.Command(test_command)
+ self.status = SelftestStatus.NO_RUN
+ self.stdout = ""
+ self.stderr = ""
+
+ def run(self):
+ if not self.exists:
+ self.stderr = "File doesn't exists."
+ return
+
+ ret, self.stdout, self.stderr = self.command.run()
+ if ret == 0:
+ self.status = SelftestStatus.PASSED
+ elif ret == 4:
+ self.status = SelftestStatus.SKIPPED
+ else:
+ self.status = SelftestStatus.FAILED
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
new file mode 100644
index 000000000000..20ea523629de
--- /dev/null
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright 2025 Google LLC
+#
+# Author: vipinsh@google.com (Vipin Sharma)
+
+import logging
+from selftest import Selftest
+from selftest import SelftestStatus
+
+logger = logging.getLogger("runner")
+
+
+class TestRunner:
+ def __init__(self, test_files):
+ self.tests = []
+
+ for test_file in test_files:
+ self.tests.append(Selftest(test_file))
+
+ def _log_result(self, test_result):
+ logger.log(test_result.status,
+ f"[{test_result.status}] {test_result.test_path}")
+ logger.info("************** STDOUT BEGIN **************")
+ logger.info(test_result.stdout)
+ logger.info("************** STDOUT END **************")
+ logger.info("************** STDERR BEGIN **************")
+ logger.info(test_result.stderr)
+ logger.info("************** STDERR END **************")
+
+ def start(self):
+ ret = 0
+
+ for test in self.tests:
+ test.run()
+ self._log_result(test)
+
+ if (test.status not in [SelftestStatus.PASSED,
+ SelftestStatus.NO_RUN,
+ SelftestStatus.SKIPPED]):
+ ret = 1
+ return ret
diff --git a/tools/testing/selftests/kvm/tests/dirty_log_perf_test/2slot_5vcpu_10iter.test b/tools/testing/selftests/kvm/tests/dirty_log_perf_test/2slot_5vcpu_10iter.test
new file mode 100644
index 000000000000..5b8d56b44a75
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/dirty_log_perf_test/2slot_5vcpu_10iter.test
@@ -0,0 +1 @@
+dirty_log_perf_test -x 2 -v 5 -i 10
diff --git a/tools/testing/selftests/kvm/tests/dirty_log_perf_test/default.test b/tools/testing/selftests/kvm/tests/dirty_log_perf_test/default.test
new file mode 100644
index 000000000000..98f423e15b46
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/dirty_log_perf_test/default.test
@@ -0,0 +1 @@
+dirty_log_perf_test
diff --git a/tools/testing/selftests/kvm/tests/dirty_log_perf_test/no_dirty_log_protect.test b/tools/testing/selftests/kvm/tests/dirty_log_perf_test/no_dirty_log_protect.test
new file mode 100644
index 000000000000..ed3490b1d1a1
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/dirty_log_perf_test/no_dirty_log_protect.test
@@ -0,0 +1 @@
+dirty_log_perf_test -g
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 02/15] KVM: selftests: Enable selftests runner to find executables in different path
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 01/15] KVM: selftest: Create KVM selftest runner Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-07-09 21:39 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner Vipin Sharma
` (13 subsequent siblings)
15 siblings, 1 reply; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add command line option, --executable/-e, to specify a directory where
test binaries are present. If this option is not provided then default
to the current directory.
Example:
python3 runner --test-dirs test -e ~/build/selftests
This option enables executing tests from out-of-tree builds.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
tools/testing/selftests/kvm/runner/__main__.py | 8 +++++++-
tools/testing/selftests/kvm/runner/selftest.py | 4 ++--
tools/testing/selftests/kvm/runner/test_runner.py | 4 ++--
3 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
index b2c85606c516..599300831504 100644
--- a/tools/testing/selftests/kvm/runner/__main__.py
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -29,6 +29,12 @@ def cli():
default=[],
help="Run tests in the given directory and all of its sub directories. Provide the space separated paths to add multiple directories.")
+ parser.add_argument("-e",
+ "--executable",
+ nargs='?',
+ default=".",
+ help="Finds the test executables in the given directory. Default is the current directory.")
+
return parser.parse_args()
@@ -85,7 +91,7 @@ def main():
args = cli()
setup_logging(args)
test_files = fetch_test_files(args)
- return TestRunner(test_files).start()
+ return TestRunner(test_files, args).start()
if __name__ == "__main__":
diff --git a/tools/testing/selftests/kvm/runner/selftest.py b/tools/testing/selftests/kvm/runner/selftest.py
index cc56c45b1c93..a0b06f150087 100644
--- a/tools/testing/selftests/kvm/runner/selftest.py
+++ b/tools/testing/selftests/kvm/runner/selftest.py
@@ -30,12 +30,12 @@ class Selftest:
Extract the test execution command from test file and executes it.
"""
- def __init__(self, test_path):
+ def __init__(self, test_path, executable_dir):
test_command = pathlib.Path(test_path).read_text().strip()
if not test_command:
raise ValueError("Empty test command in " + test_path)
- test_command = os.path.join(".", test_command)
+ test_command = os.path.join(executable_dir, test_command)
self.exists = os.path.isfile(test_command.split(maxsplit=1)[0])
self.test_path = test_path
self.command = command.Command(test_command)
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
index 20ea523629de..104f0b4c2e4e 100644
--- a/tools/testing/selftests/kvm/runner/test_runner.py
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -11,11 +11,11 @@ logger = logging.getLogger("runner")
class TestRunner:
- def __init__(self, test_files):
+ def __init__(self, test_files, args):
self.tests = []
for test_file in test_files:
- self.tests.append(Selftest(test_file))
+ self.tests.append(Selftest(test_file, args.executable))
def _log_result(self, test_result):
logger.log(test_result.status,
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 01/15] KVM: selftest: Create KVM selftest runner Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 02/15] KVM: selftests: Enable selftests runner to find executables in different path Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-07-09 21:46 ` Sean Christopherson
2025-07-09 22:25 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 04/15] KVM: selftests: Add option to save selftest runner output to a directory Vipin Sharma
` (12 subsequent siblings)
15 siblings, 2 replies; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add a command line argument in KVM selftest runner to limit amount of
time (seconds) given to a test for execution. Kill the test if it exceeds the
given value. Define a new SelftestStatus.TIMED_OUT to denote a selftest
final result. Add terminal color for status messages of timed out tests.
Set the default value of 120 seconds for all tests.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
.../testing/selftests/kvm/runner/__main__.py | 10 +++++++-
tools/testing/selftests/kvm/runner/command.py | 4 +++-
.../testing/selftests/kvm/runner/selftest.py | 23 +++++++++++--------
.../selftests/kvm/runner/test_runner.py | 2 +-
4 files changed, 27 insertions(+), 12 deletions(-)
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
index 599300831504..f7f679be0e03 100644
--- a/tools/testing/selftests/kvm/runner/__main__.py
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -35,6 +35,12 @@ def cli():
default=".",
help="Finds the test executables in the given directory. Default is the current directory.")
+ parser.add_argument("-t",
+ "--timeout",
+ default=120,
+ type=int,
+ help="Timeout, in seconds, before runner kills the running test. (Default: 120 seconds)")
+
return parser.parse_args()
@@ -42,6 +48,7 @@ def setup_logging(args):
class TerminalColorFormatter(logging.Formatter):
reset = "\033[0m"
red_bold = "\033[31;1m"
+ red = "\033[31;1m"
green = "\033[32m"
yellow = "\033[33m"
blue = "\033[34m"
@@ -50,7 +57,8 @@ def setup_logging(args):
SelftestStatus.PASSED: green,
SelftestStatus.NO_RUN: blue,
SelftestStatus.SKIPPED: yellow,
- SelftestStatus.FAILED: red_bold
+ SelftestStatus.FAILED: red_bold,
+ SelftestStatus.TIMED_OUT: red
}
def __init__(self, fmt=None, datefmt=None):
diff --git a/tools/testing/selftests/kvm/runner/command.py b/tools/testing/selftests/kvm/runner/command.py
index a63ff53a92b3..44c8e0875779 100644
--- a/tools/testing/selftests/kvm/runner/command.py
+++ b/tools/testing/selftests/kvm/runner/command.py
@@ -12,14 +12,16 @@ class Command:
Returns the exit code, std output and std error of the command.
"""
- def __init__(self, command):
+ def __init__(self, command, timeout):
self.command = command
+ self.timeout = timeout
def run(self):
run_args = {
"universal_newlines": True,
"shell": True,
"capture_output": True,
+ "timeout": self.timeout,
}
proc = subprocess.run(self.command, **run_args)
diff --git a/tools/testing/selftests/kvm/runner/selftest.py b/tools/testing/selftests/kvm/runner/selftest.py
index a0b06f150087..4c72108c47de 100644
--- a/tools/testing/selftests/kvm/runner/selftest.py
+++ b/tools/testing/selftests/kvm/runner/selftest.py
@@ -7,6 +7,7 @@ import command
import pathlib
import enum
import os
+import subprocess
class SelftestStatus(enum.IntEnum):
@@ -18,6 +19,7 @@ class SelftestStatus(enum.IntEnum):
NO_RUN = 22
SKIPPED = 23
FAILED = 24
+ TIMED_OUT = 25
def __str__(self):
return str.__str__(self.name)
@@ -30,7 +32,7 @@ class Selftest:
Extract the test execution command from test file and executes it.
"""
- def __init__(self, test_path, executable_dir):
+ def __init__(self, test_path, executable_dir, timeout):
test_command = pathlib.Path(test_path).read_text().strip()
if not test_command:
raise ValueError("Empty test command in " + test_path)
@@ -38,7 +40,7 @@ class Selftest:
test_command = os.path.join(executable_dir, test_command)
self.exists = os.path.isfile(test_command.split(maxsplit=1)[0])
self.test_path = test_path
- self.command = command.Command(test_command)
+ self.command = command.Command(test_command, timeout)
self.status = SelftestStatus.NO_RUN
self.stdout = ""
self.stderr = ""
@@ -48,10 +50,13 @@ class Selftest:
self.stderr = "File doesn't exists."
return
- ret, self.stdout, self.stderr = self.command.run()
- if ret == 0:
- self.status = SelftestStatus.PASSED
- elif ret == 4:
- self.status = SelftestStatus.SKIPPED
- else:
- self.status = SelftestStatus.FAILED
+ try:
+ ret, self.stdout, self.stderr = self.command.run()
+ if ret == 0:
+ self.status = SelftestStatus.PASSED
+ elif ret == 4:
+ self.status = SelftestStatus.SKIPPED
+ else:
+ self.status = SelftestStatus.FAILED
+ except subprocess.TimeoutExpired as e:
+ self.status = SelftestStatus.TIMED_OUT
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
index 104f0b4c2e4e..1409e1cfe7d5 100644
--- a/tools/testing/selftests/kvm/runner/test_runner.py
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -15,7 +15,7 @@ class TestRunner:
self.tests = []
for test_file in test_files:
- self.tests.append(Selftest(test_file, args.executable))
+ self.tests.append(Selftest(test_file, args.executable, args.timeout))
def _log_result(self, test_result):
logger.log(test_result.status,
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 04/15] KVM: selftests: Add option to save selftest runner output to a directory
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (2 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-07-09 21:52 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 05/15] KVM: selftests: Run tests concurrently in KVM selftests runner Vipin Sharma
` (11 subsequent siblings)
15 siblings, 1 reply; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add a command line flag, --output/-o, to selftest runner which enables to
save individual tests output (stdout & stderr) stream to a directory in
a hierarchical way. Create folder hierarchy same as tests hieararcy
given by --test-files and --test-dirs.
Also, add a command line flag, --append-output-time, which will append
timestamp (format YYYY.M.DD.HH.MM.SS) to the directory name given in
--output flag.
Example:
python3 runner --test-dirs test -o test_result --append_output_time
This will create test_result.2025.06.06.08.45.57 directory.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
.../testing/selftests/kvm/runner/__main__.py | 30 ++++++++++++++++--
tools/testing/selftests/kvm/runner/command.py | 31 +++++++++++++++++--
.../testing/selftests/kvm/runner/selftest.py | 8 +++--
.../selftests/kvm/runner/test_runner.py | 17 +++++-----
4 files changed, 72 insertions(+), 14 deletions(-)
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
index f7f679be0e03..54bdc248b13f 100644
--- a/tools/testing/selftests/kvm/runner/__main__.py
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -7,6 +7,8 @@ import argparse
import logging
import os
import sys
+import datetime
+import pathlib
from test_runner import TestRunner
from selftest import SelftestStatus
@@ -41,6 +43,16 @@ def cli():
type=int,
help="Timeout, in seconds, before runner kills the running test. (Default: 120 seconds)")
+ parser.add_argument("-o",
+ "--output",
+ nargs='?',
+ help="Dumps test runner output which includes each test execution result, their stdouts and stderrs hierarchically in the given directory.")
+
+ parser.add_argument("--append-output-time",
+ action="store_true",
+ default=False,
+ help="Appends timestamp to the output directory.")
+
return parser.parse_args()
@@ -71,12 +83,26 @@ def setup_logging(args):
logger = logging.getLogger("runner")
logger.setLevel(logging.INFO)
+ formatter_args = {
+ "fmt": "%(asctime)s | %(message)s",
+ "datefmt": "%H:%M:%S"
+ }
+
ch = logging.StreamHandler()
- ch_formatter = TerminalColorFormatter(fmt="%(asctime)s | %(message)s",
- datefmt="%H:%M:%S")
+ ch_formatter = TerminalColorFormatter(**formatter_args)
ch.setFormatter(ch_formatter)
logger.addHandler(ch)
+ if args.output != None:
+ if (args.append_output_time):
+ args.output += datetime.datetime.now().strftime(".%Y.%m.%d.%H.%M.%S")
+ pathlib.Path(args.output).mkdir(parents=True, exist_ok=True)
+ logging_file = os.path.join(args.output, "log")
+ fh = logging.FileHandler(logging_file)
+ fh_formatter = logging.Formatter(**formatter_args)
+ fh.setFormatter(fh_formatter)
+ logger.addHandler(fh)
+
def fetch_tests_from_dirs(scan_dirs):
test_files = []
diff --git a/tools/testing/selftests/kvm/runner/command.py b/tools/testing/selftests/kvm/runner/command.py
index 44c8e0875779..6f6b1811b490 100644
--- a/tools/testing/selftests/kvm/runner/command.py
+++ b/tools/testing/selftests/kvm/runner/command.py
@@ -4,6 +4,9 @@
# Author: vipinsh@google.com (Vipin Sharma)
import subprocess
+import pathlib
+import contextlib
+import os
class Command:
@@ -12,17 +15,39 @@ class Command:
Returns the exit code, std output and std error of the command.
"""
- def __init__(self, command, timeout):
+ def __init__(self, command, timeout, output_dir):
self.command = command
self.timeout = timeout
+ self.output_dir = output_dir
- def run(self):
+ def _run(self, output=None, error=None):
run_args = {
"universal_newlines": True,
"shell": True,
- "capture_output": True,
"timeout": self.timeout,
}
+ if output is None and error is None:
+ run_args.update({"capture_output": True})
+ else:
+ run_args.update({"stdout": output, "stderr": error})
+
proc = subprocess.run(self.command, **run_args)
return proc.returncode, proc.stdout, proc.stderr
+
+ def run(self):
+ if self.output_dir is not None:
+ pathlib.Path(self.output_dir).mkdir(parents=True, exist_ok=True)
+
+ output = None
+ error = None
+ with contextlib.ExitStack() as stack:
+ if self.output_dir is not None:
+ output_path = os.path.join(self.output_dir, "stdout")
+ output = stack.enter_context(
+ open(output_path, encoding="utf-8", mode="w"))
+
+ error_path = os.path.join(self.output_dir, "stderr")
+ error = stack.enter_context(
+ open(error_path, encoding="utf-8", mode="w"))
+ return self._run(output, error)
diff --git a/tools/testing/selftests/kvm/runner/selftest.py b/tools/testing/selftests/kvm/runner/selftest.py
index 4c72108c47de..664958c693e5 100644
--- a/tools/testing/selftests/kvm/runner/selftest.py
+++ b/tools/testing/selftests/kvm/runner/selftest.py
@@ -32,7 +32,7 @@ class Selftest:
Extract the test execution command from test file and executes it.
"""
- def __init__(self, test_path, executable_dir, timeout):
+ def __init__(self, test_path, executable_dir, timeout, output_dir):
test_command = pathlib.Path(test_path).read_text().strip()
if not test_command:
raise ValueError("Empty test command in " + test_path)
@@ -40,7 +40,11 @@ class Selftest:
test_command = os.path.join(executable_dir, test_command)
self.exists = os.path.isfile(test_command.split(maxsplit=1)[0])
self.test_path = test_path
- self.command = command.Command(test_command, timeout)
+
+ if output_dir is not None:
+ output_dir = os.path.join(output_dir, test_path.lstrip("/"))
+ self.command = command.Command(test_command, timeout, output_dir)
+
self.status = SelftestStatus.NO_RUN
self.stdout = ""
self.stderr = ""
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
index 1409e1cfe7d5..0501d77a9912 100644
--- a/tools/testing/selftests/kvm/runner/test_runner.py
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -13,19 +13,22 @@ logger = logging.getLogger("runner")
class TestRunner:
def __init__(self, test_files, args):
self.tests = []
+ self.output_dir = args.output
for test_file in test_files:
- self.tests.append(Selftest(test_file, args.executable, args.timeout))
+ self.tests.append(Selftest(test_file, args.executable,
+ args.timeout, args.output))
def _log_result(self, test_result):
logger.log(test_result.status,
f"[{test_result.status}] {test_result.test_path}")
- logger.info("************** STDOUT BEGIN **************")
- logger.info(test_result.stdout)
- logger.info("************** STDOUT END **************")
- logger.info("************** STDERR BEGIN **************")
- logger.info(test_result.stderr)
- logger.info("************** STDERR END **************")
+ if (self.output_dir is None):
+ logger.info("************** STDOUT BEGIN **************")
+ logger.info(test_result.stdout)
+ logger.info("************** STDOUT END **************")
+ logger.info("************** STDERR BEGIN **************")
+ logger.info(test_result.stderr)
+ logger.info("************** STDERR END **************")
def start(self):
ret = 0
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 05/15] KVM: selftests: Run tests concurrently in KVM selftests runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (3 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 04/15] KVM: selftests: Add option to save selftest runner output to a directory Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 06/15] KVM: selftests: Add a flag to print only test status in KVM Selftests run Vipin Sharma
` (10 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add a command line argument, --jobs, to specify how many tests can
execute concurrently. Set default to 1.
Example:
python3 runner --test-dirs tests -j 10
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
.../testing/selftests/kvm/runner/__main__.py | 6 ++++
.../selftests/kvm/runner/test_runner.py | 28 +++++++++++++------
2 files changed, 26 insertions(+), 8 deletions(-)
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
index 54bdc248b13f..48d7ce00a097 100644
--- a/tools/testing/selftests/kvm/runner/__main__.py
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -53,6 +53,12 @@ def cli():
default=False,
help="Appends timestamp to the output directory.")
+ parser.add_argument("-j",
+ "--jobs",
+ default=1,
+ type=int,
+ help="Maximum number of tests that can be run concurrently. (Default: 1)")
+
return parser.parse_args()
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
index 0501d77a9912..0a6e5e0ca0f5 100644
--- a/tools/testing/selftests/kvm/runner/test_runner.py
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -4,6 +4,8 @@
# Author: vipinsh@google.com (Vipin Sharma)
import logging
+import concurrent.futures
+
from selftest import Selftest
from selftest import SelftestStatus
@@ -14,11 +16,16 @@ class TestRunner:
def __init__(self, test_files, args):
self.tests = []
self.output_dir = args.output
+ self.jobs = args.jobs
for test_file in test_files:
self.tests.append(Selftest(test_file, args.executable,
args.timeout, args.output))
+ def _run_test(self, test):
+ test.run()
+ return test
+
def _log_result(self, test_result):
logger.log(test_result.status,
f"[{test_result.status}] {test_result.test_path}")
@@ -33,12 +40,17 @@ class TestRunner:
def start(self):
ret = 0
- for test in self.tests:
- test.run()
- self._log_result(test)
-
- if (test.status not in [SelftestStatus.PASSED,
- SelftestStatus.NO_RUN,
- SelftestStatus.SKIPPED]):
- ret = 1
+ with concurrent.futures.ProcessPoolExecutor(max_workers=self.jobs) as executor:
+ all_futures = []
+ for test in self.tests:
+ future = executor.submit(self._run_test, test)
+ all_futures.append(future)
+
+ for future in concurrent.futures.as_completed(all_futures):
+ test_result = future.result()
+ self._log_result(test_result)
+ if (test_result.status not in [SelftestStatus.PASSED,
+ SelftestStatus.NO_RUN,
+ SelftestStatus.SKIPPED]):
+ ret = 1
return ret
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 06/15] KVM: selftests: Add a flag to print only test status in KVM Selftests run
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (4 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 05/15] KVM: selftests: Run tests concurrently in KVM selftests runner Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-07-09 21:55 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 07/15] KVM: selftests: Add various print flags to KVM Selftest Runner Vipin Sharma
` (9 subsequent siblings)
15 siblings, 1 reply; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add a command line argument, --print-status, to limit content printed on
terminal by default. When this flag is passed only print final status of
tests i.e. passed, failed, timed out, etc.
Example:
python3 runner --test-dirs tests --print-status
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
tools/testing/selftests/kvm/runner/__main__.py | 5 +++++
tools/testing/selftests/kvm/runner/test_runner.py | 3 ++-
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
index 48d7ce00a097..3f11a20e76a9 100644
--- a/tools/testing/selftests/kvm/runner/__main__.py
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -59,6 +59,11 @@ def cli():
type=int,
help="Maximum number of tests that can be run concurrently. (Default: 1)")
+ parser.add_argument("--print-status",
+ action="store_true",
+ default=False,
+ help="Print only test's status and avoid printing stdout and stderr of the tests")
+
return parser.parse_args()
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
index 0a6e5e0ca0f5..474408fcab51 100644
--- a/tools/testing/selftests/kvm/runner/test_runner.py
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -17,6 +17,7 @@ class TestRunner:
self.tests = []
self.output_dir = args.output
self.jobs = args.jobs
+ self.print_status = args.print_status
for test_file in test_files:
self.tests.append(Selftest(test_file, args.executable,
@@ -29,7 +30,7 @@ class TestRunner:
def _log_result(self, test_result):
logger.log(test_result.status,
f"[{test_result.status}] {test_result.test_path}")
- if (self.output_dir is None):
+ if (self.output_dir is None and self.print_status is False):
logger.info("************** STDOUT BEGIN **************")
logger.info(test_result.stdout)
logger.info("************** STDOUT END **************")
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 07/15] KVM: selftests: Add various print flags to KVM Selftest Runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (5 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 06/15] KVM: selftests: Add a flag to print only test status in KVM Selftests run Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-07-09 22:01 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 08/15] KVM: selftests: Print sticky KVM Selftests Runner status at bottom Vipin Sharma
` (8 subsequent siblings)
15 siblings, 1 reply; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add various print flags to selectively print outputs on terminal based
on test execution status (passed, failed, timed out, skipped, no run).
Provide further options to print only particular execution status, like
print only status of failed tests.
Example: To print status, stdout and stderr for failed tests and only
print status of passed test:
python3 runner --test-dirs tests --print-failed \
--print-passed-status
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
.../testing/selftests/kvm/runner/__main__.py | 114 ++++++++++++++++++
.../selftests/kvm/runner/test_runner.py | 10 +-
2 files changed, 123 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
index 3f11a20e76a9..4406d8e4847a 100644
--- a/tools/testing/selftests/kvm/runner/__main__.py
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -64,9 +64,115 @@ def cli():
default=False,
help="Print only test's status and avoid printing stdout and stderr of the tests")
+ parser.add_argument("--print-passed",
+ action="store_true",
+ default=False,
+ help="Print passed test's stdout, stderr and status."
+ )
+
+ parser.add_argument("--print-passed-status",
+ action="store_true",
+ default=False,
+ help="Print only passed test's status."
+ )
+
+ parser.add_argument("--print-failed",
+ action="store_true",
+ default=False,
+ help="Print failed test's stdout, stderr and status."
+ )
+
+ parser.add_argument("--print-failed-status",
+ action="store_true",
+ default=False,
+ help="Print only failed test's status."
+ )
+
+ parser.add_argument("--print-skipped",
+ action="store_true",
+ default=False,
+ help="Print skipped test's stdout, stderr and status."
+ )
+
+ parser.add_argument("--print-skipped-status",
+ action="store_true",
+ default=False,
+ help="Print only skipped test's status."
+ )
+
+ parser.add_argument("--print-timed-out",
+ action="store_true",
+ default=False,
+ help="Print timed out test's stdout, stderr and status."
+ )
+
+ parser.add_argument("--print-timed-out-status",
+ action="store_true",
+ default=False,
+ help="Print only timed out test's status."
+ )
+
+ parser.add_argument("--print-no-runs",
+ action="store_true",
+ default=False,
+ help="Print stdout, stderr and status for tests which didn't run."
+ )
+
+ parser.add_argument("--print-no-runs-status",
+ action="store_true",
+ default=False,
+ help="Print only tests which didn't run."
+ )
+
return parser.parse_args()
+def level_filters(args):
+ # Levels added here will be printed by logger.
+ levels = set()
+
+ if args.print_passed or args.print_passed_status or args.print_status:
+ levels.add(SelftestStatus.PASSED)
+
+ if args.print_failed or args.print_failed_status or args.print_status:
+ levels.add(SelftestStatus.FAILED)
+
+ if args.print_skipped or args.print_skipped_status or args.print_status:
+ levels.add(SelftestStatus.SKIPPED)
+
+ if args.print_timed_out or args.print_timed_out_status or args.print_status:
+ levels.add(SelftestStatus.TIMED_OUT)
+
+ if args.print_no_runs or args.print_no_runs_status or args.print_status:
+ levels.add(SelftestStatus.NO_RUN)
+
+ # Nothing set explicitly, enable all.
+ if not levels:
+ args.print_passed = True
+ levels.add(SelftestStatus.PASSED)
+
+ args.print_failed = True
+ levels.add(SelftestStatus.FAILED)
+
+ args.print_skipped = True
+ levels.add(SelftestStatus.SKIPPED)
+
+ args.print_timed_out = True
+ levels.add(SelftestStatus.TIMED_OUT)
+
+ args.print_no_runs = True
+ levels.add(SelftestStatus.NO_RUN)
+
+ levels.add(logging.NOTSET)
+ levels.add(logging.DEBUG)
+ levels.add(logging.INFO)
+ levels.add(logging.WARN)
+ levels.add(logging.ERROR)
+ levels.add(logging.CRITICAL)
+
+ return levels
+
+
def setup_logging(args):
class TerminalColorFormatter(logging.Formatter):
reset = "\033[0m"
@@ -91,6 +197,13 @@ def setup_logging(args):
return (self.COLORS.get(record.levelno, "") +
super().format(record) + self.reset)
+ class LevelFilter:
+ def __init__(self, levels):
+ self.levels = levels
+
+ def filter(self, record):
+ return record.levelno in self.levels
+
logger = logging.getLogger("runner")
logger.setLevel(logging.INFO)
@@ -102,6 +215,7 @@ def setup_logging(args):
ch = logging.StreamHandler()
ch_formatter = TerminalColorFormatter(**formatter_args)
ch.setFormatter(ch_formatter)
+ ch.addFilter(LevelFilter(level_filters(args)))
logger.addHandler(ch)
if args.output != None:
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
index 474408fcab51..8f2372834104 100644
--- a/tools/testing/selftests/kvm/runner/test_runner.py
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -18,6 +18,13 @@ class TestRunner:
self.output_dir = args.output
self.jobs = args.jobs
self.print_status = args.print_status
+ self.print_stds = {
+ SelftestStatus.PASSED: args.print_passed,
+ SelftestStatus.FAILED: args.print_failed,
+ SelftestStatus.SKIPPED: args.print_skipped,
+ SelftestStatus.TIMED_OUT: args.print_timed_out,
+ SelftestStatus.NO_RUN: args.print_no_runs
+ }
for test_file in test_files:
self.tests.append(Selftest(test_file, args.executable,
@@ -30,7 +37,8 @@ class TestRunner:
def _log_result(self, test_result):
logger.log(test_result.status,
f"[{test_result.status}] {test_result.test_path}")
- if (self.output_dir is None and self.print_status is False):
+ if (self.output_dir is None and self.print_status is False
+ and self.print_stds.get(test_result.status, True)):
logger.info("************** STDOUT BEGIN **************")
logger.info(test_result.stdout)
logger.info("************** STDOUT END **************")
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 08/15] KVM: selftests: Print sticky KVM Selftests Runner status at bottom
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (6 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 07/15] KVM: selftests: Add various print flags to KVM Selftest Runner Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 09/15] KVM: selftests: Add a flag to print only sticky summary in the selftests runner Vipin Sharma
` (7 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Print current state of the KVM Selftest Runner during its execution.
Show it as the bottom most line, make it sticky and colored. Provide
the following information:
- Total number of tests selected for run.
- How many have executed.
- Total for each end state.
Example:
Total: 3/3 Passed: 1 Failed: 1 Skipped: 0 Timed Out: 0 No Run: 1
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
.../selftests/kvm/runner/test_runner.py | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
index 8f2372834104..e0da30d216a2 100644
--- a/tools/testing/selftests/kvm/runner/test_runner.py
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -15,6 +15,7 @@ logger = logging.getLogger("runner")
class TestRunner:
def __init__(self, test_files, args):
self.tests = []
+ self.status = {x: 0 for x in SelftestStatus}
self.output_dir = args.output
self.jobs = args.jobs
self.print_status = args.print_status
@@ -34,7 +35,17 @@ class TestRunner:
test.run()
return test
+ def _sticky_update(self):
+ print(f"\r\033[1mTotal: {self.tests_ran}/{len(self.tests)}" \
+ f"\033[32;1m Passed: {self.status[SelftestStatus.PASSED]}" \
+ f"\033[31;1m Failed: {self.status[SelftestStatus.FAILED]}" \
+ f"\033[33;1m Skipped: {self.status[SelftestStatus.SKIPPED]}"\
+ f"\033[91;1m Timed Out: {self.status[SelftestStatus.TIMED_OUT]}"\
+ f"\033[34;1m No Run: {self.status[SelftestStatus.NO_RUN]}\033[0m", end="\r")
+
def _log_result(self, test_result):
+ # Clear the status line
+ print("\033[2K", end="\r")
logger.log(test_result.status,
f"[{test_result.status}] {test_result.test_path}")
if (self.output_dir is None and self.print_status is False
@@ -46,8 +57,13 @@ class TestRunner:
logger.info(test_result.stderr)
logger.info("************** STDERR END **************")
+ self.status[test_result.status] += 1
+ # Sticky bottom line
+ self._sticky_update()
+
def start(self):
ret = 0
+ self.tests_ran = 0
with concurrent.futures.ProcessPoolExecutor(max_workers=self.jobs) as executor:
all_futures = []
@@ -57,9 +73,11 @@ class TestRunner:
for future in concurrent.futures.as_completed(all_futures):
test_result = future.result()
+ self.tests_ran += 1
self._log_result(test_result)
if (test_result.status not in [SelftestStatus.PASSED,
SelftestStatus.NO_RUN,
SelftestStatus.SKIPPED]):
ret = 1
+ print("\n")
return ret
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 09/15] KVM: selftests: Add a flag to print only sticky summary in the selftests runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (7 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 08/15] KVM: selftests: Print sticky KVM Selftests Runner status at bottom Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 10/15] KVM: selftests: Add flag to suppress all output from Selftest KVM Runner Vipin Sharma
` (6 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add a commandline flag, --sticky-summary-only, which only let sticky
summary print to the terminal and all other outputs are suppressed.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
tools/testing/selftests/kvm/runner/__main__.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
index 4406d8e4847a..2dcac1f4d1c4 100644
--- a/tools/testing/selftests/kvm/runner/__main__.py
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -124,6 +124,11 @@ def cli():
help="Print only tests which didn't run."
)
+ parser.add_argument("--sticky-summary-only",
+ action="store_true",
+ default=False,
+ help="Print only the summary status line.")
+
return parser.parse_args()
@@ -131,6 +136,9 @@ def level_filters(args):
# Levels added here will be printed by logger.
levels = set()
+ if args.sticky_summary_only:
+ return levels
+
if args.print_passed or args.print_passed_status or args.print_status:
levels.add(SelftestStatus.PASSED)
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 10/15] KVM: selftests: Add flag to suppress all output from Selftest KVM Runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (8 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 09/15] KVM: selftests: Add a flag to print only sticky summary in the selftests runner Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 11/15] KVM: selftests: Auto generate default tests for KVM Selftests Runner Vipin Sharma
` (5 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add a command line flag, --quiet, to suppress all of the output from
runner to terminal.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
.../testing/selftests/kvm/runner/__main__.py | 8 ++++++-
.../selftests/kvm/runner/test_runner.py | 21 ++++++++++++-------
2 files changed, 20 insertions(+), 9 deletions(-)
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
index 2dcac1f4d1c4..c02035a62873 100644
--- a/tools/testing/selftests/kvm/runner/__main__.py
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -129,6 +129,12 @@ def cli():
default=False,
help="Print only the summary status line.")
+ parser.add_argument("-q",
+ "--quiet",
+ action="store_true",
+ default=False,
+ help="Suppress all of the output to terminal")
+
return parser.parse_args()
@@ -136,7 +142,7 @@ def level_filters(args):
# Levels added here will be printed by logger.
levels = set()
- if args.sticky_summary_only:
+ if args.sticky_summary_only or args.quiet:
return levels
if args.print_passed or args.print_passed_status or args.print_status:
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
index e0da30d216a2..e7730880907d 100644
--- a/tools/testing/selftests/kvm/runner/test_runner.py
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -18,6 +18,7 @@ class TestRunner:
self.status = {x: 0 for x in SelftestStatus}
self.output_dir = args.output
self.jobs = args.jobs
+ self.quiet = args.quiet
self.print_status = args.print_status
self.print_stds = {
SelftestStatus.PASSED: args.print_passed,
@@ -35,17 +36,21 @@ class TestRunner:
test.run()
return test
+ def _print(self, text, end="\n"):
+ if not self.quiet:
+ print(text, end=end)
+
def _sticky_update(self):
- print(f"\r\033[1mTotal: {self.tests_ran}/{len(self.tests)}" \
- f"\033[32;1m Passed: {self.status[SelftestStatus.PASSED]}" \
- f"\033[31;1m Failed: {self.status[SelftestStatus.FAILED]}" \
- f"\033[33;1m Skipped: {self.status[SelftestStatus.SKIPPED]}"\
- f"\033[91;1m Timed Out: {self.status[SelftestStatus.TIMED_OUT]}"\
- f"\033[34;1m No Run: {self.status[SelftestStatus.NO_RUN]}\033[0m", end="\r")
+ self._print(f"\r\033[1mTotal: {self.tests_ran}/{len(self.tests)}"
+ f"\033[32;1m Passed: {self.status[SelftestStatus.PASSED]}"
+ f"\033[31;1m Failed: {self.status[SelftestStatus.FAILED]}"
+ f"\033[33;1m Skipped: {self.status[SelftestStatus.SKIPPED]}"
+ f"\033[91;1m Timed Out: {self.status[SelftestStatus.TIMED_OUT]}"
+ f"\033[34;1m No Run: {self.status[SelftestStatus.NO_RUN]}\033[0m", end="\r")
def _log_result(self, test_result):
# Clear the status line
- print("\033[2K", end="\r")
+ self._print("\033[2K", end="\r")
logger.log(test_result.status,
f"[{test_result.status}] {test_result.test_path}")
if (self.output_dir is None and self.print_status is False
@@ -79,5 +84,5 @@ class TestRunner:
SelftestStatus.NO_RUN,
SelftestStatus.SKIPPED]):
ret = 1
- print("\n")
+ self._print("")
return ret
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 11/15] KVM: selftests: Auto generate default tests for KVM Selftests Runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (9 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 10/15] KVM: selftests: Add flag to suppress all output from Selftest KVM Runner Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-07-09 23:06 ` Oliver Upton
2025-06-06 23:56 ` [PATCH v2 12/15] KVM: selftests: Add x86 auto generated test files " Vipin Sharma
` (4 subsequent siblings)
15 siblings, 1 reply; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add logic in Makefile.kvm to auto generate default test files for KVM
Selftests Runner. Preserve the hierarchy of test executables for
autogenerated files.
Autogeneration of default test files guards against missing these files
for new tests cases or rename of existing ones. These autogenerated
files will be checked in so that during git-diff one can easily identify
if the existing test files for the same test binary also needs an
update. It also add new tests automatically in the default coverage.
These files will be auto generated with each make invocation but
overhead should be very small as these are very small files and won't
be shown in git-diff unless test name changes.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
tools/testing/selftests/kvm/Makefile.kvm | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 307ef31d3557..a76502406123 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -201,6 +201,14 @@ TEST_GEN_PROGS += $(TEST_GEN_PROGS_$(ARCH))
TEST_GEN_PROGS_EXTENDED += $(TEST_GEN_PROGS_EXTENDED_$(ARCH))
LIBKVM += $(LIBKVM_$(ARCH))
+$(foreach tc, $(TEST_PROGS), $(shell mkdir -p tests/$(patsubst %.sh,%,$(tc))))
+$(foreach tc, $(TEST_PROGS), \
+ $(shell echo $(tc) > $(patsubst %.sh,tests/%/default.test,$(tc))))
+
+$(foreach tc, $(TEST_GEN_PROGS), $(shell mkdir -p tests/$(tc)))
+$(foreach tc, $(TEST_GEN_PROGS), \
+ $(shell echo $(tc) > $(patsubst %,tests/%/default.test,$(tc))))
+
OVERRIDE_TARGETS = 1
# lib.mak defines $(OUTPUT), prepends $(OUTPUT)/ to $(TEST_GEN_PROGS), and most
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 12/15] KVM: selftests: Add x86 auto generated test files for KVM Selftests Runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (10 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 11/15] KVM: selftests: Auto generate default tests for KVM Selftests Runner Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 13/15] KVM: selftests: Add arm64 " Vipin Sharma
` (3 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add auto generated test files for x86 platform.
These files will not show up in git-diff again unless a test is renamed.
In future patches more .test files will be added to run selftests with
their command line arguments.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
.../selftests/kvm/tests/access_tracking_perf_test/default.test | 1 +
tools/testing/selftests/kvm/tests/coalesced_io_test/default.test | 1 +
.../testing/selftests/kvm/tests/demand_paging_test/default.test | 1 +
tools/testing/selftests/kvm/tests/dirty_log_test/default.test | 1 +
tools/testing/selftests/kvm/tests/guest_memfd_test/default.test | 1 +
tools/testing/selftests/kvm/tests/guest_print_test/default.test | 1 +
.../selftests/kvm/tests/hardware_disable_test/default.test | 1 +
.../selftests/kvm/tests/kvm_binary_stats_test/default.test | 1 +
.../selftests/kvm/tests/kvm_create_max_vcpus/default.test | 1 +
.../testing/selftests/kvm/tests/kvm_page_table_test/default.test | 1 +
.../kvm/tests/memslot_modification_stress_test/default.test | 1 +
tools/testing/selftests/kvm/tests/memslot_perf_test/default.test | 1 +
tools/testing/selftests/kvm/tests/mmu_stress_test/default.test | 1 +
.../selftests/kvm/tests/pre_fault_memory_test/default.test | 1 +
tools/testing/selftests/kvm/tests/rseq_test/default.test | 1 +
.../selftests/kvm/tests/set_memory_region_test/default.test | 1 +
tools/testing/selftests/kvm/tests/steal_time/default.test | 1 +
.../selftests/kvm/tests/system_counter_offset_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/amx_test/default.test | 1 +
.../selftests/kvm/tests/x86/apic_bus_clock_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/cpuid_test/default.test | 1 +
.../selftests/kvm/tests/x86/cr4_cpuid_sync_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/debug_regs/default.test | 1 +
.../kvm/tests/x86/dirty_log_page_splitting_test/default.test | 1 +
.../kvm/tests/x86/exit_on_emulation_failure_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/fastops_test/default.test | 1 +
.../selftests/kvm/tests/x86/feature_msrs_test/default.test | 1 +
.../selftests/kvm/tests/x86/fix_hypercall_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/hwcr_msr_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/hyperv_clock/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/hyperv_cpuid/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/hyperv_evmcs/default.test | 1 +
.../kvm/tests/x86/hyperv_extended_hypercalls/default.test | 1 +
.../testing/selftests/kvm/tests/x86/hyperv_features/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/hyperv_ipi/default.test | 1 +
.../testing/selftests/kvm/tests/x86/hyperv_svm_test/default.test | 1 +
.../selftests/kvm/tests/x86/hyperv_tlb_flush/default.test | 1 +
.../selftests/kvm/tests/x86/kvm_buslock_test/default.test | 1 +
.../testing/selftests/kvm/tests/x86/kvm_clock_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/kvm_pv_test/default.test | 1 +
.../selftests/kvm/tests/x86/max_vcpuid_cap_test/default.test | 1 +
.../selftests/kvm/tests/x86/monitor_mwait_test/default.test | 1 +
.../selftests/kvm/tests/x86/nested_emulation_test/default.test | 1 +
.../selftests/kvm/tests/x86/nested_exceptions_test/default.test | 1 +
.../selftests/kvm/tests/x86/nx_huge_pages_test/default.test | 1 +
.../selftests/kvm/tests/x86/platform_info_test/default.test | 1 +
.../selftests/kvm/tests/x86/pmu_counters_test/default.test | 1 +
.../selftests/kvm/tests/x86/pmu_event_filter_test/default.test | 1 +
.../kvm/tests/x86/private_mem_conversions_test/default.test | 1 +
.../kvm/tests/x86/private_mem_kvm_exits_test/default.test | 1 +
.../selftests/kvm/tests/x86/recalc_apic_map_test/default.test | 1 +
.../testing/selftests/kvm/tests/x86/set_boot_cpu_id/default.test | 1 +
.../testing/selftests/kvm/tests/x86/set_sregs_test/default.test | 1 +
.../testing/selftests/kvm/tests/x86/sev_init2_tests/default.test | 1 +
.../selftests/kvm/tests/x86/sev_migrate_tests/default.test | 1 +
.../testing/selftests/kvm/tests/x86/sev_smoke_test/default.test | 1 +
.../kvm/tests/x86/smaller_maxphyaddr_emulation_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/smm_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/state_test/default.test | 1 +
.../selftests/kvm/tests/x86/svm_int_ctl_test/default.test | 1 +
.../kvm/tests/x86/svm_nested_shutdown_test/default.test | 1 +
.../kvm/tests/x86/svm_nested_soft_inject_test/default.test | 1 +
.../testing/selftests/kvm/tests/x86/svm_vmcall_test/default.test | 1 +
.../testing/selftests/kvm/tests/x86/sync_regs_test/default.test | 1 +
.../selftests/kvm/tests/x86/triple_fault_event_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/tsc_msrs_test/default.test | 1 +
.../selftests/kvm/tests/x86/tsc_scaling_sync/default.test | 1 +
.../selftests/kvm/tests/x86/ucna_injection_test/default.test | 1 +
.../selftests/kvm/tests/x86/userspace_io_test/default.test | 1 +
.../selftests/kvm/tests/x86/userspace_msr_exit_test/default.test | 1 +
.../selftests/kvm/tests/x86/vmx_apic_access_test/default.test | 1 +
.../kvm/tests/x86/vmx_close_while_nested_test/default.test | 1 +
.../selftests/kvm/tests/x86/vmx_dirty_log_test/default.test | 1 +
.../x86/vmx_exception_with_invalid_guest_state/default.test | 1 +
.../kvm/tests/x86/vmx_invalid_nested_guest_state/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/vmx_msrs_test/default.test | 1 +
.../kvm/tests/x86/vmx_nested_tsc_scaling_test/default.test | 1 +
.../selftests/kvm/tests/x86/vmx_pmu_caps_test/default.test | 1 +
.../kvm/tests/x86/vmx_preemption_timer_test/default.test | 1 +
.../kvm/tests/x86/vmx_set_nested_state_test/default.test | 1 +
.../selftests/kvm/tests/x86/vmx_tsc_adjust_test/default.test | 1 +
.../testing/selftests/kvm/tests/x86/xapic_ipi_test/default.test | 1 +
.../selftests/kvm/tests/x86/xapic_state_test/default.test | 1 +
.../testing/selftests/kvm/tests/x86/xcr0_cpuid_test/default.test | 1 +
.../testing/selftests/kvm/tests/x86/xen_shinfo_test/default.test | 1 +
.../testing/selftests/kvm/tests/x86/xen_vmcall_test/default.test | 1 +
tools/testing/selftests/kvm/tests/x86/xss_msr_test/default.test | 1 +
87 files changed, 87 insertions(+)
create mode 100644 tools/testing/selftests/kvm/tests/access_tracking_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/coalesced_io_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/demand_paging_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/dirty_log_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/guest_memfd_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/guest_print_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/hardware_disable_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/kvm_binary_stats_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/kvm_create_max_vcpus/default.test
create mode 100644 tools/testing/selftests/kvm/tests/kvm_page_table_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/memslot_modification_stress_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/memslot_perf_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/mmu_stress_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/pre_fault_memory_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/rseq_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/set_memory_region_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/steal_time/default.test
create mode 100644 tools/testing/selftests/kvm/tests/system_counter_offset_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/amx_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/apic_bus_clock_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/cpuid_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/cr4_cpuid_sync_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/debug_regs/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/dirty_log_page_splitting_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/exit_on_emulation_failure_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/fastops_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/feature_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/fix_hypercall_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hwcr_msr_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_clock/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_cpuid/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_evmcs/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_extended_hypercalls/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_features/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_ipi/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_svm_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/hyperv_tlb_flush/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/kvm_buslock_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/kvm_clock_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/kvm_pv_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/max_vcpuid_cap_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/monitor_mwait_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/nested_emulation_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/nested_exceptions_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/nx_huge_pages_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/platform_info_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/pmu_counters_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/pmu_event_filter_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/private_mem_conversions_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/private_mem_kvm_exits_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/recalc_apic_map_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/set_boot_cpu_id/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/set_sregs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/sev_init2_tests/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/sev_migrate_tests/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/sev_smoke_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/smaller_maxphyaddr_emulation_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/smm_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/state_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/svm_int_ctl_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/svm_nested_shutdown_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/svm_nested_soft_inject_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/svm_vmcall_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/sync_regs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/triple_fault_event_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/tsc_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/tsc_scaling_sync/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/ucna_injection_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/userspace_io_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/userspace_msr_exit_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_apic_access_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_close_while_nested_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_dirty_log_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_exception_with_invalid_guest_state/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_invalid_nested_guest_state/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_msrs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_nested_tsc_scaling_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_pmu_caps_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_preemption_timer_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_set_nested_state_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/vmx_tsc_adjust_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xapic_ipi_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xapic_state_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xcr0_cpuid_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xen_shinfo_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xen_vmcall_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/x86/xss_msr_test/default.test
diff --git a/tools/testing/selftests/kvm/tests/access_tracking_perf_test/default.test b/tools/testing/selftests/kvm/tests/access_tracking_perf_test/default.test
new file mode 100644
index 000000000000..93e0f5d35525
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/access_tracking_perf_test/default.test
@@ -0,0 +1 @@
+access_tracking_perf_test
diff --git a/tools/testing/selftests/kvm/tests/coalesced_io_test/default.test b/tools/testing/selftests/kvm/tests/coalesced_io_test/default.test
new file mode 100644
index 000000000000..9c3e8fb05142
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/coalesced_io_test/default.test
@@ -0,0 +1 @@
+coalesced_io_test
diff --git a/tools/testing/selftests/kvm/tests/demand_paging_test/default.test b/tools/testing/selftests/kvm/tests/demand_paging_test/default.test
new file mode 100644
index 000000000000..9ce2eaf620e1
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/demand_paging_test/default.test
@@ -0,0 +1 @@
+demand_paging_test
diff --git a/tools/testing/selftests/kvm/tests/dirty_log_test/default.test b/tools/testing/selftests/kvm/tests/dirty_log_test/default.test
new file mode 100644
index 000000000000..56a07cffab15
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/dirty_log_test/default.test
@@ -0,0 +1 @@
+dirty_log_test
diff --git a/tools/testing/selftests/kvm/tests/guest_memfd_test/default.test b/tools/testing/selftests/kvm/tests/guest_memfd_test/default.test
new file mode 100644
index 000000000000..1e68140a8a4b
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/guest_memfd_test/default.test
@@ -0,0 +1 @@
+guest_memfd_test
diff --git a/tools/testing/selftests/kvm/tests/guest_print_test/default.test b/tools/testing/selftests/kvm/tests/guest_print_test/default.test
new file mode 100644
index 000000000000..9f83cb6e379d
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/guest_print_test/default.test
@@ -0,0 +1 @@
+guest_print_test
diff --git a/tools/testing/selftests/kvm/tests/hardware_disable_test/default.test b/tools/testing/selftests/kvm/tests/hardware_disable_test/default.test
new file mode 100644
index 000000000000..bd0aae00519b
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/hardware_disable_test/default.test
@@ -0,0 +1 @@
+hardware_disable_test
diff --git a/tools/testing/selftests/kvm/tests/kvm_binary_stats_test/default.test b/tools/testing/selftests/kvm/tests/kvm_binary_stats_test/default.test
new file mode 100644
index 000000000000..fd0f99fe7b94
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/kvm_binary_stats_test/default.test
@@ -0,0 +1 @@
+kvm_binary_stats_test
diff --git a/tools/testing/selftests/kvm/tests/kvm_create_max_vcpus/default.test b/tools/testing/selftests/kvm/tests/kvm_create_max_vcpus/default.test
new file mode 100644
index 000000000000..68e5174fb84d
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/kvm_create_max_vcpus/default.test
@@ -0,0 +1 @@
+kvm_create_max_vcpus
diff --git a/tools/testing/selftests/kvm/tests/kvm_page_table_test/default.test b/tools/testing/selftests/kvm/tests/kvm_page_table_test/default.test
new file mode 100644
index 000000000000..4988836740b7
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/kvm_page_table_test/default.test
@@ -0,0 +1 @@
+kvm_page_table_test
diff --git a/tools/testing/selftests/kvm/tests/memslot_modification_stress_test/default.test b/tools/testing/selftests/kvm/tests/memslot_modification_stress_test/default.test
new file mode 100644
index 000000000000..ccb1d45ab009
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/memslot_modification_stress_test/default.test
@@ -0,0 +1 @@
+memslot_modification_stress_test
diff --git a/tools/testing/selftests/kvm/tests/memslot_perf_test/default.test b/tools/testing/selftests/kvm/tests/memslot_perf_test/default.test
new file mode 100644
index 000000000000..2de1264b631b
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/memslot_perf_test/default.test
@@ -0,0 +1 @@
+memslot_perf_test
diff --git a/tools/testing/selftests/kvm/tests/mmu_stress_test/default.test b/tools/testing/selftests/kvm/tests/mmu_stress_test/default.test
new file mode 100644
index 000000000000..0e1ed1e3b971
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/mmu_stress_test/default.test
@@ -0,0 +1 @@
+mmu_stress_test
diff --git a/tools/testing/selftests/kvm/tests/pre_fault_memory_test/default.test b/tools/testing/selftests/kvm/tests/pre_fault_memory_test/default.test
new file mode 100644
index 000000000000..4e352e629be5
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/pre_fault_memory_test/default.test
@@ -0,0 +1 @@
+pre_fault_memory_test
diff --git a/tools/testing/selftests/kvm/tests/rseq_test/default.test b/tools/testing/selftests/kvm/tests/rseq_test/default.test
new file mode 100644
index 000000000000..fabc9ae64128
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/rseq_test/default.test
@@ -0,0 +1 @@
+rseq_test
diff --git a/tools/testing/selftests/kvm/tests/set_memory_region_test/default.test b/tools/testing/selftests/kvm/tests/set_memory_region_test/default.test
new file mode 100644
index 000000000000..c3f91720e105
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/set_memory_region_test/default.test
@@ -0,0 +1 @@
+set_memory_region_test
diff --git a/tools/testing/selftests/kvm/tests/steal_time/default.test b/tools/testing/selftests/kvm/tests/steal_time/default.test
new file mode 100644
index 000000000000..5f3e20693d58
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/steal_time/default.test
@@ -0,0 +1 @@
+steal_time
diff --git a/tools/testing/selftests/kvm/tests/system_counter_offset_test/default.test b/tools/testing/selftests/kvm/tests/system_counter_offset_test/default.test
new file mode 100644
index 000000000000..4d76b9f44147
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/system_counter_offset_test/default.test
@@ -0,0 +1 @@
+system_counter_offset_test
diff --git a/tools/testing/selftests/kvm/tests/x86/amx_test/default.test b/tools/testing/selftests/kvm/tests/x86/amx_test/default.test
new file mode 100644
index 000000000000..423ce501efee
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/amx_test/default.test
@@ -0,0 +1 @@
+x86/amx_test
diff --git a/tools/testing/selftests/kvm/tests/x86/apic_bus_clock_test/default.test b/tools/testing/selftests/kvm/tests/x86/apic_bus_clock_test/default.test
new file mode 100644
index 000000000000..e985ff8520f5
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/apic_bus_clock_test/default.test
@@ -0,0 +1 @@
+x86/apic_bus_clock_test
diff --git a/tools/testing/selftests/kvm/tests/x86/cpuid_test/default.test b/tools/testing/selftests/kvm/tests/x86/cpuid_test/default.test
new file mode 100644
index 000000000000..d2b2a48b25af
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/cpuid_test/default.test
@@ -0,0 +1 @@
+x86/cpuid_test
diff --git a/tools/testing/selftests/kvm/tests/x86/cr4_cpuid_sync_test/default.test b/tools/testing/selftests/kvm/tests/x86/cr4_cpuid_sync_test/default.test
new file mode 100644
index 000000000000..0d8e182cd2ca
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/cr4_cpuid_sync_test/default.test
@@ -0,0 +1 @@
+x86/cr4_cpuid_sync_test
diff --git a/tools/testing/selftests/kvm/tests/x86/debug_regs/default.test b/tools/testing/selftests/kvm/tests/x86/debug_regs/default.test
new file mode 100644
index 000000000000..440eeb9ab33d
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/debug_regs/default.test
@@ -0,0 +1 @@
+x86/debug_regs
diff --git a/tools/testing/selftests/kvm/tests/x86/dirty_log_page_splitting_test/default.test b/tools/testing/selftests/kvm/tests/x86/dirty_log_page_splitting_test/default.test
new file mode 100644
index 000000000000..71e98eb3103c
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/dirty_log_page_splitting_test/default.test
@@ -0,0 +1 @@
+x86/dirty_log_page_splitting_test
diff --git a/tools/testing/selftests/kvm/tests/x86/exit_on_emulation_failure_test/default.test b/tools/testing/selftests/kvm/tests/x86/exit_on_emulation_failure_test/default.test
new file mode 100644
index 000000000000..351242070720
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/exit_on_emulation_failure_test/default.test
@@ -0,0 +1 @@
+x86/exit_on_emulation_failure_test
diff --git a/tools/testing/selftests/kvm/tests/x86/fastops_test/default.test b/tools/testing/selftests/kvm/tests/x86/fastops_test/default.test
new file mode 100644
index 000000000000..d51c860be0d9
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/fastops_test/default.test
@@ -0,0 +1 @@
+x86/fastops_test
diff --git a/tools/testing/selftests/kvm/tests/x86/feature_msrs_test/default.test b/tools/testing/selftests/kvm/tests/x86/feature_msrs_test/default.test
new file mode 100644
index 000000000000..57d867b6b94c
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/feature_msrs_test/default.test
@@ -0,0 +1 @@
+x86/feature_msrs_test
diff --git a/tools/testing/selftests/kvm/tests/x86/fix_hypercall_test/default.test b/tools/testing/selftests/kvm/tests/x86/fix_hypercall_test/default.test
new file mode 100644
index 000000000000..f95e114d8241
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/fix_hypercall_test/default.test
@@ -0,0 +1 @@
+x86/fix_hypercall_test
diff --git a/tools/testing/selftests/kvm/tests/x86/hwcr_msr_test/default.test b/tools/testing/selftests/kvm/tests/x86/hwcr_msr_test/default.test
new file mode 100644
index 000000000000..4a957c09db6b
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/hwcr_msr_test/default.test
@@ -0,0 +1 @@
+x86/hwcr_msr_test
diff --git a/tools/testing/selftests/kvm/tests/x86/hyperv_clock/default.test b/tools/testing/selftests/kvm/tests/x86/hyperv_clock/default.test
new file mode 100644
index 000000000000..a3ff8a689773
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/hyperv_clock/default.test
@@ -0,0 +1 @@
+x86/hyperv_clock
diff --git a/tools/testing/selftests/kvm/tests/x86/hyperv_cpuid/default.test b/tools/testing/selftests/kvm/tests/x86/hyperv_cpuid/default.test
new file mode 100644
index 000000000000..66b0d85430eb
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/hyperv_cpuid/default.test
@@ -0,0 +1 @@
+x86/hyperv_cpuid
diff --git a/tools/testing/selftests/kvm/tests/x86/hyperv_evmcs/default.test b/tools/testing/selftests/kvm/tests/x86/hyperv_evmcs/default.test
new file mode 100644
index 000000000000..aa34233db0c1
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/hyperv_evmcs/default.test
@@ -0,0 +1 @@
+x86/hyperv_evmcs
diff --git a/tools/testing/selftests/kvm/tests/x86/hyperv_extended_hypercalls/default.test b/tools/testing/selftests/kvm/tests/x86/hyperv_extended_hypercalls/default.test
new file mode 100644
index 000000000000..4fdfc3714d54
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/hyperv_extended_hypercalls/default.test
@@ -0,0 +1 @@
+x86/hyperv_extended_hypercalls
diff --git a/tools/testing/selftests/kvm/tests/x86/hyperv_features/default.test b/tools/testing/selftests/kvm/tests/x86/hyperv_features/default.test
new file mode 100644
index 000000000000..dc486adbe756
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/hyperv_features/default.test
@@ -0,0 +1 @@
+x86/hyperv_features
diff --git a/tools/testing/selftests/kvm/tests/x86/hyperv_ipi/default.test b/tools/testing/selftests/kvm/tests/x86/hyperv_ipi/default.test
new file mode 100644
index 000000000000..b21cc166efdb
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/hyperv_ipi/default.test
@@ -0,0 +1 @@
+x86/hyperv_ipi
diff --git a/tools/testing/selftests/kvm/tests/x86/hyperv_svm_test/default.test b/tools/testing/selftests/kvm/tests/x86/hyperv_svm_test/default.test
new file mode 100644
index 000000000000..0e973fbf7e5a
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/hyperv_svm_test/default.test
@@ -0,0 +1 @@
+x86/hyperv_svm_test
diff --git a/tools/testing/selftests/kvm/tests/x86/hyperv_tlb_flush/default.test b/tools/testing/selftests/kvm/tests/x86/hyperv_tlb_flush/default.test
new file mode 100644
index 000000000000..bcbd02d50d68
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/hyperv_tlb_flush/default.test
@@ -0,0 +1 @@
+x86/hyperv_tlb_flush
diff --git a/tools/testing/selftests/kvm/tests/x86/kvm_buslock_test/default.test b/tools/testing/selftests/kvm/tests/x86/kvm_buslock_test/default.test
new file mode 100644
index 000000000000..b9e46fc3ecb8
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/kvm_buslock_test/default.test
@@ -0,0 +1 @@
+x86/kvm_buslock_test
diff --git a/tools/testing/selftests/kvm/tests/x86/kvm_clock_test/default.test b/tools/testing/selftests/kvm/tests/x86/kvm_clock_test/default.test
new file mode 100644
index 000000000000..434a1c13d9e6
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/kvm_clock_test/default.test
@@ -0,0 +1 @@
+x86/kvm_clock_test
diff --git a/tools/testing/selftests/kvm/tests/x86/kvm_pv_test/default.test b/tools/testing/selftests/kvm/tests/x86/kvm_pv_test/default.test
new file mode 100644
index 000000000000..c785934d98a8
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/kvm_pv_test/default.test
@@ -0,0 +1 @@
+x86/kvm_pv_test
diff --git a/tools/testing/selftests/kvm/tests/x86/max_vcpuid_cap_test/default.test b/tools/testing/selftests/kvm/tests/x86/max_vcpuid_cap_test/default.test
new file mode 100644
index 000000000000..3b36b5dcbbb3
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/max_vcpuid_cap_test/default.test
@@ -0,0 +1 @@
+x86/max_vcpuid_cap_test
diff --git a/tools/testing/selftests/kvm/tests/x86/monitor_mwait_test/default.test b/tools/testing/selftests/kvm/tests/x86/monitor_mwait_test/default.test
new file mode 100644
index 000000000000..ec31250b319a
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/monitor_mwait_test/default.test
@@ -0,0 +1 @@
+x86/monitor_mwait_test
diff --git a/tools/testing/selftests/kvm/tests/x86/nested_emulation_test/default.test b/tools/testing/selftests/kvm/tests/x86/nested_emulation_test/default.test
new file mode 100644
index 000000000000..68262f28e738
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/nested_emulation_test/default.test
@@ -0,0 +1 @@
+x86/nested_emulation_test
diff --git a/tools/testing/selftests/kvm/tests/x86/nested_exceptions_test/default.test b/tools/testing/selftests/kvm/tests/x86/nested_exceptions_test/default.test
new file mode 100644
index 000000000000..1361448740dd
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/nested_exceptions_test/default.test
@@ -0,0 +1 @@
+x86/nested_exceptions_test
diff --git a/tools/testing/selftests/kvm/tests/x86/nx_huge_pages_test/default.test b/tools/testing/selftests/kvm/tests/x86/nx_huge_pages_test/default.test
new file mode 100644
index 000000000000..5229d1f85e2f
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/nx_huge_pages_test/default.test
@@ -0,0 +1 @@
+x86/nx_huge_pages_test.sh
diff --git a/tools/testing/selftests/kvm/tests/x86/platform_info_test/default.test b/tools/testing/selftests/kvm/tests/x86/platform_info_test/default.test
new file mode 100644
index 000000000000..2fe74d5d32bd
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/platform_info_test/default.test
@@ -0,0 +1 @@
+x86/platform_info_test
diff --git a/tools/testing/selftests/kvm/tests/x86/pmu_counters_test/default.test b/tools/testing/selftests/kvm/tests/x86/pmu_counters_test/default.test
new file mode 100644
index 000000000000..c94201dbdfb3
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/pmu_counters_test/default.test
@@ -0,0 +1 @@
+x86/pmu_counters_test
diff --git a/tools/testing/selftests/kvm/tests/x86/pmu_event_filter_test/default.test b/tools/testing/selftests/kvm/tests/x86/pmu_event_filter_test/default.test
new file mode 100644
index 000000000000..c64ec1b8fba4
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/pmu_event_filter_test/default.test
@@ -0,0 +1 @@
+x86/pmu_event_filter_test
diff --git a/tools/testing/selftests/kvm/tests/x86/private_mem_conversions_test/default.test b/tools/testing/selftests/kvm/tests/x86/private_mem_conversions_test/default.test
new file mode 100644
index 000000000000..933293725304
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/private_mem_conversions_test/default.test
@@ -0,0 +1 @@
+x86/private_mem_conversions_test
diff --git a/tools/testing/selftests/kvm/tests/x86/private_mem_kvm_exits_test/default.test b/tools/testing/selftests/kvm/tests/x86/private_mem_kvm_exits_test/default.test
new file mode 100644
index 000000000000..a2dd9876cb54
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/private_mem_kvm_exits_test/default.test
@@ -0,0 +1 @@
+x86/private_mem_kvm_exits_test
diff --git a/tools/testing/selftests/kvm/tests/x86/recalc_apic_map_test/default.test b/tools/testing/selftests/kvm/tests/x86/recalc_apic_map_test/default.test
new file mode 100644
index 000000000000..bc01f48e2c3e
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/recalc_apic_map_test/default.test
@@ -0,0 +1 @@
+x86/recalc_apic_map_test
diff --git a/tools/testing/selftests/kvm/tests/x86/set_boot_cpu_id/default.test b/tools/testing/selftests/kvm/tests/x86/set_boot_cpu_id/default.test
new file mode 100644
index 000000000000..79ff27988952
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/set_boot_cpu_id/default.test
@@ -0,0 +1 @@
+x86/set_boot_cpu_id
diff --git a/tools/testing/selftests/kvm/tests/x86/set_sregs_test/default.test b/tools/testing/selftests/kvm/tests/x86/set_sregs_test/default.test
new file mode 100644
index 000000000000..fc02c20dda19
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/set_sregs_test/default.test
@@ -0,0 +1 @@
+x86/set_sregs_test
diff --git a/tools/testing/selftests/kvm/tests/x86/sev_init2_tests/default.test b/tools/testing/selftests/kvm/tests/x86/sev_init2_tests/default.test
new file mode 100644
index 000000000000..9d839dba35c1
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/sev_init2_tests/default.test
@@ -0,0 +1 @@
+x86/sev_init2_tests
diff --git a/tools/testing/selftests/kvm/tests/x86/sev_migrate_tests/default.test b/tools/testing/selftests/kvm/tests/x86/sev_migrate_tests/default.test
new file mode 100644
index 000000000000..f0579e499629
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/sev_migrate_tests/default.test
@@ -0,0 +1 @@
+x86/sev_migrate_tests
diff --git a/tools/testing/selftests/kvm/tests/x86/sev_smoke_test/default.test b/tools/testing/selftests/kvm/tests/x86/sev_smoke_test/default.test
new file mode 100644
index 000000000000..2550f6eff2d6
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/sev_smoke_test/default.test
@@ -0,0 +1 @@
+x86/sev_smoke_test
diff --git a/tools/testing/selftests/kvm/tests/x86/smaller_maxphyaddr_emulation_test/default.test b/tools/testing/selftests/kvm/tests/x86/smaller_maxphyaddr_emulation_test/default.test
new file mode 100644
index 000000000000..1382d34b4863
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/smaller_maxphyaddr_emulation_test/default.test
@@ -0,0 +1 @@
+x86/smaller_maxphyaddr_emulation_test
diff --git a/tools/testing/selftests/kvm/tests/x86/smm_test/default.test b/tools/testing/selftests/kvm/tests/x86/smm_test/default.test
new file mode 100644
index 000000000000..4437978a3e77
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/smm_test/default.test
@@ -0,0 +1 @@
+x86/smm_test
diff --git a/tools/testing/selftests/kvm/tests/x86/state_test/default.test b/tools/testing/selftests/kvm/tests/x86/state_test/default.test
new file mode 100644
index 000000000000..9f222c727d7f
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/state_test/default.test
@@ -0,0 +1 @@
+x86/state_test
diff --git a/tools/testing/selftests/kvm/tests/x86/svm_int_ctl_test/default.test b/tools/testing/selftests/kvm/tests/x86/svm_int_ctl_test/default.test
new file mode 100644
index 000000000000..51fe44a36633
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/svm_int_ctl_test/default.test
@@ -0,0 +1 @@
+x86/svm_int_ctl_test
diff --git a/tools/testing/selftests/kvm/tests/x86/svm_nested_shutdown_test/default.test b/tools/testing/selftests/kvm/tests/x86/svm_nested_shutdown_test/default.test
new file mode 100644
index 000000000000..642895b3cb5b
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/svm_nested_shutdown_test/default.test
@@ -0,0 +1 @@
+x86/svm_nested_shutdown_test
diff --git a/tools/testing/selftests/kvm/tests/x86/svm_nested_soft_inject_test/default.test b/tools/testing/selftests/kvm/tests/x86/svm_nested_soft_inject_test/default.test
new file mode 100644
index 000000000000..6947cbcb1da5
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/svm_nested_soft_inject_test/default.test
@@ -0,0 +1 @@
+x86/svm_nested_soft_inject_test
diff --git a/tools/testing/selftests/kvm/tests/x86/svm_vmcall_test/default.test b/tools/testing/selftests/kvm/tests/x86/svm_vmcall_test/default.test
new file mode 100644
index 000000000000..68f22e39b1d6
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/svm_vmcall_test/default.test
@@ -0,0 +1 @@
+x86/svm_vmcall_test
diff --git a/tools/testing/selftests/kvm/tests/x86/sync_regs_test/default.test b/tools/testing/selftests/kvm/tests/x86/sync_regs_test/default.test
new file mode 100644
index 000000000000..1c88e7df625a
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/sync_regs_test/default.test
@@ -0,0 +1 @@
+x86/sync_regs_test
diff --git a/tools/testing/selftests/kvm/tests/x86/triple_fault_event_test/default.test b/tools/testing/selftests/kvm/tests/x86/triple_fault_event_test/default.test
new file mode 100644
index 000000000000..868d1d396abd
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/triple_fault_event_test/default.test
@@ -0,0 +1 @@
+x86/triple_fault_event_test
diff --git a/tools/testing/selftests/kvm/tests/x86/tsc_msrs_test/default.test b/tools/testing/selftests/kvm/tests/x86/tsc_msrs_test/default.test
new file mode 100644
index 000000000000..b9e8ce298fe8
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/tsc_msrs_test/default.test
@@ -0,0 +1 @@
+x86/tsc_msrs_test
diff --git a/tools/testing/selftests/kvm/tests/x86/tsc_scaling_sync/default.test b/tools/testing/selftests/kvm/tests/x86/tsc_scaling_sync/default.test
new file mode 100644
index 000000000000..92dbcfdab5d0
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/tsc_scaling_sync/default.test
@@ -0,0 +1 @@
+x86/tsc_scaling_sync
diff --git a/tools/testing/selftests/kvm/tests/x86/ucna_injection_test/default.test b/tools/testing/selftests/kvm/tests/x86/ucna_injection_test/default.test
new file mode 100644
index 000000000000..135561c81334
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/ucna_injection_test/default.test
@@ -0,0 +1 @@
+x86/ucna_injection_test
diff --git a/tools/testing/selftests/kvm/tests/x86/userspace_io_test/default.test b/tools/testing/selftests/kvm/tests/x86/userspace_io_test/default.test
new file mode 100644
index 000000000000..9584eb38ca61
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/userspace_io_test/default.test
@@ -0,0 +1 @@
+x86/userspace_io_test
diff --git a/tools/testing/selftests/kvm/tests/x86/userspace_msr_exit_test/default.test b/tools/testing/selftests/kvm/tests/x86/userspace_msr_exit_test/default.test
new file mode 100644
index 000000000000..a2bbdd1a2469
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/userspace_msr_exit_test/default.test
@@ -0,0 +1 @@
+x86/userspace_msr_exit_test
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_apic_access_test/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_apic_access_test/default.test
new file mode 100644
index 000000000000..561e8b72a074
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_apic_access_test/default.test
@@ -0,0 +1 @@
+x86/vmx_apic_access_test
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_close_while_nested_test/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_close_while_nested_test/default.test
new file mode 100644
index 000000000000..fca3eaff03d4
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_close_while_nested_test/default.test
@@ -0,0 +1 @@
+x86/vmx_close_while_nested_test
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_dirty_log_test/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_dirty_log_test/default.test
new file mode 100644
index 000000000000..0332d8fbfec4
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_dirty_log_test/default.test
@@ -0,0 +1 @@
+x86/vmx_dirty_log_test
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_exception_with_invalid_guest_state/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_exception_with_invalid_guest_state/default.test
new file mode 100644
index 000000000000..8d89b505f680
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_exception_with_invalid_guest_state/default.test
@@ -0,0 +1 @@
+x86/vmx_exception_with_invalid_guest_state
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_invalid_nested_guest_state/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_invalid_nested_guest_state/default.test
new file mode 100644
index 000000000000..df2df912448b
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_invalid_nested_guest_state/default.test
@@ -0,0 +1 @@
+x86/vmx_invalid_nested_guest_state
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_msrs_test/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_msrs_test/default.test
new file mode 100644
index 000000000000..677b1bdda6af
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_msrs_test/default.test
@@ -0,0 +1 @@
+x86/vmx_msrs_test
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_nested_tsc_scaling_test/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_nested_tsc_scaling_test/default.test
new file mode 100644
index 000000000000..c8400ea69265
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_nested_tsc_scaling_test/default.test
@@ -0,0 +1 @@
+x86/vmx_nested_tsc_scaling_test
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_pmu_caps_test/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_pmu_caps_test/default.test
new file mode 100644
index 000000000000..a641daa14081
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_pmu_caps_test/default.test
@@ -0,0 +1 @@
+x86/vmx_pmu_caps_test
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_preemption_timer_test/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_preemption_timer_test/default.test
new file mode 100644
index 000000000000..aba7235296f4
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_preemption_timer_test/default.test
@@ -0,0 +1 @@
+x86/vmx_preemption_timer_test
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_set_nested_state_test/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_set_nested_state_test/default.test
new file mode 100644
index 000000000000..4ee968527e4a
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_set_nested_state_test/default.test
@@ -0,0 +1 @@
+x86/vmx_set_nested_state_test
diff --git a/tools/testing/selftests/kvm/tests/x86/vmx_tsc_adjust_test/default.test b/tools/testing/selftests/kvm/tests/x86/vmx_tsc_adjust_test/default.test
new file mode 100644
index 000000000000..10c93b9145e7
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/vmx_tsc_adjust_test/default.test
@@ -0,0 +1 @@
+x86/vmx_tsc_adjust_test
diff --git a/tools/testing/selftests/kvm/tests/x86/xapic_ipi_test/default.test b/tools/testing/selftests/kvm/tests/x86/xapic_ipi_test/default.test
new file mode 100644
index 000000000000..e1b4250bc93b
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/xapic_ipi_test/default.test
@@ -0,0 +1 @@
+x86/xapic_ipi_test
diff --git a/tools/testing/selftests/kvm/tests/x86/xapic_state_test/default.test b/tools/testing/selftests/kvm/tests/x86/xapic_state_test/default.test
new file mode 100644
index 000000000000..e700e34638b1
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/xapic_state_test/default.test
@@ -0,0 +1 @@
+x86/xapic_state_test
diff --git a/tools/testing/selftests/kvm/tests/x86/xcr0_cpuid_test/default.test b/tools/testing/selftests/kvm/tests/x86/xcr0_cpuid_test/default.test
new file mode 100644
index 000000000000..fd8909f37682
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/xcr0_cpuid_test/default.test
@@ -0,0 +1 @@
+x86/xcr0_cpuid_test
diff --git a/tools/testing/selftests/kvm/tests/x86/xen_shinfo_test/default.test b/tools/testing/selftests/kvm/tests/x86/xen_shinfo_test/default.test
new file mode 100644
index 000000000000..b948bbaed045
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/xen_shinfo_test/default.test
@@ -0,0 +1 @@
+x86/xen_shinfo_test
diff --git a/tools/testing/selftests/kvm/tests/x86/xen_vmcall_test/default.test b/tools/testing/selftests/kvm/tests/x86/xen_vmcall_test/default.test
new file mode 100644
index 000000000000..63b943d0544c
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/xen_vmcall_test/default.test
@@ -0,0 +1 @@
+x86/xen_vmcall_test
diff --git a/tools/testing/selftests/kvm/tests/x86/xss_msr_test/default.test b/tools/testing/selftests/kvm/tests/x86/xss_msr_test/default.test
new file mode 100644
index 000000000000..f8322c6d0117
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/x86/xss_msr_test/default.test
@@ -0,0 +1 @@
+x86/xss_msr_test
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 13/15] KVM: selftests: Add arm64 auto generated test files for KVM Selftests Runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (11 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 12/15] KVM: selftests: Add x86 auto generated test files " Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 14/15] KVM: selftests: Add s390 " Vipin Sharma
` (2 subsequent siblings)
15 siblings, 0 replies; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add auto generated test files for ARM platform.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
tools/testing/selftests/kvm/tests/arch_timer/default.test | 1 +
.../selftests/kvm/tests/arm64/aarch32_id_regs/default.test | 1 +
.../selftests/kvm/tests/arm64/arch_timer_edge_cases/default.test | 1 +
.../selftests/kvm/tests/arm64/debug-exceptions/default.test | 1 +
tools/testing/selftests/kvm/tests/arm64/hypercalls/default.test | 1 +
tools/testing/selftests/kvm/tests/arm64/mmio_abort/default.test | 1 +
tools/testing/selftests/kvm/tests/arm64/no-vgic-v3/default.test | 1 +
.../selftests/kvm/tests/arm64/page_fault_test/default.test | 1 +
tools/testing/selftests/kvm/tests/arm64/psci_test/default.test | 1 +
tools/testing/selftests/kvm/tests/arm64/set_id_regs/default.test | 1 +
.../testing/selftests/kvm/tests/arm64/smccc_filter/default.test | 1 +
.../selftests/kvm/tests/arm64/vcpu_width_config/default.test | 1 +
tools/testing/selftests/kvm/tests/arm64/vgic_init/default.test | 1 +
tools/testing/selftests/kvm/tests/arm64/vgic_irq/default.test | 1 +
.../selftests/kvm/tests/arm64/vgic_lpi_stress/default.test | 1 +
.../selftests/kvm/tests/arm64/vpmu_counter_access/default.test | 1 +
tools/testing/selftests/kvm/tests/get-reg-list/default.test | 1 +
17 files changed, 17 insertions(+)
create mode 100644 tools/testing/selftests/kvm/tests/arch_timer/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/aarch32_id_regs/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/arch_timer_edge_cases/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/debug-exceptions/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/hypercalls/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/mmio_abort/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/no-vgic-v3/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/page_fault_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/psci_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/set_id_regs/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/smccc_filter/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/vcpu_width_config/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/vgic_init/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/vgic_irq/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/vgic_lpi_stress/default.test
create mode 100644 tools/testing/selftests/kvm/tests/arm64/vpmu_counter_access/default.test
create mode 100644 tools/testing/selftests/kvm/tests/get-reg-list/default.test
diff --git a/tools/testing/selftests/kvm/tests/arch_timer/default.test b/tools/testing/selftests/kvm/tests/arch_timer/default.test
new file mode 100644
index 000000000000..97cac72515e2
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arch_timer/default.test
@@ -0,0 +1 @@
+arch_timer
diff --git a/tools/testing/selftests/kvm/tests/arm64/aarch32_id_regs/default.test b/tools/testing/selftests/kvm/tests/arm64/aarch32_id_regs/default.test
new file mode 100644
index 000000000000..e9dc42eacb27
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/aarch32_id_regs/default.test
@@ -0,0 +1 @@
+arm64/aarch32_id_regs
diff --git a/tools/testing/selftests/kvm/tests/arm64/arch_timer_edge_cases/default.test b/tools/testing/selftests/kvm/tests/arm64/arch_timer_edge_cases/default.test
new file mode 100644
index 000000000000..ebc7f45fab7e
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/arch_timer_edge_cases/default.test
@@ -0,0 +1 @@
+arm64/arch_timer_edge_cases
diff --git a/tools/testing/selftests/kvm/tests/arm64/debug-exceptions/default.test b/tools/testing/selftests/kvm/tests/arm64/debug-exceptions/default.test
new file mode 100644
index 000000000000..dc427a64e1cd
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/debug-exceptions/default.test
@@ -0,0 +1 @@
+arm64/debug-exceptions
diff --git a/tools/testing/selftests/kvm/tests/arm64/hypercalls/default.test b/tools/testing/selftests/kvm/tests/arm64/hypercalls/default.test
new file mode 100644
index 000000000000..628731a70178
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/hypercalls/default.test
@@ -0,0 +1 @@
+arm64/hypercalls
diff --git a/tools/testing/selftests/kvm/tests/arm64/mmio_abort/default.test b/tools/testing/selftests/kvm/tests/arm64/mmio_abort/default.test
new file mode 100644
index 000000000000..94e2aef05572
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/mmio_abort/default.test
@@ -0,0 +1 @@
+arm64/mmio_abort
diff --git a/tools/testing/selftests/kvm/tests/arm64/no-vgic-v3/default.test b/tools/testing/selftests/kvm/tests/arm64/no-vgic-v3/default.test
new file mode 100644
index 000000000000..5fa28e3d6f9c
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/no-vgic-v3/default.test
@@ -0,0 +1 @@
+arm64/no-vgic-v3
diff --git a/tools/testing/selftests/kvm/tests/arm64/page_fault_test/default.test b/tools/testing/selftests/kvm/tests/arm64/page_fault_test/default.test
new file mode 100644
index 000000000000..dbf95af263d8
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/page_fault_test/default.test
@@ -0,0 +1 @@
+arm64/page_fault_test
diff --git a/tools/testing/selftests/kvm/tests/arm64/psci_test/default.test b/tools/testing/selftests/kvm/tests/arm64/psci_test/default.test
new file mode 100644
index 000000000000..1cbe0cee5277
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/psci_test/default.test
@@ -0,0 +1 @@
+arm64/psci_test
diff --git a/tools/testing/selftests/kvm/tests/arm64/set_id_regs/default.test b/tools/testing/selftests/kvm/tests/arm64/set_id_regs/default.test
new file mode 100644
index 000000000000..e305642fda26
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/set_id_regs/default.test
@@ -0,0 +1 @@
+arm64/set_id_regs
diff --git a/tools/testing/selftests/kvm/tests/arm64/smccc_filter/default.test b/tools/testing/selftests/kvm/tests/arm64/smccc_filter/default.test
new file mode 100644
index 000000000000..98dd986522a1
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/smccc_filter/default.test
@@ -0,0 +1 @@
+arm64/smccc_filter
diff --git a/tools/testing/selftests/kvm/tests/arm64/vcpu_width_config/default.test b/tools/testing/selftests/kvm/tests/arm64/vcpu_width_config/default.test
new file mode 100644
index 000000000000..d00fd74341ac
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/vcpu_width_config/default.test
@@ -0,0 +1 @@
+arm64/vcpu_width_config
diff --git a/tools/testing/selftests/kvm/tests/arm64/vgic_init/default.test b/tools/testing/selftests/kvm/tests/arm64/vgic_init/default.test
new file mode 100644
index 000000000000..7835ba6157b5
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/vgic_init/default.test
@@ -0,0 +1 @@
+arm64/vgic_init
diff --git a/tools/testing/selftests/kvm/tests/arm64/vgic_irq/default.test b/tools/testing/selftests/kvm/tests/arm64/vgic_irq/default.test
new file mode 100644
index 000000000000..387612d50530
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/vgic_irq/default.test
@@ -0,0 +1 @@
+arm64/vgic_irq
diff --git a/tools/testing/selftests/kvm/tests/arm64/vgic_lpi_stress/default.test b/tools/testing/selftests/kvm/tests/arm64/vgic_lpi_stress/default.test
new file mode 100644
index 000000000000..c31424bc5a88
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/vgic_lpi_stress/default.test
@@ -0,0 +1 @@
+arm64/vgic_lpi_stress
diff --git a/tools/testing/selftests/kvm/tests/arm64/vpmu_counter_access/default.test b/tools/testing/selftests/kvm/tests/arm64/vpmu_counter_access/default.test
new file mode 100644
index 000000000000..505743df541f
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/arm64/vpmu_counter_access/default.test
@@ -0,0 +1 @@
+arm64/vpmu_counter_access
diff --git a/tools/testing/selftests/kvm/tests/get-reg-list/default.test b/tools/testing/selftests/kvm/tests/get-reg-list/default.test
new file mode 100644
index 000000000000..07b9427b9f83
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/get-reg-list/default.test
@@ -0,0 +1 @@
+get-reg-list
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 14/15] KVM: selftests: Add s390 auto generated test files for KVM Selftests Runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (12 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 13/15] KVM: selftests: Add arm64 " Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 15/15] KVM: selftests: Add riscv " Vipin Sharma
2025-07-09 22:25 ` [PATCH v2 00/15] Add KVM Selftests runner Sean Christopherson
15 siblings, 0 replies; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add auto generated test files for s390 platforms.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
tools/testing/selftests/kvm/tests/s390/cmma_test/default.test | 1 +
.../selftests/kvm/tests/s390/cpumodel_subfuncs_test/default.test | 1 +
tools/testing/selftests/kvm/tests/s390/debug_test/default.test | 1 +
tools/testing/selftests/kvm/tests/s390/memop/default.test | 1 +
tools/testing/selftests/kvm/tests/s390/resets/default.test | 1 +
.../selftests/kvm/tests/s390/shared_zeropage_test/default.test | 1 +
.../testing/selftests/kvm/tests/s390/sync_regs_test/default.test | 1 +
tools/testing/selftests/kvm/tests/s390/tprot/default.test | 1 +
.../testing/selftests/kvm/tests/s390/ucontrol_test/default.test | 1 +
9 files changed, 9 insertions(+)
create mode 100644 tools/testing/selftests/kvm/tests/s390/cmma_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/cpumodel_subfuncs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/debug_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/memop/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/resets/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/shared_zeropage_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/sync_regs_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/tprot/default.test
create mode 100644 tools/testing/selftests/kvm/tests/s390/ucontrol_test/default.test
diff --git a/tools/testing/selftests/kvm/tests/s390/cmma_test/default.test b/tools/testing/selftests/kvm/tests/s390/cmma_test/default.test
new file mode 100644
index 000000000000..c6655b3e9472
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/s390/cmma_test/default.test
@@ -0,0 +1 @@
+s390/cmma_test
diff --git a/tools/testing/selftests/kvm/tests/s390/cpumodel_subfuncs_test/default.test b/tools/testing/selftests/kvm/tests/s390/cpumodel_subfuncs_test/default.test
new file mode 100644
index 000000000000..d02545ae1f3b
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/s390/cpumodel_subfuncs_test/default.test
@@ -0,0 +1 @@
+s390/cpumodel_subfuncs_test
diff --git a/tools/testing/selftests/kvm/tests/s390/debug_test/default.test b/tools/testing/selftests/kvm/tests/s390/debug_test/default.test
new file mode 100644
index 000000000000..bd0a9c8965b2
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/s390/debug_test/default.test
@@ -0,0 +1 @@
+s390/debug_test
diff --git a/tools/testing/selftests/kvm/tests/s390/memop/default.test b/tools/testing/selftests/kvm/tests/s390/memop/default.test
new file mode 100644
index 000000000000..76073a59f58d
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/s390/memop/default.test
@@ -0,0 +1 @@
+s390/memop
diff --git a/tools/testing/selftests/kvm/tests/s390/resets/default.test b/tools/testing/selftests/kvm/tests/s390/resets/default.test
new file mode 100644
index 000000000000..133beeb1260b
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/s390/resets/default.test
@@ -0,0 +1 @@
+s390/resets
diff --git a/tools/testing/selftests/kvm/tests/s390/shared_zeropage_test/default.test b/tools/testing/selftests/kvm/tests/s390/shared_zeropage_test/default.test
new file mode 100644
index 000000000000..8ea3ec559fda
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/s390/shared_zeropage_test/default.test
@@ -0,0 +1 @@
+s390/shared_zeropage_test
diff --git a/tools/testing/selftests/kvm/tests/s390/sync_regs_test/default.test b/tools/testing/selftests/kvm/tests/s390/sync_regs_test/default.test
new file mode 100644
index 000000000000..7eacd6f9bd2d
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/s390/sync_regs_test/default.test
@@ -0,0 +1 @@
+s390/sync_regs_test
diff --git a/tools/testing/selftests/kvm/tests/s390/tprot/default.test b/tools/testing/selftests/kvm/tests/s390/tprot/default.test
new file mode 100644
index 000000000000..78e603c3b671
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/s390/tprot/default.test
@@ -0,0 +1 @@
+s390/tprot
diff --git a/tools/testing/selftests/kvm/tests/s390/ucontrol_test/default.test b/tools/testing/selftests/kvm/tests/s390/ucontrol_test/default.test
new file mode 100644
index 000000000000..211cceee1bbd
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/s390/ucontrol_test/default.test
@@ -0,0 +1 @@
+s390/ucontrol_test
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 15/15] KVM: selftests: Add riscv auto generated test files for KVM Selftests Runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (13 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 14/15] KVM: selftests: Add s390 " Vipin Sharma
@ 2025-06-06 23:56 ` Vipin Sharma
2025-06-09 12:54 ` Andrew Jones
2025-07-09 22:25 ` [PATCH v2 00/15] Add KVM Selftests runner Sean Christopherson
15 siblings, 1 reply; 27+ messages in thread
From: Vipin Sharma @ 2025-06-06 23:56 UTC (permalink / raw)
To: kvm, kvmarm, kvm-riscv, linux-arm-kernel
Cc: seanjc, pbonzini, anup, borntraeger, frankja, imbrenda, maz,
oliver.upton, dmatlack, Vipin Sharma
Add auto generated test files for s390 platforms.
Signed-off-by: Vipin Sharma <vipinsh@google.com>
---
tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test | 1 +
.../testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test | 1 +
2 files changed, 2 insertions(+)
create mode 100644 tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test
create mode 100644 tools/testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test
diff --git a/tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test b/tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test
new file mode 100644
index 000000000000..d34b4b9b77ae
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test
@@ -0,0 +1 @@
+riscv/ebreak_test
diff --git a/tools/testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test b/tools/testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test
new file mode 100644
index 000000000000..5abb62c51097
--- /dev/null
+++ b/tools/testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test
@@ -0,0 +1 @@
+riscv/sbi_pmu_test
--
2.50.0.rc0.604.gd4ff7b7c86-goog
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH v2 15/15] KVM: selftests: Add riscv auto generated test files for KVM Selftests Runner
2025-06-06 23:56 ` [PATCH v2 15/15] KVM: selftests: Add riscv " Vipin Sharma
@ 2025-06-09 12:54 ` Andrew Jones
0 siblings, 0 replies; 27+ messages in thread
From: Andrew Jones @ 2025-06-09 12:54 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, seanjc, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton, dmatlack
On Fri, Jun 06, 2025 at 04:56:19PM -0700, Vipin Sharma wrote:
> Add auto generated test files for s390 platforms.
^ riscv
Thanks,
drew
>
> Signed-off-by: Vipin Sharma <vipinsh@google.com>
> ---
> tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test | 1 +
> .../testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test | 1 +
> 2 files changed, 2 insertions(+)
> create mode 100644 tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test
> create mode 100644 tools/testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test
>
> diff --git a/tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test b/tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test
> new file mode 100644
> index 000000000000..d34b4b9b77ae
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/tests/riscv/ebreak_test/default.test
> @@ -0,0 +1 @@
> +riscv/ebreak_test
> diff --git a/tools/testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test b/tools/testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test
> new file mode 100644
> index 000000000000..5abb62c51097
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/tests/riscv/sbi_pmu_test/default.test
> @@ -0,0 +1 @@
> +riscv/sbi_pmu_test
> --
> 2.50.0.rc0.604.gd4ff7b7c86-goog
>
>
> --
> kvm-riscv mailing list
> kvm-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kvm-riscv
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 02/15] KVM: selftests: Enable selftests runner to find executables in different path
2025-06-06 23:56 ` [PATCH v2 02/15] KVM: selftests: Enable selftests runner to find executables in different path Vipin Sharma
@ 2025-07-09 21:39 ` Sean Christopherson
0 siblings, 0 replies; 27+ messages in thread
From: Sean Christopherson @ 2025-07-09 21:39 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton, dmatlack
On Fri, Jun 06, 2025, Vipin Sharma wrote:
> Add command line option, --executable/-e, to specify a directory where
> test binaries are present. If this option is not provided then default
> to the current directory.
>
> Example:
> python3 runner --test-dirs test -e ~/build/selftests
>
> This option enables executing tests from out-of-tree builds.
>
> Signed-off-by: Vipin Sharma <vipinsh@google.com>
> ---
> tools/testing/selftests/kvm/runner/__main__.py | 8 +++++++-
> tools/testing/selftests/kvm/runner/selftest.py | 4 ++--
> tools/testing/selftests/kvm/runner/test_runner.py | 4 ++--
> 3 files changed, 11 insertions(+), 5 deletions(-)
>
> diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
> index b2c85606c516..599300831504 100644
> --- a/tools/testing/selftests/kvm/runner/__main__.py
> +++ b/tools/testing/selftests/kvm/runner/__main__.py
> @@ -29,6 +29,12 @@ def cli():
> default=[],
> help="Run tests in the given directory and all of its sub directories. Provide the space separated paths to add multiple directories.")
>
> + parser.add_argument("-e",
> + "--executable",
"executable" is kinda odd to me, as that suggests a single, specific executable.
-p/--path seems more aligned with how this concept is typically described in Linux.
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner
2025-06-06 23:56 ` [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner Vipin Sharma
@ 2025-07-09 21:46 ` Sean Christopherson
2025-07-09 22:25 ` Sean Christopherson
1 sibling, 0 replies; 27+ messages in thread
From: Sean Christopherson @ 2025-07-09 21:46 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton, dmatlack
On Fri, Jun 06, 2025, Vipin Sharma wrote:
> diff --git a/tools/testing/selftests/kvm/runner/command.py b/tools/testing/selftests/kvm/runner/command.py
> index a63ff53a92b3..44c8e0875779 100644
> --- a/tools/testing/selftests/kvm/runner/command.py
> +++ b/tools/testing/selftests/kvm/runner/command.py
> @@ -12,14 +12,16 @@ class Command:
> Returns the exit code, std output and std error of the command.
> """
>
> - def __init__(self, command):
> + def __init__(self, command, timeout):
> self.command = command
> + self.timeout = timeout
>
> def run(self):
> run_args = {
> "universal_newlines": True,
> "shell": True,
> "capture_output": True,
> + "timeout": self.timeout,
> }
> @@ -48,10 +50,13 @@ class Selftest:
> self.stderr = "File doesn't exists."
> return
>
> - ret, self.stdout, self.stderr = self.command.run()
> - if ret == 0:
> - self.status = SelftestStatus.PASSED
> - elif ret == 4:
> - self.status = SelftestStatus.SKIPPED
> - else:
> - self.status = SelftestStatus.FAILED
> + try:
> + ret, self.stdout, self.stderr = self.command.run()
I don't see any value in having both command.py and selftest.py. TimeoutExpired
*really* should be handled by Command, especially with respect to stdout/stderr
(more on that later), but that complicates converting return codes to SelftestStatus.
And even if we do figure out a clean split, one of Command or Selftest would end
but being little more than a wrapper or trampoline, with more boilerplate code
than novel logic.
I don't anticipate turning this into a general execution framework. The entire
purpose is to run a selftest, so I think it makes sense to scrap Command and just
have Selftest deal with running the executable.
> + if ret == 0:
> + self.status = SelftestStatus.PASSED
> + elif ret == 4:
> + self.status = SelftestStatus.SKIPPED
> + else:
> + self.status = SelftestStatus.FAILED
> + except subprocess.TimeoutExpired as e:
> + self.status = SelftestStatus.TIMED_OUT
> diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
> index 104f0b4c2e4e..1409e1cfe7d5 100644
> --- a/tools/testing/selftests/kvm/runner/test_runner.py
> +++ b/tools/testing/selftests/kvm/runner/test_runner.py
> @@ -15,7 +15,7 @@ class TestRunner:
> self.tests = []
>
> for test_file in test_files:
> - self.tests.append(Selftest(test_file, args.executable))
> + self.tests.append(Selftest(test_file, args.executable, args.timeout))
>
> def _log_result(self, test_result):
> logger.log(test_result.status,
> --
> 2.50.0.rc0.604.gd4ff7b7c86-goog
>
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 04/15] KVM: selftests: Add option to save selftest runner output to a directory
2025-06-06 23:56 ` [PATCH v2 04/15] KVM: selftests: Add option to save selftest runner output to a directory Vipin Sharma
@ 2025-07-09 21:52 ` Sean Christopherson
0 siblings, 0 replies; 27+ messages in thread
From: Sean Christopherson @ 2025-07-09 21:52 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton, dmatlack
On Fri, Jun 06, 2025, Vipin Sharma wrote:
> - def run(self):
> + def _run(self, output=None, error=None):
> run_args = {
> "universal_newlines": True,
> "shell": True,
> - "capture_output": True,
> "timeout": self.timeout,
> }
>
> + if output is None and error is None:
> + run_args.update({"capture_output": True})
The runner needs to check that its min version, whatever that ends up being, is
satisfied.
capture_output requires 3.7, but nothing in the runner actually checks that the
min version is met. If you hadn't mentioned running into a problem with 3.6, I
don't know that I would have figured out what was going wrong all that quickly.
There's also no reason to use capture_output, which appears to be the only
3.7+ dependency. Just pass subprocess.PIPE for stdout and stderr.
> + else:
> + run_args.update({"stdout": output, "stderr": error})
> +
> proc = subprocess.run(self.command, **run_args)
> return proc.returncode, proc.stdout, proc.stderr
> +
> + def run(self):
> + if self.output_dir is not None:
> + pathlib.Path(self.output_dir).mkdir(parents=True, exist_ok=True)
> +
> + output = None
> + error = None
> + with contextlib.ExitStack() as stack:
> + if self.output_dir is not None:
> + output_path = os.path.join(self.output_dir, "stdout")
> + output = stack.enter_context(
> + open(output_path, encoding="utf-8", mode="w"))
> +
> + error_path = os.path.join(self.output_dir, "stderr")
> + error = stack.enter_context(
> + open(error_path, encoding="utf-8", mode="w"))
> + return self._run(output, error)
> diff --git a/tools/testing/selftests/kvm/runner/selftest.py b/tools/testing/selftests/kvm/runner/selftest.py
> index 4c72108c47de..664958c693e5 100644
> --- a/tools/testing/selftests/kvm/runner/selftest.py
> +++ b/tools/testing/selftests/kvm/runner/selftest.py
> @@ -32,7 +32,7 @@ class Selftest:
> Extract the test execution command from test file and executes it.
> """
>
> - def __init__(self, test_path, executable_dir, timeout):
> + def __init__(self, test_path, executable_dir, timeout, output_dir):
> test_command = pathlib.Path(test_path).read_text().strip()
> if not test_command:
> raise ValueError("Empty test command in " + test_path)
> @@ -40,7 +40,11 @@ class Selftest:
> test_command = os.path.join(executable_dir, test_command)
> self.exists = os.path.isfile(test_command.split(maxsplit=1)[0])
> self.test_path = test_path
> - self.command = command.Command(test_command, timeout)
> +
> + if output_dir is not None:
> + output_dir = os.path.join(output_dir, test_path.lstrip("/"))
> + self.command = command.Command(test_command, timeout, output_dir)
> +
> self.status = SelftestStatus.NO_RUN
> self.stdout = ""
> self.stderr = ""
> diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
> index 1409e1cfe7d5..0501d77a9912 100644
> --- a/tools/testing/selftests/kvm/runner/test_runner.py
> +++ b/tools/testing/selftests/kvm/runner/test_runner.py
> @@ -13,19 +13,22 @@ logger = logging.getLogger("runner")
> class TestRunner:
> def __init__(self, test_files, args):
> self.tests = []
> + self.output_dir = args.output
>
> for test_file in test_files:
> - self.tests.append(Selftest(test_file, args.executable, args.timeout))
> + self.tests.append(Selftest(test_file, args.executable,
> + args.timeout, args.output))
>
> def _log_result(self, test_result):
> logger.log(test_result.status,
> f"[{test_result.status}] {test_result.test_path}")
Previous patch, but I missed it there. Print the *name* of the result, not the
integer, which is arbitrary magic. i.e
logger.log(test_result.status,
f"[{test_result.status.name}] {test_result.test_path}")
> - logger.info("************** STDOUT BEGIN **************")
> - logger.info(test_result.stdout)
> - logger.info("************** STDOUT END **************")
> - logger.info("************** STDERR BEGIN **************")
> - logger.info(test_result.stderr)
> - logger.info("************** STDERR END **************")
> + if (self.output_dir is None):
Ugh. I want both. Recording to disk shouldn't prevent the user from seeing
real-time data.
Rather than redirect to a file, always pipe to stdout/stderr, and then simply
write to the appropriate files after the subprocess completes. That'll also force
the issue on fixing a bug with timeouts, where the runner doesn't capture stdout
or stderr.
> + logger.info("************** STDOUT BEGIN **************")
> + logger.info(test_result.stdout)
> + logger.info("************** STDOUT END **************")
> + logger.info("************** STDERR BEGIN **************")
> + logger.info(test_result.stderr)
> + logger.info("************** STDERR END **************")
This is unnecessarily verbose. The logger spits out a timestamp, just use that
to demarcate, e.g.
logger.info("*** stdout ***\n" + test_result.stdout)
logger.info("*** stderr ***\n" + test_result.stderr)
yields
14:52:29 | *** stdout ***
Random seed: 0x6b8b4567
14:52:29 | *** stderr ***
==== Test Assertion Failure ====
x86/state_test.c:316: memcmp(®s1, ®s2, sizeof(regs2))
pid=168652 tid=168652 errno=4 - Interrupted system call
1 0x0000000000402300: main at state_test.c:316 (discriminator 1)
2 0x000000000041e413: __libc_start_call_main at libc-start.o:?
3 0x00000000004205bc: __libc_start_main_impl at ??:?
4 0x00000000004027a0: _start at ??:?
Unexpected register values after vcpu_load_state; rdi: 7ff68b1f3040 rsi: 0
>
> def start(self):
> ret = 0
> --
> 2.50.0.rc0.604.gd4ff7b7c86-goog
>
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 06/15] KVM: selftests: Add a flag to print only test status in KVM Selftests run
2025-06-06 23:56 ` [PATCH v2 06/15] KVM: selftests: Add a flag to print only test status in KVM Selftests run Vipin Sharma
@ 2025-07-09 21:55 ` Sean Christopherson
0 siblings, 0 replies; 27+ messages in thread
From: Sean Christopherson @ 2025-07-09 21:55 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton, dmatlack
On Fri, Jun 06, 2025, Vipin Sharma wrote:
> Add a command line argument, --print-status, to limit content printed on
> terminal by default. When this flag is passed only print final status of
> tests i.e. passed, failed, timed out, etc.
>
> Example:
> python3 runner --test-dirs tests --print-status
>
> Signed-off-by: Vipin Sharma <vipinsh@google.com>
> ---
> tools/testing/selftests/kvm/runner/__main__.py | 5 +++++
> tools/testing/selftests/kvm/runner/test_runner.py | 3 ++-
> 2 files changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
> index 48d7ce00a097..3f11a20e76a9 100644
> --- a/tools/testing/selftests/kvm/runner/__main__.py
> +++ b/tools/testing/selftests/kvm/runner/__main__.py
> @@ -59,6 +59,11 @@ def cli():
> type=int,
> help="Maximum number of tests that can be run concurrently. (Default: 1)")
>
> + parser.add_argument("--print-status",
> + action="store_true",
> + default=False,
> + help="Print only test's status and avoid printing stdout and stderr of the tests")
> +
> return parser.parse_args()
>
>
> diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
> index 0a6e5e0ca0f5..474408fcab51 100644
> --- a/tools/testing/selftests/kvm/runner/test_runner.py
> +++ b/tools/testing/selftests/kvm/runner/test_runner.py
> @@ -17,6 +17,7 @@ class TestRunner:
> self.tests = []
> self.output_dir = args.output
> self.jobs = args.jobs
> + self.print_status = args.print_status
>
> for test_file in test_files:
> self.tests.append(Selftest(test_file, args.executable,
> @@ -29,7 +30,7 @@ class TestRunner:
> def _log_result(self, test_result):
> logger.log(test_result.status,
> f"[{test_result.status}] {test_result.test_path}")
> - if (self.output_dir is None):
> + if (self.output_dir is None and self.print_status is False):
This flag confused the hell out of me. It's not at all obvious that it *reduces*
output, and that it overrides all the other --print-xxx options. I see no reason
to provide this. I suspect most users will want curated console information, or
none at all.
If we keep this, the runner should explicitly disallow --print-status with any
other --print-xxx option.
> logger.info("************** STDOUT BEGIN **************")
> logger.info(test_result.stdout)
> logger.info("************** STDOUT END **************")
> --
> 2.50.0.rc0.604.gd4ff7b7c86-goog
>
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 07/15] KVM: selftests: Add various print flags to KVM Selftest Runner
2025-06-06 23:56 ` [PATCH v2 07/15] KVM: selftests: Add various print flags to KVM Selftest Runner Vipin Sharma
@ 2025-07-09 22:01 ` Sean Christopherson
0 siblings, 0 replies; 27+ messages in thread
From: Sean Christopherson @ 2025-07-09 22:01 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton, dmatlack
On Fri, Jun 06, 2025, Vipin Sharma wrote:
> Add various print flags to selectively print outputs on terminal based
> on test execution status (passed, failed, timed out, skipped, no run).
>
> Provide further options to print only particular execution status, like
> print only status of failed tests.
>
> Example: To print status, stdout and stderr for failed tests and only
> print status of passed test:
>
> python3 runner --test-dirs tests --print-failed \
> --print-passed-status
>
> Signed-off-by: Vipin Sharma <vipinsh@google.com>
> ---
> .../testing/selftests/kvm/runner/__main__.py | 114 ++++++++++++++++++
> .../selftests/kvm/runner/test_runner.py | 10 +-
> 2 files changed, 123 insertions(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
> index 3f11a20e76a9..4406d8e4847a 100644
> --- a/tools/testing/selftests/kvm/runner/__main__.py
> +++ b/tools/testing/selftests/kvm/runner/__main__.py
> @@ -64,9 +64,115 @@ def cli():
> default=False,
> help="Print only test's status and avoid printing stdout and stderr of the tests")
>
> + parser.add_argument("--print-passed",
> + action="store_true",
> + default=False,
> + help="Print passed test's stdout, stderr and status."
> + )
> +
> + parser.add_argument("--print-passed-status",
> + action="store_true",
> + default=False,
> + help="Print only passed test's status."
> + )
Waaay too many booleans :-)
And they don't provide the right granularity. E.g. I don't want stdout for FAILED,
I just want stderr for the assert.
After some fiddling, I came up with this:
parser.add_argument("--print-passed", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
help="blah"
)
parser.add_argument("--print-failed", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
help="Full = print each test's stdout, stderr and status; status = only status."
)
parser.add_argument("--print-skipped", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
help="Print skipped test's stdout, stderr and status."
)
parser.add_argument("--print-timed-out", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
help="Print timed out test's stdout, stderr and status."
)
parser.add_argument("--print-no-run", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
help="Print stdout, stderr and status for tests which didn't run."
)
The const+nargs magic makes a bare option equivalent to "full", e.g. --print-timed-out
and --print-timed-out=full are the same. That gives the user complete control
over the output, with _less_ code in the runner. E.g. to print stdout and/or stderr:
print_level = self.print_stds.get(test_result.status);
if (print_level == "full" or print_level == "stdout"):
logger.info("*** stdout ***\n" + test_result.stdout)
if (print_level == "full" or print_level == "stderr"):
logger.info("*** stderr ***\n" + test_result.stderr)
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 00/15] Add KVM Selftests runner
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
` (14 preceding siblings ...)
2025-06-06 23:56 ` [PATCH v2 15/15] KVM: selftests: Add riscv " Vipin Sharma
@ 2025-07-09 22:25 ` Sean Christopherson
15 siblings, 0 replies; 27+ messages in thread
From: Sean Christopherson @ 2025-07-09 22:25 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton, dmatlack
On Fri, Jun 06, 2025, Vipin Sharma wrote:
> tools/testing/selftests/kvm/.gitignore | 4 +-
> tools/testing/selftests/kvm/Makefile.kvm | 8 +
> .../testing/selftests/kvm/runner/__main__.py | 271 ++++++++++++++++++
> tools/testing/selftests/kvm/runner/command.py | 53 ++++
> .../testing/selftests/kvm/runner/selftest.py | 66 +++++
> .../selftests/kvm/runner/test_runner.py | 88 ++++++
Overall, looks great! I think the only significant feedback is on the command
line options.
One thing we probably need is a README of some form, to explain how this works
and to give some examples. Outside of you and I, I doubt anyone will know how
to use this :-)
Here's the full diff of the modifications I made to massage things to my liking.
It's not complete, e.g. I punted on the help messages and didn't change to
-p/--path, but otherwise it seems to work?
---
.../testing/selftests/kvm/runner/__main__.py | 86 +++----------------
tools/testing/selftests/kvm/runner/command.py | 53 ------------
.../testing/selftests/kvm/runner/selftest.py | 39 +++++++--
.../selftests/kvm/runner/test_runner.py | 20 ++---
4 files changed, 53 insertions(+), 145 deletions(-)
delete mode 100644 tools/testing/selftests/kvm/runner/command.py
diff --git a/tools/testing/selftests/kvm/runner/__main__.py b/tools/testing/selftests/kvm/runner/__main__.py
index c02035a62873..0105835c557c 100644
--- a/tools/testing/selftests/kvm/runner/__main__.py
+++ b/tools/testing/selftests/kvm/runner/__main__.py
@@ -59,71 +59,26 @@ def cli():
type=int,
help="Maximum number of tests that can be run concurrently. (Default: 1)")
- parser.add_argument("--print-status",
- action="store_true",
- default=False,
- help="Print only test's status and avoid printing stdout and stderr of the tests")
-
- parser.add_argument("--print-passed",
- action="store_true",
- default=False,
- help="Print passed test's stdout, stderr and status."
- )
-
- parser.add_argument("--print-passed-status",
- action="store_true",
- default=False,
- help="Print only passed test's status."
- )
-
- parser.add_argument("--print-failed",
- action="store_true",
- default=False,
- help="Print failed test's stdout, stderr and status."
+ parser.add_argument("--print-passed", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
+ help="blah"
)
- parser.add_argument("--print-failed-status",
- action="store_true",
- default=False,
- help="Print only failed test's status."
+ parser.add_argument("--print-failed", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
+ help="Full = print each test's stdout, stderr and status; status = only status."
)
- parser.add_argument("--print-skipped",
- action="store_true",
- default=False,
+ parser.add_argument("--print-skipped", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
help="Print skipped test's stdout, stderr and status."
)
- parser.add_argument("--print-skipped-status",
- action="store_true",
- default=False,
- help="Print only skipped test's status."
- )
-
- parser.add_argument("--print-timed-out",
- action="store_true",
- default=False,
+ parser.add_argument("--print-timed-out", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
help="Print timed out test's stdout, stderr and status."
)
- parser.add_argument("--print-timed-out-status",
- action="store_true",
- default=False,
- help="Print only timed out test's status."
- )
-
- parser.add_argument("--print-no-runs",
- action="store_true",
- default=False,
+ parser.add_argument("--print-no-run", default="full", const="full", nargs='?', choices=["off", "full", "stderr", "stdout", "status"],
help="Print stdout, stderr and status for tests which didn't run."
)
- parser.add_argument("--print-no-runs-status",
- action="store_true",
- default=False,
- help="Print only tests which didn't run."
- )
-
parser.add_argument("--sticky-summary-only",
action="store_true",
default=False,
@@ -145,36 +100,19 @@ def level_filters(args):
if args.sticky_summary_only or args.quiet:
return levels
- if args.print_passed or args.print_passed_status or args.print_status:
+ if args.print_passed != "off":
levels.add(SelftestStatus.PASSED)
- if args.print_failed or args.print_failed_status or args.print_status:
+ if args.print_failed != "off":
levels.add(SelftestStatus.FAILED)
- if args.print_skipped or args.print_skipped_status or args.print_status:
+ if args.print_skipped != "off":
levels.add(SelftestStatus.SKIPPED)
- if args.print_timed_out or args.print_timed_out_status or args.print_status:
+ if args.print_timed_out != "off":
levels.add(SelftestStatus.TIMED_OUT)
- if args.print_no_runs or args.print_no_runs_status or args.print_status:
- levels.add(SelftestStatus.NO_RUN)
-
- # Nothing set explicitly, enable all.
- if not levels:
- args.print_passed = True
- levels.add(SelftestStatus.PASSED)
-
- args.print_failed = True
- levels.add(SelftestStatus.FAILED)
-
- args.print_skipped = True
- levels.add(SelftestStatus.SKIPPED)
-
- args.print_timed_out = True
- levels.add(SelftestStatus.TIMED_OUT)
-
- args.print_no_runs = True
+ if args.print_no_run != "off":
levels.add(SelftestStatus.NO_RUN)
levels.add(logging.NOTSET)
diff --git a/tools/testing/selftests/kvm/runner/command.py b/tools/testing/selftests/kvm/runner/command.py
deleted file mode 100644
index 6f6b1811b490..000000000000
--- a/tools/testing/selftests/kvm/runner/command.py
+++ /dev/null
@@ -1,53 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# Copyright 2025 Google LLC
-#
-# Author: vipinsh@google.com (Vipin Sharma)
-
-import subprocess
-import pathlib
-import contextlib
-import os
-
-
-class Command:
- """Executes a command in shell.
-
- Returns the exit code, std output and std error of the command.
- """
-
- def __init__(self, command, timeout, output_dir):
- self.command = command
- self.timeout = timeout
- self.output_dir = output_dir
-
- def _run(self, output=None, error=None):
- run_args = {
- "universal_newlines": True,
- "shell": True,
- "timeout": self.timeout,
- }
-
- if output is None and error is None:
- run_args.update({"capture_output": True})
- else:
- run_args.update({"stdout": output, "stderr": error})
-
- proc = subprocess.run(self.command, **run_args)
- return proc.returncode, proc.stdout, proc.stderr
-
- def run(self):
- if self.output_dir is not None:
- pathlib.Path(self.output_dir).mkdir(parents=True, exist_ok=True)
-
- output = None
- error = None
- with contextlib.ExitStack() as stack:
- if self.output_dir is not None:
- output_path = os.path.join(self.output_dir, "stdout")
- output = stack.enter_context(
- open(output_path, encoding="utf-8", mode="w"))
-
- error_path = os.path.join(self.output_dir, "stderr")
- error = stack.enter_context(
- open(error_path, encoding="utf-8", mode="w"))
- return self._run(output, error)
diff --git a/tools/testing/selftests/kvm/runner/selftest.py b/tools/testing/selftests/kvm/runner/selftest.py
index 664958c693e5..1ec1ddfbf034 100644
--- a/tools/testing/selftests/kvm/runner/selftest.py
+++ b/tools/testing/selftests/kvm/runner/selftest.py
@@ -3,7 +3,6 @@
#
# Author: vipinsh@google.com (Vipin Sharma)
-import command
import pathlib
import enum
import os
@@ -37,17 +36,18 @@ class Selftest:
if not test_command:
raise ValueError("Empty test command in " + test_path)
- test_command = os.path.join(executable_dir, test_command)
- self.exists = os.path.isfile(test_command.split(maxsplit=1)[0])
+ self.command = os.path.join(executable_dir, test_command)
+ self.exists = os.path.isfile(self.command.split(maxsplit=1)[0])
self.test_path = test_path
if output_dir is not None:
output_dir = os.path.join(output_dir, test_path.lstrip("/"))
- self.command = command.Command(test_command, timeout, output_dir)
self.status = SelftestStatus.NO_RUN
self.stdout = ""
self.stderr = ""
+ self.timeout = timeout
+ self.output_dir = output_dir
def run(self):
if not self.exists:
@@ -55,12 +55,37 @@ class Selftest:
return
try:
- ret, self.stdout, self.stderr = self.command.run()
- if ret == 0:
+ run_args = {
+ "universal_newlines": True,
+ "shell": True,
+ "stdout": subprocess.PIPE,
+ "stderr": subprocess.PIPE,
+ "timeout": self.timeout,
+ }
+ proc = subprocess.run(self.command, **run_args)
+
+ self.stdout = proc.stdout
+ self.stderr = proc.stderr
+
+ if proc.returncode == 0:
self.status = SelftestStatus.PASSED
- elif ret == 4:
+ elif proc.returncode == 4:
self.status = SelftestStatus.SKIPPED
else:
self.status = SelftestStatus.FAILED
except subprocess.TimeoutExpired as e:
+ self.stdout = e.stdout
+ self.stderr = e.stderr
+
self.status = SelftestStatus.TIMED_OUT
+
+ if self.output_dir is not None:
+ pathlib.Path(self.output_dir).mkdir(parents=True, exist_ok=True)
+
+ output_path = os.path.join(self.output_dir, "stdout")
+ with open(output_path, encoding="utf-8", mode="w") as f:
+ f.write(self.stdout)
+
+ error_path = os.path.join(self.output_dir, "stderr")
+ with open(error_path, encoding="utf-8", mode="w") as f:
+ f.write(self.stderr)
diff --git a/tools/testing/selftests/kvm/runner/test_runner.py b/tools/testing/selftests/kvm/runner/test_runner.py
index e7730880907d..a285a711a686 100644
--- a/tools/testing/selftests/kvm/runner/test_runner.py
+++ b/tools/testing/selftests/kvm/runner/test_runner.py
@@ -19,13 +19,12 @@ class TestRunner:
self.output_dir = args.output
self.jobs = args.jobs
self.quiet = args.quiet
- self.print_status = args.print_status
self.print_stds = {
SelftestStatus.PASSED: args.print_passed,
SelftestStatus.FAILED: args.print_failed,
SelftestStatus.SKIPPED: args.print_skipped,
SelftestStatus.TIMED_OUT: args.print_timed_out,
- SelftestStatus.NO_RUN: args.print_no_runs
+ SelftestStatus.NO_RUN: args.print_no_run
}
for test_file in test_files:
@@ -52,15 +51,14 @@ class TestRunner:
# Clear the status line
self._print("\033[2K", end="\r")
logger.log(test_result.status,
- f"[{test_result.status}] {test_result.test_path}")
- if (self.output_dir is None and self.print_status is False
- and self.print_stds.get(test_result.status, True)):
- logger.info("************** STDOUT BEGIN **************")
- logger.info(test_result.stdout)
- logger.info("************** STDOUT END **************")
- logger.info("************** STDERR BEGIN **************")
- logger.info(test_result.stderr)
- logger.info("************** STDERR END **************")
+ f"[{test_result.status.name}] {test_result.test_path}")
+
+ print_level = self.print_stds.get(test_result.status);
+ if (print_level == "full" or print_level == "stdout"):
+ logger.info("*** stdout ***\n" + test_result.stdout)
+
+ if (print_level == "full" or print_level == "stderr"):
+ logger.info("*** stderr ***\n" + test_result.stderr)
self.status[test_result.status] += 1
# Sticky bottom line
base-commit: 611829e42fb47b99ff2b6c75637aec2410739611
--
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner
2025-06-06 23:56 ` [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner Vipin Sharma
2025-07-09 21:46 ` Sean Christopherson
@ 2025-07-09 22:25 ` Sean Christopherson
1 sibling, 0 replies; 27+ messages in thread
From: Sean Christopherson @ 2025-07-09 22:25 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton, dmatlack
On Fri, Jun 06, 2025, Vipin Sharma wrote:
> @@ -48,10 +50,13 @@ class Selftest:
> self.stderr = "File doesn't exists."
> return
>
> - ret, self.stdout, self.stderr = self.command.run()
> - if ret == 0:
> - self.status = SelftestStatus.PASSED
> - elif ret == 4:
> - self.status = SelftestStatus.SKIPPED
> - else:
> - self.status = SelftestStatus.FAILED
> + try:
> + ret, self.stdout, self.stderr = self.command.run()
> + if ret == 0:
> + self.status = SelftestStatus.PASSED
> + elif ret == 4:
> + self.status = SelftestStatus.SKIPPED
> + else:
> + self.status = SelftestStatus.FAILED
> + except subprocess.TimeoutExpired as e:
> + self.status = SelftestStatus.TIMED_OUT
This needs to grab stderr and stdout from "e", otherwise there's never any output
to the console for timeouts.
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 11/15] KVM: selftests: Auto generate default tests for KVM Selftests Runner
2025-06-06 23:56 ` [PATCH v2 11/15] KVM: selftests: Auto generate default tests for KVM Selftests Runner Vipin Sharma
@ 2025-07-09 23:06 ` Oliver Upton
2025-07-10 0:18 ` Sean Christopherson
0 siblings, 1 reply; 27+ messages in thread
From: Oliver Upton @ 2025-07-09 23:06 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, seanjc, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, dmatlack
On Fri, Jun 06, 2025 at 04:56:15PM -0700, Vipin Sharma wrote:
> Add logic in Makefile.kvm to auto generate default test files for KVM
> Selftests Runner. Preserve the hierarchy of test executables for
> autogenerated files.
>
> Autogeneration of default test files guards against missing these files
> for new tests cases or rename of existing ones. These autogenerated
> files will be checked in so that during git-diff one can easily identify
> if the existing test files for the same test binary also needs an
> update. It also add new tests automatically in the default coverage.
I'm not sure I'm sold on the merits of committing these to the tree. My
preference would be to prioritize signal to noise and just commit the
'interesting' test configurations and generate the defaults from
$(TEST_PROGS) somewhere they're gitignored.
There's no amount of foolproofing that'll prevent folks from renaming a
test name w/o updating the 'interesting' test configurations that depend
upon it. Seriously -- I'm sure I'll manage to break it at least once :)
On top of that, there's a lot of selftests that take no arguments. Not
sure what we gain having a duplicated definition for these ones outside
of $(TEST_PROGS).
Thanks,
Oliver
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 11/15] KVM: selftests: Auto generate default tests for KVM Selftests Runner
2025-07-09 23:06 ` Oliver Upton
@ 2025-07-10 0:18 ` Sean Christopherson
0 siblings, 0 replies; 27+ messages in thread
From: Sean Christopherson @ 2025-07-10 0:18 UTC (permalink / raw)
To: Oliver Upton
Cc: Vipin Sharma, kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini,
anup, borntraeger, frankja, imbrenda, maz, dmatlack
On Wed, Jul 09, 2025, Oliver Upton wrote:
> On Fri, Jun 06, 2025 at 04:56:15PM -0700, Vipin Sharma wrote:
> > Add logic in Makefile.kvm to auto generate default test files for KVM
> > Selftests Runner. Preserve the hierarchy of test executables for
> > autogenerated files.
> >
> > Autogeneration of default test files guards against missing these files
> > for new tests cases or rename of existing ones. These autogenerated
> > files will be checked in so that during git-diff one can easily identify
> > if the existing test files for the same test binary also needs an
> > update. It also add new tests automatically in the default coverage.
>
> I'm not sure I'm sold on the merits of committing these to the tree. My
> preference would be to prioritize signal to noise and just commit the
> 'interesting' test configurations and generate the defaults from
> $(TEST_PROGS) somewhere they're gitignored.
Yeah, I don't love it either, even though I suggested it.
> There's no amount of foolproofing that'll prevent folks from renaming a
> test name w/o updating the 'interesting' test configurations that depend
> upon it. Seriously -- I'm sure I'll manage to break it at least once :)
I was more worried about clobbering someone's (poorly named) default.test, e.g.
if the user modifies or creates their own default.test and the building selftests
overwrites their file.
The main reason I pushed for committing the default testcases is to avoid having
to figure out when and where to generate the testcases. I don't like the idea
of generating testcase on *every* build; it's not slow, but ugh it seem beyond
wasteful, and it feels weird to dump files into the output directory that the
user might not want.
The obvious alternative would be to add a dedicated make command. The more I
think about it, the more I think that's probably the way to go. We already have
to `make headers_install`, so having to do `make testcases_install` or whatever
doesn't seem too onerous. The biggest conundrum is probably what to put in a
.gitignore for folks that do in-tree builds. I guess maybe we could add
default.test to the .gitignore? That'd "document" that that name is reserved,
i.e. would help prevent people from trying to create their own default.test
files.
And since the runner supports listing multiple testcase directories, it'd be
quite easy to "install" the defaults to a dedicated directory while also pulling
testcases that are commited to the repo.
> On top of that, there's a lot of selftests that take no arguments. Not
> sure what we gain having a duplicated definition for these ones outside
> of $(TEST_PROGS).
The defaults are purely a way to communicate the existence of the tests to the
runner. My hacky bash scripts for running selftests copies all binaries into a
directly and then runs everything in the "output" directory, but that has obvious
flaws :-)
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 01/15] KVM: selftest: Create KVM selftest runner
2025-06-06 23:56 ` [PATCH v2 01/15] KVM: selftest: Create KVM selftest runner Vipin Sharma
@ 2025-07-10 0:20 ` Sean Christopherson
0 siblings, 0 replies; 27+ messages in thread
From: Sean Christopherson @ 2025-07-10 0:20 UTC (permalink / raw)
To: Vipin Sharma
Cc: kvm, kvmarm, kvm-riscv, linux-arm-kernel, pbonzini, anup,
borntraeger, frankja, imbrenda, maz, oliver.upton, dmatlack
On Fri, Jun 06, 2025, Vipin Sharma wrote:
> + parser.add_argument("--test-files",
> + nargs="*",
> + default=[],
> + help="Test files to run. Provide the space separated test file paths")
> +
> + parser.add_argument("--test-dirs",
> + nargs="*",
> + default=[],
> + help="Run tests in the given directory and all of its sub directories. Provide the space separated paths to add multiple directories.")
Almost forgot. My vote would be to make these easier to reference, e.g.
-t/--testcases
-d/--dirs
or if folks like the --test-{files,dirs} names, just add the single char shorthand?
--
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2025-07-10 0:23 UTC | newest]
Thread overview: 27+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-06 23:56 [PATCH v2 00/15] Add KVM Selftests runner Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 01/15] KVM: selftest: Create KVM selftest runner Vipin Sharma
2025-07-10 0:20 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 02/15] KVM: selftests: Enable selftests runner to find executables in different path Vipin Sharma
2025-07-09 21:39 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 03/15] KVM: selftests: Add timeout option in selftests runner Vipin Sharma
2025-07-09 21:46 ` Sean Christopherson
2025-07-09 22:25 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 04/15] KVM: selftests: Add option to save selftest runner output to a directory Vipin Sharma
2025-07-09 21:52 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 05/15] KVM: selftests: Run tests concurrently in KVM selftests runner Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 06/15] KVM: selftests: Add a flag to print only test status in KVM Selftests run Vipin Sharma
2025-07-09 21:55 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 07/15] KVM: selftests: Add various print flags to KVM Selftest Runner Vipin Sharma
2025-07-09 22:01 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 08/15] KVM: selftests: Print sticky KVM Selftests Runner status at bottom Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 09/15] KVM: selftests: Add a flag to print only sticky summary in the selftests runner Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 10/15] KVM: selftests: Add flag to suppress all output from Selftest KVM Runner Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 11/15] KVM: selftests: Auto generate default tests for KVM Selftests Runner Vipin Sharma
2025-07-09 23:06 ` Oliver Upton
2025-07-10 0:18 ` Sean Christopherson
2025-06-06 23:56 ` [PATCH v2 12/15] KVM: selftests: Add x86 auto generated test files " Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 13/15] KVM: selftests: Add arm64 " Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 14/15] KVM: selftests: Add s390 " Vipin Sharma
2025-06-06 23:56 ` [PATCH v2 15/15] KVM: selftests: Add riscv " Vipin Sharma
2025-06-09 12:54 ` Andrew Jones
2025-07-09 22:25 ` [PATCH v2 00/15] Add KVM Selftests runner Sean Christopherson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).