Re: [RFC PATCH v5 0/6] KVM: x86: add per-vCPU exits disable capability

Zhi Wang
2023-01-18T09:13:45Z

On Fri, 13 Jan 2023 22:01:08 +0000
Kechen Lu <kechenl@nvidia.com> wrote:

Hi:

checkpatch.pl throws a lot of warning and errors when I was trying
this series. Can you fix them?

total: 470 errors, 22 warnings, 464 lines checked

> Summary
> ===========
> Introduce support of vCPU-scoped ioctl with KVM_CAP_X86_DISABLE_EXITS
> cap for disabling exits to enable finer-grained VM exits disabling
> on per vCPU scales instead of whole guest. This patch series enabled
> the vCPU-scoped exits control and toggling.
>
> Motivation
> ============
> In use cases like Windows guest running heavy CPU-bound
> workloads, disabling HLT VM-exits could mitigate host sched ctx switch
> overhead. Simply HLT disabling on all vCPUs could bring
> performance benefits, but if no pCPUs reserved for host threads, could
> happened to the forced preemption as host does not know the time to do
> the schedule for other host threads want to run. With this patch, we
> could only disable part of vCPUs HLT exits for one guest, this still
> keeps performance benefits, and also shows resiliency to host stressing
> workload running at the same time.
>
> Performance and Testing
> =========================
> In the host stressing workload experiment with Windows guest heavy
> CPU-bound workloads, it shows good resiliency and having the ~3%
> performance improvement. E.g. Passmark running in a Windows guest
> with this patch disabling HLT exits on only half of vCPUs still
> showing 2.4% higher main score v/s baseline.
>
> Tested everything on AMD machines.
>
> v4->v5 :
> - Drop the usage of KVM request, keep the VM-scoped exits disable
> as the existing design, and only allow per-vCPU settings to
> override the per-VM settings (Sean Christopherson)
> - Refactor the disable exits selftest without introducing any
> new prerequisite patch, tests per-vCPU exits disable and overrides,
> and per-VM exits disable
>
> v3->v4 (Chao Gao) :
> - Use kvm vCPU request KVM_REQ_DISABLE_EXIT to perform the arch
> VMCS updating (patch 5)
> - Fix selftests redundant arguments (patch 7)
> - Merge overlapped fix bits from patch 4 to patch 3
>
> v2->v3 (Sean Christopherson) :
> - Reject KVM_CAP_X86_DISABLE_EXITS if userspace disable MWAIT exits
> when MWAIT is not allowed in guest (patch 3)
> - Make userspace able to re-enable previously disabled exits (patch 4)
> - Add mwait/pause/cstate exits flag toggling instead of only hlt
> exits (patch 5)
> - Add selftests for KVM_CAP_X86_DISABLE_EXITS (patch 7)
>
> v1->v2 (Sean Christopherson) :
> - Add explicit restriction for VM-scoped exits disabling to be called
> before vCPUs creation (patch 1)
> - Use vCPU ioctl instead of 64bit vCPU bitmask (patch 5), and make exits
> disable flags check purely for vCPU instead of VM (patch 2)
>
> Best Regards,
> Kechen
>
> Kechen Lu (3):
> KVM: x86: Move *_in_guest power management flags to vCPU scope
> KVM: x86: add vCPU scoped toggling for disabled exits
> KVM: selftests: Add tests for VM and vCPU cap
> KVM_CAP_X86_DISABLE_EXITS
>
> Sean Christopherson (3):
> KVM: x86: only allow exits disable before vCPUs created
> KVM: x86: Reject disabling of MWAIT interception when not allowed
> KVM: x86: Let userspace re-enable previously disabled exits
>
> Documentation/virt/kvm/api.rst | 8 +-
> arch/x86/include/asm/kvm-x86-ops.h | 1 +
> arch/x86/include/asm/kvm_host.h | 7 +
> arch/x86/kvm/cpuid.c | 4 +-
> arch/x86/kvm/lapic.c | 7 +-
> arch/x86/kvm/svm/nested.c | 4 +-
> arch/x86/kvm/svm/svm.c | 42 +-
> arch/x86/kvm/vmx/vmx.c | 53 +-
> arch/x86/kvm/x86.c | 69 ++-
> arch/x86/kvm/x86.h | 16 +-
> include/uapi/linux/kvm.h | 4 +-
> tools/testing/selftests/kvm/Makefile | 1 +
> .../selftests/kvm/x86_64/disable_exits_test.c | 457 ++++++++++++++++++
> 13 files changed, 626 insertions(+), 47 deletions(-)
> create mode 100644 tools/testing/selftests/kvm/x86_64/disable_exits_test.c
>