Re: [RFC PATCH v5 0/6] KVM: x86: add per-vCPU exits disable capability

[Zhi Wang <zhi.wang.linux@gmail.com>]

On Fri, 13 Jan 2023 22:01:08 +0000
Kechen Lu <kechenl@nvidia.com> wrote:

Hi:

checkpatch.pl throws a lot of warning and errors when I was trying
this series. Can you fix them?

total: 470 errors, 22 warnings, 464 lines checked

> Summary
> ===========
> Introduce support of vCPU-scoped ioctl with KVM_CAP_X86_DISABLE_EXITS
> cap for disabling exits to enable finer-grained VM exits disabling
> on per vCPU scales instead of whole guest. This patch series enabled
> the vCPU-scoped exits control and toggling.
> 
> Motivation
> ============
> In use cases like Windows guest running heavy CPU-bound
> workloads, disabling HLT VM-exits could mitigate host sched ctx switch
> overhead. Simply HLT disabling on all vCPUs could bring
> performance benefits, but if no pCPUs reserved for host threads, could
> happened to the forced preemption as host does not know the time to do
> the schedule for other host threads want to run. With this patch, we
> could only disable part of vCPUs HLT exits for one guest, this still
> keeps performance benefits, and also shows resiliency to host stressing
> workload running at the same time.
> 
> Performance and Testing
> =========================
> In the host stressing workload experiment with Windows guest heavy
> CPU-bound workloads, it shows good resiliency and having the ~3%
> performance improvement. E.g. Passmark running in a Windows guest
> with this patch disabling HLT exits on only half of vCPUs still
> showing 2.4% higher main score v/s baseline.
> 
> Tested everything on AMD machines.
> 
> v4->v5 :
> - Drop the usage of KVM request, keep the VM-scoped exits disable
>   as the existing design, and only allow per-vCPU settings to
>   override the per-VM settings (Sean Christopherson)
> - Refactor the disable exits selftest without introducing any
>   new prerequisite patch, tests per-vCPU exits disable and overrides,
>   and per-VM exits disable
> 
> v3->v4 (Chao Gao) :
> - Use kvm vCPU request KVM_REQ_DISABLE_EXIT to perform the arch
>   VMCS updating (patch 5)
> - Fix selftests redundant arguments (patch 7)
> - Merge overlapped fix bits from patch 4 to patch 3
> 
> v2->v3 (Sean Christopherson) :
> - Reject KVM_CAP_X86_DISABLE_EXITS if userspace disable MWAIT exits
>   when MWAIT is not allowed in guest (patch 3)
> - Make userspace able to re-enable previously disabled exits (patch 4)
> - Add mwait/pause/cstate exits flag toggling instead of only hlt
>   exits (patch 5)
> - Add selftests for KVM_CAP_X86_DISABLE_EXITS (patch 7)
> 
> v1->v2 (Sean Christopherson) :
> - Add explicit restriction for VM-scoped exits disabling to be called
>   before vCPUs creation (patch 1)
> - Use vCPU ioctl instead of 64bit vCPU bitmask (patch 5), and make exits
>   disable flags check purely for vCPU instead of VM (patch 2)
> 
> Best Regards,
> Kechen
> 
> Kechen Lu (3):
>   KVM: x86: Move *_in_guest power management flags to vCPU scope
>   KVM: x86: add vCPU scoped toggling for disabled exits
>   KVM: selftests: Add tests for VM and vCPU cap
>     KVM_CAP_X86_DISABLE_EXITS
> 
> Sean Christopherson (3):
>   KVM: x86: only allow exits disable before vCPUs created
>   KVM: x86: Reject disabling of MWAIT interception when not allowed
>   KVM: x86: Let userspace re-enable previously disabled exits
> 
>  Documentation/virt/kvm/api.rst                |   8 +-
>  arch/x86/include/asm/kvm-x86-ops.h            |   1 +
>  arch/x86/include/asm/kvm_host.h               |   7 +
>  arch/x86/kvm/cpuid.c                          |   4 +-
>  arch/x86/kvm/lapic.c                          |   7 +-
>  arch/x86/kvm/svm/nested.c                     |   4 +-
>  arch/x86/kvm/svm/svm.c                        |  42 +-
>  arch/x86/kvm/vmx/vmx.c                        |  53 +-
>  arch/x86/kvm/x86.c                            |  69 ++-
>  arch/x86/kvm/x86.h                            |  16 +-
>  include/uapi/linux/kvm.h                      |   4 +-
>  tools/testing/selftests/kvm/Makefile          |   1 +
>  .../selftests/kvm/x86_64/disable_exits_test.c | 457 ++++++++++++++++++
>  13 files changed, 626 insertions(+), 47 deletions(-)
>  create mode 100644 tools/testing/selftests/kvm/x86_64/disable_exits_test.c
>