[PATCH for-5.0 0/4] spapr: Use less XIVE HW resources in KVM

[PATCH for-5.0 0/4] spapr: Use less XIVE HW resources in KVM

[Greg Kurz]

On POWER9 systems, the XICS-on-XIVE and XIVE KVM devices currently
allocate a bunch of VPs in the XIVE HW to accomodate the highest
VCPU id that may be possibly used in a VM. This limits the number
of VMs that can run with an in-kernel interrupt controller to 63
per POWER9 chip, irrespectively of its number of HW threads, eg.
up to 96 on a POWER9 Nimbus socket. This is an unfortunate waste
of scarce HW resources since a typical VM doesn't need that much
VPs to run.

This series exploits new attributes of the XICS-on-XIVE and XIVE
KVM devices that allow userspace to tune the numbers of VPs it
really needs.

Patches 1 to 3 are preliminary work to teach the XICS and XIVE
backends about the range of needed VCPU ids, according to the
maximum number of VCPUs specified in the QEMU command line.

Patch 5 and 6 do the actual work of configuring the KVM devices,
based on new defines brought by a patch 4. RFC since the patches
for KVM are still being discussed on the kvm-ppc list:

https://patchwork.ozlabs.org/project/kvm-ppc/list/?series=132910

As a bonus, patch 7 allows the latest machine type to automatically
set int KVM the guest core stride (VSMT) to be equal to the number
of threads per core (-smp threads=N). This makes VCPU ids contiguous
and allows to reduce the VP consumption even more.

Both KVM and QEMU changes are available here:
https://github.com/gkurz/linux/commits/xive-nr-servers-5.3
https://github.com/gkurz/qemu/commits/xive-nr-servers-for-4.2
---

Greg Kurz (4):
      linux-headers: Update
      spapr: Pass the maximum number of vCPUs to the KVM interrupt controller
      spapr/xics: Configure number of servers in KVM
      spapr/xive: Configure number of servers in KVM


 hw/intc/spapr_xive.c                         |    6 ++++--
 hw/intc/spapr_xive_kvm.c                     |   26 +++++++++++++++++++++++---
 hw/intc/xics_kvm.c                           |   24 +++++++++++++++++++++---
 hw/intc/xics_spapr.c                         |    5 +++--
 hw/ppc/spapr_irq.c                           |    8 +++++---
 include/hw/ppc/spapr_irq.h                   |   10 ++++++++--
 include/hw/ppc/spapr_xive.h                  |    3 ++-
 include/hw/ppc/xics_spapr.h                  |    3 ++-
 include/standard-headers/linux/ethtool.h     |    6 ++++++
 include/standard-headers/linux/virtio_ring.h |    2 +-
 linux-headers/asm-arm/kvm.h                  |    3 ++-
 linux-headers/asm-arm64/kvm.h                |    5 ++++-
 linux-headers/asm-mips/unistd_n32.h          |    1 +
 linux-headers/asm-mips/unistd_n64.h          |    1 +
 linux-headers/asm-mips/unistd_o32.h          |    1 +
 linux-headers/asm-powerpc/kvm.h              |    3 +++
 linux-headers/linux/kvm.h                    |   11 +++++++++++
 linux-headers/linux/psp-sev.h                |    3 +++
 18 files changed, 101 insertions(+), 20 deletions(-)

Re: [PATCH for-5.0 0/4] spapr: Use less XIVE HW resources in KVM

[Greg Kurz]

Oops clumsy fingers, please ignore that ... :)

On Tue, 26 Nov 2019 16:50:38 +0100
Greg Kurz <groug@kaod.org> wrote:

> On POWER9 systems, the XICS-on-XIVE and XIVE KVM devices currently
> allocate a bunch of VPs in the XIVE HW to accomodate the highest
> VCPU id that may be possibly used in a VM. This limits the number
> of VMs that can run with an in-kernel interrupt controller to 63
> per POWER9 chip, irrespectively of its number of HW threads, eg.
> up to 96 on a POWER9 Nimbus socket. This is an unfortunate waste
> of scarce HW resources since a typical VM doesn't need that much
> VPs to run.
> 
> This series exploits new attributes of the XICS-on-XIVE and XIVE
> KVM devices that allow userspace to tune the numbers of VPs it
> really needs.
> 
> Patches 1 to 3 are preliminary work to teach the XICS and XIVE
> backends about the range of needed VCPU ids, according to the
> maximum number of VCPUs specified in the QEMU command line.
> 
> Patch 5 and 6 do the actual work of configuring the KVM devices,
> based on new defines brought by a patch 4. RFC since the patches
> for KVM are still being discussed on the kvm-ppc list:
> 
> https://patchwork.ozlabs.org/project/kvm-ppc/list/?series=132910
> 
> As a bonus, patch 7 allows the latest machine type to automatically
> set int KVM the guest core stride (VSMT) to be equal to the number
> of threads per core (-smp threads=N). This makes VCPU ids contiguous
> and allows to reduce the VP consumption even more.
> 
> Both KVM and QEMU changes are available here:
> https://github.com/gkurz/linux/commits/xive-nr-servers-5.3
> https://github.com/gkurz/qemu/commits/xive-nr-servers-for-4.2
> ---
> 
> Greg Kurz (4):
>       linux-headers: Update
>       spapr: Pass the maximum number of vCPUs to the KVM interrupt controller
>       spapr/xics: Configure number of servers in KVM
>       spapr/xive: Configure number of servers in KVM
> 
> 
>  hw/intc/spapr_xive.c                         |    6 ++++--
>  hw/intc/spapr_xive_kvm.c                     |   26 +++++++++++++++++++++++---
>  hw/intc/xics_kvm.c                           |   24 +++++++++++++++++++++---
>  hw/intc/xics_spapr.c                         |    5 +++--
>  hw/ppc/spapr_irq.c                           |    8 +++++---
>  include/hw/ppc/spapr_irq.h                   |   10 ++++++++--
>  include/hw/ppc/spapr_xive.h                  |    3 ++-
>  include/hw/ppc/xics_spapr.h                  |    3 ++-
>  include/standard-headers/linux/ethtool.h     |    6 ++++++
>  include/standard-headers/linux/virtio_ring.h |    2 +-
>  linux-headers/asm-arm/kvm.h                  |    3 ++-
>  linux-headers/asm-arm64/kvm.h                |    5 ++++-
>  linux-headers/asm-mips/unistd_n32.h          |    1 +
>  linux-headers/asm-mips/unistd_n64.h          |    1 +
>  linux-headers/asm-mips/unistd_o32.h          |    1 +
>  linux-headers/asm-powerpc/kvm.h              |    3 +++
>  linux-headers/linux/kvm.h                    |   11 +++++++++++
>  linux-headers/linux/psp-sev.h                |    3 +++
>  18 files changed, 101 insertions(+), 20 deletions(-)
> 
>