Re: [PATCH] kvm: x86: Increase MAX_VCPUS to 710

[Vitaly Kuznetsov <vkuznets@redhat.com>]

Eduardo Habkost <ehabkost@redhat.com> writes:

> Support for 710 VCPUs has been tested by Red Hat since RHEL-8.4.
> Increase KVM_MAX_VCPUS and KVM_SOFT_MAX_VCPUS to 710.
>
> For reference, visible effects of changing KVM_MAX_VCPUS are:
> - KVM_CAP_MAX_VCPUS and KVM_CAP_NR_VCPUS will now return 710 (of course)
> - Default value for CPUID[HYPERV_CPUID_IMPLEMENT_LIMITS (00x40000005)].EAX
>   will now be 710
> - Bitmap stack variables that will grow:
>   - At kvm_hv_flush_tlb()  kvm_hv_send_ipi():
>     - Sparse VCPU bitmap (vp_bitmap) will be 96 bytes long
>     - vcpu_bitmap will be 92 bytes long
>   - vcpu_bitmap at bioapic_write_indirect() will be 92 bytes long
>     once patch "KVM: x86: Fix stack-out-of-bounds memory access
>     from ioapic_write_indirect()" is applied
>
> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> ---
>  arch/x86/include/asm/kvm_host.h | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index af6ce8d4c86a..f76fae42bf45 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -37,8 +37,8 @@
>  
>  #define __KVM_HAVE_ARCH_VCPU_DEBUGFS
>  
> -#define KVM_MAX_VCPUS 288
> -#define KVM_SOFT_MAX_VCPUS 240
> +#define KVM_MAX_VCPUS 710

Out of pure curiosity, where did 710 came from? Is this some particular
hardware which was used for testing (weird number btw). Should we maybe
go to e.g. 1024 for the sake of the beauty of powers of two? :-)

> +#define KVM_SOFT_MAX_VCPUS 710

Do we really need KVM_SOFT_MAX_VCPUS which is equal to KVM_MAX_VCPUS?

Reading 

commit 8c3ba334f8588e1d5099f8602cf01897720e0eca
Author: Sasha Levin <levinsasha928@gmail.com>
Date:   Mon Jul 18 17:17:15 2011 +0300

    KVM: x86: Raise the hard VCPU count limit

the idea behind KVM_SOFT_MAX_VCPUS was to allow developers to test high
vCPU numbers without claiming such configurations as supported.

I have two alternative suggestions:
1) Drop KVM_SOFT_MAX_VCPUS completely.
2) Raise it to a higher number (e.g. 2048)

>  #define KVM_MAX_VCPU_ID 1023

1023 may not be enough now. I rememeber there was a suggestion to make
max_vcpus configurable via module parameter and this question was
raised:

https://lore.kernel.org/lkml/878s292k75.fsf@vitty.brq.redhat.com/

TL;DR: to support EPYC-like topologies we need to keep
 KVM_MAX_VCPU_ID = 4 * KVM_MAX_VCPUS

>  /* memory slots that are not exposed to userspace */
>  #define KVM_PRIVATE_MEM_SLOTS 3

-- 
Vitaly