Re: [PATCH] target/ppc: handle vcpu hotplug failure gracefully

[Harsh Prateek Bora <harshpb@linux.ibm.com>]

+ qemu-devel

On 4/23/24 11:40, Harsh Prateek Bora wrote:
> On ppc64, the PowerVM hypervisor runs with limited memory and a VCPU
> creation during hotplug may fail during kvm_ioctl for KVM_CREATE_VCPU,
> leading to termination of guest since errp is set to &error_fatal while
> calling kvm_init_vcpu. This unexpected behaviour can be avoided by
> pre-creating vcpu and parking it on success or return error otherwise.
> This enables graceful error delivery for any vcpu hotplug failures while
> the guest can keep running.
> 
> Based on api refactoring to create/park vcpus introduced in 1/8 of patch series:
> https://lore.kernel.org/qemu-devel/20240312020000.12992-2-salil.mehta@huawei.com/
> 
> Tested OK by repeatedly doing a hotplug/unplug of vcpus as below:
> 
>   #virsh setvcpus hotplug 40
>   #virsh setvcpus hotplug 70
> error: internal error: unable to execute QEMU command 'device_add':
> kvmppc_cpu_realize: vcpu hotplug failed with -12
> 
> Reported-by: Anushree Mathur <anushree.mathur@linux.vnet.ibm.com>
> Suggested-by: Shivaprasad G Bhat <sbhat@linux.ibm.com>
> Suggested-by: Vaibhav Jain <vaibhav@linux.ibm.com>
> Signed-off by: Harsh Prateek Bora <harshpb@linux.ibm.com>
> ---
> ---
>   target/ppc/kvm.c | 42 ++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 42 insertions(+)
> 
> diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
> index 8231feb2d4..c887f6dfa0 100644
> --- a/target/ppc/kvm.c
> +++ b/target/ppc/kvm.c
> @@ -48,6 +48,8 @@
>   #include "qemu/mmap-alloc.h"
>   #include "elf.h"
>   #include "sysemu/kvm_int.h"
> +#include "sysemu/kvm.h"
> +#include "hw/core/accel-cpu.h"
>   
>   #define PROC_DEVTREE_CPU      "/proc/device-tree/cpus/"
>   
> @@ -2339,6 +2341,43 @@ static void alter_insns(uint64_t *word, uint64_t flags, bool on)
>       }
>   }
>   
> +static int max_cpu_index = 0;
> +
> +static bool kvmppc_cpu_realize(CPUState *cs, Error **errp)
> +{
> +    int ret;
> +
> +    cs->cpu_index = max_cpu_index++;
> +
> +    POWERPC_CPU(cs)->vcpu_id = cs->cpu_index;
> +
> +    if (cs->parent_obj.hotplugged) {
> +        /* create and park to fail gracefully in case vcpu hotplug fails */
> +        ret = kvm_create_vcpu(cs);
> +        if (!ret) {
> +            kvm_park_vcpu(cs);
> +        } else {
> +            max_cpu_index--;
> +            error_setg(errp, "%s: vcpu hotplug failed with %d",
> +                             __func__, ret);
> +            return false;
> +        }
> +    }
> +    return true;
> +}
> +
> +static void kvmppc_cpu_unrealize(CPUState *cpu)
> +{
> +    if (POWERPC_CPU(cpu)->vcpu_id == (max_cpu_index - 1)) {
> +    /* only reclaim vcpuid if its the last one assigned
> +     * as reclaiming random vcpuid for parked vcpus may lead
> +     * to unexpected behaviour due to an existing kernel bug
> +     * when drc_index doesnt get reclaimed as expected.
> +     */
> +        max_cpu_index--;
> +    }
> +}
> +
>   static void kvmppc_host_cpu_class_init(ObjectClass *oc, void *data)
>   {
>       PowerPCCPUClass *pcc = POWERPC_CPU_CLASS(oc);
> @@ -2963,4 +3002,7 @@ bool kvm_arch_cpu_check_are_resettable(void)
>   
>   void kvm_arch_accel_class_init(ObjectClass *oc)
>   {
> +    AccelClass *ac = ACCEL_CLASS(oc);
> +    ac->cpu_common_realize = kvmppc_cpu_realize;
> +    ac->cpu_common_unrealize = kvmppc_cpu_unrealize;
>   }