Re: [PATCH RFC V3 17/29] arm/virt: Release objects for *disabled* possible vCPUs after init

[Gavin Shan <gshan@redhat.com>]

Hi Salil,

On 8/21/24 2:40 AM, Salil Mehta wrote:
> 
> I don’t understand this clearly. Are  you suggesting to reuse only single
> vCPU object to initialize all KVM vCPUs not yet plugged? If yes, then
> I'm not sure what do we gain here by adding this complexity? It does
> not consume time or resources because we are not realizing any of these
> vCPU object in any case.
> 

First of all, it seems we have different names and terms for those cold-booted vCPUs
and hotpluggable vCPUs. For example, vCPU-0 and vCPU-1 are cold-booted vCPUs while
vCPU-2 and vCPU-3 are hotpluggable vCPUs when we have '-smp maxcpus=4,cpus=2'. Lets
stick to convention and terms for easier discussion.

The idea is avoid instantiating hotpluggable vCPUs in virtmach_init() and released in the
same function for those hotpluggable vCPUs. As I can understand, those hotpluggable vCPU
instances are serving for two purposes: (1) Relax the constraint that all vCPU's (kvm)
file descriptor have to be created and populated; (2) Help to instantiate and realize
GICv3 object.

For (1), I don't think we have to instantiate those hotpluggable vCPUs at all. In the above
example where we have command line '-smp maxcpus=4,cpus=2', it's unnecessary to instantiate
vCPU-3 and vCPU-4 to create and populate their KVM file descriptors. A vCPU's KVM file
descriptor is create and populated by the following ioctls and function calls. When the first
vCPU (vCPU-0) is realized, the property corresponding to "&init" is fixed for all vCPUs. It
means all vCPUs have same properties except the "vcpu_index".

   ioctl(vm-fd,   KVM_CREATE_VCPU,   vcpu_index);
   ioctl(vcpu-fd, KVM_ARM_VCPU_INIT, &init);
   kvm_park_vcpu(cs);

A vCPU's properties are determined by two sources and both are global. It means all vCPUs
should have same properties: (a) Feature registers returned from the host. The function
kvm_arm_get_host_cpu_features() is called for once, meaning this source is same to all vCPUs;
(b) The parameters provided by user through '-cpu host,sve=off' are translated to global
properties and applied to all vCPUs when they're instantiated.

       (a)                                            (b)

   aarch64_host_initfn                          qemu_init
   kvm_arm_set_cpu_features_from_host           parse_cpu_option
     kvm_arm_get_host_cpu_features              cpu_common_parse_features
                                                qdev_prop_register_global
                                                  :
                                                device_post_init
                                                qdev_prop_set_globals

For (2), I'm still looking into the GICv3 code for better understanding. Until now, I don't
see we need the instantiated hotpluggable vCPUs either. For example, the redistributor regions
can be exposed based on 'maxcpus' instead of 'cpus'. The IRQ connection and teardown can be
dynamically done by connecting the board with GICv3 through callbacks in ARMGICv3CommonClass.
The connection between GICv3CPUState and CPUARMState also can be done dynamically.

Thanks,
Gavin