Re: [PATCH 5/5] Documentation: kvm: introduce "VM plane" concept

[Sean Christopherson <seanjc@google.com>]

On Wed, Oct 23, 2024, Paolo Bonzini wrote:
> @@ -6398,6 +6415,46 @@ the capability to be present.
>  `flags` must currently be zero.
>  
>  
> +.. _KVM_CREATE_PLANE:
> +
> +4.144 KVM_CREATE_PLANE
> +----------------------
> +
> +:Capability: KVM_CAP_PLANE
> +:Architectures: none
> +:Type: vm ioctl
> +:Parameters: plane id
> +:Returns: a VM fd that can be used to control the new plane.
> +
> +Creates a new *plane*, i.e. a separate privilege level for the
> +virtual machine.  Each plane has its own memory attributes,
> +which can be used to enable more restricted permissions than
> +what is allowed with ``KVM_SET_USER_MEMORY_REGION``.
> +
> +Each plane has a numeric id that is used when communicating
> +with KVM through the :ref:`kvm_run <kvm_run>` struct.  While
> +KVM is currently agnostic to whether low ids are more or less
> +privileged, it is expected that this will not always be the
> +case in the future.  For example KVM in the future may use
> +the plane id when planes are supported by hardware (as is the
> +case for VMPLs in AMD), or if KVM supports accelerated plane
> +switch operations (as might be the case for Hyper-V VTLs).
> +
> +4.145 KVM_CREATE_VCPU_PLANE
> +---------------------------
> +
> +:Capability: KVM_CAP_PLANE
> +:Architectures: none
> +:Type: vm ioctl (non default plane)
> +:Parameters: vcpu file descriptor for the default plane
> +:Returns: a vCPU fd that can be used to control the new plane
> +          for the vCPU.
> +
> +Adds a vCPU to a plane; the new vCPU's id comes from the vCPU
> +file descriptor that is passed in the argument.  Note that
> + because of how the API is defined, planes other than plane 0
> +can only have a subset of the ids that are available in plane 0.

Hmm, was there a reason why we decided to add KVM_CREATE_VCPU_PLANE, as opposed
to having KVM_CREATE_PLANE create vCPUs?  IIRC, we talked about being able to
provide the new FD, but that would be easy enough to handle in KVM_CREATE_PLANE,
e.g. with an array of fds.

E.g. is the expectation that userspace will create all planes before creating
any vCPUs?

My concern with relying on userspace to create vCPUs is that it will mean KVM
will need to support, or at least not blow up on, VMs with multiple planes, but
only a subset of vCPUs at planes > 0.  Given the snafus with vcpus_array, it's
not at all hard to imagine scenarios where KVM tries to access a NULL vCPU in
a different plane.