Re: [PATCH 02/19] x86/fpu: Prepare KVM for dynamically enabled states

Paolo Bonzini
2021-12-13T12:45:57Z

On 12/13/21 13:00, Thomas Gleixner wrote:
> On Mon, Dec 13 2021 at 10:12, Paolo Bonzini wrote:
>> On 12/8/21 01:03, Yang Zhong wrote:
>>> - user_xfeatures
>>>
>>> Track which features are currently enabled for the vCPU
>>
>> Please rename to alloc_xfeatures
>
> That name makes no sense at all. This has nothing to do with alloc.

Isn't that the features for which space is currently allocated?

fpstate_realloc does

+ if (guest_fpu) {
+ newfps->is_guest = true;
+ newfps->is_confidential = curfps->is_confidential;
+ guest_fpu->user_xfeatures |= xfeatures;
+ }
+

and kvm_check_guest_realloc_fpstate does

+ if ((guest_fpu->user_xfeatures & request) != request) {
+ vcpu->arch.guest_fpu.realloc_request |= request;
+ return true;
+ }

Reading "user_xfeatures" in there is cryptic, it seems like it's
something related to the userspace thread or group that has invoked the
KVM ioctl. If it's renamed to alloc_xfeatures, then this:

+ missing = request & ~guest_fpu->alloc_xfeatures;
+ if (missing) {
+ vcpu->arch.guest_fpu.realloc_request |= missing;
+ return true;
+ }

makes it obvious that the allocation is for features that are requested
but haven't been allocated in the xstate yet.

>>> - user_perm
>>>
>>> Copied from guest_perm of the group leader thread. The first
>>> vCPU which does the copy locks the guest_perm
>>
>> Please rename to perm_xfeatures.
>
> All of that is following the naming conventions in the FPU code related
> to permissions etc.

perm or guest_perm is just fine as well; but user_perm is not (there's
no preexisting reference to user_perm in the code, in fact, as far as I
can see).

>>> - realloc_request
>>>
>>> KVM sets this field to request dynamically-enabled features
>>> which require reallocation of @fpstate
>>
>> This field should be in vcpu->arch, and there is no need for
>> fpu_guest_realloc_fpstate. Rename __xfd_enable_feature to
>> fpu_enable_xfd_feature and add it to the public API, then just do
>>
>> if (unlikely(vcpu->arch.xfd_realloc_request)) {
>> u64 request = vcpu->arch.xfd_realloc_request;
>> ret = fpu_enable_xfd(request, enter_guest);
>> }
>>
>> to kvm_put_guest_fpu.
>
> Why? Yet another export of FPU internals just because?

It's one function more and one field less. I prefer another export of
FPU internals, to a write to a random field with undocumented invariants.

For example, why WARN_ON_ONCE if enter_guest == true? If you enter the
guest after the host has restored MSR_IA32_XFD with KVM_SET_MSR, the
WARN_ON_ONCE can actually fire. It would be just fine to skip the
reallocation until enter_guest == false, which is you actually XSAVE
into the guest_fpu.

As an aside, realloc_request (if it survives, see below) shouldn't be
added until patch 6.

> Also what clears the reallocation request and what is the @enter_guest
> argument supposed to help with?

Nothing---make it

if (unlikely(vcpu->arch.xfd_realloc_request)) {
u64 request = vcpu->arch.xfd_realloc_request;
ret = fpu_enable_xfd(request, &vcpu->arch.guest_fpu);
if (!ret)
vcpu->arch.xfd_realloc_request = 0;
}

but in fact, I'm not sure why the request has to be delayed at all. The
obvious implementation of a write to XFD, after all the validity checks,
is just

/* This function just calls xfd_enable_feature. */
r = fpu_guest_alloc_xfeatures(&vcpu->arch.guest_fpu,
vcpu->arch.xcr0 & ~xfd);
/*
* An error means that userspace has screwed up by not doing
* arch_prctl(ARCH_REQ_XCOMP_GUEST_PERM). If we are here coming
* from a ioctl fail it, if the guest has done WRMSR or XSETBV
* it will inject a #GP.
*/
if (r < 0)
return 1;

vcpu->arch.xfd = xfd;

with none of the kvm_check_guest_realloc_fpstate or KVM_EXIT_FPU_REALLOC
business. No field and a simple, self-contained external API. The same
code works for __kvm_set_xcr as well, just with "xcr0 & ~vcpu->arch.xfd"
as the second argument instead.

(Maybe I'm missing why KVM_EXIT_FPU_REALLOC is needed, but I'm not
ashamed to say that, given that new userspace API was added with
documentation or tests. The only comment in the code is:

+ /*
+ * Check if fpstate reallocate is required. If yes, then
+ * let the fpu core do reallocation and update xfd;
+ * otherwise, update xfd here.
+ */
+ if (kvm_check_guest_realloc_fpstate(vcpu, data)) {
+ vcpu->run->exit_reason = KVM_EXIT_FPU_REALLOC;
+ vcpu->arch.complete_userspace_io =
+ kvm_skip_emulated_instruction;
+ return KVM_MSR_RET_USERSPACE;
+ }
+

which has nothing to do with the actual content of either
kvm_check_guest_realloc_fpstate or the "then" branch. Userspace can
just do ARCH_REQ_XCOMP_GUEST_PERM based on the guest CPUID, before
KVM_SET_CPUID2, KVM_SET_MSR or KVM_SET_XCR).

Paolo