[PATCH 1/4] KVM: x86: Handle TIF_NEED_FPU_LOAD in kvm_{load,put}_guest_fpu()

Sean Christopherson
2020-01-17T06:26:59Z

Handle TIF_NEED_FPU_LOAD similar to how fpu__copy() handles the flag
when duplicating FPU state to a new task struct. TIF_NEED_FPU_LOAD can
be set any time control is transferred out of KVM, be it voluntarily,
e.g. if I/O is triggered during a KVM call to get_user_pages, or
involuntarily, e.g. if softirq runs after an IRQ occurs. Therefore,
KVM must account for TIF_NEED_FPU_LOAD whenever it is (potentially)
accessing CPU FPU state.

Fixes: 5f409e20b7945 ("x86/fpu: Defer FPU state load until return to userspace")
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
---
arch/x86/kvm/x86.c | 27 ++++++++++++++++++++++++---
1 file changed, 24 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index cf917139de6b..0c7211491f98 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8476,8 +8476,20 @@ static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu)
{
fpregs_lock();

- copy_fpregs_to_fpstate(vcpu->arch.user_fpu);
- /* PKRU is separately restored in kvm_x86_ops->run. */
+ /*
+ * If userspace's FPU state is not resident in the CPU registers, just
+ * memcpy() from current, else save CPU state directly to user_fpu.
+ */
+ if (test_thread_flag(TIF_NEED_FPU_LOAD))
+ memcpy(&vcpu->arch.user_fpu->state, &current->thread.fpu.state,
+ fpu_kernel_xstate_size);
+ else
+ copy_fpregs_to_fpstate(vcpu->arch.user_fpu);
+
+ /*
+ * Load guest's FPU state to the CPU registers. PKRU is separately
+ * loaded in kvm_x86_ops->run.
+ */
__copy_kernel_to_fpregs(&vcpu->arch.guest_fpu->state,
~XFEATURE_MASK_PKRU);

@@ -8492,7 +8504,16 @@ static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu)
{
fpregs_lock();

- copy_fpregs_to_fpstate(vcpu->arch.guest_fpu);
+ /*
+ * If guest's FPU state is not resident in the CPU registers, just
+ * memcpy() from current, else save CPU state directly to guest_fpu.
+ */
+ if (test_thread_flag(TIF_NEED_FPU_LOAD))
+ memcpy(&vcpu->arch.guest_fpu->state, &current->thread.fpu.state,
+ fpu_kernel_xstate_size);
+ else
+ copy_fpregs_to_fpstate(vcpu->arch.guest_fpu);
+
copy_kernel_to_fpregs(&vcpu->arch.user_fpu->state);

fpregs_mark_activate();
--
2.24.1