Re: [PATCH v3] kvm/fpu: Enable fully eager restore kvm FPU

[Marcelo Tosatti <mtosatti@redhat.com>]

On Mon, Sep 17, 2012 at 02:07:43AM +0000, Hao, Xudong wrote:
> > -----Original Message-----
> > From: Avi Kivity [mailto:avi@redhat.com]
> > Sent: Friday, September 14, 2012 12:40 AM
> > To: Marcelo Tosatti
> > Cc: Hao, Xudong; kvm@vger.kernel.org; Zhang, Xiantao
> > Subject: Re: [PATCH v3] kvm/fpu: Enable fully eager restore kvm FPU
> > 
> > On 09/13/2012 07:29 PM, Marcelo Tosatti wrote:
> > > On Thu, Sep 13, 2012 at 01:26:36PM -0300, Marcelo Tosatti wrote:
> > >> On Wed, Sep 12, 2012 at 04:10:24PM +0800, Xudong Hao wrote:
> > >> > Enable KVM FPU fully eager restore, if there is other FPU state which isn't
> > >> > tracked by CR0.TS bit.
> > >> >
> > >> > v3 changes from v2:
> > >> > - Make fpu active explicitly while guest xsave is enabling and non-lazy
> > xstate bit
> > >> > exist.
> > >>
> > >> How about a "guest_xcr0_can_lazy_saverestore" bool to control this?
> > >> It only needs to be updated when guest xcr0 is updated.
> > >>
> > >> That seems cleaner. Avi?
> > >
> > > Reasoning below.
> > >
> > >> > v2 changes from v1:
> > >> > - Expand KVM_XSTATE_LAZY to 64 bits before negating it.
> > >> >
> > >> > Signed-off-by: Xudong Hao <xudong.hao@intel.com>
> > >> > ---
> > >> >  arch/x86/include/asm/kvm.h |    4 ++++
> > >> >  arch/x86/kvm/vmx.c         |    2 ++
> > >> >  arch/x86/kvm/x86.c         |   15 ++++++++++++++-
> > >> >  3 files changed, 20 insertions(+), 1 deletions(-)
> > >> >
> > >> > diff --git a/arch/x86/include/asm/kvm.h b/arch/x86/include/asm/kvm.h
> > >> > index 521bf25..4c27056 100644
> > >> > --- a/arch/x86/include/asm/kvm.h
> > >> > +++ b/arch/x86/include/asm/kvm.h
> > >> > @@ -8,6 +8,8 @@
> > >> >
> > >> >  #include <linux/types.h>
> > >> >  #include <linux/ioctl.h>
> > >> > +#include <asm/user.h>
> > >> > +#include <asm/xsave.h>
> > >> >
> > >> >  /* Select x86 specific features in <linux/kvm.h> */
> > >> >  #define __KVM_HAVE_PIT
> > >> > @@ -30,6 +32,8 @@
> > >> >  /* Architectural interrupt line count. */
> > >> >  #define KVM_NR_INTERRUPTS 256
> > >> >
> > >> > +#define KVM_XSTATE_LAZY	(XSTATE_FP | XSTATE_SSE | XSTATE_YMM)
> > >> > +
> > >> >  struct kvm_memory_alias {
> > >> >  	__u32 slot;  /* this has a different namespace than memory slots */
> > >> >  	__u32 flags;
> > >> > diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> > >> > index 248c2b4..853e875 100644
> > >> > --- a/arch/x86/kvm/vmx.c
> > >> > +++ b/arch/x86/kvm/vmx.c
> > >> > @@ -3028,6 +3028,8 @@ static void vmx_set_cr0(struct kvm_vcpu *vcpu,
> > unsigned long cr0)
> > >> >
> > >> >  	if (!vcpu->fpu_active)
> > >> >  		hw_cr0 |= X86_CR0_TS | X86_CR0_MP;
> > >> > +	else
> > >> > +		hw_cr0 &= ~(X86_CR0_TS | X86_CR0_MP);
> > >> >
> > >> >  	vmcs_writel(CR0_READ_SHADOW, cr0);
> > >> >  	vmcs_writel(GUEST_CR0, hw_cr0);
> > >> > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> > >> > index 20f2266..183cf60 100644
> > >> > --- a/arch/x86/kvm/x86.c
> > >> > +++ b/arch/x86/kvm/x86.c
> > >> > @@ -560,6 +560,8 @@ int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32
> > index, u64 xcr)
> > >> >  		return 1;
> > >> >  	if (xcr0 & ~host_xcr0)
> > >> >  		return 1;
> > >> > +	if (xcr0 & ~((u64)KVM_XSTATE_LAZY))
> > >> > +		vcpu->fpu_active = 1;
> > >
> > > This is confusing. The variable allows to decrease the number of places
> > > the decision is made.
> > 
> > Better to have a helper function (lazy_fpu_allowed(), for example).
> > Variables raise the question of whether they are maintained correctly.
> > 
> 
> I realized to modifying the fpu_active variable is incorrect, it must update exception bitmap.
> To avoid the cr0 and xcrs setting order for live migrate case, how about calling fpu_activate() in kvm_set_xcr()? I can add code comments in this function calling.

The objective of the change is to disable lazy fpu loading (that is,
host fpu loaded in guest and vice-versa), when some bit except the
initial tree bits set in guest XCR0 (initial tree being XSTATE_FP|XSTATE_SSE|
XSTATE_YMM). Yes?

If i get that right, then the suggestion seems to be:

static bool lazy_fpu_allowed()
{
	return (vcpu->arch.xcr0 & ~((u64)KVM_XSTATE_LAZY));
}

On guest entry:
        if (!lazy_fpu_allowed(vcpu)) 
                kvm_x86_ops->fpu_activate(vcpu);
        if (vcpu->fpu_active)
                kvm_load_guest_fpu(vcpu);


Does that make sense?

> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index be6d549..e4646d9 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -574,6 +574,9 @@ int kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr)
>         kvm_inject_gp(vcpu, 0);
>         return 1;
>     }
> +   if (xcr & ~((u64)KVM_XSTATE_LAZY))
> +       /* Allow fpu eager restore */
> +       kvm_x86_ops->fpu_activate(vcpu);
>     return 0;
>  }
> 
> Thanks,
> -Xudong
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html