KVM causes #GP on XRSTORS

KVM causes #GP on XRSTORS

[Nadav Amit]

Fenghua,

I got KVM (v3.17) crashing on a machine that supports XRSTORS - It appears to get a #GP when it is trying to load the guest FPU.
One reason for the #GP is that XCOMP_BV[63] is zeroed on the guest_fpu, but I am not sure it is the only problem.
Was KVM ever tested with XRSTORS?

Thanks,
Nadav

RE: KVM causes #GP on XRSTORS

[Yu, Fenghua]

> From: Nadav Amit [mailto:nadav.amit@gmail.com]
> Sent: Thursday, November 20, 2014 8:34 AM
> To: Paolo Bonzini; Yu, Fenghua
> Cc: kvm list
> Subject: KVM causes #GP on XRSTORS
> 
> Fenghua,
> 
> I got KVM (v3.17) crashing on a machine that supports XRSTORS - It appears
> to get a #GP when it is trying to load the guest FPU.
> One reason for the #GP is that XCOMP_BV[63] is zeroed on the guest_fpu,
> but I am not sure it is the only problem.
> Was KVM ever tested with XRSTORS?

In dmesg, do you see "xsave: ......... using compacted form"?

If you use kernel "noxsaves", does KVM work fine?

Thanks.

-Fenghua

Re: KVM causes #GP on XRSTORS

[Wanpeng Li]

Hi Nadav,
On Thu, Nov 20, 2014 at 06:34:04PM +0200, Nadav Amit wrote:
>Fenghua,
>
>I got KVM (v3.17) crashing on a machine that supports XRSTORS - It appears to get a #GP when it is trying to load the guest FPU.
>One reason for the #GP is that XCOMP_BV[63] is zeroed on the guest_fpu, but I am not sure it is the only problem.
>Was KVM ever tested with XRSTORS?

Current kvm and qemu use standard format and xsaves/xrstors should use
compact format, in addition, vmx is still not enabled for xsaves/xrstors 
in kvm.

Regards,
Wanpeng Li 

>
>Thanks,
>Nadav
>
>--
>To unsubscribe from this list: send the line "unsubscribe kvm" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: KVM causes #GP on XRSTORS

[Nadav Amit]

> On Nov 21, 2014, at 02:10, Wanpeng Li <wanpeng.li@linux.intel.com> wrote:
> 
> Hi Nadav,
> On Thu, Nov 20, 2014 at 06:34:04PM +0200, Nadav Amit wrote:
>> Fenghua,
>> 
>> I got KVM (v3.17) crashing on a machine that supports XRSTORS - It appears to get a #GP when it is trying to load the guest FPU.
>> One reason for the #GP is that XCOMP_BV[63] is zeroed on the guest_fpu, but I am not sure it is the only problem.
>> Was KVM ever tested with XRSTORS?
> 
> Current kvm and qemu use standard format and xsaves/xrstors should use
> compact format, in addition, vmx is still not enabled for xsaves/xrstors 
> in kvm.

Sorry, but I don’t quite understand. Should KVM work on a machine that supports xsaves/xrstors?
I am not referring to whether KVM exposes the capability to the guest, but to whether KVM should work at all.

Thanks,
Nadav

Re: KVM causes #GP on XRSTORS

[Paolo Bonzini]

On 20/11/2014 17:34, Nadav Amit wrote:
> Fenghua,
> 
> I got KVM (v3.17) crashing on a machine that supports XRSTORS - It appears to get a #GP when it is trying to load the guest FPU.
> One reason for the #GP is that XCOMP_BV[63] is zeroed on the guest_fpu, but I am not sure it is the only problem.
> Was KVM ever tested with XRSTORS?

What is the content of the CPUID[EAX=13,ECX=0] and CPUID[EAX=13,ECX=1]
leaves on the host?

Fenghua, which processors have XSAVEC, which have XGETBV with ECX=1, and
which have XSAVES?  We need to expose this in QEMU, for which I can send
a patch later today or next week (CCing Eduardo for this).

We will also have to uncompact the XSAVE area either in KVM_GET_XSAVE or
in QEMU.  It's probably not hard to do it in the kernel.

Paolo

Re: KVM causes #GP on XRSTORS

[Paolo Bonzini]

On 21/11/2014 15:46, Paolo Bonzini wrote:
> Fenghua, which processors have XSAVEC, which have XGETBV with ECX=1, and
> which have XSAVES?  We need to expose this in QEMU, for which I can send
> a patch later today or next week (CCing Eduardo for this).

Actually no change in QEMU is needed to hide XSAVES; the KVM patch I
just sent should be enough for "-cpu host" to work.

We still need the information on processor support though, in order to
enable the feature with the right -cpu options.  I assume Nadav was
using "-cpu host".

Paolo

Re: KVM causes #GP on XRSTORS

[Wanpeng Li]

Hi Paolo,
On Fri, Nov 21, 2014 at 03:46:55PM +0100, Paolo Bonzini wrote:
>
>
>On 20/11/2014 17:34, Nadav Amit wrote:
>> Fenghua,
>> 
>> I got KVM (v3.17) crashing on a machine that supports XRSTORS - It appears to get a #GP when it is trying to load the guest FPU.
>> One reason for the #GP is that XCOMP_BV[63] is zeroed on the guest_fpu, but I am not sure it is the only problem.
>> Was KVM ever tested with XRSTORS?
>
>What is the content of the CPUID[EAX=13,ECX=0] and CPUID[EAX=13,ECX=1]
>leaves on the host?
>
>Fenghua, which processors have XSAVEC, which have XGETBV with ECX=1, and
>which have XSAVES?  We need to expose this in QEMU, for which I can send

Skylake-client supports XSAVES.

>a patch later today or next week (CCing Eduardo for this).

Actually I do the xsaves work in kvm/qemu recently and the patches are
complete, I will start to test it before send out.

Regards,
Wanpeng Li 

>
>We will also have to uncompact the XSAVE area either in KVM_GET_XSAVE or
>in QEMU.  It's probably not hard to do it in the kernel.
>
>Paolo
>--
>To unsubscribe from this list: send the line "unsubscribe kvm" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: KVM causes #GP on XRSTORS

[Eduardo Habkost]

On Fri, Nov 21, 2014 at 03:46:55PM +0100, Paolo Bonzini wrote:
> 
> 
> On 20/11/2014 17:34, Nadav Amit wrote:
> > Fenghua,
> > 
> > I got KVM (v3.17) crashing on a machine that supports XRSTORS - It appears to get a #GP when it is trying to load the guest FPU.
> > One reason for the #GP is that XCOMP_BV[63] is zeroed on the guest_fpu, but I am not sure it is the only problem.
> > Was KVM ever tested with XRSTORS?
> 
> What is the content of the CPUID[EAX=13,ECX=0] and CPUID[EAX=13,ECX=1]
> leaves on the host?
> 
> Fenghua, which processors have XSAVEC, which have XGETBV with ECX=1, and
> which have XSAVES?  We need to expose this in QEMU, for which I can send
> a patch later today or next week (CCing Eduardo for this).
> 
> We will also have to uncompact the XSAVE area either in KVM_GET_XSAVE or
> in QEMU.  It's probably not hard to do it in the kernel.

Oh, now I see that KVM uses xsave_state(), which uses XSAVEOPT and
XSAVES depending on the host CPU features. I didn't know that.

QEMU already expects the standard format to be part of the KVM_GET_XSAVE
ABI. So this would already cause problems in host CPUs supporting
XSAVES, even if XSAVES is not exposed to guests yet.

-- 
Eduardo