Re: [Xen-devel] [PATCH 10/24] xen: mask XSAVE from cpuid

[Jeremy Fitzhardinge <jeremy@goop.org>]

H. Peter Anvin wrote:
> Jeremy Fitzhardinge wrote:
>   
>> Jan Beulich wrote:
>>     
>>> As pointed out on an earlier thread, it seems inappropriate to do probing
>>> like this when there is a cpuid feature flag (osxsave) that can be
>>> used to
>>> determine whether XSAVE can be used. And even without that flag,
>>> simply reading CR4 and checking whether osxsave is set there would
>>> suffice. This is under the assumption that Xen's to-be-done
>>> implementation
>>> of XSAVE support would match that of FXSAVE (Xen turns its support on
>>> unconditionally and for all [pv] guests).
>>>       
>> I didn't want to make too many assumptions about how Xen's XSAVE support
>> would look.  In particular, I thought it might virtualize the state of
>> OSXSAVE to give the guest the honour of appearing to enable it.  A guest
>> kernel may get confused if it starts with OSXSAVE set, as it may use it
>> to control its own init logic.
>>     
>
> That wouldn't be an issue if you use the *native* CPUID to look for
> OSXSAVE early on, since such virtualization would only be visible though
> the PV interface, right?
>
> It seems cleaner than probing, to be sure...
>   

Well, at the moment the problem is that cpuid (both PV and native) show 
XSAVE, but Xen prevents cr4.OSXSAVE from being set, crashing the 
kernel.  There's now a patch in Xen to mask XSAVE in CPUID, so that 
guests don't try to use it; the patch in the kernel is just to support 
non-bleeding-edge versions of Xen.

There have been some patches floating around for Xen support of XSAVE, 
but I think there are some issues with the variable-sized CPU context 
and save/restore/migrate, so they've been put on the backburner until 
there's a real need for them.  I haven't looked at them, but I wouldn't 
have assumed that Xen would necessarily set OSXSAVE for itself, or 
require guests to do so (if a guest can make do with a simpler CPU 
context structure, then that might be simpler for things like 
cross-architecture migration, etc).  I think that the only safe 
assumption is that XSAVE is available iff cpuid.XSAVE is set, modulo the 
bug mentioned above.

I guess if we support XSAVE for any vcpu, all the pcpus must have 
OSXSAVE set, and we rely on the fact that the XSAVE format is compatible 
with FXSAVE where they overlap.  But I really don't know what happens 
when guests use xsetbv and how that might be virtualized/paravirtualized.

    J