Re: XSAVE save/restore shortcomings

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 06/09/13 13:35, Jan Beulich wrote:
>>>> On 06.09.13 at 13:57, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> Linux publishes the XSAVE blob to userspace as an NT_X86_XSTATE note in
>> core dumps, but does not include CPUID data.  If offsets change, it will
>> not be possible to examine a core dump without knowing the processor on
>> which it was produced.
> Design mistake.
>
>> For our beloved hypervisors, if offsets can change it will be
>> _impossible_ to migrate from a machine to another if they have different
>> offsets.  It will be impossible (lest you disable XSAVE and thus all
>> processor features starting with AVX) to have clusters of heterogeneous
>> virtualization hosts.  This is because (a) the guest might have stored
>> XSAVE areas at arbitrary places in the source, and the destination will
>> not be able to restore them; (b) there are no vmexits for
>> XSAVE/XSAVEOPT/XRSTOR, and anyway they would be too slow.
> No, this is precisely what this thread is about: Migration can work
> with the offsets varying, but only if the there's no attempt to save
> the whole blob in one go. There needs to be a save record per
> state component, and that save record needs to include (implicitly
> or explicitly) include the size of that blob; the offset within the
> save area is of no interest then - the consuming system simply
> needs to put it at the offset within its save area that corresponds
> to the respective state component.
>
> The fact that we're currently dependent on the offsets is again a
> design flaw.

There is a further complication - There would need to be full guest OS
support for the offsets possibly changing across migrate.  Simply having
Xen able to fix up the vcpu state doesn't prevent the guest OS from
getting it wrong.

~Andrew

>
>> So please Intel, pretty please do not modify the XSAVE offsets, and
>> clarify this as soon as possible.
> I'd like to ask for the exact opposite: Drop any mention of specific
> numbers from the documentation, except when used as an
> example for a particular implementation (it's that, btw, how I
> interpret the numbers given in the AVX-512 spec). Not allowing
> compaction is going to be harmful sooner or later (as processors
> show up that implement new state, but may not implement all
> previous features, namely if some of them turn out to be relatively
> useless: AMD's LWP appears to be an example of such a feature,
> at least judging by - as you alluded to - no OS actually making use
> of it by now, and at least some people seem to think of MPX as
> being pretty useless too).
>
> Jan
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel