Re: [PATCH 084/208] x86/fpu: Rename xsave.header::xstate_bv to 'xfeatures'

[Ingo Molnar <mingo@kernel.org>]

* Dave Hansen <dave.hansen@linux.intel.com> wrote:

> On 05/05/2015 11:16 PM, Ingo Molnar wrote:
> > Btw., does Intel have any special plans with xstate compaction?
> > 
> > AFAICS in Linux we just want to enable xfeat_mask_all to the max, 
> > including compaction, and never really modify it (in the task's 
> > lifetime).
> 
> Special plans?

I.e. are there any plans beyond using it strictly for full state 
save/restore.

> If we do an XRSTORS on it before we do an XSAVES, then we need to 
> worry.  But, if we do an XSAVES, the CPU will set it up for us.
>
> > I'm also wondering whether there will be any real 'holes' in the 
> > xfeatures capability masks of future CPUs: right now xfeatures 
> > tend to be already 'compacted' (because new CPUs tend to support 
> > all xfeatures), so compaction mostly appears to be an academic 
> > feature. Or is there already hardware out there where it matter?
> 
> There is a hole in the SDM today.  See section 2.6 in the currently 
> released 054 version.  I also know of actual hardware platforms with 
> holes.  *PLUS*, someone can always shot down CPUID bits in their 
> hypervisor or with kernel command-line options.

I see, so MPX (bits 3 and 4) aren't there yet.

Btw., there's a new xfeature it appears:

  XCR0.PKRU (bit 9): If 1, the XSAVE feature set can be used to manage 
  the PKRU register (see Section 2.7).

and bit 8 is a hole again.

Btw., regarding XCR0.PKRU: that enables 'Protection Keys' in the PTE 
format. What's the main purpose of these keys? They seem to duplicate 
the read/write bits in the PTE, with the exception that they don't 
impact instruction fetches. So is this used to allow user-space to 
execute but otherwise not read instructions?

Or some other purpose I missed?

In any case, these holes are really minor at the moment, and the 
question is, what is the performance difference between a 'compactede' 
XSAVE*/XRSTOR* pair, versus a standard format one?

> > Maybe once we get AVX512 in addition to MPX we can use compaction 
> > materially: as there will be lots of tasks without MPX state but 
> > with AVX512 state - in fact I suspect that will be the common 
> > case.
> 
> Right.
> 
> But we'd need to get to a point where we are calling 'xsaves' with a 
> Requested Feature BitMask (aka RFBM[]) that had holes in it.  As it 
> stands today, we always call it with RFBM=-1 and so we always have 
> XCOMP_BV = XCR0.

XCOMP_BV must also have bit 63 set.

  13.8.1
  Standard Form of XRSTOR

  The standard from of XRSTOR performs additional fault checking. 
  Either of the following conditions causes ageneral-protection 
  exception (#GP):

  The XSTATE_BV field of the XSAVE header sets a bit that is not set 
  in XCR0. Bytes 23:8 of the XSAVE header are not all 0 (this implies 
                                                         ^^^^^^^^^^^^
  that all bits in XCOMP_BV are 0).
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Note the part I underlined: all of XCOMP_BV has to be 0 for any 
standard form of XRSTOR, and if we use a compacted form, bit 63 must 
be set:


this is why bit 63 is a nonsensical interface: it being nonzero 
already tells the hardware that we requested compaction ...

> We'd need to determine which fields are in the init state before we 
> do an xsaves.

Why? I don't think that's necessary.

The way I read the SDM both the 'init' and the 'modified' 
optimizations are mostly automatic: the CPU determines it 
automatically when a state component is (or returned to!) init state, 
and signals that via the relevant bit in XSTATE_BV being zeroed out.

This is what the SDM says about XSAVES (section 13.11 in the 054 SDM):

  — If state component i is in its initial configuration, XSTATE_BV[i] 
    may be written with either 0 or 1.

so XSAVES itself performs the first step of the 'init optimization', 
automatically: it will opportunistically write 0 to the relevant bit 
in XSTATE_BV and won't save the state.

Once there's 0 in XSTATE_BV, put there by XSAVES, the XRSTOR 
instruction is able to perform the other half of the optimization: by 
not restoring it but initializing it (if needed).

XSAVES will also set up XSTATE_BV and XCOMP_BV so that XRSTOR does not 
have to worry about it, it will do a compacted restore.

Thanks,

	Ingo