Re: 4.2-rc2: early boot memory corruption from FPU rework

[Ingo Molnar <mingo@kernel.org>]

* Dave Hansen <dave.hansen@linux.intel.com> wrote:

> On 05/05/2015 10:49 AM, Ingo Molnar wrote:
> > @@ -574,12 +573,10 @@ static void setup_init_fpu_buf(void)
> >  	on_boot_cpu = 0;
> >  
> >  	/*
> > -	 * Setup init_xstate_buf to represent the init state of
> > +	 * Setup init_xstate_ctx to represent the init state of
> >  	 * all the features managed by the xsave
> >  	 */
> > -	init_xstate_buf = alloc_bootmem_align(xstate_size,
> > -					      __alignof__(struct xsave_struct));
> > -	fx_finit(&init_xstate_buf->i387);
> > +	fx_finit(&init_xstate_ctx.i387);
> 
> This is causing memory corruption in 4.2-rc2.
> 
> We do not know the size of the 'init_xstate_buf' before we boot.  It's
> completely enumerated in CPUID leaves but it is not static by any means.
>  This commit when applied (3e5e126774) tries to replace the dynamic
> allocation with a static one.  When we do the first 'xrstor' (in
> copy_xregs_to_kernel_booting()) it overruns init_fpstate and corrupts
> the next chunk of memory (which is xfeatures_mask in my case).
> 
> I'm seeing this on a system with states not represented in
> XSTATE_RESERVE (XSTATE_ZMM_Hi256 / XSTATE_OPMASK / XSTATE_Hi16_ZMM).
> The systems affected are not widely available, but this is something
> that we absolutely do not want to see regress.
> 
> This bug could also occur if a future CPU decided to change the amount
> of storage allocated for a given xstate feature (which would be
> architecturally OK).
> 
> According to the commit:
> 
> >     This removes the last bootmem allocation from the FPU init path, allowing
> >     it to be called earlier in the boot sequence.
> 
> so we can't easily just revert this, although I'm not 100% that this is
> before bootmem is availalble.
> 
> This patch works around the problem, btw:
> 
> 	https://www.sr71.net/~dave/intel/bloat-xsave-gunk-2.patch

Yeah, so I got this prototype hardware boot crash reported in private mail and 
decoded it and after some debugging I suggested the +PAGE_SIZE hack - possibly you 
got that hack from the same person?

My suggestion was to solve this properly: if we list xstate features as supported 
then we should size their max size correctly. The AVX bits are currently not 
properly enumerated and sized - and I refuse to add feature support to the kernel 
where per task CPU state fields that the kernel saves/restores are opaque...

So please add proper AVX512 support structures to fpu/types.h and size 
XSTATE_RESERVE correctly - or alternatively we can remove the current incomplete 
AVX512 bits.

Thanks,

	Ingo