Re: [PATCH 0/6] x86: add missing vzeroupper instructions

[Eric Biggers <ebiggers@kernel.org>]

On Sun, Aug 16, 2026 at 04:14:50PM +0100, David Laight wrote:
> On Sat, 15 Aug 2026 13:57:44 -0700
> Eric Biggers <ebiggers@kernel.org> wrote:
> 
> > Assembly code using YMM or ZMM registers is supposed to end with the
> > vzeroupper instruction in order to avoid degrading the performance of
> > any later SSE code.  Since this only affects performance and not
> > correctness, it is sometimes overlooked.  Most kernel code does it
> > correctly, but a few cases were missed.  This series fixes them.
> > 
> > It should be easiest to take the full series through the x86 tree.
> 
> Would it be better to an an unconditional vzeroupper in kernel_fpu_end()?
> It could go in kernel_fpu_start() but that might have a bigger effect
> on latency.
> (I assume there is one in kernel_fpu_start() if it actually saves
> the user registers?)
> 
> Looking the latency/uops seems reasonably on everything 'recent' except
> zen-1 and knights-landing.
> Although the microcode patch for zen-2 might make that a lot worse.
> 
> 	David  

I would like to do that, but there are some issues:

- In some cases, within a single kernel-mode FPU section the kernel
  executes AVX instructions, then SSE instructions afterwards.
  It typically occurs when large input lengths are optimized specially
  with AVX and then shorter lengths fall back to SSE code.
  chacha_dosimd() in lib/crypto/x86/chacha.h is an example of this.

  In these cases the internal vzeroupper is definitely needed.

- Unnecessary overhead if only SSE instructions are used, which is still
  frequent since we don't provide both SSE and AVX versions of the same
  code when the AVX doesn't provide a notable performance benefit.
  lib/crypto/x86/sha256-ni-asm.S is an example of this.

- Further divergence from the userspace ABI, which can be annoying when
  importing assembly code to or from userspace projects, or developing
  or testing the assembly files in userspace.

As for kernel_fpu_begin(), no, it doesn't do vzeroupper.

I do think that some years down the line, we'll drop the use of SSE in
the kernel entirely.  At that point, vzeroupper in kernel_fpu_end()
would make sense.

But until then, I think we should stick with the existing, standard
convention of having the vzeroupper at the end of the assembly routines.

- Eric