[PATCH RESEND] raid6: add missing vzeroupper to AVX-512 code

[PATCH RESEND] raid6: add missing vzeroupper to AVX-512 code

[Eric Biggers]

Since the AVX-512 optimized RAID6 code uses ZMM registers, execute
vzeroupper before returning from it.  This is needed to avoid degrading
the performance of any later SSE code that may happen to be executed.

Fixes: e0a491c12968 ("lib/raid6: Add AVX512 optimized gen_syndrome functions")
Fixes: 13c520b2993c ("lib/raid6: Add AVX512 optimized recovery functions")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---

This didn't get taken through the x86 tree.  Andrew, it seems you're
taking patches to lib/raid/.  Can you apply this one?

 lib/raid/raid6/x86/avx512.c       | 6 ++++++
 lib/raid/raid6/x86/recov_avx512.c | 2 ++
 2 files changed, 8 insertions(+)

diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index e671eb5bde63..772bfc4af6df 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -78,6 +78,7 @@ static void raid6_avx5121_gen_syndrome(int disks, size_t bytes, void **ptrs)
 	}
 
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -137,6 +138,7 @@ static void raid6_avx5121_xor_syndrome(int disks, int start, int stop,
 	}
 
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -208,6 +210,7 @@ static void raid6_avx5122_gen_syndrome(int disks, size_t bytes, void **ptrs)
 	}
 
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -292,6 +295,7 @@ static void raid6_avx5122_xor_syndrome(int disks, int start, int stop,
 	}
 
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -396,6 +400,7 @@ static void raid6_avx5124_gen_syndrome(int disks, size_t bytes, void **ptrs)
 	}
 
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -529,6 +534,7 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop,
 			       "m" (q[d+128]), "m" (q[d+192]));
 	}
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 const struct raid6_calls raid6_avx512x4 = {
diff --git a/lib/raid/raid6/x86/recov_avx512.c b/lib/raid/raid6/x86/recov_avx512.c
index ec72d5a30c01..299a3f044d61 100644
--- a/lib/raid/raid6/x86/recov_avx512.c
+++ b/lib/raid/raid6/x86/recov_avx512.c
@@ -211,6 +211,7 @@ static void raid6_2data_recov_avx512(int disks, size_t bytes, int faila,
 #endif
 	}
 
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -353,6 +354,7 @@ static void raid6_datap_recov_avx512(int disks, size_t bytes, int faila,
 #endif
 	}
 
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 

base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
prerequisite-patch-id: ebf41d9b9e92708245f6ae0657896ff3f8ef7d0d
prerequisite-patch-id: f85f9c7e8bdd6fcfd394edb80f2263d7189e138d
-- 
2.55.0

Re: [PATCH RESEND] raid6: add missing vzeroupper to AVX-512 code

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing `vzeroupper` in AVX and AVX2 RAID6 and XOR implementations
- [High] Missing `vzeroupper` in ARIA GFNI AVX512 crypto routines
--

commit c86f8a80ee519a29f4ac71611d625cca2868db38
Author: Eric Biggers <ebiggers@kernel.org>
raid6: add missing vzeroupper to AVX-512 code

This commit adds vzeroupper to the AVX-512 optimized RAID6 routines before
returning, preventing performance degradation of subsequent SSE code.

> diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
> --- a/lib/raid/raid6/x86/avx512.c
> +++ b/lib/raid/raid6/x86/avx512.c
> @@ -78,6 +78,7 @@ static void raid6_avx5121_gen_syndrome(int disks, size_t bytes, void **ptrs)
>  	}
>  
>  	asm volatile("sfence" : : : "memory");
> +	asm volatile("vzeroupper");
>  	kernel_fpu_end();
>  }

[Severity: High]
This is a pre-existing issue, but does this incomplete fix also need to
address the conceptually identical AVX/AVX2 implementations?

For example, in lib/raid/raid6/x86/avx2.c:raid6_avx21_gen_syndrome():

	asm volatile("sfence" : : : "memory");
	kernel_fpu_end();

And similarly in lib/raid/raid6/x86/recov_avx2.c:raid6_2data_recov_avx2()
and lib/raid/xor/x86/xor-avx.c:xor_gen_avx(). These routines use YMM
registers but return to other kernel paths without clearing the upper state.

[Severity: High]
This isn't a bug introduced by this patch, but are there other areas in the
kernel that need a similar fix? For example, in the crypto subsystem, the
ARIA GFNI AVX512 routines dirty ZMM registers but omit the vzeroupper.

arch/x86/crypto/aria_gfni_avx512_glue.c:aria_avx512_ctr_encrypt():

			kernel_fpu_begin();
			aria_ops.aria_ctr_crypt_64way(ctx, dst, src,
						      &req_ctx->keystream[0],
						      walk.iv);
			kernel_fpu_end();

And in arch/x86/crypto/aria-gfni-avx512-asm_64.S:aria_gfni_avx512_encrypt_64way(),
the assembly implementation returns without issuing a vzeroupper to clear
the upper state of the ZMM registers.

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831212316.213896-1-ebiggers@kernel.org?part=1

Re: [PATCH RESEND] raid6: add missing vzeroupper to AVX-512 code

[Eric Biggers]

On Mon, Aug 31, 2026 at 10:07:54PM +0000, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> 
> Pre-existing issues:
> - [High] Missing `vzeroupper` in AVX and AVX2 RAID6 and XOR implementations
> - [High] Missing `vzeroupper` in ARIA GFNI AVX512 crypto routines

Already addressed by other patches I sent.

- Eric