[PATCH RESEND] raid6: add missing vzeroupper to AVX2 code

[PATCH RESEND] raid6: add missing vzeroupper to AVX2 code

[Eric Biggers]

Since the AVX2 optimized RAID6 code uses YMM registers, execute
vzeroupper before returning from it.  This is needed to avoid degrading
the performance of any later SSE code that may happen to be executed.

Fixes: 2c935842bdb4 ("lib/raid6: Add AVX2 optimized gen_syndrome functions")
Fixes: 7056741fd9fc ("lib/raid6: Add AVX2 optimized recovery functions")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
---

This didn't get taken through the x86 tree.  Andrew, it seems you're
taking patches to lib/raid/.  Can you apply this one?

 lib/raid/raid6/x86/avx2.c       | 6 ++++++
 lib/raid/raid6/x86/recov_avx2.c | 2 ++
 2 files changed, 8 insertions(+)

diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
index 7d829c669ea7..3cc2fe7ac42c 100644
--- a/lib/raid/raid6/x86/avx2.c
+++ b/lib/raid/raid6/x86/avx2.c
@@ -67,6 +67,7 @@ static void raid6_avx21_gen_syndrome(int disks, size_t bytes, void **ptrs)
 	}
 
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -115,6 +116,7 @@ static void raid6_avx21_xor_syndrome(int disks, int start, int stop,
 	}
 
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -175,6 +177,7 @@ static void raid6_avx22_gen_syndrome(int disks, size_t bytes, void **ptrs)
 	}
 
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -243,6 +246,7 @@ static void raid6_avx22_xor_syndrome(int disks, int start, int stop,
 	}
 
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -334,6 +338,7 @@ static void raid6_avx24_gen_syndrome(int disks, size_t bytes, void **ptrs)
 	}
 
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -444,6 +449,7 @@ static void raid6_avx24_xor_syndrome(int disks, int start, int stop,
 		asm volatile("vmovntdq %%ymm14,%0" : "=m" (q[d+96]));
 	}
 	asm volatile("sfence" : : : "memory");
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
diff --git a/lib/raid/raid6/x86/recov_avx2.c b/lib/raid/raid6/x86/recov_avx2.c
index a714a780a2d8..820871046e30 100644
--- a/lib/raid/raid6/x86/recov_avx2.c
+++ b/lib/raid/raid6/x86/recov_avx2.c
@@ -176,6 +176,7 @@ static void raid6_2data_recov_avx2(int disks, size_t bytes, int faila,
 #endif
 	}
 
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 
@@ -293,6 +294,7 @@ static void raid6_datap_recov_avx2(int disks, size_t bytes, int faila,
 #endif
 	}
 
+	asm volatile("vzeroupper");
 	kernel_fpu_end();
 }
 

base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
prerequisite-patch-id: ebf41d9b9e92708245f6ae0657896ff3f8ef7d0d
-- 
2.55.0

Re: [PATCH RESEND] raid6: add missing vzeroupper to AVX2 code

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The patch adds `vzeroupper` to AVX2 functions to avoid AVX-to-SSE transition penalties, but fails to apply the same necessary fix to the corresponding AVX-512 functions which suffer from the exact same issue.
--

commit 51aca22e4c3be53f90a7f330d0ca103da02290ab
Author: Eric Biggers <ebiggers@kernel.org>

raid6: add missing vzeroupper to AVX2 code

This patch adds the vzeroupper instruction to the AVX2 optimized RAID6
code before returning. This prevents degrading the performance of
subsequent legacy SSE code by avoiding hardware transition penalties.

> diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
> index 7d829c669ea79..3cc2fe7ac42c5 100644
> --- a/lib/raid/raid6/x86/avx2.c
> +++ b/lib/raid/raid6/x86/avx2.c
> @@ -67,6 +67,7 @@ static void raid6_avx21_gen_syndrome(int disks, size_t bytes, void **ptrs)
>  	}
>  
>  	asm volatile("sfence" : : : "memory");
> +	asm volatile("vzeroupper");

[Severity: High]
This isn't a bug introduced by this patch, but do the AVX-512 versions of
this code need the same fix?

In lib/raid/raid6/x86/avx512.c:raid6_avx5121_gen_syndrome():

	asm volatile("sfence" : : : "memory");
	kernel_fpu_end();

And in lib/raid/raid6/x86/recov_avx512.c:raid6_2data_recov_avx512():

	}

	kernel_fpu_end();

The AVX-512 syndrome generation and recovery functions exit leaving ZMM
registers dirty without calling vzeroupper. Without vzeroupper after using
ZMM0-ZMM15 registers, the CPU remains in a dirty upper state. This flaw
appears to be mirrored across all AVX-512 functions in these files.

If subsequent kernel code on the same CPU core executes legacy 128-bit SSE
instructions before the state is cleared, could this cause severe hardware
transition penalties that degrade overall crypto or networking performance
and cause a performance regression?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831212308.213855-1-ebiggers@kernel.org?part=1