The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Miroslav Cimerman <mc@raidlab.org>
To: Song Liu <song@kernel.org>, Yu Kuai <yukuai@fygo.io>,
	Andrew Morton <akpm@linux-foundation.org>
Cc: Li Nan <magiclinan@didiglobal.com>, Xiao Ni <xiao@kernel.org>,
	Eric Biggers <ebiggers@kernel.org>,
	Christoph Hellwig <hch@lst.de>,
	David Laight <david.laight.linux@gmail.com>,
	Markus Stockhausen <markus.stockhausen@collogia.de>,
	linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH 3/3] raid6: prefetch Q with prefetcht0 in the x86 x4 xor_syndrome functions
Date: Sun, 22 Nov 2026 03:29:33 +0000	[thread overview]
Message-ID: <20261122032933.2306875-4-mc@raidlab.org> (raw)
In-Reply-To: <20261122032933.2306875-1-mc@raidlab.org>

The x4 xor_syndrome() functions for SSE2, AVX2 and AVX-512 prefetch Q
with prefetchnta. That hint is meant for data that is used once. Q is
read, updated and written back with normal stores, and md calls
xor_syndrome() twice on the same P/Q page for each write.

Prefetch Q with prefetcht0, which keeps it in all cache levels. The data
blocks are read once and keep prefetchnta.

Time per write measured in a userspace harness in md's read-modify-write
pattern (two xor_syndrome() calls on the same P/Q page), avx512x4 on a
Xeon Gold 6130, 2-24 data disks, write to the first, middle and last
data disk, median of 10 passes:

  P and Q in DRAM:      -2.2% (geometric mean), up to -5.7%
  P and Q in L3:        +0.2%
  P and Q in L1/L2:     +0.3%

The gain comes with 2-8 data disks, where Q is a larger part of the
work. The SSE2 and AVX2 versions were not measured.

Suggested-by: Eric Biggers <ebiggers@kernel.org>
Link: https://lore.kernel.org/linux-raid/20261116210440.GE229817@google.com/
Signed-off-by: Miroslav Cimerman <mc@raidlab.org>
---
 lib/raid/raid6/x86/avx2.c   | 4 ++--
 lib/raid/raid6/x86/avx512.c | 4 ++--
 lib/raid/raid6/x86/sse2.c   | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
index d3f488a9c1a5..5a49079ace68 100644
--- a/lib/raid/raid6/x86/avx2.c
+++ b/lib/raid/raid6/x86/avx2.c
@@ -408,8 +408,8 @@ static void raid6_avx24_xor_syndrome(int disks, int start, int stop,
 			asm volatile("vpxor %ymm13,%ymm12,%ymm12");
 			asm volatile("vpxor %ymm15,%ymm14,%ymm14");
 		}
-		asm volatile("prefetchnta %0" :: "m" (q[d]));
-		asm volatile("prefetchnta %0" :: "m" (q[d+64]));
+		asm volatile("prefetcht0 %0" :: "m" (q[d]));
+		asm volatile("prefetcht0 %0" :: "m" (q[d+64]));
 		/* P/Q left side optimization */
 		for (z = start-1 ; z >= 0 ; z--) {
 			asm volatile("vpxor %ymm5,%ymm5,%ymm5");
diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index 33d7639da5b3..38077109ab0f 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -476,8 +476,8 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop,
 				       "m" (dptr[z][d+128]),
 				       "m" (dptr[z][d+192]));
 		}
-		asm volatile("prefetchnta %0\n\t"
-			     "prefetchnta %1\n\t"
+		asm volatile("prefetcht0 %0\n\t"
+			     "prefetcht0 %1\n\t"
 			     :
 			     : "m" (q[d]), "m" (q[d+128]));
 		/* P/Q left side optimization */
diff --git a/lib/raid/raid6/x86/sse2.c b/lib/raid/raid6/x86/sse2.c
index a0709916289c..c27a4c8254ba 100644
--- a/lib/raid/raid6/x86/sse2.c
+++ b/lib/raid/raid6/x86/sse2.c
@@ -405,8 +405,8 @@ static void raid6_sse24_xor_syndrome(int disks, int start, int stop,
 			asm volatile("pxor %xmm13,%xmm12");
 			asm volatile("pxor %xmm15,%xmm14");
 		}
-		asm volatile("prefetchnta %0" :: "m" (q[d]));
-		asm volatile("prefetchnta %0" :: "m" (q[d+32]));
+		asm volatile("prefetcht0 %0" :: "m" (q[d]));
+		asm volatile("prefetcht0 %0" :: "m" (q[d+32]));
 		/* P/Q left side optimization */
 		for ( z = start-1 ; z >= 0 ; z-- ) {
 			asm volatile("pxor %xmm5,%xmm5");
-- 
2.53.0


      parent reply	other threads:[~2026-11-22  3:29 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-22  3:29 [PATCH 0/3] raid6: x86: asm constraints, sfence and prefetch in xor_syndrome Miroslav Cimerman
2026-11-22  3:29 ` [PATCH 1/3] raid6: declare the AVX-512 P/Q stores as asm outputs Miroslav Cimerman
2026-11-22  3:29 ` [PATCH 2/3] raid6: remove the sfence from the x86 xor_syndrome functions Miroslav Cimerman
2026-11-22  3:29 ` Miroslav Cimerman [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261122032933.2306875-4-mc@raidlab.org \
    --to=mc@raidlab.org \
    --cc=akpm@linux-foundation.org \
    --cc=david.laight.linux@gmail.com \
    --cc=ebiggers@kernel.org \
    --cc=hch@lst.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-raid@vger.kernel.org \
    --cc=magiclinan@didiglobal.com \
    --cc=markus.stockhausen@collogia.de \
    --cc=song@kernel.org \
    --cc=xiao@kernel.org \
    --cc=yukuai@fygo.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox