From: Miroslav Cimerman <mc@raidlab.org>
To: Song Liu <song@kernel.org>, Yu Kuai <yukuai@fygo.io>,
Andrew Morton <akpm@linux-foundation.org>
Cc: Li Nan <magiclinan@didiglobal.com>, Xiao Ni <xiao@kernel.org>,
Eric Biggers <ebiggers@kernel.org>,
Christoph Hellwig <hch@lst.de>,
David Laight <david.laight.linux@gmail.com>,
Markus Stockhausen <markus.stockhausen@collogia.de>,
linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH 1/3] raid6: declare the AVX-512 P/Q stores as asm outputs
Date: Sun, 22 Nov 2026 03:29:31 +0000 [thread overview]
Message-ID: <20261122032933.2306875-2-mc@raidlab.org> (raw)
In-Reply-To: <20261122032933.2306875-1-mc@raidlab.org>
The AVX-512 gen_syndrome() and xor_syndrome() functions store P and Q in
inline asm that lists these memory operands as inputs only. The compiler
may then assume that the asm does not change P and Q. Each function ends
with an sfence whose asm has a "memory" clobber, and the constraints
technically rely on it.
Declare the stored P and Q as outputs: "=m" where the asm only writes
them, and "+m" for Q in xor_syndrome(), which the same asm reads and
writes back. The SSE2 and AVX2 functions already declare their stores as
outputs.
With clang the generated code does not change: it already loads the data
pointers from ptrs[] inside the loop over the block. gcc 15.2 now does
the same, because a byte store may alias the pointer array. Five of the
six functions get one or two more instructions.
Suggested-by: Eric Biggers <ebiggers@kernel.org>
Link: https://lore.kernel.org/linux-raid/20261116210440.GE229817@google.com/
Signed-off-by: Miroslav Cimerman <mc@raidlab.org>
---
lib/raid/raid6/x86/avx512.c | 30 ++++++++++++------------------
1 file changed, 12 insertions(+), 18 deletions(-)
diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index dac0edb9189e..7dc0a45cd2fb 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -73,8 +73,7 @@ static void raid6_avx5121_gen_syndrome(int disks, size_t bytes, void **ptrs)
"vpxorq %%zmm2,%%zmm2,%%zmm2\n\t"
"vmovntdq %%zmm4,%1\n\t"
"vpxorq %%zmm4,%%zmm4,%%zmm4"
- :
- : "m" (p[d]), "m" (q[d]));
+ : "=m" (p[d]), "=m" (q[d]));
}
asm volatile("sfence" : : : "memory");
@@ -133,8 +132,7 @@ static void raid6_avx5121_xor_syndrome(int disks, int start, int stop,
/* Don't use movntdq for r/w memory area < cache line */
"vmovdqa64 %%zmm4,%0\n\t"
"vmovdqa64 %%zmm2,%1"
- :
- : "m" (q[d]), "m" (p[d]));
+ : "+m" (q[d]), "=m" (p[d]));
}
asm volatile("sfence" : : : "memory");
@@ -204,9 +202,8 @@ static void raid6_avx5122_gen_syndrome(int disks, size_t bytes, void **ptrs)
"vmovntdq %%zmm3,%1\n\t"
"vmovntdq %%zmm4,%2\n\t"
"vmovntdq %%zmm6,%3"
- :
- : "m" (p[d]), "m" (p[d+64]), "m" (q[d]),
- "m" (q[d+64]));
+ : "=m" (p[d]), "=m" (p[d+64]), "=m" (q[d]),
+ "=m" (q[d+64]));
}
asm volatile("sfence" : : : "memory");
@@ -289,9 +286,8 @@ static void raid6_avx5122_xor_syndrome(int disks, int start, int stop,
"vmovdqa64 %%zmm6,%1\n\t"
"vmovdqa64 %%zmm2,%2\n\t"
"vmovdqa64 %%zmm3,%3"
- :
- : "m" (q[d]), "m" (q[d+64]), "m" (p[d]),
- "m" (p[d+64]));
+ : "+m" (q[d]), "+m" (q[d+64]), "=m" (p[d]),
+ "=m" (p[d+64]));
}
asm volatile("sfence" : : : "memory");
@@ -393,10 +389,9 @@ static void raid6_avx5124_gen_syndrome(int disks, size_t bytes, void **ptrs)
"vpxorq %%zmm12,%%zmm12,%%zmm12\n\t"
"vmovntdq %%zmm14,%7\n\t"
"vpxorq %%zmm14,%%zmm14,%%zmm14"
- :
- : "m" (p[d]), "m" (p[d+64]), "m" (p[d+128]),
- "m" (p[d+192]), "m" (q[d]), "m" (q[d+64]),
- "m" (q[d+128]), "m" (q[d+192]));
+ : "=m" (p[d]), "=m" (p[d+64]), "=m" (p[d+128]),
+ "=m" (p[d+192]), "=m" (q[d]), "=m" (q[d+64]),
+ "=m" (q[d+128]), "=m" (q[d+192]));
}
asm volatile("sfence" : : : "memory");
@@ -528,10 +523,9 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop,
"vmovdqa64 %%zmm6,%5\n\t"
"vmovdqa64 %%zmm12,%6\n\t"
"vmovdqa64 %%zmm14,%7"
- :
- : "m" (p[d]), "m" (p[d+64]), "m" (p[d+128]),
- "m" (p[d+192]), "m" (q[d]), "m" (q[d+64]),
- "m" (q[d+128]), "m" (q[d+192]));
+ : "=m" (p[d]), "=m" (p[d+64]), "=m" (p[d+128]),
+ "=m" (p[d+192]), "+m" (q[d]), "+m" (q[d+64]),
+ "+m" (q[d+128]), "+m" (q[d+192]));
}
asm volatile("sfence" : : : "memory");
asm volatile("vzeroupper");
--
2.53.0
next prev parent reply other threads:[~2026-11-22 3:29 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-22 3:29 [PATCH 0/3] raid6: x86: asm constraints, sfence and prefetch in xor_syndrome Miroslav Cimerman
2026-11-22 3:29 ` Miroslav Cimerman [this message]
2026-11-22 3:29 ` [PATCH 2/3] raid6: remove the sfence from the x86 xor_syndrome functions Miroslav Cimerman
2026-11-22 3:29 ` [PATCH 3/3] raid6: prefetch Q with prefetcht0 in the x86 x4 " Miroslav Cimerman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261122032933.2306875-2-mc@raidlab.org \
--to=mc@raidlab.org \
--cc=akpm@linux-foundation.org \
--cc=david.laight.linux@gmail.com \
--cc=ebiggers@kernel.org \
--cc=hch@lst.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-raid@vger.kernel.org \
--cc=magiclinan@didiglobal.com \
--cc=markus.stockhausen@collogia.de \
--cc=song@kernel.org \
--cc=xiao@kernel.org \
--cc=yukuai@fygo.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox