* Re: [PATCH 7/8] s390/mm: use free_reserved_page() in vmem_free_pages()
From: Heiko Carstens @ 2026-05-11 14:21 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: David S. Miller, Andreas Larsson, Mike Rapoport, Andrew Morton,
Alexander Gordeev, Gerald Schaefer, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, Madhavan Srinivasan,
Michael Ellerman, Nicholas Piggin, Christophe Leroy (CS GROUP),
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Suren Baghdasaryan, Michal Hocko, sparclinux, linux-kernel,
linux-mm, linux-s390, linuxppc-dev
In-Reply-To: <20260511-bootmem_info_prep-v1-7-3fb0be6fc688@kernel.org>
On Mon, May 11, 2026 at 04:05:35PM +0200, David Hildenbrand (Arm) wrote:
> We never select CONFIG_HAVE_BOOTMEM_INFO_NODE on s390. Therefore,
> free_bootmem_page() nowadays always translates to free_reserved_page().
>
> Let's use free_reserved_page() to replace the free_bootmem_page() loop.
> We can stop including bootmem_info.h.
>
> Likely, vmemmap freeing code could be factored out into the core in the
> future.
>
> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> ---
> arch/s390/mm/vmem.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/arch/s390/mm/vmem.c b/arch/s390/mm/vmem.c
> index eeadff45e0e1..d8b2a60e0c33 100644
> --- a/arch/s390/mm/vmem.c
> +++ b/arch/s390/mm/vmem.c
> @@ -4,7 +4,6 @@
> */
>
> #include <linux/memory_hotplug.h>
> -#include <linux/bootmem_info.h>
> #include <linux/cpufeature.h>
> #include <linux/memblock.h>
> #include <linux/pfn.h>
> @@ -51,7 +50,7 @@ static void vmem_free_pages(unsigned long addr, int order, struct vmem_altmap *a
> if (PageReserved(page)) {
> /* allocated from memblock */
> while (nr_pages--)
> - free_bootmem_page(page++);
> + free_reserved_page(page++);
What about the implicit call of kmemleak_free_part_phys() which gets
removed with this?
^ permalink raw reply
* Re: [PATCH 7/8] s390/mm: use free_reserved_page() in vmem_free_pages()
From: David Hildenbrand (Arm) @ 2026-05-11 14:24 UTC (permalink / raw)
To: Heiko Carstens
Cc: David S. Miller, Andreas Larsson, Mike Rapoport, Andrew Morton,
Alexander Gordeev, Gerald Schaefer, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, Madhavan Srinivasan,
Michael Ellerman, Nicholas Piggin, Christophe Leroy (CS GROUP),
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Suren Baghdasaryan, Michal Hocko, sparclinux, linux-kernel,
linux-mm, linux-s390, linuxppc-dev
In-Reply-To: <20260511142157.9589E90-hca@linux.ibm.com>
On 5/11/26 16:21, Heiko Carstens wrote:
> On Mon, May 11, 2026 at 04:05:35PM +0200, David Hildenbrand (Arm) wrote:
>> We never select CONFIG_HAVE_BOOTMEM_INFO_NODE on s390. Therefore,
>> free_bootmem_page() nowadays always translates to free_reserved_page().
>>
>> Let's use free_reserved_page() to replace the free_bootmem_page() loop.
>> We can stop including bootmem_info.h.
>>
>> Likely, vmemmap freeing code could be factored out into the core in the
>> future.
>>
>> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
>> ---
>> arch/s390/mm/vmem.c | 3 +--
>> 1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/arch/s390/mm/vmem.c b/arch/s390/mm/vmem.c
>> index eeadff45e0e1..d8b2a60e0c33 100644
>> --- a/arch/s390/mm/vmem.c
>> +++ b/arch/s390/mm/vmem.c
>> @@ -4,7 +4,6 @@
>> */
>>
>> #include <linux/memory_hotplug.h>
>> -#include <linux/bootmem_info.h>
>> #include <linux/cpufeature.h>
>> #include <linux/memblock.h>
>> #include <linux/pfn.h>
>> @@ -51,7 +50,7 @@ static void vmem_free_pages(unsigned long addr, int order, struct vmem_altmap *a
>> if (PageReserved(page)) {
>> /* allocated from memblock */
>> while (nr_pages--)
>> - free_bootmem_page(page++);
>> + free_reserved_page(page++);
>
> What about the implicit call of kmemleak_free_part_phys() which gets
> removed with this?
Thanks for taking a look!
See patch #4:
https://lore.kernel.org/r/20260511-bootmem_info_prep-v1-4-3fb0be6fc688@kernel.org
--
Cheers,
David
^ permalink raw reply
* Re: [PATCH 7/8] s390/mm: use free_reserved_page() in vmem_free_pages()
From: Heiko Carstens @ 2026-05-11 15:22 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: David S. Miller, Andreas Larsson, Mike Rapoport, Andrew Morton,
Alexander Gordeev, Gerald Schaefer, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, Madhavan Srinivasan,
Michael Ellerman, Nicholas Piggin, Christophe Leroy (CS GROUP),
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Suren Baghdasaryan, Michal Hocko, sparclinux, linux-kernel,
linux-mm, linux-s390, linuxppc-dev
In-Reply-To: <9e313bd2-cc90-4b9f-9d6c-7dac8d9ec9db@kernel.org>
On Mon, May 11, 2026 at 04:24:16PM +0200, David Hildenbrand (Arm) wrote:
> On 5/11/26 16:21, Heiko Carstens wrote:
> > On Mon, May 11, 2026 at 04:05:35PM +0200, David Hildenbrand (Arm) wrote:
> >> We never select CONFIG_HAVE_BOOTMEM_INFO_NODE on s390. Therefore,
> >> free_bootmem_page() nowadays always translates to free_reserved_page().
> >>
> >> Let's use free_reserved_page() to replace the free_bootmem_page() loop.
> >> We can stop including bootmem_info.h.
> >>
> >> Likely, vmemmap freeing code could be factored out into the core in the
> >> future.
> >>
> >> Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
> >> ---
> >> arch/s390/mm/vmem.c | 3 +--
> >> 1 file changed, 1 insertion(+), 2 deletions(-)
...
> > What about the implicit call of kmemleak_free_part_phys() which gets
> > removed with this?
>
> Thanks for taking a look!
>
> See patch #4:
>
> https://lore.kernel.org/r/20260511-bootmem_info_prep-v1-4-3fb0be6fc688@kernel.org
Heh, right, you can't expect from me that I would read more than one
patch of a series :)
Acked-by: Heiko Carstens <hca@linux.ibm.com>
^ permalink raw reply
* Re: [PATCH 7/8] s390/mm: use free_reserved_page() in vmem_free_pages()
From: David Hildenbrand (Arm) @ 2026-05-11 15:28 UTC (permalink / raw)
To: Heiko Carstens
Cc: David S. Miller, Andreas Larsson, Mike Rapoport, Andrew Morton,
Alexander Gordeev, Gerald Schaefer, Vasily Gorbik,
Christian Borntraeger, Sven Schnelle, Madhavan Srinivasan,
Michael Ellerman, Nicholas Piggin, Christophe Leroy (CS GROUP),
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka,
Suren Baghdasaryan, Michal Hocko, sparclinux, linux-kernel,
linux-mm, linux-s390, linuxppc-dev
In-Reply-To: <20260511152226.9589G9f-hca@linux.ibm.com>
On 5/11/26 17:22, Heiko Carstens wrote:
> On Mon, May 11, 2026 at 04:24:16PM +0200, David Hildenbrand (Arm) wrote:
>> On 5/11/26 16:21, Heiko Carstens wrote:
> ...
>>> What about the implicit call of kmemleak_free_part_phys() which gets
>>> removed with this?
>>
>> Thanks for taking a look!
>>
>> See patch #4:
>>
>> https://lore.kernel.org/r/20260511-bootmem_info_prep-v1-4-3fb0be6fc688@kernel.org
>
> Heh, right, you can't expect from me that I would read more than one
> patch of a series :)
:)
Let me add a hint that a recent patch changed that.
Thanks!
--
Cheers,
David
^ permalink raw reply
* Re: IBM Power S822LC: pci 0021:0d:00.0: xHCI HW did not halt within 32000 usec status = 0x0
From: Paul Menzel @ 2026-05-11 21:57 UTC (permalink / raw)
To: Michal Pecio
Cc: Mathias Nyman, Greg Kroah-Hartman, linux-usb, LKML, linuxppc-dev
In-Reply-To: <20260506193037.6de9e355.michal.pecio@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2370 bytes --]
[Cc: +linuxppc-dev]
Dear Michal,
Thank you for your reply.
Am 06.05.26 um 19:30 schrieb Michal Pecio:
> On Wed, 6 May 2026 18:06:20 +0200, Paul Menzel wrote:
>> On the IBM Power S822LC (8335-GCA POWER8), rebooting into Linux 7.1-rc2+
>> with kexec results in the warning below:
>>
>> [ 0.000000] Linux version 7.1.0-rc2+ (x@b) (gcc (Ubuntu 11.2.0-7ubuntu2) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.37) #3 SMP PREEMPT Wed May 6 08:50:5
>> […]
>> [ 0.000000] Hardware name: 8335-GCA POWER8 (raw) 0x4d0200 opal:skiboot-5.4.8-5787ad3 PowerNV
>> […]
>> [ 1.593760] NET: Registered PF_UNIX/PF_LOCAL protocol family
>> [ 1.593859] pci 0021:0d:00.0: enabling device (0140 -> 0142)
>> [ 1.627080] pci 0021:0d:00.0: xHCI HW did not halt within 32000 usec status = 0x0
>> [ 1.627094] pci 0021:0d:00.0: quirk_usb_early_handoff+0x0/0x300 took 32465 usecs
>> [ 1.627123] PCI: CLS 0 bytes, default 128
> Does it work any better if kexecing other kernel versions?
No, the problem goes as far back as 5.17-rc7. (I didn’t try anything
before.)
> What if you increase XHCI_MAX_HALT_USEC by 10* or 100* ?
I have to test this.
> Does the controller work normally after this warning?
It does not look like it. In the log attached to my report, later on
there is:
[ 1.739374] xhci_hcd 0021:0d:00.0: xHCI Host Controller
[ 1.739431] xhci_hcd 0021:0d:00.0: new USB bus registered,
assigned bus number 1
[ 1.794727] Freeing initrd memory: 52928K
[ 1.801984] xhci_hcd 0021:0d:00.0: Host halt failed, -110
[ 1.801988] xhci_hcd 0021:0d:00.0: can't setup: -110
[ 1.802137] xhci_hcd 0021:0d:00.0: USB bus 1 deregistered
[ 1.802154] xhci_hcd 0021:0d:00.0: init 0021:0d:00.0 fail, -110
[ 1.802250] xhci_hcd 0021:0d:00.0: probe with driver xhci_hcd
failed with error -110
`lsusb` also does not list the device. But I need to check on hardware.
Kind regards,
Paul
PS: Claude Sonnet 4.6 cooked up the attached patch, which does *not*
help though, but does get it to the return code 0x10, which Claude
replied to with:
> ● The status change 0x0 → 0x10 is meaningful: 0x10 is PCD (Port Change Detect, bit 4),
> HCHalted=0. The old-kernel reset (from our commit) did take effect …
Please excuse, if I attach/cite hallucinations.
[-- Attachment #2: 0001-usb-xhci-Reset-controller-on-kexec-to-prevent-stale-.patch --]
[-- Type: text/x-patch, Size: 3340 bytes --]
From b110d128ebe662a02319af6899d8dc50759a1147 Mon Sep 17 00:00:00 2001
From: Paul Menzel <pmenzel@molgen.mpg.de>
Date: Mon, 11 May 2026 17:33:35 +0200
Subject: [PATCH] usb: xhci: Reset controller on kexec to prevent stale state
in next kernel
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
When kexec is used to boot into a new Linux kernel, xhci_shutdown() only
halts the xHCI controller (CMD_RUN=0, waits for STS_HALT) without
resetting it. After halt, the controller still holds stale internal state
from the old kernel: the DCBAA pointer, scratchpad buffer address, event
and command ring dequeue pointers, and so on, all pointing into the old
kernel's now-invalid memory.
If platform firmware (e.g. OPAL on IBM POWER8) then touches the USB
controller during the kexec transition – for example to provide a USB
keyboard console – it may attempt to use those stale ring-buffer
pointers. This leaves the controller in an undefined, running state
(STS_HALT=0) when the new kernel arrives. The new kernel's early
quirk_usb_handoff_xhci() fixup then fails to halt the controller and
the xHCI driver can no longer initialise it, producing:
IBM Power S822LC: pci 0021:0d:00.0: xHCI HW did not halt within
32000 usec status = 0x0
A Petitboot-to-Linux kexec chain works correctly because Petitboot, as
the active USB owner, properly halted the controller; the new kernel
finds it cleanly halted.
Fix this by issuing a controller reset (CMD_RESET) after the halt
whenever kexec is in progress. The reset clears all internal state
(DCBAA, scratchpad, ring buffers, port state) so that any firmware or
the incoming kernel always starts from a well-known, initialisation-ready
state. This mirrors the approach already used for the
XHCI_SPURIOUS_WAKEUP and XHCI_RESET_TO_DEFAULT quirks.
Link: https://lore.kernel.org/all/fb68e15d-b8f2-42ac-aa65-0d9fedcfcdbd@molgen.mpg.de/
Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
---
drivers/usb/host/xhci.c | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c
index a54f5b57f2055..55031d03ad40e 100644
--- a/drivers/usb/host/xhci.c
+++ b/drivers/usb/host/xhci.c
@@ -9,6 +9,7 @@
*/
#include <linux/jiffies.h>
+#include <linux/kexec.h>
#include <linux/pci.h>
#include <linux/iommu.h>
#include <linux/iopoll.h>
@@ -775,11 +776,16 @@ void xhci_shutdown(struct usb_hcd *hcd)
xhci_halt(xhci);
/*
- * Workaround for spurious wakeps at shutdown with HSW, and for boot
- * firmware delay in ADL-P PCH if port are left in U3 at shutdown
+ * Workaround for spurious wakeups at shutdown with HSW, and for boot
+ * firmware delay in ADL-P PCH if ports are left in U3 at shutdown.
+ * Also reset on kexec to leave the controller in a clean state with
+ * all internal state (DCBAA, scratchpad, rings) cleared, so the next
+ * kernel can initialize it without interference from stale pointers
+ * into the old kernel's memory.
*/
if (xhci->quirks & XHCI_SPURIOUS_WAKEUP ||
- xhci->quirks & XHCI_RESET_TO_DEFAULT)
+ xhci->quirks & XHCI_RESET_TO_DEFAULT ||
+ kexec_in_progress)
xhci_reset(xhci, XHCI_RESET_SHORT_USEC);
spin_unlock_irq(&xhci->lock);
--
2.54.0
^ permalink raw reply related
* [PATCH AUTOSEL 7.0-5.10] powerpc/vmx: avoid KASAN instrumentation in enter_vmx_ops() for kexec
From: Sasha Levin @ 2026-05-11 22:19 UTC (permalink / raw)
To: patches, stable
Cc: Sourabh Jain, Aboorva Devarajan, Ritesh Harjani (IBM),
Madhavan Srinivasan, Sasha Levin, mpe, linuxppc-dev, linux-kernel
In-Reply-To: <20260511221931.2370053-1-sashal@kernel.org>
From: Sourabh Jain <sourabhjain@linux.ibm.com>
[ Upstream commit 38e989d504fc52900a3786b7144fb53cd67e0389 ]
The kexec sequence invokes enter_vmx_ops() via copy_page() with the MMU
disabled. In this context, code must not rely on normal virtual address
translations or trigger page faults.
With KASAN enabled, functions get instrumented and may access shadow
memory using regular address translation. When executed with the MMU
off, this can lead to page faults (bad_page_fault) from which the
kernel cannot recover in the kexec path, resulting in a hang.
The kexec path sets preempt_count to HARDIRQ_OFFSET before entering
the MMU-off copy sequence.
current_thread_info()->preempt_count = HARDIRQ_OFFSET
kexec_sequence(..., copy_with_mmu_off = 1)
-> kexec_copy_flush(image)
copy_segments()
-> copy_page(dest, addr)
bl enter_vmx_ops()
if (in_interrupt())
return 0
beq .Lnonvmx_copy
Since kexec sets preempt_count to HARDIRQ_OFFSET, in_interrupt()
evaluates to true and enter_vmx_ops() returns early.
As in_interrupt() (and preempt_count()) are always inlined, mark
enter_vmx_ops() with __no_sanitize_address to avoid KASAN
instrumentation and shadow memory access with MMU disabled, helping
kexec boot fine with KASAN enabled.
Reported-by: Aboorva Devarajan <aboorvad@linux.ibm.com>
Reviewed-by: Aboorva Devarajan <aboorvad@linux.ibm.com>
Tested-by: Aboorva Devarajan <aboorvad@linux.ibm.com>
Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260407124349.1698552-2-sourabhjain@linux.ibm.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
LLM Generated explanations, may be completely bogus:
## Phase 1: Commit Message Forensics
Record: subsystem `powerpc/vmx`; action verb `avoid`; intent is to
prevent KASAN instrumentation of `enter_vmx_ops()` when it is reached
from the kexec MMU-off copy path.
Record: tags parsed from the candidate message:
- `Reported-by: Aboorva Devarajan <aboorvad@linux.ibm.com>`
- `Reviewed-by: Aboorva Devarajan <aboorvad@linux.ibm.com>`
- `Tested-by: Aboorva Devarajan <aboorvad@linux.ibm.com>`
- `Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>`
- `Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>`
- `Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>`
- `Link: https://patch.msgid.link/20260407124349.1698552-2-
sourabhjain@linux.ibm.com`
- No `Fixes:` tag in this commit. Absence is not a negative signal.
Record: bug description from the message: `kexec_sequence()` may call
`copy_page()` with the MMU disabled; on PowerPC Book3S this can enter
`copypage_power7.S`, which calls `enter_vmx_ops()`. If KASAN instruments
`enter_vmx_ops()`, the instrumentation may access shadow memory using
normal virtual translation, which is invalid with MMU off and can cause
an unrecoverable `bad_page_fault`/hang in kexec.
Record: hidden bug fix: yes. Although phrased as “avoid
instrumentation,” this fixes a real boot/hang failure in the kexec/kdump
path under KASAN.
## Phase 2: Diff Analysis
Record: one file changed: `arch/powerpc/lib/vmx-helper.c`, with 8
insertions and 1 deletion in the fetched v3 patch. The only functional
change is changing `int enter_vmx_ops(void)` to `int
__no_sanitize_address enter_vmx_ops(void)`; the remaining additions are
explanatory comments.
Record: modified function: `enter_vmx_ops()`.
Record: before behavior: `enter_vmx_ops()` could be KASAN-instrumented.
During kexec with MMU off, the function first checks `in_interrupt()`
and should return `0`, but KASAN prologue/body instrumentation can run
before or around normal code and touch shadow memory.
Record: after behavior: `__no_sanitize_address` suppresses KASAN
instrumentation for this function, preserving the intended early return
path when `preempt_count` contains `HARDIRQ_OFFSET`.
Record: bug category: sanitizer/real-mode correctness bug causing kexec
hang. This is a crash/hang class fix, not a cleanup.
Record: fix quality: small, surgical, and low risk. It does not alter
normal logic, public API, data structures, or userspace-visible
behavior. The only behavioral effect is disabling KASAN instrumentation
for one helper that can run in an MMU-off path.
## Phase 3: Git History Investigation
Record: local `HEAD` is a stable release commit, not the candidate. `git
log --grep` on current history, `master`, and `power-next` did not find
the candidate commit locally, so there was no commit hash available for
`b4 dig -c`.
Record: `git blame` on current `arch/powerpc/lib/vmx-helper.c` shows the
unannotated `enter_vmx_ops()` present in the local tree; due this
repository’s history shape, blame attributes the original lines to a
merge/root history point, so the exact original introduction commit was
not verifiable locally.
Record: no candidate `Fixes:` tag to follow. The companion patch in the
same series has `Fixes: 2ab2d5794f14 ("powerpc/kasan: Disable address
sanitization in kexec paths")`; `git show` confirms that commit disabled
sanitization for PowerPC kexec real-mode paths in 2022.
Record: related recent local history: `6bc9c0a905228` changed VMX
usercopy flow and added export context around the same file. That
affects hunk context for older stable trees, but not the actual one-line
fix.
Record: author context: `git log --author="Sourabh Jain" --
arch/powerpc/...` shows multiple PowerPC kexec/crash/fadump commits.
`MAINTAINERS` lists Madhavan Srinivasan and Michael Ellerman as PowerPC
maintainers; Madhavan signed off the candidate.
Record: dependencies: no source-level prerequisite for the annotation
itself. Function `enter_vmx_ops()` and `__no_sanitize_address` exist in
checked stable tags. Functional completeness is best with the companion
patch fixing `KASAN_SANITIZE_core_$(BITS).o`, because review/test
discussion says both patches together make KASAN kexec succeed.
## Phase 4: Mailing List And External Research
Record: `b4 am` for message id
`20260407124349.1698552-2-sourabhjain@linux.ibm.com` fetched a v3 two-
patch series. It reported total patches: 2, current tree apply clean,
and DKIM signature valid for patch 2.
Record: `b4 am -c` checked for newer revisions and did not report a
newer v4. v3 appears to be the latest fetched revision.
Record: `b4 mbox` fetched the v3 thread. Patch 2 was addressed to
`linuxppc-dev` and CC’d PowerPC/kexec stakeholders including Michael
Ellerman, Madhavan Srinivasan, Mahesh Salgaonkar, Hari Bathini, Daniel
Axtens, Venkat Rao Bagalkote, Aboorva Devarajan, and Ritesh Harjani.
Record: `WebFetch` of lore URLs was blocked by Anubis, but `b4` and the
`yhbt.net` lore mirror provided the thread content.
Record: v2 review discussion verified:
- Ritesh Harjani reviewed patch 2 and said “LGTM,” granting `Reviewed-
by`.
- Aboorva Devarajan reported an actual KASAN-enabled kexec hang on
pseries PowerVM: system reached “kexec: Starting switchover sequence”
and then hung.
- Aboorva tested that “with both the patches applied, kexec completes
successfully with KASAN enabled.”
- v1 annotated both `enter_vmx_ops()` and `exit_vmx_ops()`; v2 removed
`exit_vmx_ops()` annotation and added the explanatory comment. This
shows review-driven narrowing of the fix.
Record: stable-specific discussion: Ritesh explicitly suggested `Cc:
stable@vger.kernel.org` for patch 1. I did not find a direct stable
nomination for patch 2, but patch 2 was reviewed and tested as part of
the same two-patch functional fix.
## Phase 5: Code Semantic Analysis
Record: modified function: `enter_vmx_ops()`.
Record: callers found by exact search:
- `arch/powerpc/lib/copypage_power7.S`
- `arch/powerpc/lib/memcpy_power7.S`
- `arch/powerpc/lib/memcmp_64.S`
Record: relevant kexec call chain verified in code:
`reboot(LINUX_REBOOT_CMD_KEXEC)` -> `kernel_kexec()` ->
`machine_kexec()` -> `default_machine_kexec()` -> `kexec_sequence()` ->
`kexec_copy_flush()` -> `copy_segments()` -> `copy_page()` ->
`copypage_power7.S` -> `enter_vmx_ops()`.
Record: key callees/logic in `enter_vmx_ops()`: `in_interrupt()`,
`preempt_disable()`, and `enable_kernel_altivec()`. In the kexec MMU-off
path, `default_machine_kexec()` sets
`current_thread_info()->preempt_count = HARDIRQ_OFFSET`, and
`include/linux/preempt.h` defines `in_interrupt()` via `irq_count()`,
which observes the hardirq bits.
Record: reachability: reachable from privileged kexec/kdump paths. It is
not unprivileged-user reachable, but kdump reliability is operationally
important.
Record: similar patterns: PowerPC already disables KASAN for sensitive
real-mode/interrupt code in multiple Makefiles and uses
`__no_sanitize_address` in PowerPC interrupt/stack code, so the
attribute is consistent with local practice.
## Phase 6: Cross-Referencing And Stable Tree Analysis
Record: buggy code exists in stable tags checked: `v5.4`, `v5.10`,
`v5.15`, `v6.1`, `v6.6`, `v6.12`, `v6.18`, `v6.19`, and `v7.0` all have
`enter_vmx_ops()` without `__no_sanitize_address`.
Record: the kexec `copy_segments()` -> `copy_page()` path and
`preempt_count = HARDIRQ_OFFSET` were verified in `v5.4`, `v5.10`,
`v5.15`, `v6.1`, and current/v7.0-era code.
Record: `copypage_power7.S` calls `enter_vmx_ops()` in `v5.4`, `v5.10`,
`v5.15`, and current code.
Record: `__no_sanitize_address` exists in compiler headers in checked
stable tags, including `v5.4`, `v5.10`, `v6.1`, `v6.6`, and current.
Record: backport difficulty: clean on current `7.0.y` per `git apply
--check` and `b4`. Older stable trees may need a tiny context adjustment
because `EXPORT_SYMBOL(exit_vmx_usercopy)` is not present in older
versions, but the actual function signature is present and the backport
is straightforward.
Record: related fixes already in stable: not verified as already
present; local stable tags checked do not contain the candidate
annotation.
## Phase 7: Subsystem And Maintainer Context
Record: subsystem is PowerPC architecture library/kexec path:
`arch/powerpc/lib` and `arch/powerpc/kexec`.
Record: criticality: important/platform-specific. It affects PowerPC
systems using kexec/kdump with KASAN enabled, especially pseries/PowerVM
cases verified in review. It is not universal across all architectures.
Record: subsystem activity: local history shows recent PowerPC VMX
usercopy and kexec/crash/fadump activity, and the patch was handled
through the PowerPC maintainer path.
## Phase 8: Impact And Risk Assessment
Record: affected users: PowerPC Book3S/PowerVM-style systems using
kexec/kdump with KASAN-enabled kernels and VMX copy paths.
Record: trigger conditions: kexec/kdump switchover, MMU-off copy path,
KASAN enabled, and `copy_page()` selecting the Power7 VMX copy path. The
triggering operation is privileged, but the failure impacts crash dump
capture and reboot into the new kernel.
Record: failure mode: system hang during kexec/kdump. Review discussion
includes an observed hang after “kexec: Starting switchover sequence,”
and the commit message explains unrecoverable `bad_page_fault` risk with
MMU off. Severity: HIGH to CRITICAL for affected debug/crash-dump
deployments.
Record: benefit: high for affected PowerPC KASAN/kdump users because it
prevents a hard hang and restores kexec completion.
Record: risk: very low. One function annotation plus comment; no logic
change. The main concern is that stable maintainers should also take the
companion Makefile fix for full KASAN/kexec coverage.
## Phase 9: Final Synthesis
Record: evidence for backporting:
- Fixes a real, reviewed, and tested kexec hang with KASAN enabled.
- Aboorva Devarajan reported and tested the failure/fix.
- Ritesh Harjani reviewed the final approach.
- The fix is one function attribute and explanatory comment.
- It affects a serious operational path: kexec/kdump boot into a
new/capture kernel.
- The buggy code and call path exist across active stable versions
checked.
- Backport is clean for current `7.0.y` and trivial for older stable
trees.
Record: evidence against backporting:
- The issue is config/platform specific: PowerPC plus KASAN plus
kexec/kdump plus VMX copy path.
- Older stable trees may require minor context adjustment.
- The commit is part of a two-patch series; the companion Makefile KASAN
fix should be included for the tested complete fix.
Record: unresolved questions:
- Exact original introduction commit for `enter_vmx_ops()` could not be
verified from local history.
- Direct lore `WebFetch` was blocked by Anubis; I used `b4` and a lore
mirror instead.
- I did not build-test the backport.
Stable rules checklist:
1. Obviously correct and tested: yes. The annotation directly suppresses
KASAN instrumentation on the one helper that can execute with MMU
off; tested-by is present.
2. Fixes a real bug: yes. Review discussion reports an actual kexec hang
with KASAN.
3. Important issue: yes. Failure mode is kexec/kdump hang.
4. Small and contained: yes. One file, one function annotation.
5. No new features/APIs: yes.
6. Can apply to stable: yes for current `7.0.y`; older trees may need
trivial context adjustment.
Exception category: not a device ID/quirk/build/doc exception. It is a
normal stability fix.
## Verification
- [Phase 1] Parsed the supplied commit message and fetched patch mbox;
confirmed tags, no `Fixes:` tag in patch 2, and the linked message id.
- [Phase 2] Inspected fetched v3 diff; confirmed `arch/powerpc/lib/vmx-
helper.c` has 8 insertions and 1 deletion, changing only
`enter_vmx_ops()`.
- [Phase 3] Ran `git log --grep` on current history, `master`, and
`power-next`; candidate not present locally.
- [Phase 3] Ran `git blame` around `enter_vmx_ops()`; local history
could not identify true original introduction beyond repository
history shape.
- [Phase 3] Ran `git show 2ab2d5794f14`; confirmed related PowerPC kexec
KASAN sanitization commit from 2022.
- [Phase 4] Ran `b4 am` and `b4 am -c`; fetched v3 two-patch series, no
newer revision reported, patch 2 DKIM valid.
- [Phase 4] Ran `b4 mbox`; confirmed original recipients and patch
content.
- [Phase 4] Used lore mirror for v2 thread; confirmed Ritesh review,
Aboorva tested-by/reported hang, and v1-to-v2 narrowing.
- [Phase 5] Searched call sites; confirmed `copypage_power7.S`,
`memcpy_power7.S`, and `memcmp_64.S` call `enter_vmx_ops()`.
- [Phase 5] Read kexec and assembly code; confirmed kexec path reaches
`copy_page()` with MMU-off logic and `preempt_count = HARDIRQ_OFFSET`.
- [Phase 6] Checked stable tags `v5.4` through `v7.0`; confirmed
`enter_vmx_ops()` exists without the annotation and the relevant
kexec/copy path exists in checked trees.
- [Phase 6] Ran `git apply --check` on current tree for patch 2 and the
series; clean on current `7.0.y`.
- [Phase 7] Checked `MAINTAINERS`; confirmed PowerPC maintainers and
that Madhavan Srinivasan is listed for PowerPC.
- [Phase 8] Verified failure mode from review discussion: KASAN-enabled
kexec hangs after switchover; with both patches applied, kexec
completes.
This should be backported, ideally together with the companion
`powerpc/kdump: fix KASAN sanitization flag for core_$(BITS).o` patch
for the tested complete KASAN/kexec fix.
**YES**
arch/powerpc/lib/vmx-helper.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/lib/vmx-helper.c b/arch/powerpc/lib/vmx-helper.c
index 554b248002b4f..57e897b60db86 100644
--- a/arch/powerpc/lib/vmx-helper.c
+++ b/arch/powerpc/lib/vmx-helper.c
@@ -52,7 +52,14 @@ int exit_vmx_usercopy(void)
}
EXPORT_SYMBOL(exit_vmx_usercopy);
-int enter_vmx_ops(void)
+/*
+ * Can be called from kexec copy_page() path with MMU off. The kexec
+ * code sets preempt_count to HARDIRQ_OFFSET so we return early here.
+ * Since in_interrupt() is always inline, __no_sanitize_address on this
+ * function is sufficient to avoid KASAN shadow memory accesses in real
+ * mode.
+ */
+int __no_sanitize_address enter_vmx_ops(void)
{
if (in_interrupt())
return 0;
--
2.53.0
^ permalink raw reply related
* [PATCH AUTOSEL 7.0-6.1] powerpc/pasemi: Drop redundant res assignment
From: Sasha Levin @ 2026-05-11 22:19 UTC (permalink / raw)
To: patches, stable
Cc: Krzysztof Kozlowski, Madhavan Srinivasan, Sasha Levin, mpe,
linuxppc-dev, linux-kernel
In-Reply-To: <20260511221931.2370053-1-sashal@kernel.org>
From: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
[ Upstream commit f583bd5f64d40e083dde5bb22846c4d93e59d471 ]
Return value of pas_add_bridge() is not used, so code can be simplified
to fix W=1 clang warnings:
arch/powerpc/platforms/pasemi/pci.c:275:6: error: variable 'res' set but not used [-Werror,-Wunused-but-set-variable]
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260317130823.240279-4-krzysztof.kozlowski@oss.qualcomm.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
LLM Generated explanations, may be completely bogus:
## Phase Walkthrough
### Phase 1: Commit Message Forensics
Record 1.1: Subsystem `powerpc/pasemi`; action verb `Drop`; intent is to
remove an unused local variable assignment in PA Semi PCI init.
Record 1.2: Tags found: `Signed-off-by: Krzysztof Kozlowski`, `Signed-
off-by: Madhavan Srinivasan`, `Link: https://patch.msgid.link/2026031713
0823.240279-4-krzysztof.kozlowski@oss.qualcomm.com`. No `Fixes:`,
`Reported-by:`, `Tested-by:`, `Reviewed-by:`, `Acked-by:`, or `Cc:
stable`.
Record 1.3: The commit explicitly describes a Clang `W=1` build warning
promoted to an error: `variable 'res' set but not used
[-Werror,-Wunused-but-set-variable]`. No runtime symptom, crash, data
corruption, or user-visible functional failure is described.
Record 1.4: This is not a hidden runtime bug fix. It is a build-warning
cleanup that can matter when building powerpc with warning-as-error
settings.
### Phase 2: Diff Analysis
Record 2.1: One file changed: `arch/powerpc/platforms/pasemi/pci.c`, `1
insertion(+), 2 deletions(-)`. Only `pas_pci_init()` is modified. Scope
is single-file surgical.
Record 2.2: Before: `pas_pci_init()` declared `int res;` and assigned
`res = pas_add_bridge(np);`, then never read `res`. After: it still
calls `pas_add_bridge(np);` but does not assign the return value. The
execution path and side effects are unchanged.
Record 2.3: Bug category: build warning/build error under specific
compiler options. Mechanism: remove unused-but-set variable. No resource
lifetime, locking, memory safety, reference counting, or logic behavior
changes.
Record 2.4: Fix quality is obviously correct by inspection: the function
call remains, only the unused local storage is removed. Regression risk
is very low because runtime behavior is unchanged.
### Phase 3: Git History Investigation
Record 3.1: `git blame` on current stable code shows the exact changed
lines are present in the checked-out stable tree. Deeper history shows
the unused `res` assignment was introduced by `250a93501d626`
(`powerpc/pasemi: Search for PCI root bus by compatible property`),
first described by `git describe` as `v4.19-rc1~110^2~83`.
Record 3.2: No `Fixes:` tag is present. Manual history identified
`250a93501d626` as the introducing commit for the exact unused
assignment pattern.
Record 3.3: Recent file history shows only the candidate commit and
unrelated treewide allocation changes in `next-20260508`; no
prerequisite pasemi PCI refactor was found.
Record 3.4: The author has other powerpc cleanup commits nearby,
including the sibling PS3 warning fix. `MAINTAINERS` identifies Madhavan
Srinivasan and Michael Ellerman as powerpc maintainers; Madhavan
committed this patch.
Record 3.5: The sibling commit `8333e4916040e` is part of the same
cleanup series but is independent. This pasemi patch applies standalone
to the current 7.0.5 checkout.
### Phase 4: Mailing List And External Research
Record 4.1: `b4 dig -c f583bd5f64d40` found the original submission by
patch-id. It found the January submission and the March resend
corresponding to the commit `Link:`. Direct WebFetch to
lore/patch.msgid.link was blocked by Anubis.
Record 4.2: `b4 dig -w` showed appropriate powerpc maintainers and
LLVM/compiler-warning stakeholders were copied: Madhavan Srinivasan,
Michael Ellerman, Nicholas Piggin, Christophe Leroy, Nathan Chancellor,
LLVM list, linuxppc-dev, and others.
Record 4.3: No bug report link or reporter tag exists. The thread
describes a compiler warning/build-cleanliness issue, not a runtime bug
report.
Record 4.4: The patch is part of a two-patch series with `powerpc/ps3:
Drop redundant result assignment`; the other patch is the same class of
cleanup and is not a dependency.
Record 4.5: The downloaded thread contains no `stable` mention. A direct
lore stable search was attempted but blocked by Anubis, so no stable-
list archive result could be independently verified.
### Phase 5: Code Semantic Analysis
Record 5.1: Modified function: `pas_pci_init()`.
Record 5.2: Callers: `pas_pci_init()` is assigned to `.discover_phbs` in
the PA Semi machine descriptor in
`arch/powerpc/platforms/pasemi/setup.c`. `pas_add_bridge()` is static
and, in `next-20260508`, is called only from `pas_pci_init()`.
Record 5.3: Relevant callees in the affected path include
`pci_set_flags()`, `of_find_compatible_node()`, `pas_add_bridge()`, and
`of_node_put()`. `pas_add_bridge()` allocates/configures the PCI
controller and scans OF ranges/ISA bridge state.
Record 5.4: Reachability is boot-time/platform-init only for PA Semi
systems. It is not syscall-reachable and has no unprivileged runtime
trigger.
Record 5.5: Similar pattern found: sibling commit `8333e4916040e`
removes an unused assignment in PS3 platform code for the same warning
class.
### Phase 6: Stable Tree Analysis
Record 6.1: The exact unused `res = pas_add_bridge(np);` pattern exists
in verified tags `v4.19`, `v5.10`, `v5.15`, `v6.1`, `v6.6`, `v6.12`,
`v6.17`, `v6.18`, `v6.19`, and `v7.0`. It was not present in the same
form in `v4.14` or `v4.9`.
Record 6.2: Backport difficulty is clean for the current 7.0.5 tree,
verified by `git format-patch -1 --stdout f583bd5f64d40 | git apply
--check`. Older stable trees may need minor context adjustment because
surrounding code differs in older releases.
Record 6.3: No earlier same-subject fix exists in `v7.0`; related same-
series warning fixes were found only in `next-20260508`.
### Phase 7: Subsystem Context
Record 7.1: Subsystem is `arch/powerpc/platforms/pasemi`, a platform-
specific powerpc PCI init path. Criticality is peripheral/platform-
specific, not core kernel.
Record 7.2: Recent pasemi file history is low churn. This is mature
platform code with little recent activity.
### Phase 8: Impact And Risk
Record 8.1: Affected population: PA Semi powerpc platform builders,
especially Clang `W=1` builds with powerpc `-Werror` behavior.
Record 8.2: Trigger condition is build-time only. No runtime trigger and
no unprivileged-user trigger were verified.
Record 8.3: Failure mode is build failure under specific warning/error
settings, not crash/corruption/deadlock. Severity is medium for affected
builders, low for runtime users.
Record 8.4: Benefit is narrow but real under the stable build-fix
exception. Risk is extremely low because the generated runtime behavior
should be unchanged: the same function call remains and only an unused
local assignment is removed.
### Phase 9: Final Synthesis
Record 9.1: Evidence for backporting: fixes a verified compiler
warning/error class; code exists across many stable-era releases; patch
is tiny, standalone, and behavior-preserving; applies cleanly to current
stable checkout. Evidence against: no runtime bug, no default-build
failure was verified, no stable nomination, platform-specific impact.
Unresolved: direct stable lore search was blocked, and I did not run a
full Clang `W=1` build.
Record 9.2: Stable rules checklist: obviously correct: yes by
inspection; tested: no explicit `Tested-by` and no local full build;
fixes a real issue: yes as a build-warning/error issue under specific
settings; important: only under build-fix exception, not runtime-
critical; small/contained: yes, 3-line single-function change; no new
feature/API: yes; applies to stable: current 7.0.5 yes, older trees
likely simple but not all tested.
Record 9.3: Exception category: build fix. This is the only reason to
take it; it is not a runtime bug fix.
Record 9.4: Decision: backport as a low-risk build fix for Clang
`W=1`/powerpc warning-as-error builds. The benefit is narrow, but the
patch is minimal, standalone, and behavior-preserving.
## Verification
- [Phase 1] `git show --format=fuller --stat --patch f583bd5f64d40`
verified commit message, tags, and exact diff.
- [Phase 2] Diff verified one file, one function, `1 insertion`, `2
deletions`.
- [Phase 3] `git blame` and `git show 250a93501d626` verified the unused
assignment pattern was introduced by `250a93501d626`, described as
before `v4.19-rc1`.
- [Phase 4] `b4 dig -c`, `-a`, `-w`, and `b4 mbox` verified the patch
submission, resend, recipients, and lack of candidate-specific stable
tags in the downloaded thread.
- [Phase 5] `rg` and `git grep` verified `pas_pci_init()` call placement
and `pas_add_bridge()` call sites.
- [Phase 6] `git grep` verified the pattern in `v4.19`, `v5.10`,
`v5.15`, `v6.1`, `v6.6`, `v6.12`, `v6.17`, `v6.18`, `v6.19`, and
`v7.0`; `git apply --check` verified clean apply to current 7.0.5.
- [Phase 7] `MAINTAINERS` verified powerpc maintainers and subsystem
ownership.
- [Phase 8] `arch/powerpc/Kconfig.debug` and `arch/powerpc/Kbuild`
verified powerpc warning-as-error configuration exists.
- UNVERIFIED: I did not run a full Clang `W=1` build, and direct lore
stable search was blocked by Anubis.
**YES**
arch/powerpc/platforms/pasemi/pci.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/pasemi/pci.c b/arch/powerpc/platforms/pasemi/pci.c
index 60f990a336c47..2df9552746529 100644
--- a/arch/powerpc/platforms/pasemi/pci.c
+++ b/arch/powerpc/platforms/pasemi/pci.c
@@ -272,13 +272,12 @@ void __init pas_pci_init(void)
{
struct device_node *root = of_find_node_by_path("/");
struct device_node *np;
- int res;
pci_set_flags(PCI_SCAN_ALL_PCIE_DEVS);
np = of_find_compatible_node(root, NULL, "pasemi,rootbus");
if (np) {
- res = pas_add_bridge(np);
+ pas_add_bridge(np);
of_node_put(np);
}
of_node_put(root);
--
2.53.0
^ permalink raw reply related
* [PATCH AUTOSEL 7.0-6.18] powerpc/pseries/htmdump: Free the global buffers in htmdump module exit
From: Sasha Levin @ 2026-05-11 22:19 UTC (permalink / raw)
To: patches, stable
Cc: Athira Rajeev, Madhavan Srinivasan, Sasha Levin, mpe,
linuxppc-dev, linux-kernel
In-Reply-To: <20260511221931.2370053-1-sashal@kernel.org>
From: Athira Rajeev <atrajeev@linux.ibm.com>
[ Upstream commit 54d54f33813d7911555226b4220737177a2ba8d6 ]
htmdump modules uses global memory buffers to capture
details like capabilities, status of specified HTM, read the
trace buffer. These are initialized during module init and
hence needs to be freed in module exit.
Patch adds freeing of the memory in module exit. The change
also includes minor clean up for the variable name. The
read call back for the debugfs interface file saves filp->private_data
to local variable name which is same as global variable
name for the memory buffers. Rename these local variable
names.
Signed-off-by: Athira Rajeev <atrajeev@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260314132432.25581-1-atrajeev@linux.ibm.com
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
LLM Generated explanations, may be completely bogus:
## Summary
This commit fixes a real resource leak in
`arch/powerpc/platforms/pseries/htmdump.c`: `htm_status_buf`,
`htm_info_buf`, and `htm_caps_buf` are each allocated with
`kmalloc(PAGE_SIZE, GFP_KERNEL)` during module/debugfs initialization,
but only `htm_buf` is freed in `htmdump_exit()`. The patch adds the
three missing `kfree()` calls. The local-variable renames are cleanup
and are also used by the following two patches in the same posted
series.
The benefit is modest because the leak is on module unload, not a hot
path, and the feature is PowerPC pseries/debugfs-specific. But it is a
real leak in a `default m` module, the fix is trivial, isolated, and low
risk. For stable trees that contain the v6.16-era `htmstatus`,
`htminfo`, and `htmcaps` additions, this is suitable.
## Phase Walkthrough
### Phase 1: Commit Message Forensics
Record 1.1: Subsystem is `powerpc/pseries/htmdump`; action verb is
`Free`; claimed intent is freeing global HTM debugfs buffers during
module exit.
Record 1.2: Tags in the supplied commit message: `Signed-off-by: Athira
Rajeev <atrajeev@linux.ibm.com>`, `Signed-off-by: Madhavan Srinivasan
<maddy@linux.ibm.com>`, `Link:
https://patch.msgid.link/20260314132432.25581-1-atrajeev@linux.ibm.com`.
No `Fixes:`, `Reported-by:`, `Tested-by:`, `Reviewed-by:`, `Acked-by:`,
or `Cc: stable@vger.kernel.org` tag was present in this commit.
Record 1.3: The body describes global memory buffers initialized in
module init and not freed in module exit. The user-visible symptom is
unreclaimed kernel memory after unloading the module. No crash, stack
trace, reporter, affected-version statement, or reproducer is given.
Record 1.4: This is a hidden cleanup-style bug fix: the functional
change is missing resource release on module exit.
### Phase 2: Diff Analysis
Record 2.1: One file changed:
`arch/powerpc/platforms/pseries/htmdump.c`, 17 insertions and 14
deletions in the posted patch. Modified functions are `htmdump_read()`,
`htmstatus_read()`, `htminfo_read()`, `htmcaps_read()`, and
`htmdump_exit()`. Scope is single-file surgical.
Record 2.2: The read-callback hunks rename local variables from names
shadowing globals to `*_data`; behavior is unchanged. The exit hunk
changes `htmdump_exit()` from freeing only `htm_buf` to freeing
`htm_buf`, `htm_status_buf`, `htm_info_buf`, and `htm_caps_buf`.
Record 2.3: Bug category is resource leak. Verified allocations are
three `PAGE_SIZE` buffers for status/info/caps in
`htmdump_init_debugfs()` without corresponding frees before this patch.
Record 2.4: Fix quality is high: `kfree(NULL)` is safe, the buffers are
global module-owned allocations, and debugfs removal already happens
before freeing. Regression risk is very low. The only non-functional
churn is local-variable rename.
### Phase 3: Git History Investigation
Record 3.1: `git blame` on the current stable checkout points the
relevant lines at the repository boundary merge `f3f5edc5e41e0`, so I
followed explicit commit objects. `htm_status_buf` was added by
`627cf584f4c3`, `htm_info_buf` by `dea7384e14e7`, and `htm_caps_buf` by
`143a2584627c`, all described by `git describe --contains`/history as
v6.16-rc1-era commits.
Record 3.2: Candidate has no `Fixes:` tag, so no direct Fixes target
applies. Related series patches 2/3 and 3/3 have `Fixes:` tags for
`dea7384e14e7` and `627cf584f4c3`, but this patch does not depend on
those fixes.
Record 3.3: Recent `htmdump` history shows the v6.16 additions for
status/info/setup/flags/caps and a header include fix. The candidate is
patch 1/3 in a series; patch 1 is standalone, while patches 2/3 and 3/3
textually build on its variable renames.
Record 3.4: Author Athira Rajeev authored the original htmdump expansion
commits; Madhavan Srinivasan committed them. This was verified from `git
show` for the status/info/caps commits.
Record 3.5: No prerequisite is needed for patch 1 beyond the buffers
existing. An isolated `git apply --check` of patch 1 succeeds on the
current 7.0 stable checkout.
### Phase 4: Mailing List And External Research
Record 4.1: `b4 dig` could not be meaningfully run because the candidate
commit object/hash is not present in the checked-out refs; `b4 dig -h`
confirmed it requires `-c <commitish>`. I used `b4 mbox -c` with
message-id `20260314132432.25581-1-atrajeev@linux.ibm.com`; it fetched a
3-message thread and reported no newer revision.
Record 4.2: Original recipients from the mbox were
`maddy@linux.ibm.com`, `linuxppc-dev@lists.ozlabs.org`, and CCs
including `hbathini`, `sshegde`, and IBM HTM-related recipients. No
review replies, NAKs, or stable nominations were present in the fetched
mbox.
Record 4.3: No `Reported-by` or bug-report link exists for this
candidate. `WebFetch` to lore/patch.msgid was blocked by Anubis, but `b4
mbox` successfully fetched the thread.
Record 4.4: The patch is part of a 3-patch series. Patch 1 fixes exit
cleanup; patch 2 fixes `htminfo_read()` offset handling; patch 3 fixes
`htmstatus_read()` offset handling. Patch 1 can apply standalone, and
later patches depend textually on its renames.
Record 4.5: Stable-specific lore WebFetch was blocked by Anubis.
WebSearch for the exact subject and stable context found no specific
stable discussion. Local `git log --grep` checks of `v6.18`, `v6.19`,
`v7.0`, and `linux-rolling-stable` found no exact-subject match.
### Phase 5: Code Semantic Analysis
Record 5.1: Modified functions are `htmdump_read()`, `htmstatus_read()`,
`htminfo_read()`, `htmcaps_read()`, and `htmdump_exit()`.
Record 5.2: Callers/entry points are verified through file operations
and module macros: `htmdump_exit()` is registered with `module_exit()`,
reads are reached through debugfs files created by
`debugfs_create_file()` with the corresponding fops.
Record 5.3: Key callees are `debugfs_remove_recursive()`, `kfree()`,
`htm_hcall_wrapper()`, `virt_to_phys()`, and
`simple_read_from_buffer()`. The functional fix only adds `kfree()`
calls in exit.
Record 5.4: Reachability is module lifecycle and debugfs based. The leak
triggers on module unload after successful allocation of the
status/info/caps buffers. Unprivileged triggering was not verified and
is not relied on; Kconfig shows `HTMDUMP` is `tristate`, `default m`,
and depends on `PPC_PSERIES && DEBUG_FS`.
Record 5.5: Similar pattern found: `htm_buf` was already freed in
`htmdump_exit()`, while the three later-added global buffers were not.
### Phase 6: Stable Tree Analysis
Record 6.1: `v6.15` has `htmdump.c` with only `htm_buf`; `v6.16`,
`v6.17`, `v6.18`, `v6.19`, and `v7.0` have the three additional buffers
and only free `htm_buf`. So the bug affects stable trees containing the
v6.16 htmdump expansions, not older trees lacking those buffers.
Record 6.2: Backport difficulty is low for affected trees. The patch’s
base blob matches `v6.16`/`v6.17`/current 7.0 content for the relevant
file, and isolated patch 1 applies cleanly to the current checkout.
Record 6.3: No exact-subject related fix was found in checked `v6.18`,
`v6.19`, `v7.0`, or `linux-rolling-stable`.
### Phase 7: Subsystem And Maintainer Context
Record 7.1: Subsystem is PowerPC pseries platform driver/debugfs
support. Criticality is peripheral/platform-specific, not core kernel-
wide.
Record 7.2: `git log -20 v7.0 -- arch/powerpc/platforms/pseries` shows
active pseries development, but this particular fix is contained to
`htmdump.c`.
### Phase 8: Impact And Risk
Record 8.1: Affected users are PowerPC pseries users with
`CONFIG_HTMDUMP` enabled, primarily using the debugfs HTM dump module.
Record 8.2: Trigger is successful module initialization followed by
module exit/unload. Repeated load/unload can accumulate unreclaimed
pages until reboot. This is not verified as unprivileged-triggerable.
Record 8.3: Failure mode is kernel memory/resource leak, three
`PAGE_SIZE` allocations per unload for the later buffers. Severity is
medium: real and persistent until reboot, but not a crash/security/data-
corruption fix and not a hot path.
Record 8.4: Benefit is moderate for affected stable users; risk is very
low because the functional change is only three `kfree()` calls after
debugfs removal.
### Phase 9: Final Synthesis
Record 9.1: Evidence for backporting: real resource leak, affects stable
trees v6.16 and newer that contain the expanded htmdump module, fix is
tiny and obviously correct, isolated patch applies cleanly. Evidence
against: peripheral platform-specific module, leak occurs on module
unload rather than common runtime path, no reporter/testing/stable tags
in this patch. Unresolved: no full lore review discussion was available
beyond the 3-patch mbox; direct WebFetch lore pages were blocked.
Record 9.2: Stable rules checklist: obviously correct yes; fixes a real
bug yes; important enough is borderline but acceptable as a real
resource leak with very low regression risk; small and contained yes; no
new feature/API yes; applies to affected stable trees yes for current
7.0 and likely cleanly where file matches v6.16+.
Record 9.3: No automatic exception category applies: this is not a
device ID, quirk, DT update, build fix, or documentation-only fix.
Record 9.4: Decision is YES for stable trees that contain
`htm_status_buf`, `htm_info_buf`, and `htm_caps_buf`.
## Verification
- Phase 1: Parsed supplied commit message and b4-fetched patch 1;
confirmed no `Fixes:`, `Reported-by`, `Tested-by`, review, or stable
tag in patch 1.
- Phase 2: Read `arch/powerpc/platforms/pseries/htmdump.c`; confirmed
allocations and missing frees; inspected patch hunks from
`/tmp/htmdump-20260314.mbx`.
- Phase 3: Ran `git blame -L 410,485`; followed explicit commits
`627cf584f4c3`, `dea7384e14e7`, and `143a2584627c` with `git show`;
checked author/subsystem history.
- Phase 4: `b4 mbox -c` fetched the 3-message thread; direct `WebFetch`
lore URLs were blocked; WebSearch found no exact stable discussion.
- Phase 5: Used `rg`, `ReadFile`, and Kconfig/Makefile/docs to verify
module/debugfs reachability and `CONFIG_HTMDUMP` as `tristate`,
`default m`.
- Phase 6: Checked `v6.15`, `v6.16`, `v6.17`, `v6.18`, `v6.19`, and
`v7.0` file contents; isolated patch 1 passed `git apply --check`.
- Phase 8: Verified failure mode from concrete `kmalloc(PAGE_SIZE)`
allocations and missing matching `kfree()` calls.
- Unverified: whether every downstream stable branch has identical
context; direct stable-list lore search was blocked by Anubis.
**YES**
arch/powerpc/platforms/pseries/htmdump.c | 31 +++++++++++++-----------
1 file changed, 17 insertions(+), 14 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/htmdump.c b/arch/powerpc/platforms/pseries/htmdump.c
index 742ec52c9d4df..93f0cc2dc7fb6 100644
--- a/arch/powerpc/platforms/pseries/htmdump.c
+++ b/arch/powerpc/platforms/pseries/htmdump.c
@@ -86,7 +86,7 @@ static ssize_t htm_return_check(long rc)
static ssize_t htmdump_read(struct file *filp, char __user *ubuf,
size_t count, loff_t *ppos)
{
- void *htm_buf = filp->private_data;
+ void *htm_buf_data = filp->private_data;
unsigned long page, read_size, available;
loff_t offset;
long rc, ret;
@@ -100,7 +100,7 @@ static ssize_t htmdump_read(struct file *filp, char __user *ubuf,
* - last three values are address, size and offset
*/
rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip,
- htmtype, H_HTM_OP_DUMP_DATA, virt_to_phys(htm_buf),
+ htmtype, H_HTM_OP_DUMP_DATA, virt_to_phys(htm_buf_data),
PAGE_SIZE, page);
ret = htm_return_check(rc);
@@ -112,7 +112,7 @@ static ssize_t htmdump_read(struct file *filp, char __user *ubuf,
available = PAGE_SIZE;
read_size = min(count, available);
*ppos += read_size;
- return simple_read_from_buffer(ubuf, count, &offset, htm_buf, available);
+ return simple_read_from_buffer(ubuf, count, &offset, htm_buf_data, available);
}
static const struct file_operations htmdump_fops = {
@@ -226,7 +226,7 @@ static int htmstart_get(void *data, u64 *val)
static ssize_t htmstatus_read(struct file *filp, char __user *ubuf,
size_t count, loff_t *ppos)
{
- void *htm_status_buf = filp->private_data;
+ void *htm_status_data = filp->private_data;
long rc, ret;
u64 *num_entries;
u64 to_copy;
@@ -238,7 +238,7 @@ static ssize_t htmstatus_read(struct file *filp, char __user *ubuf,
* - last three values as addr, size and offset
*/
rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip,
- htmtype, H_HTM_OP_STATUS, virt_to_phys(htm_status_buf),
+ htmtype, H_HTM_OP_STATUS, virt_to_phys(htm_status_data),
PAGE_SIZE, 0);
ret = htm_return_check(rc);
@@ -255,13 +255,13 @@ static ssize_t htmstatus_read(struct file *filp, char __user *ubuf,
* So total count to copy is:
* 32 bytes (for first 7 fields) + (number of HTM entries * entry size)
*/
- num_entries = htm_status_buf + 0x10;
+ num_entries = htm_status_data + 0x10;
if (htmtype == 0x2)
htmstatus_flag = 0x8;
else
htmstatus_flag = 0x6;
to_copy = 32 + (be64_to_cpu(*num_entries) * htmstatus_flag);
- return simple_read_from_buffer(ubuf, count, ppos, htm_status_buf, to_copy);
+ return simple_read_from_buffer(ubuf, count, ppos, htm_status_data, to_copy);
}
static const struct file_operations htmstatus_fops = {
@@ -273,7 +273,7 @@ static const struct file_operations htmstatus_fops = {
static ssize_t htminfo_read(struct file *filp, char __user *ubuf,
size_t count, loff_t *ppos)
{
- void *htm_info_buf = filp->private_data;
+ void *htm_info_data = filp->private_data;
long rc, ret;
u64 *num_entries;
u64 to_copy;
@@ -284,7 +284,7 @@ static ssize_t htminfo_read(struct file *filp, char __user *ubuf,
* - last three values as addr, size and offset
*/
rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip,
- htmtype, H_HTM_OP_DUMP_SYSPROC_CONF, virt_to_phys(htm_info_buf),
+ htmtype, H_HTM_OP_DUMP_SYSPROC_CONF, virt_to_phys(htm_info_data),
PAGE_SIZE, 0);
ret = htm_return_check(rc);
@@ -301,15 +301,15 @@ static ssize_t htminfo_read(struct file *filp, char __user *ubuf,
* So total count to copy is:
* 32 bytes (for first 5 fields) + (number of HTM entries * entry size)
*/
- num_entries = htm_info_buf + 0x10;
+ num_entries = htm_info_data + 0x10;
to_copy = 32 + (be64_to_cpu(*num_entries) * 16);
- return simple_read_from_buffer(ubuf, count, ppos, htm_info_buf, to_copy);
+ return simple_read_from_buffer(ubuf, count, ppos, htm_info_data, to_copy);
}
static ssize_t htmcaps_read(struct file *filp, char __user *ubuf,
size_t count, loff_t *ppos)
{
- void *htm_caps_buf = filp->private_data;
+ void *htm_caps_data = filp->private_data;
long rc, ret;
/*
@@ -319,7 +319,7 @@ static ssize_t htmcaps_read(struct file *filp, char __user *ubuf,
* and zero
*/
rc = htm_hcall_wrapper(htmflags, nodeindex, nodalchipindex, coreindexonchip,
- htmtype, H_HTM_OP_CAPABILITIES, virt_to_phys(htm_caps_buf),
+ htmtype, H_HTM_OP_CAPABILITIES, virt_to_phys(htm_caps_data),
0x80, 0);
ret = htm_return_check(rc);
@@ -328,7 +328,7 @@ static ssize_t htmcaps_read(struct file *filp, char __user *ubuf,
return ret;
}
- return simple_read_from_buffer(ubuf, count, ppos, htm_caps_buf, 0x80);
+ return simple_read_from_buffer(ubuf, count, ppos, htm_caps_data, 0x80);
}
static const struct file_operations htminfo_fops = {
@@ -482,6 +482,9 @@ static void __exit htmdump_exit(void)
{
debugfs_remove_recursive(htmdump_debugfs_dir);
kfree(htm_buf);
+ kfree(htm_status_buf);
+ kfree(htm_info_buf);
+ kfree(htm_caps_buf);
}
module_init(htmdump_init);
--
2.53.0
^ permalink raw reply related
* Re: [PATCH] drivers/base/memory: make memory block get/put explicit
From: Muchun Song @ 2026-05-12 3:11 UTC (permalink / raw)
To: David Hildenbrand
Cc: Muchun Song, Oscar Salvador, Greg Kroah-Hartman,
Rafael J. Wysocki, Danilo Krummrich, Andrew Morton,
Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport,
Suren Baghdasaryan, Michal Hocko, Madhavan Srinivasan,
Michael Ellerman, Nicholas Piggin, Christophe Leroy,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger, Sven Schnelle, linux-mm, driver-core,
linux-kernel, linuxppc-dev, linux-s390
In-Reply-To: <3234B4D3-8202-4E79-B85B-8B6373BB76F6@linux.dev>
> On May 11, 2026, at 21:23, Muchun Song <muchun.song@linux.dev> wrote:
>
>
>
>> On May 11, 2026, at 20:22, David Hildenbrand (Arm) <david@kernel.org> wrote:
>>
>> On 5/11/26 13:18, Muchun Song wrote:
>>> Rename the memory block lookup helper to make the acquired reference
>>> explicit, add memory_block_put() to wrap put_device(), and collapse the
>>> redundant section-number wrapper into a single block-id based lookup
>>> interface.
>>>
>>> This makes it clearer to callers that a successful lookup holds a
>>> reference that must be dropped, reducing the chance of forgetting the
>>> matching put and leaking the memory block device reference.
>>
>> Better mention some of the other changes here, like removing find_memory_block().
>
> Will do.
>
>>
>> [...]
>>
>>> unlock_device_hotplug();
>>> diff --git a/include/linux/memory.h b/include/linux/memory.h
>>> index 5bb5599c6b2b..29edef1f975c 100644
>>> --- a/include/linux/memory.h
>>> +++ b/include/linux/memory.h
>>> @@ -158,7 +158,11 @@ int create_memory_block_devices(unsigned long start, unsigned long size,
>>> void remove_memory_block_devices(unsigned long start, unsigned long size);
>>> extern void memory_dev_init(void);
>>> extern int memory_notify(enum memory_block_state state, void *v);
>>> -extern struct memory_block *find_memory_block(unsigned long section_nr);
>>> +extern struct memory_block *memory_block_get(unsigned long block_id);
>>
>> While at it, please drop the "extern".
>
> OK.
>
>>
>>> +static inline void memory_block_put(struct memory_block *mem)
>>> +{
>>> + put_device(&mem->dev);
>>> +}
>>> typedef int (*walk_memory_blocks_func_t)(struct memory_block *, void *);
>>> extern int walk_memory_blocks(unsigned long start, unsigned long size,
>>> void *arg, walk_memory_blocks_func_t func);
>>> @@ -171,7 +175,6 @@ struct memory_group *memory_group_find_by_id(int mgid);
>>> typedef int (*walk_memory_groups_func_t)(struct memory_group *, void *);
>>> int walk_dynamic_memory_groups(int nid, walk_memory_groups_func_t func,
>>> struct memory_group *excluded, void *arg);
>>> -struct memory_block *find_memory_block_by_id(unsigned long block_id);
>>> #define hotplug_memory_notifier(fn, pri) ({ \
>>> static __meminitdata struct notifier_block fn##_mem_nb =\
>>> { .notifier_call = fn, .priority = pri };\
>>> diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
>>> index 462d8dcd636d..890c6453e887 100644
>>> --- a/mm/memory_hotplug.c
>>> +++ b/mm/memory_hotplug.c
>>> @@ -1417,14 +1417,13 @@ static void remove_memory_blocks_and_altmaps(u64 start, u64 size)
>>> struct vmem_altmap *altmap = NULL;
>>> struct memory_block *mem;
>>>
>>> - mem = find_memory_block(pfn_to_section_nr(PFN_DOWN(cur_start)));
>>> + mem = memory_block_get(phys_to_block_id(cur_start));
>>> if (WARN_ON_ONCE(!mem))
>>> continue;
>>>
>>> altmap = mem->altmap;
>>> mem->altmap = NULL;
>>> - /* drop the ref. we got via find_memory_block() */
>>> - put_device(&mem->dev);
>>> + memory_block_put(mem);
>>
>> Would guards come in handy here?
>
> You mean to introduce something like:
>
> scoped_guard(memory_block, id) {
> }
>
> Right? If yes, I will give it a try.
Hi David,
Did I get that right?
5 files changed, 26 insertions(+), 35 deletions(-)
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index 5b5d41089e81..eba48320fc30 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -997,7 +997,6 @@ int walk_memory_blocks(unsigned long start, unsigned long size,
{
const unsigned long start_block_id = phys_to_block_id(start);
const unsigned long end_block_id = phys_to_block_id(start + size - 1);
- struct memory_block *mem;
unsigned long block_id;
int ret = 0;
@@ -1005,12 +1004,11 @@ int walk_memory_blocks(unsigned long start, unsigned long size,
return 0;
for (block_id = start_block_id; block_id <= end_block_id; block_id++) {
- mem = memory_block_get(block_id);
+ CLASS(memory_block_get, mem)(block_id);
if (!mem)
continue;
ret = func(mem, arg);
- memory_block_put(mem);
if (ret)
break;
}
@@ -1218,23 +1216,19 @@ int walk_dynamic_memory_groups(int nid, walk_memory_groups_func_t func,
void memblk_nr_poison_inc(unsigned long pfn)
{
const unsigned long block_id = pfn_to_block_id(pfn);
- struct memory_block *mem = memory_block_get(block_id);
+ CLASS(memory_block_get, mem)(block_id);
- if (mem) {
+ if (mem)
atomic_long_inc(&mem->nr_hwpoison);
- memory_block_put(mem);
- }
}
void memblk_nr_poison_sub(unsigned long pfn, long i)
{
const unsigned long block_id = pfn_to_block_id(pfn);
- struct memory_block *mem = memory_block_get(block_id);
+ CLASS(memory_block_get, mem)(block_id);
- if (mem) {
+ if (mem)
atomic_long_sub(i, &mem->nr_hwpoison);
- memory_block_put(mem);
- }
}
static unsigned long memblk_nr_poison(struct memory_block *mem)
diff --git a/drivers/base/node.c b/drivers/base/node.c
index b3333ca92090..32a5431cbb60 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -845,15 +845,12 @@ static void register_memory_blocks_under_nodes(void)
continue;
for (block_id = start_block_id; block_id <= end_block_id; block_id++) {
- struct memory_block *mem;
-
- mem = memory_block_get(block_id);
+ CLASS(memory_block_get, mem)(block_id);
if (!mem)
continue;
memory_block_add_nid_early(mem, nid);
do_register_memory_block_under_node(nid, mem);
- memory_block_put(mem);
}
}
diff --git a/drivers/s390/char/sclp_mem.c b/drivers/s390/char/sclp_mem.c
index 6df1926d4c62..c9a1a7adcab6 100644
--- a/drivers/s390/char/sclp_mem.c
+++ b/drivers/s390/char/sclp_mem.c
@@ -237,13 +237,12 @@ static ssize_t sclp_config_mem_store(struct kobject *kobj, struct kobj_attribute
} else {
if (!sclp_mem->config)
goto out_unlock;
- mem = memory_block_get(phys_to_block_id(addr));
- if (mem->state != MEM_OFFLINE) {
- memory_block_put(mem);
- rc = -EBUSY;
- goto out_unlock;
+ scoped_class(memory_block_get, mem, phys_to_block_id(addr)) {
+ if (mem->state != MEM_OFFLINE) {
+ rc = -EBUSY;
+ goto out_unlock;
+ }
}
- memory_block_put(mem);
sclp_mem_change_state(addr, block_size, 0);
__remove_memory(addr, block_size);
#ifdef CONFIG_KASAN
@@ -293,12 +292,12 @@ static ssize_t sclp_memmap_on_memory_store(struct kobject *kobj, struct kobj_att
return rc;
block_size = memory_block_size_bytes();
sclp_mem = container_of(kobj, struct sclp_mem, kobj);
- mem = memory_block_get(phys_to_block_id(sclp_mem->id * block_size));
- if (!mem) {
- WRITE_ONCE(sclp_mem->memmap_on_memory, value);
- } else {
- memory_block_put(mem);
- rc = -EBUSY;
+ scoped_class(memory_block_get, mem,
+ phys_to_block_id(sclp_mem->id * block_size)) {
+ if (!mem)
+ WRITE_ONCE(sclp_mem->memmap_on_memory, value);
+ else
+ rc = -EBUSY;
}
unlock_device_hotplug();
return rc ? rc : count;
diff --git a/include/linux/memory.h b/include/linux/memory.h
index 463dc02f6cff..fb67041941a8 100644
--- a/include/linux/memory.h
+++ b/include/linux/memory.h
@@ -18,6 +18,7 @@
#include <linux/node.h>
#include <linux/compiler.h>
+#include <linux/cleanup.h>
#include <linux/mutex.h>
#include <linux/memory_hotplug.h>
@@ -163,6 +164,8 @@ static inline void memory_block_put(struct memory_block *mem)
{
put_device(&mem->dev);
}
+DEFINE_CLASS(memory_block_get, struct memory_block *, if (_T) memory_block_put(_T),
+ memory_block_get(block_id), unsigned long block_id)
typedef int (*walk_memory_blocks_func_t)(struct memory_block *, void *);
extern int walk_memory_blocks(unsigned long start, unsigned long size,
void *arg, walk_memory_blocks_func_t func);
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 890c6453e887..3f89ffb286d6 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1415,15 +1415,13 @@ static void remove_memory_blocks_and_altmaps(u64 start, u64 size)
for (cur_start = start; cur_start < start + size;
cur_start += memblock_size) {
struct vmem_altmap *altmap = NULL;
- struct memory_block *mem;
-
- mem = memory_block_get(phys_to_block_id(cur_start));
- if (WARN_ON_ONCE(!mem))
- continue;
+ scoped_class(memory_block_get, mem, phys_to_block_id(cur_start)) {
+ if (WARN_ON_ONCE(!mem))
+ continue;
- altmap = mem->altmap;
- mem->altmap = NULL;
- memory_block_put(mem);
+ altmap = mem->altmap;
+ mem->altmap = NULL;
+ }
remove_memory_block_devices(cur_start, memblock_size);
Thanks,
Muchun
>
>>
>> In general
>>
>> Acked-by: David Hildenbrand (Arm) <david@kernel.org>
>
> Thanks.
>
> Muchun
>
>>
>> --
>> Cheers,
>>
>> David
^ permalink raw reply related
* [PATCH 01/19] btrfs: require at least 4 devices for RAID 6
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
While the RAID6 algorithm could in theory support 3 devices by just
copying the data disk to the two parity disks, this version is not only
useless because it is a suboptimal version of 3-way mirroring, but also
broken with various crashes and incorrect parity generation in various
architecture-optimized implementations. Disallow it similar to mdraid
which requires at least 4 devices for RAID 6.
Fixes: 53b381b3abeb ("Btrfs: RAID5 and RAID6")
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
fs/btrfs/volumes.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index a88e68f90564..0b54b97bdad8 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -159,7 +159,7 @@ const struct btrfs_raid_attr btrfs_raid_array[BTRFS_NR_RAID_TYPES] = {
.sub_stripes = 1,
.dev_stripes = 1,
.devs_max = 0,
- .devs_min = 3,
+ .devs_min = 4,
.tolerated_failures = 2,
.devs_increment = 1,
.ncopies = 1,
--
2.53.0
^ permalink raw reply related
* [PATCH 03/19] raid6: remove __KERNEL__ ifdefs
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
With the test code ported to kernel space, none of this is required.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/raid/pq.h | 90 --------------------------------
lib/raid6/algos.c | 12 -----
lib/raid6/altivec.uc | 10 +---
lib/raid6/avx2.c | 2 +-
lib/raid6/avx512.c | 2 +-
lib/raid6/loongarch.h | 38 --------------
lib/raid6/loongarch_simd.c | 3 +-
lib/raid6/mktables.c | 14 -----
lib/raid6/mmx.c | 2 +-
lib/raid6/neon.c | 6 ---
lib/raid6/recov_avx2.c | 2 +-
lib/raid6/recov_avx512.c | 2 +-
lib/raid6/recov_loongarch_simd.c | 3 +-
lib/raid6/recov_neon.c | 6 ---
lib/raid6/recov_ssse3.c | 2 +-
lib/raid6/rvv.h | 11 +---
lib/raid6/sse1.c | 2 +-
lib/raid6/sse2.c | 2 +-
lib/raid6/vpermxor.uc | 7 ---
lib/raid6/x86.h | 75 --------------------------
20 files changed, 15 insertions(+), 276 deletions(-)
delete mode 100644 lib/raid6/loongarch.h
delete mode 100644 lib/raid6/x86.h
diff --git a/include/linux/raid/pq.h b/include/linux/raid/pq.h
index 08c5995ea980..d26788fada58 100644
--- a/include/linux/raid/pq.h
+++ b/include/linux/raid/pq.h
@@ -8,8 +8,6 @@
#ifndef LINUX_RAID_RAID6_H
#define LINUX_RAID_RAID6_H
-#ifdef __KERNEL__
-
#include <linux/blkdev.h>
#include <linux/mm.h>
@@ -19,59 +17,6 @@ static inline void *raid6_get_zero_page(void)
return page_address(ZERO_PAGE(0));
}
-#else /* ! __KERNEL__ */
-/* Used for testing in user space */
-
-#include <errno.h>
-#include <inttypes.h>
-#include <stddef.h>
-#include <string.h>
-#include <sys/mman.h>
-#include <sys/time.h>
-#include <sys/types.h>
-
-/* Not standard, but glibc defines it */
-#define BITS_PER_LONG __WORDSIZE
-
-typedef uint8_t u8;
-typedef uint16_t u16;
-typedef uint32_t u32;
-typedef uint64_t u64;
-
-#ifndef PAGE_SIZE
-# define PAGE_SIZE 4096
-#endif
-#ifndef PAGE_SHIFT
-# define PAGE_SHIFT 12
-#endif
-extern const char raid6_empty_zero_page[PAGE_SIZE];
-
-#define __init
-#define __exit
-#ifndef __attribute_const__
-# define __attribute_const__ __attribute__((const))
-#endif
-#define noinline __attribute__((noinline))
-
-#define preempt_enable()
-#define preempt_disable()
-#define cpu_has_feature(x) 1
-#define enable_kernel_altivec()
-#define disable_kernel_altivec()
-
-#undef EXPORT_SYMBOL
-#define EXPORT_SYMBOL(sym)
-#undef EXPORT_SYMBOL_GPL
-#define EXPORT_SYMBOL_GPL(sym)
-#define MODULE_LICENSE(licence)
-#define MODULE_DESCRIPTION(desc)
-#define subsys_initcall(x)
-#define module_exit(x)
-
-#define IS_ENABLED(x) (x)
-#define CONFIG_RAID6_PQ_BENCHMARK 1
-#endif /* __KERNEL__ */
-
/* Routine choices */
struct raid6_calls {
void (*gen_syndrome)(int, size_t, void **);
@@ -165,39 +110,4 @@ extern void (*raid6_2data_recov)(int disks, size_t bytes, int faila, int failb,
extern void (*raid6_datap_recov)(int disks, size_t bytes, int faila,
void **ptrs);
-/* Some definitions to allow code to be compiled for testing in userspace */
-#ifndef __KERNEL__
-
-# define jiffies raid6_jiffies()
-# define printk printf
-# define pr_err(format, ...) fprintf(stderr, format, ## __VA_ARGS__)
-# define pr_info(format, ...) fprintf(stdout, format, ## __VA_ARGS__)
-# define GFP_KERNEL 0
-# define __get_free_pages(x, y) ((unsigned long)mmap(NULL, PAGE_SIZE << (y), \
- PROT_READ|PROT_WRITE, \
- MAP_PRIVATE|MAP_ANONYMOUS,\
- 0, 0))
-# define free_pages(x, y) munmap((void *)(x), PAGE_SIZE << (y))
-
-static inline void cpu_relax(void)
-{
- /* Nothing */
-}
-
-#undef HZ
-#define HZ 1000
-static inline uint32_t raid6_jiffies(void)
-{
- struct timeval tv;
- gettimeofday(&tv, NULL);
- return tv.tv_sec*1000 + tv.tv_usec/1000;
-}
-
-static inline void *raid6_get_zero_page(void)
-{
- return raid6_empty_zero_page;
-}
-
-#endif /* ! __KERNEL__ */
-
#endif /* LINUX_RAID_RAID6_H */
diff --git a/lib/raid6/algos.c b/lib/raid6/algos.c
index 5a9f4882e18d..985c60bb00a4 100644
--- a/lib/raid6/algos.c
+++ b/lib/raid6/algos.c
@@ -12,13 +12,8 @@
*/
#include <linux/raid/pq.h>
-#ifndef __KERNEL__
-#include <sys/mman.h>
-#include <stdio.h>
-#else
#include <linux/module.h>
#include <linux/gfp.h>
-#endif
#include <kunit/visibility.h>
struct raid6_calls raid6_call;
@@ -123,14 +118,7 @@ const struct raid6_recov_calls *const raid6_recov_algos[] = {
};
EXPORT_SYMBOL_IF_KUNIT(raid6_recov_algos);
-#ifdef __KERNEL__
#define RAID6_TIME_JIFFIES_LG2 4
-#else
-/* Need more time to be stable in userspace */
-#define RAID6_TIME_JIFFIES_LG2 9
-#define time_before(x, y) ((x) < (y))
-#endif
-
#define RAID6_TEST_DISKS 8
#define RAID6_TEST_DISKS_ORDER 3
diff --git a/lib/raid6/altivec.uc b/lib/raid6/altivec.uc
index d20ed0d11411..2c59963e58f9 100644
--- a/lib/raid6/altivec.uc
+++ b/lib/raid6/altivec.uc
@@ -27,10 +27,8 @@
#ifdef CONFIG_ALTIVEC
#include <altivec.h>
-#ifdef __KERNEL__
-# include <asm/cputable.h>
-# include <asm/switch_to.h>
-#endif /* __KERNEL__ */
+#include <asm/cputable.h>
+#include <asm/switch_to.h>
/*
* This is the C data type to use. We use a vector of
@@ -113,11 +111,7 @@ int raid6_have_altivec(void);
int raid6_have_altivec(void)
{
/* This assumes either all CPUs have Altivec or none does */
-# ifdef __KERNEL__
return cpu_has_feature(CPU_FTR_ALTIVEC);
-# else
- return 1;
-# endif
}
#endif
diff --git a/lib/raid6/avx2.c b/lib/raid6/avx2.c
index 059024234dce..a1a5213918af 100644
--- a/lib/raid6/avx2.c
+++ b/lib/raid6/avx2.c
@@ -14,7 +14,7 @@
*/
#include <linux/raid/pq.h>
-#include "x86.h"
+#include <asm/fpu/api.h>
static const struct raid6_avx2_constants {
u64 x1d[4];
diff --git a/lib/raid6/avx512.c b/lib/raid6/avx512.c
index 009bd0adeebf..874998bcd7d7 100644
--- a/lib/raid6/avx512.c
+++ b/lib/raid6/avx512.c
@@ -18,7 +18,7 @@
*/
#include <linux/raid/pq.h>
-#include "x86.h"
+#include <asm/fpu/api.h>
static const struct raid6_avx512_constants {
u64 x1d[8];
diff --git a/lib/raid6/loongarch.h b/lib/raid6/loongarch.h
deleted file mode 100644
index acfc33ce7056..000000000000
--- a/lib/raid6/loongarch.h
+++ /dev/null
@@ -1,38 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/*
- * Copyright (C) 2023 WANG Xuerui <git@xen0n.name>
- *
- * raid6/loongarch.h
- *
- * Definitions common to LoongArch RAID-6 code only
- */
-
-#ifndef _LIB_RAID6_LOONGARCH_H
-#define _LIB_RAID6_LOONGARCH_H
-
-#ifdef __KERNEL__
-
-#include <asm/cpu-features.h>
-#include <asm/fpu.h>
-
-#else /* for user-space testing */
-
-#include <sys/auxv.h>
-
-/* have to supply these defines for glibc 2.37- and musl */
-#ifndef HWCAP_LOONGARCH_LSX
-#define HWCAP_LOONGARCH_LSX (1 << 4)
-#endif
-#ifndef HWCAP_LOONGARCH_LASX
-#define HWCAP_LOONGARCH_LASX (1 << 5)
-#endif
-
-#define kernel_fpu_begin()
-#define kernel_fpu_end()
-
-#define cpu_has_lsx (getauxval(AT_HWCAP) & HWCAP_LOONGARCH_LSX)
-#define cpu_has_lasx (getauxval(AT_HWCAP) & HWCAP_LOONGARCH_LASX)
-
-#endif /* __KERNEL__ */
-
-#endif /* _LIB_RAID6_LOONGARCH_H */
diff --git a/lib/raid6/loongarch_simd.c b/lib/raid6/loongarch_simd.c
index aa5d9f924ca3..72f4d92d4876 100644
--- a/lib/raid6/loongarch_simd.c
+++ b/lib/raid6/loongarch_simd.c
@@ -10,7 +10,8 @@
*/
#include <linux/raid/pq.h>
-#include "loongarch.h"
+#include <asm/cpu-features.h>
+#include <asm/fpu.h>
/*
* The vector algorithms are currently priority 0, which means the generic
diff --git a/lib/raid6/mktables.c b/lib/raid6/mktables.c
index 3be03793237c..3de1dbf6846c 100644
--- a/lib/raid6/mktables.c
+++ b/lib/raid6/mktables.c
@@ -56,9 +56,7 @@ int main(int argc, char *argv[])
uint8_t v;
uint8_t exptbl[256], invtbl[256];
- printf("#ifdef __KERNEL__\n");
printf("#include <linux/export.h>\n");
- printf("#endif\n");
printf("#include <linux/raid/pq.h>\n");
/* Compute multiplication table */
@@ -76,9 +74,7 @@ int main(int argc, char *argv[])
printf("\t},\n");
}
printf("};\n");
- printf("#ifdef __KERNEL__\n");
printf("EXPORT_SYMBOL(raid6_gfmul);\n");
- printf("#endif\n");
/* Compute vector multiplication table */
printf("\nconst u8 __attribute__((aligned(256)))\n"
@@ -101,9 +97,7 @@ int main(int argc, char *argv[])
printf("\t},\n");
}
printf("};\n");
- printf("#ifdef __KERNEL__\n");
printf("EXPORT_SYMBOL(raid6_vgfmul);\n");
- printf("#endif\n");
/* Compute power-of-2 table (exponent) */
v = 1;
@@ -120,9 +114,7 @@ int main(int argc, char *argv[])
}
}
printf("};\n");
- printf("#ifdef __KERNEL__\n");
printf("EXPORT_SYMBOL(raid6_gfexp);\n");
- printf("#endif\n");
/* Compute log-of-2 table */
printf("\nconst u8 __attribute__((aligned(256)))\n"
@@ -140,9 +132,7 @@ int main(int argc, char *argv[])
}
}
printf("};\n");
- printf("#ifdef __KERNEL__\n");
printf("EXPORT_SYMBOL(raid6_gflog);\n");
- printf("#endif\n");
/* Compute inverse table x^-1 == x^254 */
printf("\nconst u8 __attribute__((aligned(256)))\n"
@@ -155,9 +145,7 @@ int main(int argc, char *argv[])
}
}
printf("};\n");
- printf("#ifdef __KERNEL__\n");
printf("EXPORT_SYMBOL(raid6_gfinv);\n");
- printf("#endif\n");
/* Compute inv(2^x + 1) (exponent-xor-inverse) table */
printf("\nconst u8 __attribute__((aligned(256)))\n"
@@ -169,9 +157,7 @@ int main(int argc, char *argv[])
(j == 7) ? '\n' : ' ');
}
printf("};\n");
- printf("#ifdef __KERNEL__\n");
printf("EXPORT_SYMBOL(raid6_gfexi);\n");
- printf("#endif\n");
return 0;
}
diff --git a/lib/raid6/mmx.c b/lib/raid6/mmx.c
index 3a5bf53a297b..e411f0cfbd95 100644
--- a/lib/raid6/mmx.c
+++ b/lib/raid6/mmx.c
@@ -14,7 +14,7 @@
#ifdef CONFIG_X86_32
#include <linux/raid/pq.h>
-#include "x86.h"
+#include <asm/fpu/api.h>
/* Shared with raid6/sse1.c */
const struct raid6_mmx_constants {
diff --git a/lib/raid6/neon.c b/lib/raid6/neon.c
index 6d9474ce6da9..47b8bb0afc65 100644
--- a/lib/raid6/neon.c
+++ b/lib/raid6/neon.c
@@ -6,13 +6,7 @@
*/
#include <linux/raid/pq.h>
-
-#ifdef __KERNEL__
#include <asm/simd.h>
-#else
-#define scoped_ksimd()
-#define cpu_has_neon() (1)
-#endif
/*
* There are 2 reasons these wrappers are kept in a separate compilation unit
diff --git a/lib/raid6/recov_avx2.c b/lib/raid6/recov_avx2.c
index 97d598d2535c..19fbd9c4dce6 100644
--- a/lib/raid6/recov_avx2.c
+++ b/lib/raid6/recov_avx2.c
@@ -5,7 +5,7 @@
*/
#include <linux/raid/pq.h>
-#include "x86.h"
+#include <asm/fpu/api.h>
static int raid6_has_avx2(void)
{
diff --git a/lib/raid6/recov_avx512.c b/lib/raid6/recov_avx512.c
index 7986120ca444..143f4976b2ad 100644
--- a/lib/raid6/recov_avx512.c
+++ b/lib/raid6/recov_avx512.c
@@ -7,7 +7,7 @@
*/
#include <linux/raid/pq.h>
-#include "x86.h"
+#include <asm/fpu/api.h>
static int raid6_has_avx512(void)
{
diff --git a/lib/raid6/recov_loongarch_simd.c b/lib/raid6/recov_loongarch_simd.c
index 93dc515997a1..eb3a1e79f01f 100644
--- a/lib/raid6/recov_loongarch_simd.c
+++ b/lib/raid6/recov_loongarch_simd.c
@@ -11,7 +11,8 @@
*/
#include <linux/raid/pq.h>
-#include "loongarch.h"
+#include <asm/cpu-features.h>
+#include <asm/fpu.h>
/*
* Unlike with the syndrome calculation algorithms, there's no boot-time
diff --git a/lib/raid6/recov_neon.c b/lib/raid6/recov_neon.c
index 9d99aeabd31a..13d5df718c15 100644
--- a/lib/raid6/recov_neon.c
+++ b/lib/raid6/recov_neon.c
@@ -5,14 +5,8 @@
*/
#include <linux/raid/pq.h>
-
-#ifdef __KERNEL__
#include <asm/simd.h>
#include "neon.h"
-#else
-#define scoped_ksimd()
-#define cpu_has_neon() (1)
-#endif
static int raid6_has_neon(void)
{
diff --git a/lib/raid6/recov_ssse3.c b/lib/raid6/recov_ssse3.c
index 2e849185c32b..146cdbf465bd 100644
--- a/lib/raid6/recov_ssse3.c
+++ b/lib/raid6/recov_ssse3.c
@@ -4,7 +4,7 @@
*/
#include <linux/raid/pq.h>
-#include "x86.h"
+#include <asm/fpu/api.h>
static int raid6_has_ssse3(void)
{
diff --git a/lib/raid6/rvv.h b/lib/raid6/rvv.h
index 6d0708a2c8a4..b0a71b375962 100644
--- a/lib/raid6/rvv.h
+++ b/lib/raid6/rvv.h
@@ -7,17 +7,8 @@
* Definitions for RISC-V RAID-6 code
*/
-#ifdef __KERNEL__
-#include <asm/vector.h>
-#else
-#define kernel_vector_begin()
-#define kernel_vector_end()
-#include <sys/auxv.h>
-#include <asm/hwcap.h>
-#define has_vector() (getauxval(AT_HWCAP) & COMPAT_HWCAP_ISA_V)
-#endif
-
#include <linux/raid/pq.h>
+#include <asm/vector.h>
static int rvv_has_vector(void)
{
diff --git a/lib/raid6/sse1.c b/lib/raid6/sse1.c
index 692fa3a93bf0..794d5cfa0306 100644
--- a/lib/raid6/sse1.c
+++ b/lib/raid6/sse1.c
@@ -19,7 +19,7 @@
#ifdef CONFIG_X86_32
#include <linux/raid/pq.h>
-#include "x86.h"
+#include <asm/fpu/api.h>
/* Defined in raid6/mmx.c */
extern const struct raid6_mmx_constants {
diff --git a/lib/raid6/sse2.c b/lib/raid6/sse2.c
index 2930220249c9..f9edf8a8d1c4 100644
--- a/lib/raid6/sse2.c
+++ b/lib/raid6/sse2.c
@@ -13,7 +13,7 @@
*/
#include <linux/raid/pq.h>
-#include "x86.h"
+#include <asm/fpu/api.h>
static const struct raid6_sse_constants {
u64 x1d[2];
diff --git a/lib/raid6/vpermxor.uc b/lib/raid6/vpermxor.uc
index 1bfb127fbfe8..a8e76b1c956e 100644
--- a/lib/raid6/vpermxor.uc
+++ b/lib/raid6/vpermxor.uc
@@ -25,10 +25,8 @@
#include <altivec.h>
#include <asm/ppc-opcode.h>
-#ifdef __KERNEL__
#include <asm/cputable.h>
#include <asm/switch_to.h>
-#endif
typedef vector unsigned char unative_t;
#define NSIZE sizeof(unative_t)
@@ -85,13 +83,8 @@ int raid6_have_altivec_vpermxor(void);
int raid6_have_altivec_vpermxor(void)
{
/* Check if arch has both altivec and the vpermxor instructions */
-# ifdef __KERNEL__
return (cpu_has_feature(CPU_FTR_ALTIVEC_COMP) &&
cpu_has_feature(CPU_FTR_ARCH_207S));
-# else
- return 1;
-#endif
-
}
#endif
diff --git a/lib/raid6/x86.h b/lib/raid6/x86.h
deleted file mode 100644
index 9a6ff37115e7..000000000000
--- a/lib/raid6/x86.h
+++ /dev/null
@@ -1,75 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/* ----------------------------------------------------------------------- *
- *
- * Copyright 2002-2004 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
-/*
- * raid6/x86.h
- *
- * Definitions common to x86 and x86-64 RAID-6 code only
- */
-
-#ifndef LINUX_RAID_RAID6X86_H
-#define LINUX_RAID_RAID6X86_H
-
-#if (defined(__i386__) || defined(__x86_64__)) && !defined(__arch_um__)
-
-#ifdef __KERNEL__ /* Real code */
-
-#include <asm/fpu/api.h>
-
-#else /* Dummy code for user space testing */
-
-static inline void kernel_fpu_begin(void)
-{
-}
-
-static inline void kernel_fpu_end(void)
-{
-}
-
-#define __aligned(x) __attribute__((aligned(x)))
-
-#define X86_FEATURE_MMX (0*32+23) /* Multimedia Extensions */
-#define X86_FEATURE_FXSR (0*32+24) /* FXSAVE and FXRSTOR instructions
- * (fast save and restore) */
-#define X86_FEATURE_XMM (0*32+25) /* Streaming SIMD Extensions */
-#define X86_FEATURE_XMM2 (0*32+26) /* Streaming SIMD Extensions-2 */
-#define X86_FEATURE_XMM3 (4*32+ 0) /* "pni" SSE-3 */
-#define X86_FEATURE_SSSE3 (4*32+ 9) /* Supplemental SSE-3 */
-#define X86_FEATURE_AVX (4*32+28) /* Advanced Vector Extensions */
-#define X86_FEATURE_AVX2 (9*32+ 5) /* AVX2 instructions */
-#define X86_FEATURE_AVX512F (9*32+16) /* AVX-512 Foundation */
-#define X86_FEATURE_AVX512DQ (9*32+17) /* AVX-512 DQ (Double/Quad granular)
- * Instructions
- */
-#define X86_FEATURE_AVX512BW (9*32+30) /* AVX-512 BW (Byte/Word granular)
- * Instructions
- */
-#define X86_FEATURE_AVX512VL (9*32+31) /* AVX-512 VL (128/256 Vector Length)
- * Extensions
- */
-#define X86_FEATURE_MMXEXT (1*32+22) /* AMD MMX extensions */
-
-/* Should work well enough on modern CPUs for testing */
-static inline int boot_cpu_has(int flag)
-{
- u32 eax, ebx, ecx, edx;
-
- eax = (flag & 0x100) ? 7 :
- (flag & 0x20) ? 0x80000001 : 1;
- ecx = 0;
-
- asm volatile("cpuid"
- : "+a" (eax), "=b" (ebx), "=d" (edx), "+c" (ecx));
-
- return ((flag & 0x100 ? ebx :
- (flag & 0x80) ? ecx : edx) >> (flag & 31)) & 1;
-}
-
-#endif /* ndef __KERNEL__ */
-
-#endif
-#endif
--
2.53.0
^ permalink raw reply related
* cleanup the RAID6 P/Q library v2
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
Hi all,
this series cleans up the RAID6 P/Q library to match the recent updates
to the RAID 5 XOR library and other CRC/crypto libraries. This includes
providing properly documented external interfaces, hiding the internals,
using static_call instead of indirect calls and turning the user space
test suite into an in-kernel kunit test which is also extended to
improve coverage.
Note that this changes registration so that non-priority algorithms are
not registered, which greatly helps with the benchmark time at boot time.
I'd like to encourage all architecture maintainers to see if they can
further optimized this by registering as few as possible algorithms when
there is a clear benefit in optimized or more unrolled implementations.
This series sits on top of the "cleanup the RAID5 XOR library v3" series.
A git tree is also available here:
git://git.infradead.org/users/hch/misc.git lib-raid6
Gitweb:
https://git.infradead.org/?p=users/hch/misc.git;a=shortlog;h=refs/heads/lib-raid6
Changes since v1:
- fix arm64 objdir != srcdir builds
- call the kunit module raid6_kunit.ko from the beginning
- update MAINTAINERS
- don't require preemptible context and apply the same restrictions as
the merged version of the XOR API
- fix the arm64 default in Kconfig
- pick the last registered (and presumably most optimized) algorithm when
benchmarking is disabled
- port over the randomization fixes from the XOR series
- misc other kunit cleanups
- require at least 4 devices for RAID6 to skip broken special cases
Diffstat:
b/Documentation/crypto/async-tx-api.rst | 4
b/MAINTAINERS | 2
b/crypto/async_tx/async_pq.c | 9
b/crypto/async_tx/async_raid6_recov.c | 9
b/drivers/dma/bcm-sba-raid.c | 1
b/drivers/md/raid5.c | 4
b/fs/btrfs/raid56.c | 8
b/fs/btrfs/volumes.c | 2
b/include/linux/raid/pq.h | 233 +------------
b/include/linux/raid/pq_tables.h | 19 +
b/lib/Kconfig | 11
b/lib/Makefile | 1
b/lib/raid/Kconfig | 33 +
b/lib/raid/Makefile | 2
b/lib/raid/raid6/Makefile | 126 +++++++
b/lib/raid/raid6/algos.c | 383 ++++++++++++++++++++++
b/lib/raid/raid6/algos.h | 41 ++
b/lib/raid/raid6/arm/neon.c | 23 -
b/lib/raid/raid6/arm/neon.uc | 2
b/lib/raid/raid6/arm/pq_arch.h | 24 +
b/lib/raid/raid6/arm/recov_neon.c | 27 -
b/lib/raid/raid6/arm/recov_neon_inner.c | 2
b/lib/raid/raid6/arm64/pq_arch.h | 1
b/lib/raid/raid6/int.uc | 10
b/lib/raid/raid6/loongarch/loongarch_simd.c | 31 -
b/lib/raid/raid6/loongarch/pq_arch.h | 23 +
b/lib/raid/raid6/loongarch/recov_loongarch_simd.c | 39 --
b/lib/raid/raid6/mktables.c | 28 -
b/lib/raid/raid6/powerpc/altivec.uc | 32 -
b/lib/raid/raid6/powerpc/pq_arch.h | 31 +
b/lib/raid/raid6/powerpc/vpermxor.uc | 29 -
b/lib/raid/raid6/recov.c | 62 ---
b/lib/raid/raid6/riscv/pq_arch.h | 21 +
b/lib/raid/raid6/riscv/recov_rvv.c | 14
b/lib/raid/raid6/riscv/rvv.h | 26 -
b/lib/raid/raid6/s390/pq_arch.h | 15
b/lib/raid/raid6/s390/recov_s390xc.c | 14
b/lib/raid/raid6/s390/s390vx.uc | 15
b/lib/raid/raid6/tests/Makefile | 3
b/lib/raid/raid6/tests/raid6_kunit.c | 321 ++++++++++++++++++
b/lib/raid/raid6/x86/avx2.c | 47 --
b/lib/raid/raid6/x86/avx512.c | 57 +--
b/lib/raid/raid6/x86/mmx.c | 39 --
b/lib/raid/raid6/x86/pq_arch.h | 97 +++++
b/lib/raid/raid6/x86/recov_avx2.c | 22 -
b/lib/raid/raid6/x86/recov_avx512.c | 26 -
b/lib/raid/raid6/x86/recov_ssse3.c | 23 -
b/lib/raid/raid6/x86/sse1.c | 49 --
b/lib/raid/raid6/x86/sse2.c | 47 --
lib/raid6/Makefile | 83 ----
lib/raid6/algos.c | 291 ----------------
lib/raid6/loongarch.h | 38 --
lib/raid6/test/.gitignore | 3
lib/raid6/test/Makefile | 156 --------
lib/raid6/test/test.c | 152 --------
lib/raid6/x86.h | 75 ----
56 files changed, 1369 insertions(+), 1517 deletions(-)
^ permalink raw reply
* [PATCH 02/19] raid6: turn the userspace test harness into a kunit test
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Currently the raid6 code can be compiled as userspace code to run the
test suite. Convert that to be a kunit case with minimal changes to
avoid mutating global state so that we can drop this requirement.
Note that this is not a good kunit test case yet and will need a lot more
work, but that is deferred until the raid6 code is moved to it's new
place, which is easier if the userspace makefile doesn't need adjustments
for the new location first.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/raid/pq.h | 3 -
lib/Kconfig | 11 +++
lib/raid6/Makefile | 2 +-
lib/raid6/algos.c | 5 +-
lib/raid6/recov.c | 34 ---------
lib/raid6/test/Makefile | 155 +--------------------------------------
lib/raid6/test/test.c | 158 +++++++++++++++++++++-------------------
7 files changed, 101 insertions(+), 267 deletions(-)
diff --git a/include/linux/raid/pq.h b/include/linux/raid/pq.h
index 2467b3be15c9..08c5995ea980 100644
--- a/include/linux/raid/pq.h
+++ b/include/linux/raid/pq.h
@@ -144,7 +144,6 @@ extern const struct raid6_calls raid6_neonx8;
/* Algorithm list */
extern const struct raid6_calls * const raid6_algos[];
extern const struct raid6_recov_calls *const raid6_recov_algos[];
-int raid6_select_algo(void);
/* Return values from chk_syndrome */
#define RAID6_OK 0
@@ -165,8 +164,6 @@ extern void (*raid6_2data_recov)(int disks, size_t bytes, int faila, int failb,
void **ptrs);
extern void (*raid6_datap_recov)(int disks, size_t bytes, int faila,
void **ptrs);
-void raid6_dual_recov(int disks, size_t bytes, int faila, int failb,
- void **ptrs);
/* Some definitions to allow code to be compiled for testing in userspace */
#ifndef __KERNEL__
diff --git a/lib/Kconfig b/lib/Kconfig
index 00a9509636c1..bffe015a6c10 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -11,6 +11,17 @@ menu "Library routines"
config RAID6_PQ
tristate
+config RAID6_PQ_KUNIT_TEST
+ tristate "KUnit tests for RAID6 PQ functions" if !KUNIT_ALL_TESTS
+ depends on KUNIT
+ depends on RAID6_PQ
+ default KUNIT_ALL_TESTS
+ help
+ Unit tests for the RAID6 PQ library functions.
+
+ This is intended to help people writing architecture-specific
+ optimized versions. If unsure, say N.
+
config RAID6_PQ_BENCHMARK
bool "Automatically choose fastest RAID6 PQ functions"
depends on RAID6_PQ
diff --git a/lib/raid6/Makefile b/lib/raid6/Makefile
index 5be0a4e60ab1..6fd048c127b6 100644
--- a/lib/raid6/Makefile
+++ b/lib/raid6/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_RAID6_PQ) += raid6_pq.o
+obj-$(CONFIG_RAID6_PQ) += raid6_pq.o test/
raid6_pq-y += algos.o recov.o tables.o int1.o int2.o int4.o \
int8.o
diff --git a/lib/raid6/algos.c b/lib/raid6/algos.c
index 799e0e5eac26..5a9f4882e18d 100644
--- a/lib/raid6/algos.c
+++ b/lib/raid6/algos.c
@@ -19,6 +19,7 @@
#include <linux/module.h>
#include <linux/gfp.h>
#endif
+#include <kunit/visibility.h>
struct raid6_calls raid6_call;
EXPORT_SYMBOL_GPL(raid6_call);
@@ -86,6 +87,7 @@ const struct raid6_calls * const raid6_algos[] = {
&raid6_intx1,
NULL
};
+EXPORT_SYMBOL_IF_KUNIT(raid6_algos);
void (*raid6_2data_recov)(int, size_t, int, int, void **);
EXPORT_SYMBOL_GPL(raid6_2data_recov);
@@ -119,6 +121,7 @@ const struct raid6_recov_calls *const raid6_recov_algos[] = {
&raid6_recov_intx1,
NULL
};
+EXPORT_SYMBOL_IF_KUNIT(raid6_recov_algos);
#ifdef __KERNEL__
#define RAID6_TIME_JIFFIES_LG2 4
@@ -239,7 +242,7 @@ static inline const struct raid6_calls *raid6_choose_gen(
/* Try to pick the best algorithm */
/* This code uses the gfmul table as convenient data set to abuse */
-int __init raid6_select_algo(void)
+static int __init raid6_select_algo(void)
{
const int disks = RAID6_TEST_DISKS;
diff --git a/lib/raid6/recov.c b/lib/raid6/recov.c
index b5e47c008b41..8d113196632e 100644
--- a/lib/raid6/recov.c
+++ b/lib/raid6/recov.c
@@ -99,37 +99,3 @@ const struct raid6_recov_calls raid6_recov_intx1 = {
.name = "intx1",
.priority = 0,
};
-
-#ifndef __KERNEL__
-/* Testing only */
-
-/* Recover two failed blocks. */
-void raid6_dual_recov(int disks, size_t bytes, int faila, int failb, void **ptrs)
-{
- if ( faila > failb ) {
- int tmp = faila;
- faila = failb;
- failb = tmp;
- }
-
- if ( failb == disks-1 ) {
- if ( faila == disks-2 ) {
- /* P+Q failure. Just rebuild the syndrome. */
- raid6_call.gen_syndrome(disks, bytes, ptrs);
- } else {
- /* data+Q failure. Reconstruct data from P,
- then rebuild syndrome. */
- /* NOT IMPLEMENTED - equivalent to RAID-5 */
- }
- } else {
- if ( failb == disks-2 ) {
- /* data+P failure. */
- raid6_datap_recov(disks, bytes, faila, ptrs);
- } else {
- /* data+data failure. */
- raid6_2data_recov(disks, bytes, faila, failb, ptrs);
- }
- }
-}
-
-#endif
diff --git a/lib/raid6/test/Makefile b/lib/raid6/test/Makefile
index 09bbe2b14cce..520381ea71d7 100644
--- a/lib/raid6/test/Makefile
+++ b/lib/raid6/test/Makefile
@@ -1,156 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
-#
-# This is a simple Makefile to test some of the RAID-6 code
-# from userspace.
-#
-pound := \#
+obj-$(CONFIG_RAID6_PQ_KUNIT_TEST) += raid6_kunit.o
-# Adjust as desired
-CC = gcc
-OPTFLAGS = -O2
-CFLAGS = -I.. -I ../../../include -g $(OPTFLAGS)
-LD = ld
-AWK = awk -f
-AR = ar
-RANLIB = ranlib
-OBJS = int1.o int2.o int4.o int8.o int16.o int32.o recov.o algos.o tables.o
-
-ARCH := $(shell uname -m 2>/dev/null | sed -e /s/i.86/i386/)
-ifeq ($(ARCH),i386)
- CFLAGS += -DCONFIG_X86_32
- IS_X86 = yes
-endif
-ifeq ($(ARCH),x86_64)
- CFLAGS += -DCONFIG_X86_64
- IS_X86 = yes
-endif
-
-ifeq ($(ARCH),arm)
- CFLAGS += -I../../../arch/arm/include -mfpu=neon
- HAS_NEON = yes
-endif
-ifeq ($(ARCH),aarch64)
- CFLAGS += -I../../../arch/arm64/include
- HAS_NEON = yes
-endif
-
-ifeq ($(findstring riscv,$(ARCH)),riscv)
- CFLAGS += -I../../../arch/riscv/include -DCONFIG_RISCV=1
- HAS_RVV = yes
-endif
-
-ifeq ($(findstring ppc,$(ARCH)),ppc)
- CFLAGS += -I../../../arch/powerpc/include
- HAS_ALTIVEC := $(shell printf '$(pound)include <altivec.h>\nvector int a;\n' |\
- gcc -c -x c - >/dev/null && rm ./-.o && echo yes)
-endif
-
-ifeq ($(ARCH),loongarch64)
- CFLAGS += -I../../../arch/loongarch/include -DCONFIG_LOONGARCH=1
- CFLAGS += $(shell echo 'vld $$vr0, $$zero, 0' | \
- gcc -c -x assembler - >/dev/null 2>&1 && \
- rm ./-.o && echo -DCONFIG_CPU_HAS_LSX=1)
- CFLAGS += $(shell echo 'xvld $$xr0, $$zero, 0' | \
- gcc -c -x assembler - >/dev/null 2>&1 && \
- rm ./-.o && echo -DCONFIG_CPU_HAS_LASX=1)
-endif
-
-ifeq ($(IS_X86),yes)
- OBJS += mmx.o sse1.o sse2.o avx2.o recov_ssse3.o recov_avx2.o avx512.o recov_avx512.o
- CFLAGS += -DCONFIG_X86
-else ifeq ($(HAS_NEON),yes)
- OBJS += neon.o neon1.o neon2.o neon4.o neon8.o recov_neon.o recov_neon_inner.o
- CFLAGS += -DCONFIG_KERNEL_MODE_NEON=1
-else ifeq ($(HAS_ALTIVEC),yes)
- CFLAGS += -DCONFIG_ALTIVEC
- OBJS += altivec1.o altivec2.o altivec4.o altivec8.o \
- vpermxor1.o vpermxor2.o vpermxor4.o vpermxor8.o
-else ifeq ($(ARCH),loongarch64)
- OBJS += loongarch_simd.o recov_loongarch_simd.o
-else ifeq ($(HAS_RVV),yes)
- OBJS += rvv.o recov_rvv.o
- CFLAGS += -DCONFIG_RISCV_ISA_V=1
-endif
-
-.c.o:
- $(CC) $(CFLAGS) -c -o $@ $<
-
-%.c: ../%.c
- cp -f $< $@
-
-%.uc: ../%.uc
- cp -f $< $@
-
-all: raid6.a raid6test
-
-raid6.a: $(OBJS)
- rm -f $@
- $(AR) cq $@ $^
- $(RANLIB) $@
-
-raid6test: test.c raid6.a
- $(CC) $(CFLAGS) -o raid6test $^
-
-neon1.c: neon.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=1 < neon.uc > $@
-
-neon2.c: neon.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=2 < neon.uc > $@
-
-neon4.c: neon.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=4 < neon.uc > $@
-
-neon8.c: neon.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=8 < neon.uc > $@
-
-altivec1.c: altivec.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=1 < altivec.uc > $@
-
-altivec2.c: altivec.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=2 < altivec.uc > $@
-
-altivec4.c: altivec.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=4 < altivec.uc > $@
-
-altivec8.c: altivec.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=8 < altivec.uc > $@
-
-vpermxor1.c: vpermxor.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=1 < vpermxor.uc > $@
-
-vpermxor2.c: vpermxor.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=2 < vpermxor.uc > $@
-
-vpermxor4.c: vpermxor.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=4 < vpermxor.uc > $@
-
-vpermxor8.c: vpermxor.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=8 < vpermxor.uc > $@
-
-int1.c: int.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=1 < int.uc > $@
-
-int2.c: int.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=2 < int.uc > $@
-
-int4.c: int.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=4 < int.uc > $@
-
-int8.c: int.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=8 < int.uc > $@
-
-int16.c: int.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=16 < int.uc > $@
-
-int32.c: int.uc ../unroll.awk
- $(AWK) ../unroll.awk -vN=32 < int.uc > $@
-
-tables.c: mktables
- ./mktables > tables.c
-
-clean:
- rm -f *.o *.a mktables mktables.c *.uc int*.c altivec*.c vpermxor*.c neon*.c tables.c raid6test
-
-spotless: clean
- rm -f *~
+raid6_kunit-y += test.o
diff --git a/lib/raid6/test/test.c b/lib/raid6/test/test.c
index 841a55242aba..ab4fda17395b 100644
--- a/lib/raid6/test/test.c
+++ b/lib/raid6/test/test.c
@@ -1,43 +1,37 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002-2007 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6test.c
+ * Copyright 2002-2007 H. Peter Anvin - All Rights Reserved
*
- * Test RAID-6 recovery with various algorithms
+ * Test RAID-6 recovery algorithms.
*/
-#include <stdlib.h>
-#include <stdio.h>
-#include <string.h>
+#include <kunit/test.h>
+#include <linux/prandom.h>
#include <linux/raid/pq.h>
-#define NDISKS 16 /* Including P and Q */
+MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING");
+
+#define RAID6_KUNIT_SEED 42
-const char raid6_empty_zero_page[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
+#define NDISKS 16 /* Including P and Q */
-char *dataptrs[NDISKS];
-char data[NDISKS][PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
-char recovi[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
-char recovj[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
+static struct rnd_state rng;
+static void *dataptrs[NDISKS];
+static char data[NDISKS][PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
+static char recovi[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
+static char recovj[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
static void makedata(int start, int stop)
{
- int i, j;
+ int i;
for (i = start; i <= stop; i++) {
- for (j = 0; j < PAGE_SIZE; j++)
- data[i][j] = rand();
-
+ prandom_bytes_state(&rng, data[i], PAGE_SIZE);
dataptrs[i] = data[i];
}
}
-static char disk_type(int d)
+static char member_type(int d)
{
switch (d) {
case NDISKS-2:
@@ -49,104 +43,118 @@ static char disk_type(int d)
}
}
-static int test_disks(int i, int j)
+static void test_disks(struct kunit *test, const struct raid6_calls *calls,
+ const struct raid6_recov_calls *ra, int faila, int failb)
{
- int erra, errb;
-
memset(recovi, 0xf0, PAGE_SIZE);
memset(recovj, 0xba, PAGE_SIZE);
- dataptrs[i] = recovi;
- dataptrs[j] = recovj;
-
- raid6_dual_recov(NDISKS, PAGE_SIZE, i, j, (void **)&dataptrs);
-
- erra = memcmp(data[i], recovi, PAGE_SIZE);
- errb = memcmp(data[j], recovj, PAGE_SIZE);
-
- if (i < NDISKS-2 && j == NDISKS-1) {
- /* We don't implement the DQ failure scenario, since it's
- equivalent to a RAID-5 failure (XOR, then recompute Q) */
- erra = errb = 0;
+ dataptrs[faila] = recovi;
+ dataptrs[failb] = recovj;
+
+ if (failb == NDISKS - 1) {
+ /*
+ * We don't implement the data+Q failure scenario, since it
+ * is equivalent to a RAID-5 failure (XOR, then recompute Q).
+ */
+ if (faila != NDISKS - 2)
+ goto skip;
+
+ /* P+Q failure. Just rebuild the syndrome. */
+ calls->gen_syndrome(NDISKS, PAGE_SIZE, dataptrs);
+ } else if (failb == NDISKS - 2) {
+ /* data+P failure. */
+ ra->datap(NDISKS, PAGE_SIZE, faila, dataptrs);
} else {
- printf("algo=%-8s faila=%3d(%c) failb=%3d(%c) %s\n",
- raid6_call.name,
- i, disk_type(i),
- j, disk_type(j),
- (!erra && !errb) ? "OK" :
- !erra ? "ERRB" :
- !errb ? "ERRA" : "ERRAB");
+ /* data+data failure. */
+ ra->data2(NDISKS, PAGE_SIZE, faila, failb, dataptrs);
}
- dataptrs[i] = data[i];
- dataptrs[j] = data[j];
-
- return erra || errb;
+ KUNIT_EXPECT_MEMEQ_MSG(test, data[faila], recovi, PAGE_SIZE,
+ "algo=%-8s/%-8s faila miscompared: %3d[%c] (failb=%3d[%c])\n",
+ calls->name, ra->name,
+ faila, member_type(faila),
+ failb, member_type(failb));
+ KUNIT_EXPECT_MEMEQ_MSG(test, data[failb], recovj, PAGE_SIZE,
+ "algo=%-8s/%-8s failb miscompared: %3d[%c] (faila=%3d[%c])\n",
+ calls->name, ra->name,
+ failb, member_type(failb),
+ faila, member_type(faila));
+
+skip:
+ dataptrs[faila] = data[faila];
+ dataptrs[failb] = data[failb];
}
-int main(int argc, char *argv[])
+static void raid6_test(struct kunit *test)
{
const struct raid6_calls *const *algo;
const struct raid6_recov_calls *const *ra;
int i, j, p1, p2;
- int err = 0;
-
- makedata(0, NDISKS-1);
for (ra = raid6_recov_algos; *ra; ra++) {
if ((*ra)->valid && !(*ra)->valid())
continue;
- raid6_2data_recov = (*ra)->data2;
- raid6_datap_recov = (*ra)->datap;
-
- printf("using recovery %s\n", (*ra)->name);
-
for (algo = raid6_algos; *algo; algo++) {
- if ((*algo)->valid && !(*algo)->valid())
- continue;
+ const struct raid6_calls *calls = *algo;
- raid6_call = **algo;
+ if (calls->valid && !calls->valid())
+ continue;
/* Nuke syndromes */
- memset(data[NDISKS-2], 0xee, 2*PAGE_SIZE);
+ memset(data[NDISKS - 2], 0xee, PAGE_SIZE);
+ memset(data[NDISKS - 1], 0xee, PAGE_SIZE);
/* Generate assumed good syndrome */
- raid6_call.gen_syndrome(NDISKS, PAGE_SIZE,
+ calls->gen_syndrome(NDISKS, PAGE_SIZE,
(void **)&dataptrs);
for (i = 0; i < NDISKS-1; i++)
for (j = i+1; j < NDISKS; j++)
- err += test_disks(i, j);
+ test_disks(test, calls, *ra, i, j);
- if (!raid6_call.xor_syndrome)
+ if (!calls->xor_syndrome)
continue;
for (p1 = 0; p1 < NDISKS-2; p1++)
for (p2 = p1; p2 < NDISKS-2; p2++) {
/* Simulate rmw run */
- raid6_call.xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
+ calls->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
(void **)&dataptrs);
makedata(p1, p2);
- raid6_call.xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
+ calls->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
(void **)&dataptrs);
for (i = 0; i < NDISKS-1; i++)
for (j = i+1; j < NDISKS; j++)
- err += test_disks(i, j);
+ test_disks(test, calls,
+ *ra, i, j);
}
}
- printf("\n");
}
+}
- printf("\n");
- /* Pick the best algorithm test */
- raid6_select_algo();
-
- if (err)
- printf("\n*** ERRORS FOUND ***\n");
+static struct kunit_case raid6_test_cases[] = {
+ KUNIT_CASE(raid6_test),
+ {},
+};
- return err;
+static int raid6_suite_init(struct kunit_suite *suite)
+{
+ prandom_seed_state(&rng, RAID6_KUNIT_SEED);
+ makedata(0, NDISKS - 1);
+ return 0;
}
+
+static struct kunit_suite raid6_test_suite = {
+ .name = "raid6",
+ .test_cases = raid6_test_cases,
+ .suite_init = raid6_suite_init,
+};
+kunit_test_suite(raid6_test_suite);
+
+MODULE_DESCRIPTION("Unit test for the XOR library functions");
+MODULE_LICENSE("GPL");
--
2.53.0
^ permalink raw reply related
* [PATCH 04/19] raid6: move to lib/raid/
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Move the raid6 code to live in lib/raid/ with the XOR code, and change
the internal organization so that each architecture has a subdirectory
similar to the CRC, crypto and XOR libraries, and fix up the Makefile to
only build files actually needed.
Also move the kunit test case from the history test/ subdirectory to
tests/ and use the normal naming scheme for it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
MAINTAINERS | 2 +-
lib/Kconfig | 22 ----
lib/Makefile | 1 -
lib/raid/Kconfig | 22 ++++
lib/raid/Makefile | 2 +-
lib/{ => raid}/raid6/.gitignore | 0
lib/raid/raid6/Makefile | 120 ++++++++++++++++++
lib/{ => raid}/raid6/algos.c | 0
lib/{raid6 => raid/raid6/arm}/neon.c | 0
lib/{raid6 => raid/raid6/arm}/neon.h | 0
lib/{raid6 => raid/raid6/arm}/neon.uc | 2 +-
lib/{raid6 => raid/raid6/arm}/recov_neon.c | 2 +-
.../raid6/arm}/recov_neon_inner.c | 2 +-
lib/{ => raid}/raid6/int.uc | 0
.../raid6/loongarch}/loongarch_simd.c | 0
.../raid6/loongarch}/recov_loongarch_simd.c | 0
lib/{ => raid}/raid6/mktables.c | 0
lib/{raid6 => raid/raid6/powerpc}/altivec.uc | 4 -
lib/{raid6 => raid/raid6/powerpc}/vpermxor.uc | 3 -
lib/{ => raid}/raid6/recov.c | 0
lib/{raid6 => raid/raid6/riscv}/recov_rvv.c | 0
lib/{raid6 => raid/raid6/riscv}/rvv.c | 0
lib/{raid6 => raid/raid6/riscv}/rvv.h | 0
lib/{raid6 => raid/raid6/s390}/recov_s390xc.c | 0
lib/{raid6 => raid/raid6/s390}/s390vx.uc | 0
lib/{raid6/test => raid/raid6/tests}/Makefile | 2 -
.../test.c => raid/raid6/tests/raid6_kunit.c} | 0
lib/{ => raid}/raid6/unroll.awk | 0
lib/{raid6 => raid/raid6/x86}/avx2.c | 0
lib/{raid6 => raid/raid6/x86}/avx512.c | 0
lib/{raid6 => raid/raid6/x86}/mmx.c | 4 -
lib/{raid6 => raid/raid6/x86}/recov_avx2.c | 0
lib/{raid6 => raid/raid6/x86}/recov_avx512.c | 0
lib/{raid6 => raid/raid6/x86}/recov_ssse3.c | 0
lib/{raid6 => raid/raid6/x86}/sse1.c | 4 -
lib/{raid6 => raid/raid6/x86}/sse2.c | 0
lib/raid6/Makefile | 83 ------------
lib/raid6/test/.gitignore | 3 -
38 files changed, 147 insertions(+), 131 deletions(-)
rename lib/{ => raid}/raid6/.gitignore (100%)
create mode 100644 lib/raid/raid6/Makefile
rename lib/{ => raid}/raid6/algos.c (100%)
rename lib/{raid6 => raid/raid6/arm}/neon.c (100%)
rename lib/{raid6 => raid/raid6/arm}/neon.h (100%)
rename lib/{raid6 => raid/raid6/arm}/neon.uc (99%)
rename lib/{raid6 => raid/raid6/arm}/recov_neon.c (99%)
rename lib/{raid6 => raid/raid6/arm}/recov_neon_inner.c (99%)
rename lib/{ => raid}/raid6/int.uc (100%)
rename lib/{raid6 => raid/raid6/loongarch}/loongarch_simd.c (100%)
rename lib/{raid6 => raid/raid6/loongarch}/recov_loongarch_simd.c (100%)
rename lib/{ => raid}/raid6/mktables.c (100%)
rename lib/{raid6 => raid/raid6/powerpc}/altivec.uc (98%)
rename lib/{raid6 => raid/raid6/powerpc}/vpermxor.uc (98%)
rename lib/{ => raid}/raid6/recov.c (100%)
rename lib/{raid6 => raid/raid6/riscv}/recov_rvv.c (100%)
rename lib/{raid6 => raid/raid6/riscv}/rvv.c (100%)
rename lib/{raid6 => raid/raid6/riscv}/rvv.h (100%)
rename lib/{raid6 => raid/raid6/s390}/recov_s390xc.c (100%)
rename lib/{raid6 => raid/raid6/s390}/s390vx.uc (100%)
rename lib/{raid6/test => raid/raid6/tests}/Makefile (77%)
rename lib/{raid6/test/test.c => raid/raid6/tests/raid6_kunit.c} (100%)
rename lib/{ => raid}/raid6/unroll.awk (100%)
rename lib/{raid6 => raid/raid6/x86}/avx2.c (100%)
rename lib/{raid6 => raid/raid6/x86}/avx512.c (100%)
rename lib/{raid6 => raid/raid6/x86}/mmx.c (99%)
rename lib/{raid6 => raid/raid6/x86}/recov_avx2.c (100%)
rename lib/{raid6 => raid/raid6/x86}/recov_avx512.c (100%)
rename lib/{raid6 => raid/raid6/x86}/recov_ssse3.c (100%)
rename lib/{raid6 => raid/raid6/x86}/sse1.c (99%)
rename lib/{raid6 => raid/raid6/x86}/sse2.c (100%)
delete mode 100644 lib/raid6/Makefile
delete mode 100644 lib/raid6/test/.gitignore
diff --git a/MAINTAINERS b/MAINTAINERS
index b2040011a386..00b0c448e462 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -24814,7 +24814,7 @@ F: drivers/md/md*
F: drivers/md/raid*
F: include/linux/raid/
F: include/uapi/linux/raid/
-F: lib/raid6/
+F: lib/raid/raid6/
SOLIDRUN CLEARFOG SUPPORT
M: Russell King <linux@armlinux.org.uk>
diff --git a/lib/Kconfig b/lib/Kconfig
index bffe015a6c10..b87f954a14bc 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -8,28 +8,6 @@ config BINARY_PRINTF
menu "Library routines"
-config RAID6_PQ
- tristate
-
-config RAID6_PQ_KUNIT_TEST
- tristate "KUnit tests for RAID6 PQ functions" if !KUNIT_ALL_TESTS
- depends on KUNIT
- depends on RAID6_PQ
- default KUNIT_ALL_TESTS
- help
- Unit tests for the RAID6 PQ library functions.
-
- This is intended to help people writing architecture-specific
- optimized versions. If unsure, say N.
-
-config RAID6_PQ_BENCHMARK
- bool "Automatically choose fastest RAID6 PQ functions"
- depends on RAID6_PQ
- default y
- help
- Benchmark all available RAID6 PQ functions on init and choose the
- fastest one.
-
config LINEAR_RANGES
tristate
diff --git a/lib/Makefile b/lib/Makefile
index f33a24bf1c19..6e72d2c1cce7 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -167,7 +167,6 @@ obj-$(CONFIG_LZ4_DECOMPRESS) += lz4/
obj-$(CONFIG_ZSTD_COMPRESS) += zstd/
obj-$(CONFIG_ZSTD_DECOMPRESS) += zstd/
obj-$(CONFIG_XZ_DEC) += xz/
-obj-$(CONFIG_RAID6_PQ) += raid6/
lib-$(CONFIG_DECOMPRESS_GZIP) += decompress_inflate.o
lib-$(CONFIG_DECOMPRESS_BZIP2) += decompress_bunzip2.o
diff --git a/lib/raid/Kconfig b/lib/raid/Kconfig
index 5ab2b0a7be4c..e39f6d667792 100644
--- a/lib/raid/Kconfig
+++ b/lib/raid/Kconfig
@@ -28,3 +28,25 @@ config XOR_KUNIT_TEST
This is intended to help people writing architecture-specific
optimized versions. If unsure, say N.
+
+config RAID6_PQ
+ tristate
+
+config RAID6_PQ_KUNIT_TEST
+ tristate "KUnit tests for RAID6 PQ functions" if !KUNIT_ALL_TESTS
+ depends on KUNIT
+ depends on RAID6_PQ
+ default KUNIT_ALL_TESTS
+ help
+ Unit tests for the RAID6 PQ library functions.
+
+ This is intended to help people writing architecture-specific
+ optimized versions. If unsure, say N.
+
+config RAID6_PQ_BENCHMARK
+ bool "Automatically choose fastest RAID6 PQ functions"
+ depends on RAID6_PQ
+ default y
+ help
+ Benchmark all available RAID6 PQ functions on init and choose the
+ fastest one.
diff --git a/lib/raid/Makefile b/lib/raid/Makefile
index 3540fe846dc4..6fc5eeb53df0 100644
--- a/lib/raid/Makefile
+++ b/lib/raid/Makefile
@@ -1,3 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
-obj-y += xor/
+obj-y += xor/ raid6/
diff --git a/lib/raid6/.gitignore b/lib/raid/raid6/.gitignore
similarity index 100%
rename from lib/raid6/.gitignore
rename to lib/raid/raid6/.gitignore
diff --git a/lib/raid/raid6/Makefile b/lib/raid/raid6/Makefile
new file mode 100644
index 000000000000..886a1771e78d
--- /dev/null
+++ b/lib/raid/raid6/Makefile
@@ -0,0 +1,120 @@
+# SPDX-License-Identifier: GPL-2.0
+
+hostprogs += mktables
+
+obj-$(CONFIG_RAID6_PQ) += raid6_pq.o tests/
+
+raid6_pq-y += algos.o tables.o
+
+# generic integer generation and recovery implementation
+raid6_pq-y += int1.o int2.o int4.o int8.o
+raid6_pq-y += recov.o
+
+# architecture-specific generation and recovery implementations:
+raid6_pq-$(CONFIG_KERNEL_MODE_NEON) += arm/neon.o \
+ arm/neon1.o \
+ arm/neon2.o \
+ arm/neon4.o \
+ arm/neon8.o \
+ arm/recov_neon.o \
+ arm/recov_neon_inner.o
+raid6_pq-$(CONFIG_LOONGARCH) += loongarch/loongarch_simd.o \
+ loongarch/recov_loongarch_simd.o
+raid6_pq-$(CONFIG_ALTIVEC) += powerpc/altivec1.o \
+ powerpc/altivec2.o \
+ powerpc/altivec4.o \
+ powerpc/altivec8.o \
+ powerpc/vpermxor1.o \
+ powerpc/vpermxor2.o \
+ powerpc/vpermxor4.o \
+ powerpc/vpermxor8.o
+raid6_pq-$(CONFIG_RISCV_ISA_V) += riscv/rvv.o \
+ riscv/recov_rvv.o
+raid6_pq-$(CONFIG_S390) += s390/s390vx8.o \
+ s390/recov_s390xc.o
+ifeq ($(CONFIG_X86),y)
+raid6_pq-$(CONFIG_X86_32) += x86/mmx.o \
+ x86/sse1.o
+endif
+raid6_pq-$(CONFIG_X86) += x86/sse2.o \
+ x86/avx2.o \
+ x86/avx512.o \
+ x86/recov_ssse3.o \
+ x86/recov_avx2.o \
+ x86/recov_avx512.o
+
+CFLAGS_arm/neon1.o += $(CC_FLAGS_FPU)
+CFLAGS_arm/neon2.o += $(CC_FLAGS_FPU)
+CFLAGS_arm/neon4.o += $(CC_FLAGS_FPU)
+CFLAGS_arm/neon8.o += $(CC_FLAGS_FPU)
+CFLAGS_arm/recov_neon_inner.o += $(CC_FLAGS_FPU)
+CFLAGS_REMOVE_arm/neon1.o += $(CC_FLAGS_NO_FPU)
+CFLAGS_REMOVE_arm/neon2.o += $(CC_FLAGS_NO_FPU)
+CFLAGS_REMOVE_arm/neon4.o += $(CC_FLAGS_NO_FPU)
+CFLAGS_REMOVE_arm/neon8.o += $(CC_FLAGS_NO_FPU)
+CFLAGS_REMOVE_arm/recov_neon_inner.o += $(CC_FLAGS_NO_FPU)
+
+ifeq ($(CONFIG_ALTIVEC),y)
+altivec_flags := -maltivec $(call cc-option,-mabi=altivec)
+# Enable <altivec.h>
+altivec_flags += -isystem $(shell $(CC) -print-file-name=include)
+
+CFLAGS_powerpc/altivec1.o += $(altivec_flags)
+CFLAGS_powerpc/altivec2.o += $(altivec_flags)
+CFLAGS_powerpc/altivec4.o += $(altivec_flags)
+CFLAGS_powerpc/altivec8.o += $(altivec_flags)
+CFLAGS_powerpc/vpermxor1.o += $(altivec_flags)
+CFLAGS_powerpc/vpermxor2.o += $(altivec_flags)
+CFLAGS_powerpc/vpermxor4.o += $(altivec_flags)
+CFLAGS_powerpc/vpermxor8.o += $(altivec_flags)
+
+ifdef CONFIG_CC_IS_CLANG
+# clang ppc port does not yet support -maltivec when -msoft-float is
+# enabled. A future release of clang will resolve this
+# https://llvm.org/pr31177
+CFLAGS_REMOVE_powerpc/altivec1.o += -msoft-float
+CFLAGS_REMOVE_powerpc/altivec2.o += -msoft-float
+CFLAGS_REMOVE_powerpc/altivec4.o += -msoft-float
+CFLAGS_REMOVE_powerpc/altivec8.o += -msoft-float
+CFLAGS_REMOVE_powerpc/vpermxor1.o += -msoft-float
+CFLAGS_REMOVE_powerpc/vpermxor2.o += -msoft-float
+CFLAGS_REMOVE_powerpc/vpermxor4.o += -msoft-float
+CFLAGS_REMOVE_powerpc/vpermxor8.o += -msoft-float
+endif # CONFIG_CC_IS_CLANG
+endif # CONFIG_ALTIVEC
+
+quiet_cmd_mktable = TABLE $@
+ cmd_mktable = $(obj)/mktables > $@
+
+targets += tables.c
+$(obj)/tables.c: $(obj)/mktables FORCE
+ $(call if_changed,mktable)
+
+quiet_cmd_unroll = UNROLL $@
+ cmd_unroll = $(AWK) -v N=$* -f $(src)/unroll.awk < $< > $@
+
+targets += int1.c int2.c int4.c int8.c
+$(obj)/int%.c: $(src)/int.uc $(src)/unroll.awk FORCE
+ $(call if_changed,unroll)
+
+targets += arm/neon1.c arm/neon2.c arm/neon4.c arm/neon8.c
+$(obj)/arm/neon%.c: $(src)/arm/neon.uc $(src)/unroll.awk FORCE
+ $(call if_changed,unroll)
+
+targets += powerpc/altivec1.c \
+ powerpc/altivec2.c \
+ powerpc/altivec4.c \
+ powerpc/altivec8.c
+$(obj)/powerpc/altivec%.c: $(src)/powerpc/altivec.uc $(src)/unroll.awk FORCE
+ $(call if_changed,unroll)
+
+targets += powerpc/vpermxor1.c \
+ powerpc/vpermxor2.c \
+ powerpc/vpermxor4.c \
+ powerpc/vpermxor8.c
+$(obj)/powerpc/vpermxor%.c: $(src)/powerpc/vpermxor.uc $(src)/unroll.awk FORCE
+ $(call if_changed,unroll)
+
+targets += s390/s390vx8.c
+$(obj)/s390/s390vx%.c: $(src)/s390/s390vx.uc $(src)/unroll.awk FORCE
+ $(call if_changed,unroll)
diff --git a/lib/raid6/algos.c b/lib/raid/raid6/algos.c
similarity index 100%
rename from lib/raid6/algos.c
rename to lib/raid/raid6/algos.c
diff --git a/lib/raid6/neon.c b/lib/raid/raid6/arm/neon.c
similarity index 100%
rename from lib/raid6/neon.c
rename to lib/raid/raid6/arm/neon.c
diff --git a/lib/raid6/neon.h b/lib/raid/raid6/arm/neon.h
similarity index 100%
rename from lib/raid6/neon.h
rename to lib/raid/raid6/arm/neon.h
diff --git a/lib/raid6/neon.uc b/lib/raid/raid6/arm/neon.uc
similarity index 99%
rename from lib/raid6/neon.uc
rename to lib/raid/raid6/arm/neon.uc
index 355270af0cd6..14a9fc2c60fa 100644
--- a/lib/raid6/neon.uc
+++ b/lib/raid/raid6/arm/neon.uc
@@ -25,7 +25,7 @@
*/
#include <arm_neon.h>
-#include "neon.h"
+#include "arm/neon.h"
typedef uint8x16_t unative_t;
diff --git a/lib/raid6/recov_neon.c b/lib/raid/raid6/arm/recov_neon.c
similarity index 99%
rename from lib/raid6/recov_neon.c
rename to lib/raid/raid6/arm/recov_neon.c
index 13d5df718c15..5a48fcc762e8 100644
--- a/lib/raid6/recov_neon.c
+++ b/lib/raid/raid6/arm/recov_neon.c
@@ -6,7 +6,7 @@
#include <linux/raid/pq.h>
#include <asm/simd.h>
-#include "neon.h"
+#include "arm/neon.h"
static int raid6_has_neon(void)
{
diff --git a/lib/raid6/recov_neon_inner.c b/lib/raid/raid6/arm/recov_neon_inner.c
similarity index 99%
rename from lib/raid6/recov_neon_inner.c
rename to lib/raid/raid6/arm/recov_neon_inner.c
index f9e7e8f5a151..53c355efa7ff 100644
--- a/lib/raid6/recov_neon_inner.c
+++ b/lib/raid/raid6/arm/recov_neon_inner.c
@@ -5,7 +5,7 @@
*/
#include <arm_neon.h>
-#include "neon.h"
+#include "arm/neon.h"
#ifdef CONFIG_ARM
/*
diff --git a/lib/raid6/int.uc b/lib/raid/raid6/int.uc
similarity index 100%
rename from lib/raid6/int.uc
rename to lib/raid/raid6/int.uc
diff --git a/lib/raid6/loongarch_simd.c b/lib/raid/raid6/loongarch/loongarch_simd.c
similarity index 100%
rename from lib/raid6/loongarch_simd.c
rename to lib/raid/raid6/loongarch/loongarch_simd.c
diff --git a/lib/raid6/recov_loongarch_simd.c b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
similarity index 100%
rename from lib/raid6/recov_loongarch_simd.c
rename to lib/raid/raid6/loongarch/recov_loongarch_simd.c
diff --git a/lib/raid6/mktables.c b/lib/raid/raid6/mktables.c
similarity index 100%
rename from lib/raid6/mktables.c
rename to lib/raid/raid6/mktables.c
diff --git a/lib/raid6/altivec.uc b/lib/raid/raid6/powerpc/altivec.uc
similarity index 98%
rename from lib/raid6/altivec.uc
rename to lib/raid/raid6/powerpc/altivec.uc
index 2c59963e58f9..130d3d3dd42c 100644
--- a/lib/raid6/altivec.uc
+++ b/lib/raid/raid6/powerpc/altivec.uc
@@ -24,8 +24,6 @@
#include <linux/raid/pq.h>
-#ifdef CONFIG_ALTIVEC
-
#include <altivec.h>
#include <asm/cputable.h>
#include <asm/switch_to.h>
@@ -122,5 +120,3 @@ const struct raid6_calls raid6_altivec$# = {
"altivecx$#",
0
};
-
-#endif /* CONFIG_ALTIVEC */
diff --git a/lib/raid6/vpermxor.uc b/lib/raid/raid6/powerpc/vpermxor.uc
similarity index 98%
rename from lib/raid6/vpermxor.uc
rename to lib/raid/raid6/powerpc/vpermxor.uc
index a8e76b1c956e..595f20aaf4cf 100644
--- a/lib/raid6/vpermxor.uc
+++ b/lib/raid/raid6/powerpc/vpermxor.uc
@@ -21,8 +21,6 @@
*/
#include <linux/raid/pq.h>
-#ifdef CONFIG_ALTIVEC
-
#include <altivec.h>
#include <asm/ppc-opcode.h>
#include <asm/cputable.h>
@@ -95,4 +93,3 @@ const struct raid6_calls raid6_vpermxor$# = {
"vpermxor$#",
0
};
-#endif
diff --git a/lib/raid6/recov.c b/lib/raid/raid6/recov.c
similarity index 100%
rename from lib/raid6/recov.c
rename to lib/raid/raid6/recov.c
diff --git a/lib/raid6/recov_rvv.c b/lib/raid/raid6/riscv/recov_rvv.c
similarity index 100%
rename from lib/raid6/recov_rvv.c
rename to lib/raid/raid6/riscv/recov_rvv.c
diff --git a/lib/raid6/rvv.c b/lib/raid/raid6/riscv/rvv.c
similarity index 100%
rename from lib/raid6/rvv.c
rename to lib/raid/raid6/riscv/rvv.c
diff --git a/lib/raid6/rvv.h b/lib/raid/raid6/riscv/rvv.h
similarity index 100%
rename from lib/raid6/rvv.h
rename to lib/raid/raid6/riscv/rvv.h
diff --git a/lib/raid6/recov_s390xc.c b/lib/raid/raid6/s390/recov_s390xc.c
similarity index 100%
rename from lib/raid6/recov_s390xc.c
rename to lib/raid/raid6/s390/recov_s390xc.c
diff --git a/lib/raid6/s390vx.uc b/lib/raid/raid6/s390/s390vx.uc
similarity index 100%
rename from lib/raid6/s390vx.uc
rename to lib/raid/raid6/s390/s390vx.uc
diff --git a/lib/raid6/test/Makefile b/lib/raid/raid6/tests/Makefile
similarity index 77%
rename from lib/raid6/test/Makefile
rename to lib/raid/raid6/tests/Makefile
index 520381ea71d7..87a001b22847 100644
--- a/lib/raid6/test/Makefile
+++ b/lib/raid/raid6/tests/Makefile
@@ -1,5 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_RAID6_PQ_KUNIT_TEST) += raid6_kunit.o
-
-raid6_kunit-y += test.o
diff --git a/lib/raid6/test/test.c b/lib/raid/raid6/tests/raid6_kunit.c
similarity index 100%
rename from lib/raid6/test/test.c
rename to lib/raid/raid6/tests/raid6_kunit.c
diff --git a/lib/raid6/unroll.awk b/lib/raid/raid6/unroll.awk
similarity index 100%
rename from lib/raid6/unroll.awk
rename to lib/raid/raid6/unroll.awk
diff --git a/lib/raid6/avx2.c b/lib/raid/raid6/x86/avx2.c
similarity index 100%
rename from lib/raid6/avx2.c
rename to lib/raid/raid6/x86/avx2.c
diff --git a/lib/raid6/avx512.c b/lib/raid/raid6/x86/avx512.c
similarity index 100%
rename from lib/raid6/avx512.c
rename to lib/raid/raid6/x86/avx512.c
diff --git a/lib/raid6/mmx.c b/lib/raid/raid6/x86/mmx.c
similarity index 99%
rename from lib/raid6/mmx.c
rename to lib/raid/raid6/x86/mmx.c
index e411f0cfbd95..7e9810669347 100644
--- a/lib/raid6/mmx.c
+++ b/lib/raid/raid6/x86/mmx.c
@@ -11,8 +11,6 @@
* MMX implementation of RAID-6 syndrome functions
*/
-#ifdef CONFIG_X86_32
-
#include <linux/raid/pq.h>
#include <asm/fpu/api.h>
@@ -135,5 +133,3 @@ const struct raid6_calls raid6_mmxx2 = {
"mmxx2",
0
};
-
-#endif
diff --git a/lib/raid6/recov_avx2.c b/lib/raid/raid6/x86/recov_avx2.c
similarity index 100%
rename from lib/raid6/recov_avx2.c
rename to lib/raid/raid6/x86/recov_avx2.c
diff --git a/lib/raid6/recov_avx512.c b/lib/raid/raid6/x86/recov_avx512.c
similarity index 100%
rename from lib/raid6/recov_avx512.c
rename to lib/raid/raid6/x86/recov_avx512.c
diff --git a/lib/raid6/recov_ssse3.c b/lib/raid/raid6/x86/recov_ssse3.c
similarity index 100%
rename from lib/raid6/recov_ssse3.c
rename to lib/raid/raid6/x86/recov_ssse3.c
diff --git a/lib/raid6/sse1.c b/lib/raid/raid6/x86/sse1.c
similarity index 99%
rename from lib/raid6/sse1.c
rename to lib/raid/raid6/x86/sse1.c
index 794d5cfa0306..deecdd72ceec 100644
--- a/lib/raid6/sse1.c
+++ b/lib/raid/raid6/x86/sse1.c
@@ -16,8 +16,6 @@
* worthwhile as a separate implementation.
*/
-#ifdef CONFIG_X86_32
-
#include <linux/raid/pq.h>
#include <asm/fpu/api.h>
@@ -155,5 +153,3 @@ const struct raid6_calls raid6_sse1x2 = {
"sse1x2",
1 /* Has cache hints */
};
-
-#endif
diff --git a/lib/raid6/sse2.c b/lib/raid/raid6/x86/sse2.c
similarity index 100%
rename from lib/raid6/sse2.c
rename to lib/raid/raid6/x86/sse2.c
diff --git a/lib/raid6/Makefile b/lib/raid6/Makefile
deleted file mode 100644
index 6fd048c127b6..000000000000
--- a/lib/raid6/Makefile
+++ /dev/null
@@ -1,83 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_RAID6_PQ) += raid6_pq.o test/
-
-raid6_pq-y += algos.o recov.o tables.o int1.o int2.o int4.o \
- int8.o
-
-raid6_pq-$(CONFIG_X86) += recov_ssse3.o recov_avx2.o mmx.o sse1.o sse2.o avx2.o avx512.o recov_avx512.o
-raid6_pq-$(CONFIG_ALTIVEC) += altivec1.o altivec2.o altivec4.o altivec8.o \
- vpermxor1.o vpermxor2.o vpermxor4.o vpermxor8.o
-raid6_pq-$(CONFIG_KERNEL_MODE_NEON) += neon.o neon1.o neon2.o neon4.o neon8.o recov_neon.o recov_neon_inner.o
-raid6_pq-$(CONFIG_S390) += s390vx8.o recov_s390xc.o
-raid6_pq-$(CONFIG_LOONGARCH) += loongarch_simd.o recov_loongarch_simd.o
-raid6_pq-$(CONFIG_RISCV_ISA_V) += rvv.o recov_rvv.o
-
-hostprogs += mktables
-
-ifeq ($(CONFIG_ALTIVEC),y)
-altivec_flags := -maltivec $(call cc-option,-mabi=altivec)
-# Enable <altivec.h>
-altivec_flags += -isystem $(shell $(CC) -print-file-name=include)
-
-ifdef CONFIG_CC_IS_CLANG
-# clang ppc port does not yet support -maltivec when -msoft-float is
-# enabled. A future release of clang will resolve this
-# https://llvm.org/pr31177
-CFLAGS_REMOVE_altivec1.o += -msoft-float
-CFLAGS_REMOVE_altivec2.o += -msoft-float
-CFLAGS_REMOVE_altivec4.o += -msoft-float
-CFLAGS_REMOVE_altivec8.o += -msoft-float
-CFLAGS_REMOVE_vpermxor1.o += -msoft-float
-CFLAGS_REMOVE_vpermxor2.o += -msoft-float
-CFLAGS_REMOVE_vpermxor4.o += -msoft-float
-CFLAGS_REMOVE_vpermxor8.o += -msoft-float
-endif
-endif
-
-quiet_cmd_unroll = UNROLL $@
- cmd_unroll = $(AWK) -v N=$* -f $(src)/unroll.awk < $< > $@
-
-targets += int1.c int2.c int4.c int8.c
-$(obj)/int%.c: $(src)/int.uc $(src)/unroll.awk FORCE
- $(call if_changed,unroll)
-
-CFLAGS_altivec1.o += $(altivec_flags)
-CFLAGS_altivec2.o += $(altivec_flags)
-CFLAGS_altivec4.o += $(altivec_flags)
-CFLAGS_altivec8.o += $(altivec_flags)
-targets += altivec1.c altivec2.c altivec4.c altivec8.c
-$(obj)/altivec%.c: $(src)/altivec.uc $(src)/unroll.awk FORCE
- $(call if_changed,unroll)
-
-CFLAGS_vpermxor1.o += $(altivec_flags)
-CFLAGS_vpermxor2.o += $(altivec_flags)
-CFLAGS_vpermxor4.o += $(altivec_flags)
-CFLAGS_vpermxor8.o += $(altivec_flags)
-targets += vpermxor1.c vpermxor2.c vpermxor4.c vpermxor8.c
-$(obj)/vpermxor%.c: $(src)/vpermxor.uc $(src)/unroll.awk FORCE
- $(call if_changed,unroll)
-
-CFLAGS_neon1.o += $(CC_FLAGS_FPU)
-CFLAGS_neon2.o += $(CC_FLAGS_FPU)
-CFLAGS_neon4.o += $(CC_FLAGS_FPU)
-CFLAGS_neon8.o += $(CC_FLAGS_FPU)
-CFLAGS_recov_neon_inner.o += $(CC_FLAGS_FPU)
-CFLAGS_REMOVE_neon1.o += $(CC_FLAGS_NO_FPU)
-CFLAGS_REMOVE_neon2.o += $(CC_FLAGS_NO_FPU)
-CFLAGS_REMOVE_neon4.o += $(CC_FLAGS_NO_FPU)
-CFLAGS_REMOVE_neon8.o += $(CC_FLAGS_NO_FPU)
-CFLAGS_REMOVE_recov_neon_inner.o += $(CC_FLAGS_NO_FPU)
-targets += neon1.c neon2.c neon4.c neon8.c
-$(obj)/neon%.c: $(src)/neon.uc $(src)/unroll.awk FORCE
- $(call if_changed,unroll)
-
-targets += s390vx8.c
-$(obj)/s390vx%.c: $(src)/s390vx.uc $(src)/unroll.awk FORCE
- $(call if_changed,unroll)
-
-quiet_cmd_mktable = TABLE $@
- cmd_mktable = $(obj)/mktables > $@
-
-targets += tables.c
-$(obj)/tables.c: $(obj)/mktables FORCE
- $(call if_changed,mktable)
diff --git a/lib/raid6/test/.gitignore b/lib/raid6/test/.gitignore
deleted file mode 100644
index 1b68a77f348f..000000000000
--- a/lib/raid6/test/.gitignore
+++ /dev/null
@@ -1,3 +0,0 @@
-/int.uc
-/neon.uc
-/raid6test
--
2.53.0
^ permalink raw reply related
* [PATCH 05/19] raid6: remove unused defines in pq.h
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
These are not used anywhere in the kernel.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
include/linux/raid/pq.h | 6 ------
1 file changed, 6 deletions(-)
diff --git a/include/linux/raid/pq.h b/include/linux/raid/pq.h
index d26788fada58..5e7e743b83f5 100644
--- a/include/linux/raid/pq.h
+++ b/include/linux/raid/pq.h
@@ -90,12 +90,6 @@ extern const struct raid6_calls raid6_neonx8;
extern const struct raid6_calls * const raid6_algos[];
extern const struct raid6_recov_calls *const raid6_recov_algos[];
-/* Return values from chk_syndrome */
-#define RAID6_OK 0
-#define RAID6_P_BAD 1
-#define RAID6_Q_BAD 2
-#define RAID6_PQ_BAD 3
-
/* Galois field tables */
extern const u8 raid6_gfmul[256][256] __attribute__((aligned(256)));
extern const u8 raid6_vgfmul[256][32] __attribute__((aligned(256)));
--
2.53.0
^ permalink raw reply related
* [PATCH 06/19] raid6: remove raid6_get_zero_page
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Just open code it as in other places in the kernel.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
crypto/async_tx/async_pq.c | 2 +-
crypto/async_tx/async_raid6_recov.c | 4 ++--
include/linux/raid/pq.h | 6 ------
lib/raid/raid6/arm/recov_neon.c | 6 +++---
lib/raid/raid6/loongarch/recov_loongarch_simd.c | 12 ++++++------
lib/raid/raid6/recov.c | 6 +++---
lib/raid/raid6/riscv/recov_rvv.c | 6 +++---
lib/raid/raid6/s390/recov_s390xc.c | 6 +++---
lib/raid/raid6/x86/recov_avx2.c | 6 +++---
lib/raid/raid6/x86/recov_avx512.c | 6 +++---
lib/raid/raid6/x86/recov_ssse3.c | 6 +++---
11 files changed, 30 insertions(+), 36 deletions(-)
diff --git a/crypto/async_tx/async_pq.c b/crypto/async_tx/async_pq.c
index 9e4bb7fbde25..0ce6f07b4e0d 100644
--- a/crypto/async_tx/async_pq.c
+++ b/crypto/async_tx/async_pq.c
@@ -119,7 +119,7 @@ do_sync_gen_syndrome(struct page **blocks, unsigned int *offsets, int disks,
for (i = 0; i < disks; i++) {
if (blocks[i] == NULL) {
BUG_ON(i > disks - 3); /* P or Q can't be zero */
- srcs[i] = raid6_get_zero_page();
+ srcs[i] = page_address(ZERO_PAGE(0));
} else {
srcs[i] = page_address(blocks[i]) + offsets[i];
diff --git a/crypto/async_tx/async_raid6_recov.c b/crypto/async_tx/async_raid6_recov.c
index 539ea5b378dc..f2dc6af6e6a7 100644
--- a/crypto/async_tx/async_raid6_recov.c
+++ b/crypto/async_tx/async_raid6_recov.c
@@ -414,7 +414,7 @@ async_raid6_2data_recov(int disks, size_t bytes, int faila, int failb,
async_tx_quiesce(&submit->depend_tx);
for (i = 0; i < disks; i++)
if (blocks[i] == NULL)
- ptrs[i] = raid6_get_zero_page();
+ ptrs[i] = page_address(ZERO_PAGE(0));
else
ptrs[i] = page_address(blocks[i]) + offs[i];
@@ -497,7 +497,7 @@ async_raid6_datap_recov(int disks, size_t bytes, int faila,
async_tx_quiesce(&submit->depend_tx);
for (i = 0; i < disks; i++)
if (blocks[i] == NULL)
- ptrs[i] = raid6_get_zero_page();
+ ptrs[i] = page_address(ZERO_PAGE(0));
else
ptrs[i] = page_address(blocks[i]) + offs[i];
diff --git a/include/linux/raid/pq.h b/include/linux/raid/pq.h
index 5e7e743b83f5..f27a866c287f 100644
--- a/include/linux/raid/pq.h
+++ b/include/linux/raid/pq.h
@@ -11,12 +11,6 @@
#include <linux/blkdev.h>
#include <linux/mm.h>
-/* This should be const but the raid6 code is too convoluted for that. */
-static inline void *raid6_get_zero_page(void)
-{
- return page_address(ZERO_PAGE(0));
-}
-
/* Routine choices */
struct raid6_calls {
void (*gen_syndrome)(int, size_t, void **);
diff --git a/lib/raid/raid6/arm/recov_neon.c b/lib/raid/raid6/arm/recov_neon.c
index 5a48fcc762e8..9993bda5d3a6 100644
--- a/lib/raid/raid6/arm/recov_neon.c
+++ b/lib/raid/raid6/arm/recov_neon.c
@@ -29,10 +29,10 @@ static void raid6_2data_recov_neon(int disks, size_t bytes, int faila,
* delta p and delta q
*/
dp = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 2] = dp;
dq = (u8 *)ptrs[failb];
- ptrs[failb] = raid6_get_zero_page();
+ ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
@@ -66,7 +66,7 @@ static void raid6_datap_recov_neon(int disks, size_t bytes, int faila,
* Use the dead data page as temporary storage for delta q
*/
dq = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
diff --git a/lib/raid/raid6/loongarch/recov_loongarch_simd.c b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
index eb3a1e79f01f..4d4563209647 100644
--- a/lib/raid/raid6/loongarch/recov_loongarch_simd.c
+++ b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
@@ -43,10 +43,10 @@ static void raid6_2data_recov_lsx(int disks, size_t bytes, int faila,
* delta p and delta q
*/
dp = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 2] = dp;
dq = (u8 *)ptrs[failb];
- ptrs[failb] = raid6_get_zero_page();
+ ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
@@ -198,7 +198,7 @@ static void raid6_datap_recov_lsx(int disks, size_t bytes, int faila,
* Use the dead data page as temporary storage for delta q
*/
dq = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
@@ -317,10 +317,10 @@ static void raid6_2data_recov_lasx(int disks, size_t bytes, int faila,
* delta p and delta q
*/
dp = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 2] = dp;
dq = (u8 *)ptrs[failb];
- ptrs[failb] = raid6_get_zero_page();
+ ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
@@ -437,7 +437,7 @@ static void raid6_datap_recov_lasx(int disks, size_t bytes, int faila,
* Use the dead data page as temporary storage for delta q
*/
dq = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
diff --git a/lib/raid/raid6/recov.c b/lib/raid/raid6/recov.c
index 8d113196632e..211e1df28963 100644
--- a/lib/raid/raid6/recov.c
+++ b/lib/raid/raid6/recov.c
@@ -31,10 +31,10 @@ static void raid6_2data_recov_intx1(int disks, size_t bytes, int faila,
Use the dead data pages as temporary storage for
delta p and delta q */
dp = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-2] = dp;
dq = (u8 *)ptrs[failb];
- ptrs[failb] = raid6_get_zero_page();
+ ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
@@ -72,7 +72,7 @@ static void raid6_datap_recov_intx1(int disks, size_t bytes, int faila,
/* Compute syndrome with zero for the missing data page
Use the dead data page as temporary storage for delta q */
dq = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
diff --git a/lib/raid/raid6/riscv/recov_rvv.c b/lib/raid/raid6/riscv/recov_rvv.c
index 40c393206b6a..f77d9c430687 100644
--- a/lib/raid/raid6/riscv/recov_rvv.c
+++ b/lib/raid/raid6/riscv/recov_rvv.c
@@ -158,10 +158,10 @@ static void raid6_2data_recov_rvv(int disks, size_t bytes, int faila,
* delta p and delta q
*/
dp = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 2] = dp;
dq = (u8 *)ptrs[failb];
- ptrs[failb] = raid6_get_zero_page();
+ ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
@@ -196,7 +196,7 @@ static void raid6_datap_recov_rvv(int disks, size_t bytes, int faila,
* Use the dead data page as temporary storage for delta q
*/
dq = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
diff --git a/lib/raid/raid6/s390/recov_s390xc.c b/lib/raid/raid6/s390/recov_s390xc.c
index 487018f81192..0f32217b7123 100644
--- a/lib/raid/raid6/s390/recov_s390xc.c
+++ b/lib/raid/raid6/s390/recov_s390xc.c
@@ -34,10 +34,10 @@ static void raid6_2data_recov_s390xc(int disks, size_t bytes, int faila,
Use the dead data pages as temporary storage for
delta p and delta q */
dp = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-2] = dp;
dq = (u8 *)ptrs[failb];
- ptrs[failb] = raid6_get_zero_page();
+ ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
@@ -81,7 +81,7 @@ static void raid6_datap_recov_s390xc(int disks, size_t bytes, int faila,
/* Compute syndrome with zero for the missing data page
Use the dead data page as temporary storage for delta q */
dq = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
diff --git a/lib/raid/raid6/x86/recov_avx2.c b/lib/raid/raid6/x86/recov_avx2.c
index 19fbd9c4dce6..325310c81e1c 100644
--- a/lib/raid/raid6/x86/recov_avx2.c
+++ b/lib/raid/raid6/x86/recov_avx2.c
@@ -28,10 +28,10 @@ static void raid6_2data_recov_avx2(int disks, size_t bytes, int faila,
Use the dead data pages as temporary storage for
delta p and delta q */
dp = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-2] = dp;
dq = (u8 *)ptrs[failb];
- ptrs[failb] = raid6_get_zero_page();
+ ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
@@ -196,7 +196,7 @@ static void raid6_datap_recov_avx2(int disks, size_t bytes, int faila,
/* Compute syndrome with zero for the missing data page
Use the dead data page as temporary storage for delta q */
dq = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
diff --git a/lib/raid/raid6/x86/recov_avx512.c b/lib/raid/raid6/x86/recov_avx512.c
index 143f4976b2ad..08de77fcb8bd 100644
--- a/lib/raid/raid6/x86/recov_avx512.c
+++ b/lib/raid/raid6/x86/recov_avx512.c
@@ -37,10 +37,10 @@ static void raid6_2data_recov_avx512(int disks, size_t bytes, int faila,
*/
dp = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-2] = dp;
dq = (u8 *)ptrs[failb];
- ptrs[failb] = raid6_get_zero_page();
+ ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
@@ -238,7 +238,7 @@ static void raid6_datap_recov_avx512(int disks, size_t bytes, int faila,
*/
dq = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
diff --git a/lib/raid/raid6/x86/recov_ssse3.c b/lib/raid/raid6/x86/recov_ssse3.c
index 146cdbf465bd..002bef1e0847 100644
--- a/lib/raid/raid6/x86/recov_ssse3.c
+++ b/lib/raid/raid6/x86/recov_ssse3.c
@@ -30,10 +30,10 @@ static void raid6_2data_recov_ssse3(int disks, size_t bytes, int faila,
Use the dead data pages as temporary storage for
delta p and delta q */
dp = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-2] = dp;
dq = (u8 *)ptrs[failb];
- ptrs[failb] = raid6_get_zero_page();
+ ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
@@ -203,7 +203,7 @@ static void raid6_datap_recov_ssse3(int disks, size_t bytes, int faila,
/* Compute syndrome with zero for the missing data page
Use the dead data page as temporary storage for delta q */
dq = (u8 *)ptrs[faila];
- ptrs[faila] = raid6_get_zero_page();
+ ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
raid6_call.gen_syndrome(disks, bytes, ptrs);
--
2.53.0
^ permalink raw reply related
* [PATCH 07/19] raid6: use named initializers for struct raid6_calls
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/arm/neon.c | 9 +++----
lib/raid/raid6/int.uc | 8 +++---
lib/raid/raid6/loongarch/loongarch_simd.c | 18 ++++++-------
lib/raid/raid6/powerpc/altivec.uc | 8 +++---
lib/raid/raid6/powerpc/vpermxor.uc | 8 +++---
lib/raid/raid6/riscv/rvv.h | 9 +++----
lib/raid/raid6/s390/s390vx.uc | 10 +++----
lib/raid/raid6/x86/avx2.c | 33 ++++++++++++-----------
lib/raid/raid6/x86/avx512.c | 33 ++++++++++++-----------
lib/raid/raid6/x86/mmx.c | 16 +++++------
lib/raid/raid6/x86/sse1.c | 18 ++++++-------
lib/raid/raid6/x86/sse2.c | 30 ++++++++++-----------
12 files changed, 95 insertions(+), 105 deletions(-)
diff --git a/lib/raid/raid6/arm/neon.c b/lib/raid/raid6/arm/neon.c
index 47b8bb0afc65..c21da59ab48f 100644
--- a/lib/raid/raid6/arm/neon.c
+++ b/lib/raid/raid6/arm/neon.c
@@ -40,11 +40,10 @@
start, stop, (unsigned long)bytes, ptrs);\
} \
struct raid6_calls const raid6_neonx ## _n = { \
- raid6_neon ## _n ## _gen_syndrome, \
- raid6_neon ## _n ## _xor_syndrome, \
- raid6_have_neon, \
- "neonx" #_n, \
- 0 \
+ .gen_syndrome = raid6_neon ## _n ## _gen_syndrome, \
+ .xor_syndrome = raid6_neon ## _n ## _xor_syndrome, \
+ .valid = raid6_have_neon, \
+ .name = "neonx" #_n, \
}
static int raid6_have_neon(void)
diff --git a/lib/raid/raid6/int.uc b/lib/raid/raid6/int.uc
index 1ba56c3fa482..4f5f2869e21e 100644
--- a/lib/raid/raid6/int.uc
+++ b/lib/raid/raid6/int.uc
@@ -139,9 +139,7 @@ static void raid6_int$#_xor_syndrome(int disks, int start, int stop,
}
const struct raid6_calls raid6_intx$# = {
- raid6_int$#_gen_syndrome,
- raid6_int$#_xor_syndrome,
- NULL, /* always valid */
- "int" NSTRING "x$#",
- 0
+ .gen_syndrome = raid6_int$#_gen_syndrome,
+ .xor_syndrome = raid6_int$#_xor_syndrome,
+ .name = "int" NSTRING "x$#",
};
diff --git a/lib/raid/raid6/loongarch/loongarch_simd.c b/lib/raid/raid6/loongarch/loongarch_simd.c
index 72f4d92d4876..1b4cd1512d05 100644
--- a/lib/raid/raid6/loongarch/loongarch_simd.c
+++ b/lib/raid/raid6/loongarch/loongarch_simd.c
@@ -244,11 +244,10 @@ static void raid6_lsx_xor_syndrome(int disks, int start, int stop,
}
const struct raid6_calls raid6_lsx = {
- raid6_lsx_gen_syndrome,
- raid6_lsx_xor_syndrome,
- raid6_has_lsx,
- "lsx",
- .priority = 0 /* see the comment near the top of the file for reason */
+ .gen_syndrome = raid6_lsx_gen_syndrome,
+ .xor_syndrome = raid6_lsx_xor_syndrome,
+ .valid = raid6_has_lsx,
+ .name = "lsx",
};
#undef NSIZE
@@ -413,11 +412,10 @@ static void raid6_lasx_xor_syndrome(int disks, int start, int stop,
}
const struct raid6_calls raid6_lasx = {
- raid6_lasx_gen_syndrome,
- raid6_lasx_xor_syndrome,
- raid6_has_lasx,
- "lasx",
- .priority = 0 /* see the comment near the top of the file for reason */
+ .gen_syndrome = raid6_lasx_gen_syndrome,
+ .xor_syndrome = raid6_lasx_xor_syndrome,
+ .valid = raid6_has_lasx,
+ .name = "lasx",
};
#undef NSIZE
#endif /* CONFIG_CPU_HAS_LASX */
diff --git a/lib/raid/raid6/powerpc/altivec.uc b/lib/raid/raid6/powerpc/altivec.uc
index 130d3d3dd42c..084ead768ddb 100644
--- a/lib/raid/raid6/powerpc/altivec.uc
+++ b/lib/raid/raid6/powerpc/altivec.uc
@@ -114,9 +114,7 @@ int raid6_have_altivec(void)
#endif
const struct raid6_calls raid6_altivec$# = {
- raid6_altivec$#_gen_syndrome,
- NULL, /* XOR not yet implemented */
- raid6_have_altivec,
- "altivecx$#",
- 0
+ .gen_syndrome = raid6_altivec$#_gen_syndrome,
+ .valid = raid6_have_altivec,
+ .name = "altivecx$#",
};
diff --git a/lib/raid/raid6/powerpc/vpermxor.uc b/lib/raid/raid6/powerpc/vpermxor.uc
index 595f20aaf4cf..bb2c3a316ae8 100644
--- a/lib/raid/raid6/powerpc/vpermxor.uc
+++ b/lib/raid/raid6/powerpc/vpermxor.uc
@@ -87,9 +87,7 @@ int raid6_have_altivec_vpermxor(void)
#endif
const struct raid6_calls raid6_vpermxor$# = {
- raid6_vpermxor$#_gen_syndrome,
- NULL,
- raid6_have_altivec_vpermxor,
- "vpermxor$#",
- 0
+ .gen_syndrome = raid6_vpermxor$#_gen_syndrome,
+ .valid = raid6_have_altivec_vpermxor,
+ .name = "vpermxor$#",
};
diff --git a/lib/raid/raid6/riscv/rvv.h b/lib/raid/raid6/riscv/rvv.h
index b0a71b375962..0d430a4c5f08 100644
--- a/lib/raid/raid6/riscv/rvv.h
+++ b/lib/raid/raid6/riscv/rvv.h
@@ -39,9 +39,8 @@ static int rvv_has_vector(void)
kernel_vector_end(); \
} \
struct raid6_calls const raid6_rvvx ## _n = { \
- raid6_rvv ## _n ## _gen_syndrome, \
- raid6_rvv ## _n ## _xor_syndrome, \
- rvv_has_vector, \
- "rvvx" #_n, \
- 0 \
+ .gen_syndrome = raid6_rvv ## _n ## _gen_syndrome, \
+ .xor_syndrome = raid6_rvv ## _n ## _xor_syndrome, \
+ .valid = rvv_has_vector, \
+ .name = "rvvx" #_n, \
}
diff --git a/lib/raid/raid6/s390/s390vx.uc b/lib/raid/raid6/s390/s390vx.uc
index 8aa53eb2f395..97c5d5d9dcf9 100644
--- a/lib/raid/raid6/s390/s390vx.uc
+++ b/lib/raid/raid6/s390/s390vx.uc
@@ -127,9 +127,9 @@ static int raid6_s390vx$#_valid(void)
}
const struct raid6_calls raid6_s390vx$# = {
- raid6_s390vx$#_gen_syndrome,
- raid6_s390vx$#_xor_syndrome,
- raid6_s390vx$#_valid,
- "vx128x$#",
- 1
+ .gen_syndrome = raid6_s390vx$#_gen_syndrome,
+ .xor_syndrome = raid6_s390vx$#_xor_syndrome,
+ .valid = raid6_s390vx$#_valid,
+ .name = "vx128x$#",
+ .priority = 1,
};
diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
index a1a5213918af..aab8b624c635 100644
--- a/lib/raid/raid6/x86/avx2.c
+++ b/lib/raid/raid6/x86/avx2.c
@@ -128,11 +128,12 @@ static void raid6_avx21_xor_syndrome(int disks, int start, int stop,
}
const struct raid6_calls raid6_avx2x1 = {
- raid6_avx21_gen_syndrome,
- raid6_avx21_xor_syndrome,
- raid6_have_avx2,
- "avx2x1",
- .priority = 2 /* Prefer AVX2 over priority 1 (SSE2 and others) */
+ .gen_syndrome = raid6_avx21_gen_syndrome,
+ .xor_syndrome = raid6_avx21_xor_syndrome,
+ .valid = raid6_have_avx2,
+ .name = "avx2x1",
+ /* Prefer AVX2 over priority 1 (SSE2 and others) */
+ .priority = 2,
};
/*
@@ -258,11 +259,12 @@ static void raid6_avx22_xor_syndrome(int disks, int start, int stop,
}
const struct raid6_calls raid6_avx2x2 = {
- raid6_avx22_gen_syndrome,
- raid6_avx22_xor_syndrome,
- raid6_have_avx2,
- "avx2x2",
- .priority = 2 /* Prefer AVX2 over priority 1 (SSE2 and others) */
+ .gen_syndrome = raid6_avx22_gen_syndrome,
+ .xor_syndrome = raid6_avx22_xor_syndrome,
+ .valid = raid6_have_avx2,
+ .name = "avx2x2",
+ /* Prefer AVX2 over priority 1 (SSE2 and others) */
+ .priority = 2,
};
#ifdef CONFIG_X86_64
@@ -461,10 +463,11 @@ static void raid6_avx24_xor_syndrome(int disks, int start, int stop,
}
const struct raid6_calls raid6_avx2x4 = {
- raid6_avx24_gen_syndrome,
- raid6_avx24_xor_syndrome,
- raid6_have_avx2,
- "avx2x4",
- .priority = 2 /* Prefer AVX2 over priority 1 (SSE2 and others) */
+ .gen_syndrome = raid6_avx24_gen_syndrome,
+ .xor_syndrome = raid6_avx24_xor_syndrome,
+ .valid = raid6_have_avx2,
+ .name = "avx2x4",
+ /* Prefer AVX2 over priority 1 (SSE2 and others) */
+ .priority = 2,
};
#endif /* CONFIG_X86_64 */
diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index 874998bcd7d7..47636b16632f 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -156,11 +156,12 @@ static void raid6_avx5121_xor_syndrome(int disks, int start, int stop,
}
const struct raid6_calls raid6_avx512x1 = {
- raid6_avx5121_gen_syndrome,
- raid6_avx5121_xor_syndrome,
- raid6_have_avx512,
- "avx512x1",
- .priority = 2 /* Prefer AVX512 over priority 1 (SSE2 and others) */
+ .gen_syndrome = raid6_avx5121_gen_syndrome,
+ .xor_syndrome = raid6_avx5121_xor_syndrome,
+ .valid = raid6_have_avx512,
+ .name = "avx512x1",
+ /* Prefer AVX512 over priority 1 (SSE2 and others) */
+ .priority = 2,
};
/*
@@ -313,11 +314,12 @@ static void raid6_avx5122_xor_syndrome(int disks, int start, int stop,
}
const struct raid6_calls raid6_avx512x2 = {
- raid6_avx5122_gen_syndrome,
- raid6_avx5122_xor_syndrome,
- raid6_have_avx512,
- "avx512x2",
- .priority = 2 /* Prefer AVX512 over priority 1 (SSE2 and others) */
+ .gen_syndrome = raid6_avx5122_gen_syndrome,
+ .xor_syndrome = raid6_avx5122_xor_syndrome,
+ .valid = raid6_have_avx512,
+ .name = "avx512x2",
+ /* Prefer AVX512 over priority 1 (SSE2 and others) */
+ .priority = 2,
};
#ifdef CONFIG_X86_64
@@ -551,10 +553,11 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop,
kernel_fpu_end();
}
const struct raid6_calls raid6_avx512x4 = {
- raid6_avx5124_gen_syndrome,
- raid6_avx5124_xor_syndrome,
- raid6_have_avx512,
- "avx512x4",
- .priority = 2 /* Prefer AVX512 over priority 1 (SSE2 and others) */
+ .gen_syndrome = raid6_avx5124_gen_syndrome,
+ .xor_syndrome = raid6_avx5124_xor_syndrome,
+ .valid = raid6_have_avx512,
+ .name = "avx512x4",
+ /* Prefer AVX512 over priority 1 (SSE2 and others) */
+ .priority = 2,
};
#endif
diff --git a/lib/raid/raid6/x86/mmx.c b/lib/raid/raid6/x86/mmx.c
index 7e9810669347..22b9fdaa705f 100644
--- a/lib/raid/raid6/x86/mmx.c
+++ b/lib/raid/raid6/x86/mmx.c
@@ -68,11 +68,9 @@ static void raid6_mmx1_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
const struct raid6_calls raid6_mmxx1 = {
- raid6_mmx1_gen_syndrome,
- NULL, /* XOR not yet implemented */
- raid6_have_mmx,
- "mmxx1",
- 0
+ .gen_syndrome = raid6_mmx1_gen_syndrome,
+ .valid = raid6_have_mmx,
+ .name = "mmxx1",
};
/*
@@ -127,9 +125,7 @@ static void raid6_mmx2_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
const struct raid6_calls raid6_mmxx2 = {
- raid6_mmx2_gen_syndrome,
- NULL, /* XOR not yet implemented */
- raid6_have_mmx,
- "mmxx2",
- 0
+ .gen_syndrome = raid6_mmx2_gen_syndrome,
+ .valid = raid6_have_mmx,
+ .name = "mmxx2",
};
diff --git a/lib/raid/raid6/x86/sse1.c b/lib/raid/raid6/x86/sse1.c
index deecdd72ceec..fad214a430d8 100644
--- a/lib/raid/raid6/x86/sse1.c
+++ b/lib/raid/raid6/x86/sse1.c
@@ -84,11 +84,10 @@ static void raid6_sse11_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
const struct raid6_calls raid6_sse1x1 = {
- raid6_sse11_gen_syndrome,
- NULL, /* XOR not yet implemented */
- raid6_have_sse1_or_mmxext,
- "sse1x1",
- 1 /* Has cache hints */
+ .gen_syndrome = raid6_sse11_gen_syndrome,
+ .valid = raid6_have_sse1_or_mmxext,
+ .name = "sse1x1",
+ .priority = 1, /* Has cache hints */
};
/*
@@ -147,9 +146,8 @@ static void raid6_sse12_gen_syndrome(int disks, size_t bytes, void **ptrs)
}
const struct raid6_calls raid6_sse1x2 = {
- raid6_sse12_gen_syndrome,
- NULL, /* XOR not yet implemented */
- raid6_have_sse1_or_mmxext,
- "sse1x2",
- 1 /* Has cache hints */
+ .gen_syndrome = raid6_sse12_gen_syndrome,
+ .valid = raid6_have_sse1_or_mmxext,
+ .name = "sse1x2",
+ .priority = 1, /* Has cache hints */
};
diff --git a/lib/raid/raid6/x86/sse2.c b/lib/raid/raid6/x86/sse2.c
index f9edf8a8d1c4..1b28e858a1d4 100644
--- a/lib/raid/raid6/x86/sse2.c
+++ b/lib/raid/raid6/x86/sse2.c
@@ -133,11 +133,11 @@ static void raid6_sse21_xor_syndrome(int disks, int start, int stop,
}
const struct raid6_calls raid6_sse2x1 = {
- raid6_sse21_gen_syndrome,
- raid6_sse21_xor_syndrome,
- raid6_have_sse2,
- "sse2x1",
- 1 /* Has cache hints */
+ .gen_syndrome = raid6_sse21_gen_syndrome,
+ .xor_syndrome = raid6_sse21_xor_syndrome,
+ .valid = raid6_have_sse2,
+ .name = "sse2x1",
+ .priority = 1, /* Has cache hints */
};
/*
@@ -263,11 +263,11 @@ static void raid6_sse22_xor_syndrome(int disks, int start, int stop,
}
const struct raid6_calls raid6_sse2x2 = {
- raid6_sse22_gen_syndrome,
- raid6_sse22_xor_syndrome,
- raid6_have_sse2,
- "sse2x2",
- 1 /* Has cache hints */
+ .gen_syndrome = raid6_sse22_gen_syndrome,
+ .xor_syndrome = raid6_sse22_xor_syndrome,
+ .valid = raid6_have_sse2,
+ .name = "sse2x2",
+ .priority = 1, /* Has cache hints */
};
#ifdef CONFIG_X86_64
@@ -470,11 +470,11 @@ static void raid6_sse24_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_sse2x4 = {
- raid6_sse24_gen_syndrome,
- raid6_sse24_xor_syndrome,
- raid6_have_sse2,
- "sse2x4",
- 1 /* Has cache hints */
+ .gen_syndrome = raid6_sse24_gen_syndrome,
+ .xor_syndrome = raid6_sse24_xor_syndrome,
+ .valid = raid6_have_sse2,
+ .name = "sse2x4",
+ .priority = 1, /* Has cache hints */
};
#endif /* CONFIG_X86_64 */
--
2.53.0
^ permalink raw reply related
* [PATCH 09/19] raid6: hide internals
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Split out two new headers from the public pq.h:
- lib/raid/raid6/algos.h contains the algorithm lists private to
lib/raid/raid6
- include/linux/raid/pq_tables.h contains the tables also used by
async_tx providers.
The public include/linux/pq.h is now limited to the public interface for
the consumers of the RAID6 PQ API.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
crypto/async_tx/async_pq.c | 1 +
crypto/async_tx/async_raid6_recov.c | 1 +
drivers/dma/bcm-sba-raid.c | 1 +
include/linux/raid/pq.h | 96 ++-----------------
include/linux/raid/pq_tables.h | 19 ++++
lib/raid/raid6/Makefile | 2 +
lib/raid/raid6/algos.c | 3 +-
lib/raid/raid6/algos.h | 82 ++++++++++++++++
lib/raid/raid6/arm/neon.c | 2 +-
lib/raid/raid6/arm/recov_neon.c | 2 +
lib/raid/raid6/int.uc | 2 +-
lib/raid/raid6/loongarch/loongarch_simd.c | 2 +-
.../raid6/loongarch/recov_loongarch_simd.c | 2 +
lib/raid/raid6/mktables.c | 2 +-
lib/raid/raid6/powerpc/altivec.uc | 2 +-
lib/raid/raid6/powerpc/vpermxor.uc | 2 +-
lib/raid/raid6/recov.c | 2 +
lib/raid/raid6/riscv/recov_rvv.c | 2 +
lib/raid/raid6/riscv/rvv.h | 2 +-
lib/raid/raid6/s390/recov_s390xc.c | 2 +
lib/raid/raid6/s390/s390vx.uc | 2 +-
lib/raid/raid6/tests/raid6_kunit.c | 2 +-
lib/raid/raid6/x86/avx2.c | 3 +-
lib/raid/raid6/x86/avx512.c | 3 +-
lib/raid/raid6/x86/mmx.c | 3 +-
lib/raid/raid6/x86/recov_avx2.c | 2 +
lib/raid/raid6/x86/recov_avx512.c | 2 +
lib/raid/raid6/x86/recov_ssse3.c | 2 +
lib/raid/raid6/x86/sse1.c | 3 +-
lib/raid/raid6/x86/sse2.c | 3 +-
30 files changed, 151 insertions(+), 103 deletions(-)
create mode 100644 include/linux/raid/pq_tables.h
create mode 100644 lib/raid/raid6/algos.h
diff --git a/crypto/async_tx/async_pq.c b/crypto/async_tx/async_pq.c
index f3574f80d1df..27f99349e310 100644
--- a/crypto/async_tx/async_pq.c
+++ b/crypto/async_tx/async_pq.c
@@ -8,6 +8,7 @@
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/raid/pq.h>
+#include <linux/raid/pq_tables.h>
#include <linux/async_tx.h>
#include <linux/gfp.h>
diff --git a/crypto/async_tx/async_raid6_recov.c b/crypto/async_tx/async_raid6_recov.c
index 305ea1421a3e..e53870d84bc5 100644
--- a/crypto/async_tx/async_raid6_recov.c
+++ b/crypto/async_tx/async_raid6_recov.c
@@ -11,6 +11,7 @@
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/raid/pq.h>
+#include <linux/raid/pq_tables.h>
#include <linux/async_tx.h>
#include <linux/dmaengine.h>
diff --git a/drivers/dma/bcm-sba-raid.c b/drivers/dma/bcm-sba-raid.c
index ed037fa883f6..0de03611252e 100644
--- a/drivers/dma/bcm-sba-raid.c
+++ b/drivers/dma/bcm-sba-raid.c
@@ -40,6 +40,7 @@
#include <linux/platform_device.h>
#include <linux/slab.h>
#include <linux/raid/pq.h>
+#include <linux/raid/pq_tables.h>
#include "dmaengine.h"
diff --git a/include/linux/raid/pq.h b/include/linux/raid/pq.h
index 662c2669f63f..760dd41a10b0 100644
--- a/include/linux/raid/pq.h
+++ b/include/linux/raid/pq.h
@@ -1,15 +1,13 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2003 H. Peter Anvin - All Rights Reserved
+/*
+ * Copyright 2003 H. Peter Anvin - All Rights Reserved
*
- * ----------------------------------------------------------------------- */
-
-#ifndef LINUX_RAID_RAID6_H
-#define LINUX_RAID_RAID6_H
+ * Public interface to the RAID6 P/Q calculation and recovery library.
+ */
+#ifndef LINUX_RAID_PQ_H
+#define LINUX_RAID_PQ_H
-#include <linux/blkdev.h>
-#include <linux/mm.h>
+#include <linux/types.h>
/*
* While the RAID6 algorithm could in theory support 3 devices by just copying
@@ -30,82 +28,4 @@ void raid6_recov_2data(int disks, size_t bytes, int faila, int failb,
void raid6_recov_datap(int disks, size_t bytes, int faila,
void **ptrs);
-/* Routine choices */
-struct raid6_calls {
- void (*gen_syndrome)(int, size_t, void **);
- void (*xor_syndrome)(int, int, int, size_t, void **);
- int (*valid)(void); /* Returns 1 if this routine set is usable */
- const char *name; /* Name of this routine set */
- int priority; /* Relative priority ranking if non-zero */
-};
-
-/* Various routine sets */
-extern const struct raid6_calls raid6_intx1;
-extern const struct raid6_calls raid6_intx2;
-extern const struct raid6_calls raid6_intx4;
-extern const struct raid6_calls raid6_intx8;
-extern const struct raid6_calls raid6_mmxx1;
-extern const struct raid6_calls raid6_mmxx2;
-extern const struct raid6_calls raid6_sse1x1;
-extern const struct raid6_calls raid6_sse1x2;
-extern const struct raid6_calls raid6_sse2x1;
-extern const struct raid6_calls raid6_sse2x2;
-extern const struct raid6_calls raid6_sse2x4;
-extern const struct raid6_calls raid6_altivec1;
-extern const struct raid6_calls raid6_altivec2;
-extern const struct raid6_calls raid6_altivec4;
-extern const struct raid6_calls raid6_altivec8;
-extern const struct raid6_calls raid6_avx2x1;
-extern const struct raid6_calls raid6_avx2x2;
-extern const struct raid6_calls raid6_avx2x4;
-extern const struct raid6_calls raid6_avx512x1;
-extern const struct raid6_calls raid6_avx512x2;
-extern const struct raid6_calls raid6_avx512x4;
-extern const struct raid6_calls raid6_s390vx8;
-extern const struct raid6_calls raid6_vpermxor1;
-extern const struct raid6_calls raid6_vpermxor2;
-extern const struct raid6_calls raid6_vpermxor4;
-extern const struct raid6_calls raid6_vpermxor8;
-extern const struct raid6_calls raid6_lsx;
-extern const struct raid6_calls raid6_lasx;
-extern const struct raid6_calls raid6_rvvx1;
-extern const struct raid6_calls raid6_rvvx2;
-extern const struct raid6_calls raid6_rvvx4;
-extern const struct raid6_calls raid6_rvvx8;
-
-struct raid6_recov_calls {
- void (*data2)(int, size_t, int, int, void **);
- void (*datap)(int, size_t, int, void **);
- int (*valid)(void);
- const char *name;
- int priority;
-};
-
-extern const struct raid6_recov_calls raid6_recov_intx1;
-extern const struct raid6_recov_calls raid6_recov_ssse3;
-extern const struct raid6_recov_calls raid6_recov_avx2;
-extern const struct raid6_recov_calls raid6_recov_avx512;
-extern const struct raid6_recov_calls raid6_recov_s390xc;
-extern const struct raid6_recov_calls raid6_recov_neon;
-extern const struct raid6_recov_calls raid6_recov_lsx;
-extern const struct raid6_recov_calls raid6_recov_lasx;
-extern const struct raid6_recov_calls raid6_recov_rvv;
-
-extern const struct raid6_calls raid6_neonx1;
-extern const struct raid6_calls raid6_neonx2;
-extern const struct raid6_calls raid6_neonx4;
-extern const struct raid6_calls raid6_neonx8;
-
-/* Algorithm list */
-extern const struct raid6_calls * const raid6_algos[];
-extern const struct raid6_recov_calls *const raid6_recov_algos[];
-
-/* Galois field tables */
-extern const u8 raid6_gfmul[256][256] __attribute__((aligned(256)));
-extern const u8 raid6_vgfmul[256][32] __attribute__((aligned(256)));
-extern const u8 raid6_gfexp[256] __attribute__((aligned(256)));
-extern const u8 raid6_gflog[256] __attribute__((aligned(256)));
-extern const u8 raid6_gfinv[256] __attribute__((aligned(256)));
-extern const u8 raid6_gfexi[256] __attribute__((aligned(256)));
-
-#endif /* LINUX_RAID_RAID6_H */
+#endif /* LINUX_RAID_PQ_H */
diff --git a/include/linux/raid/pq_tables.h b/include/linux/raid/pq_tables.h
new file mode 100644
index 000000000000..7b1ebe675677
--- /dev/null
+++ b/include/linux/raid/pq_tables.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright 2003 H. Peter Anvin - All Rights Reserved
+ *
+ * Galois field tables for the Linux RAID6 P/Q parity algorithm.
+ */
+#ifndef _LINUX_RAID_PQ_TABLES_H
+#define _LINUX_RAID_PQ_TABLES_H
+
+#include <linux/types.h>
+
+extern const u8 raid6_gfmul[256][256] __attribute__((aligned(256)));
+extern const u8 raid6_vgfmul[256][32] __attribute__((aligned(256)));
+extern const u8 raid6_gfexp[256] __attribute__((aligned(256)));
+extern const u8 raid6_gflog[256] __attribute__((aligned(256)));
+extern const u8 raid6_gfinv[256] __attribute__((aligned(256)));
+extern const u8 raid6_gfexi[256] __attribute__((aligned(256)));
+
+#endif /* _LINUX_RAID_PQ_TABLES_H */
diff --git a/lib/raid/raid6/Makefile b/lib/raid/raid6/Makefile
index 886a1771e78d..f64f6d32f28b 100644
--- a/lib/raid/raid6/Makefile
+++ b/lib/raid/raid6/Makefile
@@ -2,6 +2,8 @@
hostprogs += mktables
+ccflags-y += -I $(src)
+
obj-$(CONFIG_RAID6_PQ) += raid6_pq.o tests/
raid6_pq-y += algos.o tables.o
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index 683b97cb94ad..af31a1feb6e7 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -11,10 +11,11 @@
* Algorithm list and algorithm selection for RAID-6
*/
-#include <linux/raid/pq.h>
#include <linux/module.h>
#include <linux/gfp.h>
+#include <linux/raid/pq.h>
#include <kunit/visibility.h>
+#include "algos.h"
static const struct raid6_recov_calls *raid6_recov_algo;
diff --git a/lib/raid/raid6/algos.h b/lib/raid/raid6/algos.h
new file mode 100644
index 000000000000..e5f1098d2179
--- /dev/null
+++ b/lib/raid/raid6/algos.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright 2003 H. Peter Anvin - All Rights Reserved
+ */
+#ifndef _PQ_IMPL_H
+#define _PQ_IMPL_H
+
+#include <linux/raid/pq_tables.h>
+
+/* Routine choices */
+struct raid6_calls {
+ const char *name;
+ void (*gen_syndrome)(int disks, size_t bytes, void **ptrs);
+ void (*xor_syndrome)(int disks, int start, int stop, size_t bytes,
+ void **ptrs);
+ int (*valid)(void); /* Returns 1 if this routine set is usable */
+ int priority; /* Relative priority ranking if non-zero */
+};
+
+/* Various routine sets */
+extern const struct raid6_calls raid6_intx1;
+extern const struct raid6_calls raid6_intx2;
+extern const struct raid6_calls raid6_intx4;
+extern const struct raid6_calls raid6_intx8;
+extern const struct raid6_calls raid6_mmxx1;
+extern const struct raid6_calls raid6_mmxx2;
+extern const struct raid6_calls raid6_sse1x1;
+extern const struct raid6_calls raid6_sse1x2;
+extern const struct raid6_calls raid6_sse2x1;
+extern const struct raid6_calls raid6_sse2x2;
+extern const struct raid6_calls raid6_sse2x4;
+extern const struct raid6_calls raid6_altivec1;
+extern const struct raid6_calls raid6_altivec2;
+extern const struct raid6_calls raid6_altivec4;
+extern const struct raid6_calls raid6_altivec8;
+extern const struct raid6_calls raid6_avx2x1;
+extern const struct raid6_calls raid6_avx2x2;
+extern const struct raid6_calls raid6_avx2x4;
+extern const struct raid6_calls raid6_avx512x1;
+extern const struct raid6_calls raid6_avx512x2;
+extern const struct raid6_calls raid6_avx512x4;
+extern const struct raid6_calls raid6_s390vx8;
+extern const struct raid6_calls raid6_vpermxor1;
+extern const struct raid6_calls raid6_vpermxor2;
+extern const struct raid6_calls raid6_vpermxor4;
+extern const struct raid6_calls raid6_vpermxor8;
+extern const struct raid6_calls raid6_lsx;
+extern const struct raid6_calls raid6_lasx;
+extern const struct raid6_calls raid6_rvvx1;
+extern const struct raid6_calls raid6_rvvx2;
+extern const struct raid6_calls raid6_rvvx4;
+extern const struct raid6_calls raid6_rvvx8;
+
+struct raid6_recov_calls {
+ const char *name;
+ void (*data2)(int disks, size_t bytes, int faila, int failb,
+ void **ptrs);
+ void (*datap)(int disks, size_t bytes, int faila, void **ptrs);
+ int (*valid)(void);
+ int priority;
+};
+
+extern const struct raid6_recov_calls raid6_recov_intx1;
+extern const struct raid6_recov_calls raid6_recov_ssse3;
+extern const struct raid6_recov_calls raid6_recov_avx2;
+extern const struct raid6_recov_calls raid6_recov_avx512;
+extern const struct raid6_recov_calls raid6_recov_s390xc;
+extern const struct raid6_recov_calls raid6_recov_neon;
+extern const struct raid6_recov_calls raid6_recov_lsx;
+extern const struct raid6_recov_calls raid6_recov_lasx;
+extern const struct raid6_recov_calls raid6_recov_rvv;
+
+extern const struct raid6_calls raid6_neonx1;
+extern const struct raid6_calls raid6_neonx2;
+extern const struct raid6_calls raid6_neonx4;
+extern const struct raid6_calls raid6_neonx8;
+
+/* Algorithm list */
+extern const struct raid6_calls * const raid6_algos[];
+extern const struct raid6_recov_calls *const raid6_recov_algos[];
+
+#endif /* _PQ_IMPL_H */
diff --git a/lib/raid/raid6/arm/neon.c b/lib/raid/raid6/arm/neon.c
index c21da59ab48f..bd4ec4c86ee8 100644
--- a/lib/raid/raid6/arm/neon.c
+++ b/lib/raid/raid6/arm/neon.c
@@ -5,8 +5,8 @@
* Copyright (C) 2013 Linaro Ltd <ard.biesheuvel@linaro.org>
*/
-#include <linux/raid/pq.h>
#include <asm/simd.h>
+#include "algos.h"
/*
* There are 2 reasons these wrappers are kept in a separate compilation unit
diff --git a/lib/raid/raid6/arm/recov_neon.c b/lib/raid/raid6/arm/recov_neon.c
index 4eb0efb44750..e1d1d19fc9a8 100644
--- a/lib/raid/raid6/arm/recov_neon.c
+++ b/lib/raid/raid6/arm/recov_neon.c
@@ -4,8 +4,10 @@
* Copyright (C) 2017 Linaro Ltd. <ard.biesheuvel@linaro.org>
*/
+#include <linux/mm.h>
#include <linux/raid/pq.h>
#include <asm/simd.h>
+#include "algos.h"
#include "arm/neon.h"
static int raid6_has_neon(void)
diff --git a/lib/raid/raid6/int.uc b/lib/raid/raid6/int.uc
index 4f5f2869e21e..e63bd5a9c2ed 100644
--- a/lib/raid/raid6/int.uc
+++ b/lib/raid/raid6/int.uc
@@ -18,7 +18,7 @@
* This file is postprocessed using unroll.awk
*/
-#include <linux/raid/pq.h>
+#include "algos.h"
/*
* This is the C data type to use
diff --git a/lib/raid/raid6/loongarch/loongarch_simd.c b/lib/raid/raid6/loongarch/loongarch_simd.c
index 1b4cd1512d05..f77d11ce676e 100644
--- a/lib/raid/raid6/loongarch/loongarch_simd.c
+++ b/lib/raid/raid6/loongarch/loongarch_simd.c
@@ -9,9 +9,9 @@
* Copyright 2002-2004 H. Peter Anvin
*/
-#include <linux/raid/pq.h>
#include <asm/cpu-features.h>
#include <asm/fpu.h>
+#include "algos.h"
/*
* The vector algorithms are currently priority 0, which means the generic
diff --git a/lib/raid/raid6/loongarch/recov_loongarch_simd.c b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
index 7d4d349322b3..0bbdc8b5c2e7 100644
--- a/lib/raid/raid6/loongarch/recov_loongarch_simd.c
+++ b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
@@ -10,9 +10,11 @@
* Author: Jim Kukunas <james.t.kukunas@linux.intel.com>
*/
+#include <linux/mm.h>
#include <linux/raid/pq.h>
#include <asm/cpu-features.h>
#include <asm/fpu.h>
+#include "algos.h"
/*
* Unlike with the syndrome calculation algorithms, there's no boot-time
diff --git a/lib/raid/raid6/mktables.c b/lib/raid/raid6/mktables.c
index 3de1dbf6846c..97a17493bbd8 100644
--- a/lib/raid/raid6/mktables.c
+++ b/lib/raid/raid6/mktables.c
@@ -57,7 +57,7 @@ int main(int argc, char *argv[])
uint8_t exptbl[256], invtbl[256];
printf("#include <linux/export.h>\n");
- printf("#include <linux/raid/pq.h>\n");
+ printf("#include \"algos.h\"\n");
/* Compute multiplication table */
printf("\nconst u8 __attribute__((aligned(256)))\n"
diff --git a/lib/raid/raid6/powerpc/altivec.uc b/lib/raid/raid6/powerpc/altivec.uc
index 084ead768ddb..eb4a448cc88e 100644
--- a/lib/raid/raid6/powerpc/altivec.uc
+++ b/lib/raid/raid6/powerpc/altivec.uc
@@ -22,7 +22,7 @@
* bracked this with preempt_disable/enable or in a lock)
*/
-#include <linux/raid/pq.h>
+#include "algos.h"
#include <altivec.h>
#include <asm/cputable.h>
diff --git a/lib/raid/raid6/powerpc/vpermxor.uc b/lib/raid/raid6/powerpc/vpermxor.uc
index bb2c3a316ae8..ec61f30bec11 100644
--- a/lib/raid/raid6/powerpc/vpermxor.uc
+++ b/lib/raid/raid6/powerpc/vpermxor.uc
@@ -20,11 +20,11 @@
* This instruction was introduced in POWER8 - ISA v2.07.
*/
-#include <linux/raid/pq.h>
#include <altivec.h>
#include <asm/ppc-opcode.h>
#include <asm/cputable.h>
#include <asm/switch_to.h>
+#include "algos.h"
typedef vector unsigned char unative_t;
#define NSIZE sizeof(unative_t)
diff --git a/lib/raid/raid6/recov.c b/lib/raid/raid6/recov.c
index cc7e4dc1eaa6..735ab4013771 100644
--- a/lib/raid/raid6/recov.c
+++ b/lib/raid/raid6/recov.c
@@ -13,7 +13,9 @@
* the syndrome.)
*/
+#include <linux/mm.h>
#include <linux/raid/pq.h>
+#include "algos.h"
/* Recover two failed data blocks. */
static void raid6_2data_recov_intx1(int disks, size_t bytes, int faila,
diff --git a/lib/raid/raid6/riscv/recov_rvv.c b/lib/raid/raid6/riscv/recov_rvv.c
index 3ff39826e33f..02120d245e22 100644
--- a/lib/raid/raid6/riscv/recov_rvv.c
+++ b/lib/raid/raid6/riscv/recov_rvv.c
@@ -4,7 +4,9 @@
* Author: Chunyan Zhang <zhangchunyan@iscas.ac.cn>
*/
+#include <linux/mm.h>
#include <linux/raid/pq.h>
+#include "algos.h"
#include "rvv.h"
static void __raid6_2data_recov_rvv(int bytes, u8 *p, u8 *q, u8 *dp,
diff --git a/lib/raid/raid6/riscv/rvv.h b/lib/raid/raid6/riscv/rvv.h
index 0d430a4c5f08..c293130d798b 100644
--- a/lib/raid/raid6/riscv/rvv.h
+++ b/lib/raid/raid6/riscv/rvv.h
@@ -7,8 +7,8 @@
* Definitions for RISC-V RAID-6 code
*/
-#include <linux/raid/pq.h>
#include <asm/vector.h>
+#include "algos.h"
static int rvv_has_vector(void)
{
diff --git a/lib/raid/raid6/s390/recov_s390xc.c b/lib/raid/raid6/s390/recov_s390xc.c
index 2bc4c85174de..e7b3409f21e2 100644
--- a/lib/raid/raid6/s390/recov_s390xc.c
+++ b/lib/raid/raid6/s390/recov_s390xc.c
@@ -6,7 +6,9 @@
* Author(s): Martin Schwidefsky <schwidefsky@de.ibm.com>
*/
+#include <linux/mm.h>
#include <linux/raid/pq.h>
+#include "algos.h"
static inline void xor_block(u8 *p1, u8 *p2)
{
diff --git a/lib/raid/raid6/s390/s390vx.uc b/lib/raid/raid6/s390/s390vx.uc
index 97c5d5d9dcf9..aba3515eacac 100644
--- a/lib/raid/raid6/s390/s390vx.uc
+++ b/lib/raid/raid6/s390/s390vx.uc
@@ -12,8 +12,8 @@
*/
#include <linux/cpufeature.h>
-#include <linux/raid/pq.h>
#include <asm/fpu.h>
+#include "algos.h"
#define NSIZE 16
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index ab4fda17395b..9b71f22fa19a 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -7,7 +7,7 @@
#include <kunit/test.h>
#include <linux/prandom.h>
-#include <linux/raid/pq.h>
+#include "../algos.h"
MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING");
diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
index aab8b624c635..0bf831799082 100644
--- a/lib/raid/raid6/x86/avx2.c
+++ b/lib/raid/raid6/x86/avx2.c
@@ -13,8 +13,9 @@
*
*/
-#include <linux/raid/pq.h>
+#include <asm/cpufeature.h>
#include <asm/fpu/api.h>
+#include "algos.h"
static const struct raid6_avx2_constants {
u64 x1d[4];
diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index 47636b16632f..98ed42fb0a46 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -17,8 +17,9 @@
*
*/
-#include <linux/raid/pq.h>
+#include <asm/cpufeature.h>
#include <asm/fpu/api.h>
+#include "algos.h"
static const struct raid6_avx512_constants {
u64 x1d[8];
diff --git a/lib/raid/raid6/x86/mmx.c b/lib/raid/raid6/x86/mmx.c
index 22b9fdaa705f..052d9f010bfe 100644
--- a/lib/raid/raid6/x86/mmx.c
+++ b/lib/raid/raid6/x86/mmx.c
@@ -11,8 +11,9 @@
* MMX implementation of RAID-6 syndrome functions
*/
-#include <linux/raid/pq.h>
+#include <asm/cpufeature.h>
#include <asm/fpu/api.h>
+#include "algos.h"
/* Shared with raid6/sse1.c */
const struct raid6_mmx_constants {
diff --git a/lib/raid/raid6/x86/recov_avx2.c b/lib/raid/raid6/x86/recov_avx2.c
index bef82a38d8eb..06c6e05763bc 100644
--- a/lib/raid/raid6/x86/recov_avx2.c
+++ b/lib/raid/raid6/x86/recov_avx2.c
@@ -4,8 +4,10 @@
* Author: Jim Kukunas <james.t.kukunas@linux.intel.com>
*/
+#include <linux/mm.h>
#include <linux/raid/pq.h>
#include <asm/fpu/api.h>
+#include "algos.h"
static int raid6_has_avx2(void)
{
diff --git a/lib/raid/raid6/x86/recov_avx512.c b/lib/raid/raid6/x86/recov_avx512.c
index 06c70e771eaa..850bb962b514 100644
--- a/lib/raid/raid6/x86/recov_avx512.c
+++ b/lib/raid/raid6/x86/recov_avx512.c
@@ -6,8 +6,10 @@
* Author: Megha Dey <megha.dey@linux.intel.com>
*/
+#include <linux/mm.h>
#include <linux/raid/pq.h>
#include <asm/fpu/api.h>
+#include "algos.h"
static int raid6_has_avx512(void)
{
diff --git a/lib/raid/raid6/x86/recov_ssse3.c b/lib/raid/raid6/x86/recov_ssse3.c
index 5ca7d56f23d8..95589c33003a 100644
--- a/lib/raid/raid6/x86/recov_ssse3.c
+++ b/lib/raid/raid6/x86/recov_ssse3.c
@@ -3,8 +3,10 @@
* Copyright (C) 2012 Intel Corporation
*/
+#include <linux/mm.h>
#include <linux/raid/pq.h>
#include <asm/fpu/api.h>
+#include "algos.h"
static int raid6_has_ssse3(void)
{
diff --git a/lib/raid/raid6/x86/sse1.c b/lib/raid/raid6/x86/sse1.c
index fad214a430d8..7004255a0bb1 100644
--- a/lib/raid/raid6/x86/sse1.c
+++ b/lib/raid/raid6/x86/sse1.c
@@ -16,8 +16,9 @@
* worthwhile as a separate implementation.
*/
-#include <linux/raid/pq.h>
+#include <asm/cpufeature.h>
#include <asm/fpu/api.h>
+#include "algos.h"
/* Defined in raid6/mmx.c */
extern const struct raid6_mmx_constants {
diff --git a/lib/raid/raid6/x86/sse2.c b/lib/raid/raid6/x86/sse2.c
index 1b28e858a1d4..f30be4ee14d0 100644
--- a/lib/raid/raid6/x86/sse2.c
+++ b/lib/raid/raid6/x86/sse2.c
@@ -12,8 +12,9 @@
*
*/
-#include <linux/raid/pq.h>
+#include <asm/cpufeature.h>
#include <asm/fpu/api.h>
+#include "algos.h"
static const struct raid6_sse_constants {
u64 x1d[2];
--
2.53.0
^ permalink raw reply related
* [PATCH 08/19] raid6: improve the public interface
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Stop directly calling into function pointers from users of the RAID6 PQ
API, and provide exported functions with proper documentation and
API guarantees asserts where applicable instead.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
Documentation/crypto/async-tx-api.rst | 4 +-
crypto/async_tx/async_pq.c | 6 +-
crypto/async_tx/async_raid6_recov.c | 4 +-
drivers/md/raid5.c | 4 +-
fs/btrfs/raid56.c | 8 +-
include/linux/raid/pq.h | 28 ++--
lib/raid/raid6/algos.c | 139 +++++++++++++++++-
lib/raid/raid6/arm/recov_neon.c | 4 +-
.../raid6/loongarch/recov_loongarch_simd.c | 8 +-
lib/raid/raid6/recov.c | 4 +-
lib/raid/raid6/riscv/recov_rvv.c | 4 +-
lib/raid/raid6/s390/recov_s390xc.c | 4 +-
lib/raid/raid6/x86/recov_avx2.c | 4 +-
lib/raid/raid6/x86/recov_avx512.c | 4 +-
lib/raid/raid6/x86/recov_ssse3.c | 4 +-
15 files changed, 181 insertions(+), 48 deletions(-)
diff --git a/Documentation/crypto/async-tx-api.rst b/Documentation/crypto/async-tx-api.rst
index f88a7809385e..49fcfc66314a 100644
--- a/Documentation/crypto/async-tx-api.rst
+++ b/Documentation/crypto/async-tx-api.rst
@@ -82,9 +82,9 @@ xor_val xor a series of source buffers and set a flag if the
pq generate the p+q (raid6 syndrome) from a series of source buffers
pq_val validate that a p and or q buffer are in sync with a given series of
sources
-datap (raid6_datap_recov) recover a raid6 data block and the p block
+datap (raid6_recov_datap) recover a raid6 data block and the p block
from the given sources
-2data (raid6_2data_recov) recover 2 raid6 data blocks from the given
+2data (raid6_recov_2data) recover 2 raid6 data blocks from the given
sources
======== ====================================================================
diff --git a/crypto/async_tx/async_pq.c b/crypto/async_tx/async_pq.c
index 0ce6f07b4e0d..f3574f80d1df 100644
--- a/crypto/async_tx/async_pq.c
+++ b/crypto/async_tx/async_pq.c
@@ -131,11 +131,11 @@ do_sync_gen_syndrome(struct page **blocks, unsigned int *offsets, int disks,
}
}
if (submit->flags & ASYNC_TX_PQ_XOR_DST) {
- BUG_ON(!raid6_call.xor_syndrome);
+ BUG_ON(!raid6_can_xor_syndrome());
if (start >= 0)
- raid6_call.xor_syndrome(disks, start, stop, len, srcs);
+ raid6_xor_syndrome(disks, start, stop, len, srcs);
} else
- raid6_call.gen_syndrome(disks, len, srcs);
+ raid6_gen_syndrome(disks, len, srcs);
async_tx_sync_epilog(submit);
}
diff --git a/crypto/async_tx/async_raid6_recov.c b/crypto/async_tx/async_raid6_recov.c
index f2dc6af6e6a7..305ea1421a3e 100644
--- a/crypto/async_tx/async_raid6_recov.c
+++ b/crypto/async_tx/async_raid6_recov.c
@@ -418,7 +418,7 @@ async_raid6_2data_recov(int disks, size_t bytes, int faila, int failb,
else
ptrs[i] = page_address(blocks[i]) + offs[i];
- raid6_2data_recov(disks, bytes, faila, failb, ptrs);
+ raid6_recov_2data(disks, bytes, faila, failb, ptrs);
async_tx_sync_epilog(submit);
@@ -501,7 +501,7 @@ async_raid6_datap_recov(int disks, size_t bytes, int faila,
else
ptrs[i] = page_address(blocks[i]) + offs[i];
- raid6_datap_recov(disks, bytes, faila, ptrs);
+ raid6_recov_datap(disks, bytes, faila, ptrs);
async_tx_sync_epilog(submit);
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 0d76e82f4506..ebcb19317670 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -6955,7 +6955,7 @@ raid5_store_rmw_level(struct mddev *mddev, const char *page, size_t len)
if (kstrtoul(page, 10, &new))
return -EINVAL;
- if (new != PARITY_DISABLE_RMW && !raid6_call.xor_syndrome)
+ if (new != PARITY_DISABLE_RMW && !raid6_can_xor_syndrome())
return -EINVAL;
if (new != PARITY_DISABLE_RMW &&
@@ -7646,7 +7646,7 @@ static struct r5conf *setup_conf(struct mddev *mddev)
conf->level = mddev->new_level;
if (conf->level == 6) {
conf->max_degraded = 2;
- if (raid6_call.xor_syndrome)
+ if (raid6_can_xor_syndrome())
conf->rmw_level = PARITY_ENABLE_RMW;
else
conf->rmw_level = PARITY_DISABLE_RMW;
diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c
index 08ee8f316d96..dabc9522e881 100644
--- a/fs/btrfs/raid56.c
+++ b/fs/btrfs/raid56.c
@@ -1410,7 +1410,7 @@ static void generate_pq_vertical_step(struct btrfs_raid_bio *rbio, unsigned int
rbio_qstripe_paddr(rbio, sector_nr, step_nr));
assert_rbio(rbio);
- raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
+ raid6_gen_syndrome(rbio->real_stripes, step, pointers);
} else {
/* raid5 */
memcpy(pointers[rbio->nr_data], pointers[0], step);
@@ -1987,10 +1987,10 @@ static void recover_vertical_step(struct btrfs_raid_bio *rbio,
}
if (failb == rbio->real_stripes - 2) {
- raid6_datap_recov(rbio->real_stripes, step,
+ raid6_recov_datap(rbio->real_stripes, step,
faila, pointers);
} else {
- raid6_2data_recov(rbio->real_stripes, step,
+ raid6_recov_2data(rbio->real_stripes, step,
faila, failb, pointers);
}
} else {
@@ -2644,7 +2644,7 @@ static bool verify_one_parity_step(struct btrfs_raid_bio *rbio,
if (has_qstripe) {
assert_rbio(rbio);
/* RAID6, call the library function to fill in our P/Q. */
- raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
+ raid6_gen_syndrome(rbio->real_stripes, step, pointers);
} else {
/* RAID5. */
memcpy(pointers[nr_data], pointers[0], step);
diff --git a/include/linux/raid/pq.h b/include/linux/raid/pq.h
index f27a866c287f..662c2669f63f 100644
--- a/include/linux/raid/pq.h
+++ b/include/linux/raid/pq.h
@@ -11,6 +11,25 @@
#include <linux/blkdev.h>
#include <linux/mm.h>
+/*
+ * While the RAID6 algorithm could in theory support 3 devices by just copying
+ * the data disk to the two parity disks, this configuration is not only useless
+ * because it is a suboptimal version of 3-way mirroring, but also easy to get
+ * wrong in architecture-optimized implementations due to special casing, so
+ * don't support it.
+ */
+#define RAID6_MIN_DISKS 4
+
+void raid6_gen_syndrome(int disks, size_t bytes, void **ptrs);
+void raid6_xor_syndrome(int disks, int start, int stop, size_t bytes,
+ void **ptrs);
+bool raid6_can_xor_syndrome(void);
+
+void raid6_recov_2data(int disks, size_t bytes, int faila, int failb,
+ void **ptrs);
+void raid6_recov_datap(int disks, size_t bytes, int faila,
+ void **ptrs);
+
/* Routine choices */
struct raid6_calls {
void (*gen_syndrome)(int, size_t, void **);
@@ -20,9 +39,6 @@ struct raid6_calls {
int priority; /* Relative priority ranking if non-zero */
};
-/* Selected algorithm */
-extern struct raid6_calls raid6_call;
-
/* Various routine sets */
extern const struct raid6_calls raid6_intx1;
extern const struct raid6_calls raid6_intx2;
@@ -92,10 +108,4 @@ extern const u8 raid6_gflog[256] __attribute__((aligned(256)));
extern const u8 raid6_gfinv[256] __attribute__((aligned(256)));
extern const u8 raid6_gfexi[256] __attribute__((aligned(256)));
-/* Recovery routines */
-extern void (*raid6_2data_recov)(int disks, size_t bytes, int faila, int failb,
- void **ptrs);
-extern void (*raid6_datap_recov)(int disks, size_t bytes, int faila,
- void **ptrs);
-
#endif /* LINUX_RAID_RAID6_H */
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index 985c60bb00a4..683b97cb94ad 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -16,8 +16,85 @@
#include <linux/gfp.h>
#include <kunit/visibility.h>
-struct raid6_calls raid6_call;
-EXPORT_SYMBOL_GPL(raid6_call);
+static const struct raid6_recov_calls *raid6_recov_algo;
+
+/* Selected algorithm */
+static struct raid6_calls raid6_call;
+
+/**
+ * raid6_gen_syndrome - generate RAID6 P/Q parity
+ * @disks: number of "disks" to operate on including parity
+ * @bytes: length in bytes of each vector
+ * @ptrs: @disks size array of memory pointers
+ *
+ * Generate @bytes worth of RAID6 P and Q parity in @ptrs[@disks - 2] and
+ * @ptrs[@disks - 1] respectively from the memory pointed to by @ptrs[0] to
+ * @ptrs[@disks - 3].
+ *
+ * @disks must be at least 3, and the memory pointed to by each member of @ptrs
+ * must be at least 64-byte aligned. @bytes must be non-zero and a multiple of
+ * 512.
+ *
+ * See https://kernel.org/pub/linux/kernel/people/hpa/raid6.pdf for underlying
+ * algorithm.
+ */
+void raid6_gen_syndrome(int disks, size_t bytes, void **ptrs)
+{
+ WARN_ON_ONCE(!in_task() || irqs_disabled() || softirq_count());
+ WARN_ON_ONCE(bytes & 511);
+ WARN_ON_ONCE(disks < RAID6_MIN_DISKS);
+
+ raid6_call.gen_syndrome(disks, bytes, ptrs);
+}
+EXPORT_SYMBOL_GPL(raid6_gen_syndrome);
+
+/**
+ * raid6_xor_syndrome - update RAID6 P/Q parity
+ * @disks: number of "disks" to operate on including parity
+ * @start: first index into @disk to update
+ * @stop: last index into @disk to update
+ * @bytes: length in bytes of each vector
+ * @ptrs: @disks size array of memory pointers
+ *
+ * Update @bytes worth of RAID6 P and Q parity in @ptrs[@disks - 2] and
+ * @ptrs[@disks - 1] respectively for the memory pointed to by
+ * @ptrs[@start..@stop].
+ *
+ * This is used to update parity in place using the following sequence:
+ *
+ * 1) call raid6_xor_syndrome(disk, start, stop, ...) for the existing data.
+ * 2) update the the data in @ptrs[@start..@stop].
+ * 3) call raid6_xor_syndrome(disk, start, stop, ...) for the new data.
+ *
+ * Data between @start and @stop that is not changed should be filled
+ * with a pointer to the kernel zero page.
+ *
+ * @disks must be at least 3, and the memory pointed to by each member of @ptrs
+ * must be at least 64-byte aligned. @bytes must be non-zero and a multiple of
+ * 512. @stop must be larger or equal to @start.
+ */
+void raid6_xor_syndrome(int disks, int start, int stop, size_t bytes,
+ void **ptrs)
+{
+ WARN_ON_ONCE(!in_task() || irqs_disabled() || softirq_count());
+ WARN_ON_ONCE(bytes & 511);
+ WARN_ON_ONCE(disks < RAID6_MIN_DISKS);
+ WARN_ON_ONCE(stop < start);
+
+ raid6_call.xor_syndrome(disks, start, stop, bytes, ptrs);
+}
+EXPORT_SYMBOL_GPL(raid6_xor_syndrome);
+
+/*
+ * raid6_can_xor_syndrome - check if raid6_xor_syndrome() can be used
+ *
+ * Returns %true if raid6_can_xor_syndrome() can be used, else %false.
+ */
+bool raid6_can_xor_syndrome(void)
+{
+ return !!raid6_call.xor_syndrome;
+}
+EXPORT_SYMBOL_GPL(raid6_can_xor_syndrome);
const struct raid6_calls * const raid6_algos[] = {
#if defined(__i386__) && !defined(__arch_um__)
@@ -84,11 +161,58 @@ const struct raid6_calls * const raid6_algos[] = {
};
EXPORT_SYMBOL_IF_KUNIT(raid6_algos);
-void (*raid6_2data_recov)(int, size_t, int, int, void **);
-EXPORT_SYMBOL_GPL(raid6_2data_recov);
+/**
+ * raid6_recov_2data - recover two missing data disks
+ * @disks: number of "disks" to operate on including parity
+ * @bytes: length in bytes of each vector
+ * @faila: first failed data disk index
+ * @failb: second failed data disk index
+ * @ptrs: @disks size array of memory pointers
+ *
+ * Rebuild @bytes of missing data in @ptrs[@faila] and @ptrs[@failb] from the
+ * data in the remaining disks and the two parities pointed to by the other
+ * indices between 0 and @disks - 1 in @ptrs. @disks includes the data disks
+ * and the two parities. @faila must be smaller than @failb.
+ *
+ * Memory pointed to by each pointer in @ptrs must be page aligned and is
+ * limited to %PAGE_SIZE.
+ */
+void raid6_recov_2data(int disks, size_t bytes, int faila, int failb,
+ void **ptrs)
+{
+ WARN_ON_ONCE(!in_task() || irqs_disabled() || softirq_count());
+ WARN_ON_ONCE(bytes & 511);
+ WARN_ON_ONCE(bytes > PAGE_SIZE);
+ WARN_ON_ONCE(failb <= faila);
+
+ raid6_recov_algo->data2(disks, bytes, faila, failb, ptrs);
+}
+EXPORT_SYMBOL_GPL(raid6_recov_2data);
+
+/**
+ * raid6_recov_datap - recover a missing data disk and missing P-parity
+ * @disks: number of "disks" to operate on including parity
+ * @bytes: length in bytes of each vector
+ * @faila: failed data disk index
+ * @ptrs: @disks size array of memory pointers
+ *
+ * Rebuild @bytes of missing data in @ptrs[@faila] and the missing P-parity in
+ * @ptrs[@disks - 2] from the data in the remaining disks and the Q-parity
+ * pointed to by the other indices between 0 and @disks - 1 in @ptrs. @disks
+ * includes the data disks and the two parities.
+ *
+ * Memory pointed to by each pointer in @ptrs must be page aligned and is
+ * limited to %PAGE_SIZE.
+ */
+void raid6_recov_datap(int disks, size_t bytes, int faila, void **ptrs)
+{
+ WARN_ON_ONCE(!in_task() || irqs_disabled() || softirq_count());
+ WARN_ON_ONCE(bytes & 511);
+ WARN_ON_ONCE(bytes > PAGE_SIZE);
-void (*raid6_datap_recov)(int, size_t, int, void **);
-EXPORT_SYMBOL_GPL(raid6_datap_recov);
+ raid6_recov_algo->datap(disks, bytes, faila, ptrs);
+}
+EXPORT_SYMBOL_GPL(raid6_recov_datap);
const struct raid6_recov_calls *const raid6_recov_algos[] = {
#ifdef CONFIG_X86
@@ -133,8 +257,7 @@ static inline const struct raid6_recov_calls *raid6_choose_recov(void)
best = *algo;
if (best) {
- raid6_2data_recov = best->data2;
- raid6_datap_recov = best->datap;
+ raid6_recov_algo = best;
pr_info("raid6: using %s recovery algorithm\n", best->name);
} else
diff --git a/lib/raid/raid6/arm/recov_neon.c b/lib/raid/raid6/arm/recov_neon.c
index 9993bda5d3a6..4eb0efb44750 100644
--- a/lib/raid/raid6/arm/recov_neon.c
+++ b/lib/raid/raid6/arm/recov_neon.c
@@ -35,7 +35,7 @@ static void raid6_2data_recov_neon(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -69,7 +69,7 @@ static void raid6_datap_recov_neon(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/loongarch/recov_loongarch_simd.c b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
index 4d4563209647..7d4d349322b3 100644
--- a/lib/raid/raid6/loongarch/recov_loongarch_simd.c
+++ b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
@@ -49,7 +49,7 @@ static void raid6_2data_recov_lsx(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -201,7 +201,7 @@ static void raid6_datap_recov_lsx(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
@@ -323,7 +323,7 @@ static void raid6_2data_recov_lasx(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -440,7 +440,7 @@ static void raid6_datap_recov_lasx(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/recov.c b/lib/raid/raid6/recov.c
index 211e1df28963..cc7e4dc1eaa6 100644
--- a/lib/raid/raid6/recov.c
+++ b/lib/raid/raid6/recov.c
@@ -37,7 +37,7 @@ static void raid6_2data_recov_intx1(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -75,7 +75,7 @@ static void raid6_datap_recov_intx1(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/riscv/recov_rvv.c b/lib/raid/raid6/riscv/recov_rvv.c
index f77d9c430687..3ff39826e33f 100644
--- a/lib/raid/raid6/riscv/recov_rvv.c
+++ b/lib/raid/raid6/riscv/recov_rvv.c
@@ -164,7 +164,7 @@ static void raid6_2data_recov_rvv(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -199,7 +199,7 @@ static void raid6_datap_recov_rvv(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/s390/recov_s390xc.c b/lib/raid/raid6/s390/recov_s390xc.c
index 0f32217b7123..2bc4c85174de 100644
--- a/lib/raid/raid6/s390/recov_s390xc.c
+++ b/lib/raid/raid6/s390/recov_s390xc.c
@@ -40,7 +40,7 @@ static void raid6_2data_recov_s390xc(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -84,7 +84,7 @@ static void raid6_datap_recov_s390xc(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/x86/recov_avx2.c b/lib/raid/raid6/x86/recov_avx2.c
index 325310c81e1c..bef82a38d8eb 100644
--- a/lib/raid/raid6/x86/recov_avx2.c
+++ b/lib/raid/raid6/x86/recov_avx2.c
@@ -34,7 +34,7 @@ static void raid6_2data_recov_avx2(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -199,7 +199,7 @@ static void raid6_datap_recov_avx2(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/x86/recov_avx512.c b/lib/raid/raid6/x86/recov_avx512.c
index 08de77fcb8bd..06c70e771eaa 100644
--- a/lib/raid/raid6/x86/recov_avx512.c
+++ b/lib/raid/raid6/x86/recov_avx512.c
@@ -43,7 +43,7 @@ static void raid6_2data_recov_avx512(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -241,7 +241,7 @@ static void raid6_datap_recov_avx512(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/x86/recov_ssse3.c b/lib/raid/raid6/x86/recov_ssse3.c
index 002bef1e0847..5ca7d56f23d8 100644
--- a/lib/raid/raid6/x86/recov_ssse3.c
+++ b/lib/raid/raid6/x86/recov_ssse3.c
@@ -36,7 +36,7 @@ static void raid6_2data_recov_ssse3(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -206,7 +206,7 @@ static void raid6_datap_recov_ssse3(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
--
2.53.0
^ permalink raw reply related
* [PATCH 10/19] raid6: rework the init helpers
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Match the xor version with two initcalls for the built-in case to delay
calibrartion. This prepares for adding non-calibration init code.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/Kconfig | 11 +
lib/raid/raid6/Makefile | 4 +
lib/raid/raid6/algos.c | 309 ++++++++----------
lib/raid/raid6/algos.h | 67 +---
lib/raid/raid6/arm/neon.c | 6 -
lib/raid/raid6/arm/pq_arch.h | 24 ++
lib/raid/raid6/arm/recov_neon.c | 7 -
lib/raid/raid6/arm64/pq_arch.h | 1 +
lib/raid/raid6/loongarch/loongarch_simd.c | 12 -
lib/raid/raid6/loongarch/pq_arch.h | 23 ++
.../raid6/loongarch/recov_loongarch_simd.c | 14 -
lib/raid/raid6/powerpc/altivec.uc | 10 -
lib/raid/raid6/powerpc/pq_arch.h | 31 ++
lib/raid/raid6/powerpc/vpermxor.uc | 11 -
lib/raid/raid6/recov.c | 2 -
lib/raid/raid6/riscv/pq_arch.h | 21 ++
lib/raid/raid6/riscv/recov_rvv.c | 2 -
lib/raid/raid6/riscv/rvv.h | 6 -
lib/raid/raid6/s390/pq_arch.h | 15 +
lib/raid/raid6/s390/recov_s390xc.c | 2 -
lib/raid/raid6/s390/s390vx.uc | 7 -
lib/raid/raid6/tests/raid6_kunit.c | 23 +-
lib/raid/raid6/x86/avx2.c | 14 -
lib/raid/raid6/x86/avx512.c | 19 --
lib/raid/raid6/x86/mmx.c | 8 -
lib/raid/raid6/x86/pq_arch.h | 97 ++++++
lib/raid/raid6/x86/recov_avx2.c | 8 -
lib/raid/raid6/x86/recov_avx512.c | 12 -
lib/raid/raid6/x86/recov_ssse3.c | 9 -
lib/raid/raid6/x86/sse1.c | 12 -
lib/raid/raid6/x86/sse2.c | 15 -
31 files changed, 393 insertions(+), 409 deletions(-)
create mode 100644 lib/raid/raid6/arm/pq_arch.h
create mode 100644 lib/raid/raid6/arm64/pq_arch.h
create mode 100644 lib/raid/raid6/loongarch/pq_arch.h
create mode 100644 lib/raid/raid6/powerpc/pq_arch.h
create mode 100644 lib/raid/raid6/riscv/pq_arch.h
create mode 100644 lib/raid/raid6/s390/pq_arch.h
create mode 100644 lib/raid/raid6/x86/pq_arch.h
diff --git a/lib/raid/Kconfig b/lib/raid/Kconfig
index e39f6d667792..978cd6ba08ac 100644
--- a/lib/raid/Kconfig
+++ b/lib/raid/Kconfig
@@ -32,6 +32,17 @@ config XOR_KUNIT_TEST
config RAID6_PQ
tristate
+# selected by architectures that provide an optimized PQ implementation
+config RAID6_PQ_ARCH
+ depends on RAID6_PQ
+ default y if KERNEL_MODE_NEON # arm32/arm64
+ default y if LOONGARCH
+ default y if ALTIVEC # powerpc
+ default y if RISCV_ISA_V
+ default y if S390
+ default y if X86
+ bool
+
config RAID6_PQ_KUNIT_TEST
tristate "KUnit tests for RAID6 PQ functions" if !KUNIT_ALL_TESTS
depends on KUNIT
diff --git a/lib/raid/raid6/Makefile b/lib/raid/raid6/Makefile
index f64f6d32f28b..ed1f512b88af 100644
--- a/lib/raid/raid6/Makefile
+++ b/lib/raid/raid6/Makefile
@@ -4,6 +4,10 @@ hostprogs += mktables
ccflags-y += -I $(src)
+ifeq ($(CONFIG_RAID6_PQ_ARCH),y)
+CFLAGS_algos.o += -I$(src)/$(SRCARCH)
+endif
+
obj-$(CONFIG_RAID6_PQ) += raid6_pq.o tests/
raid6_pq-y += algos.o tables.o
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index af31a1feb6e7..5bd953032e55 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -17,6 +17,9 @@
#include <kunit/visibility.h>
#include "algos.h"
+#define RAID6_MAX_ALGOS 16
+static const struct raid6_calls *raid6_algos[RAID6_MAX_ALGOS];
+static unsigned int raid6_nr_algos;
static const struct raid6_recov_calls *raid6_recov_algo;
/* Selected algorithm */
@@ -97,71 +100,6 @@ bool raid6_can_xor_syndrome(void)
}
EXPORT_SYMBOL_GPL(raid6_can_xor_syndrome);
-const struct raid6_calls * const raid6_algos[] = {
-#if defined(__i386__) && !defined(__arch_um__)
- &raid6_avx512x2,
- &raid6_avx512x1,
- &raid6_avx2x2,
- &raid6_avx2x1,
- &raid6_sse2x2,
- &raid6_sse2x1,
- &raid6_sse1x2,
- &raid6_sse1x1,
- &raid6_mmxx2,
- &raid6_mmxx1,
-#endif
-#if defined(__x86_64__) && !defined(__arch_um__)
- &raid6_avx512x4,
- &raid6_avx512x2,
- &raid6_avx512x1,
- &raid6_avx2x4,
- &raid6_avx2x2,
- &raid6_avx2x1,
- &raid6_sse2x4,
- &raid6_sse2x2,
- &raid6_sse2x1,
-#endif
-#ifdef CONFIG_ALTIVEC
- &raid6_vpermxor8,
- &raid6_vpermxor4,
- &raid6_vpermxor2,
- &raid6_vpermxor1,
- &raid6_altivec8,
- &raid6_altivec4,
- &raid6_altivec2,
- &raid6_altivec1,
-#endif
-#if defined(CONFIG_S390)
- &raid6_s390vx8,
-#endif
-#ifdef CONFIG_KERNEL_MODE_NEON
- &raid6_neonx8,
- &raid6_neonx4,
- &raid6_neonx2,
- &raid6_neonx1,
-#endif
-#ifdef CONFIG_LOONGARCH
-#ifdef CONFIG_CPU_HAS_LASX
- &raid6_lasx,
-#endif
-#ifdef CONFIG_CPU_HAS_LSX
- &raid6_lsx,
-#endif
-#endif
-#ifdef CONFIG_RISCV_ISA_V
- &raid6_rvvx1,
- &raid6_rvvx2,
- &raid6_rvvx4,
- &raid6_rvvx8,
-#endif
- &raid6_intx8,
- &raid6_intx4,
- &raid6_intx2,
- &raid6_intx1,
- NULL
-};
-EXPORT_SYMBOL_IF_KUNIT(raid6_algos);
-
/**
* raid6_recov_2data - recover two missing data disks
* @disks: number of "disks" to operate on including parity
@@ -215,119 +153,57 @@ void raid6_recov_datap(int disks, size_t bytes, int faila, void **ptrs)
}
EXPORT_SYMBOL_GPL(raid6_recov_datap);
-const struct raid6_recov_calls *const raid6_recov_algos[] = {
-#ifdef CONFIG_X86
- &raid6_recov_avx512,
- &raid6_recov_avx2,
- &raid6_recov_ssse3,
-#endif
-#ifdef CONFIG_S390
- &raid6_recov_s390xc,
-#endif
-#if defined(CONFIG_KERNEL_MODE_NEON)
- &raid6_recov_neon,
-#endif
-#ifdef CONFIG_LOONGARCH
-#ifdef CONFIG_CPU_HAS_LASX
- &raid6_recov_lasx,
-#endif
-#ifdef CONFIG_CPU_HAS_LSX
- &raid6_recov_lsx,
-#endif
-#endif
-#ifdef CONFIG_RISCV_ISA_V
- &raid6_recov_rvv,
-#endif
- &raid6_recov_intx1,
- NULL
-};
-EXPORT_SYMBOL_IF_KUNIT(raid6_recov_algos);
-
#define RAID6_TIME_JIFFIES_LG2 4
#define RAID6_TEST_DISKS 8
#define RAID6_TEST_DISKS_ORDER 3
-static inline const struct raid6_recov_calls *raid6_choose_recov(void)
+static int raid6_choose_gen(void *(*const dptrs)[RAID6_TEST_DISKS],
+ const int disks)
{
- const struct raid6_recov_calls *const *algo;
- const struct raid6_recov_calls *best;
-
- for (best = NULL, algo = raid6_recov_algos; *algo; algo++)
- if (!best || (*algo)->priority > best->priority)
- if (!(*algo)->valid || (*algo)->valid())
- best = *algo;
+ /* work on the second half of the disks */
+ int start = (disks >> 1) - 1, stop = disks - 3;
+ const struct raid6_calls *best = NULL;
+ unsigned long bestgenperf = 0;
+ unsigned int i;
- if (best) {
- raid6_recov_algo = best;
+ for (i = 0; i < raid6_nr_algos; i++) {
+ const struct raid6_calls *algo = raid6_algos[i];
+ unsigned long perf = 0, j0, j1;
- pr_info("raid6: using %s recovery algorithm\n", best->name);
- } else
- pr_err("raid6: Yikes! No recovery algorithm found!\n");
-
- return best;
-}
+ preempt_disable();
+ j0 = jiffies;
+ while ((j1 = jiffies) == j0)
+ cpu_relax();
+ while (time_before(jiffies,
+ j1 + (1<<RAID6_TIME_JIFFIES_LG2))) {
+ algo->gen_syndrome(disks, PAGE_SIZE, *dptrs);
+ perf++;
+ }
+ preempt_enable();
-static inline const struct raid6_calls *raid6_choose_gen(
- void *(*const dptrs)[RAID6_TEST_DISKS], const int disks)
-{
- unsigned long perf, bestgenperf, j0, j1;
- int start = (disks>>1)-1, stop = disks-3; /* work on the second half of the disks */
- const struct raid6_calls *const *algo;
- const struct raid6_calls *best;
-
- for (bestgenperf = 0, best = NULL, algo = raid6_algos; *algo; algo++) {
- if (!best || (*algo)->priority >= best->priority) {
- if ((*algo)->valid && !(*algo)->valid())
- continue;
-
- if (!IS_ENABLED(CONFIG_RAID6_PQ_BENCHMARK)) {
- best = *algo;
- break;
- }
-
- perf = 0;
-
- preempt_disable();
- j0 = jiffies;
- while ((j1 = jiffies) == j0)
- cpu_relax();
- while (time_before(jiffies,
- j1 + (1<<RAID6_TIME_JIFFIES_LG2))) {
- (*algo)->gen_syndrome(disks, PAGE_SIZE, *dptrs);
- perf++;
- }
- preempt_enable();
-
- if (perf > bestgenperf) {
- bestgenperf = perf;
- best = *algo;
- }
- pr_info("raid6: %-8s gen() %5ld MB/s\n", (*algo)->name,
- (perf * HZ * (disks-2)) >>
- (20 - PAGE_SHIFT + RAID6_TIME_JIFFIES_LG2));
+ if (perf > bestgenperf) {
+ bestgenperf = perf;
+ best = algo;
}
+ pr_info("raid6: %-8s gen() %5ld MB/s\n", algo->name,
+ (perf * HZ * (disks-2)) >>
+ (20 - PAGE_SHIFT + RAID6_TIME_JIFFIES_LG2));
}
if (!best) {
pr_err("raid6: Yikes! No algorithm found!\n");
- goto out;
+ return -EINVAL;
}
raid6_call = *best;
- if (!IS_ENABLED(CONFIG_RAID6_PQ_BENCHMARK)) {
- pr_info("raid6: skipped pq benchmark and selected %s\n",
- best->name);
- goto out;
- }
-
pr_info("raid6: using algorithm %s gen() %ld MB/s\n",
best->name,
(bestgenperf * HZ * (disks - 2)) >>
(20 - PAGE_SHIFT + RAID6_TIME_JIFFIES_LG2));
if (best->xor_syndrome) {
- perf = 0;
+ unsigned long perf = 0, j0, j1;
preempt_disable();
j0 = jiffies;
@@ -346,8 +222,7 @@ static inline const struct raid6_calls *raid6_choose_gen(
(20 - PAGE_SHIFT + RAID6_TIME_JIFFIES_LG2 + 1));
}
-out:
- return best;
+ return 0;
}
@@ -357,12 +232,17 @@ static inline const struct raid6_calls *raid6_choose_gen(
static int __init raid6_select_algo(void)
{
const int disks = RAID6_TEST_DISKS;
-
- const struct raid6_calls *gen_best;
- const struct raid6_recov_calls *rec_best;
char *disk_ptr, *p;
void *dptrs[RAID6_TEST_DISKS];
int i, cycle;
+ int error;
+
+ if (!IS_ENABLED(CONFIG_RAID6_PQ_BENCHMARK) || raid6_nr_algos == 1) {
+ pr_info("raid6: skipped pq benchmark and selected %s\n",
+ raid6_algos[raid6_nr_algos - 1]->name);
+ raid6_call = *raid6_algos[raid6_nr_algos - 1];
+ return 0;
+ }
/* prepare the buffer and fill it circularly with gfmul table */
disk_ptr = (char *)__get_free_pages(GFP_KERNEL, RAID6_TEST_DISKS_ORDER);
@@ -385,22 +265,115 @@ static int __init raid6_select_algo(void)
memcpy(p, raid6_gfmul, (disks - 2) * PAGE_SIZE % 65536);
/* select raid gen_syndrome function */
- gen_best = raid6_choose_gen(&dptrs, disks);
-
- /* select raid recover functions */
- rec_best = raid6_choose_recov();
+ error = raid6_choose_gen(&dptrs, disks);
free_pages((unsigned long)disk_ptr, RAID6_TEST_DISKS_ORDER);
- return gen_best && rec_best ? 0 : -EINVAL;
+ return error;
+}
+
+void __init raid6_algo_add(const struct raid6_calls *algo)
+{
+ if (WARN_ON_ONCE(raid6_nr_algos == RAID6_MAX_ALGOS))
+ return;
+ raid6_algos[raid6_nr_algos++] = algo;
+}
+
+void __init raid6_algo_add_default(void)
+{
+ raid6_algo_add(&raid6_intx1);
+ raid6_algo_add(&raid6_intx2);
+ raid6_algo_add(&raid6_intx4);
+ raid6_algo_add(&raid6_intx8);
+}
+
+void __init raid6_recov_algo_add(const struct raid6_recov_calls *algo)
+{
+ if (WARN_ON_ONCE(raid6_recov_algo))
+ return;
+ raid6_recov_algo = algo;
+}
+
+#ifdef CONFIG_RAID6_PQ_ARCH
+#include "pq_arch.h"
+#else
+static inline void arch_raid6_init(void)
+{
+ raid6_algo_add_default();
+}
+#endif /* CONFIG_RAID6_PQ_ARCH */
+
+static int __init raid6_init(void)
+{
+ /*
+ * Architectures providing arch_raid6_init must add all PQ generation
+ * algorithms they want to consider in arch_raid6_init(), including
+ * the generic ones using raid6_algo_add_default() if wanted.
+ */
+ arch_raid6_init();
+
+ /*
+ * Architectures don't have to set a recovery algorithm, we'll just pick
+ * the generic integer one if none was set.
+ */
+ if (!raid6_recov_algo)
+ raid6_recov_algo = &raid6_recov_intx1;
+ pr_info("raid6: using %s recovery algorithm\n", raid6_recov_algo->name);
+
+#ifdef MODULE
+ return raid6_select_algo();
+#else
+ return 0;
+#endif
}
-static void raid6_exit(void)
+static void __exit raid6_exit(void)
{
- do { } while (0);
}
-subsys_initcall(raid6_select_algo);
+/*
+ * When built-in we must register the default template before md, but we don't
+ * want calibration to run that early as that would delay the boot process.
+ */
+#ifndef MODULE
+__initcall(raid6_select_algo);
+#endif
+core_initcall(raid6_init);
module_exit(raid6_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("RAID6 Q-syndrome calculations");
+
+#if IS_ENABLED(CONFIG_RAID6_PQ_KUNIT_TEST)
+const struct raid6_calls *raid6_algo_find(unsigned int idx)
+{
+ if (idx >= raid6_nr_algos) {
+ /*
+ * Always include the simplest generic integer implementation in
+ * the unit tests as a baseline.
+ */
+ if (idx == raid6_nr_algos &&
+ raid6_algos[0] != &raid6_intx1)
+ return &raid6_intx1;
+ return NULL;
+ }
+ return raid6_algos[idx];
+}
+EXPORT_SYMBOL_IF_KUNIT(raid6_algo_find);
+
+const struct raid6_recov_calls *raid6_recov_algo_find(unsigned int idx)
+{
+ switch (idx) {
+ case 0:
+ /* always test the generic integer implementation */
+ return &raid6_recov_intx1;
+ case 1:
+ /* test the optimized implementation if there is one */
+ if (raid6_recov_algo != &raid6_recov_intx1)
+ return raid6_recov_algo;
+ return NULL;
+ default:
+ return NULL;
+ }
+}
+EXPORT_SYMBOL_IF_KUNIT(raid6_recov_algo_find);
+#endif /* CONFIG_RAID6_PQ_KUNIT_TEST */
diff --git a/lib/raid/raid6/algos.h b/lib/raid/raid6/algos.h
index e5f1098d2179..43f636be183f 100644
--- a/lib/raid/raid6/algos.h
+++ b/lib/raid/raid6/algos.h
@@ -5,6 +5,7 @@
#ifndef _PQ_IMPL_H
#define _PQ_IMPL_H
+#include <linux/init.h>
#include <linux/raid/pq_tables.h>
/* Routine choices */
@@ -13,70 +14,28 @@ struct raid6_calls {
void (*gen_syndrome)(int disks, size_t bytes, void **ptrs);
void (*xor_syndrome)(int disks, int start, int stop, size_t bytes,
void **ptrs);
- int (*valid)(void); /* Returns 1 if this routine set is usable */
- int priority; /* Relative priority ranking if non-zero */
};
-/* Various routine sets */
-extern const struct raid6_calls raid6_intx1;
-extern const struct raid6_calls raid6_intx2;
-extern const struct raid6_calls raid6_intx4;
-extern const struct raid6_calls raid6_intx8;
-extern const struct raid6_calls raid6_mmxx1;
-extern const struct raid6_calls raid6_mmxx2;
-extern const struct raid6_calls raid6_sse1x1;
-extern const struct raid6_calls raid6_sse1x2;
-extern const struct raid6_calls raid6_sse2x1;
-extern const struct raid6_calls raid6_sse2x2;
-extern const struct raid6_calls raid6_sse2x4;
-extern const struct raid6_calls raid6_altivec1;
-extern const struct raid6_calls raid6_altivec2;
-extern const struct raid6_calls raid6_altivec4;
-extern const struct raid6_calls raid6_altivec8;
-extern const struct raid6_calls raid6_avx2x1;
-extern const struct raid6_calls raid6_avx2x2;
-extern const struct raid6_calls raid6_avx2x4;
-extern const struct raid6_calls raid6_avx512x1;
-extern const struct raid6_calls raid6_avx512x2;
-extern const struct raid6_calls raid6_avx512x4;
-extern const struct raid6_calls raid6_s390vx8;
-extern const struct raid6_calls raid6_vpermxor1;
-extern const struct raid6_calls raid6_vpermxor2;
-extern const struct raid6_calls raid6_vpermxor4;
-extern const struct raid6_calls raid6_vpermxor8;
-extern const struct raid6_calls raid6_lsx;
-extern const struct raid6_calls raid6_lasx;
-extern const struct raid6_calls raid6_rvvx1;
-extern const struct raid6_calls raid6_rvvx2;
-extern const struct raid6_calls raid6_rvvx4;
-extern const struct raid6_calls raid6_rvvx8;
-
struct raid6_recov_calls {
const char *name;
void (*data2)(int disks, size_t bytes, int faila, int failb,
void **ptrs);
void (*datap)(int disks, size_t bytes, int faila, void **ptrs);
- int (*valid)(void);
- int priority;
};
-extern const struct raid6_recov_calls raid6_recov_intx1;
-extern const struct raid6_recov_calls raid6_recov_ssse3;
-extern const struct raid6_recov_calls raid6_recov_avx2;
-extern const struct raid6_recov_calls raid6_recov_avx512;
-extern const struct raid6_recov_calls raid6_recov_s390xc;
-extern const struct raid6_recov_calls raid6_recov_neon;
-extern const struct raid6_recov_calls raid6_recov_lsx;
-extern const struct raid6_recov_calls raid6_recov_lasx;
-extern const struct raid6_recov_calls raid6_recov_rvv;
+void __init raid6_algo_add(const struct raid6_calls *algo);
+void __init raid6_algo_add_default(void);
+void __init raid6_recov_algo_add(const struct raid6_recov_calls *algo);
-extern const struct raid6_calls raid6_neonx1;
-extern const struct raid6_calls raid6_neonx2;
-extern const struct raid6_calls raid6_neonx4;
-extern const struct raid6_calls raid6_neonx8;
+/* for the kunit test */
+const struct raid6_calls *raid6_algo_find(unsigned int idx);
+const struct raid6_recov_calls *raid6_recov_algo_find(unsigned int idx);
-/* Algorithm list */
-extern const struct raid6_calls * const raid6_algos[];
-extern const struct raid6_recov_calls *const raid6_recov_algos[];
+/* generic implementations */
+extern const struct raid6_calls raid6_intx1;
+extern const struct raid6_calls raid6_intx2;
+extern const struct raid6_calls raid6_intx4;
+extern const struct raid6_calls raid6_intx8;
+extern const struct raid6_recov_calls raid6_recov_intx1;
#endif /* _PQ_IMPL_H */
diff --git a/lib/raid/raid6/arm/neon.c b/lib/raid/raid6/arm/neon.c
index bd4ec4c86ee8..341c61af675e 100644
--- a/lib/raid/raid6/arm/neon.c
+++ b/lib/raid/raid6/arm/neon.c
@@ -42,15 +42,9 @@
struct raid6_calls const raid6_neonx ## _n = { \
.gen_syndrome = raid6_neon ## _n ## _gen_syndrome, \
.xor_syndrome = raid6_neon ## _n ## _xor_syndrome, \
- .valid = raid6_have_neon, \
.name = "neonx" #_n, \
}
-static int raid6_have_neon(void)
-{
- return cpu_has_neon();
-}
-
RAID6_NEON_WRAPPER(1);
RAID6_NEON_WRAPPER(2);
RAID6_NEON_WRAPPER(4);
diff --git a/lib/raid/raid6/arm/pq_arch.h b/lib/raid/raid6/arm/pq_arch.h
new file mode 100644
index 000000000000..a1f4cc4961e9
--- /dev/null
+++ b/lib/raid/raid6/arm/pq_arch.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <asm/neon.h>
+
+extern const struct raid6_calls raid6_neonx1;
+extern const struct raid6_calls raid6_neonx2;
+extern const struct raid6_calls raid6_neonx4;
+extern const struct raid6_calls raid6_neonx8;
+extern const struct raid6_recov_calls raid6_recov_neon;
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ if (cpu_has_neon()) {
+ raid6_algo_add(&raid6_neonx1);
+ raid6_algo_add(&raid6_neonx2);
+ raid6_algo_add(&raid6_neonx4);
+ raid6_algo_add(&raid6_neonx8);
+ }
+
+ raid6_algo_add_default();
+
+ if (cpu_has_neon())
+ raid6_recov_algo_add(&raid6_recov_neon);
+}
diff --git a/lib/raid/raid6/arm/recov_neon.c b/lib/raid/raid6/arm/recov_neon.c
index e1d1d19fc9a8..1524050d09b7 100644
--- a/lib/raid/raid6/arm/recov_neon.c
+++ b/lib/raid/raid6/arm/recov_neon.c
@@ -10,11 +10,6 @@
#include "algos.h"
#include "arm/neon.h"
-static int raid6_has_neon(void)
-{
- return cpu_has_neon();
-}
-
static void raid6_2data_recov_neon(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -87,7 +82,5 @@ static void raid6_datap_recov_neon(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_neon = {
.data2 = raid6_2data_recov_neon,
.datap = raid6_datap_recov_neon,
- .valid = raid6_has_neon,
.name = "neon",
- .priority = 10,
};
diff --git a/lib/raid/raid6/arm64/pq_arch.h b/lib/raid/raid6/arm64/pq_arch.h
new file mode 100644
index 000000000000..27ff564d7594
--- /dev/null
+++ b/lib/raid/raid6/arm64/pq_arch.h
@@ -0,0 +1 @@
+#include "arm/pq_arch.h"
diff --git a/lib/raid/raid6/loongarch/loongarch_simd.c b/lib/raid/raid6/loongarch/loongarch_simd.c
index f77d11ce676e..c1eb53fafd27 100644
--- a/lib/raid/raid6/loongarch/loongarch_simd.c
+++ b/lib/raid/raid6/loongarch/loongarch_simd.c
@@ -26,11 +26,6 @@
#ifdef CONFIG_CPU_HAS_LSX
#define NSIZE 16
-static int raid6_has_lsx(void)
-{
- return cpu_has_lsx;
-}
-
static void raid6_lsx_gen_syndrome(int disks, size_t bytes, void **ptrs)
{
u8 **dptr = (u8 **)ptrs;
@@ -246,7 +241,6 @@ static void raid6_lsx_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_lsx = {
.gen_syndrome = raid6_lsx_gen_syndrome,
.xor_syndrome = raid6_lsx_xor_syndrome,
- .valid = raid6_has_lsx,
.name = "lsx",
};
@@ -256,11 +250,6 @@ const struct raid6_calls raid6_lsx = {
#ifdef CONFIG_CPU_HAS_LASX
#define NSIZE 32
-static int raid6_has_lasx(void)
-{
- return cpu_has_lasx;
-}
-
static void raid6_lasx_gen_syndrome(int disks, size_t bytes, void **ptrs)
{
u8 **dptr = (u8 **)ptrs;
@@ -414,7 +403,6 @@ static void raid6_lasx_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_lasx = {
.gen_syndrome = raid6_lasx_gen_syndrome,
.xor_syndrome = raid6_lasx_xor_syndrome,
- .valid = raid6_has_lasx,
.name = "lasx",
};
#undef NSIZE
diff --git a/lib/raid/raid6/loongarch/pq_arch.h b/lib/raid/raid6/loongarch/pq_arch.h
new file mode 100644
index 000000000000..cd4ee2df3f37
--- /dev/null
+++ b/lib/raid/raid6/loongarch/pq_arch.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <asm/cpu-features.h>
+
+extern const struct raid6_calls raid6_lsx;
+extern const struct raid6_calls raid6_lasx;
+
+extern const struct raid6_recov_calls raid6_recov_lsx;
+extern const struct raid6_recov_calls raid6_recov_lasx;
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ if (IS_ENABLED(CONFIG_CPU_HAS_LSX) && cpu_has_lsx)
+ raid6_algo_add(&raid6_lsx);
+ if (IS_ENABLED(CONFIG_CPU_HAS_LASX) && cpu_has_lasx)
+ raid6_algo_add(&raid6_lasx);
+ raid6_algo_add_default();
+
+ if (IS_ENABLED(CONFIG_CPU_HAS_LASX) && cpu_has_lasx)
+ raid6_recov_algo_add(&raid6_recov_lasx);
+ else if (IS_ENABLED(CONFIG_CPU_HAS_LSX) && cpu_has_lsx)
+ raid6_recov_algo_add(&raid6_recov_lsx);
+}
diff --git a/lib/raid/raid6/loongarch/recov_loongarch_simd.c b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
index 0bbdc8b5c2e7..87a2313bbb4f 100644
--- a/lib/raid/raid6/loongarch/recov_loongarch_simd.c
+++ b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
@@ -24,11 +24,6 @@
*/
#ifdef CONFIG_CPU_HAS_LSX
-static int raid6_has_lsx(void)
-{
- return cpu_has_lsx;
-}
-
static void raid6_2data_recov_lsx(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -291,18 +286,11 @@ static void raid6_datap_recov_lsx(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_lsx = {
.data2 = raid6_2data_recov_lsx,
.datap = raid6_datap_recov_lsx,
- .valid = raid6_has_lsx,
.name = "lsx",
- .priority = 1,
};
#endif /* CONFIG_CPU_HAS_LSX */
#ifdef CONFIG_CPU_HAS_LASX
-static int raid6_has_lasx(void)
-{
- return cpu_has_lasx;
-}
-
static void raid6_2data_recov_lasx(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -509,8 +497,6 @@ static void raid6_datap_recov_lasx(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_lasx = {
.data2 = raid6_2data_recov_lasx,
.datap = raid6_datap_recov_lasx,
- .valid = raid6_has_lasx,
.name = "lasx",
- .priority = 2,
};
#endif /* CONFIG_CPU_HAS_LASX */
diff --git a/lib/raid/raid6/powerpc/altivec.uc b/lib/raid/raid6/powerpc/altivec.uc
index eb4a448cc88e..c5429fb71dd6 100644
--- a/lib/raid/raid6/powerpc/altivec.uc
+++ b/lib/raid/raid6/powerpc/altivec.uc
@@ -104,17 +104,7 @@ static void raid6_altivec$#_gen_syndrome(int disks, size_t bytes, void **ptrs)
preempt_enable();
}
-int raid6_have_altivec(void);
-#if $# == 1
-int raid6_have_altivec(void)
-{
- /* This assumes either all CPUs have Altivec or none does */
- return cpu_has_feature(CPU_FTR_ALTIVEC);
-}
-#endif
-
const struct raid6_calls raid6_altivec$# = {
.gen_syndrome = raid6_altivec$#_gen_syndrome,
- .valid = raid6_have_altivec,
.name = "altivecx$#",
};
diff --git a/lib/raid/raid6/powerpc/pq_arch.h b/lib/raid/raid6/powerpc/pq_arch.h
new file mode 100644
index 000000000000..ecae7d3be131
--- /dev/null
+++ b/lib/raid/raid6/powerpc/pq_arch.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <asm/cputable.h>
+
+extern const struct raid6_calls raid6_altivec1;
+extern const struct raid6_calls raid6_altivec2;
+extern const struct raid6_calls raid6_altivec4;
+extern const struct raid6_calls raid6_altivec8;
+extern const struct raid6_calls raid6_vpermxor1;
+extern const struct raid6_calls raid6_vpermxor2;
+extern const struct raid6_calls raid6_vpermxor4;
+extern const struct raid6_calls raid6_vpermxor8;
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ /* This assumes either all CPUs have Altivec or none does */
+ if (cpu_has_feature(CPU_FTR_ALTIVEC)) {
+ raid6_algo_add(&raid6_altivec1);
+ raid6_algo_add(&raid6_altivec2);
+ raid6_algo_add(&raid6_altivec4);
+ raid6_algo_add(&raid6_altivec8);
+ }
+ if (cpu_has_feature(CPU_FTR_ALTIVEC_COMP) &&
+ cpu_has_feature(CPU_FTR_ARCH_207S)) {
+ raid6_algo_add(&raid6_vpermxor1);
+ raid6_algo_add(&raid6_vpermxor2);
+ raid6_algo_add(&raid6_vpermxor4);
+ raid6_algo_add(&raid6_vpermxor8);
+ }
+ raid6_algo_add_default();
+}
diff --git a/lib/raid/raid6/powerpc/vpermxor.uc b/lib/raid/raid6/powerpc/vpermxor.uc
index ec61f30bec11..e8964361aaef 100644
--- a/lib/raid/raid6/powerpc/vpermxor.uc
+++ b/lib/raid/raid6/powerpc/vpermxor.uc
@@ -76,18 +76,7 @@ static void raid6_vpermxor$#_gen_syndrome(int disks, size_t bytes, void **ptrs)
preempt_enable();
}
-int raid6_have_altivec_vpermxor(void);
-#if $# == 1
-int raid6_have_altivec_vpermxor(void)
-{
- /* Check if arch has both altivec and the vpermxor instructions */
- return (cpu_has_feature(CPU_FTR_ALTIVEC_COMP) &&
- cpu_has_feature(CPU_FTR_ARCH_207S));
-}
-#endif
-
const struct raid6_calls raid6_vpermxor$# = {
.gen_syndrome = raid6_vpermxor$#_gen_syndrome,
- .valid = raid6_have_altivec_vpermxor,
.name = "vpermxor$#",
};
diff --git a/lib/raid/raid6/recov.c b/lib/raid/raid6/recov.c
index 735ab4013771..76eb2aef3667 100644
--- a/lib/raid/raid6/recov.c
+++ b/lib/raid/raid6/recov.c
@@ -97,7 +97,5 @@ static void raid6_datap_recov_intx1(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_intx1 = {
.data2 = raid6_2data_recov_intx1,
.datap = raid6_datap_recov_intx1,
- .valid = NULL,
.name = "intx1",
- .priority = 0,
};
diff --git a/lib/raid/raid6/riscv/pq_arch.h b/lib/raid/raid6/riscv/pq_arch.h
new file mode 100644
index 000000000000..52dd01e9fc42
--- /dev/null
+++ b/lib/raid/raid6/riscv/pq_arch.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <asm/vector.h>
+
+extern const struct raid6_calls raid6_rvvx1;
+extern const struct raid6_calls raid6_rvvx2;
+extern const struct raid6_calls raid6_rvvx4;
+extern const struct raid6_calls raid6_rvvx8;
+extern const struct raid6_recov_calls raid6_recov_rvv;
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ if (has_vector()) {
+ raid6_algo_add(&raid6_rvvx1);
+ raid6_algo_add(&raid6_rvvx2);
+ raid6_algo_add(&raid6_rvvx4);
+ raid6_algo_add(&raid6_rvvx8);
+ raid6_recov_algo_add(&raid6_recov_rvv);
+ }
+ raid6_algo_add_default();
+}
diff --git a/lib/raid/raid6/riscv/recov_rvv.c b/lib/raid/raid6/riscv/recov_rvv.c
index 02120d245e22..2305940276dd 100644
--- a/lib/raid/raid6/riscv/recov_rvv.c
+++ b/lib/raid/raid6/riscv/recov_rvv.c
@@ -218,7 +218,5 @@ static void raid6_datap_recov_rvv(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_rvv = {
.data2 = raid6_2data_recov_rvv,
.datap = raid6_datap_recov_rvv,
- .valid = rvv_has_vector,
.name = "rvv",
- .priority = 1,
};
diff --git a/lib/raid/raid6/riscv/rvv.h b/lib/raid/raid6/riscv/rvv.h
index c293130d798b..3a7c2468b1ea 100644
--- a/lib/raid/raid6/riscv/rvv.h
+++ b/lib/raid/raid6/riscv/rvv.h
@@ -10,11 +10,6 @@
#include <asm/vector.h>
#include "algos.h"
-static int rvv_has_vector(void)
-{
- return has_vector();
-}
-
#define RAID6_RVV_WRAPPER(_n) \
static void raid6_rvv ## _n ## _gen_syndrome(int disks, \
size_t bytes, void **ptrs) \
@@ -41,6 +36,5 @@ static int rvv_has_vector(void)
struct raid6_calls const raid6_rvvx ## _n = { \
.gen_syndrome = raid6_rvv ## _n ## _gen_syndrome, \
.xor_syndrome = raid6_rvv ## _n ## _xor_syndrome, \
- .valid = rvv_has_vector, \
.name = "rvvx" #_n, \
}
diff --git a/lib/raid/raid6/s390/pq_arch.h b/lib/raid/raid6/s390/pq_arch.h
new file mode 100644
index 000000000000..95d14c342306
--- /dev/null
+++ b/lib/raid/raid6/s390/pq_arch.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <linux/cpufeature.h>
+
+extern const struct raid6_calls raid6_s390vx8;
+extern const struct raid6_recov_calls raid6_recov_s390xc;
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ if (cpu_has_vx())
+ raid6_algo_add(&raid6_s390vx8);
+ else
+ raid6_algo_add_default();
+ raid6_recov_algo_add(&raid6_recov_s390xc);
+}
diff --git a/lib/raid/raid6/s390/recov_s390xc.c b/lib/raid/raid6/s390/recov_s390xc.c
index e7b3409f21e2..08d56896e5ea 100644
--- a/lib/raid/raid6/s390/recov_s390xc.c
+++ b/lib/raid/raid6/s390/recov_s390xc.c
@@ -112,7 +112,5 @@ static void raid6_datap_recov_s390xc(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_s390xc = {
.data2 = raid6_2data_recov_s390xc,
.datap = raid6_datap_recov_s390xc,
- .valid = NULL,
.name = "s390xc",
- .priority = 1,
};
diff --git a/lib/raid/raid6/s390/s390vx.uc b/lib/raid/raid6/s390/s390vx.uc
index aba3515eacac..e5cf9054be2a 100644
--- a/lib/raid/raid6/s390/s390vx.uc
+++ b/lib/raid/raid6/s390/s390vx.uc
@@ -121,15 +121,8 @@ static void raid6_s390vx$#_xor_syndrome(int disks, int start, int stop,
kernel_fpu_end(&vxstate, KERNEL_VXR);
}
-static int raid6_s390vx$#_valid(void)
-{
- return cpu_has_vx();
-}
-
const struct raid6_calls raid6_s390vx$# = {
.gen_syndrome = raid6_s390vx$#_gen_syndrome,
.xor_syndrome = raid6_s390vx$#_xor_syndrome,
- .valid = raid6_s390vx$#_valid,
.name = "vx128x$#",
- .priority = 1,
};
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index 9b71f22fa19a..daaa28e96ff1 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -88,19 +88,20 @@ static void test_disks(struct kunit *test, const struct raid6_calls *calls,
static void raid6_test(struct kunit *test)
{
- const struct raid6_calls *const *algo;
- const struct raid6_recov_calls *const *ra;
int i, j, p1, p2;
+ unsigned int r, g;
- for (ra = raid6_recov_algos; *ra; ra++) {
- if ((*ra)->valid && !(*ra)->valid())
- continue;
+ for (r = 0; ; r++) {
+ const struct raid6_recov_calls *ra = raid6_recov_algo_find(r);
- for (algo = raid6_algos; *algo; algo++) {
- const struct raid6_calls *calls = *algo;
+ if (!ra)
+ break;
- if (calls->valid && !calls->valid())
- continue;
+ for (g = 0; ; g++) {
+ const struct raid6_calls *calls = raid6_algo_find(g);
+
+ if (!calls)
+ break;
/* Nuke syndromes */
memset(data[NDISKS - 2], 0xee, PAGE_SIZE);
@@ -112,7 +113,7 @@ static void raid6_test(struct kunit *test)
for (i = 0; i < NDISKS-1; i++)
for (j = i+1; j < NDISKS; j++)
- test_disks(test, calls, *ra, i, j);
+ test_disks(test, calls, ra, i, j);
if (!calls->xor_syndrome)
continue;
@@ -130,7 +131,7 @@ static void raid6_test(struct kunit *test)
for (i = 0; i < NDISKS-1; i++)
for (j = i+1; j < NDISKS; j++)
test_disks(test, calls,
- *ra, i, j);
+ ra, i, j);
}
}
diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
index 0bf831799082..7efd94e6a87a 100644
--- a/lib/raid/raid6/x86/avx2.c
+++ b/lib/raid/raid6/x86/avx2.c
@@ -24,11 +24,6 @@ static const struct raid6_avx2_constants {
0x1d1d1d1d1d1d1d1dULL, 0x1d1d1d1d1d1d1d1dULL,},
};
-static int raid6_have_avx2(void)
-{
- return boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_AVX);
-}
-
/*
* Plain AVX2 implementation
*/
@@ -131,10 +126,7 @@ static void raid6_avx21_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx2x1 = {
.gen_syndrome = raid6_avx21_gen_syndrome,
.xor_syndrome = raid6_avx21_xor_syndrome,
- .valid = raid6_have_avx2,
.name = "avx2x1",
- /* Prefer AVX2 over priority 1 (SSE2 and others) */
- .priority = 2,
};
/*
@@ -262,10 +254,7 @@ static void raid6_avx22_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx2x2 = {
.gen_syndrome = raid6_avx22_gen_syndrome,
.xor_syndrome = raid6_avx22_xor_syndrome,
- .valid = raid6_have_avx2,
.name = "avx2x2",
- /* Prefer AVX2 over priority 1 (SSE2 and others) */
- .priority = 2,
};
#ifdef CONFIG_X86_64
@@ -466,9 +455,6 @@ static void raid6_avx24_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx2x4 = {
.gen_syndrome = raid6_avx24_gen_syndrome,
.xor_syndrome = raid6_avx24_xor_syndrome,
- .valid = raid6_have_avx2,
.name = "avx2x4",
- /* Prefer AVX2 over priority 1 (SSE2 and others) */
- .priority = 2,
};
#endif /* CONFIG_X86_64 */
diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index 98ed42fb0a46..0772e798b742 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -30,16 +30,6 @@ static const struct raid6_avx512_constants {
0x1d1d1d1d1d1d1d1dULL, 0x1d1d1d1d1d1d1d1dULL,},
};
-static int raid6_have_avx512(void)
-{
- return boot_cpu_has(X86_FEATURE_AVX2) &&
- boot_cpu_has(X86_FEATURE_AVX) &&
- boot_cpu_has(X86_FEATURE_AVX512F) &&
- boot_cpu_has(X86_FEATURE_AVX512BW) &&
- boot_cpu_has(X86_FEATURE_AVX512VL) &&
- boot_cpu_has(X86_FEATURE_AVX512DQ);
-}
-
static void raid6_avx5121_gen_syndrome(int disks, size_t bytes, void **ptrs)
{
u8 **dptr = (u8 **)ptrs;
@@ -159,10 +149,7 @@ static void raid6_avx5121_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx512x1 = {
.gen_syndrome = raid6_avx5121_gen_syndrome,
.xor_syndrome = raid6_avx5121_xor_syndrome,
- .valid = raid6_have_avx512,
.name = "avx512x1",
- /* Prefer AVX512 over priority 1 (SSE2 and others) */
- .priority = 2,
};
/*
@@ -317,10 +304,7 @@ static void raid6_avx5122_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx512x2 = {
.gen_syndrome = raid6_avx5122_gen_syndrome,
.xor_syndrome = raid6_avx5122_xor_syndrome,
- .valid = raid6_have_avx512,
.name = "avx512x2",
- /* Prefer AVX512 over priority 1 (SSE2 and others) */
- .priority = 2,
};
#ifdef CONFIG_X86_64
@@ -556,9 +540,6 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx512x4 = {
.gen_syndrome = raid6_avx5124_gen_syndrome,
.xor_syndrome = raid6_avx5124_xor_syndrome,
- .valid = raid6_have_avx512,
.name = "avx512x4",
- /* Prefer AVX512 over priority 1 (SSE2 and others) */
- .priority = 2,
};
#endif
diff --git a/lib/raid/raid6/x86/mmx.c b/lib/raid/raid6/x86/mmx.c
index 052d9f010bfe..3228c335965a 100644
--- a/lib/raid/raid6/x86/mmx.c
+++ b/lib/raid/raid6/x86/mmx.c
@@ -22,12 +22,6 @@ const struct raid6_mmx_constants {
0x1d1d1d1d1d1d1d1dULL,
};
-static int raid6_have_mmx(void)
-{
- /* Not really "boot_cpu" but "all_cpus" */
- return boot_cpu_has(X86_FEATURE_MMX);
-}
-
/*
* Plain MMX implementation
*/
@@ -70,7 +64,6 @@ static void raid6_mmx1_gen_syndrome(int disks, size_t bytes, void **ptrs)
const struct raid6_calls raid6_mmxx1 = {
.gen_syndrome = raid6_mmx1_gen_syndrome,
- .valid = raid6_have_mmx,
.name = "mmxx1",
};
@@ -127,6 +120,5 @@ static void raid6_mmx2_gen_syndrome(int disks, size_t bytes, void **ptrs)
const struct raid6_calls raid6_mmxx2 = {
.gen_syndrome = raid6_mmx2_gen_syndrome,
- .valid = raid6_have_mmx,
.name = "mmxx2",
};
diff --git a/lib/raid/raid6/x86/pq_arch.h b/lib/raid/raid6/x86/pq_arch.h
new file mode 100644
index 000000000000..0493fc3a089c
--- /dev/null
+++ b/lib/raid/raid6/x86/pq_arch.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <asm/cpufeature.h>
+
+extern const struct raid6_calls raid6_mmxx1;
+extern const struct raid6_calls raid6_mmxx2;
+extern const struct raid6_calls raid6_sse1x1;
+extern const struct raid6_calls raid6_sse1x2;
+extern const struct raid6_calls raid6_sse2x1;
+extern const struct raid6_calls raid6_sse2x2;
+extern const struct raid6_calls raid6_sse2x4;
+extern const struct raid6_calls raid6_avx2x1;
+extern const struct raid6_calls raid6_avx2x2;
+extern const struct raid6_calls raid6_avx2x4;
+extern const struct raid6_calls raid6_avx512x1;
+extern const struct raid6_calls raid6_avx512x2;
+extern const struct raid6_calls raid6_avx512x4;
+
+extern const struct raid6_recov_calls raid6_recov_ssse3;
+extern const struct raid6_recov_calls raid6_recov_avx2;
+extern const struct raid6_recov_calls raid6_recov_avx512;
+
+static inline int raid6_has_avx512(void)
+{
+ return boot_cpu_has(X86_FEATURE_AVX2) &&
+ boot_cpu_has(X86_FEATURE_AVX) &&
+ boot_cpu_has(X86_FEATURE_AVX512F) &&
+ boot_cpu_has(X86_FEATURE_AVX512BW) &&
+ boot_cpu_has(X86_FEATURE_AVX512VL) &&
+ boot_cpu_has(X86_FEATURE_AVX512DQ);
+}
+
+static inline bool raid6_has_avx2(void)
+{
+ return boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_AVX);
+}
+
+static inline bool raid6_has_ssse3(void)
+{
+ return boot_cpu_has(X86_FEATURE_XMM) &&
+ boot_cpu_has(X86_FEATURE_XMM2) &&
+ boot_cpu_has(X86_FEATURE_SSSE3);
+}
+
+static inline bool raid6_has_sse2(void)
+{
+ return boot_cpu_has(X86_FEATURE_MMX) &&
+ boot_cpu_has(X86_FEATURE_FXSR) &&
+ boot_cpu_has(X86_FEATURE_XMM) &&
+ boot_cpu_has(X86_FEATURE_XMM2);
+}
+
+static inline bool raid6_has_sse1_or_mmxext(void)
+{
+ return boot_cpu_has(X86_FEATURE_MMX) &&
+ (boot_cpu_has(X86_FEATURE_XMM) ||
+ boot_cpu_has(X86_FEATURE_MMXEXT));
+}
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ if (raid6_has_avx2()) {
+ if (raid6_has_avx512()) {
+ raid6_algo_add(&raid6_avx512x1);
+ raid6_algo_add(&raid6_avx512x2);
+ if (IS_ENABLED(CONFIG_X86_64))
+ raid6_algo_add(&raid6_avx512x4);
+ }
+ raid6_algo_add(&raid6_avx2x1);
+ raid6_algo_add(&raid6_avx2x2);
+ if (IS_ENABLED(CONFIG_X86_64))
+ raid6_algo_add(&raid6_avx2x4);
+ } else if (IS_ENABLED(CONFIG_X86_64) || raid6_has_sse2()) {
+ /* x86_64 can assume SSE2 as baseline */
+ raid6_algo_add(&raid6_sse2x1);
+ raid6_algo_add(&raid6_sse2x2);
+ if (IS_ENABLED(CONFIG_X86_64))
+ raid6_algo_add(&raid6_sse2x4);
+ } else if (raid6_has_sse1_or_mmxext()) {
+ raid6_algo_add(&raid6_sse1x1);
+ raid6_algo_add(&raid6_sse1x2);
+ raid6_algo_add_default();
+ } else {
+ if (boot_cpu_has(X86_FEATURE_MMX)) {
+ raid6_algo_add(&raid6_mmxx1);
+ raid6_algo_add(&raid6_mmxx2);
+ }
+ raid6_algo_add_default();
+ }
+
+ if (raid6_has_avx512())
+ raid6_recov_algo_add(&raid6_recov_avx512);
+ else if (raid6_has_avx2())
+ raid6_recov_algo_add(&raid6_recov_avx2);
+ else if (raid6_has_ssse3())
+ raid6_recov_algo_add(&raid6_recov_ssse3);
+}
diff --git a/lib/raid/raid6/x86/recov_avx2.c b/lib/raid/raid6/x86/recov_avx2.c
index 06c6e05763bc..a714a780a2d8 100644
--- a/lib/raid/raid6/x86/recov_avx2.c
+++ b/lib/raid/raid6/x86/recov_avx2.c
@@ -9,12 +9,6 @@
#include <asm/fpu/api.h>
#include "algos.h"
-static int raid6_has_avx2(void)
-{
- return boot_cpu_has(X86_FEATURE_AVX2) &&
- boot_cpu_has(X86_FEATURE_AVX);
-}
-
static void raid6_2data_recov_avx2(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -305,11 +299,9 @@ static void raid6_datap_recov_avx2(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_avx2 = {
.data2 = raid6_2data_recov_avx2,
.datap = raid6_datap_recov_avx2,
- .valid = raid6_has_avx2,
#ifdef CONFIG_X86_64
.name = "avx2x2",
#else
.name = "avx2x1",
#endif
- .priority = 2,
};
diff --git a/lib/raid/raid6/x86/recov_avx512.c b/lib/raid/raid6/x86/recov_avx512.c
index 850bb962b514..ec72d5a30c01 100644
--- a/lib/raid/raid6/x86/recov_avx512.c
+++ b/lib/raid/raid6/x86/recov_avx512.c
@@ -11,16 +11,6 @@
#include <asm/fpu/api.h>
#include "algos.h"
-static int raid6_has_avx512(void)
-{
- return boot_cpu_has(X86_FEATURE_AVX2) &&
- boot_cpu_has(X86_FEATURE_AVX) &&
- boot_cpu_has(X86_FEATURE_AVX512F) &&
- boot_cpu_has(X86_FEATURE_AVX512BW) &&
- boot_cpu_has(X86_FEATURE_AVX512VL) &&
- boot_cpu_has(X86_FEATURE_AVX512DQ);
-}
-
static void raid6_2data_recov_avx512(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -369,11 +359,9 @@ static void raid6_datap_recov_avx512(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_avx512 = {
.data2 = raid6_2data_recov_avx512,
.datap = raid6_datap_recov_avx512,
- .valid = raid6_has_avx512,
#ifdef CONFIG_X86_64
.name = "avx512x2",
#else
.name = "avx512x1",
#endif
- .priority = 3,
};
diff --git a/lib/raid/raid6/x86/recov_ssse3.c b/lib/raid/raid6/x86/recov_ssse3.c
index 95589c33003a..700bd2c865ec 100644
--- a/lib/raid/raid6/x86/recov_ssse3.c
+++ b/lib/raid/raid6/x86/recov_ssse3.c
@@ -8,13 +8,6 @@
#include <asm/fpu/api.h>
#include "algos.h"
-static int raid6_has_ssse3(void)
-{
- return boot_cpu_has(X86_FEATURE_XMM) &&
- boot_cpu_has(X86_FEATURE_XMM2) &&
- boot_cpu_has(X86_FEATURE_SSSE3);
-}
-
static void raid6_2data_recov_ssse3(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -320,11 +313,9 @@ static void raid6_datap_recov_ssse3(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_ssse3 = {
.data2 = raid6_2data_recov_ssse3,
.datap = raid6_datap_recov_ssse3,
- .valid = raid6_has_ssse3,
#ifdef CONFIG_X86_64
.name = "ssse3x2",
#else
.name = "ssse3x1",
#endif
- .priority = 1,
};
diff --git a/lib/raid/raid6/x86/sse1.c b/lib/raid/raid6/x86/sse1.c
index 7004255a0bb1..6ebdcf824e00 100644
--- a/lib/raid/raid6/x86/sse1.c
+++ b/lib/raid/raid6/x86/sse1.c
@@ -25,14 +25,6 @@ extern const struct raid6_mmx_constants {
u64 x1d;
} raid6_mmx_constants;
-static int raid6_have_sse1_or_mmxext(void)
-{
- /* Not really boot_cpu but "all_cpus" */
- return boot_cpu_has(X86_FEATURE_MMX) &&
- (boot_cpu_has(X86_FEATURE_XMM) ||
- boot_cpu_has(X86_FEATURE_MMXEXT));
-}
-
/*
* Plain SSE1 implementation
*/
@@ -86,9 +78,7 @@ static void raid6_sse11_gen_syndrome(int disks, size_t bytes, void **ptrs)
const struct raid6_calls raid6_sse1x1 = {
.gen_syndrome = raid6_sse11_gen_syndrome,
- .valid = raid6_have_sse1_or_mmxext,
.name = "sse1x1",
- .priority = 1, /* Has cache hints */
};
/*
@@ -148,7 +138,5 @@ static void raid6_sse12_gen_syndrome(int disks, size_t bytes, void **ptrs)
const struct raid6_calls raid6_sse1x2 = {
.gen_syndrome = raid6_sse12_gen_syndrome,
- .valid = raid6_have_sse1_or_mmxext,
.name = "sse1x2",
- .priority = 1, /* Has cache hints */
};
diff --git a/lib/raid/raid6/x86/sse2.c b/lib/raid/raid6/x86/sse2.c
index f30be4ee14d0..7049c8512f35 100644
--- a/lib/raid/raid6/x86/sse2.c
+++ b/lib/raid/raid6/x86/sse2.c
@@ -22,15 +22,6 @@ static const struct raid6_sse_constants {
{ 0x1d1d1d1d1d1d1d1dULL, 0x1d1d1d1d1d1d1d1dULL },
};
-static int raid6_have_sse2(void)
-{
- /* Not really boot_cpu but "all_cpus" */
- return boot_cpu_has(X86_FEATURE_MMX) &&
- boot_cpu_has(X86_FEATURE_FXSR) &&
- boot_cpu_has(X86_FEATURE_XMM) &&
- boot_cpu_has(X86_FEATURE_XMM2);
-}
-
/*
* Plain SSE2 implementation
*/
@@ -136,9 +127,7 @@ static void raid6_sse21_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_sse2x1 = {
.gen_syndrome = raid6_sse21_gen_syndrome,
.xor_syndrome = raid6_sse21_xor_syndrome,
- .valid = raid6_have_sse2,
.name = "sse2x1",
- .priority = 1, /* Has cache hints */
};
/*
@@ -266,9 +255,7 @@ static void raid6_sse22_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_sse2x2 = {
.gen_syndrome = raid6_sse22_gen_syndrome,
.xor_syndrome = raid6_sse22_xor_syndrome,
- .valid = raid6_have_sse2,
.name = "sse2x2",
- .priority = 1, /* Has cache hints */
};
#ifdef CONFIG_X86_64
@@ -473,9 +460,7 @@ static void raid6_sse24_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_sse2x4 = {
.gen_syndrome = raid6_sse24_gen_syndrome,
.xor_syndrome = raid6_sse24_xor_syndrome,
- .valid = raid6_have_sse2,
.name = "sse2x4",
- .priority = 1, /* Has cache hints */
};
#endif /* CONFIG_X86_64 */
--
2.53.0
^ permalink raw reply related
* [PATCH 11/19] raid6: use static_call for gen_syndrom and xor_syndrom
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Avoid indirect calls for P/Q parity generation.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/algos.c | 18 ++++++++++++------
1 file changed, 12 insertions(+), 6 deletions(-)
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index 5bd953032e55..65b75236ecf8 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -14,6 +14,7 @@
#include <linux/module.h>
#include <linux/gfp.h>
#include <linux/raid/pq.h>
+#include <linux/static_call.h>
#include <kunit/visibility.h>
#include "algos.h"
@@ -23,7 +24,8 @@ static unsigned int raid6_nr_algos;
static const struct raid6_recov_calls *raid6_recov_algo;
/* Selected algorithm */
-static struct raid6_calls raid6_call;
+DEFINE_STATIC_CALL_NULL(raid6_gen_syndrome_impl, *raid6_intx1.gen_syndrome);
+DEFINE_STATIC_CALL_NULL(raid6_xor_syndrome_impl, *raid6_intx1.xor_syndrome);
/**
* raid6_gen_syndrome - generate RAID6 P/Q parity
@@ -48,7 +50,7 @@ void raid6_gen_syndrome(int disks, size_t bytes, void **ptrs)
WARN_ON_ONCE(bytes & 511);
WARN_ON_ONCE(disks < RAID6_MIN_DISKS);
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ static_call(raid6_gen_syndrome_impl)(disks, bytes, ptrs);
}
EXPORT_SYMBOL_GPL(raid6_gen_syndrome);
@@ -85,7 +87,7 @@ void raid6_xor_syndrome(int disks, int start, int stop, size_t bytes,
WARN_ON_ONCE(disks < RAID6_MIN_DISKS);
WARN_ON_ONCE(stop < start);
- raid6_call.xor_syndrome(disks, start, stop, bytes, ptrs);
+ static_call(raid6_xor_syndrome_impl)(disks, start, stop, bytes, ptrs);
}
EXPORT_SYMBOL_GPL(raid6_xor_syndrome);
@@ -96,7 +98,7 @@ EXPORT_SYMBOL_GPL(raid6_xor_syndrome);
*/
bool raid6_can_xor_syndrome(void)
{
- return !!raid6_call.xor_syndrome;
+ return !!static_call_query(raid6_xor_syndrome_impl);
}
EXPORT_SYMBOL_GPL(raid6_can_xor_syndrome);
@@ -195,7 +197,8 @@ static int raid6_choose_gen(void *(*const dptrs)[RAID6_TEST_DISKS],
return -EINVAL;
}
- raid6_call = *best;
+ static_call_update(raid6_gen_syndrome_impl, best->gen_syndrome);
+ static_call_update(raid6_xor_syndrome_impl, best->xor_syndrome);
pr_info("raid6: using algorithm %s gen() %ld MB/s\n",
best->name,
@@ -240,7 +243,10 @@ static int __init raid6_select_algo(void)
if (!IS_ENABLED(CONFIG_RAID6_PQ_BENCHMARK) || raid6_nr_algos == 1) {
pr_info("raid6: skipped pq benchmark and selected %s\n",
raid6_algos[raid6_nr_algos - 1]->name);
- raid6_call = *raid6_algos[raid6_nr_algos - 1];
+ static_call_update(raid6_gen_syndrome_impl,
+ raid6_algos[raid6_nr_algos - 1]->gen_syndrome);
+ static_call_update(raid6_xor_syndrome_impl,
+ raid6_algos[raid6_nr_algos - 1]->xor_syndrome);
return 0;
}
--
2.53.0
^ permalink raw reply related
* [PATCH 12/19] raid6: use static_call for raid6_recov_2data and raid6_recov_datap
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Avoid expensive indirect calls for the recovery routines as well.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/algos.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index 65b75236ecf8..d333318e3301 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -26,6 +26,8 @@ static const struct raid6_recov_calls *raid6_recov_algo;
/* Selected algorithm */
DEFINE_STATIC_CALL_NULL(raid6_gen_syndrome_impl, *raid6_intx1.gen_syndrome);
DEFINE_STATIC_CALL_NULL(raid6_xor_syndrome_impl, *raid6_intx1.xor_syndrome);
+DEFINE_STATIC_CALL_NULL(raid6_recov_2data_impl, *raid6_recov_intx1.data2);
+DEFINE_STATIC_CALL_NULL(raid6_recov_datap_impl, *raid6_recov_intx1.datap);
/**
* raid6_gen_syndrome - generate RAID6 P/Q parity
@@ -126,7 +128,7 @@ void raid6_recov_2data(int disks, size_t bytes, int faila, int failb,
WARN_ON_ONCE(bytes > PAGE_SIZE);
WARN_ON_ONCE(failb <= faila);
- raid6_recov_algo->data2(disks, bytes, faila, failb, ptrs);
+ static_call(raid6_recov_2data_impl)(disks, bytes, faila, failb, ptrs);
}
EXPORT_SYMBOL_GPL(raid6_recov_2data);
@@ -151,7 +153,7 @@ void raid6_recov_datap(int disks, size_t bytes, int faila, void **ptrs)
WARN_ON_ONCE(bytes & 511);
WARN_ON_ONCE(bytes > PAGE_SIZE);
- raid6_recov_algo->datap(disks, bytes, faila, ptrs);
+ static_call(raid6_recov_datap_impl)(disks, bytes, faila, ptrs);
}
EXPORT_SYMBOL_GPL(raid6_recov_datap);
@@ -324,6 +326,8 @@ static int __init raid6_init(void)
*/
if (!raid6_recov_algo)
raid6_recov_algo = &raid6_recov_intx1;
+ static_call_update(raid6_recov_2data_impl, raid6_recov_algo->data2);
+ static_call_update(raid6_recov_datap_impl, raid6_recov_algo->datap);
pr_info("raid6: using %s recovery algorithm\n", raid6_recov_algo->name);
#ifdef MODULE
--
2.53.0
^ permalink raw reply related
* [PATCH 13/19] raid6: update top of file comments
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Drop the pointless mention of the file name, and use standard formatting
for the top of file comments.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/algos.c | 8 +-------
lib/raid/raid6/arm/neon.c | 2 +-
lib/raid/raid6/mktables.c | 12 +++---------
lib/raid/raid6/recov.c | 14 ++++----------
lib/raid/raid6/riscv/rvv.h | 2 --
lib/raid/raid6/x86/avx2.c | 13 ++++---------
lib/raid/raid6/x86/avx512.c | 18 ++++++------------
lib/raid/raid6/x86/mmx.c | 10 ++--------
lib/raid/raid6/x86/sse1.c | 18 ++++++------------
lib/raid/raid6/x86/sse2.c | 9 +--------
10 files changed, 28 insertions(+), 78 deletions(-)
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index d333318e3301..35c276683cf7 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -1,12 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6/algos.c
+ * Copyright 2002 H. Peter Anvin - All Rights Reserved
*
* Algorithm list and algorithm selection for RAID-6
*/
diff --git a/lib/raid/raid6/arm/neon.c b/lib/raid/raid6/arm/neon.c
index 341c61af675e..af90869aaffc 100644
--- a/lib/raid/raid6/arm/neon.c
+++ b/lib/raid/raid6/arm/neon.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
- * linux/lib/raid6/neon.c - RAID6 syndrome calculation using ARM NEON intrinsics
+ * RAID6 syndrome calculation using ARM NEON intrinsics
*
* Copyright (C) 2013 Linaro Ltd <ard.biesheuvel@linaro.org>
*/
diff --git a/lib/raid/raid6/mktables.c b/lib/raid/raid6/mktables.c
index 97a17493bbd8..b6327b562fdb 100644
--- a/lib/raid/raid6/mktables.c
+++ b/lib/raid/raid6/mktables.c
@@ -1,15 +1,9 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002-2007 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * mktables.c
+ * Copyright 2002-2007 H. Peter Anvin - All Rights Reserved
*
- * Make RAID-6 tables. This is a host user space program to be run at
- * compile time.
+ * Make RAID-6 tables. This is a host user space program to be run at compile
+ * time.
*/
#include <stdio.h>
diff --git a/lib/raid/raid6/recov.c b/lib/raid/raid6/recov.c
index 76eb2aef3667..3fa53bc3fde4 100644
--- a/lib/raid/raid6/recov.c
+++ b/lib/raid/raid6/recov.c
@@ -1,16 +1,10 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6/recov.c
+ * Copyright 2002 H. Peter Anvin - All Rights Reserved
*
- * RAID-6 data recovery in dual failure mode. In single failure mode,
- * use the RAID-5 algorithm (or, in the case of Q failure, just reconstruct
- * the syndrome.)
+ * RAID-6 data recovery in dual failure mode. In single failure mode, use the
+ * RAID-5 algorithm (or, in the case of Q failure, just reconstruct the
+ * syndrome.)
*/
#include <linux/mm.h>
diff --git a/lib/raid/raid6/riscv/rvv.h b/lib/raid/raid6/riscv/rvv.h
index 3a7c2468b1ea..df0e3637cae8 100644
--- a/lib/raid/raid6/riscv/rvv.h
+++ b/lib/raid/raid6/riscv/rvv.h
@@ -2,8 +2,6 @@
/*
* Copyright 2024 Institute of Software, CAS.
*
- * raid6/rvv.h
- *
* Definitions for RISC-V RAID-6 code
*/
diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
index 7efd94e6a87a..7d829c669ea7 100644
--- a/lib/raid/raid6/x86/avx2.c
+++ b/lib/raid/raid6/x86/avx2.c
@@ -1,16 +1,11 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright (C) 2012 Intel Corporation
- * Author: Yuanhan Liu <yuanhan.liu@linux.intel.com>
+/*
+ * Copyright (C) 2012 Intel Corporation
+ * Author: Yuanhan Liu <yuanhan.liu@linux.intel.com>
*
- * Based on sse2.c: Copyright 2002 H. Peter Anvin - All Rights Reserved
+ * Based on sse2.c: Copyright 2002 H. Peter Anvin - All Rights Reserved
*
- * ----------------------------------------------------------------------- */
-
-/*
* AVX2 implementation of RAID-6 syndrome functions
- *
*/
#include <asm/cpufeature.h>
diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index 0772e798b742..e671eb5bde63 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -1,20 +1,14 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- --------------------------------------------------------
- *
- * Copyright (C) 2016 Intel Corporation
+/*
+ * Copyright (C) 2016 Intel Corporation
*
- * Author: Gayatri Kammela <gayatri.kammela@intel.com>
- * Author: Megha Dey <megha.dey@linux.intel.com>
+ * Author: Gayatri Kammela <gayatri.kammela@intel.com>
+ * Author: Megha Dey <megha.dey@linux.intel.com>
*
- * Based on avx2.c: Copyright 2012 Yuanhan Liu All Rights Reserved
- * Based on sse2.c: Copyright 2002 H. Peter Anvin - All Rights Reserved
+ * Based on avx2.c: Copyright 2012 Yuanhan Liu All Rights Reserved
+ * Based on sse2.c: Copyright 2002 H. Peter Anvin - All Rights Reserved
*
- * -----------------------------------------------------------------------
- */
-
-/*
* AVX512 implementation of RAID-6 syndrome functions
- *
*/
#include <asm/cpufeature.h>
diff --git a/lib/raid/raid6/x86/mmx.c b/lib/raid/raid6/x86/mmx.c
index 3228c335965a..afa82536142d 100644
--- a/lib/raid/raid6/x86/mmx.c
+++ b/lib/raid/raid6/x86/mmx.c
@@ -1,14 +1,8 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6/mmx.c
+ * Copyright 2002 H. Peter Anvin - All Rights Reserved
*
- * MMX implementation of RAID-6 syndrome functions
+ * MMX implementation of RAID-6 syndrome functions.
*/
#include <asm/cpufeature.h>
diff --git a/lib/raid/raid6/x86/sse1.c b/lib/raid/raid6/x86/sse1.c
index 6ebdcf824e00..f4b260df522a 100644
--- a/lib/raid/raid6/x86/sse1.c
+++ b/lib/raid/raid6/x86/sse1.c
@@ -1,19 +1,13 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6/sse1.c
+ * Copyright 2002 H. Peter Anvin - All Rights Reserved
*
- * SSE-1/MMXEXT implementation of RAID-6 syndrome functions
+ * SSE-1/MMXEXT implementation of RAID-6 syndrome functions.
*
- * This is really an MMX implementation, but it requires SSE-1 or
- * AMD MMXEXT for prefetch support and a few other features. The
- * support for nontemporal memory accesses is enough to make this
- * worthwhile as a separate implementation.
+ * This is really an MMX implementation, but it requires SSE-1 or AMD MMXEXT for
+ * prefetch support and a few other features. The support for nontemporal
+ * memory accesses is enough to make this worthwhile as a separate
+ * implementation.
*/
#include <asm/cpufeature.h>
diff --git a/lib/raid/raid6/x86/sse2.c b/lib/raid/raid6/x86/sse2.c
index 7049c8512f35..43b09ce58270 100644
--- a/lib/raid/raid6/x86/sse2.c
+++ b/lib/raid/raid6/x86/sse2.c
@@ -1,15 +1,8 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6/sse2.c
+ * Copyright 2002 H. Peter Anvin - All Rights Reserved
*
* SSE-2 implementation of RAID-6 syndrome functions
- *
*/
#include <asm/cpufeature.h>
--
2.53.0
^ permalink raw reply related
* [PATCH 14/19] raid6_kunit: use KUNIT_CASE_PARAM
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
The raid6 test combines various generation and recovery algorithms. Use
KUNIT_CASE_PARAM and provide a generator that iterates over the possible
combinations instead of looping inside a single test instance.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/tests/raid6_kunit.c | 120 ++++++++++++++++-------------
1 file changed, 68 insertions(+), 52 deletions(-)
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index daaa28e96ff1..f55b081b6b13 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -21,6 +21,15 @@ static char data[NDISKS][PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
static char recovi[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
static char recovj[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
+struct test_args {
+ unsigned int recov_idx;
+ const struct raid6_recov_calls *recov;
+ unsigned int gen_idx;
+ const struct raid6_calls *gen;
+};
+
+static struct test_args args;
+
static void makedata(int start, int stop)
{
int i;
@@ -43,9 +52,10 @@ static char member_type(int d)
}
}
-static void test_disks(struct kunit *test, const struct raid6_calls *calls,
- const struct raid6_recov_calls *ra, int faila, int failb)
+static void test_recover(struct kunit *test, int faila, int failb)
{
+ const struct test_args *ta = test->param_value;
+
memset(recovi, 0xf0, PAGE_SIZE);
memset(recovj, 0xba, PAGE_SIZE);
@@ -61,25 +71,23 @@ static void test_disks(struct kunit *test, const struct raid6_calls *calls,
goto skip;
/* P+Q failure. Just rebuild the syndrome. */
- calls->gen_syndrome(NDISKS, PAGE_SIZE, dataptrs);
+ ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, dataptrs);
} else if (failb == NDISKS - 2) {
/* data+P failure. */
- ra->datap(NDISKS, PAGE_SIZE, faila, dataptrs);
+ ta->recov->datap(NDISKS, PAGE_SIZE, faila, dataptrs);
} else {
/* data+data failure. */
- ra->data2(NDISKS, PAGE_SIZE, faila, failb, dataptrs);
+ ta->recov->data2(NDISKS, PAGE_SIZE, faila, failb, dataptrs);
}
KUNIT_EXPECT_MEMEQ_MSG(test, data[faila], recovi, PAGE_SIZE,
- "algo=%-8s/%-8s faila miscompared: %3d[%c] (failb=%3d[%c])\n",
- calls->name, ra->name,
- faila, member_type(faila),
- failb, member_type(failb));
+ "faila miscompared: %3d[%c] (failb=%3d[%c])\n",
+ faila, member_type(faila),
+ failb, member_type(failb));
KUNIT_EXPECT_MEMEQ_MSG(test, data[failb], recovj, PAGE_SIZE,
- "algo=%-8s/%-8s failb miscompared: %3d[%c] (faila=%3d[%c])\n",
- calls->name, ra->name,
- failb, member_type(failb),
- faila, member_type(faila));
+ "failb miscompared: %3d[%c] (faila=%3d[%c])\n",
+ failb, member_type(failb),
+ faila, member_type(faila));
skip:
dataptrs[faila] = data[faila];
@@ -88,58 +96,66 @@ static void test_disks(struct kunit *test, const struct raid6_calls *calls,
static void raid6_test(struct kunit *test)
{
+ const struct test_args *ta = test->param_value;
int i, j, p1, p2;
- unsigned int r, g;
-
- for (r = 0; ; r++) {
- const struct raid6_recov_calls *ra = raid6_recov_algo_find(r);
-
- if (!ra)
- break;
-
- for (g = 0; ; g++) {
- const struct raid6_calls *calls = raid6_algo_find(g);
-
- if (!calls)
- break;
- /* Nuke syndromes */
- memset(data[NDISKS - 2], 0xee, PAGE_SIZE);
- memset(data[NDISKS - 1], 0xee, PAGE_SIZE);
+ /* Nuke syndromes */
+ memset(data[NDISKS - 2], 0xee, PAGE_SIZE);
+ memset(data[NDISKS - 1], 0xee, PAGE_SIZE);
- /* Generate assumed good syndrome */
- calls->gen_syndrome(NDISKS, PAGE_SIZE,
- (void **)&dataptrs);
+ /* Generate assumed good syndrome */
+ ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, (void **)&dataptrs);
- for (i = 0; i < NDISKS-1; i++)
- for (j = i+1; j < NDISKS; j++)
- test_disks(test, calls, ra, i, j);
+ for (i = 0; i < NDISKS - 1; i++)
+ for (j = i + 1; j < NDISKS; j++)
+ test_recover(test, i, j);
- if (!calls->xor_syndrome)
- continue;
+ if (!ta->gen->xor_syndrome)
+ return;
- for (p1 = 0; p1 < NDISKS-2; p1++)
- for (p2 = p1; p2 < NDISKS-2; p2++) {
+ for (p1 = 0; p1 < NDISKS - 2; p1++) {
+ for (p2 = p1; p2 < NDISKS - 2; p2++) {
+ /* Simulate rmw run */
+ ta->gen->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
+ (void **)&dataptrs);
+ makedata(p1, p2);
+ ta->gen->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
+ (void **)&dataptrs);
- /* Simulate rmw run */
- calls->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
- (void **)&dataptrs);
- makedata(p1, p2);
- calls->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
- (void **)&dataptrs);
+ for (i = 0; i < NDISKS - 1; i++)
+ for (j = i + 1; j < NDISKS; j++)
+ test_recover(test, i, j);
+ }
+ }
+}
- for (i = 0; i < NDISKS-1; i++)
- for (j = i+1; j < NDISKS; j++)
- test_disks(test, calls,
- ra, i, j);
- }
+static const void *raid6_gen_params(struct kunit *test, const void *prev,
+ char *desc)
+{
+ if (!prev) {
+ memset(&args, 0, sizeof(args));
+next_algo:
+ args.recov_idx = 0;
+ args.gen = raid6_algo_find(args.gen_idx);
+ if (!args.gen)
+ return NULL;
+ }
- }
+ if (args.recov)
+ args.recov_idx++;
+ args.recov = raid6_recov_algo_find(args.recov_idx);
+ if (!args.recov) {
+ args.gen_idx++;
+ goto next_algo;
}
+
+ snprintf(desc, KUNIT_PARAM_DESC_SIZE, "gen=%s recov=%s",
+ args.gen->name, args.recov->name);
+ return &args;
}
static struct kunit_case raid6_test_cases[] = {
- KUNIT_CASE(raid6_test),
+ KUNIT_CASE_PARAM(raid6_test, raid6_gen_params),
{},
};
--
2.53.0
^ permalink raw reply related
* [PATCH 15/19] raid6_kunit: dynamically allocate data buffers using vmalloc
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Use vmalloc for the data buffers instead of using static .data allocations.
This provides for better out of bounds checking and avoids wasting kernel
memory after the test has run. vmalloc is used instead of kmalloc to
provide for better out of bounds access checking as in other kunit tests.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/tests/raid6_kunit.c | 77 ++++++++++++++++++++++++------
1 file changed, 62 insertions(+), 15 deletions(-)
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index f55b081b6b13..a4b65ccc9d20 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -7,19 +7,20 @@
#include <kunit/test.h>
#include <linux/prandom.h>
+#include <linux/vmalloc.h>
#include "../algos.h"
MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING");
#define RAID6_KUNIT_SEED 42
+#define RAID6_KUNIT_MAX_FAILURES 2
#define NDISKS 16 /* Including P and Q */
static struct rnd_state rng;
static void *dataptrs[NDISKS];
-static char data[NDISKS][PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
-static char recovi[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
-static char recovj[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
+static void *test_buffers[NDISKS];
+static void *test_recov_buffers[RAID6_KUNIT_MAX_FAILURES];
struct test_args {
unsigned int recov_idx;
@@ -35,8 +36,8 @@ static void makedata(int start, int stop)
int i;
for (i = start; i <= stop; i++) {
- prandom_bytes_state(&rng, data[i], PAGE_SIZE);
- dataptrs[i] = data[i];
+ prandom_bytes_state(&rng, test_buffers[i], PAGE_SIZE);
+ dataptrs[i] = test_buffers[i];
}
}
@@ -55,12 +56,13 @@ static char member_type(int d)
static void test_recover(struct kunit *test, int faila, int failb)
{
const struct test_args *ta = test->param_value;
+ int i;
- memset(recovi, 0xf0, PAGE_SIZE);
- memset(recovj, 0xba, PAGE_SIZE);
+ for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++)
+ memset(test_recov_buffers[i], 0xf0, PAGE_SIZE);
- dataptrs[faila] = recovi;
- dataptrs[failb] = recovj;
+ dataptrs[faila] = test_recov_buffers[0];
+ dataptrs[failb] = test_recov_buffers[1];
if (failb == NDISKS - 1) {
/*
@@ -80,18 +82,20 @@ static void test_recover(struct kunit *test, int faila, int failb)
ta->recov->data2(NDISKS, PAGE_SIZE, faila, failb, dataptrs);
}
- KUNIT_EXPECT_MEMEQ_MSG(test, data[faila], recovi, PAGE_SIZE,
+ KUNIT_EXPECT_MEMEQ_MSG(test, test_buffers[faila], test_recov_buffers[0],
+ PAGE_SIZE,
"faila miscompared: %3d[%c] (failb=%3d[%c])\n",
faila, member_type(faila),
failb, member_type(failb));
- KUNIT_EXPECT_MEMEQ_MSG(test, data[failb], recovj, PAGE_SIZE,
+ KUNIT_EXPECT_MEMEQ_MSG(test, test_buffers[failb], test_recov_buffers[1],
+ PAGE_SIZE,
"failb miscompared: %3d[%c] (faila=%3d[%c])\n",
failb, member_type(failb),
faila, member_type(faila));
skip:
- dataptrs[faila] = data[faila];
- dataptrs[failb] = data[failb];
+ dataptrs[faila] = test_buffers[faila];
+ dataptrs[failb] = test_buffers[failb];
}
static void raid6_test(struct kunit *test)
@@ -100,8 +104,8 @@ static void raid6_test(struct kunit *test)
int i, j, p1, p2;
/* Nuke syndromes */
- memset(data[NDISKS - 2], 0xee, PAGE_SIZE);
- memset(data[NDISKS - 1], 0xee, PAGE_SIZE);
+ memset(test_buffers[NDISKS - 2], 0xee, PAGE_SIZE);
+ memset(test_buffers[NDISKS - 1], 0xee, PAGE_SIZE);
/* Generate assumed good syndrome */
ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, (void **)&dataptrs);
@@ -161,15 +165,58 @@ static struct kunit_case raid6_test_cases[] = {
static int raid6_suite_init(struct kunit_suite *suite)
{
+ int i;
+
prandom_seed_state(&rng, RAID6_KUNIT_SEED);
+
+ /*
+ * Allocate the test buffer using vmalloc() with a page-aligned length
+ * so that it is immediately followed by a guard page. This allows
+ * buffer overreads to be detected, even in assembly code.
+ */
+ for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++) {
+ test_recov_buffers[i] = vmalloc(PAGE_SIZE);
+ if (!test_recov_buffers[i])
+ goto out_free_recov_buffers;
+ }
+ for (i = 0; i < NDISKS; i++) {
+ test_buffers[i] = vmalloc(PAGE_SIZE);
+ if (!test_buffers[i])
+ goto out_free_buffers;
+ }
+
makedata(0, NDISKS - 1);
+
return 0;
+
+out_free_buffers:
+ for (i = 0; i < NDISKS; i++)
+ vfree(test_buffers[i]);
+ memset(test_buffers, 0, sizeof(test_buffers));
+out_free_recov_buffers:
+ for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++)
+ vfree(test_recov_buffers[i]);
+ memset(test_recov_buffers, 0, sizeof(test_recov_buffers));
+ return -ENOMEM;
+}
+
+static void raid6_suite_exit(struct kunit_suite *suite)
+{
+ int i;
+
+ for (i = 0; i < NDISKS; i++)
+ vfree(test_buffers[i]);
+ memset(test_buffers, 0, sizeof(test_buffers));
+ for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++)
+ vfree(test_recov_buffers[i]);
+ memset(test_recov_buffers, 0, sizeof(test_recov_buffers));
}
static struct kunit_suite raid6_test_suite = {
.name = "raid6",
.test_cases = raid6_test_cases,
.suite_init = raid6_suite_init,
+ .suite_exit = raid6_suite_exit,
};
kunit_test_suite(raid6_test_suite);
--
2.53.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox