* Re: [PATCH 01/19] btrfs: require at least 4 devices for RAID 6
From: Christoph Hellwig @ 2026-05-15 4:37 UTC (permalink / raw)
To: H. Peter Anvin
Cc: kreijack, Goffredo Baroncelli, Christoph Hellwig, David Sterba,
Andrew Morton, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Huacai Chen, WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, Herbert Xu, Dan Williams, Chris Mason,
David Sterba, Arnd Bergmann, Song Liu, Yu Kuai, Li Nan,
linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <0507CCEF-0548-442F-8703-1D006B5E068B@zytor.com>
On Thu, May 14, 2026 at 12:57:53PM -0700, H. Peter Anvin wrote:
> That's what I'm saying – it should invoke the RAID-1 code under the
> cover (as with 3 disks, D = P = Q.)
Yes, if the btrfs maintainer cared for this setup that is what should
be done.
^ permalink raw reply
* Re: [PATCH 01/19] btrfs: require at least 4 devices for RAID 6
From: Christoph Hellwig @ 2026-05-15 4:37 UTC (permalink / raw)
To: kreijack
Cc: Christoph Hellwig, David Sterba, Andrew Morton, Catalin Marinas,
Will Deacon, Ard Biesheuvel, Huacai Chen, WANG Xuerui,
Madhavan Srinivasan, Michael Ellerman, Nicholas Piggin,
Christophe Leroy (CS GROUP), Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H. Peter Anvin, Herbert Xu, Dan Williams, Chris Mason,
David Sterba, Arnd Bergmann, Song Liu, Yu Kuai, Li Nan,
linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <0a8d1ff4-f5a2-49e9-aa45-d25dbe4ded40@libero.it>
On Thu, May 14, 2026 at 09:51:59PM +0200, Goffredo Baroncelli wrote:
> I think that the David concern is : "what happens for an already
> existing btrfs raid6 3 disks filesystem when the user upgrade the kernel ?"
> (I am thinking when a new BG needs to be allocated)...
Then it will cleanly fail to mount instead of constantly corrupting data
and memory with every write, yes. Which clearly suggest that such
file systems don't exist in the wild.
But if btrfs wants to keep supporting this I'll just add a _unsafe
version without the check in the core library.
^ permalink raw reply
* Re: [PATCH V4 0/3] md/nvme: Enable PCI P2PDMA support for RAID0 and NVMe Multipath
From: Christoph Hellwig @ 2026-05-15 4:35 UTC (permalink / raw)
To: Chaitanya Kulkarni
Cc: song, yukuai, linan122, kbusch, axboe, sagi, linux-block,
linux-raid, linux-nvme, kmodukuri
In-Reply-To: <20260513185153.95552-1-kch@nvidia.com>
Still looks good to me as per the reviews.
^ permalink raw reply
* Re: [PATCH 01/19] btrfs: require at least 4 devices for RAID 6
From: H. Peter Anvin @ 2026-05-14 19:57 UTC (permalink / raw)
To: kreijack, Goffredo Baroncelli, Christoph Hellwig, David Sterba
Cc: Andrew Morton, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Huacai Chen, WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, Herbert Xu, Dan Williams, Chris Mason,
David Sterba, Arnd Bergmann, Song Liu, Yu Kuai, Li Nan,
linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <0a8d1ff4-f5a2-49e9-aa45-d25dbe4ded40@libero.it>
On May 14, 2026 12:51:59 PM PDT, Goffredo Baroncelli <kreijack@libero.it> wrote:
>On 13/05/2026 07.47, Christoph Hellwig wrote:
>> On Tue, May 12, 2026 at 01:42:31PM +0200, David Sterba wrote:
>
>>
>>> The degenerate modes of
>>> raid0, 5, or 6 are explicit as a possible middle step when converting
>>> profiles. We can use a fallback implementation for this case if the
>>> accelerated implementations cannot do it.
>>
>> This is not about a degenerated mode. For a degenerated RAID 6, parity
>> generation uses the RAID 5 XOR routines as the second parity will be
>> missing. This is about generating two parities for a single data disk,
>> which must be explicitly selected.
>>
>
>I think that the David concern is : "what happens for an already
>existing btrfs raid6 3 disks filesystem when the user upgrade the kernel ?"
>(I am thinking when a new BG needs to be allocated)...
>
>BR
>GB
>
That's what I'm saying – it should invoke the RAID-1 code under the cover (as with 3 disks, D = P = Q.)
^ permalink raw reply
* Re: [PATCH 01/19] btrfs: require at least 4 devices for RAID 6
From: Goffredo Baroncelli @ 2026-05-14 19:51 UTC (permalink / raw)
To: Christoph Hellwig, David Sterba
Cc: Andrew Morton, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Huacai Chen, WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260513054742.GA1018@lst.de>
On 13/05/2026 07.47, Christoph Hellwig wrote:
> On Tue, May 12, 2026 at 01:42:31PM +0200, David Sterba wrote:
>
>> The degenerate modes of
>> raid0, 5, or 6 are explicit as a possible middle step when converting
>> profiles. We can use a fallback implementation for this case if the
>> accelerated implementations cannot do it.
>
> This is not about a degenerated mode. For a degenerated RAID 6, parity
> generation uses the RAID 5 XOR routines as the second parity will be
> missing. This is about generating two parities for a single data disk,
> which must be explicitly selected.
>
I think that the David concern is : "what happens for an already
existing btrfs raid6 3 disks filesystem when the user upgrade the kernel ?"
(I am thinking when a new BG needs to be allocated)...
BR
GB
--
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5
^ permalink raw reply
* Re: [PATCH 01/19] btrfs: require at least 4 devices for RAID 6
From: David Sterba @ 2026-05-13 20:19 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Andrew Morton, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Huacai Chen, WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260513054742.GA1018@lst.de>
On Wed, May 13, 2026 at 07:47:42AM +0200, Christoph Hellwig wrote:
> On Tue, May 12, 2026 at 01:42:31PM +0200, David Sterba wrote:
> > On Tue, May 12, 2026 at 07:20:41AM +0200, Christoph Hellwig wrote:
> > > While the RAID6 algorithm could in theory support 3 devices by just
> > > copying the data disk to the two parity disks, this version is not only
> > > useless because it is a suboptimal version of 3-way mirroring, but also
> > > broken with various crashes and incorrect parity generation in various
> > > architecture-optimized implementations. Disallow it similar to mdraid
> > > which requires at least 4 devices for RAID 6.
> > >
> > > Fixes: 53b381b3abeb ("Btrfs: RAID5 and RAID6")
> > > Signed-off-by: Christoph Hellwig <hch@lst.de>
> >
> > This patch should have been sent separately as it has user visible
> > impact and can potentially break some setups.
>
> It _is_ sent out separate.
It's public interface change of btrfs but in a patch series cleaning
up some library code, I noticed it by accident.
> > The degenerate modes of
> > raid0, 5, or 6 are explicit as a possible middle step when converting
> > profiles. We can use a fallback implementation for this case if the
> > accelerated implementations cannot do it.
>
> This is not about a degenerated mode. For a degenerated RAID 6, parity
> generation uses the RAID 5 XOR routines as the second parity will be
> missing. This is about generating two parities for a single data disk,
> which must be explicitly selected.
The calcuation is a different than what I'm concened about, changing
minimum devices from 3 to 4 is a breaking change. If the library won't
provide the xor/parity functions then we'll have to add a fallback for
this special case.
^ permalink raw reply
* [PATCH V4 3/3] nvme-multipath: enable PCI P2PDMA for multipath devices
From: Chaitanya Kulkarni @ 2026-05-13 18:51 UTC (permalink / raw)
To: song, yukuai, linan122, kbusch, axboe, hch, sagi
Cc: linux-block, linux-raid, linux-nvme, kmodukuri,
Pranjal Shrivastava, Nitesh Shetty, Chaitanya Kulkarni
In-Reply-To: <20260513185153.95552-1-kch@nvidia.com>
From: Kiran Kumar Modukuri <kmodukuri@nvidia.com>
NVMe multipath does not expose BLK_FEAT_PCI_P2PDMA on the head disk
even when all underlying controllers support it.
Set BLK_FEAT_PCI_P2PDMA unconditionally in nvme_mpath_alloc_disk()
alongside the other features. nvme_update_ns_info_block() already
calls queue_limits_stack_bdev() to stack each path's limits onto the
head disk, which routes through blk_stack_limits(). The core now
clears BLK_FEAT_PCI_P2PDMA automatically if any path (e.g., FC) does
not support it, consistent with how BLK_FEAT_NOWAIT and BLK_FEAT_POLL
are handled.
Tested-by: Pranjal Shrivastava<praan@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Reviewed-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Kiran Kumar Modukuri <kmodukuri@nvidia.com>
Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
drivers/nvme/host/multipath.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index 263161cb8ac0..ff442bbf2937 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -730,7 +730,7 @@ int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl, struct nvme_ns_head *head)
blk_set_stacking_limits(&lim);
lim.dma_alignment = 3;
lim.features |= BLK_FEAT_IO_STAT | BLK_FEAT_NOWAIT |
- BLK_FEAT_POLL | BLK_FEAT_ATOMIC_WRITES;
+ BLK_FEAT_POLL | BLK_FEAT_ATOMIC_WRITES | BLK_FEAT_PCI_P2PDMA;
if (head->ids.csi == NVME_CSI_ZNS)
lim.features |= BLK_FEAT_ZONED;
--
2.39.5
^ permalink raw reply related
* [PATCH V4 2/3] md: propagate BLK_FEAT_PCI_P2PDMA from member devices to RAID device
From: Chaitanya Kulkarni @ 2026-05-13 18:51 UTC (permalink / raw)
To: song, yukuai, linan122, kbusch, axboe, hch, sagi
Cc: linux-block, linux-raid, linux-nvme, kmodukuri,
Pranjal Shrivastava, Xiao Ni, Chaitanya Kulkarni
In-Reply-To: <20260513185153.95552-1-kch@nvidia.com>
From: Kiran Kumar Modukuri <kmodukuri@nvidia.com>
MD RAID does not propagate BLK_FEAT_PCI_P2PDMA from member devices to
the RAID device, preventing peer-to-peer DMA through the RAID layer even
when all underlying devices support it.
Enable BLK_FEAT_PCI_P2PDMA unconditionally in raid0, raid1 and raid10
personalities during queue limits setup. blk_stack_limits() clears it
automatically if any member device lacks support, consistent with how
BLK_FEAT_NOWAIT and BLK_FEAT_POLL are handled in the block core.
Parity RAID personalities (raid4/5/6) are excluded because they require
CPU access to data pages for parity computation, which is incompatible
with P2P mappings.
Tested with RAID0/1/10 arrays containing multiple NVMe devices with
P2PDMA support, confirming that peer-to-peer transfers work correctly
through the RAID layer.
Tested-by: Pranjal Shrivastava<praan@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Reviewed-by: Xiao Ni <xni@redhat.com>
Signed-off-by: Kiran Kumar Modukuri <kmodukuri@nvidia.com>
Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
drivers/md/raid0.c | 1 +
drivers/md/raid1.c | 1 +
drivers/md/raid10.c | 1 +
3 files changed, 3 insertions(+)
diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c
index 5e38a51e349a..2cdaf7495d92 100644
--- a/drivers/md/raid0.c
+++ b/drivers/md/raid0.c
@@ -392,6 +392,7 @@ static int raid0_set_limits(struct mddev *mddev)
lim.io_opt = lim.io_min * mddev->raid_disks;
lim.chunk_sectors = mddev->chunk_sectors;
lim.features |= BLK_FEAT_ATOMIC_WRITES;
+ lim.features |= BLK_FEAT_PCI_P2PDMA;
err = mddev_stack_rdev_limits(mddev, &lim, MDDEV_STACK_INTEGRITY);
if (err)
return err;
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index 64d970e2ef50..cc628a1be52c 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -3208,6 +3208,7 @@ static int raid1_set_limits(struct mddev *mddev)
lim.max_hw_wzeroes_unmap_sectors = 0;
lim.logical_block_size = mddev->logical_block_size;
lim.features |= BLK_FEAT_ATOMIC_WRITES;
+ lim.features |= BLK_FEAT_PCI_P2PDMA;
err = mddev_stack_rdev_limits(mddev, &lim, MDDEV_STACK_INTEGRITY);
if (err)
return err;
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 39085e7dd6d2..f905dc391b74 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -3941,6 +3941,7 @@ static int raid10_set_queue_limits(struct mddev *mddev)
lim.chunk_sectors = mddev->chunk_sectors;
lim.io_opt = lim.io_min * raid10_nr_stripes(conf);
lim.features |= BLK_FEAT_ATOMIC_WRITES;
+ lim.features |= BLK_FEAT_PCI_P2PDMA;
err = mddev_stack_rdev_limits(mddev, &lim, MDDEV_STACK_INTEGRITY);
if (err)
return err;
--
2.39.5
^ permalink raw reply related
* [PATCH V4 1/3] block: clear BLK_FEAT_PCI_P2PDMA in blk_stack_limits() for non-supporting devices
From: Chaitanya Kulkarni @ 2026-05-13 18:51 UTC (permalink / raw)
To: song, yukuai, linan122, kbusch, axboe, hch, sagi
Cc: linux-block, linux-raid, linux-nvme, kmodukuri,
Chaitanya Kulkarni, Pranjal Shrivastava, Nitesh Shetty
In-Reply-To: <20260513185153.95552-1-kch@nvidia.com>
BLK_FEAT_NOWAIT and BLK_FEAT_POLL are cleared in blk_stack_limits()
when an underlying device does not support them. Apply the same
treatment to BLK_FEAT_PCI_P2PDMA: stacking drivers set it
unconditionally and rely on the core to clear it whenever a
non-supporting member device is stacked.
Tested-by: Pranjal Shrivastava<praan@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Reviewed-by: Nitesh Shetty <nj.shetty@samsung.com>
Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
block/blk-settings.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/block/blk-settings.c b/block/blk-settings.c
index 78c83817b9d3..8274631290db 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -795,6 +795,8 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b,
t->features &= ~BLK_FEAT_NOWAIT;
if (!(b->features & BLK_FEAT_POLL))
t->features &= ~BLK_FEAT_POLL;
+ if (!(b->features & BLK_FEAT_PCI_P2PDMA))
+ t->features &= ~BLK_FEAT_PCI_P2PDMA;
t->flags |= (b->flags & BLK_FLAG_MISALIGNED);
--
2.39.5
^ permalink raw reply related
* [PATCH V4 0/3] md/nvme: Enable PCI P2PDMA support for RAID0 and NVMe Multipath
From: Chaitanya Kulkarni @ 2026-05-13 18:51 UTC (permalink / raw)
To: song, yukuai, linan122, kbusch, axboe, hch, sagi
Cc: linux-block, linux-raid, linux-nvme, kmodukuri,
Chaitanya Kulkarni
Hi,
This patch series extends PCI peer-to-peer DMA (P2PDMA) support to enable
direct data transfers between PCIe devices through RAID and NVMe multipath
block layers.
Current Linux kernel P2PDMA infrastructure supports direct peer-to-peer
transfers, but this support is not propagated through certain storage
stacks like MD RAID and NVMe multipath. This adds two patches for
MD RAID 0/1/10 and NVMe to propogate P2PDMA support through the
storage stack.
All four test scenarios demonstrate that P2PDMA capabilities are correctly
propagated through both the MD RAID layer (patch 2/3) and NVMe multipath
layer (patch 3/3). Direct peer-to-peer transfers complete successfully with
full data integrity verification, confirming that:
1. RAID devices properly inherit P2PDMA capability from member devices
2. NVMe multipath devices correctly expose P2PDMA support
3. P2P memory buffers can be used for transfers involving both types
4. Data integrity is maintained across all transfer combinations
I've added blktest log as well at the end.
Repo:-
git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux.git
Branch HEAD:-
commit 5d75bcab6af0916283c2675d831d39fe4d360b11 (origin/for-next)
Merge: feab7e5ae6ee 87d0740b7c4c
Author: Jens Axboe <axboe@kernel.dk>
Date: Wed May 13 07:55:56 2026 -0600
Merge branch 'block-7.1' into for-next
* block-7.1:
selftests: ublk: cap nthreads to kernel's actual nr_hw_queues
-ck
Changes from V3:-
1. Rebase on latest code and add review and tested by tags.
Changes from V2:-
1. Unconditionally set the BLK_FEAT_PCI_P2PDMA for md and nvme multipath.
(Christoph)
2. Add a prep patch to diable BLK_FEAT_PCI_P2PDMA in the blk_stack_limit().
(christoph)
Changes from V1:-
- Update patch 1 to explicitly support MD RAID 0/1/10.
- Fix signoff chain order for patch 2.
- Clear BLK_FEAT_PCI_P2PDMA in nvme_mpath_add_disk() when a newly
added path does not support it, to handle multipath across different
transports.
- Add nvme multipath test log for mixed transport TCP and PCIe.
Chaitanya Kulkarni (1):
block: clear BLK_FEAT_PCI_P2PDMA in blk_stack_limits() for
non-supporting devices
Kiran Kumar Modukuri (2):
md: propagate BLK_FEAT_PCI_P2PDMA from member devices to RAID device
nvme-multipath: enable PCI P2PDMA for multipath devices
block/blk-settings.c | 2 ++
drivers/md/raid0.c | 1 +
drivers/md/raid1.c | 1 +
drivers/md/raid10.c | 1 +
drivers/nvme/host/multipath.c | 2 +-
5 files changed, 6 insertions(+), 1 deletion(-)
++ for t in loop tcp
++ echo '################NVMET_TRTYPES=loop############'
################NVMET_TRTYPES=loop############
++ NVMET_TRTYPES=loop
++ ./check nvme
nvme/002 (tr=loop) (create many subsystems and test discovery) [passed]
runtime 33.808s ... 34.634s
nvme/003 (tr=loop) (test if we're sending keep-alives to a discovery controller) [passed]
runtime 10.222s ... 10.239s
nvme/004 (tr=loop) (test nvme and nvmet UUID NS descriptors) [passed]
runtime 0.683s ... 0.651s
nvme/005 (tr=loop) (reset local loopback target) [passed]
runtime 0.974s ... 1.001s
nvme/006 (tr=loop bd=device) (create an NVMeOF target) [passed]
runtime 0.088s ... 0.094s
nvme/006 (tr=loop bd=file) (create an NVMeOF target) [passed]
runtime 0.082s ... 0.087s
nvme/008 (tr=loop bd=device) (create an NVMeOF host) [passed]
runtime 0.652s ... 0.658s
nvme/008 (tr=loop bd=file) (create an NVMeOF host) [passed]
runtime 0.634s ... 0.658s
nvme/010 (tr=loop bd=device) (run data verification fio job) [passed]
runtime 9.426s ... 9.469s
nvme/010 (tr=loop bd=file) (run data verification fio job) [passed]
runtime 41.443s ... 43.716s
nvme/012 (tr=loop bd=device) (run mkfs and data verification fio) [passed]
runtime 47.971s ... 51.404s
nvme/012 (tr=loop bd=file) (run mkfs and data verification fio) [passed]
runtime 38.937s ... 38.971s
nvme/014 (tr=loop bd=device) (flush a command from host) [passed]
runtime 9.257s ... 9.208s
nvme/014 (tr=loop bd=file) (flush a command from host) [passed]
runtime 8.874s ... 8.863s
nvme/016 (tr=loop) (create/delete many NVMeOF block device-backed ns and test discovery) [passed]
runtime 20.101s ... 19.387s
nvme/017 (tr=loop) (create/delete many file-ns and test discovery) [passed]
runtime 22.534s ... 21.919s
nvme/018 (tr=loop) (unit test NVMe-oF out of range access on a file backend) [passed]
runtime 0.629s ... 0.653s
nvme/019 (tr=loop bd=device) (test NVMe DSM Discard command) [passed]
runtime 0.638s ... 0.657s
nvme/019 (tr=loop bd=file) (test NVMe DSM Discard command) [passed]
runtime 0.624s ... 0.611s
nvme/021 (tr=loop bd=device) (test NVMe list command) [passed]
runtime 0.633s ... 0.639s
nvme/021 (tr=loop bd=file) (test NVMe list command) [passed]
runtime 0.647s ... 0.632s
nvme/022 (tr=loop bd=device) (test NVMe reset command) [passed]
runtime 1.008s ... 1.011s
nvme/022 (tr=loop bd=file) (test NVMe reset command) [passed]
runtime 1.010s ... 1.006s
nvme/023 (tr=loop bd=device) (test NVMe smart-log command) [passed]
runtime 0.628s ... 0.623s
nvme/023 (tr=loop bd=file) (test NVMe smart-log command) [passed]
runtime 0.624s ... 0.627s
nvme/025 (tr=loop bd=device) (test NVMe effects-log) [passed]
runtime 0.647s ... 0.647s
nvme/025 (tr=loop bd=file) (test NVMe effects-log) [passed]
runtime 0.638s ... 0.636s
nvme/026 (tr=loop bd=device) (test NVMe ns-descs) [passed]
runtime 0.628s ... 0.647s
nvme/026 (tr=loop bd=file) (test NVMe ns-descs) [passed]
runtime 0.623s ... 0.637s
nvme/027 (tr=loop bd=device) (test NVMe ns-rescan command) [passed]
runtime 0.657s ... 0.658s
nvme/027 (tr=loop bd=file) (test NVMe ns-rescan command) [passed]
runtime 0.650s ... 0.641s
nvme/028 (tr=loop bd=device) (test NVMe list-subsys) [passed]
runtime 0.618s ... 0.642s
nvme/028 (tr=loop bd=file) (test NVMe list-subsys) [passed]
runtime 0.631s ... 0.627s
nvme/029 (tr=loop) (test userspace IO via nvme-cli read/write interface) [passed]
runtime 0.954s ... 0.952s
nvme/030 (tr=loop) (ensure the discovery generation counter is updated appropriately) [passed]
runtime 0.432s ... 0.437s
nvme/031 (tr=loop) (test deletion of NVMeOF controllers immediately after setup) [passed]
runtime 5.813s ... 5.999s
nvme/038 (tr=loop) (test deletion of NVMeOF subsystem without enabling) [passed]
runtime 0.035s ... 0.034s
nvme/040 (tr=loop) (test nvme fabrics controller reset/disconnect operation during I/O) [passed]
runtime 7.029s ... 7.022s
nvme/041 (tr=loop) (Create authenticated connections) [passed]
runtime 0.654s ... 0.684s
nvme/042 (tr=loop) (Test dhchap key types for authenticated connections) [passed]
runtime 3.866s ... 3.856s
nvme/043 (tr=loop) (Test hash and DH group variations for authenticated connections) [passed]
runtime 4.834s ... 4.886s
nvme/044 (tr=loop) (Test bi-directional authentication) [passed]
runtime 1.237s ... 1.328s
nvme/045 (tr=loop) (Test re-authentication) [passed]
runtime 1.570s ... 1.609s
nvme/047 (tr=loop) (test different queue types for fabric transports) [not run]
nvme_trtype=loop is not supported in this test
nvme/048 (tr=loop) (Test queue count changes on reconnect) [not run]
nvme_trtype=loop is not supported in this test
nvme/051 (tr=loop) (test nvmet concurrent ns enable/disable) [passed]
runtime 1.362s ... 1.407s
nvme/052 (tr=loop) (Test file-ns creation/deletion under one subsystem) [passed]
runtime 6.258s ... 6.282s
nvme/054 (tr=loop) (Test the NVMe reservation feature) [passed]
runtime 0.754s ... 0.769s
nvme/055 (tr=loop) (Test nvme write to a loop target ns just after ns is disabled) [not run]
kernel option DEBUG_ATOMIC_SLEEP has not been enabled
nvme/056 (tr=loop) (enable zero copy offload and run rw traffic) [not run]
Remote target required but NVME_TARGET_CONTROL is not set
nvme_trtype=loop is not supported in this test
kernel option ULP_DDP has not been enabled
module nvme_tcp does not have parameter ddp_offload
KERNELSRC not set
Kernel sources do not have tools/net/ynl/cli.py
NVME_IFACE not set
nvme/057 (tr=loop) (test nvme fabrics controller ANA failover during I/O) [passed]
runtime 27.227s ... 27.146s
nvme/058 (tr=loop) (test rapid namespace remapping) [passed]
runtime 4.646s ... 4.746s
nvme/060 (tr=loop) (test nvme fabrics target reset) [not run]
nvme_trtype=loop is not supported in this test
nvme/061 (tr=loop) (test fabric target teardown and setup during I/O) [not run]
nvme_trtype=loop is not supported in this test
nvme/062 (tr=loop) (Create TLS-encrypted connections) [not run]
nvme_trtype=loop is not supported in this test
nvme/063 (tr=loop) (Create authenticated TCP connections with secure concatenation) [not run]
nvme_trtype=loop is not supported in this test
nvme/065 (tr=loop) (test unmap write zeroes sysfs interface with nvmet devices) [passed]
runtime 2.310s ... 2.270s
++ for t in loop tcp
++ echo '################NVMET_TRTYPES=tcp############'
################NVMET_TRTYPES=tcp############
++ NVMET_TRTYPES=tcp
++ ./check nvme
nvme/002 (tr=tcp) (create many subsystems and test discovery) [not run]
nvme_trtype=tcp is not supported in this test
nvme/003 (tr=tcp) (test if we're sending keep-alives to a discovery controller) [passed]
runtime 10.279s ... 10.271s
nvme/004 (tr=tcp) (test nvme and nvmet UUID NS descriptors) [passed]
runtime 0.350s ... 0.404s
nvme/005 (tr=tcp) (reset local loopback target) [passed]
runtime 0.434s ... 0.466s
nvme/006 (tr=tcp bd=device) (create an NVMeOF target) [passed]
runtime 0.097s ... 0.098s
nvme/006 (tr=tcp bd=file) (create an NVMeOF target) [passed]
runtime 0.089s ... 0.094s
nvme/008 (tr=tcp bd=device) (create an NVMeOF host) [passed]
runtime 0.348s ... 0.361s
nvme/008 (tr=tcp bd=file) (create an NVMeOF host) [passed]
runtime 0.370s ... 0.395s
nvme/010 (tr=tcp bd=device) (run data verification fio job) [passed]
runtime 76.430s ... 78.689s
nvme/010 (tr=tcp bd=file) (run data verification fio job) [passed]
runtime 120.407s ... 116.963s
nvme/012 (tr=tcp bd=device) (run mkfs and data verification fio) [passed]
runtime 85.918s ... 84.974s
nvme/012 (tr=tcp bd=file) (run mkfs and data verification fio) [passed]
runtime 120.326s ... 120.123s
nvme/014 (tr=tcp bd=device) (flush a command from host) [passed]
runtime 9.603s ... 9.522s
nvme/014 (tr=tcp bd=file) (flush a command from host) [passed]
runtime 9.319s ... 9.297s
nvme/016 (tr=tcp) (create/delete many NVMeOF block device-backed ns and test discovery) [not run]
nvme_trtype=tcp is not supported in this test
nvme/017 (tr=tcp) (create/delete many file-ns and test discovery) [not run]
nvme_trtype=tcp is not supported in this test
nvme/018 (tr=tcp) (unit test NVMe-oF out of range access on a file backend) [passed]
runtime 0.334s ... 0.384s
nvme/019 (tr=tcp bd=device) (test NVMe DSM Discard command) [passed]
runtime 0.342s ... 0.377s
nvme/019 (tr=tcp bd=file) (test NVMe DSM Discard command) [passed]
runtime 0.334s ... 0.368s
nvme/021 (tr=tcp bd=device) (test NVMe list command) [passed]
runtime 0.357s ... 0.400s
nvme/021 (tr=tcp bd=file) (test NVMe list command) [passed]
runtime 0.356s ... 0.397s
nvme/022 (tr=tcp bd=device) (test NVMe reset command) [passed]
runtime 0.469s ... 0.504s
nvme/022 (tr=tcp bd=file) (test NVMe reset command) [passed]
runtime 0.441s ... 0.512s
nvme/023 (tr=tcp bd=device) (test NVMe smart-log command) [passed]
runtime 0.348s ... 0.387s
nvme/023 (tr=tcp bd=file) (test NVMe smart-log command) [passed]
runtime 0.330s ... 0.372s
nvme/025 (tr=tcp bd=device) (test NVMe effects-log) [passed]
runtime 0.351s ... 0.384s
nvme/025 (tr=tcp bd=file) (test NVMe effects-log) [passed]
runtime 0.362s ... 0.377s
nvme/026 (tr=tcp bd=device) (test NVMe ns-descs) [passed]
runtime 0.343s ... 0.359s
nvme/026 (tr=tcp bd=file) (test NVMe ns-descs) [passed]
runtime 0.317s ... 0.361s
nvme/027 (tr=tcp bd=device) (test NVMe ns-rescan command) [passed]
runtime 0.375s ... 0.412s
nvme/027 (tr=tcp bd=file) (test NVMe ns-rescan command) [passed]
runtime 0.388s ... 0.392s
nvme/028 (tr=tcp bd=device) (test NVMe list-subsys) [passed]
runtime 0.355s ... 0.362s
nvme/028 (tr=tcp bd=file) (test NVMe list-subsys) [passed]
runtime 0.342s ... 0.371s
nvme/029 (tr=tcp) (test userspace IO via nvme-cli read/write interface) [passed]
runtime 0.698s ... 0.742s
nvme/030 (tr=tcp) (ensure the discovery generation counter is updated appropriately) [passed]
runtime 0.402s ... 0.416s
nvme/031 (tr=tcp) (test deletion of NVMeOF controllers immediately after setup) [passed]
runtime 3.068s ... 3.161s
nvme/038 (tr=tcp) (test deletion of NVMeOF subsystem without enabling) [passed]
runtime 0.041s ... 0.042s
nvme/040 (tr=tcp) (test nvme fabrics controller reset/disconnect operation during I/O) [passed]
runtime 6.442s ... 6.478s
nvme/041 (tr=tcp) (Create authenticated connections) [passed]
runtime 0.408s ... 0.407s
nvme/042 (tr=tcp) (Test dhchap key types for authenticated connections) [passed]
runtime 1.803s ... 1.879s
nvme/043 (tr=tcp) (Test hash and DH group variations for authenticated connections) [passed]
runtime 2.484s ... 2.515s
nvme/044 (tr=tcp) (Test bi-directional authentication) [passed]
runtime 0.732s ... 0.732s
nvme/045 (tr=tcp) (Test re-authentication) [passed]
runtime 1.322s ... 1.310s
nvme/047 (tr=tcp) (test different queue types for fabric transports) [passed]
runtime 1.863s ... 1.879s
nvme/048 (tr=tcp) (Test queue count changes on reconnect) [passed]
runtime 6.525s ... 5.523s
nvme/051 (tr=tcp) (test nvmet concurrent ns enable/disable) [passed]
runtime 1.336s ... 1.361s
nvme/052 (tr=tcp) (Test file-ns creation/deletion under one subsystem) [not run]
nvme_trtype=tcp is not supported in this test
nvme/054 (tr=tcp) (Test the NVMe reservation feature) [passed]
runtime 0.493s ... 0.491s
nvme/055 (tr=tcp) (Test nvme write to a loop target ns just after ns is disabled) [not run]
nvme_trtype=tcp is not supported in this test
kernel option DEBUG_ATOMIC_SLEEP has not been enabled
nvme/056 (tr=tcp) (enable zero copy offload and run rw traffic) [not run]
Remote target required but NVME_TARGET_CONTROL is not set
kernel option ULP_DDP has not been enabled
module nvme_tcp does not have parameter ddp_offload
KERNELSRC not set
Kernel sources do not have tools/net/ynl/cli.py
NVME_IFACE not set
nvme/057 (tr=tcp) (test nvme fabrics controller ANA failover during I/O) [passed]
runtime 25.906s ... 25.963s
nvme/058 (tr=tcp) (test rapid namespace remapping) [passed]
runtime 3.145s ... 3.126s
nvme/060 (tr=tcp) (test nvme fabrics target reset) [passed]
runtime 20.395s ... 19.399s
nvme/061 (tr=tcp) (test fabric target teardown and setup during I/O) [passed]
runtime 8.549s ... 8.617s
nvme/062 (tr=tcp) (Create TLS-encrypted connections) [failed]
runtime 1.608s ... 5.200s
--- tests/nvme/062.out 2026-01-28 12:04:48.888356244 -0800
+++ /mnt/sda/blktests/results/nodev_tr_tcp/nvme/062.out.bad 2026-05-13 11:33:33.891029529 -0700
@@ -2,9 +2,13 @@
Test unencrypted connection w/ tls not required
disconnected 1 controller(s)
Test encrypted connection w/ tls not required
-disconnected 1 controller(s)
+FAIL: nvme connect return error code
+WARNING: connection is not encrypted
+disconnected 0 controller(s)
...
(Run 'diff -u tests/nvme/062.out /mnt/sda/blktests/results/nodev_tr_tcp/nvme/062.out.bad' to see the entire diff)
nvme/063 (tr=tcp) (Create authenticated TCP connections with secure concatenation) [passed]
runtime 1.838s ... 2.011s
nvme/065 (tr=tcp) (test unmap write zeroes sysfs interface with nvmet devices) [passed]
runtime 1.749s ... 1.758s
++ ./manage-rdma-nvme.sh --cleanup
====== RDMA NVMe Setup ======
RDMA Type: siw
Interface: auto-detect
[INFO] Checking prerequisites...
[INFO] Prerequisites check passed
[INFO] Loading RDMA module: siw
[INFO] Module siw loaded successfully
[INFO] Creating RDMA links...
[INFO] Creating RDMA link: ens5_siw
[INFO] Created RDMA link: ens5_siw -> ens5
++ ./manage-rdma-nvme.sh --status
====== RDMA Configuration Status ======
====== RDMA Network Configuration Status ======
Loaded Modules:
siw 217088
nvmet 258048
RDMA Links:
link ens5_siw/1 state ACTIVE physical_state LINK_UP netdev ens5
Network Interfaces (RDMA-capable):
Interface: ens5
IPv4: 192.168.0.46
IPv6: fe80::5054:98ff:fe76:5440%ens5
blktests Configuration:
Transport Address: 192.168.0.46:4420
Transport Type: rdma
Command: NVMET_TRTYPES=rdma ./check nvme/
NVMe RDMA Controllers:
None
=================================================
++ echo '################NVMET_TRTYPES=rdma############'
################NVMET_TRTYPES=rdma############
++ NVMET_TRTYPES=rdma
++ ./check nvme
nvme/002 (tr=rdma) (create many subsystems and test discovery) [not run]
nvme_trtype=rdma is not supported in this test
nvme/003 (tr=rdma) (test if we're sending keep-alives to a discovery controller) [passed]
runtime 10.315s ... 10.293s
nvme/004 (tr=rdma) (test nvme and nvmet UUID NS descriptors) [passed]
runtime 0.695s ... 0.750s
nvme/005 (tr=rdma) (reset local loopback target) [passed]
runtime 0.974s ... 1.046s
nvme/006 (tr=rdma bd=device) (create an NVMeOF target) [passed]
runtime 0.136s ... 0.149s
nvme/006 (tr=rdma bd=file) (create an NVMeOF target) [passed]
runtime 0.127s ... 0.140s
nvme/008 (tr=rdma bd=device) (create an NVMeOF host) [passed]
runtime 0.684s ... 0.685s
nvme/008 (tr=rdma bd=file) (create an NVMeOF host) [passed]
runtime 0.672s ... 0.729s
nvme/010 (tr=rdma bd=device) (run data verification fio job) [passed]
runtime 35.255s ... 35.280s
nvme/010 (tr=rdma bd=file) (run data verification fio job) [passed]
runtime 68.570s ... 66.260s
nvme/012 (tr=rdma bd=device) (run mkfs and data verification fio) [passed]
runtime 47.482s ... 42.242s
nvme/012 (tr=rdma bd=file) (run mkfs and data verification fio) [passed]
runtime 61.407s ... 62.511s
nvme/014 (tr=rdma bd=device) (flush a command from host) [passed]
runtime 9.855s ... 9.321s
nvme/014 (tr=rdma bd=file) (flush a command from host) [passed]
runtime 9.919s ... 9.175s
nvme/016 (tr=rdma) (create/delete many NVMeOF block device-backed ns and test discovery) [not run]
nvme_trtype=rdma is not supported in this test
nvme/017 (tr=rdma) (create/delete many file-ns and test discovery) [not run]
nvme_trtype=rdma is not supported in this test
nvme/018 (tr=rdma) (unit test NVMe-oF out of range access on a file backend) [passed]
runtime 0.654s ... 0.737s
nvme/019 (tr=rdma bd=device) (test NVMe DSM Discard command) [passed]
runtime 0.664s ... 0.738s
nvme/019 (tr=rdma bd=file) (test NVMe DSM Discard command) [passed]
runtime 0.649s ... 0.730s
nvme/021 (tr=rdma bd=device) (test NVMe list command) [passed]
runtime 0.676s ... 0.729s
nvme/021 (tr=rdma bd=file) (test NVMe list command) [passed]
runtime 0.673s ... 0.720s
nvme/022 (tr=rdma bd=device) (test NVMe reset command) [passed]
runtime 1.021s ... 1.066s
nvme/022 (tr=rdma bd=file) (test NVMe reset command) [passed]
runtime 1.015s ... 1.055s
nvme/023 (tr=rdma bd=device) (test NVMe smart-log command) [passed]
runtime 0.633s ... 0.718s
nvme/023 (tr=rdma bd=file) (test NVMe smart-log command) [passed]
runtime 0.656s ... 0.673s
nvme/025 (tr=rdma bd=device) (test NVMe effects-log) [passed]
runtime 0.706s ... 0.715s
nvme/025 (tr=rdma bd=file) (test NVMe effects-log) [passed]
runtime 0.688s ... 0.733s
nvme/026 (tr=rdma bd=device) (test NVMe ns-descs) [passed]
runtime 0.686s ... 0.690s
nvme/026 (tr=rdma bd=file) (test NVMe ns-descs) [passed]
runtime 0.669s ... 0.684s
nvme/027 (tr=rdma bd=device) (test NVMe ns-rescan command) [passed]
runtime 0.703s ... 0.731s
nvme/027 (tr=rdma bd=file) (test NVMe ns-rescan command) [passed]
runtime 0.707s ... 0.772s
nvme/028 (tr=rdma bd=device) (test NVMe list-subsys) [passed]
runtime 0.680s ... 0.712s
nvme/028 (tr=rdma bd=file) (test NVMe list-subsys) [passed]
runtime 0.670s ... 0.709s
nvme/029 (tr=rdma) (test userspace IO via nvme-cli read/write interface) [passed]
runtime 1.077s ... 1.087s
nvme/030 (tr=rdma) (ensure the discovery generation counter is updated appropriately) [passed]
runtime 0.511s ... 0.560s
nvme/031 (tr=rdma) (test deletion of NVMeOF controllers immediately after setup) [passed]
runtime 5.869s ... 6.106s
nvme/038 (tr=rdma) (test deletion of NVMeOF subsystem without enabling) [passed]
runtime 0.084s ... 0.085s
nvme/040 (tr=rdma) (test nvme fabrics controller reset/disconnect operation during I/O) [passed]
runtime 7.024s ... 6.994s
nvme/041 (tr=rdma) (Create authenticated connections) [passed]
runtime 0.718s ... 0.769s
nvme/042 (tr=rdma) (Test dhchap key types for authenticated connections) [passed]
runtime 3.759s ... 3.968s
nvme/043 (tr=rdma) (Test hash and DH group variations for authenticated connections) [passed]
runtime 4.487s ... 4.880s
nvme/044 (tr=rdma) (Test bi-directional authentication) [passed]
runtime 1.279s ... 1.393s
nvme/045 (tr=rdma) (Test re-authentication) [passed]
runtime 1.837s ... 1.892s
nvme/047 (tr=rdma) (test different queue types for fabric transports) [passed]
runtime 2.642s ... 2.700s
nvme/048 (tr=rdma) (Test queue count changes on reconnect) [passed]
runtime 5.810s ... 6.860s
nvme/051 (tr=rdma) (test nvmet concurrent ns enable/disable) [passed]
runtime 1.479s ... 1.408s
nvme/052 (tr=rdma) (Test file-ns creation/deletion under one subsystem) [not run]
nvme_trtype=rdma is not supported in this test
nvme/054 (tr=rdma) (Test the NVMe reservation feature) [passed]
runtime 0.799s ... 0.854s
nvme/055 (tr=rdma) (Test nvme write to a loop target ns just after ns is disabled) [not run]
nvme_trtype=rdma is not supported in this test
kernel option DEBUG_ATOMIC_SLEEP has not been enabled
nvme/056 (tr=rdma) (enable zero copy offload and run rw traffic) [not run]
Remote target required but NVME_TARGET_CONTROL is not set
nvme_trtype=rdma is not supported in this test
kernel option ULP_DDP has not been enabled
module nvme_tcp does not have parameter ddp_offload
KERNELSRC not set
Kernel sources do not have tools/net/ynl/cli.py
NVME_IFACE not set
nvme/057 (tr=rdma) (test nvme fabrics controller ANA failover during I/O) [passed]
runtime 26.984s ... 27.074s
nvme/058 (tr=rdma) (test rapid namespace remapping) [passed]
runtime 4.444s ... 4.553s
nvme/060 (tr=rdma) (test nvme fabrics target reset) [passed]
runtime 20.696s ... 22.308s
nvme/061 (tr=rdma) (test fabric target teardown and setup during I/O) [passed]
runtime 15.375s ... 15.348s
nvme/062 (tr=rdma) (Create TLS-encrypted connections) [not run]
nvme_trtype=rdma is not supported in this test
nvme/063 (tr=rdma) (Create authenticated TCP connections with secure concatenation) [not run]
nvme_trtype=rdma is not supported in this test
nvme/065 (tr=rdma) (test unmap write zeroes sysfs interface with nvmet devices) [passed]
runtime 2.369s ... 2.466s
++ ./manage-rdma-nvme.sh --cleanup
====== RDMA NVMe Cleanup ======
[INFO] Disconnecting NVMe RDMA controllers...
[INFO] No NVMe RDMA controllers to disconnect
[INFO] Removing RDMA links...
[INFO] No RDMA links to remove
[INFO] Unloading NVMe RDMA modules...
[INFO] Unloading module: nvme_rdma
[INFO] Module nvme_rdma unloaded
[INFO] Unloading module: nvmet_rdma
[INFO] Module nvmet_rdma unloaded
[INFO] Unloading module: nvmet
--
2.39.5
^ permalink raw reply
* Re: [PATCH 01/19] btrfs: require at least 4 devices for RAID 6
From: H. Peter Anvin @ 2026-05-13 16:14 UTC (permalink / raw)
To: Christoph Hellwig, Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, Herbert Xu, Dan Williams, Chris Mason,
David Sterba, Arnd Bergmann, Song Liu, Yu Kuai, Li Nan,
linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-2-hch@lst.de>
On May 11, 2026 10:20:41 PM PDT, Christoph Hellwig <hch@lst.de> wrote:
>While the RAID6 algorithm could in theory support 3 devices by just
>copying the data disk to the two parity disks, this version is not only
>useless because it is a suboptimal version of 3-way mirroring, but also
>broken with various crashes and incorrect parity generation in various
>architecture-optimized implementations. Disallow it similar to mdraid
>which requires at least 4 devices for RAID 6.
>
>Fixes: 53b381b3abeb ("Btrfs: RAID5 and RAID6")
>Signed-off-by: Christoph Hellwig <hch@lst.de>
>---
> fs/btrfs/volumes.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
>index a88e68f90564..0b54b97bdad8 100644
>--- a/fs/btrfs/volumes.c
>+++ b/fs/btrfs/volumes.c
>@@ -159,7 +159,7 @@ const struct btrfs_raid_attr btrfs_raid_array[BTRFS_NR_RAID_TYPES] = {
> .sub_stripes = 1,
> .dev_stripes = 1,
> .devs_max = 0,
>- .devs_min = 3,
>+ .devs_min = 4,
> .tolerated_failures = 2,
> .devs_increment = 1,
> .ncopies = 1,
Yes, if anyone cares about < 4 disks for the RAID-6 case (or < 3 for the RAID-4/5 case), just use the RAID-1 code.
^ permalink raw reply
* Re: [PATCH 01/19] btrfs: require at least 4 devices for RAID 6
From: Christoph Hellwig @ 2026-05-13 5:47 UTC (permalink / raw)
To: David Sterba
Cc: Christoph Hellwig, Andrew Morton, Catalin Marinas, Will Deacon,
Ard Biesheuvel, Huacai Chen, WANG Xuerui, Madhavan Srinivasan,
Michael Ellerman, Nicholas Piggin, Christophe Leroy (CS GROUP),
Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger, Sven Schnelle, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin,
Herbert Xu, Dan Williams, Chris Mason, David Sterba,
Arnd Bergmann, Song Liu, Yu Kuai, Li Nan, linux-kernel,
linux-arm-kernel, loongarch, linuxppc-dev, linux-riscv,
linux-s390, linux-crypto, linux-btrfs, linux-arch, linux-raid
In-Reply-To: <20260512114231.GG2558453@suse.cz>
On Tue, May 12, 2026 at 01:42:31PM +0200, David Sterba wrote:
> On Tue, May 12, 2026 at 07:20:41AM +0200, Christoph Hellwig wrote:
> > While the RAID6 algorithm could in theory support 3 devices by just
> > copying the data disk to the two parity disks, this version is not only
> > useless because it is a suboptimal version of 3-way mirroring, but also
> > broken with various crashes and incorrect parity generation in various
> > architecture-optimized implementations. Disallow it similar to mdraid
> > which requires at least 4 devices for RAID 6.
> >
> > Fixes: 53b381b3abeb ("Btrfs: RAID5 and RAID6")
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
>
> This patch should have been sent separately as it has user visible
> impact and can potentially break some setups.
It _is_ sent out separate.
> The degenerate modes of
> raid0, 5, or 6 are explicit as a possible middle step when converting
> profiles. We can use a fallback implementation for this case if the
> accelerated implementations cannot do it.
This is not about a degenerated mode. For a degenerated RAID 6, parity
generation uses the RAID 5 XOR routines as the second parity will be
missing. This is about generating two parities for a single data disk,
which must be explicitly selected.
^ permalink raw reply
* Re: [PATCH 01/19] btrfs: require at least 4 devices for RAID 6
From: David Sterba @ 2026-05-12 11:42 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Andrew Morton, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Huacai Chen, WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-2-hch@lst.de>
On Tue, May 12, 2026 at 07:20:41AM +0200, Christoph Hellwig wrote:
> While the RAID6 algorithm could in theory support 3 devices by just
> copying the data disk to the two parity disks, this version is not only
> useless because it is a suboptimal version of 3-way mirroring, but also
> broken with various crashes and incorrect parity generation in various
> architecture-optimized implementations. Disallow it similar to mdraid
> which requires at least 4 devices for RAID 6.
>
> Fixes: 53b381b3abeb ("Btrfs: RAID5 and RAID6")
> Signed-off-by: Christoph Hellwig <hch@lst.de>
This patch should have been sent separately as it has user visible
impact and can potentially break some setups. The degenerate modes of
raid0, 5, or 6 are explicit as a possible middle step when converting
profiles. We can use a fallback implementation for this case if the
accelerated implementations cannot do it.
^ permalink raw reply
* Re: cleanup the RAID6 P/Q library v2
From: Ard Biesheuvel @ 2026-05-12 9:50 UTC (permalink / raw)
To: Christoph Hellwig, Andrew Morton
Cc: Catalin Marinas, Will Deacon, Huacai Chen, WANG Xuerui,
Madhavan Srinivasan, Michael Ellerman, Nicholas Piggin,
Christophe Leroy (CS GROUP), Paul Walmsley, Palmer Dabbelt,
Albert Ou, Alexandre Ghiti, Heiko Carstens, Vasily Gorbik,
Alexander Gordeev, Christian Borntraeger, Sven Schnelle,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86,
H . Peter Anvin, Herbert Xu, Dan Williams, Chris Mason,
David Sterba, Arnd Bergmann, Song Liu, Yu Kuai, Li Nan,
linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
On Tue, 12 May 2026, at 07:20, Christoph Hellwig wrote:
> Hi all,
>
> this series cleans up the RAID6 P/Q library to match the recent updates
> to the RAID 5 XOR library and other CRC/crypto libraries. This includes
> providing properly documented external interfaces, hiding the internals,
> using static_call instead of indirect calls and turning the user space
> test suite into an in-kernel kunit test which is also extended to
> improve coverage.
>
> Note that this changes registration so that non-priority algorithms are
> not registered, which greatly helps with the benchmark time at boot time.
> I'd like to encourage all architecture maintainers to see if they can
> further optimized this by registering as few as possible algorithms when
> there is a clear benefit in optimized or more unrolled implementations.
>
> This series sits on top of the "cleanup the RAID5 XOR library v3" series.
>
> A git tree is also available here:
>
> git://git.infradead.org/users/hch/misc.git lib-raid6
>
> Gitweb:
>
>
> https://git.infradead.org/?p=users/hch/misc.git;a=shortlog;h=refs/heads/lib-raid6
>
> Changes since v1:
> - fix arm64 objdir != srcdir builds
> - call the kunit module raid6_kunit.ko from the beginning
> - update MAINTAINERS
> - don't require preemptible context and apply the same restrictions as
> the merged version of the XOR API
> - fix the arm64 default in Kconfig
> - pick the last registered (and presumably most optimized) algorithm when
> benchmarking is disabled
> - port over the randomization fixes from the XOR series
> - misc other kunit cleanups
> - require at least 4 devices for RAID6 to skip broken special cases
>
Tested-by: Ard Biesheuvel <ardb@kernel.org> # kunit only on arm64
Acked-by: Ard Biesheuvel <ardb@kernel.org>
^ permalink raw reply
* [PATCH 19/19] raid6_kunit: randomize buffer alignment
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Add code to add random alignment to the buffers to test the case where
they are not page aligned, and to move the buffers to the end of the
allocation so that they are next to the vmalloc guard page.
This does not include the recovery buffers as the recovery requires
page alignment.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/tests/raid6_kunit.c | 41 +++++++++++++++++++++++++-----
1 file changed, 35 insertions(+), 6 deletions(-)
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index d6ac777dcaee..7b45c7be36fc 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -21,6 +21,7 @@ MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING");
static struct rnd_state rng;
static void *test_buffers[RAID6_KUNIT_MAX_BUFFERS];
+static void *aligned_buffers[RAID6_KUNIT_MAX_BUFFERS];
static void *test_recov_buffers[RAID6_KUNIT_MAX_FAILURES];
static size_t test_buflen;
@@ -50,6 +51,14 @@ static unsigned int random_nr_buffers(void)
RAID6_MIN_DISKS;
}
+/* Generate a random alignment that is a multiple of 64. */
+static unsigned int random_alignment(unsigned int max_alignment)
+{
+ if (max_alignment == 0)
+ return 0;
+ return (rand32() % (max_alignment + 1)) & ~63;
+}
+
static void makedata(int start, int stop)
{
int i;
@@ -80,7 +89,7 @@ static void test_recover_one(struct kunit *test, unsigned int nr_buffers,
for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++)
memset(test_recov_buffers[i], 0xf0, test_buflen);
- memcpy(dataptrs, test_buffers, sizeof(dataptrs));
+ memcpy(dataptrs, aligned_buffers, sizeof(dataptrs));
dataptrs[faila] = test_recov_buffers[0];
dataptrs[failb] = test_recov_buffers[1];
@@ -102,13 +111,13 @@ static void test_recover_one(struct kunit *test, unsigned int nr_buffers,
ta->recov->data2(nr_buffers, len, faila, failb, dataptrs);
}
- KUNIT_EXPECT_MEMEQ_MSG(test, test_buffers[faila], test_recov_buffers[0],
+ KUNIT_EXPECT_MEMEQ_MSG(test, aligned_buffers[faila], dataptrs[faila],
len,
"faila miscompared: %3d[%c] buffers %u len %u (failb=%3d[%c])\n",
faila, member_type(nr_buffers, faila),
nr_buffers, len,
failb, member_type(nr_buffers, failb));
- KUNIT_EXPECT_MEMEQ_MSG(test, test_buffers[failb], test_recov_buffers[1],
+ KUNIT_EXPECT_MEMEQ_MSG(test, aligned_buffers[failb], dataptrs[failb],
len,
"failb miscompared: %3d[%c] buffers %u len %u (faila=%3d[%c])\n",
failb, member_type(nr_buffers, failb),
@@ -152,9 +161,9 @@ static void test_rmw_one(struct kunit *test, unsigned int nr_buffers,
{
const struct test_args *ta = test->param_value;
- ta->gen->xor_syndrome(nr_buffers, p1, p2, len, test_buffers);
+ ta->gen->xor_syndrome(nr_buffers, p1, p2, len, aligned_buffers);
makedata(p1, p2);
- ta->gen->xor_syndrome(nr_buffers, p1, p2, len, test_buffers);
+ ta->gen->xor_syndrome(nr_buffers, p1, p2, len, aligned_buffers);
test_recover(test, nr_buffers, len);
}
@@ -178,13 +187,33 @@ static void raid6_test_one(struct kunit *test)
const struct test_args *ta = test->param_value;
unsigned int nr_buffers = random_nr_buffers();
unsigned int len = random_length(RAID6_KUNIT_MAX_BYTES);
+ unsigned int max_alignment;
+ int i;
/* Nuke syndromes */
memset(test_buffers[nr_buffers - 2], 0xee, test_buflen);
memset(test_buffers[nr_buffers - 1], 0xee, test_buflen);
+ /*
+ * If we're not using the entire buffer size, inject randomize alignment
+ * into the buffer.
+ */
+ max_alignment = RAID6_KUNIT_MAX_BYTES - len;
+ if (rand32() % 2 == 0) {
+ /* Use random alignments mod 64 */
+ for (i = 0; i < nr_buffers; i++)
+ aligned_buffers[i] = test_buffers[i] +
+ random_alignment(max_alignment);
+ } else {
+ /* Go up to the guard page, to catch buffer overreads */
+ unsigned int align = test_buflen - len;
+
+ for (i = 0; i < nr_buffers; i++)
+ aligned_buffers[i] = test_buffers[i] + align;
+ }
+
/* Generate assumed good syndrome */
- ta->gen->gen_syndrome(nr_buffers, len, test_buffers);
+ ta->gen->gen_syndrome(nr_buffers, len, aligned_buffers);
test_recover(test, nr_buffers, len);
--
2.53.0
^ permalink raw reply related
* [PATCH 18/19] raid6_kunit: randomize parameters and increase limits
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
The current test has double-quadratic behavior in the selection for
the updated ("XORed") disks, and in the selection of updated pointers,
which makes scaling it to more tests difficult. At the same time it
only ever tests with the maximum number of disks, which leaves a
coverage hole for smaller ones.
Fix this by randomizing the total number, failed disks and regions
to update, and increasing the upper number of tests disks.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/tests/raid6_kunit.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index 775a0051f9a4..d6ac777dcaee 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -8,6 +8,7 @@
#include <kunit/test.h>
#include <linux/prandom.h>
#include <linux/vmalloc.h>
+#include <linux/raid/pq.h>
#include "../algos.h"
MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING");
@@ -43,6 +44,12 @@ static unsigned int random_length(unsigned int max_length)
return round_up((rand32() % max_length) + 1, 512);
}
+static unsigned int random_nr_buffers(void)
+{
+ return (rand32() % (RAID6_KUNIT_MAX_BUFFERS - (RAID6_MIN_DISKS - 1))) +
+ RAID6_MIN_DISKS;
+}
+
static void makedata(int start, int stop)
{
int i;
@@ -169,9 +176,7 @@ static void test_rmw(struct kunit *test, unsigned int nr_buffers,
static void raid6_test_one(struct kunit *test)
{
const struct test_args *ta = test->param_value;
- /* including P/Q we need at least three buffers */
- unsigned int nr_buffers =
- (rand32() % (RAID6_KUNIT_MAX_BUFFERS - 2)) + 3;
+ unsigned int nr_buffers = random_nr_buffers();
unsigned int len = random_length(RAID6_KUNIT_MAX_BYTES);
/* Nuke syndromes */
--
2.53.0
^ permalink raw reply related
* [PATCH 17/19] raid6_kunit: randomize parameters and increase limits
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
The current test has double-quadratic behavior in the selection for
the updated ("XORed") disks, and in the selection of updated pointers,
which makes scaling it to more tests difficult. At the same time it
only ever tests with the maximum number of disks, which leaves a
coverage hole for smaller ones.
Fix this by randomizing the total number, failed disks and regions
to update, and increasing the upper number of tests disks.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/tests/raid6_kunit.c | 184 ++++++++++++++++++++---------
1 file changed, 126 insertions(+), 58 deletions(-)
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index 28b4467977c5..775a0051f9a4 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -13,13 +13,15 @@
MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING");
#define RAID6_KUNIT_SEED 42
+#define RAID6_KUNIT_NUM_TEST_ITERS 10
+#define RAID6_KUNIT_MAX_BUFFERS 64 /* Including P and Q */
#define RAID6_KUNIT_MAX_FAILURES 2
-
-#define NDISKS 16 /* Including P and Q */
+#define RAID6_KUNIT_MAX_BYTES PAGE_SIZE
static struct rnd_state rng;
-static void *test_buffers[NDISKS];
+static void *test_buffers[RAID6_KUNIT_MAX_BUFFERS];
static void *test_recov_buffers[RAID6_KUNIT_MAX_FAILURES];
+static size_t test_buflen;
struct test_args {
unsigned int recov_idx;
@@ -30,102 +32,167 @@ struct test_args {
static struct test_args args;
+static u32 rand32(void)
+{
+ return prandom_u32_state(&rng);
+}
+
+/* Generate a random length that is a multiple of 512. */
+static unsigned int random_length(unsigned int max_length)
+{
+ return round_up((rand32() % max_length) + 1, 512);
+}
+
static void makedata(int start, int stop)
{
int i;
for (i = start; i <= stop; i++)
- prandom_bytes_state(&rng, test_buffers[i], PAGE_SIZE);
+ prandom_bytes_state(&rng, test_buffers[i], test_buflen);
}
-static char member_type(int d)
+static char member_type(unsigned int nr_buffers, int d)
{
- switch (d) {
- case NDISKS-2:
+ if (d == nr_buffers - 2)
return 'P';
- case NDISKS-1:
+ if (d == nr_buffers - 1)
return 'Q';
- default:
- return 'D';
- }
+ return 'D';
}
-static void test_recover(struct kunit *test, int faila, int failb)
+static void test_recover_one(struct kunit *test, unsigned int nr_buffers,
+ unsigned int len, int faila, int failb)
{
const struct test_args *ta = test->param_value;
- void *dataptrs[NDISKS];
+ void *dataptrs[RAID6_KUNIT_MAX_BUFFERS];
int i;
+ if (faila > failb)
+ swap(faila, failb);
+
for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++)
- memset(test_recov_buffers[i], 0xf0, PAGE_SIZE);
+ memset(test_recov_buffers[i], 0xf0, test_buflen);
memcpy(dataptrs, test_buffers, sizeof(dataptrs));
dataptrs[faila] = test_recov_buffers[0];
dataptrs[failb] = test_recov_buffers[1];
- if (failb == NDISKS - 1) {
+ if (failb == nr_buffers - 1) {
/*
* We don't implement the data+Q failure scenario, since it
* is equivalent to a RAID-5 failure (XOR, then recompute Q).
*/
- if (faila != NDISKS - 2)
+ if (WARN_ON_ONCE(faila != nr_buffers - 2))
return;
/* P+Q failure. Just rebuild the syndrome. */
- ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, dataptrs);
- } else if (failb == NDISKS - 2) {
+ ta->gen->gen_syndrome(nr_buffers, len, dataptrs);
+ } else if (failb == nr_buffers - 2) {
/* data+P failure. */
- ta->recov->datap(NDISKS, PAGE_SIZE, faila, dataptrs);
+ ta->recov->datap(nr_buffers, len, faila, dataptrs);
} else {
/* data+data failure. */
- ta->recov->data2(NDISKS, PAGE_SIZE, faila, failb, dataptrs);
+ ta->recov->data2(nr_buffers, len, faila, failb, dataptrs);
}
KUNIT_EXPECT_MEMEQ_MSG(test, test_buffers[faila], test_recov_buffers[0],
- PAGE_SIZE,
- "faila miscompared: %3d[%c] (failb=%3d[%c])\n",
- faila, member_type(faila),
- failb, member_type(failb));
+ len,
+ "faila miscompared: %3d[%c] buffers %u len %u (failb=%3d[%c])\n",
+ faila, member_type(nr_buffers, faila),
+ nr_buffers, len,
+ failb, member_type(nr_buffers, failb));
KUNIT_EXPECT_MEMEQ_MSG(test, test_buffers[failb], test_recov_buffers[1],
- PAGE_SIZE,
- "failb miscompared: %3d[%c] (faila=%3d[%c])\n",
- failb, member_type(failb),
- faila, member_type(faila));
+ len,
+ "failb miscompared: %3d[%c] buffers %u len %u (faila=%3d[%c])\n",
+ failb, member_type(nr_buffers, failb),
+ nr_buffers, len,
+ faila, member_type(nr_buffers, faila));
}
-static void raid6_test(struct kunit *test)
+static void test_recover(struct kunit *test, unsigned int nr_buffers,
+ unsigned int len)
+{
+ unsigned int nr_data = nr_buffers - 2;
+ int iterations, i;
+
+ /* Test P+Q recovery */
+ test_recover_one(test, nr_buffers, len, nr_data, nr_buffers - 1);
+
+ /* Test data+P recovery */
+ for (i = 0; i < nr_buffers - 2; i++)
+ test_recover_one(test, nr_buffers, len, i, nr_data);
+
+ /* Double data failure is impossible with a single data disk */
+ if (nr_data == 1)
+ return;
+
+ /* Test data+data recovery using random sampling */
+ iterations = nr_buffers * 2; /* should provide good enough coverage */
+ for (i = 0; i < iterations; i++) {
+ int faila = rand32() % nr_data, failb;
+
+ do {
+ failb = rand32() % nr_data;
+ } while (failb == faila);
+
+ test_recover_one(test, nr_buffers, len, faila, failb);
+ }
+}
+
+/* Simulate rmw run */
+static void test_rmw_one(struct kunit *test, unsigned int nr_buffers,
+ unsigned int len, int p1, int p2)
{
const struct test_args *ta = test->param_value;
- int i, j, p1, p2;
+
+ ta->gen->xor_syndrome(nr_buffers, p1, p2, len, test_buffers);
+ makedata(p1, p2);
+ ta->gen->xor_syndrome(nr_buffers, p1, p2, len, test_buffers);
+ test_recover(test, nr_buffers, len);
+}
+
+static void test_rmw(struct kunit *test, unsigned int nr_buffers,
+ unsigned int len)
+{
+ int iterations = nr_buffers / 2, i;
+
+ for (i = 0; i < iterations; i++) {
+ int p1 = rand32() % (nr_buffers - 2);
+ int p2 = rand32() % (nr_buffers - 2);
+
+ if (p2 < p1)
+ swap(p1, p2);
+ test_rmw_one(test, nr_buffers, len, p1, p2);
+ }
+}
+
+static void raid6_test_one(struct kunit *test)
+{
+ const struct test_args *ta = test->param_value;
+ /* including P/Q we need at least three buffers */
+ unsigned int nr_buffers =
+ (rand32() % (RAID6_KUNIT_MAX_BUFFERS - 2)) + 3;
+ unsigned int len = random_length(RAID6_KUNIT_MAX_BYTES);
/* Nuke syndromes */
- memset(test_buffers[NDISKS - 2], 0xee, PAGE_SIZE);
- memset(test_buffers[NDISKS - 1], 0xee, PAGE_SIZE);
+ memset(test_buffers[nr_buffers - 2], 0xee, test_buflen);
+ memset(test_buffers[nr_buffers - 1], 0xee, test_buflen);
/* Generate assumed good syndrome */
- ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, test_buffers);
+ ta->gen->gen_syndrome(nr_buffers, len, test_buffers);
- for (i = 0; i < NDISKS - 1; i++)
- for (j = i + 1; j < NDISKS; j++)
- test_recover(test, i, j);
+ test_recover(test, nr_buffers, len);
- if (!ta->gen->xor_syndrome)
- return;
+ if (ta->gen->xor_syndrome)
+ test_rmw(test, nr_buffers, len);
+}
- for (p1 = 0; p1 < NDISKS - 2; p1++) {
- for (p2 = p1; p2 < NDISKS - 2; p2++) {
- /* Simulate rmw run */
- ta->gen->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
- test_buffers);
- makedata(p1, p2);
- ta->gen->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
- test_buffers);
-
- for (i = 0; i < NDISKS - 1; i++)
- for (j = i + 1; j < NDISKS; j++)
- test_recover(test, i, j);
- }
- }
+static void raid6_test(struct kunit *test)
+{
+ int i;
+
+ for (i = 0; i < RAID6_KUNIT_NUM_TEST_ITERS; i++)
+ raid6_test_one(test);
}
static const void *raid6_gen_params(struct kunit *test, const void *prev,
@@ -169,23 +236,24 @@ static int raid6_suite_init(struct kunit_suite *suite)
* so that it is immediately followed by a guard page. This allows
* buffer overreads to be detected, even in assembly code.
*/
+ test_buflen = round_up(RAID6_KUNIT_MAX_BYTES, PAGE_SIZE);
for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++) {
- test_recov_buffers[i] = vmalloc(PAGE_SIZE);
+ test_recov_buffers[i] = vmalloc(test_buflen);
if (!test_recov_buffers[i])
goto out_free_recov_buffers;
}
- for (i = 0; i < NDISKS; i++) {
- test_buffers[i] = vmalloc(PAGE_SIZE);
+ for (i = 0; i < RAID6_KUNIT_MAX_BUFFERS; i++) {
+ test_buffers[i] = vmalloc(test_buflen);
if (!test_buffers[i])
goto out_free_buffers;
}
- makedata(0, NDISKS - 1);
+ makedata(0, RAID6_KUNIT_MAX_BUFFERS - 1);
return 0;
out_free_buffers:
- for (i = 0; i < NDISKS; i++)
+ for (i = 0; i < RAID6_KUNIT_MAX_BUFFERS; i++)
vfree(test_buffers[i]);
memset(test_buffers, 0, sizeof(test_buffers));
out_free_recov_buffers:
@@ -199,7 +267,7 @@ static void raid6_suite_exit(struct kunit_suite *suite)
{
int i;
- for (i = 0; i < NDISKS; i++)
+ for (i = 0; i < RAID6_KUNIT_MAX_BUFFERS; i++)
vfree(test_buffers[i]);
memset(test_buffers, 0, sizeof(test_buffers));
for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++)
--
2.53.0
^ permalink raw reply related
* [PATCH 16/19] raid6_kunit: cleanup dataptr handling
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Move the global dataptr array into test_recover() as all sites that fill
data or parity can use test_buffers directly, and this localized the
override for the failed slots to the recovery testing routine.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/tests/raid6_kunit.c | 19 +++++++------------
1 file changed, 7 insertions(+), 12 deletions(-)
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index a4b65ccc9d20..28b4467977c5 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -18,7 +18,6 @@ MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING");
#define NDISKS 16 /* Including P and Q */
static struct rnd_state rng;
-static void *dataptrs[NDISKS];
static void *test_buffers[NDISKS];
static void *test_recov_buffers[RAID6_KUNIT_MAX_FAILURES];
@@ -35,10 +34,8 @@ static void makedata(int start, int stop)
{
int i;
- for (i = start; i <= stop; i++) {
+ for (i = start; i <= stop; i++)
prandom_bytes_state(&rng, test_buffers[i], PAGE_SIZE);
- dataptrs[i] = test_buffers[i];
- }
}
static char member_type(int d)
@@ -56,11 +53,13 @@ static char member_type(int d)
static void test_recover(struct kunit *test, int faila, int failb)
{
const struct test_args *ta = test->param_value;
+ void *dataptrs[NDISKS];
int i;
for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++)
memset(test_recov_buffers[i], 0xf0, PAGE_SIZE);
+ memcpy(dataptrs, test_buffers, sizeof(dataptrs));
dataptrs[faila] = test_recov_buffers[0];
dataptrs[failb] = test_recov_buffers[1];
@@ -70,7 +69,7 @@ static void test_recover(struct kunit *test, int faila, int failb)
* is equivalent to a RAID-5 failure (XOR, then recompute Q).
*/
if (faila != NDISKS - 2)
- goto skip;
+ return;
/* P+Q failure. Just rebuild the syndrome. */
ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, dataptrs);
@@ -92,10 +91,6 @@ static void test_recover(struct kunit *test, int faila, int failb)
"failb miscompared: %3d[%c] (faila=%3d[%c])\n",
failb, member_type(failb),
faila, member_type(faila));
-
-skip:
- dataptrs[faila] = test_buffers[faila];
- dataptrs[failb] = test_buffers[failb];
}
static void raid6_test(struct kunit *test)
@@ -108,7 +103,7 @@ static void raid6_test(struct kunit *test)
memset(test_buffers[NDISKS - 1], 0xee, PAGE_SIZE);
/* Generate assumed good syndrome */
- ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, (void **)&dataptrs);
+ ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, test_buffers);
for (i = 0; i < NDISKS - 1; i++)
for (j = i + 1; j < NDISKS; j++)
@@ -121,10 +116,10 @@ static void raid6_test(struct kunit *test)
for (p2 = p1; p2 < NDISKS - 2; p2++) {
/* Simulate rmw run */
ta->gen->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
- (void **)&dataptrs);
+ test_buffers);
makedata(p1, p2);
ta->gen->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
- (void **)&dataptrs);
+ test_buffers);
for (i = 0; i < NDISKS - 1; i++)
for (j = i + 1; j < NDISKS; j++)
--
2.53.0
^ permalink raw reply related
* [PATCH 15/19] raid6_kunit: dynamically allocate data buffers using vmalloc
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Use vmalloc for the data buffers instead of using static .data allocations.
This provides for better out of bounds checking and avoids wasting kernel
memory after the test has run. vmalloc is used instead of kmalloc to
provide for better out of bounds access checking as in other kunit tests.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/tests/raid6_kunit.c | 77 ++++++++++++++++++++++++------
1 file changed, 62 insertions(+), 15 deletions(-)
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index f55b081b6b13..a4b65ccc9d20 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -7,19 +7,20 @@
#include <kunit/test.h>
#include <linux/prandom.h>
+#include <linux/vmalloc.h>
#include "../algos.h"
MODULE_IMPORT_NS("EXPORTED_FOR_KUNIT_TESTING");
#define RAID6_KUNIT_SEED 42
+#define RAID6_KUNIT_MAX_FAILURES 2
#define NDISKS 16 /* Including P and Q */
static struct rnd_state rng;
static void *dataptrs[NDISKS];
-static char data[NDISKS][PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
-static char recovi[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
-static char recovj[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
+static void *test_buffers[NDISKS];
+static void *test_recov_buffers[RAID6_KUNIT_MAX_FAILURES];
struct test_args {
unsigned int recov_idx;
@@ -35,8 +36,8 @@ static void makedata(int start, int stop)
int i;
for (i = start; i <= stop; i++) {
- prandom_bytes_state(&rng, data[i], PAGE_SIZE);
- dataptrs[i] = data[i];
+ prandom_bytes_state(&rng, test_buffers[i], PAGE_SIZE);
+ dataptrs[i] = test_buffers[i];
}
}
@@ -55,12 +56,13 @@ static char member_type(int d)
static void test_recover(struct kunit *test, int faila, int failb)
{
const struct test_args *ta = test->param_value;
+ int i;
- memset(recovi, 0xf0, PAGE_SIZE);
- memset(recovj, 0xba, PAGE_SIZE);
+ for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++)
+ memset(test_recov_buffers[i], 0xf0, PAGE_SIZE);
- dataptrs[faila] = recovi;
- dataptrs[failb] = recovj;
+ dataptrs[faila] = test_recov_buffers[0];
+ dataptrs[failb] = test_recov_buffers[1];
if (failb == NDISKS - 1) {
/*
@@ -80,18 +82,20 @@ static void test_recover(struct kunit *test, int faila, int failb)
ta->recov->data2(NDISKS, PAGE_SIZE, faila, failb, dataptrs);
}
- KUNIT_EXPECT_MEMEQ_MSG(test, data[faila], recovi, PAGE_SIZE,
+ KUNIT_EXPECT_MEMEQ_MSG(test, test_buffers[faila], test_recov_buffers[0],
+ PAGE_SIZE,
"faila miscompared: %3d[%c] (failb=%3d[%c])\n",
faila, member_type(faila),
failb, member_type(failb));
- KUNIT_EXPECT_MEMEQ_MSG(test, data[failb], recovj, PAGE_SIZE,
+ KUNIT_EXPECT_MEMEQ_MSG(test, test_buffers[failb], test_recov_buffers[1],
+ PAGE_SIZE,
"failb miscompared: %3d[%c] (faila=%3d[%c])\n",
failb, member_type(failb),
faila, member_type(faila));
skip:
- dataptrs[faila] = data[faila];
- dataptrs[failb] = data[failb];
+ dataptrs[faila] = test_buffers[faila];
+ dataptrs[failb] = test_buffers[failb];
}
static void raid6_test(struct kunit *test)
@@ -100,8 +104,8 @@ static void raid6_test(struct kunit *test)
int i, j, p1, p2;
/* Nuke syndromes */
- memset(data[NDISKS - 2], 0xee, PAGE_SIZE);
- memset(data[NDISKS - 1], 0xee, PAGE_SIZE);
+ memset(test_buffers[NDISKS - 2], 0xee, PAGE_SIZE);
+ memset(test_buffers[NDISKS - 1], 0xee, PAGE_SIZE);
/* Generate assumed good syndrome */
ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, (void **)&dataptrs);
@@ -161,15 +165,58 @@ static struct kunit_case raid6_test_cases[] = {
static int raid6_suite_init(struct kunit_suite *suite)
{
+ int i;
+
prandom_seed_state(&rng, RAID6_KUNIT_SEED);
+
+ /*
+ * Allocate the test buffer using vmalloc() with a page-aligned length
+ * so that it is immediately followed by a guard page. This allows
+ * buffer overreads to be detected, even in assembly code.
+ */
+ for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++) {
+ test_recov_buffers[i] = vmalloc(PAGE_SIZE);
+ if (!test_recov_buffers[i])
+ goto out_free_recov_buffers;
+ }
+ for (i = 0; i < NDISKS; i++) {
+ test_buffers[i] = vmalloc(PAGE_SIZE);
+ if (!test_buffers[i])
+ goto out_free_buffers;
+ }
+
makedata(0, NDISKS - 1);
+
return 0;
+
+out_free_buffers:
+ for (i = 0; i < NDISKS; i++)
+ vfree(test_buffers[i]);
+ memset(test_buffers, 0, sizeof(test_buffers));
+out_free_recov_buffers:
+ for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++)
+ vfree(test_recov_buffers[i]);
+ memset(test_recov_buffers, 0, sizeof(test_recov_buffers));
+ return -ENOMEM;
+}
+
+static void raid6_suite_exit(struct kunit_suite *suite)
+{
+ int i;
+
+ for (i = 0; i < NDISKS; i++)
+ vfree(test_buffers[i]);
+ memset(test_buffers, 0, sizeof(test_buffers));
+ for (i = 0; i < RAID6_KUNIT_MAX_FAILURES; i++)
+ vfree(test_recov_buffers[i]);
+ memset(test_recov_buffers, 0, sizeof(test_recov_buffers));
}
static struct kunit_suite raid6_test_suite = {
.name = "raid6",
.test_cases = raid6_test_cases,
.suite_init = raid6_suite_init,
+ .suite_exit = raid6_suite_exit,
};
kunit_test_suite(raid6_test_suite);
--
2.53.0
^ permalink raw reply related
* [PATCH 14/19] raid6_kunit: use KUNIT_CASE_PARAM
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
The raid6 test combines various generation and recovery algorithms. Use
KUNIT_CASE_PARAM and provide a generator that iterates over the possible
combinations instead of looping inside a single test instance.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/tests/raid6_kunit.c | 120 ++++++++++++++++-------------
1 file changed, 68 insertions(+), 52 deletions(-)
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index daaa28e96ff1..f55b081b6b13 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -21,6 +21,15 @@ static char data[NDISKS][PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
static char recovi[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
static char recovj[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE)));
+struct test_args {
+ unsigned int recov_idx;
+ const struct raid6_recov_calls *recov;
+ unsigned int gen_idx;
+ const struct raid6_calls *gen;
+};
+
+static struct test_args args;
+
static void makedata(int start, int stop)
{
int i;
@@ -43,9 +52,10 @@ static char member_type(int d)
}
}
-static void test_disks(struct kunit *test, const struct raid6_calls *calls,
- const struct raid6_recov_calls *ra, int faila, int failb)
+static void test_recover(struct kunit *test, int faila, int failb)
{
+ const struct test_args *ta = test->param_value;
+
memset(recovi, 0xf0, PAGE_SIZE);
memset(recovj, 0xba, PAGE_SIZE);
@@ -61,25 +71,23 @@ static void test_disks(struct kunit *test, const struct raid6_calls *calls,
goto skip;
/* P+Q failure. Just rebuild the syndrome. */
- calls->gen_syndrome(NDISKS, PAGE_SIZE, dataptrs);
+ ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, dataptrs);
} else if (failb == NDISKS - 2) {
/* data+P failure. */
- ra->datap(NDISKS, PAGE_SIZE, faila, dataptrs);
+ ta->recov->datap(NDISKS, PAGE_SIZE, faila, dataptrs);
} else {
/* data+data failure. */
- ra->data2(NDISKS, PAGE_SIZE, faila, failb, dataptrs);
+ ta->recov->data2(NDISKS, PAGE_SIZE, faila, failb, dataptrs);
}
KUNIT_EXPECT_MEMEQ_MSG(test, data[faila], recovi, PAGE_SIZE,
- "algo=%-8s/%-8s faila miscompared: %3d[%c] (failb=%3d[%c])\n",
- calls->name, ra->name,
- faila, member_type(faila),
- failb, member_type(failb));
+ "faila miscompared: %3d[%c] (failb=%3d[%c])\n",
+ faila, member_type(faila),
+ failb, member_type(failb));
KUNIT_EXPECT_MEMEQ_MSG(test, data[failb], recovj, PAGE_SIZE,
- "algo=%-8s/%-8s failb miscompared: %3d[%c] (faila=%3d[%c])\n",
- calls->name, ra->name,
- failb, member_type(failb),
- faila, member_type(faila));
+ "failb miscompared: %3d[%c] (faila=%3d[%c])\n",
+ failb, member_type(failb),
+ faila, member_type(faila));
skip:
dataptrs[faila] = data[faila];
@@ -88,58 +96,66 @@ static void test_disks(struct kunit *test, const struct raid6_calls *calls,
static void raid6_test(struct kunit *test)
{
+ const struct test_args *ta = test->param_value;
int i, j, p1, p2;
- unsigned int r, g;
-
- for (r = 0; ; r++) {
- const struct raid6_recov_calls *ra = raid6_recov_algo_find(r);
-
- if (!ra)
- break;
-
- for (g = 0; ; g++) {
- const struct raid6_calls *calls = raid6_algo_find(g);
-
- if (!calls)
- break;
- /* Nuke syndromes */
- memset(data[NDISKS - 2], 0xee, PAGE_SIZE);
- memset(data[NDISKS - 1], 0xee, PAGE_SIZE);
+ /* Nuke syndromes */
+ memset(data[NDISKS - 2], 0xee, PAGE_SIZE);
+ memset(data[NDISKS - 1], 0xee, PAGE_SIZE);
- /* Generate assumed good syndrome */
- calls->gen_syndrome(NDISKS, PAGE_SIZE,
- (void **)&dataptrs);
+ /* Generate assumed good syndrome */
+ ta->gen->gen_syndrome(NDISKS, PAGE_SIZE, (void **)&dataptrs);
- for (i = 0; i < NDISKS-1; i++)
- for (j = i+1; j < NDISKS; j++)
- test_disks(test, calls, ra, i, j);
+ for (i = 0; i < NDISKS - 1; i++)
+ for (j = i + 1; j < NDISKS; j++)
+ test_recover(test, i, j);
- if (!calls->xor_syndrome)
- continue;
+ if (!ta->gen->xor_syndrome)
+ return;
- for (p1 = 0; p1 < NDISKS-2; p1++)
- for (p2 = p1; p2 < NDISKS-2; p2++) {
+ for (p1 = 0; p1 < NDISKS - 2; p1++) {
+ for (p2 = p1; p2 < NDISKS - 2; p2++) {
+ /* Simulate rmw run */
+ ta->gen->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
+ (void **)&dataptrs);
+ makedata(p1, p2);
+ ta->gen->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
+ (void **)&dataptrs);
- /* Simulate rmw run */
- calls->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
- (void **)&dataptrs);
- makedata(p1, p2);
- calls->xor_syndrome(NDISKS, p1, p2, PAGE_SIZE,
- (void **)&dataptrs);
+ for (i = 0; i < NDISKS - 1; i++)
+ for (j = i + 1; j < NDISKS; j++)
+ test_recover(test, i, j);
+ }
+ }
+}
- for (i = 0; i < NDISKS-1; i++)
- for (j = i+1; j < NDISKS; j++)
- test_disks(test, calls,
- ra, i, j);
- }
+static const void *raid6_gen_params(struct kunit *test, const void *prev,
+ char *desc)
+{
+ if (!prev) {
+ memset(&args, 0, sizeof(args));
+next_algo:
+ args.recov_idx = 0;
+ args.gen = raid6_algo_find(args.gen_idx);
+ if (!args.gen)
+ return NULL;
+ }
- }
+ if (args.recov)
+ args.recov_idx++;
+ args.recov = raid6_recov_algo_find(args.recov_idx);
+ if (!args.recov) {
+ args.gen_idx++;
+ goto next_algo;
}
+
+ snprintf(desc, KUNIT_PARAM_DESC_SIZE, "gen=%s recov=%s",
+ args.gen->name, args.recov->name);
+ return &args;
}
static struct kunit_case raid6_test_cases[] = {
- KUNIT_CASE(raid6_test),
+ KUNIT_CASE_PARAM(raid6_test, raid6_gen_params),
{},
};
--
2.53.0
^ permalink raw reply related
* [PATCH 13/19] raid6: update top of file comments
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Drop the pointless mention of the file name, and use standard formatting
for the top of file comments.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/algos.c | 8 +-------
lib/raid/raid6/arm/neon.c | 2 +-
lib/raid/raid6/mktables.c | 12 +++---------
lib/raid/raid6/recov.c | 14 ++++----------
lib/raid/raid6/riscv/rvv.h | 2 --
lib/raid/raid6/x86/avx2.c | 13 ++++---------
lib/raid/raid6/x86/avx512.c | 18 ++++++------------
lib/raid/raid6/x86/mmx.c | 10 ++--------
lib/raid/raid6/x86/sse1.c | 18 ++++++------------
lib/raid/raid6/x86/sse2.c | 9 +--------
10 files changed, 28 insertions(+), 78 deletions(-)
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index d333318e3301..35c276683cf7 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -1,12 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6/algos.c
+ * Copyright 2002 H. Peter Anvin - All Rights Reserved
*
* Algorithm list and algorithm selection for RAID-6
*/
diff --git a/lib/raid/raid6/arm/neon.c b/lib/raid/raid6/arm/neon.c
index 341c61af675e..af90869aaffc 100644
--- a/lib/raid/raid6/arm/neon.c
+++ b/lib/raid/raid6/arm/neon.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
- * linux/lib/raid6/neon.c - RAID6 syndrome calculation using ARM NEON intrinsics
+ * RAID6 syndrome calculation using ARM NEON intrinsics
*
* Copyright (C) 2013 Linaro Ltd <ard.biesheuvel@linaro.org>
*/
diff --git a/lib/raid/raid6/mktables.c b/lib/raid/raid6/mktables.c
index 97a17493bbd8..b6327b562fdb 100644
--- a/lib/raid/raid6/mktables.c
+++ b/lib/raid/raid6/mktables.c
@@ -1,15 +1,9 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002-2007 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * mktables.c
+ * Copyright 2002-2007 H. Peter Anvin - All Rights Reserved
*
- * Make RAID-6 tables. This is a host user space program to be run at
- * compile time.
+ * Make RAID-6 tables. This is a host user space program to be run at compile
+ * time.
*/
#include <stdio.h>
diff --git a/lib/raid/raid6/recov.c b/lib/raid/raid6/recov.c
index 76eb2aef3667..3fa53bc3fde4 100644
--- a/lib/raid/raid6/recov.c
+++ b/lib/raid/raid6/recov.c
@@ -1,16 +1,10 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6/recov.c
+ * Copyright 2002 H. Peter Anvin - All Rights Reserved
*
- * RAID-6 data recovery in dual failure mode. In single failure mode,
- * use the RAID-5 algorithm (or, in the case of Q failure, just reconstruct
- * the syndrome.)
+ * RAID-6 data recovery in dual failure mode. In single failure mode, use the
+ * RAID-5 algorithm (or, in the case of Q failure, just reconstruct the
+ * syndrome.)
*/
#include <linux/mm.h>
diff --git a/lib/raid/raid6/riscv/rvv.h b/lib/raid/raid6/riscv/rvv.h
index 3a7c2468b1ea..df0e3637cae8 100644
--- a/lib/raid/raid6/riscv/rvv.h
+++ b/lib/raid/raid6/riscv/rvv.h
@@ -2,8 +2,6 @@
/*
* Copyright 2024 Institute of Software, CAS.
*
- * raid6/rvv.h
- *
* Definitions for RISC-V RAID-6 code
*/
diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
index 7efd94e6a87a..7d829c669ea7 100644
--- a/lib/raid/raid6/x86/avx2.c
+++ b/lib/raid/raid6/x86/avx2.c
@@ -1,16 +1,11 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright (C) 2012 Intel Corporation
- * Author: Yuanhan Liu <yuanhan.liu@linux.intel.com>
+/*
+ * Copyright (C) 2012 Intel Corporation
+ * Author: Yuanhan Liu <yuanhan.liu@linux.intel.com>
*
- * Based on sse2.c: Copyright 2002 H. Peter Anvin - All Rights Reserved
+ * Based on sse2.c: Copyright 2002 H. Peter Anvin - All Rights Reserved
*
- * ----------------------------------------------------------------------- */
-
-/*
* AVX2 implementation of RAID-6 syndrome functions
- *
*/
#include <asm/cpufeature.h>
diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index 0772e798b742..e671eb5bde63 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -1,20 +1,14 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- --------------------------------------------------------
- *
- * Copyright (C) 2016 Intel Corporation
+/*
+ * Copyright (C) 2016 Intel Corporation
*
- * Author: Gayatri Kammela <gayatri.kammela@intel.com>
- * Author: Megha Dey <megha.dey@linux.intel.com>
+ * Author: Gayatri Kammela <gayatri.kammela@intel.com>
+ * Author: Megha Dey <megha.dey@linux.intel.com>
*
- * Based on avx2.c: Copyright 2012 Yuanhan Liu All Rights Reserved
- * Based on sse2.c: Copyright 2002 H. Peter Anvin - All Rights Reserved
+ * Based on avx2.c: Copyright 2012 Yuanhan Liu All Rights Reserved
+ * Based on sse2.c: Copyright 2002 H. Peter Anvin - All Rights Reserved
*
- * -----------------------------------------------------------------------
- */
-
-/*
* AVX512 implementation of RAID-6 syndrome functions
- *
*/
#include <asm/cpufeature.h>
diff --git a/lib/raid/raid6/x86/mmx.c b/lib/raid/raid6/x86/mmx.c
index 3228c335965a..afa82536142d 100644
--- a/lib/raid/raid6/x86/mmx.c
+++ b/lib/raid/raid6/x86/mmx.c
@@ -1,14 +1,8 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6/mmx.c
+ * Copyright 2002 H. Peter Anvin - All Rights Reserved
*
- * MMX implementation of RAID-6 syndrome functions
+ * MMX implementation of RAID-6 syndrome functions.
*/
#include <asm/cpufeature.h>
diff --git a/lib/raid/raid6/x86/sse1.c b/lib/raid/raid6/x86/sse1.c
index 6ebdcf824e00..f4b260df522a 100644
--- a/lib/raid/raid6/x86/sse1.c
+++ b/lib/raid/raid6/x86/sse1.c
@@ -1,19 +1,13 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6/sse1.c
+ * Copyright 2002 H. Peter Anvin - All Rights Reserved
*
- * SSE-1/MMXEXT implementation of RAID-6 syndrome functions
+ * SSE-1/MMXEXT implementation of RAID-6 syndrome functions.
*
- * This is really an MMX implementation, but it requires SSE-1 or
- * AMD MMXEXT for prefetch support and a few other features. The
- * support for nontemporal memory accesses is enough to make this
- * worthwhile as a separate implementation.
+ * This is really an MMX implementation, but it requires SSE-1 or AMD MMXEXT for
+ * prefetch support and a few other features. The support for nontemporal
+ * memory accesses is enough to make this worthwhile as a separate
+ * implementation.
*/
#include <asm/cpufeature.h>
diff --git a/lib/raid/raid6/x86/sse2.c b/lib/raid/raid6/x86/sse2.c
index 7049c8512f35..43b09ce58270 100644
--- a/lib/raid/raid6/x86/sse2.c
+++ b/lib/raid/raid6/x86/sse2.c
@@ -1,15 +1,8 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/* -*- linux-c -*- ------------------------------------------------------- *
- *
- * Copyright 2002 H. Peter Anvin - All Rights Reserved
- *
- * ----------------------------------------------------------------------- */
-
/*
- * raid6/sse2.c
+ * Copyright 2002 H. Peter Anvin - All Rights Reserved
*
* SSE-2 implementation of RAID-6 syndrome functions
- *
*/
#include <asm/cpufeature.h>
--
2.53.0
^ permalink raw reply related
* [PATCH 12/19] raid6: use static_call for raid6_recov_2data and raid6_recov_datap
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Avoid expensive indirect calls for the recovery routines as well.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/algos.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index 65b75236ecf8..d333318e3301 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -26,6 +26,8 @@ static const struct raid6_recov_calls *raid6_recov_algo;
/* Selected algorithm */
DEFINE_STATIC_CALL_NULL(raid6_gen_syndrome_impl, *raid6_intx1.gen_syndrome);
DEFINE_STATIC_CALL_NULL(raid6_xor_syndrome_impl, *raid6_intx1.xor_syndrome);
+DEFINE_STATIC_CALL_NULL(raid6_recov_2data_impl, *raid6_recov_intx1.data2);
+DEFINE_STATIC_CALL_NULL(raid6_recov_datap_impl, *raid6_recov_intx1.datap);
/**
* raid6_gen_syndrome - generate RAID6 P/Q parity
@@ -126,7 +128,7 @@ void raid6_recov_2data(int disks, size_t bytes, int faila, int failb,
WARN_ON_ONCE(bytes > PAGE_SIZE);
WARN_ON_ONCE(failb <= faila);
- raid6_recov_algo->data2(disks, bytes, faila, failb, ptrs);
+ static_call(raid6_recov_2data_impl)(disks, bytes, faila, failb, ptrs);
}
EXPORT_SYMBOL_GPL(raid6_recov_2data);
@@ -151,7 +153,7 @@ void raid6_recov_datap(int disks, size_t bytes, int faila, void **ptrs)
WARN_ON_ONCE(bytes & 511);
WARN_ON_ONCE(bytes > PAGE_SIZE);
- raid6_recov_algo->datap(disks, bytes, faila, ptrs);
+ static_call(raid6_recov_datap_impl)(disks, bytes, faila, ptrs);
}
EXPORT_SYMBOL_GPL(raid6_recov_datap);
@@ -324,6 +326,8 @@ static int __init raid6_init(void)
*/
if (!raid6_recov_algo)
raid6_recov_algo = &raid6_recov_intx1;
+ static_call_update(raid6_recov_2data_impl, raid6_recov_algo->data2);
+ static_call_update(raid6_recov_datap_impl, raid6_recov_algo->datap);
pr_info("raid6: using %s recovery algorithm\n", raid6_recov_algo->name);
#ifdef MODULE
--
2.53.0
^ permalink raw reply related
* [PATCH 11/19] raid6: use static_call for gen_syndrom and xor_syndrom
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Avoid indirect calls for P/Q parity generation.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/raid6/algos.c | 18 ++++++++++++------
1 file changed, 12 insertions(+), 6 deletions(-)
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index 5bd953032e55..65b75236ecf8 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -14,6 +14,7 @@
#include <linux/module.h>
#include <linux/gfp.h>
#include <linux/raid/pq.h>
+#include <linux/static_call.h>
#include <kunit/visibility.h>
#include "algos.h"
@@ -23,7 +24,8 @@ static unsigned int raid6_nr_algos;
static const struct raid6_recov_calls *raid6_recov_algo;
/* Selected algorithm */
-static struct raid6_calls raid6_call;
+DEFINE_STATIC_CALL_NULL(raid6_gen_syndrome_impl, *raid6_intx1.gen_syndrome);
+DEFINE_STATIC_CALL_NULL(raid6_xor_syndrome_impl, *raid6_intx1.xor_syndrome);
/**
* raid6_gen_syndrome - generate RAID6 P/Q parity
@@ -48,7 +50,7 @@ void raid6_gen_syndrome(int disks, size_t bytes, void **ptrs)
WARN_ON_ONCE(bytes & 511);
WARN_ON_ONCE(disks < RAID6_MIN_DISKS);
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ static_call(raid6_gen_syndrome_impl)(disks, bytes, ptrs);
}
EXPORT_SYMBOL_GPL(raid6_gen_syndrome);
@@ -85,7 +87,7 @@ void raid6_xor_syndrome(int disks, int start, int stop, size_t bytes,
WARN_ON_ONCE(disks < RAID6_MIN_DISKS);
WARN_ON_ONCE(stop < start);
- raid6_call.xor_syndrome(disks, start, stop, bytes, ptrs);
+ static_call(raid6_xor_syndrome_impl)(disks, start, stop, bytes, ptrs);
}
EXPORT_SYMBOL_GPL(raid6_xor_syndrome);
@@ -96,7 +98,7 @@ EXPORT_SYMBOL_GPL(raid6_xor_syndrome);
*/
bool raid6_can_xor_syndrome(void)
{
- return !!raid6_call.xor_syndrome;
+ return !!static_call_query(raid6_xor_syndrome_impl);
}
EXPORT_SYMBOL_GPL(raid6_can_xor_syndrome);
@@ -195,7 +197,8 @@ static int raid6_choose_gen(void *(*const dptrs)[RAID6_TEST_DISKS],
return -EINVAL;
}
- raid6_call = *best;
+ static_call_update(raid6_gen_syndrome_impl, best->gen_syndrome);
+ static_call_update(raid6_xor_syndrome_impl, best->xor_syndrome);
pr_info("raid6: using algorithm %s gen() %ld MB/s\n",
best->name,
@@ -240,7 +243,10 @@ static int __init raid6_select_algo(void)
if (!IS_ENABLED(CONFIG_RAID6_PQ_BENCHMARK) || raid6_nr_algos == 1) {
pr_info("raid6: skipped pq benchmark and selected %s\n",
raid6_algos[raid6_nr_algos - 1]->name);
- raid6_call = *raid6_algos[raid6_nr_algos - 1];
+ static_call_update(raid6_gen_syndrome_impl,
+ raid6_algos[raid6_nr_algos - 1]->gen_syndrome);
+ static_call_update(raid6_xor_syndrome_impl,
+ raid6_algos[raid6_nr_algos - 1]->xor_syndrome);
return 0;
}
--
2.53.0
^ permalink raw reply related
* [PATCH 10/19] raid6: rework the init helpers
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Match the xor version with two initcalls for the built-in case to delay
calibrartion. This prepares for adding non-calibration init code.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
lib/raid/Kconfig | 11 +
lib/raid/raid6/Makefile | 4 +
lib/raid/raid6/algos.c | 309 ++++++++----------
lib/raid/raid6/algos.h | 67 +---
lib/raid/raid6/arm/neon.c | 6 -
lib/raid/raid6/arm/pq_arch.h | 24 ++
lib/raid/raid6/arm/recov_neon.c | 7 -
lib/raid/raid6/arm64/pq_arch.h | 1 +
lib/raid/raid6/loongarch/loongarch_simd.c | 12 -
lib/raid/raid6/loongarch/pq_arch.h | 23 ++
.../raid6/loongarch/recov_loongarch_simd.c | 14 -
lib/raid/raid6/powerpc/altivec.uc | 10 -
lib/raid/raid6/powerpc/pq_arch.h | 31 ++
lib/raid/raid6/powerpc/vpermxor.uc | 11 -
lib/raid/raid6/recov.c | 2 -
lib/raid/raid6/riscv/pq_arch.h | 21 ++
lib/raid/raid6/riscv/recov_rvv.c | 2 -
lib/raid/raid6/riscv/rvv.h | 6 -
lib/raid/raid6/s390/pq_arch.h | 15 +
lib/raid/raid6/s390/recov_s390xc.c | 2 -
lib/raid/raid6/s390/s390vx.uc | 7 -
lib/raid/raid6/tests/raid6_kunit.c | 23 +-
lib/raid/raid6/x86/avx2.c | 14 -
lib/raid/raid6/x86/avx512.c | 19 --
lib/raid/raid6/x86/mmx.c | 8 -
lib/raid/raid6/x86/pq_arch.h | 97 ++++++
lib/raid/raid6/x86/recov_avx2.c | 8 -
lib/raid/raid6/x86/recov_avx512.c | 12 -
lib/raid/raid6/x86/recov_ssse3.c | 9 -
lib/raid/raid6/x86/sse1.c | 12 -
lib/raid/raid6/x86/sse2.c | 15 -
31 files changed, 393 insertions(+), 409 deletions(-)
create mode 100644 lib/raid/raid6/arm/pq_arch.h
create mode 100644 lib/raid/raid6/arm64/pq_arch.h
create mode 100644 lib/raid/raid6/loongarch/pq_arch.h
create mode 100644 lib/raid/raid6/powerpc/pq_arch.h
create mode 100644 lib/raid/raid6/riscv/pq_arch.h
create mode 100644 lib/raid/raid6/s390/pq_arch.h
create mode 100644 lib/raid/raid6/x86/pq_arch.h
diff --git a/lib/raid/Kconfig b/lib/raid/Kconfig
index e39f6d667792..978cd6ba08ac 100644
--- a/lib/raid/Kconfig
+++ b/lib/raid/Kconfig
@@ -32,6 +32,17 @@ config XOR_KUNIT_TEST
config RAID6_PQ
tristate
+# selected by architectures that provide an optimized PQ implementation
+config RAID6_PQ_ARCH
+ depends on RAID6_PQ
+ default y if KERNEL_MODE_NEON # arm32/arm64
+ default y if LOONGARCH
+ default y if ALTIVEC # powerpc
+ default y if RISCV_ISA_V
+ default y if S390
+ default y if X86
+ bool
+
config RAID6_PQ_KUNIT_TEST
tristate "KUnit tests for RAID6 PQ functions" if !KUNIT_ALL_TESTS
depends on KUNIT
diff --git a/lib/raid/raid6/Makefile b/lib/raid/raid6/Makefile
index f64f6d32f28b..ed1f512b88af 100644
--- a/lib/raid/raid6/Makefile
+++ b/lib/raid/raid6/Makefile
@@ -4,6 +4,10 @@ hostprogs += mktables
ccflags-y += -I $(src)
+ifeq ($(CONFIG_RAID6_PQ_ARCH),y)
+CFLAGS_algos.o += -I$(src)/$(SRCARCH)
+endif
+
obj-$(CONFIG_RAID6_PQ) += raid6_pq.o tests/
raid6_pq-y += algos.o tables.o
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index af31a1feb6e7..5bd953032e55 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -17,6 +17,9 @@
#include <kunit/visibility.h>
#include "algos.h"
+#define RAID6_MAX_ALGOS 16
+static const struct raid6_calls *raid6_algos[RAID6_MAX_ALGOS];
+static unsigned int raid6_nr_algos;
static const struct raid6_recov_calls *raid6_recov_algo;
/* Selected algorithm */
@@ -97,71 +100,6 @@ bool raid6_can_xor_syndrome(void)
}
EXPORT_SYMBOL_GPL(raid6_can_xor_syndrome);
-const struct raid6_calls * const raid6_algos[] = {
-#if defined(__i386__) && !defined(__arch_um__)
- &raid6_avx512x2,
- &raid6_avx512x1,
- &raid6_avx2x2,
- &raid6_avx2x1,
- &raid6_sse2x2,
- &raid6_sse2x1,
- &raid6_sse1x2,
- &raid6_sse1x1,
- &raid6_mmxx2,
- &raid6_mmxx1,
-#endif
-#if defined(__x86_64__) && !defined(__arch_um__)
- &raid6_avx512x4,
- &raid6_avx512x2,
- &raid6_avx512x1,
- &raid6_avx2x4,
- &raid6_avx2x2,
- &raid6_avx2x1,
- &raid6_sse2x4,
- &raid6_sse2x2,
- &raid6_sse2x1,
-#endif
-#ifdef CONFIG_ALTIVEC
- &raid6_vpermxor8,
- &raid6_vpermxor4,
- &raid6_vpermxor2,
- &raid6_vpermxor1,
- &raid6_altivec8,
- &raid6_altivec4,
- &raid6_altivec2,
- &raid6_altivec1,
-#endif
-#if defined(CONFIG_S390)
- &raid6_s390vx8,
-#endif
-#ifdef CONFIG_KERNEL_MODE_NEON
- &raid6_neonx8,
- &raid6_neonx4,
- &raid6_neonx2,
- &raid6_neonx1,
-#endif
-#ifdef CONFIG_LOONGARCH
-#ifdef CONFIG_CPU_HAS_LASX
- &raid6_lasx,
-#endif
-#ifdef CONFIG_CPU_HAS_LSX
- &raid6_lsx,
-#endif
-#endif
-#ifdef CONFIG_RISCV_ISA_V
- &raid6_rvvx1,
- &raid6_rvvx2,
- &raid6_rvvx4,
- &raid6_rvvx8,
-#endif
- &raid6_intx8,
- &raid6_intx4,
- &raid6_intx2,
- &raid6_intx1,
- NULL
-};
-EXPORT_SYMBOL_IF_KUNIT(raid6_algos);
-
/**
* raid6_recov_2data - recover two missing data disks
* @disks: number of "disks" to operate on including parity
@@ -215,119 +153,57 @@ void raid6_recov_datap(int disks, size_t bytes, int faila, void **ptrs)
}
EXPORT_SYMBOL_GPL(raid6_recov_datap);
-const struct raid6_recov_calls *const raid6_recov_algos[] = {
-#ifdef CONFIG_X86
- &raid6_recov_avx512,
- &raid6_recov_avx2,
- &raid6_recov_ssse3,
-#endif
-#ifdef CONFIG_S390
- &raid6_recov_s390xc,
-#endif
-#if defined(CONFIG_KERNEL_MODE_NEON)
- &raid6_recov_neon,
-#endif
-#ifdef CONFIG_LOONGARCH
-#ifdef CONFIG_CPU_HAS_LASX
- &raid6_recov_lasx,
-#endif
-#ifdef CONFIG_CPU_HAS_LSX
- &raid6_recov_lsx,
-#endif
-#endif
-#ifdef CONFIG_RISCV_ISA_V
- &raid6_recov_rvv,
-#endif
- &raid6_recov_intx1,
- NULL
-};
-EXPORT_SYMBOL_IF_KUNIT(raid6_recov_algos);
-
#define RAID6_TIME_JIFFIES_LG2 4
#define RAID6_TEST_DISKS 8
#define RAID6_TEST_DISKS_ORDER 3
-static inline const struct raid6_recov_calls *raid6_choose_recov(void)
+static int raid6_choose_gen(void *(*const dptrs)[RAID6_TEST_DISKS],
+ const int disks)
{
- const struct raid6_recov_calls *const *algo;
- const struct raid6_recov_calls *best;
-
- for (best = NULL, algo = raid6_recov_algos; *algo; algo++)
- if (!best || (*algo)->priority > best->priority)
- if (!(*algo)->valid || (*algo)->valid())
- best = *algo;
+ /* work on the second half of the disks */
+ int start = (disks >> 1) - 1, stop = disks - 3;
+ const struct raid6_calls *best = NULL;
+ unsigned long bestgenperf = 0;
+ unsigned int i;
- if (best) {
- raid6_recov_algo = best;
+ for (i = 0; i < raid6_nr_algos; i++) {
+ const struct raid6_calls *algo = raid6_algos[i];
+ unsigned long perf = 0, j0, j1;
- pr_info("raid6: using %s recovery algorithm\n", best->name);
- } else
- pr_err("raid6: Yikes! No recovery algorithm found!\n");
-
- return best;
-}
+ preempt_disable();
+ j0 = jiffies;
+ while ((j1 = jiffies) == j0)
+ cpu_relax();
+ while (time_before(jiffies,
+ j1 + (1<<RAID6_TIME_JIFFIES_LG2))) {
+ algo->gen_syndrome(disks, PAGE_SIZE, *dptrs);
+ perf++;
+ }
+ preempt_enable();
-static inline const struct raid6_calls *raid6_choose_gen(
- void *(*const dptrs)[RAID6_TEST_DISKS], const int disks)
-{
- unsigned long perf, bestgenperf, j0, j1;
- int start = (disks>>1)-1, stop = disks-3; /* work on the second half of the disks */
- const struct raid6_calls *const *algo;
- const struct raid6_calls *best;
-
- for (bestgenperf = 0, best = NULL, algo = raid6_algos; *algo; algo++) {
- if (!best || (*algo)->priority >= best->priority) {
- if ((*algo)->valid && !(*algo)->valid())
- continue;
-
- if (!IS_ENABLED(CONFIG_RAID6_PQ_BENCHMARK)) {
- best = *algo;
- break;
- }
-
- perf = 0;
-
- preempt_disable();
- j0 = jiffies;
- while ((j1 = jiffies) == j0)
- cpu_relax();
- while (time_before(jiffies,
- j1 + (1<<RAID6_TIME_JIFFIES_LG2))) {
- (*algo)->gen_syndrome(disks, PAGE_SIZE, *dptrs);
- perf++;
- }
- preempt_enable();
-
- if (perf > bestgenperf) {
- bestgenperf = perf;
- best = *algo;
- }
- pr_info("raid6: %-8s gen() %5ld MB/s\n", (*algo)->name,
- (perf * HZ * (disks-2)) >>
- (20 - PAGE_SHIFT + RAID6_TIME_JIFFIES_LG2));
+ if (perf > bestgenperf) {
+ bestgenperf = perf;
+ best = algo;
}
+ pr_info("raid6: %-8s gen() %5ld MB/s\n", algo->name,
+ (perf * HZ * (disks-2)) >>
+ (20 - PAGE_SHIFT + RAID6_TIME_JIFFIES_LG2));
}
if (!best) {
pr_err("raid6: Yikes! No algorithm found!\n");
- goto out;
+ return -EINVAL;
}
raid6_call = *best;
- if (!IS_ENABLED(CONFIG_RAID6_PQ_BENCHMARK)) {
- pr_info("raid6: skipped pq benchmark and selected %s\n",
- best->name);
- goto out;
- }
-
pr_info("raid6: using algorithm %s gen() %ld MB/s\n",
best->name,
(bestgenperf * HZ * (disks - 2)) >>
(20 - PAGE_SHIFT + RAID6_TIME_JIFFIES_LG2));
if (best->xor_syndrome) {
- perf = 0;
+ unsigned long perf = 0, j0, j1;
preempt_disable();
j0 = jiffies;
@@ -346,8 +222,7 @@ static inline const struct raid6_calls *raid6_choose_gen(
(20 - PAGE_SHIFT + RAID6_TIME_JIFFIES_LG2 + 1));
}
-out:
- return best;
+ return 0;
}
@@ -357,12 +232,17 @@ static inline const struct raid6_calls *raid6_choose_gen(
static int __init raid6_select_algo(void)
{
const int disks = RAID6_TEST_DISKS;
-
- const struct raid6_calls *gen_best;
- const struct raid6_recov_calls *rec_best;
char *disk_ptr, *p;
void *dptrs[RAID6_TEST_DISKS];
int i, cycle;
+ int error;
+
+ if (!IS_ENABLED(CONFIG_RAID6_PQ_BENCHMARK) || raid6_nr_algos == 1) {
+ pr_info("raid6: skipped pq benchmark and selected %s\n",
+ raid6_algos[raid6_nr_algos - 1]->name);
+ raid6_call = *raid6_algos[raid6_nr_algos - 1];
+ return 0;
+ }
/* prepare the buffer and fill it circularly with gfmul table */
disk_ptr = (char *)__get_free_pages(GFP_KERNEL, RAID6_TEST_DISKS_ORDER);
@@ -385,22 +265,115 @@ static int __init raid6_select_algo(void)
memcpy(p, raid6_gfmul, (disks - 2) * PAGE_SIZE % 65536);
/* select raid gen_syndrome function */
- gen_best = raid6_choose_gen(&dptrs, disks);
-
- /* select raid recover functions */
- rec_best = raid6_choose_recov();
+ error = raid6_choose_gen(&dptrs, disks);
free_pages((unsigned long)disk_ptr, RAID6_TEST_DISKS_ORDER);
- return gen_best && rec_best ? 0 : -EINVAL;
+ return error;
+}
+
+void __init raid6_algo_add(const struct raid6_calls *algo)
+{
+ if (WARN_ON_ONCE(raid6_nr_algos == RAID6_MAX_ALGOS))
+ return;
+ raid6_algos[raid6_nr_algos++] = algo;
+}
+
+void __init raid6_algo_add_default(void)
+{
+ raid6_algo_add(&raid6_intx1);
+ raid6_algo_add(&raid6_intx2);
+ raid6_algo_add(&raid6_intx4);
+ raid6_algo_add(&raid6_intx8);
+}
+
+void __init raid6_recov_algo_add(const struct raid6_recov_calls *algo)
+{
+ if (WARN_ON_ONCE(raid6_recov_algo))
+ return;
+ raid6_recov_algo = algo;
+}
+
+#ifdef CONFIG_RAID6_PQ_ARCH
+#include "pq_arch.h"
+#else
+static inline void arch_raid6_init(void)
+{
+ raid6_algo_add_default();
+}
+#endif /* CONFIG_RAID6_PQ_ARCH */
+
+static int __init raid6_init(void)
+{
+ /*
+ * Architectures providing arch_raid6_init must add all PQ generation
+ * algorithms they want to consider in arch_raid6_init(), including
+ * the generic ones using raid6_algo_add_default() if wanted.
+ */
+ arch_raid6_init();
+
+ /*
+ * Architectures don't have to set a recovery algorithm, we'll just pick
+ * the generic integer one if none was set.
+ */
+ if (!raid6_recov_algo)
+ raid6_recov_algo = &raid6_recov_intx1;
+ pr_info("raid6: using %s recovery algorithm\n", raid6_recov_algo->name);
+
+#ifdef MODULE
+ return raid6_select_algo();
+#else
+ return 0;
+#endif
}
-static void raid6_exit(void)
+static void __exit raid6_exit(void)
{
- do { } while (0);
}
-subsys_initcall(raid6_select_algo);
+/*
+ * When built-in we must register the default template before md, but we don't
+ * want calibration to run that early as that would delay the boot process.
+ */
+#ifndef MODULE
+__initcall(raid6_select_algo);
+#endif
+core_initcall(raid6_init);
module_exit(raid6_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("RAID6 Q-syndrome calculations");
+
+#if IS_ENABLED(CONFIG_RAID6_PQ_KUNIT_TEST)
+const struct raid6_calls *raid6_algo_find(unsigned int idx)
+{
+ if (idx >= raid6_nr_algos) {
+ /*
+ * Always include the simplest generic integer implementation in
+ * the unit tests as a baseline.
+ */
+ if (idx == raid6_nr_algos &&
+ raid6_algos[0] != &raid6_intx1)
+ return &raid6_intx1;
+ return NULL;
+ }
+ return raid6_algos[idx];
+}
+EXPORT_SYMBOL_IF_KUNIT(raid6_algo_find);
+
+const struct raid6_recov_calls *raid6_recov_algo_find(unsigned int idx)
+{
+ switch (idx) {
+ case 0:
+ /* always test the generic integer implementation */
+ return &raid6_recov_intx1;
+ case 1:
+ /* test the optimized implementation if there is one */
+ if (raid6_recov_algo != &raid6_recov_intx1)
+ return raid6_recov_algo;
+ return NULL;
+ default:
+ return NULL;
+ }
+}
+EXPORT_SYMBOL_IF_KUNIT(raid6_recov_algo_find);
+#endif /* CONFIG_RAID6_PQ_KUNIT_TEST */
diff --git a/lib/raid/raid6/algos.h b/lib/raid/raid6/algos.h
index e5f1098d2179..43f636be183f 100644
--- a/lib/raid/raid6/algos.h
+++ b/lib/raid/raid6/algos.h
@@ -5,6 +5,7 @@
#ifndef _PQ_IMPL_H
#define _PQ_IMPL_H
+#include <linux/init.h>
#include <linux/raid/pq_tables.h>
/* Routine choices */
@@ -13,70 +14,28 @@ struct raid6_calls {
void (*gen_syndrome)(int disks, size_t bytes, void **ptrs);
void (*xor_syndrome)(int disks, int start, int stop, size_t bytes,
void **ptrs);
- int (*valid)(void); /* Returns 1 if this routine set is usable */
- int priority; /* Relative priority ranking if non-zero */
};
-/* Various routine sets */
-extern const struct raid6_calls raid6_intx1;
-extern const struct raid6_calls raid6_intx2;
-extern const struct raid6_calls raid6_intx4;
-extern const struct raid6_calls raid6_intx8;
-extern const struct raid6_calls raid6_mmxx1;
-extern const struct raid6_calls raid6_mmxx2;
-extern const struct raid6_calls raid6_sse1x1;
-extern const struct raid6_calls raid6_sse1x2;
-extern const struct raid6_calls raid6_sse2x1;
-extern const struct raid6_calls raid6_sse2x2;
-extern const struct raid6_calls raid6_sse2x4;
-extern const struct raid6_calls raid6_altivec1;
-extern const struct raid6_calls raid6_altivec2;
-extern const struct raid6_calls raid6_altivec4;
-extern const struct raid6_calls raid6_altivec8;
-extern const struct raid6_calls raid6_avx2x1;
-extern const struct raid6_calls raid6_avx2x2;
-extern const struct raid6_calls raid6_avx2x4;
-extern const struct raid6_calls raid6_avx512x1;
-extern const struct raid6_calls raid6_avx512x2;
-extern const struct raid6_calls raid6_avx512x4;
-extern const struct raid6_calls raid6_s390vx8;
-extern const struct raid6_calls raid6_vpermxor1;
-extern const struct raid6_calls raid6_vpermxor2;
-extern const struct raid6_calls raid6_vpermxor4;
-extern const struct raid6_calls raid6_vpermxor8;
-extern const struct raid6_calls raid6_lsx;
-extern const struct raid6_calls raid6_lasx;
-extern const struct raid6_calls raid6_rvvx1;
-extern const struct raid6_calls raid6_rvvx2;
-extern const struct raid6_calls raid6_rvvx4;
-extern const struct raid6_calls raid6_rvvx8;
-
struct raid6_recov_calls {
const char *name;
void (*data2)(int disks, size_t bytes, int faila, int failb,
void **ptrs);
void (*datap)(int disks, size_t bytes, int faila, void **ptrs);
- int (*valid)(void);
- int priority;
};
-extern const struct raid6_recov_calls raid6_recov_intx1;
-extern const struct raid6_recov_calls raid6_recov_ssse3;
-extern const struct raid6_recov_calls raid6_recov_avx2;
-extern const struct raid6_recov_calls raid6_recov_avx512;
-extern const struct raid6_recov_calls raid6_recov_s390xc;
-extern const struct raid6_recov_calls raid6_recov_neon;
-extern const struct raid6_recov_calls raid6_recov_lsx;
-extern const struct raid6_recov_calls raid6_recov_lasx;
-extern const struct raid6_recov_calls raid6_recov_rvv;
+void __init raid6_algo_add(const struct raid6_calls *algo);
+void __init raid6_algo_add_default(void);
+void __init raid6_recov_algo_add(const struct raid6_recov_calls *algo);
-extern const struct raid6_calls raid6_neonx1;
-extern const struct raid6_calls raid6_neonx2;
-extern const struct raid6_calls raid6_neonx4;
-extern const struct raid6_calls raid6_neonx8;
+/* for the kunit test */
+const struct raid6_calls *raid6_algo_find(unsigned int idx);
+const struct raid6_recov_calls *raid6_recov_algo_find(unsigned int idx);
-/* Algorithm list */
-extern const struct raid6_calls * const raid6_algos[];
-extern const struct raid6_recov_calls *const raid6_recov_algos[];
+/* generic implementations */
+extern const struct raid6_calls raid6_intx1;
+extern const struct raid6_calls raid6_intx2;
+extern const struct raid6_calls raid6_intx4;
+extern const struct raid6_calls raid6_intx8;
+extern const struct raid6_recov_calls raid6_recov_intx1;
#endif /* _PQ_IMPL_H */
diff --git a/lib/raid/raid6/arm/neon.c b/lib/raid/raid6/arm/neon.c
index bd4ec4c86ee8..341c61af675e 100644
--- a/lib/raid/raid6/arm/neon.c
+++ b/lib/raid/raid6/arm/neon.c
@@ -42,15 +42,9 @@
struct raid6_calls const raid6_neonx ## _n = { \
.gen_syndrome = raid6_neon ## _n ## _gen_syndrome, \
.xor_syndrome = raid6_neon ## _n ## _xor_syndrome, \
- .valid = raid6_have_neon, \
.name = "neonx" #_n, \
}
-static int raid6_have_neon(void)
-{
- return cpu_has_neon();
-}
-
RAID6_NEON_WRAPPER(1);
RAID6_NEON_WRAPPER(2);
RAID6_NEON_WRAPPER(4);
diff --git a/lib/raid/raid6/arm/pq_arch.h b/lib/raid/raid6/arm/pq_arch.h
new file mode 100644
index 000000000000..a1f4cc4961e9
--- /dev/null
+++ b/lib/raid/raid6/arm/pq_arch.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <asm/neon.h>
+
+extern const struct raid6_calls raid6_neonx1;
+extern const struct raid6_calls raid6_neonx2;
+extern const struct raid6_calls raid6_neonx4;
+extern const struct raid6_calls raid6_neonx8;
+extern const struct raid6_recov_calls raid6_recov_neon;
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ if (cpu_has_neon()) {
+ raid6_algo_add(&raid6_neonx1);
+ raid6_algo_add(&raid6_neonx2);
+ raid6_algo_add(&raid6_neonx4);
+ raid6_algo_add(&raid6_neonx8);
+ }
+
+ raid6_algo_add_default();
+
+ if (cpu_has_neon())
+ raid6_recov_algo_add(&raid6_recov_neon);
+}
diff --git a/lib/raid/raid6/arm/recov_neon.c b/lib/raid/raid6/arm/recov_neon.c
index e1d1d19fc9a8..1524050d09b7 100644
--- a/lib/raid/raid6/arm/recov_neon.c
+++ b/lib/raid/raid6/arm/recov_neon.c
@@ -10,11 +10,6 @@
#include "algos.h"
#include "arm/neon.h"
-static int raid6_has_neon(void)
-{
- return cpu_has_neon();
-}
-
static void raid6_2data_recov_neon(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -87,7 +82,5 @@ static void raid6_datap_recov_neon(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_neon = {
.data2 = raid6_2data_recov_neon,
.datap = raid6_datap_recov_neon,
- .valid = raid6_has_neon,
.name = "neon",
- .priority = 10,
};
diff --git a/lib/raid/raid6/arm64/pq_arch.h b/lib/raid/raid6/arm64/pq_arch.h
new file mode 100644
index 000000000000..27ff564d7594
--- /dev/null
+++ b/lib/raid/raid6/arm64/pq_arch.h
@@ -0,0 +1 @@
+#include "arm/pq_arch.h"
diff --git a/lib/raid/raid6/loongarch/loongarch_simd.c b/lib/raid/raid6/loongarch/loongarch_simd.c
index f77d11ce676e..c1eb53fafd27 100644
--- a/lib/raid/raid6/loongarch/loongarch_simd.c
+++ b/lib/raid/raid6/loongarch/loongarch_simd.c
@@ -26,11 +26,6 @@
#ifdef CONFIG_CPU_HAS_LSX
#define NSIZE 16
-static int raid6_has_lsx(void)
-{
- return cpu_has_lsx;
-}
-
static void raid6_lsx_gen_syndrome(int disks, size_t bytes, void **ptrs)
{
u8 **dptr = (u8 **)ptrs;
@@ -246,7 +241,6 @@ static void raid6_lsx_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_lsx = {
.gen_syndrome = raid6_lsx_gen_syndrome,
.xor_syndrome = raid6_lsx_xor_syndrome,
- .valid = raid6_has_lsx,
.name = "lsx",
};
@@ -256,11 +250,6 @@ const struct raid6_calls raid6_lsx = {
#ifdef CONFIG_CPU_HAS_LASX
#define NSIZE 32
-static int raid6_has_lasx(void)
-{
- return cpu_has_lasx;
-}
-
static void raid6_lasx_gen_syndrome(int disks, size_t bytes, void **ptrs)
{
u8 **dptr = (u8 **)ptrs;
@@ -414,7 +403,6 @@ static void raid6_lasx_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_lasx = {
.gen_syndrome = raid6_lasx_gen_syndrome,
.xor_syndrome = raid6_lasx_xor_syndrome,
- .valid = raid6_has_lasx,
.name = "lasx",
};
#undef NSIZE
diff --git a/lib/raid/raid6/loongarch/pq_arch.h b/lib/raid/raid6/loongarch/pq_arch.h
new file mode 100644
index 000000000000..cd4ee2df3f37
--- /dev/null
+++ b/lib/raid/raid6/loongarch/pq_arch.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <asm/cpu-features.h>
+
+extern const struct raid6_calls raid6_lsx;
+extern const struct raid6_calls raid6_lasx;
+
+extern const struct raid6_recov_calls raid6_recov_lsx;
+extern const struct raid6_recov_calls raid6_recov_lasx;
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ if (IS_ENABLED(CONFIG_CPU_HAS_LSX) && cpu_has_lsx)
+ raid6_algo_add(&raid6_lsx);
+ if (IS_ENABLED(CONFIG_CPU_HAS_LASX) && cpu_has_lasx)
+ raid6_algo_add(&raid6_lasx);
+ raid6_algo_add_default();
+
+ if (IS_ENABLED(CONFIG_CPU_HAS_LASX) && cpu_has_lasx)
+ raid6_recov_algo_add(&raid6_recov_lasx);
+ else if (IS_ENABLED(CONFIG_CPU_HAS_LSX) && cpu_has_lsx)
+ raid6_recov_algo_add(&raid6_recov_lsx);
+}
diff --git a/lib/raid/raid6/loongarch/recov_loongarch_simd.c b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
index 0bbdc8b5c2e7..87a2313bbb4f 100644
--- a/lib/raid/raid6/loongarch/recov_loongarch_simd.c
+++ b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
@@ -24,11 +24,6 @@
*/
#ifdef CONFIG_CPU_HAS_LSX
-static int raid6_has_lsx(void)
-{
- return cpu_has_lsx;
-}
-
static void raid6_2data_recov_lsx(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -291,18 +286,11 @@ static void raid6_datap_recov_lsx(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_lsx = {
.data2 = raid6_2data_recov_lsx,
.datap = raid6_datap_recov_lsx,
- .valid = raid6_has_lsx,
.name = "lsx",
- .priority = 1,
};
#endif /* CONFIG_CPU_HAS_LSX */
#ifdef CONFIG_CPU_HAS_LASX
-static int raid6_has_lasx(void)
-{
- return cpu_has_lasx;
-}
-
static void raid6_2data_recov_lasx(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -509,8 +497,6 @@ static void raid6_datap_recov_lasx(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_lasx = {
.data2 = raid6_2data_recov_lasx,
.datap = raid6_datap_recov_lasx,
- .valid = raid6_has_lasx,
.name = "lasx",
- .priority = 2,
};
#endif /* CONFIG_CPU_HAS_LASX */
diff --git a/lib/raid/raid6/powerpc/altivec.uc b/lib/raid/raid6/powerpc/altivec.uc
index eb4a448cc88e..c5429fb71dd6 100644
--- a/lib/raid/raid6/powerpc/altivec.uc
+++ b/lib/raid/raid6/powerpc/altivec.uc
@@ -104,17 +104,7 @@ static void raid6_altivec$#_gen_syndrome(int disks, size_t bytes, void **ptrs)
preempt_enable();
}
-int raid6_have_altivec(void);
-#if $# == 1
-int raid6_have_altivec(void)
-{
- /* This assumes either all CPUs have Altivec or none does */
- return cpu_has_feature(CPU_FTR_ALTIVEC);
-}
-#endif
-
const struct raid6_calls raid6_altivec$# = {
.gen_syndrome = raid6_altivec$#_gen_syndrome,
- .valid = raid6_have_altivec,
.name = "altivecx$#",
};
diff --git a/lib/raid/raid6/powerpc/pq_arch.h b/lib/raid/raid6/powerpc/pq_arch.h
new file mode 100644
index 000000000000..ecae7d3be131
--- /dev/null
+++ b/lib/raid/raid6/powerpc/pq_arch.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <asm/cputable.h>
+
+extern const struct raid6_calls raid6_altivec1;
+extern const struct raid6_calls raid6_altivec2;
+extern const struct raid6_calls raid6_altivec4;
+extern const struct raid6_calls raid6_altivec8;
+extern const struct raid6_calls raid6_vpermxor1;
+extern const struct raid6_calls raid6_vpermxor2;
+extern const struct raid6_calls raid6_vpermxor4;
+extern const struct raid6_calls raid6_vpermxor8;
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ /* This assumes either all CPUs have Altivec or none does */
+ if (cpu_has_feature(CPU_FTR_ALTIVEC)) {
+ raid6_algo_add(&raid6_altivec1);
+ raid6_algo_add(&raid6_altivec2);
+ raid6_algo_add(&raid6_altivec4);
+ raid6_algo_add(&raid6_altivec8);
+ }
+ if (cpu_has_feature(CPU_FTR_ALTIVEC_COMP) &&
+ cpu_has_feature(CPU_FTR_ARCH_207S)) {
+ raid6_algo_add(&raid6_vpermxor1);
+ raid6_algo_add(&raid6_vpermxor2);
+ raid6_algo_add(&raid6_vpermxor4);
+ raid6_algo_add(&raid6_vpermxor8);
+ }
+ raid6_algo_add_default();
+}
diff --git a/lib/raid/raid6/powerpc/vpermxor.uc b/lib/raid/raid6/powerpc/vpermxor.uc
index ec61f30bec11..e8964361aaef 100644
--- a/lib/raid/raid6/powerpc/vpermxor.uc
+++ b/lib/raid/raid6/powerpc/vpermxor.uc
@@ -76,18 +76,7 @@ static void raid6_vpermxor$#_gen_syndrome(int disks, size_t bytes, void **ptrs)
preempt_enable();
}
-int raid6_have_altivec_vpermxor(void);
-#if $# == 1
-int raid6_have_altivec_vpermxor(void)
-{
- /* Check if arch has both altivec and the vpermxor instructions */
- return (cpu_has_feature(CPU_FTR_ALTIVEC_COMP) &&
- cpu_has_feature(CPU_FTR_ARCH_207S));
-}
-#endif
-
const struct raid6_calls raid6_vpermxor$# = {
.gen_syndrome = raid6_vpermxor$#_gen_syndrome,
- .valid = raid6_have_altivec_vpermxor,
.name = "vpermxor$#",
};
diff --git a/lib/raid/raid6/recov.c b/lib/raid/raid6/recov.c
index 735ab4013771..76eb2aef3667 100644
--- a/lib/raid/raid6/recov.c
+++ b/lib/raid/raid6/recov.c
@@ -97,7 +97,5 @@ static void raid6_datap_recov_intx1(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_intx1 = {
.data2 = raid6_2data_recov_intx1,
.datap = raid6_datap_recov_intx1,
- .valid = NULL,
.name = "intx1",
- .priority = 0,
};
diff --git a/lib/raid/raid6/riscv/pq_arch.h b/lib/raid/raid6/riscv/pq_arch.h
new file mode 100644
index 000000000000..52dd01e9fc42
--- /dev/null
+++ b/lib/raid/raid6/riscv/pq_arch.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <asm/vector.h>
+
+extern const struct raid6_calls raid6_rvvx1;
+extern const struct raid6_calls raid6_rvvx2;
+extern const struct raid6_calls raid6_rvvx4;
+extern const struct raid6_calls raid6_rvvx8;
+extern const struct raid6_recov_calls raid6_recov_rvv;
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ if (has_vector()) {
+ raid6_algo_add(&raid6_rvvx1);
+ raid6_algo_add(&raid6_rvvx2);
+ raid6_algo_add(&raid6_rvvx4);
+ raid6_algo_add(&raid6_rvvx8);
+ raid6_recov_algo_add(&raid6_recov_rvv);
+ }
+ raid6_algo_add_default();
+}
diff --git a/lib/raid/raid6/riscv/recov_rvv.c b/lib/raid/raid6/riscv/recov_rvv.c
index 02120d245e22..2305940276dd 100644
--- a/lib/raid/raid6/riscv/recov_rvv.c
+++ b/lib/raid/raid6/riscv/recov_rvv.c
@@ -218,7 +218,5 @@ static void raid6_datap_recov_rvv(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_rvv = {
.data2 = raid6_2data_recov_rvv,
.datap = raid6_datap_recov_rvv,
- .valid = rvv_has_vector,
.name = "rvv",
- .priority = 1,
};
diff --git a/lib/raid/raid6/riscv/rvv.h b/lib/raid/raid6/riscv/rvv.h
index c293130d798b..3a7c2468b1ea 100644
--- a/lib/raid/raid6/riscv/rvv.h
+++ b/lib/raid/raid6/riscv/rvv.h
@@ -10,11 +10,6 @@
#include <asm/vector.h>
#include "algos.h"
-static int rvv_has_vector(void)
-{
- return has_vector();
-}
-
#define RAID6_RVV_WRAPPER(_n) \
static void raid6_rvv ## _n ## _gen_syndrome(int disks, \
size_t bytes, void **ptrs) \
@@ -41,6 +36,5 @@ static int rvv_has_vector(void)
struct raid6_calls const raid6_rvvx ## _n = { \
.gen_syndrome = raid6_rvv ## _n ## _gen_syndrome, \
.xor_syndrome = raid6_rvv ## _n ## _xor_syndrome, \
- .valid = rvv_has_vector, \
.name = "rvvx" #_n, \
}
diff --git a/lib/raid/raid6/s390/pq_arch.h b/lib/raid/raid6/s390/pq_arch.h
new file mode 100644
index 000000000000..95d14c342306
--- /dev/null
+++ b/lib/raid/raid6/s390/pq_arch.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <linux/cpufeature.h>
+
+extern const struct raid6_calls raid6_s390vx8;
+extern const struct raid6_recov_calls raid6_recov_s390xc;
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ if (cpu_has_vx())
+ raid6_algo_add(&raid6_s390vx8);
+ else
+ raid6_algo_add_default();
+ raid6_recov_algo_add(&raid6_recov_s390xc);
+}
diff --git a/lib/raid/raid6/s390/recov_s390xc.c b/lib/raid/raid6/s390/recov_s390xc.c
index e7b3409f21e2..08d56896e5ea 100644
--- a/lib/raid/raid6/s390/recov_s390xc.c
+++ b/lib/raid/raid6/s390/recov_s390xc.c
@@ -112,7 +112,5 @@ static void raid6_datap_recov_s390xc(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_s390xc = {
.data2 = raid6_2data_recov_s390xc,
.datap = raid6_datap_recov_s390xc,
- .valid = NULL,
.name = "s390xc",
- .priority = 1,
};
diff --git a/lib/raid/raid6/s390/s390vx.uc b/lib/raid/raid6/s390/s390vx.uc
index aba3515eacac..e5cf9054be2a 100644
--- a/lib/raid/raid6/s390/s390vx.uc
+++ b/lib/raid/raid6/s390/s390vx.uc
@@ -121,15 +121,8 @@ static void raid6_s390vx$#_xor_syndrome(int disks, int start, int stop,
kernel_fpu_end(&vxstate, KERNEL_VXR);
}
-static int raid6_s390vx$#_valid(void)
-{
- return cpu_has_vx();
-}
-
const struct raid6_calls raid6_s390vx$# = {
.gen_syndrome = raid6_s390vx$#_gen_syndrome,
.xor_syndrome = raid6_s390vx$#_xor_syndrome,
- .valid = raid6_s390vx$#_valid,
.name = "vx128x$#",
- .priority = 1,
};
diff --git a/lib/raid/raid6/tests/raid6_kunit.c b/lib/raid/raid6/tests/raid6_kunit.c
index 9b71f22fa19a..daaa28e96ff1 100644
--- a/lib/raid/raid6/tests/raid6_kunit.c
+++ b/lib/raid/raid6/tests/raid6_kunit.c
@@ -88,19 +88,20 @@ static void test_disks(struct kunit *test, const struct raid6_calls *calls,
static void raid6_test(struct kunit *test)
{
- const struct raid6_calls *const *algo;
- const struct raid6_recov_calls *const *ra;
int i, j, p1, p2;
+ unsigned int r, g;
- for (ra = raid6_recov_algos; *ra; ra++) {
- if ((*ra)->valid && !(*ra)->valid())
- continue;
+ for (r = 0; ; r++) {
+ const struct raid6_recov_calls *ra = raid6_recov_algo_find(r);
- for (algo = raid6_algos; *algo; algo++) {
- const struct raid6_calls *calls = *algo;
+ if (!ra)
+ break;
- if (calls->valid && !calls->valid())
- continue;
+ for (g = 0; ; g++) {
+ const struct raid6_calls *calls = raid6_algo_find(g);
+
+ if (!calls)
+ break;
/* Nuke syndromes */
memset(data[NDISKS - 2], 0xee, PAGE_SIZE);
@@ -112,7 +113,7 @@ static void raid6_test(struct kunit *test)
for (i = 0; i < NDISKS-1; i++)
for (j = i+1; j < NDISKS; j++)
- test_disks(test, calls, *ra, i, j);
+ test_disks(test, calls, ra, i, j);
if (!calls->xor_syndrome)
continue;
@@ -130,7 +131,7 @@ static void raid6_test(struct kunit *test)
for (i = 0; i < NDISKS-1; i++)
for (j = i+1; j < NDISKS; j++)
test_disks(test, calls,
- *ra, i, j);
+ ra, i, j);
}
}
diff --git a/lib/raid/raid6/x86/avx2.c b/lib/raid/raid6/x86/avx2.c
index 0bf831799082..7efd94e6a87a 100644
--- a/lib/raid/raid6/x86/avx2.c
+++ b/lib/raid/raid6/x86/avx2.c
@@ -24,11 +24,6 @@ static const struct raid6_avx2_constants {
0x1d1d1d1d1d1d1d1dULL, 0x1d1d1d1d1d1d1d1dULL,},
};
-static int raid6_have_avx2(void)
-{
- return boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_AVX);
-}
-
/*
* Plain AVX2 implementation
*/
@@ -131,10 +126,7 @@ static void raid6_avx21_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx2x1 = {
.gen_syndrome = raid6_avx21_gen_syndrome,
.xor_syndrome = raid6_avx21_xor_syndrome,
- .valid = raid6_have_avx2,
.name = "avx2x1",
- /* Prefer AVX2 over priority 1 (SSE2 and others) */
- .priority = 2,
};
/*
@@ -262,10 +254,7 @@ static void raid6_avx22_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx2x2 = {
.gen_syndrome = raid6_avx22_gen_syndrome,
.xor_syndrome = raid6_avx22_xor_syndrome,
- .valid = raid6_have_avx2,
.name = "avx2x2",
- /* Prefer AVX2 over priority 1 (SSE2 and others) */
- .priority = 2,
};
#ifdef CONFIG_X86_64
@@ -466,9 +455,6 @@ static void raid6_avx24_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx2x4 = {
.gen_syndrome = raid6_avx24_gen_syndrome,
.xor_syndrome = raid6_avx24_xor_syndrome,
- .valid = raid6_have_avx2,
.name = "avx2x4",
- /* Prefer AVX2 over priority 1 (SSE2 and others) */
- .priority = 2,
};
#endif /* CONFIG_X86_64 */
diff --git a/lib/raid/raid6/x86/avx512.c b/lib/raid/raid6/x86/avx512.c
index 98ed42fb0a46..0772e798b742 100644
--- a/lib/raid/raid6/x86/avx512.c
+++ b/lib/raid/raid6/x86/avx512.c
@@ -30,16 +30,6 @@ static const struct raid6_avx512_constants {
0x1d1d1d1d1d1d1d1dULL, 0x1d1d1d1d1d1d1d1dULL,},
};
-static int raid6_have_avx512(void)
-{
- return boot_cpu_has(X86_FEATURE_AVX2) &&
- boot_cpu_has(X86_FEATURE_AVX) &&
- boot_cpu_has(X86_FEATURE_AVX512F) &&
- boot_cpu_has(X86_FEATURE_AVX512BW) &&
- boot_cpu_has(X86_FEATURE_AVX512VL) &&
- boot_cpu_has(X86_FEATURE_AVX512DQ);
-}
-
static void raid6_avx5121_gen_syndrome(int disks, size_t bytes, void **ptrs)
{
u8 **dptr = (u8 **)ptrs;
@@ -159,10 +149,7 @@ static void raid6_avx5121_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx512x1 = {
.gen_syndrome = raid6_avx5121_gen_syndrome,
.xor_syndrome = raid6_avx5121_xor_syndrome,
- .valid = raid6_have_avx512,
.name = "avx512x1",
- /* Prefer AVX512 over priority 1 (SSE2 and others) */
- .priority = 2,
};
/*
@@ -317,10 +304,7 @@ static void raid6_avx5122_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx512x2 = {
.gen_syndrome = raid6_avx5122_gen_syndrome,
.xor_syndrome = raid6_avx5122_xor_syndrome,
- .valid = raid6_have_avx512,
.name = "avx512x2",
- /* Prefer AVX512 over priority 1 (SSE2 and others) */
- .priority = 2,
};
#ifdef CONFIG_X86_64
@@ -556,9 +540,6 @@ static void raid6_avx5124_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_avx512x4 = {
.gen_syndrome = raid6_avx5124_gen_syndrome,
.xor_syndrome = raid6_avx5124_xor_syndrome,
- .valid = raid6_have_avx512,
.name = "avx512x4",
- /* Prefer AVX512 over priority 1 (SSE2 and others) */
- .priority = 2,
};
#endif
diff --git a/lib/raid/raid6/x86/mmx.c b/lib/raid/raid6/x86/mmx.c
index 052d9f010bfe..3228c335965a 100644
--- a/lib/raid/raid6/x86/mmx.c
+++ b/lib/raid/raid6/x86/mmx.c
@@ -22,12 +22,6 @@ const struct raid6_mmx_constants {
0x1d1d1d1d1d1d1d1dULL,
};
-static int raid6_have_mmx(void)
-{
- /* Not really "boot_cpu" but "all_cpus" */
- return boot_cpu_has(X86_FEATURE_MMX);
-}
-
/*
* Plain MMX implementation
*/
@@ -70,7 +64,6 @@ static void raid6_mmx1_gen_syndrome(int disks, size_t bytes, void **ptrs)
const struct raid6_calls raid6_mmxx1 = {
.gen_syndrome = raid6_mmx1_gen_syndrome,
- .valid = raid6_have_mmx,
.name = "mmxx1",
};
@@ -127,6 +120,5 @@ static void raid6_mmx2_gen_syndrome(int disks, size_t bytes, void **ptrs)
const struct raid6_calls raid6_mmxx2 = {
.gen_syndrome = raid6_mmx2_gen_syndrome,
- .valid = raid6_have_mmx,
.name = "mmxx2",
};
diff --git a/lib/raid/raid6/x86/pq_arch.h b/lib/raid/raid6/x86/pq_arch.h
new file mode 100644
index 000000000000..0493fc3a089c
--- /dev/null
+++ b/lib/raid/raid6/x86/pq_arch.h
@@ -0,0 +1,97 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include <asm/cpufeature.h>
+
+extern const struct raid6_calls raid6_mmxx1;
+extern const struct raid6_calls raid6_mmxx2;
+extern const struct raid6_calls raid6_sse1x1;
+extern const struct raid6_calls raid6_sse1x2;
+extern const struct raid6_calls raid6_sse2x1;
+extern const struct raid6_calls raid6_sse2x2;
+extern const struct raid6_calls raid6_sse2x4;
+extern const struct raid6_calls raid6_avx2x1;
+extern const struct raid6_calls raid6_avx2x2;
+extern const struct raid6_calls raid6_avx2x4;
+extern const struct raid6_calls raid6_avx512x1;
+extern const struct raid6_calls raid6_avx512x2;
+extern const struct raid6_calls raid6_avx512x4;
+
+extern const struct raid6_recov_calls raid6_recov_ssse3;
+extern const struct raid6_recov_calls raid6_recov_avx2;
+extern const struct raid6_recov_calls raid6_recov_avx512;
+
+static inline int raid6_has_avx512(void)
+{
+ return boot_cpu_has(X86_FEATURE_AVX2) &&
+ boot_cpu_has(X86_FEATURE_AVX) &&
+ boot_cpu_has(X86_FEATURE_AVX512F) &&
+ boot_cpu_has(X86_FEATURE_AVX512BW) &&
+ boot_cpu_has(X86_FEATURE_AVX512VL) &&
+ boot_cpu_has(X86_FEATURE_AVX512DQ);
+}
+
+static inline bool raid6_has_avx2(void)
+{
+ return boot_cpu_has(X86_FEATURE_AVX2) && boot_cpu_has(X86_FEATURE_AVX);
+}
+
+static inline bool raid6_has_ssse3(void)
+{
+ return boot_cpu_has(X86_FEATURE_XMM) &&
+ boot_cpu_has(X86_FEATURE_XMM2) &&
+ boot_cpu_has(X86_FEATURE_SSSE3);
+}
+
+static inline bool raid6_has_sse2(void)
+{
+ return boot_cpu_has(X86_FEATURE_MMX) &&
+ boot_cpu_has(X86_FEATURE_FXSR) &&
+ boot_cpu_has(X86_FEATURE_XMM) &&
+ boot_cpu_has(X86_FEATURE_XMM2);
+}
+
+static inline bool raid6_has_sse1_or_mmxext(void)
+{
+ return boot_cpu_has(X86_FEATURE_MMX) &&
+ (boot_cpu_has(X86_FEATURE_XMM) ||
+ boot_cpu_has(X86_FEATURE_MMXEXT));
+}
+
+static __always_inline void __init arch_raid6_init(void)
+{
+ if (raid6_has_avx2()) {
+ if (raid6_has_avx512()) {
+ raid6_algo_add(&raid6_avx512x1);
+ raid6_algo_add(&raid6_avx512x2);
+ if (IS_ENABLED(CONFIG_X86_64))
+ raid6_algo_add(&raid6_avx512x4);
+ }
+ raid6_algo_add(&raid6_avx2x1);
+ raid6_algo_add(&raid6_avx2x2);
+ if (IS_ENABLED(CONFIG_X86_64))
+ raid6_algo_add(&raid6_avx2x4);
+ } else if (IS_ENABLED(CONFIG_X86_64) || raid6_has_sse2()) {
+ /* x86_64 can assume SSE2 as baseline */
+ raid6_algo_add(&raid6_sse2x1);
+ raid6_algo_add(&raid6_sse2x2);
+ if (IS_ENABLED(CONFIG_X86_64))
+ raid6_algo_add(&raid6_sse2x4);
+ } else if (raid6_has_sse1_or_mmxext()) {
+ raid6_algo_add(&raid6_sse1x1);
+ raid6_algo_add(&raid6_sse1x2);
+ raid6_algo_add_default();
+ } else {
+ if (boot_cpu_has(X86_FEATURE_MMX)) {
+ raid6_algo_add(&raid6_mmxx1);
+ raid6_algo_add(&raid6_mmxx2);
+ }
+ raid6_algo_add_default();
+ }
+
+ if (raid6_has_avx512())
+ raid6_recov_algo_add(&raid6_recov_avx512);
+ else if (raid6_has_avx2())
+ raid6_recov_algo_add(&raid6_recov_avx2);
+ else if (raid6_has_ssse3())
+ raid6_recov_algo_add(&raid6_recov_ssse3);
+}
diff --git a/lib/raid/raid6/x86/recov_avx2.c b/lib/raid/raid6/x86/recov_avx2.c
index 06c6e05763bc..a714a780a2d8 100644
--- a/lib/raid/raid6/x86/recov_avx2.c
+++ b/lib/raid/raid6/x86/recov_avx2.c
@@ -9,12 +9,6 @@
#include <asm/fpu/api.h>
#include "algos.h"
-static int raid6_has_avx2(void)
-{
- return boot_cpu_has(X86_FEATURE_AVX2) &&
- boot_cpu_has(X86_FEATURE_AVX);
-}
-
static void raid6_2data_recov_avx2(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -305,11 +299,9 @@ static void raid6_datap_recov_avx2(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_avx2 = {
.data2 = raid6_2data_recov_avx2,
.datap = raid6_datap_recov_avx2,
- .valid = raid6_has_avx2,
#ifdef CONFIG_X86_64
.name = "avx2x2",
#else
.name = "avx2x1",
#endif
- .priority = 2,
};
diff --git a/lib/raid/raid6/x86/recov_avx512.c b/lib/raid/raid6/x86/recov_avx512.c
index 850bb962b514..ec72d5a30c01 100644
--- a/lib/raid/raid6/x86/recov_avx512.c
+++ b/lib/raid/raid6/x86/recov_avx512.c
@@ -11,16 +11,6 @@
#include <asm/fpu/api.h>
#include "algos.h"
-static int raid6_has_avx512(void)
-{
- return boot_cpu_has(X86_FEATURE_AVX2) &&
- boot_cpu_has(X86_FEATURE_AVX) &&
- boot_cpu_has(X86_FEATURE_AVX512F) &&
- boot_cpu_has(X86_FEATURE_AVX512BW) &&
- boot_cpu_has(X86_FEATURE_AVX512VL) &&
- boot_cpu_has(X86_FEATURE_AVX512DQ);
-}
-
static void raid6_2data_recov_avx512(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -369,11 +359,9 @@ static void raid6_datap_recov_avx512(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_avx512 = {
.data2 = raid6_2data_recov_avx512,
.datap = raid6_datap_recov_avx512,
- .valid = raid6_has_avx512,
#ifdef CONFIG_X86_64
.name = "avx512x2",
#else
.name = "avx512x1",
#endif
- .priority = 3,
};
diff --git a/lib/raid/raid6/x86/recov_ssse3.c b/lib/raid/raid6/x86/recov_ssse3.c
index 95589c33003a..700bd2c865ec 100644
--- a/lib/raid/raid6/x86/recov_ssse3.c
+++ b/lib/raid/raid6/x86/recov_ssse3.c
@@ -8,13 +8,6 @@
#include <asm/fpu/api.h>
#include "algos.h"
-static int raid6_has_ssse3(void)
-{
- return boot_cpu_has(X86_FEATURE_XMM) &&
- boot_cpu_has(X86_FEATURE_XMM2) &&
- boot_cpu_has(X86_FEATURE_SSSE3);
-}
-
static void raid6_2data_recov_ssse3(int disks, size_t bytes, int faila,
int failb, void **ptrs)
{
@@ -320,11 +313,9 @@ static void raid6_datap_recov_ssse3(int disks, size_t bytes, int faila,
const struct raid6_recov_calls raid6_recov_ssse3 = {
.data2 = raid6_2data_recov_ssse3,
.datap = raid6_datap_recov_ssse3,
- .valid = raid6_has_ssse3,
#ifdef CONFIG_X86_64
.name = "ssse3x2",
#else
.name = "ssse3x1",
#endif
- .priority = 1,
};
diff --git a/lib/raid/raid6/x86/sse1.c b/lib/raid/raid6/x86/sse1.c
index 7004255a0bb1..6ebdcf824e00 100644
--- a/lib/raid/raid6/x86/sse1.c
+++ b/lib/raid/raid6/x86/sse1.c
@@ -25,14 +25,6 @@ extern const struct raid6_mmx_constants {
u64 x1d;
} raid6_mmx_constants;
-static int raid6_have_sse1_or_mmxext(void)
-{
- /* Not really boot_cpu but "all_cpus" */
- return boot_cpu_has(X86_FEATURE_MMX) &&
- (boot_cpu_has(X86_FEATURE_XMM) ||
- boot_cpu_has(X86_FEATURE_MMXEXT));
-}
-
/*
* Plain SSE1 implementation
*/
@@ -86,9 +78,7 @@ static void raid6_sse11_gen_syndrome(int disks, size_t bytes, void **ptrs)
const struct raid6_calls raid6_sse1x1 = {
.gen_syndrome = raid6_sse11_gen_syndrome,
- .valid = raid6_have_sse1_or_mmxext,
.name = "sse1x1",
- .priority = 1, /* Has cache hints */
};
/*
@@ -148,7 +138,5 @@ static void raid6_sse12_gen_syndrome(int disks, size_t bytes, void **ptrs)
const struct raid6_calls raid6_sse1x2 = {
.gen_syndrome = raid6_sse12_gen_syndrome,
- .valid = raid6_have_sse1_or_mmxext,
.name = "sse1x2",
- .priority = 1, /* Has cache hints */
};
diff --git a/lib/raid/raid6/x86/sse2.c b/lib/raid/raid6/x86/sse2.c
index f30be4ee14d0..7049c8512f35 100644
--- a/lib/raid/raid6/x86/sse2.c
+++ b/lib/raid/raid6/x86/sse2.c
@@ -22,15 +22,6 @@ static const struct raid6_sse_constants {
{ 0x1d1d1d1d1d1d1d1dULL, 0x1d1d1d1d1d1d1d1dULL },
};
-static int raid6_have_sse2(void)
-{
- /* Not really boot_cpu but "all_cpus" */
- return boot_cpu_has(X86_FEATURE_MMX) &&
- boot_cpu_has(X86_FEATURE_FXSR) &&
- boot_cpu_has(X86_FEATURE_XMM) &&
- boot_cpu_has(X86_FEATURE_XMM2);
-}
-
/*
* Plain SSE2 implementation
*/
@@ -136,9 +127,7 @@ static void raid6_sse21_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_sse2x1 = {
.gen_syndrome = raid6_sse21_gen_syndrome,
.xor_syndrome = raid6_sse21_xor_syndrome,
- .valid = raid6_have_sse2,
.name = "sse2x1",
- .priority = 1, /* Has cache hints */
};
/*
@@ -266,9 +255,7 @@ static void raid6_sse22_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_sse2x2 = {
.gen_syndrome = raid6_sse22_gen_syndrome,
.xor_syndrome = raid6_sse22_xor_syndrome,
- .valid = raid6_have_sse2,
.name = "sse2x2",
- .priority = 1, /* Has cache hints */
};
#ifdef CONFIG_X86_64
@@ -473,9 +460,7 @@ static void raid6_sse24_xor_syndrome(int disks, int start, int stop,
const struct raid6_calls raid6_sse2x4 = {
.gen_syndrome = raid6_sse24_gen_syndrome,
.xor_syndrome = raid6_sse24_xor_syndrome,
- .valid = raid6_have_sse2,
.name = "sse2x4",
- .priority = 1, /* Has cache hints */
};
#endif /* CONFIG_X86_64 */
--
2.53.0
^ permalink raw reply related
* [PATCH 08/19] raid6: improve the public interface
From: Christoph Hellwig @ 2026-05-12 5:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Huacai Chen,
WANG Xuerui, Madhavan Srinivasan, Michael Ellerman,
Nicholas Piggin, Christophe Leroy (CS GROUP), Paul Walmsley,
Palmer Dabbelt, Albert Ou, Alexandre Ghiti, Heiko Carstens,
Vasily Gorbik, Alexander Gordeev, Christian Borntraeger,
Sven Schnelle, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, x86, H. Peter Anvin, Herbert Xu, Dan Williams,
Chris Mason, David Sterba, Arnd Bergmann, Song Liu, Yu Kuai,
Li Nan, linux-kernel, linux-arm-kernel, loongarch, linuxppc-dev,
linux-riscv, linux-s390, linux-crypto, linux-btrfs, linux-arch,
linux-raid
In-Reply-To: <20260512052230.2947683-1-hch@lst.de>
Stop directly calling into function pointers from users of the RAID6 PQ
API, and provide exported functions with proper documentation and
API guarantees asserts where applicable instead.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
Documentation/crypto/async-tx-api.rst | 4 +-
crypto/async_tx/async_pq.c | 6 +-
crypto/async_tx/async_raid6_recov.c | 4 +-
drivers/md/raid5.c | 4 +-
fs/btrfs/raid56.c | 8 +-
include/linux/raid/pq.h | 28 ++--
lib/raid/raid6/algos.c | 139 +++++++++++++++++-
lib/raid/raid6/arm/recov_neon.c | 4 +-
.../raid6/loongarch/recov_loongarch_simd.c | 8 +-
lib/raid/raid6/recov.c | 4 +-
lib/raid/raid6/riscv/recov_rvv.c | 4 +-
lib/raid/raid6/s390/recov_s390xc.c | 4 +-
lib/raid/raid6/x86/recov_avx2.c | 4 +-
lib/raid/raid6/x86/recov_avx512.c | 4 +-
lib/raid/raid6/x86/recov_ssse3.c | 4 +-
15 files changed, 181 insertions(+), 48 deletions(-)
diff --git a/Documentation/crypto/async-tx-api.rst b/Documentation/crypto/async-tx-api.rst
index f88a7809385e..49fcfc66314a 100644
--- a/Documentation/crypto/async-tx-api.rst
+++ b/Documentation/crypto/async-tx-api.rst
@@ -82,9 +82,9 @@ xor_val xor a series of source buffers and set a flag if the
pq generate the p+q (raid6 syndrome) from a series of source buffers
pq_val validate that a p and or q buffer are in sync with a given series of
sources
-datap (raid6_datap_recov) recover a raid6 data block and the p block
+datap (raid6_recov_datap) recover a raid6 data block and the p block
from the given sources
-2data (raid6_2data_recov) recover 2 raid6 data blocks from the given
+2data (raid6_recov_2data) recover 2 raid6 data blocks from the given
sources
======== ====================================================================
diff --git a/crypto/async_tx/async_pq.c b/crypto/async_tx/async_pq.c
index 0ce6f07b4e0d..f3574f80d1df 100644
--- a/crypto/async_tx/async_pq.c
+++ b/crypto/async_tx/async_pq.c
@@ -131,11 +131,11 @@ do_sync_gen_syndrome(struct page **blocks, unsigned int *offsets, int disks,
}
}
if (submit->flags & ASYNC_TX_PQ_XOR_DST) {
- BUG_ON(!raid6_call.xor_syndrome);
+ BUG_ON(!raid6_can_xor_syndrome());
if (start >= 0)
- raid6_call.xor_syndrome(disks, start, stop, len, srcs);
+ raid6_xor_syndrome(disks, start, stop, len, srcs);
} else
- raid6_call.gen_syndrome(disks, len, srcs);
+ raid6_gen_syndrome(disks, len, srcs);
async_tx_sync_epilog(submit);
}
diff --git a/crypto/async_tx/async_raid6_recov.c b/crypto/async_tx/async_raid6_recov.c
index f2dc6af6e6a7..305ea1421a3e 100644
--- a/crypto/async_tx/async_raid6_recov.c
+++ b/crypto/async_tx/async_raid6_recov.c
@@ -418,7 +418,7 @@ async_raid6_2data_recov(int disks, size_t bytes, int faila, int failb,
else
ptrs[i] = page_address(blocks[i]) + offs[i];
- raid6_2data_recov(disks, bytes, faila, failb, ptrs);
+ raid6_recov_2data(disks, bytes, faila, failb, ptrs);
async_tx_sync_epilog(submit);
@@ -501,7 +501,7 @@ async_raid6_datap_recov(int disks, size_t bytes, int faila,
else
ptrs[i] = page_address(blocks[i]) + offs[i];
- raid6_datap_recov(disks, bytes, faila, ptrs);
+ raid6_recov_datap(disks, bytes, faila, ptrs);
async_tx_sync_epilog(submit);
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 0d76e82f4506..ebcb19317670 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -6955,7 +6955,7 @@ raid5_store_rmw_level(struct mddev *mddev, const char *page, size_t len)
if (kstrtoul(page, 10, &new))
return -EINVAL;
- if (new != PARITY_DISABLE_RMW && !raid6_call.xor_syndrome)
+ if (new != PARITY_DISABLE_RMW && !raid6_can_xor_syndrome())
return -EINVAL;
if (new != PARITY_DISABLE_RMW &&
@@ -7646,7 +7646,7 @@ static struct r5conf *setup_conf(struct mddev *mddev)
conf->level = mddev->new_level;
if (conf->level == 6) {
conf->max_degraded = 2;
- if (raid6_call.xor_syndrome)
+ if (raid6_can_xor_syndrome())
conf->rmw_level = PARITY_ENABLE_RMW;
else
conf->rmw_level = PARITY_DISABLE_RMW;
diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c
index 08ee8f316d96..dabc9522e881 100644
--- a/fs/btrfs/raid56.c
+++ b/fs/btrfs/raid56.c
@@ -1410,7 +1410,7 @@ static void generate_pq_vertical_step(struct btrfs_raid_bio *rbio, unsigned int
rbio_qstripe_paddr(rbio, sector_nr, step_nr));
assert_rbio(rbio);
- raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
+ raid6_gen_syndrome(rbio->real_stripes, step, pointers);
} else {
/* raid5 */
memcpy(pointers[rbio->nr_data], pointers[0], step);
@@ -1987,10 +1987,10 @@ static void recover_vertical_step(struct btrfs_raid_bio *rbio,
}
if (failb == rbio->real_stripes - 2) {
- raid6_datap_recov(rbio->real_stripes, step,
+ raid6_recov_datap(rbio->real_stripes, step,
faila, pointers);
} else {
- raid6_2data_recov(rbio->real_stripes, step,
+ raid6_recov_2data(rbio->real_stripes, step,
faila, failb, pointers);
}
} else {
@@ -2644,7 +2644,7 @@ static bool verify_one_parity_step(struct btrfs_raid_bio *rbio,
if (has_qstripe) {
assert_rbio(rbio);
/* RAID6, call the library function to fill in our P/Q. */
- raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
+ raid6_gen_syndrome(rbio->real_stripes, step, pointers);
} else {
/* RAID5. */
memcpy(pointers[nr_data], pointers[0], step);
diff --git a/include/linux/raid/pq.h b/include/linux/raid/pq.h
index f27a866c287f..662c2669f63f 100644
--- a/include/linux/raid/pq.h
+++ b/include/linux/raid/pq.h
@@ -11,6 +11,25 @@
#include <linux/blkdev.h>
#include <linux/mm.h>
+/*
+ * While the RAID6 algorithm could in theory support 3 devices by just copying
+ * the data disk to the two parity disks, this configuration is not only useless
+ * because it is a suboptimal version of 3-way mirroring, but also easy to get
+ * wrong in architecture-optimized implementations due to special casing, so
+ * don't support it.
+ */
+#define RAID6_MIN_DISKS 4
+
+void raid6_gen_syndrome(int disks, size_t bytes, void **ptrs);
+void raid6_xor_syndrome(int disks, int start, int stop, size_t bytes,
+ void **ptrs);
+bool raid6_can_xor_syndrome(void);
+
+void raid6_recov_2data(int disks, size_t bytes, int faila, int failb,
+ void **ptrs);
+void raid6_recov_datap(int disks, size_t bytes, int faila,
+ void **ptrs);
+
/* Routine choices */
struct raid6_calls {
void (*gen_syndrome)(int, size_t, void **);
@@ -20,9 +39,6 @@ struct raid6_calls {
int priority; /* Relative priority ranking if non-zero */
};
-/* Selected algorithm */
-extern struct raid6_calls raid6_call;
-
/* Various routine sets */
extern const struct raid6_calls raid6_intx1;
extern const struct raid6_calls raid6_intx2;
@@ -92,10 +108,4 @@ extern const u8 raid6_gflog[256] __attribute__((aligned(256)));
extern const u8 raid6_gfinv[256] __attribute__((aligned(256)));
extern const u8 raid6_gfexi[256] __attribute__((aligned(256)));
-/* Recovery routines */
-extern void (*raid6_2data_recov)(int disks, size_t bytes, int faila, int failb,
- void **ptrs);
-extern void (*raid6_datap_recov)(int disks, size_t bytes, int faila,
- void **ptrs);
-
#endif /* LINUX_RAID_RAID6_H */
diff --git a/lib/raid/raid6/algos.c b/lib/raid/raid6/algos.c
index 985c60bb00a4..683b97cb94ad 100644
--- a/lib/raid/raid6/algos.c
+++ b/lib/raid/raid6/algos.c
@@ -16,8 +16,85 @@
#include <linux/gfp.h>
#include <kunit/visibility.h>
-struct raid6_calls raid6_call;
-EXPORT_SYMBOL_GPL(raid6_call);
+static const struct raid6_recov_calls *raid6_recov_algo;
+
+/* Selected algorithm */
+static struct raid6_calls raid6_call;
+
+/**
+ * raid6_gen_syndrome - generate RAID6 P/Q parity
+ * @disks: number of "disks" to operate on including parity
+ * @bytes: length in bytes of each vector
+ * @ptrs: @disks size array of memory pointers
+ *
+ * Generate @bytes worth of RAID6 P and Q parity in @ptrs[@disks - 2] and
+ * @ptrs[@disks - 1] respectively from the memory pointed to by @ptrs[0] to
+ * @ptrs[@disks - 3].
+ *
+ * @disks must be at least 3, and the memory pointed to by each member of @ptrs
+ * must be at least 64-byte aligned. @bytes must be non-zero and a multiple of
+ * 512.
+ *
+ * See https://kernel.org/pub/linux/kernel/people/hpa/raid6.pdf for underlying
+ * algorithm.
+ */
+void raid6_gen_syndrome(int disks, size_t bytes, void **ptrs)
+{
+ WARN_ON_ONCE(!in_task() || irqs_disabled() || softirq_count());
+ WARN_ON_ONCE(bytes & 511);
+ WARN_ON_ONCE(disks < RAID6_MIN_DISKS);
+
+ raid6_call.gen_syndrome(disks, bytes, ptrs);
+}
+EXPORT_SYMBOL_GPL(raid6_gen_syndrome);
+
+/**
+ * raid6_xor_syndrome - update RAID6 P/Q parity
+ * @disks: number of "disks" to operate on including parity
+ * @start: first index into @disk to update
+ * @stop: last index into @disk to update
+ * @bytes: length in bytes of each vector
+ * @ptrs: @disks size array of memory pointers
+ *
+ * Update @bytes worth of RAID6 P and Q parity in @ptrs[@disks - 2] and
+ * @ptrs[@disks - 1] respectively for the memory pointed to by
+ * @ptrs[@start..@stop].
+ *
+ * This is used to update parity in place using the following sequence:
+ *
+ * 1) call raid6_xor_syndrome(disk, start, stop, ...) for the existing data.
+ * 2) update the the data in @ptrs[@start..@stop].
+ * 3) call raid6_xor_syndrome(disk, start, stop, ...) for the new data.
+ *
+ * Data between @start and @stop that is not changed should be filled
+ * with a pointer to the kernel zero page.
+ *
+ * @disks must be at least 3, and the memory pointed to by each member of @ptrs
+ * must be at least 64-byte aligned. @bytes must be non-zero and a multiple of
+ * 512. @stop must be larger or equal to @start.
+ */
+void raid6_xor_syndrome(int disks, int start, int stop, size_t bytes,
+ void **ptrs)
+{
+ WARN_ON_ONCE(!in_task() || irqs_disabled() || softirq_count());
+ WARN_ON_ONCE(bytes & 511);
+ WARN_ON_ONCE(disks < RAID6_MIN_DISKS);
+ WARN_ON_ONCE(stop < start);
+
+ raid6_call.xor_syndrome(disks, start, stop, bytes, ptrs);
+}
+EXPORT_SYMBOL_GPL(raid6_xor_syndrome);
+
+/*
+ * raid6_can_xor_syndrome - check if raid6_xor_syndrome() can be used
+ *
+ * Returns %true if raid6_can_xor_syndrome() can be used, else %false.
+ */
+bool raid6_can_xor_syndrome(void)
+{
+ return !!raid6_call.xor_syndrome;
+}
+EXPORT_SYMBOL_GPL(raid6_can_xor_syndrome);
const struct raid6_calls * const raid6_algos[] = {
#if defined(__i386__) && !defined(__arch_um__)
@@ -84,11 +161,58 @@ const struct raid6_calls * const raid6_algos[] = {
};
EXPORT_SYMBOL_IF_KUNIT(raid6_algos);
-void (*raid6_2data_recov)(int, size_t, int, int, void **);
-EXPORT_SYMBOL_GPL(raid6_2data_recov);
+/**
+ * raid6_recov_2data - recover two missing data disks
+ * @disks: number of "disks" to operate on including parity
+ * @bytes: length in bytes of each vector
+ * @faila: first failed data disk index
+ * @failb: second failed data disk index
+ * @ptrs: @disks size array of memory pointers
+ *
+ * Rebuild @bytes of missing data in @ptrs[@faila] and @ptrs[@failb] from the
+ * data in the remaining disks and the two parities pointed to by the other
+ * indices between 0 and @disks - 1 in @ptrs. @disks includes the data disks
+ * and the two parities. @faila must be smaller than @failb.
+ *
+ * Memory pointed to by each pointer in @ptrs must be page aligned and is
+ * limited to %PAGE_SIZE.
+ */
+void raid6_recov_2data(int disks, size_t bytes, int faila, int failb,
+ void **ptrs)
+{
+ WARN_ON_ONCE(!in_task() || irqs_disabled() || softirq_count());
+ WARN_ON_ONCE(bytes & 511);
+ WARN_ON_ONCE(bytes > PAGE_SIZE);
+ WARN_ON_ONCE(failb <= faila);
+
+ raid6_recov_algo->data2(disks, bytes, faila, failb, ptrs);
+}
+EXPORT_SYMBOL_GPL(raid6_recov_2data);
+
+/**
+ * raid6_recov_datap - recover a missing data disk and missing P-parity
+ * @disks: number of "disks" to operate on including parity
+ * @bytes: length in bytes of each vector
+ * @faila: failed data disk index
+ * @ptrs: @disks size array of memory pointers
+ *
+ * Rebuild @bytes of missing data in @ptrs[@faila] and the missing P-parity in
+ * @ptrs[@disks - 2] from the data in the remaining disks and the Q-parity
+ * pointed to by the other indices between 0 and @disks - 1 in @ptrs. @disks
+ * includes the data disks and the two parities.
+ *
+ * Memory pointed to by each pointer in @ptrs must be page aligned and is
+ * limited to %PAGE_SIZE.
+ */
+void raid6_recov_datap(int disks, size_t bytes, int faila, void **ptrs)
+{
+ WARN_ON_ONCE(!in_task() || irqs_disabled() || softirq_count());
+ WARN_ON_ONCE(bytes & 511);
+ WARN_ON_ONCE(bytes > PAGE_SIZE);
-void (*raid6_datap_recov)(int, size_t, int, void **);
-EXPORT_SYMBOL_GPL(raid6_datap_recov);
+ raid6_recov_algo->datap(disks, bytes, faila, ptrs);
+}
+EXPORT_SYMBOL_GPL(raid6_recov_datap);
const struct raid6_recov_calls *const raid6_recov_algos[] = {
#ifdef CONFIG_X86
@@ -133,8 +257,7 @@ static inline const struct raid6_recov_calls *raid6_choose_recov(void)
best = *algo;
if (best) {
- raid6_2data_recov = best->data2;
- raid6_datap_recov = best->datap;
+ raid6_recov_algo = best;
pr_info("raid6: using %s recovery algorithm\n", best->name);
} else
diff --git a/lib/raid/raid6/arm/recov_neon.c b/lib/raid/raid6/arm/recov_neon.c
index 9993bda5d3a6..4eb0efb44750 100644
--- a/lib/raid/raid6/arm/recov_neon.c
+++ b/lib/raid/raid6/arm/recov_neon.c
@@ -35,7 +35,7 @@ static void raid6_2data_recov_neon(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -69,7 +69,7 @@ static void raid6_datap_recov_neon(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/loongarch/recov_loongarch_simd.c b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
index 4d4563209647..7d4d349322b3 100644
--- a/lib/raid/raid6/loongarch/recov_loongarch_simd.c
+++ b/lib/raid/raid6/loongarch/recov_loongarch_simd.c
@@ -49,7 +49,7 @@ static void raid6_2data_recov_lsx(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -201,7 +201,7 @@ static void raid6_datap_recov_lsx(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
@@ -323,7 +323,7 @@ static void raid6_2data_recov_lasx(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -440,7 +440,7 @@ static void raid6_datap_recov_lasx(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/recov.c b/lib/raid/raid6/recov.c
index 211e1df28963..cc7e4dc1eaa6 100644
--- a/lib/raid/raid6/recov.c
+++ b/lib/raid/raid6/recov.c
@@ -37,7 +37,7 @@ static void raid6_2data_recov_intx1(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -75,7 +75,7 @@ static void raid6_datap_recov_intx1(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/riscv/recov_rvv.c b/lib/raid/raid6/riscv/recov_rvv.c
index f77d9c430687..3ff39826e33f 100644
--- a/lib/raid/raid6/riscv/recov_rvv.c
+++ b/lib/raid/raid6/riscv/recov_rvv.c
@@ -164,7 +164,7 @@ static void raid6_2data_recov_rvv(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -199,7 +199,7 @@ static void raid6_datap_recov_rvv(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks - 1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/s390/recov_s390xc.c b/lib/raid/raid6/s390/recov_s390xc.c
index 0f32217b7123..2bc4c85174de 100644
--- a/lib/raid/raid6/s390/recov_s390xc.c
+++ b/lib/raid/raid6/s390/recov_s390xc.c
@@ -40,7 +40,7 @@ static void raid6_2data_recov_s390xc(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -84,7 +84,7 @@ static void raid6_datap_recov_s390xc(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/x86/recov_avx2.c b/lib/raid/raid6/x86/recov_avx2.c
index 325310c81e1c..bef82a38d8eb 100644
--- a/lib/raid/raid6/x86/recov_avx2.c
+++ b/lib/raid/raid6/x86/recov_avx2.c
@@ -34,7 +34,7 @@ static void raid6_2data_recov_avx2(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -199,7 +199,7 @@ static void raid6_datap_recov_avx2(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/x86/recov_avx512.c b/lib/raid/raid6/x86/recov_avx512.c
index 08de77fcb8bd..06c70e771eaa 100644
--- a/lib/raid/raid6/x86/recov_avx512.c
+++ b/lib/raid/raid6/x86/recov_avx512.c
@@ -43,7 +43,7 @@ static void raid6_2data_recov_avx512(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -241,7 +241,7 @@ static void raid6_datap_recov_avx512(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
diff --git a/lib/raid/raid6/x86/recov_ssse3.c b/lib/raid/raid6/x86/recov_ssse3.c
index 002bef1e0847..5ca7d56f23d8 100644
--- a/lib/raid/raid6/x86/recov_ssse3.c
+++ b/lib/raid/raid6/x86/recov_ssse3.c
@@ -36,7 +36,7 @@ static void raid6_2data_recov_ssse3(int disks, size_t bytes, int faila,
ptrs[failb] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dp;
@@ -206,7 +206,7 @@ static void raid6_datap_recov_ssse3(int disks, size_t bytes, int faila,
ptrs[faila] = page_address(ZERO_PAGE(0));
ptrs[disks-1] = dq;
- raid6_call.gen_syndrome(disks, bytes, ptrs);
+ raid6_gen_syndrome(disks, bytes, ptrs);
/* Restore pointer table */
ptrs[faila] = dq;
--
2.53.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox