* Re: [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Christoph Hellwig @ 2026-05-26 6:46 UTC (permalink / raw)
To: Qu Wenruo; +Cc: dsterba, Christoph Hellwig, linux-btrfs, linux-raid
In-Reply-To: <e5eafd64-2704-42de-ad7b-e4f170c451e4@suse.com>
On Mon, May 25, 2026 at 08:09:54PM +0930, Qu Wenruo wrote:
>> It's a lazy hack at best and fix on absolutely wrong layer. The library
>> should provide the support for the edge case. But we disagree on that.
>
> I strongly disagree.
>
> Firstly on the layer to fix, let me be clear, if something writes like
> RAID1, reads like RAID1, repairs like RAID1, then it's RAID1.
*nod*
> With that mindset, changing chunk type at read/add time is exactly the
> correct layer to fix.
It also is amazingly simple!
^ permalink raw reply
* Re: [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Christoph Hellwig @ 2026-05-26 6:45 UTC (permalink / raw)
To: David Sterba; +Cc: Christoph Hellwig, Qu Wenruo, linux-btrfs, linux-raid
In-Reply-To: <20260525101707.GV12792@suse.cz>
On Mon, May 25, 2026 at 12:17:07PM +0200, David Sterba wrote:
> > It absolutely is. Adding fast path workarounds for this is completely
> > stupid when it can be trivially handled on the mount side.
>
> It's a lazy hack at best and fix on absolutely wrong layer. The library
> should provide the support for the edge case. But we disagree on that.
No, the library should not provide hacks for something it never intended
to support, and which is a bad idea. It's also something the
implementation that the library was factored out from for btrfs use
never supported.
So adding the workaround to btrfs, which caused the problem is a
requirement. Doing it to only create mount time overhead instead of
burdening the I/O path is a very smart way to solve the problem,
while adding hacks to the I/O fastpath is everything but smart.
But in the end this is for the btrfs maintainers to decide.
^ permalink raw reply
* [PATCH 30/34] md-bitmap: Convert read_file_page and write_file_page to bh_submit()
From: Matthew Wilcox (Oracle) @ 2026-05-25 17:19 UTC (permalink / raw)
To: Jan Kara
Cc: Matthew Wilcox (Oracle), Christian Brauner, Christoph Hellwig,
linux-fsdevel, linux-raid
In-Reply-To: <20260525171931.4144395-1-willy@infradead.org>
Avoid an extra indirect function call by using bh_submit() instead of
submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: linux-raid@vger.kernel.org
---
drivers/md/md-bitmap.c | 27 ++++++++++++++-------------
1 file changed, 14 insertions(+), 13 deletions(-)
diff --git a/drivers/md/md-bitmap.c b/drivers/md/md-bitmap.c
index 028b9ca8ce52..c0a4559186e2 100644
--- a/drivers/md/md-bitmap.c
+++ b/drivers/md/md-bitmap.c
@@ -502,6 +502,18 @@ static void write_sb_page(struct bitmap *bitmap, unsigned long pg_index,
static void md_bitmap_file_kick(struct bitmap *bitmap);
#ifdef CONFIG_MD_BITMAP_FILE
+static void end_bitmap_write(struct bio *bio)
+{
+ bool uptodate = bio->bi_status == BLK_STS_OK;
+ struct buffer_head *bh = bio_endio_bh(bio);
+ struct bitmap *bitmap = bh->b_private;
+
+ if (!uptodate)
+ set_bit(BITMAP_WRITE_ERROR, &bitmap->flags);
+ if (atomic_dec_and_test(&bitmap->pending_writes))
+ wake_up(&bitmap->write_wait);
+}
+
static void write_file_page(struct bitmap *bitmap, struct page *page, int wait)
{
struct buffer_head *bh = page_buffers(page);
@@ -510,7 +522,7 @@ static void write_file_page(struct bitmap *bitmap, struct page *page, int wait)
atomic_inc(&bitmap->pending_writes);
set_buffer_locked(bh);
set_buffer_mapped(bh);
- submit_bh(REQ_OP_WRITE | REQ_SYNC, bh);
+ bh_submit(bh, REQ_OP_WRITE | REQ_SYNC, end_bitmap_write);
bh = bh->b_this_page;
}
@@ -519,16 +531,6 @@ static void write_file_page(struct bitmap *bitmap, struct page *page, int wait)
atomic_read(&bitmap->pending_writes) == 0);
}
-static void end_bitmap_write(struct buffer_head *bh, int uptodate)
-{
- struct bitmap *bitmap = bh->b_private;
-
- if (!uptodate)
- set_bit(BITMAP_WRITE_ERROR, &bitmap->flags);
- if (atomic_dec_and_test(&bitmap->pending_writes))
- wake_up(&bitmap->write_wait);
-}
-
static void free_buffers(struct page *page)
{
struct buffer_head *bh;
@@ -592,12 +594,11 @@ static int read_file_page(struct file *file, unsigned long index,
else
count -= blocksize;
- bh->b_end_io = end_bitmap_write;
bh->b_private = bitmap;
atomic_inc(&bitmap->pending_writes);
set_buffer_locked(bh);
set_buffer_mapped(bh);
- submit_bh(REQ_OP_READ, bh);
+ bh_submit(bh, REQ_OP_READ, end_bitmap_write);
}
blk_cur++;
bh = bh->b_this_page;
--
2.47.3
^ permalink raw reply related
* Re: --bitmap=lockless across server reboot
From: Anton Gavriliuk @ 2026-05-25 17:11 UTC (permalink / raw)
To: yukuai; +Cc: linux-raid
In-Reply-To: <CAAiJnjoh9rTHh4gRu2=7A4C7EFP4yfBED94qmeS_ATDyB0yevQ@mail.gmail.com>
I remembered how to reproduce.
The raid is gone after I performed precondition for performance tests
and reboot.
After reboot there are no issues in dmesg and messages.
[root@qdevice ~]# fio --name=raid5_final --rw=write --bs=2048k
--filename=/dev/md0 --direct=1 --numjobs=1 --iodepth=64
--group_reporting --ioengine=io_uring --time_based --runtime=1800
raid5_final: (g=0): rw=write, bs=(R) 2048KiB-2048KiB, (W)
2048KiB-2048KiB, (T) 2048KiB-2048KiB, ioengine=io_uring, iodepth=64
fio-3.42-29-gd569
Starting 1 process
Jobs: 1 (f=1): [W(1)][100.0%][w=8316MiB/s][w=4158 IOPS][eta 00m:00s]
raid5_final: (groupid=0, jobs=1): err= 0: pid=5070: Mon May 25 19:56:09 2026
write: IOPS=4088, BW=8176MiB/s (8574MB/s)(14.0TiB/1800013msec)
slat (usec): min=15, max=1287, avg=36.23, stdev=11.79
clat (usec): min=2072, max=43284, avg=15618.22, stdev=3312.23
lat (usec): min=2094, max=43313, avg=15654.45, stdev=3312.52
clat percentiles (usec):
| 1.00th=[ 8356], 5.00th=[10421], 10.00th=[11469], 20.00th=[12911],
| 30.00th=[13829], 40.00th=[14615], 50.00th=[15533], 60.00th=[16319],
| 70.00th=[17171], 80.00th=[18220], 90.00th=[19792], 95.00th=[21103],
| 99.00th=[23987], 99.50th=[25297], 99.90th=[28181], 99.95th=[29492],
| 99.99th=[32375]
bw ( MiB/s): min= 7736, max= 9132, per=100.00%, avg=8178.90,
stdev=166.52, samples=3599
iops : min= 3868, max= 4566, avg=4089.42, stdev=83.27, samples=3599
lat (msec) : 4=0.01%, 10=3.68%, 20=87.01%, 50=9.32%
cpu : usr=10.21%, sys=5.19%, ctx=5637748, majf=0, minf=754452
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=0,7358832,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0.00ns, window=0.00ns, percentile=100.00%, depth=64
Run status group 0 (all jobs):
WRITE: bw=8176MiB/s (8574MB/s), 8176MiB/s-8176MiB/s
(8574MB/s-8574MB/s), io=14.0TiB (15.4TB), run=1800013-1800013msec
Disk stats (read/write):
md0: ios=20/14716476, sectors=2560/30139342848, merge=0/0,
ticks=0/125528858, in_queue=125528858, util=100.00%,
aggrios=68/7358896, aggsectors=3046/7535444480, aggrmerge=0/0,
aggrticks=6/413853, aggrin_queue=413859, aggrutil=21.24%
nvme0n1: ios=48/7358896, sectors=1680/7535444480, merge=0/0,
ticks=6/408945, in_queue=408951, util=20.67%
nvme3n1: ios=67/7358896, sectors=3992/7535444480, merge=0/0,
ticks=4/433875, in_queue=433879, util=21.24%
nvme2n1: ios=118/7358896, sectors=2960/7535444480, merge=0/0,
ticks=7/405682, in_queue=405689, util=20.45%
nvme5n1: ios=75/7358896, sectors=5136/7535444480, merge=0/0,
ticks=9/407693, in_queue=407702, util=20.42%
nvme1n1: ios=51/7358896, sectors=2064/7535444480, merge=0/0,
ticks=7/409760, in_queue=409767, util=20.59%
nvme4n1: ios=54/7358896, sectors=2448/7535444480, merge=0/0,
ticks=5/417163, in_queue=417168, util=20.58%
[root@qdevice ~]#
[root@qdevice ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
nvme6n1 259:1 0 447.1G 0 disk
├─nvme6n1p1 259:2 0 1G 0 part /boot/efi
├─nvme6n1p2 259:3 0 1G 0 part /boot
├─nvme6n1p3 259:4 0 220G 0 part /
├─nvme6n1p4 259:5 0 1G 0 part
└─nvme6n1p5 259:6 0 220G 0 part
nvme0n1 259:9 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme1n1 259:10 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme4n1 259:15 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme2n1 259:16 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme5n1 259:17 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme3n1 259:18 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
[root@qdevice ~]#
[root@qdevice ~]# reboot
login as: root
root@10.72.14.186's password:
Web console: https://qdevice:9090/ or https://10.72.14.186:9090/
Last login: Mon May 25 19:25:38 2026 from 10.72.12.178
[root@qdevice ~]# uptime
20:01:59 up 0 min, 1 user, load average: 0.87, 0.26, 0.09
[root@qdevice ~]#
[root@qdevice ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
nvme5n1 259:1 0 447.1G 0 disk
├─nvme5n1p1 259:2 0 1G 0 part /boot/efi
├─nvme5n1p2 259:3 0 1G 0 part /boot
├─nvme5n1p3 259:4 0 220G 0 part /
├─nvme5n1p4 259:5 0 1G 0 part
└─nvme5n1p5 259:6 0 220G 0 part
nvme1n1 259:8 0 2.9T 0 disk
nvme3n1 259:10 0 2.9T 0 disk
nvme4n1 259:13 0 2.9T 0 disk
nvme2n1 259:14 0 2.9T 0 disk
nvme0n1 259:17 0 2.9T 0 disk
nvme6n1 259:18 0 2.9T 0 disk
[root@qdevice ~]#
[root@qdevice ~]# uptime
20:02:40 up 1 min, 1 user, load average: 0.46, 0.24, 0.09
[root@qdevice ~]#
Anton
пн, 25 мая 2026 г. в 18:08, Anton Gavriliuk <antosha20xx@gmail.com>:
>
> Hhmm.... I just re-checked it with latest kernel (7.1-rc5) on Fedora
> Server 44, and now it works, log below. I don't remember on which
> kernel I saw that issue (7.1-rc3 or 7.1-rc4)
>
> BTW, earlier you mentioned that you are working on raid5 performance
> improvements,
>
> "“BTW we're working on performance
> improvement for large raid5 arrays for at most 64 disks, results looks great.
> However, it might take sometime before we push our work to upstream.”
>
> Are there any progress ?, I'm very interested to test on PCIe 5.0 NVMe drives.
>
> This is Fedora Server 44 (up to date) with compiled kernel 7.1-rc5
>
> [root@qdevice ~]# cat /etc/*release
> Fedora release 44 (Forty Four)
> NAME="Fedora Linux"
> VERSION="44 (Server Edition)"
> RELEASE_TYPE=stable
> ID=fedora
> VERSION_ID=44
> VERSION_CODENAME=""
> PRETTY_NAME="Fedora Linux 44 (Server Edition)"
> ANSI_COLOR="0;38;2;60;110;180"
> LOGO=fedora-logo-icon
> CPE_NAME="cpe:/o:fedoraproject:fedora:44"
> HOME_URL="https://fedoraproject.org/"
> DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f44/"
> SUPPORT_URL="https://ask.fedoraproject.org/"
> BUG_REPORT_URL="https://bugzilla.redhat.com/"
> REDHAT_BUGZILLA_PRODUCT="Fedora"
> REDHAT_BUGZILLA_PRODUCT_VERSION=44
> REDHAT_SUPPORT_PRODUCT="Fedora"
> REDHAT_SUPPORT_PRODUCT_VERSION=44
> SUPPORT_END=2027-05-19
> VARIANT="Server Edition"
> VARIANT_ID=server
> Fedora release 44 (Forty Four)
> Fedora release 44 (Forty Four)
> [root@qdevice ~]#
> [root@qdevice ~]# uname -a
> Linux qdevice 7.1.0-rc5 #1 SMP PREEMPT_DYNAMIC Mon May 25 15:02:20
> EEST 2026 x86_64 GNU/Linux
> [root@qdevice ~]#
> [root@qdevice mdadm]# lsblk
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
> nvme6n1 259:1 0 447.1G 0 disk
> ├─nvme6n1p1 259:2 0 1G 0 part /boot/efi
> ├─nvme6n1p2 259:3 0 1G 0 part /boot
> ├─nvme6n1p3 259:4 0 220G 0 part /
> ├─nvme6n1p4 259:5 0 1G 0 part
> └─nvme6n1p5 259:6 0 220G 0 part
> nvme3n1 259:10 0 2.9T 0 disk
> nvme1n1 259:11 0 2.9T 0 disk
> nvme0n1 259:12 0 2.9T 0 disk
> nvme4n1 259:16 0 2.9T 0 disk
> nvme2n1 259:17 0 2.9T 0 disk
> nvme5n1 259:18 0 2.9T 0 disk
> [root@qdevice mdadm]#
> [root@qdevice mdadm]# mdadm --create --verbose /dev/md0 --level=6
> --bitmap=lockless --raid-devices=6 /dev/nvme0n1 /dev/nvme1n1
> /dev/nvme2n1 /dev/nvme3n1 /dev/nvme4n1 /dev/nvme5n1
> mdadm: Experimental lockless bitmap, use at your own risk!
> mdadm: layout defaults to left-symmetric
> mdadm: layout defaults to left-symmetric
> mdadm: chunk size defaults to 512K
> mdadm: /dev/nvme0n1 appears to be part of a raid array:
> level=raid6 devices=5 ctime=Wed May 20 12:45:26 2026
> mdadm: /dev/nvme1n1 appears to be part of a raid array:
> level=raid6 devices=5 ctime=Wed May 20 12:45:26 2026
> mdadm: /dev/nvme2n1 appears to be part of a raid array:
> level=raid6 devices=5 ctime=Wed May 20 12:45:26 2026
> mdadm: /dev/nvme3n1 appears to be part of a raid array:
> level=raid6 devices=5 ctime=Wed May 20 12:45:26 2026
> mdadm: /dev/nvme4n1 appears to be part of a raid array:
> level=raid6 devices=5 ctime=Wed May 20 12:45:26 2026
> mdadm: /dev/nvme5n1 appears to be part of a raid array:
> level=raid6 devices=6 ctime=Wed May 20 11:45:38 2026
> mdadm: size set to 3125484544K
> Continue creating array [y/N]? y
> mdadm: Defaulting to version 1.2 metadata
> mdadm: array /dev/md0 started.
> [root@qdevice mdadm]#
> [root@qdevice mdadm]# lsblk
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
> nvme6n1 259:1 0 447.1G 0 disk
> ├─nvme6n1p1 259:2 0 1G 0 part /boot/efi
> ├─nvme6n1p2 259:3 0 1G 0 part /boot
> ├─nvme6n1p3 259:4 0 220G 0 part /
> ├─nvme6n1p4 259:5 0 1G 0 part
> └─nvme6n1p5 259:6 0 220G 0 part
> nvme3n1 259:10 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> nvme1n1 259:11 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> nvme0n1 259:12 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> nvme4n1 259:16 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> nvme2n1 259:17 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> nvme5n1 259:18 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> [root@qdevice mdadm]#
> [root@qdevice mdadm]# reboot
> [root@qdevice mdadm]#
>
> After reboot,
>
> [root@qdevice ~]# lsblk
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
> nvme6n1 259:1 0 447.1G 0 disk
> ├─nvme6n1p1 259:2 0 1G 0 part /boot/efi
> ├─nvme6n1p2 259:3 0 1G 0 part /boot
> ├─nvme6n1p3 259:4 0 220G 0 part /
> ├─nvme6n1p4 259:5 0 1G 0 part
> └─nvme6n1p5 259:6 0 220G 0 part
> nvme2n1 259:9 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> nvme1n1 259:10 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> nvme4n1 259:15 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> nvme0n1 259:16 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> nvme3n1 259:17 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> nvme5n1 259:18 0 2.9T 0 disk
> └─md0 9:0 0 11.6T 0 raid6
> [root@qdevice ~]#
>
>
> Anton
>
> пн, 25 мая 2026 г. в 08:32, Yu Kuai <yukuai@fnnas.com>:
> >
> > Hi,
> >
> > 在 2026/5/20 1:28, Anton Gavriliuk 写道:
> > > Hi
> > >
> > > Does md-raid lockless bitmap support server's reboot ?
> >
> > Yes, of course it should support reboot.
> >
> > >
> > > I don't see md-raid6 created with --bitmap=lockless after reboot.
> >
> > Do you check kernel log if there is any failure log? And do you try
> > mdadm -a manually?
> >
> > BTW, pPlease report with your environment and test script in details,
> > especially kernel version and mdadm version.
> >
> > >
> > > Anton
> > >
> > --
> > Thansk,
> > Kuai
^ permalink raw reply
* Re: --bitmap=lockless across server reboot
From: Anton Gavriliuk @ 2026-05-25 15:08 UTC (permalink / raw)
To: yukuai; +Cc: linux-raid
In-Reply-To: <5d4b72d7-7e03-463c-82a1-63adcae20344@fygo.io>
Hhmm.... I just re-checked it with latest kernel (7.1-rc5) on Fedora
Server 44, and now it works, log below. I don't remember on which
kernel I saw that issue (7.1-rc3 or 7.1-rc4)
BTW, earlier you mentioned that you are working on raid5 performance
improvements,
"“BTW we're working on performance
improvement for large raid5 arrays for at most 64 disks, results looks great.
However, it might take sometime before we push our work to upstream.”
Are there any progress ?, I'm very interested to test on PCIe 5.0 NVMe drives.
This is Fedora Server 44 (up to date) with compiled kernel 7.1-rc5
[root@qdevice ~]# cat /etc/*release
Fedora release 44 (Forty Four)
NAME="Fedora Linux"
VERSION="44 (Server Edition)"
RELEASE_TYPE=stable
ID=fedora
VERSION_ID=44
VERSION_CODENAME=""
PRETTY_NAME="Fedora Linux 44 (Server Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:44"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f44/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=44
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=44
SUPPORT_END=2027-05-19
VARIANT="Server Edition"
VARIANT_ID=server
Fedora release 44 (Forty Four)
Fedora release 44 (Forty Four)
[root@qdevice ~]#
[root@qdevice ~]# uname -a
Linux qdevice 7.1.0-rc5 #1 SMP PREEMPT_DYNAMIC Mon May 25 15:02:20
EEST 2026 x86_64 GNU/Linux
[root@qdevice ~]#
[root@qdevice mdadm]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
nvme6n1 259:1 0 447.1G 0 disk
├─nvme6n1p1 259:2 0 1G 0 part /boot/efi
├─nvme6n1p2 259:3 0 1G 0 part /boot
├─nvme6n1p3 259:4 0 220G 0 part /
├─nvme6n1p4 259:5 0 1G 0 part
└─nvme6n1p5 259:6 0 220G 0 part
nvme3n1 259:10 0 2.9T 0 disk
nvme1n1 259:11 0 2.9T 0 disk
nvme0n1 259:12 0 2.9T 0 disk
nvme4n1 259:16 0 2.9T 0 disk
nvme2n1 259:17 0 2.9T 0 disk
nvme5n1 259:18 0 2.9T 0 disk
[root@qdevice mdadm]#
[root@qdevice mdadm]# mdadm --create --verbose /dev/md0 --level=6
--bitmap=lockless --raid-devices=6 /dev/nvme0n1 /dev/nvme1n1
/dev/nvme2n1 /dev/nvme3n1 /dev/nvme4n1 /dev/nvme5n1
mdadm: Experimental lockless bitmap, use at your own risk!
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 512K
mdadm: /dev/nvme0n1 appears to be part of a raid array:
level=raid6 devices=5 ctime=Wed May 20 12:45:26 2026
mdadm: /dev/nvme1n1 appears to be part of a raid array:
level=raid6 devices=5 ctime=Wed May 20 12:45:26 2026
mdadm: /dev/nvme2n1 appears to be part of a raid array:
level=raid6 devices=5 ctime=Wed May 20 12:45:26 2026
mdadm: /dev/nvme3n1 appears to be part of a raid array:
level=raid6 devices=5 ctime=Wed May 20 12:45:26 2026
mdadm: /dev/nvme4n1 appears to be part of a raid array:
level=raid6 devices=5 ctime=Wed May 20 12:45:26 2026
mdadm: /dev/nvme5n1 appears to be part of a raid array:
level=raid6 devices=6 ctime=Wed May 20 11:45:38 2026
mdadm: size set to 3125484544K
Continue creating array [y/N]? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
[root@qdevice mdadm]#
[root@qdevice mdadm]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
nvme6n1 259:1 0 447.1G 0 disk
├─nvme6n1p1 259:2 0 1G 0 part /boot/efi
├─nvme6n1p2 259:3 0 1G 0 part /boot
├─nvme6n1p3 259:4 0 220G 0 part /
├─nvme6n1p4 259:5 0 1G 0 part
└─nvme6n1p5 259:6 0 220G 0 part
nvme3n1 259:10 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme1n1 259:11 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme0n1 259:12 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme4n1 259:16 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme2n1 259:17 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme5n1 259:18 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
[root@qdevice mdadm]#
[root@qdevice mdadm]# reboot
[root@qdevice mdadm]#
After reboot,
[root@qdevice ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
nvme6n1 259:1 0 447.1G 0 disk
├─nvme6n1p1 259:2 0 1G 0 part /boot/efi
├─nvme6n1p2 259:3 0 1G 0 part /boot
├─nvme6n1p3 259:4 0 220G 0 part /
├─nvme6n1p4 259:5 0 1G 0 part
└─nvme6n1p5 259:6 0 220G 0 part
nvme2n1 259:9 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme1n1 259:10 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme4n1 259:15 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme0n1 259:16 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme3n1 259:17 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
nvme5n1 259:18 0 2.9T 0 disk
└─md0 9:0 0 11.6T 0 raid6
[root@qdevice ~]#
Anton
пн, 25 мая 2026 г. в 08:32, Yu Kuai <yukuai@fnnas.com>:
>
> Hi,
>
> 在 2026/5/20 1:28, Anton Gavriliuk 写道:
> > Hi
> >
> > Does md-raid lockless bitmap support server's reboot ?
>
> Yes, of course it should support reboot.
>
> >
> > I don't see md-raid6 created with --bitmap=lockless after reboot.
>
> Do you check kernel log if there is any failure log? And do you try
> mdadm -a manually?
>
> BTW, pPlease report with your environment and test script in details,
> especially kernel version and mdadm version.
>
> >
> > Anton
> >
> --
> Thansk,
> Kuai
^ permalink raw reply
* Re: [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Qu Wenruo @ 2026-05-25 10:39 UTC (permalink / raw)
To: dsterba, Christoph Hellwig; +Cc: linux-btrfs, linux-raid
In-Reply-To: <20260525101707.GV12792@suse.cz>
在 2026/5/25 19:47, David Sterba 写道:
> On Mon, May 25, 2026 at 08:00:30AM +0200, Christoph Hellwig wrote:
>> On Sat, May 23, 2026 at 04:23:44PM +0200, David Sterba wrote:
>>> On Fri, May 22, 2026 at 06:53:53PM +0930, Qu Wenruo wrote:
>>>> Recently kernel RAID56 lib is trying to remove the unexpected
>>>> single-data-RAID56 (2 disks RAID5 or 3 disk RAID5) support, meanwhile
>>>> btrfs still supports such setup, which means in the long run btrfs has
>>>> to handle such corner case by ourselves.
>>>>
>>>> Thankfully single-data-RAID56 is really RAID1/RAID1C3, since data and
>>>> P/Q stripes all match each other, rotation also makes no difference.
>>>>
>>>> This patch will disguise those single-data-RAID56 chunks as
>>>> RAID1/RAID1C3 chunks.
>>>
>>> I don't think this is the right way to fix it. Calculations are an
>>> implementation detail and should be done at the time of the xor_gen or
>>> raid6_call, not touching the upper level structures related to format.
>>
>> It absolutely is. Adding fast path workarounds for this is completely
>> stupid when it can be trivially handled on the mount side.
>
> It's a lazy hack at best and fix on absolutely wrong layer. The library
> should provide the support for the edge case. But we disagree on that.
I strongly disagree.
Firstly on the layer to fix, let me be clear, if something writes like
RAID1, reads like RAID1, repairs like RAID1, then it's RAID1.
With that mindset, changing chunk type at read/add time is exactly the
correct layer to fix.
Or let me change the title of the patch, ENHANCE single-data-RAID56
handling to get rid all the corner cases.
Do you still think that's the wrong correct layer to fix?
So the wrong layer argument doesn't stand. If you're so focused on
fixing the problem in the library layer, then all you can see is to put
some adhoc fixes to emulate the library fix.
Furthermore, we are already doing all the warning about 2-disk raid5,
now I'm providing a solution that is as mature as RAID1, now you're
complaining about the simplicity but wants all the complex striping and
reduced read performance?
I don't see any point in this.
^ permalink raw reply
* Re: [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Qu Wenruo @ 2026-05-25 10:25 UTC (permalink / raw)
To: dsterba; +Cc: linux-btrfs, hch, linux-raid
In-Reply-To: <20260525101236.GU12792@suse.cz>
在 2026/5/25 19:42, David Sterba 写道:
> On Sun, May 24, 2026 at 01:43:54PM +0930, Qu Wenruo wrote:
>> I tried the idea, and it doesn't look elegant at all, we need a lot of
>> if () checks around every xor_gen()/raid6_call.
>>
>> Sure the end result is not that huge (and I have only compile tested
>> it), but if we had some parameters passed wrongly, or some corner case
>> missed, it can be very hard to notice.
>>
>> Just paste the adhoc fix for reference:
>
> I ended up with something similar, but replaced the open coded ifs with
> a helper so there's not that much churn at the call sites.
>
>> diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c
>> index 08ee8f316d96..50fefe33bbe3 100644
>> --- a/fs/btrfs/raid56.c
>> +++ b/fs/btrfs/raid56.c
>> @@ -1410,11 +1410,17 @@ static void generate_pq_vertical_step(struct
>> btrfs_raid_bio *rbio, unsigned int
>> rbio_qstripe_paddr(rbio, sector_nr, step_nr));
>>
>> assert_rbio(rbio);
>> - raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
>> + if (rbio->nr_data > 1) {
>> + raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
>> + } else {
>> + memcpy(pointers[1], pointers[0], step);
>> + memcpy(pointers[2], pointers[0], step);
>> + }
>> } else {
>> /* raid5 */
>> memcpy(pointers[rbio->nr_data], pointers[0], step);
>> - xor_gen(pointers[rbio->nr_data], pointers + 1, rbio->nr_data - 1,
>> + if (rbio->nr_data > 1)
>> + xor_gen(pointers[rbio->nr_data], pointers + 1, rbio->nr_data - 1,
>> step);
>> }
>> for (stripe = stripe - 1; stripe >= 0; stripe--)
>> @@ -1987,11 +1993,17 @@ static void recover_vertical_step(struct
>> btrfs_raid_bio *rbio,
>> }
>>
>> if (failb == rbio->real_stripes - 2) {
>> - raid6_datap_recov(rbio->real_stripes, step,
>> - faila, pointers);
>> + if (unlikely(rbio->nr_data == 1))
>> + memcpy(pointers[0], pointers[rbio->real_stripes - 1], step);
>> + else
>> + raid6_datap_recov(rbio->real_stripes, step,
>> + faila, pointers);
>> } else {
>> - raid6_2data_recov(rbio->real_stripes, step,
>> - faila, failb, pointers);
>> + if (unlikely(rbio->nr_data == 1))
>> + memcpy(pointers[0], pointers[rbio->real_stripes - 2], step);
>
> The 2data recovery needs to do 2 memcpy
Nope. The recovery is only to repair the data stripe.
We do not need to bother the failed P or Q stripe here.
And I think this explains exactly why doing adhoc fix is a stupid idea
in the first place.
We're wasting time on useless corner cases, and corner cases inside
corner cases.
It makes no sense to waste all the time reviewing those useless details,
when we have a much better solution to avoid all those hassles.
^ permalink raw reply
* Re: [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: David Sterba @ 2026-05-25 10:17 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Qu Wenruo, linux-btrfs, linux-raid
In-Reply-To: <20260525060030.GB3370@lst.de>
On Mon, May 25, 2026 at 08:00:30AM +0200, Christoph Hellwig wrote:
> On Sat, May 23, 2026 at 04:23:44PM +0200, David Sterba wrote:
> > On Fri, May 22, 2026 at 06:53:53PM +0930, Qu Wenruo wrote:
> > > Recently kernel RAID56 lib is trying to remove the unexpected
> > > single-data-RAID56 (2 disks RAID5 or 3 disk RAID5) support, meanwhile
> > > btrfs still supports such setup, which means in the long run btrfs has
> > > to handle such corner case by ourselves.
> > >
> > > Thankfully single-data-RAID56 is really RAID1/RAID1C3, since data and
> > > P/Q stripes all match each other, rotation also makes no difference.
> > >
> > > This patch will disguise those single-data-RAID56 chunks as
> > > RAID1/RAID1C3 chunks.
> >
> > I don't think this is the right way to fix it. Calculations are an
> > implementation detail and should be done at the time of the xor_gen or
> > raid6_call, not touching the upper level structures related to format.
>
> It absolutely is. Adding fast path workarounds for this is completely
> stupid when it can be trivially handled on the mount side.
It's a lazy hack at best and fix on absolutely wrong layer. The library
should provide the support for the edge case. But we disagree on that.
^ permalink raw reply
* Re: [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: David Sterba @ 2026-05-25 10:12 UTC (permalink / raw)
To: Qu Wenruo; +Cc: linux-btrfs, hch, linux-raid
In-Reply-To: <1b5c39fe-df28-4a21-bace-fc742498945e@suse.com>
On Sun, May 24, 2026 at 01:43:54PM +0930, Qu Wenruo wrote:
> I tried the idea, and it doesn't look elegant at all, we need a lot of
> if () checks around every xor_gen()/raid6_call.
>
> Sure the end result is not that huge (and I have only compile tested
> it), but if we had some parameters passed wrongly, or some corner case
> missed, it can be very hard to notice.
>
> Just paste the adhoc fix for reference:
I ended up with something similar, but replaced the open coded ifs with
a helper so there's not that much churn at the call sites.
> diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c
> index 08ee8f316d96..50fefe33bbe3 100644
> --- a/fs/btrfs/raid56.c
> +++ b/fs/btrfs/raid56.c
> @@ -1410,11 +1410,17 @@ static void generate_pq_vertical_step(struct
> btrfs_raid_bio *rbio, unsigned int
> rbio_qstripe_paddr(rbio, sector_nr, step_nr));
>
> assert_rbio(rbio);
> - raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
> + if (rbio->nr_data > 1) {
> + raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
> + } else {
> + memcpy(pointers[1], pointers[0], step);
> + memcpy(pointers[2], pointers[0], step);
> + }
> } else {
> /* raid5 */
> memcpy(pointers[rbio->nr_data], pointers[0], step);
> - xor_gen(pointers[rbio->nr_data], pointers + 1, rbio->nr_data - 1,
> + if (rbio->nr_data > 1)
> + xor_gen(pointers[rbio->nr_data], pointers + 1, rbio->nr_data - 1,
> step);
> }
> for (stripe = stripe - 1; stripe >= 0; stripe--)
> @@ -1987,11 +1993,17 @@ static void recover_vertical_step(struct
> btrfs_raid_bio *rbio,
> }
>
> if (failb == rbio->real_stripes - 2) {
> - raid6_datap_recov(rbio->real_stripes, step,
> - faila, pointers);
> + if (unlikely(rbio->nr_data == 1))
> + memcpy(pointers[0], pointers[rbio->real_stripes - 1], step);
> + else
> + raid6_datap_recov(rbio->real_stripes, step,
> + faila, pointers);
> } else {
> - raid6_2data_recov(rbio->real_stripes, step,
> - faila, failb, pointers);
> + if (unlikely(rbio->nr_data == 1))
> + memcpy(pointers[0], pointers[rbio->real_stripes - 2], step);
The 2data recovery needs to do 2 memcpy
> + else
> + raid6_2data_recov(rbio->real_stripes, step,
> + faila, failb, pointers);
> }
> } else {
> void *p;
> @@ -2002,6 +2014,9 @@ static void recover_vertical_step(struct
> btrfs_raid_bio *rbio,
> /* Copy parity block into failed block to start with */
> memcpy(pointers[faila], pointers[rbio->nr_data], step);
>
> + if (rbio->nr_data == 1)
> + goto cleanup;
> +
> /* Rearrange the pointer array */
> p = pointers[faila];
> for (stripe_nr = faila; stripe_nr < rbio->nr_data - 1;
> @@ -2644,11 +2659,17 @@ static bool verify_one_parity_step(struct
> btrfs_raid_bio *rbio,
> if (has_qstripe) {
> assert_rbio(rbio);
> /* RAID6, call the library function to fill in our P/Q. */
> - raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
> + if (rbio->nr_data > 1) {
> + raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
> + } else {
> + memcpy(pointers[1], pointers[0], step);
> + memcpy(pointers[2], pointers[0], step);
> + }
> } else {
> /* RAID5. */
> memcpy(pointers[nr_data], pointers[0], step);
> - xor_gen(pointers[nr_data], pointers + 1, nr_data - 1, step);
> + if (rbio->nr_data > 1)
> + xor_gen(pointers[nr_data], pointers + 1, nr_data - 1, step);
> }
>
> /* Check scrubbing parity and repair it. */
> --
> 2.54.0
^ permalink raw reply
* Re: [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Christoph Hellwig @ 2026-05-25 6:00 UTC (permalink / raw)
To: David Sterba; +Cc: Qu Wenruo, linux-btrfs, hch, linux-raid
In-Reply-To: <20260523142344.GC183694@twin.jikos.cz>
On Sat, May 23, 2026 at 04:23:44PM +0200, David Sterba wrote:
> On Fri, May 22, 2026 at 06:53:53PM +0930, Qu Wenruo wrote:
> > Recently kernel RAID56 lib is trying to remove the unexpected
> > single-data-RAID56 (2 disks RAID5 or 3 disk RAID5) support, meanwhile
> > btrfs still supports such setup, which means in the long run btrfs has
> > to handle such corner case by ourselves.
> >
> > Thankfully single-data-RAID56 is really RAID1/RAID1C3, since data and
> > P/Q stripes all match each other, rotation also makes no difference.
> >
> > This patch will disguise those single-data-RAID56 chunks as
> > RAID1/RAID1C3 chunks.
>
> I don't think this is the right way to fix it. Calculations are an
> implementation detail and should be done at the time of the xor_gen or
> raid6_call, not touching the upper level structures related to format.
It absolutely is. Adding fast path workarounds for this is completely
stupid when it can be trivially handled on the mount side.
^ permalink raw reply
* Re: --bitmap=lockless across server reboot
From: Yu Kuai @ 2026-05-25 5:32 UTC (permalink / raw)
To: Anton Gavriliuk, linux-raid, yukuai
In-Reply-To: <CAAiJnjpPAur30W4FO2sTZcYWc24yFafKWGR=BZWZsh2JnbrRpQ@mail.gmail.com>
Hi,
在 2026/5/20 1:28, Anton Gavriliuk 写道:
> Hi
>
> Does md-raid lockless bitmap support server's reboot ?
Yes, of course it should support reboot.
>
> I don't see md-raid6 created with --bitmap=lockless after reboot.
Do you check kernel log if there is any failure log? And do you try
mdadm -a manually?
BTW, pPlease report with your environment and test script in details,
especially kernel version and mdadm version.
>
> Anton
>
--
Thansk,
Kuai
^ permalink raw reply
* [PATCH v3 2/2] md/raid10: bound reused r10bio devs[] walks by used_nr_devs
From: Chen Cheng @ 2026-05-25 1:55 UTC (permalink / raw)
To: Yu Kuai, linux-raid; +Cc: Chen Cheng, linux-kernel
In-Reply-To: <20260525015520.2565423-1-chencheng@fnnas.com>
From: Chen Cheng <chencheng@fnnas.com>
After reshape changes raid_disks, an in-flight r10bio from the old geometry
can still be completed or freed later. In that case, using the current
geometry to walk r10_bio->devs[] is unsafe. A failure was reproduced with a
simple write workload while reshaping a raid10 array from 4 disks to 5 disks.
e.g.:
mdadm -C /dev/md777 -l10 -n4 /dev/sda /dev/sdb /dev/sdc /dev/sdd
mkfs.ext4 /dev/md777
mount /dev/md777 /mnt/test
fsstress -d /mnt/test -n 24000 -p 8 -l 24 &
mdadm /dev/md777 --add /dev/sde
mdadm --grow /dev/md777 --raid-devices=5 \
--backup-file=/tmp/md-reshape-backup
the sequence above can trigger:
BUG: KASAN: slab-out-of-bounds in free_r10bio+0x1c4/0x260 [raid10]
Read of size 8 at addr ffff00008c2dfac8 by task ksoftirqd/0/15
free_r10bio
raid_end_bio_io
one_write_done
raid10_end_write_request
The buggy object was 200 bytes long, which matches an r10bio with space for
only four devs[] entries. However, put_all_bios() and find_bio_disk() walk
r10_bio->devs[] using the current conf->geo.raid_disks value. Once reshape
switches conf->geo.raid_disks from 4 to 5, an old 4-slot r10bio can be
completed or freed as if it had 5 slots, and the walk overruns devs[4]. The
same stale-width mismatch can also surface during a 5-disk to 4-disk reshape.
Track the number of valid devs[] entries in each reused r10bio with
used_nr_devs. Initialize it whenever an r10bio is prepared for regular I/O,
discard, or resync/recovery/reshape work, and use it to bound devs[] walks
in put_all_bios() and find_bio_disk().
Signed-off-by: Chen Cheng <chencheng@fnnas.com>
---
drivers/md/raid10.c | 8 ++++++--
drivers/md/raid10.h | 2 ++
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 64677dbe5152..d4695fa9c076 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -273,11 +273,11 @@ static void r10buf_pool_free(void *__r10_bio, void *data)
static void put_all_bios(struct r10conf *conf, struct r10bio *r10_bio)
{
int i;
- for (i = 0; i < conf->geo.raid_disks; i++) {
+ for (i = 0; i < r10_bio->used_nr_devs; i++) {
struct bio **bio = & r10_bio->devs[i].bio;
if (!BIO_SPECIAL(*bio))
bio_put(*bio);
*bio = NULL;
bio = &r10_bio->devs[i].repl_bio;
@@ -370,11 +370,11 @@ static int find_bio_disk(struct r10conf *conf, struct r10bio *r10_bio,
struct bio *bio, int *slotp, int *replp)
{
int slot;
int repl = 0;
- for (slot = 0; slot < conf->geo.raid_disks; slot++) {
+ for (slot = 0; slot < r10_bio->used_nr_devs; slot++) {
if (r10_bio->devs[slot].bio == bio)
break;
if (r10_bio->devs[slot].repl_bio == bio) {
repl = 1;
break;
@@ -1553,10 +1553,11 @@ static void __make_request(struct mddev *mddev, struct bio *bio, int sectors)
r10_bio->mddev = mddev;
r10_bio->sector = bio->bi_iter.bi_sector;
r10_bio->state = 0;
r10_bio->read_slot = -1;
+ r10_bio->used_nr_devs = conf->geo.raid_disks;
memset(r10_bio->devs, 0, sizeof(r10_bio->devs[0]) *
conf->geo.raid_disks);
if (bio_data_dir(bio) == READ)
raid10_read_request(mddev, bio, r10_bio, true);
@@ -1740,10 +1741,11 @@ static int raid10_handle_discard(struct mddev *mddev, struct bio *bio)
retry_discard:
r10_bio = mempool_alloc(conf->r10bio_pool, GFP_NOIO);
r10_bio->mddev = mddev;
r10_bio->state = 0;
r10_bio->sectors = 0;
+ r10_bio->used_nr_devs = geo->raid_disks;
memset(r10_bio->devs, 0, sizeof(r10_bio->devs[0]) * geo->raid_disks);
wait_blocked_dev(mddev, r10_bio);
/*
* For far layout it needs more than one r10bio to cover all regions.
@@ -3074,10 +3076,12 @@ static struct r10bio *raid10_alloc_init_r10buf(struct r10conf *conf)
test_bit(MD_RECOVERY_RESHAPE, &conf->mddev->recovery))
nalloc = conf->copies; /* resync */
else
nalloc = 2; /* recovery */
+ r10bio->used_nr_devs = nalloc;
+
for (i = 0; i < nalloc; i++) {
bio = r10bio->devs[i].bio;
rp = bio->bi_private;
bio_reset(bio, NULL, 0);
bio->bi_private = rp;
diff --git a/drivers/md/raid10.h b/drivers/md/raid10.h
index b711626a5db7..4751119f9770 100644
--- a/drivers/md/raid10.h
+++ b/drivers/md/raid10.h
@@ -125,10 +125,12 @@ struct r10bio {
struct bio *master_bio;
/*
* if the IO is in READ direction, then this is where we read
*/
int read_slot;
+ /* Used to bound devs[] walks when the object is reused. */
+ unsigned int used_nr_devs;
struct list_head retry_list;
/*
* if the IO is in WRITE direction, then multiple bios are used,
* one for each copy.
--
2.54.0
^ permalink raw reply related
* [PATCH v3 1/2] md/raid10: make r10bio_pool use fixed-size objects
From: Chen Cheng @ 2026-05-25 1:55 UTC (permalink / raw)
To: Yu Kuai, linux-raid; +Cc: Chen Cheng, linux-kernel
In-Reply-To: <20260525015520.2565423-1-chencheng@fnnas.com>
From: Chen Cheng <chencheng@fnnas.com>
raid10 currently allocates r10bio_pool objects with conf->geo.raid_disks,
which makes regular r10bio objects geometry-dependent.
That model breaks down across reshape. mempool objects are preallocated and
reused, so a reshape that changes the number of raid disks can leave old
r10bio objects in the regular I/O pool with a devs[] array sized for the
previous geometry. After the geometry switch, those stale objects may be
reused or later freed under the new layout, creating a width mismatch
between the reused r10bio and the current array geometry.
For example, during a 4-disk to 5-disk reshape, an r10bio allocated before
the geometry switch has room for only 4 devs[] entries. After reshape
updates conf->geo.raid_disks to 5, that stale object can be reused under
the new geometry. Code such as __make_request(), put_all_bios(), and
find_bio_disk() may then access devs[] using the new geometry and step
past the end of the old 4-slot object, leading to slab out-of-bounds
accesses.
The root problem is that regular r10bio pool objects are geometry-dependent,
while mempool elements are preallocated and reused across requests.
Switch r10bio_pool to a fixed-size kmalloc mempool so regular I/O objects no
longer carry an allocation width tied to the current geometry. Use the same
fixed-size allocation rule for the standalone r10bio allocated from
r10buf_pool_alloc().
Signed-off-by: Chen Cheng <chencheng@fnnas.com>
---
drivers/md/raid10.c | 57 ++++++++++++++++++++++++++++++++++-----------
drivers/md/raid10.h | 2 +-
2 files changed, 45 insertions(+), 14 deletions(-)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 39085e7dd6d2..64677dbe5152 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -101,17 +101,32 @@ static void end_reshape(struct r10conf *conf);
static inline struct r10bio *get_resync_r10bio(struct bio *bio)
{
return get_resync_pages(bio)->raid_bio;
}
-static void * r10bio_pool_alloc(gfp_t gfp_flags, void *data)
+static inline unsigned int calc_r10bio_pool_disks(struct mddev *mddev)
{
- struct r10conf *conf = data;
- int size = offsetof(struct r10bio, devs[conf->geo.raid_disks]);
+ /* If delta_disks < 0, use bigger r10bio->devs[] is ok. */
+ return mddev->raid_disks + max(0, mddev->delta_disks);
+}
+
+static inline int calc_r10bio_size(struct mddev *mddev)
+{
+ return offsetof(struct r10bio, devs[calc_r10bio_pool_disks(mddev)]);
+}
+
+static mempool_t *create_r10bio_pool(struct mddev *mddev)
+{
+ int size = calc_r10bio_size(mddev);
+
+ return mempool_create_kmalloc_pool(NR_RAID_BIOS, size);
+}
+
+static struct r10bio *alloc_r10bio(struct mddev *mddev, gfp_t gfp_flags)
+{
+ int size = calc_r10bio_size(mddev);
- /* allocate a r10bio with room for raid_disks entries in the
- * bios array */
return kzalloc(size, gfp_flags);
}
#define RESYNC_SECTORS (RESYNC_BLOCK_SIZE >> 9)
/* amount of memory to reserve for resync requests */
@@ -135,11 +150,11 @@ static void * r10buf_pool_alloc(gfp_t gfp_flags, void *data)
struct bio *bio;
int j;
int nalloc, nalloc_rp;
struct resync_pages *rps;
- r10_bio = r10bio_pool_alloc(gfp_flags, conf);
+ r10_bio = alloc_r10bio(conf->mddev, gfp_flags);
if (!r10_bio)
return NULL;
if (test_bit(MD_RECOVERY_SYNC, &conf->mddev->recovery) ||
test_bit(MD_RECOVERY_RESHAPE, &conf->mddev->recovery))
@@ -275,11 +290,11 @@ static void put_all_bios(struct r10conf *conf, struct r10bio *r10_bio)
static void free_r10bio(struct r10bio *r10_bio)
{
struct r10conf *conf = r10_bio->mddev->private;
put_all_bios(conf, r10_bio);
- mempool_free(r10_bio, &conf->r10bio_pool);
+ mempool_free(r10_bio, conf->r10bio_pool);
}
static void put_buf(struct r10bio *r10_bio)
{
struct r10conf *conf = r10_bio->mddev->private;
@@ -1529,11 +1544,11 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio,
static void __make_request(struct mddev *mddev, struct bio *bio, int sectors)
{
struct r10conf *conf = mddev->private;
struct r10bio *r10_bio;
- r10_bio = mempool_alloc(&conf->r10bio_pool, GFP_NOIO);
+ r10_bio = mempool_alloc(conf->r10bio_pool, GFP_NOIO);
r10_bio->master_bio = bio;
r10_bio->sectors = sectors;
r10_bio->mddev = mddev;
@@ -1721,11 +1736,11 @@ static int raid10_handle_discard(struct mddev *mddev, struct bio *bio)
last_stripe_index *= geo->far_copies;
end_disk_offset = (bio_end & geo->chunk_mask) +
(last_stripe_index << geo->chunk_shift);
retry_discard:
- r10_bio = mempool_alloc(&conf->r10bio_pool, GFP_NOIO);
+ r10_bio = mempool_alloc(conf->r10bio_pool, GFP_NOIO);
r10_bio->mddev = mddev;
r10_bio->state = 0;
r10_bio->sectors = 0;
memset(r10_bio->devs, 0, sizeof(r10_bio->devs[0]) * geo->raid_disks);
wait_blocked_dev(mddev, r10_bio);
@@ -3821,11 +3836,11 @@ static int setup_geo(struct geom *geo, struct mddev *mddev, enum geo_type new)
static void raid10_free_conf(struct r10conf *conf)
{
if (!conf)
return;
- mempool_exit(&conf->r10bio_pool);
+ mempool_destroy(conf->r10bio_pool);
kfree(conf->mirrors);
kfree(conf->mirrors_old);
kfree(conf->mirrors_new);
safe_put_page(conf->tmppage);
bioset_exit(&conf->bio_split);
@@ -3868,13 +3883,12 @@ static struct r10conf *setup_conf(struct mddev *mddev)
if (!conf->tmppage)
goto out;
conf->geo = geo;
conf->copies = copies;
- err = mempool_init(&conf->r10bio_pool, NR_RAID_BIOS, r10bio_pool_alloc,
- rbio_pool_free, conf);
- if (err)
+ conf->r10bio_pool = create_r10bio_pool(mddev);
+ if (!conf->r10bio_pool)
goto out;
err = bioset_init(&conf->bio_split, BIO_POOL_SIZE, 0, 0);
if (err)
goto out;
@@ -4363,10 +4377,11 @@ static int raid10_start_reshape(struct mddev *mddev)
struct geom new;
struct r10conf *conf = mddev->private;
struct md_rdev *rdev;
int spares = 0;
int ret;
+ mempool_t *new_pool;
if (test_bit(MD_RECOVERY_RUNNING, &mddev->recovery))
return -EBUSY;
if (setup_geo(&new, mddev, geo_start) != conf->copies)
@@ -4398,11 +4413,23 @@ static int raid10_start_reshape(struct mddev *mddev)
return -EINVAL;
if (spares < mddev->delta_disks)
return -EINVAL;
+ mddev_unlock(mddev);
+ mddev_suspend_and_lock_nointr(mddev);
conf->offset_diff = min_offset_diff;
+ if (mddev->delta_disks > 0) {
+ new_pool = create_r10bio_pool(mddev);
+ if (!new_pool) {
+ mddev_unlock_and_resume(mddev);
+ mddev_lock_nointr(mddev);
+ return -ENOMEM;
+ }
+ mempool_destroy(conf->r10bio_pool);
+ conf->r10bio_pool = new_pool;
+ }
spin_lock_irq(&conf->device_lock);
if (conf->mirrors_new) {
memcpy(conf->mirrors_new, conf->mirrors,
sizeof(struct raid10_info)*conf->prev.raid_disks);
smp_mb();
@@ -4415,20 +4442,24 @@ static int raid10_start_reshape(struct mddev *mddev)
smp_mb();
if (mddev->reshape_backwards) {
sector_t size = raid10_size(mddev, 0, 0);
if (size < mddev->array_sectors) {
spin_unlock_irq(&conf->device_lock);
+ mddev_unlock_and_resume(mddev);
+ mddev_lock_nointr(mddev);
pr_warn("md/raid10:%s: array size must be reduce before number of disks\n",
mdname(mddev));
return -EINVAL;
}
mddev->resync_max_sectors = size;
conf->reshape_progress = size;
} else
conf->reshape_progress = 0;
conf->reshape_safe = conf->reshape_progress;
spin_unlock_irq(&conf->device_lock);
+ mddev_unlock_and_resume(mddev);
+ mddev_lock_nointr(mddev);
if (mddev->delta_disks && mddev->bitmap) {
struct mdp_superblock_1 *sb = NULL;
sector_t oldsize, newsize;
diff --git a/drivers/md/raid10.h b/drivers/md/raid10.h
index ec79d87fb92f..b711626a5db7 100644
--- a/drivers/md/raid10.h
+++ b/drivers/md/raid10.h
@@ -85,11 +85,11 @@ struct r10conf {
int have_replacement; /* There is at least one
* replacement device.
*/
wait_queue_head_t wait_barrier;
- mempool_t r10bio_pool;
+ mempool_t *r10bio_pool;
mempool_t r10buf_pool;
struct page *tmppage;
struct bio_set bio_split;
/* When taking over an array from a different personality, we store
--
2.54.0
^ permalink raw reply related
* [PATCH v3 0/2] md/raid10: fix r10bio width mismatches across reshape
From: Chen Cheng @ 2026-05-25 1:55 UTC (permalink / raw)
To: Yu Kuai, linux-raid; +Cc: Chen Cheng, linux-kernel
From: Chen Cheng <chencheng@fnnas.com>
Hi,
This series fixes slab out-of-bounds accesses in raid10 when reshape changes
the number of raid disks while regular I/O is still reusing r10bio objects
allocated under the previous geometry.
The bug is reproducible with a simple 4-disk to 5-disk reshape under write
load, for example:
mdadm -C /dev/md777 -l10 -n4 /dev/sda /dev/sdb /dev/sdc /dev/sdd
mkfs.ext4 /dev/md777
mount /dev/md777 /mnt/test
fsstress -d /mnt/test -n 24000 -p 8 -l 24 &
mdadm /dev/md777 --add /dev/sde
mdadm --grow /dev/md777 --raid-devices=5 \
--backup-file=/tmp/md-reshape-backup
Without these changes, an r10bio allocated under the old geometry can later be
reused, initialized, or freed after conf->geo.raid_disks has switched to the
new geometry. This creates width mismatches between the object and the current
devs[] walk/initialization width, which can trigger KASAN reports such as
slab-out-of-bounds in __make_request(), put_all_bios(), or find_bio_disk().
This series addresses the problem in two steps:
1. make the regular r10bio pool fixed-size across reshape transitions, and
move the pool rebuild into the freeze window before the live geometry
switch;
2. track the number of valid devs[] entries in each reused r10bio and use
that recorded width when walking devs[] after reshape.
Changes in v3:
- Replace freeze_array()/unfreeze_array() in raid10_start_reshape() with
mddev_suspend_and_lock_nointr()/mddev_unlock_and_resume(). freeze_array()
returns when nr_pending == nr_queued, which still allows retry-list items
to hold pool objects; mddev_suspend() provides the correct upper-layer
quiesce interface. (Suggested by Yu Kuai)
Changes in v2:
- add this cover letter
- convert r10bio_pool to a fixed-size kmalloc mempool
- rebuild r10bio_pool inside the freeze window before switching live reshape
geometry
- switch raid10_quiesce() to freeze_array()/unfreeze_array()
Testing:
- reproduced the original KASAN slab-out-of-bounds on 4-disk -> 5-disk
raid10 reshape with fsstress
- verified that this series fixes that reproducer
- exercised the 5-disk -> 4-disk reshape direction as well
Thanks,
Chen Cheng
Chen Cheng (2):
md/raid10: make r10bio_pool use fixed-size objects
md/raid10: bound reused r10bio devs[] walks by used_nr_devs
drivers/md/raid10.c | 65 ++++++++++++++++++++++++++++++++++-----------
drivers/md/raid10.h | 4 ++-
2 files changed, 53 insertions(+), 16 deletions(-)
--
2.54.0
^ permalink raw reply
* LBS setup
From: Piergiorgio Sartor @ 2026-05-24 10:27 UTC (permalink / raw)
To: linux-raid
Hi all,
I've some questions about some recent
report from kernel.
I assume from 7.x.y, from what I understood.
Not sure about the <= 6.18 reported.
In the boot log there is:
md0: echo current LBS to md/logical_block_size to prevent data loss issues from LBS changes.
Note: After setting, array will not be assembled in old kernels (<= 6.18)
I found the docs stating:
logical_block_size
... Write the value before starting array. ...
Now, should we really copy the value from:
/sys/class/block/md0/queue/logical_block_size
to:
/sys/class/block/md0/md/logical_block_size
???
Note, the first in my system is 512, the
second is 0.
Second, the docs says "before starting array".
In case this is due (copy the value), how to do
for a system array?
I mean, the array where the whole OS lies.
If needed, is there any way to do this automatically?
Using mdadm or something?
Thanks,
bye,
--
piergiorgio
^ permalink raw reply
* Re: [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Qu Wenruo @ 2026-05-24 4:13 UTC (permalink / raw)
To: dsterba; +Cc: linux-btrfs, hch, linux-raid
In-Reply-To: <e1961351-d563-4d73-b9f3-30063bbe7c4d@suse.com>
在 2026/5/24 08:16, Qu Wenruo 写道:
>
>
> 在 2026/5/23 23:53, David Sterba 写道:
>> On Fri, May 22, 2026 at 06:53:53PM +0930, Qu Wenruo wrote:
>>> Recently kernel RAID56 lib is trying to remove the unexpected
>>> single-data-RAID56 (2 disks RAID5 or 3 disk RAID5) support, meanwhile
>>> btrfs still supports such setup, which means in the long run btrfs has
>>> to handle such corner case by ourselves.
>>>
>>> Thankfully single-data-RAID56 is really RAID1/RAID1C3, since data and
>>> P/Q stripes all match each other, rotation also makes no difference.
>>>
>>> This patch will disguise those single-data-RAID56 chunks as
>>> RAID1/RAID1C3 chunks.
>>
>> I don't think this is the right way to fix it. Calculations are an
>> implementation detail and should be done at the time of the xor_gen or
>> raid6_call, not touching the upper level structures related to format.
>>
>> Instead, please add helpers that take the same parameters and do the
>> switch if there's the edge case of raid5 or raid6, using memcpy. Thanks.
>
> I'd say going dedicated memcpy() is an adhoc fix, and not any easier to
> maintain.
>
> Now you have two paths, one from the raid56 lib, one from the btrfs
> specific handling for the single-data specific corner case, and that no
> one else is going to review/utilize that corner case.
>
> From this point of view, reusing the existing RAID1/RAID1C3 is way more
> reliable and sane.
I tried the idea, and it doesn't look elegant at all, we need a lot of
if () checks around every xor_gen()/raid6_call.
Sure the end result is not that huge (and I have only compile tested
it), but if we had some parameters passed wrongly, or some corner case
missed, it can be very hard to notice.
Just paste the adhoc fix for reference:
diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c
index 08ee8f316d96..50fefe33bbe3 100644
--- a/fs/btrfs/raid56.c
+++ b/fs/btrfs/raid56.c
@@ -1410,11 +1410,17 @@ static void generate_pq_vertical_step(struct
btrfs_raid_bio *rbio, unsigned int
rbio_qstripe_paddr(rbio, sector_nr, step_nr));
assert_rbio(rbio);
- raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
+ if (rbio->nr_data > 1) {
+ raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
+ } else {
+ memcpy(pointers[1], pointers[0], step);
+ memcpy(pointers[2], pointers[0], step);
+ }
} else {
/* raid5 */
memcpy(pointers[rbio->nr_data], pointers[0], step);
- xor_gen(pointers[rbio->nr_data], pointers + 1, rbio->nr_data - 1,
+ if (rbio->nr_data > 1)
+ xor_gen(pointers[rbio->nr_data], pointers + 1, rbio->nr_data - 1,
step);
}
for (stripe = stripe - 1; stripe >= 0; stripe--)
@@ -1987,11 +1993,17 @@ static void recover_vertical_step(struct
btrfs_raid_bio *rbio,
}
if (failb == rbio->real_stripes - 2) {
- raid6_datap_recov(rbio->real_stripes, step,
- faila, pointers);
+ if (unlikely(rbio->nr_data == 1))
+ memcpy(pointers[0], pointers[rbio->real_stripes - 1], step);
+ else
+ raid6_datap_recov(rbio->real_stripes, step,
+ faila, pointers);
} else {
- raid6_2data_recov(rbio->real_stripes, step,
- faila, failb, pointers);
+ if (unlikely(rbio->nr_data == 1))
+ memcpy(pointers[0], pointers[rbio->real_stripes - 2], step);
+ else
+ raid6_2data_recov(rbio->real_stripes, step,
+ faila, failb, pointers);
}
} else {
void *p;
@@ -2002,6 +2014,9 @@ static void recover_vertical_step(struct
btrfs_raid_bio *rbio,
/* Copy parity block into failed block to start with */
memcpy(pointers[faila], pointers[rbio->nr_data], step);
+ if (rbio->nr_data == 1)
+ goto cleanup;
+
/* Rearrange the pointer array */
p = pointers[faila];
for (stripe_nr = faila; stripe_nr < rbio->nr_data - 1;
@@ -2644,11 +2659,17 @@ static bool verify_one_parity_step(struct
btrfs_raid_bio *rbio,
if (has_qstripe) {
assert_rbio(rbio);
/* RAID6, call the library function to fill in our P/Q. */
- raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
+ if (rbio->nr_data > 1) {
+ raid6_call.gen_syndrome(rbio->real_stripes, step, pointers);
+ } else {
+ memcpy(pointers[1], pointers[0], step);
+ memcpy(pointers[2], pointers[0], step);
+ }
} else {
/* RAID5. */
memcpy(pointers[nr_data], pointers[0], step);
- xor_gen(pointers[nr_data], pointers + 1, nr_data - 1, step);
+ if (rbio->nr_data > 1)
+ xor_gen(pointers[nr_data], pointers + 1, nr_data - 1, step);
}
/* Check scrubbing parity and repair it. */
--
2.54.0
^ permalink raw reply related
* Re: [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Qu Wenruo @ 2026-05-23 22:46 UTC (permalink / raw)
To: dsterba; +Cc: linux-btrfs, hch, linux-raid
In-Reply-To: <20260523142344.GC183694@twin.jikos.cz>
在 2026/5/23 23:53, David Sterba 写道:
> On Fri, May 22, 2026 at 06:53:53PM +0930, Qu Wenruo wrote:
>> Recently kernel RAID56 lib is trying to remove the unexpected
>> single-data-RAID56 (2 disks RAID5 or 3 disk RAID5) support, meanwhile
>> btrfs still supports such setup, which means in the long run btrfs has
>> to handle such corner case by ourselves.
>>
>> Thankfully single-data-RAID56 is really RAID1/RAID1C3, since data and
>> P/Q stripes all match each other, rotation also makes no difference.
>>
>> This patch will disguise those single-data-RAID56 chunks as
>> RAID1/RAID1C3 chunks.
>
> I don't think this is the right way to fix it. Calculations are an
> implementation detail and should be done at the time of the xor_gen or
> raid6_call, not touching the upper level structures related to format.
>
> Instead, please add helpers that take the same parameters and do the
> switch if there's the edge case of raid5 or raid6, using memcpy. Thanks.
I'd say going dedicated memcpy() is an adhoc fix, and not any easier to
maintain.
Now you have two paths, one from the raid56 lib, one from the btrfs
specific handling for the single-data specific corner case, and that no
one else is going to review/utilize that corner case.
From this point of view, reusing the existing RAID1/RAID1C3 is way more
reliable and sane.
^ permalink raw reply
* Re: [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: David Sterba @ 2026-05-23 14:23 UTC (permalink / raw)
To: Qu Wenruo; +Cc: linux-btrfs, hch, linux-raid
In-Reply-To: <ce02ebf8a9a6f5e2357b838df9d7ecce1f0cfd4b.1779412152.git.wqu@suse.com>
On Fri, May 22, 2026 at 06:53:53PM +0930, Qu Wenruo wrote:
> Recently kernel RAID56 lib is trying to remove the unexpected
> single-data-RAID56 (2 disks RAID5 or 3 disk RAID5) support, meanwhile
> btrfs still supports such setup, which means in the long run btrfs has
> to handle such corner case by ourselves.
>
> Thankfully single-data-RAID56 is really RAID1/RAID1C3, since data and
> P/Q stripes all match each other, rotation also makes no difference.
>
> This patch will disguise those single-data-RAID56 chunks as
> RAID1/RAID1C3 chunks.
I don't think this is the right way to fix it. Calculations are an
implementation detail and should be done at the time of the xor_gen or
raid6_call, not touching the upper level structures related to format.
Instead, please add helpers that take the same parameters and do the
switch if there's the edge case of raid5 or raid6, using memcpy. Thanks.
^ permalink raw reply
* Re: [PATCH RFC] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Christoph Hellwig @ 2026-05-22 12:08 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Christoph Hellwig, Qu Wenruo, linux-btrfs, linux-raid
In-Reply-To: <a8de8d07-d262-4f91-96eb-473fcded6549@gmx.com>
On Fri, May 22, 2026 at 07:00:25PM +0930, Qu Wenruo wrote:
> Not sure what specific workload you may want to verify, but my planned
> workload is:
>
> - Create single-data-RAID56 btrfs
> - Fsstress it
> - Unmount
>
> - Mount with one device missing, readonly
> - Do a readonly scrub, should be no error
> - Retry with the remaining device(s)
fsstress doesn't actually check data ingtegrity. So fsx is usually
a better choice for these kinds of tests. Otherwise this sounds good.
^ permalink raw reply
* Re: [PATCH v2 2/3] md/raid1,raid10: fix error-path detection with md_cloned_bio()
From: Abd-Alrhman Masalkhi @ 2026-05-22 11:02 UTC (permalink / raw)
To: Yu Kuai, Xiao Ni; +Cc: song, xni, neilb, shli, linux-raid, linux-kernel, yukuai
In-Reply-To: <aab7e7ea-737a-4bc6-a50c-f6ed7b38c37d@fygo.io>
On Fri, May 22, 2026 at 12:23 +0800, Yu Kuai wrote:
> Hi,
>
> 在 2026/5/21 17:14, Abd-Alrhman Masalkhi 写道:
>> The original bio resubmitted via bio_list is not executing in the raid1d
>> thread context, so blocking on is_suspended() is correct and will not
>> cause a deadlock. The suspension deadlock risk only exists for
>> md_cloned_bio executing in the raid1d thread context in the error path.
>> And in this case it is always true that if we are executing in the
>> raid1d thread context the bio will already be a md_cloned_bio on entry
>> to md_handle_request(), even if current->bio_list was set, because (and as
>> @Xiao has mentioned) every time we split a bio in the error path we are
>> resubmitting the md_cloned_bio remainder, never the original bio.
>
> I do understand this, if original bio or resubmitted bio only split at
> raid1_read_request, there is no problem. What I feel is not safe is that
> bio can split somewhere else.
>
> This is possible for raid5, However, take a closer look at raid1/10, I
> feel it's not possible for resubmitted io to split in bio_split_to_limits.
> Even if it happens, it's probably someone else to blame.
>
Agreed. Since bio_split_to_limits() has been mentioned, I am wondering
whether a resubmitted md_cloned_bio could end up being split there for
raid456. I have not looked into the raid456 code yet; after finishing
raid1 and raid10, I will move on to the others.
I am considering skipping the redundant call to bio_split_to_limits()
when md_cloned_bio() is true. However, I am not sure whether this would
be safe for raid456.
> --
> Thansk,
> Kuai
--
Best Regards,
Abd-Alrhman
^ permalink raw reply
* Re: [PATCH RFC] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Qu Wenruo @ 2026-05-22 9:30 UTC (permalink / raw)
To: Christoph Hellwig, Qu Wenruo; +Cc: linux-btrfs, linux-raid
In-Reply-To: <20260522085211.GA6702@lst.de>
在 2026/5/22 18:22, Christoph Hellwig 写道:
> Hi Qu,
>
> this does looks sensible to me. Maybe add an xfstests case that
> exercises these using loop devices?
Sure, I'll add a test case to verify the single-data-RAID56 behavior for
btrfs.
Not sure what specific workload you may want to verify, but my planned
workload is:
- Create single-data-RAID56 btrfs
- Fsstress it
- Unmount
- Mount with one device missing, readonly
- Do a readonly scrub, should be no error
- Retry with the remaining device(s)
If you have something else to add, I'm pretty happy to add.
Thanks,
Qu
^ permalink raw reply
* [PATCH 2/3] btrfs: remove duplicated block group type assignment
From: Qu Wenruo @ 2026-05-22 9:23 UTC (permalink / raw)
To: linux-btrfs; +Cc: hch, linux-raid
In-Reply-To: <cover.1779412152.git.wqu@suse.com>
In the function fill_dummy_bgs(), bg->flags is assigned twice.
Just remove the second assignment.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
fs/btrfs/block-group.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index a0cb0db68c9a..41f65a3165f8 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -2627,7 +2627,6 @@ static int fill_dummy_bgs(struct btrfs_fs_info *fs_info)
bg->flags = map->type;
bg->cached = BTRFS_CACHE_FINISHED;
bg->used = map->chunk_len;
- bg->flags = map->type;
bg->space_info = btrfs_find_space_info(fs_info, bg->flags);
ret = btrfs_add_block_group_cache(bg);
/*
--
2.54.0
^ permalink raw reply related
* [PATCH 1/3] btrfs: remove btrfs_chunk_map::io_(align|width) members
From: Qu Wenruo @ 2026-05-22 9:23 UTC (permalink / raw)
To: linux-btrfs; +Cc: hch, linux-raid
In-Reply-To: <cover.1779412152.git.wqu@suse.com>
Those two members are read from on-disk metadata, but never utilized.
And for new chunks we always set those members to BTRFS_STRIPE_LEN
anyway.
Thus there is no need to keep them inside btrfs_chunk_map.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
fs/btrfs/volumes.c | 4 ----
fs/btrfs/volumes.h | 2 --
2 files changed, 6 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 93a923e4ecaf..daa8f1ddf713 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -6019,8 +6019,6 @@ static struct btrfs_block_group *create_chunk(struct btrfs_trans_handle *trans,
map->chunk_len = ctl->chunk_size;
map->stripe_size = ctl->stripe_size;
map->type = type;
- map->io_align = BTRFS_STRIPE_LEN;
- map->io_width = BTRFS_STRIPE_LEN;
map->sub_stripes = ctl->sub_stripes;
map->num_stripes = ctl->num_stripes;
@@ -7583,8 +7581,6 @@ static int read_one_chunk(struct btrfs_key *key, struct extent_buffer *leaf,
map->start = logical;
map->chunk_len = length;
map->num_stripes = num_stripes;
- map->io_width = btrfs_chunk_io_width(leaf, chunk);
- map->io_align = btrfs_chunk_io_align(leaf, chunk);
map->type = type;
/*
* We can't use the sub_stripes value, as for profiles other than
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index 96904d18f686..f24385b599dc 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -633,8 +633,6 @@ struct btrfs_chunk_map {
u64 chunk_len;
u64 stripe_size;
u64 type;
- int io_align;
- int io_width;
int num_stripes;
int sub_stripes;
struct btrfs_io_stripe stripes[];
--
2.54.0
^ permalink raw reply related
* [PATCH 3/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Qu Wenruo @ 2026-05-22 9:23 UTC (permalink / raw)
To: linux-btrfs; +Cc: hch, linux-raid
In-Reply-To: <cover.1779412152.git.wqu@suse.com>
Recently kernel RAID56 lib is trying to remove the unexpected
single-data-RAID56 (2 disks RAID5 or 3 disk RAID5) support, meanwhile
btrfs still supports such setup, which means in the long run btrfs has
to handle such corner case by ourselves.
Thankfully single-data-RAID56 is really RAID1/RAID1C3, since data and
P/Q stripes all match each other, rotation also makes no difference.
This patch will disguise those single-data-RAID56 chunks as
RAID1/RAID1C3 chunks.
This is done at two timings:
- Chunk read
- Chunk allocation
This is done by introducing btrfs_chunk_map::on_disk_type member, which
stores the type read from the on-disk metadata.
Meanwhile btrfs_chunk_map::type is calculated using on_disk_type.
For most profiles @type matches @on_disk_type, but for
single-data-RAID56, the @type will be RAID1/RAID1C3.
This method has a minimal impact on the fs, all other operations like
scrub and read-repair, are all based on the chunk map type, so the
disguise method will require no extra modification to those call sites.
Although there are still some locations that are checking against
block_group->flags, e.g. scrub. Those call sites will still get extra
limits assuming the bg is RAID56. But it should not cause any extra
problem.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
fs/btrfs/block-group.c | 2 +-
fs/btrfs/volumes.c | 21 ++++++++++++++++++---
fs/btrfs/volumes.h | 8 ++++++++
3 files changed, 27 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 41f65a3165f8..5c9fc7ed4c3e 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -2624,7 +2624,7 @@ static int fill_dummy_bgs(struct btrfs_fs_info *fs_info)
/* Fill dummy cache as FULL */
bg->length = map->chunk_len;
- bg->flags = map->type;
+ bg->flags = map->on_disk_type;
bg->cached = BTRFS_CACHE_FINISHED;
bg->used = map->chunk_len;
bg->space_info = btrfs_find_space_info(fs_info, bg->flags);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index daa8f1ddf713..a754ab7a538f 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -6000,6 +6000,19 @@ struct btrfs_chunk_map *btrfs_alloc_chunk_map(int num_stripes, gfp_t gfp)
return map;
}
+static void set_real_chunk_type(struct btrfs_chunk_map *map)
+{
+ map->type = map->on_disk_type;
+ if (likely((map->on_disk_type & BTRFS_BLOCK_GROUP_RAID56_MASK) == 0 ||
+ nr_data_stripes(map) > 1))
+ return;
+ if (map->on_disk_type & BTRFS_BLOCK_GROUP_RAID5)
+ map->type |= BTRFS_BLOCK_GROUP_RAID1;
+ else
+ map->type |= BTRFS_BLOCK_GROUP_RAID1C3;
+ map->type &= ~BTRFS_BLOCK_GROUP_RAID56_MASK;
+}
+
static struct btrfs_block_group *create_chunk(struct btrfs_trans_handle *trans,
struct alloc_chunk_ctl *ctl,
struct btrfs_device_info *devices_info)
@@ -6018,9 +6031,10 @@ static struct btrfs_block_group *create_chunk(struct btrfs_trans_handle *trans,
map->start = start;
map->chunk_len = ctl->chunk_size;
map->stripe_size = ctl->stripe_size;
- map->type = type;
+ map->on_disk_type = type;
map->sub_stripes = ctl->sub_stripes;
map->num_stripes = ctl->num_stripes;
+ set_real_chunk_type(map);
for (int i = 0; i < ctl->ndevs; i++) {
for (int j = 0; j < ctl->dev_stripes; j++) {
@@ -6199,7 +6213,7 @@ int btrfs_chunk_alloc_add_chunk_item(struct btrfs_trans_handle *trans,
btrfs_set_stack_chunk_length(chunk, bg->length);
btrfs_set_stack_chunk_owner(chunk, BTRFS_EXTENT_TREE_OBJECTID);
btrfs_set_stack_chunk_stripe_len(chunk, BTRFS_STRIPE_LEN);
- btrfs_set_stack_chunk_type(chunk, map->type);
+ btrfs_set_stack_chunk_type(chunk, map->on_disk_type);
btrfs_set_stack_chunk_num_stripes(chunk, map->num_stripes);
btrfs_set_stack_chunk_io_align(chunk, BTRFS_STRIPE_LEN);
btrfs_set_stack_chunk_io_width(chunk, BTRFS_STRIPE_LEN);
@@ -7581,7 +7595,7 @@ static int read_one_chunk(struct btrfs_key *key, struct extent_buffer *leaf,
map->start = logical;
map->chunk_len = length;
map->num_stripes = num_stripes;
- map->type = type;
+ map->on_disk_type = type;
/*
* We can't use the sub_stripes value, as for profiles other than
* RAID10, they may have 0 as sub_stripes for filesystems created by
@@ -7592,6 +7606,7 @@ static int read_one_chunk(struct btrfs_key *key, struct extent_buffer *leaf,
*/
map->sub_stripes = btrfs_raid_array[index].sub_stripes;
map->verified_stripes = 0;
+ set_real_chunk_type(map);
if (num_stripes > 0)
map->stripe_size = btrfs_calc_stripe_length(map);
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index f24385b599dc..f75f516afe0d 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -632,7 +632,15 @@ struct btrfs_chunk_map {
u64 start;
u64 chunk_len;
u64 stripe_size;
+ /*
+ * The real type that is utilized during logical address mapping.
+ *
+ * For most profiles it matches @on_disk_type, but for single-data-RAID56,
+ * the real type will be set to RAID1/RAID1C3, to avoid unsupported
+ * operations from raid56 lib.
+ */
u64 type;
+ u64 on_disk_type;
int num_stripes;
int sub_stripes;
struct btrfs_io_stripe stripes[];
--
2.54.0
^ permalink raw reply related
* [PATCH 0/3] btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
From: Qu Wenruo @ 2026-05-22 9:23 UTC (permalink / raw)
To: linux-btrfs; +Cc: hch, linux-raid
[CHANGELOG]
RFC->v1:
- Remove unused btrfs_chunk_map::io_(align|width) members
To make space for the new member.
- Fix a duplicated bg->flags assignment in fill_dummy_bgs()
Just exposed by the last patch which also touches that code.
- Use a dedicated member to record the on-disk chunk type
So it's less fragile than the RFC patch.
The single-data-RAID56 (2 disks raid5 or 3 disks raid6) is not a feature
that should be supported in the first place, thus raid56 is going to
remove such support.
Meanwhile rejecting single-data-RAID56 will cause existing btrfs users
unable to mount their fs, even if there is only a single
empty chunk with that flag, mostly caused by some degraded mount.
So to avoid impacting existing users, implement an internal
single-data-RAID56 to RAID1/RAID1C3 mapping, to co-operate with the
upstream raid56 lib changes.
Qu Wenruo (3):
btrfs: remove btrfs_chunk_map::io_(align|width) members
btrfs: remove duplicated block group type assignment
btrfs: disguise single-data-RAID56 as RAID1/RAID1C3
fs/btrfs/block-group.c | 3 +--
fs/btrfs/volumes.c | 25 ++++++++++++++++++-------
fs/btrfs/volumes.h | 10 ++++++++--
3 files changed, 27 insertions(+), 11 deletions(-)
--
2.54.0
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox