Re: some help for improvement in btrfs

Qu Wenruo
2022-08-16T08:19:15Z

On 2022/8/16 10:47, hmsjwzb wrote:
> Hi Qu,
>
> Sorry for interrupt you so many times.
>
> As for
> scrub level checks at RAID56 substripe write time.
>
> Is this feature available in latest linux-next branch?

Nope, no one is working on that, thus no patches at all.

> Or may I need to get patches from mail list.
> What is the core function of this feature ?

The following small script would explain it pretty well:

mkfs.btrfs -f -m raid5 -d raid5 -b 1G $dev1 $dev2 $dev3
mount $dev1 $mnt

xfs_io -f -c "pwrite -S 0xee 0 64K" $mnt/file1
sync
umount $mnt

# Currupt data stripe 1 of full stripe of above 64K write
xfs_io -f -c "pwrite -S 0xff 119865344 64K" $dev1

mount $dev1 $mnt

# Do a new write into data stripe 2,
# We will trigger a RMW, which will use on-disk (corrupted) data to
# generate new P/Q.
xfs_io -f -c "pwrite -S 0xee 0 64K" -c sync $mnt/file2

# Now we can no longer read file1, as its data is corrupted, and
# above write generated new P/Q using corrupted data stripe 1,
# preventing us to recover the data stripe 1.
cat $mnt/file1 > /dev/null
umount $mnt

Above script is the best way to demonstrate the "destructive RMW".
Although this is not btrfs specific (other RAID56 is also affected),
it's definitely a real problem.

There are several different directions to solve it:

- A way to add CSUM for P/Q stripes
In theory this should be the easiest way implementation wise.
We can easily know if a P/Q stripe is correct, then before doing
RMW, we verify the result of P/Q.
If the result doesn't match, we know some data stripe(s) are
corrupted, then rebuild the data first before write.

Unfortunately, this needs a on-disk format.

- Full stripe verification before writes
This means, before we submit sub-stripe writes, we use some scrub like
method to verify all data stripes first.
Then we can do recovery if needed, then do writes.

Unfortunately, scrub-like checks has quite some limitations.
Regular scrub only works on RO block groups, thus extent tree and csum
tree are consistent.
But for RAID56 writes, we have no such luxury, I'm not 100% sure if
this can even pass stress tests.

Thanks,
Qu

>
> I think I may use qemu and gdb to get basic understanding about this feature.
>
> Thanks,
> Flint
>
> On 8/15/22 04:54, Qu Wenruo wrote:
>> scrub level checks at RAID56 substripe write time.