Re: some help for improvement in btrfs

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2022/8/16 10:47, hmsjwzb wrote:
> Hi Qu,
>
> Sorry for interrupt you so many times.
>
> As for
> 	scrub level checks at RAID56 substripe write time.
>
> Is this feature available in latest linux-next branch?

Nope, no one is working on that, thus no patches at all.

> Or may I need to get patches from mail list.
> What is the core function of this feature ?

The following small script would explain it pretty well:

   mkfs.btrfs -f -m raid5 -d raid5 -b 1G $dev1 $dev2 $dev3
   mount $dev1 $mnt

   xfs_io -f -c "pwrite -S 0xee 0 64K" $mnt/file1
   sync
   umount $mnt

   # Currupt data stripe 1 of full stripe of above 64K write
   xfs_io -f -c "pwrite -S 0xff 119865344 64K" $dev1

   mount $dev1 $mnt

   # Do a new write into data stripe 2,
   # We will trigger a RMW, which will use on-disk (corrupted) data to
   # generate new P/Q.
   xfs_io -f -c "pwrite -S 0xee 0 64K" -c sync $mnt/file2

   # Now we can no longer read file1, as its data is corrupted, and
   # above write generated new P/Q using corrupted data stripe 1,
   # preventing us to recover the data stripe 1.
   cat $mnt/file1 > /dev/null
   umount $mnt

Above script is the best way to demonstrate the "destructive RMW".
Although this is not btrfs specific (other RAID56 is also affected),
it's definitely a real problem.

There are several different directions to solve it:

- A way to add CSUM for P/Q stripes
   In theory this should be the easiest way implementation wise.
   We can easily know if a P/Q stripe is correct, then before doing
   RMW, we verify the result of P/Q.
   If the result doesn't match, we know some data stripe(s) are
   corrupted, then rebuild the data first before write.

   Unfortunately, this needs a on-disk format.

- Full stripe verification before writes
   This means, before we submit sub-stripe writes, we use some scrub like
   method to verify all data stripes first.
   Then we can do recovery if needed, then do writes.

   Unfortunately, scrub-like checks has quite some limitations.
   Regular scrub only works on RO block groups, thus extent tree and csum
   tree are consistent.
   But for RAID56 writes, we have no such luxury, I'm not 100% sure if
   this can even pass stress tests.

Thanks,
Qu

>
> I think I may use qemu and gdb to get basic understanding about this feature.
>
> Thanks,
> Flint
>
> On 8/15/22 04:54, Qu Wenruo wrote:
>> scrub level checks at RAID56 substripe write time.