Re: [PATCH DRAFT] btrfs: RAID56J journal on-disk format draft

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2022/5/25 17:00, Christoph Hellwig wrote:
> On Tue, May 24, 2022 at 07:02:34PM +0200, David Sterba wrote:
>> Well, that does not sound encouraging. One option discussed in the past
>> how to fix the write hole was to always do full RMW cycle. Having a "not
>> fast journal at all" would require a format change and have probably a
>> comparable performance drop.
>
> So maybe I'm just dumb, but what is the problem with only using
> raid56 for data, forbidding nowcow for it and thus avoiding the
> problem entirely?

The problem is, we can have partial write for RAID56, no matter if we
use NODATACOW or not.

For example, we have a very typical 3 disks RAID5:

	0	32K	64K
Disk 1  |DDDDDDD|       |
Disk 2  |ddddddd|ddddddd|
Disk 3  |PPPPPPP|PPPPPPP|


D = old data, it's there for a while.
d = new data, we want to write.

Now bio for disk 1 and disk 3 finished, but before bio for disk2 can
finish, we hit power loss.

Btrfs reverts to old data, so we should still only see D, but no new data d.

So far so good.

But what if disk 1 now disappear?

To read out old data (DDDDD), we need to rebuild using disk2 and disk3.

But please note that, now Disk 3 has the new parity, but disk2 is still
old data.

Now the recovered data will be wrong, and not pass btrfs csum check.

This is the write-hole problem, it's not screwing up all our data by a
sudden, but corrupt out data bytes by bytes each time we hit a power loss.

Thanks,
Qu