Re: RAID56 discussion related to RST. (Was "Re: [RFC ONLY 0/8] btrfs: introduce raid-stripe-tree")

Johannes Thumshirn
2022-07-13T12:43:19Z

On 13.07.22 14:01, Qu Wenruo wrote:
>
>
> On 2022/7/13 19:43, Johannes Thumshirn wrote:
>> On 13.07.22 12:54, Qu Wenruo wrote:
>>>
>>>
>>> On 2022/5/16 22:31, Johannes Thumshirn wrote:
>>>> Introduce a raid-stripe-tree to record writes in a RAID environment.
>>>>
>>>> In essence this adds another address translation layer between the logical
>>>> and the physical addresses in btrfs and is designed to close two gaps. The
>>>> first is the ominous RAID-write-hole we suffer from with RAID5/6 and the
>>>> second one is the inability of doing RAID with zoned block devices due to the
>>>> constraints we have with REQ_OP_ZONE_APPEND writes.
>>>
>>> Here I want to discuss about something related to RAID56 and RST.
>>>
>>> One of my long existing concern is, P/Q stripes have a higher update
>>> frequency, thus with certain transaction commit/data writeback timing,
>>> wouldn't it cause the device storing P/Q stripes go out of space before
>>> the data stripe devices?
>>
>> P/Q stripes on a dedicated drive would be RAID4, which we don't have.
>
> I'm just using one block group as an example.
>
> Sure, the next bg can definitely go somewhere else.
>
> But inside one bg, we are still using one zone for the bg, right?

Ok maybe I'm not understanding the code in volumes.c correctly, but
doesn't __btrfs_map_block() calculate a rotation per stripe-set?

I'm looking at this code:

/* Build raid_map */
if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK && need_raid_map &&
(need_full_stripe(op) || mirror_num > 1)) {
u64 tmp;
unsigned rot;

/* Work out the disk rotation on this stripe-set */
div_u64_rem(stripe_nr, num_stripes, &rot);

/* Fill in the logical address of each stripe */
tmp = stripe_nr * data_stripes;
for (i = 0; i < data_stripes; i++)
bioc->raid_map[(i + rot) % num_stripes] =
em->start + (tmp + i) * map->stripe_len;

bioc->raid_map[(i + rot) % map->num_stripes] = RAID5_P_STRIPE;
if (map->type & BTRFS_BLOCK_GROUP_RAID6)
bioc->raid_map[(i + rot + 1) % num_stripes] =
RAID6_Q_STRIPE;

sort_parity_stripes(bioc, num_stripes);
}

So then in your example we have something like this:

Write of 4k D1:

0 32K 64K
Disk 1 |D1 |
Disk 2 | |
Disk 3 |P1 |

Write of 4k D2, the new parity is P2 the old P1 parity is obsolete

0 32K 64K
Disk 1 |D1 |
Disk 2 |P2 |
Disk 3 |P1D2 |

Write of new 4k D1 with P3

0 32K 64K
Disk 1 |D1P3 |
Disk 2 |P2D1 |
Disk 3 |P1D2 |

and so on.