conf->min_offset_diff calculation timing of growing raid5

conf->min_offset_diff calculation timing of growing raid5

[Amy Chiang]

Hi,
I was trying to find a way to make rehape fast but safe. Since growing
a 3-SSD raid5 array to 4-SSD raid6 with backup file specified takes
180 minutes and it only takes 30 minutes if there's no backup file.
I tried to do it with command
          mdadm --grow /dev/md0 --level=6 --raid-device=4

I didn't specify --backup-file because I thought mdadm would give me a
warning if there's a possibility that I might lose my data if this
operation crash.
From the trace I added in function reshape_request in kernel I found
        1. metadata only updated every 10 seconds
        2. conf->min_offset_diff is always 0 thus makes no difference
to write/safe/readpos calculation.

Then I made a test, I cut off the power before the first metadata
update but the new stripe has been overlapped with the old data. The
reshape started from the beginning after I turn on the power and the
array started again. This sudden power loss actually crashed my data.

And if I calculate conf->min_offset_diff before reshape started, none
of the problem mentioned above would happen and it takes only around
30 minutes for doing reshape.

So I was wondering if calculating conf->min_offset_diff before doing
reshape can keep data safe when there's no backup file, and if it can,
why did kernel calculate this value when array starts instead of
before doing reshape.

Thanks,

Re: conf->min_offset_diff calculation timing of growing raid5

[Wols Lists]

On 02/01/18 03:49, Amy Chiang wrote:
> I didn't specify --backup-file because I thought mdadm would give me a
> warning if there's a possibility that I might lose my data if this
> operation crash.

I'm slightly surprised at your results (although I shouldn't be :-).
Backup files are now almost entirely legacy - a modern setup should
pretty much never need them - which is why I'm surprised at your
results, I would have expected mdadm to ignore the backup-file option.

In the old days, a reshape used to copy a stripe to backup, and then
rewrite it. Rinse, lather and repeat. So the i/o overhead was high. Now,
with "data offset" and superblock 1, mdadm calculates where the reshaped
array is going to be, then starts copying from the old to array to where
the new array does not overlap the old array. This makes a window where
there are two copies of the data. When it runs out of space, it moves
the window to create more space and carries on copying. As you can see,
this is a lot less work ...

Cheers,
Wol