Re: Tried to replace a drive in a raid 1 and all hell broke loose

[Nicholas D Steeves <nsteeves@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2639 bytes --]

Graham Cobb <g.btrfs@cobb.me.uk> writes:

> On 30/05/2022 21:47, Forza wrote:
>> 
>> 
>> I had a discussion with some Windows users, and they did exactly the
>> same thing - yanked the mirror out and then inserted it again. 4 times
>> out of 5 it "worked" and they got upset when it didn't work the last time.
>> 
>> So, with that said, there is room to improve documentation, man pages
>> and guides to help users find the information they need to check their
>> system correctly.
>> 
>> For now, mounting each mirror independently and then combine them again
>> is not good for Btrfs. This use-case seems to be unhandled.
>
> Sounds like btrfs should do something like assign the filesystem a
> completely new UUID (updated onto all the superblocks present at the
> time) if you mount degraded. To prevent any disks not present at that
> time from being reintroduced later.
>
> A bit drastic but that is what is really happening with a degraded
> mount: you are creating a new filesystem, with some of the contents
> inherited from an old one, and some missing.
>

Yes, I agree something should be done, but I'm not sure this is it.
Rather than this, I wonder why a multidisk profile of btrfs doesn't
do something like the following:

1. Maintain a list of devices that are part of the filesystem, using
/dev/disk/by-id or by-uuid identifiers.  At fs creation, these are
added to the "good list"

2. If ever the filesystem is mounted degraded, the IDs of missing
device[s] should be moved to a "bad list", and permanently blocked from
use.

3. If ever those IDs reappear (ie: they match an element of the "bad
list"), a warning should be emitted in the kernel log, and btrfs-progs
tools should warn that a "wipefs" of those devices is required before
readding them.

4. It also seems like it would be user-friendly to emit a warning if
ever single block groups are found on a on what should be a 100%
profile=raid{1,10,c3,c4,5,6} filesystem, because this is a dangerous
situation to be in.  This would signal that an urgent rebalance is
required after step #3.


Qu, is this possible with the current on-disk format?  If not, then
could something like this (specifically the "bad device list" at #2)
please be included in the design of the next on-disk format?  Ideally it
would be nice if reattached disks could replay all transactions since
they were detached, so maybe the future on-disk format could also
reserve a field for this?  The silent creation of profile=single blocks
makes using Btrfs profile=raid{1,c3,c4} risky when compared to ZFS
mirrors.

Regards,
Nicholas

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]