Re: regression: drive was detected as raid member due to metadata on partition

["Sven Köhler" <sven.koehler@gmail.com>]

Hi,

Am 10.04.24 um 03:56 schrieb Li Nan:
> Hi, Köhler
> 
> 在 2024/4/9 7:31, Sven Köhler 写道:
>> Hi,
>>
>> I was shocked to find that upon reboot, my Linux machine was detecting 
>> /dev/sd[abcd] as members of a raid array. It would assign those 
>> members to /dev/md4. It would not run the raid arrays /dev/mdX with 
>> members /dev/sda[abcd]X for X=1,2,3,4 as it usually did for the past 
>> couple of years.
>>
>> My server was probably a unicorn in the sense that it used metadata 
>> version 0.90. This version of software RAID metadata is stored at the 
>> _end_ of a partition. In my case, /dev/sda4 would be the last 
>> partition on drive /dev/sda. I confirmed with mdadm --examine that 
>> metadata with the identical UUID would be found on both /dev/sda4 and 
>> /dev/sda.
>>
> 
> I am trying to reproduce it, but after reboot, md0 started with members
> /dev/sd[bc]2 correctly. And mdadm will waring if assemble by 'mdadm -A'.
> 
>    # mdadm -CR /dev/md0 -l1 -n2 /dev/sd[bc]2 --metadata=0.9
>    # mdadm -S --scan
>    # mdadm -A --scan
>    mdadm: WARNING /dev/sde2 and /dev/sde appear to have very similar 
> superblocks.
>          If they are really different, please --zero the superblock on one
>          If they are the same or overlap, please remove one from the
>          DEVICE list in mdadm.conf.
>    mdadm: No arrays found in config file or automatically
> 
> Can you tell me how you create and config the RAID?

I should have mentioned the mdadm and kernel version. I am using mdadm 
4.3-2 and linux-lts 6.6.23-1 on Arch Linux.

I created the array very similar to what you did:
mdadm --create /dev/md4 --level=6 --raid-devices=4 --metadata=0.90 
/dev/sd[abcd]4

My mdadm.conf looks like this:
DEVICE partitions
ARRAY /dev/md/4 metadata=0.90  UUID=...

And /proc/partitions looks like this:

major minor  #blocks  name
    8        0 2930266584 sda
    8        1    1048576 sda1
    8        2   33554432 sda2
    8        3   10485760 sda3
    8        4 2885176775 sda4
    8       16 2930266584 sdb
    8       17    1048576 sdb1
    8       18   33554432 sdb2
    8       19   10485760 sdb3
    8       20 2885176775 sdb4
    8       32 2930266584 sdc
    8       33    1048576 sdc1
    8       34   33554432 sdc2
    8       35   10485760 sdc3
    8       36 2885176775 sdc4
    8       48 2930266584 sdd
    8       49    1048576 sdd1
    8       50   33554432 sdd2
    8       51   10485760 sdd3
    8       52 2885176775 sdd4


Interestingly, sda, sdb, etc. are included. So "DEVICE partitions" 
actually considers them.


>> Here's what I think went wrong: I believe either the kernel or mdadm 
>> (likely the latter) was seeing the metadata at the end of /dev/sda and 
>> ignored the fact that the location of the metadata was actually owned 
>> by a partition (namely /dev/sda4). The same happened for /dev/sd[bcd] 
>> and thus I ended up with /dev/md4 being started with members 
>> /dev/sda[abcd] instead of members /dev/sda[abcd]4.
>>
>> This behavior started recently. I saw in the logs that I had updated 
>> mdadm but also the Linux kernel. mdadm and an appropriate mdadm.conf 
>> is part of my initcpio. My mdadm.conf lists the arrays with their 
>> metadata version and their UUID.
>>
>> Starting a RAID array with members /dev/sda[abcd] somehow removed the 
>> partitions of the drives. The partition table would still be present, 
>> but the partitions would disappear from /dev. So /dev/sda[abcd]1-3 
>> were not visible anymore and thus /dev/md1-3 would not be started.
>>
>> I strongly believe that mdadm should ignore any metadata - regardless 
>> of the version - that is at a location owned by any of the partitions. 
>> While I'm not 100% sure how to implement that, the following might 
>> also work: first scan the partitions for metadata, then ignore if the 
>> parent device has metadata with a UUID previously found.
>>
>>
>> I did the right thing and converted my RAID arrays to metadata 1.2, 
>> but I'd like to save other from the adrenaline shock.
>>
>>
>>
>> Kind Regards,
>>    Sven
>>
>> .
>