Re: Weirdness with DDF arrays (mdadm 3.3)

[Martin Wilck <mwilck@arcor.de>]

On 09/14/2013 05:25 PM, Francis Moreau wrote:
> Hi Martin,
> 
> On Sat, Sep 14, 2013 at 4:16 PM, Martin Wilck <mwilck@arcor.de> wrote:
>> Hi Francis, hi Neil,
>>
>>> Still testing MD arrays using DDF metadata and find another possible issues :)
>>>
>>> I'm creating a new DDF array containing 2 disks. After that
>>> /proc/mdstat looks correct:
>>>
>>> # cat /proc/mdstat
>>> Personalities : [raid1]
>>> md124 : active raid1 loop0[1] loop1[0]
>>>       84416 blocks super external:/md125/0 [2/2] [UU]
>>>
>>> md125 : inactive loop1[1](S) loop0[0](S)
>>>       65536 blocks super external:ddf
>>>
>>> Now I'm stopping the array and restart it by incrementaly adding the 2 disks:
>>> # mdadm --stop /dev/md124
>>> # mdadm --stop /dev/md125
>>> # mdadm -IRs /dev/loop0
>>
>> This is wrong, because -IRs "wills can the mapfile for arrays that are
>> being incrementally assembled snd will try to start any that are not
>> already started".
>>
>> mdadm -IRs will first add /dev/loop0, then see that there is an
>> incomplete array, and start it.
>>
>>> # mdadm -IRs /dev/loop1
>>
>> Now you add /dev/loop1, but as the array is already started, it will be
>> added as a spare. That's what you see below.
> 
> Ah you're right, sorry for the noise, however I still doesn't
> understand my last point, please see the end of the email.
> 
>>
>> However, there is room for improvement here. The array hasn't been
>> written to, so even if it is started, it should be possible to re-add
>> the second disk cleanly.
>>
>> Looking into that.
>>
> 
> thanks
> 
>> Martin
>>
>>
>>> # cat /proc/mdstat
>>> Personalities : [raid1]
>>> md124 : active (auto-read-only) raid1 loop1[2] loop0[0]
>>>       84416 blocks super external:/md125/0 [2/1] [_U]
>>>
>>> md125 : inactive loop1[1](S) loop0[0](S)
>>>       65536 blocks super external:ddf
>>>
>>> Parsing mdstat content tells me disk "loop1" have a role number equal
>>> to 2 which is greater than 1 indicating that "loop1" is a spare disk
>>> and the "[_U]" below indicates "loop1" is down".
>>>
>>> Why is "loop1" down now ?
>>>
>>> I decided to still use the md device by creating a new partition on it:
>>> # fdisk /dev/md124
>>> ...
>>> Calling ioctl() to re-read partition table.
>>> Syncing disks.
>>>
>>> Now inspecting /proc/mdstat:
>>>
>>> # cat /proc/mdstat
>>> Personalities : [raid1]
>>> md124 : active raid1 loop1[2] loop0[0]
>>>       84416 blocks super external:/md125/0 [2/2] [UU]
>>>
>>> md125 : inactive loop1[1](S) loop0[0](S)
>>>       65536 blocks super external:ddf
>>>
>>> which looks even weirder: "loop1[2]" indicates that the disk is a
>>> spare one whereas "[UU]" tells me the opposite.
>>>
>>> Could you tell me if I'm wrong in my interpretation or what's going wrong ?
> 
> What about the loop1 in spare and [UU] indicating that loop1 is used ?

After you added loop1, the array was in read-auto state. Rebuild doesn't
start in this state.

When you created the partition table, the array went in active state and
was rebuilt. When you looked at mdstat again, the rebuild was already
finished. Therefore you got "[UU]" after that.

Wrt loop1[2], I think you interpret the [2] wrongly. It seems to be the
kernel index of the device somehow. The mdstat parsing code of mdadm
doesn't look at this number. If you look at
/sys/class/block/md124/md/dev-loop*/slot, the number should be correct -
I tried it here.

Perhaps someone else can comment on this, too.

/proc/mdstat can be pretty cryptic. If you are uncertain about anything,
look under /sys.

Martin

> 
> 
> Thanks