Re: Strange / inconsistent behavior with mdadm -I -R

[NeilBrown <neilb@suse.de>]

[-- Attachment #1: Type: text/plain, Size: 3224 bytes --]

On Tue, 19 Mar 2013 19:29:44 +0100 Martin Wilck <mwilck@arcor.de> wrote:

> On 03/18/2013 12:35 AM, NeilBrown wrote:
> 
> > With -R, it only stays inactive until we have the minimum devices needed for
> > a functioning array.  Then it will switch to 'read-auto' or, if the array is
> > known and all expected devices are present, it will switch to 'active'.
> 
> That's the point - I see the array switch to active *before* all
> expected devices are present. That causes further additions to fail.
> 
> > Maybe if you could provide a sequence of "mdadm" commands that produces an
> > outcome different to what you would expect - that would reduce the chance
> > that I get confused.
> 
> Here is the sequence of mdadm commands to illustrate what I mean.
> I have a RAID10 array and I add the devices using mdadm -I -R in the
> sequence 1,3,2,4. After adding device 3, the array will be started in
> auto-read-only mode, which is fine.
> 
> But then as soon as the next disk (/dev/tosh/rd2) is added, the array
> switches to "active" although it is neither written to, nor all disks
> have been added yet. Consequently, adding disk 4 fails.
> 
> I expected the array to remain "auto-read-only" until either all 4
> devices are present, or it is opened for writing. This is how the
> container case is behaving (almost - it doesn't switch to active
> automatically until it's written to).
> 
> # ./mdadm -C /dev/md0 -l 1 -n 4 /dev/tosh/rd[1-4] -pn2
> mdadm: array /dev/md0 started.
> (wait for initial build to finish)
> # mdadm -S /dev/md0
> mdadm: stopped /dev/md0
> # ./mdadm -v -I /dev/tosh/rd1 -R
> mdadm: /dev/tosh/rd1 attached to /dev/md/0, not enough to start (1).
> # ./mdadm -v -I /dev/tosh/rd3 -R
> mdadm: /dev/tosh/rd3 attached to /dev/md/0, which has been started.
> # cat /proc/mdstat
> Personalities : [raid1] [raid10]
> md0 : active (auto-read-only) raid10 dm-6[2] dm-4[0]
>       2094080 blocks super 1.2 512K chunks 2 near-copies [4/2] [U_U_]
> # ./mdadm -v -I /dev/tosh/rd2 -R; cat /proc/mdstat
> mdadm: /dev/tosh/rd2 attached to /dev/md/0 which is already active.
> Personalities : [raid1] [raid10]
> md0 : active raid10 dm-5[1] dm-6[2] dm-4[0]
>       2094080 blocks super 1.2 512K chunks 2 near-copies [4/2] [U_U_]
>       [>....................]  recovery =  0.0% (0/1047040)
> finish=1090.6min speed=0K/sec
> (wait for recovery to finish)
> # ./mdadm -v -I /dev/tosh/rd4 -R
> mdadm: can only add /dev/tosh/rd4 to /dev/md/0 as a spare, and
> force-spare is not set.
> mdadm: failed to add /dev/tosh/rd4 to existing array /dev/md/0: Invalid
> argument.
> 
> Thanks,
> Martin

Thanks, that makes is all very clear.

The problem is that the ADD_NEW_DISK ioctl automatically converts from
read-auto to active.
There are two approaches I could take to addressing this.
1/ change ADD_NEW_DISK to not cause that conversion.  I think that would need
   to be conditional as some times it really should be changed.
2/ change mdadm to not use ADD_NEW_DISK but instead add the disk my setting it
   up via sysfs.

I'm not sure which is best and neither is completely straight forward.
So for now:  I'll get back to you.

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]