problems with raid=noautodetect

problems with raid=noautodetect

[Florian Dazinger]

hi list,
I read somewhere that it would be better not to rely on the 
autodetect-mechanism in the kernel at boot time, but rather to set up 
/etc/mdadm.conf accordingly and boot with raid=noautodetect. Well, I 
tried that :)

I set up /etc/mdadm.conf for my 2 raid5 arrays:

---- snip ----
# mountpoint: /home/media
ARRAY  /dev/md0
    level=raid5
    UUID=86ed1434:43380717:4abf124e:970d843a
    devices=/dev/sda1,/dev/sdb1,/dev/sdd3

# mountpoint: /mnt/raid
ARRAY  /dev/md1
    level=raid5
    UUID=baf59fb5:f4805e7a:91a77644:af3dde17
#   devices=/dev/sda2,/dev/sdb2,/dev/sdd2
---- snap ----

and rebooted with raid=noautodetect. It booted fine, but the 3rd disks 
from each array (/dev/sdd2 and /dev/sdd3) were removed, so I had 2 
degraded raid5 arrays. It was possible to readd them with sth. like:

mdadm /dev/md0 -a /dev/sdd3
(synced and /proc/mdstat showed [UUU])

but after the next reboot, the two partitions were again removed 
([UU_])?! This was a reproducible error, I tried it several times with 
different /etc/mdadm.conf settings (ARRAY-statement with UUID=, 
devices=, UUID+devices, etc.).

I´m now running autodetect again, all raid arrays are working fine, but 
can anyone explain this strange behaviour?
(kernel-2.6.16.14, amd64)

thanks,
florian

PS: please cc me, as I´m not subscribed to the list
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: problems with raid=noautodetect

[Neil Brown]

On Monday May 22, florian.dazinger@umit.at wrote:
> hi list,
> I read somewhere that it would be better not to rely on the 
> autodetect-mechanism in the kernel at boot time, but rather to set up 
> /etc/mdadm.conf accordingly and boot with raid=noautodetect. Well, I 
> tried that :)
> 
> I set up /etc/mdadm.conf for my 2 raid5 arrays:
> 
> ---- snip ----
> # mountpoint: /home/media
> ARRAY  /dev/md0
>     level=raid5
>     UUID=86ed1434:43380717:4abf124e:970d843a
>     devices=/dev/sda1,/dev/sdb1,/dev/sdd3
> 
> # mountpoint: /mnt/raid
> ARRAY  /dev/md1
>     level=raid5
>     UUID=baf59fb5:f4805e7a:91a77644:af3dde17
> #   devices=/dev/sda2,/dev/sdb2,/dev/sdd2
> ---- snap ----

Presumably you have a 'DEVICE' line in mdadm.conf too?  What is it.
My first guess is that it isn't listing /dev/sdd? somehow.

Otherwise, can you add a '-v' to the mdadm command that assembles the
array, and capture the output.  That might be helpful.

NeilBrown


> 
> and rebooted with raid=noautodetect. It booted fine, but the 3rd disks 
> from each array (/dev/sdd2 and /dev/sdd3) were removed, so I had 2 
> degraded raid5 arrays. It was possible to readd them with sth. like:
> 
> mdadm /dev/md0 -a /dev/sdd3
> (synced and /proc/mdstat showed [UUU])
> 
> but after the next reboot, the two partitions were again removed 
> ([UU_])?! This was a reproducible error, I tried it several times with 
> different /etc/mdadm.conf settings (ARRAY-statement with UUID=, 
> devices=, UUID+devices, etc.).
> 
> I´m now running autodetect again, all raid arrays are working fine, but 
> can anyone explain this strange behaviour?
> (kernel-2.6.16.14, amd64)
> 
> thanks,
> florian
> 
> PS: please cc me, as I´m not subscribed to the list
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: problems with raid=noautodetect - solved

[Florian Dazinger]

Neil Brown wrote:
> Presumably you have a 'DEVICE' line in mdadm.conf too?  What is it.
> My first guess is that it isn't listing /dev/sdd? somehow.
> 
> Otherwise, can you add a '-v' to the mdadm command that assembles the
> array, and capture the output.  That might be helpful.
> 
> NeilBrown
> 
stupid me! I had a DEVICE section, but somehow forgot about my /dev/sdd 
drive.

sorry for the noise..

thanks,
florian

Re: problems with raid=noautodetect - solved

[Nix]

On 24 May 2006, Florian Dazinger uttered the following:
> Neil Brown wrote:
>> Presumably you have a 'DEVICE' line in mdadm.conf too?  What is it.
>> My first guess is that it isn't listing /dev/sdd? somehow.
>> Otherwise, can you add a '-v' to the mdadm command that assembles the
>> array, and capture the output.  That might be helpful.
>> NeilBrown
>>
> stupid me! I had a DEVICE section, but somehow forgot about my /dev/sdd drive.

`DEVICE partitions' is generally preferable for that reason, unless
you have entries in /proc/partitions which you explicitly want to
exclude from scanning for RAID superblocks.

-- 
`On a scale of 1-10, X's "brokenness rating" is 1.1, but that's only
 because bringing Windows into the picture rescaled "brokenness" by
 a factor of 10.' --- Peter da Silva

Re: problems with raid=noautodetect

[Luca Berra]

On Tue, May 23, 2006 at 08:39:26AM +1000, Neil Brown wrote:
>Presumably you have a 'DEVICE' line in mdadm.conf too?  What is it.
>My first guess is that it isn't listing /dev/sdd? somehow.

Neil,
i am seeing a lot of people that fall in this same error, and i would
propose a way of avoiding this problem

1) make "DEVICE partitions" the default if no device line is specified.
2) deprecate the "DEVICE" keyword issuing a warning when it is found in
the configuration file
3) introduce "DEVICEFILTER" or similar keyword with the same meaning at
the actual "DEVICE" keyboard
4) optionally add an "EXCLUDEDEVICE" keyword with the opposite meaning.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: problems with raid=noautodetect

[Luca Berra]

On Fri, May 26, 2006 at 09:53:08AM +0200, Luca Berra wrote:
>On Tue, May 23, 2006 at 08:39:26AM +1000, Neil Brown wrote:
>>Presumably you have a 'DEVICE' line in mdadm.conf too?  What is it.
>>My first guess is that it isn't listing /dev/sdd? somehow.
>
>Neil,
>i am seeing a lot of people that fall in this same error, and i would
>propose a way of avoiding this problem
>
>1) make "DEVICE partitions" the default if no device line is specified.
oops,
just read your 2.5 announce, you already did that :)
>2) deprecate the "DEVICE" keyword issuing a warning when it is found in
>the configuration file
>3) introduce "DEVICEFILTER" or similar keyword with the same meaning at
>the actual "DEVICE" keyboard
>4) optionally add an "EXCLUDEDEVICE" keyword with the opposite meaning.
>

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: problems with raid=noautodetect

[Neil Brown]

On Friday May 26, bluca@comedia.it wrote:
> On Tue, May 23, 2006 at 08:39:26AM +1000, Neil Brown wrote:
> >Presumably you have a 'DEVICE' line in mdadm.conf too?  What is it.
> >My first guess is that it isn't listing /dev/sdd? somehow.
> 
> Neil,
> i am seeing a lot of people that fall in this same error, and i would
> propose a way of avoiding this problem
> 
> 1) make "DEVICE partitions" the default if no device line is specified.

As you note, we think alike on this :-)

> 2) deprecate the "DEVICE" keyword issuing a warning when it is found in
> the configuration file

Not sure I'm so keen on that, at least not in the near term.


> 3) introduce "DEVICEFILTER" or similar keyword with the same meaning at
> the actual "DEVICE" keyboard

If it has the same meaning, why not leave it called 'DEVICE'???

However, there is at least the beginnings of a good idea here.

If we assume there is a list of devices provided by a (possibly
default) 'DEVICE' line, then 

DEVICEFILTER   !pattern1 !pattern2 pattern3 pattern4

could mean that any device in that list which matches pattern 1 or 2
is immediately discarded, and remaining device that matches patterns 3
or 4 are included, and the remainder are discard.

The rule could be that the default is to include any devices that
don't match a !pattern, unless there is a pattern without a '!', in
which case the default is to reject non-accepted patterns.
Is that straight forward enough, or do I need an
  order allow,deny
like apache has?


Thanks for the suggestion.

NeilBrown

Re: problems with raid=noautodetect

[Luca Berra]

On Mon, May 29, 2006 at 02:21:09PM +1000, Neil Brown wrote:
>> 3) introduce "DEVICEFILTER" or similar keyword with the same meaning at
>> the actual "DEVICE" keyboard
>
>If it has the same meaning, why not leave it called 'DEVICE'???

the idea was to warn people that write

DEVICE /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
ARRAY /dev/md0 .......

that it might break since disk naming is not guaranteed to be constant.

>However, there is at least the beginnings of a good idea here.
>
>If we assume there is a list of devices provided by a (possibly
>default) 'DEVICE' line, then 
>
>DEVICEFILTER   !pattern1 !pattern2 pattern3 pattern4
>
>could mean that any device in that list which matches pattern 1 or 2
>is immediately discarded, and remaining device that matches patterns 3
>or 4 are included, and the remainder are discard.
>
>The rule could be that the default is to include any devices that
>don't match a !pattern, unless there is a pattern without a '!', in
>which case the default is to reject non-accepted patterns.
>Is that straight forward enough, or do I need an
>  order allow,deny
>like apache has?
>
I think that documenting the feature would be enough
L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: problems with raid=noautodetect

[Michael Tokarev]

Neil Brown wrote:
> On Friday May 26, bluca@comedia.it wrote:
[]
> If we assume there is a list of devices provided by a (possibly
> default) 'DEVICE' line, then 
> 
> DEVICEFILTER   !pattern1 !pattern2 pattern3 pattern4
> 
> could mean that any device in that list which matches pattern 1 or 2
> is immediately discarded, and remaining device that matches patterns 3
> or 4 are included, and the remainder are discard.
> 
> The rule could be that the default is to include any devices that
> don't match a !pattern, unless there is a pattern without a '!', in
> which case the default is to reject non-accepted patterns.
> Is that straight forward enough, or do I need an
>   order allow,deny
> like apache has?

I'd suggest the following.

"All the other devices" are included or excluded from the list of devices
to consider based on the last component in the DEVICE line.  Ie. if it
ends up at !dev, all the rest of devices are included.  If it ends up at
dev (w/o !), all the rest are excluded.  If memory serves me right, it's
how squid ACLs works.

There's no need to introduce new keyword.  Given this rule, a line

 DEVICE a b c

will do exactly as it does now.  Line

 DEVICE a b c !d

is somewhat redundant - it's the same as DEVICE !d
Ie, if the list ends up at !stuff, append `partitions' (or *) to it.

Ofcourse mixing !s and !!s is useful, like to say use all sda* but not
sda1:

 DEVICE !sda1 sda*

(and nothing else).

And the default is to have `DEVICE partitions'.

The only possible issue I see here is that with udev, it's possible to
use, say, /dev/disk/by-id/*-like stuff (don't remember exact directory
layout) -- symlinked to /dev/sd* according to the disk serial number or
something like that -- for this to work, mdadm needs to use glob()
internally.

/mjt

Re: problems with raid=noautodetect

[Luca Berra]

On Mon, May 29, 2006 at 12:38:25PM +0400, Michael Tokarev wrote:
>Neil Brown wrote:
>> On Friday May 26, bluca@comedia.it wrote:
>I'd suggest the following.
>
>"All the other devices" are included or excluded from the list of devices
>to consider based on the last component in the DEVICE line.  Ie. if it
>ends up at !dev, all the rest of devices are included.  If it ends up at
>dev (w/o !), all the rest are excluded.  If memory serves me right, it's
>how squid ACLs works.
>
>There's no need to introduce new keyword.  Given this rule, a line

as i said the new keyword is to warn on configurations that do not
account for changing device-ids, and if we change the syntax a new
keyword would make it clearer. In case the user tries to use a new
configuration on an old mdadm.

>The only possible issue I see here is that with udev, it's possible to
>use, say, /dev/disk/by-id/*-like stuff (don't remember exact directory
>layout) -- symlinked to /dev/sd* according to the disk serial number or
>something like that -- for this to work, mdadm needs to use glob()
>internally.

uhm
i think that we would better translate any device found on a DEVICE (or
DEVICEFILTER) line to the corresponding major/minor number and blacklist
based on that.
nothing prevents someone to have an udev rule that creates a device
node, instead of symlinking.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: problems with raid=noautodetect

[Bill Davidsen]

Neil Brown wrote:

>On Friday May 26, bluca@comedia.it wrote:
>  
>
>>On Tue, May 23, 2006 at 08:39:26AM +1000, Neil Brown wrote:
>>    
>>
>>>Presumably you have a 'DEVICE' line in mdadm.conf too?  What is it.
>>>My first guess is that it isn't listing /dev/sdd? somehow.
>>>      
>>>
>>Neil,
>>i am seeing a lot of people that fall in this same error, and i would
>>propose a way of avoiding this problem
>>
>>1) make "DEVICE partitions" the default if no device line is specified.
>>    
>>
>
>As you note, we think alike on this :-)
>
>  
>
>>2) deprecate the "DEVICE" keyword issuing a warning when it is found in
>>the configuration file
>>    
>>
>
>Not sure I'm so keen on that, at least not in the near term.
>
Let's not start warning and depreciating powerful features because they 
can be misused... If I wanted someone to make decisions for me I would 
be using this software at all.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: problems with raid=noautodetect

[Luca Berra]

On Tue, May 30, 2006 at 01:10:24PM -0400, Bill Davidsen wrote:
>>>2) deprecate the "DEVICE" keyword issuing a warning when it is found in
>>>the configuration file
>>
>>Not sure I'm so keen on that, at least not in the near term.
>>
>Let's not start warning and depreciating powerful features because they 
>can be misused... If I wanted someone to make decisions for me I would 
>be using this software at all.

you cut the rest of the mail.
i did not propose to deprecate the feature,
just the keyword.

but, ok,
just go on writing 
DEVICE /dev/sda1
DEVICE /dev/sdb1
ARRAY /dev/md0 devices=/dev/sda1,/dev/sdb1

then come on the list and complain when it stops working.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: problems with raid=noautodetect

[Bill Davidsen]

Luca Berra wrote:

> On Tue, May 30, 2006 at 01:10:24PM -0400, Bill Davidsen wrote:
>
>>>> 2) deprecate the "DEVICE" keyword issuing a warning when it is 
>>>> found in
>>>> the configuration file
>>>
>>>
>>> Not sure I'm so keen on that, at least not in the near term.
>>>
>> Let's not start warning and depreciating powerful features because 
>> they can be misused... If I wanted someone to make decisions for me I 
>> would be using this software at all.
>
>
> you cut the rest of the mail.

Trimming the part about which I make no comment is usually a good thing.

> i did not propose to deprecate the feature,
> just the keyword.

"A rose by any other name would still smell as sweet." In other words, 
the capability is still able to be misused, and changing the name or 
generating error messages will only cause work and concern for people 
using the feature.

>
> but, ok,
> just go on writing DEVICE /dev/sda1
> DEVICE /dev/sdb1
> ARRAY /dev/md0 devices=/dev/sda1,/dev/sdb1
>
> then come on the list and complain when it stops working. 


What I suggest is that the feature keep working, and no one will 
complain. If there is a missing partition the error messages are clear. 
The feature is mainly used when there are partitions or drives which 
should not be examined, and "stops working" only when a hardware config 
has changed.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979