mdadm --fail requires writeable drive.

mdadm --fail requires writeable drive.

[Benjamin ESTRABAUD]

Hi,

I noticed that mdadm --fail will only work if a disk is present and is 
writeable.

mdadm's Manage.c seems to first make sure the drive can be "stat"ed 
before proceeding (which will fail if the drive is gone), and then 
seemingly try to write "Faulty" to the drive's superblock as well as 
notifying MD of the drive failure.

However, in many cases, a drive must be set faulty because it has gone 
offline. MD will do a very good job to set the drive faulty itself, 
provided that IOs are running on the array. If no IOs are running on the 
array, removing a drive and then trying to set it as failed will not 
work. Trying to "-r" the drive will also fail since the drive is not yet 
marked as failed, so deemed still in use.

Looking through the code it seemed to me that MD could be told to fail a 
drive even if that drive was no longer present (the /dev/sdX device node 
file still is in this case), in the same way as remove works. Would that 
be possible? Am I missing something here?

Thanks in advance for your help.

Regards,

Ben - MPSTOR

Re: mdadm --fail requires writeable drive.

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1218 bytes --]

On Wed, 13 Nov 2013 12:17:21 +0000 Benjamin ESTRABAUD <be@mpstor.com> wrote:

> Hi,
> 
> I noticed that mdadm --fail will only work if a disk is present and is 
> writeable.
> 
> mdadm's Manage.c seems to first make sure the drive can be "stat"ed 
> before proceeding (which will fail if the drive is gone), and then 
> seemingly try to write "Faulty" to the drive's superblock as well as 
> notifying MD of the drive failure.
> 
> However, in many cases, a drive must be set faulty because it has gone 
> offline. MD will do a very good job to set the drive faulty itself, 
> provided that IOs are running on the array. If no IOs are running on the 
> array, removing a drive and then trying to set it as failed will not 
> work. Trying to "-r" the drive will also fail since the drive is not yet 
> marked as failed, so deemed still in use.
> 
> Looking through the code it seemed to me that MD could be told to fail a 
> drive even if that drive was no longer present (the /dev/sdX device node 
> file still is in this case), in the same way as remove works. Would that 
> be possible? Am I missing something here?

Does
  mdadm /dev/mdXX --fail detached

do what you want?

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: mdadm --fail requires writeable drive.

[Benjamin ESTRABAUD]

Hi Neil,

This does work but I was looking for a way to explicitely set a 
particular device as Failed (I want to process devices one by one).

But right now I was reading from the array to force it detecting the 
failure as opposed to using "detached" which orders of magnitude better.

Thanks for the tip, I'll use that for now.

Regards,
Ben.

On 14/11/13 00:44, NeilBrown wrote:
> On Wed, 13 Nov 2013 12:17:21 +0000 Benjamin ESTRABAUD <be@mpstor.com> wrote:
>
>> Hi,
>>
>> I noticed that mdadm --fail will only work if a disk is present and is
>> writeable.
>>
>> mdadm's Manage.c seems to first make sure the drive can be "stat"ed
>> before proceeding (which will fail if the drive is gone), and then
>> seemingly try to write "Faulty" to the drive's superblock as well as
>> notifying MD of the drive failure.
>>
>> However, in many cases, a drive must be set faulty because it has gone
>> offline. MD will do a very good job to set the drive faulty itself,
>> provided that IOs are running on the array. If no IOs are running on the
>> array, removing a drive and then trying to set it as failed will not
>> work. Trying to "-r" the drive will also fail since the drive is not yet
>> marked as failed, so deemed still in use.
>>
>> Looking through the code it seemed to me that MD could be told to fail a
>> drive even if that drive was no longer present (the /dev/sdX device node
>> file still is in this case), in the same way as remove works. Would that
>> be possible? Am I missing something here?
>
> Does
>    mdadm /dev/mdXX --fail detached
>
> do what you want?
>
> NeilBrown
>

Re: mdadm --fail requires writeable drive.

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

On Thu, 14 Nov 2013 17:33:01 +0000 Benjamin ESTRABAUD <be@mpstor.com> wrote:

> Hi Neil,
> 
> This does work but I was looking for a way to explicitely set a 
> particular device as Failed (I want to process devices one by one).
> 
> But right now I was reading from the array to force it detecting the 
> failure as opposed to using "detached" which orders of magnitude better.
> 
> Thanks for the tip, I'll use that for now.

If you really want to just fail one of them you can used the "kernel name"
like "sda1".
If you
   ls -d /sys/block/mdXX/md/dev-*

you will see the devices that are though to be part of the array.  The part
of the name after "dev-" is the "kernel name".
"mdadm /dev/mdXX --fail" will accept a "kernel name" and will mark just that
device as faulty.

NeilBrown


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: mdadm --fail requires writeable drive.

[Benjamin ESTRABAUD]

On 15/11/13 04:46, NeilBrown wrote:
> On Thu, 14 Nov 2013 17:33:01 +0000 Benjamin ESTRABAUD <be@mpstor.com> wrote:
>
>> Hi Neil,
>>
>> This does work but I was looking for a way to explicitely set a
>> particular device as Failed (I want to process devices one by one).
>>
>> But right now I was reading from the array to force it detecting the
>> failure as opposed to using "detached" which orders of magnitude better.
>>
>> Thanks for the tip, I'll use that for now.
>

Hi Neil,

> If you really want to just fail one of them you can used the "kernel name"
> like "sda1".
> If you
>     ls -d /sys/block/mdXX/md/dev-*
>

Thanks a lot for this! This is exactly what I was looking for and it 
worked like a charm.

> you will see the devices that are though to be part of the array.  The part
> of the name after "dev-" is the "kernel name".
> "mdadm /dev/mdXX --fail" will accept a "kernel name" and will mark just that
> device as faulty.
>

That's perfect. Thanks a lot!

> NeilBrown
>

Ben.