mdadm --detail works, mdadm --stop says "does not appear to be an md device"

mdadm --detail works, mdadm --stop says "does not appear to be an md device"

[Marc MERLIN]

gargamel:~# cat /proc/mdstat 
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath] 
md6 : active raid5 sdn1[3](F) sdt1[0](F) sds1[1](F) sdq1[2](F)
      23441547264 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/0] [_____]
gargamel:~# mdadm --stop /dev/md6
mdadm: /dev/md6 does not appear to be an md device
gargamel:~# mdadm --detail /dev/md6
/dev/md6:
           Version : 1.2
     Creation Time : Wed Jul  8 10:09:21 2020
        Raid Level : raid5
        Array Size : 23441547264 (22355.60 GiB 24004.14 GB)
     Used Dev Size : 5860386816 (5588.90 GiB 6001.04 GB)
      Raid Devices : 5
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Fri Mar 17 09:17:28 2023
             State : clean, FAILED 
    Active Devices : 0
    Failed Devices : 4
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : ppl

    Number   Major   Minor   RaidDevice State
       -       0        0        0      removed
       -       0        0        1      removed
       -       0        0        2      removed
       -       0        0        3      removed
       -       0        0        4      removed


How do I clear this without a reboot?

gargamel:~# uname -r
5.19.7


-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08

Re: mdadm --detail works, mdadm --stop says "does not appear to be an md device"

[Mariusz Tkaczyk]

On Fri, 17 Mar 2023 10:38:10 -0700
Marc MERLIN <marc@merlins.org> wrote:

> gargamel:~# cat /proc/mdstat 
> Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
> [multipath] md6 : active raid5 sdn1[3](F) sdt1[0](F) sds1[1](F) sdq1[2](F)
>       23441547264 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/0]
> [_____] gargamel:~# mdadm --stop /dev/md6
> mdadm: /dev/md6 does not appear to be an md device
> gargamel:~# mdadm --detail /dev/md6
> /dev/md6:
>            Version : 1.2
>      Creation Time : Wed Jul  8 10:09:21 2020
>         Raid Level : raid5
>         Array Size : 23441547264 (22355.60 GiB 24004.14 GB)
>      Used Dev Size : 5860386816 (5588.90 GiB 6001.04 GB)
>       Raid Devices : 5
>      Total Devices : 4
>        Persistence : Superblock is persistent
> 
>        Update Time : Fri Mar 17 09:17:28 2023
>              State : clean, FAILED 
>     Active Devices : 0
>     Failed Devices : 4
>      Spare Devices : 0
> 
>             Layout : left-symmetric
>         Chunk Size : 512K
> 
> Consistency Policy : ppl
> 
>     Number   Major   Minor   RaidDevice State
>        -       0        0        0      removed
>        -       0        0        1      removed
>        -       0        0        2      removed
>        -       0        0        3      removed
>        -       0        0        4      removed
> 
> 
> How do I clear this without a reboot?
> 
> gargamel:~# uname -r
> 5.19.7
> 
> 

Hi,
mdadm is unable to complete this task because it cannot ensure that it is safe
to stop the array. It cannot open the array with O_EXCL.
If it is mounted then it may hang if filesystem needs to flush some data.

Please, try umount the array if it mounted somewhere and then try:

# echo inactive > /sys/block/md6/md/array_state
# echo clear > /sys/block/md6/md/array_state

Thanks,
Mariusz

Re: mdadm --detail works, mdadm --stop says "does not appear to be an md device"

[Marc MERLIN]

On Mon, Mar 20, 2023 at 03:36:39PM +0100, Mariusz Tkaczyk wrote:
> Hi,
> mdadm is unable to complete this task because it cannot ensure that it is safe
> to stop the array. It cannot open the array with O_EXCL.
> If it is mounted then it may hang if filesystem needs to flush some data.
 
Thanks for the reply. The array was not mounted, that said, given that
it was fully down, there wasn't a way to flush the data if it had been
(cable problem to an enclosure, all the drives disappeared at once)

> Please, try umount the array if it mounted somewhere and then try:
> 
> # echo inactive > /sys/block/md6/md/array_state
> # echo clear > /sys/block/md6/md/array_state

I can try this next time (already had to reboot), thanks.

That said, mdadm should output a better message in this case
> > mdadm: /dev/md6 does not appear to be an md device
is clearly wrong 

Is that something easy to fix/improve?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/  

Re: mdadm --detail works, mdadm --stop says "does not appear to be an md device"

[Mariusz Tkaczyk]

On Mon, 20 Mar 2023 07:50:35 -0700
Marc MERLIN <marc@merlins.org> wrote:

> On Mon, Mar 20, 2023 at 03:36:39PM +0100, Mariusz Tkaczyk wrote:
> > Hi,
> > mdadm is unable to complete this task because it cannot ensure that it is
> > safe to stop the array. It cannot open the array with O_EXCL.
> > If it is mounted then it may hang if filesystem needs to flush some data.  
>  
> Thanks for the reply. The array was not mounted, that said, given that
> it was fully down, there wasn't a way to flush the data if it had been
> (cable problem to an enclosure, all the drives disappeared at once)
> 
> > Please, try umount the array if it mounted somewhere and then try:
> > 
> > # echo inactive > /sys/block/md6/md/array_state
> > # echo clear > /sys/block/md6/md/array_state  
> 
> I can try this next time (already had to reboot), thanks.
> 
> That said, mdadm should output a better message in this case
> > > mdadm: /dev/md6 does not appear to be an md device  
> is clearly wrong 
> 
> Is that something easy to fix/improve?

Oh, sorry my bad, please see the code:
https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/tree/mdopen.c#n472

We are failing to "understand" the array:
https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/tree/util.c#n229
It has nothing to do with open and O_EXCL. I didn't dig into to determine why.

Anyway, now error seems to be reasonable but maybe we should be able to tract
this array as valid? I requires more work and analysis so it is not simple fix.

Thanks,
Mariusz

Re: mdadm --detail works, mdadm --stop says "does not appear to be an md device"

[Marc MERLIN]

On Mon, Mar 20, 2023 at 04:16:59PM +0100, Mariusz Tkaczyk wrote:
> On Mon, 20 Mar 2023 07:50:35 -0700
> Marc MERLIN <marc@merlins.org> wrote:
> 
> > On Mon, Mar 20, 2023 at 03:36:39PM +0100, Mariusz Tkaczyk wrote:
> > > Hi,
> > > mdadm is unable to complete this task because it cannot ensure that it is
> > > safe to stop the array. It cannot open the array with O_EXCL.
> > > If it is mounted then it may hang if filesystem needs to flush some data.  
> >  
> > Thanks for the reply. The array was not mounted, that said, given that
> > it was fully down, there wasn't a way to flush the data if it had been
> > (cable problem to an enclosure, all the drives disappeared at once)
> > 
> > > Please, try umount the array if it mounted somewhere and then try:
> > > 
> > > # echo inactive > /sys/block/md6/md/array_state
> > > # echo clear > /sys/block/md6/md/array_state  
> > 
> > I can try this next time (already had to reboot), thanks.
> > 
> > That said, mdadm should output a better message in this case
> > > > mdadm: /dev/md6 does not appear to be an md device  
> > is clearly wrong 
> > 
> > Is that something easy to fix/improve?
> 
> Oh, sorry my bad, please see the code:
> https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/tree/mdopen.c#n472
> 
> We are failing to "understand" the array:
> https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/tree/util.c#n229
> It has nothing to do with open and O_EXCL. I didn't dig into to determine why.
> 
> Anyway, now error seems to be reasonable but maybe we should be able to tract
> this array as valid? I requires more work and analysis so it is not simple fix.

You are definitely more knowledgeable about this than I am.
All I can say is that the array was down, not mounted, and I couldn't
stop it without a reboot, and that's a problem.

Any way to force stop in a case like this, would be quite welcome :)

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08

Re: mdadm --detail works, mdadm --stop says "does not appear to be an md device"

[Mariusz Tkaczyk]

On Mon, 20 Mar 2023 19:01:01 -0700
Marc MERLIN <marc@merlins.org> wrote:

> On Mon, Mar 20, 2023 at 04:16:59PM +0100, Mariusz Tkaczyk wrote:
> > On Mon, 20 Mar 2023 07:50:35 -0700
> > Marc MERLIN <marc@merlins.org> wrote:
> >   
> > > On Mon, Mar 20, 2023 at 03:36:39PM +0100, Mariusz Tkaczyk wrote:  
> > > > Hi,
> > > > mdadm is unable to complete this task because it cannot ensure that it
> > > > is safe to stop the array. It cannot open the array with O_EXCL.
> > > > If it is mounted then it may hang if filesystem needs to flush some
> > > > data.    
> > >  
> > > Thanks for the reply. The array was not mounted, that said, given that
> > > it was fully down, there wasn't a way to flush the data if it had been
> > > (cable problem to an enclosure, all the drives disappeared at once)
> > >   
> > > > Please, try umount the array if it mounted somewhere and then try:
> > > > 
> > > > # echo inactive > /sys/block/md6/md/array_state
> > > > # echo clear > /sys/block/md6/md/array_state    
> > > 
> > > I can try this next time (already had to reboot), thanks.
> > > 
> > > That said, mdadm should output a better message in this case  
> > > > > mdadm: /dev/md6 does not appear to be an md device    
> > > is clearly wrong 
> > > 
> > > Is that something easy to fix/improve?  
> > 
> > Oh, sorry my bad, please see the code:
> > https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/tree/mdopen.c#n472
> > 
> > We are failing to "understand" the array:
> > https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/tree/util.c#n229
> > It has nothing to do with open and O_EXCL. I didn't dig into to determine
> > why.
> > 
> > Anyway, now error seems to be reasonable but maybe we should be able to
> > tract this array as valid? I requires more work and analysis so it is not
> > simple fix.  
> 
> You are definitely more knowledgeable about this than I am.
> All I can say is that the array was down, not mounted, and I couldn't
> stop it without a reboot, and that's a problem.
> 
> Any way to force stop in a case like this, would be quite welcome :)
> 
> Thanks,
> Marc

Jes, how you see this?

Thanks,
Mariusz

Re: mdadm --detail works, mdadm --stop says "does not appear to be an md device"

[Jes Sorensen]

On 3/22/23 03:11, Mariusz Tkaczyk wrote:
> On Mon, 20 Mar 2023 19:01:01 -0700
> Marc MERLIN <marc@merlins.org> wrote:
> 
>> On Mon, Mar 20, 2023 at 04:16:59PM +0100, Mariusz Tkaczyk wrote:
>>> On Mon, 20 Mar 2023 07:50:35 -0700
>>> Marc MERLIN <marc@merlins.org> wrote:
>>>   
>>>> On Mon, Mar 20, 2023 at 03:36:39PM +0100, Mariusz Tkaczyk wrote:  
>>>>> Hi,
>>>>> mdadm is unable to complete this task because it cannot ensure that it
>>>>> is safe to stop the array. It cannot open the array with O_EXCL.
>>>>> If it is mounted then it may hang if filesystem needs to flush some
>>>>> data.    
>>>>  
>>>> Thanks for the reply. The array was not mounted, that said, given that
>>>> it was fully down, there wasn't a way to flush the data if it had been
>>>> (cable problem to an enclosure, all the drives disappeared at once)
>>>>   
>>>>> Please, try umount the array if it mounted somewhere and then try:
>>>>>
>>>>> # echo inactive > /sys/block/md6/md/array_state
>>>>> # echo clear > /sys/block/md6/md/array_state    
>>>>
>>>> I can try this next time (already had to reboot), thanks.
>>>>
>>>> That said, mdadm should output a better message in this case  
>>>>>> mdadm: /dev/md6 does not appear to be an md device    
>>>> is clearly wrong 
>>>>
>>>> Is that something easy to fix/improve?  
>>>
>>> Oh, sorry my bad, please see the code:
>>> https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/tree/mdopen.c#n472
>>>
>>> We are failing to "understand" the array:
>>> https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/tree/util.c#n229
>>> It has nothing to do with open and O_EXCL. I didn't dig into to determine
>>> why.
>>>
>>> Anyway, now error seems to be reasonable but maybe we should be able to
>>> tract this array as valid? I requires more work and analysis so it is not
>>> simple fix.  
>>
>> You are definitely more knowledgeable about this than I am.
>> All I can say is that the array was down, not mounted, and I couldn't
>> stop it without a reboot, and that's a problem.
>>
>> Any way to force stop in a case like this, would be quite welcome :)
>>
>> Thanks,
>> Marc
> 
> Jes, how you see this?

If we can force stop it with a big fat warning, then I think that would
be a good feature to add. The must reboot requirement is not exactly ideal.

Cheers,
Jes

Re: mdadm --detail works, mdadm --stop says "does not appear to be an md device"

[Marc MERLIN]

On Wed, Mar 22, 2023 at 02:59:12PM -0400, Jes Sorensen wrote:
> >> You are definitely more knowledgeable about this than I am.
> >> All I can say is that the array was down, not mounted, and I couldn't
> >> stop it without a reboot, and that's a problem.
> >>
> >> Any way to force stop in a case like this, would be quite welcome :)
> > 
> > Jes, how you see this?
> 
> If we can force stop it with a big fat warning, then I think that would
> be a good feature to add. The must reboot requirement is not exactly ideal.

Hi Jes, how goes?

That would be lovely, even if I realize it's an unusual corner case.

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08