Re: MD Remnants After --stop

[NeilBrown <neilb@suse.com>]

[-- Attachment #1: Type: text/plain, Size: 4794 bytes --]

On Wed, Nov 23 2016, Marc Smith wrote:

> Hi,
>
> Sorry, I'm not trying to beat a dead horse here, but I do feel
> something has changed... I just tested with Linux 4.5.2 and when
> stopping an md array (with mdadm --stop) the entry in /sys/block/ is
> removed, and even the /dev/mdXXX and /dev/md/name link are removed
> properly.
>
> When testing with Linux 4.9-rc3, the entries in /sys/block/ still
> remain (array_state attribute value is "clear") after using mdadm
> --stop and the /dev/mdXXX device exists (the /dev/md/name link is
> removed, by udev I assume).

With the latest (git) mdadm, when events are reported by "udevadm monitor"??

I only see remove events, and the entries from /dev and /sys are
removed.

If I could reproduce your problem, I would fix it...

NeilBrown

>
> Looks like Linux 4.9 is at rc6 now -- have there been any changes that
> would correct this behavior? Or is this expected behavior with the
> latest version? Not sure when this changed, but I did go back to 4.5.2
> and confirmed everything is removed correctly in that version, not
> sure if this is different starting in 4.9, or something between 4.5
> and 4.9.
>
> Can anyone else confirm with Linux 4.9 that the /sys/block/mdXXX entry
> lingers after using mdadm --stop? I suppose it could be some other
> system that is causing this on my machines. I tested using the latest
> from the master branch of mdadm and get the same result.
>
>
> Thanks for your time and help.
>
> --Marc
>
>
> On Mon, Nov 21, 2016 at 9:08 AM, Marc Smith <marc.smith@mcc.edu> wrote:
>> On Sun, Nov 20, 2016 at 10:42 PM, NeilBrown <neilb@suse.com> wrote:
>>> On Sat, Nov 19 2016, Marc Smith wrote:
>>>
>>>> On Mon, Nov 7, 2016 at 12:44 AM, NeilBrown <neilb@suse.com> wrote:
>>>>> On Sat, Nov 05 2016, Marc Smith wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> It may be that I've never noticed this before, so maybe its not a
>>>>>> problem... after using '--stop' to deactivate/stop an MD array, there
>>>>>> are remnants of it lingering, namely an entry in /sys/block (eg,
>>>>>> /sys/block/md127) and the device node in /dev remains (eg,
>>>>>> /dev/md127).
>>>>>>
>>>>>> Is this normal? Like I said, it probably is, and I've just never
>>>>>> noticed it before. I assume its not going to hurt anything, but is
>>>>>> there a way to clean it up, without rebooting? Obviously I could
>>>>>> remove the /dev entry, but what about /sys/block?
>>>>>>
>>>>>
>>>>> You can remove them both by running
>>>>>   mdadm -S /dev/md127
>>>>>
>>>>> but they'll probably just reappear again.
>>>>>
>>>>> This seems to be an on-going battle between md and udev.  I've "fixed"
>>>>> it at least once, but it keeps coming back.
>>>>>
>>>>> When md removes the md127 device, a message is sent to udev.
>>>>> As part of its response to this message, udev tries to open /dev/md127.
>>>>> Because of the rather unusual way that md devices are created (it made
>>>>> sense nearly 20 years ago when it was designed), opening /dev/md127
>>>>> causes md to create device md127 again.
>>>>>
>>>>> You could
>>>>>   mv /dev/md127 /dev/md127X
>>>>>   mdadm -S /dev/md127X
>>>>>   rm /dev/md127X
>>>>> that stop udev from opening /dev/md127.  It seems to work reliably.
>>>>>
>>>>> md used to generate a CHANGE event before the REMOVE event, and only the
>>>>> CHANGE event caused udev to open the device file.  I removed that and
>>>>> the problem went away.  Apparently some change has happened to udev and
>>>>> now it opens the file in response to REMOVE as well.
>>>>
>>>> I used "udevadm monitor -pku" to watch the events when running "mdadm
>>>> --stop /dev/md127" and this is what I see:
>>>>
>>>> --snip--
>>>> KERNEL[163074.119778] change   /devices/virtual/block/md127 (block)
>>>> ACTION=change
>>>> DEVNAME=/dev/md127
>>>> DEVPATH=/devices/virtual/block/md127
>>>> DEVTYPE=disk
>>>> MAJOR=9
>>>> MINOR=127
>>>> SEQNUM=3701
>>>> SUBSYSTEM=block
>>>>
>>>> UDEV  [163074.121569] change   /devices/virtual/block/md127 (block)
>>>> ACTION=change
>>>> DEVNAME=/dev/md127
>>>> DEVPATH=/devices/virtual/block/md127
>>>> DEVTYPE=disk
>>>> MAJOR=9
>>>> MINOR=127
>>>> SEQNUM=3701
>>>> SUBSYSTEM=block
>>>> SYSTEMD_READY=0
>>>> USEC_INITIALIZED=370470
>>>> --snip--
>>>>
>>>> I don't see any 'remove' event generated. I should mention if I hadn't
>>>> already that I'm testing md-cluster (--bitmap=clustered), and
>>>> currently using Linux 4.9-rc3.
>>>
>>> What version of mdadm are you using?
>>
>> v3.4
>>
>>
>>> You need one which contains
>>> Commit: 229e66cb9689 ("Manage.c: Only issue change events for kernels older than 2.6.28")
>>>
>>> which hasn't made it into a release yet.  But if you are playing with
>>> md-cluster, I would guess you are using the latest from git...
>>
>> Wasn't, but I will now. Thanks.
>>
>> --Marc
>>
>>
>>>
>>> NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]