* question about mdmon --takeover @ 2013-07-31 14:30 Francis Moreau 2013-08-05 6:59 ` NeilBrown 0 siblings, 1 reply; 9+ messages in thread From: Francis Moreau @ 2013-07-31 14:30 UTC (permalink / raw) To: linux-raid Hello list, I thought that using "--takeover" would hint mdmon to replace existing mdmon process, and therefore the old one would exit somehow. However after several "mdmon --takeover" I can see this: $ ps aux | grep dmon root 233 0.0 0.2 80388 10752 ? SLsl 14:02 0:00 @dmon --offroot md127 root 3326 0.0 0.2 14920 10820 ? SLsl 15:16 0:00 mdmon --takeover md127 root 3343 0.0 0.2 14920 10820 ? SLsl 15:17 0:00 mdmon --takeover md127 Is this expected ? Thanks. -- Francis ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: question about mdmon --takeover 2013-07-31 14:30 question about mdmon --takeover Francis Moreau @ 2013-08-05 6:59 ` NeilBrown 2013-08-28 17:14 ` Francis Moreau 0 siblings, 1 reply; 9+ messages in thread From: NeilBrown @ 2013-08-05 6:59 UTC (permalink / raw) To: Francis Moreau; +Cc: linux-raid [-- Attachment #1: Type: text/plain, Size: 1204 bytes --] On Wed, 31 Jul 2013 16:30:34 +0200 Francis Moreau <francis.moro@gmail.com> wrote: > Hello list, > > I thought that using "--takeover" would hint mdmon to replace existing > mdmon process, and therefore the old one would exit somehow. > > However after several "mdmon --takeover" I can see this: > $ ps aux | grep dmon > root 233 0.0 0.2 80388 10752 ? SLsl 14:02 0:00 @dmon > --offroot md127 > root 3326 0.0 0.2 14920 10820 ? SLsl 15:16 0:00 mdmon > --takeover md127 > root 3343 0.0 0.2 14920 10820 ? SLsl 15:17 0:00 mdmon > --takeover md127 > > Is this expected ? > > Thanks. Nope. That's not expected. mdmon should send SIGTERM to the old mdmon and then wait for it to exit. If the new and old mdmon were compiled different and look for the pid file in different directories that might explain what you see. If you compile mdadm from source it will use /run/mdadm. However if your distro doesn't have /run the the distro-provided mdadm will be compiled differently. It is safe to kill the old mdmons once a new one is running, but you should probably find out what the incompatibility is... NeilBrown [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 828 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: question about mdmon --takeover 2013-08-05 6:59 ` NeilBrown @ 2013-08-28 17:14 ` Francis Moreau 2013-08-28 19:42 ` Francis Moreau 2013-08-31 20:21 ` Francis Moreau 0 siblings, 2 replies; 9+ messages in thread From: Francis Moreau @ 2013-08-28 17:14 UTC (permalink / raw) To: NeilBrown; +Cc: linux-raid Hello Neil, Sorry for the late reply. On Mon, Aug 5, 2013 at 8:59 AM, NeilBrown <neilb@suse.de> wrote: > On Wed, 31 Jul 2013 16:30:34 +0200 Francis Moreau <francis.moro@gmail.com> > wrote: > >> Hello list, >> >> I thought that using "--takeover" would hint mdmon to replace existing >> mdmon process, and therefore the old one would exit somehow. >> >> However after several "mdmon --takeover" I can see this: >> $ ps aux | grep dmon >> root 233 0.0 0.2 80388 10752 ? SLsl 14:02 0:00 @dmon >> --offroot md127 >> root 3326 0.0 0.2 14920 10820 ? SLsl 15:16 0:00 mdmon >> --takeover md127 >> root 3343 0.0 0.2 14920 10820 ? SLsl 15:17 0:00 mdmon >> --takeover md127 >> >> Is this expected ? >> >> Thanks. > > Nope. That's not expected. > > mdmon should send SIGTERM to the old mdmon and then wait for it to exit. > > If the new and old mdmon were compiled different and look for the pid file in > different directories that might explain what you see. > > If you compile mdadm from source it will use /run/mdadm. However if your > distro doesn't have /run the the distro-provided mdadm will be compiled > differently. > It doesn't to be the case. Actually sending SIGTERM manually to mdmon has no effects. # mdadm --version mdadm - v3.2.6 - 25th October 2012 # ps aux | grep dmon root 235 0.1 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon --offroot md127 root 339 0.0 1.0 15044 10944 ? SLsl 19:08 0:00 /sbin/mdmon --takeover md127 # cat /run/mdadm/md127.pid 339 # kill -SIGTERM 339 # ps aux | grep dmon root 235 0.0 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon --offroot md127 root 339 0.0 1.0 15044 10944 ? SLsl 19:08 0:00 /sbin/mdmon --takeover md127 # ps aux | grep dmon root 235 0.0 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon --offroot md127 root 339 0.0 1.0 15044 10944 ? SLsl 19:08 0:00 /sbin/mdmon --takeover md127 root 2352 0.1 1.0 15076 10976 ? SLsl 19:12 0:00 mdmon --takeover /dev/md127 # cat /run/mdadm/md127.pid 2352 # pkill -SIGTERM mdmon [root@localhost ~]# ps aux | grep dmon root 235 0.0 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon --offroot md127 root 339 0.0 1.0 80580 10944 ? SLsl 19:08 0:00 /sbin/mdmon --takeover md127 root 2352 0.0 1.0 80612 10976 ? SLsl 19:12 0:00 mdmon --takeover /dev/md127 Can't you reproduce ? Thanks -- Francis ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: question about mdmon --takeover 2013-08-28 17:14 ` Francis Moreau @ 2013-08-28 19:42 ` Francis Moreau 2013-08-31 20:21 ` Francis Moreau 1 sibling, 0 replies; 9+ messages in thread From: Francis Moreau @ 2013-08-28 19:42 UTC (permalink / raw) To: NeilBrown; +Cc: linux-raid On Wed, Aug 28, 2013 at 7:14 PM, Francis Moreau <francis.moro@gmail.com> wrote: > Hello Neil, > > > Sorry for the late reply. > > On Mon, Aug 5, 2013 at 8:59 AM, NeilBrown <neilb@suse.de> wrote: >> On Wed, 31 Jul 2013 16:30:34 +0200 Francis Moreau <francis.moro@gmail.com> >> wrote: >> >>> Hello list, >>> >>> I thought that using "--takeover" would hint mdmon to replace existing >>> mdmon process, and therefore the old one would exit somehow. >>> >>> However after several "mdmon --takeover" I can see this: >>> $ ps aux | grep dmon >>> root 233 0.0 0.2 80388 10752 ? SLsl 14:02 0:00 @dmon >>> --offroot md127 >>> root 3326 0.0 0.2 14920 10820 ? SLsl 15:16 0:00 mdmon >>> --takeover md127 >>> root 3343 0.0 0.2 14920 10820 ? SLsl 15:17 0:00 mdmon >>> --takeover md127 >>> >>> Is this expected ? >>> >>> Thanks. >> >> Nope. That's not expected. >> >> mdmon should send SIGTERM to the old mdmon and then wait for it to exit. >> >> If the new and old mdmon were compiled different and look for the pid file in >> different directories that might explain what you see. >> >> If you compile mdadm from source it will use /run/mdadm. However if your >> distro doesn't have /run the the distro-provided mdadm will be compiled >> differently. >> > > It doesn't to be the case. > > Actually sending SIGTERM manually to mdmon has no effects. > > > # mdadm --version > mdadm - v3.2.6 - 25th October 2012 > > # ps aux | grep dmon > root 235 0.1 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon > --offroot md127 > root 339 0.0 1.0 15044 10944 ? SLsl 19:08 0:00 > /sbin/mdmon --takeover md127 > > # cat /run/mdadm/md127.pid > 339 > > # kill -SIGTERM 339 > # ps aux | grep dmon > root 235 0.0 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon > --offroot md127 > root 339 0.0 1.0 15044 10944 ? SLsl 19:08 0:00 > /sbin/mdmon --takeover md127 > > # ps aux | grep dmon > root 235 0.0 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon > --offroot md127 > root 339 0.0 1.0 15044 10944 ? SLsl 19:08 0:00 > /sbin/mdmon --takeover md127 > root 2352 0.1 1.0 15076 10976 ? SLsl 19:12 0:00 mdmon > --takeover /dev/md127 > > # cat /run/mdadm/md127.pid > 2352 > > # pkill -SIGTERM mdmon > [root@localhost ~]# ps aux | grep dmon > root 235 0.0 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon > --offroot md127 > root 339 0.0 1.0 80580 10944 ? SLsl 19:08 0:00 > /sbin/mdmon --takeover md127 > root 2352 0.0 1.0 80612 10976 ? SLsl 19:12 0:00 mdmon > --takeover /dev/md127 > > Can't you reproduce ? > and running latest git head is even worse, mdmon can't be started at all: # mdadm --version mdadm - v3.3-rc2-58-g2cdd5ce - 28th August 2013 # ls /run/mdadm/ autorebuild.pid map # mdmon --takeover /dev/md127 # ps aux | grep dmon root 2347 0.0 0.0 9680 848 ttyS0 S+ 21:40 0:00 grep --color dmon # ls /run/mdadm/ autorebuild.pid map -- Francis ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: question about mdmon --takeover 2013-08-28 17:14 ` Francis Moreau 2013-08-28 19:42 ` Francis Moreau @ 2013-08-31 20:21 ` Francis Moreau 2013-09-02 2:04 ` NeilBrown 1 sibling, 1 reply; 9+ messages in thread From: Francis Moreau @ 2013-08-31 20:21 UTC (permalink / raw) To: Martin Wilck; +Cc: linux-raid, NeilBrown Hello Martin, I'm adding you in the discussion because you might have an idea on what's going on with my array which is using DDF metadata. On Wed, Aug 28, 2013 at 7:14 PM, Francis Moreau <francis.moro@gmail.com> wrote: > Hello Neil, > > > Sorry for the late reply. > > On Mon, Aug 5, 2013 at 8:59 AM, NeilBrown <neilb@suse.de> wrote: >> On Wed, 31 Jul 2013 16:30:34 +0200 Francis Moreau <francis.moro@gmail.com> >> wrote: >> >>> Hello list, >>> >>> I thought that using "--takeover" would hint mdmon to replace existing >>> mdmon process, and therefore the old one would exit somehow. >>> >>> However after several "mdmon --takeover" I can see this: >>> $ ps aux | grep dmon >>> root 233 0.0 0.2 80388 10752 ? SLsl 14:02 0:00 @dmon >>> --offroot md127 >>> root 3326 0.0 0.2 14920 10820 ? SLsl 15:16 0:00 mdmon >>> --takeover md127 >>> root 3343 0.0 0.2 14920 10820 ? SLsl 15:17 0:00 mdmon >>> --takeover md127 >>> >>> Is this expected ? >>> >>> Thanks. >> >> Nope. That's not expected. >> >> mdmon should send SIGTERM to the old mdmon and then wait for it to exit. >> >> If the new and old mdmon were compiled different and look for the pid file in >> different directories that might explain what you see. >> >> If you compile mdadm from source it will use /run/mdadm. However if your >> distro doesn't have /run the the distro-provided mdadm will be compiled >> differently. >> > > It doesn't to be the case. > > Actually sending SIGTERM manually to mdmon has no effects. > > > # mdadm --version > mdadm - v3.2.6 - 25th October 2012 > > # ps aux | grep dmon > root 235 0.1 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon > --offroot md127 > root 339 0.0 1.0 15044 10944 ? SLsl 19:08 0:00 > /sbin/mdmon --takeover md127 > > # cat /run/mdadm/md127.pid > 339 > > # kill -SIGTERM 339 > # ps aux | grep dmon > root 235 0.0 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon > --offroot md127 > root 339 0.0 1.0 15044 10944 ? SLsl 19:08 0:00 > /sbin/mdmon --takeover md127 > > # ps aux | grep dmon > root 235 0.0 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon > --offroot md127 > root 339 0.0 1.0 15044 10944 ? SLsl 19:08 0:00 > /sbin/mdmon --takeover md127 > root 2352 0.1 1.0 15076 10976 ? SLsl 19:12 0:00 mdmon > --takeover /dev/md127 > > # cat /run/mdadm/md127.pid > 2352 > > # pkill -SIGTERM mdmon > [root@localhost ~]# ps aux | grep dmon > root 235 0.0 1.0 80612 10976 ? SLsl 19:08 0:00 @dmon > --offroot md127 > root 339 0.0 1.0 80580 10944 ? SLsl 19:08 0:00 > /sbin/mdmon --takeover md127 > root 2352 0.0 1.0 80612 10976 ? SLsl 19:12 0:00 mdmon > --takeover /dev/md127 > > Can't you reproduce ? > Here are some additionnal information: # cat /proc/mdstat Personalities : [raid1] md126 : active raid1 sda[1] sdb[0] 2064384 blocks super external:/md127/0 [2/2] [UU] md127 : inactive sda[1](S) sdb[0](S) 65536 blocks super external:ddf unused devices: <none> # mdadm -E /dev/md126 /dev/md126: MBR Magic : aa55 Partition[0] : 367447 sectors at 2048 (type 82) Partition[1] : 3755997 sectors at 372708 (type 05) [root@localhost ~]# mdadm -E /dev/md127 /dev/md127: Magic : de11de11 Version : 01.02.00 Controller GUID : 4C696E75:782D4D44:20202020:2020206C:6F63616C:686F7374 (Linux-MD localhost) Container GUID : 4C696E75:782D4D44:DEADBEEF:00000000:3F4FB732:8435623D (Linux-MD 08/28/13 22:27:30) Seq : 00000001 Redundant hdr : no Virtual Disks : 1 VD GUID[0] : 4C696E75:782D4D44:DEADBEEF:00000000:3F4FB739:E0C8B16E (Linux-MD 08/28/13 22:27:37) unit[0] : 126 state[0] : Optimal, Not Consistent init state[0] : Fully Initialised access[0] : Read/Write Name[0] : array1 Raid Devices[0] : 2 (0 1) Raid Level[0] : RAID1 Device Size[0] : 2064384 Array Size[0] : 2064384 Physical Disks : 2 Number RefNo Size Device Type/State 0 2cf00056 2064384K /dev/sda active/Online 1 b342fbdc 2064384K /dev/sdb active/Online I used GDB to trace what's going on in the monitor (thread) when sending a SIGTERM to the mdmon process: Breakpoint 1, wait_and_act (container=0x141a2c0, nowait=0) at monitor.c:634 634 for (a = *aap; a ; a = a->next) { (gdb) n 632 rv = 0; (gdb) 633 dirty_arrays = 0; (gdb) 634 for (a = *aap; a ; a = a->next) { (gdb) 649 int ret = read_and_act(a); (gdb) p *a $2 = { info = { array = { major_version = 0, minor_version = 0, patch_version = 0, ctime = 0, level = 1, size = 0, nr_disks = 0, raid_disks = 2, md_minor = 0, not_persistent = 0, utime = 0, state = 0, active_disks = 0, working_disks = 0, failed_disks = 0, spare_disks = 0, layout = 0, chunk_size = 0 }, disk = { number = 0, major = 0, minor = 0, raid_disk = 0, state = 0 }, events = 0, uuid = {0, 0, 0, 0}, name = '\000' <repeats 32 times>, data_offset = 0, component_size = 4128768, custom_array_size = 0, reshape_active = 0, reshape_progress = 0, recovery_blocked = 0, { resync_start = 18446744073709551615, recovery_start = 18446744073709551615 }, bitmap_offset = 0, safe_mode_delay = 0, new_level = 0, delta_disks = 0, new_layout = 0, new_chunk = 0, errors = 0, cache_size = 0, mismatch_cnt = 0, text_version = '\000' <repeats 49 times>, container_member = 0, container_enough = 0, sys_name = "md126", '\000' <repeats 14 times>, devs = 0x14213c0, next = 0x0, recovery_fd = 0, state_fd = 12, prev_state = 0, curr_state = 0, next_state = 0 }, container = 0x141a2c0, next = 0x0, replaces = 0x0, to_remove = 0, action_fd = 11, resync_start_fd = 13, metadata_fd = 14, sync_completed_fd = 15, last_checkpoint = 3872640, prev_state = active, curr_state = active, next_state = bad_word, prev_action = idle, curr_action = idle, next_action = bad_action, check_degraded = 0, check_reshape = 0, devnum = 126 } (gdb) n 636 if (a->replaces && !discard_this) { (gdb) 648 if (a->container && !a->to_remove) { (gdb) 649 int ret = read_and_act(a); (gdb) 656 if (sigterm && !(ret & ARRAY_DIRTY)) (gdb) p ret $3 = 1 It seems that the array is dirty, and that's the reason why the monitor never terminates. I got the same behaviour if I'm starting a new mdmon process on this array using --takeover switch. It seems weird since in my understanding --takeover indicate mdmon to replace the current mdmon process even if the array is dirty, no ? Do you have any idea on this behaviour ? -- Francis ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: question about mdmon --takeover 2013-08-31 20:21 ` Francis Moreau @ 2013-09-02 2:04 ` NeilBrown 2013-09-02 7:56 ` Francis Moreau 0 siblings, 1 reply; 9+ messages in thread From: NeilBrown @ 2013-09-02 2:04 UTC (permalink / raw) To: Francis Moreau; +Cc: Martin Wilck, linux-raid [-- Attachment #1: Type: text/plain, Size: 468 bytes --] On Sat, 31 Aug 2013 22:21:36 +0200 Francis Moreau <francis.moro@gmail.com> wrote: > Hello Martin, > > I'm adding you in the discussion because you might have an idea on > what's going on with my array which is using DDF metadata. Hi Francis, I think this problem will be fixed by Martin's latest patch set. The root cause seems to be that safe_mode_delay isn't being set properly which is the same root cause as the other problem you had. NeilBrown [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 828 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: question about mdmon --takeover 2013-09-02 2:04 ` NeilBrown @ 2013-09-02 7:56 ` Francis Moreau 2013-09-02 8:10 ` NeilBrown 0 siblings, 1 reply; 9+ messages in thread From: Francis Moreau @ 2013-09-02 7:56 UTC (permalink / raw) To: NeilBrown; +Cc: Martin Wilck, linux-raid Hi Neil, On Mon, Sep 2, 2013 at 4:04 AM, NeilBrown <neilb@suse.de> wrote: > On Sat, 31 Aug 2013 22:21:36 +0200 Francis Moreau <francis.moro@gmail.com> > wrote: > >> Hello Martin, >> >> I'm adding you in the discussion because you might have an idea on >> what's going on with my array which is using DDF metadata. > > Hi Francis, > I think this problem will be fixed by Martin's latest patch set. > The root cause seems to be that safe_mode_delay isn't being set properly > which is the same root cause as the other problem you had. > That's correct, this patch fixes this issue as well. May I ask you when the 3.3 release is planned ? Thanks -- Francis ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: question about mdmon --takeover 2013-09-02 7:56 ` Francis Moreau @ 2013-09-02 8:10 ` NeilBrown 2013-09-02 8:27 ` Francis Moreau 0 siblings, 1 reply; 9+ messages in thread From: NeilBrown @ 2013-09-02 8:10 UTC (permalink / raw) To: Francis Moreau; +Cc: Martin Wilck, linux-raid [-- Attachment #1: Type: text/plain, Size: 954 bytes --] On Mon, 2 Sep 2013 09:56:26 +0200 Francis Moreau <francis.moro@gmail.com> wrote: > Hi Neil, > > On Mon, Sep 2, 2013 at 4:04 AM, NeilBrown <neilb@suse.de> wrote: > > On Sat, 31 Aug 2013 22:21:36 +0200 Francis Moreau <francis.moro@gmail.com> > > wrote: > > > >> Hello Martin, > >> > >> I'm adding you in the discussion because you might have an idea on > >> what's going on with my array which is using DDF metadata. > > > > Hi Francis, > > I think this problem will be fixed by Martin's latest patch set. > > The root cause seems to be that safe_mode_delay isn't being set properly > > which is the same root cause as the other problem you had. > > > > That's correct, this patch fixes this issue as well. Thanks. > > May I ask you when the 3.3 release is planned ? I was waiting for a confirmation from you and one other from Martin. So you have just brought it closer. Tuesday or Wednesday I expect. NeilBrown [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 828 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: question about mdmon --takeover 2013-09-02 8:10 ` NeilBrown @ 2013-09-02 8:27 ` Francis Moreau 0 siblings, 0 replies; 9+ messages in thread From: Francis Moreau @ 2013-09-02 8:27 UTC (permalink / raw) To: NeilBrown; +Cc: Martin Wilck, linux-raid On Mon, Sep 2, 2013 at 10:10 AM, NeilBrown <neilb@suse.de> wrote: > On Mon, 2 Sep 2013 09:56:26 +0200 Francis Moreau <francis.moro@gmail.com> > wrote: > >> Hi Neil, >> >> On Mon, Sep 2, 2013 at 4:04 AM, NeilBrown <neilb@suse.de> wrote: >> > On Sat, 31 Aug 2013 22:21:36 +0200 Francis Moreau <francis.moro@gmail.com> >> > wrote: >> > >> >> Hello Martin, >> >> >> >> I'm adding you in the discussion because you might have an idea on >> >> what's going on with my array which is using DDF metadata. >> > >> > Hi Francis, >> > I think this problem will be fixed by Martin's latest patch set. >> > The root cause seems to be that safe_mode_delay isn't being set properly >> > which is the same root cause as the other problem you had. >> > >> >> That's correct, this patch fixes this issue as well. > > Thanks. > >> >> May I ask you when the 3.3 release is planned ? > > I was waiting for a confirmation from you and one other from Martin. > So you have just brought it closer. > > Tuesday or Wednesday I expect. Great news. -- Francis ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2013-09-02 8:27 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-07-31 14:30 question about mdmon --takeover Francis Moreau 2013-08-05 6:59 ` NeilBrown 2013-08-28 17:14 ` Francis Moreau 2013-08-28 19:42 ` Francis Moreau 2013-08-31 20:21 ` Francis Moreau 2013-09-02 2:04 ` NeilBrown 2013-09-02 7:56 ` Francis Moreau 2013-09-02 8:10 ` NeilBrown 2013-09-02 8:27 ` Francis Moreau
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).