* md-cluster Module Requirement
@ 2016-08-26 14:40 Marc Smith
2016-09-01 2:18 ` Guoqing Jiang
0 siblings, 1 reply; 6+ messages in thread
From: Marc Smith @ 2016-08-26 14:40 UTC (permalink / raw)
To: linux-raid
Hi,
I'm attempting to use md-cluster from Linux 4.5.2 with mdadm 3.4 and
I'm running into this when attempting to create a RAID1 device with
the clustered bitmap:
--snip--
[64782.619968] md: bind<dm-4>
[64782.629336] md: bind<dm-3>
[64782.630008] md/raid1:md127: active with 2 out of 2 mirrors
[64782.630528] md-cluster module not found.
[64782.630530] md127: Could not setup cluster service (-2)
[64782.630531] md127: bitmap file superblock:
[64782.630532] magic: 6d746962
[64782.630533] version: 5
[64782.630534] uuid: 10fee18f.f553d7f2.deb926f1.c7c4bd4b
[64782.630534] events: 0
[64782.630535] events cleared: 0
[64782.630536] state: 00000000
[64782.630537] chunksize: 67108864 B
[64782.630537] daemon sleep: 5s
[64782.630538] sync size: 878956288 KB
[64782.630539] max write behind: 0
[64782.630541] md127: failed to create bitmap (-2)
[64782.630577] md: md127 stopped.
[64782.630581] md: unbind<dm-3>
[64782.635133] md: export_rdev(dm-3)
[64782.635145] md: unbind<dm-4>
[64782.643111] md: export_rdev(dm-4)
--snip--
I'm using md-cluster built-in, not as a module:
# zcat /proc/config.gz | grep MD_CLUSTER
CONFIG_MD_CLUSTER=y
It seems the driver is attempting to load the 'md-cluster' module
(from drivers/md/md.c):
--snip--
err = request_module("md-cluster");
if (err) {
pr_err("md-cluster module not found.\n");
return -ENOENT;
}
--snip--
I looked at linux-next and it appears this code is the same; is there
a test we can do before attempting to load the module in the case that
its built-in, or is there some other requirement that md-cluster needs
to be built as a module?
Thanks for your time.
--Marc
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: md-cluster Module Requirement 2016-08-26 14:40 md-cluster Module Requirement Marc Smith @ 2016-09-01 2:18 ` Guoqing Jiang 2016-09-01 5:52 ` NeilBrown 0 siblings, 1 reply; 6+ messages in thread From: Guoqing Jiang @ 2016-09-01 2:18 UTC (permalink / raw) To: Marc Smith, linux-raid On 08/26/2016 10:40 AM, Marc Smith wrote: > Hi, > > I'm attempting to use md-cluster from Linux 4.5.2 with mdadm 3.4 and > I'm running into this when attempting to create a RAID1 device with > the clustered bitmap: > > --snip-- > [64782.619968] md: bind<dm-4> > [64782.629336] md: bind<dm-3> > [64782.630008] md/raid1:md127: active with 2 out of 2 mirrors > [64782.630528] md-cluster module not found. > [64782.630530] md127: Could not setup cluster service (-2) > [64782.630531] md127: bitmap file superblock: > [64782.630532] magic: 6d746962 > [64782.630533] version: 5 > [64782.630534] uuid: 10fee18f.f553d7f2.deb926f1.c7c4bd4b > [64782.630534] events: 0 > [64782.630535] events cleared: 0 > [64782.630536] state: 00000000 > [64782.630537] chunksize: 67108864 B > [64782.630537] daemon sleep: 5s > [64782.630538] sync size: 878956288 KB > [64782.630539] max write behind: 0 > [64782.630541] md127: failed to create bitmap (-2) > [64782.630577] md: md127 stopped. > [64782.630581] md: unbind<dm-3> > [64782.635133] md: export_rdev(dm-3) > [64782.635145] md: unbind<dm-4> > [64782.643111] md: export_rdev(dm-4) > --snip-- > > I'm using md-cluster built-in, not as a module: > # zcat /proc/config.gz | grep MD_CLUSTER > CONFIG_MD_CLUSTER=y > > It seems the driver is attempting to load the 'md-cluster' module > (from drivers/md/md.c): > --snip-- > err = request_module("md-cluster"); > if (err) { > pr_err("md-cluster module not found.\n"); > return -ENOENT; > } > --snip-- > > I looked at linux-next and it appears this code is the same; is there > a test we can do before attempting to load the module in the case that > its built-in, or is there some other requirement that md-cluster needs > to be built as a module? Yes, we need some additional modules corosync/pacemaker and dlm, pls refer to http://www.spinics.net/lists/raid/msg47863.html, HTH. Regards, Guoqing ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: md-cluster Module Requirement 2016-09-01 2:18 ` Guoqing Jiang @ 2016-09-01 5:52 ` NeilBrown 2016-09-01 9:30 ` Guoqing Jiang 0 siblings, 1 reply; 6+ messages in thread From: NeilBrown @ 2016-09-01 5:52 UTC (permalink / raw) To: Guoqing Jiang, Marc Smith, linux-raid [-- Attachment #1: Type: text/plain, Size: 3169 bytes --] On Thu, Sep 01 2016, Guoqing Jiang wrote: > On 08/26/2016 10:40 AM, Marc Smith wrote: >> Hi, >> >> I'm attempting to use md-cluster from Linux 4.5.2 with mdadm 3.4 and >> I'm running into this when attempting to create a RAID1 device with >> the clustered bitmap: >> >> --snip-- >> [64782.619968] md: bind<dm-4> >> [64782.629336] md: bind<dm-3> >> [64782.630008] md/raid1:md127: active with 2 out of 2 mirrors >> [64782.630528] md-cluster module not found. >> [64782.630530] md127: Could not setup cluster service (-2) >> [64782.630531] md127: bitmap file superblock: >> [64782.630532] magic: 6d746962 >> [64782.630533] version: 5 >> [64782.630534] uuid: 10fee18f.f553d7f2.deb926f1.c7c4bd4b >> [64782.630534] events: 0 >> [64782.630535] events cleared: 0 >> [64782.630536] state: 00000000 >> [64782.630537] chunksize: 67108864 B >> [64782.630537] daemon sleep: 5s >> [64782.630538] sync size: 878956288 KB >> [64782.630539] max write behind: 0 >> [64782.630541] md127: failed to create bitmap (-2) >> [64782.630577] md: md127 stopped. >> [64782.630581] md: unbind<dm-3> >> [64782.635133] md: export_rdev(dm-3) >> [64782.635145] md: unbind<dm-4> >> [64782.643111] md: export_rdev(dm-4) >> --snip-- >> >> I'm using md-cluster built-in, not as a module: >> # zcat /proc/config.gz | grep MD_CLUSTER >> CONFIG_MD_CLUSTER=y >> >> It seems the driver is attempting to load the 'md-cluster' module >> (from drivers/md/md.c): >> --snip-- >> err = request_module("md-cluster"); >> if (err) { >> pr_err("md-cluster module not found.\n"); >> return -ENOENT; >> } >> --snip-- I think this code is wrong. It should be more like: diff --git a/drivers/md/md.c b/drivers/md/md.c index d646f6e444f0..09036add7f33 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -7612,15 +7612,13 @@ int md_setup_cluster(struct mddev *mddev, int nodes) { int err; - err = request_module("md-cluster"); - if (err) { - pr_err("md-cluster module not found.\n"); - return -ENOENT; - } + if (!md_cluster_ops) + request_module("md-cluster"); spin_lock(&pers_lock); if (!md_cluster_ops || !try_module_get(md_cluster_mod)) { spin_unlock(&pers_lock); + pr_err("md-cluster module not found.\n"); return -ENOENT; } spin_unlock(&pers_lock); >> >> I looked at linux-next and it appears this code is the same; is there >> a test we can do before attempting to load the module in the case that >> its built-in, or is there some other requirement that md-cluster needs >> to be built as a module? > > Yes, we need some additional modules corosync/pacemaker and dlm, That doesn't explain the error message though. If MD_CLUSTER is built in, then DLM must be too. NeilBrown > pls refer to http://www.spinics.net/lists/raid/msg47863.html, HTH. > > Regards, > Guoqing > > -- > To unsubscribe from this list: send the line "unsubscribe linux-raid" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 800 bytes --] ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: md-cluster Module Requirement 2016-09-01 5:52 ` NeilBrown @ 2016-09-01 9:30 ` Guoqing Jiang 2016-09-01 10:51 ` NeilBrown 0 siblings, 1 reply; 6+ messages in thread From: Guoqing Jiang @ 2016-09-01 9:30 UTC (permalink / raw) To: NeilBrown, Marc Smith, linux-raid On 09/01/2016 01:52 AM, NeilBrown wrote: > On Thu, Sep 01 2016, Guoqing Jiang wrote: > >> On 08/26/2016 10:40 AM, Marc Smith wrote: >>> Hi, >>> >>> I'm attempting to use md-cluster from Linux 4.5.2 with mdadm 3.4 and >>> I'm running into this when attempting to create a RAID1 device with >>> the clustered bitmap: >>> >>> --snip-- >>> [64782.619968] md: bind<dm-4> >>> [64782.629336] md: bind<dm-3> >>> [64782.630008] md/raid1:md127: active with 2 out of 2 mirrors >>> [64782.630528] md-cluster module not found. >>> [64782.630530] md127: Could not setup cluster service (-2) >>> [64782.630531] md127: bitmap file superblock: >>> [64782.630532] magic: 6d746962 >>> [64782.630533] version: 5 >>> [64782.630534] uuid: 10fee18f.f553d7f2.deb926f1.c7c4bd4b >>> [64782.630534] events: 0 >>> [64782.630535] events cleared: 0 >>> [64782.630536] state: 00000000 >>> [64782.630537] chunksize: 67108864 B >>> [64782.630537] daemon sleep: 5s >>> [64782.630538] sync size: 878956288 KB >>> [64782.630539] max write behind: 0 >>> [64782.630541] md127: failed to create bitmap (-2) >>> [64782.630577] md: md127 stopped. >>> [64782.630581] md: unbind<dm-3> >>> [64782.635133] md: export_rdev(dm-3) >>> [64782.635145] md: unbind<dm-4> >>> [64782.643111] md: export_rdev(dm-4) >>> --snip-- >>> >>> I'm using md-cluster built-in, not as a module: >>> # zcat /proc/config.gz | grep MD_CLUSTER >>> CONFIG_MD_CLUSTER=y >>> >>> It seems the driver is attempting to load the 'md-cluster' module >>> (from drivers/md/md.c): >>> --snip-- >>> err = request_module("md-cluster"); >>> if (err) { >>> pr_err("md-cluster module not found.\n"); >>> return -ENOENT; >>> } >>> --snip-- > I think this code is wrong. It should be more like: > > diff --git a/drivers/md/md.c b/drivers/md/md.c > index d646f6e444f0..09036add7f33 100644 > --- a/drivers/md/md.c > +++ b/drivers/md/md.c > @@ -7612,15 +7612,13 @@ int md_setup_cluster(struct mddev *mddev, int nodes) > { > int err; > > - err = request_module("md-cluster"); > - if (err) { > - pr_err("md-cluster module not found.\n"); > - return -ENOENT; > - } > + if (!md_cluster_ops) > + request_module("md-cluster"); > > spin_lock(&pers_lock); > if (!md_cluster_ops || !try_module_get(md_cluster_mod)) { > spin_unlock(&pers_lock); > + pr_err("md-cluster module not found.\n"); > return -ENOENT; > } > spin_unlock(&pers_lock); Thanks, how about below changes? --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -7619,20 +7619,19 @@ EXPORT_SYMBOL(unregister_md_cluster_operations); int md_setup_cluster(struct mddev *mddev, int nodes) { - int err; - - err = request_module("md-cluster"); - if (err) { - pr_err("md-cluster module not found.\n"); - return -ENOENT; - } - spin_lock(&pers_lock); - if (!md_cluster_ops || !try_module_get(md_cluster_mod)) { - spin_unlock(&pers_lock); - return -ENOENT; + if (!md_cluster_ops) { + /* load module and ensure it won't be unloaded */ + if (!request_module("md-cluster") && + !try_module_get(md_cluster_mod)) { + pr_err("md-cluster module found.\n"); + spin_unlock(&pers_lock); + } else { + pr_err("md-cluster module not found.\n"); + spin_unlock(&pers_lock); + return -ENODEV; + } } - spin_unlock(&pers_lock); return md_cluster_ops->join(mddev, nodes); } >>> I looked at linux-next and it appears this code is the same; is there >>> a test we can do before attempting to load the module in the case that >>> its built-in, or is there some other requirement that md-cluster needs >>> to be built as a module? >> Yes, we need some additional modules corosync/pacemaker and dlm, > That doesn't explain the error message though. You are right, I should read carefully :( > If MD_CLUSTER is built in, then DLM must be too. Sure, then only need to ensure cluster is configured right. Best Regards, Guoqing ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: md-cluster Module Requirement 2016-09-01 9:30 ` Guoqing Jiang @ 2016-09-01 10:51 ` NeilBrown 2016-09-01 15:45 ` Marc Smith 0 siblings, 1 reply; 6+ messages in thread From: NeilBrown @ 2016-09-01 10:51 UTC (permalink / raw) To: Guoqing Jiang, Marc Smith, linux-raid [-- Attachment #1: Type: text/plain, Size: 2098 bytes --] On Thu, Sep 01 2016, Guoqing Jiang wrote: > > Thanks, how about below changes? > > --- a/drivers/md/md.c > +++ b/drivers/md/md.c > @@ -7619,20 +7619,19 @@ EXPORT_SYMBOL(unregister_md_cluster_operations); > > int md_setup_cluster(struct mddev *mddev, int nodes) > { > - int err; > - > - err = request_module("md-cluster"); > - if (err) { > - pr_err("md-cluster module not found.\n"); > - return -ENOENT; > - } > - > spin_lock(&pers_lock); > - if (!md_cluster_ops || !try_module_get(md_cluster_mod)) { > - spin_unlock(&pers_lock); > - return -ENOENT; > + if (!md_cluster_ops) { > + /* load module and ensure it won't be unloaded */ > + if (!request_module("md-cluster") && Calling request_module() under a spin_lock is not OK. request_module() needs to wait while the module is loaded. Just call request_module, ignore the error, then take the spinlock and see if the module is registered. NeilBrown > + !try_module_get(md_cluster_mod)) { > + pr_err("md-cluster module found.\n"); > + spin_unlock(&pers_lock); > + } else { > + pr_err("md-cluster module not found.\n"); > + spin_unlock(&pers_lock); > + return -ENODEV; > + } > } > - spin_unlock(&pers_lock); > > return md_cluster_ops->join(mddev, nodes); > } > >>>> I looked at linux-next and it appears this code is the same; is there >>>> a test we can do before attempting to load the module in the case that >>>> its built-in, or is there some other requirement that md-cluster needs >>>> to be built as a module? >>> Yes, we need some additional modules corosync/pacemaker and dlm, >> That doesn't explain the error message though. > > You are right, I should read carefully :( > >> If MD_CLUSTER is built in, then DLM must be too. > > Sure, then only need to ensure cluster is configured right. > > Best Regards, > Guoqing [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 800 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: md-cluster Module Requirement 2016-09-01 10:51 ` NeilBrown @ 2016-09-01 15:45 ` Marc Smith 0 siblings, 0 replies; 6+ messages in thread From: Marc Smith @ 2016-09-01 15:45 UTC (permalink / raw) To: NeilBrown; +Cc: Guoqing Jiang, linux-raid Thanks; I assume an updated patch will ensue? If so, I'll hang tight for that and test it once posted. --Marc On Thu, Sep 1, 2016 at 6:51 AM, NeilBrown <neilb@suse.com> wrote: > On Thu, Sep 01 2016, Guoqing Jiang wrote: > >> >> Thanks, how about below changes? >> >> --- a/drivers/md/md.c >> +++ b/drivers/md/md.c >> @@ -7619,20 +7619,19 @@ EXPORT_SYMBOL(unregister_md_cluster_operations); >> >> int md_setup_cluster(struct mddev *mddev, int nodes) >> { >> - int err; >> - >> - err = request_module("md-cluster"); >> - if (err) { >> - pr_err("md-cluster module not found.\n"); >> - return -ENOENT; >> - } >> - >> spin_lock(&pers_lock); >> - if (!md_cluster_ops || !try_module_get(md_cluster_mod)) { >> - spin_unlock(&pers_lock); >> - return -ENOENT; >> + if (!md_cluster_ops) { >> + /* load module and ensure it won't be unloaded */ >> + if (!request_module("md-cluster") && > > Calling request_module() under a spin_lock is not OK. request_module() > needs to wait while the module is loaded. > > Just call request_module, ignore the error, then take the spinlock and > see if the module is registered. > > NeilBrown > >> + !try_module_get(md_cluster_mod)) { >> + pr_err("md-cluster module found.\n"); >> + spin_unlock(&pers_lock); >> + } else { >> + pr_err("md-cluster module not found.\n"); >> + spin_unlock(&pers_lock); >> + return -ENODEV; >> + } >> } >> - spin_unlock(&pers_lock); >> >> return md_cluster_ops->join(mddev, nodes); >> } >> >>>>> I looked at linux-next and it appears this code is the same; is there >>>>> a test we can do before attempting to load the module in the case that >>>>> its built-in, or is there some other requirement that md-cluster needs >>>>> to be built as a module? >>>> Yes, we need some additional modules corosync/pacemaker and dlm, >>> That doesn't explain the error message though. >> >> You are right, I should read carefully :( >> >>> If MD_CLUSTER is built in, then DLM must be too. >> >> Sure, then only need to ensure cluster is configured right. >> >> Best Regards, >> Guoqing ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-09-01 15:45 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-08-26 14:40 md-cluster Module Requirement Marc Smith 2016-09-01 2:18 ` Guoqing Jiang 2016-09-01 5:52 ` NeilBrown 2016-09-01 9:30 ` Guoqing Jiang 2016-09-01 10:51 ` NeilBrown 2016-09-01 15:45 ` Marc Smith
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox