[PATCH] md-cluster: make md-cluster also can work when compiled into kernel

[PATCH] md-cluster: make md-cluster also can work when compiled into kernel

[Guoqing Jiang]

The md-cluster is compiled as module by default,
if it is compiled by built-in way, then we can't
make md-cluster works.

[64782.630008] md/raid1:md127: active with 2 out of 2 mirrors
[64782.630528] md-cluster module not found.
[64782.630530] md127: Could not setup cluster service (-2)

Fixes: edb39c9 ("Introduce md_cluster_operations to handle cluster functions")
Cc: stable@vger.kernel.org # v4.1+
Cc: NeilBrown <neilb@suse.com>
Reported-by: Marc Smith <marc.smith@mcc.edu>
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
 drivers/md/md.c | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)

diff --git a/drivers/md/md.c b/drivers/md/md.c
index bdbbb6e1..3b19d21 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -7619,20 +7619,17 @@ EXPORT_SYMBOL(unregister_md_cluster_operations);
 
 int md_setup_cluster(struct mddev *mddev, int nodes)
 {
-	int err;
-
-	err = request_module("md-cluster");
-	if (err) {
-		pr_err("md-cluster module not found.\n");
-		return -ENOENT;
-	}
-
-	spin_lock(&pers_lock);
-	if (!md_cluster_ops || !try_module_get(md_cluster_mod)) {
+	if (!md_cluster_ops) {
+		/* load module and ensure it won't be unloaded */
+		request_module("md-cluster");
+		spin_lock(&pers_lock);
+		if (!try_module_get(md_cluster_mod)) {
+			pr_err("can't get md-cluster module reference.\n");
+			spin_unlock(&pers_lock);
+			return -ENODEV;
+		}
 		spin_unlock(&pers_lock);
-		return -ENOENT;
 	}
-	spin_unlock(&pers_lock);
 
 	return md_cluster_ops->join(mddev, nodes);
 }
-- 
2.6.6

Re: [PATCH] md-cluster: make md-cluster also can work when compiled into kernel

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1877 bytes --]

On Fri, Sep 02 2016, Guoqing Jiang wrote:

> The md-cluster is compiled as module by default,
> if it is compiled by built-in way, then we can't
> make md-cluster works.
>
> [64782.630008] md/raid1:md127: active with 2 out of 2 mirrors
> [64782.630528] md-cluster module not found.
> [64782.630530] md127: Could not setup cluster service (-2)
>
> Fixes: edb39c9 ("Introduce md_cluster_operations to handle cluster functions")
> Cc: stable@vger.kernel.org # v4.1+
> Cc: NeilBrown <neilb@suse.com>
> Reported-by: Marc Smith <marc.smith@mcc.edu>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
>  drivers/md/md.c | 21 +++++++++------------
>  1 file changed, 9 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index bdbbb6e1..3b19d21 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -7619,20 +7619,17 @@ EXPORT_SYMBOL(unregister_md_cluster_operations);
>  
>  int md_setup_cluster(struct mddev *mddev, int nodes)
>  {
> -	int err;
> -
> -	err = request_module("md-cluster");
> -	if (err) {
> -		pr_err("md-cluster module not found.\n");
> -		return -ENOENT;
> -	}
> -
> -	spin_lock(&pers_lock);
> -	if (!md_cluster_ops || !try_module_get(md_cluster_mod)) {
> +	if (!md_cluster_ops) {
> +		/* load module and ensure it won't be unloaded */
> +		request_module("md-cluster");
> +		spin_lock(&pers_lock);
> +		if (!try_module_get(md_cluster_mod)) {
> +			pr_err("can't get md-cluster module reference.\n");
> +			spin_unlock(&pers_lock);
> +			return -ENODEV;
> +		}
>  		spin_unlock(&pers_lock);
> -		return -ENOENT;
>  	}
> -	spin_unlock(&pers_lock);
>  
>  	return md_cluster_ops->join(mddev, nodes);
>  }
> -- 
> 2.6.6

No good.  If md_cluster_ops is set, try_module_get() won't be called, so
it will be possible to unload the module while it is in use.

NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

[PATCH V2] md-cluster: make md-cluster also can work when compiled into kernel

[Guoqing Jiang]

The md-cluster is compiled as module by default,
if it is compiled by built-in way, then we can't
make md-cluster works.

[64782.630008] md/raid1:md127: active with 2 out of 2 mirrors
[64782.630528] md-cluster module not found.
[64782.630530] md127: Could not setup cluster service (-2)

Fixes: edb39c9 ("Introduce md_cluster_operations to handle cluster functions")
Cc: stable@vger.kernel.org # v4.1+
Cc: NeilBrown <neilb@suse.com>
Reported-by: Marc Smith <marc.smith@mcc.edu>
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
Changes:
1. call try_module_get if md_cluster_ops is already set,
otherwise try_module_get/module_put are unbalanced.

 drivers/md/md.c | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)

diff --git a/drivers/md/md.c b/drivers/md/md.c
index 67642ba..6ac5abe 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -7610,16 +7610,12 @@ EXPORT_SYMBOL(unregister_md_cluster_operations);
 
 int md_setup_cluster(struct mddev *mddev, int nodes)
 {
-	int err;
-
-	err = request_module("md-cluster");
-	if (err) {
-		pr_err("md-cluster module not found.\n");
-		return -ENOENT;
-	}
-
+	if (!md_cluster_ops)
+		request_module("md-cluster");
 	spin_lock(&pers_lock);
-	if (!md_cluster_ops || !try_module_get(md_cluster_mod)) {
+	/* ensure module won't be unloaded */
+	if (!try_module_get(md_cluster_mod)) {
+		pr_err("can't get md-cluster module reference.\n");
 		spin_unlock(&pers_lock);
 		return -ENOENT;
 	}
-- 
2.6.6

Re: [PATCH V2] md-cluster: make md-cluster also can work when compiled into kernel

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2193 bytes --]

On Sun, Sep 04 2016, Guoqing Jiang wrote:

> The md-cluster is compiled as module by default,
> if it is compiled by built-in way, then we can't
> make md-cluster works.
>
> [64782.630008] md/raid1:md127: active with 2 out of 2 mirrors
> [64782.630528] md-cluster module not found.
> [64782.630530] md127: Could not setup cluster service (-2)
>
> Fixes: edb39c9 ("Introduce md_cluster_operations to handle cluster functions")
> Cc: stable@vger.kernel.org # v4.1+

The above results in you sending email to
   stable@vger.kernel.org#v4.1+
which is not a valid address.

The correct form for comments in email address is to use parentheses.
e.g.
   Cc: stable@vger.kernel.org (v4.1+)

> Cc: NeilBrown <neilb@suse.com>
> Reported-by: Marc Smith <marc.smith@mcc.edu>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> Changes:
> 1. call try_module_get if md_cluster_ops is already set,
> otherwise try_module_get/module_put are unbalanced.
>
>  drivers/md/md.c | 14 +++++---------
>  1 file changed, 5 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 67642ba..6ac5abe 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -7610,16 +7610,12 @@ EXPORT_SYMBOL(unregister_md_cluster_operations);
>  
>  int md_setup_cluster(struct mddev *mddev, int nodes)
>  {
> -	int err;
> -
> -	err = request_module("md-cluster");
> -	if (err) {
> -		pr_err("md-cluster module not found.\n");
> -		return -ENOENT;
> -	}
> -
> +	if (!md_cluster_ops)
> +		request_module("md-cluster");
>  	spin_lock(&pers_lock);
> -	if (!md_cluster_ops || !try_module_get(md_cluster_mod)) {
> +	/* ensure module won't be unloaded */
> +	if (!try_module_get(md_cluster_mod)) {

Why did you drop the "!md_cluster_ops" test?

What happens if the md-cluster module cannot be loaded?
md_cluster_ops will be NULL and md_cluster_mod will be NULL.
try_module_get(NULL) succeeds, so with this patch md_setup_cluster()
will succeed if the module is needed but fails to load.

NeilBrown



> +		pr_err("can't get md-cluster module reference.\n");
>  		spin_unlock(&pers_lock);
>  		return -ENOENT;
>  	}
> -- 
> 2.6.6

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: [PATCH V2] md-cluster: make md-cluster also can work when compiled into kernel

[Guoqing Jiang]

On 09/04/2016 07:24 PM, NeilBrown wrote:
> On Sun, Sep 04 2016, Guoqing Jiang wrote:
>
>> The md-cluster is compiled as module by default,
>> if it is compiled by built-in way, then we can't
>> make md-cluster works.
>>
>> [64782.630008] md/raid1:md127: active with 2 out of 2 mirrors
>> [64782.630528] md-cluster module not found.
>> [64782.630530] md127: Could not setup cluster service (-2)
>>
>> Fixes: edb39c9 ("Introduce md_cluster_operations to handle cluster functions")
>> Cc: stable@vger.kernel.org # v4.1+
> The above results in you sending email to
>     stable@vger.kernel.org#v4.1+
> which is not a valid address.
>
> The correct form for comments in email address is to use parentheses.
> e.g.
>     Cc: stable@vger.kernel.org (v4.1+)

Thanks for correct it!

>> Cc: NeilBrown <neilb@suse.com>
>> Reported-by: Marc Smith <marc.smith@mcc.edu>
>> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
>> ---
>> Changes:
>> 1. call try_module_get if md_cluster_ops is already set,
>> otherwise try_module_get/module_put are unbalanced.
>>
>>   drivers/md/md.c | 14 +++++---------
>>   1 file changed, 5 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/md/md.c b/drivers/md/md.c
>> index 67642ba..6ac5abe 100644
>> --- a/drivers/md/md.c
>> +++ b/drivers/md/md.c
>> @@ -7610,16 +7610,12 @@ EXPORT_SYMBOL(unregister_md_cluster_operations);
>>   
>>   int md_setup_cluster(struct mddev *mddev, int nodes)
>>   {
>> -	int err;
>> -
>> -	err = request_module("md-cluster");
>> -	if (err) {
>> -		pr_err("md-cluster module not found.\n");
>> -		return -ENOENT;
>> -	}
>> -
>> +	if (!md_cluster_ops)
>> +		request_module("md-cluster");
>>   	spin_lock(&pers_lock);
>> -	if (!md_cluster_ops || !try_module_get(md_cluster_mod)) {
>> +	/* ensure module won't be unloaded */
>> +	if (!try_module_get(md_cluster_mod)) {
> Why did you drop the "!md_cluster_ops" test?
>
> What happens if the md-cluster module cannot be loaded?
> md_cluster_ops will be NULL and md_cluster_mod will be NULL.
> try_module_get(NULL) succeeds, so with this patch md_setup_cluster()
> will succeed if the module is needed but fails to load.

Yes, I just find try_module_get(NULL) could return true, it seems not
correct for the return value, maybe it need to be changed to:

             return module ? ret : false;

Anyway, I will send v3 with add "!md_cluster_ops" test back.

Thanks,
Guoqing

Re: [PATCH V2] md-cluster: make md-cluster also can work when compiled into kernel

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2783 bytes --]

On Mon, Sep 05 2016, Guoqing Jiang wrote:

> On 09/04/2016 07:24 PM, NeilBrown wrote:
>> On Sun, Sep 04 2016, Guoqing Jiang wrote:
>>
>>> The md-cluster is compiled as module by default,
>>> if it is compiled by built-in way, then we can't
>>> make md-cluster works.
>>>
>>> [64782.630008] md/raid1:md127: active with 2 out of 2 mirrors
>>> [64782.630528] md-cluster module not found.
>>> [64782.630530] md127: Could not setup cluster service (-2)
>>>
>>> Fixes: edb39c9 ("Introduce md_cluster_operations to handle cluster functions")
>>> Cc: stable@vger.kernel.org # v4.1+
>> The above results in you sending email to
>>     stable@vger.kernel.org#v4.1+
>> which is not a valid address.
>>
>> The correct form for comments in email address is to use parentheses.
>> e.g.
>>     Cc: stable@vger.kernel.org (v4.1+)
>
> Thanks for correct it!
>
>>> Cc: NeilBrown <neilb@suse.com>
>>> Reported-by: Marc Smith <marc.smith@mcc.edu>
>>> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
>>> ---
>>> Changes:
>>> 1. call try_module_get if md_cluster_ops is already set,
>>> otherwise try_module_get/module_put are unbalanced.
>>>
>>>   drivers/md/md.c | 14 +++++---------
>>>   1 file changed, 5 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/drivers/md/md.c b/drivers/md/md.c
>>> index 67642ba..6ac5abe 100644
>>> --- a/drivers/md/md.c
>>> +++ b/drivers/md/md.c
>>> @@ -7610,16 +7610,12 @@ EXPORT_SYMBOL(unregister_md_cluster_operations);
>>>   
>>>   int md_setup_cluster(struct mddev *mddev, int nodes)
>>>   {
>>> -	int err;
>>> -
>>> -	err = request_module("md-cluster");
>>> -	if (err) {
>>> -		pr_err("md-cluster module not found.\n");
>>> -		return -ENOENT;
>>> -	}
>>> -
>>> +	if (!md_cluster_ops)
>>> +		request_module("md-cluster");
>>>   	spin_lock(&pers_lock);
>>> -	if (!md_cluster_ops || !try_module_get(md_cluster_mod)) {
>>> +	/* ensure module won't be unloaded */
>>> +	if (!try_module_get(md_cluster_mod)) {
>> Why did you drop the "!md_cluster_ops" test?
>>
>> What happens if the md-cluster module cannot be loaded?
>> md_cluster_ops will be NULL and md_cluster_mod will be NULL.
>> try_module_get(NULL) succeeds, so with this patch md_setup_cluster()
>> will succeed if the module is needed but fails to load.
>
> Yes, I just find try_module_get(NULL) could return true, it seems not
> correct for the return value, maybe it need to be changed to:
>
>              return module ? ret : false;

If a module is built into the kernel, then the module pointer that code
will use with be NULL, but you still want try_module_get() and
module_put() to work.  So I think returning 'true' for NULL is correct.

>
> Anyway, I will send v3 with add "!md_cluster_ops" test back.

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

[PATCH V3] md-cluster: make md-cluster also can work when compiled into kernel

[Guoqing Jiang]

The md-cluster is compiled as module by default,
if it is compiled by built-in way, then we can't
make md-cluster works.

[64782.630008] md/raid1:md127: active with 2 out of 2 mirrors
[64782.630528] md-cluster module not found.
[64782.630530] md127: Could not setup cluster service (-2)

Fixes: edb39c9 ("Introduce md_cluster_operations to handle cluster functions")
Cc: stable@vger.kernel.org (v4.1+)
Cc: NeilBrown <neilb@suse.com>
Reported-by: Marc Smith <marc.smith@mcc.edu>
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
V3 changes:
1. add the "!md_cluster_ops" test back
2. fix wrong mail info of stable kernel

V2 changes:
1. call try_module_get if md_cluster_ops is already set,
   otherwise try_module_get/module_put are unbalanced.

 drivers/md/md.c | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/drivers/md/md.c b/drivers/md/md.c
index 67642ba..915e84d 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -7610,16 +7610,12 @@ EXPORT_SYMBOL(unregister_md_cluster_operations);
 
 int md_setup_cluster(struct mddev *mddev, int nodes)
 {
-	int err;
-
-	err = request_module("md-cluster");
-	if (err) {
-		pr_err("md-cluster module not found.\n");
-		return -ENOENT;
-	}
-
+	if (!md_cluster_ops)
+		request_module("md-cluster");
 	spin_lock(&pers_lock);
+	/* ensure module won't be unloaded */
 	if (!md_cluster_ops || !try_module_get(md_cluster_mod)) {
+		pr_err("can't find md-cluster module or get it's reference.\n");
 		spin_unlock(&pers_lock);
 		return -ENOENT;
 	}
-- 
2.6.6

Re: [PATCH V3] md-cluster: make md-cluster also can work when compiled into kernel

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 1790 bytes --]

On Mon, Sep 05 2016, Guoqing Jiang wrote:

> The md-cluster is compiled as module by default,
> if it is compiled by built-in way, then we can't
> make md-cluster works.
>
> [64782.630008] md/raid1:md127: active with 2 out of 2 mirrors
> [64782.630528] md-cluster module not found.
> [64782.630530] md127: Could not setup cluster service (-2)
>
> Fixes: edb39c9 ("Introduce md_cluster_operations to handle cluster functions")
> Cc: stable@vger.kernel.org (v4.1+)
> Cc: NeilBrown <neilb@suse.com>
> Reported-by: Marc Smith <marc.smith@mcc.edu>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> V3 changes:
> 1. add the "!md_cluster_ops" test back
> 2. fix wrong mail info of stable kernel
>
> V2 changes:
> 1. call try_module_get if md_cluster_ops is already set,
>    otherwise try_module_get/module_put are unbalanced.
>
>  drivers/md/md.c | 12 ++++--------
>  1 file changed, 4 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 67642ba..915e84d 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -7610,16 +7610,12 @@ EXPORT_SYMBOL(unregister_md_cluster_operations);
>  
>  int md_setup_cluster(struct mddev *mddev, int nodes)
>  {
> -	int err;
> -
> -	err = request_module("md-cluster");
> -	if (err) {
> -		pr_err("md-cluster module not found.\n");
> -		return -ENOENT;
> -	}
> -
> +	if (!md_cluster_ops)
> +		request_module("md-cluster");
>  	spin_lock(&pers_lock);
> +	/* ensure module won't be unloaded */
>  	if (!md_cluster_ops || !try_module_get(md_cluster_mod)) {
> +		pr_err("can't find md-cluster module or get it's reference.\n");
>  		spin_unlock(&pers_lock);
>  		return -ENOENT;
>  	}
> -- 
> 2.6.6

Reviewed-by: NeilBrown <neilb@suse.com>

Thanks,
NeilBrown

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: [PATCH V3] md-cluster: make md-cluster also can work when compiled into kernel

[Shaohua Li]

On Sun, Sep 04, 2016 at 10:17:28PM -0400, Guoqing Jiang wrote:
> The md-cluster is compiled as module by default,
> if it is compiled by built-in way, then we can't
> make md-cluster works.
> 
> [64782.630008] md/raid1:md127: active with 2 out of 2 mirrors
> [64782.630528] md-cluster module not found.
> [64782.630530] md127: Could not setup cluster service (-2)
> 
> Fixes: edb39c9 ("Introduce md_cluster_operations to handle cluster functions")
> Cc: stable@vger.kernel.org (v4.1+)
> Cc: NeilBrown <neilb@suse.com>
> Reported-by: Marc Smith <marc.smith@mcc.edu>
> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> ---
> V3 changes:
> 1. add the "!md_cluster_ops" test back
> 2. fix wrong mail info of stable kernel
> 
> V2 changes:
> 1. call try_module_get if md_cluster_ops is already set,
>    otherwise try_module_get/module_put are unbalanced.

applied, thanks!