Re: [PATCH v4 1/2] cpu-hotplug: export cpu_hotplug_enable/cpu_hotplug_disable

[Vitaly Kuznetsov <vkuznets@redhat.com>]

Vitaly Kuznetsov <vkuznets@redhat.com> writes:

> Loaded Hyper-V module will use these functions to disable CPU hotplug
> under certain circumstances. Convert cpu_hotplug_disabled to a counter
> (protected by cpu_add_remove_lock) to support e.g. disable -> disable ->
> enable call sequences.
>
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  Documentation/power/suspend-and-cpuhotplug.txt |  6 +++---
>  kernel/cpu.c                                   | 11 ++++++-----
>  2 files changed, 9 insertions(+), 8 deletions(-)
>
> diff --git a/Documentation/power/suspend-and-cpuhotplug.txt b/Documentation/power/suspend-and-cpuhotplug.txt
> index 2850df3..2fc9095 100644
> --- a/Documentation/power/suspend-and-cpuhotplug.txt
> +++ b/Documentation/power/suspend-and-cpuhotplug.txt
> @@ -72,7 +72,7 @@ More details follow:
>                                          |
>                                          v
>                             Disable regular cpu hotplug
> -                        by setting cpu_hotplug_disabled=1
> +                        by increasing cpu_hotplug_disabled
>                                          |
>                                          v
>                              Release cpu_add_remove_lock
> @@ -89,7 +89,7 @@ Resuming back is likewise, with the counterparts being (in the order of
>  execution during resume):
>  * enable_nonboot_cpus() which involves:
>     |  Acquire cpu_add_remove_lock
> -   |  Reset cpu_hotplug_disabled to 0, thereby enabling regular cpu hotplug
> +   |  Decrease cpu_hotplug_disabled, thereby enabling regular cpu hotplug
>     |  Call _cpu_up() [for all those cpus in the frozen_cpus mask, in a loop]
>     |  Release cpu_add_remove_lock
>     v
> @@ -120,7 +120,7 @@ after the entire cycle is complete (i.e., suspend + resume).
>                             Acquire cpu_add_remove_lock
>                                          |
>                                          v
> -                          If cpu_hotplug_disabled is 1
> +                          If cpu_hotplug_disabled > 0
>                                  return gracefully
>                                          |
>                                          |
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 94bbe46..b6e42d2 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -190,17 +190,18 @@ void cpu_hotplug_done(void)
>  void cpu_hotplug_disable(void)
>  {
>  	cpu_maps_update_begin();
> -	cpu_hotplug_disabled = 1;
> +	cpu_hotplug_disabled++;
>  	cpu_maps_update_done();
>  }
> +EXPORT_SYMBOL_GPL(cpu_hotplug_disable);
>
>  void cpu_hotplug_enable(void)
>  {
>  	cpu_maps_update_begin();
> -	cpu_hotplug_disabled = 0;
> +	WARN_ON(--cpu_hotplug_disabled < 0);
>  	cpu_maps_update_done();
>  }
> -
> +EXPORT_SYMBOL_GPL(cpu_hotplug_enable);
>  #endif	/* CONFIG_HOTPLUG_CPU */
>
>  /* Need to know about CPUs going up/down? */
> @@ -600,7 +601,7 @@ int disable_nonboot_cpus(void)
>  	if (!error) {
>  		BUG_ON(num_online_cpus() > 1);
>  		/* Make sure the CPUs won't be enabled by someone else */
> -		cpu_hotplug_disabled = 1;
> +		cpu_hotplug_disabled++;

Actually, there is an issue here: all disable_nonboot_cpus() users do
enable_nonboot_cpus() in case of failure (yesterday I sent a patch to
fix the last one: https://lkml.org/lkml/2015/6/24/379) but we do
cpu_hotplug_disabled++ only in case of success. I think we need to do
cpu_hotplug_disabled++ here unconditionally.

I'll fix and resend.


>  	} else {
>  		pr_err("Non-boot CPUs are not disabled\n");
>  	}
> @@ -622,7 +623,7 @@ void __ref enable_nonboot_cpus(void)
>
>  	/* Allow everyone to use the CPU hotplug again */
>  	cpu_maps_update_begin();
> -	cpu_hotplug_disabled = 0;
> +	WARN_ON(--cpu_hotplug_disabled < 0);

(here we don't really know if we had a success or a failure in
previously called disable_nonboot_cpus()).

>  	if (cpumask_empty(frozen_cpus))
>  		goto out;

-- 
  Vitaly