Re: [PATCH] acpi, cpuidle: Register with cpuidle even if cpu is onlined after boot (beyond maxcpus)

["Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>]

On 06/25/2012 07:23 PM, Thomas Renninger wrote:

> On Monday, June 25, 2012 01:25:43 PM Srivatsa S. Bhat wrote:
>>
>> Daniel Lezcano noticed that after booting with maxcpus=X, if we online the
>> remaining cpus by writing: echo 1 > /sys/devices/system/cpu/cpuY/online, then
>> for the newly onlined cpus, the cpuidle directory is not found under
>> /sys/devices/system/cpu/cpuY.
>>
>> Partly, the reason for this is that acpi restricts the initialization to cpus
>> within the maxcpus limit. (See commit 75cbfb9 "ACPI: Do not try to set up acpi
>> processor stuff on cores exceeding maxcpus="). The maxcpus= kernel parameter is
>> used to restrict the number of cpus brought up during boot. That doesn't mean
>> that we should hard restrict the bring up of the remaining cpus later on.
> 
> Sorry, but IMO it exaclty does mean that (adding more general lists for
> further comments).
> 
> If you can online more cores than maxcpus= via sysfs, this sounds like a bug.
> Not the other way around.
> 
> Compare with Documentation/kernel-parameters.txt:
>         maxcpus=        [SMP] Maximum number of processors that an SMP kernel
>                         should make use of.  maxcpus=n : n >= 0 limits the
>                         kernel to using 'n' processors.  n=0 is a special case,
>                         it is equivalent to "nosmp", which also disables
>                         the IO APIC.
> 
> Chances that you run into more problems are high.


Right, I agree on that. So, IMHO, maxcpus=X doesn't mean that the kernel must and
should forbid any new cpus from coming online, but in the interest of avoiding
problems/complications in some obscure paths, I guess it makes sense to avoid
onlining new cpus beyond maxcpus.

In any case, I was just trying to see why the simple removal of the setup_max_cpus
check in acpi_processor_add() wasn't enough to expose the cpuidle directories under
the new cpus.. and while debugging that, I came up with this patch. I don't mind
if this doesn't get picked up.

> It would help if it is explained why at all this would be needed.
> 


Right, the usecase of why somebody would like to online new cpus beyond maxcpus
doesn't look all that solid anyway. So I am OK with leaving the code as it is now.

> If there really is a valid use-case, possibly a bootcpus= param could get
> defined or above documentation is adjusted. But this would need thorough
> double checking, because I expect maxcpus=X was never intended to bring up
> more than X cores later via sysfs and I expect there are more
> places where things have to get ajusted.
> 


Yep, that sounds reasonable.

Regards,
Srivatsa S. Bhat

> 
>> So
>> teach acpi to play nice with maxcpus= parameter, and perform the initialization
>> for all present cpus, but defer their startup to the point when they are
>> actually onlined later.
>>
>> But that alone won't suffice as a fix, because there is one more thing that
>> the present but !online cpus lack - the need_hotplug_init flag.
>> The need_hotplug_init flag is set only for physically hotplugged cpus and not
>> for cpus which were already present but not brought online during boot (due to
>> the maxcpus= parameter). For cpus with this flag set, during the online
>> operation, acpi_cpu_soft_notify() takes care of the registration with cpuidle.
>> So, in order to allow the present but !online cpus to be onlined later with
>> full features (by making use of acpi_cpu_soft_notify()), set their
>> need_hotplug_init flag.
>>
>> Reported-by: Daniel Lezcano <daniel.lezcano@linaro.org>
>> Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
>>
>> ---
>>
>> Daniel, this patch works for me. Does it work for you as well?
>>
>>  drivers/acpi/processor_driver.c |   16 +++++++++++-----
>>  1 files changed, 11 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/acpi/processor_driver.c b/drivers/acpi/processor_driver.c
>> index 0734086..f29d30f 100644
>> --- a/drivers/acpi/processor_driver.c
>> +++ b/drivers/acpi/processor_driver.c
>> @@ -551,11 +551,6 @@ static int __cpuinit acpi_processor_add(struct acpi_device *device)
>>  		return 0;
>>  	}
>>  
>> -#ifdef CONFIG_SMP
>> -	if (pr->id >= setup_max_cpus && pr->id != 0)
>> -		return 0;
>> -#endif
>> -
>>  	BUG_ON((pr->id >= nr_cpu_ids) || (pr->id < 0));
>>  
>>  	/*
>> @@ -580,6 +575,17 @@ static int __cpuinit acpi_processor_add(struct acpi_device *device)
>>  		goto err_clear_processor;
>>  	}
>>  
>> +#ifdef CONFIG_SMP
>> +	if (pr->id >= setup_max_cpus && pr->id != 0) {
>> +		/*
>> +		 * Don't start cpus beyond maxcpus now. But allow them to
>> +		 * to be brought online later.
>> +		 */
>> +		pr->flags.need_hotplug_init = 1;
>> +		return 0;
>> +	}
>> +#endif
>> +
>>  	/*
>>  	 * Do not start hotplugged CPUs now, but when they
>>  	 * are onlined the first time
>>
>>
>>