Re: [PATCH v2 1/3] powerpc/rtas: use device model APIs and serialization during LPM

[Nathan Lynch <nathanl@linux.ibm.com>]

Tyrel Datwyler <tyreld@linux.ibm.com> writes:
> On 8/2/19 12:29 PM, Nathan Lynch wrote:
>> diff --git a/arch/powerpc/kernel/rtas.c b/arch/powerpc/kernel/rtas.c
>> index 5faf0a64c92b..05824eb4323b 100644
>> --- a/arch/powerpc/kernel/rtas.c
>> +++ b/arch/powerpc/kernel/rtas.c
>> @@ -871,15 +871,17 @@ static int rtas_cpu_state_change_mask(enum rtas_cpu_state state,
>>  		return 0;
>> 
>>  	for_each_cpu(cpu, cpus) {
>> +		struct device *dev = get_cpu_device(cpu);
>> +
>>  		switch (state) {
>>  		case DOWN:
>> -			cpuret = cpu_down(cpu);
>> +			cpuret = device_offline(dev);
>>  			break;
>>  		case UP:
>> -			cpuret = cpu_up(cpu);
>> +			cpuret = device_online(dev);
>
> Not that I have a problem with using the core device api, but as an FYI we had
> discussed in the past introducing one shot functions in kernel/cpu.c for doing
> our take down, bring up where cpu_update_maps() can be held for the whole
> process. The thought was maybe it would be useful generically being able to
> online/offline a bulk subset.

Thanks, I've discussed something along those lines with Gautham and it
may be more desirable in the longer term than having the arch code
perform the locking.

However, I think it would be even better to avoid bringing up all the
offline CPUs only to shut them down again on the destination. Ideally we
could nudge offline threads into H_JOIN without doing all the work
associated with cpuhp callbacks and generating hotplug event noise. I'm
concerned about the overhead incurred by the current design on large
LPAR configurations.

Regardless, I'd expect that this fix shouldn't have to wait for the
implementation of either of those ideas.


>>  			break;
>>  		}
>> -		if (cpuret) {
>> +		if (cpuret < 0) {
>>  			pr_debug("%s: cpu_%s for cpu#%d returned %d.\n",
>>  					__func__,
>>  					((state == UP) ? "up" : "down"),
>> @@ -968,6 +970,8 @@ int rtas_ibm_suspend_me(u64 handle)
>>  	data.token = rtas_token("ibm,suspend-me");
>>  	data.complete = &done;
>> 
>> +	lock_device_hotplug();
>> +
>
> Does taking the device hotplug lock suffice to prevent races with sysfs attempts
> to hotplug (on/off) cpus?

Yes.

> And if so we can strip out the code that checks the mask to see if we
> raced, correct?

I hope so, but I'm not sure, and it's harmless to leave in for
now. There could be other code which (like LPM) initiates cpu
online/offline outside of sysfs.