Re: [PATCH 01/51] CPU hotplug: Provide lockless versions of callback registration functions

["Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>]

On 02/12/2014 11:48 AM, Srivatsa S. Bhat wrote:
> On 02/12/2014 02:21 AM, Toshi Kani wrote:
>> On Wed, 2014-02-12 at 00:50 +0530, Srivatsa S. Bhat wrote:
>>> On 02/11/2014 11:05 PM, Toshi Kani wrote:
>>  :
>>>> How about this?  foo_cpu_notifier returns NOP when foo_notifier_ready is
>>>> false.
>>>>
>>>>         register_cpu_notifier(&foobar_cpu_notifier);
>>>>
>>>>         get_online_cpus();
>>>>
>>>>         for_each_online_cpu(cpu)
>>>>                 init_cpu(cpu);
>>>>
>>>> 	foo_notifier_ready = true;
>>>>
>>>>         put_online_cpus();
>>>>
>>>
>>> Nah, that looks a lot like some quick-n-dirty hack ;-(
>>> It would also amount to burdening the various subsystems to add weird-looking
>>> pieces of code such as this in their callbacks:
>>>
>>> 	if (!foo_notifier_ready)
>>> 		return NOTIFY_OK;
>>>
>>> This only makes it all the more evident that the callback registration APIs
>>> exposed by the CPU hotplug core is poorly designed.
>>>
>>> What we need instead, is an elegant, well-defined and easy-to-use set of
>>> interfaces/APIs exposed by the core CPU hotplug code to the various
>>> subsystems. I don't think we should worry so much about the fact that
>>> we can't use the familiar get/put_online_cpus() in this type of callback
>>> registration scenario. We can introduce a sane set of APIs that work
>>> well in such situations and use them consistently.
>>
>>> For example, something like the code snippet shown below looks pretty
>>> neat to me:
>>>
>>> 	cpu_notifier_register_begin();
>>>
>>> 	for_each_online_cpu(cpu)
>>> 		init_cpu(cpu);
>>>
>>> 	register_cpu_notifier(&foobar_cpu_notifier);
>>>
>>> 	cpu_notifier_register_done();
>>>
>>> What do you think?
>>
>> I agree that it is cleaner for the callers as long as people understand
>> how to use them.  Can you document them properly so that they know when
>> they need to use them instead of the familiar get/put_online_cpus()?
>>
> 
> Sure.. I had updated the documentation with the semantics introduced in
> this patchset, in patch 2:
> 
> http://thread.gmane.org/gmane.linux.kernel/1641638/focus=1641695
> 
> Similarly I'll keep the docs updated with these new APIs in v2 as well.
>

For now, however, let us not add the new rw-semaphore to the CPU hotplug
core yet. Its very unlikely that we'll see any performance issue immediately,
due to serialized initialization of cpu hotplug notifiers, since early boot
is mostly sequential anyway.

Some time in the future, if we start hitting bottlenecks in the cpu hotplug
notifier registration phase (perhaps when we implement parallel CPU boot-up
infrastructure), then we can directly use the rw-semaphore solution, since
we have already worked it out. Besides, like Gautham said, we might want
to be more careful and have a very good justification before adding more
locks to the CPU hotplug core code. So we'll add the new rw-sempahore if
and when it becomes necessary.

I'll post the v2 with the earlier design itself, by adding the new symbols
cpu_notifier_register_begin/done() (to enhance the readability) and map
them to cpu_maps_update_begin/done().

Thank you!

Regards,
Srivatsa S. Bhat