Re: [RFC PATCH 0/6] CPU hotplug: Reverse invocation of notifiers during CPU hotplug

["Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>]

On 07/25/2012 10:00 PM, Thomas Gleixner wrote:
> On Wed, 25 Jul 2012, Srivatsa S. Bhat wrote:
>> On 07/25/2012 08:27 PM, Alan Stern wrote:
>> One of the other ideas to improve the hotplug notifier stuff that came up during some
>> of the discussions was to implement explicit dependency tracking between the notifiers
>> and perhaps get rid of the priority numbers that are currently being used to provide
>> some sort of ordering between the callbacks. Links to some of the related discussions
>> are provided below.
> 
> The current code which brings up/down a CPU (mostly architecture
> specific) code is comnpletely asymetric.
> 
> We really want a fully symetric state machine here, which also gives
> us the proper invocation points for the other subsystems callbacks.
> 
> While I thought about having a full dependency tracking system, I'm
> quite convinced by now, that hotplug is a rather linear sequence which
> does not provide much room for paralell setup/teardown.
> 
> At least we should start with a simple linear chain.
> 
> The problem with the current notifiers is, that we only have ordering
> for a few specific callbacks, but we don't have the faintest idea in
> which order all other random stuff is brought up and torn down.
> 
> So I started experimenting with the following:
> 
> struct hotplug_event {
>        int (*bring_up)(unsigned int cpu);
>        int (*tear_down)(unsigned int cpu);
> };
> 
> enum hotplug_events {
>      CPU_HOTPLUG_START,
>      CPU_HOTPLUG_CREATE_THREADS,
>      CPU_HOTPLUG_INIT_TIMERS,
>      ...
>      CPU_HOTPLUG_KICK_CPU,
>      ...
>      CPU_HOTPLUG_START_THREADS,
>      ...
>      CPU_HOTPLUG_SET_ONLINE,
>      ...
>      CPU_HOTPLUG_MAX_EVENTS,
> };
> 
> Now I have two arrays:
> 
> struct hotplug_event hotplug_events_bp[CPU_HOTPLUG_MAX_EVENTS];
> struct hotplug_event hotplug_events_ap[CPU_HOTPLUG_MAX_EVENTS];
>    
> The _bp one is the list of events which are executed on the active cpu
> and the _ap ones are those executed on the hotplugged cpu.
> 
> The core code advances the events in sync steps, so both BP and AP can
> issue a stop on the process and cause a rollback.

What exactly does "sync steps" mean in this context? Also, for the CPU
offline event, the event could start off with both the BP and the AP being
the same CPU.. Does this design take care of that case?

Regards,
Srivatsa S. Bhat

> 
> Most of the callbacks can be added to the arrays at compile time, just
> the stuff which is in modules requires an register/unregister
> interface.
> 
> Though in any case the enum gives us a very explicit ordering of
> setup/teardown, so rollback or partial online/offline should be simple
> to achieve.
> 
> The only drawback is that it will prevent out of tree modules to use
> the hotplug infrastructure, but I really couldn't care less.
> 
> Thoughts?