Re: [PATCH v6 3/3] CPU hotplug, smp: Flush any pending IPI callbacks before CPU offline

["Srivatsa S. Bhat" <srivatsa.bhat@linux.vnet.ibm.com>]

On 05/23/2014 06:57 PM, Frederic Weisbecker wrote:
> On Fri, May 23, 2014 at 03:42:35PM +0530, Srivatsa S. Bhat wrote:
>> During CPU offline, in the stop-machine loop, we use 2 separate stages to
>> disable interrupts, to ensure that the CPU going offline doesn't get any new
>> IPIs from the other CPUs after it has gone offline.
>>
[...]
>> +/**
>> + * generic_smp_call_function_single_interrupt - Execute SMP IPI callbacks
>> + *
>> + * Invoked by arch to handle an IPI for call function single.
>> + * Must be called with interrupts disabled.
>> + */
>> +void generic_smp_call_function_single_interrupt(void)
>> +{
>> +	static bool warned;
>> +
>> +	WARN_ON(!irqs_disabled());
>> +
>>  	/*
>>  	 * Shouldn't receive this interrupt on a cpu that is not yet online.
>>  	 */
>>  	if (unlikely(!cpu_online(smp_processor_id()) && !warned)) {
>> +		struct llist_node *entry;
>> +		struct call_single_data *csd;
>> +
>>  		warned = true;
>>  		WARN(1, "IPI on offline CPU %d\n", smp_processor_id());
>>  
>> +		entry = llist_del_all(&__get_cpu_var(call_single_queue));
> 
> This is deleting all the entries, the call to flush_smp_call_function_queue()
> will then miss these.
>

Argh! And I thought I had finally nailed it this time. :-(

Regards,
Srivatsa S. Bhat
 
>> +		entry = llist_reverse_order(entry);
> 
> 
> 
>> +
>>  		/*
>>  		 * We don't have to use the _safe() variant here
>>  		 * because we are not invoking the IPI handlers yet.
>> @@ -206,10 +250,7 @@ void generic_smp_call_function_single_interrupt(void)
>>  				csd->func);
>>  	}
>>  
>> -	llist_for_each_entry_safe(csd, csd_next, entry, llist) {
>> -		csd->func(csd->info);
>> -		csd_unlock(csd);
>> -	}
>> +	flush_smp_call_function_queue();
>>  }
>>  
>>  /*
>>
>