Re: [PATCH v6 2/3] CPU hotplug, stop-machine: Plug race-window that leads to "IPI-to-offline-CPU"

Srivatsa S. Bhat
2014-05-23T15:32:20Z

On 05/23/2014 08:51 PM, Peter Zijlstra wrote:
> On Fri, May 23, 2014 at 03:42:20PM +0530, Srivatsa S. Bhat wrote:
>> Re-enable interrupts Re-enable interrupts
>>
>> The pending IPI is noted
>> immediately, but alas,
>> the CPU is offline at
>> this point.
>>
>
> So wasn't clear_local_APIC() (and the arch function __cpu_disable() in
> general) wipe all pending interrup state?
>

Apparently not. Atleast powerpc explicitly re-enables only the IPIs..

A snippet from xics_migrate_irqs_away() in powerpc, which is called
in the cpu-disable phase:

/* Interrupts are disabled. */
void xics_migrate_irqs_away(void)
{
[...]
/* Reject any interrupt that was queued to us... */
icp_ops->set_priority(0);

[...]
/* Allow IPIs again... */
icp_ops->set_priority(DEFAULT_PRIORITY);

for_each_irq_desc(virq, desc) {
[...]
/* We only need to migrate enabled IRQS */
if (!desc->action)
continue;

[...]

/* We need to get IPIs still. */
if (irq == XICS_IPI || irq == XICS_IRQ_SPURIOUS)
continue;
[...]

}

But that said, it should have cleared any pending IPIs and allowed
only *new* IPIs (for whatever reason). Atleast the warning I hit
indicates that that didn't happen, and the old IPI really was taken
later.

Ben, Paul, any thoughts on this?