Re: Q: force_quiescent_state && cpu_online_map

Paul E. McKenney
2008-11-11T17:25:42Z

On Tue, Nov 11, 2008 at 06:03:27PM +0100, Oleg Nesterov wrote:
> I don't think this matters, but still...
>
> force_quiescent_state:
>
> * cpu_online_map is updated by the _cpu_down()
> * using __stop_machine(). Since we're in irqs disabled
> * section, __stop_machine() is not exectuting, hence
> * the cpu_online_map is stable.
> *
> * However, a cpu might have been offlined _just_ before
> * we disabled irqs while entering here.
> * And rcu subsystem might not yet have handled the CPU_DEAD
> * notification, leading to the offlined cpu's bit
> * being set in the rcp->cpumask.
> *
> * Hence cpumask = (rcp->cpumask & cpu_online_map) to prevent
> * sending smp_reschedule() to an offlined CPU.
> */
> cpus_and(cpumask, rcp->cpumask, cpu_online_map);
> cpu_clear(rdp->cpu, cpumask);
> for_each_cpu_mask_nr(cpu, cpumask)
> smp_send_reschedule(cpu);
>
> However,
>
> // called by __stop_machine take_cpu_down()
> arch/x86/kernel/smpboot.c:cpu_disable_common()
>
> /*
> * HACK:
> * Allow any queued timer interrupts to get serviced
> * This is only a temporary solution until we cleanup
> * fixup_irqs as we do for IA64.
> */
> local_irq_enable();
> mdelay(1);
> local_irq_disable();
> ...
> remove_cpu_from_maps(cpu);
>
> So it is possible to send the ipi to the dying CPU. I know nothing
> about this low-level irq code, most probably this is harmless. We
> already did clear_local_APIC(), but I don't understand what it does.

Indeed, some of the things I am doing as part of the hierarchical RCU
implementation need to be applied to preemptable RCU. :-/

Thanx, Paul