Re: Bug in on_each_cpu?

Andrew Morton
2007-03-01T15:29:08Z

On Thu, 01 Mar 2007 03:34:18 -0800 Zachary Amsden <zach@vmware.com> wrote:

> > Why is it a bug? Because there's a deadlock where this CPU is waiting for
> > CPU A to take the IPI, but CPU A is waiting (with interrupts disabled) for
> > this CPU to take an IPI.
> >
>
> Then the bug is not in on_each_cpu(). It is in the usage of
> clock_was_set(). For example, look at do_settimeofday in kernel/timer.c:
>
> write_sequnlock_irqrestore(&xtime_lock, flags);
>
> /* signal hrtimers about time change */
> clock_was_set();
>
> return 0;

Perhaps a WARN_ON(irqs_disabled()) in clock_was_set() would help. But probably
the one in smp_call_function() will suffice.

> And timekeeping_resume has similar code (and called from a sysdev
> callback, so I don't know what the interrupt state should be ). Either
> the write_sequnlock_irqrestore is redundant, and should be merely an
> write_sequnlock_irq, or the callsite is not prepared to handle enabling
> interrupts temporarily as must be done for on_each_cpu(), which is a
> pretty scary scenario.
>
> What would be really, really nice would be to statically check all
> callsites that issue irq disables actually keep irqs disabled.
> Presumably, there was a reason they disabled irqs, and re-enabling them
> underneath their noses, even if it is to avoid a race, breaks the logic
> behind that reason.

yup. I once played with adding warnings in places like spin_lock_irq(),
but there were false positives from places which were odd-but-correct.

It would be worth revisiting however.