Re: WARN_ON_ONCE(in_nmi()) hit in irq_work_queue_on

[Frederic Weisbecker <fweisbec@gmail.com>]

On Thu, Aug 07, 2014 at 10:13:21AM +0200, Peter Zijlstra wrote:
> On Thu, Aug 07, 2014 at 01:44:58AM +0200, Frederic Weisbecker wrote:
> > In fact the problem has arised since the recent irq work patches I did.
> 
> No, those just added the WARN, previously we send the resched IPI, and
> that's equally wrong from NMI context.

Well the scheduler IPI was rather used for remote kicks before we had remote irq
work. This includes local kicks as well as the caller could need to kick
anywhere. As in inc_nr_running().

But for strict local kicks, as in perf, we were using tick_nohz_full_kick() which has
been using irq work for a while. But it got broken when we replaced it to call irq_work_queue_on
instead of irq_work_queue.

> 
> > There I've changed tick_nohz_full_kick() to use irq_work_queue_on() instead
> > of irq_work_queue() so it has become NMI unsafe by accident.
> > 
> > So I'd rather suggest this instead of queuing two levels of irq_work:
> 
> > +void tick_nohz_full_kick(void)
> > +{
> > +	if (!tick_nohz_full_cpu(smp_processor_id()))
> > +		irq_work_queue(&__get_cpu_var(nohz_full_kick_work));
> > +}
> 
> Indeed, that's better. Thanks!