Re: [tip:timers/core] hrtimer: Allow concurrent hrtimer_start() for self restarting timers

[Sasha Levin <sasha.levin@oracle.com>]

On 05/13/2015 09:43 AM, Peter Zijlstra wrote:
> On Tue, May 12, 2015 at 09:52:09AM -0400, Sasha Levin wrote:
>> Hey Peter,
>>
>> I seem to be hitting this warning with the latest -next (2015-05-11):
>>
>> [3344291.055800] WARNING: CPU: 0 PID: 9422 at kernel/time/hrtimer.c:802 hrtimer_forward+0x1f9/0x330()
> 
> Indeed.
> 
> So I can't seem to come up with a pretty solution :-(
> 
> The 'problem' is creating a periodic timer that can stop when there's no
> work left and ensuring (re)start doesn't get lost or looses overruns.
> 
> The problem with the current code is that the re-start can come before
> the callback does fwd, at which point the fwd will spew the above
> warning (rightly so).
> 
> Now, conceptually its easy to detect if you're before or after the fwd
> by comparing the expiration time against the current time. Of course,
> that's expensive (and racy) because we don't have the current time.
> 
> Alternatively one could cache this state inside the timer, but then
> everybody pays the overhead of maintaining this extra state, and that is
> undesired.
> 
> The only other option that I could see is the external timer_active
> variable, which I tried to kill before.
> 
> So I suppose the below (compile tested) patch should fix things, but
> seeing how I've been up since 4am I might just have missed something
> obvious :-)

I know exactly how you feel...

> Almost-Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>

Original warning is gone, but now I get:

[  281.213520] INFO: trying to register non-static key.
[  281.214251] the code is fine but needs lockdep annotation.
[  281.214251] turning off the locking correctness validator.
[  281.214251] CPU: 0 PID: 10071 Comm: trinity-main Tainted: G        W       4.1.0-rc3-next-20150512-sasha-00051-gd40f47b-dirty #2213
[  281.214251]  ffffffffae4a5b20 00000000cbbe31dc ffff88000267f698 ffffffffa7c8e8ab
[  281.214251]  0000000000000000 0000000000000000 ffff88000267f848 ffffffff9e303567
[  281.214251]  ffff88001d015260 0000000000000004 ffff88001d014d20 0000000000000005
[  281.214251] Call Trace:
[  281.214251] dump_stack (lib/dump_stack.c:52)
[  281.214251] __lock_acquire (kernel/locking/lockdep.c:786 kernel/locking/lockdep.c:3134)
[  281.214251] ? x86_schedule_events (arch/x86/kernel/cpu/perf_event.c:849 (discriminator 2))
[  281.214251] ? lockdep_reset_lock (kernel/locking/lockdep.c:3105)
[  281.214251] ? x86_pmu_commit_txn (arch/x86/kernel/cpu/perf_event.c:1098)
[  281.214251] ? x86_pmu_commit_txn (arch/x86/kernel/cpu/perf_event.c:1731)
[  281.214251] ? x86_pmu_cancel_txn (arch/x86/kernel/cpu/perf_event.c:1720)
[  281.214251] ? perf_event_update_userpage (include/linux/rcupdate.h:917 kernel/events/core.c:4256)
[  281.214251] lock_acquire (kernel/locking/lockdep.c:3658)
[  281.214251] ? perf_mux_hrtimer_restart (kernel/events/core.c:807)
[  281.214251] _raw_spin_lock_irqsave (include/linux/spinlock_api_smp.h:119 kernel/locking/spinlock.c:159)
[  281.214251] ? perf_mux_hrtimer_restart (kernel/events/core.c:807)
[  281.214251] perf_mux_hrtimer_restart (kernel/events/core.c:807)
[  281.214251] group_sched_in (kernel/events/core.c:1963)
[  281.214251] ? sched_clock_cpu (kernel/sched/clock.c:311)
[  281.214251] ctx_sched_in (kernel/events/core.c:2725 kernel/events/core.c:2756)
[  281.214251] perf_event_sched_in (kernel/events/core.c:2025)
[  281.214251] __perf_install_in_context (kernel/events/core.c:2082)
[  281.214251] ? perf_mux_hrtimer_handler (kernel/events/core.c:2035)
[  281.214251] remote_function (kernel/events/core.c:74)
[  281.214251] ? pmu_dev_release (kernel/events/core.c:66)
[  281.214251] generic_exec_single (kernel/smp.c:157)
[  281.214251] ? pmu_dev_release (kernel/events/core.c:66)
[  281.214251] smp_call_function_single (kernel/smp.c:300)
[  281.214251] ? generic_exec_single (kernel/smp.c:273)
[  281.214251] ? __lock_is_held (kernel/locking/lockdep.c:3572)
[  281.214251] task_function_call (kernel/events/core.c:101)
[  281.214251] ? remote_function (kernel/events/core.c:92)
[  281.214251] ? perf_mux_hrtimer_handler (kernel/events/core.c:2035)
[  281.214251] perf_install_in_context (kernel/events/core.c:2121)
[  281.214251] ? add_event_to_ctx (kernel/events/core.c:2102)
[  281.214251] ? alloc_perf_context (kernel/events/core.c:3306)
[  281.214251] ? perf_event_alloc (kernel/events/core.c:7569)
[  281.214251] SYSC_perf_event_open (kernel/events/core.c:8133)
[  281.214251] ? perf_event_alloc (kernel/events/core.c:7856)
[  281.214251] ? syscall_trace_enter_phase1 (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1480)
[  281.214251] SyS_perf_event_open (kernel/events/core.c:7853)
[  281.214251] tracesys_phase2 (arch/x86/kernel/entry_64.S:346)


Thanks,
Sasha