Re: [BUG, bisect] hrtimer: severe lag after suspend & resume

[Ingo Molnar <mingo@kernel.org>]

* Thomas Gleixner <tglx@linutronix.de> wrote:

> On Thu, 4 Jun 2015, John Stultz wrote:
> > On Wed, Jun 3, 2015 at 5:56 PM, Jeremiah Mahler <jmmahler@gmail.com> wrote:
> > So I suspect the problem is the change to clock_was_set_seq in
> > timekeeping_update is done prior to mirroring the time state to the
> > shadow-timekeeper. Thus the next time we do update_wall_time() the
> > updated sequence is overwritten by whats in the shadow copy. The
> > attached patch moving the modification up seems to avoid the issue for
> > me.
> 
> Duh, yes.
>  
> > Thomas: Looking at the problematic change, I'm not a big fan of it. Caching 
> > timekeeping state here in the hrtimer code has been a source of bugs in the 
> > past, and I'm not sure I see how avoiding copying 24bytes is that big of a 
> > win. Especially since it adds more state to the timekeeper and hrtimer base 
> > that we have to read and mange.
> 
> It's not about copying 24 bytes. It's about touching 3 cache lines for nothing. 
> In situations where we run high frequency periodic timers on clock monotonic and 
> nothing is going on in the other clock domains, which is a pretty common 
> situation, this is measurable in terms of cache utilization. [...]

It's not just about 'touching': it's about _dirtying_ cachelines from a globally 
executed function (timekeeping), which is then accessed by per-CPU functionality 
(hrtimers).

That makes it far more expensive, it has similar scalability limiting effects as a 
global lock - while if we do it smart it can perform as essentially lockless code 
in most cases.

Thanks,

	Ingo