Re: [PATCH 2/3] sched, timer: Use atomics for thread_group_cputimer to improve scalability

[Jason Low <jason.low2@hp.com>]

On Wed, 2015-04-15 at 15:32 +0200, Peter Zijlstra wrote:
> On Wed, Apr 15, 2015 at 03:25:36PM +0200, Frederic Weisbecker wrote:
> > On Tue, Apr 14, 2015 at 04:09:45PM -0700, Jason Low wrote:
> > >  void thread_group_cputimer(struct task_struct *tsk, struct task_cputime *times)
> > >  {
> > >  	struct thread_group_cputimer *cputimer = &tsk->signal->cputimer;
> > >  	struct task_cputime sum;
> > >  
> > >  	if (!cputimer->running) {
> > >  		/*
> > >  		 * The POSIX timer interface allows for absolute time expiry
> > >  		 * values through the TIMER_ABSTIME flag, therefore we have
> > > +		 * to synchronize the timer to the clock every time we start it.
> > >  		 */
> > >  		thread_group_cputime(tsk, &sum);
> > > +		update_gt_cputime(cputimer, &sum);
> > > +		/* Start 'running' after update_gt_cputime() */
> > > +		smp_store_release(&cputimer->running, 1);
> > 
> > This barrier should be mirrored somewhere but I can't see where in this patch.
> > Maybe in another one in the series. Or maybe there is already a barrier in the
> > existing code that I'm missing. I would expect to see it in account_group_*_time().
> > In any case, there should be comment about what it mirrors.
> 
> I think it should be in cputimer_running(), which should use
> smp_load_acquire() to read cputimer->running.
> 
> That way you guarantee that everything observing 'running' will indeed
> observe the initialized state.

So I intended the smp_store_release() here to be mainly for
documentation purposes, to say that we would like to set running after
the update.

With patch 3/3, even if running happens to get set earlier, the worst
case scenario is that update_gt_cputime may have to do go through some
retry logic. This isn't much of a performance issue in practice
(especially compared to adding smp_load_acquire() to hot paths), since
we only enter this path when we need to enable the timers.

In that case, I'm wondering if should just convert this back to
WRITE_ONCE(cputimer->running, 1) and avoid adding barriers to the hot
paths?

Thanks,
Jason