Re: [PATCH 2/3] sched, timer: Use atomics for thread_group_cputimer to improve scalability

Jason Low
2015-04-15T20:05:08Z

On Wed, 2015-04-15 at 15:32 +0200, Peter Zijlstra wrote:
> On Wed, Apr 15, 2015 at 03:25:36PM +0200, Frederic Weisbecker wrote:
> > On Tue, Apr 14, 2015 at 04:09:45PM -0700, Jason Low wrote:
> > > void thread_group_cputimer(struct task_struct *tsk, struct task_cputime *times)
> > > {
> > > struct thread_group_cputimer *cputimer = &tsk->signal->cputimer;
> > > struct task_cputime sum;
> > >
> > > if (!cputimer->running) {
> > > /*
> > > * The POSIX timer interface allows for absolute time expiry
> > > * values through the TIMER_ABSTIME flag, therefore we have
> > > + * to synchronize the timer to the clock every time we start it.
> > > */
> > > thread_group_cputime(tsk, &sum);
> > > + update_gt_cputime(cputimer, &sum);
> > > + /* Start 'running' after update_gt_cputime() */
> > > + smp_store_release(&cputimer->running, 1);
> >
> > This barrier should be mirrored somewhere but I can't see where in this patch.
> > Maybe in another one in the series. Or maybe there is already a barrier in the
> > existing code that I'm missing. I would expect to see it in account_group_*_time().
> > In any case, there should be comment about what it mirrors.
>
> I think it should be in cputimer_running(), which should use
> smp_load_acquire() to read cputimer->running.
>
> That way you guarantee that everything observing 'running' will indeed
> observe the initialized state.

So I intended the smp_store_release() here to be mainly for
documentation purposes, to say that we would like to set running after
the update.

With patch 3/3, even if running happens to get set earlier, the worst
case scenario is that update_gt_cputime may have to do go through some
retry logic. This isn't much of a performance issue in practice
(especially compared to adding smp_load_acquire() to hot paths), since
we only enter this path when we need to enable the timers.

In that case, I'm wondering if should just convert this back to
WRITE_ONCE(cputimer->running, 1) and avoid adding barriers to the hot
paths?

Thanks,
Jason