Re: [PATCH] sched: Use READ_ONCE()/WRITE_ONCE() in task_cpu()/__set_task_cpu()

[Andrea Parri <andrea.parri@amarulasolutions.com>]

On Mon, Jan 21, 2019 at 01:25:26PM +0100, Peter Zijlstra wrote:
> On Mon, Jan 21, 2019 at 11:51:21AM +0100, Andrea Parri wrote:
> > On Wed, Jan 16, 2019 at 07:42:18PM +0100, Andrea Parri wrote:
> > > The smp_wmb() in move_queued_task() (c.f., __set_task_cpu()) pairs with
> > > the composition of the dependency and the ACQUIRE in task_rq_lock():
> > > 
> > > 	move_queued_task()		task_rq_lock()
> > > 
> > > 	[S] ->on_rq = MIGRATING		[L] rq = task_rq()
> > > 	WMB (__set_task_cpu())		ACQUIRE (rq->lock);
> > > 	[S] ->cpu = new_cpu		[L] ->on_rq
> > > 
> > > where "[L] rq = task_rq()" is ordered before "ACQUIRE (rq->lock)" by an
> > > address dependency and, in turn, "ACQUIRE (rq->lock)" is ordered before
> > > "[L] ->on_rq" by the ACQUIRE itself.
> > > 
> > > Use READ_ONCE() to load ->cpu in task_rq() (c.f., task_cpu()) to honour
> > > this address dependency between loads; also, mark the store to ->cpu in
> > > __set_task_cpu() by using WRITE_ONCE() in order to tell the compiler to
> > > not mess/tear this (synchronizing) memory access.
> > 
> > In the light of the recent discussion about the integration of plain
> > accesses in the LKMM (c.f., e.g., [1] and discussion thereof), I was
> > considering even further changes to this in order to "reinforce" the
> > above smp_wmb().  Here's two approaches (one of):
> > 
> >  1) replace this smp_wmb()+WRITE_ONCE() with an smp_store_release();
> > 
> >  2) or keep this smp_wmb()+WRITE_ONCE(), but use {WRITE,READ}_ONCE()
> >     also for the accesses to ->on_rq.
> 
> That should be the least painful I think. Note that we never store a
> value larger than a single byte in that word, so tearing shouldn't be a
> problem, but yes, that makes it all neat and tidy.

Thank you for the suggestion; I'll send the revisited patch shortly.

  Andrea