Re: [PATCH 2/2] [PATCH] sched: Add smp_rmb() in task rq locking cycles

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Tue, Feb 17, 2015 at 07:36:36PM +0100, Peter Zijlstra wrote:
> On Tue, Feb 17, 2015 at 08:05:32AM -0800, Paul E. McKenney wrote:
> > > FWIW, we should probably update that table to include control
> > > dependencies too; we didn't (formally) have those back then I think.
> > > 
> > > The blob under SMP BARRIER PAIRING does not mention pairing with control
> > > dependencies; and I'm rather sure I've done so.
> > 
> > Yep, they should pair as well, though the pairing is limited.
> > No transitivity, of course.
> > 
> > So the straightforward approach requires eighteen bits per cell, though
> > some of them are a bit, ummm, "unusual".  
> 
> Right, I think the idea was to not mark with 'X' when very unusual,
> otherwise you do indeed obtain the below 'trivial' matrix.
> 
> > Sixteen of these are given by
> > Scenarios 0-15 in http://lwn.net/Articles/573436/, with the barrier on
> > the side corresponding to the first column and the barrier on the top
> > corresponding to the second column.  The seventeenth bit says whether
> > you get transitivity chaining after the top access, assuming that it
> > happens later.  The eighteenth bit says whether you get transitivity
> > chaining before the side access, assuming that it happens earlier.
> > 
> > The following is a rough first guess, filling in only the diagonal.
> > Some of the entries are no doubt wrong, and getting them right requires
> > something like 7*7*18 test cases, which will take some time.  So, is
> > something like this really helpful?
> 
> 
> >       |   mb  |  wmb  |  rmb  |  rbd  |  acq  |  rel  |  ctl  |
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >    mb | 3ffff |   X   |   X   |   X   |   X   |   X   |   X   +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   wmb |   X   | 01000 |   X   |   X   |   X   |   X   |   X   +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   rmb |   X   |   X   | 00000 |   X   |   X   |   X   |   X   +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   rbd |   X   |   X   |   X   | 00000 |   X   |   X   |   X   +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   acq |   X   |   X   |   X   |   X   | 00020 |   X   |   X   +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   rel |   X   |   X   |   X   |   X   |   X   | 0cc00 |   X   +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> >   ctl |   X   |   X   |   X   |   X   |   X   |   X   | 00020 +
> >  -----+-------+-------+-------+-------+-------+-------+-------+
> 
> So maybe make two tables; one with 'obvious' pairings, which would
> include things like mb - {mb,rmb,wmb}; rmb-wmb; acq-rel; ctl-mb; etc.
> 
> That table is for people to quickly check 'easy'; like yes wmb-rbd makes
> sense and rmb-rbd doesn't appear to make sense, I need more reading up.
> 
> After that do the 'funny' table, which will explain further possible
> pairings in more detail, like the rmb-rbd pairing.
> 
> I'm not entirely sure we want to do the 7*7*18 state table, that's a lot
> of work to exhaustively generate. We should be lazy and demand fill when
> people come to us.

I could do a table per communication style.  For example, message
passing looks like this (give or take likely errors in the table):

	Side CPU	Top CPU
	--------	-------
	X = 1;		r1 = Y;
	<some barrier>	<some barrier>
	Y = 1;		r2 = X;

	assert(r1 == 0 || r2 == 1);


      |   mb  |  wmb  |  rmb  |  rbd  |  acq  |  rel  |  ctl  |
 -----+-------+-------+-------+-------+-------+-------+-------+
   mb |   Y   |       |   Y   |   y   |   Y   |       |   Y   +
 -----+-------+-------+-------+-------+-------+-------+-------+
  wmb |   Y   |       |   Y   |   y   |   Y   |       |   Y   +
 -----+-------+-------+-------+-------+-------+-------+-------+
  rmb |       |       |       |       |       |       |       +
 -----+-------+-------+-------+-------+-------+-------+-------+
  rbd |       |       |       |       |       |       |       +
 -----+-------+-------+-------+-------+-------+-------+-------+
  acq |       |       |       |       |       |       |       +
 -----+-------+-------+-------+-------+-------+-------+-------+
  rel |   Y   |       |   Y   |   y   |   Y   |       |   Y   +
 -----+-------+-------+-------+-------+-------+-------+-------+
  ctl |       |       |       |       |       |       |       +
 -----+-------+-------+-------+-------+-------+-------+-------+

Here "Y" says that the barrier pair works, "y" says that it can
work but requires an artificial dependency, and " " says that
it does not work.

							Thanx, Paul