Re: [PATCH RFC tip/core/rcu] rcu: direct algorithmic SRCU implementation

[Mathieu Desnoyers <mathieu.desnoyers@efficios.com>]

* Peter Zijlstra (peterz@infradead.org) wrote:
> On Thu, 2012-02-16 at 07:18 -0500, Mathieu Desnoyers wrote:
> > 
> > Hrm, I think we'd need a little more than just lock/unlock ordering
> > guarantees. Let's consider the following, where the stores would be
> > expected to be seen as "store A before store B" by CPU 2
> > 
> > CPU 0             CPU 1               CPU 2
> > 
> >                                       load B, smp_rmb, load A in loop,
> >                                       expecting that when updated A is
> >                                       observed, B is always observed as
> >                                       updated too.
> > store A
> > (lock is permeable:
> > outside can leak
> > inside)
> > lock(rq->lock)
> > 
> >       -> migration ->
> > 
> >                   unlock(rq->lock)
> >                   (lock is permeable:
> >                   outside can leak inside)
> >                   store B
> 
> You got the pairing the wrong way around, I suggested:
> 
>   store A
> 
>   switch-out
>     UNLOCK
> 
>   	-> migration ->
> 
> 			switch-in
> 			  LOCK
> 
> 			store B
> 
> While both LOCK and UNLOCK are semi-permeable, A won't pass the UNLOCK
> and B won't pass the LOCK.
> 
> Yes, A can pass switch-out LOCK, but that doesn't matter much since the
> switch-in cannot happen until we've passed UNLOCK.
> 
> And yes B can pass switch-in UNLOCK, but again, I can't see that being a
> problem since the LOCK will avoid it being visible before A.

Ah, so this is what I missed: the context switch has its lock/unlock
pair, the following migration is performed under its own lock/unlock
pair, and the following context switch also has its lock/unlock pair. So
yes, this should be sufficient to act as a full memory barrier.

> 
> > Does that make sense, or should I get my first morning coffee ? :) 
> 
> Probably.. but that's not saying I'm not wrong ;-)

It does pass my 1st morning coffee test still, so it looks good, at
least to me. :-)

Back to the initial subject: I think it would be important for general
code understanding that when RCU operates tricks on per-cpu variables
based on scheduler migration memory ordering assumption, that it tells
so explicitely, rather than claiming that the memory barriers match
those at RCU read lock/unlock sites, which is not quite right.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com