Re: [RESEND PATCH v5] locking/pvqspinlock: Relax cmpxchg's to improve performance on some archs

[Will Deacon <will.deacon@arm.com>]

On Mon, Aug 14, 2017 at 11:01:10AM -0400, Waiman Long wrote:
> On 08/14/2017 08:01 AM, Will Deacon wrote:
> > On Fri, Aug 11, 2017 at 11:06:01AM +0200, Peter Zijlstra wrote:
> >> On Thu, Aug 10, 2017 at 02:18:30PM -0400, Waiman Long wrote:
> >>> On 08/10/2017 12:22 PM, Waiman Long wrote:
> >>>> On 08/10/2017 12:15 PM, Peter Zijlstra wrote:
> >>>>> Might as well do an explicit:
> >>>>>
> >>>>> 	smp_mb__before_atomic()
> >>>>> 	cmpxchg_relaxed()
> >>>>> 	smp_mb__after_atomic()
> >>>>>
> >>>>> I suppose and not introduce new primitives.
> >>> I think we don't need smp_mb__after_atomic(). The read has to be fully
> >>> ordered, but the write part may not need it as the control dependency of
> >>> the old value should guard against incorrect action. Right?
> >> You'd think that, but IIRC there was something funny about using the SC
> >> return flag for control dependencies. Will?
> > Yeah, that's right, you can't use the STXR status flag to create control
> > dependencies.
> >
> > Will
> 
> Actually, the code sequence that I plan to use are:
> 
>         smp_mb__before_atomic();
>         if (cmpxchg_relaxed(&pn->state, vcpu_halted, vcpu_hashed)
>             != vcpu_halted)
>                 return;
> 
>         WRITE_ONCE(l->locked, _Q_SLOW_VAL);
>         (void)pv_hash(lock, pn);
> 
> I am planning to use the comparison of the returned value (pn->state)
> again vcpu_halted as the control dependency. I don't see how the status
> flag of STXR is affecting this.

Thanks for the context. I agree that you've got a control dependency in this
case, so the WRITE_ONCE will be ordered after the LL part of the cmpxchg. It
could still be reordered with respect to the write part, however.

Will