RE: rte_ring move head question for machines with relaxed MO (arm/ppc)

[Konstantin Ananyev <konstantin.ananyev@huawei.com>]

> > > > > 1. rte_ring_generic_pvt.h:
> > > > > =====================
> > > > >
> > > > > pseudo-c-code                                      //        related armv8 instructions
> > > > > --------------------                                                 --------------------------------------
> > > > >  head.load()                                          //        ldr [head]
> > > > >  rte_smp_rmb()                                    //        dmb ishld
> > > > >  opposite_tail.load()                            //        ldr [opposite_tail]
> > > > >  ...
> > > > >  rte_atomic32_cmpset(head, ...)      //        ldrex[head];... stlex[head]
> > > > >
> > > > >
> > > > > 2. rte_ring_c11_pvt.h
> > > > > =====================
> > > > >
> > > > > pseudo-c-code                                       //        related armv8 instructions
> > > > > --------------------                                                 --------------------------------------
> > > > > head.atomic_load(relaxed)                 //        ldr[head]
> > > > > atomic_thread_fence(acquire)           //        dmb ish
> > > > > opposite_tail.atomic_load(acquire)   //        lda[opposite_tail]
> > > > > ...
> > > > > head.atomic_cas(..., relaxed)              //        ldrex[haed]; ... strex[head]
> > > > >
> > > > >
> > > > > 3.   rte_ring_hts_elem_pvt.h
> > > > > ==========================
> > > > >
> > > > > pseudo-c-code                                       //        related armv8 instructions
> > > > > --------------------                                                 --------------------------------------
> > > > > head.atomic_load(acquire)                //        lda [head]
> > > > > opposite_tail.load()                             //        ldr [opposite_tail]
> > > > > ...
> > > > > head.atomic_cas(..., acquire)            //         ldaex[head]; ... strex[head]
> > > > >
> > > > > The questions that arose from these observations:
> > > > > a) are all 3 approaches equivalent in terms of functionality?
> > > > Different, lda (Load with acquire semantics) and ldr (load) are different.
> > >
> > > I understand that, my question was:
> > > lda {head]; ldr[tail]
> > > vs
> > > ldr [head]; dmb ishld; ldr [tail];
> > >
> > > Is there any difference in terms of functionality (memory ops
> > ordering/observability)?
> >
> > To be more precise:
> >
> > lda {head]; ldr[tail]
> > vs
> > ldr [head]; dmb ishld; ldr [tail];
> > vs
> > ldr [head]; dmb ishld; lda [tail];
> >
> > what would be the difference between these 3 cases?
> 
> Case A: lda {head]; ldr[tail]
> load of the head will be observed by the memory subsystem
> before the load of the tail.
> 
> Case B: ldr [head]; dmb ishld; ldr [tail];
> load of the head will be observed by the memory subsystem
> Before the load of the tail.
> 
> 
> Essentially both cases A and B are the same.
> They preserve following program orders.
> LOAD-LOAD
> LOAD-STORE

Ok, that is crystal clear, thanks for explanation.
 

> Case C: ldr [head]; dmb ishld; lda [tail];
> load of the head will be observed by the memory subsystem
> before the load of the tail. 

Ok.

> In addition, any load or store program
> order after lda[tail] will not be observed by the memory subsystem
> before the load of the tail. 

Ok... the question is why we need that extra hoisting barrier here?
From what unwanted  re-orderings we are protecting here?
Does it mean that without it, ldrex/strex (CAS) can be reordered with load[cons.tail]?

Actually, we probably need to look at whole picture:

in rte_ring_generic_pvt.h
=====================

ldr [prod.head]
dmb ishld
ldr [cons.tail]
...
/* cas */
ldrex [prod.head]
stlex [prod.head]   /* sink barrier */

in rte_ring_c11_pvt.h
=====================

ldr [prod.head]
dmb ishld
lda [cons.tail]          /* exrea hoist */
...
/* cas */
ldrex [prod.head]
strex [prod.head]  

So, in _genereic_ we don't have that extra hoist barrier after load[con.tail],
but we have extra sink barrier at cas(prod.tail).

If that's correct observation, can we change _c11_ implementation to match
_generic_ one by:

 atomic_load(prod.head, releaxed);
 atomic_thread_fence(acquire);
 atomic_load(cons.tail, releaxed);
....
atomic_cas(prod.head, release, relaxed);
?
  
From my understanding that should help to make these 2 implantations
Identical, and then hopefully we can get rid of rte_ring_generic_pvt.h.