Re: RDMA and memory ordering

[Gabriele Svelto <gabriele.svelto-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

  Hi Anuj,

On 10/11/2013 11:46, Anuj Kalia wrote:
> How can this happen in the presence of memory barriers? With barriers,
> A[i].counter should be updated later and therefore should always be
> smaller than A[i].value.

memory barriers such as mfence synchronize memory operations from the 
point of view of CPUs only. Practically this means that the stores you 
wrote might go out to memory in a different order than what the 
processor sees and external devices such as a PCIe HCAs might thus see a 
different ordering even in the presence of memory barriers.

To ensure that an external devices sees your store in the order you 
meant you will need some form of external barrier though I do not know 
if it is possible at all in userspace and besides it will be a fragile 
solution.

Instead I would suggest you to use verbs atomic operations such as 
IBV_WR_ATOMIC_CMP_AND_SWP and IBV_WR_ATOMIC_FETCH_AND_ADD to implement 
what you have in mind.

  Gabriele
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html