Re: [RFC PATCH 2/2] RDMA/rxe: Add RDMA Atomic Write operation

[Jason Gunthorpe <jgg@nvidia.com>]

On Fri, Jan 07, 2022 at 02:15:25AM +0000, yangx.jy@fujitsu.com wrote:
> On 2022/1/6 21:00, Jason Gunthorpe wrote:
> > On Thu, Jan 06, 2022 at 10:52:47AM +0000, yangx.jy@fujitsu.com wrote:
> >> On 2022/1/6 7:53, Jason Gunthorpe wrote:
> >>> On Thu, Dec 30, 2021 at 04:39:01PM -0500, Tom Talpey wrote:
> >>>
> >>>> Because RXE is a software provider, I believe the most natural approach
> >>>> here is to use an atomic64_set(dst, *src).
> >>> A smp_store_release() is most likely sufficient.
> >> Hi Jason, Tom
> >>
> >> Is smp_store_mb() better here? It calls WRITE_ONCE + smb_mb/barrier().
> >> I think the semantics of 'atomic write' is to do atomic write and make
> >> the 8-byte data reach the memory.
> > No, it is not 'data reach memory' it is a 'release' in that if the CPU
> > later does an 'acquire' on the written data it is guarenteed to see
> > all the preceeding writes.
> Hi Jason, Tom
> 
> Sorry for the wrong statement. I mean that the semantics of 'atomic 
> write' is to write an 8-byte value atomically and make the 8-byte value 
> visible for all CPUs.
> 'smp_store_release' makes all the preceding writes visible for all CPUs 
> before doing an atomic write. I think this guarantee should be done by 
> the preceding 'flush'.

That isn't what the spec says by my reading, and it would be a useless
primitive to allow the ATOMIC WRITE to become visible before any data
it might be guarding.

> 'smp_store_mb' does an atomic write and then makes the atomic write 
> visible for all CPUs. Subsequent 'flush' is only used to make the atomic 
> write persistent.

persistent is a different subject.

Jason