Mellanox implementation for atomic operations

Mellanox implementation for atomic operations

[lihaidong]

Hi,
   I have a question about atomic operations.
   According to IB specification o10-48, all atomic operation request made to the same HCA, referencing the same physical memory are serialized with respect to each other.  I know this should be complied with if HCA supports atomic operations, right?
   According to  IB specification o10-49, all atomic operations requests that referencing the same physical memory are serialized with respect to each other. This means that atomic operations performed by processors should serialized with atomic operations performed by HCAs, too, if they were referencing the same physical memory.
  I want to know whether Mellanox implementation for atomic operations comply with o10-49 or not.
  if not ,to what extent it comply with the rule? 
  I also was intrested in how this rule is complied with by others vendors?

2010-05-10 
lihaidong 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mellanox implementation for atomic operations

[Dotan Barak]

Hi.

On 10/05/2010 08:42, lihaidong wrote:
> Hi,
>    I have a question about atomic operations.
>    According to IB specification o10-48, all atomic operation request made to the same HCA, referencing the same physical memory are serialized with respect to each other.  I know this should be complied with if HCA supports atomic operations, right?
>   
Right.
>    According to  IB specification o10-49, all atomic operations requests that referencing the same physical memory are serialized with respect to each other. This means that atomic operations performed by processors should serialized with atomic operations performed by HCAs, too, if they were referencing the same physical memory.
>   
So far so good.
>   I want to know whether Mellanox implementation for atomic operations comply with o10-49 or not.
>   if not ,to what extent it comply with the rule? 
>   I also was intrested in how this rule is complied with by others vendors?
>
>   
Let give you a general answer:
The struct ibv_device_attr contains the atomic_cap attribute, this
attribute defines the atomicity
level that the HCA support (None, only within the HCA, between all HCAs
(global)).

I think that your code should check this attribute
(This way your code will support all vendors HCAs).

As much as i know, atomic operations are only supported within one HCA.

I hope that this answer helped you ..
Dotan

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mellanox implementation for atomic operations

[Ralph Campbell]

The ib_qib driver supports atomic IB operations and
they are global since it does it in the host software
instead of PCIe bus transactions which don't have
global atomic support (yet).

On Tue, 2010-06-15 at 13:50 -0700, Dotan Barak wrote:
> Hi.
> 
> On 10/05/2010 08:42, lihaidong wrote:
> > Hi,
> >    I have a question about atomic operations.
> >    According to IB specification o10-48, all atomic operation request made to the same HCA, referencing the same physical memory are serialized with respect to each other.  I know this should be complied with if HCA supports atomic operations, right?
> >   
> Right.
> >    According to  IB specification o10-49, all atomic operations requests that referencing the same physical memory are serialized with respect to each other. This means that atomic operations performed by processors should serialized with atomic operations performed by HCAs, too, if they were referencing the same physical memory.
> >   
> So far so good.
> >   I want to know whether Mellanox implementation for atomic operations comply with o10-49 or not.
> >   if not ,to what extent it comply with the rule? 
> >   I also was intrested in how this rule is complied with by others vendors?
> >
> >   
> Let give you a general answer:
> The struct ibv_device_attr contains the atomic_cap attribute, this
> attribute defines the atomicity
> level that the HCA support (None, only within the HCA, between all HCAs
> (global)).
> 
> I think that your code should check this attribute
> (This way your code will support all vendors HCAs).
> 
> As much as i know, atomic operations are only supported within one HCA.
> 
> I hope that this answer helped you ..
> Dotan
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html