Resolving IP addr to GID with librdmacm

Resolving IP addr to GID with librdmacm

[Ramu]

Hi, 
I am trying to explore the details of librdmacm.  When I ran qperf app (with
librdmacm as underlying user space cm module) on my Infiniband setup, it all
went fine i.e conn establishment and data path.  In this case, I had ib_ipoib
module loaded on both the sides and I gave remote IP addr associated with net
interface (ib0/ib1 etc.) exposed by infiniband hw (from Mellanox).

One of the doc from Infiniband hw vendor says that we need to have ib_ipoib
module loaded at both sides in order to map IP addr to GID.  But I am trying to
understand the dependency on ib_ipoib module.  I ran the below experiment to
explore on this.  

I have disabled ib0/ib1 interface (described above) and gave the remote IP addr
as the one associated with eth0 (normal LAN interface) then qperf client failed
with error "unexpected event from RDMA CM: Address error".  

In the code, it looks like during resolving IP addr to GID, one of the kernel
module (ib_addr) is using neigh_lookup(..) to get the hardware address of the
given IP.  I did not get the dependency on the module ib_ipoib to resolve remote
IP addr to GID.  Am I missing anything here? Please explain.

Thanks & Regards,
Ramu    

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Resolving IP addr to GID with librdmacm

[Hefty, Sean]

> In the code, it looks like during resolving IP addr to GID, one of the kernel
> module (ib_addr) is using neigh_lookup(..) to get the hardware address of the
> given IP.  I did not get the dependency on the module ib_ipoib to resolve
> remote
> IP addr to GID.  Am I missing anything here? Please explain.

The IP address is mapped to a hardware address, typically done using ARP for IPv4, which goes over the ipoib device.  The returned hardware address must be in the format defined by the ipoib protocol.  That format includes the GID as part of the hardware address.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Resolving IP addr to GID with librdmacm

[Ramu]

Hi,
Does IPoIB module depend on Subnet manager/Subnet Administration components for
any service while resolving IP address to GID.  Is there any link available
which describes the architectural details of IPoIB module ?

Thanks & Regards,
Ramu

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Resolving IP addr to GID with librdmacm

[Hefty, Sean]

> Does IPoIB module depend on Subnet manager/Subnet Administration components
> for
> any service while resolving IP address to GID.

Yes - it relies on multicast groups having been setup and SA queries.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html