RE: [ewg] rdma/cm: disallow loopback address for iwarp devices

Sean Hefty
2010-02-08T23:48:16Z

>Sean, can you try openmpi? It fails for me, and yet ucmatose succeeds.
>I don't understand the difference yet...

I believe the issue is that rdma_bind_addr succeeds (returns 0), but no device
is assigned to the rdma_cm_id (verbs field is NULL).

This was a change from commit 6f8372b69c3198e06cecb1df2cb9682d0c55e657:

The defined behavior of rdma_bind_addr is to associate an RDMA
device with an rdma_cm_id, as long as the user specified a non-
zero address. (ie they weren't just trying to reserve a port)
Currently, if the loopback address is passed to rdma_bind_addr,
no device is associated with the rdma_cm_id. Fix this.

There are two places where rdma_bind_addr() is called in the openmpi source code
(based on a tarball download of the trunk). One is btl_openib_iwarp.c:

rc = rdma_bind_addr(cm_id, ipaddr);
if (rc || !cm_id->verbs) {
rc = OMPI_SUCCESS;
goto out3;
}

The other is btl_openib_connect_rdmacm.c, but that deals with listening. I
can't quickly determine if btl_openib_iwarp.c is usually used for IB or not.

So, to fully keep the behavior of 2.6.32, rdma_bind_addr for 127.0.0.1 should
succeed, but not assign a device. I think this was the change from commit
..c55e657 that changed the behavior:

@@ -2089,7 +2096,9 @@ int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr
*addr)
if (!cma_comp_exch(id_priv, CMA_IDLE, CMA_ADDR_BOUND))
return -EINVAL;

- if (!cma_any_addr(addr)) {
+ if (cma_loopback_addr(addr)) {
+ ret = cma_bind_loopback(id_priv);
+ } else if (!cma_zero_addr(addr)) {
ret = rdma_translate_ip(addr, &id->route.addr.dev_addr);
if (ret)
goto err1;

I'll see if reverting this gives the desired(?) behavior.

- Sean