[PATCH] IB/CMA: fail qkey setting for RDMA_PS_IPOIB and IB_LINK_LAYER_ETHERNET

[PATCH] IB/CMA: fail qkey setting for RDMA_PS_IPOIB and IB_LINK_LAYER_ETHERNET

[Moni Shoua]

From: Moni Shoua <monis-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

In general, when link layer is ETHERNET it is wrong to use IPoIB port space since
no IPoIB interface is available. Specifically, setting qkey when port space is
RDMA_PS_IPOIB, requires SA query which is impossible when link layer is IB_LINK_LAYER_ETHERNET.

Signed-off-by: Moni Shoua <monis-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/core/cma.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index b6a33b3..9bf83d5 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -314,6 +314,9 @@ static int cma_set_qkey(struct rdma_id_private *id_priv)
 		id_priv->qkey = RDMA_UDP_QKEY;
 		break;
 	case RDMA_PS_IPOIB:
+		if (rdma_port_get_link_layer(id_priv->cma_dev->device, id_priv->id.port_num) ==
+			IB_LINK_LAYER_ETHERNET)
+			return -EINVAL;
 		ib_addr_get_mgid(&id_priv->id.route.addr.dev_addr, &rec.mgid);
 		ret = ib_sa_get_mcmember_rec(id_priv->id.device,
 					     id_priv->id.port_num, &rec.mgid,
-- 
1.7.4.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH] IB/CMA: fail qkey setting for RDMA_PS_IPOIB and IB_LINK_LAYER_ETHERNET

[Hefty, Sean]

> In general, when link layer is ETHERNET it is wrong to use IPoIB port space
> since
> no IPoIB interface is available. Specifically, setting qkey when port space is
> RDMA_PS_IPOIB, requires SA query which is impossible when link layer is
> IB_LINK_LAYER_ETHERNET.

Can you describe the problem that the current code causes?  Does it lead to a kernel crash?

Maybe the issue is that port space ipoib should never be associated with a non-IB device, with a check added to cma_acquire_dev().

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] IB/CMA: fail qkey setting for RDMA_PS_IPOIB and IB_LINK_LAYER_ETHERNET

[Moni Shoua]

Yes, the kernel crashed after watchdog detected a lockup. This
happened while running udaddy -p 0x2 with RoCE. The crash isn't 100%
reproducible but I have a pair of nodes where that used to crash with
probability of 50%.
The fix you suggest seems to detect the mismatch between port space
and link layer earlier, so I guess it handles the bug better. I'll
rewrite, retest and send a fix on Sunday.

- monis

On Thu, Jul 7, 2011 at 7:55 PM, Hefty, Sean <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
>> In general, when link layer is ETHERNET it is wrong to use IPoIB port space
>> since
>> no IPoIB interface is available. Specifically, setting qkey when port space is
>> RDMA_PS_IPOIB, requires SA query which is impossible when link layer is
>> IB_LINK_LAYER_ETHERNET.
>
> Can you describe the problem that the current code causes?  Does it lead to a kernel crash?
>
> Maybe the issue is that port space ipoib should never be associated with a non-IB device, with a check added to cma_acquire_dev().
>
> - Sean
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] IB/CMA: fail qkey setting for RDMA_PS_IPOIB and IB_LINK_LAYER_ETHERNET

[Or Gerlitz]

>
> Can you describe the problem that the current code causes?  Does it lead to a kernel crash?
> Maybe the issue is that port space ipoib should never be associated with a non-IB device, with a check added to cma_acquire_dev().

Sean, Moni,

I believe that crashing the kernel from user space should be 
avoided/fixed anyway, is that
crash possible also under IB?

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html