Re: rdma provider module references

Steve Wise
2010-12-16T15:48:32Z

However I guess NFS/RDMA is behind the RDMA CM, which is supposed to
> handle device removal. In that code it seems to end up in
> cma_process_remove(), which appears at first glance to do the right
> things to destroy all connections etc.
>

Function cma_process_remove() calls cma_remove_id_dev() for each cm_id bound to the device being removed. Function
cma_remove_id_dev() calls the event handler function for each cm_id and passes a RDMA_CM_EVENT_DEVICE_REMOVAL event.
The NFSRDMA server marks the RPC transport as XPT_CLOSE, but doesn't immediately destroy the cm_id in the event handler
function. This is in net/sunrpc/xprtrdma/svc_rdma_transport.c / rdma_cma_handler(). That's the issue methinks. Each
RDMA kernel user must destroy all the resources in the event handler function itself. These cannot be scheduled or
deferred in any way given the current design.