RE: Problem with RDMA device removal architecture

Sean Hefty
2010-03-26T17:08:12Z

>In other words, I think we want the ucma context to stay around until
>the application destroys it (via explicit means or via exit). But the
>rdma_cm_id gets destroyed immediately upon receiving a DEVICE_REMOVE event.

Yes - this is what I was trying to say.

The problem is trying to destroy the rdma_cm_id 'immediately' upon receiving a
device remove event. The user could be in a separate downcall into the rdma_cm
at the time the device removal occurs, and we cannot call rdma_destroy_id while
calling rdma_foo() for the same id. Plus, 'foo' may be 'destroy_id', and we
can't destroy the id twice.

For the trivial case, if the rdma_cm_id is not in use, we just return -1 to the
remove device callback to destroy the id. But if the rdma_cm_id is in use, then
we must schedule the destruction of the rdma_cm_id to a separate thread and
synchronize that thread against user space destroying the id before the thread
can run.