Re: Problem with RDMA device removal architecture

[Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>]

Sean Hefty wrote:
>> In other words, I think we want the ucma context to stay around until
>> the application destroys it (via explicit means or via exit).   But the
>> rdma_cm_id gets destroyed immediately upon receiving a DEVICE_REMOVE event.
>>     
>
> Yes - this is what I was trying to say.
>
> The problem is trying to destroy the rdma_cm_id 'immediately' upon receiving a
> device remove event.  The user could be in a separate downcall into the rdma_cm
> at the time the device removal occurs, and we cannot call rdma_destroy_id while
> calling rdma_foo() for the same id.  Plus, 'foo' may be 'destroy_id', and we
> can't destroy the id twice.
>
> For the trivial case, if the rdma_cm_id is not in use, we just return -1 to the
> remove device callback to destroy the id.  But if the rdma_cm_id is in use, then
> we must schedule the destruction of the rdma_cm_id to a separate thread and
> synchronize that thread against user space destroying the id before the thread
> can run.
>
>   

I think you don't want to schedule another thread.  The semantics of the 
device removal and eeh events is that when the driver returns, all usage 
of the device is stopped and it can be removed, reset, etc.  Because of 
this, these events are run on a thread so they can block if needed.

Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html