Re: how to re-use a QP for a new connection

[Shirley Ma <shirley.ma-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>]

On 06/20/2014 03:30 PM, Chuck Lever wrote:
> Hi Shirley-
> 
> I’ve found that to move the QP back to the IB_QPS_INIT state, I need to
> call ib_modify_qp() with a specific set of attributes, including the
> pkey_index and port_num.
> 
> rdma_init_qp_attr() extracts those attributes. But, when I try to call it
> after rdma_disconnect(), the rdma_cm_id is not in the RDMA_CM_IDLE state,
> and the call fails.
> 
> So I can’t get the QP back to the INIT state unless the rdma_cm_id has
> somehow been reset.

I see, we need to have rdma_reset_id() to change the cm_id state to RDMA_CM_IDLE.
 
> I suppose I could call rdma_init_qp_attr() while the transport is still
> connected, and save the returned attributes.
Maybe we can save ib_qp_attr in xprtrdma rpcrdma_ia?

> On Jun 20, 2014, at 6:24 PM, Shirley Ma <shirley.ma-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> wrote:
> 
>> The QP can be reused. The rdma_id_private has a field reuseaddr. What additional change is needed besides rdma_set_reuseaddr?
>>
>> Shirley
>>
>> On 06/20/2014 02:17 PM, Hefty, Sean wrote:
>>>> During a remote transport disconnect, the QP leaves RTS.
>>>>
>>>> xprtrdma deals with this in a separate transport connect worker process,
>>>> where it creates a new id and qp, and replaces the existing id and qp.
>>>>
>>>> Unfortunately there are parts of xprtrdma (namely FRMR deregistration)
>>>> that are not easy to serialize with this reconnect logic.
>>>>
>>>> Re-using the QP would mean no serialization would be needed between
>>>> transport reconnect and FRMR deregistration.
>>>>
>>>> If QP re-use is not supported, though, it's not worth considering any
>>>> further.
>>>
>>> It may be possible to reuse the QP, just not the rdma_cm_id without additional code changes.  Reuse of the rdma_cm_id may also require changes in the underlying IB/iWarp CMs.
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
> 
> --
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html