Re: [SPDK] #416879 qp recovery: outstanding requests

[Walker, Benjamin <benjamin.walker at intel.com>]

[-- Attachment #1: Type: text/plain, Size: 1205 bytes --]

On Tue, 2018-07-17 at 22:41 +0000, Philipp Skadorov wrote:
> Hi Benjamin,
>  
> I have played with SoftRoCE and run through the real SNIC IB driver sources to
> see how it is possible to continue with the outstanding requests after the QP
> is recovered.
> When the QP goes into an error state (async event: IB_EVENT_QP_FATAL) and
> drains the CQ, it sends responses back with the error code IB_WC_WR_FLUSH_ERR
> (5) which makes the outstanding requests in SPDK sort of invalidated. It looks
> to me that dropping those outstanding SPDK requests and freeing resources the
> best way to go.

Ok - I assume there is some handling we need to implement on the initiator side
to deal with the IB_WC_WR_FLUSH_ERR responses. Maybe we should retry those on
the initiator side once the RDMA queue pair recovers? I just looked at the
specification and it doesn't have much to say on the issue beyond errors may
cause the RDMA QP to be terminated, and that the details were up to the specific
transport specification (Infiniband Verbs in this case).

I wonder if the other NVMe-oF initiator implementations attempt any sort of
error handling, or if they just terminate the QP.

Thanks,
Ben