From mboxrd@z Thu Jan  1 00:00:00 1970
Content-Type: multipart/mixed; boundary="===============6920664723077128016=="
MIME-Version: 1.0
From: Walker, Benjamin <benjamin.walker at intel.com>
Subject: Re: [SPDK] #416879 qp recovery: outstanding requests
Date: Tue, 17 Jul 2018 23:10:20 +0000
Message-ID: <c51ea42ad6b67b45d93e8817616f65ad24a08f35.camel@intel.com>
In-Reply-To: DM6PR04MB49887F7606E0309BE1959E24865C0@DM6PR04MB4988.namprd04.prod.outlook.com
List-ID: <spdk@lists.01.org>
To: spdk@lists.01.org

--===============6920664723077128016==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable

On Tue, 2018-07-17 at 22:41 +0000, Philipp Skadorov wrote:
> Hi Benjamin,
>  =

> I have played with SoftRoCE and run through the real SNIC IB driver sourc=
es to
> see how it is possible to continue with the outstanding requests after th=
e QP
> is recovered.
> When the QP goes into an error state (async event: IB_EVENT_QP_FATAL) and
> drains the CQ, it sends responses back with the error code IB_WC_WR_FLUSH=
_ERR
> (5) which makes the outstanding requests in SPDK sort of invalidated. It =
looks
> to me that dropping those outstanding SPDK requests and freeing resources=
 the
> best way to go.

Ok - I assume there is some handling we need to implement on the initiator =
side
to deal with the IB_WC_WR_FLUSH_ERR responses. Maybe we should retry those =
on
the initiator side once the RDMA queue pair recovers? I just looked at the
specification and it doesn't have much to say on the issue beyond errors may
cause the RDMA QP to be terminated, and that the details were up to the spe=
cific
transport specification (Infiniband Verbs in this case).

I wonder if the other NVMe-oF initiator implementations attempt any sort of
error handling, or if they just terminate the QP.

Thanks,
Ben


--===============6920664723077128016==--