From: Sasha Kotchubievsky <sashakot at dev.mellanox.co.il>
To: spdk@lists.01.org
Subject: Re: [SPDK] RDMA QP leak on spdk nvmf target
Date: Mon, 17 Dec 2018 21:21:21 +0200 [thread overview]
Message-ID: <cf4b8a8c-0f4e-b2a7-27e5-e8f53b77f4ac@dev.mellanox.co.il> (raw)
In-Reply-To: 20181217190248.GA20210@chelsio.com
[-- Attachment #1: Type: text/plain, Size: 2384 bytes --]
Hi,
I believe,
https://github.com/spdk/spdk/commit/7da9f8faba8712e655fc75240b2a2dc6660e0e61
recently merged into "master" should fix the problem.
Best regards
Sasha
On 12/17/2018 9:02 PM, Potnuri Bharat Teja wrote:
> Hi All,
> With recent spdk code, RDMA QPs are not destroyed on the nvmf target after each
> spdk perf run until target is cleared/stopped.
>
> SPDK nvmf target is using drain WR completion logic to destroy the RDMA QP. But
> even after send/recv drain completions are received RDMA QP is not destroyed as
> the rqpair->refcnt is 0 in spdk_nvmf_rdma_qpair_destroy().
>
> I believe rqpair refcnt needs to be incremented before spdk_nvmf_rdma_qpair_destroy().
> Here is my experimental patch that fixes the issue, Please let me know if this
> qualifies for a patch.
> --- a/lib/nvmf/rdma.c
> +++ b/lib/nvmf/rdma.c
> @@ -2586,9 +2595,10 @@ spdk_nvmf_rdma_poller_poll(struct spdk_nvmf_rdma_transport *rtransport,
> case RDMA_WR_TYPE_DRAIN_RECV:
> rqpair = SPDK_CONTAINEROF(rdma_wr, struct spdk_nvmf_rdma_qpair, drain_recv_wr);
> assert(rqpair->disconnect_flags & RDMA_QP_DISCONNECTING);
> SPDK_DEBUGLOG(SPDK_LOG_RDMA, "Drained QP RECV %u (%p)\n", rqpair->qpair.qid, rqpair);
> rqpair->disconnect_flags |= RDMA_QP_RECV_DRAINED;
> if (rqpair->disconnect_flags & RDMA_QP_SEND_DRAINED) {
> + spdk_nvmf_rdma_qpair_inc_refcnt(rqpair);
> spdk_nvmf_rdma_qpair_destroy(rqpair);
> }
> /* Continue so that this does not trigger the disconnect path below. */
> @@ -2596,9 +2606,10 @@ spdk_nvmf_rdma_poller_poll(struct spdk_nvmf_rdma_transport *rtransport,
> case RDMA_WR_TYPE_DRAIN_SEND:
> rqpair = SPDK_CONTAINEROF(rdma_wr, struct spdk_nvmf_rdma_qpair, drain_send_wr);
> assert(rqpair->disconnect_flags & RDMA_QP_DISCONNECTING);
> SPDK_DEBUGLOG(SPDK_LOG_RDMA, "Drained QP SEND %u (%p)\n", rqpair->qpair.qid, rqpair);
> rqpair->disconnect_flags |= RDMA_QP_SEND_DRAINED;
> if (rqpair->disconnect_flags & RDMA_QP_RECV_DRAINED) {
> + spdk_nvmf_rdma_qpair_inc_refcnt(rqpair);
> spdk_nvmf_rdma_qpair_destroy(rqpair);
> }
> /* Continue so that this does not trigger the disconnect path below. */
> ----
>
> Thanks,
> Bharat.
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
next reply other threads:[~2018-12-17 19:21 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-17 19:21 Sasha Kotchubievsky [this message]
-- strict thread matches above, loose matches on Subject: below --
2018-12-17 20:04 [SPDK] RDMA QP leak on spdk nvmf target Potnuri Bharat Teja
2018-12-17 19:02 Potnuri Bharat Teja
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cf4b8a8c-0f4e-b2a7-27e5-e8f53b77f4ac@dev.mellanox.co.il \
--to=spdk@lists.01.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox