public inbox for linux-rdma@vger.kernel.org
 help / color / mirror / Atom feed
From: Dennis Dalessandro <dennis.dalessandro@intel.com>
To: dledford@redhat.com
Cc: linux-rdma@vger.kernel.org,
	Mike Marciniszyn <mike.marciniszyn@intel.com>,
	stable@vger.kernel.org, Kaike Wan <kaike.wan@intel.com>
Subject: [PATCH for-next 02/16] IB/{qib, hfi1}: Avoid flow control testing for RDMA write operation
Date: Mon, 21 Aug 2017 18:26:20 -0700	[thread overview]
Message-ID: <20170822012619.32701.53092.stgit@scvm10.sc.intel.com> (raw)
In-Reply-To: <20170822011657.32701.22207.stgit@scvm10.sc.intel.com>

From: Mike Marciniszyn <mike.marciniszyn@intel.com>

Section 9.7.7.2.5 of the 1.3 IBTA spec clearly says that receive
credits should never apply to RDMA write.

qib and hfi1 were doing that.  The following situation will result
in a QP hang:
- A prior SEND or RDMA_WRITE with immmediate consumed the last
  credit for a QP using RC receive buffer credits
- The prior op is acked so there are no more acks
- The peer ULP fails to post receive for some reason
- An RDMA write sees that the credits are exhausted and waits
- The peer ULP posts receive buffers
- The ULP posts a send or RDMA write that will be hung

The fix is to avoid the credit test for the RDMA write operation.

Cc: <stable@vger.kernel.org>
Reviewed-by: Kaike Wan <kaike.wan@intel.com>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
---
 drivers/infiniband/hw/hfi1/rc.c    |    3 ++-
 drivers/infiniband/hw/qib/qib_rc.c |    4 ++--
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/rc.c b/drivers/infiniband/hw/hfi1/rc.c
index 99defcc..e1cf0c0 100644
--- a/drivers/infiniband/hw/hfi1/rc.c
+++ b/drivers/infiniband/hw/hfi1/rc.c
@@ -442,7 +442,7 @@ int hfi1_make_rc_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps)
 		case IB_WR_RDMA_WRITE:
 			if (newreq && !(qp->s_flags & RVT_S_UNLIMITED_CREDIT))
 				qp->s_lsn++;
-			/* FALLTHROUGH */
+			goto no_flow_control;
 		case IB_WR_RDMA_WRITE_WITH_IMM:
 			/* If no credit, return. */
 			if (!(qp->s_flags & RVT_S_UNLIMITED_CREDIT) &&
@@ -450,6 +450,7 @@ int hfi1_make_rc_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps)
 				qp->s_flags |= RVT_S_WAIT_SSN_CREDIT;
 				goto bail;
 			}
+no_flow_control:
 			put_ib_reth_vaddr(
 				wqe->rdma_wr.remote_addr,
 				&ohdr->u.rc.reth);
diff --git a/drivers/infiniband/hw/qib/qib_rc.c b/drivers/infiniband/hw/qib/qib_rc.c
index 4ddbcac..e9a9173 100644
--- a/drivers/infiniband/hw/qib/qib_rc.c
+++ b/drivers/infiniband/hw/qib/qib_rc.c
@@ -348,7 +348,7 @@ int qib_make_rc_req(struct rvt_qp *qp, unsigned long *flags)
 		case IB_WR_RDMA_WRITE:
 			if (newreq && !(qp->s_flags & RVT_S_UNLIMITED_CREDIT))
 				qp->s_lsn++;
-			/* FALLTHROUGH */
+			goto no_flow_control;
 		case IB_WR_RDMA_WRITE_WITH_IMM:
 			/* If no credit, return. */
 			if (!(qp->s_flags & RVT_S_UNLIMITED_CREDIT) &&
@@ -356,7 +356,7 @@ int qib_make_rc_req(struct rvt_qp *qp, unsigned long *flags)
 				qp->s_flags |= RVT_S_WAIT_SSN_CREDIT;
 				goto bail;
 			}
-
+no_flow_control:
 			ohdr->u.rc.reth.vaddr =
 				cpu_to_be64(wqe->rdma_wr.remote_addr);
 			ohdr->u.rc.reth.rkey =

  reply	other threads:[~2017-08-22  1:26 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-08-22  1:26 [PATCH for-next 00/16] IB/hfi1, qib, rdmavt: patches for next 08/21/2017 Dennis Dalessandro
2017-08-22  1:26 ` Dennis Dalessandro [this message]
     [not found] ` <20170822011657.32701.22207.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
2017-08-22  1:26   ` [PATCH for-next 01/16] IB/rdmavt: Use rvt_put_swqe() in rvt_clear_mr_ref() Dennis Dalessandro
2017-08-22  1:26   ` [PATCH for-next 03/16] IB/qib: Remove unnecessary memory allocation for boardname Dennis Dalessandro
2017-08-22  1:26   ` [PATCH for-next 04/16] IB/qib: Stricter bounds checking for copy and array access Dennis Dalessandro
2017-08-22  1:26   ` [PATCH for-next 05/16] IB/hfi1: Ratelimit prints from sdma_interrupt Dennis Dalessandro
2017-08-22  1:26   ` [PATCH for-next 06/16] IB/hfi1: Improve local kmem_cache_alloc performance Dennis Dalessandro
2017-08-22  1:26   ` [PATCH for-next 07/16] IB/hfi1: Clean up hfi1_user_exp_rcv_setup function Dennis Dalessandro
2017-08-22  1:26   ` [PATCH for-next 08/16] IB/hfi1: Clean up user_sdma_send_pkts() function Dennis Dalessandro
2017-08-22  1:27   ` [PATCH for-next 09/16] IB/hfi1: Clean up pin_vector_pages() function Dennis Dalessandro
     [not found]     ` <20170822012702.32701.90032.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
2017-08-22 15:46       ` Leon Romanovsky
     [not found]         ` <20170822154659.GE1724-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2017-08-22 18:39           ` Harish Chegondi
     [not found]             ` <599C7A74.9010301-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-08-23  4:49               ` Leon Romanovsky
     [not found]                 ` <20170823044900.GK1724-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2017-08-28  5:00                   ` Harish Chegondi
2017-08-22  1:27   ` [PATCH for-next 10/16] IB/hfi1: Fix the bail out code in " Dennis Dalessandro
2017-08-22  1:27   ` [PATCH for-next 11/16] IB/hfi1: Remove duplicate definitions of num_user_pages() function Dennis Dalessandro
2017-08-22  1:27   ` [PATCH for-next 12/16] IB/hfi1: Move structure definitions from user_exp_rcv.c to user_exp_rcv.h Dennis Dalessandro
2017-08-22  1:27   ` [PATCH for-next 13/16] IB/hfi1: Move structure and MACRO definitions in user_sdma.c to user_sdma.h Dennis Dalessandro
     [not found]     ` <20170822012728.32701.38661.stgit-9QXIwq+3FY+1XWohqUldA0EOCMrvLtNR@public.gmane.org>
2017-08-22 15:40       ` Leon Romanovsky
     [not found]         ` <20170822154025.GD1724-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2017-08-28  5:04           ` Harish Chegondi
2017-08-22  1:27   ` [PATCH for-next 14/16] IB/hfi1: Fix whitespace alignment issue for MAD Dennis Dalessandro
2017-08-22  1:27   ` [PATCH for-next 15/16] IB/hfi1: Add received request info to qp_stats Dennis Dalessandro
2017-08-22  1:27   ` [PATCH for-next 16/16] IB/hfi1: Add opcode states " Dennis Dalessandro
2017-08-28 23:16   ` [PATCH for-next 00/16] IB/hfi1, qib, rdmavt: patches for next 08/21/2017 Doug Ledford

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170822012619.32701.53092.stgit@scvm10.sc.intel.com \
    --to=dennis.dalessandro@intel.com \
    --cc=dledford@redhat.com \
    --cc=kaike.wan@intel.com \
    --cc=linux-rdma@vger.kernel.org \
    --cc=mike.marciniszyn@intel.com \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox