Linux RDMA and InfiniBand development
 help / color / mirror / Atom feed
From: Md Haris Iqbal <haris.iqbal@ionos.com>
To: linux-rdma@vger.kernel.org
Cc: leon@kernel.org, jgg@ziepe.ca, haris.iqbal@ionos.com,
	jinpu.wang@ionos.com,
	Grzegorz Prajsner <grzegorz.prajsner@ionos.com>
Subject: [PATCH v2 for-next 02/11] RDMA/rtrs-clt: Fix need_inv setting in error case
Date: Wed, 21 Aug 2024 13:22:08 +0200	[thread overview]
Message-ID: <20240821112217.41827-3-haris.iqbal@ionos.com> (raw)
In-Reply-To: <20240821112217.41827-1-haris.iqbal@ionos.com>

From: Jack Wang <jinpu.wang@ionos.com>

In some cases need_inv can be missed for write requests, additionally
driver has to handle missing invalidates for write requests. While at
it, remove the else case from write invalidate path as it is possible
to reach there.

Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Grzegorz Prajsner <grzegorz.prajsner@ionos.com>
---
 drivers/infiniband/ulp/rtrs/rtrs-clt.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
index 66ac4dba990f..d09018c11ece 100644
--- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c
+++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c
@@ -391,11 +391,12 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno,
 	clt_path = to_clt_path(con->c.path);
 
 	if (req->sg_cnt) {
-		if (req->dir == DMA_FROM_DEVICE && req->need_inv) {
+		if (req->need_inv) {
 			/*
-			 * We are here to invalidate read requests
+			 * We are here to invalidate read/write requests
 			 * ourselves.  In normal scenario server should
-			 * send INV for all read requests, but
+			 * send INV for all read requests, we do chained local
+			 * invalidate for write requests, but
 			 * we are here, thus two things could happen:
 			 *
 			 *    1.  this is failover, when errno != 0
@@ -422,14 +423,6 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno,
 					  req->mr->rkey, err);
 			} else if (can_wait) {
 				wait_for_completion(&req->inv_comp);
-			} else {
-				/*
-				 * Something went wrong, so request will be
-				 * completed from INV callback.
-				 */
-				WARN_ON_ONCE(1);
-
-				return;
 			}
 			if (!refcount_dec_and_test(&req->ref))
 				return;
@@ -1146,6 +1139,7 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req)
 		};
 		wr = &rwr.wr;
 		fr_en = true;
+		req->need_inv = true;
 		refcount_inc(&req->ref);
 	}
 	/*
@@ -1164,6 +1158,10 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req)
 			    clt_path->hca_port);
 		if (req->mp_policy == MP_POLICY_MIN_INFLIGHT)
 			atomic_dec(&clt_path->stats->inflight);
+		if (req->need_inv) {
+			req->need_inv = false;
+			refcount_dec(&req->ref);
+		}
 		if (req->sg_cnt)
 			ib_dma_unmap_sg(clt_path->s.dev->ib_dev, req->sglist,
 					req->sg_cnt, req->dir);
-- 
2.25.1


  parent reply	other threads:[~2024-08-21 11:23 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-08-21 11:22 [PATCH v2 for-next 00/11] Misc patches for RTRS Md Haris Iqbal
2024-08-21 11:22 ` [PATCH v2 for-next 01/11] RDMA/rtrs: For HB error add additional clt/srv specific logging Md Haris Iqbal
2024-08-21 11:22 ` Md Haris Iqbal [this message]
2024-08-21 11:22 ` [PATCH v2 for-next 03/11] RDMA/rtrs-clt: Rate limit errors in IO path Md Haris Iqbal
2024-08-21 11:22 ` [PATCH v2 for-next 04/11] RDMA/rtrs: Reset hb_missed_cnt after receiving other traffic from peer Md Haris Iqbal
2024-08-21 11:22 ` [PATCH v2 for-next 05/11] RDMA/rtrs-clt: Reuse need_inval from mr Md Haris Iqbal
2024-08-21 11:22 ` [PATCH v2 for-next 06/11] RDMA/rtrs-clt: Reset cid to con_num - 1 to stay in bounds Md Haris Iqbal
2024-08-21 11:22 ` [PATCH v2 for-next 07/11] RDMA/rtrs-clt: Print request type for errors Md Haris Iqbal
2024-08-21 11:22 ` [PATCH v2 for-next 08/11] RDMA/rtrs-srv: Avoid null pointer deref during path establishment Md Haris Iqbal
2024-08-21 11:22 ` [PATCH v2 for-next 09/11] RDMA/rtrs: register ib event handler Md Haris Iqbal
2024-08-21 11:22 ` [PATCH v2 for-next 10/11] RDMA/rtrs-clt: Do local invalidate after write io completion Md Haris Iqbal
2024-08-21 11:22 ` [PATCH v2 for-next 11/11] RDMA/rtrs-clt: Remove an extra space Md Haris Iqbal
2024-08-28 11:00 ` [PATCH v2 for-next 00/11] Misc patches for RTRS Haris Iqbal
2024-08-28 11:05 ` Leon Romanovsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240821112217.41827-3-haris.iqbal@ionos.com \
    --to=haris.iqbal@ionos.com \
    --cc=grzegorz.prajsner@ionos.com \
    --cc=jgg@ziepe.ca \
    --cc=jinpu.wang@ionos.com \
    --cc=leon@kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox