Distributed Replicated Block Device (DRBD) development
 help / color / mirror / Atom feed
From: "zhengbing.huang" <zhengbing.huang@easystack.cn>
To: drbd-dev@lists.linbit.com
Subject: [PATCH 10/11] drbd_transport_rdma: introduce timeout for rdma_connect
Date: Mon, 24 Jun 2024 13:46:18 +0800	[thread overview]
Message-ID: <20240624054619.23212-10-zhengbing.huang@easystack.cn> (raw)
In-Reply-To: <20240624054619.23212-1-zhengbing.huang@easystack.cn>

From: Dongsheng Yang <dongsheng.yang@easystack.cn>

Signed-off-by: Dongsheng Yang <dongsheng.yang@easystack.cn>
---
 drbd/drbd_transport_rdma.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/drbd/drbd_transport_rdma.c b/drbd/drbd_transport_rdma.c
index 811f1a20a..0cd639254 100644
--- a/drbd/drbd_transport_rdma.c
+++ b/drbd/drbd_transport_rdma.c
@@ -140,11 +140,13 @@ union dtr_immediate {
 
 enum dtr_state_bits {
 	DSB_CONNECT_REQ,
+	DSB_CONNECTING,
 	DSB_CONNECTED,
 	DSB_ERROR,
 };
 
 #define DSM_CONNECT_REQ   (1 << DSB_CONNECT_REQ)
+#define DSM_CONNECTING    (1 << DSB_CONNECTING)
 #define DSM_CONNECTED     (1 << DSB_CONNECTED)
 #define DSM_ERROR         (1 << DSB_ERROR)
 
@@ -1033,6 +1035,7 @@ static int dtr_cma_accept(struct dtr_listener *listener, struct rdma_cm_id *new_
 		return -EAGAIN;
 	}
 
+	set_bit(DSB_CONNECTING, &cm->state);
 	err = rdma_accept(new_cm_id, &dtr_conn_param);
 	if (err)
 		kref_put(&cm->kref, dtr_destroy_cm);
@@ -1163,6 +1166,7 @@ static void dtr_cma_connect_work_fn(struct work_struct *work)
 	}
 
 	kref_get(&cm->kref); /* Expecting RDMA_CM_EVENT_ESTABLISHED */
+	set_bit(DSB_CONNECTING, &cm->state);
 	err = rdma_connect(cm->id, &dtr_conn_param);
 	if (err) {
 		kref_put(&cm->kref, dtr_destroy_cm); /* no RDMA_CM_EVENT_ESTABLISHED */
@@ -1170,6 +1174,15 @@ static void dtr_cma_connect_work_fn(struct work_struct *work)
 		goto out;
 	}
 
+	err = wait_event_timeout(cm->state_wq,
+			   !test_bit(DSB_CONNECTING, &cm->state), 20*HZ);
+
+	if (err == 0 && test_and_clear_bit(DSB_CONNECTING, &cm->state)) {
+		kref_put(&cm->kref, dtr_destroy_cm);
+		tr_err(transport, "rdma_connect timeout\n");
+		goto out;
+	}
+
 	kref_put(&cm->kref, dtr_destroy_cm); /* for work */
 	return;
 out:
@@ -1293,6 +1306,9 @@ static int dtr_cma_event_handler(struct rdma_cm_id *cm_id, struct rdma_cm_event
 
 	case RDMA_CM_EVENT_ESTABLISHED:
 		// pr_info("%s: RDMA_CM_EVENT_ESTABLISHED\n", cm->name);
+		if (!test_and_clear_bit(DSB_CONNECTING, &cm->state))
+			return 0;
+		wake_up(&cm->state_wq);
 		/* cm->state = DSM_CONNECTED; is set later in the work item */
 		/* This is called for active and passive connections */
 
@@ -1313,6 +1329,8 @@ static int dtr_cma_event_handler(struct rdma_cm_id *cm_id, struct rdma_cm_event
 		// pr_info("%s: RDMA_CM_EVENT_REJECTED\n", cm->name);
 		// pr_info("event = %d, status = %d\n", event->event, event->status);
 		set_bit(DSB_ERROR, &cm->state);
+		if (!test_and_clear_bit(DSB_CONNECTING, &cm->state))
+			return 0;
 
 		dtr_cma_retry_connect(cm->path, cm);
 		break;
-- 
2.27.0


  parent reply	other threads:[~2024-06-24  7:27 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-06-24  5:46 [PATCH 01/11] drbd_nl: dont allow detating to be inttrupted in waiting D_DETACHING to DISKLESS zhengbing.huang
2024-06-24  5:46 ` [PATCH 02/11] drbd_receiver: get_ldev before use device->ldev for drbd_reconsider_queue_parameters() zhengbing.huang
2024-06-28  9:35   ` Philipp Reisner
2024-06-24  5:46 ` [PATCH 03/11] drbd_transport_rdma: put kref for cm in dtr_path_established in error path zhengbing.huang
2024-06-28  9:40   ` Philipp Reisner
2024-07-01  2:07     ` Dongsheng Yang
2024-07-01  2:48       ` Dongsheng Yang
2024-10-16 16:44         ` Philipp Reisner
2024-10-17  6:42           ` Zhengbing
2024-06-24  5:46 ` [PATCH 04/11] drbd_transport_rdma: dont schedule retry_connect_work in active is false zhengbing.huang
2024-06-28 11:51   ` Philipp Reisner
2024-07-01  2:11     ` Dongsheng Yang
2024-06-24  5:46 ` [PATCH 05/11] drbd_transport_rdma: dont break in dtr_tx_cq_event_handler if (cm->state != DSM_CONNECTED) zhengbing.huang
2024-06-28 12:07   ` Philipp Reisner
2024-07-01  2:23     ` Dongsheng Yang
2024-06-24  5:46 ` [PATCH 06/11] drbd_transport_rdma: put kref in error path zhengbing.huang
2024-06-28 12:12   ` Philipp Reisner
2024-06-24  5:46 ` [PATCH 07/11] drbd_transport_rdma: put kref in dtr_remap_tx_desc error zhengbing.huang
2024-06-28 12:19   ` Philipp Reisner
2024-07-01  2:28     ` Dongsheng Yang
2024-06-24  5:46 ` [PATCH 08/11] drbd_transport_rdma: fix a race between dtr_connect and drbd_thread_stop zhengbing.huang
2024-06-28 12:36   ` Philipp Reisner
2024-07-01  2:30     ` Dongsheng Yang
2024-06-24  5:46 ` [PATCH 09/11] drbd_transport_rdma: introduce timeout for rdma_disocnnect zhengbing.huang
2024-06-24  5:46 ` zhengbing.huang [this message]
2024-06-24  5:46 ` [PATCH 11/11] drbd_transport_rdma: wake up state_wq after clear DSB_CONNECTED in dtr_tx_timeout_work_fn zhengbing.huang
2024-06-28  9:10 ` [PATCH 01/11] drbd_nl: dont allow detating to be inttrupted in waiting D_DETACHING to DISKLESS Philipp Reisner
2024-07-01  2:02   ` Dongsheng Yang
2024-07-01 10:00     ` Philipp Reisner
2024-07-02  1:45       ` Dongsheng Yang
2024-07-03 14:31         ` [PATCH] drbd: make drbd_adm_detach() interruptible Philipp Reisner
2024-07-04  2:59           ` Zhengbing

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240624054619.23212-10-zhengbing.huang@easystack.cn \
    --to=zhengbing.huang@easystack.cn \
    --cc=drbd-dev@lists.linbit.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox