Linux CIFS filesystem development
 help / color / mirror / Atom feed
From: Namjae Jeon <linkinjeon@kernel.org>
To: linux-cifs@vger.kernel.org
Cc: smfrench@gmail.com, Namjae Jeon <linkinjeon@kernel.org>,
	Tom Talpey <tom@talpey.com>, David Howells <dhowells@redhat.com>,
	Hyunchul Lee <hyc.lee@gmail.com>, Long Li <longli@microsoft.com>
Subject: [PATCH] cifs: smbdirect: use the max_sge the hw reports
Date: Wed,  3 Aug 2022 11:20:42 +0900	[thread overview]
Message-ID: <20220803022042.10543-1-linkinjeon@kernel.org> (raw)

In Soft-iWARP, smbdirect does not work in cifs client.
The hardcoding max_sge is large in cifs, but need smaller value for
soft-iWARP. Add SMBDIRECT_MIN_SGE macro as 6 and use the max_sge
the hw reports instead of hardcoding 16 sge's.

Cc: Tom Talpey <tom@talpey.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Hyunchul Lee <hyc.lee@gmail.com>
Cc: Long Li <longli@microsoft.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
---
 fs/cifs/smbdirect.c | 15 ++++++++++-----
 fs/cifs/smbdirect.h |  3 ++-
 2 files changed, 12 insertions(+), 6 deletions(-)

diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index 5fbbec22bcc8..bb68702362f7 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -1518,7 +1518,7 @@ static int allocate_caches_and_workqueue(struct smbd_connection *info)
 static struct smbd_connection *_smbd_get_connection(
 	struct TCP_Server_Info *server, struct sockaddr *dstaddr, int port)
 {
-	int rc;
+	int rc, max_sge;
 	struct smbd_connection *info;
 	struct rdma_conn_param conn_param;
 	struct ib_qp_init_attr qp_attr;
@@ -1562,13 +1562,13 @@ static struct smbd_connection *_smbd_get_connection(
 	info->max_receive_size = smbd_max_receive_size;
 	info->keep_alive_interval = smbd_keep_alive_interval;
 
-	if (info->id->device->attrs.max_send_sge < SMBDIRECT_MAX_SGE) {
+	if (info->id->device->attrs.max_send_sge < SMBDIRECT_MIN_SGE) {
 		log_rdma_event(ERR,
 			"warning: device max_send_sge = %d too small\n",
 			info->id->device->attrs.max_send_sge);
 		log_rdma_event(ERR, "Queue Pair creation may fail\n");
 	}
-	if (info->id->device->attrs.max_recv_sge < SMBDIRECT_MAX_SGE) {
+	if (info->id->device->attrs.max_recv_sge < SMBDIRECT_MIN_SGE) {
 		log_rdma_event(ERR,
 			"warning: device max_recv_sge = %d too small\n",
 			info->id->device->attrs.max_recv_sge);
@@ -1593,13 +1593,18 @@ static struct smbd_connection *_smbd_get_connection(
 		goto alloc_cq_failed;
 	}
 
+	max_sge = min3(info->id->device->attrs.max_send_sge,
+		       info->id->device->attrs.max_recv_sge,
+		       SMBDIRECT_MAX_SGE);
+	max_sge = max(max_sge, SMBDIRECT_MIN_SGE);
+
 	memset(&qp_attr, 0, sizeof(qp_attr));
 	qp_attr.event_handler = smbd_qp_async_error_upcall;
 	qp_attr.qp_context = info;
 	qp_attr.cap.max_send_wr = info->send_credit_target;
 	qp_attr.cap.max_recv_wr = info->receive_credit_max;
-	qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SGE;
-	qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_SGE;
+	qp_attr.cap.max_send_sge = max_sge;
+	qp_attr.cap.max_recv_sge = max_sge;
 	qp_attr.cap.max_inline_data = 0;
 	qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
 	qp_attr.qp_type = IB_QPT_RC;
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
index a87fca82a796..8b81301e4d4c 100644
--- a/fs/cifs/smbdirect.h
+++ b/fs/cifs/smbdirect.h
@@ -225,7 +225,8 @@ struct smbd_buffer_descriptor_v1 {
 	__le32 length;
 } __packed;
 
-/* Default maximum number of SGEs in a RDMA send/recv */
+/* Default maximum/minimum number of SGEs in a RDMA send/recv */
+#define SMBDIRECT_MIN_SGE	6
 #define SMBDIRECT_MAX_SGE	16
 /* The context for a SMBD request */
 struct smbd_request {
-- 
2.25.1


             reply	other threads:[~2022-08-03  2:21 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-03  2:20 Namjae Jeon [this message]
2022-08-03 14:26 ` [PATCH] cifs: smbdirect: use the max_sge the hw reports Tom Talpey
2022-08-03 14:34   ` Tom Talpey
2022-08-04  5:58     ` Namjae Jeon
2022-08-04 12:58       ` Tom Talpey

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220803022042.10543-1-linkinjeon@kernel.org \
    --to=linkinjeon@kernel.org \
    --cc=dhowells@redhat.com \
    --cc=hyc.lee@gmail.com \
    --cc=linux-cifs@vger.kernel.org \
    --cc=longli@microsoft.com \
    --cc=smfrench@gmail.com \
    --cc=tom@talpey.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox