From: Namjae Jeon <linkinjeon@kernel.org>
To: linux-cifs@vger.kernel.org
Cc: smfrench@gmail.com, Namjae Jeon <linkinjeon@kernel.org>,
Tom Talpey <tom@talpey.com>, David Howells <dhowells@redhat.com>,
Hyunchul Lee <hyc.lee@gmail.com>, Long Li <longli@microsoft.com>
Subject: [PATCH] cifs: smbdirect: use the max_sge the hw reports
Date: Wed, 3 Aug 2022 11:20:42 +0900 [thread overview]
Message-ID: <20220803022042.10543-1-linkinjeon@kernel.org> (raw)
In Soft-iWARP, smbdirect does not work in cifs client.
The hardcoding max_sge is large in cifs, but need smaller value for
soft-iWARP. Add SMBDIRECT_MIN_SGE macro as 6 and use the max_sge
the hw reports instead of hardcoding 16 sge's.
Cc: Tom Talpey <tom@talpey.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Hyunchul Lee <hyc.lee@gmail.com>
Cc: Long Li <longli@microsoft.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
---
fs/cifs/smbdirect.c | 15 ++++++++++-----
fs/cifs/smbdirect.h | 3 ++-
2 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c
index 5fbbec22bcc8..bb68702362f7 100644
--- a/fs/cifs/smbdirect.c
+++ b/fs/cifs/smbdirect.c
@@ -1518,7 +1518,7 @@ static int allocate_caches_and_workqueue(struct smbd_connection *info)
static struct smbd_connection *_smbd_get_connection(
struct TCP_Server_Info *server, struct sockaddr *dstaddr, int port)
{
- int rc;
+ int rc, max_sge;
struct smbd_connection *info;
struct rdma_conn_param conn_param;
struct ib_qp_init_attr qp_attr;
@@ -1562,13 +1562,13 @@ static struct smbd_connection *_smbd_get_connection(
info->max_receive_size = smbd_max_receive_size;
info->keep_alive_interval = smbd_keep_alive_interval;
- if (info->id->device->attrs.max_send_sge < SMBDIRECT_MAX_SGE) {
+ if (info->id->device->attrs.max_send_sge < SMBDIRECT_MIN_SGE) {
log_rdma_event(ERR,
"warning: device max_send_sge = %d too small\n",
info->id->device->attrs.max_send_sge);
log_rdma_event(ERR, "Queue Pair creation may fail\n");
}
- if (info->id->device->attrs.max_recv_sge < SMBDIRECT_MAX_SGE) {
+ if (info->id->device->attrs.max_recv_sge < SMBDIRECT_MIN_SGE) {
log_rdma_event(ERR,
"warning: device max_recv_sge = %d too small\n",
info->id->device->attrs.max_recv_sge);
@@ -1593,13 +1593,18 @@ static struct smbd_connection *_smbd_get_connection(
goto alloc_cq_failed;
}
+ max_sge = min3(info->id->device->attrs.max_send_sge,
+ info->id->device->attrs.max_recv_sge,
+ SMBDIRECT_MAX_SGE);
+ max_sge = max(max_sge, SMBDIRECT_MIN_SGE);
+
memset(&qp_attr, 0, sizeof(qp_attr));
qp_attr.event_handler = smbd_qp_async_error_upcall;
qp_attr.qp_context = info;
qp_attr.cap.max_send_wr = info->send_credit_target;
qp_attr.cap.max_recv_wr = info->receive_credit_max;
- qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SGE;
- qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_SGE;
+ qp_attr.cap.max_send_sge = max_sge;
+ qp_attr.cap.max_recv_sge = max_sge;
qp_attr.cap.max_inline_data = 0;
qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
qp_attr.qp_type = IB_QPT_RC;
diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h
index a87fca82a796..8b81301e4d4c 100644
--- a/fs/cifs/smbdirect.h
+++ b/fs/cifs/smbdirect.h
@@ -225,7 +225,8 @@ struct smbd_buffer_descriptor_v1 {
__le32 length;
} __packed;
-/* Default maximum number of SGEs in a RDMA send/recv */
+/* Default maximum/minimum number of SGEs in a RDMA send/recv */
+#define SMBDIRECT_MIN_SGE 6
#define SMBDIRECT_MAX_SGE 16
/* The context for a SMBD request */
struct smbd_request {
--
2.25.1
next reply other threads:[~2022-08-03 2:21 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-03 2:20 Namjae Jeon [this message]
2022-08-03 14:26 ` [PATCH] cifs: smbdirect: use the max_sge the hw reports Tom Talpey
2022-08-03 14:34 ` Tom Talpey
2022-08-04 5:58 ` Namjae Jeon
2022-08-04 12:58 ` Tom Talpey
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220803022042.10543-1-linkinjeon@kernel.org \
--to=linkinjeon@kernel.org \
--cc=dhowells@redhat.com \
--cc=hyc.lee@gmail.com \
--cc=linux-cifs@vger.kernel.org \
--cc=longli@microsoft.com \
--cc=smfrench@gmail.com \
--cc=tom@talpey.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox