* [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io
@ 2025-08-06 17:35 Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 01/18] smb: client: remove unused enum smbd_connection_status Stefan Metzmacher
` (20 more replies)
0 siblings, 21 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
Hi,
this is the next step towards a common smbdirect layer
between cifs.ko and ksmbd.ko, with the aim to provide
a socket layer for userspace usage at the end of the road.
This patchset focuses on the usage of a common
smbdirect_send_io and related structures in smbdirect_socket.send_io.
Note only patches 01-08 are intended to be merged soon,
while the ksmbd patches 09-18 are only posted for
completeness (as discussed with Namjae) to get early feedback.
I used the following xfstests as regression tests:
cifs/001 generic/001 generic/002 generic/005 generic/006 generic/007 generic/010 generic/011
Between cifs.ko against ksmbd.ko via siw.ko in all combinations
with and without the patchset on each side.
Stefan Metzmacher (18):
smb: client: remove unused enum smbd_connection_status
smb: smbdirect: add SMBDIRECT_RECV_IO_MAX_SGE
smb: client: make use of SMBDIRECT_RECV_IO_MAX_SGE
smb: smbdirect: introduce struct smbdirect_send_io
smb: client: make use of struct smbdirect_send_io
smb: smbdirect: add smbdirect_socket.{send,recv}_io.mem.{cache,pool}
smb: client: make use of
smbdirect_socket.{send,recv}_io.mem.{cache,pool}
smb: server: make use of common smbdirect_pdu.h
smb: server: make use of common smbdirect.h
smb: server: make use of common smbdirect_socket
smb: server: make use of common smbdirect_socket_parameters
smb: server: make use of smbdirect_socket->recv_io.expected
smb: server: make use of struct smbdirect_recv_io
smb: server: make use of smbdirect_socket.recv_io.free.{list,lock}
smb: server: make use of smbdirect_socket.recv_io.reassembly.*
smb: server: make use of SMBDIRECT_RECV_IO_MAX_SGE
smb: server: make use of struct smbdirect_send_io
smb: server: make use of
smbdirect_socket.{send,recv}_io.mem.{cache,pool}
fs/smb/client/smbdirect.c | 112 ++--
fs/smb/client/smbdirect.h | 38 --
fs/smb/common/smbdirect/smbdirect_socket.h | 54 ++
fs/smb/server/connection.c | 4 +-
fs/smb/server/connection.h | 10 +-
fs/smb/server/smb2pdu.c | 11 +-
fs/smb/server/smb2pdu.h | 6 -
fs/smb/server/transport_rdma.c | 742 +++++++++++----------
fs/smb/server/transport_rdma.h | 41 --
9 files changed, 500 insertions(+), 518 deletions(-)
--
2.43.0
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH 01/18] smb: client: remove unused enum smbd_connection_status
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 02/18] smb: smbdirect: add SMBDIRECT_RECV_IO_MAX_SGE Stefan Metzmacher
` (19 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Steve French, Tom Talpey, Long Li
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.h | 10 ----------
1 file changed, 10 deletions(-)
diff --git a/fs/smb/client/smbdirect.h b/fs/smb/client/smbdirect.h
index 9df434f6bb8c..0463fde1bf26 100644
--- a/fs/smb/client/smbdirect.h
+++ b/fs/smb/client/smbdirect.h
@@ -33,16 +33,6 @@ enum keep_alive_status {
KEEP_ALIVE_SENT,
};
-enum smbd_connection_status {
- SMBD_CREATED,
- SMBD_CONNECTING,
- SMBD_CONNECTED,
- SMBD_NEGOTIATE_FAILED,
- SMBD_DISCONNECTING,
- SMBD_DISCONNECTED,
- SMBD_DESTROYED
-};
-
/*
* The context for the SMBDirect transport
* Everything related to the transport is here. It has several logical parts
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 02/18] smb: smbdirect: add SMBDIRECT_RECV_IO_MAX_SGE
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 01/18] smb: client: remove unused enum smbd_connection_status Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 03/18] smb: client: make use of SMBDIRECT_RECV_IO_MAX_SGE Stefan Metzmacher
` (18 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
This will allow the client and server specific defines to be
replaced.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/common/smbdirect/smbdirect_socket.h | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/fs/smb/common/smbdirect/smbdirect_socket.h b/fs/smb/common/smbdirect/smbdirect_socket.h
index 3ae834ca3af1..7270fcee1048 100644
--- a/fs/smb/common/smbdirect/smbdirect_socket.h
+++ b/fs/smb/common/smbdirect/smbdirect_socket.h
@@ -92,6 +92,13 @@ struct smbdirect_socket {
struct smbdirect_recv_io {
struct smbdirect_socket *socket;
struct ib_cqe cqe;
+
+ /*
+ * For now we only use a single SGE
+ * as we have just one large buffer
+ * per posted recv.
+ */
+#define SMBDIRECT_RECV_IO_MAX_SGE 1
struct ib_sge sge;
/* Link to free or reassembly list */
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 03/18] smb: client: make use of SMBDIRECT_RECV_IO_MAX_SGE
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 01/18] smb: client: remove unused enum smbd_connection_status Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 02/18] smb: smbdirect: add SMBDIRECT_RECV_IO_MAX_SGE Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 04/18] smb: smbdirect: introduce struct smbdirect_send_io Stefan Metzmacher
` (17 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Steve French, Tom Talpey, Long Li
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 4 ++--
fs/smb/client/smbdirect.h | 3 ---
2 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index 5217a8122a94..5d1fa83583f6 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -1563,7 +1563,7 @@ static struct smbd_connection *_smbd_get_connection(
sp->keepalive_interval_msec = smbd_keep_alive_interval * 1000;
if (sc->ib.dev->attrs.max_send_sge < SMBDIRECT_MAX_SEND_SGE ||
- sc->ib.dev->attrs.max_recv_sge < SMBDIRECT_MAX_RECV_SGE) {
+ sc->ib.dev->attrs.max_recv_sge < SMBDIRECT_RECV_IO_MAX_SGE) {
log_rdma_event(ERR,
"device %.*s max_send_sge/max_recv_sge = %d/%d too small\n",
IB_DEVICE_NAME_MAX,
@@ -1595,7 +1595,7 @@ static struct smbd_connection *_smbd_get_connection(
qp_attr.cap.max_send_wr = sp->send_credit_target;
qp_attr.cap.max_recv_wr = sp->recv_credit_max;
qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SEND_SGE;
- qp_attr.cap.max_recv_sge = SMBDIRECT_MAX_RECV_SGE;
+ qp_attr.cap.max_recv_sge = SMBDIRECT_RECV_IO_MAX_SGE;
qp_attr.cap.max_inline_data = 0;
qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
qp_attr.qp_type = IB_QPT_RC;
diff --git a/fs/smb/client/smbdirect.h b/fs/smb/client/smbdirect.h
index 0463fde1bf26..81b55c0de552 100644
--- a/fs/smb/client/smbdirect.h
+++ b/fs/smb/client/smbdirect.h
@@ -139,9 +139,6 @@ struct smbd_request {
u8 packet[];
};
-/* Maximum number of SGEs used by smbdirect.c in any receive work request */
-#define SMBDIRECT_MAX_RECV_SGE 1
-
/* Create a SMBDirect session */
struct smbd_connection *smbd_get_connection(
struct TCP_Server_Info *server, struct sockaddr *dstaddr);
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 04/18] smb: smbdirect: introduce struct smbdirect_send_io
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (2 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 03/18] smb: client: make use of SMBDIRECT_RECV_IO_MAX_SGE Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 05/18] smb: client: make use of " Stefan Metzmacher
` (16 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
This will be used in client and server soon
in order to replace smbd_request/smb_direct_sendmsg.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/common/smbdirect/smbdirect_socket.h | 24 ++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/fs/smb/common/smbdirect/smbdirect_socket.h b/fs/smb/common/smbdirect/smbdirect_socket.h
index 7270fcee1048..4660c05c358f 100644
--- a/fs/smb/common/smbdirect/smbdirect_socket.h
+++ b/fs/smb/common/smbdirect/smbdirect_socket.h
@@ -89,6 +89,30 @@ struct smbdirect_socket {
} recv_io;
};
+struct smbdirect_send_io {
+ struct smbdirect_socket *socket;
+ struct ib_cqe cqe;
+
+ /*
+ * The SGE entries for this work request
+ *
+ * The first points to the packet header
+ */
+#define SMBDIRECT_SEND_IO_MAX_SGE 6
+ size_t num_sge;
+ struct ib_sge sge[SMBDIRECT_SEND_IO_MAX_SGE];
+
+ /*
+ * Link to the list of sibling smbdirect_send_io
+ * messages.
+ */
+ struct list_head sibling_list;
+ struct ib_send_wr wr;
+
+ /* SMBD packet header follows this structure */
+ u8 packet[];
+};
+
struct smbdirect_recv_io {
struct smbdirect_socket *socket;
struct ib_cqe cqe;
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 05/18] smb: client: make use of struct smbdirect_send_io
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (3 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 04/18] smb: smbdirect: introduce struct smbdirect_send_io Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 06/18] smb: smbdirect: add smbdirect_socket.{send,recv}_io.mem.{cache,pool} Stefan Metzmacher
` (15 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Steve French, Tom Talpey, Long Li
The server will also use this soon, so that we can
split out common helper functions in future.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 45 ++++++++++++++++++++-------------------
fs/smb/client/smbdirect.h | 16 --------------
2 files changed, 23 insertions(+), 38 deletions(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index 5d1fa83583f6..c367efef8c7a 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -255,7 +255,7 @@ smbd_qp_async_error_upcall(struct ib_event *event, void *context)
}
}
-static inline void *smbd_request_payload(struct smbd_request *request)
+static inline void *smbdirect_send_io_payload(struct smbdirect_send_io *request)
{
return (void *)request->packet;
}
@@ -269,12 +269,13 @@ static inline void *smbdirect_recv_io_payload(struct smbdirect_recv_io *response
static void send_done(struct ib_cq *cq, struct ib_wc *wc)
{
int i;
- struct smbd_request *request =
- container_of(wc->wr_cqe, struct smbd_request, cqe);
- struct smbd_connection *info = request->info;
- struct smbdirect_socket *sc = &info->socket;
+ struct smbdirect_send_io *request =
+ container_of(wc->wr_cqe, struct smbdirect_send_io, cqe);
+ struct smbdirect_socket *sc = request->socket;
+ struct smbd_connection *info =
+ container_of(sc, struct smbd_connection, socket);
- log_rdma_send(INFO, "smbd_request 0x%p completed wc->status=%d\n",
+ log_rdma_send(INFO, "smbdirect_send_io 0x%p completed wc->status=%d\n",
request, wc->status);
for (i = 0; i < request->num_sge; i++)
@@ -291,12 +292,12 @@ static void send_done(struct ib_cq *cq, struct ib_wc *wc)
return;
}
- if (atomic_dec_and_test(&request->info->send_pending))
- wake_up(&request->info->wait_send_pending);
+ if (atomic_dec_and_test(&info->send_pending))
+ wake_up(&info->wait_send_pending);
- wake_up(&request->info->wait_post_send);
+ wake_up(&info->wait_post_send);
- mempool_free(request, request->info->request_mempool);
+ mempool_free(request, info->request_mempool);
}
static void dump_smbdirect_negotiate_resp(struct smbdirect_negotiate_resp *resp)
@@ -688,16 +689,16 @@ static int smbd_post_send_negotiate_req(struct smbd_connection *info)
struct smbdirect_socket_parameters *sp = &sc->parameters;
struct ib_send_wr send_wr;
int rc = -ENOMEM;
- struct smbd_request *request;
+ struct smbdirect_send_io *request;
struct smbdirect_negotiate_req *packet;
request = mempool_alloc(info->request_mempool, GFP_KERNEL);
if (!request)
return rc;
- request->info = info;
+ request->socket = sc;
- packet = smbd_request_payload(request);
+ packet = smbdirect_send_io_payload(request);
packet->min_version = cpu_to_le16(SMBDIRECT_V1);
packet->max_version = cpu_to_le16(SMBDIRECT_V1);
packet->reserved = 0;
@@ -794,7 +795,7 @@ static int manage_keep_alive_before_sending(struct smbd_connection *info)
/* Post the send request */
static int smbd_post_send(struct smbd_connection *info,
- struct smbd_request *request)
+ struct smbdirect_send_io *request)
{
struct smbdirect_socket *sc = &info->socket;
struct smbdirect_socket_parameters *sp = &sc->parameters;
@@ -843,7 +844,7 @@ static int smbd_post_send_iter(struct smbd_connection *info,
int i, rc;
int header_length;
int data_length;
- struct smbd_request *request;
+ struct smbdirect_send_io *request;
struct smbdirect_data_transfer *packet;
int new_credits = 0;
@@ -888,14 +889,14 @@ static int smbd_post_send_iter(struct smbd_connection *info,
goto err_alloc;
}
- request->info = info;
+ request->socket = sc;
memset(request->sge, 0, sizeof(request->sge));
/* Fill in the data payload to find out how much data we can add */
if (iter) {
struct smb_extract_to_rdma extract = {
.nr_sge = 1,
- .max_sge = SMBDIRECT_MAX_SEND_SGE,
+ .max_sge = SMBDIRECT_SEND_IO_MAX_SGE,
.sge = request->sge,
.device = sc->ib.dev,
.local_dma_lkey = sc->ib.pd->local_dma_lkey,
@@ -917,7 +918,7 @@ static int smbd_post_send_iter(struct smbd_connection *info,
}
/* Fill in the packet header */
- packet = smbd_request_payload(request);
+ packet = smbdirect_send_io_payload(request);
packet->credits_requested = cpu_to_le16(sp->send_credit_target);
new_credits = manage_credits_prior_sending(info);
@@ -1447,11 +1448,11 @@ static int allocate_caches_and_workqueue(struct smbd_connection *info)
if (WARN_ON_ONCE(sp->max_recv_size < sizeof(struct smbdirect_data_transfer)))
return -ENOMEM;
- scnprintf(name, MAX_NAME_LEN, "smbd_request_%p", info);
+ scnprintf(name, MAX_NAME_LEN, "smbdirect_send_io_%p", info);
info->request_cache =
kmem_cache_create(
name,
- sizeof(struct smbd_request) +
+ sizeof(struct smbdirect_send_io) +
sizeof(struct smbdirect_data_transfer),
0, SLAB_HWCACHE_ALIGN, NULL);
if (!info->request_cache)
@@ -1562,7 +1563,7 @@ static struct smbd_connection *_smbd_get_connection(
sp->max_recv_size = smbd_max_receive_size;
sp->keepalive_interval_msec = smbd_keep_alive_interval * 1000;
- if (sc->ib.dev->attrs.max_send_sge < SMBDIRECT_MAX_SEND_SGE ||
+ if (sc->ib.dev->attrs.max_send_sge < SMBDIRECT_SEND_IO_MAX_SGE ||
sc->ib.dev->attrs.max_recv_sge < SMBDIRECT_RECV_IO_MAX_SGE) {
log_rdma_event(ERR,
"device %.*s max_send_sge/max_recv_sge = %d/%d too small\n",
@@ -1594,7 +1595,7 @@ static struct smbd_connection *_smbd_get_connection(
qp_attr.qp_context = info;
qp_attr.cap.max_send_wr = sp->send_credit_target;
qp_attr.cap.max_recv_wr = sp->recv_credit_max;
- qp_attr.cap.max_send_sge = SMBDIRECT_MAX_SEND_SGE;
+ qp_attr.cap.max_send_sge = SMBDIRECT_SEND_IO_MAX_SGE;
qp_attr.cap.max_recv_sge = SMBDIRECT_RECV_IO_MAX_SGE;
qp_attr.cap.max_inline_data = 0;
qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
diff --git a/fs/smb/client/smbdirect.h b/fs/smb/client/smbdirect.h
index 81b55c0de552..a8380bccf623 100644
--- a/fs/smb/client/smbdirect.h
+++ b/fs/smb/client/smbdirect.h
@@ -123,22 +123,6 @@ struct smbd_connection {
unsigned int count_send_empty;
};
-/* Maximum number of SGEs used by smbdirect.c in any send work request */
-#define SMBDIRECT_MAX_SEND_SGE 6
-
-/* The context for a SMBD request */
-struct smbd_request {
- struct smbd_connection *info;
- struct ib_cqe cqe;
-
- /* the SGE entries for this work request */
- struct ib_sge sge[SMBDIRECT_MAX_SEND_SGE];
- int num_sge;
-
- /* SMBD packet header follows this structure */
- u8 packet[];
-};
-
/* Create a SMBDirect session */
struct smbd_connection *smbd_get_connection(
struct TCP_Server_Info *server, struct sockaddr *dstaddr);
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 06/18] smb: smbdirect: add smbdirect_socket.{send,recv}_io.mem.{cache,pool}
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (4 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 05/18] smb: client: make use of " Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 07/18] smb: client: make use of smbdirect_socket.{send,recv}_io.mem.{cache,pool} Stefan Metzmacher
` (14 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
This will be the common location memory caches and pools.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/common/smbdirect/smbdirect_socket.h | 23 ++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/fs/smb/common/smbdirect/smbdirect_socket.h b/fs/smb/common/smbdirect/smbdirect_socket.h
index 4660c05c358f..3c4a8d627aa3 100644
--- a/fs/smb/common/smbdirect/smbdirect_socket.h
+++ b/fs/smb/common/smbdirect/smbdirect_socket.h
@@ -39,6 +39,20 @@ struct smbdirect_socket {
struct smbdirect_socket_parameters parameters;
+ /*
+ * The state for posted send buffers
+ */
+ struct {
+ /*
+ * Memory pools for preallocating
+ * smbdirect_send_io buffers
+ */
+ struct {
+ struct kmem_cache *cache;
+ mempool_t *pool;
+ } mem;
+ } send_io;
+
/*
* The state for posted receive buffers
*/
@@ -52,6 +66,15 @@ struct smbdirect_socket {
SMBDIRECT_EXPECT_DATA_TRANSFER = 3,
} expected;
+ /*
+ * Memory pools for preallocating
+ * smbdirect_recv_io buffers
+ */
+ struct {
+ struct kmem_cache *cache;
+ mempool_t *pool;
+ } mem;
+
/*
* The list of free smbdirect_recv_io
* structures
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 07/18] smb: client: make use of smbdirect_socket.{send,recv}_io.mem.{cache,pool}
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (5 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 06/18] smb: smbdirect: add smbdirect_socket.{send,recv}_io.mem.{cache,pool} Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 08/18] smb: server: make use of common smbdirect_pdu.h Stefan Metzmacher
` (13 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Steve French, Tom Talpey, Long Li
This will allow common helper functions to be created later.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 65 ++++++++++++++++++++-------------------
fs/smb/client/smbdirect.h | 9 ------
2 files changed, 34 insertions(+), 40 deletions(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index c367efef8c7a..6c2af00be44c 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -287,7 +287,7 @@ static void send_done(struct ib_cq *cq, struct ib_wc *wc)
if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_SEND) {
log_rdma_send(ERR, "wc->status=%d wc->opcode=%d\n",
wc->status, wc->opcode);
- mempool_free(request, info->request_mempool);
+ mempool_free(request, sc->send_io.mem.pool);
smbd_disconnect_rdma_connection(info);
return;
}
@@ -297,7 +297,7 @@ static void send_done(struct ib_cq *cq, struct ib_wc *wc)
wake_up(&info->wait_post_send);
- mempool_free(request, info->request_mempool);
+ mempool_free(request, sc->send_io.mem.pool);
}
static void dump_smbdirect_negotiate_resp(struct smbdirect_negotiate_resp *resp)
@@ -692,7 +692,7 @@ static int smbd_post_send_negotiate_req(struct smbd_connection *info)
struct smbdirect_send_io *request;
struct smbdirect_negotiate_req *packet;
- request = mempool_alloc(info->request_mempool, GFP_KERNEL);
+ request = mempool_alloc(sc->send_io.mem.pool, GFP_KERNEL);
if (!request)
return rc;
@@ -751,7 +751,7 @@ static int smbd_post_send_negotiate_req(struct smbd_connection *info)
smbd_disconnect_rdma_connection(info);
dma_mapping_failed:
- mempool_free(request, info->request_mempool);
+ mempool_free(request, sc->send_io.mem.pool);
return rc;
}
@@ -883,7 +883,7 @@ static int smbd_post_send_iter(struct smbd_connection *info,
goto wait_send_queue;
}
- request = mempool_alloc(info->request_mempool, GFP_KERNEL);
+ request = mempool_alloc(sc->send_io.mem.pool, GFP_KERNEL);
if (!request) {
rc = -ENOMEM;
goto err_alloc;
@@ -977,7 +977,7 @@ static int smbd_post_send_iter(struct smbd_connection *info,
request->sge[i].addr,
request->sge[i].length,
DMA_TO_DEVICE);
- mempool_free(request, info->request_mempool);
+ mempool_free(request, sc->send_io.mem.pool);
/* roll back receive credits and credits to be offered */
spin_lock(&info->lock_new_credits_offered);
@@ -1235,7 +1235,7 @@ static int allocate_receive_buffers(struct smbd_connection *info, int num_buf)
init_waitqueue_head(&info->wait_receive_queues);
for (i = 0; i < num_buf; i++) {
- response = mempool_alloc(info->response_mempool, GFP_KERNEL);
+ response = mempool_alloc(sc->recv_io.mem.pool, GFP_KERNEL);
if (!response)
goto allocate_failed;
@@ -1255,17 +1255,18 @@ static int allocate_receive_buffers(struct smbd_connection *info, int num_buf)
list_del(&response->list);
info->count_receive_queue--;
- mempool_free(response, info->response_mempool);
+ mempool_free(response, sc->recv_io.mem.pool);
}
return -ENOMEM;
}
static void destroy_receive_buffers(struct smbd_connection *info)
{
+ struct smbdirect_socket *sc = &info->socket;
struct smbdirect_recv_io *response;
while ((response = get_receive_buffer(info)))
- mempool_free(response, info->response_mempool);
+ mempool_free(response, sc->recv_io.mem.pool);
}
/* Implement idle connection timer [MS-SMBD] 3.1.6.2 */
@@ -1377,11 +1378,11 @@ void smbd_destroy(struct TCP_Server_Info *server)
rdma_destroy_id(sc->rdma.cm_id);
/* free mempools */
- mempool_destroy(info->request_mempool);
- kmem_cache_destroy(info->request_cache);
+ mempool_destroy(sc->send_io.mem.pool);
+ kmem_cache_destroy(sc->send_io.mem.cache);
- mempool_destroy(info->response_mempool);
- kmem_cache_destroy(info->response_cache);
+ mempool_destroy(sc->recv_io.mem.pool);
+ kmem_cache_destroy(sc->recv_io.mem.cache);
sc->status = SMBDIRECT_SOCKET_DESTROYED;
@@ -1429,12 +1430,14 @@ int smbd_reconnect(struct TCP_Server_Info *server)
static void destroy_caches_and_workqueue(struct smbd_connection *info)
{
+ struct smbdirect_socket *sc = &info->socket;
+
destroy_receive_buffers(info);
destroy_workqueue(info->workqueue);
- mempool_destroy(info->response_mempool);
- kmem_cache_destroy(info->response_cache);
- mempool_destroy(info->request_mempool);
- kmem_cache_destroy(info->request_cache);
+ mempool_destroy(sc->recv_io.mem.pool);
+ kmem_cache_destroy(sc->recv_io.mem.cache);
+ mempool_destroy(sc->send_io.mem.pool);
+ kmem_cache_destroy(sc->send_io.mem.cache);
}
#define MAX_NAME_LEN 80
@@ -1449,19 +1452,19 @@ static int allocate_caches_and_workqueue(struct smbd_connection *info)
return -ENOMEM;
scnprintf(name, MAX_NAME_LEN, "smbdirect_send_io_%p", info);
- info->request_cache =
+ sc->send_io.mem.cache =
kmem_cache_create(
name,
sizeof(struct smbdirect_send_io) +
sizeof(struct smbdirect_data_transfer),
0, SLAB_HWCACHE_ALIGN, NULL);
- if (!info->request_cache)
+ if (!sc->send_io.mem.cache)
return -ENOMEM;
- info->request_mempool =
+ sc->send_io.mem.pool =
mempool_create(sp->send_credit_target, mempool_alloc_slab,
- mempool_free_slab, info->request_cache);
- if (!info->request_mempool)
+ mempool_free_slab, sc->send_io.mem.cache);
+ if (!sc->send_io.mem.pool)
goto out1;
scnprintf(name, MAX_NAME_LEN, "smbdirect_recv_io_%p", info);
@@ -1472,17 +1475,17 @@ static int allocate_caches_and_workqueue(struct smbd_connection *info)
sizeof(struct smbdirect_data_transfer)),
.usersize = sp->max_recv_size - sizeof(struct smbdirect_data_transfer),
};
- info->response_cache =
+ sc->recv_io.mem.cache =
kmem_cache_create(name,
sizeof(struct smbdirect_recv_io) + sp->max_recv_size,
&response_args, SLAB_HWCACHE_ALIGN);
- if (!info->response_cache)
+ if (!sc->recv_io.mem.cache)
goto out2;
- info->response_mempool =
+ sc->recv_io.mem.pool =
mempool_create(sp->recv_credit_max, mempool_alloc_slab,
- mempool_free_slab, info->response_cache);
- if (!info->response_mempool)
+ mempool_free_slab, sc->recv_io.mem.cache);
+ if (!sc->recv_io.mem.pool)
goto out3;
scnprintf(name, MAX_NAME_LEN, "smbd_%p", info);
@@ -1501,13 +1504,13 @@ static int allocate_caches_and_workqueue(struct smbd_connection *info)
out5:
destroy_workqueue(info->workqueue);
out4:
- mempool_destroy(info->response_mempool);
+ mempool_destroy(sc->recv_io.mem.pool);
out3:
- kmem_cache_destroy(info->response_cache);
+ kmem_cache_destroy(sc->recv_io.mem.cache);
out2:
- mempool_destroy(info->request_mempool);
+ mempool_destroy(sc->send_io.mem.pool);
out1:
- kmem_cache_destroy(info->request_cache);
+ kmem_cache_destroy(sc->send_io.mem.cache);
return -ENOMEM;
}
diff --git a/fs/smb/client/smbdirect.h b/fs/smb/client/smbdirect.h
index a8380bccf623..0d4d45428c85 100644
--- a/fs/smb/client/smbdirect.h
+++ b/fs/smb/client/smbdirect.h
@@ -105,15 +105,6 @@ struct smbd_connection {
struct workqueue_struct *workqueue;
struct delayed_work idle_timer_work;
- /* Memory pool for preallocating buffers */
- /* request pool for RDMA send */
- struct kmem_cache *request_cache;
- mempool_t *request_mempool;
-
- /* response pool for RDMA receive */
- struct kmem_cache *response_cache;
- mempool_t *response_mempool;
-
/* for debug purposes */
unsigned int count_get_receive_buffer;
unsigned int count_put_receive_buffer;
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 08/18] smb: server: make use of common smbdirect_pdu.h
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (6 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 07/18] smb: client: make use of smbdirect_socket.{send,recv}_io.mem.{cache,pool} Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-07 15:47 ` ksmbd common smbdirect headers for 6.17-rc1? (Re: [PATCH 08/18] smb: server: make use of common smbdirect_pdu.h) Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 09/18] smb: server: make use of common smbdirect.h Stefan Metzmacher
` (12 subsequent siblings)
20 siblings, 1 reply; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon,
Hyunchul Lee, Meetakshi Setiya
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Hyunchul Lee <hyc.lee@gmail.com>
Cc: Meetakshi Setiya <meetakshisetiyaoss@gmail.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/transport_rdma.c | 49 +++++++++++++++++-----------------
fs/smb/server/transport_rdma.h | 41 ----------------------------
2 files changed, 25 insertions(+), 65 deletions(-)
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index 8d366db5f605..275199fef4e5 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -23,12 +23,13 @@
#include "connection.h"
#include "smb_common.h"
#include "../common/smb2status.h"
+#include "../common/smbdirect/smbdirect_pdu.h"
#include "transport_rdma.h"
#define SMB_DIRECT_PORT_IWARP 5445
#define SMB_DIRECT_PORT_INFINIBAND 445
-#define SMB_DIRECT_VERSION_LE cpu_to_le16(0x0100)
+#define SMB_DIRECT_VERSION_LE cpu_to_le16(SMBDIRECT_V1)
/* SMB_DIRECT negotiation timeout in seconds */
#define SMB_DIRECT_NEGOTIATE_TIMEOUT 120
@@ -472,8 +473,8 @@ static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
{
switch (recvmsg->type) {
case SMB_DIRECT_MSG_DATA_TRANSFER: {
- struct smb_direct_data_transfer *req =
- (struct smb_direct_data_transfer *)recvmsg->packet;
+ struct smbdirect_data_transfer *req =
+ (struct smbdirect_data_transfer *)recvmsg->packet;
struct smb2_hdr *hdr = (struct smb2_hdr *)(recvmsg->packet
+ le32_to_cpu(req->data_offset));
ksmbd_debug(RDMA,
@@ -485,8 +486,8 @@ static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
break;
}
case SMB_DIRECT_MSG_NEGOTIATE_REQ: {
- struct smb_direct_negotiate_req *req =
- (struct smb_direct_negotiate_req *)recvmsg->packet;
+ struct smbdirect_negotiate_req *req =
+ (struct smbdirect_negotiate_req *)recvmsg->packet;
ksmbd_debug(RDMA,
"MinVersion: %u, MaxVersion: %u, CreditRequested: %u, MaxSendSize: %u, MaxRecvSize: %u, MaxFragmentedSize: %u\n",
le16_to_cpu(req->min_version),
@@ -540,7 +541,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
switch (recvmsg->type) {
case SMB_DIRECT_MSG_NEGOTIATE_REQ:
- if (wc->byte_len < sizeof(struct smb_direct_negotiate_req)) {
+ if (wc->byte_len < sizeof(struct smbdirect_negotiate_req)) {
put_recvmsg(t, recvmsg);
smb_direct_disconnect_rdma_connection(t);
return;
@@ -552,13 +553,13 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
wake_up_interruptible(&t->wait_status);
return;
case SMB_DIRECT_MSG_DATA_TRANSFER: {
- struct smb_direct_data_transfer *data_transfer =
- (struct smb_direct_data_transfer *)recvmsg->packet;
+ struct smbdirect_data_transfer *data_transfer =
+ (struct smbdirect_data_transfer *)recvmsg->packet;
unsigned int data_length;
int avail_recvmsg_count, receive_credits;
if (wc->byte_len <
- offsetof(struct smb_direct_data_transfer, padding)) {
+ offsetof(struct smbdirect_data_transfer, padding)) {
put_recvmsg(t, recvmsg);
smb_direct_disconnect_rdma_connection(t);
return;
@@ -566,7 +567,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
data_length = le32_to_cpu(data_transfer->data_length);
if (data_length) {
- if (wc->byte_len < sizeof(struct smb_direct_data_transfer) +
+ if (wc->byte_len < sizeof(struct smbdirect_data_transfer) +
(u64)data_length) {
put_recvmsg(t, recvmsg);
smb_direct_disconnect_rdma_connection(t);
@@ -598,7 +599,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
&t->send_credits);
if (le16_to_cpu(data_transfer->flags) &
- SMB_DIRECT_RESPONSE_REQUESTED)
+ SMBDIRECT_FLAG_RESPONSE_REQUESTED)
queue_work(smb_direct_wq, &t->send_immediate_work);
if (atomic_read(&t->send_credits) > 0)
@@ -664,7 +665,7 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
unsigned int size, int unused)
{
struct smb_direct_recvmsg *recvmsg;
- struct smb_direct_data_transfer *data_transfer;
+ struct smbdirect_data_transfer *data_transfer;
int to_copy, to_read, data_read, offset;
u32 data_length, remaining_data_length, data_offset;
int rc;
@@ -1001,7 +1002,7 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
struct smb_direct_sendmsg **sendmsg_out)
{
struct smb_direct_sendmsg *sendmsg;
- struct smb_direct_data_transfer *packet;
+ struct smbdirect_data_transfer *packet;
int header_length;
int ret;
@@ -1010,7 +1011,7 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
return PTR_ERR(sendmsg);
/* Fill in the packet header */
- packet = (struct smb_direct_data_transfer *)sendmsg->packet;
+ packet = (struct smbdirect_data_transfer *)sendmsg->packet;
packet->credits_requested = cpu_to_le16(t->send_credit_target);
packet->credits_granted = cpu_to_le16(manage_credits_prior_sending(t));
@@ -1033,11 +1034,11 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
le32_to_cpu(packet->remaining_data_length));
/* Map the packet to DMA */
- header_length = sizeof(struct smb_direct_data_transfer);
+ header_length = sizeof(struct smbdirect_data_transfer);
/* If this is a packet without payload, don't send padding */
if (!size)
header_length =
- offsetof(struct smb_direct_data_transfer, padding);
+ offsetof(struct smbdirect_data_transfer, padding);
sendmsg->sge[0].addr = ib_dma_map_single(t->cm_id->device,
(void *)packet,
@@ -1212,7 +1213,7 @@ static int smb_direct_writev(struct ksmbd_transport *t,
int remaining_data_length;
int start, i, j;
int max_iov_size = st->max_send_size -
- sizeof(struct smb_direct_data_transfer);
+ sizeof(struct smbdirect_data_transfer);
int ret;
struct kvec vec;
struct smb_direct_send_ctx send_ctx;
@@ -1560,18 +1561,18 @@ static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
int failed)
{
struct smb_direct_sendmsg *sendmsg;
- struct smb_direct_negotiate_resp *resp;
+ struct smbdirect_negotiate_resp *resp;
int ret;
sendmsg = smb_direct_alloc_sendmsg(t);
if (IS_ERR(sendmsg))
return -ENOMEM;
- resp = (struct smb_direct_negotiate_resp *)sendmsg->packet;
+ resp = (struct smbdirect_negotiate_resp *)sendmsg->packet;
if (failed) {
memset(resp, 0, sizeof(*resp));
- resp->min_version = cpu_to_le16(0x0100);
- resp->max_version = cpu_to_le16(0x0100);
+ resp->min_version = SMB_DIRECT_VERSION_LE;
+ resp->max_version = SMB_DIRECT_VERSION_LE;
resp->status = STATUS_NOT_SUPPORTED;
} else {
resp->status = STATUS_SUCCESS;
@@ -1803,7 +1804,7 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
snprintf(name, sizeof(name), "smb_direct_rqst_pool_%p", t);
t->sendmsg_cache = kmem_cache_create(name,
sizeof(struct smb_direct_sendmsg) +
- sizeof(struct smb_direct_negotiate_resp),
+ sizeof(struct smbdirect_negotiate_resp),
0, SLAB_HWCACHE_ALIGN, NULL);
if (!t->sendmsg_cache)
return -ENOMEM;
@@ -1936,7 +1937,7 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
{
struct smb_direct_transport *st = smb_trans_direct_transfort(t);
struct smb_direct_recvmsg *recvmsg;
- struct smb_direct_negotiate_req *req;
+ struct smbdirect_negotiate_req *req;
int ret;
ksmbd_debug(RDMA, "Waiting for SMB_DIRECT negotiate request\n");
@@ -1955,7 +1956,7 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
if (ret == -ECONNABORTED)
goto out;
- req = (struct smb_direct_negotiate_req *)recvmsg->packet;
+ req = (struct smbdirect_negotiate_req *)recvmsg->packet;
st->max_recv_size = min_t(int, st->max_recv_size,
le32_to_cpu(req->preferred_send_size));
st->max_send_size = min_t(int, st->max_send_size,
diff --git a/fs/smb/server/transport_rdma.h b/fs/smb/server/transport_rdma.h
index 77aee4e5c9dc..0fb692c40e21 100644
--- a/fs/smb/server/transport_rdma.h
+++ b/fs/smb/server/transport_rdma.h
@@ -11,47 +11,6 @@
#define SMBD_MIN_IOSIZE (512 * 1024)
#define SMBD_MAX_IOSIZE (16 * 1024 * 1024)
-/* SMB DIRECT negotiation request packet [MS-SMBD] 2.2.1 */
-struct smb_direct_negotiate_req {
- __le16 min_version;
- __le16 max_version;
- __le16 reserved;
- __le16 credits_requested;
- __le32 preferred_send_size;
- __le32 max_receive_size;
- __le32 max_fragmented_size;
-} __packed;
-
-/* SMB DIRECT negotiation response packet [MS-SMBD] 2.2.2 */
-struct smb_direct_negotiate_resp {
- __le16 min_version;
- __le16 max_version;
- __le16 negotiated_version;
- __le16 reserved;
- __le16 credits_requested;
- __le16 credits_granted;
- __le32 status;
- __le32 max_readwrite_size;
- __le32 preferred_send_size;
- __le32 max_receive_size;
- __le32 max_fragmented_size;
-} __packed;
-
-#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001
-
-/* SMB DIRECT data transfer packet with payload [MS-SMBD] 2.2.3 */
-struct smb_direct_data_transfer {
- __le16 credits_requested;
- __le16 credits_granted;
- __le16 flags;
- __le16 reserved;
- __le32 remaining_data_length;
- __le32 data_offset;
- __le32 data_length;
- __le32 padding;
- __u8 buffer[];
-} __packed;
-
#ifdef CONFIG_SMB_SERVER_SMBDIRECT
int ksmbd_rdma_init(void);
void ksmbd_rdma_destroy(void);
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 09/18] smb: server: make use of common smbdirect.h
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (7 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 08/18] smb: server: make use of common smbdirect_pdu.h Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 10/18] smb: server: make use of common smbdirect_socket Stefan Metzmacher
` (11 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon,
Hyunchul Lee, Meetakshi Setiya
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Hyunchul Lee <hyc.lee@gmail.com>
Cc: Meetakshi Setiya <meetakshisetiyaoss@gmail.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/connection.c | 4 ++--
fs/smb/server/connection.h | 10 ++++++----
fs/smb/server/smb2pdu.c | 11 ++++++-----
fs/smb/server/smb2pdu.h | 6 ------
fs/smb/server/transport_rdma.c | 7 ++++---
5 files changed, 18 insertions(+), 20 deletions(-)
diff --git a/fs/smb/server/connection.c b/fs/smb/server/connection.c
index 3f04a2977ba8..6bfb29e36fa2 100644
--- a/fs/smb/server/connection.c
+++ b/fs/smb/server/connection.c
@@ -243,7 +243,7 @@ int ksmbd_conn_write(struct ksmbd_work *work)
int ksmbd_conn_rdma_read(struct ksmbd_conn *conn,
void *buf, unsigned int buflen,
- struct smb2_buffer_desc_v1 *desc,
+ struct smbdirect_buffer_descriptor_v1 *desc,
unsigned int desc_len)
{
int ret = -EINVAL;
@@ -257,7 +257,7 @@ int ksmbd_conn_rdma_read(struct ksmbd_conn *conn,
int ksmbd_conn_rdma_write(struct ksmbd_conn *conn,
void *buf, unsigned int buflen,
- struct smb2_buffer_desc_v1 *desc,
+ struct smbdirect_buffer_descriptor_v1 *desc,
unsigned int desc_len)
{
int ret = -EINVAL;
diff --git a/fs/smb/server/connection.h b/fs/smb/server/connection.h
index dd3e0e3f7bf0..b18a518de508 100644
--- a/fs/smb/server/connection.h
+++ b/fs/smb/server/connection.h
@@ -19,6 +19,8 @@
#include "smb_common.h"
#include "ksmbd_work.h"
+struct smbdirect_buffer_descriptor_v1;
+
#define KSMBD_SOCKET_BACKLOG 16
enum {
@@ -127,11 +129,11 @@ struct ksmbd_transport_ops {
unsigned int remote_key);
int (*rdma_read)(struct ksmbd_transport *t,
void *buf, unsigned int len,
- struct smb2_buffer_desc_v1 *desc,
+ struct smbdirect_buffer_descriptor_v1 *desc,
unsigned int desc_len);
int (*rdma_write)(struct ksmbd_transport *t,
void *buf, unsigned int len,
- struct smb2_buffer_desc_v1 *desc,
+ struct smbdirect_buffer_descriptor_v1 *desc,
unsigned int desc_len);
void (*free_transport)(struct ksmbd_transport *kt);
};
@@ -157,11 +159,11 @@ bool ksmbd_conn_lookup_dialect(struct ksmbd_conn *c);
int ksmbd_conn_write(struct ksmbd_work *work);
int ksmbd_conn_rdma_read(struct ksmbd_conn *conn,
void *buf, unsigned int buflen,
- struct smb2_buffer_desc_v1 *desc,
+ struct smbdirect_buffer_descriptor_v1 *desc,
unsigned int desc_len);
int ksmbd_conn_rdma_write(struct ksmbd_conn *conn,
void *buf, unsigned int buflen,
- struct smb2_buffer_desc_v1 *desc,
+ struct smbdirect_buffer_descriptor_v1 *desc,
unsigned int desc_len);
void ksmbd_conn_enqueue_request(struct ksmbd_work *work);
void ksmbd_conn_try_dequeue_request(struct ksmbd_work *work);
diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c
index 6fc6ad63d004..e5699c4ee2e9 100644
--- a/fs/smb/server/smb2pdu.c
+++ b/fs/smb/server/smb2pdu.c
@@ -23,6 +23,7 @@
#include "asn1.h"
#include "connection.h"
#include "transport_ipc.h"
+#include "../common/smbdirect/smbdirect.h"
#include "transport_rdma.h"
#include "vfs.h"
#include "vfs_cache.h"
@@ -6662,7 +6663,7 @@ static noinline int smb2_read_pipe(struct ksmbd_work *work)
}
static int smb2_set_remote_key_for_rdma(struct ksmbd_work *work,
- struct smb2_buffer_desc_v1 *desc,
+ struct smbdirect_buffer_descriptor_v1 *desc,
__le32 Channel,
__le16 ChannelInfoLength)
{
@@ -6698,7 +6699,7 @@ static ssize_t smb2_read_rdma_channel(struct ksmbd_work *work,
int err;
err = ksmbd_conn_rdma_write(work->conn, data_buf, length,
- (struct smb2_buffer_desc_v1 *)
+ (struct smbdirect_buffer_descriptor_v1 *)
((char *)req + le16_to_cpu(req->ReadChannelInfoOffset)),
le16_to_cpu(req->ReadChannelInfoLength));
if (err)
@@ -6769,7 +6770,7 @@ int smb2_read(struct ksmbd_work *work)
goto out;
}
err = smb2_set_remote_key_for_rdma(work,
- (struct smb2_buffer_desc_v1 *)
+ (struct smbdirect_buffer_descriptor_v1 *)
((char *)req + ch_offset),
req->Channel,
req->ReadChannelInfoLength);
@@ -6964,7 +6965,7 @@ static ssize_t smb2_write_rdma_channel(struct ksmbd_work *work,
return -ENOMEM;
ret = ksmbd_conn_rdma_read(work->conn, data_buf, length,
- (struct smb2_buffer_desc_v1 *)
+ (struct smbdirect_buffer_descriptor_v1 *)
((char *)req + le16_to_cpu(req->WriteChannelInfoOffset)),
le16_to_cpu(req->WriteChannelInfoLength));
if (ret < 0) {
@@ -7029,7 +7030,7 @@ int smb2_write(struct ksmbd_work *work)
goto out;
}
err = smb2_set_remote_key_for_rdma(work,
- (struct smb2_buffer_desc_v1 *)
+ (struct smbdirect_buffer_descriptor_v1 *)
((char *)req + ch_offset),
req->Channel,
req->WriteChannelInfoLength);
diff --git a/fs/smb/server/smb2pdu.h b/fs/smb/server/smb2pdu.h
index 16ae8a10490b..5163d5241b90 100644
--- a/fs/smb/server/smb2pdu.h
+++ b/fs/smb/server/smb2pdu.h
@@ -136,12 +136,6 @@ struct create_posix_rsp {
u8 SidBuffer[44];
} __packed;
-struct smb2_buffer_desc_v1 {
- __le64 offset;
- __le32 token;
- __le32 length;
-} __packed;
-
#define SMB2_0_IOCTL_IS_FSCTL 0x00000001
struct smb_sockaddr_in {
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index 275199fef4e5..584f89b88c71 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -23,6 +23,7 @@
#include "connection.h"
#include "smb_common.h"
#include "../common/smb2status.h"
+#include "../common/smbdirect/smbdirect.h"
#include "../common/smbdirect/smbdirect_pdu.h"
#include "transport_rdma.h"
@@ -1335,7 +1336,7 @@ static void write_done(struct ib_cq *cq, struct ib_wc *wc)
static int smb_direct_rdma_xmit(struct smb_direct_transport *t,
void *buf, int buf_len,
- struct smb2_buffer_desc_v1 *desc,
+ struct smbdirect_buffer_descriptor_v1 *desc,
unsigned int desc_len,
bool is_read)
{
@@ -1465,7 +1466,7 @@ static int smb_direct_rdma_xmit(struct smb_direct_transport *t,
static int smb_direct_rdma_write(struct ksmbd_transport *t,
void *buf, unsigned int buflen,
- struct smb2_buffer_desc_v1 *desc,
+ struct smbdirect_buffer_descriptor_v1 *desc,
unsigned int desc_len)
{
return smb_direct_rdma_xmit(smb_trans_direct_transfort(t), buf, buflen,
@@ -1474,7 +1475,7 @@ static int smb_direct_rdma_write(struct ksmbd_transport *t,
static int smb_direct_rdma_read(struct ksmbd_transport *t,
void *buf, unsigned int buflen,
- struct smb2_buffer_desc_v1 *desc,
+ struct smbdirect_buffer_descriptor_v1 *desc,
unsigned int desc_len)
{
return smb_direct_rdma_xmit(smb_trans_direct_transfort(t), buf, buflen,
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 10/18] smb: server: make use of common smbdirect_socket
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (8 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 09/18] smb: server: make use of common smbdirect.h Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 11/18] smb: server: make use of common smbdirect_socket_parameters Stefan Metzmacher
` (10 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon,
Hyunchul Lee, Meetakshi Setiya
This is the next step in the direction of a common smbdirect layer.
Currently only structures are shared, but that will change
over time until everything is shared.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Hyunchul Lee <hyc.lee@gmail.com>
Cc: Meetakshi Setiya <meetakshisetiyaoss@gmail.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/transport_rdma.c | 249 ++++++++++++++++++---------------
1 file changed, 137 insertions(+), 112 deletions(-)
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index 584f89b88c71..1c852c8601c6 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -25,6 +25,7 @@
#include "../common/smb2status.h"
#include "../common/smbdirect/smbdirect.h"
#include "../common/smbdirect/smbdirect_pdu.h"
+#include "../common/smbdirect/smbdirect_socket.h"
#include "transport_rdma.h"
#define SMB_DIRECT_PORT_IWARP 5445
@@ -89,26 +90,14 @@ static struct smb_direct_listener {
static struct workqueue_struct *smb_direct_wq;
-enum smb_direct_status {
- SMB_DIRECT_CS_NEW = 0,
- SMB_DIRECT_CS_CONNECTED,
- SMB_DIRECT_CS_DISCONNECTING,
- SMB_DIRECT_CS_DISCONNECTED,
-};
-
struct smb_direct_transport {
struct ksmbd_transport transport;
- enum smb_direct_status status;
+ struct smbdirect_socket socket;
+
bool full_packet_received;
wait_queue_head_t wait_status;
- struct rdma_cm_id *cm_id;
- struct ib_cq *send_cq;
- struct ib_cq *recv_cq;
- struct ib_pd *pd;
- struct ib_qp *qp;
-
int max_send_size;
int max_recv_size;
int max_fragmented_send_size;
@@ -267,8 +256,10 @@ smb_direct_recvmsg *get_free_recvmsg(struct smb_direct_transport *t)
static void put_recvmsg(struct smb_direct_transport *t,
struct smb_direct_recvmsg *recvmsg)
{
+ struct smbdirect_socket *sc = &t->socket;
+
if (likely(recvmsg->sge.length != 0)) {
- ib_dma_unmap_single(t->cm_id->device,
+ ib_dma_unmap_single(sc->ib.dev,
recvmsg->sge.addr,
recvmsg->sge.length,
DMA_FROM_DEVICE);
@@ -312,17 +303,20 @@ static void smb_direct_disconnect_rdma_work(struct work_struct *work)
struct smb_direct_transport *t =
container_of(work, struct smb_direct_transport,
disconnect_work);
+ struct smbdirect_socket *sc = &t->socket;
- if (t->status == SMB_DIRECT_CS_CONNECTED) {
- t->status = SMB_DIRECT_CS_DISCONNECTING;
- rdma_disconnect(t->cm_id);
+ if (sc->status == SMBDIRECT_SOCKET_CONNECTED) {
+ sc->status = SMBDIRECT_SOCKET_DISCONNECTING;
+ rdma_disconnect(sc->rdma.cm_id);
}
}
static void
smb_direct_disconnect_rdma_connection(struct smb_direct_transport *t)
{
- if (t->status == SMB_DIRECT_CS_CONNECTED)
+ struct smbdirect_socket *sc = &t->socket;
+
+ if (sc->status == SMBDIRECT_SOCKET_CONNECTED)
queue_work(smb_direct_wq, &t->disconnect_work);
}
@@ -330,8 +324,9 @@ static void smb_direct_send_immediate_work(struct work_struct *work)
{
struct smb_direct_transport *t = container_of(work,
struct smb_direct_transport, send_immediate_work);
+ struct smbdirect_socket *sc = &t->socket;
- if (t->status != SMB_DIRECT_CS_CONNECTED)
+ if (sc->status != SMBDIRECT_SOCKET_CONNECTED)
return;
smb_direct_post_send_data(t, NULL, NULL, 0, 0);
@@ -340,16 +335,20 @@ static void smb_direct_send_immediate_work(struct work_struct *work)
static struct smb_direct_transport *alloc_transport(struct rdma_cm_id *cm_id)
{
struct smb_direct_transport *t;
+ struct smbdirect_socket *sc;
struct ksmbd_conn *conn;
t = kzalloc(sizeof(*t), KSMBD_DEFAULT_GFP);
if (!t)
return NULL;
+ sc = &t->socket;
- t->cm_id = cm_id;
+ sc->rdma.cm_id = cm_id;
cm_id->context = t;
- t->status = SMB_DIRECT_CS_NEW;
+ sc->ib.dev = sc->rdma.cm_id->device;
+
+ sc->status = SMBDIRECT_SOCKET_CREATED;
init_waitqueue_head(&t->wait_status);
spin_lock_init(&t->reassembly_queue_lock);
@@ -393,6 +392,7 @@ static void smb_direct_free_transport(struct ksmbd_transport *kt)
static void free_transport(struct smb_direct_transport *t)
{
+ struct smbdirect_socket *sc = &t->socket;
struct smb_direct_recvmsg *recvmsg;
wake_up_interruptible(&t->wait_send_credits);
@@ -405,11 +405,11 @@ static void free_transport(struct smb_direct_transport *t)
cancel_delayed_work_sync(&t->post_recv_credits_work);
cancel_work_sync(&t->send_immediate_work);
- if (t->qp) {
- ib_drain_qp(t->qp);
- ib_mr_pool_destroy(t->qp, &t->qp->rdma_mrs);
- t->qp = NULL;
- rdma_destroy_qp(t->cm_id);
+ if (sc->ib.qp) {
+ ib_drain_qp(sc->ib.qp);
+ ib_mr_pool_destroy(sc->ib.qp, &sc->ib.qp->rdma_mrs);
+ sc->ib.qp = NULL;
+ rdma_destroy_qp(sc->rdma.cm_id);
}
ksmbd_debug(RDMA, "drain the reassembly queue\n");
@@ -426,14 +426,14 @@ static void free_transport(struct smb_direct_transport *t)
} while (recvmsg);
t->reassembly_data_length = 0;
- if (t->send_cq)
- ib_free_cq(t->send_cq);
- if (t->recv_cq)
- ib_free_cq(t->recv_cq);
- if (t->pd)
- ib_dealloc_pd(t->pd);
- if (t->cm_id)
- rdma_destroy_id(t->cm_id);
+ if (sc->ib.send_cq)
+ ib_free_cq(sc->ib.send_cq);
+ if (sc->ib.recv_cq)
+ ib_free_cq(sc->ib.recv_cq);
+ if (sc->ib.pd)
+ ib_dealloc_pd(sc->ib.pd);
+ if (sc->rdma.cm_id)
+ rdma_destroy_id(sc->rdma.cm_id);
smb_direct_destroy_pools(t);
ksmbd_conn_free(KSMBD_TRANS(t)->conn);
@@ -456,14 +456,15 @@ static struct smb_direct_sendmsg
static void smb_direct_free_sendmsg(struct smb_direct_transport *t,
struct smb_direct_sendmsg *msg)
{
+ struct smbdirect_socket *sc = &t->socket;
int i;
if (msg->num_sge > 0) {
- ib_dma_unmap_single(t->cm_id->device,
+ ib_dma_unmap_single(sc->ib.dev,
msg->sge[0].addr, msg->sge[0].length,
DMA_TO_DEVICE);
for (i = 1; i < msg->num_sge; i++)
- ib_dma_unmap_page(t->cm_id->device,
+ ib_dma_unmap_page(sc->ib.dev,
msg->sge[i].addr, msg->sge[i].length,
DMA_TO_DEVICE);
}
@@ -518,9 +519,11 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
{
struct smb_direct_recvmsg *recvmsg;
struct smb_direct_transport *t;
+ struct smbdirect_socket *sc;
recvmsg = container_of(wc->wr_cqe, struct smb_direct_recvmsg, cqe);
t = recvmsg->transport;
+ sc = &t->socket;
if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_RECV) {
put_recvmsg(t, recvmsg);
@@ -549,7 +552,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
}
t->negotiation_requested = true;
t->full_packet_received = true;
- t->status = SMB_DIRECT_CS_CONNECTED;
+ sc->status = SMBDIRECT_SOCKET_CONNECTED;
enqueue_reassembly(t, recvmsg, 0);
wake_up_interruptible(&t->wait_status);
return;
@@ -631,17 +634,18 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
static int smb_direct_post_recv(struct smb_direct_transport *t,
struct smb_direct_recvmsg *recvmsg)
{
+ struct smbdirect_socket *sc = &t->socket;
struct ib_recv_wr wr;
int ret;
- recvmsg->sge.addr = ib_dma_map_single(t->cm_id->device,
+ recvmsg->sge.addr = ib_dma_map_single(sc->ib.dev,
recvmsg->packet, t->max_recv_size,
DMA_FROM_DEVICE);
- ret = ib_dma_mapping_error(t->cm_id->device, recvmsg->sge.addr);
+ ret = ib_dma_mapping_error(sc->ib.dev, recvmsg->sge.addr);
if (ret)
return ret;
recvmsg->sge.length = t->max_recv_size;
- recvmsg->sge.lkey = t->pd->local_dma_lkey;
+ recvmsg->sge.lkey = sc->ib.pd->local_dma_lkey;
recvmsg->cqe.done = recv_done;
wr.wr_cqe = &recvmsg->cqe;
@@ -649,10 +653,10 @@ static int smb_direct_post_recv(struct smb_direct_transport *t,
wr.sg_list = &recvmsg->sge;
wr.num_sge = 1;
- ret = ib_post_recv(t->qp, &wr, NULL);
+ ret = ib_post_recv(sc->ib.qp, &wr, NULL);
if (ret) {
pr_err("Can't post recv: %d\n", ret);
- ib_dma_unmap_single(t->cm_id->device,
+ ib_dma_unmap_single(sc->ib.dev,
recvmsg->sge.addr, recvmsg->sge.length,
DMA_FROM_DEVICE);
recvmsg->sge.length = 0;
@@ -671,9 +675,10 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
u32 data_length, remaining_data_length, data_offset;
int rc;
struct smb_direct_transport *st = smb_trans_direct_transfort(t);
+ struct smbdirect_socket *sc = &st->socket;
again:
- if (st->status != SMB_DIRECT_CS_CONNECTED) {
+ if (sc->status != SMBDIRECT_SOCKET_CONNECTED) {
pr_err("disconnected\n");
return -ENOTCONN;
}
@@ -783,7 +788,7 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
ksmbd_debug(RDMA, "wait_event on more data\n");
rc = wait_event_interruptible(st->wait_reassembly_queue,
st->reassembly_data_length >= size ||
- st->status != SMB_DIRECT_CS_CONNECTED);
+ sc->status != SMBDIRECT_SOCKET_CONNECTED);
if (rc)
return -EINTR;
@@ -885,10 +890,11 @@ static int manage_credits_prior_sending(struct smb_direct_transport *t)
static int smb_direct_post_send(struct smb_direct_transport *t,
struct ib_send_wr *wr)
{
+ struct smbdirect_socket *sc = &t->socket;
int ret;
atomic_inc(&t->send_pending);
- ret = ib_post_send(t->qp, wr, NULL);
+ ret = ib_post_send(sc->ib.qp, wr, NULL);
if (ret) {
pr_err("failed to post send: %d\n", ret);
if (atomic_dec_and_test(&t->send_pending))
@@ -953,6 +959,7 @@ static int wait_for_credits(struct smb_direct_transport *t,
wait_queue_head_t *waitq, atomic_t *total_credits,
int needed)
{
+ struct smbdirect_socket *sc = &t->socket;
int ret;
do {
@@ -962,9 +969,9 @@ static int wait_for_credits(struct smb_direct_transport *t,
atomic_add(needed, total_credits);
ret = wait_event_interruptible(*waitq,
atomic_read(total_credits) >= needed ||
- t->status != SMB_DIRECT_CS_CONNECTED);
+ sc->status != SMBDIRECT_SOCKET_CONNECTED);
- if (t->status != SMB_DIRECT_CS_CONNECTED)
+ if (sc->status != SMBDIRECT_SOCKET_CONNECTED)
return -ENOTCONN;
else if (ret < 0)
return ret;
@@ -1002,6 +1009,7 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
int size, int remaining_data_length,
struct smb_direct_sendmsg **sendmsg_out)
{
+ struct smbdirect_socket *sc = &t->socket;
struct smb_direct_sendmsg *sendmsg;
struct smbdirect_data_transfer *packet;
int header_length;
@@ -1041,11 +1049,11 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
header_length =
offsetof(struct smbdirect_data_transfer, padding);
- sendmsg->sge[0].addr = ib_dma_map_single(t->cm_id->device,
+ sendmsg->sge[0].addr = ib_dma_map_single(sc->ib.dev,
(void *)packet,
header_length,
DMA_TO_DEVICE);
- ret = ib_dma_mapping_error(t->cm_id->device, sendmsg->sge[0].addr);
+ ret = ib_dma_mapping_error(sc->ib.dev, sendmsg->sge[0].addr);
if (ret) {
smb_direct_free_sendmsg(t, sendmsg);
return ret;
@@ -1053,7 +1061,7 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
sendmsg->num_sge = 1;
sendmsg->sge[0].length = header_length;
- sendmsg->sge[0].lkey = t->pd->local_dma_lkey;
+ sendmsg->sge[0].lkey = sc->ib.pd->local_dma_lkey;
*sendmsg_out = sendmsg;
return 0;
@@ -1107,10 +1115,11 @@ static int post_sendmsg(struct smb_direct_transport *t,
struct smb_direct_send_ctx *send_ctx,
struct smb_direct_sendmsg *msg)
{
+ struct smbdirect_socket *sc = &t->socket;
int i;
for (i = 0; i < msg->num_sge; i++)
- ib_dma_sync_single_for_device(t->cm_id->device,
+ ib_dma_sync_single_for_device(sc->ib.dev,
msg->sge[i].addr, msg->sge[i].length,
DMA_TO_DEVICE);
@@ -1146,6 +1155,7 @@ static int smb_direct_post_send_data(struct smb_direct_transport *t,
struct kvec *iov, int niov,
int remaining_data_length)
{
+ struct smbdirect_socket *sc = &t->socket;
int i, j, ret;
struct smb_direct_sendmsg *msg;
int data_length;
@@ -1171,7 +1181,7 @@ static int smb_direct_post_send_data(struct smb_direct_transport *t,
int sg_cnt;
sg_init_table(sg, SMB_DIRECT_MAX_SEND_SGES - 1);
- sg_cnt = get_mapped_sg_list(t->cm_id->device,
+ sg_cnt = get_mapped_sg_list(sc->ib.dev,
iov[i].iov_base, iov[i].iov_len,
sg, SMB_DIRECT_MAX_SEND_SGES - 1,
DMA_TO_DEVICE);
@@ -1182,7 +1192,7 @@ static int smb_direct_post_send_data(struct smb_direct_transport *t,
} else if (sg_cnt + msg->num_sge > SMB_DIRECT_MAX_SEND_SGES) {
pr_err("buffer not fitted into sges\n");
ret = -E2BIG;
- ib_dma_unmap_sg(t->cm_id->device, sg, sg_cnt,
+ ib_dma_unmap_sg(sc->ib.dev, sg, sg_cnt,
DMA_TO_DEVICE);
goto err;
}
@@ -1191,7 +1201,7 @@ static int smb_direct_post_send_data(struct smb_direct_transport *t,
sge = &msg->sge[msg->num_sge];
sge->addr = sg_dma_address(&sg[j]);
sge->length = sg_dma_len(&sg[j]);
- sge->lkey = t->pd->local_dma_lkey;
+ sge->lkey = sc->ib.pd->local_dma_lkey;
msg->num_sge++;
}
}
@@ -1211,6 +1221,7 @@ static int smb_direct_writev(struct ksmbd_transport *t,
bool need_invalidate, unsigned int remote_key)
{
struct smb_direct_transport *st = smb_trans_direct_transfort(t);
+ struct smbdirect_socket *sc = &st->socket;
int remaining_data_length;
int start, i, j;
int max_iov_size = st->max_send_size -
@@ -1219,7 +1230,7 @@ static int smb_direct_writev(struct ksmbd_transport *t,
struct kvec vec;
struct smb_direct_send_ctx send_ctx;
- if (st->status != SMB_DIRECT_CS_CONNECTED)
+ if (sc->status != SMBDIRECT_SOCKET_CONNECTED)
return -ENOTCONN;
//FIXME: skip RFC1002 header..
@@ -1300,7 +1311,9 @@ static void smb_direct_free_rdma_rw_msg(struct smb_direct_transport *t,
struct smb_direct_rdma_rw_msg *msg,
enum dma_data_direction dir)
{
- rdma_rw_ctx_destroy(&msg->rw_ctx, t->qp, t->qp->port,
+ struct smbdirect_socket *sc = &t->socket;
+
+ rdma_rw_ctx_destroy(&msg->rw_ctx, sc->ib.qp, sc->ib.qp->port,
msg->sgt.sgl, msg->sgt.nents, dir);
sg_free_table_chained(&msg->sgt, SG_CHUNK_SIZE);
kfree(msg);
@@ -1340,6 +1353,7 @@ static int smb_direct_rdma_xmit(struct smb_direct_transport *t,
unsigned int desc_len,
bool is_read)
{
+ struct smbdirect_socket *sc = &t->socket;
struct smb_direct_rdma_rw_msg *msg, *next_msg;
int i, ret;
DECLARE_COMPLETION_ONSTACK(completion);
@@ -1349,7 +1363,7 @@ static int smb_direct_rdma_xmit(struct smb_direct_transport *t,
int credits_needed;
unsigned int desc_buf_len, desc_num = 0;
- if (t->status != SMB_DIRECT_CS_CONNECTED)
+ if (sc->status != SMBDIRECT_SOCKET_CONNECTED)
return -ENOTCONN;
if (buf_len > t->max_rdma_rw_size)
@@ -1419,7 +1433,7 @@ static int smb_direct_rdma_xmit(struct smb_direct_transport *t,
goto out;
}
- ret = rdma_rw_ctx_init(&msg->rw_ctx, t->qp, t->qp->port,
+ ret = rdma_rw_ctx_init(&msg->rw_ctx, sc->ib.qp, sc->ib.qp->port,
msg->sgt.sgl,
get_buf_page_count(desc_buf, desc_buf_len),
0,
@@ -1440,11 +1454,11 @@ static int smb_direct_rdma_xmit(struct smb_direct_transport *t,
/* concatenate work requests of rdma_rw_ctxs */
first_wr = NULL;
list_for_each_entry_reverse(msg, &msg_list, list) {
- first_wr = rdma_rw_ctx_wrs(&msg->rw_ctx, t->qp, t->qp->port,
+ first_wr = rdma_rw_ctx_wrs(&msg->rw_ctx, sc->ib.qp, sc->ib.qp->port,
&msg->cqe, first_wr);
}
- ret = ib_post_send(t->qp, first_wr, NULL);
+ ret = ib_post_send(sc->ib.qp, first_wr, NULL);
if (ret) {
pr_err("failed to post send wr for RDMA R/W: %d\n", ret);
goto out;
@@ -1485,20 +1499,22 @@ static int smb_direct_rdma_read(struct ksmbd_transport *t,
static void smb_direct_disconnect(struct ksmbd_transport *t)
{
struct smb_direct_transport *st = smb_trans_direct_transfort(t);
+ struct smbdirect_socket *sc = &st->socket;
- ksmbd_debug(RDMA, "Disconnecting cm_id=%p\n", st->cm_id);
+ ksmbd_debug(RDMA, "Disconnecting cm_id=%p\n", sc->rdma.cm_id);
smb_direct_disconnect_rdma_work(&st->disconnect_work);
wait_event_interruptible(st->wait_status,
- st->status == SMB_DIRECT_CS_DISCONNECTED);
+ sc->status == SMBDIRECT_SOCKET_DISCONNECTED);
free_transport(st);
}
static void smb_direct_shutdown(struct ksmbd_transport *t)
{
struct smb_direct_transport *st = smb_trans_direct_transfort(t);
+ struct smbdirect_socket *sc = &st->socket;
- ksmbd_debug(RDMA, "smb-direct shutdown cm_id=%p\n", st->cm_id);
+ ksmbd_debug(RDMA, "smb-direct shutdown cm_id=%p\n", sc->rdma.cm_id);
smb_direct_disconnect_rdma_work(&st->disconnect_work);
}
@@ -1507,28 +1523,29 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
struct rdma_cm_event *event)
{
struct smb_direct_transport *t = cm_id->context;
+ struct smbdirect_socket *sc = &t->socket;
ksmbd_debug(RDMA, "RDMA CM event. cm_id=%p event=%s (%d)\n",
cm_id, rdma_event_msg(event->event), event->event);
switch (event->event) {
case RDMA_CM_EVENT_ESTABLISHED: {
- t->status = SMB_DIRECT_CS_CONNECTED;
+ sc->status = SMBDIRECT_SOCKET_CONNECTED;
wake_up_interruptible(&t->wait_status);
break;
}
case RDMA_CM_EVENT_DEVICE_REMOVAL:
case RDMA_CM_EVENT_DISCONNECTED: {
- ib_drain_qp(t->qp);
+ ib_drain_qp(sc->ib.qp);
- t->status = SMB_DIRECT_CS_DISCONNECTED;
+ sc->status = SMBDIRECT_SOCKET_DISCONNECTED;
wake_up_interruptible(&t->wait_status);
wake_up_interruptible(&t->wait_reassembly_queue);
wake_up(&t->wait_send_credits);
break;
}
case RDMA_CM_EVENT_CONNECT_ERROR: {
- t->status = SMB_DIRECT_CS_DISCONNECTED;
+ sc->status = SMBDIRECT_SOCKET_DISCONNECTED;
wake_up_interruptible(&t->wait_status);
break;
}
@@ -1544,9 +1561,10 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
static void smb_direct_qpair_handler(struct ib_event *event, void *context)
{
struct smb_direct_transport *t = context;
+ struct smbdirect_socket *sc = &t->socket;
ksmbd_debug(RDMA, "Received QP event. cm_id=%p, event=%s (%d)\n",
- t->cm_id, ib_event_msg(event->event), event->event);
+ sc->rdma.cm_id, ib_event_msg(event->event), event->event);
switch (event->event) {
case IB_EVENT_CQ_ERR:
@@ -1561,6 +1579,7 @@ static void smb_direct_qpair_handler(struct ib_event *event, void *context)
static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
int failed)
{
+ struct smbdirect_socket *sc = &t->socket;
struct smb_direct_sendmsg *sendmsg;
struct smbdirect_negotiate_resp *resp;
int ret;
@@ -1591,10 +1610,10 @@ static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
cpu_to_le32(t->max_fragmented_recv_size);
}
- sendmsg->sge[0].addr = ib_dma_map_single(t->cm_id->device,
+ sendmsg->sge[0].addr = ib_dma_map_single(sc->ib.dev,
(void *)resp, sizeof(*resp),
DMA_TO_DEVICE);
- ret = ib_dma_mapping_error(t->cm_id->device, sendmsg->sge[0].addr);
+ ret = ib_dma_mapping_error(sc->ib.dev, sendmsg->sge[0].addr);
if (ret) {
smb_direct_free_sendmsg(t, sendmsg);
return ret;
@@ -1602,7 +1621,7 @@ static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
sendmsg->num_sge = 1;
sendmsg->sge[0].length = sizeof(*resp);
- sendmsg->sge[0].lkey = t->pd->local_dma_lkey;
+ sendmsg->sge[0].lkey = sc->ib.pd->local_dma_lkey;
ret = post_sendmsg(t, NULL, sendmsg);
if (ret) {
@@ -1617,19 +1636,20 @@ static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
static int smb_direct_accept_client(struct smb_direct_transport *t)
{
+ struct smbdirect_socket *sc = &t->socket;
struct rdma_conn_param conn_param;
struct ib_port_immutable port_immutable;
u32 ird_ord_hdr[2];
int ret;
memset(&conn_param, 0, sizeof(conn_param));
- conn_param.initiator_depth = min_t(u8, t->cm_id->device->attrs.max_qp_rd_atom,
+ conn_param.initiator_depth = min_t(u8, sc->ib.dev->attrs.max_qp_rd_atom,
SMB_DIRECT_CM_INITIATOR_DEPTH);
conn_param.responder_resources = 0;
- t->cm_id->device->ops.get_port_immutable(t->cm_id->device,
- t->cm_id->port_num,
- &port_immutable);
+ sc->ib.dev->ops.get_port_immutable(sc->ib.dev,
+ sc->rdma.cm_id->port_num,
+ &port_immutable);
if (port_immutable.core_cap_flags & RDMA_CORE_PORT_IWARP) {
ird_ord_hdr[0] = conn_param.responder_resources;
ird_ord_hdr[1] = 1;
@@ -1643,7 +1663,7 @@ static int smb_direct_accept_client(struct smb_direct_transport *t)
conn_param.rnr_retry_count = SMB_DIRECT_CM_RNR_RETRY;
conn_param.flow_control = 0;
- ret = rdma_accept(t->cm_id, &conn_param);
+ ret = rdma_accept(sc->rdma.cm_id, &conn_param);
if (ret) {
pr_err("error at rdma_accept: %d\n", ret);
return ret;
@@ -1683,15 +1703,18 @@ static int smb_direct_prepare_negotiation(struct smb_direct_transport *t)
static unsigned int smb_direct_get_max_fr_pages(struct smb_direct_transport *t)
{
+ struct smbdirect_socket *sc = &t->socket;
+
return min_t(unsigned int,
- t->cm_id->device->attrs.max_fast_reg_page_list_len,
+ sc->ib.dev->attrs.max_fast_reg_page_list_len,
256);
}
static int smb_direct_init_params(struct smb_direct_transport *t,
struct ib_qp_cap *cap)
{
- struct ib_device *device = t->cm_id->device;
+ struct smbdirect_socket *sc = &t->socket;
+ struct ib_device *device = sc->ib.dev;
int max_send_sges, max_rw_wrs, max_send_wrs;
unsigned int max_sge_per_wr, wrs_per_credit;
@@ -1851,34 +1874,35 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
static int smb_direct_create_qpair(struct smb_direct_transport *t,
struct ib_qp_cap *cap)
{
+ struct smbdirect_socket *sc = &t->socket;
int ret;
struct ib_qp_init_attr qp_attr;
int pages_per_rw;
- t->pd = ib_alloc_pd(t->cm_id->device, 0);
- if (IS_ERR(t->pd)) {
+ sc->ib.pd = ib_alloc_pd(sc->ib.dev, 0);
+ if (IS_ERR(sc->ib.pd)) {
pr_err("Can't create RDMA PD\n");
- ret = PTR_ERR(t->pd);
- t->pd = NULL;
+ ret = PTR_ERR(sc->ib.pd);
+ sc->ib.pd = NULL;
return ret;
}
- t->send_cq = ib_alloc_cq(t->cm_id->device, t,
+ sc->ib.send_cq = ib_alloc_cq(sc->ib.dev, t,
smb_direct_send_credit_target + cap->max_rdma_ctxs,
0, IB_POLL_WORKQUEUE);
- if (IS_ERR(t->send_cq)) {
+ if (IS_ERR(sc->ib.send_cq)) {
pr_err("Can't create RDMA send CQ\n");
- ret = PTR_ERR(t->send_cq);
- t->send_cq = NULL;
+ ret = PTR_ERR(sc->ib.send_cq);
+ sc->ib.send_cq = NULL;
goto err;
}
- t->recv_cq = ib_alloc_cq(t->cm_id->device, t,
+ sc->ib.recv_cq = ib_alloc_cq(sc->ib.dev, t,
t->recv_credit_max, 0, IB_POLL_WORKQUEUE);
- if (IS_ERR(t->recv_cq)) {
+ if (IS_ERR(sc->ib.recv_cq)) {
pr_err("Can't create RDMA recv CQ\n");
- ret = PTR_ERR(t->recv_cq);
- t->recv_cq = NULL;
+ ret = PTR_ERR(sc->ib.recv_cq);
+ sc->ib.recv_cq = NULL;
goto err;
}
@@ -1888,22 +1912,22 @@ static int smb_direct_create_qpair(struct smb_direct_transport *t,
qp_attr.cap = *cap;
qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR;
qp_attr.qp_type = IB_QPT_RC;
- qp_attr.send_cq = t->send_cq;
- qp_attr.recv_cq = t->recv_cq;
+ qp_attr.send_cq = sc->ib.send_cq;
+ qp_attr.recv_cq = sc->ib.recv_cq;
qp_attr.port_num = ~0;
- ret = rdma_create_qp(t->cm_id, t->pd, &qp_attr);
+ ret = rdma_create_qp(sc->rdma.cm_id, sc->ib.pd, &qp_attr);
if (ret) {
pr_err("Can't create RDMA QP: %d\n", ret);
goto err;
}
- t->qp = t->cm_id->qp;
- t->cm_id->event_handler = smb_direct_cm_handler;
+ sc->ib.qp = sc->rdma.cm_id->qp;
+ sc->rdma.cm_id->event_handler = smb_direct_cm_handler;
pages_per_rw = DIV_ROUND_UP(t->max_rdma_rw_size, PAGE_SIZE) + 1;
- if (pages_per_rw > t->cm_id->device->attrs.max_sgl_rd) {
- ret = ib_mr_pool_init(t->qp, &t->qp->rdma_mrs,
+ if (pages_per_rw > sc->ib.dev->attrs.max_sgl_rd) {
+ ret = ib_mr_pool_init(sc->ib.qp, &sc->ib.qp->rdma_mrs,
t->max_rw_credits, IB_MR_TYPE_MEM_REG,
t->pages_per_rw_credit, 0);
if (ret) {
@@ -1915,21 +1939,21 @@ static int smb_direct_create_qpair(struct smb_direct_transport *t,
return 0;
err:
- if (t->qp) {
- t->qp = NULL;
- rdma_destroy_qp(t->cm_id);
+ if (sc->ib.qp) {
+ sc->ib.qp = NULL;
+ rdma_destroy_qp(sc->rdma.cm_id);
}
- if (t->recv_cq) {
- ib_destroy_cq(t->recv_cq);
- t->recv_cq = NULL;
+ if (sc->ib.recv_cq) {
+ ib_destroy_cq(sc->ib.recv_cq);
+ sc->ib.recv_cq = NULL;
}
- if (t->send_cq) {
- ib_destroy_cq(t->send_cq);
- t->send_cq = NULL;
+ if (sc->ib.send_cq) {
+ ib_destroy_cq(sc->ib.send_cq);
+ sc->ib.send_cq = NULL;
}
- if (t->pd) {
- ib_dealloc_pd(t->pd);
- t->pd = NULL;
+ if (sc->ib.pd) {
+ ib_dealloc_pd(sc->ib.pd);
+ sc->ib.pd = NULL;
}
return ret;
}
@@ -1937,6 +1961,7 @@ static int smb_direct_create_qpair(struct smb_direct_transport *t,
static int smb_direct_prepare(struct ksmbd_transport *t)
{
struct smb_direct_transport *st = smb_trans_direct_transfort(t);
+ struct smbdirect_socket *sc = &st->socket;
struct smb_direct_recvmsg *recvmsg;
struct smbdirect_negotiate_req *req;
int ret;
@@ -1944,9 +1969,9 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
ksmbd_debug(RDMA, "Waiting for SMB_DIRECT negotiate request\n");
ret = wait_event_interruptible_timeout(st->wait_status,
st->negotiation_requested ||
- st->status == SMB_DIRECT_CS_DISCONNECTED,
+ sc->status == SMBDIRECT_SOCKET_DISCONNECTED,
SMB_DIRECT_NEGOTIATE_TIMEOUT * HZ);
- if (ret <= 0 || st->status == SMB_DIRECT_CS_DISCONNECTED)
+ if (ret <= 0 || sc->status == SMBDIRECT_SOCKET_DISCONNECTED)
return ret < 0 ? ret : -ETIMEDOUT;
recvmsg = get_first_reassembly(st);
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 11/18] smb: server: make use of common smbdirect_socket_parameters
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (9 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 10/18] smb: server: make use of common smbdirect_socket Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 12/18] smb: server: make use of smbdirect_socket->recv_io.expected Stefan Metzmacher
` (9 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon,
Hyunchul Lee, Meetakshi Setiya
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Hyunchul Lee <hyc.lee@gmail.com>
Cc: Meetakshi Setiya <meetakshisetiyaoss@gmail.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/transport_rdma.c | 85 ++++++++++++++++++----------------
1 file changed, 44 insertions(+), 41 deletions(-)
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index 1c852c8601c6..d81f1694a9af 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -98,12 +98,6 @@ struct smb_direct_transport {
bool full_packet_received;
wait_queue_head_t wait_status;
- int max_send_size;
- int max_recv_size;
- int max_fragmented_send_size;
- int max_fragmented_recv_size;
- int max_rdma_rw_size;
-
spinlock_t reassembly_queue_lock;
struct list_head reassembly_queue;
int reassembly_data_length;
@@ -114,13 +108,11 @@ struct smb_direct_transport {
spinlock_t receive_credit_lock;
int recv_credits;
int count_avail_recvmsg;
- int recv_credit_max;
int recv_credit_target;
spinlock_t recvmsg_queue_lock;
struct list_head recvmsg_queue;
- int send_credit_target;
atomic_t send_credits;
spinlock_t lock_new_recv_credits;
int new_recv_credits;
@@ -635,16 +627,18 @@ static int smb_direct_post_recv(struct smb_direct_transport *t,
struct smb_direct_recvmsg *recvmsg)
{
struct smbdirect_socket *sc = &t->socket;
+ struct smbdirect_socket_parameters *sp = &sc->parameters;
struct ib_recv_wr wr;
int ret;
recvmsg->sge.addr = ib_dma_map_single(sc->ib.dev,
- recvmsg->packet, t->max_recv_size,
+ recvmsg->packet,
+ sp->max_recv_size,
DMA_FROM_DEVICE);
ret = ib_dma_mapping_error(sc->ib.dev, recvmsg->sge.addr);
if (ret)
return ret;
- recvmsg->sge.length = t->max_recv_size;
+ recvmsg->sge.length = sp->max_recv_size;
recvmsg->sge.lkey = sc->ib.pd->local_dma_lkey;
recvmsg->cqe.done = recv_done;
@@ -1010,6 +1004,7 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
struct smb_direct_sendmsg **sendmsg_out)
{
struct smbdirect_socket *sc = &t->socket;
+ struct smbdirect_socket_parameters *sp = &sc->parameters;
struct smb_direct_sendmsg *sendmsg;
struct smbdirect_data_transfer *packet;
int header_length;
@@ -1021,7 +1016,7 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
/* Fill in the packet header */
packet = (struct smbdirect_data_transfer *)sendmsg->packet;
- packet->credits_requested = cpu_to_le16(t->send_credit_target);
+ packet->credits_requested = cpu_to_le16(sp->send_credit_target);
packet->credits_granted = cpu_to_le16(manage_credits_prior_sending(t));
packet->flags = 0;
@@ -1222,9 +1217,10 @@ static int smb_direct_writev(struct ksmbd_transport *t,
{
struct smb_direct_transport *st = smb_trans_direct_transfort(t);
struct smbdirect_socket *sc = &st->socket;
+ struct smbdirect_socket_parameters *sp = &sc->parameters;
int remaining_data_length;
int start, i, j;
- int max_iov_size = st->max_send_size -
+ int max_iov_size = sp->max_send_size -
sizeof(struct smbdirect_data_transfer);
int ret;
struct kvec vec;
@@ -1354,6 +1350,7 @@ static int smb_direct_rdma_xmit(struct smb_direct_transport *t,
bool is_read)
{
struct smbdirect_socket *sc = &t->socket;
+ struct smbdirect_socket_parameters *sp = &sc->parameters;
struct smb_direct_rdma_rw_msg *msg, *next_msg;
int i, ret;
DECLARE_COMPLETION_ONSTACK(completion);
@@ -1366,7 +1363,7 @@ static int smb_direct_rdma_xmit(struct smb_direct_transport *t,
if (sc->status != SMBDIRECT_SOCKET_CONNECTED)
return -ENOTCONN;
- if (buf_len > t->max_rdma_rw_size)
+ if (buf_len > sp->max_read_write_size)
return -EINVAL;
/* calculate needed credits */
@@ -1580,6 +1577,7 @@ static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
int failed)
{
struct smbdirect_socket *sc = &t->socket;
+ struct smbdirect_socket_parameters *sp = &sc->parameters;
struct smb_direct_sendmsg *sendmsg;
struct smbdirect_negotiate_resp *resp;
int ret;
@@ -1601,13 +1599,13 @@ static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
resp->negotiated_version = SMB_DIRECT_VERSION_LE;
resp->reserved = 0;
resp->credits_requested =
- cpu_to_le16(t->send_credit_target);
+ cpu_to_le16(sp->send_credit_target);
resp->credits_granted = cpu_to_le16(manage_credits_prior_sending(t));
- resp->max_readwrite_size = cpu_to_le32(t->max_rdma_rw_size);
- resp->preferred_send_size = cpu_to_le32(t->max_send_size);
- resp->max_receive_size = cpu_to_le32(t->max_recv_size);
+ resp->max_readwrite_size = cpu_to_le32(sp->max_read_write_size);
+ resp->preferred_send_size = cpu_to_le32(sp->max_send_size);
+ resp->max_receive_size = cpu_to_le32(sp->max_recv_size);
resp->max_fragmented_size =
- cpu_to_le32(t->max_fragmented_recv_size);
+ cpu_to_le32(sp->max_fragmented_recv_size);
}
sendmsg->sge[0].addr = ib_dma_map_single(sc->ib.dev,
@@ -1714,6 +1712,7 @@ static int smb_direct_init_params(struct smb_direct_transport *t,
struct ib_qp_cap *cap)
{
struct smbdirect_socket *sc = &t->socket;
+ struct smbdirect_socket_parameters *sp = &sc->parameters;
struct ib_device *device = sc->ib.dev;
int max_send_sges, max_rw_wrs, max_send_wrs;
unsigned int max_sge_per_wr, wrs_per_credit;
@@ -1721,10 +1720,10 @@ static int smb_direct_init_params(struct smb_direct_transport *t,
/* need 3 more sge. because a SMB_DIRECT header, SMB2 header,
* SMB2 response could be mapped.
*/
- t->max_send_size = smb_direct_max_send_size;
- max_send_sges = DIV_ROUND_UP(t->max_send_size, PAGE_SIZE) + 3;
+ sp->max_send_size = smb_direct_max_send_size;
+ max_send_sges = DIV_ROUND_UP(sp->max_send_size, PAGE_SIZE) + 3;
if (max_send_sges > SMB_DIRECT_MAX_SEND_SGES) {
- pr_err("max_send_size %d is too large\n", t->max_send_size);
+ pr_err("max_send_size %d is too large\n", sp->max_send_size);
return -EINVAL;
}
@@ -1735,9 +1734,9 @@ static int smb_direct_init_params(struct smb_direct_transport *t,
* are needed for MR registration, RDMA R/W, local & remote
* MR invalidation.
*/
- t->max_rdma_rw_size = smb_direct_max_read_write_size;
+ sp->max_read_write_size = smb_direct_max_read_write_size;
t->pages_per_rw_credit = smb_direct_get_max_fr_pages(t);
- t->max_rw_credits = DIV_ROUND_UP(t->max_rdma_rw_size,
+ t->max_rw_credits = DIV_ROUND_UP(sp->max_read_write_size,
(t->pages_per_rw_credit - 1) *
PAGE_SIZE);
@@ -1778,20 +1777,20 @@ static int smb_direct_init_params(struct smb_direct_transport *t,
t->recv_credits = 0;
t->count_avail_recvmsg = 0;
- t->recv_credit_max = smb_direct_receive_credit_max;
+ sp->recv_credit_max = smb_direct_receive_credit_max;
t->recv_credit_target = 10;
t->new_recv_credits = 0;
- t->send_credit_target = smb_direct_send_credit_target;
+ sp->send_credit_target = smb_direct_send_credit_target;
atomic_set(&t->send_credits, 0);
atomic_set(&t->rw_credits, t->max_rw_credits);
- t->max_send_size = smb_direct_max_send_size;
- t->max_recv_size = smb_direct_max_receive_size;
- t->max_fragmented_recv_size = smb_direct_max_fragmented_recv_size;
+ sp->max_send_size = smb_direct_max_send_size;
+ sp->max_recv_size = smb_direct_max_receive_size;
+ sp->max_fragmented_recv_size = smb_direct_max_fragmented_recv_size;
cap->max_send_wr = max_send_wrs;
- cap->max_recv_wr = t->recv_credit_max;
+ cap->max_recv_wr = sp->recv_credit_max;
cap->max_send_sge = max_sge_per_wr;
cap->max_recv_sge = SMB_DIRECT_MAX_RECV_SGES;
cap->max_inline_data = 0;
@@ -1821,6 +1820,8 @@ static void smb_direct_destroy_pools(struct smb_direct_transport *t)
static int smb_direct_create_pools(struct smb_direct_transport *t)
{
+ struct smbdirect_socket *sc = &t->socket;
+ struct smbdirect_socket_parameters *sp = &sc->parameters;
char name[80];
int i;
struct smb_direct_recvmsg *recvmsg;
@@ -1833,7 +1834,7 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
if (!t->sendmsg_cache)
return -ENOMEM;
- t->sendmsg_mempool = mempool_create(t->send_credit_target,
+ t->sendmsg_mempool = mempool_create(sp->send_credit_target,
mempool_alloc_slab, mempool_free_slab,
t->sendmsg_cache);
if (!t->sendmsg_mempool)
@@ -1842,20 +1843,20 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
snprintf(name, sizeof(name), "smb_direct_resp_%p", t);
t->recvmsg_cache = kmem_cache_create(name,
sizeof(struct smb_direct_recvmsg) +
- t->max_recv_size,
+ sp->max_recv_size,
0, SLAB_HWCACHE_ALIGN, NULL);
if (!t->recvmsg_cache)
goto err;
t->recvmsg_mempool =
- mempool_create(t->recv_credit_max, mempool_alloc_slab,
+ mempool_create(sp->recv_credit_max, mempool_alloc_slab,
mempool_free_slab, t->recvmsg_cache);
if (!t->recvmsg_mempool)
goto err;
INIT_LIST_HEAD(&t->recvmsg_queue);
- for (i = 0; i < t->recv_credit_max; i++) {
+ for (i = 0; i < sp->recv_credit_max; i++) {
recvmsg = mempool_alloc(t->recvmsg_mempool, KSMBD_DEFAULT_GFP);
if (!recvmsg)
goto err;
@@ -1863,7 +1864,7 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
recvmsg->sge.length = 0;
list_add(&recvmsg->list, &t->recvmsg_queue);
}
- t->count_avail_recvmsg = t->recv_credit_max;
+ t->count_avail_recvmsg = sp->recv_credit_max;
return 0;
err:
@@ -1875,6 +1876,7 @@ static int smb_direct_create_qpair(struct smb_direct_transport *t,
struct ib_qp_cap *cap)
{
struct smbdirect_socket *sc = &t->socket;
+ struct smbdirect_socket_parameters *sp = &sc->parameters;
int ret;
struct ib_qp_init_attr qp_attr;
int pages_per_rw;
@@ -1898,7 +1900,7 @@ static int smb_direct_create_qpair(struct smb_direct_transport *t,
}
sc->ib.recv_cq = ib_alloc_cq(sc->ib.dev, t,
- t->recv_credit_max, 0, IB_POLL_WORKQUEUE);
+ sp->recv_credit_max, 0, IB_POLL_WORKQUEUE);
if (IS_ERR(sc->ib.recv_cq)) {
pr_err("Can't create RDMA recv CQ\n");
ret = PTR_ERR(sc->ib.recv_cq);
@@ -1925,7 +1927,7 @@ static int smb_direct_create_qpair(struct smb_direct_transport *t,
sc->ib.qp = sc->rdma.cm_id->qp;
sc->rdma.cm_id->event_handler = smb_direct_cm_handler;
- pages_per_rw = DIV_ROUND_UP(t->max_rdma_rw_size, PAGE_SIZE) + 1;
+ pages_per_rw = DIV_ROUND_UP(sp->max_read_write_size, PAGE_SIZE) + 1;
if (pages_per_rw > sc->ib.dev->attrs.max_sgl_rd) {
ret = ib_mr_pool_init(sc->ib.qp, &sc->ib.qp->rdma_mrs,
t->max_rw_credits, IB_MR_TYPE_MEM_REG,
@@ -1962,6 +1964,7 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
{
struct smb_direct_transport *st = smb_trans_direct_transfort(t);
struct smbdirect_socket *sc = &st->socket;
+ struct smbdirect_socket_parameters *sp = &sc->parameters;
struct smb_direct_recvmsg *recvmsg;
struct smbdirect_negotiate_req *req;
int ret;
@@ -1983,14 +1986,14 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
goto out;
req = (struct smbdirect_negotiate_req *)recvmsg->packet;
- st->max_recv_size = min_t(int, st->max_recv_size,
+ sp->max_recv_size = min_t(int, sp->max_recv_size,
le32_to_cpu(req->preferred_send_size));
- st->max_send_size = min_t(int, st->max_send_size,
+ sp->max_send_size = min_t(int, sp->max_send_size,
le32_to_cpu(req->max_receive_size));
- st->max_fragmented_send_size =
+ sp->max_fragmented_send_size =
le32_to_cpu(req->max_fragmented_size);
- st->max_fragmented_recv_size =
- (st->recv_credit_max * st->max_recv_size) / 2;
+ sp->max_fragmented_recv_size =
+ (sp->recv_credit_max * sp->max_recv_size) / 2;
ret = smb_direct_send_negotiate_response(st, ret);
out:
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 12/18] smb: server: make use of smbdirect_socket->recv_io.expected
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (10 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 11/18] smb: server: make use of common smbdirect_socket_parameters Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 13/18] smb: server: make use of struct smbdirect_recv_io Stefan Metzmacher
` (8 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Namjae Jeon, Steve French, Tom Talpey
The expected incoming message type can be per connection.
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/transport_rdma.c | 44 +++++++++++++++++++---------------
1 file changed, 25 insertions(+), 19 deletions(-)
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index d81f1694a9af..14d338a380a3 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -141,10 +141,6 @@ struct smb_direct_transport {
#define KSMBD_TRANS(t) ((struct ksmbd_transport *)&((t)->transport))
#define SMBD_TRANS(t) ((struct smb_direct_transport *)container_of(t, \
struct smb_direct_transport, transport))
-enum {
- SMB_DIRECT_MSG_NEGOTIATE_REQ = 0,
- SMB_DIRECT_MSG_DATA_TRANSFER
-};
static const struct ksmbd_transport_ops ksmbd_smb_direct_transport_ops;
@@ -168,7 +164,6 @@ struct smb_direct_sendmsg {
struct smb_direct_recvmsg {
struct smb_direct_transport *transport;
struct list_head list;
- int type;
struct ib_sge sge;
struct ib_cqe cqe;
bool first_segment;
@@ -465,8 +460,10 @@ static void smb_direct_free_sendmsg(struct smb_direct_transport *t,
static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
{
- switch (recvmsg->type) {
- case SMB_DIRECT_MSG_DATA_TRANSFER: {
+ struct smbdirect_socket *sc = &recvmsg->transport->socket;
+
+ switch (sc->recv_io.expected) {
+ case SMBDIRECT_EXPECT_DATA_TRANSFER: {
struct smbdirect_data_transfer *req =
(struct smbdirect_data_transfer *)recvmsg->packet;
struct smb2_hdr *hdr = (struct smb2_hdr *)(recvmsg->packet
@@ -477,9 +474,9 @@ static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
le16_to_cpu(req->credits_requested),
req->data_length, req->remaining_data_length,
hdr->ProtocolId, hdr->Command);
- break;
+ return 0;
}
- case SMB_DIRECT_MSG_NEGOTIATE_REQ: {
+ case SMBDIRECT_EXPECT_NEGOTIATE_REQ: {
struct smbdirect_negotiate_req *req =
(struct smbdirect_negotiate_req *)recvmsg->packet;
ksmbd_debug(RDMA,
@@ -499,12 +496,15 @@ static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
128 * 1024)
return -ECONNABORTED;
- break;
+ return 0;
}
- default:
- return -EINVAL;
+ case SMBDIRECT_EXPECT_NEGOTIATE_REP:
+ /* client only */
+ break;
}
- return 0;
+
+ /* This is an internal error */
+ return -EINVAL;
}
static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
@@ -535,8 +535,8 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
ib_dma_sync_single_for_cpu(wc->qp->device, recvmsg->sge.addr,
recvmsg->sge.length, DMA_FROM_DEVICE);
- switch (recvmsg->type) {
- case SMB_DIRECT_MSG_NEGOTIATE_REQ:
+ switch (sc->recv_io.expected) {
+ case SMBDIRECT_EXPECT_NEGOTIATE_REQ:
if (wc->byte_len < sizeof(struct smbdirect_negotiate_req)) {
put_recvmsg(t, recvmsg);
smb_direct_disconnect_rdma_connection(t);
@@ -548,7 +548,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
enqueue_reassembly(t, recvmsg, 0);
wake_up_interruptible(&t->wait_status);
return;
- case SMB_DIRECT_MSG_DATA_TRANSFER: {
+ case SMBDIRECT_EXPECT_DATA_TRANSFER: {
struct smbdirect_data_transfer *data_transfer =
(struct smbdirect_data_transfer *)recvmsg->packet;
unsigned int data_length;
@@ -613,12 +613,15 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
return;
}
+ case SMBDIRECT_EXPECT_NEGOTIATE_REP:
+ /* client only */
+ break;
}
/*
* This is an internal error!
*/
- WARN_ON_ONCE(recvmsg->type != SMB_DIRECT_MSG_DATA_TRANSFER);
+ WARN_ON_ONCE(sc->recv_io.expected != SMBDIRECT_EXPECT_DATA_TRANSFER);
put_recvmsg(t, recvmsg);
smb_direct_disconnect_rdma_connection(t);
}
@@ -807,7 +810,6 @@ static void smb_direct_post_recv_credits(struct work_struct *work)
if (!recvmsg)
break;
- recvmsg->type = SMB_DIRECT_MSG_DATA_TRANSFER;
recvmsg->first_segment = false;
ret = smb_direct_post_recv(t, recvmsg);
@@ -1606,6 +1608,8 @@ static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
resp->max_receive_size = cpu_to_le32(sp->max_recv_size);
resp->max_fragmented_size =
cpu_to_le32(sp->max_fragmented_recv_size);
+
+ sc->recv_io.expected = SMBDIRECT_EXPECT_DATA_TRANSFER;
}
sendmsg->sge[0].addr = ib_dma_map_single(sc->ib.dev,
@@ -1671,13 +1675,15 @@ static int smb_direct_accept_client(struct smb_direct_transport *t)
static int smb_direct_prepare_negotiation(struct smb_direct_transport *t)
{
+ struct smbdirect_socket *sc = &t->socket;
int ret;
struct smb_direct_recvmsg *recvmsg;
+ sc->recv_io.expected = SMBDIRECT_EXPECT_NEGOTIATE_REQ;
+
recvmsg = get_free_recvmsg(t);
if (!recvmsg)
return -ENOMEM;
- recvmsg->type = SMB_DIRECT_MSG_NEGOTIATE_REQ;
ret = smb_direct_post_recv(t, recvmsg);
if (ret) {
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 13/18] smb: server: make use of struct smbdirect_recv_io
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (11 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 12/18] smb: server: make use of smbdirect_socket->recv_io.expected Stefan Metzmacher
@ 2025-08-06 17:35 ` Stefan Metzmacher
2025-08-06 17:36 ` [PATCH 14/18] smb: server: make use of smbdirect_socket.recv_io.free.{list,lock} Stefan Metzmacher
` (7 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:35 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Namjae Jeon, Steve French, Tom Talpey
This will allow us to move helper functions into common code soon
as the client already uses smbdirect_recv_io.
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/transport_rdma.c | 61 +++++++++++++++-------------------
1 file changed, 26 insertions(+), 35 deletions(-)
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index 14d338a380a3..26d70396b0c1 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -161,15 +161,6 @@ struct smb_direct_sendmsg {
u8 packet[];
};
-struct smb_direct_recvmsg {
- struct smb_direct_transport *transport;
- struct list_head list;
- struct ib_sge sge;
- struct ib_cqe cqe;
- bool first_segment;
- u8 packet[];
-};
-
struct smb_direct_rdma_rw_msg {
struct smb_direct_transport *t;
struct ib_cqe cqe;
@@ -212,7 +203,7 @@ smb_trans_direct_transfort(struct ksmbd_transport *t)
}
static inline void
-*smb_direct_recvmsg_payload(struct smb_direct_recvmsg *recvmsg)
+*smbdirect_recv_io_payload(struct smbdirect_recv_io *recvmsg)
{
return (void *)recvmsg->packet;
}
@@ -225,14 +216,14 @@ static inline bool is_receive_credit_post_required(int receive_credits,
}
static struct
-smb_direct_recvmsg *get_free_recvmsg(struct smb_direct_transport *t)
+smbdirect_recv_io *get_free_recvmsg(struct smb_direct_transport *t)
{
- struct smb_direct_recvmsg *recvmsg = NULL;
+ struct smbdirect_recv_io *recvmsg = NULL;
spin_lock(&t->recvmsg_queue_lock);
if (!list_empty(&t->recvmsg_queue)) {
recvmsg = list_first_entry(&t->recvmsg_queue,
- struct smb_direct_recvmsg,
+ struct smbdirect_recv_io,
list);
list_del(&recvmsg->list);
}
@@ -241,7 +232,7 @@ smb_direct_recvmsg *get_free_recvmsg(struct smb_direct_transport *t)
}
static void put_recvmsg(struct smb_direct_transport *t,
- struct smb_direct_recvmsg *recvmsg)
+ struct smbdirect_recv_io *recvmsg)
{
struct smbdirect_socket *sc = &t->socket;
@@ -259,7 +250,7 @@ static void put_recvmsg(struct smb_direct_transport *t,
}
static void enqueue_reassembly(struct smb_direct_transport *t,
- struct smb_direct_recvmsg *recvmsg,
+ struct smbdirect_recv_io *recvmsg,
int data_length)
{
spin_lock(&t->reassembly_queue_lock);
@@ -276,11 +267,11 @@ static void enqueue_reassembly(struct smb_direct_transport *t,
spin_unlock(&t->reassembly_queue_lock);
}
-static struct smb_direct_recvmsg *get_first_reassembly(struct smb_direct_transport *t)
+static struct smbdirect_recv_io *get_first_reassembly(struct smb_direct_transport *t)
{
if (!list_empty(&t->reassembly_queue))
return list_first_entry(&t->reassembly_queue,
- struct smb_direct_recvmsg, list);
+ struct smbdirect_recv_io, list);
else
return NULL;
}
@@ -380,7 +371,7 @@ static void smb_direct_free_transport(struct ksmbd_transport *kt)
static void free_transport(struct smb_direct_transport *t)
{
struct smbdirect_socket *sc = &t->socket;
- struct smb_direct_recvmsg *recvmsg;
+ struct smbdirect_recv_io *recvmsg;
wake_up_interruptible(&t->wait_send_credits);
@@ -458,9 +449,9 @@ static void smb_direct_free_sendmsg(struct smb_direct_transport *t,
mempool_free(msg, t->sendmsg_mempool);
}
-static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
+static int smb_direct_check_recvmsg(struct smbdirect_recv_io *recvmsg)
{
- struct smbdirect_socket *sc = &recvmsg->transport->socket;
+ struct smbdirect_socket *sc = recvmsg->socket;
switch (sc->recv_io.expected) {
case SMBDIRECT_EXPECT_DATA_TRANSFER: {
@@ -509,13 +500,13 @@ static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
{
- struct smb_direct_recvmsg *recvmsg;
+ struct smbdirect_recv_io *recvmsg;
struct smb_direct_transport *t;
struct smbdirect_socket *sc;
- recvmsg = container_of(wc->wr_cqe, struct smb_direct_recvmsg, cqe);
- t = recvmsg->transport;
- sc = &t->socket;
+ recvmsg = container_of(wc->wr_cqe, struct smbdirect_recv_io, cqe);
+ sc = recvmsg->socket;
+ t = container_of(sc, struct smb_direct_transport, socket);
if (wc->status != IB_WC_SUCCESS || wc->opcode != IB_WC_RECV) {
put_recvmsg(t, recvmsg);
@@ -627,7 +618,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
}
static int smb_direct_post_recv(struct smb_direct_transport *t,
- struct smb_direct_recvmsg *recvmsg)
+ struct smbdirect_recv_io *recvmsg)
{
struct smbdirect_socket *sc = &t->socket;
struct smbdirect_socket_parameters *sp = &sc->parameters;
@@ -666,7 +657,7 @@ static int smb_direct_post_recv(struct smb_direct_transport *t,
static int smb_direct_read(struct ksmbd_transport *t, char *buf,
unsigned int size, int unused)
{
- struct smb_direct_recvmsg *recvmsg;
+ struct smbdirect_recv_io *recvmsg;
struct smbdirect_data_transfer *data_transfer;
int to_copy, to_read, data_read, offset;
u32 data_length, remaining_data_length, data_offset;
@@ -703,7 +694,7 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
offset = st->first_entry_offset;
while (data_read < size) {
recvmsg = get_first_reassembly(st);
- data_transfer = smb_direct_recvmsg_payload(recvmsg);
+ data_transfer = smbdirect_recv_io_payload(recvmsg);
data_length = le32_to_cpu(data_transfer->data_length);
remaining_data_length =
le32_to_cpu(data_transfer->remaining_data_length);
@@ -796,7 +787,7 @@ static void smb_direct_post_recv_credits(struct work_struct *work)
{
struct smb_direct_transport *t = container_of(work,
struct smb_direct_transport, post_recv_credits_work.work);
- struct smb_direct_recvmsg *recvmsg;
+ struct smbdirect_recv_io *recvmsg;
int receive_credits, credits = 0;
int ret;
@@ -1677,7 +1668,7 @@ static int smb_direct_prepare_negotiation(struct smb_direct_transport *t)
{
struct smbdirect_socket *sc = &t->socket;
int ret;
- struct smb_direct_recvmsg *recvmsg;
+ struct smbdirect_recv_io *recvmsg;
sc->recv_io.expected = SMBDIRECT_EXPECT_NEGOTIATE_REQ;
@@ -1806,7 +1797,7 @@ static int smb_direct_init_params(struct smb_direct_transport *t,
static void smb_direct_destroy_pools(struct smb_direct_transport *t)
{
- struct smb_direct_recvmsg *recvmsg;
+ struct smbdirect_recv_io *recvmsg;
while ((recvmsg = get_free_recvmsg(t)))
mempool_free(recvmsg, t->recvmsg_mempool);
@@ -1830,7 +1821,7 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
struct smbdirect_socket_parameters *sp = &sc->parameters;
char name[80];
int i;
- struct smb_direct_recvmsg *recvmsg;
+ struct smbdirect_recv_io *recvmsg;
snprintf(name, sizeof(name), "smb_direct_rqst_pool_%p", t);
t->sendmsg_cache = kmem_cache_create(name,
@@ -1846,9 +1837,9 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
if (!t->sendmsg_mempool)
goto err;
- snprintf(name, sizeof(name), "smb_direct_resp_%p", t);
+ snprintf(name, sizeof(name), "smbdirect_recv_io_pool_%p", t);
t->recvmsg_cache = kmem_cache_create(name,
- sizeof(struct smb_direct_recvmsg) +
+ sizeof(struct smbdirect_recv_io) +
sp->max_recv_size,
0, SLAB_HWCACHE_ALIGN, NULL);
if (!t->recvmsg_cache)
@@ -1866,7 +1857,7 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
recvmsg = mempool_alloc(t->recvmsg_mempool, KSMBD_DEFAULT_GFP);
if (!recvmsg)
goto err;
- recvmsg->transport = t;
+ recvmsg->socket = sc;
recvmsg->sge.length = 0;
list_add(&recvmsg->list, &t->recvmsg_queue);
}
@@ -1971,7 +1962,7 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
struct smb_direct_transport *st = smb_trans_direct_transfort(t);
struct smbdirect_socket *sc = &st->socket;
struct smbdirect_socket_parameters *sp = &sc->parameters;
- struct smb_direct_recvmsg *recvmsg;
+ struct smbdirect_recv_io *recvmsg;
struct smbdirect_negotiate_req *req;
int ret;
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 14/18] smb: server: make use of smbdirect_socket.recv_io.free.{list,lock}
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (12 preceding siblings ...)
2025-08-06 17:35 ` [PATCH 13/18] smb: server: make use of struct smbdirect_recv_io Stefan Metzmacher
@ 2025-08-06 17:36 ` Stefan Metzmacher
2025-08-06 17:36 ` [PATCH 15/18] smb: server: make use of smbdirect_socket.recv_io.reassembly.* Stefan Metzmacher
` (6 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:36 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Namjae Jeon, Steve French, Tom Talpey
This is already used by the client and will allow us to
add common helper functions soon.
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/transport_rdma.c | 27 ++++++++++++---------------
1 file changed, 12 insertions(+), 15 deletions(-)
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index 26d70396b0c1..aebd29242a2b 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -110,9 +110,6 @@ struct smb_direct_transport {
int count_avail_recvmsg;
int recv_credit_target;
- spinlock_t recvmsg_queue_lock;
- struct list_head recvmsg_queue;
-
atomic_t send_credits;
spinlock_t lock_new_recv_credits;
int new_recv_credits;
@@ -218,16 +215,17 @@ static inline bool is_receive_credit_post_required(int receive_credits,
static struct
smbdirect_recv_io *get_free_recvmsg(struct smb_direct_transport *t)
{
+ struct smbdirect_socket *sc = &t->socket;
struct smbdirect_recv_io *recvmsg = NULL;
- spin_lock(&t->recvmsg_queue_lock);
- if (!list_empty(&t->recvmsg_queue)) {
- recvmsg = list_first_entry(&t->recvmsg_queue,
+ spin_lock(&sc->recv_io.free.lock);
+ if (!list_empty(&sc->recv_io.free.list)) {
+ recvmsg = list_first_entry(&sc->recv_io.free.list,
struct smbdirect_recv_io,
list);
list_del(&recvmsg->list);
}
- spin_unlock(&t->recvmsg_queue_lock);
+ spin_unlock(&sc->recv_io.free.lock);
return recvmsg;
}
@@ -244,9 +242,9 @@ static void put_recvmsg(struct smb_direct_transport *t,
recvmsg->sge.length = 0;
}
- spin_lock(&t->recvmsg_queue_lock);
- list_add(&recvmsg->list, &t->recvmsg_queue);
- spin_unlock(&t->recvmsg_queue_lock);
+ spin_lock(&sc->recv_io.free.lock);
+ list_add(&recvmsg->list, &sc->recv_io.free.list);
+ spin_unlock(&sc->recv_io.free.lock);
}
static void enqueue_reassembly(struct smb_direct_transport *t,
@@ -326,6 +324,9 @@ static struct smb_direct_transport *alloc_transport(struct rdma_cm_id *cm_id)
sc->ib.dev = sc->rdma.cm_id->device;
+ INIT_LIST_HEAD(&sc->recv_io.free.list);
+ spin_lock_init(&sc->recv_io.free.lock);
+
sc->status = SMBDIRECT_SOCKET_CREATED;
init_waitqueue_head(&t->wait_status);
@@ -338,8 +339,6 @@ static struct smb_direct_transport *alloc_transport(struct rdma_cm_id *cm_id)
init_waitqueue_head(&t->wait_rw_credits);
spin_lock_init(&t->receive_credit_lock);
- spin_lock_init(&t->recvmsg_queue_lock);
- INIT_LIST_HEAD(&t->recvmsg_queue);
init_waitqueue_head(&t->wait_send_pending);
atomic_set(&t->send_pending, 0);
@@ -1851,15 +1850,13 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
if (!t->recvmsg_mempool)
goto err;
- INIT_LIST_HEAD(&t->recvmsg_queue);
-
for (i = 0; i < sp->recv_credit_max; i++) {
recvmsg = mempool_alloc(t->recvmsg_mempool, KSMBD_DEFAULT_GFP);
if (!recvmsg)
goto err;
recvmsg->socket = sc;
recvmsg->sge.length = 0;
- list_add(&recvmsg->list, &t->recvmsg_queue);
+ list_add(&recvmsg->list, &sc->recv_io.free.list);
}
t->count_avail_recvmsg = sp->recv_credit_max;
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 15/18] smb: server: make use of smbdirect_socket.recv_io.reassembly.*
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (13 preceding siblings ...)
2025-08-06 17:36 ` [PATCH 14/18] smb: server: make use of smbdirect_socket.recv_io.free.{list,lock} Stefan Metzmacher
@ 2025-08-06 17:36 ` Stefan Metzmacher
2025-08-06 17:36 ` [PATCH 16/18] smb: server: make use of SMBDIRECT_RECV_IO_MAX_SGE Stefan Metzmacher
` (5 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:36 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Namjae Jeon, Steve French, Tom Talpey
This is also used by the client and will allow us to introduce
common helper functions soon.
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/transport_rdma.c | 90 ++++++++++++++++------------------
1 file changed, 43 insertions(+), 47 deletions(-)
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index aebd29242a2b..7fcd80c329d7 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -95,16 +95,8 @@ struct smb_direct_transport {
struct smbdirect_socket socket;
- bool full_packet_received;
wait_queue_head_t wait_status;
- spinlock_t reassembly_queue_lock;
- struct list_head reassembly_queue;
- int reassembly_data_length;
- int reassembly_queue_length;
- int first_entry_offset;
- wait_queue_head_t wait_reassembly_queue;
-
spinlock_t receive_credit_lock;
int recv_credits;
int count_avail_recvmsg;
@@ -251,9 +243,11 @@ static void enqueue_reassembly(struct smb_direct_transport *t,
struct smbdirect_recv_io *recvmsg,
int data_length)
{
- spin_lock(&t->reassembly_queue_lock);
- list_add_tail(&recvmsg->list, &t->reassembly_queue);
- t->reassembly_queue_length++;
+ struct smbdirect_socket *sc = &t->socket;
+
+ spin_lock(&sc->recv_io.reassembly.lock);
+ list_add_tail(&recvmsg->list, &sc->recv_io.reassembly.list);
+ sc->recv_io.reassembly.queue_length++;
/*
* Make sure reassembly_data_length is updated after list and
* reassembly_queue_length are updated. On the dequeue side
@@ -261,14 +255,16 @@ static void enqueue_reassembly(struct smb_direct_transport *t,
* if reassembly_queue_length and list is up to date
*/
virt_wmb();
- t->reassembly_data_length += data_length;
- spin_unlock(&t->reassembly_queue_lock);
+ sc->recv_io.reassembly.data_length += data_length;
+ spin_unlock(&sc->recv_io.reassembly.lock);
}
static struct smbdirect_recv_io *get_first_reassembly(struct smb_direct_transport *t)
{
- if (!list_empty(&t->reassembly_queue))
- return list_first_entry(&t->reassembly_queue,
+ struct smbdirect_socket *sc = &t->socket;
+
+ if (!list_empty(&sc->recv_io.reassembly.list))
+ return list_first_entry(&sc->recv_io.reassembly.list,
struct smbdirect_recv_io, list);
else
return NULL;
@@ -330,11 +326,11 @@ static struct smb_direct_transport *alloc_transport(struct rdma_cm_id *cm_id)
sc->status = SMBDIRECT_SOCKET_CREATED;
init_waitqueue_head(&t->wait_status);
- spin_lock_init(&t->reassembly_queue_lock);
- INIT_LIST_HEAD(&t->reassembly_queue);
- t->reassembly_data_length = 0;
- t->reassembly_queue_length = 0;
- init_waitqueue_head(&t->wait_reassembly_queue);
+ spin_lock_init(&sc->recv_io.reassembly.lock);
+ INIT_LIST_HEAD(&sc->recv_io.reassembly.list);
+ sc->recv_io.reassembly.data_length = 0;
+ sc->recv_io.reassembly.queue_length = 0;
+ init_waitqueue_head(&sc->recv_io.reassembly.wait_queue);
init_waitqueue_head(&t->wait_send_credits);
init_waitqueue_head(&t->wait_rw_credits);
@@ -391,17 +387,17 @@ static void free_transport(struct smb_direct_transport *t)
ksmbd_debug(RDMA, "drain the reassembly queue\n");
do {
- spin_lock(&t->reassembly_queue_lock);
+ spin_lock(&sc->recv_io.reassembly.lock);
recvmsg = get_first_reassembly(t);
if (recvmsg) {
list_del(&recvmsg->list);
- spin_unlock(&t->reassembly_queue_lock);
+ spin_unlock(&sc->recv_io.reassembly.lock);
put_recvmsg(t, recvmsg);
} else {
- spin_unlock(&t->reassembly_queue_lock);
+ spin_unlock(&sc->recv_io.reassembly.lock);
}
} while (recvmsg);
- t->reassembly_data_length = 0;
+ sc->recv_io.reassembly.data_length = 0;
if (sc->ib.send_cq)
ib_free_cq(sc->ib.send_cq);
@@ -533,7 +529,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
return;
}
t->negotiation_requested = true;
- t->full_packet_received = true;
+ sc->recv_io.reassembly.full_packet_received = true;
sc->status = SMBDIRECT_SOCKET_CONNECTED;
enqueue_reassembly(t, recvmsg, 0);
wake_up_interruptible(&t->wait_status);
@@ -560,13 +556,13 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
return;
}
- if (t->full_packet_received)
+ if (sc->recv_io.reassembly.full_packet_received)
recvmsg->first_segment = true;
if (le32_to_cpu(data_transfer->remaining_data_length))
- t->full_packet_received = false;
+ sc->recv_io.reassembly.full_packet_received = false;
else
- t->full_packet_received = true;
+ sc->recv_io.reassembly.full_packet_received = true;
spin_lock(&t->receive_credit_lock);
receive_credits = --(t->recv_credits);
@@ -597,7 +593,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
if (data_length) {
enqueue_reassembly(t, recvmsg, (int)data_length);
- wake_up_interruptible(&t->wait_reassembly_queue);
+ wake_up_interruptible(&sc->recv_io.reassembly.wait_queue);
} else
put_recvmsg(t, recvmsg);
@@ -675,7 +671,7 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
* the only one reading from the front of the queue. The transport
* may add more entries to the back of the queue at the same time
*/
- if (st->reassembly_data_length >= size) {
+ if (sc->recv_io.reassembly.data_length >= size) {
int queue_length;
int queue_removed = 0;
@@ -687,10 +683,10 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
* updated in SOFTIRQ as more data is received
*/
virt_rmb();
- queue_length = st->reassembly_queue_length;
+ queue_length = sc->recv_io.reassembly.queue_length;
data_read = 0;
to_read = size;
- offset = st->first_entry_offset;
+ offset = sc->recv_io.reassembly.first_entry_offset;
while (data_read < size) {
recvmsg = get_first_reassembly(st);
data_transfer = smbdirect_recv_io_payload(recvmsg);
@@ -733,9 +729,9 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
if (queue_length) {
list_del(&recvmsg->list);
} else {
- spin_lock_irq(&st->reassembly_queue_lock);
+ spin_lock_irq(&sc->recv_io.reassembly.lock);
list_del(&recvmsg->list);
- spin_unlock_irq(&st->reassembly_queue_lock);
+ spin_unlock_irq(&sc->recv_io.reassembly.lock);
}
queue_removed++;
put_recvmsg(st, recvmsg);
@@ -748,10 +744,10 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
data_read += to_copy;
}
- spin_lock_irq(&st->reassembly_queue_lock);
- st->reassembly_data_length -= data_read;
- st->reassembly_queue_length -= queue_removed;
- spin_unlock_irq(&st->reassembly_queue_lock);
+ spin_lock_irq(&sc->recv_io.reassembly.lock);
+ sc->recv_io.reassembly.data_length -= data_read;
+ sc->recv_io.reassembly.queue_length -= queue_removed;
+ spin_unlock_irq(&sc->recv_io.reassembly.lock);
spin_lock(&st->receive_credit_lock);
st->count_avail_recvmsg += queue_removed;
@@ -763,18 +759,18 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
spin_unlock(&st->receive_credit_lock);
}
- st->first_entry_offset = offset;
+ sc->recv_io.reassembly.first_entry_offset = offset;
ksmbd_debug(RDMA,
"returning to thread data_read=%d reassembly_data_length=%d first_entry_offset=%d\n",
- data_read, st->reassembly_data_length,
- st->first_entry_offset);
+ data_read, sc->recv_io.reassembly.data_length,
+ sc->recv_io.reassembly.first_entry_offset);
read_rfc1002_done:
return data_read;
}
ksmbd_debug(RDMA, "wait_event on more data\n");
- rc = wait_event_interruptible(st->wait_reassembly_queue,
- st->reassembly_data_length >= size ||
+ rc = wait_event_interruptible(sc->recv_io.reassembly.wait_queue,
+ sc->recv_io.reassembly.data_length >= size ||
sc->status != SMBDIRECT_SOCKET_CONNECTED);
if (rc)
return -EINTR;
@@ -1529,7 +1525,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
sc->status = SMBDIRECT_SOCKET_DISCONNECTED;
wake_up_interruptible(&t->wait_status);
- wake_up_interruptible(&t->wait_reassembly_queue);
+ wake_up_interruptible(&sc->recv_io.reassembly.wait_queue);
wake_up(&t->wait_send_credits);
break;
}
@@ -1991,10 +1987,10 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
ret = smb_direct_send_negotiate_response(st, ret);
out:
- spin_lock_irq(&st->reassembly_queue_lock);
- st->reassembly_queue_length--;
+ spin_lock_irq(&sc->recv_io.reassembly.lock);
+ sc->recv_io.reassembly.queue_length--;
list_del(&recvmsg->list);
- spin_unlock_irq(&st->reassembly_queue_lock);
+ spin_unlock_irq(&sc->recv_io.reassembly.lock);
put_recvmsg(st, recvmsg);
return ret;
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 16/18] smb: server: make use of SMBDIRECT_RECV_IO_MAX_SGE
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (14 preceding siblings ...)
2025-08-06 17:36 ` [PATCH 15/18] smb: server: make use of smbdirect_socket.recv_io.reassembly.* Stefan Metzmacher
@ 2025-08-06 17:36 ` Stefan Metzmacher
2025-08-06 17:36 ` [PATCH 17/18] smb: server: make use of struct smbdirect_send_io Stefan Metzmacher
` (4 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:36 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Namjae Jeon, Steve French, Tom Talpey
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/transport_rdma.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index 7fcd80c329d7..8021225df200 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -37,7 +37,6 @@
#define SMB_DIRECT_NEGOTIATE_TIMEOUT 120
#define SMB_DIRECT_MAX_SEND_SGES 6
-#define SMB_DIRECT_MAX_RECV_SGES 1
/*
* Default maximum number of RDMA read/write outstanding on this connection
@@ -1760,7 +1759,7 @@ static int smb_direct_init_params(struct smb_direct_transport *t,
return -EINVAL;
}
- if (device->attrs.max_recv_sge < SMB_DIRECT_MAX_RECV_SGES) {
+ if (device->attrs.max_recv_sge < SMBDIRECT_RECV_IO_MAX_SGE) {
pr_err("warning: device max_recv_sge = %d too small\n",
device->attrs.max_recv_sge);
return -EINVAL;
@@ -1784,7 +1783,7 @@ static int smb_direct_init_params(struct smb_direct_transport *t,
cap->max_send_wr = max_send_wrs;
cap->max_recv_wr = sp->recv_credit_max;
cap->max_send_sge = max_sge_per_wr;
- cap->max_recv_sge = SMB_DIRECT_MAX_RECV_SGES;
+ cap->max_recv_sge = SMBDIRECT_RECV_IO_MAX_SGE;
cap->max_inline_data = 0;
cap->max_rdma_ctxs = t->max_rw_credits;
return 0;
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 17/18] smb: server: make use of struct smbdirect_send_io
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (15 preceding siblings ...)
2025-08-06 17:36 ` [PATCH 16/18] smb: server: make use of SMBDIRECT_RECV_IO_MAX_SGE Stefan Metzmacher
@ 2025-08-06 17:36 ` Stefan Metzmacher
2025-08-06 17:36 ` [PATCH 18/18] smb: server: make use of smbdirect_socket.{send,recv}_io.mem.{cache,pool} Stefan Metzmacher
` (3 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:36 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Namjae Jeon, Steve French, Tom Talpey
This is already used by the client and will allow us
to use common helper functions soon.
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/transport_rdma.c | 81 +++++++++++++++-------------------
1 file changed, 36 insertions(+), 45 deletions(-)
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index 8021225df200..a7671db57705 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -36,8 +36,6 @@
/* SMB_DIRECT negotiation timeout in seconds */
#define SMB_DIRECT_NEGOTIATE_TIMEOUT 120
-#define SMB_DIRECT_MAX_SEND_SGES 6
-
/*
* Default maximum number of RDMA read/write outstanding on this connection
* This value is possibly decreased during QP creation on hardware limit
@@ -139,16 +137,6 @@ struct smb_direct_send_ctx {
unsigned int remote_key;
};
-struct smb_direct_sendmsg {
- struct smb_direct_transport *transport;
- struct ib_send_wr wr;
- struct list_head list;
- int num_sge;
- struct ib_sge sge[SMB_DIRECT_MAX_SEND_SGES];
- struct ib_cqe cqe;
- u8 packet[];
-};
-
struct smb_direct_rdma_rw_msg {
struct smb_direct_transport *t;
struct ib_cqe cqe;
@@ -411,22 +399,23 @@ static void free_transport(struct smb_direct_transport *t)
ksmbd_conn_free(KSMBD_TRANS(t)->conn);
}
-static struct smb_direct_sendmsg
+static struct smbdirect_send_io
*smb_direct_alloc_sendmsg(struct smb_direct_transport *t)
{
- struct smb_direct_sendmsg *msg;
+ struct smbdirect_socket *sc = &t->socket;
+ struct smbdirect_send_io *msg;
msg = mempool_alloc(t->sendmsg_mempool, KSMBD_DEFAULT_GFP);
if (!msg)
return ERR_PTR(-ENOMEM);
- msg->transport = t;
- INIT_LIST_HEAD(&msg->list);
+ msg->socket = sc;
+ INIT_LIST_HEAD(&msg->sibling_list);
msg->num_sge = 0;
return msg;
}
static void smb_direct_free_sendmsg(struct smb_direct_transport *t,
- struct smb_direct_sendmsg *msg)
+ struct smbdirect_send_io *msg)
{
struct smbdirect_socket *sc = &t->socket;
int i;
@@ -822,12 +811,14 @@ static void smb_direct_post_recv_credits(struct work_struct *work)
static void send_done(struct ib_cq *cq, struct ib_wc *wc)
{
- struct smb_direct_sendmsg *sendmsg, *sibling;
+ struct smbdirect_send_io *sendmsg, *sibling;
struct smb_direct_transport *t;
+ struct smbdirect_socket *sc;
struct list_head *pos, *prev, *end;
- sendmsg = container_of(wc->wr_cqe, struct smb_direct_sendmsg, cqe);
- t = sendmsg->transport;
+ sendmsg = container_of(wc->wr_cqe, struct smbdirect_send_io, cqe);
+ sc = sendmsg->socket;
+ t = container_of(sc, struct smb_direct_transport, socket);
ksmbd_debug(RDMA, "Send completed. status='%s (%d)', opcode=%d\n",
ib_wc_status_msg(wc->status), wc->status,
@@ -846,13 +837,13 @@ static void send_done(struct ib_cq *cq, struct ib_wc *wc)
/* iterate and free the list of messages in reverse. the list's head
* is invalid.
*/
- for (pos = &sendmsg->list, prev = pos->prev, end = sendmsg->list.next;
+ for (pos = &sendmsg->sibling_list, prev = pos->prev, end = sendmsg->sibling_list.next;
prev != end; pos = prev, prev = prev->prev) {
- sibling = container_of(pos, struct smb_direct_sendmsg, list);
+ sibling = container_of(pos, struct smbdirect_send_io, sibling_list);
smb_direct_free_sendmsg(t, sibling);
}
- sibling = container_of(pos, struct smb_direct_sendmsg, list);
+ sibling = container_of(pos, struct smbdirect_send_io, sibling_list);
smb_direct_free_sendmsg(t, sibling);
}
@@ -900,18 +891,18 @@ static int smb_direct_flush_send_list(struct smb_direct_transport *t,
struct smb_direct_send_ctx *send_ctx,
bool is_last)
{
- struct smb_direct_sendmsg *first, *last;
+ struct smbdirect_send_io *first, *last;
int ret;
if (list_empty(&send_ctx->msg_list))
return 0;
first = list_first_entry(&send_ctx->msg_list,
- struct smb_direct_sendmsg,
- list);
+ struct smbdirect_send_io,
+ sibling_list);
last = list_last_entry(&send_ctx->msg_list,
- struct smb_direct_sendmsg,
- list);
+ struct smbdirect_send_io,
+ sibling_list);
last->wr.send_flags = IB_SEND_SIGNALED;
last->wr.wr_cqe = &last->cqe;
@@ -929,7 +920,7 @@ static int smb_direct_flush_send_list(struct smb_direct_transport *t,
atomic_add(send_ctx->wr_cnt, &t->send_credits);
wake_up(&t->wait_send_credits);
list_for_each_entry_safe(first, last, &send_ctx->msg_list,
- list) {
+ sibling_list) {
smb_direct_free_sendmsg(t, first);
}
}
@@ -988,11 +979,11 @@ static int calc_rw_credits(struct smb_direct_transport *t,
static int smb_direct_create_header(struct smb_direct_transport *t,
int size, int remaining_data_length,
- struct smb_direct_sendmsg **sendmsg_out)
+ struct smbdirect_send_io **sendmsg_out)
{
struct smbdirect_socket *sc = &t->socket;
struct smbdirect_socket_parameters *sp = &sc->parameters;
- struct smb_direct_sendmsg *sendmsg;
+ struct smbdirect_send_io *sendmsg;
struct smbdirect_data_transfer *packet;
int header_length;
int ret;
@@ -1095,7 +1086,7 @@ static int get_mapped_sg_list(struct ib_device *device, void *buf, int size,
static int post_sendmsg(struct smb_direct_transport *t,
struct smb_direct_send_ctx *send_ctx,
- struct smb_direct_sendmsg *msg)
+ struct smbdirect_send_io *msg)
{
struct smbdirect_socket *sc = &t->socket;
int i;
@@ -1115,14 +1106,14 @@ static int post_sendmsg(struct smb_direct_transport *t,
msg->wr.wr_cqe = NULL;
msg->wr.send_flags = 0;
if (!list_empty(&send_ctx->msg_list)) {
- struct smb_direct_sendmsg *last;
+ struct smbdirect_send_io *last;
last = list_last_entry(&send_ctx->msg_list,
- struct smb_direct_sendmsg,
- list);
+ struct smbdirect_send_io,
+ sibling_list);
last->wr.next = &msg->wr;
}
- list_add_tail(&msg->list, &send_ctx->msg_list);
+ list_add_tail(&msg->sibling_list, &send_ctx->msg_list);
send_ctx->wr_cnt++;
return 0;
}
@@ -1139,9 +1130,9 @@ static int smb_direct_post_send_data(struct smb_direct_transport *t,
{
struct smbdirect_socket *sc = &t->socket;
int i, j, ret;
- struct smb_direct_sendmsg *msg;
+ struct smbdirect_send_io *msg;
int data_length;
- struct scatterlist sg[SMB_DIRECT_MAX_SEND_SGES - 1];
+ struct scatterlist sg[SMBDIRECT_SEND_IO_MAX_SGE - 1];
ret = wait_for_send_credits(t, send_ctx);
if (ret)
@@ -1162,16 +1153,16 @@ static int smb_direct_post_send_data(struct smb_direct_transport *t,
struct ib_sge *sge;
int sg_cnt;
- sg_init_table(sg, SMB_DIRECT_MAX_SEND_SGES - 1);
+ sg_init_table(sg, SMBDIRECT_SEND_IO_MAX_SGE - 1);
sg_cnt = get_mapped_sg_list(sc->ib.dev,
iov[i].iov_base, iov[i].iov_len,
- sg, SMB_DIRECT_MAX_SEND_SGES - 1,
+ sg, SMBDIRECT_SEND_IO_MAX_SGE - 1,
DMA_TO_DEVICE);
if (sg_cnt <= 0) {
pr_err("failed to map buffer\n");
ret = -ENOMEM;
goto err;
- } else if (sg_cnt + msg->num_sge > SMB_DIRECT_MAX_SEND_SGES) {
+ } else if (sg_cnt + msg->num_sge > SMBDIRECT_SEND_IO_MAX_SGE) {
pr_err("buffer not fitted into sges\n");
ret = -E2BIG;
ib_dma_unmap_sg(sc->ib.dev, sg, sg_cnt,
@@ -1565,7 +1556,7 @@ static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
{
struct smbdirect_socket *sc = &t->socket;
struct smbdirect_socket_parameters *sp = &sc->parameters;
- struct smb_direct_sendmsg *sendmsg;
+ struct smbdirect_send_io *sendmsg;
struct smbdirect_negotiate_resp *resp;
int ret;
@@ -1713,7 +1704,7 @@ static int smb_direct_init_params(struct smb_direct_transport *t,
*/
sp->max_send_size = smb_direct_max_send_size;
max_send_sges = DIV_ROUND_UP(sp->max_send_size, PAGE_SIZE) + 3;
- if (max_send_sges > SMB_DIRECT_MAX_SEND_SGES) {
+ if (max_send_sges > SMBDIRECT_SEND_IO_MAX_SGE) {
pr_err("max_send_size %d is too large\n", sp->max_send_size);
return -EINVAL;
}
@@ -1817,9 +1808,9 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
int i;
struct smbdirect_recv_io *recvmsg;
- snprintf(name, sizeof(name), "smb_direct_rqst_pool_%p", t);
+ snprintf(name, sizeof(name), "smbdirect_send_io_pool_%p", t);
t->sendmsg_cache = kmem_cache_create(name,
- sizeof(struct smb_direct_sendmsg) +
+ sizeof(struct smbdirect_send_io) +
sizeof(struct smbdirect_negotiate_resp),
0, SLAB_HWCACHE_ALIGN, NULL);
if (!t->sendmsg_cache)
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* [PATCH 18/18] smb: server: make use of smbdirect_socket.{send,recv}_io.mem.{cache,pool}
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (16 preceding siblings ...)
2025-08-06 17:36 ` [PATCH 17/18] smb: server: make use of struct smbdirect_send_io Stefan Metzmacher
@ 2025-08-06 17:36 ` Stefan Metzmacher
2025-08-06 17:41 ` [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (2 subsequent siblings)
20 siblings, 0 replies; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:36 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: metze, Namjae Jeon, Steve French, Tom Talpey
This will allow common helper functions to be created later.
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/server/transport_rdma.c | 50 ++++++++++++++++------------------
1 file changed, 23 insertions(+), 27 deletions(-)
diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
index a7671db57705..b59a03e2dbcf 100644
--- a/fs/smb/server/transport_rdma.c
+++ b/fs/smb/server/transport_rdma.c
@@ -109,11 +109,6 @@ struct smb_direct_transport {
wait_queue_head_t wait_send_credits;
wait_queue_head_t wait_rw_credits;
- mempool_t *sendmsg_mempool;
- struct kmem_cache *sendmsg_cache;
- mempool_t *recvmsg_mempool;
- struct kmem_cache *recvmsg_cache;
-
wait_queue_head_t wait_send_pending;
atomic_t send_pending;
@@ -405,7 +400,7 @@ static struct smbdirect_send_io
struct smbdirect_socket *sc = &t->socket;
struct smbdirect_send_io *msg;
- msg = mempool_alloc(t->sendmsg_mempool, KSMBD_DEFAULT_GFP);
+ msg = mempool_alloc(sc->send_io.mem.pool, KSMBD_DEFAULT_GFP);
if (!msg)
return ERR_PTR(-ENOMEM);
msg->socket = sc;
@@ -429,7 +424,7 @@ static void smb_direct_free_sendmsg(struct smb_direct_transport *t,
msg->sge[i].addr, msg->sge[i].length,
DMA_TO_DEVICE);
}
- mempool_free(msg, t->sendmsg_mempool);
+ mempool_free(msg, sc->send_io.mem.pool);
}
static int smb_direct_check_recvmsg(struct smbdirect_recv_io *recvmsg)
@@ -1782,22 +1777,23 @@ static int smb_direct_init_params(struct smb_direct_transport *t,
static void smb_direct_destroy_pools(struct smb_direct_transport *t)
{
+ struct smbdirect_socket *sc = &t->socket;
struct smbdirect_recv_io *recvmsg;
while ((recvmsg = get_free_recvmsg(t)))
- mempool_free(recvmsg, t->recvmsg_mempool);
+ mempool_free(recvmsg, sc->recv_io.mem.pool);
- mempool_destroy(t->recvmsg_mempool);
- t->recvmsg_mempool = NULL;
+ mempool_destroy(sc->recv_io.mem.pool);
+ sc->recv_io.mem.pool = NULL;
- kmem_cache_destroy(t->recvmsg_cache);
- t->recvmsg_cache = NULL;
+ kmem_cache_destroy(sc->recv_io.mem.cache);
+ sc->recv_io.mem.cache = NULL;
- mempool_destroy(t->sendmsg_mempool);
- t->sendmsg_mempool = NULL;
+ mempool_destroy(sc->send_io.mem.pool);
+ sc->send_io.mem.pool = NULL;
- kmem_cache_destroy(t->sendmsg_cache);
- t->sendmsg_cache = NULL;
+ kmem_cache_destroy(sc->send_io.mem.cache);
+ sc->send_io.mem.cache = NULL;
}
static int smb_direct_create_pools(struct smb_direct_transport *t)
@@ -1809,35 +1805,35 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
struct smbdirect_recv_io *recvmsg;
snprintf(name, sizeof(name), "smbdirect_send_io_pool_%p", t);
- t->sendmsg_cache = kmem_cache_create(name,
+ sc->send_io.mem.cache = kmem_cache_create(name,
sizeof(struct smbdirect_send_io) +
sizeof(struct smbdirect_negotiate_resp),
0, SLAB_HWCACHE_ALIGN, NULL);
- if (!t->sendmsg_cache)
+ if (!sc->send_io.mem.cache)
return -ENOMEM;
- t->sendmsg_mempool = mempool_create(sp->send_credit_target,
+ sc->send_io.mem.pool = mempool_create(sp->send_credit_target,
mempool_alloc_slab, mempool_free_slab,
- t->sendmsg_cache);
- if (!t->sendmsg_mempool)
+ sc->send_io.mem.cache);
+ if (!sc->send_io.mem.pool)
goto err;
snprintf(name, sizeof(name), "smbdirect_recv_io_pool_%p", t);
- t->recvmsg_cache = kmem_cache_create(name,
+ sc->recv_io.mem.cache = kmem_cache_create(name,
sizeof(struct smbdirect_recv_io) +
sp->max_recv_size,
0, SLAB_HWCACHE_ALIGN, NULL);
- if (!t->recvmsg_cache)
+ if (!sc->recv_io.mem.cache)
goto err;
- t->recvmsg_mempool =
+ sc->recv_io.mem.pool =
mempool_create(sp->recv_credit_max, mempool_alloc_slab,
- mempool_free_slab, t->recvmsg_cache);
- if (!t->recvmsg_mempool)
+ mempool_free_slab, sc->recv_io.mem.cache);
+ if (!sc->recv_io.mem.pool)
goto err;
for (i = 0; i < sp->recv_credit_max; i++) {
- recvmsg = mempool_alloc(t->recvmsg_mempool, KSMBD_DEFAULT_GFP);
+ recvmsg = mempool_alloc(sc->recv_io.mem.pool, KSMBD_DEFAULT_GFP);
if (!recvmsg)
goto err;
recvmsg->socket = sc;
--
2.43.0
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (17 preceding siblings ...)
2025-08-06 17:36 ` [PATCH 18/18] smb: server: make use of smbdirect_socket.{send,recv}_io.mem.{cache,pool} Stefan Metzmacher
@ 2025-08-06 17:41 ` Stefan Metzmacher
2025-08-06 20:21 ` Steve French
2025-08-08 0:22 ` Namjae Jeon
2025-08-11 0:06 ` Namjae Jeon
20 siblings, 1 reply; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-06 17:41 UTC (permalink / raw)
To: linux-cifs, samba-technical; +Cc: Steve French, Tom Talpey, Namjae Jeon
Am 06.08.25 um 19:35 schrieb Stefan Metzmacher via samba-technical:
> Hi,
>
> this is the next step towards a common smbdirect layer
> between cifs.ko and ksmbd.ko, with the aim to provide
> a socket layer for userspace usage at the end of the road.
>
> This patchset focuses on the usage of a common
> smbdirect_send_io and related structures in smbdirect_socket.send_io.
>
> Note only patches 01-08 are intended to be merged soon,
Sorry it's just 01-07.
metze
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io
2025-08-06 17:41 ` [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
@ 2025-08-06 20:21 ` Steve French
2025-08-07 23:21 ` Namjae Jeon
0 siblings, 1 reply; 26+ messages in thread
From: Steve French @ 2025-08-06 20:21 UTC (permalink / raw)
To: Stefan Metzmacher
Cc: linux-cifs, samba-technical, Tom Talpey, Namjae Jeon,
Meetakshi Setiya
tentatively merged those 7 additional small patches into cifs-2.6.git
for-next pending more review and testing.
On Wed, Aug 6, 2025 at 12:41 PM Stefan Metzmacher <metze@samba.org> wrote:
>
> Am 06.08.25 um 19:35 schrieb Stefan Metzmacher via samba-technical:
> > Hi,
> >
> > this is the next step towards a common smbdirect layer
> > between cifs.ko and ksmbd.ko, with the aim to provide
> > a socket layer for userspace usage at the end of the road.
> >
> > This patchset focuses on the usage of a common
> > smbdirect_send_io and related structures in smbdirect_socket.send_io.
> >
> > Note only patches 01-08 are intended to be merged soon,
>
> Sorry it's just 01-07.
>
> metze
--
Thanks,
Steve
^ permalink raw reply [flat|nested] 26+ messages in thread
* ksmbd common smbdirect headers for 6.17-rc1? (Re: [PATCH 08/18] smb: server: make use of common smbdirect_pdu.h)
2025-08-06 17:35 ` [PATCH 08/18] smb: server: make use of common smbdirect_pdu.h Stefan Metzmacher
@ 2025-08-07 15:47 ` Stefan Metzmacher
2025-08-08 0:10 ` Namjae Jeon
0 siblings, 1 reply; 26+ messages in thread
From: Stefan Metzmacher @ 2025-08-07 15:47 UTC (permalink / raw)
To: Namjae Jeon, Steve French
Cc: Meetakshi Setiya, Tom Talpey, Hyunchul Lee, Steve French,
samba-technical, linux-cifs
Hi Namjae,
maybe I'm able to get to a point where we just have this:
struct smb_direct_transport {
struct ksmbd_transport transport;
struct smbdirect_socket socket;
};
If I get there tomorrow evening I'm wondering if I should also post that
patchset for inclusion into 6.17-rc1.
Should I try that or would this be for 6.18 anyway?
What do you think?
metze
Am 06.08.25 um 19:35 schrieb Stefan Metzmacher via samba-technical:
> Cc: Steve French <smfrench@gmail.com>
> Cc: Tom Talpey <tom@talpey.com>
> Cc: Long Li <longli@microsoft.com>
> Cc: Namjae Jeon <linkinjeon@kernel.org>
> Cc: Hyunchul Lee <hyc.lee@gmail.com>
> Cc: Meetakshi Setiya <meetakshisetiyaoss@gmail.com>
> Cc: linux-cifs@vger.kernel.org
> Cc: samba-technical@lists.samba.org
> Signed-off-by: Stefan Metzmacher <metze@samba.org>
> ---
> fs/smb/server/transport_rdma.c | 49 +++++++++++++++++-----------------
> fs/smb/server/transport_rdma.h | 41 ----------------------------
> 2 files changed, 25 insertions(+), 65 deletions(-)
>
> diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
> index 8d366db5f605..275199fef4e5 100644
> --- a/fs/smb/server/transport_rdma.c
> +++ b/fs/smb/server/transport_rdma.c
> @@ -23,12 +23,13 @@
> #include "connection.h"
> #include "smb_common.h"
> #include "../common/smb2status.h"
> +#include "../common/smbdirect/smbdirect_pdu.h"
> #include "transport_rdma.h"
>
> #define SMB_DIRECT_PORT_IWARP 5445
> #define SMB_DIRECT_PORT_INFINIBAND 445
>
> -#define SMB_DIRECT_VERSION_LE cpu_to_le16(0x0100)
> +#define SMB_DIRECT_VERSION_LE cpu_to_le16(SMBDIRECT_V1)
>
> /* SMB_DIRECT negotiation timeout in seconds */
> #define SMB_DIRECT_NEGOTIATE_TIMEOUT 120
> @@ -472,8 +473,8 @@ static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
> {
> switch (recvmsg->type) {
> case SMB_DIRECT_MSG_DATA_TRANSFER: {
> - struct smb_direct_data_transfer *req =
> - (struct smb_direct_data_transfer *)recvmsg->packet;
> + struct smbdirect_data_transfer *req =
> + (struct smbdirect_data_transfer *)recvmsg->packet;
> struct smb2_hdr *hdr = (struct smb2_hdr *)(recvmsg->packet
> + le32_to_cpu(req->data_offset));
> ksmbd_debug(RDMA,
> @@ -485,8 +486,8 @@ static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
> break;
> }
> case SMB_DIRECT_MSG_NEGOTIATE_REQ: {
> - struct smb_direct_negotiate_req *req =
> - (struct smb_direct_negotiate_req *)recvmsg->packet;
> + struct smbdirect_negotiate_req *req =
> + (struct smbdirect_negotiate_req *)recvmsg->packet;
> ksmbd_debug(RDMA,
> "MinVersion: %u, MaxVersion: %u, CreditRequested: %u, MaxSendSize: %u, MaxRecvSize: %u, MaxFragmentedSize: %u\n",
> le16_to_cpu(req->min_version),
> @@ -540,7 +541,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
>
> switch (recvmsg->type) {
> case SMB_DIRECT_MSG_NEGOTIATE_REQ:
> - if (wc->byte_len < sizeof(struct smb_direct_negotiate_req)) {
> + if (wc->byte_len < sizeof(struct smbdirect_negotiate_req)) {
> put_recvmsg(t, recvmsg);
> smb_direct_disconnect_rdma_connection(t);
> return;
> @@ -552,13 +553,13 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
> wake_up_interruptible(&t->wait_status);
> return;
> case SMB_DIRECT_MSG_DATA_TRANSFER: {
> - struct smb_direct_data_transfer *data_transfer =
> - (struct smb_direct_data_transfer *)recvmsg->packet;
> + struct smbdirect_data_transfer *data_transfer =
> + (struct smbdirect_data_transfer *)recvmsg->packet;
> unsigned int data_length;
> int avail_recvmsg_count, receive_credits;
>
> if (wc->byte_len <
> - offsetof(struct smb_direct_data_transfer, padding)) {
> + offsetof(struct smbdirect_data_transfer, padding)) {
> put_recvmsg(t, recvmsg);
> smb_direct_disconnect_rdma_connection(t);
> return;
> @@ -566,7 +567,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
>
> data_length = le32_to_cpu(data_transfer->data_length);
> if (data_length) {
> - if (wc->byte_len < sizeof(struct smb_direct_data_transfer) +
> + if (wc->byte_len < sizeof(struct smbdirect_data_transfer) +
> (u64)data_length) {
> put_recvmsg(t, recvmsg);
> smb_direct_disconnect_rdma_connection(t);
> @@ -598,7 +599,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
> &t->send_credits);
>
> if (le16_to_cpu(data_transfer->flags) &
> - SMB_DIRECT_RESPONSE_REQUESTED)
> + SMBDIRECT_FLAG_RESPONSE_REQUESTED)
> queue_work(smb_direct_wq, &t->send_immediate_work);
>
> if (atomic_read(&t->send_credits) > 0)
> @@ -664,7 +665,7 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
> unsigned int size, int unused)
> {
> struct smb_direct_recvmsg *recvmsg;
> - struct smb_direct_data_transfer *data_transfer;
> + struct smbdirect_data_transfer *data_transfer;
> int to_copy, to_read, data_read, offset;
> u32 data_length, remaining_data_length, data_offset;
> int rc;
> @@ -1001,7 +1002,7 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
> struct smb_direct_sendmsg **sendmsg_out)
> {
> struct smb_direct_sendmsg *sendmsg;
> - struct smb_direct_data_transfer *packet;
> + struct smbdirect_data_transfer *packet;
> int header_length;
> int ret;
>
> @@ -1010,7 +1011,7 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
> return PTR_ERR(sendmsg);
>
> /* Fill in the packet header */
> - packet = (struct smb_direct_data_transfer *)sendmsg->packet;
> + packet = (struct smbdirect_data_transfer *)sendmsg->packet;
> packet->credits_requested = cpu_to_le16(t->send_credit_target);
> packet->credits_granted = cpu_to_le16(manage_credits_prior_sending(t));
>
> @@ -1033,11 +1034,11 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
> le32_to_cpu(packet->remaining_data_length));
>
> /* Map the packet to DMA */
> - header_length = sizeof(struct smb_direct_data_transfer);
> + header_length = sizeof(struct smbdirect_data_transfer);
> /* If this is a packet without payload, don't send padding */
> if (!size)
> header_length =
> - offsetof(struct smb_direct_data_transfer, padding);
> + offsetof(struct smbdirect_data_transfer, padding);
>
> sendmsg->sge[0].addr = ib_dma_map_single(t->cm_id->device,
> (void *)packet,
> @@ -1212,7 +1213,7 @@ static int smb_direct_writev(struct ksmbd_transport *t,
> int remaining_data_length;
> int start, i, j;
> int max_iov_size = st->max_send_size -
> - sizeof(struct smb_direct_data_transfer);
> + sizeof(struct smbdirect_data_transfer);
> int ret;
> struct kvec vec;
> struct smb_direct_send_ctx send_ctx;
> @@ -1560,18 +1561,18 @@ static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
> int failed)
> {
> struct smb_direct_sendmsg *sendmsg;
> - struct smb_direct_negotiate_resp *resp;
> + struct smbdirect_negotiate_resp *resp;
> int ret;
>
> sendmsg = smb_direct_alloc_sendmsg(t);
> if (IS_ERR(sendmsg))
> return -ENOMEM;
>
> - resp = (struct smb_direct_negotiate_resp *)sendmsg->packet;
> + resp = (struct smbdirect_negotiate_resp *)sendmsg->packet;
> if (failed) {
> memset(resp, 0, sizeof(*resp));
> - resp->min_version = cpu_to_le16(0x0100);
> - resp->max_version = cpu_to_le16(0x0100);
> + resp->min_version = SMB_DIRECT_VERSION_LE;
> + resp->max_version = SMB_DIRECT_VERSION_LE;
> resp->status = STATUS_NOT_SUPPORTED;
> } else {
> resp->status = STATUS_SUCCESS;
> @@ -1803,7 +1804,7 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
> snprintf(name, sizeof(name), "smb_direct_rqst_pool_%p", t);
> t->sendmsg_cache = kmem_cache_create(name,
> sizeof(struct smb_direct_sendmsg) +
> - sizeof(struct smb_direct_negotiate_resp),
> + sizeof(struct smbdirect_negotiate_resp),
> 0, SLAB_HWCACHE_ALIGN, NULL);
> if (!t->sendmsg_cache)
> return -ENOMEM;
> @@ -1936,7 +1937,7 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
> {
> struct smb_direct_transport *st = smb_trans_direct_transfort(t);
> struct smb_direct_recvmsg *recvmsg;
> - struct smb_direct_negotiate_req *req;
> + struct smbdirect_negotiate_req *req;
> int ret;
>
> ksmbd_debug(RDMA, "Waiting for SMB_DIRECT negotiate request\n");
> @@ -1955,7 +1956,7 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
> if (ret == -ECONNABORTED)
> goto out;
>
> - req = (struct smb_direct_negotiate_req *)recvmsg->packet;
> + req = (struct smbdirect_negotiate_req *)recvmsg->packet;
> st->max_recv_size = min_t(int, st->max_recv_size,
> le32_to_cpu(req->preferred_send_size));
> st->max_send_size = min_t(int, st->max_send_size,
> diff --git a/fs/smb/server/transport_rdma.h b/fs/smb/server/transport_rdma.h
> index 77aee4e5c9dc..0fb692c40e21 100644
> --- a/fs/smb/server/transport_rdma.h
> +++ b/fs/smb/server/transport_rdma.h
> @@ -11,47 +11,6 @@
> #define SMBD_MIN_IOSIZE (512 * 1024)
> #define SMBD_MAX_IOSIZE (16 * 1024 * 1024)
>
> -/* SMB DIRECT negotiation request packet [MS-SMBD] 2.2.1 */
> -struct smb_direct_negotiate_req {
> - __le16 min_version;
> - __le16 max_version;
> - __le16 reserved;
> - __le16 credits_requested;
> - __le32 preferred_send_size;
> - __le32 max_receive_size;
> - __le32 max_fragmented_size;
> -} __packed;
> -
> -/* SMB DIRECT negotiation response packet [MS-SMBD] 2.2.2 */
> -struct smb_direct_negotiate_resp {
> - __le16 min_version;
> - __le16 max_version;
> - __le16 negotiated_version;
> - __le16 reserved;
> - __le16 credits_requested;
> - __le16 credits_granted;
> - __le32 status;
> - __le32 max_readwrite_size;
> - __le32 preferred_send_size;
> - __le32 max_receive_size;
> - __le32 max_fragmented_size;
> -} __packed;
> -
> -#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001
> -
> -/* SMB DIRECT data transfer packet with payload [MS-SMBD] 2.2.3 */
> -struct smb_direct_data_transfer {
> - __le16 credits_requested;
> - __le16 credits_granted;
> - __le16 flags;
> - __le16 reserved;
> - __le32 remaining_data_length;
> - __le32 data_offset;
> - __le32 data_length;
> - __le32 padding;
> - __u8 buffer[];
> -} __packed;
> -
> #ifdef CONFIG_SMB_SERVER_SMBDIRECT
> int ksmbd_rdma_init(void);
> void ksmbd_rdma_destroy(void);
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io
2025-08-06 20:21 ` Steve French
@ 2025-08-07 23:21 ` Namjae Jeon
0 siblings, 0 replies; 26+ messages in thread
From: Namjae Jeon @ 2025-08-07 23:21 UTC (permalink / raw)
To: Steve French
Cc: Stefan Metzmacher, linux-cifs, samba-technical, Tom Talpey,
Meetakshi Setiya
On Thu, Aug 7, 2025 at 5:21 AM Steve French <smfrench@gmail.com> wrote:
>
> tentatively merged those 7 additional small patches into cifs-2.6.git
> for-next pending more review and testing.
Could you apply ksmbd smbdirect patches from Metze to for-next branch also ?
Thanks.
>
> On Wed, Aug 6, 2025 at 12:41 PM Stefan Metzmacher <metze@samba.org> wrote:
> >
> > Am 06.08.25 um 19:35 schrieb Stefan Metzmacher via samba-technical:
> > > Hi,
> > >
> > > this is the next step towards a common smbdirect layer
> > > between cifs.ko and ksmbd.ko, with the aim to provide
> > > a socket layer for userspace usage at the end of the road.
> > >
> > > This patchset focuses on the usage of a common
> > > smbdirect_send_io and related structures in smbdirect_socket.send_io.
> > >
> > > Note only patches 01-08 are intended to be merged soon,
> >
> > Sorry it's just 01-07.
> >
> > metze
>
>
>
> --
> Thanks,
>
> Steve
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: ksmbd common smbdirect headers for 6.17-rc1? (Re: [PATCH 08/18] smb: server: make use of common smbdirect_pdu.h)
2025-08-07 15:47 ` ksmbd common smbdirect headers for 6.17-rc1? (Re: [PATCH 08/18] smb: server: make use of common smbdirect_pdu.h) Stefan Metzmacher
@ 2025-08-08 0:10 ` Namjae Jeon
0 siblings, 0 replies; 26+ messages in thread
From: Namjae Jeon @ 2025-08-08 0:10 UTC (permalink / raw)
To: Stefan Metzmacher
Cc: Steve French, Meetakshi Setiya, Tom Talpey, Hyunchul Lee,
Steve French, samba-technical, linux-cifs
On Fri, Aug 8, 2025 at 12:47 AM Stefan Metzmacher <metze@samba.org> wrote:
>
> Hi Namjae,
>
> maybe I'm able to get to a point where we just have this:
>
> struct smb_direct_transport {
> struct ksmbd_transport transport;
>
> struct smbdirect_socket socket;
> };
>
> If I get there tomorrow evening I'm wondering if I should also post that
> patchset for inclusion into 6.17-rc1.
>
> Should I try that or would this be for 6.18 anyway?
>
> What do you think?
It seems a bit late to merge into 6.17-rc1, so I've told Steve to
merge your patches into #for-next.
I think you can work on #for-next as a base and send new patches to the list.
Thanks.
> metze
>
> Am 06.08.25 um 19:35 schrieb Stefan Metzmacher via samba-technical:
> > Cc: Steve French <smfrench@gmail.com>
> > Cc: Tom Talpey <tom@talpey.com>
> > Cc: Long Li <longli@microsoft.com>
> > Cc: Namjae Jeon <linkinjeon@kernel.org>
> > Cc: Hyunchul Lee <hyc.lee@gmail.com>
> > Cc: Meetakshi Setiya <meetakshisetiyaoss@gmail.com>
> > Cc: linux-cifs@vger.kernel.org
> > Cc: samba-technical@lists.samba.org
> > Signed-off-by: Stefan Metzmacher <metze@samba.org>
> > ---
> > fs/smb/server/transport_rdma.c | 49 +++++++++++++++++-----------------
> > fs/smb/server/transport_rdma.h | 41 ----------------------------
> > 2 files changed, 25 insertions(+), 65 deletions(-)
> >
> > diff --git a/fs/smb/server/transport_rdma.c b/fs/smb/server/transport_rdma.c
> > index 8d366db5f605..275199fef4e5 100644
> > --- a/fs/smb/server/transport_rdma.c
> > +++ b/fs/smb/server/transport_rdma.c
> > @@ -23,12 +23,13 @@
> > #include "connection.h"
> > #include "smb_common.h"
> > #include "../common/smb2status.h"
> > +#include "../common/smbdirect/smbdirect_pdu.h"
> > #include "transport_rdma.h"
> >
> > #define SMB_DIRECT_PORT_IWARP 5445
> > #define SMB_DIRECT_PORT_INFINIBAND 445
> >
> > -#define SMB_DIRECT_VERSION_LE cpu_to_le16(0x0100)
> > +#define SMB_DIRECT_VERSION_LE cpu_to_le16(SMBDIRECT_V1)
> >
> > /* SMB_DIRECT negotiation timeout in seconds */
> > #define SMB_DIRECT_NEGOTIATE_TIMEOUT 120
> > @@ -472,8 +473,8 @@ static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
> > {
> > switch (recvmsg->type) {
> > case SMB_DIRECT_MSG_DATA_TRANSFER: {
> > - struct smb_direct_data_transfer *req =
> > - (struct smb_direct_data_transfer *)recvmsg->packet;
> > + struct smbdirect_data_transfer *req =
> > + (struct smbdirect_data_transfer *)recvmsg->packet;
> > struct smb2_hdr *hdr = (struct smb2_hdr *)(recvmsg->packet
> > + le32_to_cpu(req->data_offset));
> > ksmbd_debug(RDMA,
> > @@ -485,8 +486,8 @@ static int smb_direct_check_recvmsg(struct smb_direct_recvmsg *recvmsg)
> > break;
> > }
> > case SMB_DIRECT_MSG_NEGOTIATE_REQ: {
> > - struct smb_direct_negotiate_req *req =
> > - (struct smb_direct_negotiate_req *)recvmsg->packet;
> > + struct smbdirect_negotiate_req *req =
> > + (struct smbdirect_negotiate_req *)recvmsg->packet;
> > ksmbd_debug(RDMA,
> > "MinVersion: %u, MaxVersion: %u, CreditRequested: %u, MaxSendSize: %u, MaxRecvSize: %u, MaxFragmentedSize: %u\n",
> > le16_to_cpu(req->min_version),
> > @@ -540,7 +541,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
> >
> > switch (recvmsg->type) {
> > case SMB_DIRECT_MSG_NEGOTIATE_REQ:
> > - if (wc->byte_len < sizeof(struct smb_direct_negotiate_req)) {
> > + if (wc->byte_len < sizeof(struct smbdirect_negotiate_req)) {
> > put_recvmsg(t, recvmsg);
> > smb_direct_disconnect_rdma_connection(t);
> > return;
> > @@ -552,13 +553,13 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
> > wake_up_interruptible(&t->wait_status);
> > return;
> > case SMB_DIRECT_MSG_DATA_TRANSFER: {
> > - struct smb_direct_data_transfer *data_transfer =
> > - (struct smb_direct_data_transfer *)recvmsg->packet;
> > + struct smbdirect_data_transfer *data_transfer =
> > + (struct smbdirect_data_transfer *)recvmsg->packet;
> > unsigned int data_length;
> > int avail_recvmsg_count, receive_credits;
> >
> > if (wc->byte_len <
> > - offsetof(struct smb_direct_data_transfer, padding)) {
> > + offsetof(struct smbdirect_data_transfer, padding)) {
> > put_recvmsg(t, recvmsg);
> > smb_direct_disconnect_rdma_connection(t);
> > return;
> > @@ -566,7 +567,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
> >
> > data_length = le32_to_cpu(data_transfer->data_length);
> > if (data_length) {
> > - if (wc->byte_len < sizeof(struct smb_direct_data_transfer) +
> > + if (wc->byte_len < sizeof(struct smbdirect_data_transfer) +
> > (u64)data_length) {
> > put_recvmsg(t, recvmsg);
> > smb_direct_disconnect_rdma_connection(t);
> > @@ -598,7 +599,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)
> > &t->send_credits);
> >
> > if (le16_to_cpu(data_transfer->flags) &
> > - SMB_DIRECT_RESPONSE_REQUESTED)
> > + SMBDIRECT_FLAG_RESPONSE_REQUESTED)
> > queue_work(smb_direct_wq, &t->send_immediate_work);
> >
> > if (atomic_read(&t->send_credits) > 0)
> > @@ -664,7 +665,7 @@ static int smb_direct_read(struct ksmbd_transport *t, char *buf,
> > unsigned int size, int unused)
> > {
> > struct smb_direct_recvmsg *recvmsg;
> > - struct smb_direct_data_transfer *data_transfer;
> > + struct smbdirect_data_transfer *data_transfer;
> > int to_copy, to_read, data_read, offset;
> > u32 data_length, remaining_data_length, data_offset;
> > int rc;
> > @@ -1001,7 +1002,7 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
> > struct smb_direct_sendmsg **sendmsg_out)
> > {
> > struct smb_direct_sendmsg *sendmsg;
> > - struct smb_direct_data_transfer *packet;
> > + struct smbdirect_data_transfer *packet;
> > int header_length;
> > int ret;
> >
> > @@ -1010,7 +1011,7 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
> > return PTR_ERR(sendmsg);
> >
> > /* Fill in the packet header */
> > - packet = (struct smb_direct_data_transfer *)sendmsg->packet;
> > + packet = (struct smbdirect_data_transfer *)sendmsg->packet;
> > packet->credits_requested = cpu_to_le16(t->send_credit_target);
> > packet->credits_granted = cpu_to_le16(manage_credits_prior_sending(t));
> >
> > @@ -1033,11 +1034,11 @@ static int smb_direct_create_header(struct smb_direct_transport *t,
> > le32_to_cpu(packet->remaining_data_length));
> >
> > /* Map the packet to DMA */
> > - header_length = sizeof(struct smb_direct_data_transfer);
> > + header_length = sizeof(struct smbdirect_data_transfer);
> > /* If this is a packet without payload, don't send padding */
> > if (!size)
> > header_length =
> > - offsetof(struct smb_direct_data_transfer, padding);
> > + offsetof(struct smbdirect_data_transfer, padding);
> >
> > sendmsg->sge[0].addr = ib_dma_map_single(t->cm_id->device,
> > (void *)packet,
> > @@ -1212,7 +1213,7 @@ static int smb_direct_writev(struct ksmbd_transport *t,
> > int remaining_data_length;
> > int start, i, j;
> > int max_iov_size = st->max_send_size -
> > - sizeof(struct smb_direct_data_transfer);
> > + sizeof(struct smbdirect_data_transfer);
> > int ret;
> > struct kvec vec;
> > struct smb_direct_send_ctx send_ctx;
> > @@ -1560,18 +1561,18 @@ static int smb_direct_send_negotiate_response(struct smb_direct_transport *t,
> > int failed)
> > {
> > struct smb_direct_sendmsg *sendmsg;
> > - struct smb_direct_negotiate_resp *resp;
> > + struct smbdirect_negotiate_resp *resp;
> > int ret;
> >
> > sendmsg = smb_direct_alloc_sendmsg(t);
> > if (IS_ERR(sendmsg))
> > return -ENOMEM;
> >
> > - resp = (struct smb_direct_negotiate_resp *)sendmsg->packet;
> > + resp = (struct smbdirect_negotiate_resp *)sendmsg->packet;
> > if (failed) {
> > memset(resp, 0, sizeof(*resp));
> > - resp->min_version = cpu_to_le16(0x0100);
> > - resp->max_version = cpu_to_le16(0x0100);
> > + resp->min_version = SMB_DIRECT_VERSION_LE;
> > + resp->max_version = SMB_DIRECT_VERSION_LE;
> > resp->status = STATUS_NOT_SUPPORTED;
> > } else {
> > resp->status = STATUS_SUCCESS;
> > @@ -1803,7 +1804,7 @@ static int smb_direct_create_pools(struct smb_direct_transport *t)
> > snprintf(name, sizeof(name), "smb_direct_rqst_pool_%p", t);
> > t->sendmsg_cache = kmem_cache_create(name,
> > sizeof(struct smb_direct_sendmsg) +
> > - sizeof(struct smb_direct_negotiate_resp),
> > + sizeof(struct smbdirect_negotiate_resp),
> > 0, SLAB_HWCACHE_ALIGN, NULL);
> > if (!t->sendmsg_cache)
> > return -ENOMEM;
> > @@ -1936,7 +1937,7 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
> > {
> > struct smb_direct_transport *st = smb_trans_direct_transfort(t);
> > struct smb_direct_recvmsg *recvmsg;
> > - struct smb_direct_negotiate_req *req;
> > + struct smbdirect_negotiate_req *req;
> > int ret;
> >
> > ksmbd_debug(RDMA, "Waiting for SMB_DIRECT negotiate request\n");
> > @@ -1955,7 +1956,7 @@ static int smb_direct_prepare(struct ksmbd_transport *t)
> > if (ret == -ECONNABORTED)
> > goto out;
> >
> > - req = (struct smb_direct_negotiate_req *)recvmsg->packet;
> > + req = (struct smbdirect_negotiate_req *)recvmsg->packet;
> > st->max_recv_size = min_t(int, st->max_recv_size,
> > le32_to_cpu(req->preferred_send_size));
> > st->max_send_size = min_t(int, st->max_send_size,
> > diff --git a/fs/smb/server/transport_rdma.h b/fs/smb/server/transport_rdma.h
> > index 77aee4e5c9dc..0fb692c40e21 100644
> > --- a/fs/smb/server/transport_rdma.h
> > +++ b/fs/smb/server/transport_rdma.h
> > @@ -11,47 +11,6 @@
> > #define SMBD_MIN_IOSIZE (512 * 1024)
> > #define SMBD_MAX_IOSIZE (16 * 1024 * 1024)
> >
> > -/* SMB DIRECT negotiation request packet [MS-SMBD] 2.2.1 */
> > -struct smb_direct_negotiate_req {
> > - __le16 min_version;
> > - __le16 max_version;
> > - __le16 reserved;
> > - __le16 credits_requested;
> > - __le32 preferred_send_size;
> > - __le32 max_receive_size;
> > - __le32 max_fragmented_size;
> > -} __packed;
> > -
> > -/* SMB DIRECT negotiation response packet [MS-SMBD] 2.2.2 */
> > -struct smb_direct_negotiate_resp {
> > - __le16 min_version;
> > - __le16 max_version;
> > - __le16 negotiated_version;
> > - __le16 reserved;
> > - __le16 credits_requested;
> > - __le16 credits_granted;
> > - __le32 status;
> > - __le32 max_readwrite_size;
> > - __le32 preferred_send_size;
> > - __le32 max_receive_size;
> > - __le32 max_fragmented_size;
> > -} __packed;
> > -
> > -#define SMB_DIRECT_RESPONSE_REQUESTED 0x0001
> > -
> > -/* SMB DIRECT data transfer packet with payload [MS-SMBD] 2.2.3 */
> > -struct smb_direct_data_transfer {
> > - __le16 credits_requested;
> > - __le16 credits_granted;
> > - __le16 flags;
> > - __le16 reserved;
> > - __le32 remaining_data_length;
> > - __le32 data_offset;
> > - __le32 data_length;
> > - __le32 padding;
> > - __u8 buffer[];
> > -} __packed;
> > -
> > #ifdef CONFIG_SMB_SERVER_SMBDIRECT
> > int ksmbd_rdma_init(void);
> > void ksmbd_rdma_destroy(void);
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (18 preceding siblings ...)
2025-08-06 17:41 ` [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
@ 2025-08-08 0:22 ` Namjae Jeon
2025-08-11 0:06 ` Namjae Jeon
20 siblings, 0 replies; 26+ messages in thread
From: Namjae Jeon @ 2025-08-08 0:22 UTC (permalink / raw)
To: Stefan Metzmacher
Cc: linux-cifs, samba-technical, Steve French, Tom Talpey, Long Li
On Thu, Aug 7, 2025 at 2:36 AM Stefan Metzmacher <metze@samba.org> wrote:
>
> Hi,
>
> this is the next step towards a common smbdirect layer
> between cifs.ko and ksmbd.ko, with the aim to provide
> a socket layer for userspace usage at the end of the road.
>
> This patchset focuses on the usage of a common
> smbdirect_send_io and related structures in smbdirect_socket.send_io.
>
> Note only patches 01-08 are intended to be merged soon,
> while the ksmbd patches 09-18 are only posted for
> completeness (as discussed with Namjae) to get early feedback.
>
> I used the following xfstests as regression tests:
> cifs/001 generic/001 generic/002 generic/005 generic/006 generic/007 generic/010 generic/011
>
> Between cifs.ko against ksmbd.ko via siw.ko in all combinations
> with and without the patchset on each side.
>
> Stefan Metzmacher (18):
> smb: client: remove unused enum smbd_connection_status
> smb: smbdirect: add SMBDIRECT_RECV_IO_MAX_SGE
> smb: client: make use of SMBDIRECT_RECV_IO_MAX_SGE
> smb: smbdirect: introduce struct smbdirect_send_io
> smb: client: make use of struct smbdirect_send_io
> smb: smbdirect: add smbdirect_socket.{send,recv}_io.mem.{cache,pool}
> smb: client: make use of
> smbdirect_socket.{send,recv}_io.mem.{cache,pool}
> smb: server: make use of common smbdirect_pdu.h
> smb: server: make use of common smbdirect.h
> smb: server: make use of common smbdirect_socket
> smb: server: make use of common smbdirect_socket_parameters
> smb: server: make use of smbdirect_socket->recv_io.expected
> smb: server: make use of struct smbdirect_recv_io
> smb: server: make use of smbdirect_socket.recv_io.free.{list,lock}
> smb: server: make use of smbdirect_socket.recv_io.reassembly.*
> smb: server: make use of SMBDIRECT_RECV_IO_MAX_SGE
> smb: server: make use of struct smbdirect_send_io
> smb: server: make use of
> smbdirect_socket.{send,recv}_io.mem.{cache,pool}
Acked-by: Namjae Jeon <linkinjeon@kernel.org>
Thanks!
>
> fs/smb/client/smbdirect.c | 112 ++--
> fs/smb/client/smbdirect.h | 38 --
> fs/smb/common/smbdirect/smbdirect_socket.h | 54 ++
> fs/smb/server/connection.c | 4 +-
> fs/smb/server/connection.h | 10 +-
> fs/smb/server/smb2pdu.c | 11 +-
> fs/smb/server/smb2pdu.h | 6 -
> fs/smb/server/transport_rdma.c | 742 +++++++++++----------
> fs/smb/server/transport_rdma.h | 41 --
> 9 files changed, 500 insertions(+), 518 deletions(-)
>
> --
> 2.43.0
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
` (19 preceding siblings ...)
2025-08-08 0:22 ` Namjae Jeon
@ 2025-08-11 0:06 ` Namjae Jeon
20 siblings, 0 replies; 26+ messages in thread
From: Namjae Jeon @ 2025-08-11 0:06 UTC (permalink / raw)
To: Stefan Metzmacher
Cc: linux-cifs, samba-technical, Steve French, Tom Talpey, Long Li
On Thu, Aug 7, 2025 at 2:36 AM Stefan Metzmacher <metze@samba.org> wrote:
>
> Hi,
>
> this is the next step towards a common smbdirect layer
> between cifs.ko and ksmbd.ko, with the aim to provide
> a socket layer for userspace usage at the end of the road.
>
> This patchset focuses on the usage of a common
> smbdirect_send_io and related structures in smbdirect_socket.send_io.
>
> Note only patches 01-08 are intended to be merged soon,
> while the ksmbd patches 09-18 are only posted for
> completeness (as discussed with Namjae) to get early feedback.
>
> I used the following xfstests as regression tests:
> cifs/001 generic/001 generic/002 generic/005 generic/006 generic/007 generic/010 generic/011
>
> Between cifs.ko against ksmbd.ko via siw.ko in all combinations
> with and without the patchset on each side.
>
> Stefan Metzmacher (18):
> smb: client: remove unused enum smbd_connection_status
> smb: smbdirect: add SMBDIRECT_RECV_IO_MAX_SGE
> smb: client: make use of SMBDIRECT_RECV_IO_MAX_SGE
> smb: smbdirect: introduce struct smbdirect_send_io
> smb: client: make use of struct smbdirect_send_io
> smb: smbdirect: add smbdirect_socket.{send,recv}_io.mem.{cache,pool}
> smb: client: make use of
> smbdirect_socket.{send,recv}_io.mem.{cache,pool}
> smb: server: make use of common smbdirect_pdu.h
> smb: server: make use of common smbdirect.h
> smb: server: make use of common smbdirect_socket
> smb: server: make use of common smbdirect_socket_parameters
> smb: server: make use of smbdirect_socket->recv_io.expected
> smb: server: make use of struct smbdirect_recv_io
> smb: server: make use of smbdirect_socket.recv_io.free.{list,lock}
> smb: server: make use of smbdirect_socket.recv_io.reassembly.*
> smb: server: make use of SMBDIRECT_RECV_IO_MAX_SGE
> smb: server: make use of struct smbdirect_send_io
> smb: server: make use of
> smbdirect_socket.{send,recv}_io.mem.{cache,pool}
Applied ksmbd's patches to #ksmbd-for-next-next.
Thanks!
>
> fs/smb/client/smbdirect.c | 112 ++--
> fs/smb/client/smbdirect.h | 38 --
> fs/smb/common/smbdirect/smbdirect_socket.h | 54 ++
> fs/smb/server/connection.c | 4 +-
> fs/smb/server/connection.h | 10 +-
> fs/smb/server/smb2pdu.c | 11 +-
> fs/smb/server/smb2pdu.h | 6 -
> fs/smb/server/transport_rdma.c | 742 +++++++++++----------
> fs/smb/server/transport_rdma.h | 41 --
> 9 files changed, 500 insertions(+), 518 deletions(-)
>
> --
> 2.43.0
>
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2025-08-11 0:06 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-08-06 17:35 [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 01/18] smb: client: remove unused enum smbd_connection_status Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 02/18] smb: smbdirect: add SMBDIRECT_RECV_IO_MAX_SGE Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 03/18] smb: client: make use of SMBDIRECT_RECV_IO_MAX_SGE Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 04/18] smb: smbdirect: introduce struct smbdirect_send_io Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 05/18] smb: client: make use of " Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 06/18] smb: smbdirect: add smbdirect_socket.{send,recv}_io.mem.{cache,pool} Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 07/18] smb: client: make use of smbdirect_socket.{send,recv}_io.mem.{cache,pool} Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 08/18] smb: server: make use of common smbdirect_pdu.h Stefan Metzmacher
2025-08-07 15:47 ` ksmbd common smbdirect headers for 6.17-rc1? (Re: [PATCH 08/18] smb: server: make use of common smbdirect_pdu.h) Stefan Metzmacher
2025-08-08 0:10 ` Namjae Jeon
2025-08-06 17:35 ` [PATCH 09/18] smb: server: make use of common smbdirect.h Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 10/18] smb: server: make use of common smbdirect_socket Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 11/18] smb: server: make use of common smbdirect_socket_parameters Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 12/18] smb: server: make use of smbdirect_socket->recv_io.expected Stefan Metzmacher
2025-08-06 17:35 ` [PATCH 13/18] smb: server: make use of struct smbdirect_recv_io Stefan Metzmacher
2025-08-06 17:36 ` [PATCH 14/18] smb: server: make use of smbdirect_socket.recv_io.free.{list,lock} Stefan Metzmacher
2025-08-06 17:36 ` [PATCH 15/18] smb: server: make use of smbdirect_socket.recv_io.reassembly.* Stefan Metzmacher
2025-08-06 17:36 ` [PATCH 16/18] smb: server: make use of SMBDIRECT_RECV_IO_MAX_SGE Stefan Metzmacher
2025-08-06 17:36 ` [PATCH 17/18] smb: server: make use of struct smbdirect_send_io Stefan Metzmacher
2025-08-06 17:36 ` [PATCH 18/18] smb: server: make use of smbdirect_socket.{send,recv}_io.mem.{cache,pool} Stefan Metzmacher
2025-08-06 17:41 ` [PATCH 00/18] smb: smbdirect: more use of common structures e.g. smbdirect_send_io Stefan Metzmacher
2025-08-06 20:21 ` Steve French
2025-08-07 23:21 ` Namjae Jeon
2025-08-08 0:22 ` Namjae Jeon
2025-08-11 0:06 ` Namjae Jeon
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).