* [PATCH 01/10] smb: smbdirect: introduce smbdirect_mr_io.{kref,mutex} and SMBDIRECT_MR_DISABLED
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
@ 2025-10-12 19:10 ` Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 02/10] smb: client: change smbd_deregister_mr() to return void Stefan Metzmacher
` (9 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-12 19:10 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
This will be used in the next commits in order to improve the
client code.
A broken connection can just disable the smbdirect_mr_io while
keeping the memory arround for the caller.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/common/smbdirect/smbdirect_socket.h | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/fs/smb/common/smbdirect/smbdirect_socket.h b/fs/smb/common/smbdirect/smbdirect_socket.h
index db22a1d0546b..361db7f9f623 100644
--- a/fs/smb/common/smbdirect/smbdirect_socket.h
+++ b/fs/smb/common/smbdirect/smbdirect_socket.h
@@ -437,13 +437,22 @@ enum smbdirect_mr_state {
SMBDIRECT_MR_READY,
SMBDIRECT_MR_REGISTERED,
SMBDIRECT_MR_INVALIDATED,
- SMBDIRECT_MR_ERROR
+ SMBDIRECT_MR_ERROR,
+ SMBDIRECT_MR_DISABLED
};
struct smbdirect_mr_io {
struct smbdirect_socket *socket;
struct ib_cqe cqe;
+ /*
+ * We can have up to two references:
+ * 1. by the connection
+ * 2. by the registration
+ */
+ struct kref kref;
+ struct mutex mutex;
+
struct list_head list;
enum smbdirect_mr_state state;
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 02/10] smb: client: change smbd_deregister_mr() to return void
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 01/10] smb: smbdirect: introduce smbdirect_mr_io.{kref,mutex} and SMBDIRECT_MR_DISABLED Stefan Metzmacher
@ 2025-10-12 19:10 ` Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 03/10] smb: client: let destroy_mr_list() call list_del(&mr->list) Stefan Metzmacher
` (8 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-12 19:10 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
No callers checks the return value and this makes further
changes easier.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 4 +---
fs/smb/client/smbdirect.h | 2 +-
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index 316f398c70f4..a20aa2ddf57d 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -2612,7 +2612,7 @@ static void local_inv_done(struct ib_cq *cq, struct ib_wc *wc)
* and we have to locally invalidate the buffer to prevent data is being
* modified by remote peer after upper layer consumes it
*/
-int smbd_deregister_mr(struct smbdirect_mr_io *smbdirect_mr)
+void smbd_deregister_mr(struct smbdirect_mr_io *smbdirect_mr)
{
struct ib_send_wr *wr;
struct smbdirect_socket *sc = smbdirect_mr->socket;
@@ -2662,8 +2662,6 @@ int smbd_deregister_mr(struct smbdirect_mr_io *smbdirect_mr)
done:
if (atomic_dec_and_test(&sc->mr_io.used.count))
wake_up(&sc->mr_io.cleanup.wait_queue);
-
- return rc;
}
static bool smb_set_sge(struct smb_extract_to_rdma *rdma,
diff --git a/fs/smb/client/smbdirect.h b/fs/smb/client/smbdirect.h
index d67ac5ddaff4..577d37dbeb8a 100644
--- a/fs/smb/client/smbdirect.h
+++ b/fs/smb/client/smbdirect.h
@@ -60,7 +60,7 @@ int smbd_send(struct TCP_Server_Info *server,
struct smbdirect_mr_io *smbd_register_mr(
struct smbd_connection *info, struct iov_iter *iter,
bool writing, bool need_invalidate);
-int smbd_deregister_mr(struct smbdirect_mr_io *mr);
+void smbd_deregister_mr(struct smbdirect_mr_io *mr);
#else
#define cifs_rdma_enabled(server) 0
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 03/10] smb: client: let destroy_mr_list() call list_del(&mr->list)
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 01/10] smb: smbdirect: introduce smbdirect_mr_io.{kref,mutex} and SMBDIRECT_MR_DISABLED Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 02/10] smb: client: change smbd_deregister_mr() to return void Stefan Metzmacher
@ 2025-10-12 19:10 ` Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 04/10] smb: client: let destroy_mr_list() remove locked from the list Stefan Metzmacher
` (7 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-12 19:10 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
This makes the code clearer and will make further changes easier.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index a20aa2ddf57d..b7be67dacd09 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -2363,6 +2363,7 @@ static void destroy_mr_list(struct smbdirect_socket *sc)
mr->sgt.nents, mr->dir);
ib_dereg_mr(mr->mr);
kfree(mr->sgt.sgl);
+ list_del(&mr->list);
kfree(mr);
}
}
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 04/10] smb: client: let destroy_mr_list() remove locked from the list
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
` (2 preceding siblings ...)
2025-10-12 19:10 ` [PATCH 03/10] smb: client: let destroy_mr_list() call list_del(&mr->list) Stefan Metzmacher
@ 2025-10-12 19:10 ` Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 05/10] smb: client: improve logic in allocate_mr_list() Stefan Metzmacher
` (6 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-12 19:10 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
This should make sure get_mr() can't see the removed entries.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index b7be67dacd09..b974ca4e0b2e 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -2355,9 +2355,16 @@ static void smbd_mr_recovery_work(struct work_struct *work)
static void destroy_mr_list(struct smbdirect_socket *sc)
{
struct smbdirect_mr_io *mr, *tmp;
+ LIST_HEAD(all_list);
+ unsigned long flags;
disable_work_sync(&sc->mr_io.recovery_work);
- list_for_each_entry_safe(mr, tmp, &sc->mr_io.all.list, list) {
+
+ spin_lock_irqsave(&sc->mr_io.all.lock, flags);
+ list_splice_tail_init(&sc->mr_io.all.list, &all_list);
+ spin_unlock_irqrestore(&sc->mr_io.all.lock, flags);
+
+ list_for_each_entry_safe(mr, tmp, &all_list, list) {
if (mr->state == SMBDIRECT_MR_INVALIDATED)
ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl,
mr->sgt.nents, mr->dir);
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 05/10] smb: client: improve logic in allocate_mr_list()
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
` (3 preceding siblings ...)
2025-10-12 19:10 ` [PATCH 04/10] smb: client: let destroy_mr_list() remove locked from the list Stefan Metzmacher
@ 2025-10-12 19:10 ` Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 06/10] smb: client: improve logic in smbd_register_mr() Stefan Metzmacher
` (5 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-12 19:10 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
- use 'mr' as variable name
- use goto lables for easier cleanup
- use destroy_mr_list()
- style fixes
- INIT_WORK(&sc->mr_io.recovery_work, smbd_mr_recovery_work) on success
This will make further changes easier.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 65 +++++++++++++++++++++------------------
1 file changed, 35 insertions(+), 30 deletions(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index b974ca4e0b2e..658ca11cb26c 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -2385,10 +2385,9 @@ static void destroy_mr_list(struct smbdirect_socket *sc)
static int allocate_mr_list(struct smbdirect_socket *sc)
{
struct smbdirect_socket_parameters *sp = &sc->parameters;
- int i;
- struct smbdirect_mr_io *smbdirect_mr, *tmp;
-
- INIT_WORK(&sc->mr_io.recovery_work, smbd_mr_recovery_work);
+ struct smbdirect_mr_io *mr;
+ int ret;
+ u32 i;
if (sp->responder_resources == 0) {
log_rdma_mr(ERR, "responder_resources negotiated as 0\n");
@@ -2397,42 +2396,48 @@ static int allocate_mr_list(struct smbdirect_socket *sc)
/* Allocate more MRs (2x) than hardware responder_resources */
for (i = 0; i < sp->responder_resources * 2; i++) {
- smbdirect_mr = kzalloc(sizeof(*smbdirect_mr), GFP_KERNEL);
- if (!smbdirect_mr)
- goto cleanup_entries;
- smbdirect_mr->mr = ib_alloc_mr(sc->ib.pd, sc->mr_io.type,
- sp->max_frmr_depth);
- if (IS_ERR(smbdirect_mr->mr)) {
+ mr = kzalloc(sizeof(*mr), GFP_KERNEL);
+ if (!mr) {
+ ret = -ENOMEM;
+ goto kzalloc_mr_failed;
+ }
+
+ mr->mr = ib_alloc_mr(sc->ib.pd,
+ sc->mr_io.type,
+ sp->max_frmr_depth);
+ if (IS_ERR(mr->mr)) {
+ ret = PTR_ERR(mr->mr);
log_rdma_mr(ERR, "ib_alloc_mr failed mr_type=%x max_frmr_depth=%x\n",
sc->mr_io.type, sp->max_frmr_depth);
- goto out;
+ goto ib_alloc_mr_failed;
}
- smbdirect_mr->sgt.sgl = kcalloc(sp->max_frmr_depth,
- sizeof(struct scatterlist),
- GFP_KERNEL);
- if (!smbdirect_mr->sgt.sgl) {
+
+ mr->sgt.sgl = kcalloc(sp->max_frmr_depth,
+ sizeof(struct scatterlist),
+ GFP_KERNEL);
+ if (!mr->sgt.sgl) {
+ ret = -ENOMEM;
log_rdma_mr(ERR, "failed to allocate sgl\n");
- ib_dereg_mr(smbdirect_mr->mr);
- goto out;
+ goto kcalloc_sgl_failed;
}
- smbdirect_mr->state = SMBDIRECT_MR_READY;
- smbdirect_mr->socket = sc;
+ mr->state = SMBDIRECT_MR_READY;
+ mr->socket = sc;
- list_add_tail(&smbdirect_mr->list, &sc->mr_io.all.list);
+ list_add_tail(&mr->list, &sc->mr_io.all.list);
atomic_inc(&sc->mr_io.ready.count);
}
+
+ INIT_WORK(&sc->mr_io.recovery_work, smbd_mr_recovery_work);
+
return 0;
-out:
- kfree(smbdirect_mr);
-cleanup_entries:
- list_for_each_entry_safe(smbdirect_mr, tmp, &sc->mr_io.all.list, list) {
- list_del(&smbdirect_mr->list);
- ib_dereg_mr(smbdirect_mr->mr);
- kfree(smbdirect_mr->sgt.sgl);
- kfree(smbdirect_mr);
- }
- return -ENOMEM;
+kcalloc_sgl_failed:
+ ib_dereg_mr(mr->mr);
+ib_alloc_mr_failed:
+ kfree(mr);
+kzalloc_mr_failed:
+ destroy_mr_list(sc);
+ return ret;
}
/*
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 06/10] smb: client: improve logic in smbd_register_mr()
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
` (4 preceding siblings ...)
2025-10-12 19:10 ` [PATCH 05/10] smb: client: improve logic in allocate_mr_list() Stefan Metzmacher
@ 2025-10-12 19:10 ` Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 07/10] smb: client: improve logic in smbd_deregister_mr() Stefan Metzmacher
` (4 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-12 19:10 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
- use 'mr' as variable name
- style fixes
This will make further changes easier.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 52 +++++++++++++++++----------------------
1 file changed, 23 insertions(+), 29 deletions(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index 658ca11cb26c..a863b6fff87a 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -2517,9 +2517,8 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,
{
struct smbdirect_socket *sc = &info->socket;
struct smbdirect_socket_parameters *sp = &sc->parameters;
- struct smbdirect_mr_io *smbdirect_mr;
+ struct smbdirect_mr_io *mr;
int rc, num_pages;
- enum dma_data_direction dir;
struct ib_reg_wr *reg_wr;
num_pages = iov_iter_npages(iter, sp->max_frmr_depth + 1);
@@ -2530,49 +2529,45 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,
return NULL;
}
- smbdirect_mr = get_mr(sc);
- if (!smbdirect_mr) {
+ mr = get_mr(sc);
+ if (!mr) {
log_rdma_mr(ERR, "get_mr returning NULL\n");
return NULL;
}
- dir = writing ? DMA_FROM_DEVICE : DMA_TO_DEVICE;
- smbdirect_mr->dir = dir;
- smbdirect_mr->need_invalidate = need_invalidate;
- smbdirect_mr->sgt.nents = 0;
- smbdirect_mr->sgt.orig_nents = 0;
+ mr->dir = writing ? DMA_FROM_DEVICE : DMA_TO_DEVICE;
+ mr->need_invalidate = need_invalidate;
+ mr->sgt.nents = 0;
+ mr->sgt.orig_nents = 0;
log_rdma_mr(INFO, "num_pages=0x%x count=0x%zx depth=%u\n",
num_pages, iov_iter_count(iter), sp->max_frmr_depth);
- smbd_iter_to_mr(iter, &smbdirect_mr->sgt, sp->max_frmr_depth);
+ smbd_iter_to_mr(iter, &mr->sgt, sp->max_frmr_depth);
- rc = ib_dma_map_sg(sc->ib.dev, smbdirect_mr->sgt.sgl,
- smbdirect_mr->sgt.nents, dir);
+ rc = ib_dma_map_sg(sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir);
if (!rc) {
log_rdma_mr(ERR, "ib_dma_map_sg num_pages=%x dir=%x rc=%x\n",
- num_pages, dir, rc);
+ num_pages, mr->dir, rc);
goto dma_map_error;
}
- rc = ib_map_mr_sg(smbdirect_mr->mr, smbdirect_mr->sgt.sgl,
- smbdirect_mr->sgt.nents, NULL, PAGE_SIZE);
- if (rc != smbdirect_mr->sgt.nents) {
+ rc = ib_map_mr_sg(mr->mr, mr->sgt.sgl, mr->sgt.nents, NULL, PAGE_SIZE);
+ if (rc != mr->sgt.nents) {
log_rdma_mr(ERR,
- "ib_map_mr_sg failed rc = %d nents = %x\n",
- rc, smbdirect_mr->sgt.nents);
+ "ib_map_mr_sg failed rc = %d nents = %x\n",
+ rc, mr->sgt.nents);
goto map_mr_error;
}
- ib_update_fast_reg_key(smbdirect_mr->mr,
- ib_inc_rkey(smbdirect_mr->mr->rkey));
- reg_wr = &smbdirect_mr->wr;
+ ib_update_fast_reg_key(mr->mr, ib_inc_rkey(mr->mr->rkey));
+ reg_wr = &mr->wr;
reg_wr->wr.opcode = IB_WR_REG_MR;
- smbdirect_mr->cqe.done = register_mr_done;
- reg_wr->wr.wr_cqe = &smbdirect_mr->cqe;
+ mr->cqe.done = register_mr_done;
+ reg_wr->wr.wr_cqe = &mr->cqe;
reg_wr->wr.num_sge = 0;
reg_wr->wr.send_flags = IB_SEND_SIGNALED;
- reg_wr->mr = smbdirect_mr->mr;
- reg_wr->key = smbdirect_mr->mr->rkey;
+ reg_wr->mr = mr->mr;
+ reg_wr->key = mr->mr->rkey;
reg_wr->access = writing ?
IB_ACCESS_REMOTE_WRITE | IB_ACCESS_LOCAL_WRITE :
IB_ACCESS_REMOTE_READ;
@@ -2584,18 +2579,17 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,
*/
rc = ib_post_send(sc->ib.qp, ®_wr->wr, NULL);
if (!rc)
- return smbdirect_mr;
+ return mr;
log_rdma_mr(ERR, "ib_post_send failed rc=%x reg_wr->key=%x\n",
rc, reg_wr->key);
/* If all failed, attempt to recover this MR by setting it SMBDIRECT_MR_ERROR*/
map_mr_error:
- ib_dma_unmap_sg(sc->ib.dev, smbdirect_mr->sgt.sgl,
- smbdirect_mr->sgt.nents, smbdirect_mr->dir);
+ ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir);
dma_map_error:
- smbdirect_mr->state = SMBDIRECT_MR_ERROR;
+ mr->state = SMBDIRECT_MR_ERROR;
if (atomic_dec_and_test(&sc->mr_io.used.count))
wake_up(&sc->mr_io.cleanup.wait_queue);
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 07/10] smb: client: improve logic in smbd_deregister_mr()
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
` (5 preceding siblings ...)
2025-10-12 19:10 ` [PATCH 06/10] smb: client: improve logic in smbd_register_mr() Stefan Metzmacher
@ 2025-10-12 19:10 ` Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 08/10] smb: client: call ib_dma_unmap_sg if mr->sgt.nents is not 0 Stefan Metzmacher
` (3 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-12 19:10 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
- use 'mr' as variable name
- style fixes
This will make further changes easier.
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 35 ++++++++++++++++-------------------
1 file changed, 16 insertions(+), 19 deletions(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index a863b6fff87a..af0642e94d7e 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -2619,44 +2619,41 @@ static void local_inv_done(struct ib_cq *cq, struct ib_wc *wc)
* and we have to locally invalidate the buffer to prevent data is being
* modified by remote peer after upper layer consumes it
*/
-void smbd_deregister_mr(struct smbdirect_mr_io *smbdirect_mr)
+void smbd_deregister_mr(struct smbdirect_mr_io *mr)
{
- struct ib_send_wr *wr;
- struct smbdirect_socket *sc = smbdirect_mr->socket;
- int rc = 0;
+ struct smbdirect_socket *sc = mr->socket;
+
+ if (mr->need_invalidate) {
+ struct ib_send_wr *wr = &mr->inv_wr;
+ int rc;
- if (smbdirect_mr->need_invalidate) {
/* Need to finish local invalidation before returning */
- wr = &smbdirect_mr->inv_wr;
wr->opcode = IB_WR_LOCAL_INV;
- smbdirect_mr->cqe.done = local_inv_done;
- wr->wr_cqe = &smbdirect_mr->cqe;
+ mr->cqe.done = local_inv_done;
+ wr->wr_cqe = &mr->cqe;
wr->num_sge = 0;
- wr->ex.invalidate_rkey = smbdirect_mr->mr->rkey;
+ wr->ex.invalidate_rkey = mr->mr->rkey;
wr->send_flags = IB_SEND_SIGNALED;
- init_completion(&smbdirect_mr->invalidate_done);
+ init_completion(&mr->invalidate_done);
rc = ib_post_send(sc->ib.qp, wr, NULL);
if (rc) {
log_rdma_mr(ERR, "ib_post_send failed rc=%x\n", rc);
smbd_disconnect_rdma_connection(sc);
goto done;
}
- wait_for_completion(&smbdirect_mr->invalidate_done);
- smbdirect_mr->need_invalidate = false;
+ wait_for_completion(&mr->invalidate_done);
+ mr->need_invalidate = false;
} else
/*
* For remote invalidation, just set it to SMBDIRECT_MR_INVALIDATED
* and defer to mr_recovery_work to recover the MR for next use
*/
- smbdirect_mr->state = SMBDIRECT_MR_INVALIDATED;
+ mr->state = SMBDIRECT_MR_INVALIDATED;
- if (smbdirect_mr->state == SMBDIRECT_MR_INVALIDATED) {
- ib_dma_unmap_sg(
- sc->ib.dev, smbdirect_mr->sgt.sgl,
- smbdirect_mr->sgt.nents,
- smbdirect_mr->dir);
- smbdirect_mr->state = SMBDIRECT_MR_READY;
+ if (mr->state == SMBDIRECT_MR_INVALIDATED) {
+ ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir);
+ mr->state = SMBDIRECT_MR_READY;
if (atomic_inc_return(&sc->mr_io.ready.count) == 1)
wake_up(&sc->mr_io.ready.wait_queue);
} else
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 08/10] smb: client: call ib_dma_unmap_sg if mr->sgt.nents is not 0
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
` (6 preceding siblings ...)
2025-10-12 19:10 ` [PATCH 07/10] smb: client: improve logic in smbd_deregister_mr() Stefan Metzmacher
@ 2025-10-12 19:10 ` Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 09/10] smb: client: let destroy_mr_list() call ib_dereg_mr() before ib_dma_unmap_sg() Stefan Metzmacher
` (2 subsequent siblings)
10 siblings, 0 replies; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-12 19:10 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
This seems to be the more reliable way to check if we need to
call ib_dma_unmap_sg().
Fixes: c7398583340a ("CIFS: SMBD: Implement RDMA memory registration")
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index af0642e94d7e..21dcd326af3d 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -2365,9 +2365,8 @@ static void destroy_mr_list(struct smbdirect_socket *sc)
spin_unlock_irqrestore(&sc->mr_io.all.lock, flags);
list_for_each_entry_safe(mr, tmp, &all_list, list) {
- if (mr->state == SMBDIRECT_MR_INVALIDATED)
- ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl,
- mr->sgt.nents, mr->dir);
+ if (mr->sgt.nents)
+ ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir);
ib_dereg_mr(mr->mr);
kfree(mr->sgt.sgl);
list_del(&mr->list);
@@ -2589,6 +2588,7 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,
ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir);
dma_map_error:
+ mr->sgt.nents = 0;
mr->state = SMBDIRECT_MR_ERROR;
if (atomic_dec_and_test(&sc->mr_io.used.count))
wake_up(&sc->mr_io.cleanup.wait_queue);
@@ -2651,8 +2651,12 @@ void smbd_deregister_mr(struct smbdirect_mr_io *mr)
*/
mr->state = SMBDIRECT_MR_INVALIDATED;
- if (mr->state == SMBDIRECT_MR_INVALIDATED) {
+ if (mr->sgt.nents) {
ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir);
+ mr->sgt.nents = 0;
+ }
+
+ if (mr->state == SMBDIRECT_MR_INVALIDATED) {
mr->state = SMBDIRECT_MR_READY;
if (atomic_inc_return(&sc->mr_io.ready.count) == 1)
wake_up(&sc->mr_io.ready.wait_queue);
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 09/10] smb: client: let destroy_mr_list() call ib_dereg_mr() before ib_dma_unmap_sg()
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
` (7 preceding siblings ...)
2025-10-12 19:10 ` [PATCH 08/10] smb: client: call ib_dma_unmap_sg if mr->sgt.nents is not 0 Stefan Metzmacher
@ 2025-10-12 19:10 ` Stefan Metzmacher
2025-10-12 19:10 ` [PATCH 10/10] smb: client: let destroy_mr_list() keep smbdirect_mr_io memory if registered Stefan Metzmacher
2025-10-13 3:05 ` [PATCH 00/10] improve smbdirect_mr_io lifetime Steve French
10 siblings, 0 replies; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-12 19:10 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
This is more consistent as we call ib_dma_unmap_sg() only
when the memory is no longer registered.
This is the same pattern as calling ib_dma_unmap_sg() after
IB_WR_LOCAL_INV.
Fixes: c7398583340a ("CIFS: SMBD: Implement RDMA memory registration")
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index 21dcd326af3d..c3330e43488f 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -2365,9 +2365,10 @@ static void destroy_mr_list(struct smbdirect_socket *sc)
spin_unlock_irqrestore(&sc->mr_io.all.lock, flags);
list_for_each_entry_safe(mr, tmp, &all_list, list) {
+ if (mr->mr)
+ ib_dereg_mr(mr->mr);
if (mr->sgt.nents)
ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir);
- ib_dereg_mr(mr->mr);
kfree(mr->sgt.sgl);
list_del(&mr->list);
kfree(mr);
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH 10/10] smb: client: let destroy_mr_list() keep smbdirect_mr_io memory if registered
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
` (8 preceding siblings ...)
2025-10-12 19:10 ` [PATCH 09/10] smb: client: let destroy_mr_list() call ib_dereg_mr() before ib_dma_unmap_sg() Stefan Metzmacher
@ 2025-10-12 19:10 ` Stefan Metzmacher
2025-10-15 7:20 ` Stefan Metzmacher
2025-10-13 3:05 ` [PATCH 00/10] improve smbdirect_mr_io lifetime Steve French
10 siblings, 1 reply; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-12 19:10 UTC (permalink / raw)
To: linux-cifs, samba-technical
Cc: metze, Steve French, Tom Talpey, Long Li, Namjae Jeon
If a smbdirect_mr_io structure if still visible to callers of
smbd_register_mr() we can't free the related memory when the
connection is disconnected! Otherwise smbd_deregister_mr()
will crash.
Now we use a mutex and refcounting in order to keep the
memory around if the connection is disconnected.
It means smbd_deregister_mr() can be called at any later time to free
the memory, which is no longer referenced by nor referencing the
connection.
It also means smbd_destroy() no longer needs to wait for
mr_io.used.count to become 0.
Fixes: 050b8c374019 ("smbd: Make upper layer decide when to destroy the transport")
Cc: Steve French <smfrench@gmail.com>
Cc: Tom Talpey <tom@talpey.com>
Cc: Long Li <longli@microsoft.com>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: linux-cifs@vger.kernel.org
Cc: samba-technical@lists.samba.org
Signed-off-by: Stefan Metzmacher <metze@samba.org>
---
fs/smb/client/smbdirect.c | 145 +++++++++++++++++++++++++++++++++-----
1 file changed, 126 insertions(+), 19 deletions(-)
diff --git a/fs/smb/client/smbdirect.c b/fs/smb/client/smbdirect.c
index c3330e43488f..e78b4ceb6d32 100644
--- a/fs/smb/client/smbdirect.c
+++ b/fs/smb/client/smbdirect.c
@@ -1624,19 +1624,7 @@ void smbd_destroy(struct TCP_Server_Info *server)
log_rdma_event(INFO, "free receive buffers\n");
destroy_receive_buffers(sc);
- /*
- * For performance reasons, memory registration and deregistration
- * are not locked by srv_mutex. It is possible some processes are
- * blocked on transport srv_mutex while holding memory registration.
- * Release the transport srv_mutex to allow them to hit the failure
- * path when sending data, and then release memory registrations.
- */
log_rdma_event(INFO, "freeing mr list\n");
- while (atomic_read(&sc->mr_io.used.count)) {
- cifs_server_unlock(server);
- msleep(1000);
- cifs_server_lock(server);
- }
destroy_mr_list(sc);
ib_free_cq(sc->ib.send_cq);
@@ -2352,6 +2340,46 @@ static void smbd_mr_recovery_work(struct work_struct *work)
}
}
+static void smbd_mr_disable_locked(struct smbdirect_mr_io *mr)
+{
+ struct smbdirect_socket *sc = mr->socket;
+
+ lockdep_assert_held(&mr->mutex);
+
+ if (mr->state == SMBDIRECT_MR_DISABLED)
+ return;
+
+ if (mr->mr)
+ ib_dereg_mr(mr->mr);
+ if (mr->sgt.nents)
+ ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir);
+ kfree(mr->sgt.sgl);
+
+ mr->mr = NULL;
+ mr->sgt.sgl = NULL;
+ mr->sgt.nents = 0;
+
+ mr->state = SMBDIRECT_MR_DISABLED;
+}
+
+static void smbd_mr_free_locked(struct kref *kref)
+{
+ struct smbdirect_mr_io *mr =
+ container_of(kref, struct smbdirect_mr_io, kref);
+
+ lockdep_assert_held(&mr->mutex);
+
+ /*
+ * smbd_mr_disable_locked() should already be called!
+ */
+ if (WARN_ON_ONCE(mr->state != SMBDIRECT_MR_DISABLED))
+ smbd_mr_disable_locked(mr);
+
+ mutex_unlock(&mr->mutex);
+ mutex_destroy(&mr->mutex);
+ kfree(mr);
+}
+
static void destroy_mr_list(struct smbdirect_socket *sc)
{
struct smbdirect_mr_io *mr, *tmp;
@@ -2365,13 +2393,31 @@ static void destroy_mr_list(struct smbdirect_socket *sc)
spin_unlock_irqrestore(&sc->mr_io.all.lock, flags);
list_for_each_entry_safe(mr, tmp, &all_list, list) {
- if (mr->mr)
- ib_dereg_mr(mr->mr);
- if (mr->sgt.nents)
- ib_dma_unmap_sg(sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir);
- kfree(mr->sgt.sgl);
+ mutex_lock(&mr->mutex);
+
+ smbd_mr_disable_locked(mr);
list_del(&mr->list);
- kfree(mr);
+ mr->socket = NULL;
+
+ /*
+ * No kref_put_mutex() as it's already locked.
+ *
+ * If smbd_mr_free_locked() is called
+ * and the mutex is unlocked and mr is gone,
+ * in that case kref_put() returned 1.
+ *
+ * If kref_put() returned 0 we know that
+ * smbd_mr_free_locked() didn't
+ * run. Not by us nor by anyone else, as we
+ * still hold the mutex, so we need to unlock.
+ *
+ * If the mr is still registered it will
+ * be dangling (detached from the connection
+ * waiting for smbd_deregister_mr() to be
+ * called in order to free the memory.
+ */
+ if (!kref_put(&mr->kref, smbd_mr_free_locked))
+ mutex_unlock(&mr->mutex);
}
}
@@ -2402,6 +2448,9 @@ static int allocate_mr_list(struct smbdirect_socket *sc)
goto kzalloc_mr_failed;
}
+ kref_init(&mr->kref);
+ mutex_init(&mr->mutex);
+
mr->mr = ib_alloc_mr(sc->ib.pd,
sc->mr_io.type,
sp->max_frmr_depth);
@@ -2471,6 +2520,7 @@ static struct smbdirect_mr_io *get_mr(struct smbdirect_socket *sc)
list_for_each_entry(ret, &sc->mr_io.all.list, list) {
if (ret->state == SMBDIRECT_MR_READY) {
ret->state = SMBDIRECT_MR_REGISTERED;
+ kref_get(&ret->kref);
spin_unlock_irqrestore(&sc->mr_io.all.lock, flags);
atomic_dec(&sc->mr_io.ready.count);
atomic_inc(&sc->mr_io.used.count);
@@ -2535,6 +2585,8 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,
return NULL;
}
+ mutex_lock(&mr->mutex);
+
mr->dir = writing ? DMA_FROM_DEVICE : DMA_TO_DEVICE;
mr->need_invalidate = need_invalidate;
mr->sgt.nents = 0;
@@ -2578,8 +2630,16 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,
* on the next ib_post_send when we actually send I/O to remote peer
*/
rc = ib_post_send(sc->ib.qp, ®_wr->wr, NULL);
- if (!rc)
+ if (!rc) {
+ /*
+ * get_mr() gave us a reference
+ * via kref_get(&mr->kref), we keep that and let
+ * the caller use smbd_deregister_mr()
+ * to remove it again.
+ */
+ mutex_unlock(&mr->mutex);
return mr;
+ }
log_rdma_mr(ERR, "ib_post_send failed rc=%x reg_wr->key=%x\n",
rc, reg_wr->key);
@@ -2596,6 +2656,25 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,
smbd_disconnect_rdma_connection(sc);
+ /*
+ * get_mr() gave us a reference
+ * via kref_get(&mr->kref), we need to remove it again
+ * on error.
+ *
+ * No kref_put_mutex() as it's already locked.
+ *
+ * If smbd_mr_free_locked() is called
+ * and the mutex is unlocked and mr is gone,
+ * in that case kref_put() returned 1.
+ *
+ * If kref_put() returned 0 we know that
+ * smbd_mr_free_locked() didn't
+ * run. Not by us nor by anyone else, as we
+ * still hold the mutex, so we need to unlock.
+ */
+ if (!kref_put(&mr->kref, smbd_mr_free_locked))
+ mutex_unlock(&mr->mutex);
+
return NULL;
}
@@ -2624,6 +2703,15 @@ void smbd_deregister_mr(struct smbdirect_mr_io *mr)
{
struct smbdirect_socket *sc = mr->socket;
+ mutex_lock(&mr->mutex);
+ if (mr->state == SMBDIRECT_MR_DISABLED)
+ goto put_kref;
+
+ if (sc->status != SMBDIRECT_SOCKET_CONNECTED) {
+ smbd_mr_disable_locked(mr);
+ goto put_kref;
+ }
+
if (mr->need_invalidate) {
struct ib_send_wr *wr = &mr->inv_wr;
int rc;
@@ -2640,6 +2728,7 @@ void smbd_deregister_mr(struct smbdirect_mr_io *mr)
rc = ib_post_send(sc->ib.qp, wr, NULL);
if (rc) {
log_rdma_mr(ERR, "ib_post_send failed rc=%x\n", rc);
+ smbd_mr_disable_locked(mr);
smbd_disconnect_rdma_connection(sc);
goto done;
}
@@ -2671,6 +2760,24 @@ void smbd_deregister_mr(struct smbdirect_mr_io *mr)
done:
if (atomic_dec_and_test(&sc->mr_io.used.count))
wake_up(&sc->mr_io.cleanup.wait_queue);
+
+put_kref:
+ /*
+ * No kref_put_mutex() as it's already locked.
+ *
+ * If smbd_mr_free_locked() is called
+ * and the mutex is unlocked and mr is gone,
+ * in that case kref_put() returned 1.
+ *
+ * If kref_put() returned 0 we know that
+ * smbd_mr_free_locked() didn't
+ * run. Not by us nor by anyone else, as we
+ * still hold the mutex, so we need to unlock
+ * and keep the mr in SMBDIRECT_MR_READY or
+ * SMBDIRECT_MR_ERROR state.
+ */
+ if (!kref_put(&mr->kref, smbd_mr_free_locked))
+ mutex_unlock(&mr->mutex);
}
static bool smb_set_sge(struct smb_extract_to_rdma *rdma,
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [PATCH 10/10] smb: client: let destroy_mr_list() keep smbdirect_mr_io memory if registered
2025-10-12 19:10 ` [PATCH 10/10] smb: client: let destroy_mr_list() keep smbdirect_mr_io memory if registered Stefan Metzmacher
@ 2025-10-15 7:20 ` Stefan Metzmacher
2025-10-15 12:47 ` Steve French
0 siblings, 1 reply; 14+ messages in thread
From: Stefan Metzmacher @ 2025-10-15 7:20 UTC (permalink / raw)
To: Steve French
Cc: Tom Talpey, Long Li, Namjae Jeon, linux-cifs, samba-technical
Hi Steve,
as already discussed, one additional hunk is needed...
> @@ -2402,6 +2448,9 @@ static int allocate_mr_list(struct smbdirect_socket *sc)
> goto kzalloc_mr_failed;
> }
>
> + kref_init(&mr->kref);
> + mutex_init(&mr->mutex);
> +
> mr->mr = ib_alloc_mr(sc->ib.pd,
> sc->mr_io.type,
> sp->max_frmr_depth);
Here we're missing the following hunk:
@@ -2483,6 +2483,7 @@ static int allocate_mr_list(struct smbdirect_socket *sc)
kcalloc_sgl_failed:
ib_dereg_mr(mr->mr);
ib_alloc_mr_failed:
+ mutex_destroy(&mr->mutex);
kfree(mr);
kzalloc_mr_failed:
destroy_mr_list(sc);
I fixed it in my for-6.18/fs-smb-20251015-v2 branch,
up to commit e4418cd1d5d80a8c24530ac0a41a5451c44f82bf.
git fetch https://git.samba.org/metze/linux/wip.git for-6.18/fs-smb-20251015-v2
The above hunk is the only difference to the current sfrench-cifs-2.6/for-next
(at commit 7949ce089965bd025a8d46dbaa2f5d0a2bd4ec77), I only moved my commits
to the top. So you can just replace 7949ce089965bd025a8d46dbaa2f5d0a2bd4ec77 by
e4418cd1d5d80a8c24530ac0a41a5451c44f82bf.
Thanks!
metze
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [PATCH 10/10] smb: client: let destroy_mr_list() keep smbdirect_mr_io memory if registered
2025-10-15 7:20 ` Stefan Metzmacher
@ 2025-10-15 12:47 ` Steve French
0 siblings, 0 replies; 14+ messages in thread
From: Steve French @ 2025-10-15 12:47 UTC (permalink / raw)
To: Stefan Metzmacher
Cc: Tom Talpey, Long Li, Namjae Jeon, linux-cifs, samba-technical
cifs-2.6.git for-next updated with newer version of "smb: client: let
destroy_mr_list() keep smbdirect_mr_io memory if registered"
Let me know if additional changes needed or if you spot any issues
On Wed, Oct 15, 2025 at 2:20 AM Stefan Metzmacher <metze@samba.org> wrote:
>
> Hi Steve,
>
> as already discussed, one additional hunk is needed...
>
> > @@ -2402,6 +2448,9 @@ static int allocate_mr_list(struct smbdirect_socket *sc)
> > goto kzalloc_mr_failed;
> > }
> >
> > + kref_init(&mr->kref);
> > + mutex_init(&mr->mutex);
> > +
> > mr->mr = ib_alloc_mr(sc->ib.pd,
> > sc->mr_io.type,
> > sp->max_frmr_depth);
>
> Here we're missing the following hunk:
>
> @@ -2483,6 +2483,7 @@ static int allocate_mr_list(struct smbdirect_socket *sc)
> kcalloc_sgl_failed:
> ib_dereg_mr(mr->mr);
> ib_alloc_mr_failed:
> + mutex_destroy(&mr->mutex);
> kfree(mr);
> kzalloc_mr_failed:
> destroy_mr_list(sc);
>
> I fixed it in my for-6.18/fs-smb-20251015-v2 branch,
> up to commit e4418cd1d5d80a8c24530ac0a41a5451c44f82bf.
> git fetch https://git.samba.org/metze/linux/wip.git for-6.18/fs-smb-20251015-v2
>
> The above hunk is the only difference to the current sfrench-cifs-2.6/for-next
> (at commit 7949ce089965bd025a8d46dbaa2f5d0a2bd4ec77), I only moved my commits
> to the top. So you can just replace 7949ce089965bd025a8d46dbaa2f5d0a2bd4ec77 by
> e4418cd1d5d80a8c24530ac0a41a5451c44f82bf.
>
> Thanks!
> metze
>
--
Thanks,
Steve
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 00/10] improve smbdirect_mr_io lifetime
2025-10-12 19:10 [PATCH 00/10] improve smbdirect_mr_io lifetime Stefan Metzmacher
` (9 preceding siblings ...)
2025-10-12 19:10 ` [PATCH 10/10] smb: client: let destroy_mr_list() keep smbdirect_mr_io memory if registered Stefan Metzmacher
@ 2025-10-13 3:05 ` Steve French
10 siblings, 0 replies; 14+ messages in thread
From: Steve French @ 2025-10-13 3:05 UTC (permalink / raw)
To: Stefan Metzmacher
Cc: linux-cifs, samba-technical, Tom Talpey, Long Li, Namjae Jeon
merged into cifs-2.6.git for-next pending more testing
On Sun, Oct 12, 2025 at 2:10 PM Stefan Metzmacher <metze@samba.org> wrote:
>
> Hi,
>
> these patches improve and simplify our handling of
> smbdirect_mr_io structures and their lifetime.
>
> smbd_register_mr() returns a pointer to struct smbdirect_mr_io
> and smbd_deregister_mr() gives the pointer back.
>
> But currently the memory itself is managed by the connection
> (struct smbdirect_socket) and smbd_destroy() has a strange
> wait loop in order to wait for smbd_deregister_mr() being
> called. It means code in smbd_destroy() is aware of
> the server mutex in the generic smb client handling above
> the transport layer.
>
> These patches do some cleanups and fixes before changing
> the logic to use a kref and a mutex in order to allow
> smbd_deregister_mr() being called after smbd_destroy()
> as the memory of smbdirect_mr_io will stay in memory
> but will be detached from the connection.
>
> This makes the code independent of cifs_server_[un]lock()
> and will allow us to move more smbdirect code into common
> functions (shared between client and server).
>
> I think these should go into 6.18.
>
> Stefan Metzmacher (10):
> smb: smbdirect: introduce smbdirect_mr_io.{kref,mutex} and
> SMBDIRECT_MR_DISABLED
> smb: client: change smbd_deregister_mr() to return void
> smb: client: let destroy_mr_list() call list_del(&mr->list)
> smb: client: let destroy_mr_list() remove locked from the list
> smb: client: improve logic in allocate_mr_list()
> smb: client: improve logic in smbd_register_mr()
> smb: client: improve logic in smbd_deregister_mr()
> smb: client: call ib_dma_unmap_sg if mr->sgt.nents is not 0
> smb: client: let destroy_mr_list() call ib_dereg_mr() before
> ib_dma_unmap_sg()
> smb: client: let destroy_mr_list() keep smbdirect_mr_io memory if
> registered
>
> fs/smb/client/smbdirect.c | 312 ++++++++++++++-------
> fs/smb/client/smbdirect.h | 2 +-
> fs/smb/common/smbdirect/smbdirect_socket.h | 11 +-
> 3 files changed, 224 insertions(+), 101 deletions(-)
>
> --
> 2.43.0
>
--
Thanks,
Steve
^ permalink raw reply [flat|nested] 14+ messages in thread