public inbox for linux-nvme@lists.infradead.org
 help / color / mirror / Atom feed
* [PATCH 0/5] nvme-fc: misc small improvents
@ 2025-10-28 15:26 Daniel Wagner
  2025-10-28 15:26 ` [PATCH 1/5] nvme-fc: don't hold rport lock when putting ctrl Daniel Wagner
                   ` (4 more replies)
  0 siblings, 5 replies; 18+ messages in thread
From: Daniel Wagner @ 2025-10-28 15:26 UTC (permalink / raw)
  To: Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel, Daniel Wagner

I've collected a bunch of patches from the last debugging session. I think it's
worth documenting which conditions are expected in the cleanup path by adding a
bunch of WARNs.

Also found a deadlock in the nvme-fc code, so this one should definitly go in.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
Daniel Wagner (5):
      nvme-fc: don't hold rport lock when putting ctrl
      nvme-fc: check all request and response have been processed
      nvmet-fcloop: check all request and response have been processed
      nvmet-fcloop: remove unused lsdir member.
      nvmet-fc: use pr_* print macros instead of dev_*

 drivers/nvme/host/fc.c       |  4 ++++
 drivers/nvme/target/fc.c     | 48 +++++++++++++++++++-------------------------
 drivers/nvme/target/fcloop.c |  9 ++++++---
 3 files changed, 31 insertions(+), 30 deletions(-)
---
base-commit: 77a4fe6a06e265bd94d2b3cdc87fb3cde877a05b
change-id: 20251028-nvmet-fcloop-fixes-4e57e82f1f15

Best regards,
-- 
Daniel Wagner <wagi@kernel.org>



^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH 1/5] nvme-fc: don't hold rport lock when putting ctrl
  2025-10-28 15:26 [PATCH 0/5] nvme-fc: misc small improvents Daniel Wagner
@ 2025-10-28 15:26 ` Daniel Wagner
  2025-10-29  8:51   ` Christoph Hellwig
  2025-11-04 10:51   ` Hannes Reinecke
  2025-10-28 15:26 ` [PATCH 2/5] nvme-fc: check all request and response have been processed Daniel Wagner
                   ` (3 subsequent siblings)
  4 siblings, 2 replies; 18+ messages in thread
From: Daniel Wagner @ 2025-10-28 15:26 UTC (permalink / raw)
  To: Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel, Daniel Wagner

nvme_fc_ctrl_put can acquire the rport lock when freeing the
ctrl object:

nvme_fc_ctrl_put
  nvme_fc_ctrl_free
    spin_lock_irqsave(rport->lock)

Thus we can't hold the rport lock when calling nvme_fc_ctrl_put.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 drivers/nvme/host/fc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 03987f497a5b..2c0ea843ae57 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -1488,7 +1488,9 @@ nvme_fc_match_disconn_ls(struct nvme_fc_rport *rport,
 		if (ret)
 			/* leave the ctrl get reference */
 			break;
+		spin_unlock_irqrestore(&rport->lock, flags);
 		nvme_fc_ctrl_put(ctrl);
+		spin_lock_irqsave(&rport->lock, flags);
 	}
 
 	spin_unlock_irqrestore(&rport->lock, flags);

-- 
2.51.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH 2/5] nvme-fc: check all request and response have been processed
  2025-10-28 15:26 [PATCH 0/5] nvme-fc: misc small improvents Daniel Wagner
  2025-10-28 15:26 ` [PATCH 1/5] nvme-fc: don't hold rport lock when putting ctrl Daniel Wagner
@ 2025-10-28 15:26 ` Daniel Wagner
  2025-10-29  8:51   ` Christoph Hellwig
  2025-11-04 10:51   ` Hannes Reinecke
  2025-10-28 15:26 ` [PATCH 3/5] nvmet-fcloop: " Daniel Wagner
                   ` (2 subsequent siblings)
  4 siblings, 2 replies; 18+ messages in thread
From: Daniel Wagner @ 2025-10-28 15:26 UTC (permalink / raw)
  To: Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel, Daniel Wagner

When the rport is removed there shouldn't be any in flight request or
responses.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 drivers/nvme/host/fc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 2c0ea843ae57..dcb7fc2ca0b7 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -520,6 +520,8 @@ nvme_fc_free_rport(struct kref *ref)
 
 	WARN_ON(rport->remoteport.port_state != FC_OBJSTATE_DELETED);
 	WARN_ON(!list_empty(&rport->ctrl_list));
+	WARN_ON(!list_empty(&rport->ls_req_list));
+	WARN_ON(!list_empty(&rport->ls_rcv_list));
 
 	/* remove from lport list */
 	spin_lock_irqsave(&nvme_fc_lock, flags);

-- 
2.51.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH 3/5] nvmet-fcloop: check all request and response have been processed
  2025-10-28 15:26 [PATCH 0/5] nvme-fc: misc small improvents Daniel Wagner
  2025-10-28 15:26 ` [PATCH 1/5] nvme-fc: don't hold rport lock when putting ctrl Daniel Wagner
  2025-10-28 15:26 ` [PATCH 2/5] nvme-fc: check all request and response have been processed Daniel Wagner
@ 2025-10-28 15:26 ` Daniel Wagner
  2025-10-29  8:52   ` Christoph Hellwig
  2025-11-04 10:52   ` Hannes Reinecke
  2025-10-28 15:26 ` [PATCH 4/5] nvmet-fcloop: remove unused lsdir member Daniel Wagner
  2025-10-28 15:26 ` [PATCH 5/5] nvmet-fc: use pr_* print macros instead of dev_* Daniel Wagner
  4 siblings, 2 replies; 18+ messages in thread
From: Daniel Wagner @ 2025-10-28 15:26 UTC (permalink / raw)
  To: Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel, Daniel Wagner

When the remoteport or the targetport are removed check that there are
no inflight requests or responses.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 drivers/nvme/target/fcloop.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
index 5dffcc5becae..4e429a1ea2bd 100644
--- a/drivers/nvme/target/fcloop.c
+++ b/drivers/nvme/target/fcloop.c
@@ -1111,8 +1111,10 @@ fcloop_remoteport_delete(struct nvme_fc_remote_port *remoteport)
 	rport->nport->rport = NULL;
 	spin_unlock_irqrestore(&fcloop_lock, flags);
 
-	if (put_port)
+	if (put_port) {
+		WARN_ON(!list_empty(&rport->ls_list));
 		fcloop_nport_put(rport->nport);
+	}
 }
 
 static void
@@ -1130,8 +1132,10 @@ fcloop_targetport_delete(struct nvmet_fc_target_port *targetport)
 	tport->nport->tport = NULL;
 	spin_unlock_irqrestore(&fcloop_lock, flags);
 
-	if (put_port)
+	if (put_port) {
+		WARN_ON(!list_empty(&tport->ls_list));
 		fcloop_nport_put(tport->nport);
+	}
 }
 
 #define	FCLOOP_HW_QUEUES		4

-- 
2.51.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH 4/5] nvmet-fcloop: remove unused lsdir member.
  2025-10-28 15:26 [PATCH 0/5] nvme-fc: misc small improvents Daniel Wagner
                   ` (2 preceding siblings ...)
  2025-10-28 15:26 ` [PATCH 3/5] nvmet-fcloop: " Daniel Wagner
@ 2025-10-28 15:26 ` Daniel Wagner
  2025-10-29  8:52   ` Christoph Hellwig
  2025-11-04 10:52   ` Hannes Reinecke
  2025-10-28 15:26 ` [PATCH 5/5] nvmet-fc: use pr_* print macros instead of dev_* Daniel Wagner
  4 siblings, 2 replies; 18+ messages in thread
From: Daniel Wagner @ 2025-10-28 15:26 UTC (permalink / raw)
  To: Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel, Daniel Wagner

Nothing is using lsdir member in struct fcloop_lsreq.

Signed-off-by: Daniel Wagner <wagi@kernel.org>
---
 drivers/nvme/target/fcloop.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
index 4e429a1ea2bd..c30e9a3e014f 100644
--- a/drivers/nvme/target/fcloop.c
+++ b/drivers/nvme/target/fcloop.c
@@ -254,7 +254,6 @@ struct fcloop_nport {
 struct fcloop_lsreq {
 	struct nvmefc_ls_req		*lsreq;
 	struct nvmefc_ls_rsp		ls_rsp;
-	int				lsdir;	/* H2T or T2H */
 	int				status;
 	struct list_head		ls_list; /* fcloop_rport->ls_list */
 };

-- 
2.51.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [PATCH 5/5] nvmet-fc: use pr_* print macros instead of dev_*
  2025-10-28 15:26 [PATCH 0/5] nvme-fc: misc small improvents Daniel Wagner
                   ` (3 preceding siblings ...)
  2025-10-28 15:26 ` [PATCH 4/5] nvmet-fcloop: remove unused lsdir member Daniel Wagner
@ 2025-10-28 15:26 ` Daniel Wagner
  2025-10-28 15:47   ` Daniel Wagner
                     ` (2 more replies)
  4 siblings, 3 replies; 18+ messages in thread
From: Daniel Wagner @ 2025-10-28 15:26 UTC (permalink / raw)
  To: Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel, Daniel Wagner

Many of the nvmet-fc log messages cannot print the device used, because
it's not there yet:

  (NULL device *): {0:0} Association deleted

Use the pr_* macros consistently throughout the module and match the
output of the nvme-fc module.

Using port:association ids are more useful when debugging what's going
on, because these match now with the log entries from nvme-fc.
---
 drivers/nvme/target/fc.c | 48 +++++++++++++++++++++---------------------------
 1 file changed, 21 insertions(+), 27 deletions(-)

diff --git a/drivers/nvme/target/fc.c b/drivers/nvme/target/fc.c
index 7d84527d5a43..0d9784004c9b 100644
--- a/drivers/nvme/target/fc.c
+++ b/drivers/nvme/target/fc.c
@@ -490,8 +490,7 @@ nvmet_fc_xmt_disconnect_assoc(struct nvmet_fc_tgt_assoc *assoc)
 			sizeof(*discon_rqst) + sizeof(*discon_acc) +
 			tgtport->ops->lsrqst_priv_sz), GFP_KERNEL);
 	if (!lsop) {
-		dev_info(tgtport->dev,
-			"{%d:%d} send Disconnect Association failed: ENOMEM\n",
+		pr_info("{%d:%d}: send Disconnect Association failed: ENOMEM\n",
 			tgtport->fc_target_port.port_num, assoc->a_id);
 		return;
 	}
@@ -513,8 +512,7 @@ nvmet_fc_xmt_disconnect_assoc(struct nvmet_fc_tgt_assoc *assoc)
 	ret = nvmet_fc_send_ls_req_async(tgtport, lsop,
 				nvmet_fc_disconnect_assoc_done);
 	if (ret) {
-		dev_info(tgtport->dev,
-			"{%d:%d} XMT Disconnect Association failed: %d\n",
+		pr_info("{%d:%d}: XMT Disconnect Association failed: %d\n",
 			tgtport->fc_target_port.port_num, assoc->a_id, ret);
 		kfree(lsop);
 	}
@@ -1187,8 +1185,7 @@ nvmet_fc_target_assoc_free(struct kref *ref)
 	if (oldls)
 		nvmet_fc_xmt_ls_rsp(tgtport, oldls);
 	ida_free(&tgtport->assoc_cnt, assoc->a_id);
-	dev_info(tgtport->dev,
-		"{%d:%d} Association freed\n",
+	pr_info("{%d:%d}: Association freed\n",
 		tgtport->fc_target_port.port_num, assoc->a_id);
 	kfree(assoc);
 }
@@ -1224,8 +1221,7 @@ nvmet_fc_delete_target_assoc(struct nvmet_fc_tgt_assoc *assoc)
 			flush_workqueue(assoc->queues[i]->work_q);
 	}
 
-	dev_info(tgtport->dev,
-		"{%d:%d} Association deleted\n",
+	pr_info("{%d:%d}: Association deleted\n",
 		tgtport->fc_target_port.port_num, assoc->a_id);
 
 	nvmet_fc_tgtport_put(tgtport);
@@ -1716,9 +1712,9 @@ nvmet_fc_ls_create_association(struct nvmet_fc_tgtport *tgtport,
 	}
 
 	if (ret) {
-		dev_err(tgtport->dev,
-			"Create Association LS failed: %s\n",
-			validation_errors[ret]);
+		pr_err("{%d}: Create Association LS failed: %s\n",
+		       tgtport->fc_target_port.port_num,
+		       validation_errors[ret]);
 		iod->lsrsp->rsplen = nvme_fc_format_rjt(acc,
 				sizeof(*acc), rqst->w0.ls_cmd,
 				FCNVME_RJT_RC_LOGIC,
@@ -1730,8 +1726,7 @@ nvmet_fc_ls_create_association(struct nvmet_fc_tgtport *tgtport,
 	atomic_set(&queue->connected, 1);
 	queue->sqhd = 0;	/* best place to init value */
 
-	dev_info(tgtport->dev,
-		"{%d:%d} Association created\n",
+	pr_info("{%d:%d}: Association created\n",
 		tgtport->fc_target_port.port_num, iod->assoc->a_id);
 
 	/* format a response */
@@ -1809,9 +1804,9 @@ nvmet_fc_ls_create_connection(struct nvmet_fc_tgtport *tgtport,
 	}
 
 	if (ret) {
-		dev_err(tgtport->dev,
-			"Create Connection LS failed: %s\n",
-			validation_errors[ret]);
+		pr_err("{%d}: Create Connection LS failed: %s\n",
+		       tgtport->fc_target_port.port_num,
+		       validation_errors[ret]);
 		iod->lsrsp->rsplen = nvme_fc_format_rjt(acc,
 				sizeof(*acc), rqst->w0.ls_cmd,
 				(ret == VERR_NO_ASSOC) ?
@@ -1871,9 +1866,9 @@ nvmet_fc_ls_disconnect(struct nvmet_fc_tgtport *tgtport,
 	}
 
 	if (ret || !assoc) {
-		dev_err(tgtport->dev,
-			"Disconnect LS failed: %s\n",
-			validation_errors[ret]);
+		pr_err("{%d}: Disconnect LS failed: %s\n",
+		       tgtport->fc_target_port.port_num,
+		       validation_errors[ret]);
 		iod->lsrsp->rsplen = nvme_fc_format_rjt(acc,
 				sizeof(*acc), rqst->w0.ls_cmd,
 				(ret == VERR_NO_ASSOC) ?
@@ -1907,8 +1902,7 @@ nvmet_fc_ls_disconnect(struct nvmet_fc_tgtport *tgtport,
 	spin_unlock_irqrestore(&tgtport->lock, flags);
 
 	if (oldls) {
-		dev_info(tgtport->dev,
-			"{%d:%d} Multiple Disconnect Association LS's "
+		pr_info("{%d:%d}: Multiple Disconnect Association LS's "
 			"received\n",
 			tgtport->fc_target_port.port_num, assoc->a_id);
 		/* overwrite good response with bogus failure */
@@ -2051,8 +2045,8 @@ nvmet_fc_rcv_ls_req(struct nvmet_fc_target_port *target_port,
 	struct fcnvme_ls_rqst_w0 *w0 = (struct fcnvme_ls_rqst_w0 *)lsreqbuf;
 
 	if (lsreqbuf_len > sizeof(union nvmefc_ls_requests)) {
-		dev_info(tgtport->dev,
-			"RCV %s LS failed: payload too large (%d)\n",
+		pr_info("{%d}: RCV %s LS failed: payload too large (%d)\n",
+			tgtport->fc_target_port.port_num,
 			(w0->ls_cmd <= NVME_FC_LAST_LS_CMD_VALUE) ?
 				nvmefc_ls_names[w0->ls_cmd] : "",
 			lsreqbuf_len);
@@ -2060,8 +2054,8 @@ nvmet_fc_rcv_ls_req(struct nvmet_fc_target_port *target_port,
 	}
 
 	if (!nvmet_fc_tgtport_get(tgtport)) {
-		dev_info(tgtport->dev,
-			"RCV %s LS failed: target deleting\n",
+		pr_info("{%d}: RCV %s LS failed: target deleting\n",
+			tgtport->fc_target_port.port_num,
 			(w0->ls_cmd <= NVME_FC_LAST_LS_CMD_VALUE) ?
 				nvmefc_ls_names[w0->ls_cmd] : "");
 		return -ESHUTDOWN;
@@ -2069,8 +2063,8 @@ nvmet_fc_rcv_ls_req(struct nvmet_fc_target_port *target_port,
 
 	iod = nvmet_fc_alloc_ls_iod(tgtport);
 	if (!iod) {
-		dev_info(tgtport->dev,
-			"RCV %s LS failed: context allocation failed\n",
+		pr_info("{%d}: RCV %s LS failed: context allocation failed\n",
+			tgtport->fc_target_port.port_num,
 			(w0->ls_cmd <= NVME_FC_LAST_LS_CMD_VALUE) ?
 				nvmefc_ls_names[w0->ls_cmd] : "");
 		nvmet_fc_tgtport_put(tgtport);

-- 
2.51.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* Re: [PATCH 5/5] nvmet-fc: use pr_* print macros instead of dev_*
  2025-10-28 15:26 ` [PATCH 5/5] nvmet-fc: use pr_* print macros instead of dev_* Daniel Wagner
@ 2025-10-28 15:47   ` Daniel Wagner
  2025-10-29  8:52   ` Christoph Hellwig
  2025-11-04 10:54   ` Hannes Reinecke
  2 siblings, 0 replies; 18+ messages in thread
From: Daniel Wagner @ 2025-10-28 15:47 UTC (permalink / raw)
  To: Daniel Wagner
  Cc: Justin Tee, Christoph Hellwig, Keith Busch, James Smart,
	Jens Axboe, linux-nvme, linux-kernel

On Tue, Oct 28, 2025 at 04:26:24PM +0100, Daniel Wagner wrote:
> Many of the nvmet-fc log messages cannot print the device used, because
> it's not there yet:
> 
>   (NULL device *): {0:0} Association deleted
> 
> Use the pr_* macros consistently throughout the module and match the
> output of the nvme-fc module.
> 
> Using port:association ids are more useful when debugging what's going
> on, because these match now with the log entries from nvme-fc.

Signed-off-by: Daniel Wagner <wagi@kernel.org>


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 1/5] nvme-fc: don't hold rport lock when putting ctrl
  2025-10-28 15:26 ` [PATCH 1/5] nvme-fc: don't hold rport lock when putting ctrl Daniel Wagner
@ 2025-10-29  8:51   ` Christoph Hellwig
  2025-11-04 10:51   ` Hannes Reinecke
  1 sibling, 0 replies; 18+ messages in thread
From: Christoph Hellwig @ 2025-10-29  8:51 UTC (permalink / raw)
  To: Daniel Wagner
  Cc: Justin Tee, Christoph Hellwig, Keith Busch, James Smart,
	Jens Axboe, linux-nvme, linux-kernel

On Tue, Oct 28, 2025 at 04:26:20PM +0100, Daniel Wagner wrote:
> nvme_fc_ctrl_put can acquire the rport lock when freeing the
> ctrl object:
> 
> nvme_fc_ctrl_put
>   nvme_fc_ctrl_free
>     spin_lock_irqsave(rport->lock)
> 
> Thus we can't hold the rport lock when calling nvme_fc_ctrl_put.
> 
> Signed-off-by: Daniel Wagner <wagi@kernel.org>

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 2/5] nvme-fc: check all request and response have been processed
  2025-10-28 15:26 ` [PATCH 2/5] nvme-fc: check all request and response have been processed Daniel Wagner
@ 2025-10-29  8:51   ` Christoph Hellwig
  2025-11-04 10:51   ` Hannes Reinecke
  1 sibling, 0 replies; 18+ messages in thread
From: Christoph Hellwig @ 2025-10-29  8:51 UTC (permalink / raw)
  To: Daniel Wagner
  Cc: Justin Tee, Christoph Hellwig, Keith Busch, James Smart,
	Jens Axboe, linux-nvme, linux-kernel

On Tue, Oct 28, 2025 at 04:26:21PM +0100, Daniel Wagner wrote:
> When the rport is removed there shouldn't be any in flight request or
> responses.
> 
> Signed-off-by: Daniel Wagner <wagi@kernel.org>

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 3/5] nvmet-fcloop: check all request and response have been processed
  2025-10-28 15:26 ` [PATCH 3/5] nvmet-fcloop: " Daniel Wagner
@ 2025-10-29  8:52   ` Christoph Hellwig
  2025-11-04 10:52   ` Hannes Reinecke
  1 sibling, 0 replies; 18+ messages in thread
From: Christoph Hellwig @ 2025-10-29  8:52 UTC (permalink / raw)
  To: Daniel Wagner
  Cc: Justin Tee, Christoph Hellwig, Keith Busch, James Smart,
	Jens Axboe, linux-nvme, linux-kernel

On Tue, Oct 28, 2025 at 04:26:22PM +0100, Daniel Wagner wrote:
> When the remoteport or the targetport are removed check that there are
> no inflight requests or responses.
> 
> Signed-off-by: Daniel Wagner <wagi@kernel.org>

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 4/5] nvmet-fcloop: remove unused lsdir member.
  2025-10-28 15:26 ` [PATCH 4/5] nvmet-fcloop: remove unused lsdir member Daniel Wagner
@ 2025-10-29  8:52   ` Christoph Hellwig
  2025-11-04 10:52   ` Hannes Reinecke
  1 sibling, 0 replies; 18+ messages in thread
From: Christoph Hellwig @ 2025-10-29  8:52 UTC (permalink / raw)
  To: Daniel Wagner
  Cc: Justin Tee, Christoph Hellwig, Keith Busch, James Smart,
	Jens Axboe, linux-nvme, linux-kernel

On Tue, Oct 28, 2025 at 04:26:23PM +0100, Daniel Wagner wrote:
> Nothing is using lsdir member in struct fcloop_lsreq.
> 
> Signed-off-by: Daniel Wagner <wagi@kernel.org>

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 5/5] nvmet-fc: use pr_* print macros instead of dev_*
  2025-10-28 15:26 ` [PATCH 5/5] nvmet-fc: use pr_* print macros instead of dev_* Daniel Wagner
  2025-10-28 15:47   ` Daniel Wagner
@ 2025-10-29  8:52   ` Christoph Hellwig
  2025-11-04 10:54   ` Hannes Reinecke
  2 siblings, 0 replies; 18+ messages in thread
From: Christoph Hellwig @ 2025-10-29  8:52 UTC (permalink / raw)
  To: Daniel Wagner
  Cc: Justin Tee, Christoph Hellwig, Keith Busch, James Smart,
	Jens Axboe, linux-nvme, linux-kernel

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 1/5] nvme-fc: don't hold rport lock when putting ctrl
  2025-10-28 15:26 ` [PATCH 1/5] nvme-fc: don't hold rport lock when putting ctrl Daniel Wagner
  2025-10-29  8:51   ` Christoph Hellwig
@ 2025-11-04 10:51   ` Hannes Reinecke
  2025-11-04 12:32     ` Daniel Wagner
  1 sibling, 1 reply; 18+ messages in thread
From: Hannes Reinecke @ 2025-11-04 10:51 UTC (permalink / raw)
  To: Daniel Wagner, Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel

On 10/28/25 16:26, Daniel Wagner wrote:
> nvme_fc_ctrl_put can acquire the rport lock when freeing the
> ctrl object:
> 
> nvme_fc_ctrl_put
>    nvme_fc_ctrl_free
>      spin_lock_irqsave(rport->lock)
> 
> Thus we can't hold the rport lock when calling nvme_fc_ctrl_put.
> 
> Signed-off-by: Daniel Wagner <wagi@kernel.org>
> ---
>   drivers/nvme/host/fc.c | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index 03987f497a5b..2c0ea843ae57 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -1488,7 +1488,9 @@ nvme_fc_match_disconn_ls(struct nvme_fc_rport *rport,
>   		if (ret)
>   			/* leave the ctrl get reference */
>   			break;
> +		spin_unlock_irqrestore(&rport->lock, flags);
>   		nvme_fc_ctrl_put(ctrl);
> +		spin_lock_irqsave(&rport->lock, flags);
>   	}
>   
>   	spin_unlock_irqrestore(&rport->lock, flags);
> 
In theory, yes.
In practice we're getting a reference to the controller at the start
of the loop, so nvme_fc_ctrl_put() should just decrement the refcount.
But _if_ someone manages to sneak in an drop the reference while we are
within that loop then the entire logic falls apart as the controller
will be deleted and the next loop iteration will trip over an
uninitialized pointer.
So you would need to modify the loop to use
loop_for_each_entry_safe() to avoid this, too.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
hare@suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 2/5] nvme-fc: check all request and response have been processed
  2025-10-28 15:26 ` [PATCH 2/5] nvme-fc: check all request and response have been processed Daniel Wagner
  2025-10-29  8:51   ` Christoph Hellwig
@ 2025-11-04 10:51   ` Hannes Reinecke
  1 sibling, 0 replies; 18+ messages in thread
From: Hannes Reinecke @ 2025-11-04 10:51 UTC (permalink / raw)
  To: Daniel Wagner, Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel

On 10/28/25 16:26, Daniel Wagner wrote:
> When the rport is removed there shouldn't be any in flight request or
> responses.
> 
> Signed-off-by: Daniel Wagner <wagi@kernel.org>
> ---
>   drivers/nvme/host/fc.c | 2 ++
>   1 file changed, 2 insertions(+)
> 
Reviewed-by: Hannes Reinecke <hare@suse.de>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
hare@suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 3/5] nvmet-fcloop: check all request and response have been processed
  2025-10-28 15:26 ` [PATCH 3/5] nvmet-fcloop: " Daniel Wagner
  2025-10-29  8:52   ` Christoph Hellwig
@ 2025-11-04 10:52   ` Hannes Reinecke
  1 sibling, 0 replies; 18+ messages in thread
From: Hannes Reinecke @ 2025-11-04 10:52 UTC (permalink / raw)
  To: Daniel Wagner, Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel

On 10/28/25 16:26, Daniel Wagner wrote:
> When the remoteport or the targetport are removed check that there are
> no inflight requests or responses.
> 
> Signed-off-by: Daniel Wagner <wagi@kernel.org>
> ---
>   drivers/nvme/target/fcloop.c | 8 ++++++--
>   1 file changed, 6 insertions(+), 2 deletions(-)
> 
Reviewed-by: Hannes Reinecke <hare@suse.de>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
hare@suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 4/5] nvmet-fcloop: remove unused lsdir member.
  2025-10-28 15:26 ` [PATCH 4/5] nvmet-fcloop: remove unused lsdir member Daniel Wagner
  2025-10-29  8:52   ` Christoph Hellwig
@ 2025-11-04 10:52   ` Hannes Reinecke
  1 sibling, 0 replies; 18+ messages in thread
From: Hannes Reinecke @ 2025-11-04 10:52 UTC (permalink / raw)
  To: Daniel Wagner, Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel

On 10/28/25 16:26, Daniel Wagner wrote:
> Nothing is using lsdir member in struct fcloop_lsreq.
> 
> Signed-off-by: Daniel Wagner <wagi@kernel.org>
> ---
>   drivers/nvme/target/fcloop.c | 1 -
>   1 file changed, 1 deletion(-)
> 
> diff --git a/drivers/nvme/target/fcloop.c b/drivers/nvme/target/fcloop.c
> index 4e429a1ea2bd..c30e9a3e014f 100644
> --- a/drivers/nvme/target/fcloop.c
> +++ b/drivers/nvme/target/fcloop.c
> @@ -254,7 +254,6 @@ struct fcloop_nport {
>   struct fcloop_lsreq {
>   	struct nvmefc_ls_req		*lsreq;
>   	struct nvmefc_ls_rsp		ls_rsp;
> -	int				lsdir;	/* H2T or T2H */
>   	int				status;
>   	struct list_head		ls_list; /* fcloop_rport->ls_list */
>   };
> 
Reviewed-by: Hannes Reinecke <hare@suse.de>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
hare@suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 5/5] nvmet-fc: use pr_* print macros instead of dev_*
  2025-10-28 15:26 ` [PATCH 5/5] nvmet-fc: use pr_* print macros instead of dev_* Daniel Wagner
  2025-10-28 15:47   ` Daniel Wagner
  2025-10-29  8:52   ` Christoph Hellwig
@ 2025-11-04 10:54   ` Hannes Reinecke
  2 siblings, 0 replies; 18+ messages in thread
From: Hannes Reinecke @ 2025-11-04 10:54 UTC (permalink / raw)
  To: Daniel Wagner, Justin Tee, Christoph Hellwig, Keith Busch
  Cc: James Smart, Jens Axboe, linux-nvme, linux-kernel

On 10/28/25 16:26, Daniel Wagner wrote:
> Many of the nvmet-fc log messages cannot print the device used, because
> it's not there yet:
> 
>    (NULL device *): {0:0} Association deleted
> 
> Use the pr_* macros consistently throughout the module and match the
> output of the nvme-fc module.
> 
> Using port:association ids are more useful when debugging what's going
> on, because these match now with the log entries from nvme-fc.
> ---
>   drivers/nvme/target/fc.c | 48 +++++++++++++++++++++---------------------------
>   1 file changed, 21 insertions(+), 27 deletions(-)
> 
Reviewed-by: Hannes Reinecke <hare@suse.de>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
hare@suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH 1/5] nvme-fc: don't hold rport lock when putting ctrl
  2025-11-04 10:51   ` Hannes Reinecke
@ 2025-11-04 12:32     ` Daniel Wagner
  0 siblings, 0 replies; 18+ messages in thread
From: Daniel Wagner @ 2025-11-04 12:32 UTC (permalink / raw)
  To: Hannes Reinecke
  Cc: Daniel Wagner, Justin Tee, Christoph Hellwig, Keith Busch,
	James Smart, Jens Axboe, linux-nvme, linux-kernel

On Tue, Nov 04, 2025 at 11:51:09AM +0100, Hannes Reinecke wrote:
> On 10/28/25 16:26, Daniel Wagner wrote:
> > nvme_fc_ctrl_put can acquire the rport lock when freeing the
> > ctrl object:
> > 
> > nvme_fc_ctrl_put
> >    nvme_fc_ctrl_free
> >      spin_lock_irqsave(rport->lock)
> > 
> > Thus we can't hold the rport lock when calling nvme_fc_ctrl_put.
> > 
> > Signed-off-by: Daniel Wagner <wagi@kernel.org>
> > ---
> >   drivers/nvme/host/fc.c | 2 ++
> >   1 file changed, 2 insertions(+)
> > 
> > diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> > index 03987f497a5b..2c0ea843ae57 100644
> > --- a/drivers/nvme/host/fc.c
> > +++ b/drivers/nvme/host/fc.c
> > @@ -1488,7 +1488,9 @@ nvme_fc_match_disconn_ls(struct nvme_fc_rport *rport,
> >   		if (ret)
> >   			/* leave the ctrl get reference */
> >   			break;
> > +		spin_unlock_irqrestore(&rport->lock, flags);
> >   		nvme_fc_ctrl_put(ctrl);
> > +		spin_lock_irqsave(&rport->lock, flags);
> >   	}
> >   	spin_unlock_irqrestore(&rport->lock, flags);
> > 
> In theory, yes.

I hit this in practice ;)


^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2025-11-04 12:32 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-28 15:26 [PATCH 0/5] nvme-fc: misc small improvents Daniel Wagner
2025-10-28 15:26 ` [PATCH 1/5] nvme-fc: don't hold rport lock when putting ctrl Daniel Wagner
2025-10-29  8:51   ` Christoph Hellwig
2025-11-04 10:51   ` Hannes Reinecke
2025-11-04 12:32     ` Daniel Wagner
2025-10-28 15:26 ` [PATCH 2/5] nvme-fc: check all request and response have been processed Daniel Wagner
2025-10-29  8:51   ` Christoph Hellwig
2025-11-04 10:51   ` Hannes Reinecke
2025-10-28 15:26 ` [PATCH 3/5] nvmet-fcloop: " Daniel Wagner
2025-10-29  8:52   ` Christoph Hellwig
2025-11-04 10:52   ` Hannes Reinecke
2025-10-28 15:26 ` [PATCH 4/5] nvmet-fcloop: remove unused lsdir member Daniel Wagner
2025-10-29  8:52   ` Christoph Hellwig
2025-11-04 10:52   ` Hannes Reinecke
2025-10-28 15:26 ` [PATCH 5/5] nvmet-fc: use pr_* print macros instead of dev_* Daniel Wagner
2025-10-28 15:47   ` Daniel Wagner
2025-10-29  8:52   ` Christoph Hellwig
2025-11-04 10:54   ` Hannes Reinecke

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox