* [PATCH v4 01/15] nvmet: Rapid Path Failure Recovery set controller identify fields
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-30 10:37 ` Hannes Reinecke
2026-03-28 0:43 ` [PATCH v4 02/15] nvmet/debugfs: Export controller CIU and CIRN via debugfs Mohamed Khalfella
` (13 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
TP8028 Rapid Path Failure Recovery defined new fields in controller
identify response. The newly defined fields are:
- CIU (Controller Instance UNIQUIFIER): is an 8bit non-zero value that
is assigned a random value when controller is first created. The value
is expected to be incremented when RDY bit in CSTS register is asserted
- CIRN (Controller Instance Random Number): is 64bit random value that
gets generated when controller is created. CIRN is regenerated everytime
RDY bit is CSTS register is asserted.
- CCRL (Cross-Controller Reset Limit) is an 8bit value that defines the
maximum number of in-progress controller reset operations. CCRL is
hardcoded to 4 as recommended by TP8028.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
---
drivers/nvme/target/admin-cmd.c | 5 +++++
drivers/nvme/target/core.c | 9 +++++++++
drivers/nvme/target/nvmet.h | 2 ++
include/linux/nvme.h | 10 ++++++++--
4 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index ca5b08ce1211..ec09e30eca18 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -695,6 +695,11 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
id->cntlid = cpu_to_le16(ctrl->cntlid);
id->ver = cpu_to_le32(ctrl->subsys->ver);
+ if (!nvmet_is_disc_subsys(ctrl->subsys)) {
+ id->ciu = ctrl->ciu;
+ id->cirn = cpu_to_le64(ctrl->cirn);
+ id->ccrl = NVMF_CCR_LIMIT;
+ }
/* XXX: figure out what to do about RTD3R/RTD3 */
id->oaes = cpu_to_le32(NVMET_AEN_CFG_OPTIONAL);
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 9238e13bd480..e8b945a01f35 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -1396,6 +1396,10 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
return;
}
+ if (!nvmet_is_disc_subsys(ctrl->subsys)) {
+ ctrl->ciu = ((u8)(ctrl->ciu + 1)) ? : 1;
+ ctrl->cirn = get_random_u64();
+ }
ctrl->csts = NVME_CSTS_RDY;
/*
@@ -1661,6 +1665,11 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
}
ctrl->cntlid = ret;
+ if (!nvmet_is_disc_subsys(ctrl->subsys)) {
+ ctrl->ciu = get_random_u8() ? : 1;
+ ctrl->cirn = get_random_u64();
+ }
+
/*
* Discovery controllers may use some arbitrary high value
* in order to cleanup stale discovery sessions
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 319d6a5e9cf0..2181ac45ae7f 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -264,7 +264,9 @@ struct nvmet_ctrl {
uuid_t hostid;
u16 cntlid;
+ u8 ciu;
u32 kato;
+ u64 cirn;
struct nvmet_port *port;
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index 655d194f8e72..7746b6d30349 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -21,6 +21,8 @@
#define NVMF_TRADDR_SIZE 256
#define NVMF_TSAS_SIZE 256
+#define NVMF_CCR_LIMIT 4
+
#define NVME_DISC_SUBSYS_NAME "nqn.2014-08.org.nvmexpress.discovery"
#define NVME_NSID_ALL 0xffffffff
@@ -328,7 +330,10 @@ struct nvme_id_ctrl {
__le16 crdt1;
__le16 crdt2;
__le16 crdt3;
- __u8 rsvd134[122];
+ __u8 rsvd134[1];
+ __u8 ciu;
+ __le64 cirn;
+ __u8 rsvd144[112];
__le16 oacs;
__u8 acl;
__u8 aerl;
@@ -389,7 +394,8 @@ struct nvme_id_ctrl {
__u8 msdbd;
__u8 rsvd1804[2];
__u8 dctype;
- __u8 rsvd1807[241];
+ __u8 ccrl;
+ __u8 rsvd1808[240];
struct nvme_id_power_state psd[32];
__u8 vs[1024];
};
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v4 01/15] nvmet: Rapid Path Failure Recovery set controller identify fields
2026-03-28 0:43 ` [PATCH v4 01/15] nvmet: Rapid Path Failure Recovery set controller identify fields Mohamed Khalfella
@ 2026-03-30 10:37 ` Hannes Reinecke
0 siblings, 0 replies; 25+ messages in thread
From: Hannes Reinecke @ 2026-03-30 10:37 UTC (permalink / raw)
To: Mohamed Khalfella, Justin Tee, Naresh Gottumukkala, Paul Ely,
Chaitanya Kulkarni, Jens Axboe, Keith Busch, Sagi Grimberg,
James Smart
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel
On 3/28/26 01:43, Mohamed Khalfella wrote:
> TP8028 Rapid Path Failure Recovery defined new fields in controller
> identify response. The newly defined fields are:
>
> - CIU (Controller Instance UNIQUIFIER): is an 8bit non-zero value that
> is assigned a random value when controller is first created. The value
> is expected to be incremented when RDY bit in CSTS register is asserted
> - CIRN (Controller Instance Random Number): is 64bit random value that
> gets generated when controller is created. CIRN is regenerated everytime
> RDY bit is CSTS register is asserted.
> - CCRL (Cross-Controller Reset Limit) is an 8bit value that defines the
> maximum number of in-progress controller reset operations. CCRL is
> hardcoded to 4 as recommended by TP8028.
>
> Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
> ---
> drivers/nvme/target/admin-cmd.c | 5 +++++
> drivers/nvme/target/core.c | 9 +++++++++
> drivers/nvme/target/nvmet.h | 2 ++
> include/linux/nvme.h | 10 ++++++++--
> 4 files changed, 24 insertions(+), 2 deletions(-)
>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 02/15] nvmet/debugfs: Export controller CIU and CIRN via debugfs
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 01/15] nvmet: Rapid Path Failure Recovery set controller identify fields Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 03/15] nvmet: Implement CCR nvme command Mohamed Khalfella
` (12 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
Export ctrl->ciu and ctrl->cirn as debugfs files under controller
debugfs directory.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/target/debugfs.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/drivers/nvme/target/debugfs.c b/drivers/nvme/target/debugfs.c
index 5dcbd5aa86e1..1300adf6c1fb 100644
--- a/drivers/nvme/target/debugfs.c
+++ b/drivers/nvme/target/debugfs.c
@@ -152,6 +152,23 @@ static int nvmet_ctrl_tls_concat_show(struct seq_file *m, void *p)
}
NVMET_DEBUGFS_ATTR(nvmet_ctrl_tls_concat);
#endif
+static int nvmet_ctrl_instance_ciu_show(struct seq_file *m, void *p)
+{
+ struct nvmet_ctrl *ctrl = m->private;
+
+ seq_printf(m, "%02x\n", ctrl->ciu);
+ return 0;
+}
+NVMET_DEBUGFS_ATTR(nvmet_ctrl_instance_ciu);
+
+static int nvmet_ctrl_instance_cirn_show(struct seq_file *m, void *p)
+{
+ struct nvmet_ctrl *ctrl = m->private;
+
+ seq_printf(m, "%016llx\n", ctrl->cirn);
+ return 0;
+}
+NVMET_DEBUGFS_ATTR(nvmet_ctrl_instance_cirn);
int nvmet_debugfs_ctrl_setup(struct nvmet_ctrl *ctrl)
{
@@ -184,6 +201,10 @@ int nvmet_debugfs_ctrl_setup(struct nvmet_ctrl *ctrl)
debugfs_create_file("tls_key", S_IRUSR, ctrl->debugfs_dir, ctrl,
&nvmet_ctrl_tls_key_fops);
#endif
+ debugfs_create_file("ciu", S_IRUSR, ctrl->debugfs_dir, ctrl,
+ &nvmet_ctrl_instance_ciu_fops);
+ debugfs_create_file("cirn", S_IRUSR, ctrl->debugfs_dir, ctrl,
+ &nvmet_ctrl_instance_cirn_fops);
return 0;
}
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v4 03/15] nvmet: Implement CCR nvme command
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 01/15] nvmet: Rapid Path Failure Recovery set controller identify fields Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 02/15] nvmet/debugfs: Export controller CIU and CIRN via debugfs Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-30 10:45 ` Hannes Reinecke
2026-03-28 0:43 ` [PATCH v4 04/15] nvmet: Implement CCR logpage Mohamed Khalfella
` (11 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
Defined by TP8028 Rapid Path Failure Recovery, CCR (Cross-Controller
Reset) command is an nvme command issued to source controller by
initiator to reset impacted controller. Implement CCR command for linux
nvme target.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
---
drivers/nvme/target/admin-cmd.c | 74 ++++++++++++++++++++++++++++++++
drivers/nvme/target/core.c | 76 +++++++++++++++++++++++++++++++++
drivers/nvme/target/nvmet.h | 13 ++++++
include/linux/nvme.h | 23 ++++++++++
4 files changed, 186 insertions(+)
diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index ec09e30eca18..0a37c0eeebb5 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -376,7 +376,9 @@ static void nvmet_get_cmd_effects_admin(struct nvmet_ctrl *ctrl,
log->acs[nvme_admin_get_features] =
log->acs[nvme_admin_async_event] =
log->acs[nvme_admin_keep_alive] =
+ log->acs[nvme_admin_cross_ctrl_reset] =
cpu_to_le32(NVME_CMD_EFFECTS_CSUPP);
+
}
static void nvmet_get_cmd_effects_nvm(struct nvme_effects_log *log)
@@ -1613,6 +1615,75 @@ void nvmet_execute_keep_alive(struct nvmet_req *req)
nvmet_req_complete(req, status);
}
+void nvmet_execute_cross_ctrl_reset(struct nvmet_req *req)
+{
+ struct nvmet_ctrl *ictrl, *sctrl = req->sq->ctrl;
+ struct nvme_command *cmd = req->cmd;
+ struct nvmet_ccr *ccr, *new_ccr;
+ int ccr_active, ccr_total;
+ u16 cntlid, status = NVME_SC_SUCCESS;
+
+ cntlid = le16_to_cpu(cmd->ccr.icid);
+ if (sctrl->cntlid == cntlid) {
+ req->error_loc =
+ offsetof(struct nvme_cross_ctrl_reset_cmd, icid);
+ status = NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
+ goto out;
+ }
+
+ /* Find and get impacted controller */
+ ictrl = nvmet_ctrl_find_get_ccr(sctrl->subsys, sctrl->hostnqn,
+ cmd->ccr.ciu, cntlid,
+ le64_to_cpu(cmd->ccr.cirn));
+ if (!ictrl) {
+ /* Immediate Reset Successful */
+ nvmet_set_result(req, 1);
+ status = NVME_SC_SUCCESS;
+ goto out;
+ }
+
+ ccr_total = ccr_active = 0;
+ mutex_lock(&sctrl->lock);
+ list_for_each_entry(ccr, &sctrl->ccr_list, entry) {
+ if (ccr->ctrl == ictrl) {
+ status = NVME_SC_CCR_IN_PROGRESS | NVME_STATUS_DNR;
+ goto out_unlock;
+ }
+
+ ccr_total++;
+ if (ccr->ctrl)
+ ccr_active++;
+ }
+
+ if (ccr_active >= NVMF_CCR_LIMIT) {
+ status = NVME_SC_CCR_LIMIT_EXCEEDED;
+ goto out_unlock;
+ }
+ if (ccr_total >= NVMF_CCR_PER_PAGE) {
+ status = NVME_SC_CCR_LOGPAGE_FULL;
+ goto out_unlock;
+ }
+
+ new_ccr = kmalloc_obj(*new_ccr, GFP_KERNEL);
+ if (!new_ccr) {
+ status = NVME_SC_INTERNAL;
+ goto out_unlock;
+ }
+
+ new_ccr->ciu = cmd->ccr.ciu;
+ new_ccr->icid = cntlid;
+ new_ccr->ctrl = ictrl;
+ list_add_tail(&new_ccr->entry, &sctrl->ccr_list);
+
+out_unlock:
+ mutex_unlock(&sctrl->lock);
+ if (status == NVME_SC_SUCCESS)
+ nvmet_ctrl_fatal_error(ictrl);
+ nvmet_ctrl_put(ictrl);
+out:
+ nvmet_req_complete(req, status);
+}
+
u32 nvmet_admin_cmd_data_len(struct nvmet_req *req)
{
struct nvme_command *cmd = req->cmd;
@@ -1690,6 +1761,9 @@ u16 nvmet_parse_admin_cmd(struct nvmet_req *req)
case nvme_admin_keep_alive:
req->execute = nvmet_execute_keep_alive;
return 0;
+ case nvme_admin_cross_ctrl_reset:
+ req->execute = nvmet_execute_cross_ctrl_reset;
+ return 0;
default:
return nvmet_report_invalid_opcode(req);
}
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index e8b945a01f35..2e0c31d82bad 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -117,6 +117,20 @@ u16 nvmet_zero_sgl(struct nvmet_req *req, off_t off, size_t len)
return 0;
}
+void nvmet_ctrl_cleanup_ccrs(struct nvmet_ctrl *ctrl, bool all)
+{
+ struct nvmet_ccr *ccr, *tmp;
+
+ lockdep_assert_held(&ctrl->lock);
+
+ list_for_each_entry_safe(ccr, tmp, &ctrl->ccr_list, entry) {
+ if (all || ccr->ctrl == NULL) {
+ list_del(&ccr->entry);
+ kfree(ccr);
+ }
+ }
+}
+
static u32 nvmet_max_nsid(struct nvmet_subsys *subsys)
{
struct nvmet_ns *cur;
@@ -1399,6 +1413,7 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
if (!nvmet_is_disc_subsys(ctrl->subsys)) {
ctrl->ciu = ((u8)(ctrl->ciu + 1)) ? : 1;
ctrl->cirn = get_random_u64();
+ nvmet_ctrl_cleanup_ccrs(ctrl, false);
}
ctrl->csts = NVME_CSTS_RDY;
@@ -1504,6 +1519,35 @@ struct nvmet_ctrl *nvmet_ctrl_find_get(const char *subsysnqn,
return ctrl;
}
+struct nvmet_ctrl *nvmet_ctrl_find_get_ccr(struct nvmet_subsys *subsys,
+ const char *hostnqn, u8 ciu,
+ u16 cntlid, u64 cirn)
+{
+ struct nvmet_ctrl *ctrl, *ictrl = NULL;
+ bool found = false;
+
+ mutex_lock(&subsys->lock);
+ list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
+ if (ctrl->cntlid != cntlid)
+ continue;
+
+ /* Avoid racing with a controller that is becoming ready */
+ mutex_lock(&ctrl->lock);
+ if (ctrl->ciu == ciu && ctrl->cirn == cirn)
+ found = true;
+ mutex_unlock(&ctrl->lock);
+
+ if (found) {
+ if (kref_get_unless_zero(&ctrl->ref))
+ ictrl = ctrl;
+ break;
+ }
+ };
+ mutex_unlock(&subsys->lock);
+
+ return ictrl;
+}
+
u16 nvmet_check_ctrl_status(struct nvmet_req *req)
{
if (unlikely(!(req->sq->ctrl->cc & NVME_CC_ENABLE))) {
@@ -1629,6 +1673,7 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
subsys->clear_ids = 1;
#endif
+ INIT_LIST_HEAD(&ctrl->ccr_list);
INIT_WORK(&ctrl->async_event_work, nvmet_async_event_work);
INIT_LIST_HEAD(&ctrl->async_events);
INIT_RADIX_TREE(&ctrl->p2p_ns_map, GFP_KERNEL);
@@ -1739,12 +1784,43 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
}
EXPORT_SYMBOL_GPL(nvmet_alloc_ctrl);
+static void nvmet_ctrl_complete_pending_ccr(struct nvmet_ctrl *ctrl)
+{
+ struct nvmet_subsys *subsys = ctrl->subsys;
+ struct nvmet_ctrl *sctrl;
+ struct nvmet_ccr *ccr;
+
+ lockdep_assert_held(&subsys->lock);
+
+ /* Cleanup all CCRs issued by ctrl as source controller */
+ mutex_lock(&ctrl->lock);
+ nvmet_ctrl_cleanup_ccrs(ctrl, true);
+ mutex_unlock(&ctrl->lock);
+
+ /*
+ * Find all CCRs targeting ctrl as impacted controller and
+ * set ccr->ctrl to NULL. This tells the source controller
+ * that CCR completed successfully.
+ */
+ list_for_each_entry(sctrl, &subsys->ctrls, subsys_entry) {
+ mutex_lock(&sctrl->lock);
+ list_for_each_entry(ccr, &sctrl->ccr_list, entry) {
+ if (ccr->ctrl == ctrl) {
+ ccr->ctrl = NULL;
+ break;
+ }
+ }
+ mutex_unlock(&sctrl->lock);
+ }
+}
+
static void nvmet_ctrl_free(struct kref *ref)
{
struct nvmet_ctrl *ctrl = container_of(ref, struct nvmet_ctrl, ref);
struct nvmet_subsys *subsys = ctrl->subsys;
mutex_lock(&subsys->lock);
+ nvmet_ctrl_complete_pending_ccr(ctrl);
nvmet_ctrl_destroy_pr(ctrl);
nvmet_release_p2p_ns_map(ctrl);
list_del(&ctrl->subsys_entry);
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index 2181ac45ae7f..b9eb044ded19 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -268,6 +268,7 @@ struct nvmet_ctrl {
u32 kato;
u64 cirn;
+ struct list_head ccr_list;
struct nvmet_port *port;
u32 aen_enabled;
@@ -314,6 +315,13 @@ struct nvmet_ctrl {
struct nvmet_pr_log_mgr pr_log_mgr;
};
+struct nvmet_ccr {
+ struct nvmet_ctrl *ctrl;
+ struct list_head entry;
+ u16 icid;
+ u8 ciu;
+};
+
struct nvmet_subsys {
enum nvme_subsys_type type;
@@ -577,6 +585,7 @@ void nvmet_req_free_sgls(struct nvmet_req *req);
void nvmet_execute_set_features(struct nvmet_req *req);
void nvmet_execute_get_features(struct nvmet_req *req);
void nvmet_execute_keep_alive(struct nvmet_req *req);
+void nvmet_execute_cross_ctrl_reset(struct nvmet_req *req);
u16 nvmet_check_cqid(struct nvmet_ctrl *ctrl, u16 cqid, bool create);
u16 nvmet_check_io_cqid(struct nvmet_ctrl *ctrl, u16 cqid, bool create);
@@ -619,6 +628,10 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args);
struct nvmet_ctrl *nvmet_ctrl_find_get(const char *subsysnqn,
const char *hostnqn, u16 cntlid,
struct nvmet_req *req);
+struct nvmet_ctrl *nvmet_ctrl_find_get_ccr(struct nvmet_subsys *subsys,
+ const char *hostnqn, u8 ciu,
+ u16 cntlid, u64 cirn);
+void nvmet_ctrl_cleanup_ccrs(struct nvmet_ctrl *ctrl, bool all);
void nvmet_ctrl_put(struct nvmet_ctrl *ctrl);
u16 nvmet_check_ctrl_status(struct nvmet_req *req);
ssize_t nvmet_ctrl_host_traddr(struct nvmet_ctrl *ctrl,
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index 7746b6d30349..bd3b3f2a5377 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -22,6 +22,7 @@
#define NVMF_TSAS_SIZE 256
#define NVMF_CCR_LIMIT 4
+#define NVMF_CCR_PER_PAGE 511
#define NVME_DISC_SUBSYS_NAME "nqn.2014-08.org.nvmexpress.discovery"
@@ -1222,6 +1223,22 @@ struct nvme_zone_mgmt_recv_cmd {
__le32 cdw14[2];
};
+struct nvme_cross_ctrl_reset_cmd {
+ __u8 opcode;
+ __u8 flags;
+ __u16 command_id;
+ __le32 nsid;
+ __le64 rsvd2[2];
+ union nvme_data_ptr dptr;
+ __le16 icid;
+ __u8 ciu;
+ __u8 rsvd10;
+ __le32 cdw11;
+ __le64 cirn;
+ __le32 cdw14;
+ __le32 cdw15;
+};
+
struct nvme_io_mgmt_recv_cmd {
__u8 opcode;
__u8 flags;
@@ -1320,6 +1337,7 @@ enum nvme_admin_opcode {
nvme_admin_virtual_mgmt = 0x1c,
nvme_admin_nvme_mi_send = 0x1d,
nvme_admin_nvme_mi_recv = 0x1e,
+ nvme_admin_cross_ctrl_reset = 0x38,
nvme_admin_dbbuf = 0x7C,
nvme_admin_format_nvm = 0x80,
nvme_admin_security_send = 0x81,
@@ -1353,6 +1371,7 @@ enum nvme_admin_opcode {
nvme_admin_opcode_name(nvme_admin_virtual_mgmt), \
nvme_admin_opcode_name(nvme_admin_nvme_mi_send), \
nvme_admin_opcode_name(nvme_admin_nvme_mi_recv), \
+ nvme_admin_opcode_name(nvme_admin_cross_ctrl_reset), \
nvme_admin_opcode_name(nvme_admin_dbbuf), \
nvme_admin_opcode_name(nvme_admin_format_nvm), \
nvme_admin_opcode_name(nvme_admin_security_send), \
@@ -2006,6 +2025,7 @@ struct nvme_command {
struct nvme_dbbuf dbbuf;
struct nvme_directive_cmd directive;
struct nvme_io_mgmt_recv_cmd imr;
+ struct nvme_cross_ctrl_reset_cmd ccr;
};
};
@@ -2170,6 +2190,9 @@ enum {
NVME_SC_PMR_SAN_PROHIBITED = 0x123,
NVME_SC_ANA_GROUP_ID_INVALID = 0x124,
NVME_SC_ANA_ATTACH_FAILED = 0x125,
+ NVME_SC_CCR_IN_PROGRESS = 0x13f,
+ NVME_SC_CCR_LOGPAGE_FULL = 0x140,
+ NVME_SC_CCR_LIMIT_EXCEEDED = 0x141,
/*
* I/O Command Set Specific - NVM commands:
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v4 03/15] nvmet: Implement CCR nvme command
2026-03-28 0:43 ` [PATCH v4 03/15] nvmet: Implement CCR nvme command Mohamed Khalfella
@ 2026-03-30 10:45 ` Hannes Reinecke
2026-03-31 16:38 ` Mohamed Khalfella
0 siblings, 1 reply; 25+ messages in thread
From: Hannes Reinecke @ 2026-03-30 10:45 UTC (permalink / raw)
To: Mohamed Khalfella, Justin Tee, Naresh Gottumukkala, Paul Ely,
Chaitanya Kulkarni, Jens Axboe, Keith Busch, Sagi Grimberg,
James Smart
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel
On 3/28/26 01:43, Mohamed Khalfella wrote:
> Defined by TP8028 Rapid Path Failure Recovery, CCR (Cross-Controller
> Reset) command is an nvme command issued to source controller by
> initiator to reset impacted controller. Implement CCR command for linux
> nvme target.
>
> Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
> ---
> drivers/nvme/target/admin-cmd.c | 74 ++++++++++++++++++++++++++++++++
> drivers/nvme/target/core.c | 76 +++++++++++++++++++++++++++++++++
> drivers/nvme/target/nvmet.h | 13 ++++++
> include/linux/nvme.h | 23 ++++++++++
> 4 files changed, 186 insertions(+)
>
> diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
> index ec09e30eca18..0a37c0eeebb5 100644
> --- a/drivers/nvme/target/admin-cmd.c
> +++ b/drivers/nvme/target/admin-cmd.c
> @@ -376,7 +376,9 @@ static void nvmet_get_cmd_effects_admin(struct nvmet_ctrl *ctrl,
> log->acs[nvme_admin_get_features] =
> log->acs[nvme_admin_async_event] =
> log->acs[nvme_admin_keep_alive] =
> + log->acs[nvme_admin_cross_ctrl_reset] =
> cpu_to_le32(NVME_CMD_EFFECTS_CSUPP);
> +
> }
>
> static void nvmet_get_cmd_effects_nvm(struct nvme_effects_log *log)
> @@ -1613,6 +1615,75 @@ void nvmet_execute_keep_alive(struct nvmet_req *req)
> nvmet_req_complete(req, status);
> }
>
> +void nvmet_execute_cross_ctrl_reset(struct nvmet_req *req)
> +{
> + struct nvmet_ctrl *ictrl, *sctrl = req->sq->ctrl;
> + struct nvme_command *cmd = req->cmd;
> + struct nvmet_ccr *ccr, *new_ccr;
> + int ccr_active, ccr_total;
> + u16 cntlid, status = NVME_SC_SUCCESS;
> +
> + cntlid = le16_to_cpu(cmd->ccr.icid);
> + if (sctrl->cntlid == cntlid) {
> + req->error_loc =
> + offsetof(struct nvme_cross_ctrl_reset_cmd, icid);
> + status = NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
> + goto out;
> + }
> +
> + /* Find and get impacted controller */
> + ictrl = nvmet_ctrl_find_get_ccr(sctrl->subsys, sctrl->hostnqn,
> + cmd->ccr.ciu, cntlid,
> + le64_to_cpu(cmd->ccr.cirn));
> + if (!ictrl) {
> + /* Immediate Reset Successful */
> + nvmet_set_result(req, 1);
> + status = NVME_SC_SUCCESS;
> + goto out;
> + }
> +
> + ccr_total = ccr_active = 0;
> + mutex_lock(&sctrl->lock);
> + list_for_each_entry(ccr, &sctrl->ccr_list, entry) {
> + if (ccr->ctrl == ictrl) {
> + status = NVME_SC_CCR_IN_PROGRESS | NVME_STATUS_DNR;
> + goto out_unlock;
> + }
> +
> + ccr_total++;
> + if (ccr->ctrl)
> + ccr_active++;
> + }
> +
> + if (ccr_active >= NVMF_CCR_LIMIT) {
> + status = NVME_SC_CCR_LIMIT_EXCEEDED;
> + goto out_unlock;
> + }
> + if (ccr_total >= NVMF_CCR_PER_PAGE) {
> + status = NVME_SC_CCR_LOGPAGE_FULL;
> + goto out_unlock;
> + }
> +
> + new_ccr = kmalloc_obj(*new_ccr, GFP_KERNEL);
> + if (!new_ccr) {
> + status = NVME_SC_INTERNAL;
> + goto out_unlock;
> + }
> +
> + new_ccr->ciu = cmd->ccr.ciu;
> + new_ccr->icid = cntlid;
> + new_ccr->ctrl = ictrl;
> + list_add_tail(&new_ccr->entry, &sctrl->ccr_list);
> +
> +out_unlock:
> + mutex_unlock(&sctrl->lock);
> + if (status == NVME_SC_SUCCESS)
> + nvmet_ctrl_fatal_error(ictrl);
> + nvmet_ctrl_put(ictrl);
> +out:
> + nvmet_req_complete(req, status);
> +}
> +
> u32 nvmet_admin_cmd_data_len(struct nvmet_req *req)
> {
> struct nvme_command *cmd = req->cmd;
> @@ -1690,6 +1761,9 @@ u16 nvmet_parse_admin_cmd(struct nvmet_req *req)
> case nvme_admin_keep_alive:
> req->execute = nvmet_execute_keep_alive;
> return 0;
> + case nvme_admin_cross_ctrl_reset:
> + req->execute = nvmet_execute_cross_ctrl_reset;
> + return 0;
> default:
> return nvmet_report_invalid_opcode(req);
> }
> diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
> index e8b945a01f35..2e0c31d82bad 100644
> --- a/drivers/nvme/target/core.c
> +++ b/drivers/nvme/target/core.c
> @@ -117,6 +117,20 @@ u16 nvmet_zero_sgl(struct nvmet_req *req, off_t off, size_t len)
> return 0;
> }
>
> +void nvmet_ctrl_cleanup_ccrs(struct nvmet_ctrl *ctrl, bool all)
> +{
> + struct nvmet_ccr *ccr, *tmp;
> +
> + lockdep_assert_held(&ctrl->lock);
> +
> + list_for_each_entry_safe(ccr, tmp, &ctrl->ccr_list, entry) {
> + if (all || ccr->ctrl == NULL) {
> + list_del(&ccr->entry);
> + kfree(ccr);
> + }
> + }
> +}
> +
> static u32 nvmet_max_nsid(struct nvmet_subsys *subsys)
> {
> struct nvmet_ns *cur;
> @@ -1399,6 +1413,7 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
> if (!nvmet_is_disc_subsys(ctrl->subsys)) {
> ctrl->ciu = ((u8)(ctrl->ciu + 1)) ? : 1;
> ctrl->cirn = get_random_u64();
> + nvmet_ctrl_cleanup_ccrs(ctrl, false);
> }
> ctrl->csts = NVME_CSTS_RDY;
>
> @@ -1504,6 +1519,35 @@ struct nvmet_ctrl *nvmet_ctrl_find_get(const char *subsysnqn,
> return ctrl;
> }
>
> +struct nvmet_ctrl *nvmet_ctrl_find_get_ccr(struct nvmet_subsys *subsys,
> + const char *hostnqn, u8 ciu,
> + u16 cntlid, u64 cirn)
> +{
> + struct nvmet_ctrl *ctrl, *ictrl = NULL;
> + bool found = false;
> +
> + mutex_lock(&subsys->lock);
> + list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
> + if (ctrl->cntlid != cntlid)
> + continue;
> +
> + /* Avoid racing with a controller that is becoming ready */
> + mutex_lock(&ctrl->lock);
> + if (ctrl->ciu == ciu && ctrl->cirn == cirn)
> + found = true;
> + mutex_unlock(&ctrl->lock);
> +
> + if (found) {
> + if (kref_get_unless_zero(&ctrl->ref))
> + ictrl = ctrl;
> + break;
> + }
> + };
> + mutex_unlock(&subsys->lock);
> +
> + return ictrl;
> +}
> +
> u16 nvmet_check_ctrl_status(struct nvmet_req *req)
> {
> if (unlikely(!(req->sq->ctrl->cc & NVME_CC_ENABLE))) {
> @@ -1629,6 +1673,7 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
> subsys->clear_ids = 1;
> #endif
>
> + INIT_LIST_HEAD(&ctrl->ccr_list);
> INIT_WORK(&ctrl->async_event_work, nvmet_async_event_work);
> INIT_LIST_HEAD(&ctrl->async_events);
> INIT_RADIX_TREE(&ctrl->p2p_ns_map, GFP_KERNEL);
> @@ -1739,12 +1784,43 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
> }
> EXPORT_SYMBOL_GPL(nvmet_alloc_ctrl);
>
> +static void nvmet_ctrl_complete_pending_ccr(struct nvmet_ctrl *ctrl)
> +{
> + struct nvmet_subsys *subsys = ctrl->subsys;
> + struct nvmet_ctrl *sctrl;
> + struct nvmet_ccr *ccr;
> +
> + lockdep_assert_held(&subsys->lock);
> +
> + /* Cleanup all CCRs issued by ctrl as source controller */
> + mutex_lock(&ctrl->lock);
> + nvmet_ctrl_cleanup_ccrs(ctrl, true);
> + mutex_unlock(&ctrl->lock);
> +
> + /*
> + * Find all CCRs targeting ctrl as impacted controller and
> + * set ccr->ctrl to NULL. This tells the source controller
> + * that CCR completed successfully.
> + */
> + list_for_each_entry(sctrl, &subsys->ctrls, subsys_entry) {
> + mutex_lock(&sctrl->lock);
> + list_for_each_entry(ccr, &sctrl->ccr_list, entry) {
> + if (ccr->ctrl == ctrl) {
> + ccr->ctrl = NULL;
> + break;
> + }
> + }
> + mutex_unlock(&sctrl->lock);
> + }
> +}
> +
Do I see this correct that with this implementation a CCR is only
complete once the controller resets? IOW the CCR has to wait for
the controller to be reset, but it does not invoke a controller reset
itself?
Is that intended?
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v4 03/15] nvmet: Implement CCR nvme command
2026-03-30 10:45 ` Hannes Reinecke
@ 2026-03-31 16:38 ` Mohamed Khalfella
0 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-31 16:38 UTC (permalink / raw)
To: Hannes Reinecke
Cc: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart, Aaron Dailey,
Randy Jennings, Dhaval Giani, linux-nvme, linux-kernel
On Mon 2026-03-30 12:45:57 +0200, Hannes Reinecke wrote:
> On 3/28/26 01:43, Mohamed Khalfella wrote:
> > Defined by TP8028 Rapid Path Failure Recovery, CCR (Cross-Controller
> > Reset) command is an nvme command issued to source controller by
> > initiator to reset impacted controller. Implement CCR command for linux
> > nvme target.
> >
> > Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
> > ---
> > drivers/nvme/target/admin-cmd.c | 74 ++++++++++++++++++++++++++++++++
> > drivers/nvme/target/core.c | 76 +++++++++++++++++++++++++++++++++
> > drivers/nvme/target/nvmet.h | 13 ++++++
> > include/linux/nvme.h | 23 ++++++++++
> > 4 files changed, 186 insertions(+)
> >
> > diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
> > index ec09e30eca18..0a37c0eeebb5 100644
> > --- a/drivers/nvme/target/admin-cmd.c
> > +++ b/drivers/nvme/target/admin-cmd.c
> > @@ -376,7 +376,9 @@ static void nvmet_get_cmd_effects_admin(struct nvmet_ctrl *ctrl,
> > log->acs[nvme_admin_get_features] =
> > log->acs[nvme_admin_async_event] =
> > log->acs[nvme_admin_keep_alive] =
> > + log->acs[nvme_admin_cross_ctrl_reset] =
> > cpu_to_le32(NVME_CMD_EFFECTS_CSUPP);
> > +
> > }
> >
> > static void nvmet_get_cmd_effects_nvm(struct nvme_effects_log *log)
> > @@ -1613,6 +1615,75 @@ void nvmet_execute_keep_alive(struct nvmet_req *req)
> > nvmet_req_complete(req, status);
> > }
> >
> > +void nvmet_execute_cross_ctrl_reset(struct nvmet_req *req)
> > +{
> > + struct nvmet_ctrl *ictrl, *sctrl = req->sq->ctrl;
> > + struct nvme_command *cmd = req->cmd;
> > + struct nvmet_ccr *ccr, *new_ccr;
> > + int ccr_active, ccr_total;
> > + u16 cntlid, status = NVME_SC_SUCCESS;
> > +
> > + cntlid = le16_to_cpu(cmd->ccr.icid);
> > + if (sctrl->cntlid == cntlid) {
> > + req->error_loc =
> > + offsetof(struct nvme_cross_ctrl_reset_cmd, icid);
> > + status = NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
> > + goto out;
> > + }
> > +
> > + /* Find and get impacted controller */
> > + ictrl = nvmet_ctrl_find_get_ccr(sctrl->subsys, sctrl->hostnqn,
> > + cmd->ccr.ciu, cntlid,
> > + le64_to_cpu(cmd->ccr.cirn));
> > + if (!ictrl) {
> > + /* Immediate Reset Successful */
> > + nvmet_set_result(req, 1);
> > + status = NVME_SC_SUCCESS;
> > + goto out;
> > + }
> > +
> > + ccr_total = ccr_active = 0;
> > + mutex_lock(&sctrl->lock);
> > + list_for_each_entry(ccr, &sctrl->ccr_list, entry) {
> > + if (ccr->ctrl == ictrl) {
> > + status = NVME_SC_CCR_IN_PROGRESS | NVME_STATUS_DNR;
> > + goto out_unlock;
> > + }
> > +
> > + ccr_total++;
> > + if (ccr->ctrl)
> > + ccr_active++;
> > + }
> > +
> > + if (ccr_active >= NVMF_CCR_LIMIT) {
> > + status = NVME_SC_CCR_LIMIT_EXCEEDED;
> > + goto out_unlock;
> > + }
> > + if (ccr_total >= NVMF_CCR_PER_PAGE) {
> > + status = NVME_SC_CCR_LOGPAGE_FULL;
> > + goto out_unlock;
> > + }
> > +
> > + new_ccr = kmalloc_obj(*new_ccr, GFP_KERNEL);
> > + if (!new_ccr) {
> > + status = NVME_SC_INTERNAL;
> > + goto out_unlock;
> > + }
> > +
> > + new_ccr->ciu = cmd->ccr.ciu;
> > + new_ccr->icid = cntlid;
> > + new_ccr->ctrl = ictrl;
> > + list_add_tail(&new_ccr->entry, &sctrl->ccr_list);
> > +
> > +out_unlock:
> > + mutex_unlock(&sctrl->lock);
> > + if (status == NVME_SC_SUCCESS)
> > + nvmet_ctrl_fatal_error(ictrl);
> > + nvmet_ctrl_put(ictrl);
> > +out:
> > + nvmet_req_complete(req, status);
> > +}
> > +
> > u32 nvmet_admin_cmd_data_len(struct nvmet_req *req)
> > {
> > struct nvme_command *cmd = req->cmd;
> > @@ -1690,6 +1761,9 @@ u16 nvmet_parse_admin_cmd(struct nvmet_req *req)
> > case nvme_admin_keep_alive:
> > req->execute = nvmet_execute_keep_alive;
> > return 0;
> > + case nvme_admin_cross_ctrl_reset:
> > + req->execute = nvmet_execute_cross_ctrl_reset;
> > + return 0;
> > default:
> > return nvmet_report_invalid_opcode(req);
> > }
> > diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
> > index e8b945a01f35..2e0c31d82bad 100644
> > --- a/drivers/nvme/target/core.c
> > +++ b/drivers/nvme/target/core.c
> > @@ -117,6 +117,20 @@ u16 nvmet_zero_sgl(struct nvmet_req *req, off_t off, size_t len)
> > return 0;
> > }
> >
> > +void nvmet_ctrl_cleanup_ccrs(struct nvmet_ctrl *ctrl, bool all)
> > +{
> > + struct nvmet_ccr *ccr, *tmp;
> > +
> > + lockdep_assert_held(&ctrl->lock);
> > +
> > + list_for_each_entry_safe(ccr, tmp, &ctrl->ccr_list, entry) {
> > + if (all || ccr->ctrl == NULL) {
> > + list_del(&ccr->entry);
> > + kfree(ccr);
> > + }
> > + }
> > +}
> > +
> > static u32 nvmet_max_nsid(struct nvmet_subsys *subsys)
> > {
> > struct nvmet_ns *cur;
> > @@ -1399,6 +1413,7 @@ static void nvmet_start_ctrl(struct nvmet_ctrl *ctrl)
> > if (!nvmet_is_disc_subsys(ctrl->subsys)) {
> > ctrl->ciu = ((u8)(ctrl->ciu + 1)) ? : 1;
> > ctrl->cirn = get_random_u64();
> > + nvmet_ctrl_cleanup_ccrs(ctrl, false);
> > }
> > ctrl->csts = NVME_CSTS_RDY;
> >
> > @@ -1504,6 +1519,35 @@ struct nvmet_ctrl *nvmet_ctrl_find_get(const char *subsysnqn,
> > return ctrl;
> > }
> >
> > +struct nvmet_ctrl *nvmet_ctrl_find_get_ccr(struct nvmet_subsys *subsys,
> > + const char *hostnqn, u8 ciu,
> > + u16 cntlid, u64 cirn)
> > +{
> > + struct nvmet_ctrl *ctrl, *ictrl = NULL;
> > + bool found = false;
> > +
> > + mutex_lock(&subsys->lock);
> > + list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
> > + if (ctrl->cntlid != cntlid)
> > + continue;
> > +
> > + /* Avoid racing with a controller that is becoming ready */
> > + mutex_lock(&ctrl->lock);
> > + if (ctrl->ciu == ciu && ctrl->cirn == cirn)
> > + found = true;
> > + mutex_unlock(&ctrl->lock);
> > +
> > + if (found) {
> > + if (kref_get_unless_zero(&ctrl->ref))
> > + ictrl = ctrl;
> > + break;
> > + }
> > + };
> > + mutex_unlock(&subsys->lock);
> > +
> > + return ictrl;
> > +}
> > +
> > u16 nvmet_check_ctrl_status(struct nvmet_req *req)
> > {
> > if (unlikely(!(req->sq->ctrl->cc & NVME_CC_ENABLE))) {
> > @@ -1629,6 +1673,7 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
> > subsys->clear_ids = 1;
> > #endif
> >
> > + INIT_LIST_HEAD(&ctrl->ccr_list);
> > INIT_WORK(&ctrl->async_event_work, nvmet_async_event_work);
> > INIT_LIST_HEAD(&ctrl->async_events);
> > INIT_RADIX_TREE(&ctrl->p2p_ns_map, GFP_KERNEL);
> > @@ -1739,12 +1784,43 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
> > }
> > EXPORT_SYMBOL_GPL(nvmet_alloc_ctrl);
> >
> > +static void nvmet_ctrl_complete_pending_ccr(struct nvmet_ctrl *ctrl)
> > +{
> > + struct nvmet_subsys *subsys = ctrl->subsys;
> > + struct nvmet_ctrl *sctrl;
> > + struct nvmet_ccr *ccr;
> > +
> > + lockdep_assert_held(&subsys->lock);
> > +
> > + /* Cleanup all CCRs issued by ctrl as source controller */
> > + mutex_lock(&ctrl->lock);
> > + nvmet_ctrl_cleanup_ccrs(ctrl, true);
> > + mutex_unlock(&ctrl->lock);
> > +
> > + /*
> > + * Find all CCRs targeting ctrl as impacted controller and
> > + * set ccr->ctrl to NULL. This tells the source controller
> > + * that CCR completed successfully.
> > + */
> > + list_for_each_entry(sctrl, &subsys->ctrls, subsys_entry) {
> > + mutex_lock(&sctrl->lock);
> > + list_for_each_entry(ccr, &sctrl->ccr_list, entry) {
> > + if (ccr->ctrl == ctrl) {
> > + ccr->ctrl = NULL;
> > + break;
> > + }
> > + }
> > + mutex_unlock(&sctrl->lock);
> > + }
> > +}
> > +
>
> Do I see this correct that with this implementation a CCR is only
> complete once the controller resets? IOW the CCR has to wait for
> the controller to be reset, but it does not invoke a controller reset
> itself?
>
> Is that intended?
nvmet_execute_cross_ctrl_reset() calls nvmet_ctrl_fatal_error() to cause
impacted controller to fail. CCR is completed when the impacted
controller exits.
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 04/15] nvmet: Implement CCR logpage
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (2 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 03/15] nvmet: Implement CCR nvme command Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 05/15] nvmet: Send an AEN on CCR completion Mohamed Khalfella
` (10 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
Defined by TP8028 Rapid Path Failure Recovery, CCR (Cross-Controller
Reset) log page contains an entry for each CCR request submitted to
source controller. Implement CCR logpage for nvme linux target.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/target/admin-cmd.c | 44 +++++++++++++++++++++++++++++++++
include/linux/nvme.h | 29 ++++++++++++++++++++++
2 files changed, 73 insertions(+)
diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index 0a37c0eeebb5..305a5d9b5450 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -220,6 +220,7 @@ static void nvmet_execute_get_supported_log_pages(struct nvmet_req *req)
logs->lids[NVME_LOG_FEATURES] = cpu_to_le32(NVME_LIDS_LSUPP);
logs->lids[NVME_LOG_RMI] = cpu_to_le32(NVME_LIDS_LSUPP);
logs->lids[NVME_LOG_RESERVATION] = cpu_to_le32(NVME_LIDS_LSUPP);
+ logs->lids[NVME_LOG_CCR] = cpu_to_le32(NVME_LIDS_LSUPP);
status = nvmet_copy_to_sgl(req, 0, logs, sizeof(*logs));
kfree(logs);
@@ -607,6 +608,47 @@ static void nvmet_execute_get_log_page_features(struct nvmet_req *req)
nvmet_req_complete(req, status);
}
+static void nvmet_execute_get_log_page_ccr(struct nvmet_req *req)
+{
+ struct nvmet_ctrl *ctrl = req->sq->ctrl;
+ struct nvmet_ccr *ccr;
+ struct nvme_ccr_log *log;
+ int index = 0;
+ u16 status;
+
+ log = kzalloc(sizeof(*log), GFP_KERNEL);
+ if (!log) {
+ status = NVME_SC_INTERNAL;
+ goto out;
+ }
+
+ mutex_lock(&ctrl->lock);
+ list_for_each_entry(ccr, &ctrl->ccr_list, entry) {
+ u8 flags = NVME_CCR_FLAGS_VALIDATED | NVME_CCR_FLAGS_INITIATED;
+ u8 status = ccr->ctrl ? NVME_CCR_STATUS_IN_PROGRESS :
+ NVME_CCR_STATUS_SUCCESS;
+
+ log->entries[index].icid = cpu_to_le16(ccr->icid);
+ log->entries[index].ciu = ccr->ciu;
+ log->entries[index].acid = cpu_to_le16(0xffff);
+ log->entries[index].ccrs = status;
+ log->entries[index].ccrf = flags;
+ index++;
+ }
+
+ /* Cleanup completed CCRs if requested */
+ if (req->cmd->get_log_page.lsp & 0x1)
+ nvmet_ctrl_cleanup_ccrs(ctrl, false);
+ mutex_unlock(&ctrl->lock);
+
+ log->ne = cpu_to_le16(index);
+ nvmet_clear_aen_bit(req, NVME_AEN_BIT_CCR_COMPLETE);
+ status = nvmet_copy_to_sgl(req, 0, log, sizeof(*log));
+ kfree(log);
+out:
+ nvmet_req_complete(req, status);
+}
+
static void nvmet_execute_get_log_page(struct nvmet_req *req)
{
if (!nvmet_check_transfer_len(req, nvmet_get_log_page_len(req->cmd)))
@@ -640,6 +682,8 @@ static void nvmet_execute_get_log_page(struct nvmet_req *req)
return nvmet_execute_get_log_page_rmi(req);
case NVME_LOG_RESERVATION:
return nvmet_execute_get_log_page_resv(req);
+ case NVME_LOG_CCR:
+ return nvmet_execute_get_log_page_ccr(req);
}
pr_debug("unhandled lid %d on qid %d\n",
req->cmd->get_log_page.lid, req->sq->qid);
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index bd3b3f2a5377..b792d488f72e 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -1432,6 +1432,7 @@ enum {
NVME_LOG_FDP_CONFIGS = 0x20,
NVME_LOG_DISC = 0x70,
NVME_LOG_RESERVATION = 0x80,
+ NVME_LOG_CCR = 0x1E,
NVME_FWACT_REPL = (0 << 3),
NVME_FWACT_REPL_ACTV = (1 << 3),
NVME_FWACT_ACTV = (2 << 3),
@@ -1455,6 +1456,34 @@ enum {
NVME_FIS_CSCPE = 1 << 21,
};
+/* NVMe Cross-Controller Reset Status */
+enum {
+ NVME_CCR_STATUS_IN_PROGRESS,
+ NVME_CCR_STATUS_SUCCESS,
+ NVME_CCR_STATUS_FAILED,
+};
+
+/* NVMe Cross-Controller Reset Flags */
+enum {
+ NVME_CCR_FLAGS_VALIDATED = 0x01,
+ NVME_CCR_FLAGS_INITIATED = 0x02,
+};
+
+struct nvme_ccr_log_entry {
+ __le16 icid;
+ __u8 ciu;
+ __u8 rsvd3;
+ __le16 acid;
+ __u8 ccrs;
+ __u8 ccrf;
+};
+
+struct nvme_ccr_log {
+ __le16 ne;
+ __u8 rsvd2[6];
+ struct nvme_ccr_log_entry entries[NVMF_CCR_PER_PAGE];
+};
+
/* NVMe Namespace Write Protect State */
enum {
NVME_NS_NO_WRITE_PROTECT = 0,
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v4 05/15] nvmet: Send an AEN on CCR completion
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (3 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 04/15] nvmet: Implement CCR logpage Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 06/15] nvme: Rapid Path Failure Recovery read controller identify fields Mohamed Khalfella
` (9 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
Send an AEN to initiator when impacted controller exists. The
notification points to CCR log page that initiator can read to check
which CCR operation completed.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/target/core.c | 25 ++++++++++++++++++++++---
drivers/nvme/target/nvmet.h | 3 ++-
include/linux/nvme.h | 3 +++
3 files changed, 27 insertions(+), 4 deletions(-)
diff --git a/drivers/nvme/target/core.c b/drivers/nvme/target/core.c
index 2e0c31d82bad..4f1f0562cef8 100644
--- a/drivers/nvme/target/core.c
+++ b/drivers/nvme/target/core.c
@@ -205,7 +205,7 @@ static void nvmet_async_event_work(struct work_struct *work)
nvmet_async_events_process(ctrl);
}
-void nvmet_add_async_event(struct nvmet_ctrl *ctrl, u8 event_type,
+static void nvmet_add_async_event_locked(struct nvmet_ctrl *ctrl, u8 event_type,
u8 event_info, u8 log_page)
{
struct nvmet_async_event *aen;
@@ -218,13 +218,19 @@ void nvmet_add_async_event(struct nvmet_ctrl *ctrl, u8 event_type,
aen->event_info = event_info;
aen->log_page = log_page;
- mutex_lock(&ctrl->lock);
list_add_tail(&aen->entry, &ctrl->async_events);
- mutex_unlock(&ctrl->lock);
queue_work(nvmet_aen_wq, &ctrl->async_event_work);
}
+void nvmet_add_async_event(struct nvmet_ctrl *ctrl, u8 event_type,
+ u8 event_info, u8 log_page)
+{
+ mutex_lock(&ctrl->lock);
+ nvmet_add_async_event_locked(ctrl, event_type, event_info, log_page);
+ mutex_unlock(&ctrl->lock);
+}
+
static void nvmet_add_to_changed_ns_log(struct nvmet_ctrl *ctrl, __le32 nsid)
{
u32 i;
@@ -1784,6 +1790,18 @@ struct nvmet_ctrl *nvmet_alloc_ctrl(struct nvmet_alloc_ctrl_args *args)
}
EXPORT_SYMBOL_GPL(nvmet_alloc_ctrl);
+static void nvmet_ctrl_notify_ccr(struct nvmet_ctrl *ctrl)
+{
+ lockdep_assert_held(&ctrl->lock);
+
+ if (nvmet_aen_bit_disabled(ctrl, NVME_AEN_BIT_CCR_COMPLETE))
+ return;
+
+ nvmet_add_async_event_locked(ctrl, NVME_AER_NOTICE,
+ NVME_AER_NOTICE_CCR_COMPLETED,
+ NVME_LOG_CCR);
+}
+
static void nvmet_ctrl_complete_pending_ccr(struct nvmet_ctrl *ctrl)
{
struct nvmet_subsys *subsys = ctrl->subsys;
@@ -1807,6 +1825,7 @@ static void nvmet_ctrl_complete_pending_ccr(struct nvmet_ctrl *ctrl)
list_for_each_entry(ccr, &sctrl->ccr_list, entry) {
if (ccr->ctrl == ctrl) {
ccr->ctrl = NULL;
+ nvmet_ctrl_notify_ccr(sctrl);
break;
}
}
diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h
index b9eb044ded19..6546be25901a 100644
--- a/drivers/nvme/target/nvmet.h
+++ b/drivers/nvme/target/nvmet.h
@@ -44,7 +44,8 @@
* Supported optional AENs:
*/
#define NVMET_AEN_CFG_OPTIONAL \
- (NVME_AEN_CFG_NS_ATTR | NVME_AEN_CFG_ANA_CHANGE)
+ (NVME_AEN_CFG_NS_ATTR | NVME_AEN_CFG_ANA_CHANGE | \
+ NVME_AEN_CFG_CCR_COMPLETE)
#define NVMET_DISC_AEN_CFG_OPTIONAL \
(NVME_AEN_CFG_DISC_CHANGE)
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index b792d488f72e..1fbb5b268303 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -860,12 +860,14 @@ enum {
NVME_AER_NOTICE_FW_ACT_STARTING = 0x01,
NVME_AER_NOTICE_ANA = 0x03,
NVME_AER_NOTICE_DISC_CHANGED = 0xf0,
+ NVME_AER_NOTICE_CCR_COMPLETED = 0xf4,
};
enum {
NVME_AEN_BIT_NS_ATTR = 8,
NVME_AEN_BIT_FW_ACT = 9,
NVME_AEN_BIT_ANA_CHANGE = 11,
+ NVME_AEN_BIT_CCR_COMPLETE = 20,
NVME_AEN_BIT_DISC_CHANGE = 31,
};
@@ -873,6 +875,7 @@ enum {
NVME_AEN_CFG_NS_ATTR = 1 << NVME_AEN_BIT_NS_ATTR,
NVME_AEN_CFG_FW_ACT = 1 << NVME_AEN_BIT_FW_ACT,
NVME_AEN_CFG_ANA_CHANGE = 1 << NVME_AEN_BIT_ANA_CHANGE,
+ NVME_AEN_CFG_CCR_COMPLETE = 1 << NVME_AEN_BIT_CCR_COMPLETE,
NVME_AEN_CFG_DISC_CHANGE = 1 << NVME_AEN_BIT_DISC_CHANGE,
};
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v4 06/15] nvme: Rapid Path Failure Recovery read controller identify fields
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (4 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 05/15] nvmet: Send an AEN on CCR completion Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 07/15] nvme: Introduce FENCING and FENCED controller states Mohamed Khalfella
` (8 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
TP8028 Rapid path failure added new fileds to controller identify
response. Read CIU (Controller Instance Uniquifier), CIRN (Controller
Instance Random Number), and CCRL (Cross-Controller Reset Limit) from
controller identify response. Expose CIU and CIRN as sysfs attributes
so the values can be used directrly by user if needed.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/core.c | 4 ++++
drivers/nvme/host/nvme.h | 10 ++++++++++
drivers/nvme/host/sysfs.c | 23 +++++++++++++++++++++++
3 files changed, 37 insertions(+)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 766e9cc4ffca..7a07c23aefdb 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3567,6 +3567,10 @@ static int nvme_init_identify(struct nvme_ctrl *ctrl)
ctrl->crdt[1] = le16_to_cpu(id->crdt2);
ctrl->crdt[2] = le16_to_cpu(id->crdt3);
+ ctrl->ciu = id->ciu;
+ ctrl->cirn = le64_to_cpu(id->cirn);
+ atomic_set(&ctrl->ccr_limit, id->ccrl);
+
ctrl->oacs = le16_to_cpu(id->oacs);
ctrl->oncs = le16_to_cpu(id->oncs);
ctrl->mtfa = le16_to_cpu(id->mtfa);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 9971045dbc05..234f3872a212 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -382,11 +382,14 @@ struct nvme_ctrl {
u16 crdt[3];
u16 oncs;
u8 dmrl;
+ u8 ciu;
u32 dmrsl;
+ u64 cirn;
u16 oacs;
u16 sqsize;
u32 max_namespaces;
atomic_t abort_limit;
+ atomic_t ccr_limit;
u8 vwc;
u32 vs;
u32 sgls;
@@ -1280,4 +1283,11 @@ static inline bool nvme_multi_css(struct nvme_ctrl *ctrl)
return (ctrl->ctrl_config & NVME_CC_CSS_MASK) == NVME_CC_CSS_CSI;
}
+static inline unsigned long nvme_fence_timeout_ms(struct nvme_ctrl *ctrl)
+{
+ if (ctrl->ctratt & NVME_CTRL_ATTR_TBKAS)
+ return 3 * ctrl->kato * 1000;
+ return 2 * ctrl->kato * 1000;
+}
+
#endif /* _NVME_H */
diff --git a/drivers/nvme/host/sysfs.c b/drivers/nvme/host/sysfs.c
index 16c6fea4b2db..f182a26b38b0 100644
--- a/drivers/nvme/host/sysfs.c
+++ b/drivers/nvme/host/sysfs.c
@@ -388,6 +388,27 @@ nvme_show_int_function(queue_count);
nvme_show_int_function(sqsize);
nvme_show_int_function(kato);
+static ssize_t nvme_sysfs_ciu_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+
+ return sysfs_emit(buf, "%02x\n", ctrl->ciu);
+}
+static DEVICE_ATTR(ciu, S_IRUSR, nvme_sysfs_ciu_show, NULL);
+
+static ssize_t nvme_sysfs_cirn_show(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
+
+ return sysfs_emit(buf, "%016llx\n", ctrl->cirn);
+}
+static DEVICE_ATTR(cirn, S_IRUSR, nvme_sysfs_cirn_show, NULL);
+
+
static ssize_t nvme_sysfs_delete(struct device *dev,
struct device_attribute *attr, const char *buf,
size_t count)
@@ -756,6 +777,8 @@ static struct attribute *nvme_dev_attrs[] = {
&dev_attr_numa_node.attr,
&dev_attr_queue_count.attr,
&dev_attr_sqsize.attr,
+ &dev_attr_ciu.attr,
+ &dev_attr_cirn.attr,
&dev_attr_hostnqn.attr,
&dev_attr_hostid.attr,
&dev_attr_ctrl_loss_tmo.attr,
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v4 07/15] nvme: Introduce FENCING and FENCED controller states
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (5 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 06/15] nvme: Rapid Path Failure Recovery read controller identify fields Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-30 10:46 ` Hannes Reinecke
2026-03-28 0:43 ` [PATCH v4 08/15] nvme: Implement cross-controller reset recovery Mohamed Khalfella
` (7 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
FENCING is a new controller state that a LIVE controller enters when an
error is encountered. While in FENCING state, inflight IOs that timeout
are not canceled because they should be held until either CCR succeeds
or time-based recovery completes. While the queues remain alive, requests
are not allowed to be sent in this state, and the controller cannot be
reset or deleted. This is intentional because resetting or deleting the
controller results in canceling inflight IOs.
FENCED is a short-term state the controller enters before it is reset.
It exists only to prevent manual resets from happening while controller
is in FENCING state.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
---
drivers/nvme/host/core.c | 27 +++++++++++++++++++++++++--
drivers/nvme/host/nvme.h | 4 ++++
drivers/nvme/host/sysfs.c | 2 ++
3 files changed, 31 insertions(+), 2 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 7a07c23aefdb..824a1193bec8 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -574,10 +574,29 @@ bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
break;
}
break;
+ case NVME_CTRL_FENCING:
+ switch (old_state) {
+ case NVME_CTRL_LIVE:
+ changed = true;
+ fallthrough;
+ default:
+ break;
+ }
+ break;
+ case NVME_CTRL_FENCED:
+ switch (old_state) {
+ case NVME_CTRL_FENCING:
+ changed = true;
+ fallthrough;
+ default:
+ break;
+ }
+ break;
case NVME_CTRL_RESETTING:
switch (old_state) {
case NVME_CTRL_NEW:
case NVME_CTRL_LIVE:
+ case NVME_CTRL_FENCED:
changed = true;
fallthrough;
default:
@@ -760,6 +779,8 @@ blk_status_t nvme_fail_nonready_command(struct nvme_ctrl *ctrl,
if (state != NVME_CTRL_DELETING_NOIO &&
state != NVME_CTRL_DELETING &&
+ state != NVME_CTRL_FENCING &&
+ state != NVME_CTRL_FENCED &&
state != NVME_CTRL_DEAD &&
!test_bit(NVME_CTRL_FAILFAST_EXPIRED, &ctrl->flags) &&
!blk_noretry_request(rq) && !(rq->cmd_flags & REQ_NVME_MPATH))
@@ -802,10 +823,12 @@ bool __nvme_check_ready(struct nvme_ctrl *ctrl, struct request *rq,
req->cmd->fabrics.fctype == nvme_fabrics_type_auth_receive))
return true;
break;
- default:
- break;
+ case NVME_CTRL_FENCING:
+ case NVME_CTRL_FENCED:
case NVME_CTRL_DEAD:
return false;
+ default:
+ break;
}
}
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 234f3872a212..45e58434cf30 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -305,6 +305,8 @@ static inline u16 nvme_req_qid(struct request *req)
enum nvme_ctrl_state {
NVME_CTRL_NEW,
NVME_CTRL_LIVE,
+ NVME_CTRL_FENCING,
+ NVME_CTRL_FENCED,
NVME_CTRL_RESETTING,
NVME_CTRL_CONNECTING,
NVME_CTRL_DELETING,
@@ -831,6 +833,8 @@ static inline bool nvme_state_terminal(struct nvme_ctrl *ctrl)
switch (nvme_ctrl_state(ctrl)) {
case NVME_CTRL_NEW:
case NVME_CTRL_LIVE:
+ case NVME_CTRL_FENCING:
+ case NVME_CTRL_FENCED:
case NVME_CTRL_RESETTING:
case NVME_CTRL_CONNECTING:
return false;
diff --git a/drivers/nvme/host/sysfs.c b/drivers/nvme/host/sysfs.c
index f182a26b38b0..6ae29fe431dc 100644
--- a/drivers/nvme/host/sysfs.c
+++ b/drivers/nvme/host/sysfs.c
@@ -443,6 +443,8 @@ static ssize_t nvme_sysfs_show_state(struct device *dev,
static const char *const state_name[] = {
[NVME_CTRL_NEW] = "new",
[NVME_CTRL_LIVE] = "live",
+ [NVME_CTRL_FENCING] = "fencing",
+ [NVME_CTRL_FENCED] = "fenced",
[NVME_CTRL_RESETTING] = "resetting",
[NVME_CTRL_CONNECTING] = "connecting",
[NVME_CTRL_DELETING] = "deleting",
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v4 07/15] nvme: Introduce FENCING and FENCED controller states
2026-03-28 0:43 ` [PATCH v4 07/15] nvme: Introduce FENCING and FENCED controller states Mohamed Khalfella
@ 2026-03-30 10:46 ` Hannes Reinecke
0 siblings, 0 replies; 25+ messages in thread
From: Hannes Reinecke @ 2026-03-30 10:46 UTC (permalink / raw)
To: Mohamed Khalfella, Justin Tee, Naresh Gottumukkala, Paul Ely,
Chaitanya Kulkarni, Jens Axboe, Keith Busch, Sagi Grimberg,
James Smart
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel
On 3/28/26 01:43, Mohamed Khalfella wrote:
> FENCING is a new controller state that a LIVE controller enters when an
> error is encountered. While in FENCING state, inflight IOs that timeout
> are not canceled because they should be held until either CCR succeeds
> or time-based recovery completes. While the queues remain alive, requests
> are not allowed to be sent in this state, and the controller cannot be
> reset or deleted. This is intentional because resetting or deleting the
> controller results in canceling inflight IOs.
>
> FENCED is a short-term state the controller enters before it is reset.
> It exists only to prevent manual resets from happening while controller
> is in FENCING state.
>
> Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
> ---
> drivers/nvme/host/core.c | 27 +++++++++++++++++++++++++--
> drivers/nvme/host/nvme.h | 4 ++++
> drivers/nvme/host/sysfs.c | 2 ++
> 3 files changed, 31 insertions(+), 2 deletions(-)
>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 08/15] nvme: Implement cross-controller reset recovery
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (6 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 07/15] nvme: Introduce FENCING and FENCED controller states Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-30 10:50 ` Hannes Reinecke
2026-03-28 0:43 ` [PATCH v4 09/15] nvme: Implement cross-controller reset completion Mohamed Khalfella
` (6 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
A host that has more than one path connecting to an nvme subsystem
typically has an nvme controller associated with every path. This is
mostly applicable to nvmeof. If one path goes down, inflight IOs on that
path should not be retried immediately on another path because this
could lead to data corruption as described in TP4129. TP8028 defines
cross-controller reset mechanism that can be used by host to terminate
IOs on the failed path using one of the remaining healthy paths. Only
after IOs are terminated, or long enough time passes as defined by
TP4129, inflight IOs should be retried on another path. Implement core
cross-controller reset shared logic to be used by the transports.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
---
drivers/nvme/host/constants.c | 1 +
drivers/nvme/host/core.c | 145 ++++++++++++++++++++++++++++++++++
drivers/nvme/host/nvme.h | 9 +++
3 files changed, 155 insertions(+)
diff --git a/drivers/nvme/host/constants.c b/drivers/nvme/host/constants.c
index dc90df9e13a2..f679efd5110e 100644
--- a/drivers/nvme/host/constants.c
+++ b/drivers/nvme/host/constants.c
@@ -46,6 +46,7 @@ static const char * const nvme_admin_ops[] = {
[nvme_admin_virtual_mgmt] = "Virtual Management",
[nvme_admin_nvme_mi_send] = "NVMe Send MI",
[nvme_admin_nvme_mi_recv] = "NVMe Receive MI",
+ [nvme_admin_cross_ctrl_reset] = "Cross Controller Reset",
[nvme_admin_dbbuf] = "Doorbell Buffer Config",
[nvme_admin_format_nvm] = "Format NVM",
[nvme_admin_security_send] = "Security Send",
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 824a1193bec8..5603ae36444f 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -554,6 +554,150 @@ void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl)
}
EXPORT_SYMBOL_GPL(nvme_cancel_admin_tagset);
+static struct nvme_ctrl *nvme_find_ctrl_ccr(struct nvme_ctrl *ictrl,
+ u32 min_cntlid)
+{
+ struct nvme_subsystem *subsys = ictrl->subsys;
+ struct nvme_ctrl *ctrl, *sctrl = NULL;
+ unsigned long flags;
+
+ mutex_lock(&nvme_subsystems_lock);
+ list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
+ if (ctrl->cntlid < min_cntlid)
+ continue;
+
+ if (atomic_dec_if_positive(&ctrl->ccr_limit) < 0)
+ continue;
+
+ spin_lock_irqsave(&ctrl->lock, flags);
+ if (ctrl->state != NVME_CTRL_LIVE) {
+ spin_unlock_irqrestore(&ctrl->lock, flags);
+ atomic_inc(&ctrl->ccr_limit);
+ continue;
+ }
+
+ /*
+ * We got a good candidate source controller that is locked and
+ * LIVE. However, no guarantee ctrl will not be deleted after
+ * ctrl->lock is released. Get a ref of both ctrl and admin_q
+ * so they do not disappear until we are done with them.
+ */
+ WARN_ON_ONCE(!blk_get_queue(ctrl->admin_q));
+ nvme_get_ctrl(ctrl);
+ spin_unlock_irqrestore(&ctrl->lock, flags);
+ sctrl = ctrl;
+ break;
+ }
+ mutex_unlock(&nvme_subsystems_lock);
+ return sctrl;
+}
+
+static void nvme_put_ctrl_ccr(struct nvme_ctrl *sctrl)
+{
+ atomic_inc(&sctrl->ccr_limit);
+ blk_put_queue(sctrl->admin_q);
+ nvme_put_ctrl(sctrl);
+}
+
+static int nvme_issue_wait_ccr(struct nvme_ctrl *sctrl, struct nvme_ctrl *ictrl,
+ unsigned long deadline)
+{
+ struct nvme_ccr_entry ccr = { };
+ union nvme_result res = { 0 };
+ struct nvme_command c = { };
+ unsigned long flags, now, tmo = 0;
+ bool completed = false;
+ int ret = 0;
+ u32 result;
+
+ init_completion(&ccr.complete);
+ ccr.ictrl = ictrl;
+
+ spin_lock_irqsave(&sctrl->lock, flags);
+ list_add_tail(&ccr.list, &sctrl->ccr_list);
+ spin_unlock_irqrestore(&sctrl->lock, flags);
+
+ c.ccr.opcode = nvme_admin_cross_ctrl_reset;
+ c.ccr.ciu = ictrl->ciu;
+ c.ccr.icid = cpu_to_le16(ictrl->cntlid);
+ c.ccr.cirn = cpu_to_le64(ictrl->cirn);
+ ret = __nvme_submit_sync_cmd(sctrl->admin_q, &c, &res,
+ NULL, 0, NVME_QID_ANY, 0);
+ if (ret) {
+ ret = -EIO;
+ goto out;
+ }
+
+ result = le32_to_cpu(res.u32);
+ if (result & 0x01) /* Immediate Reset Successful */
+ goto out;
+
+ now = jiffies;
+ if (time_before(now, deadline))
+ tmo = min_t(unsigned long,
+ secs_to_jiffies(ictrl->kato), deadline - now);
+
+ if (!wait_for_completion_timeout(&ccr.complete, tmo)) {
+ ret = -ETIMEDOUT;
+ goto out;
+ }
+
+ completed = true;
+
+out:
+ spin_lock_irqsave(&sctrl->lock, flags);
+ list_del(&ccr.list);
+ spin_unlock_irqrestore(&sctrl->lock, flags);
+ if (completed) {
+ if (ccr.ccrs == NVME_CCR_STATUS_SUCCESS)
+ return 0;
+ return -EREMOTEIO;
+ }
+ return ret;
+}
+
+int nvme_fence_ctrl(struct nvme_ctrl *ictrl)
+{
+ unsigned long deadline, timeout;
+ struct nvme_ctrl *sctrl;
+ u32 min_cntlid = 0;
+ int ret;
+
+ timeout = nvme_fence_timeout_ms(ictrl);
+ dev_info(ictrl->device, "attempting CCR, timeout %lums\n", timeout);
+
+ deadline = jiffies + msecs_to_jiffies(timeout);
+ while (time_is_after_jiffies(deadline)) {
+ sctrl = nvme_find_ctrl_ccr(ictrl, min_cntlid);
+ if (!sctrl) {
+ dev_dbg(ictrl->device,
+ "failed to find source controller\n");
+ return -EIO;
+ }
+
+ ret = nvme_issue_wait_ccr(sctrl, ictrl, deadline);
+ if (!ret) {
+ dev_info(ictrl->device, "CCR succeeded using %s\n",
+ dev_name(sctrl->device));
+ nvme_put_ctrl_ccr(sctrl);
+ return 0;
+ }
+
+ min_cntlid = sctrl->cntlid + 1;
+ nvme_put_ctrl_ccr(sctrl);
+
+ if (ret == -EIO) /* CCR command failed */
+ continue;
+
+ /* CCR operation failed or timed out */
+ return ret;
+ }
+
+ dev_info(ictrl->device, "CCR operation timeout\n");
+ return -ETIMEDOUT;
+}
+EXPORT_SYMBOL_GPL(nvme_fence_ctrl);
+
bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
enum nvme_ctrl_state new_state)
{
@@ -5116,6 +5260,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
mutex_init(&ctrl->scan_lock);
INIT_LIST_HEAD(&ctrl->namespaces);
+ INIT_LIST_HEAD(&ctrl->ccr_list);
xa_init(&ctrl->cels);
ctrl->dev = dev;
ctrl->ops = ops;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 45e58434cf30..f2bcff9ccd25 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -333,6 +333,13 @@ enum nvme_ctrl_flags {
NVME_CTRL_FROZEN = 6,
};
+struct nvme_ccr_entry {
+ struct list_head list;
+ struct completion complete;
+ struct nvme_ctrl *ictrl;
+ u8 ccrs;
+};
+
struct nvme_ctrl {
bool comp_seen;
bool identified;
@@ -350,6 +357,7 @@ struct nvme_ctrl {
struct blk_mq_tag_set *tagset;
struct blk_mq_tag_set *admin_tagset;
struct list_head namespaces;
+ struct list_head ccr_list;
struct mutex namespaces_lock;
struct srcu_struct srcu;
struct device ctrl_device;
@@ -868,6 +876,7 @@ blk_status_t nvme_host_path_error(struct request *req);
bool nvme_cancel_request(struct request *req, void *data);
void nvme_cancel_tagset(struct nvme_ctrl *ctrl);
void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl);
+int nvme_fence_ctrl(struct nvme_ctrl *ctrl);
bool nvme_change_ctrl_state(struct nvme_ctrl *ctrl,
enum nvme_ctrl_state new_state);
int nvme_disable_ctrl(struct nvme_ctrl *ctrl, bool shutdown);
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v4 08/15] nvme: Implement cross-controller reset recovery
2026-03-28 0:43 ` [PATCH v4 08/15] nvme: Implement cross-controller reset recovery Mohamed Khalfella
@ 2026-03-30 10:50 ` Hannes Reinecke
2026-03-31 16:47 ` Mohamed Khalfella
0 siblings, 1 reply; 25+ messages in thread
From: Hannes Reinecke @ 2026-03-30 10:50 UTC (permalink / raw)
To: Mohamed Khalfella, Justin Tee, Naresh Gottumukkala, Paul Ely,
Chaitanya Kulkarni, Jens Axboe, Keith Busch, Sagi Grimberg,
James Smart
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel
On 3/28/26 01:43, Mohamed Khalfella wrote:
> A host that has more than one path connecting to an nvme subsystem
> typically has an nvme controller associated with every path. This is
> mostly applicable to nvmeof. If one path goes down, inflight IOs on that
> path should not be retried immediately on another path because this
> could lead to data corruption as described in TP4129. TP8028 defines
> cross-controller reset mechanism that can be used by host to terminate
> IOs on the failed path using one of the remaining healthy paths. Only
> after IOs are terminated, or long enough time passes as defined by
> TP4129, inflight IOs should be retried on another path. Implement core
> cross-controller reset shared logic to be used by the transports.
>
> Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
> ---
> drivers/nvme/host/constants.c | 1 +
> drivers/nvme/host/core.c | 145 ++++++++++++++++++++++++++++++++++
> drivers/nvme/host/nvme.h | 9 +++
> 3 files changed, 155 insertions(+)
>
> diff --git a/drivers/nvme/host/constants.c b/drivers/nvme/host/constants.c
> index dc90df9e13a2..f679efd5110e 100644
> --- a/drivers/nvme/host/constants.c
> +++ b/drivers/nvme/host/constants.c
> @@ -46,6 +46,7 @@ static const char * const nvme_admin_ops[] = {
> [nvme_admin_virtual_mgmt] = "Virtual Management",
> [nvme_admin_nvme_mi_send] = "NVMe Send MI",
> [nvme_admin_nvme_mi_recv] = "NVMe Receive MI",
> + [nvme_admin_cross_ctrl_reset] = "Cross Controller Reset",
> [nvme_admin_dbbuf] = "Doorbell Buffer Config",
> [nvme_admin_format_nvm] = "Format NVM",
> [nvme_admin_security_send] = "Security Send",
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 824a1193bec8..5603ae36444f 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -554,6 +554,150 @@ void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl)
> }
> EXPORT_SYMBOL_GPL(nvme_cancel_admin_tagset);
>
> +static struct nvme_ctrl *nvme_find_ctrl_ccr(struct nvme_ctrl *ictrl,
> + u32 min_cntlid)
> +{
> + struct nvme_subsystem *subsys = ictrl->subsys;
> + struct nvme_ctrl *ctrl, *sctrl = NULL;
> + unsigned long flags;
> +
> + mutex_lock(&nvme_subsystems_lock);
> + list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
> + if (ctrl->cntlid < min_cntlid)
> + continue;
> +
> + if (atomic_dec_if_positive(&ctrl->ccr_limit) < 0)
> + continue;
> +
> + spin_lock_irqsave(&ctrl->lock, flags);
> + if (ctrl->state != NVME_CTRL_LIVE) {
> + spin_unlock_irqrestore(&ctrl->lock, flags);
> + atomic_inc(&ctrl->ccr_limit);
> + continue;
> + }
> +
> + /*
> + * We got a good candidate source controller that is locked and
> + * LIVE. However, no guarantee ctrl will not be deleted after
> + * ctrl->lock is released. Get a ref of both ctrl and admin_q
> + * so they do not disappear until we are done with them.
> + */
> + WARN_ON_ONCE(!blk_get_queue(ctrl->admin_q));
> + nvme_get_ctrl(ctrl);
> + spin_unlock_irqrestore(&ctrl->lock, flags);
> + sctrl = ctrl;
> + break;
> + }
> + mutex_unlock(&nvme_subsystems_lock);
> + return sctrl;
> +}
> +
> +static void nvme_put_ctrl_ccr(struct nvme_ctrl *sctrl)
> +{
> + atomic_inc(&sctrl->ccr_limit);
> + blk_put_queue(sctrl->admin_q);
> + nvme_put_ctrl(sctrl);
> +}
> +
> +static int nvme_issue_wait_ccr(struct nvme_ctrl *sctrl, struct nvme_ctrl *ictrl,
> + unsigned long deadline)
> +{
> + struct nvme_ccr_entry ccr = { };
> + union nvme_result res = { 0 };
> + struct nvme_command c = { };
> + unsigned long flags, now, tmo = 0;
> + bool completed = false;
> + int ret = 0;
> + u32 result;
> +
> + init_completion(&ccr.complete);
> + ccr.ictrl = ictrl;
> +
> + spin_lock_irqsave(&sctrl->lock, flags);
> + list_add_tail(&ccr.list, &sctrl->ccr_list);
> + spin_unlock_irqrestore(&sctrl->lock, flags);
> +
> + c.ccr.opcode = nvme_admin_cross_ctrl_reset;
> + c.ccr.ciu = ictrl->ciu;
> + c.ccr.icid = cpu_to_le16(ictrl->cntlid);
> + c.ccr.cirn = cpu_to_le64(ictrl->cirn);
> + ret = __nvme_submit_sync_cmd(sctrl->admin_q, &c, &res,
> + NULL, 0, NVME_QID_ANY, 0);
> + if (ret) {
> + ret = -EIO;
> + goto out;
> + }
> +
> + result = le32_to_cpu(res.u32);
> + if (result & 0x01) /* Immediate Reset Successful */
> + goto out;
> +
> + now = jiffies;
> + if (time_before(now, deadline))
> + tmo = min_t(unsigned long,
> + secs_to_jiffies(ictrl->kato), deadline - now);
> +
> + if (!wait_for_completion_timeout(&ccr.complete, tmo)) {
> + ret = -ETIMEDOUT;
> + goto out;
> + }
> +
> + completed = true;
> +
> +out:
> + spin_lock_irqsave(&sctrl->lock, flags);
> + list_del(&ccr.list);
> + spin_unlock_irqrestore(&sctrl->lock, flags);
> + if (completed) {
> + if (ccr.ccrs == NVME_CCR_STATUS_SUCCESS)
> + return 0;
> + return -EREMOTEIO;
> + }
> + return ret;
> +}
> +
> +int nvme_fence_ctrl(struct nvme_ctrl *ictrl)
> +{
> + unsigned long deadline, timeout;
> + struct nvme_ctrl *sctrl;
> + u32 min_cntlid = 0;
> + int ret;
> +
> + timeout = nvme_fence_timeout_ms(ictrl);
> + dev_info(ictrl->device, "attempting CCR, timeout %lums\n", timeout);
> +
> + deadline = jiffies + msecs_to_jiffies(timeout);
> + while (time_is_after_jiffies(deadline)) {
> + sctrl = nvme_find_ctrl_ccr(ictrl, min_cntlid);
> + if (!sctrl) {
> + dev_dbg(ictrl->device,
> + "failed to find source controller\n");
> + return -EIO;
> + }
> +
> + ret = nvme_issue_wait_ccr(sctrl, ictrl, deadline);
> + if (!ret) {
> + dev_info(ictrl->device, "CCR succeeded using %s\n",
> + dev_name(sctrl->device));
> + nvme_put_ctrl_ccr(sctrl);
> + return 0;
> + }
> +
> + min_cntlid = sctrl->cntlid + 1;
> + nvme_put_ctrl_ccr(sctrl);
> +
> + if (ret == -EIO) /* CCR command failed */
> + continue;
> +
> + /* CCR operation failed or timed out */
> + return ret;
> + }
> +
> + dev_info(ictrl->device, "CCR operation timeout\n");
> + return -ETIMEDOUT;
> +}
Please restructure the loop.
Having a comment 'CCR operation failed or timed out',
returning a status, and then have a comment
'CCR operation timeout' _after_ the return is confusing.
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v4 08/15] nvme: Implement cross-controller reset recovery
2026-03-30 10:50 ` Hannes Reinecke
@ 2026-03-31 16:47 ` Mohamed Khalfella
0 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-31 16:47 UTC (permalink / raw)
To: Hannes Reinecke
Cc: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart, Aaron Dailey,
Randy Jennings, Dhaval Giani, linux-nvme, linux-kernel
On Mon 2026-03-30 12:50:24 +0200, Hannes Reinecke wrote:
> On 3/28/26 01:43, Mohamed Khalfella wrote:
> > A host that has more than one path connecting to an nvme subsystem
> > typically has an nvme controller associated with every path. This is
> > mostly applicable to nvmeof. If one path goes down, inflight IOs on that
> > path should not be retried immediately on another path because this
> > could lead to data corruption as described in TP4129. TP8028 defines
> > cross-controller reset mechanism that can be used by host to terminate
> > IOs on the failed path using one of the remaining healthy paths. Only
> > after IOs are terminated, or long enough time passes as defined by
> > TP4129, inflight IOs should be retried on another path. Implement core
> > cross-controller reset shared logic to be used by the transports.
> >
> > Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
> > ---
> > drivers/nvme/host/constants.c | 1 +
> > drivers/nvme/host/core.c | 145 ++++++++++++++++++++++++++++++++++
> > drivers/nvme/host/nvme.h | 9 +++
> > 3 files changed, 155 insertions(+)
> >
> > diff --git a/drivers/nvme/host/constants.c b/drivers/nvme/host/constants.c
> > index dc90df9e13a2..f679efd5110e 100644
> > --- a/drivers/nvme/host/constants.c
> > +++ b/drivers/nvme/host/constants.c
> > @@ -46,6 +46,7 @@ static const char * const nvme_admin_ops[] = {
> > [nvme_admin_virtual_mgmt] = "Virtual Management",
> > [nvme_admin_nvme_mi_send] = "NVMe Send MI",
> > [nvme_admin_nvme_mi_recv] = "NVMe Receive MI",
> > + [nvme_admin_cross_ctrl_reset] = "Cross Controller Reset",
> > [nvme_admin_dbbuf] = "Doorbell Buffer Config",
> > [nvme_admin_format_nvm] = "Format NVM",
> > [nvme_admin_security_send] = "Security Send",
> > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> > index 824a1193bec8..5603ae36444f 100644
> > --- a/drivers/nvme/host/core.c
> > +++ b/drivers/nvme/host/core.c
> > @@ -554,6 +554,150 @@ void nvme_cancel_admin_tagset(struct nvme_ctrl *ctrl)
> > }
> > EXPORT_SYMBOL_GPL(nvme_cancel_admin_tagset);
> >
> > +static struct nvme_ctrl *nvme_find_ctrl_ccr(struct nvme_ctrl *ictrl,
> > + u32 min_cntlid)
> > +{
> > + struct nvme_subsystem *subsys = ictrl->subsys;
> > + struct nvme_ctrl *ctrl, *sctrl = NULL;
> > + unsigned long flags;
> > +
> > + mutex_lock(&nvme_subsystems_lock);
> > + list_for_each_entry(ctrl, &subsys->ctrls, subsys_entry) {
> > + if (ctrl->cntlid < min_cntlid)
> > + continue;
> > +
> > + if (atomic_dec_if_positive(&ctrl->ccr_limit) < 0)
> > + continue;
> > +
> > + spin_lock_irqsave(&ctrl->lock, flags);
> > + if (ctrl->state != NVME_CTRL_LIVE) {
> > + spin_unlock_irqrestore(&ctrl->lock, flags);
> > + atomic_inc(&ctrl->ccr_limit);
> > + continue;
> > + }
> > +
> > + /*
> > + * We got a good candidate source controller that is locked and
> > + * LIVE. However, no guarantee ctrl will not be deleted after
> > + * ctrl->lock is released. Get a ref of both ctrl and admin_q
> > + * so they do not disappear until we are done with them.
> > + */
> > + WARN_ON_ONCE(!blk_get_queue(ctrl->admin_q));
> > + nvme_get_ctrl(ctrl);
> > + spin_unlock_irqrestore(&ctrl->lock, flags);
> > + sctrl = ctrl;
> > + break;
> > + }
> > + mutex_unlock(&nvme_subsystems_lock);
> > + return sctrl;
> > +}
> > +
> > +static void nvme_put_ctrl_ccr(struct nvme_ctrl *sctrl)
> > +{
> > + atomic_inc(&sctrl->ccr_limit);
> > + blk_put_queue(sctrl->admin_q);
> > + nvme_put_ctrl(sctrl);
> > +}
> > +
> > +static int nvme_issue_wait_ccr(struct nvme_ctrl *sctrl, struct nvme_ctrl *ictrl,
> > + unsigned long deadline)
> > +{
> > + struct nvme_ccr_entry ccr = { };
> > + union nvme_result res = { 0 };
> > + struct nvme_command c = { };
> > + unsigned long flags, now, tmo = 0;
> > + bool completed = false;
> > + int ret = 0;
> > + u32 result;
> > +
> > + init_completion(&ccr.complete);
> > + ccr.ictrl = ictrl;
> > +
> > + spin_lock_irqsave(&sctrl->lock, flags);
> > + list_add_tail(&ccr.list, &sctrl->ccr_list);
> > + spin_unlock_irqrestore(&sctrl->lock, flags);
> > +
> > + c.ccr.opcode = nvme_admin_cross_ctrl_reset;
> > + c.ccr.ciu = ictrl->ciu;
> > + c.ccr.icid = cpu_to_le16(ictrl->cntlid);
> > + c.ccr.cirn = cpu_to_le64(ictrl->cirn);
> > + ret = __nvme_submit_sync_cmd(sctrl->admin_q, &c, &res,
> > + NULL, 0, NVME_QID_ANY, 0);
> > + if (ret) {
> > + ret = -EIO;
> > + goto out;
> > + }
> > +
> > + result = le32_to_cpu(res.u32);
> > + if (result & 0x01) /* Immediate Reset Successful */
> > + goto out;
> > +
> > + now = jiffies;
> > + if (time_before(now, deadline))
> > + tmo = min_t(unsigned long,
> > + secs_to_jiffies(ictrl->kato), deadline - now);
> > +
> > + if (!wait_for_completion_timeout(&ccr.complete, tmo)) {
> > + ret = -ETIMEDOUT;
> > + goto out;
> > + }
> > +
> > + completed = true;
> > +
> > +out:
> > + spin_lock_irqsave(&sctrl->lock, flags);
> > + list_del(&ccr.list);
> > + spin_unlock_irqrestore(&sctrl->lock, flags);
> > + if (completed) {
> > + if (ccr.ccrs == NVME_CCR_STATUS_SUCCESS)
> > + return 0;
> > + return -EREMOTEIO;
> > + }
> > + return ret;
> > +}
> > +
> > +int nvme_fence_ctrl(struct nvme_ctrl *ictrl)
> > +{
> > + unsigned long deadline, timeout;
> > + struct nvme_ctrl *sctrl;
> > + u32 min_cntlid = 0;
> > + int ret;
> > +
> > + timeout = nvme_fence_timeout_ms(ictrl);
> > + dev_info(ictrl->device, "attempting CCR, timeout %lums\n", timeout);
> > +
> > + deadline = jiffies + msecs_to_jiffies(timeout);
> > + while (time_is_after_jiffies(deadline)) {
> > + sctrl = nvme_find_ctrl_ccr(ictrl, min_cntlid);
> > + if (!sctrl) {
> > + dev_dbg(ictrl->device,
> > + "failed to find source controller\n");
> > + return -EIO;
> > + }
> > +
> > + ret = nvme_issue_wait_ccr(sctrl, ictrl, deadline);
> > + if (!ret) {
> > + dev_info(ictrl->device, "CCR succeeded using %s\n",
> > + dev_name(sctrl->device));
> > + nvme_put_ctrl_ccr(sctrl);
> > + return 0;
> > + }
> > +
> > + min_cntlid = sctrl->cntlid + 1;
> > + nvme_put_ctrl_ccr(sctrl);
> > +
> > + if (ret == -EIO) /* CCR command failed */
> > + continue;
> > +
> > + /* CCR operation failed or timed out */
> > + return ret;
> > + }
> > +
> > + dev_info(ictrl->device, "CCR operation timeout\n");
> > + return -ETIMEDOUT;
> > +}
>
> Please restructure the loop.
> Having a comment 'CCR operation failed or timed out',
> returning a status, and then have a comment
> 'CCR operation timeout' _after_ the return is confusing.
I can change /* CCR operation failed or timed out */ to something like
/*
* Source controller accepted CCR command but CCR operation
* timed out or failed. Retrying another path is not likely
* to succeed, return an error.
*/
And change the log line "CCR operation timeout\n" outside the while
loop to "fencing timedout\n".
Will this help?
>
> Cheers,
>
> Hannes
> --
> Dr. Hannes Reinecke Kernel Storage Architect
> hare@suse.de +49 911 74053 688
> SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
> HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 09/15] nvme: Implement cross-controller reset completion
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (7 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 08/15] nvme: Implement cross-controller reset recovery Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-30 10:53 ` Hannes Reinecke
2026-03-28 0:43 ` [PATCH v4 10/15] nvme-tcp: Use CCR to recover controller that hits an error Mohamed Khalfella
` (5 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
An nvme source controller that issues CCR command expects to receive an
NVME_AER_NOTICE_CCR_COMPLETED when pending CCR succeeds or fails. Add
sctrl->ccr_work to read NVME_LOG_CCR logpage and wakeup any thread
waiting on CCR completion.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
---
drivers/nvme/host/core.c | 49 +++++++++++++++++++++++++++++++++++++++-
drivers/nvme/host/nvme.h | 1 +
2 files changed, 49 insertions(+), 1 deletion(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 5603ae36444f..793f203bfc38 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1920,7 +1920,8 @@ EXPORT_SYMBOL_GPL(nvme_set_queue_count);
#define NVME_AEN_SUPPORTED \
(NVME_AEN_CFG_NS_ATTR | NVME_AEN_CFG_FW_ACT | \
- NVME_AEN_CFG_ANA_CHANGE | NVME_AEN_CFG_DISC_CHANGE)
+ NVME_AEN_CFG_ANA_CHANGE | NVME_AEN_CFG_CCR_COMPLETE | \
+ NVME_AEN_CFG_DISC_CHANGE)
static void nvme_enable_aen(struct nvme_ctrl *ctrl)
{
@@ -4873,6 +4874,47 @@ static void nvme_get_fw_slot_info(struct nvme_ctrl *ctrl)
kfree(log);
}
+static void nvme_ccr_work(struct work_struct *work)
+{
+ struct nvme_ctrl *ctrl = container_of(work, struct nvme_ctrl, ccr_work);
+ struct nvme_ccr_entry *ccr;
+ struct nvme_ccr_log_entry *entry;
+ struct nvme_ccr_log *log;
+ unsigned long flags;
+ int ret, i;
+
+ log = kmalloc(sizeof(*log), GFP_KERNEL);
+ if (!log)
+ return;
+
+ ret = nvme_get_log(ctrl, 0, NVME_LOG_CCR, 0x01,
+ 0x00, log, sizeof(*log), 0);
+ if (ret)
+ goto out;
+
+ spin_lock_irqsave(&ctrl->lock, flags);
+ for (i = 0; i < le16_to_cpu(log->ne); i++) {
+ entry = &log->entries[i];
+ if (entry->ccrs == NVME_CCR_STATUS_IN_PROGRESS)
+ continue;
+
+ list_for_each_entry(ccr, &ctrl->ccr_list, list) {
+ struct nvme_ctrl *ictrl = ccr->ictrl;
+
+ if (ictrl->cntlid != le16_to_cpu(entry->icid) ||
+ ictrl->ciu != entry->ciu)
+ continue;
+
+ /* Complete matching entry */
+ ccr->ccrs = entry->ccrs;
+ complete(&ccr->complete);
+ }
+ }
+ spin_unlock_irqrestore(&ctrl->lock, flags);
+out:
+ kfree(log);
+}
+
static void nvme_fw_act_work(struct work_struct *work)
{
struct nvme_ctrl *ctrl = container_of(work,
@@ -4949,6 +4991,9 @@ static bool nvme_handle_aen_notice(struct nvme_ctrl *ctrl, u32 result)
case NVME_AER_NOTICE_DISC_CHANGED:
ctrl->aen_result = result;
break;
+ case NVME_AER_NOTICE_CCR_COMPLETED:
+ queue_work(nvme_wq, &ctrl->ccr_work);
+ break;
default:
dev_warn(ctrl->device, "async event result %08x\n", result);
}
@@ -5144,6 +5189,7 @@ void nvme_stop_ctrl(struct nvme_ctrl *ctrl)
nvme_stop_failfast_work(ctrl);
flush_work(&ctrl->async_event_work);
cancel_work_sync(&ctrl->fw_act_work);
+ cancel_work_sync(&ctrl->ccr_work);
if (ctrl->ops->stop_ctrl)
ctrl->ops->stop_ctrl(ctrl);
}
@@ -5267,6 +5313,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
ctrl->quirks = quirks;
ctrl->numa_node = NUMA_NO_NODE;
INIT_WORK(&ctrl->scan_work, nvme_scan_work);
+ INIT_WORK(&ctrl->ccr_work, nvme_ccr_work);
INIT_WORK(&ctrl->async_event_work, nvme_async_event_work);
INIT_WORK(&ctrl->fw_act_work, nvme_fw_act_work);
INIT_WORK(&ctrl->delete_work, nvme_delete_ctrl_work);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index f2bcff9ccd25..776ee8aa5a93 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -419,6 +419,7 @@ struct nvme_ctrl {
struct nvme_effects_log *effects;
struct xarray cels;
struct work_struct scan_work;
+ struct work_struct ccr_work;
struct work_struct async_event_work;
struct delayed_work ka_work;
struct delayed_work failfast_work;
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v4 09/15] nvme: Implement cross-controller reset completion
2026-03-28 0:43 ` [PATCH v4 09/15] nvme: Implement cross-controller reset completion Mohamed Khalfella
@ 2026-03-30 10:53 ` Hannes Reinecke
2026-03-31 16:55 ` Mohamed Khalfella
0 siblings, 1 reply; 25+ messages in thread
From: Hannes Reinecke @ 2026-03-30 10:53 UTC (permalink / raw)
To: Mohamed Khalfella, Justin Tee, Naresh Gottumukkala, Paul Ely,
Chaitanya Kulkarni, Jens Axboe, Keith Busch, Sagi Grimberg,
James Smart
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel
On 3/28/26 01:43, Mohamed Khalfella wrote:
> An nvme source controller that issues CCR command expects to receive an
> NVME_AER_NOTICE_CCR_COMPLETED when pending CCR succeeds or fails. Add
> sctrl->ccr_work to read NVME_LOG_CCR logpage and wakeup any thread
> waiting on CCR completion.
>
> Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
> ---
> drivers/nvme/host/core.c | 49 +++++++++++++++++++++++++++++++++++++++-
> drivers/nvme/host/nvme.h | 1 +
> 2 files changed, 49 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 5603ae36444f..793f203bfc38 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -1920,7 +1920,8 @@ EXPORT_SYMBOL_GPL(nvme_set_queue_count);
>
> #define NVME_AEN_SUPPORTED \
> (NVME_AEN_CFG_NS_ATTR | NVME_AEN_CFG_FW_ACT | \
> - NVME_AEN_CFG_ANA_CHANGE | NVME_AEN_CFG_DISC_CHANGE)
> + NVME_AEN_CFG_ANA_CHANGE | NVME_AEN_CFG_CCR_COMPLETE | \
> + NVME_AEN_CFG_DISC_CHANGE)
>
> static void nvme_enable_aen(struct nvme_ctrl *ctrl)
> {
> @@ -4873,6 +4874,47 @@ static void nvme_get_fw_slot_info(struct nvme_ctrl *ctrl)
> kfree(log);
> }
>
> +static void nvme_ccr_work(struct work_struct *work)
> +{
> + struct nvme_ctrl *ctrl = container_of(work, struct nvme_ctrl, ccr_work);
> + struct nvme_ccr_entry *ccr;
> + struct nvme_ccr_log_entry *entry;
> + struct nvme_ccr_log *log;
> + unsigned long flags;
> + int ret, i;
> +
> + log = kmalloc(sizeof(*log), GFP_KERNEL);
> + if (!log)
> + return;
> +
> + ret = nvme_get_log(ctrl, 0, NVME_LOG_CCR, 0x01,
> + 0x00, log, sizeof(*log), 0);
> + if (ret)
> + goto out;
> +
> + spin_lock_irqsave(&ctrl->lock, flags);
> + for (i = 0; i < le16_to_cpu(log->ne); i++) {
> + entry = &log->entries[i];
> + if (entry->ccrs == NVME_CCR_STATUS_IN_PROGRESS)
> + continue;
> +
> + list_for_each_entry(ccr, &ctrl->ccr_list, list) {
> + struct nvme_ctrl *ictrl = ccr->ictrl;
> +
> + if (ictrl->cntlid != le16_to_cpu(entry->icid) ||
> + ictrl->ciu != entry->ciu)
> + continue;
> +
> + /* Complete matching entry */
> + ccr->ccrs = entry->ccrs;
> + complete(&ccr->complete);
> + }
> + }
> + spin_unlock_irqrestore(&ctrl->lock, flags);
> +out:
> + kfree(log);
> +}
> +
> static void nvme_fw_act_work(struct work_struct *work)
> {
> struct nvme_ctrl *ctrl = container_of(work,
> @@ -4949,6 +4991,9 @@ static bool nvme_handle_aen_notice(struct nvme_ctrl *ctrl, u32 result)
> case NVME_AER_NOTICE_DISC_CHANGED:
> ctrl->aen_result = result;
> break;
> + case NVME_AER_NOTICE_CCR_COMPLETED:
> + queue_work(nvme_wq, &ctrl->ccr_work);
> + break;
> default:
> dev_warn(ctrl->device, "async event result %08x\n", result);
> }
> @@ -5144,6 +5189,7 @@ void nvme_stop_ctrl(struct nvme_ctrl *ctrl)
> nvme_stop_failfast_work(ctrl);
> flush_work(&ctrl->async_event_work);
> cancel_work_sync(&ctrl->fw_act_work);
> + cancel_work_sync(&ctrl->ccr_work);
> if (ctrl->ops->stop_ctrl)
> ctrl->ops->stop_ctrl(ctrl);
> }
> @@ -5267,6 +5313,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
> ctrl->quirks = quirks;
> ctrl->numa_node = NUMA_NO_NODE;
> INIT_WORK(&ctrl->scan_work, nvme_scan_work);
> + INIT_WORK(&ctrl->ccr_work, nvme_ccr_work);
> INIT_WORK(&ctrl->async_event_work, nvme_async_event_work);
> INIT_WORK(&ctrl->fw_act_work, nvme_fw_act_work);
> INIT_WORK(&ctrl->delete_work, nvme_delete_ctrl_work);
> diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> index f2bcff9ccd25..776ee8aa5a93 100644
> --- a/drivers/nvme/host/nvme.h
> +++ b/drivers/nvme/host/nvme.h
> @@ -419,6 +419,7 @@ struct nvme_ctrl {
> struct nvme_effects_log *effects;
> struct xarray cels;
> struct work_struct scan_work;
> + struct work_struct ccr_work;
> struct work_struct async_event_work;
> struct delayed_work ka_work;
> struct delayed_work failfast_work;
Hmm. The 'nvme_fence_ctrl' operation introduced in the previous patch
is synchronous, yet in this patch we're looking a a log page to figure
out if the cross-controller reset is complete.
Which is slightly irritating.
Wouldn't it be better to make the 'nvme_fence_ctrl' operation
asynchronous, and then have a separate function to wait for the fence
operation to complete (which then could look at log pages etc)?
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v4 09/15] nvme: Implement cross-controller reset completion
2026-03-30 10:53 ` Hannes Reinecke
@ 2026-03-31 16:55 ` Mohamed Khalfella
0 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-31 16:55 UTC (permalink / raw)
To: Hannes Reinecke
Cc: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart, Aaron Dailey,
Randy Jennings, Dhaval Giani, linux-nvme, linux-kernel
On Mon 2026-03-30 12:53:07 +0200, Hannes Reinecke wrote:
> On 3/28/26 01:43, Mohamed Khalfella wrote:
> > An nvme source controller that issues CCR command expects to receive an
> > NVME_AER_NOTICE_CCR_COMPLETED when pending CCR succeeds or fails. Add
> > sctrl->ccr_work to read NVME_LOG_CCR logpage and wakeup any thread
> > waiting on CCR completion.
> >
> > Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
> > ---
> > drivers/nvme/host/core.c | 49 +++++++++++++++++++++++++++++++++++++++-
> > drivers/nvme/host/nvme.h | 1 +
> > 2 files changed, 49 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> > index 5603ae36444f..793f203bfc38 100644
> > --- a/drivers/nvme/host/core.c
> > +++ b/drivers/nvme/host/core.c
> > @@ -1920,7 +1920,8 @@ EXPORT_SYMBOL_GPL(nvme_set_queue_count);
> >
> > #define NVME_AEN_SUPPORTED \
> > (NVME_AEN_CFG_NS_ATTR | NVME_AEN_CFG_FW_ACT | \
> > - NVME_AEN_CFG_ANA_CHANGE | NVME_AEN_CFG_DISC_CHANGE)
> > + NVME_AEN_CFG_ANA_CHANGE | NVME_AEN_CFG_CCR_COMPLETE | \
> > + NVME_AEN_CFG_DISC_CHANGE)
> >
> > static void nvme_enable_aen(struct nvme_ctrl *ctrl)
> > {
> > @@ -4873,6 +4874,47 @@ static void nvme_get_fw_slot_info(struct nvme_ctrl *ctrl)
> > kfree(log);
> > }
> >
> > +static void nvme_ccr_work(struct work_struct *work)
> > +{
> > + struct nvme_ctrl *ctrl = container_of(work, struct nvme_ctrl, ccr_work);
> > + struct nvme_ccr_entry *ccr;
> > + struct nvme_ccr_log_entry *entry;
> > + struct nvme_ccr_log *log;
> > + unsigned long flags;
> > + int ret, i;
> > +
> > + log = kmalloc(sizeof(*log), GFP_KERNEL);
> > + if (!log)
> > + return;
> > +
> > + ret = nvme_get_log(ctrl, 0, NVME_LOG_CCR, 0x01,
> > + 0x00, log, sizeof(*log), 0);
> > + if (ret)
> > + goto out;
> > +
> > + spin_lock_irqsave(&ctrl->lock, flags);
> > + for (i = 0; i < le16_to_cpu(log->ne); i++) {
> > + entry = &log->entries[i];
> > + if (entry->ccrs == NVME_CCR_STATUS_IN_PROGRESS)
> > + continue;
> > +
> > + list_for_each_entry(ccr, &ctrl->ccr_list, list) {
> > + struct nvme_ctrl *ictrl = ccr->ictrl;
> > +
> > + if (ictrl->cntlid != le16_to_cpu(entry->icid) ||
> > + ictrl->ciu != entry->ciu)
> > + continue;
> > +
> > + /* Complete matching entry */
> > + ccr->ccrs = entry->ccrs;
> > + complete(&ccr->complete);
> > + }
> > + }
> > + spin_unlock_irqrestore(&ctrl->lock, flags);
> > +out:
> > + kfree(log);
> > +}
> > +
> > static void nvme_fw_act_work(struct work_struct *work)
> > {
> > struct nvme_ctrl *ctrl = container_of(work,
> > @@ -4949,6 +4991,9 @@ static bool nvme_handle_aen_notice(struct nvme_ctrl *ctrl, u32 result)
> > case NVME_AER_NOTICE_DISC_CHANGED:
> > ctrl->aen_result = result;
> > break;
> > + case NVME_AER_NOTICE_CCR_COMPLETED:
> > + queue_work(nvme_wq, &ctrl->ccr_work);
> > + break;
> > default:
> > dev_warn(ctrl->device, "async event result %08x\n", result);
> > }
> > @@ -5144,6 +5189,7 @@ void nvme_stop_ctrl(struct nvme_ctrl *ctrl)
> > nvme_stop_failfast_work(ctrl);
> > flush_work(&ctrl->async_event_work);
> > cancel_work_sync(&ctrl->fw_act_work);
> > + cancel_work_sync(&ctrl->ccr_work);
> > if (ctrl->ops->stop_ctrl)
> > ctrl->ops->stop_ctrl(ctrl);
> > }
> > @@ -5267,6 +5313,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
> > ctrl->quirks = quirks;
> > ctrl->numa_node = NUMA_NO_NODE;
> > INIT_WORK(&ctrl->scan_work, nvme_scan_work);
> > + INIT_WORK(&ctrl->ccr_work, nvme_ccr_work);
> > INIT_WORK(&ctrl->async_event_work, nvme_async_event_work);
> > INIT_WORK(&ctrl->fw_act_work, nvme_fw_act_work);
> > INIT_WORK(&ctrl->delete_work, nvme_delete_ctrl_work);
> > diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
> > index f2bcff9ccd25..776ee8aa5a93 100644
> > --- a/drivers/nvme/host/nvme.h
> > +++ b/drivers/nvme/host/nvme.h
> > @@ -419,6 +419,7 @@ struct nvme_ctrl {
> > struct nvme_effects_log *effects;
> > struct xarray cels;
> > struct work_struct scan_work;
> > + struct work_struct ccr_work;
> > struct work_struct async_event_work;
> > struct delayed_work ka_work;
> > struct delayed_work failfast_work;
>
> Hmm. The 'nvme_fence_ctrl' operation introduced in the previous patch
> is synchronous, yet in this patch we're looking a a log page to figure
> out if the cross-controller reset is complete.
> Which is slightly irritating.
> Wouldn't it be better to make the 'nvme_fence_ctrl' operation
> asynchronous, and then have a separate function to wait for the fence
> operation to complete (which then could look at log pages etc)?
True nvme_fence_ctrl() is synchronous, but it runs in from ctrl->fencing_work.
What is it that you find irritating about nvme_fence_ctrl()?
>
> Cheers,
>
> Hannes
> --
> Dr. Hannes Reinecke Kernel Storage Architect
> hare@suse.de +49 911 74053 688
> SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
> HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 10/15] nvme-tcp: Use CCR to recover controller that hits an error
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (8 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 09/15] nvme: Implement cross-controller reset completion Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-30 11:00 ` Hannes Reinecke
2026-03-28 0:43 ` [PATCH v4 11/15] nvme-rdma: " Mohamed Khalfella
` (4 subsequent siblings)
14 siblings, 1 reply; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
An alive nvme controller that hits an error now will move to FENCING
state instead of RESETTING state. ctrl->fencing_work attempts CCR to
terminate inflight IOs. Regardless of the success or failure of CCR
operation the controller is transitioned to RESETTING state to continue
error recovery process.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
---
drivers/nvme/host/tcp.c | 30 +++++++++++++++++++++++++++++-
1 file changed, 29 insertions(+), 1 deletion(-)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 243dab830dc8..6393ec2b3b55 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -194,6 +194,7 @@ struct nvme_tcp_ctrl {
struct sockaddr_storage src_addr;
struct nvme_ctrl ctrl;
+ struct work_struct fencing_work;
struct work_struct err_work;
struct delayed_work connect_work;
struct nvme_tcp_request async_req;
@@ -612,6 +613,12 @@ static void nvme_tcp_init_recv_ctx(struct nvme_tcp_queue *queue)
static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
{
+ if (nvme_change_ctrl_state(ctrl, NVME_CTRL_FENCING)) {
+ dev_warn(ctrl->device, "starting controller fencing\n");
+ queue_work(nvme_wq, &to_tcp_ctrl(ctrl)->fencing_work);
+ return;
+ }
+
if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_RESETTING))
return;
@@ -2471,12 +2478,29 @@ static void nvme_tcp_reconnect_ctrl_work(struct work_struct *work)
nvme_tcp_reconnect_or_remove(ctrl, ret);
}
+static void nvme_tcp_fencing_work(struct work_struct *work)
+{
+ struct nvme_tcp_ctrl *tcp_ctrl = container_of(work,
+ struct nvme_tcp_ctrl, fencing_work);
+ struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
+ int ret;
+
+ ret = nvme_fence_ctrl(ctrl);
+ if (ret)
+ dev_info(ctrl->device, "CCR failed with error %d\n", ret);
+
+ nvme_change_ctrl_state(ctrl, NVME_CTRL_FENCED);
+ if (nvme_change_ctrl_state(ctrl, NVME_CTRL_RESETTING))
+ queue_work(nvme_reset_wq, &tcp_ctrl->err_work);
+}
+
static void nvme_tcp_error_recovery_work(struct work_struct *work)
{
struct nvme_tcp_ctrl *tcp_ctrl = container_of(work,
struct nvme_tcp_ctrl, err_work);
struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
+ flush_work(&to_tcp_ctrl(ctrl)->fencing_work);
if (nvme_tcp_key_revoke_needed(ctrl))
nvme_auth_revoke_tls_key(ctrl);
nvme_stop_keep_alive(ctrl);
@@ -2519,6 +2543,7 @@ static void nvme_reset_ctrl_work(struct work_struct *work)
container_of(work, struct nvme_ctrl, reset_work);
int ret;
+ flush_work(&to_tcp_ctrl(ctrl)->fencing_work);
if (nvme_tcp_key_revoke_needed(ctrl))
nvme_auth_revoke_tls_key(ctrl);
nvme_stop_ctrl(ctrl);
@@ -2644,13 +2669,15 @@ static enum blk_eh_timer_return nvme_tcp_timeout(struct request *rq)
struct nvme_tcp_cmd_pdu *pdu = nvme_tcp_req_cmd_pdu(req);
struct nvme_command *cmd = &pdu->cmd;
int qid = nvme_tcp_queue_id(req->queue);
+ enum nvme_ctrl_state state;
dev_warn(ctrl->device,
"I/O tag %d (%04x) type %d opcode %#x (%s) QID %d timeout\n",
rq->tag, nvme_cid(rq), pdu->hdr.type, cmd->common.opcode,
nvme_fabrics_opcode_str(qid, cmd), qid);
- if (nvme_ctrl_state(ctrl) != NVME_CTRL_LIVE) {
+ state = nvme_ctrl_state(ctrl);
+ if (state != NVME_CTRL_LIVE && state != NVME_CTRL_FENCING) {
/*
* If we are resetting, connecting or deleting we should
* complete immediately because we may block controller
@@ -2905,6 +2932,7 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev,
INIT_DELAYED_WORK(&ctrl->connect_work,
nvme_tcp_reconnect_ctrl_work);
+ INIT_WORK(&ctrl->fencing_work, nvme_tcp_fencing_work);
INIT_WORK(&ctrl->err_work, nvme_tcp_error_recovery_work);
INIT_WORK(&ctrl->ctrl.reset_work, nvme_reset_ctrl_work);
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v4 10/15] nvme-tcp: Use CCR to recover controller that hits an error
2026-03-28 0:43 ` [PATCH v4 10/15] nvme-tcp: Use CCR to recover controller that hits an error Mohamed Khalfella
@ 2026-03-30 11:00 ` Hannes Reinecke
0 siblings, 0 replies; 25+ messages in thread
From: Hannes Reinecke @ 2026-03-30 11:00 UTC (permalink / raw)
To: Mohamed Khalfella, Justin Tee, Naresh Gottumukkala, Paul Ely,
Chaitanya Kulkarni, Jens Axboe, Keith Busch, Sagi Grimberg,
James Smart
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel
On 3/28/26 01:43, Mohamed Khalfella wrote:
> An alive nvme controller that hits an error now will move to FENCING
> state instead of RESETTING state. ctrl->fencing_work attempts CCR to
> terminate inflight IOs. Regardless of the success or failure of CCR
> operation the controller is transitioned to RESETTING state to continue
> error recovery process.
>
> Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
> ---
> drivers/nvme/host/tcp.c | 30 +++++++++++++++++++++++++++++-
> 1 file changed, 29 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index 243dab830dc8..6393ec2b3b55 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -194,6 +194,7 @@ struct nvme_tcp_ctrl {
> struct sockaddr_storage src_addr;
> struct nvme_ctrl ctrl;
>
> + struct work_struct fencing_work;
> struct work_struct err_work;
> struct delayed_work connect_work;
> struct nvme_tcp_request async_req;
> @@ -612,6 +613,12 @@ static void nvme_tcp_init_recv_ctx(struct nvme_tcp_queue *queue)
>
> static void nvme_tcp_error_recovery(struct nvme_ctrl *ctrl)
> {
> + if (nvme_change_ctrl_state(ctrl, NVME_CTRL_FENCING)) {
> + dev_warn(ctrl->device, "starting controller fencing\n");
> + queue_work(nvme_wq, &to_tcp_ctrl(ctrl)->fencing_work);
> + return;
> + }
> +
> if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_RESETTING))
> return;
>
> @@ -2471,12 +2478,29 @@ static void nvme_tcp_reconnect_ctrl_work(struct work_struct *work)
> nvme_tcp_reconnect_or_remove(ctrl, ret);
> }
>
> +static void nvme_tcp_fencing_work(struct work_struct *work)
> +{
> + struct nvme_tcp_ctrl *tcp_ctrl = container_of(work,
> + struct nvme_tcp_ctrl, fencing_work);
> + struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
> + int ret;
> +
> + ret = nvme_fence_ctrl(ctrl);
> + if (ret)
> + dev_info(ctrl->device, "CCR failed with error %d\n", ret);
> +
> + nvme_change_ctrl_state(ctrl, NVME_CTRL_FENCED);
> + if (nvme_change_ctrl_state(ctrl, NVME_CTRL_RESETTING))
> + queue_work(nvme_reset_wq, &tcp_ctrl->err_work);
> +}
> +
To follow up to my comments on the previous patches:
As we're already using a work queue, wouldn't it be better / easier
to make 'nvme_fence_ctrl()' asynchronous and explicitely call
'nvme_fence_wait()' or something here?
> static void nvme_tcp_error_recovery_work(struct work_struct *work)
> {
> struct nvme_tcp_ctrl *tcp_ctrl = container_of(work,
> struct nvme_tcp_ctrl, err_work);
> struct nvme_ctrl *ctrl = &tcp_ctrl->ctrl;
>
> + flush_work(&to_tcp_ctrl(ctrl)->fencing_work);
> if (nvme_tcp_key_revoke_needed(ctrl))
> nvme_auth_revoke_tls_key(ctrl);
> nvme_stop_keep_alive(ctrl);
> @@ -2519,6 +2543,7 @@ static void nvme_reset_ctrl_work(struct work_struct *work)
> container_of(work, struct nvme_ctrl, reset_work);
> int ret;
>
> + flush_work(&to_tcp_ctrl(ctrl)->fencing_work);
> if (nvme_tcp_key_revoke_needed(ctrl))
> nvme_auth_revoke_tls_key(ctrl);
> nvme_stop_ctrl(ctrl);
> @@ -2644,13 +2669,15 @@ static enum blk_eh_timer_return nvme_tcp_timeout(struct request *rq)
> struct nvme_tcp_cmd_pdu *pdu = nvme_tcp_req_cmd_pdu(req);
> struct nvme_command *cmd = &pdu->cmd;
> int qid = nvme_tcp_queue_id(req->queue);
> + enum nvme_ctrl_state state;
>
> dev_warn(ctrl->device,
> "I/O tag %d (%04x) type %d opcode %#x (%s) QID %d timeout\n",
> rq->tag, nvme_cid(rq), pdu->hdr.type, cmd->common.opcode,
> nvme_fabrics_opcode_str(qid, cmd), qid);
>
> - if (nvme_ctrl_state(ctrl) != NVME_CTRL_LIVE) {
> + state = nvme_ctrl_state(ctrl);
> + if (state != NVME_CTRL_LIVE && state != NVME_CTRL_FENCING) {
> /*
> * If we are resetting, connecting or deleting we should
> * complete immediately because we may block controller
Do we need to call nvme_tcp_error_recovery() even if the controller is
in 'FENCING'?
Don't we just need to return BLK_EH_RESET_TIMER when in 'FENCING' ?
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v4 11/15] nvme-rdma: Use CCR to recover controller that hits an error
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (9 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 10/15] nvme-tcp: Use CCR to recover controller that hits an error Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 12/15] nvme-fc: Refactor IO error recovery Mohamed Khalfella
` (3 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
An alive nvme controller that hits an error now will move to FENCING
state instead of RESETTING state. ctrl->fencing_work attempts CCR to
terminate inflight IOs. Regardless of the success or failure of CCR
operation the controller is transitioned to RESETTING state to continue
error recovery process.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
---
drivers/nvme/host/rdma.c | 30 +++++++++++++++++++++++++++++-
1 file changed, 29 insertions(+), 1 deletion(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 57111139e84f..b42798781619 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -106,6 +106,7 @@ struct nvme_rdma_ctrl {
/* other member variables */
struct blk_mq_tag_set tag_set;
+ struct work_struct fencing_work;
struct work_struct err_work;
struct nvme_rdma_qe async_event_sqe;
@@ -1120,11 +1121,28 @@ static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
nvme_rdma_reconnect_or_remove(ctrl, ret);
}
+static void nvme_rdma_fencing_work(struct work_struct *work)
+{
+ struct nvme_rdma_ctrl *rdma_ctrl = container_of(work,
+ struct nvme_rdma_ctrl, fencing_work);
+ struct nvme_ctrl *ctrl = &rdma_ctrl->ctrl;
+ int ret;
+
+ ret = nvme_fence_ctrl(ctrl);
+ if (ret)
+ dev_info(ctrl->device, "CCR failed with error %d\n", ret);
+
+ nvme_change_ctrl_state(ctrl, NVME_CTRL_FENCED);
+ if (nvme_change_ctrl_state(ctrl, NVME_CTRL_RESETTING))
+ queue_work(nvme_reset_wq, &rdma_ctrl->err_work);
+}
+
static void nvme_rdma_error_recovery_work(struct work_struct *work)
{
struct nvme_rdma_ctrl *ctrl = container_of(work,
struct nvme_rdma_ctrl, err_work);
+ flush_work(&ctrl->fencing_work);
nvme_stop_keep_alive(&ctrl->ctrl);
flush_work(&ctrl->ctrl.async_event_work);
nvme_rdma_teardown_io_queues(ctrl, false);
@@ -1147,6 +1165,12 @@ static void nvme_rdma_error_recovery_work(struct work_struct *work)
static void nvme_rdma_error_recovery(struct nvme_rdma_ctrl *ctrl)
{
+ if (nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_FENCING)) {
+ dev_warn(ctrl->ctrl.device, "starting controller fencing\n");
+ queue_work(nvme_wq, &ctrl->fencing_work);
+ return;
+ }
+
if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_RESETTING))
return;
@@ -1957,13 +1981,15 @@ static enum blk_eh_timer_return nvme_rdma_timeout(struct request *rq)
struct nvme_rdma_ctrl *ctrl = queue->ctrl;
struct nvme_command *cmd = req->req.cmd;
int qid = nvme_rdma_queue_idx(queue);
+ enum nvme_ctrl_state state;
dev_warn(ctrl->ctrl.device,
"I/O tag %d (%04x) opcode %#x (%s) QID %d timeout\n",
rq->tag, nvme_cid(rq), cmd->common.opcode,
nvme_fabrics_opcode_str(qid, cmd), qid);
- if (nvme_ctrl_state(&ctrl->ctrl) != NVME_CTRL_LIVE) {
+ state = nvme_ctrl_state(&ctrl->ctrl);
+ if (state != NVME_CTRL_LIVE && state != NVME_CTRL_FENCING) {
/*
* If we are resetting, connecting or deleting we should
* complete immediately because we may block controller
@@ -2169,6 +2195,7 @@ static void nvme_rdma_reset_ctrl_work(struct work_struct *work)
container_of(work, struct nvme_rdma_ctrl, ctrl.reset_work);
int ret;
+ flush_work(&ctrl->fencing_work);
nvme_stop_ctrl(&ctrl->ctrl);
nvme_rdma_shutdown_ctrl(ctrl, false);
@@ -2281,6 +2308,7 @@ static struct nvme_rdma_ctrl *nvme_rdma_alloc_ctrl(struct device *dev,
INIT_DELAYED_WORK(&ctrl->reconnect_work,
nvme_rdma_reconnect_ctrl_work);
+ INIT_WORK(&ctrl->fencing_work, nvme_rdma_fencing_work);
INIT_WORK(&ctrl->err_work, nvme_rdma_error_recovery_work);
INIT_WORK(&ctrl->ctrl.reset_work, nvme_rdma_reset_ctrl_work);
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v4 12/15] nvme-fc: Refactor IO error recovery
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (10 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 11/15] nvme-rdma: " Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 13/15] nvme-fc: Use CCR to recover controller that hits an error Mohamed Khalfella
` (2 subsequent siblings)
14 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
Added new nvme_fc_start_ioerr_recovery() to trigger error recovery
instead of directly queueing ctrl->ioerr_work. nvme_fc_error_recovery()
now called only from ctrl->ioerr_work has been updated to not depend on
nvme_reset_ctrl() to handle error recovery. nvme_fc_error_recovery()
effectively resets the controller and attempts reconnection if needed.
This makes nvme-fc ioerr handling similar to other fabric transports.
Update nvme_fc_timeout() to not abort timed out IOs. IOs aborted from
nvme_fc_timeout() are not accounted for in ctrl->iocnt and this causes
nvme_fc_delete_association() not to wait for them. Instead of aborting
IOs nvme_fc_timeout() calls nvme_fc_start_ioerr_recovery() to start IO
error recovery. Since error recovery runs in ctrl->ioerr_work this
change fixes the issue reported in the link below.
Link: https://lore.kernel.org/all/20250529214928.2112990-1-mkhalfella@purestorage.com/
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
---
drivers/nvme/host/fc.c | 119 +++++++++++++++++++++++------------------
1 file changed, 66 insertions(+), 53 deletions(-)
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index e1bb4707183c..6797eb17917f 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -227,6 +227,10 @@ static DEFINE_IDA(nvme_fc_ctrl_cnt);
static struct device *fc_udev_device;
static void nvme_fc_complete_rq(struct request *rq);
+static void nvme_fc_start_ioerr_recovery(struct nvme_fc_ctrl *ctrl,
+ char *errmsg);
+static void __nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl,
+ bool start_queues);
/* *********************** FC-NVME Port Management ************************ */
@@ -788,7 +792,7 @@ nvme_fc_ctrl_connectivity_loss(struct nvme_fc_ctrl *ctrl)
"Reconnect", ctrl->cnum);
set_bit(ASSOC_FAILED, &ctrl->flags);
- nvme_reset_ctrl(&ctrl->ctrl);
+ nvme_fc_start_ioerr_recovery(ctrl, "Connectivity Loss");
}
/**
@@ -985,7 +989,7 @@ fc_dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nents,
static void nvme_fc_ctrl_put(struct nvme_fc_ctrl *);
static int nvme_fc_ctrl_get(struct nvme_fc_ctrl *);
-static void nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg);
+static void nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl);
static void
__nvme_fc_finish_ls_req(struct nvmefc_ls_req_op *lsop)
@@ -1567,9 +1571,8 @@ nvme_fc_ls_disconnect_assoc(struct nvmefc_ls_rcv_op *lsop)
* for the association have been ABTS'd by
* nvme_fc_delete_association().
*/
-
- /* fail the association */
- nvme_fc_error_recovery(ctrl, "Disconnect Association LS received");
+ nvme_fc_start_ioerr_recovery(ctrl,
+ "Disconnect Association LS received");
/* release the reference taken by nvme_fc_match_disconn_ls() */
nvme_fc_ctrl_put(ctrl);
@@ -1871,7 +1874,22 @@ nvme_fc_ctrl_ioerr_work(struct work_struct *work)
struct nvme_fc_ctrl *ctrl =
container_of(work, struct nvme_fc_ctrl, ioerr_work);
- nvme_fc_error_recovery(ctrl, "transport detected io error");
+ /*
+ * if an error (io timeout, etc) while (re)connecting, the remote
+ * port requested terminating of the association (disconnect_ls)
+ * or an error (timeout or abort) occurred on an io while creating
+ * the controller. Abort any ios on the association and let the
+ * create_association error path resolve things.
+ */
+ if (nvme_ctrl_state(&ctrl->ctrl) == NVME_CTRL_CONNECTING) {
+ __nvme_fc_abort_outstanding_ios(ctrl, true);
+ dev_warn(ctrl->ctrl.device,
+ "NVME-FC{%d}: transport error during (re)connect\n",
+ ctrl->cnum);
+ return;
+ }
+
+ nvme_fc_error_recovery(ctrl);
}
/*
@@ -1892,6 +1910,24 @@ char *nvme_fc_io_getuuid(struct nvmefc_fcp_req *req)
}
EXPORT_SYMBOL_GPL(nvme_fc_io_getuuid);
+static void nvme_fc_start_ioerr_recovery(struct nvme_fc_ctrl *ctrl,
+ char *errmsg)
+{
+ enum nvme_ctrl_state state = nvme_ctrl_state(&ctrl->ctrl);
+
+ if (state == NVME_CTRL_CONNECTING || state == NVME_CTRL_DELETING ||
+ state == NVME_CTRL_DELETING_NOIO) {
+ queue_work(nvme_reset_wq, &ctrl->ioerr_work);
+ return;
+ }
+
+ if (nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_RESETTING)) {
+ dev_warn(ctrl->ctrl.device, "NVME-FC{%d}: starting error recovery %s\n",
+ ctrl->cnum, errmsg);
+ queue_work(nvme_reset_wq, &ctrl->ioerr_work);
+ }
+}
+
static void
nvme_fc_fcpio_done(struct nvmefc_fcp_req *req)
{
@@ -2049,9 +2085,8 @@ nvme_fc_fcpio_done(struct nvmefc_fcp_req *req)
nvme_fc_complete_rq(rq);
check_error:
- if (terminate_assoc &&
- nvme_ctrl_state(&ctrl->ctrl) != NVME_CTRL_RESETTING)
- queue_work(nvme_reset_wq, &ctrl->ioerr_work);
+ if (terminate_assoc)
+ nvme_fc_start_ioerr_recovery(ctrl, "io error");
}
static int
@@ -2495,39 +2530,6 @@ __nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues)
nvme_unquiesce_admin_queue(&ctrl->ctrl);
}
-static void
-nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl, char *errmsg)
-{
- enum nvme_ctrl_state state = nvme_ctrl_state(&ctrl->ctrl);
-
- /*
- * if an error (io timeout, etc) while (re)connecting, the remote
- * port requested terminating of the association (disconnect_ls)
- * or an error (timeout or abort) occurred on an io while creating
- * the controller. Abort any ios on the association and let the
- * create_association error path resolve things.
- */
- if (state == NVME_CTRL_CONNECTING) {
- __nvme_fc_abort_outstanding_ios(ctrl, true);
- dev_warn(ctrl->ctrl.device,
- "NVME-FC{%d}: transport error during (re)connect\n",
- ctrl->cnum);
- return;
- }
-
- /* Otherwise, only proceed if in LIVE state - e.g. on first error */
- if (state != NVME_CTRL_LIVE)
- return;
-
- dev_warn(ctrl->ctrl.device,
- "NVME-FC{%d}: transport association event: %s\n",
- ctrl->cnum, errmsg);
- dev_warn(ctrl->ctrl.device,
- "NVME-FC{%d}: resetting controller\n", ctrl->cnum);
-
- nvme_reset_ctrl(&ctrl->ctrl);
-}
-
static enum blk_eh_timer_return nvme_fc_timeout(struct request *rq)
{
struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(rq);
@@ -2536,24 +2538,14 @@ static enum blk_eh_timer_return nvme_fc_timeout(struct request *rq)
struct nvme_fc_cmd_iu *cmdiu = &op->cmd_iu;
struct nvme_command *sqe = &cmdiu->sqe;
- /*
- * Attempt to abort the offending command. Command completion
- * will detect the aborted io and will fail the connection.
- */
dev_info(ctrl->ctrl.device,
"NVME-FC{%d.%d}: io timeout: opcode %d fctype %d (%s) w10/11: "
"x%08x/x%08x\n",
ctrl->cnum, qnum, sqe->common.opcode, sqe->fabrics.fctype,
nvme_fabrics_opcode_str(qnum, sqe),
sqe->common.cdw10, sqe->common.cdw11);
- if (__nvme_fc_abort_op(ctrl, op))
- nvme_fc_error_recovery(ctrl, "io timeout abort failed");
- /*
- * the io abort has been initiated. Have the reset timer
- * restarted and the abort completion will complete the io
- * shortly. Avoids a synchronous wait while the abort finishes.
- */
+ nvme_fc_start_ioerr_recovery(ctrl, "io timeout");
return BLK_EH_RESET_TIMER;
}
@@ -3352,6 +3344,27 @@ nvme_fc_reset_ctrl_work(struct work_struct *work)
}
}
+static void
+nvme_fc_error_recovery(struct nvme_fc_ctrl *ctrl)
+{
+ nvme_stop_keep_alive(&ctrl->ctrl);
+ nvme_stop_ctrl(&ctrl->ctrl);
+ flush_work(&ctrl->ctrl.async_event_work);
+
+ /* will block while waiting for io to terminate */
+ nvme_fc_delete_association(ctrl);
+
+ /* Do not reconnect if controller is being deleted */
+ if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_CONNECTING))
+ return;
+
+ if (ctrl->rport->remoteport.port_state == FC_OBJSTATE_ONLINE) {
+ queue_delayed_work(nvme_wq, &ctrl->connect_work, 0);
+ return;
+ }
+
+ nvme_fc_reconnect_or_delete(ctrl, -ENOTCONN);
+}
static const struct nvme_ctrl_ops nvme_fc_ctrl_ops = {
.name = "fc",
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v4 13/15] nvme-fc: Use CCR to recover controller that hits an error
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (11 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 12/15] nvme-fc: Refactor IO error recovery Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 14/15] nvme-fc: Hold inflight requests while in FENCING state Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 15/15] nvme-fc: Do not cancel requests in io taget before it is initialized Mohamed Khalfella
14 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
An alive nvme controller that hits an error now will move to FENCING
state instead of RESETTING state. ctrl->fencing_work attempts CCR to
terminate inflight IOs. Regardless of the success or failure of CCR
operation the controller is transitioned to RESETTING state to continue
error recovery process.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
---
drivers/nvme/host/fc.c | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 6797eb17917f..9f6b95415f25 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -166,6 +166,7 @@ struct nvme_fc_ctrl {
struct blk_mq_tag_set admin_tag_set;
struct blk_mq_tag_set tag_set;
+ struct work_struct fencing_work;
struct work_struct ioerr_work;
struct delayed_work connect_work;
@@ -1868,6 +1869,22 @@ __nvme_fc_fcpop_chk_teardowns(struct nvme_fc_ctrl *ctrl,
}
}
+static void nvme_fc_fencing_work(struct work_struct *work)
+{
+ struct nvme_fc_ctrl *fc_ctrl =
+ container_of(work, struct nvme_fc_ctrl, fencing_work);
+ struct nvme_ctrl *ctrl = &fc_ctrl->ctrl;
+ int ret;
+
+ ret = nvme_fence_ctrl(ctrl);
+ if (ret)
+ dev_info(ctrl->device, "CCR failed with error %d\n", ret);
+
+ nvme_change_ctrl_state(ctrl, NVME_CTRL_FENCED);
+ if (nvme_change_ctrl_state(ctrl, NVME_CTRL_RESETTING))
+ queue_work(nvme_reset_wq, &fc_ctrl->ioerr_work);
+}
+
static void
nvme_fc_ctrl_ioerr_work(struct work_struct *work)
{
@@ -1889,6 +1906,7 @@ nvme_fc_ctrl_ioerr_work(struct work_struct *work)
return;
}
+ flush_work(&ctrl->fencing_work);
nvme_fc_error_recovery(ctrl);
}
@@ -1921,6 +1939,14 @@ static void nvme_fc_start_ioerr_recovery(struct nvme_fc_ctrl *ctrl,
return;
}
+ if (nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_FENCING)) {
+ dev_warn(ctrl->ctrl.device,
+ "NVME-FC{%d}: starting controller fencing %s\n",
+ ctrl->cnum, errmsg);
+ queue_work(nvme_wq, &ctrl->fencing_work);
+ return;
+ }
+
if (nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_RESETTING)) {
dev_warn(ctrl->ctrl.device, "NVME-FC{%d}: starting error recovery %s\n",
ctrl->cnum, errmsg);
@@ -3321,6 +3347,7 @@ nvme_fc_reset_ctrl_work(struct work_struct *work)
struct nvme_fc_ctrl *ctrl =
container_of(work, struct nvme_fc_ctrl, ctrl.reset_work);
+ flush_work(&ctrl->fencing_work);
nvme_stop_ctrl(&ctrl->ctrl);
/* will block will waiting for io to terminate */
@@ -3496,6 +3523,7 @@ nvme_fc_alloc_ctrl(struct device *dev, struct nvmf_ctrl_options *opts,
INIT_WORK(&ctrl->ctrl.reset_work, nvme_fc_reset_ctrl_work);
INIT_DELAYED_WORK(&ctrl->connect_work, nvme_fc_connect_ctrl_work);
+ INIT_WORK(&ctrl->fencing_work, nvme_fc_fencing_work);
INIT_WORK(&ctrl->ioerr_work, nvme_fc_ctrl_ioerr_work);
spin_lock_init(&ctrl->lock);
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v4 14/15] nvme-fc: Hold inflight requests while in FENCING state
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (12 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 13/15] nvme-fc: Use CCR to recover controller that hits an error Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
2026-03-28 0:43 ` [PATCH v4 15/15] nvme-fc: Do not cancel requests in io taget before it is initialized Mohamed Khalfella
14 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
While in FENCING state, aborted inflight IOs should be held until fencing
is done. Update nvme_fc_fcpio_done() to not complete aborted requests or
requests with transport errors. These held requests will be canceled in
nvme_fc_delete_association() after fencing is done. nvme_fc_fcpio_done()
avoids racing with canceling aborted requests by making sure we complete
successful requests before waking up the waiting thread.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
Signed-off-by: James Smart <jsmart833426@gmail.com>
---
drivers/nvme/host/fc.c | 61 +++++++++++++++++++++++++++++++++++-------
1 file changed, 51 insertions(+), 10 deletions(-)
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 9f6b95415f25..eea5a90d936b 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -172,7 +172,7 @@ struct nvme_fc_ctrl {
struct kref ref;
unsigned long flags;
- u32 iocnt;
+ atomic_t iocnt;
wait_queue_head_t ioabort_wait;
struct nvme_fc_fcp_op aen_ops[NVME_NR_AEN_COMMANDS];
@@ -1823,7 +1823,7 @@ __nvme_fc_abort_op(struct nvme_fc_ctrl *ctrl, struct nvme_fc_fcp_op *op)
atomic_set(&op->state, opstate);
else if (test_bit(FCCTRL_TERMIO, &ctrl->flags)) {
op->flags |= FCOP_FLAGS_TERMIO;
- ctrl->iocnt++;
+ atomic_inc(&ctrl->iocnt);
}
spin_unlock_irqrestore(&ctrl->lock, flags);
@@ -1853,20 +1853,29 @@ nvme_fc_abort_aen_ops(struct nvme_fc_ctrl *ctrl)
}
static inline void
+__nvme_fc_fcpop_count_one_down(struct nvme_fc_ctrl *ctrl)
+{
+ if (atomic_dec_return(&ctrl->iocnt) == 0)
+ wake_up(&ctrl->ioabort_wait);
+}
+
+static inline bool
__nvme_fc_fcpop_chk_teardowns(struct nvme_fc_ctrl *ctrl,
struct nvme_fc_fcp_op *op, int opstate)
{
unsigned long flags;
+ bool ret = false;
if (opstate == FCPOP_STATE_ABORTED) {
spin_lock_irqsave(&ctrl->lock, flags);
if (test_bit(FCCTRL_TERMIO, &ctrl->flags) &&
op->flags & FCOP_FLAGS_TERMIO) {
- if (!--ctrl->iocnt)
- wake_up(&ctrl->ioabort_wait);
+ ret = true;
}
spin_unlock_irqrestore(&ctrl->lock, flags);
}
+
+ return ret;
}
static void nvme_fc_fencing_work(struct work_struct *work)
@@ -1966,7 +1975,8 @@ nvme_fc_fcpio_done(struct nvmefc_fcp_req *req)
struct nvme_command *sqe = &op->cmd_iu.sqe;
__le16 status = cpu_to_le16(NVME_SC_SUCCESS << 1);
union nvme_result result;
- bool terminate_assoc = true;
+ bool op_term, terminate_assoc = true;
+ enum nvme_ctrl_state state;
int opstate;
/*
@@ -2099,16 +2109,38 @@ nvme_fc_fcpio_done(struct nvmefc_fcp_req *req)
done:
if (op->flags & FCOP_FLAGS_AEN) {
nvme_complete_async_event(&queue->ctrl->ctrl, status, &result);
- __nvme_fc_fcpop_chk_teardowns(ctrl, op, opstate);
+ if (__nvme_fc_fcpop_chk_teardowns(ctrl, op, opstate))
+ __nvme_fc_fcpop_count_one_down(ctrl);
atomic_set(&op->state, FCPOP_STATE_IDLE);
op->flags = FCOP_FLAGS_AEN; /* clear other flags */
nvme_fc_ctrl_put(ctrl);
goto check_error;
}
- __nvme_fc_fcpop_chk_teardowns(ctrl, op, opstate);
+ /*
+ * We can not access op after the request is completed because it can
+ * be reused immediately. At the same time we want to wakeup the thread
+ * waiting for ongoing IOs _after_ requests are completed. This is
+ * necessary because that thread will start canceling inflight IOs
+ * and we want to avoid request completion racing with cancellation.
+ */
+ op_term = __nvme_fc_fcpop_chk_teardowns(ctrl, op, opstate);
+
+ /*
+ * If we are going to terminate associations and the controller is
+ * LIVE or FENCING, then do not complete this request now. Let error
+ * recovery cancel this request when it is safe to do so.
+ */
+ state = nvme_ctrl_state(&ctrl->ctrl);
+ if (terminate_assoc &&
+ (state == NVME_CTRL_LIVE || state == NVME_CTRL_FENCING))
+ goto check_op_term;
+
if (!nvme_try_complete_req(rq, status, result))
nvme_fc_complete_rq(rq);
+check_op_term:
+ if (op_term)
+ __nvme_fc_fcpop_count_one_down(ctrl);
check_error:
if (terminate_assoc)
@@ -2747,7 +2779,8 @@ nvme_fc_start_fcp_op(struct nvme_fc_ctrl *ctrl, struct nvme_fc_queue *queue,
* cmd with the csn was supposed to arrive.
*/
opstate = atomic_xchg(&op->state, FCPOP_STATE_COMPLETE);
- __nvme_fc_fcpop_chk_teardowns(ctrl, op, opstate);
+ if (__nvme_fc_fcpop_chk_teardowns(ctrl, op, opstate))
+ __nvme_fc_fcpop_count_one_down(ctrl);
if (!(op->flags & FCOP_FLAGS_AEN)) {
nvme_fc_unmap_data(ctrl, op->rq, op);
@@ -3216,7 +3249,7 @@ nvme_fc_delete_association(struct nvme_fc_ctrl *ctrl)
spin_lock_irqsave(&ctrl->lock, flags);
set_bit(FCCTRL_TERMIO, &ctrl->flags);
- ctrl->iocnt = 0;
+ atomic_set(&ctrl->iocnt, 0);
spin_unlock_irqrestore(&ctrl->lock, flags);
__nvme_fc_abort_outstanding_ios(ctrl, false);
@@ -3225,11 +3258,19 @@ nvme_fc_delete_association(struct nvme_fc_ctrl *ctrl)
nvme_fc_abort_aen_ops(ctrl);
/* wait for all io that had to be aborted */
+ wait_event(ctrl->ioabort_wait, atomic_read(&ctrl->iocnt) == 0);
spin_lock_irq(&ctrl->lock);
- wait_event_lock_irq(ctrl->ioabort_wait, ctrl->iocnt == 0, ctrl->lock);
clear_bit(FCCTRL_TERMIO, &ctrl->flags);
spin_unlock_irq(&ctrl->lock);
+ /*
+ * At this point all inflight requests have been successfully
+ * aborted. Now it is safe to cancel all requests we decided
+ * not to complete in nvme_fc_fcpio_done().
+ */
+ nvme_cancel_tagset(&ctrl->ctrl);
+ nvme_cancel_admin_tagset(&ctrl->ctrl);
+
nvme_fc_term_aen_ops(ctrl);
/*
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v4 15/15] nvme-fc: Do not cancel requests in io taget before it is initialized
2026-03-28 0:43 [PATCH v4 00/15] TP8028 Rapid Path Failure Recovery Mohamed Khalfella
` (13 preceding siblings ...)
2026-03-28 0:43 ` [PATCH v4 14/15] nvme-fc: Hold inflight requests while in FENCING state Mohamed Khalfella
@ 2026-03-28 0:43 ` Mohamed Khalfella
14 siblings, 0 replies; 25+ messages in thread
From: Mohamed Khalfella @ 2026-03-28 0:43 UTC (permalink / raw)
To: Justin Tee, Naresh Gottumukkala, Paul Ely, Chaitanya Kulkarni,
Jens Axboe, Keith Busch, Sagi Grimberg, James Smart,
Hannes Reinecke
Cc: Aaron Dailey, Randy Jennings, Dhaval Giani, linux-nvme,
linux-kernel, Mohamed Khalfella
A new nvme-fc controller in CONNECTING state sees admin request timeout
schedules ctrl->ioerr_work to abort inflight requests. This ends up
calling __nvme_fc_abort_outstanding_ios() which aborts requests in both
admin and io tagsets. In case fc_ctrl->tag_set was not initialized we
see the warning below. This is because ctrl.queue_count is initialized
early in nvme_fc_alloc_ctrl().
nvme nvme0: NVME-FC{0}: starting error recovery Connectivity Loss
INFO: trying to register non-static key.
The code is fine but needs lockdep annotation, or maybe
lpfc 0000:ab:00.0: queue 0 connect admin queue failed (-6).
you didn't initialize this object before use?
turning off the locking correctness validator.
Workqueue: nvme-reset-wq nvme_fc_ctrl_ioerr_work [nvme_fc]
Call Trace:
<TASK>
dump_stack_lvl+0x57/0x80
register_lock_class+0x567/0x580
__lock_acquire+0x330/0xb90
lock_acquire.part.0+0xad/0x210
blk_mq_tagset_busy_iter+0xf9/0xc00
__nvme_fc_abort_outstanding_ios+0x23f/0x320 [nvme_fc]
nvme_fc_ctrl_ioerr_work+0x172/0x210 [nvme_fc]
process_one_work+0x82c/0x1450
worker_thread+0x5ee/0xfd0
kthread+0x3a0/0x750
ret_from_fork+0x439/0x670
ret_from_fork_asm+0x1a/0x30
</TASK>
Update the check in __nvme_fc_abort_outstanding_ios() confirm that io
tagset was created before iterating over busy requests. Also make sure
to cancel ctrl->ioerr_work before removing io tagset.
Signed-off-by: Mohamed Khalfella <mkhalfella@purestorage.com>
Signed-off-by: James Smart <jsmart833426@gmail.com>
---
drivers/nvme/host/fc.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index eea5a90d936b..e342d522145a 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -2554,7 +2554,7 @@ __nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues)
* io requests back to the block layer as part of normal completions
* (but with error status).
*/
- if (ctrl->ctrl.queue_count > 1) {
+ if (ctrl->ctrl.queue_count > 1 && ctrl->ctrl.tagset) {
nvme_quiesce_io_queues(&ctrl->ctrl);
nvme_sync_io_queues(&ctrl->ctrl);
blk_mq_tagset_busy_iter(&ctrl->tag_set,
@@ -2951,6 +2951,11 @@ nvme_fc_create_io_queues(struct nvme_fc_ctrl *ctrl)
out_delete_hw_queues:
nvme_fc_delete_hw_io_queues(ctrl);
out_cleanup_tagset:
+ /*
+ * In CONNECTING state ctrl->ioerr_work will abort both admin
+ * and io tagsets. Cancel it first before removing io tagset.
+ */
+ cancel_work_sync(&ctrl->ioerr_work);
nvme_remove_io_tag_set(&ctrl->ctrl);
nvme_fc_free_io_queues(ctrl);
--
2.52.0
^ permalink raw reply related [flat|nested] 25+ messages in thread