[PATCH 00/29] ibmvfc: Add NVMe-FC support

[PATCH 00/29] ibmvfc: Add NVMe-FC support

[Tyrel Datwyler]

This patch series adds NVMe-FC protocol support to the ibmvfc driver,
enabling IBM POWER virtual Fibre Channel adapters to handle both SCSI
and NVMe storage traffic through a unified driver architecture.

The implementation leverages the existing multi-queue infrastructure
and extends the driver's protocol handling to support NVMe-FC alongside
traditional SCSI FCP. Key architectural changes include:

- Refactoring the driver into protocol-agnostic core (ibmvfc-core.c)
  and protocol-specific modules (ibmvfc-nvme.c for NVMe-FC)
- Extending the target discovery and management state machine to handle
  NVMe/FC targets with appropriate protocol-specific login sequences
- Implementing the nvme-fc LLDD (Low-Level Device Driver) callbacks for
  queue management, command submission, and abort handling
- Adding support for NVMe-FC specific MAD (Management Adapter Data)
  operations including target discovery, port login, process login,
  and implicit logout/move login
- Implementing NVMe FCP and Link Services (LS) command submission and
  abort paths using the existing sub-CRQ infrastructure

The series is organized to introduce changes incrementally:

Patches 1-8: Infrastructure and refactoring
- Move target lists to protocol-specific channel groups
- Split NVMe support into separate source file
- Add protocol interface definitions and helper functions
- Initialize NVMe channel configuration

Patches 9-16: NVMe/FC MAD operations and target management
- Implement discovery, port login, process login, query target
- Add target allocation based on protocol
- Update state machine for NVMe/FC target processing
- Handle NVMe/FC target deletion

Patches 17-20: Local and remote port registration
- Implement nvme-fc local port registration after fabric login
- Process NVMe/FC rports in work thread
- Extend debug visibility for NVMe components

Patches 21-26: LLDD callback implementation
- Implement queue mapping callbacks
- Add LS and FCP command submission paths
- Implement NVMe FCP and LS abort handling

Patches 27-29: Error handling and debugging
- Fail NVMe requests during transport reset


Tyrel Datwyler (29):
  ibmvfc: move target list from host to protocol specific channel groups
  ibmvfc: add NVMe/FC protocol interface definitions
  ibmvfc: split NVMe support into separate source file and add transport
    stubs
  ibmvfc: initialize NVMe channel configuration during driver probe
  ibmvfc: alloc/dealloc sub-queues for nvme channels
  ibmvfc: add logic for protocol specific fabric logins
  ibmvfc: add wrapper to get vhost associated with a channel struct
  ibmvfc: add helper for creating protocol specific discovery event
  ibmvfc: add helper to check NVMe/FC support with active channels
  ibmvfc: allocate and free NVMe channel group discover buffer
  ibmvfc: send NVMe target discovery MAD
  ibmvfc: add NVMe/FC Implicit Logout and Move Login support
  ibmvfc: add NVMe/FC Port Login support
  ibmvfc: add NVMe/FC Process Login support
  ibmvfc: add NVMe/FC Query Target support
  ibmvfc: allocate targets based on protocol
  ibmvfc: delete NVMe/FC targets as well as SCSI
  ibmvfc: update state machine to process NVMe/FC targets
  ibmvfc: implement NVMe/FC stubs for local/remote port registration
  ibmvfc: register local nvme fc port after fabric login
  ibmvfc: process NVMe/FC rports in work thread
  ibmvfc: extend ibmvfc_debug visibility to ibmvfc-nvme.h
  ibmvfc: declare global function definitions
  ibmvfc: implement LLDD callbacks for mapping nvme-fc queues
  ibmvfc: implement nvme-fc LS submission transport callback
  ibmvfc: implement nvme-fc IO command submission callback
  ibmvfc: implement nvme-fc LS abort handling callback
  ibmvfc: implement nvme-fc FCP abort callback
  ibmvfc: fail nvme-fc fcp-io and ls requests during transport reset

 drivers/scsi/ibmvscsi/Makefile                |   2 +
 .../scsi/ibmvscsi/{ibmvfc.c => ibmvfc-core.c} | 709 ++++++++++++++----
 drivers/scsi/ibmvscsi/ibmvfc-nvme.c           | 565 ++++++++++++++
 drivers/scsi/ibmvscsi/ibmvfc-nvme.h           |  48 ++
 drivers/scsi/ibmvscsi/ibmvfc.h                | 209 +++++-
 5 files changed, 1360 insertions(+), 173 deletions(-)
 rename drivers/scsi/ibmvscsi/{ibmvfc.c => ibmvfc-core.c} (89%)
 create mode 100644 drivers/scsi/ibmvscsi/ibmvfc-nvme.c
 create mode 100644 drivers/scsi/ibmvscsi/ibmvfc-nvme.h

-- 
2.54.0

[PATCH 01/29] ibmvfc: move target list from host to protocol specific channel groups

[Tyrel Datwyler]

Prepare the driver for protocol-specific target management by moving
the target list and target count out of struct ibmvfc_host and into
struct ibmvfc_channels.

Today the driver only maintains a single SCSI target list, but NVMe/FC
support will require separate target tracking for each protocol-specific
channel group. Update the existing target iteration, allocation, and
discovery paths to use the SCSI channel group's target list instead of a
host-wide list.

This is a preparatory refactoring only. No functional change is intended
for existing SCSI operation.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc.c | 52 +++++++++++++++++-----------------
 drivers/scsi/ibmvscsi/ibmvfc.h |  4 +--
 2 files changed, 28 insertions(+), 28 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c
index 3dd2adda195e..912901436442 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc.c
@@ -682,7 +682,7 @@ static void ibmvfc_link_down(struct ibmvfc_host *vhost,
 
 	ENTER;
 	scsi_block_requests(vhost->host);
-	list_for_each_entry(tgt, &vhost->targets, queue)
+	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 		ibmvfc_del_tgt(tgt);
 	ibmvfc_set_host_state(vhost, state);
 	ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_TGT_DEL);
@@ -715,7 +715,7 @@ static void ibmvfc_init_host(struct ibmvfc_host *vhost)
 		memset(vhost->async_crq.msgs.async, 0, PAGE_SIZE);
 		vhost->async_crq.cur = 0;
 
-		list_for_each_entry(tgt, &vhost->targets, queue) {
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 			if (vhost->client_migrated)
 				tgt->need_login = 1;
 			else
@@ -1232,7 +1232,7 @@ static struct ibmvfc_target *__ibmvfc_get_target(struct scsi_target *starget)
 	struct ibmvfc_host *vhost = shost_priv(shost);
 	struct ibmvfc_target *tgt;
 
-	list_for_each_entry(tgt, &vhost->targets, queue)
+	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 		if (tgt->target_id == starget->id) {
 			kref_get(&tgt->kref);
 			return tgt;
@@ -1832,7 +1832,7 @@ static void ibmvfc_relogin(struct scsi_device *sdev)
 	unsigned long flags;
 
 	spin_lock_irqsave(vhost->host->host_lock, flags);
-	list_for_each_entry(tgt, &vhost->targets, queue) {
+	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 		if (rport == tgt->rport) {
 			ibmvfc_del_tgt(tgt);
 			break;
@@ -2130,7 +2130,7 @@ static int ibmvfc_bsg_plogi(struct ibmvfc_host *vhost, unsigned int port_id)
 
 	ENTER;
 	spin_lock_irqsave(vhost->host->host_lock, flags);
-	list_for_each_entry(tgt, &vhost->targets, queue) {
+	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 		if (tgt->scsi_id == port_id) {
 			issue_login = 0;
 			break;
@@ -3102,7 +3102,7 @@ static void ibmvfc_terminate_rport_io(struct fc_rport *rport)
 
 	spin_lock_irqsave(shost->host_lock, flags);
 	found = 0;
-	list_for_each_entry(tgt, &vhost->targets, queue) {
+	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 		if (tgt->scsi_id == rport->port_id) {
 			found++;
 			break;
@@ -3242,7 +3242,7 @@ static void ibmvfc_handle_async(struct ibmvfc_async_crq *crq,
 	case IBMVFC_AE_ELS_LOGO:
 	case IBMVFC_AE_ELS_PRLO:
 	case IBMVFC_AE_ELS_PLOGI:
-		list_for_each_entry(tgt, &vhost->targets, queue) {
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 			if (!crq->scsi_id && !crq->wwpn && !crq->node_name)
 				break;
 			if (crq->scsi_id && cpu_to_be64(tgt->scsi_id) != crq->scsi_id)
@@ -4863,14 +4863,14 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,
 
 	/* Look to see if we already have a target allocated for this SCSI ID or WWPN */
 	spin_lock_irqsave(vhost->host->host_lock, flags);
-	list_for_each_entry(tgt, &vhost->targets, queue) {
+	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 		if (tgt->wwpn == wwpn) {
 			wtgt = tgt;
 			break;
 		}
 	}
 
-	list_for_each_entry(tgt, &vhost->targets, queue) {
+	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 		if (tgt->scsi_id == scsi_id) {
 			stgt = tgt;
 			break;
@@ -4927,7 +4927,7 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,
 	ibmvfc_init_tgt(tgt, ibmvfc_tgt_implicit_logout);
 	spin_lock_irqsave(vhost->host->host_lock, flags);
 	tgt->cancel_key = vhost->task_set++;
-	list_add_tail(&tgt->queue, &vhost->targets);
+	list_add_tail(&tgt->queue, &vhost->scsi_scrqs.targets);
 
 unlock_out:
 	spin_unlock_irqrestore(vhost->host->host_lock, flags);
@@ -4945,7 +4945,7 @@ static int ibmvfc_alloc_targets(struct ibmvfc_host *vhost)
 {
 	int i, rc;
 
-	for (i = 0, rc = 0; !rc && i < vhost->num_targets; i++)
+	for (i = 0, rc = 0; !rc && i < vhost->scsi_scrqs.num_targets; i++)
 		rc = ibmvfc_alloc_target(vhost, &vhost->scsi_scrqs.disc_buf[i]);
 
 	return rc;
@@ -4966,8 +4966,8 @@ static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt)
 	switch (mad_status) {
 	case IBMVFC_MAD_SUCCESS:
 		ibmvfc_dbg(vhost, "Discover Targets succeeded\n");
-		vhost->num_targets = min_t(u32, be32_to_cpu(rsp->num_written),
-					   max_targets);
+		vhost->scsi_scrqs.num_targets = min_t(u32, be32_to_cpu(rsp->num_written),
+						      max_targets);
 		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_ALLOC_TGTS);
 		break;
 	case IBMVFC_MAD_FAILED:
@@ -5383,7 +5383,7 @@ static int ibmvfc_dev_init_to_do(struct ibmvfc_host *vhost)
 {
 	struct ibmvfc_target *tgt;
 
-	list_for_each_entry(tgt, &vhost->targets, queue) {
+	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 		if (tgt->action == IBMVFC_TGT_ACTION_INIT ||
 		    tgt->action == IBMVFC_TGT_ACTION_INIT_WAIT)
 			return 1;
@@ -5403,7 +5403,7 @@ static int ibmvfc_dev_logo_to_do(struct ibmvfc_host *vhost)
 {
 	struct ibmvfc_target *tgt;
 
-	list_for_each_entry(tgt, &vhost->targets, queue) {
+	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 		if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT ||
 		    tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT)
 			return 1;
@@ -5433,10 +5433,10 @@ static int __ibmvfc_work_to_do(struct ibmvfc_host *vhost)
 	case IBMVFC_HOST_ACTION_QUERY_TGTS:
 		if (vhost->discovery_threads == disc_threads)
 			return 0;
-		list_for_each_entry(tgt, &vhost->targets, queue)
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 			if (tgt->action == IBMVFC_TGT_ACTION_INIT)
 				return 1;
-		list_for_each_entry(tgt, &vhost->targets, queue)
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 			if (tgt->action == IBMVFC_TGT_ACTION_INIT_WAIT)
 				return 0;
 		return 1;
@@ -5444,10 +5444,10 @@ static int __ibmvfc_work_to_do(struct ibmvfc_host *vhost)
 	case IBMVFC_HOST_ACTION_TGT_DEL_FAILED:
 		if (vhost->discovery_threads == disc_threads)
 			return 0;
-		list_for_each_entry(tgt, &vhost->targets, queue)
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 			if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT)
 				return 1;
-		list_for_each_entry(tgt, &vhost->targets, queue)
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 			if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT)
 				return 0;
 		return 1;
@@ -5635,12 +5635,12 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)
 			vhost->job_step(vhost);
 		break;
 	case IBMVFC_HOST_ACTION_QUERY:
-		list_for_each_entry(tgt, &vhost->targets, queue)
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 			ibmvfc_init_tgt(tgt, ibmvfc_tgt_query_target);
 		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY_TGTS);
 		break;
 	case IBMVFC_HOST_ACTION_QUERY_TGTS:
-		list_for_each_entry(tgt, &vhost->targets, queue) {
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 			if (tgt->action == IBMVFC_TGT_ACTION_INIT) {
 				tgt->job_step(tgt);
 				break;
@@ -5652,7 +5652,7 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)
 		break;
 	case IBMVFC_HOST_ACTION_TGT_DEL:
 	case IBMVFC_HOST_ACTION_TGT_DEL_FAILED:
-		list_for_each_entry(tgt, &vhost->targets, queue) {
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 			if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT) {
 				tgt->job_step(tgt);
 				break;
@@ -5664,7 +5664,7 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)
 			return;
 		}
 
-		list_for_each_entry(tgt, &vhost->targets, queue) {
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 			if (tgt->action == IBMVFC_TGT_ACTION_DEL_RPORT) {
 				tgt_dbg(tgt, "Deleting rport\n");
 				rport = tgt->rport;
@@ -5739,7 +5739,7 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)
 		spin_lock_irqsave(vhost->host->host_lock, flags);
 		break;
 	case IBMVFC_HOST_ACTION_TGT_INIT:
-		list_for_each_entry(tgt, &vhost->targets, queue) {
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 			if (tgt->action == IBMVFC_TGT_ACTION_INIT) {
 				tgt->job_step(tgt);
 				break;
@@ -6276,7 +6276,7 @@ static void ibmvfc_rport_add_thread(struct work_struct *work)
 		if (vhost->state != IBMVFC_ACTIVE)
 			break;
 
-		list_for_each_entry(tgt, &vhost->targets, queue) {
+		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
 			if (tgt->add_rport) {
 				did_work = 1;
 				tgt->add_rport = 0;
@@ -6341,7 +6341,7 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id)
 	shost->nr_hw_queues = mq_enabled ? min(max_scsi_queues, nr_scsi_hw_queues) : 1;
 
 	vhost = shost_priv(shost);
-	INIT_LIST_HEAD(&vhost->targets);
+	INIT_LIST_HEAD(&vhost->scsi_scrqs.targets);
 	INIT_LIST_HEAD(&vhost->purge);
 	sprintf(vhost->name, IBMVFC_NAME);
 	vhost->host = shost;
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index c73ed2314ad0..0e259e9d2e9b 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -828,6 +828,8 @@ struct ibmvfc_channels {
 	unsigned int active_queues;
 	unsigned int desired_queues;
 	unsigned int max_queues;
+	int num_targets;
+	struct list_head targets;
 	int disc_buf_sz;
 	struct ibmvfc_discover_targets_entry *disc_buf;
 	dma_addr_t disc_buf_dma;
@@ -871,8 +873,6 @@ struct ibmvfc_host {
 #define IBMVFC_TRACE_SIZE	(sizeof(struct ibmvfc_trace_entry) * IBMVFC_NUM_TRACE_ENTRIES)
 	struct ibmvfc_trace_entry *trace;
 	atomic_t trace_index;
-	int num_targets;
-	struct list_head targets;
 	struct list_head purge;
 	struct device *dev;
 	struct dma_pool *sg_pool;
-- 
2.54.0

[PATCH 02/29] ibmvfc: add NVMe/FC protocol interface definitions

[Tyrel Datwyler]

Add the protocol definitions for client-VIOS interface updates needed to
support NVMe/FC over the ibmvfc NPIV transport.

Extend the ibmvfc interface with:

- NVMe/FC-specific capability bits and opcodes
- protocol-specific channel and queue definitions
- updated channel enquiry/setup fields for NVMe queues
- v3 command layout support for protocol-specific payloads

These changes provide the common header and interface plumbing needed by
later patches that add NVMe/FC login, discovery, remote-port handling,
and I/O submission.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc.h | 153 +++++++++++++++++++++++++--------
 1 file changed, 119 insertions(+), 34 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index 0e259e9d2e9b..f8a2bf92da41 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -13,6 +13,8 @@
 #include <linux/list.h>
 #include <linux/types.h>
 #include <scsi/viosrp.h>
+#include <linux/nvme.h>
+#include <linux/nvme-fc.h>
 
 #define IBMVFC_NAME	"ibmvfc"
 #define IBMVFC_DRIVER_VERSION		"1.0.11"
@@ -92,6 +94,7 @@ enum ibmvfc_cmd_status_flags {
 	IBMVFC_FC_SCSI_ERROR		= 0x0008,
 	IBMVFC_HW_EVENT_LOGGED		= 0x0010,
 	IBMVFC_VIOS_LOGGED		= 0x0020,
+	IBMVFC_FC_NVME_STATUS		= 0x0040,
 };
 
 enum ibmvfc_fabric_mapped_errors {
@@ -124,20 +127,37 @@ enum ibmvfc_vios_errors {
 	IBMVFC_COMMAND_FAILED			= 0x8000,
 };
 
+enum ibmvfc_fc_nvme_errors {
+	IBMVFC_NVMS_VALID_ERSP		= 0x0001,
+	IBMVFC_NVMS_VALID_NODMA_CQE	= 0x0002,
+};
+
 enum ibmvfc_mad_types {
 	IBMVFC_NPIV_LOGIN		= 0x0001,
-	IBMVFC_DISC_TARGETS	= 0x0002,
+	IBMVFC_DISC_TARGETS		= 0x0002,
+	IBMVFC_DISC_NVMF_TARGETS	= 0x0003,
 	IBMVFC_PORT_LOGIN		= 0x0004,
-	IBMVFC_PROCESS_LOGIN	= 0x0008,
-	IBMVFC_QUERY_TARGET	= 0x0010,
+	IBMVFC_NVMF_PORT_LOGIN		= 0x0005,
+	IBMVFC_PROCESS_LOGIN		= 0x0008,
+	IBMVFC_NVMF_PROCESS_LOGIN	= 0x0009,
+	IBMVFC_QUERY_TARGET		= 0x0010,
+	IBMVFC_NVMF_QUERY_TARGET	= 0x0011,
 	IBMVFC_MOVE_LOGIN		= 0x0020,
-	IBMVFC_IMPLICIT_LOGOUT	= 0x0040,
-	IBMVFC_PASSTHRU		= 0x0200,
-	IBMVFC_TMF_MAD		= 0x0100,
-	IBMVFC_NPIV_LOGOUT	= 0x0800,
-	IBMVFC_CHANNEL_ENQUIRY	= 0x1000,
-	IBMVFC_CHANNEL_SETUP	= 0x2000,
-	IBMVFC_CONNECTION_INFO	= 0x4000,
+	IBMVFC_NVMF_MOVE_LOGIN		= 0x0021,
+	IBMVFC_IMPLICIT_LOGOUT		= 0x0040,
+	IBMVFC_NVMF_IMPLICIT_LOGOUT	= 0x0041,
+	IBMVFC_RNID			= 0x0080,
+	IBMVFC_NVMF_RNID		= 0x0081,
+	IBMVFC_TMF_MAD			= 0x0100,
+	IBMVFC_NVMF_TMF_MAD		= 0x0101,
+	IBMVFC_PASSTHRU			= 0x0200,
+	IBMVFC_NVMF_PASSTHRU		= 0x0201,
+	IBMVFC_NPIV_LOGOUT		= 0x0800,
+	IBMVFC_CHANNEL_ENQUIRY		= 0x1000,
+	IBMVFC_CHANNEL_SETUP		= 0x2000,
+	IBMVFC_CONNECTION_INFO		= 0x4000,
+	IBMVFC_FABRIC_LOGIN		= 0x8000,
+	IBMVFC_NVMF_FABRIC_LOGIN	= 0x8001,
 };
 
 struct ibmvfc_mad_common {
@@ -175,11 +195,16 @@ struct ibmvfc_npiv_login {
 #define IBMVFC_FLUSH_ON_HALT		0x02
 	__be32 max_cmds;
 	__be64 capabilities;
-#define IBMVFC_CAN_MIGRATE		0x01
-#define IBMVFC_CAN_USE_CHANNELS		0x02
-#define IBMVFC_CAN_HANDLE_FPIN		0x04
-#define IBMVFC_CAN_USE_MAD_VERSION	0x08
-#define IBMVFC_CAN_SEND_VF_WWPN		0x10
+#define IBMVFC_CAN_MIGRATE		0x001
+#define IBMVFC_CAN_USE_CHANNELS		0x002
+#define IBMVFC_CAN_HANDLE_FPIN		0x004
+#define IBMVFC_CAN_USE_MAD_VERSION	0x008
+#define IBMVFC_CAN_SEND_VF_WWPN		0x010
+#define IBMVFC_YES_NVMEOF		0x020
+#define IBMVFC_YES_SCSI			0x040
+#define IBMVFC_CAN_USE_WWPN_ALL		0x080
+#define IBMVFC_USE_ASYNC_SUBQ		0x100
+#define IBMVFC_CAN_USE_NOOP_CMD		0x200
 	__be64 node_name;
 	struct srp_direct_buf async;
 	u8 partition_name[IBMVFC_MAX_NAME];
@@ -219,13 +244,18 @@ struct ibmvfc_npiv_login_resp {
 	__be16 error;
 	__be32 flags;
 #define IBMVFC_NATIVE_FC		0x01
-	__be32 reserved;
+	__be32 possible_nports;
 	__be64 capabilities;
-#define IBMVFC_CAN_FLUSH_ON_HALT	0x08
-#define IBMVFC_CAN_SUPPRESS_ABTS	0x10
-#define IBMVFC_MAD_VERSION_CAP		0x20
-#define IBMVFC_HANDLE_VF_WWPN		0x40
-#define IBMVFC_CAN_SUPPORT_CHANNELS	0x80
+#define IBMVFC_CAN_FLUSH_ON_HALT	0x0008
+#define IBMVFC_CAN_SUPPRESS_ABTS	0x0010
+#define IBMVFC_MAD_VERSION_CAP		0x0020
+#define IBMVFC_HANDLE_VF_WWPN		0x0040
+#define IBMVFC_CAN_SUPPORT_CHANNELS	0x0080
+#define IBMVFC_SUPPORT_NVMEOF		0x0100
+#define IBMVFC_SUPPORT_SCSI		0x0200
+#define IBMVFC_SUPPORT_WWPN_ALL		0x0400
+#define IBMVFC_ASYNC_SUBQ		0x0800
+#define IBMVFC_SUPPORT_NOOP_CMD		0x1000
 	__be32 max_cmds;
 	__be32 scsi_id_sz;
 	__be64 max_dma_len;
@@ -238,7 +268,7 @@ struct ibmvfc_npiv_login_resp {
 	u8 port_loc_code[IBMVFC_MAX_NAME];
 	u8 drc_name[IBMVFC_MAX_NAME];
 	struct ibmvfc_service_parms service_parms;
-	__be64 reserved2;
+	__be64 reserved;
 } __packed __aligned(8);
 
 union ibmvfc_npiv_login_data {
@@ -246,6 +276,17 @@ union ibmvfc_npiv_login_data {
 	struct ibmvfc_npiv_login_resp resp;
 } __packed __aligned(8);
 
+struct ibmvfc_fabric_login_mad {
+	struct ibmvfc_mad_common common;
+	__be64 flags;
+	__be64 capabilities;
+	__be64 nport_id;
+	__be16 status;
+	__be16 error;
+	__be32 reserved;
+	__be64 reserved2[16];
+} __packed __aligned(8);
+
 struct ibmvfc_discover_targets_entry {
 	__be32 scsi_id;
 	__be32 pad;
@@ -287,6 +328,7 @@ enum ibmvfc_fc_type {
 	IBMVFC_FABRIC_BUSY	= 0x04,
 	IBMVFC_PORT_BUSY		= 0x05,
 	IBMVFC_BASIC_REJECT	= 0x06,
+	IBMVFC_FC4_LS_REJECT	= 0x07,
 };
 
 enum ibmvfc_gs_explain {
@@ -377,20 +419,27 @@ struct ibmvfc_query_tgt {
 struct ibmvfc_implicit_logout {
 	struct ibmvfc_mad_common common;
 	__be64 old_scsi_id;
-	__be64 reserved[2];
+	__be64 reserved[8];
+	__be64 target_wwpn;
 } __packed __aligned(8);
 
 struct ibmvfc_tmf {
 	struct ibmvfc_mad_common common;
 	__be64 scsi_id;
-	struct scsi_lun lun;
+	union {
+		struct scsi_lun lun;
+		__be64 assoc_id;
+	};
 	__be32 flags;
-#define IBMVFC_TMF_ABORT_TASK		0x02
-#define IBMVFC_TMF_ABORT_TASK_SET	0x04
-#define IBMVFC_TMF_LUN_RESET		0x10
-#define IBMVFC_TMF_TGT_RESET		0x20
-#define IBMVFC_TMF_LUA_VALID		0x40
-#define IBMVFC_TMF_SUPPRESS_ABTS	0x80
+#define IBMVFC_TMF_ABORT_TASK		0x002
+#define IBMVFC_TMF_ABORT_TASK_SET	0x004
+#define IBMVFC_TMF_LUN_RESET		0x010
+#define IBMVFC_TMF_TGT_RESET		0x020
+#define IBMVFC_TMF_LUA_VALID		0x040
+#define IBMVFC_TMF_SUPPRESS_ABTS	0x080
+#define IBMVFC_TMF_NVMF_ASSOC		0x100
+#define IBMVFC_TMF_NVMF_TARGET		0x200
+#define IBMVFC_TMF_BITMASK_VALID	0x400
 	__be32 cancel_key;
 	__be32 my_cancel_key;
 	__be32 pad;
@@ -446,6 +495,8 @@ enum ibmvfc_cmd_flags {
 	IBMVFC_WRITE		= 0x0008,
 	IBMVFC_TMF			= 0x0080,
 	IBMVFC_CLASS_3_ERR	= 0x0100,
+	IBMVFC_NVMEOF_PROTOCOL	= 0x0200,
+	IBMVFC_NVMF_SLER	= 0x0400,
 };
 
 enum ibmvfc_fc_task_attr {
@@ -493,7 +544,7 @@ struct ibmvfc_cmd {
 	__be64 tgt_scsi_id;
 	__be64 tag;
 	__be64 target_wwpn;
-	__be64 reserved3;
+	__be64 assoc_id;
 	union {
 		struct {
 			struct ibmvfc_fcp_cmd_iu iu;
@@ -504,6 +555,15 @@ struct ibmvfc_cmd {
 			struct ibmvfc_fcp_cmd_iu iu;
 			struct ibmvfc_fcp_rsp rsp;
 		} v2;
+		struct {
+			__be64 reserved5[4];
+			struct ibmvfc_fcp_cmd_iu iu;
+			struct ibmvfc_fcp_rsp rsp;
+		} v3scsi;
+		struct {
+			__be64 reserved[4];
+			struct nvme_fc_cmd_iu iu;
+		} v3nvme;
 	};
 } __packed __aligned(8);
 
@@ -522,6 +582,9 @@ struct ibmvfc_passthru_iu {
 	__be32 flags;
 #define IBMVFC_FC_ELS		0x01
 #define IBMVFC_FC_CT_IU		0x02
+#define IBMVFC_PT_PRLI		0x04
+#define IBMVFC_FC4_LS_OTH	0x08
+#define IBMVFC_FC4_LS_DSC_CTRL	0x10
 	__be32 cancel_key;
 #define IBMVFC_PASSTHRU_CANCEL_KEY	0x80000000
 #define IBMVFC_INTERNAL_CANCEL_KEY	0x80000001
@@ -559,7 +622,7 @@ struct ibmvfc_channel_setup_mad {
 	struct srp_direct_buf buffer;
 } __packed __aligned(8);
 
-#define IBMVFC_MAX_CHANNELS	502
+#define IBMVFC_MAX_CHANNELS	501
 
 struct ibmvfc_channel_setup {
 	__be32 flags;
@@ -574,6 +637,7 @@ struct ibmvfc_channel_setup {
 	struct srp_direct_buf buffer;
 	__be64 reserved2[5];
 	__be64 channel_handles[IBMVFC_MAX_CHANNELS];
+	__be64 async_sub_crq_handle;
 } __packed __aligned(8);
 
 struct ibmvfc_connection_info {
@@ -620,7 +684,8 @@ struct ibmvfc_trace_entry {
 
 enum ibmvfc_crq_formats {
 	IBMVFC_CMD_FORMAT		= 0x01,
-	IBMVFC_ASYNC_EVENT	= 0x02,
+	IBMVFC_ASYNC_EVENT		= 0x02,
+	IBMVFC_NOOP			= 0x03,
 	IBMVFC_MAD_FORMAT		= 0x04,
 };
 
@@ -639,6 +704,9 @@ enum ibmvfc_async_event {
 	IBMVFC_AE_RESUME			= 0x0800,
 	IBMVFC_AE_ADAPTER_FAILED	= 0x1000,
 	IBMVFC_AE_FPIN			= 0x2000,
+	IBMVFC_NVME_DISCONNECT		= 0x4000,
+	IBMVFC_NVMEOF_PROTO_AVAIL	= 0x40000000,
+	IBMVFC_SCSI_PROTO_AVAIL		= 0x80000000,
 };
 
 struct ibmvfc_async_desc {
@@ -683,13 +751,30 @@ struct ibmvfc_async_crq {
 	volatile __be64 scsi_id;
 	volatile __be64 wwpn;
 	volatile __be64 node_name;
-	__be64 reserved;
+	__be64 assoc_id;
+} __packed __aligned(8);
+
+struct ibmvfc_async_sub_crq {
+	volatile u8 valid;
+	u8 flags;
+#define IBMVFC_ASYNC_ID_IS_ASSOC_ID	0x01
+	u8 link_state;
+	u8 fpin_status;
+	__be16 event;
+	__be16 pad;
+	__be64 wwpn;
+	__be64 nport_id;
+	union {
+		__be64 node_name;
+		__be64 assoc_id;
+	} id;
 } __packed __aligned(8);
 
 union ibmvfc_iu {
 	struct ibmvfc_mad_common mad_common;
 	struct ibmvfc_npiv_login_mad npiv_login;
 	struct ibmvfc_npiv_logout_mad npiv_logout;
+	struct ibmvfc_fabric_login_mad fabric_login;
 	struct ibmvfc_discover_targets discover_targets;
 	struct ibmvfc_port_login plogi;
 	struct ibmvfc_process_login prli;
-- 
2.54.0

[PATCH 03/29] ibmvfc: split NVMe support into separate source file and add transport stubs

[Tyrel Datwyler]

Rename ibmvfc.c to ibmvfc-core.c as first step in decoupling each
protocol from the core driver logic. Add ibmvfc-nvme.[ch] files, and
register an nvme_fc_port_template with empty callback stubs.

Add empty registration functions definitions for local and remote ports.

No functional NVMe/FC support is added yet.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/Makefile                |  2 +
 .../scsi/ibmvscsi/{ibmvfc.c => ibmvfc-core.c} | 19 +++-
 drivers/scsi/ibmvscsi/ibmvfc-nvme.c           | 87 +++++++++++++++++++
 drivers/scsi/ibmvscsi/ibmvfc-nvme.h           | 32 +++++++
 drivers/scsi/ibmvscsi/ibmvfc.h                | 10 ++-
 5 files changed, 146 insertions(+), 4 deletions(-)
 rename drivers/scsi/ibmvscsi/{ibmvfc.c => ibmvfc-core.c} (99%)
 create mode 100644 drivers/scsi/ibmvscsi/ibmvfc-nvme.c
 create mode 100644 drivers/scsi/ibmvscsi/ibmvfc-nvme.h

diff --git a/drivers/scsi/ibmvscsi/Makefile b/drivers/scsi/ibmvscsi/Makefile
index 5eb1cb1a0028..9408c7f4cdee 100644
--- a/drivers/scsi/ibmvscsi/Makefile
+++ b/drivers/scsi/ibmvscsi/Makefile
@@ -1,3 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
+ibmvfc-objs := ibmvfc-core.o ibmvfc-nvme.o
+
 obj-$(CONFIG_SCSI_IBMVSCSI)	+= ibmvscsi.o
 obj-$(CONFIG_SCSI_IBMVFC)	+= ibmvfc.o
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
similarity index 99%
rename from drivers/scsi/ibmvscsi/ibmvfc.c
rename to drivers/scsi/ibmvscsi/ibmvfc-core.c
index 912901436442..4e45d23221d6 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -1,10 +1,11 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
 /*
  * ibmvfc.c -- driver for IBM Power Virtual Fibre Channel Adapter
  *
  * Written By: Brian King <brking@linux.vnet.ibm.com>, IBM Corporation
  *
- * Copyright (C) IBM Corporation, 2008
+ * Copyright (C) IBM Corporation, 2008-2026
  */
 
 #include <linux/module.h>
@@ -46,6 +47,9 @@ static unsigned int cls3_error = IBMVFC_CLS3_ERROR;
 static unsigned int mq_enabled = IBMVFC_MQ;
 static unsigned int nr_scsi_hw_queues = IBMVFC_SCSI_HW_QUEUES;
 static unsigned int nr_scsi_channels = IBMVFC_SCSI_CHANNELS;
+static unsigned int nvme_enabled = IBMVFC_NVME;
+static unsigned int nr_nvme_hw_queues = IBMVFC_NVME_HW_QUEUES;
+static unsigned int nr_nvme_channels = IBMVFC_NVME_CHANNELS;
 static unsigned int mig_channels_only = IBMVFC_MIG_NO_SUB_TO_CRQ;
 static unsigned int mig_no_less_channels = IBMVFC_MIG_NO_N_TO_M;
 
@@ -74,6 +78,16 @@ module_param_named(mig_no_less_channels, mig_no_less_channels, uint, S_IRUGO);
 MODULE_PARM_DESC(mig_no_less_channels, "Prevent migration to system with less channels. "
 		 "[Default=" __stringify(IBMVFC_MIG_NO_N_TO_M) "]");
 
+module_param_named(nvme, nvme_enabled, uint, S_IRUGO);
+MODULE_PARM_DESC(nvme, "Enable NVMe over FC support. "
+		 "[Default=" __stringify(IBMVFC_NVME) "]");
+module_param_named(nvme_host_queues, nr_nvme_hw_queues, uint, S_IRUGO);
+MODULE_PARM_DESC(nvme_host_queues, "Number of NVMeoF Host submission queues. "
+		 "[Default=" __stringify(IBMVFC_NVME_HW_QUEUES) "]");
+module_param_named(nvme_hw_channels, nr_nvme_channels, uint, S_IRUGO);
+MODULE_PARM_DESC(nvme_hw_channels, "Number of hw NVMeoF channels to request. "
+		 "[Default=" __stringify(IBMVFC_NVME_CHANNELS) "]");
+
 module_param_named(init_timeout, init_timeout, uint, S_IRUGO | S_IWUSR);
 MODULE_PARM_DESC(init_timeout, "Initialization timeout in seconds. "
 		 "[Default=" __stringify(IBMVFC_INIT_TIMEOUT) "]");
@@ -468,6 +482,7 @@ static const struct {
 	{ IBMVFC_FABRIC_BUSY, "fabric busy" },
 	{ IBMVFC_PORT_BUSY, "port busy" },
 	{ IBMVFC_BASIC_REJECT, "basic reject" },
+	{ IBMVFC_FC4_LS_REJECT, "fc4 ls reject" },
 };
 
 static const char *unknown_fc_type = "unknown fc type";
diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
new file mode 100644
index 000000000000..4a66cde8a8d2
--- /dev/null
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
@@ -0,0 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+/*
+ * ibmvfc-nvme.c -- IBM Power Virtual Fibre Channel NVMeoF HBA driver
+ *
+ * Written By: Tyrel Datwyler <tyreld@linux.ibm.com>, IBM Corporation
+ *
+ * Copyright (C) IBM Corporation, 2026
+ */
+
+#include <scsi/scsi_transport_fc.h>
+
+#include "ibmvfc-nvme.h"
+
+static void ibmvfc_nvme_localport_delete(struct nvme_fc_local_port *lport)
+{
+}
+
+static void ibmvfc_nvme_remoteport_delete(struct nvme_fc_remote_port *rport)
+{
+}
+
+static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport,
+			      struct nvme_fc_remote_port *rport,
+			      struct nvmefc_ls_req *ls_req)
+{
+	return 0;
+}
+
+static void ibmvfc_nvme_ls_abort(struct nvme_fc_local_port *lport,
+				struct nvme_fc_remote_port *rport,
+				struct nvmefc_ls_req *ls_abort)
+{
+}
+
+static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport,
+			      struct nvme_fc_remote_port *rport,
+			      void *hw_queue_handle,
+			      struct nvmefc_fcp_req *fcp_req)
+{
+	return 0;
+}
+
+static void ibmvfc_nvme_fcp_abort(struct nvme_fc_local_port *lport,
+				  struct nvme_fc_remote_port *rport,
+				  void *hw_queue_handle,
+				  struct nvmefc_fcp_req *abort_req)
+{
+}
+
+static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = {
+	.localport_delete	= ibmvfc_nvme_localport_delete,
+	.remoteport_delete	= ibmvfc_nvme_remoteport_delete,
+	.create_queue		= NULL,
+	.delete_queue		= NULL,
+	.ls_req			= ibmvfc_nvme_ls_req,
+	.ls_abort		= ibmvfc_nvme_ls_abort,
+	.fcp_io			= ibmvfc_nvme_fcp_io,
+	.fcp_abort		= ibmvfc_nvme_fcp_abort,
+	.map_queues		= NULL,
+	.max_hw_queues		= IBMVFC_NVME_HW_QUEUES,
+	.max_sgl_segments	= 1024,
+	.max_dif_sgl_segments	= 64,
+	.dma_boundary		= 0xFFFFFFFF,
+	.local_priv_sz		= sizeof(struct ibmvfc_host *),
+	.remote_priv_sz		= sizeof(struct ibmvfc_target *),
+	.lsrqst_priv_sz		= sizeof(struct ibmvfc_event *),
+	.fcprqst_priv_sz	= sizeof(struct ibmvfc_event *),
+};
+
+int ibmvfc_nvme_register_remoteport(struct ibmvfc_target *tgt)
+{
+	return 0;
+}
+
+void ibmvfc_nvme_unregister_remoteport(struct ibmvfc_target *tgt)
+{
+}
+
+int ibmvfc_nvme_register(struct ibmvfc_host *vhost)
+{
+	return 0;
+}
+
+void ibmvfc_nvme_unregister(struct ibmvfc_host *vhost)
+{
+}
diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
new file mode 100644
index 000000000000..97e267871df2
--- /dev/null
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
@@ -0,0 +1,32 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+/*
+ * ibmvfc-nvme.h -- IBM Power Virtual Fibre Channel NVMeoF HBA driver
+ *
+ * Written By: Tyrel Datwyler <tyreld@linux.ibm.com>, IBM Corporation
+ *
+ * Copyright (C) IBM Corporation, 2026
+ */
+
+#ifndef _IBMVFC_NVME_H
+#define _IBMVFC_NVME_H
+
+#include <uapi/scsi/fc/fc_fs.h>
+#include <uapi/scsi/fc/fc_els.h>
+#include <linux/nvme-fc-driver.h>
+
+#include "ibmvfc.h"
+
+#define IBMVFC_NVME		0
+#define IBMVFC_NVME_HW_QUEUES	8
+#define IBMVFC_MAX_NVME_QUEUES	16
+#define IBMVFC_NVME_CHANNELS	8
+
+struct ibmvfc_host;
+struct ibmvfc_target;
+
+int ibmvfc_nvme_register_remoteport(struct ibmvfc_target *tgt);
+void ibmvfc_nvme_unregister_remoteport(struct ibmvfc_target *tgt);
+int ibmvfc_nvme_register(struct ibmvfc_host *vhost);
+void ibmvfc_nvme_unregister(struct ibmvfc_host *vhost);
+#endif
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index f8a2bf92da41..f137b61ce422 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -1,22 +1,27 @@
 /* SPDX-License-Identifier: GPL-2.0-or-later */
+
 /*
  * ibmvfc.h -- driver for IBM Power Virtual Fibre Channel Adapter
  *
  * Written By: Brian King <brking@linux.vnet.ibm.com>, IBM Corporation
  *
- * Copyright (C) IBM Corporation, 2008
+ * Copyright (C) IBM Corporation, 2008-2026
  */
 
 #ifndef _IBMVFC_H
 #define _IBMVFC_H
 
+#include <linux/interrupt.h>
 #include <linux/list.h>
 #include <linux/types.h>
+#include <scsi/scsi_device.h>
 #include <scsi/viosrp.h>
 #include <linux/nvme.h>
 #include <linux/nvme-fc.h>
 
-#define IBMVFC_NAME	"ibmvfc"
+#include "ibmvfc-nvme.h"
+
+#define IBMVFC_NAME			"ibmvfc"
 #define IBMVFC_DRIVER_VERSION		"1.0.11"
 #define IBMVFC_DRIVER_DATE		"(April 12, 2013)"
 
@@ -984,6 +989,7 @@ struct ibmvfc_host {
 	unsigned int mq_enabled:1;
 	unsigned int using_channels:1;
 	unsigned int do_enquiry:1;
+	unsigned int nvme_enabled:1;
 	unsigned int aborting_passthru:1;
 	unsigned int scan_complete:1;
 	int scan_timeout;
-- 
2.54.0

[PATCH 04/29] ibmvfc: initialize NVMe channel configuration during driver probe

[Tyrel Datwyler]

Initialize the host's NVMe channel-group state during probe.

Set up the NVMe channel list head, desired queue count, maximum queue
count, protocol identifier, and enablement state alongside the existing
SCSI channel-group initialization in ibmvfc_probe().

This prepares the driver with a NVMe/FC channel group that can will be
used by later patches for NVMe queue allocation, discovery buffers,
target management, and IO submission.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 8 +++++++-
 drivers/scsi/ibmvscsi/ibmvfc.h      | 1 +
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 4e45d23221d6..5732ccf2ac1c 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -6337,7 +6337,8 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id)
 	struct device *dev = &vdev->dev;
 	int rc = -ENOMEM;
 	unsigned int online_cpus = num_online_cpus();
-	unsigned int max_scsi_queues = min((unsigned int)IBMVFC_MAX_SCSI_QUEUES, online_cpus);
+	unsigned int max_scsi_queues = min_t(unsigned int, IBMVFC_MAX_SCSI_QUEUES, online_cpus);
+	unsigned int max_nvme_queues = min_t(unsigned int, IBMVFC_MAX_NVME_QUEUES, online_cpus);
 
 	ENTER;
 	shost = scsi_host_alloc(&driver_template, sizeof(*vhost));
@@ -6357,6 +6358,7 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id)
 
 	vhost = shost_priv(shost);
 	INIT_LIST_HEAD(&vhost->scsi_scrqs.targets);
+	INIT_LIST_HEAD(&vhost->nvme_scrqs.targets);
 	INIT_LIST_HEAD(&vhost->purge);
 	sprintf(vhost->name, IBMVFC_NAME);
 	vhost->host = shost;
@@ -6371,6 +6373,10 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id)
 	vhost->scsi_scrqs.protocol = IBMVFC_PROTO_SCSI;
 	vhost->using_channels = 0;
 	vhost->do_enquiry = 1;
+	vhost->nvme_enabled = mq_enabled ? nvme_enabled : 0;
+	vhost->nvme_scrqs.desired_queues = min(max_nvme_queues, nr_nvme_channels);
+	vhost->nvme_scrqs.max_queues = max_nvme_queues;
+	vhost->nvme_scrqs.protocol = IBMVFC_PROTO_NVME;
 	vhost->scan_timeout = 0;
 
 	strcpy(vhost->partition_name, "UNKNOWN");
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index f137b61ce422..01c49d81b135 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -970,6 +970,7 @@ struct ibmvfc_host {
 	struct ibmvfc_queue crq;
 	struct ibmvfc_queue async_crq;
 	struct ibmvfc_channels scsi_scrqs;
+	struct ibmvfc_channels nvme_scrqs;
 	struct ibmvfc_npiv_login login_info;
 	union ibmvfc_npiv_login_data *login_buf;
 	dma_addr_t login_buf_dma;
-- 
2.54.0

[PATCH 05/29] ibmvfc: alloc/dealloc sub-queues for nvme channels

[Tyrel Datwyler]

Allocate, register, deregister, and release NVMe subordinate CRQs
alongside the existing SCSI sub-CRQs.

Update the CRQ reset and re-enable paths to tear down and recreate NVMe
sub-queues, extend sub-CRQ initialization to allocate NVMe channels when
enabled, and release NVMe channel resources during adapter teardown.

This keeps the NVMe queue lifecycle aligned with the existing SCSI queue
lifecycle so both protocols are reset consistently across probe, remove,
and connection recovery.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 5732ccf2ac1c..6f5e8b3cbfc8 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -945,6 +945,7 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost)
 	unsigned long flags;
 
 	ibmvfc_dereg_sub_crqs(vhost, &vhost->scsi_scrqs);
+	ibmvfc_dereg_sub_crqs(vhost, &vhost->nvme_scrqs);
 
 	/* Re-enable the CRQ */
 	do {
@@ -964,6 +965,7 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost)
 	spin_unlock_irqrestore(vhost->host->host_lock, flags);
 
 	ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs);
+	ibmvfc_reg_sub_crqs(vhost, &vhost->nvme_scrqs);
 
 	return rc;
 }
@@ -983,6 +985,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost)
 	struct ibmvfc_queue *crq = &vhost->crq;
 
 	ibmvfc_dereg_sub_crqs(vhost, &vhost->scsi_scrqs);
+	ibmvfc_dereg_sub_crqs(vhost, &vhost->nvme_scrqs);
 
 	/* Close the CRQ */
 	do {
@@ -1016,6 +1019,7 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost)
 	spin_unlock_irqrestore(vhost->host->host_lock, flags);
 
 	ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs);
+	ibmvfc_reg_sub_crqs(vhost, &vhost->nvme_scrqs);
 
 	return rc;
 }
@@ -6109,6 +6113,13 @@ static void ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost)
 
 	ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs);
 
+	if (vhost->nvme_enabled) {
+		if (ibmvfc_alloc_channels(vhost, &vhost->nvme_scrqs))
+			vhost->nvme_enabled = 0;
+		else
+			ibmvfc_reg_sub_crqs(vhost, &vhost->nvme_scrqs);
+	}
+
 	LEAVE;
 }
 
@@ -6137,8 +6148,10 @@ static void ibmvfc_release_sub_crqs(struct ibmvfc_host *vhost)
 		return;
 
 	ibmvfc_dereg_sub_crqs(vhost, &vhost->scsi_scrqs);
-
 	ibmvfc_release_channels(vhost, &vhost->scsi_scrqs);
+
+	ibmvfc_dereg_sub_crqs(vhost, &vhost->nvme_scrqs);
+	ibmvfc_release_channels(vhost, &vhost->nvme_scrqs);
 	LEAVE;
 }
 
-- 
2.54.0

[PATCH 06/29] ibmvfc: add logic for protocol specific fabric logins

[Tyrel Datwyler]

Add support for the protocol-specific fabric login flow introduced by
the updated client/VIOS interface.

After NPIV login, a VIOS that advertises protocol-specific support
requires separate fabric login MADs for SCSI and NVMe/FC. Track whether
SCSI and NVMe/FC fabric login are needed, extend channel enquiry/setup
handling to negotiate both SCSI and NVMe queue counts, and issue the
appropriate fabric login MADs before target discovery begins.

Also update command layout selection so the driver uses the v3 command
format when the VIOS advertises NVMe/FC-capable framing.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 230 ++++++++++++++++++++++++----
 drivers/scsi/ibmvscsi/ibmvfc.h      |  11 +-
 2 files changed, 206 insertions(+), 35 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 6f5e8b3cbfc8..93c32fa162f8 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -210,7 +210,9 @@ static int ibmvfc_check_caps(struct ibmvfc_host *vhost, unsigned long cap_flags)
 static struct ibmvfc_fcp_cmd_iu *ibmvfc_get_fcp_iu(struct ibmvfc_host *vhost,
 						   struct ibmvfc_cmd *vfc_cmd)
 {
-	if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN))
+	if (ibmvfc_check_caps(vhost, IBMVFC_SUPPORT_NVMEOF))
+		return &vfc_cmd->v3scsi.iu;
+	else if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN))
 		return &vfc_cmd->v2.iu;
 	else
 		return &vfc_cmd->v1.iu;
@@ -219,7 +221,9 @@ static struct ibmvfc_fcp_cmd_iu *ibmvfc_get_fcp_iu(struct ibmvfc_host *vhost,
 static struct ibmvfc_fcp_rsp *ibmvfc_get_fcp_rsp(struct ibmvfc_host *vhost,
 						 struct ibmvfc_cmd *vfc_cmd)
 {
-	if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN))
+	if (ibmvfc_check_caps(vhost, IBMVFC_SUPPORT_NVMEOF))
+		return &vfc_cmd->v3scsi.rsp;
+	else if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN))
 		return &vfc_cmd->v2.rsp;
 	else
 		return &vfc_cmd->v1.rsp;
@@ -961,6 +965,8 @@ static int ibmvfc_reenable_crq_queue(struct ibmvfc_host *vhost)
 	spin_lock(vhost->crq.q_lock);
 	vhost->do_enquiry = 1;
 	vhost->using_channels = 0;
+	vhost->do_scsi_login = 0;
+	vhost->do_nvme_login = 0;
 	spin_unlock(vhost->crq.q_lock);
 	spin_unlock_irqrestore(vhost->host->host_lock, flags);
 
@@ -1000,6 +1006,8 @@ static int ibmvfc_reset_crq(struct ibmvfc_host *vhost)
 	vhost->logged_in = 0;
 	vhost->do_enquiry = 1;
 	vhost->using_channels = 0;
+	vhost->do_scsi_login = 0;
+	vhost->do_nvme_login = 0;
 
 	/* Clean out the queue */
 	memset(crq->msgs.crq, 0, PAGE_SIZE);
@@ -1512,7 +1520,7 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost)
 	max_cmds = scsi_qdepth + IBMVFC_NUM_INTERNAL_REQ;
 	if (mq_enabled)
 		max_cmds += (scsi_qdepth + IBMVFC_NUM_INTERNAL_SUBQ_REQ) *
-			vhost->scsi_scrqs.desired_queues;
+			(vhost->scsi_scrqs.desired_queues + vhost->nvme_scrqs.desired_queues);
 
 	memset(login_info, 0, sizeof(*login_info));
 
@@ -1530,8 +1538,14 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost)
 	login_info->max_cmds = cpu_to_be32(max_cmds);
 	login_info->capabilities = cpu_to_be64(IBMVFC_CAN_MIGRATE | IBMVFC_CAN_SEND_VF_WWPN);
 
-	if (vhost->mq_enabled || vhost->using_channels)
+	if (vhost->mq_enabled || vhost->using_channels) {
 		login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_USE_CHANNELS);
+		if (vhost->nvme_enabled) {
+			login_info->capabilities |= cpu_to_be64(IBMVFC_YES_NVMEOF);
+			login_info->capabilities |= cpu_to_be64(IBMVFC_YES_SCSI);
+			login_info->capabilities |= cpu_to_be64(IBMVFC_CAN_USE_WWPN_ALL);
+		}
+	}
 
 	login_info->async.va = cpu_to_be64(vhost->async_crq.msg_token);
 	login_info->async.len = cpu_to_be32(async_crq->size *
@@ -1954,11 +1968,13 @@ static struct ibmvfc_cmd *ibmvfc_init_vfc_cmd(struct ibmvfc_event *evt, struct s
 	size_t offset;
 
 	memset(vfc_cmd, 0, sizeof(*vfc_cmd));
-	if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN)) {
+	if (ibmvfc_check_caps(vhost, IBMVFC_SUPPORT_NVMEOF))
+		offset = offsetof(struct ibmvfc_cmd, v3scsi.rsp);
+	else if (ibmvfc_check_caps(vhost, IBMVFC_HANDLE_VF_WWPN))
 		offset = offsetof(struct ibmvfc_cmd, v2.rsp);
-		vfc_cmd->target_wwpn = cpu_to_be64(rport->port_name);
-	} else
+	else
 		offset = offsetof(struct ibmvfc_cmd, v1.rsp);
+	vfc_cmd->target_wwpn = cpu_to_be64(rport->port_name);
 	vfc_cmd->resp.va = cpu_to_be64(be64_to_cpu(evt->crq.ioba) + offset);
 	vfc_cmd->resp.len = cpu_to_be32(sizeof(*rsp));
 	vfc_cmd->frame_type = cpu_to_be32(IBMVFC_SCSI_FCP_TYPE);
@@ -5042,14 +5058,141 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)
 		ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD);
 }
 
+static void ibmvfc_fabric_login_nvme_done(struct ibmvfc_event *evt)
+{
+	struct ibmvfc_host *vhost = evt->vhost;
+	struct ibmvfc_fabric_login_mad *rsp = &evt->xfer_iu->fabric_login;
+	u32 mad_status = be16_to_cpu(rsp->common.status);
+	int level = IBMVFC_DEFAULT_LOG_LEVEL;
+
+	switch (mad_status) {
+	case IBMVFC_MAD_SUCCESS:
+		ibmvfc_dbg(vhost, "NVMe fabric login succeeded\n");
+		break;
+	case IBMVFC_MAD_FAILED:
+		if (ibmvfc_retry_cmd(be16_to_cpu(rsp->status), be16_to_cpu(rsp->error)))
+			level += ibmvfc_retry_host_init(vhost);
+		else
+			ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD);
+		ibmvfc_log(vhost, level, "NVMe fabric login failed: %s (%x:%x)\n",
+			   ibmvfc_get_cmd_error(be16_to_cpu(rsp->status), be16_to_cpu(rsp->error)),
+			   be16_to_cpu(rsp->status), be16_to_cpu(rsp->error));
+		ibmvfc_free_event(evt);
+		return;
+	case IBMVFC_MAD_DRIVER_FAILED:
+		ibmvfc_free_event(evt);
+		return;
+	default:
+		dev_err(vhost->dev, "Invalid NVMe fabric login response: 0x%x\n", mad_status);
+		ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD);
+		break;
+	}
+
+	ibmvfc_free_event(evt);
+
+	ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY);
+	wake_up(&vhost->work_wait_q);
+}
+
+static void ibmvfc_fabric_login_nvme(struct ibmvfc_host *vhost)
+{
+	struct ibmvfc_fabric_login_mad *mad;
+	struct ibmvfc_event *evt = ibmvfc_get_reserved_event(&vhost->crq);
+
+	if (!evt) {
+		ibmvfc_retry_host_init(vhost);
+		return;
+	}
+
+	ibmvfc_init_event(evt, ibmvfc_fabric_login_nvme_done, IBMVFC_MAD_FORMAT);
+	mad = &evt->iu.fabric_login;
+	memset(mad, 0, sizeof(*mad));
+	mad->common.version = cpu_to_be32(1);
+	mad->common.opcode = cpu_to_be32(IBMVFC_NVMF_FABRIC_LOGIN);
+	mad->common.length = cpu_to_be16(sizeof(*mad));
+
+	ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT);
+
+	if (!ibmvfc_send_event(evt, vhost, default_timeout))
+		ibmvfc_dbg(vhost, "Send NVMe fabric login\n");
+	else
+		ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD);
+}
+
+static void ibmvfc_fabric_login_scsi_done(struct ibmvfc_event *evt)
+{
+	struct ibmvfc_host *vhost = evt->vhost;
+	struct ibmvfc_fabric_login_mad *rsp = &evt->xfer_iu->fabric_login;
+	u32 mad_status = be16_to_cpu(rsp->common.status);
+	int level = IBMVFC_DEFAULT_LOG_LEVEL;
+
+	switch (mad_status) {
+	case IBMVFC_MAD_SUCCESS:
+		ibmvfc_dbg(vhost, "SCSI fabric login succeeded\n");
+		break;
+	case IBMVFC_MAD_FAILED:
+		if (ibmvfc_retry_cmd(be16_to_cpu(rsp->status), be16_to_cpu(rsp->error)))
+			level += ibmvfc_retry_host_init(vhost);
+		else
+			ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD);
+		ibmvfc_log(vhost, level, "SCSI fabric login failed: %s (%x:%x)\n",
+			   ibmvfc_get_cmd_error(be16_to_cpu(rsp->status), be16_to_cpu(rsp->error)),
+			   be16_to_cpu(rsp->status), be16_to_cpu(rsp->error));
+		ibmvfc_free_event(evt);
+		return;
+	case IBMVFC_MAD_DRIVER_FAILED:
+		ibmvfc_free_event(evt);
+		return;
+	default:
+		dev_err(vhost->dev, "Invalid SCSI fabric login response: 0x%x\n", mad_status);
+		ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD);
+		break;
+	}
+
+	ibmvfc_free_event(evt);
+
+	if (vhost->do_nvme_login) {
+		ibmvfc_fabric_login_nvme(vhost);
+	} else {
+		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY);
+		wake_up(&vhost->work_wait_q);
+	}
+}
+
+static void ibmvfc_fabric_login_scsi(struct ibmvfc_host *vhost)
+{
+	struct ibmvfc_fabric_login_mad *mad;
+	struct ibmvfc_event *evt = ibmvfc_get_reserved_event(&vhost->crq);
+
+	if (!evt) {
+		ibmvfc_retry_host_init(vhost);
+		return;
+	}
+
+	ibmvfc_init_event(evt, ibmvfc_fabric_login_scsi_done, IBMVFC_MAD_FORMAT);
+	mad = &evt->iu.fabric_login;
+	memset(mad, 0, sizeof(*mad));
+	mad->common.version = cpu_to_be32(1);
+	mad->common.opcode = cpu_to_be32(IBMVFC_FABRIC_LOGIN);
+	mad->common.length = cpu_to_be16(sizeof(*mad));
+
+	ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT);
+
+	if (!ibmvfc_send_event(evt, vhost, default_timeout))
+		ibmvfc_dbg(vhost, "Send SCSI fabric login\n");
+	else
+		ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD);
+}
+
 static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt)
 {
 	struct ibmvfc_host *vhost = evt->vhost;
 	struct ibmvfc_channel_setup *setup = vhost->channel_setup_buf;
-	struct ibmvfc_channels *scrqs = &vhost->scsi_scrqs;
+	struct ibmvfc_channels *scsi = &vhost->scsi_scrqs;
+	struct ibmvfc_channels *nvme = &vhost->nvme_scrqs;
 	u32 mad_status = be16_to_cpu(evt->xfer_iu->channel_setup.common.status);
 	int level = IBMVFC_DEFAULT_LOG_LEVEL;
-	int flags, active_queues, i;
+	int flags, i;
 
 	ibmvfc_free_event(evt);
 
@@ -5058,22 +5201,28 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt)
 		ibmvfc_dbg(vhost, "Channel Setup succeeded\n");
 		flags = be32_to_cpu(setup->flags);
 		vhost->do_enquiry = 0;
-		active_queues = be32_to_cpu(setup->num_scsi_subq_channels);
-		scrqs->active_queues = active_queues;
+		scsi->active_queues = be32_to_cpu(setup->num_scsi_subq_channels);
+		nvme->active_queues = be32_to_cpu(setup->num_nvme_subq_channels);
 
 		if (flags & IBMVFC_CHANNELS_CANCELED) {
 			ibmvfc_dbg(vhost, "Channels Canceled\n");
 			vhost->using_channels = 0;
-		} else {
-			if (active_queues)
-				vhost->using_channels = 1;
-			for (i = 0; i < active_queues; i++)
-				scrqs->scrqs[i].vios_cookie =
-					be64_to_cpu(setup->channel_handles[i]);
-
-			ibmvfc_dbg(vhost, "Using %u channels\n",
-				   vhost->scsi_scrqs.active_queues);
+			break;
 		}
+
+		if (scsi->active_queues || nvme->active_queues)
+			vhost->using_channels = 1;
+		for (i = 0; i < scsi->active_queues; i++)
+			scsi->scrqs[i].vios_cookie =
+				be64_to_cpu(setup->channel_handles[i]);
+		for (i = 0; i < nvme->active_queues; i++)
+			nvme->scrqs[i].vios_cookie =
+				be64_to_cpu(setup->channel_handles[scsi->active_queues + i]);
+
+		ibmvfc_dbg(vhost, "Using %u SCSI channels\n",
+			   scsi->active_queues);
+		ibmvfc_dbg(vhost, "Using %u NVMe channels\n",
+			   nvme->active_queues);
 		break;
 	case IBMVFC_MAD_FAILED:
 		level += ibmvfc_retry_host_init(vhost);
@@ -5088,8 +5237,12 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt)
 		return;
 	}
 
-	ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY);
-	wake_up(&vhost->work_wait_q);
+	if (vhost->do_scsi_login) {
+		ibmvfc_fabric_login_scsi(vhost);
+	} else {
+		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY);
+		wake_up(&vhost->work_wait_q);
+	}
 }
 
 static void ibmvfc_channel_setup(struct ibmvfc_host *vhost)
@@ -5097,9 +5250,12 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost)
 	struct ibmvfc_channel_setup_mad *mad;
 	struct ibmvfc_channel_setup *setup_buf = vhost->channel_setup_buf;
 	struct ibmvfc_event *evt = ibmvfc_get_reserved_event(&vhost->crq);
-	struct ibmvfc_channels *scrqs = &vhost->scsi_scrqs;
-	unsigned int num_channels =
-		min(scrqs->desired_queues, vhost->max_vios_scsi_channels);
+	struct ibmvfc_channels *scsi = &vhost->scsi_scrqs;
+	struct ibmvfc_channels *nvme = &vhost->nvme_scrqs;
+	unsigned int scsi_channels =
+		min(scsi->desired_queues, vhost->max_vios_scsi_channels);
+	unsigned int nvme_channels =
+		min(nvme->desired_queues, vhost->max_vios_nvme_channels);
 	int level = IBMVFC_DEFAULT_LOG_LEVEL;
 	int i;
 
@@ -5110,12 +5266,17 @@ static void ibmvfc_channel_setup(struct ibmvfc_host *vhost)
 	}
 
 	memset(setup_buf, 0, sizeof(*setup_buf));
-	if (num_channels == 0)
+	if (!scsi_channels && !nvme_channels)
 		setup_buf->flags = cpu_to_be32(IBMVFC_CANCEL_CHANNELS);
 	else {
-		setup_buf->num_scsi_subq_channels = cpu_to_be32(num_channels);
-		for (i = 0; i < num_channels; i++)
-			setup_buf->channel_handles[i] = cpu_to_be64(scrqs->scrqs[i].cookie);
+		setup_buf->num_scsi_subq_channels = cpu_to_be32(scsi_channels);
+		setup_buf->num_nvme_subq_channels = cpu_to_be32(nvme_channels);
+		for (i = 0; i < scsi_channels; i++)
+			setup_buf->channel_handles[i] =
+				cpu_to_be64(scsi->scrqs[i].cookie);
+		for (i = 0; i < nvme_channels; i++)
+			setup_buf->channel_handles[scsi_channels + i] =
+				cpu_to_be64(nvme->scrqs[i].cookie);
 	}
 
 	ibmvfc_init_event(evt, ibmvfc_channel_setup_done, IBMVFC_MAD_FORMAT);
@@ -5146,6 +5307,7 @@ static void ibmvfc_channel_enquiry_done(struct ibmvfc_event *evt)
 	case IBMVFC_MAD_SUCCESS:
 		ibmvfc_dbg(vhost, "Channel Enquiry succeeded\n");
 		vhost->max_vios_scsi_channels = be32_to_cpu(rsp->num_scsi_subq_channels);
+		vhost->max_vios_nvme_channels = be32_to_cpu(rsp->num_nvme_subq_channels);
 		ibmvfc_free_event(evt);
 		break;
 	case IBMVFC_MAD_FAILED:
@@ -5280,8 +5442,14 @@ static void ibmvfc_npiv_login_done(struct ibmvfc_event *evt)
 	vhost->host->can_queue = be32_to_cpu(rsp->max_cmds) - IBMVFC_NUM_INTERNAL_REQ;
 	vhost->host->max_sectors = npiv_max_sectors;
 
-	if (ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPORT_CHANNELS) && vhost->do_enquiry) {
-		ibmvfc_channel_enquiry(vhost);
+
+	if (ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPORT_CHANNELS)) {
+		if (ibmvfc_check_caps(vhost, IBMVFC_SUPPORT_SCSI))
+			vhost->do_scsi_login = 1;
+		if (ibmvfc_check_caps(vhost, IBMVFC_SUPPORT_NVMEOF))
+			vhost->do_nvme_login = 1;
+		if (vhost->do_enquiry)
+			ibmvfc_channel_enquiry(vhost);
 	} else {
 		vhost->do_enquiry = 0;
 		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY);
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index 01c49d81b135..454968de925b 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -617,9 +617,9 @@ struct ibmvfc_channel_enquiry {
 #define IBMVFC_SUPPORT_VARIABLE_SUBQ_MSG	0x02
 #define IBMVFC_NO_N_TO_M_CHANNELS_SUPPORT	0x04
 	__be32 num_scsi_subq_channels;
-	__be32 num_nvmeof_subq_channels;
+	__be32 num_nvme_subq_channels;
 	__be32 num_scsi_vas_channels;
-	__be32 num_nvmeof_vas_channels;
+	__be32 num_nvme_vas_channels;
 } __packed __aligned(8);
 
 struct ibmvfc_channel_setup_mad {
@@ -636,9 +636,9 @@ struct ibmvfc_channel_setup {
 #define IBMVFC_CHANNELS_CANCELED	0x04
 	__be32 reserved;
 	__be32 num_scsi_subq_channels;
-	__be32 num_nvmeof_subq_channels;
+	__be32 num_nvme_subq_channels;
 	__be32 num_scsi_vas_channels;
-	__be32 num_nvmeof_vas_channels;
+	__be32 num_nvme_vas_channels;
 	struct srp_direct_buf buffer;
 	__be64 reserved2[5];
 	__be64 channel_handles[IBMVFC_MAX_CHANNELS];
@@ -979,6 +979,7 @@ struct ibmvfc_host {
 	int log_level;
 	struct mutex passthru_mutex;
 	unsigned int max_vios_scsi_channels;
+	unsigned int max_vios_nvme_channels;
 	int task_set;
 	int init_retries;
 	int discovery_threads;
@@ -991,6 +992,8 @@ struct ibmvfc_host {
 	unsigned int using_channels:1;
 	unsigned int do_enquiry:1;
 	unsigned int nvme_enabled:1;
+	unsigned int do_scsi_login:1;
+	unsigned int do_nvme_login:1;
 	unsigned int aborting_passthru:1;
 	unsigned int scan_complete:1;
 	int scan_timeout;
-- 
2.54.0

[PATCH 07/29] ibmvfc: add wrapper to get vhost associated with a channel struct

[Tyrel Datwyler]

Add ibmvfc_channels_to_vhost() to recover the parent struct ibmvfc_host
from a protocol-specific struct ibmvfc_channels.

Later patches need to operate on either the SCSI or NVMe channel group
and still access host-wide state such as the primary CRQ, device, and
logging context. Centralize that mapping in a helper instead of open-
coding container lookups at each call site.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index 454968de925b..67f546ff092e 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -1011,6 +1011,16 @@ struct ibmvfc_host {
 	wait_queue_head_t work_wait_q;
 };
 
+static inline struct ibmvfc_host *ibmvfc_channels_to_vhost(struct ibmvfc_channels *channels)
+{
+	if (channels->protocol == IBMVFC_PROTO_SCSI)
+		return container_of(channels, struct ibmvfc_host, scsi_scrqs);
+	else if (channels->protocol == IBMVFC_PROTO_NVME)
+		return container_of(channels, struct ibmvfc_host, nvme_scrqs);
+
+	return NULL;
+}
+
 #define DBG_CMD(CMD) do { if (ibmvfc_debug) CMD; } while (0)
 
 #define tgt_dbg(t, fmt, ...)			\
-- 
2.54.0

[PATCH 08/29] ibmvfc: add helper for creating protocol specific discovery event

[Tyrel Datwyler]

Refactor discover-target event creation so it can be shared by both SCSI
and NVMe/FC discovery.

Introduce a helper that takes a protocol-specific channel group, selects
the correct discover-target opcode, and maps the corresponding discovery
buffer into the MAD.

This is a preparatory cleanup for issuing protocol-specific discovery
MADs in later patches.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 39 ++++++++++++++++++++---------
 1 file changed, 27 insertions(+), 12 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 93c32fa162f8..8186e9321af5 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -5023,6 +5023,32 @@ static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt)
 	wake_up(&vhost->work_wait_q);
 }
 
+static struct ibmvfc_event *ibmvfc_get_disc_event(struct ibmvfc_channels *channels)
+{
+	struct ibmvfc_discover_targets *mad;
+	struct ibmvfc_host *vhost = ibmvfc_channels_to_vhost(channels);
+	struct ibmvfc_event *evt = ibmvfc_get_reserved_event(&vhost->crq);
+
+	if (!evt)
+		return NULL;
+
+	ibmvfc_init_event(evt, ibmvfc_discover_targets_done, IBMVFC_MAD_FORMAT);
+	mad = &evt->iu.discover_targets;
+	memset(mad, 0, sizeof(*mad));
+	mad->common.version = cpu_to_be32(1);
+	if (channels->protocol == IBMVFC_PROTO_SCSI)
+		mad->common.opcode = cpu_to_be32(IBMVFC_DISC_TARGETS);
+	else
+		mad->common.opcode = cpu_to_be32(IBMVFC_DISC_NVMF_TARGETS);
+	mad->common.length = cpu_to_be16(sizeof(*mad));
+	mad->bufflen = cpu_to_be32(channels->disc_buf_sz);
+	mad->buffer.va = cpu_to_be64(channels->disc_buf_dma);
+	mad->buffer.len = cpu_to_be32(channels->disc_buf_sz);
+	mad->flags = cpu_to_be32(IBMVFC_DISC_TGT_PORT_ID_WWPN_LIST);
+
+	return evt;
+}
+
 /**
  * ibmvfc_discover_targets - Send Discover Targets MAD
  * @vhost:	ibmvfc host struct
@@ -5030,8 +5056,7 @@ static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt)
  **/
 static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)
 {
-	struct ibmvfc_discover_targets *mad;
-	struct ibmvfc_event *evt = ibmvfc_get_reserved_event(&vhost->crq);
+	struct ibmvfc_event *evt = ibmvfc_get_disc_event(&vhost->scsi_scrqs);
 	int level = IBMVFC_DEFAULT_LOG_LEVEL;
 
 	if (!evt) {
@@ -5040,16 +5065,6 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)
 		return;
 	}
 
-	ibmvfc_init_event(evt, ibmvfc_discover_targets_done, IBMVFC_MAD_FORMAT);
-	mad = &evt->iu.discover_targets;
-	memset(mad, 0, sizeof(*mad));
-	mad->common.version = cpu_to_be32(1);
-	mad->common.opcode = cpu_to_be32(IBMVFC_DISC_TARGETS);
-	mad->common.length = cpu_to_be16(sizeof(*mad));
-	mad->bufflen = cpu_to_be32(vhost->scsi_scrqs.disc_buf_sz);
-	mad->buffer.va = cpu_to_be64(vhost->scsi_scrqs.disc_buf_dma);
-	mad->buffer.len = cpu_to_be32(vhost->scsi_scrqs.disc_buf_sz);
-	mad->flags = cpu_to_be32(IBMVFC_DISC_TGT_PORT_ID_WWPN_LIST);
 	ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT);
 
 	if (!ibmvfc_send_event(evt, vhost, default_timeout))
-- 
2.54.0

[PATCH 09/29] ibmvfc: add helper to check NVMe/FC support with active channels

[Tyrel Datwyler]

It can be the case that NVMeoF is enabled on both the client and VIOS,
but no queues are configured making the need to do NVMe target discovery
pointless. Add a helper to short hand the capabilities check and active
NVMe queue check.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 8186e9321af5..9cd688762150 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -207,6 +207,12 @@ static int ibmvfc_check_caps(struct ibmvfc_host *vhost, unsigned long cap_flags)
 	return (host_caps & cap_flags) ? 1 : 0;
 }
 
+static int ibmvfc_nvme_active(struct ibmvfc_host *vhost)
+{
+	return (ibmvfc_check_caps(vhost, IBMVFC_SUPPORT_NVMEOF) &&
+		vhost->nvme_scrqs.active_queues);
+}
+
 static struct ibmvfc_fcp_cmd_iu *ibmvfc_get_fcp_iu(struct ibmvfc_host *vhost,
 						   struct ibmvfc_cmd *vfc_cmd)
 {
-- 
2.54.0

[PATCH 10/29] ibmvfc: allocate and free NVMe channel group discover buffer

[Tyrel Datwyler]

Allocate a discovery buffer for the NVMe channel group and free it on
all teardown and error paths.

The existing discovery-buffer allocation only covered the SCSI channel
group. This patch is prepratory for sending NVMe/FC target discovery
MAD.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 9cd688762150..efac82c48258 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -6365,6 +6365,7 @@ static void ibmvfc_free_mem(struct ibmvfc_host *vhost)
 	mempool_destroy(vhost->tgt_pool);
 	kfree(vhost->trace);
 	ibmvfc_free_disc_buf(vhost->dev, &vhost->scsi_scrqs);
+	ibmvfc_free_disc_buf(vhost->dev, &vhost->nvme_scrqs);
 	dma_free_coherent(vhost->dev, sizeof(*vhost->login_buf),
 			  vhost->login_buf, vhost->login_buf_dma);
 	dma_free_coherent(vhost->dev, sizeof(*vhost->channel_setup_buf),
@@ -6427,12 +6428,15 @@ static int ibmvfc_alloc_mem(struct ibmvfc_host *vhost)
 	if (ibmvfc_alloc_disc_buf(dev, &vhost->scsi_scrqs))
 		goto free_login_buffer;
 
+	if (ibmvfc_alloc_disc_buf(dev, &vhost->nvme_scrqs))
+		goto free_scsi_disc_buffer;
+
 	vhost->trace = kzalloc_objs(struct ibmvfc_trace_entry,
 				    IBMVFC_NUM_TRACE_ENTRIES);
 	atomic_set(&vhost->trace_index, -1);
 
 	if (!vhost->trace)
-		goto free_scsi_disc_buffer;
+		goto free_nvme_disc_buffer;
 
 	vhost->tgt_pool = mempool_create_kmalloc_pool(IBMVFC_TGT_MEMPOOL_SZ,
 						      sizeof(struct ibmvfc_target));
@@ -6458,6 +6462,8 @@ static int ibmvfc_alloc_mem(struct ibmvfc_host *vhost)
 	mempool_destroy(vhost->tgt_pool);
 free_trace:
 	kfree(vhost->trace);
+free_nvme_disc_buffer:
+	ibmvfc_free_disc_buf(dev, &vhost->nvme_scrqs);
 free_scsi_disc_buffer:
 	ibmvfc_free_disc_buf(dev, &vhost->scsi_scrqs);
 free_login_buffer:
-- 
2.54.0

[PATCH 11/29] ibmvfc: send NVMe target discovery MAD

[Tyrel Datwyler]

Extend target discovery to send protocol-specific discover-target MADs
for NVMe/FC.

Use the protocol-aware discovery helper to build an NVMe discover-target
request, submit it when NVMe/FC support is active, and process the
returned target count using the NVMe channel group's discovery buffer.

This allows the driver to discover NVMe/FC targets in parallel with the
existing SCSI discovery flow while keeping protocol-specific target data
separate.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 50 +++++++++++++++++++++++++----
 1 file changed, 43 insertions(+), 7 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index efac82c48258..53480d150042 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -173,6 +173,11 @@ static const struct {
 	{ IBMVFC_FC_SCSI_ERROR, IBMVFC_COMMAND_FAILED, DID_ERROR, 0, 1, "PRLI to device failed." },
 };
 
+const char *proto_type[] = {
+	"SCSI",
+	"NVMe",
+};
+
 static void ibmvfc_npiv_login(struct ibmvfc_host *);
 static void ibmvfc_tgt_send_prli(struct ibmvfc_target *);
 static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *);
@@ -5001,19 +5006,30 @@ static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt)
 {
 	struct ibmvfc_host *vhost = evt->vhost;
 	struct ibmvfc_discover_targets *rsp = &evt->xfer_iu->discover_targets;
+	struct ibmvfc_channels *channels;
 	u32 mad_status = be16_to_cpu(rsp->common.status);
+	u32 opcode = be32_to_cpu(rsp->common.opcode);
 	int level = IBMVFC_DEFAULT_LOG_LEVEL;
 
+	if (opcode == IBMVFC_DISC_TARGETS)
+		channels = &vhost->scsi_scrqs;
+	else
+		channels = &vhost->nvme_scrqs;
+
 	switch (mad_status) {
 	case IBMVFC_MAD_SUCCESS:
-		ibmvfc_dbg(vhost, "Discover Targets succeeded\n");
-		vhost->scsi_scrqs.num_targets = min_t(u32, be32_to_cpu(rsp->num_written),
-						      max_targets);
+		ibmvfc_dbg(vhost, "Discover %s Targets succeeded\n",
+			   proto_type[channels->protocol]);
+		channels->num_targets = min_t(u32, be32_to_cpu(rsp->num_written),
+					      max_targets);
+		ibmvfc_dbg(vhost, "%d %s targets found\n", channels->num_targets,
+			   proto_type[channels->protocol]);
 		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_ALLOC_TGTS);
 		break;
 	case IBMVFC_MAD_FAILED:
 		level += ibmvfc_retry_host_init(vhost);
-		ibmvfc_log(vhost, level, "Discover Targets failed: %s (%x:%x)\n",
+		ibmvfc_log(vhost, level, "Discover %s Targets failed: %s (%x:%x)\n",
+			   proto_type[channels->protocol],
 			   ibmvfc_get_cmd_error(be16_to_cpu(rsp->status), be16_to_cpu(rsp->error)),
 			   be16_to_cpu(rsp->status), be16_to_cpu(rsp->error));
 		break;
@@ -5066,7 +5082,7 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)
 	int level = IBMVFC_DEFAULT_LOG_LEVEL;
 
 	if (!evt) {
-		ibmvfc_log(vhost, level, "Discover Targets failed: no available events\n");
+		ibmvfc_log(vhost, level, "Discover SCSI Targets failed: no available events\n");
 		ibmvfc_hard_reset_host(vhost);
 		return;
 	}
@@ -5074,9 +5090,29 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)
 	ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT);
 
 	if (!ibmvfc_send_event(evt, vhost, default_timeout))
-		ibmvfc_dbg(vhost, "Sent discover targets\n");
+		ibmvfc_dbg(vhost, "Sent discover SCSI targets\n");
 	else
-		ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD);
+		goto link_down;
+
+	if (!ibmvfc_nvme_active(vhost))
+		return;
+
+	evt = ibmvfc_get_disc_event(&vhost->nvme_scrqs);
+	if (!evt) {
+		ibmvfc_log(vhost, level, "Discover NVMe Targets failed: no available events\n");
+		ibmvfc_hard_reset_host(vhost);
+		return;
+	}
+
+	if (!ibmvfc_send_event(evt, vhost, default_timeout))
+		ibmvfc_dbg(vhost, "Sent discover NVMe targets\n");
+	else
+		goto link_down;
+
+	return;
+
+link_down:
+	ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD);
 }
 
 static void ibmvfc_fabric_login_nvme_done(struct ibmvfc_event *evt)
-- 
2.54.0

[PATCH 12/29] ibmvfc: add NVMe/FC Implicit Logout and Move Login support

[Tyrel Datwyler]

Add protocol-specific handling for implicit logout and move-login
operations on NVMe/FC targets.

Select the NVMe/FC-specific implicit logout opcode when operating on an
NVMe target and update the associated logging so protocol-specific
operations are visible in debug output. This extends the existing target
relogin and migration-related flows to work with NVMe targets as well as
SCSI targets.

These changes are needed so target reauthentication and target movement
continue to work once NVMe/FC targets are added to the driver's state
machine.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 33 +++++++++++++++++++----------
 1 file changed, 22 insertions(+), 11 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 53480d150042..3e3d77e0d517 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -4316,7 +4316,7 @@ static void ibmvfc_tgt_implicit_logout_done(struct ibmvfc_event *evt)
 
 	switch (status) {
 	case IBMVFC_MAD_SUCCESS:
-		tgt_dbg(tgt, "Implicit Logout succeeded\n");
+		tgt_dbg(tgt, "%s Implicit Logout succeeded\n", proto_type[tgt->protocol]);
 		break;
 	case IBMVFC_MAD_DRIVER_FAILED:
 		kref_put(&tgt->kref, ibmvfc_release_tgt);
@@ -4324,7 +4324,8 @@ static void ibmvfc_tgt_implicit_logout_done(struct ibmvfc_event *evt)
 		return;
 	case IBMVFC_MAD_FAILED:
 	default:
-		tgt_err(tgt, "Implicit Logout failed: rc=0x%02X\n", status);
+		tgt_err(tgt, "%s Implicit Logout failed: rc=0x%02X\n",
+			proto_type[tgt->protocol], status);
 		break;
 	}
 
@@ -4357,7 +4358,10 @@ static struct ibmvfc_event *__ibmvfc_tgt_get_implicit_logout_evt(struct ibmvfc_t
 	mad = &evt->iu.implicit_logout;
 	memset(mad, 0, sizeof(*mad));
 	mad->common.version = cpu_to_be32(1);
-	mad->common.opcode = cpu_to_be32(IBMVFC_IMPLICIT_LOGOUT);
+	if (tgt->protocol == IBMVFC_PROTO_SCSI)
+		mad->common.opcode = cpu_to_be32(IBMVFC_IMPLICIT_LOGOUT);
+	else
+		mad->common.opcode = cpu_to_be32(IBMVFC_NVMF_IMPLICIT_LOGOUT);
 	mad->common.length = cpu_to_be16(sizeof(*mad));
 	mad->old_scsi_id = cpu_to_be64(tgt->scsi_id);
 	return evt;
@@ -4393,7 +4397,7 @@ static void ibmvfc_tgt_implicit_logout(struct ibmvfc_target *tgt)
 		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
 		kref_put(&tgt->kref, ibmvfc_release_tgt);
 	} else
-		tgt_dbg(tgt, "Sent Implicit Logout\n");
+		tgt_dbg(tgt, "%s Sent Implicit Logout\n", proto_type[tgt->protocol]);
 }
 
 /**
@@ -4423,7 +4427,8 @@ static void ibmvfc_tgt_implicit_logout_and_del_done(struct ibmvfc_event *evt)
 	else
 		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_DEL_AND_LOGOUT_RPORT);
 
-	tgt_dbg(tgt, "Implicit Logout %s\n", (status == IBMVFC_MAD_SUCCESS) ? "succeeded" : "failed");
+	tgt_dbg(tgt, "%s Implicit Logout %s\n", proto_type[tgt->protocol],
+		(status == IBMVFC_MAD_SUCCESS) ? "succeeded" : "failed");
 	kref_put(&tgt->kref, ibmvfc_release_tgt);
 	wake_up(&vhost->work_wait_q);
 }
@@ -4456,7 +4461,7 @@ static void ibmvfc_tgt_implicit_logout_and_del(struct ibmvfc_target *tgt)
 		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_DEL_RPORT);
 		kref_put(&tgt->kref, ibmvfc_release_tgt);
 	} else
-		tgt_dbg(tgt, "Sent Implicit Logout\n");
+		tgt_dbg(tgt, "%s Sent Implicit Logout\n", proto_type[tgt->protocol]);
 }
 
 /**
@@ -4476,7 +4481,8 @@ static void ibmvfc_tgt_move_login_done(struct ibmvfc_event *evt)
 	ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
 	switch (status) {
 	case IBMVFC_MAD_SUCCESS:
-		tgt_dbg(tgt, "Move Login succeeded for new scsi_id: %llX\n", tgt->new_scsi_id);
+		tgt_dbg(tgt, "%s Move Login succeeded for new scsi_id: %llX\n",
+			proto_type[tgt->protocol], tgt->new_scsi_id);
 		tgt->ids.node_name = wwn_to_u64(rsp->service_parms.node_name);
 		tgt->ids.port_name = wwn_to_u64(rsp->service_parms.port_name);
 		tgt->scsi_id = tgt->new_scsi_id;
@@ -4497,8 +4503,9 @@ static void ibmvfc_tgt_move_login_done(struct ibmvfc_event *evt)
 		level += ibmvfc_retry_tgt_init(tgt, ibmvfc_tgt_move_login);
 
 		tgt_log(tgt, level,
-			"Move Login failed: new scsi_id: %llX, flags:%x, vios_flags:%x, rc=0x%02X\n",
-			tgt->new_scsi_id, be32_to_cpu(rsp->flags), be16_to_cpu(rsp->vios_flags),
+			"%s Move Login failed: new scsi_id: %llX, flags:%x, vios_flags:%x, rc=0x%02X\n",
+			proto_type[tgt->protocol], tgt->new_scsi_id,
+			be32_to_cpu(rsp->flags), be16_to_cpu(rsp->vios_flags),
 			status);
 		break;
 	}
@@ -4538,7 +4545,10 @@ static void ibmvfc_tgt_move_login(struct ibmvfc_target *tgt)
 	move = &evt->iu.move_login;
 	memset(move, 0, sizeof(*move));
 	move->common.version = cpu_to_be32(1);
-	move->common.opcode = cpu_to_be32(IBMVFC_MOVE_LOGIN);
+	if (tgt->protocol == IBMVFC_PROTO_SCSI)
+		move->common.opcode = cpu_to_be32(IBMVFC_MOVE_LOGIN);
+	else
+		move->common.opcode = cpu_to_be32(IBMVFC_NVMF_MOVE_LOGIN);
 	move->common.length = cpu_to_be16(sizeof(*move));
 
 	move->old_scsi_id = cpu_to_be64(tgt->scsi_id);
@@ -4551,7 +4561,8 @@ static void ibmvfc_tgt_move_login(struct ibmvfc_target *tgt)
 		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_DEL_RPORT);
 		kref_put(&tgt->kref, ibmvfc_release_tgt);
 	} else
-		tgt_dbg(tgt, "Sent Move Login for new scsi_id: %llX\n", tgt->new_scsi_id);
+		tgt_dbg(tgt, "Sent %s Move Login for new scsi_id: %llX\n",
+			proto_type[tgt->protocol], tgt->new_scsi_id);
 }
 
 /**
-- 
2.54.0

[PATCH 13/29] ibmvfc: add NVMe/FC Port Login support

[Tyrel Datwyler]

Expand the target login path to issue the NVMe/FC-specific port login
MAD for NVMe targets.

Select the correct PLOGI MAD opcode based on the target protocol and
include the protocol name in success and failure logging. The rest of
the target login flow remains shared with the existing SCSI
implementation.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 3e3d77e0d517..2c54d0b9add4 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -4210,7 +4210,7 @@ static void ibmvfc_tgt_plogi_done(struct ibmvfc_event *evt)
 	ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
 	switch (status) {
 	case IBMVFC_MAD_SUCCESS:
-		tgt_dbg(tgt, "Port Login succeeded\n");
+		tgt_dbg(tgt, "%s Port Login succeeded\n", proto_type[tgt->protocol]);
 		if (tgt->ids.port_name &&
 		    tgt->ids.port_name != wwn_to_u64(rsp->service_parms.port_name)) {
 			vhost->reinit = 1;
@@ -4238,9 +4238,9 @@ static void ibmvfc_tgt_plogi_done(struct ibmvfc_event *evt)
 		else
 			ibmvfc_del_tgt(tgt);
 
-		tgt_log(tgt, level, "Port Login failed: %s (%x:%x) %s (%x) %s (%x) rc=0x%02X\n",
-			ibmvfc_get_cmd_error(be16_to_cpu(rsp->status), be16_to_cpu(rsp->error)),
-					     be16_to_cpu(rsp->status), be16_to_cpu(rsp->error),
+		tgt_log(tgt, level, "%s Port Login failed: %s (%x:%x) %s (%x) %s (%x) rc=0x%02X\n",
+			proto_type[tgt->protocol], ibmvfc_get_cmd_error(be16_to_cpu(rsp->status),
+			be16_to_cpu(rsp->error)), be16_to_cpu(rsp->status), be16_to_cpu(rsp->error),
 			ibmvfc_get_fc_type(be16_to_cpu(rsp->fc_type)), be16_to_cpu(rsp->fc_type),
 			ibmvfc_get_ls_explain(be16_to_cpu(rsp->fc_explain)), be16_to_cpu(rsp->fc_explain), status);
 		break;
@@ -4286,7 +4286,10 @@ static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *tgt)
 	} else {
 		plogi->common.version = cpu_to_be32(1);
 	}
-	plogi->common.opcode = cpu_to_be32(IBMVFC_PORT_LOGIN);
+	if (tgt->protocol == IBMVFC_PROTO_SCSI)
+		plogi->common.opcode = cpu_to_be32(IBMVFC_PORT_LOGIN);
+	else
+		plogi->common.opcode = cpu_to_be32(IBMVFC_NVMF_PORT_LOGIN);
 	plogi->common.length = cpu_to_be16(sizeof(*plogi));
 	plogi->scsi_id = cpu_to_be64(tgt->scsi_id);
 
@@ -4295,7 +4298,7 @@ static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *tgt)
 		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
 		kref_put(&tgt->kref, ibmvfc_release_tgt);
 	} else
-		tgt_dbg(tgt, "Sent port login\n");
+		tgt_dbg(tgt, "Sent %s port login\n", proto_type[tgt->protocol]);
 }
 
 /**
-- 
2.54.0

[PATCH 14/29] ibmvfc: add NVMe/FC Process Login support

[Tyrel Datwyler]

Extend PRLI handling code to support NVMe/FC targets.

When the target protocol is NVMe/FC, issue the NVMe process login MAD,
set the NVMe FC-4 type, and populate NVMe-specific service parameters.
On completion, decode the returned PRLI service parameters and derive
the appropriate remote-port roles for NVMe initiator, target, and
discovery ports.

Keep the existing SCSI PRLI flow unchanged while allowing the common
target state machine to complete login for NVMe/FC targets.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 55 ++++++++++++++++++++---------
 drivers/scsi/ibmvscsi/ibmvfc.h      |  6 ++++
 2 files changed, 45 insertions(+), 16 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 2c54d0b9add4..b45cd0183fb5 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -4088,11 +4088,12 @@ static void ibmvfc_tgt_prli_done(struct ibmvfc_event *evt)
 	ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
 	switch (status) {
 	case IBMVFC_MAD_SUCCESS:
-		tgt_dbg(tgt, "Process Login succeeded: %X %02X %04X\n",
-			parms->type, parms->flags, parms->service_parms);
+		tgt_dbg(tgt, "%s Process Login succeeded: %X %02X %04X\n",
+			proto_type[tgt->protocol], parms->type, parms->flags,
+			parms->service_parms);
 
+		index = ibmvfc_get_prli_rsp(be16_to_cpu(parms->flags));
 		if (parms->type == IBMVFC_SCSI_FCP_TYPE) {
-			index = ibmvfc_get_prli_rsp(be16_to_cpu(parms->flags));
 			if (prli_rsp[index].logged_in) {
 				if (be16_to_cpu(parms->flags) & IBMVFC_PRLI_EST_IMG_PAIR) {
 					tgt->need_login = 0;
@@ -4108,6 +4109,22 @@ static void ibmvfc_tgt_prli_done(struct ibmvfc_event *evt)
 				ibmvfc_retry_tgt_init(tgt, ibmvfc_tgt_send_prli);
 			else
 				ibmvfc_del_tgt(tgt);
+		} else if (parms->type == IBMVFC_NVME_FCP_TYPE) {
+			if (prli_rsp[index].logged_in) {
+				/* For FC-NVMe PRLI the Image Pair field is always set to zero (see 6.3.3) */
+				tgt->need_login = 0;
+				tgt->ids.roles = 0;
+				if (be32_to_cpu(parms->service_parms) & IBMVFC_PRLI_NVME_TARGET)
+					tgt->ids.roles |= FC_PORT_ROLE_NVME_TARGET;
+				if (be32_to_cpu(parms->service_parms) & IBMVFC_PRLI_NVME_INITIATOR)
+					tgt->ids.roles |= FC_PORT_ROLE_NVME_INITIATOR;
+				if (be32_to_cpu(parms->service_parms) & IBMVFC_PRLI_NVME_DISCOVERY)
+					tgt->ids.roles |= FC_PORT_ROLE_NVME_DISCOVERY;
+				tgt->add_rport = 1;
+			} else if (prli_rsp[index].retry)
+				ibmvfc_retry_tgt_init(tgt, ibmvfc_tgt_send_prli);
+			else
+				ibmvfc_del_tgt(tgt);
 		} else
 			ibmvfc_del_tgt(tgt);
 		break;
@@ -4128,9 +4145,10 @@ static void ibmvfc_tgt_prli_done(struct ibmvfc_event *evt)
 		else
 			ibmvfc_del_tgt(tgt);
 
-		tgt_log(tgt, level, "Process Login failed: %s (%x:%x) rc=0x%02X\n",
-			ibmvfc_get_cmd_error(be16_to_cpu(rsp->status), be16_to_cpu(rsp->error)),
-			be16_to_cpu(rsp->status), be16_to_cpu(rsp->error), status);
+		tgt_log(tgt, level, "%s Process Login failed: %s (%x:%x) rc=0x%02X\n",
+			proto_type[tgt->protocol], ibmvfc_get_cmd_error(be16_to_cpu(rsp->status),
+			be16_to_cpu(rsp->error)), be16_to_cpu(rsp->status),
+			be16_to_cpu(rsp->error), status);
 		break;
 	}
 
@@ -4172,25 +4190,30 @@ static void ibmvfc_tgt_send_prli(struct ibmvfc_target *tgt)
 	} else {
 		prli->common.version = cpu_to_be32(1);
 	}
-	prli->common.opcode = cpu_to_be32(IBMVFC_PROCESS_LOGIN);
+	if (tgt->protocol == IBMVFC_PROTO_SCSI) {
+		prli->common.opcode = cpu_to_be32(IBMVFC_PROCESS_LOGIN);
+		prli->parms.type = IBMVFC_SCSI_FCP_TYPE;
+		prli->parms.flags = cpu_to_be16(IBMVFC_PRLI_EST_IMG_PAIR);
+		prli->parms.service_parms = cpu_to_be32(IBMVFC_PRLI_INITIATOR_FUNC);
+		prli->parms.service_parms |= cpu_to_be32(IBMVFC_PRLI_READ_FCP_XFER_RDY_DISABLED);
+
+		if (cls3_error)
+			prli->parms.service_parms |= cpu_to_be32(IBMVFC_PRLI_RETRY);
+	} else {
+		prli->common.opcode = cpu_to_be32(IBMVFC_NVMF_PROCESS_LOGIN);
+		prli->parms.type = IBMVFC_NVME_FCP_TYPE;
+		prli->parms.service_parms = cpu_to_be32(IBMVFC_PRLI_NVME_INITIATOR);
+	}
 	prli->common.length = cpu_to_be16(sizeof(*prli));
 	prli->scsi_id = cpu_to_be64(tgt->scsi_id);
 
-	prli->parms.type = IBMVFC_SCSI_FCP_TYPE;
-	prli->parms.flags = cpu_to_be16(IBMVFC_PRLI_EST_IMG_PAIR);
-	prli->parms.service_parms = cpu_to_be32(IBMVFC_PRLI_INITIATOR_FUNC);
-	prli->parms.service_parms |= cpu_to_be32(IBMVFC_PRLI_READ_FCP_XFER_RDY_DISABLED);
-
-	if (cls3_error)
-		prli->parms.service_parms |= cpu_to_be32(IBMVFC_PRLI_RETRY);
-
 	ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_INIT_WAIT);
 	if (ibmvfc_send_event(evt, vhost, default_timeout)) {
 		vhost->discovery_threads--;
 		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
 		kref_put(&tgt->kref, ibmvfc_release_tgt);
 	} else
-		tgt_dbg(tgt, "Sent process login\n");
+		tgt_dbg(tgt, "%s Sent process login\n", proto_type[tgt->protocol]);
 }
 
 /**
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index 67f546ff092e..66025e6ffeed 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -381,6 +381,7 @@ struct ibmvfc_move_login {
 struct ibmvfc_prli_svc_parms {
 	u8 type;
 #define IBMVFC_SCSI_FCP_TYPE		0x08
+#define IBMVFC_NVME_FCP_TYPE		0x28
 	u8 type_ext;
 	__be16 flags;
 #define IBMVFC_PRLI_ORIG_PA_VALID			0x8000
@@ -396,6 +397,11 @@ struct ibmvfc_prli_svc_parms {
 #define IBMVFC_PRLI_TARGET_FUNC			0x00000010
 #define IBMVFC_PRLI_READ_FCP_XFER_RDY_DISABLED	0x00000002
 #define IBMVFC_PRLI_WR_FCP_XFER_RDY_DISABLED	0x00000001
+#define IBMVFC_PRLI_NVME_PI_CTRL		0x00000200
+#define IBMVFC_PRLI_NVME_SLER			0x00000100
+#define IBMVFC_PRLI_NVME_INITIATOR		0x00000020
+#define IBMVFC_PRLI_NVME_TARGET			0x00000010
+#define IBMVFC_PRLI_NVME_DISCOVERY		0x00000008
 } __packed __aligned(4);
 
 struct ibmvfc_process_login {
-- 
2.54.0

[PATCH 15/29] ibmvfc: add NVMe/FC Query Target support

[Tyrel Datwyler]

Add protocol-specific query-target support for NVMe/FC targets.

Use the NVMe query-target specific MAD when querying an NVMe target and
update the associated debug and error logging to include the target
protocol.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index b45cd0183fb5..363bf75d6244 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -4849,7 +4849,7 @@ static void ibmvfc_tgt_query_target_done(struct ibmvfc_event *evt)
 	ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
 	switch (status) {
 	case IBMVFC_MAD_SUCCESS:
-		tgt_dbg(tgt, "Query Target succeeded\n");
+		tgt_dbg(tgt, "%s Query Target succeeded\n", proto_type[tgt->protocol]);
 		if (be64_to_cpu(rsp->scsi_id) != tgt->scsi_id)
 			ibmvfc_del_tgt(tgt);
 		else
@@ -4871,9 +4871,9 @@ static void ibmvfc_tgt_query_target_done(struct ibmvfc_event *evt)
 		else
 			ibmvfc_del_tgt(tgt);
 
-		tgt_log(tgt, level, "Query Target failed: %s (%x:%x) %s (%x) %s (%x) rc=0x%02X\n",
-			ibmvfc_get_cmd_error(be16_to_cpu(rsp->status), be16_to_cpu(rsp->error)),
-			be16_to_cpu(rsp->status), be16_to_cpu(rsp->error),
+		tgt_log(tgt, level, "%s Query Target failed: %s (%x:%x) %s (%x) %s (%x) rc=0x%02X\n",
+			proto_type[tgt->protocol], ibmvfc_get_cmd_error(be16_to_cpu(rsp->status),
+			be16_to_cpu(rsp->error)), be16_to_cpu(rsp->status), be16_to_cpu(rsp->error),
 			ibmvfc_get_fc_type(be16_to_cpu(rsp->fc_type)), be16_to_cpu(rsp->fc_type),
 			ibmvfc_get_gs_explain(be16_to_cpu(rsp->fc_explain)), be16_to_cpu(rsp->fc_explain),
 			status);
@@ -4913,7 +4913,10 @@ static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt)
 	query_tgt = &evt->iu.query_tgt;
 	memset(query_tgt, 0, sizeof(*query_tgt));
 	query_tgt->common.version = cpu_to_be32(1);
-	query_tgt->common.opcode = cpu_to_be32(IBMVFC_QUERY_TARGET);
+	if (tgt->protocol == IBMVFC_PROTO_SCSI)
+		query_tgt->common.opcode = cpu_to_be32(IBMVFC_QUERY_TARGET);
+	else
+		query_tgt->common.opcode = cpu_to_be32(IBMVFC_NVMF_QUERY_TARGET);
 	query_tgt->common.length = cpu_to_be16(sizeof(*query_tgt));
 	query_tgt->wwpn = cpu_to_be64(tgt->ids.port_name);
 
@@ -4923,7 +4926,7 @@ static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt)
 		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
 		kref_put(&tgt->kref, ibmvfc_release_tgt);
 	} else
-		tgt_dbg(tgt, "Sent Query Target\n");
+		tgt_dbg(tgt, "Sent %s Query Target\n", proto_type[tgt->protocol]);
 }
 
 /**
-- 
2.54.0

[PATCH 16/29] ibmvfc: allocate targets based on protocol

[Tyrel Datwyler]

Allocate discovered targets onto the channel-group list that matches
their protocol.

When a target is created, use the discovered protocol type to decide
which list it belongs on. This keeps protocol-specific discovery
results isolated and allows later state-machine and remote-port code to
walk the correct target set.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 363bf75d6244..4b95e4344947 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -4938,25 +4938,32 @@ static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt)
  *	0 on success / other on failure
  **/
 static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,
-			       struct ibmvfc_discover_targets_entry *target)
+			       struct ibmvfc_discover_targets_entry *target,
+			       enum ibmvfc_protocol protocol)
 {
 	struct ibmvfc_target *stgt = NULL;
 	struct ibmvfc_target *wtgt = NULL;
 	struct ibmvfc_target *tgt;
+	struct ibmvfc_channels *channels;
 	unsigned long flags;
 	u64 scsi_id = be32_to_cpu(target->scsi_id) & IBMVFC_DISC_TGT_SCSI_ID_MASK;
 	u64 wwpn = be64_to_cpu(target->wwpn);
 
+	if (protocol == IBMVFC_PROTO_SCSI)
+		channels = &vhost->scsi_scrqs;
+	else
+		channels = &vhost->nvme_scrqs;
+
 	/* Look to see if we already have a target allocated for this SCSI ID or WWPN */
 	spin_lock_irqsave(vhost->host->host_lock, flags);
-	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
+	list_for_each_entry(tgt, &channels->targets, queue) {
 		if (tgt->wwpn == wwpn) {
 			wtgt = tgt;
 			break;
 		}
 	}
 
-	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
+	list_for_each_entry(tgt, &channels->targets, queue) {
 		if (tgt->scsi_id == scsi_id) {
 			stgt = tgt;
 			break;
@@ -5004,6 +5011,7 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,
 
 	tgt = mempool_alloc(vhost->tgt_pool, GFP_NOIO);
 	memset(tgt, 0, sizeof(*tgt));
+	tgt->protocol = protocol;
 	tgt->scsi_id = scsi_id;
 	tgt->wwpn = wwpn;
 	tgt->vhost = vhost;
@@ -5013,7 +5021,7 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,
 	ibmvfc_init_tgt(tgt, ibmvfc_tgt_implicit_logout);
 	spin_lock_irqsave(vhost->host->host_lock, flags);
 	tgt->cancel_key = vhost->task_set++;
-	list_add_tail(&tgt->queue, &vhost->scsi_scrqs.targets);
+	list_add_tail(&tgt->queue, &channels->targets);
 
 unlock_out:
 	spin_unlock_irqrestore(vhost->host->host_lock, flags);
@@ -5032,7 +5040,11 @@ static int ibmvfc_alloc_targets(struct ibmvfc_host *vhost)
 	int i, rc;
 
 	for (i = 0, rc = 0; !rc && i < vhost->scsi_scrqs.num_targets; i++)
-		rc = ibmvfc_alloc_target(vhost, &vhost->scsi_scrqs.disc_buf[i]);
+		rc = ibmvfc_alloc_target(vhost, &vhost->scsi_scrqs.disc_buf[i],
+					 vhost->scsi_scrqs.protocol);
+	for (i = 0; !rc && i < vhost->nvme_scrqs.num_targets; i++)
+		rc = ibmvfc_alloc_target(vhost, &vhost->nvme_scrqs.disc_buf[i],
+					 vhost->nvme_scrqs.protocol);
 
 	return rc;
 }
-- 
2.54.0

[PATCH 17/29] ibmvfc: delete NVMe/FC targets as well as SCSI

[Tyrel Datwyler]

Extend target deletion paths to process NVMe targets as well as SCSI
targets.

Update link-down, and host reinitialization flows to walk both the SCSI
and NVMe target lists when marking targets for deletion. This ensures
that protocol-specific target state stays consistent across adapter
resets and fabric events.

Rename ibmvfc_relogin to ibmvfc_scsi_relogin as it acts on a scsi
command.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 4b95e4344947..13e513bfd0a8 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -714,6 +714,8 @@ static void ibmvfc_link_down(struct ibmvfc_host *vhost,
 	scsi_block_requests(vhost->host);
 	list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 		ibmvfc_del_tgt(tgt);
+	list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue)
+		ibmvfc_del_tgt(tgt);
 	ibmvfc_set_host_state(vhost, state);
 	ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_TGT_DEL);
 	vhost->events_to_log |= IBMVFC_AE_LINKDOWN;
@@ -751,6 +753,12 @@ static void ibmvfc_init_host(struct ibmvfc_host *vhost)
 			else
 				ibmvfc_del_tgt(tgt);
 		}
+		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue) {
+			if (vhost->client_migrated)
+				tgt->need_login = 1;
+			else
+				ibmvfc_del_tgt(tgt);
+		}
 
 		scsi_block_requests(vhost->host);
 		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT);
@@ -1868,7 +1876,7 @@ static void ibmvfc_log_error(struct ibmvfc_event *evt)
  * @sdev:	scsi device struct
  *
  **/
-static void ibmvfc_relogin(struct scsi_device *sdev)
+static void ibmvfc_scsi_relogin(struct scsi_device *sdev)
 {
 	struct ibmvfc_host *vhost = shost_priv(sdev->host);
 	struct fc_rport *rport = starget_to_rport(scsi_target(sdev));
@@ -1920,7 +1928,7 @@ static void ibmvfc_scsi_done(struct ibmvfc_event *evt)
 				memcpy(cmnd->sense_buffer, rsp->data.sense + rsp_len, sense_len);
 			if ((be16_to_cpu(vfc_cmd->status) & IBMVFC_VIOS_FAILURE) &&
 			    (be16_to_cpu(vfc_cmd->error) == IBMVFC_PLOGI_REQUIRED))
-				ibmvfc_relogin(cmnd->device);
+				ibmvfc_scsi_relogin(cmnd->device);
 
 			if (!cmnd->result && (!scsi_get_resid(cmnd) || (rsp->flags & FCP_RESID_OVER)))
 				cmnd->result = (DID_ERROR << 16);
-- 
2.54.0

[PATCH 18/29] ibmvfc: update state machine to process NVMe/FC targets

[Tyrel Datwyler]

Update the host work loop and target state-machine helpers to process
NVMe targets in addition to SCSI targets.

Check both protocol-specific target lists when determining whether there
is initialization or logout work pending, and extend the query, target
init, and target delete phases to dispatch work for NVMe targets using
the same common state-machine callbacks.

This allows the existing discovery and login state machine to drive
NVMe/FC targets through query, login, and deletion without duplicating
the control flow.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 42 +++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 13e513bfd0a8..9a6a885aa57e 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -5692,6 +5692,11 @@ static int ibmvfc_dev_init_to_do(struct ibmvfc_host *vhost)
 		    tgt->action == IBMVFC_TGT_ACTION_INIT_WAIT)
 			return 1;
 	}
+	list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue) {
+		if (tgt->action == IBMVFC_TGT_ACTION_INIT ||
+		    tgt->action == IBMVFC_TGT_ACTION_INIT_WAIT)
+			return 1;
+	}
 
 	return 0;
 }
@@ -5712,6 +5717,11 @@ static int ibmvfc_dev_logo_to_do(struct ibmvfc_host *vhost)
 		    tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT)
 			return 1;
 	}
+	list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue) {
+		if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT ||
+		    tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT)
+			return 1;
+	}
 	return 0;
 }
 
@@ -5740,9 +5750,15 @@ static int __ibmvfc_work_to_do(struct ibmvfc_host *vhost)
 		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 			if (tgt->action == IBMVFC_TGT_ACTION_INIT)
 				return 1;
+		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue)
+			if (tgt->action == IBMVFC_TGT_ACTION_INIT)
+				return 1;
 		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 			if (tgt->action == IBMVFC_TGT_ACTION_INIT_WAIT)
 				return 0;
+		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue)
+			if (tgt->action == IBMVFC_TGT_ACTION_INIT_WAIT)
+				return 0;
 		return 1;
 	case IBMVFC_HOST_ACTION_TGT_DEL:
 	case IBMVFC_HOST_ACTION_TGT_DEL_FAILED:
@@ -5751,9 +5767,15 @@ static int __ibmvfc_work_to_do(struct ibmvfc_host *vhost)
 		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 			if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT)
 				return 1;
+		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue)
+			if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT)
+				return 1;
 		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 			if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT)
 				return 0;
+		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue)
+			if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT_WAIT)
+				return 0;
 		return 1;
 	case IBMVFC_HOST_ACTION_LOGO:
 	case IBMVFC_HOST_ACTION_INIT:
@@ -5941,6 +5963,8 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)
 	case IBMVFC_HOST_ACTION_QUERY:
 		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue)
 			ibmvfc_init_tgt(tgt, ibmvfc_tgt_query_target);
+		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue)
+			ibmvfc_init_tgt(tgt, ibmvfc_tgt_query_target);
 		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY_TGTS);
 		break;
 	case IBMVFC_HOST_ACTION_QUERY_TGTS:
@@ -5950,6 +5974,12 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)
 				break;
 			}
 		}
+		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue) {
+			if (tgt->action == IBMVFC_TGT_ACTION_INIT) {
+				tgt->job_step(tgt);
+				break;
+			}
+		}
 
 		if (!ibmvfc_dev_init_to_do(vhost))
 			ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_TGT_DEL);
@@ -5962,6 +5992,12 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)
 				break;
 			}
 		}
+		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue) {
+			if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT) {
+				tgt->job_step(tgt);
+				break;
+			}
+		}
 
 		if (ibmvfc_dev_logo_to_do(vhost)) {
 			spin_unlock_irqrestore(vhost->host->host_lock, flags);
@@ -6049,6 +6085,12 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)
 				break;
 			}
 		}
+		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue) {
+			if (tgt->action == IBMVFC_TGT_ACTION_INIT) {
+				tgt->job_step(tgt);
+				break;
+			}
+		}
 
 		if (!ibmvfc_dev_init_to_do(vhost))
 			ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_TGT_DEL_FAILED);
-- 
2.54.0

[PATCH 19/29] ibmvfc: implement NVMe/FC stubs for local/remote port registration

[Tyrel Datwyler]

Implement the initial NVMe/FC local-port and remote-port registration
functions that notify the NVMe-FC midlayr of port discovery and loss.

Register the local port with the NVMe-FC transport, register discovered
remote ports against that local port, and add matching unregister paths
that wait for the NVMe-FC core to complete asynchronous deletion before
dropping driver references. Also store driver-private host and target
pointers in the registered NVMe-FC port objects.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 79 ++++++++++++++++++++++++++++-
 drivers/scsi/ibmvscsi/ibmvfc-nvme.h |  4 +-
 drivers/scsi/ibmvscsi/ibmvfc.h      |  4 ++
 3 files changed, 83 insertions(+), 4 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
index 4a66cde8a8d2..202e8d0b0081 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
@@ -14,10 +14,18 @@
 
 static void ibmvfc_nvme_localport_delete(struct nvme_fc_local_port *lport)
 {
+	struct ibmvfc_host *vhost = lport->private;
+
+	vhost->nvme_local_port = NULL;
+	complete(&vhost->nvme_delete_done);
 }
 
 static void ibmvfc_nvme_remoteport_delete(struct nvme_fc_remote_port *rport)
 {
+	struct ibmvfc_target *tgt = rport->private;
+
+	tgt->nvme_remote_port = NULL;
+	complete(&tgt->nvme_delete_done);
 }
 
 static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport,
@@ -70,18 +78,85 @@ static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = {
 
 int ibmvfc_nvme_register_remoteport(struct ibmvfc_target *tgt)
 {
-	return 0;
+	struct ibmvfc_host *vhost = tgt->vhost;
+	struct nvme_fc_port_info pinfo;
+	int rc;
+
+	if (!vhost->nvme_local_port) {
+		dev_err(vhost->dev, "Attempt to register NVMe fc remoteport without valid localport\n");
+		return -EINVAL;
+	}
+
+	memset(&pinfo, 0, sizeof(struct nvme_fc_port_info));
+	pinfo.node_name = tgt->ids.node_name;
+	pinfo.port_name = tgt->ids.port_name;
+	pinfo.port_id = tgt->ids.port_id;
+	pinfo.port_role = FC_PORT_ROLE_NVME_TARGET;
+
+	rc = nvme_fc_register_remoteport(vhost->nvme_local_port, &pinfo,
+					 &tgt->nvme_remote_port);
+
+	if (!rc) {
+		ibmvfc_log(vhost, 2, "register_remoteport: traddr=nn-0x%llx:pn-0x%llx PortID:%x\n",
+			   pinfo.node_name, pinfo.port_name, pinfo.port_id);
+		tgt->nvme_remote_port->private = tgt;
+	}
+
+	return rc;
 }
 
 void ibmvfc_nvme_unregister_remoteport(struct ibmvfc_target *tgt)
 {
+	struct ibmvfc_host *vhost = tgt->vhost;
+	struct nvme_fc_remote_port *rport = tgt->nvme_remote_port;
+	int rc;
+
+	if (!tgt->nvme_remote_port)
+		return;
+
+	ibmvfc_log(vhost, 2, "unregister_remoteport: traddr=nn-0x%llx:pn-0x%llx PortID:%x\n",
+		   rport->node_name, rport->port_name, rport->port_id);
+	init_completion(&tgt->nvme_delete_done);
+	rc = nvme_fc_unregister_remoteport(tgt->nvme_remote_port);
+
+	if (!rc) {
+		wait_for_completion(&tgt->nvme_delete_done);
+		tgt->nvme_remote_port->private = NULL;
+	}
 }
 
 int ibmvfc_nvme_register(struct ibmvfc_host *vhost)
 {
-	return 0;
+	struct nvme_fc_port_info pinfo;
+	int rc;
+
+	pinfo.node_name = fc_host_node_name(vhost->host);
+	pinfo.port_name = fc_host_port_name(vhost->host);
+	pinfo.port_id = fc_host_port_id(vhost->host);
+	pinfo.port_role = FC_PORT_ROLE_NVME_INITIATOR;
+	pinfo.dev_loss_tmo = 0;
+
+	rc = nvme_fc_register_localport(&pinfo, &ibmvfc_nvme_fc_transport,
+					vhost->dev, &vhost->nvme_local_port);
+
+	if (!rc) {
+		ibmvfc_log(vhost, 2, "register_localport: host-traddr=nn-0x%llx:pn-0x%llx on portID:%x\n",
+			   pinfo.node_name, pinfo.port_name, pinfo.port_id);
+		vhost->nvme_local_port->private = vhost;
+	} else
+		dev_err(vhost->dev, "Failed to register NVMe fc localport (%d)\n", rc);
+
+	return rc;
 }
 
 void ibmvfc_nvme_unregister(struct ibmvfc_host *vhost)
 {
+	int rc;
+
+	if (vhost->nvme_local_port) {
+		init_completion(&vhost->nvme_delete_done);
+		rc = nvme_fc_unregister_localport(vhost->nvme_local_port);
+		if (!rc)
+			wait_for_completion(&vhost->nvme_delete_done);
+	}
 }
diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
index 97e267871df2..0465e8719881 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
@@ -11,8 +11,8 @@
 #ifndef _IBMVFC_NVME_H
 #define _IBMVFC_NVME_H
 
-#include <uapi/scsi/fc/fc_fs.h>
-#include <uapi/scsi/fc/fc_els.h>
+#include <scsi/fc/fc_fs.h>
+#include <scsi/fc/fc_els.h>
 #include <linux/nvme-fc-driver.h>
 
 #include "ibmvfc.h"
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index 66025e6ffeed..d8c2e5f1fdec 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -839,6 +839,8 @@ struct ibmvfc_target {
 	void (*job_step) (struct ibmvfc_target *);
 	struct timer_list timer;
 	struct kref kref;
+	struct completion nvme_delete_done;
+	struct nvme_fc_remote_port *nvme_remote_port;
 };
 
 /* a unit of work for the hosting partition */
@@ -1015,6 +1017,8 @@ struct ibmvfc_host {
 	struct work_struct rport_add_work_q;
 	wait_queue_head_t init_wait_q;
 	wait_queue_head_t work_wait_q;
+	struct nvme_fc_local_port *nvme_local_port;
+	struct completion nvme_delete_done;
 };
 
 static inline struct ibmvfc_host *ibmvfc_channels_to_vhost(struct ibmvfc_channels *channels)
-- 
2.54.0

[PATCH 20/29] ibmvfc: register local nvme fc port after fabric login

[Tyrel Datwyler]

Register the local NVMe/FC port only after fabric login has completed.

The VIOS returns the client port ID in the fabric login response, and
that port ID is required to populate the local-port information passed
to the NVMe-FC midlayer. Delay local-port registration until that data
is available and update the registration helper accordingly.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c |  3 +++
 drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 18 +++++++++++++++---
 2 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 9a6a885aa57e..2c7ecf7bdde9 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -5184,6 +5184,8 @@ static void ibmvfc_fabric_login_nvme_done(struct ibmvfc_event *evt)
 
 	switch (mad_status) {
 	case IBMVFC_MAD_SUCCESS:
+		fc_host_port_id(vhost->host) = be64_to_cpu(rsp->nport_id);
+		ibmvfc_nvme_register(vhost);
 		ibmvfc_dbg(vhost, "NVMe fabric login succeeded\n");
 		break;
 	case IBMVFC_MAD_FAILED:
@@ -5245,6 +5247,7 @@ static void ibmvfc_fabric_login_scsi_done(struct ibmvfc_event *evt)
 
 	switch (mad_status) {
 	case IBMVFC_MAD_SUCCESS:
+		fc_host_port_id(vhost->host) = be64_to_cpu(rsp->nport_id);
 		ibmvfc_dbg(vhost, "SCSI fabric login succeeded\n");
 		break;
 	case IBMVFC_MAD_FAILED:
diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
index 202e8d0b0081..fc4337fc9b3f 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
@@ -137,14 +137,17 @@ int ibmvfc_nvme_register(struct ibmvfc_host *vhost)
 	pinfo.dev_loss_tmo = 0;
 
 	rc = nvme_fc_register_localport(&pinfo, &ibmvfc_nvme_fc_transport,
-					vhost->dev, &vhost->nvme_local_port);
+					get_device(vhost->dev),
+					&vhost->nvme_local_port);
 
 	if (!rc) {
 		ibmvfc_log(vhost, 2, "register_localport: host-traddr=nn-0x%llx:pn-0x%llx on portID:%x\n",
 			   pinfo.node_name, pinfo.port_name, pinfo.port_id);
 		vhost->nvme_local_port->private = vhost;
-	} else
+	} else {
 		dev_err(vhost->dev, "Failed to register NVMe fc localport (%d)\n", rc);
+		put_device(vhost->dev);
+	}
 
 	return rc;
 }
@@ -154,9 +157,18 @@ void ibmvfc_nvme_unregister(struct ibmvfc_host *vhost)
 	int rc;
 
 	if (vhost->nvme_local_port) {
+		ibmvfc_log(vhost, 2, "unregister_localport: host-traddr=nn-0x%llx:pn-0x%llx on portID:%x\n",
+			   vhost->nvme_local_port->node_name,
+			   vhost->nvme_local_port->port_name,
+			   vhost->nvme_local_port->port_id);
 		init_completion(&vhost->nvme_delete_done);
 		rc = nvme_fc_unregister_localport(vhost->nvme_local_port);
-		if (!rc)
+		if (!rc) {
 			wait_for_completion(&vhost->nvme_delete_done);
+			vhost->nvme_local_port->private = NULL;
+		} else
+			dev_err(vhost->dev, "Failed to unregister NVMe fc localport (%d)\n", rc);
+
+		put_device(vhost->dev);
 	}
 }
-- 
2.54.0

[PATCH 21/29] ibmvfc: process NVMe/FC rports in work thread

[Tyrel Datwyler]

Add an NVMe-specific remote-port add helper and update the rport worker
thread to walk the NVMe target list, register new NVMe remote ports, and
rescan existing ones through the NVMe-FC midlayer. Also handle delete
and delete-with-logout transitions for NVMe remote ports in the same
worker context used for SCSI rports.

This keeps remote-port registration serialized in the existing worker
model while allowing NVMe targets to participate in the common target
state machine.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 68 ++++++++++++++++++++++++++++-
 1 file changed, 67 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 2c7ecf7bdde9..5f6ee99d0dba 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -5829,6 +5829,51 @@ static void ibmvfc_log_ae(struct ibmvfc_host *vhost, int events)
 		fc_host_post_event(vhost->host, fc_get_event_number(), FCH_EVT_LINKUP, 0);
 }
 
+/**
+ * ibmvfc_tgt_add_nvme_rport - Tell the FC transport about a new remote port
+ * @tgt:		ibmvfc target struct
+ *
+ **/
+static void ibmvfc_tgt_add_nvme_rport(struct ibmvfc_target *tgt)
+{
+	struct ibmvfc_host *vhost = tgt->vhost;
+	struct nvme_fc_remote_port *rport;
+	unsigned long flags;
+
+	tgt_dbg(tgt, "Adding NVMe rport\n");
+	ibmvfc_nvme_register_remoteport(tgt);
+	spin_lock_irqsave(vhost->host->host_lock, flags);
+	rport = tgt->nvme_remote_port;
+
+	if (rport && tgt->action == IBMVFC_TGT_ACTION_DEL_RPORT) {
+		tgt_dbg(tgt, "Deleting NVMe rport\n");
+		list_del(&tgt->queue);
+		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_DELETED_RPORT);
+		spin_unlock_irqrestore(vhost->host->host_lock, flags);
+		ibmvfc_nvme_unregister_remoteport(tgt);
+		timer_delete_sync(&tgt->timer);
+		kref_put(&tgt->kref, ibmvfc_release_tgt);
+		return;
+	} else if (rport && tgt->action == IBMVFC_TGT_ACTION_DEL_AND_LOGOUT_RPORT) {
+		tgt_dbg(tgt, "Deleting NVMe rport with outstanding I/O\n");
+		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_LOGOUT_DELETED_RPORT);
+		tgt->init_retries = 0;
+		spin_unlock_irqrestore(vhost->host->host_lock, flags);
+		ibmvfc_nvme_unregister_remoteport(tgt);
+		return;
+	} else if (rport && tgt->action == IBMVFC_TGT_ACTION_DELETED_RPORT) {
+		spin_unlock_irqrestore(vhost->host->host_lock, flags);
+		return;
+	}
+
+	if (rport) {
+		tgt_dbg(tgt, "NVMe rport add succeeded\n");
+		tgt->target_id = tgt->nvme_remote_port->port_id;
+	} else
+		tgt_dbg(tgt, "NVMe rport add failed\n");
+	spin_unlock_irqrestore(vhost->host->host_lock, flags);
+}
+
 /**
  * ibmvfc_tgt_add_rport - Tell the FC transport about a new remote port
  * @tgt:		ibmvfc target struct
@@ -6630,6 +6675,7 @@ static void ibmvfc_rport_add_thread(struct work_struct *work)
 						 rport_add_work_q);
 	struct ibmvfc_target *tgt;
 	struct fc_rport *rport;
+	struct nvme_fc_remote_port *nvme_rport;
 	unsigned long flags;
 	int did_work;
 
@@ -6663,7 +6709,27 @@ static void ibmvfc_rport_add_thread(struct work_struct *work)
 				break;
 			}
 		}
-	} while(did_work);
+
+		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue) {
+			if (tgt->add_rport) {
+				did_work = 1;
+				tgt->add_rport = 0;
+				kref_get(&tgt->kref);
+				nvme_rport = tgt->nvme_remote_port;
+				if (!nvme_rport) {
+					spin_unlock_irqrestore(vhost->host->host_lock, flags);
+					ibmvfc_tgt_add_nvme_rport(tgt);
+				} else {
+					spin_unlock_irqrestore(vhost->host->host_lock, flags);
+					nvme_fc_rescan_remoteport(nvme_rport);
+				}
+
+				kref_put(&tgt->kref, ibmvfc_release_tgt);
+				spin_lock_irqsave(vhost->host->host_lock, flags);
+				break;
+			}
+		}
+	} while (did_work);
 
 	if (vhost->state == IBMVFC_ACTIVE)
 		vhost->scan_complete = 1;
-- 
2.54.0

[PATCH 22/29] ibmvfc: extend ibmvfc_debug visibility to ibmvfc-nvme.h

[Tyrel Datwyler]

Export ibmvfc_debug so the NVMe support code can use the
existing ibmvfc_dbg logging macro.

The debug control variable is currently file-local to the core driver,
which prevents protocol-specific code in ibmvfc-nvme.c from using the
shared debug infrastructure. Make the variable global within the module
and declare it in ibmvfc-nvme.h.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 3 ++-
 drivers/scsi/ibmvscsi/ibmvfc-nvme.h | 2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 5f6ee99d0dba..7e6912fba899 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -41,7 +41,6 @@ static unsigned int max_requests = IBMVFC_MAX_REQUESTS_DEFAULT;
 static u16 max_sectors = IBMVFC_MAX_SECTORS;
 static u16 scsi_qdepth = IBMVFC_SCSI_QDEPTH;
 static unsigned int disc_threads = IBMVFC_MAX_DISC_THREADS;
-static unsigned int ibmvfc_debug = IBMVFC_DEBUG;
 static unsigned int log_level = IBMVFC_DEFAULT_LOG_LEVEL;
 static unsigned int cls3_error = IBMVFC_CLS3_ERROR;
 static unsigned int mq_enabled = IBMVFC_MQ;
@@ -53,6 +52,8 @@ static unsigned int nr_nvme_channels = IBMVFC_NVME_CHANNELS;
 static unsigned int mig_channels_only = IBMVFC_MIG_NO_SUB_TO_CRQ;
 static unsigned int mig_no_less_channels = IBMVFC_MIG_NO_N_TO_M;
 
+unsigned int ibmvfc_debug = IBMVFC_DEBUG;
+
 static LIST_HEAD(ibmvfc_head);
 static DEFINE_SPINLOCK(ibmvfc_driver_lock);
 static struct scsi_transport_template *ibmvfc_transport_template;
diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
index 0465e8719881..3aa285788795 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
@@ -22,6 +22,8 @@
 #define IBMVFC_MAX_NVME_QUEUES	16
 #define IBMVFC_NVME_CHANNELS	8
 
+extern unsigned int ibmvfc_debug;
+
 struct ibmvfc_host;
 struct ibmvfc_target;
 
-- 
2.54.0

[PATCH 23/29] ibmvfc: declare global function definitions

[Tyrel Datwyler]

Some common functions will require visibility by both SCSI and NVMe
protocols. Make common ibmvfc helper routines available to the NVMe
support code.

Remove static from the core event allocation, event initialization,
event free, target release, command error, and event send helpers, and
declare them in ibmvfc.h. This allows ibmvfc-nvme.c to reuse the
existing event and target infrastructure.

No functional change is intended.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 22 ++++++----------------
 drivers/scsi/ibmvscsi/ibmvfc.h      | 17 +++++++++++++++++
 2 files changed, 23 insertions(+), 16 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 7e6912fba899..177d341ce7bc 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -206,13 +206,6 @@ static long h_reg_sub_crq(unsigned long unit_address, unsigned long ioba,
 	return rc;
 }
 
-static int ibmvfc_check_caps(struct ibmvfc_host *vhost, unsigned long cap_flags)
-{
-	u64 host_caps = be64_to_cpu(vhost->login_buf->resp.capabilities);
-
-	return (host_caps & cap_flags) ? 1 : 0;
-}
-
 static int ibmvfc_nvme_active(struct ibmvfc_host *vhost)
 {
 	return (ibmvfc_check_caps(vhost, IBMVFC_SUPPORT_NVMEOF) &&
@@ -354,7 +347,7 @@ static int ibmvfc_get_err_index(u16 status, u16 error)
  * Return value:
  *	error description string
  **/
-static const char *ibmvfc_get_cmd_error(u16 status, u16 error)
+const char *ibmvfc_get_cmd_error(u16 status, u16 error)
 {
 	int rc = ibmvfc_get_err_index(status, error);
 	if (rc >= 0)
@@ -1076,7 +1069,7 @@ static int ibmvfc_valid_event(struct ibmvfc_event_pool *pool,
  * @evt:	ibmvfc_event to be freed
  *
  **/
-static void ibmvfc_free_event(struct ibmvfc_event *evt)
+void ibmvfc_free_event(struct ibmvfc_event *evt)
 {
 	struct ibmvfc_event_pool *pool = &evt->queue->evt_pool;
 	unsigned long flags;
@@ -1410,7 +1403,7 @@ static void ibmvfc_set_rport_dev_loss_tmo(struct fc_rport *rport, u32 timeout)
  * @kref:		kref struct
  *
  **/
-static void ibmvfc_release_tgt(struct kref *kref)
+void ibmvfc_release_tgt(struct kref *kref)
 {
 	struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);
 	kfree(tgt);
@@ -1588,7 +1581,7 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost)
  *
  * Returns a free event from the pool.
  **/
-static struct ibmvfc_event *__ibmvfc_get_event(struct ibmvfc_queue *queue, int reserved)
+struct ibmvfc_event *__ibmvfc_get_event(struct ibmvfc_queue *queue, int reserved)
 {
 	struct ibmvfc_event *evt = NULL;
 	unsigned long flags;
@@ -1612,9 +1605,6 @@ static struct ibmvfc_event *__ibmvfc_get_event(struct ibmvfc_queue *queue, int r
 	return evt;
 }
 
-#define ibmvfc_get_event(queue) __ibmvfc_get_event(queue, 0)
-#define ibmvfc_get_reserved_event(queue) __ibmvfc_get_event(queue, 1)
-
 /**
  * ibmvfc_locked_done - Calls evt completion with host_lock held
  * @evt:	ibmvfc evt to complete
@@ -1639,7 +1629,7 @@ static void ibmvfc_locked_done(struct ibmvfc_event *evt)
  * @done:	Routine to call when the event is responded to
  * @format:	SRP or MAD format
  **/
-static void ibmvfc_init_event(struct ibmvfc_event *evt,
+void ibmvfc_init_event(struct ibmvfc_event *evt,
 			      void (*done) (struct ibmvfc_event *), u8 format)
 {
 	evt->cmnd = NULL;
@@ -1764,7 +1754,7 @@ static void ibmvfc_timeout(struct timer_list *t)
  *
  * Returns the value returned from ibmvfc_send_crq(). (Zero for success)
  **/
-static int ibmvfc_send_event(struct ibmvfc_event *evt,
+int ibmvfc_send_event(struct ibmvfc_event *evt,
 			     struct ibmvfc_host *vhost, unsigned long timeout)
 {
 	__be64 *crq_as_u64 = (__be64 *) &evt->crq;
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index d8c2e5f1fdec..ece1f379c269 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -1021,6 +1021,23 @@ struct ibmvfc_host {
 	struct completion nvme_delete_done;
 };
 
+struct ibmvfc_event *__ibmvfc_get_event(struct ibmvfc_queue *queue, int reserved);
+#define ibmvfc_get_event(queue) __ibmvfc_get_event(queue, 0)
+#define ibmvfc_get_reserved_event(queue) __ibmvfc_get_event(queue, 1)
+
+void ibmvfc_init_event(struct ibmvfc_event *evt, void (*done) (struct ibmvfc_event *), u8 format);
+void ibmvfc_free_event(struct ibmvfc_event *evt);
+void ibmvfc_release_tgt(struct kref *kref);
+int ibmvfc_send_event(struct ibmvfc_event *evt, struct ibmvfc_host *vhost, unsigned long timeout);
+const char *ibmvfc_get_cmd_error(u16 status, u16 error);
+
+static inline int ibmvfc_check_caps(struct ibmvfc_host *vhost, unsigned long cap_flags)
+{
+	u64 host_caps = be64_to_cpu(vhost->login_buf->resp.capabilities);
+
+	return (host_caps & cap_flags) ? 1 : 0;
+}
+
 static inline struct ibmvfc_host *ibmvfc_channels_to_vhost(struct ibmvfc_channels *channels)
 {
 	if (channels->protocol == IBMVFC_PROTO_SCSI)
-- 
2.54.0

[PATCH 24/29] ibmvfc: implement LLDD callbacks for mapping nvme-fc queues

[Tyrel Datwyler]

Implement the NVMe-FC queue create and delete callbacks and map NVMe
controller queues onto ibmvfc hardware queues.

Use qidx of NVMe controller queue to map onto a ibmvfc_queue channel.
The Admin queue is always qidx 0 and general practice among other
drivers is to map both the Admin queue and first IO queue to the same HW
queue. Add a new ibmvfc_nvme_qhandle struct that will be used as the
opaque queue handle by the NVMe-FC layer when issuing fcp IO.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 38 +++++++++++++++++++++++++++--
 drivers/scsi/ibmvscsi/ibmvfc-nvme.h |  7 ++++++
 2 files changed, 43 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
index fc4337fc9b3f..1108d11d6b2d 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
@@ -28,6 +28,40 @@ static void ibmvfc_nvme_remoteport_delete(struct nvme_fc_remote_port *rport)
 	complete(&tgt->nvme_delete_done);
 }
 
+static int ibmvfc_nvme_create_queue(struct nvme_fc_local_port *lport, unsigned int qidx,
+				    u16 qsize, void **handle)
+{
+	struct ibmvfc_host *vhost = lport->private;
+	struct ibmvfc_nvme_qhandle *qhandle;
+
+	if (!vhost->nvme_scrqs.active_queues)
+		return -ENODEV;
+
+	qhandle = kzalloc_obj(struct ibmvfc_nvme_qhandle);
+	if (!qhandle)
+		return -ENOMEM;
+
+	qhandle->cpu_id = raw_smp_processor_id();
+	qhandle->qidx = qidx;
+
+	/* Admin and first IO queue are both mapped to index 0 */
+	if (qidx)
+		qhandle->index = (qidx - 1) % vhost->nvme_scrqs.active_queues;
+	else
+		qhandle->index = qidx;
+
+	qhandle->queue = &vhost->nvme_scrqs.scrqs[qhandle->index];
+
+	*handle = qhandle;
+	return 0;
+}
+
+static void ibmvfc_nvme_delete_queue(struct nvme_fc_local_port *lport, unsigned int qidx,
+				     void *handle)
+{
+	kfree(handle);
+}
+
 static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport,
 			      struct nvme_fc_remote_port *rport,
 			      struct nvmefc_ls_req *ls_req)
@@ -59,8 +93,8 @@ static void ibmvfc_nvme_fcp_abort(struct nvme_fc_local_port *lport,
 static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = {
 	.localport_delete	= ibmvfc_nvme_localport_delete,
 	.remoteport_delete	= ibmvfc_nvme_remoteport_delete,
-	.create_queue		= NULL,
-	.delete_queue		= NULL,
+	.create_queue		= ibmvfc_nvme_create_queue,
+	.delete_queue		= ibmvfc_nvme_delete_queue,
 	.ls_req			= ibmvfc_nvme_ls_req,
 	.ls_abort		= ibmvfc_nvme_ls_abort,
 	.fcp_io			= ibmvfc_nvme_fcp_io,
diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
index 3aa285788795..4c6146048a6f 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
@@ -27,6 +27,13 @@ extern unsigned int ibmvfc_debug;
 struct ibmvfc_host;
 struct ibmvfc_target;
 
+struct ibmvfc_nvme_qhandle {
+	unsigned int qidx;
+	u16 cpu_id;
+	unsigned long index;
+	struct ibmvfc_queue *queue;
+};
+
 int ibmvfc_nvme_register_remoteport(struct ibmvfc_target *tgt);
 void ibmvfc_nvme_unregister_remoteport(struct ibmvfc_target *tgt);
 int ibmvfc_nvme_register(struct ibmvfc_host *vhost);
-- 
2.54.0

[PATCH 25/29] ibmvfc: implement nvme-fc LS submission transport callback

[Tyrel Datwyler]

NVMe FC Link Service commands are required to use the ibmvfc_passthru
MAD. Initialize a pssthru mad for the target port including the DMA
addresses for the FC4_LS request and response as well as the max length
of each IU as provided in the nvmefc_ls_req struct. FC4_LS commands are
sent via the primary CRQ. Further, store the assoc_id during a create
association request as this is a required field in our vfc_cmd struct
for nvme_fcp_io commands.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 89 +++++++++++++++++++++++++++++
 drivers/scsi/ibmvscsi/ibmvfc-nvme.h |  8 +++
 drivers/scsi/ibmvscsi/ibmvfc.h      |  2 +
 3 files changed, 99 insertions(+)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
index 1108d11d6b2d..506135c1a34e 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
@@ -62,10 +62,99 @@ static void ibmvfc_nvme_delete_queue(struct nvme_fc_local_port *lport, unsigned
 	kfree(handle);
 }
 
+static void ibmvfc_ls_req_done(struct ibmvfc_event *evt)
+{
+	struct ibmvfc_target *tgt = evt->tgt;
+	struct ibmvfc_passthru_mad *mad = &evt->xfer_iu->passthru;
+	struct fcnvme_ls_rqst_w0 *ls_rqst;
+	struct fcnvme_ls_cr_assoc_acc *ls_resp;
+	u32 status = be16_to_cpu(mad->common.status);
+	int rc = 0;
+
+	ls_rqst = (struct fcnvme_ls_rqst_w0 *)evt->ls_req->rqstaddr;
+	ls_resp = (struct fcnvme_ls_cr_assoc_acc *)evt->ls_req->rspaddr;
+
+	switch (status) {
+	case IBMVFC_MAD_SUCCESS:
+		tgt_dbg(tgt, "ls_req succeeded\n");
+		if ((ls_rqst->ls_cmd == FCNVME_LS_CREATE_ASSOCIATION) &&
+		    (ls_resp->hdr.w0.ls_cmd == FCNVME_LS_ACC)) {
+			tgt->assoc_id = be64_to_cpu(ls_resp->associd.association_id);
+			tgt_dbg(tgt, "assoc_id 0x%llx\n", tgt->assoc_id);
+		}
+		break;
+	case IBMVFC_MAD_DRIVER_FAILED:
+		break;
+	case IBMVFC_MAD_FAILED:
+	default:
+		tgt_info(tgt, "ls_req failed: %s (%x:%x) rc=0x%02X\n",
+			 ibmvfc_get_cmd_error(be16_to_cpu(mad->iu.status), be16_to_cpu(mad->iu.error)),
+			 be16_to_cpu(mad->iu.status), be16_to_cpu(mad->iu.error), status);
+		break;
+	}
+
+	if (status)
+		rc = -EIO;
+
+	evt->ls_req->done(evt->ls_req, rc);
+
+	kref_put(&tgt->kref, ibmvfc_release_tgt);
+	ibmvfc_free_event(evt);
+}
+
+static void ibmvfc_init_ls_req(struct ibmvfc_event *evt, struct nvmefc_ls_req *ls_req)
+{
+	struct ibmvfc_passthru_mad *mad = &evt->iu.passthru;
+
+	memset(mad, 0, sizeof(*mad));
+	mad->common.version = cpu_to_be32(2);
+	mad->common.opcode = cpu_to_be32(IBMVFC_NVMF_PASSTHRU);
+	mad->common.length = cpu_to_be16(sizeof(*mad) - sizeof(mad->fc_iu) - sizeof(mad->iu));
+	mad->cmd_ioba.va = cpu_to_be64((u64)be64_to_cpu(evt->crq.ioba) +
+				       offsetof(struct ibmvfc_passthru_mad, iu));
+	mad->cmd_ioba.len = cpu_to_be32(sizeof(mad->iu));
+	mad->iu.cmd_len = cpu_to_be32(ls_req->rqstlen);
+	mad->iu.rsp_len = cpu_to_be32(ls_req->rsplen);
+	mad->iu.cmd.va = cpu_to_be64(ls_req->rqstdma);
+	mad->iu.cmd.len = cpu_to_be32(ls_req->rqstlen);
+	mad->iu.rsp.va = cpu_to_be64(ls_req->rspdma);
+	mad->iu.rsp.len = cpu_to_be32(ls_req->rsplen);
+}
+
 static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport,
 			      struct nvme_fc_remote_port *rport,
 			      struct nvmefc_ls_req *ls_req)
 {
+	struct ibmvfc_host *vhost = lport->private;
+	struct ibmvfc_target *tgt = rport->private;
+	struct ibmvfc_passthru_mad *mad;
+	struct ibmvfc_event *evt;
+
+	kref_get(&tgt->kref);
+	evt = ibmvfc_get_event(&vhost->crq);
+	if (!evt) {
+		kref_put(&tgt->kref, ibmvfc_release_tgt);
+		return -EBUSY;
+	}
+
+	ibmvfc_init_event(evt, ibmvfc_ls_req_done, IBMVFC_MAD_FORMAT);
+	evt->tgt = tgt;
+	evt->ls_req = ls_req;
+	ls_req->private = evt;
+
+	ibmvfc_init_ls_req(evt, ls_req);
+	mad = &evt->iu.passthru;
+	mad->iu.flags = cpu_to_be32(IBMVFC_FC4_LS_DSC_CTRL);
+	mad->iu.scsi_id = cpu_to_be64(tgt->scsi_id);
+	mad->iu.cancel_key = cpu_to_be32((u64)evt);
+	mad->iu.target_wwpn = cpu_to_be64(tgt->wwpn);
+
+	ibmvfc_dbg(vhost, "nvme_ls_req\n");
+	if (ibmvfc_send_event(evt, vhost, IBMVFC_FC4_LS_PLUS_CANCEL_TIMEOUT)) {
+		kref_put(&tgt->kref, ibmvfc_release_tgt);
+		return -ENXIO;
+	}
+
 	return 0;
 }
 
diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
index 4c6146048a6f..e245b6ed0875 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
@@ -14,6 +14,8 @@
 #include <scsi/fc/fc_fs.h>
 #include <scsi/fc/fc_els.h>
 #include <linux/nvme-fc-driver.h>
+#include <linux/nvme.h>
+#include <linux/nvme-fc.h>
 
 #include "ibmvfc.h"
 
@@ -22,10 +24,16 @@
 #define IBMVFC_MAX_NVME_QUEUES	16
 #define IBMVFC_NVME_CHANNELS	8
 
+#define IBMVFC_FC4_LS_TIMEOUT	15
+#define IBMVFC_FC4_LS_CANCEL_TIMEOUT	45
+#define IBMVFC_FC4_LS_PLUS_CANCEL_TIMEOUT	\
+	(IBMVFC_FC4_LS_TIMEOUT + IBMVFC_FC4_LS_CANCEL_TIMEOUT)
+
 extern unsigned int ibmvfc_debug;
 
 struct ibmvfc_host;
 struct ibmvfc_target;
+struct ibmvfc_queue;
 
 struct ibmvfc_nvme_qhandle {
 	unsigned int qidx;
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index ece1f379c269..9f6705e604fd 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -824,6 +824,7 @@ struct ibmvfc_target {
 	u64 scsi_id;
 	u64 wwpn;
 	u64 new_scsi_id;
+	u64 assoc_id;
 	struct fc_rport *rport;
 	int target_id;
 	enum ibmvfc_target_action action;
@@ -851,6 +852,7 @@ struct ibmvfc_event {
 	struct ibmvfc_queue *queue;
 	struct ibmvfc_target *tgt;
 	struct scsi_cmnd *cmnd;
+	struct nvmefc_ls_req *ls_req;
 	atomic_t free;
 	atomic_t active;
 	union ibmvfc_iu *xfer_iu;
-- 
2.54.0

[PATCH 26/29] ibmvfc: implement nvme-fc IO command submission callback

[Tyrel Datwyler]

Add helpers to initialize an ibmvfc command from an nvmefc_fcp_req,
map request scatterlists into either an inline descriptor or an external
DMA pool list, and submit the request on the selected NVMe hardware
queue. On completion, translate ibmvfc status into the NVMe-FC response
format, including transferred length and CQE handling for no-DMA
responses.

Also store the NVMe request pointer in struct ibmvfc_event so the
completion path can finish the original request.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c |   2 +-
 drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 140 +++++++++++++++++++++++++++-
 drivers/scsi/ibmvscsi/ibmvfc.h      |   1 +
 3 files changed, 141 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index 177d341ce7bc..a7183493cf96 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -1539,7 +1539,7 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost)
 
 	login_info->ostype = cpu_to_be32(IBMVFC_OS_LINUX);
 	login_info->max_dma_len = cpu_to_be64(max_sectors << 9);
-	login_info->max_payload = cpu_to_be32(sizeof(struct ibmvfc_fcp_cmd_iu));
+	login_info->max_payload = cpu_to_be32(sizeof(struct nvme_fc_cmd_iu));
 	login_info->max_response = cpu_to_be32(sizeof(struct ibmvfc_fcp_rsp));
 	login_info->partition_num = cpu_to_be32(vhost->partition_number);
 	login_info->vfc_frame_version = cpu_to_be32(1);
diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
index 506135c1a34e..bff469d0b47d 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
@@ -8,6 +8,7 @@
  * Copyright (C) IBM Corporation, 2026
  */
 
+#include <linux/dmapool.h>
 #include <scsi/scsi_transport_fc.h>
 
 #include "ibmvfc-nvme.h"
@@ -164,12 +165,149 @@ static void ibmvfc_nvme_ls_abort(struct nvme_fc_local_port *lport,
 {
 }
 
+static void ibmvfc_nvme_done(struct ibmvfc_event *evt)
+{
+	struct ibmvfc_cmd *vfc_cmd = &evt->xfer_iu->cmd;
+	struct nvmefc_fcp_req *fcp_req = evt->fcp_req;
+	struct nvme_fc_ersp_iu *ersp = (struct nvme_fc_ersp_iu *)fcp_req->rspaddr;
+	struct nvme_completion *cqe = &ersp->cqe;
+	struct nvme_command *sqe = &((struct nvme_fc_cmd_iu *)fcp_req->cmdaddr)->sqe;
+
+	ibmvfc_dbg(evt->vhost, "fc_done: (%x:%x)\n", be16_to_cpu(vfc_cmd->status),
+		   be16_to_cpu(vfc_cmd->error));
+	ibmvfc_dbg(evt->vhost, "fc_done: cmdlen: %d, rsplen %d, payload_len %d\n",
+		   fcp_req->cmdlen, fcp_req->rsplen, fcp_req->payload_length);
+
+	fcp_req->status = 0;
+	if (!vfc_cmd->status) {
+		fcp_req->rcv_rsplen = NVME_FC_SIZEOF_ZEROS_RSP;
+		fcp_req->transferred_length = fcp_req->payload_length;
+	} else if (be16_to_cpu(vfc_cmd->status) & IBMVFC_FC_NVME_STATUS) {
+		fcp_req->rcv_rsplen = sizeof(struct nvme_fc_ersp_iu);
+		fcp_req->transferred_length = be32_to_cpu(ersp->xfrd_len);
+		if (be16_to_cpu(vfc_cmd->error) & IBMVFC_NVMS_VALID_NODMA_CQE)
+			cqe->command_id = sqe->common.command_id;
+	} else {
+		fcp_req->rcv_rsplen = 0;
+		fcp_req->transferred_length = 0;
+		fcp_req->status = NVME_SC_INTERNAL;
+	}
+
+	fcp_req->done(fcp_req);
+	ibmvfc_free_event(evt);
+}
+
+static struct ibmvfc_cmd *ibmvfc_nvme_init_vfc_cmd(struct ibmvfc_event *evt,
+						   struct nvme_fc_remote_port *rport,
+						   struct nvmefc_fcp_req *fcp_req)
+{
+	struct ibmvfc_target *tgt = rport->private;
+	struct ibmvfc_cmd *vfc_cmd = &evt->iu.cmd;
+
+	memset(vfc_cmd, 0, sizeof(*vfc_cmd));
+
+	vfc_cmd->resp.va = cpu_to_be64(fcp_req->rspdma);
+	vfc_cmd->resp.len = cpu_to_be32(fcp_req->rsplen);
+	vfc_cmd->frame_type = cpu_to_be32(IBMVFC_NVME_FCP_TYPE);
+	vfc_cmd->flags |= cpu_to_be16(IBMVFC_NVMEOF_PROTOCOL);
+	vfc_cmd->payload_len = cpu_to_be32(fcp_req->cmdlen);
+	vfc_cmd->resp_len = cpu_to_be32(fcp_req->rsplen);
+	vfc_cmd->cancel_key = cpu_to_be32((u64)evt);
+	vfc_cmd->target_wwpn = cpu_to_be64(rport->port_name);
+	vfc_cmd->tgt_scsi_id = cpu_to_be64(rport->port_id);
+	vfc_cmd->assoc_id = cpu_to_be64(tgt->assoc_id);
+
+	memcpy(&vfc_cmd->v3nvme.iu, fcp_req->cmdaddr, fcp_req->cmdlen);
+
+	return vfc_cmd;
+}
+
+static void ibmvfc_nvme_map_sg_list(struct nvmefc_fcp_req *fcp_req,
+				    struct srp_direct_buf *md)
+{
+	int i;
+	struct scatterlist *sg;
+
+	for_each_sg(fcp_req->first_sgl, sg, fcp_req->sg_cnt, i) {
+		md[i].va = cpu_to_be64(sg_dma_address(sg));
+		md[i].len = cpu_to_be32(sg_dma_len(sg));
+		md[i].key = 0;
+	}
+}
+
+static int ibmvfc_nvme_map_sg_data(struct nvmefc_fcp_req *fcp_req,
+				    struct ibmvfc_event *evt,
+				    struct ibmvfc_cmd *vfc_cmd)
+{
+	struct srp_direct_buf *data = &vfc_cmd->ioba;
+	struct ibmvfc_host *vhost = evt->vhost;
+
+	if (!fcp_req->sg_cnt) {
+		vfc_cmd->flags |= cpu_to_be16(IBMVFC_NO_MEM_DESC);
+		return 0;
+	}
+
+	if (fcp_req->io_dir == NVMEFC_FCP_WRITE)
+		vfc_cmd->flags |= cpu_to_be16(IBMVFC_WRITE);
+	else
+		vfc_cmd->flags |= cpu_to_be16(IBMVFC_READ);
+
+	if (fcp_req->sg_cnt == 1) {
+		ibmvfc_nvme_map_sg_list(fcp_req, data);
+		return 0;
+	}
+
+	vfc_cmd->flags |= cpu_to_be16(IBMVFC_SCATTERLIST);
+
+	if (!evt->ext_list) {
+		evt->ext_list = dma_pool_alloc(vhost->sg_pool, GFP_ATOMIC,
+					       &evt->ext_list_token);
+
+		if (!evt->ext_list)
+			return -ENOMEM;
+	}
+
+	ibmvfc_nvme_map_sg_list(fcp_req, evt->ext_list);
+
+	data->va = cpu_to_be64(evt->ext_list_token);
+	data->len = cpu_to_be32(fcp_req->sg_cnt * sizeof(struct srp_direct_buf));
+	data->key = 0;
+	return 0;
+}
+
 static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport,
 			      struct nvme_fc_remote_port *rport,
 			      void *hw_queue_handle,
 			      struct nvmefc_fcp_req *fcp_req)
 {
-	return 0;
+	struct ibmvfc_host *vhost = lport->private;
+	struct ibmvfc_nvme_qhandle *qhandle = hw_queue_handle;
+	struct ibmvfc_cmd *vfc_cmd;
+	struct ibmvfc_event *evt;
+	int rc;
+
+	ibmvfc_dbg(vhost, "nvme_fcp_io\n");
+	evt = ibmvfc_get_event(qhandle->queue);
+	if (!evt)
+		return -EBUSY;
+
+	evt->hwq = qhandle->index;
+	ibmvfc_dbg(vhost, "vfc-nvme-mq-%d\n", evt->hwq);
+
+	ibmvfc_init_event(evt, ibmvfc_nvme_done, IBMVFC_CMD_FORMAT);
+	evt->fcp_req = fcp_req;
+	fcp_req->private = evt;
+
+	vfc_cmd = ibmvfc_nvme_init_vfc_cmd(evt, rport, fcp_req);
+
+	vfc_cmd->correlation = cpu_to_be64((u64)evt);
+
+	if (likely(!(rc = ibmvfc_nvme_map_sg_data(fcp_req, evt, vfc_cmd))))
+		return ibmvfc_send_event(evt, vhost, 0);
+
+	ibmvfc_free_event(evt);
+
+	return rc;
 }
 
 static void ibmvfc_nvme_fcp_abort(struct nvme_fc_local_port *lport,
diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
index 9f6705e604fd..ca80ceffe53a 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc.h
+++ b/drivers/scsi/ibmvscsi/ibmvfc.h
@@ -853,6 +853,7 @@ struct ibmvfc_event {
 	struct ibmvfc_target *tgt;
 	struct scsi_cmnd *cmnd;
 	struct nvmefc_ls_req *ls_req;
+	struct nvmefc_fcp_req *fcp_req;
 	atomic_t free;
 	atomic_t active;
 	union ibmvfc_iu *xfer_iu;
-- 
2.54.0

[PATCH 27/29] ibmvfc: implement nvme-fc LS abort handling callback

[Tyrel Datwyler]

Implement the NVMe FC-LS abort callback by issuing an ibmvfc cancel MAD
to the VIOS for the outstanding link-service request.

Use the saved event pointer from the original FC-LS request to identify
the command to cancel, submit the cancel operation, and complete the
abort request based on the returned status.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 64 +++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
index bff469d0b47d..18e8657abc44 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
@@ -13,6 +13,8 @@
 
 #include "ibmvfc-nvme.h"
 
+static unsigned int default_timeout = IBMVFC_DEFAULT_TIMEOUT;
+
 static void ibmvfc_nvme_localport_delete(struct nvme_fc_local_port *lport)
 {
 	struct ibmvfc_host *vhost = lport->private;
@@ -159,10 +161,72 @@ static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport,
 	return 0;
 }
 
+static void ibmvfc_sync_nvme_completion(struct ibmvfc_event *evt)
+{
+	/* copy the response back */
+	if (evt->sync_iu)
+		*evt->sync_iu = *evt->xfer_iu;
+
+	complete(&evt->comp);
+}
+
+static void ibmvfc_init_ls_abort(struct ibmvfc_event *evt, struct nvmefc_ls_req *ls_abort)
+{
+	struct ibmvfc_tmf *tmf;
+	struct ibmvfc_event *abt_evt = ls_abort->private;
+	struct ibmvfc_target *tgt = abt_evt->tgt;
+	struct ibmvfc_host *vhost = evt->vhost;
+
+	tmf = &evt->iu.tmf;
+	memset(tmf, 0, sizeof(*tmf));
+	tmf->common.version = cpu_to_be32(2);
+	tmf->target_wwpn = cpu_to_be64(tgt->wwpn);
+	tmf->common.opcode = cpu_to_be32(IBMVFC_NVMF_TMF_MAD);
+	tmf->common.length = cpu_to_be16(sizeof(*tmf));
+	if (vhost->state != IBMVFC_ACTIVE)
+		if (!ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPRESS_ABTS))
+			tmf->flags = cpu_to_be32(IBMVFC_TMF_SUPPRESS_ABTS);
+	tmf->cancel_key = cpu_to_be32((u64)abt_evt);
+	tmf->my_cancel_key = cpu_to_be32((u64)evt);
+	tmf->assoc_id = cpu_to_be64(tgt->assoc_id);
+
+	init_completion(&evt->comp);
+}
+
 static void ibmvfc_nvme_ls_abort(struct nvme_fc_local_port *lport,
 				struct nvme_fc_remote_port *rport,
 				struct nvmefc_ls_req *ls_abort)
 {
+	struct ibmvfc_host *vhost = lport->private;
+	struct ibmvfc_target *tgt = rport->private;
+	struct ibmvfc_event *evt;
+	union ibmvfc_iu rsp;
+	unsigned long flags;
+	u16 status;
+
+	evt = ibmvfc_get_event(&vhost->crq);
+	if (!vhost->logged_in || !evt)
+		return;
+
+	spin_lock_irqsave(vhost->host->host_lock, flags);
+	kref_get(&tgt->kref);
+	ibmvfc_init_event(evt, ibmvfc_sync_nvme_completion, IBMVFC_MAD_FORMAT);
+	ibmvfc_init_ls_abort(evt, ls_abort);
+	evt->sync_iu = &rsp;
+
+	if (ibmvfc_send_event(evt, vhost, default_timeout))
+		goto out;
+
+	spin_unlock_irqrestore(vhost->host->host_lock, flags);
+
+	wait_for_completion(&evt->comp);
+	status = be16_to_cpu(rsp.mad_common.status);
+	spin_lock_irqsave(vhost->host->host_lock, flags);
+	ibmvfc_free_event(evt);
+out:
+	spin_unlock_irqrestore(vhost->host->host_lock, flags);
+	ibmvfc_dbg(vhost, "ls_abort: cancel failed with rc=%x\n", status);
+	kref_put(&tgt->kref, ibmvfc_release_tgt);
 }
 
 static void ibmvfc_nvme_done(struct ibmvfc_event *evt)
-- 
2.54.0

[PATCH 28/29] ibmvfc: implement nvme-fc FCP abort callback

[Tyrel Datwyler]

Implement the NVMe-FC FCP abort callback by issuing an NVMF cancel MAD
on the same submission queue used by the original FCP request.

Use the original request event stored in abort_req->private to recover
the associated ibmvfc queue, then allocate a new event from that queue
so the cancel is sent on the matching nvme_scrq. Factor the TMF setup
into a dedicated helper, mirroring the LS abort path, and populate the
cancel key, task tag, target WWPN, and association ID needed for the
VIOS NVMF abort request.

The abort path sends the cancel synchronously, waits for completion,
frees the temporary event, and logs non-zero MAD status values for
debugging.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-nvme.c | 64 +++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
index 18e8657abc44..92937f9aa464 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
@@ -374,11 +374,75 @@ static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport,
 	return rc;
 }
 
+static void ibmvfc_init_fcp_abort(struct ibmvfc_event *evt,
+				  struct nvmefc_fcp_req *abort_req)
+{
+	struct ibmvfc_tmf *tmf;
+	struct ibmvfc_event *abt_evt = abort_req->private;
+	struct ibmvfc_target *tgt = abt_evt->tgt;
+
+	tmf = &evt->iu.tmf;
+	memset(tmf, 0, sizeof(*tmf));
+	tmf->common.version = cpu_to_be32(2);
+	tmf->common.opcode = cpu_to_be32(IBMVFC_NVMF_TMF_MAD);
+	tmf->common.length = cpu_to_be16(sizeof(*tmf));
+	tmf->flags = cpu_to_be32(IBMVFC_TMF_ABORT_TASK | IBMVFC_TMF_NVMF_ASSOC);
+	tmf->cancel_key = cpu_to_be32((u64)abt_evt);
+	tmf->my_cancel_key = cpu_to_be32((u64)evt);
+	tmf->target_wwpn = cpu_to_be64(tgt->wwpn);
+	tmf->assoc_id = cpu_to_be64(tgt->assoc_id);
+	tmf->task_tag = cpu_to_be64((u64)abt_evt);
+
+	init_completion(&evt->comp);
+}
+
 static void ibmvfc_nvme_fcp_abort(struct nvme_fc_local_port *lport,
 				  struct nvme_fc_remote_port *rport,
 				  void *hw_queue_handle,
 				  struct nvmefc_fcp_req *abort_req)
 {
+	struct ibmvfc_host *vhost = lport->private;
+	struct ibmvfc_target *tgt = rport->private;
+	struct ibmvfc_event *evt, *abt_evt = abort_req->private;
+	struct ibmvfc_queue *queue;
+	union ibmvfc_iu rsp;
+	unsigned long flags;
+	u16 status = 0;
+
+	if (!abt_evt)
+		return;
+
+	queue = abt_evt->queue;
+	if (!vhost->logged_in || !queue)
+		return;
+
+	evt = ibmvfc_get_event(queue);
+	if (!evt)
+		return;
+
+	spin_lock_irqsave(queue->q_lock, flags);
+	kref_get(&tgt->kref);
+	ibmvfc_init_event(evt, ibmvfc_sync_nvme_completion, IBMVFC_MAD_FORMAT);
+	ibmvfc_init_fcp_abort(evt, abort_req);
+	evt->sync_iu = &rsp;
+
+	if (ibmvfc_send_event(evt, vhost, default_timeout))
+		goto out;
+
+	spin_unlock_irqrestore(queue->q_lock, flags);
+
+	wait_for_completion(&evt->comp);
+	status = be16_to_cpu(rsp.mad_common.status);
+
+	spin_lock_irqsave(queue->q_lock, flags);
+	ibmvfc_free_event(evt);
+out:
+	spin_unlock_irqrestore(queue->q_lock, flags);
+
+	if (status)
+		ibmvfc_dbg(vhost, "fcp_abort: cancel failed with rc=%x\n", status);
+
+	kref_put(&tgt->kref, ibmvfc_release_tgt);
 }
 
 static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = {
-- 
2.54.0

[PATCH 29/29] ibmvfc: fail nvme-fc fcp-io and ls requests during transport reset

[Tyrel Datwyler]

The current purge code for flushing outstanding commands during a
transport reset only deals with SCSI commands. Rename the
ibmvfc_scsi_eh_done completion handler to ibmvfc_vfc_eh_done and wire it
to correctly complete nvme fcp and ls commands when flushing the
inflight command list during a reset.

Signed-off-by: Tyrel Datwyler <tyreld@linux.ibm.com>
---
 drivers/scsi/ibmvscsi/ibmvfc-core.c | 30 ++++++++++++++++++++---------
 1 file changed, 21 insertions(+), 9 deletions(-)

diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
index a7183493cf96..aeb5b8902aa5 100644
--- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
+++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
@@ -1092,20 +1092,28 @@ void ibmvfc_free_event(struct ibmvfc_event *evt)
 }
 
 /**
- * ibmvfc_scsi_eh_done - EH done function for queuecommand commands
+ * ibmvfc_vfc_eh_done - EH done function for queued IO
  * @evt:	ibmvfc event struct
  *
- * This function does not setup any error status, that must be done
- * before this function gets called.
+ * This function does not setup any error status for scsi commands, that must be
+ * done before this function gets called.
  **/
-static void ibmvfc_scsi_eh_done(struct ibmvfc_event *evt)
+static void ibmvfc_vfc_eh_done(struct ibmvfc_event *evt)
 {
 	struct scsi_cmnd *cmnd = evt->cmnd;
+	struct nvmefc_ls_req *ls_req = evt->ls_req;
+	struct nvmefc_fcp_req *fcp_req = evt->fcp_req;
 
 	if (cmnd) {
 		scsi_dma_unmap(cmnd);
 		scsi_done(cmnd);
-	}
+	} else if (fcp_req) {
+		fcp_req->rcv_rsplen = 0;
+		fcp_req->transferred_length = 0;
+		fcp_req->status = NVME_SC_INTERNAL;
+		fcp_req->done(fcp_req);
+	} else if (ls_req)
+		ls_req->done(ls_req, -EIO);
 
 	ibmvfc_free_event(evt);
 }
@@ -1146,8 +1154,10 @@ static void ibmvfc_fail_request(struct ibmvfc_event *evt, int error_code)
 	BUG_ON(!atomic_dec_and_test(&evt->active));
 	if (evt->cmnd) {
 		evt->cmnd->result = (error_code << 16);
-		evt->done = ibmvfc_scsi_eh_done;
-	} else
+		evt->done = ibmvfc_vfc_eh_done;
+	} else if (evt->fcp_req || evt->ls_req)
+		evt->done = ibmvfc_vfc_eh_done;
+	else
 		evt->xfer_iu->mad_common.status = cpu_to_be16(IBMVFC_MAD_DRIVER_FAILED);
 
 	timer_delete(&evt->timer);
@@ -1816,8 +1826,10 @@ int ibmvfc_send_event(struct ibmvfc_event *evt,
 		dev_err(vhost->dev, "Send error (rc=%d)\n", rc);
 		if (evt->cmnd) {
 			evt->cmnd->result = DID_ERROR << 16;
-			evt->done = ibmvfc_scsi_eh_done;
-		} else
+			evt->done = ibmvfc_vfc_eh_done;
+		} else if (evt->fcp_req || evt->ls_req)
+			evt->done = ibmvfc_vfc_eh_done;
+		else
 			evt->xfer_iu->mad_common.status = cpu_to_be16(IBMVFC_MAD_CRQ_ERROR);
 
 		evt->done(evt);
-- 
2.54.0

Re: [PATCH 01/29] ibmvfc: move target list from host to protocol specific channel groups

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The driver schedules a background work item (`vhost->rport_add_work_q`) but never explicitly flushes or cancels it during host removal or module unloading, leading to a leaked work struct and a potential use-after-free.
--

commit 455b96a57ea40cd18201b8e8603264b5cfb84e0c
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: move target list from host to protocol specific channel groups

Prepare the driver for protocol-specific target management by moving
the target list and target count out of struct ibmvfc_host and into
struct ibmvfc_channels.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc.c b/drivers/scsi/ibmvscsi/ibmvfc.c
> index 3dd2adda195ee..9129014364423 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc.c
[ ... ]
> @@ -6276,7 +6276,7 @@ static void ibmvfc_rport_add_thread(struct work_struct *work)
>  		if (vhost->state != IBMVFC_ACTIVE)
>  			break;
>  
> -		list_for_each_entry(tgt, &vhost->targets, queue) {
> +		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {

[Severity: High]
This isn't a bug introduced by this patch, but does the driver leak the
rport_add_work_q on module removal?

In ibmvfc_remove(), the vhost structure is freed via scsi_host_put() after
teardown:

drivers/scsi/ibmvscsi/ibmvfc.c:ibmvfc_remove() {
    ...
	kthread_stop(vhost->work_thread);
	fc_remove_host(vhost->host);
	scsi_remove_host(vhost->host);
    ...
	scsi_host_put(vhost->host);
}

However, the teardown sequence never explicitly flushes or cancels
vhost->rport_add_work_q.

If the work thread is actively running or pending on the system workqueue:

drivers/scsi/ibmvscsi/ibmvfc.c:ibmvfc_rport_add_thread() {
	struct ibmvfc_host *vhost = container_of(work, struct ibmvfc_host,
						 rport_add_work_q);
	...
	spin_lock_irqsave(vhost->host->host_lock, flags);
    ...
}

Could it execute after the vhost is deallocated, leading to a use-after-free
when accessing the lock or other memory?

>  			if (tgt->add_rport) {
>  				did_work = 1;
>  				tgt->add_rport = 0;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=1

Re: [PATCH 12/29] ibmvfc: add NVMe/FC Implicit Logout and Move Login support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing NULL pointer check on the event allocated for implicit logout in `ibmvfc_tgt_implicit_logout_and_del()`, leading to a crash in `ibmvfc_send_event()`.
--

commit d7f6f97d0c832e12c02c910b6675c0a3be2fb197
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: add NVMe/FC Implicit Logout and Move Login support

This patch adds protocol-specific handling for implicit logout and
move-login operations on NVMe/FC targets. It extends the existing target
relogin and migration flows by selecting NVMe/FC-specific opcodes and
updating the associated logging.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index 53480d150042e..3e3d77e0d5172 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
[ ... ]
> @@ -4456,7 +4461,7 @@ static void ibmvfc_tgt_implicit_logout_and_del(struct ibmvfc_target *tgt)
>  		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_DEL_RPORT);
>  		kref_put(&tgt->kref, ibmvfc_release_tgt);
>  	} else
> -		tgt_dbg(tgt, "Sent Implicit Logout\n");
> +		tgt_dbg(tgt, "%s Sent Implicit Logout\n", proto_type[tgt->protocol]);
>  }

[Severity: High]
This is a pre-existing issue, but does ibmvfc_tgt_implicit_logout_and_del()
need to check if evt is NULL before calling ibmvfc_send_event()?

If the reserved event pool is exhausted, __ibmvfc_tgt_get_implicit_logout_evt()
can return NULL:

ibmvfc-core.c:__ibmvfc_tgt_get_implicit_logout_evt() {
        ...
        evt = ibmvfc_get_reserved_event(&vhost->crq);
        if (!evt)
                return NULL;
        ...
}

Unlike other callers such as ibmvfc_tgt_implicit_logout(), this function
lacks a NULL check and passes the evt pointer directly to ibmvfc_send_event(),
which unconditionally dereferences it:

ibmvfc-core.c:ibmvfc_send_event() {
        __be64 *crq_as_u64 = (__be64 *) &evt->crq;
        ...
}

Could this lead to a NULL pointer dereference under heavy queue pressure
or concurrent target teardowns?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=12

Re: [PATCH 03/29] ibmvfc: split NVMe support into separate source file and add transport stubs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The driver misuses the NVMe-FC transport private data API by overwriting the `fcp_req->private` pointer instead of storing data within the allocated memory, leading to a kernel panic during AEN teardown.
--

commit ff3f397be994beea2b3c34de8ca81ef76dd5fe89
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: split NVMe support into separate source file and add transport stubs

Rename ibmvfc.c to ibmvfc-core.c as first step in decoupling each
protocol from the core driver logic. Add ibmvfc-nvme.[ch] files, and
register an nvme_fc_port_template with empty callback stubs.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> new file mode 100644
> index 0000000000000..4a66cde8a8d20
> --- /dev/null
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
[ ... ]
> +static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = {
> +	.localport_delete	= ibmvfc_nvme_localport_delete,
> +	.remoteport_delete	= ibmvfc_nvme_remoteport_delete,
> +	.create_queue		= NULL,
> +	.delete_queue		= NULL,
> +	.ls_req			= ibmvfc_nvme_ls_req,
> +	.ls_abort		= ibmvfc_nvme_ls_abort,
> +	.fcp_io			= ibmvfc_nvme_fcp_io,
> +	.fcp_abort		= ibmvfc_nvme_fcp_abort,
> +	.map_queues		= NULL,
> +	.max_hw_queues		= IBMVFC_NVME_HW_QUEUES,
> +	.max_sgl_segments	= 1024,
> +	.max_dif_sgl_segments	= 64,
> +	.dma_boundary		= 0xFFFFFFFF,
> +	.local_priv_sz		= sizeof(struct ibmvfc_host *),
> +	.remote_priv_sz		= sizeof(struct ibmvfc_target *),
> +	.lsrqst_priv_sz		= sizeof(struct ibmvfc_event *),
> +	.fcprqst_priv_sz	= sizeof(struct ibmvfc_event *),

[Severity: High]
Will this misconfigure the transport private data API?

The NVMe-FC core allocates an 8-byte private data area for AEN requests using
kzalloc() in nvme_fc_init_aen_ops() based on the fcprqst_priv_sz requested
here.

If the driver later overwrites the fcp_req->private pointer directly with a
statically allocated ibmvfc_event pointer (e.g., fcp_req->private = evt;)
instead of storing it inside the allocated memory space, this will lead to a
kernel panic.

During controller teardown (e.g., device removal or transport error recovery),
nvme_fc_term_aen_ops() calls kfree(aen_op->fcp_req.private). Because the
pointer would have been overwritten, kfree() is called on the non-kmalloc'd
event pointer, resulting in a SLUB bug and memory leak.

> +};

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=3

Re: [PATCH 06/29] ibmvfc: add logic for protocol specific fabric logins

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:

New issues:
- [High] Host initialization stalls indefinitely if `do_enquiry` is false during NPIV login completion.
- [High] NVMe fabric login is entirely skipped if SCSI fabric login is not required.
- [High] Concurrent target discovery MADs cause a data race and state machine failure, dropping targets.
- [High] Stale `do_scsi_login` and `do_nvme_login` flags persist across NPIV logins, causing unsupported MAD requests.

Pre-existing issues:
- [High] Missing bounds check on VIOS-provided queue counts causes arbitrary out-of-bounds heap write.
- [High] Integer overflow in `max_cmds` leads to SCSI command queue depth under-allocation and negative `can_queue` bounds.
--

commit d57e35c4c58c31909c3240ab400f9b9c8c0b9a8b
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: add logic for protocol specific fabric logins
    
This commit adds support for the protocol-specific fabric login flow introduced
by the updated client/VIOS interface. It tracks whether SCSI and NVMe/FC fabric
logins are needed and negotiates queue counts for both protocols. The driver
now issues appropriate fabric login MADs before target discovery begins.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index 6f5e8b3cbfc82..93c32fa162f83 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
[ ... ]
> @@ -1512,7 +1520,7 @@ static void ibmvfc_set_login_info(struct ibmvfc_host *vhost)
>  	max_cmds = scsi_qdepth + IBMVFC_NUM_INTERNAL_REQ;
>  	if (mq_enabled)
>  		max_cmds += (scsi_qdepth + IBMVFC_NUM_INTERNAL_SUBQ_REQ) *
> -			vhost->scsi_scrqs.desired_queues;
> +			(vhost->scsi_scrqs.desired_queues + vhost->nvme_scrqs.desired_queues);
>  
>  	memset(login_info, 0, sizeof(*login_info));

[Severity: High]
This is a pre-existing issue, but does max_cmds overflow here?

The max_cmds variable is a u16. When mq_enabled is true, adding the nvme
desired queues effectively doubles the multiplier. On systems with high core
counts, this calculation could silently overflow the 65,535 limit of the u16.

If the wrapped value is smaller than IBMVFC_NUM_INTERNAL_REQ, can_queue
becomes negative, which could induce failures during SCSI command allocations.

[ ... ]
> @@ -5058,22 +5201,28 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt)
[ ... ]
> -		active_queues = be32_to_cpu(setup->num_scsi_subq_channels);
> -		scrqs->active_queues = active_queues;
> +		scsi->active_queues = be32_to_cpu(setup->num_scsi_subq_channels);
> +		nvme->active_queues = be32_to_cpu(setup->num_nvme_subq_channels);
>  
>  		if (flags & IBMVFC_CHANNELS_CANCELED) {
>  			ibmvfc_dbg(vhost, "Channels Canceled\n");
>  			vhost->using_channels = 0;
> -		} else {
> -			if (active_queues)
> -				vhost->using_channels = 1;
> -			for (i = 0; i < active_queues; i++)
> -				scrqs->scrqs[i].vios_cookie =
> -					be64_to_cpu(setup->channel_handles[i]);
> -
> -			ibmvfc_dbg(vhost, "Using %u channels\n",
> -				   vhost->scsi_scrqs.active_queues);
> +			break;
>  		}
> +
> +		if (scsi->active_queues || nvme->active_queues)
> +			vhost->using_channels = 1;
> +		for (i = 0; i < scsi->active_queues; i++)
> +			scsi->scrqs[i].vios_cookie =
> +				be64_to_cpu(setup->channel_handles[i]);
> +		for (i = 0; i < nvme->active_queues; i++)
> +			nvme->scrqs[i].vios_cookie =
> +				be64_to_cpu(setup->channel_handles[scsi->active_queues + i]);

[Severity: High]
This isn't a bug introduced by this patch, but do these loops need bounds
checking on the VIOS-provided queue counts?

The driver assigns the untrusted num_scsi_subq_channels and
num_nvme_subq_channels values directly to active_queues. The code then uses
these values as boundary limits in the for loops to write into the scrqs
arrays.

Could a buggy or compromised VIOS returning a massive queue count cause an
out-of-bounds array write into the kernel heap?

[ ... ]
> @@ -5088,8 +5237,12 @@ static void ibmvfc_channel_setup_done(struct ibmvfc_event *evt)
>  		return;
>  	}
>  
> -	ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY);
> -	wake_up(&vhost->work_wait_q);
> +	if (vhost->do_scsi_login) {
> +		ibmvfc_fabric_login_scsi(vhost);
> +	} else {
> +		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY);
> +		wake_up(&vhost->work_wait_q);
> +	}
>  }

[Severity: High]
Does this bypass NVMe fabric login entirely if SCSI fabric login is not
required?

If a VIOS advertises IBMVFC_SUPPORT_NVMEOF but not IBMVFC_SUPPORT_SCSI,
vhost->do_scsi_login will be false while vhost->do_nvme_login is true.
Because vhost->do_scsi_login is false, execution falls into the else block
and advances the state to QUERY.

Since NVMe fabric login is only chained sequentially from the SCSI login
completion handler, does this prevent NVMe target discovery entirely on
NVMe-only setups?

[ ... ]
> @@ -5280,8 +5442,14 @@ static void ibmvfc_npiv_login_done(struct ibmvfc_event *evt)
>  	vhost->host->can_queue = be32_to_cpu(rsp->max_cmds) - IBMVFC_NUM_INTERNAL_REQ;
>  	vhost->host->max_sectors = npiv_max_sectors;
>  
> -	if (ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPORT_CHANNELS) && vhost->do_enquiry) {
> -		ibmvfc_channel_enquiry(vhost);
> +
> +	if (ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPORT_CHANNELS)) {
> +		if (ibmvfc_check_caps(vhost, IBMVFC_SUPPORT_SCSI))
> +			vhost->do_scsi_login = 1;
> +		if (ibmvfc_check_caps(vhost, IBMVFC_SUPPORT_NVMEOF))
> +			vhost->do_nvme_login = 1;

[Severity: High]
Can stale do_scsi_login and do_nvme_login flags persist across NPIV logins
here?

The flags are conditionally set based on NPIV login capabilities, but they are
never cleared before checking. If a VIOS drops these capabilities in a
subsequent NPIV login (for example, following a partition migration to a VIOS
with different support), the flags might erroneously remain set to 1.

Could this cause the driver to issue unsupported IBMVFC_FABRIC_LOGIN MADs to
the new VIOS?

> +		if (vhost->do_enquiry)
> +			ibmvfc_channel_enquiry(vhost);
>  	} else {
>  		vhost->do_enquiry = 0;
>  		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_QUERY);

[Severity: High]
If vhost->do_enquiry is false, does the host initialization stall indefinitely?

The structural change placed the do_enquiry check inside the capabilities
check block. If IBMVFC_CAN_SUPPORT_CHANNELS is true but vhost->do_enquiry is
false (e.g. following a link flap), the driver sets the login flags and
completes execution of the if block.

Because it bypasses the else block, it skips setting IBMVFC_HOST_ACTION_QUERY
and waking the work queue. Does this leave the host state machine permanently
stalled in IBMVFC_HOST_ACTION_INIT_WAIT with no outstanding events to advance
it?

[Severity: High]
Since both protocols are now supported, do concurrent target discovery MADs
cause a data race?

Looking at ibmvfc_discover_targets(), it appears to issue both SCSI and NVMe
target discovery MADs to the VIOS concurrently:

ibmvfc_discover_targets()
    evt = ibmvfc_get_disc_event(&vhost->scsi_scrqs);
    ibmvfc_send_event(evt, vhost, default_timeout);
    
    evt = ibmvfc_get_disc_event(&vhost->nvme_scrqs);
    ibmvfc_send_event(evt, vhost, default_timeout);

When the first MAD completes, ibmvfc_discover_targets_done() immediately
advances the host action to ALLOC_TGTS and wakes the allocation worker
thread.

Does the worker thread iterate over both discovery buffers while the second
MAD is still in flight? Additionally, when the slower MAD completes, its
request to set the host action is ignored because the state is no longer
INIT_WAIT, potentially dropping the targets from the slower discovery
operation.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=6

Re: [PATCH 19/29] ibmvfc: implement NVMe/FC stubs for local/remote port registration

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Unconditional NULL pointer dereference in NVMe-FC local and remote port unregistration paths.
--

commit 70b1c2d3d0ca257b156db2ba118e3a6f944e519d
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: implement NVMe/FC stubs for local/remote port registration

Implement the initial NVMe/FC local-port and remote-port registration
functions that notify the NVMe-FC midlayr of port discovery and loss.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> index 4a66cde8a8d20..202e8d0b00813 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> @@ -14,10 +14,18 @@
>  
>  static void ibmvfc_nvme_localport_delete(struct nvme_fc_local_port *lport)
>  {
> +	struct ibmvfc_host *vhost = lport->private;
> +
> +	vhost->nvme_local_port = NULL;
> +	complete(&vhost->nvme_delete_done);
>  }
>  
>  static void ibmvfc_nvme_remoteport_delete(struct nvme_fc_remote_port *rport)
>  {
> +	struct ibmvfc_target *tgt = rport->private;
> +
> +	tgt->nvme_remote_port = NULL;
> +	complete(&tgt->nvme_delete_done);
>  }

[ ... ]

> @@ -70,18 +78,85 @@ static struct nvme_fc_port_template ibmvfc_nvme_fc_transport = {
>  
>  int ibmvfc_nvme_register_remoteport(struct ibmvfc_target *tgt)
>  {

[ ... ]

>  void ibmvfc_nvme_unregister_remoteport(struct ibmvfc_target *tgt)
>  {
> +	struct ibmvfc_host *vhost = tgt->vhost;
> +	struct nvme_fc_remote_port *rport = tgt->nvme_remote_port;
> +	int rc;
> +
> +	if (!tgt->nvme_remote_port)
> +		return;
> +
> +	ibmvfc_log(vhost, 2, "unregister_remoteport: traddr=nn-0x%llx:pn-0x%llx PortID:%x\n",
> +		   rport->node_name, rport->port_name, rport->port_id);
> +	init_completion(&tgt->nvme_delete_done);
> +	rc = nvme_fc_unregister_remoteport(tgt->nvme_remote_port);
> +
> +	if (!rc) {
> +		wait_for_completion(&tgt->nvme_delete_done);
> +		tgt->nvme_remote_port->private = NULL;

[Severity: High]
Does this code unconditionally dereference a NULL pointer?

When nvme_fc_unregister_remoteport() finishes, the asynchronous delete
callback ibmvfc_nvme_remoteport_delete() is called. This callback executes:

    tgt->nvme_remote_port = NULL;
    complete(&tgt->nvme_delete_done);

After waking up from wait_for_completion(), tgt->nvme_remote_port will
always be NULL, meaning the access to tgt->nvme_remote_port->private will
cause a kernel panic.

> +	}
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=19

Re: [PATCH 04/29] ibmvfc: initialize NVMe channel configuration during driver probe

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] The user-configured module parameter for the number of NVMe hardware submission queues (`nvme_host_queues`) is ignored during NVMe queue initialization, preventing users from capping the maximum number of NVMe host queues.
- [Medium] Bitfield truncation of unsigned integer module parameters can silently disable NVMe or create an inconsistent state.
--

commit 8b0247aa812866f78c7701f707162de73d472772
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: initialize NVMe channel configuration during driver probe

Initialize the host's NVMe channel-group state during probe. This prepares the
driver with a NVMe/FC channel group that can will be used by later patches for
NVMe queue allocation, discovery buffers, target management, and IO
submission.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> @@ -6337,7 +6337,8 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id)
>  	struct device *dev = &vdev->dev;
>  	int rc = -ENOMEM;
>  	unsigned int online_cpus = num_online_cpus();
> -	unsigned int max_scsi_queues = min((unsigned int)IBMVFC_MAX_SCSI_QUEUES, online_cpus);
> +	unsigned int max_scsi_queues = min_t(unsigned int, IBMVFC_MAX_SCSI_QUEUES, online_cpus);
> +	unsigned int max_nvme_queues = min_t(unsigned int, IBMVFC_MAX_NVME_QUEUES, online_cpus);
>  
>  	ENTER;
>  	shost = scsi_host_alloc(&driver_template, sizeof(*vhost));

[ ... ]

> @@ -6371,6 +6373,10 @@ static int ibmvfc_probe(struct vio_dev *vdev, const struct vio_device_id *id)
>  	vhost->scsi_scrqs.protocol = IBMVFC_PROTO_SCSI;
>  	vhost->using_channels = 0;
>  	vhost->do_enquiry = 1;
> +	vhost->nvme_enabled = mq_enabled ? nvme_enabled : 0;

[Severity: Medium]
Could this result in unintended bitfield truncation if an even value is passed
via the module parameter?

Since vhost->nvme_enabled is a 1-bit bitfield, assigning an even value
directly will truncate it to 0, unintentionally disabling NVMe.

Additionally, since the ternary operator checks the un-normalized mq_enabled
integer directly, couldn't this create an inconsistent state where the ternary
evaluates to true, but the underlying vhost->mq_enabled field was truncated
to 0?

> +	vhost->nvme_scrqs.desired_queues = min(max_nvme_queues, nr_nvme_channels);
> +	vhost->nvme_scrqs.max_queues = max_nvme_queues;

[Severity: Medium]
Should the maximum queue calculations be constrained by the nr_nvme_hw_queues
module parameter?

The NVMe queue initialization currently ignores the user-configured
nr_nvme_hw_queues limit, unconditionally defaulting to the max_nvme_queues
limit based on IBMVFC_MAX_NVME_QUEUES and online CPUs.

> +	vhost->nvme_scrqs.protocol = IBMVFC_PROTO_NVME;
>  	vhost->scan_timeout = 0;
>  
>  	strcpy(vhost->partition_name, "UNKNOWN");

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=4

Re: [PATCH 22/29] ibmvfc: extend ibmvfc_debug visibility to ibmvfc-nvme.h

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Core driver debug variable declared in protocol-specific header
--

commit 719e4b33f0990bbcd15f1244a436cf69d36807ef
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: extend ibmvfc_debug visibility to ibmvfc-nvme.h

Export ibmvfc_debug so the NVMe support code can use the
existing ibmvfc_dbg logging macro.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
> index 0465e87198817..3aa2857887950 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.h
> @@ -22,6 +22,8 @@
>  #define IBMVFC_MAX_NVME_QUEUES	16
>  #define IBMVFC_NVME_CHANNELS	8
>  
> +extern unsigned int ibmvfc_debug;
> +

[Severity: Low]
This isn't a bug, but should this core driver debug variable be declared in
the core ibmvfc.h header rather than this protocol-specific header?

Declaring it here creates a confusing dependency where the core header relies
on a protocol header for its macro dependencies.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=22

Re: [PATCH 17/29] ibmvfc: delete NVMe/FC targets as well as SCSI

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] Fabric ELS events (LOGO, PRLO, PLOGI) fail to process NVMe targets, violating the commit's claim regarding consistent fabric event handling.

Pre-existing issues:
- [High] An integer overflow in the calculation of `sense_len + rsp_len` bypasses bounds checking, leading to an out-of-bounds read and write in `memcpy()`.
--

commit 56df0a05e56764e1456bdc6176dcf26a3a1bc6b0
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: delete NVMe/FC targets as well as SCSI

This commit extends target deletion paths to process NVMe targets alongside
SCSI targets. It updates link-down and host reinitialization flows to walk
both lists to maintain consistent target state. It also renames
ibmvfc_relogin to ibmvfc_scsi_relogin.

>     Update link-down, and host reinitialization flows to walk both the SCSI
>     and NVMe target lists when marking targets for deletion. This ensures
>     that protocol-specific target state stays consistent across adapter
>     resets and fabric events.

[Severity: High]
Does the current patch fully handle fabric events for NVMe targets?

The commit message mentions maintaining consistent target state across fabric
events, but it appears ibmvfc_handle_async in
drivers/scsi/ibmvscsi/ibmvfc-core.c was not updated to process the
nvme_scrqs.targets list.

When an NVMe target logs out of the fabric and the adapter sends an async
IBMVFC_AE_ELS_LOGO event, the ELS event handler loop only searches
scsi_scrqs.targets. Can this cause stale NVMe targets to remain in memory
indefinitely and break the target state machine?

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index 4b95e43449473..13e513bfd0a86 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c

[ ... ]

> @@ -1920,7 +1928,7 @@ static void ibmvfc_scsi_done(struct ibmvfc_event *evt)
>  				memcpy(cmnd->sense_buffer, rsp->data.sense + rsp_len, sense_len);

[Severity: High]
This is a pre-existing issue, but can an integer overflow occur here when
calculating sense_len + rsp_len in ibmvfc_scsi_done?

If a buggy or compromised virtual I/O server returns a large fcp_sense_len
like 0xFFFFFFFE and a small fcp_rsp_len like 4, their sum could wrap around
to 2.

This wrapped value would bypass the greater than SCSI_SENSE_BUFFERSIZE
bounds check. Since rsp_len <= 8 would still be true, could the memcpy call
attempt to copy nearly 4GB of data and cause memory corruption?

>  			if ((be16_to_cpu(vfc_cmd->status) & IBMVFC_VIOS_FAILURE) &&
>  			    (be16_to_cpu(vfc_cmd->error) == IBMVFC_PLOGI_REQUIRED))
> -				ibmvfc_relogin(cmnd->device);
> +				ibmvfc_scsi_relogin(cmnd->device);
>  
>  			if (!cmnd->result && (!scsi_get_resid(cmnd) || (rsp->flags & FCP_RESID_OVER)))
>  				cmnd->result = (DID_ERROR << 16);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=17

Re: [PATCH 14/29] ibmvfc: add NVMe/FC Process Login support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Self-deadlock on `host_lock` if a management CRQ event fails to send.
--

commit ee31c1669f64561bd4a4e52947529fea4cf6545e
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: add NVMe/FC Process Login support

This commit extends the PRLI handling code to support NVMe/FC targets.
It adds NVMe-specific service parameters and sets the remote-port roles
upon successful process login. The existing SCSI PRLI flow is preserved.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index 2c54d0b9add49..b45cd0183fb51 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
[ ... ]
> @@ -4172,25 +4190,30 @@ static void ibmvfc_tgt_send_prli(struct ibmvfc_target *tgt)
[ ... ]
>  	ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_INIT_WAIT);
>  	if (ibmvfc_send_event(evt, vhost, default_timeout)) {

[Severity: High]
This is a pre-existing issue, but can this call path trigger a deadlock?

When ibmvfc_tgt_send_prli() is called by the target's job step from
ibmvfc_do_work(), the host_lock is already held.

If ibmvfc_send_event() fails (e.g. due to a hypercall failure), it
synchronously invokes the completion callback evt->done(evt).

Because the event was initialized with IBMVFC_MAD_FORMAT via
ibmvfc_init_event(), evt->done is set to ibmvfc_locked_done(), which
attempts to acquire the host_lock again:

    spin_lock_irqsave(evt->vhost->host->host_lock, flags);

Would this cause a hard deadlock on the CPU if the underlying hardware
encounters an error and fails to send the process login?

>  		vhost->discovery_threads--;
>  		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
>  		kref_put(&tgt->kref, ibmvfc_release_tgt);
>  	} else
> -		tgt_dbg(tgt, "Sent process login\n");
> +		tgt_dbg(tgt, "%s Sent process login\n", proto_type[tgt->protocol]);
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=14

Re: [PATCH 11/29] ibmvfc: send NVMe target discovery MAD

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Parallel SCSI and NVMe discover targets MADs race to advance the host initialization state machine, causing the slower protocol's targets to be permanently ignored and breaking retry handling.
- [Low] The newly added `proto_type` array is missing the `static` keyword, making it a globally visible symbol.
--

commit e89ce7e682793b58e5019c746ffde18602fe635b
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: send NVMe target discovery MAD

This commit extends target discovery to send protocol-specific
discover-target MADs for NVMe/FC. It introduces a parallel discovery flow
for NVMe alongside SCSI targets.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index efac82c48258a..53480d150042e 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> @@ -173,6 +173,11 @@ static const struct {
>  	{ IBMVFC_FC_SCSI_ERROR, IBMVFC_COMMAND_FAILED, DID_ERROR, 0, 1, "PRLI to device failed." },
>  };
>  
> +const char *proto_type[] = {
> +	"SCSI",
> +	"NVMe",
> +};

[Severity: Low]
This isn't a bug, but can this array be declared as static? It is currently
exposed to the global kernel namespace which could lead to linker
collisions.

> +
>  static void ibmvfc_npiv_login(struct ibmvfc_host *);
>  static void ibmvfc_tgt_send_prli(struct ibmvfc_target *);
>  static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *);

[ ... ]

> @@ -5001,19 +5006,30 @@ static void ibmvfc_discover_targets_done(struct ibmvfc_event *evt)
>  {
>  	struct ibmvfc_host *vhost = evt->vhost;
>  	struct ibmvfc_discover_targets *rsp = &evt->xfer_iu->discover_targets;
> +	struct ibmvfc_channels *channels;
>  	u32 mad_status = be16_to_cpu(rsp->common.status);
> +	u32 opcode = be32_to_cpu(rsp->common.opcode);
>  	int level = IBMVFC_DEFAULT_LOG_LEVEL;
>  
> +	if (opcode == IBMVFC_DISC_TARGETS)
> +		channels = &vhost->scsi_scrqs;
> +	else
> +		channels = &vhost->nvme_scrqs;
> +
>  	switch (mad_status) {
>  	case IBMVFC_MAD_SUCCESS:
> -		ibmvfc_dbg(vhost, "Discover Targets succeeded\n");
> -		vhost->scsi_scrqs.num_targets = min_t(u32, be32_to_cpu(rsp->num_written),
> -						      max_targets);
> +		ibmvfc_dbg(vhost, "Discover %s Targets succeeded\n",
> +			   proto_type[channels->protocol]);
> +		channels->num_targets = min_t(u32, be32_to_cpu(rsp->num_written),
> +					      max_targets);
> +		ibmvfc_dbg(vhost, "%d %s targets found\n", channels->num_targets,
> +			   proto_type[channels->protocol]);
>  		ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_ALLOC_TGTS);

[Severity: High]
Does this code create a race condition between the SCSI and NVMe discovery
MADs?

If both SCSI and NVMe discoveries are running asynchronously, whichever MAD
completes first will call ibmvfc_set_host_action() and advance the host
state from IBMVFC_HOST_ACTION_INIT_WAIT to IBMVFC_HOST_ACTION_ALLOC_TGTS.

When the slower MAD subsequently completes, ibmvfc_set_host_action()
will ignore the transition to IBMVFC_HOST_ACTION_ALLOC_TGTS because the
state is no longer IBMVFC_HOST_ACTION_INIT_WAIT. This causes the targets
from the slower protocol to be permanently ignored.

Additionally, if the first MAD fails, could the state machine revert to
IBMVFC_HOST_ACTION_INIT and issue a new batch of discovery commands
while the slower MAD is still in-flight?

>  		break;
>  	case IBMVFC_MAD_FAILED:
>  		level += ibmvfc_retry_host_init(vhost);

[ ... ]

> @@ -5074,9 +5090,29 @@ static void ibmvfc_discover_targets(struct ibmvfc_host *vhost)
>  	ibmvfc_set_host_action(vhost, IBMVFC_HOST_ACTION_INIT_WAIT);
>  
>  	if (!ibmvfc_send_event(evt, vhost, default_timeout))
> -		ibmvfc_dbg(vhost, "Sent discover targets\n");
> +		ibmvfc_dbg(vhost, "Sent discover SCSI targets\n");
>  	else
> -		ibmvfc_link_down(vhost, IBMVFC_LINK_DEAD);
> +		goto link_down;
> +
> +	if (!ibmvfc_nvme_active(vhost))
> +		return;
> +
> +	evt = ibmvfc_get_disc_event(&vhost->nvme_scrqs);
> +	if (!evt) {
> +		ibmvfc_log(vhost, level, "Discover NVMe Targets failed: no available events\n");
> +		ibmvfc_hard_reset_host(vhost);
> +		return;
> +	}
> +
> +	if (!ibmvfc_send_event(evt, vhost, default_timeout))
> +		ibmvfc_dbg(vhost, "Sent discover NVMe targets\n");
> +	else
> +		goto link_down;
> +
> +	return;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=11

Re: [PATCH 15/29] ibmvfc: add NVMe/FC Query Target support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [Low] The global array `proto_type` is missing the `static` keyword, causing namespace pollution.

Pre-existing issues:
- [High] Memory allocated from a mempool is freed directly via kfree(), permanently depleting the mempool reserve under memory pressure.
--

commit edf90dca8246218c80f44b69406be60422857d3c
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: add NVMe/FC Query Target support

Add protocol-specific query-target support for NVMe/FC targets. It uses the
NVMe query-target specific MAD when querying an NVMe target and updates the
associated debug and error logging to include the target protocol.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index b45cd0183fb51..363bf75d62443 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> @@ -4849,7 +4849,7 @@ static void ibmvfc_tgt_query_target_done(struct ibmvfc_event *evt)
>  	ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
>  	switch (status) {
>  	case IBMVFC_MAD_SUCCESS:
> -		tgt_dbg(tgt, "Query Target succeeded\n");
> +		tgt_dbg(tgt, "%s Query Target succeeded\n", proto_type[tgt->protocol]);

[Severity: Low]
Does proto_type need the static keyword? Since it is a generic identifier
defined at file scope without being in a header:

drivers/scsi/ibmvscsi/ibmvfc-core.c:
const char *proto_type[] = {
	"SCSI",
	"NVMe",
};

Could this cause symbol collisions in the global kernel namespace during
linking?

>  		if (be64_to_cpu(rsp->scsi_id) != tgt->scsi_id)
>  			ibmvfc_del_tgt(tgt);
>  		else

[ ... ]

> @@ -4913,7 +4913,10 @@ static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt)
>  	query_tgt = &evt->iu.query_tgt;
>  	memset(query_tgt, 0, sizeof(*query_tgt));
>  	query_tgt->common.version = cpu_to_be32(1);
> -	query_tgt->common.opcode = cpu_to_be32(IBMVFC_QUERY_TARGET);
> +	if (tgt->protocol == IBMVFC_PROTO_SCSI)
> +		query_tgt->common.opcode = cpu_to_be32(IBMVFC_QUERY_TARGET);
> +	else
> +		query_tgt->common.opcode = cpu_to_be32(IBMVFC_NVMF_QUERY_TARGET);
>  	query_tgt->common.length = cpu_to_be16(sizeof(*query_tgt));
>  	query_tgt->wwpn = cpu_to_be64(tgt->ids.port_name);
>  
> @@ -4923,7 +4926,7 @@ static void ibmvfc_tgt_query_target(struct ibmvfc_target *tgt)
>  		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
>  		kref_put(&tgt->kref, ibmvfc_release_tgt);

[Severity: High]
This is a pre-existing issue, but does ibmvfc_release_tgt() incorrectly free
the target?

The target is originally allocated using mempool_alloc():

drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_alloc_target() {
    ...
	tgt = mempool_alloc(vhost->tgt_pool, GFP_NOIO);
    ...
}

However, it is freed directly via kfree() instead of mempool_free():

drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_release_tgt() {
	struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);
	kfree(tgt);
}

Under memory pressure, mempool_alloc() falls back to its preallocated
reserve. Will bypassing mempool_free() permanently leak the reserved
elements, eventually causing deadlocks when the pool is depleted and
mempool_alloc() blocks waiting for elements to be returned?

>  	} else
> -		tgt_dbg(tgt, "Sent Query Target\n");
> +		tgt_dbg(tgt, "Sent %s Query Target\n", proto_type[tgt->protocol]);
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=15

Re: [PATCH 13/29] ibmvfc: add NVMe/FC Port Login support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Targets allocated from a mempool are incorrectly freed using `kfree()` directly, permanently depleting the mempool's reserved elements.
--

commit 6a208595368e371e1b9ab8d0fccb04902f87daf8
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: add NVMe/FC Port Login support
    
Expand the target login path to issue the NVMe/FC-specific port login
MAD for NVMe targets. Select the correct PLOGI MAD opcode based on the
target protocol and include the protocol name in success and failure
logging.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index 3e3d77e0d5172..2c54d0b9add49 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c

[ ... ]

> @@ -4286,7 +4286,10 @@ static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *tgt)
>  	} else {
>  		plogi->common.version = cpu_to_be32(1);
>  	}
> -	plogi->common.opcode = cpu_to_be32(IBMVFC_PORT_LOGIN);
> +	if (tgt->protocol == IBMVFC_PROTO_SCSI)
> +		plogi->common.opcode = cpu_to_be32(IBMVFC_PORT_LOGIN);
> +	else
> +		plogi->common.opcode = cpu_to_be32(IBMVFC_NVMF_PORT_LOGIN);
>  	plogi->common.length = cpu_to_be16(sizeof(*plogi));
>  	plogi->scsi_id = cpu_to_be64(tgt->scsi_id);
>  
> @@ -4295,7 +4298,7 @@ static void ibmvfc_tgt_send_plogi(struct ibmvfc_target *tgt)
>  		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_NONE);
>  		kref_put(&tgt->kref, ibmvfc_release_tgt);

[Severity: High]
This isn't a bug introduced by this patch, but does the target release
function deplete the mempool's reserved elements during memory pressure?

The target structure is allocated from the driver's mempool:

drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_alloc_target() {
    ...
    tgt = mempool_alloc(vhost->tgt_pool, GFP_NOIO);
    ...
}

However, it appears ibmvfc_release_tgt() uses kfree() instead of
mempool_free():

drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_release_tgt() {
    struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);
    kfree(tgt);
}

When the system is under memory pressure, mempool_alloc() falls back to
using its pre-allocated reserve elements.

If these elements are returned to the general slab allocator via kfree()
instead of mempool_free(), wouldn't this permanently deplete the mempool's
reserve and cause subsequent allocations under memory pressure to hang
indefinitely?

>  	} else
> -		tgt_dbg(tgt, "Sent port login\n");
> +		tgt_dbg(tgt, "Sent %s port login\n", proto_type[tgt->protocol]);
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=13

Re: [PATCH 02/29] ibmvfc: add NVMe/FC protocol interface definitions

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Unconditional size expansion of `struct ibmvfc_implicit_logout` breaks backwards compatibility with older VIOS versions.
--

commit c4f31afebbe28cbb64ba03689b7e7ed327cb6d9e
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: add NVMe/FC protocol interface definitions

This commit adds the protocol definitions for client-VIOS interface updates
needed to support NVMe/FC over the ibmvfc NPIV transport. It extends the
ibmvfc interface with NVMe/FC-specific capabilities, queue definitions,
and command layout support.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc.h b/drivers/scsi/ibmvscsi/ibmvfc.h
> index 0e259e9d2e9bb..f8a2bf92da410 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc.h
> +++ b/drivers/scsi/ibmvscsi/ibmvfc.h

[ ... ]

> @@ -377,20 +419,27 @@ struct ibmvfc_query_tgt {
>  struct ibmvfc_implicit_logout {
>  	struct ibmvfc_mad_common common;
>  	__be64 old_scsi_id;
> -	__be64 reserved[2];
> +	__be64 reserved[8];
> +	__be64 target_wwpn;
>  } __packed __aligned(8);

[Severity: High]
Will this unconditional size expansion break backwards compatibility with older
Virtual I/O Server (VIOS) firmware?

Since the driver calculates the MAD length dynamically using sizeof():

drivers/scsi/ibmvscsi/ibmvfc.c:__ibmvfc_tgt_get_implicit_logout_evt() {
    ...
    mad->common.length = cpu_to_be16(sizeof(*mad));
    ...
}

This changes the length sent for version 1 IBMVFC_IMPLICIT_LOGOUT operations
from 40 bytes to 96 bytes. Could older VIOS versions that strictly validate
MAD lengths against the version 1 specification reject this payload, breaking
target teardown and error recovery?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=2

Re: [PATCH 05/29] ibmvfc: alloc/dealloc sub-queues for nvme channels

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Failure to register NVMe sub-queues silently disables SCSI multiqueue support due to a shared state flag.
--

commit 6eb0f2ef7c51067f6c97ea8c13211afa2e19dcc4
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: alloc/dealloc sub-queues for nvme channels

This patch allocates, registers, deregisters, and releases NVMe subordinate
CRQs alongside the existing SCSI sub-CRQs. It integrates the NVMe queue
lifecycle with the existing SCSI queue lifecycle to ensure both protocols
are reset consistently.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index 5732ccf2ac1c0..6f5e8b3cbfc82 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c

[ ... ]

> @@ -6109,6 +6113,13 @@ static void ibmvfc_init_sub_crqs(struct ibmvfc_host *vhost)
>  
>  	ibmvfc_reg_sub_crqs(vhost, &vhost->scsi_scrqs);
>  
> +	if (vhost->nvme_enabled) {
> +		if (ibmvfc_alloc_channels(vhost, &vhost->nvme_scrqs))
> +			vhost->nvme_enabled = 0;
> +		else
> +			ibmvfc_reg_sub_crqs(vhost, &vhost->nvme_scrqs);

[Severity: High]
Does a failure to register NVMe sub-queues unintentionally disable SCSI
multiqueue support here?

If ibmvfc_reg_sub_crqs() fails while registering the NVMe channels, it handles
the error by unregistering the partial channels and setting a shared host flag:

ibmvfc_reg_sub_crqs() {
    ...
    if (ibmvfc_register_channel(vhost, channels, i)) {
        for (j = i; j > 0; j--)
            ...
        vhost->do_enquiry = 0;
        return;
    }
}

Since ibmvfc_reg_sub_crqs() is now called for NVMe right after successfully
registering the SCSI sub-queues, setting vhost->do_enquiry to 0 upon an NVMe
registration failure will cause the driver to skip channel enquiry during the
NPIV login sequence. 

Will this cause the successfully registered SCSI channels to be silently
ignored if the hypervisor rejects the NVMe sub-crqs allocation?

> +	}
> +
>  	LEAVE;
>  }
>

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=5

Re: [PATCH 18/29] ibmvfc: update state machine to process NVMe/FC targets

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] NVMe targets leak memory and become permanently stranded because their cleanup logic is omitted from the target deletion phase in ibmvfc_do_work().
- [High] Asynchronous ELS events (LOGO, PRLO, PLOGI) from NVMe targets are ignored because the event handler only loops over the SCSI target list.
--

commit 2221ef8b59f01b903ca3e2cc56b9bd27be8a9dc1
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: update state machine to process NVMe/FC targets

Update the host work loop and target state-machine helpers to process
NVMe targets in addition to SCSI targets.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index 13e513bfd0a86..9a6a885aa57ed 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
[ ... ]
> @@ -5962,6 +5992,12 @@ static void ibmvfc_do_work(struct ibmvfc_host *vhost)
>  				break;
>  			}
>  		}
> +		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue) {
> +			if (tgt->action == IBMVFC_TGT_ACTION_LOGOUT_RPORT) {
> +				tgt->job_step(tgt);
> +				break;
> +			}
> +		}
>  
>  		if (ibmvfc_dev_logo_to_do(vhost)) {
>  			spin_unlock_irqrestore(vhost->host->host_lock, flags);

[Severity: High]
Does this code also need to loop over nvme_scrqs.targets for final target
deletion?

Looking further down in ibmvfc_do_work(), there is a loop handling
IBMVFC_TGT_ACTION_DEL_RPORT and IBMVFC_TGT_ACTION_DEL_AND_LOGOUT_RPORT:

		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
			if (tgt->action == IBMVFC_TGT_ACTION_DEL_RPORT) {
				tgt_dbg(tgt, "Deleting rport\n");
				rport = tgt->rport;
				tgt->rport = NULL;
				list_del(&tgt->queue);
				ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_DELETED_RPORT);
				...

Since this loop only processes scsi_scrqs.targets, wouldn't NVMe targets in
the DEL_RPORT state be leaked and stranded?

[ ... ]

[Severity: High]
Are asynchronous ELS events from NVMe targets being ignored?

Looking at ibmvfc_handle_async(), which processes events like LOGO, PRLO,
and PLOGI, the loop only iterates over the SCSI targets:

	case IBMVFC_AE_ELS_LOGO:
	case IBMVFC_AE_ELS_PRLO:
	case IBMVFC_AE_ELS_PLOGI:
		list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {

Since the NVMe targets are now in a separate nvme_scrqs.targets list,
wouldn't an explicit logout or state change from an NVMe target be dropped,
leaving the driver with stale state?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=18

Re: [PATCH 16/29] ibmvfc: allocate targets based on protocol

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Target structures allocated from a mempool are incorrectly freed using kfree() instead of mempool_free(), permanently depleting the mempool reserve.
--

commit 2827414150eab249a5dd4c4a093ced5d74ae8737
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: allocate targets based on protocol

Allocate discovered targets onto the channel-group list that matches
their protocol.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index 363bf75d62443..4b95e43449473 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c

[ ... ]

> @@ -5004,6 +5011,7 @@ static int ibmvfc_alloc_target(struct ibmvfc_host *vhost,
>  
>  	tgt = mempool_alloc(vhost->tgt_pool, GFP_NOIO);
>  	memset(tgt, 0, sizeof(*tgt));
> +	tgt->protocol = protocol;

[Severity: High]
This is a pre-existing issue and was not introduced by the patch under
review, but since we are looking at this allocation path, is the target
structure freed correctly when released?

This function allocates tgt from a mempool, but looking at the reference
count release routine ibmvfc_release_tgt(), the object is freed directly to
the slab allocator:

drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_release_tgt() {
    struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);
    kfree(tgt);
}

Because it is freed via kfree() instead of mempool_free(), the mempool
internal reserve count is never replenished and waiting threads are not woken
up. Under memory pressure, subsequent mempool_alloc() calls with GFP_NOIO
could block indefinitely on the mempool waitqueue once the reserve is
depleted.

>  	tgt->scsi_id = scsi_id;
>  	tgt->wwpn = wwpn;
>  	tgt->vhost = vhost;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=16

Re: [PATCH 20/29] ibmvfc: register local nvme fc port after fabric login

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

New issues:
- [High] Guaranteed NULL pointer dereference in `ibmvfc_nvme_unregister()` upon successful local port unregistration.
- [High] Permanent memory, device reference, and state leak due to unconditional NVMe local port registration without teardown.
- [High] Sleeping in atomic context bug by calling `nvme_fc_register_localport()` from a CRQ response handler.

Pre-existing issues:
- [High] Guaranteed NULL pointer dereference in `ibmvfc_nvme_unregister_remoteport()`.
--

commit 84b5a798e09cfb51baf820040579b6001b3aae40
Author: Tyrel Datwyler <tyreld@linux.ibm.com>
Date:   Tue Jun 23 01:30:26 2026 +0000

ibmvfc: register local nvme fc port after fabric login

Register the local NVMe/FC port only after fabric login has completed.
The VIOS returns the client port ID in the fabric login response, and
that port ID is required to populate the local-port information passed
to the NVMe-FC midlayer.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index 9a6a885aa57ed..2c7ecf7bdde93 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> @@ -5184,6 +5184,8 @@ static void ibmvfc_fabric_login_nvme_done(struct ibmvfc_event *evt)
>  
>  	switch (mad_status) {
>  	case IBMVFC_MAD_SUCCESS:
> +		fc_host_port_id(vhost->host) = be64_to_cpu(rsp->nport_id);
> +		ibmvfc_nvme_register(vhost);

[Severity: High]
Does this code sleep in atomic context?

It looks like ibmvfc_fabric_login_nvme_done() is executed as a callback
from ibmvfc_locked_done(), which holds the host_lock spinlock with
interrupts disabled:

ibmvfc_locked_done() {
    spin_lock_irqsave(evt->vhost->host->host_lock, flags);
    evt->_done(evt);
    spin_unlock_irqrestore(evt->vhost->host->host_lock, flags);
}

The callback invokes ibmvfc_nvme_register(), which calls the NVMe FC midlayer
function nvme_fc_register_localport(). This midlayer function makes GFP_KERNEL
allocations, which could lead to deadlocks or panics when atomic sleep checks
are enabled.

Additionally, does this introduce a permanent resource leak?

This function unconditionally registers a new NVMe local port upon every
successful fabric login. These logins can happen repeatedly during probe,
adapter resets, or network link flaps. Since it never unregisters the
previously registered port before grabbing a new device reference, it
appears the memory, device references, and NVMe transport state will leak.

>  		ibmvfc_dbg(vhost, "NVMe fabric login succeeded\n");
>  		break;
>  	case IBMVFC_MAD_FAILED:

[ ... ]

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> index 202e8d0b00813..fc4337fc9b3fb 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c

[ ... ]

> @@ -154,9 +157,18 @@ void ibmvfc_nvme_unregister(struct ibmvfc_host *vhost)
>  	int rc;
>  
>  	if (vhost->nvme_local_port) {
> +		ibmvfc_log(vhost, 2, "unregister_localport: host-traddr=nn-0x%llx:pn-0x%llx on portID:%x\n",
> +			   vhost->nvme_local_port->node_name,
> +			   vhost->nvme_local_port->port_name,
> +			   vhost->nvme_local_port->port_id);
>  		init_completion(&vhost->nvme_delete_done);
>  		rc = nvme_fc_unregister_localport(vhost->nvme_local_port);
> -		if (!rc)
> +		if (!rc) {
>  			wait_for_completion(&vhost->nvme_delete_done);
> +			vhost->nvme_local_port->private = NULL;

[Severity: High]
Can this dereference a NULL pointer?

The completion for port deletion is signaled by the midlayer callback
ibmvfc_nvme_localport_delete(), which explicitly clears the local port
pointer:

ibmvfc_nvme_localport_delete() {
    vhost->nvme_local_port = NULL;
    complete(&vhost->nvme_delete_done);
}

When wait_for_completion() returns here, vhost->nvme_local_port is
guaranteed to be NULL, meaning the assignment to private will
unconditionally cause a crash.

> +		} else
> +			dev_err(vhost->dev, "Failed to unregister NVMe fc localport (%d)\n", rc);
> +
> +		put_device(vhost->dev);
>  	}
>  }

[ ... ]

> static void ibmvfc_nvme_unregister_remoteport(struct ibmvfc_target *tgt)
> {
> 	if (tgt->nvme_remote_port) {
> 		init_completion(&tgt->nvme_delete_done);
> 		nvme_fc_unregister_remoteport(tgt->nvme_remote_port);
> 		wait_for_completion(&tgt->nvme_delete_done);
> 		tgt->nvme_remote_port->private = NULL;

[Severity: High]
This is a pre-existing issue, but does this suffer from the same NULL
pointer dereference issue as the local port unregister function?

Similar to the local port, the completion is signaled by
ibmvfc_nvme_remoteport_delete(), which sets tgt->nvme_remote_port = NULL
before completing. This means tgt->nvme_remote_port will be NULL when the
wait finishes, leading to an unconditional crash here during target teardown.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=20

Re: [PATCH 21/29] ibmvfc: process NVMe/FC rports in work thread

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

New issues:
- [Critical] Deterministic NULL pointer dereference during NVMe remote port and local port unregistration.
- [High] NVMe targets marked for deletion are completely ignored by the main state machine, resulting in permanent resource leaks and stuck target states.
- [High] `ibmvfc_nvme_unregister` is dead code, permanently leaking the NVMe local port on module unload.

Pre-existing issues:
- [Critical] Concurrent target deletion during port addition completely leaks the newly registered remote port, leading to a Use-After-Free of the private data pointer for NVMe targets.
- [High] Target structures allocated from a mempool are incorrectly freed using `kfree()`, draining the mempool reserves and leading to hangs under memory pressure.
--

commit f73360d74019cb783c285fb8eef798a237334c34
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: process NVMe/FC rports in work thread

This commit adds an NVMe-specific remote-port add helper and updates
the rport worker thread to walk the NVMe target list. It handles
registration, rescan, and deletion transitions for NVMe remote
ports in the same worker context used for SCSI rports.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c
[ ... ]
> +static void ibmvfc_tgt_add_nvme_rport(struct ibmvfc_target *tgt)
> +{
> +	struct ibmvfc_host *vhost = tgt->vhost;
[ ... ]
> +	if (rport && tgt->action == IBMVFC_TGT_ACTION_DEL_RPORT) {
> +		tgt_dbg(tgt, "Deleting NVMe rport\n");
> +		list_del(&tgt->queue);
> +		ibmvfc_set_tgt_action(tgt, IBMVFC_TGT_ACTION_DELETED_RPORT);
> +		spin_unlock_irqrestore(vhost->host->host_lock, flags);
> +		ibmvfc_nvme_unregister_remoteport(tgt);

[Severity: Critical]
Does ibmvfc_nvme_unregister_remoteport() contain a null pointer dereference?

When unregistering the remote port, the completion callback appears to
explicitly set the tgt->nvme_remote_port pointer to NULL:

drivers/scsi/ibmvscsi/ibmvfc-nvme.c:ibmvfc_nvme_unregister_remoteport() {
    ...
    rc = nvme_fc_unregister_remoteport(tgt->nvme_remote_port);
    if (!rc) {
        wait_for_completion(&tgt->nvme_delete_done);
        tgt->nvme_remote_port->private = NULL;
    }
}

Will this attempt to dereference tgt->nvme_remote_port right after it was
set to NULL by ibmvfc_nvme_remoteport_delete before the completion signaled?
The same pattern seems to exist in ibmvfc_nvme_unregister() for the local
port.

> +		timer_delete_sync(&tgt->timer);
> +		kref_put(&tgt->kref, ibmvfc_release_tgt);

[Severity: High]
This is a pre-existing issue, but does ibmvfc_release_tgt() incorrectly free
mempool-allocated targets?

The targets are allocated via mempool_alloc(vhost->tgt_pool, GFP_NOIO) in
ibmvfc_alloc_targets(), but the kref release function calls kfree(tgt):

drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_release_tgt() {
    struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);
    kfree(tgt);
}

Does this bypass the mempool reserves and eventually cause mempool_alloc()
to hang under memory pressure?

> +		return;
> +	} else if (rport && tgt->action == IBMVFC_TGT_ACTION_DEL_AND_LOGOUT_RPORT) {
[ ... ]
> +	} else if (rport && tgt->action == IBMVFC_TGT_ACTION_DELETED_RPORT) {
> +		spin_unlock_irqrestore(vhost->host->host_lock, flags);
> +		return;
> +	}

[Severity: Critical]
This is a pre-existing issue, but can this early return leak the newly
registered remote port?

If tgt->action is transitioned to IBMVFC_TGT_ACTION_DELETED_RPORT by the
state machine while the worker thread drops the lock to register the port,
we return early here without unregistering the newly created port.

For NVMe targets, tgt->nvme_remote_port->private points to the tgt
structure. If the tgt structure is subsequently freed via kref_put(), could
this cause a use-after-free in the NVMe midlayer when it uses the dangling
private data pointer?

[ ... ]
> @@ -6630,6 +6675,7 @@ static void ibmvfc_rport_add_thread(struct work_struct *work)
[ ... ]
> -	} while(did_work);
> +
> +		list_for_each_entry(tgt, &vhost->nvme_scrqs.targets, queue) {
> +			if (tgt->add_rport) {

[Severity: High]
Are NVMe targets marked for deletion completely ignored by the main state
machine?

The normal deletion path clears tgt->add_rport, which causes this worker
thread to ignore them. However, looking at the main state machine in
ibmvfc_do_work(), the deletion loop exists only for SCSI targets:

drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_do_work() {
    ...
    list_for_each_entry(tgt, &vhost->scsi_scrqs.targets, queue) {
        if (tgt->action == IBMVFC_TGT_ACTION_DEL_RPORT) {
            ...
}

Since the equivalent loop is omitted for nvme_scrqs.targets, do targets
marked for deletion remain unlinked and permanently leaked?

[Severity: High]
On a related note, does the driver fail to unregister the NVMe local port on
module unload?

The teardown path in ibmvfc_remove() removes the SCSI host via
scsi_remove_host() but does not appear to invoke the NVMe equivalent logic.
There is an ibmvfc_nvme_unregister() function provided in the code, but it
seems to be completely dead code that is never called.

Could this lead to a permanent resource leak and potential use-after-free
in the NVMe midlayer if it interacts with the removed host?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=21

Re: [PATCH 23/29] ibmvfc: declare global function definitions

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

Pre-existing issues:
- [High] Objects allocated via mempool_alloc are incorrectly freed using kfree, breaking mempool semantics and causing deadlock under memory pressure.
- [High] Non-synchronous timer cancellation during host removal leads to Use-After-Free on `vhost` in the timer callback.
- [High] Recursive deadlock in ibmvfc_send_event() error path when sending management (MAD format) events.
--

commit 433fbb97fc83e37f1339b6c6598956ee0f1ae0a5
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: declare global function definitions

Some common functions will require visibility by both SCSI and NVMe
protocols. Make common ibmvfc helper routines available to the NVMe
support code.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index 7e6912fba899d..177d341ce7bc6 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c

[ ... ]

> @@ -1410,7 +1403,7 @@ static void ibmvfc_set_rport_dev_loss_tmo(struct fc_rport *rport, u32 timeout)
>   * @kref:		kref struct
>   *
>   **/
> -static void ibmvfc_release_tgt(struct kref *kref)
> +void ibmvfc_release_tgt(struct kref *kref)
>  {
>  	struct ibmvfc_target *tgt = container_of(kref, struct ibmvfc_target, kref);
>  	kfree(tgt);
>  }

[Severity: High]
This isn't a bug introduced by this patch, but does this leak reserved objects
to the slab allocator and risk deadlock?

Since the target is allocated from tgt_pool (a mempool), shouldn't it be freed
with mempool_free() rather than kfree()?

Under memory pressure, if mempool allocations fail, mempool_alloc() sleeps
waiting for objects to return. Freeing directly to the slab bypasses the
mempool and its waitqueue is never woken.

[ ... ]

> @@ -1639,7 +1629,7 @@ static void ibmvfc_locked_done(struct ibmvfc_event *evt)

[Severity: High]
This isn't a bug introduced by this patch, but does this code path contain a
recursive deadlock when sending management (MAD format) events?

Management commands are often issued from a context that already holds
vhost->host->host_lock. If ibmvfc_send_event() encounters a local error, it
synchronously calls evt->done(evt):

ibmvfc_send_event() {
    ...
    if (rc) {
        ...
        evt->done(evt);
    }
}

For MAD format events, evt->done is often assigned to ibmvfc_locked_done(),
which unconditionally attempts to lock the same spinlock again:

ibmvfc_locked_done() {
    ...
    spin_lock_irqsave(evt->vhost->host->host_lock, flags);
    ...
}

Will this cause a fatal recursive deadlock?

>   * @done:	Routine to call when the event is responded to
>   * @format:	SRP or MAD format
>   **/
> -static void ibmvfc_init_event(struct ibmvfc_event *evt,
> +void ibmvfc_init_event(struct ibmvfc_event *evt,

[ ... ]

> @@ -1764,7 +1754,7 @@ static void ibmvfc_timeout(struct timer_list *t)

[Severity: High]
This isn't a bug introduced by this patch, but can ibmvfc_timeout() execute
concurrently and cause a use-after-free on vhost?

During teardown, ibmvfc_purge_requests() cancels each event's timer using the
non-synchronous timer_delete() while holding host_lock.

If the timer callback ibmvfc_timeout() is concurrently firing on another CPU,
it blocks waiting for that lock:

ibmvfc_timeout() {
    ...
    ibmvfc_reset_host(vhost);
}

ibmvfc_reset_host() then calls spin_lock_irqsave(vhost->host->host_lock, flags).

Once the teardown drops the lock and frees vhost, the timer callback acquires
the lock and operates on the freed vhost object. Could this lead to a
use-after-free?

>   *
>   * Returns the value returned from ibmvfc_send_crq(). (Zero for success)
>   **/
> -static int ibmvfc_send_event(struct ibmvfc_event *evt,
> +int ibmvfc_send_event(struct ibmvfc_event *evt,
>  			     struct ibmvfc_host *vhost, unsigned long timeout)

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=23

Re: [PATCH 29/29] ibmvfc: fail nvme-fc fcp-io and ls requests during transport reset

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

New issues:
- [Critical] Pooled event structures retain stale NVMe request pointers, leading to exploitable Use-After-Free and double completions when non-NVMe commands fail.
- [High] Target kref is permanently leaked when an NVMe LS request fails or is purged.

Pre-existing issues:
- [High] Returning SCSI_MLQUEUE_HOST_BUSY for NVMe requests on transport closure causes permanent IO failures instead of requeuing.
- [Medium] NVMe FCP requests are not purged during a transport reset, rendering the patch's FCP functionality dead code.
--

commit 88ce7ce7f11c11cfca700c6919820cf4d9865c8a
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

    ibmvfc: fail nvme-fc fcp-io and ls requests during transport reset
    
    The current purge code for flushing outstanding commands during a
    transport reset only deals with SCSI commands. Rename the
    ibmvfc_scsi_eh_done completion handler to ibmvfc_vfc_eh_done and wire it
    to correctly complete nvme fcp and ls commands when flushing the
    inflight command list during a reset.

> The current purge code for flushing outstanding commands during a
> transport reset only deals with SCSI commands. Rename the
> ibmvfc_scsi_eh_done completion handler to ibmvfc_vfc_eh_done and wire it
> to correctly complete nvme fcp and ls commands when flushing the
> inflight command list during a reset.

[Severity: Medium]
This isn't a bug introduced by this patch, but does the purge code actually
evaluate inflight NVMe FCP commands?

Looking at ibmvfc_purge_requests(), the loop iterates over vhost->crq and
vhost->scsi_scrqs.scrqs:

drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_purge_requests() {
    ...
    for (i = 0; i < hwqs; i++) {
        spin_lock_irqsave(queues[i].q_lock, flags);
    ...
}

It seems to entirely miss vhost->nvme_scrqs.scrqs where FCP requests are
queued. If so, inflight NVMe FCP commands might be orphaned during a reset,
making the new FCP completion handling unreachable.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-core.c b/drivers/scsi/ibmvscsi/ibmvfc-core.c
> index a7183493cf965..aeb5b8902aa55 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-core.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-core.c

[ ... ]

> @@ -1092,20 +1092,28 @@ void ibmvfc_free_event(struct ibmvfc_event *evt)
>  }
>  
>  /**
> - * ibmvfc_scsi_eh_done - EH done function for queuecommand commands
> + * ibmvfc_vfc_eh_done - EH done function for queued IO
>   * @evt:	ibmvfc event struct
>   *
> - * This function does not setup any error status, that must be done
> - * before this function gets called.
> + * This function does not setup any error status for scsi commands, that must be
> + * done before this function gets called.
>   **/
> -static void ibmvfc_scsi_eh_done(struct ibmvfc_event *evt)
> +static void ibmvfc_vfc_eh_done(struct ibmvfc_event *evt)
>  {
>  	struct scsi_cmnd *cmnd = evt->cmnd;
> +	struct nvmefc_ls_req *ls_req = evt->ls_req;
> +	struct nvmefc_fcp_req *fcp_req = evt->fcp_req;
>  
>  	if (cmnd) {
>  		scsi_dma_unmap(cmnd);
>  		scsi_done(cmnd);
> -	}
> +	} else if (fcp_req) {
> +		fcp_req->rcv_rsplen = 0;
> +		fcp_req->transferred_length = 0;
> +		fcp_req->status = NVME_SC_INTERNAL;
> +		fcp_req->done(fcp_req);
> +	} else if (ls_req)
> +		ls_req->done(ls_req, -EIO);

[Severity: High]
Does this error handler leak the target kref for LS requests?

In ibmvfc_nvme_ls_req(), a reference is taken via kref_get(&tgt->kref) which
is typically dropped in the success path by ibmvfc_ls_req_done().

Since this patch replaces the completion handler with ibmvfc_vfc_eh_done(),
the transport request is completed here but kref_put(&evt->tgt->kref, ...)
is never called, potentially leaving the target structure leaked.

>  
>  	ibmvfc_free_event(evt);
>  }
> @@ -1146,8 +1154,10 @@ static void ibmvfc_fail_request(struct ibmvfc_event *evt, int error_code)
>  	BUG_ON(!atomic_dec_and_test(&evt->active));
>  	if (evt->cmnd) {
>  		evt->cmnd->result = (error_code << 16);
> -		evt->done = ibmvfc_scsi_eh_done;
> -	} else
> +		evt->done = ibmvfc_vfc_eh_done;
> +	} else if (evt->fcp_req || evt->ls_req)
> +		evt->done = ibmvfc_vfc_eh_done;
> +	else

[Severity: Critical]
Can these checks be triggered by stale NVMe pointers on non-NVMe commands?

When ibmvfc_get_event() reallocates an event structure from the pool,
ibmvfc_init_event() doesn't clear evt->fcp_req or evt->ls_req to NULL:

drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_init_event() {
    ...
    evt->cmnd = NULL;
    evt->sync_iu = NULL;
    evt->eh_comp = NULL;
    ...
}

If a non-NVMe command (e.g., a MAD command) is allocated an event with stale
NVMe pointers and fails here, will it incorrectly replace the completion
handler with ibmvfc_vfc_eh_done() and later use-after-free the stale NVMe
pointers?

>  		evt->xfer_iu->mad_common.status = cpu_to_be16(IBMVFC_MAD_DRIVER_FAILED);
>  
>  	timer_delete(&evt->timer);

[ ... ]

> @@ -1816,8 +1826,10 @@ int ibmvfc_send_event(struct ibmvfc_event *evt,

[Severity: High]
This isn't a bug introduced by this patch, but earlier in ibmvfc_send_event(),
it hardcodes a return of SCSI_MLQUEUE_HOST_BUSY on H_CLOSED without
distinguishing between SCSI and NVMe commands:

drivers/scsi/ibmvscsi/ibmvfc-core.c:ibmvfc_send_event() {
    ...
    if (rc == H_CLOSED) {
        ...
        return SCSI_MLQUEUE_HOST_BUSY;
    }
    ...
}

This value (0x1055) is propagated back up through ibmvfc_nvme_fcp_io()
directly to the NVMe FC transport layer. Since 0x1055 is neither 0 nor
-EBUSY, will the NVMe FC core treat it as a hard failure and abort the
request with BLK_STS_IOERR instead of handling the queue pressure
appropriately?

>  		dev_err(vhost->dev, "Send error (rc=%d)\n", rc);
>  		if (evt->cmnd) {
>  			evt->cmnd->result = DID_ERROR << 16;
> -			evt->done = ibmvfc_scsi_eh_done;
> -		} else
> +			evt->done = ibmvfc_vfc_eh_done;
> +		} else if (evt->fcp_req || evt->ls_req)
> +			evt->done = ibmvfc_vfc_eh_done;
> +		else
>  			evt->xfer_iu->mad_common.status = cpu_to_be16(IBMVFC_MAD_CRQ_ERROR);
>  
>  		evt->done(evt);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=29

Re: [PATCH 28/29] ibmvfc: implement nvme-fc FCP abort callback

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [High] Synchronous wait in `fcp_abort` violates the NVMe-FC transport API and blocks the block layer timeout handler.
- [High] Lack of synchronization against command completion allows `fcp_abort` to abort the wrong I/O request and causes a Use-After-Free of `ibmvfc_event`.
- [High] Indefinite hang in `wait_for_completion` when an NVMe MAD command times out, due to missing NVMe queue cleanup in the host reset path.
- [High] Lock inversion (AB-BA) deadlock between the hardware queue lock and the SCSI host lock during the failure path of `ibmvfc_send_event`.
- [High] Uninitialized `evt->tgt` pointer in FCP IO requests causes a NULL pointer dereference or Use-After-Free in the abort path.
- [Medium] Event pool exhaustion prevents aborts from issuing during hardware queue saturation, causing the transport to hang.
--

commit 0e079891124178111c0e0055ee9e67515aff453b
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: implement nvme-fc FCP abort callback

This commit implements the NVMe-FC FCP abort callback by issuing an NVMF
cancel MAD on the same submission queue used by the original FCP request.
It factors the TMF setup into a dedicated helper and sends the cancel
synchronously, waiting for completion.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> index 18e8657abc444..92937f9aa4649 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> @@ -374,11 +374,75 @@ static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport,
[ ... ]
> +static void ibmvfc_init_fcp_abort(struct ibmvfc_event *evt,
> +				  struct nvmefc_fcp_req *abort_req)
> +{
> +	struct ibmvfc_tmf *tmf;
> +	struct ibmvfc_event *abt_evt = abort_req->private;
> +	struct ibmvfc_target *tgt = abt_evt->tgt;
> +
> +	tmf = &evt->iu.tmf;
> +	memset(tmf, 0, sizeof(*tmf));
> +	tmf->common.version = cpu_to_be32(2);
> +	tmf->common.opcode = cpu_to_be32(IBMVFC_NVMF_TMF_MAD);
> +	tmf->common.length = cpu_to_be16(sizeof(*tmf));
> +	tmf->flags = cpu_to_be32(IBMVFC_TMF_ABORT_TASK | IBMVFC_TMF_NVMF_ASSOC);
> +	tmf->cancel_key = cpu_to_be32((u64)abt_evt);
> +	tmf->my_cancel_key = cpu_to_be32((u64)evt);
> +	tmf->target_wwpn = cpu_to_be64(tgt->wwpn);

[Severity: High]
Does this dereference a NULL or stale pointer?

It appears that when an NVMe FCP request is submitted via
ibmvfc_nvme_fcp_io(), an ibmvfc_event is allocated but its evt->tgt
field is never initialized. Because evt->tgt was never set during
submission, could it contain NULL (from pool zeroing) or a stale pointer
from a previous LS request when we extract it here?

> +	tmf->assoc_id = cpu_to_be64(tgt->assoc_id);
> +	tmf->task_tag = cpu_to_be64((u64)abt_evt);
> +
> +	init_completion(&evt->comp);
> +}
> +
> +static void ibmvfc_nvme_fcp_abort(struct nvme_fc_local_port *lport,
> +				  struct nvme_fc_remote_port *rport,
> +				  void *hw_queue_handle,
> +				  struct nvmefc_fcp_req *abort_req)
> +{
> +	struct ibmvfc_host *vhost = lport->private;
> +	struct ibmvfc_target *tgt = rport->private;
> +	struct ibmvfc_event *evt, *abt_evt = abort_req->private;
> +	struct ibmvfc_queue *queue;
> +	union ibmvfc_iu rsp;
> +	unsigned long flags;
> +	u16 status = 0;
> +
> +	if (!abt_evt)
> +		return;
> +
> +	queue = abt_evt->queue;

[Severity: High]
Is it possible for the original command to complete concurrently at the
hardware level before this code accesses abt_evt?

If the interrupt handler returns abt_evt to the free pool, could this
cause a use-after-free? Furthermore, if the event is reallocated to a
new I/O request on another CPU, could the TMF payload be populated with
the new event's tag, causing the VIOS to abort the wrong I/O request?

> +	if (!vhost->logged_in || !queue)
> +		return;
> +
> +	evt = ibmvfc_get_event(queue);
> +	if (!evt)
> +		return;

[Severity: Medium]
Will this silently drop the abort request if the standard event pool is
exhausted?

Since aborts are typically invoked when commands are timing out (often
due to submission queue saturation), should this use a reserved event?
If this returns NULL without sending the abort, could the command hang
indefinitely since the NVMe FC transport assumes the abort was initiated?

> +
> +	spin_lock_irqsave(queue->q_lock, flags);
> +	kref_get(&tgt->kref);
> +	ibmvfc_init_event(evt, ibmvfc_sync_nvme_completion, IBMVFC_MAD_FORMAT);
> +	ibmvfc_init_fcp_abort(evt, abort_req);
> +	evt->sync_iu = &rsp;
> +
> +	if (ibmvfc_send_event(evt, vhost, default_timeout))

[Severity: High]
Can this cause a lock inversion (AB-BA) deadlock?

We acquire queue->q_lock before calling ibmvfc_send_event(). If
ibmvfc_send_event() encounters an error, it calls evt->done(evt).
Because the event is initialized with IBMVFC_MAD_FORMAT, the callback is
ibmvfc_locked_done(), which unconditionally acquires host_lock.

This creates an acquisition order of queue->q_lock -> host_lock. However,
the driver's reset path in ibmvfc_reset_host() acquires host_lock first,
then iterates over the hardware queues to acquire queue->q_lock inside
ibmvfc_purge_requests().

> +		goto out;
> +
> +	spin_unlock_irqrestore(queue->q_lock, flags);
> +
> +	wait_for_completion(&evt->comp);

[Severity: High]
Does this synchronous wait violate the NVMe-FC transport API?

The NVMe FC transport requires fcp_abort to be an asynchronous operation.
Because fcp_abort is invoked by the block layer's timeout worker, waiting
synchronously for the abort MAD to complete could stall the timeout
processing thread for the system until the VIOS responds.

[Severity: High]
Can this wait_for_completion() hang indefinitely if the NVMe MAD command
times out?

The fcp_abort TMF event is allocated on the NVMe queue (nvme_scrqs).
If the VIOS fails to respond to the cancel MAD, ibmvfc_send_event() times
out and triggers a host reset, invoking ibmvfc_purge_requests().

However, ibmvfc_purge_requests() iterates only over the legacy CRQ and
vhost->scsi_scrqs, entirely ignoring vhost->nvme_scrqs. Because the event
is never failed by the purge logic, could this wait forever?

> +	status = be16_to_cpu(rsp.mad_common.status);
> +

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=28

Re: [PATCH 24/29] ibmvfc: implement LLDD callbacks for mapping nvme-fc queues

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Lockless double-read of `vhost->nvme_scrqs.active_queues` can cause a division-by-zero kernel panic.
--

commit fd68a3e765846c041d31997c784f4e4a733bb8aa
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: implement LLDD callbacks for mapping nvme-fc queues

This commit implements the NVMe-FC queue create and delete callbacks. It maps
NVMe controller queues onto ibmvfc hardware queues using the qidx.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> index fc4337fc9b3fb..1108d11d6b2db 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c

[ ... ]

> @@ -28,6 +28,40 @@ static void ibmvfc_nvme_remoteport_delete(struct nvme_fc_remote_port *rport)
>  	complete(&tgt->nvme_delete_done);
>  }
>  
> +static int ibmvfc_nvme_create_queue(struct nvme_fc_local_port *lport, unsigned int qidx,
> +				    u16 qsize, void **handle)
> +{
> +	struct ibmvfc_host *vhost = lport->private;
> +	struct ibmvfc_nvme_qhandle *qhandle;
> +
> +	if (!vhost->nvme_scrqs.active_queues)
> +		return -ENODEV;
> +
> +	qhandle = kzalloc_obj(struct ibmvfc_nvme_qhandle);
> +	if (!qhandle)
> +		return -ENOMEM;
> +
> +	qhandle->cpu_id = raw_smp_processor_id();
> +	qhandle->qidx = qidx;
> +
> +	/* Admin and first IO queue are both mapped to index 0 */
> +	if (qidx)
> +		qhandle->index = (qidx - 1) % vhost->nvme_scrqs.active_queues;

[Severity: High]
Could a lockless double-read of vhost->nvme_scrqs.active_queues lead to a
division by zero?

If a concurrent host reset or channel setup failure sets active_queues to 0,
it seems the compiler could reload the value for the modulo operation since
the memory allocation acts as a compiler barrier.

Would reading active_queues once into a local variable prevent a potential
kernel panic here?

> +	else
> +		qhandle->index = qidx;
> +
> +	qhandle->queue = &vhost->nvme_scrqs.scrqs[qhandle->index];
> +
> +	*handle = qhandle;
> +	return 0;
> +}

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=24

Re: [PATCH 25/29] ibmvfc: implement nvme-fc LS submission transport callback

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The driver accesses NVMe-FC response payloads (`evt->ls_req->rspaddr` and `fcp_req->rspaddr`) from CPU memory while they are still mapped for DMA, before the NVMe-FC core unmaps or syncs them. On architectures using SWIOTLB bounce buffers (like PowerPC Secure Execution), this causes the driver to read stale data and its manual payload writes to be clobbered by the subsequent DMA sync.
--

commit 1915a13751880ed8af4917b3092d7cf5b63c79eb
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: implement nvme-fc LS submission transport callback
    
NVMe FC Link Service commands are required to use the ibmvfc_passthru
MAD. Initialize a pssthru mad for the target port including the DMA
addresses for the FC4_LS request and response as well as the max length
of each IU as provided in the nvmefc_ls_req struct. FC4_LS commands are
sent via the primary CRQ. Further, store the assoc_id during a create
association request as this is a required field in our vfc_cmd struct
for nvme_fcp_io commands.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> index 1108d11d6b2db..506135c1a34e3 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c

[ ... ]

> @@ -62,10 +62,99 @@ static void ibmvfc_nvme_delete_queue(struct nvme_fc_local_port *lport, unsigned
>  	kfree(handle);
>  }
>  
> +static void ibmvfc_ls_req_done(struct ibmvfc_event *evt)
> +{
> +	struct ibmvfc_target *tgt = evt->tgt;
> +	struct ibmvfc_passthru_mad *mad = &evt->xfer_iu->passthru;
> +	struct fcnvme_ls_rqst_w0 *ls_rqst;
> +	struct fcnvme_ls_cr_assoc_acc *ls_resp;
> +	u32 status = be16_to_cpu(mad->common.status);
> +	int rc = 0;
> +
> +	ls_rqst = (struct fcnvme_ls_rqst_w0 *)evt->ls_req->rqstaddr;
> +	ls_resp = (struct fcnvme_ls_cr_assoc_acc *)evt->ls_req->rspaddr;
> +
> +	switch (status) {
> +	case IBMVFC_MAD_SUCCESS:
> +		tgt_dbg(tgt, "ls_req succeeded\n");
> +		if ((ls_rqst->ls_cmd == FCNVME_LS_CREATE_ASSOCIATION) &&
> +		    (ls_resp->hdr.w0.ls_cmd == FCNVME_LS_ACC)) {
> +			tgt->assoc_id = be64_to_cpu(ls_resp->associd.association_id);

[Severity: High]
Does this code read stale data from the response buffer when bounce buffers
are active?

Since the response payload at evt->ls_req->rspaddr is mapped for DMA, and the
NVMe-FC core hasn't unmapped or synced it yet, reading the association_id
here without a prior dma_sync_single_for_cpu() could result in reading stale
memory on architectures using SWIOTLB bounce buffers (like PowerPC Secure
Execution).

Additionally, does a similar issue occur in ibmvfc_nvme_done()? If the driver
reads ersp->xfrd_len from rspaddr and writes to cqe->command_id before
calling fcp_req->done(), the subsequent DMA sync inside fcp_req->done()
could overwrite the manually written command_id with unmodified bounce buffer
memory, causing the I/O to fail.

> +			tgt_dbg(tgt, "assoc_id 0x%llx\n", tgt->assoc_id);
> +		}
> +		break;
> +	case IBMVFC_MAD_DRIVER_FAILED:
> +		break;

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=25

Re: [PATCH 26/29] ibmvfc: implement nvme-fc IO command submission callback

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Buffer overflow in NVMe scatterlist mapping when sg_cnt exceeds SG_ALL.
- [High] Returning SCSI-specific or bare memory allocation errors from `fcp_io` causes the NVMe-FC core to permanently fail retriable commands.
- [High] Overwriting `fcp_req->private` causes an invalid `kfree` of an interior pool pointer and a memory leak.
- [High] Stale data is read from the ERSP buffer when the target responds with `IBMVFC_NVMS_VALID_NODMA_CQE`.
--

commit 6b37a58cc483eca7d86fda91190fd35dfdfe359a
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: implement nvme-fc IO command submission callback

This commit adds helpers to initialize an ibmvfc command from an
nvmefc_fcp_req. It handles scatterlist mapping, submits the request on
the NVMe hardware queue, and handles command completion translation.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> index 506135c1a34e3..bff469d0b47df 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c

[ ... ]

> @@ -164,12 +165,149 @@ static void ibmvfc_nvme_ls_abort(struct nvme_fc_local_port *lport,
>  {
>  }
>  
> +static void ibmvfc_nvme_done(struct ibmvfc_event *evt)
> +{
> +	struct ibmvfc_cmd *vfc_cmd = &evt->xfer_iu->cmd;
> +	struct nvmefc_fcp_req *fcp_req = evt->fcp_req;
> +	struct nvme_fc_ersp_iu *ersp = (struct nvme_fc_ersp_iu *)fcp_req->rspaddr;
> +	struct nvme_completion *cqe = &ersp->cqe;
> +	struct nvme_command *sqe = &((struct nvme_fc_cmd_iu *)fcp_req->cmdaddr)->sqe;
> +
> +	ibmvfc_dbg(evt->vhost, "fc_done: (%x:%x)\n", be16_to_cpu(vfc_cmd->status),
> +		   be16_to_cpu(vfc_cmd->error));
> +	ibmvfc_dbg(evt->vhost, "fc_done: cmdlen: %d, rsplen %d, payload_len %d\n",
> +		   fcp_req->cmdlen, fcp_req->rsplen, fcp_req->payload_length);
> +
> +	fcp_req->status = 0;
> +	if (!vfc_cmd->status) {
> +		fcp_req->rcv_rsplen = NVME_FC_SIZEOF_ZEROS_RSP;
> +		fcp_req->transferred_length = fcp_req->payload_length;
> +	} else if (be16_to_cpu(vfc_cmd->status) & IBMVFC_FC_NVME_STATUS) {
> +		fcp_req->rcv_rsplen = sizeof(struct nvme_fc_ersp_iu);
> +		fcp_req->transferred_length = be32_to_cpu(ersp->xfrd_len);
> +		if (be16_to_cpu(vfc_cmd->error) & IBMVFC_NVMS_VALID_NODMA_CQE)
> +			cqe->command_id = sqe->common.command_id;

[Severity: High]
Does this code read stale data from the ERSP buffer when the target
responds with IBMVFC_NVMS_VALID_NODMA_CQE?

If the completion queue entry was not DMA'd into the ersp buffer,
ersp->xfrd_len, cqe->status, and cqe->result are not initialized,
but this path proceeds to use be32_to_cpu(ersp->xfrd_len) to set the
transferred length.

Since the NVMe-FC core reuses the request structure without zeroing
the rspaddr buffer between commands, this can cause the driver to
read stale data from previous commands, resulting in data corruption
and validation failures.

> +	} else {
> +		fcp_req->rcv_rsplen = 0;
> +		fcp_req->transferred_length = 0;
> +		fcp_req->status = NVME_SC_INTERNAL;
> +	}
> +
> +	fcp_req->done(fcp_req);
> +	ibmvfc_free_event(evt);
> +}

[ ... ]

> +static int ibmvfc_nvme_map_sg_data(struct nvmefc_fcp_req *fcp_req,
> +				    struct ibmvfc_event *evt,
> +				    struct ibmvfc_cmd *vfc_cmd)
> +{
> +	struct srp_direct_buf *data = &vfc_cmd->ioba;
> +	struct ibmvfc_host *vhost = evt->vhost;
> +
> +	if (!fcp_req->sg_cnt) {
> +		vfc_cmd->flags |= cpu_to_be16(IBMVFC_NO_MEM_DESC);
> +		return 0;
> +	}
> +
> +	if (fcp_req->io_dir == NVMEFC_FCP_WRITE)
> +		vfc_cmd->flags |= cpu_to_be16(IBMVFC_WRITE);
> +	else
> +		vfc_cmd->flags |= cpu_to_be16(IBMVFC_READ);
> +
> +	if (fcp_req->sg_cnt == 1) {
> +		ibmvfc_nvme_map_sg_list(fcp_req, data);
> +		return 0;
> +	}
> +
> +	vfc_cmd->flags |= cpu_to_be16(IBMVFC_SCATTERLIST);
> +
> +	if (!evt->ext_list) {
> +		evt->ext_list = dma_pool_alloc(vhost->sg_pool, GFP_ATOMIC,
> +					       &evt->ext_list_token);
> +
> +		if (!evt->ext_list)
> +			return -ENOMEM;
> +	}
> +
> +	ibmvfc_nvme_map_sg_list(fcp_req, evt->ext_list);

[Severity: High]
Can this cause a buffer overflow when fcp_req->sg_cnt exceeds SG_ALL?

The NVMe-FC transport template sets max_sgl_segments to 1024, allowing
requests with up to 1024 scatterlist elements.

However, the external list is allocated from vhost->sg_pool, which is
sized at SG_ALL (128) elements. If fcp_req->sg_cnt is greater than 128,
ibmvfc_nvme_map_sg_list() will iterate up to sg_cnt and write
out of bounds, corrupting adjacent DMA pool memory.

> +
> +	data->va = cpu_to_be64(evt->ext_list_token);
> +	data->len = cpu_to_be32(fcp_req->sg_cnt * sizeof(struct srp_direct_buf));
> +	data->key = 0;
> +	return 0;
> +}
> +
>  static int ibmvfc_nvme_fcp_io(struct nvme_fc_local_port *lport,
>  			      struct nvme_fc_remote_port *rport,
>  			      void *hw_queue_handle,
>  			      struct nvmefc_fcp_req *fcp_req)
>  {
> -	return 0;
> +	struct ibmvfc_host *vhost = lport->private;
> +	struct ibmvfc_nvme_qhandle *qhandle = hw_queue_handle;
> +	struct ibmvfc_cmd *vfc_cmd;
> +	struct ibmvfc_event *evt;
> +	int rc;
> +
> +	ibmvfc_dbg(vhost, "nvme_fcp_io\n");
> +	evt = ibmvfc_get_event(qhandle->queue);
> +	if (!evt)
> +		return -EBUSY;
> +
> +	evt->hwq = qhandle->index;
> +	ibmvfc_dbg(vhost, "vfc-nvme-mq-%d\n", evt->hwq);
> +
> +	ibmvfc_init_event(evt, ibmvfc_nvme_done, IBMVFC_CMD_FORMAT);
> +	evt->fcp_req = fcp_req;
> +	fcp_req->private = evt;

[Severity: High]
Could overwriting fcp_req->private cause an invalid kfree and memory leak?

The NVMe-FC core allocates the private area for the driver (specified by
fcprqst_priv_sz) and stores the pointer in fcp_req->private. For
Asynchronous Event Notifications (AENs), this memory is allocated via
kzalloc.

By overwriting the private pointer with evt here, during controller resets
or teardown, the NVMe-FC core will call kfree(aen_op->fcp_req.private).
Because the pointer now points to evt (which resides within the contiguous
pool->events array), this results in freeing an interior pointer, triggering
a kernel bug.

> +
> +	vfc_cmd = ibmvfc_nvme_init_vfc_cmd(evt, rport, fcp_req);
> +
> +	vfc_cmd->correlation = cpu_to_be64((u64)evt);
> +
> +	if (likely(!(rc = ibmvfc_nvme_map_sg_data(fcp_req, evt, vfc_cmd))))
> +		return ibmvfc_send_event(evt, vhost, 0);
> +
> +	ibmvfc_free_event(evt);
> +
> +	return rc;

[Severity: High]
Does returning rc directly bypass the expected -EBUSY retry mechanism?

The ibmvfc_nvme_fcp_io callback can return -ENOMEM from map_sg_data or
SCSI_MLQUEUE_HOST_BUSY (0x1055) from ibmvfc_send_event on temporary
allocation or queue closure failures.

The NVMe-FC core expects fcp_io to return exactly -EBUSY to trigger a
requeue (BLK_STS_RESOURCE). For any other non-zero return value,
nvme_fc_start_fcp_op returns BLK_STS_IOERR, translating transient
hardware constraints into permanent I/O errors and data loss for the
application.

>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=26

Re: [PATCH 27/29] ibmvfc: implement nvme-fc LS abort handling callback

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider:
- [High] The allocated ibmvfc_event structure is leaked on early return if the virtual host is not logged in.
- [High] Recursive spinlock self-deadlock or Use-After-Free on `ibmvfc_send_event()` failure.
- [High] Unsynchronized access to a potentially freed event structure (`abt_evt`) via `ls_abort->private` leads to a Use-After-Free.
- [High] Event leak when `ibmvfc_send_event()` fails, bypassing the freeing logic.
- [Medium] Logic inversion defeats the `IBMVFC_CAN_SUPPRESS_ABTS` capability check.
- [Low] Uninitialized stack variable `status` is read and printed on the error path.
- [Low] Returning a SCSI midlayer error code (`SCSI_MLQUEUE_HOST_BUSY`) to the NVMe FC transport causes unintended API violations.
--

commit e8f5bb9c65a6243c394bbc91b4a31f61c71ee322
Author: Tyrel Datwyler <tyreld@linux.ibm.com>

ibmvfc: implement nvme-fc LS abort handling callback

Implement the NVMe FC-LS abort callback by issuing an ibmvfc cancel MAD
to the VIOS for the outstanding link-service request.

Use the saved event pointer from the original FC-LS request to identify
the command to cancel, submit the cancel operation, and complete the
abort request based on the returned status.

> diff --git a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> index bff469d0b47df..18e8657abc444 100644
> --- a/drivers/scsi/ibmvscsi/ibmvfc-nvme.c
> +++ b/drivers/scsi/ibmvscsi/ibmvfc-nvme.c

[ ... ]

> @@ -159,10 +161,72 @@ static int ibmvfc_nvme_ls_req(struct nvme_fc_local_port *lport,

[ ... ]

> +static void ibmvfc_init_ls_abort(struct ibmvfc_event *evt, struct nvmefc_ls_req *ls_abort)
> +{
> +	struct ibmvfc_tmf *tmf;
> +	struct ibmvfc_event *abt_evt = ls_abort->private;
> +	struct ibmvfc_target *tgt = abt_evt->tgt;

[Severity: High]
Can this sequence actually occur? If the NVMe FC transport invokes the abort
callback concurrently with the normal completion of the LS request, could
ibmvfc_ls_req_done() complete the request and free abt_evt to the pool before
we access abt_evt->tgt here?

> +	struct ibmvfc_host *vhost = evt->vhost;
> +
> +	tmf = &evt->iu.tmf;
> +	memset(tmf, 0, sizeof(*tmf));
> +	tmf->common.version = cpu_to_be32(2);
> +	tmf->target_wwpn = cpu_to_be64(tgt->wwpn);
> +	tmf->common.opcode = cpu_to_be32(IBMVFC_NVMF_TMF_MAD);
> +	tmf->common.length = cpu_to_be16(sizeof(*tmf));
> +	if (vhost->state != IBMVFC_ACTIVE)
> +		if (!ibmvfc_check_caps(vhost, IBMVFC_CAN_SUPPRESS_ABTS))
> +			tmf->flags = cpu_to_be32(IBMVFC_TMF_SUPPRESS_ABTS);

[Severity: Medium]
Does this logic correctly evaluate the capability? The check evaluates to true
only when the adapter does not support suppressing ABTS, which appears to
set IBMVFC_TMF_SUPPRESS_ABTS on adapters that do not support it, and omits
it on adapters that do.

> +	tmf->cancel_key = cpu_to_be32((u64)abt_evt);
> +	tmf->my_cancel_key = cpu_to_be32((u64)evt);
> +	tmf->assoc_id = cpu_to_be64(tgt->assoc_id);
> +
> +	init_completion(&evt->comp);
> +}
> +
>  static void ibmvfc_nvme_ls_abort(struct nvme_fc_local_port *lport,
>  				struct nvme_fc_remote_port *rport,
>  				struct nvmefc_ls_req *ls_abort)
>  {
> +	struct ibmvfc_host *vhost = lport->private;
> +	struct ibmvfc_target *tgt = rport->private;
> +	struct ibmvfc_event *evt;
> +	union ibmvfc_iu rsp;
> +	unsigned long flags;
> +	u16 status;
> +
> +	evt = ibmvfc_get_event(&vhost->crq);
> +	if (!vhost->logged_in || !evt)
> +		return;

[Severity: High]
Does this code leak the event structure? If vhost->logged_in is false,
ibmvfc_get_event() might successfully allocate an event from the vhost->crq
pool, but returning immediately here leaves the allocated event unreleased.
Repeated abort attempts could permanently exhaust the event pool.

> +
> +	spin_lock_irqsave(vhost->host->host_lock, flags);
> +	kref_get(&tgt->kref);
> +	ibmvfc_init_event(evt, ibmvfc_sync_nvme_completion, IBMVFC_MAD_FORMAT);
> +	ibmvfc_init_ls_abort(evt, ls_abort);
> +	evt->sync_iu = &rsp;
> +
> +	if (ibmvfc_send_event(evt, vhost, default_timeout))

[Severity: High]
Can this lead to a recursive self-deadlock or a use-after-free?

If ibmvfc_send_event() fails, it invokes evt->done(evt) synchronously:

ibmvfc_send_event()
    if (rc)
        evt->done(evt);

If the event is fresh, evt->done remains ibmvfc_locked_done(), which attempts
to acquire vhost->host->host_lock. Since this lock is already held here before
calling ibmvfc_send_event(), will this self-deadlock?

Additionally, because ibmvfc_init_event() doesn't zero evt->ls_req or
evt->fcp_req, could recycled events contain stale pointers that cause a
use-after-free when ibmvfc_send_event() evaluates those fields and replaces
evt->done with ibmvfc_vfc_eh_done() on failure?

[Severity: Low]
On a related note regarding ibmvfc_send_event(), when the receive queue is
closed it returns SCSI_MLQUEUE_HOST_BUSY. In places like ibmvfc_nvme_fcp_io()
that directly return the output of ibmvfc_send_event(), does returning a
positive SCSI midlayer error code to the NVMe FC transport cause unintended
API violations? The NVMe FC transport typically expects 0 for success or
standard negative error codes.

> +		goto out;

[Severity: High]
Does this goto skip the event cleanup? Although ibmvfc_send_event()'s error
path executes evt->done, ibmvfc_sync_nvme_completion() only calls complete()
on the event wait structure, meaning the event is never explicitly freed and
could be permanently leaked.

> +
> +	spin_unlock_irqrestore(vhost->host->host_lock, flags);
> +
> +	wait_for_completion(&evt->comp);
> +	status = be16_to_cpu(rsp.mad_common.status);
> +	spin_lock_irqsave(vhost->host->host_lock, flags);
> +	ibmvfc_free_event(evt);
> +out:
> +	spin_unlock_irqrestore(vhost->host->host_lock, flags);
> +	ibmvfc_dbg(vhost, "ls_abort: cancel failed with rc=%x\n", status);

[Severity: Low]
Is status uninitialized when we reach the out label? If ibmvfc_send_event()
fails and triggers the goto, status is never assigned and might leak
uninitialized stack memory into the logs.

> +	kref_put(&tgt->kref, ibmvfc_release_tgt);
>  }

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260623013035.3436640-1-tyreld@linux.ibm.com?part=27