[PATCH 00/14] lpfc multiqueue performance fixes

[PATCH 00/14] lpfc multiqueue performance fixes

[Hannes Reinecke]

Hi all,

here's a patchset to improve lpfc multiqueue performance.
They are a reworked version of those which I've used for
the performance tests I've presented at VAULT 2016.

The patches address two issues in the lpfc driver:
- Decouple abort I/O handling.
  Currently the lpfc driver has to synchronize I/O completions
  with pending aborts, which involves taking a spinlock during
  I/O completion. With this patchset aborts can run independent
  of the I/O completion, and the spinlock can be avoided.
- Array-based command allocation.
  The original driver was using a list-based command allocation,
  which required us to take a lock during command submission.
  This patchset implements an array-based command allocation where
  each element is addressed by the command tag. As the blocklayer
  ensures that we'll never see busy tags we can avoid the spinlock.

To decoupling aborts and I/O completion works as the firmware will
always send an I/O completion, even if the command has been aborted.
So we can as well keep the I/O running when an abort is active (ie
the block timeout function will return BLK_EH_RESET_TIMER), and
complete the command _after_ the aborts are finished. This way
we can ensure that no aborts will be running by the time the I/O
completion (ie ->scsi_done()) is called.
As there are potentially other drivers which could benefit from this
I've added a new SCSI host template flag 'abort_completions'.

The patchset is based on linux v4.7, and can be found at

http://git.kernel.org/hare/scsi-devel/h/lpfc.v3

As usual, comments and reviews are welcome.

Hannes Reinecke (14):
  block: don't check request size in blk_cloned_rq_check_limits()
  lpfc: remove unused 'pring' argument
  lpfc: Add config option 'lpfc_enable_stats'
  scsi: add 'abort_completions' host template flag
  lpfc: do not wait for completion when aborting commands
  lpfc: display manual queue assignment
  scsi: modify can_queue after calling mq_allocate()
  blk-mq: add blk_mq_resize_tag_set()
  scsi: Implement scsi_mq_resize_tags()
  lpfc: use bitflag for exch_busy
  lpfc: set LPFC_CMD_ABORTED when a command is put on the aborted list
  lpfc: move to array based command allocation for SLI-4
  lpfc: LPFC_CMD_RRQ_ACTIVE flag to mark commands
  lpfc: Complete scsi commands after RRQ has completed

 block/blk-core.c              |   5 -
 block/blk-mq.c                |  15 +++
 drivers/scsi/lpfc/lpfc.h      |   2 +
 drivers/scsi/lpfc/lpfc_attr.c |  20 +++-
 drivers/scsi/lpfc/lpfc_init.c |  54 +++++++++-
 drivers/scsi/lpfc/lpfc_scsi.c | 222 +++++++++++++++++++++++++++++-------------
 drivers/scsi/lpfc/lpfc_scsi.h |   7 +-
 drivers/scsi/lpfc/lpfc_sli.c  |  77 ++++++++++++---
 drivers/scsi/scsi_error.c     |   7 +-
 drivers/scsi/scsi_lib.c       |  14 ++-
 include/linux/blk-mq.h        |   1 +
 include/scsi/scsi_host.h      |   7 ++
 12 files changed, 337 insertions(+), 94 deletions(-)

-- 
1.8.5.6

[PATCH 01/14] block: don't check request size in blk_cloned_rq_check_limits()

[Hannes Reinecke]

When checking a cloned request there is no need to check
the overall request size; this won't have changed even
when resubmitting to another queue.
Without this patch ppc64le on ibmvfc fails to boot.

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 block/blk-core.c | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index 2475b1c7..e108bf0 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -2160,11 +2160,6 @@ EXPORT_SYMBOL(submit_bio);
 static int blk_cloned_rq_check_limits(struct request_queue *q,
 				      struct request *rq)
 {
-	if (blk_rq_sectors(rq) > blk_queue_get_max_sectors(q, rq->cmd_flags)) {
-		printk(KERN_ERR "%s: over max size limit.\n", __func__);
-		return -EIO;
-	}
-
 	/*
 	 * queue's settings related to segment counting like q->bounce_pfn
 	 * may differ from that of other stacking queues.
-- 
1.8.5.6

[PATCH 02/14] lpfc: remove unused 'pring' argument

[Hannes Reinecke]

---
 drivers/scsi/lpfc/lpfc_sli.c | 17 ++++++-----------
 1 file changed, 6 insertions(+), 11 deletions(-)

diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index 70edf21..5e38a35 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -2657,7 +2657,6 @@ lpfc_sli_process_unsol_iocb(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
 /**
  * lpfc_sli_iocbq_lookup - Find command iocb for the given response iocb
  * @phba: Pointer to HBA context object.
- * @pring: Pointer to driver SLI ring object.
  * @prspiocb: Pointer to response iocb object.
  *
  * This function looks up the iocb_lookup table to get the command iocb
@@ -2668,7 +2667,6 @@ lpfc_sli_process_unsol_iocb(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
  **/
 static struct lpfc_iocbq *
 lpfc_sli_iocbq_lookup(struct lpfc_hba *phba,
-		      struct lpfc_sli_ring *pring,
 		      struct lpfc_iocbq *prspiocb)
 {
 	struct lpfc_iocbq *cmd_iocb = NULL;
@@ -2697,7 +2695,6 @@ lpfc_sli_iocbq_lookup(struct lpfc_hba *phba,
 /**
  * lpfc_sli_iocbq_lookup_by_tag - Find command iocb for the iotag
  * @phba: Pointer to HBA context object.
- * @pring: Pointer to driver SLI ring object.
  * @iotag: IOCB tag.
  *
  * This function looks up the iocb_lookup table to get the command iocb
@@ -2708,7 +2705,7 @@ lpfc_sli_iocbq_lookup(struct lpfc_hba *phba,
  **/
 static struct lpfc_iocbq *
 lpfc_sli_iocbq_lookup_by_tag(struct lpfc_hba *phba,
-			     struct lpfc_sli_ring *pring, uint16_t iotag)
+			     uint16_t iotag)
 {
 	struct lpfc_iocbq *cmd_iocb;
 
@@ -2755,7 +2752,7 @@ lpfc_sli_process_sol_iocb(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
 
 	/* Based on the iotag field, get the cmd IOCB from the txcmplq */
 	spin_lock_irqsave(&phba->hbalock, iflag);
-	cmdiocbp = lpfc_sli_iocbq_lookup(phba, pring, saveq);
+	cmdiocbp = lpfc_sli_iocbq_lookup(phba, saveq);
 	spin_unlock_irqrestore(&phba->hbalock, iflag);
 
 	if (cmdiocbp) {
@@ -3088,8 +3085,7 @@ lpfc_sli_handle_fast_ring_event(struct lpfc_hba *phba,
 				break;
 			}
 
-			cmdiocbq = lpfc_sli_iocbq_lookup(phba, pring,
-							 &rspiocbq);
+			cmdiocbq = lpfc_sli_iocbq_lookup(phba, &rspiocbq);
 			if (unlikely(!cmdiocbq))
 				break;
 			if (cmdiocbq->iocb_flag & LPFC_DRIVER_ABORTED)
@@ -3284,8 +3280,7 @@ lpfc_sli_sp_handle_rspiocb(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
 		case LPFC_ABORT_IOCB:
 			cmdiocbp = NULL;
 			if (irsp->ulpCommand != CMD_XRI_ABORTED_CX)
-				cmdiocbp = lpfc_sli_iocbq_lookup(phba, pring,
-								 saveq);
+				cmdiocbp = lpfc_sli_iocbq_lookup(phba, saveq);
 			if (cmdiocbp) {
 				/* Call the specified completion routine */
 				if (cmdiocbp->iocb_cmpl) {
@@ -11711,7 +11706,7 @@ lpfc_sli4_els_wcqe_to_rspiocbq(struct lpfc_hba *phba,
 	spin_lock_irqsave(&pring->ring_lock, iflags);
 	pring->stats.iocb_event++;
 	/* Look up the ELS command IOCB and create pseudo response IOCB */
-	cmdiocbq = lpfc_sli_iocbq_lookup_by_tag(phba, pring,
+	cmdiocbq = lpfc_sli_iocbq_lookup_by_tag(phba,
 				bf_get(lpfc_wcqe_c_request_tag, wcqe));
 	spin_unlock_irqrestore(&pring->ring_lock, iflags);
 
@@ -12302,7 +12297,7 @@ lpfc_sli4_fp_handle_fcp_wcqe(struct lpfc_hba *phba, struct lpfc_queue *cq,
 	/* Look up the FCP command IOCB and create pseudo response IOCB */
 	spin_lock_irqsave(&pring->ring_lock, iflags);
 	pring->stats.iocb_event++;
-	cmdiocbq = lpfc_sli_iocbq_lookup_by_tag(phba, pring,
+	cmdiocbq = lpfc_sli_iocbq_lookup_by_tag(phba,
 				bf_get(lpfc_wcqe_c_request_tag, wcqe));
 	spin_unlock_irqrestore(&pring->ring_lock, iflags);
 	if (unlikely(!cmdiocbq)) {
-- 
1.8.5.6

[PATCH 03/14] lpfc: Add config option 'lpfc_enable_stats'

[Hannes Reinecke]

Add a module option lpfc_enable_stats to enable or disable
HBA statistics. Gathering statistics is done in the hot path
during lpfc_scsi_cmd_iocb_cmpl() and require the host_lock,
so disabling them increases the overall performance.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/lpfc/lpfc.h      |  1 +
 drivers/scsi/lpfc/lpfc_attr.c | 10 ++++++++++
 drivers/scsi/lpfc/lpfc_scsi.c | 21 +++++++++------------
 3 files changed, 20 insertions(+), 12 deletions(-)

diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
index d5bd420..bb53b81 100644
--- a/drivers/scsi/lpfc/lpfc.h
+++ b/drivers/scsi/lpfc/lpfc.h
@@ -759,6 +759,7 @@ struct lpfc_hba {
 #define LPFC_FDMI_NO_SUPPORT	0	/* FDMI not supported */
 #define LPFC_FDMI_SUPPORT	1	/* FDMI supported? */
 	uint32_t cfg_enable_SmartSAN;
+	uint32_t cfg_enable_stats;
 	lpfc_vpd_t vpd;		/* vital product data */
 
 	struct pci_dev *pcidev;
diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c
index cfec2ec..241bbd6 100644
--- a/drivers/scsi/lpfc/lpfc_attr.c
+++ b/drivers/scsi/lpfc/lpfc_attr.c
@@ -4697,6 +4697,14 @@ LPFC_ATTR_RW(XLanePriority, 0, 0x0, 0x7f, "CS_CTL for Express Lane Feature.");
 LPFC_ATTR_R(enable_bg, 0, 0, 1, "Enable BlockGuard Support");
 
 /*
+# lpfc_enable_stats:  Enable HBA statistics
+#       0  = HBA statistics disabled
+#       1  = HBA statistics enabled (default)
+# Value range is [0,1]. Default value is 1
+*/
+LPFC_ATTR_RW(enable_stats, 1, 0, 1, "Enable HBA statistics.");
+
+/*
 # lpfc_fcp_look_ahead: Look ahead for completions in FCP start routine
 #       0  = disabled (default)
 #       1  = enabled
@@ -4876,6 +4884,7 @@ struct device_attribute *lpfc_hba_attrs[] = {
 	&dev_attr_lpfc_sriov_hw_max_virtfn,
 	&dev_attr_protocol,
 	&dev_attr_lpfc_xlane_supported,
+	&dev_attr_lpfc_enable_stats,
 	NULL,
 };
 
@@ -5866,6 +5875,7 @@ lpfc_get_cfgparam(struct lpfc_hba *phba)
 	lpfc_request_firmware_upgrade_init(phba, lpfc_req_fw_upgrade);
 	lpfc_suppress_link_up_init(phba, lpfc_suppress_link_up);
 	lpfc_iocb_cnt_init(phba, lpfc_iocb_cnt);
+	lpfc_enable_stats_init(phba, lpfc_enable_stats);
 	phba->cfg_enable_dss = 1;
 	return;
 }
diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index 3bd0be6..fde5b00 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -195,31 +195,24 @@ lpfc_sli4_set_rsp_sgl_last(struct lpfc_hba *phba,
  *
  * This function is called when there is a command completion and this
  * function updates the statistical data for the command completion.
+ * Needs to be called under shost->host_lock.
  **/
 static void
-lpfc_update_stats(struct lpfc_hba *phba, struct  lpfc_scsi_buf *lpfc_cmd)
+lpfc_update_stats(struct lpfc_hba *phba, struct lpfc_vport *vport,
+		  struct lpfc_scsi_buf *lpfc_cmd)
 {
 	struct lpfc_rport_data *rdata = lpfc_cmd->rdata;
 	struct lpfc_nodelist *pnode = rdata->pnode;
-	struct scsi_cmnd *cmd = lpfc_cmd->pCmd;
-	unsigned long flags;
-	struct Scsi_Host  *shost = cmd->device->host;
-	struct lpfc_vport *vport = (struct lpfc_vport *) shost->hostdata;
 	unsigned long latency;
 	int i;
 
-	if (cmd->result)
-		return;
-
 	latency = jiffies_to_msecs((long)jiffies - (long)lpfc_cmd->start_time);
 
-	spin_lock_irqsave(shost->host_lock, flags);
 	if (!vport->stat_data_enabled ||
 		vport->stat_data_blocked ||
 		!pnode ||
 		!pnode->lat_data ||
 		(phba->bucket_type == LPFC_NO_BUCKET)) {
-		spin_unlock_irqrestore(shost->host_lock, flags);
 		return;
 	}
 
@@ -239,7 +232,6 @@ lpfc_update_stats(struct lpfc_hba *phba, struct  lpfc_scsi_buf *lpfc_cmd)
 	}
 
 	pnode->lat_data[i].cmd_count++;
-	spin_unlock_irqrestore(shost->host_lock, flags);
 }
 
 /**
@@ -4100,7 +4092,12 @@ lpfc_scsi_cmd_iocb_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pIocbIn,
 				 scsi_get_resid(cmd));
 	}
 
-	lpfc_update_stats(phba, lpfc_cmd);
+	if (!cmd->result && phba->cfg_enable_stats) {
+		spin_lock_irqsave(shost->host_lock, flags);
+		lpfc_update_stats(phba, vport, lpfc_cmd);
+		spin_unlock_irqrestore(shost->host_lock, flags);
+	}
+
 	if (vport->cfg_max_scsicmpl_time &&
 	   time_after(jiffies, lpfc_cmd->start_time +
 		msecs_to_jiffies(vport->cfg_max_scsicmpl_time))) {
-- 
1.8.5.6

[PATCH 04/14] scsi: add 'abort_completions' host template flag

[Hannes Reinecke]

Some HBAs will always send a command completion on the original
command after a ABORT TASK TMF has been send.
For those it's pointless to wait for the TMF itself to finish,
but we can as well reset the timer for the failed command and
wait for the command completion to finish up the command.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/scsi_error.c | 7 +++++--
 include/scsi/scsi_host.h  | 6 ++++++
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/scsi_error.c b/drivers/scsi/scsi_error.c
index a8b610e..e4a1592 100644
--- a/drivers/scsi/scsi_error.c
+++ b/drivers/scsi/scsi_error.c
@@ -285,8 +285,11 @@ enum blk_eh_timer_return scsi_times_out(struct request *req)
 		rtn = host->hostt->eh_timed_out(scmd);
 
 	if (rtn == BLK_EH_NOT_HANDLED) {
-		if (!host->hostt->no_async_abort &&
-		    scsi_abort_command(scmd) == SUCCESS)
+		if (host->hostt->abort_completions &&
+		    scsi_try_to_abort_cmd(host->hostt, scmd) == SUCCESS)
+			return BLK_EH_RESET_TIMER;
+		else if (!host->hostt->no_async_abort &&
+			 scsi_abort_command(scmd) == SUCCESS)
 			return BLK_EH_NOT_HANDLED;
 
 		set_host_byte(scmd, DID_TIME_OUT);
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index 76e9d27..2af2397 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -449,6 +449,12 @@ struct scsi_host_template {
 	unsigned no_async_abort:1;
 
 	/*
+	 * True if aborts will trigger a command completion
+	 * of the failed command
+	 */
+	unsigned abort_completions:1;
+
+	/*
 	 * Countdown for host blocking with no commands outstanding.
 	 */
 	unsigned int max_host_blocked;
-- 
1.8.5.6

[PATCH 05/14] lpfc: do not wait for completion when aborting commands

[Hannes Reinecke]

The current synchronization between lpfc_scsi_cmd_iocb_cmpl()
and lpfc_abort_handler() requires us to take two spin_locks,
which kills performance for high end setups.

Looking at it, the synchronization is done for two reasons:
- Avoiding an abort for a command which is already completed
- Waiting for the completion to finish before proceeding

However, in both cases we rely on a completion to come in
for _both_ commands, the original and the abort TMF.
So there is no need to wait for the completion of the TMF;
once the TMF is underway we will be getting a completion for
both.

So we can check if the 'abort_completions' flag is set in the
host template and do away with these synchronisation points
altogether to speed up I/O processing.

At the same time the original code is left in place so one can
select the original behaviour by simply disabling the
'abort_completions' host template flag.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/lpfc/lpfc_scsi.c | 80 +++++++++++++++++++++++++------------------
 1 file changed, 47 insertions(+), 33 deletions(-)

diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index fde5b00..fab387c 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -4133,23 +4133,32 @@ lpfc_scsi_cmd_iocb_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pIocbIn,
 
 	lpfc_scsi_unprep_dma_buf(phba, lpfc_cmd);
 
+	/*
+	 * If the 'abort_completions' flag is set we can
+	 * release the scsi_buf before calling 'done', thereby
+	 * avoiding a race condition between aborts and scsi_done
+	 */
+	if (shost->hostt->abort_completions)
+		lpfc_release_scsi_buf(phba, lpfc_cmd);
+
 	/* The sdev is not guaranteed to be valid post scsi_done upcall. */
 	cmd->scsi_done(cmd);
 
-	spin_lock_irqsave(&phba->hbalock, flags);
-	lpfc_cmd->pCmd = NULL;
-	spin_unlock_irqrestore(&phba->hbalock, flags);
-
-	/*
-	 * If there is a thread waiting for command completion
-	 * wake up the thread.
-	 */
-	spin_lock_irqsave(shost->host_lock, flags);
-	if (lpfc_cmd->waitq)
-		wake_up(lpfc_cmd->waitq);
-	spin_unlock_irqrestore(shost->host_lock, flags);
+	if (!shost->hostt->abort_completions) {
+		spin_lock_irqsave(&phba->hbalock, flags);
+		lpfc_cmd->pCmd = NULL;
+		spin_unlock_irqrestore(&phba->hbalock, flags);
 
-	lpfc_release_scsi_buf(phba, lpfc_cmd);
+		/*
+		 * If there is a thread waiting for command completion
+		 * wake up the thread.
+		 */
+		spin_lock_irqsave(shost->host_lock, flags);
+		if (lpfc_cmd->waitq)
+			wake_up(lpfc_cmd->waitq);
+		spin_unlock_irqrestore(shost->host_lock, flags);
+		lpfc_release_scsi_buf(phba, lpfc_cmd);
+	}
 }
 
 /**
@@ -4677,7 +4686,6 @@ lpfc_abort_handler(struct scsi_cmnd *cmnd)
 	struct lpfc_sli_ring *pring_s4;
 	int ring_number, ret_val;
 	unsigned long flags, iflags;
-	DECLARE_WAIT_QUEUE_HEAD_ONSTACK(waitq);
 
 	status = fc_block_scsi_eh(cmnd);
 	if (status != 0 && status != SUCCESS)
@@ -4690,7 +4698,7 @@ lpfc_abort_handler(struct scsi_cmnd *cmnd)
 		lpfc_printf_vlog(vport, KERN_WARNING, LOG_FCP,
 			"3168 SCSI Layer abort requested I/O has been "
 			"flushed by LLD.\n");
-		return FAILED;
+		return shost->hostt->abort_completions ? SUCCESS : FAILED;
 	}
 
 	lpfc_cmd = (struct lpfc_scsi_buf *)cmnd->host_scribble;
@@ -4700,7 +4708,7 @@ lpfc_abort_handler(struct scsi_cmnd *cmnd)
 			 "2873 SCSI Layer I/O Abort Request IO CMPL Status "
 			 "x%x ID %d LUN %llu\n",
 			 SUCCESS, cmnd->device->id, cmnd->device->lun);
-		return SUCCESS;
+		return shost->hostt->abort_completions ? SUCCESS : FAILED;
 	}
 
 	iocb = &lpfc_cmd->cur_iocbq;
@@ -4710,7 +4718,7 @@ lpfc_abort_handler(struct scsi_cmnd *cmnd)
 		lpfc_printf_vlog(vport, KERN_WARNING, LOG_FCP,
 			"3169 SCSI Layer abort requested I/O has been "
 			"cancelled by LLD.\n");
-		return FAILED;
+		return shost->hostt->abort_completions ? SUCCESS : FAILED;
 	}
 	/*
 	 * If pCmd field of the corresponding lpfc_scsi_buf structure
@@ -4802,24 +4810,28 @@ lpfc_abort_handler(struct scsi_cmnd *cmnd)
 			&phba->sli.ring[LPFC_FCP_RING], HA_R0RE_REQ);
 
 wait_for_cmpl:
-	lpfc_cmd->waitq = &waitq;
-	/* Wait for abort to complete */
-	wait_event_timeout(waitq,
-			  (lpfc_cmd->pCmd != cmnd),
-			   msecs_to_jiffies(2*vport->cfg_devloss_tmo*1000));
+	if (!shost->hostt->abort_completions) {
+		DECLARE_WAIT_QUEUE_HEAD_ONSTACK(waitq);
 
-	spin_lock_irqsave(shost->host_lock, flags);
-	lpfc_cmd->waitq = NULL;
-	spin_unlock_irqrestore(shost->host_lock, flags);
+		lpfc_cmd->waitq = &waitq;
+		/* Wait for abort to complete */
+		wait_event_timeout(waitq,
+			(lpfc_cmd->pCmd != cmnd),
+			msecs_to_jiffies(2*vport->cfg_devloss_tmo*1000));
 
-	if (lpfc_cmd->pCmd == cmnd) {
-		ret = FAILED;
-		lpfc_printf_vlog(vport, KERN_ERR, LOG_FCP,
-				 "0748 abort handler timed out waiting "
-				 "for abortng I/O (xri:x%x) to complete: "
-				 "ret %#x, ID %d, LUN %llu\n",
-				 iocb->sli4_xritag, ret,
-				 cmnd->device->id, cmnd->device->lun);
+		spin_lock_irqsave(shost->host_lock, flags);
+		lpfc_cmd->waitq = NULL;
+		spin_unlock_irqrestore(shost->host_lock, flags);
+
+		if (lpfc_cmd->pCmd == cmnd) {
+			ret = FAILED;
+			lpfc_printf_vlog(vport, KERN_ERR, LOG_FCP,
+				"0748 abort handler timed out waiting "
+				"for abortng I/O (xri:x%x) to complete: "
+				"ret %#x, ID %d, LUN %llu\n",
+				iocb->sli4_xritag, ret,
+				cmnd->device->id, cmnd->device->lun);
+		}
 	}
 	goto out;
 
@@ -5927,6 +5939,7 @@ struct scsi_host_template lpfc_template = {
 	.vendor_id		= LPFC_NL_VENDOR_ID,
 	.change_queue_depth	= scsi_change_queue_depth,
 	.track_queue_depth	= 1,
+	.abort_completions	= 1,
 };
 
 struct scsi_host_template lpfc_vport_template = {
@@ -5950,4 +5963,5 @@ struct scsi_host_template lpfc_vport_template = {
 	.max_sectors		= 0xFFFF,
 	.change_queue_depth	= scsi_change_queue_depth,
 	.track_queue_depth	= 1,
+	.abort_completions	= 1,
 };
-- 
1.8.5.6

[PATCH 06/14] lpfc: display manual queue assignment

[Hannes Reinecke]

If no interrupt affinity is requested the driver should still
list the queue-to-interrupt mapping so that userspace tools
can setup the correct interrupt affinity.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/lpfc/lpfc_attr.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c
index 241bbd6..801b2b0 100644
--- a/drivers/scsi/lpfc/lpfc_attr.c
+++ b/drivers/scsi/lpfc/lpfc_attr.c
@@ -4306,7 +4306,7 @@ lpfc_fcp_cpu_map_show(struct device *dev, struct device_attribute *attr,
 	struct lpfc_vport *vport = (struct lpfc_vport *)shost->hostdata;
 	struct lpfc_hba   *phba = vport->phba;
 	struct lpfc_vector_map_info *cpup;
-	int  len = 0;
+	int  len = 0, idx;
 
 	if ((phba->sli_rev != LPFC_SLI_REV4) ||
 	    (phba->intr_type != MSIX))
@@ -4315,8 +4315,14 @@ lpfc_fcp_cpu_map_show(struct device *dev, struct device_attribute *attr,
 	switch (phba->cfg_fcp_cpu_map) {
 	case 0:
 		len += snprintf(buf + len, PAGE_SIZE-len,
-				"fcp_cpu_map: No mapping (%d)\n",
+				"fcp_cpu_map: Manual mapping (%d)\n",
 				phba->cfg_fcp_cpu_map);
+		for (idx = 0; idx < phba->cfg_fcp_io_channel; idx++) {
+			len += snprintf(buf + len, PAGE_SIZE-len,
+					"io_chan %02d IRQ %d\n",
+					phba->sli4_hba.msix_entries[idx].entry,
+					phba->sli4_hba.msix_entries[idx].vector);
+		}
 		return len;
 	case 1:
 		len += snprintf(buf + len, PAGE_SIZE-len,
-- 
1.8.5.6

[PATCH 07/14] scsi: modify can_queue after calling mq_allocate()

[Hannes Reinecke]

When allocating a multiqueue structure the number of tags might
have been reduced due to memory pressure.
So update can_queue with the new value.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/scsi_lib.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index c71344a..e8fea8e 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2099,6 +2099,7 @@ struct request_queue *scsi_mq_alloc_queue(struct scsi_device *sdev)
 int scsi_mq_setup_tags(struct Scsi_Host *shost)
 {
 	unsigned int cmd_size, sgl_size, tbl_size;
+	int ret;
 
 	tbl_size = shost->sg_tablesize;
 	if (tbl_size > SG_CHUNK_SIZE)
@@ -2119,7 +2120,12 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
 		BLK_ALLOC_POLICY_TO_MQ_FLAG(shost->hostt->tag_alloc_policy);
 	shost->tag_set.driver_data = shost;
 
-	return blk_mq_alloc_tag_set(&shost->tag_set);
+	ret = blk_mq_alloc_tag_set(&shost->tag_set);
+	if (!ret) {
+		/* queue depth might have been adjusted */
+		shost->can_queue = shost->tag_set.queue_depth;
+	}
+	return ret;
 }
 
 void scsi_mq_destroy_tags(struct Scsi_Host *shost)
-- 
1.8.5.6

[PATCH 08/14] blk-mq: add blk_mq_resize_tag_set()

[Hannes Reinecke]

Add a function to resize the number of tags in an existing tag set.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 block/blk-mq.c         | 15 +++++++++++++++
 include/linux/blk-mq.h |  1 +
 2 files changed, 16 insertions(+)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 29cbc1b..e2afe45 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2385,6 +2385,21 @@ void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues)
 }
 EXPORT_SYMBOL_GPL(blk_mq_update_nr_hw_queues);
 
+int blk_mq_resize_tag_set(struct blk_mq_tag_set *set, int new_tags)
+{
+	int i, ret = 0;
+
+	for (i = 0; i < set->nr_hw_queues; i++) {
+		if (!set->tags[i])
+			continue;
+		ret = blk_mq_tag_update_depth(set->tags[i], new_tags);
+		if (ret)
+			break;
+	}
+	return ret;
+}
+EXPORT_SYMBOL(blk_mq_resize_tag_set);
+
 void blk_mq_disable_hotplug(void)
 {
 	mutex_lock(&all_q_mutex);
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index 2498fdf..8b603b6 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -181,6 +181,7 @@ void blk_mq_unregister_disk(struct gendisk *);
 
 int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set);
 void blk_mq_free_tag_set(struct blk_mq_tag_set *set);
+int blk_mq_resize_tag_set(struct blk_mq_tag_set *set, int new_tags);
 
 void blk_mq_flush_plug_list(struct blk_plug *plug, bool from_schedule);
 
-- 
1.8.5.6

[PATCH 09/14] scsi: Implement scsi_mq_resize_tags()

[Hannes Reinecke]

Add a function to resize the tagspace for a scsi host.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/scsi_lib.c  | 6 ++++++
 include/scsi/scsi_host.h | 1 +
 2 files changed, 7 insertions(+)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index e8fea8e..56c2724 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2133,6 +2133,12 @@ void scsi_mq_destroy_tags(struct Scsi_Host *shost)
 	blk_mq_free_tag_set(&shost->tag_set);
 }
 
+int scsi_mq_resize_tags(struct Scsi_Host *shost, int max_tags)
+{
+	return blk_mq_resize_tag_set(&shost->tag_set, max_tags);
+}
+EXPORT_SYMBOL_GPL(scsi_mq_resize_tags);
+
 /*
  * Function:    scsi_block_requests()
  *
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index 2af2397..7cdfbd7 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -826,6 +826,7 @@ static inline int scsi_host_scan_allowed(struct Scsi_Host *shost)
 
 extern void scsi_unblock_requests(struct Scsi_Host *);
 extern void scsi_block_requests(struct Scsi_Host *);
+extern int scsi_mq_resize_tags(struct Scsi_Host *, int);
 
 struct class_container;
 
-- 
1.8.5.6

[PATCH 10/14] lpfc: use bitflag for exch_busy

[Hannes Reinecke]

Signed-off-by: Hannes Reinecke <hare@suse.de>
---
 drivers/scsi/lpfc/lpfc_scsi.c | 19 +++++++++++--------
 drivers/scsi/lpfc/lpfc_scsi.h |  4 +++-
 drivers/scsi/lpfc/lpfc_sli.c  |  5 ++++-
 3 files changed, 18 insertions(+), 10 deletions(-)

diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index fab387c..4354713 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -556,7 +556,7 @@ lpfc_sli4_fcp_xri_aborted(struct lpfc_hba *phba,
 		&phba->sli4_hba.lpfc_abts_scsi_buf_list, list) {
 		if (psb->cur_iocbq.sli4_xritag == xri) {
 			list_del(&psb->list);
-			psb->exch_busy = 0;
+			clear_bit(LPFC_CMD_EXCH_BUSY, &psb->flags);
 			psb->status = IOSTAT_SUCCESS;
 			spin_unlock(
 				&phba->sli4_hba.abts_scsi_buf_list_lock);
@@ -588,7 +588,7 @@ lpfc_sli4_fcp_xri_aborted(struct lpfc_hba *phba,
 		if (iocbq->sli4_xritag != xri)
 			continue;
 		psb = container_of(iocbq, struct lpfc_scsi_buf, cur_iocbq);
-		psb->exch_busy = 0;
+		clear_bit(LPFC_CMD_EXCH_BUSY, &psb->flags);
 		spin_unlock_irqrestore(&phba->hbalock, iflag);
 		if (!list_empty(&pring->txq))
 			lpfc_worker_wake_up(phba);
@@ -675,10 +675,12 @@ lpfc_sli4_post_scsi_sgl_list(struct lpfc_hba *phba,
 						psb->cur_iocbq.sli4_xritag);
 				if (status) {
 					/* failure, put on abort scsi list */
-					psb->exch_busy = 1;
+					set_bit(LPFC_CMD_EXCH_BUSY,
+						&psb->flags);
 				} else {
 					/* success, put on SCSI buffer list */
-					psb->exch_busy = 0;
+					clear_bit(LPFC_CMD_EXCH_BUSY,
+						  &psb->flags);
 					psb->status = IOSTAT_SUCCESS;
 					num_posted++;
 				}
@@ -708,10 +710,10 @@ lpfc_sli4_post_scsi_sgl_list(struct lpfc_hba *phba,
 					 struct lpfc_scsi_buf, list);
 			if (status) {
 				/* failure, put on abort scsi list */
-				psb->exch_busy = 1;
+				set_bit(LPFC_CMD_EXCH_BUSY, &psb->flags);
 			} else {
 				/* success, put on SCSI buffer list */
-				psb->exch_busy = 0;
+				clear_bit(LPFC_CMD_EXCH_BUSY, &psb->flags);
 				psb->status = IOSTAT_SUCCESS;
 				num_posted++;
 			}
@@ -1089,7 +1091,7 @@ lpfc_release_scsi_buf_s4(struct lpfc_hba *phba, struct lpfc_scsi_buf *psb)
 	psb->nonsg_phys = 0;
 	psb->prot_seg_cnt = 0;
 
-	if (psb->exch_busy) {
+	if (test_bit(LPFC_CMD_EXCH_BUSY, &psb->flags)) {
 		spin_lock_irqsave(&phba->sli4_hba.abts_scsi_buf_list_lock,
 					iflag);
 		psb->pCmd = NULL;
@@ -3923,7 +3925,8 @@ lpfc_scsi_cmd_iocb_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pIocbIn,
 	lpfc_cmd->result = (pIocbOut->iocb.un.ulpWord[4] & IOERR_PARAM_MASK);
 	lpfc_cmd->status = pIocbOut->iocb.ulpStatus;
 	/* pick up SLI4 exhange busy status from HBA */
-	lpfc_cmd->exch_busy = pIocbOut->iocb_flag & LPFC_EXCHANGE_BUSY;
+	if (pIocbOut->iocb_flag & LPFC_EXCHANGE_BUSY)
+		set_bit(LPFC_CMD_EXCH_BUSY, &lpfc_cmd->flags);
 
 #ifdef CONFIG_SCSI_LPFC_DEBUG_FS
 	if (lpfc_cmd->prot_data_type) {
diff --git a/drivers/scsi/lpfc/lpfc_scsi.h b/drivers/scsi/lpfc/lpfc_scsi.h
index 18b9260..a3658ce 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.h
+++ b/drivers/scsi/lpfc/lpfc_scsi.h
@@ -134,7 +134,7 @@ struct lpfc_scsi_buf {
 
 	uint32_t timeout;
 
-	uint16_t exch_busy;     /* SLI4 hba reported XB on complete WCQE */
+	uint16_t xx_exch_busy;     /* SLI4 hba reported XB on complete WCQE */
 	uint16_t status;	/* From IOCB Word 7- ulpStatus */
 	uint32_t result;	/* From IOCB Word 4. */
 
@@ -143,6 +143,8 @@ struct lpfc_scsi_buf {
 				 * to dma_unmap_sg. */
 	uint32_t prot_seg_cnt;  /* seg_cnt's counterpart for protection data */
 
+	unsigned long flags;
+#define LPFC_CMD_EXCH_BUSY 1
 	dma_addr_t nonsg_phys;	/* Non scatter-gather physical address. */
 
 	/*
diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index 5e38a35..52b1fa1 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -10442,7 +10442,10 @@ lpfc_sli_wake_iocb_wait(struct lpfc_hba *phba,
 		!(cmdiocbq->iocb_flag & LPFC_IO_LIBDFC)) {
 		lpfc_cmd = container_of(cmdiocbq, struct lpfc_scsi_buf,
 			cur_iocbq);
-		lpfc_cmd->exch_busy = rspiocbq->iocb_flag & LPFC_EXCHANGE_BUSY;
+		if (rspiocbq->iocb_flag & LPFC_EXCHANGE_BUSY)
+			set_bit(LPFC_CMD_EXCH_BUSY, &lpfc_cmd->flags);
+		else
+			clear_bit(LPFC_CMD_EXCH_BUSY, &lpfc_cmd->flags);
 	}
 
 	pdone_q = cmdiocbq->context_un.wait_queue;
-- 
1.8.5.6

[PATCH 11/14] lpfc: set LPFC_CMD_ABORTED when a command is put on the aborted list

[Hannes Reinecke]

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/lpfc/lpfc_init.c | 1 +
 drivers/scsi/lpfc/lpfc_scsi.c | 3 +++
 drivers/scsi/lpfc/lpfc_scsi.h | 1 +
 3 files changed, 5 insertions(+)

diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index b43f7ac..d2a6302 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -1028,6 +1028,7 @@ lpfc_hba_down_post_s4(struct lpfc_hba *phba)
 	spin_unlock_irq(&phba->hbalock);
 
 	list_for_each_entry_safe(psb, psb_next, &aborts, list) {
+		clear_bit(LPFC_CMD_ABORTED, &psb->flags);
 		psb->pCmd = NULL;
 		psb->status = IOSTAT_SUCCESS;
 	}
diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index 4354713..3111a9d 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -556,6 +556,7 @@ lpfc_sli4_fcp_xri_aborted(struct lpfc_hba *phba,
 		&phba->sli4_hba.lpfc_abts_scsi_buf_list, list) {
 		if (psb->cur_iocbq.sli4_xritag == xri) {
 			list_del(&psb->list);
+			clear_bit(LPFC_CMD_ABORTED, &psb->flags);
 			clear_bit(LPFC_CMD_EXCH_BUSY, &psb->flags);
 			psb->status = IOSTAT_SUCCESS;
 			spin_unlock(
@@ -1092,6 +1093,8 @@ lpfc_release_scsi_buf_s4(struct lpfc_hba *phba, struct lpfc_scsi_buf *psb)
 	psb->prot_seg_cnt = 0;
 
 	if (test_bit(LPFC_CMD_EXCH_BUSY, &psb->flags)) {
+		if (!test_and_set_bit(LPFC_CMD_ABORTED, &psb->flags))
+			return;
 		spin_lock_irqsave(&phba->sli4_hba.abts_scsi_buf_list_lock,
 					iflag);
 		psb->pCmd = NULL;
diff --git a/drivers/scsi/lpfc/lpfc_scsi.h b/drivers/scsi/lpfc/lpfc_scsi.h
index a3658ce..4e8f0bd 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.h
+++ b/drivers/scsi/lpfc/lpfc_scsi.h
@@ -145,6 +145,7 @@ struct lpfc_scsi_buf {
 
 	unsigned long flags;
 #define LPFC_CMD_EXCH_BUSY 1
+#define LPFC_CMD_ABORTED   2
 	dma_addr_t nonsg_phys;	/* Non scatter-gather physical address. */
 
 	/*
-- 
1.8.5.6

[PATCH 12/14] lpfc: move to array based command allocation for SLI-4

[Hannes Reinecke]

On high-end arrays the list-based command allocation becomes a
bottleneck as the lock needs to be taken for each command allocation.
On the other hand the current blk-mq/scsi-mq infrastructure ensures
that a tag is never being reused. So this patch moves the command
allocation to an array-based structure, indexed by the command tag.
With this we can avoid taking a lock during command allocation, and
just mark the command as 'in-use' by setting a flag.
This allows for proper house-keeping in case the HBA needs to be
resetted.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/lpfc/lpfc.h      |  1 +
 drivers/scsi/lpfc/lpfc_init.c | 53 +++++++++++++++++++++++++--
 drivers/scsi/lpfc/lpfc_scsi.c | 84 +++++++++++++++++++++++++++++++++++++------
 drivers/scsi/lpfc/lpfc_scsi.h |  7 ++--
 4 files changed, 128 insertions(+), 17 deletions(-)

diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
index bb53b81..289cc50 100644
--- a/drivers/scsi/lpfc/lpfc.h
+++ b/drivers/scsi/lpfc/lpfc.h
@@ -840,6 +840,7 @@ struct lpfc_hba {
 	uint64_t bg_reftag_err_cnt;
 
 	/* fastpath list. */
+	struct lpfc_scsi_buf **lpfc_scsi_buf_arr;
 	spinlock_t scsi_buf_list_get_lock;  /* SCSI buf alloc list lock */
 	spinlock_t scsi_buf_list_put_lock;  /* SCSI buf free list lock */
 	struct list_head lpfc_scsi_buf_list_get;
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index d2a6302..55ed075 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -3071,6 +3071,20 @@ lpfc_scsi_free(struct lpfc_hba *phba)
 	}
 	spin_unlock(&phba->scsi_buf_list_get_lock);
 
+	if (phba->lpfc_scsi_buf_arr) {
+		int idx;
+		for (idx = 0; idx < phba->cfg_hba_queue_depth; idx++) {
+			sb = phba->lpfc_scsi_buf_arr[idx];
+			if (!sb)
+				continue;
+			clear_bit(LPFC_CMD_QUEUED, &sb->flags);
+			list_del(&sb->list);
+			pci_pool_free(phba->lpfc_scsi_dma_buf_pool, sb->data,
+				      sb->dma_handle);
+			kfree(sb);
+			phba->total_scsi_bufs--;
+		}
+	}
 	/* Release all the lpfc_iocbq entries maintained by this host. */
 	list_for_each_entry_safe(io, io_next, &phba->lpfc_iocb_list, list) {
 		list_del(&io->list);
@@ -3212,6 +3226,18 @@ lpfc_sli4_xri_sgl_update(struct lpfc_hba *phba)
 			phba->sli4_hba.scsi_xri_cnt,
 			phba->sli4_hba.scsi_xri_max);
 
+	if (phba->lpfc_scsi_buf_arr) {
+		for (i = 0; i < phba->cfg_hba_queue_depth; i++) {
+			psb = phba->lpfc_scsi_buf_arr[i];
+			if (psb) {
+				if (test_and_set_bit(LPFC_CMD_QUEUED,
+						     &psb->flags))
+					continue;
+				list_add_tail(&psb->list, &scsi_sgl_list);
+			}
+		}
+	}
+
 	spin_lock_irq(&phba->scsi_buf_list_get_lock);
 	spin_lock(&phba->scsi_buf_list_put_lock);
 	list_splice_init(&phba->lpfc_scsi_buf_list_get, &scsi_sgl_list);
@@ -3228,6 +3254,9 @@ lpfc_sli4_xri_sgl_update(struct lpfc_hba *phba)
 			list_remove_head(&scsi_sgl_list, psb,
 					 struct lpfc_scsi_buf, list);
 			if (psb) {
+				clear_bit(LPFC_CMD_QUEUED, &psb->flags);
+				if (phba->lpfc_scsi_buf_arr)
+					phba->lpfc_scsi_buf_arr[psb->iotag] = NULL;
 				pci_pool_free(phba->lpfc_scsi_dma_buf_pool,
 					      psb->data, psb->dma_handle);
 				kfree(psb);
@@ -3258,8 +3287,17 @@ lpfc_sli4_xri_sgl_update(struct lpfc_hba *phba)
 	list_splice_init(&scsi_sgl_list, &phba->lpfc_scsi_buf_list_get);
 	INIT_LIST_HEAD(&phba->lpfc_scsi_buf_list_put);
 	spin_unlock(&phba->scsi_buf_list_put_lock);
-	spin_unlock_irq(&phba->scsi_buf_list_get_lock);
 
+	if (phba->lpfc_scsi_buf_arr) {
+		for (i = 0; i < phba->cfg_hba_queue_depth; i++) {
+			psb = phba->lpfc_scsi_buf_arr[i];
+			if (psb) {
+				clear_bit(LPFC_CMD_QUEUED, &psb->flags);
+				list_del_init(&psb->list);
+			}
+		}
+	}
+	spin_unlock_irq(&phba->scsi_buf_list_get_lock);
 	return 0;
 
 out_free_mem:
@@ -3329,7 +3367,8 @@ lpfc_create_port(struct lpfc_hba *phba, int instance, struct device *dev)
 	 * scsi_add_host will fail. This will be adjusted later based on the
 	 * max xri value determined in hba setup.
 	 */
-	shost->can_queue = phba->cfg_hba_queue_depth - 10;
+	shost->can_queue = (phba->cfg_hba_queue_depth - 10) /
+		phba->cfg_fcp_io_channel;
 	if (dev != &phba->pcidev->dev) {
 		shost->transportt = lpfc_vport_transport_template;
 		vport->port_type = LPFC_NPIV_PORT;
@@ -3338,6 +3377,13 @@ lpfc_create_port(struct lpfc_hba *phba, int instance, struct device *dev)
 		vport->port_type = LPFC_PHYSICAL_PORT;
 	}
 
+	if (shost_use_blk_mq(shost) && phba->sli_rev == LPFC_SLI_REV4) {
+		phba->lpfc_scsi_buf_arr = kzalloc(sizeof(struct lpfc_scsi_buf *) *
+						  phba->cfg_hba_queue_depth, GFP_KERNEL);
+		if (!phba->lpfc_scsi_buf_arr)
+			goto out_put_shost;
+	}
+
 	/* Initialize all internally managed lists. */
 	INIT_LIST_HEAD(&vport->fc_nodes);
 	INIT_LIST_HEAD(&vport->rcv_buffer_list);
@@ -6312,7 +6358,8 @@ lpfc_post_init_setup(struct lpfc_hba *phba)
 	 * adjust the value of can_queue.
 	 */
 	shost = pci_get_drvdata(phba->pcidev);
-	shost->can_queue = phba->cfg_hba_queue_depth - 10;
+	shost->can_queue = (phba->cfg_hba_queue_depth - 10) /
+		phba->cfg_fcp_io_channel;
 	if (phba->sli3_options & LPFC_SLI3_BG_ENABLED)
 		lpfc_setup_bg(phba, shost);
 
diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index 3111a9d..a3eb5ff 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -746,9 +746,19 @@ int
 lpfc_sli4_repost_scsi_sgl_list(struct lpfc_hba *phba)
 {
 	LIST_HEAD(post_sblist);
-	int num_posted, rc = 0;
+	int i, num_posted, rc = 0;
 
 	/* get all SCSI buffers need to repost to a local list */
+	if (phba->lpfc_scsi_buf_arr) {
+		struct lpfc_scsi_buf *psb;
+
+		for (i = 0; i < phba->cfg_hba_queue_depth; i++) {
+			psb = phba->lpfc_scsi_buf_arr[i];
+			if (psb &&
+			    !test_and_set_bit(LPFC_CMD_QUEUED, &psb->flags))
+				list_add(&psb->list, &post_sblist);
+		}
+	}
 	spin_lock_irq(&phba->scsi_buf_list_get_lock);
 	spin_lock(&phba->scsi_buf_list_put_lock);
 	list_splice_init(&phba->lpfc_scsi_buf_list_get, &post_sblist);
@@ -913,6 +923,12 @@ lpfc_new_scsi_buf_s4(struct lpfc_vport *vport, int num_to_alloc)
 		psb->dma_phys_bpl = pdma_phys_bpl;
 
 		/* add the scsi buffer to a post list */
+		if (phba->lpfc_scsi_buf_arr) {
+			int idx = phba->total_scsi_bufs + bcnt;
+			psb->iotag = idx;
+			phba->lpfc_scsi_buf_arr[idx] = psb;
+			set_bit(LPFC_CMD_QUEUED, &psb->flags);
+		}
 		list_add_tail(&psb->list, &post_sblist);
 		spin_lock_irq(&phba->scsi_buf_list_get_lock);
 		phba->sli4_hba.scsi_xri_cnt++;
@@ -1105,9 +1121,13 @@ lpfc_release_scsi_buf_s4(struct lpfc_hba *phba, struct lpfc_scsi_buf *psb)
 	} else {
 		psb->pCmd = NULL;
 		psb->cur_iocbq.iocb_flag = LPFC_IO_FCP;
-		spin_lock_irqsave(&phba->scsi_buf_list_put_lock, iflag);
-		list_add_tail(&psb->list, &phba->lpfc_scsi_buf_list_put);
-		spin_unlock_irqrestore(&phba->scsi_buf_list_put_lock, iflag);
+		if (phba->lpfc_scsi_buf_arr)
+			clear_bit(LPFC_CMD_QUEUED, &psb->flags);
+		else {
+			spin_lock_irqsave(&phba->scsi_buf_list_put_lock, iflag);
+			list_add_tail(&psb->list, &phba->lpfc_scsi_buf_list_put);
+			spin_unlock_irqrestore(&phba->scsi_buf_list_put_lock, iflag);
+		}
 	}
 }
 
@@ -4533,7 +4553,7 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd)
 	struct lpfc_hba   *phba = vport->phba;
 	struct lpfc_rport_data *rdata;
 	struct lpfc_nodelist *ndlp;
-	struct lpfc_scsi_buf *lpfc_cmd;
+	struct lpfc_scsi_buf *lpfc_cmd = NULL;
 	struct fc_rport *rport = starget_to_rport(scsi_target(cmnd->device));
 	int err;
 
@@ -4566,7 +4586,28 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd)
 	if (atomic_read(&ndlp->cmd_pending) >= ndlp->cmd_qdepth)
 		goto out_tgt_busy;
 
-	lpfc_cmd = lpfc_get_scsi_buf(phba, ndlp);
+	if (phba->lpfc_scsi_buf_arr) {
+		u32 tag = blk_mq_unique_tag(cmnd->request);
+		u16 hwq = blk_mq_unique_tag_to_hwq(tag);
+		u16 idx = blk_mq_unique_tag_to_tag(tag);
+
+		idx = idx * phba->cfg_fcp_io_channel + hwq;
+		if (idx >= phba->cfg_hba_queue_depth) {
+			lpfc_printf_vlog(vport, KERN_ERR, LOG_SCSI_CMD,
+					 "9034 iotag %x too large\n", idx);
+		} else
+			lpfc_cmd = phba->lpfc_scsi_buf_arr[idx];
+		if (!lpfc_cmd)
+			lpfc_printf_vlog(vport, KERN_ERR, LOG_SCSI_CMD,
+					 "9035 iotag %x invalid\n", idx);
+		else if (test_and_set_bit(LPFC_CMD_QUEUED, &lpfc_cmd->flags)) {
+			lpfc_printf_vlog(vport, KERN_INFO, LOG_SCSI_CMD,
+					 "9036 iotag %x hwq %x busy\n",
+					 lpfc_cmd->iotag, hwq);
+			lpfc_cmd = NULL;
+		}
+	} else
+		lpfc_cmd = lpfc_get_scsi_buf(phba, ndlp);
 	if (lpfc_cmd == NULL) {
 		lpfc_rampdown_queue_depth(phba);
 
@@ -4962,7 +5003,7 @@ lpfc_send_taskmgmt(struct lpfc_vport *vport, struct lpfc_rport_data *rdata,
 		    uint8_t task_mgmt_cmd)
 {
 	struct lpfc_hba   *phba = vport->phba;
-	struct lpfc_scsi_buf *lpfc_cmd;
+	struct lpfc_scsi_buf *lpfc_cmd = NULL;
 	struct lpfc_iocbq *iocbq;
 	struct lpfc_iocbq *iocbqrsp;
 	struct lpfc_nodelist *pnode = rdata->pnode;
@@ -4972,7 +5013,21 @@ lpfc_send_taskmgmt(struct lpfc_vport *vport, struct lpfc_rport_data *rdata,
 	if (!pnode || !NLP_CHK_NODE_ACT(pnode))
 		return FAILED;
 
-	lpfc_cmd = lpfc_get_scsi_buf(phba, rdata->pnode);
+	if (phba->lpfc_scsi_buf_arr) {
+		int idx;
+		for (idx = 0; idx < phba->cfg_hba_queue_depth; idx++) {
+			lpfc_cmd = phba->lpfc_scsi_buf_arr[idx];
+			if (test_and_set_bit(LPFC_CMD_QUEUED,
+					     &lpfc_cmd->flags)) {
+				ret = 0;
+				break;
+			}
+			ret = -EBUSY;
+		}
+		if (ret < 0)
+			lpfc_cmd = NULL;
+	} else
+		lpfc_cmd = lpfc_get_scsi_buf(phba, rdata->pnode);
 	if (lpfc_cmd == NULL)
 		return FAILED;
 	lpfc_cmd->timeout = phba->cfg_task_mgmt_tmo;
@@ -5483,10 +5538,12 @@ lpfc_slave_alloc(struct scsi_device *sdev)
 	 * extra.  This list of scsi bufs exists for the lifetime of the driver.
 	 */
 	total = phba->total_scsi_bufs;
-	num_to_alloc = vport->cfg_lun_queue_depth + 2;
+	num_to_alloc = (vport->cfg_lun_queue_depth + 2) *
+		phba->cfg_fcp_io_channel;
 
 	/* If allocated buffers are enough do nothing */
-	if ((sdev_cnt * (vport->cfg_lun_queue_depth + 2)) < total)
+	if (!shost_use_blk_mq(sdev->host) &&
+	    (sdev_cnt * (vport->cfg_lun_queue_depth + 2)) < total)
 		return 0;
 
 	/* Allow some exchanges to be available always to complete discovery */
@@ -5514,8 +5571,13 @@ lpfc_slave_alloc(struct scsi_device *sdev)
 					 "Allocated %d buffers.\n",
 					 num_to_alloc, num_allocated);
 	}
-	if (num_allocated > 0)
+	if (num_allocated > 0) {
 		phba->total_scsi_bufs += num_allocated;
+		if (shost_use_blk_mq(sdev->host)) {
+			int num_tags = num_allocated / phba->cfg_fcp_io_channel;
+			scsi_mq_resize_tags(sdev->host, num_tags);
+		}
+	}
 	return 0;
 }
 
diff --git a/drivers/scsi/lpfc/lpfc_scsi.h b/drivers/scsi/lpfc/lpfc_scsi.h
index 4e8f0bd..a07341e 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.h
+++ b/drivers/scsi/lpfc/lpfc_scsi.h
@@ -134,7 +134,7 @@ struct lpfc_scsi_buf {
 
 	uint32_t timeout;
 
-	uint16_t xx_exch_busy;     /* SLI4 hba reported XB on complete WCQE */
+	uint16_t iotag;
 	uint16_t status;	/* From IOCB Word 7- ulpStatus */
 	uint32_t result;	/* From IOCB Word 4. */
 
@@ -144,8 +144,9 @@ struct lpfc_scsi_buf {
 	uint32_t prot_seg_cnt;  /* seg_cnt's counterpart for protection data */
 
 	unsigned long flags;
-#define LPFC_CMD_EXCH_BUSY 1
-#define LPFC_CMD_ABORTED   2
+#define LPFC_CMD_EXCH_BUSY 0
+#define LPFC_CMD_ABORTED   1
+#define LPFC_CMD_QUEUED    2
 	dma_addr_t nonsg_phys;	/* Non scatter-gather physical address. */
 
 	/*
-- 
1.8.5.6

[PATCH 13/14] lpfc: LPFC_CMD_RRQ_ACTIVE flag to mark commands

[Hannes Reinecke]

During aborts the xri might not be valid, so this patch
introduces a new command flag 'LPFC_CMD_RRQ_ACTIVE' which
signals that an RRQ is outstanding for this command and
the xri cannot be used ATM.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/lpfc/lpfc_scsi.c |  7 +++++++
 drivers/scsi/lpfc/lpfc_scsi.h |  1 +
 drivers/scsi/lpfc/lpfc_sli.c  | 43 ++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 50 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index a3eb5ff..348cae6 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -571,6 +571,7 @@ lpfc_sli4_fcp_xri_aborted(struct lpfc_hba *phba,
 			if (ndlp) {
 				lpfc_set_rrq_active(phba, ndlp,
 					psb->cur_iocbq.sli4_lxritag, rxid, 1);
+				set_bit(LPFC_CMD_RRQ_ACTIVE, &psb->flags);
 				lpfc_sli4_abts_err_handler(phba, ndlp, axri);
 			}
 			lpfc_release_scsi_buf_s4(phba, psb);
@@ -4093,6 +4094,7 @@ lpfc_scsi_cmd_iocb_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pIocbIn,
 				lpfc_set_rrq_active(phba, pnode,
 					lpfc_cmd->cur_iocbq.sli4_lxritag,
 					0, 0);
+				set_bit(LPFC_CMD_RRQ_ACTIVE, &lpfc_cmd->flags);
 			}
 		/* else: fall through */
 		default:
@@ -4605,6 +4607,11 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd)
 					 "9036 iotag %x hwq %x busy\n",
 					 lpfc_cmd->iotag, hwq);
 			lpfc_cmd = NULL;
+		} else if (test_bit(LPFC_CMD_RRQ_ACTIVE, &lpfc_cmd->flags)) {
+			lpfc_printf_vlog(vport, KERN_INFO, LOG_SCSI_CMD,
+					 "9037 iotag %x hwq %x rrq active\n",
+					 lpfc_cmd->iotag, hwq);
+			lpfc_cmd = NULL;
 		}
 	} else
 		lpfc_cmd = lpfc_get_scsi_buf(phba, ndlp);
diff --git a/drivers/scsi/lpfc/lpfc_scsi.h b/drivers/scsi/lpfc/lpfc_scsi.h
index a07341e..017ae28 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.h
+++ b/drivers/scsi/lpfc/lpfc_scsi.h
@@ -147,6 +147,7 @@ struct lpfc_scsi_buf {
 #define LPFC_CMD_EXCH_BUSY 0
 #define LPFC_CMD_ABORTED   1
 #define LPFC_CMD_QUEUED    2
+#define LPFC_CMD_RRQ_ACTIVE 3
 	dma_addr_t nonsg_phys;	/* Non scatter-gather physical address. */
 
 	/*
diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index 52b1fa1..e5cd212 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -658,6 +658,26 @@ lpfc_clr_rrq_active(struct lpfc_hba *phba,
 		goto out;
 
 	if (test_and_clear_bit(xritag, ndlp->active_rrqs_xri_bitmap)) {
+		unsigned long iflag;
+		int i;
+		struct lpfc_iocbq *iocbq;
+		struct lpfc_scsi_buf *psb;
+
+		spin_lock_irqsave(&phba->hbalock, iflag);
+		for (i = 1; i <= phba->sli.last_iotag; i++) {
+			iocbq = phba->sli.iocbq_lookup[i];
+			if (!(iocbq->iocb_flag & LPFC_IO_FCP) ||
+			    (iocbq->iocb_flag & LPFC_IO_LIBDFC))
+				continue;
+			if (iocbq->sli4_lxritag != rrq->xritag)
+				continue;
+
+			psb = container_of(iocbq, struct lpfc_scsi_buf,
+					   cur_iocbq);
+			clear_bit(LPFC_CMD_RRQ_ACTIVE, &psb->flags);
+			break;
+		}
+		spin_unlock_irqrestore(&phba->hbalock, iflag);
 		rrq->send_rrq = 0;
 		rrq->xritag = 0;
 		rrq->rrq_stop_time = 0;
@@ -15271,9 +15291,30 @@ lpfc_sli4_seq_abort_rsp(struct lpfc_vport *vport,
 	else
 		xri = rxid;
 	lxri = lpfc_sli4_xri_inrange(phba, xri);
-	if (lxri != NO_XRI)
+	if (lxri != NO_XRI) {
+		unsigned long iflag;
+		int i;
+		struct lpfc_iocbq *iocbq;
+		struct lpfc_scsi_buf *psb;
+
 		lpfc_set_rrq_active(phba, ndlp, lxri,
 			(xri == oxid) ? rxid : oxid, 0);
+		spin_lock_irqsave(&phba->hbalock, iflag);
+		for (i = 1; i <= phba->sli.last_iotag; i++) {
+			iocbq = phba->sli.iocbq_lookup[i];
+			if (!(iocbq->iocb_flag & LPFC_IO_FCP) ||
+			    (iocbq->iocb_flag & LPFC_IO_LIBDFC))
+				continue;
+			if (iocbq->sli4_lxritag != lxri)
+				continue;
+
+			psb = container_of(iocbq, struct lpfc_scsi_buf,
+					   cur_iocbq);
+			set_bit(LPFC_CMD_RRQ_ACTIVE, &psb->flags);
+			break;
+		}
+		spin_unlock_irqrestore(&phba->hbalock, iflag);
+	}
 	/* For BA_ABTS from exchange responder, if the logical xri with
 	 * the oxid maps to the FCP XRI range, the port no longer has
 	 * that exchange context, send a BLS_RJT. Override the IOCB for
-- 
1.8.5.6

[PATCH 14/14] lpfc: Complete scsi commands after RRQ has completed

[Hannes Reinecke]

The lpfc driver cannot re-use xri until RRQ after aborts has
completed. With this patch the scsi command remains valid until
RRQ has completed and only then scsi_done() is called.
This will cause an extended timeout for the command under abort,
but the driver will never re-use invalid xris.

Signed-off-by: Hannes Reinecke <hare@suse.com>
---
 drivers/scsi/lpfc/lpfc_scsi.c | 10 ++++++----
 drivers/scsi/lpfc/lpfc_sli.c  | 14 +++++++++++++-
 2 files changed, 19 insertions(+), 5 deletions(-)

diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index 348cae6..bbba1c2 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -1084,7 +1084,6 @@ lpfc_release_scsi_buf_s3(struct lpfc_hba *phba, struct lpfc_scsi_buf *psb)
 	psb->prot_seg_cnt = 0;
 
 	spin_lock_irqsave(&phba->scsi_buf_list_put_lock, iflag);
-	psb->pCmd = NULL;
 	psb->cur_iocbq.iocb_flag = LPFC_IO_FCP;
 	list_add_tail(&psb->list, &phba->lpfc_scsi_buf_list_put);
 	spin_unlock_irqrestore(&phba->scsi_buf_list_put_lock, iflag);
@@ -1114,13 +1113,11 @@ lpfc_release_scsi_buf_s4(struct lpfc_hba *phba, struct lpfc_scsi_buf *psb)
 			return;
 		spin_lock_irqsave(&phba->sli4_hba.abts_scsi_buf_list_lock,
 					iflag);
-		psb->pCmd = NULL;
 		list_add_tail(&psb->list,
 			&phba->sli4_hba.lpfc_abts_scsi_buf_list);
 		spin_unlock_irqrestore(&phba->sli4_hba.abts_scsi_buf_list_lock,
 					iflag);
 	} else {
-		psb->pCmd = NULL;
 		psb->cur_iocbq.iocb_flag = LPFC_IO_FCP;
 		if (phba->lpfc_scsi_buf_arr)
 			clear_bit(LPFC_CMD_QUEUED, &psb->flags);
@@ -4166,8 +4163,13 @@ lpfc_scsi_cmd_iocb_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pIocbIn,
 	 * release the scsi_buf before calling 'done', thereby
 	 * avoiding a race condition between aborts and scsi_done
 	 */
-	if (shost->hostt->abort_completions)
+	if (shost->hostt->abort_completions) {
 		lpfc_release_scsi_buf(phba, lpfc_cmd);
+		if (test_bit(LPFC_CMD_EXCH_BUSY, &lpfc_cmd->flags) &&
+		    test_bit(LPFC_CMD_ABORTED, &lpfc_cmd->flags))
+			return;
+		lpfc_cmd->pCmd = NULL;
+	}
 
 	/* The sdev is not guaranteed to be valid post scsi_done upcall. */
 	cmd->scsi_done(cmd);
diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index e5cd212..01b55c2 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -662,6 +662,7 @@ lpfc_clr_rrq_active(struct lpfc_hba *phba,
 		int i;
 		struct lpfc_iocbq *iocbq;
 		struct lpfc_scsi_buf *psb;
+		struct scsi_cmnd *cmd;
 
 		spin_lock_irqsave(&phba->hbalock, iflag);
 		for (i = 1; i <= phba->sli.last_iotag; i++) {
@@ -674,7 +675,18 @@ lpfc_clr_rrq_active(struct lpfc_hba *phba,
 
 			psb = container_of(iocbq, struct lpfc_scsi_buf,
 					   cur_iocbq);
-			clear_bit(LPFC_CMD_RRQ_ACTIVE, &psb->flags);
+			if (!test_and_clear_bit(LPFC_CMD_RRQ_ACTIVE,
+					       &psb->flags))
+				continue;
+
+			cmd = psb->pCmd;
+			if (cmd) {
+				psb->pCmd = NULL;
+				if (!cmd->result)
+					cmd->result = ScsiResult(DID_ABORT,
+							SAM_STAT_TASK_ABORTED);
+				cmd->scsi_done(cmd);
+			}
 			break;
 		}
 		spin_unlock_irqrestore(&phba->hbalock, iflag);
-- 
1.8.5.6

Re: [PATCH 08/14] blk-mq: add blk_mq_resize_tag_set()

[Bart Van Assche]

On 06/02/2016 07:39 AM, Hannes Reinecke wrote:
> +int blk_mq_resize_tag_set(struct blk_mq_tag_set *set, int new_tags)
> +{
> +	int i, ret = 0;
> +
> +	for (i = 0; i < set->nr_hw_queues; i++) {
> +		if (!set->tags[i])
> +			continue;
> +		ret = blk_mq_tag_update_depth(set->tags[i], new_tags);
> +		if (ret)
> +			break;
> +	}
> +	return ret;
> +}
> +EXPORT_SYMBOL(blk_mq_resize_tag_set);

(+Jens)

This code does not change the size of the bitmaps allocated by 
bt_alloc() so I think the word "resize" in the function name is 
misleading. How about using the name blk_mq_update_depth()? Have you 
considered a more clear name for the "new_tags" argument, e.g. tdepth? 
Please document that this argument represents the sum of the regular and 
reserved tags. Additionally, I don't think it is safe to call this 
function while I/O is in progress. Please either add code to stop and 
restart I/O inside this function or document this clearly.

Thanks,

Bart.

Re: [PATCH 09/14] scsi: Implement scsi_mq_resize_tags()

[Bart Van Assche]

On 06/02/2016 07:39 AM, Hannes Reinecke wrote:
> +int scsi_mq_resize_tags(struct Scsi_Host *shost, int max_tags)
> +{
> +	return blk_mq_resize_tag_set(&shost->tag_set, max_tags);
> +}
> +EXPORT_SYMBOL_GPL(scsi_mq_resize_tags);

What is the meaning of the max_tags argument? Does it include the 
reserved tag count or not? If not, shouldn't the number of reserved tags 
be added before calling blk_mq_resize_tag_set()? Please document the 
meaning of the max_tags argument.

Please also consider choosing a name for this function that is more 
consistent with the function names used in the block layer e.g. 
scsi_mq_update_tag_depth().

Thanks,

Bart.