[PATCH RESEND 0/7] New CCINs and several patches for new improvement

[PATCH RESEND 0/7] New CCINs and several patches for new improvement

[wenxiong]

Hi James,

I re-submitted the following patches for ipr device driver. I have re-tested
these patches with our latest adapters.

These patches are:

(1)Remove extended delay bit on GSCSI reads/writes ops.
(2)Add new device id to support Z06 LTD new adapter.
(3)Handle early EEH.
Above 3 patches were submitted on 1/21.

(4)Get rid of superfluous call to pci_disable_msi/msix()
(5)Use pci_enable_msi_range()/pci_enable_msix_range()
Above 2 patches were submiited by Alexander Gordeev

(6)Format HACM overlay ID 0x21.
(7)Add new device id to support Grand Canyon adapters.
Above patches were submitted on 2/15/2014

Let me know if you have questions about this.

Thanks for your help!
Wendy
-- 

[PATCH RESEND 1/7] ipr: Remove extended delay bit on GSCSI reads/writes ops

[wenxiong]

[-- Attachment #1: delay --]
[-- Type: text/plain, Size: 1826 bytes --]

This patch removes extended delay bit on GSCSI reads/writes ops, the
performance will be significanly better.

Signed-off-by: Wen Xiong <wenxiong@linux.vnet.ibm.com>

---
 drivers/scsi/ipr.c |    6 +++++-
 drivers/scsi/ipr.h |    1 +
 2 files changed, 6 insertions(+), 1 deletion(-)

Index: b/drivers/scsi/ipr.c
===================================================================
--- a/drivers/scsi/ipr.c	2014-03-11 17:11:04.487766647 -0500
+++ b/drivers/scsi/ipr.c	2014-03-11 17:12:07.218702192 -0500
@@ -1143,6 +1143,7 @@ static void ipr_init_res_entry(struct ip
 	res->add_to_ml = 0;
 	res->del_from_ml = 0;
 	res->resetting_device = 0;
+	res->reset_occurred = 0;
 	res->sdev = NULL;
 	res->sata_port = NULL;
 
@@ -5015,6 +5016,7 @@ static int __ipr_eh_dev_reset(struct scs
 	} else
 		rc = ipr_device_reset(ioa_cfg, res);
 	res->resetting_device = 0;
+	res->reset_occurred = 1;
 
 	LEAVE;
 	return rc ? FAILED : SUCCESS;
@@ -6183,8 +6185,10 @@ static int ipr_queuecommand(struct Scsi_
 			ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_ULEN_CHK;
 
 		ioarcb->cmd_pkt.flags_hi |= IPR_FLAGS_HI_NO_LINK_DESC;
-		if (ipr_is_gscsi(res))
+		if (ipr_is_gscsi(res) && res->reset_occurred) {
+			res->reset_occurred = 0;
 			ioarcb->cmd_pkt.flags_lo |= IPR_FLAGS_LO_DELAY_AFTER_RST;
+		}
 		ioarcb->cmd_pkt.flags_lo |= IPR_FLAGS_LO_ALIGNED_BFR;
 		ioarcb->cmd_pkt.flags_lo |= ipr_get_task_attributes(scsi_cmd);
 	}
Index: b/drivers/scsi/ipr.h
===================================================================
--- a/drivers/scsi/ipr.h	2014-03-11 17:10:58.577766209 -0500
+++ b/drivers/scsi/ipr.h	2014-03-11 17:12:07.218702192 -0500
@@ -1252,6 +1252,7 @@ struct ipr_resource_entry {
 	u8 add_to_ml:1;
 	u8 del_from_ml:1;
 	u8 resetting_device:1;
+	u8 reset_occurred:1;
 
 	u32 bus;		/* AKA channel */
 	u32 target;		/* AKA id */

-- 

Re: [PATCH RESEND 1/7] ipr: Remove extended delay bit on GSCSI reads/writes ops

[Brian King]

Ack patches 1-7

Acked-by: Brian King <brking@linux.vnet.ibm.com>

-- 
Brian King
Power Linux I/O
IBM Linux Technology Center

[PATCH RESEND 2/7] ipr: Add new CCIN definition for new hardware support

[wenxiong]

[-- Attachment #1: z06_ltd_devid --]
[-- Type: text/plain, Size: 1460 bytes --]

Add the appropriate definition and table entry for new hardware support.

Signed-off-by: Wen Xiong <wenxiong@linux.vnet.ibm.com>

---
 drivers/scsi/ipr.c |    2 ++
 drivers/scsi/ipr.h |    1 +
 2 files changed, 3 insertions(+)

Index: b/drivers/scsi/ipr.c
===================================================================
--- a/drivers/scsi/ipr.c	2014-03-11 17:12:07.218702192 -0500
+++ b/drivers/scsi/ipr.c	2014-03-11 17:13:10.759012001 -0500
@@ -9998,6 +9998,8 @@ static struct pci_device_id ipr_pci_tabl
 	{ PCI_VENDOR_ID_IBM, PCI_DEVICE_ID_IBM_CROCODILE,
 		PCI_VENDOR_ID_IBM, IPR_SUBS_DEV_ID_57D9, 0, 0, 0 },
 	{ PCI_VENDOR_ID_IBM, PCI_DEVICE_ID_IBM_CROCODILE,
+		PCI_VENDOR_ID_IBM, IPR_SUBS_DEV_ID_57DA, 0, 0, 0 },
+	{ PCI_VENDOR_ID_IBM, PCI_DEVICE_ID_IBM_CROCODILE,
 		PCI_VENDOR_ID_IBM, IPR_SUBS_DEV_ID_57EB, 0, 0, 0 },
 	{ PCI_VENDOR_ID_IBM, PCI_DEVICE_ID_IBM_CROCODILE,
 		PCI_VENDOR_ID_IBM, IPR_SUBS_DEV_ID_57EC, 0, 0, 0 },
Index: b/drivers/scsi/ipr.h
===================================================================
--- a/drivers/scsi/ipr.h	2014-03-11 17:12:07.218702192 -0500
+++ b/drivers/scsi/ipr.h	2014-03-11 17:13:10.759012001 -0500
@@ -101,6 +101,7 @@
 #define IPR_SUBS_DEV_ID_57D7    0x03FF
 #define IPR_SUBS_DEV_ID_57D8    0x03FE
 #define IPR_SUBS_DEV_ID_57D9    0x046D
+#define IPR_SUBS_DEV_ID_57DA    0x04CA
 #define IPR_SUBS_DEV_ID_57EB    0x0474
 #define IPR_SUBS_DEV_ID_57EC    0x0475
 #define IPR_SUBS_DEV_ID_57ED    0x0499

-- 

[PATCH RESEND 3/7] ipr: Handle early EEH

[wenxiong]

[-- Attachment #1: early_eeh --]
[-- Type: text/plain, Size: 17323 bytes --]

From:  Brian King <brking@linux.vnet.ibm.com>

If, when the ipr driver loads, the adapter is in an EEH error state,
it will currently oops and not be able to recover, as it attempts
to access memory that has not yet been allocated. We've seen this
occur in some kexec scenarios. The following patch fixes the oops
and also allows the driver to recover from these probe time EEH errors.

Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
---

 drivers/scsi/ipr.c |  265 +++++++++++++++++++++++++++++++++++------------------
 drivers/scsi/ipr.h |    3 
 2 files changed, 179 insertions(+), 89 deletions(-)

Index: b/drivers/scsi/ipr.h
===================================================================
--- a/drivers/scsi/ipr.h	2014-03-11 17:13:10.759012001 -0500
+++ b/drivers/scsi/ipr.h	2014-03-11 17:13:24.279949356 -0500
@@ -231,6 +231,7 @@
 #define IPR_WAIT_FOR_RESET_TIMEOUT		(2 * HZ)
 #define IPR_CHECK_FOR_RESET_TIMEOUT		(HZ / 10)
 #define IPR_WAIT_FOR_BIST_TIMEOUT		(2 * HZ)
+#define IPR_PCI_ERROR_RECOVERY_TIMEOUT	(120 * HZ)
 #define IPR_PCI_RESET_TIMEOUT			(HZ / 2)
 #define IPR_SIS32_DUMP_TIMEOUT			(15 * HZ)
 #define IPR_SIS64_DUMP_TIMEOUT			(40 * HZ)
@@ -1443,6 +1444,7 @@ struct ipr_ioa_cfg {
 	u8 dump_timeout:1;
 	u8 cfg_locked:1;
 	u8 clear_isr:1;
+	u8 probe_done:1;
 
 	u8 revid;
 
@@ -1521,6 +1523,7 @@ struct ipr_ioa_cfg {
 
 	wait_queue_head_t reset_wait_q;
 	wait_queue_head_t msi_wait_q;
+	wait_queue_head_t eeh_wait_q;
 
 	struct ipr_dump *dump;
 	enum ipr_sdt_state sdt_state;
Index: b/drivers/scsi/ipr.c
===================================================================
--- a/drivers/scsi/ipr.c	2014-03-11 17:13:10.759012001 -0500
+++ b/drivers/scsi/ipr.c	2014-03-11 17:13:24.279949356 -0500
@@ -8645,6 +8645,25 @@ static int ipr_reset_freeze(struct ipr_c
 }
 
 /**
+ * ipr_pci_mmio_enabled - Called when MMIO has been re-enabled
+ * @pdev:	PCI device struct
+ *
+ * Description: This routine is called to tell us that the MMIO
+ * access to the IOA has been restored
+ */
+static pci_ers_result_t ipr_pci_mmio_enabled(struct pci_dev *pdev)
+{
+	unsigned long flags = 0;
+	struct ipr_ioa_cfg *ioa_cfg = pci_get_drvdata(pdev);
+
+	spin_lock_irqsave(ioa_cfg->host->host_lock, flags);
+	if (!ioa_cfg->probe_done)
+		pci_save_state(pdev);
+	spin_unlock_irqrestore(ioa_cfg->host->host_lock, flags);
+	return PCI_ERS_RESULT_NEED_RESET;
+}
+
+/**
  * ipr_pci_frozen - Called when slot has experienced a PCI bus error.
  * @pdev:	PCI device struct
  *
@@ -8658,7 +8677,8 @@ static void ipr_pci_frozen(struct pci_de
 	struct ipr_ioa_cfg *ioa_cfg = pci_get_drvdata(pdev);
 
 	spin_lock_irqsave(ioa_cfg->host->host_lock, flags);
-	_ipr_initiate_ioa_reset(ioa_cfg, ipr_reset_freeze, IPR_SHUTDOWN_NONE);
+	if (ioa_cfg->probe_done)
+		_ipr_initiate_ioa_reset(ioa_cfg, ipr_reset_freeze, IPR_SHUTDOWN_NONE);
 	spin_unlock_irqrestore(ioa_cfg->host->host_lock, flags);
 }
 
@@ -8676,11 +8696,14 @@ static pci_ers_result_t ipr_pci_slot_res
 	struct ipr_ioa_cfg *ioa_cfg = pci_get_drvdata(pdev);
 
 	spin_lock_irqsave(ioa_cfg->host->host_lock, flags);
-	if (ioa_cfg->needs_warm_reset)
-		ipr_initiate_ioa_reset(ioa_cfg, IPR_SHUTDOWN_NONE);
-	else
-		_ipr_initiate_ioa_reset(ioa_cfg, ipr_reset_restore_cfg_space,
-					IPR_SHUTDOWN_NONE);
+	if (ioa_cfg->probe_done) {
+		if (ioa_cfg->needs_warm_reset)
+			ipr_initiate_ioa_reset(ioa_cfg, IPR_SHUTDOWN_NONE);
+		else
+			_ipr_initiate_ioa_reset(ioa_cfg, ipr_reset_restore_cfg_space,
+						IPR_SHUTDOWN_NONE);
+	} else
+		wake_up_all(&ioa_cfg->eeh_wait_q);
 	spin_unlock_irqrestore(ioa_cfg->host->host_lock, flags);
 	return PCI_ERS_RESULT_RECOVERED;
 }
@@ -8699,17 +8722,20 @@ static void ipr_pci_perm_failure(struct 
 	int i;
 
 	spin_lock_irqsave(ioa_cfg->host->host_lock, flags);
-	if (ioa_cfg->sdt_state == WAIT_FOR_DUMP)
-		ioa_cfg->sdt_state = ABORT_DUMP;
-	ioa_cfg->reset_retries = IPR_NUM_RESET_RELOAD_RETRIES - 1;
-	ioa_cfg->in_ioa_bringdown = 1;
-	for (i = 0; i < ioa_cfg->hrrq_num; i++) {
-		spin_lock(&ioa_cfg->hrrq[i]._lock);
-		ioa_cfg->hrrq[i].allow_cmds = 0;
-		spin_unlock(&ioa_cfg->hrrq[i]._lock);
-	}
-	wmb();
-	ipr_initiate_ioa_reset(ioa_cfg, IPR_SHUTDOWN_NONE);
+	if (ioa_cfg->probe_done) {
+		if (ioa_cfg->sdt_state == WAIT_FOR_DUMP)
+			ioa_cfg->sdt_state = ABORT_DUMP;
+		ioa_cfg->reset_retries = IPR_NUM_RESET_RELOAD_RETRIES - 1;
+		ioa_cfg->in_ioa_bringdown = 1;
+		for (i = 0; i < ioa_cfg->hrrq_num; i++) {
+			spin_lock(&ioa_cfg->hrrq[i]._lock);
+			ioa_cfg->hrrq[i].allow_cmds = 0;
+			spin_unlock(&ioa_cfg->hrrq[i]._lock);
+		}
+		wmb();
+		ipr_initiate_ioa_reset(ioa_cfg, IPR_SHUTDOWN_NONE);
+	} else
+		wake_up_all(&ioa_cfg->eeh_wait_q);
 	spin_unlock_irqrestore(ioa_cfg->host->host_lock, flags);
 }
 
@@ -8729,7 +8755,7 @@ static pci_ers_result_t ipr_pci_error_de
 	switch (state) {
 	case pci_channel_io_frozen:
 		ipr_pci_frozen(pdev);
-		return PCI_ERS_RESULT_NEED_RESET;
+		return PCI_ERS_RESULT_CAN_RECOVER;
 	case pci_channel_io_perm_failure:
 		ipr_pci_perm_failure(pdev);
 		return PCI_ERS_RESULT_DISCONNECT;
@@ -8759,6 +8785,7 @@ static int ipr_probe_ioa_part2(struct ip
 	ENTER;
 	spin_lock_irqsave(ioa_cfg->host->host_lock, host_lock_flags);
 	dev_dbg(&ioa_cfg->pdev->dev, "ioa_cfg adx: 0x%p\n", ioa_cfg);
+	ioa_cfg->probe_done = 1;
 	if (ioa_cfg->needs_hard_reset) {
 		ioa_cfg->needs_hard_reset = 0;
 		ipr_initiate_ioa_reset(ioa_cfg, IPR_SHUTDOWN_NONE);
@@ -9034,16 +9061,6 @@ static int ipr_alloc_mem(struct ipr_ioa_
 	if (!ioa_cfg->vpd_cbs)
 		goto out_free_res_entries;
 
-	for (i = 0; i < ioa_cfg->hrrq_num; i++) {
-		INIT_LIST_HEAD(&ioa_cfg->hrrq[i].hrrq_free_q);
-		INIT_LIST_HEAD(&ioa_cfg->hrrq[i].hrrq_pending_q);
-		spin_lock_init(&ioa_cfg->hrrq[i]._lock);
-		if (i == 0)
-			ioa_cfg->hrrq[i].lock = ioa_cfg->host->host_lock;
-		else
-			ioa_cfg->hrrq[i].lock = &ioa_cfg->hrrq[i]._lock;
-	}
-
 	if (ipr_alloc_cmd_blks(ioa_cfg))
 		goto out_free_vpd_cbs;
 
@@ -9144,6 +9161,48 @@ static void ipr_initialize_bus_attr(stru
 }
 
 /**
+ * ipr_init_regs - Initialize IOA registers
+ * @ioa_cfg:	ioa config struct
+ *
+ * Return value:
+ * 	none
+ **/
+static void ipr_init_regs(struct ipr_ioa_cfg *ioa_cfg)
+{
+	const struct ipr_interrupt_offsets *p;
+	struct ipr_interrupts *t;
+	void __iomem *base;
+
+	p = &ioa_cfg->chip_cfg->regs;
+	t = &ioa_cfg->regs;
+	base = ioa_cfg->hdw_dma_regs;
+
+	t->set_interrupt_mask_reg = base + p->set_interrupt_mask_reg;
+	t->clr_interrupt_mask_reg = base + p->clr_interrupt_mask_reg;
+	t->clr_interrupt_mask_reg32 = base + p->clr_interrupt_mask_reg32;
+	t->sense_interrupt_mask_reg = base + p->sense_interrupt_mask_reg;
+	t->sense_interrupt_mask_reg32 = base + p->sense_interrupt_mask_reg32;
+	t->clr_interrupt_reg = base + p->clr_interrupt_reg;
+	t->clr_interrupt_reg32 = base + p->clr_interrupt_reg32;
+	t->sense_interrupt_reg = base + p->sense_interrupt_reg;
+	t->sense_interrupt_reg32 = base + p->sense_interrupt_reg32;
+	t->ioarrin_reg = base + p->ioarrin_reg;
+	t->sense_uproc_interrupt_reg = base + p->sense_uproc_interrupt_reg;
+	t->sense_uproc_interrupt_reg32 = base + p->sense_uproc_interrupt_reg32;
+	t->set_uproc_interrupt_reg = base + p->set_uproc_interrupt_reg;
+	t->set_uproc_interrupt_reg32 = base + p->set_uproc_interrupt_reg32;
+	t->clr_uproc_interrupt_reg = base + p->clr_uproc_interrupt_reg;
+	t->clr_uproc_interrupt_reg32 = base + p->clr_uproc_interrupt_reg32;
+
+	if (ioa_cfg->sis64) {
+		t->init_feedback_reg = base + p->init_feedback_reg;
+		t->dump_addr_reg = base + p->dump_addr_reg;
+		t->dump_data_reg = base + p->dump_data_reg;
+		t->endian_swap_reg = base + p->endian_swap_reg;
+	}
+}
+
+/**
  * ipr_init_ioa_cfg - Initialize IOA config struct
  * @ioa_cfg:	ioa config struct
  * @host:		scsi host struct
@@ -9155,9 +9214,7 @@ static void ipr_initialize_bus_attr(stru
 static void ipr_init_ioa_cfg(struct ipr_ioa_cfg *ioa_cfg,
 			     struct Scsi_Host *host, struct pci_dev *pdev)
 {
-	const struct ipr_interrupt_offsets *p;
-	struct ipr_interrupts *t;
-	void __iomem *base;
+	int i;
 
 	ioa_cfg->host = host;
 	ioa_cfg->pdev = pdev;
@@ -9177,6 +9234,7 @@ static void ipr_init_ioa_cfg(struct ipr_
 	INIT_WORK(&ioa_cfg->work_q, ipr_worker_thread);
 	init_waitqueue_head(&ioa_cfg->reset_wait_q);
 	init_waitqueue_head(&ioa_cfg->msi_wait_q);
+	init_waitqueue_head(&ioa_cfg->eeh_wait_q);
 	ioa_cfg->sdt_state = INACTIVE;
 
 	ipr_initialize_bus_attr(ioa_cfg);
@@ -9187,44 +9245,33 @@ static void ipr_init_ioa_cfg(struct ipr_
 		host->max_lun = IPR_MAX_SIS64_LUNS_PER_TARGET;
 		if (ipr_max_devs > IPR_MAX_SIS64_DEVS)
 			ioa_cfg->max_devs_supported = IPR_MAX_SIS64_DEVS;
+		ioa_cfg->cfg_table_size = (sizeof(struct ipr_config_table_hdr64)
+					   + ((sizeof(struct ipr_config_table_entry64)
+					       * ioa_cfg->max_devs_supported)));
 	} else {
 		host->max_id = IPR_MAX_NUM_TARGETS_PER_BUS;
 		host->max_lun = IPR_MAX_NUM_LUNS_PER_TARGET;
 		if (ipr_max_devs > IPR_MAX_PHYSICAL_DEVS)
 			ioa_cfg->max_devs_supported = IPR_MAX_PHYSICAL_DEVS;
+		ioa_cfg->cfg_table_size = (sizeof(struct ipr_config_table_hdr)
+					   + ((sizeof(struct ipr_config_table_entry)
+					       * ioa_cfg->max_devs_supported)));
 	}
+
 	host->max_channel = IPR_MAX_BUS_TO_SCAN;
 	host->unique_id = host->host_no;
 	host->max_cmd_len = IPR_MAX_CDB_LEN;
 	host->can_queue = ioa_cfg->max_cmds;
 	pci_set_drvdata(pdev, ioa_cfg);
 
-	p = &ioa_cfg->chip_cfg->regs;
-	t = &ioa_cfg->regs;
-	base = ioa_cfg->hdw_dma_regs;
-
-	t->set_interrupt_mask_reg = base + p->set_interrupt_mask_reg;
-	t->clr_interrupt_mask_reg = base + p->clr_interrupt_mask_reg;
-	t->clr_interrupt_mask_reg32 = base + p->clr_interrupt_mask_reg32;
-	t->sense_interrupt_mask_reg = base + p->sense_interrupt_mask_reg;
-	t->sense_interrupt_mask_reg32 = base + p->sense_interrupt_mask_reg32;
-	t->clr_interrupt_reg = base + p->clr_interrupt_reg;
-	t->clr_interrupt_reg32 = base + p->clr_interrupt_reg32;
-	t->sense_interrupt_reg = base + p->sense_interrupt_reg;
-	t->sense_interrupt_reg32 = base + p->sense_interrupt_reg32;
-	t->ioarrin_reg = base + p->ioarrin_reg;
-	t->sense_uproc_interrupt_reg = base + p->sense_uproc_interrupt_reg;
-	t->sense_uproc_interrupt_reg32 = base + p->sense_uproc_interrupt_reg32;
-	t->set_uproc_interrupt_reg = base + p->set_uproc_interrupt_reg;
-	t->set_uproc_interrupt_reg32 = base + p->set_uproc_interrupt_reg32;
-	t->clr_uproc_interrupt_reg = base + p->clr_uproc_interrupt_reg;
-	t->clr_uproc_interrupt_reg32 = base + p->clr_uproc_interrupt_reg32;
-
-	if (ioa_cfg->sis64) {
-		t->init_feedback_reg = base + p->init_feedback_reg;
-		t->dump_addr_reg = base + p->dump_addr_reg;
-		t->dump_data_reg = base + p->dump_data_reg;
-		t->endian_swap_reg = base + p->endian_swap_reg;
+	for (i = 0; i < ARRAY_SIZE(ioa_cfg->hrrq); i++) {
+		INIT_LIST_HEAD(&ioa_cfg->hrrq[i].hrrq_free_q);
+		INIT_LIST_HEAD(&ioa_cfg->hrrq[i].hrrq_pending_q);
+		spin_lock_init(&ioa_cfg->hrrq[i]._lock);
+		if (i == 0)
+			ioa_cfg->hrrq[i].lock = ioa_cfg->host->host_lock;
+		else
+			ioa_cfg->hrrq[i].lock = &ioa_cfg->hrrq[i]._lock;
 	}
 }
 
@@ -9247,6 +9294,26 @@ ipr_get_chip_info(const struct pci_devic
 	return NULL;
 }
 
+/**
+ * ipr_wait_for_pci_err_recovery - Wait for any PCI error recovery to complete
+ *						during probe time
+ * @ioa_cfg:	ioa config struct
+ *
+ * Return value:
+ * 	None
+ **/
+static void ipr_wait_for_pci_err_recovery(struct ipr_ioa_cfg *ioa_cfg)
+{
+	struct pci_dev *pdev = ioa_cfg->pdev;
+
+	if (pci_channel_offline(pdev)) {
+		wait_event_timeout(ioa_cfg->eeh_wait_q,
+				   !pci_channel_offline(pdev),
+				   IPR_PCI_ERROR_RECOVERY_TIMEOUT);
+		pci_restore_state(pdev);
+	}
+}
+
 static int ipr_enable_msix(struct ipr_ioa_cfg *ioa_cfg)
 {
 	struct msix_entry entries[IPR_MAX_MSIX_VECTORS];
@@ -9261,6 +9328,7 @@ static int ipr_enable_msix(struct ipr_io
 			vectors = err;
 
 	if (err < 0) {
+		ipr_wait_for_pci_err_recovery(ioa_cfg);
 		pci_disable_msix(ioa_cfg->pdev);
 		return err;
 	}
@@ -9284,6 +9352,7 @@ static int ipr_enable_msi(struct ipr_ioa
 			vectors = err;
 
 	if (err < 0) {
+		ipr_wait_for_pci_err_recovery(ioa_cfg);
 		pci_disable_msi(ioa_cfg->pdev);
 		return err;
 	}
@@ -9438,19 +9507,13 @@ static int ipr_probe_ioa(struct pci_dev 
 
 	ENTER;
 
-	if ((rc = pci_enable_device(pdev))) {
-		dev_err(&pdev->dev, "Cannot enable adapter\n");
-		goto out;
-	}
-
 	dev_info(&pdev->dev, "Found IOA with IRQ: %d\n", pdev->irq);
-
 	host = scsi_host_alloc(&driver_template, sizeof(*ioa_cfg));
 
 	if (!host) {
 		dev_err(&pdev->dev, "call to scsi_host_alloc failed!\n");
 		rc = -ENOMEM;
-		goto out_disable;
+		goto out;
 	}
 
 	ioa_cfg = (struct ipr_ioa_cfg *)host->hostdata;
@@ -9480,6 +9543,8 @@ static int ipr_probe_ioa(struct pci_dev 
 
 	ioa_cfg->revid = pdev->revision;
 
+	ipr_init_ioa_cfg(ioa_cfg, host, pdev);
+
 	ipr_regs_pci = pci_resource_start(pdev, 0);
 
 	rc = pci_request_regions(pdev, IPR_NAME);
@@ -9489,22 +9554,35 @@ static int ipr_probe_ioa(struct pci_dev 
 		goto out_scsi_host_put;
 	}
 
+	rc = pci_enable_device(pdev);
+
+	if (rc || pci_channel_offline(pdev)) {
+		if (pci_channel_offline(pdev)) {
+			ipr_wait_for_pci_err_recovery(ioa_cfg);
+			rc = pci_enable_device(pdev);
+		}
+
+		if (rc) {
+			dev_err(&pdev->dev, "Cannot enable adapter\n");
+			ipr_wait_for_pci_err_recovery(ioa_cfg);
+			goto out_release_regions;
+		}
+	}
+
 	ipr_regs = pci_ioremap_bar(pdev, 0);
 
 	if (!ipr_regs) {
 		dev_err(&pdev->dev,
 			"Couldn't map memory range of registers\n");
 		rc = -ENOMEM;
-		goto out_release_regions;
+		goto out_disable;
 	}
 
 	ioa_cfg->hdw_dma_regs = ipr_regs;
 	ioa_cfg->hdw_dma_regs_pci = ipr_regs_pci;
 	ioa_cfg->ioa_mailbox = ioa_cfg->chip_cfg->mailbox + ipr_regs;
 
-	ipr_init_ioa_cfg(ioa_cfg, host, pdev);
-
-	pci_set_master(pdev);
+	ipr_init_regs(ioa_cfg);
 
 	if (ioa_cfg->sis64) {
 		rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
@@ -9512,7 +9590,6 @@ static int ipr_probe_ioa(struct pci_dev 
 			dev_dbg(&pdev->dev, "Failed to set 64 bit PCI DMA mask\n");
 			rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
 		}
-
 	} else
 		rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
 
@@ -9526,10 +9603,15 @@ static int ipr_probe_ioa(struct pci_dev 
 
 	if (rc != PCIBIOS_SUCCESSFUL) {
 		dev_err(&pdev->dev, "Write of cache line size failed\n");
+		ipr_wait_for_pci_err_recovery(ioa_cfg);
 		rc = -EIO;
 		goto cleanup_nomem;
 	}
 
+	/* Issue MMIO read to ensure card is not in EEH */
+	interrupts = readl(ioa_cfg->regs.sense_interrupt_reg);
+	ipr_wait_for_pci_err_recovery(ioa_cfg);
+
 	if (ipr_number_of_msix > IPR_MAX_MSIX_VECTORS) {
 		dev_err(&pdev->dev, "The max number of MSIX is %d\n",
 			IPR_MAX_MSIX_VECTORS);
@@ -9548,10 +9630,22 @@ static int ipr_probe_ioa(struct pci_dev 
 		dev_info(&pdev->dev, "Cannot enable MSI.\n");
 	}
 
+	pci_set_master(pdev);
+
+	if (pci_channel_offline(pdev)) {
+		ipr_wait_for_pci_err_recovery(ioa_cfg);
+		pci_set_master(pdev);
+		if (pci_channel_offline(pdev)) {
+			rc = -EIO;
+			goto out_msi_disable;
+		}
+	}
+
 	if (ioa_cfg->intr_flag == IPR_USE_MSI ||
 	    ioa_cfg->intr_flag == IPR_USE_MSIX) {
 		rc = ipr_test_msi(ioa_cfg, pdev);
 		if (rc == -EOPNOTSUPP) {
+			ipr_wait_for_pci_err_recovery(ioa_cfg);
 			if (ioa_cfg->intr_flag == IPR_USE_MSI) {
 				ioa_cfg->intr_flag &= ~IPR_USE_MSI;
 				pci_disable_msi(pdev);
@@ -9581,30 +9675,12 @@ static int ipr_probe_ioa(struct pci_dev 
 				(unsigned int)num_online_cpus(),
 				(unsigned int)IPR_MAX_HRRQ_NUM);
 
-	/* Save away PCI config space for use following IOA reset */
-	rc = pci_save_state(pdev);
-
-	if (rc != PCIBIOS_SUCCESSFUL) {
-		dev_err(&pdev->dev, "Failed to save PCI config space\n");
-		rc = -EIO;
-		goto out_msi_disable;
-	}
-
 	if ((rc = ipr_save_pcix_cmd_reg(ioa_cfg)))
 		goto out_msi_disable;
 
 	if ((rc = ipr_set_pcix_cmd_reg(ioa_cfg)))
 		goto out_msi_disable;
 
-	if (ioa_cfg->sis64)
-		ioa_cfg->cfg_table_size = (sizeof(struct ipr_config_table_hdr64)
-				+ ((sizeof(struct ipr_config_table_entry64)
-				* ioa_cfg->max_devs_supported)));
-	else
-		ioa_cfg->cfg_table_size = (sizeof(struct ipr_config_table_hdr)
-				+ ((sizeof(struct ipr_config_table_entry)
-				* ioa_cfg->max_devs_supported)));
-
 	rc = ipr_alloc_mem(ioa_cfg);
 	if (rc < 0) {
 		dev_err(&pdev->dev,
@@ -9612,6 +9688,15 @@ static int ipr_probe_ioa(struct pci_dev 
 		goto out_msi_disable;
 	}
 
+	/* Save away PCI config space for use following IOA reset */
+	rc = pci_save_state(pdev);
+
+	if (rc != PCIBIOS_SUCCESSFUL) {
+		dev_err(&pdev->dev, "Failed to save PCI config space\n");
+		rc = -EIO;
+		goto cleanup_nolog;
+	}
+
 	/*
 	 * If HRRQ updated interrupt is not masked, or reset alert is set,
 	 * the card is in an unknown state and needs a hard reset
@@ -9668,18 +9753,19 @@ out:
 cleanup_nolog:
 	ipr_free_mem(ioa_cfg);
 out_msi_disable:
+	ipr_wait_for_pci_err_recovery(ioa_cfg);
 	if (ioa_cfg->intr_flag == IPR_USE_MSI)
 		pci_disable_msi(pdev);
 	else if (ioa_cfg->intr_flag == IPR_USE_MSIX)
 		pci_disable_msix(pdev);
 cleanup_nomem:
 	iounmap(ipr_regs);
+out_disable:
+	pci_disable_device(pdev);
 out_release_regions:
 	pci_release_regions(pdev);
 out_scsi_host_put:
 	scsi_host_put(host);
-out_disable:
-	pci_disable_device(pdev);
 	goto out;
 }
 
@@ -10017,6 +10103,7 @@ MODULE_DEVICE_TABLE(pci, ipr_pci_table);
 
 static const struct pci_error_handlers ipr_err_handler = {
 	.error_detected = ipr_pci_error_detected,
+	.mmio_enabled = ipr_pci_mmio_enabled,
 	.slot_reset = ipr_pci_slot_reset,
 };
 

-- 

[PATCH RESEND 4/7] ipr: Get rid of superfluous call to pci_disbale_msi/msix()

[wenxiong]

[-- Attachment #1: 001_msix --]
[-- Type: text/plain, Size: 936 bytes --]

From: Alexander Gordeev <agordeev@redhat.com>

There is no need to call pci_disable_msi() or pci_disable_msix()
in case the call to pci_enable_msi() or pci_enable_msix() failed.

Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
Acked-by: Brian King <brking@linux.vnet.ibm.com>
---
 drivers/scsi/ipr.c |    2 --
 1 file changed, 2 deletions(-)

Index: b/drivers/scsi/ipr.c
===================================================================
--- a/drivers/scsi/ipr.c	2014-03-11 17:16:08.849955755 -0500
+++ b/drivers/scsi/ipr.c	2014-03-11 17:18:06.709957199 -0500
@@ -9329,7 +9329,6 @@ static int ipr_enable_msix(struct ipr_io
 
 	if (err < 0) {
 		ipr_wait_for_pci_err_recovery(ioa_cfg);
-		pci_disable_msix(ioa_cfg->pdev);
 		return err;
 	}
 
@@ -9353,7 +9352,6 @@ static int ipr_enable_msi(struct ipr_ioa
 
 	if (err < 0) {
 		ipr_wait_for_pci_err_recovery(ioa_cfg);
-		pci_disable_msi(ioa_cfg->pdev);
 		return err;
 	}
 

-- 

Re: [PATCH RESEND 4/7] ipr: Get rid of superfluous call to pci_disbale_msi/msix()

[James Bottomley]

On Wed, 2014-03-12 at 16:08 -0500, wenxiong@vmlinux.vnet.ibm.com wrote:
> From: Alexander Gordeev <agordeev@redhat.com>
> 
> There is no need to call pci_disable_msi() or pci_disable_msix()
> in case the call to pci_enable_msi() or pci_enable_msix() failed.
> 
> Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
> Acked-by: Brian King <brking@linux.vnet.ibm.com>
> ---

This needs a Signed-off-by: from you as well, since you transmitted the
patch to me.  I assume it was just omitted in error and have added it in
the git repository.

James

Re: [PATCH RESEND 4/7] ipr: Get rid of superfluous call to pci_disbale_msi/msix()

[Brian King]

On 03/19/2014 02:55 PM, James Bottomley wrote:
> On Wed, 2014-03-12 at 16:08 -0500, wenxiong@vmlinux.vnet.ibm.com wrote:
>> From: Alexander Gordeev <agordeev@redhat.com>
>>
>> There is no need to call pci_disable_msi() or pci_disable_msix()
>> in case the call to pci_enable_msi() or pci_enable_msix() failed.
>>
>> Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
>> Acked-by: Brian King <brking@linux.vnet.ibm.com>
>> ---
> 
> This needs a Signed-off-by: from you as well, since you transmitted the
> patch to me.  I assume it was just omitted in error and have added it in
> the git repository.

Correcting Wendy's email address...

-Brian

-- 
Brian King
Power Linux I/O
IBM Linux Technology Center

Re: [PATCH RESEND 4/7] ipr: Get rid of superfluous call to pci_disbale_msi/msix()

[James Bottomley]

On Wed, 2014-03-19 at 15:26 -0500, Brian King wrote:
> On 03/19/2014 02:55 PM, James Bottomley wrote:
> > On Wed, 2014-03-12 at 16:08 -0500, wenxiong@vmlinux.vnet.ibm.com wrote:
> >> From: Alexander Gordeev <agordeev@redhat.com>
> >>
> >> There is no need to call pci_disable_msi() or pci_disable_msix()
> >> in case the call to pci_enable_msi() or pci_enable_msix() failed.
> >>
> >> Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
> >> Acked-by: Brian King <brking@linux.vnet.ibm.com>
> >> ---
> > 
> > This needs a Signed-off-by: from you as well, since you transmitted the
> > patch to me.  I assume it was just omitted in error and have added it in
> > the git repository.
> 
> Correcting Wendy's email address...

Humf, that means it's wrong in the Author field of the git commits
too ... please fix whatever script is sending the wrong email address.

Thanks,

James

[PATCH RESEND 5/7] ipr: Use pci_enable_msi_range() and pci_enable_msix_range()

[wenxiong]

[-- Attachment #1: 002_msix --]
[-- Type: text/plain, Size: 2958 bytes --]

From: Alexander Gordeev <agordeev@redhat.com>

As result deprecation of MSI-X/MSI enablement functions
pci_enable_msix() and pci_enable_msi_block() all drivers
using these two interfaces need to be updated to use the
new pci_enable_msi_range() and pci_enable_msix_range()
interfaces.

Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
Acked-by: Brian King <brking@linux.vnet.ibm.com>
---
 drivers/scsi/ipr.c |   47 ++++++++++++++++++-----------------------------
 1 file changed, 18 insertions(+), 29 deletions(-)

Index: b/drivers/scsi/ipr.c
===================================================================
--- a/drivers/scsi/ipr.c	2014-03-11 17:18:06.709957199 -0500
+++ b/drivers/scsi/ipr.c	2014-03-11 17:18:20.169018184 -0500
@@ -9317,51 +9317,40 @@ static void ipr_wait_for_pci_err_recover
 static int ipr_enable_msix(struct ipr_ioa_cfg *ioa_cfg)
 {
 	struct msix_entry entries[IPR_MAX_MSIX_VECTORS];
-	int i, err, vectors;
+	int i, vectors;
 
 	for (i = 0; i < ARRAY_SIZE(entries); ++i)
 		entries[i].entry = i;
 
-	vectors = ipr_number_of_msix;
-
-	while ((err = pci_enable_msix(ioa_cfg->pdev, entries, vectors)) > 0)
-			vectors = err;
-
-	if (err < 0) {
+	vectors = pci_enable_msix_range(ioa_cfg->pdev,
+					entries, 1, ipr_number_of_msix);
+	if (vectors < 0) {
 		ipr_wait_for_pci_err_recovery(ioa_cfg);
-		return err;
+		return vectors;
 	}
 
-	if (!err) {
-		for (i = 0; i < vectors; i++)
-			ioa_cfg->vectors_info[i].vec = entries[i].vector;
-		ioa_cfg->nvectors = vectors;
-	}
+	for (i = 0; i < vectors; i++)
+		ioa_cfg->vectors_info[i].vec = entries[i].vector;
+	ioa_cfg->nvectors = vectors;
 
-	return err;
+	return 0;
 }
 
 static int ipr_enable_msi(struct ipr_ioa_cfg *ioa_cfg)
 {
-	int i, err, vectors;
+	int i, vectors;
 
-	vectors = ipr_number_of_msix;
-
-	while ((err = pci_enable_msi_block(ioa_cfg->pdev, vectors)) > 0)
-			vectors = err;
-
-	if (err < 0) {
+	vectors = pci_enable_msi_range(ioa_cfg->pdev, 1, ipr_number_of_msix);
+	if (vectors < 0) {
 		ipr_wait_for_pci_err_recovery(ioa_cfg);
-		return err;
+		return vectors;
 	}
 
-	if (!err) {
-		for (i = 0; i < vectors; i++)
-			ioa_cfg->vectors_info[i].vec = ioa_cfg->pdev->irq + i;
-		ioa_cfg->nvectors = vectors;
-	}
+	for (i = 0; i < vectors; i++)
+		ioa_cfg->vectors_info[i].vec = ioa_cfg->pdev->irq + i;
+	ioa_cfg->nvectors = vectors;
 
-	return err;
+	return 0;
 }
 
 static void name_msi_vectors(struct ipr_ioa_cfg *ioa_cfg)
@@ -9426,7 +9415,7 @@ static irqreturn_t ipr_test_intr(int irq
  * ipr_test_msi - Test for Message Signaled Interrupt (MSI) support.
  * @pdev:		PCI device struct
  *
- * Description: The return value from pci_enable_msi() can not always be
+ * Description: The return value from pci_enable_msi_range() can not always be
  * trusted.  This routine sets up and initiates a test interrupt to determine
  * if the interrupt is received via the ipr_test_intr() service routine.
  * If the tests fails, the driver will fall back to LSI.

-- 

[PATCH RESEND 6/7] ipr: Format HCAM overlay ID 0x21

[wenxiong]

[-- Attachment #1: overlay_21 --]
[-- Type: text/plain, Size: 3897 bytes --]

This patch adds formatting error overlay 0x21 to improve debug capabilities.

Signed-off-by: Wen Xiong <wenxiong@linux.vnet.ibm.com>

---
 drivers/scsi/ipr.c |   39 +++++++++++++++++++++++++++++++++++++++
 drivers/scsi/ipr.h |   14 ++++++++++++++
 2 files changed, 53 insertions(+)

Index: b/drivers/scsi/ipr.c
===================================================================
--- a/drivers/scsi/ipr.c	2014-03-11 17:18:20.169018184 -0500
+++ b/drivers/scsi/ipr.c	2014-03-11 17:20:48.289943030 -0500
@@ -2368,6 +2368,42 @@ static void ipr_log_generic_error(struct
 }
 
 /**
+ * ipr_log_sis64_device_error - Log a cache error.
+ * @ioa_cfg:	ioa config struct
+ * @hostrcb:	hostrcb struct
+ *
+ * Return value:
+ * 	none
+ **/
+static void ipr_log_sis64_device_error(struct ipr_ioa_cfg *ioa_cfg,
+					 struct ipr_hostrcb *hostrcb)
+{
+	struct ipr_hostrcb_type_21_error *error;
+	char buffer[IPR_MAX_RES_PATH_LENGTH];
+
+	error = &hostrcb->hcam.u.error64.u.type_21_error;
+
+	ipr_err("-----Failing Device Information-----\n");
+	ipr_err("World Wide Unique ID: %08X%08X%08X%08X\n",
+		be32_to_cpu(error->wwn[0]), be32_to_cpu(error->wwn[1]),
+		 be32_to_cpu(error->wwn[2]), be32_to_cpu(error->wwn[3]));
+	ipr_err("Device Resource Path: %s\n",
+		__ipr_format_res_path(error->res_path,
+				      buffer, sizeof(buffer)));
+	error->primary_problem_desc[sizeof(error->primary_problem_desc) - 1] = '\0';
+	error->second_problem_desc[sizeof(error->second_problem_desc) - 1] = '\0';
+	ipr_err("Primary Problem Description: %s\n", error->primary_problem_desc);
+	ipr_err("Secondary Problem Description:  %s\n", error->second_problem_desc);
+	ipr_err("SCSI Sense Data: \n");
+	ipr_log_hex_data(ioa_cfg, error->sense_data, sizeof(error->sense_data));
+	ipr_err("SCSI Command Descriptor Block: \n");
+	ipr_log_hex_data(ioa_cfg, error->cdb, sizeof(error->cdb));
+
+	ipr_err("Additional IOA Data: \n");
+	ipr_log_hex_data(ioa_cfg, error->ioa_data, be32_to_cpu(error->length_of_error));
+}
+
+/**
  * ipr_get_error - Find the specfied IOASC in the ipr_error_table.
  * @ioasc:	IOASC
  *
@@ -2468,6 +2504,9 @@ static void ipr_handle_log_data(struct i
 	case IPR_HOST_RCB_OVERLAY_ID_20:
 		ipr_log_fabric_error(ioa_cfg, hostrcb);
 		break;
+	case IPR_HOST_RCB_OVERLAY_ID_21:
+		ipr_log_sis64_device_error(ioa_cfg, hostrcb);
+		break;
 	case IPR_HOST_RCB_OVERLAY_ID_23:
 		ipr_log_sis64_config_error(ioa_cfg, hostrcb);
 		break;
Index: b/drivers/scsi/ipr.h
===================================================================
--- a/drivers/scsi/ipr.h	2014-03-11 17:13:24.279949356 -0500
+++ b/drivers/scsi/ipr.h	2014-03-11 17:20:48.289943030 -0500
@@ -899,6 +899,18 @@ struct ipr_hostrcb_type_01_error {
 	__be32 ioa_data[236];
 }__attribute__((packed, aligned (4)));
 
+struct ipr_hostrcb_type_21_error {
+	__be32 wwn[4];
+	u8 res_path[8];
+	u8 primary_problem_desc[32];
+	u8 second_problem_desc[32];
+	__be32 sense_data[8];
+	__be32 cdb[4];
+	__be32 residual_trans_length;
+	__be32 length_of_error;
+	__be32 ioa_data[236];
+}__attribute__((packed, aligned (4)));
+
 struct ipr_hostrcb_type_02_error {
 	struct ipr_vpd ioa_vpd;
 	struct ipr_vpd cfc_vpd;
@@ -1128,6 +1140,7 @@ struct ipr_hostrcb64_error {
 		struct ipr_hostrcb_type_ff_error type_ff_error;
 		struct ipr_hostrcb_type_12_error type_12_error;
 		struct ipr_hostrcb_type_17_error type_17_error;
+		struct ipr_hostrcb_type_21_error type_21_error;
 		struct ipr_hostrcb_type_23_error type_23_error;
 		struct ipr_hostrcb_type_24_error type_24_error;
 		struct ipr_hostrcb_type_30_error type_30_error;
@@ -1171,6 +1184,7 @@ struct ipr_hcam {
 #define IPR_HOST_RCB_OVERLAY_ID_16				0x16
 #define IPR_HOST_RCB_OVERLAY_ID_17				0x17
 #define IPR_HOST_RCB_OVERLAY_ID_20				0x20
+#define IPR_HOST_RCB_OVERLAY_ID_21				0x21
 #define IPR_HOST_RCB_OVERLAY_ID_23				0x23
 #define IPR_HOST_RCB_OVERLAY_ID_24				0x24
 #define IPR_HOST_RCB_OVERLAY_ID_26				0x26

-- 

[PATCH RESEND 7/7] ipr: Add new CCIN definition for Grand Canyon support

[wenxiong]

[-- Attachment #1: grand_canyon_id --]
[-- Type: text/plain, Size: 1560 bytes --]

Add the appropriate definition and table entry for new hardware support.

Signed-off-by: Wen Xiong <wenxiong@linux.vnet.ibm.com>

---
 drivers/scsi/ipr.c |    6 ++++++
 drivers/scsi/ipr.h |    3 +++
 2 files changed, 9 insertions(+)

Index: b/drivers/scsi/ipr.c
===================================================================
--- a/drivers/scsi/ipr.c	2014-03-11 17:20:48.289943030 -0500
+++ b/drivers/scsi/ipr.c	2014-03-11 17:21:01.018707528 -0500
@@ -10123,6 +10123,12 @@ static struct pci_device_id ipr_pci_tabl
 		PCI_VENDOR_ID_IBM, IPR_SUBS_DEV_ID_57EF, 0, 0, 0 },
 	{ PCI_VENDOR_ID_IBM, PCI_DEVICE_ID_IBM_CROCODILE,
 		PCI_VENDOR_ID_IBM, IPR_SUBS_DEV_ID_57F0, 0, 0, 0 },
+	{ PCI_VENDOR_ID_IBM, PCI_DEVICE_ID_IBM_CROCODILE,
+		PCI_VENDOR_ID_IBM, IPR_SUBS_DEV_ID_2CCA, 0, 0, 0 },
+	{ PCI_VENDOR_ID_IBM, PCI_DEVICE_ID_IBM_CROCODILE,
+		PCI_VENDOR_ID_IBM, IPR_SUBS_DEV_ID_2CD2, 0, 0, 0 },
+	{ PCI_VENDOR_ID_IBM, PCI_DEVICE_ID_IBM_CROCODILE,
+		PCI_VENDOR_ID_IBM, IPR_SUBS_DEV_ID_2CCD, 0, 0, 0 },
 	{ }
 };
 MODULE_DEVICE_TABLE(pci, ipr_pci_table);
Index: b/drivers/scsi/ipr.h
===================================================================
--- a/drivers/scsi/ipr.h	2014-03-11 17:20:48.289943030 -0500
+++ b/drivers/scsi/ipr.h	2014-03-11 17:21:01.018707528 -0500
@@ -108,6 +108,9 @@
 #define IPR_SUBS_DEV_ID_57EE    0x049A
 #define IPR_SUBS_DEV_ID_57EF    0x049B
 #define IPR_SUBS_DEV_ID_57F0    0x049C
+#define IPR_SUBS_DEV_ID_2CCA	0x04C7
+#define IPR_SUBS_DEV_ID_2CD2	0x04C8
+#define IPR_SUBS_DEV_ID_2CCD	0x04C9
 #define IPR_NAME				"ipr"
 
 /*

--