[PATCH]: Fusion-MPT slowness workaround

[PATCH]: Fusion-MPT slowness workaround

[Moore, Eric Dean]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

On Tuesday, November 01, 2005 11:12 AM, Martin Devera wrote: 
> > NACK. Disabling DV is not the solution.
> > 
> > I have a fix I will post sometime this week.
> 
> excellent. Just to make it clear, I posted this one as quick 
> workaround not
> as final solution attempt.
> Please consider CCing me when the patch will be available - 
> I'll be pleased
> to test it :-)
> 

Here is a patch I would like to have tested to address the
performance issue many have posted to the forum.

I've added spin-lock to control domain validation, and 
alternate controller bringup which are occuring at the same time,
and causing dv to fail.  Thanks to Kenneth Chen and Bodo Stroesser.

Eric Moore
LSI Logic



[-- Attachment #2: performance.patch --]
[-- Type: application/octet-stream, Size: 4194 bytes --]

diff -uarN b/drivers/message/fusion/mptbase.c a/drivers/message/fusion/mptbase.c
--- b/drivers/message/fusion/mptbase.c	2005-10-27 18:02:08.000000000 -0600
+++ a/drivers/message/fusion/mptbase.c	2005-11-01 12:25:35.000000000 -0700
@@ -1121,6 +1121,38 @@
 
 /*=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
 /*
+ *	mpt_bringup_adapter - This is a wrapper function for mpt_do_ioc_recovery
+ *	@ioc: Pointer to MPT adapter structure
+ *	@sleepFlag: Use schedule if CAN_SLEEP else use udelay.
+ *
+ *	This routine performs all the steps necessary to bring the IOC
+ *	to a OPERATIONAL state.
+ *
+ *      Special Note: This function was added with spin lock's so as to allow
+ *      the dv(domain validation) work thread to succeed on the other channel
+ *      that maybe occuring at the same time when this function is called.
+ *      Without this lock, the dv would fail when message frames were
+ *      requested during hba bringup on the alternate ioc.
+ */
+static int
+mpt_bringup_adapter(MPT_ADAPTER *ioc, int sleepFlag)
+{
+int r;
+
+	if(ioc->alt_ioc)
+		spin_lock(&ioc->alt_ioc->initializing_hba_lock);
+
+	r = mpt_do_ioc_recovery(ioc, MPT_HOSTEVENT_IOC_BRINGUP,
+	    CAN_SLEEP);
+
+	if(ioc->alt_ioc)
+		spin_unlock(&ioc->alt_ioc->initializing_hba_lock);
+
+return r;
+}
+
+/*=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
+/*
  *	mpt_attach - Install a PCI intelligent MPT adapter.
  *	@pdev: Pointer to pci_dev structure
  *
@@ -1187,6 +1219,7 @@
 	ioc->pcidev = pdev;
 	ioc->diagPending = 0;
 	spin_lock_init(&ioc->diagLock);
+	spin_lock_init(&ioc->initializing_hba_lock);
 
 	/* Initialize the event logging.
 	 */
@@ -1409,8 +1442,7 @@
 	 */
 	mpt_detect_bound_ports(ioc, pdev);
 
-	if ((r = mpt_do_ioc_recovery(ioc,
-	  MPT_HOSTEVENT_IOC_BRINGUP, CAN_SLEEP)) != 0) {
+	if ((r = mpt_bringup_adapter(ioc, CAN_SLEEP)) != 0){
 		printk(KERN_WARNING MYNAM
 		  ": WARNING - %s did not initialize properly! (%d)\n",
 		  ioc->name, r);
@@ -6300,7 +6332,6 @@
 EXPORT_SYMBOL(mpt_free_fw_memory);
 EXPORT_SYMBOL(mptbase_sas_persist_operation);
 
-
 /*=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
 /*
  *	fusion_init - Fusion MPT base driver initialization routine.
diff -uarN b/drivers/message/fusion/mptbase.h a/drivers/message/fusion/mptbase.h
--- b/drivers/message/fusion/mptbase.h	2005-10-27 18:02:08.000000000 -0600
+++ a/drivers/message/fusion/mptbase.h	2005-11-01 12:08:50.000000000 -0700
@@ -601,6 +601,7 @@
 	int			 DoneCtx;
 	int			 TaskCtx;
 	int			 InternalCtx;
+	spinlock_t		 initializing_hba_lock;
 	struct list_head	 list;
 	struct net_device	*netdev;
 	struct list_head	 sas_topology;
diff -uarN b/drivers/message/fusion/mptscsih.c a/drivers/message/fusion/mptscsih.c
--- b/drivers/message/fusion/mptscsih.c	2005-10-27 18:02:08.000000000 -0600
+++ a/drivers/message/fusion/mptscsih.c	2005-11-01 12:24:27.000000000 -0700
@@ -3549,6 +3549,9 @@
 
 	ddvprintk((MYIOC_s_WARN_FMT "Timer Expired! Cmd %p\n", hd->ioc->name, hd->cmdPtr));
 
+	if(spin_is_locked(&hd->ioc->initializing_hba_lock))
+		spin_unlock(&hd->ioc->initializing_hba_lock);
+
 	if (hd->cmdPtr) {
 		MPIHeader_t *cmd = (MPIHeader_t *)hd->cmdPtr;
 
@@ -3690,7 +3693,7 @@
 	in_isr = in_interrupt();
 	if (in_isr) {
 		dprintk((MYIOC_s_WARN_FMT "Internal SCSI IO request not allowed in ISR context!\n",
-       				hd->ioc->name));
+				hd->ioc->name));
 		return -EPERM;
 	}
 
@@ -3789,11 +3792,14 @@
 		return -EFAULT;
 	}
 
+	spin_lock(&hd->ioc->initializing_hba_lock);
+
 	/* Get and Populate a free Frame
 	 */
 	if ((mf = mpt_get_msg_frame(hd->ioc->InternalCtx, hd->ioc)) == NULL) {
 		ddvprintk((MYIOC_s_WARN_FMT "No msg frames!\n",
 					hd->ioc->name));
+		spin_unlock(&hd->ioc->initializing_hba_lock);
 		return -EBUSY;
 	}
 
@@ -3883,6 +3889,7 @@
 	add_timer(&hd->timer);
 	mpt_put_msg_frame(hd->ioc->InternalCtx, hd->ioc, mf);
 	wait_event(hd->scandv_waitq, hd->scandv_wait_done);
+	spin_unlock(&hd->ioc->initializing_hba_lock);
 
 	if (hd->pLocal) {
 		rc = hd->pLocal->completion;

RE: [PATCH]: Fusion-MPT slowness workaround

[Chen, Kenneth W]

Moore, Eric Dean wrote on Tuesday, November 01, 2005 11:35 AM
> Here is a patch I would like to have tested to address the
> performance issue many have posted to the forum.
> 
> I've added spin-lock to control domain validation, and 
> alternate controller bringup which are occuring at the same time,
> and causing dv to fail.  Thanks to Kenneth Chen and Bodo Stroesser.

With the patch, Kernel hung at boot (2.6.14 is used).

boot log as following:

[other stuff snipped] ....
Fusion MPT base driver 3.03.03
Copyright (c) 1999-2005 LSI Logic Corporation
Fusion MPT SPI Host driver 3.03.03
GSI 28 (level, low) -> CPU 0 (0xc618) vector 49
ACPI: PCI Interrupt 0000:06:02.0[A] -> GSI 28 (level, low) -> IRQ 49
mptbase: Initiating ioc0 bringup
ioc0: 53C1030: Capabilities={Initiator}
scsi0 : ioc0: LSI53C1030, FwRev=01030a00h, Ports=1, MaxQ=255, IRQ=49
  Vendor: MAXTOR    Model: ATLAS10K3_18_SCA  Rev: B000
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sda: 35916548 512-byte hdwr sectors (18389 MB)
SCSI device sda: drive cache: write back
SCSI device sda: 35916548 512-byte hdwr sectors (18389 MB)
SCSI device sda: drive cache: write back
 sda: sda1 sda2 sda3
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
  Vendor: QUANTUM   Model: ATLAS IV 9 SCA    Rev: 0B0B
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sdb: 17942584 512-byte hdwr sectors (9187 MB)
SCSI device sdb: drive cache: write back
SCSI device sdb: 17942584 512-byte hdwr sectors (9187 MB)
SCSI device sdb: drive cache: write back
 sdb: sdb1
Attached scsi disk sdb at scsi0, channel 0, id 1, lun 0
  Vendor: SEAGATE   Model: ST336752LC        Rev: 0004
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sdc: 71687369 512-byte hdwr sectors (36704 MB)
SCSI device sdc: drive cache: write back
SCSI device sdc: 71687369 512-byte hdwr sectors (36704 MB)
SCSI device sdc: drive cache: write back
 sdc: sdc1 sdc2
Attached scsi disk sdc at scsi0, channel 0, id 2, lun 0
  Vendor: ESG-SHV   Model: SCA HSBP M17      Rev: 1.0D
  Type:   Processor                          ANSI SCSI revision: 02

RE: [PATCH]: Fusion-MPT slowness workaround

[Moore, Eric Dean]

[-- Attachment #1: Type: text/plain, Size: 483 bytes --]

On Tuesday, November 01, 2005 2:39 PM, Chen, Kenneth W wrote:
> > 
> > I've added spin-lock to control domain validation, and 
> > alternate controller bringup which are occuring at the same time,
> > and causing dv to fail.  Thanks to Kenneth Chen and Bodo Stroesser.
> 
> With the patch, Kernel hung at boot (2.6.14 is used).
> 

Try this patch instead.  Use it over 2.6.14, not over the
previous post.

This patch is one that Bodo provided.  Let me know the results.

Eric Moore


[-- Attachment #2: performance-take2.patch --]
[-- Type: application/octet-stream, Size: 2763 bytes --]

diff -uarN b/drivers/message/fusion/mptbase.c a/drivers/message/fusion/mptbase.c
--- b/drivers/message/fusion/mptbase.c	2005-10-27 18:02:08.000000000 -0600
+++ a/drivers/message/fusion/mptbase.c	2005-11-01 15:03:12.000000000 -0700
@@ -118,6 +118,7 @@
 static int	last_drv_idx = -1;
 
 static DECLARE_WAIT_QUEUE_HEAD(mpt_waitq);
+static DECLARE_WAIT_QUEUE_HEAD(mpt_host_bringup_waitq);
 
 /*=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
 /*
@@ -1659,6 +1660,7 @@
 						ioc->alt_ioc->name));
 				CHIPREG_WRITE32(&ioc->alt_ioc->chip->IntMask, ~(MPI_HIM_RIM));
 				ioc->alt_ioc->active = 1;
+				wake_up(&mpt_host_bringup_waitq);
 			}
 
 		} else {
@@ -1783,6 +1785,7 @@
 				ioc->alt_ioc->name));
 		CHIPREG_WRITE32(&ioc->alt_ioc->chip->IntMask, ~(MPI_HIM_RIM));
 		ioc->alt_ioc->active = 1;
+		wake_up(&mpt_host_bringup_waitq);
 	}
 
 	/*  Enable MPT base driver management of EventNotification
@@ -1893,6 +1896,15 @@
 	return ret;
 }
 
+int
+mpt_wait_active(MPT_ADAPTER *ioc)
+{
+	if (ioc->active == 0)
+		return wait_event_timeout(mpt_host_bringup_waitq,
+		    ioc->active == 1, HZ*15) <=0;
+	return 0;
+}
+
 /*=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
 /*
  *	mpt_detect_bound_ports - Search for PCI bus/dev_function
@@ -6299,7 +6311,7 @@
 EXPORT_SYMBOL(mpt_alloc_fw_memory);
 EXPORT_SYMBOL(mpt_free_fw_memory);
 EXPORT_SYMBOL(mptbase_sas_persist_operation);
-
+EXPORT_SYMBOL(mpt_wait_active);
 
 /*=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
 /*
diff -uarN b/drivers/message/fusion/mptbase.h a/drivers/message/fusion/mptbase.h
--- b/drivers/message/fusion/mptbase.h	2005-10-27 18:02:08.000000000 -0600
+++ a/drivers/message/fusion/mptbase.h	2005-11-01 14:59:05.000000000 -0700
@@ -966,6 +966,7 @@
 extern int	 mpt_resume(struct pci_dev *pdev);
 #endif
 extern int	 mpt_register(MPT_CALLBACK cbfunc, MPT_DRIVER_CLASS dclass);
+extern int	 mpt_wait_active(MPT_ADAPTER *ioc);
 extern void	 mpt_deregister(int cb_idx);
 extern int	 mpt_event_register(int cb_idx, MPT_EVHANDLER ev_cbfunc);
 extern void	 mpt_event_deregister(int cb_idx);
diff -uarN b/drivers/message/fusion/mptscsih.c a/drivers/message/fusion/mptscsih.c
--- b/drivers/message/fusion/mptscsih.c	2005-10-27 18:02:08.000000000 -0600
+++ a/drivers/message/fusion/mptscsih.c	2005-11-01 15:01:16.000000000 -0700
@@ -3789,6 +3789,12 @@
 		return -EFAULT;
 	}
 
+	if (mpt_wait_active(hd->ioc)) {
+		ddvprintk((MYIOC_s_WARN_FMT "wait_event: no msg frames!\n",
+		    hd->ioc->name));
+		return -EBUSY;
+	}
+
 	/* Get and Populate a free Frame
 	 */
 	if ((mf = mpt_get_msg_frame(hd->ioc->InternalCtx, hd->ioc)) == NULL) {

RE: [PATCH]: Fusion-MPT slowness workaround

[Chen, Kenneth W]

Moore, Eric Dean wrote on Tuesday, November 01, 2005 2:09 PM
> On Tuesday, November 01, 2005 2:39 PM, Chen, Kenneth W wrote:
> > > 
> > > I've added spin-lock to control domain validation, and 
> > > alternate controller bringup which are occuring at the same time,
> > > and causing dv to fail.  Thanks to Kenneth Chen and Bodo Stroesser.
> > 
> > With the patch, Kernel hung at boot (2.6.14 is used).
> > 
> 
> Try this patch instead.  Use it over 2.6.14, not over the
> previous post.
> 
> This patch is one that Bodo provided.  Let me know the results.


Looks OK, kernel boots and DV appears to be running correctly.

- Ken

Re: [PATCH]: Fusion-MPT slowness workaround

[Martin Devera]

Chen, Kenneth W wrote:
> Moore, Eric Dean wrote on Tuesday, November 01, 2005 2:09 PM
> 
>>On Tuesday, November 01, 2005 2:39 PM, Chen, Kenneth W wrote:
>>
>>>>I've added spin-lock to control domain validation, and 
>>>>alternate controller bringup which are occuring at the same time,
>>>>and causing dv to fail.  Thanks to Kenneth Chen and Bodo Stroesser.
>>>
>>>With the patch, Kernel hung at boot (2.6.14 is used).
>>>
>>
>>Try this patch instead.  Use it over 2.6.14, not over the
>>previous post.
>>
>>This patch is one that Bodo provided.  Let me know the results.
> 
> Looks OK, kernel boots and DV appears to be running correctly.

Not here. Both performance*patch patches boot ok here but I/O remains
slow in both cases. I double checked against mixing patches but only
disabling DV works here.
Tested with 2.6.14 kernel, no modules.

regards, Martin