[Bug 15688] mptsas & poor performance

[bugzilla-daemon@bugzilla.kernel.org]

https://bugzilla.kernel.org/show_bug.cgi?id=15688





--- Comment #3 from Brian Sullivan <bexamous@gmail.com>  2010-04-08 15:36:49 ---
I think the only changes I made to 4.18.00.00 was to just not use mptlan
(since I have no need and it wouldn't compile on 2.6.34) and make a few
changes to mptsas.c (only to get it to compile on 2.6.34).

diff ./mptlinux-4.18.00.00/dkms.conf /usr/src/mptlinux-4.18.00.00/dkms.conf
13d12
< MAKE[6]="make -C ${kernel_source_dir}
SUBDIRS=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build modules"
28,29c27,28
< BUILT_MODULE_NAME[3]="mptlan"
< DEST_MODULE_NAME[3]="mptlan"
---
> BUILT_MODULE_NAME[3]="mptspi"
> DEST_MODULE_NAME[3]="mptspi"
30a30,31
> #MODULES_CONF_OBSOLETES[3]="mptbase,mptscsih,mptctl,mptlan"
> MODULES_CONF_ALIAS_TYPE[3]="scsi_hostadapter"
32,33c33,34
< BUILT_MODULE_NAME[4]="mptspi"
< DEST_MODULE_NAME[4]="mptspi"
---
> BUILT_MODULE_NAME[4]="mptsas"
> DEST_MODULE_NAME[4]="mptsas"
35d35
< #MODULES_CONF_OBSOLETES[4]="mptbase,mptscsih,mptctl,mptlan"
38,39c38,39
< BUILT_MODULE_NAME[5]="mptsas"
< DEST_MODULE_NAME[5]="mptsas"
---
> BUILT_MODULE_NAME[5]="mptfc"
> DEST_MODULE_NAME[5]="mptfc"
41,45d40
< MODULES_CONF_ALIAS_TYPE[5]="scsi_hostadapter"
<
< BUILT_MODULE_NAME[6]="mptfc"
< DEST_MODULE_NAME[6]="mptfc"
< DEST_MODULE_LOCATION[6]="/kernel/drivers/message/fusion"
47a43
>
diff ./mptlinux-4.18.00.00/Makefile /usr/src/mptlinux-4.18.00.00/Makefile
2c2
< # LSI Logic  mpt fusion
---
> # LSI mpt fusion
19c19
< obj-$(CONFIG_FUSION_LAN)    += mptlan.o
---
> #obj-$(CONFIG_FUSION_LAN)    += mptlan.o
diff ./mptlinux-4.18.00.00/mptsas.c /usr/src/mptlinux-4.18.00.00/mptsas.c
2437,2438c2437,2438
<             ioc->name, __func__, req->bio->bi_vcnt, req->data_len,
<             rsp->bio->bi_vcnt, rsp->data_len);
---
>             ioc->name, __func__, req->bio->bi_vcnt, blk_rq_bytes(req),
>             rsp->bio->bi_vcnt, blk_rq_bytes(rsp));
2455c2455
<     smpreq->RequestDataLength = cpu_to_le16(req->data_len - 4);
---
>     smpreq->RequestDataLength = cpu_to_le16(blk_rq_bytes(req) - 4);
2485c2485
<     flagsLength |= (req->data_len - 4);
---
>     flagsLength |= (blk_rq_bytes(req) - 4);
2488c2488
<                       req->data_len, PCI_DMA_BIDIRECTIONAL);
---
>                       blk_rq_bytes(req), PCI_DMA_BIDIRECTIONAL);
2501c2501
<     flagsLength |= rsp->data_len + 4;
---
>     flagsLength |= blk_rq_bytes(rsp) + 4;
2503c2503
<                       rsp->data_len, PCI_DMA_BIDIRECTIONAL);
---
>                       blk_rq_bytes(rsp), PCI_DMA_BIDIRECTIONAL);
2534,2535c2534,2535
<         req->data_len = 0;
<         rsp->data_len -= smprep->ResponseDataLength;
---
>         req->resid_len = 0;
>         rsp->resid_len -= smprep->ResponseDataLength;
2544c2544
<         pci_unmap_single(ioc->pcidev, dma_addr_out, req->data_len,
---
>         pci_unmap_single(ioc->pcidev, dma_addr_out, blk_rq_bytes(req),
2547c2547
<         pci_unmap_single(ioc->pcidev, dma_addr_in, rsp->data_len,
---
>         pci_unmap_single(ioc->pcidev, dma_addr_in, blk_rq_bytes(rsp),


BTW This was Bonnie++ output when btrfs was still empty, 388MB/sec write
526MB/sec read:
Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec
%CP
nine            24G   259  95 388802  57 127153  51   221  94 526553  66
167.9 331
Latency             30004us    1451ms     378ms   23627us     118ms
104ms
Version  1.96       ------Sequential Create------ --------Random
Create--------
nine                -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec
%CP
                 16  8406  24 +++++ +++ 11176  49 14249  40 +++++ +++ 11370
49
Latency               498us     441us     926us     601us      43us
1628us
1.96,1.96,nine,1,1270361591,24G,,259,95,388802,57,127153,51,221,94,526553,66,167.9,331,16,,,,,8406,24,+++++,+++,11176,49,14249,40,+++++,+++,11370,49,30004us,1451ms,378ms,23627us,118ms,104ms,498us,441us,926us,601us,43us,1628us

The <200MB/sec and 395MB/sec i was referring to was just using dd to write
zeros.  Later on I ran bonnie and saved this.

Also system:
Tyan mobo wiht two E5410 CPUs, 12GB RAM, using onboard LSI 1068E.  The LSI
connects to a HP SAS Expander.  All drives are connected to the HP SAS
Expander.  The drives are all 2TB WD GP drives WD20EADS, not the newer EARS
version.


On Thu, Apr 8, 2010 at 3:31 AM, <bugzilla-daemon@bugzilla.kernel.org> wrote:

> https://bugzilla.kernel.org/show_bug.cgi?id=15688
>
>
>
>
>
> --- Comment #2 from kdesai <kashyap.desai@lsi.com>  2010-04-08 10:31:17
> ---
> Andrew,
>
> Today I tried same steps as mentioned by you. In my case I am able to see
> both
> the drivers
> Performance is similar.
>
> 3.4.14 is driver version available at kernel.org
> 4.22.00.00 is driver LSI internally uses. [4.18.00.00 does not have support
> for
> 2.6.34 kernel]
> I guess you must have done some changes to make sure 4.18.00.00 works with
> 2.6.34-020634rc1.
>
> Both the case I am getting 190~210MB/sec.
>
> Can you help me to reproduce ~395MB/sec using 4.18.00.00 ?
>
>
> ~Kashyap
>
>
>
>
> > -----Original Message-----
> > From: linux-scsi-owner@vger.kernel.org [mailto:linux-scsi-
> > owner@vger.kernel.org] On Behalf Of Andrew Morton
> > Sent: Tuesday, April 06, 2010 2:05 AM
> > To: linux-scsi@vger.kernel.org; Moore, Eric
> > Cc: bugzilla-daemon@bugzilla.kernel.org; bugme-
> > daemon@bugzilla.kernel.org; bexamous@gmail.com
> > Subject: Re: [Bugme-new] [Bug 15688] New: mptsas & poor performance
> >
> >
> > (switched to email.  Please respond via emailed reply-to-all, not via
> > the
> > bugzilla web interface).
> >
> > scsi_drivers-other reports don't appear to be coming out on the
> > linux-scsi list.
> >
> > On Sat, 3 Apr 2010 21:16:31 GMT
> > bugzilla-daemon@bugzilla.kernel.org wrote:
> >
> > > https://bugzilla.kernel.org/show_bug.cgi?id=15688
> > >
> > >            Summary: mptsas & poor performance
> > >            Product: SCSI Drivers
> > >            Version: 2.5
> > >     Kernel Version: 2.6.34-020634rc1
> > >           Platform: All
> > >         OS/Version: Linux
> > >               Tree: Mainline
> > >             Status: NEW
> > >           Severity: normal
> > >           Priority: P1
> > >          Component: Other
> > >         AssignedTo: scsi_drivers-other@kernel-bugs.osdl.org
> > >         ReportedBy: bexamous@gmail.com
> > >         Regression: No
> > >
> > >
> > > The mptsas module from LSI's website, v4.18.00.00, performs much
> > better than
> > > the mptsas module included in the kernel, v3.something.
> > >
> > > I'm only comparing sequential writes (dd if=/dev/zero of=zeros
> > bs=1M), but...
> > >
> > > 6 drive MD RAID5 with fresh btrfs, writing 10GB of zeros with dd,
> > > kernel version:  <200MB/sec
> > > LSI's v4.18.00.00: 395MB/sec
> > >
> > > Pretty big difference.  I've gone back and forth a few times,
> > > enabling/disabling write cache on the drives, enabling/disabling ioc,
> > > enabling/disabling filesystem barriers...  I can make small changes
> > in
> > > performance but nothing compares to simply updating to v4.18.00.00.
> > >
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-scsi"
> > in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You reported the bug.
>

-- 
Configure bugmail: https://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.