* Slow I/O performance on SAS1064
@ 2014-03-05 17:50 Markus
2014-03-05 18:21 ` Mark Knecht
0 siblings, 1 reply; 6+ messages in thread
From: Markus @ 2014-03-05 17:50 UTC (permalink / raw)
To: Nagalakshmi Nandigama, Sreekanth Reddy, support,
DL-MPTFusionLinux, linux-scsi, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 5346 bytes --]
Hi
I have problem with SATA disks at my Sunfire v245 and its lsi controller they are very slow.
So I test it with dd and while dd in a nother terminal i see dstat results of sda:
Copy from ramdisk to sda.
=== Test ext4 ===
DD
root@outpost:/ramdisk# dd if=debian-7.4.0-sparc-netinst.iso of=/root/test.dd
353344+0 records in
353344+0 records out
180912128 bytes (181 MB) copied, 24.6384 s, 7.3 MB/s
Dstat:
----total-cpu-usage---- --dsk/sda-- ---io/sda--
usr sys idl wai hiq siq| read writ| read writ
1 0 100 0 0 0| 0 0 | 0 0
0 0 100 0 0 0| 0 0 | 0 0
2 1 97 0 0 0| 0 0 | 0 0
1 5 94 0 0 0| 0 0 | 0 0
7 43 50 0 0 0| 0 0 | 0 0
7 43 50 0 0 0| 0 0 | 0 0
10 40 50 0 0 0| 0 0 | 0 0
12 45 41 3 0 0| 0 0 | 0 0
6 42 36 17 0 1| 0 512k| 0 1.00
0 0 0 99 0 0| 0 4096k| 0 8.00
0 2 0 98 0 0| 0 5120k| 0 10.0
0 1 0 99 0 0| 0 7680k| 0 15.0
1 0 0 100 0 0| 0 5632k| 0 11.0
0 0 0 99 0 0| 0 7168k| 0 14.0
1 1 0 99 0 0| 0 6656k| 0 13.0
0 0 0 100 0 0| 0 6144k| 0 12.0
1 0 0 100 0 0| 0 7680k| 0 15.0
0 1 22 78 0 0| 0 4608k| 0 9.00
0 0 50 50 0 0| 0 3072k| 0 6.00
1 0 50 50 0 1| 0 7168k| 0 14.0
1 1 50 49 0 0|4096B 5120k|1.00 10.0
0 0 50 50 0 0| 0 8704k| 0 17.0
0 0 50 49 0 0| 0 7680k| 0 15.0
0 1 50 50 0 0| 0 7168k| 0 14.0
1 0 49 49 0 0| 0 7168k| 0 14.0
1 0 50 50 0 0| 0 7680k| 0 15.0
0 0 50 50 0 0| 0 7168k| 0 14.0
0 0 50 50 0 0| 0 2560k| 0 5.00
0 1 30 69 0 0| 0 3624k| 0 8.00
1 0 0 99 0 0| 0 4100k| 0 9.00
0 0 0 100 0 0| 0 5632k| 0 11.0
0 0 0 99 0 0| 0 3584k| 0 7.00
1 0 0 100 0 0| 0 4608k| 0 9.00
0 0 0 99 0 1| 0 5120k| 0 10.0
1 0 4 96 0 0| 0 2560k| 0 6.00
0 0 50 50 0 0| 0 3072k| 0 6.00
0 0 49 50 0 0| 0 5120k| 0 10.0
1 0 50 50 0 0| 0 5120k| 0 10.0
0 0 50 50 0 0| 0 4096k| 0 8.00
1 0 50 50 0 0| 0 5120k| 0 10.0
0 0 50 50 0 0| 0 5120k| 0 10.0
0 1 57 42 0 0| 0 1125k| 0 17.0
0 0 100 0 0 0| 0 0 | 0 0
1 0 100 0 0 0| 0 0 | 0 0
1 0 100 0 0 0| 0 0 | 0 0
0 0 100 0 0 0| 0 0 | 0 0
0 0 100 0 0 0| 0 0 | 0 0
-------------------------------------------------------------------------------
Information:
Kernel 3.13.5 (stable release)
Debian7 (stable)
hdparm v9.39
Controller
LSI SAS1064
cat /proc/mpt/ioc0/info
ioc0:
ProductID = 0x2701 (LSISAS1064 A3)
FWVersion = 0x01080400
MsgVersion = 0x0105
FirstWhoInit = 0x00
EventState = 0x00
CurrentHostMfaHighAddr = 0x00000000
CurrentSenseBufferHighAddr = 0x00000000
MaxChainDepth = 0x60 frames
MinBlockSize = 0x20 bytes
RequestFrames @ 0xfffffc137f602800 (Dma @ 0x00000000c000a800)
{CurReqSz=128} x {CurReqDepth=511} = 65408 bytes ^= 0x10000
{MaxReqSz=128} {MaxReqDepth=511}
Frames @ 0xfffffc137f600000 (Dma @ 0x00000000c0008000)
{CurRepSz=80} x {CurRepDepth=128} = 10240 bytes ^= 0x2880
{MaxRepSz=0} {MaxRepDepth=511}
MaxDevices = 63
MaxBuses = 1
PortNumber = 1 (of 1)
cat /proc/mpt/version
mptlinux-3.04.20
Fusion MPT base driver
Fusion MPT SAS host driver
Fusion MPT ioctl driver
-------------------------------------------------------------------------------
Hdparm Information:
hdparm -I /dev/sda
/dev/sda:
SG_IO: bad/missing sense data, sb[]: 70 00 05 00 00 00 00 0a 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ATA device, with non-removable media
Standards:
Likely used: 1
Configuration:
Logical max current
cylinders 0 0
heads 0 0
sectors/track 0 0
--
Logical/Physical Sector size: 512 bytes
device size with M = 1024*1024: 0 MBytes
device size with M = 1000*1000: 0 MBytes
cache/buffer size = unknown
Capabilities:
IORDY not likely
Cannot perform double-word IO
R/W multiple sector transfer: not supported
DMA: not supported
PIO: pio0
-------------------------------------------------------------------------------
Moduls load:
lsmod | grep mpt
mptctl 27359 0
mptsas 45861 5
mptscsih 25765 1 mptsas
mptbase 75530 3 mptctl,mptsas,mptscsih
scsi_transport_sas 21624 1 mptsas
-------------------------------------------------------------------------------
The hdparm result looks like there is somethink not right . There were no features supported but why ?
Greetings Markus
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Slow I/O performance on SAS1064
2014-03-05 17:50 Markus
@ 2014-03-05 18:21 ` Mark Knecht
2014-03-05 21:40 ` markus
0 siblings, 1 reply; 6+ messages in thread
From: Mark Knecht @ 2014-03-05 18:21 UTC (permalink / raw)
To: Markus
Cc: Nagalakshmi Nandigama, Sreekanth Reddy, support,
DL-MPTFusionLinux, linux-scsi, Linux Kernel List
On Wed, Mar 5, 2014 at 9:50 AM, Markus <markus@kola.li> wrote:
<SNIP>
> The hdparm result looks like there is somethink not right . There were no features supported but why ?
Does the HDD have S.M.A.R.T. features? Possibly
smartctl -a /dev/sda
would provide some additional visibility?
- Mark
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Slow I/O performance on SAS1064
2014-03-05 18:21 ` Mark Knecht
@ 2014-03-05 21:40 ` markus
2014-03-06 20:51 ` Mark Knecht
0 siblings, 1 reply; 6+ messages in thread
From: markus @ 2014-03-05 21:40 UTC (permalink / raw)
To: Mark Knecht
Cc: Nagalakshmi Nandigama, Sreekanth Reddy, support,
DL-MPTFusionLinux, linux-scsi, Linux Kernel List
[-- Attachment #1: Type: text/plain, Size: 4819 bytes --]
On Wed, Mar 05, 2014 at 10:21:07AM -0800, Mark Knecht wrote:
> On Wed, Mar 5, 2014 at 9:50 AM, Markus <markus@kola.li> wrote:
> <SNIP>
>
> > The hdparm result looks like there is somethink not right . There were no features supported but why ?
>
>
> Does the HDD have S.M.A.R.T. features? Possibly
>
> smartctl -a /dev/sda
root@outpost:~# smartctl -a /dev/sda
smartctl 5.41 2011-06-09 r3365 [sparc64-linux-3.13.5-mar] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
Probable ATA device behind a SAT layer
Try an additional '-d ata' or '-d sat' argument.
so i try it with scsi.
root@outpost:~# smartctl -d scsi -a /dev/sda
smartctl 5.41 2011-06-09 r3365 [sparc64-linux-3.13.5-mar] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
User Capacity: 2,000,398,934,016 bytes [2.00 TB]
Logical block size: 512 bytes
Logical Unit id: 0x50000394e2380537
Serial number: 7365P4KNT
Device type: disk
Local Time is: Wed Mar 5 22:32:09 2014 CET
Device supports SMART and is Disabled
Temperature Warning Disabled or Not Supported
SMART Health Status: OK
Error Counter logging not supported
Device does not support Self Test logging
>
> would provide some additional visibility?
.....
[ 98.037209] Uniform Multi-Platform E-IDE driver
[ 98.107564] SLAB: Unable to allocate memory on node 0 (gfp=0x40d1)
[ 98.188985] cache: dma-kmalloc-512, object size: 512, order: 0
[ 98.268127] node 0: slabs: 0/0, objs: 0/0, free: 0
[ 98.333538] sr: out of memory.
[ 98.373628] cdrom: Uniform CD-ROM driver Revision: 3.20
[ 98.443390] sr 0:0:0:0: Attached scsi CD-ROM sr0
[ 98.448375] sr 0:0:0:0: Attached scsi generic sg0 type 5
[ 103.198705] scsi2 : ioc0: LSISAS1064 A3, FwRev=01080400h, Ports=1, MaxQ=511, IRQ=10
[ 103.329805] mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 0, phy 0, sas_addr 0x1221000000000000
[ 103.462762] scsi 2:0:0:0: Direct-Access ATA TOSHIBA MQ01ABB2 0U PQ: 0 ANSI: 5
[ 103.571610] scsi 2:0:0:0: Attached scsi generic sg1 type 0
[ 103.645418] mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 1, phy 1, sas_addr 0x1221000001000000
[ 103.777635] scsi 2:0:1:0: Direct-Access ATA TOSHIBA MQ01ABB2 0U PQ: 0 ANSI: 5
[ 103.886607] scsi 2:0:1:0: Attached scsi generic sg2 type 0
[ 103.960517] mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 2, phy 2, sas_addr 0x1221000002000000
[ 104.093109] scsi 2:0:2:0: Direct-Access ATA TOSHIBA MQ01ABB2 0U PQ: 0 ANSI: 5
[ 104.201973] scsi 2:0:2:0: Attached scsi generic sg3 type 0
[ 104.275830] mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 3, phy 3, sas_addr 0x1221000003000000
[ 104.408412] scsi 2:0:3:0: Direct-Access ATA TOSHIBA MQ01ABB2 0U PQ: 0 ANSI: 5
[ 104.517379] scsi 2:0:3:0: Attached scsi generic sg4 type 0
[ 104.618997] sd 2:0:0:0: [sda] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[ 104.721185] sd 2:0:3:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[ 104.823771] sd 2:0:1:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[ 104.824017] sd 2:0:2:0: [sdc] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[ 104.828691] sd 2:0:0:0: [sda] Write Protect is off
[ 104.828697] sd 2:0:0:0: [sda] Mode Sense: 67 00 00 08
[ 104.829940] sd 2:0:2:0: [sdc] Write Protect is off
[ 104.829945] sd 2:0:2:0: [sdc] Mode Sense: 67 00 00 08
[ 104.830488] sd 2:0:3:0: [sdd] Write Protect is off
[ 104.830493] sd 2:0:3:0: [sdd] Mode Sense: 67 00 00 08
[ 104.833225] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 104.834471] sd 2:0:2:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 104.835118] sd 2:0:3:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 105.583175] sd 2:0:1:0: [sdb] Write Protect is off
[ 105.623801] sda: sda1 sda2 sda3
[ 105.651118] sdc: sdc1 sdc9
[ 105.654024] sdd: sdd1 sdd9
[ 105.761966] sd 2:0:1:0: [sdb] Mode Sense: 67 00 00 08
[ 105.769029] sd 2:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 105.775293] sd 2:0:2:0: [sdc] Attached SCSI disk
[ 105.775941] sd 2:0:3:0: [sdd] Attached SCSI disk
[ 106.014357] sd 2:0:0:0: [sda] Attached SCSI disk
[ 106.158864] sdb: sdb1 sdb9
[ 106.205879] sd 2:0:1:0: [sdb] Attached SCSI disk
[ 106.502387] random: nonblocking pool is initialized
[ 106.792978] device-mapper: uevent: version 1.0.3
[ 106.861434] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel@redhat.com
[ 108.085062] bio: create slab <bio-1> at 1
.....
>
> - Mark
Markus
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Slow I/O performance on SAS1064
2014-03-05 21:40 ` markus
@ 2014-03-06 20:51 ` Mark Knecht
2014-03-07 9:46 ` markus
0 siblings, 1 reply; 6+ messages in thread
From: Mark Knecht @ 2014-03-06 20:51 UTC (permalink / raw)
To: markus
Cc: Nagalakshmi Nandigama, Sreekanth Reddy, support,
DL-MPTFusionLinux, linux-scsi, Linux Kernel List
On Wed, Mar 5, 2014 at 1:40 PM, markus <markus@kola.li> wrote:
> On Wed, Mar 05, 2014 at 10:21:07AM -0800, Mark Knecht wrote:
>> On Wed, Mar 5, 2014 at 9:50 AM, Markus <markus@kola.li> wrote:
>> <SNIP>
>>
>> > The hdparm result looks like there is somethink not right . There were no features supported but why ?
>>
>>
>> Does the HDD have S.M.A.R.T. features? Possibly
>>
>> smartctl -a /dev/sda
> root@outpost:~# smartctl -a /dev/sda
> smartctl 5.41 2011-06-09 r3365 [sparc64-linux-3.13.5-mar] (local build)
> Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
>
>
> Probable ATA device behind a SAT layer
> Try an additional '-d ata' or '-d sat' argument.
>
> so i try it with scsi.
>
> root@outpost:~# smartctl -d scsi -a /dev/sda
> smartctl 5.41 2011-06-09 r3365 [sparc64-linux-3.13.5-mar] (local build)
> Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
>
> User Capacity: 2,000,398,934,016 bytes [2.00 TB]
> Logical block size: 512 bytes
> Logical Unit id: 0x50000394e2380537
> Serial number: 7365P4KNT
> Device type: disk
> Local Time is: Wed Mar 5 22:32:09 2014 CET
> Device supports SMART and is Disabled
> Temperature Warning Disabled or Not Supported
> SMART Health Status: OK
>
> Error Counter logging not supported
> Device does not support Self Test logging
I am not familiar with this message:
"Device supports SMART and is Disabled"
but there are lots of posts out there with folks asking about it.
Possibly you can figure out what's causing smart not to turn on, get
it turned on and get some info.
Sorry I cannot help more.
Good luck,
Mark
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Slow I/O performance on SAS1064
2014-03-06 20:51 ` Mark Knecht
@ 2014-03-07 9:46 ` markus
0 siblings, 0 replies; 6+ messages in thread
From: markus @ 2014-03-07 9:46 UTC (permalink / raw)
To: Mark Knecht
Cc: Nagalakshmi Nandigama, Sreekanth Reddy, support,
DL-MPTFusionLinux, linux-scsi, Linux Kernel List
[-- Attachment #1: Type: text/plain, Size: 4040 bytes --]
On Thu, Mar 06, 2014 at 12:51:27PM -0800, Mark Knecht wrote:
> On Wed, Mar 5, 2014 at 1:40 PM, markus <markus@kola.li> wrote:
> > On Wed, Mar 05, 2014 at 10:21:07AM -0800, Mark Knecht wrote:
> >> On Wed, Mar 5, 2014 at 9:50 AM, Markus <markus@kola.li> wrote:
> >> <SNIP>
> I am not familiar with this message:
>
> "Device supports SMART and is Disabled"
>
> but there are lots of posts out there with folks asking about it.
> Possibly you can figure out what's causing smart not to turn on, get
> it turned on and get some info.
right , thats why i writing to the scsi and lsi devs , because die disk supports smart and lots of other features, but the controller driver failes to enable support for this.
Now i get that the mpt driver provides /dev/sg[0-1]
root@outpost:~# smartctl -d scsi -x /dev/sg1
smartctl 6.2 2013-07-26 r3841 [sparc64-linux-3.13.5-mar] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
User Capacity: 2,000,398,934,016 bytes [2.00 TB]
Logical block size: 512 bytes
Logical Unit id: 0x50000394e2380537
Serial number: 7365P4KNT
Device type: disk
Local Time is: Fri Mar 7 09:54:47 2014 CET
SMART support is: Available - device has SMART capability.
SMART support is: Disabled
Temperature Warning: Disabled or Not Supported
Read Cache is: Enabled
Writeback Cache is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK
Error Counter logging not supported
Device does not support Self Test logging
Device does not support Background scan results logging
scsiPrintSasPhy Log Sense Failed [unsupported field in scsi command]
so i had read those posts and i tried but it won't work.
root@outpost:~/lsiutil# smartctl -r scsiioctl,3 -s on -d scsi /dev/sg1
smartctl 6.2 2013-07-26 r3841 [sparc64-linux-3.13.5-mar] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
[inquiry: 12 01 00 01 fc 00 ]
scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=0 milliseconds resid=500
Incoming data, len=8:
00 00 00 00 04 00 80 83 89
[inquiry: 12 00 00 00 24 00 ]
scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=4 milliseconds resid=0
Incoming data, len=36:
00 00 00 05 12 45 00 00 02 41 54 41 20 20 20 20 20
10 54 4f 53 48 49 42 41 20 4d 51 30 31 41 42 42 32
20 30 55 20 20
=== START OF ENABLE/DISABLE COMMANDS SECTION ===
[mode sense(6): 1a 00 1c 00 40 00 ]
scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=4 milliseconds resid=40
Incoming data, len=24:
00 17 00 00 08 00 00 00 00 00 00 02 00 1c 0a 08 06
10 08 00 00 00 00 00 00 00
[mode sense(6): 1a 00 5c 00 40 00 ]
scsi_status=0x0, host_status=0x0, driver_status=0x0
info=0x0 duration=0 milliseconds resid=40
Incoming data, len=24:
00 17 00 00 08 00 00 00 00 00 00 02 00 1c 0a 00 00
10 00 00 00 00 00 00 00 00
00 00 00 00 08 00 00 00 00 00 00 02 00 1c 0a 08 06
10 08 00 00 00 00 00 00 00
[mode select(6): 15 10 00 00 18 00 ]
Outgoing data, len=24:
scsi_status=0x2, host_status=0x0, driver_status=0x8
info=0x1 duration=0 milliseconds resid=24
>>> Sense buffer, len=18:
00 70 00 0b 00 00 00 00 0a 00 00 00 00 00 00 00 00
10 00 00
status=2: sense_key=b asc=0 ascq=0
unable to enable Exception control and warning [aborted command]
>
> Sorry I cannot help more.
>
> Good luck,
> Mark
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Slow I/O performance on SAS1064
[not found] <CAN7X1Umb_wyTCk9w590AnSTFgL=3CC5x8zGNiGZoAdNEej9TCQ@mail.gmail.com>
@ 2014-03-08 20:14 ` markus
0 siblings, 0 replies; 6+ messages in thread
From: markus @ 2014-03-08 20:14 UTC (permalink / raw)
To: Purush Gupta
Cc: Nagalakshmi Nandigama, Sreekanth Reddy, support,
DL-MPTFusionLinux, linux-scsi, Linux Kernel List
[-- Attachment #1: Type: text/plain, Size: 842 bytes --]
On Fri, Mar 07, 2014 at 08:24:48AM -0800, Purush Gupta wrote:
> I noticed you had benchmarked only sda target, is the behavior same on
> other targets?
>
Yes the other disk are equal to sda , but they are used with zfs and there were other preformance problems because of the multible disk access.
config:
NAME STATE READ WRITE CKSUM
home ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
sdb ONLINE 0 0 3
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
> Since they are SCSI targets, have you explored sg_utils [
> http://sg.danny.cz/sg/uu_index.html]
>
> And for benchmarking I also recommend FIO https://github.com/axboe/fio
>
Thanks for the tips.
> Good luck,
> Purush
>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-03-08 20:09 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CAN7X1Umb_wyTCk9w590AnSTFgL=3CC5x8zGNiGZoAdNEej9TCQ@mail.gmail.com>
2014-03-08 20:14 ` Slow I/O performance on SAS1064 markus
2014-03-05 17:50 Markus
2014-03-05 18:21 ` Mark Knecht
2014-03-05 21:40 ` markus
2014-03-06 20:51 ` Mark Knecht
2014-03-07 9:46 ` markus
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox