* Possible mpt2sas 4k sector bug
@ 2012-01-30 14:19 Lars Karlslund
0 siblings, 0 replies; only message in thread
From: Lars Karlslund @ 2012-01-30 14:19 UTC (permalink / raw)
To: linux-kernel
Hi,
I have assembled a system with a LSI 9116i controller (aka 9201-16i),
and have hit a weird problem with basic disk-access.
Doing writes with anything lower than 4k block size, generates lots of
reads on the drives.
These are the system details, basically it's an Ubuntu 11.10 64-bit with
latest updates.
System is on SSD drive on onboard SATA controller (sda).
8 x 1TB drives with 512bytes sector size on LSI controller (sdb to sdi)
1 x 750GB drive with 512bytes sector size also on LSI controller (sdj -
just to verify that it's not the model of drives that are the problem)
Problem is, doing everything but 4k+ writes on the drives, generates
massive reads from the drives, before doing the actual writes.
# dd if=/dev/zero of=/dev/sdb bs=512 count=1000000
1000000+0 records in
1000000+0 records out
512000000 bytes (512 MB) copied, 21.5398 s, 23.8 MB/s
dstat output while doing this:
# dstat -Dsdb
You did not select any stats, using -cdngy by default.
----total-cpu-usage---- --dsk/sdb-- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 100 0 0 0|1592B 1590B| 0 0 | 0 0 | 59 97
0 0 100 0 0 0| 0 0 | 330B 836B| 0 0 | 121 149
0 0 100 0 0 0| 0 0 | 540B 2750B| 0 0 | 70 118
0 1 88 10 0 0| 31M 0 | 497B 114B| 0 0 |8057 24k
0 1 88 11 0 0| 32M 0 |1212B 19k| 0 0 |8268 24k
0 1 88 11 0 0| 32M 0 | 151B 322B| 0 0 |8235 24k
0 2 88 11 0 0| 32M 0 | 151B 322B| 0 0 |8219 25k
0 1 88 11 0 0| 32M 0 | 186B 322B| 0 0 |8219 24k
0 1 88 11 0 0| 32M 0 | 271B 322B| 0 0 |8225 24k
0 2 88 11 0 0| 32M 0 | 126B 322B| 0 0 |8202 24k
0 1 88 11 0 0| 32M 0 | 271B 322B| 0 0 |8179 24k
0 1 88 11 0 0| 32M 0 | 211B 322B| 0 0 |8229 25k
0 1 88 11 0 0| 32M 0 | 66B 322B| 0 0 |8208 24k
0 1 88 11 0 0| 32M 0 | 151B 322B| 0 0 |8201 24k
0 1 88 11 0 0| 32M 0 | 66B 322B| 0 0 |8217 24k
0 1 86 13 0 0| 11M 57M| 653B 322B| 0 0 |3119 8828
0 1 88 12 0 0|2576k 76M| 149B 338B| 0 0 | 874 2061
0 1 88 12 0 0| 14M 39M| 66B 338B| 0 0 |3853 11k
0 1 88 11 0 0| 15M 47M| 149B 338B| 0 0 |3912 11k
0 1 88 12 0 0|5832k 64M| 66B 338B| 0 0 |1664 4507
0 0 88 12 0 0|2792k 72M| 390B 494B| 0 0 | 946 2253
0 1 88 11 0 0| 21M 22M| 150B 322B| 0 0 |5463 16k
0 1 88 11 0 0| 13M 47M| 360B 2570B| 0 0 |3521 10k
0 2 87 11 0 0| 24M 26M| 906B 15k| 0 0 |6487 19k
0 1 92 7 0 0| 336k 41M| 66B 338B| 0 0 | 805 929
0 0 100 0 0 0| 0 0 | 616B 2618B| 0 0 | 62 96
0 0 100 0 0 0| 0 0 |1085B 14k| 0 0 | 87 144
0 0 100 0 0 0| 0 0 | 66B 306B| 0 0 | 44 71
0 0 100 0 0 0| 0 0 | 329B 486B| 0 0 | 89 134
0 0 100 0 0 0| 0 0 | 66B 306B| 0 0 | 67 111
0 0 100 0 0 0| 0 0 | 149B 306B| 0 0 | 61 106
I.e. lots of reads before the writes??
Doing dd with bs=4k fixes this, giving expected behaviour:
# dd if=/dev/zero of=/dev/sdb bs=4k count=125000
125000+0 records in
125000+0 records out
512000000 bytes (512 MB) copied, 6.16785 s, 83.0 MB/s
And dstat output while doing this:
# dstat -Dsdb
You did not select any stats, using -cdngy by default.
----total-cpu-usage---- --dsk/sdb-- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
0 0 100 0 0 0|3180B 3177B| 0 0 | 0 0 | 59 98
0 0 100 0 0 0| 0 0 | 508B 3098B| 0 0 | 158 173
0 3 94 2 0 0| 0 15M| 262B 114B| 0 0 | 320 242
0 0 86 14 0 0| 0 82M| 0 0 | 0 0 | 239 146
0 0 88 12 0 0| 0 83M| 772B 10k| 0 0 | 261 141
0 0 87 12 0 0| 0 83M| 239B 322B| 0 0 | 258 150
0 0 88 12 0 0| 0 83M| 210B 322B| 0 0 | 252 123
0 0 88 12 0 0| 0 81M| 210B 322B| 0 0 | 229 164
0 1 88 11 0 0| 336k 62M| 186B 322B| 0 0 | 521 843
0 0 100 0 0 0| 0 0 | 210B 338B| 0 0 | 252 206
0 0 100 0 0 0| 0 0 | 540B 2780B| 0 0 | 79 128
0 0 100 0 0 0| 0 0 | 643B 10k| 0 0 | 71 117
Details of system:
# lspci | grep LSI
05:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic
SAS2116 PCI-Express Fusion-MPT SAS-2 [Meteor] (rev 02)
# uname -a
Linux storage-ng 3.2.2-030202-generic #201201252035 SMP Thu Jan 26
01:36:10 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
(also tested with 3.0.0-15server)
# fdisk /dev/sdb
Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x39411ba9
Controller firmware upgraded from 7 (version on board when I got it), to
12, verified with dmesg:
# dmesg | grep mpt2sas
[ 3.122653] mpt2sas version 10.100.00.00 loaded
[ 3.123197] mpt2sas 0000:05:00.0: PCI INT A -> GSI 16 (level, low) ->
IRQ 16
[ 3.123218] mpt2sas 0000:05:00.0: setting latency timer to 64
[ 3.123225] mpt2sas0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total
mem (8176204 kB)
[ 3.123303] mpt2sas 0000:05:00.0: irq 46 for MSI/MSI-X
[ 3.123320] mpt2sas0-msix0: PCI-MSI-X enabled: IRQ 46
[ 3.123329] mpt2sas0: iomem(0x00000000f8ffc000),
mapped(0xffffc900117b8000), size(16384)
[ 3.123338] mpt2sas0: ioport(0x000000000000e000), size(256)
[ 3.409371] mpt2sas0: sending message unit reset !!
[ 3.417363] mpt2sas0: message unit reset: SUCCESS
[ 3.585191] mpt2sas0: Allocated physical memory: size(15199 kB)
[ 3.585203] mpt2sas0: Current Controller Queue Depth(7385), Max
Controller Queue Depth(7632)
[ 3.585209] mpt2sas0: Scatter Gather Elements per IO(128)
[ 3.817976] mpt2sas0: LSISAS2116: FWVersion(12.00.00.00),
ChipRevision(0x02), BiosVersion(07.23.01.00)
[ 3.817991] mpt2sas0: Protocol=(Initiator,Target),
Capabilities=(TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
[ 3.818250] mpt2sas0: sending port enable !!
[ 3.823717] mpt2sas0: host_add: handle(0x0001),
sas_addr(0x500062b2000c3600), phys(16)
[ 3.833476] mpt2sas0: port enable: SUCCESS
So now I'm stuck ... any ideas?
Please cc to my mail, as I am not subscribed here.
Thanks,
Lars Karlslund
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2012-01-30 14:26 UTC | newest]
Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-01-30 14:19 Possible mpt2sas 4k sector bug Lars Karlslund
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.