3ware disk latency?

[Jan Kasprzak <kas@fi.muni.cz>]

	Hi all,

I have upgraded my machine from 3ware 7508 with 8x 250GB ATA drives
to 3ware 9550SX-8LP with 8x 500GB SATA-II drives, and I have found that
while the overall throughput of the disks is higher than before,
the latency - measured as the number of messages per time unit Qmail can
deliver - is far worse than before[1]. I have tried to use deadline
and cfq schedulers, but it made no visible speedup.

	I have found the following two years old mail by Jens Axboe:
http://www.uwsg.iu.edu/hypermail/linux/kernel/0409.0/1330.html
- the problem at that time was that the device had its internal
command queue deeper than the nr_requests of the block device layer,
so the I/O scheduler couldn't do anything.  I was surprised
that the situation is still the same: I have

/sys/block/sd[a-h]/queue/nr_requests == 128, and
/sys/devices/pci0000:00/0000:00:0b.0/0000:01:03.0/host0/target*/*/queue_depth == 254

I have verified that even on older 3ware drivers (7508) the situation
is the same. On the newer hardware it is probably more visible, because
the controller actually has bigger on-board cache (128MB vs. 32MB, I think).

	I have tried to lower the /sys/devices/.../queue_depth to 4
and disable the NCQ, and the latency got a bit better. But it is still
nowhere near to where it was on the older HW.

	Does anybody experience the similar latency problems with
3ware 95xx controllers? Thanks,

-Yenya

[1] the old configuration peaked at ~2k-4k messages per 5 minutes,
	with 1k-2k messages/5min being pretty normal, while the new one
	has maximum throughput of 1k messages per 5 minutes, but the normal
	speed is much lower - some low hundreds messages per 5 minutes.
	The new system runs about the same load as the previous one,
	and the layout of disks is also the same (just the newer drives
	are bigger, of course).


-- 
| Jan "Yenya" Kasprzak  <kas at {fi.muni.cz - work | yenya.net - private}> |
| GPG: ID 1024/D3498839      Fingerprint 0D99A7FB206605D7 8B35FCDE05B18A5E |
| http://www.fi.muni.cz/~kas/    Journal: http://www.fi.muni.cz/~kas/blog/ |
> I will never go to meetings again because I think  face to face meetings <
> are the biggest waste of time you can ever have.        --Linus Torvalds <