Seagate harddrives problems with SB7x0 AHCI

[William King <quentusrex@gmail.com>]

Problem:
I have run into what appears to be a driver problem with the Seagate 
Barracuda 7200.12 drives. The symptom is io load of 10-15 times the load 
for other drives. To the point the box becomes fully io-bound and 
unresponsive when even installing apt packages or rsync of a 100MB file.

Background:
I have two servers with nearly identical hardware: Alpha has Seagate 
Barracuda 7200.12 1TB drives model: ST31000528AS and Bravo that has 
Western Digital Cavaliar Black 1TB model: 'WDC WD1001FALS-0'. Both 
servers have Ubuntu 11.04 64bit server installed, and have fully updated 
to the 2.3.38-10 kernel. More hardware info and logs can be located 
here: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/815540

The server hardware manufacturer page is:
http://www.supermicro.com/Aplus/system/1U/1042/AS-1042G-TF.cfm

Command outputs:
# lspci |grep SATA
00:11.0 SATA controller: ATI Technologies Inc SB7x0/SB8x0/SB9x0 SATA 
Controller [AHCI mode]

# uname -a
Linux alpha-mule 2.6.38-10-server #46-Ubuntu SMP Tue Jun 28 16:31:00 UTC 
2011 x86_64 x86_64 x86_64 GNU/Linux


Steps to confirm/reproduce and narrow scope:
I have taken the hard drives from Bravo, put them into Alpha reinstalled 
and there is no problem. Also I have placed some older Seagate Barracuda 
7200.9 80GB drives in Alpha and the problem does not exist then either. 
I used iotop and iostats to watch the reported io load as well as the 
read/write speeds to the disk. I then used rsync with a bandwidth 
limitation to measure the performance difference between the two servers 
in terms of server load and iotop reported IO%.

A cat of /proc/interrupts show an order of magnitude more interrupts for 
ahci on Alpha than Bravo for the same amount of traffic during an rsync.

I have hdparm -t /dev/sda test and have seen a performance of ~50MB/s on 
the Seagate 7200.12 drive while the specs and other benchmarks claim 
speeds of up to 140MB/s. The Western Digital in the same hardware are 
able to reach speeds of ~105MB/s and the Seagate 7200.9 gets ~65MB/s. 
The hard drive was not in use at the time of the hdparm test.

I took a perf capture of a fio random read test on each machines. The 
test completes in 4 seconds on Alpha and 2 minutes 45 seconds on Bravo 
and the perf stats show all the extra time being spent in the kernel 
scheduler section.

I initially had the Seagate hard drives on firmware CC46 then updated 
them to the latest firmware of CC49. No change detected between firmwares.

I have setup alpha to be able to rebuild the Ubuntu Kernel with patches 
so I can easily test new changes. Also I have the linux kernel git tree 
cloned so while it would be possible, I would be more comfortable 
testing on the Ubuntu version of the 2.6.38-10 kernel.

-William King