AMD SP5100 SATA low perfomance

AMD SP5100 SATA low perfomance

[Rus V. Brushkoff]

Hello All !

 I'm observing the problem with AMD SP5100 SATA controller. The
problem appears as very low (2-3MB/s) write perfomance. Reading drops
too, but not such dramatically (from 100Mb/s on other hardware to
50-60Mb/s on SP5100). This problem is checked on several kernels
(2.6.x, 3.2.x, 3.4.x) including latest 3.5 and with all possible SATA
BIOS settings. The hardware is Tyan S8005 m/b (latest BIOS v1.03), AMD
X6 1045T, 8GB ECC memory and Hitachi HTS725050A7E630 drive.  Moreover
- I've checked 2 different identical boxes - it is 100% reproducable
on both.
 So I'm asking for help for resolving this issue. The same HDD with
another SATA controller (checked with Asus M2NPV and M5A78 m/b) shows
normal write (150Mb/s-300Mb/s) perfomance. The SMART for HDD is good,
contains no errors and passes the short test.
The http://sfinxsoft.com/tyan URL contains dmesg (dmesg-bad file) and
some screenshots. Can provide any additional info by request. It is
looks like HDD fimrware bug, but Hitachi support refused to help
saying that this is "third-party" problems and they will not support
the end users ;(

Thanks in advance

-- 
Rus

Re: AMD SP5100 SATA low perfomance

[Borislav Petkov]

On Tue, Jul 24, 2012 at 09:34:57PM +0300, Rus V. Brushkoff wrote:
> Hello All !
> 
>  I'm observing the problem with AMD SP5100 SATA controller. The
> problem appears as very low (2-3MB/s) write perfomance. Reading drops
> too, but not such dramatically (from 100Mb/s on other hardware to
> 50-60Mb/s on SP5100). This problem is checked on several kernels
> (2.6.x, 3.2.x, 3.4.x) including latest 3.5 and with all possible SATA
> BIOS settings. The hardware is Tyan S8005 m/b (latest BIOS v1.03), AMD
> X6 1045T, 8GB ECC memory and Hitachi HTS725050A7E630 drive.  Moreover
> - I've checked 2 different identical boxes - it is 100% reproducable
> on both.

Have you tried doing the same test on the same mobo but with another
drive, possibly by another manufacturer? Do you see the same write
slowdown with the second drive too?

Thanks.

-- 
Regards/Gruss,
    Boris.

Re: AMD SP5100 SATA low perfomance

[Rus V. Brushkoff]

 I will do the tests with another drive (from WD) today. But I've
fogot to say that I have another S8005 m/b with SP5100 that working
perfectly with these particular Hitachi drives (please see boot log
for it in dmesg-good at the same URL). So the problem can be in init
procedure for AMD south bridge or SATA controller. The differences
between m/b is very little - working has onboard audio, not working
hasn't. The diff in boot log is present too in file diff.txt

>>  I'm observing the problem with AMD SP5100 SATA controller. The
>> problem appears as very low (2-3MB/s) write perfomance. Reading drops
>> too, but not such dramatically (from 100Mb/s on other hardware to
>> 50-60Mb/s on SP5100). This problem is checked on several kernels
>> (2.6.x, 3.2.x, 3.4.x) including latest 3.5 and with all possible SATA
>> BIOS settings. The hardware is Tyan S8005 m/b (latest BIOS v1.03), AMD
>> X6 1045T, 8GB ECC memory and Hitachi HTS725050A7E630 drive.  Moreover
>> - I've checked 2 different identical boxes - it is 100% reproducable
>> on both.
>
> Have you tried doing the same test on the same mobo but with another
> drive, possibly by another manufacturer? Do you see the same write
> slowdown with the second drive too?
>
> Thanks.
>
> --
> Regards/Gruss,
>     Boris.



-- 
Rus

Re: AMD SP5100 SATA low perfomance

[Borislav Petkov]

On Wed, Jul 25, 2012 at 09:47:16AM +0300, Rus V. Brushkoff wrote:
>  I will do the tests with another drive (from WD) today. But I've
> fogot to say that I have another S8005 m/b with SP5100 that working
> perfectly with these particular Hitachi drives (please see boot log
> for it in dmesg-good at the same URL). So the problem can be in init
> procedure for AMD south bridge or SATA controller. The differences
> between m/b is very little - working has onboard audio, not working
> hasn't. The diff in boot log is present too in file diff.txt

One other thing you could try: can you downgrade the BIOS on one of the
bad Tyan boards (the -LE ones, right?) and retry the test again?

Any results on the test with the other drive?

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: AMD SP5100 SATA low perfomance

[Rus V. Brushkoff]

>
> One other thing you could try: can you downgrade the BIOS on one of the
> bad Tyan boards (the -LE ones, right?) and retry the test again?

Already did this - I've tried the 1.02 BIOS version (can try earlier
as the earlier versions do not have my CPU id needed for starting) .
The results are the same - poor HDD perfomance.

>
> Any results on the test with the other drive?

On the way - will post results in several hours.

-- 
Rus

Re: AMD SP5100 SATA low perfomance

[Rus V. Brushkoff]

>> Any results on the test with the other drive?

The test results with dmes

>
> On the way - will post results in several hours.
>
> --
> Rus



-- 
Rus

Re: AMD SP5100 SATA low perfomance

[Rus V. Brushkoff]

>> Any results on the test with the other drive?

Sorry, pressed Send button during edit.

The test results with 2 WD drives,

1. Desktop 500GB 7200 rpm WD5000AKX drive
      [good S8005GM2NR board] read speed - 127 Mb/s
      [good S8005GM2NR board] write speed - 124 Mb/s
      [bad S8005GM2NR-LE] board read spead - 127 Mb/s
      [bad S8005GM2NR-LE] board write speed - 124 Mb/s

2. Mobile 250GB 5400 rpm WD2500BEVS
      [good S8005GM2NR board] read speed - 57 Mb/s
      [good S8005GM2NR board] write speed - 55 Mb/s
      [bad S8005GM2NR-LE] board read spead - 37 Mb/s (!)
      [bad S8005GM2NR-LE] board write speed - 30 Mb/s (!)

So at least the perfomance dropped with different drive too and the drop is
nearly 20Mb/s.

I've made another test, trying to exclude the SP52100 - I've used the
external PCI SATA controller from Silicon Image Sil3112A, so the two
drives - Hitachi and WD were connected to PCI SATA board instead of
onboard SP5100, test results below :

1. Hitachi HTS725050A7E630 500GB drive
      [good S8005GM2NR board] read speed - 88 Mb/s
      [good S8005GM2NR board] write speed - 80-91 Mb/s (!)
      [bad S8005GM2NR-LE] board read spead - 35 Mb/s
      [bad S8005GM2NR-LE] board write speed - 0.9 Mb/s (!)

2. Desktop 500GB WD5000AKX drive
      [good S8005GM2NR board] read speed - 98 Mb/s
      [good S8005GM2NR board] write speed - 85 Mb/s
      [bad S8005GM2NR-LE] board read spead - 96 Mb/s
      [bad S8005GM2NR-LE] board write speed - 87 Mb/s

So the low write perfomance do not depends even from SATA controller,
but depends from the HDD and moherboard combination. Can we see (or
debug) what is happenning on bad  S8005GM2NR-LE motherboard during
write ?

-- 
Rus

Re: AMD SP5100 SATA low perfomance

[Rus V. Brushkoff]

Hello,

You will laugh, but I've found the issue. These particular drives do
not like the fan vibration and lowering the spin speed ;) Sh*t ...
Sorry for bothering all of you and big thanks for support !

Rus

2012/7/26 Rus V. Brushkoff <brushkoff@gmail.com>:
>> Yes, probably. Right about now I'm all out of ideas about what on the
>> motherboard could be causing this. Maybe we should ask Tyan for help or
>> whether they've seen a similar issue.
>
> Already asked, but they can't reproduce this as I think that they do
> not use Linux for tests ;(
>
>>
>>> Can we see (or debug) what is happenning on bad S8005GM2NR-LE
>>> motherboard during write ?
>>
>> I don't think so, without special hardware.
>
> Hmm, what exact hardware needed for this ?
>
>>
>> Just a stab in the dark: you've done all those with the systemrescuecd
>> which has kernel 3.2, correct? Would it be possible to redo those tests,
>> especially the ones which show the slowdown, with the latest kernel -
>> 3.5 - to see whether the issue still persists?
>
> I've tried the systemrescuecd 3.0.0-beta too, its running 3.5 kernel.
>
> Rus
>
> --
> Rus



-- 
Rus

Re: AMD SP5100 SATA low perfomance

[Borislav Petkov]

On Fri, Jul 27, 2012 at 11:23:56AM +0300, Rus V. Brushkoff wrote:
> Hello,
> 
> You will laugh, but I've found the issue. These particular drives do
> not like the fan vibration and lowering the spin speed ;) Sh*t ...

"Sh*t" really only begins to describe this. So let me get this
correctly: your Hitachi hdds weren't mounted in a very stable manner to
the mobo chassis and vibrations from the fan caused them to fail writes
and retry a _lot_ leading to the write performance drop?

And the WD ones don't have a problem with the vibrations?

Wow! Looks like some idiot was saving on production material/design to
make the drives cheaper. Talk about a major serious FAIL.

:-)

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: AMD SP5100 SATA low perfomance

[Rus V. Brushkoff]

> "Sh*t" really only begins to describe this. So let me get this
> correctly: your Hitachi hdds weren't mounted in a very stable manner to
> the mobo chassis and vibrations from the fan caused them to fail writes
> and retry a _lot_ leading to the write performance drop?

The HDD's _was_ mounted very tight and stable, so the FAN vibration is
propagated to HDD without the problems, next the internal
accelerometer founds this vibration and lowers all the HDD operations
! If I get out the HDD from the case and mechanically isolate it from
the FAN vibration the speed momentally grows to the max value !

> And the WD ones don't have a problem with the vibrations?

The desktop WD do not have this issue as it has much more mass than
mobile one. As you can see by tests made by me - the mobile WD behaves
nearly same as Hitachi one - it lowers the HDD operations by -20Mb/s
value.

> Wow! Looks like some idiot was saving on production material/design to
> make the drives cheaper. Talk about a major serious FAIL.

This idiot is me ;) I do not take this issue in account.

Rus

-- 
Rus

Re: AMD SP5100 SATA low perfomance

[Borislav Petkov]

On Fri, Jul 27, 2012 at 12:22:14PM +0300, Rus V. Brushkoff wrote:
> > "Sh*t" really only begins to describe this. So let me get this
> > correctly: your Hitachi hdds weren't mounted in a very stable manner to
> > the mobo chassis and vibrations from the fan caused them to fail writes
> > and retry a _lot_ leading to the write performance drop?
> 
> The HDD's _was_ mounted very tight and stable, so the FAN vibration is
> propagated to HDD without the problems, next the internal
> accelerometer founds this vibration and lowers all the HDD operations
> ! If I get out the HDD from the case and mechanically isolate it from
> the FAN vibration the speed momentally grows to the max value !
> 
> > And the WD ones don't have a problem with the vibrations?
> 
> The desktop WD do not have this issue as it has much more mass than
> mobile one. As you can see by tests made by me - the mobile WD behaves
> nearly same as Hitachi one - it lowers the HDD operations by -20Mb/s
> value.

So the Hitachi ones are lighter than the WD ones and behave like mobile
hdds in that respect.

Oh well, good to know for next time when buying hdds.

I have one final question. You said:

"The same HDD with another SATA controller (checked with Asus M2NPV and
M5A78 m/b) shows normal write (150Mb/s-300Mb/s) perfomance."

Does that mean that the Hitachi hdd which was performing badly in the
Tyan board, didn't receive vibrations when connected to the Asus mobo
simply because the Asus mobo is in a different chassis which absorbs
vibrations somehow?

All I'm saying is, if you switch mobos and use the same chassis which
generates vibrations, you should be seeing the same performance drop
with the Asus board and this way confirm your findings.

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: AMD SP5100 SATA low perfomance

[Rus V. Brushkoff]

> So the Hitachi ones are lighter than the WD ones and behave like mobile
> hdds in that respect.

Yes, these Hitachi is lighter and more sensitive to vibrations - even
hanging on wires is sufficient for twice perfomance drop - the whole
vibro silence is requred for it to reach the max speed. I think this
is mostly the hardware design flaw of this particular model - will
return all of them.

> "The same HDD with another SATA controller (checked with Asus M2NPV and
> M5A78 m/b) shows normal write (150Mb/s-300Mb/s) perfomance."
>
> Does that mean that the Hitachi hdd which was performing badly in the
> Tyan board, didn't receive vibrations when connected to the Asus mobo
> simply because the Asus mobo is in a different chassis which absorbs
> vibrations somehow?

 Yes, as the Asus mobo as the "working good" S8005GM2NR one has the
different chassis that has very small vibration, so for tests I did
not put the HDD inside - it was simply laying on chassis surface -
this is why connecting the same HDD shows the normal perfomance.

> All I'm saying is, if you switch mobos and use the same chassis which
> generates vibrations, you should be seeing the same performance drop
> with the Asus board and this way confirm your findings.

 Sure. I've made the simple test - took out the HDD from the vibrating
case - run dd and see the 100Mb/s perfomance. Next simply puted the
HDD on the vibrating 1U case surface and run the dd test again - the
perfomance immediatly dropped to 20Mb/s. This is constantly
reproducable ;) These HDD is unusable where the fans exists - seems
like everywhere ;)

-- 
Rus

Re: AMD SP5100 SATA low perfomance

[Dave Howorth]

Rus V. Brushkoff wrote:
>> So the Hitachi ones are lighter than the WD ones and behave like mobile
>> hdds in that respect.
> 
> Yes, these Hitachi is lighter and more sensitive to vibrations - even
> hanging on wires is sufficient for twice perfomance drop - the whole
> vibro silence is requred for it to reach the max speed. I think this
> is mostly the hardware design flaw of this particular model - will
> return all of them.

Well, you said that the 2.5" WD drives showed the same symptoms, so I
don't see how you conclude the problem is Hitachi-specific. It seems
more likely to be an interaction between some (or all) 2.5" drives and
the particular case/fan combination.

How are the 2.5" drives mounted in the cases?

Perhaps it would be fixable by using low-vibration fans or fan mounting
kits. Perhaps mounting the drives on a lump of solid metal to bring them
up to the same mass as a 3.5" drive would fix it. Perhaps resilient
mounting of the drives would fix it. Perhaps a stiffer/heavier case
would fix it.

Now that you know the problem is hardware-related, the same symptoms
should show up using Windows (or any other OS). So maybe Hitachi (and WD
and the case and fan manufacturers) might now be interested in finding a
better cure, or at least warning potential customers.

Re: AMD SP5100 SATA low perfomance

[Rus V. Brushkoff]

>> Yes, these Hitachi is lighter and more sensitive to vibrations - even
>> hanging on wires is sufficient for twice perfomance drop - the whole
>> vibro silence is requred for it to reach the max speed. I think this
>> is mostly the hardware design flaw of this particular model - will
>> return all of them.
>
> Well, you said that the 2.5" WD drives showed the same symptoms, so I
> don't see how you conclude the problem is Hitachi-specific. It seems
> more likely to be an interaction between some (or all) 2.5" drives and
> the particular case/fan combination.

The WD drives dropped perfomance by 20Mb/s but Hitachi by factor 5, I
think this is big difference.

>
> How are the 2.5" drives mounted in the cases?

Using the stock Tyan GT14 tight alluminium HDD mount case.

>
> Perhaps it would be fixable by using low-vibration fans or fan mounting
> kits. Perhaps mounting the drives on a lump of solid metal to bring them
> up to the same mass as a 3.5" drive would fix it. Perhaps resilient
> mounting of the drives would fix it. Perhaps a stiffer/heavier case
> would fix it.

 Tested, probed, tried - do not work with this model. These Hitachi
drives are too light (nearly 90 gr) and do not like even the lowest
vibration. This is the worst vibrate-sensitive drives I ever saw. The
worst thing that the allowable vibration level is different for each
drive - I've tested 5 drives of the same model - each shows different
perfomance drop from the same vibration level - write speed varied
from 0.7Mb/s to 30Mb/sec, while no-vibration level is 116-120Mb/sec.

> Now that you know the problem is hardware-related, the same symptoms
> should show up using Windows (or any other OS). So maybe Hitachi (and WD
> and the case and fan manufacturers) might now be interested in finding a
> better cure, or at least warning potential customers.

Hitachi Technical Support Center (one Niko Levenetz) refused to help
me from the very start leaving me with this problematic drive problem
alone. Seems like they do not bother at all about supporting their
products for end users. I will never buy the Hitachi drives again not
because they are not work (this can be fixed at least) but because of
refusing to support end users. F*ck you Hitachi ! (c) Linus ;)

-- 
Rus