libata sata_via unusual behaviour

libata sata_via unusual behaviour

[Paolo]

Hello!

 It seems that me and many others are experiencing poor sata performances
 in the last months, probably with 2.6.20+ kernels.

 an example of these problems is here:

 http://ubuntuforums.org/showthread.php?p=4845777

 unfortunately it seems that sata subsystem on linux is little to none
 tweakable, and there are no informations both on the linux-ata
 official page and on the web in general.

 Hope you can make some clarity about the situation
 Thanks in advance and best regards,
 Paolo

Re: libata sata_via unusual behaviour

[Alan Cox]

On Thu, 1 May 2008 20:58:37 +0200
Paolo <paoletto@gmail.com> wrote:

> Hello!
> 
>  It seems that me and many others are experiencing poor sata performances
>  in the last months, probably with 2.6.20+ kernels.
> 
>  an example of these problems is here:
> 
>  http://ubuntuforums.org/showthread.php?p=4845777
> 
>  unfortunately it seems that sata subsystem on linux is little to none
>  tweakable

It doesn't normally need tweaking, and the hdparm numbers show that it is
working correctly in the example you give. There may be problems higher
up the stack but the ATA layer appears to be doing just fine.

Perhaps the Ubuntu engineers could be persuaded to profile their problem
systems and identify what is causing the slow down - LVM certain has been
problematic for some setups if LVM is in use.

Alan

Re: libata sata_via unusual behaviour

[Mark Lord]

Alan Cox wrote:
> On Thu, 1 May 2008 20:58:37 +0200
> Paolo <paoletto@gmail.com> wrote:
> 
>> Hello!
>>
>>  It seems that me and many others are experiencing poor sata performances
>>  in the last months, probably with 2.6.20+ kernels.
>>
>>  an example of these problems is here:
>>
>>  http://ubuntuforums.org/showthread.php?p=4845777
>>
>>  unfortunately it seems that sata subsystem on linux is little to none
>>  tweakable
> 
> It doesn't normally need tweaking, and the hdparm numbers show that it is
> working correctly in the example you give. There may be problems higher
> up the stack but the ATA layer appears to be doing just fine.
..

Yup.  The drive seems to be doing fine, according to the info you've provided.

But the "-T" (big T) number looks *very* slow:  220.02 MB/sec
That number has nothing to do with the drive.  Rather, it's an indication
of how fast the CPU and memory are together, and that's about 1/20 to 1/10
of a modern system.

Note that it's not a useful number in any absolute sense, but only when
compared with a "-T" value from some other system or Linux distro.

Strange.

Re: libata sata_via unusual behaviour

[Paolo]

Hi Alan, hi all

so far, i did some more tests.

First of all, i'm using ext3 filesystem (and it might happen it's very
slow, according with my tests results)

What i did:

Trying ubuntu Hardy LiveCD (to check it's not because
1) im using a hand made compiled kernel
2) i might have removed some package which is actually needed for disk
performances
3) the upgrade screwed something )

So far, from LiveCD i did the same i did before, that is:

root@ubuntu:/media/disk/movies# time cat Lo\ squalo.avi >/dev/null

real    5m16.352s
user    0m0.132s
sys     0m6.244s
root@ubuntu:/media/disk/movies# ls -la Lo\ squalo.avi
-rw-r----- 1 1000 1001 1559521280 2008-03-30 06:37 Lo squalo.avi

it means roughly those old 5mb/sec.
In the meanwhile i checked with top, and all the cpu cycles were taken
by the disk (about 98.0%wa and 0.0%id)

ok so not my fault. Maybe ext3? my volumes looks like:
root@rivendell:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1             28834716  17088924  10281068  63% /
[...]
/dev/sda3            931618688 613099244 314519444  67% /local

So i got a 30gb partition, and a 900gb partition (roughly).

I tried to copy the file into the smaller partition ( into / )
and:

root@rivendell:/# time cp -rf /local/movies/Lo\ squalo.avi /

real    6m21.978s
user    0m0.150s
sys     0m18.170s
root@rivendell:/# time cat /Lo\ squalo.avi >/dev/null

real    1m33.913s
user    0m0.150s
sys     0m5.370s
root@rivendell:/# time cat /local/movies/Lo\ squalo.avi >/dev/null

real    6m13.549s
user    0m0.230s
sys     0m5.930s

During this tests, the first and the last command still showed like 98.0%wa
while with the second, (reading the file from the / fs) i had about
50-60%wa and some idle cycles (about 20% or so, sometimes 0% but not
always) with about 16mb/sec transfer rate (which is still slow, to
me.. again, hdparm -tT shows:
root@rivendell:/# hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   476 MB in  2.00 seconds = 237.68 MB/sec
 Timing buffered disk reads:  220 MB in  3.02 seconds =  72.80 MB/sec )

This test shows to me that it might be a filesystem issue, but still,
is it possible it's so expensive? i mean, even from a 30gb partition
(which is the norm) reading operations take almost all the cpu power
(even if it's finally faster).
Ok, my cpu is not lighting fast (AthlonXP 800mhz) but i had almost
better performances on a pentiumII 350mhz and 320gb IDE hard drive
(debian testing on ext3).

do you have any clue about what could i do to improve fs performances?
Thank you in advance and excuse me for the boring mail!
Paolo

On 5/1/08, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> On Thu, 1 May 2008 21:16:24 +0200
> Paolo <paoletto@gmail.com> wrote:
>
>> Umh..
>>
>> Thank you for your answer!
>> Actually, i think the same. Hdparm tests shows good results.
>
> hdparm results mean the disk setup is ok
>
>> what i dont get is why the cpu load raise to keep idle to 0% like if
>> it was pio mode, even if it is in udma6..
>> What else could be, other than the libata layer not setted in dma
>> (which is not)?
>
> Anything above the disk driver layer - file system, block queue setup
> or something happening which causes a lot of I/O in small chunks (which
> ruins a disks performance). This is usually distro level stuff so best
> analysed by the distribution people.
>
>> ps: no, im not using LVM. But ubuntu is using their uuid stuff in
>> fstab to boot which i quite dont know at all..
>> maybe that?
>
> No that wouldn't explain it at all. The uuid/fstab stuff is just boot
> time stuff so doesn't get in the way.
>
> What file system are you using ?
>

Re: libata sata_via unusual behaviour

[Paolo]

Actually, i was curious to see how my old Pentium2 performs, so i
tried an ubuntu gutsy on it.
Surprising results:

root@erebor:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1             14421344   2177664  11511120  16% /
[...]
/dev/sda3            273522480 260268272  12230208  96% /local

(so one 15gb partition, and one 270gb partition, both ext3)

hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   158 MB in  2.00 seconds =  78.93 MB/sec
 Timing buffered disk reads:   84 MB in  3.03 seconds =  27.74 MB/sec

crappy results, BUT:

root@erebor:~# time cat /local/SAT/aba/bigfile.avi >/dev/null

real    0m24.637s
user    0m0.236s
sys     0m5.676s
root@erebor:~# ls -la  /local/SAT/aba/bigfile.avi
-rw-rw-rw- 1 root root 732903424 2007-05-29 22:43 /local/SAT/aba/bigfile.avi

which means about 28mb/sec (what hdparm -t state). Here i still get
0%id cpu, but at least it goes fast. Now: this machine has an intel
i440bx chipset.
The other one a Via KT600.

Both of them run or ran ubuntu gutsy (now the new one has hardy, but
before , with gutsy, i had same results)

So... is sata_via buggy with via VT8237 soutbridge?


On 5/17/08, Paolo <paoletto@gmail.com> wrote:
> Hi Alan, hi all
>
> so far, i did some more tests.
>
> First of all, i'm using ext3 filesystem (and it might happen it's very
> slow, according with my tests results)
>
> What i did:
>
> Trying ubuntu Hardy LiveCD (to check it's not because
> 1) im using a hand made compiled kernel
> 2) i might have removed some package which is actually needed for disk
> performances
> 3) the upgrade screwed something )
>
> So far, from LiveCD i did the same i did before, that is:
>
> root@ubuntu:/media/disk/movies# time cat Lo\ squalo.avi >/dev/null
>
> real    5m16.352s
> user    0m0.132s
> sys     0m6.244s
> root@ubuntu:/media/disk/movies# ls -la Lo\ squalo.avi
> -rw-r----- 1 1000 1001 1559521280 2008-03-30 06:37 Lo squalo.avi
>
> it means roughly those old 5mb/sec.
> In the meanwhile i checked with top, and all the cpu cycles were taken
> by the disk (about 98.0%wa and 0.0%id)
>
> ok so not my fault. Maybe ext3? my volumes looks like:
> root@rivendell:~# df
> Filesystem           1K-blocks      Used Available Use% Mounted on
> /dev/sda1             28834716  17088924  10281068  63% /
> [...]
> /dev/sda3            931618688 613099244 314519444  67% /local
>
> So i got a 30gb partition, and a 900gb partition (roughly).
>
> I tried to copy the file into the smaller partition ( into / )
> and:
>
> root@rivendell:/# time cp -rf /local/movies/Lo\ squalo.avi /
>
> real    6m21.978s
> user    0m0.150s
> sys     0m18.170s
> root@rivendell:/# time cat /Lo\ squalo.avi >/dev/null
>
> real    1m33.913s
> user    0m0.150s
> sys     0m5.370s
> root@rivendell:/# time cat /local/movies/Lo\ squalo.avi >/dev/null
>
> real    6m13.549s
> user    0m0.230s
> sys     0m5.930s
>
> During this tests, the first and the last command still showed like 98.0%wa
> while with the second, (reading the file from the / fs) i had about
> 50-60%wa and some idle cycles (about 20% or so, sometimes 0% but not
> always) with about 16mb/sec transfer rate (which is still slow, to
> me.. again, hdparm -tT shows:
> root@rivendell:/# hdparm -tT /dev/sda
>
> /dev/sda:
>  Timing cached reads:   476 MB in  2.00 seconds = 237.68 MB/sec
>  Timing buffered disk reads:  220 MB in  3.02 seconds =  72.80 MB/sec )
>
> This test shows to me that it might be a filesystem issue, but still,
> is it possible it's so expensive? i mean, even from a 30gb partition
> (which is the norm) reading operations take almost all the cpu power
> (even if it's finally faster).
> Ok, my cpu is not lighting fast (AthlonXP 800mhz) but i had almost
> better performances on a pentiumII 350mhz and 320gb IDE hard drive
> (debian testing on ext3).
>
> do you have any clue about what could i do to improve fs performances?
> Thank you in advance and excuse me for the boring mail!
> Paolo
>
> On 5/1/08, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> On Thu, 1 May 2008 21:16:24 +0200
>> Paolo <paoletto@gmail.com> wrote:
>>
>>> Umh..
>>>
>>> Thank you for your answer!
>>> Actually, i think the same. Hdparm tests shows good results.
>>
>> hdparm results mean the disk setup is ok
>>
>>> what i dont get is why the cpu load raise to keep idle to 0% like if
>>> it was pio mode, even if it is in udma6..
>>> What else could be, other than the libata layer not setted in dma
>>> (which is not)?
>>
>> Anything above the disk driver layer - file system, block queue setup
>> or something happening which causes a lot of I/O in small chunks (which
>> ruins a disks performance). This is usually distro level stuff so best
>> analysed by the distribution people.
>>
>>> ps: no, im not using LVM. But ubuntu is using their uuid stuff in
>>> fstab to boot which i quite dont know at all..
>>> maybe that?
>>
>> No that wouldn't explain it at all. The uuid/fstab stuff is just boot
>> time stuff so doesn't get in the way.
>>
>> What file system are you using ?
>>
>

Re: libata sata_via unusual behaviour

[Alan Cox]

> real    5m16.352s
> user    0m0.132s
> sys     0m6.244s

So the CPU usage is ok

> root@ubuntu:/media/disk/movies# ls -la Lo\ squalo.avi
> -rw-r----- 1 1000 1001 1559521280 2008-03-30 06:37 Lo squalo.avi
> 
> it means roughly those old 5mb/sec.
> In the meanwhile i checked with top, and all the cpu cycles were taken
> by the disk (about 98.0%wa and 0.0%id)

98.0%wa - wait time, not CPU time.

Its still a very slow copy rate unless the disks are on the same IDE
channel. How is the disk set up - ext3 direct or via raid/lvm ?

Also is a copy of the movie to /dev/null slow or fast, ditto a copy
from /dev/zero to disk of that sort of size ?

Alan

Re: libata sata_via unusual behaviour

[Paolo]

So, let me clarify a bit:

i did the same test in 3 different configuration:
"cat bigfile >/dev/null"

3 configuration were

1) Machine A (the slow one) from the big partition (1000gb) - got 5mb/s
2) Machine A from the small partition - got 16mb(s
3) Machine B from 300gb partition - got 28mb/s

Machine A: AthlonXP 800 -Via KT600 - SATA disk western digital -
direct ext3 filesystems (no raid-no lvm-no nothing)

Machine B: Pentium 2 350 - i440bx - seagate IDE drive - direct ext3 filesystems

I also did another test: putting a cpu intensive process  in the
background with nice -n 20. The test showed the process was not being
executed almost at all, so those %wa cpu cycles are somehow cpu cycles
since they dont let other processes execute. (in fact when i do stuff
with the disk my system became incredibly slow)

Finally  i tried the test you suggested me
root@rivendell:~# time dd if=/dev/zero of=/tmp/testfile bs=1024000 count=1000
1000+0 records in
1000+0 records out
1024000000 bytes (1,0 GB) copied, 16,9609 s, 60,4 MB/s

real    0m17.119s
user    0m0.010s
sys     0m12.890s

with a sample cpu usage of:
Cpu(s):  0.7%us, 15.0%sy,  0.0%ni, 76.1%id,  6.6%wa,  0.0%hi,  1.7%si,  0.0%st


root@rivendell:~# time dd if=/dev/zero of=/local/testfile bs=1024000 count=1000
1000+0 records in
1000+0 records out
1024000000 bytes (1,0 GB) copied, 23,3957 s, 43,8 MB/s

real    0m23.898s
user    0m0.010s
sys     0m14.090s

with a sample cpu usage of
Cpu(s):  1.3%us, 50.7%sy,  0.0%ni,  0.0%id, 45.4%wa,  0.7%hi,  2.0%si,  0.0%st


( / was the 30gb partition, /local the 900gb)




On 5/18/08, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> real    5m16.352s
>> user    0m0.132s
>> sys     0m6.244s
>
> So the CPU usage is ok
>
>> root@ubuntu:/media/disk/movies# ls -la Lo\ squalo.avi
>> -rw-r----- 1 1000 1001 1559521280 2008-03-30 06:37 Lo squalo.avi
>>
>> it means roughly those old 5mb/sec.
>> In the meanwhile i checked with top, and all the cpu cycles were taken
>> by the disk (about 98.0%wa and 0.0%id)
>
> 98.0%wa - wait time, not CPU time.
>
> Its still a very slow copy rate unless the disks are on the same IDE
> channel. How is the disk set up - ext3 direct or via raid/lvm ?
>
> Also is a copy of the movie to /dev/null slow or fast, ditto a copy
> from /dev/zero to disk of that sort of size ?
>
> Alan
>