Poor SATA / RAID performance (2.6.11 and promise SATAII150 TX4)

Poor SATA / RAID performance (2.6.11 and promise SATAII150 TX4)

[Tim Harvey]

Greetings,

I'm attempting to benchmark software RAID5 on a system with:
  - Promise SATAII150 TX4 card
  - 4 Segate ST3300831AS drives 
  - custom built kernel 2.6.11 (to get driver for promise SATAIITX4)
  - FC3 install
  - EPIA M10000 mainboard, 256MB memory

The tools I'm familiar with for benchmarking a PATA based RAID system are:
  - hdparm
  - dd

Here are some interesting stats from my system:

[root@epiam10k ~]# more /proc/mdstat
Personalities : [raid5] 
md0 : active raid5 sdd1[3] sdc1[2] sdb1[1] sda1[0]
      879100608 blocks level 5, 4k chunk, algorithm 2 [4/4] [UUUU]
      
unused devices: <none>
[root@epiam10k ~]# hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads:  116 MB in  3.02 seconds =  38.45 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for
device
[root@epiam10k ~]# hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads:  104 MB in  3.05 seconds =  34.10 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for
device
[root@epiam10k ~]# hdparm -t /dev/md0

/dev/md0:
 Timing buffered disk reads:   72 MB in  3.03 seconds =  23.79 MB/sec

[root@epiam10k ~]# vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
 1  0      0   3628   8984 142704    0    0   423    23 1023   112 30  3 61  6

Observations:
  - the performance of a raw SATA device (/dev/sda in the above example) seems
low when I compare it to a PATA drive from a previous system (which would get
about 45MB/sec)
  - the performance of the RAID5 array (/dev/md0) seems very low - I expect
quite an increase over a single device due to striping
  - the number of interrupts per second (1023) seems very high

Questions:
  - is hdparm the right tool for looking at SATA devices?
  - is the error regarding the ioctl an issue?
  - how do I set/get the DMA modes for the SATA controller and SATA drives?
  - why would the interrupts per sec reported from vmstat be so high?

Thanks for any suggesitons.  Please 'cc' me in any replies to the list.

Tim Harvey

Re: Poor SATA / RAID performance (2.6.11 and promise SATAII150 TX4)

[J.A. Magallon]

On 03.31, Tim Harvey wrote:
> Greetings,
> 
> I'm attempting to benchmark software RAID5 on a system with:
>   - Promise SATAII150 TX4 card
>   - 4 Segate ST3300831AS drives 
>   - custom built kernel 2.6.11 (to get driver for promise SATAIITX4)
>   - FC3 install
>   - EPIA M10000 mainboard, 256MB memory
> 
> The tools I'm familiar with for benchmarking a PATA based RAID system are:
>   - hdparm
>   - dd
> 
> Here are some interesting stats from my system:
> 

Here goes my setup/results, FWIW...
Hard:  Dual PIII@9333
       2 Promise FastTrak S150 TX4 (rev 02)
       6x Maxtor 7Y250M0, 250Gb, 3 on each card.
Soft:
> [root@epiam10k ~]# more /proc/mdstat
> Personalities : [raid5] 
> md0 : active raid5 sdd1[3] sdc1[2] sdb1[1] sda1[0]
>       879100608 blocks level 5, 4k chunk, algorithm 2 [4/4] [UUUU]
>       
> unused devices: <none>

nada:~# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid5] [multipath] [raid6] [raid10] 
md0 : active raid5 sdf1[5] sde1[4] sdd1[3] sdc1[2] sdb1[1] sda1[0]
      1225557760 blocks level 5, 256k chunk, algorithm 2 [6/6] [UUUUUU]
      
unused devices: <none>

> [root@epiam10k ~]# hdparm -t /dev/sda
> 
> /dev/sda:
>  Timing buffered disk reads:  116 MB in  3.02 seconds =  38.45 MB/sec
> HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for
> device
> [root@epiam10k ~]# hdparm -t /dev/sda1
> 
> /dev/sda1:
>  Timing buffered disk reads:  104 MB in  3.05 seconds =  34.10 MB/sec
> HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for
> device

nada:~# hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   900 MB in  2.00 seconds = 449.84 MB/sec
 Timing buffered disk reads:  172 MB in  3.03 seconds =  56.70 MB/sec

> [root@epiam10k ~]# hdparm -t /dev/md0
> 
> /dev/md0:
>  Timing buffered disk reads:   72 MB in  3.03 seconds =  23.79 MB/sec
> 

nada:~# hdparm -tT /dev/md0

/dev/md0:
 Timing cached reads:   912 MB in  2.01 seconds = 454.48 MB/sec
 Timing buffered disk reads:  178 MB in  3.04 seconds =  58.60 MB/sec

> [root@epiam10k ~]# vmstat
> procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
>  1  0      0   3628   8984 142704    0    0   423    23 1023   112 30  3 61  6
> 
> Observations:
>   - the performance of a raw SATA device (/dev/sda in the above example) seems
> low when I compare it to a PATA drive from a previous system (which would get
> about 45MB/sec)
>   - the performance of the RAID5 array (/dev/md0) seems very low - I expect
> quite an increase over a single device due to striping
>   - the number of interrupts per second (1023) seems very high
> 

It is not marvelous in my case, but slightly better than yours...

--
J.A. Magallon <jamagallon()able!es>     \               Software is like sex:
werewolf!able!es                         \         It's better when it's free
Mandrakelinux release 10.2 (Cooker) for i586
Linux 2.6.11-jam10 (gcc 3.4.3 (Mandrakelinux 10.2 3.4.3-7mdk)) #1

Re: Poor SATA / RAID performance (2.6.11 and promise SATAII150 TX4)

[Lee Revell]

On Wed, 2005-03-30 at 19:12 -0800, Tim Harvey wrote:
>   - the number of interrupts per second (1023) seems very high

Why?  You have 1000 timer interrupts every second, plus 23 from other
sources.

Lee