Interesting Performance with Marvell

Interesting Performance with Marvell

[Terrence Martin]

So I am running a few tests with two Marvell chipset based systems now 
that I have a working driver. Thanks for all the help with that.  I am 
seeing some strange results though in performance. Basically my 4 port 
Marvell seems to be much faster on writes only than my 8 port.

Here are my bonie++ tests

I use a basic bonnie++ -u daemon -d bonnie on XFS file systems.
Where I change the scheduler from anticipatory I note it.

MV88SX6081 8-port SATA II PCI-X Controller (rev 09)
4 disk raid 0
Linux software raid 0
mdadm --create /dev/md2 --chunk=256 --level=0 --raid-devices=4 /dev/sdc1 
/dev/sdd1 /dev/sde1 /dev/sdf1
ST3400832AS
3.2Ghz Xeon 4GB RAM Dual Socket
4GB RAM
Supermicro X6DVL-EG2
2.6.15-rc5
XFS
CFQ Scheduler
Version  1.03       ------Sequential Output------ --Sequential Input- 
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  
/sec %CP
osg-fs1.local    8G 48009  99 56987  11 27792   7 50606  97 191749  19 
360.8   0
                    ------Sequential Create------ --------Random 
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
                 16  3724  18 +++++ +++  2549  11  2543  15 +++++ +++  
1772   7
osg-fs1.local,8G,48009,99,56987,11,27792,7,50606,97,191749,19,360.8,0,16,3724,18,+++++,+++,2549,11,2543,15,+++++,+++,1772,7

Marvell Technology Group Ltd. MV88SX6041 4-port SATA II PCI-X Controller 
(rev 03)
2 Disk RAID0
ST3200826AS
3.2Ghz Xeon 4GB RAM Dual Socket
4GB RAM
Supermicro X6DHR-TG
2.6.15-rc5
XFS
mdadm --create /dev/md2 --chunk=256 --level=0 --raid-devices=2 /dev/sdc1 
/dev/sdd1
Version  1.03       ------Sequential Output------ --Sequential Input- 
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  
/sec %CP
osg-gw-2.t2.ucsd 8G 48118  99 154782  36 45164  11 49961  96 133858  13 
371.7   0
                    ------Sequential Create------ --------Random 
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
                 16  9288  46 +++++ +++  4842  19  5011  25 +++++ +++  
4061  21
osg-gw-2.t2.ucsd.edu,8G,48118,99,154782,36,45164,11,49961,96,133858,13,371.7,0,16,9288,46,+++++,+++,4842,19,5011,25,+++++,+++,4061,21

What I find interesting about these two runs is that the two disk RAID0 
is much faster on writes than the 4 disk RAID0 with much better balance. 
Memory, Kernel, CPU are all the same. The motherboards are different and 
I have noted them. The 4 port is onboard, the 8port is a PCI-X card in a 
PCI-X slot.

Here is the onboard Marvell with only one drive. Even with one drive I 
seem to be able to beat the write performance of the 4 drive raid0 on 
the PCI-X controller. I cannot imagine the 400 GB Seagates are slower on 
writes by that much over the 200GB seagates.

Marvell Technology Group Ltd. MV88SX6041 4-port SATA II PCI-X Controller 
(rev 03)
1 Disk no raid
ST3200826AS
3.2Ghz Xeon 4GB RAM Dual Socket
4GB RAM
Supermicro X6DHR-TG
2.6.15-rc5
XFS
Version  1.03       ------Sequential Output------ --Sequential Input- 
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  
/sec %CP
osg-gw-2.t2.ucsd 8G 47861  99 75955  16 28891   6 35003  66 67958   6 
202.2   0
                    ------Sequential Create------ --------Random 
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
                 16  5807  26 +++++ +++  5190  19  5460  25 +++++ +++  
3712  15
osg-gw-2.t2.ucsd.edu,8G,47861,99,75955,16,28891,6,35003,66,67958,6,202.2,0,16,5807,26,+++++,+++,5190,19,5460,25,+++++,+++,3712,15

Here is the 8 port again, this time with 2 disk raid0. I have also gone 
back to the default anticipatory scheduler.

MV88SX6081 8-port SATA II PCI-X Controller (rev 09)
2 disk raid 0
Linux software raid 0
mdadm --create /dev/md2 --chunk=256 --level=0 --raid-devices=4 /dev/sdc1 
/dev/sdd1 /dev/sde1 /dev/sdf1
ST3400832AS
3.2Ghz Xeon 4GB RAM Dual Socket
4GB RAM
Supermicro X6DVL-EG2
2.6.15-rc5
XFS
Version  1.03       ------Sequential Output------ --Sequential Input- 
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  
/sec %CP
osg-fs1.local    8G 29588  61 33894   7 21057   5 47502  91 134689  13 
397.5   0
                    ------Sequential Create------ --------Random 
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- 
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
                 16  2236  10 +++++ +++  1492   6  1794   8 +++++ +++  
1175   5
osg-fs1.local,8G,29588,61,33894,7,21057,5,47502,91,134689,13,397.5,0,16,2236,10,+++++,+++,1492,6,1794,8,+++++,+++,1175,5

Again the read performance is very nice. The write performance though is 
not so good.

So is this driver? Hardware?

Sebastian what are the mfg of the Marvell controller you are using?

Terrence

Re: Interesting Performance with Marvell

[Sebastian Kuzminsky]

Terrence Martin <tmartin@physics.ucsd.edu> wrote:
> Sebastian what are the mfg of the Marvell controller you are using?

Sonnet.  It's their "Tempo-X 8 SATA" card.  PCI-X, 8 external SATA
ports and no internal ports.  I'll post performance numbers soon or
after Newtonmas.

Who made you 8-port card?


-- 
Sebastian Kuzminsky

Re: Interesting Performance with Marvell

[Terrence Martin]

Sebastian Kuzminsky wrote:

>Terrence Martin <tmartin@physics.ucsd.edu> wrote:
>  
>
>>Sebastian what are the mfg of the Marvell controller you are using?
>>    
>>
>
>Sonnet.  It's their "Tempo-X 8 SATA" card.  PCI-X, 8 external SATA
>ports and no internal ports.  I'll post performance numbers soon or
>after Newtonmas.
>
>Who made you 8-port card?
>
>
>  
>
Supermicro is the manufacturer of the board. As far as I can tell it is 
just a PCI-X SATA controller with the Marvell chipset driving 8 
interfaces. I am evaluating various SATA controllers, raid and non-raid 
to see if any actually can provide decent performance. So far 3ware 
9550SX is ok, the Areca 1110 is not that impressive using their driver 
at least. The Marvell 8 port addin card is comparable to the onboard 
Marvell 4 port for reads delivering 190MB/s but it is much slower on 
writes for some reason.

I know that drivers can make a significant difference. With 3ware 9500 
4LP cards the 3ware driver provides about twice the performance of the 
one that comes with redhats RHEL4.2 kernels. I have not tried kernel.org 
kernels on that card. 9550SX is a 3ware provided driver.

I just see a lot of variance between cards and cost seldom seems to 
indicate performance. The supermicro card is about $125US so is good 
value if I can figure out why the poor write performance compared to the 
onboard. The onboard I am happy with so far. My opterons have the 
marvell onboard as well, 4 port. I will be checking those next.

Terrence