Marvell 88SX7042 [4 Port SATA PCI-Express x4] Support/Questions

Marvell 88SX7042 [4 Port SATA PCI-Express x4] Support/Questions

[Justin Piszcz]

Jeff has been working on this chipset/patching/etc:
http://forum.soft32.com/linux/PATCH-sata_mv-HighPoint-2310-support-88SX7042-ftopict338643.html

Manual/specs:
http://www.startech.com/Share/ProductSpecs/PDF/PEXSATA24E.pdf
http://www.rosewill.com/RosewillSoftware/UserManual%20for%20RC-218-WEB%20v1.0.pdf

Looks like there are a couple cards sporting this new chip at the moment, 
here is one:
http://www.newegg.com/Product/Product.aspx?Item=N82E16816132018

http://en.wikipedia.org/wiki/List_of_Marvell_Technology_Group_chipsets#88SXxxxx_SATA_Controllers_Chipsets
It also appears to be supported in the latest kernel.

If one bought enough of these, one could possibly achieve speeds in 
excess(!) of 1 gigabyte/second with enough drives and SW RAID in Linux.
Does anyone have such a card? I would be interested if it could sustain 
the maximum rate from each disk without any contention.

(with an old 965 chipset for example and a lot of 2-port PCI-e cards)
dd if=/dev/sda of=/dev/null bs=1M
dd if=/dev/sdb of=/dev/null bs=1M

The total read speed increases linearly with each additional disk.  Up to 
~700-800 megabytes per second with 12 disks (as reported by dstat) (or 
vmstat 1)..  Same with write (but slower, obviously)...

So my questions are:
1. Does anyone have one of these cards? (not the highpoint with the same 
chip) as that is a RAID card.
2. How 'experimental' is it?

Any other comments?

There are motherboards with multiple PCI-e x8 slots (5) if you look around 
hard enough, imagine 5 of these cards with 20 hard drives, you could have 
one very fast raid array..!

Justin.

Re: Marvell 88SX7042 [4 Port SATA PCI-Express x4] Support/Questions

[Mark Lord]

Justin Piszcz wrote:
> Jeff has been working on this chipset/patching/etc:
..

Actually, I'm the one working on that chipset etc..  :)

> Looks like there are a couple cards sporting this new chip at the moment
..

There are lots of them out there, including Sonnet, Highpoint, and others.

> If one bought enough of these, one could possibly achieve speeds in 
> excess(!) of 1 gigabyte/second with enough drives and SW RAID in Linux.
> Does anyone have such a card? I would be interested if it could sustain 
> the maximum rate from each disk without any contention.
..

Sure, if the bus and memory are fast enough.  You'll hit the Linux/libata
transactions/sec limit at some point, but I don't know what that is.

Eg. Here's a 4X (I think) card in my PCIe video slot:

beefy:~# mdadm --create /dev/md0 --chunk=256 -l 0 --raid-devices=4 /dev/sd[bcde]
mdadm: array /dev/md0 started.

beefy:~# hdparm -t --direct /dev/sd[bcde] /dev/md0

/dev/sdb:
 Timing O_DIRECT disk reads:  218 MB in  3.01 seconds =  72.46 MB/sec

/dev/sdc:
 Timing O_DIRECT disk reads:  256 MB in  3.01 seconds =  85.00 MB/sec

/dev/sdd:
 Timing O_DIRECT disk reads:  192 MB in  3.01 seconds =  63.79 MB/sec

/dev/sde:
 Timing O_DIRECT disk reads:  256 MB in  3.01 seconds =  85.07 MB/sec

/dev/md0:
 Timing O_DIRECT disk reads:  768 MB in  3.00 seconds = 255.69 MB/sec

Similar results happen without the "--direct" flag as well.

> 2. How 'experimental' is it? 

The 7042 chipset is working rather well right now.
Older Marvell chips should still be considered "experimental"
for the time being.

Cheers

Re: Marvell 88SX7042 [4 Port SATA PCI-Express x4] Support/Questions

[Andre Tomt]

Mark Lord wrote:
> Justin Piszcz wrote:
>> If one bought enough of these, one could possibly achieve speeds in 
>> excess(!) of 1 gigabyte/second with enough drives and SW RAID in Linux.
>> Does anyone have such a card? I would be interested if it could 
>> sustain the maximum rate from each disk without any contention.
> ..
> 
> Sure, if the bus and memory are fast enough.  You'll hit the Linux/libata
> transactions/sec limit at some point, but I don't know what that is.

I'm seeing 800MB/s on ICH8/9R class chipsets on concurrent non-raid I/O, 
however that's the limit of the PCIe bus used between the ICH (south, 
IO) and the MCH (north, memory/cpu/video-pcie hub). ICH10 is supposed to 
double that by moving up to PCI-e 2.0 on that link AFAIK.

> Eg. Here's a 4X (I think) card in my PCIe video slot:

Ah, of course making use of the video PCIe slot by putting as many ports 
as you can on it will improve the situation somewhat as its generally 
connected to the MCH directly. (mm, any plain/non-raid 88SX7xxx PCIe 
cards with 8 ports available?)

Re: Marvell 88SX7042 [4 Port SATA PCI-Express x4] Support/Questions

[Grant Grundler]

On Sun, May 25, 2008 at 10:47 AM, Mark Lord <liml@rtr.ca> wrote:
...
> Sure, if the bus and memory are fast enough.  You'll hit the Linux/libata
> transactions/sec limit at some point, but I don't know what that is.
>
> Eg. Here's a 4X (I think) card in my PCIe video slot:
>
> beefy:~# mdadm --create /dev/md0 --chunk=256 -l 0 --raid-devices=4
> /dev/sd[bcde]
> mdadm: array /dev/md0 started.
>
> beefy:~# hdparm -t --direct /dev/sd[bcde] /dev/md0
>
> /dev/sdb:
> Timing O_DIRECT disk reads:  218 MB in  3.01 seconds =  72.46 MB/sec
>
> /dev/sdc:
> Timing O_DIRECT disk reads:  256 MB in  3.01 seconds =  85.00 MB/sec
>
> /dev/sdd:
> Timing O_DIRECT disk reads:  192 MB in  3.01 seconds =  63.79 MB/sec
>
> /dev/sde:
> Timing O_DIRECT disk reads:  256 MB in  3.01 seconds =  85.07 MB/sec
>
> /dev/md0:
> Timing O_DIRECT disk reads:  768 MB in  3.00 seconds = 255.69 MB/sec

The bottle neck here is with the four disks.
Modern 1TB 7200 rpm disks have raw throughput rates up to around
100MB/s. Anything in the last 4-5 years should do better than 60MB/s.

In theory, ~16 disks (4 disks per 7042 port using Port Mulitpliers),
one should be able to hit 200+ MB/s per 7042 port and approach 900MB/s
total. Getting more than 900Mb/s will depend on the chipset. IIRC, the
MMRBC of 128 bytes will result in about 15% protocol overhead in the
PCI-e link.
Bigger PCI-e "burst" will result in dramatically less overhead (IIRC,
256Byte == 9% overhead.)

hth,
grant