Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

[Song Zhao]

Hi, 

I've been doing some benchmarks with this board, it is terribly disppointing. 
Has anyone had similar experiences?

The hardware spec is:
Dual 2.8GHz Xeon, 3ware Escalade 7850 (7500-8) 12 port IDE RAID controller, 
RAID 10, 4x 1GB DDR SDRAM Registered ECC, 2x 80GB WD HDD, 10x 120GB WD HDD, 
ServerWorks Grand Champion LE. 

I am running RH7.3 with 2.4.20 kernel. The performance of this box is about 
half of an almost identical box (Supermicro X5DP8-G2 mobo, E7501 chipset)

Also, this board can't even boot with 8x 1GB memory modules plugged in (8 DIMM 
slots in total). This is a relative new board and I can't find anything 
relevant on the net. 

cheers,
Song Zhao

Re: Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

[Jurriaan]

From: Song Zhao <song.zhao@nuix.com.au>
Date: Thu, Feb 20, 2003 at 08:34:28PM -0500
> Hi, 
> 
> I've been doing some benchmarks with this board, it is terribly disppointing. 
> Has anyone had similar experiences?

How many people do you guess own such hardware? Not me :-(

> 
> The hardware spec is:
> Dual 2.8GHz Xeon, 3ware Escalade 7850 (7500-8) 12 port IDE RAID controller, 
> RAID 10, 4x 1GB DDR SDRAM Registered ECC, 2x 80GB WD HDD, 10x 120GB WD HDD, 
> ServerWorks Grand Champion LE. 
> 
> I am running RH7.3 with 2.4.20 kernel. The performance of this box is about 
> half of an almost identical box (Supermicro X5DP8-G2 mobo, E7501 chipset)

what does

cat /proc/mtrr

say?

> 
> Also, this board can't even boot with 8x 1GB memory modules plugged in (8 DIMM 
> slots in total). This is a relative new board and I can't find anything 
> relevant on the net. 
> 
"can't boot" as in crashes halfway during linux or doesn't even start
lilo?

HTH,
Jurriaan
-- 
We are lost, we are freaks
We are crippled, we are weak
We are the heirs, we are the true heirs to all the world.
        New Model Army - Ballad of Bodmin Pill
GNU/Linux 2.5.61 SMP/ReiserFS 2x2105 bogomips load av: 0.93 1.05 0.60

Re: Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

[Song Zhao]

On Thu, 20 Feb 2003 05:21 am, Jurriaan wrote:
> From: Song Zhao <song.zhao@nuix.com.au>
> Date: Thu, Feb 20, 2003 at 08:34:28PM -0500
>
> > Hi,
> >
> > I've been doing some benchmarks with this board, it is terribly
> > disppointing. Has anyone had similar experiences?
>
> How many people do you guess own such hardware? Not me :-(
>
> > The hardware spec is:
> > Dual 2.8GHz Xeon, 3ware Escalade 7850 (7500-8) 12 port IDE RAID
> > controller, RAID 10, 4x 1GB DDR SDRAM Registered ECC, 2x 80GB WD HDD, 10x
> > 120GB WD HDD, ServerWorks Grand Champion LE.
> >
> > I am running RH7.3 with 2.4.20 kernel. The performance of this box is
> > about half of an almost identical box (Supermicro X5DP8-G2 mobo, E7501
> > chipset)
>
> what does
>
> cat /proc/mtrr
>
> say?

sorry I can't tell you right now, as I have just replaced the motherboard with 
another Supermicro mobo, this time a E7500 chipset. As soon as I get it back 
online, I'll let you know. 

>
> > Also, this board can't even boot with 8x 1GB memory modules plugged in (8
> > DIMM slots in total). This is a relative new board and I can't find
> > anything relevant on the net.
>
> "can't boot" as in crashes halfway during linux or doesn't even start
> lilo?

It doesn't even start Lilo, it hangs after it checks memory, 3ware card and 
network card. 

>
> HTH,
> Jurriaan

Re: Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

[Bruce Harada]

On Thu, 20 Feb 2003 22:01:15 -0500
Song Zhao <song.zhao@nuix.com.au> wrote:

> > > Also, this board can't even boot with 8x 1GB memory modules plugged in
> > > (8 DIMM slots in total). This is a relative new board and I can't find
> > > anything relevant on the net.
> >
> > "can't boot" as in crashes halfway during linux or doesn't even start
> > lilo?
> 
> It doesn't even start Lilo, it hangs after it checks memory, 3ware card and 
> network card. 

In that case, it has nothing to do with Linux... Have you checked to see if
there's any BIOS updates?

Re: Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

[Song Zhao]

On Thu, 20 Feb 2003 07:51 am, Bruce Harada wrote:
> On Thu, 20 Feb 2003 22:01:15 -0500
>
> Song Zhao <song.zhao@nuix.com.au> wrote:
> > > > Also, this board can't even boot with 8x 1GB memory modules plugged
> > > > in (8 DIMM slots in total). This is a relative new board and I can't
> > > > find anything relevant on the net.
> > >
> > > "can't boot" as in crashes halfway during linux or doesn't even start
> > > lilo?
> >
> > It doesn't even start Lilo, it hangs after it checks memory, 3ware card
> > and network card.
>
> In that case, it has nothing to do with Linux... Have you checked to see if
> there's any BIOS updates?

I haven't checked for BIOS updates yet, we were planning to try a different 
brand of memory modules. The chipset is supposed to be able to support up to 
16GB. 8GB should definitely be supported by the default BIOS revision. I put 
4GB in it, and it boots up fine. 

Also, this has nothing to do with the fact that it runs slow when its up and 
running with 4GB. 

Re: Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

[Song Zhao]

On Thu, 20 Feb 2003 06:06 am, Jurriaan wrote:
> From: Song Zhao <song.zhao@nuix.com.au>
> Date: Thu, Feb 20, 2003 at 10:01:15PM -0500
>
> > > > Also, this board can't even boot with 8x 1GB memory modules plugged
> > > > in (8 DIMM slots in total). This is a relative new board and I can't
> > > > find anything relevant on the net.
> > >
> > > "can't boot" as in crashes halfway during linux or doesn't even start
> > > lilo?
> >
> > It doesn't even start Lilo, it hangs after it checks memory, 3ware card
> > and network card.
>
> A bios issue? That would be something for the people you bought it from,
> then.

Our supplier is in the process of contacting Supermicro about it. My other 
Supermicro X5DP8-G2 (E7501 chipset) is running beautifully. 

Re: Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

[Dave Jones]

On Thu, Feb 20, 2003 at 08:34:28PM -0500, Song Zhao wrote:

 > Dual 2.8GHz Xeon, 3ware Escalade 7850 (7500-8) 12 port IDE RAID controller, 
 > RAID 10, 4x 1GB DDR SDRAM Registered ECC, 2x 80GB WD HDD, 10x 120GB WD HDD, 
 > ServerWorks Grand Champion LE. 
 > I am running RH7.3 with 2.4.20 kernel. The performance of this box is about 
 > half of an almost identical box (Supermicro X5DP8-G2 mobo, E7501 chipset)

You mentioned nothing about what sort of performance you were measuring.
Disk, network, memory bandwidth etc.., however at a complete guess you
are hitting this..

mtrr: Serverworks LE detected. Write-combining disabled.

This workaround was for older sewerworks chipsets which were
buggy. Rumour has it that revisions 6 and above are ok.
I have a patch pending for 2.5, if it turns out to be stable,
it should also get merged back to 2.4

		Dave

-- 
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs

Re: Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

[Song Zhao]

To: Manish Lachwani <manish@Zambeel.com>

On Thu, 20 Feb 2003 12:54 pm, you wrote:
> WHat kind of benchmarks are you running? I had evaluated a Supermicro board
> with e7500 chipset and it showed good performance. Are you benchmarking
> network, memory, disk? Note that 3ware controllers operate at 33 MHZ PCI
> and if the nics share the same PCI bus with 3ware, they too will operate at
> 33 MHZ PCI and not 133 MHZ. Have you disabled hyperthreading in the BIOS?

I did some disk I/O and CPU benchmarks, including bonnie++, hdparm, nbench,
unixbench, dbench, tiotest. I haven't done any network/memory testing yet,
the onboard Broadcom and Intel gigabit controllers both seem to work quite
well. I have taken care to make sure that the 3ware card has its own PCI bus,
in this case slot 5. I have not disabled hyperthreading but I am not sure why
I would.

On that note, I have also done benchmarking with a Supermicro E7500 board
(P4DPR, Intel Gigabit 82544GC onboard controller), the tests I ran included
Netperf, Netperf3, NetPIPE (TCP), Tbench, nttcp, Aim9 and the ones I
mentioned above. It seemed that if I had two identical machines hooked up
directly (back to back with CAT5E crossover cable), the performance would be
a lot better and is very consistent. I tried different combinations
HT=on/off, CPU affinity, IRQ affinity, SMP/UP. Some results were:

Supermicro
=========
Netperf Result (MB/s) - 112.21
Netperf3 Result (MB/s) - 112.22
Tbench Result (MB/s) - 114.48
Nttcp Result (Mb/s) - 946.55

However, if I hook it up to a different machine, problems started to occur.
For example, with the Tyan S2720 Thunder i7500  board (E7500 chipset, Intel
82544EI onboard gigabit controller), I see good performance on the Tyan but
not Supermicro.

Supermicro
=========
Netperf Result (MB/s) - 29.23
Netperf3 Result (MB/s) - 78.19
Tbench Result (MB/s) - 117.32
Nttcp Result (Mb/s) - 646.77

Tyan
====
Netperf Result (MB/s) - 112.22
Netperf3 Result (MB/s) - 112.02
Tbench Result (MB/s) - can't find it
Nttcp Result (Mb/s) - 945.9

-------------------------------------------------------

Re: Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

[Song Zhao]

Date: Fri, 21 Feb 2003 12:24:10 -0500
From: Song Zhao <song.zhao@nuix.com.au>
To: Dave Jones <davej@codemonkey.org.uk>

On Thu, 20 Feb 2003 09:57 am, you wrote:
> On Thu, Feb 20, 2003 at 08:34:28PM -0500, Song Zhao wrote:
>  > Dual 2.8GHz Xeon, 3ware Escalade 7850 (7500-8) 12 port IDE RAID
>  > controller, RAID 10, 4x 1GB DDR SDRAM Registered ECC, 2x 80GB WD HDD,
>  > 10x 120GB WD HDD, ServerWorks Grand Champion LE.
>  > I am running RH7.3 with 2.4.20 kernel. The performance of this box is
>  > about half of an almost identical box (Supermicro X5DP8-G2 mobo, E7501
>  > chipset)
>
> You mentioned nothing about what sort of performance you were measuring.
> Disk, network, memory bandwidth etc.., however at a complete guess you
> are hitting this..

I did some disk I/O and CPU benchmarks, including bonnie++, hdparm, nbench,
unixbench, dbench, tiotest. I haven't done any network/memory testing yet.

Here is a rough comparison of E7500, E7501 and the ServerWorks Chipset:

==========================================================================

| Benchmark 				| E7500E7501 	| ServerWorks 	| GrandChampion LE 	|

==========================================================================

| Nbench (integer index) 		| 33.47 		| 38.78 		| 10.61 			|

==========================================================================

| Nbench (floating-point index) 	| 27.03 		| 32.05 		| 20.87 			|

==========================================================================

| Unixbench index			| 329 		| 349.5 		| 141.6 			|

==========================================================================

| Hdparm -t 				| 70.33MB/s 	| 73.73MB/s 	| 46.04MB/s 		|

==========================================================================

| Hdparm -T 				| 512MB/s 		| 673.68MB/s 	| 673.68MB/s 		|

==========================================================================

| Tiobench (write) 			| 41.2MB/s		| 42.09MB/s	| 44.37MB/s		|

==========================================================================

| Tiobench (random write) 		| 7.03MB/s 	| 9.73MB/s 	| 10.87MB/s 		|

==========================================================================

| Tiobench (read) 			| 1520.05MB/s 	| 1528.2MB/s 	| 1267.15MB/s 		|

==========================================================================

| Tiobench (random read) 		| 1391.36MB/s 	| 1471.28MB/s	| 1041.67MB/s 		|

==========================================================================

> mtrr: Serverworks LE detected. Write-combining disabled.

from dmesg:

mtrr: v1.40 (20010327) Richard Gooch (rgooch@atnf.csiro.au)
mtrr: detected mtrr type: Intel

ServerWorks CSB5: IDE controller on PCI bus 00 dev 79
ServerWorks CSB5: chipset revision 147
ServerWorks CSB5: not 100% native mode: will probe irqs later

from lspci (please note that I took out 3ware card which I mentioned in my
previous mail):

00:00.0 Host bridge: ServerWorks: Unknown device 0014 (rev 31)
00:00.1 Host bridge: ServerWorks: Unknown device 0014
00:00.2 Host bridge: ServerWorks: Unknown device 0014
00:02.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
00:04.0 Ethernet controller: Intel Corp.: Unknown device 100e (rev 02)
00:0f.0 ISA bridge: ServerWorks CSB5 South Bridge (rev 93)
00:0f.1 IDE interface: ServerWorks CSB5 IDE Controller (rev 93)
00:0f.3 Host bridge: ServerWorks: Unknown device 0225
00:10.0 Host bridge: ServerWorks: Unknown device 0101 (rev 03)
00:10.2 Host bridge: ServerWorks: Unknown device 0101 (rev 03)
00:11.0 Host bridge: ServerWorks: Unknown device 0101 (rev 03)
00:11.2 Host bridge: ServerWorks: Unknown device 0101 (rev 03)
01:03.0 Ethernet controller: BROADCOM Corporation NetXtreme BCM5703X Gigabit
Ethernet (rev 02)

> This workaround was for older sewerworks chipsets which were
> buggy. Rumour has it that revisions 6 and above are ok.
> I have a patch pending for 2.5, if it turns out to be stable,
> it should also get merged back to 2.4
>
> 		Dave

-------------------------------------------------------

Re: Supermicro X5DL8-GG (ServerWorks Grandchampion LE chipset) slow

[Song Zhao]

On Sat, 22 Feb 2003 02:43 pm, you wrote:
> > >  > 10x 120GB WD HDD, ServerWorks Grand Champion LE.
> > >  > I am running RH7.3 with 2.4.20 kernel. The performance of this box
> > >  > is about half of an almost identical box (Supermicro X5DP8-G2 mobo,
> > >  > E7501
>
> well, I've rarely seen anyone claiming good performance for any SW chipset,
> especially compared to i75xx's.

I wonder where the problem lies? Is it hardware/software?

>
> > Here is a rough comparison of E7500, E7501 and the ServerWorks Chipset:
>
> I don't really understand the column headed "E7500E7501".  which is it?
> 7500 (dual PC1600) or 7501 (dual pc2100)?
>
> > | Benchmark 				| E7500E7501 	| ServerWorks 	| GrandChampion LE 	|
> >
> > =========================================================================
> >=
> >
> > | Nbench (integer index) 		| 33.47 		| 38.78 		| 10.61 			|
>
> oh, maybe the headers are just broken?  I can readily believe that

You are right, the header is broken. 

> 7500 is 33, 7501 is a little higher, and GCLE is a lot lower.
> remember that this benchmark spends most of its time in strcpy/strcmp...
>
> hmm, I'd be curious to see whether lmbench indicates the GCLE's memory
> latency is much higher than Intel's.  your hdparm -t score indicates that
> the GCLE doesn't have a memory *bandwidth* problem.

Yeah, I noticed that too, buffer cache read is pretty impressive actually. 

I haven't had a chance to run lmbench as it takes about 5 hours to complete. 
Will probably do it within these couple of days.