[LARTC] New hardware

[LARTC] New hardware

[Jurijs Petrovs]

[-- Attachment #1.1: Type: text/plain, Size: 1216 bytes --]

Hi! 

I want to upgrade hardware on my router (iptables, htb, >1000 users).
Now it is 
based on usual desktop PC (Intel Prescott P4 3.00 Ghz, 1 Gb RAM). The
reason of hardware upgrade is growing up number of users, also we are
planning to increase upstream link from 100 Mbit/s to 1 Gbit/s.
Iptables rules are now optimized with ipset tool, for tc I'm using
hash tables as well. So I didn't know how to increase performance of
my router without hardware upgrade... 

I’d appreciate if someone gives me advice – what system
shows the best performance: 

1) Server based router (with dual core Xeon CPU) 
2) Desktop based router (with newest Intel Core 2 Duo CPU) 
3) what about AMD (I have no personal experience with them)? 

As I know, 64-bit architecture in router specific tasks doesn’t
give any performance boost? Maybe there are some other nuances that I
need to know (kernel hacking, hardware hints)? 

I’m going to use standard Intel E1000 NIC for 1 Gbit/s speed
line, is it a good choice? 

Thank in advance! 

Advertisement:

Es iesu uz mobilo festivalu "re:loud"!
14 grupas, 2 skatuves, DJ telts, atrakcijas
1.septembri Mezaparka estrade www.reloud.lv

[-- Attachment #1.2: Type: text/html, Size: 3822 bytes --]

[-- Attachment #2: Type: text/plain, Size: 143 bytes --]

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] New hardware

[Marek Kierdelewicz]

> Hi! 

Hi

> reason of hardware upgrade is growing up number of users, also we are
> planning to increase upstream link from 100 Mbit/s to 1 Gbit/s.

If you want to move around 1Gbit/s you need PCI-X or PCI-E network
interfaces (PCI bus would be a bottleneck).

> 1) Server based router (with dual core Xeon CPU) 

I've been using for some time now router with 2xsingle-core Xeon CPU
for servicing 2k+ users. It works well. 

> 2) Desktop based router (with newest Intel Core 2 Duo CPU) 

Did't have a pleasure. I'd like to hear some opinions on that one too.

> 3) what about AMD (I have no personal experience with them)? 

I'm trying to use unorthodox hardware setup for fast router:

Dual core AMD64
desktop motherboard with 2xPCIe x16 slots (SLI capable motherboard)
two PCIe x4 intel E1000 Server network adapters

Preliminary tests look promising. Network adapters mentioned above
should in theory do better then previous PCI-X network adapters.

> 
> As I know, 64-bit architecture in router specific tasks doesn’t
> give any performance boost? Maybe there are some other nuances that I
> need to know (kernel hacking, hardware hints)? 

I think 64-bit architectures can do much better then 32-bit ones... at
least on 64-bit kernel.

> 
> I’m going to use standard Intel E1000 NIC for 1 Gbit/s speed
> line, is it a good choice? 

Best choice as far as I know. Anyone would sugest something else?

Regards,
-- 
Marek Kierdelewicz
Kierownik Dzia?u Systemów Sieciowych, KoBa
Manager of Network Systems Department, KoBa
tel. (85) 7406466; fax. (85) 7406467
e-mail: admin@koba.pl
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] New hardware

[Mohan Sundaram]

I used a SunFire x2100 which has 2 Ghz Opteron and PCI-Xpress slot into 
which I plugged in a dual port gigE Intel PCI-Xpress card.

Got a raw forwarding performance of 800Kpps for 64B packets. This beat a 
CISCO 3845 which gave 600Kpps.

Raw forwarding is one measure though it is not the end-all.

Mohan
Marek Kierdelewicz wrote:
>> Hi! 
> 
> Hi
> 
>> reason of hardware upgrade is growing up number of users, also we are
>> planning to increase upstream link from 100 Mbit/s to 1 Gbit/s.
> 
> If you want to move around 1Gbit/s you need PCI-X or PCI-E network
> interfaces (PCI bus would be a bottleneck).
> 
>> 1) Server based router (with dual core Xeon CPU) 
> 
> I've been using for some time now router with 2xsingle-core Xeon CPU
> for servicing 2k+ users. It works well. 
> 
>> 2) Desktop based router (with newest Intel Core 2 Duo CPU) 
> 
> Did't have a pleasure. I'd like to hear some opinions on that one too.
> 
>> 3) what about AMD (I have no personal experience with them)? 
> 
> I'm trying to use unorthodox hardware setup for fast router:
> 
> Dual core AMD64
> desktop motherboard with 2xPCIe x16 slots (SLI capable motherboard)
> two PCIe x4 intel E1000 Server network adapters
> 
> Preliminary tests look promising. Network adapters mentioned above
> should in theory do better then previous PCI-X network adapters.
> 
>> As I know, 64-bit architecture in router specific tasks doesn’t
>> give any performance boost? Maybe there are some other nuances that I
>> need to know (kernel hacking, hardware hints)? 
> 
> I think 64-bit architectures can do much better then 32-bit ones... at
> least on 64-bit kernel.
> 
>> I’m going to use standard Intel E1000 NIC for 1 Gbit/s speed
>> line, is it a good choice? 
> 
> Best choice as far as I know. Anyone would sugest something else?
> 
> Regards,

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] New hardware

[Marek Kierdelewicz]

Hi there,

> I used a SunFire x2100 which has 2 Ghz Opteron and PCI-Xpress slot
> into which I plugged in a dual port gigE Intel PCI-Xpress card.

What was the software configuration of this host? What kernel
have you used for the test (compiled for x86 or amd64; dual-core
aware sheduler or not).

Out of the curiosity ... dual port nic generates one interrupt or two
per port? I asume the first is more probable. This leads to another
question - has interrupt been balanced between cores? 

> Got a raw forwarding performance of 800Kpps for 64B packets. This
> beat a CISCO 3845 which gave 600Kpps.

Thanks for the input :).

Right now my production border router (BGP+some firewall rules+some QoS,
vlans @ P4 3GHz Linux 2.6, 2xBroadcom PCI-X, 1xIntel E1000 PCI-X) is 91%
saturated (CPU) at peek hours and during the time it forwards 344,8Kpps
with avg. packet sized 69 bytes.

I'll try to determine raw forwarding performance when box goes out
of production env, so we'll have a clear comparison picture.

regards,
-- 
Marek Kierdelewicz
Kierownik Dzia?u Systemów Sieciowych, KoBa
Manager of Network Systems Department, KoBa
tel. (85) 7406466; fax. (85) 7406467
e-mail: admin@koba.pl
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] New hardware

[Mohan Sundaram]

Marek Kierdelewicz wrote:
> Hi there,
> 
>> I used a SunFire x2100 which has 2 Ghz Opteron and PCI-Xpress slot
>> into which I plugged in a dual port gigE Intel PCI-Xpress card.
> 
> What was the software configuration of this host? What kernel
> have you used for the test (compiled for x86 or amd64; dual-core
> aware sheduler or not).
> 
stock x86 32 bit kernel 2.4.20. No dual core optimisations.
> Out of the curiosity ... dual port nic generates one interrupt or two
> per port? I asume the first is more probable. This leads to another
> question - has interrupt been balanced between cores? 
I'm unaware of how it was configured.
> 
>> Got a raw forwarding performance of 800Kpps for 64B packets. This
>> beat a CISCO 3845 which gave 600Kpps.
> 
> Thanks for the input :).
> 
> Right now my production border router (BGP+some firewall rules+some QoS,
> vlans @ P4 3GHz Linux 2.6, 2xBroadcom PCI-X, 1xIntel E1000 PCI-X) is 91%
> saturated (CPU) at peek hours and during the time it forwards 344,8Kpps
> with avg. packet sized 69 bytes.
> 
The Opteron was 100% utilised at 800Kpps for 64B pkts. I could do 
bi-directional line rate on gigE for 512/1500B pkts (approx 500/164Kpps) 
  with 65%/25% cpu utilisation
> I'll try to determine raw forwarding performance when box goes out
> of production env, so we'll have a clear comparison picture.
> 
> regards,

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] New hardware

[Mohan Sundaram]

Marek Kierdelewicz wrote:
> Hi there,
> 
>> I used a SunFire x2100 which has 2 Ghz Opteron and PCI-Xpress slot
>> into which I plugged in a dual port gigE Intel PCI-Xpress card.
> 
> What was the software configuration of this host? What kernel
> have you used for the test (compiled for x86 or amd64; dual-core
> aware sheduler or not).
> 
> Out of the curiosity ... dual port nic generates one interrupt or two
> per port? I asume the first is more probable. This leads to another
> question - has interrupt been balanced between cores? 
> 
>> Got a raw forwarding performance of 800Kpps for 64B packets. This
>> beat a CISCO 3845 which gave 600Kpps.
> 
> Thanks for the input :).
> 
> Right now my production border router (BGP+some firewall rules+some QoS,
> vlans @ P4 3GHz Linux 2.6, 2xBroadcom PCI-X, 1xIntel E1000 PCI-X) is 91%
> saturated (CPU) at peek hours and during the time it forwards 344,8Kpps
> with avg. packet sized 69 bytes.
> 
> I'll try to determine raw forwarding performance when box goes out
> of production env, so we'll have a clear comparison picture.
> 
> regards,

The beauty was the price. $725 for the SUN hardware and $200 for the NIC 
card. The CISCO 3845 would cost 9K street minimum.

Mohan
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc