From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jurijs Petrovs Date: Mon, 21 Aug 2006 15:46:18 +0000 Subject: [LARTC] New hardware Message-Id: <1156175178.44e9d54a18bd1@www.inbox.lv> MIME-Version: 1 Content-Type: multipart/mixed; boundary="===============0121982951==" List-Id: To: lartc@vger.kernel.org --===============0121982951== Content-Transfer-Encoding: 7bit Content-Type: multipart/alternative; boundary="-moq11561751784c2e86598877692fc9da84b55b64d28a" ---moq11561751784c2e86598877692fc9da84b55b64d28a Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi! =0A=0AI want to upgrade hardware on my router (iptables, htb, >1000 use= rs).=0ANow it is =0Abased on usual desktop PC (Intel Prescott P4 3.00 Ghz, = 1 Gb RAM). The=0Areason of hardware upgrade is growing up number of users, = also we are=0Aplanning to increase upstream link from 100 Mbit/s to 1 Gbit/= s.=0AIptables rules are now optimized with ipset tool, for tc I'm using=0Ah= ash tables as well. So I didn't know how to increase performance of=0Amy ro= uter without hardware upgrade... =0A=0AI’d appreciate if someone give= s me advice – what system=0Ashows the best performance: =0A=0A1) Serv= er based router (with dual core Xeon CPU) =0A2) Desktop based router (with = newest Intel Core 2 Duo CPU) =0A3) what about AMD (I have no personal exper= ience with them)? =0A=0AAs I know, 64-bit architecture in router specific t= asks doesn’t=0Agive any performance boost? Maybe there are some other= nuances that I=0Aneed to know (kernel hacking, hardware hints)? =0A=0AI&rs= quo;m going to use standard Intel E1000 NIC for 1 Gbit/s speed=0Aline, is i= t a good choice? =0A=0AThank in advance! =0A Advertisement: Es iesu uz mobilo festivalu "re:loud"! 14 grupas, 2 skatuves, DJ telts, atrakcijas 1.septembri Mezaparka estrade www.reloud.lv ---moq11561751784c2e86598877692fc9da84b55b64d28a Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi!

I want to upgrade hardware on my router (ip= tables, htb, >1000 users). Now it is
based on usual desktop PC (In= tel Prescott P4 3.00 Ghz, 1 Gb RAM). The reason of hardware upgrade is grow= ing up number of users, also we are planning to increase upstream link from= 100 Mbit/s to 1 Gbit/s. Iptables rules are now optimized with ipset tool, = for tc I'm using hash tables as well. So I didn't know how to increase perf= ormance of my router without hardware upgrade...

I’d appr= eciate if someone gives me advice – what system shows the best perfor= mance:

1) Server based router (with dual core Xeon CPU)
2= ) Desktop based router (with newest Intel Core 2 Duo CPU)
3) what abo= ut AMD (I have no personal experience with them)?

As I know, 64= -bit architecture in router specific tasks doesn’t give any performan= ce boost? Maybe there are some other nuances that I need to know (kernel ha= cking, hardware hints)?

I’m going to use standard Intel E= 1000 NIC for 1 Gbit/s speed line, is it a good choice?

Thank in= advance!
__________
Adver= tisement:

Es iesu uz mobilo festivalu =84re:loud=94!
14 grupas, 2 skatuves, DJ telts, atrakcijas
1.septembri Mezaparka estrade www.re= loud.lv
=0A ---moq11561751784c2e86598877692fc9da84b55b64d28a-- --===============0121982951== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc --===============0121982951==-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marek Kierdelewicz Date: Mon, 21 Aug 2006 16:16:12 +0000 Subject: Re: [LARTC] New hardware Message-Id: <20060821181612.25e0d962@localhost.localdomain> List-Id: References: <1156175178.44e9d54a18bd1@www.inbox.lv> In-Reply-To: <1156175178.44e9d54a18bd1@www.inbox.lv> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable To: lartc@vger.kernel.org > Hi!=20 Hi > reason of hardware upgrade is growing up number of users, also we are > planning to increase upstream link from 100 Mbit/s to 1 Gbit/s. If you want to move around 1Gbit/s you need PCI-X or PCI-E network interfaces (PCI bus would be a bottleneck). > 1) Server based router (with dual core Xeon CPU)=20 I've been using for some time now router with 2xsingle-core Xeon CPU for servicing 2k+ users. It works well.=20 > 2) Desktop based router (with newest Intel Core 2 Duo CPU)=20 Did't have a pleasure. I'd like to hear some opinions on that one too. > 3) what about AMD (I have no personal experience with them)?=20 I'm trying to use unorthodox hardware setup for fast router: Dual core AMD64 desktop motherboard with 2xPCIe x16 slots (SLI capable motherboard) two PCIe x4 intel E1000 Server network adapters Preliminary tests look promising. Network adapters mentioned above should in theory do better then previous PCI-X network adapters. >=20 > As I know, 64-bit architecture in router specific tasks doesn’t > give any performance boost? Maybe there are some other nuances that I > need to know (kernel hacking, hardware hints)?=20 I think 64-bit architectures can do much better then 32-bit ones... at least on 64-bit kernel. >=20 > I’m going to use standard Intel E1000 NIC for 1 Gbit/s speed > line, is it a good choice?=20 Best choice as far as I know. Anyone would sugest something else? Regards, --=20 Marek Kierdelewicz Kierownik Dzia?u System=F3w Sieciowych, KoBa Manager of Network Systems Department, KoBa tel. (85) 7406466; fax. (85) 7406467 e-mail: admin@koba.pl _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mohan Sundaram Date: Tue, 22 Aug 2006 02:21:24 +0000 Subject: Re: [LARTC] New hardware Message-Id: <44EA6754.6000909@vsnl.com> List-Id: References: <1156175178.44e9d54a18bd1@www.inbox.lv> In-Reply-To: <1156175178.44e9d54a18bd1@www.inbox.lv> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: lartc@vger.kernel.org I used a SunFire x2100 which has 2 Ghz Opteron and PCI-Xpress slot into which I plugged in a dual port gigE Intel PCI-Xpress card. Got a raw forwarding performance of 800Kpps for 64B packets. This beat a CISCO 3845 which gave 600Kpps. Raw forwarding is one measure though it is not the end-all. Mohan Marek Kierdelewicz wrote: >> Hi! > > Hi > >> reason of hardware upgrade is growing up number of users, also we are >> planning to increase upstream link from 100 Mbit/s to 1 Gbit/s. > > If you want to move around 1Gbit/s you need PCI-X or PCI-E network > interfaces (PCI bus would be a bottleneck). > >> 1) Server based router (with dual core Xeon CPU) > > I've been using for some time now router with 2xsingle-core Xeon CPU > for servicing 2k+ users. It works well. > >> 2) Desktop based router (with newest Intel Core 2 Duo CPU) > > Did't have a pleasure. I'd like to hear some opinions on that one too. > >> 3) what about AMD (I have no personal experience with them)? > > I'm trying to use unorthodox hardware setup for fast router: > > Dual core AMD64 > desktop motherboard with 2xPCIe x16 slots (SLI capable motherboard) > two PCIe x4 intel E1000 Server network adapters > > Preliminary tests look promising. Network adapters mentioned above > should in theory do better then previous PCI-X network adapters. > >> As I know, 64-bit architecture in router specific tasks doesn’t >> give any performance boost? Maybe there are some other nuances that I >> need to know (kernel hacking, hardware hints)? > > I think 64-bit architectures can do much better then 32-bit ones... at > least on 64-bit kernel. > >> I’m going to use standard Intel E1000 NIC for 1 Gbit/s speed >> line, is it a good choice? > > Best choice as far as I know. Anyone would sugest something else? > > Regards, _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marek Kierdelewicz Date: Wed, 23 Aug 2006 07:51:41 +0000 Subject: Re: [LARTC] New hardware Message-Id: <20060823095141.4eedcc80@localhost.localdomain> List-Id: References: <1156175178.44e9d54a18bd1@www.inbox.lv> In-Reply-To: <1156175178.44e9d54a18bd1@www.inbox.lv> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable To: lartc@vger.kernel.org Hi there, > I used a SunFire x2100 which has 2 Ghz Opteron and PCI-Xpress slot > into which I plugged in a dual port gigE Intel PCI-Xpress card. What was the software configuration of this host? What kernel have you used for the test (compiled for x86 or amd64; dual-core aware sheduler or not). Out of the curiosity ... dual port nic generates one interrupt or two per port? I asume the first is more probable. This leads to another question - has interrupt been balanced between cores?=20 > Got a raw forwarding performance of 800Kpps for 64B packets. This > beat a CISCO 3845 which gave 600Kpps. Thanks for the input :). Right now my production border router (BGP+some firewall rules+some QoS, vlans @ P4 3GHz Linux 2.6, 2xBroadcom PCI-X, 1xIntel E1000 PCI-X) is 91% saturated (CPU) at peek hours and during the time it forwards 344,8Kpps with avg. packet sized 69 bytes. I'll try to determine raw forwarding performance when box goes out of production env, so we'll have a clear comparison picture. regards, --=20 Marek Kierdelewicz Kierownik Dzia?u System=F3w Sieciowych, KoBa Manager of Network Systems Department, KoBa tel. (85) 7406466; fax. (85) 7406467 e-mail: admin@koba.pl _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mohan Sundaram Date: Wed, 23 Aug 2006 08:17:53 +0000 Subject: Re: [LARTC] New hardware Message-Id: <44EC0C61.7010505@vsnl.com> List-Id: References: <1156175178.44e9d54a18bd1@www.inbox.lv> In-Reply-To: <1156175178.44e9d54a18bd1@www.inbox.lv> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: lartc@vger.kernel.org Marek Kierdelewicz wrote: > Hi there, > >> I used a SunFire x2100 which has 2 Ghz Opteron and PCI-Xpress slot >> into which I plugged in a dual port gigE Intel PCI-Xpress card. > > What was the software configuration of this host? What kernel > have you used for the test (compiled for x86 or amd64; dual-core > aware sheduler or not). > stock x86 32 bit kernel 2.4.20. No dual core optimisations. > Out of the curiosity ... dual port nic generates one interrupt or two > per port? I asume the first is more probable. This leads to another > question - has interrupt been balanced between cores? I'm unaware of how it was configured. > >> Got a raw forwarding performance of 800Kpps for 64B packets. This >> beat a CISCO 3845 which gave 600Kpps. > > Thanks for the input :). > > Right now my production border router (BGP+some firewall rules+some QoS, > vlans @ P4 3GHz Linux 2.6, 2xBroadcom PCI-X, 1xIntel E1000 PCI-X) is 91% > saturated (CPU) at peek hours and during the time it forwards 344,8Kpps > with avg. packet sized 69 bytes. > The Opteron was 100% utilised at 800Kpps for 64B pkts. I could do bi-directional line rate on gigE for 512/1500B pkts (approx 500/164Kpps) with 65%/25% cpu utilisation > I'll try to determine raw forwarding performance when box goes out > of production env, so we'll have a clear comparison picture. > > regards, _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mohan Sundaram Date: Wed, 23 Aug 2006 08:21:19 +0000 Subject: Re: [LARTC] New hardware Message-Id: <44EC0D2F.7050306@vsnl.com> List-Id: References: <1156175178.44e9d54a18bd1@www.inbox.lv> In-Reply-To: <1156175178.44e9d54a18bd1@www.inbox.lv> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: lartc@vger.kernel.org Marek Kierdelewicz wrote: > Hi there, > >> I used a SunFire x2100 which has 2 Ghz Opteron and PCI-Xpress slot >> into which I plugged in a dual port gigE Intel PCI-Xpress card. > > What was the software configuration of this host? What kernel > have you used for the test (compiled for x86 or amd64; dual-core > aware sheduler or not). > > Out of the curiosity ... dual port nic generates one interrupt or two > per port? I asume the first is more probable. This leads to another > question - has interrupt been balanced between cores? > >> Got a raw forwarding performance of 800Kpps for 64B packets. This >> beat a CISCO 3845 which gave 600Kpps. > > Thanks for the input :). > > Right now my production border router (BGP+some firewall rules+some QoS, > vlans @ P4 3GHz Linux 2.6, 2xBroadcom PCI-X, 1xIntel E1000 PCI-X) is 91% > saturated (CPU) at peek hours and during the time it forwards 344,8Kpps > with avg. packet sized 69 bytes. > > I'll try to determine raw forwarding performance when box goes out > of production env, so we'll have a clear comparison picture. > > regards, The beauty was the price. $725 for the SUN hardware and $200 for the NIC card. The CISCO 3845 would cost 9K street minimum. Mohan _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc