NAT performance

NAT performance

[Christophe SUIRE]

Hi,

I have done some tests, and i'm surprise with the poor result with NAT.
I have a linux firewall, 2.6.8 kernel, one card for public network,  
and one card for the lan.
All cards are giga bit cards.
I have 10 PC which are each in a VLAN, and with a gateway which is  
the virtual VLAN interface under the firewall link with the lan card.
I have 5 switch with a 100Mbit/s uplink to the firewall (with a giga  
bit backbone switch). I have 2PC under each switch. So in theory each  
PC have 50Mbit/s of bandwidth.
Each PC have 10 alias ip, so i have 10 networks with 10 virtuals  
clients under each network.
So each virtual client (100) have 5Mbit/s of bandwidth.
On the firewall each vlan network is SNAT to go out to the internet.
My bandwidth test is done with TPTEST, and a TPTEST server under the  
public network of the firewall.
My procedure is : tcp-receive of 50Mo
     launch the test for 1 virtual client and get the time
     launch the test for 2 virtuals clients together and get the time  
for each
     ....
     launch the test for 100 virtuals ...

When i do my test without NAT, just routing, the total bandwidth used  
is near to 500Mbit/s, which is great !
But when i do my test with NAT, the total bandwidth used is near to  
170Mbit/s !!! So i have an import drop of the performance !
And this bandwidth is the same from 20 virtuals clients to 100  
virtuals clients.
So i understand that NAT need to rewrite all packets .. but here the  
performance is very poor.
If someone can explain me why ??

Thanks a lot
Christophe SUIRE

Re: NAT performance

[Taylor, Grant]

Christophe SUIRE wrote:
> Hi,
> 
> I have done some tests, and i'm surprise with the poor result with NAT.
> I have a linux firewall, 2.6.8 kernel, one card for public network,  and 
> one card for the lan.
> All cards are giga bit cards.
> I have 10 PC which are each in a VLAN, and with a gateway which is  the 
> virtual VLAN interface under the firewall link with the lan card.
> I have 5 switch with a 100Mbit/s uplink to the firewall (with a giga  
> bit backbone switch). I have 2PC under each switch. So in theory each  
> PC have 50Mbit/s of bandwidth.
> Each PC have 10 alias ip, so i have 10 networks with 10 virtuals  
> clients under each network.
> So each virtual client (100) have 5Mbit/s of bandwidth.
> On the firewall each vlan network is SNAT to go out to the internet.
> My bandwidth test is done with TPTEST, and a TPTEST server under the  
> public network of the firewall.
> My procedure is : tcp-receive of 50Mo
>     launch the test for 1 virtual client and get the time
>     launch the test for 2 virtuals clients together and get the time  
> for each
>     ....
>     launch the test for 100 virtuals ...
> 
> When i do my test without NAT, just routing, the total bandwidth used  
> is near to 500Mbit/s, which is great !
> But when i do my test with NAT, the total bandwidth used is near to  
> 170Mbit/s !!! So i have an import drop of the performance !
> And this bandwidth is the same from 20 virtuals clients to 100  virtuals 
> clients.
> So i understand that NAT need to rewrite all packets .. but here the  
> performance is very poor.
> If someone can explain me why ??

What are the specs on the system you are using as the firewall?



Grant. . . .

Re: NAT performance

[Christophe SUIRE]

Hi,

The hardware is :
     Xeon 3,4Ghz
     1Go of RAM
     Intel Giga cards for public an private network (e1000 driver)
     2 SCSI U320 Raid 1 hard drives

The system is :
     Debian Sarge
     Kernel 2.6.8
     iptables 1.2.11

Thanks a lot.
--  
Christophe

Le 13 mai 05 à 02:03, Taylor, Grant a écrit :

> Christophe SUIRE wrote:
>
>> Hi,
>> I have done some tests, and i'm surprise with the poor result with  
>> NAT.
>> I have a linux firewall, 2.6.8 kernel, one card for public  
>> network,  and one card for the lan.
>> All cards are giga bit cards.
>> I have 10 PC which are each in a VLAN, and with a gateway which  
>> is  the virtual VLAN interface under the firewall link with the  
>> lan card.
>> I have 5 switch with a 100Mbit/s uplink to the firewall (with a  
>> giga  bit backbone switch). I have 2PC under each switch. So in  
>> theory each  PC have 50Mbit/s of bandwidth.
>> Each PC have 10 alias ip, so i have 10 networks with 10 virtuals   
>> clients under each network.
>> So each virtual client (100) have 5Mbit/s of bandwidth.
>> On the firewall each vlan network is SNAT to go out to the internet.
>> My bandwidth test is done with TPTEST, and a TPTEST server under  
>> the  public network of the firewall.
>> My procedure is : tcp-receive of 50Mo
>>     launch the test for 1 virtual client and get the time
>>     launch the test for 2 virtuals clients together and get the  
>> time  for each
>>     ....
>>     launch the test for 100 virtuals ...
>> When i do my test without NAT, just routing, the total bandwidth  
>> used  is near to 500Mbit/s, which is great !
>> But when i do my test with NAT, the total bandwidth used is near  
>> to  170Mbit/s !!! So i have an import drop of the performance !
>> And this bandwidth is the same from 20 virtuals clients to 100   
>> virtuals clients.
>> So i understand that NAT need to rewrite all packets .. but here  
>> the  performance is very poor.
>> If someone can explain me why ??
>>
>
> What are the specs on the system you are using as the firewall?
>
>
>
> Grant. . . .
>
>
>

Re: NAT performance

[Christophe SUIRE]

Hi,

This not a problem with the network card, because when i do the test  
with only firewall routing i have a total bandwidth used near to  
500Mbit/s.
But when i add an SNAT translation for each network (10) the total  
bandwidth used is near 170Mbit/s.
So why this important difference without an with NAT ??

Thx.
-- 
Christophe Suire


Le 13 mai 05 à 10:24, R. DuFresne a écrit :

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Fri, 13 May 2005, Christophe SUIRE wrote:
>
>
>> Hi,
>>
>> The hardware is :
>>    Xeon 3,4Ghz
>>    1Go of RAM
>>    Intel Giga cards for public an private network (e1000 driver)
>>    2 SCSI U320 Raid 1 hard drives
>>
>> The system is :
>>    Debian Sarge
>>    Kernel 2.6.8
>>    iptables 1.2.11
>>
>> Thanks a lot.
>> -- Christophe
>>
>> Le 13 mai 05 à 02:03, Taylor, Grant a écrit :
>>
>>
>>> Christophe SUIRE wrote:
>>>
>>>> Hi,
>>>> I have done some tests, and i'm surprise with the poor result  
>>>> with NAT.
>>>> I have a linux firewall, 2.6.8 kernel, one card for public  
>>>> network,  and one card for the lan.
>>>> All cards are giga bit cards.
>>>> I have 10 PC which are each in a VLAN, and with a gateway which  
>>>> is  the virtual VLAN interface under the firewall link with the  
>>>> lan card.
>>>> I have 5 switch with a 100Mbit/s uplink to the firewall (with a  
>>>> giga bit backbone switch). I have 2PC under each switch. So in  
>>>> theory each PC have 50Mbit/s of bandwidth.
>>>> Each PC have 10 alias ip, so i have 10 networks with 10 virtuals  
>>>> clients under each network.
>>>> So each virtual client (100) have 5Mbit/s of bandwidth.
>>>> On the firewall each vlan network is SNAT to go out to the  
>>>> internet.
>>>> My bandwidth test is done with TPTEST, and a TPTEST server under  
>>>> the public network of the firewall.
>>>> My procedure is : tcp-receive of 50Mo
>>>>     launch the test for 1 virtual client and get the time
>>>>     launch the test for 2 virtuals clients together and get the  
>>>> time for each
>>>>     ....
>>>>     launch the test for 100 virtuals ...
>>>> When i do my test without NAT, just routing, the total bandwidth  
>>>> used is near to 500Mbit/s, which is great !
>>>> But when i do my test with NAT, the total bandwidth used is near  
>>>> to 170Mbit/s !!! So i have an import drop of the performance !
>>>> And this bandwidth is the same from 20 virtuals clients to 100   
>>>> virtuals clients.
>>>> So i understand that NAT need to rewrite all packets .. but here  
>>>> the performance is very poor.
>>>> If someone can explain me why ??
>>>>
>
>
> If you are doing this all at once, you are likely saturating the  
> choke points <firewalls> network interface.  All traffic is passing  
> there, unless of course you have multi interfaces.
>
>
> Thanks,
>
>
> Ron DuFresne
> - -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>         admin & senior security consultant:  sysinfo.com
>                         http://sysinfo.com
> Key fingerprint = 9401 4B13 B918 164C 647A  E838 B2DF AFCC 94B0 6629
>
> ...We waste time looking for the perfect lover
> instead of creating the perfect love.
>
>                 -Tom Robbins <Still Life With Woodpecker>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
>
> iD8DBQFChGRUst+vzJSwZikRAuN4AKCKNvvsHiK5f6bH7i5R47n7Ha2KRACfae+y
> xBwrHTBFFkRA+uEM1wTHkXA=
> =+THk
> -----END PGP SIGNATURE-----

Re: NAT performance

[Jozsef Kadlecsik]

On Fri, 13 May 2005, Christophe SUIRE wrote:

> This not a problem with the network card, because when i do the test
> with only firewall routing i have a total bandwidth used near to
> 500Mbit/s.
> But when i add an SNAT translation for each network (10) the total
> bandwidth used is near 170Mbit/s.
> So why this important difference without an with NAT ??

NAT *is* expensive. Have a look at the paper on netfilter performance
tests and comparisons at http://people.netfilter.org/kadlec/nftest.pdf

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: NAT performance

[Christophe SUIRE]

Hi,

A great paper !
But i have done some bandwidth tests which is quiet different ..
I know that NAT *is* expensive .. but i'm surprise that there no  
bandwidth difference between 20 client and 100 client.
So i would like to understand why this limit ?
Thx

-- 
Christophe Suire


Le 13 mai 05 à 11:22, Jozsef Kadlecsik a écrit :

> On Fri, 13 May 2005, Christophe SUIRE wrote:
>
>
>> This not a problem with the network card, because when i do the test
>> with only firewall routing i have a total bandwidth used near to
>> 500Mbit/s.
>> But when i add an SNAT translation for each network (10) the total
>> bandwidth used is near 170Mbit/s.
>> So why this important difference without an with NAT ??
>>
>
> NAT *is* expensive. Have a look at the paper on netfilter performance
> tests and comparisons at http://people.netfilter.org/kadlec/nftest.pdf
>
> Best regards,
> Jozsef
> -
> E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
> PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
> Address : KFKI Research Institute for Particle and Nuclear Physics
>           H-1525 Budapest 114, POB. 49, Hungary
>
>

Re: NAT performance

[Jozsef Kadlecsik]

On Fri, 13 May 2005, Christophe SUIRE wrote:

> A great paper !
> But i have done some bandwidth tests which is quiet different ..
> I know that NAT *is* expensive .. but i'm surprise that there no
> bandwidth difference between 20 client and 100 client.
> So i would like to understand why this limit ?

Sorry, but I have to ask: don't you hit the limit of ip_conntrack_max?

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: NAT performance

[Jozsef Kadlecsik]

On Fri, 13 May 2005, Jozsef Kadlecsik wrote:

> On Fri, 13 May 2005, Christophe SUIRE wrote:
>
> > A great paper !
> > But i have done some bandwidth tests which is quiet different ..
> > I know that NAT *is* expensive .. but i'm surprise that there no
> > bandwidth difference between 20 client and 100 client.
> > So i would like to understand why this limit ?
>
> Sorry, but I have to ask: don't you hit the limit of ip_conntrack_max?

Probably not. But as far as I see, you cannot specify the source IP
address at tptest. Are you sure the IP address aliases were used at all?

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: NAT performance

[Christophe SUIRE]

No i don't change the value : 65528
But my test is to receive 50Mo of data, so the conntrack is  
established only one time for each client .. so i should have only  
100 conntracking entries

Thx.
-- 
Christophe Suire

Le 13 mai 05 à 11:57, Jozsef Kadlecsik a écrit :

> On Fri, 13 May 2005, Christophe SUIRE wrote:
>
>
>> A great paper !
>> But i have done some bandwidth tests which is quiet different ..
>> I know that NAT *is* expensive .. but i'm surprise that there no
>> bandwidth difference between 20 client and 100 client.
>> So i would like to understand why this limit ?
>>
>
> Sorry, but I have to ask: don't you hit the limit of ip_conntrack_max?
>
> Best regards,
> Jozsef
> -
> E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
> PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
> Address : KFKI Research Institute for Particle and Nuclear Physics
>           H-1525 Budapest 114, POB. 49, Hungary
>
>

Re: NAT performance

[Christophe SUIRE]

You can bind on ip address so ip alias are used ..

   -b <local address>        Bind to local address/interface

--  
Christophe Suire


Le 13 mai 05 à 12:14, Jozsef Kadlecsik a écrit :

> On Fri, 13 May 2005, Jozsef Kadlecsik wrote:
>
>
>> On Fri, 13 May 2005, Christophe SUIRE wrote:
>>
>>
>>> A great paper !
>>> But i have done some bandwidth tests which is quiet different ..
>>> I know that NAT *is* expensive .. but i'm surprise that there no
>>> bandwidth difference between 20 client and 100 client.
>>> So i would like to understand why this limit ?
>>>
>>
>> Sorry, but I have to ask: don't you hit the limit of  
>> ip_conntrack_max?
>>
>
> Probably not. But as far as I see, you cannot specify the source IP
> address at tptest. Are you sure the IP address aliases were used at  
> all?
>
> Best regards,
> Jozsef
> -
> E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
> PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
> Address : KFKI Research Institute for Particle and Nuclear Physics
>           H-1525 Budapest 114, POB. 49, Hungary
>
>

Re: NAT performance

[Feizhou]

Jozsef Kadlecsik wrote:
> On Fri, 13 May 2005, Christophe SUIRE wrote:
> 
> 
>>This not a problem with the network card, because when i do the test
>>with only firewall routing i have a total bandwidth used near to
>>500Mbit/s.
>>But when i add an SNAT translation for each network (10) the total
>>bandwidth used is near 170Mbit/s.
>>So why this important difference without an with NAT ??
> 
> 
> NAT *is* expensive. Have a look at the paper on netfilter performance
> tests and comparisons at http://people.netfilter.org/kadlec/nftest.pdf

the ip_conntrack module sucks. Lovely cpu chewer.