largescale NAT

largescale NAT

[DEXTER]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Okay. As nobody answered my question - except one in private - i'll try
to rephrase it.

So.. there is a dormitory about 400+ students in it. They are always
like to play games on net, and when one starts something like GameSpy it
connects to a whole bunch of servers on all kinds of ports, and it is
just ONE user, and not to mention other p2p programs like DC, Emule,
Bittorrent, etc. Than you can imagine what happens when 100 or more
users want to play online at the same time -> it eats up lots of port on
the NAT box (all of the ports).

and this is where the problem lies.

The documenation on NAT is great
(http://www.netfilter.org/documentation/HOWTO//NAT-HOWTO.html) but it is
good when you have only few machines, or lots of machines but not with
gamers, downloaders....
Why isn't there any documentation on how linux handles free, and
occupied ports on a NAT box, how to fine tune the box when lots of user
are behind it, etc..?

So my question is: How to handle this situation? I mean, just 1 NAT
linux box is able to handle all the 400+ users with the above
situations? Or we have to do something like CONNLIMIT on source ips? Is
there possibilities to balance the traffic on 2 or more NAT box?

thx.

- --
You can find my public PGP key here:
Tu peux trouver mon public PGP clef ici:
A nyilvanos PGP kulcsomat innen tudod letolteni:

http://koli.kando.hu/dexter/publickey.asc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xBC788404

.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDsWpXd/8YWbx4hAQRAjxgAJ9Zw/hBW2NBYU2D4HXhKHfWMDvOAgCgocEs
CDrM5sU1HYUeBmrNgyhNHGQ=
=9w4i
-----END PGP SIGNATURE-----

Re: largescale NAT

[KOVACS Krisztian]

  Hi,

On Tuesday 27 December 2005 17.22, DEXTER wrote:
> Why isn't there any documentation on how linux handles free, and
> occupied ports on a NAT box, how to fine tune the box when lots of user
> are behind it, etc..?
>
> So my question is: How to handle this situation? I mean, just 1 NAT
> linux box is able to handle all the 400+ users with the above
> situations? Or we have to do something like CONNLIMIT on source ips? Is
> there possibilities to balance the traffic on 2 or more NAT box?

  Whether or not your NAT box will be able to handle the situation mainly 
depends on your traffic pattern and available uplink bandwidth. For 400+ 
users the number of available ports on a single IP seems to be a serious 
limitation, so first of all you'd probably have to get multiple routable IP 
addresses and share connections between those multiple public IPs.

  The algorithm Netfilter uses to balance connections between multiple 
public IPs depends on the kernel version, so it's hard to say more about 
this without this crucial piece of information.

-- 
KOVACS Krisztian

Re: largescale NAT

[DEXTER]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thank you for your answer, but what if we have only one public(external)
ip? :)

Alex Sirbu wrote:
> Hi,
>  
> I have a situation similar with yours : I have almost 1024 computers
> connected to Internet through a single ( but powerful ) NAT box. To
> solve the problem, I use multiple IP addresses for NAT . I have a rule
> that sound like this ( it's just an example, not the real world ) :
>  
> iptables -t nat -A POSTROUTING  -s 192.160.100.0/22
> <http://192.160.100.0/22> -o eth0 -j SNAT --to-source 10.0.0.1-10.0.0.63
>  
> Obviously, you should replace 192.160.100.0/22
> <http://192.160.100.0/22>  with your internal IP address pool  and
> 10.0.0.1 <http://10.0.0.1> with your external IP. Note that the more
> external IP-s you use, the less chances to have port conflict on the NAT
> box.
>  
> Normally, on every IP you can have a fixed number of ports usable at the
> same time. If you just multiply the number of the IP addresses used for
> NAT , you will have more available ports, so more resources for new
> connections. But be aware that you will need a huge amount of memory to
> handle the connection tracking.  My NAT box have 2GB bytes of RAM , and
> it use more than 1.2 GB for NAT , on an average of 25.000-30.000
> simultaneous connections, using 65% of a 100FDX Internet connection ,
> running QoS too ( IMQ and HTB ).
>  
>  
>  
> 
> 
>  
> On 12/27/05, *DEXTER* <dexter@fiktiv.szgtikol.kando.hu
> <mailto:dexter@fiktiv.szgtikol.kando.hu>> wrote:
> 
>     -----BEGIN PGP SIGNED MESSAGE-----
>     Hash: SHA1
> 
>     Okay. As nobody answered my question - except one in private - i'll try
>     to rephrase it.
> 
>     So.. there is a dormitory about 400+ students in it. They are always
>     like to play games on net, and when one starts something like GameSpy it
>     connects to a whole bunch of servers on all kinds of ports, and it is
>     just ONE user, and not to mention other p2p programs like DC, Emule,
>     Bittorrent, etc. Than you can imagine what happens when 100 or more
>     users want to play online at the same time -> it eats up lots of
>     port on
>     the NAT box (all of the ports).
> 
>     and this is where the problem lies.
> 
>     The documenation on NAT is great
>     (http://www.netfilter.org/documentation/HOWTO//NAT-HOWTO.html
>     <http://www.netfilter.org/documentation/HOWTO//NAT-HOWTO.html>) but
>     it is
>     good when you have only few machines, or lots of machines but not with
>     gamers, downloaders....
>     Why isn't there any documentation on how linux handles free, and
>     occupied ports on a NAT box, how to fine tune the box when lots of user
>     are behind it, etc..?
> 
>     So my question is: How to handle this situation? I mean, just 1 NAT
>     linux box is able to handle all the 400+ users with the above
>     situations? Or we have to do something like CONNLIMIT on source ips? Is
>     there possibilities to balance the traffic on 2 or more NAT box?
> 
>     thx.
> 
>     - --
>     You can find my public PGP key here:
>     Tu peux trouver mon public PGP clef ici:
>     A nyilvanos PGP kulcsomat innen tudod letolteni:
> 
>     http://koli.kando.hu/dexter/publickey.asc
>     http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xBC788404
>     <http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xBC788404>
> 
> 
> 
> 
> -- 
> 
> Alex Sirbu

- --
You can find my public PGP key here:
Tu peux trouver mon public PGP clef ici:
A nyilvanos PGP kulcsomat innen tudod letolteni:

http://koli.kando.hu/dexter/publickey.asc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xBC788404

.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDsX7Ud/8YWbx4hAQRAoHlAJ4xIe+TD/dtyawFYVMkX25emzUg4QCgmgjC
09UvtS5lYpMWnLDDA1aH6iA=
=DALM
-----END PGP SIGNATURE-----

RE: largescale NAT

[Rob Sterenborg]

> Thank you for your answer, but what if we have only one
> public(external) ip? :)

Isn't it possible to limit the number of connections per (source) IP ?
http://www.netfilter.org/projects/patch-o-matic/pom-base.html#pom-base-c
onnlimit

Maybe not so nice for your users but things have to keep working, don't
they ?


Gr,
Rob

Re: largescale NAT

[DEXTER]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

It is possible, but i've already mentioned that:
> "Or we have to do something like CONNLIMIT on source ips?"

but thx.

Rob Sterenborg wrote:
>>Thank you for your answer, but what if we have only one
>>public(external) ip? :)
> 
> 
> Isn't it possible to limit the number of connections per (source) IP ?
> http://www.netfilter.org/projects/patch-o-matic/pom-base.html#pom-base-c
> onnlimit
> 
> Maybe not so nice for your users but things have to keep working, don't
> they ?
> 
> 
> Gr,
> Rob
> 
> 
> 

- --
You can find my public PGP key here:
Tu peux trouver mon public PGP clef ici:
A nyilvanos PGP kulcsomat innen tudod letolteni:

http://koli.kando.hu/dexter/publickey.asc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xBC788404

.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDsY56d/8YWbx4hAQRAoFoAKCR3D8euC1QeA2KGcwpAFxrfa8qPQCeKl4R
t/w231lpsmZJO5RHb/HYjT8=
=9Q/t
-----END PGP SIGNATURE-----

Re: largescale NAT

[Ishwar Rattan]

On Tue, 27 Dec 2005, DEXTER wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Thank you for your answer, but what if we have only one public(external)
> ip? :)

Either go for conenction limit or get more public IPs. There are
only so many ports on NAT box and it can process packets only
so fast :-(

-ishwar

Re: largescale NAT

[René Pfeiffer]

[-- Attachment #1: Type: text/plain, Size: 915 bytes --]

On Dec 27, 2005 at 1722 +0100, DEXTER appeared and said:
> [...]
> So my question is: How to handle this situation? I mean, just 1 NAT
> linux box is able to handle all the 400+ users with the above
> situations? Or we have to do something like CONNLIMIT on source ips? Is
> there possibilities to balance the traffic on 2 or more NAT box?

Just a wild guess, I would try to look at the timeouts in
/proc/sys/net/ipv4/netfilter/ and see if you can reduce some of them. In
addition to that I would increase the IP pool for NAT on the outgoing
interface(s). You don't need two boxes for that, just multiple IP
addresses on the interface.

Best wishes,
René.

-- 
  )\._.,--....,'``.      Let GNU/Linux work for you while you take a nap.
 /,   _.. \   _\  (`._ ,. R. Pfeiffer <lynx at luchs.at> + http://web.luchs.at/
`._.-(,_..'--(,_..'`-.;.'  - System administration + Consulting + Teaching -


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]