Re: Using iptables with high volume mail

[John Little <jlittle_97@yahoo.com>]

----- Original Message ----
> From: Thomas Jacob <jacob@internet24.de>
> To: John Little <jlittle_97@yahoo.com>
> Cc: Gáspár Lajos <swifty@freemail.hu>; netfilter@vger.kernel.org
> Sent: Friday, October 2, 2009 8:31:16 AM
> Subject: Re: Using iptables with high volume mail
> 
> > @thomas Thanks for those metrics.  We are looking to see if the connections 
> per second is
> > generated with our current devices.  What we do know is that our max
> >outbound connections will get as high as 16000 for a period of time >
> >(maybe 2-4 hours) and will occasionally burst up to around 20000.
> 
> I am guessing that means existing parallel connections, not new
> connections per second (cps), the kind of server box I was referring
> to can easily sustain millions of those, given enough
> memory for the tables (The last number I remember was <300byte per
> connection in the conntrack table + space for entries into the routing
> cache for each different IP). Slabtop is your friend here.
> 
> What matters most is
> what happens in each time slice, not so much how many connections
> you have in the connection hash table (you can tune that table with
> with /proc/sys/net/ipv4/netfilter/ip_conntrack_max
> and /sys/module/ip_conntrack/parameters/hashsize).
> 
> > How does that compare to the metrics that you mentioned earlier?
> 
> Well, any Switch/Router with SNMP support allows you to track bytes and
> packets per second, so you could collect some data on the current
> situation with that (www.cacti.net is a very nice tool).
> 
> As for new connections per second, once you have the iptables box
> running you can get this info with lnstat -f ip_conntrack/column new.
> 
> If you have a reasonably good switch/router in the datapath, you could
> also use port mirroring to get a copy of the data stream and then
> count all tcp/syn packets to port 25 to give you a rough idea
> about the number of connections per second.
> 
> However, emails per time should be pretty much the same as connections
> per time, unless you open several tcp connections over the nat box
> for each email, and I see no reason why you would need to do that ;)


Ok thanks.

We have some stats now:

Packets per second:  avg 6221 max 41,810
 
Connections peak: avg 7263  max 22,981
 
New connections per second: avg 102 max 1029 

Given your numbers of 8000 cps and the above comments it would seem that we are well within any types of overload issues with any decent off the shelf server equipped with two dual core CPUs and the necessary memory.  If I allocate 500 bytes per connection at the max connections I would need ~87Mb + machine overhead.  That's not much in today's world of servers.

Am I looking at this properly?

Thanks,
John