Re: IP Accounting and performance

[Joel Newkirk <netfilter@newkirk.us>]

On Sunday 08 December 2002 04:18 pm, Gerald Galster wrote:
> Hi all,
>
> Perhaps you can give me some hints on a performance problem that I'm
> currently experiencing with iptables.
>
> The situation is as follows:
>
> I have a firewall currently running kernel 2.4.20, Celeron 1 GHz and
> 512 MB of RAM
> that should do traffic accounting based on single IP addresses. I
> thought it would be more
> efficient to use iptables than writing a standalone application using
> pcap or the like.

You will probably get your best speed (and usability of the resulting 
data) if you just use one of the accounting daemons that can be paired 
with the ULOG target, and use the two rules below.  Google/linux search 
on "Netfilter ULOG accounting"  
http://www.google.com/linux?num=100&restrict=linux&hl=en&lr=&ie=ISO-8859-1&q=netfilter+ULOG+accounting&btnG=Google+Search  
turns up some useful links...  That said, if you still want to do it 
directly in your ruleset, read on! :^)

> I need to add filtering rules like
>
> /sbin/iptables -A FORWARD -o eth0 -s ip_address/32
> /sbin/iptables -A FORWARD -i eth0 -d ip_address/32

Is this the complete rule?  If you don't have a target it will count and 
pass to the next rule.  Since all you said you are doing is counting 
that may seem OK, but it means that each packet is tested against each 
and every rule...  

If you can use a -j ACCEPT target for them it will count, and prevent 
testing against following rules.  This will seriously speed up testing 
for IP's that appear early in the list, but won't help later ones at 
all.  Best is to break the address range down into smaller chunks.

Something expanded from the following might work well:

/sbin/iptables -N ip01to31in
/sbin/iptables -N ip32to63out
/sbin/iptables -A ip01to31in -d 10.11.12.13 -j RETURN
/sbin/iptables -A FORWARD -i eth0 -d 10.11.12.0/27 -j ip01to31in
/sbin/iptables -A ip32to63out -s 10.11.12.33 -j RETURN
/sbin/iptables -A FORWARD -o eth0 -s 10.11.12.32/27 -j ip32to63out

This creates a user-defined chain (two in this sample fragment) to test a 
range of addresses, which can be broken down in whatever fashion gives 
you either an even distribution of rules among the userdef chains, or 
simply two chains and two FORWARD rules for each subnet.  (Obviously the 
breakdown must occur in some fashion that is easy to test, IE based on 
masking the IPs)  The latter would allow you to use the FORWARD chain 
counts to see overall numbers for each subnet, but if one subnet is 5 
machines and another is 1000 this will obviously be seriously 
unbalanced.  You can also further break down from a user defined chain 
to several more sub-chains using the same technique if this is useful.

> for about six class-C networks (this means about 3000 iptables rules).

Your end result can be twelve user-defined chains, one in/one out for 
each subnet, and twelve rules in FORWARD.

Using the RETURN target in the user-defined chains simply passes the 
packet back to the originating chain (IE, FORWARD) and skips testing 
against the remaining rules in the chain.  You can substitute ACCEPT (or 
DROP or REJECT) with no problem here as well.  RETURN shouldn't incur 
much more processing time, since the packet should only match and be 
passed to ONE userdef chain, and the main FORWARD chain should be fairly 
short this way, so it will only test against a few more rules after it 
RETURNS to FORWARD.  Using RETURN in a sub-chain will also allow you to 
perform 'real' FORWARD filtering after the counting has taken place.

Also, as someone else suggested, you can have a script that creates the 
rules in a loop, or perhaps six (one per subnet).  Fanciest and most 
efficient traversal-wise would be a balanced tree of rules, (which could 
bring you down to a max of 2-3 dozen tests for each packet, instead of 
thousands right now) the construction of which is beyond the scope of 
this reply... :^P

Whatever you do, if you handle this with iptables rules make sure you use 
the most restrictive rules possible for your situation at the upper 
levels (IE in FORWARD itself) to prevent traffic being tested that 
doesn't need to be.  If all you're interested in is HTTP to the 
internet, for example, test that in the FORWARD rules before reaching 
the subchains.

j

> The average throughput is around 3 Mbits / second.
>
> After I've added those rules, the latency in ping times to a machine
> behind
> the firewall increases from 30 ms to over 200 ms ...
>
> Now my question is if I can speed those things up ... do you have any
> ideas?
>
> Thanks in advance.
>
> Regards,
> Gerald