Performance of iptables-restore on large rule sets

Performance of iptables-restore on large rule sets

[Steve Bergman]

I have a large rule set (~53000 rules) that I sometimes load using 
iptables-restore.  (It takes almost an hour.

Googling around tells me that the loop detection code in the kernel is 
slow with large rule sets.  The only thing  that seems odd to me is that 
throughout the entire loading process, iptables-restore is consistently 
at about 67% user and33% system processor time according to vmstat.  If 
the slowness is in the kernel, shouldn't I be seeing a high and ever 
increasing amount of "system" time?

Kernel is 2.6.9-1.681_FC3.  Iptables is iptables-1.2.11-3.1.FC3.

Thanks for any insights,
Steve Bergman

Re: Performance of iptables-restore on large rule sets

[Martin Josefsson]

[-- Attachment #1: Type: text/plain, Size: 1314 bytes --]

On Fri, 2005-01-28 at 12:56 -0600, Steve Bergman wrote:
> I have a large rule set (~53000 rules) that I sometimes load using 
> iptables-restore.  (It takes almost an hour.
> 
> Googling around tells me that the loop detection code in the kernel is 
> slow with large rule sets.  The only thing  that seems odd to me is that 
> throughout the entire loading process, iptables-restore is consistently 
> at about 67% user and33% system processor time according to vmstat.  If 
> the slowness is in the kernel, shouldn't I be seeing a high and ever 
> increasing amount of "system" time?

The loop checking takes place in userspace.

> Kernel is 2.6.9-1.681_FC3.  Iptables is iptables-1.2.11-3.1.FC3.

Please try what is going to be released as iptables 1.3.0
You can get the latest snapshot here:
ftp://ftp.netfilter.org/pub/iptables/snapshot/iptables-1.3.0-20050127.tar.bz2

Read the file called INSTALL to see how to compile and install it. (and
make sure you are actually using the new version after it's installed,
either by using the absolute patch, /usr/local/sbin/iptables or by
uninstalling the iptables rpm)

It contains a rewrite of libiptc which is the library that performs the
ruleset modifications, it's much faster now.

I hope it improves your situation.

-- 
/Martin

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Performance of iptables-restore on large rule sets

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 1094 bytes --]

On Fri, Jan 28, 2005 at 12:56:30PM -0600, Steve Bergman wrote:
> I have a large rule set (~53000 rules) that I sometimes load using 
> iptables-restore.  (It takes almost an hour.

That's really slow.  I've seen multiple minutes, but an hour?  What kind
of system is this?  How does the ruleset look like?  Maybe some dns
resolvals are timing out?

> Googling around tells me that the loop detection code in the kernel is 
> slow with large rule sets.  

That's wrong.  What used to be slow is libiptc.  iptables-1.2.11 should
actually already be significantly faster than all prior versions.

Please try the current pre-1.3.0 snapshots from
ftp://ftp.netfilter.org/pub/iptables/snapshot

Please report back if they solve your performance issue.

> Steve Bergman
-- 
- Harald Welte <laforge@gnumonks.org>               http://www.gnumonks.org/
============================================================================
"Privacy in residential applications is a desirable marketing option."
                                                  (ETSI EN 300 175-7 Ch. A6)

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]