How many rules were supported iptables?

How many rules were supported iptables?

[이근수]

Hi All.
I have a problem that is how many rules were supported iptables.
The program which I maintain and repair generated iptables rules automatically.
Now, I encounter this problem.
Source and destination IP address is written range by user.( ex, 1.1.1.1~1.1.1.10 ) then our program generated 10 IP address(1.1.1.1, 1.1.1.2, 1.1.1.3, … , 1.1.1.10) and make 10 iptables rules.
Unfortunately, user wrote iptables rules like this, 10,000 rules is generated.
　“ iptalbes –A FORWARD –p tcp –s 1.1.1.1~1.1.1.100 –d 2.2.2.1~2.2.2.100 –j QUEUE”.
Do 10,000 rules operate safely?? Or Some rules don’t operate normally??
If some rules don’t operate normally, how many rules iptables does support??

Re: How many rules were supported iptables?

[Maik Hentsche]

Zitat von 이근수 <kslee109@gmail.com>:

> If some rules don't operate normally, how many rules iptables does
> support??

According to my own tests on a dual Opteron 2,4 GHz-System with 2 GB
RAM, more than 5000 rules decreased the throughput dramatically to
abround 4 MBit/s an a 100MBit interface, even through, these were empty
rules with no -j target. Having such a target will surely decrease the
throughput even more. Thus is discourage you from using so many rules.

so long
Maik

Re: How many rules were supported iptables?

[scott comer (sccomer)]

[-- Attachment #1: Type: text/plain, Size: 506 bytes --]

in our application, a sccp specific gateway, users outside the firewall 
authenticate using secure web browser, then we open 3 ports for them by 
inserting rules into a subchain of the input chain. in trials we could 
authenticate a max of 10,281 users (30,843 rules). after that the system 
hangs.

we are using kernel  2.6.16-1.2115_FC4 and iptables 1.3.5.

we did not have time to investigate the exact cause of the hang. i have 
no idea how big the ruleset was. how does one find that out?

scott out

How many rules were supported iptables?

[이근수]

Hi All.
I have a problem that is how many rules were supported iptables.
The program which I maintain and repair generated iptables rules automatically.
Now, I encounter this problem.
Source and destination IP address is written range by user.( ex, 1.1.1.1~1.1.1.10 ) then our program generated 10 IP address(1.1.1.1, 1.1.1.2, 1.1.1.3, … , 1.1.1.10) and make 10 iptables rules.
Unfortunately, user wrote iptables rules like this, 10,000 rules is generated.
　“ iptalbes –A FORWARD –p tcp –s 1.1.1.1~1.1.1.100 –d 2.2.2.1~2.2.2.100 –j QUEUE”.
Do 10,000 rules operate safely?? Or Some rules don’t operate normally??
If some rules don’t operate normally, how many rules iptables does support??

Re: How many rules were supported iptables?

[Peter]

이근수 wrote:

>Hi All.
>I have a problem that is how many rules were supported iptables.
>The program which I maintain and repair generated iptables rules automatically.
>Now, I encounter this problem.
>Source and destination IP address is written range by user.( ex, 1.1.1.1~1.1.1.10 ) then our program generated 10 IP address(1.1.1.1, 1.1.1.2, 1.1.1.3, … , 1.1.1.10) and make 10 iptables rules.
>Unfortunately, user wrote iptables rules like this, 10,000 rules is generated.
>　“ iptalbes –A FORWARD –p tcp –s 1.1.1.1~1.1.1.100 –d 2.2.2.1~2.2.2.100 –j QUEUE”.
>Do 10,000 rules operate safely?? Or Some rules don’t operate normally??
>If some rules don’t operate normally, how many rules iptables does support??
>
>
>  
>
It's the 4 MB in total size of rules. (This is the maximum size of a 
chunc of data copy from userspace to kernel space in
one syscall)

Re: How many rules were supported iptables?

[Eric Dumazet]

Peter a écrit :
> 이근수 wrote:
> 
>> Hi All.
>> I have a problem that is how many rules were supported iptables.
>> The program which I maintain and repair generated iptables rules 
>> automatically.
>> Now, I encounter this problem.
>> Source and destination IP address is written range by user.( ex, 
>> 1.1.1.1~1.1.1.10 ) then our program generated 10 IP address(1.1.1.1, 
>> 1.1.1.2, 1.1.1.3, … , 1.1.1.10) and make 10 iptables rules.
>> Unfortunately, user wrote iptables rules like this, 10,000 rules is 
>> generated.
>> 　“ iptalbes –A FORWARD –p tcp –s 1.1.1.1~1.1.1.100 –d 
>> 2.2.2.1~2.2.2.100 –j QUEUE”.
>> Do 10,000 rules operate safely?? Or Some rules don’t operate normally??
>> If some rules don’t operate normally, how many rules iptables does 
>> support??
>>
>>
>>  
>>
> It's the 4 MB in total size of rules. (This is the maximum size of a 
> chunc of data copy from userspace to kernel space in
> one syscall)
Hi Peter

I was not aware on this 4MB limit.
Could you please tell us where this limit is applied in kernel sources ?

Thank you

Re: How many rules were supported iptables?

[Jesper Dangaard Brouer]

On Fri, 20 Oct 2006, Peter wrote:

> ??? <kslee109@gmail.com> wrote:
>
<cut>
>> ...
>> Do 10,000 rules operate safely?? Or Some rules dont operate normally??
>> If some rules dont operate normally, how many rules iptables does 
>> support??
>> 
> It's the 4 MB in total size of rules. (This is the maximum size of a 
> chunc of data copy from userspace to kernel space in one syscall)


It is NOT true, that there exist a 4 MB size limit of rules.

Its quite easy to disprove. I have made some tests where I can transfer a 
ruleset of 13 Mb, between userspace and kernel (howto see ruleset size 
see note#1).

The ruleset consists of 12256 chains and 59239 rules.  Regarding 
performance issues: I have constructed the ruleset as a subnet-tree, where 
a worst-case match result in touching max 45 rules.  Throughput wise with 
this ruleset, I can route 94Mbit/s through the Linux box on a 100Mbit/s 
netcard.

Problem: The execution time of iptables take around 14 seconds.  This is 
caused by the way iptables/libiptc parse the ruleset blob, in libiptc.c 
function iptcc_find_chain_by_offset.  We should actually fix this... I 
have raised this issue before on the list, but the discussion died...

The real limit:

To test the limits of iptables, I also tried to load a ruleset with 24878 
chains and 120250 rules, with a ruleset blob size around 30Mb. (Iptables 
init function now takes 70 seconds!!!). With a ruleset that large iptables 
actually gave up and reported:

  "Memory allocation problem" (errno:12)

And the kernel syslog reported:

  "kernel: allocation failed: out of vmalloc space - use vmalloc=<size> to increase size."

This error, is caused by, that the kernel cannot allocate enough vmalloc 
memory.  Propably just before it swaps the rulesets. (This can be 
increased at boot time vmalloc=xxx and the available vmalloc memory can 
be seen in /proc/meminfo).


Cheers,
   Jesper Brouer

--
-------------------------------------------------------------------
MSc. Master of Computer Science
Dept. of Computer Science, University of Copenhagen
Author of http://www.adsl-optimizer.dk
-------------------------------------------------------------------

Notes:

Note#1:
  The easies way to see the size of the ruleset blob is to compile libiptc
  with -DIPTC_DEBUG2 and look at the filesize of
  /tmp/libiptc-so_get_entries.blob and /tmp/libiptc-so_set_replace.blob).

RE: How many rules were supported iptables?

[Paul C Diem]

On Fri, 20 Oct 2006, Peter wrote:
(snip)
> Problem: The execution time of iptables take around 14 seconds.  This is
> caused by the way iptables/libiptc parse the ruleset blob, in libiptc.c
> function iptcc_find_chain_by_offset.  We should actually fix this... I
> have raised this issue before on the list, but the discussion died...
(snip)

I posted a patch to fix this a few months ago. Unfortunately, I'm not being
familiar with maintaining my mods with cvs, the proper way to submit patches
and the nfsim test suggested by Patrick McHardy. If someone can do that,
here's a diff of the changes I've made to libiptc.c including the
modification to load/use the chain entry offset for use by the parse and the
modification to insert new chains in the proper order and not sort chains on
init:

Paul C. Diem
PCDiem@FoxValley.net

diff -Nurp iptables-1.3.5-org/libiptc/libiptc.c
iptables-1.3.5/libiptc/libiptc.c | more
--- iptables-1.3.5-org/libiptc/libiptc.c        2006-01-30
02:43:09.000000000 -0600
+++ iptables-1.3.5/libiptc/libiptc.c    2006-07-15 15:04:24.000000000 -0500
@@ -307,6 +307,12 @@ static struct rule_head *iptcc_get_rule_
 static struct chain_head *
 iptcc_find_chain_by_offset(TC_HANDLE_T handle, unsigned int offset)
 {
+/*+PCD*/
+       /*
+       * Use the offset of the chain entry in the ipt entry comefrom
+       * field as loaded by iptcc_find_chain_by_offset.
+       */
+#if 0
        struct list_head *pos;

        if (list_empty(&handle->chains))
@@ -317,6 +323,11 @@ iptcc_find_chain_by_offset(TC_HANDLE_T h
                if (offset >= c->head_offset && offset <= c->foot_offset)
                        return c;
        }
+#else
+      STRUCT_ENTRY * e = ((STRUCT_ENTRY *)((char
*)handle->entries->entrytable + offset));
+      return (struct chain_head *)((char *)e + e->comefrom);
+#endif
+/*-PCD*/

        return NULL;
 }
@@ -420,7 +431,13 @@ static void __iptcc_p_add_chain(TC_HANDL
        c->head_offset = offset;
        c->index = *num;

-       iptc_insert_chain(h, c);
+/*+PCD
+* Don't waste cycles finding alphabetical place to insert chains
+* on load. Do that when we're adding a new chain instead.
+*/
+//     iptc_insert_chain(h, c);
+       list_add_tail(&c->list, &h->chains);
+/*-PCD*/

        h->chain_iterator_cur = c;
 }
@@ -494,6 +511,15 @@ new_rule:
                r->index = *num;
                r->offset = offset;
                memcpy(r->entry, e, e->next_offset);
+/*+PCD*/
+               /*
+               * Load the offset of the chain entry relative to the
+               * ipt entry into the ipt entry comefrom field for use
+               * by iptcc_find_chain_by_offset in the second pass.
+               */
+               if (h->chain_iterator_cur->num_rules == 0)
+                       e->comefrom = (char *)h->chain_iterator_cur - (char
*)e;
+/*-PCD*/
                r->counter_map.maptype = COUNTER_MAP_NORMAL_MAP;
                r->counter_map.mappos = r->index;

@@ -534,7 +560,6 @@ out_inc:
        return 0;
 }

-
 /* parse an iptables blob into it's pieces */
 static int parse_table(TC_HANDLE_T h)
 {
@@ -545,7 +570,6 @@ static int parse_table(TC_HANDLE_T h)
        /* First pass: over ruleset blob */
        ENTRY_ITERATE(h->entries->entrytable, h->entries->size,
                        cache_add_entry, h, &prev, &num);
-
        /* Second pass: fixup parsed data from first pass */
        list_for_each_entry(c, &h->chains, list) {
                struct rule_head *r;
@@ -1048,7 +1072,8 @@ TC_NEXT_RULE(const STRUCT_ENTRY *prev, T
        DEBUGP_C("next=%p, head=%p...", &r->list,
                &(*handle)->rule_iterator_cur->chain->rules);

-       if (&r->list == &(*handle)->rule_iterator_cur->chain->rules) {
+       if (r == (struct rule_head *)(&(*handle)->rule_iterator_cur->list)
||
+               &r->list == &(*handle)->rule_iterator_cur->chain->rules) {
                (*handle)->rule_iterator_cur = NULL;
                DEBUGP_C("finished, returning NULL\n");
                return NULL;
@@ -1781,8 +1806,12 @@ TC_CREATE_CHAIN(const IPT_CHAINLABEL cha
        }

        DEBUGP("Creating chain `%s'\n", chain);
-       list_add_tail(&c->list, &(*handle)->chains);
-
+/*+PCD
+* Insert new chain alphabetically.
+*/
+//     list_add_tail(&c->list, &(*handle)->chains);
+       iptc_insert_chain(*handle, c);
+/*-PCD*/
        set_changed(*handle);

        return 1;

Re: How many rules were supported iptables?

[m.innocenti]

이근수 ha scritto:
> Hi All.
> I have a problem that is how many rules were supported iptables.
> The program which I maintain and repair generated iptables rules automatically.
> Now, I encounter this problem.
> Source and destination IP address is written range by user.( ex, 1.1.1.1~1.1.1.10 ) then our program generated 10 IP address(1.1.1.1, 1.1.1.2, 1.1.1.3, … , 1.1.1.10) and make 10 iptables rules.
> Unfortunately, user wrote iptables rules like this, 10,000 rules is generated.
> 　“ iptalbes –A FORWARD –p tcp –s 1.1.1.1~1.1.1.100 –d 2.2.2.1~2.2.2.100 –j QUEUE”.

I think you should use the module iprange or ipset.

> Do 10,000 rules operate safely?? Or Some rules don’t operate normally??

10000 rules in one chain have a great impact on performance
(http://people.netfilter.org/kadlec/nftest.pdf).



-- 
**********************************************************************
Marco Innocenti              Gruppo Infrastruttura e Sicurezza
CINECA                       phone:+39 0516171553 / fax:+39 0516132198
Via Magnanelli 6/3           e-mail: innocenti@cineca.it
40033 Casalecchio di Reno    Bologna (Italia)
**********************************************************************

How many rules were supported iptables?

["이근수"]

Hi.
I have a problem that is how many rules were supported iptables.
The program which I maintain and repair generated iptables rules automatically.
Now, I encounter this problem.
Source and destination IP address is written range by user.( ex, 1.1.1.1~1.1.1.10 )
then our program generated 10 IP address(1.1.1.1, 1.1.1.2, 1.1.1.3, … , 1.1.1.10) and make 10 iptables rules.
Unfortunately, user wrote iptables rules like this, 10,000 rules is generated.
  iptalbes –A FORWARD –p tcp –s 1.1.1.1~1.1.1.100 –d 2.2.2.1~2.2.2.100 –j QUEUE.
Do 10,000 rules operate safely?? Or Some rules don’t operate normally??
If some rules don’t operate normally, how many rules iptables does support??------------------------------------------------------------------------
NAVER :: Korea's No.1 search portal
www.naver.com