2.4 kernels and max # of rules with iptables

2.4 kernels and max # of rules with iptables

[Jason Wever]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

Hi All,

I don't know if anyone else has run into this or not, but over in Gentoo
we've had a user report[1] of a "hard limit" as to the number of rules
you can have with iptables on the 2.4 kernels before it starts erroring
out. Currently, it seems that the limit is 857 rules, and the error
given is "iptables: Memory allocation problem".

I've been able to confirm this behavior on 2.4.31 (using
iptables 1.2.11 and 1.3.1) and was able to load more than 10,000 rules
in 2.6.12-rc3 before I gave up.

Current Gentoo userland is using either kernel headers from 2.4.23 (for
stable keywords) and 2.4.26 (for testing keywords).

If anyone has any suggestions or pointers, I'd be glad to hear them.

[1] - https://bugs.gentoo.org/show_bug.cgi?id=75668

Thanks,
-- 
Jason Wever
Gentoo/Sparc Team Co-Lead

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.4 kernels and max # of rules with iptables

[David S. Miller]

From: Jason Wever <weeve@gentoo.org>
Date: Sun, 19 Jun 2005 20:07:12 -0600

> I've been able to confirm this behavior on 2.4.31 (using
> iptables 1.2.11 and 1.3.1) and was able to load more than 10,000 rules
> in 2.6.12-rc3 before I gave up.

64-bit or 32-bit userland binaries?

He could be hitting the kmalloc() limit via the netfilter
32-bit userland compat code in:

       arch/sparc64/kernel/sys_sparc32.c:do_netfilter_replace()

You have to make sure you replicate the netfilter commands
his system is running, in order to determine where the limit
is actually coming from.

Re: 2.4 kernels and max # of rules with iptables

[Jason Wever]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]

On Sun, 19 Jun 2005 19:20:34 -0700 (PDT)
"David S. Miller" <davem@davemloft.net> wrote:

> 64-bit or 32-bit userland binaries?

32 bit userland binaries.  Currently we don't have a working 64 bit
environment that is suggested for general use.
 
> He could be hitting the kmalloc() limit via the netfilter
> 32-bit userland compat code in:
> 
>        arch/sparc64/kernel/sys_sparc32.c:do_netfilter_replace()
> 
> You have to make sure you replicate the netfilter commands
> his system is running, in order to determine where the limit
> is actually coming from.

I used a C program[1] written by Gustavo Zacarias to replicate here and
I'll post a follow-up once I get a response from the user.

[1] - http://dev.gentoo.org/~gustavoz/source/iptables_test.c

Thanks,
-- 
Jason Wever
Gentoo/Sparc Team Co-Lead

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.4 kernels and max # of rules with iptables

[Jason Wever]

[-- Attachment #1: Type: text/plain, Size: 397 bytes --]

On Sun, 19 Jun 2005 20:27:12 -0600
Jason Wever <weeve@gentoo.org> wrote:

> I used a C program[1] written by Gustavo Zacarias to replicate here
> and I'll post a follow-up once I get a response from the user.
> 
> [1] - http://dev.gentoo.org/~gustavoz/source/iptables_test.c

User posted his ruleset and it also fails at 857 rules.

Cheers,
-- 
Jason Wever
Gentoo/Sparc Team Co-Lead

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.4 kernels and max # of rules with iptables

[Josh Grebe]

Good Morning,

I threw a little printk patch at Jason and we determined that to be 
correct. In do_netfilter_replace(), on about line 2924, the call
        krepl = (struct ipt_replace *)kmalloc(kreplsize, GFP_KERNEL);
results in krepl being NULL after a pile of rules have been added.

I compiled a statically linked iptables binary and Jason was able to 
add over 7k rules before he stopped it, the 32 bit iptables bombs 
before 900 rules.

As far as changing this behaviour, I plead ignorance.

Thanks,

Josh


Jason Wever wrote:
> On Sun, 19 Jun 2005 19:20:34 -0700 (PDT)
> "David S. Miller" <davem@davemloft.net> wrote:
> 
> 
>>64-bit or 32-bit userland binaries?
> 
> 
> 32 bit userland binaries.  Currently we don't have a working 64 bit
> environment that is suggested for general use.
>  
> 
>>He could be hitting the kmalloc() limit via the netfilter
>>32-bit userland compat code in:
>>
>>       arch/sparc64/kernel/sys_sparc32.c:do_netfilter_replace()
>>

Re: 2.4 kernels and max # of rules with iptables

[David S. Miller]

From: Josh Grebe <josh@brokedown.net>
Date: Wed, 17 Aug 2005 08:06:06 -0500

> I threw a little printk patch at Jason and we determined that to be 
> correct. In do_netfilter_replace(), on about line 2924, the call
>         krepl = (struct ipt_replace *)kmalloc(kreplsize, GFP_KERNEL);
> results in krepl being NULL after a pile of rules have been added.
> 
> I compiled a statically linked iptables binary and Jason was able to 
> add over 7k rules before he stopped it, the 32 bit iptables bombs 
> before 900 rules.
> 
> As far as changing this behaviour, I plead ignorance.

So how large was kreplsize in the failing call?

Re: 2.4 kernels and max # of rules with iptables

[Josh Grebe]

[-- Attachment #1: Type: text/plain, Size: 983 bytes --]

I guess that would help...

kmalloc failed, size of kreplsize is 939608



David S. Miller wrote:
> From: Josh Grebe <josh@brokedown.net>
> Date: Wed, 17 Aug 2005 08:06:06 -0500
> 
> 
>>I threw a little printk patch at Jason and we determined that to be 
>>correct. In do_netfilter_replace(), on about line 2924, the call
>>        krepl = (struct ipt_replace *)kmalloc(kreplsize, GFP_KERNEL);
>>results in krepl being NULL after a pile of rules have been added.
>>
>>I compiled a statically linked iptables binary and Jason was able to 
>>add over 7k rules before he stopped it, the 32 bit iptables bombs 
>>before 900 rules.
>>
>>As far as changing this behaviour, I plead ignorance.
> 
> 
> So how large was kreplsize in the failing call?
> -
> To unsubscribe from this list: send the line "unsubscribe sparclinux" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> !DSPAM:43038246130492096615112!
> 

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: 2.4 kernels and max # of rules with iptables

[David S. Miller]

From: Josh Grebe <josh@brokedown.net>
Subject: Re: 2.4 kernels and max # of rules with iptables
Date: Wed, 17 Aug 2005 13:53:38 -0500

> I guess that would help...
> 
> kmalloc failed, size of kreplsize is 939608

Yes, when running 32-bit programs on a 64-bit kernel,
there will be a hard limit of how many ipt_replace
entries can be loaded because we have to copy the
user's rule set into a kernel copy and therefore we
have to allocate kmalloc() memory to hold that.

You _could_ experiment with making the code there use
vmalloc()/vfree() instead, and if you get it working cleanly I'd be
happy to add such a patch to fix this.

Re: 2.4 kernels and max # of rules with iptables

[Gustavo Zacarias]

[-- Attachment #1.1: Type: text/plain, Size: 286 bytes --]

David S. Miller wrote:

> You _could_ experiment with making the code there use
> vmalloc()/vfree() instead, and if you get it working cleanly I'd be
> happy to add such a patch to fix this.

Tested with 5000+ rules.
Does it look clean enough?

-- 
Gustavo Zacarias
Gentoo/SPARC monkey

[-- Attachment #1.2: 2.4-sparc64-netfilter-ioctl32.patch --]
[-- Type: text/x-patch, Size: 1246 bytes --]

diff -Nura linux-2.4.31/arch/sparc64/kernel/sys_sparc32.c linux-2.4.31.netfilter/arch/sparc64/kernel/sys_sparc32.c
--- linux-2.4.31/arch/sparc64/kernel/sys_sparc32.c	2005-04-03 22:42:19.000000000 -0300
+++ linux-2.4.31.netfilter/arch/sparc64/kernel/sys_sparc32.c	2005-08-22 12:34:14.000000000 -0300
@@ -50,6 +50,7 @@
 #include <linux/in.h>
 #include <linux/icmpv6.h>
 #include <linux/sysctl.h>
+#include <linux/vmalloc.h>
 #include <linux/dnotify.h>
 #include <linux/netfilter_ipv4/ip_tables.h>
 
@@ -2919,12 +2920,12 @@
 	if (optlen != kreplsize)
 		return -ENOPROTOOPT;
 
-	krepl = (struct ipt_replace *)kmalloc(kreplsize, GFP_KERNEL);
+	krepl = (struct ipt_replace *)vmalloc(kreplsize);
 	if (krepl == NULL)
 		return -ENOMEM;
 
 	if (copy_from_user(krepl, optval, kreplsize)) {
-		kfree(krepl);
+		vfree(krepl);
 		return -EFAULT;
 	}
 
@@ -2935,7 +2936,7 @@
 	krepl->counters = (struct ipt_counters *)kmalloc(
 					kcountersize, GFP_KERNEL);
 	if (krepl->counters == NULL) {
-		kfree(krepl);
+		vfree(krepl);
 		return -ENOMEM;
 	}
 
@@ -2949,8 +2950,8 @@
 		copy_to_user(counters32, krepl->counters, kcountersize))
 			ret = -EFAULT;
 
-	kfree(krepl->counters);
-	kfree(krepl);
+	vfree(krepl->counters);
+	vfree(krepl);
 
 	return ret;
 }

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: 2.4 kernels and max # of rules with iptables

[David S. Miller]

From: Gustavo Zacarias <gustavoz@gentoo.org>
Date: Mon, 22 Aug 2005 14:10:28 -0300

> David S. Miller wrote:
> 
> > You _could_ experiment with making the code there use
> > vmalloc()/vfree() instead, and if you get it working cleanly I'd be
> > happy to add such a patch to fix this.
> 
> Tested with 5000+ rules.
> Does it look clean enough?

Looks good to me, I'll pass this along to the netfilter maintainers.

Re: 2.4 kernels and max # of rules with iptables

[Gustavo Zacarias]

[-- Attachment #1.1: Type: text/plain, Size: 331 bytes --]

David S. Miller wrote:

> Looks good to me, I'll pass this along to the netfilter maintainers.

Actually i'm still doing a kmalloc for replsize->counter, that may work,
but since we're cleaning up it should be a vmalloc too (specially since
i'm doing a vfree).
Second time is the charm :)

-- 
Gustavo Zacarias
Gentoo/SPARC monkey

[-- Attachment #1.2: 2.4-netfilter-ioctl32.patch --]
[-- Type: text/x-patch, Size: 1430 bytes --]

diff -Nura linux-2.4.31/arch/sparc64/kernel/sys_sparc32.c linux-2.4.31.netfilter/arch/sparc64/kernel/sys_sparc32.c
--- linux-2.4.31/arch/sparc64/kernel/sys_sparc32.c	2005-04-03 22:42:19.000000000 -0300
+++ linux-2.4.31.netfilter/arch/sparc64/kernel/sys_sparc32.c	2005-08-22 14:14:40.000000000 -0300
@@ -50,6 +50,7 @@
 #include <linux/in.h>
 #include <linux/icmpv6.h>
 #include <linux/sysctl.h>
+#include <linux/vmalloc.h>
 #include <linux/dnotify.h>
 #include <linux/netfilter_ipv4/ip_tables.h>
 
@@ -2919,12 +2920,12 @@
 	if (optlen != kreplsize)
 		return -ENOPROTOOPT;
 
-	krepl = (struct ipt_replace *)kmalloc(kreplsize, GFP_KERNEL);
+	krepl = (struct ipt_replace *)vmalloc(kreplsize);
 	if (krepl == NULL)
 		return -ENOMEM;
 
 	if (copy_from_user(krepl, optval, kreplsize)) {
-		kfree(krepl);
+		vfree(krepl);
 		return -EFAULT;
 	}
 
@@ -2932,10 +2933,9 @@
 		((struct ipt_replace32 *)krepl)->counters);
 
 	kcountersize = krepl->num_counters * sizeof(struct ipt_counters);
-	krepl->counters = (struct ipt_counters *)kmalloc(
-					kcountersize, GFP_KERNEL);
+	krepl->counters = (struct ipt_counters *)vmalloc(kcountersize);
 	if (krepl->counters == NULL) {
-		kfree(krepl);
+		vfree(krepl);
 		return -ENOMEM;
 	}
 
@@ -2949,8 +2949,8 @@
 		copy_to_user(counters32, krepl->counters, kcountersize))
 			ret = -EFAULT;
 
-	kfree(krepl->counters);
-	kfree(krepl);
+	vfree(krepl->counters);
+	vfree(krepl);
 
 	return ret;
 }

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]