lots of oopses

lots of oopses

[Amin Azez]

Cc: l7-filter-developers and netfilter-devel

I'm getting lots of oopses with 2.6.17.1 SMP, on a 2 cpu (4 core)
machine testing layer7 at 90Mb/s. (The oops dump worries me, maybe I'm
reading it wrong but it seems to a two column dump of 2 threads, but I
have 4 cores)

I also think these are layer7 related because I can't get them to occur
without a layer7 rule in iptables, although the oops doesn't always
occur in the layer7 module. I also suspect it is related to conntrack
handling.

I have pablos recent set of 8 conntrack patches applied, as well as the
recent layer7 regex-smp-safeness and the layer7 concurrency patch.

There are also quite a few other patches for iptables modules not used
in this example ruleset. In the later oops I have reduced the number of
loaded modules significantly and still obtain the oops; although the
examples here do have all the modules loaded.

I generate the oops using packETH and udp with randomized source
addresses to maximize conntrack creation

My iptables-save is:
# Generated by iptables-save v1.3.5-20060629 on Tue Aug 15 10:49:39 2006
*raw
:PREROUTING ACCEPT [938:92791]
:OUTPUT ACCEPT [848:491202]
COMMIT
# Completed on Tue Aug 15 10:49:39 2006
# Generated by iptables-save v1.3.5-20060629 on Tue Aug 15 10:49:39 2006
*nat
:PREROUTING ACCEPT [125:17240]
:POSTROUTING ACCEPT [6:372]
:OUTPUT ACCEPT [6:372]
COMMIT
# Completed on Tue Aug 15 10:49:39 2006
# Generated by iptables-save v1.3.5-20060629 on Tue Aug 15 10:49:39 2006
*mangle
:PREROUTING ACCEPT [938:92791]
:INPUT ACCEPT [851:80777]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [848:491202]
:POSTROUTING ACCEPT [848:491202]
-A FORWARD -m layer7 --l7proto edonkey -j RETURN
COMMIT
# Completed on Tue Aug 15 10:49:39 2006
# Generated by iptables-save v1.3.5-20060629 on Tue Aug 15 10:49:39 2006
*filter
:INPUT ACCEPT [851:80777]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [848:491202]
COMMIT
# Completed on Tue Aug 15 10:49:39 2006

The most recent oops where I forgot to unload most of the modules, but
was running my layer7 load monitor crashed at a rule-add just after a
conntrack flush. Layer7 is clearly implicated in this one. There is
another oops further down where this is not the case.

+ iptables -t mangle -D FORWARD 1
+ conntrack -F
+ iptables -t mangle -A FORWARD [17179991.580000] Oops: 0000 [#1]
[17179991.580000] SMP
[17179991.580000] Modules linked in: ipt_vlan ipt_connrate ebt_mark
ebtable_nat
ebtable_filter ebtable_broute ebtables ipt_ULOG ipt_ttl ipt_TOS ipt_tos
ipt_TCPMSS ipt_SAME ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE
ipt_LOG ipt_iprange ipt_hashlimit ipt_ECN ipt_ecn ipt_DSCP ipt_dscp
ipt_ah ipt_addrtype iptable_raw iptable_nat iptable_mangle
iptable_filter ip_tables ip_queue cls_fw sch_prio sch_sfq sch_htb crc32
cls_u32 8021q bridge llc ipt_condition ipt_account ipt_recent tg3 e1000
e100 mii ip_conntrack_netlink ip_nat ipt_SET xt_tcpudp xt_tcpmss
xt_string xt_state xt_sctp xt_realm xt_policy xt_pkttype xt_physdev
xt_multiport xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_dccp
xt_conntrack xt_connmark xt_connbytes xt_comment xt_NOTRACK xt_NFQUEUE
xt_MARK xt_CONNMARK ip_conntrack nfnetlink xt_CLASSIFY ipt_layer7
ipt_time ipt_set x_tables ip_set_portmap ip_set_nethash ip_set_macipmap
ip_set_iptree ip_set_ipporthash ip_set_ipmap ip_set_iphash ip_set
ata_piix libata sd_mod scsi_mod
[17179991.580000] CPU:    0
[17179991.580000] EIP:    0060:[<f88de6b1>]    Not tainted VLI
[17179991.580000] EFLAGS: 00010202   (2.6.17.1-smp-dbamK #34)
[17179991.580000] EIP is at match+0x154/0x30a [ipt_layer7]
[17179991.580000] eax: 0000021c   ebx: e84de9e8   ecx: 0000021c   edx:
e84de9e8
[17179991.580000] esi: f8b651b8   edi: f88dec96   ebp: f8b651b8   esp:
f6053b84
[17179991.580000] ds: 007b   es: 007b   ss: 0068
[17179991.580000] Process conntrack (pid: 3067, threadinfo=f6052000
task=f7b39540)
[17179991.580000] Stack: e84de9e8 ecca0c3c 0000021c 00000000 00000000
f5db2780 e3e912e4 f8b65198
[17179991.580000]        f654a000 f8b65128 00000070 f8a2b261 f54fe880
f654a000 f654a000 f88e03a0
[17179991.580000]        f8b651b8 00000000 00000014 f6053bf4 00000000
f895e9cc f8b69498 f8b65000
[17179991.580000] Call Trace:
[17179991.580000]  <f8a2b261> ipt_do_table+0x239/0x4cd [ip_tables]
<f895e037> ipt_route_hook+0x37/0x3b [iptable_mangle]
[17179991.580000]  <c02b0a19> nf_iterate+0x6f/0xaa  <f8a675b9>
br_nf_forward_finish+0x0/0x106 [bridge]
[17179991.580000]  <c02b0abf> nf_hook_slow+0x6b/0xf7  <f8a675b9>
br_nf_forward_finish+0x0/0x106 [bridge]
[17179991.580000]  <f8a6779f> br_nf_forward_ip+0xe0/0x173 [bridge]
<f8a675b9> br_nf_forward_finish+0x0/0x106 [bridge]
[17179991.580000]  <c02b0a19> nf_iterate+0x6f/0xaa  <f8a61f64>
br_forward_finish+0x0/0x5b [bridge]
[17179991.580000]  <c02b0abf> nf_hook_slow+0x6b/0xf7  <f8a61f64>
br_forward_finish+0x0/0x5b [bridge]
[17179991.580000]  <f8a6207d> __br_forward+0x57/0x6e [bridge]
<f8a61f64> br_forward_finish+0x0/0x5b [bridge]
[17179991.580000]  <f8a62d25> br_handle_frame_finish+0xe8/0x138 [bridge]
 <f8a66c38> br_nf_pre_routing_finish+0x135/0x322 [bridge]
[17179991.580000]  <f8a62c3d> br_handle_frame_finish+0x0/0x138 [bridge]
 <f89b1884> ip_nat_in+0x43/0xb0 [iptable_nat]
[17179991.580000]  <f8a66b03> br_nf_pre_routing_finish+0x0/0x322
[bridge]  <c02b0a19> nf_iterate+0x6f/0xaa
[17179991.580000]  <f8a66b03> br_nf_pre_routing_finish+0x0/0x322
[bridge]  <c02b0abf> nf_hook_slow+0x6b/0xf7
[17179991.580000]  <f8a66b03> br_nf_pre_routing_finish+0x0/0x322
[bridge]  <f8a673cd> br_nf_pre_routing+0x264/0x3e4 [bridge]
[17179991.580000]  <f8a66b03> br_nf_pre_routing_finish+0x0/0x322
[bridge]  <c02b0a19> nf_iterate+0x6f/0xaa
[17179991.580000]  <f8a62c3d> br_handle_frame_finish+0x0/0x138 [bridge]
 <c02b0abf> nf_hook_slow+0x6b/0xf7
[17179991.580000]  <f8a62c3d> br_handle_frame_finish+0x0/0x138 [bridge]
 <f8a62ed1> br_handle_frame+0x122/0x1d5 [bridge]
[17179991.580000]  <f8a62c3d> br_handle_frame_finish+0x0/0x138 [bridge]
 <c0297e7f> netif_receive_skb+0x1b8/0x3ee
[17179991.580000]  <c0298139> process_backlog+0x84/0x109  <c029824c>
net_rx_action+0x8e/0x15f
[17179991.580000]  <c012321a> __do_softirq+0xc2/0xd4  <c012325e>
do_softirq+0x32/0x34
[17179991.580000]  <c010534f> do_IRQ+0x3b/0x66  <c0103696>
common_interrupt+0x1a/0x20
[17179991.580000] Code: 98 03 8e f8 00 00 00 01 8b 83 58 01 00 00 85 c0
0f 84 31 01 00 00 8b 54 24 30 80 7a 30 00 0f 84 ee 00 00 00 bf 96 ec 8d
f8 89 ee <ac> ae 75 08 84 c0 75 f8 31 c0 eb 04 19 c0 0c 01 85 c0 c7 44 24
[17179991.580000] EIP: [<f88de6b1>] match+0x154/0x30a [ipt_layer7]
SS:ESP 0068:f6053b84
[17179991.580000]  <0>Kernel panic - not syncing: Fatal exception in
interrupt
[17179991.580000]



Layer7 is not obviously implicated in this one, but I think it is related.

[17183769.024000] Oops: 0000 [#1]
[17183769.024000] SMP
[17183769.024000] Modules linked in: ipt_vlan ipt_connrate ebt_mark
ebtable_nat
ebtable_filter ebtable_broute ebtables ipt_ULOG ipt_ttl ipt_TOS ipt_tos
ipt_TCPMSS ipt_SAME ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE
ipt_LOG ipt_iprange ipt_hashlimit ipt_ECN ipt_ecn ipt_DSCP ipt_dscp
ipt_ah ipt_addrtype iptable_raw iptable_nat iptable_mangle
iptable_filter ip_tables ip_queue cls_fw sch_prio sch_sfq sch_htb crc32
cls_u32 8021q bridge llc ipt_condition ipt_account ipt_recent tg3 e1000
e100 mii ip_conntrack_netlink ip_nat ipt_SET xt_tcpudp xt_tcpmss
xt_string xt_state xt_sctp xt_realm xt_policy xt_pkttype xt_physdev
xt_multiport xt_mark xt_mac xt_limit xt_length xt_helper xt_esp xt_dccp
xt_conntrack xt_connmark xt_connbytes xt_comment xt_NOTRACK xt_NFQUEUE
xt_MARK xt_CONNMARK ip_conntrack nfnetlink xt_CLASSIFY ipt_layer7
ipt_time ipt_set x_tables ip_set_portmap ip_set_nethash ip_set_macipmap
ip_set_iptree ip_set_ipporthash ip_set_ipmap ip_set_iphash ip_set
ata_piix libata sd_mod scsi_mod
[17183769.024000] CPU:    0
[17183769.024000] EIP:    0060:[<f8a2b0f9>]    Not tainted VLI
[17183769.024000] EFLAGS: 00010286   (2.6.17.1-smp-dbamK #34)
[17183769.024000] EIP is at ipt_do_table+0xd1/0x4cd [ip_tables]
[17183769.024000] eax: f8b8e2e8   ebx: f8ba5b48   ecx: 00000000   edx:
00000001
[17183769.024000] esi: f700d000   edi: f8ba7c90   ebp: 00000070   esp:
f7e87bb4
[17183769.024000] ds: 007b   es: 007b   ss: 0068
[17183769.024000] Process ksoftirqd/0 (pid: 3, threadinfo=f7e86000
task=f7e1a070)
[17183769.024000] Stack: f4ff9a80 f700d000 f700d000 f88e03a0 f8ba5b68
00000000 00000014 f7e87bf4
[17183769.024000]        00000000 f895e9cc f8b8e2e8 f8b8e000 f700d000
f700d000 00000000 e1e56420
[17183769.024000]        00000000 f895ea18 f7e87c74 80000000 c04ba010
f895e037 f7e87cac 00000002
[17183769.024000] Call Trace:
[17183769.024000]  <f895e037> ipt_route_hook+0x37/0x3b [iptable_mangle]
 <c02b0a19> nf_iterate+0x6f/0xaa
[17183769.024000]  <f8a675b9> br_nf_forward_finish+0x0/0x106 [bridge]
<c02b0abf> nf_hook_slow+0x6b/0xf7
[17183769.024000]  <f8a675b9> br_nf_forward_finish+0x0/0x106 [bridge]
<f8a6779f> br_nf_forward_ip+0xe0/0x173 [bridge]
[17183769.024000]  <f8a675b9> br_nf_forward_finish+0x0/0x106 [bridge]
<c02b0a19> nf_iterate+0x6f/0xaa
[17183769.024000]  <f8a61f64> br_forward_finish+0x0/0x5b [bridge]
<c02b0abf> nf_hook_slow+0x6b/0xf7
[17183769.024000]  <f8a61f64> br_forward_finish+0x0/0x5b [bridge]
<f8a6207d> __br_forward+0x57/0x6e [bridge]
[17183769.024000]  <f8a61f64> br_forward_finish+0x0/0x5b [bridge]
<f8a62d25> br_handle_frame_finish+0xe8/0x138 [bridge]
[17183769.024000]  <f8a66c38> br_nf_pre_routing_finish+0x135/0x322
[bridge]  <f8a62c3d> br_handle_frame_finish+0x0/0x138 [bridge]
[17183769.024000]  <f89b1884> ip_nat_in+0x43/0xb0 [iptable_nat]
<f8a66b03> br_nf_pre_routing_finish+0x0/0x322 [bridge]
[17183769.024000]  <c02b0a19> nf_iterate+0x6f/0xaa  <f8a66b03>
br_nf_pre_routing_finish+0x0/0x322 [bridge]
[17183769.024000]  <c02b0abf> nf_hook_slow+0x6b/0xf7  <f8a66b03>
br_nf_pre_routing_finish+0x0/0x322 [bridge]
[17183769.024000]  <f8a673cd> br_nf_pre_routing+0x264/0x3e4 [bridge]
<f8a66b03> br_nf_pre_routing_finish+0x0/0x322 [bridge]
[17183769.024000]  <c02b0a19> nf_iterate+0x6f/0xaa  <f8a62c3d>
br_handle_frame_finish+0x0/0x138 [bridge]
[17183769.024000]  <c02b0abf> nf_hook_slow+0x6b/0xf7  <f8a62c3d>
br_handle_frame_finish+0x0/0x138 [bridge]
[17183769.024000]  <f8a62ed1> br_handle_frame+0x122/0x1d5 [bridge]
<f8a62c3d> br_handle_frame_finish+0x0/0x138 [bridge]
[17183769.024000]  <c0297e7f> netif_receive_skb+0x1b8/0x3ee  <c0298139>
process_backlog+0x84/0x109
[17183769.024000]  <c029824c> net_rx_action+0x8e/0x15f  <c012321a>
__do_softirq+0xc2/0xd4
[17183769.024000]  <c01236c9> ksoftirqd+0x0/0xbd  <c012325e>
do_softirq+0x32/0x34
[17183769.024000]  <c0123744> ksoftirqd+0x7b/0xbd  <c01312b9>
kthread+0xb7/0xbd
[17183769.024000]  <c0131202> kthread+0x0/0xbd  <c01010b5>
kernel_thread_helper+0x5/0xb
[17183769.024000] Code: 44 24 5c 8b 54 24 2c 03 7c 86 0c 03 54 86 20 89
6c 24 20 89 54 24 28 85 ff 0f 84 b6 03 00 00 8b 44 24 28 85 c0 0f 84 81
03 00 00 <0f> b6 5f 53 89 d8 24 08 84 c0 0f 84 5e 03 00 00 8b 47 08 8b 4c
[17183769.024000] EIP: [<f8a2b0f9>] ipt_do_table+0xd1/0x4cd [ip_tables]
SS:ESP 0068:f7e87bb4
[17183769.024000]  <0>Kernel panic - not syncing: Fatal exception in
interrupt
[17183769.024000]


(Is this stackdump one or two threads?)

I'm doing more tests to locate the cause.

Sam

Re: lots of oopses

[Patrick McHardy]

[-- Attachment #1: Type: text/plain, Size: 459 bytes --]

Amin Azez wrote:
> The most recent oops where I forgot to unload most of the modules, but
> was running my layer7 load monitor crashed at a rule-add just after a
> conntrack flush. Layer7 is clearly implicated in this one. There is
> another oops further down where this is not the case.

The second one looks like a race in ipt_tables when changing the
ruleset (the attached patch should fix that). The first one looks
like a l7 bug.

BTW, what is ipt_vlan?

[-- Attachment #2: x --]
[-- Type: text/plain, Size: 2246 bytes --]

[NETFILTER]: ip_tables: fix table locking in ipt_do_table

table->private might change because of ruleset changes, don't use it without
holding the lock.

Signed-off-by: Patrick McHardy <kaber@trash.net>

---
commit 338fe5c67e8fb799c9e3470331db6f3c60a31b1e
tree 2dc15d63244ed18a8035ae483ae2d722e7fbcf62
parent 32ce9bc41528c327b1353713b2108d2213128dee
author Patrick McHardy <kaber@trash.net> Tue, 15 Aug 2006 16:06:57 +0200
committer Patrick McHardy <kaber@trash.net> Tue, 15 Aug 2006 16:06:57 +0200

 net/ipv4/netfilter/arp_tables.c |    3 ++-
 net/ipv4/netfilter/ip_tables.c  |    3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/net/ipv4/netfilter/arp_tables.c b/net/ipv4/netfilter/arp_tables.c
index df4854c..8d1d7a6 100644
--- a/net/ipv4/netfilter/arp_tables.c
+++ b/net/ipv4/netfilter/arp_tables.c
@@ -236,7 +236,7 @@ unsigned int arpt_do_table(struct sk_buf
 	struct arpt_entry *e, *back;
 	const char *indev, *outdev;
 	void *table_base;
-	struct xt_table_info *private = table->private;
+	struct xt_table_info *private;
 
 	/* ARP header, plus 2 device addresses, plus 2 IP addresses.  */
 	if (!pskb_may_pull((*pskb), (sizeof(struct arphdr) +
@@ -248,6 +248,7 @@ unsigned int arpt_do_table(struct sk_buf
 	outdev = out ? out->name : nulldevname;
 
 	read_lock_bh(&table->lock);
+	private = table->private;
 	table_base = (void *)private->entries[smp_processor_id()];
 	e = get_entry(table_base, private->hook_entry[hook]);
 	back = get_entry(table_base, private->underflow[hook]);
diff --git a/net/ipv4/netfilter/ip_tables.c b/net/ipv4/netfilter/ip_tables.c
index f316ff5..048514f 100644
--- a/net/ipv4/netfilter/ip_tables.c
+++ b/net/ipv4/netfilter/ip_tables.c
@@ -230,7 +230,7 @@ ipt_do_table(struct sk_buff **pskb,
 	const char *indev, *outdev;
 	void *table_base;
 	struct ipt_entry *e, *back;
-	struct xt_table_info *private = table->private;
+	struct xt_table_info *private;
 
 	/* Initialization */
 	ip = (*pskb)->nh.iph;
@@ -247,6 +247,7 @@ ipt_do_table(struct sk_buff **pskb,
 
 	read_lock_bh(&table->lock);
 	IP_NF_ASSERT(table->valid_hooks & (1 << hook));
+	private = table->private;
 	table_base = (void *)private->entries[smp_processor_id()];
 	e = get_entry(table_base, private->hook_entry[hook]);
 

Re: lots of oopses

[Amin Azez]

Patrick McHardy wrote:
> Amin Azez wrote:
>   
>> The most recent oops where I forgot to unload most of the modules, but
>> was running my layer7 load monitor crashed at a rule-add just after a
>> conntrack flush. Layer7 is clearly implicated in this one. There is
>> another oops further down where this is not the case.
>>     
>
> The second one looks like a race in ipt_tables when changing the
> ruleset (the attached patch should fix that). The first one looks
> like a l7 bug.
>
>   
Thanks Patrick, you are top.
> BTW, what is ipt_vlan?
>   
I think I posted it here a year ago, but I'll do so again if you want it.
It matches on vlan-id.

It was said that strictly this is a layer 2 thing and not for iptables; 
I find it useful though;-
which iptables rules should be applied may depend on vlan stuff, and 
sometimes it seems like there isn't enough mark to go around...

I like the iptables/ebtables seperation but sometimes it seems like they 
should be able to share each-others matches, like one big happy table 
with a few extra points of inspection. Anyway...

thanks again.

Sam
> ------------------------------------------------------------------------
>
> [NETFILTER]: ip_tables: fix table locking in ipt_do_table
>
> table->private might change because of ruleset changes, don't use it without
> holding the lock.
>
> Signed-off-by: Patrick McHardy <kaber@trash.net>
>
> ---
> commit 338fe5c67e8fb799c9e3470331db6f3c60a31b1e
> tree 2dc15d63244ed18a8035ae483ae2d722e7fbcf62
> parent 32ce9bc41528c327b1353713b2108d2213128dee
> author Patrick McHardy <kaber@trash.net> Tue, 15 Aug 2006 16:06:57 +0200
> committer Patrick McHardy <kaber@trash.net> Tue, 15 Aug 2006 16:06:57 +0200
>
>  net/ipv4/netfilter/arp_tables.c |    3 ++-
>  net/ipv4/netfilter/ip_tables.c  |    3 ++-
>  2 files changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/net/ipv4/netfilter/arp_tables.c b/net/ipv4/netfilter/arp_tables.c
> index df4854c..8d1d7a6 100644
> --- a/net/ipv4/netfilter/arp_tables.c
> +++ b/net/ipv4/netfilter/arp_tables.c
> @@ -236,7 +236,7 @@ unsigned int arpt_do_table(struct sk_buf
>  	struct arpt_entry *e, *back;
>  	const char *indev, *outdev;
>  	void *table_base;
> -	struct xt_table_info *private = table->private;
> +	struct xt_table_info *private;
>  
>  	/* ARP header, plus 2 device addresses, plus 2 IP addresses.  */
>  	if (!pskb_may_pull((*pskb), (sizeof(struct arphdr) +
> @@ -248,6 +248,7 @@ unsigned int arpt_do_table(struct sk_buf
>  	outdev = out ? out->name : nulldevname;
>  
>  	read_lock_bh(&table->lock);
> +	private = table->private;
>  	table_base = (void *)private->entries[smp_processor_id()];
>  	e = get_entry(table_base, private->hook_entry[hook]);
>  	back = get_entry(table_base, private->underflow[hook]);
> diff --git a/net/ipv4/netfilter/ip_tables.c b/net/ipv4/netfilter/ip_tables.c
> index f316ff5..048514f 100644
> --- a/net/ipv4/netfilter/ip_tables.c
> +++ b/net/ipv4/netfilter/ip_tables.c
> @@ -230,7 +230,7 @@ ipt_do_table(struct sk_buff **pskb,
>  	const char *indev, *outdev;
>  	void *table_base;
>  	struct ipt_entry *e, *back;
> -	struct xt_table_info *private = table->private;
> +	struct xt_table_info *private;
>  
>  	/* Initialization */
>  	ip = (*pskb)->nh.iph;
> @@ -247,6 +247,7 @@ ipt_do_table(struct sk_buff **pskb,
>  
>  	read_lock_bh(&table->lock);
>  	IP_NF_ASSERT(table->valid_hooks & (1 << hook));
> +	private = table->private;
>  	table_base = (void *)private->entries[smp_processor_id()];
>  	e = get_entry(table_base, private->hook_entry[hook]);
>  
>   

Re: lots of oopses

[Patrick McHardy]

Amin Azez wrote:
> Patrick McHardy wrote:
> 
>> BTW, what is ipt_vlan?
>>   
> 
> I think I posted it here a year ago, but I'll do so again if you want it.
> It matches on vlan-id.
> 
> It was said that strictly this is a layer 2 thing and not for iptables;
> I find it useful though;-
> which iptables rules should be applied may depend on vlan stuff, and
> sometimes it seems like there isn't enough mark to go around...
> 
> I like the iptables/ebtables seperation but sometimes it seems like they
> should be able to share each-others matches, like one big happy table
> with a few extra points of inspection. Anyway...

Agreed. It should be possible for ebtables to use all iptables matches
looking only at packet data, but not necessarily the other way around.
Unfortunately ebtables is in large parts a copy of iptables, with just
enough differences to prevent it from using x_tables.

Re: lots of oopses

[Patrick McHardy]

Amin Azez wrote:
> Patrick McHardy wrote:
> 
>> Unfortunately ebtables is in large parts a copy of iptables, with just
>> enough differences to prevent it from using x_tables.
> 
> Hmm, now I'm interested, what are the problematic differences?

Mostly slightly different structure layout.

ipt_vlan

[Amin Azez]

[-- Attachment #1: Type: text/plain, Size: 382 bytes --]

Attached is my ipt_vlan patch for 2.6.17 and 2.6.11, and the iptables
patch to go with it.
It's based on the mac match.

I think the iptables patch is wrong, the way I freak the extension
makefile needs reviewing, but it does compile.

it doesn't require any vlan interfaces to be set up on the box; unless
you want to route (I guess);
I do bridging and get to match on vlan.

Sam


[-- Attachment #2: vlan.2.6.17.patch --]
[-- Type: text/x-patch, Size: 4415 bytes --]

Index: linux-2.6.17.1/include/linux/netfilter_ipv4/ipt_vlan.h
===================================================================
--- /dev/null
+++ linux-2.6.17.1/include/linux/netfilter_ipv4/ipt_vlan.h
@@ -0,0 +1,8 @@
+#ifndef _IPT_VLAN_H
+#define _IPT_VLAN_H
+
+struct ipt_vlan_info {
+    unsigned short vlan;
+    int invert;
+};
+#endif /*_IPT_VLAN_H*/
Index: linux-2.6.17.1/net/ipv4/netfilter/Kconfig
===================================================================
--- linux-2.6.17.1.orig/net/ipv4/netfilter/Kconfig
+++ linux-2.6.17.1/net/ipv4/netfilter/Kconfig
@@ -235,6 +235,15 @@ config IP_NF_IPTABLES
 	  To compile it as a module, choose M here.  If unsure, say N.
 
 # The matches.
+config IP_NF_MATCH_VLAN
+	tristate "VLAN address match support"
+	depends on IP_NF_IPTABLES
+	help
+	  VLAN matching allows you to match packets based on the vlan
+	  tag of the packet, if your switch fowards them
+
+	  To compile it as a module, choose M here.  If unsure, say N.
+
 config IP_NF_MATCH_LAYER7
 	tristate "Layer 7 match support (EXPERIMENTAL)"
 	depends on IP_NF_IPTABLES && IP_NF_CT_ACCT && IP_NF_CONNTRACK && EXPERIMENTAL
Index: linux-2.6.17.1/net/ipv4/netfilter/Makefile
===================================================================
--- linux-2.6.17.1.orig/net/ipv4/netfilter/Makefile
+++ linux-2.6.17.1/net/ipv4/netfilter/Makefile
@@ -64,6 +64,8 @@ obj-$(CONFIG_IP_NF_MATCH_ADDRTYPE) += ip
 
 obj-$(CONFIG_IP_NF_MATCH_LAYER7) += ipt_layer7.o
 
+obj-$(CONFIG_IP_NF_MATCH_VLAN) += ipt_vlan.o
+
 # targets
 obj-$(CONFIG_IP_NF_TARGET_REJECT) += ipt_REJECT.o
 obj-$(CONFIG_IP_NF_TARGET_TOS) += ipt_TOS.o
Index: linux-2.6.17.1/net/ipv4/netfilter/ipt_vlan.c
===================================================================
--- /dev/null
+++ linux-2.6.17.1/net/ipv4/netfilter/ipt_vlan.c
@@ -0,0 +1,85 @@
+/* Kernel module to match VLAN parameters based on ipt_mac */
+
+/* (C) 1999-2001 Paul `Rusty' Russell
+ * (C) 2002-2004 Netfilter Core Team <coreteam@netfilter.org>
+ * (C) UFO Mechanic <azez@ufomechanic.net>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/module.h>
+#include <linux/skbuff.h>
+#include <linux/if_ether.h>
+#include <linux/if_vlan.h>
+
+#include <linux/netfilter_ipv4/ipt_vlan.h>
+#include <linux/netfilter_ipv4/ip_tables.h>
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("UFO Mechanic <azez@ufomechanic.net>");
+MODULE_DESCRIPTION("iptables vlan matching module");
+
+#define IS_VLAN_IP (skb->protocol == __constant_htons(ETH_P_8021Q) &&    \
+	hdr->h_vlan_encapsulated_proto == __constant_htons(ETH_P_IP)) /* &&  \ brnf_filter_vlan_tagged) */
+#define IS_VLAN_IPV6 (skb->protocol == __constant_htons(ETH_P_8021Q) &&    \
+	hdr->h_vlan_encapsulated_proto == __constant_htons(ETH_P_IPV6)) /* &&  \ brnf_filter_vlan_tagged) */
+#define IS_VLAN_ARP (skb->protocol == __constant_htons(ETH_P_8021Q) &&   \
+	hdr->h_vlan_encapsulated_proto == __constant_htons(ETH_P_ARP)) /* && \ brnf_filter_vlan_tagged) */
+
+static int
+match(	const struct sk_buff *skb,
+	const struct net_device *in,
+	const struct net_device *out,
+	const struct xt_match *match,
+	const void *matchinfo,
+	int offset,
+	int *hotdrop)
+{
+    const struct ipt_vlan_info *info = matchinfo;
+    struct vlan_ethhdr *hdr = vlan_eth_hdr(skb);
+
+/* should we use: static inline int __vlan_get_tag(struct sk_buff *skb, unsigned short *tag) */
+    /* Is it even a VLAN packet? */
+    if ((IS_VLAN_IP || IS_VLAN_IPV6 || IS_VLAN_ARP)) {
+	    /* If so, compare... */
+	    return (( (ntohs(hdr->h_vlan_TCI)==info->vlan) ^ info->invert));
+    }
+    return 0 ^ info->invert;
+}
+
+static int
+ipt_vlan_checkentry(const char *tablename,
+	const struct ipt_ip *ip,
+	const struct xt_match *match,
+	void *matchinfo,
+	unsigned int matchsize,
+	unsigned int hook_mask)
+{
+	if (matchsize != IPT_ALIGN(sizeof(struct ipt_vlan_info)))
+		return 0;
+
+	return 1;
+}
+
+static struct ipt_match vlan_match = {
+	.name		= "vlan",
+	.match		= &match,
+	.checkentry	= &ipt_vlan_checkentry,
+	.matchsize	= sizeof(struct ipt_vlan_info),
+	.me		= THIS_MODULE,
+};
+
+static int __init init(void)
+{
+	return ipt_register_match(&vlan_match);
+}
+
+static void __exit fini(void)
+{
+	ipt_unregister_match(&vlan_match);
+}
+
+module_init(init);
+module_exit(fini);

[-- Attachment #3: vlan.2.6.11.patch --]
[-- Type: text/x-patch, Size: 4417 bytes --]

diff -Nru ../linux-2.6.11.7-reference/include/linux/netfilter_ipv4/ipt_vlan.h ./include/linux/netfilter_ipv4/ipt_vlan.h
--- ../linux-2.6.11.7-reference/include/linux/netfilter_ipv4/ipt_vlan.h	1970-01-01 01:00:00.000000000 +0100
+++ ./include/linux/netfilter_ipv4/ipt_vlan.h	2005-07-13 14:59:24.000000000 +0100
@@ -0,0 +1,8 @@
+#ifndef _IPT_VLAN_H
+#define _IPT_VLAN_H
+
+struct ipt_vlan_info {
+    unsigned short vlan;
+    int invert;
+};
+#endif /*_IPT_VLAN_H*/
diff -Nru ../linux-2.6.11.7-reference/net/ipv4/netfilter/ipt_vlan.c ./net/ipv4/netfilter/ipt_vlan.c
--- ../linux-2.6.11.7-reference/net/ipv4/netfilter/ipt_vlan.c	1970-01-01 01:00:00.000000000 +0100
+++ ./net/ipv4/netfilter/ipt_vlan.c	2005-07-22 09:49:01.000000000 +0100
@@ -0,0 +1,82 @@
+/* Kernel module to match VLAN parameters based on ipt_mac */
+
+/* (C) 1999-2001 Paul `Rusty' Russell
+ * (C) 2002-2004 Netfilter Core Team <coreteam@netfilter.org>
+ * (C) UFO Mechanic <azez@ufomechanic.net>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/module.h>
+#include <linux/skbuff.h>
+#include <linux/if_ether.h>
+#include <linux/if_vlan.h>
+
+#include <linux/netfilter_ipv4/ipt_vlan.h>
+#include <linux/netfilter_ipv4/ip_tables.h>
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("UFO Mechanic <azez@ufomechanic.net>");
+MODULE_DESCRIPTION("iptables vlan matching module");
+
+#define IS_VLAN_IP (skb->protocol == __constant_htons(ETH_P_8021Q) &&    \
+	hdr->h_vlan_encapsulated_proto == __constant_htons(ETH_P_IP)) /* &&  \ brnf_filter_vlan_tagged) */
+#define IS_VLAN_IPV6 (skb->protocol == __constant_htons(ETH_P_8021Q) &&    \
+	hdr->h_vlan_encapsulated_proto == __constant_htons(ETH_P_IPV6)) /* &&  \ brnf_filter_vlan_tagged) */
+#define IS_VLAN_ARP (skb->protocol == __constant_htons(ETH_P_8021Q) &&   \
+	hdr->h_vlan_encapsulated_proto == __constant_htons(ETH_P_ARP)) /* && \ brnf_filter_vlan_tagged) */
+
+static int
+match(const struct sk_buff *skb,
+      const struct net_device *in,
+      const struct net_device *out,
+      const void *matchinfo,
+      int offset,
+      int *hotdrop)
+{
+    const struct ipt_vlan_info *info = matchinfo;
+    struct vlan_ethhdr *hdr = vlan_eth_hdr(skb);
+
+/* should we use: static inline int __vlan_get_tag(struct sk_buff *skb, unsigned short *tag) */
+    /* Is it even a VLAN packet? */
+    if ((IS_VLAN_IP || IS_VLAN_IPV6 || IS_VLAN_ARP)) {
+	    /* If so, compare... */
+	    return (( (ntohs(hdr->h_vlan_TCI)==info->vlan) ^ info->invert));
+    }
+    return 0 ^ info->invert;
+}
+
+static int
+ipt_vlan_checkentry(const char *tablename,
+		   const struct ipt_ip *ip,
+		   void *matchinfo,
+		   unsigned int matchsize,
+		   unsigned int hook_mask)
+{
+	if (matchsize != IPT_ALIGN(sizeof(struct ipt_vlan_info)))
+		return 0;
+
+	return 1;
+}
+
+static struct ipt_match vlan_match = {
+	.name		= "vlan",
+	.match		= &match,
+	.checkentry	= &ipt_vlan_checkentry,
+	.me		= THIS_MODULE,
+};
+
+static int __init init(void)
+{
+	return ipt_register_match(&vlan_match);
+}
+
+static void __exit fini(void)
+{
+	ipt_unregister_match(&vlan_match);
+}
+
+module_init(init);
+module_exit(fini);
--- kernel/net/ipv4/netfilter/Makefile.orig	2005-07-22 12:12:46.000000000 +0100
+++ kernel/net/ipv4/netfilter/Makefile	2005-07-22 12:13:08.000000000 +0100
@@ -45,6 +45,7 @@
 obj-$(CONFIG_IP_NF_MATCH_SCTP) += ipt_sctp.o
 obj-$(CONFIG_IP_NF_MATCH_MARK) += ipt_mark.o
 obj-$(CONFIG_IP_NF_MATCH_MAC) += ipt_mac.o
+obj-$(CONFIG_IP_NF_MATCH_VLAN) += ipt_vlan.o
 obj-$(CONFIG_IP_NF_MATCH_IPRANGE) += ipt_iprange.o
 obj-$(CONFIG_IP_NF_MATCH_PKTTYPE) += ipt_pkttype.o
 obj-$(CONFIG_IP_NF_MATCH_MULTIPORT) += ipt_multiport.o
--- kernel/net/ipv4/netfilter/Kconfig.orig	2005-07-22 12:13:18.000000000 +0100
+++ kernel/net/ipv4/netfilter/Kconfig	2005-07-22 12:14:37.000000000 +0100
@@ -183,6 +183,15 @@
 	  Unless you know what you're doing, leave it at the default of 2kB.
 
 
+config IP_NF_MATCH_VLAN
+	tristate "VLAN address match support"
+	depends on IP_NF_IPTABLES
+	help
+	  VLAN matching allows you to match packets based on the vlan
+	  tag of the packet, if your switch fowards them
+
+	  To compile it as a module, choose M here.  If unsure, say N.
+
 config IP_NF_MATCH_PKTTYPE
 	tristate "Packet type match support"
 	depends on IP_NF_IPTABLES

[-- Attachment #4: iptables.vlan.patch --]
[-- Type: text/x-patch, Size: 2707 bytes --]

--- extensions/Makefile.orig    2006-06-30 10:41:38.000000000 +0100
+++ extensions/Makefile 2006-06-30 10:42:00.000000000 +0100
@@ -14,2 +14,4 @@

+PF_EXT_SLIB+=vlan
+
 # Optionals
--- extensions/libipt_vlan.c	2005-09-26 14:26:01.000000000 +0100
+++ extensions/libipt_vlan.c	2005-09-26 14:18:17.000000000 +0100
@@ -0,0 +1,105 @@
+/* Shared library add-on to iptables to add VLAN address support. */
+#include <stdio.h>
+#include <netdb.h>
+#include <string.h>
+#include <stdlib.h>
+#include <getopt.h>
+#if defined(__GLIBC__) && __GLIBC__ == 2
+#include <net/ethernet.h>
+#else
+#include <linux/if_ether.h>
+#endif
+#include <iptables.h>
+#include <linux/netfilter_ipv4/ipt_vlan.h>
+
+/* Function which prints out usage message. */
+static void
+help(void)
+{
+	printf(
+"VLAN v%s options:\n"
+" --vlan [!] <vlan_id>\n"
+"				Match source VLAN id\n"
+"\n", IPTABLES_VERSION);
+}
+
+static struct option opts[] = {
+	{ "vlan", 1, 0, '1' },
+	{0}
+};
+
+/* Function which parses command options; returns true if it
+   ate an option */
+static int
+parse(int c, char **argv, int invert, unsigned int *flags,
+      const struct ipt_entry *entry,
+      unsigned int *nfcache,
+      struct ipt_entry_match **match)
+{
+	struct ipt_vlan_info *vlaninfo = (struct ipt_vlan_info *)(*match)->data;
+
+	switch (c) {
+	case '1':
+		check_inverse(optarg, &invert, &optind, 0);
+		vlaninfo->vlan=atoi(argv[optind-1]);
+		if (invert)
+			vlaninfo->invert = 1;
+		*flags = 1;
+		break;
+
+	default:
+		return 0;
+	}
+
+	return 1;
+}
+
+/* Final check; must have specified --vlan. */
+static void final_check(unsigned int flags)
+{
+	if (!flags)
+		exit_error(PARAMETER_PROBLEM,
+			   "You must specify `--vlan'");
+}
+
+/* Prints out the matchinfo. */
+static void
+print(const struct ipt_ip *ip,
+      const struct ipt_entry_match *match,
+      int numeric)
+{
+	printf("vlan ");
+
+	if (((struct ipt_vlan_info *)match->data)->invert)
+		printf("! ");
+
+	printf("%d ",((struct ipt_vlan_info *)match->data)->vlan);
+}
+
+/* Saves the union ipt_matchinfo in parsable form to stdout. */
+static void save(const struct ipt_ip *ip, const struct ipt_entry_match *match)
+{
+	if (((struct ipt_vlan_info *)match->data)->invert)
+		printf("! ");
+
+	printf("--vlan %d ",((struct ipt_vlan_info *)match->data)->vlan);
+}
+
+static struct iptables_match vlan = { 
+	.next		= NULL,
+ 	.name		= "vlan",
+	.version	= IPTABLES_VERSION,
+	.size		= IPT_ALIGN(sizeof(struct ipt_vlan_info)),
+	.userspacesize	= IPT_ALIGN(sizeof(struct ipt_vlan_info)),
+	.help		= &help,
+	.parse		= &parse,
+	.final_check	= &final_check,
+	.print		= &print,
+	.save		= &save,
+	.extra_opts	= opts
+};
+
+void _init(void)
+{
+	register_match(&vlan);
+}

Re: ipt_vlan

[Patrick McHardy]

Amin Azez wrote:
> Attached is my ipt_vlan patch for 2.6.17 and 2.6.11, and the iptables
> patch to go with it. It's based on the mac match.

This looks useful. If we hadn't already got ebt_vlan (which seems
to do the same thing with a few extra features) I would be tempted
to ask you to submit it :)

> I think the iptables patch is wrong, the way I freak the extension
> makefile needs reviewing, but it does compile.

> --- extensions/Makefile.orig    2006-06-30 10:41:38.000000000 +0100
> +++ extensions/Makefile 2006-06-30 10:42:00.000000000 +0100
> @@ -14,2 +14,4 @@
> 
> +PF_EXT_SLIB+=vlan
> +

Its fine this way, although the usual way for extensions is to either
add them to the long list at the top or add a .test script.