Hacking iptables 1.3.5 for 2.6.17

Hacking iptables 1.3.5 for 2.6.17

[Amin Azez]

Whats the official way to add extra extensions targets extensions/Makefile?

I generally patch extensions/Makefile to add to PF_EXT_SLIB, like
PF_EXT_SLIB+=new_extension

but then I get complaints about it being listed twice. Is there a neater
way to express extra extensions to the Makefile?

Also;

I don't think this problem is specific to iptables, but general to
building anything against a kernel.

If a kernel is built with the -O=dir option to place build files outside
of the source tree, then version.h, autoconf.h and compile.h are written
to dir/include/linux instead of src/include/linux

This means that
make KERNEL_DIR=src
is not enough, as dir also needs to be in the search path (or dir/include)

Probably it needs to be before KERNEL_DIR in the search path.



Sam

Re: Hacking iptables 1.3.5 for 2.6.17

[Yasuyuki KOZAKAI]

From: Amin Azez <azez@ufomechanic.net>
Date: Fri, 30 Jun 2006 10:10:50 +0100

> Whats the official way to add extra extensions targets extensions/Makefile?
> 
> I generally patch extensions/Makefile to add to PF_EXT_SLIB, like
> PF_EXT_SLIB+=new_extension
>
> but then I get complaints about it being listed twice. Is there a neater
> way to express extra extensions to the Makefile?

You can build your extension with iptables by putting .new_extension-test
(or .new_extension-test6 if it is for ip6tables) which tests whether kernel
source support the extension. Please look extensions/.test-* .

> Also;
> 
> I don't think this problem is specific to iptables, but general to
> building anything against a kernel.
> 
> If a kernel is built with the -O=dir option to place build files outside
> of the source tree, then version.h, autoconf.h and compile.h are written
> to dir/include/linux instead of src/include/linux
> 
> This means that
> make KERNEL_DIR=src
> is not enough, as dir also needs to be in the search path (or dir/include)
> 
> Probably it needs to be before KERNEL_DIR in the search path.

The automatically generated files at kernel compilation are not necessary
to build iptables, then KERNEL_DIR is enough. Or do I miss something ?

-- Yasuyuki Kozakai

Re: Hacking iptables 1.3.5 for 2.6.17

[Yasuyuki KOZAKAI]

From: Amin Azez <azez@ufomechanic.net>
Date: Fri, 30 Jun 2006 11:44:33 +0100

> Well, my src/linux-2.6.17.1/include/linux/config.h (line 6) tries to
> include linux/autoconf.h, which is now part of the O=dir build output.
> If it can't find my build autoconf.h then it falls back to
> /usr/include/linux/autoconf.h which is:
> #error Invalid kernel header included in userspace
> 
> In case this much is bogus, here are the include dependancies;
> 
> Extensions found: IPv4:CLUSTERIP IPv4:connbytes IPv4:dccp IPv4:recent
> IPv4:string IPv6:ah IPv6:esp IPv6:frag IPv6:ipv6header IPv6:hbh IPv6:dst
> IPv6:REJECT IPv6:rt
> cc -O2 -Wall -Wunused -I/opt/KERNEL/linux-2.6.17.1//include -Iinclude/
> -DIPTABLES_VERSION=\"1.3.5-20060629\" -D_UNKNOWN_KERNEL_POINTER_SIZE
> -fPIC -o extensions/libipt_ah_sh.o -c extensions/libipt_ah.c
> In file included from /opt/KERNEL/linux-2.6.17.1/include/linux/config.h:6,
> from /opt/KERNEL/linux-2.6.17.1/include/linux/netfilter_ipv4.h:8,
> from /opt/KERNEL/linux-2.6.17.1/include/linux/netfilter_ipv4/ip_tables.h:26,
> from include/libiptc/libiptc.h:6,
> from include/iptables.h:5,
> from extensions/libipt_ah.c:8:
> /usr/include/linux/autoconf.h:1:2: #error Invalid kernel header included
> in usersp

I was seeing the recent git tree that the above problem was fixed.

As you says, netfilter_ipv4.h in 2.6.17.x has unnecessary
'#include <config.h>' outside of __KERNEL__. But it seemd to be deleted
in recent kernel as follows.

> commit 62c4f0a2d5a188f73a94f2cb8ea0dba3e7cf0a7f
> Author: David Woodhouse <dwmw2@infradead.org>
> Date:   Wed Apr 26 12:56:16 2006 +0100
> 
>     Don't include linux/config.h from anywhere else in include/
> 
>     Signed-off-by: David Woodhouse <dwmw2@infradead.org> 

I don't think/know that this goes to stable tree, though.

-- Yasuyuki Kozakai

xtables Re: Hacking iptables 1.3.5 for 2.6.17

[Amin Azez]

Yasuyuki KOZAKAI wrote:
> As you says, netfilter_ipv4.h in 2.6.17.x has unnecessary
> '#include <config.h>' outside of __KERNEL__. But it seemd to be deleted
> in recent kernel as follows.
> 
>> commit 62c4f0a2d5a188f73a94f2cb8ea0dba3e7cf0a7f
>> Author: David Woodhouse <dwmw2@infradead.org>
>> Date:   Wed Apr 26 12:56:16 2006 +0100
>>
>>     Don't include linux/config.h from anywhere else in include/
>>
>>     Signed-off-by: David Woodhouse <dwmw2@infradead.org> 
> 
> I don't think/know that this goes to stable tree, though.

Thanks; thats helpful and also solves a long-standing puzzlement for me.
(Into my quilt stack it goes)

Next question relates to the port to xtables and friends, and what I
need to do to convert iptables targets.

I notice a lot of things have moved to net/netfilter/xt_*.c

Is this an absolute requirement, or is there some compatability system?

I'm porting the layer7 match first, currently when adding a layer7 rule
I get (dmesg):
[71987.513531] ip_tables: layer7 match: invalid size 0 != 8452

I know where the error is coming from I just wonder if I have to do a
complete conversion now, or a quick fix and fuller conversion later...

thanks

Sam

Re: xtables Re: Hacking iptables 1.3.5 for 2.6.17

[Sam Liddicott]

Amin Azez wrote:
> Yasuyuki KOZAKAI wrote:
>   
>> As you says, netfilter_ipv4.h in 2.6.17.x has unnecessary
>> '#include <config.h>' outside of __KERNEL__. But it seemd to be deleted
>> in recent kernel as follows.
>>
>>     
>>> commit 62c4f0a2d5a188f73a94f2cb8ea0dba3e7cf0a7f
>>> Author: David Woodhouse <dwmw2@infradead.org>
>>> Date:   Wed Apr 26 12:56:16 2006 +0100
>>>
>>>     Don't include linux/config.h from anywhere else in include/
>>>
>>>     Signed-off-by: David Woodhouse <dwmw2@infradead.org> 
>>>       
>> I don't think/know that this goes to stable tree, though.
>>     
>
> Thanks; thats helpful and also solves a long-standing puzzlement for me.
> (Into my quilt stack it goes)
>
> Next question relates to the port to xtables and friends, and what I
> need to do to convert iptables targets.
>
> I notice a lot of things have moved to net/netfilter/xt_*.c
>
> Is this an absolute requirement, or is there some compatability system?
>   
Sorry for this foolish question; I have the answer just by diffing what
is left in net/ipv4/netfilter/ with earlier kernels
please forgive my laziness.

Sam

Re: xtables Re: Hacking iptables 1.3.5 for 2.6.17

[Patrick McHardy]

Amin Azez wrote:
> I notice a lot of things have moved to net/netfilter/xt_*.c
> 
> Is this an absolute requirement, or is there some compatability system?

No requirement, things can just stay where they are.

> I'm porting the layer7 match first, currently when adding a layer7 rule
> I get (dmesg):
> [71987.513531] ip_tables: layer7 match: invalid size 0 != 8452

8452? Thats ridiculous much .. anyway, you need to initialize the
.matchsize field to sizeof(struct ...) (the one shared with userspace).

Re: xtables Re: Hacking iptables 1.3.5 for 2.6.17

[Amin Azez]

Patrick McHardy wrote:
> Amin Azez wrote:
>> I notice a lot of things have moved to net/netfilter/xt_*.c
>>
>> Is this an absolute requirement, or is there some compatability system?
> 
> No requirement, things can just stay where they are.
> 
>> I'm porting the layer7 match first, currently when adding a layer7 rule
>> I get (dmesg):
>> [71987.513531] ip_tables: layer7 match: invalid size 0 != 8452
> 
> 8452? Thats ridiculous much 

too right! Layer7 permits up to 8192 for the regex - like anyone is
going to want to run a regex that long on every packet (well the first
few of each connection)

Whats anyones feelings on flexible sizes of iptables match structs?
Or even on making the kernel size struct bigger?

Right now layer7 does a linear search through a cache of pre-compiled
regexes with the original regex as the key - PER PACKET - it would make
much more sense to store a pointer to the cached regex in the matchinfo
- which is what I'm doing now; but the lack of extra kernelside fields
is restrictive, so I'm canibalizing (in an ugly way) the last few bytes
of the refex field.

>.. anyway, you need to initialize the
> .matchsize field to sizeof(struct ...) (the one shared with userspace).

aye, that works fine, thanks.
To my surprise I'm getting checkinfo called lots of times, I thought it
would only be called once when a new rule was being accepted
kernel-side. Do you know why it would be called many times (with the
same value matchinfo pointer)

Sam

Re: xtables Re: Hacking iptables 1.3.5 for 2.6.17

[Patrick McHardy]

Amin Azez wrote:
> Patrick McHardy wrote:
> 
>>Amin Azez wrote:
>>
>>>I notice a lot of things have moved to net/netfilter/xt_*.c
>>>
>>>Is this an absolute requirement, or is there some compatability system?
>>
>>No requirement, things can just stay where they are.
>>
>>
>>>I'm porting the layer7 match first, currently when adding a layer7 rule
>>>I get (dmesg):
>>>[71987.513531] ip_tables: layer7 match: invalid size 0 != 8452
>>
>>8452? Thats ridiculous much 
> 
> 
> too right! Layer7 permits up to 8192 for the regex - like anyone is
> going to want to run a regex that long on every packet (well the first
> few of each connection)
> 
> Whats anyones feelings on flexible sizes of iptables match structs?

Its not possible with the current architecture, and overcoming this
would probably not be worth it measured in both time and the resulting
uglyness compared to just introducing a netlink interface.

> Right now layer7 does a linear search through a cache of pre-compiled
> regexes with the original regex as the key - PER PACKET - it would make
> much more sense to store a pointer to the cached regex in the matchinfo
> - which is what I'm doing now; but the lack of extra kernelside fields
> is restrictive, so I'm canibalizing (in an ugly way) the last few bytes
> of the refex field.

Thats a bit of a different problem. Per-instance private data will soon
be possible in a nicer way.

>>.. anyway, you need to initialize the
>>.matchsize field to sizeof(struct ...) (the one shared with userspace).
> 
> 
> aye, that works fine, thanks.
> To my surprise I'm getting checkinfo called lots of times, I thought it
> would only be called once when a new rule was being accepted
> kernel-side. Do you know why it would be called many times (with the
> same value matchinfo pointer)

It gets called whenever you add a new rule. Thats a result of the
atomic-exchange of the entire ruleset, each "generation" of your
ruleset (while adding rules incrementally) is an entirely new
ruleset to the kernel.

Re: xtables Re: Hacking iptables 1.3.5 for 2.6.17

[Amin Azez]

Patrick McHardy wrote:
> Amin Azez wrote:
>> To my surprise I'm getting checkinfo called lots of times, I thought it
>> would only be called once when a new rule was being accepted
>> kernel-side. Do you know why it would be called many times (with the
>> same value matchinfo pointer)
> 
> It gets called whenever you add a new rule. Thats a result of the
> atomic-exchange of the entire ruleset, each "generation" of your
> ruleset (while adding rules incrementally) is an entirely new
> ruleset to the kernel.


iptables -Z dsrateentries -t mangle

causes the checkentry to all be called again.
(It just so happens I was doing that every 10 seconds)

I guess iptables -z is not as lightweight as I thought, it perhaps
re-applies all the rules - or maybe checkentry is wrongly being called
from -z?

Or maybe checkentry can do more these days and I need to check why it is
called

Sam

Re: xtables Re: Hacking iptables 1.3.5 for 2.6.17

[Patrick McHardy]

Amin Azez wrote:
> iptables -Z dsrateentries -t mangle
> 
> causes the checkentry to all be called again.
> (It just so happens I was doing that every 10 seconds)
> 
> I guess iptables -z is not as lightweight as I thought, it perhaps
> re-applies all the rules - or maybe checkentry is wrongly being called
> from -z?

I'm actually not sure how this is implemented, I guess it just
dumps the ruleset to userspace, changes counters and pumps
it back to the kernel again (which means it needs to be revalidated).