[ANNOUNCE] ipset-4.4 released

[ANNOUNCE] ipset-4.4 released

[Jozsef Kadlecsik]

Hi,

ipset-4.4 has just been released with one important fix and some small 
corrections:

Kernel part changes:
  - The ipporthash, ipportiphash and ipportnethash set types did 
    not work with mixed "src" and "dst" direction parameters of the "set" 
    and  "SET" iptables match and target (reported by Dash Four)
  - Errorneous semaphore handling in error path fixed (reported by 
    Jan Engelhardt, bugzilla id 668) 

Userspace changes:
   - Manpage fix to make it clear how ipset works on setlist type 
     of sets (John Brendler, bugzilla id 640) 

Please note, if you use ipporthash, ipportiphash or ipportnethash type of 
sets with mixed direction parameters of the "set" and "SET" iptables match 
and target, then you must upgrade.

You can download the new version from the usual places:
	http://ipset.netfilter.org
	git://git.netfilter.org/ipset.git

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

ipset-4.4 on 2.6.16.60 kernel

[Mr Dash Four]

I am trying to install ipset on the above kernel version after 
successfully recompiling and installing my iptables v1.3.7.

When I unzip ipset-4.4.tar.bz2, then copy both 
kernel/include/linux/netfilter_ipv4/ip_set.h AND 
kernel/include/linux/netfilter_ipv4/ipt_set.h to 
include/linux/netfilter_ipv4 (the latter was also needed for building 
iptables as well, though the installation page did not mention that as a 
requirement) and then ran make I've got the following error:

=================
  CC [M]  /root/ipset-4.4/kernel/ipt_set.o
/root/ipset-4.4/kernel/ipt_set.c: In function `checkentry':
/root/ipset-4.4/kernel/ipt_set.c:167: warning: implicit declaration of 
function `IPT_ALIGN'
/root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_init':
/root/ipset-4.4/kernel/ipt_set.c:244: warning: passing arg 1 of 
`xt_register_match' makes integer from pointer without a cast
/root/ipset-4.4/kernel/ipt_set.c:244: error: too few arguments to 
function `xt_register_match'
/root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_fini':
/root/ipset-4.4/kernel/ipt_set.c:249: warning: passing arg 1 of 
`xt_unregister_match' makes integer from pointer without a cast
/root/ipset-4.4/kernel/ipt_set.c:249: error: too few arguments to 
function `xt_unregister_match'
make[2]: *** [/root/ipset-4.4/kernel/ipt_set.o] Error 1
make[1]: *** [_module_/root/ipset-4.4/kernel] Error 2
=================

Looking at the source of ipt_set.c I think this is what causes the error:

=================
#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,16)
#include <linux/netfilter_ipv4/ip_tables.h>
#define xt_register_match       ipt_register_match
#define xt_unregister_match     ipt_unregister_match
#define xt_match                ipt_match
#else
#include <linux/netfilter/x_tables.h>
#endif
#include <linux/netfilter_ipv4/ip_set.h>
#include <linux/netfilter_ipv4/ipt_set.h>
.....
static int __init ipt_ipset_init(void)
{
        return xt_register_match(&set_match);
}

static void __exit ipt_ipset_fini(void)
{
        xt_unregister_match(&set_match);
}
=================

I looked at both x_tables.h and ip_tables.h files and they do contain 
the following:

x_tables.h
~~~~~~~~~~
extern int xt_register_match(int af, struct xt_match *target);
extern void xt_unregister_match(int af, struct xt_match *target);

ip_tables.h
~~~~~~~~~~~
#define ipt_register_match(mtch) xt_register_match(AF_INET, mtch)
#define ipt_unregister_match(mtch) xt_unregister_match(AF_INET, mtch)

According to the above as my kernel version appears to be > 2,6,16 
x_tables.h include triggers and it defines the 2 functions, but 
requiring two parameters instead of the one defined in ipt_set.c! I 
tried to change the version to KERNEL_VERSION(2,6,17) to force include 
of netfilter_ipv4/ip_tables.h - that passes, though for this file I get 
the following warning:

=================
  CC [M]  /root/ipset-4.4/kernel/ipt_set.o
/root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_init':
/root/ipset-4.4/kernel/ipt_set.c:244: warning: implicit declaration of 
function `ipt_register_match'
/root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_fini':
/root/ipset-4.4/kernel/ipt_set.c:249: warning: implicit declaration of 
function `ipt_unregister_match'
=================

and then get a similar error for ipt_SET.c:

=================
  CC [M]  /root/ipset-4.4/kernel/ipt_SET.o
/root/ipset-4.4/kernel/ipt_SET.c: In function `checkentry':
/root/ipset-4.4/kernel/ipt_SET.c:157: warning: implicit declaration of 
function `IPT_ALIGN'
/root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_init':
/root/ipset-4.4/kernel/ipt_SET.c:247: warning: passing arg 1 of 
`xt_register_target' makes integer from pointer without a cast
/root/ipset-4.4/kernel/ipt_SET.c:247: error: too few arguments to 
function `xt_register_target'
/root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_fini':
/root/ipset-4.4/kernel/ipt_SET.c:252: warning: passing arg 1 of 
`xt_unregister_target' makes integer from pointer without a cast
/root/ipset-4.4/kernel/ipt_SET.c:252: error: too few arguments to 
function `xt_unregister_target'
make[2]: *** [/root/ipset-4.4/kernel/ipt_SET.o] Error 1
make[1]: *** [_module_/root/ipset-4.4/kernel] Error 2
=================

Trying the same 'trick' for ipt_SET.c won't work though, I am getting this:

=================
  CC [M]  /root/ipset-4.4/kernel/ipt_SET.o
/root/ipset-4.4/kernel/ipt_SET.c:24:1: warning: "XT_CONTINUE" redefined
In file included from include/linux/netfilter_ipv4/ip_tables.h:28,
                 from /root/ipset-4.4/kernel/ipt_SET.c:20:
include/linux/netfilter/x_tables.h:17:1: warning: this is the location 
of the previous definition
/root/ipset-4.4/kernel/ipt_SET.c: In function `target':
/root/ipset-4.4/kernel/ipt_SET.c:94: error: `XT_CONTINUE' undeclared 
(first use in this function)
/root/ipset-4.4/kernel/ipt_SET.c:94: error: (Each undeclared identifier 
is reported only once
/root/ipset-4.4/kernel/ipt_SET.c:94: error: for each function it appears 
in.)
/root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_init':
/root/ipset-4.4/kernel/ipt_SET.c:247: warning: implicit declaration of 
function `ipt_register_target'
/root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_fini':
/root/ipset-4.4/kernel/ipt_SET.c:252: warning: implicit declaration of 
function `ipt_unregister_target'
make[2]: *** [/root/ipset-4.4/kernel/ipt_SET.o] Error 1
make[1]: *** [_module_/root/ipset-4.4/kernel] Error 2
make[1]: Leaving directory 
`/usr/src/expresscore/distrib/build/sources/kernel-runtime/linux-2.6.16.60'
make: *** [modules] Error 2
=================

XT_CONTINUE is defined as IPT_CONTINUE, which, in ip_tables.h is defined 
as ... XT_CONTINUE in ip_tables.h! After a bit more digging I found that 
netfilter/x_tables.h defines XT_CONTINUE as 0xFFFFFFFF, so I tried to 
replace this in ipt_SET.c, but when tried make again I've got this:

=================
  CC [M]  /root/ipset-4.4/kernel/ipt_set.o
/root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_init':
/root/ipset-4.4/kernel/ipt_set.c:244: warning: implicit declaration of 
function `ipt_register_match'
/root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_fini':
/root/ipset-4.4/kernel/ipt_set.c:249: warning: implicit declaration of 
function `ipt_unregister_match'
  CC [M]  /root/ipset-4.4/kernel/ipt_SET.o
/root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_init':
/root/ipset-4.4/kernel/ipt_SET.c:247: warning: implicit declaration of 
function `ipt_register_target'
/root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_fini':
/root/ipset-4.4/kernel/ipt_SET.c:252: warning: implicit declaration of 
function `ipt_unregister_target'

  Building modules, stage 2.
  MODPOST
*** Warning: "ipt_unregister_match" [/root/ipset-4.4/kernel/ipt_set.ko] 
undefined!
*** Warning: "ipt_register_match" [/root/ipset-4.4/kernel/ipt_set.ko] 
undefined!
*** Warning: "ipt_unregister_target" [/root/ipset-4.4/kernel/ipt_SET.ko] 
undefined!
*** Warning: "ipt_register_target" [/root/ipset-4.4/kernel/ipt_SET.ko] 
undefined!
=================

Where I am now stuck and would appreciate a bit of help.

My iptables 1.3.7 compiled and installed successfully (from what I can 
gather it added two additional files in /usr/lib/iptables - 
libipt_set.so and libipt_SET.so), so I don't think this is iptables problem.


> ipset-4.4 has just been released with one important fix and some small 
> corrections:
>
> Kernel part changes:
>   - The ipporthash, ipportiphash and ipportnethash set types did 
>     not work with mixed "src" and "dst" direction parameters of the "set" 
>     and  "SET" iptables match and target (reported by Dash Four)
>   - Errorneous semaphore handling in error path fixed (reported by 
>     Jan Engelhardt, bugzilla id 668) 
>   

Re: ipset-4.4 on 2.6.16.60 kernel

[Jozsef Kadlecsik]

On Sat, 6 Nov 2010, Mr Dash Four wrote:

> I am trying to install ipset on the above kernel version after successfully
> recompiling and installing my iptables v1.3.7.
> 
> When I unzip ipset-4.4.tar.bz2, then copy both
> kernel/include/linux/netfilter_ipv4/ip_set.h AND
> kernel/include/linux/netfilter_ipv4/ipt_set.h to include/linux/netfilter_ipv4
> (the latter was also needed for building iptables as well, though the
> installation page did not mention that as a requirement) and then ran make
> I've got the following error:
> 
> =================
>  CC [M]  /root/ipset-4.4/kernel/ipt_set.o
> /root/ipset-4.4/kernel/ipt_set.c: In function `checkentry':
> /root/ipset-4.4/kernel/ipt_set.c:167: warning: implicit declaration of
> function `IPT_ALIGN'
> /root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_init':
> /root/ipset-4.4/kernel/ipt_set.c:244: warning: passing arg 1 of
> `xt_register_match' makes integer from pointer without a cast
> /root/ipset-4.4/kernel/ipt_set.c:244: error: too few arguments to function
> `xt_register_match'
> /root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_fini':
> /root/ipset-4.4/kernel/ipt_set.c:249: warning: passing arg 1 of
> `xt_unregister_match' makes integer from pointer without a cast
> /root/ipset-4.4/kernel/ipt_set.c:249: error: too few arguments to function
> `xt_unregister_match'
> make[2]: *** [/root/ipset-4.4/kernel/ipt_set.o] Error 1
> make[1]: *** [_module_/root/ipset-4.4/kernel] Error 2
> =================
> 
> Looking at the source of ipt_set.c I think this is what causes the error:
> 
> =================
> #if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,16)
> #include <linux/netfilter_ipv4/ip_tables.h>
> #define xt_register_match       ipt_register_match
> #define xt_unregister_match     ipt_unregister_match
> #define xt_match                ipt_match
> #else
> #include <linux/netfilter/x_tables.h>
> #endif
> #include <linux/netfilter_ipv4/ip_set.h>
> #include <linux/netfilter_ipv4/ipt_set.h>
> .....
> static int __init ipt_ipset_init(void)
> {
>        return xt_register_match(&set_match);
> }
> 
> static void __exit ipt_ipset_fini(void)
> {
>        xt_unregister_match(&set_match);
> }
> =================
> 
> I looked at both x_tables.h and ip_tables.h files and they do contain the
> following:
> 
> x_tables.h
> ~~~~~~~~~~
> extern int xt_register_match(int af, struct xt_match *target);
> extern void xt_unregister_match(int af, struct xt_match *target);
> 
> ip_tables.h
> ~~~~~~~~~~~
> #define ipt_register_match(mtch) xt_register_match(AF_INET, mtch)
> #define ipt_unregister_match(mtch) xt_unregister_match(AF_INET, mtch)
> 
> According to the above as my kernel version appears to be > 2,6,16 x_tables.h
> include triggers and it defines the 2 functions, but requiring two parameters
> instead of the one defined in ipt_set.c! I tried to change the version to
> KERNEL_VERSION(2,6,17) to force include of netfilter_ipv4/ip_tables.h - that
> passes, though for this file I get the following warning:

The problem is that the API changed somewhere along the 2.6.16.x tree and 
it cannot be expressed by

#if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,16)
...

So it is not possible to provide an ipset-4.x source tree which could 
support 2.6.16, 2.6.16.x and 2.6.17 as well.

 
> =================
>  CC [M]  /root/ipset-4.4/kernel/ipt_set.o
> /root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_init':
> /root/ipset-4.4/kernel/ipt_set.c:244: warning: implicit declaration of
> function `ipt_register_match'
> /root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_fini':
> /root/ipset-4.4/kernel/ipt_set.c:249: warning: implicit declaration of
> function `ipt_unregister_match'
> ================

No, you cannot solve the compatibility that way. Please write back the 
kernel version check to KERNEL_VERSION(2,6,16) and manually modify all 
xt_register_match|target, xt_unregister_match|target calls in ipt_set.c 
and ipt_SET.c speficying two arguments, the first one filled out as 
AF_INET, eg:

static int __init ipt_ipset_init(void)
{
       return xt_register_match(AF_INET, &set_match);
}
 
> and then get a similar error for ipt_SET.c:
> 
> =================
>  CC [M]  /root/ipset-4.4/kernel/ipt_SET.o
> /root/ipset-4.4/kernel/ipt_SET.c: In function `checkentry':
> /root/ipset-4.4/kernel/ipt_SET.c:157: warning: implicit declaration of
> function `IPT_ALIGN'
> /root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_init':
> /root/ipset-4.4/kernel/ipt_SET.c:247: warning: passing arg 1 of
> `xt_register_target' makes integer from pointer without a cast
> /root/ipset-4.4/kernel/ipt_SET.c:247: error: too few arguments to function
> `xt_register_target'
> /root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_fini':
> /root/ipset-4.4/kernel/ipt_SET.c:252: warning: passing arg 1 of
> `xt_unregister_target' makes integer from pointer without a cast
> /root/ipset-4.4/kernel/ipt_SET.c:252: error: too few arguments to function
> `xt_unregister_target'
> make[2]: *** [/root/ipset-4.4/kernel/ipt_SET.o] Error 1
> make[1]: *** [_module_/root/ipset-4.4/kernel] Error 2
> =================
> 
> Trying the same 'trick' for ipt_SET.c won't work though, I am getting this:
> 
> =================
>  CC [M]  /root/ipset-4.4/kernel/ipt_SET.o
> /root/ipset-4.4/kernel/ipt_SET.c:24:1: warning: "XT_CONTINUE" redefined
> In file included from include/linux/netfilter_ipv4/ip_tables.h:28,
>                 from /root/ipset-4.4/kernel/ipt_SET.c:20:
> include/linux/netfilter/x_tables.h:17:1: warning: this is the location of the
> previous definition
> /root/ipset-4.4/kernel/ipt_SET.c: In function `target':
> /root/ipset-4.4/kernel/ipt_SET.c:94: error: `XT_CONTINUE' undeclared (first
> use in this function)
> /root/ipset-4.4/kernel/ipt_SET.c:94: error: (Each undeclared identifier is
> reported only once
> /root/ipset-4.4/kernel/ipt_SET.c:94: error: for each function it appears in.)
> /root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_init':
> /root/ipset-4.4/kernel/ipt_SET.c:247: warning: implicit declaration of
> function `ipt_register_target'
> /root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_fini':
> /root/ipset-4.4/kernel/ipt_SET.c:252: warning: implicit declaration of
> function `ipt_unregister_target'
> make[2]: *** [/root/ipset-4.4/kernel/ipt_SET.o] Error 1
> make[1]: *** [_module_/root/ipset-4.4/kernel] Error 2
> make[1]: Leaving directory
> `/usr/src/expresscore/distrib/build/sources/kernel-runtime/linux-2.6.16.60'
> make: *** [modules] Error 2
> =================
> 
> XT_CONTINUE is defined as IPT_CONTINUE, which, in ip_tables.h is defined as
> ... XT_CONTINUE in ip_tables.h! After a bit more digging I found that
> netfilter/x_tables.h defines XT_CONTINUE as 0xFFFFFFFF, so I tried to replace
> this in ipt_SET.c, but when tried make again I've got this:
> 
> =================
>  CC [M]  /root/ipset-4.4/kernel/ipt_set.o
> /root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_init':
> /root/ipset-4.4/kernel/ipt_set.c:244: warning: implicit declaration of
> function `ipt_register_match'
> /root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_fini':
> /root/ipset-4.4/kernel/ipt_set.c:249: warning: implicit declaration of
> function `ipt_unregister_match'
>  CC [M]  /root/ipset-4.4/kernel/ipt_SET.o
> /root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_init':
> /root/ipset-4.4/kernel/ipt_SET.c:247: warning: implicit declaration of
> function `ipt_register_target'
> /root/ipset-4.4/kernel/ipt_SET.c: In function `ipt_SET_fini':
> /root/ipset-4.4/kernel/ipt_SET.c:252: warning: implicit declaration of
> function `ipt_unregister_target'
> 
>  Building modules, stage 2.
>  MODPOST
> *** Warning: "ipt_unregister_match" [/root/ipset-4.4/kernel/ipt_set.ko]
> undefined!
> *** Warning: "ipt_register_match" [/root/ipset-4.4/kernel/ipt_set.ko]
> undefined!
> *** Warning: "ipt_unregister_target" [/root/ipset-4.4/kernel/ipt_SET.ko]
> undefined!
> *** Warning: "ipt_register_target" [/root/ipset-4.4/kernel/ipt_SET.ko]
> undefined!
> =================
> 
> Where I am now stuck and would appreciate a bit of help.
> 
> My iptables 1.3.7 compiled and installed successfully (from what I can gather
> it added two additional files in /usr/lib/iptables - libipt_set.so and
> libipt_SET.so), so I don't think this is iptables problem.
> 
> 
> > ipset-4.4 has just been released with one important fix and some small
> > corrections:
> > 
> > Kernel part changes:
> >   - The ipporthash, ipportiphash and ipportnethash set types did     not
> > work with mixed "src" and "dst" direction parameters of the "set"     and
> > "SET" iptables match and target (reported by Dash Four)
> >   - Errorneous semaphore handling in error path fixed (reported by     Jan
> > Engelhardt, bugzilla id 668)   

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipset-4.4 on 2.6.16.60 kernel

[Mr Dash Four]

>> =================
>>  CC [M]  /root/ipset-4.4/kernel/ipt_set.o
>> /root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_init':
>> /root/ipset-4.4/kernel/ipt_set.c:244: warning: implicit declaration of
>> function `ipt_register_match'
>> /root/ipset-4.4/kernel/ipt_set.c: In function `ipt_ipset_fini':
>> /root/ipset-4.4/kernel/ipt_set.c:249: warning: implicit declaration of
>> function `ipt_unregister_match'
>> ================
>>     
>
> No, you cannot solve the compatibility that way. Please write back the 
> kernel version check to KERNEL_VERSION(2,6,16) and manually modify all 
> xt_register_match|target, xt_unregister_match|target calls in ipt_set.c 
> and ipt_SET.c speficying two arguments, the first one filled out as 
> AF_INET, eg:
>
> static int __init ipt_ipset_init(void)
> {
>        return xt_register_match(AF_INET, &set_match);
> }
>   
That is exactly what I did, and it compiled without error. After that I 
did KERNEL_DIR=... make install and installed the whole thing.

HOWEVER, it still does not work!

After reboot, when I try 'ipset --version' it tells me it does not know 
the kernel version, so I looked at /lib/modules/2.6.16.60/ and saw that 
in a directory called 'extra' there were all the ipset modules sitting. 
So what I did is modprobe all .ko files to load them into the memory. 
lsmod confirmed it that they are loaded without errors.

So, hopeful that I finally cracked it I executed 'iptables -I blacklist 
1 -m set --match-set test dst -j DROP' (I created the treemap called 
'test' prior to that) and got this message:

iptables v1.3.7: Unknown arg `--match-set'
Try `iptables -h' or 'iptables --help' for more information.

Looked in /usr/lib/iptables/ and there are two additional files 
libipt_set.so and libipt_SET.so, which were installed by the newly 
compiled version of iptables so don't know why it does not work!

[SOLVED] Re: ipset-4.4 on 2.6.16.60 kernel

[Mr Dash Four]

> That is exactly what I did, and it compiled without error. After that 
> I did KERNEL_DIR=... make install and installed the whole thing.
>
> HOWEVER, it still does not work!
>
> After reboot, when I try 'ipset --version' it tells me it does not 
> know the kernel version, so I looked at /lib/modules/2.6.16.60/ and 
> saw that in a directory called 'extra' there were all the ipset 
> modules sitting. So what I did is modprobe all .ko files to load them 
> into the memory. lsmod confirmed it that they are loaded without errors.
>
> So, hopeful that I finally cracked it I executed 'iptables -I 
> blacklist 1 -m set --match-set test dst -j DROP' (I created the 
> treemap called 'test' prior to that) and got this message:
>
> iptables v1.3.7: Unknown arg `--match-set'
> Try `iptables -h' or 'iptables --help' for more information.
>
> Looked in /usr/lib/iptables/ and there are two additional files 
> libipt_set.so and libipt_SET.so, which were installed by the newly 
> compiled version of iptables so don't know why it does not work!

Please ignore the above - it has been a long day and I clearly had too 
much on my plate today. IT WORKS! I've just got the syntax wrong - in 
'older' iptables version the syntax is not --match-set, but just --set 
and I completely forgot about this.

One more thing - it would be nice if you could update the iptables 
1.3.7. section on the ipset installation page to state that both 
ip_set.h and ipt_set.h are needed for re-compilation of iptables in 
order to make the whole thing work, so that others like me don't bang 
their heads against the wall in the future.