[xtables-addons][PATCH 0/2] Misc ipset issues

[xtables-addons][PATCH 0/2] Misc ipset issues

[Kevin Locke]

Hello All,

I just wanted to express my thanks for writing/maintaining ipset and
xtables-addons.  I have found them to be very useful.

I ran across a few issues getting ipset to work on my system and
wanted to bring them up here for consideration.

The first is that in 2.6.33-rc1, sk_buff->iff was renamed to
sk_buff->skb_iif, which breaks compilation on kernels going forward.

The second issue is that the setlist module is currently not being
built which results in "ipset v4.1: Unknown set type" when attempting
to create a set of this type (as documented in the man page).  I'm not
sure if this is intentional (if it is, feel free to ignore that
patch), but in my experience it has worked quite well with the
exception of -T not working as expected (or at all AFAICT).

Another issue, for which I did not include a patch, is how automatic
resizing of hash tables is handled.  If I restore a file (created
outside ipset) which contains somewhere near (but less than) 65000
entries which do not hash to unique values I start getting log
messages like the following:

/usr/src/modules/xtables-addons/ipset/ip_set_nethash.c: nethash_retry: rehashing of set setname triggered: hashsize grows from 44319 to 66478
/usr/src/modules/xtables-addons/ipset/ip_set_nethash.c: nethash_retry: rehashing of set setname triggered: hashsize grows from 66478 to 99717

and ipset -R silently fails to restore the rest of the file (returning
exit code 0).  I realize that there may be some code to deal with this
during save (or when adding entries using -A), but it would be very
helpful if the user could be warned about the failure during -R as
well.  As a side note:  One use case is when building a large set it
is significantly (on the order of 1000 times on my test system) faster
to build the list and use -R than individually with -A).

Thanks again for all of your work,
Kevin

P.S.  Please CC me in replies as I am not subscribed to this list.

Kevin Locke (2):
  Rename skb->iif to skb->skb_iif for after 2.6.32
  Build ip_set_setlist.ko module

 extensions/compat_skbuff.h |    5 ++++-
 extensions/ipset/Kbuild    |    2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

[xtables-addons][PATCH 1/2] Rename skb->iif to skb->skb_iif for after 2.6.32

[Kevin Locke]

The rename occurred in 8964be4a9a5ca8cab1219bb046db2f6d1936227c,
which will hit mainline in 2.6.33.

Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
---
 extensions/compat_skbuff.h |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/extensions/compat_skbuff.h b/extensions/compat_skbuff.h
index 2778fb4..df210f5 100644
--- a/extensions/compat_skbuff.h
+++ b/extensions/compat_skbuff.h
@@ -25,9 +25,12 @@ static inline struct rtable *skb_rtable(const struct sk_buff *skb)
 #	define skb_ifindex(skb) \
 		(((skb)->input_dev != NULL) ? (skb)->input_dev->ifindex : 0)
 #	define skb_nfmark(skb) (((struct sk_buff *)(skb))->nfmark)
-#else
+#elif LINUX_VERSION_CODE <= KERNEL_VERSION(2, 6, 32)
 #	define skb_ifindex(skb) (skb)->iif
 #	define skb_nfmark(skb) (((struct sk_buff *)(skb))->mark)
+#else
+#	define skb_ifindex(skb) (skb)->skb_iif
+#	define skb_nfmark(skb) (((struct sk_buff *)(skb))->mark)
 #endif
 
 #ifdef CONFIG_NETWORK_SECMARK
-- 
1.6.5

[xtables-addons][PATCH 2/2] Build ip_set_setlist.ko module

[Kevin Locke]

Not sure if this was overlooked, or if it is not in a suitable state for
general use, but lets give it a shot and see how it works.

Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
---
 extensions/ipset/Kbuild |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/extensions/ipset/Kbuild b/extensions/ipset/Kbuild
index 80ff927..c881b3a 100644
--- a/extensions/ipset/Kbuild
+++ b/extensions/ipset/Kbuild
@@ -3,4 +3,4 @@
 obj-m += ipt_set.o ipt_SET.o
 obj-m += ip_set.o ip_set_ipmap.o ip_set_portmap.o ip_set_macipmap.o
 obj-m += ip_set_iphash.o ip_set_nethash.o ip_set_ipporthash.o
-obj-m += ip_set_iptree.o ip_set_iptreemap.o
+obj-m += ip_set_iptree.o ip_set_iptreemap.o ip_set_setlist.o
-- 
1.6.5

Re: [xtables-addons][PATCH 0/2] Misc ipset issues

[Jozsef Kadlecsik]

Hi,

On Sat, 19 Dec 2009, Kevin Locke wrote:

> I ran across a few issues getting ipset to work on my system and
> wanted to bring them up here for consideration.
> 
> The first is that in 2.6.33-rc1, sk_buff->iff was renamed to
> sk_buff->skb_iif, which breaks compilation on kernels going forward.
> 
> The second issue is that the setlist module is currently not being
> built which results in "ipset v4.1: Unknown set type" when attempting
> to create a set of this type (as documented in the man page).  I'm not
> sure if this is intentional (if it is, feel free to ignore that
> patch), but in my experience it has worked quite well with the
> exception of -T not working as expected (or at all AFAICT).

These are xtables-addons related issues, Jan'll take care of them.
 
> Another issue, for which I did not include a patch, is how automatic
> resizing of hash tables is handled.  If I restore a file (created
> outside ipset) which contains somewhere near (but less than) 65000
> entries which do not hash to unique values I start getting log
> messages like the following:
> 
> /usr/src/modules/xtables-addons/ipset/ip_set_nethash.c: nethash_retry: rehashing of set setname triggered: hashsize grows from 44319 to 66478
> /usr/src/modules/xtables-addons/ipset/ip_set_nethash.c: nethash_retry: rehashing of set setname triggered: hashsize grows from 66478 to 99717
> 
> and ipset -R silently fails to restore the rest of the file (returning
> exit code 0).  I realize that there may be some code to deal with this
> during save (or when adding entries using -A), but it would be very
> helpful if the user could be warned about the failure during -R as
> well. 

You are hitting the internal (and somewhat artifical) limit of max 65535 
entries in a set. When rehashing, the next tried hash size was larger than 
the limit so it failed. It should be reported back, you are right.

> As a side note:  One use case is when building a large set it
> is significantly (on the order of 1000 times on my test system) faster
> to build the list and use -R than individually with -A).

That's normal, restore mode spares a lot of system calls.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: [xtables-addons][PATCH 0/2] Misc ipset issues

[Jan Engelhardt]

On Saturday 2009-12-19 21:50, Kevin Locke wrote:
>
>I just wanted to express my thanks for writing/maintaining ipset and
>xtables-addons.  I have found them to be very useful.
>
>I ran across a few issues getting ipset to work on my system and
>wanted to bring them up here for consideration.
>
>The first is that in 2.6.33-rc1, sk_buff->iff was renamed to
>sk_buff->skb_iif, which breaks compilation on kernels going forward.
>
>The second issue is that the setlist module is currently not being
>built which results in "ipset v4.1: Unknown set type" when attempting
>to create a set of this type (as documented in the man page).  I'm not
>sure if this is intentional (if it is, feel free to ignore that
>patch), but in my experience it has worked quite well with the
>exception of -T not working as expected (or at all AFAICT).

Thanks for the two patches, I applied them.