[ipset] Minor non-blocking "sleep" bugs

[ipset] Minor non-blocking "sleep" bugs

[Ismaël BALLO]

Hi,

I use ipset 2.2.9a on kernel 2.6.19.7 compiled (from kernel.org) with
these options
(iptables 1.3.6)
I have a minor bug (non blocking) when I load ipsets ( it seems it
happens when I have a large numbers of bindings)
I try too with  (256) Maximum number of IP sets       (same behaviour)

<M> IP set support
                            (512) Maximum number of IP sets
                          (1024) Hash size for bindings of IP sets
                          <M>   ipmap set support
                          <M>   macipmap set support
                          <M>   portmap set support
                         <M>   iphash set support
                          <M>   nethash set support
                          <M>   ipporthash set support
                          <M>   iptree set support
                          <M>   set match support
                          <M>   SET target support


and other : What can we chose best parameters for hashsize, probes, resize ?


Mar 23 14:45:10 fwa01 kernel: BUG: sleeping function called from
invalid context at mm/slab.c:3007
Mar 23 14:45:10 fwa01 kernel: in_atomic():1, irqs_disabled():0
Mar 23 14:45:10 fwa01 kernel:  [<c0158e6c>] kmem_cache_alloc+0x1b/0x55
Mar 23 14:45:10 fwa01 kernel:  [<f89a77b8>] ip_set_hash_add+0xe7/0x142 [ip_set]
Mar 23 14:45:10 fwa01 kernel:  [<f89a8680>]
ip_set_sockfn_get+0x9bb/0xac8 [ip_set]
Mar 23 14:45:10 fwa01 kernel:  [<c02fac5c>] _read_unlock_irq+0x5/0x7
Mar 23 14:45:10 fwa01 kernel:  [<c01463bf>]
__do_page_cache_readahead+0x118/0x201
Mar 23 14:45:10 fwa01 kernel:  [<c0115086>] __wake_up+0x32/0x43
Mar 23 14:45:10 fwa01 kernel:  [<c02b2500>] nf_sockopt+0x64/0xe2
Mar 23 14:45:10 fwa01 kernel:  [<c02b259e>] nf_getsockopt+0x20/0x25
Mar 23 14:45:10 fwa01 kernel:  [<c02bcdaf>] ip_getsockopt+0x547/0x581
Mar 23 14:45:10 fwa01 kernel:  [<c016d453>] touch_atime+0x60/0x91
Mar 23 14:45:10 fwa01 kernel:  [<c01414b0>] do_generic_mapping_read+0x414/0x45b
Mar 23 14:45:10 fwa01 kernel:  [<c01431bc>] generic_file_aio_read+0x1a6/0x1ee
Mar 23 14:45:10 fwa01 kernel:  [<c014097a>] file_read_actor+0x0/0xdb
Mar 23 14:45:10 fwa01 kernel:  [<c01cfcfa>] vsnprintf+0x44e/0x48c
Mar 23 14:45:10 fwa01 kernel:  [<c017fb74>] inotify_d_instantiate+0x44/0x72
Mar 23 14:45:10 fwa01 kernel:  [<c016a427>] d_rehash+0x26/0x32
Mar 23 14:45:10 fwa01 kernel:  [<c0292be3>] sock_attach_fd+0x72/0xd5
Mar 23 14:45:10 fwa01 kernel:  [<c02943fe>] sock_common_getsockopt+0x1d/0x22
Mar 23 14:45:10 fwa01 kernel:  [<c02929c4>] sys_getsockopt+0x7d/0x9c
Mar 23 14:45:10 fwa01 kernel:  [<c02941f4>] sys_socketcall+0x22a/0x261
Mar 23 14:45:10 fwa01 kernel:  [<c0102a99>] sysenter_past_esp+0x56/0x79
Mar 23 14:45:10 fwa01 kernel:  [<c02f007b>] xfrm_user_rcv_msg+0x124/0x143
Mar 23 14:45:11 fwa01 kernel:  =======================


ipset -R < ipsets

# Generated by ipset 2.2.9a on Mon Mar 26 17:41:02 2007
-N ADM_SET1 iphash --hashsize 1024 --probes 4 --resize 25
-A ADM_SET1  192.168.50.66
-A ADM_SET1 192.168.50.67
-A ADM_SET1 192.168.50.10
-A ADM_SET1  192.168.50.1
-A ADM_SET1 192.168.50.64
-A ADM_SET1 192.168.50.11
-A ADM_SET1 192.168.50.70
-A ADM_SET1  192.168.50.34
-A ADM_SET1 192.168.50.68
-A ADM_SET1 192.168.50.97
-A ADM_SET1  192.168.50.65
-A ADM_SET1 192.168.50.12
-A ADM_SET1 192.168.50.71
-A ADM_SET1 192.168.50.98
-A ADM_SET1  192.168.50.69
-A ADM_SET1 192.168.50.3
-A ADM_SET1 192.168.50.33
-N ADM_SET2 iphash --hashsize 1024 --probes 4 --resize 25
-A ADM_SET2 192.168.51.26
-A ADM_SET2 192.168.50.45
-B ADM_SET1 192.168.50.98 -b ADM_SET2
-B ADM_SET1  192.168.50.34 -b ADM_SET2
-B ADM_SET1 192.168.50.1 -b ADM_SET2
-B ADM_SET1 192.168.50.64 -b ADM_SET2
-B ADM_SET1  192.168.50.71 -b ADM_SET2
-B ADM_SET1 192.168.50.97 -b ADM_SET2
-B ADM_SET1 192.168.50.3 -b ADM_SET2
-B ADM_SET1  192.168.50.65 -b ADM_SET2
-B ADM_SET1 192.168.50.11 -b ADM_SET2
-B ADM_SET1 192.168.50.69 -b ADM_SET2
-B ADM_SET1  192.168.50.67 -b ADM_SET2
-B ADM_SET1 192.168.50.66 -b ADM_SET2
-B ADM_SET1 192.168.50.33 -b ADM_SET2
-B ADM_SET1  192.168.50.70 -b ADM_SET2
-B ADM_SET1 192.168.50.12 -b ADM_SET2
-B ADM_SET1 192.168.50.68 -b ADM_SET2
-B ADM_SET1  192.168.50.10 -b ADM_SET2
COMMIT
# Completed on Mon Mar 26 17:41:02 2007

Thanks in advance.

Re: [ipset] Minor non-blocking "sleep" bugs

[Jozsef Kadlecsik]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1470 bytes --]

On Tue, 27 Mar 2007, Ismaël BALLO wrote:

> I use ipset 2.2.9a on kernel 2.6.19.7 compiled (from kernel.org) with
> these options
> (iptables 1.3.6)
> I have a minor bug (non blocking) when I load ipsets ( it seems it
> happens when I have a large numbers of bindings)
>
> Mar 23 14:45:10 fwa01 kernel: BUG: sleeping function called from
> invalid context at mm/slab.c:3007
> Mar 23 14:45:10 fwa01 kernel: in_atomic():1, irqs_disabled():0
> Mar 23 14:45:10 fwa01 kernel:  [<c0158e6c>] kmem_cache_alloc+0x1b/0x55
> Mar 23 14:45:10 fwa01 kernel:  [<f89a77b8>] ip_set_hash_add+0xe7/0x142 
> [ip_set]

That's due to a stupid bug of mine in the flag of kmalloc. The fixed 
kernel source can be downloaded from the svn repository or as the 
patch-o-matic-ng-20070328.tar.bz2 snapshot from the ipset webpage. Thank 
you for the bugreport.

> and other : What can we chose best parameters for hashsize, probes, resize ?

There is no golden path: either you pay by memory for speed (i.e. large 
hashsize, small probes, large resize percentage) or reversed. The defaults 
of the iphash and nethash types are towards the other end, i.e. spare with 
the memory requirement and loose thus speed.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
           H-1525 Budapest 114, POB. 49, Hungary

Re: [ipset] Minor non-blocking "sleep" bugs

[Ismaël BALLO]

Hi,
Thanks for the reply Jozsef.

The compilation fails unless you put
u_int32_t  min_ip, max_ip; (instead of  __be32 )
in  KERNEL_DIR/ include/linux/netfilter_ipv4/ipt_iprange.h

But I have an other pb.
When I want to flush and delete all rules.
(after ipset -U :all: :all: ; ipset -F ; ipset -X and iptables -D <on
appropriate rules using sets >)

Sometimes, references stays on some sets.

How can I really destroy them ?

Thanks in advance

2007/3/28, Jozsef Kadlecsik <kadlec@blackhole.kfki.hu>:

> On Tue, 27 Mar 2007, Ismaël BALLO wrote:
>
> > I use ipset 2.2.9a on kernel 2.6.19.7 compiled (from kernel.org) with
> > these options
> > (iptables 1.3.6)
> > I have a minor bug (non blocking) when I load ipsets ( it seems it
> > happens when I have a large numbers of bindings)
> >
> > Mar 23 14:45:10 fwa01 kernel: BUG: sleeping function called from
> > invalid context at mm/slab.c:3007
> > Mar 23 14:45:10 fwa01 kernel: in_atomic():1, irqs_disabled():0
> > Mar 23 14:45:10 fwa01 kernel:  [<c0158e6c>] kmem_cache_alloc+0x1b/0x55
> > Mar 23 14:45:10 fwa01 kernel:  [<f89a77b8>] ip_set_hash_add+0xe7/0x142
> > [ip_set]
>
> That's due to a stupid bug of mine in the flag of kmalloc. The fixed
> kernel source can be downloaded from the svn repository or as the
> patch-o-matic-ng-20070328.tar.bz2 snapshot from the ipset webpage. Thank
> you for the bugreport.
>
> > and other : What can we chose best parameters for hashsize, probes, resize ?
>
> There is no golden path: either you pay by memory for speed (i.e. large
> hashsize, small probes, large resize percentage) or reversed. The defaults
> of the iphash and nethash types are towards the other end, i.e. spare with
> the memory requirement and loose thus speed.
>
> Best regards,
> Jozsef
> -
> E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
> PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
> Address : KFKI Research Institute for Particle and Nuclear Physics
>            H-1525 Budapest 114, POB. 49, Hungary

Re: [ipset] Minor non-blocking "sleep" bugs

[Jozsef Kadlecsik]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 857 bytes --]

Hi,

On Thu, 29 Mar 2007, Ismaël BALLO wrote:

> The compilation fails unless you put
> u_int32_t  min_ip, max_ip; (instead of  __be32 )
> in  KERNEL_DIR/ include/linux/netfilter_ipv4/ipt_iprange.h

That's an independent problem, not related to ipset ;-).

> When I want to flush and delete all rules.
> (after ipset -U :all: :all: ; ipset -F ; ipset -X and iptables -D <on
> appropriate rules using sets >)
>
> Sometimes, references stays on some sets.

The order is important: you cannot destroy a set if you haven't 
deleted previously the iptables rule referencing the set.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
           H-1525 Budapest 114, POB. 49, Hungary

Re: [ipset] Minor non-blocking "sleep" bugs

[Ismaël BALLO]

Jozsef Kadlecsik a écrit :
> Hi,
>
> On Thu, 29 Mar 2007, Ismaël BALLO wrote:
>
>> The compilation fails unless you put
>> u_int32_t  min_ip, max_ip; (instead of  __be32 )
>> in  KERNEL_DIR/ include/linux/netfilter_ipv4/ipt_iprange.h
>
> That's an independent problem, not related to ipset ;-).
Ok.
>
>> When I want to flush and delete all rules.
>> (after ipset -U :all: :all: ; ipset -F ; ipset -X and iptables -D <on
>> appropriate rules using sets >)
>>
>> Sometimes, references stays on some sets.
>
> The order is important: you cannot destroy a set if you haven't 
> deleted previously the iptables rule referencing the set.
>
Sorru,  you're right.( I've written too quick ..)
But the problem is still there

 1 - iptables -D <on all appropriate rules using sets > ...
 2 - ipset -U :all: :all: ; ipset -F ; ipset -X

Sometimes, there still exists references.
How can we see them ?Is there a way to flush them ?
> Best regards,
> Jozsef
> -
> E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
> PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
> Address : KFKI Research Institute for Particle and Nuclear Physics
>           H-1525 Budapest 114, POB. 49, Hungary

Re: [ipset] Minor non-blocking "sleep" bugs

[Jozsef Kadlecsik]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 696 bytes --]

On Thu, 29 Mar 2007, Ismaël BALLO wrote:

> But the problem is still there
>
> 1 - iptables -D <on all appropriate rules using sets > ...
> 2 - ipset -U :all: :all: ; ipset -F ; ipset -X

That should be enough to destroy the sets.

> Sometimes, there still exists references.
> How can we see them ?Is there a way to flush them ?

Send me the example (in private) which triggers the problem
so that I could reproduce it.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
           H-1525 Budapest 114, POB. 49, Hungary