ipset hash:net,iface - can not add more than 64 interfaces

ipset hash:net,iface - can not add more than 64 interfaces

[Марк Коренберг]

for i in `seq 0 70`; do ip link del dummy$i; done;
for i in `seq 0 70`; do ip link add type dummy; done;
for i in `seq 0 70`; do ipset add qwe 0.0.0.0/0,dummy$i; done;

Reveals the problem. Only 64 records can be added, but there are no
obvious restrictions on that. I s it possible to increase the limit ?

-- 
Segmentation fault

Re: ipset hash:net,iface - can not add more than 64 interfaces

[Jozsef Kadlecsik]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]

Hi,

On Tue, 28 Nov 2023, Марк Коренберг wrote:

> for i in `seq 0 70`; do ip link del dummy$i; done;
> for i in `seq 0 70`; do ip link add type dummy; done;
> for i in `seq 0 70`; do ipset add qwe 0.0.0.0/0,dummy$i; done;
> 
> Reveals the problem. Only 64 records can be added, but there are no
> obvious restrictions on that. I s it possible to increase the limit ?

It is intentional. Such elements can be stored in the same hash bucket 
only and 64 is the max size I'm willing to sacrifice for that. Please 
note, that's a huge number and means linear evaluation, i.e. loosing 
performance.

Best regards,
Jozsef
-- 
E-mail  : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.hu
PGP key : https://wigner.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipset hash:net,iface - can not add more than 64 interfaces

[Марк Коренберг]

Okay, got it.

Is there any options to store interface indices internally (instead of
names) ? i.e. if I renamed an interface, it would also “rename” in
ipset (actually just listing it would resolve indices to current
names). This feature would speed up matching ipset in network stack
because it does not require resolving index to name.

вт, 28 нояб. 2023 г. в 09:48, Jozsef Kadlecsik <kadlec@netfilter.org>:
>
> Hi,
>
> On Tue, 28 Nov 2023, Марк Коренберг wrote:
>
> > for i in `seq 0 70`; do ip link del dummy$i; done;
> > for i in `seq 0 70`; do ip link add type dummy; done;
> > for i in `seq 0 70`; do ipset add qwe 0.0.0.0/0,dummy$i; done;
> >
> > Reveals the problem. Only 64 records can be added, but there are no
> > obvious restrictions on that. I s it possible to increase the limit ?
>
> It is intentional. Such elements can be stored in the same hash bucket
> only and 64 is the max size I'm willing to sacrifice for that. Please
> note, that's a huge number and means linear evaluation, i.e. loosing
> performance.
>
> Best regards,
> Jozsef
> --
> E-mail  : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.hu
> PGP key : https://wigner.hu/~kadlec/pgp_public_key.txt
> Address : Wigner Research Centre for Physics
>           H-1525 Budapest 114, POB. 49, Hungary



-- 
Segmentation fault

Re: ipset hash:net,iface - can not add more than 64 interfaces

[Марк Коренберг]

Actually, I need an ipset that matches against list of interfaces
(without networks associated). Are there any ways ?

вт, 28 нояб. 2023 г. в 09:48, Jozsef Kadlecsik <kadlec@netfilter.org>:
>
> Hi,
>
> On Tue, 28 Nov 2023, Марк Коренберг wrote:
>
> > for i in `seq 0 70`; do ip link del dummy$i; done;
> > for i in `seq 0 70`; do ip link add type dummy; done;
> > for i in `seq 0 70`; do ipset add qwe 0.0.0.0/0,dummy$i; done;
> >
> > Reveals the problem. Only 64 records can be added, but there are no
> > obvious restrictions on that. I s it possible to increase the limit ?
>
> It is intentional. Such elements can be stored in the same hash bucket
> only and 64 is the max size I'm willing to sacrifice for that. Please
> note, that's a huge number and means linear evaluation, i.e. loosing
> performance.
>
> Best regards,
> Jozsef
> --
> E-mail  : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.hu
> PGP key : https://wigner.hu/~kadlec/pgp_public_key.txt
> Address : Wigner Research Centre for Physics
>           H-1525 Budapest 114, POB. 49, Hungary



-- 
Segmentation fault

Re: ipset hash:net,iface - can not add more than 64 interfaces

[Jozsef Kadlecsik]

[-- Attachment #1: Type: text/plain, Size: 1813 bytes --]

On Tue, 28 Nov 2023, Марк Коренберг wrote:

> Is there any options to store interface indices internally (instead of 
> names) ? i.e. if I renamed an interface, it would also “rename” in ipset 
> (actually just listing it would resolve indices to current names). This 
> feature would speed up matching ipset in network stack because it does 
> not require resolving index to name.

No, ipset does not support storing interface indices instead of names.

Best regards,
Jozsef
 
> вт, 28 нояб. 2023 г. в 09:48, Jozsef Kadlecsik <kadlec@netfilter.org>:
> >
> > Hi,
> >
> > On Tue, 28 Nov 2023, Марк Коренберг wrote:
> >
> > > for i in `seq 0 70`; do ip link del dummy$i; done;
> > > for i in `seq 0 70`; do ip link add type dummy; done;
> > > for i in `seq 0 70`; do ipset add qwe 0.0.0.0/0,dummy$i; done;
> > >
> > > Reveals the problem. Only 64 records can be added, but there are no
> > > obvious restrictions on that. I s it possible to increase the limit ?
> >
> > It is intentional. Such elements can be stored in the same hash bucket
> > only and 64 is the max size I'm willing to sacrifice for that. Please
> > note, that's a huge number and means linear evaluation, i.e. loosing
> > performance.
> >
> > Best regards,
> > Jozsef
> > --
> > E-mail  : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.hu
> > PGP key : https://wigner.hu/~kadlec/pgp_public_key.txt
> > Address : Wigner Research Centre for Physics
> >           H-1525 Budapest 114, POB. 49, Hungary
> 
> 
> 
> -- 
> Segmentation fault
> 

-- 
E-mail  : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.hu
PGP key : https://wigner.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipset hash:net,iface - can not add more than 64 interfaces

[Jozsef Kadlecsik]

[-- Attachment #1: Type: text/plain, Size: 1781 bytes --]

On Tue, 28 Nov 2023, Марк Коренберг wrote:

> Actually, I need an ipset that matches against list of interfaces
> (without networks associated). Are there any ways ?

No, that's not possible in ipset either.

However, I'd suggest you to explore nftables where there are no such 
internal limitation than in ipset, supports matching interface indices or 
names and can store just interface names/indices in an nftables set too.

Best regards,
Jozsef 
> вт, 28 нояб. 2023 г. в 09:48, Jozsef Kadlecsik <kadlec@netfilter.org>:
> >
> > Hi,
> >
> > On Tue, 28 Nov 2023, Марк Коренберг wrote:
> >
> > > for i in `seq 0 70`; do ip link del dummy$i; done;
> > > for i in `seq 0 70`; do ip link add type dummy; done;
> > > for i in `seq 0 70`; do ipset add qwe 0.0.0.0/0,dummy$i; done;
> > >
> > > Reveals the problem. Only 64 records can be added, but there are no
> > > obvious restrictions on that. I s it possible to increase the limit ?
> >
> > It is intentional. Such elements can be stored in the same hash bucket
> > only and 64 is the max size I'm willing to sacrifice for that. Please
> > note, that's a huge number and means linear evaluation, i.e. loosing
> > performance.
> >
> > Best regards,
> > Jozsef
> > --
> > E-mail  : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.hu
> > PGP key : https://wigner.hu/~kadlec/pgp_public_key.txt
> > Address : Wigner Research Centre for Physics
> >           H-1525 Budapest 114, POB. 49, Hungary
> 
> 
> 
> -- 
> Segmentation fault
> 

-- 
E-mail  : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.hu
PGP key : https://wigner.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipset hash:net,iface - can not add more than 64 interfaces

[Марк Коренберг]

Thank you for the suggestion. Will benchmark in my conditions.

вт, 28 нояб. 2023 г. в 12:06, Fatih USTA <fatihusta86@gmail.com>:
>
> You can use type of list. Separate 64 elements per set. After that append into list.
>
> ipset create dummy_ifaces list:set
>
> ipset create dummy0_0 hash:net,iface
> ipset create dummy0_1 hash:net,iface
> ipset create dummy0_2 hash:net,iface
>
> ipset add dummy_ifaces dummy0_0
> ipset add dummy_ifaces dummy0_1
> ipset add dummy_ifaces dummy0_2
>
>
> On Tue, Nov 28, 2023, 12:34 Jozsef Kadlecsik <kadlec@netfilter.org> wrote:
>>
>> On Tue, 28 Nov 2023, Марк Коренберг wrote:
>>
>> > Actually, I need an ipset that matches against list of interfaces
>> > (without networks associated). Are there any ways ?
>>
>> No, that's not possible in ipset either.
>>
>> However, I'd suggest you to explore nftables where there are no such
>> internal limitation than in ipset, supports matching interface indices or
>> names and can store just interface names/indices in an nftables set too.
>>
>> Best regards,
>> Jozsef
>> > вт, 28 нояб. 2023 г. в 09:48, Jozsef Kadlecsik <kadlec@netfilter.org>:
>> > >
>> > > Hi,
>> > >
>> > > On Tue, 28 Nov 2023, Марк Коренберг wrote:
>> > >
>> > > > for i in `seq 0 70`; do ip link del dummy$i; done;
>> > > > for i in `seq 0 70`; do ip link add type dummy; done;
>> > > > for i in `seq 0 70`; do ipset add qwe 0.0.0.0/0,dummy$i; done;
>> > > >
>> > > > Reveals the problem. Only 64 records can be added, but there are no
>> > > > obvious restrictions on that. I s it possible to increase the limit ?
>> > >
>> > > It is intentional. Such elements can be stored in the same hash bucket
>> > > only and 64 is the max size I'm willing to sacrifice for that. Please
>> > > note, that's a huge number and means linear evaluation, i.e. loosing
>> > > performance.
>> > >
>> > > Best regards,
>> > > Jozsef
>> > > --
>> > > E-mail  : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.hu
>> > > PGP key : https://wigner.hu/~kadlec/pgp_public_key.txt
>> > > Address : Wigner Research Centre for Physics
>> > >           H-1525 Budapest 114, POB. 49, Hungary
>> >
>> >
>> >
>> > --
>> > Segmentation fault
>> >
>>
>> --
>> E-mail  : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.hu
>> PGP key : https://wigner.hu/~kadlec/pgp_public_key.txt
>> Address : Wigner Research Centre for Physics
>>           H-1525 Budapest 114, POB. 49, Hungary



-- 
Segmentation fault