Optimize fails on a large ruleset

Optimize fails on a large ruleset

[sixene]

Hi,
I hope this is the right channel to reach out for support, the wiki 
mentioned this mailing list.
I'm having trouble optimizing my large ruleset of 26000+ lines.
When I run 'nft -c -o -f ruleset.nft', after the while processes, I get 
the following error:
nft: optimize.c:423: merge_verdict_stmts: Assertion `0' failed.
Aborted (core dumped)

My ruleset mostly consists of just 'ip saddr x.x.x.x/xx counter packets 
0 bytes 0 drop'
Hope you can help, as I'm facing very bad performance with this list, 
however I have no choice because I need to block all of these addresses.
Thanks!

(i could not attach the file due to your service's policies, however i 
am using x4bnet's lists_vpn on github)

Optimize fails on a large ruleset

[Sixene]

Hi,
I hope this is the right channel to reach out for support, the wiki
mentioned this mailing list.
I'm having trouble optimizing my large ruleset of 26000+ lines.
When I run 'nft -c -o -f ruleset.nft', after the while processes, I
get the following error:
nft: optimize.c:423: merge_verdict_stmts: Assertion `0' failed.
Aborted (core dumped)

My ruleset mostly consists of just 'ip saddr x.x.x.x/xx counter
packets 0 bytes 0 drop'
Hope you can help, as I'm facing very bad performance with this list,
however I have no choice because I need to block all of these
addresses.
Thanks!

(i could not attach the file due to your service's policies, however i
am using x4bnet's lists_vpn on github)

Re: Optimize fails on a large ruleset

[Pablo Neira Ayuso]

Hi,

On Fri, Nov 17, 2023 at 03:40:09PM +0100, Sixene wrote:
> Hi,
> I hope this is the right channel to reach out for support, the wiki
> mentioned this mailing list.
> I'm having trouble optimizing my large ruleset of 26000+ lines.
> When I run 'nft -c -o -f ruleset.nft', after the while processes, I
> get the following error:
> nft: optimize.c:423: merge_verdict_stmts: Assertion `0' failed.
> Aborted (core dumped)

Please, make sure you you run on latest nftables version.

> My ruleset mostly consists of just 'ip saddr x.x.x.x/xx counter
> packets 0 bytes 0 drop'
> Hope you can help, as I'm facing very bad performance with this list,
> however I have no choice because I need to block all of these
> addresses.

Please, send a simple reproducer.

Thanks.

Re: Optimize fails on a large ruleset

[A L]

---- From: Sixene <notsixene@gmail.com> -- Sent: 2023-11-17 - 15:40 ----

> Hi,
> I hope this is the right channel to reach out for support, the wiki
> mentioned this mailing list.
> I'm having trouble optimizing my large ruleset of 26000+ lines.
> When I run 'nft -c -o -f ruleset.nft', after the while processes, I
> get the following error:
> nft: optimize.c:423: merge_verdict_stmts: Assertion `0' failed.
> Aborted (core dumped)
> 
> My ruleset mostly consists of just 'ip saddr x.x.x.x/xx counter
> packets 0 bytes 0 drop'
> Hope you can help, as I'm facing very bad performance with this list,
> however I have no choice because I need to block all of these
> addresses.
> Thanks!

A workaround would be to use ipset. But it only works with the iptables version netfilter, AFAIK. Ipset is optimised to handle large sets of addresses. 

https://ipset.netfilter.org/


> 
> (i could not attach the file due to your service's policies, however i
> am using x4bnet's lists_vpn on github)

Re: Optimize fails on a large ruleset

[Pablo Neira Ayuso]

[-- Attachment #1: Type: text/plain, Size: 462 bytes --]

On Fri, Nov 17, 2023 at 05:42:59PM +0100, Sixene wrote:
> Hi,
> After checking via dnf, it seems I'm running the latest version already.
> After some investigation I found out I had a lot of duplicate entries,
> after fixing this, I now get the error "Segmentation fault (core
> dumped)" with the same command.

No crash with nftables 1.0.9, what nftables version are you using?

I am attaching the output with your ruleset, running:

nft -c -o -f notsixene.nft

[-- Attachment #2: output.txt --]
[-- Type: text/plain, Size: 3365 bytes --]

Merging:
notsixene.nft:4:9-60:         ip saddr 1.12.32.0/23 counter packets 0 bytes 0 drop
notsixene.nft:5:9-59:         ip saddr 1.14.0.0/15 counter packets 0 bytes 0 drop
notsixene.nft:6:9-60:         ip saddr 1.44.96.0/24 counter packets 0 bytes 0 drop
notsixene.nft:7:9-60:         ip saddr 1.116.0.0/15 counter packets 0 bytes 0 drop
notsixene.nft:8:9-61:         ip saddr 1.178.32.0/19 counter packets 0 bytes 0 drop
notsixene.nft:9:9-60:         ip saddr 1.247.4.0/24 counter packets 0 bytes 0 drop
notsixene.nft:10:9-61:         ip saddr 1.255.30.0/24 counter packets 0 bytes 0 drop
into:
	ip saddr { 1.12.32.0/23, 1.14.0.0/15, 1.44.96.0/24, 1.116.0.0/15, 1.178.32.0/19, 1.247.4.0/24, 1.255.30.0/24 } counter drop
Merging:
notsixene.nft:172:9-57:         tcp dport 9090 ct state { new, untracked } accept
notsixene.nft:173:9-55:         tcp dport 80 ct state { new, untracked } accept
notsixene.nft:174:9-58:         tcp dport 25565 ct state { new, untracked } accept
notsixene.nft:175:9-58:         tcp dport 25566 ct state { new, untracked } accept
into:
	tcp dport . ct state { 9090 . new, 9090 . untracked, 80 . new, 80 . untracked, 25565 . new, 25565 . untracked, 25566 . new, 25566 . untracked } accept
Merging:
notsixene.nft:176:9-58:         udp dport 25565 ct state { new, untracked } accept
notsixene.nft:177:9-58:         udp dport 25566 ct state { new, untracked } accept
into:
	ct state . udp dport { new . 25565, untracked . 25565, new . 25566, untracked . 25566 } accept
Merging:
notsixene.nft:178:9-58:         tcp dport 27015 ct state { new, untracked } accept
notsixene.nft:179:9-56:         tcp dport 443 ct state { new, untracked } accept
notsixene.nft:180:9-57:         tcp dport 8092 ct state { new, untracked } accept
notsixene.nft:181:9-57:         tcp dport 8093 ct state { new, untracked } accept
into:
	tcp dport . ct state { 27015 . new, 27015 . untracked, 443 . new, 443 . untracked, 8092 . new, 8092 . untracked, 8093 . new, 8093 . untracked } accept
Merging:
notsixene.nft:182:9-57:         udp dport 8092 ct state { new, untracked } accept
notsixene.nft:183:9-57:         udp dport 8093 ct state { new, untracked } accept
into:
	ct state . udp dport { new . 8092, untracked . 8092, new . 8093, untracked . 8093 } accept
Merging:
notsixene.nft:184:9-57:         tcp dport 8080 ct state { new, untracked } accept
notsixene.nft:185:9-57:         tcp dport 8181 ct state { new, untracked } accept
notsixene.nft:186:9-57:         tcp dport 4430 ct state { new, untracked } accept
notsixene.nft:187:9-58:         tcp dport 34523 ct state { new, untracked } accept
notsixene.nft:188:9-57:         tcp dport 8000 ct state { new, untracked } accept
notsixene.nft:189:9-57:         tcp dport 8010 ct state { new, untracked } accept
into:
	tcp dport . ct state { 8080 . new, 8080 . untracked, 8181 . new, 8181 . untracked, 4430 . new, 4430 . untracked, 34523 . new, 34523 . untracked, 8000 . new, 8000 . untracked, 8010 . new, 8010 . untracked } accept
Merging:
notsixene.nft:314:9-45:         icmpv6 type nd-neighbor-advert accept
notsixene.nft:315:9-46:         icmpv6 type nd-neighbor-solicit accept
notsixene.nft:316:9-43:         icmpv6 type nd-router-advert accept
notsixene.nft:317:9-38:         icmpv6 type nd-redirect accept
into:
	icmpv6 type { nd-neighbor-advert, nd-neighbor-solicit, nd-router-advert, nd-redirect } accept