nftables: named set for ipv4 networks

nftables: named set for ipv4 networks

[Leon Merten Lohse]

Hi,

is there a way to migrate a hash:net type ipset to nftables?
We use this to implement a blacklist where we block a large number of
networks.

I tried using a type ipv4_addr named set as follows:

flush ruleset
table inet filter {
  set blacklist_v4 { type ipv4_addr; }
}
add element inet filter blacklist_v4 { 10.0.0.0/8 }

However, this results in:
Error: Set member cannot be prefix, missing interval flag on declaration
add element inet filter blacklist_v4 { 10.0.0.0/8 }
                                       ^^^^^^^^^^
Is ipv4_addr the wrong type in this case? I could not find any
documentation on it.
Using networks in anonymous sets seems to work flawlessly.

Best,
Leon

System:
# nft --version
nftables v0.6 (Support Edward Snowden)
# uname -a
Linux maracuja 4.7.0-0.bpo.1-amd64 #1 SMP Debian 4.7.5-1~bpo8+2
(2016-10-01) x86_64 GNU/Linux

Re: nftables: named set for ipv4 networks

[Pablo Neira Ayuso]

On Sun, Oct 23, 2016 at 09:38:22PM +0200, Leon Merten Lohse wrote:
> Hi,
> 
> is there a way to migrate a hash:net type ipset to nftables?
> We use this to implement a blacklist where we block a large number of
> networks.
> 
> I tried using a type ipv4_addr named set as follows:
> 
> flush ruleset
> table inet filter {
>   set blacklist_v4 { type ipv4_addr; }
> }
> add element inet filter blacklist_v4 { 10.0.0.0/8 }
> 
> However, this results in:
> Error: Set member cannot be prefix, missing interval flag on declaration
> add element inet filter blacklist_v4 { 10.0.0.0/8 }
>                                        ^^^^^^^^^^
> Is ipv4_addr the wrong type in this case? I could not find any
> documentation on it.
> Using networks in anonymous sets seems to work flawlessly.

Missing "flags interval;" here, so this should be:

        table inet filter {
           set blacklist_v4 { type ipv4_addr; flags interval; }
        }

With named sets, you have to specify this flag since the kernel uses
to select what is the best data structure uses to represent what you
need.

Re: nftables: named set for ipv4 networks

[Leon Merten Lohse]

(repost to list. sorry for the duplicate)

Thank you, Pablo!
Unfortunately this results in a segfault for me. I was able to boil the
ruleset down to a minimal example that segfaults. 

# nft -f segfault.ruleset 
Segmentation fault

Even if I made a severe error, it should not segfault, right? ..

Versions of nftables and kernel are as in my previous emails.

Best 
Leon

---

# segfault.ruleset

flush ruleset

table inet filter {
  # blocked networks
  set blacklist_v4 { type ipv4_addr; flags interval; }
}

# this would be included from a generated file
add element inet filter blacklist_v4 {
192.168.0.1/24,
}



On Thu, 27 Oct 2016 21:23:38 +0200
Pablo Neira Ayuso <pablo@netfilter.org> wrote:

> On Sun, Oct 23, 2016 at 09:38:22PM +0200, Leon Merten Lohse wrote:  
> > Hi,
> > 
> > is there a way to migrate a hash:net type ipset to nftables?
> > We use this to implement a blacklist where we block a large number
> > of networks.
> > 
> > I tried using a type ipv4_addr named set as follows:
> > 
> > flush ruleset
> > table inet filter {
> >   set blacklist_v4 { type ipv4_addr; }
> > }
> > add element inet filter blacklist_v4 { 10.0.0.0/8 }
> > 
> > However, this results in:
> > Error: Set member cannot be prefix, missing interval flag on
> > declaration add element inet filter blacklist_v4 { 10.0.0.0/8 }
> >                                        ^^^^^^^^^^
> > Is ipv4_addr the wrong type in this case? I could not find any
> > documentation on it.
> > Using networks in anonymous sets seems to work flawlessly.    
> 
> Missing "flags interval;" here, so this should be:
> 
>         table inet filter {
>            set blacklist_v4 { type ipv4_addr; flags interval; }
>         }
> 
> With named sets, you have to specify this flag since the kernel uses
> to select what is the best data structure uses to represent what you
> need.
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter"
> in the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html  

Re: nftables: named set for ipv4 networks

[Pablo Neira Ayuso]

On Thu, Oct 27, 2016 at 11:41:52PM +0200, Leon Merten Lohse wrote:
> (repost to list. sorry for the duplicate)
> 
> Thank you, Pablo!
> Unfortunately this results in a segfault for me. I was able to boil the
> ruleset down to a minimal example that segfaults. 
> 
> # nft -f segfault.ruleset 
> Segmentation fault
> 
> Even if I made a severe error, it should not segfault, right? ..
> 
> Versions of nftables and kernel are as in my previous emails.
> 
> Best 
> Leon
> 
> ---
> 
> # segfault.ruleset
> 
> flush ruleset
> 
> table inet filter {
>   # blocked networks
>   set blacklist_v4 { type ipv4_addr; flags interval; }
> }
> 
> # this would be included from a generated file
> add element inet filter blacklist_v4 {
> 192.168.0.1/24,
> }

This is working here.

Your nft version is probably missing this patch:

http://git.netfilter.org/nftables/commit/?id=24c9e993eeff0ed41875aa7efef5dd69db4eb537

Could you please grab a nftables git.netfilter.org clone and confirm
this works for you?

We're planning for the nft 0.7 release, so it would be good if you can
help us confirm that the problem is already gone.

Thanks.

Re: nftables: named set for ipv4 networks

[Leon Merten Lohse]

On Fri, 28 Oct 2016 10:04:19 +0200
Pablo Neira Ayuso <pablo@netfilter.org> wrote:

> Could you please grab a nftables git.netfilter.org clone and confirm
> this works for you?

It seems to work with the latest git version -- no segmentation fault.
Thank you!

However, I stumbled onto the next problem: 
When I load this ruleset twice, it complains that "interval overlaps
with an existing one" even though I explicitly do a "flush ruleset" at
the beginning of the file.
This problem does not occur if I "nft flush ruleset" first and then
load the ruleset.
Do I have to explicitly flush the sets, somehow?

Best
Leon

Re: nftables: named set for ipv4 networks

[Leon Merten Lohse]

I would really appreciate if someone could comment on this.

The problem is that nft complains about overlapping intervals in the
set, if I load the following ruleset twice.
This occurs even though there is a `flush ruleset' directive at the
beginning of the ruleset. As far as I understand it, all sets should be
empty after that.
Calling `nft flush ruleset' beforehand works around this but is no
longer atomic.

Best
Leon

flush ruleset

table inet filter {
  set blacklist_v4 { type ipv4_addr; flags interval; }
}

add element inet filter blacklist_v4 {
192.168.0.1/24,
}

On Fri, 28 Oct 2016 16:23:53 +0200
Leon Merten Lohse <leon@green-side.de> wrote:

> When I load this ruleset twice, it complains that "interval overlaps
> with an existing one" even though I explicitly do a "flush ruleset" at
> the beginning of the file.
> This problem does not occur if I "nft flush ruleset" first and then
> load the ruleset.
> Do I have to explicitly flush the sets, somehow?

Re: nftables: named set for ipv4 networks

[Arturo Borrero Gonzalez]

On 2 November 2016 at 11:30, Leon Merten Lohse <leon@green-side.de> wrote:
> I would really appreciate if someone could comment on this.
>
> The problem is that nft complains about overlapping intervals in the
> set, if I load the following ruleset twice.
> This occurs even though there is a `flush ruleset' directive at the
> beginning of the ruleset. As far as I understand it, all sets should be
> empty after that.
> Calling `nft flush ruleset' beforehand works around this but is no
> longer atomic.
>

Yes, this is probably some bug in the nft cache.

Will send a testcase so we don't forget about fixing this.

Re: nftables: named set for ipv4 networks

[Leon Merten Lohse]

Is there a fix in sight for this? Can I be of assistance somehow?

Best
Leon

On Thu, 3 Nov 2016 12:29:28 +0100
Arturo Borrero Gonzalez <arturo@debian.org> wrote:

> On 2 November 2016 at 11:30, Leon Merten Lohse <leon@green-side.de>
> wrote:
> > I would really appreciate if someone could comment on this.
> >
> > The problem is that nft complains about overlapping intervals in the
> > set, if I load the following ruleset twice.
> > This occurs even though there is a `flush ruleset' directive at the
> > beginning of the ruleset. As far as I understand it, all sets
> > should be empty after that.
> > Calling `nft flush ruleset' beforehand works around this but is no
> > longer atomic.
> >  
> 
> Yes, this is probably some bug in the nft cache.
> 
> Will send a testcase so we don't forget about fixing this.
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter"
> in the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html