Nftables / ipset / multiple tables

Nftables / ipset / multiple tables

[Mark Morgan]

I am trying to convert from iptables to nftables. So far I really like
the way nftables is structured, it will allow me to cleanup my
iptables settings and I expect it to be much more manageable over the
long term.

However, I'm encountering an issue trying to translate an
iptables/ipset feature into nftables in a sane way.

Today I use ipsets, which I can maintain separate from my iptable
firewall rules. This is a nice convenience - I can add blacklists,
manage CIDR block sets, etc, all in an efficient package (ipset),
without having to restart my entire firewall rule set.

As best I can tell, nftables does not support external ipset,
presumably because it has its own internal version. It would be great
if it could support external ipset, then I could have the best of both
worlds.

For now, I converted some of my ipsets to nftables format. The problem
I'm having is that while there is an "include" directive which allows
me to separate out my ipsets into manageable separate files, there is
no mechanism I can see to actually reference these sets. Example:

# nftables.conf
@include "nftables.foobar"
table inet filter { /* stuff in here */ }

# nftables.foobar
table ip foobar { set country_block { /*blah blah*/ } }

Now I want to reference "set country_block" from my inet filters. I
can't find any documentation on how to reference a set in another
table. The "@country_block" reference seems to be scoped to only the
current table, with no way I can find to have it reference a set in
another table. I really do not want to store all of my ip sets in a
single large file/table. That would be worse than what I have today
with iptables.

Am I missing an feature that perhaps isn't documented that would make
this all work, or is there a forth coming feature that might make this
possible to do in the future?

Re: Nftables / ipset / multiple tables

[Leon Merten Lohse]

--repost to list--

As far as I know, named sets are only visible in the current table.

The way I do it is that I create the named set in the table I need it 
and populate it in an included file.
Translated to your example it would look like this:

# nftables.conf
table inet filter {
set country_block { /* set_type ie `type ipv4_addr' */ }
/* more stuff in here */
}
@include "nftables.foobar"


# nftables.foobar
add element inet filter country_block {
/* set elements here */
}


The real downside of this is that it does not allow to access this set 
from another table.
For example, I have a mac based blacklist where port 80 is redirected to 
a portal page.
Normally I would have to know the blocked macs in 3 tables, because I 
want the filter table to be inet but I also need nat chains for the 
dnats, which are not allowed in inet tables. I solved this by marking 
the packets very early in prerouting in the inet table and checking for 
the mark in the nat chains of the ip and ipv6 nat chains.
tl;dr: Currently it is not always feasible to translate old ipset 
approaches to nftables 1 to 1.

Best
Leon

On 2016-12-13 05:39, Mark Morgan wrote:
> I am trying to convert from iptables to nftables. So far I really like
> the way nftables is structured, it will allow me to cleanup my
> iptables settings and I expect it to be much more manageable over the
> long term.
> 
> However, I'm encountering an issue trying to translate an
> iptables/ipset feature into nftables in a sane way.
> 
> Today I use ipsets, which I can maintain separate from my iptable
> firewall rules. This is a nice convenience - I can add blacklists,
> manage CIDR block sets, etc, all in an efficient package (ipset),
> without having to restart my entire firewall rule set.
> 
> As best I can tell, nftables does not support external ipset,
> presumably because it has its own internal version. It would be great
> if it could support external ipset, then I could have the best of both
> worlds.
> 
> For now, I converted some of my ipsets to nftables format. The problem
> I'm having is that while there is an "include" directive which allows
> me to separate out my ipsets into manageable separate files, there is
> no mechanism I can see to actually reference these sets. Example:
> 
> # nftables.conf
> @include "nftables.foobar"
> table inet filter { /* stuff in here */ }
> 
> # nftables.foobar
> table ip foobar { set country_block { /*blah blah*/ } }
> 
> Now I want to reference "set country_block" from my inet filters. I
> can't find any documentation on how to reference a set in another
> table. The "@country_block" reference seems to be scoped to only the
> current table, with no way I can find to have it reference a set in
> another table. I really do not want to store all of my ip sets in a
> single large file/table. That would be worse than what I have today
> with iptables.
> 
> Am I missing an feature that perhaps isn't documented that would make
> this all work, or is there a forth coming feature that might make this
> possible to do in the future?
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Nftables / ipset / multiple tables

[Pablo Neira Ayuso]

On Mon, Dec 12, 2016 at 08:39:47PM -0800, Mark Morgan wrote:
[...]
> For now, I converted some of my ipsets to nftables format. The problem
> I'm having is that while there is an "include" directive which allows
> me to separate out my ipsets into manageable separate files, there is
> no mechanism I can see to actually reference these sets. Example:
> 
> # nftables.conf
> @include "nftables.foobar"
> table inet filter { /* stuff in here */ }
> 
> # nftables.foobar
> table ip foobar { set country_block { /*blah blah*/ } }
> 
> Now I want to reference "set country_block" from my inet filters. I
> can't find any documentation on how to reference a set in another
> table. The "@country_block" reference seems to be scoped to only the
> current table, with no way I can find to have it reference a set in
> another table. I really do not want to store all of my ip sets in a
> single large file/table. That would be worse than what I have today
> with iptables.
> 
> Am I missing an feature that perhaps isn't documented that would make
> this all work, or is there a forth coming feature that might make this
> possible to do in the future?

I'd suggest you use a variable definition, eg.

# cat nftables.conf
include "./nftables.foobar"

table ip filter {
        set country_block {
                type ipv4_addr
                elements = $country_block
        }
}
# cat nftables.foobar
define country_block = { 1.2.3.4, \
                         4.3.2.1 }

Thus you maintain a single file with the address list.

Re: Nftables / ipset / multiple tables

[Mark Morgan]

Great ideas, thank you both. I tried Pablo's suggestion and have an
error. Here's what it looks like:

# nftables.conf
include "/etc/nftables.country-block"
table inet filter {
  set country-block {
    type ipv4_addr; flags interval;
    elements = $country_block_list
  }
}

# nftables.country-block
define country_block_list = {
          1.160.0.0/12, 1.192.0.0/13, 1.200.0.0/16, 1.202.0.0/15, 1.204.0.0/14,
          # .... many lines here
        223.240.0.0/13, 223.4.0.0/14, 223.64.0.0/11
    }

And when I try to reload nftables, I receive the error:

Dec 13 19:56:18 ip-172-31-46-95 nft[563]: /etc/nftables.conf:82:16-16:
Error: syntax error, unexpected '$', expecting '{'
Dec 13 19:56:18 ip-172-31-46-95 nft[563]:     elements = $country_block_list
Dec 13 19:56:18 ip-172-31-46-95 nft[563]:                ^

Did I do something wrong? Versions if it matters (Arch Linux):

> uname -r -m
4.4.36-1-ec2-lts x86_64
> nft --version
nftables v0.6 (Support Edward Snowden)

I tried Leon's suggested approach and it just coredumps:

# nftables.conf
table inet filter {
  set country-block {
    type ipv4_addr; flags interval;
  }
}
include "/etc/nftables.country-block"

# nftables.country-block
add element inet filter country-block {
        1.160.0.0/12, 1.192.0.0/13, 1.200.0.0/16, 1.202.0.0/15, 1.204.0.0/14,
        # ... blah blah
        223.240.0.0/13, 223.4.0.0/14, 223.64.0.0/11
    }

-- Unit nftables.service has begun starting up.
Dec 13 20:00:35 ip-172-31-46-95 kernel: nft[616]: segfault at 70 ip
000000000041d662 sp 00007ffc1b3f8db0 error 4 in nft[400000+56000]
Dec 13 20:00:35 ip-172-31-46-95 systemd[1]: Started Process Core Dump
(PID 618/UID 0).
-- Subject: Unit systemd-coredump@2-618-0.service has finished start-up

Also in case size is a possible issue, here's word count on my
country-block file:

> wc nftables.country-block
  213   952 16532 nftables.country-block

Skipping 10-15 words of meta information, that's approximately ~940
CIDR blocks in the list.

Should I try different versions of nftables or a newer kernel?  Or for
the core dumping issue, feel free to point me at some docs for how to
capture useful debug information for the devs and I'm happy to read up
and submit a bug report.

Re: Nftables / ipset / multiple tables

[Pablo Neira Ayuso]

On Tue, Dec 13, 2016 at 08:06:27PM -0800, Mark Morgan wrote:
> Great ideas, thank you both. I tried Pablo's suggestion and have an
> error. Here's what it looks like:
> 
> # nftables.conf
> include "/etc/nftables.country-block"
> table inet filter {
>   set country-block {
>     type ipv4_addr; flags interval;
>     elements = $country_block_list
>   }
> }
> 
> # nftables.country-block
> define country_block_list = {
>           1.160.0.0/12, 1.192.0.0/13, 1.200.0.0/16, 1.202.0.0/15, 1.204.0.0/14,
>           # .... many lines here
>         223.240.0.0/13, 223.4.0.0/14, 223.64.0.0/11
>     }
> 
> And when I try to reload nftables, I receive the error:
> 
> Dec 13 19:56:18 ip-172-31-46-95 nft[563]: /etc/nftables.conf:82:16-16:
> Error: syntax error, unexpected '$', expecting '{'
> Dec 13 19:56:18 ip-172-31-46-95 nft[563]:     elements = $country_block_list
> Dec 13 19:56:18 ip-172-31-46-95 nft[563]:                ^
> 
> Did I do something wrong? Versions if it matters (Arch Linux):
>
> > uname -r -m
> 4.4.36-1-ec2-lts x86_64
> > nft --version
> nftables v0.6 (Support Edward Snowden)

This is fixed at git.netfilter.org, it would be great if you can test
this and confirm. We're preparing a release soon.

What I sent you works here.

> I tried Leon's suggested approach and it just coredumps:
> 
> # nftables.conf
> table inet filter {
>   set country-block {
>     type ipv4_addr; flags interval;
>   }
> }
> include "/etc/nftables.country-block"
> 
> # nftables.country-block
> add element inet filter country-block {
>         1.160.0.0/12, 1.192.0.0/13, 1.200.0.0/16, 1.202.0.0/15, 1.204.0.0/14,
>         # ... blah blah
>         223.240.0.0/13, 223.4.0.0/14, 223.64.0.0/11
>     }
> 
> -- Unit nftables.service has begun starting up.
> Dec 13 20:00:35 ip-172-31-46-95 kernel: nft[616]: segfault at 70 ip
> 000000000041d662 sp 00007ffc1b3f8db0 error 4 in nft[400000+56000]
> Dec 13 20:00:35 ip-172-31-46-95 systemd[1]: Started Process Core Dump
> (PID 618/UID 0).
> -- Subject: Unit systemd-coredump@2-618-0.service has finished start-up
> 
> Also in case size is a possible issue, here's word count on my
> country-block file:
> 
> > wc nftables.country-block
>   213   952 16532 nftables.country-block
> 
> Skipping 10-15 words of meta information, that's approximately ~940
> CIDR blocks in the list.
> 
> Should I try different versions of nftables or a newer kernel?  Or for
> the core dumping issue, feel free to point me at some docs for how to
> capture useful debug information for the devs and I'm happy to read up
> and submit a bug report.

Yes, try newer libnftnl and nftables at git.netfilter.org and get back
to us if you still have problems.

Re: Nftables / ipset / multiple tables

[Mark Morgan]

I can confirm nft built from source at git.netfilter.org works as
expected for both solutions given in this thread. Thanks!


On Wed, Dec 14, 2016 at 2:15 PM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> On Tue, Dec 13, 2016 at 08:06:27PM -0800, Mark Morgan wrote:
>> Great ideas, thank you both. I tried Pablo's suggestion and have an
>> error. Here's what it looks like:
>>
>> # nftables.conf
>> include "/etc/nftables.country-block"
>> table inet filter {
>>   set country-block {
>>     type ipv4_addr; flags interval;
>>     elements = $country_block_list
>>   }
>> }
>>
>> # nftables.country-block
>> define country_block_list = {
>>           1.160.0.0/12, 1.192.0.0/13, 1.200.0.0/16, 1.202.0.0/15, 1.204.0.0/14,
>>           # .... many lines here
>>         223.240.0.0/13, 223.4.0.0/14, 223.64.0.0/11
>>     }
>>
>> And when I try to reload nftables, I receive the error:
>>
>> Dec 13 19:56:18 ip-172-31-46-95 nft[563]: /etc/nftables.conf:82:16-16:
>> Error: syntax error, unexpected '$', expecting '{'
>> Dec 13 19:56:18 ip-172-31-46-95 nft[563]:     elements = $country_block_list
>> Dec 13 19:56:18 ip-172-31-46-95 nft[563]:                ^
>>
>> Did I do something wrong? Versions if it matters (Arch Linux):
>>
>> > uname -r -m
>> 4.4.36-1-ec2-lts x86_64
>> > nft --version
>> nftables v0.6 (Support Edward Snowden)
>
> This is fixed at git.netfilter.org, it would be great if you can test
> this and confirm. We're preparing a release soon.
>
> What I sent you works here.
>
>> I tried Leon's suggested approach and it just coredumps:
>>
>> # nftables.conf
>> table inet filter {
>>   set country-block {
>>     type ipv4_addr; flags interval;
>>   }
>> }
>> include "/etc/nftables.country-block"
>>
>> # nftables.country-block
>> add element inet filter country-block {
>>         1.160.0.0/12, 1.192.0.0/13, 1.200.0.0/16, 1.202.0.0/15, 1.204.0.0/14,
>>         # ... blah blah
>>         223.240.0.0/13, 223.4.0.0/14, 223.64.0.0/11
>>     }
>>
>> -- Unit nftables.service has begun starting up.
>> Dec 13 20:00:35 ip-172-31-46-95 kernel: nft[616]: segfault at 70 ip
>> 000000000041d662 sp 00007ffc1b3f8db0 error 4 in nft[400000+56000]
>> Dec 13 20:00:35 ip-172-31-46-95 systemd[1]: Started Process Core Dump
>> (PID 618/UID 0).
>> -- Subject: Unit systemd-coredump@2-618-0.service has finished start-up
>>
>> Also in case size is a possible issue, here's word count on my
>> country-block file:
>>
>> > wc nftables.country-block
>>   213   952 16532 nftables.country-block
>>
>> Skipping 10-15 words of meta information, that's approximately ~940
>> CIDR blocks in the list.
>>
>> Should I try different versions of nftables or a newer kernel?  Or for
>> the core dumping issue, feel free to point me at some docs for how to
>> capture useful debug information for the devs and I'm happy to read up
>> and submit a bug report.
>
> Yes, try newer libnftnl and nftables at git.netfilter.org and get back
> to us if you still have problems.