Combine ipv4 and ipv6 in a set

Combine ipv4 and ipv6 in a set

[Daniel]

Hi,

nft 1.06 Debian12. Is it possible in a set to combine ipv4 and ipv6 ? If 
not, does it exist another method to do this ?

-- 
Daniel

Re: Combine ipv4 and ipv6 in a set

[Kerin Millar]

On Tue, 30 Jan 2024, at 10:17 AM, Daniel wrote:
> Hi,
>
> nft 1.06 Debian12. Is it possible in a set to combine ipv4 and ipv6 ? If 
> not, does it exist another method to do this ?

Combining is impossible. However, the value of an ipv6_addr element is permitted to be an IPv4-mapped IPv6 address.

-- 
Kerin Millar

Re: Combine ipv4 and ipv6 in a set

[Daniel]

Le 30/01/2024 à 11:39, Kerin Millar a écrit :
> On Tue, 30 Jan 2024, at 10:17 AM, Daniel wrote:
>> Hi,
>>
>> nft 1.06 Debian12. Is it possible in a set to combine ipv4 and ipv6 ? If
>> not, does it exist another method to do this ?
> Combining is impossible. However, the value of an ipv6_addr element is permitted to be an IPv4-mapped IPv6 address.

As simple it is, thanks.

Re: Combine ipv4 and ipv6 in a set

[Daniel]

Le 30/01/2024 à 13:13, Daniel a écrit :
> 
> Le 30/01/2024 à 11:39, Kerin Millar a écrit :
>> On Tue, 30 Jan 2024, at 10:17 AM, Daniel wrote:
>>> Hi,
>>>
>>> nft 1.06 Debian12. Is it possible in a set to combine ipv4 and ipv6 ? If
>>> not, does it exist another method to do this ?
>> Combining is impossible. However, the value of an ipv6_addr element is 
>> permitted to be an IPv4-mapped IPv6 address.
> 
> As simple it is, thanks.

I have a question: if I want to check a source ipv4 address against a 
mapped ipv4, how to configure the rule ? I get a datatype mismatch as 
the set is of type addr_ipv6

add rule inet filter input ip saddr @reaction-banned-ips drop is not 
accepted.

Re: Combine ipv4 and ipv6 in a set

[Marc Haber]

On Tue, Jan 30, 2024 at 10:39:57AM +0000, Kerin Millar wrote:
> On Tue, 30 Jan 2024, at 10:17 AM, Daniel wrote:
> > Hi,
> >
> > nft 1.06 Debian12. Is it possible in a set to combine ipv4 and ipv6 ? If 
> > not, does it exist another method to do this ?
> 
> Combining is impossible.

This is one of my pet peeves with nft, actually. For iptables, there was
tooling like ferm which made it possible to write dual-stack rule sets
very easily. This kind of tooling seems to be completely missing in the
nftables world. Am I missing something here?

> However, the value of an ipv6_addr element is permitted to be an IPv4-mapped IPv6 address.

Does nft have a function to convert an IPv4 address to an IPv4-mapped
address? Will the rule set do the intended thing? Is an ipv6 rule with
an IPv4 mapped address fully equivalent with a proper IPv4 rule?

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Re: Combine ipv4 and ipv6 in a set

[Kim B. Heino]

> This is one of my pet peeves with nft, actually. For iptables, there
> was tooling like ferm which made it possible to write dual-stack rule
> sets very easily. This kind of tooling seems to be completely missing
> in the nftables world. Am I missing something here?

Foomuuri [https://github.com/FoobarOy/foomuuri] makes it easy to write
dual-stack firewall rule set in text format. Not really ferm-like, but
might be what you are looking for?

Full disclousure: I'm the author of Foomuuri.

Re: Combine ipv4 and ipv6 in a set

[Marc Haber]

On Tue, Jan 30, 2024 at 03:22:47PM +0200, Kim B. Heino wrote:
> > This is one of my pet peeves with nft, actually. For iptables, there
> > was tooling like ferm which made it possible to write dual-stack rule
> > sets very easily. This kind of tooling seems to be completely missing
> > in the nftables world. Am I missing something here?
> 
> Foomuuri [https://github.com/FoobarOy/foomuuri] makes it easy to write
> dual-stack firewall rule set in text format. Not really ferm-like, but
> might be what you are looking for?

That looks interesting, but it is an entirely different language that
needs to be learned and understood. I will look into it. Where do the
rules created by Foomuuri end up? Can they be inspected by using nft
show tables?

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Re: Combine ipv4 and ipv6 in a set

[Kim B. Heino]

> > Foomuuri [https://github.com/FoobarOy/foomuuri] makes it easy 
>
> That looks interesting, but it is an entirely different language that
> needs to be learned and understood. I will look into it. Where do the
> rules created by Foomuuri end up? Can they be inspected by using nft
> show tables?

Command "nft list table inet foomuuri" (or "foomuuri list") will show
them. They are also saved to /var/lib/foomuuri/ directory.

You can dry-run it with "foomuuri check" and then take a look at above
directory.

This is probably getting bit off-topic here so we can continue in
Foomuuri's github discussions.

Re: Combine ipv4 and ipv6 in a set

[Kerin Millar]

On Tue, 30 Jan 2024, at 1:08 PM, Marc Haber wrote:
> On Tue, Jan 30, 2024 at 10:39:57AM +0000, Kerin Millar wrote:
>> On Tue, 30 Jan 2024, at 10:17 AM, Daniel wrote:
>> > Hi,
>> >
>> > nft 1.06 Debian12. Is it possible in a set to combine ipv4 and ipv6 ? If 
>> > not, does it exist another method to do this ?
>> 
>> Combining is impossible.
>
> This is one of my pet peeves with nft, actually. For iptables, there was
> tooling like ferm which made it possible to write dual-stack rule sets
> very easily. This kind of tooling seems to be completely missing in the
> nftables world. Am I missing something here?

Quite possibly. Currently, nftables supports:

- mixed rulesets (using tables bearing the "inet" family)
- mixed rules (wherever it makes sense)
- first-class sets of any kind (irrespective of the type of table enclosing them)

Granted, one cannot create a set that is typed in such a way that an element can be either an IPv4 or IPv6 address/interval. Conversely, iptables does not natively support sets at all, though it can integrate with sets that are managed by ipset(8). Now, can an ipset contain addresses of mixed types? No, it cannot.

# ipset create myset hash:ip
# ipset add myset 127.0.0.1
# ipset add myset ::1
ipset v7.19: Syntax error: cannot parse ::1: resolving to IPv4 address failed

# ipset destroy myset
# ipset create myset hash:ip family inet6
# ipset add myset ::1
# ipset add myset 127.0.0.1
ipset v7.19: Syntax error: cannot parse 127.0.0.1: resolving to IPv6 address failed

As far as the present topic is concerned, the only tangible advantage that ipset has is the ability to create a set whose sole purpose is to act as a superset of other - potentially mixed - sets. This advantage is rather diminished by the fact that one also has to two manage two entirely separate rulesets with iptables and ip6tables, notwithstanding that wrappers such as fermi exist. At any rate, the follow nftables ruleset is valid.

table inet filter {
        set block4 {
                type ipv4_addr
        }
        set block6 {
                type ipv6_addr
        }
        chain INPUT {
                type filter hook input priority filter; policy accept
                ip saddr @block4 drop
                ip6 saddr @block6 drop
        }
}

>
>> However, the value of an ipv6_addr element is permitted to be an IPv4-mapped IPv6 address.
>
> Does nft have a function to convert an IPv4 address to an IPv4-mapped
> address? Will the rule set do the intended thing? Is an ipv6 rule with
> an IPv4 mapped address fully equivalent with a proper IPv4 rule?

I do not know, as I have not yet attempted to use them in an ipv6_addr set (it would waste memory). That said, my expectation would be that they have to be specified in the appropriate format and that they would only be applicable to dual-stack applications. In that case, they might sometimes prove helpful, particularly as Linux defaults to having the "net.ipv6.bindv6only" sysctl be set to "0".

-- 
Kerin Millar

Re: Combine ipv4 and ipv6 in a set

[Slavko]

Dňa 30. januára 2024 15:17:32 UTC používateľ Kerin Millar <kfm@plushkava.net> napísal:

>Granted, one cannot create a set that is typed in such a way that an element can be either an IPv4 or IPv6 address/interval. 

Nowadays IPv6 becomes more and more common. While allmost all
can stay on IP(v4) only host, not all can use IPv6 only host (yet, as many
services are still IPv4 only). In other words, many will have dual stack,
to can access (or be accessible for) all and they will need dual stack FW,
and IMO will need it for many years.

Having separate support for IPv4 and IPv6 was acceptable at time, when
ip6tables was born, but nowadays IMO firewall cannot be named modern,
if any of its part separates that. And any argument (memory, complexity,
etc) against it is pointles, as dual stacks are (and will be) here.

Nftables now has inet family, that is great step from iptables. But still
requires to maintain separate rules in it for anything with network layer
address, eg. mentioned sets (and for icmp/icmp6 too). I hope, that it
is temporary state only and will be improved soon.

regards


-- 
Slavko
https://www.slavino.sk/

Re: Combine ipv4 and ipv6 in a set

[Kerin Millar]

On Tue, 30 Jan 2024, at 5:00 PM, Slavko wrote:
> Dňa 30. januára 2024 15:17:32 UTC používateľ Kerin Millar 
> <kfm@plushkava.net> napísal:
>
>>Granted, one cannot create a set that is typed in such a way that an element can be either an IPv4 or IPv6 address/interval. 
>
> Nowadays IPv6 becomes more and more common. While allmost all
> can stay on IP(v4) only host, not all can use IPv6 only host (yet, as many
> services are still IPv4 only). In other words, many will have dual stack,
> to can access (or be accessible for) all and they will need dual stack FW,
> and IMO will need it for many years.
>
> Having separate support for IPv4 and IPv6 was acceptable at time, when
> ip6tables was born, but nowadays IMO firewall cannot be named modern,
> if any of its part separates that. And any argument (memory, complexity,
> etc) against it is pointles, as dual stacks are (and will be) here.

It is easy to say that it is pointless if not the one to be responsible for implementing and maintaining the code and trying to take into account diverse - and occasionally conflicting - user desires. There have been various set-related bugs in nftables over the years. Complexity surely matters to someone. There are multiple open bugs now that concern both performance and memory usage. Efficiency surely matters to someone.

>
> Nftables now has inet family, that is great step from iptables. But still
> requires to maintain separate rules in it for anything with network layer
> address, eg. mentioned sets (and for icmp/icmp6 too). I hope, that it
> is temporary state only and will be improved soon.

For it to improve, you could put forward a concrete suggestion as to how it might be improved, be it supporting logical disjunctions in rules, supporting a generic address type in sets or whatever else. That would, at least, be a step along the road to (potentially) convincing whoever is going to do the work that it is justified.

On my part, and despite having been a user of nftables for many years now, I would prefer to see its QA and documentation improve ahead of - though not wholly at the expense of - new features being added.

--
Kerin Millar

Re: Combine ipv4 and ipv6 in a set

[Slavko]

Dňa 30. januára 2024 17:57:21 UTC používateľ Kerin Millar <kfm@plushkava.net> napísal:

>It is easy to say that it is pointless if not the one to be responsible for implementing and maintaining the code and trying to take into account diverse - and occasionally conflicting - user desires. There have been various set-related bugs in nftables over the years. Complexity surely matters to someone. There are multiple open bugs now that concern both performance and memory usage. Efficiency surely matters to someone.

Please, aproximate my English, it is far from good...

I mean mostly that, that IPv6 becomes reality, and soon or later (the later
is IMO more realistic) will replace IPv4 at all, thus all L3 addresses will
become 128 bites long. Sure, here will be always penalty in using 128b
addresses in compare to 32b addresses, but world is (slowly) moving
to IPv6, thus that penalty is something, which cannot be avoided.

Debate these penalties sounds for me exactly as debate when 64b
OSs started. I remember "arguments" in that time -- twice of memory
consumption, twice of binary sizes, etc... While all these arguments
are (or can be) right, they are pointless, as IPv6 is here and is not
avoidable in future and we all should to learn to live with that, despite
if we want or don't want it.

>For it to improve, you could put forward a concrete suggestion as to how it might be improved, be it supporting logical disjunctions in rules, supporting a generic address type in sets or whatever else.

I am not able to provide real suggestions, as i don't speak C and i
have minimal knowledge about netfilter/nftables/kernel internals,
thus i can be totally wrong.

Anyway, i  have some ideas from user (admin) point of view, of
course. I can imagine this:

+ sets will store protocol agnostic L3 addresses and it doesn't
  matter (for me) if IPv4 will be mapped to IPv6 or these L3 sets
  will internally maintain two sets -- one for IPv4 and one for IPv6
+ nftables syntax will provide some sort of abstraction, as it already
  provides eg. for UDP/TCP ports (th dport), eg. "nh saddr" (nh as
  network header)
+ nftables will provide abstraction for ICMP(v6) types, something
  as it already provides for reject, to one can build common rule
  for both, including the one common set/verdict map -- for me
  it doesn't matter if it will internally map namex to protocol's proper
  values, or specific names must be prefixed eg. by ipv4 and/or ipv6

> That would, at least, be a step along the road to (potentially) convincing whoever is going to do the work that it is justified.

Yes, that was initial idea of my reply, we all (users, admins, devs)
have to provide own point of view and find best possible solution.

>On my part, and despite having been a user of nftables for many years now, I would prefer to see its QA and documentation improve ahead of - though not wholly at the expense of - new features being added.

While i played with nftables for years, i use it only on one of
my servers yet, and even this machine uses mix of iptables
and nftables rules (the real blocklists are still using ipset),
the ip(6)tables is used only to drop traffic, all other logic is
done from nftables. Nothing extra, just slightly more than
basic firewall, it uses sets to limit (per IP/net) logging,
connections, dynamic blocking, and every used set must be
doubled...

My nft sets doesn't contain huge amount of addresses, as long
time blocking is done by ipsets, with (semi) persistent and from
fail2ban (up some tens of days).

I implemented it about year ago, and yes, the documentation
is hard to read. Most of it is ip family only, some parts are
hard to understand, and even more hard to combine together.
But i cannot be sure, if it is not by lack of my English... But
my feel from docs is, that they are exactly as my notes --
perfect for me, but hard to understand for all others ;-)

I found as very useful examples from announces in this ML's
archive. IMO these announces should be copied to (or be
linked from) the wiki.

regards


-- 
Slavko
https://www.slavino.sk/

Re: Combine ipv4 and ipv6 in a set

[Kerin Millar]

Hi Slavko,

On Tue, 30 Jan 2024 19:34:30 +0000
Slavko <linux@slavino.sk> wrote:

> Dňa 30. januára 2024 17:57:21 UTC používateľ Kerin Millar <kfm@plushkava.net> napísal:
> 
> >It is easy to say that it is pointless if not the one to be responsible for implementing and maintaining the code and trying to take into account diverse - and occasionally conflicting - user desires. There have been various set-related bugs in nftables over the years. Complexity surely matters to someone. There are multiple open bugs now that concern both performance and memory usage. Efficiency surely matters to someone.
> 
> Please, aproximate my English, it is far from good...
> 
> I mean mostly that, that IPv6 becomes reality, and soon or later (the later
> is IMO more realistic) will replace IPv4 at all, thus all L3 addresses will
> become 128 bites long. Sure, here will be always penalty in using 128b
> addresses in compare to 32b addresses, but world is (slowly) moving
> to IPv6, thus that penalty is something, which cannot be avoided.
> 
> Debate these penalties sounds for me exactly as debate when 64b
> OSs started. I remember "arguments" in that time -- twice of memory
> consumption, twice of binary sizes, etc... While all these arguments
> are (or can be) right, they are pointless, as IPv6 is here and is not
> avoidable in future and we all should to learn to live with that, despite
> if we want or don't want it.

I see. Fair enough.

> 
> >For it to improve, you could put forward a concrete suggestion as to how it might be improved, be it supporting logical disjunctions in rules, supporting a generic address type in sets or whatever else.
> 
> I am not able to provide real suggestions, as i don't speak C and i
> have minimal knowledge about netfilter/nftables/kernel internals,
> thus i can be totally wrong.
> 
> Anyway, i  have some ideas from user (admin) point of view, of
> course. I can imagine this:
> 
> + sets will store protocol agnostic L3 addresses and it doesn't
>   matter (for me) if IPv4 will be mapped to IPv6 or these L3 sets
>   will internally maintain two sets -- one for IPv4 and one for IPv6

Now that you mention it, I thought that I recalled someone filing a related issue at bugzilla but it seems that my memory is playing tricks on me. I can find no such issue now, although there is one that (sort of) proposed an equivalent to the list:set type of ipset. That one was rejected.

> + nftables syntax will provide some sort of abstraction, as it already
>   provides eg. for UDP/TCP ports (th dport), eg. "nh saddr" (nh as
>   network header)

It would require some new syntax and/or grammar, indeed.

> + nftables will provide abstraction for ICMP(v6) types, something
>   as it already provides for reject, to one can build common rule
>   for both, including the one common set/verdict map -- for me
>   it doesn't matter if it will internally map namex to protocol's proper
>   values, or specific names must be prefixed eg. by ipv4 and/or ipv6

I could see that working well enough for types that both protocols have in common, such as echo-request.

> 
> > That would, at least, be a step along the road to (potentially) convincing whoever is going to do the work that it is justified.
> 
> Yes, that was initial idea of my reply, we all (users, admins, devs)
> have to provide own point of view and find best possible solution.

Yes, absolutely.

> 
> >On my part, and despite having been a user of nftables for many years now, I would prefer to see its QA and documentation improve ahead of - though not wholly at the expense of - new features being added.
> 
> While i played with nftables for years, i use it only on one of
> my servers yet, and even this machine uses mix of iptables
> and nftables rules (the real blocklists are still using ipset),
> the ip(6)tables is used only to drop traffic, all other logic is
> done from nftables. Nothing extra, just slightly more than
> basic firewall, it uses sets to limit (per IP/net) logging,
> connections, dynamic blocking, and every used set must be
> doubled...
> 
> My nft sets doesn't contain huge amount of addresses, as long
> time blocking is done by ipsets, with (semi) persistent and from
> fail2ban (up some tens of days).
> 
> I implemented it about year ago, and yes, the documentation
> is hard to read. Most of it is ip family only, some parts are
> hard to understand, and even more hard to combine together.
> But i cannot be sure, if it is not by lack of my English... But
> my feel from docs is, that they are exactly as my notes --
> perfect for me, but hard to understand for all others ;-)

That seems a fair assessment to me. While iptables has the advantage of being simpler in many ways, the iptables(8) man page probably does a better job of on-boarding new users to the underlying concepts. Rusty Russell's guides and HOWTOs were also rather good in their day. Technical writing is a talent unto itself and some people are just naturally good at it (no offense intended to the current team, of course).

> 
> I found as very useful examples from announces in this ML's
> archive. IMO these announces should be copied to (or be
> linked from) the wiki.

I agree. We (this includes myself) could be making more of an effort to do this.

-- 
Kerin Millar

Re: Combine ipv4 and ipv6 in a set

[Marc Haber]

On Tue, Jan 30, 2024 at 10:05:48PM +0000, Kerin Millar wrote:
> On Tue, 30 Jan 2024 19:34:30 +0000
> Slavko <linux@slavino.sk> wrote:
> > + sets will store protocol agnostic L3 addresses and it doesn't
> >   matter (for me) if IPv4 will be mapped to IPv6 or these L3 sets
> >   will internally maintain two sets -- one for IPv4 and one for IPv6
> 
> Now that you mention it, I thought that I recalled someone filing a related issue at bugzilla but it seems that my memory is playing tricks on me. I can find no such issue now, although there is one that (sort of) proposed an equivalent to the list:set type of ipset. That one was rejected.

For what it's worth, I had a look in bugzilla regarding easy rule
writing two years or so ago, found a number of related bugs, none of
them with a visible reaction, and decided to spend my time with more
productive tasks. Sadly, I didn't take bookmarks, and looking again
earlier this week didn't show any results. I might have looked
differently back then

> That seems a fair assessment to me. While iptables has the advantage of being simpler in many ways, the iptables(8) man page probably does a better job of on-boarding new users to the underlying concepts. Rusty Russell's guides and HOWTOs were also rather good in their day. Technical writing is a talent unto itself and some people are just naturally good at it (no offense intended to the current team, of course).

Docs in Linux netfilter have been a problem in the last ten or so years.
I remember myself suddenly in a situation of SIP no longer going through
my (unchanged) iptables firewall after a kernel update to find a blog
entry saying that you now need to have a rule sending a packet to the
conntrack/nat helper explicitly for the helpers to work. I didn't find
any place in the official docs where this major change was mentioned.

I am sure that many of the new, nifty features of iptables, netfilter,
nft remain grossly unused because they're not mentioned in rusty's
documents which are still the gold standard of Linux firewall
documentation.

Isn't that sad?

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Re: Combine ipv4 and ipv6 in a set

[Marc Haber]

On Tue, Jan 30, 2024 at 07:34:30PM +0000, Slavko wrote:
> Dňa 30. januára 2024 17:57:21 UTC používateľ Kerin Millar <kfm@plushkava.net> napísal:
> 
> >It is easy to say that it is pointless if not the one to be responsible for implementing and maintaining the code and trying to take into account diverse - and occasionally conflicting - user desires. There have been various set-related bugs in nftables over the years. Complexity surely matters to someone. There are multiple open bugs now that concern both performance and memory usage. Efficiency surely matters to someone.
> 
> I mean mostly that, that IPv6 becomes reality, and soon or later (the later
> is IMO more realistic) will replace IPv4 at all, thus all L3 addresses will
> become 128 bites long. Sure, here will be always penalty in using 128b
> addresses in compare to 32b addresses, but world is (slowly) moving
> to IPv6, thus that penalty is something, which cannot be avoided.

IPv4 and IPv6 are distinct protocols that should be handled differently
from each other in the low level parts of our systems. It is the
responsibility of the high-level tools to make handling this easier for
the human user. In the nftables architecture, that would be nft,
allowing the user to freely use IPv4 and IPv6 addresses in the rule set
and translate this for the low-level parts of the architecture where
efficiency matters.

A IPv6 has longer addresses, yes. But IPv6 also doesn't need NAT and
other crutches that are efficiency killers in other parts of our
systems. And it is also easier to use for the human user ONCE we have
learned to live in a world without the kludges that we currently need to
keep IPv4 alive.

> + sets will store protocol agnostic L3 addresses and it doesn't
>   matter (for me) if IPv4 will be mapped to IPv6 or these L3 sets
>   will internally maintain two sets -- one for IPv4 and one for IPv6
> + nftables syntax will provide some sort of abstraction, as it already
>   provides eg. for UDP/TCP ports (th dport), eg. "nh saddr" (nh as
>   network header)

Agreed.

> + nftables will provide abstraction for ICMP(v6) types, something
>   as it already provides for reject, to one can build common rule
>   for both, including the one common set/verdict map -- for me
>   it doesn't matter if it will internally map namex to protocol's proper
>   values, or specific names must be prefixed eg. by ipv4 and/or ipv6

I'd rather not do this, at least not in the beginning. ICMPv4 and ICMPv6
are distinct protocols, especially when looking at neighbor discovery.
Having a pseudo-protocol ICMPv46 will probably make it easier to handle
things like echo, echo reply, host unreachable etc, but one will need
different handling of ICMPv4 and ICMPv6. It was hard enough to teach
firewall people that IPv6 will cease working if you don't alow ICMPv6 at
all.

> >On my part, and despite having been a user of nftables for many years now, I would prefer to see its QA and documentation improve ahead of - though not wholly at the expense of - new features being added.
> 
> While i played with nftables for years, i use it only on one of
> my servers yet, and even this machine uses mix of iptables
> and nftables rules (the real blocklists are still using ipset),
> the ip(6)tables is used only to drop traffic, all other logic is
> done from nftables. Nothing extra, just slightly more than
> basic firewall, it uses sets to limit (per IP/net) logging,
> connections, dynamic blocking, and every used set must be
> doubled...
> 
> My nft sets doesn't contain huge amount of addresses, as long
> time blocking is done by ipsets, with (semi) persistent and from
> fail2ban (up some tens of days).

I am using nftables also just on one machine, while my bigger
installations still use iptables(-legacy) and ferm (which precludes me
from using ipset because ferm doesn't support that).

And I find writing nftables rules not fun, maybe because the syntax is
so similiar-but-still-different from ferm (which is something that nft
should NOT change, I need to adapt here) and/or because it's so clumsy
to write dual-stack rulesets.

> I implemented it about year ago, and yes, the documentation
> is hard to read. Most of it is ip family only, some parts are
> hard to understand, and even more hard to combine together.
> But i cannot be sure, if it is not by lack of my English... But
> my feel from docs is, that they are exactly as my notes --
> perfect for me, but hard to understand for all others ;-)

The docs, manpages and the wiki are in desperate need of a technical
writer, who will provide a more educational approach. I think that the
wiki is helpful if you already know your way around nftables, but at the
moment it looks to me that having in-depth knowledge of iptables is more
a liability than an asset when trying to move to nftables.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Re: Combine ipv4 and ipv6 in a set

[Marc Haber]

Hi,

On Tue, Jan 30, 2024 at 05:57:21PM +0000, Kerin Millar wrote:
> It is easy to say that it is pointless if not the one to be responsible for implementing and maintaining the code and trying to take into account diverse - and occasionally conflicting - user desires. There have been various set-related bugs in nftables over the years. Complexity surely matters to someone. There are multiple open bugs now that concern both performance and memory usage. Efficiency surely matters to someone.

That responsibility should have been considered before taking the
decision to dump iptables. iptables is used everywhere and elsewhere,
while nftables is the new kid on the block. Currently, changing over
from iptables to nftables needs not only dropping all your know-how, but
also dropping nearly all of your available tooling.

This is a considerable step backwards regarding ease of use, and not
having an easy way to write dual-stack rules hits the early adopters who
have been using IPv6 for decades. Those would also be the early adopters
for nftables, but making their lives harder in writing firewall rules
will prevent nftables migrations.

> > Nftables now has inet family, that is great step from iptables. But still
> > requires to maintain separate rules in it for anything with network layer
> > address, eg. mentioned sets (and for icmp/icmp6 too). I hope, that it
> > is temporary state only and will be improved soon.
> 
> For it to improve, you could put forward a concrete suggestion as to how it might be improved, be it supporting logical disjunctions in rules, supporting a generic address type in sets or whatever else. That would, at least, be a step along the road to (potentially) convincing whoever is going to do the work that it is justified.

My suggestion would be to allow IPv4 and IPv6 addresses mixed in
definition and sets, and disregard invalid combinations in rule
generation. At least that's what existing iptables tools do, and
emulating this behavior would probably ease the transition.

I am open to other discussions, but I doubt that I would be helpful
because I don't know enough about nftables at the moment.

> On my part, and despite having been a user of nftables for many years
> now, I would prefer to see its QA and documentation improve ahead of -
> though not wholly at the expense of - new features being added.

QA and documentation is important, of course.

Greetings
Marc


-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Re: Combine ipv4 and ipv6 in a set

[Marc Haber]

On Tue, Jan 30, 2024 at 05:00:50PM +0000, Slavko wrote:
> Dňa 30. januára 2024 15:17:32 UTC používateľ Kerin Millar <kfm@plushkava.net> napísal:
> 
> >Granted, one cannot create a set that is typed in such a way that an element can be either an IPv4 or IPv6 address/interval. 
> 
> Nowadays IPv6 becomes more and more common. While allmost all
> can stay on IP(v4) only host, not all can use IPv6 only host (yet, as many
> services are still IPv4 only). In other words, many will have dual stack,
> to can access (or be accessible for) all and they will need dual stack FW,
> and IMO will need it for many years.

The correct way to do it would be to develop/install for IPv6 only TODAY
and use a transition technology (NAT64/DNS64 for example for client
access, and reverse proxies for services) to support IPv4 as a legacy
protocol.

> Having separate support for IPv4 and IPv6 was acceptable at time, when
> ip6tables was born, but nowadays IMO firewall cannot be named modern,
> if any of its part separates that. And any argument (memory, complexity,
> etc) against it is pointles, as dual stacks are (and will be) here.

I find it ok to have separated rule set for IPv4 and IPv6. They are
different protocols in the first place. But it should be possible to
hide this separation from the person writing the rules, making it
possible to have a definition for a dual-stack host, creating dual-stack
rules without even noticing that.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Re: Combine ipv4 and ipv6 in a set

[Marc Haber]

On Tue, Jan 30, 2024 at 03:17:32PM +0000, Kerin Millar wrote:
> On Tue, 30 Jan 2024, at 1:08 PM, Marc Haber wrote:
> > This is one of my pet peeves with nft, actually. For iptables, there was
> > tooling like ferm which made it possible to write dual-stack rule sets
> > very easily. This kind of tooling seems to be completely missing in the
> > nftables world. Am I missing something here?
> 
> Quite possibly. Currently, nftables supports:
> 
> - mixed rulesets (using tables bearing the "inet" family)
> - mixed rules (wherever it makes sense)
> - first-class sets of any kind (irrespective of the type of table enclosing them)

That nicely shows another issue with nft: Docs. Neither the search
function on the nftables wiki nor the nft manual page have matches for
"mixed" or "first-class".

How would I use a mixed ruleset? How would I use a mixed rule?

> Granted, one cannot create a set that is typed in such a way that an
> element can be either an IPv4 or IPv6 address/interval. Conversely,
> iptables does not natively support sets at all, though it can
> integrate with sets that are managed by ipset(8). Now, can an ipset
> contain addresses of mixed types? No, it cannot.

In ferm, a tool that has been available for over 20 years, you can
(beginning ten years ago, I think) write things like:

@def $h_cadencia=(192.168.251.9 2001:db8:42bc:a102::9:100);
@def $h_chasse=(192.168.181.161 2001:db8:42bc:a181::a1:100);
@def $g_dnsrec=($h_cadencia $h_chasse)

domain (ip ip6) {
  table filter {
    chain FORWARD {
      iface internal0 daddr $g_dnsrec proto (tcp udp) dport 53 ACCEPT;
    }
  }
}

which creates (if I calculated correctly) four IPv4 rules and four IPv6
rules automatically. Ferm solves this trivially by dropping any
generated rules where address type and domain doesn't match.

This approach of course makes the sheer number of rules explode, but it
makes writing rules easier. My time is precious, CPU time is generously
available. In the situation where CPU time is precious as well, I might
be willing to spend some more time writing _efficient_ rules but that's
like 2 percent of all the cases and I am not willing to make my life
harder in the other 98 percent.

> As far as the present topic is concerned, the only tangible advantage
> that ipset has is the ability to create a set whose sole purpose is to
> act as a superset of other - potentially mixed - sets. This advantage
> is rather diminished by the fact that one also has to two manage two
> entirely separate rulesets with iptables and ip6tables,
> notwithstanding that wrappers such as fermi exist.

Of course one could have an nft preprocessor solving those issues, but
given that nft is already open source, reads a text file and configures
the nftables kernel backend, the natural approach would be to modify nft
to not only translate input 1:1 to kernel entries, but also add some
bells and whistles that makes life easier for those people writing
firewall rules.

With iptables, where people are encouraged to put their firewall rules
into shell scripts, things were entirely different, and this approach
was very inconvenient (proven by the fact that many iptables frontends
didnt write iptables scripts but files that could be read by
iptables-restore.

> At any rate, the follow nftables ruleset is valid.
> 
> table inet filter {
>         set block4 {
>                 type ipv4_addr
>         }
>         set block6 {
>                 type ipv6_addr
>         }
>         chain INPUT {
>                 type filter hook input priority filter; policy accept
>                 ip saddr @block4 drop
>                 ip6 saddr @block6 drop
>         }
> }

Isnt that clumsy! Wouldn't it be so much nicer to be able to write:

table inet filter {
        set block46 {
                type ipv46_addr
        }
        chain INPUT {
                type filter hook input priority filter; policy accept
                ip46 saddr @block46 drop
        }
}

Having to write distinct rules makes things more complex and
error-prone.

> >> However, the value of an ipv6_addr element is permitted to be an IPv4-mapped IPv6 address.
> >
> > Does nft have a function to convert an IPv4 address to an IPv4-mapped
> > address? Will the rule set do the intended thing? Is an ipv6 rule with
> > an IPv4 mapped address fully equivalent with a proper IPv4 rule?
> 

> I do not know, as I have not yet attempted to use them in an ipv6_addr
> set (it would waste memory).

I surely hope that are beyond the discussion of having to waste memory.
The vast majority of machines have way than enough memory to hold a few
128 bit entries, and if that is really a premium, a given mixed set can
be pulled apart automatically in preprocessing.

First make it work, then make it efficient. Don't trade human time
against machine time unless in very rare corner cases (I currently
happen to work in one of those corner cases, but I still say it's the
wrong way).

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Re: Combine ipv4 and ipv6 in a set

[Daniel]

Le 31/01/2024 à 11:54, Marc Haber a écrit :
[...]
>  >> At any rate, the follow nftables ruleset is valid. >> >> table inet 
filter { set block4 { type ipv4_addr } set block6 { type >> ipv6_addr } 
chain INPUT { type filter hook input priority filter; >> policy accept 
ip saddr @block4 drop ip6 saddr @block6 drop } } > Isnt that clumsy! 
Wouldn't it be so much nicer to be able to write: > > table inet filter 
{ set block46 { type ipv46_addr } chain INPUT { > type filter hook input 
priority filter; policy accept ip46 saddr > @block46 drop } } > > Having 
to write distinct rules makes things more complex and > error-prone.
table inet filter {
         set block6 {
                 type ipv6_addr
         }

chain INPUT {
                 type filter hook input priority filter; policy accept
                 ip4to6 saddr @block6 drop
                 ip6 saddr @block6 drop
         }

With ip4to6 -type/cmd not existing at this time in nftables- 
transforming an ipv4 in ipv4-ipv6 mapped eg 127.0.0.1 => 
::ffff:127.0.0.1 At this time we are already able to map ipv4 addresses 
in ipv6 tables like

nft add element inet filter ip-banned-ip6 { ::ffff:1:2:3:4 }

[...]

-- 
Daniel

Re: Combine ipv4 and ipv6 in a set

[Marc Haber]

On Wed, Jan 31, 2024 at 12:55:12PM +0100, Daniel wrote:
> table inet filter {
>         set block6 {
>                 type ipv6_addr
>         }
> 
> chain INPUT {
>                 type filter hook input priority filter; policy accept
>                 ip4to6 saddr @block6 drop
>                 ip6 saddr @block6 drop
>         }

Still, duplicated rule. That should be optionally invisible to the user.

> With ip4to6 -type/cmd not existing at this time in nftables- transforming an
> ipv4 in ipv4-ipv6 mapped eg 127.0.0.1 => ::ffff:127.0.0.1 At this time we
> are already able to map ipv4 addresses in ipv6 tables like
> 
> nft add element inet filter ip-banned-ip6 { ::ffff:1:2:3:4 }

That immediately wakes my wish to have the address converted
automatically, and the natural place to do this would be in nft
proper. It is already reading in human readable code and generates
kernel configuration.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Re: Combine ipv4 and ipv6 in a set

[Kerin Millar]

Hi Marc,

On Wed, 31 Jan 2024, at 10:54 AM, Marc Haber wrote:
> On Tue, Jan 30, 2024 at 03:17:32PM +0000, Kerin Millar wrote:
>> On Tue, 30 Jan 2024, at 1:08 PM, Marc Haber wrote:
>> > This is one of my pet peeves with nft, actually. For iptables, there was
>> > tooling like ferm which made it possible to write dual-stack rule sets
>> > very easily. This kind of tooling seems to be completely missing in the
>> > nftables world. Am I missing something here?
>> 
>> Quite possibly. Currently, nftables supports:
>> 
>> - mixed rulesets (using tables bearing the "inet" family)
>> - mixed rules (wherever it makes sense)
>> - first-class sets of any kind (irrespective of the type of table enclosing them)
>
> That nicely shows another issue with nft: Docs. Neither the search
> function on the nftables wiki nor the nft manual page have matches for
> "mixed" or "first-class".
>
> How would I use a mixed ruleset? How would I use a mixed rule?

It wasn't intended as jargon, so the onus is upon me to clarify.

Firstly, I mentioned both tables and the "inet" family. If you search for inet in the man page, you should land in the ADDRESS FAMILY section, which indicates that the purpose of an address family is to define the type of packets that can be processed. It goes on to say that "all nftables objects exist in address family specific namespaces, therefore all identifiers include an address family".

Now, if we look at the defined grammar for creating tables in the TABLES section, it is - in part - presented as "{add | create} table [family] table". That shows that that the family can be specified at the time of creating a table. But wait, why is it in square brackets? It means that the argument specifying the family is optional. It goes on to say, "when no address family is specified, ip is used by default".

In summary, a table must always have an address family. If you do not explicitly specify the family at the time of creation, the family shall be "ip". If you specify "inet" as the family, the objects within the table (hooks, chains, rules etc) will be able to handle *both* IPv4 and IPv6 packets. That is what I meant by mixed.

Secondly, by mixed rules, I meant that generalised rules within an inet table can handle both IPv4 and IPv6 packets. Consider a very simple rule such as "tcp dport 22 accept". It will match both IPv4 and IPv6 without issue. As you well know, iptables is incapable of doing this.

Thirdly, by first-class, I meant that sets are an intrinsic feature of nftables. That is, they are not a feature bolted on by way of a separate kernel subsystem, along a separate userspace tool to manage them (ipset), requiring an iptables extension to glue them together. Consequently, it is possible to use sets in many interesting ways which are simply not possible in iptables. I am aware that this comes at the cost of some administrative complexity.

>
>> Granted, one cannot create a set that is typed in such a way that an
>> element can be either an IPv4 or IPv6 address/interval. Conversely,
>> iptables does not natively support sets at all, though it can
>> integrate with sets that are managed by ipset(8). Now, can an ipset
>> contain addresses of mixed types? No, it cannot.
>
> In ferm, a tool that has been available for over 20 years, you can
> (beginning ten years ago, I think) write things like:
>
> @def $h_cadencia=(192.168.251.9 2001:db8:42bc:a102::9:100);
> @def $h_chasse=(192.168.181.161 2001:db8:42bc:a181::a1:100);
> @def $g_dnsrec=($h_cadencia $h_chasse)
>
> domain (ip ip6) {
>   table filter {
>     chain FORWARD {
>       iface internal0 daddr $g_dnsrec proto (tcp udp) dport 53 ACCEPT;
>     }
>   }
> }

I would agree that fermi is a fine wrapper and that its author has put considerable thought into its syntax.

>
> which creates (if I calculated correctly) four IPv4 rules and four IPv6
> rules automatically. Ferm solves this trivially by dropping any
> generated rules where address type and domain doesn't match.
>
> This approach of course makes the sheer number of rules explode, but it
> makes writing rules easier. My time is precious, CPU time is generously
> available. In the situation where CPU time is precious as well, I might
> be willing to spend some more time writing _efficient_ rules but that's
> like 2 percent of all the cases and I am not willing to make my life
> harder in the other 98 percent.
>
>> As far as the present topic is concerned, the only tangible advantage
>> that ipset has is the ability to create a set whose sole purpose is to
>> act as a superset of other - potentially mixed - sets. This advantage
>> is rather diminished by the fact that one also has to two manage two
>> entirely separate rulesets with iptables and ip6tables,
>> notwithstanding that wrappers such as fermi exist.
>
> Of course one could have an nft preprocessor solving those issues, but
> given that nft is already open source, reads a text file and configures
> the nftables kernel backend, the natural approach would be to modify nft
> to not only translate input 1:1 to kernel entries, but also add some
> bells and whistles that makes life easier for those people writing
> firewall rules.
>
> With iptables, where people are encouraged to put their firewall rules
> into shell scripts, things were entirely different, and this approach
> was very inconvenient (proven by the fact that many iptables frontends
> didnt write iptables scripts but files that could be read by
> iptables-restore.
>
>> At any rate, the follow nftables ruleset is valid.
>> 
>> table inet filter {
>>         set block4 {
>>                 type ipv4_addr
>>         }
>>         set block6 {
>>                 type ipv6_addr
>>         }
>>         chain INPUT {
>>                 type filter hook input priority filter; policy accept
>>                 ip saddr @block4 drop
>>                 ip6 saddr @block6 drop
>>         }
>> }
>
> Isnt that clumsy! Wouldn't it be so much nicer to be able to write:

I don't find it particularly clumsy; it is rather low on my list of nftables-related grievances. I don't know whether you are a programmer yourself but, sometimes, I wonder whether there is a culture clash of sorts between programming admins and non-programming admins. Though I am no Netfilter hacker, it is intuitive to me that sets are strongly typed and that the protocols are different. That being said ...

>
> table inet filter {
>         set block46 {
>                 type ipv46_addr
>         }
>         chain INPUT {
>                 type filter hook input priority filter; policy accept
>                 ip46 saddr @block46 drop
>         }
> }
>
> Having to write distinct rules makes things more complex and
> error-prone.

... I can certainly appreciate this perspective and why it might ease the cognitive burden for maintaining some rulesets. 

-- 
Kerin Millar

Re: Combine ipv4 and ipv6 in a set

[Slavko]

Dňa 31. januára 2024 13:02:57 UTC používateľ Kerin Millar <kfm@plushkava.net> napísal:

>Firstly, I mentioned both tables and the "inet" family. If you search for inet in the man page, you should land in the ADDRESS FAMILY section, which indicates that the purpose of an address family is to define the type of packets that can be processed. It goes on to say that "all nftables objects exist in address family specific namespaces, therefore all identifiers include an address family".

I check manpage now, 1.0.6 (as is in debian bookworm) and from its
ADDRESS FAMILY section is nor clean (at least for me) the order of
inet and ip/ip6 tables processing. It is even not clearly stated here,
that packet will be processed in both, the inet and the ip/ip6.

I know, that hook's priority comes into play, it is worth to mention,
if here are hooks with the same priority in both (as inet and ip/ip6)
the order is ... or order is not defined.

>>> table inet filter {
>>>         set block4 {
>>>                 type ipv4_addr
>>>         }
>>>         set block6 {
>>>                 type ipv6_addr
>>>         }
>>>         chain INPUT {
>>>                 type filter hook input priority filter; policy accept
>>>                 ip saddr @block4 drop
>>>                 ip6 saddr @block6 drop
>>>         }
>>> }

This can be OK with two sets, as it really doesn't matter if you have
one ot two rules/sets. But when you start to really use them and you
need eg. 10 sets, you have to define every twice, things become
more complicated...

When i start to play with these sets i was too constrained by iptables's
approach by that, that every iptables's table becomed separate
nftables table, thus one cannot share common sets between eg. raw
and filter tables. It took some time to realize, that i need to switch my
mind and that i can define "raw" and "filter" hooks in one table and thus
share the same set in both (eg. fill content in filter and drop in raw).

Second thing which took time to understand was, why in hell i need
as many sets. After some time i realized, that it is price for flexibility.
In iptables many extensions uses some storage (recent, limit,
hashlimit, connlimit) and these storages are managed by extension
itself, and user needs only to setup its name or even nothing (for
limit). In nftables world one can use sets to achieve the same result,
but have to create sets/storage by self, but can customize them in
simple way (without change kernel module options) and that is
great.

But if one use these features intensive, all sets and rules must
be twiced in inet table (exactly as in separated ip/ip6 tables), the
amount of sets increases significantly. Yes, these "sets" was twiced
in ip(6)tables too, but that was "hidden" in extension code, now the
inet table exposes that twicing. And as amount of sets increases,
listing them becomes more and more not user friendly, as one can
list either all table's sets or one concrete set, nothing in between.

BTW, i am curious, how big difference (in performance/time) is to
search IPv4 vs. IPv6 address in set? Is it worth to consider?

I start to more and more think, that using of inet family (as in current
state) is good only for simple/basic FWs, without any advanced
features, where are L3 addresses used only to static distingushing
of access or where rulesets are maintained by some external tool
(in separate tables).

From my point of view, integration of sets created impressive
features in nftables, but in result they are too constrained (in L3
mean), by the same way as they was in iptables, or even worse,
as now ipset allow to use combined sets and add/update it from
the same name (thus rule too) in both, the iptables and ip6tables
(yes, as mentioned already, that is not possible from command
line -- yet?).

Another thing, which is not clean for me, is memory usage of sets.
I read on multiple places about huge memory usage with big sets,
but nowhere i found real comparison. Nor in memory usage, nor in
performance of nftables's sets vs. ipset sets. Now i don't know
if these memory problems was in early stage of sets development
(and are solved now), or they was caused by suboptimal usage of
tools, or it is by design. Recently i noticed in docs, that there is
memory/performance switch for sets, but what that means in
real usage? Again, no benchmark is documented, to one can
properly decide which one to use. Or, at least, commands/way
how to measure difference by self...

regards


-- 
Slavko
https://www.slavino.sk/

Re: Combine ipv4 and ipv6 in a set

[Kerin Millar]

Hi Slavko,

On Wed, 31 Jan 2024 20:23:54 +0000
Slavko <linux@slavino.sk> wrote:

> Dňa 31. januára 2024 13:02:57 UTC používateľ Kerin Millar <kfm@plushkava.net> napísal:
> 
> >Firstly, I mentioned both tables and the "inet" family. If you search for inet in the man page, you should land in the ADDRESS FAMILY section, which indicates that the purpose of an address family is to define the type of packets that can be processed. It goes on to say that "all nftables objects exist in address family specific namespaces, therefore all identifiers include an address family".
> 
> I check manpage now, 1.0.6 (as is in debian bookworm) and from its
> ADDRESS FAMILY section is nor clean (at least for me) the order of
> inet and ip/ip6 tables processing. It is even not clearly stated here,
> that packet will be processed in both, the inet and the ip/ip6.

I think that the effect of "inet" family is to register hooks - if any - for both the ip and ip6 families. Yet, even knowing that doesn't help to address the question.

> 
> I know, that hook's priority comes into play, it is worth to mention,
> if here are hooks with the same priority in both (as inet and ip/ip6)
> the order is ... or order is not defined.

Yes, it should be made clear - even if only to explicitly say that the behaviour is undefined (it would be better than nothing).

> 
> >>> table inet filter {
> >>>         set block4 {
> >>>                 type ipv4_addr
> >>>         }
> >>>         set block6 {
> >>>                 type ipv6_addr
> >>>         }
> >>>         chain INPUT {
> >>>                 type filter hook input priority filter; policy accept
> >>>                 ip saddr @block4 drop
> >>>                 ip6 saddr @block6 drop
> >>>         }
> >>> }
> 
> This can be OK with two sets, as it really doesn't matter if you have
> one ot two rules/sets. But when you start to really use them and you
> need eg. 10 sets, you have to define every twice, things become
> more complicated...
> 
> When i start to play with these sets i was too constrained by iptables's
> approach by that, that every iptables's table becomed separate
> nftables table, thus one cannot share common sets between eg. raw
> and filter tables. It took some time to realize, that i need to switch my
> mind and that i can define "raw" and "filter" hooks in one table and thus
> share the same set in both (eg. fill content in filter and drop in raw).

This has also annoyed me on several occasions. Though I have been using nftables for a fairly long time, I still find it more natural to organise rulesets based on the conventions of iptables. Old habits die hard, as the saying goes. Incidentally, there is an open bug concerning this.

https://bugzilla.netfilter.org/show_bug.cgi?id=1472

> 
> Second thing which took time to understand was, why in hell i need
> as many sets. After some time i realized, that it is price for flexibility.
> In iptables many extensions uses some storage (recent, limit,
> hashlimit, connlimit) and these storages are managed by extension
> itself, and user needs only to setup its name or even nothing (for
> limit). In nftables world one can use sets to achieve the same result,
> but have to create sets/storage by self, but can customize them in
> simple way (without change kernel module options) and that is
> great.
> 
> But if one use these features intensive, all sets and rules must
> be twiced in inet table (exactly as in separated ip/ip6 tables), the
> amount of sets increases significantly. Yes, these "sets" was twiced
> in ip(6)tables too, but that was "hidden" in extension code, now the
> inet table exposes that twicing. And as amount of sets increases,
> listing them becomes more and more not user friendly, as one can
> list either all table's sets or one concrete set, nothing in between.

All of this is true.

> 
> BTW, i am curious, how big difference (in performance/time) is to
> search IPv4 vs. IPv6 address in set? Is it worth to consider?

I do not know but I would expect a lookup to be very efficient in both cases.

> 
> I start to more and more think, that using of inet family (as in current
> state) is good only for simple/basic FWs, without any advanced
> features, where are L3 addresses used only to static distingushing
> of access or where rulesets are maintained by some external tool
> (in separate tables).
> 
> From my point of view, integration of sets created impressive
> features in nftables, but in result they are too constrained (in L3
> mean), by the same way as they was in iptables, or even worse,
> as now ipset allow to use combined sets and add/update it from
> the same name (thus rule too) in both, the iptables and ip6tables
> (yes, as mentioned already, that is not possible from command
> line -- yet?).
> 
> Another thing, which is not clean for me, is memory usage of sets.
> I read on multiple places about huge memory usage with big sets,
> but nowhere i found real comparison. Nor in memory usage, nor in
> performance of nftables's sets vs. ipset sets. Now i don't know
> if these memory problems was in early stage of sets development
> (and are solved now), or they was caused by suboptimal usage of
> tools, or it is by design. Recently i noticed in docs, that there is
> memory/performance switch for sets, but what that means in
> real usage? Again, no benchmark is documented, to one can
> properly decide which one to use. Or, at least, commands/way
> how to measure difference by self...

I haven't paid much attention to memory usage. Benchmarking performance is fairly straightforward with the use of the shell, especially bash because it offers some useful features such as:

- a time builtin that can measure arbitrarily complex commands (not only "SIMPLE COMMANDS" as defined by the manual)
- C-style for loops: for ((i = 0; i < 1000; i++)); do ...; done
- a printf builtin that can print timestamps without the need for an external utility

As concerns timestamps, bash >=5.0 can even produce timestamps with microsecond resolution.

$ printf '%(%F %T)T.%s\n' -1 "${EPOCHREALTIME: -6}"
2024-01-31 22:01:42.286354

You might find it interesting to look at existing issues concerning the use of sets by visiting the following link.

https://bugzilla.netfilter.org/showdependencytree.cgi?id=1461&hide_resolved=0

Of those, #1584 concerns "high memory requirements", where Pablo appears to be using https://valgrind.org/docs/manual/ms-manual.html to profile memory usage.

-- 
Kerin Millar

Re: Combine ipv4 and ipv6 in a set

[Kevin P. Fleming]

On Wed, Jan 31, 2024, at 23:10, Kerin Millar wrote:
> This has also annoyed me on several occasions. Though I have been using 
> nftables for a fairly long time, I still find it more natural to 
> organise rulesets based on the conventions of iptables. Old habits die 
> hard, as the saying goes. Incidentally, there is an open bug concerning 
> this.

When I switched to nftables last year (after having used iptables since it was created), I was tempted to do the same, but forced myself to rethink my table structures.

In the end I landed on using address family 'inet' for all filtering, 'ip' for NAT, and both 'ip' and 'ip6' for netdev (because those rules are entirely about matching addresses). This does mean that my 'inet' table contains many seemingly-duplicate rules, but I don't really care since the result is so much cleaner and easier to understand than it was with iptables. Also my nftables rules are machine-generated so that's not visible at the layer where I do admin work :)

Re: Combine ipv4 and ipv6 in a set

[Kerin Millar]

On Thu, 01 Feb 2024 07:06:08 +0100
"Kevin P. Fleming" <lists.netfilter@kevin.km6g.us> wrote:

> On Wed, Jan 31, 2024, at 23:10, Kerin Millar wrote:
> > This has also annoyed me on several occasions. Though I have been using 
> > nftables for a fairly long time, I still find it more natural to 
> > organise rulesets based on the conventions of iptables. Old habits die 
> > hard, as the saying goes. Incidentally, there is an open bug concerning 
> > this.
> 
> When I switched to nftables last year (after having used iptables since it was created), I was tempted to do the same, but forced myself to rethink my table structures.
> 
> In the end I landed on using address family 'inet' for all filtering, 'ip' for NAT, and both 'ip' and 'ip6' for netdev (because those rules are entirely about matching addresses). This does mean that my 'inet' table contains many seemingly-duplicate rules, but I don't really care since the result is so much cleaner and easier to understand than it was with iptables. Also my nftables rules are machine-generated so that's not visible at the layer where I do admin work :)

I ended up using fewer tables, while naming several chains so as to incorporate a faux namespace e.g. "nat/PREROUTING". It works well enough but I cannot say that I have come to like it.

-- 
Kerin Millar

Re: Combine ipv4 and ipv6 in a set

[Slavko]

Dňa 31. 1. o 23:10 Kerin Millar napísal(a):

> This has also annoyed me on several occasions. Though I have been using nftables for a fairly long time, I still find it more natural to organise rulesets based on the conventions of iptables. Old habits die hard, as the saying goes. Incidentally, there is an open bug concerning this.

Perhaps you know that, but IMO it can be useful for some...

IMO the root of confusion comes from fact, that both (iptables/nft) uses 
the same name -- "table". I don't want to discuss if it was or wasn't 
good decision, as it doesn't matter, it is only name.

In iptables table more or less defines chain's hook priority:

    CHAIN => hook type
    TABLE => hook priority

In nftables one have (can) to define hook type and its priority by self, 
thus that purpose of ipt TABLEs is gone and main purpose of table 
changed to define family (and group objects) and name it.

I fight with that hook's priorities for long time, i was not able to get 
it, until symbolic names for priorities was introduced. And after math 
can be used in its definition (eg. filter + 5) it is really simple now. 
Great job.

>> BTW, i am curious, how big difference (in performance/time) is to
>> search IPv4 vs. IPv6 address in set? Is it worth to consider?
> 
> I do not know but I would expect a lookup to be very efficient in both cases.

I know only very little how ipsets stores items internally, mostly only 
that it uses hashes and tries, and i know nothing about sets. I know, 
that these tries are efficient way to search in large datasets, with 
near to constant search time. But if someone complains about performance 
(IP vs. IPv6) of ipsets/sets, there must be difference. The question is, 
if it is important/significant.

> I haven't paid much attention to memory usage. Benchmarking performance is fairly straightforward with the use of the shell, especially bash because it offers some useful features such as:

I mean performance in packet flow, how long it tooks to find (or not 
find) match or update/add from FW rule with ipset and with set, if both 
uses the same dataset size. If size of ipset/set matter, etc. Will be 
measured command time the same for particular action as from packet 
flow? That comparison will reveal, if sets are real replacement of 
ipsets in all (or most) cases or not. Or will be adding/removing item(s) 
from command line time the same as from packet flow? (IMO not, as 
command invocation plays role too)

BTW i recently realized how important can be command time, first some 
attack (which fills ipsets with thousands IPs), then kernel update, thus 
restart was needed, and it reveals how inefficient way fail2ban uses to 
restore ipset bans :-D But that is not nft related (and it was short 
time shock)...

> You might find it interesting to look at existing issues concerning the use of sets by visiting the following link.

thanks, i will check them from time to time...

> Of those, #1584 concerns "high memory requirements", where Pablo appears to be using https://valgrind.org/docs/manual/ms-manual.html to profile memory usage.

I meet valgrind only two or three times, related to some bug finding, 
but i got full guide from devs (do that, run that and post that or so), 
without understanding what i am doing...

Ipset shows in output (list) how much memory particular set uses, can i 
get that info from set listing (to compare)?

-- 
Slavko

Re: Combine ipv4 and ipv6 in a set

[Kerin Millar]

Hi Slavko,

On Thu, 1 Feb 2024, at 10:50 AM, Slavko wrote:
> Dňa 31. 1. o 23:10 Kerin Millar napísal(a):
>
>> This has also annoyed me on several occasions. Though I have been using nftables for a fairly long time, I still find it more natural to organise rulesets based on the conventions of iptables. Old habits die hard, as the saying goes. Incidentally, there is an open bug concerning this.
>
> Perhaps you know that, but IMO it can be useful for some...
>
> IMO the root of confusion comes from fact, that both (iptables/nft) uses 
> the same name -- "table". I don't want to discuss if it was or wasn't 
> good decision, as it doesn't matter, it is only name.
>
> In iptables table more or less defines chain's hook priority:
>
>     CHAIN => hook type
>     TABLE => hook priority
>
> In nftables one have (can) to define hook type and its priority by self, 
> thus that purpose of ipt TABLEs is gone and main purpose of table 
> changed to define family (and group objects) and name it.

Quite. It's nothing more than an "object" namespace with an associated address family. In what way it resembles a table, I do not know.

>
> I fight with that hook's priorities for long time, i was not able to get 
> it, until symbolic names for priorities was introduced. And after math 
> can be used in its definition (eg. filter + 5) it is really simple now. 
> Great job.
>
>>> BTW, i am curious, how big difference (in performance/time) is to
>>> search IPv4 vs. IPv6 address in set? Is it worth to consider?
>> 
>> I do not know but I would expect a lookup to be very efficient in both cases.
>
> I know only very little how ipsets stores items internally, mostly only 
> that it uses hashes and tries, and i know nothing about sets. I know, 
> that these tries are efficient way to search in large datasets, with 
> near to constant search time. But if someone complains about performance 
> (IP vs. IPv6) of ipsets/sets, there must be difference. The question is, 
> if it is important/significant.
>
>> I haven't paid much attention to memory usage. Benchmarking performance is fairly straightforward with the use of the shell, especially bash because it offers some useful features such as:
>
> I mean performance in packet flow, how long it tooks to find (or not 
> find) match or update/add from FW rule with ipset and with set, if both 
> uses the same dataset size. If size of ipset/set matter, etc. Will be 
> measured command time the same for particular action as from packet 
> flow? That comparison will reveal, if sets are real replacement of 
> ipsets in all (or most) cases or not. Or will be adding/removing item(s) 
> from command line time the same as from packet flow? (IMO not, as 
> command invocation plays role too)

I don't think that there is an abundance of material on this topic but I recall seeing one or two articles on the web concerning this sort of performance measurement (by well known Netfilter developers/contributors).

>
> BTW i recently realized how important can be command time, first some 
> attack (which fills ipsets with thousands IPs), then kernel update, thus 
> restart was needed, and it reveals how inefficient way fail2ban uses to 
> restore ipset bans :-D But that is not nft related (and it was short 
> time shock)...
>
>> You might find it interesting to look at existing issues concerning the use of sets by visiting the following link.
>
> thanks, i will check them from time to time...
>
>> Of those, #1584 concerns "high memory requirements", where Pablo appears to be using https://valgrind.org/docs/manual/ms-manual.html to profile memory usage.
>
> I meet valgrind only two or three times, related to some bug finding, 
> but i got full guide from devs (do that, run that and post that or so), 
> without understanding what i am doing...
>
> Ipset shows in output (list) how much memory particular set uses, can i 
> get that info from set listing (to compare)?

Try:

nft -t list sets

Alternatively:

nft -jt list sets | jq -r '.nftables[1:] | .[].set | [ .table, .family, .name, .size ] | @csv'

The problem is that the size isn't always shown. I complained about it here:

https://bugzilla.netfilter.org/show_bug.cgi?id=1717.

-- 
Kerin Millar

Re: Combine ipv4 and ipv6 in a set

[Slavko]

Dňa 1. februára 2024 12:48:31 UTC používateľ Kerin Millar <kfm@plushkava.net> napísal:

>The problem is that the size isn't always shown. I complained about it here:

I afraid, that it shows configured size limit, not real items count and does not
show memory usage (on my version)...

regards


-- 
Slavko
https://www.slavino.sk/

Re: Combine ipv4 and ipv6 in a set

[Kerin Millar]

On Thu, 1 Feb 2024, at 1:42 PM, Slavko wrote:
> Dňa 1. februára 2024 12:48:31 UTC používateľ Kerin Millar 
> <kfm@plushkava.net> napísal:
>
>>The problem is that the size isn't always shown. I complained about it here:
>
> I afraid, that it shows configured size limit, not real items count and does not
> show memory usage (on my version)...

Ah, yes. That won't do at all. Sorry.

-- 
Kerin Millar

Re: Combine ipv4 and ipv6 in a set

[Pablo Neira Ayuso]

On Wed, Jan 31, 2024 at 08:23:54PM +0000, Slavko wrote:
> Dňa 31. januára 2024 13:02:57 UTC používateľ Kerin Millar <kfm@plushkava.net> napísal:
> 
[...]
> I check manpage now, 1.0.6 (as is in debian bookworm) and from its
> ADDRESS FAMILY section is nor clean (at least for me) the order of
> inet and ip/ip6 tables processing. It is even not clearly stated here,
> that packet will be processed in both, the inet and the ip/ip6.

There is a command to display the datapath hook pipeline per device:

# nft list hooks device eth0
family ip {
        hook ingress {
                 0000000000 chain netdev x y [nf_tables]
        }
        hook input {
                 0000000000 chain inet x y [nf_tables]
        }
        hook forward {
                -0000000225 selinux_ip_forward
        }
        hook output {
                -0000000225 selinux_ip_output
        }
        hook postrouting {
                +0000000225 selinux_ip_postroute
        }
}
family ip6 {
        hook ingress {
                 0000000000 chain netdev x y [nf_tables]
        }
        hook input {
                 0000000000 chain inet x y [nf_tables]
        }
        hook forward {
                -0000000225 selinux_ip_forward
        }
        hook output {
                -0000000225 selinux_ip_output
        }
        hook postrouting {
                +0000000225 selinux_ip_postroute
        }
}
family bridge {
        hook ingress {
                 0000000000 chain netdev x y [nf_tables]
        }
}

[ The quick example above is rather silly, because it was taken from a
  VM with selinux hooks, one single inet/chain at input and one hook at
  netdev. ]

But it shows a view per family, for an IPv6 packet...

- going for local process follows ingress hook, entering nf_tables
  chain netdev x y, then in case it is local traffic it enters chain
  inet x y [nf_tables].

- being forwarded goes through ingress, then forward selinux hook at
  priority -225, then postrouting selinux hook again.

- leaving from local process follows output and postrouting selinux
  hooks.

- entering the bridge layer, it gets evaluated by the ingress hook
  too.

It is a flat representation, so you still have to understand how
routing determines what hook the packet visits.

There is a terse refering in the manpage:

LISTING
           list { secmarks | synproxys | flow tables | meters | hooks } [family]
           list { secmarks | synproxys | flow tables | meters | hooks } table [family] table
           list ct { timeout | expectation | helper | helpers } table [family] table

       Inspect configured objects. list hooks shows the full hook
       pipeline, including those registered by kernel modules,
       such as nf_conntrack.

which is a really shame :)

Someone up for contributing an improvement for the manpage, including
a better example than above and the explaination?

Re: Combine ipv4 and ipv6 in a set

[Slavko]

Dňa 31. 1. o 23:36 Pablo Neira Ayuso napísal(a):

> There is a command to display the datapath hook pipeline per device:
> 
> # nft list hooks device eth0

That is nice, except that it returns empty output here: nft 1.0.6, 
kernel 6.1.69, inside LXC container... No error reported, just no output.

regards

-- 
Slavko

Re: Combine ipv4 and ipv6 in a set

[Eric]

On Wednesday, January 31st, 2024 at 14:36, Pablo Neira Ayuso <pablo@netfilter.org> wrote:

> On Wed, Jan 31, 2024 at 08:23:54PM +0000, Slavko wrote:
> 
> > Dňa 31. januára 2024 13:02:57 UTC používateľ Kerin Millar kfm@plushkava.net napísal:
> 
> [...]
> 
> > I check manpage now, 1.0.6 (as is in debian bookworm) and from its
> > ADDRESS FAMILY section is nor clean (at least for me) the order of
> > inet and ip/ip6 tables processing. It is even not clearly stated here,
> > that packet will be processed in both, the inet and the ip/ip6.
> 
> 
> There is a command to display the datapath hook pipeline per device:
> 
> # nft list hooks device eth0
> family ip {
> hook ingress {
> 0000000000 chain netdev x y [nf_tables]
> }

Pablo,

Is there a minimum kernel version required to get these to work?  I've
tried it on 5.15 and 6.1, both of which just spin for a second and
produce nothing.  

I also tried a bunch of other 'list' commands and got 'list ct expectation'
to seg fault on 5.15, but it's fine on 6.1.  (And all of these other 'list
<state>' commands produce nothing on my machines.)

Eric

Re: Combine ipv4 and ipv6 in a set

[Pablo Neira Ayuso]

On Fri, Feb 02, 2024 at 01:38:46AM +0000, Eric wrote:
> On Wednesday, January 31st, 2024 at 14:36, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> 
> > On Wed, Jan 31, 2024 at 08:23:54PM +0000, Slavko wrote:
> > 
> > > Dňa 31. januára 2024 13:02:57 UTC používateľ Kerin Millar kfm@plushkava.net napísal:
> > 
> > [...]
> > 
> > > I check manpage now, 1.0.6 (as is in debian bookworm) and from its
> > > ADDRESS FAMILY section is nor clean (at least for me) the order of
> > > inet and ip/ip6 tables processing. It is even not clearly stated here,
> > > that packet will be processed in both, the inet and the ip/ip6.
> > 
> > 
> > There is a command to display the datapath hook pipeline per device:
> > 
> > # nft list hooks device eth0
> > family ip {
> > hook ingress {
> > 0000000000 chain netdev x y [nf_tables]
> > }
> 
> Pablo,
> 
> Is there a minimum kernel version required to get these to work?  I've
> tried it on 5.15 and 6.1, both of which just spin for a second and
> produce nothing.

Works fine with -stable 5.15 and 6.1 here.

Does your kernel turn on this?

CONFIG_NETFILTER_NETLINK_HOOK=m

> I also tried a bunch of other 'list' commands and got 'list ct expectation'
> to seg fault on 5.15, but it's fine on 6.1.

I have just tested on 5.15 and 6.1 and this works fine.

Userspace nft version and reproducer?

> (And all of these other 'list <state>' commands produce nothing on my machines.)

If you did not define such objects in your ruleset, then it is normal
that this produces nothing.

Re: Combine ipv4 and ipv6 in a set

[Slavko]

[-- Attachment #1: Type: text/plain, Size: 560 bytes --]

Ahoj,

Dňa Fri, 2 Feb 2024 10:23:04 +0100 Pablo Neira Ayuso
<pablo@netfilter.org> napísal:

> Does your kernel turn on this?
> 
> CONFIG_NETFILTER_NETLINK_HOOK=m

Unfortunately, the debian's stock kernel doesn't enable that.

But how one have to realize, that kernel module is missing, from success
return code and/or no output at all? And i checked dmesg (kernel log)
output too, nothing related in it.

IMO, nft has report that by some way, and document that needing, at
least in manpage.

regards

-- 
Slavko
https://www.slavino.sk

[-- Attachment #2: Digitálny podpis OpenPGP --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Combine ipv4 and ipv6 in a set

[Eric]

On Friday, February 2nd, 2024 at 01:23, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> Works fine with -stable 5.15 and 6.1 here.
> 
> Does your kernel turn on this?
> 
> CONFIG_NETFILTER_NETLINK_HOOK=m

Like Slavko, my test devices (both OpenWrt builds) have

# CONFIG_NETFILTER_NETLINK_HOOK is not set

I'll have to check and see how much it grows the kernel when enabled, as this
seems like it would be useful to have on by default.


Re the seg fault, I'm still trying to find a minimal reproducer, but it looks
like just about anything fires it off with k5.15 (I see it on x86, MIPS and
aarch64 devices).

Eric

Re: Combine ipv4 and ipv6 in a set

[Pablo Neira Ayuso]

On Sat, Feb 03, 2024 at 03:49:54PM +0000, Eric wrote:
> On Friday, February 2nd, 2024 at 01:23, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > Works fine with -stable 5.15 and 6.1 here.
> > 
> > Does your kernel turn on this?
> > 
> > CONFIG_NETFILTER_NETLINK_HOOK=m
> 
> Like Slavko, my test devices (both OpenWrt builds) have
> 
> # CONFIG_NETFILTER_NETLINK_HOOK is not set
> 
> I'll have to check and see how much it grows the kernel when enabled, as this
> seems like it would be useful to have on by default.

It should not take much, it is a simple netlink interface that
iterates over a few existing datastructures.

> Re the seg fault, I'm still trying to find a minimal reproducer, but it looks
> like just about anything fires it off with k5.15 (I see it on x86, MIPS and
> aarch64 devices).

Strange that userspace crashes depending on the kernel version.

What is your userspace nft version there?

Thanks.

Re: Combine ipv4 and ipv6 in a set

[Eric]

On Monday, February 5th, 2024 at 01:47, Pablo Neira Ayuso <pablo@netfilter.org> wrote:

> Strange that userspace crashes depending on the kernel version.
> 
> What is your userspace nft version there?

This is on the current release of OpenWrt so previous release of nft.  Same behavior on x86, aarch64 and mips.  All other commands from my monkey testing just return silently, only this one fails.

$ uname -a
Linux rtr00 5.15.137 #0 SMP Tue Nov 14 13:38:11 2023 x86_64 GNU/Linux

$ nft -V
nftables v1.0.8 (Old Doc Yak #2)
  cli:          no
  json:         yes
  minigmp:      yes
  libxtables:   no

$ nft list ct expectation table inet fw4
Segmentation fault

Re: Combine ipv4 and ipv6 in a set

[Pablo Neira Ayuso]

On Mon, Feb 05, 2024 at 05:02:16PM +0000, Eric wrote:
> On Monday, February 5th, 2024 at 01:47, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> 
> > Strange that userspace crashes depending on the kernel version.
> > 
> > What is your userspace nft version there?
> 
> This is on the current release of OpenWrt so previous release of nft.  Same behavior on x86, aarch64 and mips.  All other commands from my monkey testing just return silently, only this one fails.
> 
> $ uname -a
> Linux rtr00 5.15.137 #0 SMP Tue Nov 14 13:38:11 2023 x86_64 GNU/Linux
> 
> $ nft -V
> nftables v1.0.8 (Old Doc Yak #2)
>   cli:          no
>   json:         yes
>   minigmp:      yes
>   libxtables:   no
> 
> $ nft list ct expectation table inet fw4
> Segmentation fault

Fix is already available in v1.0.9:

commit a8ff324dc64fd76f7d218d3d94c5885250951258
Author: Florian Westphal <fw@strlen.de>
Date:   Fri Jul 28 21:04:13 2023 +0200

    ct expectation: fix 'list object x' vs. 'list objects in table' confusion
    
    Just like "ct timeout", "ct expectation" is in need of the same fix,
    we get segfault on "nft list ct expectation table t", if table t exists.
    
    This is the exact same pattern as resolved for "ct timeout" in commit
    1d2e22fc0521 ("ct timeout: fix 'list object x' vs. 'list objects in table' confusion").
    
    Signed-off-by: Florian Westphal <fw@strlen.de>

Re: Combine ipv4 and ipv6 in a set

[Eric]

On Monday, February 5th, 2024 at 09:08, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> Fix is already available in v1.0.9:

Excellent!  Thanks.

Re: Combine ipv4 and ipv6 in a set

[Marc Haber]

On Wed, Jan 31, 2024 at 01:02:57PM +0000, Kerin Millar wrote:
> On Wed, 31 Jan 2024, at 10:54 AM, Marc Haber wrote:
> > On Tue, Jan 30, 2024 at 03:17:32PM +0000, Kerin Millar wrote:
> >> On Tue, 30 Jan 2024, at 1:08 PM, Marc Haber wrote:
> >> > This is one of my pet peeves with nft, actually. For iptables, there was
> >> > tooling like ferm which made it possible to write dual-stack rule sets
> >> > very easily. This kind of tooling seems to be completely missing in the
> >> > nftables world. Am I missing something here?
> >> 
> >> Quite possibly. Currently, nftables supports:
> >> 
> >> - mixed rulesets (using tables bearing the "inet" family)
> >> - mixed rules (wherever it makes sense)
> >> - first-class sets of any kind (irrespective of the type of table enclosing them)
> >
> > That nicely shows another issue with nft: Docs. Neither the search
> > function on the nftables wiki nor the nft manual page have matches for
> > "mixed" or "first-class".
> >
> > How would I use a mixed ruleset? How would I use a mixed rule?
> 
> It wasn't intended as jargon, so the onus is upon me to clarify.
> 
> Firstly, I mentioned both tables and the "inet" family. If you search for inet in the man page, you should land in the ADDRESS FAMILY section, which indicates that the purpose of an address family is to define the type of packets that can be processed. It goes on to say that "all nftables objects exist in address family specific namespaces, therefore all identifiers include an address family".
> 
> Now, if we look at the defined grammar for creating tables in the TABLES section, it is - in part - presented as "{add | create} table [family] table". That shows that that the family can be specified at the time of creating a table. But wait, why is it in square brackets? It means that the argument specifying the family is optional. It goes on to say, "when no address family is specified, ip is used by default".
> 
> In summary, a table must always have an address family. If you do not explicitly specify the family at the time of creation, the family shall be "ip". If you specify "inet" as the family, the objects within the table (hooks, chains, rules etc) will be able to handle *both* IPv4 and IPv6 packets. That is what I meant by mixed.

Thanks for the clarification. This is what I am already using, but I am
not having formal create statements. I have definitions like:

table inet filter {
	chain input {
▸       ▸       type filter hook input priority 0; policy drop;↲
▸       ▸       ct state invalid drop comment "Drop invalid packets"↲
▸       ▸       ct state established,related accept comment "Accept established, related"↲
▸       ▸       iifname lo accept comment "Accept lo"↲
▸       ▸       icmpv6 type { nd-neighbor-solicit, nd-neighbor-advert, echo-request } accept↲
▸       ▸       icmp type { echo-request } accept↲
▸       ▸       ip  saddr $ssh_from4 tcp dport ssh accept↲
▸       ▸       ip6 saddr $ssh_from6 tcp dport ssh accept↲
	}
}

It is the two last rules defined that I gripe with: I need to define two
variables, and I need to define two rules. That's the problem.

> Secondly, by mixed rules, I meant that generalised rules within an inet table can handle both IPv4 and IPv6 packets. Consider a very simple rule such as "tcp dport 22 accept". It will match both IPv4 and IPv6 without issue. As you well know, iptables is incapable of doing this.

I must admit that I very rarely have rules that don't contain a list of
IP addresses.

> > In ferm, a tool that has been available for over 20 years, you can
> > (beginning ten years ago, I think) write things like:
> >
> > @def $h_cadencia=(192.168.251.9 2001:db8:42bc:a102::9:100);
> > @def $h_chasse=(192.168.181.161 2001:db8:42bc:a181::a1:100);
> > @def $g_dnsrec=($h_cadencia $h_chasse)
> >
> > domain (ip ip6) {
> >   table filter {
> >     chain FORWARD {
> >       iface internal0 daddr $g_dnsrec proto (tcp udp) dport 53 ACCEPT;
> >     }
> >   }
> > }
> 
> I would agree that fermi is a fine wrapper and that its author has put considerable thought into its syntax.

ferm (sic!) is an acronym for "for easy rule making". No connection to
Enrico ;-)   People mock it as being an "iptables macro assembler", and
that's exactly what I like in it. nft is still just an assembler, it
needs better macros, to stay in that imagery.

ferm's syntax is quite similiar to what nft uses. It just seems natural
to extend nft to become a better tool.

> >> At any rate, the follow nftables ruleset is valid.
> >> 
> >> table inet filter {
> >>         set block4 {
> >>                 type ipv4_addr
> >>         }
> >>         set block6 {
> >>                 type ipv6_addr
> >>         }
> >>         chain INPUT {
> >>                 type filter hook input priority filter; policy accept
> >>                 ip saddr @block4 drop
> >>                 ip6 saddr @block6 drop
> >>         }
> >> }
> >
> > Isnt that clumsy! Wouldn't it be so much nicer to be able to write:
> 
> I don't find it particularly clumsy; it is rather low on my list of nftables-related grievances. I don't know whether you are a programmer yourself but, sometimes, I wonder whether there is a culture clash of sorts between programming admins and non-programming admins. Though I am no Netfilter hacker, it is intuitive to me that sets are strongly typed and that the protocols are different. That being said ...

It is my particular grievance because it means manually duplicating work
for the people writing the rules. They're prone to make mistakes. The
error class of IPv4 and IPv6 firewall rules bot being in sync is quite
hard to debug, I'd rather have both protocols break together (and to be
fixed together).

I don't have a formal development background, I have always been more
Ops than Dev, and you probably don't want to read my code. I see
development as providing tools to users. And nft is not doing a
particularly good job for the users, the people actually writing rules.

Sadly, my limited development background and my limited time prevent me
from writing my own nft processor (which would probably not generate nft
input but write the rules directly to the kernel just as nft does).

> ... I can certainly appreciate this perspective and why it might ease the cognitive burden for maintaining some rulesets. 

Thanks for listening. This discussion surely not only helps the two of
us to learn about the other side, but it might also be inspiration for
the audience of the mailing list.

Greetings
Marc

-- 
-----------------------------------------------------------------------------
Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421