Dynamically appending addresses to a named set

Dynamically appending addresses to a named set

[Lars Noodén]

Hello,

In NFTables, I have created a named set called 'bar' in the chain input
in the table foo.  I can add elements to the set manually,

# nft add element ip foo bar { 192.168.2.2 }

However, I am not able to guess the syntax to have a regular NFTables
rule do the appending automatically.  I've tried a lot of permutations
of the following, but always with fatal errors,

# nft add rule foo input tcp dport 22 counter add @bar { ip saddr }
Error: Could not process rule: Operation not supported
add rule foo input tcp dport 22 counter add @bar { ip saddr }

# nft add rule foo input tcp dport 22 add element @bar { ip saddr }
Error: syntax error, unexpected element, expecting @ or '$'
add rule foo input tcp dport 22 add element @bar { ip saddr }

What would be the correct syntax to have a rule which automatically adds
a source IP address to the named set when certain conditions are met?

/Lars

Re: Dynamically appending addresses to a named set

[Kerin Millar]

On Wed, 12 Mar 2025, at 4:08 PM, Lars Noodén wrote:
> Hello,
>
> In NFTables, I have created a named set called 'bar' in the chain input
> in the table foo.  I can add elements to the set manually,
>
> # nft add element ip foo bar { 192.168.2.2 }
>
> However, I am not able to guess the syntax to have a regular NFTables
> rule do the appending automatically.  I've tried a lot of permutations
> of the following, but always with fatal errors,
>
> # nft add rule foo input tcp dport 22 counter add @bar { ip saddr }
> Error: Could not process rule: Operation not supported
> add rule foo input tcp dport 22 counter add @bar { ip saddr }

For the kernel to raise ENOTSUP does not indicate an error of syntax. The bytecode intended for the nftables VM will already have been compiled at this point.

I suspect that your set has been declared with the "interval" flag in effect, in which case updates from the packet path are not allowed. As far as I can tell, this constraint is undocumented.

-- 
Kerin Millar

Re: Dynamically appending addresses to a named set

[Pablo Neira Ayuso]

On Wed, Mar 12, 2025 at 07:44:25PM +0000, Kerin Millar wrote:
> On Wed, 12 Mar 2025, at 4:08 PM, Lars Noodén wrote:
> > Hello,
> >
> > In NFTables, I have created a named set called 'bar' in the chain input
> > in the table foo.  I can add elements to the set manually,
> >
> > # nft add element ip foo bar { 192.168.2.2 }
> >
> > However, I am not able to guess the syntax to have a regular NFTables
> > rule do the appending automatically.  I've tried a lot of permutations
> > of the following, but always with fatal errors,
> >
> > # nft add rule foo input tcp dport 22 counter add @bar { ip saddr }
> > Error: Could not process rule: Operation not supported
> > add rule foo input tcp dport 22 counter add @bar { ip saddr }
> 
> For the kernel to raise ENOTSUP does not indicate an error of syntax. The bytecode intended for the nftables VM will already have been compiled at this point.
> 
> I suspect that your set has been declared with the "interval" flag in effect, in which case updates from the packet path are not allowed. As far as I can tell, this constraint is undocumented.

Maybe Lars forgot to set on the flags dynamic;

Where is you set declaration?

Re: Dynamically appending addresses to a named set

[Kerin Millar]

On Wed, 12 Mar 2025, at 7:48 PM, Pablo Neira Ayuso wrote:
> On Wed, Mar 12, 2025 at 07:44:25PM +0000, Kerin Millar wrote:
>> On Wed, 12 Mar 2025, at 4:08 PM, Lars Noodén wrote:
>> > Hello,
>> >
>> > In NFTables, I have created a named set called 'bar' in the chain input
>> > in the table foo.  I can add elements to the set manually,
>> >
>> > # nft add element ip foo bar { 192.168.2.2 }
>> >
>> > However, I am not able to guess the syntax to have a regular NFTables
>> > rule do the appending automatically.  I've tried a lot of permutations
>> > of the following, but always with fatal errors,
>> >
>> > # nft add rule foo input tcp dport 22 counter add @bar { ip saddr }
>> > Error: Could not process rule: Operation not supported
>> > add rule foo input tcp dport 22 counter add @bar { ip saddr }
>> 
>> For the kernel to raise ENOTSUP does not indicate an error of syntax. The bytecode intended for the nftables VM will already have been compiled at this point.
>> 
>> I suspect that your set has been declared with the "interval" flag in effect, in which case updates from the packet path are not allowed. As far as I can tell, this constraint is undocumented.
>
> Maybe Lars forgot to set on the flags dynamic;

Not to go off on a tangent but that was my initial thought. Yet, neglecting to specify the "dynamic" flag does not seem able to cause the error that Lars describes.

# cat test.nft
flush ruleset
table ip foo {
        set bar { type ipv4_addr; }
        chain input {}
}
# nft -f test.nft
# nft 'add rule foo input tcp dport 22 counter add @bar { ip saddr }'; echo "$?"
0

That's with nftables v1.1.1 and Linux 6.12.13. Ultimately, the set becomes a dynamic one, even though it was not declared to that effect. I find it surprising because:

- the behaviour of the implementation directly contradicts the manual
- it seems difficult to fathom, predict and explain the behaviour
- it casts doubt on the purpose of the flag (where is it even useful to declare it?)

I wonder, also, how nft(8) can possibly be assured of my intent in a situation such as this. Imagine a scenario in which the admin adjusts a set so as to be non-dynamic and forgets to remove a rule that updates from the packet path prior to reloading a ruleset. Could it not simply abort upon encountering the rule and require for the admin to resolve the innate contradiction?

Or, is "dynamic" destined to become a historical artifact and be retained only for the purposes of backwards-compatibility?

-- 
Kerin Millar

Re: Dynamically appending addresses to a named set

[Pablo Neira Ayuso]

On Wed, Mar 12, 2025 at 09:50:20PM +0000, Kerin Millar wrote:
> On Wed, 12 Mar 2025, at 7:48 PM, Pablo Neira Ayuso wrote:
> > On Wed, Mar 12, 2025 at 07:44:25PM +0000, Kerin Millar wrote:
> >> On Wed, 12 Mar 2025, at 4:08 PM, Lars Noodén wrote:
> >> > Hello,
> >> >
> >> > In NFTables, I have created a named set called 'bar' in the chain input
> >> > in the table foo.  I can add elements to the set manually,
> >> >
> >> > # nft add element ip foo bar { 192.168.2.2 }
> >> >
> >> > However, I am not able to guess the syntax to have a regular NFTables
> >> > rule do the appending automatically.  I've tried a lot of permutations
> >> > of the following, but always with fatal errors,
> >> >
> >> > # nft add rule foo input tcp dport 22 counter add @bar { ip saddr }
> >> > Error: Could not process rule: Operation not supported
> >> > add rule foo input tcp dport 22 counter add @bar { ip saddr }
> >> 
> >> For the kernel to raise ENOTSUP does not indicate an error of syntax. The bytecode intended for the nftables VM will already have been compiled at this point.
> >> 
> >> I suspect that your set has been declared with the "interval" flag in effect, in which case updates from the packet path are not allowed. As far as I can tell, this constraint is undocumented.
> >
> > Maybe Lars forgot to set on the flags dynamic;
> 
> Not to go off on a tangent but that was my initial thought. Yet, neglecting to specify the "dynamic" flag does not seem able to cause the error that Lars describes.
> 
> # cat test.nft
> flush ruleset
> table ip foo {
>         set bar { type ipv4_addr; }
>         chain input {}
> }
> # nft -f test.nft
> # nft 'add rule foo input tcp dport 22 counter add @bar { ip saddr }'; echo "$?"
> 0

If you list the ruleset above, the dynamic flag shows up.

> That's with nftables v1.1.1 and Linux 6.12.13. Ultimately, the set becomes a dynamic one, even though it was not declared to that effect. I find it surprising because:
> 
> - the behaviour of the implementation directly contradicts the manual
> - it seems difficult to fathom, predict and explain the behaviour
> - it casts doubt on the purpose of the flag (where is it even useful to declare it?)
> 
> I wonder, also, how nft(8) can possibly be assured of my intent in a situation such as this. Imagine a scenario in which the admin adjusts a set so as to be non-dynamic and forgets to remove a rule that updates from the packet path prior to reloading a ruleset. Could it not simply abort upon encountering the rule and require for the admin to resolve the innate contradiction?
>
> Or, is "dynamic" destined to become a historical artifact and be retained only for the purposes of backwards-compatibility?

The ruleset above provides sufficient context to infer that the
dynamic flag is needed, but that might not be the case in all
circunstances. The dynamic flag cannot be inferred in all cases like
the one above.

Without Lars' set declaration, the question is incomplete and it is
not easy to answer.

Re: Dynamically appending addresses to a named set

[Lars Noodén]

Thanks.

On 3/12/25 21:44, Kerin Millar wrote:
 > I suspect that your set has been declared with the "interval" flag
 > in effect, in which case updates from the packet path are not
 > allowed. As far as I can tell, this constraint is undocumented.

Yes, it is the case that the interval flag has been set.  It seems that
the interval flag necessary when the set shall contain a mixture of
single IPv4 addresses and subnets with elements:

	"Error: You must add 'flags interval' to your
	 set declaration if you want to add prefix elements"

and with auto-merge:

	"Error: auto-merge only works with interval sets"

It is necessary to have auto-merge since the individual IPv4 addresses
and subnets get added in an unpredictable manner and may overlap.

On 3/13/25 00:06, Pablo Neira Ayuso wrote:
> The ruleset above provides sufficient context to infer that the
> dynamic flag is needed, but that might not be the case in all
> circunstances. The dynamic flag cannot be inferred in all cases like
> the one above.
>
> Without Lars' set declaration, the question is incomplete and it is
> not easy to answer.

The following is basically the set up:

table ip foo {
         set bar {
                 type ipv4_addr
                 flags interval
                 auto-merge
                 elements = { 192.168.2.0/24 }
         }
	chain input {}
}

The two important parts to retain are auto-merge and pre-defined
elements (and be able to add to the elements later).

/Lars

Re: Dynamically appending addresses to a named set

[Pablo Neira Ayuso]

On Thu, Mar 13, 2025 at 06:31:03AM +0200, Lars Noodén wrote:
> Thanks.
> 
> On 3/12/25 21:44, Kerin Millar wrote:
> > I suspect that your set has been declared with the "interval" flag
> > in effect, in which case updates from the packet path are not
> > allowed. As far as I can tell, this constraint is undocumented.
> 
> Yes, it is the case that the interval flag has been set.  It seems that
> the interval flag necessary when the set shall contain a mixture of
> single IPv4 addresses and subnets with elements:
> 
> 	"Error: You must add 'flags interval' to your
> 	 set declaration if you want to add prefix elements"
> 
> and with auto-merge:
> 
> 	"Error: auto-merge only works with interval sets"
> 
> It is necessary to have auto-merge since the individual IPv4 addresses
> and subnets get added in an unpredictable manner and may overlap.
> 
> On 3/13/25 00:06, Pablo Neira Ayuso wrote:
> > The ruleset above provides sufficient context to infer that the
> > dynamic flag is needed, but that might not be the case in all
> > circunstances. The dynamic flag cannot be inferred in all cases like
> > the one above.
> > 
> > Without Lars' set declaration, the question is incomplete and it is
> > not easy to answer.
> 
> The following is basically the set up:
> 
> table ip foo {
>         set bar {
>                 type ipv4_addr
>                 flags interval
>                 auto-merge
>                 elements = { 192.168.2.0/24 }
>         }
> 	chain input {}
> }
> 
> The two important parts to retain are auto-merge and pre-defined
> elements (and be able to add to the elements later).

which together with your previous rule means that:

# nft add rule foo input tcp dport 22 counter add @bar { ip saddr }
Error: Could not process rule: Operation not supported
add rule foo input tcp dport 22 counter add @bar { ip saddr }
                                        ^^^^^^^^^^^^^^^^^^^^^

reports EOPNOTSUP because interval sets do not currently support
updates from the packet path.

Re: Dynamically appending addresses to a named set

[Lars Noodén]

On 3/13/25 10:59, Pablo Neira Ayuso wrote:
> reports EOPNOTSUP because interval sets do not currently support
> updates from the packet path.

Thanks.  Is there a way around that using nftables or is the main
workaround that I write it to a log and then have a script parse the log
file to then call nftables with the address settings?

/Lars