Advantage(s) of static over dynamic nftables sets?

Advantage(s) of static over dynamic nftables sets?

[Frank Myhr]

Hi,

As an nftables newbie I was a bit surprised to discover that defining a 
set as static prevents adding or deleting elements not only from the 
packet path but also from the nft command line:

#  nft add element ip ip_filter static_set { a.b.c.d }
Error: Could not process rule: Device or resource busy

Which is easily remedied by defining the set as dynamic instead.

So now I wonder: why not define every set as dynamic? Which would allow 
modification of any set's elements without having to reload the entire 
firewall -- thereby preserving accumulated counters and other stateful 
objects. Would performance and/or memory usage take a significant hit by 
doing this?

Thanks,
Frank

Re: Advantage(s) of static over dynamic nftables sets?

[Florian Westphal]

Frank Myhr <fmyhr@fhmtech.com> wrote:
> As an nftables newbie I was a bit surprised to discover that defining a set
> as static prevents adding or deleting elements not only from the packet path
> but also from the nft command line:
> 
> #  nft add element ip ip_filter static_set { a.b.c.d }
> Error: Could not process rule: Device or resource busy

Yes, such set is immutable.

> Which is easily remedied by defining the set as dynamic instead.
> 
> So now I wonder: why not define every set as dynamic?

Sets that are made static allow kernel to pick a more efficient
representation for the set type.

> Which would allow
> modification of any set's elements without having to reload the entire
> firewall -- thereby preserving accumulated counters and other stateful
> objects. Would performance and/or memory usage take a significant hit by
> doing this?

I don't think so, but this will probably depend a lot on the
system in question and on the type of elements stored.

Re: Advantage(s) of static over dynamic nftables sets?

[Frank Myhr]

On 2020/03/18 17:35, Florian Westphal wrote:
> Frank Myhr <fmyhr@fhmtech.com> wrote:
>> So now I wonder: why not define every set as dynamic?
> 
> Sets that are made static allow kernel to pick a more efficient
> representation for the set type.

Thanks, Florian. Good to know.


>> Would performance and/or memory usage take a significant hit by
>> [defining all sets dynamic]?
> 
> I don't think so, but this will probably depend a lot on the
> system in question and on the type of elements stored.

Makes sense, thanks. I suppose efficiency gets progressively worse for 
element types with larger possible ranges (# of bits), something like?:

inet_proto < inet_service < mark, ipv4_addr < ether_addr < ipv6_addr

I suppose intervals are more efficient than an equivalent group of 
single elements?

Is a concatenation like ipv4_addr . inet_service as efficient as a pure 
data type with the same number of bits?

Thanks,
Frank

Re: Advantage(s) of static over dynamic nftables sets?

[Florian Westphal]

Frank Myhr <fmyhr@fhmtech.com> wrote:
> > > Would performance and/or memory usage take a significant hit by
> > > [defining all sets dynamic]?
> > 
> > I don't think so, but this will probably depend a lot on the
> > system in question and on the type of elements stored.
> 
> Makes sense, thanks. I suppose efficiency gets progressively worse for
> element types with larger possible ranges (# of bits), something like?:

Right now (assuming static), size <= 16 bits is most efficient,
then <= 32 bit, rest uses hash table.

> I suppose intervals are more efficient than an equivalent group of single
> elements?

Memory-wise yes, performance-wise no.

> Is a concatenation like ipv4_addr . inet_service as efficient as a pure data
> type with the same number of bits?

Yes, as it makes no difference for the storage.  The kernel doesn't know
what its storing, it only knows the size of the element.

It does store a 'type' information, but that is only used by nftables
so it knows how to format the elements when listing the ruleset.

Re: Advantage(s) of static over dynamic nftables sets?

[Frank Myhr]

On 2020/03/18 18:51, Florian Westphal wrote:

> Right now (assuming static), size <= 16 bits is most efficient,
> then <= 32 bit, rest uses hash table.

Very good to know. Thanks so much.


>> I suppose intervals are more efficient than an equivalent group of single
>> elements?
> 
> Memory-wise yes, performance-wise no.

Also good to know.


>> Is a concatenation like ipv4_addr . inet_service as efficient as a pure data
>> type with the same number of bits?
> 
> Yes, as it makes no difference for the storage.  The kernel doesn't know
> what its storing, it only knows the size of the element.
> 
> It does store a 'type' information, but that is only used by nftables
> so it knows how to format the elements when listing the ruleset.

Thanks for confirming, and for all of your detailed answers.

If you find the time (and if it's comprehensible to a non-kernel 
programmer): How does a dynamic set change the above situation? Does it 
use a hash table for everything?

Thanks,
Frank

Re: Advantage(s) of static over dynamic nftables sets?

[Florian Westphal]

Frank Myhr <fmyhr@fhmtech.com> wrote:
> > > I suppose intervals are more efficient than an equivalent group of single
> > > elements?
> > 
> > Memory-wise yes, performance-wise no.
> 
> Also good to know.
> 
> > > Is a concatenation like ipv4_addr . inet_service as efficient as a pure data
> > > type with the same number of bits?
> > 
> > Yes, as it makes no difference for the storage.  The kernel doesn't know
> > what its storing, it only knows the size of the element.
> > 
> > It does store a 'type' information, but that is only used by nftables
> > so it knows how to format the elements when listing the ruleset.
> 
> Thanks for confirming, and for all of your detailed answers.
> 
> If you find the time (and if it's comprehensible to a non-kernel
> programmer): How does a dynamic set change the above situation? Does it use
> a hash table for everything?

Yes, dynamic set always uses 'rhashtable' backend, thats the only type
that supports insertion from the packet path.

We have 3 hash tables, a simple one for 32bit keys, one for static sets
(table size decided at creation time) and rhashtable (table size is
auto-resized as needed).

Re: Advantage(s) of static over dynamic nftables sets?

[Frank Myhr]

On 2020/03/18 20:08, Florian Westphal wrote:

> Yes, dynamic set always uses 'rhashtable' backend, thats the only type
> that supports insertion from the packet path.
> 
> We have 3 hash tables, a simple one for 32bit keys, one for static sets
> (table size decided at creation time) and rhashtable (table size is
> auto-resized as needed).

Thank you! I now have a much better idea of what's going on "behind the 
curtain." I'll stop pestering you with questions, for now. :-)