Re: Combine ipv4 and ipv6 in a set

["Kerin Millar" <kfm@plushkava.net>]

Hi Slavko,

On Thu, 1 Feb 2024, at 10:50 AM, Slavko wrote:
> Dňa 31. 1. o 23:10 Kerin Millar napísal(a):
>
>> This has also annoyed me on several occasions. Though I have been using nftables for a fairly long time, I still find it more natural to organise rulesets based on the conventions of iptables. Old habits die hard, as the saying goes. Incidentally, there is an open bug concerning this.
>
> Perhaps you know that, but IMO it can be useful for some...
>
> IMO the root of confusion comes from fact, that both (iptables/nft) uses 
> the same name -- "table". I don't want to discuss if it was or wasn't 
> good decision, as it doesn't matter, it is only name.
>
> In iptables table more or less defines chain's hook priority:
>
>     CHAIN => hook type
>     TABLE => hook priority
>
> In nftables one have (can) to define hook type and its priority by self, 
> thus that purpose of ipt TABLEs is gone and main purpose of table 
> changed to define family (and group objects) and name it.

Quite. It's nothing more than an "object" namespace with an associated address family. In what way it resembles a table, I do not know.

>
> I fight with that hook's priorities for long time, i was not able to get 
> it, until symbolic names for priorities was introduced. And after math 
> can be used in its definition (eg. filter + 5) it is really simple now. 
> Great job.
>
>>> BTW, i am curious, how big difference (in performance/time) is to
>>> search IPv4 vs. IPv6 address in set? Is it worth to consider?
>> 
>> I do not know but I would expect a lookup to be very efficient in both cases.
>
> I know only very little how ipsets stores items internally, mostly only 
> that it uses hashes and tries, and i know nothing about sets. I know, 
> that these tries are efficient way to search in large datasets, with 
> near to constant search time. But if someone complains about performance 
> (IP vs. IPv6) of ipsets/sets, there must be difference. The question is, 
> if it is important/significant.
>
>> I haven't paid much attention to memory usage. Benchmarking performance is fairly straightforward with the use of the shell, especially bash because it offers some useful features such as:
>
> I mean performance in packet flow, how long it tooks to find (or not 
> find) match or update/add from FW rule with ipset and with set, if both 
> uses the same dataset size. If size of ipset/set matter, etc. Will be 
> measured command time the same for particular action as from packet 
> flow? That comparison will reveal, if sets are real replacement of 
> ipsets in all (or most) cases or not. Or will be adding/removing item(s) 
> from command line time the same as from packet flow? (IMO not, as 
> command invocation plays role too)

I don't think that there is an abundance of material on this topic but I recall seeing one or two articles on the web concerning this sort of performance measurement (by well known Netfilter developers/contributors).

>
> BTW i recently realized how important can be command time, first some 
> attack (which fills ipsets with thousands IPs), then kernel update, thus 
> restart was needed, and it reveals how inefficient way fail2ban uses to 
> restore ipset bans :-D But that is not nft related (and it was short 
> time shock)...
>
>> You might find it interesting to look at existing issues concerning the use of sets by visiting the following link.
>
> thanks, i will check them from time to time...
>
>> Of those, #1584 concerns "high memory requirements", where Pablo appears to be using https://valgrind.org/docs/manual/ms-manual.html to profile memory usage.
>
> I meet valgrind only two or three times, related to some bug finding, 
> but i got full guide from devs (do that, run that and post that or so), 
> without understanding what i am doing...
>
> Ipset shows in output (list) how much memory particular set uses, can i 
> get that info from set listing (to compare)?

Try:

nft -t list sets

Alternatively:

nft -jt list sets | jq -r '.nftables[1:] | .[].set | [ .table, .family, .name, .size ] | @csv'

The problem is that the size isn't always shown. I complained about it here:

https://bugzilla.netfilter.org/show_bug.cgi?id=1717.

-- 
Kerin Millar