Re: Is there an efficient way to delete multiple elements from a set?

["Kerin Millar" <kfm@plushkava.net>]

On Wed, 31 Jan 2024, at 8:14 AM, Anton wrote:
> Hello, I've been experimenting with nftables sets for the purpose of
> geoip blocking. Let's say I'd like to add ip blocks for multiple
> countries to a blacklist or to a whitelist. Perhaps the most efficient
> way to do that would be by combining all required ip blocks in one set
> (for each family). However since country ip blocks are a moving
> target, I would need to regularly refresh parts of that set. My idea
> was to delete all ip addresses corresponding to an ip block from the
> set and then add the updated ip block. The problem is, this is very
> slow. While adding an ip block takes (in my VM) 0.09s, deleting all
> ip's from that same block takes 14.5s.
>
> This is how I'm doing the deletion and the time measurement:
> printf '%s\n' "delete element inet test testset { $(cat test.set) };"
> | /usr/bin/time -f %es nft -f -
>
> (the test.set file stores a comma-separated list of subnets)
>
> Is there a more efficient way to do this? I could of course flush the
> set and rebuild it every time I need to update some part of it, but I
> thought I'd ask before deciding to implement that.

Indeed there is. Your method isn't optimal because the command substitution causes the shell to needlessly assign memory for the sole purpose of containing the entirety of the output of cat(1). Further, cat(1) will have to complete before the printf builtin can even be executed, which defeats the asynchronicity of the pipeline. You should feed nft(8) with a command list instead.

{
	printf 'delete element inet test testset { '
	cat test.set
	printf ' }'
} | nft -f -

If you would prefer to express it as a one-liner then you may, in which case you must also terminate each of the three commands in the { command list } with a semicolon.

-- 
Kerin Millar