Re: [PATCH nf,v2] netfilter: nft_set_rbtree: allocate same array size on updates

[Chris Arges <carges@cloudflare.com>]

On 2026-03-07 01:11:24, Pablo Neira Ayuso wrote:
> The array resize function increments the size of the array in
> NFT_ARRAY_EXTRA_SIZE slots for each update, this is unnecesarily
> increasing the array size.
> 
> To determine the number of array slots:
> 
> - Use NFT_ARRAY_EXTRA_SIZE for new sets.
> - Use the current maximum number of intervals in the live array.
> 
> Reported-by: Chris Arges <carges@cloudflare.com>
> Fixes: 7e43e0a1141d ("netfilter: nft_set_rbtree: translate rbtree to array for binary search")
> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> ---
> v2: fix crash with new sets, reported by Florian.
> 
>  net/netfilter/nft_set_rbtree.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 

Pablo,

I was able to test with this patch applied on top of v6.18.16; however the
memory consumption remained high and similar to v6.18.16 without this patch.

My VM reproducer runs the services and checks for slab memory increases. In a
passing test case, the unreclaimable slab memory reaches about 1.4G and
stabilizes. In a failure test case unreclaimable slab memory goes beyond 4.4G
then the process gets OOM killed due to cgroup memory limits.

Also, using this reproducer I re-verified that this patch is what changes
slab memory.stat characteristics from within the cgroup:
- 7e43e0a1141d netfilter: nft_set_rbtree: translate rbtree to array for binary search

Finally, I reverted the following patches from v6.18.16 and re-tested using my
reproducer:
- 648946966a08 netfilter: nft_set_rbtree: validate open interval overlap
- 782f2688128e netfilter: nft_set_rbtree: validate element belonging to interval
- 35f83a75529a netfilter: nft_set_rbtree: don't gc elements on insert
- 5599fa810b50 netfilter: nft_set_rbtree: remove seqcount_rwlock_t
- 2aa34191f06f netfilter: nft_set_rbtree: use binary search array in get command
- 7e43e0a1141d netfilter: nft_set_rbtree: translate rbtree to array for binary search
In this instance memory allocations were again around 1.4G.

Any suggestions for other tests, I can rebuild with memory debugging config as
well.

Also, could there be an option here to opt-out of this performance optimization
in favor of retaining existing memory characteristics?

Thanks,
--chris