Re: [PATCH nf,v4] netfilter: nft_set_rbtree: revisit array resize logic

[Chris Arges <carges@cloudflare.com>]

On 2026-03-18 17:57:54, Pablo Neira Ayuso wrote:
> On Wed, Mar 18, 2026 at 10:46:56AM -0500, Chris Arges wrote:
> > On 2026-03-17 18:07:21, Pablo Neira Ayuso wrote:
> > > Chris Arges reports high memory consumption with thousands of
> > > containers, this patch revisits the array allocation logic.
> > > 
> > > For anonymous sets, start by 16 slots (which takes 256 bytes on x86_64).
> > > Expand it by x2 until threshold of 512 slots is reached, over that
> > > threshold, expand it by x1.5.
> > > 
> > > For non-anonymous set, start by 1024 slots in the array (which takes 16
> > > Kbytes initially on x86_64). Expand it by x1.5.
> > > 
> > > Use set->ndeact to subtract deactivated elements when calculating the
> > > number of the slots in the array, otherwise the array size array gets
> > > increased artifically. Add special case shrink logic to deal with flush
> > > set too.
> > > 
> > > The shrink logic is skipped by anonymous sets.
> > > 
> > > Use check_add_overflow() to calculate the new array size.
> > > 
> > > Add a WARN_ON_ONCE check to make sure elements fit into the new array
> > > size.
> > > 
> > > Reported-by: Chris Arges <carges@cloudflare.com>
> > > Fixes: 7e43e0a1141d ("netfilter: nft_set_rbtree: translate rbtree to array for binary search")
> > > Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> > > ---
> > > v4: use maybe_grow: goto tag instead of grow:
> > >     Add note in commit description: "The shrink logic is skipped by anonymous sets."
> > > 
> > 
> > I will be able to testing this more in depth early next week. Just to confirm,
> > this patch requires this to be applied first?
> > https://lore.kernel.org/netfilter-devel/20260307001124.2897063-1-pablo@netfilter.org/
> 
> Yes, it requires that fix to be applied first.

Thanks, these two patches applied on 6.18 stable show slab unreclaimable memory
leveling out at 1.5GB for my local reproducer. I'll be deploying this to a
wider set of machines for more real-world testing this week.

So far seems good.
--chris