Re: [PATCH nf,v4] netfilter: nft_set_rbtree: revisit array resize logic

Chris Arges
2026-03-26T00:42:55Z

On 2026-03-23 12:29:30, Chris Arges wrote:
> On 2026-03-18 17:57:54, Pablo Neira Ayuso wrote:
> > On Wed, Mar 18, 2026 at 10:46:56AM -0500, Chris Arges wrote:
> > > On 2026-03-17 18:07:21, Pablo Neira Ayuso wrote:
> > > > Chris Arges reports high memory consumption with thousands of
> > > > containers, this patch revisits the array allocation logic.
> > > >
> > > > For anonymous sets, start by 16 slots (which takes 256 bytes on x86_64).
> > > > Expand it by x2 until threshold of 512 slots is reached, over that
> > > > threshold, expand it by x1.5.
> > > >
> > > > For non-anonymous set, start by 1024 slots in the array (which takes 16
> > > > Kbytes initially on x86_64). Expand it by x1.5.
> > > >
> > > > Use set->ndeact to subtract deactivated elements when calculating the
> > > > number of the slots in the array, otherwise the array size array gets
> > > > increased artifically. Add special case shrink logic to deal with flush
> > > > set too.
> > > >
> > > > The shrink logic is skipped by anonymous sets.
> > > >
> > > > Use check_add_overflow() to calculate the new array size.
> > > >
> > > > Add a WARN_ON_ONCE check to make sure elements fit into the new array
> > > > size.
> > > >
> > > > Reported-by: Chris Arges <carges@cloudflare.com>
> > > > Fixes: 7e43e0a1141d ("netfilter: nft_set_rbtree: translate rbtree to array for binary search")
> > > > Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
> > > > ---
> > > > v4: use maybe_grow: goto tag instead of grow:
> > > > Add note in commit description: "The shrink logic is skipped by anonymous sets."
> > > >
> > >
> > > I will be able to testing this more in depth early next week. Just to confirm,
> > > this patch requires this to be applied first?
> > > https://lore.kernel.org/netfilter-devel/20260307001124.2897063-1-pablo@netfilter.org/
> >
> > Yes, it requires that fix to be applied first.
>
> Thanks, these two patches applied on 6.18 stable show slab unreclaimable memory
> leveling out at 1.5GB for my local reproducer. I'll be deploying this to a
> wider set of machines for more real-world testing this week.
>
> So far seems good.
> --chris

We have deployed this successfully and memory usage looks good in production.

Tested-by: Chris Arges <carges@cloudflare.com>

--chris