expose named set element count

expose named set element count

[Trevor Cordes]

Hi!  I have an application that can create very large named sets
(dynamically) of IPs.  For monitoring and testing purposes it is useful to
get a count of elements currently in a set.

Currently I an running "nft list table inet mytable |wc".  But once the ip
set grows to much over 100k IPs, that command can take around 90 minutes
on my system!  During that time the nft command takes 98% CPU.  That makes
monitoring in this manner nearly impossible.

I think the in-kernel code tracks the current number of elements.  Can we
expose this through the Netlink interface so users can get an instant
count of elements with virtually 0 CPU usage?  (Note, I'm not talking the
"size" data point, I'm talking the actual IPs currently in the set.)

Less importantly, I'm wondering what is making nft take so long to list
these huge sets.  Maybe the IP numeric sorting and formatting?  If it's
all the sorting, maybe an nft option to disable sorting would help?  Or an
option to disable formatting and have nft just dump the whole wad of IPs
on one line per set?

Thanks so much.  Please CC: my on replies because I'm not sure my
subscribe request is working or going anywhere.  The listserv may not like
my mail smarthost.

Re: expose named set element count

[Pablo Neira Ayuso]

On Tue, Oct 20, 2026 at 08:17:44AM -0500, Trevor Cordes wrote:
> Hi!  I have an application that can create very large named sets
> (dynamically) of IPs.  For monitoring and testing purposes it is useful to
> get a count of elements currently in a set.
> 
> Currently I an running "nft list table inet mytable |wc".  But once the ip
> set grows to much over 100k IPs, that command can take around 90 minutes
> on my system!  During that time the nft command takes 98% CPU.  That makes
> monitoring in this manner nearly impossible.
> 
> I think the in-kernel code tracks the current number of elements.  Can we
> expose this through the Netlink interface so users can get an instant
> count of elements with virtually 0 CPU usage?  (Note, I'm not talking the
> "size" data point, I'm talking the actual IPs currently in the set.)
> 
> Less importantly, I'm wondering what is making nft take so long to list
> these huge sets.  Maybe the IP numeric sorting and formatting?  If it's
> all the sorting, maybe an nft option to disable sorting would help?  Or an
> option to disable formatting and have nft just dump the whole wad of IPs
> on one line per set?
> 
> Thanks so much.  Please CC: my on replies because I'm not sure my
> subscribe request is working or going anywhere.  The listserv may not like
> my mail smarthost.

There is a count field in more recent userspace nft and kernels, what
nft and kernel versions are you running?

Re: expose named set element count

[Pablo Neira Ayuso]

On Tue, Oct 20, 2026 at 06:35:31PM +0200, Pablo Neira Ayuso wrote:
> On Tue, Oct 20, 2026 at 08:17:44AM -0500, Trevor Cordes wrote:
> > Hi!  I have an application that can create very large named sets
> > (dynamically) of IPs.  For monitoring and testing purposes it is useful to
> > get a count of elements currently in a set.
> > 
> > Currently I an running "nft list table inet mytable |wc".  But once the ip
> > set grows to much over 100k IPs, that command can take around 90 minutes
> > on my system!  During that time the nft command takes 98% CPU.  That makes
> > monitoring in this manner nearly impossible.
> > 
> > I think the in-kernel code tracks the current number of elements.  Can we
> > expose this through the Netlink interface so users can get an instant
> > count of elements with virtually 0 CPU usage?  (Note, I'm not talking the
> > "size" data point, I'm talking the actual IPs currently in the set.)
> > 
> > Less importantly, I'm wondering what is making nft take so long to list
> > these huge sets.  Maybe the IP numeric sorting and formatting?  If it's
> > all the sorting, maybe an nft option to disable sorting would help?  Or an
> > option to disable formatting and have nft just dump the whole wad of IPs
> > on one line per set?
> > 
> > Thanks so much.  Please CC: my on replies because I'm not sure my
> > subscribe request is working or going anywhere.  The listserv may not like
> > my mail smarthost.
> 
> There is a count field in more recent userspace nft and kernels, what
> nft and kernel versions are you running?

nft list ruleset
table ip t {
        set s {
                type ipv4_addr
                size 10 # count 1
                counter
                elements = { 1.1.1.1 counter packets 1 bytes 11 }
        }
}

Userspace supports is available since nftables 1.1.5:
$ git describe --contains 8ff87c8fd483eb3fdab5839cccaf100b593fe92e
v1.1.4~48

And Linux kernel needs to >= v6.17:
$ git describe --contains 0014af802193a
v6.16-rc1~132^2~190^2