Re: [PATCH] netfilter: nf_tables: coalesce multiple notifications into one skbuff

[Phil Sutter <phil@nwl.cc>]

Hi Pablo,

On Thu, Aug 27, 2020 at 07:28:42PM +0200, Pablo Neira Ayuso wrote:
> On x86_64, each notification results in one skbuff allocation which
> consumes at least 768 bytes due to the skbuff overhead.
> 
> This patch coalesces several notifications into one single skbuff, so
> each notification consumes at least ~211 bytes, that ~3.5 times less
> memory consumption. As a result, this is reducing the chances to exhaust
> the netlink socket receive buffer.
> 
> Rule of thumb is that each notification batch only contains netlink
> messages whose report flag is the same, nfnetlink_send() requires this
> to do appropriately delivery to userspace, either via unicast (echo
> mode) or multicast (monitor mode).
> 
> The skbuff control buffer is used to annotate the report flag for later
> handling at the new coalescing routine.
> 
> The batch skbuff notification size is NLMSG_GOODSIZE, using a larger
> skbuff would allow for more socket receiver buffer savings (to amortize
> the cost of the skbuff even more), however, going over that size might
> break userspace applications, so let's be conservative and stick to
> NLMSG_GOODSIZE.

With this patch in place on top of your other one ("netfilter:
nfnetlink: nfnetlink_unicast() reports EAGAIN instead of ENOBUFS"),
firewalld (as well as nft with same input) now report:

| netlink: Error: Could not process rule: No space left on device

The JSON snippet causing the problem is indeed quite big though:

| % grep "\"\(add\|insert\)\"" fail.pp.json | wc -l
| 462

Eric told me he plans to split initial ruleset creation into several
chunks though, so this should at least not be a blocker for firewalld.

something seems to be fishy, though. Here's a reproducer:

| #!/bin/bash
| 
| numrules="$1"
| 
| nft flush ruleset
| (
| 	echo "add table t"
| 	echo "add chain t c"
| 	for ((i = 0; i < $numrules; i++)); do
| 		echo "add rule t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 mark 0x23 counter accept"
| 	done
| ) | nft -ef -

It starts failing at 13, which is not much. Interestingly, it fails outside of
the container, too. And it even echoes part of the commands:

| add table ip t
| add chain ip t c
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| add rule ip t c ip saddr 10.0.0.1 ip daddr 10.0.0.2 tcp dport 27374 meta mark 0x00000023 counter packets 0 bytes 0 accept
| netlink: Error: Could not process rule: No space left on device

Is this a bug in your patch?

Cheers, Phil