netfilter-devel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH nf-next 00/11] netfilter: nf_tables: reduce transaction log memory usage
@ 2024-05-13 13:00 Florian Westphal
  2024-05-13 13:00 ` [PATCH nf-next 01/11] netfilter: nf_tables: make struct nft_trans first member of derived subtypes Florian Westphal
                   ` (10 more replies)
  0 siblings, 11 replies; 20+ messages in thread
From: Florian Westphal @ 2024-05-13 13:00 UTC (permalink / raw)
  To: netfilter-devel; +Cc: Florian Westphal

The transaction log can grow to huge values.
Insertion of 1.000.000 elements into a set, or flushing a set with
1.000.000 elements will eat 128 byte per element, i.e. 128 MiBi.

This series compacts the structures. After this series, struct
nft_trans_elem can be allocated from kmalloc-96 slab, resulting
in a 25% memory reduction.

To further reduce flush/mass-insert several approaches come
to mind:

1. allow struct nft_trans_elem to hold several elements.
2. add a kernel-internal, dedicated nft_trans_elem_batch that
   is only used for flushing (similar to 1).
3. Remove 'struct net' from nft_trans struct.  This reduces
   size of nft_trans_elem to 64 bytes, which would halve memory
   needs compared to the current state.

I have tried to do 3), its possible but not very elegant.

You can have a look at the general idea at
https://git.kernel.org/pub/scm/linux/kernel/git/fwestphal/nf-next.git/commit/?h=nft_trans_compact_01&id=5269e591563204490b9fad6ae1e33810a9f4c39d

I have started to look at 1) too, but unlike this compaction
series it looks like this will make things even more complex
as we'll need to be careful wrt. appending more set elements to
an already-queued nft_trans_elem (must be same msg_type, same set,
etc).

This series has seen brief testing with kasan+kmemleak and
nftables.git selftests.

Feedback and comments welcome.

Florian Westphal (11):
  netfilter: nf_tables: make struct nft_trans first member of derived subtypes
  netfilter: nf_tables: move bind list_head into relevant subtypes
  netfilter: nf_tables: compact chain+ft transaction objects
  netfilter: nf_tables: reduce trans->ctx.table references
  netfilter: nf_tables: pass nft_chain to destroy function, not nft_ctx
  netfilter: nf_tables: pass more specific nft_trans_chain where possible
  netfilter: nf_tables: avoid usage of embedded nft_ctx
  netfilter: nf_tables: store chain pointer in rule transaction
  netfilter: nf_tables: reduce trans->ctx.chain references
  netfilter: nf_tables: pass nft_table to destroy function
  netfilter: nf_tables: do not store nft_ctx in transaction objects

 include/net/netfilter/nf_tables.h | 152 +++++++----
 net/netfilter/nf_tables_api.c     | 402 +++++++++++++++++-------------
 net/netfilter/nf_tables_offload.c |  40 +--
 net/netfilter/nft_immediate.c     |   2 +-
 4 files changed, 363 insertions(+), 233 deletions(-)

-- 
2.43.2


^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2024-06-26 11:28 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-05-13 13:00 [PATCH nf-next 00/11] netfilter: nf_tables: reduce transaction log memory usage Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 01/11] netfilter: nf_tables: make struct nft_trans first member of derived subtypes Florian Westphal
2024-06-18  8:28   ` Pablo Neira Ayuso
2024-06-18  9:20     ` Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 02/11] netfilter: nf_tables: move bind list_head into relevant subtypes Florian Westphal
2024-06-18  8:24   ` Pablo Neira Ayuso
2024-06-18  9:21     ` Florian Westphal
2024-06-24 19:16   ` Pablo Neira Ayuso
2024-06-24 21:18     ` Florian Westphal
2024-06-25 18:49       ` Pablo Neira Ayuso
2024-06-26 11:28         ` Pablo Neira Ayuso
2024-05-13 13:00 ` [PATCH nf-next 03/11] netfilter: nf_tables: compact chain+ft transaction objects Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 04/11] netfilter: nf_tables: reduce trans->ctx.table references Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 05/11] netfilter: nf_tables: pass nft_chain to destroy function, not nft_ctx Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 06/11] netfilter: nf_tables: pass more specific nft_trans_chain where possible Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 07/11] netfilter: nf_tables: avoid usage of embedded nft_ctx Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 08/11] netfilter: nf_tables: store chain pointer in rule transaction Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 09/11] netfilter: nf_tables: reduce trans->ctx.chain references Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 10/11] netfilter: nf_tables: pass nft_table to destroy function Florian Westphal
2024-05-13 13:00 ` [PATCH nf-next 11/11] netfilter: nf_tables: do not store nft_ctx in transaction objects Florian Westphal

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).