[iptables PATCH 0/3] libxtables: Fix for pointless socket() calls

[Phil Sutter <phil@nwl.cc>]

The motivation for this series was a bug report claiming a near 100%
slowdown of iptables-restore when passed a large number of rules
containing conntrack match between two kernel versions. Turns out the
curlprit kernel change was within SELinux and in fact a performance
optimization, namely an introduced hash table mapping from security
context string to SID. This hash table insert, which happened for each
new socket, slowed iptables-restore down considerably.

The actual problem exposed by the above was that iptables-restore opens
a surprisingly large number of sockets when restoring said ruleset. This
stems from bugs in extension compatibility checks done during extension
registration (actually, "full registration").

One of the problems was that incompatible older revsions of an extension
never were never dropped from the pending list, and thus retried for
each rule using the extension. Coincidently, conntrack revision 0
matches this criteria.

Another problem was a (likely) accidental recursion of
xtables_fully_register_pending_*() via xtables_find_*(). In combination
with incompatible match revisions stuck in pending list, this caused
even more extra compatibility checks.

Solve all these problems by making pending extension lists sorted by
(descending) revision number. If at least one revision was compatible
with the kernel, any following incompatible ones may safely be dropped.
This should on one hand get rid of the repeated compatibility checks
while on the other maintain the presumptions stated in commit
3b2530ce7a0d6 ("xtables: Do not register matches/targets with
incompatible revision").

Patch 1 establishes the needed sorting in pending extension lists,
patch 2 then simplifies xtables_fully_register_pending_*() functions.
Patch 3 is strictly speaking not necessary but nice to have as it
streamlines array-based extension registrators with the extension
sorting.

Phil Sutter (3):
  libxtables: Make sure extensions register in revision order
  libxtables: Simplify pending extension registration
  libxtables: Register multiple extensions in ascending order

 libxtables/xtables.c | 206 +++++++++++++++++++++----------------------
 1 file changed, 99 insertions(+), 107 deletions(-)

-- 
2.28.0