Re: [PATCH v6 2/2] netfilter: nfnetlink_queue: optimize verdict lookup with hash table

[Florian Westphal <fw@strlen.de>]

Scott Mitchell <scott.k.mitch1@gmail.com> wrote:
> > > +#define NFQNL_HASH_MAX_SIZE        131072
> >
> > Is there a use case for such a large table?
> 
> Order of magnitude goal is to gracefully handle 64k verdicts in a
> queue (w/ out of order verdicting).

Ouch.  I fear this will need way more work, we will have to implement
some form of memory accounting for the queued skbs, e.g. by tracking
queued bytes instead of queue length.

nfqueue comes from a time when GSO did not exist, now even a single
skb can easily have 2mb worth of data.

> > What is the deal-breaker wrt. rhashtable so that one would start to
> > reimplement the features it already offers?
> 
> Agreed if global rhashtable is within the ballpark of v6 performance
> it would be preferred. I've implemented the global rhashtable approach
> locally and I've also implemented an isolated test harness to assess
> performance so we have data to drive the decision.
>
> I captured the rationale for current approach here:
> https://lore.kernel.org/netfilter-devel/CAFn2buB-Pnn_kXFov+GEPST=XCbHwyW5HhidLMotqJxYoaW-+A@mail.gmail.com/#t.

OK, but I'm not keen on maintaining an rhashtable clone in nfqueue.

If the shrinker logic in rhashtable has bad effects then
maybe its better to extend rhashtable first so its behaviour can
be influenced better, e.g. by adding a delayed shrink process that
is canceled when the low watermark is below threshold for less than
X seconds.