Re: [PATCH v6 2/2] netfilter: nfnetlink_queue: optimize verdict lookup with hash table

[Florian Westphal <fw@strlen.de>]

Scott Mitchell <scott.k.mitch1@gmail.com> wrote:
> > > > +#define NFQNL_HASH_MAX_SIZE        131072
> > >
> > > Is there a use case for such a large table?
> >
> > Order of magnitude goal is to gracefully handle 64k verdicts in a
> > queue (w/ out of order verdicting).
> > Ouch.  I fear this will need way more work, we will have to implement
> > some form of memory accounting for the queued skbs, e.g. by tracking
> > queued bytes instead of queue length.
> >
> > nfqueue comes from a time when GSO did not exist, now even a single
> > skb can easily have 2mb worth of data.
> 
> I agree byte-based memory accounting would be valuable for preventing
> memory exhaustion with large queues (especially with GSO). However, I
> believe this is orthogonal to the hash verdict lookup optimization
> (hash table itself has bounded memory overhead, skb memory pressure
> exists today with the linear list). Does that align with your
> thinking?

Yes, this is an existing bug.

> For my use case, packet sizes are bounded and NFQA_CFG_QUEUE_MAXLEN
> provides sufficient protection.

Its sufficient for cooperative use cases only, we have to get
rid of NFQA_CFG_QUEUE_MAXLEN (resp. translate it to a byte
approximation) soon.

If you have time it would be good if you could followup.
If not, I can see if I can make cycles available to do this.

Unfotunately its not that simple due to 64k queues, so the
accouting will have to be pernet and not per queue.