Re: nfnetlink_queue -- why linear lookup ?

[Pablo Neira Ayuso <pablo@netfilter.org>]

On Sun, Aug 15, 2021 at 08:47:04PM +0200, alexandre.ferrieux@orange.com wrote:
> 
> 
> On 8/15/21 4:12 PM, Pablo Neira Ayuso wrote:
> > On Sun, Aug 15, 2021 at 03:32:30PM +0200, alexandre.ferrieux@orange.com wrote:
> >>
> > > [...] I was just worried that people would >> object to adding even the slightest overhead (hash_add/hash_del) to the
> > > existing code path, that satisfies 99% of uses (LIFO). What do you think ?
> > 
> > It should be possible to maintain both the list and the hashtable,
> > AFAICS, the batch callback still needs the queue_list.
> 
> Yes, but to maintain the hashtable, we need to bother the "normal" code path
> with hash_add/del. Not much, but still, some overhead...

Probably you can collect some numbers to make sure this is not a
theoretical issue.

> > > > > PS: what is the intended dominant use case for batch verdicts ?
> > > > > Issuing a batch containing several packets helps to amortize the
> > > > cost of the syscall.
> > > 
> > > Yes, but that could also be achieved by passing an array of ids.
> > 
> > You mean, one single sendmsg() with several netlink messages, that
> > would also work to achieve a similar batching effect.
> 
> Yes, a full spectrum of batching methods are possible. If we're to minimize
> the number of bytes crossing the kernel/user boundary though, an array of
> ids looks like the way to go (4 bytes per packet, assuming uint32 ids).

Are you proposing a new batching mechanism?

> > > The extra constraint of using a (contiguous) range means that there
> > > is no outlier.  This seems to imply that ranges are no help when
> > > flows are multiplexed. Or maybe, the assumption was that bursts tend
> > > to be homogeneous ?
> > 
> > What is your usecase?
> 
> For O(1) lookup:
> 
> My primary motivation is for transparent proxies and userland qdiscs. In
> both cases, specific packets must be held for some time and reinjected at a
> later time which is not computed by a simple, fixed delay, but rather
> triggered by some external event.
> 
> My secondary motivation is that the netfilter queue is a beautifully
> asynchronous mechanism, and absolutely nothing in its definition limits it
> to the dumb FIFO-of-instantaneous-drop-decisions that seems to dominate
> sample code.

I see. Thanks for telling me.

> For the deprecation of id-range-based batching:
> 
> It seems that as soon as two different packet streams are muxed in the
> queue, one deserving verdict A and the other verdict B, contiguous id ranges
> of a given verdict may be very small. But I realize I'm 20 years late to
> complain :)

As I said, you can place several netlink messages in one single
sendmsg() call, then they do not need to be contiguous.

> That being said, the Doxygen of the userland nfqueue API mentions being
> DEPRECATED... So what is the recommended replacement ?

What API are you refering to specifically?