libnfnetlink dependency elimination

libnfnetlink dependency elimination

[Duncan Roe]

Some of these documented changes haven't happened yet.

Duncan Roe (1):
  doc: First update for libnfnetlink-based API over libmnl

 src/libnetfilter_queue.c | 56 +++++++++++++++++++++++++++-------------
 1 file changed, 38 insertions(+), 18 deletions(-)

-- 
2.35.8

[PATCH libnetfilter_queue 1/1] doc: First update for libnfnetlink-based API over libmnl

[Duncan Roe]

 - make it clear at the outset that there are 2 APIs
 - remove DEPRECATED tags, instead insert warning at top of these pages
 - update gdb options in compile line
 - remove the Library Setup line that follows
 - re-work how to increase default socket buffer size
   (i.e. other than by calling nfnl_rcvbufsiz()).

Signed-off-by: Duncan Roe <duncan_roe@optusnet.com.au>
---
 src/libnetfilter_queue.c | 56 +++++++++++++++++++++++++++-------------
 1 file changed, 38 insertions(+), 18 deletions(-)

diff --git a/src/libnetfilter_queue.c b/src/libnetfilter_queue.c
index bf67a19..54db391 100644
--- a/src/libnetfilter_queue.c
+++ b/src/libnetfilter_queue.c
@@ -41,12 +41,22 @@
  * libnetfilter_queue is a userspace library providing an API to packets that
  * have been queued by the kernel packet filter. It is is part of a system that
  * replaces the old ip_queue / libipq mechanism (withdrawn in kernel 3.5).
+ * \n
+ * libnetfilter_queue in fact offers 2 different APIs:
+ *   -# The modern API which provides helper functions for some
+ * libmnl functions. Users call other libmnl functions directly.
+ * The documentation calls this the **mnl** API.
+ *   -# An older API which provided wrappers for all relevant
+ * libnfnetlink functions.
+ * This API uses libmnl calls now, but its use in new software is discouraged.
+ * The documentation calls this the **nfnl** API.
+ * libnfnetlink itself is deprecated and will eventually be removed.
  *
  * libnetfilter_queue homepage is:
  * 	https://netfilter.org/projects/libnetfilter_queue/
  *
  <h1>Dependencies</h1>
- * libnetfilter_queue requires libmnl, libnfnetlink and a kernel that includes
+ * libnetfilter_queue requires libmnl and a kernel that includes
  * the Netfilter NFQUEUE over NFNETLINK interface (i.e. 2.6.14 or later).
  *
  * <h1>Main Features</h1>
@@ -86,18 +96,8 @@
  * nf-queue.c source file.
  * Simple compile line:
  * \verbatim
-gcc -g3 -ggdb -Wall -lmnl -lnetfilter_queue -o nf-queue nf-queue.c
+gcc -g3 -gdwarf-4 -Wall -lmnl -lnetfilter_queue -o nf-queue nf-queue.c
 \endverbatim
- *The doxygen documentation
- * \htmlonly
-<a class="el" href="group__LibrarySetup.html">LibrarySetup </a>
-\endhtmlonly
- * \manonly
-\fBLibrarySetup\fP\
-\endmanonly
- * is Deprecated and
- * incompatible with non-deprecated functions. It is hoped to produce a
- * corresponding non-deprecated (*Current*) topic soon.
  *
  * Somewhat outdated but possibly providing some insight into
  * libnetfilter_queue usage is the following
@@ -109,7 +109,7 @@ gcc -g3 -ggdb -Wall -lmnl -lnetfilter_queue -o nf-queue nf-queue.c
  * recv() may return -1 and errno is set to ENOBUFS in case that your
  * application is not fast enough to retrieve the packets from the kernel.
  * In that case, you can increase the socket buffer size by means of
- * nfnl_rcvbufsiz(). Although this delays the appearance of ENOBUFS errors,
+ * setsocketopt(). Although this delays the appearance of ENOBUFS errors,
  * you may hit it again sooner or later. The next section provides some hints
  * on how to obtain the best performance for your application.
  *
@@ -117,7 +117,11 @@ gcc -g3 -ggdb -Wall -lmnl -lnetfilter_queue -o nf-queue nf-queue.c
  * To improve your libnetfilter_queue application in terms of performance,
  * you may consider the following tweaks:
  *
- * - increase the default socket buffer size by means of nfnl_rcvbufsiz().
+ * - increase the default socket buffer size.
+ * Use setsocketopt() with SOL_SOCKET and SO_RCVBUFFORCE on the netlink socket
+ * fd returned by mnl_socket_get_fd()
+ * (software using the old nfnl API calls nfq_fd()).
+ * Software calling nfnl_rcvbufsiz() will continue to be supported.
  * - set nice value of your process to -20 (maximum priority).
  * - set the CPU affinity of your process to a spare core that is not used
  * to handle NIC interruptions.
@@ -247,7 +251,11 @@ struct nfnl_handle *nfq_nfnlh(struct nfq_handle *h)
 
 /**
  *
- * \defgroup Queue Queue handling [DEPRECATED]
+ * \defgroup Queue Queue handling
+ *
+ * \warning
+ * This page describes functions from the old nfnl API.
+ * Consider using the mnl API for new projects.
  *
  * Once libnetfilter_queue library has been initialised (See
  * \link LibrarySetup \endlink), it is possible to bind the program to a
@@ -335,7 +343,11 @@ int nfq_fd(struct nfq_handle *h)
  */
 
 /**
- * \defgroup LibrarySetup Library setup [DEPRECATED]
+ * \defgroup LibrarySetup Library setup
+ *
+ * \warning
+ * This page describes functions from the old nfnl API.
+ * Consider using the mnl API for new projects.
  *
  * Library initialisation is made in two steps.
  *
@@ -977,7 +989,11 @@ int nfq_set_verdict_mark(struct nfq_q_handle *qh, uint32_t id,
  *************************************************************/
 
 /**
- * \defgroup Parsing Message parsing functions [DEPRECATED]
+ * \defgroup Parsing Message parsing functions
+ *
+ * \warning
+ * This page describes functions from the old nfnl API.
+ * Consider using the mnl API for new projects.
  *
  * \manonly
 .SH SYNOPSIS
@@ -1385,7 +1401,11 @@ do {								\
 } while (0)
 
 /**
- * \defgroup Printing Printing [DEPRECATED]
+ * \defgroup Printing Printing
+ *
+ * \warning
+ * This page describes functions from the old nfnl API.
+ * Consider using the mnl API for new projects.
  *
  * \manonly
 .SH SYNOPSIS
-- 
2.35.8

Re: libnfnetlink dependency elimination

[Pablo Neira Ayuso]

On Sun, Nov 12, 2023 at 05:59:21PM +1100, Duncan Roe wrote:
> Some of these documented changes haven't happened yet.

Then we have to start by changes first, not the other way around.

Re: libnfnetlink dependency elimination (doc)

[Duncan Roe]

Hi Pablo,

On Tue, Nov 14, 2023 at 04:24:11PM +0100, Pablo Neira Ayuso wrote:
> On Sun, Nov 12, 2023 at 05:59:21PM +1100, Duncan Roe wrote:
> > Some of these documented changes haven't happened yet.
>
> Then we have to start by changes first, not the other way around.

Yes I know that, obviously:)

The point here is that nfnl_rcvbufsiz() has been advertised in the main page of
libnetfilter_queue HTML for a long time and there are likely a number of systems
out there that use it. When libnfnetlink is removed, libnetfilter_queue will
have to provide nfnl_rcvbufsiz() or those systems will start failing.

I have in mind that although libnetfilter_queue will provide nfnl_rcvbufsiz(),
there will be no documentation for it.

You will see in
https://patchwork.ozlabs.org/project/netfilter-devel/patch/20231112065922.3414-2-duncan_roe@optusnet.com.au/
I replaced the advice to use nfnl_rcvbufsiz() (in 2 places) with advice to use
setsocketopt(). I only mentioned that programs calling nfnl_rcvbufsiz() will
continue to run.

So I offered this patch as the only documentation of how to use
nfnl_rcvbufsiz(). I need it for my testing, but it's fine with me if you don't
want to take it.

Cheers ... Duncan.

*** Scrub that last message

[Duncan Roe]

Sorry, that was reply to another message

Re: libnfnetlink dependency elimination (doc)

[Duncan Roe]

Hi Pablo,

On Tue, Nov 14, 2023 at 04:24:11PM +0100, Pablo Neira Ayuso wrote:
> On Sun, Nov 12, 2023 at 05:59:21PM +1100, Duncan Roe wrote:
> > Some of these documented changes haven't happened yet.
>
> Then we have to start by changes first, not the other way around.

It's my development model to document before coding. That way:
 - I know what I'm working towards
 - The documentation often improves with time. That would be unlikely if I left
   it until last

Could you therefore just treat these patches as RFC (for now) and reply if you'd
like to see changes,

There wiil be a v2 of this one with a more positive take on using the nfnl API.

Cheers ... Duncan.

Re: libnfnetlink dependency elimination (doc)

[Pablo Neira Ayuso]

On Wed, Nov 15, 2023 at 10:04:47AM +1100, Duncan Roe wrote:
> Hi Pablo,
> 
> On Tue, Nov 14, 2023 at 04:24:11PM +0100, Pablo Neira Ayuso wrote:
> > On Sun, Nov 12, 2023 at 05:59:21PM +1100, Duncan Roe wrote:
> > > Some of these documented changes haven't happened yet.
> >
> > Then we have to start by changes first, not the other way around.
> 
> Yes I know that, obviously:)
> 
> The point here is that nfnl_rcvbufsiz() has been advertised in the main page of
> libnetfilter_queue HTML for a long time and there are likely a number of systems
> out there that use it. When libnfnetlink is removed, libnetfilter_queue will
> have to provide nfnl_rcvbufsiz() or those systems will start failing.

There is nfq_fd() and setsockopt() that you can use:

        setsockopt(fd, SOL_SOCKET, SO_RCVBUFFORCE, &size, socklen);
        setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &size, socklen);

libnfnetlink is **deprecated** and it will be removed at some point,
the git log shows that it has less and less users.

main libnfnetlink users are the libnetfilter_* libraries.

No new application should be using libnfnetlink in 2023.

Re: libnfnetlink dependency elimination (doc)

[Duncan Roe]

On Wed, Nov 15, 2023 at 10:27:34AM +0100, Pablo Neira Ayuso wrote:
> On Wed, Nov 15, 2023 at 10:04:47AM +1100, Duncan Roe wrote:
> > Hi Pablo,
> >
> > On Tue, Nov 14, 2023 at 04:24:11PM +0100, Pablo Neira Ayuso wrote:
> > > On Sun, Nov 12, 2023 at 05:59:21PM +1100, Duncan Roe wrote:
> > > > Some of these documented changes haven't happened yet.
> > >
> > > Then we have to start by changes first, not the other way around.
> >
> > Yes I know that, obviously:)
> >
> > The point here is that nfnl_rcvbufsiz() has been advertised in the main page of
> > libnetfilter_queue HTML for a long time and there are likely a number of systems
> > out there that use it. When libnfnetlink is removed, libnetfilter_queue will
> > have to provide nfnl_rcvbufsiz() or those systems will start failing.
>
> There is nfq_fd() and setsockopt() that you can use:
>
>         setsockopt(fd, SOL_SOCKET, SO_RCVBUFFORCE, &size, socklen);
>         setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &size, socklen);

Yes
>
> libnfnetlink is **deprecated** and it will be removed at some point,
> the git log shows that it has less and less users.

Good to know
>
> main libnfnetlink users are the libnetfilter_* libraries.

Yes
>
> No new application should be using libnfnetlink in 2023.

Yes

But please see thread starting
> [PATCH libnetfilter_queue] utils: Add example of setting socket buffer size
I meant to send my reply to that email, not this one.

Again sorry for the confusion I've caused.

Cheers ... Duncan.

Re: libnfnetlink dependency elimination (doc)

[Pablo Neira Ayuso]

On Wed, Nov 15, 2023 at 10:43:52PM +1100, Duncan Roe wrote:
> On Wed, Nov 15, 2023 at 10:27:34AM +0100, Pablo Neira Ayuso wrote:
> > On Wed, Nov 15, 2023 at 10:04:47AM +1100, Duncan Roe wrote:
> > > Hi Pablo,
> > >
> > > On Tue, Nov 14, 2023 at 04:24:11PM +0100, Pablo Neira Ayuso wrote:
> > > > On Sun, Nov 12, 2023 at 05:59:21PM +1100, Duncan Roe wrote:
> > > > > Some of these documented changes haven't happened yet.
> > > >
> > > > Then we have to start by changes first, not the other way around.
> > >
> > > Yes I know that, obviously:)
> > >
> > > The point here is that nfnl_rcvbufsiz() has been advertised in the main page of
> > > libnetfilter_queue HTML for a long time and there are likely a number of systems
> > > out there that use it. When libnfnetlink is removed, libnetfilter_queue will
> > > have to provide nfnl_rcvbufsiz() or those systems will start failing.
> >
> > There is nfq_fd() and setsockopt() that you can use:
> >
> >         setsockopt(fd, SOL_SOCKET, SO_RCVBUFFORCE, &size, socklen);
> >         setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &size, socklen);
> 
> Yes
> >
> > libnfnetlink is **deprecated** and it will be removed at some point,
> > the git log shows that it has less and less users.
> 
> Good to know
> >
> > main libnfnetlink users are the libnetfilter_* libraries.
> 
> Yes
> >
> > No new application should be using libnfnetlink in 2023.
> 
> Yes
> 
> But please see thread starting
> > [PATCH libnetfilter_queue] utils: Add example of setting socket buffer size
> I meant to send my reply to that email, not this one.
> 
> Again sorry for the confusion I've caused.

Apologies, I got lost. Huge burden here.

I suggest to restart from scratch and only one submission and thread at a time :)

Thanks.