can libnetfilter_conntrack be used to write a userspace connection tracker?

can libnetfilter_conntrack be used to write a userspace connection tracker?

[Sam Roberts]

I'm working on a connection tracker for a RPC-like protocol (over TCP).

I believe that by inspecting packets using nfqueue, and
creating/destroying expectations using nfconntrack, I can do a
connection tracker in user-space.

In order to remove nfqueue from the mix, I've been looking at the
conntrack code, trying to figure out whether even notifications about
connection status can include the TCP data that I need to inspect, the
data that's in the skbs provided to kernel module conntrack helpers. I
haven't been able to be certain what libnfconntrack can/cannot do, but
it seems outside of the usage that the command line tools and
conntrack daemon need, so I suspect its not possible.

Can somebody confirm my suspicions?

Thank you.
Sam

Re: can libnetfilter_conntrack be used to write a userspace connection tracker?

[Pablo Neira Ayuso]

On 12/01/11 22:25, Sam Roberts wrote:
> I'm working on a connection tracker for a RPC-like protocol (over TCP).
> 
> I believe that by inspecting packets using nfqueue, and
> creating/destroying expectations using nfconntrack, I can do a
> connection tracker in user-space.
> 
> In order to remove nfqueue from the mix, I've been looking at the
> conntrack code, trying to figure out whether even notifications about
> connection status can include the TCP data that I need to inspect, the
> data that's in the skbs provided to kernel module conntrack helpers. I
> haven't been able to be certain what libnfconntrack can/cannot do, but
> it seems outside of the usage that the command line tools and
> conntrack daemon need, so I suspect its not possible.
> 
> Can somebody confirm my suspicions?

You can implement a user-space conntrack helper with NFQUEUE and
libnetfilter_conntrack:

http://people.netfilter.org/pablo/userspace-conntrack-helpers/

That's a proof-of-concept, ideally there would be a generic daemon so
you can develop your own plugins for state tracking upon it.

That daemon's on my TODO list.

You require Linux kernel >= 2.6.37

Re: can libnetfilter_conntrack be used to write a userspace connection tracker?

[Sam Roberts]

On Wed, Jan 12, 2011 at 1:36 PM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> You can implement a user-space conntrack helper with NFQUEUE and
> libnetfilter_conntrack:
>
> http://people.netfilter.org/pablo/userspace-conntrack-helpers/
>
> You require Linux kernel >= 2.6.37

I've upgraded the kernel to
http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.37-rc2-maverick/.

I began experiencing this bug
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=612272
For me, conntrack -E expect is exiting immediately, on an almost unloaded box.

So I built the two conntrack projects from git master to see if that
would help - and got a kernel bug (see other message).

Are there any other requirements? Would it be better if I build my own
kernel? Are there any other dependency issues between the various
players here: libnfnetlink, libnfconntrack, conntrack, and the kernel?

Or is there a limit to the users of conntrack? Can my own daemon
(modelled on the ftp userspace example of yours), conntrack -E, and
other users of conntrack coexist?

Thanks,
Sam

Re: can libnetfilter_conntrack be used to write a userspace connection tracker?

[Pablo Neira Ayuso]

[-- Attachment #1: Type: text/plain, Size: 1282 bytes --]

On 16/02/11 01:08, Sam Roberts wrote:
> On Wed, Jan 12, 2011 at 1:36 PM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
>> You can implement a user-space conntrack helper with NFQUEUE and
>> libnetfilter_conntrack:
>>
>> http://people.netfilter.org/pablo/userspace-conntrack-helpers/
>>
>> You require Linux kernel >= 2.6.37
> 
> I've upgraded the kernel to
> http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.37-rc2-maverick/.
> 
> I began experiencing this bug
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=612272
> For me, conntrack -E expect is exiting immediately, on an almost unloaded box.
> 
> So I built the two conntrack projects from git master to see if that
> would help - and got a kernel bug (see other message).
> 
> Are there any other requirements? Would it be better if I build my own
> kernel? Are there any other dependency issues between the various
> players here: libnfnetlink, libnfconntrack, conntrack, and the kernel?
> 
> Or is there a limit to the users of conntrack? Can my own daemon
> (modelled on the ftp userspace example of yours), conntrack -E, and
> other users of conntrack coexist?

Probably you have hit one of the bugs that went into 2.6.37. Please, try
the patch attached. IIRC, this is fixed in -stable and 2.6.38 and later
kernels.

[-- Attachment #2: 0001-netfilter-fix-export-secctx-error-handling.patch --]
[-- Type: text/x-patch, Size: 4120 bytes --]

>From cba85b532e4aabdb97f44c18987d45141fd93faa Mon Sep 17 00:00:00 2001
From: Pablo Neira Ayuso <pablo@netfilter.org>
Date: Thu, 6 Jan 2011 11:25:00 -0800
Subject: [PATCH] netfilter: fix export secctx error handling

In 1ae4de0cdf855305765592647025bde55e85e451, the secctx was exported
via the /proc/net/netfilter/nf_conntrack and ctnetlink interfaces
instead of the secmark.

That patch introduced the use of security_secid_to_secctx() which may
return a non-zero value on error.

In one of my setups, I have NF_CONNTRACK_SECMARK enabled but no
security modules. Thus, security_secid_to_secctx() returns a negative
value that results in the breakage of the /proc and `conntrack -L'
outputs. To fix this, we skip the inclusion of secctx if the
aforementioned function fails.

This patch also fixes the dynamic netlink message size calculation
if security_secid_to_secctx() returns an error, since its logic is
also wrong.

This problem exists in Linux kernel >= 2.6.37.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
---
 .../netfilter/nf_conntrack_l3proto_ipv4_compat.c   |    2 +-
 net/netfilter/nf_conntrack_netlink.c               |   25 +++++++++++--------
 net/netfilter/nf_conntrack_standalone.c            |    2 +-
 3 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/net/ipv4/netfilter/nf_conntrack_l3proto_ipv4_compat.c b/net/ipv4/netfilter/nf_conntrack_l3proto_ipv4_compat.c
index 37f8adb..63f60fc 100644
--- a/net/ipv4/netfilter/nf_conntrack_l3proto_ipv4_compat.c
+++ b/net/ipv4/netfilter/nf_conntrack_l3proto_ipv4_compat.c
@@ -97,7 +97,7 @@ static int ct_show_secctx(struct seq_file *s, const struct nf_conn *ct)
 
 	ret = security_secid_to_secctx(ct->secmark, &secctx, &len);
 	if (ret)
-		return ret;
+		return 0;
 
 	ret = seq_printf(s, "secctx=%s ", secctx);
 
diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c
index b729ace..0cdba50 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -254,7 +254,7 @@ ctnetlink_dump_secctx(struct sk_buff *skb, const struct nf_conn *ct)
 
 	ret = security_secid_to_secctx(ct->secmark, &secctx, &len);
 	if (ret)
-		return ret;
+		return 0;
 
 	ret = -1;
 	nest_secctx = nla_nest_start(skb, CTA_SECCTX | NLA_F_NESTED);
@@ -453,16 +453,22 @@ ctnetlink_counters_size(const struct nf_conn *ct)
 	       ;
 }
 
-#ifdef CONFIG_NF_CONNTRACK_SECMARK
-static int ctnetlink_nlmsg_secctx_size(const struct nf_conn *ct)
+static inline int
+ctnetlink_secctx_size(const struct nf_conn *ct)
 {
-	int len;
+#ifdef CONFIG_NF_CONNTRACK_SECMARK
+	int len, ret;
 
-	security_secid_to_secctx(ct->secmark, NULL, &len);
+	ret = security_secid_to_secctx(ct->secmark, NULL, &len);
+	if (ret)
+		return 0;
 
-	return sizeof(char) * len;
-}
+	return nla_total_size(0) /* CTA_SECCTX */
+	       + nla_total_size(sizeof(char) * len); /* CTA_SECCTX_NAME */
+#else
+	return 0;
 #endif
+}
 
 static inline size_t
 ctnetlink_nlmsg_size(const struct nf_conn *ct)
@@ -479,10 +485,7 @@ ctnetlink_nlmsg_size(const struct nf_conn *ct)
 	       + nla_total_size(0) /* CTA_PROTOINFO */
 	       + nla_total_size(0) /* CTA_HELP */
 	       + nla_total_size(NF_CT_HELPER_NAME_LEN) /* CTA_HELP_NAME */
-#ifdef CONFIG_NF_CONNTRACK_SECMARK
-	       + nla_total_size(0) /* CTA_SECCTX */
-	       + nla_total_size(ctnetlink_nlmsg_secctx_size(ct)) /* CTA_SECCTX_NAME */
-#endif
+	       + ctnetlink_secctx_size(ct)
 #ifdef CONFIG_NF_NAT_NEEDED
 	       + 2 * nla_total_size(0) /* CTA_NAT_SEQ_ADJ_ORIG|REPL */
 	       + 6 * nla_total_size(sizeof(u_int32_t)) /* CTA_NAT_SEQ_OFFSET */
diff --git a/net/netfilter/nf_conntrack_standalone.c b/net/netfilter/nf_conntrack_standalone.c
index 0fb6570..b4d7f0f 100644
--- a/net/netfilter/nf_conntrack_standalone.c
+++ b/net/netfilter/nf_conntrack_standalone.c
@@ -118,7 +118,7 @@ static int ct_show_secctx(struct seq_file *s, const struct nf_conn *ct)
 
 	ret = security_secid_to_secctx(ct->secmark, &secctx, &len);
 	if (ret)
-		return ret;
+		return 0;
 
 	ret = seq_printf(s, "secctx=%s ", secctx);
 
-- 
1.7.2.3

Re: can libnetfilter_conntrack be used to write a userspace connection tracker?

[Sam Roberts]

On Wed, Feb 16, 2011 at 5:20 AM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> Probably you have hit one of the bugs that went into 2.6.37. Please, try
> the patch attached. IIRC, this is fixed in -stable and 2.6.38 and later
> kernels.

Since I'm not yet building my kernels from source, its easier for me
to build a newer kernel than to find an old one and patch it.

Stable is 2.6.37, I'll try 2.6.38-rc5.

userspace connection trackers seems a bit bleading edge, I'd be happy
to build your latest code from git if you point me to it.

I'm now two steps back since upgrading from ubuntu's default kernel
2.6..35 and tools 0.9.14.

It used to be everything but setting expectations was working for me,
but I no longer get updates at all about the conntrack table, and
neither does conntrack -E or -L:

% sudo conntrack -L conntrack
conntrack v0.9.15 (conntrack-tools): 0 flow entries have been shown.

% sudo cat /proc/net/nf_conntrack
ipv4     2 unknown  2 530 src=0.0.0.0 dst=224.0.0.1 [UNREPLIED]
src=224.0.0.1 dst=0.0.0.0 mark=0 ipv4     2 tcp      6 45 CLOSE_WAIT
src=127.0.0.1 dst=127.0.0.1 sport=35780 dport=9999 src=127.0.0.1
dst=127.0.0.1 sport=9999 dport=35780 [ASSURED] mark=0 ipv4     2 tcp
   6 108 SYN_SENT src=127.0.0.1 dst=127.0.0.1 sport=58000 dport=36011
[UNREPLIED] src=127.0.0.1 dst=127.0.0.1 sport=36011 dport=58000 mark=0

% conntrack --version
conntrack v0.9.15 (conntrack-tools)

% grep -i version /usr/local/lib/pkgconfig/libn* /usr/lib/pkgconfig/libn
/usr/local/lib/pkgconfig/libnetfilter_conntrack.pc:Version: 0.9.0
/usr/local/lib/pkgconfig/libnetfilter_queue.pc:Version: 1.0.0
/usr/local/lib/pkgconfig/libnfnetlink.pc:Version: 1.0.0
grep: /usr/lib/pkgconfig/libn: No such file or directory

% uname -a
Linux samtu 2.6.37-020637rc2-generic #201011160905 SMP Tue Nov 16
10:15:47 UTC 2010 i686 GNU/Linux

Cheers,
Sam

Re: can libnetfilter_conntrack be used to write a userspace connection tracker?

[Pablo Neira Ayuso]

On 16/02/11 18:52, Sam Roberts wrote:
> On Wed, Feb 16, 2011 at 5:20 AM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
>> Probably you have hit one of the bugs that went into 2.6.37. Please, try
>> the patch attached. IIRC, this is fixed in -stable and 2.6.38 and later
>> kernels.
> 
> Since I'm not yet building my kernels from source, its easier for me
> to build a newer kernel than to find an old one and patch it.
> 
> Stable is 2.6.37, I'll try 2.6.38-rc5.

I'm using 2.6.37 with the patch that I sent you in one of my firewalls:

$ uname -a
Linux debian2 2.6.37 #7 SMP Mon Feb 7 10:34:10 UTC 2011 x86_64 GNU/Linux

Everything works fine.

> userspace connection trackers seems a bit bleading edge, I'd be happy
> to build your latest code from git if you point me to it.

http://git.netfilter.org/cgi-bin/gitweb.cgi?p=conntrack-tools.git;a=summary

It's stable, I'll release 1.0 soon.

Some more work can be done on it to port it to libmnl, add H323 and SIP
support, active-active support, among tons of many other improvements.
But that can be done in the future.

> I'm now two steps back since upgrading from ubuntu's default kernel
> 2.6..35 and tools 0.9.14.
> 
> It used to be everything but setting expectations was working for me,
> but I no longer get updates at all about the conntrack table, and
> neither does conntrack -E or -L:
> 
> % sudo conntrack -L conntrack
> conntrack v0.9.15 (conntrack-tools): 0 flow entries have been shown.

As said, here works fine with the patch that I attached ;-)