conntrack accounting

conntrack accounting

[Ben Lentz]

Greetings list!
I am considering using the conntrack-tools userspace package to perform 
byte level accounting for iptables by reading events from the connection 
tracking table for completed connections and logging the statistics for 
the stateful connection to syslog. It appears that conntrack was really 
designed to keep redundant firewalls' state tables in sync, but I'm 
intrigued by it's ability to use the new connection tracking and state 
notification features in netfilter without having to parse or poll 
/proc/net/ip_conntrack.

The goal I'm trying to accomplish is similar to that of:
conntrack -E conntrack -e DESTROY | logger -t conntrack &

which gives me the ability to log completed (e.g. entered the DESTROY 
state) connections to syslog from kernel-triggered events. It's plenty 
hackish though... it'd be nicer to have an actual daemon that fork()s 
and detaches and closes file descriptors and communicates with syslog 
directly. I understand that a patch has been contributed to allow 
conntrackd to use syslog, but it appears that the logging facility in 
conntrackd is limited to recording startup, shutdown, and error 
information. In any event, the current incarnation of conntrackd does 
not support the long-term recording of event messages.

What would you folks recommend to accomplish this goal? Am I simply 
using the wrong tool here, or is it worthwhile to get a-patchin'?

If more appropriate, I'll repost this in netfilter failover, but since 
I'm not actually looking to do failover (at the moment) I'd figure I'd 
start here.

Thanks in advance for any information or opinions you can provide.

Re: conntrack accounting

[Pablo Neira Ayuso]

Hi,

Ben Lentz wrote:
> I am considering using the conntrack-tools userspace package to perform
> byte level accounting for iptables by reading events from the connection
> tracking table for completed connections and logging the statistics for
> the stateful connection to syslog. It appears that conntrack was really
> designed to keep redundant firewalls' state tables in sync, but I'm
> intrigued by it's ability to use the new connection tracking and state
> notification features in netfilter without having to parse or poll
> /proc/net/ip_conntrack.
> 
> The goal I'm trying to accomplish is similar to that of:
> conntrack -E conntrack -e DESTROY | logger -t conntrack &

I just committed a patch to SVN which implements this for the statistics
mode. Have a look at the doc/stats/conntrackd.conf example file and
enable logging to give it a try. This will be available in the upcoming
conntrack-tool 0.9.6 release. Don't forget to run conntrackd with the -S
option.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: conntrack accounting

[Ben Lentz]

> I just committed a patch to SVN which implements this for the statistics
> mode. Have a look at the doc/stats/conntrackd.conf example file and
> enable logging to give it a try. This will be available in the upcoming
> conntrack-tool 0.9.6 release. Don't forget to run conntrackd with the -S
> option.
>   
This sounds great! However, I appear to be having some trouble. I 
checked out, built, and installed conntrack-tools 0.9.6 7164 and 
libnetfilter_conntrack-0.0.87 7164 and am running conntrackd -S. I still 
have libnfnetlink-0.0.30. I don't seem to be getting any statistics 
logging either in Syslog mode or LogFile mode.

- If I set the Stats section to Syslog on, it seems to crash on the 
first attempt to log:
select(5, [3 4], NULL, NULL, {0, 199092}) = 1 (in [4], left {0, 111000})
rt_sigprocmask(SIG_BLOCK, [INT TERM CHLD], NULL, 8) = 0
recvmsg(4, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000004}, 
msg_iov(1)=[...], msg_controllen=0, msg_flags=0}, MSG_PEEK) = 164
recvfrom(4, ""..., 8192, 0, {sa_family=AF_NETLINK, pid=0, 
groups=00000004}, [12]) = 164
time(NULL)                              = 1199383171
open("/etc/localtime", O_RDONLY)        = 6
fstat64(6, {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
fstat64(6, {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 
0) = 0xb7f97000
read(6, ""..., 4096)                    = 3519
close(6)                                = 0
munmap(0xb7f97000, 4096)                = 0
stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++

- If I set the Stats section to Logfile on (or LogFile filename), it 
doesn't crash, but generates a similar error each time it goes to log a 
connection. I can correlate connections about to close with
$ sudo watch --interval=0.1 'cat /proc/net/ip_conntrack | grep 
"^[a-z]\{3\} *[0-9]* *0"'

to errors in a strace on conntrackd:
rt_sigprocmask(SIG_BLOCK, [INT TERM CHLD], NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [INT TERM CHLD], NULL, 8) = 0
gettimeofday({1199383388, 670177}, NULL) = 0
gettimeofday({1199383388, 670286}, NULL) = 0
select(5, [3 4], NULL, NULL, {0, 198979}) = 1 (in [4], left {0, 47000})
rt_sigprocmask(SIG_BLOCK, [INT TERM CHLD], NULL, 8) = 0
recvmsg(4, {msg_name(12)={sa_family=AF_NETLINK, pid=0, groups=00000004}, 
msg_iov(1)=[...], msg_controllen=0, msg_flags=0}, MSG_PEEK) = 164
recvfrom(4, ""..., 8192, 0, {sa_family=AF_NETLINK, pid=0, 
groups=00000004}, [12]) = 164
recvfrom(4, 0xbfae01e0, 8192, 0, 0xbfae01ac, 0xbfae01b8) = -1 EAGAIN 
(Resource temporarily unavailable)
rt_sigprocmask(SIG_UNBLOCK, [INT TERM CHLD], NULL, 8) = 0
gettimeofday({1199383388, 822810}, NULL) = 0
gettimeofday({1199383388, 822856}, NULL) = 0

Configuration file is as follows:
$ grep -v '^$\|^#\|^\W#' /etc/conntrackd/conntrackd.conf
General {
        HashSize 8192
        HashLimit 65535
        LogFile on
        Syslog off
        LockFile /var/lock/conntrack.lock
        UNIX {
                Path /tmp/sync.sock
                Backlog 20
        }
        SocketBufferSize 262142
        SocketBufferSizeMaxGrown 655355
}
Stats {
        LogFile on
        Syslog off
}
IgnoreTrafficFor {
}
IgnoreProtocol {
}

Platform is CentOS 5, kernel 2.6.18.

Please let me know if I've done something dumb or if there's anything I 
can do to provide more useful debugging information. It's been a long 
while since I've been in gdb, so I might need some help with that...

Re: conntrack accounting

[Ben Lentz]

> This sounds great! However, I appear to be having some trouble. I 
> checked out, built, and installed conntrack-tools 0.9.6 7164 and 
> libnetfilter_conntrack-0.0.87 7164 and am running conntrackd -S. I 
> still have libnfnetlink-0.0.30. I don't seem to be getting any 
> statistics logging either in Syslog mode or LogFile mode.
I've made some progress... it turns out that the statistics logging via 
LogFile mode works only if I start conntrackd with -C 
/etc/conntrackd/conntrackd.conf... for some reason, leaving the -C 
option off and defaulting to this configuration file doesn't work, even 
though a strace shows a successful open to the file upon initialization.

Syslog statistics mode still crashes... but only if I *disable* LogFile, 
too.

This works perfectly:
Stats {
        LogFile on
        Syslog on
}
This crashes:
Stats {
        LogFile off
        Syslog on
}

CentOS 5.0, kernel 2.6.18, no SELinux, liberal iptables configuration, 
sysklogd 1.4.1

Is there a more appropriate place to report this information?

Thanks for your work on this thus far, this looks like a great addition 
to conntrack-tools!

Re: conntrack accounting

[Ben Lentz]

[-- Attachment #1: Type: text/plain, Size: 174 bytes --]


> Syslog statistics mode still crashes... but only if I *disable* 
> LogFile, too.
>
I've fixed the crash when stats LogFile is off and stats Syslog is on, 
patch attached.

[-- Attachment #2: conntrack-tools_stats-syslog.patch --]
[-- Type: text/x-patch, Size: 553 bytes --]

--- src/log.c.orig	2008-01-04 16:26:50.000000000 -0500
+++ src/log.c	2008-01-04 16:30:26.000000000 -0500
@@ -99,13 +99,14 @@
 	time_t t;
 	char buf[1024];
 	char *tmp;
+		
+	t = time(NULL);
+	ctime_r(&t, buf);
+	tmp = buf + strlen(buf);
+	buf[strlen(buf)-1]='\t';
+	nfct_snprintf(buf+strlen(buf), 1024-strlen(buf), ct, 0, 0, 0);
 
 	if (fd) {
-		t = time(NULL);
-		ctime_r(&t, buf);
-		tmp = buf + strlen(buf);
-		buf[strlen(buf)-1]='\t';
-		nfct_snprintf(buf+strlen(buf), 1024-strlen(buf), ct, 0, 0, 0);
 		fprintf(fd, "%s\n", buf);
 		fflush(fd);
 	}

Re: conntrack accounting

[Pablo Neira Ayuso]

Hi Ben,

Ben Lentz wrote:
>> Syslog statistics mode still crashes... but only if I *disable*
>> LogFile, too.
>>
> I've fixed the crash when stats LogFile is off and stats Syslog is on,
> patch attached.

Applied. Thanks a lot for investigating and fixing this.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers