libnfnetlink_conntrack performance

libnfnetlink_conntrack performance

[Morten Isaksen]

Hi!

Do anyone have som experience with the performance of libnfnetlink_conntrack?

I am using it to track and log NFCT_T_NEW and NFCT_T_DESTROY events. I
am testing it on a firewall with about 15 Mbit/s traffic and 18-22K
entries in ip_conntrack. The only difference I can spot so far is 300
more context switches pr. sec than usual.

The events are collected in batches of 7 and then sent in an UDP
packet to a log server.

-- 
Morten Isaksen
http://www.misak.dk/blog/

Re: libnfnetlink_conntrack performance

[Pablo Neira Ayuso]

Morten Isaksen wrote:
> Do anyone have som experience with the performance of libnfnetlink_conntrack?
> 
> I am using it to track and log NFCT_T_NEW and NFCT_T_DESTROY events. I
> am testing it on a firewall with about 15 Mbit/s traffic and 18-22K
> entries in ip_conntrack. The only difference I can spot so far is 300
> more context switches pr. sec than usual.

Using conntrack_events example available under utils/ with a little hack 
(removed the counter).

$ ./bench 192.168.1.2 10000 # generate 10000 HTTP GET requests
time taken: 19.613752000 seconds
request per seconds: 509.846344

Cyclesoak says:
System load: 10.1%
System load:  1.6%
System load:  3.4%
System load:  6.8%
System load: 13.7%
System load: 23.2%
System load:  4.2%
System load:  1.7%
System load:  3.5%
System load:  6.8%
System load: 13.6%
System load: 20.8%
System load:  6.8%
System load:  1.7%
System load:  3.4%
System load:  7.0%
System load: 14.1%
System load: 22.5%
System load:  5.4%
System load:  1.7%
System load:  2.1%

processor       : 1
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 33
model name      : Dual Core AMD Opteron(tm) Processor 275
stepping        : 2
cpu MHz         : 2210.189
cache size      : 1024 KB

One optimization that we can apply here is that libnetfilter_conntrack 
listen to update events even if you don't want to do it. These 
introduces an extra cost, of course. I have a patch somewhere to improve 
this, even I plan to introduce more fine grain events groups like 
protocol/state specific, eg. only listen to TCP syn_sent events.

> The events are collected in batches of 7 and then sent in an UDP
> packet to a log server.

Did you ever have a look at conntrackd? It is available in the 
conntrack-tools package that I'll release today. It has a statistics 
mode that still needs some work, this option that you have implemented 
could be quite interesting for it.

-- 
The dawn of the fourth age of Linux firewalling is coming; a time of 
great struggle and heroic deeds -- J.Kadlecsik got inspired by J.Morris

Re: libnfnetlink_conntrack performance

[Morten Isaksen]

Hi!

On 5/23/07, Pablo Neira Ayuso <pablo@netfilter.org> wrote:

> One optimization that we can apply here is that libnetfilter_conntrack
> listen to update events even if you don't want to do it. These
> introduces an extra cost, of course. I have a patch somewhere to improve
> this, even I plan to introduce more fine grain events groups like
> protocol/state specific, eg. only listen to TCP syn_sent events.

I am very interested in this patch.

> > The events are collected in batches of 7 and then sent in an UDP
> > packet to a log server.
>
> Did you ever have a look at conntrackd? It is available in the
> conntrack-tools package that I'll release today. It has a statistics
> mode that still needs some work, this option that you have implemented
> could be quite interesting for it.

I looked at it briefly, but as far as I could tell it did not support
the log-part I am interested in, so I decided to write my own log
program (very much based on the conntrack_event program). Also from a
performance point of view I like the program as small as possible.

-- 
Morten Isaksen
http://www.misak.dk/blog/