[RFC,ANNOUNCE] conntrack daemon (stateful replication)

[RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Pablo Neira Ayuso]

Hi,

I've been working on a pet project during the last months. Part of this 
stuff is related with my works in the university.

The project is called `conntrackd' that is the conntrack userspace daemon.

Features:
---------

- Stateful replication: the daemon keeps a cache of internal events via 
libnetfilter_conntrack and a cache of external event received from the 
other node.
- Support for classical Primary/Backup settings
- Support for Active/Active settings (two machines max. per VRRP instance)
- Support for NAT: It recognizes NAT'ed connections and handles them 
properly.
- UDP traffic ignore facility
- ICMP traffic ignore facility
- Ignore loopback traffic (not customizable at the moment)
- Ignore traffic for certain set of machines: Useful to ignore traffic 
for the firewall since we just want to replicate conntracks that 
represent forwarded connections.
- Dump internal and external caches via UNIX sockets
- Flush internal, external caches and conntrack table
- The communication between daemons is done in NETLINK format, so the 
protocol used is based on NETLINK over IP, to ensure backward compatibility.
- Configuration via file
- Log file support

I'm still generalizing this a bit so it can be also used for statistics 
purposes: since a replica of the conntrack table (cache) is kept in 
userspace, the dumping process would not need to request the information 
to the kernel.

The software is released under GPLv2 and it is available at:

http://people.netfilter.org/pablo/conntrackd/

The remaining issues are:
-------------------------

- Support for IPv6.

- Evaluation: I'll be getting some results to evaluate the *performance 
drop* that could suppose to enable replication in linux firewall based 
on this solution. Expect results soon.
- Better integration with keepalived: This is the most important issue 
and my major concern now. I'm happy with keepalived, but the interface 
provided to communicate events (based on shell scripts) is not so much.
- Checksum messages going through the network.
- Security: A dedicated link is required to communicate nodes that 
conform the cluster, otherwise third parties could pick up information 
about the connections processed by the firewall.
- More testing

Requirements:
-------------

- linux kernel >= 2.6.16 with [ip|nf]_conntrack_netlink support
- libnfnetlink from SVN
- libnetfilter_conntrack from SVN + plus patch inside the doc/ directory.
- keepalived (tested here with 1.1.11 available in debian)

Installation:
-------------

- make sure that multicast traffic sent by conntrackd is received in the 
dedicated interfaces:
iptables -I INPUT -d 225.0.0.50 -j ACCEPT
iptables -I OUTPUT -d 225.0.0.50 -j ACCEPT

- install libnfnetlink and libnetfilter_conntrack from SVN (and apply 
the patch available in doc/)
- classical ./configure; make; make install
- copy doc/conntrackd.conf to /etc/conntrackd/, this directory can be 
overrided with the -C option.
- copy doc/script_*.sh where keepalived can find them.


Running:
--------

# conntrackd -d
# conntrackd -i # dump internal events cache
# conntrackd -e # dump external cache
# conntrackd -k # kill conntrackd
# conntrackd -f # flush internal, external caches and conntrack table

I am going to write some docs at the same time that I improve the daemon.

BTW, I sent a PDF file to netfilter-core but exceeded maximum size a 
bit, could you accept it? I wrote a small article for USENIX's :LOGIN; 
about the connection tracking system that will be released in June. I 
can't release it for public before that date but I sent you a copy in 
private. Although all that it contains is well known by you ;) but at 
least have a look at the acknowledgement section.

Hope that you like it.

-- 
The dawn of the fourth age of Linux firewalling is coming; a time of 
great struggle and heroic deeds -- J.Kadlecsik got inspired by J.Morris

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Patrick McHardy]

Pablo Neira Ayuso wrote:
> I've been working on a pet project during the last months. Part of this
> stuff is related with my works in the university.
> 
> The project is called `conntrackd' that is the conntrack userspace daemon.

Interesting work (and nice featureset), I like the idea of doing this
in userspace. IIRC Holger Eitzenberger is also working on something
similar. Do you have performance measurements how this compares to
ct_sync?

> BTW, I sent a PDF file to netfilter-core but exceeded maximum size a
> bit, could you accept it? I wrote a small article for USENIX's :LOGIN;
> about the connection tracking system that will be released in June. I
> can't release it for public before that date but I sent you a copy in
> private. Although all that it contains is well known by you ;) but at
> least have a look at the acknowledgement section.

I'd be interested, please send it to me in private. Thanks.

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Pablo Neira Ayuso]

Patrick McHardy wrote:
> Pablo Neira Ayuso wrote:
> 
>>I've been working on a pet project during the last months. Part of this
>>stuff is related with my works in the university.
>>
>>The project is called `conntrackd' that is the conntrack userspace daemon.
> 
> Interesting work (and nice featureset), I like the idea of doing this
> in userspace. IIRC Holger Eitzenberger is also working on something
> similar. Do you have performance measurements how this compares to
> ct_sync?

I'll spend tomorrow morning to do that, as soon as I get some results 
I'll post them. Hm, I didn't know that Holger was working on something 
similar, it could be nice if we can join efforts.

>>BTW, I sent a PDF file to netfilter-core but exceeded maximum size a
>>bit, could you accept it? I wrote a small article for USENIX's :LOGIN;
>>about the connection tracking system that will be released in June. I
>>can't release it for public before that date but I sent you a copy in
>>private. Although all that it contains is well known by you ;) but at
>>least have a look at the acknowledgement section.
> 
> I'd be interested, please send it to me in private. Thanks.

Done.

-- 
The dawn of the fourth age of Linux firewalling is coming; a time of 
great struggle and heroic deeds -- J.Kadlecsik got inspired by J.Morris

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Holger Eitzenberger]

On Mon, May 29, 2006 at 01:32:15AM +0200, Patrick McHardy wrote:

> > I've been working on a pet project during the last months. Part of this
> > stuff is related with my works in the university.
> > 
> > The project is called `conntrackd' that is the conntrack userspace daemon.

> Interesting work (and nice featureset), I like the idea of doing this
> in userspace. IIRC Holger Eitzenberger is also working on something
> similar. Do you have performance measurements how this compares to
> ct_sync?

Hi Pablo, Patrick,

yes, for me it started as a proof-of-concept and matured in an almost
feature complete version I will release soon.  It's called ctsyncd.

Pablo, I will check your code, at a first glance your solution has a
richer feature set than my solution.  Mabe we should join our efforts.

Regards.  /holger

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Jeho-Park]

good job !

it looks so interesting ~ thanks


jeho park


Pablo Neira Ayuso wrote:

> Hi,
>
> I've been working on a pet project during the last months. Part of 
> this stuff is related with my works in the university.
>
> The project is called `conntrackd' that is the conntrack userspace 
> daemon.
>
> Features:
> ---------
>
> - Stateful replication: the daemon keeps a cache of internal events 
> via libnetfilter_conntrack and a cache of external event received from 
> the other node.
> - Support for classical Primary/Backup settings
> - Support for Active/Active settings (two machines max. per VRRP 
> instance)
> - Support for NAT: It recognizes NAT'ed connections and handles them 
> properly.
> - UDP traffic ignore facility
> - ICMP traffic ignore facility
> - Ignore loopback traffic (not customizable at the moment)
> - Ignore traffic for certain set of machines: Useful to ignore traffic 
> for the firewall since we just want to replicate conntracks that 
> represent forwarded connections.
> - Dump internal and external caches via UNIX sockets
> - Flush internal, external caches and conntrack table
> - The communication between daemons is done in NETLINK format, so the 
> protocol used is based on NETLINK over IP, to ensure backward 
> compatibility.
> - Configuration via file
> - Log file support
>
> I'm still generalizing this a bit so it can be also used for 
> statistics purposes: since a replica of the conntrack table (cache) is 
> kept in userspace, the dumping process would not need to request the 
> information to the kernel.
>
> The software is released under GPLv2 and it is available at:
>
> http://people.netfilter.org/pablo/conntrackd/
>
> The remaining issues are:
> -------------------------
>
> - Support for IPv6.
>
> - Evaluation: I'll be getting some results to evaluate the 
> *performance drop* that could suppose to enable replication in linux 
> firewall based on this solution. Expect results soon.
> - Better integration with keepalived: This is the most important issue 
> and my major concern now. I'm happy with keepalived, but the interface 
> provided to communicate events (based on shell scripts) is not so much.
> - Checksum messages going through the network.
> - Security: A dedicated link is required to communicate nodes that 
> conform the cluster, otherwise third parties could pick up information 
> about the connections processed by the firewall.
> - More testing
>
> Requirements:
> -------------
>
> - linux kernel >= 2.6.16 with [ip|nf]_conntrack_netlink support
> - libnfnetlink from SVN
> - libnetfilter_conntrack from SVN + plus patch inside the doc/ directory.
> - keepalived (tested here with 1.1.11 available in debian)
>
> Installation:
> -------------
>
> - make sure that multicast traffic sent by conntrackd is received in 
> the dedicated interfaces:
> iptables -I INPUT -d 225.0.0.50 -j ACCEPT
> iptables -I OUTPUT -d 225.0.0.50 -j ACCEPT
>
> - install libnfnetlink and libnetfilter_conntrack from SVN (and apply 
> the patch available in doc/)
> - classical ./configure; make; make install
> - copy doc/conntrackd.conf to /etc/conntrackd/, this directory can be 
> overrided with the -C option.
> - copy doc/script_*.sh where keepalived can find them.
>
>
> Running:
> --------
>
> # conntrackd -d
> # conntrackd -i # dump internal events cache
> # conntrackd -e # dump external cache
> # conntrackd -k # kill conntrackd
> # conntrackd -f # flush internal, external caches and conntrack table
>
> I am going to write some docs at the same time that I improve the daemon.
>
> BTW, I sent a PDF file to netfilter-core but exceeded maximum size a 
> bit, could you accept it? I wrote a small article for USENIX's :LOGIN; 
> about the connection tracking system that will be released in June. I 
> can't release it for public before that date but I sent you a copy in 
> private. Although all that it contains is well known by you ;) but at 
> least have a look at the acknowledgement section.
>
> Hope that you like it.
>


-- 
--
Jeho-Park <jhpark-nf@kernelproject.org> or <linuxpark@infnis.com>

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1098 bytes --]



On Mon, 29 May 2006, Pablo Neira Ayuso wrote:

> Hi,
Hi,

> I've been working on a pet project during the last months. Part of this stuff 
> is related with my works in the university.
>
> The project is called `conntrackd' that is the conntrack userspace daemon.

Great news! Thank you. :)

> The remaining issues are:
> - Security: A dedicated link is required to communicate nodes that conform 
> the cluster, otherwise third parties could pick up information about the 
> connections processed by the firewall.

AFAIK this can be solved by additional vlan.

> Requirements:
> -------------
>
> - linux kernel >= 2.6.16 with [ip|nf]_conntrack_netlink support
> - libnfnetlink from SVN
> - libnetfilter_conntrack from SVN + plus patch inside the doc/ directory.

Current versions of libnfnetlink/libnetfilter_conntrack/conntrack/etc are 
quite old. Are there any plans for releasing new versions based on SVN?

What about TCP window tracking? AFAIR, last time when I checked 
conntrack_netlink did not handle it at all.

Best regards,

 				Krzysztof Olędzki

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Eric Leblond]

Hi,

> The project is called `conntrackd' that is the conntrack userspace daemon.
> - Stateful replication: the daemon keeps a cache of internal events via
> libnetfilter_conntrack and a cache of external event received from the
> other node.

That sounds good !

> - Support for classical Primary/Backup settings
...
>
> - install libnfnetlink and libnetfilter_conntrack from SVN (and apply
> the patch available in doc/)

A quick look at the patch seems to show that API has changed again :-(

Userspace handling of conntrack is a GREAT thing and conntrackd is one of
the elements that put it on the top of the great Netfilter things. But, it
definitively needs a fixed API because all related projects are at the
same point : "use svn and patch it!"

All nfnetlink related libraries should really been put in delivery state.
Userspace application (conntrackd, NuFW, ...) are waiting for it but they
need stability. For now, we can't even say to our users: "Use 0.999
release and it will work.".
This is the same for articles or documentations that could help spreading
the good word. They have nothing stable to relay on.

I think userspace libraries related to the new nfnetlink stuffs should be
released more frequently. Another way could be to have a stable and a
developement release but it has a cost.

Netfilter has an enormous advantage over other filtering system thanks to
this new features. Don't waste it for a simple release policy problem.

BR and thanks for the great work,
--
Éric Leblond

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Pablo Neira Ayuso]

Eric Leblond wrote:
>>The project is called `conntrackd' that is the conntrack userspace daemon.
>>- Stateful replication: the daemon keeps a cache of internal events via
>>libnetfilter_conntrack and a cache of external event received from the
>>other node.
> 
> 
> That sounds good !
> 
> 
>>- Support for classical Primary/Backup settings
> 
> ...
> 
>>- install libnfnetlink and libnetfilter_conntrack from SVN (and apply
>>the patch available in doc/)
> 
> 
> A quick look at the patch seems to show that API has changed again :-(
> 
> Userspace handling of conntrack is a GREAT thing and conntrackd is one of
> the elements that put it on the top of the great Netfilter things. But, it
> definitively needs a fixed API because all related projects are at the
> same point : "use svn and patch it!"

I missed this email. Eric, this is still a work in progress, and I 
didn't ever apply that patch. The new patch included in conntrackd (yet 
unapplied) doesn't change the API. I understand your concerns about the API.

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]

On Mon, May 29, 2006 at 12:09:59AM +0200, Pablo Neira Ayuso wrote:
> Hi,
> 
> I've been working on a pet project during the last months. Part of
> this stuff is related with my works in the university.
> 
> The project is called `conntrackd' that is the conntrack userspace daemon.

great.  I'll have a look at the code ASAP.

> BTW, I sent a PDF file to netfilter-core but exceeded maximum size a
> bit, could you accept it? I wrote a small article for USENIX's :LOGIN;
> about the connection tracking system that will be released in June. I
> can't release it for public before that date but I sent you a copy in
> private. Although all that it contains is well known by you ;) but at
> least have a look at the acknowledgement section.

Please re-send.  Somehow I didn't find it in the list of to-be-moderated
postings for netfilter-core (might have been accidentially considered as
spam).

Looking forward to reading it!

Keep up the good work,

-- 
- Harald Welte <laforge@netfilter.org>                 http://netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Holger Eitzenberger]

On Mon, May 29, 2006 at 12:09:59AM +0200, Pablo Neira Ayuso wrote:

> I've been working on a pet project during the last months. Part of this 
> stuff is related with my works in the university.

Hi Pablo,

I am working on a daemon called 'ctsyncd', which for me started as a
proof-of-concept and is now almost in a state where I can release it
to the public.  My current objective is simple master/slave scenario
without active/active, and I am almost done.

Hopefully I am able to look at your sources.  With your great knowledge
of libnetfilter_conntrack and my programming skills we should consider
joining our efforts.  But first I will release my code for public review
within a few days.  


Stay tuned.

/holger


> - Stateful replication: the daemon keeps a cache of internal events via 
> libnetfilter_conntrack and a cache of external event received from the 
> other node.
> - Support for classical Primary/Backup settings
> - Support for Active/Active settings (two machines max. per VRRP instance)
> - Support for NAT: It recognizes NAT'ed connections and handles them 
> properly.
> - UDP traffic ignore facility
> - ICMP traffic ignore facility
> - Ignore loopback traffic (not customizable at the moment)
> - Ignore traffic for certain set of machines: Useful to ignore traffic 
> for the firewall since we just want to replicate conntracks that 
> represent forwarded connections.
> - Dump internal and external caches via UNIX sockets
> - Flush internal, external caches and conntrack table
> - The communication between daemons is done in NETLINK format, so the 
> protocol used is based on NETLINK over IP, to ensure backward compatibility.
> - Configuration via file

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Pasi Kärkkäinen]

Hello!

Any updates to these projects? Files available somewhere? I think many
people from this list would like to test and help with these daemons..

- Pasi 

On Tue, May 30, 2006 at 08:58:05AM +0200, Holger Eitzenberger wrote:
> On Mon, May 29, 2006 at 12:09:59AM +0200, Pablo Neira Ayuso wrote:
> 
> > I've been working on a pet project during the last months. Part of this 
> > stuff is related with my works in the university.
> 
> Hi Pablo,
> 
> I am working on a daemon called 'ctsyncd', which for me started as a
> proof-of-concept and is now almost in a state where I can release it
> to the public.  My current objective is simple master/slave scenario
> without active/active, and I am almost done.
> 
> Hopefully I am able to look at your sources.  With your great knowledge
> of libnetfilter_conntrack and my programming skills we should consider
> joining our efforts.  But first I will release my code for public review
> within a few days.  
> 
> 
> Stay tuned.
> 
> /holger
> 
> 
> > - Stateful replication: the daemon keeps a cache of internal events via 
> > libnetfilter_conntrack and a cache of external event received from the 
> > other node.
> > - Support for classical Primary/Backup settings
> > - Support for Active/Active settings (two machines max. per VRRP instance)
> > - Support for NAT: It recognizes NAT'ed connections and handles them 
> > properly.
> > - UDP traffic ignore facility
> > - ICMP traffic ignore facility
> > - Ignore loopback traffic (not customizable at the moment)
> > - Ignore traffic for certain set of machines: Useful to ignore traffic 
> > for the firewall since we just want to replicate conntracks that 
> > represent forwarded connections.
> > - Dump internal and external caches via UNIX sockets
> > - Flush internal, external caches and conntrack table
> > - The communication between daemons is done in NETLINK format, so the 
> > protocol used is based on NETLINK over IP, to ensure backward compatibility.
> > - Configuration via file
> 
> 
> 

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Pablo Neira Ayuso]

Pasi Kärkkäinen wrote:
> Hello!
> 
> Any updates to these projects? Files available somewhere? I think many
> people from this list would like to test and help with these daemons..

The only thing available at the moment:

http://people.netfilter.org/pablo/conntrackd/

I know, this requires an appropiate documentation and a webpage but I'm 
working on it, any help on the webpage (something simple) could come handy.

-- 
The dawn of the fourth age of Linux firewalling is coming; a time of 
great struggle and heroic deeds -- J.Kadlecsik got inspired by J.Morris

Re: [RFC,ANNOUNCE] conntrack daemon (stateful replication)

[Angel Mieres]

Hi Pasi, 

Im honored to told you that advance in this project is positive and im
sure netfilter(with Pablo leading) are doing all they can. 
I have the chance to test conntrackd and i will be pleased to help you
to test it ;)

Best Regards,
Angel M.
El jue, 17-08-2006 a las 16:28 +0200, Pablo Neira Ayuso escribió:
> Pasi Kärkkäinen wrote:
> > Hello!
> > 
> > Any updates to these projects? Files available somewhere? I think many
> > people from this list would like to test and help with these daemons..
> 
> The only thing available at the moment:
> 
> http://people.netfilter.org/pablo/conntrackd/
> 
> I know, this requires an appropiate documentation and a webpage but I'm 
> working on it, any help on the webpage (something simple) could come handy.
> 
-- 
Angel Mieres - amieres@eneotecnologia.com
///////////////////////////////////////// Gentoo has you...