What's required for a stateful firewall + ipvs in 2.6 kernel?

What's required for a stateful firewall + ipvs in 2.6 kernel?

[Brian Ghidinelli]

I'm trying to get a handle on whether or not it's possible to set up the 
following on a redundant pair of boxes:

1. Stateful iptables firewall
2. LVS director (keepalived)
3. DNAT, SNAT and fwmarks
4. Connection synchronization for failover

I currently have CentOS/RHEL 5 running 1, 2 and 3 above but the RHEL 
2.6.18-* kernels don't export LVS connections to netfilter resulting in 
lots of INVALID packets on return traffic from real servers.  It also 
prevents connection synchronization to the backup fw/director for 
failover.  Google has been giving me conflicting results on the 
following questions:

* Do the antefacto patches allow netfilter to access connections managed 
by ipvs and support DNAT, SNAT and fwmarks used in the LVS configuration?

* Has anyone gotten this to work on RHEL/CentOS via a kernel recompile 
with the antefacto patches?

If so, is there anything needed beyond the following?:

1. Recompile CentOS kernel (2.6.18 ok?) with Antefacto patches 
(http://www.ssi.bg/~ja/nfct/)

2. Setup conntrackd - will mirror the connection information 
synchronized by keepalived at the netfilter level.  Will conntrackd work 
on RHEL/CentOS 5.2?

Are libntnetlink or libnetfilter_conntrack required?  I have been 
reading all day but don't yet follow how all of the pieces go together. 
  Many thanks for any advice here...


Brian

Re: What's required for a stateful firewall + ipvs in 2.6 kernel?

[Grant Taylor]

On 09/09/08 18:47, Brian Ghidinelli wrote:
> I'm trying to get a handle on whether or not it's possible to set up the 
> following on a redundant pair of boxes:
> 
> 1. Stateful iptables firewall
> 2. LVS director (keepalived)
> 3. DNAT, SNAT and fwmarks
> 4. Connection synchronization for failover

You should easily be able to get SPI (1), NAT (3), and failover (4) 
between multiple systems.  However I'm not sure if you will get LVS (2) 
to play properly in this or not.  Traditionally LVS worked independently 
/ completely out side of IPTables (1 and 3) and thus was not able to be 
synchronized / failed over (4) between multiple boxen.  This does not 
mean that it can not be done, just that it is not going to be documented 
in the usual locations if it is possible.

> * Do the antefacto patches allow netfilter to access connections managed 
> by ipvs and support DNAT, SNAT and fwmarks used in the LVS configuration?

Based on the (below) referenced web page from Julian, yes to some extent 
it does..

> 2. Setup conntrackd - will mirror the connection information 
> synchronized by keepalived at the netfilter level.  Will conntrackd work 
> on RHEL/CentOS 5.2?

It is my (mis)understanding that keepalived does not do the 
synchronization, rather just the monitoring of things.  Conntrackd will 
do the synchronization for NetFilter.

As far as whether or not conntrackd will work on RHEL/CentOS, it should. 
  I don't know of any reason you can't compile it and get it to work. 
You may have to change some underlying libraries if versions are not 
correct (I don't know b/c I run different distro(s)).

> Are libntnetlink or libnetfilter_conntrack required?  I have been 
> reading all day but don't yet follow how all of the pieces go together. 

I don't know.  If you read the documentation with conntrackd you should 
be able to find out if libnetlink / libnetfilter are needed or not.  I 
would not be surprised if you need libnetfilter.



Grant. . . .

Re: What's required for a stateful firewall + ipvs in 2.6 kernel?

[Brian Ghidinelli]

Grant Taylor wrote:
>> 1. Stateful iptables firewall
>> 2. LVS director (keepalived)
>> 3. DNAT, SNAT and fwmarks
>> 4. Connection synchronization for failover
> 
 > ...
> synchronized / failed over (4) between multiple boxen.  This does not 
> mean that it can not be done, just that it is not going to be documented 
> in the usual locations if it is possible.

That's the issue... there are a lot of posts about LVS and netfilter on 
Austintek.com and other sites but the dates range from 2000 to 2006 or 
so making it hard to figure out what's current.

In sysadmining, I don't really like to be the pioneer. :)  No one else 
has turned an RHEL box into a Firewall + LVS Director?


> It is my (mis)understanding that keepalived does not do the 
> synchronization, rather just the monitoring of things.  Conntrackd will 
> do the synchronization for NetFilter.

I believe keepalived synchronizes the LVS connections between ipvs on 
the two boxes.  There is a config option "lvs_sync_daemon_inteface" for 
this (as I understand it).

This is only half the picture though, and conntrackd appears to solve 
the other half by also keeping netfilter in sync about which connections 
are already established or related so iptables rules don't kill valid 
sessions.

So in the end I suppose the real question is whether or not anyone has 
successfully used the Antefacto patches on RHEL?  I will try the 
lvs-users mailing list for that one...

Thanks for the help Grant,


Brian

Re: What's required for a stateful firewall + ipvs in 2.6 kernel?

[Grant Taylor]

On 09/10/08 12:00, Brian Ghidinelli wrote:
> That's the issue... there are a lot of posts about LVS and netfilter on 
> Austintek.com and other sites but the dates range from 2000 to 2006 or 
> so making it hard to figure out what's current.

*nod*  This is the case with a lot of things, not just LVS.

> In sysadmining, I don't really like to be the pioneer. :)  No one else 
> has turned an RHEL box into a Firewall + LVS Director?

I doubt that you are the first, but I don't know that others have 
documented things for people to find.

> I believe keepalived synchronizes the LVS connections between ipvs on 
> the two boxes.  There is a config option "lvs_sync_daemon_inteface" for 
> this (as I understand it).

Ok...

> This is only half the picture though, and conntrackd appears to solve 
> the other half by also keeping netfilter in sync about which connections 
> are already established or related so iptables rules don't kill valid 
> sessions.

*nod*

> So in the end I suppose the real question is whether or not anyone has 
> successfully used the Antefacto patches on RHEL?  I will try the 
> lvs-users mailing list for that one...

Please follow up with what you find so others searching this archive in 
the future will have some information.

> Thanks for the help Grant,

You are welcome.



Grant. . . .

Re: What's required for a stateful firewall + ipvs in 2.6 kernel?

[Pablo Neira Ayuso]

Brian Ghidinelli wrote:
> I'm trying to get a handle on whether or not it's possible to set up the
> following on a redundant pair of boxes:
> 
> 1. Stateful iptables firewall
> 2. LVS director (keepalived)
> 3. DNAT, SNAT and fwmarks
> 4. Connection synchronization for failover
> 
> I currently have CentOS/RHEL 5 running 1, 2 and 3 above but the RHEL
> 2.6.18-* kernels don't export LVS connections to netfilter resulting in
> lots of INVALID packets on return traffic from real servers.  It also
> prevents connection synchronization to the backup fw/director for
> failover.  Google has been giving me conflicting results on the
> following questions:
> 
> * Do the antefacto patches allow netfilter to access connections managed
> by ipvs and support DNAT, SNAT and fwmarks used in the LVS configuration?
> 
> * Has anyone gotten this to work on RHEL/CentOS via a kernel recompile
> with the antefacto patches?
> 
> If so, is there anything needed beyond the following?:
> 
> 1. Recompile CentOS kernel (2.6.18 ok?) with Antefacto patches
> (http://www.ssi.bg/~ja/nfct/)

The last time that I had a look at the antefacto patch it look to me
like a hack. IIRC, the problem is the LVS design (at least time ago when
I had a look at it) as it bypasses the network stack. This screws up the
possibility of having stateful firewalling and LVS.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: What's required for a stateful firewall + ipvs in 2.6 kernel?

[Grant Taylor]

On 09/23/08 05:09, Pablo Neira Ayuso wrote:
> The last time that I had a look at the antefacto patch it look to me 
> like a hack. IIRC, the problem is the LVS design (at least time ago 
> when I had a look at it) as it bypasses the network stack. This 
> screws up the possibility of having stateful firewalling and LVS.

I can offer a recent confirmation (with in the last three months) that 
LVS does indeed still interfere with firewalling.



Grant. . . .