Netfilter cluster / Invalid state problem

Netfilter cluster / Invalid state problem

[Pierre Westeel]

Hi,

I have a Linux netfilter cluster with keepalived to perform high avalaibility.
The master runs a 2.6.12 kernel and the Backup runs a 2.4.26 kernel. (
both from kernel.org without patch )

I have the same iptables config on  firewalls  ( Only usefuls rules
are pasted below ) :

-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

-A INPUT -m state --state INVALID -j drop_invalid
-A FORWARD -m state --state INVALID -j drop_invalid
-A OUTPUT -m state --state INVALID -j drop_invalid
-A drop_invalid -j LOG --log-prefix "INVALID state -- DENY " --log-level 7
-A drop_invalid -j DROP

---

-A INPUT -d 10.24.240.0/255.255.248.0 -m state --state NEW -j ACCEPT
-A INPUT -s 10.24.240.0/255.255.248.0 -m state --state NEW -j ACCEPT
-A OUTPUT -d 10.24.240.0/255.255.248.0 -m state --state NEW -j ACCEPT
-A OUTPUT -s 10.24.240.0/255.255.248.0 -m state --state NEW -j ACCEPT
-A FORWARD -d 10.24.240.0/255.255.248.0 -m state --state NEW -j ACCEPT
-A FORWARD -s 10.24.240.0/255.255.248.0 -m state --state NEW -j ACCEPT

---

When I swap all the connections on the backup firewall with stopping
keepalived daemon on master, the connections are correctly forwarded
through the backup and I can see the new entry in
/proc/net/ip_conntrack
( This is a VNC stream , the connexion is never Idle so the sequence
number is increasing quickly )

BUT when i restart the keepalived daemon to make connection go back
through the master firewall 30 seconds after the first swap, i get the
following logs :

INVALID state -- DENY IN=eth0.730 OUT=eth0.732 SRC=172.18.130.194
DST=10.24.247.253 LEN=46 TOS=0x00 PREC=0x00 TTL=126 ID=46274 DF
PROTO=TCP SPT=1522 DPT=5901 WINDOW=17520 RES=0x00 ACK PSH URGP=0
INVALID state -- DENY IN=eth0.732 OUT=eth0.730 SRC=10.24.247.253
DST=172.18.130.194 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=30732 DF
PROTO=TCP SPT=5901 DPT=1522 WINDOW=5840 RES=0x00 ACK URGP=0
INVALID state -- DENY IN=eth0.732 OUT=eth0.730 SRC=10.24.247.253
DST=172.18.130.194 LEN=1040 TOS=0x00 PREC=0x00 TTL=63 ID=30733 DF
PROTO=TCP SPT=5901 DPT=1522 WINDOW=5840 RES=0x00 ACK PSH URGP=0
INVALID state -- DENY IN=eth0.730 OUT=eth0.732 SRC=172.18.130.194
DST=10.24.247.253 LEN=46 TOS=0x00 PREC=0x00 TTL=126 ID=46276 DF
PROTO=TCP SPT=1522 DPT=5901 WINDOW=17520 RES=0x00 ACK PSH URGP=0

But the connection is still present in  /proc/net/ip_conntrack :

tcp      6 431933 ESTABLISHED src=172.18.130.194 dst=10.24.247.253
sport=1522 dport=5901 src=10.24.247.253 dst=172.18.130.194 sport=5901
dport=1522 [ASSURED] use=1

When I swap the master with the backup it works correctly. ( 2.4.26 ->
2.6.12 -> 2.4.26 instead of 2.6.12 -> 2.4.26 -> 2.6.12 )
It seems that 2.6.12 is checking TCP  sequence number for conntracking
... but I don't use patch-o-matic.

This configuration works well with 2 firewall with  2.4.26 kernel

Any ideas ?

Regards,

---
Pierre Westeel

Re: Netfilter cluster / Invalid state problem

[Pablo Neira]

Pierre Westeel wrote:
> I have a Linux netfilter cluster with keepalived to perform high avalaibility.
> The master runs a 2.6.12 kernel and the Backup runs a 2.4.26 kernel. (
> both from kernel.org without patch )
> 
> When I swap all the connections on the backup firewall with stopping
> keepalived daemon on master, the connections are correctly forwarded
> through the backup and I can see the new entry in
> /proc/net/ip_conntrack
> ( This is a VNC stream , the connexion is never Idle so the sequence
> number is increasing quickly )
> 
> BUT when i restart the keepalived daemon to make connection go back
> through the master firewall 30 seconds after the first swap, i get the
> following logs :
> 
> INVALID state -- DENY IN=eth0.730 OUT=eth0.732 SRC=172.18.130.194
> DST=10.24.247.253 LEN=46 TOS=0x00 PREC=0x00 TTL=126 ID=46274 DF
> PROTO=TCP SPT=1522 DPT=5901 WINDOW=17520 RES=0x00 ACK PSH URGP=

Try this:

# echo 1 > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_be_liberal

on the linux-2.6 box. See that later linux kernels 2.6 include TCP 
tracking active by default. Setting ip_conntrack_tcp_be_liberal to 1 
relaxes the in-window checkings. As you can figure out, if linux-2.6 
takes over, it won't know anything about the current active connections, 
so it will consider that they are invalid.

--
Pablo