Conntrack rule timeout problem

Conntrack rule timeout problem

[Pat Riehecky]

I seem to be capturing way more packets than I intend (or even expect!).
I am running squid and have the firewall rules below running on it.  For
some reason I am capturing hundreds of packets that I don't think should
be caught.

I have increased the timeouts in /proc/ (via sysctl) to fix this, but no
dice.  Anyone have any idea why the sample packet below would be
captured?  It is getting picked up by either the 
-A INPUT -p tcp -m tcp ! --syn -m conntrack --ctstate INVALID -j DROP
but sometimes the
-A INPUT -p tcp -m tcp ! --syn -m state --state NEW -j DROP

The packet looks to have been requested by squid, it is coming on port
80...  I also seem to be having the same behavior on the squid side
where the FIN/ACK packets are being caught by the conntrack rule...

I know I have something wrong, just what exactly is eluding me...

Any help would be helpful!

--- sample capture A ---
IN=eth0 OUT= MAC=00:0a:e9:6b:9e:ba:00:10:80:ec:37:5f:08:00
SRC=199.239.137.245 DST=XXX.YYY.ZZZ.AAA LEN=64 TOS=0x00 PREC=0x00 TTL=56
ID=620 DF PROTO=TCP SPT=80 DPT=60338 WINDOW=37260 RES=0x00 ACK URGP=0

--sample capture B ---
IN=eth0 OUT= MAC=00:0a:a9:60:9e:ba:00:10:80:ec:37:5f:08:00
SRC=222.28.249.164 DST=XXX.YYY.ZZZ.AAA LEN=40 TOS=0x00 PREC=0x00 TTL=109
ID=34912 DF PROTO=TCP SPT=1802 DPT=3128 WINDOW=65535 RES=0x00 ACK FIN
URGP=0

--- sysctl.conf ---
net.ipv4.netfilter.ip_conntrack_tcp_be_liberal=0
net.ipv4.netfilter.ip_conntrack_udp_timeout = 20
net.ipv4.netfilter.ip_conntrack_udp_timeout_stream = 95
net.ipv4.netfilter.ip_conntrack_generic_timeout=650
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close=20
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait=180

--- rules ---
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m state
--state NEW -j DROP
-A INPUT -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack
--ctstate INVALID -j DROP
-A INPUT -p tcp -m tcp --tcp-flags SYN,ACK SYN,ACK -m state --state NEW
-j REJECT --reject-with tcp-reset
-A INPUT -p icmp -m icmp --icmp-type 8 -m limit --limit 4/sec
--limit-burst 9 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -j DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 113 -j REJECT --reject-with tcp-reset
-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 3128 -m state --state NEW -j ACCEPT
-A INPUT -j DROP
-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -s 255.0.0.0/255.0.0.0 -j DROP
-A OUTPUT -s 0.0.0.0/255.0.0.0 -j DROP
-A OUTPUT -s 192.168.0.0/255.255.0.0 -j DROP
-A OUTPUT -p icmp -m icmp --icmp-type 8 -m limit --limit 9/sec
--limit-burst 15 -j ACCEPT
-A OUTPUT -p icmp -m icmp --icmp-type 8 -j DROP
-A OUTPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
-A OUTPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
-A OUTPUT -p tcp -m multiport --dports 80,443 -m state --state NEW -j
ACCEPT
-A OUTPUT -j DROP

Re: Conntrack rule timeout problem

[Gáspár Lajos]

Hi,

Pat Riehecky írta:
> I seem to be capturing way more packets than I intend (or even expect!).
> I am running squid and have the firewall rules below running on it.  For
> some reason I am capturing hundreds of packets that I don't think should
> be caught.
>   
Maybe someone is scanning you....
> I have increased the timeouts in /proc/ (via sysctl) to fix this, but no
> dice.  Anyone have any idea why the sample packet below would be
> captured?  It is getting picked up by either the 
> -A INPUT -p tcp -m tcp ! --syn -m conntrack --ctstate INVALID -j DROP
> but sometimes the
> -A INPUT -p tcp -m tcp ! --syn -m state --state NEW -j DROP
>   
Take a look on nmap...
> The packet looks to have been requested by squid, it is coming on port
> 80...  I also seem to be having the same behavior on the squid side
> where the FIN/ACK packets are being caught by the conntrack rule...
>
> I know I have something wrong, just what exactly is eluding me...
>
> Any help would be helpful!
>   
Swifty

Re: Conntrack rule timeout problem

[Pat Riehecky]

On Wed, 2007-05-23 at 14:47 +0200, Gáspár Lajos wrote:
> Hi,
> 
> Pat Riehecky írta:
> > I seem to be capturing way more packets than I intend (or even expect!).
> > I am running squid and have the firewall rules below running on it.  For
> > some reason I am capturing hundreds of packets that I don't think should
> > be caught.
> >   
> Maybe someone is scanning you....
> > I have increased the timeouts in /proc/ (via sysctl) to fix this, but no
> > dice.  Anyone have any idea why the sample packet below would be
> > captured?  It is getting picked up by either the 
> > -A INPUT -p tcp -m tcp ! --syn -m conntrack --ctstate INVALID -j DROP
> > but sometimes the
> > -A INPUT -p tcp -m tcp ! --syn -m state --state NEW -j DROP
> >   
> Take a look on nmap...
> > The packet looks to have been requested by squid, it is coming on port
> > 80...  I also seem to be having the same behavior on the squid side
> > where the FIN/ACK packets are being caught by the conntrack rule...
> >
> > I know I have something wrong, just what exactly is eluding me...
> >
> > Any help would be helpful!
> >   
> Swifty
> 

I am about 90% certain that I am not being scanned as a bunch of the
dropped packets are coming from places like the New York Times,
Microsoft, and Google.  Admittedly they could be spoofed IP addresses.
but the packets are all coming from 80 or 443 and they are all destined
for TCP Ports in the ephemeral range.  Additionally in my squid logs I
have a corresponding entry requesting data from that server.

All evidence I have points to some sort of conntrack timeout.
Occasionally I can find the IP addresses in the output from iptstate,
but... 

Thanks for the ideas, any chance for more theories?
Pat

Re: Conntrack rule timeout problem

[Gáspár Lajos]

Pat Riehecky írta:
> I am about 90% certain that I am not being scanned as a bunch of the
> dropped packets are coming from places like the New York Times,
> Microsoft, and Google.  Admittedly they could be spoofed IP addresses.
> but the packets are all coming from 80 or 443 and they are all destined
> for TCP Ports in the ephemeral range.  Additionally in my squid logs I
> have a corresponding entry requesting data from that server.
>
>   
Well... Read this:

http://www.hackinthebox.org/modules.php?op=modload&name=News&file=article&sid=10640&mode=thread&order=0&thold=0

The interesting part starts at *"Camouflaging your ip address"...*
> All evidence I have points to some sort of conntrack timeout.
> Occasionally I can find the IP addresses in the output from iptstate,
> but... 
>
> Thanks for the ideas, any chance for more theories?
> Pat
>   
Swifty

Re: Conntrack rule timeout problem

[Pat Riehecky]

That is an excellent article!

I attempted to simplify the oddities I am seeing to avoid being overly
complex, but it seems that was in error....

Here is a packet that was caught on its way out of my server, it cannot
be part of a FORWARD chain as my FORWARD chain looks like this

:FORWARD DROP [0:0]
-A FORWARD -j LOG --log-prefix "Default DROP (FORWARD): " 
-A FORWARD -j DROP 

Simply put the answer to any forward is "NO"

The packet is :
Default DROP (OUTPUT): IN= OUT=eth0 SRC=192.168.12.74 DST=66.28.242.99
LEN=344 TOS=0x00 PREC=0x00 TTL=64 ID=13688 DF PROTO=TCP SPT=60155 DPT=80
WINDOW=5840 RES=0x00 ACK FIN URGP=0

It tries for a while and then gives up.  This feels identical to the
input scenario.  The last packet seems to not be getting through as
RELATED/ESTABLISHED.  After studying the flow with iptstate 
(a bit poorly, but it was a start) this seems to occur when a connection
is closed - but not when all connections are closed.  This leads me to
believe that it has to be related to conntrack.

Is my reasoning on this flawed?
Pat

On Wed, 2007-05-23 at 18:27 +0200, Gáspár Lajos wrote:
> Pat Riehecky írta:
> > I am about 90% certain that I am not being scanned as a bunch of the
> > dropped packets are coming from places like the New York Times,
> > Microsoft, and Google.  Admittedly they could be spoofed IP addresses.
> > but the packets are all coming from 80 or 443 and they are all destined
> > for TCP Ports in the ephemeral range.  Additionally in my squid logs I
> > have a corresponding entry requesting data from that server.
> >
> >   
> Well... Read this:
> 
> http://www.hackinthebox.org/modules.php?op=modload&name=News&file=article&sid=10640&mode=thread&order=0&thold=0
> 
> The interesting part starts at *"Camouflaging your ip address"...*
> > All evidence I have points to some sort of conntrack timeout.
> > Occasionally I can find the IP addresses in the output from iptstate,
> > but... 
> >
> > Thanks for the ideas, any chance for more theories?
> > Pat
> >   
> Swifty
>