Ugly issue with conntrack

Ugly issue with conntrack

[Juan Antonio]

Hello everyone,

I have a extrange issue with a conntrack entry. There is a nat server
configure in this way

    DMZ  194.139.30.0/23  ---  194.139.30.16  nat 192.168.12.100 ---- 
192.168.12.0/24  private network

The nat machine does postrouting in all traffic from the private network
to DMZ, and there is no problem but in one server in the DMZ with
windows 2008 server the traffic doesn't return to the origin, I can see
the traffic with tcpdump like this

17:19:23.971978 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto:
ICMP (1), length: 84) 192.168.12.91 > 194.139.30.62: ICMP echo request,
id 12075, seq 1, length 64    <-----   The echo request original OK

17:19:23.972094 IP (tos 0x0, ttl  63, id 0, offset 0, flags [DF], proto:
ICMP (1), length: 84) 194.139.30.16 > 194.139.30.62: ICMP echo request,
id 12075, seq 1, length 64    <------ Masquerade the source IP OK

17:19:23.972164 IP (tos 0x0, ttl 128, id 25050, offset 0, flags [none],
proto: ICMP (1), length: 84) 194.139.30.62 > 194.139.30.16: ICMP echo
reply, id 12075, seq 1, length 64    <------- The echo reply OK

¿?¿?¿?   <-----------    Lost echo reply not OK

There isn't the packet from 194.139.30.16 to 192.168.12.91 despite off
the conntrack table show

cat /proc/net/ip_conntrack | grep '30.62'
icmp     1 29 src=192.168.12.91 dst=194.139.30.62 type=8 code=0 id=12075
packets=11 bytes=924 [UNREPLIED] src=194.139.30.62 dst=194.139.30.16
type=0 code=0 id=12075 packets=0 bytes=0 mark=0 use=1

The packet in tcpdump match on the conntrack entry. "id 12075" in both
cases, but if I LOG the traffic with the LOG iptables target I see the
reply in INPUT table not in the FORWARD.

Thank you and sorry for me bad english.

Re: Ugly issue with conntrack

[Mart Frauenlob]

On 16.03.2010 18:15, pushakk@limbo.ari.es wrote:
> Hello everyone,
> 
> I have a extrange issue with a conntrack entry. There is a nat server
> configure in this way
> 
>     DMZ  194.139.30.0/23  ---  194.139.30.16  nat 192.168.12.100 ---- 
> 192.168.12.0/24  private network
> 
> The nat machine does postrouting in all traffic from the private network
> to DMZ, and there is no problem but in one server in the DMZ with
> windows 2008 server the traffic doesn't return to the origin, I can see
> the traffic with tcpdump like this
> 
> 17:19:23.971978 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto:
> ICMP (1), length: 84) 192.168.12.91 > 194.139.30.62: ICMP echo request,
> id 12075, seq 1, length 64    <-----   The echo request original OK
> 
> 17:19:23.972094 IP (tos 0x0, ttl  63, id 0, offset 0, flags [DF], proto:
> ICMP (1), length: 84) 194.139.30.16 > 194.139.30.62: ICMP echo request,
> id 12075, seq 1, length 64    <------ Masquerade the source IP OK
> 
> 17:19:23.972164 IP (tos 0x0, ttl 128, id 25050, offset 0, flags [none],
> proto: ICMP (1), length: 84) 194.139.30.62 > 194.139.30.16: ICMP echo
> reply, id 12075, seq 1, length 64    <------- The echo reply OK
> 
> ¿?¿?¿?   <-----------    Lost echo reply not OK
> 
> There isn't the packet from 194.139.30.16 to 192.168.12.91 despite off
> the conntrack table show
> 
> cat /proc/net/ip_conntrack | grep '30.62'
> icmp     1 29 src=192.168.12.91 dst=194.139.30.62 type=8 code=0 id=12075
> packets=11 bytes=924 [UNREPLIED] src=194.139.30.62 dst=194.139.30.16
> type=0 code=0 id=12075 packets=0 bytes=0 mark=0 use=1
> 
> The packet in tcpdump match on the conntrack entry. "id 12075" in both
> cases, but if I LOG the traffic with the LOG iptables target I see the
> reply in INPUT table not in the FORWARD.
> 
> Thank you and sorry for me bad english.
> 
This behaviour indicates, that conntrack classifies the traffic into
state INVALID. Thus it is not natted, as stateful nat needs traffic to
be valid for conntrack.
I don't know why it happens in that particular case, but you can try to
debug it a little more.
If your kernel supports it, you can set
/proc/sys/net/netfilter/nf_conntrack_log_invalid to 1.

-m state --state INVALID -j LOG --log-level debug ....
in INPUT/FORWARD.

Also providing iptables-save output, kernel version, etc.. is helpful.

As this only happens with win2008, you might try to find some IP
settings there?

Best regards

Mart

Re: Ugly issue with conntrack

[Juan Antonio]

Hi Mart,

thank you very much for your reply, I will look for INVALID STATE doc 
and try LOG this traffic. I suposse the issue is related with the 
windows 2008 server because if I config the same public ip to another 
server this reply the echoes corectly so I think it's not a intrinsic 
netfilter issue.

I suposse the problem is in some special config in the 2008 server, this 
is a client machine so I have no access to the system.

Thank you.

El 17/03/10 16:33, Mart Frauenlob escribió:
> On 16.03.2010 18:15, pushakk@limbo.ari.es wrote:
>    
>> Hello everyone,
>>
>> I have a extrange issue with a conntrack entry. There is a nat server
>> configure in this way
>>
>>      DMZ  194.139.30.0/23  ---  194.139.30.16  nat 192.168.12.100 ----
>> 192.168.12.0/24  private network
>>
>> The nat machine does postrouting in all traffic from the private network
>> to DMZ, and there is no problem but in one server in the DMZ with
>> windows 2008 server the traffic doesn't return to the origin, I can see
>> the traffic with tcpdump like this
>>
>> 17:19:23.971978 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto:
>> ICMP (1), length: 84) 192.168.12.91>  194.139.30.62: ICMP echo request,
>> id 12075, seq 1, length 64<-----   The echo request original OK
>>
>> 17:19:23.972094 IP (tos 0x0, ttl  63, id 0, offset 0, flags [DF], proto:
>> ICMP (1), length: 84) 194.139.30.16>  194.139.30.62: ICMP echo request,
>> id 12075, seq 1, length 64<------ Masquerade the source IP OK
>>
>> 17:19:23.972164 IP (tos 0x0, ttl 128, id 25050, offset 0, flags [none],
>> proto: ICMP (1), length: 84) 194.139.30.62>  194.139.30.16: ICMP echo
>> reply, id 12075, seq 1, length 64<------- The echo reply OK
>>
>> ¿?¿?¿?<-----------    Lost echo reply not OK
>>
>> There isn't the packet from 194.139.30.16 to 192.168.12.91 despite off
>> the conntrack table show
>>
>> cat /proc/net/ip_conntrack | grep '30.62'
>> icmp     1 29 src=192.168.12.91 dst=194.139.30.62 type=8 code=0 id=12075
>> packets=11 bytes=924 [UNREPLIED] src=194.139.30.62 dst=194.139.30.16
>> type=0 code=0 id=12075 packets=0 bytes=0 mark=0 use=1
>>
>> The packet in tcpdump match on the conntrack entry. "id 12075" in both
>> cases, but if I LOG the traffic with the LOG iptables target I see the
>> reply in INPUT table not in the FORWARD.http://www.google.com/firefox
>>
>> Thank you and sorry for me bad english.
>>
>>      
> This behaviour indicates, that conntrack classifies the traffic into
> state INVALID. Thus it is not natted, as stateful nat needs traffic to
> be valid for conntrack.
> I don't know why it happens in that particular case, but you can try to
> debug it a little more.
> If your kernel supports it, you can set
> /proc/sys/net/netfilter/nf_conntrack_log_invalid to 1.
>
> -m state --state INVALID -j LOG --log-level debug ....
> in INPUT/FORWARD.
>
> Also providing iptables-save output, kernel version, etc.. is helpful.
>
> As this only happens with win2008, you might try to find some IP
> settings there?
>
> Best regards
>
> Mart
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>    

Re: Ugly issue with conntrack

[Juan Antonio]

Hello again Mart,

I have recover some more data about the problem.

uname -r
2.6.17-5mdv

iptables --version
iptables v1.3.5

ping -c 1 tcpdump out

(tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: ICMP (1), length:
84) 192.168.12.91 > 195.139.30.62: ICMP echo request, id 30011, seq 1,
length 64

(tos 0x0, ttl  63, id 0, offset 0, flags [DF], proto: ICMP (1), length:
84) 195.139.30.16 > 195.139.30.62: ICMP echo request, id 30011, seq 1,
length 64

(tos 0x0, ttl 128, id 6454, offset 0, flags [none], proto: ICMP (1),
length: 84) 195.139.30.62 > 195.139.30.16: ICMP echo reply, id 30011,
seq 1, length 64

no more traffic here

iptables LOG rules

INPUT chain
target     prot opt source               destination
LOG        icmp --  0.0.0.0/0            195.139.30.62      LOG flags 0
level 7 prefix `dst-nexus-input '
LOG        icmp --  195.139.30.62       0.0.0.0/0           LOG flags 0
level 7 prefix `src-nexus-input '

FORWARD chain
target     prot opt source               destination
LOG        icmp --  0.0.0.0/0            195.139.30.62      LOG flags 0
level 7 prefix `dst-nexus-forward '
LOG        icmp --  195.139.30.62       0.0.0.0/0           LOG flags 0
level 7 prefix `src-nexus-forward '

and this rules LOG
Mar 18 09:33:14 thunderNAT kernel: dst-nexus-forward IN=eth0 OUT=eth1
SRC=192.168.12.91 DST=195.139.30.62 LEN=84 TOS=0x00 PREC=0x00 TTL=63
ID=0 DF PROTO=ICMP
TYPE=8 CODE=0 ID=30011 SEQ=1

Mar 18 09:33:14 thunderNAT kernel: src-nexus-input IN=eth1 OUT=
MAC=00:0e:0c:c6:b0:49:00:1e:c9:b9:db:9d:08:00 SRC=195.139.30.62
DST=195.248.230.16 LEN=84 TOS=
0x00 PREC=0x00 TTL=128 ID=6454 PROTO=ICMP TYPE=0 CODE=1 ID=30011 SEQ=1

Like will see the reply enter in the src-nexus-input chain despite off
the conntrack entry show

icmp     1 29 src=192.168.12.91 dst=195.139.30.62 type=8 code=0 id=30011
packets=19 bytes=1596 [UNREPLIED] src=195.139.30.62 dst=195.139.30.16
type=0 code=0 id=30011 packets=0 bytes=0 mark=0 use=1

same id but no match.

I have LOG the invalid conntrack states but it doesn't match anything
about this traffic.

Tahnk you again for your help.

El 17/03/10 16:33, Mart Frauenlob escribió:
> On 16.03.2010 18:15, pushakk@limbo.ari.es wrote:
>    
>> Hello everyone,
>>
>> I have a extrange issue with a conntrack entry. There is a nat server
>> configure in this way
>>
>>      DMZ  194.139.30.0/23  ---  194.139.30.16  nat 192.168.12.100 ----
>> 192.168.12.0/24  private network
>>
>> The nat machine does postrouting in all traffic from the private network
>> to DMZ, and there is no problem but in one server in the DMZ with
>> windows 2008 server the traffic doesn't return to the origin, I can see
>> the traffic with tcpdump like this
>>
>> 17:19:23.971978 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto:
>> ICMP (1), length: 84) 192.168.12.91>  194.139.30.62: ICMP echo request,
>> id 12075, seq 1, length 64<-----   The echo request original OK
>>
>> 17:19:23.972094 IP (tos 0x0, ttl  63, id 0, offset 0, flags [DF], proto:
>> ICMP (1), length: 84) 194.139.30.16>  194.139.30.62: ICMP echo request,
>> id 12075, seq 1, length 64<------ Masquerade the source IP OK
>>
>> 17:19:23.972164 IP (tos 0x0, ttl 128, id 25050, offset 0, flags [none],
>> proto: ICMP (1), length: 84) 194.139.30.62>  194.139.30.16: ICMP echo
>> reply, id 12075, seq 1, length 64<------- The echo reply OK
>>
>> ¿?¿?¿?<-----------    Lost echo reply not OK
>>
>> There isn't the packet from 194.139.30.16 to 192.168.12.91 despite off
>> the conntrack table show
>>
>> cat /proc/net/ip_conntrack | grep '30.62'
>> icmp     1 29 src=192.168.12.91 dst=194.139.30.62 type=8 code=0 id=12075
>> packets=11 bytes=924 [UNREPLIED] src=194.139.30.62 dst=194.139.30.16
>> type=0 code=0 id=12075 packets=0 bytes=0 mark=0 use=1
>>
>> The packet in tcpdump match on the conntrack entry. "id 12075" in both
>> cases, but if I LOG the traffic with the LOG iptables target I see the
>> reply in INPUT table not in the FORWARD.http://www.google.com/firefox
>>
>> Thank you and sorry for me bad english.
>>
>>      
> This behaviour indicates, that conntrack classifies the traffic into
> state INVALID. Thus it is not natted, as stateful nat needs traffic to
> be valid for conntrack.
> I don't know why it happens in that particular case, but you can try to
> debug it a little more.
> If your kernel supports it, you can set
> /proc/sys/net/netfilter/nf_conntrack_log_invalid to 1.
>
> -m state --state INVALID -j LOG --log-level debug ....
> in INPUT/FORWARD.
>
> Also providing iptables-save output, kernel version, etc.. is helpful.
>
> As this only happens with win2008, you might try to find some IP
> settings there?
>
> Best regards
>
> Mart
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>    

--
To unsubscribe from this list: send the line "unsubscribe netfilter" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html