* [Fwd: Re: Networking messed up, bad checksum, incorrect length]
@ 2006-10-29 23:26 Carlos Velasco
2006-10-30 8:10 ` Jozsef Kadlecsik
0 siblings, 1 reply; 4+ messages in thread
From: Carlos Velasco @ 2006-10-29 23:26 UTC (permalink / raw)
To: netfilter-devel
[-- Attachment #1: Type: text/plain, Size: 7494 bytes --]
I'm forwarding a mail from Linux Kernel, as it seems a bug in Netfilter
in 2.6.18.
Source server is:
Linux Merak 2.6.18.1 #1 SMP Fri Oct 27 20:56:26 CEST 2006 x86_64 x86_64
x86_64 GNU/Linux
Modules:
Module Size Used by
ipv6 279392 32
xt_tcpudp 7680 37
iptable_nat 11012 0
ip_nat 20396 1 iptable_nat
ipt_LOG 11008 1
ipt_REJECT 8960 2
xt_state 6528 39
ip_conntrack_tftp 8280 0
ip_conntrack_ftp 11088 0
ip_conntrack 52484 5
iptable_nat,ip_nat,xt_state,ip_conntrack_tftp,ip_conntrack_ftp
iptable_filter 7168 1
ip_tables 23656 2 iptable_nat,iptable_filter
x_tables 17800 6
xt_tcpudp,iptable_nat,ipt_LOG,ipt_REJECT,xt_state,ip_tables
tg3 111364 0
# iptables -L
Chain INPUT (policy DROP)
target prot opt source destination
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere state
RELATED,ESTABLISHED
ACCEPT udp -- anywhere anywhere state NEW
udp dpt:ntp
ACCEPT udp -- anywhere anywhere state NEW
udp spts:bootps:bootpc dpt:bootps
ACCEPT tcp -- anywhere anywhere state NEW
tcp dpt:http
ACCEPT tcp -- anywhere anywhere state NEW
tcp dpt:https
ACCEPT udp -- anywhere anywhere state NEW
udp dpt:domain
ACCEPT tcp -- anywhere anywhere state NEW
tcp dpt:domain
ACCEPT tcp -- anywhere anywhere state NEW
tcp dpt:smtp
ACCEPT tcp -- anywhere anywhere state NEW
tcp dpt:pop3
ACCEPT tcp -- anywhere anywhere state NEW
tcp dpt:imap
ACCEPT tcp -- 192.168.192.0/24 anywhere state NEW
tcp dpt:ssh
ACCEPT tcp -- 192.168.128.0/24 anywhere state NEW
tcp dpt:ssh
ACCEPT tcp -- 192.168.129.0/24 anywhere state NEW
tcp dpt:ssh
ACCEPT tcp -- 192.168.192.0/24 anywhere state NEW
tcp dpt:telnet
ACCEPT tcp -- 192.168.128.0/24 anywhere state NEW
tcp dpt:telnet
ACCEPT tcp -- 192.168.129.0/24 anywhere state NEW
tcp dpt:telnet
ACCEPT tcp -- merak.nimastelecom.com merak.nimastelecom.com state
NEW tcp dpts:bacula-dir:bacula-sd
ACCEPT tcp -- 192.168.192.210 merak.nimastelecom.com state
NEW tcp dpt:bacula-sd
ACCEPT udp -- merak.nimastelecom.com merak.nimastelecom.com state
NEW udp dpt:snmp
ACCEPT udp -- ROUTER merak.nimastelecom.com state
NEW udp dpt:snmptrap
ACCEPT tcp -- anywhere anywhere state NEW
tcp dpt:ftp
ACCEPT tcp -- 192.168.192.210 anywhere state NEW
tcp dpt:swat
ACCEPT tcp -- 192.168.192.0/24 anywhere state NEW
tcp dpt:netbios-ssn
ACCEPT tcp -- 192.168.128.0/24 anywhere state NEW
tcp dpt:netbios-ssn
ACCEPT tcp -- 192.168.129.0/24 anywhere state NEW
tcp dpt:netbios-ssn
ACCEPT tcp -- 192.168.192.0/24 anywhere state NEW
tcp dpt:microsoft-ds
ACCEPT tcp -- 192.168.128.0/24 anywhere state NEW
tcp dpt:microsoft-ds
ACCEPT tcp -- 192.168.129.0/24 anywhere state NEW
tcp dpt:microsoft-ds
ACCEPT udp -- 192.168.192.0/24 anywhere state NEW
udp dpt:netbios-ns
ACCEPT udp -- 192.168.128.0/24 anywhere state NEW
udp dpt:netbios-ns
ACCEPT udp -- 192.168.129.0/24 anywhere state NEW
udp dpt:netbios-ns
ACCEPT udp -- 192.168.192.0/24 anywhere state NEW
udp dpt:netbios-dgm
ACCEPT udp -- 192.168.128.0/24 anywhere state NEW
udp dpt:netbios-dgm
ACCEPT udp -- 192.168.129.0/24 anywhere state NEW
udp dpt:netbios-dgm
ACCEPT tcp -- ROUTER anywhere state NEW
tcp dpt:shell
ACCEPT udp -- ROUTER anywhere state NEW
udp dpt:syslog
ACCEPT tcp -- 192.168.192.201 anywhere state NEW
tcp dpt:shell
ACCEPT udp -- 192.168.192.201 anywhere state NEW
udp dpt:syslog
ACCEPT icmp -- anywhere anywhere state NEW
icmp echo-request
LOG all -- anywhere anywhere LOG level
warning prefix `FIREWALL:INPUT '
REJECT tcp -- anywhere anywhere reject-with
tcp-reset
REJECT all -- anywhere anywhere reject-with
icmp-port-unreachable
Chain FORWARD (policy DROP)
target prot opt source destination
Chain OUTPUT (policy DROP)
target prot opt source destination
ACCEPT all -- anywhere anywhere
Dropped packets showed in kernel log are:
Oct 29 23:29:30 kern:warning FIREWALL:INPUT IN=eth0 OUT=
MAC=00:13:21:cc:54:a6:00:0d:bc:a0:e2:82:08:00 SRC=193.147.150.12 DST=192.16
8.128.182 LEN=64 TOS=0x00 PREC=0x00 TTL=54 ID=50099 DF PROTO=TCP SPT=25
DPT=45020 WINDOW=39672 RES=0x00 ACK URGP=0
Oct 29 23:29:30 kern:warning FIREWALL:INPUT IN=eth0 OUT=
MAC=00:13:21:cc:54:a6:00:0d:bc:a0:e2:82:08:00 SRC=193.147.150.12 DST=192.16
8.128.182 LEN=64 TOS=0x00 PREC=0x00 TTL=54 ID=50100 DF PROTO=TCP SPT=25
DPT=45020 WINDOW=39672 RES=0x00 ACK URGP=0
Disabling Netfilter with:
/sbin/iptables --policy INPUT ACCEPT
/sbin/iptables --policy OUTPUT ACCEPT
/sbin/iptables --policy FORWARD ACCEPT
/sbin/iptables --flush
/sbin/iptables -t nat --flush
/sbin/iptables -t mangle --flush
/sbin/iptables --delete-chain
/sbin/iptables -t nat --delete-chain
/sbin/iptables -t mangle --delete-chain
Makes the transaction success.
Sniffer traces attached.
-------- Mensaje original --------
Asunto: Re: Networking messed up, bad checksum, incorrect length
Fecha: Sun, 29 Oct 2006 23:40:44 +0100
De: Carlos Velasco <lkml@newipnet.com>
Para: linux-kernel@vger.kernel.org, linux-net@vger.kernel.org
Referencias: <E1GdJNf-0002xT-00@gondolin.me.apana.org.au>
Following with this issue, disabling TSO does _not_ solve the problem in
the lost of connection in an outbound email.
It seems like a bug in Netfilter, not sure how to debug TCP conntrack,
but it seems as if Netfilter lost the Established state for this connection.
Looking at the traces, the only strange thing is that using
tcpdump/libpcap in the server I see packets being sent before the
previuos ACK is received, but I think this can't be posible, as the ACK
number of this packet is needed before sending the next.
I'm attaching the last traces taken with tso disabled.
smtptrace_flash1_bad2.pcap <- packets taken in the destination server
smtptrace_merak_bad2.pcap <- packets tanken in the source server (the
one with problem).
Analyzing the traces, it seems that a data packet is lost in transit to
destination server, so the destination server send DUP ACKs trying to
force the source server to resend this lost packet. But Netfilter is
filtering these DUP ACKs (showed in the kernel logs) and so, the source
server give up and reset the connection.
I think this to be a bug in Netfilter code. Again, any help would be
apreciated.
Regards,
Carlos Velasco
CCNP & CCDP Cisco Certified Network Professional
[-- Attachment #2: smtptrace_merak_bad2.pcap --]
[-- Type: application/octet-stream, Size: 5503 bytes --]
[-- Attachment #3: smtptrace_flash1_bad2.pcap --]
[-- Type: application/octet-stream, Size: 6047 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Fwd: Re: Networking messed up, bad checksum, incorrect length]
2006-10-29 23:26 [Fwd: Re: Networking messed up, bad checksum, incorrect length] Carlos Velasco
@ 2006-10-30 8:10 ` Jozsef Kadlecsik
2006-10-30 8:29 ` Jozsef Kadlecsik
0 siblings, 1 reply; 4+ messages in thread
From: Jozsef Kadlecsik @ 2006-10-30 8:10 UTC (permalink / raw)
To: Carlos Velasco; +Cc: netfilter-devel
On Mon, 30 Oct 2006, Carlos Velasco wrote:
> I'm forwarding a mail from Linux Kernel, as it seems a bug in Netfilter
> in 2.6.18.
The TCP session dies because a NAT device between the communicating
parties does not adjust the sequence numbers in the SACK fields.
Is there a NATing device, which is not identical with the machine running
2.6.28, between the client and the server?
Best regards,
Jozsef
-
E-mail : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
H-1525 Budapest 114, POB. 49, Hungary
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Fwd: Re: Networking messed up, bad checksum, incorrect length]
2006-10-30 8:10 ` Jozsef Kadlecsik
@ 2006-10-30 8:29 ` Jozsef Kadlecsik
2006-11-08 13:17 ` Carlos Velasco
0 siblings, 1 reply; 4+ messages in thread
From: Jozsef Kadlecsik @ 2006-10-30 8:29 UTC (permalink / raw)
To: Carlos Velasco; +Cc: netfilter-devel
On Mon, 30 Oct 2006, Jozsef Kadlecsik wrote:
> On Mon, 30 Oct 2006, Carlos Velasco wrote:
>
> > I'm forwarding a mail from Linux Kernel, as it seems a bug in Netfilter
> > in 2.6.18.
>
> The TCP session dies because a NAT device between the communicating
> parties does not adjust the sequence numbers in the SACK fields.
>
> Is there a NATing device, which is not identical with the machine running
> 2.6.28, between the client and the server?
No, it's not a NATing device. It's a 'smart' box which munges the TCP
sequence numbers and misses to do so in the SACK fields: the first packet
from both recordings:
23:29:27.912908 IP (tos 0x0, ttl 64, id 51776, offset 0, flags [DF],
length: 60) 192.168.128.182.45020 > 193.147.150.12.25: S [tcp sum ok]
426197099:426197099(0) win 5840
<mss 1460,sackOK,timestamp 46238251 0,nop,wscale 7>
23:29:28.017284 IP (tos 0x0, ttl 54, id 51776, offset 0, flags [DF],
length: 60) 84.77.121.105.45020 > 193.147.150.12.25: S [tcp sum ok]
888737236:888737236(0) win 5840
<mss 1380,sackOK,timestamp 46238251 0,nop,wscale 7>
It's definitely not a 2.6.18 bug.
Best regards,
Jozsef
-
E-mail : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
H-1525 Budapest 114, POB. 49, Hungary
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Fwd: Re: Networking messed up, bad checksum, incorrect length]
2006-10-30 8:29 ` Jozsef Kadlecsik
@ 2006-11-08 13:17 ` Carlos Velasco
0 siblings, 0 replies; 4+ messages in thread
From: Carlos Velasco @ 2006-11-08 13:17 UTC (permalink / raw)
To: Jozsef Kadlecsik; +Cc: netfilter-devel
Jozsef Kadlecsik escribió:
> On Mon, 30 Oct 2006, Jozsef Kadlecsik wrote:
>
>> On Mon, 30 Oct 2006, Carlos Velasco wrote:
>>
>>> I'm forwarding a mail from Linux Kernel, as it seems a bug in Netfilter
>>> in 2.6.18.
>> The TCP session dies because a NAT device between the communicating
>> parties does not adjust the sequence numbers in the SACK fields.
>>
>> Is there a NATing device, which is not identical with the machine running
>> 2.6.28, between the client and the server?
>
> No, it's not a NATing device. It's a 'smart' box which munges the TCP
> sequence numbers and misses to do so in the SACK fields: the first packet
> from both recordings:
Yes, you are absolutely right. This is not a linux/netfilter issue.
After a little research we saw a Cisco PIX firewall behind the receiver
SMTP server doing ISN randomization.
I contacted Cisco and a bug has been raised for this issue:
Bug number: CSCse14419
http://www.cisco.com/cgi-bin/bugtool/onebug.pl?bugid=CSCse14419
As a workaround, ISN randomization can be disabled for TCP connections,
like this:
===
access-list WAROUNDTCP extended permit tcp any any
class-map WAROUNDTCP
match access-list WAROUNDTCP
policy-map global_policy
class WAROUNDTCP
set connection random-sequence-number disable
===
I thought it's a good thing to post this email for future refence if
this issue comes back again in the future.
Regards,
Carlos Velasco
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2006-11-08 13:17 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-10-29 23:26 [Fwd: Re: Networking messed up, bad checksum, incorrect length] Carlos Velasco
2006-10-30 8:10 ` Jozsef Kadlecsik
2006-10-30 8:29 ` Jozsef Kadlecsik
2006-11-08 13:17 ` Carlos Velasco
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.