Can anyone tell wat's going on here?

Can anyone tell wat's going on here?

[Sergio Penkale]

Hi, I have a problem that's driving me crazy:
I have a nat box running Debian Sarge under a 2.6.11.7 kernel. I compiled
this kernel myself, which could be the source of problems.
The nat box itself works great, but masquerading has a strange problem,
some things work and others don't. For example some web pages (like google)
work great, and some others (like hotmail for example), don't work at all,
while in the nat box all of them work. 
I've tried https pages on the clients and they work, so the problem seems to
be at the level of packets, and it only affects the traffic that gets masqueraded.
I have a lot of iptables rules, but it makes no difference using only trivial
masquerading rules. That is: flushing everything, setting all policies to ACCEPT
and adding "-A POSTROUTING -s 10.10.10.0/255.255.255.0 -o ppp0 -j MASQUERADE" in nat.
Usinf SNAT instead of MASQUERADE doesn't make any difference either

The problem also arises when doing ssh: i login to a remote machine and do an ls
of a directory with many files. If i make this from the nat box it works fine,
but if i repeat those same steps from a masqueraded client i only get one or
two listing lines and then it just hangs.
Here's the tethereal output of this: 201.252.166.212 is my ip and xxx.xxx.xxx.xxx
is a remote machine, i've already loged in and i start capturing when i send the
ls command. From the nat box it works ok:

palangana:~# tethereal -i ppp0 '(host xxx.xxx.xxx.xxx) or (port 22)'
Capturing on ppp0
   0.000000 201.252.166.212 -> xxx.xxx.xxx.xxx SSH Encrypted request packet len=48
   0.159113 xxx.xxx.xxx.xxx -> 201.252.166.212 SSH Encrypted response packet len=48
   0.159304 201.252.166.212 -> xxx.xxx.xxx.xxx TCP 4874 > ssh [ACK] Seq=48 Ack=48 Win=2540 Len=0 TSV=59055726 TSER=229209706
   0.210101 xxx.xxx.xxx.xxx -> 201.252.166.212 SSH Encrypted response packet len=144
   0.210259 201.252.166.212 -> xxx.xxx.xxx.xxx TCP 4874 > ssh [ACK] Seq=48 Ack=192 Win=2812 Len=0 TSV=59055777 TSER=229209709
   0.383142 xxx.xxx.xxx.xxx -> 201.252.166.212 SSH Encrypted response packet len=1440
   0.383456 201.252.166.212 -> xxx.xxx.xxx.xxx TCP 4874 > ssh [ACK] Seq=48 Ack=1632 Win=3532 Len=0 TSV=59055950 TSER=229209716
   0.463144 xxx.xxx.xxx.xxx -> 201.252.166.212 SSH Encrypted response packet len=1440
   0.463349 201.252.166.212 -> xxx.xxx.xxx.xxx TCP 4874 > ssh [ACK] Seq=48 Ack=3072 Win=4252 Len=0 TSV=59056030 TSER=229209716
   0.463154 xxx.xxx.xxx.xxx -> 201.252.166.212 SSH Encrypted response packet len=16
   0.463481 201.252.166.212 -> xxx.xxx.xxx.xxx TCP 4874 > ssh [ACK] Seq=48 Ack=3088 Win=4252 Len=0 TSV=59056030 TSER=229209717

but from a client this happens:

palangana:~# tethereal -i ppp0 '(host xxx.xxx.xxx.xxx ) or (port 22)'
Capturing on ppp0
   0.000000 201.252.166.212 -> xxx.xxx.xxx.xxx  SSH Encrypted request packet len=48
   0.074706 xxx.xxx.xxx.xxx  -> 201.252.166.212 SSH Encrypted response packet len=48
   0.104737 xxx.xxx.xxx.xxx  -> 201.252.166.212 SSH Encrypted response packet len=144
   0.105109 201.252.166.212 -> xxx.xxx.xxx.xxx  TCP 1356 > ssh [ACK] Seq=48 Ack=192 Win=64399 Len=0
   0.118746 xxx.xxx.xxx.xxx  -> 201.252.166.212 SSH Encrypted response packet len=144
   0.226971 201.252.166.212 -> xxx.xxx.xxx.xxx  TCP 1356 > ssh [ACK] Seq=48 Ack=336 Win=64255 Len=0
   0.367701 xxx.xxx.xxx.xxx  -> 201.252.166.212 SSH [TCP Previous segment lost] Encrypted response packet len=1292
   0.368144 201.252.166.212 -> xxx.xxx.xxx.xxx  TCP [TCP Dup ACK 6#1] 1356 > ssh [ACK] Seq=48 Ack=336 Win=64255 Len=0 SLE=1796 SRE=3088

I don't know whether my ack on package #6 gets lost, or if i'm missing some packets
sent by xxx.xxx.xxx.xxx.
I could repeat this behavior anytime i want.

My main problem is that i don't know wat the problem is, i don't know if it's 
iptables-related, kernel-related or what. I haven't tried other kernels mainly
because it's a lot of trouble to get the driver of my USB ADSL modem to work
between different kernel versions.
I was using debian woody with a 2.4.19 kernel and everything worked fine,
the problem arised when i upgraded to sarge. The main configuration didn't
change during the upgrade.
Any suggestion on what the problem could be or at least how to trace it
would be much appreciated.

regards,
Sergio.

Re: Can anyone tell wat's going on here?

[Steven M Campbell]

Sergio Penkale wrote:
>
> My main problem is that i don't know wat the problem is, i don't know 
> if it's iptables-related, kernel-related or what. I haven't tried 
> other kernels mainly
> because it's a lot of trouble to get the driver of my USB ADSL modem 
> to work
> between different kernel versions.
> I was using debian woody with a 2.4.19 kernel and everything worked fine,
> the problem arised when i upgraded to sarge. The main configuration 
> didn't
> change during the upgrade.
> Any suggestion on what the problem could be or at least how to trace it
> would be much appreciated.
>

Have you checked 'netstat -in' for interface errors on the ethernet 
connection, I am suspicious that you just have a duplex mismatch between 
the host and client.  Worth a look.

Re: Can anyone tell wat's going on here?

[Sergio Penkale]

On Wed, 11 May 2005, Steven M Campbell wrote:

> Have you checked 'netstat -in' for interface errors on the ethernet 
> connection, I am suspicious that you just have a duplex mismatch between the 
> host and client.  Worth a look.

Thanks for the reply, but the ethernet link seems ok, i'm using squid 
until i can fix this and it works fine.
Besides, if i go back to woody (i made an image of my previous distro) it 
works ok.

This is the output of netstat, the client host is on eth0.

palangana:~# netstat -in
Kernel Interface table
Iface   MTU Met   RX-OK RX-ERR RX-DRP RX-OVR   TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0   1500 0      2649      0      0      0    1809      0      0      0 BMRU
eth1   1500 0      7563      0      0      0    7444      0      0      1 BMU
lo    16436 0       106      0      0      0     106      0      0      0 LRU
nas0   1500 0     94132      0      0      0   74268     10      0     10 BMRU
ppp0   1492 0     91901      0      0      0   72046      0      0      0 MOPRU

Re: Can anyone tell wat's going on here?

[Jason Opperisano]

On Wed, May 11, 2005 at 12:49:39PM -0300, Sergio Penkale wrote:
> Hi, I have a problem that's driving me crazy:
> I have a nat box running Debian Sarge under a 2.6.11.7 kernel. I compiled
> this kernel myself, which could be the source of problems.
> The nat box itself works great, but masquerading has a strange problem,
> some things work and others don't. For example some web pages (like google)
> work great, and some others (like hotmail for example), don't work at all,
> while in the nat box all of them work. 
> I've tried https pages on the clients and they work, so the problem seems to
> be at the level of packets, and it only affects the traffic that gets 
> masqueraded.
> I have a lot of iptables rules, but it makes no difference using only 
> trivial
> masquerading rules. That is: flushing everything, setting all policies to 
> ACCEPT
> and adding "-A POSTROUTING -s 10.10.10.0/255.255.255.0 -o ppp0 -j 
> MASQUERADE" in nat.
> Usinf SNAT instead of MASQUERADE doesn't make any difference either
> 
> The problem also arises when doing ssh: i login to a remote machine and do 
> an ls
> of a directory with many files. If i make this from the nat box it works 
> fine,
> but if i repeat those same steps from a masqueraded client i only get one or
> two listing lines and then it just hangs.
> Here's the tethereal output of this: 201.252.166.212 is my ip and 
> xxx.xxx.xxx.xxx
> is a remote machine, i've already loged in and i start capturing when i 
> send the
> ls command. From the nat box it works ok:

your symptoms scream 'MTU Issue' at me.  try this rule and see if it
fixes it:

  iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
    -j TCPMSS --clamp-mss-to-pmtu

-j

--
"Brian: Seriously, who buys a novelty fire extinguisher?
 Peter: I'll tell you who: someone who cares enough about physical
 comedy to put his entire family into serious danger, that's who."
        --Family Guy

Re: Can anyone tell wat's going on here?

[Taylor, Grant]

> palangana:~# tethereal -i ppp0 '(host xxx.xxx.xxx.xxx) or (port 22)' Capturing on ppp0
>   0.000000 201.252.166.212 -> xxx.xxx.xxx.xxx SSH Encrypted request packet len=48
>   0.159113 xxx.xxx.xxx.xxx -> 201.252.166.212 SSH Encrypted response packet len=48
>   0.159304 201.252.166.212 -> xxx.xxx.xxx.xxx TCP 4874 > ssh [ACK] Seq=48 Ack=48 Win=2540 Len=0 TSV=59055726 TSER=229209706
>   0.210101 xxx.xxx.xxx.xxx -> 201.252.166.212 SSH Encrypted response packet len=144
>   0.210259 201.252.166.212 -> xxx.xxx.xxx.xxx TCP 4874 > ssh [ACK] Seq=48 Ack=192 Win=2812 Len=0 TSV=59055777 TSER=229209709
>   0.383142 xxx.xxx.xxx.xxx -> 201.252.166.212 SSH Encrypted response packet len=1440
>   0.383456 201.252.166.212 -> xxx.xxx.xxx.xxx TCP 4874 > ssh [ACK] Seq=48 Ack=1632 Win=3532 Len=0 TSV=59055950 TSER=229209716
>   0.463144 xxx.xxx.xxx.xxx -> 201.252.166.212 SSH Encrypted response packet len=1440
>   0.463349 201.252.166.212 -> xxx.xxx.xxx.xxx TCP 4874 > ssh [ACK] Seq=48 Ack=3072 Win=4252 Len=0 TSV=59056030 TSER=229209716
>   0.463154 xxx.xxx.xxx.xxx -> 201.252.166.212 SSH Encrypted response packet len=16
>   0.463481 201.252.166.212 -> xxx.xxx.xxx.xxx TCP 4874 > ssh [ACK] Seq=48 Ack=3088 Win=4252 Len=0 TSV=59056030 TSER=229209717
> 
> but from a client this happens:
> 
> palangana:~# tethereal -i ppp0 '(host xxx.xxx.xxx.xxx ) or (port 22)' Capturing on ppp0
>   0.000000 201.252.166.212 -> xxx.xxx.xxx.xxx  SSH Encrypted request packet len=48
>   0.074706 xxx.xxx.xxx.xxx  -> 201.252.166.212 SSH Encrypted response packet len=48
>   0.104737 xxx.xxx.xxx.xxx  -> 201.252.166.212 SSH Encrypted response packet len=144
>   0.105109 201.252.166.212 -> xxx.xxx.xxx.xxx  TCP 1356 > ssh [ACK] Seq=48 Ack=192 Win=64399 Len=0
>   0.118746 xxx.xxx.xxx.xxx  -> 201.252.166.212 SSH Encrypted response packet len=144
>   0.226971 201.252.166.212 -> xxx.xxx.xxx.xxx  TCP 1356 > ssh [ACK] Seq=48 Ack=336 Win=64255 Len=0
>   0.367701 xxx.xxx.xxx.xxx  -> 201.252.166.212 SSH [TCP Previous segment lost] Encrypted response packet len=1292
>   0.368144 201.252.166.212 -> xxx.xxx.xxx.xxx  TCP [TCP Dup ACK 6#1] 1356 > ssh [ACK] Seq=48 Ack=336 Win=64255 Len=0 SLE=1796 SRE=3088

One thing jumps out to me, namely your TCP window size is different when you are working from your client (NATed) system.  I'm not familiar with dealing with MTU issues, but someone else on this list (Jason do you have any comments?) might be able to help you more than I can.

The other thing that comes to mind is that you might be having problem with your USB modem driver.  I personally try to avoid USB (for many different reasons) and thus can not help you with that either.  Sorry.  :(



Grant. . . .

Re: Can anyone tell wat's going on here?

[Sergio Penkale]

On Wed, 11 May 2005, Jason Opperisano wrote:

> your symptoms scream 'MTU Issue' at me.  try this rule and see if it
> fixes it:
>
>  iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
>    -j TCPMSS --clamp-mss-to-pmtu

It worked!!! Thanks!!
You have no idea how much i fought with this.

Thanks, Sergio.