Packets being natted (unwanted) when asymmtric routing

Packets being natted (unwanted) when asymmtric routing

[David Harris]

Hello,

I have the following scenario, any insight into why this is happening
would be great:

HTTP initial SYN comes in eth4 out eth0.
SYN-ACK comes back in eth0 out eth4.
ACK comes in eth4 out eth0.
GET request comes in eth4 out eth0.

This is all fine and good. Then:

The response from the HTTP server comes in eth1 and goes out eth4.

The problem is, my linux box changes the source port from port 80 to
something else and I have no idea why.  This obviously causes the
request to not work

SYN
=-=-=-=
11:16:57.442624 eth4 < 10.175.130.221.3326 > 74.52.32.85.http: S
1975949470:1975949470(0) win 65535 <mss 1380,nop,nop,sackOK> (DF)
11:16:57.442668 eth0 > 66.129.118.229.3326 > 74.52.32.85.http: S
1975949470:1975949470(0) win 65535 <mss 1380,nop,nop,sackOK> (DF)

SYN-ACK
=-=-=-=
11:16:57.579041 eth0 < 74.52.32.85.http > 66.129.118.229.3326: S
344376811:344376811(0) ack 1975949471 win 5840 <mss
1460,nop,nop,sackOK> (DF)
11:16:57.579049 eth4 > 74.52.32.85.http > 10.175.130.221.3326: S
344376811:344376811(0) ack 1975949471 win 5840 <mss
1460,nop,nop,sackOK> (DF)

SYN
=-=-=-=
11:16:57.716492 eth4 < 10.175.130.221.3326 > 74.52.32.85.http: .
1:1(0) ack 1 win 65535 (DF)
11:16:57.716498 eth0 > 66.129.118.229.3326 > 74.52.32.85.http: .
1:1(0) ack 1 win 65535 (DF)

HTTP GET
=-=-=-=
11:16:58.447934 eth4 < 10.175.130.221.3326 > 74.52.32.85.http: P
1:196(195) ack 1 win 65535 (DF)
11:16:58.447948 eth0 > 66.129.118.229.3326 > 74.52.32.85.http: P
1:196(195) ack 1 win 65535 (DF)

HTTP RESPONSE - Here is the problem. The websense is replying on
behalf of the HTTP server with a 302, with the goal of redirecting the
client to a blocked page. It comes in eth1 from the websense, then it
goes out eth4 where the client is, which is good. But the linux box
has changed the source port from 80 to 126. This is the problem.
=-=-=-=
11:16:58.450321 eth1 < 74.52.32.85.http > 10.175.130.221.3326: FP
1:148(147) ack 196 win 1024 [tos 0x10] 11:16:58.450340 eth4 >
74.52.32.85.126 > 10.175.130.221.3326: FP 344376812:344376959(147) ack
1975949666 win 1024 [tos 0x10]

Thanks,
David Harris