TCP connection fails in a asymmetric routing situation

[Florian Westphal <fw@strlen.de>]

https://bugzilla.redhat.com/show_bug.cgi?id=2051413

Gist is:
as of 878aed8db324bec64f3c3f956e64d5ae7375a5de
(" netfilter: nat: force port remap to prevent shadowing well-known
 port"), tcp connections won't get established with asymmetric routing
setups.

Workaround: Block conntrack for  LAN-LAN2 traffic by
iptables  -t raw -A PREROUTING -j CT --notrack
Or: echo 0 > /proc/sys/net/netfilter/nf_conntrack_tcp_loose

I'd guess that is because conntrack picks up the flow on syn-ack rather
than syn, snat check then thinks that source port is < 16384 and dest
port is large, so we do port rewrite but we do it on syn-ack and
connection cannot complete because client and server have different
views of the source ports involved.

Question is on how this can be prevented. I see a few solutions:

1. Change ct->local_origin to "ct->no_srcremap" (or a new status bit)
that indicates that this should not have src remap done, just like we
do for locally generated connections.

2. Add a new "mid-stream" status bit, then bypass the entire -t nat
logic if its set. nf_nat_core would create a null binding for the
flow, this also bypasses the "src remap" code.

3. Simpler version: from tcp conntrack, set the nat-done status bits
if its a mid-stream pickup.

Downside: nat engine (as-is) won't create a null binding, so connection
will not be known to nat engine for masquerade source port clash
detection.

I would go for 2) unless you have a better suggestion/idea.

Thanks!