Re: TCP connection fails in a asymmetric routing situation

["H.Janssen" <hmmsjan@kpnplanet.nl>]

Dear Mr. Westphal,

The problem seems to be solved with the patch below in kernel 
5.16.12-200.fc35.x86_64. Few lines offset during patch, but it patches 
and compiles.

No idea about possible side effects, but ports are no longer manipulated..

contrack -E --dst 192.168.10.12

     [NEW] tcp      6 300 ESTABLISHED src=192.168.1.47 dst=192.168.10.12 
sport=80 dport=52044 [UNREPLIED] src=192.168.10.12 dst=192.168.1.47 
sport=52044 dport=80
  [UPDATE] tcp      6 120 FIN_WAIT src=192.168.1.47 dst=192.168.10.12 
sport=80 dport=52044 [UNREPLIED] src=192.168.10.12 dst=192.168.1.47 
sport=52044 dport=80
[DESTROY] tcp      6 FIN_WAIT src=192.168.1.47 dst=192.168.10.12 
sport=80 dport=52040 [UNREPLIED] src=192.168.10.12 dst=192.168.1.47 
sport=52040 dport=80
[DESTROY] tcp      6 FIN_WAIT src=192.168.1.47 dst=192.168.10.12 
sport=80 dport=52042 [UNREPLIED] src=192.168.10.12 dst=192.168.1.47 
sport=52042 dport=80

Thanks and kind regards





On 3/2/22 11:59, Florian Westphal wrote:
> Florian Westphal <fw@strlen.de> wrote:
>> 1. Change ct->local_origin to "ct->no_srcremap" (or a new status bit)
>> that indicates that this should not have src remap done, just like we
>> do for locally generated connections.
>>
>> 2. Add a new "mid-stream" status bit, then bypass the entire -t nat
>> logic if its set. nf_nat_core would create a null binding for the
>> flow, this also bypasses the "src remap" code.
>>
>> 3. Simpler version: from tcp conntrack, set the nat-done status bits
>> if its a mid-stream pickup.
>>
>> Downside: nat engine (as-is) won't create a null binding, so connection
>> will not be known to nat engine for masquerade source port clash
>> detection.
>>
>> I would go for 2) unless you have a better suggestion/idea.
> Something like this:
>
> diff --git a/include/uapi/linux/netfilter/nf_conntrack_common.h b/include/uapi/linux/netfilter/nf_conntrack_common.h
> --- a/include/uapi/linux/netfilter/nf_conntrack_common.h
> +++ b/include/uapi/linux/netfilter/nf_conntrack_common.h
> @@ -118,15 +118,19 @@ enum ip_conntrack_status {
>   	IPS_HW_OFFLOAD_BIT = 15,
>   	IPS_HW_OFFLOAD = (1 << IPS_HW_OFFLOAD_BIT),
>   
> +	/* Connection started mid-transfer, origin/reply might be inversed */
> +	IPS_MID_STREAM_BIT = 16,
> +	IPS_MID_STREAM = (1 << IPS_MID_STREAM_BIT),
> +
>   	/* Be careful here, modifying these bits can make things messy,
>   	 * so don't let users modify them directly.
>   	 */
>   	IPS_UNCHANGEABLE_MASK = (IPS_NAT_DONE_MASK | IPS_NAT_MASK |
>   				 IPS_EXPECTED | IPS_CONFIRMED | IPS_DYING |
>   				 IPS_SEQ_ADJUST | IPS_TEMPLATE | IPS_UNTRACKED |
> -				 IPS_OFFLOAD | IPS_HW_OFFLOAD),
> +				 IPS_OFFLOAD | IPS_HW_OFFLOAD | IPS_MID_STREAM_BIT),
>   
> -	__IPS_MAX_BIT = 16,
> +	__IPS_MAX_BIT = 17,
>   };
>   
>   /* Connection tracking event types */
> diff --git a/net/netfilter/nf_conntrack_proto_tcp.c b/net/netfilter/nf_conntrack_proto_tcp.c
> --- a/net/netfilter/nf_conntrack_proto_tcp.c
> +++ b/net/netfilter/nf_conntrack_proto_tcp.c
> @@ -537,6 +537,8 @@ static bool tcp_in_window(struct nf_conn *ct,
>   			 * its history is lost for us.
>   			 * Let's try to use the data from the packet.
>   			 */
> +			set_bit(IPS_MID_STREAM_BIT, &ct->status);
> +
>   			sender->td_end = end;
>   			swin = win << sender->td_scale;
>   			sender->td_maxwin = (swin == 0 ? 1 : swin);
> @@ -816,6 +818,8 @@ static noinline bool tcp_new(struct nf_conn *ct, const struct sk_buff *skb,
>   		 * its history is lost for us.
>   		 * Let's try to use the data from the packet.
>   		 */
> +		set_bit(IPS_MID_STREAM_BIT, &ct->status);
> +
>   		ct->proto.tcp.seen[0].td_end =
>   			segment_seq_plus_len(ntohl(th->seq), skb->len,
>   					     dataoff, th);
> diff --git a/net/netfilter/nf_nat_core.c b/net/netfilter/nf_nat_core.c
> --- a/net/netfilter/nf_nat_core.c
> +++ b/net/netfilter/nf_nat_core.c
> @@ -545,6 +545,12 @@ get_unique_tuple(struct nf_conntrack_tuple *tuple,
>   
>   	zone = nf_ct_zone(ct);
>   
> +	if (unlikely(test_bit(IPS_MID_STREAM_BIT, &ct->status))) {
> +		/* Can't NAT if connection was picked up mid-stream (e.g. tcp) */
> +		*tuple = *orig_tuple;
> +		return;
> +	}
> +
>   	if (maniptype == NF_NAT_MANIP_SRC &&
>   	    !random_port &&
>   	    !ct->local_origin)
> @@ -781,7 +787,7 @@ nf_nat_inet_fn(void *priv, struct sk_buff *skb,
>   			unsigned int ret;
>   			int i;
>   
> -			if (!e)
> +			if (!e || unlikely(test_bit(IPS_MID_STREAM_BIT, &ct->status)))
>   				goto null_bind;
>   
>   			for (i = 0; i < e->num_hook_entries; i++) {

-- 
H.Janssen
Lekerwaard 36
1824HC Alkmaar
06-58971047