Re: [BUG REPORT] netfilter: DNS/SNAT Issue in Kubernetes Environment

[Florian Westphal <fw@strlen.de>]

Yafang Shao <laoar.shao@gmail.com> wrote:
> After applying commit d8f84a9bc7c4, only one entry remains:
> $ cat /proc/net/nf_conntrack| grep 10.242.249.78
> ipv4     2 udp      17 106 src=10.242.249.78 dst=169.254.1.2
> sport=34616 dport=53 src=127.0.0.1 dst=10.242.249.78 sport=53
> dport=34616 [ASSURED] mark=0 zone=0 use=2

Makes sense to me, thats what would be expected, at least from ct state, no?
(I inderstand that things are not working as expected from DNS pov).

> After the additional custom hack, the entries now show two records:
> $ cat /proc/net/nf_conntrack| grep 10.242.249.78
> ipv4     2 udp      17 27 src=169.254.1.2 dst=10.242.249.78 sport=53
> dport=46858 [UNREPLIED] src=10.242.249.78 dst=169.254.1.2 sport=46858
> dport=53 mark=0 zone=0 use=2
> ipv4     2 udp      17 27 src=10.242.249.78 dst=169.254.1.2
> sport=46858 dport=53 src=127.0.0.1 dst=10.242.249.78 sport=53
> dport=46858 mark=0 zone=0 use=2

That makes no sense to me whatsoever.

The second entry looks correct/as expected:
10.242.249.78 -> 169.254.1.2  46858 -> 53    DNATed to 127.0.0.1:53  10.242.249.78:46858

... so we would expect replies coming from 127.0.0.1:53.

But the other entry makes no sense to me.

src=169.254.1.2   dst=10.242.249.78  sport=53 dport=46858 [UNREPLIED] src=10.242.249.78 dst=169.254.1.2 sport=46858 dport=53 mark=0 zone=0 use=2

This means conntrack saw a packet, not matching any existing entry for this:
169.254.1.2:53 -> 10.242.249.78:46858

... and that makes no sense to me.
The reply should be coming from 127.0.0.1:53.

I suspect stack refuses to send a packet from 127.0.0.1 to foreign/nonlocal address?

As far as conntrack is concerned, the origin 169.254.1.2:53 is a new flow.

We do expect this:
127.0.0.1:53 -> 10.242.249.78:46858, which would be classified as matching response to the
existing entry.

Do you have any load balancing, bridging etc. going on that would result in cloned
packets leaving the system, where one is going out unmodified?

Is route_localnet sysctl enabled? I have never tried such lo stunts myself.