Query on nftables DNAT for localhost-to-localhost traffic in IPv6 or without route_localnet

Query on nftables DNAT for localhost-to-localhost traffic in IPv6 or without route_localnet

[Antonio Ojea]

Hi,

We (kubernetes) are currently exploring options for port forwarding
traffic that originates from localhost and is also destined for
localhost, to redirect it to a different destination IP address and
port [1].

We can use the route_localnet sysctl parameter, however, that does not
work for IPv6.

We are trying to avoid solutions that rely on eBPF or userspace
proxies to reduce the maintenance load, and hoping to find a solution
within the kernel's networking stack.

Any guidance or suggestions you could provide would be greatly appreciated.

[1]: https://github.com/kubernetes/kubernetes/issues/132955

Re: Query on nftables DNAT for localhost-to-localhost traffic in IPv6 or without route_localnet

[Florian Westphal]

Antonio Ojea <antonio.ojea.garcia@gmail.com> wrote:
> We (kubernetes) are currently exploring options for port forwarding
> traffic that originates from localhost and is also destined for
> localhost, to redirect it to a different destination IP address and
> port [1].

Don't think its a good idea, has much higher risk of exposing
credentials.  Maybe fixable by placing macsec or ipsec tunnel.

> We can use the route_localnet sysctl parameter, however, that does not
> work for IPv6.

Seems no kernel changes are needed, but its ugly because daddr ::1 has
to be concealed in prerouting to prevent RT6_LOOKUP_F_IFACE flag:

        if (rt6_need_strict(&fl6->daddr) && dev->type != ARPHRD_PIMREG)
                flags |= RT6_LOOKUP_F_IFACE;

... in ip6_route_input_lookup().  This seems to do the trick:

define fakein6 = dead::1ce
table inet test {
        chain nat_pr {
                type nat hook postrouting priority srcnat ; policy accept;
                ct status dnat ct original ip6 saddr ::1 masquerade
        }

        chain nat_out {
                type nat hook output priority dstnat ; policy accept;
                ip6 daddr ::1 tcp dport 12345 dnat to [dead:beef:0:227:300::3]:22
        }

        chain pre {
                type filter hook prerouting priority 0 ; policy accept;
                ct status dnat,snat ct original ip6 saddr ::1 ct original ip6 daddr ::1 ip6 daddr set $fakein6 comment "daddr is ::1 but that forces strict route lookup"
        }

        chain in {
                type filter hook input priority 0 ; policy accept;
                ct status dnat,snat ct original ip6 saddr ::1 ct original ip6 daddr ::1 ip6 daddr set ::1 comment " get rid if fakein6"
        }
}

$ ip -6 addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet6 dead::1ce/128 scope global
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever

$ uname -sr ; ssh -p 12345 ::1 uname -sr
Linux 6.15.8-200.fc42.x86_64
Linux 6.1.0-37-amd64

Re: Query on nftables DNAT for localhost-to-localhost traffic in IPv6 or without route_localnet

[Pablo Neira Ayuso]

Hi Florian,

On Mon, Aug 04, 2025 at 10:25:37AM +0200, Florian Westphal wrote:
> Antonio Ojea <antonio.ojea.garcia@gmail.com> wrote:
> > We (kubernetes) are currently exploring options for port forwarding
> > traffic that originates from localhost and is also destined for
> > localhost, to redirect it to a different destination IP address and
> > port [1].
> 
> Don't think its a good idea, has much higher risk of exposing
> credentials.  Maybe fixable by placing macsec or ipsec tunnel.
> 
> > We can use the route_localnet sysctl parameter, however, that does not
> > work for IPv6.
> 
> Seems no kernel changes are needed, but its ugly because daddr ::1 has
> to be concealed in prerouting to prevent RT6_LOOKUP_F_IFACE flag:
> 
>         if (rt6_need_strict(&fl6->daddr) && dev->type != ARPHRD_PIMREG)
>                 flags |= RT6_LOOKUP_F_IFACE;
>
> ... in ip6_route_input_lookup().

>This seems to do the trick:

To simplify this example below, would it be possible to extend nft_fib
to attach DST_METADATA in prerouting to modify the ip6_route_input_lookup()
behaviour? This is similar to the conntrack template, but for routing.

> define fakein6 = dead::1ce
> table inet test {
>         chain nat_pr {
>                 type nat hook postrouting priority srcnat ; policy accept;
>                 ct status dnat ct original ip6 saddr ::1 masquerade
>         }
> 
>         chain nat_out {
>                 type nat hook output priority dstnat ; policy accept;
>                 ip6 daddr ::1 tcp dport 12345 dnat to [dead:beef:0:227:300::3]:22
>         }
> 
>         chain pre {
>                 type filter hook prerouting priority 0 ; policy accept;
>                 ct status dnat,snat ct original ip6 saddr ::1 ct original ip6 daddr ::1 ip6 daddr set $fakein6 comment "daddr is ::1 but that forces strict route lookup"
>         }
> 
>         chain in {
>                 type filter hook input priority 0 ; policy accept;
>                 ct status dnat,snat ct original ip6 saddr ::1 ct original ip6 daddr ::1 ip6 daddr set ::1 comment " get rid if fakein6"
>         }
> }
> 
> $ ip -6 addr show dev lo
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
>     inet6 dead::1ce/128 scope global
>        valid_lft forever preferred_lft forever
>     inet6 ::1/128 scope host noprefixroute
>        valid_lft forever preferred_lft forever
> 
> $ uname -sr ; ssh -p 12345 ::1 uname -sr
> Linux 6.15.8-200.fc42.x86_64
> Linux 6.1.0-37-amd64
> 

Re: Query on nftables DNAT for localhost-to-localhost traffic in IPv6 or without route_localnet

[Florian Westphal]

Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> >This seems to do the trick:
> 
> To simplify this example below, would it be possible to extend nft_fib
> to attach DST_METADATA in prerouting to modify the ip6_route_input_lookup()
> behaviour? This is similar to the conntrack template, but for routing.

skb_valid_dst() doesn't consider DST_METADATA as a valid dst, afaics the
dst is then discarded and we end up in the same code paths.

But I think we could extend nft_fib to attach a route/dst.

But at this time I don't want to spend time on enabling such hacks
(lo-to-remote-dst-nat) unless there is a good use case for it.

Re: Query on nftables DNAT for localhost-to-localhost traffic in IPv6 or without route_localnet

[Pablo Neira Ayuso]

On Tue, Aug 12, 2025 at 01:17:43PM +0200, Florian Westphal wrote:
> Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > >This seems to do the trick:
> > 
> > To simplify this example below, would it be possible to extend nft_fib
> > to attach DST_METADATA in prerouting to modify the ip6_route_input_lookup()
> > behaviour? This is similar to the conntrack template, but for routing.
> 
> skb_valid_dst() doesn't consider DST_METADATA as a valid dst, afaics the
> dst is then discarded and we end up in the same code paths.
>
> But I think we could extend nft_fib to attach a route/dst.

Then ip6_route_input_lookup() needs to be updated, and it would be
good if there is a flag somewhere to specify that the existing route
is intentional to skip this:

        skb_dst_drop(skb);
        skb_dst_set_noref(skb, ip6_route_input_lookup(net, skb->dev,
                                                      &fl6, skb, flags));

> But at this time I don't want to spend time on enabling such hacks
> (lo-to-remote-dst-nat) unless there is a good use case for it.

I am not familiar with this use-case.

Re: Query on nftables DNAT for localhost-to-localhost traffic in IPv6 or without route_localnet

[Antonio Ojea]

>
> > But at this time I don't want to spend time on enabling such hacks
> > (lo-to-remote-dst-nat) unless there is a good use case for it.
>
> I am not familiar with this use-case.

only one use case in kubernetes and one we were trying to avoid to the
point we removed the functionality in the new implementation of
kube-proxy with nftables, but since it came back recently I just
considered it worth to ask.
I will try to expose the use case simplifying some of the technical
details, container runtimes need to pull images from registries (think
of it as a webserver to serve container images).
For efficiency and to reduce network traffic, it's common to run a
local registry mirror or cache. Localhost is a simple and reliable
target for the container runtime that pulls the images, and by
convention they treat localhost registries as "secure" by default,
meaning they don't require a valid TLS certificate setup.
Kubernetes has a networking option to forward a specific port in a
node to a virtual ip with a set of endpoints, and people were using
this functionality to implement the local cache mechanism ...
So personally I do not feel this is a good use case for netfilter to
implement something new based on it