nftables: masquerading not applied consistently

nftables: masquerading not applied consistently

[Thilo-Alexander Ginkel]

Hello everyone,

I have a somewhat complex VPN setup involving a Ubuntu-based VPN
gateway ("vpn-gw") that terminates multiple VPNs based on WireGuard
and OpenVPN. Also involved is a core router ("gw01") that performs
routing among different VLANs comprising the LAN. One of the OpenVPN
interfaces ("tun252")  requires masquerading, which I implemented
through the following nftables config snippet:

table ip nat {
    chain postrouting {
        type nat hook postrouting priority 100; policy accept

        [...]
        oifname "tun252" counter masquerade
    }
}

This works fine if traffic originates on the LAN. Things start acting
weird (i.e., no masquerading is applied when the traffic is forwarded
to tun252) when the traffic originates on one of the WireGuard
interfaces. The path of the packets in this case is: vpn-gw[wireguard]
-> vpn-gw[eth0.250] -> gw01[eth0.250] ->  gw01[eth0.252] ->
vpn-gw[eth0.252] -> vpn-gw[tun252]

In contrast, traffic that takes the following course is masqueraded
correctly: gw01[eth0.10] ->  gw01[eth0.252] -> vpn-gw[eth0.252] ->
vpn-gw[tun252]

Any idea what may be causing this and how to fix this?

Thanks,
Thilo

Re: nftables: masquerading not applied consistently

[Florian Westphal]

Thilo-Alexander Ginkel <thilo@ginkel.com> wrote:
> I have a somewhat complex VPN setup involving a Ubuntu-based VPN
> gateway ("vpn-gw") that terminates multiple VPNs based on WireGuard
> and OpenVPN. Also involved is a core router ("gw01") that performs
> routing among different VLANs comprising the LAN. One of the OpenVPN
> interfaces ("tun252")  requires masquerading, which I implemented
> through the following nftables config snippet:
> 
> table ip nat {
>     chain postrouting {
>         type nat hook postrouting priority 100; policy accept
> 
>         [...]
>         oifname "tun252" counter masquerade
>     }
> }
> 
> This works fine if traffic originates on the LAN. Things start acting
> weird (i.e., no masquerading is applied when the traffic is forwarded
> to tun252) when the traffic originates on one of the WireGuard
> interfaces. The path of the packets in this case is: vpn-gw[wireguard]
> -> vpn-gw[eth0.250] -> gw01[eth0.250] ->  gw01[eth0.252] ->
> vpn-gw[eth0.252] -> vpn-gw[tun252]

I.e., traffic passes vpn-gw twice?  If so, that won't work.

On first pass (wireguard decap), packet x is picked up by
connection tracking and then has NAT applied to it.
As no matching NAT rule exists, the connection gets a 'do nothing'
binding.    When packet comes back, it will match the existing
'do nothing' connection and the nat table is never consulted.

If thats the case, you will either need to turn off contrack for packets
coming from wireguard (use 'notrack' keyword in raw table) or, if you
need nat for this part as well, place the two rounds the packet takes in
different connection tracking zones, so that the packet coming back
from gw01/vlan252 is seen as a new flow, rather than an old packet
matching a known connection.

Re: nftables: masquerading not applied consistently

[Thilo-Alexander Ginkel]

On Wed, Jul 8, 2020 at 1:10 AM Florian Westphal <fw@strlen.de> wrote:
> I.e., traffic passes vpn-gw twice?

Yes.

> If so, that won't work.
>
> On first pass (wireguard decap), packet x is picked up by
> connection tracking and then has NAT applied to it.
> As no matching NAT rule exists, the connection gets a 'do nothing'
> binding.    When packet comes back, it will match the existing
> 'do nothing' connection and the nat table is never consulted.
>
> If thats the case, you will either need to turn off contrack for packets
> coming from wireguard (use 'notrack' keyword in raw table) or, if you
> need nat for this part as well, place the two rounds the packet takes in
> different connection tracking zones, so that the packet coming back
> from gw01/vlan252 is seen as a new flow, rather than an old packet
> matching a known connection.

As the WireGuard interface in question doesn not need NAT, I'll try
the "notrack" approach.

Thanks for your speedy help!

Regards,
Thilo

Re: nftables: masquerading not applied consistently

[Reindl Harald]

Am 08.07.20 um 09:28 schrieb Thilo-Alexander Ginkel:
> On Wed, Jul 8, 2020 at 1:10 AM Florian Westphal <fw@strlen.de> wrote:
>> On first pass (wireguard decap), packet x is picked up by
>> connection tracking and then has NAT applied to it.
>> As no matching NAT rule exists, the connection gets a 'do nothing'
>> binding.    When packet comes back, it will match the existing
>> 'do nothing' connection and the nat table is never consulted.
>>
>> If thats the case, you will either need to turn off contrack for packets
>> coming from wireguard (use 'notrack' keyword in raw table) or, if you
>> need nat for this part as well, place the two rounds the packet takes in
>> different connection tracking zones, so that the packet coming back
>> from gw01/vlan252 is seen as a new flow, rather than an old packet
>> matching a known connection.
> 
> As the WireGuard interface in question doesn not need NAT, I'll try
> the "notrack" approach.

what about limit your NAT rules to the interface where you need it? on
our nat router we have 3 interfaces

* lan
* wan
* vpn

NAT is strictly limited to "wan" and so any wireguard related traffic
would never hit as well as lan-lan forwarding

Chain PREROUTING (policy ACCEPT 15 packets, 2524 bytes)
num   pkts bytes target     prot opt in     out     source
 destination
1       14  2464 NETMAP     all  --  *      *       0.0.0.0/0
 172.17.0.0/24        to:172.16.0.0/24

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source
 destination
1        0     0 NETMAP     all  --  *      wan     172.16.0.0/24
 0.0.0.0/0            to:172.17.0.0/24

Re: nftables: masquerading not applied consistently

[Thilo-Alexander Ginkel]

On Wed, Jul 8, 2020 at 12:32 PM Reindl Harald <h.reindl@thelounge.net> wrote:
> what about limit your NAT rules to the interface where you need it? on
> our nat router we have 3 interfaces
>
> * lan
> * wan
> * vpn
>
> NAT is strictly limited to "wan" and so any wireguard related traffic
> would never hit as well as lan-lan forwarding

That's what I am IMHO already doing. Only traffic routed to tun252 is
NATed. While WAN traffic also needs NATing, this is a different story
in my setup as it isn't handled vpn-gw.

Regards,
Thilo