Packets not traversing postrouting chain

[Bradley Hook <bhook@kansasdeaf.gov>]

Hello.

I've got an Ubuntu Server 24.04 LTS system that is doing some NAT on
our network. We had an ISP fail and I'm trying to reroute traffic over
a different link, but some of the SNAT rules are not being applied.
The traces seem to indicate that the packets that are causing problems
aren't even traversing the postrouting chain.

On the LAN side, we have several VLANs. Some are completely private
addressing (10.x.x.x), and others have public addresses from ISP-B.
I'll use 192.168.x.x addresses as a stand-in for those public
addresses in my examples.

On the WAN side, we have two separate ISPs. ISP-B has been our
primary, and ISP-A has been our backup. I'll use 172.16.x.x addresses
for ISP-A.

Due to various off-site problems, ISP-B is down. We changed routing
rules and added some NAT rules to try and get the traffic working
while ISP-B is down.

Private addresses and some public addresses are NATing just fine. But,
some of the public addresses don't appear to be traversing the
postrouting chain at all.

I'll use 8.8.8.8 as a placeholder for Internet targets we are trying to reach.

We've gutted the ruleset to try and figure out what is going on, so it
is very simple and provided at the end of this email.

The issue we are seeing is that packets from 192.168.122.252 to
8.8.8.8 are not traversing the postrouting chain at all. We can see
the packets leaving the interface without NAT applied. We can see the
packets hitting the forward chain with the trace. Other traffic from
other subnets are being masqueraded just fine. We just aren't seeing
the packets from 192.168.122.x/24 hit any postrouting rules at all.

Any suggestions as to what I might be missing here or other
troubleshooting steps to take?

table inet filter { # handle 12
        chain input { # handle 1
                type filter hook input priority filter; policy accept;
        }

        chain forward { # handle 2
                type filter hook forward priority filter; policy accept;
                ip saddr 192.168.122.0/24 ip daddr 8.8.8.8 meta
nftrace set 1 # handle 15
        }

        chain output { # handle 3
                type filter hook output priority filter; policy accept;
        }

        chain prerouting { # handle 4
                type nat hook prerouting priority dstnat; policy accept;
                ip daddr 10.10.10.10 meta l4proto { tcp, udp } th
dport 53 dnat ip to 10.222.128.10 # handle 9
        }

        chain postrouting { # handle 5
                type nat hook postrouting priority srcnat; policy accept;
                ip saddr 192.168.122.0/24 ip daddr 8.8.8.8 meta
nftrace set 1 # handle 16
                oifname "mvISPa" ip saddr 192.168.122.0/24 snat ip to
172.16.169.201 # handle 13
                oifname { "mvISPa", "mvISPb", "vrrpISPa4", "vrrpISPb4"
} ip saddr 10.0.0.0/8 masquerade # handle 10
                oifname { "mvISPa", "vrrpISPa4" } ip saddr
192.168.120.0/21 masquerade # handle 11
        }
}

Respectfully,
~Bradley Hook, J.D.
Network Administrator
Google Certified Project Manager
Kansas State Schools for the Deaf and the Blind
bhook@kansasdeaf.gov
Mobile: 913-275-9982

-- 
*Kansas State Schools for the Deaf and the Blind Confidentiality Notice**:* 
 
The information contained in this e-mail transmission is confidential and 
legally protected.  It is intended for the sole use of the individual(s) 
entity named in the message header.  If you are not the intended recipient, 
you are hereby notified that any dissemination or copying of this 
information is strictly prohibited.  If you received this message in error, 
please notify the sender of the error and delete this message and any 
attachments.