VPN nftables

VPN nftables

[Martin Brampton]

On a Debian 12 VPS with openvpn installed, I am having difficulty 
finding an nftable ruleset that works. The following is the current 
ruleset, excluding the set definitions. I can connect to the VPN, but 
then cannot access any services.

I'm aware there are overlapping rules, but that is because I'm 
attempting to use a standard ruleset across all servers, with additions 
according to the services needed. Any thoughts?

table inet firewall {
     chain input {
         type filter hook input priority filter; policy accept;
         ct state { established, related } accept
         iifname "lo" accept
         icmpv6 type { destination-unreachable, packet-too-big, 
time-exceeded, parameter-problem, echo-request, mld-listener-query, 
nd-router-solicit, nd-router-advert, nd-neighbor-solicit, 
nd-neighbor-advert } accept
         ip protocol icmp icmp type echo-request ip saddr @servers_ipv4 
accept
         ip protocol icmp icmp type echo-request ip saddr 127.0.0.1 accept
         ip saddr @spamhaus_blacklist drop
         ip saddr @blacklist_v4 drop
         tcp dport 443 accept
         iifname "tun0" accept
         tcp dport 22 ip saddr @always_ipv4 accept
         tcp dport 22 ip saddr @privileged_ipv4 accept
         tcp dport 22 ip saddr @tunnelin_ipv4 accept
         tcp dport 22 ip saddr @security_v4 accept
         tcp dport 22 ip saddr 193.107.20.105 accept
         tcp dport 22 ip6 saddr @always_ipv6 accept
         tcp dport 22 ip6 saddr @privileged_ipv6 accept
         tcp dport 22 ip6 saddr @tunnelin_ipv6 accept
         tcp dport 22 ip6 saddr @security_v6 accept
         tcp dport 22 log prefix "Denied SSH: " drop
         udp dport 60000-61000 ip saddr @always_ipv4 accept
         udp dport 60000-61000 ip saddr @privileged_ipv4 accept
         udp dport 60000-61000 ip6 saddr @always_ipv6 accept
         udp dport 60000-61000 ip6 saddr @privileged_ipv6 accept
         udp dport 60000-61000 log prefix "Denied MOSH: " drop
         ip saddr @munin_host_v4 tcp dport 4949 accept
         ip6 saddr @munin_host_v6 tcp dport 4949 accept
         tcp dport 3306 ip saddr 127.0.0.1 accept
         log prefix "nftables input dropped: " flags all counter packets 
86 bytes 9203 drop
     }

     chain forward {
         type filter hook forward priority filter; policy drop;
         ct state { established, related } accept
         iifname "tun0" oifname "eth0" accept
         iifname "eth0" oifname "tun0" ct state new accept
         log prefix "nftables forward dropped: " flags all counter 
packets 0 bytes 0 drop
     }

     chain output {
         type filter hook output priority filter; policy drop;
         ct state { established, related } accept
         ip protocol icmp icmp type echo-request ip daddr @servers_ipv4 
accept
         ip protocol icmp icmp type echo-request ip daddr 127.0.0.1 accept
         icmpv6 type { destination-unreachable, packet-too-big, 
time-exceeded, parameter-problem, echo-request, mld-listener-query, 
nd-router-solicit, nd-router-advert, nd-neighbor-solicit, 
nd-neighbor-advert } accept
         ip protocol { tcp, udp } th dport 53 accept
         tcp dport 123 accept
         tcp dport { 80, 443 } accept
         ip daddr @postmark_v4 tcp dport 587 accept
         ip daddr @postmarklists_v4 tcp dport 587 accept
         ip daddr @smtp2go_v4 tcp dport 587 accept
         ip6 daddr @postmark_v6 tcp dport 587 accept
         ip6 daddr @postmarklists_v6 tcp dport 587 accept
         ip6 daddr @smtp2go_v6 tcp dport 587 accept
         ip daddr @vault_v4 tcp dport 22 accept
         ip6 daddr @vault_v6 tcp dport 22 accept
         ip daddr 127.0.0.1 tcp dport 3306 accept
         ip protocol tcp accept
         ip6 nexthdr tcp accept
         ip protocol udp accept
         ip6 nexthdr udp accept
     }

     chain prerouting {
         type nat hook prerouting priority dstnat; policy accept;
     }

     chain postrouting {
         type nat hook postrouting priority srcnat; policy accept;
     }
}

Re: VPN nftables

[Daniel]

In the output set policy to accept or add new to ct state

Le 10/10/2024 à 18:18, Martin Brampton a écrit :
> On a Debian 12 VPS with openvpn installed, I am having difficulty
> finding an nftable ruleset that works. The following is the current
> ruleset, excluding the set definitions. I can connect to the VPN, but
> then cannot access any services.
>
> I'm aware there are overlapping rules, but that is because I'm
> attempting to use a standard ruleset across all servers, with additions
> according to the services needed. Any thoughts?
>
> table inet firewall {
>       chain input {
>           type filter hook input priority filter; policy accept;
>           ct state { established, related } accept
>           iifname "lo" accept
>           icmpv6 type { destination-unreachable, packet-too-big,
> time-exceeded, parameter-problem, echo-request, mld-listener-query,
> nd-router-solicit, nd-router-advert, nd-neighbor-solicit,
> nd-neighbor-advert } accept
>           ip protocol icmp icmp type echo-request ip saddr @servers_ipv4
> accept
>           ip protocol icmp icmp type echo-request ip saddr 127.0.0.1 accept
>           ip saddr @spamhaus_blacklist drop
>           ip saddr @blacklist_v4 drop
>           tcp dport 443 accept
>           iifname "tun0" accept
>           tcp dport 22 ip saddr @always_ipv4 accept
>           tcp dport 22 ip saddr @privileged_ipv4 accept
>           tcp dport 22 ip saddr @tunnelin_ipv4 accept
>           tcp dport 22 ip saddr @security_v4 accept
>           tcp dport 22 ip saddr 193.107.20.105 accept
>           tcp dport 22 ip6 saddr @always_ipv6 accept
>           tcp dport 22 ip6 saddr @privileged_ipv6 accept
>           tcp dport 22 ip6 saddr @tunnelin_ipv6 accept
>           tcp dport 22 ip6 saddr @security_v6 accept
>           tcp dport 22 log prefix "Denied SSH: " drop
>           udp dport 60000-61000 ip saddr @always_ipv4 accept
>           udp dport 60000-61000 ip saddr @privileged_ipv4 accept
>           udp dport 60000-61000 ip6 saddr @always_ipv6 accept
>           udp dport 60000-61000 ip6 saddr @privileged_ipv6 accept
>           udp dport 60000-61000 log prefix "Denied MOSH: " drop
>           ip saddr @munin_host_v4 tcp dport 4949 accept
>           ip6 saddr @munin_host_v6 tcp dport 4949 accept
>           tcp dport 3306 ip saddr 127.0.0.1 accept
>           log prefix "nftables input dropped: " flags all counter packets
> 86 bytes 9203 drop
>       }
>
>       chain forward {
>           type filter hook forward priority filter; policy drop;
>           ct state { established, related } accept
>           iifname "tun0" oifname "eth0" accept
>           iifname "eth0" oifname "tun0" ct state new accept
>           log prefix "nftables forward dropped: " flags all counter
> packets 0 bytes 0 drop
>       }
>
>       chain output {
>           type filter hook output priority filter; policy drop;
>           ct state { established, related } accept
>           ip protocol icmp icmp type echo-request ip daddr @servers_ipv4
> accept
>           ip protocol icmp icmp type echo-request ip daddr 127.0.0.1 accept
>           icmpv6 type { destination-unreachable, packet-too-big,
> time-exceeded, parameter-problem, echo-request, mld-listener-query,
> nd-router-solicit, nd-router-advert, nd-neighbor-solicit,
> nd-neighbor-advert } accept
>           ip protocol { tcp, udp } th dport 53 accept
>           tcp dport 123 accept
>           tcp dport { 80, 443 } accept
>           ip daddr @postmark_v4 tcp dport 587 accept
>           ip daddr @postmarklists_v4 tcp dport 587 accept
>           ip daddr @smtp2go_v4 tcp dport 587 accept
>           ip6 daddr @postmark_v6 tcp dport 587 accept
>           ip6 daddr @postmarklists_v6 tcp dport 587 accept
>           ip6 daddr @smtp2go_v6 tcp dport 587 accept
>           ip daddr @vault_v4 tcp dport 22 accept
>           ip6 daddr @vault_v6 tcp dport 22 accept
>           ip daddr 127.0.0.1 tcp dport 3306 accept
>           ip protocol tcp accept
>           ip6 nexthdr tcp accept
>           ip protocol udp accept
>           ip6 nexthdr udp accept
>       }
>
>       chain prerouting {
>           type nat hook prerouting priority dstnat; policy accept;
>       }
>
>       chain postrouting {
>           type nat hook postrouting priority srcnat; policy accept;
>       }
> }

-- 
Daniel Huhardeaux
+33.368460088@tootai.net	      sip:820@sip.tootai.net
      					    tootaiNET

Re: VPN nftables

[Martin Brampton]

Thanks. Unfortunately that doesn't help.

In fact, it seems that installing and configuring nftables destroys openvpn.

 > In the output set policy to accept or add new to ct state

Re: VPN nftables

[Kerin Millar]

On Thu, 10 Oct 2024, at 8:25 PM, Martin Brampton wrote:
> Thanks. Unfortunately that doesn't help.
>
> In fact, it seems that installing and configuring nftables destroys openvpn.

In your initial post, you describe your problem as not being able to access any services. That does not amount to a concrete description of a test case. Upon your peer being connected, what exactly have you tried where the outcome is not as expected? What were the diagnostic messages reported, if any? If you were to focus on - and detail - a particular scenario, it could make it easier for other readers to help you diagnose the underlying problem.

Additionally, you may find it beneficial to define a rule that enables the nftrace mode for packets of interest. That is, those that you suspect are being erroneously dropped. I usually do so in a prerouting hook. For example:

chain prerouting {
    type filter hook prerouting priority raw; policy accept;

    # Trace any TCP packet arriving that is destined for port 22 at any host
    tcp dport 22 meta nftrace set 1
}

You may then run "nft monitor trace" and observe the trajectory of the packet(s) through your ruleset in real time.

-- 
Kerin Millar

Re: VPN nftables

[Martin Brampton]

Thanks, I'm sure that makes sense. But I'm on the point of abandoning 
the use of nftables with openvpn. I've already spent several days on 
this. Other servers (about 15 in all) are running with nftables, and I 
would prefer total consistency, but not at any price.

When I say can't access any services, I mean literally that. I can 
create a new server, install openvpn, connect to it and use services 
like ssh, mosh, https, imaps... And I can do that with an iptables firewall.

But as soon as I add nftables (removing iptables) and connect to the 
server as a VPN, mosh sessions stop, web access ceases, mail access 
ceases. Given that the ruleset opens all output ports, on the face of 
it, that should not happen.

And from that point I cannot find any way back to a working VPN server, 
which makes testing harder, and is disastrous for a live VPN server.


On 10/10/2024 20:56, Kerin Millar wrote:
> 
> In your initial post, you describe your problem as not being able to access any services. That does not amount to a concrete description of a test case. Upon your peer being connected, what exactly have you tried where the outcome is not as expected? What were the diagnostic messages reported, if any? If you were to focus on - and detail - a particular scenario, it could make it easier for other readers to help you diagnose the underlying problem.
> 
> Additionally, you may find it beneficial to define a rule that enables the nftrace mode for packets of interest. That is, those that you suspect are being erroneously dropped. I usually do so in a prerouting hook. For example:
> 
> chain prerouting {
>      type filter hook prerouting priority raw; policy accept;
> 
>      # Trace any TCP packet arriving that is destined for port 22 at any host
>      tcp dport 22 meta nftrace set 1
> }
> 
> You may then run "nft monitor trace" and observe the trajectory of the packet(s) through your ruleset in real time.
> 

Re: VPN nftables

[Reindl Harald]

Am 11.10.24 um 08:40 schrieb Martin Brampton:
> When I say can't access any services, I mean literally that. I can
> create a new server, install openvpn, connect to it and use services
> like ssh, mosh, https, imaps... And I can do that with an iptables firewall.
> 
> But as soon as I add nftables (removing iptables) and connect to the
> server as a VPN, mosh sessions stop, web access ceases, mail access
> ceases. Given that the ruleset opens all output ports, on the face of
> it, that should not happen.
did you also remove all iptables rules?

sadly you can have both at the same time in kernel with all sort of 
funny results

Re: VPN nftables

[Martin Brampton]

Yes, I actually uninstalled iptables.


On 11/10/2024 08:34, Reindl Harald wrote:
> 
> did you also remove all iptables rules?
> 
> sadly you can have both at the same time in kernel with all sort of 
> funny results
> 
> 
> 

Re: VPN nftables

[Kerin Millar]

On Fri, 11 Oct 2024, at 7:40 AM, Martin Brampton wrote:
> Thanks, I'm sure that makes sense. But I'm on the point of abandoning 
> the use of nftables with openvpn. I've already spent several days on 
> this. Other servers (about 15 in all) are running with nftables, and I 
> would prefer total consistency, but not at any price.
>
> When I say can't access any services, I mean literally that. I can 
> create a new server, install openvpn, connect to it and use services 
> like ssh, mosh, https, imaps... And I can do that with an iptables firewall.
>
> But as soon as I add nftables (removing iptables) and connect to the 
> server as a VPN, mosh sessions stop, web access ceases, mail access 
> ceases. Given that the ruleset opens all output ports, on the face of 
> it, that should not happen.
>
> And from that point I cannot find any way back to a working VPN server, 
> which makes testing harder, and is disastrous for a live VPN server.

I take it that there is no out-of-band management system in place for the affected server? If so, you might consider whether the kernel is panicking. Given the use of Debian 12, it would be surprising but by no means impossible.

You can test rulesets more safely by using tmux or GNU screen and running "nft -f /path/to/ruleset; sleep 10; nft flush ruleset". Assuming that you are able to access the server after the ruleset has been flushed, there would be no practical impediment to tracing as a debugging method.

-- 
Kerin Millar