Re: Issue migrating "iptables -m socket --transparent" into nftables

["Nirgal Vourgère" <contact_vgernf@nirgal.com>]

[-- Attachment #1: Type: text/plain, Size: 2765 bytes --]

On Monday, 17 August 2020 21:34:06 CEST Florian Westphal wrote:
> Nirgal Vourgère <contact_vgernf@nirgal.com> wrote:
> > 
> >     ip rule add fwmark 1 lookup haproxy
> >     ip route add local default dev lo table haproxy
> > 
> > My firewall rules have
> > 
> >     iptables -t mangle -A PREROUTING -m socket --transparent -j MARK --set-mark 1
> 
> [..]
> 
> > I tried this nft rule:
> > 
> >     table inet haproxy {
> >         chain prerouting {
> >             type filter hook prerouting priority -150; policy accept;
> >             socket transparent 1 mark set 0x00000001
> >         }
> >     }
> > 
> > It does work, but all traffic is routed to the haproxy socket, including outbound masqueraded connection… I mean when a box on the lan side connects to a foreign https server, the connection is grabbed by haproxy, which is not what I want.
> 
> I don't understand how the iptables rule would not do exactly the same
> thing, there is nothing that checks interface names or addresses.
> 
> Are you sure there is nothing in the iptables rule set that
> makes the socket rule only handle those packets that should be redirected?

I am sure. Yes.

Here's the output of my iptables generated "nft list ruleset", only the fragment regarding the mangle tables generated by iptables and ip6tables:

table ip mangle {
	chain PREROUTING {
		type filter hook prerouting priority -150; policy accept;
		# socket --transparent counter packets 83537 bytes 53363874 meta mark set 0x1 
	}

	chain INPUT {
		type filter hook input priority -150; policy accept;
	}

	chain FORWARD {
		type filter hook forward priority -150; policy accept;
	}

	chain OUTPUT {
		type route hook output priority -150; policy accept;
	}

	chain POSTROUTING {
		type filter hook postrouting priority -150; policy accept;
	}
}
table ip6 mangle {
	chain PREROUTING {
		type filter hook prerouting priority -150; policy accept;
		# socket --transparent counter packets 3 bytes 180 meta mark set 0x1 
	}

	chain INPUT {
		type filter hook input priority -150; policy accept;
	}

	chain FORWARD {
		type filter hook forward priority -150; policy accept;
	}

	chain OUTPUT {
		type route hook output priority -150; policy accept;
	}

	chain POSTROUTING {
		type filter hook postrouting priority -150; policy accept;
	}
}

This works.

No protocol check, no ip check, no port, just a simple brutal "iptables -t mangle -A PREROUTING -m socket --transparent -j MARK --set-mark 1"
Attached are the whole firewall mangle fragments, one works. The other does not.

Maybe there's some magic in the old transparent module, that silently add some conditions?

I've been using that set up on a whole bunch of servers.

[-- Attachment #2: fw_ok.sh --]
[-- Type: text/plain, Size: 1621 bytes --]

#!/bin/bash
ENABLE_HAPROXY=1
HA_RT_TABLE="haproxy"
RT_TABLES=/etc/iproute2/rt_tables
HAPROXY_IPMARK=1 # Id for packets to go to haproxy

ip46tables() {
    # simple function that run rule on both IPv4 and IPv6
    iptables "$@"
    ip6tables "$@"
}

###############################################################################
# Marking packets for haproxy
###############################################################################

ip46tables -t mangle --flush PREROUTING
if [ -n "$ENABLE_HAPROXY" ]
then
        sysctl -q -w net.ipv4.ip_nonlocal_bind=1
        ip46tables -t mangle -A PREROUTING -m socket --transparent -j MARK --set-mark $HAPROXY_IPMARK

        if grep -q $HA_RT_TABLE $RT_TABLES
        then
                for ipversion in 4 6
                do
                        # all packets marked by HAPROXY_IPMARK should be routed using $HA_RT_TABLE
                        ip -$ipversion rule del fwmark $HAPROXY_IPMARK
                        ip -$ipversion rule add fwmark $HAPROXY_IPMARK lookup $HA_RT_TABLE

                        # default for routing table $HA_RT_TABLE is to try local bind
                        # Note that net.ipv4.ip_nonlocal_bind=1
                        ip -$ipversion route flush table $HA_RT_TABLE
                        ip -$ipversion route add local default dev lo table $HA_RT_TABLE
                done
        else
                $LOGGER -p user.crit -- "$RT_TABLES does not have $HA_RT_TABLE entry. Consider running \"echo 100 $HA_RT_TABLE >> $RT_TABLES\". haproxy rules disabled."
        fi
else
        sysctl -q -w net.ipv4.ip_nonlocal_bind=0
fi


[-- Attachment #3: fw_nok.sh --]
[-- Type: text/plain, Size: 1600 bytes --]

#!/bin/bash
ENABLE_HAPROXY=1
HA_RT_TABLE="haproxy"
RT_TABLES=/etc/iproute2/rt_tables
HAPROXY_IPMARK=1 # Id for packets to go to haproxy

###############################################################################
# Marking packets for haproxy
###############################################################################

if [ -n "$ENABLE_HAPROXY" ]
then
        sysctl -q -w net.ipv4.ip_nonlocal_bind=1

        nft create table inet haproxy
        nft -- add chain inet haproxy prerouting \{ type filter hook prerouting priority -150\; \}
        nft add rule inet haproxy prerouting socket transparent 1 meta mark set $HAPROXY_IPMARK
        if grep -q $HA_RT_TABLE $RT_TABLES
        then
                for ipversion in 4 6
                do
                        # all packets marked by HAPROXY_IPMARK should be routed using $HA_RT_TABLE
                        ip -$ipversion rule del fwmark $HAPROXY_IPMARK
                        ip -$ipversion rule add fwmark $HAPROXY_IPMARK lookup $HA_RT_TABLE

                        # default for routing table $HA_RT_TABLE is to try local bind
                        # Note that net.ipv4.ip_nonlocal_bind=1
                        ip -$ipversion route flush table $HA_RT_TABLE
                        ip -$ipversion route add local default dev lo table $HA_RT_TABLE
                done
        else
                $LOGGER -p user.crit -- "$RT_TABLES does not have $HA_RT_TABLE entry. Consider running \"echo 100 $HA_RT_TABLE >> $RT_TABLES\". haproxy rules disabled."
        fi
else
        sysctl -q -w net.ipv4.ip_nonlocal_bind=0
fi