Re: nftables / DHCP / NAT

Re: nftables / DHCP / NAT

[Pablo Neira Ayuso]

On Fri, Oct 27, 2023 at 06:32:45PM +0200, Volodymyr Litovka wrote:
> The question - what I'm doing wrong?

Description, ruleset and topology look a bit convoluted :-)

To start with:

        iifname "inspan" ...

is not really required, because you chain is already hooked at
"inspan" device see your chain declaration:

table netdev inspan {
    chain rewrit {
        # Drop everything except Radius Accounting and DHCP packets
        type filter hook ingress device "inspan" priority filter; policy drop;

Then, to forward packets to some other box from the 'netdev' family,
use the 'fwd' statement:

        udp dport 67 udp dport set 10067 counter fwd to 100.64.0.66 device "eth0"

This rule above is mangling your UDP destination port from 67 to
10067, then it send the packet to 100.64.0.66 and device "eth0". The
destination MAC address is updated by the neighbour layer so you do
not have to bother with "ether daddr set ...".

Re: nftables / DHCP / NAT

[Volodymyr Litovka]

Hi Pablo,

On 10/30/23 09:41, Pablo Neira Ayuso wrote:
> iifname "inspan" ...
>
> is not really required, because you chain is already hooked at
> "inspan" device see your chain declaration:
thanks for that.

> Then, to forward packets to some other box from the 'netdev' family,
> use the 'fwd' statement:
>
>          udp dport 67 udp dport set 10067 counter fwd to 100.64.0.66 device "eth0"
>
> This rule above is mangling your UDP destination port from 67 to
> 10067, then it send the packet to 100.64.0.66 and device "eth0". The
> destination MAC address is updated by the neighbour layer so you do
> not have to bother with "ether daddr set ...".

the basic idea of this construction is to use later load balancing 
(https://wiki.nftables.org/wiki-nftables/index.php/Load_balancing) 
between multiple destinations, in the section

table ip todos {
     chain enat {
         type nat hook prerouting priority dstnat;
         udp dport 10067 counter dnat to 100.64.0.15:10067
         udp dport 11813 counter dnat to 100.64.0.15:11813
     }
}

so on the first step (netdev) I'm setting dst mac to local (so packet 
will not be dropped as "alien", because I receive on this box mirrored 
(SPAN) traffic, where dst mac is not this box) and then load-balance it 
between multiple destinations using NAT/LB. As far as I understand, 
'fwd' is for forwarding to a single destination.

I will appreciate any suggestion on how to solve this task - either fix 
what I'm trying to do or using another way :-)

Thank you.

-- 
Volodymyr Litovka
   "Vision without Execution is Hallucination." -- Thomas Edison

Re: nftables / DHCP / NAT

[Pablo Neira Ayuso]

On Mon, Oct 30, 2023 at 12:57:52PM +0100, Volodymyr Litovka wrote:
[...]
> the basic idea of this construction is to use later load balancing
> (https://wiki.nftables.org/wiki-nftables/index.php/Load_balancing) between
> multiple destinations
>
[...]
>
> so on the first step (netdev) I'm setting dst mac to local (so packet will
> not be dropped as "alien", because I receive on this box mirrored (SPAN)
> traffic, where dst mac is not this box) and then load-balance it between
> multiple destinations using NAT/LB. As far as I understand, 'fwd' is for
> forwarding to a single destination.

You do not need to pass up packets to the IP stack for this purpose
and use stateful NAT, it makes things complicated.

You can combine the fwd statement with the numgen expression:

           ... fwd ip to numgen inc mod 2 map { \
                               0 : 192.168.10.100, \
                               1 : 192.168.10.110 } device "enp0s25"

this performs round-robin packet distribution over the two destination
IP addresses.

So the example in the wiki above also works fine for the fwd
statement.

If you would like to perform flow-based load balancing, you need to
create a dynamic set and store what destination IP address is used for
a given IP source for persistency.

Re: nftables / DHCP / NAT

[Volodymyr Litovka]

Hi Pablo,

On 10/30/23 09:41, Pablo Neira Ayuso wrote:
> Then, to forward packets to some other box from the 'netdev' family,
> use the 'fwd' statement:
>
>          udp dport 67 udp dport set 10067 counter fwd to 100.64.0.66 device "eth0"
>
> This rule above is mangling your UDP destination port from 67 to
> 10067, then it send the packet to 100.64.0.66 and device "eth0". The
> destination MAC address is updated by the neighbour layer so you do
> not have to bother with "ether daddr set ..."

This works, thanks. But - all packets are duplicated :)

If I run on the receiving host the command (in short - extract what I'm 
interested in)

tshark -i enp1s0 -l -f 'port 10067 and ether[42:1]==2' -d 
udp.port=10067,dhcp  -V -Y 'dhcp.option.dhcp==5' -V -E "separator=|" -E 
"quote=n" -E "occurrence=a" -T fields -e 
"dhcp.option.agent_information_option.value" -e "dhcp.ip.client"

then when using the old version of rules (just to check and compare) -
udp dport 67 meta pkttype set other ether daddr set 96:9f:7c:d3:c3:66 ip 
daddr set 100.64.0.15 udp dport set 10067 counter accept

then tshark shows everything is ok, but if using -
udp dport 67 udp dport set 10067 counter fwd ip to 100.64.0.15 device enp1s0

( which comes to
# nft list ruleset
table netdev inspan {
     chain rewrit {
         type filter hook ingress device "inspan" priority filter; 
policy drop;
         udp dport 67 udp dport set 10067 counter packets 9510 bytes 
4137201 fwd ip to 100.64.0.15 device "enp1s0"
     }
}

then every packet (same running tshark) is duplicated:

64702...31333430|x.x.x2.238
64702...31333430|x.x.x2.238
48522...312e3430|x.x.x4.168
48522...312e3430|x.x.x4.168
49534...312e3430|x.x.x5.119
49534...312e3430|x.x.x5.119
6c632...2f392d34|x.x.x5.183
6c632...2f392d34|x.x.x5.183

Linux 6.5.0-060500-generic #202308271831 SMP PREEMPT_DYNAMIC Sun Aug 27 
22:37:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
nftables v1.0.2 (Lester Gooch)

Any ideas why this can happen?

Thank you.

-- 
Volodymyr Litovka
   "Vision without Execution is Hallucination." -- Thomas Edison

Re: nftables / DHCP / NAT

[Pablo Neira Ayuso]

Hi Volodymyr,

On Mon, Oct 30, 2023 at 11:20:55PM +0100, Volodymyr Litovka wrote:
> Hi Pablo,
> 
> On 10/30/23 09:41, Pablo Neira Ayuso wrote:
> > Then, to forward packets to some other box from the 'netdev' family,
> > use the 'fwd' statement:
> > 
> >          udp dport 67 udp dport set 10067 counter fwd to 100.64.0.66 device "eth0"
> > 
> > This rule above is mangling your UDP destination port from 67 to
> > 10067, then it send the packet to 100.64.0.66 and device "eth0". The
> > destination MAC address is updated by the neighbour layer so you do
> > not have to bother with "ether daddr set ..."
> 
> This works, thanks. But - all packets are duplicated :)

I see no duplicates, see below.

> If I run on the receiving host the command (in short - extract what I'm
> interested in)
> 
> tshark -i enp1s0 -l -f 'port 10067 and ether[42:1]==2' -d
> udp.port=10067,dhcp  -V -Y 'dhcp.option.dhcp==5' -V -E "separator=|" -E
> "quote=n" -E "occurrence=a" -T fields -e
> "dhcp.option.agent_information_option.value" -e "dhcp.ip.client"
> 
> then when using the old version of rules (just to check and compare) -
> udp dport 67 meta pkttype set other ether daddr set 96:9f:7c:d3:c3:66 ip
> daddr set 100.64.0.15 udp dport set 10067 counter accept
> 
> then tshark shows everything is ok, but if using -
> udp dport 67 udp dport set 10067 counter fwd ip to 100.64.0.15 device enp1s0

I had to cleanup your snippet, it contains non-breaking space (0xc2
0x0a) which makes the parser unhappy:

test.nft:2:1-1: Error: syntax error, unexpected junk
    chain rewrit {
^

This is the cleaned up result:

table netdev inspan {
    chain rewrit {
       type filter hook ingress device "inspan" priority filter; policy drop;
       udp dport 67 udp dport set 10067 counter packets 9510 bytes 4137201 fwd ip to 100.64.0.15 device "enp1s0"
    }
}

> then every packet (same running tshark) is duplicated:
> 
> 64702...31333430|x.x.x2.238
> 64702...31333430|x.x.x2.238

I suspect this shows the packet coming in (ingress) and coming out
(egress) on the same device.

I can see this in tcpdump, I made a similar ruleset that matches on
udp dport 8000, it mangles it to udp 8888 and forwards it to the same
device:

14:52:32.753056 IP 192.168.210.137.51820 > 192.168.210.130.8000: UDP, length 4
14:52:32.753149 IP 192.168.210.137.51820 > 192.168.210.130.8888: UDP, length 4

The first packet is the packet shown by tcpdump at ingress, the second
packet is the packet that is forwarded and tcpdump shows it at egress.

On the collector, I can see one single packet with UDP port 8888.

> Any ideas why this can happen?

I think what you observe from tshark / tcpdump is the packet at
ingress and then (being reflected) at egress.

Re: nftables / DHCP / NAT

[Volodymyr Litovka]

Hi Pablo,

On 10/31/23 15:05, Pablo Neira Ayuso wrote:
> I see no duplicates, see below.
thank a lot for the help. You were right and, finally, it was my 
mistake. The difference is that when I use 'ip daddr set x.x.x.x' it 
rewrites packet and receiving host is the final destination, while 'fwd' 
is frame forwarding with original ip header so this packet, being 
received, was forwarded to the original destination and that's why I 
seen two copies - one from ingress, another one - from egress.

Thanks again!

-- 
Volodymyr Litovka
   "Vision without Execution is Hallucination." -- Thomas Edison