Issues with netdev egress hooks

Issues with netdev egress hooks

[Daniel Mack]

Hi,

I am using the NFT egress hook in a netdev table with 'set' statements
to adjust the source MAC and IP addresses before duplicating packets to
another interface:

table netdev dummy {
  chain egress {
    type filter hook egress device "dummy" priority 0;
    ether type ip ether saddr set 01:02:03:04:05:06 ip saddr set 1.1.1.1
dup to "eth0"
  }
}

Does this rule look okay or am I holding it wrong?

The modification of the sender's MAC address works fine. However, the
adjustment of the source IP is applied at the wrong offset. The octets
in the raw packet that are being modified are 13 and 14, which would be
the correct offset within an IP header, but it seems that the prefixed
Ethernet header is not taken into account.

For the same reason, attempting to filter based on any details beyond
the Ethernet header also fails. The following rule does not match any
packets, even though there is a significant amount of UDP traffic:

table netdev dummy {
  chain egress {
    type filter hook egress device "dummy" priority 0;
    ether type ip ip protocol udp dup to "eth0"
  }
}

At this point, I'm not sure where to start digging to be honest and
would appreciate any guidance on how to resolve this issue.


Thanks,
Daniel

Re: Issues with netdev egress hooks

[Pablo Neira Ayuso]

Hi Daniel,

On Wed, Mar 06, 2024 at 04:43:02PM +0100, Daniel Mack wrote:
> Hi,
> 
> I am using the NFT egress hook in a netdev table with 'set' statements
> to adjust the source MAC and IP addresses before duplicating packets to
> another interface:
> 
> table netdev dummy {
>   chain egress {
>     type filter hook egress device "dummy" priority 0;
>     ether type ip ether saddr set 01:02:03:04:05:06 ip saddr set 1.1.1.1
> dup to "eth0"
>   }
> }

Is this a dummy device created via: ip link add dummy type dummy or
just a coincidence?

> Does this rule look okay or am I holding it wrong?

Rule looks correct.

> The modification of the sender's MAC address works fine. However, the
> adjustment of the source IP is applied at the wrong offset. The octets
> in the raw packet that are being modified are 13 and 14, which would be
> the correct offset within an IP header, but it seems that the prefixed
> Ethernet header is not taken into account.
> 
> For the same reason, attempting to filter based on any details beyond
> the Ethernet header also fails. The following rule does not match any
> packets, even though there is a significant amount of UDP traffic:
> 
> table netdev dummy {
>   chain egress {
>     type filter hook egress device "dummy" priority 0;
>     ether type ip ip protocol udp dup to "eth0"
>   }
> }
> 
> At this point, I'm not sure where to start digging to be honest and
> would appreciate any guidance on how to resolve this issue.

I guess you are running a kernel with

commit 0ae8e4cca78781401b17721bfb72718fdf7b4912
Author: Pablo Neira Ayuso <pablo@netfilter.org>
Date:   Thu Dec 14 11:50:12 2023 +0100

    netfilter: nf_tables: set transport offset from mac header for netdev/egress

so this is a different bug?

Re: Issues with netdev egress hooks

[Daniel Mack]

Hi Pablo,

Thanks a lot for your reply.

On 3/6/24 19:17, Pablo Neira Ayuso wrote:
> On Wed, Mar 06, 2024 at 04:43:02PM +0100, Daniel Mack wrote:
>> Hi,
>>
>> I am using the NFT egress hook in a netdev table with 'set' statements
>> to adjust the source MAC and IP addresses before duplicating packets to
>> another interface:
>>
>> table netdev dummy {
>>   chain egress {
>>     type filter hook egress device "dummy" priority 0;
>>     ether type ip ether saddr set 01:02:03:04:05:06 ip saddr set 1.1.1.1
>> dup to "eth0"
>>   }
>> }
> 
> Is this a dummy device created via: ip link add dummy type dummy or
> just a coincidence?

Yes, it's a dummy device. I'm using the egress netdev hook here because
it is executed before the device's .ndo_start_xmit() is called.

>> The modification of the sender's MAC address works fine. However, the
>> adjustment of the source IP is applied at the wrong offset. The octets
>> in the raw packet that are being modified are 13 and 14, which would be
>> the correct offset within an IP header, but it seems that the prefixed
>> Ethernet header is not taken into account.
>>
>> For the same reason, attempting to filter based on any details beyond
>> the Ethernet header also fails. The following rule does not match any
>> packets, even though there is a significant amount of UDP traffic:
>>
>> table netdev dummy {
>>   chain egress {
>>     type filter hook egress device "dummy" priority 0;
>>     ether type ip ip protocol udp dup to "eth0"
>>   }
>> }
>>
>> At this point, I'm not sure where to start digging to be honest and
>> would appreciate any guidance on how to resolve this issue.
> 
> I guess you are running a kernel with
> 
> commit 0ae8e4cca78781401b17721bfb72718fdf7b4912
> Author: Pablo Neira Ayuso <pablo@netfilter.org>
> Date:   Thu Dec 14 11:50:12 2023 +0100
> 
>     netfilter: nf_tables: set transport offset from mac header for netdev/egress
> 
> so this is a different bug?

Interesting, I did in fact run a 6.4 production kernel when I tried
this, and that didn't have that patch applied. Sorry for that oversight.

On 6.7, what I see is different but still broken:

This rules does the right thing and patches the source MAC correctly:

table netdev dummy {
  chain egress {
    type filter hook egress device dummy priority 0;
    ether saddr set 1:2:3:4:5:6 dup to eth0
  }
}

Whereas trying to patch the IP source addr leads to no packets being
forwarded at all anymore:

table netdev dummy {
  chain egress {
    type filter hook egress device dummy priority 0;
    ip saddr set 1.1.1.1 dup to eth0
  }
}

Interestingly, ether type filtering is also broken now, the following
also doesn't match any packets:

table netdev dummy {
  chain egress {
    type filter hook egress device dummy priority 0;
    ether type ip dup to eth0
  }
}

I browsed through the patches since 6.7 and couldn't find anything that
is related. Did I miss anything?


Best regards,
Daniel

Re: Issues with netdev egress hooks

[Pablo Neira Ayuso]

Hi Daniel,

On Thu, Mar 07, 2024 at 02:34:38PM +0100, Daniel Mack wrote:
> On 3/6/24 19:17, Pablo Neira Ayuso wrote:
[...]
> > I guess you are running a kernel with
> > 
> > commit 0ae8e4cca78781401b17721bfb72718fdf7b4912
> > Author: Pablo Neira Ayuso <pablo@netfilter.org>
> > Date:   Thu Dec 14 11:50:12 2023 +0100
> > 
> >     netfilter: nf_tables: set transport offset from mac header for netdev/egress
> > 
> > so this is a different bug?
> 
> Interesting, I did in fact run a 6.4 production kernel when I tried
> this, and that didn't have that patch applied. Sorry for that oversight.
> 
> On 6.7, what I see is different but still broken:

I'm here with 6.8.0-rc6+:

> This rules does the right thing and patches the source MAC correctly:
> 
> table netdev dummy {
>   chain egress {
>     type filter hook egress device dummy priority 0;
>     ether saddr set 1:2:3:4:5:6 dup to eth0
>   }
> }
> 
> Whereas trying to patch the IP source addr leads to no packets being
> forwarded at all anymore:

My setup:

# ip link set up dev dummy
# ip a a 10.141.10.1/24 dev dummy
# ip ro del local 10.141.10.1 dev dummy table local proto kernel scope host src 10.141.10.1

testing with ping to 10.141.10.1

I need to remove the local route, otherwise packets go through
loopback interface.

> table netdev dummy {
>   chain egress {
>     type filter hook egress device dummy priority 0;
>     ip saddr set 1.1.1.1 dup to eth0
>   }
> }

1) tcpdump in dummy:

17:14:42.939483 f2:20:1a:4c:c4:a1 > f2:20:1a:4c:c4:a1, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 10.141.10.1: ICMP echo request, id 46403, seq 1, length 64

2) tcpdump in eth0:

17:15:21.006853 f2:20:1a:4c:c4:a1 > f2:20:1a:4c:c4:a1, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 10.141.10.1: ICMP echo request, id 1087, seq 1, length 64

> Interestingly, ether type filtering is also broken now, the following
> also doesn't match any packets:
> 
> table netdev dummy {
>   chain egress {
>     type filter hook egress device dummy priority 0;
>     ether type ip dup to eth0
>   }
> }

1) tcpdump in dummy

17:18:13.921128 f2:20:1a:4c:c4:a1 > f2:20:1a:4c:c4:a1, ethertype IPv4 (0x0800), length 98: 10.141.10.1 > 10.141.10.1: ICMP echo request, id 137, seq 1, length 64

2) tcpdump in eth0:

17:19:00.398882 f2:20:1a:4c:c4:a1 > f2:20:1a:4c:c4:a1, ethertype IPv4 (0x0800), length 98: 10.141.10.1 > 10.141.10.1: ICMP echo request, id 21186, seq 1, length 64

> I browsed through the patches since 6.7 and couldn't find anything that
> is related. Did I miss anything?

I tried again this first example you posted:

table netdev dummy {
  chain egress {
    type filter hook egress device "dummy" priority 0;
    ether type ip ether saddr set 01:02:03:04:05:06 ip saddr set 1.1.1.1 dup to "eth0"
  }
}

tcpdump dummy:

17:22:08.390312 01:02:03:04:05:06 > f2:20:1a:4c:c4:a1, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 10.141.10.1: ICMP echo request, id 47168, seq 1, length 64

tcpdump enps25:

17:20:28.298524 01:02:03:04:05:06 > f2:20:1a:4c:c4:a1, ethertype IPv4 (0x0800), length 98: 1.1.1.1 > 10.141.10.1: ICMP echo request, id 15435, seq 1, length 64

Maybe my setup is different?