nft not logging some layer 2 frames?

nft not logging some layer 2 frames?

[Teodor Milkov]

Hi,

I've been trying the |netdev| table for ingress filtering, which seems 
to be working fine. However, I'm encountering an issue with logging.

Here's the relevant configuration snippet:

table netdev filter {
     chain eth2_input {
         type filter hook ingress device "eth2" priority filter; policy 
accept;
         vlan id 99 vlan type ip counter packets 68304372 bytes 
28196182850 accept
         vlan id 99 vlan type arp counter packets 756970 bytes 35442808 
accept
         vlan id 99 counter packets 441 bytes 49415 log accept;


As seen, the counters are updating, but logging does not seem to occur 
for certain Layer 2 frames - in this case UDLD:

18:00:55.426295 f4:4e:05:ab:cd:ef > 01:00:0c:cc:cc:cc, 802.3, length 
115: LLC, dsap SNAP (0xaa) Individual, ssap SNAP (0xaa) Command, ctrl 
0x03: oui Cisco (0x00000c), pid UDLD (0x0111), length 107: UDLDv1, Code 
Probe message (1), Flags [RT] (0x01), length 107

 From a brief review of the source code, it appears that nftables might 
only log frames of certain known types like e.g. ipv4, ipv6 and arp.

Could you confirm if this is the case, or if there's documentation or a 
configuration tweak I might have missed that could resolve this logging 
issue?

Best regards,

Teodor Milkov

Re: nft not logging some layer 2 frames?

[William N.]

On Fri, 24 May 2024 17:42:37 +0300 Teodor Milkov wrote:

>          vlan id 99 counter packets 441 bytes 49415 log accept;

Have you tried putting log before counter?

Re: nft not logging some layer 2 frames?

[Teodor Milkov]

On 24.05.24 г. 18:57 ч., William N. wrote:
> On Fri, 24 May 2024 17:42:37 +0300 Teodor Milkov wrote:
>
>>   =C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0 vlan id 99 counter packets 441 byt=
> es 49415 log accept;
>
> Have you tried putting log before counter?

I just tried putting log before counter and still nothing showing in 
logs. But does order really matter?

OTOH, if I let some of the IPv6 traffic hit that final line it gets 
logged just fine.

Re: nft not logging some layer 2 frames?

[William N.]

On Mon, 27 May 2024 13:17:50 +0300 Teodor Milkov wrote:

> I just tried putting log before counter and still nothing showing in 
> logs. But does order really matter?

Perhaps not. I just thought it might be worth trying.
What happens if you remove the counter and use only log?

I tried this:

table netdev xxx {
        chain ingress {
                type filter hook ingress device "eth0" priority -600

                counter
                log prefix "TEST: "
                drop
        }
}

and I ran 'arp' on the other end of the eth0 link. I can see logging
works, i.e. without specifying any particular protocol whatsoever.
Counter works too.

Re: nft not logging some layer 2 frames?

[Teodor Milkov]

On 27.05.24 г. 20:17 ч., William N. wrote:
> On Mon, 27 May 2024 13:17:50 +0300 Teodor Milkov wrote:
>
>> I just tried putting log before counter and still nothing showing in
>> logs. But does order really matter?
> Perhaps not. I just thought it might be worth trying.
> What happens if you remove the counter and use only log?
>
> I tried this:
>
> table netdev xxx {
>          chain ingress {
>                  type filter hook ingress device "eth0" priority -600
>
>                  counter
>                  log prefix "TEST: "
>                  drop
>          }
> }
>
> and I ran 'arp' on the other end of the eth0 link. I can see logging
> works, i.e. without specifying any particular protocol whatsoever.
> Counter works too.
>
Yes, as stated in my original message:

"From a brief review of the source code, it appears that nftables might 
only log frames of certain known types like e.g. ipv4, ipv6 and arp."

So, if you have a way to test with some other type like STP, LLDP, CDP 
or similar, maybe you'll be able to reproduce the issue.

Re: nft not logging some layer 2 frames?

[William N.]

Following the example given here:

https://www.freecodecamp.org/news/how-to-use-scapy-python-networking/

I did sendp(frame) while watching 'journalctl -f'. It showed me:

kernel: DEBUG: IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:11:22:33:44:55:90:00

Unless I am completely mistaken, this L2 frame was sent and logged.

Re: nft not logging some layer 2 frames?

[Pablo Neira Ayuso]

On Tue, May 28, 2024 at 03:01:31PM -0000, William N. wrote:
> Following the example given here:
> 
> https://www.freecodecamp.org/news/how-to-use-scapy-python-networking/
> 
> I did sendp(frame) while watching 'journalctl -f'. It showed me:
> 
> kernel: DEBUG: IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:11:22:33:44:55:90:00
> 
> Unless I am completely mistaken, this L2 frame was sent and logged.

For the record:

        ... log prefix "test: " flags ether

shows:

        test: IN=eth0 OUT= MACSRC=90:dd:b4:12:ea:f0 MACDST=ff:ff:ff:ff:ff:ff MACPROTO=8899

Re: nft not logging some layer 2 frames?

[William N.]

Thanks, Pablo.
Where can one look up the meaning of the MACPROTO number?

Re: nft not logging some layer 2 frames?

[Pablo Neira Ayuso]

On Tue, Jun 11, 2024 at 07:12:14PM -0000, William N. wrote:
> Thanks, Pablo.
> Where can one look up the meaning of the MACPROTO number?

That is the ethertype, the MACPROTO key is used for historical
reasons. Manpage can be enhanced to include this information.

Re: nft not logging some layer 2 frames?

[Teodor Milkov]

On 11.06.24 г. 21:25 ч., Pablo Neira Ayuso wrote:
> On Tue, May 28, 2024 at 03:01:31PM -0000, William N. wrote:
>> Following the example given here:
>>
>> https://www.freecodecamp.org/news/how-to-use-scapy-python-networking/
>>
>> I did sendp(frame) while watching 'journalctl -f'. It showed me:
>>
>> kernel: DEBUG: IN=eth0 OUT= MAC=ff:ff:ff:ff:ff:ff:00:11:22:33:44:55:90:00
>>
>> Unless I am completely mistaken, this L2 frame was sent and logged.
> For the record:
>
>          ... log prefix "test: " flags ether
>
> shows:
>
>          test: IN=eth0 OUT= MACSRC=90:dd:b4:12:ea:f0 MACDST=ff:ff:ff:ff:ff:ff MACPROTO=8899
>
I tried the simple nft ruleset shared by William in a previous message:

table netdev xxx {
         chain ingress {
                 type filter hook ingress device "eth-inet0" priority filter

                 ether type arp counter accept
                 ether type ip counter accept
                 vlan type arp counter accept
                 vlan type ip counter accept

                 counter limit rate 1/second log prefix "TEST: " flags ether
         }
}

The "arp" and "ip" rules I've added are there to suppress logging of 
non-relevant traffic.

Here's what traffic I'm seeing with tcpdump:

09:20:13.204053 0c:c4:7a:xx:yy:zz > ff:ff:ff:ff:ff:ff, ethertype AoE 
(0x88a2), length 60: AoE length 46, Ver 1, Flags: [none]

09:20:14.746200 b8:83:03:xx:yy:zz > 01:00:0c:cc:cc:cd, 802.3, length 50: 
LLC, dsap SNAP (0xaa) Individual, ssap SNAP (0xaa) Command, ctrl 0x03: 
oui Cisco (0x00000c), pid PVST (0x010b), length 42: STP 802.1w, Rapid 
STP, Flags [Proposal, Learn, Forward, Agreement], bridge-id 
8008.b8:83:03:39:89:00.800c, length 42

09:20:15.161805 b8:83:03:xx:yy:zz > 01:80:c2:00:00:0e, ethertype LLDP 
(0x88cc), length 279: LLDP, length 265: rackswitch-1-r6-v8

None of this is logged by nft.

Logging in general works, because if I remove some of the preceding 
rules e.g. "vlan type arp counter accept", then I'm seeing this:

Jun 26 09:25:15 coldspare1-us kernel: [2594462.618777] TEST: 
IN=eth-inet0 OUT= MACSRC=90:e2:ba:xx:yy:zz MACDST=ff:ff:ff:ff:ff:ff 
VPROTO=8100 VID=4 MACPROTO=0806 ARP HTYPE=1 PTYPE=0x0800 OPCODE=1 
MACSRC=90:e2:ba:xx:yy:zz IPSRC=aa.bb.197.2 MACDST=ff:ff:ff:ff:ff:ff 
IPDST=aa.bb.197.201

Could you confirm if a specific kernel or nft version is required for 
this? Also, should certain Ethernet driver features or offloading 
settings be disabled to accommodate these logging rules?

My kernel is "5.10.0-29-amd64 #1 SMP Debian 5.10.216-1 (2024-05-03)", 
nft v0.9.8 and ixgbe Ethernet driver.

Re: nft not logging some layer 2 frames?

[William N.]

On Wed, 26 Jun 2024 16:36:07 +0300 Teodor Milkov wrote:

> Could you confirm if a specific kernel or nft version is required for 
> this? Also, should certain Ethernet driver features or offloading 
> settings be disabled to accommodate these logging rules?

I don't know. I use Debian 12 VM, nftables v1.0.6.
Perhaps the experts here can provide better info.