mv88e6320: Failed to forward PTP multicast

mv88e6320: Failed to forward PTP multicast

[Fabio Estevam]

Hi,

We are running kernel 6.1.26 on an imx8mn-based board with a Marvell
mv88e6320 switch.

eth1 and eth2 are the mv88e6320 ports. We connect a PTP sync source to eth1
and we notice that after setting up vlan_filtering on a bridge, the PTP
packets are no longer forwarded by the switch.

Below is the networking setup.

It does not matter if we assign an IP and sniff on the br0 or on the veth2,
PTP multicast is not appearing. Some multicast like ARP does come through.
Flags on br0: multicast_snooping = 1, mcast_flood  =1, mcast_router = 1

Any ideas as to how we can get the PTP packets to be forwarded?

Thanks,

Fabio Estevam

# Add bridge
ip link add name br0 type bridge
sleep 1

# Activate VLAN filtering
ip link set dev br0 type bridge vlan_filtering 1

# Add veth pairs
 ip link add veth1 type veth peer name veth2

sleep 1
# Set Interfaces to be part of the bridge
ip link set eth1 master br0
ip link set eth2 master br0
ip link set veth1 master br0

sleep 1
# Bring down interfaces
ip link set eth1 down
ip link set eth2 down

sleep 1
# Bring up interfaces
ip link set br0 up
ip link set veth1 up
ip link set veth2 up
ip link set eth1 up
ip link set eth2 up

sleep 1
ip addr add 192.168.0.1/24 dev veth2

tcpdump -i veth2 dst port 319 or dst port 320

Re: mv88e6320: Failed to forward PTP multicast

[Andrew Lunn]

On Thu, May 04, 2023 at 03:39:17PM -0300, Fabio Estevam wrote:
> Hi,
> 
> We are running kernel 6.1.26 on an imx8mn-based board with a Marvell
> mv88e6320 switch.



> 
> eth1 and eth2 are the mv88e6320 ports. We connect a PTP sync source to eth1
> and we notice that after setting up vlan_filtering on a bridge, the PTP
> packets are no longer forwarded by the switch.
> 
> Below is the networking setup.
> 
> It does not matter if we assign an IP and sniff on the br0 or on the veth2,
> PTP multicast is not appearing. Some multicast like ARP does come through.
> Flags on br0: multicast_snooping = 1, mcast_flood  =1, mcast_router = 1

Do you see the PTP traffic on eth1?

What MAC address is the PTP traffic using? Is it a link local MAC
address? There are some range of MAC addresses which you are not
supposed to forward across a bridge. e.g. you don't forward BPDUs.
Take a look at br_handle_frame(). Maybe you can play with
group_fwd_mask.

	 Andrew

Re: mv88e6320: Failed to forward PTP multicast

[Fabio Estevam]

Hi Andrew,

On Thu, May 4, 2023 at 4:21 PM Andrew Lunn <andrew@lunn.ch> wrote:

> Do you see the PTP traffic on eth1?

Yes, PTP traffic is seen on eth1.

> What MAC address is the PTP traffic using? Is it a link local MAC
> address? There are some range of MAC addresses which you are not
> supposed to forward across a bridge. e.g. you don't forward BPDUs.
> Take a look at br_handle_frame(). Maybe you can play with
> group_fwd_mask.

In our case, it is a multicast MAC.

The same traffic flows correctly when the bridge is not VLAN aware.

After VLAN is activated:

# Activate VLAN filtering
ip link set dev br0 type bridge vlan_filtering 1

Then the flow stops.

Thanks

Re: mv88e6320: Failed to forward PTP multicast

[Andrew Lunn]

On Thu, May 04, 2023 at 04:40:53PM -0300, Fabio Estevam wrote:
> Hi Andrew,
> 
> On Thu, May 4, 2023 at 4:21 PM Andrew Lunn <andrew@lunn.ch> wrote:
> 
> > Do you see the PTP traffic on eth1?
> 
> Yes, PTP traffic is seen on eth1.

So it is not a Marvell DSA issue. The frame is making it into the
Linux bridge.

> > What MAC address is the PTP traffic using? Is it a link local MAC
> > address? There are some range of MAC addresses which you are not
> > supposed to forward across a bridge. e.g. you don't forward BPDUs.
> > Take a look at br_handle_frame(). Maybe you can play with
> > group_fwd_mask.
> 
> In our case, it is a multicast MAC.

So not 01-80-C2-00-00-0E ?

I don't know how reliable it is, but see:

https://www.ieee802.org/1/files/public/docs2012/new-tc-messenger-tc-ptp-forwarding-1112-v02.pdf

Slide 5 says:

• 01-1B-19-00-00-00 – a general group address
  • An 802.1Q VLAN Bridge would forward the frame unchanged.
• 01-80-C2-00-00-0E – Individual LAN Scope group ad
  • An 802.1Q VLAN Bridge would drop the frame.

Maybe you are falling into this second case?

You really need to find out where in the Linux bridge it is getting
dropped. It should then be obvious why.

	 Andrew

Re: mv88e6320: Failed to forward PTP multicast

[Fabio Estevam]

[-- Attachment #1: Type: text/plain, Size: 1901 bytes --]

Hi Andrew,

On Thu, May 4, 2023 at 4:40 PM Fabio Estevam <festevam@gmail.com> wrote:
>
> Hi Andrew,
>
> On Thu, May 4, 2023 at 4:21 PM Andrew Lunn <andrew@lunn.ch> wrote:
>
> > Do you see the PTP traffic on eth1?
>
> Yes, PTP traffic is seen on eth1.

I gave you incorrect information, sorry about that.

Let me restart and share the test results.

We have made 2 tests. The only difference is the absence of "ip link
set dev vswitch0 type bridge vlan_filtering 1" in test 2

Test 1.) No PTP traffic on ETH1 can be sniffed after config
----------------------------------------------------------------------------
ip addr del 172.16.42.199/22 dev eth1
# Add bridge
ip link add name vswitch0 type bridge
sleep 1
ip link set dev vswitch0 type bridge vlan_filtering 1
sleep 1
ip link set eth1 master vswitch0
ip link set eth2 master vswitch0

sleep 1
ip link set eth1 down
ip link set eth2 down

sleep 1
ip link set vswitch0 up
ip link set eth1 up
ip link set eth2 up

sleep 1
ip addr add 172.16.42.199/22 dev eth1

sleep 1
tcpdump -i eth1 dst port 319 or dst port 320
----------- ------------------------------

Test 2.) PTP traffic on ETH1 can be sniffed after config
------------------------------------------------------------------------
ip addr del 172.16.42.199/22 dev eth1

# Add bridge
ip link add name vswitch0 type bridge

sleep 1
ip link set eth1 master vswitch0
ip link set eth2 master vswitch0

sleep 1
ip link set eth1 down
ip link set eth2 down

sleep 1
ip link set vswitch0 up
ip link set eth1 up
ip link set eth2 up

sleep 1
ip addr add 172.16.42.199/22 dev eth1

sleep 1
tcpdump -i eth1 dst port 319 or dst port 320
----------- ------------------------------

Please find attached the log of PTP sniffing on eth1 from test 2.

There is no such MAC address as 01-80-C2-00-00-0E.

Thanks,

Fabio Estevam

[-- Attachment #2: ptplog.txt --]
[-- Type: text/plain, Size: 26838 bytes --]

08:21:25.544600 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:25.544920 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37954, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9812 seconds, 583488768 nanoseconds
08:21:25.808242 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:25.808242 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37955, control : 0 (sync msg), log message interval : 254, originTimeStamp : 9812 seconds, 847676798 nanoseconds
08:21:25.809324 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:25.809324 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37955, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9812 seconds, 847871703 nanoseconds
08:21:25.909782 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 106: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 92)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : announce msg, length : 64, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 9586, control : 5 (Other), log message interval : 0, originTimeStamp : 0 seconds 0 nanoseconds, origin cur utc :0, rsvd : 0, gm priority_1 : 119, gm clock class : 248, gm clock accuracy : 254, gm clock variance : 61536, gm priority_2 : 112, gm clock id : 0x1dc1fffe1873fe, steps removed : 0, time source : 0xa0
08:21:26.062908 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:26.062908 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37956, control : 0 (sync msg), log message interval : 254, originTimeStamp : 9813 seconds, 102354533 nanoseconds
08:21:26.064007 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:26.064007 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37956, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9813 seconds, 102540812 nanoseconds
08:21:26.327039 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:26.327039 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37957, control : 0 (sync msg), log message interval : 254, originTimeStamp : 9813 seconds, 366488932 nanoseconds
08:21:26.327754 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:26.328471 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37957, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9813 seconds, 366678343 nanoseconds
08:21:26.369279 00:0d:b9:48:97:74 (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0x0, ttl 10, id 13520, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.40.238.ptp-event > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : delay req msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0xdb9fffe489774, port id : 1, seq id : 7187, control : 1 (delay req msg), log message interval : 127, originTimeStamp : 0 seconds, 0 nanoseconds
08:21:26.370133 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 96: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 82)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : delay resp msg, length : 54, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 7187, control : 3 (peer delay resp msg), log message interval : 0, receiveTimeStamp : 9813 seconds, 408626462 nanoseconds, port identity : 0xdb9fffe489774, port id : 1
08:21:26.581708 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:26.581708 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37958, control : 0 (sync msg), log message interval : 254, originTimeStamp : 9813 seconds, 621163694 nanoseconds
08:21:26.582774 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:26.582775 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37958, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9813 seconds, 621350218 nanoseconds
08:21:26.735242 00:1d:c1:50:ad:ff (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 2060, offset 0, flags [none], proto UDP (17), length 152)
    172.16.41.6.49156 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:26.736448 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 102: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 88)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:26.845778 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:26.845778 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37959, control : 0 (sync msg), log message interval : 254, originTimeStamp : 9813 seconds, 885214762 nanoseconds
08:21:26.846857 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:26.846857 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37959, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9813 seconds, 885403482 nanoseconds
08:21:26.927219 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 106: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 92)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : announce msg, length : 64, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 9587, control : 5 (Other), log message interval : 0, originTimeStamp : 0 seconds 0 nanoseconds, origin cur utc :0, rsvd : 0, gm priority_1 : 119, gm clock class : 248, gm clock accuracy : 254, gm clock variance : 61536, gm priority_2 : 112, gm clock id : 0x1dc1fffe1873fe, steps removed : 0, time source : 0xa0
08:21:27.100125 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:27.100125 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37960, control : 0 (sync msg), log message interval : 254, originTimeStamp : 9814 seconds, 139553709 nanoseconds
08:21:27.101178 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:27.101178 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37960, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9814 seconds, 139741779 nanoseconds
08:21:27.249035 00:1d:c1:21:d6:00 (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.11.3892 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:27.250272 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 102: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 88)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:27.354487 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:27.354487 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37961, control : 0 (sync msg), log message interval : 254, originTimeStamp : 9814 seconds, 393917398 nanoseconds
08:21:27.355188 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:27.355532 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37961, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9814 seconds, 394103229 nanoseconds
08:21:27.540079 00:0d:b9:48:97:74 (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0x0, ttl 10, id 13730, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.40.238.ptp-event > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : delay req msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0xdb9fffe489774, port id : 1, seq id : 7188, control : 1 (delay req msg), log message interval : 127, originTimeStamp : 0 seconds, 0 nanoseconds
08:21:27.541258 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 96: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 82)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : delay resp msg, length : 54, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 7188, control : 3 (peer delay resp msg), log message interval : 0, receiveTimeStamp : 9814 seconds, 579773844 nanoseconds, port identity : 0xdb9fffe489774, port id : 1
08:21:27.609143 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:27.609143 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37962, control : 0 (sync msg), log message interval : 254, originTimeStamp : 9814 seconds, 648577677 nanoseconds
08:21:27.610210 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37962, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9814 seconds, 648769083 nanoseconds
08:21:27.610210 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:27.726585 00:0d:b9:48:97:74 (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0x0, ttl 10, id 13753, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.40.238.ptp-event > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : delay req msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0xdb9fffe489774, port id : 1, seq id : 7189, control : 1 (delay req msg), log message interval : 127, originTimeStamp : 0 seconds, 0 nanoseconds
08:21:27.727137 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 96: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 82)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : delay resp msg, length : 54, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 7189, control : 3 (peer delay resp msg), log message interval : 0, receiveTimeStamp : 9814 seconds, 765935179 nanoseconds, port identity : 0xdb9fffe489774, port id : 1
08:21:27.776918 00:1d:c1:18:80:16 (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.13.51376 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:27.777861 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 102: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 88)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:27.873225 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:27.873225 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37963, control : 0 (sync msg), log message interval : 254, originTimeStamp : 9814 seconds, 912641721 nanoseconds
08:21:27.874247 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:27.874582 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37963, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9814 seconds, 912831705 nanoseconds
08:21:27.954380 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 106: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 92)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : announce msg, length : 64, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 9588, control : 5 (Other), log message interval : 0, originTimeStamp : 0 seconds 0 nanoseconds, origin cur utc :0, rsvd : 0, gm priority_1 : 119, gm clock class : 248, gm clock accuracy : 254, gm clock variance : 61536, gm priority_2 : 112, gm clock id : 0x1dc1fffe1873fe, steps removed : 0, time source : 0xa0
08:21:28.127496 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:28.127518 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37964, control : 0 (sync msg), log message interval : 254, originTimeStamp : 9815 seconds, 167001301 nanoseconds
08:21:28.128614 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
    172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)
08:21:28.128614 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.3529 > 224.0.1.129.ptp-general: [udp sum ok] PTPv2, v1 compat : no, msg type : follow up msg, length : 44, domain : 0, reserved1 : 0, Flags [none], NS correction : 0, sub NS correction : 0, reserved2 : 0, clock identity : 0x1dc1fffe1873fe, port id : 2, seq id : 37964, control : 2 (peer delay req msg), log message interval : 254, preciseOriginTimeStamp : 9815 seconds, 167189655 nanoseconds
08:21:28.381877 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 166: (tos 0xe0, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 152)
    172.16.41.45.1607 > 224.0.1.129.ptp-event: [udp sum ok] PTPv1 (not implemented)
08:21:28.381877 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 86: (tos 0xb8, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.41.45.2504 > 224.0.1.129.ptp-event: [udp sum ok] PTPv2, v1 compat : no, msg type : sync msg, length : 44, domain : 0, reserved1 : 0, Flags [two step], NS correction : 0, sub^C
188 packets captured
188 packets received by filter
0 packets dropped by kernel

Re: mv88e6320: Failed to forward PTP multicast

[Andrew Lunn]

> I gave you incorrect information, sorry about that.
> 
> Let me restart and share the test results.
> 
> We have made 2 tests. The only difference is the absence of "ip link
> set dev vswitch0 type bridge vlan_filtering 1" in test 2

O.K, so we are back to the switch hardware.

I notice you have an extra, unneeded, down/up in you script. Is that
required to produce the problem?

> 
> Test 1.) No PTP traffic on ETH1 can be sniffed after config
> ----------------------------------------------------------------------------
> ip addr del 172.16.42.199/22 dev eth1
> # Add bridge
> ip link add name vswitch0 type bridge
> sleep 1
> ip link set dev vswitch0 type bridge vlan_filtering 1
> sleep 1
> ip link set eth1 master vswitch0
> ip link set eth2 master vswitch0
> 
> sleep 1
> ip link set eth1 down
> ip link set eth2 down
> 
> sleep 1
> ip link set vswitch0 up
> ip link set eth1 up
> ip link set eth2 up
> 
> sleep 1
> ip addr add 172.16.42.199/22 dev eth1
> 
> sleep 1
> tcpdump -i eth1 dst port 319 or dst port 320
> ----------- ------------------------------
> 
> Test 2.) PTP traffic on ETH1 can be sniffed after config
> ------------------------------------------------------------------------
> ip addr del 172.16.42.199/22 dev eth1
> 
> # Add bridge
> ip link add name vswitch0 type bridge
> 
> sleep 1
> ip link set eth1 master vswitch0
> ip link set eth2 master vswitch0
> 
> sleep 1
> ip link set eth1 down
> ip link set eth2 down
> 
> sleep 1
> ip link set vswitch0 up
> ip link set eth1 up
> ip link set eth2 up
> 
> sleep 1
> ip addr add 172.16.42.199/22 dev eth1
> 
> sleep 1
> tcpdump -i eth1 dst port 319 or dst port 320
> ----------- ------------------------------
> 
> Please find attached the log of PTP sniffing on eth1 from test 2.
> 
> There is no such MAC address as 01-80-C2-00-00-0E.

> 08:21:25.544600 00:1d:c1:18:73:fe (oui Unknown) > 01:00:5e:00:01:81 (oui Unknown), ethertype IPv4 (0x0800), length 94: (tos 0xb8, ttl 1, id 0, offset 0, flags [DF], proto UDP (17), length 80)
>     172.16.41.45.3187 > 224.0.1.129.ptp-general: [udp sum ok] PTPv1 (not implemented)

So some background....

I mention 01-80-C2-00-00-0E because the switch looks for specific
types of frames and forwards them to the CPU, independent of the
status of the port. So for example, BPDUs used for spanning tree, use
a MAC address in the range 01-80-C2-00-00-XX, so the switch forwards
them to the CPU. PTP using 01-80-C2-00-00-0E would also match and get
forwarded to the CPU.

You are using 01:00:5e:00:01:81, so that is just general
multicast. The hardware matching for PTP is probably not going to
match on that.

What might also be playing a role here, maybe, is IGMP snooping. Does
your downstream PTP client issue IGMP join requests for the group, and
does the Linux bridge see them? If IGMP snooping is active, and there
is no IGMP signalling, there is no need to actually forward the PTP
frames, since nobody is interested in them. If there is a client
interested in the traffic, you would expect to see a multicast FDB
entry added to the switch in order that it forwards the multicast to
the CPU. However, this does not really fit the vlan_filtering.

When i look at your tcpdump traffic, i don't see a VLAN for your PTP
traffic. So i'm assuming it is untagged. You also don't appear to be
setting a default VLAN PVID. If you define it, untagged traffic gets
tagged on ingress and is then handled like tagged traffic.

So this is probably why your traffic is dropped. Try:

ip link set dev vswitch0 type bridge vlan_filtering 1 vlan_default_pvid 42

So that untagged traffic gets tagged as VLAN 42.

I'm not too familiar with all this VLAN stuff. So i could be telling
your wrong information.... 'self' is also importing in way's i don't
really understand. Vladimir and Tobias are the experts here.

   Andrew

Re: mv88e6320: Failed to forward PTP multicast

[Fabio Estevam]

Hi Andrew,

On Fri, May 5, 2023 at 10:02 AM Andrew Lunn <andrew@lunn.ch> wrote:

> I notice you have an extra, unneeded, down/up in you script. Is that
> required to produce the problem?

No, this sequence is not required to reproduce the problem.

> So some background....
>
> I mention 01-80-C2-00-00-0E because the switch looks for specific
> types of frames and forwards them to the CPU, independent of the
> status of the port. So for example, BPDUs used for spanning tree, use
> a MAC address in the range 01-80-C2-00-00-XX, so the switch forwards
> them to the CPU. PTP using 01-80-C2-00-00-0E would also match and get
> forwarded to the CPU.
>
> You are using 01:00:5e:00:01:81, so that is just general
> multicast. The hardware matching for PTP is probably not going to
> match on that.

Yes, that may be the reason, but why doesn't the hardware filter allow
us to forward unknown multicasts (non-IGMP subscribed)? Even if we set it to do
so with "mcast_flood"?

> What might also be playing a role here, maybe, is IGMP snooping. Does
> your downstream PTP client issue IGMP join requests for the group, and
> does the Linux bridge see them? If IGMP snooping is active, and there
> is no IGMP signalling, there is no need to actually forward the PTP
> frames, since nobody is interested in them. If there is a client
> interested in the traffic, you would expect to see a multicast FDB
> entry added to the switch in order that it forwards the multicast to
> the CPU. However, this does not really fit the vlan_filtering.

Yes true, but we are also testing with a third-party software
(PTPTrackHound) that does
the IGMP join, and it also fails in this case.
In any case, the "bridge -d -s mdb show" shows us always the correct
subscriptions if IGMP is active.
Further, if we disable IGMP snooping, all multicasts should be forwarded, and
still, our PTP multicast is not forwarded.

> When i look at your tcpdump traffic, i don't see a VLAN for your PTP
> traffic. So i'm assuming it is untagged. You also don't appear to be
> setting a default VLAN PVID. If you define it, untagged traffic gets
> tagged on ingress and is then handled like tagged traffic.

That was to simplify the example. It makes no difference if we add
more or less VLANs, tagged or not.

After adding a bridge and making it vlan aware, then these defaults are
applied:

# bridge vlan show
port              vlan-id
eth1              1 PVID Egress Untagged
eth2              1 PVID Egress Untagged
br0              1 PVID Egress Untagged

We can play with different VIDs, but it does not make any difference,
unfortunately.

We have tried a lot of different combinations and also tried to remove
the VLAN from the bridge and reassign it, but it makes
no difference.

Thanks

Re: mv88e6320: Failed to forward PTP multicast

[Fabio Estevam]

On Fri, May 5, 2023 at 10:02 AM Andrew Lunn <andrew@lunn.ch> wrote:

> I'm not too familiar with all this VLAN stuff. So i could be telling
> your wrong information.... 'self' is also importing in way's i don't
> really understand. Vladimir and Tobias are the experts here.

Vladimir and Tobias,

Would you have any suggestions as to how to allow PTP multicast to be
forwarded when
vlan_filtering is active?

The whole thread is at:
https://lore.kernel.org/netdev/5cd6a70c-ea13-4547-958f-5806f86bfa10@lunn.ch/T/#m6453e569a98478bf5bddf09895393c3a52b91727

Thanks,

Fabio Estevam

Re: mv88e6320: Failed to forward PTP multicast

[Vladimir Oltean]

Hi Fabio,

On Wed, May 10, 2023 at 11:05:46AM -0300, Fabio Estevam wrote:
> On Fri, May 5, 2023 at 10:02 AM Andrew Lunn <andrew@lunn.ch> wrote:
> 
> > I'm not too familiar with all this VLAN stuff. So i could be telling
> > your wrong information.... 'self' is also importing in way's i don't
> > really understand. Vladimir and Tobias are the experts here.
> 
> Vladimir and Tobias,
> 
> Would you have any suggestions as to how to allow PTP multicast to be
> forwarded when
> vlan_filtering is active?
> 
> The whole thread is at:
> https://lore.kernel.org/netdev/5cd6a70c-ea13-4547-958f-5806f86bfa10@lunn.ch/T/#m6453e569a98478bf5bddf09895393c3a52b91727
> 
> Thanks,
> 
> Fabio Estevam

I checked out the v6.1.26 tag from linux-stable and I was able to
synchronize 2 stations attached to my Turris MOX (Marvell 6190) with
this commands: sudo ptp4l -i eth0 -4 -m
(also I was able to synchronize a third station behind a mvneta bridge
port foreign to the MV88E6190, using software forwarding)

My bridge configuration is VLAN-aware. FWIW, I'm using vlan_default_pvid
1000, but it should not make a difference.

In a bridging configuration where there are only 2 ports in the bridge
PVID (1 source and 1 destination), could you please run the following
command from a station attached to one of the Marvell switch ports:

board # ethtool -S lanX | grep -v ': 0'
station # mausezahn eth0 -B 224.0.1.129 -c 1000 -t udp "dp=319"
board # ethtool -S lanX | grep -v ': 0'

and tell me which counters increment?

I am also curious whether there is any difference to your setup between:
ip link add br0 type bridge
ip link set br0 type bridge vlan_filtering 1 # dynamic toggling of VLAN awareness
and:
ip link add br0 type bridge vlan_filtering 1 # static creation of VLAN-aware bridge

I've tested both forms on my setup, and both work. Who knows, maybe
something happens differently on your particular kernel, or with your
particular switch model.

Is it vanilla v6.1.26 or something else?

Re: mv88e6320: Failed to forward PTP multicast

[Tobias Waldekranz]

Hi Fabio,

On ons, maj 10, 2023 at 11:05, Fabio Estevam <festevam@gmail.com> wrote:
> On Fri, May 5, 2023 at 10:02 AM Andrew Lunn <andrew@lunn.ch> wrote:
>
>> I'm not too familiar with all this VLAN stuff. So i could be telling
>> your wrong information.... 'self' is also importing in way's i don't
>> really understand. Vladimir and Tobias are the experts here.
>
> Vladimir and Tobias,
>
> Would you have any suggestions as to how to allow PTP multicast to be
> forwarded when
> vlan_filtering is active?

If possible, could you install mdio-tools and paste the output of `mvls`
on your board from the two configurations above?

Unfortunately, you will have to patch it to support your device. Based
on a quick view of the datasheet, this should probably work:

diff --git a/src/mvls/mvls.c b/src/mvls/mvls.c
index 3ced04a..d4442c9 100644
--- a/src/mvls/mvls.c
+++ b/src/mvls/mvls.c
@@ -195,6 +195,16 @@ const struct chip chips[] = {
                .family = &opal_family,
                .n_ports = 11,
        },
+       {
+               .id = "Marvell 88E6320",
+               .family = &opal_family,
+               .n_ports = 7,
+       },
+       {
+               .id = "Marvell 88E6321",
+               .family = &opal_family,
+               .n_ports = 7,
+       },
        {
                .id = "Marvell 88E6352",
                .family = &opal_family,

Re: mv88e6320: Failed to forward PTP multicast

[Fabio Estevam]

[-- Attachment #1: Type: text/plain, Size: 2172 bytes --]

Hi Vladimir,

On Wed, May 10, 2023 at 3:28 PM Vladimir Oltean <olteanv@gmail.com> wrote:

> I checked out the v6.1.26 tag from linux-stable and I was able to
> synchronize 2 stations attached to my Turris MOX (Marvell 6190) with
> this commands: sudo ptp4l -i eth0 -4 -m
> (also I was able to synchronize a third station behind a mvneta bridge
> port foreign to the MV88E6190, using software forwarding)

Thanks for testing it, appreciate it.

> My bridge configuration is VLAN-aware. FWIW, I'm using vlan_default_pvid
> 1000, but it should not make a difference.
>
> In a bridging configuration where there are only 2 ports in the bridge
> PVID (1 source and 1 destination), could you please run the following
> command from a station attached to one of the Marvell switch ports:
>
> board # ethtool -S lanX | grep -v ': 0'
> station # mausezahn eth0 -B 224.0.1.129 -c 1000 -t udp "dp=319"
> board # ethtool -S lanX | grep -v ': 0'
>
> and tell me which counters increment?

In our tests:
eth0 is the port connected to the i.MX8MN.
eth1 and eth2 are the Marvell switch ports

Please find attached two configurations and the results.

Some notes:

- We have bridged (eth1+eth2) = br0, no matter if it is VLAN aware or not.
- PTP traffic flows correctly over eth1+eth2 (the 2 hardware switch interfaces)
- PTP traffic appears shortly, (like during 30 seconds) on the
non-VLAN-aware case
br0 interface.
- PTP traffic does not appear on the VLAN-aware br0 interface

> I am also curious whether there is any difference to your setup between:
> ip link add br0 type bridge
> ip link set br0 type bridge vlan_filtering 1 # dynamic toggling of VLAN awareness
> and:
> ip link add br0 type bridge vlan_filtering 1 # static creation of VLAN-aware bridge

It does not show any difference here.

> I've tested both forms on my setup, and both work. Who knows, maybe
> something happens differently on your particular kernel, or with your
> particular switch model.
>
> Is it vanilla v6.1.26 or something else?

It is vanilla v6.1.26 plus the devicetree for this board, plus an
audio codec patch.

Thanks,

Fabio Estevam

[-- Attachment #2: mausezahn.tar.xz --]
[-- Type: application/x-xz, Size: 1248 bytes --]

Re: mv88e6320: Failed to forward PTP multicast

[Fabio Estevam]

[-- Attachment #1: Type: text/plain, Size: 871 bytes --]

Hi Tobias,

On Wed, May 10, 2023 at 6:34 PM Tobias Waldekranz <tobias@waldekranz.com> wrote:

> If possible, could you install mdio-tools and paste the output of `mvls`
> on your board from the two configurations above?
>
> Unfortunately, you will have to patch it to support your device. Based
> on a quick view of the datasheet, this should probably work:

I have installed mdio-tools with the suggested patch to support 88E6320.

Please find attached two tests with their mvls results.

Test 1: netconfig_PTP30s_mvls_test.sh
Test 2: netconfig_NOPTP_mvls_test.sh

The only difference between these two tests is that the second one adds:
ip link set dev br0 type bridge vlan_filtering 1

In the first case, PTP traffic is seen for about 30 seconds and then it stops.
In the second case, no PTP traffic is seen at all.

Thanks,

Fabio Estevam

[-- Attachment #2: mvls.tar.xz --]
[-- Type: application/x-xz, Size: 1508 bytes --]

Re: mv88e6320: Failed to forward PTP multicast

[Vladimir Oltean]

Hi Fabio,

On Thu, May 11, 2023 at 08:03:01AM -0300, Fabio Estevam wrote:
> Hi Vladimir,
> 
> On Wed, May 10, 2023 at 3:28 PM Vladimir Oltean <olteanv@gmail.com> wrote:
> 
> > I checked out the v6.1.26 tag from linux-stable and I was able to
> > synchronize 2 stations attached to my Turris MOX (Marvell 6190) with
> > this commands: sudo ptp4l -i eth0 -4 -m
> > (also I was able to synchronize a third station behind a mvneta bridge
> > port foreign to the MV88E6190, using software forwarding)
> 
> Thanks for testing it, appreciate it.
> 
> > My bridge configuration is VLAN-aware. FWIW, I'm using vlan_default_pvid
> > 1000, but it should not make a difference.
> >
> > In a bridging configuration where there are only 2 ports in the bridge
> > PVID (1 source and 1 destination), could you please run the following
> > command from a station attached to one of the Marvell switch ports:
> >
> > board # ethtool -S lanX | grep -v ': 0'
> > station # mausezahn eth0 -B 224.0.1.129 -c 1000 -t udp "dp=319"
> > board # ethtool -S lanX | grep -v ': 0'
> >
> > and tell me which counters increment?
> 
> In our tests:
> eth0 is the port connected to the i.MX8MN.

I don't see the "eth0" name referenced in any of the attached files. By
"connected to the i.MX8MN" you mean "separate from the board under test",
right? To be more specific, it is always connected to the eth2 switch
port, correct?

> eth1 and eth2 are the Marvell switch ports
> 
> Please find attached two configurations and the results.
> 
> Some notes:
> 
> - We have bridged (eth1+eth2) = br0, no matter if it is VLAN aware or not.
> - PTP traffic flows correctly over eth1+eth2 (the 2 hardware switch interfaces)

"Flows correctly" with vlan_filtering=1? (netconfig_mausezahn_test1.sh, right)?

> - PTP traffic appears shortly, (like during 30 seconds) on the
> non-VLAN-aware case br0 interface.

Which test case is this? Both test1 and test2 are fully VLAN-aware from
the start.

What happens after 30 seconds? PTP traffic disappears? Is the bridge
still VLAN-unaware when this happens?

> - PTP traffic does not appear on the VLAN-aware br0 interface

The collected statistics are a bit noisy. The $(after - before) for
in_multicasts is 1400, and for out_multicasts is 1026. So it appears
that multicast traffic is flowing bidirectionally. Could you either stop
the other sources of traffic while repeating the experiment, or send a
higher number of mausezahn packets, so they stand out more clearly in
the delta? The mausezahn packets should have the same basic characteristics
to the switch as the PTP traffic, so you shouldn't need PTP to figure
out what's wrong for now.

Also, result_mausezahn_test1.txt says:
| Station that generates the traffic with mausezahn is connected to eth2, statistics on eth1

Could you take the statistics on the ingress switch port please? (eth2)

As for result_mausezahn_test2.txt, it says:
| Station that generates the traffic with mausezahn is connected to eth2, statistics on br0

but eth2 is not one of the ports under br0! (only eth1 and veth1 are)
So you don't expect eth2 to forward these packets, do you?

Also, the same comment: please capture the statistics at the ingress
port, not at the expected egress port. I am operating under the
assumption that in the case which doesn't work for you, the switch drops
the packets somewhere, and I want to find out the reason (which should
be on the ingress port).

Re: mv88e6320: Failed to forward PTP multicast

[Tobias Waldekranz]

On tor, maj 11, 2023 at 08:16, Fabio Estevam <festevam@gmail.com> wrote:
> Hi Tobias,
>
> On Wed, May 10, 2023 at 6:34 PM Tobias Waldekranz <tobias@waldekranz.com> wrote:
>
>> If possible, could you install mdio-tools and paste the output of `mvls`
>> on your board from the two configurations above?
>>
>> Unfortunately, you will have to patch it to support your device. Based
>> on a quick view of the datasheet, this should probably work:
>
> I have installed mdio-tools with the suggested patch to support 88E6320.

Good, that seems to at least partially work. From the VTU dump (first
table), we can see that all VLANs appear to be using FID 0, which they
shouldn't do. Do you have any patches on mv88e6xxx? It might also be a
bug in mvls, in that Pearl is not completely compatible with Opal w.r.t
to the VTU.

> Please find attached two tests with their mvls results.
>
> Test 1: netconfig_PTP30s_mvls_test.sh

In this case, we can see that the PTP group (224.0.1.129, aka
01:00:5e:00:01:81) is not in the ATU. Therefore, the message will be
flooded to all ports that allow flooding of unknown multicast, i.e. have
their "m" bit set in the "FL" column of the port table. Therefore, you
should see the packet both on the CPU and on eth2.

This assumes that there is no active config to trap PTP packets. I am
not familiar with how that is setup. To verify this, you can run a
tcpdump on the interface connected to the cpu port of the switch, with
the "-e" flag set, and check whether the PTP packets arrive with a
FORWARD or a TO_CPU tag.

> Test 2: netconfig_NOPTP_mvls_test.sh

In this case, this line in the ATU...

01:00:5e:00:01:81     0  static     -  -  .  .  .  3  4  .  .

Shows that the group is treated as registered multicast, and is
therefore only allowed to egress through port 3 (eth1) and 4 (eth2). You
should therefore see it on eth2, but it would not show up at the CPU.

I imagine that if you were to open a socket and add a membership to the
group, the packets would reach the CPU. What happens if you run:

socat udp-recvfrom:1234,ip-add-membership=224.0.1.129:br0 gopen:/dev/null &

Can you see the PTP packets on the CPU now?

> The only difference between these two tests is that the second one adds:
> ip link set dev br0 type bridge vlan_filtering 1
>
> In the first case, PTP traffic is seen for about 30 seconds and then it stops.
> In the second case, no PTP traffic is seen at all.

I would guess that this is when br0 gives up on finding a multicast
router on the network, and assumes the role of querier. How does the
output of mvls change before and after this point in time?

Re: mv88e6320: Failed to forward PTP multicast

[Fabio Estevam]

Hi Vladimir,

On Thu, May 11, 2023 at 8:46 AM Vladimir Oltean <olteanv@gmail.com> wrote:

> I don't see the "eth0" name referenced in any of the attached files. By
> "connected to the i.MX8MN" you mean "separate from the board under test",
> right? To be more specific, it is always connected to the eth2 switch
> port, correct?

Our system looks like this:

https://ibb.co/F7d1tnK

I realized that I provided several scripts and that made things confusing.
Sorry about that.

I found a simpler and more reliable way to reproduce the problem.

Script 1: Non-VLAN-aware bridge script

ip link add br0 type bridge vlan_filtering 0
ip link set eth1 master br0
ip link set eth2 master br0
ip addr add 192.168.0.97/24 dev br0
ip link set br0 up
tcpdump -i br0 dst port 319 or dst port 320

Script 2: VLAN-aware bridge script

ip link add br0 type bridge vlan_filtering 1
ip link set eth1 master br0
ip link set eth2 master br0
ip addr add 192.168.0.97/24 dev br0
ip link set br0 up
tcpdump -i br0 dst port 319 or dst port 320

(only difference between them is 'vlan_filtering 0' versus 'vlan_filtering 1').

On the Linux PC, I run:

"sudo ptp4l -i eth0 -m"

When I run script 1 on the imx8mn board, tcpdump constantly shows the
PTP traffic.

When I run script 2 on the imx8mn board, tcpdump shows the PTP traffic only
for a short duration of time (from 10 seconds to about 1 minute) and then
it no longer shows the PTP traffic.

When I get into this "blocked" situation if I restart the bridge manually:
ip link set br0 down
ip link set br0 up

Then tcpdump starts showing the PTP traffic again, but only for a
short duration of time, and stops again.

Now that I have a more reliable way to reproduce the issue, I can run
more tests/debugging.
Please let me know if you have any suggestions.

Thanks a lot,

Fabio Estevam

Re: mv88e6320: Failed to forward PTP multicast

[Andrew Lunn]

> When I get into this "blocked" situation if I restart the bridge manually:
> ip link set br0 down
> ip link set br0 up
> 
> Then tcpdump starts showing the PTP traffic again, but only for a
> short duration of time, and stops again.
> 
> Now that I have a more reliable way to reproduce the issue, I can run
> more tests/debugging.
> Please let me know if you have any suggestions.

This behaviour sounds like IGMP snooping, or something like that. The
bridge is adding in an entry to say don't send the traffic to the CPU,
nobody is interested in it.

I would add some debug prints into mv88e6xxx_port_fdb_add(),
mv88e6xxx_port_fdb_del() mv88e6xxx_port_mdb_add() and
mv88e6xxx_port_mdb_del() and see what entries are getting. You can
then backtrack and see why the bridge is adding them.

Also, Tobias asked about the type of frame being passed from the
switch to the host for PTP frames. Is it TO_CPU or FORWARD?  tcpdump
-e on the FEC interface will show you additional information in the
DSA header.

   Andrew

Re: mv88e6320: Failed to forward PTP multicast

[Fabio Estevam]

Hi Tobias,

On Thu, May 11, 2023 at 8:56 AM Tobias Waldekranz <tobias@waldekranz.com> wrote:

> I imagine that if you were to open a socket and add a membership to the
> group, the packets would reach the CPU. What happens if you run:
>
> socat udp-recvfrom:1234,ip-add-membership=224.0.1.129:br0 gopen:/dev/null &
>
> Can you see the PTP packets on the CPU now?

Yes, this seems to do the trick!

After adding a membership to the group, I don't see the PTP traffic
getting blocked anymore on a VLAN-aware configuration.

I will be running more tests, but this seems to be very promising.

Thanks a lot!

Fabio Estevam

Re: mv88e6320: Failed to forward PTP multicast

[Vladimir Oltean]

On Tue, May 16, 2023 at 06:29:13PM +0200, Andrew Lunn wrote:
> > When I get into this "blocked" situation if I restart the bridge manually:
> > ip link set br0 down
> > ip link set br0 up
> > 
> > Then tcpdump starts showing the PTP traffic again, but only for a
> > short duration of time, and stops again.
> > 
> > Now that I have a more reliable way to reproduce the issue, I can run
> > more tests/debugging.
> > Please let me know if you have any suggestions.
> 
> This behaviour sounds like IGMP snooping, or something like that. The
> bridge is adding in an entry to say don't send the traffic to the CPU,
> nobody is interested in it.

This is more to Fabio: right now I don't really understand what's the
problem. In the initial message you reported that the switch doesn't
forward IPv4 PTP between bridged ports, but I guess that's not true?
Or if it is, the lack of IPv4 PTP in the tcpdump output on br0 is
completely unrelated. It would help if you could restate the real issue.

> I would add some debug prints into mv88e6xxx_port_fdb_add(),
> mv88e6xxx_port_fdb_del() mv88e6xxx_port_mdb_add() and
> mv88e6xxx_port_mdb_del() and see what entries are getting. You can
> then backtrack and see why the bridge is adding them.

FWIW, there's also a more "modern" way of debugging in net-next, which
is either to put "trace_event=dsa" in the kernel cmdline and to
cat /sys/kernel/debug/tracing/trace, or to run "trace-cmd record -e dsa <command ...>"
followed by "trace-cmd report".

> 
> Also, Tobias asked about the type of frame being passed from the
> switch to the host for PTP frames. Is it TO_CPU or FORWARD?  tcpdump
> -e on the FEC interface will show you additional information in the
> DSA header.

Seeing that these come and go to the host (eth0) based on the presence
of the 01:00:5e:00:01:81 entry in the ATU, I'd say these come with a
FORWARD tag, but indeed it would be good to confirm. Only
MV88E6XXX_VID_STANDALONE is installed with vlan.policy = true,
the others aren't, so I don't see a reason why IPv4 PTP would reach the
CPU via a trap here.

Re: mv88e6320: Failed to forward PTP multicast

[Vladimir Oltean]

On Tue, May 16, 2023 at 03:10:00PM -0300, Fabio Estevam wrote:
> Hi Tobias,
> 
> On Thu, May 11, 2023 at 8:56 AM Tobias Waldekranz <tobias@waldekranz.com> wrote:
> 
> > I imagine that if you were to open a socket and add a membership to the
> > group, the packets would reach the CPU. What happens if you run:
> >
> > socat udp-recvfrom:1234,ip-add-membership=224.0.1.129:br0 gopen:/dev/null &
> >
> > Can you see the PTP packets on the CPU now?
> 
> Yes, this seems to do the trick!
> 
> After adding a membership to the group, I don't see the PTP traffic
> getting blocked anymore on a VLAN-aware configuration.
> 
> I will be running more tests, but this seems to be very promising.

Slightly unrelated to the original topic and probably completely
unrelated to Fabio's actual issue.

I'm completely inapt when it comes to IP multicast. Tobias, does the
fact that br0 have mcast_router 1 mean that the CPU port should be a
recipient of all multicast traffic, registered or unregistered?

Re: mv88e6320: Failed to forward PTP multicast

[Andrew Lunn]

> Slightly unrelated to the original topic and probably completely
> unrelated to Fabio's actual issue.
> 
> I'm completely inapt when it comes to IP multicast. Tobias, does the
> fact that br0 have mcast_router 1 mean that the CPU port should be a
> recipient of all multicast traffic, registered or unregistered?

It is a long time since i did much with multicast. But my
understanding is that a multicast router should be taking part in
IGMP. If there is a group member on the other side of the gateway, the
router should indicate it has interest in the group so traffic flows
to it. It can then forward the traffic between its multicast
interfaces, based on that interest.

	Andrew

Re: mv88e6320: Failed to forward PTP multicast

[Tobias Waldekranz]

On ons, maj 17, 2023 at 19:07, Andrew Lunn <andrew@lunn.ch> wrote:
>> Slightly unrelated to the original topic and probably completely
>> unrelated to Fabio's actual issue.
>> 
>> I'm completely inapt when it comes to IP multicast. Tobias, does the
>> fact that br0 have mcast_router 1 mean that the CPU port should be a
>> recipient of all multicast traffic, registered or unregistered?

If the bridge is the elected querier, then yes, br0 should receive _all_
IP multicast.  However, I believe that mcast_router=1 only means that
the bridge is allowed to take part in the election, if you have another
querier/router somewhere on the LAN, then that might be elected.

Assuming that br0 is the elected querier, Fabio is probably hitting a
long-standing issue with the MDB-implementation in the DSA/mv88e6xxx
layers.  Typically, drivers will ensure that router ports (and host
mrouter) receive _unknown_ multicast.  However, existing _registered_
groups still won't reach these ports.

In all hardware that I've seen, this must be handled by iterating over
all known (IP) groups in the MDB, ORing in the new router port (which
could be the CPU port in the host mrouter case).  Conversely, when new
groups are registered, you must also make sure to OR in all existing
router ports.

Deletions are even more perilous, because now each bit in a hardware MDB
(ATU) entry might have been set for either one or both of two reasons:

1. Because an IGMP/MLD report or a static configuration said so.
2. Because a router/querier is located behind it.

This means that you need to cache all this information in software to
know when it is safe to clear a bit in the hardware.

I have made a draft implementation of this, which I do not have access
to at the moment.  I'll poke around and see if I can find it.

> It is a long time since i did much with multicast. But my
> understanding is that a multicast router should be taking part in
> IGMP. If there is a group member on the other side of the gateway, the
> router should indicate it has interest in the group so traffic flows
> to it. It can then forward the traffic between its multicast
> interfaces, based on that interest.

Yes, that would have been the sane thing to do.  Indeed, if you're
running a protocol like PIM-SM, it could have been done that way.  Alas,
we also have flood-and-prune based protocols like DVMRP to deal with,
where the router has to flood all groups to all other routers until it
receives a prune message.  This is why, AFAIK, the standard says that a
router port needs to also receive unregistered groups.