Re: [BUG] mlx5: VLAN-aware bridge drops all traffic in legacy eswitch mode without promiscuous

[Dragos Tatulea <dtatulea@nvidia.com>]

Hi,

On 24.04.26 13:07, bryan wrote:
> Good day,
>
> I wanted to check whether there is an open bug report or known fix in
> progress for an issue that has been affecting mlx5 users (specifically
> ConnectX-4 Lx, but likely broader from what I have seen other
> reporting) since at least 2021:
>
> When an mlx5 interface is added as a port to a VLAN-aware Linux bridge
> (bridge-vlan-aware yes / vlan_filtering 1) in legacy eswitch mode, all
> traffic stops passing through the bridge. Both tagged and untagged
> traffic is affected. The same configuration works correctly with non-
> mlx5 NICs (tested Intel, Chelsio cards).
>
Is this even with one vlan? I ran a flow on a CX4LX pair with one vlan
and vlan_filtering set and traffic seems to be flowing normally.
Something like:

# IFACE=eth2
# VID=100
# ip link add br0 type bridge vlan_filtering 1
# ip link set "$IFACE" master br0
# bridge vlan add vid "$VID" dev "$IFACE"
# bridge vlan add vid "$VID" dev br0 self
# ip link add link br0 name "br0.$VID" type vlan id "$VID"
# ip addr add 10.0.0.1/24 dev br0
# ip addr add "10.0.$VID.1/24" dev "br0.$VID"
# ip link set "$IFACE" up
# ip link set br0 up
# ip link set "br0.$VID" up

From the other side where I have a similar setup I can ping
br0.100.

Tested on a CX4LX with FW version 28.48.1000 and kernel 6.18.
eth2 is a PF in legacy switchdev mode.

> [...]
> This is well documented in community forums but does not appear to have
> been formally reported to netdev that I have been able to find. My
> apologies in advance if this has been reported and I wasn't able to
> locate it. Here are a couple of forum examples where this is discussed
> among other affected users:
>
> - NVIDIA Developer Forum (opened 2021, unresolved):
>
> https://forums.developer.nvidia.com/t/vlan-aware-linux-bridging-is-not-functional-on-connectx4lx-card-unless-manually-put-in-promiscuous-mode/206083
>
> - Proxmox Forum thread (2023, ongoing):
>
> https://forum.proxmox.com/threads/mellanox-connectx-4-lx-and-brigde-vlan-aware-on-proxmox-8-0-1.130902/
>
> - Community writeup with analysis:
>   https://www.apalrd.net/posts/2023/tip_mellanox/
>
This last link seems the only one that provides some extra data. From it
I can see that the amount of VLAN ids > what the FW supports. This could
result in loss of traffic for the vlan ids > 512. Do you also see in
your dmesg these kinds of errors:

mlx5_core 0000:19:00.1: mlx5e_vport_context_update_vlans:179:(pid 13470): netdev vlans list size (4080) > (512) max vport list size, some vlans will be dropped

This is not a bug, simply a limit being reached.

> Has anyone bisected this or is there a fix already in progress that I
> did not find? This affects a fairly common hypervisor configuration
> (VLAN-aware bridge for VM networking) and the workarounds are not
> conducive to production use.
>
Could you provide a short repro script for this. Not being able to
reproduce the issue makes it hard to check :).

Thanks,
Dragos