Re: DSA: some questions regarding TX forwarding offload

Vladimir Oltean
2021-10-07T16:06:54Z

On Thu, Oct 07, 2021 at 02:15:23PM +0000, Alvin Šipraga wrote:
> First of all, I noticed that even with VLAN-unaware bridges, if I add
> VLANs to the bridge or ports, then my .port_vlan_add is called even
> though VLAN filtering is not enabled. That's OK and the switch will
> still receive untagged/priority-tagged frames without complaint.

There's more to that. The switch should receive and transmit any frames
regardless of VLAN tag, not just untagged and priority-tagged.

> But if I may ask, what is the point?

The point is simple. VLANs are to be used only while the bridge has VLAN
awareness turned on, however they can be added and deleted at any time.
When the bridge has VLAN awareness turned off, those VLANs are just
supposed to sit there, inactive.

ip link add br0 type bridge
ip link set swp0 master br0
bridge vlan add dev swp0 vid 100 # inactive..
ip link set br0 type bridge vlan_filtering 1 # active!
ip link set br0 type bridge vlan_filtering 0 # inactive again

Forget about switchdev, try it out with any interface, like a veth pair.
This is how the software bridge behaves, so this is how offloading ports
should behave too.

In VLAN-unaware mode, you can delete all bridge VLANs on all ports and
on the bridge device itself, and traffic will still work. So an
offloading port should follow suit.

> Even when programming a PVID on a given port of a VLAN-unaware bridge,
> the switch is not inserting 802.1Q tags on ingress frames that it
> forwards to the CPU port (tcpdump on the swpN interface still shows
> untagged frames).

The point of a PVID is not to insert 802.1Q tags in frames, it is to
classify untagged or null-tagged frames to a certain VLAN during the
ingress stage. The frame is associated with that VLAN as the packet
traverses the switch, gets used for the address table lookup,
selection of destination ports, etc. Inserting 802.1Q tags is a concern
at the egress stage of a bridge port. A packet received as untagged can
still be transmitted as untagged if the pvid of the ingress port is
egress-untagged on the egress port. There's no requirement at the
ingress stage for that packet to be tagged or not. It may even be
flooded to two egress ports, and be sent as tagged in one, but untagged
in the other.

I still feel the need to mention that the hardware PVID may or may not
be equal to the bridge's PVID, it can be different especially when VLAN
awareness in the software bridge is disabled (see above). Typically,
hardware always wants to classify packets to a VLAN. But considering
that the bridge may not have any VLAN to give you through switchdev, and
that your hardware still needs to operate => the only logical
consequence is that you need to manage the PVID committed to hardware
when the port is under a VLAN unaware bridge on your own.

> So I fail to see what capability is being offloaded to the switch
> hardware with that call to .port_vlan_add for VLAN-unaware bridges.

The capability is that you don't lag behind the bridge's state when
VLAN awareness is finally turned on. See the commit messages in the git
blame surrounding the ds->configure_vlan_while_not_filtering.

> Or _should_ the switch insert a tag for ports with a PVID?

I don't understand this question, sorry. Insert a tag towards which port?
Above you talked about the CPU port. It is the switch driver's business
whether it configures the CPU port as a VLAN trunk or not. Several
drivers do. Those drivers also set ds->untag_bridge_pvid = true, to
ensure that DSA strips those VLANs before presenting the packets further
to the network stack (hint: the network stack would like to see the
packet as it was on the wire, not with VLANs getting added where they
didn't exist). Rule of thumb though: if you don't know that you need
this feature, you don't need it.

> The second concern I have is regarding learning. We agreed now that all
> of the VLANs we add to the switch should have IVL_EN=1, meaning that the
> switch will learn in a VLAN-aware manner.

Actually the discussion was about FDB isolation and TX forwarding
offload, not about how to make a port VLAN-unaware.

> If the switch receives a frame with MAC SA=00:00:aa:aa:aa:aa, VID=1 on
> port N of a VLAN-unaware bridge, it will still have to flood a frame
> from the CPU with MAC DA=00:00:aa:aa:aa:aa if the frame doesn't have
> VID=1, rather than forwarding it directly to port N. So although the
> ports of this VLAN-unaware bridge are respecting the rules of
> VLAN-(un)awareness as I read them in the switchdev documentation,

Are they? I don't think so. It sounds like the switch classifies the
packet in a different FID on ingress from the user port compared to the
ingress from the CPU port, depending on whether that packet has a VLAN
header or not. That's quite the opposite of "VLAN-unaware".

> the hardware's learning process is still VLAN-aware and may cause more
> flooding than expected.
> My question to you is: is this acceptable behaviour?

Nope.

> To work around this, I guess I can set vlan_filtering_is_global=true. Or
> I can do some bookkeeping in the driver of VLAN settings and only
> program them when enabling VLAN filtering, which is a bit of a pain.
> Hoping you can help clear this up for me - I have a feeling I have
> misunderstood something.

You're asking me as if I knew the hardware, yet I haven't seen the
documentation even once :)

Other switches have an actual VLAN _awareness_ setting. Something like
this (not the diagram of an actual switch, just an oversimplification/
generalization):

+----------------------------------+
| packet ingresses a port |
+----------------------------------+
|
|
v
+----------------------------------+
| gets classified to the port pvid |
+----------------------------------+
|
|
v / \
/ \ / \
/ \ / \ +--------------------------+
/ \ / port \ | |
/ packet \ yes / is \ yes | packet gets reclassified |
/ has \--------->/ VLAN \-------->+ to the VID from the |
\ 802.1Q / \ aware? / | 802.1Q header |
\ header? / \ / | |
\ / \ / +--------------------------+
\ / \ / |
\ / \ / |
\ / \ / |
| no | no |
| | |
+----------------------+-----------------------------+
|
|
v / \ / \
/ \ / \ / \
/ \ / \ / \
/ is \ / is \ / is \
/classified yes / ingress \ no / ingress \ no
/ VLAN \----------------->/ port a \----->/ port in \----> drop
\ in the / \ member of / \ fallback /
\ VLAN / \ this / \ mode? /
\table? / \ VLAN? / \ /
\ / \ / \ /
\ / \ / \ /
\ / \ / \ /
| no | yes | yes
| | |
| +<-----------------+
v |
/ \ v
/ \ +--------------------------+
/ is \ | |
yes / VLAN \ | read the VLAN table entry|
drop <-------/ ingress \ | and select the FID |
\ filtering / | |
\ enabled?/ | |
\ / +--------------------------+
\ / |
\ / |
\ / |
| no |
| |
v |
+--------------------+ |
| use a port based | |
| default FID | |
+--------------------+ |
| |
+------------------------------+
|
v
+----------------------+
| FDB lookup using FID |
+----------------------+

What the Linux bridge understands by "VLAN filtering" is actually a
mixture of many things, it doesn't really distinguish between ingress
filtering and awareness, it doesn't have support for fallback mode,
it has no idea about FIDs, things like that.
But one thing is certain: when in VLAN-unaware mode, it learns FDB
entries in VID 0, and that is its encoding for "match regardless of VLAN
while VLAN-unaware". But the logic above may still run in your hardware,
you have to configure your hardware to treat switchdev FDB objects in
that way. You may need to remap VID 0 from those entries when you
offload them, but that is what it means.

So you can see that certain switches allow you to say: even if a packet
has an 802.1Q header, I want to keep the classified VLAN equal to the
ingress port's PVID. There's a reason why switches are made this way, it
simplifies things. I've no idea if your switch can do that, I did look
at your rtl8365mb_port_vlan_filtering() implementation and I did notice
that you only disable ingress filtering.

If you cannot inactivate your VLANs present in the VLAN table, then
you'll have to live with the fact that you'll need, in VLAN-unaware
mode, to configure them all for SVL, and point them all towards the same FID.

So if a packet is untagged, it is classified to the PVID, which is
mapped in the VLAN table to, say, FID 1.

If a packet is tagged with VID 100, which is missing from the VLAN
table, it will get mapped to the port-based default FID, which is 1.

If a packet is tagged with VID 101, which is present in the VLAN table,
that will need to be mapped to FID 1.

So all I'm saying is that this is one way in which VLAN-unaware bridging
could be obtained, given the circumstances. It is the best? No, because
if you configure your VLAN table entries as SVL, you can't simply do
that only for the ports which are under a VLAN-unaware bridge. The VLAN
table is global for all ports. So, yes, if one bridge is VLAN-unaware,
all bridges will have to be. We haven't really had this case before, but
I suppose ds->vlan_filtering_is_global = true can deal with it.

Anyway, it is quite a suboptimal approach, but lacking a crystal ball,
I don't know what else to tell you, I hope that you find more switch
settings which you may have dismissed before.