Changing devlink port flavor dynamically for DSA

[Florian Fainelli <f.fainelli@gmail.com>]

Hi all,

On a BCM7278 system, we have two ports of the switch: 5 and 8, that
connect to separate Ethernet MACs that the host/CPU can control. In
premise they are both interchangeable because the switch supports
configuring the management port to be either 5 or 8 and the Ethernet
MACs are two identical instances.

The Ethernet MACs are scheduled differently across the memory controller
(they have different bandwidth and priority allocations) so it is
desirable to select an Ethernet MAC capable of sustaining bandwidth and
latency for host networking. Our current (in the downstream kernel) use
case is to expose port 5 solely as a control end-point to the user and
leave it to the user how they wish to use the Ethernet MAC behind port
5. Some customers use it to bridge Wi-Fi traffic, some simply keep it
disabled. Port 5 of that switch does not make use of Broadcom tags in
that case, since ARL-based forwarding works just fine.

The current Device Tree representation that we have for that system
makes it possible for either port to be elected as the CPU port from a
DSA perspective as they both have an "ethernet" phandle property that
points to the appropriate Ethernet MAC node, because of that the DSA
framework treats them as CPU ports.

My current line of thinking is to permit a port to be configured as
either "cpu" or "user" flavor and do that through devlink. This can
create some challenges but hopefully this also paves the way for finally
supporting "multi-CPU port" configurations. I am thinking something like
this would be how I would like it to be configured:

# First configure port 8 as the new CPU port
devlink port set pci/0000:01:00.0/8 type cpu
# Now unmap port 5 from being a CPU port
devlink port set pci/0000:01:00.0/1 type eth

and this would do a simple "swap" of all user ports being now associated
with port 8, and no longer with port 5, thus permitting port 5 from
becoming a standard user port. Or maybe, we need to do this as an atomic
operation in order to avoid a switch being configured with no CPU port
anymore, so something like this instead:

devlink port set pci/0000:01:00.0/5 type eth mgmt pci/0000:01:00.0/8

The latter could also be used to define groups of ports within a switch
that has multiple CPU ports, e.g.:

# Ports 1 through 4 "bound" to CPU port 5:

for i in $(seq 0 3)
do
	devlink port set pci/0000:01:00.0/$i type eth mgmt pci/0000:01:00.0/5
done

# Ports 7 bound to CPU port 8:

devlink port set pci/0000:01:00.0/1 type eth mgmt pci/0000:01:00.0/8

Let me know what you think!

Thanks
-- 
Florian