Re: Changing devlink port flavor dynamically for DSA

[Florian Fainelli <f.fainelli@gmail.com>]

On 4/8/2020 12:51 PM, Vladimir Oltean wrote:
> Hi Florian,
> 
> On Sun, 5 Apr 2020 at 23:42, Florian Fainelli <f.fainelli@gmail.com> wrote:
>>
>> Hi all,
>>
>> On a BCM7278 system, we have two ports of the switch: 5 and 8, that
>> connect to separate Ethernet MACs that the host/CPU can control. In
>> premise they are both interchangeable because the switch supports
>> configuring the management port to be either 5 or 8 and the Ethernet
>> MACs are two identical instances.
>>
>> The Ethernet MACs are scheduled differently across the memory controller
>> (they have different bandwidth and priority allocations) so it is
>> desirable to select an Ethernet MAC capable of sustaining bandwidth and
>> latency for host networking. Our current (in the downstream kernel) use
>> case is to expose port 5 solely as a control end-point to the user and
>> leave it to the user how they wish to use the Ethernet MAC behind port
>> 5. Some customers use it to bridge Wi-Fi traffic, some simply keep it
>> disabled. Port 5 of that switch does not make use of Broadcom tags in
>> that case, since ARL-based forwarding works just fine.
>>
>> The current Device Tree representation that we have for that system
>> makes it possible for either port to be elected as the CPU port from a
>> DSA perspective as they both have an "ethernet" phandle property that
>> points to the appropriate Ethernet MAC node, because of that the DSA
>> framework treats them as CPU ports.
>>
>> My current line of thinking is to permit a port to be configured as
>> either "cpu" or "user" flavor and do that through devlink. This can
>> create some challenges but hopefully this also paves the way for finally
>> supporting "multi-CPU port" configurations. I am thinking something like
>> this would be how I would like it to be configured:
>>
>> # First configure port 8 as the new CPU port
>> devlink port set pci/0000:01:00.0/8 type cpu
>> # Now unmap port 5 from being a CPU port
>> devlink port set pci/0000:01:00.0/1 type eth
>>
>> and this would do a simple "swap" of all user ports being now associated
>> with port 8, and no longer with port 5, thus permitting port 5 from
>> becoming a standard user port. Or maybe, we need to do this as an atomic
>> operation in order to avoid a switch being configured with no CPU port
>> anymore, so something like this instead:
>>
>> devlink port set pci/0000:01:00.0/5 type eth mgmt pci/0000:01:00.0/8
>>
>> The latter could also be used to define groups of ports within a switch
>> that has multiple CPU ports, e.g.:
>>
>> # Ports 1 through 4 "bound" to CPU port 5:
>>
>> for i in $(seq 0 3)
>> do
>>         devlink port set pci/0000:01:00.0/$i type eth mgmt pci/0000:01:00.0/5
>> done
>>
>> # Ports 7 bound to CPU port 8:
>>
>> devlink port set pci/0000:01:00.0/1 type eth mgmt pci/0000:01:00.0/8
>>
>> Let me know what you think!
>>
>> Thanks
>> --
>> Florian
> 
> What is missing from your argumentation is what would the new devlink
> mechanism of changing the CPU port bring for your particular use case.
> I mean you can already remove the "ethernet" device tree property from
> port 5 and end up exactly with the configuration that you want, no?

That's what I do in our downstream tree for now, should I submit this
upstream? I doubt it would be accepted.
-- 
Florian