Re: [PATCH net-next 2/8] devlink: add PF and VF port flavours

David Ahern
2019-02-28T16:24:38Z

On 2/27/19 3:42 PM, Jakub Kicinski wrote:
> On Wed, 27 Feb 2019 21:17:27 +0100, Jiri Pirko wrote:
>> Wed, Feb 27, 2019 at 06:23:26PM CET, jakub.kicinski@netronome.com wrote:
>>> On Wed, 27 Feb 2019 13:41:35 +0100, Jiri Pirko wrote:
>>>> Wed, Feb 27, 2019 at 01:23:27PM CET, jiri@resnulli.us wrote:
>>>>> Tue, Feb 26, 2019 at 07:24:30PM CET, jakub.kicinski@netronome.com wrote:
>>>>>> Current port flavours cover simple switches and DSA. Add PF
>>>>>> and VF flavours to cover "switchdev" SR-IOV NICs.
>>>>>>
>>>>>> Example devlink user space output:
>>>>>>
>>>>>> $ devlink port
>>>>>> pci/0000:82:00.0/0: type eth netdev p4p1 flavour physical
>>>>>> pci/0000:82:00.0/10000: type eth netdev eth0 flavour pcie_pf pf 0
>>>>>> pci/0000:82:00.0/10001: type eth netdev eth1 flavour pcie_vf pf 0 vf 0
>>>>>> pci/0000:82:00.0/10002: type eth netdev eth2 flavour pcie_vf pf 0 vf 1
>>>>>
>>>>> Wait a second, howcome pf and vfs have the same PCI address?
>>>>
>>>> Oh, I think you have these as eswitch port representors. Confusing...
>>>
>>> FWIW I don't like the word representor, its a port. We don't call
>>> physical ports "representors" even though from ASIC's point of view
>>> they are exactly the same.
>>
>> My point is, they are not PFs and VFs. We have to find a way to clearly
>> see what's what.
>
> Okay, so let me explain the way I see it, and you can explain your way
> or tell me where you disagree. Those devlink ports and netdevs are pf
> ports and vf ports, which most refer to as "representor". If one sends
> packets to the netdev indicated in DEVLINK_ATTR_PORT_NETDEV_*
> attributes they will _egress_ the switch from that port. For physical
> port that means going onto the Ethernet or IB wire. For PCIe it means
> getting DMAed over the PCIe link to host memory.
>
> There is a netdev construct on the host which is in charge of that
> host memory. Maybe we shall call that host netdev?
>
> (I said I don't like "representor" for the reason that people don't
> refer to the physical port as "representor" even though it has exactly
> the semantics we are following. This distinction between behaviour of
> physical and PCI ports is what leads to confusion, I think.)
>
> Let me bring out the moose :)
>
> HOST A || HOST B
> ||
> PF A | V | V | V | V || PF B | V | V | V
> |*F |*F |*F |*F ... || |*F |*F |*F ...
> *port A0 |*port A1 | 0 | 1 | 2 | 3 ||*port B0 |*port B1 | 0 | 1 | 2
> ||
> PCI Express link || PCI Express link
> \ \ \ | | | | | / / /
> \ \ \ | | | | | / / /
> /\ \______\______\'___|___|__________|_______'____/___/___/__ /\
> || |+PF0s0|+PF0s1 |+VF0|+VF1| ...| |+PF1s0|+PF1s1|+VF0|+VF1| ||
> i || |------ ------ ----- ---- ----|--- ------ ------ ---- ----| || i
> d n H || | <<========== | || d n H
> e s O || | ==========>> | || e s O
> v t S || | SR-IOV e-switch | || v t S
> l a T || | <<========== | || l a T
> i n || | ==========>> | || i n
> n c A || | ________ _________ ________ | || n c B
> k e || | |+Phys 0 |+Phys 1 |+Phys 2 | | || k e
> || \---------------------------------------------------------/ ||
> \/ | | | \/
> | | |
> || ||
> MAC 0 || MAC 1 || MAC 2
> || ||
>
> Things marked with + are devlink ports and have port (-repr-) netdevs
> (including physical ports).
> Things marked with * are host netdevs, don't have devlink ports.
>

That would a good update to the commit message or cover letter.