Re: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports

[Jiri Pirko <jiri@resnulli.us>]

Wed, Mar 20, 2019 at 09:22:57PM CET, jakub.kicinski@netronome.com wrote:
>On Wed, 20 Mar 2019 18:24:15 +0000, Parav Pandit wrote:
>> Hi Jiri, Jakub, Samudrala Sridhar,
>> > > > > > And physical port in include/uapi/linux/devlink.h also describe
>> > > > > > that.  
>> > > > >
>> > > > > By "that" you must mean that the physical is a user facing port.  
>> > > >
>> > > > Can you please describe the difference between 'PF port' and
>> > > > 'physical port of include/uapi/linux/devlink.h'? I must have missed
>> > > > this crisp definition in discussion between you and Jiri. I am in
>> > > > meantime checking the thread.  
>> > >
>> > > Perhaps start with the cover letter which includes an ASCII drawing?
>> > >
>> > > Using Mellanox nomenclature - PF port is a "representor" for the PF
>> > > which may be on another Host (SmartNIC or multihost).  It's pretty
>> > > much the same thing as a VF port/"representor".
>> > >  
>> > Yes. We are aligned here. :-)
>> > I see your point, where in multi-host scenario, a physical port may be 1, but
>> > PF ports are 4, because of 4 PFs for 4 hosts.
>> > (just an example of 4 hosts with their own mac address sharing 1 physical
>> > port).
>> > 
>> > When there is no multihost and one to one mapping between a PF and
>> > physical links, there is some overlap between PF port and physical port
>> > attributes.
>> > I believe, such overlap is fine as long as we have unique indices for the ports.
>> > 
>> > So I am ok to have flavours as physical/cpu/dsa/pf/vf/mdev/switchport.
>> > (last 4 as new port flavours).
>> >   
>> > > Physical port is the hole on the panel of the adapter where cable goes.  
>> 
>> So my take away from above discussion are:
>> 1. Following new port flavours should be added pci_pf/pci_vf/mdev/switchport.
>> a. Switchport indicates port on the eswitch. Normally this port has rep-netdev attached to it.
>
>I don't understand the "switchport".  Surely physical ports are also
>attached to the eswitch?  And one of the main purpose of adding the
>pci_pf/pci_vf flavours was to generate phys_port_name for the port
>netdevs.
>
>Please don't use the term representor if possible.  Representor for
>most developers describes the way the netdev is implemented in the
>driver, so for Mellanox and Netronome different ports will be
>representors and non-representors.  That's why I prefer port netdev
>(attached to eswitch, has switch_id) and host netdev (PF/VF netdev,
>vNIC, VSI, etc).
>
>> b. host side port flavours are pci_pf/pci_vf/mdev which may be connected to switchport
>
>See above, pci_pf/pci_vf are needed for phys_port_name generation.

Yep, that makes sense.


>
>> 2. host side port flavours are not limited to Ethernet, as it is for devlink's port instance.
>> 
>> 3. Each port is continue to be accessed using unique port index.
>> 
>> 4. host side ports and switchport are control objects.
>> a. switch side ports reside where current eswitch object of devlink instance reside
>> b. for a given VF/PF/mdev such host side ports may be in hypervisor or VM or both 
>> depending on the privilege
>> 
>> 5. eth.mac_address, rdma.port_guid can be programmed at 
>> host port flavours by extending as $ devlink port param set...
>> (similar to devlink dev param set)
>
>You can keep restating that's your position, but I have *not* conceded
>to that.

I'm also not convinced that host dummy ports are good idea to hold
these.


>
>> 6. more host port params can be added in future when user need arise
>> 
>> 7. rep-netdev continue to be eswitch (switchport) representor at the switch side.
>> a. Hence rep-netdev cannot be used for programming host port's parameters.
>> 
>> 8. eswitch devlink instance knows when VF/PF/mdev's switchport are created/removed.
>> Hence, those will be created/deleted by eswitch.
>> Similarly for host port flavours too.
>> 
>> Does it look fine? Did I miss something?
>> We would like to progress on incremental patches for item-4 and 
>> any prep work needed to reach to item-4.