RE: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports

Parav Pandit
2019-03-18T22:12:36Z

> -----Original Message-----
> From: Jakub Kicinski <jakub.kicinski@netronome.com>
> Sent: Monday, March 18, 2019 4:30 PM
> To: Parav Pandit <parav@mellanox.com>
> Cc: Jiri Pirko <jiri@resnulli.us>; Samudrala, Sridhar
> <sridhar.samudrala@intel.com>; davem@davemloft.net;
> netdev@vger.kernel.org; oss-drivers@netronome.com
> Subject: Re: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI
> ports
>
> On Mon, 18 Mar 2019 20:35:02 +0000, Parav Pandit wrote:
> > > > > > physical and pf has some overlapping definitions.
> > > > >
> > > > > What "overlapping definitions" do physical and PF have?
> > > > PF has physically user facing port.
> > >
> > > PF doesn't "have a user facing port" in switchdev mode.
> >
> > Physical port described in include/uapi/linux/devlink.h as
> > DEVLINK_PORT_FLAVOUR_PHYSICAL is not related to switchdev or legacy
> > mode.
>
> I said "PF doesn't ...", you're now talking about physical?
>
> > As the comment block describe it is 'any kind of port physical facing
> > user'.
>
> Are you saying PCI function is physical? Just because PF stands for Physical
> Function?
>
> Physical port in devlink means a port in the front panel where networking
> cable goes.
>
> > Current mlx5 driver doesn't expose ports as physical regardless of
> > switchdev/legacy mode.
>
> Today mlx5 doesn't expose devlink ports at all.
>
> > > It's a limitation of Mellanox HW that you have some strong
> > > association there.
> > Not sure why you keep saying that. Any code reference that I should
> > look at? Or maybe you can explain what is that limitation, because I
> > am not aware of any.
>
> NIC designs originating from traditional NICs were build as pipelines from PCI
> to wire or from wire to PCI. Reportedly it makes it hard to completely
> divorce the PCI PF from the wire port (physical port).
>
> Which is why you may think that "PF has physically user facing port".
>
> > > > And physical port in include/uapi/linux/devlink.h also describe
> > > > that.
> > >
> > > By "that" you must mean that the physical is a user facing port.
> >
> > Can you please describe the difference between 'PF port' and 'physical
> > port of include/uapi/linux/devlink.h'? I must have missed this crisp
> > definition in discussion between you and Jiri. I am in meantime
> > checking the thread.
>
> Perhaps start with the cover letter which includes an ASCII drawing?
>
> Using Mellanox nomenclature - PF port is a "representor" for the PF which
> may be on another Host (SmartNIC or multihost). It's pretty much the same
> thing as a VF port/"representor".
>
Yes. We are aligned here. :-)
I see your point, where in multi-host scenario, a physical port may be 1, but PF ports are 4, because of 4 PFs for 4 hosts.
(just an example of 4 hosts with their own mac address sharing 1 physical port).

When there is no multihost and one to one mapping between a PF and physical links,
there is some overlap between PF port and physical port attributes.
I believe, such overlap is fine as long as we have unique indices for the ports.

So I am ok to have flavours as physical/cpu/dsa/pf/vf/mdev/switchport.
(last 4 as new port flavours).

> Physical port is the hole on the panel of the adapter where cable goes.