RE: [EXTERNAL] Re: [net-next PATCH v7 00/10] Introduce RVU representors

Geethasowjanya Akula
2024-07-04T13:48:43Z

>-----Original Message-----
>From: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
>Sent: Thursday, July 4, 2024 12:45 PM
>To: Geethasowjanya Akula <gakula@marvell.com>
>Cc: netdev@vger.kernel.org; linux-kernel@vger.kernel.org; kuba@kernel.org;
>davem@davemloft.net; pabeni@redhat.com; edumazet@google.com; Sunil
>Kovvuri Goutham <sgoutham@marvell.com>; Subbaraya Sundeep Bhatta
><sbhatta@marvell.com>; Hariprasad Kelam <hkelam@marvell.com>
>Subject: Re: [EXTERNAL] Re: [net-next PATCH v7 00/10] Introduce RVU
>representors
>
>On Wed, Jul 03, 2024 at 02: 34: 03PM +0000, Geethasowjanya Akula wrote: > >
>> >-----Original Message----- > >From: Michal Swiatkowski
><michal. swiatkowski@ linux. intel. com> > >Sent: Wednesday, July 3, 2024 4: 14
>PM
>On Wed, Jul 03, 2024 at 02:34:03PM +0000, Geethasowjanya Akula wrote:
>>
>>
>> >-----Original Message-----
>> >From: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
>> >Sent: Wednesday, July 3, 2024 4:14 PM
>> >To: Geethasowjanya Akula <gakula@marvell.com>
>> >Cc: netdev@vger.kernel.org; linux-kernel@vger.kernel.org;
>> >kuba@kernel.org; davem@davemloft.net; pabeni@redhat.com;
>> >edumazet@google.com; Sunil Kovvuri Goutham
><sgoutham@marvell.com>;
>> >Subbaraya Sundeep Bhatta <sbhatta@marvell.com>; Hariprasad Kelam
>> ><hkelam@marvell.com>
>> >Subject: [EXTERNAL] Re: [net-next PATCH v7 00/10] Introduce RVU
>> >representors On Fri, Jun 28, 2024 at 07:05:07PM +0530, Geetha sowjanya
>wrote:
>> >> This series adds representor support for each rvu devices.
>> >> When switchdev mode is enabled, representor netdev is registered
>> >> for each rvu device. In implementation of representor model, one
>> >> NIX HW LF with multiple SQ and RQ is reserved, where each RQ and SQ
>> >> of the LF are mapped to a representor. A loopback channel is
>> >> reserved to support packet path between representors and VFs.
>> >> CN10K silicon supports 2 types of MACs, RPM and SDP. This patch set
>> >> adds representor support for both RPM and SDP MAC interfaces.
>> >>
>> >> - Patch 1: Refactors and exports the shared service functions.
>> >> - Patch 2: Implements basic representor driver.
>> >> - Patch 3: Add devlink support to create representor netdevs that
>> >> can be used to manage VFs.
>> >> - Patch 4: Implements basec netdev_ndo_ops.
>> >> - Patch 5: Installs tcam rules to route packets between representor and
>> >> VFs.
>> >> - Patch 6: Enables fetching VF stats via representor interface
>> >> - Patch 7: Adds support to sync link state between representors and VFs .
>> >> - Patch 8: Enables configuring VF MTU via representor netdevs.
>> >> - Patch 9: Add representors for sdp MAC.
>> >> - Patch 10: Add devlink port support.
>> >>
>> >> Command to create VF representor
>> >> #devlink dev eswitch set pci/0002:1c:00.0 mode switchdev VF
>> >> representors are created for each VF when switch mode is set
>> >> switchdev on representor PCI device
>> >
>> >Does it mean that VFs needs to be created before going to switchdev
>> >mode? (in legacy mode). Keep in mind that in both mellanox and ice
>> >driver assume that VFs are created after chaning mode to switchdev
>> >(mode can't be changed if VFs).
>> No. RVU representor driver implementation is similar to mellanox and ice
>drivers.
>> It assumes that VF gets created only after switchdev mode is enabled.
>> Sorry, if above commit description is confusing. Will rewrite it.
>>
>
>Ok, but why the rvu_rep_create() is called in switching mode to switchdev
>function? In this function you are creating netdevs, only for PF representor? It
>looks like it doesn't called from other context in this patchset, so where the
>port representor netdevs for VFs are created?
>
RVU representors for PF/VFs are created when switchdev mode is set, similar to the bnxt and nfp drivers.
rvu_rep_create() will create representors based on rep_cnt (which include both PFs and VFs count)

Thanks,
Geetha.

>Thanks,
>Michal
>
>> >
>> >Different order can be problematic. For example (AFAIK) kubernetes
>> >scripts for switchdev assume that first is switching to switchdev and
>> >VFs creation is done after that.
>> >
>> >Thanks,
>> >Michal
>> >
>> >> # devlink dev eswitch set pci/0002:1c:00.0 mode switchdev # ip
>> >> link show
>> >> 25: r0p1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>mode
>> >DEFAULT group default qlen 1000
>> >> link/ether 32:0f:0f:f0:60:f1 brd ff:ff:ff:ff:ff:ff
>> >> 26: r1p1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>mode
>> >DEFAULT group default qlen 1000
>> >> link/ether 3e:5d:9a:4d:e7:7b brd ff:ff:ff:ff:ff:ff
>> >>
>> >> #devlink dev
>> >> pci/0002:01:00.0
>> >> pci/0002:02:00.0
>> >> pci/0002:03:00.0
>> >> pci/0002:04:00.0
>> >> pci/0002:05:00.0
>> >> pci/0002:06:00.0
>> >> pci/0002:07:00.0
>> >>
>> >> ~# devlink port
>> >> pci/0002:1c:00.0/0: type eth netdev r0p1v0 flavour pcipf controller
>> >> 0 pfnum 1 vfnum 0 external false splittable false
>> >> pci/0002:1c:00.0/1: type eth netdev r1p1v1 flavour pcivf controller
>> >> 0 pfnum 1 vfnum 1 external false splittable false
>> >> pci/0002:1c:00.0/2: type eth netdev r2p1v2 flavour pcivf controller
>> >> 0 pfnum 1 vfnum 2 external false splittable false
>> >> pci/0002:1c:00.0/3: type eth netdev r3p1v3 flavour pcivf controller
>> >> 0 pfnum 1 vfnum 3 external false splittable false
>> >>
>> >> -----------
>> >> v1-v2:
>> >> -Fixed build warnings.
>> >> -Address review comments provided by "Kalesh Anakkur Purayil".
>> >>
>> >> v2-v3:
>> >> - Used extack for error messages.
>> >> - As suggested reworked commit messages.
>> >> - Fixed sparse warning.
>> >>
>> >> v3-v4:
>> >> - Patch 2 & 3: Fixed coccinelle reported warnings.
>> >> - Patch 10: Added devlink port support.
>> >>
>> >> v4-v5:
>> >> - Patch 3: Removed devm_* usage in rvu_rep_create()
>> >> - Patch 3: Fixed build warnings.
>> >>
>> >> v5-v6:
>> >> - Addressed review comments provided by "Simon Horman".
>> >> - Added review tag.
>> >>
>> >> v6-v7:
>> >> - Rebased on top net-next branch.
>> >>
>> >> Geetha sowjanya (10):
>> >> octeontx2-pf: Refactoring RVU driver
>> >> octeontx2-pf: RVU representor driver
>> >> octeontx2-pf: Create representor netdev
>> >> octeontx2-pf: Add basic net_device_ops
>> >> octeontx2-af: Add packet path between representor and VF
>> >> octeontx2-pf: Get VF stats via representor
>> >> octeontx2-pf: Add support to sync link state between representor and
>> >> VFs
>> >> octeontx2-pf: Configure VF mtu via representor
>> >> octeontx2-pf: Add representors for sdp MAC
>> >> octeontx2-pf: Add devlink port support
>> >>
>> >> .../net/ethernet/marvell/octeontx2/Kconfig | 8 +
>> >> .../ethernet/marvell/octeontx2/af/Makefile | 3 +-
>> >> .../ethernet/marvell/octeontx2/af/common.h | 2 +
>> >> .../net/ethernet/marvell/octeontx2/af/mbox.h | 74 ++
>> >> .../net/ethernet/marvell/octeontx2/af/npc.h | 1 +
>> >> .../net/ethernet/marvell/octeontx2/af/rvu.c | 11 +
>> >> .../net/ethernet/marvell/octeontx2/af/rvu.h | 30 +-
>> >> .../marvell/octeontx2/af/rvu_debugfs.c | 27 -
>> >> .../marvell/octeontx2/af/rvu_devlink.c | 6 +
>> >> .../ethernet/marvell/octeontx2/af/rvu_nix.c | 81 ++-
>> >> .../marvell/octeontx2/af/rvu_npc_fs.c | 5 +
>> >> .../ethernet/marvell/octeontx2/af/rvu_reg.h | 4 +
>> >> .../ethernet/marvell/octeontx2/af/rvu_rep.c | 464 ++++++++++++
>> >> .../marvell/octeontx2/af/rvu_struct.h | 26 +
>> >> .../marvell/octeontx2/af/rvu_switch.c | 20 +-
>> >> .../ethernet/marvell/octeontx2/nic/Makefile | 2 +
>> >> .../ethernet/marvell/octeontx2/nic/cn10k.c | 4 +-
>> >> .../ethernet/marvell/octeontx2/nic/cn10k.h | 2 +-
>> >> .../marvell/octeontx2/nic/otx2_common.c | 56 +-
>> >> .../marvell/octeontx2/nic/otx2_common.h | 84 ++-
>> >> .../marvell/octeontx2/nic/otx2_devlink.c | 49 ++
>> >> .../ethernet/marvell/octeontx2/nic/otx2_pf.c | 305 +++++---
>> >> .../marvell/octeontx2/nic/otx2_txrx.c | 38 +-
>> >> .../marvell/octeontx2/nic/otx2_txrx.h | 3 +-
>> >> .../ethernet/marvell/octeontx2/nic/otx2_vf.c | 19 +-
>> >> .../net/ethernet/marvell/octeontx2/nic/rep.c | 684
>> >> ++++++++++++++++++ .../net/ethernet/marvell/octeontx2/nic/rep.h |
>> >> 53 ++
>> >> 27 files changed, 1834 insertions(+), 227 deletions(-) create
>> >> mode
>> >> 100644 drivers/net/ethernet/marvell/octeontx2/af/rvu_rep.c
>> >> create mode 100644
>> >> drivers/net/ethernet/marvell/octeontx2/nic/rep.c
>> >> create mode 100644
>> >> drivers/net/ethernet/marvell/octeontx2/nic/rep.h
>> >>
>> >> --
>> >> 2.25.1
>> >>