From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jakub Kicinski Subject: Re: phys_port_id in switchdev mode? Date: Tue, 28 Aug 2018 20:43:51 +0200 Message-ID: <20180828204351.34fe457f@cakuba.netronome.com> References: <20180828200539.1c0fe607@cakuba.netronome.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: nick viljoen , "netdev@vger.kernel.org" To: Florian Fainelli , Or Gerlitz , Simon Horman , Andy Gospodarek , "mchan@broadcom.com" , Jiri Pirko , Alexander Duyck , Frederick Botha Return-path: Received: from mail-pg1-f174.google.com ([209.85.215.174]:36025 "EHLO mail-pg1-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726961AbeH1Wgz (ORCPT ); Tue, 28 Aug 2018 18:36:55 -0400 Received: by mail-pg1-f174.google.com with SMTP id d1-v6so1144758pgo.3 for ; Tue, 28 Aug 2018 11:43:59 -0700 (PDT) In-Reply-To: <20180828200539.1c0fe607@cakuba.netronome.com> Sender: netdev-owner@vger.kernel.org List-ID: Ugh, CC: netdev.. On Tue, 28 Aug 2018 20:05:39 +0200, Jakub Kicinski wrote: > Hi! > > I wonder if we can use phys_port_id in switchdev to group together > interfaces of a single PCI PF? Here is the problem: > > With a mix of PF and VF interfaces it gets increasingly difficult to > figure out which one corresponds to which PF. We can identify which > *representor* is which, by means of phys_port_name and devlink > flavours. But if the actual VF/PF interfaces are also present on the > same host, it gets confusing when one tries to identify the PF they > came from. Generally one has to resort of matching between PCI DBDF of > the PF and VFs or read relevant info out of ethtool -i. > > In multi host scenario this is particularly painful, as there seems to > be no immediately obvious way to match PCI interface ID of a card (0, > 1, 2, 3, 4...) to the DBDF we have connected. > > Another angle to this is legacy SR-IOV NDOs. User space picks a netdev > from /sys/bus/pci/$VF_DBDF/physfn/net/ to run the NDOs on in somehow > random manner, which means we have to provide those for all devices with > link to the PF (all reprs). And we have to link them (a) because it's > right (tm) and (b) to get correct naming. The only reliable way to make > user space (libvirt) choose the repr it should run the NDOs on (which is > IMHO the corresponding PF repr) is to set phys_port_id on actual VFs, > VF reprs, PFs and PF reprs to a value corresponding to the *PCI PF*, > not the external/Ethernet port when in switchdev mode. User space > should understand phys_port_id in this context, given it was originally > introduced for matching VFs to ports. > > I hope this explanation makes sense, and is correct. Please point out > errors in my understanding, any comments would be appreciated! :) > > Jiri? Or? Others?