From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B691FC43381 for ; Wed, 27 Feb 2019 22:42:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7999B2133D for ; Wed, 27 Feb 2019 22:42:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=netronome-com.20150623.gappssmtp.com header.i=@netronome-com.20150623.gappssmtp.com header.b="FT7bgfud" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730189AbfB0Wmr (ORCPT ); Wed, 27 Feb 2019 17:42:47 -0500 Received: from mail-qk1-f195.google.com ([209.85.222.195]:34968 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730102AbfB0Wmq (ORCPT ); Wed, 27 Feb 2019 17:42:46 -0500 Received: by mail-qk1-f195.google.com with SMTP id z13so10952415qki.2 for ; Wed, 27 Feb 2019 14:42:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :organization:mime-version:content-transfer-encoding; bh=LmSbMzmW4gp9o0cSADCAOQBMa9vgHE/CAN1lBM1G1zI=; b=FT7bgfudEXE29XkyQdUHFh1jtn/9sltW4W8vcYCt2cIwhd3DbDMnwYIAP6hGTV62h/ K8/vrtf/ZgNlY1rcE8+6g0WUWEfJEob2hR27NC2EJNmDoB8bw+QyDtf02VN/2AMsTWU6 /BPeZpDD0WeVq0aAXbaNw6yTM8MzYeCITgMCj6AleiKo6+VFTZKYbDKfCDkaUzyX+LQe So1JVnt5AiY62U7xp9kkc+ioPXkAnX3/zW2R1Z9b1XOoty3pGjqQCZOYRXMRpk7CQ2xP WrhwBk+fUCI/vAD4inx8xzkFFPNkQJgThnfabJ+QW/M74E8pY92+j3/o/ooKa6Ol4Ypq 56BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=LmSbMzmW4gp9o0cSADCAOQBMa9vgHE/CAN1lBM1G1zI=; b=GEfcGMqScSNoDwPGm583OH2ltDKfG9VVWmxHrtoeAL6p35LVWeJxMZo6YWI+zuddrr gxivQt5MaFg3cH8OaNEbWrTTeLW9kdAFfxtqhmctnaQ1Pz6s91jDqiWWVZd/bhHhEyRM HhsIC9WHQRsd4/f/FTOM/pD0wJvD1fel3MVELuD3xLYEVbTMUrGqXBDgPg7ysbw+Kb0X ErIQV/UPDgJB9j3aOsRHngKTZM2KCLeYhFk1ABShSI36+d4lp8FZxkYCrnwS+OL29PCL 5Eq/WM5FOvOo3mzcI6433IYf5oX3dCsUe/K3DRsPEEt8pxe7B99WaSW27uP9u4BknTcW AtVw== X-Gm-Message-State: AHQUAubwz2r+WGLZvP/nQ0Cuqy3sfZFsj3WaMenjLAz/xEHmxNO0nqW6 gYBfB/eLedq6YZcVDHHakSp3OA== X-Google-Smtp-Source: AHgI3IZwDuJzAf5Mv9oqFypomgusxDGv82C9wydFL6rSZzhcJRNP+O9V44m7JlWBeu7qvogr9+sCzA== X-Received: by 2002:a05:620a:14b2:: with SMTP id x18mr4084682qkj.82.1551307365246; Wed, 27 Feb 2019 14:42:45 -0800 (PST) Received: from cakuba.netronome.com ([66.60.152.14]) by smtp.gmail.com with ESMTPSA id r47sm10982562qtc.48.2019.02.27.14.42.44 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 27 Feb 2019 14:42:45 -0800 (PST) Date: Wed, 27 Feb 2019 14:42:39 -0800 From: Jakub Kicinski To: Jiri Pirko Cc: davem@davemloft.net, oss-drivers@netronome.com, netdev@vger.kernel.org Subject: Re: [PATCH net-next 2/8] devlink: add PF and VF port flavours Message-ID: <20190227144239.55481885@cakuba.netronome.com> In-Reply-To: <20190227201727.GA2042@nanopsycho> References: <20190226182436.23811-1-jakub.kicinski@netronome.com> <20190226182436.23811-3-jakub.kicinski@netronome.com> <20190227122327.GA10571@nanopsycho> <20190227124135.GB10571@nanopsycho> <20190227092326.5abf417b@cakuba.netronome.com> <20190227201727.GA2042@nanopsycho> Organization: Netronome Systems, Ltd. MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Wed, 27 Feb 2019 21:17:27 +0100, Jiri Pirko wrote: > Wed, Feb 27, 2019 at 06:23:26PM CET, jakub.kicinski@netronome.com wrote: > >On Wed, 27 Feb 2019 13:41:35 +0100, Jiri Pirko wrote: > >> Wed, Feb 27, 2019 at 01:23:27PM CET, jiri@resnulli.us wrote: > >> >Tue, Feb 26, 2019 at 07:24:30PM CET, jakub.kicinski@netronome.com wrote: > >> >>Current port flavours cover simple switches and DSA. Add PF > >> >>and VF flavours to cover "switchdev" SR-IOV NICs. > >> >> > >> >>Example devlink user space output: > >> >> > >> >>$ devlink port > >> >>pci/0000:82:00.0/0: type eth netdev p4p1 flavour physical > >> >>pci/0000:82:00.0/10000: type eth netdev eth0 flavour pcie_pf pf 0 > >> >>pci/0000:82:00.0/10001: type eth netdev eth1 flavour pcie_vf pf 0 vf 0 > >> >>pci/0000:82:00.0/10002: type eth netdev eth2 flavour pcie_vf pf 0 vf 1 > >> > > >> >Wait a second, howcome pf and vfs have the same PCI address? > >> > >> Oh, I think you have these as eswitch port representors. Confusing... > > > >FWIW I don't like the word representor, its a port. We don't call > >physical ports "representors" even though from ASIC's point of view > >they are exactly the same. > > My point is, they are not PFs and VFs. We have to find a way to clearly > see what's what. Okay, so let me explain the way I see it, and you can explain your way or tell me where you disagree. Those devlink ports and netdevs are pf ports and vf ports, which most refer to as "representor". If one sends packets to the netdev indicated in DEVLINK_ATTR_PORT_NETDEV_* attributes they will _egress_ the switch from that port. For physical port that means going onto the Ethernet or IB wire. For PCIe it means getting DMAed over the PCIe link to host memory. There is a netdev construct on the host which is in charge of that host memory. Maybe we shall call that host netdev? (I said I don't like "representor" for the reason that people don't refer to the physical port as "representor" even though it has exactly the semantics we are following. This distinction between behaviour of physical and PCI ports is what leads to confusion, I think.) Let me bring out the moose :) HOST A || HOST B || PF A | V | V | V | V || PF B | V | V | V |*F |*F |*F |*F ... || |*F |*F |*F ... *port A0 |*port A1 | 0 | 1 | 2 | 3 ||*port B0 |*port B1 | 0 | 1 | 2 || PCI Express link || PCI Express link \ \ \ | | | | | / / / \ \ \ | | | | | / / / /\ \______\______\'___|___|__________|_______'____/___/___/__ /\ || |+PF0s0|+PF0s1 |+VF0|+VF1| ...| |+PF1s0|+PF1s1|+VF0|+VF1| || i || |------ ------ ----- ---- ----|--- ------ ------ ---- ----| || i d n H || | <<========== | || d n H e s O || | ==========>> | || e s O v t S || | SR-IOV e-switch | || v t S l a T || | <<========== | || l a T i n || | ==========>> | || i n n c A || | ________ _________ ________ | || n c B k e || | |+Phys 0 |+Phys 1 |+Phys 2 | | || k e || \---------------------------------------------------------/ || \/ | | | \/ | | | || || MAC 0 || MAC 1 || MAC 2 || || Things marked with + are devlink ports and have port (-repr-) netdevs (including physical ports). Things marked with * are host netdevs, don't have devlink ports.