From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD554C43381 for ; Wed, 13 Mar 2019 06:22:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 95F6B217F5 for ; Wed, 13 Mar 2019 06:22:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=resnulli-us.20150623.gappssmtp.com header.i=@resnulli-us.20150623.gappssmtp.com header.b="C8AL8SUu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726510AbfCMGRY (ORCPT ); Wed, 13 Mar 2019 02:17:24 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:51808 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725773AbfCMGRY (ORCPT ); Wed, 13 Mar 2019 02:17:24 -0400 Received: by mail-wm1-f65.google.com with SMTP id n19so582494wmi.1 for ; Tue, 12 Mar 2019 23:17:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=resnulli-us.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=VV8pbhNKDGo3Gfnhi3Ncmv9nwYE6+QrLgqukheL3+Ws=; b=C8AL8SUuOVa2GOhosuz9LBvfKubC8f85PRLDqWPB9P4yxxcHK7kgYBB/aeBdvBjDpk dDs379t72TcHxC2uA3aE1jSA30zhpoAvp8RfA/H35mTPUQzNmAhwv+AhFlUHc1RxER0q Wf8p3JPlyYiU5gsX0M8rVr1VvDwl8/Ng58WAJauRvUW21kEU1ArGJRxc00j1QZNTa2GV 21F2/bywzssCxIRyzs9n1jTgO8ewNX2VH/l1fNqgAXLoDpzxHLmUhKbSUReJoujrM3hf i9JtuFnnh5MTrk11ND1v4oPQVVU3fxbiLm0pwqXawlgbVZCK5iNPV5/eRDime2Kelj+B JBSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=VV8pbhNKDGo3Gfnhi3Ncmv9nwYE6+QrLgqukheL3+Ws=; b=f/dlquXK3RlnIvCght+s2NyJDwjfnMMkPxqBE0B9JVplz89cBuDqxP+zR/jVA7DA58 s9Xy2Ba51y/l/J9CmUethny2Rr2ilpsRhBSmNQT0m+ZOUt8B/zx2UrqYIgqkkLR8jDuT jX9jzyHigIH458sufe7oeuR6DGOUyRnQBep6kPXhIiO19ASpMpeI09ixFAoDt+ETFJjo yn9y9oXC/gLTQb3oDymAj0QylMFAUYmnoIuEkA9X7ZcxlK4a6QWAFQB8fB0O1xGoAoLt 2gOWzcpxbpXc2NRb1qvOvgqhF9qtB7SV0G2NoH7jliMpa6SF/OgW9hcM3/JWspa87jlK kkww== X-Gm-Message-State: APjAAAXG9Ka4htAsFZArnUVdFjOMZrBXSb2mqY82Zslbsb0xXu/abWm/ rlYwgxaZUhyJRVhpl5h3IKwPoM99n8I= X-Google-Smtp-Source: APXvYqwSgAQdRCJPPC0dTz4pVvXdIwejuWdr7vZLqHbvHLbzdWxTDZRtmPRBfJJqECsCo6wnHli9bQ== X-Received: by 2002:a1c:9e97:: with SMTP id h145mr871401wme.147.1552457842482; Tue, 12 Mar 2019 23:17:22 -0700 (PDT) Received: from localhost (jirka.pirko.cz. [84.16.102.26]) by smtp.gmail.com with ESMTPSA id h137sm4002543wmg.41.2019.03.12.23.17.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 12 Mar 2019 23:17:22 -0700 (PDT) Date: Wed, 13 Mar 2019 07:07:01 +0100 From: Jiri Pirko To: Jakub Kicinski Cc: davem@davemloft.net, netdev@vger.kernel.org, oss-drivers@netronome.com Subject: Re: [PATCH net-next v2 4/7] devlink: allow subports on devlink PCI ports Message-ID: <20190313060701.GB2384@nanopsycho.orion> References: <20190306122037.GB2819@nanopsycho> <20190306095638.7c028bdd@cakuba.hsd1.ca.comcast.net> <20190307094816.GA2190@nanopsycho> <20190307185202.2db37490@cakuba.hsd1.ca.comcast.net> <20190308145421.GA2888@nanopsycho.orion> <20190308110943.2ee42bc0@cakuba.hsd1.ca.comcast.net> <20190311085204.GA2194@nanopsycho> <20190311191054.36b801d6@cakuba.hsd1.ca.comcast.net> <20190312140239.GA2455@nanopsycho> <20190312135628.5250135b@cakuba.hsd1.ca.comcast.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190312135628.5250135b@cakuba.hsd1.ca.comcast.net> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Tue, Mar 12, 2019 at 09:56:28PM CET, jakub.kicinski@netronome.com wrote: >On Tue, 12 Mar 2019 15:02:39 +0100, Jiri Pirko wrote: >> Tue, Mar 12, 2019 at 03:10:54AM CET, wrote: >> >On Mon, 11 Mar 2019 09:52:04 +0100, Jiri Pirko wrote: >> >> Fri, Mar 08, 2019 at 08:09:43PM CET, wrote: >> >> >If the switchport is in the hypervisor then only the hypervisor can >> >> >control switching/forwarding, correct? >> >> >> >> Correct. >> >> >> >> >The primary use case for partitioning within a VM (of a VF) would be >> >> >containers (and DPDK)? >> >> >> >> Makes sense. >> >> >> >> >SR-IOV makes things harder. Splitting a PF is reasonably easy to grasp. >> >> >I'm trying to get a sense of is how would we control an SR-IOV >> >> >environment as a whole. >> >> >> >> You mean orchestration? >> > >> >Right, orchestration. >> > >> >To be clear on where I'm going with this - if we want to allow VFs >> >to partition themselves then they have to control what is effectively >> >a "nested" switch. A per-VF set of rules which would the get >> >> Wait. If you allow to make VF subports (I believe that is what you ment >> by VFs partition themselves), that does not mean they will have a >> separate nested switch. They would still belong under the same one. > >But that existing switch is administered by the hypervisor, how would >the VF owners install forwarding rules in a switch they don't control? They won't. > >> >"flattened" into the main eswitch rule set. If I was to choose I'd >> >really rather have this "flattening" be done on the (Linux) hypervisor >> >and not in the vendor driver and firmware. >> >> Agreed. Driver should provide one big switch. User should configure it. > >Cool, when you say user - is it the tenant or the provider? Whoever gets access to the instance. > >> >I'd much rather have the VM make a "give me another NIC" orchestration >> >call via some high level REST API than devlink. This makes the >> >configuration strictly high level to low level: >> > >> > VM -> cloud net REST API -> cloud agent -> devlink/Linux -> FW -> HW >> > >> >Without round trips via firmware. >> >> Okay. So the "devlink/Linux -> FW" part is going to happen on baremetal. >> Makes sense. >> >> >This allows for easy policy enforcement, common code to be maintained >> >in user space, in high level languages (no 0.5M LoC drivers and 10M LoC >> >firmware for every driver). It can also be used with software paths >> >like VirtIO.. >> >> Agreed. >> >> >Modelling and debugging a nested switch would be a nightmare. What >> >follows is that we probably shouldn't deal with partitioning of VFs, >> >but rather only partition via the PF devlink instance, and reassign >> >the partitions to VMs. >> >> Agreed. That must be misunderstanding, I never suggested nested >> switches. > >Cool, yes, I was making sure we weren't going in that direction :) Okay. > >> >> I originally planned to implement sriov orchestration api in devlink too. >> > >> >Interesting, would you mind elaborating? >> >> I have to think about it. But something like this: >> [...] > >I see thanks for the examples, they makes things clear! Okay. I will put together some documentation including this. I have some patches that implement some of the stuff. Your patchset also does some of that (considering you adjust a thing or two). Lets make this right.