From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5FC7C43381 for ; Mon, 4 Mar 2019 07:46:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8775120863 for ; Mon, 4 Mar 2019 07:46:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=resnulli-us.20150623.gappssmtp.com header.i=@resnulli-us.20150623.gappssmtp.com header.b="gS/qGWpm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726197AbfCDHqX (ORCPT ); Mon, 4 Mar 2019 02:46:23 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:45661 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725974AbfCDHqX (ORCPT ); Mon, 4 Mar 2019 02:46:23 -0500 Received: by mail-wr1-f66.google.com with SMTP id w17so4316663wrn.12 for ; Sun, 03 Mar 2019 23:46:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=resnulli-us.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=0QBQbmUn/70hsM48DWpDUbfm1duPPqNW3FqKDXPUYMs=; b=gS/qGWpmULoefqos9ZwcCM8et+Jth9DNCcASznELkYRdkLD0xoIu+2g4/P/MTxYYF5 tb23kXRlxKO/Gzu9UsEir0wnkG+6vELsdH9C7tB7zllqM6JOnAi69MPiaKCwp/fb1m6U JET3AjRIAS9If0ZLigb5GmvVOgxoYtidjF7BZnYZkYCT1dkhzOADmPsEWowygcdUtdNP NaKi8ng9IE7lHFwe9OqHhI4ElJrv5JCUpBiLIgvZ6JhSa4pqG4t4UmNXiDyDNXOCprvW ec73baqQsEsMezwFZtX4AqbmqLpN0VS6gw0eUdGjYFUmw77gsAUlwARphZyeQ6kb21fg 5+SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=0QBQbmUn/70hsM48DWpDUbfm1duPPqNW3FqKDXPUYMs=; b=MU07xybMMiUw7EOZQfk1VuZqrZMoRwcNhDQJCP+CwLtC7Z+gsqhI2zZGHzfXtmDhWC 19mOtqBPcPC6JCyq9IGMKwUXVr/FEPA6vdf7DOs6OaU+YlyxIZyWKSa1uy1Bb1HiuRYT pOrIhsudDYyvSX/M8xAUACUJPuhmvLNkh+wzDKsQ10bU3KxxHby+wugUKPflS8VGzorB JaJPcFjTsn0BDkt5ktU1SuEyJnh1KrczMUqeSZoCDAIux6Jd0gOSqH5a8qSESuVy/Xc0 XYICOlS2O7doFWq+xr9M8MEbRbtEh2mRgZkaGRu6S6i5cMurb7rnCfnMEl1v0uSY55Cx bRYA== X-Gm-Message-State: APjAAAX1iNSkCwGe5b7fb5D7HpKdXUbcGr3nyB4I+LBqyZsOC3ip3eIM kLpu6CL+YhXPb0H+vOq4NIdnRQ== X-Google-Smtp-Source: APXvYqxnrU0/us/y2Rj8pgbncTr1KnA3GrjgN/tR/tO9zAqo9CH1pOOkuHpC9lu6UWSlY03N6896xA== X-Received: by 2002:adf:ee45:: with SMTP id w5mr11355527wro.204.1551685580924; Sun, 03 Mar 2019 23:46:20 -0800 (PST) Received: from localhost (ip-89-177-134-16.net.upcbroadband.cz. [89.177.134.16]) by smtp.gmail.com with ESMTPSA id x22sm35468681wmc.19.2019.03.03.23.46.20 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 03 Mar 2019 23:46:20 -0800 (PST) Date: Mon, 4 Mar 2019 08:36:31 +0100 From: Jiri Pirko To: Jakub Kicinski Cc: davem@davemloft.net, netdev@vger.kernel.org, oss-drivers@netronome.com Subject: Re: [PATCH net-next v2 3/7] nfp: register devlink ports of all reprs Message-ID: <20190304073631.GU2314@nanopsycho> References: <20190301180453.17778-1-jakub.kicinski@netronome.com> <20190301180453.17778-4-jakub.kicinski@netronome.com> <20190302084347.GP2314@nanopsycho> <20190302110724.75d1bdf5@cakuba.netronome.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190302110724.75d1bdf5@cakuba.netronome.com> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Sat, Mar 02, 2019 at 08:07:24PM CET, jakub.kicinski@netronome.com wrote: >On Sat, 2 Mar 2019 09:43:47 +0100, Jiri Pirko wrote: >> Fri, Mar 01, 2019 at 07:04:49PM CET, jakub.kicinski@netronome.com wrote: >> >Register all representors as devlink ports. >> > >> >The port_index is slightly tricky to figure out, we use a bit of >> >arbitrary math to create unique IDs for PCI ports. >> > >> >Signed-off-by: Jakub Kicinski >> >--- >> > .../net/ethernet/netronome/nfp/nfp_devlink.c | 40 ++++++++++++++++++- >> > .../net/ethernet/netronome/nfp/nfp_net_repr.c | 16 +++++++- >> > 2 files changed, 53 insertions(+), 3 deletions(-) >> > >> >diff --git a/drivers/net/ethernet/netronome/nfp/nfp_devlink.c b/drivers/net/ethernet/netronome/nfp/nfp_devlink.c >> >index 9af3cb1f2f17..bf7fd9614152 100644 >> >--- a/drivers/net/ethernet/netronome/nfp/nfp_devlink.c >> >+++ b/drivers/net/ethernet/netronome/nfp/nfp_devlink.c >> >@@ -350,7 +350,8 @@ const struct devlink_ops nfp_devlink_ops = { >> > .flash_update = nfp_devlink_flash_update, >> > }; >> > >> >-int nfp_devlink_port_init(struct nfp_app *app, struct nfp_port *port) >> >+static int >> >+nfp_devlink_port_init_phys(struct devlink *devlink, struct nfp_port *port) >> > { >> > struct nfp_eth_table_port eth_port; >> > int ret; >> >@@ -368,6 +369,27 @@ int nfp_devlink_port_init(struct nfp_app *app, struct nfp_port *port) >> > return 0; >> > } >> > >> >+int nfp_devlink_port_init(struct nfp_app *app, struct nfp_port *port) >> >+{ >> >+ struct devlink *devlink = priv_to_devlink(app->pf); >> >+ >> >+ switch (port->type) { >> >+ case NFP_PORT_PHYS_PORT: >> >+ return nfp_devlink_port_init_phys(devlink, port); >> >+ case NFP_PORT_PF_PORT: >> >+ devlink_port_type_eth_set(&port->dl_port, port->netdev); >> >+ devlink_port_attrs_pci_pf_set(&port->dl_port, port->pf_id); >> >+ return 0; >> >+ case NFP_PORT_VF_PORT: >> >+ devlink_port_type_eth_set(&port->dl_port, port->netdev); >> >+ devlink_port_attrs_pci_vf_set(&port->dl_port, port->pf_id, >> >+ port->vf_id); >> >> What is the reason to expose vf/pf id for switch port? Isn't it rather >> an attribute of a peer? > >Naw, its an attribute of the port. I leave the ASIC via PF n or VF m >of PF n. Whatever is on the other side is isolated from the topology >of the ASIC. Ok. > >Is the physical port ID an attribute of the other end of the cable? > >> >+ return 0; >> >+ default: >> >+ return -EINVAL; >> >+ } >> >+} >> >+ >> > void nfp_devlink_port_clean(struct nfp_port *port) >> > { >> > } >> >@@ -376,7 +398,21 @@ int nfp_devlink_port_register(struct nfp_app *app, struct nfp_port *port) >> > { >> > struct devlink *devlink = priv_to_devlink(app->pf); >> > >> >- return devlink_port_register(devlink, &port->dl_port, port->eth_id); >> >+ switch (port->type) { >> >+ case NFP_PORT_PHYS_PORT: >> >+ return devlink_port_register(devlink, &port->dl_port, >> >+ port->eth_id); >> >+ case NFP_PORT_PF_PORT: >> >+ return devlink_port_register(devlink, &port->dl_port, >> >+ (port->pf_id + 1) * 10000 + >> >+ port->pf_split_id * 1000); >> >> Wait. What this 10000/1000 magic about? > >port_index has to be unique, I need some unique number here, as I >stated both in the commit message and the cover letter, this is >arbitrary. You can at least use some defines for that. > >I can put the datapath port identifier in there but its (a) >meaningless, (b) a bitfield, so it will look like 8972367083. And it >may change depending on the FW load, so its not stable either. > >> >+ case NFP_PORT_VF_PORT: >> >+ return devlink_port_register(devlink, &port->dl_port, >> >+ (port->pf_id + 1) * 10000 + >> >+ port->vf_id + 1); >> >+ default: >> >+ return -EINVAL; >> >+ } >> > } >> > >> > void nfp_devlink_port_unregister(struct nfp_port *port) >> >diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_repr.c b/drivers/net/ethernet/netronome/nfp/nfp_net_repr.c >> >index d2c803bb4e56..869d22760a6e 100644 >> >--- a/drivers/net/ethernet/netronome/nfp/nfp_net_repr.c >> >+++ b/drivers/net/ethernet/netronome/nfp/nfp_net_repr.c >> >@@ -292,7 +292,9 @@ nfp_repr_transfer_features(struct net_device *netdev, struct net_device *lower) >> > >> > static void nfp_repr_clean(struct nfp_repr *repr) >> > { >> >+ nfp_devlink_port_unregister(repr->port); >> > unregister_netdev(repr->netdev); >> >+ nfp_devlink_port_clean(repr->port); >> > nfp_app_repr_clean(repr->app, repr->netdev); >> > dst_release((struct dst_entry *)repr->dst); >> > nfp_port_free(repr->port); >> >@@ -395,12 +397,24 @@ int nfp_repr_init(struct nfp_app *app, struct net_device *netdev, >> > if (err) >> > goto err_clean; >> > >> >- err = register_netdev(netdev); >> >+ err = nfp_devlink_port_init(app, repr->port); >> > if (err) >> > goto err_repr_clean; >> > >> >+ err = register_netdev(netdev); >> >+ if (err) >> >+ goto err_port_clean; >> >+ >> >+ err = nfp_devlink_port_register(app, repr->port); >> >> Don't you want to take my patch ("nfp: register devlink port before >> netdev") to change order of register_netdev and devlink_port_register, >> include it to this patchset before this patch and change the order in >> this patch too? I think it would be clearer to do it from the beginning. > >This way both netdev and devlink_port can get registered fully >initialized. Otherwise we'd get two notifications. Are we trying to >establish some ordering rules to get around the rtnl locking? :) The order of devlink_port_register and register_netdev is given by layering. For example, for port change, the devlink_port is still there and registered, only the netdev is unregistered and ib_dev registered instead of vice versa. It has really no relation to rtnl locking. > >> >+ if (err) >> >+ goto err_unreg_netdev; >> >+ >> > return 0; >> > >> >+err_unreg_netdev: >> >+ unregister_netdev(repr->netdev); >> >+err_port_clean: >> >+ nfp_devlink_port_clean(repr->port); >> > err_repr_clean: >> > nfp_app_repr_clean(app, netdev); >> > err_clean: