From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DC6DC169C4 for ; Wed, 6 Feb 2019 20:13:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0F51B20844 for ; Wed, 6 Feb 2019 20:13:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="pqPIf25D" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726843AbfBFUND (ORCPT ); Wed, 6 Feb 2019 15:13:03 -0500 Received: from mail-pl1-f196.google.com ([209.85.214.196]:34764 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726401AbfBFUNC (ORCPT ); Wed, 6 Feb 2019 15:13:02 -0500 Received: by mail-pl1-f196.google.com with SMTP id w4so3620171plz.1 for ; Wed, 06 Feb 2019 12:13:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=K5cudVdNFXNJby+xaejP8gToMfIsNdYzwmKFiMkTx2I=; b=pqPIf25Dd7TDKSQ9oAY6jL0Vy+MlqyAkmpa8xYcpe+o+MFbaM/t7YDmqSDBxHLdxCA kdOboXtvIEoLLoDJnnBebk0OGMDLG/ONwWE0d0C2iRj9mHy6YgdSAoyV5vEW8pykESWA GCrqhs8La8bNTmNLq8t77QwVkB5Shfz/LjOgnAh8ZHk3mez/XRgkWMk1xjaJ42MtyPfp vbgiLrl1DvWeY+il14Gu3SmRp3PKH8TaA68/oVw2A3UltPKbUjG+gwe+g1izWKPHHtIs a9zOgBOn6ahUxwiszhjfvlhh8o5IhndAEIXrxEA5SdWfNJoNgurqc9up2HVurYBbrbUY ZgMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=K5cudVdNFXNJby+xaejP8gToMfIsNdYzwmKFiMkTx2I=; b=PDkp29yjZmheSBM4IdNoHnwGdBo58zwSDF7G5Muy/f6b38dODXEsm3TKdbHZDai+8R pH9i/OVmi22iHnNqDST9D9hg+xmgFxLyDg811hQ+nE/i2n5LEMUTYcLO/ozhB60oWgi2 VijzCzTBAs0I4d67TzqWTZaY4DMW3GeLuU/M0fi4Vm/L1B8kBvqRxVCz9Gc6bxKYR7hI VXax5luIf5zBaHk7U9JIV9bH03tweCn7lVL4VCLmmsp7PXTY9z4O2wEp6ytTqOgIJta0 L+TpyrSjuKrJeJUxYbLGoAA/Mzf9ABY6nyaUm54azKGfz5DN9T6rWq666O2XimdV6bUr aG7Q== X-Gm-Message-State: AHQUAuYN67cIznej1pBlFrkhzIZJePzWNoSfVMG4xtJIgYG9DdNZRcQ+ CDqzqFHtEbDe0I+ddcQKPCo= X-Google-Smtp-Source: AHgI3IY3E3MAA55DfCUE/QaaReKeVaz3iEBtspO/lJpGq461UM45IuCUHVPvASHt4uIKuHEIofSysg== X-Received: by 2002:a17:902:4222:: with SMTP id g31mr12385428pld.240.1549483981413; Wed, 06 Feb 2019 12:13:01 -0800 (PST) Received: from [10.67.48.231] ([192.19.223.250]) by smtp.googlemail.com with ESMTPSA id z13sm12286732pgf.84.2019.02.06.12.12.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Feb 2019 12:13:00 -0800 (PST) Subject: Re: [RFC 00/14] netlink/hierarchical stats To: Jakub Kicinski , davem@davemloft.net Cc: oss-drivers@netronome.com, netdev@vger.kernel.org, jiri@resnulli.us, andrew@lunn.ch, mkubecek@suse.cz, dsahern@gmail.com, simon.horman@netronome.com, jesse.brandeburg@intel.com, maciejromanfijalkowski@gmail.com, vasundhara-v.volam@broadcom.com, michael.chan@broadcom.com, shalomt@mellanox.com, idosch@mellanox.com References: <20190128234507.32028-1-jakub.kicinski@netronome.com> From: Florian Fainelli Openpgp: preference=signencrypt Autocrypt: addr=f.fainelli@gmail.com; prefer-encrypt=mutual; keydata= mQGiBEjPuBIRBACW9MxSJU9fvEOCTnRNqG/13rAGsj+vJqontvoDSNxRgmafP8d3nesnqPyR xGlkaOSDuu09rxuW+69Y2f1TzjFuGpBk4ysWOR85O2Nx8AJ6fYGCoeTbovrNlGT1M9obSFGQ X3IzRnWoqlfudjTO5TKoqkbOgpYqIo5n1QbEjCCwCwCg3DOH/4ug2AUUlcIT9/l3pGvoRJ0E AICDzi3l7pmC5IWn2n1mvP5247urtHFs/uusE827DDj3K8Upn2vYiOFMBhGsxAk6YKV6IP0d ZdWX6fqkJJlu9cSDvWtO1hXeHIfQIE/xcqvlRH783KrihLcsmnBqOiS6rJDO2x1eAgC8meAX SAgsrBhcgGl2Rl5gh/jkeA5ykwbxA/9u1eEuL70Qzt5APJmqVXR+kWvrqdBVPoUNy/tQ8mYc nzJJ63ng3tHhnwHXZOu8hL4nqwlYHRa9eeglXYhBqja4ZvIvCEqSmEukfivk+DlIgVoOAJbh qIWgvr3SIEuR6ayY3f5j0f2ejUMYlYYnKdiHXFlF9uXm1ELrb0YX4GMHz7QnRmxvcmlhbiBG YWluZWxsaSA8Zi5mYWluZWxsaUBnbWFpbC5jb20+iGYEExECACYCGyMGCwkIBwMCBBUCCAME FgIDAQIeAQIXgAUCVF/S8QUJHlwd3wAKCRBhV5kVtWN2DvCVAJ4u4/bPF4P3jxb4qEY8I2gS 6hG0gACffNWlqJ2T4wSSn+3o7CCZNd7SLSC5BA0ESM+4EhAQAL/o09boR9D3Vk1Tt7+gpYr3 WQ6hgYVON905q2ndEoA2J0dQxJNRw3snabHDDzQBAcqOvdi7YidfBVdKi0wxHhSuRBfuOppu pdXkb7zxuPQuSveCLqqZWRQ+Cc2QgF7SBqgznbe6Ngout5qXY5Dcagk9LqFNGhJQzUGHAsIs hap1f0B1PoUyUNeEInV98D8Xd/edM3mhO9nRpUXRK9Bvt4iEZUXGuVtZLT52nK6Wv2EZ1TiT OiqZlf1P+vxYLBx9eKmabPdm3yjalhY8yr1S1vL0gSA/C6W1o/TowdieF1rWN/MYHlkpyj9c Rpc281gAO0AP3V1G00YzBEdYyi0gaJbCEQnq8Vz1vDXFxHzyhgGz7umBsVKmYwZgA8DrrB0M oaP35wuGR3RJcaG30AnJpEDkBYHznI2apxdcuTPOHZyEilIRrBGzDwGtAhldzlBoBwE3Z3MY 31TOpACu1ZpNOMysZ6xiE35pWkwc0KYm4hJA5GFfmWSN6DniimW3pmdDIiw4Ifcx8b3mFrRO BbDIW13E51j9RjbO/nAaK9ndZ5LRO1B/8Fwat7bLzmsCiEXOJY7NNpIEpkoNoEUfCcZwmLrU +eOTPzaF6drw6ayewEi5yzPg3TAT6FV3oBsNg3xlwU0gPK3v6gYPX5w9+ovPZ1/qqNfOrbsE FRuiSVsZQ5s3AAMFD/9XjlnnVDh9GX/r/6hjmr4U9tEsM+VQXaVXqZuHKaSmojOLUCP/YVQo 7IiYaNssCS4FCPe4yrL4FJJfJAsbeyDykMN7wAnBcOkbZ9BPJPNCbqU6dowLOiy8AuTYQ48m vIyQ4Ijnb6GTrtxIUDQeOBNuQC/gyyx3nbL/lVlHbxr4tb6YkhkO6shjXhQh7nQb33FjGO4P WU11Nr9i/qoV8QCo12MQEo244RRA6VMud06y/E449rWZFSTwGqb0FS0seTcYNvxt8PB2izX+ HZA8SL54j479ubxhfuoTu5nXdtFYFj5Lj5x34LKPx7MpgAmj0H7SDhpFWF2FzcC1bjiW9mjW HaKaX23Awt97AqQZXegbfkJwX2Y53ufq8Np3e1542lh3/mpiGSilCsaTahEGrHK+lIusl6mz Joil+u3k01ofvJMK0ZdzGUZ/aPMZ16LofjFA+MNxWrZFrkYmiGdv+LG45zSlZyIvzSiG2lKy kuVag+IijCIom78P9jRtB1q1Q5lwZp2TLAJlz92DmFwBg1hyFzwDADjZ2nrDxKUiybXIgZp9 aU2d++ptEGCVJOfEW4qpWCCLPbOT7XBr+g/4H3qWbs3j/cDDq7LuVYIe+wchy/iXEJaQVeTC y5arMQorqTFWlEOgRA8OP47L9knl9i4xuR0euV6DChDrguup2aJVU4hPBBgRAgAPAhsMBQJU X9LxBQkeXB3fAAoJEGFXmRW1Y3YOj4UAn3nrFLPZekMeqX5aD/aq/dsbXSfyAKC45Go0YyxV HGuUuzv+GKZ6nsysJ7kCDQRXG8fwARAA6q/pqBi5PjHcOAUgk2/2LR5LjjesK50bCaD4JuNc YDhFR7Vs108diBtsho3w8WRd9viOqDrhLJTroVckkk74OY8r+3t1E0Dd4wHWHQZsAeUvOwDM PQMqTUBFuMi6ydzTZpFA2wBR9x6ofl8Ax+zaGBcFrRlQnhsuXLnM1uuvS39+pmzIjasZBP2H UPk5ifigXcpelKmj6iskP3c8QN6x6GjUSmYx+xUfs/GNVSU1XOZn61wgPDbgINJd/THGdqiO iJxCLuTMqlSsmh1+E1dSdfYkCb93R/0ZHvMKWlAx7MnaFgBfsG8FqNtZu3PCLfizyVYYjXbV WO1A23riZKqwrSJAATo5iTS65BuYxrFsFNPrf7TitM8E76BEBZk0OZBvZxMuOs6Z1qI8YKVK UrHVGFq3NbuPWCdRul9SX3VfOunr9Gv0GABnJ0ET+K7nspax0xqq7zgnM71QEaiaH17IFYGS sG34V7Wo3vyQzsk7qLf9Ajno0DhJ+VX43g8+AjxOMNVrGCt9RNXSBVpyv2AMTlWCdJ5KI6V4 KEzWM4HJm7QlNKE6RPoBxJVbSQLPd9St3h7mxLcne4l7NK9eNgNnneT7QZL8fL//s9K8Ns1W t60uQNYvbhKDG7+/yLcmJgjF74XkGvxCmTA1rW2bsUriM533nG9gAOUFQjURkwI8jvMAEQEA AYkCaAQYEQIACQUCVxvH8AIbAgIpCRBhV5kVtWN2DsFdIAQZAQIABgUCVxvH8AAKCRCH0Jac RAcHBIkHD/9nmfog7X2ZXMzL9ktT++7x+W/QBrSTCTmq8PK+69+INN1ZDOrY8uz6htfTLV9+ e2W6G8/7zIvODuHk7r+yQ585XbplgP0V5Xc8iBHdBgXbqnY5zBrcH+Q/oQ2STalEvaGHqNoD UGyLQ/fiKoLZTPMur57Fy1c9rTuKiSdMgnT0FPfWVDfpR2Ds0gpqWePlRuRGOoCln5GnREA/ 2MW2rWf+CO9kbIR+66j8b4RUJqIK3dWn9xbENh/aqxfonGTCZQ2zC4sLd25DQA4w1itPo+f5 V/SQxuhnlQkTOCdJ7b/mby/pNRz1lsLkjnXueLILj7gNjwTabZXYtL16z24qkDTI1x3g98R/ xunb3/fQwR8FY5/zRvXJq5us/nLvIvOmVwZFkwXc+AF+LSIajqQz9XbXeIP/BDjlBNXRZNdo dVuSU51ENcMcilPr2EUnqEAqeczsCGpnvRCLfVQeSZr2L9N4svNhhfPOEscYhhpHTh0VPyxI pPBNKq+byuYPMyk3nj814NKhImK0O4gTyCK9b+gZAVvQcYAXvSouCnTZeJRrNHJFTgTgu6E0 caxTGgc5zzQHeX67eMzrGomG3ZnIxmd1sAbgvJUDaD2GrYlulfwGWwWyTNbWRvMighVdPkSF 6XFgQaosWxkV0OELLy2N485YrTr2Uq64VKyxpncLh50e2RnyAJ9Za0Dx0yyp44iD1OvHtkEI M5kY0ACeNhCZJvZ5g4C2Lc9fcTHu8jxmEkI= Message-ID: Date: Wed, 6 Feb 2019 12:12:39 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <20190128234507.32028-1-jakub.kicinski@netronome.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 1/28/19 3:44 PM, Jakub Kicinski wrote: > Hi! > > As I tried to explain in my slides at netconf 2018 we are lacking > an expressive, standard API to report device statistics. > > Networking silicon generally maintains some IEEE 802.3 and/or RMON > statistics. Today those all end up in ethtool -S. Here is a simple > attempt (admittedly very imprecise) of counting how many names driver > authors invented for IETF RFC2819 etherStatsPkts512to1023Octets > statistics (RX and TX): > > $ git grep '".*512.*1023.*"' -- drivers/net/ | \ > sed -e 's/.*"\(.*\)".*/\1/' | sort | uniq | wc -l > 63 > > Interestingly only two drivers in the tree use the name the standard > gave us (etherStatsPkts512to1023, modulo case). > > I set out to working on this set in an attempt to give drivers a way > to express clearly to user space standard-compliant counters. > > Second most common use for custom statistics is per-queue counters. > This is where the "hierarchical" part of this set comes in, as > groups can be nested, and user space tools can handle the aggregation > inside the groups if needed. > > This set also tries to address the problem of users not knowing if > a statistic is reported by hardware or the driver. Many modern drivers > use some prefix in ethtool -S to indicate MAC/PHY stats. At a quick > glance: Netronome uses "mac.", Intel "port." and Mellanox "_phy". > In this set, netlink attributes describe whether a group of statistics > is RX or TX, maintained by device or driver. > > The purpose of this patch set is _not_ to replace ethtool -S. It is > an incredibly useful tool, and we will certainly continue using it. > However, for standard-based and commonly maintained statistics a more > structured API seems warranted. > > There are two things missing from these patches, which I initially > planned to address as well: filtering, and refresh rate control. > > Filtering doesn't need much explanation, users should be able to request > only a subset of statistics (like only SW stats or only given ID). The > bitmap of statistics in each group is there for filtering later on. > > By refresh control I mean the ability for user space to indicate how > "fresh" values it expects. Sometimes reading the HW counters requires > slow register reads or FW communication, in such cases drivers may cache > the result. (Privileged) user space should be able to add a "not older > than" timestamp to indicate how fresh statistics it expects. And vice > versa, drivers can then also put the timestamp of when the statistics > were last refreshed in the dump for more precise bandwidth estimation. Another thing that we cannot quite do with ethtool right now, at least not easily, is something like the following use case. You have some filtering/classification capable hardware, and the HW can count the number of times a rule has been hit/missed. The number of rules programmed into the HW is dynamic and depends on use case so dumping them all is not convenient for e.g.: hundreds/thousands of rules. You would want to return only the rules that are active/enabled, and not the full possible range of rules. With ethtool, this is not possible because you have to define the strings first, and in a second call, you are going to get the dump and fill in the data returned to user-space... I will review more in depth, but the idea looks great so far. > > Jakub Kicinski (14): > nfp: remove unused structure > nfp: constify parameter to nfp_port_from_netdev() > net: hstats: add basic/core functionality > net: hstats: allow hierarchies to be built > nfp: very basic hstat support > net: hstats: allow iterators > net: hstats: help in iteration over directions > nfp: hstats: make use of iteration for direction > nfp: hstats: add driver and device per queue statistics > net: hstats: add IEEE 802.3 and common IETF MIB/RMON stats > nfp: hstats: add IEEE/RMON ethernet port/MAC stats > net: hstats: add markers for partial groups > nfp: hstats: add a partial group of per-8021Q prio stats > Documentation: networking: describe new hstat API > > Documentation/networking/hstats.rst | 590 +++++++++++++++ > .../networking/hstats_flow_example.dot | 11 + > Documentation/networking/index.rst | 1 + > drivers/net/ethernet/netronome/nfp/Makefile | 1 + > .../net/ethernet/netronome/nfp/nfp_hstat.c | 474 ++++++++++++ > drivers/net/ethernet/netronome/nfp/nfp_main.c | 1 + > drivers/net/ethernet/netronome/nfp/nfp_main.h | 2 + > drivers/net/ethernet/netronome/nfp/nfp_net.h | 10 +- > .../ethernet/netronome/nfp/nfp_net_common.c | 1 + > .../net/ethernet/netronome/nfp/nfp_net_repr.h | 2 +- > drivers/net/ethernet/netronome/nfp/nfp_port.c | 2 +- > drivers/net/ethernet/netronome/nfp/nfp_port.h | 2 +- > include/linux/netdevice.h | 9 + > include/net/hstats.h | 176 +++++ > include/uapi/linux/if_link.h | 107 +++ > net/core/Makefile | 2 +- > net/core/hstats.c | 682 ++++++++++++++++++ > net/core/rtnetlink.c | 21 + > 18 files changed, 2084 insertions(+), 10 deletions(-) > create mode 100644 Documentation/networking/hstats.rst > create mode 100644 Documentation/networking/hstats_flow_example.dot > create mode 100644 drivers/net/ethernet/netronome/nfp/nfp_hstat.c > create mode 100644 include/net/hstats.h > create mode 100644 net/core/hstats.c > -- Florian