From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB8E64F213 for ; Tue, 27 Feb 2024 18:09:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709057395; cv=none; b=caDJeU5IaMk+DDU4ZC7qFG37nrY/ynqj8SNwFn1JS6IzCDBdWfazAOLW5tmpoOZL5YuDXObUy47TmplOHYArREhxkqmgAtV/CD9oUdfefa+mRyhVqMa8EPX2eJuQ/dRORVfW8EPEiTy1yi+DVqrEUU+g0zaD1aguj0sU6P25VWg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709057395; c=relaxed/simple; bh=LCdkcrl41bq1jClmkYElaHxcJh2uvHVQMSaUw7NTGFg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZfBbPnqaLCPkJKIENK0WPMF6Qzo0oos7FQb6PY9SiSK5LYNIcMUNDk+cauwp50Jr9YZYGcH+sTMvLK6loEXdUG4F2jann/xZdilgNUAJwJQAv9aIP8ImQZrCpMmv9TfwiaYITYfbuPiMCgEm12YNdNx+tOYzcT5BlYbA28OhmU0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--sdf.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Qb66CASx; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--sdf.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Qb66CASx" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-5dc90074016so4130046a12.0 for ; Tue, 27 Feb 2024 10:09:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709057393; x=1709662193; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=lEYSPjKr0MfecLHJzTD3M81cMNcO8CG0ocLCVkiG6MQ=; b=Qb66CASxlE3ZYsIKfliSqjbnHZJLe+PhJUKYiDzn/KbTw98TnEmkRamhav2fU022zg DK0Y7igr5U5Z0rB067WwNh5yXl6tONcc31BWfU5Q1o70ycUnbEaiA6mXzB2ejefsnwP2 sSqyIuKt1m94PRyBNtT9UDKnZ4KUzQJ6Pfj5ZQ5zd65bxH3RAegFkYw/UQ1PRZWth8hP 7mSDItrcSaeU4WXEXzNKeT7Z0VWEz022g7IQ5ZapMqdCzkYRnrU2tVdp2CMrw1EnHRiU Kvd36gaSb1t7WOAEXmhn/J39D/HAVYryfWFIe6AmH5boTWXVuCUM7sC6bUQ9s/Un5NfO sOgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709057393; x=1709662193; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lEYSPjKr0MfecLHJzTD3M81cMNcO8CG0ocLCVkiG6MQ=; b=UlOtzUmMtlVbs6bEmuQjb8A/3smJYxEzOkxcn2bSGj9AjC88Zqp5s9zeWZ/bUcpbOz qaQ3KjLXclni+KJV9opBGQ9DJDj0s7YuWvyW/kFgOH6pL4H0KG2rC6tnzBgUs5aa0gpp ytFM4Tf3slU39yCisSpx1xV98za855M2bh/3R5sEopBoeqR6Jvz/ik3A7jw59hyW+AWw c1bQxLEiNN0YBtP2gGPRrMne5eXwqRWYlWFBnEm1lz/fSoD453dtSkhwxrTR84D5xxWl P7CouUrzqKtHotj1qiCHbEULohH9mHQ8jLuCwHMtOgx1+wMDzRVvcuZcF8mTIvJeIGBP G5tA== X-Forwarded-Encrypted: i=1; AJvYcCUjAtxdHTw9zRvrGMHJm9YRXlAFyxP2EKhozn6nrt/XHOlvdPbfPRIse8dfJJh8XGkYQRDTzNnXBkr6pZi6xmZkkJVL+p23 X-Gm-Message-State: AOJu0YxuLRVN7Ykub/0sk6dUHQpZlrcXUlHaf3/HVUXs44Rh1Ibq1MdK K8MXS1+yxjftOmIBHTLHosMwCQD6mStyT/Hp8S5GWY9luKv+4hGnwJ2DP5X9lHSy5Q== X-Google-Smtp-Source: AGHT+IEtyzEfIjQ1IQU2b1KkMbK7OYyh/Mw+ZhHolaG5GaMnT4QyyZab1WZPhBVHhrnB92gE1IsFPR8= X-Received: from sdf.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5935]) (user=sdf job=sendgmr) by 2002:a17:902:e88a:b0:1dc:abe9:826b with SMTP id w10-20020a170902e88a00b001dcabe9826bmr311768plg.11.1709057392956; Tue, 27 Feb 2024 10:09:52 -0800 (PST) Date: Tue, 27 Feb 2024 10:09:51 -0800 In-Reply-To: <20240227072404.06ea4843@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240226211015.1244807-1-kuba@kernel.org> <20240226211015.1244807-2-kuba@kernel.org> <20240226141928.171b79fe@kernel.org> <20240227072404.06ea4843@kernel.org> Message-ID: Subject: Re: [PATCH net-next 1/3] netdev: add per-queue statistics From: Stanislav Fomichev To: Jakub Kicinski Cc: davem@davemloft.net, netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, amritha.nambiar@intel.com, danielj@nvidia.com, mst@redhat.com, michael.chan@broadcom.com, vadim.fedorenko@linux.dev Content-Type: text/plain; charset="utf-8" On 02/27, Jakub Kicinski wrote: > On Mon, 26 Feb 2024 19:37:04 -0800 Stanislav Fomichev wrote: > > On 02/26, Jakub Kicinski wrote: > > > On Mon, 26 Feb 2024 13:35:34 -0800 Stanislav Fomichev wrote: > > > > IIUC, in order to get netdev-scoped stats in v1 (vs rfc) is to not set > > > > stats-scope, right? Any reason we dropped the explicit netdev entry? > > > > It seems more robust with a separate entry and removes the ambiguity about > > > > which stats we're querying. > > > > > > The change is because I switched from enum to flags. > > > > > > I'm not 100% sure which one is going to cause fewer issues down > > > the line. It's a question of whether the next scope we add will > > > be disjoint with or subdividing previous scopes. > > > > > > I think only subdividing previous scopes makes sense. If we were > > > to add "stats per NAPI" (bad example) or "per buffer pool" or IDK what > > > other thing -- we should expose that as a new netlink command, not mix > > > it with the queues. > > > > > > The expectation is that scopes will be extended with hw vs sw, or > > > per-CPU (e.g. page pool recycling). In which case we'll want flags, > > > so that we can combine them -- ask for HW stats for a queue or hw > > > stats for the entire netdev. > > > > > > Perhaps I should rename stats -> queue-stats to make this more explicit? > > > > > > The initial version I wrote could iterate both over NAPIs and > > > queues. This could be helpful to some drivers - but I realized that it > > > would lead to rather painful user experience (does the driver maintain > > > stats per NAPI or per queue?) and tricky implementation of the device > > > level sum (device stats = Sum(queue) or Sum(queue) + Sum(NAPI)??) > > > > Yeah, same, not sure. The flags may be more flexible but a bit harder > > wrt discoverability. Assuming a somewhat ignorant spec reader/user, > > it might be hard to say which flags makes sense to combine and which isn't. > > Or, I guess, we can try to document it? > > We're talking about driver API here, so document and enforce in code > review :) But fundamentally, I don't think we should be turning this op > into a mux for all sort of stats. We can have 64k ops in the family. > > > For HW vs SW, do you think it makes sense to expose it as a scope? > > Why not have something like 'rx-packets' and 'hw-rx-packets'? > > I had that in one of the WIP versions but (a) a lot of the stats can > be maintained by either device or the driver, so we'd end up with a hw- > flavor for most of the entries, and (b) 90% of the time the user will > not care whether it's the HW or SW that counted the bytes, or GSO > segments. Similarly to how most of the users will not care about > per-queue breakdown, TBH, which made me think that from user > perspective both queue and hw vs sw are just a form of detailed > breakdown. Majority will dump the combined sw|hw stats for the device. > > I could be wrong. > > > Maybe, as you're suggesting, we should rename stats to queue-states > > and drop the score for now? When the time comes to add hw counters, > > we can revisit. For total netdev stats, we can ask the user to aggregate > > the per-queue ones? > > I'd keep the scope, and ability to show the device level aggregation. > There are drivers (bnxt, off the top of my head, but I feel like there's > more) which stash the counters when queues get freed. Without the device > level aggregation we'd need to expose that as "no queue" or "history" > or "delta" etc stats. I think that's uglier that showing the sum, which > is what user will care about 99% of the time. > > It'd be a pure rename. Ack, sounds fair!