From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59ADCC54E8D for ; Mon, 11 May 2020 17:34:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 39133206D6 for ; Mon, 11 May 2020 17:34:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hcyCwnUX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730774AbgEKReg (ORCPT ); Mon, 11 May 2020 13:34:36 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:52781 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1730215AbgEKReg (ORCPT ); Mon, 11 May 2020 13:34:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1589218474; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jCAJG5qvZrEQpE9xVUmgEKkY6ZGLKpgEhGop1JiyIl0=; b=hcyCwnUXTdYBNYFLqUg2GihL4D9uas+zLEUKIUGv/xskDRIm/IoyiczJXfYo/BSslHxCuu EfRQlwpWh/vC1qOSUP/IUt6SEic3zgNbVCxYSsAwK2CM53VzR2tf5AsKUcJT+I9umDjRrx ysh4dq1P/gu2OJsZCJiKLoFZ2Ggr+S4= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-283-gYWIgIugO4WpAxzD8FwwSQ-1; Mon, 11 May 2020 13:34:32 -0400 X-MC-Unique: gYWIgIugO4WpAxzD8FwwSQ-1 Received: by mail-wr1-f72.google.com with SMTP id g10so5563058wrr.10 for ; Mon, 11 May 2020 10:34:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=jCAJG5qvZrEQpE9xVUmgEKkY6ZGLKpgEhGop1JiyIl0=; b=UBxciJ8K1nCd2AvcZ3KRrlGiac4JAxgHvWl4lsmezhVnRrKC9HPPGTW5U1kCma7LpT VdVLLRbOb407uB8BIUpK0T2R/XVH8zVks9Rema42eSWyFa/LfQceal+7vB6NwMT9FjNV yTxn8iiWoGBysqVr2HMbE17DGXSgiaWyJ+iK824Umv0LW2vIDc6TNmngmm8I0TVbokVI fJq3AsEia4WsRrJgHs7FbY9/KDsmFjkpHXW4kGSIZBIJAFsA0x78kCbU3wa9rgS1jGOH jqCIelLclELVj5ehKMhHeAxNzRx5aDnniCbXlFTve0WXE4/RXkk9gy8U9gDICwgUtp1+ ydbg== X-Gm-Message-State: AGi0PuZy3Nb3xqaOJi372Papo2qoTBMVQFvj3TwK10CS7YsUz0+alPXF XWqZzJ1tGFrNv3oIZSR6xsjYCyBAmw+pNB1c5cQzWoOI9k5D1QEb7XWwpL6ghzEfZDTF/NlhPr2 kZ26kwCkbbK1/3WECmwucC3TSWQ== X-Received: by 2002:a5d:49ca:: with SMTP id t10mr12469227wrs.285.1589218471440; Mon, 11 May 2020 10:34:31 -0700 (PDT) X-Google-Smtp-Source: APiQypJNe6I7z0SbQYOHpqyl28aME7TxGvpzSGiIlzFVueJrlC+GRcVDtQgQEgjyS86Oa3oOuu0xgw== X-Received: by 2002:a5d:49ca:: with SMTP id t10mr12469194wrs.285.1589218471191; Mon, 11 May 2020 10:34:31 -0700 (PDT) Received: from ?IPv6:2001:b07:6468:f312:4c95:a679:8cf7:9fb6? ([2001:b07:6468:f312:4c95:a679:8cf7:9fb6]) by smtp.gmail.com with ESMTPSA id 89sm18102311wrj.37.2020.05.11.10.34.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 11 May 2020 10:34:30 -0700 (PDT) Subject: Re: [PATCH v2 0/5] Statsfs: a new ram-based file sytem for Linux kernel statistics To: Jonathan Adams Cc: Emanuele Giuseppe Esposito , kvm list , Christian Borntraeger , David Hildenbrand , Cornelia Huck , Vitaly Kuznetsov , Jim Mattson , Alexander Viro , Emanuele Giuseppe Esposito , LKML , linux-mips@vger.kernel.org, kvm-ppc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org References: <20200504110344.17560-1-eesposit@redhat.com> <29982969-92f6-b6d0-aeae-22edb401e3ac@redhat.com> From: Paolo Bonzini Message-ID: Date: Mon, 11 May 2020 19:34:29 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Hi Jonathan, I think the remaining sticky point is this one: On 11/05/20 19:02, Jonathan Adams wrote: > I think I'd characterize this slightly differently; we have a set of > statistics which are essentially "in parallel": > > - a variety of statistics, N CPUs they're available for, or > - a variety of statistics, N interfaces they're available for. > - a variety of statistics, N kvm object they're available for. > > Recreating a parallel hierarchy of statistics any time we add/subtract > a CPU or interface seems like a lot of overhead. Perhaps a better > model would be some sort of "parameter enumn" (naming is hard; > parameter set?), so when a CPU/network interface/etc is added you'd > add its ID to the "CPUs" we know about, and at removal time you'd > take it out; it would have an associated cbarg for the value getting > callback. > >> Yep, the above "not create a dentry" flag would handle the case where >> you sum things up in the kernel because the more fine grained counters >> would be overwhelming. > > nodnod; or the callback could handle the sum itself. In general for statsfs we took a more explicit approach where each addend in a sum is a separate stats_fs_source. In this version of the patches it's also a directory, but we'll take your feedback and add both the ability to hide directories (first) and to list values (second). So, in the cases of interfaces and KVM objects I would prefer to keep each addend separate. For CPUs that however would be pretty bad. Many subsystems might accumulate stats percpu for performance reason, which would then be exposed as the sum (usually). So yeah, native handling of percpu values makes sense. I think it should fit naturally into the same custom aggregation framework as hash table keys, we'll see if there's any devil in the details. Core kernel stats such as /proc/interrupts or /proc/stat are the exception here, since individual per-CPU values can be vital for debugging. For those, creating a source per stat, possibly on-the-fly at hotplug/hot-unplug time because NR_CPUS can be huge, would still be my preferred way to do it. Thanks, Paolo