From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB5BCC433E0 for ; Thu, 28 May 2020 05:24:19 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F33D4206F1 for ; Thu, 28 May 2020 05:24:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HHMOS4Dr"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HHMOS4Dr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F33D4206F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 49XbjC0bzQzDqY3 for ; Thu, 28 May 2020 15:24:15 +1000 (AEST) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=redhat.com (client-ip=207.211.31.120; helo=us-smtp-1.mimecast.com; envelope-from=pbonzini@redhat.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=HHMOS4Dr; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=HHMOS4Dr; dkim-atps=neutral Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 49XbgL4WBpzDqQv for ; Thu, 28 May 2020 15:22:35 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590643352; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J+iKV2WQ6lOBXo37IQKk0z/KDQxyIDc+E74qSJBHsLY=; b=HHMOS4DrPrizHpKaCsMF9ZJp2gMJ+/yVUKXtkEyqPna5UKescYVdsyxxo+iBKgM2S6QWcu cnjWtQ9ltSpQoczRgq1AmlCQS8dsfT8q3Q6xXEApHu70UjX/DNaa1Kh9a9eiF85kyVYO/b pfjtA1s6QduaeYaktCtaidQeL1WYcRc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590643352; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=J+iKV2WQ6lOBXo37IQKk0z/KDQxyIDc+E74qSJBHsLY=; b=HHMOS4DrPrizHpKaCsMF9ZJp2gMJ+/yVUKXtkEyqPna5UKescYVdsyxxo+iBKgM2S6QWcu cnjWtQ9ltSpQoczRgq1AmlCQS8dsfT8q3Q6xXEApHu70UjX/DNaa1Kh9a9eiF85kyVYO/b pfjtA1s6QduaeYaktCtaidQeL1WYcRc= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-113-fjhq4oJIPna2J1VpkfVIQA-1; Thu, 28 May 2020 01:22:30 -0400 X-MC-Unique: fjhq4oJIPna2J1VpkfVIQA-1 Received: by mail-ed1-f72.google.com with SMTP id ba29so6769603edb.16 for ; Wed, 27 May 2020 22:22:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=J+iKV2WQ6lOBXo37IQKk0z/KDQxyIDc+E74qSJBHsLY=; b=fileDHMEhnc2YwOuO3E4Nfqd0LRKljqKytKnoLWBDiucL6XjC46z72S8JtSXF/1Qj6 gbNKlYNqPhZ5v8IlU3wlIVHxvKGIinmCJKY7hgkxVuL/t3CAaIJc8Cqupd5KmK/FAtIH i4+y9evqxANwJlGcwm8fmOcE87NCfOHiPL3wuLD7Rxp/PiKeqC5bMKjgWeEwBisTvl8S bxCS+SwnGq/IguFl4xWE97E+EFS/d8H92soLNF1CrTzbwJ/c/VjnDKjD8dQ7QtqjkIEB tMi+t5rR4dkE7bHLPDIE6jPCzPjqBmaU8tP+JORSART03S9nVgFtNS156BSUo898NBNJ cw8g== X-Gm-Message-State: AOAM531/MfI/zyWEHckYRw5BgrKRceQYhHw/RXglDwQrHyvK/gOU1vXe HNdXh5nQDDOqN5/eDWLyL/VrMjMt9Z5U29QcyhrLCOe6EP8PvWZGH7O3LFscspLG2jIXjz/6eNx sXwKEzUHtIjoNYVLL6m56xSEQuQ== X-Received: by 2002:a50:d6d0:: with SMTP id l16mr1387311edj.317.1590643349013; Wed, 27 May 2020 22:22:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzJyu6Bj2R7xQtg6fbTx+WdDfElhcWjvy2DluItTe1wU0Un+QkFMtlI7BVZ+bXqt+0zrNq85A== X-Received: by 2002:a50:d6d0:: with SMTP id l16mr1387289edj.317.1590643348729; Wed, 27 May 2020 22:22:28 -0700 (PDT) Received: from ?IPv6:2001:b07:6468:f312:3c1c:ffba:c624:29b8? ([2001:b07:6468:f312:3c1c:ffba:c624:29b8]) by smtp.gmail.com with ESMTPSA id g23sm4521316ejo.28.2020.05.27.22.22.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 27 May 2020 22:22:28 -0700 (PDT) Subject: Re: [PATCH v3 0/7] Statsfs: a new ram-based file system for Linux kernel statistics To: David Ahern , Jakub Kicinski , Emanuele Giuseppe Esposito References: <20200526110318.69006-1-eesposit@redhat.com> <20200526153128.448bfb43@kicinski-fedora-PC1C0HJN.hsd1.ca.comcast.net> <6a754b40-b148-867d-071d-8f31c5c0d172@redhat.com> <20200527132321.54bcdf04@kicinski-fedora-PC1C0HJN.hsd1.ca.comcast.net> From: Paolo Bonzini Message-ID: Date: Thu, 28 May 2020 07:22:25 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, netdev@vger.kernel.org, Emanuele Giuseppe Esposito , linux-kernel@vger.kernel.org, kvm-ppc@vger.kernel.org, Jonathan Adams , Christian Borntraeger , Andrew Lunn , Alexander Viro , David Rientjes , linux-fsdevel@vger.kernel.org, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org, Jim Mattson Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On 28/05/20 00:21, David Ahern wrote: > On 5/27/20 3:07 PM, Paolo Bonzini wrote: >> I see what you meant now. statsfs can also be used to enumerate objects >> if one is so inclined (with the prototype in patch 7, for example, each >> network interface becomes a directory). > > there are many use cases that have 100's to 1000's have network devices. > Having a sysfs entry per device already bloats memory usage for these > use cases; another filesystem with an entry per device makes that worse. > Really the wrong direction for large scale systems. Hi David, IMO the important part for now is having a flexible kernel API for exposing statistics across multiple subsystems, so that they can be harvested in an efficient way. The userspace API is secondary, and multiple APIs can be added to cater for different usecases. For example, as of the first five patches the memory usage is the same as what is now in the mainline kernel, since all the patchset does is take existing debugfs inodes and move them to statsfs. I agree that, if the concept is extended to the whole kernel, scalability and memory usage becomes an issue; and indeed, the long-term plan is to support a binary format that is actually _more_ efficient than the status quo for large scale systems. In the meanwhile, the new filesystem can be disabled (see the difference between "STATS_FS" and "STATS_FS_API") if it imposes undesirable overhead. Thanks, Paolo