From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29947C46CD2 for ; Tue, 30 Jan 2024 17:09:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB4C56B00AC; Tue, 30 Jan 2024 12:09:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A629C6B00AD; Tue, 30 Jan 2024 12:09:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 902E66B00AF; Tue, 30 Jan 2024 12:09:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7A6DD6B00AC for ; Tue, 30 Jan 2024 12:09:49 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 48D6080A68 for ; Tue, 30 Jan 2024 17:09:49 +0000 (UTC) X-FDA: 81736614498.04.782B731 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf23.hostedemail.com (Postfix) with ESMTP id 667A7140020 for ; Tue, 30 Jan 2024 17:09:47 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=TBfb9uYH; spf=pass (imf23.hostedemail.com: domain of 3Wi25ZQoKCM0H7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3Wi25ZQoKCM0H7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706634587; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3gUAiVWxsdrX9MXtJSqjLmwRv/f/RgRmiC76ruk/jsk=; b=YwpZjlLAT3EOcwQ2dfju42TcFPE6I/P4IOHhJdSwXfLuZgjCvdKAMR3e/k7kHRsZOGvF6g H19elajnX8qDCGuDxujYDB0xPE/Q1ADrGRnpcQUTX64RIVKLS/qFF7vbATDk4MBK+AW+Sn cbSAM1DIFodZE0zeIpA7FEhyQt9zXEU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706634587; a=rsa-sha256; cv=none; b=8kVelEat50J1/NQRDJwYwmGCndT6dLTC/kEJBuqfd/TY0/8o2+iFAQCLZD5a+sLhF72UAd QPlaY0BXczYpwN9eY19zq5ciMQ+I28Q+rfgS6G8gOrkM8bDxd3TrYx01US0cRbWz03r4FN MXigm4SoIOUOIMf4FUwgAbI7OkLmwMs= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=TBfb9uYH; spf=pass (imf23.hostedemail.com: domain of 3Wi25ZQoKCM0H7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3Wi25ZQoKCM0H7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dc6b26783b4so243429276.0 for ; Tue, 30 Jan 2024 09:09:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706634586; x=1707239386; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=3gUAiVWxsdrX9MXtJSqjLmwRv/f/RgRmiC76ruk/jsk=; b=TBfb9uYHRIQfpubZ3hp/TzxiUVVfiNYcfz/9koqbEIyRL7DHWK6Ppm4TLckujtDeew agr/gWaFeCJOKU+RV4kjNc1SjHSBTuDkEogdUmpzHWUjE5I8dvjReZjQdvIZAGAV3Nr3 PQxGOgSgnljXhSuZTP0tYfGEWKoouK/8nTTZLrAbCNlnZyejeMTMfkwwLfT/I6lq0Nxg 75QVBeEDxXVR5JCZ3e23ucS1qMCBM7ZrSaOAYE7UPrkGZqNtUPpYn0/4zAPbIEnMbe0B qefpvSZ9SAwNf4hEq9Qus2JZcK2ECqPWgEqkHR4j+QiG85jvPP8OsN3iS8CW6Imf5Rh1 ImUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706634586; x=1707239386; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3gUAiVWxsdrX9MXtJSqjLmwRv/f/RgRmiC76ruk/jsk=; b=mYRM5akhhHzfCms0Cn2dxEARWmfg18LhYpcdqT2Ui6ZNHdakSof1qlBPksRDAJ16td wtK/6ZEkY3N3SLdrfGNxBC032DX9wEdl8ZSf+WSDSu8LJETUevXIYraYlNEqGxN4JtI9 QGQAgAWqlybvS0A6ekb9yVSm5n5EK9rTjAQ2FR74uBR/YK9hPdE1z8IQL8QHA+ubjitO Njoa5nBmT4QkfPSAV84jwbMcxloXs+wkZ4teRKNjKRfbJ+T2/QLyz97HmsMY21EAXLUG tk1mA1Dc8ecFVsOQdlKmHGlDdO42W2tohieGUSPCKqBRyQI89QomV9iV8ov6uoZjJG8p Boww== X-Gm-Message-State: AOJu0YxjRYO3AVO/E+YS8NGlSraOBUko3BOOQFkUTFdhvXTAEQcoQRiV qQElnIS0N5AnQ5ApeNa5v7mk2/jqoYoCMwI69TDibuHctBTA9mIgktrhS956YRzXEwnNHUV68Pk qM5GXpemV01XEZzHT6g== X-Google-Smtp-Source: AGHT+IGSAt/cicvuYR+6VzSMEHyc1YMNmMpNIIn+3acHabrN/cjKOm187soaitREtOeSDbJEryRDFwsxRAXg741d X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a05:6902:e05:b0:dc6:207d:7797 with SMTP id df5-20020a0569020e0500b00dc6207d7797mr458648ybb.3.1706634586391; Tue, 30 Jan 2024 09:09:46 -0800 (PST) Date: Tue, 30 Jan 2024 17:09:44 +0000 In-Reply-To: <2024012948-hungry-tibia-5345@gregkh> Mime-Version: 1.0 References: <20240129224204.1812062-1-souravpanda@google.com> <20240129224204.1812062-2-souravpanda@google.com> <2024012948-hungry-tibia-5345@gregkh> Message-ID: Subject: Re: [PATCH v7 1/1] mm: report per-page metadata information From: Yosry Ahmed To: Greg KH Cc: Sourav Panda , corbet@lwn.net, rafael@kernel.org, akpm@linux-foundation.org, mike.kravetz@oracle.com, muchun.song@linux.dev, rppt@kernel.org, david@redhat.com, rdunlap@infradead.org, chenlinxuan@uniontech.com, yang.yang29@zte.com.cn, tomas.mudrunka@gmail.com, bhelgaas@google.com, ivan@cloudflare.com, pasha.tatashin@soleen.com, hannes@cmpxchg.org, shakeelb@google.com, kirill.shutemov@linux.intel.com, wangkefeng.wang@huawei.com, adobriyan@gmail.com, vbabka@suse.cz, Liam.Howlett@oracle.com, surenb@google.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, willy@infradead.org, weixugc@google.com Content-Type: text/plain; charset="us-ascii" X-Rspamd-Queue-Id: 667A7140020 X-Rspam-User: X-Stat-Signature: nst9nug5rh4uj7kjpkm3efg1fbqjxzsq X-Rspamd-Server: rspam03 X-HE-Tag: 1706634587-754177 X-HE-Meta: U2FsdGVkX192QDZGISbHiiveuUcSW4FiFs3ezUWHP0le+5htl3/s3MbcFs0DGMt6GjAvKb8IrBORH9vjcv4g98Olzvx2PhAV6iwXO9Q+IQUHSx0GQKfOlWcl7JI+SnmZ15OUDCDs5vyfqJ1S2Xtj6lvmGKnbzESnoNz3pAZVyV3jrStBFqYrGFMaBtr/BB8L5DLsuni+1Gu2oo6Osns+N3kxQS3QIGxSQh2WtWyCN2MSnkTNHYPBcumdT/DLg+HQNsp+yK22ndpqrhT5/6/4s+OU99jdc9MbVVQvVaTMQUgKdVCV3UPGhWfHwmp52tEmhtUCLh/UM1cwD5HTVv/GpcUa5vscQpKBD2dNg/30AJ87ONd4Sy20igBLzEBB4F2smCGyCyxTsNg+UbKIFM6Ptxq2u/XQtQTG42CgGXFA8CQFgQjuA/rM4NgcZ1DMyHo/rnyfA5p7tH7zz9PkVr7cma5AHlgWUeOUEs6bBVvqgcRCW10Ue5FaYAUeAHosr3poj1Gl3T6mgJudA44va+iCM/heJlZloIbiuMK+ynen7IMm1N2jRZV+g5OzBuUIFyc9+Pq5pkfOdQRpH2FgdKOuSdLay6pXPCMzcym7u8ms0PibYQoLyiVYQJ04gSdN8xz+VTzNeHNcmjjRk8jF9EtQPqDgnQ7af+jZwSL9K7OJT6qdgdukgW/EONWMbLgZJKzJouZ4geW8WJa81qnTyNhfzOfIKHH8/v3laxvzP/rP8/gKNG4L+eqC1IC37TFZ4+QvNbWzXrTgumIDLRzQ1/jaw9VZGYFOQTcsIyStxANwmxn6hvEtMyhEtrHpcfkuynH+wht709atZCFZ4kgUrWx710PSGCwJAFeVNBXZoqY/xniuJ/OOfX1prO1QKAUPh+DWF9Nf9XWmP+o3sG9JONk23dZ5JAiN8lEnRAyfEx3JTK5+1hvzzKh7hynjL+lHIe6NBbxjBuIsiRtAaWlBpZA cL8SvCqL fRuZ3VCwr5O0fO4zgdGiRn/FFr1p4jxg9vAX6qy7okQH1O1IrbU4AiWuFogp3Ad8jzvMbPhCjQN2eFJ/qgVzi6Vo4QtIxmyCP5RF2tRG3zku/JWdz4ubEX6bhY+HtcyikCQPpnxhoof4iVFBr0smfh1VTAXp+KiRfFr86xdF5V/Bs9TpfZw0dp7YJ1c/Ooi+u3TI/LEAeJLpkqL82YlKhZbIjPPMT+qFGX6w2EREdU9uOTFA91y0qYpvzQ4ebhKqcl2TNzAe1v5yUUr+zo/UBFTB40bfM+r1KjFD4Ei8O7J+OJ0QZVeDJleEG6qtiEKDsgrId0GBlfoOL43TKOwedBNRPrNJDwEvG2IFETP4pIZd4ojO7zVxwb7GOTfE7IFvqINo+LrZsEYttSC2cx4ccVXtnfVwMyOXc5/yCyNRpbCYH5D3xd8kG1D5W8wAlkMKZN+1ABz7spf1yAIvYRDGdM1VCQM9oX432bPsp/nVPuTZz5srCFo/uuMSuVXeiBGHWF/kg51leQbkHWgNiWmEBYtlyXyCIWb8df0r3FGxsQWY1o3JqZZ809WDrmT6duhSiOKLXi4jb3qLpddLG4Ub5PEItNA0vPbs8zA6CH1wtf3zHfR2yv98TxEey7G/KQur7C0c/b1jRjJPiizQzf2z/zbEvFwQYUYJXE8IERZ4JQyJqcumt9cCspuMJpCoubkJ/SUg3ukx91fn0QSOqUCY/rHHphxMXd4ol4ItsthkaEpEyqQzQoTkjn6BxCPp0bh51n8iY+Qtw9KTUCF0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Jan 29, 2024 at 02:52:23PM -0800, Greg KH wrote: > On Mon, Jan 29, 2024 at 02:42:04PM -0800, Sourav Panda wrote: > > Adds two new per-node fields, namely nr_page_metadata and > > nr_page_metadata_boot, to /sys/devices/system/node/nodeN/vmstat > > and a global PageMetadata field to /proc/meminfo. This information can > > be used by users to see how much memory is being used by per-page > > metadata, which can vary depending on build configuration, machine > > architecture, and system use. > > > > Per-page metadata is the amount of memory that Linux needs in order to > > manage memory at the page granularity. The majority of such memory is > > used by "struct page" and "page_ext" data structures. In contrast to > > most other memory consumption statistics, per-page metadata might not > > be included in MemTotal. For example, MemTotal does not include memblock > > allocations but includes buddy allocations. In this patch, exported > > field nr_page_metadata in /sys/devices/system/node/nodeN/vmstat would > > exclusively track buddy allocations while nr_page_metadata_boot would > > exclusively track memblock allocations. Furthermore, PageMetadata in > > /proc/meminfo would exclusively track buddy allocations allowing it to > > be compared against MemTotal. > > > > This memory depends on build configurations, machine architectures, and > > the way system is used: > > > > Build configuration may include extra fields into "struct page", > > and enable / disable "page_ext" > > Machine architecture defines base page sizes. For example 4K x86, > > 8K SPARC, 64K ARM64 (optionally), etc. The per-page metadata > > overhead is smaller on machines with larger page sizes. > > System use can change per-page overhead by using vmemmap > > optimizations with hugetlb pages, and emulated pmem devdax pages. > > Also, boot parameters can determine whether page_ext is needed > > to be allocated. This memory can be part of MemTotal or be outside > > MemTotal depending on whether the memory was hot-plugged, booted with, > > or hugetlb memory was returned back to the system. > > > > Suggested-by: Pasha Tatashin > > Signed-off-by: Sourav Panda > > --- > > Documentation/filesystems/proc.rst | 3 +++ > > fs/proc/meminfo.c | 4 ++++ > > include/linux/mmzone.h | 4 ++++ > > include/linux/vmstat.h | 4 ++++ > > mm/hugetlb_vmemmap.c | 19 ++++++++++++++---- > > mm/mm_init.c | 3 +++ > > mm/page_alloc.c | 1 + > > mm/page_ext.c | 32 +++++++++++++++++++++--------- > > mm/sparse-vmemmap.c | 8 ++++++++ > > mm/sparse.c | 7 ++++++- > > mm/vmstat.c | 26 +++++++++++++++++++++++- > > 11 files changed, 96 insertions(+), 15 deletions(-) > > > > diff --git a/Documentation/filesystems/proc.rst b/Documentation/filesystems/proc.rst > > index 49ef12df631b..d5901d04e082 100644 > > --- a/Documentation/filesystems/proc.rst > > +++ b/Documentation/filesystems/proc.rst > > @@ -993,6 +993,7 @@ Example output. You may not have all of these fields. > > AnonPages: 4654780 kB > > Mapped: 266244 kB > > Shmem: 9976 kB > > + PageMetadata: 513419 kB > > KReclaimable: 517708 kB > > Slab: 660044 kB > > SReclaimable: 517708 kB > > Why are you adding it to the middle of the file? Are you sure the > userspace tools that parse this file today can handle an unknown field > here, and not just at the end of the file? FWIW, looking at git blame for fs/proc/meminfo.c, it seems like people have generally been adding items where it makes sense semantically, not at the end of the file. So maybe that's okay for userspace tools.