From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF7D1C636D7 for ; Fri, 17 Feb 2023 22:00:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 452A26B0071; Fri, 17 Feb 2023 17:00:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 402476B0072; Fri, 17 Feb 2023 17:00:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2CB676B0073; Fri, 17 Feb 2023 17:00:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1D9986B0071 for ; Fri, 17 Feb 2023 17:00:45 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id DDA251A09C8 for ; Fri, 17 Feb 2023 22:00:44 +0000 (UTC) X-FDA: 80478154008.30.E649CF3 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf03.hostedemail.com (Postfix) with ESMTP id 261A820032 for ; Fri, 17 Feb 2023 22:00:41 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=X9ATJS1k; spf=pass (imf03.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676671242; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LRwyH602lq+jHY+c11asskZNmz5Lehh5qVkUfe9Q44Y=; b=RsWuGuai9mIMIab1QaouK3lWp1hRrfrxYuXm1C5j17xOf8zNE6EFklgmtM8oR+HqroFZtE EaS6NQIGgp8O98Lor3IVFDiI3VXk9RD1TN+9HlhnUZOHPP03m8Z8a+YzManpulQDLn2FhQ xdpVSA8C2jjh2HVyfDFsR6jglqvQANg= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=X9ATJS1k; spf=pass (imf03.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676671242; a=rsa-sha256; cv=none; b=jQ8F4L3L7RrDhhbQHBEQZk0YHM0ni4yT0HO65te45JUwr+B57kjvGe/ed/Y5Q4x2E1HqIF B91bldQOyxgIeX8iOT1qBfUrDusTlrUs/h7mOB9A5univ0GbNczdjAPOQ6f/158SH/2L8H AderZtlqsYGbY2O1avpK3JNHLjq+gyc= Received: by mail-pl1-f170.google.com with SMTP id f8so2987121plr.10 for ; Fri, 17 Feb 2023 14:00:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1676671241; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=LRwyH602lq+jHY+c11asskZNmz5Lehh5qVkUfe9Q44Y=; b=X9ATJS1kuDIkKZk2PyhmzLACHt4QBzXA34BvI5D8MA8tNpEnOOeaLGuJXMTTYepzmq Kli/WnCBsrD6rSNGEgobVl3+ihHnYZvR6xvorAQp58x105VbCliwkp7502ZWSf5O/l0L Vtj4d+JGM2Kw583zgIq+8acVVSSuQMyueyH4LZuTMCmtt9PTZfwvKMgQY3x+efqGYXSu U5m+2KIvHYgQSBM2s6OSx0EfRLYvcmK8+lMb9UemGuYJNIQ7wzMkHAZ3hNJPlzj8gFCH I2JLmLKvDXmYuIT5S3jPYMk91DZJFU23sNsZOsRzFJqXaLeshw66rEqhqyTeL5ZOXuvr Br5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1676671241; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=LRwyH602lq+jHY+c11asskZNmz5Lehh5qVkUfe9Q44Y=; b=dMhctoBkqd9DUZ4CwonG6ucAcnU89fRPVIBKbgo0cC41mymo/3SQHXdDyxpCSQqfTl wSClB6qnEXdMTEpf3uhpUzuRw4Y0QRuu0iJqx4epMqvjWrRgppdfk0/RFcbHj4eVg1FQ 3haOQk3aQXLRQwuYC0jhJdjNrTjmCwE0KX8CEYq5am9GPOCJrRzVV+6PL5DQEt8261IF aIn+ZARaT09AqxUddpAq5htURzbncB4HqI0tAIdVFK3J0+EcLc5PwJs4cfc1mNX5l2/5 GW95TD6f9Yi4jGPiHVbudGK+EZ0Baw82TqJYYmBv9AooBQyIJJzuBzMwTTyaKHEj50ot jNuw== X-Gm-Message-State: AO0yUKUYluKqVZkGIwIkLnkUqhtVaJV5AZW479JTNkJROyFmOpPPGkqs 8k/r/WUh03hOGoueeVQmam1zPDQC8p182twxBKI= X-Google-Smtp-Source: AK7set/3OnpSvbOfNTYhyAIVnUOZgHs4/fZCU1DKvbCUMCbcb6GNChrac+A4K8L9p8iXcuViyd6ASpNLbsHzVI0IeMo= X-Received: by 2002:a17:902:f80c:b0:196:4e61:b6b5 with SMTP id ix12-20020a170902f80c00b001964e61b6b5mr532852plb.2.1676671240922; Fri, 17 Feb 2023 14:00:40 -0800 (PST) MIME-Version: 1.0 References: <6bbf2c47-05ab-b78c-3165-2eff18962d6d@linux.ibm.com> In-Reply-To: <6bbf2c47-05ab-b78c-3165-2eff18962d6d@linux.ibm.com> From: Yang Shi Date: Fri, 17 Feb 2023 14:00:29 -0800 Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] Using hardware counters to determine hot/cold pages To: Aneesh Kumar K V Cc: lsf-pc@lists.linux-foundation.org, Linux MM , Yu Zhao , Dave Hansen , Johannes Weiner Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 261A820032 X-Stat-Signature: 3yry844yt1umwftsy6p3wy8fn3pfpphf X-Rspam-User: X-HE-Tag: 1676671241-407329 X-HE-Meta: U2FsdGVkX19tuZHfztyjaUgfEIjbYbF4kdQr3ihZR9RpAdyIWG8aR+aFGMvQ/cmSOjzZAI8TAFEMJiScD5XVmiF0rct23zGsd27I3EqufArUCfOavnaShIdpob7zadD7JcGuUcqmGOqkAtPcHm8ZTFr5mf9lAsUxaBkZ4AHAk9fd3yrq6V6vU7M6P0fHN2M8igtyJcW2qnb4z5/Au3rXXn8QBPLT4zR47KqpG6FcN7lvQgqNQa9lh8ID8DYWRDHooeXzNUzR80NoXgDM/hk3du+r/Q91wd+F8T6bIww1o5pn9y+uLMNEGJodJSXh8vB+kPPN5CKGDIbs4Mfrpkdmgmevw5pPlm+bI4hhT88WX+SKKd5XWnj2os/gM2YW2HAvE1aLJet+2Wj7zusL2mlkaPrFHBymOl9p/1/qYPJzy/RgYsPp0yw5tlGXWv7IioA9HTxxJ5R2McjcEGXY4Pva0aX1LaPuLIqod08OvZ4rhaBBUbQzi0ci6YEs1D9mt5EUAwktT+THDik8fOH7eVeT0P3ZS5XNWjQOWyAIuWMJs6Pjda5dQL8RdFnXdPuOlqtEEockicVjh7Lo4gphCDAh3zclnw1peOWquXrmfP6f0IODUcYyp7xpjDKLoNnvd/oWKi1rUB8hjLguhegk8RcrxwGAkJuyEJPpd9J4kc8LrVIz+IFSuTWIo2JgX6WTXlzYmeKy2tqbfcDEzJPoVSzHrNQYKkYEPzn37GT//YkcukciQMmzkXzbX+3w+jvVUZ7KtBf6S+1Bp/ITg8ghREWDAMAgN16ViVsY13wPs7pztWT2NTMitk+86RP1loolUmqK+YsQXXR9s03uIsliuST/TDw5ie+b9S0ihC5K8q6XAdJF7t0Rbgm8ZyyHsuPge+zvwai0I50JWbPiRFuKolrPUJQ/puphO/FrNuT6NbeGUKp/y+t6PvmkHDpGBGA1rOXcAzFuhupXZMD/WLIEatg BbrUaKR5 JcE5FtuNJWWP9kTqt7u5x7We1ao+atayfwKMYrTkcXqx8fz4Szbr4OkrmbjgCdRkKwkhzSd2Dhzcph7SBXB8tY3dC70lqRL03/a63b7Y4hmVHx+k8cMyjjcE1HijLJnDzbz+xk9o20hPztI1R5ntuOMvXPdaaAh5zZCQ1T+LrUIev74+LJcVOQ8Iz4iKg20/FoBWiejWhojJzv/vQRObGnm/nrry378+0Sa+pNDcU8E8Yt/7zFsqcsC8N6IRryl0iZWF+zftZW7VdLKBZZI4r1sX0PVESep9NLIl3CZDn5E0aOA+FBhZCn1mzVipabAlNJ5KLlvM6zwoYFxzmVNqCF2Ci5aFQBx2XuzdFY54BUl1VaQyPlAczDFprcv4FgCyfsUn4G2IwjC5TvhYVje1mpRCN+bRrzNyaVMUFO0+uFPeJMq3tNmd7zPOUbKHGaMaW5hYMdLmRQaQItnDZXXfp7nMb+S2UO8o6P0zM1vy4sa0Z0sXfZ6RnHWMBaMiE7Xk1e/fhVuMSV9nOjdPvRxsE5M1XmtutXnAoImgbUYDnRoVtBWs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Feb 17, 2023 at 3:58 AM Aneesh Kumar K V wrote: > > PowerPC architecture (POWER10) supports a Hot/Cold page tracking > facility that provides access counter and access affinity details at > configurable page size granularity [1]. I have been looking at using > this counter in different areas of the kernel such as > > 1) Page reclaim/demotion > 2) THP utilization > 3) Page promotion. Not sure whether you are aware of this patchset: https://lore.kernel.org/linux-mm/20230208073533.715-1-bharata@amd.com/ ARM64 has SPE which provides similar functionality. So I hope a common framework could be provided to hide the hardware details. > > I have done some MGLRU integration and would like to discuss the > observation with the rest of the community. It is still not clear what > are the best ways to integrate these hardware counters in the Linux > kernel. Attached is the performance graph showing how the mongodb/ycsb > benchmark performs when using hardware counters with MGLRU aging. An > early RFC version of the code can be found at > https://github.com/kvaneesh/linux/commit/b472e2c8080823bb4114c286270aea3e18ffe221 > . I also expect we can get some numbers w.r.t THP usage before the > conference. > > > X axis is the amount of memory that I am removing from the system so > that I can force more memory reclaims. The total memory available is > 50GB/single NUMA node/64 CPUs,40GB database with 40GB cache > configuration. > > > [1] > https://hc32.hotchips.org/assets/program/conference/day1/HotChips2020_Server_Processors_IBM_Starke_POWER10_v33.pdf