From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 254CEECAAA2 for ; Thu, 25 Aug 2022 21:31:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F1A16B0075; Thu, 25 Aug 2022 17:31:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 87A85940008; Thu, 25 Aug 2022 17:31:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71B8A940007; Thu, 25 Aug 2022 17:31:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 605FA6B0075 for ; Thu, 25 Aug 2022 17:31:12 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 37870AB9D0 for ; Thu, 25 Aug 2022 21:31:12 +0000 (UTC) X-FDA: 79839410784.07.303C38D Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by imf01.hostedemail.com (Postfix) with ESMTP id CF05F4000E for ; Thu, 25 Aug 2022 21:31:11 +0000 (UTC) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 27PGTgSm002655 for ; Thu, 25 Aug 2022 14:31:11 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=facebook; bh=uien9I9/yhH00YAFVEobmw25WG+SJWjbrVulEVTverU=; b=Dkt7hI8aIBVy/JYVx3xACW0zEof75KS3MZZQhQIYQ3EswSFWMXy+g+CoXuZDPQi2a8GP Z8LIx1pLYalEOmpkmy2RsCsMCUgFUfLaKKwXfg4tKp7DGYP/snxeDmCNZh4NqwS4ba/x UnDlq8+JjNhotMRKHe/2nnqNESxepcYjbqQ= Received: from mail.thefacebook.com ([163.114.132.120]) by m0089730.ppops.net (PPS) with ESMTPS id 3j5xcgq03x-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 25 Aug 2022 14:31:11 -0700 Received: from snc-exhub201.TheFacebook.com (2620:10d:c085:21d::7) by snc-exhub102.TheFacebook.com (2620:10d:c085:11d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 25 Aug 2022 14:31:07 -0700 Received: from twshared22413.18.frc3.facebook.com (2620:10d:c085:108::8) by mail.thefacebook.com (2620:10d:c085:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 25 Aug 2022 14:31:01 -0700 Received: by devvm6390.atn0.facebook.com (Postfix, from userid 352741) id 093D02DC95F6; Thu, 25 Aug 2022 14:30:57 -0700 (PDT) From: To: CC: , , , , , , Alexander Zhu Subject: [RFC 0/3] THP Shrinker Date: Thu, 25 Aug 2022 14:30:51 -0700 Message-ID: X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-FB-Internal: Safe Content-Type: text/plain X-Proofpoint-GUID: _qF4o1yUNz6cdslQEh9SEDAK6ViERJny X-Proofpoint-ORIG-GUID: _qF4o1yUNz6cdslQEh9SEDAK6ViERJny X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-25_10,2022-08-25_01,2022-06-22_01 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661463071; a=rsa-sha256; cv=none; b=7ofIumvp53GS3MtiJFWAeBuSjk7+IAsbUWf6SuY4GyOzmv9B729o5oW6g+V3ruI9Toc68n YcGGaY3tO8NnU8FG/iXCt9DY8lTXJX3DGzh/UVNStnoCZ2qXmla0zOBcGTpAkjPwnasweD wj9aOr68jkxZ6Da4StKX2NbO7Ff03g8= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=Dkt7hI8a; spf=pass (imf01.hostedemail.com: domain of "prvs=9236f59be8=alexlzhu@fb.com" designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=9236f59be8=alexlzhu@fb.com"; dmarc=pass (policy=reject) header.from=fb.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661463071; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=uien9I9/yhH00YAFVEobmw25WG+SJWjbrVulEVTverU=; b=1fGchI3OMlGJNzO3oEy28pOtMcwJUz3R06S+BHyRL+shJlwamt6Ckcaq81zzbevsePDruE fKuHY0oY54JRUTQzvCSY5vz/7aO0xp76NWNbwTJm+liXbMXgb0g2U0ZbknakxA8lj8TAhh 8kMZr9JYe+faXfLvZBUsMAxKTfQKniM= X-Stat-Signature: 8ohq1siufc3z788a9m3pf5e7g54genia X-Rspamd-Queue-Id: CF05F4000E X-Rspam-User: X-Rspamd-Server: rspam06 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=Dkt7hI8a; spf=pass (imf01.hostedemail.com: domain of "prvs=9236f59be8=alexlzhu@fb.com" designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=9236f59be8=alexlzhu@fb.com"; dmarc=pass (policy=reject) header.from=fb.com X-HE-Tag: 1661463071-437342 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alexander Zhu Transparent Hugepages use a larger page size of 2MB in comparison to normal sized pages that are 4kb. A larger page size allows for fewer TLB cache misses and thus more efficient use of the CPU. Using a larger page size also results in more memory waste, which can hurt performance in som= e use cases. THPs are currently enabled in the Linux Kernel by applications in limited virtual address ranges via the madvise system call. The THP shrinker tries to find a balance between increased use of THPs, and increased use of memory. It shrinks the size of memory by removing the underutilized THPs that are identified by the thp_utilization scanner.=20 In our experiments we have noticed that the least utilized THPs are almos= t entirely unutilized. Sample Output:=20 Utilized[0-50]: 1331 680884 Utilized[51-101]: 9 3983 Utilized[102-152]: 3 1187 Utilized[153-203]: 0 0 Utilized[204-255]: 2 539 Utilized[256-306]: 5 1135 Utilized[307-357]: 1 192 Utilized[358-408]: 0 0 Utilized[409-459]: 1 57 Utilized[460-512]: 400 13 Last Scan Time: 223.98 Last Scan Duration: 70.65 Above is a sample obtained from one of our test machines when THP is alwa= ys enabled. Of the 1331 THPs in this thp_utilization sample that have from 0-50 utilized subpages, we see that there are 680884 free pages. This comes out to 680884 / (512 * 1331) =3D 99.91% zero pages in the least utilized bucket. This represents 680884 * 4KB =3D 2.7GB memory waste. Also note that the vast majority of pages are either in the least utilize= d [0-50] or most utilized [460-512] buckets. The least utilized THPs are=20 responsible for almost all of the memory waste when THP is always=20 enabled. Thus by clearing out THPs in the lowest utilization bucket we extract most of the improvement in CPU efficiency. We have seen=20 similar results on our production hosts. This patchset introduces the THP shrinker we have developed to identify and split the least utilized THPs. It includes the thp_utilization=20 changes that groups anonymous THPs into buckets, the split_huge_page() changes that identify and zap zero 4KB pages within THPs and the shrinker changes. It should be noted that the split_huge_page() changes are based off previous work done by Yu Zhao.=20 In the future, we intend to allow additional tuning to the shrinker based on workload depending on CPU/IO/Memory pressure and the=20 amount of anonymous memory. The long term goal is to eventually always=20 enable THP for all applications and deprecate madvise entirely. Alexander Zhu (3): mm: add thp_utilization metrics to debugfs mm: changes to split_huge_page() to free zero filled tail pages mm: THP low utilization shrinker Documentation/admin-guide/mm/transhuge.rst | 9 + include/linux/huge_mm.h | 9 + include/linux/list_lru.h | 24 ++ include/linux/mm_types.h | 5 + include/linux/rmap.h | 2 +- include/linux/vm_event_item.h | 2 + mm/huge_memory.c | 333 +++++++++++++++++- mm/list_lru.c | 49 +++ mm/migrate.c | 60 +++- mm/migrate_device.c | 4 +- mm/page_alloc.c | 6 + mm/vmstat.c | 2 + .../selftests/vm/split_huge_page_test.c | 58 ++- tools/testing/selftests/vm/vm_util.c | 23 ++ tools/testing/selftests/vm/vm_util.h | 1 + 15 files changed, 569 insertions(+), 18 deletions(-) --=20 2.30.2