From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A212C19F2A for ; Thu, 11 Aug 2022 21:55:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 95BE38E0001; Thu, 11 Aug 2022 17:55:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 90BBE6B0075; Thu, 11 Aug 2022 17:55:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7ABEB8E0001; Thu, 11 Aug 2022 17:55:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6B30F6B0073 for ; Thu, 11 Aug 2022 17:55:45 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3ADB912194A for ; Thu, 11 Aug 2022 21:55:45 +0000 (UTC) X-FDA: 79788669450.10.F4E3172 Received: from mail-ua1-f48.google.com (mail-ua1-f48.google.com [209.85.222.48]) by imf20.hostedemail.com (Postfix) with ESMTP id D364B1C006E for ; Thu, 11 Aug 2022 21:55:44 +0000 (UTC) Received: by mail-ua1-f48.google.com with SMTP id t21so7445195uaq.3 for ; Thu, 11 Aug 2022 14:55:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc; bh=u3u7luPR9C1c21z1wcEFU9ERQpt0EtbUvjlQMy/wwkU=; b=WHnAQw3Fn+MhoaqG0bfhnnVecOudD4nKJomuJkJtLlPc73RGk16RaZEsZCo0AyKYAG hLT55YNLUNBmyB7qfq9ukOUKEp+KSrAg5X8ZBJE5l805apiOOv22gupHbyg4DAWjLyzT 8bCnmti2bbfTONXj0S5O2a5tlaij4d2C0TjSSZfl2HxlTfCbEVgXOm7QLCateFha975K 9yrRtke8GyUrs/F8ExY/T0IqEMspYDpaI5DSmYvYardrSHIxk8BONM91hh0mmv6I/VhO h62YLXCmLV5J9pIxKqGXNKtbPS4+Luq4ECo+Zpgwf634ERs/7PE318BbIPq+RX+I3Asa Um9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc; bh=u3u7luPR9C1c21z1wcEFU9ERQpt0EtbUvjlQMy/wwkU=; b=gazlTDU0+5QDBQsSjW5I1ZHmqPzKLpljaELhwt/h7tpbTA1N0q8uhvCeanubuDRT9n ZYoA/RyWjazNrhln0Pb7p0XrQcEWIw4ml6u94vsX0XJPT6jtCiwIZIij98XMTWGV7yrp dmkB6SRjWHF0KTtVgPx15KUucoOdBF8jYIbDP4VWYbpIi8CUhvanq9nako6RjNU20qbM zP0r4TVJv6+SlJo4WtqS/ZmDli1Gq9qoIVAZp3qTxVSHIn8CMV3uc8BR1PJj9mzEUb7U vNGnRP26Xx1RV7F++bDW4LUeTFBK3H9FT0z0ktJG3yJLIUtjIe4/+4YSiscdEpX97tlK zY5Q== X-Gm-Message-State: ACgBeo1iiHcd1A059YwJOH5sv5U+pO04XL4Wnm0N5e/+8SD3eY/874Id F7POpUOlqq0oMklmfeuYr3PLeVsiMaYQfSBQO0optQ== X-Google-Smtp-Source: AA6agR6Tl+w+Cq6fu3hieYhhq29AyLFFQMUDA63A0mOeqSL+4TzW/EZ7q2sgSksfPwbPM376pPSol9onTGk92ef9F1I= X-Received: by 2002:a9f:2bcd:0:b0:387:471e:c4c1 with SMTP id f13-20020a9f2bcd000000b00387471ec4c1mr640310uaj.113.1660254943981; Thu, 11 Aug 2022 14:55:43 -0700 (PDT) MIME-Version: 1.0 References: <20220805184016.2926168-1-alexlzhu@fb.com> <0b16dbac6444bfcdfbeb4df4280354839bfe1a8f.camel@fb.com> <1F8B9D85-A735-4832-AD58-CA4BD474248D@fb.com> <868F0874-70E8-4416-B39B-DA74C9D76A40@fb.com> <3195C304-2140-4E5D-890D-AC55653193E5@fb.com> In-Reply-To: <3195C304-2140-4E5D-890D-AC55653193E5@fb.com> From: Yu Zhao Date: Thu, 11 Aug 2022 15:55:07 -0600 Message-ID: Subject: Re: [PATCH v3] mm: add thp_utilization metrics to /proc/thp_utilization To: "Alex Zhu (Kernel)" Cc: Yang Shi , Rik van Riel , Kernel Team , "linux-mm@kvack.org" , "willy@infradead.org" , "linux-kernel@vger.kernel.org" , "akpm@linux-foundation.org" , Ning Zhang , Miaohe Lin Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660254944; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=u3u7luPR9C1c21z1wcEFU9ERQpt0EtbUvjlQMy/wwkU=; b=Yj5zBexO5+pIvK7JpvdgRa/PxnbbkaecNv3tkTXDk9/+w18AD+jEpyXb2oPU/MXrLxICk2 zL6BW1jPqwVkoNOwxqD43LwnK11XkuVsKVJYN3D2NuaipYOs85bg+a684HRddsrgwOX2iY yW8M04WsEp9N8OGax20sMYx1nLfXN24= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=WHnAQw3F; spf=pass (imf20.hostedemail.com: domain of yuzhao@google.com designates 209.85.222.48 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660254944; a=rsa-sha256; cv=none; b=JaRUI3HKb9Ewdu4wiwi9Sn+xb4uSjxVGlGQuGiUpTWkUACFX5xqW7yMGm/EJ6EQl8pFEhe +blTf3XzSKZhMpjFc1GcOWh0sttuYv8lNuT5FbywIOJnpwx94iECy5+4TzNXBJO92oC1F3 MdrSUcAHyTg4D0fpvKO2e9WezIOHA7Y= X-Stat-Signature: cnaefnpi59kyyjtejhn3fjfm4b88ux4k X-Rspamd-Queue-Id: D364B1C006E Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=WHnAQw3F; spf=pass (imf20.hostedemail.com: domain of yuzhao@google.com designates 209.85.222.48 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1660254944-444019 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Aug 11, 2022 at 1:20 PM Alex Zhu (Kernel) wrote: > > Hi Yu, > > I=E2=80=99ve updated your patch set from last year to work with folio and= am testing it now. The functionality in split_huge_page() is the same as w= hat I have. Was there any follow up work done later? Yes, but it won't change the landscape any time soon (see below). So please feel free to continue along your current direction. > If not, I would like to incorporate this into what I have, and then resub= mit. Will reference the original patchset. We need this functionality for t= he shrinker, but even the changes to split_huge_page() by itself it should = show some performance improvement when used by the existing deferred_split_= huge_page(). SGTM. Thanks! A side note: I'm working on a new mode: THP=3Dauto, meaning the kernel will detect internal fragmentation of 2MB compound pages to decide whether to map them by PMDs or split them under memory pressure. The general workflow of this new mode is as follows. In the page fault path: 1. Compound pages are allocated as usual. 2. Each is mapped by 512 consecutive PTEs rather than a PMD. 3. There will be more TLB misses but the same number of page faults. 4. TLB coalescing can mitigate the performance degradation. In khugepaged: 1. Check the dirty bit in the PTEs mapping a compound page, to determine its utilization. 2. Remap compound pages that meet a certain utilization threshold by PMDs in place, i.e., no migrations. In the reclaim path, e.g., MGLRU page table scanning: 1. Decide whether compound pages mapped by PTEs should be split based on their utilizations and memory pressure, e.g., reclaim priority. 2. Clean subpages should be freed directly after split, rather than swapped= out. N.B. 1. This workflow relies on the dirty bit rather examining the content of a = page. 2. Sampling can be done by periodically switching between a PMD and 512 consecutive PTEs. 3. It only needs to hold mmap_lock for read because this special mode (512 consecutive PTEs) is not considered the split mode. 4. Don't hold your breath :) Other references: 1. https://www.usenix.org/system/files/atc20-zhu-weixi_0.pdf 2. https://www.usenix.org/system/files/osdi21-hunter.pdf