From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB6C7CD612F for ; Mon, 9 Oct 2023 20:05:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 53C1C8D0086; Mon, 9 Oct 2023 16:05:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4EBB68D0003; Mon, 9 Oct 2023 16:05:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B3A38D0086; Mon, 9 Oct 2023 16:05:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2CF3C8D0003 for ; Mon, 9 Oct 2023 16:05:12 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 025601A037F for ; Mon, 9 Oct 2023 20:05:11 +0000 (UTC) X-FDA: 81327002064.01.E762950 Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) by imf30.hostedemail.com (Postfix) with ESMTP id 3563E80002 for ; Mon, 9 Oct 2023 20:05:10 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=W9meif8C; spf=pass (imf30.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.181 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696881910; a=rsa-sha256; cv=none; b=ddaPk3NqHtYiadJ2/bzBdT5DHn9g2wHw6XLBEaoT00GNnx7DeEBEbxHbZFUON1drRwxzZW gnEZGs4Igh0wL6LEkBxOt2fKPOSq2Djiz7hsSvP8xNh2GtGGHXZ7R+ClKUvsrtNIMjUU1U DxY0bJoU2c/SSefPCFaM/ZmbikUfzUM= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=W9meif8C; spf=pass (imf30.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.181 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696881910; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8RmUseYDBSAK6ld4te5RZdIrDezIy+kRo+tEZernjK0=; b=OlJ0zyuEnhrUBJsRSkblEf/gjKeyEp/QxA4U0kVMiK6KTqkgUDOM4oqwe/Eihv9vb3+4B4 t+6ShynIoYmoSmTEbDjBt3L5oEjBJpm5eFq0+KBziqJDowBUXeiPRxJ9JVjsZu8E+WMZJc 6MJyr3+10yaXZ5+HR1tvSWyjToZjJEA= Received: by mail-qt1-f181.google.com with SMTP id d75a77b69052e-4195fe5cf73so600151cf.1 for ; Mon, 09 Oct 2023 13:05:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1696881909; x=1697486709; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=8RmUseYDBSAK6ld4te5RZdIrDezIy+kRo+tEZernjK0=; b=W9meif8CWYpjB8ktqgCDD8I3m5FSXh+jMGHq8F9BUBV+qXs2GhFENFYn2n6PUMqBme PDPcqho5PIb/XkJQLKHKkWmQ7/BGulOfgy92uVPpInkHSnPHOhsyI2lg9IYB4jdE5uSE w84u5aaBXkFk6Rz92NmjDHYBnHYmR03AibAeVT63oTCvSSm7pWPegN++i2CmYFEtvWCO L9agd6BN3+zg2Krvo3vLloBRCeNvv9wgsW2V8wX7z1JddvJKCYwpkSLTAIFS1BSjYSXY QySY2mohaFICfN2MZBaWyksOoANVABqWiOpxMD0XbRvZXYEkcogLloVsUK7NclYQowDq E4pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696881909; x=1697486709; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8RmUseYDBSAK6ld4te5RZdIrDezIy+kRo+tEZernjK0=; b=spY9xDKBX5kLFoW4dibNn4k+UleRGXg48pUfEXk8Dd5xQg3QbPYF7FAzJBzTJULOMn o8dgV2A7KVBGEKGjIt9yo3hX7snrFNsbEAt1jrOnyqjJVI8bjN7cFFgqgWrla+0vKp+i lJ4S8Fg9qcLnXTVsnI/O0rRbQbWe3uClH+PBYGDxpEyZwd7Kxwf36yocbnLeFlBc/xNc /OZQaj04roHlMIaeh0H6Zoy8RMInib1+rZwR9BSh4//UwoS7Nu0Zot49axJddEhqiysS uzfcF41+MfwpPyXpIBqNINBAmfeuuUffvgOlxSfyydf9UIagoltkcQ5ZNWciLcV9mRtq c5Wg== X-Gm-Message-State: AOJu0YyFcm5TiAX73ybpOVTUc1ENA0Gmt8GFHRr/va6yweuOcXq4dLyS nWppIqMXsanO5cJonWWE0UT2MTojldiST3t9MovhrA== X-Google-Smtp-Source: AGHT+IGuPKApUxVXeX87MEU5C5RGG7tpaE3pkFUsYZuECjF/GvRH8xUtxcnStjwfiI4PbDzwvmONgLQQENsV7Ph6E/M= X-Received: by 2002:a05:622a:1452:b0:410:88a5:92f with SMTP id v18-20020a05622a145200b0041088a5092fmr1185590qtx.0.1696881909137; Mon, 09 Oct 2023 13:05:09 -0700 (PDT) MIME-Version: 1.0 References: <20230929114421.3761121-1-ryan.roberts@arm.com> <20230929114421.3761121-7-ryan.roberts@arm.com> <2f64809e-0d0d-cc61-71ac-8d9b072efc3a@redhat.com> <25d1cdee-3da8-4728-aa0d-dc07eb28ea95@arm.com> In-Reply-To: <25d1cdee-3da8-4728-aa0d-dc07eb28ea95@arm.com> From: Yu Zhao Date: Mon, 9 Oct 2023 14:04:31 -0600 Message-ID: Subject: Re: [PATCH v6 6/9] mm: thp: Add "recommend" option for anon_orders To: Ryan Roberts Cc: David Hildenbrand , Andrew Morton , Matthew Wilcox , Yin Fengwei , Catalin Marinas , Anshuman Khandual , Yang Shi , "Huang, Ying" , Zi Yan , Luis Chamberlain , Itaru Kitayama , "Kirill A. Shutemov" , John Hubbard , David Rientjes , Vlastimil Babka , Hugh Dickins , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Suren Baghdasaryan Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 3563E80002 X-Stat-Signature: 5brtfacweg1wmmj5etib5obat1ikhgr8 X-Rspam-User: X-HE-Tag: 1696881910-100164 X-HE-Meta: U2FsdGVkX1/61/WeUgbLp4iEEUdW6cy/pShf8CCxHqXjQgnxFJeKRef3REs1UbefMkJEi1SD1aLFn+vqC+iT/dolWsq8YVR0P2v/Hi6Pe/IY68H4fQJrmu1EJasjt38MXitsDSvCmD+WcskWPgqCNPLio1/wsZBxZzrh6gn+uRgfG4GawqCnDKX5enghzHI1qIS0BZ41DjHCUGyI+chLwOGNeXvvzf0HNyi/P7UmbarMuHnHw5X40QygN/PcLekutUhLlb5KLLVFZ89LY8bqsAmNhIm+11O9Vm8BTNligov1c6M+HStZimZ8fYgRxYHfwNbGEl4YhU3aS3MlQ4AkJK8G5KQ4IT1W9KPqa+5qJbBwSc0Z7OfiyMsLylrwTG8SlQlDmHPoKOtqmXkv259YTJzCSGT9aingPazYzOSRwm6OcH+2JFyfQB7yEPEbHtyUgKgU6Upn9pENHv9pGXxwa77lZSFSMJdb4IxGb7LrpB4CIdt6DX72NI6qEWUVJV1xOH4iZD/dTA+lQoBkB8OX3qyoYxhk1QmXbDnvBVq0S5Jcia7qVSYr9fgijk8irDoD//uScC2nsePYd0x8z0O6q5kYBLPSMyQTh9O/jKqeaRjmswYjPJTOXujbSBDpDhiUpRcxa7T+IhR3FxQGWGPsmiZPEIbiV07rU6PFpmrY4uskyoPrDWuI9PoJr1yf2hayE38vgfbDiNUEBsCx9FoWE9PhcZi8vmUhjrubyJhXZ5nEKNxRFCgOp/ZfrZ5401Qp1XwKbBZKVBG9TRxnBCuZXduMqeulJjrHuZUBslCFQFaLlGKDehphhNrOmLN43UdXvSbLKCko3ME6ZC+gZFBT7eMxZbPbJ9Rc/o0QAHWs4xB8hXg4nW3iYsbLwFmbg2uyjCVgfWNlsbJhTTyj61t2Fj+Qvpjz2og0SLXkZV1c+ifKqFglrvUl1lsc6Ms3jBYbKalF3EgPDWHIPEY809S isjQ0K+g vbB9vUr2GutqIFyNRp72ZDQBhZz+r3C7VxgYlok4DQ7kBZGYoxHY9wDfJheJb3dqWvpyIejcrNAocoQcKNqJE5Va3TAam5hM9CwB//7Ut8Yu4viAnvawYqR6jJ776Ne0ZWTAbRKg1XzPOrGYLBUxNH0ap2p37qKZvdHXkVdzF8hXSjBfpnPzF2jkdhRhrmvsB1X2nMGT8rVSaHfkDbpnpdPJAdmnsr1jqWxI/XZHWL8aTLIt/7Gc1EbZ+dSKUCtoz+1PSr+QmOd7b2v2ovBHXST0fzI7RFO+iLs4v85BxVtQlJz3ikjX2RbwXsERQfKkKTe4V4wnOxRy3qLrFBJM8k9GKW014xO+qFOGlg8SjD2iC/SgxQke2pupnQWzHH29y+hn40zkERdwcmUp7JmhhBrSpAF9VNgMO0R480cbVbsPJqGTtQVScgEjB87iMkGdhZMB4zTyvXdR/dX2pXn4dRWr6mpb1ULdP7+o+HJXeeHBMTqU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Oct 9, 2023 at 5:45=E2=80=AFAM Ryan Roberts = wrote: > > On 06/10/2023 23:28, Yu Zhao wrote: > > On Fri, Oct 6, 2023 at 2:08=E2=80=AFPM David Hildenbrand wrote: > >> > >> On 29.09.23 13:44, Ryan Roberts wrote: > >>> In addition to passing a bitfield of folio orders to enable for THP, > >>> allow the string "recommend" to be written, which has the effect of > >>> causing the system to enable the orders preferred by the architecture > >>> and by the mm. The user can see what these orders are by subsequently > >>> reading back the file. > >>> > >>> Note that these recommended orders are expected to be static for a gi= ven > >>> boot of the system, and so the keyword "auto" was deliberately not us= ed, > >>> as I want to reserve it for a possible future use where the "best" or= der > >>> is chosen more dynamically at runtime. > >>> > >>> Recommended orders are determined as follows: > >>> - PMD_ORDER: The traditional THP size > >>> - arch_wants_pte_order() if implemented by the arch > >>> - PAGE_ALLOC_COSTLY_ORDER: The largest order kept on per-cpu free = list > >>> > >>> arch_wants_pte_order() can be overridden by the architecture if desir= ed. > >>> Some architectures (e.g. arm64) can coalsece TLB entries if a contigu= ous > >>> set of ptes map physically contigious, naturally aligned memory, so t= his > >>> mechanism allows the architecture to optimize as required. > >>> > >>> Here we add the default implementation of arch_wants_pte_order(), use= d > >>> when the architecture does not define it, which returns -1, implying > >>> that the HW has no preference. > >>> > >>> Signed-off-by: Ryan Roberts > >>> --- > >>> Documentation/admin-guide/mm/transhuge.rst | 4 ++++ > >>> include/linux/pgtable.h | 13 +++++++++++++ > >>> mm/huge_memory.c | 14 +++++++++++--- > >>> 3 files changed, 28 insertions(+), 3 deletions(-) > >>> > >>> diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentati= on/admin-guide/mm/transhuge.rst > >>> index 732c3b2f4ba8..d6363d4efa3a 100644 > >>> --- a/Documentation/admin-guide/mm/transhuge.rst > >>> +++ b/Documentation/admin-guide/mm/transhuge.rst > >>> @@ -187,6 +187,10 @@ pages (=3D16K if the page size is 4K). The examp= le above enables order-9 > >>> By enabling multiple orders, allocation of each order will be > >>> attempted, highest to lowest, until a successful allocation is made= . > >>> If the PMD-order is unset, then no PMD-sized THPs will be allocated= . > >>> +It is also possible to enable the recommended set of orders, which > >>> +will be optimized for the architecture and mm:: > >>> + > >>> + echo recommend >/sys/kernel/mm/transparent_hugepage/anon_orders > >>> > >>> The kernel will ignore any orders that it does not support so read = the > >>> file back to determine which orders are enabled:: > >>> diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h > >>> index af7639c3b0a3..0e110ce57cc3 100644 > >>> --- a/include/linux/pgtable.h > >>> +++ b/include/linux/pgtable.h > >>> @@ -393,6 +393,19 @@ static inline void arch_check_zapped_pmd(struct = vm_area_struct *vma, > >>> } > >>> #endif > >>> > >>> +#ifndef arch_wants_pte_order > >>> +/* > >>> + * Returns preferred folio order for pte-mapped memory. Must be in r= ange [0, > >>> + * PMD_ORDER) and must not be order-1 since THP requires large folio= s to be at > >>> + * least order-2. Negative value implies that the HW has no preferen= ce and mm > >>> + * will choose it's own default order. > >>> + */ > >>> +static inline int arch_wants_pte_order(void) > >>> +{ > >>> + return -1; > >>> +} > >>> +#endif > >>> + > >>> #ifndef __HAVE_ARCH_PTEP_GET_AND_CLEAR > >>> static inline pte_t ptep_get_and_clear(struct mm_struct *mm, > >>> unsigned long address, > >>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c > >>> index bcecce769017..e2e2d3906a21 100644 > >>> --- a/mm/huge_memory.c > >>> +++ b/mm/huge_memory.c > >>> @@ -464,10 +464,18 @@ static ssize_t anon_orders_store(struct kobject= *kobj, > >>> int err; > >>> int ret =3D count; > >>> unsigned int orders; > >>> + int arch; > >>> > >>> - err =3D kstrtouint(buf, 0, &orders); > >>> - if (err) > >>> - ret =3D -EINVAL; > >>> + if (sysfs_streq(buf, "recommend")) { > >>> + arch =3D max(arch_wants_pte_order(), PAGE_ALLOC_COSTLY_= ORDER); > >>> + orders =3D BIT(arch); > >>> + orders |=3D BIT(PAGE_ALLOC_COSTLY_ORDER); > >>> + orders |=3D BIT(PMD_ORDER); > >>> + } else { > >>> + err =3D kstrtouint(buf, 0, &orders); > >>> + if (err) > >>> + ret =3D -EINVAL; > >>> + } > >>> > >>> if (ret > 0) { > >>> orders &=3D THP_ORDERS_ALL_ANON; > >> > >> :/ don't really like that. Regarding my proposal, one could have > >> something like that in an "auto" setting for the "enabled" value, or a > >> "recommended" setting [not sure]. > > > > Me either. > > > > Again this is something I call random -- we only discussed "auto", > > and yes, the commit message above explained why "recommended" here but > > it has never surfaced in previous discussions, has it? > > The context in which we discussed "auto" was for a future aspiration to > automatically determine the order that should be used for a given allocat= ion to > balance perf vs internal fragmentation. > > The case we are talking about here is completely different; I had a pre-e= xisting > feature from previous versions of the series, which would allow the arch = to > specify its preferred order (originally proposed by Yu, IIRC). In moving = the > allocation size decision to user space, I felt that we still needed a mec= hanism > whereby the arch could express its preference. And "recommend" is what I = came up > with. > > All of the friction we are currently having is around this feature, I thi= nk? > Certainly all the links you provided in the other thread all point to > conversations skirting around it. How about I just drop it for this initi= al > patch set? Just let user space decide what sizes it wants (per David's in= terface > proposal)? I can see I'm trying to get a square peg into a round hole. Yes, and I think I've been fairly clear since the beginning: why can't the initial patchset only have what we agreed on so that it can get merged asap? Since we haven't agreed on any ABI changes (sysfs, stats, etc.), debugfs (Suren @ Android), boot parameters, etc., Kconfig is the only mergeable option at the moment. To answer your questions [1][2], i.e., why "a compile time option": it's not to make *my testing* easier; it's for *your series* to make immediate progress. [1] https://lore.kernel.org/mm-commits/137d2fc4-de8b-4dda-a51d-31ce6b29a3d0= @arm.com/ [2] https://lore.kernel.org/mm-commits/316054fd-0acb-4277-b9da-d21f0dae2d29= @arm.com/