From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65FDDC001DF for ; Tue, 1 Aug 2023 02:05:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE5742800C1; Mon, 31 Jul 2023 22:05:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C94D828007A; Mon, 31 Jul 2023 22:05:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B83DE2800C1; Mon, 31 Jul 2023 22:05:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A7EA628007A for ; Mon, 31 Jul 2023 22:05:12 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6FB3180373 for ; Tue, 1 Aug 2023 02:05:12 +0000 (UTC) X-FDA: 81073893264.18.964E7BF Received: from out-111.mta0.migadu.com (out-111.mta0.migadu.com [91.218.175.111]) by imf08.hostedemail.com (Postfix) with ESMTP id CDDAD160018 for ; Tue, 1 Aug 2023 02:05:09 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=NhjMmp6i; spf=pass (imf08.hostedemail.com: domain of muchun.song@linux.dev designates 91.218.175.111 as permitted sender) smtp.mailfrom=muchun.song@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690855510; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XFcd0PWa/LzVN9FBDTHBRxRJ7bjV/Fi/yIQ9ks3BIBY=; b=wMn9WFY39AVZDRFrpieIvf3NPeYYcImhMyyJAIERvLScC2mBDJNPTODPvqJGM3lAjcYx1r RR8dJe3uVwk7RRmmtQkqqfi6EvgGQ6HqPm/1JVYNYwEh8ewL4esj3bLRhi2o4R9J6GOX1m XV8EQMiyRI3sivStx3YK9HBZ+bc3/G4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690855510; a=rsa-sha256; cv=none; b=ugXCWGq0hY/zNWJ7K0ANjKtddNgehCUm0MxRb42olqH5QpE+1UbQmEdxTBrNJ0yxcBd+BU aWT8D1cTYrbobCAQB1avCFivBEcaX4/gzR0Z8qEaRuZ4UgelAmaYhs/JAHpki2KZWyyHnt Kss/GysxMJ0j/lM1l0+HmQ84Gk5usJs= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=NhjMmp6i; spf=pass (imf08.hostedemail.com: domain of muchun.song@linux.dev designates 91.218.175.111 as permitted sender) smtp.mailfrom=muchun.song@linux.dev; dmarc=pass (policy=none) header.from=linux.dev Message-ID: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1690855508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XFcd0PWa/LzVN9FBDTHBRxRJ7bjV/Fi/yIQ9ks3BIBY=; b=NhjMmp6icNSl4jKc/Z6RCheyP/E+4k48FsyFZaFqx2akhX8K+31gwF3m0wbD1KmregczEs iYPzs+kMmRNV1mWktvvb8YcHFJ42IqsjmjNleBMnD32nvzIAKyHo2/LkXtIbE7h5+ayft6 vKLrx7ddHvLSd6YWS1oEs28zwD9ZLZY= Date: Tue, 1 Aug 2023 10:04:59 +0800 MIME-Version: 1.0 Subject: Re: [v2 1/6] mm: hugetlb: Skip prep of tail pages when HVO is enabled To: Usama Arif Cc: linux-kernel@vger.kernel.org, fam.zheng@bytedance.com, liangma@liangbit.com, simon.evans@bytedance.com, punit.agrawal@bytedance.com, linux-mm@kvack.org, mike.kravetz@oracle.com, rppt@kernel.org References: <20230730151606.2871391-1-usama.arif@bytedance.com> <20230730151606.2871391-2-usama.arif@bytedance.com> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Muchun Song In-Reply-To: <20230730151606.2871391-2-usama.arif@bytedance.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: CDDAD160018 X-Rspam-User: X-Stat-Signature: 3jimeqs8rghxq34riam86xe5dqpcgdei X-Rspamd-Server: rspam03 X-HE-Tag: 1690855509-307234 X-HE-Meta: U2FsdGVkX1/RIYZ/T1TfZYNgvQ3E9sUEE25MYvtQp+ry0dz7W/cZbXk1fipZEIqPTMmBNch4xy3ybV0pZewp6YdOhv936KH5T2t3h/H5KmhBk7bkmDhmxxa27bnYkCwWRKAx8taF4meZdy+WyHHwSh7XkIpE3hDEmkSiZrseGrUyGiR3nMytMzNlRQFGjltlEjTxbH2hX4bFl77vDqGxa+I/N8zO3gUKdPnxlNDiryz+G0RDlEGv6lhuFH0jhqOPAGgJtpW+TkI/dX0OghZ4uiC1b9hK6NowGxmkqsLQNozJCcViP8ObMTLEBg1UDTlOGfNDej1SJ6WwTRHXQ3N3tq30hDTGWsGvf89BDk5FXb2e/RI8enWZnMV56BGW1mVVN8qVNbjuKE/aMjgv7Pwh3RMS8W5sjDqksS9k4cExHMVPqBuJR0G4XaWDEU4cQjYuztHJGINrzeW+6kxAVUKYQ1zrHQmmTDV4bKzLyJWBLPOXPlM3zdBjRZYFq8DyKDhIvm+47z7mxsHj77hoZqO34s9iJozv1erFLQs56/1bfcOwhJLnQoJRqfnXu0w62v/bjosVyVFALSjNmQyb5gomm2QLz/FNtlZRD2loQu6JYUV9xMEQauA6thg5n5MNLQ0l/4ono9IZUMW6zlUvaNeY89VfH05haygmKxz7v6tvlC5JysAWUbscozWPvChduC/MBHJg4UZYxOkpxK37YIgUQPPeqTQ/OBDaIrf1XZwK2YhuwIC3h2eUq5pHwqbVCj7HlNZHPC2EucT/GkDyMA7SQX29fe4p4wK/ArxX1Xdm37weimSXwxwbYLuSfKz6yRd2OU1P/xzZz6uhhUjHcX968Kd46bs82MFHwlCSL6M8yOLSYu2UP7Nd7Ec2Z4sohny6/PD2k/AuMT3AE01ySgUvDO1hO2V56wQ6NrEF0gFadvrBiCA7PDljf4pIrXp7hqDfKP3iHi09Q78xJY8gqs3 A94HOaf3 Sbr2aJ8tNeLYkH9kBzJMaZAMn+0ZbcHx8AXdu7czIQdhNuOGmauOlNlP5AnQMm+pNqozlNOuJKf89VBSQvn+mZuPtvJHEqA8MfVbOegWaykXw2s5NBDipTK83XNLrTUQ+zKWOgqciudngtnJjeP8U+n7uHg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2023/7/30 23:16, Usama Arif wrote: > When vmemmap is optimizable, it will free all the duplicated tail > pages in hugetlb_vmemmap_optimize while preparing the new hugepage. > Hence, there is no need to prepare them. > > For 1G x86 hugepages, it avoids preparing > 262144 - 64 = 262080 struct pages per hugepage. > > The indirection of using __prep_compound_gigantic_folio is also removed, > as it just creates extra functions to indicate demote which can be done > with the argument. > > Signed-off-by: Usama Arif > --- > mm/hugetlb.c | 32 ++++++++++++++------------------ > mm/hugetlb_vmemmap.c | 2 +- > mm/hugetlb_vmemmap.h | 15 +++++++++++---- > 3 files changed, 26 insertions(+), 23 deletions(-) > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index 64a3239b6407..541c07b6d60f 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -1942,14 +1942,23 @@ static void prep_new_hugetlb_folio(struct hstate *h, struct folio *folio, int ni > spin_unlock_irq(&hugetlb_lock); > } > > -static bool __prep_compound_gigantic_folio(struct folio *folio, > - unsigned int order, bool demote) > +static bool prep_compound_gigantic_folio(struct folio *folio, struct hstate *h, bool demote) > { > int i, j; > + int order = huge_page_order(h); > int nr_pages = 1 << order; > struct page *p; > > __folio_clear_reserved(folio); > + > + /* > + * No need to prep pages that will be freed later by hugetlb_vmemmap_optimize. > + * Hence, reduce nr_pages to the pages that will be kept. > + */ > + if (IS_ENABLED(CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP) && > + vmemmap_should_optimize(h, &folio->page)) > + nr_pages = HUGETLB_VMEMMAP_RESERVE_SIZE / sizeof(struct page); We need to initialize the refcount to zero of tail pages (see the big comment below in this function), given a situation that someone (maybe GUP) could get a ref on the tail pages when the vmemmap is optimizing, what prevent this from happening? Thanks. > + > for (i = 0; i < nr_pages; i++) { > p = folio_page(folio, i); > > @@ -2019,18 +2028,6 @@ static bool __prep_compound_gigantic_folio(struct folio *folio, > return false; > } > > -static bool prep_compound_gigantic_folio(struct folio *folio, > - unsigned int order) > -{ > - return __prep_compound_gigantic_folio(folio, order, false); > -} > - > -static bool prep_compound_gigantic_folio_for_demote(struct folio *folio, > - unsigned int order) > -{ > - return __prep_compound_gigantic_folio(folio, order, true); > -} > - > /* > * PageHuge() only returns true for hugetlbfs pages, but not for normal or > * transparent huge pages. See the PageTransHuge() documentation for more > @@ -2185,7 +2182,7 @@ static struct folio *alloc_fresh_hugetlb_folio(struct hstate *h, > if (!folio) > return NULL; > if (hstate_is_gigantic(h)) { > - if (!prep_compound_gigantic_folio(folio, huge_page_order(h))) { > + if (!prep_compound_gigantic_folio(folio, h, false)) { > /* > * Rare failure to convert pages to compound page. > * Free pages and try again - ONCE! > @@ -3201,7 +3198,7 @@ static void __init gather_bootmem_prealloc(void) > > VM_BUG_ON(!hstate_is_gigantic(h)); > WARN_ON(folio_ref_count(folio) != 1); > - if (prep_compound_gigantic_folio(folio, huge_page_order(h))) { > + if (prep_compound_gigantic_folio(folio, h, false)) { > WARN_ON(folio_test_reserved(folio)); > prep_new_hugetlb_folio(h, folio, folio_nid(folio)); > free_huge_page(page); /* add to the hugepage allocator */ > @@ -3624,8 +3621,7 @@ static int demote_free_hugetlb_folio(struct hstate *h, struct folio *folio) > subpage = folio_page(folio, i); > inner_folio = page_folio(subpage); > if (hstate_is_gigantic(target_hstate)) > - prep_compound_gigantic_folio_for_demote(inner_folio, > - target_hstate->order); > + prep_compound_gigantic_folio(inner_folio, target_hstate, true); > else > prep_compound_page(subpage, target_hstate->order); > folio_change_private(inner_folio, NULL); > diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c > index c2007ef5e9b0..b721e87de2b3 100644 > --- a/mm/hugetlb_vmemmap.c > +++ b/mm/hugetlb_vmemmap.c > @@ -486,7 +486,7 @@ int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head) > } > > /* Return true iff a HugeTLB whose vmemmap should and can be optimized. */ > -static bool vmemmap_should_optimize(const struct hstate *h, const struct page *head) > +bool vmemmap_should_optimize(const struct hstate *h, const struct page *head) > { > if (!READ_ONCE(vmemmap_optimize_enabled)) > return false; > diff --git a/mm/hugetlb_vmemmap.h b/mm/hugetlb_vmemmap.h > index 25bd0e002431..3e7978a9af73 100644 > --- a/mm/hugetlb_vmemmap.h > +++ b/mm/hugetlb_vmemmap.h > @@ -10,16 +10,17 @@ > #define _LINUX_HUGETLB_VMEMMAP_H > #include > > -#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP > -int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head); > -void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head); > - > /* > * Reserve one vmemmap page, all vmemmap addresses are mapped to it. See > * Documentation/vm/vmemmap_dedup.rst. > */ > #define HUGETLB_VMEMMAP_RESERVE_SIZE PAGE_SIZE > > +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP > +int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head); > +void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head); > +bool vmemmap_should_optimize(const struct hstate *h, const struct page *head); > + > static inline unsigned int hugetlb_vmemmap_size(const struct hstate *h) > { > return pages_per_huge_page(h) * sizeof(struct page); > @@ -51,6 +52,12 @@ static inline unsigned int hugetlb_vmemmap_optimizable_size(const struct hstate > { > return 0; > } > + > +static inline bool vmemmap_should_optimize(const struct hstate *h, const struct page *head) > +{ > + return false; > +} > + > #endif /* CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP */ > > static inline bool hugetlb_vmemmap_optimizable(const struct hstate *h)