From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC272EB64DD for ; Tue, 15 Aug 2023 03:27:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6119D90001C; Mon, 14 Aug 2023 23:27:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C38290000B; Mon, 14 Aug 2023 23:27:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4625890001C; Mon, 14 Aug 2023 23:27:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3622A90000B for ; Mon, 14 Aug 2023 23:27:08 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D74E9140724 for ; Tue, 15 Aug 2023 03:27:07 +0000 (UTC) X-FDA: 81124902894.12.95A5A6C Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf23.hostedemail.com (Postfix) with ESMTP id 7C59314000F for ; Tue, 15 Aug 2023 03:27:05 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=umf5Qjlr; dmarc=none; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692070026; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AkiwaMX9J2EStw5aS/8SkpjxSTrjAeNOcY6xWBx7phM=; b=t2yJNJ1ghHPHeYlM2s4OrDEiLJD6/vsaYQUH9+UPXtadrB+yoeKoYSDL+mkresmahKvEwB paKB0kuYascuqZx7Lx41isuco/KjI9IK+JrilpGX2D+TKEUfpVLb74zOwMoTCluty7efeQ RyNIkzE755v5o6vi+85+KgkJHBsvHGk= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=umf5Qjlr; dmarc=none; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692070026; a=rsa-sha256; cv=none; b=eyi4q56wJa53qBuovHdRpuu4mlgyqVo5IChUJCjsVoaYDEo8XEhjerWipaH3TX0d2MSRRd qLdL7m6Dwwggusl11o67DnLq+Ui+AFTPr3gOnx1noedMkaCP47iBKk3gri2IUVsWLEP/d0 b5ZqLay9uY/HYCNJM1/1LYbyxLNQt64= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=AkiwaMX9J2EStw5aS/8SkpjxSTrjAeNOcY6xWBx7phM=; b=umf5Qjlr4TVGgREeM841PEBgao U/OYkolNBqfy6XznAkCSwddr8yp9BZo/N9H1CdaqwxN06gvrn6jvEqEPblAuMZ/waLUZPVkZ5q7wx OY71tWBf9TI6QkfLbu7dgqNXSXjR5cCPXH9ZMZrrX3Q83VTs2JfbkUELVZd2w5ggZLg/uOH7iCSh7 2YnG4iRVeFk3+o/A8i46rvMVdmIyfKMXspzjCxKR1/AKExDZARAfxbaXsajqLo8bw0bsZglLNg7re Wx2OkHUSPmqEJE+LQBQ3M/ETqVLMRr+AXX3FtZWRR6GV/lEPaWXQBv+151JJoN3WDoLIzN8AfnjbB 6hPPGVtg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qVkhm-005qab-LN; Tue, 15 Aug 2023 03:26:50 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , Jens Axboe , io-uring@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 6/9] mm: Remove HUGETLB_PAGE_DTOR Date: Tue, 15 Aug 2023 04:26:42 +0100 Message-Id: <20230815032645.1393700-7-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230815032645.1393700-1-willy@infradead.org> References: <20230815032645.1393700-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 7C59314000F X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ixs6w74erjfuqonhg93myuc3ak1o8j9s X-HE-Tag: 1692070025-925524 X-HE-Meta: U2FsdGVkX18ElK/Tv2rp2PwFbO9vQzp51HnIjxBweQLtOXpFZLY+7bck1ZM/AJMQltJLDfbveGJOADQK3SUwee2+F9IbBytcRNURgQBrRCoOgzeG0HjIvlEXkyvUtayl746Uf/W5to6RFpU8S/TzsQdBXSGSiB/ejnPG/ctBM29kYY+ASezNgJKLUY6AaLUoHgBdX9B8ZEKLapmvvFHWY9r5ACHqU4ErbcEaAZk2H2jwA9yPhkYChv8AZ+pysxvEgVaX/I5SsgJ50ACJlUHdJqdk3Gei7j0I57IksJ5ltWIEKH//WaTGriJdbgzLEdFiLKPByt+I/ADC2hKugFPXlb1kYeRSHoNyPpNtT4Bw47RrvQvLgjfgYgxYmIR5n5/T4RA7GvNxdomlOT3vEToKu5mCbxohR+0QKulVjyxruYZ5joitXFpYCnUS4G9ZmlICyx5o2QxKJaRyCMVR3EQ+/HUHU50K/uTgyAMNzdgC3RsoWKazk7c5eedBly+22AKi3IYF5Hn9Xz60BPhoM207R2GFaAxjLdaDpm9kv2iAmpaLsAgf4Y1WW1opWLmtThkjHgEKKY/dSwSvwEdeBOlvEpv0Qpgb5kC7O+xbdGWohyEHzoMPAOiCStCiNk9COWPVk863+loKUlteIBAKTqPTAzQf+PP4vh1T2hlCHu3hadcNmOjudSxtuni39SD9EBHz5+UezFQTOVoFSg1p03DbYX1YExHIzUjSdqofx3LSNqbTregPHJNpe4AVtMzQBXrRtyM8WIjczfBl4SUPr1gA2w4zUGEhlqeUGDM/unTBziwp7xM7IIc8sic2ThjdXSKN357iT9GuDNmrarLHHZVX/5eftI3GygVuILaPCOf9H+9siTqQzMlEE37+lnR7Z7Jilg/qUeE5nsxlYHy/usMYoeQqU2ieXkCGTvR+6UkXnuhrj1J645W8zmtkmY62BLARcAWiO4fpVLiuBxQCqqN ivRg0MaT QrLVZfs+4QaFI7S2ywx8ad0k9U9ym4Xc/xrDBKZs2ax5x2R8RTvogY14E5rY+NK0mm2kLCGRRi40lp8vxGOFsn3rguvEWnhXBQjWkOcrfVSxKPONL5qo0y6kvQsxK5nyrOH/D+vFXAIluZA6EO3kDA/QX9lRewvCGRxeDNq7+nx3NHMIP4kbAm7JkzqLL1pNZhjdRD7dcVyfkJr9hca6qxs5ugvLEAxcdwhPk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We can use a bit in page[1].flags to indicate that this folio belongs to hugetlb instead of using a value in page[1].dtors. That lets folio_test_hugetlb() become an inline function like it should be. We can also get rid of NULL_COMPOUND_DTOR. Signed-off-by: Matthew Wilcox (Oracle) --- .../admin-guide/kdump/vmcoreinfo.rst | 10 +--- include/linux/mm.h | 2 - include/linux/page-flags.h | 21 +++++++- kernel/crash_core.c | 2 +- mm/hugetlb.c | 49 +++---------------- 5 files changed, 29 insertions(+), 55 deletions(-) diff --git a/Documentation/admin-guide/kdump/vmcoreinfo.rst b/Documentation/admin-guide/kdump/vmcoreinfo.rst index c18d94fa6470..baa1c355741d 100644 --- a/Documentation/admin-guide/kdump/vmcoreinfo.rst +++ b/Documentation/admin-guide/kdump/vmcoreinfo.rst @@ -325,8 +325,8 @@ NR_FREE_PAGES On linux-2.6.21 or later, the number of free pages is in vm_stat[NR_FREE_PAGES]. Used to get the number of free pages. -PG_lru|PG_private|PG_swapcache|PG_swapbacked|PG_slab|PG_hwpoision|PG_head_mask ------------------------------------------------------------------------------- +PG_lru|PG_private|PG_swapcache|PG_swapbacked|PG_slab|PG_hwpoision|PG_head_mask|PG_hugetlb +----------------------------------------------------------------------------------------- Page attributes. These flags are used to filter various unnecessary for dumping pages. @@ -338,12 +338,6 @@ More page attributes. These flags are used to filter various unnecessary for dumping pages. -HUGETLB_PAGE_DTOR ------------------ - -The HUGETLB_PAGE_DTOR flag denotes hugetlbfs pages. Makedumpfile -excludes these pages. - x86_64 ====== diff --git a/include/linux/mm.h b/include/linux/mm.h index cf6707a7069e..c8c8b1fd64d3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1269,9 +1269,7 @@ unsigned long nr_free_buffer_pages(void); /* Compound pages may have a special destructor */ enum compound_dtor_id { - NULL_COMPOUND_DTOR, COMPOUND_PAGE_DTOR, - HUGETLB_PAGE_DTOR, TRANSHUGE_PAGE_DTOR, NR_COMPOUND_DTORS }; diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 92a2063a0a23..01d98695e79f 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -180,6 +180,9 @@ enum pageflags { PG_has_hwpoisoned = PG_error, #endif + /* Is a hugetlb page. Stored in first tail page. */ + PG_hugetlb = PG_writeback, + /* non-lru isolated movable page */ PG_isolated = PG_reclaim, @@ -812,7 +815,23 @@ static inline void ClearPageCompound(struct page *page) #ifdef CONFIG_HUGETLB_PAGE int PageHuge(struct page *page); -bool folio_test_hugetlb(struct folio *folio); +SETPAGEFLAG(HugeTLB, hugetlb, PF_SECOND) +CLEARPAGEFLAG(HugeTLB, hugetlb, PF_SECOND) + +/** + * folio_test_hugetlb - Determine if the folio belongs to hugetlbfs + * @folio: The folio to test. + * + * Context: Any context. Caller should have a reference on the folio to + * prevent it from being turned into a tail page. + * Return: True for hugetlbfs folios, false for anon folios or folios + * belonging to other filesystems. + */ +static inline bool folio_test_hugetlb(struct folio *folio) +{ + return folio_test_large(folio) && + test_bit(PG_hugetlb, folio_flags(folio, 1)); +} #else TESTPAGEFLAG_FALSE(Huge, hugetlb) #endif diff --git a/kernel/crash_core.c b/kernel/crash_core.c index 90ce1dfd591c..dd5f87047d06 100644 --- a/kernel/crash_core.c +++ b/kernel/crash_core.c @@ -490,7 +490,7 @@ static int __init crash_save_vmcoreinfo_init(void) #define PAGE_BUDDY_MAPCOUNT_VALUE (~PG_buddy) VMCOREINFO_NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE); #ifdef CONFIG_HUGETLB_PAGE - VMCOREINFO_NUMBER(HUGETLB_PAGE_DTOR); + VMCOREINFO_NUMBER(PG_hugetlb); #define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) VMCOREINFO_NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE); #endif diff --git a/mm/hugetlb.c b/mm/hugetlb.c index bc340f5dbbd4..a1cebcee6503 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1585,25 +1585,7 @@ static inline void __clear_hugetlb_destructor(struct hstate *h, { lockdep_assert_held(&hugetlb_lock); - /* - * Very subtle - * - * For non-gigantic pages set the destructor to the normal compound - * page dtor. This is needed in case someone takes an additional - * temporary ref to the page, and freeing is delayed until they drop - * their reference. - * - * For gigantic pages set the destructor to the null dtor. This - * destructor will never be called. Before freeing the gigantic - * page destroy_compound_gigantic_folio will turn the folio into a - * simple group of pages. After this the destructor does not - * apply. - * - */ - if (hstate_is_gigantic(h)) - folio_set_compound_dtor(folio, NULL_COMPOUND_DTOR); - else - folio_set_compound_dtor(folio, COMPOUND_PAGE_DTOR); + folio_clear_hugetlb(folio); } /* @@ -1690,7 +1672,7 @@ static void add_hugetlb_folio(struct hstate *h, struct folio *folio, h->surplus_huge_pages_node[nid]++; } - folio_set_compound_dtor(folio, HUGETLB_PAGE_DTOR); + folio_set_hugetlb(folio); folio_change_private(folio, NULL); /* * We have to set hugetlb_vmemmap_optimized again as above @@ -1814,9 +1796,8 @@ static void free_hpage_workfn(struct work_struct *work) /* * The VM_BUG_ON_FOLIO(!folio_test_hugetlb(folio), folio) in * folio_hstate() is going to trigger because a previous call to - * remove_hugetlb_folio() will call folio_set_compound_dtor - * (folio, NULL_COMPOUND_DTOR), so do not use folio_hstate() - * directly. + * remove_hugetlb_folio() will clear the hugetlb bit, so do + * not use folio_hstate() directly. */ h = size_to_hstate(page_size(page)); @@ -1955,7 +1936,7 @@ static void __prep_new_hugetlb_folio(struct hstate *h, struct folio *folio) { hugetlb_vmemmap_optimize(h, &folio->page); INIT_LIST_HEAD(&folio->lru); - folio_set_compound_dtor(folio, HUGETLB_PAGE_DTOR); + folio_set_hugetlb(folio); hugetlb_set_folio_subpool(folio, NULL); set_hugetlb_cgroup(folio, NULL); set_hugetlb_cgroup_rsvd(folio, NULL); @@ -2070,28 +2051,10 @@ int PageHuge(struct page *page) if (!PageCompound(page)) return 0; folio = page_folio(page); - return folio->_folio_dtor == HUGETLB_PAGE_DTOR; + return folio_test_hugetlb(folio); } EXPORT_SYMBOL_GPL(PageHuge); -/** - * folio_test_hugetlb - Determine if the folio belongs to hugetlbfs - * @folio: The folio to test. - * - * Context: Any context. Caller should have a reference on the folio to - * prevent it from being turned into a tail page. - * Return: True for hugetlbfs folios, false for anon folios or folios - * belonging to other filesystems. - */ -bool folio_test_hugetlb(struct folio *folio) -{ - if (!folio_test_large(folio)) - return false; - - return folio->_folio_dtor == HUGETLB_PAGE_DTOR; -} -EXPORT_SYMBOL_GPL(folio_test_hugetlb); - /* * Find and lock address space (mapping) in write mode. * -- 2.40.1