From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AD60C5475B for ; Fri, 8 Mar 2024 04:31:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 44C5B6B032D; Thu, 7 Mar 2024 23:31:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3FCDE6B032E; Thu, 7 Mar 2024 23:31:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C44F6B032F; Thu, 7 Mar 2024 23:31:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1D4AC6B032D for ; Thu, 7 Mar 2024 23:31:10 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 9C1F24135F for ; Fri, 8 Mar 2024 04:31:09 +0000 (UTC) X-FDA: 81872597058.25.3EBE4C5 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf13.hostedemail.com (Postfix) with ESMTP id 292FB20003 for ; Fri, 8 Mar 2024 04:31:05 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=vWe8jjEn; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709872267; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ix31HswNvG6e4yZ4UH3l8wzhhUan5/t3AuA3RESfIuM=; b=1JLxEURF84vDWC0wrjvVQQn4RZ/+JNsfcYepG0Aaq5kDDOY7OO8556UYvIEf8AETif5W4w spgunVTTjp4GXpHJBv/QfoLz6rHudUmmfmXZ7XqasyK0WpRiAQqGTgzHLa6w16XKXnMunf tEDq58b73fXf4C7guw832o3LqCKO11U= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=vWe8jjEn; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709872267; a=rsa-sha256; cv=none; b=fuYcYyiNeFfmEZyWhyh5FPs/72bNloygz1VCjU0zlB6O8Nvr8LHquUmbb1ffcVxO4wC2Kr s6Vs3czwYtZDQs9KcHKzRrD9yXDbIGfiVP86OBlPkw4gVWy3CzmkmUsGUhfMr47AhNL2y4 wDjDIimpq3XGyNsGs0xLhVq+JqZG57s= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Ix31HswNvG6e4yZ4UH3l8wzhhUan5/t3AuA3RESfIuM=; b=vWe8jjEnikgujIi02g4mbYQ/sY R09uIO3LkPgvzJ/wsGfDP1oMpUVzOhpO789nb2NIt1JBKDgkcqrbO5yEGOTyc2zy/Dgl8JRC5JEDw 2ETQkTIaOrZFDhV9Ngyrwm51asBK3p8GBxcd6b88YNDw1GWDS+9MQOf/NqfQlNA5khfPTDu/8Akzt sjevyyvW42jp3ht52KRXIMP/e7dID+CzxSpQwO1wJ+tg0UVn34mL6ODW+JrIyPjkg08O1LU7PHWZ8 tYmwzrcTICbWP08lY+/z54ujX8oMfUFgAJA7NGHwW99Q2TfdT/oHxcNyEVE8afMMI69HLsHH2sE0y vGmQvkiA==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1riRst-0000000AiIW-2Fl5; Fri, 08 Mar 2024 04:31:03 +0000 Date: Fri, 8 Mar 2024 04:31:03 +0000 From: Matthew Wilcox To: David Hildenbrand Cc: linux-mm@kvack.org, Oscar Salvador Subject: Re: [PATCH 0/5] Remove some races around folio_test_hugetlb Message-ID: References: <20240301214712.2853147-1-willy@infradead.org> <52599fd8-76dc-4d8f-b9f2-78146fc7a518@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 292FB20003 X-Rspam-User: X-Stat-Signature: o8io9qx4hq6tdsi31b1ndafozt7fg6s1 X-Rspamd-Server: rspam01 X-HE-Tag: 1709872265-578980 X-HE-Meta: U2FsdGVkX1+z4vr9gO85kBPIV4PcUZ0gfjHARnyjdtOdiOWtwCUOVXo2p5PEznFk3XCXsKGg2ycpUVagiwsK2fX4Oe5AXmRglexKR9HYjOwD7UD5pgbWgMZVc1r8YxHvql0VkefHX9sTx4lCvLCPZ+6A0Nb7dLlfKQY7Y7dCZU/leuKBkkodAJe/ZiWl4PXZXl+Wihp5ZmioKNRIivYmtTo9gWyFDZDGQYYzYcbzCPjaxE7W4jXQUN559EynqQmamCLf2takFI1+kVXEicpOZzle9hLTHkdIAwxczRRwS/qPXCTmBRKSjXXfdyNYMY+px58ZO6sGR2c4BYCsT+wYUn6IschTwMGxQD3AhB3kuGk0hUmq1/K4uDG4OV5SaWzb/ZJrzPG6j6Aogacw7tNxZbxc7evOhTnnNxZ+9J6moUk+LUddI6GERPKQjWm9ywfSHGZG+7gfXb77zClKGsJ1mID5C9RNFzhaQjh9jbjyhTm+iqu/6TC3NDDXLF5qofTjaY0diyKZoqcqiuQAu0tHqkEIEb6ZaFI/Bd/06sXC5OsUg8MK96/wtVlojOPRIhpUs8c5HnvD+jRK3zUjLXvpNiZxAzqDs729QA6qaSlDow/PgyzZWGzOfiAfIpEBvHtM9VvWk/Lo3+HqeErmqiEpfvOp6lEoo3CPoFUB0uxA6AGOhvQFOyAdShd0bOcj3GW6yzuIiYAjrF3mjTaVAvL3UJswJgzDuHC9KaH3jHbIGhNm2pzC2p+CI+EylPb1OFnJVva/5LE9fT781dwHYlT7ou8V91peGHG9Ds8Y7cMrq74ENvd7s35OB9GDGRAHIYUPL/IMR+a9Imw/OFAlcGiGKY96Amhw1IZ1S2dMBo9aB9poUpwywoE1Nh4DDTF+VBsg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Mar 07, 2024 at 09:14:50PM +0000, Matthew Wilcox wrote: > I suppose we could use a magic value for > page[0].mapcount to indicate hugetlb, but that'd make page_mapcount() > more complex. Actually, not significantly. I haven't tested this yet, but it should work ... thoughts? It's certainly the simplest implementation of folio_test_hugetlb() that we've ever had. diff --git a/include/linux/mm.h b/include/linux/mm.h index f5a97dec5169..32281097f07e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1212,6 +1212,9 @@ static inline int page_mapcount(struct page *page) { int mapcount = atomic_read(&page->_mapcount) + 1; + /* Head page repurposes mapcount; tail page leaves at -1 */ + if (PageHeadHuge(page)) + mapcount = 0; if (unlikely(PageCompound(page))) mapcount += folio_entire_mapcount(page_folio(page)); diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 735cddc13d20..6ebb573c7195 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -190,7 +190,6 @@ enum pageflags { /* At least one page in this folio has the hwpoison flag set */ PG_has_hwpoisoned = PG_error, - PG_hugetlb = PG_active, PG_large_rmappable = PG_workingset, /* anon or file-backed */ }; @@ -829,29 +828,6 @@ TESTPAGEFLAG_FALSE(LargeRmappable, large_rmappable) #define PG_head_mask ((1UL << PG_head)) -#ifdef CONFIG_HUGETLB_PAGE -int PageHuge(struct page *page); -SETPAGEFLAG(HugeTLB, hugetlb, PF_SECOND) -CLEARPAGEFLAG(HugeTLB, hugetlb, PF_SECOND) - -/** - * folio_test_hugetlb - Determine if the folio belongs to hugetlbfs - * @folio: The folio to test. - * - * Context: Any context. Caller should have a reference on the folio to - * prevent it from being turned into a tail page. - * Return: True for hugetlbfs folios, false for anon folios or folios - * belonging to other filesystems. - */ -static inline bool folio_test_hugetlb(struct folio *folio) -{ - return folio_test_large(folio) && - test_bit(PG_hugetlb, folio_flags(folio, 1)); -} -#else -TESTPAGEFLAG_FALSE(Huge, hugetlb) -#endif - #ifdef CONFIG_TRANSPARENT_HUGEPAGE /* * PageHuge() only returns true for hugetlbfs pages, but not for @@ -907,18 +883,6 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) TESTSCFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) #endif -/* - * Check if a page is currently marked HWPoisoned. Note that this check is - * best effort only and inherently racy: there is no way to synchronize with - * failing hardware. - */ -static inline bool is_page_hwpoison(struct page *page) -{ - if (PageHWPoison(page)) - return true; - return PageHuge(page) && PageHWPoison(compound_head(page)); -} - /* * For pages that are never mapped to userspace (and aren't PageSlab), * page_type may be used. Because it is initialised to -1, we invert the @@ -935,6 +899,7 @@ static inline bool is_page_hwpoison(struct page *page) #define PG_offline 0x00000100 #define PG_table 0x00000200 #define PG_guard 0x00000400 +#define PG_hugetlb 0x00000800 #define PageType(page, flag) \ ((page->page_type & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE) @@ -1026,6 +991,37 @@ PAGE_TYPE_OPS(Table, table, pgtable) */ PAGE_TYPE_OPS(Guard, guard, guard) +#ifdef CONFIG_HUGETLB_PAGE +PAGE_TYPE_OPS(HeadHuge, hugetlb, hugetlb) +#else +TESTPAGEFLAG_FALSE(HeadHuge, hugetlb) +#endif + +/** + * PageHuge - Determine if the page belongs to hugetlbfs + * @page: The page to test. + * + * Context: Any context. + * Return: True for hugetlbfs pages, false for anon pages or pages + * belonging to other filesystems. + */ +static inline bool PageHuge(struct page *page) +{ + return folio_test_hugetlb(page_folio(page)); +} + +/* + * Check if a page is currently marked HWPoisoned. Note that this check is + * best effort only and inherently racy: there is no way to synchronize with + * failing hardware. + */ +static inline bool is_page_hwpoison(struct page *page) +{ + if (PageHWPoison(page)) + return true; + return PageHuge(page) && PageHWPoison(compound_head(page)); +} + extern bool is_free_buddy_page(struct page *page); PAGEFLAG(Isolated, isolated, PF_ANY); @@ -1092,7 +1088,7 @@ static __always_inline void __ClearPageAnonExclusive(struct page *page) */ #define PAGE_FLAGS_SECOND \ (0xffUL /* order */ | 1UL << PG_has_hwpoisoned | \ - 1UL << PG_hugetlb | 1UL << PG_large_rmappable) + 1UL << PG_large_rmappable) #define PAGE_FLAGS_PRIVATE \ (1UL << PG_private | 1UL << PG_private_2) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index ed1581b670d4..23b62df21971 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1623,7 +1623,7 @@ static inline void __clear_hugetlb_destructor(struct hstate *h, { lockdep_assert_held(&hugetlb_lock); - folio_clear_hugetlb(folio); + __folio_clear_hugetlb(folio); } /* @@ -1710,7 +1710,7 @@ static void add_hugetlb_folio(struct hstate *h, struct folio *folio, h->surplus_huge_pages_node[nid]++; } - folio_set_hugetlb(folio); + __folio_set_hugetlb(folio); folio_change_private(folio, NULL); /* * We have to set hugetlb_vmemmap_optimized again as above @@ -2048,7 +2048,7 @@ static void __prep_account_new_huge_page(struct hstate *h, int nid) static void init_new_hugetlb_folio(struct hstate *h, struct folio *folio) { - folio_set_hugetlb(folio); + __folio_set_hugetlb(folio); INIT_LIST_HEAD(&folio->lru); hugetlb_set_folio_subpool(folio, NULL); set_hugetlb_cgroup(folio, NULL); @@ -2158,22 +2158,6 @@ static bool prep_compound_gigantic_folio_for_demote(struct folio *folio, return __prep_compound_gigantic_folio(folio, order, true); } -/* - * PageHuge() only returns true for hugetlbfs pages, but not for normal or - * transparent huge pages. See the PageTransHuge() documentation for more - * details. - */ -int PageHuge(struct page *page) -{ - struct folio *folio; - - if (!PageCompound(page)) - return 0; - folio = page_folio(page); - return folio_test_hugetlb(folio); -} -EXPORT_SYMBOL_GPL(PageHuge); - /* * Find and lock address space (mapping) in write mode. *