From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 91E1EC624A5 for ; Mon, 31 Aug 2026 07:55:30 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4hYLr75F2Gz2y2f; Mon, 31 Aug 2026 17:55:03 +1000 (AEST) Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::52c" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1788162903; cv=none; b=cjzHjw7o21NBHDdrRswlzOcxTWKvLlfUNVKMXK4B+LaAv4xe6YcglwguN6uQJCRF3oOHAU3WzzX1dDr9xw6zsQi+76UeiCBT4SGm7xWHbJtebsuOjzDW+qfsHqt/i5xLMIDeBhMxJ985Lam03J+wa1AnRN9GqTfIuu8phfIFvz3aRUE2XKn8bavB51RWDbGZ+otWipCTsIAfBtnWbnaotXy9p2wqKezNIxudpUspkiB5hxAJqAhUGKu+2WwfX9c5DtMFgk2Tj/Tezo1PVFnXOOUsasMrtsy+LiZwcAHWIFKwzDJIkyztP1aVVBIx86Tkf1lOnKnC3lSki+nBXspnhQ== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1788162903; c=relaxed/relaxed; bh=4fLjmUTfKT3TRPIaKPcH4qoh94gQxw3yJaaLiOmIzFs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EkXyaI0S6DTYws9HJHuZR1dRZWrP7KFfbD4wWKenPnKUEYNHhsZIh82twbLFP3bJzqUlLeYr4klD06OXBmGFSksDIhnZs4kKfpRc1sL3E85x0FSvfz88u/2GdLNYNn/TMe3vTmzzrr5z0rRkjZXZgVXvU8Mj6Ag2VvVWutH4vRhf7EqOzRiMeZqrqdFLi3yA1oUDsxSsdRLJK501VnsmGmW0MBBNfo1MHDLcOOUPFmfNcUc7+8ztRecStSRVUYg3WpX9x60kipd13HejA9YuTyxDpXSXHjpk12H+MahYvgUm03Weaa39TkAW9H4Suo2OBmznbdCFKP0Yq2e9yUzccg== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=RmS4c+9A; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::52c; helo=mail-pg1-x52c.google.com; envelope-from=songmuchun@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=RmS4c+9A; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::52c; helo=mail-pg1-x52c.google.com; envelope-from=songmuchun@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4hYLr65VqSz2yFW for ; Mon, 31 Aug 2026 17:55:02 +1000 (AEST) Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-cb5b8572b70so3268212a12.2 for ; Mon, 31 Aug 2026 00:55:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1788162901; x=1788767701; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=4fLjmUTfKT3TRPIaKPcH4qoh94gQxw3yJaaLiOmIzFs=; b=RmS4c+9AVl0w/JY2KZTsaPoRuEwu798XhCS5CMn38o5jpq34xuq/bIUGBNHOiPfGyT 43UbkPfRY0Vz7Ps68VyKW5xLg5J9t4OuI/NOovL0sN9T0qm+MWaI4YXceMv/HDjLroqv o4MK8tqyrrpisAQYNv3R8dl1W8AWG3bcwc3B7OK7U0Lk42ttkAoBqCSl/2UNyPCDseeA BRN6Emvo0fd2x52DQkczTmGmE+RE85gYX4MHsztHFETn8JS3wg3uwgeACGSKuOGn20Y9 nKJpEmMakfN2b1vpsmJvEsbFMSkvM/yhtUaxJWf07GBN0kvIx8rWylkI22zXwvu9blNl 3FgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788162901; x=1788767701; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=4fLjmUTfKT3TRPIaKPcH4qoh94gQxw3yJaaLiOmIzFs=; b=nX3DJ079vljh8jZNHzhwdy1l4ikkpeeic3YM/LMXCKehxiq41h+dpNp2Fgu3v8DXaf I+Bt4Ndoc3XtfPbP9KDlkLJFm1sTTElp0A5MGIDUPeiIqL17umddgXn0nm5jx8Azlp+4 SF+buVpiSHjHUAfZ/FH+UYwgv4/UEos6yH1WOgK3EZfsdY/4eMs1Siy1L5FEmku0JqSF oIy2hgo91y2gx+EMbyGIxXVoyRcIRDh8e16dP8RONS+CG6TXh6a47XC0UpAX/0USJqMk C8NmB1kqVSvB9EnPNEgjNLz0A3ZHy0nGYyZJrEwUtnfGKRMmwji4FQQ3nSUsEHxJ1o4c 5Z/g== X-Forwarded-Encrypted: i=1; AHgh+RpjeZEHJoVkXOByLRAn6jrkSYUfuNkMBr0pOtaEu6Ejv+IRkin2MPGgRemlwiERnlLjf6wEiLM+3V0ljKY=@lists.ozlabs.org X-Gm-Message-State: AFuF++l99kDsRgdThlRrNxDgrvt4dn9gMCtwORZ8QuzYMGJuBrvMPrJv 6NnllGM4IWrPzknkmkftjSqK2dQzouey53Xb5TynIVPUaGDfOjF3JTCC5C1QZ1JIxpA= X-Gm-Gg: AR+sD126GaFDMWrYSsQiI36Tof4Ls9hdChkNINBksZRVfojaRA6NXS//wlZ+Kp/acNp 42QcRMBps+zZngZPq3Joznii4Vw5IsM4RzLfikOAftQ7cjrYgxllbfVr8XWYanxCQeGWsw6AleY nKq0GD9eF11SmJjTPFF1D+lGYaJ8KL59AwacVlIMdhp/4eK9knKJlRc2s1wrx7e543zghxbdxPj JiKvcWDqeuiGF3pM9hq5Z6dF/Z+LaCSCuibf0Z+alMx5s9kMVvx3fLcg2UxMJuKzxZNdC6zbaoQ QEpoe9ObXrabwiSyvnakCUdOqVIgyLxnXqdT2jC/eM1gwyqRXoPs0B7nPLpY8XPM3Wr0YTjRTAz h2eeaZnbiOI2EaI45+Bj9yftX64zGd3pegHe+oMlHl+W5snnlVLRd4uBdDChhMpp7ksPTM52U1E FXRF6iB+c8sYm2NZkP5yucY4Qk3MvGfBsq9lOSqO9hGhMTC75+U/8BNOv16MFV0Ma3Q5SsOdmEr BUgr/OgeBZpWek4EmTAB7gY X-Received: by 2002:a05:6a21:a09:b0:3b4:61f:1fec with SMTP id adf61e73a8af0-3d26668474amr44495998637.2.1788162900583; Mon, 31 Aug 2026 00:55:00 -0700 (PDT) Received: from G6L4RL2QG9.bytedance.net ([61.213.176.9]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-cc1f330f88csm3590256a12.10.2026.08.31.00.54.56 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 31 Aug 2026 00:55:00 -0700 (PDT) From: Muchun Song To: Andrew Morton , David Hildenbrand , Oscar Salvador , Madhavan Srinivasan , Michael Ellerman , Jonathan Corbet Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-doc@vger.kernel.org, Muchun Song , Lorenzo Stoakes , Mike Rapoport , Qi Zheng , Nicholas Piggin , Christophe Leroy , Randy Dunlap , Muchun Song Subject: [PATCH 07/11] mm/sparse-vmemmap: move HVO helpers to a public header Date: Mon, 31 Aug 2026 15:53:38 +0800 Message-ID: <20260831075342.57563-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260831075342.57563-1-songmuchun@bytedance.com> References: <20260831075342.57563-1-songmuchun@bytedance.com> X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Owner: List-Post: List-Archive: , List-Subscribe: , , List-Unsubscribe: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit The vmemmap optimization helpers currently live in mm/sparse.h, which is an internal MM header. That works for MM code, but prevents powerpc from using the same interfaces without including a private header. Move the declarations and inline helpers to include/linux/vmemmap-optimization.h. This is a preparatory change for powerpc, which has its own vmemmap optimization implementation and needs to use the HVO interfaces from architecture code. Signed-off-by: Muchun Song --- MAINTAINERS | 1 + include/linux/vmemmap-optimization.h | 88 ++++++++++++++++++++++++++++ mm/hugetlb.c | 2 +- mm/hugetlb_vmemmap.c | 2 +- mm/memory_hotplug.c | 1 + mm/sparse.h | 73 +---------------------- 6 files changed, 93 insertions(+), 74 deletions(-) create mode 100644 include/linux/vmemmap-optimization.h diff --git a/MAINTAINERS b/MAINTAINERS index a3167bcbe589..ccc7f75bbeb0 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12099,6 +12099,7 @@ F: Documentation/mm/hugetlbfs_reserv.rst F: Documentation/mm/vmemmap_dedup.rst F: fs/hugetlbfs/ F: include/linux/hugetlb.h +F: include/linux/vmemmap-optimization.h F: include/trace/events/hugetlbfs.h F: mm/hugetlb.c F: mm/hugetlb_cgroup.c diff --git a/include/linux/vmemmap-optimization.h b/include/linux/vmemmap-optimization.h new file mode 100644 index 000000000000..171553fabd47 --- /dev/null +++ b/include/linux/vmemmap-optimization.h @@ -0,0 +1,88 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * vmemmap-optimization.h + * + * Generic vmemmap optimization declarations. + * + * Author: Muchun Song + */ +#ifndef _LINUX_VMEMMAP_OPTIMIZATION_H +#define _LINUX_VMEMMAP_OPTIMIZATION_H + +#include + +#ifdef CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION +static inline unsigned int section_order(const struct mem_section *section) +{ + return section->order; +} + +static inline void section_set_order(struct mem_section *section, unsigned int order) +{ + VM_WARN_ON(section_order(section) && order && section_order(section) != order); + section->order = order; +} + +static inline void section_set_order_range(unsigned long pfn, unsigned long nr_pages, + unsigned int order) +{ + unsigned long section_nr = pfn_to_section_nr(pfn); + + if (!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SECTION)) + return; + + for (unsigned long i = 0; i < nr_pages / PAGES_PER_SECTION; i++) + section_set_order(__nr_to_section(section_nr + i), order); +} + +static inline unsigned int pfn_to_section_order(unsigned long pfn) +{ + return section_order(__pfn_to_section(pfn)); +} +#else +static inline unsigned int section_order(const struct mem_section *section) +{ + return 0; +} + +static inline void section_set_order(struct mem_section *section, unsigned int order) +{ +} + +static inline void section_set_order_range(unsigned long pfn, unsigned long nr_pages, + unsigned int order) +{ +} + +static inline unsigned int pfn_to_section_order(unsigned long pfn) +{ + return 0; +} +#endif /* CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION */ + +static inline bool vmemmap_optimizable_pfn(unsigned long pfn) +{ + const unsigned int order = pfn_to_section_order(pfn); + const unsigned long nr_pages = 1UL << order; + + if (!is_power_of_2(sizeof(struct page))) + return false; + + return (pfn & (nr_pages - 1)) >= VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES; +} + +static inline bool vmemmap_optimizable_order(unsigned int order) +{ + if (!IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION)) + return false; + + if (!is_power_of_2(sizeof(struct page))) + return false; + + return order >= VMEMMAP_OPTIMIZATION_MIN_ORDER; +} + +#ifdef CONFIG_SPARSEMEM_VMEMMAP +struct page *vmemmap_shared_tail_page(unsigned int order, struct zone *zone); +#endif /* CONFIG_SPARSEMEM_VMEMMAP */ +#endif /* _LINUX_VMEMMAP_OPTIMIZATION_H */ diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 8fa1bafa03d9..5aa9c0eaf2d0 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -38,6 +38,7 @@ #include #include #include +#include #include #include @@ -52,7 +53,6 @@ #include "hugetlb_cma.h" #include "hugetlb_internal.h" #include "mm_init.h" -#include "sparse.h" #include #define HUGE_BOOTMEM_ZONES_VALID BIT(0) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 4a57e6c3352c..25c4e7d2664c 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -15,10 +15,10 @@ #include #include #include +#include #include #include "hugetlb_vmemmap.h" -#include "sparse.h" #include "internal.h" /** diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index d28bafb6fd53..4f3387632883 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -43,6 +43,7 @@ #include "mm_init.h" #include "page_alloc.h" #include "shuffle.h" +#include "sparse.h" enum { MEMMAP_ON_MEMORY_DISABLE = 0, diff --git a/mm/sparse.h b/mm/sparse.h index 59b825df83b9..e511d99fc26b 100644 --- a/mm/sparse.h +++ b/mm/sparse.h @@ -9,77 +9,7 @@ #define __MM_SPARSE_H #include - -#ifdef CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION -static inline unsigned int section_order(const struct mem_section *section) -{ - return section->order; -} - -static inline void section_set_order(struct mem_section *section, unsigned int order) -{ - VM_WARN_ON(section_order(section) && order && section_order(section) != order); - section->order = order; -} - -static inline void section_set_order_range(unsigned long pfn, unsigned long nr_pages, - unsigned int order) -{ - unsigned long section_nr = pfn_to_section_nr(pfn); - - if (!IS_ALIGNED(pfn | nr_pages, PAGES_PER_SECTION)) - return; - - for (unsigned long i = 0; i < nr_pages / PAGES_PER_SECTION; i++) - section_set_order(__nr_to_section(section_nr + i), order); -} - -static inline unsigned int pfn_to_section_order(unsigned long pfn) -{ - return section_order(__pfn_to_section(pfn)); -} -#else -static inline unsigned int section_order(const struct mem_section *section) -{ - return 0; -} - -static inline void section_set_order(struct mem_section *section, unsigned int order) -{ -} - -static inline void section_set_order_range(unsigned long pfn, unsigned long nr_pages, - unsigned int order) -{ -} - -static inline unsigned int pfn_to_section_order(unsigned long pfn) -{ - return 0; -} -#endif - -static inline bool vmemmap_optimizable_pfn(unsigned long pfn) -{ - const unsigned int order = pfn_to_section_order(pfn); - const unsigned long nr_pages = 1UL << order; - - if (!is_power_of_2(sizeof(struct page))) - return false; - - return (pfn & (nr_pages - 1)) >= VMEMMAP_OPTIMIZATION_NR_STRUCT_PAGES; -} - -static inline bool vmemmap_optimizable_order(unsigned int order) -{ - if (!IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION)) - return false; - - if (!is_power_of_2(sizeof(struct page))) - return false; - - return order >= VMEMMAP_OPTIMIZATION_MIN_ORDER; -} +#include /* * mm/sparse.c @@ -139,7 +69,6 @@ static inline void sparse_sections_init(void) {} * mm/sparse-vmemmap.c */ #ifdef CONFIG_SPARSEMEM_VMEMMAP -struct page *vmemmap_shared_tail_page(unsigned int order, struct zone *zone); void sparse_init_subsection_map(void); int section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr_pages, struct vmem_altmap *altmap, struct dev_pagemap *pgmap); -- 2.54.0