From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACBE427603A for ; Wed, 9 Sep 2026 01:45:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788918330; cv=none; b=jJGgOD27Kq0u/joZOb19LkEGowA7w9ZaV1qUIpJR5wliuFn8N5cjtgWJnKGeqdLCgzV+q1yT3Vfb1YTmXh2QrMxz7mL7rSEXpsW5UxUJQiyC993hNoP/Umspm15Ko/8B4bpZCr01I/JJGGEWOejiSULZdDsIdV4dP8wlGqoPi9I= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788918330; c=relaxed/simple; bh=btEtQCjhneDzGDNodU4C4ZAKdUibzgAjy5jnP0BsfUg=; h=Date:To:From:Subject:Message-Id; b=jDmkzeoz1sDOlzyUsMpPAnbCM5wNi6JrHQbrB4NpPfMBVG6GWeoPj+YoEstN8ap8II/537tY9AgGePhPJjVv9Y1Z3Bf01ZSHjwR5ACspqUVi7Zj9w0dCPXdrXBDSSDMGSBGRwtKTlmS23YMKMwJ50yB05diE4PSsEWtUuDbIxP8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=2DUW+zsh; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="2DUW+zsh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3B2461F00A3A; Wed, 9 Sep 2026 01:45:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=korg; t=1788918328; bh=FDenMs4bQYNU1NdqwdIWEdBuzYKYSBZZ+cN63yhQqKw=; h=Date:To:From:Subject; b=2DUW+zshYe5Qm8Nz3PvH8MI3mfSEWhbrY7hbj5LKsXJXgdfw11faiAOrDRLnv0sqL G6Jso2JR61/28NHS2eIPYjlaOiqE7AJ5k3lPGXPG6n2i82s210eIftFs8LshPoKS+y UbEHuADqpX3jnCLafn29ZatQoP+V8erfDBanVPe4= Date: Tue, 08 Sep 2026 18:45:27 -0700 To: mm-commits@vger.kernel.org,rppt@kernel.org,rdunlap@infradead.org,qi.zheng@linux.dev,osalvador@suse.de,npiggin@gmail.com,mpe@ellerman.id.au,maddy@linux.ibm.com,ljs@kernel.org,david@kernel.org,corbet@lwn.net,songmuchun@bytedance.com,akpm@linux-foundation.org From: Andrew Morton Subject: + mm-sparse-vmemmap-factor-out-shared-vmemmap-tail-page-allocation.patch added to mm-new branch Message-Id: <20260909014528.3B2461F00A3A@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The patch titled Subject: mm/sparse-vmemmap: factor out shared vmemmap tail page allocation has been added to the -mm mm-new branch. Its filename is mm-sparse-vmemmap-factor-out-shared-vmemmap-tail-page-allocation.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-sparse-vmemmap-factor-out-shared-vmemmap-tail-page-allocation.patch This patch will later appear in the mm-new branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Note, mm-new is a provisional staging ground for work-in-progress patches, and acceptance into mm-new is a notification for others take notice and to finish up reviews. Please do not hesitate to respond to review feedback and post updated versions to replace or incrementally fixup patches in mm-new. The mm-new branch of mm.git is not included in linux-next If a few days of testing in mm-new is successful, the patch will me moved into mm.git's mm-unstable branch, which is included in linux-next Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via various branches at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there most days ------------------------------------------------------ From: Muchun Song Subject: mm/sparse-vmemmap: factor out shared vmemmap tail page allocation Date: Tue, 8 Sep 2026 11:03:26 +0800 HugeTLB and sparse-vmemmap each have their own helper to allocate the shared vmemmap tail page used by vmemmap optimization. Factor that logic into a common vmemmap_shared_tail_page() helper. It allocates the page through vmemmap_alloc_block(), and uses cmpxchg() to install the per-zone shared page. Expose zone->vmemmap_tails under CONFIG_SPARSEMEM_VMEMMAP to match the shared helper's build condition. This avoids a !CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION stub; when optimization is disabled, the array has no entries and the compiler folds away the unused paths, so no storage or runtime overhead is added. This removes duplicate allocation logic while still handling both the early boot and runtime paths through the same helper. Link: https://lore.kernel.org/20260908030335.96549-3-songmuchun@bytedance.com Signed-off-by: Muchun Song Acked-by: Qi Zheng Cc: David Hildenbrand Cc: Jonathan Corbet Cc: Lorenzo Stoakes Cc: Madhavan Srinivasan Cc: Michael Ellerman Cc: Mike Rapoport Cc: Nicholas Piggin Cc: Oscar Salvador Cc: Randy Dunlap Signed-off-by: Andrew Morton --- include/linux/mmzone.h | 2 - mm/hugetlb_vmemmap.c | 28 -------------- mm/sparse-vmemmap.c | 74 +++++++++++++++------------------------ mm/sparse.h | 1 4 files changed, 33 insertions(+), 72 deletions(-) --- a/include/linux/mmzone.h~mm-sparse-vmemmap-factor-out-shared-vmemmap-tail-page-allocation +++ a/include/linux/mmzone.h @@ -1156,7 +1156,7 @@ struct zone { /* Zone statistics */ atomic_long_t vm_stat[NR_VM_ZONE_STAT_ITEMS]; atomic_long_t vm_numa_event[NR_VM_NUMA_EVENT_ITEMS]; -#ifdef CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION +#ifdef CONFIG_SPARSEMEM_VMEMMAP struct page *vmemmap_tails[VMEMMAP_OPTIMIZATION_NR_ORDERS]; #endif } ____cacheline_internodealigned_in_smp; --- a/mm/hugetlb_vmemmap.c~mm-sparse-vmemmap-factor-out-shared-vmemmap-tail-page-allocation +++ a/mm/hugetlb_vmemmap.c @@ -493,32 +493,6 @@ static bool vmemmap_should_optimize_foli return true; } -static struct page *vmemmap_get_tail(unsigned int order, struct zone *zone) -{ - const unsigned int idx = order - VMEMMAP_OPTIMIZATION_MIN_ORDER; - struct page *tail, *p; - int node = zone_to_nid(zone); - - tail = READ_ONCE(zone->vmemmap_tails[idx]); - if (likely(tail)) - return tail; - - tail = alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0); - if (!tail) - return NULL; - - p = page_to_virt(tail); - for (int i = 0; i < PAGE_SIZE / sizeof(struct page); i++) - init_compound_tail(p + i, NULL, order, zone); - - if (cmpxchg(&zone->vmemmap_tails[idx], NULL, tail)) { - __free_page(tail); - tail = READ_ONCE(zone->vmemmap_tails[idx]); - } - - return tail; -} - static int __hugetlb_vmemmap_optimize_folio(const struct hstate *h, struct folio *folio, struct list_head *vmemmap_pages, @@ -535,7 +509,7 @@ static int __hugetlb_vmemmap_optimize_fo return ret; nid = folio_nid(folio); - vmemmap_tail = vmemmap_get_tail(h->order, folio_zone(folio)); + vmemmap_tail = vmemmap_shared_tail_page(h->order, folio_zone(folio)); if (!vmemmap_tail) return -ENOMEM; --- a/mm/sparse.h~mm-sparse-vmemmap-factor-out-shared-vmemmap-tail-page-allocation +++ a/mm/sparse.h @@ -139,6 +139,7 @@ static inline void sparse_sections_init( * mm/sparse-vmemmap.c */ #ifdef CONFIG_SPARSEMEM_VMEMMAP +struct page *vmemmap_shared_tail_page(unsigned int order, struct zone *zone); void sparse_init_subsection_map(void); int section_nr_vmemmap_pages(unsigned long pfn, unsigned long nr_pages, struct vmem_altmap *altmap, struct dev_pagemap *pgmap); --- a/mm/sparse-vmemmap.c~mm-sparse-vmemmap-factor-out-shared-vmemmap-tail-page-allocation +++ a/mm/sparse-vmemmap.c @@ -42,27 +42,13 @@ #include "mm_init.h" #include "sparse.h" -/* - * Allocate a block of memory to be used to back the virtual memory map - * or to back the page tables that are used to create the mapping. - * Uses the main allocators if they are available, else bootmem. - */ - -static void * __ref __earlyonly_bootmem_alloc(int node, - unsigned long size, - unsigned long align, - unsigned long goal) -{ - return memmap_alloc(size, align, goal, node, false); -} - -void * __meminit vmemmap_alloc_block(unsigned long size, int node) +void __ref *vmemmap_alloc_block(unsigned long size, int node) { /* If the main allocator is up use that, fallback to bootmem. */ if (slab_is_available()) { gfp_t gfp_mask = GFP_KERNEL|__GFP_RETRY_MAYFAIL|__GFP_NOWARN; int order = get_order(size); - static bool warned __meminitdata; + static bool warned; struct page *page; page = alloc_pages_node(node, gfp_mask, order); @@ -76,8 +62,7 @@ void * __meminit vmemmap_alloc_block(uns } return NULL; } else - return __earlyonly_bootmem_alloc(node, size, size, - __pa(MAX_DMA_ADDRESS)); + return memmap_alloc(size, size, __pa(MAX_DMA_ADDRESS), node, false); } static void * __meminit altmap_alloc_block_buf(unsigned long size, @@ -184,39 +169,40 @@ static void * __meminit vmemmap_alloc_bl return p; } -#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP -static __meminit struct page *vmemmap_get_tail(unsigned int order, struct zone *zone) +struct page __ref *vmemmap_shared_tail_page(unsigned int order, struct zone *zone) { - struct page *p, *tail; - unsigned int idx; - int node = zone_to_nid(zone); + void *addr; + struct page *page; + const unsigned int idx = order - VMEMMAP_OPTIMIZATION_MIN_ORDER; - if (WARN_ON_ONCE(order < VMEMMAP_OPTIMIZATION_MIN_ORDER)) - return NULL; - if (WARN_ON_ONCE(order > MAX_FOLIO_ORDER)) + if (WARN_ON_ONCE(idx >= ARRAY_SIZE(zone->vmemmap_tails))) return NULL; - idx = order - VMEMMAP_OPTIMIZATION_MIN_ORDER; - tail = zone->vmemmap_tails[idx]; - if (tail) - return tail; - p = vmemmap_alloc_block_zero(PAGE_SIZE, node); - if (!p) + page = READ_ONCE(zone->vmemmap_tails[idx]); + if (likely(page)) + return page; + + addr = vmemmap_alloc_block(PAGE_SIZE, zone_to_nid(zone)); + if (!addr) return NULL; - for (int i = 0; i < PAGE_SIZE / sizeof(struct page); i++) - init_compound_tail(p + i, NULL, order, zone); - tail = virt_to_page(p); - zone->vmemmap_tails[idx] = tail; + for (int i = 0; i < PAGE_SIZE / sizeof(struct page); i++) { + page = (struct page *)addr + i; + mm_zero_struct_page(page); + init_compound_tail(page, NULL, order, zone); + } - return tail; -} -#else -static inline struct page *vmemmap_get_tail(unsigned int order, struct zone *zone) -{ - return NULL; + page = virt_to_page(addr); + if (cmpxchg(&zone->vmemmap_tails[idx], NULL, page) != NULL) { + if (slab_is_available()) + __free_page(page); + else + memblock_free(addr, PAGE_SIZE); + page = READ_ONCE(zone->vmemmap_tails[idx]); + } + + return page; } -#endif static __meminit void *vmemmap_alloc_pte(unsigned long pfn, int node, struct vmem_altmap *altmap) @@ -229,7 +215,7 @@ static __meminit void *vmemmap_alloc_pte return vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap); zone = pfn_to_zone(pfn, node); - page = vmemmap_get_tail(order, zone); + page = vmemmap_shared_tail_page(order, zone); if (!page) return NULL; _ Patches currently in -mm which might be from songmuchun@bytedance.com are mm-sparse-relax-struct-mem_section-size-constraints.patch mm-sparse-vmemmap-rename-hvo-order-macros.patch mm-mm_init-skip-initializing-shared-vmemmap-tail-pages.patch mm-sparse-vmemmap-initialize-shared-tail-vmemmap-pages-on-allocation.patch mm-sparse-vmemmap-support-section-based-vmemmap-accounting.patch mm-mm_init-factor-out-pfn_to_zone.patch mm-sparse-vmemmap-move-helpers-ahead-of-future-callers.patch mm-sparse-vmemmap-support-section-based-vmemmap-optimization.patch mm-sparse-initialize-memory-sections-earlier.patch mm-hugetlb-switch-hugetlb-to-section-based-vmemmap-optimization.patch mm-sparse-vmemmap-remove-sparsemem_vmemmap_preinit-support.patch mm-sparse-inline-usemap-allocation-into-sparse_init_nid.patch mm-sparse-remove-section_map_size.patch mm-hugetlb-remove-huge_bootmem_hvo.patch mm-hugetlb-remove-huge_bootmem_cma.patch mm-hugetlb-localize-struct-huge_bootmem_page.patch mm-hugetlb-localize-huge_bootmem_zones_valid.patch mm-sparse-vmemmap-introduce-config_sparsemem_vmemmap_optimization.patch mm-sparse-vmemmap-factor-out-shared-vmemmap-tail-page-allocation.patch mm-sparse-vmemmap-open-code-init_compound_tail.patch mm-sparse-vmemmap-prepare-dax-vmemmap-population-for-section-orders.patch mm-sparse-vmemmap-set-section-order-for-device-dax.patch mm-sparse-vmemmap-switch-device-dax-to-shared-tail-vmemmap-pages.patch mm-sparse-vmemmap-move-hvo-helpers-to-a-public-header.patch powerpc-mm-switch-device-dax-to-shared-tail-vmemmap-pages.patch mm-sparse-vmemmap-drop-the-extra-tail-page-from-device-dax-reservation.patch mm-sparse-vmemmap-drop-unused-section_nr_vmemmap_pages-arguments.patch documentation-mm-update-dax-vmemmap-deduplication-docs.patch