From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794207850; b=FbAD9JdhU6N3c40NF0COm4GaH/+bNsvcIsnnDK34EhVrx4suYlWWoxiXyLkuxw/vzCZy nwbp9ZdGyY3LUtTfFcVdRUGTpRF+vF9cpOIcHQrTyogJzqeMtPEySpMlByFtfeSjG1NuR XG9TJKu3lrpOoVBGMiyg0fYmKQf6VrPn5xum1MHICiBCtqEElSv0qcfe8ARV7BBwBjSOk RgqoIywAtjVZGq+Npt5nv8xa0URitQZ1qPwoEQNl00PPj3R5UQJMotBJGwM11bx9YIhh5 YI1ArbJ1ip/MfMFfQ4YV4ULDiGWfDOeRe1nkPsqgLR6ZnViLuuMmlMhCA6bDfP9K7Ow== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794207850; h=Received:DKIM-Signature:Date:To:From:Subject:Message-Id; bh=HNGmKlZRgw7Q072dbNWP7rPlA4psIm5QPl+1aYCuVVE=; b=wNdiS6ip8ADsd0JSIQNwBCc1zeZstGI1OIh0t3LijK5qCOVKsp5I05Pho/HR6weoNAKh N7qaQujlVXZqq47oRRVXRNfKFkPecRRUV6wx6DeDdmIxehVq2gB79UQXEXGrNPirnRvHw EgHyZAYQ8PbUlYVyBOsX9P5gpR2CjZfzhNWtlQM3BL3D7/nINej7HeySyF0q+HOOpIMaY Bvu4qZt2HjOrasEAT4qLEd8PKkTUVy7fLpAOZ6SZvc1ROplWQAnR/qWlJZdUw/LwXlp4c 8mG4/dbWs2vXEpP3y36ncSs/ke68bNHfERzxTifcys5ZOqceXonqap9uXwRp7kNvCCg== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=linux-foundation.org header.i=@linux-foundation.org header.a=rsa-sha256 header.s=korg header.b=Z19nvU1C; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A4C0606A31C for ; Mon, 9 Nov 2026 07:04:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.a=rsa-sha256 header.s=korg header.b=Z19nvU1C Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3A50A1F0089F; Mon, 9 Nov 2026 07:04:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=korg; t=1794207850; bh=HNGmKlZRgw7Q072dbNWP7rPlA4psIm5QPl+1aYCuVVE=; h=Date:To:From:Subject; b=Z19nvU1CqbFT1+ZWpWH2Wc4/yBRVLXbSLUvPynItAQSfqsLzrwpgSSkqgbXgC1kll kEfHZBpwZA2LtjoPXp9c32cxhJPqtiDFZE5brb0ZC4rihudJPWxgA4y8rHHk/+4eUs +Q+I/AgvtzpwJuaO6IWWbePpv5rQZVsTRUiPtUT0= Date: Sun, 08 Nov 2026 23:04:09 -0800 To: mm-commits@vger.kernel.org,xueyuan.chen21@gmail.com,will@kernel.org, urezki@gmail.com,ryan.roberts@arm.com,rppt@kernel.org,npiggin@gmail.com, mpe@ellerman.id.au,maddy@linux.ibm.com,leo.yan@arm.com, jiangwenxiaomi@gmail.com,jiangwen6@xiaomi.com,dev.jain@arm.com, david@kernel.org,chleroy@kernel.org,catalin.marinas@arm.com, anshuman.khandual@arm.com,andrew+kernel@donnellan.id.au,baohua@kernel.org, akpm@linux-foundation.org From: Andrew Morton Subject: + mm-vmalloc-extend-page-table-walk-to-support-larger-page_shift-sizes-and-eliminate-page-table-rewalk.patch added to mm-new branch Message-Id: <20261109070410.3A50A1F0089F@smtp.kernel.org> X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list The patch titled Subject: mm/vmalloc: extend page table walk to support larger page_shift sizes and eliminate page table rewalk has been added to the -mm mm-new branch. Its filename is mm-vmalloc-extend-page-table-walk-to-support-larger-page_shift-sizes-and-eliminate-page-table-rewalk.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-vmalloc-extend-page-table-walk-to-support-larger-page_shift-sizes-and-eliminate-page-table-rewalk.patch This patch will later appear in the mm-new branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Note, mm-new is a provisional staging ground for work-in-progress patches, and acceptance into mm-new is a notification for others take notice and to finish up reviews. Please do not hesitate to respond to review feedback and post updated versions to replace or incrementally fixup patches in mm-new. The mm-new branch of mm.git is not included in linux-next If a few days of testing in mm-new is successful, the patch will me moved into mm.git's mm-unstable branch, which is included in linux-next Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via various branches at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there most days ------------------------------------------------------ From: "Barry Song (Xiaomi)" Subject: mm/vmalloc: extend page table walk to support larger page_shift sizes and eliminate page table rewalk Date: Sat, 07 Nov 2026 21:49:50 +0800 vmap_pages_range_noflush_walk() (formerly vmap_small_pages_range_noflush()) provides a clean interface by taking struct page **pages and mapping them via direct PTE iteration. This avoids the page table rewalk seen when using vmap_range_noflush() for page_shift values other than PAGE_SHIFT. Extend it to support larger page_shift values, and add PMD- and contiguous-PTE mappings as well. Rename it to vmap_pages_range_noflush_walk() since it now handles more than just small pages. For vmalloc() allocations with VM_ALLOW_HUGE_VMAP, we no longer need to iterate over pages one by one via vmap_range_noflush(), which would otherwise lead to page table rewalk. The code is now unified with the PAGE_SHIFT case by simply calling vmap_pages_range_noflush_walk(). The pages array may hold several physically discontiguous chunks that are mapped with the same @shift. If the virtual range is not aligned to the chunk size, a PMD boundary can split a chunk, and the walk resumes in the next PMD in the middle of it. arch_vmap_pte_range_map_size() only checks CONT_PTE_SIZE alignment there and could map past the end of the chunk, so clamp each PTE-level mapping to the remainder of the current chunk. Link: https://lore.kernel.org/20261107134953.748622-8-jiangwenxiaomi@gmail.com Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Wen Jiang Signed-off-by: Andrew Morton Tested-by: Xueyuan Chen Tested-by: Leo Yan Reviewed-by: Dev Jain Reviewed-by: Uladzislau Rezki (Sony) Cc: Catalin Marinas Cc: Will Deacon Cc: Ryan Roberts Cc: Anshuman Khandual Cc: David Hildenbrand Cc: Mike Rapoport Cc: Christophe Leroy Cc: Madhavan Srinivasan Cc: Michael Ellerman Cc: Nicholas Piggin Cc: Andrew Donnellan Cc: Wen Jiang --- mm/vmalloc.c | 88 ++++++++++++++++++++++++++++++------------------- 1 file changed, 54 insertions(+), 34 deletions(-) --- a/mm/vmalloc.c~mm-vmalloc-extend-page-table-walk-to-support-larger-page_shift-sizes-and-eliminate-page-table-rewalk +++ a/mm/vmalloc.c @@ -558,8 +558,11 @@ void vunmap_range(unsigned long addr, un static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { + const unsigned long chunk_pages = 1UL << (shift - PAGE_SHIFT); + unsigned long pfn, size, chunk_end; + unsigned int nr_pages; int err = 0; pte_t *pte; @@ -590,9 +593,13 @@ static int vmap_pages_pte_range(pmd_t *p break; } - set_pte_at(&init_mm, addr, pte, mk_pte(page, prot)); - (*nr)++; - } while (pte++, addr += PAGE_SIZE, addr != end); + pfn = page_to_pfn(page); + + /* A PMD boundary may split a chunk; never map past its end. */ + chunk_end = addr + ((ALIGN(pfn + 1, chunk_pages) - pfn) << PAGE_SHIFT); + size = vmap_set_ptes(pte, addr, min(end, chunk_end), pfn, prot, shift); + nr_pages = PFN_DOWN(size); + } while (pte += nr_pages, *nr += nr_pages, addr += size, addr != end); lazy_mmu_mode_disable(); *mask |= PGTBL_PTE_MODIFIED; @@ -602,60 +609,88 @@ static int vmap_pages_pte_range(pmd_t *p static int vmap_pages_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { pmd_t *pmd; unsigned long next; + int err; pmd = pmd_alloc_track(&init_mm, pud, addr, mask); if (!pmd) return -ENOMEM; do { next = pmd_addr_end(addr, end); - if (vmap_pages_pte_range(pmd, addr, next, prot, pages, nr, mask)) - return -ENOMEM; + + if (shift >= PMD_SHIFT) { + struct page *page = pages[*nr]; + phys_addr_t phys_addr; + + if (WARN_ON(!page)) + return -ENOMEM; + if (WARN_ON(!pfn_valid(page_to_pfn(page)))) + return -EINVAL; + + phys_addr = page_to_phys(page); + + if (vmap_try_huge_pmd(pmd, addr, next, phys_addr, prot, shift)) { + *mask |= PGTBL_PMD_MODIFIED; + *nr += 1U << (PMD_SHIFT - PAGE_SHIFT); + continue; + } + } + err = vmap_pages_pte_range(pmd, addr, next, prot, pages, nr, mask, shift); + if (err) + return err; } while (pmd++, addr = next, addr != end); return 0; } static int vmap_pages_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { pud_t *pud; unsigned long next; + int err; pud = pud_alloc_track(&init_mm, p4d, addr, mask); if (!pud) return -ENOMEM; do { next = pud_addr_end(addr, end); - if (vmap_pages_pmd_range(pud, addr, next, prot, pages, nr, mask)) - return -ENOMEM; + err = vmap_pages_pmd_range(pud, addr, next, prot, pages, nr, mask, shift); + if (err) + return err; } while (pud++, addr = next, addr != end); return 0; } static int vmap_pages_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { p4d_t *p4d; unsigned long next; + int err; p4d = p4d_alloc_track(&init_mm, pgd, addr, mask); if (!p4d) return -ENOMEM; do { next = p4d_addr_end(addr, end); - if (vmap_pages_pud_range(p4d, addr, next, prot, pages, nr, mask)) - return -ENOMEM; + err = vmap_pages_pud_range(p4d, addr, next, prot, pages, nr, mask, shift); + if (err) + return err; } while (p4d++, addr = next, addr != end); return 0; } -static int vmap_small_pages_range_noflush(unsigned long addr, unsigned long end, - pgprot_t prot, struct page **pages) +/* + * It can take an array of pages which are not all contiguous, but it + * may have contiguous chunks, as hinted by @shift. + */ +static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long end, + pgprot_t prot, struct page **pages, unsigned int shift) { unsigned long start = addr; pgd_t *pgd; @@ -670,7 +705,7 @@ static int vmap_small_pages_range_noflus next = pgd_addr_end(addr, end); if (pgd_bad(*pgd)) mask |= PGTBL_PGD_MODIFIED; - err = vmap_pages_p4d_range(pgd, addr, next, prot, pages, &nr, &mask); + err = vmap_pages_p4d_range(pgd, addr, next, prot, pages, &nr, &mask, shift); if (err) break; } while (pgd++, addr = next, addr != end); @@ -693,28 +728,13 @@ static int vmap_small_pages_range_noflus int __vmap_pages_range_noflush(unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, unsigned int page_shift) { - unsigned int i, nr = (end - addr) >> PAGE_SHIFT; - if (WARN_ON_ONCE(page_shift < PAGE_SHIFT)) return -EINVAL; - if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC) || - page_shift == PAGE_SHIFT) - return vmap_small_pages_range_noflush(addr, end, prot, pages); - - for (i = 0; i < nr; i += 1U << (page_shift - PAGE_SHIFT)) { - int err; - - err = vmap_range_noflush(addr, addr + (1UL << page_shift), - page_to_phys(pages[i]), prot, - page_shift); - if (err) - return err; + if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC)) + page_shift = PAGE_SHIFT; - addr += 1UL << page_shift; - } - - return 0; + return vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift); } int vmap_pages_range_noflush(unsigned long addr, unsigned long end, _ Patches currently in -mm which might be from baohua@kernel.org are arm64-vmalloc-allow-arch_vmap_pte_range_map_size-to-batch-multiple-cont_pte.patch mm-vmalloc-extend-page-table-walk-to-support-larger-page_shift-sizes-and-eliminate-page-table-rewalk.patch mm-vmalloc-map-contiguous-pages-in-batches-for-vmap-if-possible.patch mm-vmalloc-align-vm_area-so-vmap-can-batch-mappings.patch