+ mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region.patch added to mm-new branch

+ mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region.patch added to mm-new branch

[Andrew Morton]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 5907 bytes --]


The patch titled
     Subject: mm: introduce for_each_valid_pfn() and use it from reserve_bootmem_region()
has been added to the -mm mm-new branch.  Its filename is
     mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region.patch

This patch will shortly appear at
     https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region.patch

This patch will later appear in the mm-new branch at
    git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Before you just go and hit "reply", please:
   a) Consider who else should be cc'ed
   b) Prefer to cc a suitable mailing list as well
   c) Ideally: find the original patch on the mailing list and do a
      reply-to-all to that, adding suitable additional cc's

*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***

The -mm tree is included into linux-next via the mm-everything
branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
and is updated there every 2-3 working days

------------------------------------------------------
From: David Woodhouse <dwmw@amazon.co.uk>
Subject: mm: introduce for_each_valid_pfn() and use it from reserve_bootmem_region()
Date: Wed, 23 Apr 2025 14:33:37 +0100

Patch series "mm: Introduce for_each_valid_pfn()", v4.

There are cases where a nave loop over a PFN range, calling pfn_valid() on
each one, is horribly inefficient.  Ruihan Li reported the case where
memmap_init() iterates all the way from zero to a potentially large value
of ARCH_PFN_OFFSET, and we at Amazon found the reserve_bootmem_region()
one as it affects hypervisor live update.  Others are more cosmetic.

By introducing a for_each_valid_pfn() helper it can optimise away a lot of
pointless calls to pfn_valid(), skipping immediately to the next valid PFN
and also skipping *all* checks within a valid (sub)region according to the
granularity of the memory model in use.


This patch (of 7)

Especially since commit 9092d4f7a1f8 ("memblock: update initialization of
reserved pages"), the reserve_bootmem_region() function can spend a
significant amount of time iterating over every 4KiB PFN in a range,
calling pfn_valid() on each one, and ultimately doing absolutely nothing.

On a platform used for virtualization, with large NOMAP regions that
eventually get used for guest RAM, this leads to a significant increase in
steal time experienced during kexec for a live update.

Introduce for_each_valid_pfn() and use it from reserve_bootmem_region(). 
This implementation is precisely the same naïve loop that the functio
used to have, but subsequent commits will provide optimised versions for
FLATMEM and SPARSEMEM, and this version will remain for those
architectures which provide their own pfn_valid() implementation,
until/unless they also provide a matching for_each_valid_pfn().

Link: https://lkml.kernel.org/r/20250423133821.789413-1-dwmw2@infradead.org
Link: https://lkml.kernel.org/r/20250423133821.789413-2-dwmw2@infradead.org
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Ard Biesheuvel <ardb@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Marc Rutland <mark.rutland@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Ruihan Li <lrh2000@pku.edu.cn>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 include/linux/mmzone.h |   10 ++++++++++
 mm/mm_init.c           |   23 ++++++++++-------------
 2 files changed, 20 insertions(+), 13 deletions(-)

--- a/include/linux/mmzone.h~mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region
+++ a/include/linux/mmzone.h
@@ -2177,6 +2177,16 @@ void sparse_init(void);
 #define subsection_map_init(_pfn, _nr_pages) do {} while (0)
 #endif /* CONFIG_SPARSEMEM */
 
+/*
+ * Fallback case for when the architecture provides its own pfn_valid() but
+ * not a corresponding for_each_valid_pfn().
+ */
+#ifndef for_each_valid_pfn
+#define for_each_valid_pfn(_pfn, _start_pfn, _end_pfn)			\
+	for ((_pfn) = (_start_pfn); (_pfn) < (_end_pfn); (_pfn)++)	\
+		if (pfn_valid(_pfn))
+#endif
+
 #endif /* !__GENERATING_BOUNDS.H */
 #endif /* !__ASSEMBLY__ */
 #endif /* _LINUX_MMZONE_H */
--- a/mm/mm_init.c~mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region
+++ a/mm/mm_init.c
@@ -783,22 +783,19 @@ void __init_memblock init_deferred_page(
 void __meminit reserve_bootmem_region(phys_addr_t start,
 				      phys_addr_t end, int nid)
 {
-	unsigned long start_pfn = PFN_DOWN(start);
-	unsigned long end_pfn = PFN_UP(end);
+	unsigned long pfn;
 
-	for (; start_pfn < end_pfn; start_pfn++) {
-		if (pfn_valid(start_pfn)) {
-			struct page *page = pfn_to_page(start_pfn);
+	for_each_valid_pfn(pfn, PFN_DOWN(start), PFN_UP(end)) {
+		struct page *page = pfn_to_page(pfn);
 
-			__init_deferred_page(start_pfn, nid);
+		__init_deferred_page(start_pfn, nid);
 
-			/*
-			 * no need for atomic set_bit because the struct
-			 * page is not visible yet so nobody should
-			 * access it yet.
-			 */
-			__SetPageReserved(page);
-		}
+		/*
+		 * no need for atomic set_bit because the struct
+		 * page is not visible yet so nobody should
+		 * access it yet.
+		 */
+		__SetPageReserved(page);
 	}
 }
 
_

Patches currently in -mm which might be from dwmw@amazon.co.uk are

mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region.patch
mm-implement-for_each_valid_pfn-for-config_flatmem.patch
mm-implement-for_each_valid_pfn-for-config_sparsemem.patch
mm-pm-use-for_each_valid_pfn-in-kernel-power-snapshotc.patch
mm-x86-use-for_each_valid_pfn-from-__ioremap_check_ram.patch
mm-use-for_each_valid_pfn-in-memory_hotplug.patch
mm-mm_init-use-for_each_valid_pfn-in-init_unavailable_range.patch

Re: + mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region.patch added to mm-new branch

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 1930 bytes --]

On Wed, 2025-04-23 at 14:49 -0700, Andrew Morton wrote:
> The patch titled
>      Subject: mm: introduce for_each_valid_pfn() and use it from reserve_bootmem_region()
> has been added to the -mm mm-new branch.  Its filename is
>      mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region.patch
> 
> This patch will shortly appear at
>      https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region.patch
> 
> This patch will later appear in the mm-new branch at
>     git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
> 
> Before you just go and hit "reply", please:
>    a) Consider who else should be cc'ed
>    b) Prefer to cc a suitable mailing list as well
>    c) Ideally: find the original patch on the mailing list and do a
>       reply-to-all to that, adding suitable additional cc's
> 
> *** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
> 
> The -mm tree is included into linux-next via the mm-everything
> branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
> and is updated there every 2-3 working days
> 
> ------------------------------------------------------
> From: David Woodhouse <dwmw@amazon.co.uk>
> Subject: mm: introduce for_each_valid_pfn() and use it from reserve_bootmem_region()
> Date: Wed, 23 Apr 2025 14:33:37 +0100
> 
> Patch series "mm: Introduce for_each_valid_pfn()", v4.
> 
> There are cases where a nave loop over a PFN range, calling pfn_valid() on
> each one, is horribly inefficient. 

Hm, that definitely said 'naïve loop' when I sent it¹.

I don't even know how that kind of problem can happen any more. Surely
everything has been pure UTF-8 everywhere for *decades* by now? :)


¹ https://lkml.kernel.org/r/20250423133821.789413-1-dwmw2@infradead.org

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5069 bytes --]

Re: + mm-introduce-for_each_valid_pfn-and-use-it-from-reserve_bootmem_region.patch added to mm-new branch

[Andrew Morton]

On Wed, 23 Apr 2025 23:43:56 +0100 David Woodhouse <dwmw2@infradead.org> wrote:

> > There are cases where a nave loop over a PFN range, calling pfn_valid() on
> > each one, is horribly inefficient. 
> 
> Hm, that definitely said 'naïve loop' when I sent it¹.

I guess one of my eleventy scripts is stuck in the 90's.  I'll take a look.