[PATCH 0/2] KHO Fixes

[PATCH 0/2] KHO Fixes

[Changyuan Lyu]

Hi,

This patchset contains 2 fixes for the KHO series
(https://lore.kernel.org/all/20250509074635.3187114-1-changyuanl@google.com/).

Best,
Changyuan

Changyuan Lyu (1):
  memblock: show a warning if allocation in KHO scratch fails

Pasha Tatashin (1):
  KHO: init new_physxa->phys_bits to fix lockdep

 kernel/kexec_handover.c | 29 +++++++++++++++++++++++++----
 mm/memblock.c           |  3 +++
 2 files changed, 28 insertions(+), 4 deletions(-)


base-commit: 9e619cd4fefd19cdce16e169d5827bc64ae01aa1
--
2.49.0.1101.gccaa498523-goog

[PATCH 1/2] memblock: show a warning if allocation in KHO scratch fails

[Changyuan Lyu]

When we kexec into a new kernel from an old kernel with KHO
enabled, the new kernel allocates vmemmap in the scratch area.
If the KHO scratch size is too small, vmemmap allocation would
fail and cause kernel panic, like the following,

[    0.027133] Faking a node at [mem 0x0000000000000000-0x00000004ffffffff]
[    0.027877] NODE_DATA(0) allocated [mem 0x4e4bd5a00-0x4e4bfffff]
[    0.029696] sparse_init_nid: node[0] memory map backing failed. Some memory will not be available.
[    0.029698] Zone ranges:
[    0.030974]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.031627]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.032281]   Normal   [mem 0x0000000100000000-0x00000004ffffffff]
[    0.032930]   Device   empty
[    0.033251] Movable zone start for each node
[    0.033710] Early memory node ranges
[    0.034108]   node   0: [mem 0x0000000000001000-0x000000000007ffff]
[    0.034801]   node   0: [mem 0x0000000000100000-0x00000000773fffff]
[    0.035461]   node   0: [mem 0x0000000077400000-0x00000000775fffff]
[    0.036116]   node   0: [mem 0x0000000077600000-0x000000007fffffff]
[    0.036768]   node   0: [mem 0x0000000100000000-0x00000004ccbfffff]
[    0.037423]   node   0: [mem 0x00000004ccc00000-0x00000004e4bfffff]
[    0.038111] BUG: kernel NULL pointer dereference, address: 0000000000000010
[    0.038880] #PF: supervisor write access in kernel mode
[    0.039474] #PF: error_code(0x0002) - not-present page
[    0.040056] PGD 0 P4D 0
[    0.040335] Oops: Oops: 0002 [#1] SMP
[    0.040745] CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.15.0-rc4+ #275 NONE
[    0.041541] RIP: 0010:__bitmap_set+0x2b/0x80
[    0.041992] Code: 0f 1e fa 55 48 89 e5 89 f1 89 f0 c1 e8 06 48 8d 04 c7 48 c7 c7 ff ff ff ff 48 d3 e7 41 89 f0 41 83 c8 c0 44 89 c6 01 d6 78 43 <48> 09 38 48 83 c0 08 83 fe 40 72 1a 41 8d 3c 10 83 c7 40 48 c7 00
[    0.043986] RSP: 0000:ffffffff96203df0 EFLAGS: 00010047
[    0.044546] RAX: 0000000000000010 RBX: 000000000000cc00 RCX: 0000000000000000
[    0.045311] RDX: 0000000000000040 RSI: 0000000000000000 RDI: ffffffffffffffff
[    0.046075] RBP: ffffffff96203df0 R08: 00000000ffffffc0 R09: ffffffff9626c950
[    0.046830] R10: 000000000002fffd R11: 0000000000000004 R12: 0000000000008000
[    0.047574] R13: 0000000000000000 R14: 000000000000003f R15: 000000000000009b
[    0.048313] FS:  0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
[    0.049151] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.049751] CR2: 0000000000000010 CR3: 00000004d123e000 CR4: 00000000000200b0
[    0.050494] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.051238] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[    0.051978] Call Trace:
[    0.052235]  <TASK>
[    0.052455]  subsection_map_init+0xe4/0x130
[    0.052891]  free_area_init+0x217/0x3d0
[    0.053290]  zone_sizes_init+0x5e/0x80
[    0.053682]  paging_init+0x27/0x30
[    0.054046]  setup_arch+0x307/0x3e0
[    0.054422]  start_kernel+0x59/0x390
[    0.054820]  x86_64_start_reservations+0x28/0x30
[    0.055307]  x86_64_start_kernel+0x70/0x80
[    0.055736]  common_startup_64+0x13b/0x140
[    0.056165]  </TASK>
[    0.056392] CR2: 0000000000000010
[    0.056737] ---[ end trace 0000000000000000 ]---
[    0.057218] RIP: 0010:__bitmap_set+0x2b/0x80
[    0.057667] Code: 0f 1e fa 55 48 89 e5 89 f1 89 f0 c1 e8 06 48 8d 04 c7 48 c7 c7 ff ff ff ff 48 d3 e7 41 89 f0 41 83 c8 c0 44 89 c6 01 d6 78 43 <48> 09 38 48 83 c0 08 83 fe 40 72 1a 41 8d 3c 10 83 c7 40 48 c7 00
[    0.059650] RSP: 0000:ffffffff96203df0 EFLAGS: 00010047
[    0.060218] RAX: 0000000000000010 RBX: 000000000000cc00 RCX: 0000000000000000
[    0.060985] RDX: 0000000000000040 RSI: 0000000000000000 RDI: ffffffffffffffff
[    0.061728] RBP: ffffffff96203df0 R08: 00000000ffffffc0 R09: ffffffff9626c950
[    0.062486] R10: 000000000002fffd R11: 0000000000000004 R12: 0000000000008000
[    0.063228] R13: 0000000000000000 R14: 000000000000003f R15: 000000000000009b
[    0.063968] FS:  0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
[    0.064812] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.065423] CR2: 0000000000000010 CR3: 00000004d123e000 CR4: 00000000000200b0
[    0.066175] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.066926] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[    0.067678] Kernel panic - not syncing: Attempted to kill the idle task!
[    0.068403] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---

The panic above can be easily reproduced by the following steps,

1.  boot a VM with 20GiB physical memory (or larger) and kernel command
    line "kho=on kho_scratch=2m,256m,128m"
2.  echo 1 > /sys/kernel/debug/kho/out/finalize
3.  kexec to a new kernel

The current panic log above is confusing and it's hard to find the
root cause.

Add an error log to make it easier to debug such kind of panics.

Fixes: d59f43b57480 ("memblock: add support for scratch memory")
Signed-off-by: Changyuan Lyu <changyuanl@google.com>
---
 mm/memblock.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/mm/memblock.c b/mm/memblock.c
index 154f1d73b61f..ed886bfd3de7 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -1573,6 +1573,9 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
 		goto again;
 	}

+	if (flags & MEMBLOCK_KHO_SCRATCH)
+		pr_err_once("Could not allocate %pap bytes in KHO scratch\n", &size);
+
 	return 0;

 done:
--
2.49.0.1101.gccaa498523-goog

[PATCH 2/2] KHO: init new_physxa->phys_bits to fix lockdep

[Changyuan Lyu]

From: Pasha Tatashin <pasha.tatashin@soleen.com>

Lockdep shows the following warning:

INFO: trying to register non-static key.
The code is fine but needs lockdep annotation, or maybe
you didn't initialize this object before use?
turning off the locking correctness validator.

[<ffffffff810133a6>] dump_stack_lvl+0x66/0xa0
[<ffffffff8136012c>] assign_lock_key+0x10c/0x120
[<ffffffff81358bb4>] register_lock_class+0xf4/0x2f0
[<ffffffff813597ff>] __lock_acquire+0x7f/0x2c40
[<ffffffff81360cb0>] ? __pfx_hlock_conflict+0x10/0x10
[<ffffffff811707be>] ? native_flush_tlb_global+0x8e/0xa0
[<ffffffff8117096e>] ? __flush_tlb_all+0x4e/0xa0
[<ffffffff81172fc2>] ? __kernel_map_pages+0x112/0x140
[<ffffffff813ec327>] ? xa_load_or_alloc+0x67/0xe0
[<ffffffff81359556>] lock_acquire+0xe6/0x280
[<ffffffff813ec327>] ? xa_load_or_alloc+0x67/0xe0
[<ffffffff8100b9e0>] _raw_spin_lock+0x30/0x40
[<ffffffff813ec327>] ? xa_load_or_alloc+0x67/0xe0
[<ffffffff813ec327>] xa_load_or_alloc+0x67/0xe0
[<ffffffff813eb4c0>] kho_preserve_folio+0x90/0x100
[<ffffffff813ebb7f>] __kho_finalize+0xcf/0x400
[<ffffffff813ebef4>] kho_finalize+0x34/0x70

This is becase xa has its own lock, that is not initialized in
xa_load_or_alloc.

Modifiy __kho_preserve_order(), to properly call
xa_init(&new_physxa->phys_bits);

Fixes: fc33e4b44b27 ("kexec: enable KHO support for memory preservation")
Signed-off-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Signed-off-by: Changyuan Lyu <changyuanl@google.com>
---
 kernel/kexec_handover.c | 29 +++++++++++++++++++++++++----
 1 file changed, 25 insertions(+), 4 deletions(-)

diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c
index 69b953551677..f0ac6a9170f8 100644
--- a/kernel/kexec_handover.c
+++ b/kernel/kexec_handover.c
@@ -144,14 +144,35 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
 				unsigned int order)
 {
 	struct kho_mem_phys_bits *bits;
-	struct kho_mem_phys *physxa;
+	struct kho_mem_phys *physxa, *new_physxa;
 	const unsigned long pfn_high = pfn >> order;

 	might_sleep();

-	physxa = xa_load_or_alloc(&track->orders, order, sizeof(*physxa));
-	if (IS_ERR(physxa))
-		return PTR_ERR(physxa);
+	physxa = xa_load(&track->orders, order);
+	if (!physxa) {
+		new_physxa = kzalloc(sizeof(*physxa), GFP_KERNEL);
+		if (!new_physxa)
+			return -ENOMEM;
+
+		xa_init(&new_physxa->phys_bits);
+		physxa = xa_cmpxchg(&track->orders, order, NULL, new_physxa,
+				    GFP_KERNEL);
+		if (xa_is_err(physxa)) {
+			int err_ret = xa_err(physxa);
+
+			xa_destroy(&new_physxa->phys_bits);
+			kfree(new_physxa);
+
+			return err_ret;
+		}
+		if (physxa) {
+			xa_destroy(&new_physxa->phys_bits);
+			kfree(new_physxa);
+		} else {
+			physxa = new_physxa;
+		}
+	}

 	bits = xa_load_or_alloc(&physxa->phys_bits, pfn_high / PRESERVE_BITS,
 				sizeof(*bits));
--
2.49.0.1101.gccaa498523-goog

Re: [PATCH 2/2] KHO: init new_physxa->phys_bits to fix lockdep

[Mike Rapoport]

On Sun, May 18, 2025 at 07:23:15AM -0700, Changyuan Lyu wrote:
> From: Pasha Tatashin <pasha.tatashin@soleen.com>
> 
> Lockdep shows the following warning:
> 
> INFO: trying to register non-static key.
> The code is fine but needs lockdep annotation, or maybe
> you didn't initialize this object before use?
> turning off the locking correctness validator.
> 
> [<ffffffff810133a6>] dump_stack_lvl+0x66/0xa0
> [<ffffffff8136012c>] assign_lock_key+0x10c/0x120
> [<ffffffff81358bb4>] register_lock_class+0xf4/0x2f0
> [<ffffffff813597ff>] __lock_acquire+0x7f/0x2c40
> [<ffffffff81360cb0>] ? __pfx_hlock_conflict+0x10/0x10
> [<ffffffff811707be>] ? native_flush_tlb_global+0x8e/0xa0
> [<ffffffff8117096e>] ? __flush_tlb_all+0x4e/0xa0
> [<ffffffff81172fc2>] ? __kernel_map_pages+0x112/0x140
> [<ffffffff813ec327>] ? xa_load_or_alloc+0x67/0xe0
> [<ffffffff81359556>] lock_acquire+0xe6/0x280
> [<ffffffff813ec327>] ? xa_load_or_alloc+0x67/0xe0
> [<ffffffff8100b9e0>] _raw_spin_lock+0x30/0x40
> [<ffffffff813ec327>] ? xa_load_or_alloc+0x67/0xe0
> [<ffffffff813ec327>] xa_load_or_alloc+0x67/0xe0
> [<ffffffff813eb4c0>] kho_preserve_folio+0x90/0x100
> [<ffffffff813ebb7f>] __kho_finalize+0xcf/0x400
> [<ffffffff813ebef4>] kho_finalize+0x34/0x70
> 
> This is becase xa has its own lock, that is not initialized in
> xa_load_or_alloc.
> 
> Modifiy __kho_preserve_order(), to properly call
> xa_init(&new_physxa->phys_bits);
> 
> Fixes: fc33e4b44b27 ("kexec: enable KHO support for memory preservation")
> Signed-off-by: Pasha Tatashin <pasha.tatashin@soleen.com>
> Signed-off-by: Changyuan Lyu <changyuanl@google.com>
> ---
>  kernel/kexec_handover.c | 29 +++++++++++++++++++++++++----
>  1 file changed, 25 insertions(+), 4 deletions(-)
> 
> diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c
> index 69b953551677..f0ac6a9170f8 100644
> --- a/kernel/kexec_handover.c
> +++ b/kernel/kexec_handover.c
> @@ -144,14 +144,35 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
>  				unsigned int order)
>  {
>  	struct kho_mem_phys_bits *bits;
> -	struct kho_mem_phys *physxa;
> +	struct kho_mem_phys *physxa, *new_physxa;
>  	const unsigned long pfn_high = pfn >> order;
> 
>  	might_sleep();
> 
> -	physxa = xa_load_or_alloc(&track->orders, order, sizeof(*physxa));
> -	if (IS_ERR(physxa))
> -		return PTR_ERR(physxa);
> +	physxa = xa_load(&track->orders, order);
> +	if (!physxa) {
> +		new_physxa = kzalloc(sizeof(*physxa), GFP_KERNEL);
> +		if (!new_physxa)
> +			return -ENOMEM;
> +
> +		xa_init(&new_physxa->phys_bits);
> +		physxa = xa_cmpxchg(&track->orders, order, NULL, new_physxa,
> +				    GFP_KERNEL);
> +		if (xa_is_err(physxa)) {
> +			int err_ret = xa_err(physxa);
> +
> +			xa_destroy(&new_physxa->phys_bits);
> +			kfree(new_physxa);
> +
> +			return err_ret;
> +		}
> +		if (physxa) {
> +			xa_destroy(&new_physxa->phys_bits);
> +			kfree(new_physxa);
> +		} else {
> +			physxa = new_physxa;
> +		}
> +	}

You are nearly duplicating xa_load_or_alloc() here. 
Is xa_destroy() is really needed here? In the end we destroying an empty
xarray.

Unless xa_destroy() is a must something like this would be simpler IMHO:

diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c
index ef21db6c59d5..4c8303fbf97a 100644
--- a/kernel/kexec_handover.c
+++ b/kernel/kexec_handover.c
@@ -91,10 +91,12 @@ struct kho_serialization {
 	struct khoser_mem_chunk *preserved_mem_map;
 };
 
-static void *xa_load_or_alloc(struct xarray *xa, unsigned long index, size_t sz)
+static void *xa_load_or_alloc(struct xarray *xa, unsigned long index, size_t sz,
+			      bool *new)
 {
 	void *elm, *res;
 
+	*new = false;
 	elm = xa_load(xa, index);
 	if (elm)
 		return elm;
@@ -112,6 +114,7 @@ static void *xa_load_or_alloc(struct xarray *xa, unsigned long index, size_t sz)
 		return res;
 	}
 
+	*new = true;
 	return elm;
 }
 
@@ -146,15 +149,18 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
 	struct kho_mem_phys_bits *bits;
 	struct kho_mem_phys *physxa;
 	const unsigned long pfn_high = pfn >> order;
+	bool new;
 
 	might_sleep();
 
-	physxa = xa_load_or_alloc(&track->orders, order, sizeof(*physxa));
+	physxa = xa_load_or_alloc(&track->orders, order, sizeof(*physxa), &new);
 	if (IS_ERR(physxa))
 		return PTR_ERR(physxa);
+	if (new)
+		xa_init(&physxa->phys_bits);
 
 	bits = xa_load_or_alloc(&physxa->phys_bits, pfn_high / PRESERVE_BITS,
-				sizeof(*bits));
+				sizeof(*bits), &new);
 	if (IS_ERR(bits))
 		return PTR_ERR(bits);
 
And if xa_destroy() is actually required, the allocation of new xarray
should be a helper function.

>  	bits = xa_load_or_alloc(&physxa->phys_bits, pfn_high / PRESERVE_BITS,
>  				sizeof(*bits));
> --
> 2.49.0.1101.gccaa498523-goog

-- 
Sincerely yours,
Mike.

Re: [PATCH 1/2] memblock: show a warning if allocation in KHO scratch fails

[Mike Rapoport]

On Sun, May 18, 2025 at 07:23:14AM -0700, Changyuan Lyu wrote:
> When we kexec into a new kernel from an old kernel with KHO
> enabled, the new kernel allocates vmemmap in the scratch area.
> If the KHO scratch size is too small, vmemmap allocation would
> fail and cause kernel panic, like the following,
> 
> [    0.027133] Faking a node at [mem 0x0000000000000000-0x00000004ffffffff]
> [    0.027877] NODE_DATA(0) allocated [mem 0x4e4bd5a00-0x4e4bfffff]
> [    0.029696] sparse_init_nid: node[0] memory map backing failed. Some memory will not be available.
> [    0.029698] Zone ranges:
> [    0.030974]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
> [    0.031627]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
> [    0.032281]   Normal   [mem 0x0000000100000000-0x00000004ffffffff]
> [    0.032930]   Device   empty
> [    0.033251] Movable zone start for each node
> [    0.033710] Early memory node ranges
> [    0.034108]   node   0: [mem 0x0000000000001000-0x000000000007ffff]
> [    0.034801]   node   0: [mem 0x0000000000100000-0x00000000773fffff]
> [    0.035461]   node   0: [mem 0x0000000077400000-0x00000000775fffff]
> [    0.036116]   node   0: [mem 0x0000000077600000-0x000000007fffffff]
> [    0.036768]   node   0: [mem 0x0000000100000000-0x00000004ccbfffff]
> [    0.037423]   node   0: [mem 0x00000004ccc00000-0x00000004e4bfffff]
> [    0.038111] BUG: kernel NULL pointer dereference, address: 0000000000000010
> [    0.038880] #PF: supervisor write access in kernel mode
> [    0.039474] #PF: error_code(0x0002) - not-present page
> [    0.040056] PGD 0 P4D 0
> [    0.040335] Oops: Oops: 0002 [#1] SMP
> [    0.040745] CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.15.0-rc4+ #275 NONE
> [    0.041541] RIP: 0010:__bitmap_set+0x2b/0x80
> [    0.041992] Code: 0f 1e fa 55 48 89 e5 89 f1 89 f0 c1 e8 06 48 8d 04 c7 48 c7 c7 ff ff ff ff 48 d3 e7 41 89 f0 41 83 c8 c0 44 89 c6 01 d6 78 43 <48> 09 38 48 83 c0 08 83 fe 40 72 1a 41 8d 3c 10 83 c7 40 48 c7 00
> [    0.043986] RSP: 0000:ffffffff96203df0 EFLAGS: 00010047
> [    0.044546] RAX: 0000000000000010 RBX: 000000000000cc00 RCX: 0000000000000000
> [    0.045311] RDX: 0000000000000040 RSI: 0000000000000000 RDI: ffffffffffffffff
> [    0.046075] RBP: ffffffff96203df0 R08: 00000000ffffffc0 R09: ffffffff9626c950
> [    0.046830] R10: 000000000002fffd R11: 0000000000000004 R12: 0000000000008000
> [    0.047574] R13: 0000000000000000 R14: 000000000000003f R15: 000000000000009b
> [    0.048313] FS:  0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
> [    0.049151] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    0.049751] CR2: 0000000000000010 CR3: 00000004d123e000 CR4: 00000000000200b0
> [    0.050494] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [    0.051238] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [    0.051978] Call Trace:
> [    0.052235]  <TASK>
> [    0.052455]  subsection_map_init+0xe4/0x130
> [    0.052891]  free_area_init+0x217/0x3d0
> [    0.053290]  zone_sizes_init+0x5e/0x80
> [    0.053682]  paging_init+0x27/0x30
> [    0.054046]  setup_arch+0x307/0x3e0
> [    0.054422]  start_kernel+0x59/0x390
> [    0.054820]  x86_64_start_reservations+0x28/0x30
> [    0.055307]  x86_64_start_kernel+0x70/0x80
> [    0.055736]  common_startup_64+0x13b/0x140
> [    0.056165]  </TASK>
> [    0.056392] CR2: 0000000000000010
> [    0.056737] ---[ end trace 0000000000000000 ]---
> [    0.057218] RIP: 0010:__bitmap_set+0x2b/0x80
> [    0.057667] Code: 0f 1e fa 55 48 89 e5 89 f1 89 f0 c1 e8 06 48 8d 04 c7 48 c7 c7 ff ff ff ff 48 d3 e7 41 89 f0 41 83 c8 c0 44 89 c6 01 d6 78 43 <48> 09 38 48 83 c0 08 83 fe 40 72 1a 41 8d 3c 10 83 c7 40 48 c7 00
> [    0.059650] RSP: 0000:ffffffff96203df0 EFLAGS: 00010047
> [    0.060218] RAX: 0000000000000010 RBX: 000000000000cc00 RCX: 0000000000000000
> [    0.060985] RDX: 0000000000000040 RSI: 0000000000000000 RDI: ffffffffffffffff
> [    0.061728] RBP: ffffffff96203df0 R08: 00000000ffffffc0 R09: ffffffff9626c950
> [    0.062486] R10: 000000000002fffd R11: 0000000000000004 R12: 0000000000008000
> [    0.063228] R13: 0000000000000000 R14: 000000000000003f R15: 000000000000009b
> [    0.063968] FS:  0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
> [    0.064812] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    0.065423] CR2: 0000000000000010 CR3: 00000004d123e000 CR4: 00000000000200b0
> [    0.066175] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [    0.066926] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [    0.067678] Kernel panic - not syncing: Attempted to kill the idle task!
> [    0.068403] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---
> 
> The panic above can be easily reproduced by the following steps,
> 
> 1.  boot a VM with 20GiB physical memory (or larger) and kernel command
>     line "kho=on kho_scratch=2m,256m,128m"
> 2.  echo 1 > /sys/kernel/debug/kho/out/finalize
> 3.  kexec to a new kernel

This can be reproduced without KHO, just squeeze the RAM size, boot with a huge
kernel and initrd and you'll get the same panic.

The issue is that sparse_init_nid() does not treat allocation failures as
fatal and just continues with some sections being unpopulated and then
subsection_map_init() presumes all the sections are valid.

This should be fixed in mm/sparse.c regardless of KHO, maybe as simple as 

diff --git a/mm/sparse.c b/mm/sparse.c
index 3c012cf83cc2..64d071f9f037 100644
--- a/mm/sparse.c
+++ b/mm/sparse.c
@@ -197,6 +197,10 @@ void __init subsection_map_init(unsigned long pfn, unsigned long nr_pages)
 		pfns = min(nr_pages, PAGES_PER_SECTION
 				- (pfn & ~PAGE_SECTION_MASK));
 		ms = __nr_to_section(nr);
+
+		if (!ms->section_mem_map)
+			continue;
+
 		subsection_mask_set(ms->usage->subsection_map, pfn, pfns);
 
 		pr_debug("%s: sec: %lu pfns: %lu set(%d, %d)\n", __func__, nr,
 
> The current panic log above is confusing and it's hard to find the
> root cause.
> 
> Add an error log to make it easier to debug such kind of panics.
> 
> Fixes: d59f43b57480 ("memblock: add support for scratch memory")
> Signed-off-by: Changyuan Lyu <changyuanl@google.com>
> ---
>  mm/memblock.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/mm/memblock.c b/mm/memblock.c
> index 154f1d73b61f..ed886bfd3de7 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -1573,6 +1573,9 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
>  		goto again;
>  	}
> 
> +	if (flags & MEMBLOCK_KHO_SCRATCH)
> +		pr_err_once("Could not allocate %pap bytes in KHO scratch\n", &size);
> +
>  	return 0;
> 
>  done:
> --
> 2.49.0.1101.gccaa498523-goog

-- 
Sincerely yours,
Mike.

Re: [PATCH 2/2] KHO: init new_physxa->phys_bits to fix lockdep

[Pasha Tatashin]

On Sun, May 18, 2025 at 11:51 AM Mike Rapoport <rppt@kernel.org> wrote:
>
> On Sun, May 18, 2025 at 07:23:15AM -0700, Changyuan Lyu wrote:
> > From: Pasha Tatashin <pasha.tatashin@soleen.com>
> >
> > Lockdep shows the following warning:
> >
> > INFO: trying to register non-static key.
> > The code is fine but needs lockdep annotation, or maybe
> > you didn't initialize this object before use?
> > turning off the locking correctness validator.
> >
> > [<ffffffff810133a6>] dump_stack_lvl+0x66/0xa0
> > [<ffffffff8136012c>] assign_lock_key+0x10c/0x120
> > [<ffffffff81358bb4>] register_lock_class+0xf4/0x2f0
> > [<ffffffff813597ff>] __lock_acquire+0x7f/0x2c40
> > [<ffffffff81360cb0>] ? __pfx_hlock_conflict+0x10/0x10
> > [<ffffffff811707be>] ? native_flush_tlb_global+0x8e/0xa0
> > [<ffffffff8117096e>] ? __flush_tlb_all+0x4e/0xa0
> > [<ffffffff81172fc2>] ? __kernel_map_pages+0x112/0x140
> > [<ffffffff813ec327>] ? xa_load_or_alloc+0x67/0xe0
> > [<ffffffff81359556>] lock_acquire+0xe6/0x280
> > [<ffffffff813ec327>] ? xa_load_or_alloc+0x67/0xe0
> > [<ffffffff8100b9e0>] _raw_spin_lock+0x30/0x40
> > [<ffffffff813ec327>] ? xa_load_or_alloc+0x67/0xe0
> > [<ffffffff813ec327>] xa_load_or_alloc+0x67/0xe0
> > [<ffffffff813eb4c0>] kho_preserve_folio+0x90/0x100
> > [<ffffffff813ebb7f>] __kho_finalize+0xcf/0x400
> > [<ffffffff813ebef4>] kho_finalize+0x34/0x70
> >
> > This is becase xa has its own lock, that is not initialized in
> > xa_load_or_alloc.
> >
> > Modifiy __kho_preserve_order(), to properly call
> > xa_init(&new_physxa->phys_bits);
> >
> > Fixes: fc33e4b44b27 ("kexec: enable KHO support for memory preservation")
> > Signed-off-by: Pasha Tatashin <pasha.tatashin@soleen.com>
> > Signed-off-by: Changyuan Lyu <changyuanl@google.com>
> > ---
> >  kernel/kexec_handover.c | 29 +++++++++++++++++++++++++----
> >  1 file changed, 25 insertions(+), 4 deletions(-)
> >
> > diff --git a/kernel/kexec_handover.c b/kernel/kexec_handover.c
> > index 69b953551677..f0ac6a9170f8 100644
> > --- a/kernel/kexec_handover.c
> > +++ b/kernel/kexec_handover.c
> > @@ -144,14 +144,35 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
> >                               unsigned int order)
> >  {
> >       struct kho_mem_phys_bits *bits;
> > -     struct kho_mem_phys *physxa;
> > +     struct kho_mem_phys *physxa, *new_physxa;
> >       const unsigned long pfn_high = pfn >> order;
> >
> >       might_sleep();
> >
> > -     physxa = xa_load_or_alloc(&track->orders, order, sizeof(*physxa));
> > -     if (IS_ERR(physxa))
> > -             return PTR_ERR(physxa);
> > +     physxa = xa_load(&track->orders, order);
> > +     if (!physxa) {
> > +             new_physxa = kzalloc(sizeof(*physxa), GFP_KERNEL);
> > +             if (!new_physxa)
> > +                     return -ENOMEM;
> > +
> > +             xa_init(&new_physxa->phys_bits);
> > +             physxa = xa_cmpxchg(&track->orders, order, NULL, new_physxa,
> > +                                 GFP_KERNEL);
> > +             if (xa_is_err(physxa)) {
> > +                     int err_ret = xa_err(physxa);
> > +
> > +                     xa_destroy(&new_physxa->phys_bits);
> > +                     kfree(new_physxa);
> > +
> > +                     return err_ret;
> > +             }
> > +             if (physxa) {
> > +                     xa_destroy(&new_physxa->phys_bits);
> > +                     kfree(new_physxa);
> > +             } else {
> > +                     physxa = new_physxa;
> > +             }
> > +     }
>
> You are nearly duplicating xa_load_or_alloc() here.
> Is xa_destroy() is really needed here? In the end we destroying an empty
> xarray.
>
> Unless xa_destroy() is a must something like this would be simpler IMHO:

I wanted to do proper xa_destroy(), as the whole point of this patch
is to satisfy lockdep, and do a proper xa_init(). The patch fixes a
warning in linux-next, and I think should be taken as is. We can do a
separate clean-up once the series lands, where xa_load_or_alloc()
could either take another argument, or split into two functions.

Pasha

Re: [PATCH 1/2] memblock: show a warning if allocation in KHO scratch fails

[Changyuan Lyu]

Hi Mike,

On Sun, May 18, 2025 at 19:07:02 +0300, Mike Rapoport <rppt@kernel.org> wrote:
> On Sun, May 18, 2025 at 07:23:14AM -0700, Changyuan Lyu wrote:
> > When we kexec into a new kernel from an old kernel with KHO
> > enabled, the new kernel allocates vmemmap in the scratch area.
> > If the KHO scratch size is too small, vmemmap allocation would
> > fail and cause kernel panic, like the following,
> >
> > [    0.027133] Faking a node at [mem 0x0000000000000000-0x00000004ffffffff]
> > [    0.027877] NODE_DATA(0) allocated [mem 0x4e4bd5a00-0x4e4bfffff]
> > [    0.029696] sparse_init_nid: node[0] memory map backing failed. Some memory will not be available.
> > [    0.029698] Zone ranges:
> > [    0.030974]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
> > [    0.031627]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
> > [    0.032281]   Normal   [mem 0x0000000100000000-0x00000004ffffffff]
> > [    0.032930]   Device   empty
> > [    0.033251] Movable zone start for each node
> > [    0.033710] Early memory node ranges
> > [    0.034108]   node   0: [mem 0x0000000000001000-0x000000000007ffff]
> > [    0.034801]   node   0: [mem 0x0000000000100000-0x00000000773fffff]
> > [    0.035461]   node   0: [mem 0x0000000077400000-0x00000000775fffff]
> > [    0.036116]   node   0: [mem 0x0000000077600000-0x000000007fffffff]
> > [    0.036768]   node   0: [mem 0x0000000100000000-0x00000004ccbfffff]
> > [    0.037423]   node   0: [mem 0x00000004ccc00000-0x00000004e4bfffff]
> > [    0.038111] BUG: kernel NULL pointer dereference, address: 0000000000000010
> > [    0.038880] #PF: supervisor write access in kernel mode
> > [    0.039474] #PF: error_code(0x0002) - not-present page
> > [    0.040056] PGD 0 P4D 0
> > [    0.040335] Oops: Oops: 0002 [#1] SMP
> > [    0.040745] CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.15.0-rc4+ #275 NONE
> > [    0.041541] RIP: 0010:__bitmap_set+0x2b/0x80
> > [    0.041992] Code: 0f 1e fa 55 48 89 e5 89 f1 89 f0 c1 e8 06 48 8d 04 c7 48 c7 c7 ff ff ff ff 48 d3 e7 41 89 f0 41 83 c8 c0 44 89 c6 01 d6 78 43 <48> 09 38 48 83 c0 08 83 fe 40 72 1a 41 8d 3c 10 83 c7 40 48 c7 00
> > [    0.043986] RSP: 0000:ffffffff96203df0 EFLAGS: 00010047
> > [    0.044546] RAX: 0000000000000010 RBX: 000000000000cc00 RCX: 0000000000000000
> > [    0.045311] RDX: 0000000000000040 RSI: 0000000000000000 RDI: ffffffffffffffff
> > [    0.046075] RBP: ffffffff96203df0 R08: 00000000ffffffc0 R09: ffffffff9626c950
> > [    0.046830] R10: 000000000002fffd R11: 0000000000000004 R12: 0000000000008000
> > [    0.047574] R13: 0000000000000000 R14: 000000000000003f R15: 000000000000009b
> > [    0.048313] FS:  0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
> > [    0.049151] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [    0.049751] CR2: 0000000000000010 CR3: 00000004d123e000 CR4: 00000000000200b0
> > [    0.050494] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [    0.051238] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > [    0.051978] Call Trace:
> > [    0.052235]  <TASK>
> > [    0.052455]  subsection_map_init+0xe4/0x130
> > [    0.052891]  free_area_init+0x217/0x3d0
> > [    0.053290]  zone_sizes_init+0x5e/0x80
> > [    0.053682]  paging_init+0x27/0x30
> > [    0.054046]  setup_arch+0x307/0x3e0
> > [    0.054422]  start_kernel+0x59/0x390
> > [    0.054820]  x86_64_start_reservations+0x28/0x30
> > [    0.055307]  x86_64_start_kernel+0x70/0x80
> > [    0.055736]  common_startup_64+0x13b/0x140
> > [    0.056165]  </TASK>
> > [    0.056392] CR2: 0000000000000010
> > [    0.056737] ---[ end trace 0000000000000000 ]---
> > [    0.057218] RIP: 0010:__bitmap_set+0x2b/0x80
> > [    0.057667] Code: 0f 1e fa 55 48 89 e5 89 f1 89 f0 c1 e8 06 48 8d 04 c7 48 c7 c7 ff ff ff ff 48 d3 e7 41 89 f0 41 83 c8 c0 44 89 c6 01 d6 78 43 <48> 09 38 48 83 c0 08 83 fe 40 72 1a 41 8d 3c 10 83 c7 40 48 c7 00
> > [    0.059650] RSP: 0000:ffffffff96203df0 EFLAGS: 00010047
> > [    0.060218] RAX: 0000000000000010 RBX: 000000000000cc00 RCX: 0000000000000000
> > [    0.060985] RDX: 0000000000000040 RSI: 0000000000000000 RDI: ffffffffffffffff
> > [    0.061728] RBP: ffffffff96203df0 R08: 00000000ffffffc0 R09: ffffffff9626c950
> > [    0.062486] R10: 000000000002fffd R11: 0000000000000004 R12: 0000000000008000
> > [    0.063228] R13: 0000000000000000 R14: 000000000000003f R15: 000000000000009b
> > [    0.063968] FS:  0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
> > [    0.064812] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [    0.065423] CR2: 0000000000000010 CR3: 00000004d123e000 CR4: 00000000000200b0
> > [    0.066175] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [    0.066926] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> > [    0.067678] Kernel panic - not syncing: Attempted to kill the idle task!
> > [    0.068403] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---
> >
> > The panic above can be easily reproduced by the following steps,
> >
> > 1.  boot a VM with 20GiB physical memory (or larger) and kernel command
> >     line "kho=on kho_scratch=2m,256m,128m"
> > 2.  echo 1 > /sys/kernel/debug/kho/out/finalize
> > 3.  kexec to a new kernel
>
> This can be reproduced without KHO, just squeeze the RAM size, boot with a huge
> kernel and initrd and you'll get the same panic.
>
> The issue is that sparse_init_nid() does not treat allocation failures as
> fatal and just continues with some sections being unpopulated and then
> subsection_map_init() presumes all the sections are valid.
>
> This should be fixed in mm/sparse.c regardless of KHO, maybe as simple as
>
> diff --git a/mm/sparse.c b/mm/sparse.c
> index 3c012cf83cc2..64d071f9f037 100644
> --- a/mm/sparse.c
> +++ b/mm/sparse.c
> @@ -197,6 +197,10 @@ void __init subsection_map_init(unsigned long pfn, unsigned long nr_pages)
>  		pfns = min(nr_pages, PAGES_PER_SECTION
>  				- (pfn & ~PAGE_SECTION_MASK));
>  		ms = __nr_to_section(nr);
> +
> +		if (!ms->section_mem_map)
> +			continue;
> +
>  		subsection_mask_set(ms->usage->subsection_map, pfn, pfns);
>
>  		pr_debug("%s: sec: %lu pfns: %lu set(%d, %d)\n", __func__, nr,

I tried your patch and the kernel log now looks like

[    0.027562] Faking a node at [mem 0x0000000000000000-0x000000057fffffff]
[    0.028338] NODE_DATA(0) allocated [mem 0x562bd5a00-0x562bfffff]
[    0.029201] Could not allocate 0x0000000014000000 bytes in KHO scratch
[    0.030229] sparse_init_nid: node[0] memory map backing failed. Some memory will not be available.
[    0.030232] Zone ranges:
[    0.031539]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.032242]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.032952]   Normal   [mem 0x0000000100000000-0x000000057fffffff]
[    0.033658]   Device   empty
[    0.033987] Movable zone start for each node
[    0.034473] Early memory node ranges
[    0.034878]   node   0: [mem 0x0000000000001000-0x000000000007ffff]
[    0.035591]   node   0: [mem 0x0000000000100000-0x00000000773fffff]
[    0.036308]   node   0: [mem 0x0000000077400000-0x00000000775fffff]
[    0.037030]   node   0: [mem 0x0000000077600000-0x000000007fffffff]
[    0.037750]   node   0: [mem 0x0000000100000000-0x000000054abfffff]
[    0.038463]   node   0: [mem 0x000000054ac00000-0x0000000562bfffff]
[    0.039180]   node   0: [mem 0x0000000562c00000-0x000000057fffffff]
[    0.039901] Initmem setup node 0 [mem 0x0000000000001000-0x000000057fffffff]
[    0.040707] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.041401] On node 0, zone DMA: 128 pages in unavailable ranges
[    0.221829] BUG: kernel NULL pointer dereference, address: 0000000000000018
[    0.222675] #PF: supervisor read access in kernel mode
[    0.223271] #PF: error_code(0x0000) - not-present page
[    0.223859] PGD 0 P4D 0
[    0.224152] Oops: Oops: 0000 [#1] SMP
[    0.224575] CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.15.0-rc4+ #279 NONE
[    0.225439] RIP: 0010:set_pageblock_migratetype+0x97/0xd0
[    0.226069] Code: b6 c9 c1 e1 04 48 01 c8 eb 02 31 c0 48 8b 70 08 89 f9 c1 e9 07 c1 ef 0d 83 e7 03 80 e1 3c 41 b8 07 00 00 00 49 d3 e0 48 d3 e2 <48> 8b 44 fe 18 49 f7 d0 48 89 c1 4c 21 c1 48 09 d1 f0 48 0f b1 4c
[    0.228231] RSP: 0000:ffffffffa4203d58 EFLAGS: 00010046
[    0.228834] RAX: ffff8e4722bd13b0 RBX: 0000000000000000 RCX: 0000000000009b00
[    0.229664] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000000
[    0.230487] RBP: ffffffffa4203d58 R08: 0000000000000007 R09: 0000000000000000
[    0.231303] R10: ffffffffa4edc610 R11: 000000000000000c R12: 000000000054ac00
[    0.232119] R13: 0017fff000000000 R14: 0000000000000002 R15: 00000000004d8000
[    0.232937] FS:  0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
[    0.233868] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.234529] CR2: 0000000000000018 CR3: 000000055923e000 CR4: 00000000000200b0
[    0.235351] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.236171] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[    0.236990] Call Trace:
[    0.237272]  <TASK>
[    0.237514]  memmap_init_range+0x1d8/0x210
[    0.237983]  memmap_init_zone_range+0x7f/0xb0
[    0.238488]  memmap_init+0x9a/0x120
[    0.238892]  free_area_init+0x369/0x3d0
[    0.239331]  zone_sizes_init+0x5e/0x80
[    0.239765]  paging_init+0x27/0x30
[    0.240153]  setup_arch+0x307/0x3e0
[    0.240556]  start_kernel+0x59/0x390
[    0.240968]  x86_64_start_reservations+0x28/0x30
[    0.241493]  x86_64_start_kernel+0x70/0x80
[    0.241962]  common_startup_64+0x13b/0x140
[    0.242433]  </TASK>
[    0.242682] CR2: 0000000000000018
[    0.243064] ---[ end trace 0000000000000000 ]---

It seems we are just defering the panic from subsection_map_init() to
memmap_init(). To me it is still not obvious that the failure was
caused by samll KHO scratch.

I think the error log in my original patch still makes sense since it
indicates potential panics early.

> > The current panic log above is confusing and it's hard to find the
> > root cause.
> >
> > Add an error log to make it easier to debug such kind of panics.
> >
> > Fixes: d59f43b57480 ("memblock: add support for scratch memory")
> > Signed-off-by: Changyuan Lyu <changyuanl@google.com>
> > ---
> >  mm/memblock.c | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/mm/memblock.c b/mm/memblock.c
> > index 154f1d73b61f..ed886bfd3de7 100644
> > --- a/mm/memblock.c
> > +++ b/mm/memblock.c
> > @@ -1573,6 +1573,9 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
> >  		goto again;
> >  	}
> >
> > +	if (flags & MEMBLOCK_KHO_SCRATCH)
> > +		pr_err_once("Could not allocate %pap bytes in KHO scratch\n", &size);
> > +
> >  	return 0;
> >
> >  done:
> > --
> > 2.49.0.1101.gccaa498523-goog
>
> --
> Sincerely yours,
> Mike.

Best,
Changyuan

Re: [PATCH 1/2] memblock: show a warning if allocation in KHO scratch fails

[Mike Rapoport]

Hi Changyuan,

On Wed, May 21, 2025 at 12:03:10AM -0700, Changyuan Lyu wrote:
> Hi Mike,
> 
> On Sun, May 18, 2025 at 19:07:02 +0300, Mike Rapoport <rppt@kernel.org> wrote:
> >
> > This can be reproduced without KHO, just squeeze the RAM size, boot with a huge
> > kernel and initrd and you'll get the same panic.
> >
> > The issue is that sparse_init_nid() does not treat allocation failures as
> > fatal and just continues with some sections being unpopulated and then
> > subsection_map_init() presumes all the sections are valid.
> >
> > This should be fixed in mm/sparse.c regardless of KHO, maybe as simple as
> >
> > diff --git a/mm/sparse.c b/mm/sparse.c
> > index 3c012cf83cc2..64d071f9f037 100644
> > --- a/mm/sparse.c
> > +++ b/mm/sparse.c
> > @@ -197,6 +197,10 @@ void __init subsection_map_init(unsigned long pfn, unsigned long nr_pages)
> >  		pfns = min(nr_pages, PAGES_PER_SECTION
> >  				- (pfn & ~PAGE_SECTION_MASK));
> >  		ms = __nr_to_section(nr);
> > +
> > +		if (!ms->section_mem_map)
> > +			continue;
> > +
> >  		subsection_mask_set(ms->usage->subsection_map, pfn, pfns);
> >
> >  		pr_debug("%s: sec: %lu pfns: %lu set(%d, %d)\n", __func__, nr,
> 
> I tried your patch and the kernel log now looks like
> 
> [    0.027562] Faking a node at [mem 0x0000000000000000-0x000000057fffffff]
> [    0.028338] NODE_DATA(0) allocated [mem 0x562bd5a00-0x562bfffff]
> [    0.029201] Could not allocate 0x0000000014000000 bytes in KHO scratch
> [    0.030229] sparse_init_nid: node[0] memory map backing failed. Some memory will not be available.
> [    0.030232] Zone ranges:
> [    0.031539]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
> [    0.032242]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
> [    0.032952]   Normal   [mem 0x0000000100000000-0x000000057fffffff]
> [    0.033658]   Device   empty
> [    0.033987] Movable zone start for each node
> [    0.034473] Early memory node ranges
> [    0.034878]   node   0: [mem 0x0000000000001000-0x000000000007ffff]
> [    0.035591]   node   0: [mem 0x0000000000100000-0x00000000773fffff]
> [    0.036308]   node   0: [mem 0x0000000077400000-0x00000000775fffff]
> [    0.037030]   node   0: [mem 0x0000000077600000-0x000000007fffffff]
> [    0.037750]   node   0: [mem 0x0000000100000000-0x000000054abfffff]
> [    0.038463]   node   0: [mem 0x000000054ac00000-0x0000000562bfffff]
> [    0.039180]   node   0: [mem 0x0000000562c00000-0x000000057fffffff]
> [    0.039901] Initmem setup node 0 [mem 0x0000000000001000-0x000000057fffffff]
> [    0.040707] On node 0, zone DMA: 1 pages in unavailable ranges
> [    0.041401] On node 0, zone DMA: 128 pages in unavailable ranges
> [    0.221829] BUG: kernel NULL pointer dereference, address: 0000000000000018
> [    0.222675] #PF: supervisor read access in kernel mode
> [    0.223271] #PF: error_code(0x0000) - not-present page
> [    0.223859] PGD 0 P4D 0
> [    0.224152] Oops: Oops: 0000 [#1] SMP
> [    0.224575] CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.15.0-rc4+ #279 NONE
> [    0.225439] RIP: 0010:set_pageblock_migratetype+0x97/0xd0
> [    0.226069] Code: b6 c9 c1 e1 04 48 01 c8 eb 02 31 c0 48 8b 70 08 89 f9 c1 e9 07 c1 ef 0d 83 e7 03 80 e1 3c 41 b8 07 00 00 00 49 d3 e0 48 d3 e2 <48> 8b 44 fe 18 49 f7 d0 48 89 c1 4c 21 c1 48 09 d1 f0 48 0f b1 4c
> [    0.228231] RSP: 0000:ffffffffa4203d58 EFLAGS: 00010046
> [    0.228834] RAX: ffff8e4722bd13b0 RBX: 0000000000000000 RCX: 0000000000009b00
> [    0.229664] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000000
> [    0.230487] RBP: ffffffffa4203d58 R08: 0000000000000007 R09: 0000000000000000
> [    0.231303] R10: ffffffffa4edc610 R11: 000000000000000c R12: 000000000054ac00
> [    0.232119] R13: 0017fff000000000 R14: 0000000000000002 R15: 00000000004d8000
> [    0.232937] FS:  0000000000000000(0000) GS:0000000000000000(0000) knlGS:0000000000000000
> [    0.233868] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    0.234529] CR2: 0000000000000018 CR3: 000000055923e000 CR4: 00000000000200b0
> [    0.235351] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [    0.236171] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [    0.236990] Call Trace:
> [    0.237272]  <TASK>
> [    0.237514]  memmap_init_range+0x1d8/0x210
> [    0.237983]  memmap_init_zone_range+0x7f/0xb0
> [    0.238488]  memmap_init+0x9a/0x120
> [    0.238892]  free_area_init+0x369/0x3d0
> [    0.239331]  zone_sizes_init+0x5e/0x80
> [    0.239765]  paging_init+0x27/0x30
> [    0.240153]  setup_arch+0x307/0x3e0
> [    0.240556]  start_kernel+0x59/0x390
> [    0.240968]  x86_64_start_reservations+0x28/0x30
> [    0.241493]  x86_64_start_kernel+0x70/0x80
> [    0.241962]  common_startup_64+0x13b/0x140
> [    0.242433]  </TASK>
> [    0.242682] CR2: 0000000000000018
> [    0.243064] ---[ end trace 0000000000000000 ]---
> 
> It seems we are just defering the panic from subsection_map_init() to
> memmap_init(). To me it is still not obvious that the failure was
> caused by samll KHO scratch.

Small KHO scratch only exposes the issue that from one side
sparse_init_nid() does not treat OOM condition as fatal and tries to
continue with hardly noticeable error message but from the other side, we
presume that all section data was properly allocated and access it.
 
> I think the error log in my original patch still makes sense since it
> indicates potential panics early.

This will add another barely noticeable message at the same place
sparse_init_nid() reports an error.
I don't see how it will be better.

I think we should just make sparse_init_nid() panic or at least change 
"sparse_init_nid: node[0] memory map backing failed. Some memory will not be available."
to something more visible and clear. 

> Best,
> Changyuan

-- 
Sincerely yours,
Mike.

Re: [PATCH 1/2] memblock: show a warning if allocation in KHO scratch fails

[Oscar Salvador]

On Wed, May 21, 2025 at 10:43:15AM +0300, Mike Rapoport wrote:
> I think we should just make sparse_init_nid() panic or at least change 
> "sparse_init_nid: node[0] memory map backing failed. Some memory will not be available."
> to something more visible and clear. 

Panicking the system seems a bit too harsh.
Those sections will not be initialized, and sure you will lose some memory,
but still.

I think that making sure that subsection_map_init() does not access
non-initialized values is enough.
Because wrt. error message, I am not sure it can get more clear that we
failed we allocate memory to back the section and so that section will
not be activated :-)

 

-- 
Oscar Salvador
SUSE Labs

Re: [PATCH 1/2] memblock: show a warning if allocation in KHO scratch fails

[Mike Rapoport]

On Wed, May 21, 2025 at 10:48:55AM +0200, Oscar Salvador wrote:
> On Wed, May 21, 2025 at 10:43:15AM +0300, Mike Rapoport wrote:
> > I think we should just make sparse_init_nid() panic or at least change 
> > "sparse_init_nid: node[0] memory map backing failed. Some memory will not be available."
> > to something more visible and clear. 
> 
> Panicking the system seems a bit too harsh.
> Those sections will not be initialized, and sure you will lose some memory,
> but still.
> 
> I think that making sure that subsection_map_init() does not access
> non-initialized values is enough.

It's not only subsection_map_init(), next failing one is
memmap_init_range() and maybe there's more, but we can audit and fix them.
I believe all those accesses are at init time because after system is
booted we are careful to avoid accessing absent sections.

> Because wrt. error message, I am not sure it can get more clear that we
> failed we allocate memory to back the section and so that section will
> not be activated :-)

Add a dump_stack()? ;-)

> -- 
> Oscar Salvador
> SUSE Labs
> 

-- 
Sincerely yours,
Mike.