[PATCH] kho: fix global scratch size calculation

[PATCH] kho: fix global scratch size calculation

[Sourabh Jain]

KHO calculates the global scratch size based on memblock-reserved kernel
memory. It passes NUMA_NO_NODE to memblock_reserved_kern_size() for
this calculation.

When memblock_reserved_kern_size() is called with NUMA_NO_NODE, it
counts both:

- memory reserved for a specific NUMA node
- memory reserved with NUMA_NO_NODE

KHO needs to distinguish between these two types of reservations.
When calculating the size of global scratch memory, KHO only needs to
account for reservations made with NUMA_NO_NODE. Reservations made for
a specific NUMA node must not be included in the global scratch size.

Add memblock_reserved_size_nid() to calculate reserved memory for a
given reservation type and NUMA node. When NUMA_NO_NODE is passed, it
counts only memory reserved with NUMA_NO_NODE.

Use the new API for lowmem, global, and per-node KHO scratch size
calculations. For lowmem and global scratch, count only memory
reservations that were made with NUMA_NO_NODE. For per-node scratch,
count only memory reservations that were made with the corresponding
NUMA node ID.

Remove memblock_reserved_hugetlb_size() since it has the same
implementation as the new API and differs only in the memblock
reservation flag being checked. The new API handles both kernel and
HugeTLB reservations through its reservation type argument.

Define the new helper as a static function in the KHO implementation,
since it is only used by KHO and has no users outside
kernel/liveupdate/kexec_handover.c.

On powerpc, the difference can be seen in the scratch_len values
reported by:

cat /sys/kernel/debug/kho/out/scratch_len

Before this change, the global scratch allocation was 0x12000000
(288 MB):

0x1000000   (16 MB)
0x12000000  (288 MB)  <- global allocation
0x5000000   (80 MB)

After this change, the global scratch allocation is 0xd000000
(208 MB):

0x1000000   (16 MB)
0xd000000   (208 MB)  <- global allocation
0x5000000   (80 MB)

The 80 MB difference is the per-node reservation that was previously
being included in the global allocation.

The same issue also affects lowmem scratch memory, but its impact is
limited because the lowmem scratch memory calculation is restricted to
the first 4G of memory. The changes also cover the lowmem scratch
memory case.

Cc: Alexander Graf <graf@amazon.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: George Guo <guodongtai@kylinos.cn>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Pratyush Yadav <pratyush@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
---
 include/linux/memblock.h           |  1 -
 kernel/liveupdate/kexec_handover.c | 49 ++++++++++++++++++++++--------
 mm/memblock.c                      | 22 --------------
 3 files changed, 37 insertions(+), 35 deletions(-)

diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index d62db9e776cf..678fe466529a 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -487,7 +487,6 @@ static inline __init_memblock bool memblock_bottom_up(void)
 phys_addr_t memblock_phys_mem_size(void);
 phys_addr_t memblock_reserved_size(void);
 phys_addr_t memblock_reserved_kern_size(phys_addr_t limit, int nid);
-phys_addr_t memblock_reserved_hugetlb_size(phys_addr_t limit, int nid);
 unsigned long memblock_estimated_nr_free_pages(void);
 phys_addr_t memblock_start_of_DRAM(void);
 phys_addr_t memblock_end_of_DRAM(void);
diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
index 7c4d86daf86d..dc809e1e768c 100644
--- a/kernel/liveupdate/kexec_handover.c
+++ b/kernel/liveupdate/kexec_handover.c
@@ -752,6 +752,31 @@ static int __init kho_parse_scratch_size(char *p)
 }
 early_param("kho_scratch", kho_parse_scratch_size);
 
+static phys_addr_t __init_memblock memblock_reserved_size_nid(phys_addr_t limit, int nid,
+							      enum memblock_flags region_type)
+{
+	struct memblock_region *r;
+	phys_addr_t total = 0;
+
+	for_each_reserved_mem_region(r) {
+		phys_addr_t size = r->size;
+
+		if (r->base > limit)
+			break;
+
+		if (r->base + r->size > limit)
+			size = limit - r->base;
+
+#ifdef CONFIG_NUMA
+		if (nid == memblock_get_region_node(r))
+#endif
+			if (r->flags & region_type)
+				total += size;
+	}
+
+	return total;
+}
+
 static void __init scratch_size_update(void)
 {
 	/*
@@ -762,17 +787,17 @@ static void __init scratch_size_update(void)
 	if (scratch_scale) {
 		phys_addr_t size;
 
-		size = memblock_reserved_kern_size(ARCH_LOW_ADDRESS_LIMIT,
-						   NUMA_NO_NODE);
-		size -= memblock_reserved_hugetlb_size(ARCH_LOW_ADDRESS_LIMIT,
-						       NUMA_NO_NODE);
+		size = memblock_reserved_size_nid(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE,
+						  MEMBLOCK_RSRV_KERN);
+		size -= memblock_reserved_size_nid(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE,
+						   MEMBLOCK_RSRV_HUGETLB);
 		size = size * scratch_scale / 100;
 		scratch_size_lowmem = size;
 
-		size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE,
-						   NUMA_NO_NODE);
-		size -= memblock_reserved_hugetlb_size(MEMBLOCK_ALLOC_ANYWHERE,
-						       NUMA_NO_NODE);
+		size = memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, NUMA_NO_NODE,
+						  MEMBLOCK_RSRV_KERN);
+		size -= memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, NUMA_NO_NODE,
+						   MEMBLOCK_RSRV_HUGETLB);
 		size = size * scratch_scale / 100 - scratch_size_lowmem;
 		scratch_size_global = size;
 	}
@@ -790,11 +815,11 @@ static phys_addr_t __init scratch_size_node(int nid)
 	phys_addr_t size;
 
 	if (scratch_scale) {
-		size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE,
-						   nid);
+		size = memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, nid,
+						  MEMBLOCK_RSRV_KERN);
 		/* Do not count HugeTLB pages. */
-		size -= memblock_reserved_hugetlb_size(MEMBLOCK_ALLOC_ANYWHERE,
-						       nid);
+		size -= memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, nid,
+						   MEMBLOCK_RSRV_HUGETLB);
 		size = size * scratch_scale / 100;
 	} else {
 		size = scratch_size_pernode;
diff --git a/mm/memblock.c b/mm/memblock.c
index 021db49eb7fc..9da748e774ea 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -1900,28 +1900,6 @@ phys_addr_t __init_memblock memblock_reserved_size(void)
 	return memblock.reserved.total_size;
 }
 
-phys_addr_t __init_memblock memblock_reserved_hugetlb_size(phys_addr_t limit, int nid)
-{
-	struct memblock_region *r;
-	phys_addr_t total = 0;
-
-	for_each_reserved_mem_region(r) {
-		phys_addr_t size = r->size;
-
-		if (r->base > limit)
-			break;
-
-		if (r->base + r->size > limit)
-			size = limit - r->base;
-
-		if (nid == memblock_get_region_node(r) || !numa_valid_node(nid))
-			if (r->flags & MEMBLOCK_RSRV_HUGETLB)
-				total += size;
-	}
-
-	return total;
-}
-
 phys_addr_t __init_memblock memblock_reserved_kern_size(phys_addr_t limit, int nid)
 {
 	struct memblock_region *r;
-- 
2.55.0

Re: [PATCH] kho: fix global scratch size calculation

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] kho: lowmem scratch exhaustion due to missing node-specific lowmem reservations
--

commit 45b67db1813edaa6fb1bdfe7033f0598a814cd63
Author: Sourabh Jain <sourabhjain@linux.ibm.com>

kho: fix global scratch size calculation

This commit updates the scratch size calculations to correctly distinguish
between memory reserved for a specific NUMA node and memory reserved with
NUMA_NO_NODE. It introduces memblock_reserved_size_nid() to calculate
reserved memory based on reservation type and NUMA node, fixing an issue
where per-node reservations were incorrectly included in the global scratch
allocation.

> diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
> index 7c4d86daf86d5..dc809e1e768c0 100644
> --- a/kernel/liveupdate/kexec_handover.c
> +++ b/kernel/liveupdate/kexec_handover.c
[ ... ]
> @@ -787,17 +787,17 @@ static void __init scratch_size_update(void)
>  	if (scratch_scale) {
>  		phys_addr_t size;
>  
> -		size = memblock_reserved_kern_size(ARCH_LOW_ADDRESS_LIMIT,
> -						   NUMA_NO_NODE);
> -		size -= memblock_reserved_hugetlb_size(ARCH_LOW_ADDRESS_LIMIT,
> -						       NUMA_NO_NODE);
> +		size = memblock_reserved_size_nid(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE,
> +						  MEMBLOCK_RSRV_KERN);

[Severity: High]
Does checking NUMA_NO_NODE here exclude node-specific lowmem reservations
(like node-bound DMA pools) from the lowmem scratch size calculation?

If the successor kernel requires lowmem for these node-specific reservations
during early boot, could shrinking the global lowmem scratch memory lead to
memblock allocation failures and a panic?

> +		size -= memblock_reserved_size_nid(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE,
> +						   MEMBLOCK_RSRV_HUGETLB);
>  		size = size * scratch_scale / 100;
>  		scratch_size_lowmem = size;
[ ... ]
> @@ -815,11 +815,11 @@ static phys_addr_t __init scratch_size_node(int nid)
>  	phys_addr_t size;
>  
>  	if (scratch_scale) {
> -		size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE,
> -						   nid);
> +		size = memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, nid,
> +						  MEMBLOCK_RSRV_KERN);

[Severity: High]
Are the node-specific lowmem reservations that were excluded from
scratch_size_lowmem now being counted here instead? Since this counts
with MEMBLOCK_ALLOC_ANYWHERE, will these reservations potentially be satisfied
using scratch memory in highmem?

If the successor kernel restricts its memblock allocations to
MEMBLOCK_KHO_SCRATCH regions, won't it be unable to recreate its
node-specific lowmem reservations if the per-node scratch memory was placed
in highmem?

>  		/* Do not count HugeTLB pages. */
> -		size -= memblock_reserved_hugetlb_size(MEMBLOCK_ALLOC_ANYWHERE,
> -						       nid);
> +		size -= memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, nid,
> +						   MEMBLOCK_RSRV_HUGETLB);
>  		size = size * scratch_scale / 100;
>  	} else {
>  		size = scratch_size_pernode;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260922131217.698809-1-sourabhjain@linux.ibm.com?part=1

Re: [PATCH] kho: fix global scratch size calculation

[Sourabh Jain]

On 22/09/26 18:57, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] kho: lowmem scratch exhaustion due to missing node-specific lowmem reservations
> --
>
> commit 45b67db1813edaa6fb1bdfe7033f0598a814cd63
> Author: Sourabh Jain <sourabhjain@linux.ibm.com>
>
> kho: fix global scratch size calculation
>
> This commit updates the scratch size calculations to correctly distinguish
> between memory reserved for a specific NUMA node and memory reserved with
> NUMA_NO_NODE. It introduces memblock_reserved_size_nid() to calculate
> reserved memory based on reservation type and NUMA node, fixing an issue
> where per-node reservations were incorrectly included in the global scratch
> allocation.
>
>> diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
>> index 7c4d86daf86d5..dc809e1e768c0 100644
>> --- a/kernel/liveupdate/kexec_handover.c
>> +++ b/kernel/liveupdate/kexec_handover.c
> [ ... ]
>> @@ -787,17 +787,17 @@ static void __init scratch_size_update(void)
>>   	if (scratch_scale) {
>>   		phys_addr_t size;
>>   
>> -		size = memblock_reserved_kern_size(ARCH_LOW_ADDRESS_LIMIT,
>> -						   NUMA_NO_NODE);
>> -		size -= memblock_reserved_hugetlb_size(ARCH_LOW_ADDRESS_LIMIT,
>> -						       NUMA_NO_NODE);
>> +		size = memblock_reserved_size_nid(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE,
>> +						  MEMBLOCK_RSRV_KERN);
> [Severity: High]
> Does checking NUMA_NO_NODE here exclude node-specific lowmem reservations
> (like node-bound DMA pools) from the lowmem scratch size calculation?

No. Node-specific reservations are handled separately. The node-specific
scratch memory size is calculated in scratch_size_node(), so if the kernel
has made a node-specific allocation, KHO will reserve scratch memory on
the corresponding node.

> If the successor kernel requires lowmem for these node-specific reservations
> during early boot, could shrinking the global lowmem scratch memory lead to
> memblock allocation failures and a panic?


With this fix, KHO still reserves both per-node scratch memory and
lowmem scratch memory to meet the needs of the successor kernel.

Per-node scratch memory is reserved on the corresponding node.
The only change is that per-node allocations made in lowmem (below 4G 
by-default)
are no longer counted when calculating the lowmem scratch size.


>> +		size -= memblock_reserved_size_nid(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE,
>> +						   MEMBLOCK_RSRV_HUGETLB);
>>   		size = size * scratch_scale / 100;
>>   		scratch_size_lowmem = size;
> [ ... ]
>> @@ -815,11 +815,11 @@ static phys_addr_t __init scratch_size_node(int nid)
>>   	phys_addr_t size;
>>   
>>   	if (scratch_scale) {
>> -		size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE,
>> -						   nid);
>> +		size = memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, nid,
>> +						  MEMBLOCK_RSRV_KERN);
> [Severity: High]
> Are the node-specific lowmem reservations that were excluded from
> scratch_size_lowmem now being counted here instead? Since this counts
> with MEMBLOCK_ALLOC_ANYWHERE, will these reservations potentially be satisfied
> using scratch memory in highmem?

No.

>
> If the successor kernel restricts its memblock allocations to
> MEMBLOCK_KHO_SCRATCH regions, won't it be unable to recreate its
> node-specific lowmem reservations if the per-node scratch memory was placed
> in highmem?

The per-node scratch memory will be allocated on the corresponding
node. Therefore, the successor kernel should be able to recreate its
node-specific reservations, even if the per-node scratch memory is
placed in highmem.



>
>>   		/* Do not count HugeTLB pages. */
>> -		size -= memblock_reserved_hugetlb_size(MEMBLOCK_ALLOC_ANYWHERE,
>> -						       nid);
>> +		size -= memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, nid,
>> +						   MEMBLOCK_RSRV_HUGETLB);
>>   		size = size * scratch_scale / 100;
>>   	} else {
>>   		size = scratch_size_pernode;

Re: [PATCH] kho: fix global scratch size calculation

[Mike Rapoport]

Hi Sourabh,

On Tue, Sep 22, 2026 at 06:42:16PM +0530, Sourabh Jain wrote:
> KHO calculates the global scratch size based on memblock-reserved kernel
> memory. It passes NUMA_NO_NODE to memblock_reserved_kern_size() for
> this calculation.
> 
> When memblock_reserved_kern_size() is called with NUMA_NO_NODE, it
> counts both:
> 
> - memory reserved for a specific NUMA node
> - memory reserved with NUMA_NO_NODE
> 
> KHO needs to distinguish between these two types of reservations.
> When calculating the size of global scratch memory, KHO only needs to
> account for reservations made with NUMA_NO_NODE. Reservations made for
> a specific NUMA node must not be included in the global scratch size.
> 
> Add memblock_reserved_size_nid() to calculate reserved memory for a
> given reservation type and NUMA node. When NUMA_NO_NODE is passed, it
> counts only memory reserved with NUMA_NO_NODE.
> 
> Use the new API for lowmem, global, and per-node KHO scratch size
> calculations. For lowmem and global scratch, count only memory
> reservations that were made with NUMA_NO_NODE. For per-node scratch,
> count only memory reservations that were made with the corresponding
> NUMA node ID.
> 
> Remove memblock_reserved_hugetlb_size() since it has the same
> implementation as the new API and differs only in the memblock
> reservation flag being checked. The new API handles both kernel and
> HugeTLB reservations through its reservation type argument.
> 
> Define the new helper as a static function in the KHO implementation,
> since it is only used by KHO and has no users outside
> kernel/liveupdate/kexec_handover.c.
> 
> On powerpc, the difference can be seen in the scratch_len values
> reported by:
> 
> cat /sys/kernel/debug/kho/out/scratch_len
> 
> Before this change, the global scratch allocation was 0x12000000
> (288 MB):
> 
> 0x1000000   (16 MB)
> 0x12000000  (288 MB)  <- global allocation
> 0x5000000   (80 MB)
> 
> After this change, the global scratch allocation is 0xd000000
> (208 MB):
> 
> 0x1000000   (16 MB)
> 0xd000000   (208 MB)  <- global allocation
> 0x5000000   (80 MB)
> 
> The 80 MB difference is the per-node reservation that was previously
> being included in the global allocation.
> 
> The same issue also affects lowmem scratch memory, but its impact is
> limited because the lowmem scratch memory calculation is restricted to
> the first 4G of memory. The changes also cover the lowmem scratch
> memory case.
> 
> Cc: Alexander Graf <graf@amazon.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: George Guo <guodongtai@kylinos.cn>
> Cc: Mike Rapoport <rppt@kernel.org>
> Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
> Cc: Pratyush Yadav <pratyush@kernel.org>
> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
> Cc: linux-kernel@vger.kernel.org
> Cc: linux-mm@kvack.org
> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
> ---
>  include/linux/memblock.h           |  1 -
>  kernel/liveupdate/kexec_handover.c | 49 ++++++++++++++++++++++--------
>  mm/memblock.c                      | 22 --------------
>  3 files changed, 37 insertions(+), 35 deletions(-)
> 
> diff --git a/include/linux/memblock.h b/include/linux/memblock.h
> index d62db9e776cf..678fe466529a 100644
> --- a/include/linux/memblock.h
> +++ b/include/linux/memblock.h
> @@ -487,7 +487,6 @@ static inline __init_memblock bool memblock_bottom_up(void)
>  phys_addr_t memblock_phys_mem_size(void);
>  phys_addr_t memblock_reserved_size(void);
>  phys_addr_t memblock_reserved_kern_size(phys_addr_t limit, int nid);
> -phys_addr_t memblock_reserved_hugetlb_size(phys_addr_t limit, int nid);
>  unsigned long memblock_estimated_nr_free_pages(void);
>  phys_addr_t memblock_start_of_DRAM(void);
>  phys_addr_t memblock_end_of_DRAM(void);
> diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
> index 7c4d86daf86d..dc809e1e768c 100644
> --- a/kernel/liveupdate/kexec_handover.c
> +++ b/kernel/liveupdate/kexec_handover.c
> @@ -752,6 +752,31 @@ static int __init kho_parse_scratch_size(char *p)
>  }
>  early_param("kho_scratch", kho_parse_scratch_size);
>  
> +static phys_addr_t __init_memblock memblock_reserved_size_nid(phys_addr_t limit, int nid,
> +							      enum memblock_flags region_type)
> +{

Why is this in kexec_handover.c and not in memblock.c?

> +	struct memblock_region *r;
> +	phys_addr_t total = 0;
> +
> +	for_each_reserved_mem_region(r) {
> +		phys_addr_t size = r->size;
> +
> +		if (r->base > limit)
> +			break;
> +
> +		if (r->base + r->size > limit)
> +			size = limit - r->base;
> +
> +#ifdef CONFIG_NUMA
> +		if (nid == memblock_get_region_node(r))
> +#endif

Do we really need the #ifdef here?

> +			if (r->flags & region_type)
> +				total += size;
> +	}
> +
> +	return total;
> +}
> +
>  static void __init scratch_size_update(void)
>  {
>  	/*

-- 
Sincerely yours,
Mike.

Re: [PATCH] kho: fix global scratch size calculation

[Sourabh Jain]

Hello Mike,

On 26/09/26 14:36, Mike Rapoport wrote:
> Hi Sourabh,
>
> On Tue, Sep 22, 2026 at 06:42:16PM +0530, Sourabh Jain wrote:
>> KHO calculates the global scratch size based on memblock-reserved kernel
>> memory. It passes NUMA_NO_NODE to memblock_reserved_kern_size() for
>> this calculation.
>>
>> When memblock_reserved_kern_size() is called with NUMA_NO_NODE, it
>> counts both:
>>
>> - memory reserved for a specific NUMA node
>> - memory reserved with NUMA_NO_NODE
>>
>> KHO needs to distinguish between these two types of reservations.
>> When calculating the size of global scratch memory, KHO only needs to
>> account for reservations made with NUMA_NO_NODE. Reservations made for
>> a specific NUMA node must not be included in the global scratch size.
>>
>> Add memblock_reserved_size_nid() to calculate reserved memory for a
>> given reservation type and NUMA node. When NUMA_NO_NODE is passed, it
>> counts only memory reserved with NUMA_NO_NODE.
>>
>> Use the new API for lowmem, global, and per-node KHO scratch size
>> calculations. For lowmem and global scratch, count only memory
>> reservations that were made with NUMA_NO_NODE. For per-node scratch,
>> count only memory reservations that were made with the corresponding
>> NUMA node ID.
>>
>> Remove memblock_reserved_hugetlb_size() since it has the same
>> implementation as the new API and differs only in the memblock
>> reservation flag being checked. The new API handles both kernel and
>> HugeTLB reservations through its reservation type argument.
>>
>> Define the new helper as a static function in the KHO implementation,
>> since it is only used by KHO and has no users outside
>> kernel/liveupdate/kexec_handover.c.
>>
>> On powerpc, the difference can be seen in the scratch_len values
>> reported by:
>>
>> cat /sys/kernel/debug/kho/out/scratch_len
>>
>> Before this change, the global scratch allocation was 0x12000000
>> (288 MB):
>>
>> 0x1000000   (16 MB)
>> 0x12000000  (288 MB)  <- global allocation
>> 0x5000000   (80 MB)
>>
>> After this change, the global scratch allocation is 0xd000000
>> (208 MB):
>>
>> 0x1000000   (16 MB)
>> 0xd000000   (208 MB)  <- global allocation
>> 0x5000000   (80 MB)
>>
>> The 80 MB difference is the per-node reservation that was previously
>> being included in the global allocation.
>>
>> The same issue also affects lowmem scratch memory, but its impact is
>> limited because the lowmem scratch memory calculation is restricted to
>> the first 4G of memory. The changes also cover the lowmem scratch
>> memory case.
>>
>> Cc: Alexander Graf <graf@amazon.com>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Cc: George Guo <guodongtai@kylinos.cn>
>> Cc: Mike Rapoport <rppt@kernel.org>
>> Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
>> Cc: Pratyush Yadav <pratyush@kernel.org>
>> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
>> Cc: linux-kernel@vger.kernel.org
>> Cc: linux-mm@kvack.org
>> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
>> ---
>>   include/linux/memblock.h           |  1 -
>>   kernel/liveupdate/kexec_handover.c | 49 ++++++++++++++++++++++--------
>>   mm/memblock.c                      | 22 --------------
>>   3 files changed, 37 insertions(+), 35 deletions(-)
>>
>> diff --git a/include/linux/memblock.h b/include/linux/memblock.h
>> index d62db9e776cf..678fe466529a 100644
>> --- a/include/linux/memblock.h
>> +++ b/include/linux/memblock.h
>> @@ -487,7 +487,6 @@ static inline __init_memblock bool memblock_bottom_up(void)
>>   phys_addr_t memblock_phys_mem_size(void);
>>   phys_addr_t memblock_reserved_size(void);
>>   phys_addr_t memblock_reserved_kern_size(phys_addr_t limit, int nid);
>> -phys_addr_t memblock_reserved_hugetlb_size(phys_addr_t limit, int nid);
>>   unsigned long memblock_estimated_nr_free_pages(void);
>>   phys_addr_t memblock_start_of_DRAM(void);
>>   phys_addr_t memblock_end_of_DRAM(void);
>> diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
>> index 7c4d86daf86d..dc809e1e768c 100644
>> --- a/kernel/liveupdate/kexec_handover.c
>> +++ b/kernel/liveupdate/kexec_handover.c
>> @@ -752,6 +752,31 @@ static int __init kho_parse_scratch_size(char *p)
>>   }
>>   early_param("kho_scratch", kho_parse_scratch_size);
>>   
>> +static phys_addr_t __init_memblock memblock_reserved_size_nid(phys_addr_t limit, int nid,
>> +							      enum memblock_flags region_type)
>> +{
> Why is this in kexec_handover.c and not in memblock.c?

I kept it here because KHO is the only user of this. However, I'm happy 
to move it to
memblock.c if you think that would be more appropriate.

>> +	struct memblock_region *r;
>> +	phys_addr_t total = 0;
>> +
>> +	for_each_reserved_mem_region(r) {
>> +		phys_addr_t size = r->size;
>> +
>> +		if (r->base > limit)
>> +			break;
>> +
>> +		if (r->base + r->size > limit)
>> +			size = limit - r->base;
>> +
>> +#ifdef CONFIG_NUMA
>> +		if (nid == memblock_get_region_node(r))
>> +#endif
> Do we really need the #ifdef here?

Yes, otherwise, if CONFIG_NUMA is not set, the lowmem and global sizes 
would be 0.


- Sourabh Jain

>
>> +			if (r->flags & region_type)
>> +				total += size;
>> +	}
>> +
>> +	return total;
>> +}
>> +
>>   static void __init scratch_size_update(void)
>>   {
>>   	/*

Re: [PATCH] kho: fix global scratch size calculation

[Mike Rapoport]

On Tue, Sep 22, 2026 at 06:42:16PM +0530, Sourabh Jain wrote:
> KHO calculates the global scratch size based on memblock-reserved kernel
> memory. It passes NUMA_NO_NODE to memblock_reserved_kern_size() for
> this calculation.
> 
> When memblock_reserved_kern_size() is called with NUMA_NO_NODE, it
> counts both:
> 
> - memory reserved for a specific NUMA node
> - memory reserved with NUMA_NO_NODE
> 
> KHO needs to distinguish between these two types of reservations.
> When calculating the size of global scratch memory, KHO only needs to
> account for reservations made with NUMA_NO_NODE. Reservations made for
> a specific NUMA node must not be included in the global scratch size.
> 
> Add memblock_reserved_size_nid() to calculate reserved memory for a
> given reservation type and NUMA node. When NUMA_NO_NODE is passed, it
> counts only memory reserved with NUMA_NO_NODE.
> 
> Use the new API for lowmem, global, and per-node KHO scratch size
> calculations. For lowmem and global scratch, count only memory
> reservations that were made with NUMA_NO_NODE. For per-node scratch,
> count only memory reservations that were made with the corresponding
> NUMA node ID.
> 
> Remove memblock_reserved_hugetlb_size() since it has the same
> implementation as the new API and differs only in the memblock
> reservation flag being checked. The new API handles both kernel and
> HugeTLB reservations through its reservation type argument.
> 
> Define the new helper as a static function in the KHO implementation,
> since it is only used by KHO and has no users outside
> kernel/liveupdate/kexec_handover.c.

I looked again and this feels like a band-aid to me.

I think we need to step back and rethink the order of the
scratch/kho-bootmem sizing and consider how NUMA and lowmem are involved
there.
 
> On powerpc, the difference can be seen in the scratch_len values
> reported by:
> 
> cat /sys/kernel/debug/kho/out/scratch_len
> 
> Before this change, the global scratch allocation was 0x12000000
> (288 MB):
> 
> 0x1000000   (16 MB)
> 0x12000000  (288 MB)  <- global allocation
> 0x5000000   (80 MB)
> 
> After this change, the global scratch allocation is 0xd000000
> (208 MB):
> 
> 0x1000000   (16 MB)
> 0xd000000   (208 MB)  <- global allocation
> 0x5000000   (80 MB)
> 
> The 80 MB difference is the per-node reservation that was previously
> being included in the global allocation.
> 
> The same issue also affects lowmem scratch memory, but its impact is
> limited because the lowmem scratch memory calculation is restricted to
> the first 4G of memory. The changes also cover the lowmem scratch
> memory case.
> 
> Cc: Alexander Graf <graf@amazon.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: George Guo <guodongtai@kylinos.cn>
> Cc: Mike Rapoport <rppt@kernel.org>
> Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
> Cc: Pratyush Yadav <pratyush@kernel.org>
> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
> Cc: linux-kernel@vger.kernel.org
> Cc: linux-mm@kvack.org
> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
> ---
>  include/linux/memblock.h           |  1 -
>  kernel/liveupdate/kexec_handover.c | 49 ++++++++++++++++++++++--------
>  mm/memblock.c                      | 22 --------------
>  3 files changed, 37 insertions(+), 35 deletions(-)
> 
> diff --git a/include/linux/memblock.h b/include/linux/memblock.h
> index d62db9e776cf..678fe466529a 100644
> --- a/include/linux/memblock.h
> +++ b/include/linux/memblock.h
> @@ -487,7 +487,6 @@ static inline __init_memblock bool memblock_bottom_up(void)
>  phys_addr_t memblock_phys_mem_size(void);
>  phys_addr_t memblock_reserved_size(void);
>  phys_addr_t memblock_reserved_kern_size(phys_addr_t limit, int nid);
> -phys_addr_t memblock_reserved_hugetlb_size(phys_addr_t limit, int nid);
>  unsigned long memblock_estimated_nr_free_pages(void);
>  phys_addr_t memblock_start_of_DRAM(void);
>  phys_addr_t memblock_end_of_DRAM(void);
> diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
> index 7c4d86daf86d..dc809e1e768c 100644
> --- a/kernel/liveupdate/kexec_handover.c
> +++ b/kernel/liveupdate/kexec_handover.c
> @@ -752,6 +752,31 @@ static int __init kho_parse_scratch_size(char *p)
>  }
>  early_param("kho_scratch", kho_parse_scratch_size);
>  
> +static phys_addr_t __init_memblock memblock_reserved_size_nid(phys_addr_t limit, int nid,
> +							      enum memblock_flags region_type)
> +{
> +	struct memblock_region *r;
> +	phys_addr_t total = 0;
> +
> +	for_each_reserved_mem_region(r) {
> +		phys_addr_t size = r->size;
> +
> +		if (r->base > limit)
> +			break;
> +
> +		if (r->base + r->size > limit)
> +			size = limit - r->base;
> +
> +#ifdef CONFIG_NUMA
> +		if (nid == memblock_get_region_node(r))
> +#endif
> +			if (r->flags & region_type)
> +				total += size;
> +	}
> +
> +	return total;
> +}
> +
>  static void __init scratch_size_update(void)
>  {
>  	/*
> @@ -762,17 +787,17 @@ static void __init scratch_size_update(void)
>  	if (scratch_scale) {
>  		phys_addr_t size;
>  
> -		size = memblock_reserved_kern_size(ARCH_LOW_ADDRESS_LIMIT,
> -						   NUMA_NO_NODE);
> -		size -= memblock_reserved_hugetlb_size(ARCH_LOW_ADDRESS_LIMIT,
> -						       NUMA_NO_NODE);
> +		size = memblock_reserved_size_nid(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE,
> +						  MEMBLOCK_RSRV_KERN);
> +		size -= memblock_reserved_size_nid(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE,
> +						   MEMBLOCK_RSRV_HUGETLB);
>  		size = size * scratch_scale / 100;
>  		scratch_size_lowmem = size;
>  
> -		size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE,
> -						   NUMA_NO_NODE);
> -		size -= memblock_reserved_hugetlb_size(MEMBLOCK_ALLOC_ANYWHERE,
> -						       NUMA_NO_NODE);
> +		size = memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, NUMA_NO_NODE,
> +						  MEMBLOCK_RSRV_KERN);
> +		size -= memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, NUMA_NO_NODE,
> +						   MEMBLOCK_RSRV_HUGETLB);
>  		size = size * scratch_scale / 100 - scratch_size_lowmem;
>  		scratch_size_global = size;
>  	}
> @@ -790,11 +815,11 @@ static phys_addr_t __init scratch_size_node(int nid)
>  	phys_addr_t size;
>  
>  	if (scratch_scale) {
> -		size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE,
> -						   nid);
> +		size = memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, nid,
> +						  MEMBLOCK_RSRV_KERN);
>  		/* Do not count HugeTLB pages. */
> -		size -= memblock_reserved_hugetlb_size(MEMBLOCK_ALLOC_ANYWHERE,
> -						       nid);
> +		size -= memblock_reserved_size_nid(MEMBLOCK_ALLOC_ANYWHERE, nid,
> +						   MEMBLOCK_RSRV_HUGETLB);
>  		size = size * scratch_scale / 100;
>  	} else {
>  		size = scratch_size_pernode;
> diff --git a/mm/memblock.c b/mm/memblock.c
> index 021db49eb7fc..9da748e774ea 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -1900,28 +1900,6 @@ phys_addr_t __init_memblock memblock_reserved_size(void)
>  	return memblock.reserved.total_size;
>  }
>  
> -phys_addr_t __init_memblock memblock_reserved_hugetlb_size(phys_addr_t limit, int nid)
> -{
> -	struct memblock_region *r;
> -	phys_addr_t total = 0;
> -
> -	for_each_reserved_mem_region(r) {
> -		phys_addr_t size = r->size;
> -
> -		if (r->base > limit)
> -			break;
> -
> -		if (r->base + r->size > limit)
> -			size = limit - r->base;
> -
> -		if (nid == memblock_get_region_node(r) || !numa_valid_node(nid))
> -			if (r->flags & MEMBLOCK_RSRV_HUGETLB)
> -				total += size;
> -	}
> -
> -	return total;
> -}
> -
>  phys_addr_t __init_memblock memblock_reserved_kern_size(phys_addr_t limit, int nid)
>  {
>  	struct memblock_region *r;
> -- 
> 2.55.0
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH] kho: fix global scratch size calculation

[Pratyush Yadav]

On Sat, Oct 03 2026, Mike Rapoport wrote:

> On Tue, Sep 22, 2026 at 06:42:16PM +0530, Sourabh Jain wrote:
>> KHO calculates the global scratch size based on memblock-reserved kernel
>> memory. It passes NUMA_NO_NODE to memblock_reserved_kern_size() for
>> this calculation.
>> 
>> When memblock_reserved_kern_size() is called with NUMA_NO_NODE, it
>> counts both:
>> 
>> - memory reserved for a specific NUMA node
>> - memory reserved with NUMA_NO_NODE
>> 
>> KHO needs to distinguish between these two types of reservations.
>> When calculating the size of global scratch memory, KHO only needs to
>> account for reservations made with NUMA_NO_NODE. Reservations made for
>> a specific NUMA node must not be included in the global scratch size.
>> 
>> Add memblock_reserved_size_nid() to calculate reserved memory for a
>> given reservation type and NUMA node. When NUMA_NO_NODE is passed, it
>> counts only memory reserved with NUMA_NO_NODE.
>> 
>> Use the new API for lowmem, global, and per-node KHO scratch size
>> calculations. For lowmem and global scratch, count only memory
>> reservations that were made with NUMA_NO_NODE. For per-node scratch,
>> count only memory reservations that were made with the corresponding
>> NUMA node ID.
>> 
>> Remove memblock_reserved_hugetlb_size() since it has the same
>> implementation as the new API and differs only in the memblock
>> reservation flag being checked. The new API handles both kernel and
>> HugeTLB reservations through its reservation type argument.
>> 
>> Define the new helper as a static function in the KHO implementation,
>> since it is only used by KHO and has no users outside
>> kernel/liveupdate/kexec_handover.c.
>
> I looked again and this feels like a band-aid to me.
>
> I think we need to step back and rethink the order of the
> scratch/kho-bootmem sizing and consider how NUMA and lowmem are involved
> there.

Could you elaborate? How else would you calculate the sizes?

To me, the idea sounds fine (although I also share your comments on the
code style). This is what the global scratch size should have been doing
from the get-go. Today it over-allocates by counting both global
allocations and _all_ per-node allocations.

There is a separate problem with CONFIG_NUMA=n, but for that I think we
should just have a different allocation path where we only do lowmem and
global scratch.

>  
>> On powerpc, the difference can be seen in the scratch_len values
>> reported by:
>> 
>> cat /sys/kernel/debug/kho/out/scratch_len
>> 
>> Before this change, the global scratch allocation was 0x12000000
>> (288 MB):
>> 
>> 0x1000000   (16 MB)
>> 0x12000000  (288 MB)  <- global allocation
>> 0x5000000   (80 MB)
>> 
>> After this change, the global scratch allocation is 0xd000000
>> (208 MB):
>> 
>> 0x1000000   (16 MB)
>> 0xd000000   (208 MB)  <- global allocation
>> 0x5000000   (80 MB)
>> 
>> The 80 MB difference is the per-node reservation that was previously
>> being included in the global allocation.
>> 
>> The same issue also affects lowmem scratch memory, but its impact is
>> limited because the lowmem scratch memory calculation is restricted to
>> the first 4G of memory. The changes also cover the lowmem scratch
>> memory case.
>> 
>> Cc: Alexander Graf <graf@amazon.com>
>> Cc: Andrew Morton <akpm@linux-foundation.org>
>> Cc: George Guo <guodongtai@kylinos.cn>
>> Cc: Mike Rapoport <rppt@kernel.org>
>> Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
>> Cc: Pratyush Yadav <pratyush@kernel.org>
>> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
>> Cc: linux-kernel@vger.kernel.org
>> Cc: linux-mm@kvack.org
>> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
[...]

-- 
Regards,
Pratyush Yadav

Re: [PATCH] kho: fix global scratch size calculation

[Mike Rapoport]

On Mon, Oct 12, 2026 at 03:52:46PM +0200, Pratyush Yadav wrote:
> On Sat, Oct 03 2026, Mike Rapoport wrote:
> 
> > On Tue, Sep 22, 2026 at 06:42:16PM +0530, Sourabh Jain wrote:
> >> KHO calculates the global scratch size based on memblock-reserved kernel
> >> memory. It passes NUMA_NO_NODE to memblock_reserved_kern_size() for
> >> this calculation.
> >> 
> >> When memblock_reserved_kern_size() is called with NUMA_NO_NODE, it
> >> counts both:
> >> 
> >> - memory reserved for a specific NUMA node
> >> - memory reserved with NUMA_NO_NODE
> >> 
> >> KHO needs to distinguish between these two types of reservations.
> >> When calculating the size of global scratch memory, KHO only needs to
> >> account for reservations made with NUMA_NO_NODE. Reservations made for
> >> a specific NUMA node must not be included in the global scratch size.
> >> 
> >> Add memblock_reserved_size_nid() to calculate reserved memory for a
> >> given reservation type and NUMA node. When NUMA_NO_NODE is passed, it
> >> counts only memory reserved with NUMA_NO_NODE.
> >> 
> >> Use the new API for lowmem, global, and per-node KHO scratch size
> >> calculations. For lowmem and global scratch, count only memory
> >> reservations that were made with NUMA_NO_NODE. For per-node scratch,
> >> count only memory reservations that were made with the corresponding
> >> NUMA node ID.
> >> 
> >> Remove memblock_reserved_hugetlb_size() since it has the same
> >> implementation as the new API and differs only in the memblock
> >> reservation flag being checked. The new API handles both kernel and
> >> HugeTLB reservations through its reservation type argument.
> >> 
> >> Define the new helper as a static function in the KHO implementation,
> >> since it is only used by KHO and has no users outside
> >> kernel/liveupdate/kexec_handover.c.
> >
> > I looked again and this feels like a band-aid to me.
> >
> > I think we need to step back and rethink the order of the
> > scratch/kho-bootmem sizing and consider how NUMA and lowmem are involved
> > there.
> 
> Could you elaborate? How else would you calculate the sizes?
> 
> To me, the idea sounds fine (although I also share your comments on the
> code style). This is what the global scratch size should have been doing
> from the get-go. Today it over-allocates by counting both global
> allocations and _all_ per-node allocations.
 
> There is a separate problem with CONFIG_NUMA=n, but for that I think we
> should just have a different allocation path where we only do lowmem and
> global scratch.

I mean more fundamental code refactoring that takes into the account all
cases (and I could miss some)

- NUMA vs no-NUMA
- systems that have global scratch containing lowmem scratch vs those that
  don't
- and of course fix global scratch over-allocation
 
> >> On powerpc, the difference can be seen in the scratch_len values
> >> reported by:
> >> 
> >> cat /sys/kernel/debug/kho/out/scratch_len
> >> 
> >> Before this change, the global scratch allocation was 0x12000000
> >> (288 MB):
> >> 
> >> 0x1000000   (16 MB)
> >> 0x12000000  (288 MB)  <- global allocation
> >> 0x5000000   (80 MB)
> >> 
> >> After this change, the global scratch allocation is 0xd000000
> >> (208 MB):
> >> 
> >> 0x1000000   (16 MB)
> >> 0xd000000   (208 MB)  <- global allocation
> >> 0x5000000   (80 MB)
> >> 
> >> The 80 MB difference is the per-node reservation that was previously
> >> being included in the global allocation.
> >> 
> >> The same issue also affects lowmem scratch memory, but its impact is
> >> limited because the lowmem scratch memory calculation is restricted to
> >> the first 4G of memory. The changes also cover the lowmem scratch
> >> memory case.
> >> 
> >> Cc: Alexander Graf <graf@amazon.com>
> >> Cc: Andrew Morton <akpm@linux-foundation.org>
> >> Cc: George Guo <guodongtai@kylinos.cn>
> >> Cc: Mike Rapoport <rppt@kernel.org>
> >> Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
> >> Cc: Pratyush Yadav <pratyush@kernel.org>
> >> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
> >> Cc: linux-kernel@vger.kernel.org
> >> Cc: linux-mm@kvack.org
> >> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
> [...]
> 
> -- 
> Regards,
> Pratyush Yadav

-- 
Sincerely yours,
Mike.

Re: [PATCH] kho: fix global scratch size calculation

[Pratyush Yadav]

On Mon, Oct 12 2026, Mike Rapoport wrote:

> On Mon, Oct 12, 2026 at 03:52:46PM +0200, Pratyush Yadav wrote:
>> On Sat, Oct 03 2026, Mike Rapoport wrote:
>> 
>> > On Tue, Sep 22, 2026 at 06:42:16PM +0530, Sourabh Jain wrote:
>> >> KHO calculates the global scratch size based on memblock-reserved kernel
>> >> memory. It passes NUMA_NO_NODE to memblock_reserved_kern_size() for
>> >> this calculation.
>> >> 
>> >> When memblock_reserved_kern_size() is called with NUMA_NO_NODE, it
>> >> counts both:
>> >> 
>> >> - memory reserved for a specific NUMA node
>> >> - memory reserved with NUMA_NO_NODE
>> >> 
>> >> KHO needs to distinguish between these two types of reservations.
>> >> When calculating the size of global scratch memory, KHO only needs to
>> >> account for reservations made with NUMA_NO_NODE. Reservations made for
>> >> a specific NUMA node must not be included in the global scratch size.
>> >> 
>> >> Add memblock_reserved_size_nid() to calculate reserved memory for a
>> >> given reservation type and NUMA node. When NUMA_NO_NODE is passed, it
>> >> counts only memory reserved with NUMA_NO_NODE.
>> >> 
>> >> Use the new API for lowmem, global, and per-node KHO scratch size
>> >> calculations. For lowmem and global scratch, count only memory
>> >> reservations that were made with NUMA_NO_NODE. For per-node scratch,
>> >> count only memory reservations that were made with the corresponding
>> >> NUMA node ID.
>> >> 
>> >> Remove memblock_reserved_hugetlb_size() since it has the same
>> >> implementation as the new API and differs only in the memblock
>> >> reservation flag being checked. The new API handles both kernel and
>> >> HugeTLB reservations through its reservation type argument.
>> >> 
>> >> Define the new helper as a static function in the KHO implementation,
>> >> since it is only used by KHO and has no users outside
>> >> kernel/liveupdate/kexec_handover.c.
>> >
>> > I looked again and this feels like a band-aid to me.
>> >
>> > I think we need to step back and rethink the order of the
>> > scratch/kho-bootmem sizing and consider how NUMA and lowmem are involved
>> > there.
>> 
>> Could you elaborate? How else would you calculate the sizes?
>> 
>> To me, the idea sounds fine (although I also share your comments on the
>> code style). This is what the global scratch size should have been doing
>> from the get-go. Today it over-allocates by counting both global
>> allocations and _all_ per-node allocations.
>  
>> There is a separate problem with CONFIG_NUMA=n, but for that I think we
>> should just have a different allocation path where we only do lowmem and
>> global scratch.
>
> I mean more fundamental code refactoring that takes into the account all
> cases (and I could miss some)
>
> - NUMA vs no-NUMA
> - systems that have global scratch containing lowmem scratch vs those that
>   don't
> - and of course fix global scratch over-allocation

Fair enough, that makes sense.

[...]

-- 
Regards,
Pratyush Yadav

Re: [PATCH] kho: fix global scratch size calculation

[BernardoGV]

On Mon, Oct 12, 2026, Mike Rapoport wrote:
> I mean more fundamental code refactoring that takes into the account all
> cases (and I could miss some)
>
> - NUMA vs no-NUMA
> - systems that have global scratch containing lowmem scratch vs those that
>   don't
> - and of course fix global scratch over-allocation

One more case for the list: reservations made before KHO sizes scratch
that are never served from it. HugeTLB is already excluded. crashkernel
is the other big one. Sourabh mentioned it in the powerpc thread. Here
are x86 numbers from a machine that never asked for KHO.

Ubuntu 26.04's generic kernel (7.0.0-38-generic) ships
CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y, so scratch is reserved on every
boot. (26.10 has since turned the default off. I've asked for the same
in 26.04 [1].) The machine is a single-node Ryzen AI Max+ 395 with
128 GiB and no hugetlb. kdump is set up the stock Ubuntu way, which gives
crashkernel 2048M high + 256M low.

Three cold boots of the same kernel, KHO on by default:

  kho/out/scratch_len   crashkernel   no crashkernel   delta
  lowmem                   760 MiB         248 MiB      -512 = 2 x 256
  global                 8,510 MiB       4,414 MiB    -4,096 = 2 x 2,048
  per-node                4,338 MiB       4,338 MiB         0
  total                 13,608 MiB       9,000 MiB    -4,608 = 2 x 2,304

"Memory: ... reserved" goes from 18,849,832K to 11,771,924K. The
difference, 6,912 MiB, is exactly the crashkernel plus 2x crashkernel of
scratch. With kho=off it is 4,915,232K.

Both problems show up here. Without crashkernel, lowmem + global is
2 x 2,331 MiB and per-node is 2 x 2,169 MiB, so on a single node about
2,169 MiB of reservations get counted twice. That is the over-allocation,
and the total comes to roughly 4x what was reserved. crashkernel is
allocated with NUMA_NO_NODE, so it lands only in lowmem/global, at 2x.
If I read Sourabh's patch right, it would count NUMA_NO_NODE only for
global. On this machine that leaves crashkernel as nearly all of the
global scratch (about 2 x (162 + 2,304) MiB).

The cost is not only boot memory. Scratch ends up MIGRATE_CMA, so
anything that pins can't use it. Here that is amdgpu GTT on an APU. A
single process tops out at 107 GiB of a 120 GiB GTT limit with KHO on,
and at 117 GiB with kho=off.

Happy to test patches on this box.

[1] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2171297

Re: [PATCH] kho: fix global scratch size calculation

[Pratyush Yadav]

On Sat, Oct 17 2026, BernardoGV@proton.me wrote:

> On Mon, Oct 12, 2026, Mike Rapoport wrote:
>> I mean more fundamental code refactoring that takes into the account all
>> cases (and I could miss some)
>>
>> - NUMA vs no-NUMA
>> - systems that have global scratch containing lowmem scratch vs those that
>>   don't
>> - and of course fix global scratch over-allocation
>
> One more case for the list: reservations made before KHO sizes scratch
> that are never served from it. HugeTLB is already excluded. crashkernel
> is the other big one. Sourabh mentioned it in the powerpc thread. Here
> are x86 numbers from a machine that never asked for KHO.
>
> Ubuntu 26.04's generic kernel (7.0.0-38-generic) ships
> CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y, so scratch is reserved on every
> boot. (26.10 has since turned the default off. I've asked for the same
> in 26.04 [1].) The machine is a single-node Ryzen AI Max+ 395 with
> 128 GiB and no hugetlb. kdump is set up the stock Ubuntu way, which gives
> crashkernel 2048M high + 256M low.
>
> Three cold boots of the same kernel, KHO on by default:
>
>   kho/out/scratch_len   crashkernel   no crashkernel   delta
>   lowmem                   760 MiB         248 MiB      -512 = 2 x 256
>   global                 8,510 MiB       4,414 MiB    -4,096 = 2 x 2,048
>   per-node                4,338 MiB       4,338 MiB         0
>   total                 13,608 MiB       9,000 MiB    -4,608 = 2 x 2,304
>
> "Memory: ... reserved" goes from 18,849,832K to 11,771,924K. The
> difference, 6,912 MiB, is exactly the crashkernel plus 2x crashkernel of
> scratch. With kho=off it is 4,915,232K.
>
> Both problems show up here. Without crashkernel, lowmem + global is
> 2 x 2,331 MiB and per-node is 2 x 2,169 MiB, so on a single node about
> 2,169 MiB of reservations get counted twice. That is the over-allocation,
> and the total comes to roughly 4x what was reserved. crashkernel is
> allocated with NUMA_NO_NODE, so it lands only in lowmem/global, at 2x.
> If I read Sourabh's patch right, it would count NUMA_NO_NODE only for
> global. On this machine that leaves crashkernel as nearly all of the
> global scratch (about 2 x (162 + 2,304) MiB).

To remove crashkernel from scratch calculation, you'd have to allocate
it from outside of scratch, like the hugetlb pages. Crashkernel is
usually allocated earlier in boot, so we'd have to call
kho_extend_scratch() before crashkernel allocations, and I am not sure
if something will break if we do so. I suspect we at least need to check
efi_init().

Not saying it can't be done, but it isn't trivial.

>
> The cost is not only boot memory. Scratch ends up MIGRATE_CMA, so
> anything that pins can't use it. Here that is amdgpu GTT on an APU. A
> single process tops out at 107 GiB of a 120 GiB GTT limit with KHO on,
> and at 117 GiB with kho=off.

Well, my pipe dream is to have KHO entirely without scratch. You would
use the same mechanism that kho_extend_scratch() uses, to discover all
usable early boot memory, and not pass over any scratch at all (or, a
very small amount of scratch). The only problem is that it might fall
prey to fragmentation. If preserved regions are spread apart enough, you
might end up with _no_ memory to boot from. I am not sure how to make
this idea reliable yet.

>
> Happy to test patches on this box.
>
> [1] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2171297
>
>

-- 
Regards,
Pratyush Yadav