[PATCH v3 0/4] powerpc: increase MIN RMA size for CAS negotiation

[PATCH v3 0/4] powerpc: increase MIN RMA size for CAS negotiation

[Sourabh Jain]

Change RMA size from 512 MB to 768 MB which will result
in more RMA at boot time for PowerPC. When PowerPC LPAR use/uses vTPM,
Secure Boot or FADump, the 512 MB RMA memory is not sufficient for
booting. With this 512 MB RMA, GRUB2 run out of memory and unable to
load the necessary. Sometimes even usage of CDROM which requires more
memory for installation along with the options mentioned above troubles
the boot memory and result in boot failures. Increasing the RMA size
will resolves multiple out of memory issues observed in PowerPC.

Due to the change in the MIN_RMA size, there is an impact on the
additional parameter for the FADump feature in HASH MMU. The first two
patches in this series update the memory allocation policy for the
additional parameter in FADump for HASH MMU to mitigate that impact.

Patch 01/04: Exports the `MIN_RMA` so that the FADump code can access
             it.
Patch 02/04: Changes the memory reservation policy for the FADump
             additional parameter in HASH MMU.
Patch 03/04: Change MIN_RMA to 768MB
Patch 04/04: Update fadump document to include details about the
             additional parameter feature

Note: This policy change does not impact the RADIX MMU additional
      parameter for the FADump feature.

Testing Done:  
=============
The patch series has been tested with the following four scenarios, and
both the production kernel and FADump kernel boot successfully.  

- Distro: RHEL 9.6

1. Production Kernel: RADIX MMU
   FADump Kernel:     RADIX MMU
   - MIN_RMA: 768 MB
   - RMA:     1 GB
     - a. crashkernel: 4 GB
     - b. crashkernel: 1 GB

2. Production Kernel: HASH MMU
   FADump Kernel:   HASH MMU
   - MIN_RMA: 768 MB
   - RMA:     1 GB
     - a. crashkernel: 768 MB
     - b. crashkernel: 1 GB


Changelog:
==========

MIN RMA change
---------------
v1: https://lore.kernel.org/all/20241205101045.27069-1-avnish@linux.ibm.com/

v2: https://lore.kernel.org/all/20241206065545.14815-1-avnish@linux.ibm.com/
 - Added GRUB2 debug logs and Kernel traces.

Fadump Fixes
-------------
v1:
To address the FADump issue introduced due to the change in MIN_RMA, patches were sent:
https://lore.kernel.org/all/20250120173501.1147236-1-sourabhjain@linux.ibm.com/

Based on the comments on the FADump fixes patch series above, it was
suggested that it would be better to fix the FADump issue before
making the MIN_RMA change.

Merged the FADump fixes and MIN_RMA change into a single patch series.
-----------------------------------------------------------------------
v3:
 - This avoids the impact on FADump caused by the MIN_RMA change.
 - Since the policy for FADump memory reservation for the additional
   parameter is changing for HASH MMU, the corresponding FADump
   documentation is also updated to reflect this change in patch 04/04.
 - Add Reviewed-by tag

Cc: Avnish Chouhan <avnish@linux.ibm.com>
Cc: Brian King <brking@linux.ibm.com>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Cc: Sourabh Jain <sourabhjain@linux.ibm.com>

Avnish Chouhan (1):
  powerpc: increase MIN RMA size for CAS negotiation

Sourabh Jain (3):
  powerpc: export MIN RMA size
  powerpc/fadump: fix additional param memory reservation for HASH MMU
  Documentation/powerpc/fadump: add additional parameter feature details

 Documentation/ABI/testing/sysfs-kernel-fadump |  3 ++-
 .../arch/powerpc/firmware-assisted-dump.rst   | 22 +++++++++++++++++++
 arch/powerpc/include/asm/prom.h               |  2 ++
 arch/powerpc/kernel/fadump.c                  | 21 +++++++++---------
 arch/powerpc/kernel/prom_init.c               |  2 +-
 5 files changed, 38 insertions(+), 12 deletions(-)

-- 
2.48.1

[PATCH v3 1/4] powerpc: export MIN RMA size

[Sourabh Jain]

Commit 683eab94da75bc ("powerpc/fadump: setup additional parameters for
dump capture kernel") introduced the additional parameter feature in
fadump for HASH MMU with the understanding that GRUB does not use the
memory area between 640MB and 768MB for its operation.

However, the third patch ("powerpc: increase MIN RMA size for CAS
negotiation") in this series is changing the MIN RMA size to 768MB,
allowing GRUB to use memory up to 768MB. This makes the fadump
reservation for the additional parameter feature for HASH MMU
unreliable.

To address this, export the MIN_RMA so that the next patch
("powerpc/fadump: fix additional param memory reservation for HASH MMU")
can identify the correct memory range for the additional parameter
feature in fadump for HASH MMU.

Cc: Avnish Chouhan <avnish@linux.ibm.com>
Cc: Brian King <brking@linux.ibm.com>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Reviewed-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
---
 arch/powerpc/include/asm/prom.h | 2 ++
 arch/powerpc/kernel/prom_init.c | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/prom.h b/arch/powerpc/include/asm/prom.h
index c0107d8ddd8c..b4e504b914e4 100644
--- a/arch/powerpc/include/asm/prom.h
+++ b/arch/powerpc/include/asm/prom.h
@@ -17,6 +17,8 @@
 struct device_node;
 struct property;
 
+#define MIN_RMA			512		/* Minimum RMA (in MB) for CAS negotiation */
+
 #define OF_DT_BEGIN_NODE	0x1		/* Start of node, full name */
 #define OF_DT_END_NODE		0x2		/* End node */
 #define OF_DT_PROP		0x3		/* Property: name off, size,
diff --git a/arch/powerpc/kernel/prom_init.c b/arch/powerpc/kernel/prom_init.c
index 8e776ba39497..7daecf1b7c16 100644
--- a/arch/powerpc/kernel/prom_init.c
+++ b/arch/powerpc/kernel/prom_init.c
@@ -1061,7 +1061,7 @@ static const struct ibm_arch_vec ibm_architecture_vec_template __initconst = {
 		.virt_base = cpu_to_be32(0xffffffff),
 		.virt_size = cpu_to_be32(0xffffffff),
 		.load_base = cpu_to_be32(0xffffffff),
-		.min_rma = cpu_to_be32(512),		/* 512MB min RMA */
+		.min_rma = cpu_to_be32(MIN_RMA),
 		.min_load = cpu_to_be32(0xffffffff),	/* full client load */
 		.min_rma_percent = 0,	/* min RMA percentage of total RAM */
 		.max_pft_size = 48,	/* max log_2(hash table size) */
-- 
2.48.1

[PATCH v3 2/4] powerpc/fadump: fix additional param memory reservation for HASH MMU

[Sourabh Jain]

Commit 683eab94da75bc ("powerpc/fadump: setup additional parameters for
dump capture kernel") introduced the additional parameter feature in
fadump for HASH MMU with the understanding that GRUB does not use the
memory area between 640MB and 768MB for its operation.

However, the third patch in this series ("powerpc: increase MIN RMA
size for CAS negotiation") changes the MIN RMA size to 768MB, allowing
GRUB to use memory up to 768MB. This makes the fadump reservation for
the additional parameter feature for HASH MMU unreliable.

To address this, adjust the memory range for the additional parameter in
fadump for HASH MMU. This will ensure that GRUB does not overwrite the
memory reserved for fadump's additional parameter in HASH MMU.

The new policy for the memory range for the additional parameter in HASH
MMU is that the first memory block must be larger than the MIN_RMA size,
as the bootloader can use memory up to the MIN_RMA size. The range
should be between MIN_RMA and the RMA size (ppc64_rma_size), and it must
not overlap with the fadump reserved area.

Cc: Avnish Chouhan <avnish@linux.ibm.com>
Cc: Brian King <brking@linux.ibm.com>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Reviewed-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
---
 arch/powerpc/kernel/fadump.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/arch/powerpc/kernel/fadump.c b/arch/powerpc/kernel/fadump.c
index 4b371c738213..26e3d151e048 100644
--- a/arch/powerpc/kernel/fadump.c
+++ b/arch/powerpc/kernel/fadump.c
@@ -33,6 +33,7 @@
 #include <asm/fadump-internal.h>
 #include <asm/setup.h>
 #include <asm/interrupt.h>
+#include <asm/prom.h>
 
 /*
  * The CPU who acquired the lock to trigger the fadump crash should
@@ -1764,19 +1765,19 @@ void __init fadump_setup_param_area(void)
 		range_end = memblock_end_of_DRAM();
 	} else {
 		/*
-		 * Passing additional parameters is supported for hash MMU only
-		 * if the first memory block size is 768MB or higher.
+		 * Memory range for passing additional parameters for HASH MMU
+		 * must meet the following conditions:
+		 * 1. The first memory block size must be higher than the
+		 *    minimum RMA (MIN_RMA) size. Bootloader can use memory
+		 *    upto RMA size. So it should be avoided.
+		 * 2. The range should be between MIN_RMA and RMA size (ppc64_rma_size)
+		 * 3. It must not overlap with the fadump reserved area.
 		 */
-		if (ppc64_rma_size < 0x30000000)
+		if (ppc64_rma_size < MIN_RMA*1024*1024)
 			return;
 
-		/*
-		 * 640 MB to 768 MB is not used by PFW/bootloader. So, try reserving
-		 * memory for passing additional parameters in this range to avoid
-		 * being stomped on by PFW/bootloader.
-		 */
-		range_start = 0x2A000000;
-		range_end = range_start + 0x4000000;
+		range_start = MIN_RMA * 1024 * 1024;
+		range_end = min(ppc64_rma_size, fw_dump.boot_mem_top);
 	}
 
 	fw_dump.param_area = memblock_phys_alloc_range(COMMAND_LINE_SIZE,
-- 
2.48.1

[PATCH v3 3/4] powerpc: increase MIN RMA size for CAS negotiation

[Sourabh Jain]

From: Avnish Chouhan <avnish@linux.ibm.com>

Change RMA size from 512 MB to 768 MB which will result
in more RMA at boot time for PowerPC. When PowerPC LPAR use/uses vTPM,
Secure Boot or FADump, the 512 MB RMA memory is not sufficient for
booting. With this 512 MB RMA, GRUB2 run out of memory and unable to
load the necessary. Sometimes even usage of CDROM which requires more
memory for installation along with the options mentioned above troubles
the boot memory and result in boot failures. Increasing the RMA size
will resolves multiple out of memory issues observed in PowerPC.

Failure details:

1. GRUB2

kern/ieee1275/init.c:550: mm requested region of size 8513000, flags 1
kern/ieee1275/init.c:563: Cannot satisfy allocation and retain minimum runtime
space
kern/ieee1275/init.c:550: mm requested region of size 8513000, flags 0
kern/ieee1275/init.c:563: Cannot satisfy allocation and retain minimum runtime
space
kern/file.c:215: Closing `/ppc/ppc64/initrd.img' ...
kern/disk.c:297: Closing
`ieee1275//vdevice/v-scsi
@30000067/disk@8300000000000000'...
kern/disk.c:311: Closing
`ieee1275//vdevice/v-scsi
@30000067/disk@8300000000000000' succeeded.
kern/file.c:225: Closing `/ppc/ppc64/initrd.img' failed with 3.
kern/file.c:148: Opening `/ppc/ppc64/initrd.img' succeeded.
error: ../../grub-core/kern/mm.c:552:out of memory.

2. Kernel

[    0.777633] List of all partitions:
[    0.777639] No filesystem could mount root, tried:
[    0.777640]
[    0.777649] Kernel panic - not syncing: VFS: Unable to mount root fs on "" or unknown-block(0,0)
[    0.777658] CPU: 17 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.11.0-0.rc4.20.el10.ppc64le #1
[    0.777669] Hardware name: IBM,9009-22A POWER9 (architected) 0x4e0202 0xf000005 of:IBM,FW950.B0 (VL950_149) hv:phyp pSeries
[    0.777678] Call Trace:
[    0.777682] [c000000003db7b60] [c000000001119714] dump_stack_lvl+0x88/0xc4 (unreliable)
[    0.777700] [c000000003db7b90] [c00000000016c274] panic+0x174/0x460
[    0.777711] [c000000003db7c30] [c00000000200631c] mount_root_generic+0x320/0x354
[    0.777724] [c000000003db7d00] [c0000000020066f8] prepare_namespace+0x27c/0x2f4
[    0.777735] [c000000003db7d90] [c000000002005824] kernel_init_freeable+0x254/0x294
[    0.777747] [c000000003db7df0] [c00000000001131c] kernel_init+0x30/0x1c4
[    0.777757] [c000000003db7e50] [c00000000000debc] ret_from_kernel_user_thread+0x14/0x1c
[    0.777768] --- interrupt: 0 at 0x0
[    0.784238] pstore: backend (nvram) writing error (-1)
[    0.790447] Rebooting in 10 seconds..

Cc: Brian King <brking@linux.ibm.com>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Meghana Prakash <meghanaprakash@in.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Cc: Sourabh Jain <sourabhjain@linux.ibm.com>
Signed-off-by: Avnish Chouhan <avnish@linux.ibm.com>
---
 arch/powerpc/include/asm/prom.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/prom.h b/arch/powerpc/include/asm/prom.h
index b4e504b914e4..f679a11a7e7f 100644
--- a/arch/powerpc/include/asm/prom.h
+++ b/arch/powerpc/include/asm/prom.h
@@ -17,7 +17,7 @@
 struct device_node;
 struct property;
 
-#define MIN_RMA			512		/* Minimum RMA (in MB) for CAS negotiation */
+#define MIN_RMA			768		/* Minimum RMA (in MB) for CAS negotiation */
 
 #define OF_DT_BEGIN_NODE	0x1		/* Start of node, full name */
 #define OF_DT_END_NODE		0x2		/* End node */
-- 
2.48.1

[PATCH v3 4/4] Documentation/powerpc/fadump: add additional parameter feature details

[Sourabh Jain]

Update the fadump document to include details about the fadump
additional parameter feature.

The document includes the following:
- Significance of the feature
- How to use it
- Feature restrictions

No functional changes are introduced.

Cc: Avnish Chouhan <avnish@linux.ibm.com>
Cc: Brian King <brking@linux.ibm.com>
Cc: Hari Bathini <hbathini@linux.ibm.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
---
 Documentation/ABI/testing/sysfs-kernel-fadump |  3 ++-
 .../arch/powerpc/firmware-assisted-dump.rst   | 22 +++++++++++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/Documentation/ABI/testing/sysfs-kernel-fadump b/Documentation/ABI/testing/sysfs-kernel-fadump
index 2f9daa7ca55b..b64b7622e6fc 100644
--- a/Documentation/ABI/testing/sysfs-kernel-fadump
+++ b/Documentation/ABI/testing/sysfs-kernel-fadump
@@ -55,4 +55,5 @@ Date:		May 2024
 Contact:	linuxppc-dev@lists.ozlabs.org
 Description:	read/write
 		This is a special sysfs file available to setup additional
-		parameters to be passed to capture kernel.
+		parameters to be passed to capture kernel. For HASH MMU it
+		is exported only if RMA size higher than 768MB.
diff --git a/Documentation/arch/powerpc/firmware-assisted-dump.rst b/Documentation/arch/powerpc/firmware-assisted-dump.rst
index 7e37aadd1f77..7e266e749cd5 100644
--- a/Documentation/arch/powerpc/firmware-assisted-dump.rst
+++ b/Documentation/arch/powerpc/firmware-assisted-dump.rst
@@ -120,6 +120,28 @@ to ensure that crash data is preserved to process later.
    e.g.
      # echo 1 > /sys/firmware/opal/mpipl/release_core
 
+-- Support for Additional Kernel Arguments in Fadump
+   Fadump has a feature that allows passing additional kernel arguments
+   to the fadump kernel. This feature was primarily designed to disable
+   kernel functionalities that are not required for the fadump kernel
+   and to reduce its memory footprint while collecting the dump.
+
+  Command to Add Additional Kernel Parameters to Fadump:
+  e.g.
+  # echo "nr_cpus=16" > /sys/kernel/fadump/bootargs_append
+
+  The above command is sufficient to add additional arguments to fadump.
+  An explicit service restart is not required.
+
+  Command to Retrieve the Additional Fadump Arguments:
+  e.g.
+  # cat /sys/kernel/fadump/bootargs_append
+
+Note: Additional kernel arguments for fadump with HASH MMU is only
+      supported if the RMA size is greater than 768 MB. If the RMA
+      size is less than 768 MB, the kernel does not export the
+      /sys/kernel/fadump/bootargs_append sysfs node.
+
 Implementation details:
 -----------------------
 
-- 
2.48.1

Re: [PATCH v3 4/4] Documentation/powerpc/fadump: add additional parameter feature details

[Mahesh J Salgaonkar]

On 2025-01-23 17:12:54 Thu, Sourabh Jain wrote:
> Update the fadump document to include details about the fadump
> additional parameter feature.
> 
> The document includes the following:
> - Significance of the feature
> - How to use it
> - Feature restrictions
> 
> No functional changes are introduced.
> 
> Cc: Avnish Chouhan <avnish@linux.ibm.com>
> Cc: Brian King <brking@linux.ibm.com>
> Cc: Hari Bathini <hbathini@linux.ibm.com>
> Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
> ---
>  Documentation/ABI/testing/sysfs-kernel-fadump |  3 ++-
>  .../arch/powerpc/firmware-assisted-dump.rst   | 22 +++++++++++++++++++
>  2 files changed, 24 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/ABI/testing/sysfs-kernel-fadump b/Documentation/ABI/testing/sysfs-kernel-fadump
> index 2f9daa7ca55b..b64b7622e6fc 100644
> --- a/Documentation/ABI/testing/sysfs-kernel-fadump
> +++ b/Documentation/ABI/testing/sysfs-kernel-fadump
[...]
> +-- Support for Additional Kernel Arguments in Fadump
> +   Fadump has a feature that allows passing additional kernel arguments
> +   to the fadump kernel. This feature was primarily designed to disable
> +   kernel functionalities that are not required for the fadump kernel
> +   and to reduce its memory footprint while collecting the dump.
> +
> +  Command to Add Additional Kernel Parameters to Fadump:
> +  e.g.
> +  # echo "nr_cpus=16" > /sys/kernel/fadump/bootargs_append
> +
> +  The above command is sufficient to add additional arguments to fadump.
> +  An explicit service restart is not required.
> +
> +  Command to Retrieve the Additional Fadump Arguments:
> +  e.g.
> +  # cat /sys/kernel/fadump/bootargs_append
> +
> +Note: Additional kernel arguments for fadump with HASH MMU is only
> +      supported if the RMA size is greater than 768 MB. If the RMA
> +      size is less than 768 MB, the kernel does not export the
> +      /sys/kernel/fadump/bootargs_append sysfs node.

Reviewed-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>

Thanks,
-Mahesh.

> +
>  Implementation details:
>  -----------------------
>  
> -- 
> 2.48.1
> 
> 

-- 
Mahesh J Salgaonkar

Re: [PATCH v3 4/4] Documentation/powerpc/fadump: add additional parameter feature details

[Sourabh Jain]

Hello Mahesh,


On 26/01/25 17:31, Mahesh J Salgaonkar wrote:
> On 2025-01-23 17:12:54 Thu, Sourabh Jain wrote:
>> Update the fadump document to include details about the fadump
>> additional parameter feature.
>>
>> The document includes the following:
>> - Significance of the feature
>> - How to use it
>> - Feature restrictions
>>
>> No functional changes are introduced.
>>
>> Cc: Avnish Chouhan <avnish@linux.ibm.com>
>> Cc: Brian King <brking@linux.ibm.com>
>> Cc: Hari Bathini <hbathini@linux.ibm.com>
>> Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
>> Cc: Michael Ellerman <mpe@ellerman.id.au>
>> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
>> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
>> ---
>>   Documentation/ABI/testing/sysfs-kernel-fadump |  3 ++-
>>   .../arch/powerpc/firmware-assisted-dump.rst   | 22 +++++++++++++++++++
>>   2 files changed, 24 insertions(+), 1 deletion(-)
>>
>> diff --git a/Documentation/ABI/testing/sysfs-kernel-fadump b/Documentation/ABI/testing/sysfs-kernel-fadump
>> index 2f9daa7ca55b..b64b7622e6fc 100644
>> --- a/Documentation/ABI/testing/sysfs-kernel-fadump
>> +++ b/Documentation/ABI/testing/sysfs-kernel-fadump
> [...]
>> +-- Support for Additional Kernel Arguments in Fadump
>> +   Fadump has a feature that allows passing additional kernel arguments
>> +   to the fadump kernel. This feature was primarily designed to disable
>> +   kernel functionalities that are not required for the fadump kernel
>> +   and to reduce its memory footprint while collecting the dump.
>> +
>> +  Command to Add Additional Kernel Parameters to Fadump:
>> +  e.g.
>> +  # echo "nr_cpus=16" > /sys/kernel/fadump/bootargs_append
>> +
>> +  The above command is sufficient to add additional arguments to fadump.
>> +  An explicit service restart is not required.
>> +
>> +  Command to Retrieve the Additional Fadump Arguments:
>> +  e.g.
>> +  # cat /sys/kernel/fadump/bootargs_append
>> +
>> +Note: Additional kernel arguments for fadump with HASH MMU is only
>> +      supported if the RMA size is greater than 768 MB. If the RMA
>> +      size is less than 768 MB, the kernel does not export the
>> +      /sys/kernel/fadump/bootargs_append sysfs node.
> Reviewed-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>

Thanks for the review.

- Sourabh Jain

Re: [PATCH v3 4/4] Documentation/powerpc/fadump: add additional parameter feature details

[Hari Bathini]

Hi Sourabh,

Thanks for updating the documentation..

On 23/01/25 5:12 pm, Sourabh Jain wrote:
> Update the fadump document to include details about the fadump
> additional parameter feature.
> 
> The document includes the following:
> - Significance of the feature
> - How to use it
> - Feature restrictions
> 
> No functional changes are introduced.
> 
> Cc: Avnish Chouhan <avnish@linux.ibm.com>
> Cc: Brian King <brking@linux.ibm.com>
> Cc: Hari Bathini <hbathini@linux.ibm.com>
> Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Mahesh Salgaonkar <mahesh@linux.ibm.com>
> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
> ---
>   Documentation/ABI/testing/sysfs-kernel-fadump |  3 ++-
>   .../arch/powerpc/firmware-assisted-dump.rst   | 22 +++++++++++++++++++
>   2 files changed, 24 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/ABI/testing/sysfs-kernel-fadump b/Documentation/ABI/testing/sysfs-kernel-fadump
> index 2f9daa7ca55b..b64b7622e6fc 100644
> --- a/Documentation/ABI/testing/sysfs-kernel-fadump
> +++ b/Documentation/ABI/testing/sysfs-kernel-fadump
> @@ -55,4 +55,5 @@ Date:		May 2024
>   Contact:	linuxppc-dev@lists.ozlabs.org
>   Description:	read/write
>   		This is a special sysfs file available to setup additional
> -		parameters to be passed to capture kernel.
> +		parameters to be passed to capture kernel. For HASH MMU it
> +		is exported only if RMA size higher than 768MB.
> diff --git a/Documentation/arch/powerpc/firmware-assisted-dump.rst b/Documentation/arch/powerpc/firmware-assisted-dump.rst
> index 7e37aadd1f77..7e266e749cd5 100644
> --- a/Documentation/arch/powerpc/firmware-assisted-dump.rst
> +++ b/Documentation/arch/powerpc/firmware-assisted-dump.rst
> @@ -120,6 +120,28 @@ to ensure that crash data is preserved to process later.
>      e.g.
>        # echo 1 > /sys/firmware/opal/mpipl/release_core
>
    > +-- Support for Additional Kernel Arguments in Fadump

"Support to add additional kernel parameters to FADump capture kernel" ?

> +   Fadump has a feature that allows passing additional kernel arguments
> +   to the fadump kernel. This feature was primarily designed to disable
> +   kernel functionalities that are not required for the fadump kernel
> +   and to reduce its memory footprint while collecting the dump.

The below details are more appropriate in "Sysfs/debugfs files:" section
with the details going under /sys/kernel/fadump/bootargs_append:

> +
> +  Command to Add Additional Kernel Parameters to Fadump:

Do those words really need to start with Uppercase?

> +  e.g.
> +  # echo "nr_cpus=16" > /sys/kernel/fadump/bootargs_append
> +
> +  The above command is sufficient to add additional arguments to fadump.
> +  An explicit service restart is not required.
> +
> +  Command to Retrieve the Additional Fadump Arguments:
> +  e.g.
> +  # cat /sys/kernel/fadump/bootargs_append
> +


> +Note: Additional kernel arguments for fadump with HASH MMU is only
> +      supported if the RMA size is greater than 768 MB. If the RMA
> +      size is less than 768 MB, the kernel does not export the
> +      /sys/kernel/fadump/bootargs_append sysfs node.
> +

- Hari

Re: [PATCH v3 2/4] powerpc/fadump: fix additional param memory reservation for HASH MMU

[Hari Bathini]

Hi Sourabh,

On 23/01/25 5:12 pm, Sourabh Jain wrote:
> Commit 683eab94da75bc ("powerpc/fadump: setup additional parameters for
> dump capture kernel") introduced the additional parameter feature in
> fadump for HASH MMU with the understanding that GRUB does not use the
> memory area between 640MB and 768MB for its operation.
> 
> However, the third patch in this series ("powerpc: increase MIN RMA
> size for CAS negotiation") changes the MIN RMA size to 768MB, allowing
> GRUB to use memory up to 768MB. This makes the fadump reservation for
> the additional parameter feature for HASH MMU unreliable.
> 
> To address this, adjust the memory range for the additional parameter in
> fadump for HASH MMU. This will ensure that GRUB does not overwrite the
> memory reserved for fadump's additional parameter in HASH MMU.
> 
> The new policy for the memory range for the additional parameter in HASH
> MMU is that the first memory block must be larger than the MIN_RMA size,
> as the bootloader can use memory up to the MIN_RMA size. The range
> should be between MIN_RMA and the RMA size (ppc64_rma_size), and it must
> not overlap with the fadump reserved area.
> 
> Cc: Avnish Chouhan <avnish@linux.ibm.com>
> Cc: Brian King <brking@linux.ibm.com>
> Cc: Hari Bathini <hbathini@linux.ibm.com>
> Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Reviewed-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
> ---
>   arch/powerpc/kernel/fadump.c | 21 +++++++++++----------
>   1 file changed, 11 insertions(+), 10 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/fadump.c b/arch/powerpc/kernel/fadump.c
> index 4b371c738213..26e3d151e048 100644
> --- a/arch/powerpc/kernel/fadump.c
> +++ b/arch/powerpc/kernel/fadump.c
> @@ -33,6 +33,7 @@
>   #include <asm/fadump-internal.h>
>   #include <asm/setup.h>
>   #include <asm/interrupt.h>
> +#include <asm/prom.h>
>   
>   /*
>    * The CPU who acquired the lock to trigger the fadump crash should
> @@ -1764,19 +1765,19 @@ void __init fadump_setup_param_area(void)
>   		range_end = memblock_end_of_DRAM();
>   	} else {
>   		/*
> -		 * Passing additional parameters is supported for hash MMU only
> -		 * if the first memory block size is 768MB or higher.
> +		 * Memory range for passing additional parameters for HASH MMU
> +		 * must meet the following conditions:
> +		 * 1. The first memory block size must be higher than the
> +		 *    minimum RMA (MIN_RMA) size. Bootloader can use memory

> +		 *    upto RMA size. So it should be avoided.

I believe you mean "upto MIN_RMA size." here?

Rest looks good.

Reviewed-by: Hari Bathini <hbathini@linux.ibm.com>


> +		 * 2. The range should be between MIN_RMA and RMA size (ppc64_rma_size)
> +		 * 3. It must not overlap with the fadump reserved area.
>   		 */
> -		if (ppc64_rma_size < 0x30000000)
> +		if (ppc64_rma_size < MIN_RMA*1024*1024)
>   			return;
>   
> -		/*
> -		 * 640 MB to 768 MB is not used by PFW/bootloader. So, try reserving
> -		 * memory for passing additional parameters in this range to avoid
> -		 * being stomped on by PFW/bootloader.
> -		 */
> -		range_start = 0x2A000000;
> -		range_end = range_start + 0x4000000;
> +		range_start = MIN_RMA * 1024 * 1024;
> +		range_end = min(ppc64_rma_size, fw_dump.boot_mem_top);
>   	}
>   
>   	fw_dump.param_area = memblock_phys_alloc_range(COMMAND_LINE_SIZE,

Re: [PATCH v3 2/4] powerpc/fadump: fix additional param memory reservation for HASH MMU

[Sourabh Jain]

On 10/02/25 12:48, Hari Bathini wrote:
> Hi Sourabh,
>
> On 23/01/25 5:12 pm, Sourabh Jain wrote:
>> Commit 683eab94da75bc ("powerpc/fadump: setup additional parameters for
>> dump capture kernel") introduced the additional parameter feature in
>> fadump for HASH MMU with the understanding that GRUB does not use the
>> memory area between 640MB and 768MB for its operation.
>>
>> However, the third patch in this series ("powerpc: increase MIN RMA
>> size for CAS negotiation") changes the MIN RMA size to 768MB, allowing
>> GRUB to use memory up to 768MB. This makes the fadump reservation for
>> the additional parameter feature for HASH MMU unreliable.
>>
>> To address this, adjust the memory range for the additional parameter in
>> fadump for HASH MMU. This will ensure that GRUB does not overwrite the
>> memory reserved for fadump's additional parameter in HASH MMU.
>>
>> The new policy for the memory range for the additional parameter in HASH
>> MMU is that the first memory block must be larger than the MIN_RMA size,
>> as the bootloader can use memory up to the MIN_RMA size. The range
>> should be between MIN_RMA and the RMA size (ppc64_rma_size), and it must
>> not overlap with the fadump reserved area.
>>
>> Cc: Avnish Chouhan <avnish@linux.ibm.com>
>> Cc: Brian King <brking@linux.ibm.com>
>> Cc: Hari Bathini <hbathini@linux.ibm.com>
>> Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
>> Cc: Michael Ellerman <mpe@ellerman.id.au>
>> Reviewed-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
>> Signed-off-by: Sourabh Jain <sourabhjain@linux.ibm.com>
>> ---
>>   arch/powerpc/kernel/fadump.c | 21 +++++++++++----------
>>   1 file changed, 11 insertions(+), 10 deletions(-)
>>
>> diff --git a/arch/powerpc/kernel/fadump.c b/arch/powerpc/kernel/fadump.c
>> index 4b371c738213..26e3d151e048 100644
>> --- a/arch/powerpc/kernel/fadump.c
>> +++ b/arch/powerpc/kernel/fadump.c
>> @@ -33,6 +33,7 @@
>>   #include <asm/fadump-internal.h>
>>   #include <asm/setup.h>
>>   #include <asm/interrupt.h>
>> +#include <asm/prom.h>
>>     /*
>>    * The CPU who acquired the lock to trigger the fadump crash should
>> @@ -1764,19 +1765,19 @@ void __init fadump_setup_param_area(void)
>>           range_end = memblock_end_of_DRAM();
>>       } else {
>>           /*
>> -         * Passing additional parameters is supported for hash MMU only
>> -         * if the first memory block size is 768MB or higher.
>> +         * Memory range for passing additional parameters for HASH MMU
>> +         * must meet the following conditions:
>> +         * 1. The first memory block size must be higher than the
>> +         *    minimum RMA (MIN_RMA) size. Bootloader can use memory
>
>> +         *    upto RMA size. So it should be avoided.
>
> I believe you mean "upto MIN_RMA size." here?

Yeah it should MIN_RMA.

>
> Rest looks good.
>
> Reviewed-by: Hari Bathini <hbathini@linux.ibm.com>

Thanks for the review.

- Sourabh Jain

Re: [PATCH v3 0/4] powerpc: increase MIN RMA size for CAS negotiation

[Madhavan Srinivasan]

On Thu, 23 Jan 2025 17:12:50 +0530, Sourabh Jain wrote:
> Change RMA size from 512 MB to 768 MB which will result
> in more RMA at boot time for PowerPC. When PowerPC LPAR use/uses vTPM,
> Secure Boot or FADump, the 512 MB RMA memory is not sufficient for
> booting. With this 512 MB RMA, GRUB2 run out of memory and unable to
> load the necessary. Sometimes even usage of CDROM which requires more
> memory for installation along with the options mentioned above troubles
> the boot memory and result in boot failures. Increasing the RMA size
> will resolves multiple out of memory issues observed in PowerPC.
> 
> [...]

Applied to powerpc/next.

[1/4] powerpc: export MIN RMA size
      https://git.kernel.org/powerpc/c/0bdd7ff5b830fadc18254399d6340cdaa4271527
[2/4] powerpc/fadump: fix additional param memory reservation for HASH MMU
      https://git.kernel.org/powerpc/c/b7bb460624570e8169eaaa55f7a410fda1bc0c2b
[3/4] powerpc: increase MIN RMA size for CAS negotiation
      https://git.kernel.org/powerpc/c/fdc44538d57caf4e96f57d5f0d0c89c4aa079f94
[4/4] Documentation/powerpc/fadump: add additional parameter feature details
      https://git.kernel.org/powerpc/c/61c403b5d000b46b8703595ea16533d0cb2a2911

Thanks