[PATCH -next v3 0/2] support allocating crashkernel above 4G explicitly on riscv

[PATCH -next v3 0/2] support allocating crashkernel above 4G explicitly on riscv

[Chen Jiahao]

On riscv, the current crash kernel allocation logic is trying to
allocate within 32bit addressible memory region by default, if
failed, try to allocate without 4G restriction.

In need of saving DMA zone memory while allocating a relatively large
crash kernel region, allocating the reserved memory top down in
high memory, without overlapping the DMA zone, is a mature solution.
Hence this patchset introduces the parameter option crashkernel=X,[high,low].

One can reserve the crash kernel from high memory above DMA zone range
by explicitly passing "crashkernel=X,high"; or reserve a memory range
below 4G with "crashkernel=X,low". Besides, there are few rules need
to take notice:
1. "crashkernel=X,[high,low]" will be ignored if "crashkernel=size"
   is specified.
2. "crashkernel=X,low" is valid only when "crashkernel=X,high" is passed
   and there is enough memory to be allocated under 4G.
3. When allocating crashkernel above 4G and no "crashkernel=X,low" is
   specified, a 128M low memory will be allocated automatically for
   swiotlb bounce buffer.
See Documentation/admin-guide/kernel-parameters.txt for more information.

To verify loading the crashkernel, adapted kexec-tools is attached below:
https://github.com/chenjh005/kexec-tools/tree/build-test-riscv-v2

Following test cases have been performed as expected:
1) crashkernel=256M                          //low=256M
2) crashkernel=1G                            //low=1G
3) crashkernel=4G                            //high=4G, low=128M(default)
4) crashkernel=4G crashkernel=256M,high      //high=4G, low=128M(default), high is ignored
5) crashkernel=4G crashkernel=256M,low       //high=4G, low=128M(default), low is ignored
6) crashkernel=4G,high                       //high=4G, low=128M(default)
7) crashkernel=256M,low                      //low=0M, invalid
8) crashkernel=4G,high crashkernel=256M,low  //high=4G, low=256M
9) crashkernel=4G,high crashkernel=4G,low    //high=0M, low=0M, invalid
10) crashkernel=512M@0xd0000000              //low=512M
11) crashkernel=1G@0xe0000000                //high=0M, low=0M, no enough low memory, failed

Changes since [v3]:
1. Update to print warning and return explicitly on failure when
   crashkernel=size@offset is specified. Not changing the result
   in this case but making the logic more straightforward.
2. Some minor cleanup.

Changes since [v2]:
1. Update the allocation logic to ensure the high crashkernel
   region is reserved strictly above dma32_phys_limit.
2. Clean up some minor format problems.

Chen Jiahao (2):
  riscv: kdump: Implement crashkernel=X,[high,low]
  docs: kdump: Update the crashkernel description for riscv

 .../admin-guide/kernel-parameters.txt         | 15 ++--
 arch/riscv/kernel/setup.c                     |  5 ++
 arch/riscv/mm/init.c                          | 74 +++++++++++++++++--
 3 files changed, 82 insertions(+), 12 deletions(-)

-- 
2.31.1

[PATCH -next v3 1/2] riscv: kdump: Implement crashkernel=X,[high,low]

[Chen Jiahao]

On riscv, the current crash kernel allocation logic is trying to
allocate within 32bit addressible memory region by default, if
failed, try to allocate without 4G restriction.

In need of saving DMA zone memory while allocating a relatively large
crash kernel region, allocating the reserved memory top down in
high memory, without overlapping the DMA zone, is a mature solution.
Here introduce the parameter option crashkernel=X,[high,low].

One can reserve the crash kernel from high memory above DMA zone range
by explicitly passing "crashkernel=X,high"; or reserve a memory range
below 4G with "crashkernel=X,low".

Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>
---
 arch/riscv/kernel/setup.c |  5 +++
 arch/riscv/mm/init.c      | 74 ++++++++++++++++++++++++++++++++++++---
 2 files changed, 74 insertions(+), 5 deletions(-)

diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 5d3184cbf518..ea84e5047c23 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -176,6 +176,11 @@ static void __init init_resources(void)
 		if (ret < 0)
 			goto error;
 	}
+	if (crashk_low_res.start != crashk_low_res.end) {
+		ret = add_resource(&iomem_resource, &crashk_low_res);
+		if (ret < 0)
+			goto error;
+	}
 #endif
 
 #ifdef CONFIG_CRASH_DUMP
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
index 478d6763a01a..b5b457193423 100644
--- a/arch/riscv/mm/init.c
+++ b/arch/riscv/mm/init.c
@@ -1152,6 +1152,28 @@ static inline void setup_vm_final(void)
 }
 #endif /* CONFIG_MMU */
 
+/* Reserve 128M low memory by default for swiotlb buffer */
+#define DEFAULT_CRASH_KERNEL_LOW_SIZE	(128UL << 20)
+
+static int __init reserve_crashkernel_low(unsigned long long low_size)
+{
+	unsigned long long low_base;
+
+	low_base = memblock_phys_alloc_range(low_size, PMD_SIZE, 0, dma32_phys_limit);
+	if (!low_base) {
+		pr_err("cannot allocate crashkernel low memory (size:0x%llx).\n", low_size);
+		return -ENOMEM;
+	}
+
+	pr_info("crashkernel low memory reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
+		low_base, low_base + low_size, low_size >> 20);
+
+	crashk_low_res.start = low_base;
+	crashk_low_res.end = low_base + low_size - 1;
+
+	return 0;
+}
+
 /*
  * reserve_crashkernel() - reserves memory for crash kernel
  *
@@ -1163,8 +1185,12 @@ static void __init reserve_crashkernel(void)
 {
 	unsigned long long crash_base = 0;
 	unsigned long long crash_size = 0;
+	unsigned long long crash_low_size = 0;
 	unsigned long search_start = memblock_start_of_DRAM();
 	unsigned long search_end = memblock_end_of_DRAM();
+	unsigned long search_low_max = (unsigned long)dma32_phys_limit;
+	char *cmdline = boot_command_line;
+	bool fixed_base = false;
 
 	int ret = 0;
 
@@ -1180,14 +1206,37 @@ static void __init reserve_crashkernel(void)
 		return;
 	}
 
-	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
+	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
 				&crash_size, &crash_base);
-	if (ret || !crash_size)
+	if (ret == -ENOENT) {
+		/*
+		 * crashkernel=X,[high,low] can be specified or not, but
+		 * invalid value is not allowed.
+		 */
+		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
+		if (ret || !crash_size)
+			return;
+
+		/*
+		 * crashkernel=Y,low is valid only when crashkernel=X,high
+		 * is passed and high memory is reserved successful.
+		 */
+		ret = parse_crashkernel_low(cmdline, 0, &crash_low_size, &crash_base);
+		if (ret == -ENOENT)
+			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
+		else if (ret)
+			return;
+
+		search_start = search_low_max;
+	} else if (ret || !crash_size) {
+		/* Invalid argument value specified */
 		return;
+	}
 
 	crash_size = PAGE_ALIGN(crash_size);
 
 	if (crash_base) {
+		fixed_base = true;
 		search_start = crash_base;
 		search_end = crash_base + crash_size;
 	}
@@ -1201,16 +1250,31 @@ static void __init reserve_crashkernel(void)
 	 */
 	crash_base = memblock_phys_alloc_range(crash_size, PMD_SIZE,
 					       search_start,
-					       min(search_end, (unsigned long) SZ_4G));
+					       min(search_end, search_low_max));
 	if (crash_base == 0) {
-		/* Try again without restricting region to 32bit addressible memory */
+		if (fixed_base) {
+			pr_warn("crashkernel: allocating failed with given size@offset\n");
+			return;
+		}
+
+		/* Try again above the region of 32bit addressible memory */
 		crash_base = memblock_phys_alloc_range(crash_size, PMD_SIZE,
-						search_start, search_end);
+						       max(search_start, search_low_max),
+						       search_end);
 		if (crash_base == 0) {
 			pr_warn("crashkernel: couldn't allocate %lldKB\n",
 				crash_size >> 10);
 			return;
 		}
+
+		if (!crash_low_size)
+			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
+	}
+
+	if ((crash_base > dma32_phys_limit - crash_low_size) &&
+	    crash_low_size && reserve_crashkernel_low(crash_low_size)) {
+		memblock_phys_free(crash_base, crash_size);
+		return;
 	}
 
 	pr_info("crashkernel: reserved 0x%016llx - 0x%016llx (%lld MB)\n",
-- 
2.31.1

[PATCH -next v3 2/2] docs: kdump: Update the crashkernel description for riscv

[Chen Jiahao]

Now "crashkernel=" parameter on riscv has been updated to support
crashkernel=X,[high,low]. Through which we can reserve memory region
above/within 32bit addressible DMA zone.

Here update the parameter description accordingly.

Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>
---
 Documentation/admin-guide/kernel-parameters.txt | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 54702bd488eb..41865aae9eaa 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -866,7 +866,7 @@
 			memory region [offset, offset + size] for that kernel
 			image. If '@offset' is omitted, then a suitable offset
 			is selected automatically.
-			[KNL, X86-64, ARM64] Select a region under 4G first, and
+			[KNL, X86-64, ARM64, RISCV] Select a region under 4G first, and
 			fall back to reserve region above 4G when '@offset'
 			hasn't been specified.
 			See Documentation/admin-guide/kdump/kdump.rst for further details.
@@ -879,14 +879,14 @@
 			Documentation/admin-guide/kdump/kdump.rst for an example.
 
 	crashkernel=size[KMG],high
-			[KNL, X86-64, ARM64] range could be above 4G. Allow kernel
-			to allocate physical memory region from top, so could
-			be above 4G if system have more than 4G ram installed.
-			Otherwise memory region will be allocated below 4G, if
-			available.
+			[KNL, X86-64, ARM64, RISCV] range could be above 4G.
+			Allow kernel to allocate physical memory region from top,
+			so could be above 4G if system have more than 4G ram
+			installed. Otherwise memory region will be allocated
+			below 4G, if available.
 			It will be ignored if crashkernel=X is specified.
 	crashkernel=size[KMG],low
-			[KNL, X86-64, ARM64] range under 4G. When crashkernel=X,high
+			[KNL, X86-64, ARM64, RISCV] range under 4G. When crashkernel=X,high
 			is passed, kernel could allocate physical memory region
 			above 4G, that cause second kernel crash on system
 			that require some amount of low memory, e.g. swiotlb
@@ -897,6 +897,7 @@
 			size is	platform dependent.
 			  --> x86: max(swiotlb_size_or_default() + 8MiB, 256MiB)
 			  --> arm64: 128MiB
+			  --> riscv: 128MiB
 			This one lets the user specify own low range under 4G
 			for second kernel instead.
 			0: to disable low allocation.
-- 
2.31.1

Re: [PATCH -next v3 2/2] docs: kdump: Update the crashkernel description for riscv

[Guo Ren]

Reviewed-by: Guo Ren <guoren@kernel.org>

On Thu, Apr 6, 2023 at 10:05 PM Chen Jiahao <chenjiahao16@huawei.com> wrote:
>
> Now "crashkernel=" parameter on riscv has been updated to support
> crashkernel=X,[high,low]. Through which we can reserve memory region
> above/within 32bit addressible DMA zone.
>
> Here update the parameter description accordingly.
>
> Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt | 15 ++++++++-------
>  1 file changed, 8 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 54702bd488eb..41865aae9eaa 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -866,7 +866,7 @@
>                         memory region [offset, offset + size] for that kernel
>                         image. If '@offset' is omitted, then a suitable offset
>                         is selected automatically.
> -                       [KNL, X86-64, ARM64] Select a region under 4G first, and
> +                       [KNL, X86-64, ARM64, RISCV] Select a region under 4G first, and
>                         fall back to reserve region above 4G when '@offset'
>                         hasn't been specified.
>                         See Documentation/admin-guide/kdump/kdump.rst for further details.
> @@ -879,14 +879,14 @@
>                         Documentation/admin-guide/kdump/kdump.rst for an example.
>
>         crashkernel=size[KMG],high
> -                       [KNL, X86-64, ARM64] range could be above 4G. Allow kernel
> -                       to allocate physical memory region from top, so could
> -                       be above 4G if system have more than 4G ram installed.
> -                       Otherwise memory region will be allocated below 4G, if
> -                       available.
> +                       [KNL, X86-64, ARM64, RISCV] range could be above 4G.
> +                       Allow kernel to allocate physical memory region from top,
> +                       so could be above 4G if system have more than 4G ram
> +                       installed. Otherwise memory region will be allocated
> +                       below 4G, if available.
>                         It will be ignored if crashkernel=X is specified.
>         crashkernel=size[KMG],low
> -                       [KNL, X86-64, ARM64] range under 4G. When crashkernel=X,high
> +                       [KNL, X86-64, ARM64, RISCV] range under 4G. When crashkernel=X,high
>                         is passed, kernel could allocate physical memory region
>                         above 4G, that cause second kernel crash on system
>                         that require some amount of low memory, e.g. swiotlb
> @@ -897,6 +897,7 @@
>                         size is platform dependent.
>                           --> x86: max(swiotlb_size_or_default() + 8MiB, 256MiB)
>                           --> arm64: 128MiB
> +                         --> riscv: 128MiB
>                         This one lets the user specify own low range under 4G
>                         for second kernel instead.
>                         0: to disable low allocation.
> --
> 2.31.1
>


-- 
Best Regards
 Guo Ren

Re: [PATCH -next v3 1/2] riscv: kdump: Implement crashkernel=X,[high,low]

[Guo Ren]

On Thu, Apr 6, 2023 at 10:06 PM Chen Jiahao <chenjiahao16@huawei.com> wrote:
>
> On riscv, the current crash kernel allocation logic is trying to
> allocate within 32bit addressible memory region by default, if
> failed, try to allocate without 4G restriction.
>
> In need of saving DMA zone memory while allocating a relatively large
> crash kernel region, allocating the reserved memory top down in
> high memory, without overlapping the DMA zone, is a mature solution.
> Here introduce the parameter option crashkernel=X,[high,low].
>
> One can reserve the crash kernel from high memory above DMA zone range
> by explicitly passing "crashkernel=X,high"; or reserve a memory range
> below 4G with "crashkernel=X,low".
Asked-by: Guo Ren <guoren@kernel.org>

>
> Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>
> ---
>  arch/riscv/kernel/setup.c |  5 +++
>  arch/riscv/mm/init.c      | 74 ++++++++++++++++++++++++++++++++++++---
>  2 files changed, 74 insertions(+), 5 deletions(-)
>
> diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
> index 5d3184cbf518..ea84e5047c23 100644
> --- a/arch/riscv/kernel/setup.c
> +++ b/arch/riscv/kernel/setup.c
> @@ -176,6 +176,11 @@ static void __init init_resources(void)
>                 if (ret < 0)
>                         goto error;
>         }
> +       if (crashk_low_res.start != crashk_low_res.end) {
> +               ret = add_resource(&iomem_resource, &crashk_low_res);
> +               if (ret < 0)
> +                       goto error;
> +       }
>  #endif
>
>  #ifdef CONFIG_CRASH_DUMP
> diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
> index 478d6763a01a..b5b457193423 100644
> --- a/arch/riscv/mm/init.c
> +++ b/arch/riscv/mm/init.c
> @@ -1152,6 +1152,28 @@ static inline void setup_vm_final(void)
>  }
>  #endif /* CONFIG_MMU */
>
> +/* Reserve 128M low memory by default for swiotlb buffer */
> +#define DEFAULT_CRASH_KERNEL_LOW_SIZE  (128UL << 20)
> +
> +static int __init reserve_crashkernel_low(unsigned long long low_size)
> +{
> +       unsigned long long low_base;
> +
> +       low_base = memblock_phys_alloc_range(low_size, PMD_SIZE, 0, dma32_phys_limit);
> +       if (!low_base) {
> +               pr_err("cannot allocate crashkernel low memory (size:0x%llx).\n", low_size);
> +               return -ENOMEM;
> +       }
> +
> +       pr_info("crashkernel low memory reserved: 0x%016llx - 0x%016llx (%lld MB)\n",
> +               low_base, low_base + low_size, low_size >> 20);
> +
> +       crashk_low_res.start = low_base;
> +       crashk_low_res.end = low_base + low_size - 1;
> +
> +       return 0;
> +}
> +
>  /*
>   * reserve_crashkernel() - reserves memory for crash kernel
>   *
> @@ -1163,8 +1185,12 @@ static void __init reserve_crashkernel(void)
>  {
>         unsigned long long crash_base = 0;
>         unsigned long long crash_size = 0;
> +       unsigned long long crash_low_size = 0;
>         unsigned long search_start = memblock_start_of_DRAM();
>         unsigned long search_end = memblock_end_of_DRAM();
> +       unsigned long search_low_max = (unsigned long)dma32_phys_limit;
> +       char *cmdline = boot_command_line;
> +       bool fixed_base = false;
>
>         int ret = 0;
>
> @@ -1180,14 +1206,37 @@ static void __init reserve_crashkernel(void)
>                 return;
>         }
>
> -       ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
> +       ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
>                                 &crash_size, &crash_base);
> -       if (ret || !crash_size)
> +       if (ret == -ENOENT) {
> +               /*
> +                * crashkernel=X,[high,low] can be specified or not, but
> +                * invalid value is not allowed.
> +                */
> +               ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
> +               if (ret || !crash_size)
> +                       return;
> +
> +               /*
> +                * crashkernel=Y,low is valid only when crashkernel=X,high
> +                * is passed and high memory is reserved successful.
> +                */
> +               ret = parse_crashkernel_low(cmdline, 0, &crash_low_size, &crash_base);
> +               if (ret == -ENOENT)
> +                       crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
> +               else if (ret)
> +                       return;
> +
> +               search_start = search_low_max;
> +       } else if (ret || !crash_size) {
> +               /* Invalid argument value specified */
>                 return;
> +       }
>
>         crash_size = PAGE_ALIGN(crash_size);
>
>         if (crash_base) {
> +               fixed_base = true;
>                 search_start = crash_base;
>                 search_end = crash_base + crash_size;
>         }
> @@ -1201,16 +1250,31 @@ static void __init reserve_crashkernel(void)
>          */
>         crash_base = memblock_phys_alloc_range(crash_size, PMD_SIZE,
>                                                search_start,
> -                                              min(search_end, (unsigned long) SZ_4G));
> +                                              min(search_end, search_low_max));
>         if (crash_base == 0) {
> -               /* Try again without restricting region to 32bit addressible memory */
> +               if (fixed_base) {
> +                       pr_warn("crashkernel: allocating failed with given size@offset\n");
> +                       return;
> +               }
> +
> +               /* Try again above the region of 32bit addressible memory */
>                 crash_base = memblock_phys_alloc_range(crash_size, PMD_SIZE,
> -                                               search_start, search_end);
> +                                                      max(search_start, search_low_max),
> +                                                      search_end);
>                 if (crash_base == 0) {
>                         pr_warn("crashkernel: couldn't allocate %lldKB\n",
>                                 crash_size >> 10);
>                         return;
>                 }
> +
> +               if (!crash_low_size)
> +                       crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
> +       }
> +
> +       if ((crash_base > dma32_phys_limit - crash_low_size) &&
> +           crash_low_size && reserve_crashkernel_low(crash_low_size)) {
> +               memblock_phys_free(crash_base, crash_size);
> +               return;
>         }
>
>         pr_info("crashkernel: reserved 0x%016llx - 0x%016llx (%lld MB)\n",
> --
> 2.31.1
>


-- 
Best Regards
 Guo Ren

Re: [PATCH -next v3 1/2] riscv: kdump: Implement crashkernel=X,[high,low]

[Simon Horman]

On Fri, Apr 07, 2023 at 06:02:05AM +0800, Chen Jiahao wrote:
> On riscv, the current crash kernel allocation logic is trying to
> allocate within 32bit addressible memory region by default, if
> failed, try to allocate without 4G restriction.
> 
> In need of saving DMA zone memory while allocating a relatively large
> crash kernel region, allocating the reserved memory top down in
> high memory, without overlapping the DMA zone, is a mature solution.
> Here introduce the parameter option crashkernel=X,[high,low].
> 
> One can reserve the crash kernel from high memory above DMA zone range
> by explicitly passing "crashkernel=X,high"; or reserve a memory range
> below 4G with "crashkernel=X,low".
> 
> Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>

...

> @@ -1180,14 +1206,37 @@ static void __init reserve_crashkernel(void)
>  		return;
>  	}
>  
> -	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
> +	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
>  				&crash_size, &crash_base);
> -	if (ret || !crash_size)
> +	if (ret == -ENOENT) {
> +		/*
> +		 * crashkernel=X,[high,low] can be specified or not, but
> +		 * invalid value is not allowed.

nit: Perhaps something like this would be easier to correlate with the
     code that follows:

		/* Fallback to crashkernel=X,[high,low] */


> +		 */
> +		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
> +		if (ret || !crash_size)
> +			return;
> +
> +		/*
> +		 * crashkernel=Y,low is valid only when crashkernel=X,high
> +		 * is passed and high memory is reserved successful.

nit: s/successful/successfully/

> +		 */
> +		ret = parse_crashkernel_low(cmdline, 0, &crash_low_size, &crash_base);
> +		if (ret == -ENOENT)
> +			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
> +		else if (ret)
> +			return;
> +
> +		search_start = search_low_max;
> +	} else if (ret || !crash_size) {
> +		/* Invalid argument value specified */
>  		return;
> +	}

...

Re: [PATCH -next v3 1/2] riscv: kdump: Implement crashkernel=X,[high,low]

[Simon Horman]

On Fri, Apr 07, 2023 at 05:06:24PM +0800, Guo Ren wrote:
> On Thu, Apr 6, 2023 at 10:06 PM Chen Jiahao <chenjiahao16@huawei.com> wrote:
> >
> > On riscv, the current crash kernel allocation logic is trying to
> > allocate within 32bit addressible memory region by default, if
> > failed, try to allocate without 4G restriction.
> >
> > In need of saving DMA zone memory while allocating a relatively large
> > crash kernel region, allocating the reserved memory top down in
> > high memory, without overlapping the DMA zone, is a mature solution.
> > Here introduce the parameter option crashkernel=X,[high,low].
> >
> > One can reserve the crash kernel from high memory above DMA zone range
> > by explicitly passing "crashkernel=X,high"; or reserve a memory range
> > below 4G with "crashkernel=X,low".
> Asked-by: Guo Ren <guoren@kernel.org>

Perhaps 'Acked-by' :)

Re: [PATCH -next v3 2/2] docs: kdump: Update the crashkernel description for riscv

[Simon Horman]

On Fri, Apr 07, 2023 at 06:02:06AM +0800, Chen Jiahao wrote:
> Now "crashkernel=" parameter on riscv has been updated to support
> crashkernel=X,[high,low]. Through which we can reserve memory region
> above/within 32bit addressible DMA zone.
> 
> Here update the parameter description accordingly.
> 
> Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>

Reviewed-by: Simon Horman <horms@kernel.org>

Re: [PATCH -next v3 0/2] support allocating crashkernel above 4G explicitly on riscv

[Simon Horman]

On Fri, Apr 07, 2023 at 06:02:04AM +0800, Chen Jiahao wrote:
> On riscv, the current crash kernel allocation logic is trying to
> allocate within 32bit addressible memory region by default, if
> failed, try to allocate without 4G restriction.
> 
> In need of saving DMA zone memory while allocating a relatively large
> crash kernel region, allocating the reserved memory top down in
> high memory, without overlapping the DMA zone, is a mature solution.
> Hence this patchset introduces the parameter option crashkernel=X,[high,low].
> 
> One can reserve the crash kernel from high memory above DMA zone range
> by explicitly passing "crashkernel=X,high"; or reserve a memory range
> below 4G with "crashkernel=X,low". Besides, there are few rules need
> to take notice:
> 1. "crashkernel=X,[high,low]" will be ignored if "crashkernel=size"
>    is specified.
> 2. "crashkernel=X,low" is valid only when "crashkernel=X,high" is passed
>    and there is enough memory to be allocated under 4G.
> 3. When allocating crashkernel above 4G and no "crashkernel=X,low" is
>    specified, a 128M low memory will be allocated automatically for
>    swiotlb bounce buffer.
> See Documentation/admin-guide/kernel-parameters.txt for more information.
> 
> To verify loading the crashkernel, adapted kexec-tools is attached below:
> https://github.com/chenjh005/kexec-tools/tree/build-test-riscv-v2

Are you planning to submit the kexec-tools patches too?

Re: [PATCH -next v3 1/2] riscv: kdump: Implement crashkernel=X,[high,low]

[Leizhen (ThunderTown)]

On 2023/4/7 20:03, Simon Horman wrote:
> On Fri, Apr 07, 2023 at 06:02:05AM +0800, Chen Jiahao wrote:
>> On riscv, the current crash kernel allocation logic is trying to
>> allocate within 32bit addressible memory region by default, if
>> failed, try to allocate without 4G restriction.
>>
>> In need of saving DMA zone memory while allocating a relatively large
>> crash kernel region, allocating the reserved memory top down in
>> high memory, without overlapping the DMA zone, is a mature solution.
>> Here introduce the parameter option crashkernel=X,[high,low].
>>
>> One can reserve the crash kernel from high memory above DMA zone range
>> by explicitly passing "crashkernel=X,high"; or reserve a memory range
>> below 4G with "crashkernel=X,low".
>>
>> Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>
> 
> ...
> 
>> @@ -1180,14 +1206,37 @@ static void __init reserve_crashkernel(void)
>>  		return;
>>  	}
>>  
>> -	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
>> +	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
>>  				&crash_size, &crash_base);
>> -	if (ret || !crash_size)
>> +	if (ret == -ENOENT) {
>> +		/*
>> +		 * crashkernel=X,[high,low] can be specified or not, but
>> +		 * invalid value is not allowed.
> 
> nit: Perhaps something like this would be easier to correlate with the
>      code that follows:
> 
> 		/* Fallback to crashkernel=X,[high,low] */

The description "crashkernel=X,[high,low] can be specified or not" is not
correct, because crashkernel=X,high must be specified when walking into this
branch. So use Simon's comments or copy arm64's comments(it's written for
parse_crashkernel_low()).

> 
> 
>> +		 */
>> +		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
>> +		if (ret || !crash_size)
>> +			return;
>> +
>> +		/*
>> +		 * crashkernel=Y,low is valid only when crashkernel=X,high
>> +		 * is passed and high memory is reserved successful.
> 
> nit: s/successful/successfully/

Seems like the whole "and high memory is reserved successful" needs to be deleted.
Only the dependency between the two boot options should be described here,
regardless of whether their memory is successfully allocated.

> 
>> +		 */
>> +		ret = parse_crashkernel_low(cmdline, 0, &crash_low_size, &crash_base);
>> +		if (ret == -ENOENT)
>> +			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
>> +		else if (ret)
>> +			return;
>> +
>> +		search_start = search_low_max;
>> +	} else if (ret || !crash_size) {
>> +		/* Invalid argument value specified */
>>  		return;
>> +	}
> 
> ...
> .
> 

-- 
Regards,
  Zhen Lei

Re: [PATCH -next v3 1/2] riscv: kdump: Implement crashkernel=X,[high,low]

[Leizhen (ThunderTown)]

On 2023/4/7 20:58, Leizhen (ThunderTown) wrote:
> 
> 
> On 2023/4/7 20:03, Simon Horman wrote:
>> On Fri, Apr 07, 2023 at 06:02:05AM +0800, Chen Jiahao wrote:
>>> On riscv, the current crash kernel allocation logic is trying to
>>> allocate within 32bit addressible memory region by default, if
>>> failed, try to allocate without 4G restriction.
>>>
>>> In need of saving DMA zone memory while allocating a relatively large
>>> crash kernel region, allocating the reserved memory top down in
>>> high memory, without overlapping the DMA zone, is a mature solution.
>>> Here introduce the parameter option crashkernel=X,[high,low].
>>>
>>> One can reserve the crash kernel from high memory above DMA zone range
>>> by explicitly passing "crashkernel=X,high"; or reserve a memory range
>>> below 4G with "crashkernel=X,low".
>>>
>>> Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>
>>
>> ...
>>
>>> @@ -1180,14 +1206,37 @@ static void __init reserve_crashkernel(void)
>>>  		return;
>>>  	}
>>>  
>>> -	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
>>> +	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
>>>  				&crash_size, &crash_base);
>>> -	if (ret || !crash_size)
>>> +	if (ret == -ENOENT) {
>>> +		/*
>>> +		 * crashkernel=X,[high,low] can be specified or not, but
>>> +		 * invalid value is not allowed.
>>
>> nit: Perhaps something like this would be easier to correlate with the
>>      code that follows:
>>
>> 		/* Fallback to crashkernel=X,[high,low] */
> 
> The description "crashkernel=X,[high,low] can be specified or not" is not
> correct, because crashkernel=X,high must be specified when walking into this
> branch. So use Simon's comments or copy arm64's comments(it's written for
> parse_crashkernel_low()).

I rethink it a little bit, if it's relative to crashkernel=X[@offset],
that's also true.

Reviewed-by: Zhen Lei <thunder.leizhen@huawei.com>

> 
>>
>>
>>> +		 */
>>> +		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
>>> +		if (ret || !crash_size)
>>> +			return;
>>> +
>>> +		/*
>>> +		 * crashkernel=Y,low is valid only when crashkernel=X,high
>>> +		 * is passed and high memory is reserved successful.
>>
>> nit: s/successful/successfully/
> 
> Seems like the whole "and high memory is reserved successful" needs to be deleted.
> Only the dependency between the two boot options should be described here,
> regardless of whether their memory is successfully allocated.
> 
>>
>>> +		 */
>>> +		ret = parse_crashkernel_low(cmdline, 0, &crash_low_size, &crash_base);
>>> +		if (ret == -ENOENT)
>>> +			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
>>> +		else if (ret)
>>> +			return;
>>> +
>>> +		search_start = search_low_max;
>>> +	} else if (ret || !crash_size) {
>>> +		/* Invalid argument value specified */
>>>  		return;
>>> +	}
>>
>> ...
>> .
>>
> 

-- 
Regards,
  Zhen Lei

Re: [PATCH -next v3 2/2] docs: kdump: Update the crashkernel description for riscv

[Leizhen (ThunderTown)]

On 2023/4/7 6:02, Chen Jiahao wrote:
> Now "crashkernel=" parameter on riscv has been updated to support
> crashkernel=X,[high,low]. Through which we can reserve memory region
> above/within 32bit addressible DMA zone.
> 
> Here update the parameter description accordingly.

Reviewed-by: Zhen Lei <thunder.leizhen@huawei.com>

> 
> Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt | 15 ++++++++-------
>  1 file changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 54702bd488eb..41865aae9eaa 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -866,7 +866,7 @@
>  			memory region [offset, offset + size] for that kernel
>  			image. If '@offset' is omitted, then a suitable offset
>  			is selected automatically.
> -			[KNL, X86-64, ARM64] Select a region under 4G first, and
> +			[KNL, X86-64, ARM64, RISCV] Select a region under 4G first, and
>  			fall back to reserve region above 4G when '@offset'
>  			hasn't been specified.
>  			See Documentation/admin-guide/kdump/kdump.rst for further details.
> @@ -879,14 +879,14 @@
>  			Documentation/admin-guide/kdump/kdump.rst for an example.
>  
>  	crashkernel=size[KMG],high
> -			[KNL, X86-64, ARM64] range could be above 4G. Allow kernel
> -			to allocate physical memory region from top, so could
> -			be above 4G if system have more than 4G ram installed.
> -			Otherwise memory region will be allocated below 4G, if
> -			available.
> +			[KNL, X86-64, ARM64, RISCV] range could be above 4G.
> +			Allow kernel to allocate physical memory region from top,
> +			so could be above 4G if system have more than 4G ram
> +			installed. Otherwise memory region will be allocated
> +			below 4G, if available.
>  			It will be ignored if crashkernel=X is specified.
>  	crashkernel=size[KMG],low
> -			[KNL, X86-64, ARM64] range under 4G. When crashkernel=X,high
> +			[KNL, X86-64, ARM64, RISCV] range under 4G. When crashkernel=X,high
>  			is passed, kernel could allocate physical memory region
>  			above 4G, that cause second kernel crash on system
>  			that require some amount of low memory, e.g. swiotlb
> @@ -897,6 +897,7 @@
>  			size is	platform dependent.
>  			  --> x86: max(swiotlb_size_or_default() + 8MiB, 256MiB)
>  			  --> arm64: 128MiB
> +			  --> riscv: 128MiB
>  			This one lets the user specify own low range under 4G
>  			for second kernel instead.
>  			0: to disable low allocation.
> 

-- 
Regards,
  Zhen Lei

Re: [PATCH -next v3 1/2] riscv: kdump: Implement crashkernel=X,[high,low]

[chenjiahao (C)]

On 2023/4/7 20:03, Simon Horman wrote:
> On Fri, Apr 07, 2023 at 06:02:05AM +0800, Chen Jiahao wrote:
>> On riscv, the current crash kernel allocation logic is trying to
>> allocate within 32bit addressible memory region by default, if
>> failed, try to allocate without 4G restriction.
>>
>> In need of saving DMA zone memory while allocating a relatively large
>> crash kernel region, allocating the reserved memory top down in
>> high memory, without overlapping the DMA zone, is a mature solution.
>> Here introduce the parameter option crashkernel=X,[high,low].
>>
>> One can reserve the crash kernel from high memory above DMA zone range
>> by explicitly passing "crashkernel=X,high"; or reserve a memory range
>> below 4G with "crashkernel=X,low".
>>
>> Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>
> ...
>
>> @@ -1180,14 +1206,37 @@ static void __init reserve_crashkernel(void)
>>   		return;
>>   	}
>>   
>> -	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
>> +	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
>>   				&crash_size, &crash_base);
>> -	if (ret || !crash_size)
>> +	if (ret == -ENOENT) {
>> +		/*
>> +		 * crashkernel=X,[high,low] can be specified or not, but
>> +		 * invalid value is not allowed.
> nit: Perhaps something like this would be easier to correlate with the
>       code that follows:
>
> 		/* Fallback to crashkernel=X,[high,low] */
>
Agreed, this would be more concise and accurate.
>> +		 */
>> +		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
>> +		if (ret || !crash_size)
>> +			return;
>> +
>> +		/*
>> +		 * crashkernel=Y,low is valid only when crashkernel=X,high
>> +		 * is passed and high memory is reserved successful.
> nit: s/successful/successfully/

I will fix above nits and resend another version later, thanks.


>> +		 */
>> +		ret = parse_crashkernel_low(cmdline, 0, &crash_low_size, &crash_base);
>> +		if (ret == -ENOENT)
>> +			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
>> +		else if (ret)
>> +			return;
>> +
>> +		search_start = search_low_max;
>> +	} else if (ret || !crash_size) {
>> +		/* Invalid argument value specified */
>>   		return;
>> +	}
> ...

Re: [PATCH -next v3 1/2] riscv: kdump: Implement crashkernel=X,[high,low]

[chenjiahao (C)]

On 2023/4/8 10:00, Leizhen (ThunderTown) wrote:
>
> On 2023/4/7 20:58, Leizhen (ThunderTown) wrote:
>>
>> On 2023/4/7 20:03, Simon Horman wrote:
>>> On Fri, Apr 07, 2023 at 06:02:05AM +0800, Chen Jiahao wrote:
>>>> On riscv, the current crash kernel allocation logic is trying to
>>>> allocate within 32bit addressible memory region by default, if
>>>> failed, try to allocate without 4G restriction.
>>>>
>>>> In need of saving DMA zone memory while allocating a relatively large
>>>> crash kernel region, allocating the reserved memory top down in
>>>> high memory, without overlapping the DMA zone, is a mature solution.
>>>> Here introduce the parameter option crashkernel=X,[high,low].
>>>>
>>>> One can reserve the crash kernel from high memory above DMA zone range
>>>> by explicitly passing "crashkernel=X,high"; or reserve a memory range
>>>> below 4G with "crashkernel=X,low".
>>>>
>>>> Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com>
>>> ...
>>>
>>>> @@ -1180,14 +1206,37 @@ static void __init reserve_crashkernel(void)
>>>>   		return;
>>>>   	}
>>>>   
>>>> -	ret = parse_crashkernel(boot_command_line, memblock_phys_mem_size(),
>>>> +	ret = parse_crashkernel(cmdline, memblock_phys_mem_size(),
>>>>   				&crash_size, &crash_base);
>>>> -	if (ret || !crash_size)
>>>> +	if (ret == -ENOENT) {
>>>> +		/*
>>>> +		 * crashkernel=X,[high,low] can be specified or not, but
>>>> +		 * invalid value is not allowed.
>>> nit: Perhaps something like this would be easier to correlate with the
>>>       code that follows:
>>>
>>> 		/* Fallback to crashkernel=X,[high,low] */
>> The description "crashkernel=X,[high,low] can be specified or not" is not
>> correct, because crashkernel=X,high must be specified when walking into this
>> branch. So use Simon's comments or copy arm64's comments(it's written for
>> parse_crashkernel_low()).
> I rethink it a little bit, if it's relative to crashkernel=X[@offset],
> that's also true.
>
> Reviewed-by: Zhen Lei <thunder.leizhen@huawei.com>

Sure, The commit should not be ambiguous like this, Simon's comment above is

a better option.


>>>
>>>> +		 */
>>>> +		ret = parse_crashkernel_high(cmdline, 0, &crash_size, &crash_base);
>>>> +		if (ret || !crash_size)
>>>> +			return;
>>>> +
>>>> +		/*
>>>> +		 * crashkernel=Y,low is valid only when crashkernel=X,high
>>>> +		 * is passed and high memory is reserved successful.
>>> nit: s/successful/successfully/
>> Seems like the whole "and high memory is reserved successful" needs to be deleted.
>> Only the dependency between the two boot options should be described here,
>> regardless of whether their memory is successfully allocated.

The comment here is imprecise, since there is absolutely no check whether

the allocation is successful before "parse_crashkernel_low"


>>
>>>> +		 */
>>>> +		ret = parse_crashkernel_low(cmdline, 0, &crash_low_size, &crash_base);
>>>> +		if (ret == -ENOENT)
>>>> +			crash_low_size = DEFAULT_CRASH_KERNEL_LOW_SIZE;
>>>> +		else if (ret)
>>>> +			return;
>>>> +
>>>> +		search_start = search_low_max;
>>>> +	} else if (ret || !crash_size) {
>>>> +		/* Invalid argument value specified */
>>>>   		return;
>>>> +	}
>>> ...
>>> .
>>>
BR,

Jiahao

Re: [PATCH -next v3 0/2] support allocating crashkernel above 4G explicitly on riscv

[chenjiahao (C)]

On 2023/4/7 20:05, Simon Horman wrote:
> On Fri, Apr 07, 2023 at 06:02:04AM +0800, Chen Jiahao wrote:
>> On riscv, the current crash kernel allocation logic is trying to
>> allocate within 32bit addressible memory region by default, if
>> failed, try to allocate without 4G restriction.
>>
>> In need of saving DMA zone memory while allocating a relatively large
>> crash kernel region, allocating the reserved memory top down in
>> high memory, without overlapping the DMA zone, is a mature solution.
>> Hence this patchset introduces the parameter option crashkernel=X,[high,low].
>>
>> One can reserve the crash kernel from high memory above DMA zone range
>> by explicitly passing "crashkernel=X,high"; or reserve a memory range
>> below 4G with "crashkernel=X,low". Besides, there are few rules need
>> to take notice:
>> 1. "crashkernel=X,[high,low]" will be ignored if "crashkernel=size"
>>     is specified.
>> 2. "crashkernel=X,low" is valid only when "crashkernel=X,high" is passed
>>     and there is enough memory to be allocated under 4G.
>> 3. When allocating crashkernel above 4G and no "crashkernel=X,low" is
>>     specified, a 128M low memory will be allocated automatically for
>>     swiotlb bounce buffer.
>> See Documentation/admin-guide/kernel-parameters.txt for more information.
>>
>> To verify loading the crashkernel, adapted kexec-tools is attached below:
>> https://github.com/chenjh005/kexec-tools/tree/build-test-riscv-v2
> Are you planning to submit the kexec-tools patches too?

Yes, but current adaptation is a little informal. I will submit them 
later after further tests.


BR,

Jiahao

Re: [PATCH -next v3 1/2] riscv: kdump: Implement crashkernel=X,[high,low]

[Guo Ren]

On Fri, Apr 7, 2023 at 8:03 PM Simon Horman <horms@kernel.org> wrote:
>
> On Fri, Apr 07, 2023 at 05:06:24PM +0800, Guo Ren wrote:
> > On Thu, Apr 6, 2023 at 10:06 PM Chen Jiahao <chenjiahao16@huawei.com> wrote:
> > >
> > > On riscv, the current crash kernel allocation logic is trying to
> > > allocate within 32bit addressible memory region by default, if
> > > failed, try to allocate without 4G restriction.
> > >
> > > In need of saving DMA zone memory while allocating a relatively large
> > > crash kernel region, allocating the reserved memory top down in
> > > high memory, without overlapping the DMA zone, is a mature solution.
> > > Here introduce the parameter option crashkernel=X,[high,low].
> > >
> > > One can reserve the crash kernel from high memory above DMA zone range
> > > by explicitly passing "crashkernel=X,high"; or reserve a memory range
> > > below 4G with "crashkernel=X,low".
> > Asked-by: Guo Ren <guoren@kernel.org>
>
> Perhaps 'Acked-by' :)
Sorry, my typo.

Acked-by: Guo Ren <guoren@kernel.org>



-- 
Best Regards
 Guo Ren