* [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
@ 2020-07-21 7:32 ` Wei Li
0 siblings, 0 replies; 26+ messages in thread
From: Wei Li @ 2020-07-21 7:32 UTC (permalink / raw)
To: catalin.marinas, will
Cc: liwei213, saberlily.xia, puck.chen, butao, fengbaopeng2,
nsaenzjulienne, steve.capper, rppt, song.bao.hua,
linux-arm-kernel, linux-kernel, sujunfei2
For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
do not free the reserved memory for the page map, this patch do it.
Signed-off-by: Wei Li <liwei213@huawei.com>
Signed-off-by: Chen Feng <puck.chen@hisilicon.com>
Signed-off-by: Xia Qing <saberlily.xia@hisilicon.com>
---
arch/arm64/mm/init.c | 81 +++++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 71 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 1e93cfc7c47a..d1b56b47d5ba 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -441,7 +441,48 @@ void __init bootmem_init(void)
memblock_dump_all();
}
-#ifndef CONFIG_SPARSEMEM_VMEMMAP
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+#define VMEMMAP_PAGE_INUSE 0xFD
+static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
+{
+ unsigned long addr, end;
+ unsigned long next;
+ pmd_t *pmd;
+ void *page_addr;
+ phys_addr_t phys_addr;
+
+ addr = (unsigned long)pfn_to_page(start_pfn);
+ end = (unsigned long)pfn_to_page(end_pfn);
+
+ pmd = pmd_offset(pud_offset(pgd_offset_k(addr), addr), addr);
+ for (; addr < end; addr = next, pmd++) {
+ next = pmd_addr_end(addr, end);
+
+ if (!pmd_present(*pmd))
+ continue;
+
+ if (IS_ALIGNED(addr, PMD_SIZE) &&
+ IS_ALIGNED(next, PMD_SIZE)) {
+ phys_addr = __pfn_to_phys(pmd_pfn(*pmd));
+ free_bootmem(phys_addr, PMD_SIZE);
+ pmd_clear(pmd);
+ } else {
+ /* If here, we are freeing vmemmap pages. */
+ memset((void *)addr, VMEMMAP_PAGE_INUSE, next - addr);
+ page_addr = page_address(pmd_page(*pmd));
+
+ if (!memchr_inv(page_addr, VMEMMAP_PAGE_INUSE,
+ PMD_SIZE)) {
+ phys_addr = __pfn_to_phys(pmd_pfn(*pmd));
+ free_bootmem(phys_addr, PMD_SIZE);
+ pmd_clear(pmd);
+ }
+ }
+ }
+
+ flush_tlb_all();
+}
+#else
static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
{
struct page *start_pg, *end_pg;
@@ -468,31 +509,53 @@ static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
memblock_free(pg, pgend - pg);
}
+#endif
+
/*
* The mem_map array can get very big. Free the unused area of the memory map.
*/
static void __init free_unused_memmap(void)
{
- unsigned long start, prev_end = 0;
+ unsigned long start, cur_start, prev_end = 0;
struct memblock_region *reg;
for_each_memblock(memory, reg) {
- start = __phys_to_pfn(reg->base);
+ cur_start = __phys_to_pfn(reg->base);
#ifdef CONFIG_SPARSEMEM
/*
* Take care not to free memmap entries that don't exist due
* to SPARSEMEM sections which aren't present.
*/
- start = min(start, ALIGN(prev_end, PAGES_PER_SECTION));
-#endif
+ start = min(cur_start, ALIGN(prev_end, PAGES_PER_SECTION));
+
/*
- * If we had a previous bank, and there is a space between the
- * current bank and the previous, free it.
+ * Free memory in the case of:
+ * 1. if cur_start - prev_end <= PAGES_PER_SECTION,
+ * free pre_end ~ cur_start.
+ * 2. if cur_start - prev_end > PAGES_PER_SECTION,
+ * free pre_end ~ ALIGN(prev_end, PAGES_PER_SECTION).
*/
if (prev_end && prev_end < start)
free_memmap(prev_end, start);
+ /*
+ * Free memory in the case of:
+ * if cur_start - prev_end > PAGES_PER_SECTION,
+ * free ALIGN_DOWN(cur_start, PAGES_PER_SECTION) ~ cur_start.
+ */
+ if (cur_start > start &&
+ !IS_ALIGNED(cur_start, PAGES_PER_SECTION))
+ free_memmap(ALIGN_DOWN(cur_start, PAGES_PER_SECTION),
+ cur_start);
+#else
+ /*
+ * If we had a previous bank, and there is a space between the
+ * current bank and the previous, free it.
+ */
+ if (prev_end && prev_end < cur_start)
+ free_memmap(prev_end, cur_start);
+#endif
/*
* Align up here since the VM subsystem insists that the
* memmap entries are valid from the bank end aligned to
@@ -507,7 +570,6 @@ static void __init free_unused_memmap(void)
free_memmap(prev_end, ALIGN(prev_end, PAGES_PER_SECTION));
#endif
}
-#endif /* !CONFIG_SPARSEMEM_VMEMMAP */
/*
* mem_init() marks the free areas in the mem_map and tells us how much memory
@@ -524,9 +586,8 @@ void __init mem_init(void)
set_max_mapnr(max_pfn - PHYS_PFN_OFFSET);
-#ifndef CONFIG_SPARSEMEM_VMEMMAP
free_unused_memmap();
-#endif
+
/* this will put all unused low memory onto the freelists */
memblock_free_all();
--
2.15.0
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-21 7:32 ` Wei Li
(?)
@ 2020-07-21 16:29 ` kernel test robot
2020-07-22 1:58 ` 答复: " liwei
-1 siblings, 1 reply; 26+ messages in thread
From: kernel test robot @ 2020-07-21 16:29 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 4569 bytes --]
Hi Wei,
I love your patch! Yet something to improve:
[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on v5.8-rc6 next-20200721]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Wei-Li/arm64-mm-free-unused-memmap-for-sparse-memory-model-that-define-VMEMMAP/20200721-153408
base: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
config: arm64-allyesconfig (attached as .config)
compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
In file included from arch/arm64/include/asm/kernel-pgtable.h:11,
from arch/arm64/include/asm/uaccess.h:11,
from include/linux/uaccess.h:11,
from include/linux/sched/task.h:11,
from include/linux/sched/signal.h:9,
from include/linux/rcuwait.h:6,
from include/linux/percpu-rwsem.h:7,
from include/linux/fs.h:33,
from include/linux/cgroup.h:17,
from include/linux/memcontrol.h:13,
from include/linux/swap.h:9,
from arch/arm64/mm/init.c:12:
arch/arm64/mm/init.c: In function 'free_memmap':
>> include/linux/pgtable.h:113:34: error: passing argument 1 of 'pud_offset' from incompatible pointer type [-Werror=incompatible-pointer-types]
113 | #define pgd_offset(mm, address) pgd_offset_pgd((mm)->pgd, (address))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| pgd_t * {aka struct <anonymous> *}
include/linux/pgtable.h:120:32: note: in expansion of macro 'pgd_offset'
120 | #define pgd_offset_k(address) pgd_offset(&init_mm, (address))
| ^~~~~~~~~~
arch/arm64/mm/init.c:457:30: note: in expansion of macro 'pgd_offset_k'
457 | pmd = pmd_offset(pud_offset(pgd_offset_k(addr), addr), addr);
| ^~~~~~~~~~~~
In file included from arch/arm64/include/asm/pgtable-types.h:50,
from arch/arm64/include/asm/kasan.h:9,
from arch/arm64/include/asm/processor.h:36,
from include/asm-generic/qrwlock.h:14,
from ./arch/arm64/include/generated/asm/qrwlock.h:1,
from arch/arm64/include/asm/spinlock.h:8,
from include/linux/spinlock.h:89,
from include/linux/swap.h:5,
from arch/arm64/mm/init.c:12:
include/asm-generic/pgtable-nopud.h:42:40: note: expected 'p4d_t *' {aka 'struct <anonymous> *'} but argument is of type 'pgd_t *' {aka 'struct <anonymous> *'}
42 | static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address)
| ~~~~~~~^~~
>> arch/arm64/mm/init.c:467:4: error: implicit declaration of function 'free_bootmem'; did you mean 'free_initmem'? [-Werror=implicit-function-declaration]
467 | free_bootmem(phys_addr, PMD_SIZE);
| ^~~~~~~~~~~~
| free_initmem
cc1: some warnings being treated as errors
vim +/pud_offset +113 include/linux/pgtable.h
974b9b2c68f3d3 Mike Rapoport 2020-06-08 108
974b9b2c68f3d3 Mike Rapoport 2020-06-08 109 /*
974b9b2c68f3d3 Mike Rapoport 2020-06-08 110 * a shortcut to get a pgd_t in a given mm
974b9b2c68f3d3 Mike Rapoport 2020-06-08 111 */
974b9b2c68f3d3 Mike Rapoport 2020-06-08 112 #ifndef pgd_offset
974b9b2c68f3d3 Mike Rapoport 2020-06-08 @113 #define pgd_offset(mm, address) pgd_offset_pgd((mm)->pgd, (address))
974b9b2c68f3d3 Mike Rapoport 2020-06-08 114 #endif
974b9b2c68f3d3 Mike Rapoport 2020-06-08 115
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 73584 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread* 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-21 16:29 ` kernel test robot
@ 2020-07-22 1:58 ` liwei
0 siblings, 0 replies; 26+ messages in thread
From: liwei @ 2020-07-22 1:58 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 5323 bytes --]
Hi,
Thank you for your attention to this patch, I will accept your suggestion and fix the issue as soon as possible.
Thanks!
-----邮件原件-----
发件人: kernel test robot [mailto:lkp(a)intel.com]
发送时间: 2020年7月22日 0:29
收件人: liwei (CM) <liwei213@huawei.com>; catalin.marinas(a)arm.com; will(a)kernel.org
抄送: kbuild-all(a)lists.01.org; liwei (CM) <liwei213@huawei.com>; Xiaqing (A) <saberlily.xia@hisilicon.com>; Chenfeng (puck) <puck.chen@hisilicon.com>; butao <butao@hisilicon.com>; fengbaopeng <fengbaopeng2@hisilicon.com>; nsaenzjulienne(a)suse.de; steve.capper(a)arm.com; rppt(a)linux.ibm.com
主题: Re: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
Hi Wei,
I love your patch! Yet something to improve:
[auto build test ERROR on arm64/for-next/core] [also build test ERROR on v5.8-rc6 next-20200721] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Wei-Li/arm64-mm-free-unused-memmap-for-sparse-memory-model-that-define-VMEMMAP/20200721-153408
base: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
config: arm64-allyesconfig (attached as .config)
compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
In file included from arch/arm64/include/asm/kernel-pgtable.h:11,
from arch/arm64/include/asm/uaccess.h:11,
from include/linux/uaccess.h:11,
from include/linux/sched/task.h:11,
from include/linux/sched/signal.h:9,
from include/linux/rcuwait.h:6,
from include/linux/percpu-rwsem.h:7,
from include/linux/fs.h:33,
from include/linux/cgroup.h:17,
from include/linux/memcontrol.h:13,
from include/linux/swap.h:9,
from arch/arm64/mm/init.c:12:
arch/arm64/mm/init.c: In function 'free_memmap':
>> include/linux/pgtable.h:113:34: error: passing argument 1 of
>> 'pud_offset' from incompatible pointer type
>> [-Werror=incompatible-pointer-types]
113 | #define pgd_offset(mm, address) pgd_offset_pgd((mm)->pgd, (address))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| pgd_t * {aka struct <anonymous> *}
include/linux/pgtable.h:120:32: note: in expansion of macro 'pgd_offset'
120 | #define pgd_offset_k(address) pgd_offset(&init_mm, (address))
| ^~~~~~~~~~
arch/arm64/mm/init.c:457:30: note: in expansion of macro 'pgd_offset_k'
457 | pmd = pmd_offset(pud_offset(pgd_offset_k(addr), addr), addr);
| ^~~~~~~~~~~~
In file included from arch/arm64/include/asm/pgtable-types.h:50,
from arch/arm64/include/asm/kasan.h:9,
from arch/arm64/include/asm/processor.h:36,
from include/asm-generic/qrwlock.h:14,
from ./arch/arm64/include/generated/asm/qrwlock.h:1,
from arch/arm64/include/asm/spinlock.h:8,
from include/linux/spinlock.h:89,
from include/linux/swap.h:5,
from arch/arm64/mm/init.c:12:
include/asm-generic/pgtable-nopud.h:42:40: note: expected 'p4d_t *' {aka 'struct <anonymous> *'} but argument is of type 'pgd_t *' {aka 'struct <anonymous> *'}
42 | static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address)
| ~~~~~~~^~~
>> arch/arm64/mm/init.c:467:4: error: implicit declaration of function
>> 'free_bootmem'; did you mean 'free_initmem'?
>> [-Werror=implicit-function-declaration]
467 | free_bootmem(phys_addr, PMD_SIZE);
| ^~~~~~~~~~~~
| free_initmem
cc1: some warnings being treated as errors
vim +/pud_offset +113 include/linux/pgtable.h
974b9b2c68f3d3 Mike Rapoport 2020-06-08 108
974b9b2c68f3d3 Mike Rapoport 2020-06-08 109 /*
974b9b2c68f3d3 Mike Rapoport 2020-06-08 110 * a shortcut to get a pgd_t in a given mm
974b9b2c68f3d3 Mike Rapoport 2020-06-08 111 */
974b9b2c68f3d3 Mike Rapoport 2020-06-08 112 #ifndef pgd_offset
974b9b2c68f3d3 Mike Rapoport 2020-06-08 @113 #define pgd_offset(mm, address) pgd_offset_pgd((mm)->pgd, (address))
974b9b2c68f3d3 Mike Rapoport 2020-06-08 114 #endif
974b9b2c68f3d3 Mike Rapoport 2020-06-08 115
---
0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-21 7:32 ` Wei Li
@ 2020-07-22 6:07 ` Mike Rapoport
-1 siblings, 0 replies; 26+ messages in thread
From: Mike Rapoport @ 2020-07-22 6:07 UTC (permalink / raw)
To: Wei Li
Cc: song.bao.hua, linux-arm-kernel, steve.capper, catalin.marinas,
sujunfei2, linux-kernel, fengbaopeng2, saberlily.xia, puck.chen,
will, nsaenzjulienne, butao
Hi,
On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> do not free the reserved memory for the page map, this patch do it.
Are there numbers showing how much memory is actually freed?
The freeing of empty memmap would become rather complex with these
changes, do the memory savings justify it?
> Signed-off-by: Wei Li <liwei213@huawei.com>
> Signed-off-by: Chen Feng <puck.chen@hisilicon.com>
> Signed-off-by: Xia Qing <saberlily.xia@hisilicon.com>
> ---
> arch/arm64/mm/init.c | 81 +++++++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 71 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 1e93cfc7c47a..d1b56b47d5ba 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -441,7 +441,48 @@ void __init bootmem_init(void)
> memblock_dump_all();
> }
>
> -#ifndef CONFIG_SPARSEMEM_VMEMMAP
> +#ifdef CONFIG_SPARSEMEM_VMEMMAP
> +#define VMEMMAP_PAGE_INUSE 0xFD
> +static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
> +{
> + unsigned long addr, end;
> + unsigned long next;
> + pmd_t *pmd;
> + void *page_addr;
> + phys_addr_t phys_addr;
> +
> + addr = (unsigned long)pfn_to_page(start_pfn);
> + end = (unsigned long)pfn_to_page(end_pfn);
> +
> + pmd = pmd_offset(pud_offset(pgd_offset_k(addr), addr), addr);
> + for (; addr < end; addr = next, pmd++) {
> + next = pmd_addr_end(addr, end);
> +
> + if (!pmd_present(*pmd))
> + continue;
> +
> + if (IS_ALIGNED(addr, PMD_SIZE) &&
> + IS_ALIGNED(next, PMD_SIZE)) {
> + phys_addr = __pfn_to_phys(pmd_pfn(*pmd));
> + free_bootmem(phys_addr, PMD_SIZE);
> + pmd_clear(pmd);
> + } else {
> + /* If here, we are freeing vmemmap pages. */
> + memset((void *)addr, VMEMMAP_PAGE_INUSE, next - addr);
> + page_addr = page_address(pmd_page(*pmd));
> +
> + if (!memchr_inv(page_addr, VMEMMAP_PAGE_INUSE,
> + PMD_SIZE)) {
> + phys_addr = __pfn_to_phys(pmd_pfn(*pmd));
> + free_bootmem(phys_addr, PMD_SIZE);
> + pmd_clear(pmd);
> + }
> + }
> + }
> +
> + flush_tlb_all();
> +}
> +#else
> static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
> {
> struct page *start_pg, *end_pg;
> @@ -468,31 +509,53 @@ static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
> memblock_free(pg, pgend - pg);
> }
>
> +#endif
> +
> /*
> * The mem_map array can get very big. Free the unused area of the memory map.
> */
> static void __init free_unused_memmap(void)
> {
> - unsigned long start, prev_end = 0;
> + unsigned long start, cur_start, prev_end = 0;
> struct memblock_region *reg;
>
> for_each_memblock(memory, reg) {
> - start = __phys_to_pfn(reg->base);
> + cur_start = __phys_to_pfn(reg->base);
>
> #ifdef CONFIG_SPARSEMEM
> /*
> * Take care not to free memmap entries that don't exist due
> * to SPARSEMEM sections which aren't present.
> */
> - start = min(start, ALIGN(prev_end, PAGES_PER_SECTION));
> -#endif
> + start = min(cur_start, ALIGN(prev_end, PAGES_PER_SECTION));
> +
> /*
> - * If we had a previous bank, and there is a space between the
> - * current bank and the previous, free it.
> + * Free memory in the case of:
> + * 1. if cur_start - prev_end <= PAGES_PER_SECTION,
> + * free pre_end ~ cur_start.
> + * 2. if cur_start - prev_end > PAGES_PER_SECTION,
> + * free pre_end ~ ALIGN(prev_end, PAGES_PER_SECTION).
> */
> if (prev_end && prev_end < start)
> free_memmap(prev_end, start);
>
> + /*
> + * Free memory in the case of:
> + * if cur_start - prev_end > PAGES_PER_SECTION,
> + * free ALIGN_DOWN(cur_start, PAGES_PER_SECTION) ~ cur_start.
> + */
> + if (cur_start > start &&
> + !IS_ALIGNED(cur_start, PAGES_PER_SECTION))
> + free_memmap(ALIGN_DOWN(cur_start, PAGES_PER_SECTION),
> + cur_start);
> +#else
> + /*
> + * If we had a previous bank, and there is a space between the
> + * current bank and the previous, free it.
> + */
> + if (prev_end && prev_end < cur_start)
> + free_memmap(prev_end, cur_start);
> +#endif
> /*
> * Align up here since the VM subsystem insists that the
> * memmap entries are valid from the bank end aligned to
> @@ -507,7 +570,6 @@ static void __init free_unused_memmap(void)
> free_memmap(prev_end, ALIGN(prev_end, PAGES_PER_SECTION));
> #endif
> }
> -#endif /* !CONFIG_SPARSEMEM_VMEMMAP */
>
> /*
> * mem_init() marks the free areas in the mem_map and tells us how much memory
> @@ -524,9 +586,8 @@ void __init mem_init(void)
>
> set_max_mapnr(max_pfn - PHYS_PFN_OFFSET);
>
> -#ifndef CONFIG_SPARSEMEM_VMEMMAP
> free_unused_memmap();
> -#endif
> +
> /* this will put all unused low memory onto the freelists */
> memblock_free_all();
>
> --
> 2.15.0
>
--
Sincerely yours,
Mike.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
@ 2020-07-22 6:07 ` Mike Rapoport
0 siblings, 0 replies; 26+ messages in thread
From: Mike Rapoport @ 2020-07-22 6:07 UTC (permalink / raw)
To: Wei Li
Cc: catalin.marinas, will, saberlily.xia, puck.chen, butao,
fengbaopeng2, nsaenzjulienne, steve.capper, song.bao.hua,
linux-arm-kernel, linux-kernel, sujunfei2
Hi,
On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> do not free the reserved memory for the page map, this patch do it.
Are there numbers showing how much memory is actually freed?
The freeing of empty memmap would become rather complex with these
changes, do the memory savings justify it?
> Signed-off-by: Wei Li <liwei213@huawei.com>
> Signed-off-by: Chen Feng <puck.chen@hisilicon.com>
> Signed-off-by: Xia Qing <saberlily.xia@hisilicon.com>
> ---
> arch/arm64/mm/init.c | 81 +++++++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 71 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 1e93cfc7c47a..d1b56b47d5ba 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -441,7 +441,48 @@ void __init bootmem_init(void)
> memblock_dump_all();
> }
>
> -#ifndef CONFIG_SPARSEMEM_VMEMMAP
> +#ifdef CONFIG_SPARSEMEM_VMEMMAP
> +#define VMEMMAP_PAGE_INUSE 0xFD
> +static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
> +{
> + unsigned long addr, end;
> + unsigned long next;
> + pmd_t *pmd;
> + void *page_addr;
> + phys_addr_t phys_addr;
> +
> + addr = (unsigned long)pfn_to_page(start_pfn);
> + end = (unsigned long)pfn_to_page(end_pfn);
> +
> + pmd = pmd_offset(pud_offset(pgd_offset_k(addr), addr), addr);
> + for (; addr < end; addr = next, pmd++) {
> + next = pmd_addr_end(addr, end);
> +
> + if (!pmd_present(*pmd))
> + continue;
> +
> + if (IS_ALIGNED(addr, PMD_SIZE) &&
> + IS_ALIGNED(next, PMD_SIZE)) {
> + phys_addr = __pfn_to_phys(pmd_pfn(*pmd));
> + free_bootmem(phys_addr, PMD_SIZE);
> + pmd_clear(pmd);
> + } else {
> + /* If here, we are freeing vmemmap pages. */
> + memset((void *)addr, VMEMMAP_PAGE_INUSE, next - addr);
> + page_addr = page_address(pmd_page(*pmd));
> +
> + if (!memchr_inv(page_addr, VMEMMAP_PAGE_INUSE,
> + PMD_SIZE)) {
> + phys_addr = __pfn_to_phys(pmd_pfn(*pmd));
> + free_bootmem(phys_addr, PMD_SIZE);
> + pmd_clear(pmd);
> + }
> + }
> + }
> +
> + flush_tlb_all();
> +}
> +#else
> static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
> {
> struct page *start_pg, *end_pg;
> @@ -468,31 +509,53 @@ static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
> memblock_free(pg, pgend - pg);
> }
>
> +#endif
> +
> /*
> * The mem_map array can get very big. Free the unused area of the memory map.
> */
> static void __init free_unused_memmap(void)
> {
> - unsigned long start, prev_end = 0;
> + unsigned long start, cur_start, prev_end = 0;
> struct memblock_region *reg;
>
> for_each_memblock(memory, reg) {
> - start = __phys_to_pfn(reg->base);
> + cur_start = __phys_to_pfn(reg->base);
>
> #ifdef CONFIG_SPARSEMEM
> /*
> * Take care not to free memmap entries that don't exist due
> * to SPARSEMEM sections which aren't present.
> */
> - start = min(start, ALIGN(prev_end, PAGES_PER_SECTION));
> -#endif
> + start = min(cur_start, ALIGN(prev_end, PAGES_PER_SECTION));
> +
> /*
> - * If we had a previous bank, and there is a space between the
> - * current bank and the previous, free it.
> + * Free memory in the case of:
> + * 1. if cur_start - prev_end <= PAGES_PER_SECTION,
> + * free pre_end ~ cur_start.
> + * 2. if cur_start - prev_end > PAGES_PER_SECTION,
> + * free pre_end ~ ALIGN(prev_end, PAGES_PER_SECTION).
> */
> if (prev_end && prev_end < start)
> free_memmap(prev_end, start);
>
> + /*
> + * Free memory in the case of:
> + * if cur_start - prev_end > PAGES_PER_SECTION,
> + * free ALIGN_DOWN(cur_start, PAGES_PER_SECTION) ~ cur_start.
> + */
> + if (cur_start > start &&
> + !IS_ALIGNED(cur_start, PAGES_PER_SECTION))
> + free_memmap(ALIGN_DOWN(cur_start, PAGES_PER_SECTION),
> + cur_start);
> +#else
> + /*
> + * If we had a previous bank, and there is a space between the
> + * current bank and the previous, free it.
> + */
> + if (prev_end && prev_end < cur_start)
> + free_memmap(prev_end, cur_start);
> +#endif
> /*
> * Align up here since the VM subsystem insists that the
> * memmap entries are valid from the bank end aligned to
> @@ -507,7 +570,6 @@ static void __init free_unused_memmap(void)
> free_memmap(prev_end, ALIGN(prev_end, PAGES_PER_SECTION));
> #endif
> }
> -#endif /* !CONFIG_SPARSEMEM_VMEMMAP */
>
> /*
> * mem_init() marks the free areas in the mem_map and tells us how much memory
> @@ -524,9 +586,8 @@ void __init mem_init(void)
>
> set_max_mapnr(max_pfn - PHYS_PFN_OFFSET);
>
> -#ifndef CONFIG_SPARSEMEM_VMEMMAP
> free_unused_memmap();
> -#endif
> +
> /* this will put all unused low memory onto the freelists */
> memblock_free_all();
>
> --
> 2.15.0
>
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 26+ messages in thread* 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-22 6:07 ` Mike Rapoport
@ 2020-07-22 8:41 ` liwei (CM)
-1 siblings, 0 replies; 26+ messages in thread
From: liwei (CM) @ 2020-07-22 8:41 UTC (permalink / raw)
To: Mike Rapoport
Cc: Song Bao Hua (Barry Song), linux-arm-kernel@lists.infradead.org,
steve.capper@arm.com, catalin.marinas@arm.com, sujunfei,
linux-kernel@vger.kernel.org, fengbaopeng, zhaojiapeng,
Xiaqing (A), Chenfeng (puck), will@kernel.org,
nsaenzjulienne@suse.de, butao
-----邮件原件-----
发件人: Mike Rapoport [mailto:rppt@linux.ibm.com]
发送时间: 2020年7月22日 14:07
收件人: liwei (CM) <liwei213@huawei.com>
抄送: catalin.marinas@arm.com; will@kernel.org; Xiaqing (A) <saberlily.xia@hisilicon.com>; Chenfeng (puck) <puck.chen@hisilicon.com>; butao <butao@hisilicon.com>; fengbaopeng <fengbaopeng2@hisilicon.com>; nsaenzjulienne@suse.de; steve.capper@arm.com; Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>; linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; sujunfei <sujunfei2@hisilicon.com>
主题: Re: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
Hi,
On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> do not free the reserved memory for the page map, this patch do it.
Are there numbers showing how much memory is actually freed?
The freeing of empty memmap would become rather complex with these changes, do the memory savings justify it?
Hi, Mike
In the sparse memory model, the size of a section is 1 GB (SECTION_SIZE_BITS 30) by default. Therefore, when the memory is less than the size of a section, that is, when there is a hole, the patch takes effect：
1) For example, the DDR size used by our platform is 8 GB, however, 3.5 ~ 4 GB is the space where the SOC register is located. Therefore, if the page map is performed on memory3 (3 ~4 GB), 512 MB/4 KB x 64 bytes = 8 MB is wasted because the page map is not required for the 3.5~4 GB space; However, the DDR memory space of 3.5~4 GB is shift to 16~16.5 GB. In this case, memory is wasted because the page map is not required for 16.5~17 GB, the patch can also save 8 MB. Therefore, the total saved memory is 16 MB.
2) In the reserved memory, some modules need to reserve a large amount of memory (no-map attr). On our platform, the modem module needs to reserve more than 256 MB memory, and the patch can save 4 MB. Actually, if the reserved memory is greater than 128 MB, the patch can free unnecessary page map memory.
It may be possible to save some waste by reducing the section size, but free the waste page map that defines VMEMMAP is another approach similar to flat memory model and sparse memory mode that does not define VMEMMAP, and makes the entire code look more complete.
If you have a better idea, I'd be happy to discuss it with you.
Thanks!
> Signed-off-by: Wei Li <liwei213@huawei.com>
> Signed-off-by: Chen Feng <puck.chen@hisilicon.com>
> Signed-off-by: Xia Qing <saberlily.xia@hisilicon.com>
> ---
> arch/arm64/mm/init.c | 81
> +++++++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 71 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index
> 1e93cfc7c47a..d1b56b47d5ba 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -441,7 +441,48 @@ void __init bootmem_init(void)
> memblock_dump_all();
> }
>
> -#ifndef CONFIG_SPARSEMEM_VMEMMAP
> +#ifdef CONFIG_SPARSEMEM_VMEMMAP
> +#define VMEMMAP_PAGE_INUSE 0xFD
> +static inline void free_memmap(unsigned long start_pfn, unsigned long
> +end_pfn) {
> + unsigned long addr, end;
> + unsigned long next;
> + pmd_t *pmd;
> + void *page_addr;
> + phys_addr_t phys_addr;
> +
> + addr = (unsigned long)pfn_to_page(start_pfn);
> + end = (unsigned long)pfn_to_page(end_pfn);
> +
> + pmd = pmd_offset(pud_offset(pgd_offset_k(addr), addr), addr);
> + for (; addr < end; addr = next, pmd++) {
> + next = pmd_addr_end(addr, end);
> +
> + if (!pmd_present(*pmd))
> + continue;
> +
> + if (IS_ALIGNED(addr, PMD_SIZE) &&
> + IS_ALIGNED(next, PMD_SIZE)) {
> + phys_addr = __pfn_to_phys(pmd_pfn(*pmd));
> + free_bootmem(phys_addr, PMD_SIZE);
> + pmd_clear(pmd);
> + } else {
> + /* If here, we are freeing vmemmap pages. */
> + memset((void *)addr, VMEMMAP_PAGE_INUSE, next - addr);
> + page_addr = page_address(pmd_page(*pmd));
> +
> + if (!memchr_inv(page_addr, VMEMMAP_PAGE_INUSE,
> + PMD_SIZE)) {
> + phys_addr = __pfn_to_phys(pmd_pfn(*pmd));
> + free_bootmem(phys_addr, PMD_SIZE);
> + pmd_clear(pmd);
> + }
> + }
> + }
> +
> + flush_tlb_all();
> +}
> +#else
> static inline void free_memmap(unsigned long start_pfn, unsigned long
> end_pfn) {
> struct page *start_pg, *end_pg;
> @@ -468,31 +509,53 @@ static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
> memblock_free(pg, pgend - pg);
> }
>
> +#endif
> +
> /*
> * The mem_map array can get very big. Free the unused area of the memory map.
> */
> static void __init free_unused_memmap(void) {
> - unsigned long start, prev_end = 0;
> + unsigned long start, cur_start, prev_end = 0;
> struct memblock_region *reg;
>
> for_each_memblock(memory, reg) {
> - start = __phys_to_pfn(reg->base);
> + cur_start = __phys_to_pfn(reg->base);
>
> #ifdef CONFIG_SPARSEMEM
> /*
> * Take care not to free memmap entries that don't exist due
> * to SPARSEMEM sections which aren't present.
> */
> - start = min(start, ALIGN(prev_end, PAGES_PER_SECTION));
> -#endif
> + start = min(cur_start, ALIGN(prev_end, PAGES_PER_SECTION));
> +
> /*
> - * If we had a previous bank, and there is a space between the
> - * current bank and the previous, free it.
> + * Free memory in the case of:
> + * 1. if cur_start - prev_end <= PAGES_PER_SECTION,
> + * free pre_end ~ cur_start.
> + * 2. if cur_start - prev_end > PAGES_PER_SECTION,
> + * free pre_end ~ ALIGN(prev_end, PAGES_PER_SECTION).
> */
> if (prev_end && prev_end < start)
> free_memmap(prev_end, start);
>
> + /*
> + * Free memory in the case of:
> + * if cur_start - prev_end > PAGES_PER_SECTION,
> + * free ALIGN_DOWN(cur_start, PAGES_PER_SECTION) ~ cur_start.
> + */
> + if (cur_start > start &&
> + !IS_ALIGNED(cur_start, PAGES_PER_SECTION))
> + free_memmap(ALIGN_DOWN(cur_start, PAGES_PER_SECTION),
> + cur_start);
> +#else
> + /*
> + * If we had a previous bank, and there is a space between the
> + * current bank and the previous, free it.
> + */
> + if (prev_end && prev_end < cur_start)
> + free_memmap(prev_end, cur_start);
> +#endif
> /*
> * Align up here since the VM subsystem insists that the
> * memmap entries are valid from the bank end aligned to @@ -507,7
> +570,6 @@ static void __init free_unused_memmap(void)
> free_memmap(prev_end, ALIGN(prev_end, PAGES_PER_SECTION)); #endif
> }
> -#endif /* !CONFIG_SPARSEMEM_VMEMMAP */
>
> /*
> * mem_init() marks the free areas in the mem_map and tells us how
> much memory @@ -524,9 +586,8 @@ void __init mem_init(void)
>
> set_max_mapnr(max_pfn - PHYS_PFN_OFFSET);
>
> -#ifndef CONFIG_SPARSEMEM_VMEMMAP
> free_unused_memmap();
> -#endif
> +
> /* this will put all unused low memory onto the freelists */
> memblock_free_all();
>
> --
> 2.15.0
>
--
Sincerely yours,
Mike.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread* 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
@ 2020-07-22 8:41 ` liwei (CM)
0 siblings, 0 replies; 26+ messages in thread
From: liwei (CM) @ 2020-07-22 8:41 UTC (permalink / raw)
To: Mike Rapoport
Cc: catalin.marinas@arm.com, will@kernel.org, Xiaqing (A),
Chenfeng (puck), butao, fengbaopeng, nsaenzjulienne@suse.de,
steve.capper@arm.com, Song Bao Hua (Barry Song),
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, sujunfei, zhaojiapeng
-----邮件原件-----
发件人: Mike Rapoport [mailto:rppt@linux.ibm.com]
发送时间: 2020年7月22日 14:07
收件人: liwei (CM) <liwei213@huawei.com>
抄送: catalin.marinas@arm.com; will@kernel.org; Xiaqing (A) <saberlily.xia@hisilicon.com>; Chenfeng (puck) <puck.chen@hisilicon.com>; butao <butao@hisilicon.com>; fengbaopeng <fengbaopeng2@hisilicon.com>; nsaenzjulienne@suse.de; steve.capper@arm.com; Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>; linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; sujunfei <sujunfei2@hisilicon.com>
主题: Re: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
Hi,
On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> do not free the reserved memory for the page map, this patch do it.
Are there numbers showing how much memory is actually freed?
The freeing of empty memmap would become rather complex with these changes, do the memory savings justify it?
Hi, Mike
In the sparse memory model, the size of a section is 1 GB (SECTION_SIZE_BITS 30) by default. Therefore, when the memory is less than the size of a section, that is, when there is a hole, the patch takes effect：
1) For example, the DDR size used by our platform is 8 GB, however, 3.5 ~ 4 GB is the space where the SOC register is located. Therefore, if the page map is performed on memory3 (3 ~4 GB), 512 MB/4 KB x 64 bytes = 8 MB is wasted because the page map is not required for the 3.5~4 GB space; However, the DDR memory space of 3.5~4 GB is shift to 16~16.5 GB. In this case, memory is wasted because the page map is not required for 16.5~17 GB, the patch can also save 8 MB. Therefore, the total saved memory is 16 MB.
2) In the reserved memory, some modules need to reserve a large amount of memory (no-map attr). On our platform, the modem module needs to reserve more than 256 MB memory, and the patch can save 4 MB. Actually, if the reserved memory is greater than 128 MB, the patch can free unnecessary page map memory.
It may be possible to save some waste by reducing the section size, but free the waste page map that defines VMEMMAP is another approach similar to flat memory model and sparse memory mode that does not define VMEMMAP, and makes the entire code look more complete.
If you have a better idea, I'd be happy to discuss it with you.
Thanks!
> Signed-off-by: Wei Li <liwei213@huawei.com>
> Signed-off-by: Chen Feng <puck.chen@hisilicon.com>
> Signed-off-by: Xia Qing <saberlily.xia@hisilicon.com>
> ---
> arch/arm64/mm/init.c | 81
> +++++++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 71 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index
> 1e93cfc7c47a..d1b56b47d5ba 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -441,7 +441,48 @@ void __init bootmem_init(void)
> memblock_dump_all();
> }
>
> -#ifndef CONFIG_SPARSEMEM_VMEMMAP
> +#ifdef CONFIG_SPARSEMEM_VMEMMAP
> +#define VMEMMAP_PAGE_INUSE 0xFD
> +static inline void free_memmap(unsigned long start_pfn, unsigned long
> +end_pfn) {
> + unsigned long addr, end;
> + unsigned long next;
> + pmd_t *pmd;
> + void *page_addr;
> + phys_addr_t phys_addr;
> +
> + addr = (unsigned long)pfn_to_page(start_pfn);
> + end = (unsigned long)pfn_to_page(end_pfn);
> +
> + pmd = pmd_offset(pud_offset(pgd_offset_k(addr), addr), addr);
> + for (; addr < end; addr = next, pmd++) {
> + next = pmd_addr_end(addr, end);
> +
> + if (!pmd_present(*pmd))
> + continue;
> +
> + if (IS_ALIGNED(addr, PMD_SIZE) &&
> + IS_ALIGNED(next, PMD_SIZE)) {
> + phys_addr = __pfn_to_phys(pmd_pfn(*pmd));
> + free_bootmem(phys_addr, PMD_SIZE);
> + pmd_clear(pmd);
> + } else {
> + /* If here, we are freeing vmemmap pages. */
> + memset((void *)addr, VMEMMAP_PAGE_INUSE, next - addr);
> + page_addr = page_address(pmd_page(*pmd));
> +
> + if (!memchr_inv(page_addr, VMEMMAP_PAGE_INUSE,
> + PMD_SIZE)) {
> + phys_addr = __pfn_to_phys(pmd_pfn(*pmd));
> + free_bootmem(phys_addr, PMD_SIZE);
> + pmd_clear(pmd);
> + }
> + }
> + }
> +
> + flush_tlb_all();
> +}
> +#else
> static inline void free_memmap(unsigned long start_pfn, unsigned long
> end_pfn) {
> struct page *start_pg, *end_pg;
> @@ -468,31 +509,53 @@ static inline void free_memmap(unsigned long start_pfn, unsigned long end_pfn)
> memblock_free(pg, pgend - pg);
> }
>
> +#endif
> +
> /*
> * The mem_map array can get very big. Free the unused area of the memory map.
> */
> static void __init free_unused_memmap(void) {
> - unsigned long start, prev_end = 0;
> + unsigned long start, cur_start, prev_end = 0;
> struct memblock_region *reg;
>
> for_each_memblock(memory, reg) {
> - start = __phys_to_pfn(reg->base);
> + cur_start = __phys_to_pfn(reg->base);
>
> #ifdef CONFIG_SPARSEMEM
> /*
> * Take care not to free memmap entries that don't exist due
> * to SPARSEMEM sections which aren't present.
> */
> - start = min(start, ALIGN(prev_end, PAGES_PER_SECTION));
> -#endif
> + start = min(cur_start, ALIGN(prev_end, PAGES_PER_SECTION));
> +
> /*
> - * If we had a previous bank, and there is a space between the
> - * current bank and the previous, free it.
> + * Free memory in the case of:
> + * 1. if cur_start - prev_end <= PAGES_PER_SECTION,
> + * free pre_end ~ cur_start.
> + * 2. if cur_start - prev_end > PAGES_PER_SECTION,
> + * free pre_end ~ ALIGN(prev_end, PAGES_PER_SECTION).
> */
> if (prev_end && prev_end < start)
> free_memmap(prev_end, start);
>
> + /*
> + * Free memory in the case of:
> + * if cur_start - prev_end > PAGES_PER_SECTION,
> + * free ALIGN_DOWN(cur_start, PAGES_PER_SECTION) ~ cur_start.
> + */
> + if (cur_start > start &&
> + !IS_ALIGNED(cur_start, PAGES_PER_SECTION))
> + free_memmap(ALIGN_DOWN(cur_start, PAGES_PER_SECTION),
> + cur_start);
> +#else
> + /*
> + * If we had a previous bank, and there is a space between the
> + * current bank and the previous, free it.
> + */
> + if (prev_end && prev_end < cur_start)
> + free_memmap(prev_end, cur_start);
> +#endif
> /*
> * Align up here since the VM subsystem insists that the
> * memmap entries are valid from the bank end aligned to @@ -507,7
> +570,6 @@ static void __init free_unused_memmap(void)
> free_memmap(prev_end, ALIGN(prev_end, PAGES_PER_SECTION)); #endif
> }
> -#endif /* !CONFIG_SPARSEMEM_VMEMMAP */
>
> /*
> * mem_init() marks the free areas in the mem_map and tells us how
> much memory @@ -524,9 +586,8 @@ void __init mem_init(void)
>
> set_max_mapnr(max_pfn - PHYS_PFN_OFFSET);
>
> -#ifndef CONFIG_SPARSEMEM_VMEMMAP
> free_unused_memmap();
> -#endif
> +
> /* this will put all unused low memory onto the freelists */
> memblock_free_all();
>
> --
> 2.15.0
>
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-22 8:41 ` liwei (CM)
@ 2020-07-22 12:49 ` Catalin Marinas
-1 siblings, 0 replies; 26+ messages in thread
From: Catalin Marinas @ 2020-07-22 12:49 UTC (permalink / raw)
To: liwei (CM)
Cc: Song Bao Hua (Barry Song), linux-arm-kernel@lists.infradead.org,
steve.capper@arm.com, Chenfeng (puck), sujunfei,
linux-kernel@vger.kernel.org, fengbaopeng, zhaojiapeng,
Xiaqing (A), Mike Rapoport, will@kernel.org,
nsaenzjulienne@suse.de, butao
On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> Mike Rapoport wrote:
> > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> > > do not free the reserved memory for the page map, this patch do it.
> >
> > Are there numbers showing how much memory is actually freed?
> >
> > The freeing of empty memmap would become rather complex with these
> > changes, do the memory savings justify it?
>
> In the sparse memory model, the size of a section is 1 GB
> (SECTION_SIZE_BITS 30) by default.
Can we reduce SECTION_SIZE_BITS instead? Say 26?
--
Catalin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
@ 2020-07-22 12:49 ` Catalin Marinas
0 siblings, 0 replies; 26+ messages in thread
From: Catalin Marinas @ 2020-07-22 12:49 UTC (permalink / raw)
To: liwei (CM)
Cc: Mike Rapoport, will@kernel.org, Xiaqing (A), Chenfeng (puck),
butao, fengbaopeng, nsaenzjulienne@suse.de, steve.capper@arm.com,
Song Bao Hua (Barry Song), linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, sujunfei, zhaojiapeng
On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> Mike Rapoport wrote:
> > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> > > do not free the reserved memory for the page map, this patch do it.
> >
> > Are there numbers showing how much memory is actually freed?
> >
> > The freeing of empty memmap would become rather complex with these
> > changes, do the memory savings justify it?
>
> In the sparse memory model, the size of a section is 1 GB
> (SECTION_SIZE_BITS 30) by default.
Can we reduce SECTION_SIZE_BITS instead? Say 26?
--
Catalin
^ permalink raw reply [flat|nested] 26+ messages in thread
* 答复: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-22 12:49 ` Catalin Marinas
@ 2020-07-22 13:40 ` liwei (CM)
-1 siblings, 0 replies; 26+ messages in thread
From: liwei (CM) @ 2020-07-22 13:40 UTC (permalink / raw)
To: Catalin Marinas
Cc: Song Bao Hua (Barry Song), linux-arm-kernel@lists.infradead.org,
steve.capper@arm.com, Chenfeng (puck), sujunfei,
linux-kernel@vger.kernel.org, fengbaopeng, zhaojiapeng,
Xiaqing (A), Mike Rapoport, will@kernel.org,
nsaenzjulienne@suse.de, butao
-----邮件原件-----
发件人: Catalin Marinas [mailto:catalin.marinas@arm.com]
发送时间: 2020年7月22日 20:49
收件人: liwei (CM) <liwei213@huawei.com>
抄送: Mike Rapoport <rppt@linux.ibm.com>; will@kernel.org; Xiaqing (A) <saberlily.xia@hisilicon.com>; Chenfeng (puck) <puck.chen@hisilicon.com>; butao <butao@hisilicon.com>; fengbaopeng <fengbaopeng2@hisilicon.com>; nsaenzjulienne@suse.de; steve.capper@arm.com; Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>; linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; sujunfei <sujunfei2@hisilicon.com>; zhaojiapeng <zhaojiapeng@huawei.com>
主题: Re: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> Mike Rapoport wrote:
> > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > For the memory hole, sparse memory model that define
> > > SPARSEMEM_VMEMMAP do not free the reserved memory for the page map, this patch do it.
> >
> > Are there numbers showing how much memory is actually freed?
> >
> > The freeing of empty memmap would become rather complex with these
> > changes, do the memory savings justify it?
>
> In the sparse memory model, the size of a section is 1 GB
> (SECTION_SIZE_BITS 30) by default.
Can we reduce SECTION_SIZE_BITS instead? Say 26?
Hi, Catalin
Yes, you are right, reduce SECTION_SIZE_BITS to 26 can save almost the same memory as the patch.
1) However, it is not clear whether changing the section size has any other impact.
2) Just like the flat memory model and the sparse memory model that does not define VMEMMAP, both of them have their own ways to free unused memmap. I think we've given a similar way for sparse memory define VMEMMAP.
3) This explicit free unused memmap method does reduce unnecessary memory waste for users who do not notice the section size modification.
Hope you will reconsider the purpose and significance of this patch, thanks.
--
Catalin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* 答复: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
@ 2020-07-22 13:40 ` liwei (CM)
0 siblings, 0 replies; 26+ messages in thread
From: liwei (CM) @ 2020-07-22 13:40 UTC (permalink / raw)
To: Catalin Marinas
Cc: Mike Rapoport, will@kernel.org, Xiaqing (A), Chenfeng (puck),
butao, fengbaopeng, nsaenzjulienne@suse.de, steve.capper@arm.com,
Song Bao Hua (Barry Song), linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, sujunfei, zhaojiapeng
-----邮件原件-----
发件人: Catalin Marinas [mailto:catalin.marinas@arm.com]
发送时间: 2020年7月22日 20:49
收件人: liwei (CM) <liwei213@huawei.com>
抄送: Mike Rapoport <rppt@linux.ibm.com>; will@kernel.org; Xiaqing (A) <saberlily.xia@hisilicon.com>; Chenfeng (puck) <puck.chen@hisilicon.com>; butao <butao@hisilicon.com>; fengbaopeng <fengbaopeng2@hisilicon.com>; nsaenzjulienne@suse.de; steve.capper@arm.com; Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>; linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; sujunfei <sujunfei2@hisilicon.com>; zhaojiapeng <zhaojiapeng@huawei.com>
主题: Re: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> Mike Rapoport wrote:
> > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > For the memory hole, sparse memory model that define
> > > SPARSEMEM_VMEMMAP do not free the reserved memory for the page map, this patch do it.
> >
> > Are there numbers showing how much memory is actually freed?
> >
> > The freeing of empty memmap would become rather complex with these
> > changes, do the memory savings justify it?
>
> In the sparse memory model, the size of a section is 1 GB
> (SECTION_SIZE_BITS 30) by default.
Can we reduce SECTION_SIZE_BITS instead? Say 26?
Hi, Catalin
Yes, you are right, reduce SECTION_SIZE_BITS to 26 can save almost the same memory as the patch.
1) However, it is not clear whether changing the section size has any other impact.
2) Just like the flat memory model and the sparse memory model that does not define VMEMMAP, both of them have their own ways to free unused memmap. I think we've given a similar way for sparse memory define VMEMMAP.
3) This explicit free unused memmap method does reduce unnecessary memory waste for users who do not notice the section size modification.
Hope you will reconsider the purpose and significance of this patch, thanks.
--
Catalin
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 答复: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-22 13:40 ` liwei (CM)
@ 2020-07-23 11:29 ` Catalin Marinas
-1 siblings, 0 replies; 26+ messages in thread
From: Catalin Marinas @ 2020-07-23 11:29 UTC (permalink / raw)
To: liwei (CM)
Cc: Song Bao Hua (Barry Song), linux-arm-kernel@lists.infradead.org,
steve.capper@arm.com, Chenfeng (puck), sujunfei,
linux-kernel@vger.kernel.org, fengbaopeng, zhaojiapeng,
Xiaqing (A), Mike Rapoport, will@kernel.org,
nsaenzjulienne@suse.de, butao
On Wed, Jul 22, 2020 at 01:40:34PM +0000, liwei (CM) wrote:
> Catalin Marinas wrote:
> > On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> > > Mike Rapoport wrote:
> > > > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > > > For the memory hole, sparse memory model that define
> > > > > SPARSEMEM_VMEMMAP do not free the reserved memory for the page
> > > > > map, this patch do it.
> > > >
> > > > Are there numbers showing how much memory is actually freed?
> > > >
> > > > The freeing of empty memmap would become rather complex with these
> > > > changes, do the memory savings justify it?
> > >
> > > In the sparse memory model, the size of a section is 1 GB
> > > (SECTION_SIZE_BITS 30) by default.
> >
> > Can we reduce SECTION_SIZE_BITS instead? Say 26?
>
> Yes, you are right, reduce SECTION_SIZE_BITS to 26 can save almost the
> same memory as the patch.
>
> 1) However, it is not clear whether changing the section size has any
> other impact.
Well, we should analyse this.
> 2) Just like the flat memory model and the sparse memory model that
> does not define VMEMMAP, both of them have their own ways to free
> unused memmap. I think we've given a similar way for sparse memory
> define VMEMMAP.
I think we did it for flatmem initially (on arm32) and added support for
sparsemem later on, so free_unused_memmap() had to cope with sparse
sections. On arm64 we introduced vmemmap support and didn't bother with
the freeing at all because of the added complexity of the vmemmap page
tables.
I wonder whether we should just disallow flatmem and non-vmemmap
sparsemem on arm64. Is there any value in keeping them around?
> 3) This explicit free unused memmap method does reduce unnecessary
> memory waste for users who do not notice the section size
> modification.
But if we changed SECTION_SIZE_BITS in the mainline kernel, then we
wouldn't need additional code to free the unused memmap.
--
Catalin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 答复: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
@ 2020-07-23 11:29 ` Catalin Marinas
0 siblings, 0 replies; 26+ messages in thread
From: Catalin Marinas @ 2020-07-23 11:29 UTC (permalink / raw)
To: liwei (CM)
Cc: Mike Rapoport, will@kernel.org, Xiaqing (A), Chenfeng (puck),
butao, fengbaopeng, nsaenzjulienne@suse.de, steve.capper@arm.com,
Song Bao Hua (Barry Song), linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, sujunfei, zhaojiapeng
On Wed, Jul 22, 2020 at 01:40:34PM +0000, liwei (CM) wrote:
> Catalin Marinas wrote:
> > On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> > > Mike Rapoport wrote:
> > > > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > > > For the memory hole, sparse memory model that define
> > > > > SPARSEMEM_VMEMMAP do not free the reserved memory for the page
> > > > > map, this patch do it.
> > > >
> > > > Are there numbers showing how much memory is actually freed?
> > > >
> > > > The freeing of empty memmap would become rather complex with these
> > > > changes, do the memory savings justify it?
> > >
> > > In the sparse memory model, the size of a section is 1 GB
> > > (SECTION_SIZE_BITS 30) by default.
> >
> > Can we reduce SECTION_SIZE_BITS instead? Say 26?
>
> Yes, you are right, reduce SECTION_SIZE_BITS to 26 can save almost the
> same memory as the patch.
>
> 1) However, it is not clear whether changing the section size has any
> other impact.
Well, we should analyse this.
> 2) Just like the flat memory model and the sparse memory model that
> does not define VMEMMAP, both of them have their own ways to free
> unused memmap. I think we've given a similar way for sparse memory
> define VMEMMAP.
I think we did it for flatmem initially (on arm32) and added support for
sparsemem later on, so free_unused_memmap() had to cope with sparse
sections. On arm64 we introduced vmemmap support and didn't bother with
the freeing at all because of the added complexity of the vmemmap page
tables.
I wonder whether we should just disallow flatmem and non-vmemmap
sparsemem on arm64. Is there any value in keeping them around?
> 3) This explicit free unused memmap method does reduce unnecessary
> memory waste for users who do not notice the section size
> modification.
But if we changed SECTION_SIZE_BITS in the mainline kernel, then we
wouldn't need additional code to free the unused memmap.
--
Catalin
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 答复: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-23 11:29 ` Catalin Marinas
@ 2020-07-23 13:18 ` Mike Rapoport
-1 siblings, 0 replies; 26+ messages in thread
From: Mike Rapoport @ 2020-07-23 13:18 UTC (permalink / raw)
To: Catalin Marinas
Cc: Song Bao Hua (Barry Song), sujunfei,
linux-arm-kernel@lists.infradead.org, steve.capper@arm.com,
Chenfeng (puck), Xiaqing (A), linux-kernel@vger.kernel.org,
fengbaopeng, zhaojiapeng, liwei (CM), will@kernel.org,
nsaenzjulienne@suse.de, butao
On Thu, Jul 23, 2020 at 12:29:26PM +0100, Catalin Marinas wrote:
> On Wed, Jul 22, 2020 at 01:40:34PM +0000, liwei (CM) wrote:
> > Catalin Marinas wrote:
> > > On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> > > > Mike Rapoport wrote:
> > > > > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > > > > For the memory hole, sparse memory model that define
> > > > > > SPARSEMEM_VMEMMAP do not free the reserved memory for the page
> > > > > > map, this patch do it.
> > > > >
> > > > > Are there numbers showing how much memory is actually freed?
> > > > >
> > > > > The freeing of empty memmap would become rather complex with these
> > > > > changes, do the memory savings justify it?
> > > >
> > > > In the sparse memory model, the size of a section is 1 GB
> > > > (SECTION_SIZE_BITS 30) by default.
> > >
> > > Can we reduce SECTION_SIZE_BITS instead? Say 26?
> >
> > Yes, you are right, reduce SECTION_SIZE_BITS to 26 can save almost the
> > same memory as the patch.
> >
> > 1) However, it is not clear whether changing the section size has any
> > other impact.
>
> Well, we should analyse this.
>
> > 2) Just like the flat memory model and the sparse memory model that
> > does not define VMEMMAP, both of them have their own ways to free
> > unused memmap. I think we've given a similar way for sparse memory
> > define VMEMMAP.
>
> I think we did it for flatmem initially (on arm32) and added support for
> sparsemem later on, so free_unused_memmap() had to cope with sparse
> sections. On arm64 we introduced vmemmap support and didn't bother with
> the freeing at all because of the added complexity of the vmemmap page
> tables.
>
> I wonder whether we should just disallow flatmem and non-vmemmap
> sparsemem on arm64. Is there any value in keeping them around?
FLATMEM is useful for UMA systems with a single memory bank, so probably
it's worth keeping it for low end machines.
Non-vmemmap sparsemem is essentially disable in arch/arm64/Kconfig, so
for NUMA configurations SPARSEMEM_VMEMMAP is the only choice.
> > 3) This explicit free unused memmap method does reduce unnecessary
> > memory waste for users who do not notice the section size
> > modification.
>
> But if we changed SECTION_SIZE_BITS in the mainline kernel, then we
> wouldn't need additional code to free the unused memmap.
Moreover if we reduce SECTION_SIZE_BITS, we can drop
free_unused_memmap() and since the arm64 memory map for sparse will not
differ from other arches we can drop custom pfn_valid() as well.
> --
> Catalin
--
Sincerely yours,
Mike.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: 答复: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
@ 2020-07-23 13:18 ` Mike Rapoport
0 siblings, 0 replies; 26+ messages in thread
From: Mike Rapoport @ 2020-07-23 13:18 UTC (permalink / raw)
To: Catalin Marinas
Cc: liwei (CM), will@kernel.org, Xiaqing (A), Chenfeng (puck), butao,
fengbaopeng, nsaenzjulienne@suse.de, steve.capper@arm.com,
Song Bao Hua (Barry Song), linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, sujunfei, zhaojiapeng
On Thu, Jul 23, 2020 at 12:29:26PM +0100, Catalin Marinas wrote:
> On Wed, Jul 22, 2020 at 01:40:34PM +0000, liwei (CM) wrote:
> > Catalin Marinas wrote:
> > > On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> > > > Mike Rapoport wrote:
> > > > > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > > > > For the memory hole, sparse memory model that define
> > > > > > SPARSEMEM_VMEMMAP do not free the reserved memory for the page
> > > > > > map, this patch do it.
> > > > >
> > > > > Are there numbers showing how much memory is actually freed?
> > > > >
> > > > > The freeing of empty memmap would become rather complex with these
> > > > > changes, do the memory savings justify it?
> > > >
> > > > In the sparse memory model, the size of a section is 1 GB
> > > > (SECTION_SIZE_BITS 30) by default.
> > >
> > > Can we reduce SECTION_SIZE_BITS instead? Say 26?
> >
> > Yes, you are right, reduce SECTION_SIZE_BITS to 26 can save almost the
> > same memory as the patch.
> >
> > 1) However, it is not clear whether changing the section size has any
> > other impact.
>
> Well, we should analyse this.
>
> > 2) Just like the flat memory model and the sparse memory model that
> > does not define VMEMMAP, both of them have their own ways to free
> > unused memmap. I think we've given a similar way for sparse memory
> > define VMEMMAP.
>
> I think we did it for flatmem initially (on arm32) and added support for
> sparsemem later on, so free_unused_memmap() had to cope with sparse
> sections. On arm64 we introduced vmemmap support and didn't bother with
> the freeing at all because of the added complexity of the vmemmap page
> tables.
>
> I wonder whether we should just disallow flatmem and non-vmemmap
> sparsemem on arm64. Is there any value in keeping them around?
FLATMEM is useful for UMA systems with a single memory bank, so probably
it's worth keeping it for low end machines.
Non-vmemmap sparsemem is essentially disable in arch/arm64/Kconfig, so
for NUMA configurations SPARSEMEM_VMEMMAP is the only choice.
> > 3) This explicit free unused memmap method does reduce unnecessary
> > memory waste for users who do not notice the section size
> > modification.
>
> But if we changed SECTION_SIZE_BITS in the mainline kernel, then we
> wouldn't need additional code to free the unused memmap.
Moreover if we reduce SECTION_SIZE_BITS, we can drop
free_unused_memmap() and since the arm64 memory map for sparse will not
differ from other arches we can drop custom pfn_valid() as well.
> --
> Catalin
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 26+ messages in thread
* 答复: 答复: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-23 13:18 ` Mike Rapoport
@ 2020-07-24 3:40 ` liwei (CM)
-1 siblings, 0 replies; 26+ messages in thread
From: liwei (CM) @ 2020-07-24 3:40 UTC (permalink / raw)
To: Mike Rapoport, Catalin Marinas
Cc: Song Bao Hua (Barry Song), linux-arm-kernel@lists.infradead.org,
steve.capper@arm.com, Chenfeng (puck), sujunfei,
linux-kernel@vger.kernel.org, zhaojiapeng, Xiaqing (A), butao,
will@kernel.org, nsaenzjulienne@suse.de, fengbaopeng
-----邮件原件-----
发件人: Mike Rapoport [mailto:rppt@linux.ibm.com]
发送时间: 2020年7月23日 21:19
收件人: Catalin Marinas <catalin.marinas@arm.com>
抄送: liwei (CM) <liwei213@huawei.com>; will@kernel.org; Xiaqing (A) <saberlily.xia@hisilicon.com>; Chenfeng (puck) <puck.chen@hisilicon.com>; butao <butao@hisilicon.com>; fengbaopeng <fengbaopeng2@hisilicon.com>; nsaenzjulienne@suse.de; steve.capper@arm.com; Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>; linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; sujunfei <sujunfei2@hisilicon.com>; zhaojiapeng <zhaojiapeng@huawei.com>
主题: Re: 答复: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
On Thu, Jul 23, 2020 at 12:29:26PM +0100, Catalin Marinas wrote:
> On Wed, Jul 22, 2020 at 01:40:34PM +0000, liwei (CM) wrote:
> > Catalin Marinas wrote:
> > > On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> > > > Mike Rapoport wrote:
> > > > > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > > > > For the memory hole, sparse memory model that define
> > > > > > SPARSEMEM_VMEMMAP do not free the reserved memory for the
> > > > > > page map, this patch do it.
> > > > >
> > > > > Are there numbers showing how much memory is actually freed?
> > > > >
> > > > > The freeing of empty memmap would become rather complex with
> > > > > these changes, do the memory savings justify it?
> > > >
> > > > In the sparse memory model, the size of a section is 1 GB
> > > > (SECTION_SIZE_BITS 30) by default.
> > >
> > > Can we reduce SECTION_SIZE_BITS instead? Say 26?
> >
> > Yes, you are right, reduce SECTION_SIZE_BITS to 26 can save almost
> > the same memory as the patch.
> >
> > 1) However, it is not clear whether changing the section size has
> > any other impact.
>
> Well, we should analyse this.
>
> > 2) Just like the flat memory model and the sparse memory model that
> > does not define VMEMMAP, both of them have their own ways to free
> > unused memmap. I think we've given a similar way for sparse memory
> > define VMEMMAP.
>
> I think we did it for flatmem initially (on arm32) and added support
> for sparsemem later on, so free_unused_memmap() had to cope with
> sparse sections. On arm64 we introduced vmemmap support and didn't
> bother with the freeing at all because of the added complexity of the
> vmemmap page tables.
>
> I wonder whether we should just disallow flatmem and non-vmemmap
> sparsemem on arm64. Is there any value in keeping them around?
FLATMEM is useful for UMA systems with a single memory bank, so probably it's worth keeping it for low end machines.
Non-vmemmap sparsemem is essentially disable in arch/arm64/Kconfig, so for NUMA configurations SPARSEMEM_VMEMMAP is the only choice.
> > 3) This explicit free unused memmap method does reduce unnecessary
> > memory waste for users who do not notice the section size
> > modification.
>
> But if we changed SECTION_SIZE_BITS in the mainline kernel, then we
> wouldn't need additional code to free the unused memmap.
Moreover if we reduce SECTION_SIZE_BITS, we can drop
free_unused_memmap() and since the arm64 memory map for sparse will not differ from other arches we can drop custom pfn_valid() as well.
Hi, Mike & Catalin
Let's think and discuss together about the impact of directly reducing the section size：
1) Currently, the memory of PC or Mobile devices are increasing. If the section size is reduced, the consumption of the section structure will also increase.
2) If the section size is too small, memory hotplug may be affected. Hotplug add or remove a memblock means that you need to online or offline many sections. In this case, software consumption may increase.
Currently, the page map is wasted when the default section size is used. In some cases, the waste is serious. Please help to check whether the section size reduction has other impacts and whether it meets the long-term evolution.
Thanks.
> --
> Catalin
--
Sincerely yours,
Mike.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* 答复: 答复: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
@ 2020-07-24 3:40 ` liwei (CM)
0 siblings, 0 replies; 26+ messages in thread
From: liwei (CM) @ 2020-07-24 3:40 UTC (permalink / raw)
To: Mike Rapoport, Catalin Marinas
Cc: will@kernel.org, Xiaqing (A), Chenfeng (puck), butao, fengbaopeng,
nsaenzjulienne@suse.de, steve.capper@arm.com,
Song Bao Hua (Barry Song), linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, sujunfei, zhaojiapeng
-----邮件原件-----
发件人: Mike Rapoport [mailto:rppt@linux.ibm.com]
发送时间: 2020年7月23日 21:19
收件人: Catalin Marinas <catalin.marinas@arm.com>
抄送: liwei (CM) <liwei213@huawei.com>; will@kernel.org; Xiaqing (A) <saberlily.xia@hisilicon.com>; Chenfeng (puck) <puck.chen@hisilicon.com>; butao <butao@hisilicon.com>; fengbaopeng <fengbaopeng2@hisilicon.com>; nsaenzjulienne@suse.de; steve.capper@arm.com; Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>; linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; sujunfei <sujunfei2@hisilicon.com>; zhaojiapeng <zhaojiapeng@huawei.com>
主题: Re: 答复: 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
On Thu, Jul 23, 2020 at 12:29:26PM +0100, Catalin Marinas wrote:
> On Wed, Jul 22, 2020 at 01:40:34PM +0000, liwei (CM) wrote:
> > Catalin Marinas wrote:
> > > On Wed, Jul 22, 2020 at 08:41:17AM +0000, liwei (CM) wrote:
> > > > Mike Rapoport wrote:
> > > > > On Tue, Jul 21, 2020 at 03:32:03PM +0800, Wei Li wrote:
> > > > > > For the memory hole, sparse memory model that define
> > > > > > SPARSEMEM_VMEMMAP do not free the reserved memory for the
> > > > > > page map, this patch do it.
> > > > >
> > > > > Are there numbers showing how much memory is actually freed?
> > > > >
> > > > > The freeing of empty memmap would become rather complex with
> > > > > these changes, do the memory savings justify it?
> > > >
> > > > In the sparse memory model, the size of a section is 1 GB
> > > > (SECTION_SIZE_BITS 30) by default.
> > >
> > > Can we reduce SECTION_SIZE_BITS instead? Say 26?
> >
> > Yes, you are right, reduce SECTION_SIZE_BITS to 26 can save almost
> > the same memory as the patch.
> >
> > 1) However, it is not clear whether changing the section size has
> > any other impact.
>
> Well, we should analyse this.
>
> > 2) Just like the flat memory model and the sparse memory model that
> > does not define VMEMMAP, both of them have their own ways to free
> > unused memmap. I think we've given a similar way for sparse memory
> > define VMEMMAP.
>
> I think we did it for flatmem initially (on arm32) and added support
> for sparsemem later on, so free_unused_memmap() had to cope with
> sparse sections. On arm64 we introduced vmemmap support and didn't
> bother with the freeing at all because of the added complexity of the
> vmemmap page tables.
>
> I wonder whether we should just disallow flatmem and non-vmemmap
> sparsemem on arm64. Is there any value in keeping them around?
FLATMEM is useful for UMA systems with a single memory bank, so probably it's worth keeping it for low end machines.
Non-vmemmap sparsemem is essentially disable in arch/arm64/Kconfig, so for NUMA configurations SPARSEMEM_VMEMMAP is the only choice.
> > 3) This explicit free unused memmap method does reduce unnecessary
> > memory waste for users who do not notice the section size
> > modification.
>
> But if we changed SECTION_SIZE_BITS in the mainline kernel, then we
> wouldn't need additional code to free the unused memmap.
Moreover if we reduce SECTION_SIZE_BITS, we can drop
free_unused_memmap() and since the arm64 memory map for sparse will not differ from other arches we can drop custom pfn_valid() as well.
Hi, Mike & Catalin
Let's think and discuss together about the impact of directly reducing the section size：
1) Currently, the memory of PC or Mobile devices are increasing. If the section size is reduced, the consumption of the section structure will also increase.
2) If the section size is too small, memory hotplug may be affected. Hotplug add or remove a memblock means that you need to online or offline many sections. In this case, software consumption may increase.
Currently, the page map is wasted when the default section size is used. In some cases, the waste is serious. Please help to check whether the section size reduction has other impacts and whether it meets the long-term evolution.
Thanks.
> --
> Catalin
--
Sincerely yours,
Mike.
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-21 7:32 ` Wei Li
@ 2020-07-23 2:33 ` Anshuman Khandual
-1 siblings, 0 replies; 26+ messages in thread
From: Anshuman Khandual @ 2020-07-23 2:33 UTC (permalink / raw)
To: Wei Li, catalin.marinas, will
Cc: song.bao.hua, steve.capper, puck.chen, sujunfei2, linux-kernel,
rppt, nsaenzjulienne, saberlily.xia, butao, linux-arm-kernel,
fengbaopeng2
On 07/21/2020 01:02 PM, Wei Li wrote:
> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> do not free the reserved memory for the page map, this patch do it.
>
> Signed-off-by: Wei Li <liwei213@huawei.com>
> Signed-off-by: Chen Feng <puck.chen@hisilicon.com>
> Signed-off-by: Xia Qing <saberlily.xia@hisilicon.com>
> ---
> arch/arm64/mm/init.c | 81 +++++++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 71 insertions(+), 10 deletions(-)
This patch does not compile on 5.8-rc6 with defconfig.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
@ 2020-07-23 2:33 ` Anshuman Khandual
0 siblings, 0 replies; 26+ messages in thread
From: Anshuman Khandual @ 2020-07-23 2:33 UTC (permalink / raw)
To: Wei Li, catalin.marinas, will
Cc: song.bao.hua, sujunfei2, saberlily.xia, linux-arm-kernel,
steve.capper, puck.chen, linux-kernel, rppt, fengbaopeng2,
nsaenzjulienne, butao
On 07/21/2020 01:02 PM, Wei Li wrote:
> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> do not free the reserved memory for the page map, this patch do it.
>
> Signed-off-by: Wei Li <liwei213@huawei.com>
> Signed-off-by: Chen Feng <puck.chen@hisilicon.com>
> Signed-off-by: Xia Qing <saberlily.xia@hisilicon.com>
> ---
> arch/arm64/mm/init.c | 81 +++++++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 71 insertions(+), 10 deletions(-)
This patch does not compile on 5.8-rc6 with defconfig.
^ permalink raw reply [flat|nested] 26+ messages in thread
* 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
2020-07-23 2:33 ` Anshuman Khandual
@ 2020-07-23 3:28 ` liwei (CM)
-1 siblings, 0 replies; 26+ messages in thread
From: liwei (CM) @ 2020-07-23 3:28 UTC (permalink / raw)
To: Anshuman Khandual, catalin.marinas@arm.com, will@kernel.org
Cc: Song Bao Hua (Barry Song), steve.capper@arm.com, Chenfeng (puck),
sujunfei, linux-kernel@vger.kernel.org, rppt@linux.ibm.com,
nsaenzjulienne@suse.de, Xiaqing (A), butao,
linux-arm-kernel@lists.infradead.org, fengbaopeng
-----邮件原件-----
发件人: Anshuman Khandual [mailto:anshuman.khandual@arm.com]
发送时间: 2020年7月23日 10:33
收件人: liwei (CM) <liwei213@huawei.com>; catalin.marinas@arm.com; will@kernel.org
抄送: Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>; sujunfei <sujunfei2@hisilicon.com>; Xiaqing (A) <saberlily.xia@hisilicon.com>; linux-arm-kernel@lists.infradead.org; steve.capper@arm.com; Chenfeng (puck) <puck.chen@hisilicon.com>; linux-kernel@vger.kernel.org; rppt@linux.ibm.com; fengbaopeng <fengbaopeng2@hisilicon.com>; nsaenzjulienne@suse.de; butao <butao@hisilicon.com>
主题: Re: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
On 07/21/2020 01:02 PM, Wei Li wrote:
> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> do not free the reserved memory for the page map, this patch do it.
>
> Signed-off-by: Wei Li <liwei213@huawei.com>
> Signed-off-by: Chen Feng <puck.chen@hisilicon.com>
> Signed-off-by: Xia Qing <saberlily.xia@hisilicon.com>
> ---
> arch/arm64/mm/init.c | 81
> +++++++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 71 insertions(+), 10 deletions(-)
This patch does not compile on 5.8-rc6 with defconfig.
Hi,
We're working on patch v2 as soon as possible.
Thanks.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 26+ messages in thread
* 答复: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
@ 2020-07-23 3:28 ` liwei (CM)
0 siblings, 0 replies; 26+ messages in thread
From: liwei (CM) @ 2020-07-23 3:28 UTC (permalink / raw)
To: Anshuman Khandual, catalin.marinas@arm.com, will@kernel.org
Cc: Song Bao Hua (Barry Song), sujunfei, Xiaqing (A),
linux-arm-kernel@lists.infradead.org, steve.capper@arm.com,
Chenfeng (puck), linux-kernel@vger.kernel.org, rppt@linux.ibm.com,
fengbaopeng, nsaenzjulienne@suse.de, butao
-----邮件原件-----
发件人: Anshuman Khandual [mailto:anshuman.khandual@arm.com]
发送时间: 2020年7月23日 10:33
收件人: liwei (CM) <liwei213@huawei.com>; catalin.marinas@arm.com; will@kernel.org
抄送: Song Bao Hua (Barry Song) <song.bao.hua@hisilicon.com>; sujunfei <sujunfei2@hisilicon.com>; Xiaqing (A) <saberlily.xia@hisilicon.com>; linux-arm-kernel@lists.infradead.org; steve.capper@arm.com; Chenfeng (puck) <puck.chen@hisilicon.com>; linux-kernel@vger.kernel.org; rppt@linux.ibm.com; fengbaopeng <fengbaopeng2@hisilicon.com>; nsaenzjulienne@suse.de; butao <butao@hisilicon.com>
主题: Re: [PATCH] arm64: mm: free unused memmap for sparse memory model that define VMEMMAP
On 07/21/2020 01:02 PM, Wei Li wrote:
> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> do not free the reserved memory for the page map, this patch do it.
>
> Signed-off-by: Wei Li <liwei213@huawei.com>
> Signed-off-by: Chen Feng <puck.chen@hisilicon.com>
> Signed-off-by: Xia Qing <saberlily.xia@hisilicon.com>
> ---
> arch/arm64/mm/init.c | 81
> +++++++++++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 71 insertions(+), 10 deletions(-)
This patch does not compile on 5.8-rc6 with defconfig.
Hi,
We're working on patch v2 as soon as possible.
Thanks.
^ permalink raw reply [flat|nested] 26+ messages in thread