* Re: [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup [not found] <20230825190436.55045-11-mike.kravetz@oracle.com> @ 2023-08-26 5:56 ` kernel test robot 2023-08-28 9:42 ` Joao Martins 2023-08-26 18:14 ` kernel test robot 1 sibling, 1 reply; 5+ messages in thread From: kernel test robot @ 2023-08-26 5:56 UTC (permalink / raw) To: Mike Kravetz, linux-mm, linux-kernel Cc: llvm, oe-kbuild-all, Muchun Song, Joao Martins, Oscar Salvador, David Hildenbrand, Miaohe Lin, David Rientjes, Anshuman Khandual, Naoya Horiguchi, Barry Song, Michal Hocko, Matthew Wilcox, Xiongchun Duan, Andrew Morton, Linux Memory Management List, Mike Kravetz Hi Mike, kernel test robot noticed the following build errors: [auto build test ERROR on next-20230825] [cannot apply to akpm-mm/mm-everything v6.5-rc7 v6.5-rc6 v6.5-rc5 linus/master v6.5-rc7] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Mike-Kravetz/hugetlb-clear-flags-in-tail-pages-that-will-be-freed-individually/20230826-030805 base: next-20230825 patch link: https://lore.kernel.org/r/20230825190436.55045-11-mike.kravetz%40oracle.com patch subject: [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup config: s390-randconfig-001-20230826 (https://download.01.org/0day-ci/archive/20230826/202308261325.ipTttZHZ-lkp@intel.com/config) compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a) reproduce: (https://download.01.org/0day-ci/archive/20230826/202308261325.ipTttZHZ-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202308261325.ipTttZHZ-lkp@intel.com/ All error/warnings (new ones prefixed by >>): mm/hugetlb_vmemmap.c:661:6: warning: no previous prototype for function 'hugetlb_vmemmap_optimize_bulk' [-Wmissing-prototypes] 661 | void hugetlb_vmemmap_optimize_bulk(const struct hstate *h, struct page *head, | ^ mm/hugetlb_vmemmap.c:661:1: note: declare 'static' if the function is not intended to be used outside of this translation unit 661 | void hugetlb_vmemmap_optimize_bulk(const struct hstate *h, struct page *head, | ^ | static >> mm/hugetlb_vmemmap.c:667:6: warning: no previous prototype for function 'hugetlb_vmemmap_split' [-Wmissing-prototypes] 667 | void hugetlb_vmemmap_split(const struct hstate *h, struct page *head) | ^ mm/hugetlb_vmemmap.c:667:1: note: declare 'static' if the function is not intended to be used outside of this translation unit 667 | void hugetlb_vmemmap_split(const struct hstate *h, struct page *head) | ^ | static >> mm/hugetlb_vmemmap.c:698:28: error: use of undeclared identifier 'TLB_FLUSH_ALL' 698 | flush_tlb_kernel_range(0, TLB_FLUSH_ALL); | ^ 2 warnings and 1 error generated. vim +/TLB_FLUSH_ALL +698 mm/hugetlb_vmemmap.c 666 > 667 void hugetlb_vmemmap_split(const struct hstate *h, struct page *head) 668 { 669 unsigned long vmemmap_start = (unsigned long)head, vmemmap_end; 670 unsigned long vmemmap_reuse; 671 672 if (!vmemmap_should_optimize(h, head)) 673 return; 674 675 static_branch_inc(&hugetlb_optimize_vmemmap_key); 676 677 vmemmap_end = vmemmap_start + hugetlb_vmemmap_size(h); 678 vmemmap_reuse = vmemmap_start; 679 vmemmap_start += HUGETLB_VMEMMAP_RESERVE_SIZE; 680 681 /* 682 * Remap the vmemmap virtual address range [@vmemmap_start, @vmemmap_end) 683 * to the page which @vmemmap_reuse is mapped to, then free the pages 684 * which the range [@vmemmap_start, @vmemmap_end] is mapped to. 685 */ 686 if (vmemmap_remap_split(vmemmap_start, vmemmap_end, vmemmap_reuse)) 687 static_branch_dec(&hugetlb_optimize_vmemmap_key); 688 } 689 690 void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_list) 691 { 692 struct folio *folio; 693 LIST_HEAD(vmemmap_pages); 694 695 list_for_each_entry(folio, folio_list, lru) 696 hugetlb_vmemmap_split(h, &folio->page); 697 > 698 flush_tlb_kernel_range(0, TLB_FLUSH_ALL); 699 700 list_for_each_entry(folio, folio_list, lru) 701 hugetlb_vmemmap_optimize_bulk(h, &folio->page, &vmemmap_pages); 702 703 free_vmemmap_page_list(&vmemmap_pages); 704 } 705 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup 2023-08-26 5:56 ` [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup kernel test robot @ 2023-08-28 9:42 ` Joao Martins 2023-08-28 16:44 ` Mike Kravetz 0 siblings, 1 reply; 5+ messages in thread From: Joao Martins @ 2023-08-28 9:42 UTC (permalink / raw) To: kernel test robot, Mike Kravetz, linux-mm Cc: llvm, oe-kbuild-all, Muchun Song, Oscar Salvador, David Hildenbrand, Miaohe Lin, David Rientjes, Anshuman Khandual, Naoya Horiguchi, Barry Song, Michal Hocko, Matthew Wilcox, Xiongchun Duan, Andrew Morton, linux-kernel On 26/08/2023 06:56, kernel test robot wrote: > Hi Mike, > > kernel test robot noticed the following build errors: > > [auto build test ERROR on next-20230825] > [cannot apply to akpm-mm/mm-everything v6.5-rc7 v6.5-rc6 v6.5-rc5 linus/master v6.5-rc7] > [If your patch is applied to the wrong git tree, kindly drop us a note. > And when submitting patch, we suggest to use '--base' as documented in > https://git-scm.com/docs/git-format-patch#_base_tree_information] > > url: https://github.com/intel-lab-lkp/linux/commits/Mike-Kravetz/hugetlb-clear-flags-in-tail-pages-that-will-be-freed-individually/20230826-030805 > base: next-20230825 > patch link: https://lore.kernel.org/r/20230825190436.55045-11-mike.kravetz%40oracle.com > patch subject: [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup > config: s390-randconfig-001-20230826 (https://download.01.org/0day-ci/archive/20230826/202308261325.ipTttZHZ-lkp@intel.com/config) > compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a) > reproduce: (https://download.01.org/0day-ci/archive/20230826/202308261325.ipTttZHZ-lkp@intel.com/reproduce) > > If you fix the issue in a separate patch/commit (i.e. not just a new version of > the same patch/commit), kindly add following tags > | Reported-by: kernel test robot <lkp@intel.com> > | Closes: https://lore.kernel.org/oe-kbuild-all/202308261325.ipTttZHZ-lkp@intel.com/ > > All error/warnings (new ones prefixed by >>): > [...] >>> mm/hugetlb_vmemmap.c:698:28: error: use of undeclared identifier 'TLB_FLUSH_ALL' > 698 | flush_tlb_kernel_range(0, TLB_FLUSH_ALL); > | ^ > 2 warnings and 1 error generated. > > TLB_FLUSH_ALL is x86 only so what I wrote above is wrong in what should be architecture independent. The way I should have written the global TLB flush is to use flush_tlb_all(), which is what is implemented by the arch. The alternative is to compose a start/end tuple in the top-level optimize-folios function as we iterate over folios to remap, and flush via flush_tlb_kernel_range(). But this would likely only be relevant on x86 only, that is to optimize the flushing of 3 contiguous 2M hugetlb pages (~24 vmemmap pages) as that's where the TLB flush ceiling is put (31 pages) for per-page VA flush, before falling back to a global TLB flush. Weren't sure of the added complexity for dubious benefit thus kept it in global TLB flush. > vim +/TLB_FLUSH_ALL +698 mm/hugetlb_vmemmap.c > > 666 > > 667 void hugetlb_vmemmap_split(const struct hstate *h, struct page *head) > 668 { > 669 unsigned long vmemmap_start = (unsigned long)head, vmemmap_end; > 670 unsigned long vmemmap_reuse; > 671 > 672 if (!vmemmap_should_optimize(h, head)) > 673 return; > 674 > 675 static_branch_inc(&hugetlb_optimize_vmemmap_key); > 676 > 677 vmemmap_end = vmemmap_start + hugetlb_vmemmap_size(h); > 678 vmemmap_reuse = vmemmap_start; > 679 vmemmap_start += HUGETLB_VMEMMAP_RESERVE_SIZE; > 680 > 681 /* > 682 * Remap the vmemmap virtual address range [@vmemmap_start, @vmemmap_end) > 683 * to the page which @vmemmap_reuse is mapped to, then free the pages > 684 * which the range [@vmemmap_start, @vmemmap_end] is mapped to. > 685 */ > 686 if (vmemmap_remap_split(vmemmap_start, vmemmap_end, vmemmap_reuse)) > 687 static_branch_dec(&hugetlb_optimize_vmemmap_key); > 688 } > 689 > 690 void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_list) > 691 { > 692 struct folio *folio; > 693 LIST_HEAD(vmemmap_pages); > 694 > 695 list_for_each_entry(folio, folio_list, lru) > 696 hugetlb_vmemmap_split(h, &folio->page); > 697 > > 698 flush_tlb_kernel_range(0, TLB_FLUSH_ALL); > 699 > 700 list_for_each_entry(folio, folio_list, lru) > 701 hugetlb_vmemmap_optimize_bulk(h, &folio->page, &vmemmap_pages); > 702 > 703 free_vmemmap_page_list(&vmemmap_pages); > 704 } > 705 > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup 2023-08-28 9:42 ` Joao Martins @ 2023-08-28 16:44 ` Mike Kravetz 2023-08-29 3:47 ` Muchun Song 0 siblings, 1 reply; 5+ messages in thread From: Mike Kravetz @ 2023-08-28 16:44 UTC (permalink / raw) To: Joao Martins Cc: kernel test robot, linux-mm, llvm, oe-kbuild-all, Muchun Song, Oscar Salvador, David Hildenbrand, Miaohe Lin, David Rientjes, Anshuman Khandual, Naoya Horiguchi, Barry Song, Michal Hocko, Matthew Wilcox, Xiongchun Duan, Andrew Morton, linux-kernel On 08/28/23 10:42, Joao Martins wrote: > On 26/08/2023 06:56, kernel test robot wrote: > > Hi Mike, > > > > kernel test robot noticed the following build errors: > > > > [auto build test ERROR on next-20230825] > > [cannot apply to akpm-mm/mm-everything v6.5-rc7 v6.5-rc6 v6.5-rc5 linus/master v6.5-rc7] > > [If your patch is applied to the wrong git tree, kindly drop us a note. > > And when submitting patch, we suggest to use '--base' as documented in > > https://git-scm.com/docs/git-format-patch#_base_tree_information] > > > > url: https://github.com/intel-lab-lkp/linux/commits/Mike-Kravetz/hugetlb-clear-flags-in-tail-pages-that-will-be-freed-individually/20230826-030805 > > base: next-20230825 > > patch link: https://lore.kernel.org/r/20230825190436.55045-11-mike.kravetz%40oracle.com > > patch subject: [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup > > config: s390-randconfig-001-20230826 (https://download.01.org/0day-ci/archive/20230826/202308261325.ipTttZHZ-lkp@intel.com/config) > > compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a) > > reproduce: (https://download.01.org/0day-ci/archive/20230826/202308261325.ipTttZHZ-lkp@intel.com/reproduce) > > > > If you fix the issue in a separate patch/commit (i.e. not just a new version of > > the same patch/commit), kindly add following tags > > | Reported-by: kernel test robot <lkp@intel.com> > > | Closes: https://lore.kernel.org/oe-kbuild-all/202308261325.ipTttZHZ-lkp@intel.com/ > > > > All error/warnings (new ones prefixed by >>): > > > > [...] > > >>> mm/hugetlb_vmemmap.c:698:28: error: use of undeclared identifier 'TLB_FLUSH_ALL' > > 698 | flush_tlb_kernel_range(0, TLB_FLUSH_ALL); > > | ^ > > 2 warnings and 1 error generated. > > > > > > TLB_FLUSH_ALL is x86 only so what I wrote above is wrong in what should be > architecture independent. The way I should have written the global TLB flush is > to use flush_tlb_all(), which is what is implemented by the arch. > > The alternative is to compose a start/end tuple in the top-level optimize-folios > function as we iterate over folios to remap, and flush via > flush_tlb_kernel_range(). But this would likely only be relevant on x86 only, > that is to optimize the flushing of 3 contiguous 2M hugetlb pages (~24 vmemmap > pages) as that's where the TLB flush ceiling is put (31 pages) for per-page VA > flush, before falling back to a global TLB flush. Weren't sure of the added > complexity for dubious benefit thus kept it in global TLB flush. Thanks Joao. I added my share of build issues to this RFC as can be seen in the bot responses to other patches. My assumption is that these build issues will not prevent people from looking into and commenting on the bigger performance issue that was the reason for this series. The build issues would of course be resolved if there is some concensus that this is the way to move forward to address this issue. If the build issues are a stumbling block for anyone to look at this bigger issue, let me know and I will fix them all ASAP. -- Mike Kravetz ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup 2023-08-28 16:44 ` Mike Kravetz @ 2023-08-29 3:47 ` Muchun Song 0 siblings, 0 replies; 5+ messages in thread From: Muchun Song @ 2023-08-29 3:47 UTC (permalink / raw) To: Mike Kravetz Cc: Joao Martins, kernel test robot, Linux-MM, llvm, oe-kbuild-all, Muchun Song, Oscar Salvador, David Hildenbrand, Miaohe Lin, David Rientjes, Anshuman Khandual, Naoya Horiguchi, Barry Song, Michal Hocko, Matthew Wilcox, Xiongchun Duan, Andrew Morton, LKML > On Aug 29, 2023, at 00:44, Mike Kravetz <mike.kravetz@oracle.com> wrote: > > On 08/28/23 10:42, Joao Martins wrote: >> On 26/08/2023 06:56, kernel test robot wrote: >>> Hi Mike, >>> >>> kernel test robot noticed the following build errors: >>> >>> [auto build test ERROR on next-20230825] >>> [cannot apply to akpm-mm/mm-everything v6.5-rc7 v6.5-rc6 v6.5-rc5 linus/master v6.5-rc7] >>> [If your patch is applied to the wrong git tree, kindly drop us a note. >>> And when submitting patch, we suggest to use '--base' as documented in >>> https://git-scm.com/docs/git-format-patch#_base_tree_information] >>> >>> url: https://github.com/intel-lab-lkp/linux/commits/Mike-Kravetz/hugetlb-clear-flags-in-tail-pages-that-will-be-freed-individually/20230826-030805 >>> base: next-20230825 >>> patch link: https://lore.kernel.org/r/20230825190436.55045-11-mike.kravetz%40oracle.com >>> patch subject: [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup >>> config: s390-randconfig-001-20230826 (https://download.01.org/0day-ci/archive/20230826/202308261325.ipTttZHZ-lkp@intel.com/config) >>> compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project.git 4a5ac14ee968ff0ad5d2cc1ffa0299048db4c88a) >>> reproduce: (https://download.01.org/0day-ci/archive/20230826/202308261325.ipTttZHZ-lkp@intel.com/reproduce) >>> >>> If you fix the issue in a separate patch/commit (i.e. not just a new version of >>> the same patch/commit), kindly add following tags >>> | Reported-by: kernel test robot <lkp@intel.com> >>> | Closes: https://lore.kernel.org/oe-kbuild-all/202308261325.ipTttZHZ-lkp@intel.com/ >>> >>> All error/warnings (new ones prefixed by >>): >>> >> >> [...] >> >>>>> mm/hugetlb_vmemmap.c:698:28: error: use of undeclared identifier 'TLB_FLUSH_ALL' >>> 698 | flush_tlb_kernel_range(0, TLB_FLUSH_ALL); >>> | ^ >>> 2 warnings and 1 error generated. >>> >>> >> >> TLB_FLUSH_ALL is x86 only so what I wrote above is wrong in what should be >> architecture independent. The way I should have written the global TLB flush is >> to use flush_tlb_all(), which is what is implemented by the arch. >> >> The alternative is to compose a start/end tuple in the top-level optimize-folios >> function as we iterate over folios to remap, and flush via >> flush_tlb_kernel_range(). But this would likely only be relevant on x86 only, >> that is to optimize the flushing of 3 contiguous 2M hugetlb pages (~24 vmemmap >> pages) as that's where the TLB flush ceiling is put (31 pages) for per-page VA >> flush, before falling back to a global TLB flush. Weren't sure of the added >> complexity for dubious benefit thus kept it in global TLB flush. > > Thanks Joao. > > I added my share of build issues to this RFC as can be seen in the bot > responses to other patches. > > My assumption is that these build issues will not prevent people from > looking into and commenting on the bigger performance issue that was the > reason for this series. The build issues would of course be resolved if > there is some concensus that this is the way to move forward to address > this issue. If the build issues are a stumbling block for anyone to > look at this bigger issue, let me know and I will fix them all ASAP. No need to update. But I need some time to look. Muchun, Thanks. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup [not found] <20230825190436.55045-11-mike.kravetz@oracle.com> 2023-08-26 5:56 ` [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup kernel test robot @ 2023-08-26 18:14 ` kernel test robot 1 sibling, 0 replies; 5+ messages in thread From: kernel test robot @ 2023-08-26 18:14 UTC (permalink / raw) To: Mike Kravetz, linux-mm, linux-kernel Cc: oe-kbuild-all, Muchun Song, Joao Martins, Oscar Salvador, David Hildenbrand, Miaohe Lin, David Rientjes, Anshuman Khandual, Naoya Horiguchi, Barry Song, Michal Hocko, Matthew Wilcox, Xiongchun Duan, Andrew Morton, Linux Memory Management List, Mike Kravetz Hi Mike, kernel test robot noticed the following build warnings: [auto build test WARNING on next-20230825] [cannot apply to akpm-mm/mm-everything v6.5-rc7 v6.5-rc6 v6.5-rc5 linus/master v6.5-rc7] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Mike-Kravetz/hugetlb-clear-flags-in-tail-pages-that-will-be-freed-individually/20230826-030805 base: next-20230825 patch link: https://lore.kernel.org/r/20230825190436.55045-11-mike.kravetz%40oracle.com patch subject: [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup config: x86_64-randconfig-r031-20230826 (https://download.01.org/0day-ci/archive/20230827/202308270248.eJT8UDG3-lkp@intel.com/config) compiler: gcc-11 (Debian 11.3.0-12) 11.3.0 reproduce: (https://download.01.org/0day-ci/archive/20230827/202308270248.eJT8UDG3-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202308270248.eJT8UDG3-lkp@intel.com/ All warnings (new ones prefixed by >>): >> mm/hugetlb_vmemmap.c:40: warning: Function parameter or member 'flags' not described in 'vmemmap_remap_walk' vim +40 mm/hugetlb_vmemmap.c f41f2ed43ca525 Muchun Song 2021-06-30 19 998a2997885f73 Muchun Song 2022-06-28 20 /** 998a2997885f73 Muchun Song 2022-06-28 21 * struct vmemmap_remap_walk - walk vmemmap page table 998a2997885f73 Muchun Song 2022-06-28 22 * 998a2997885f73 Muchun Song 2022-06-28 23 * @remap_pte: called for each lowest-level entry (PTE). 998a2997885f73 Muchun Song 2022-06-28 24 * @nr_walked: the number of walked pte. 998a2997885f73 Muchun Song 2022-06-28 25 * @reuse_page: the page which is reused for the tail vmemmap pages. 998a2997885f73 Muchun Song 2022-06-28 26 * @reuse_addr: the virtual address of the @reuse_page page. 998a2997885f73 Muchun Song 2022-06-28 27 * @vmemmap_pages: the list head of the vmemmap pages that can be freed 998a2997885f73 Muchun Song 2022-06-28 28 * or is mapped from. 506a27a4627ab7 Joao Martins 2023-08-25 29 * @flags used to modify behavior in bulk operations 998a2997885f73 Muchun Song 2022-06-28 30 */ 998a2997885f73 Muchun Song 2022-06-28 31 struct vmemmap_remap_walk { 998a2997885f73 Muchun Song 2022-06-28 32 void (*remap_pte)(pte_t *pte, unsigned long addr, 998a2997885f73 Muchun Song 2022-06-28 33 struct vmemmap_remap_walk *walk); 998a2997885f73 Muchun Song 2022-06-28 34 unsigned long nr_walked; 998a2997885f73 Muchun Song 2022-06-28 35 struct page *reuse_page; 998a2997885f73 Muchun Song 2022-06-28 36 unsigned long reuse_addr; 998a2997885f73 Muchun Song 2022-06-28 37 struct list_head *vmemmap_pages; 506a27a4627ab7 Joao Martins 2023-08-25 38 #define VMEMMAP_REMAP_ONLY_SPLIT BIT(0) 506a27a4627ab7 Joao Martins 2023-08-25 39 unsigned long flags; 998a2997885f73 Muchun Song 2022-06-28 @40 }; 998a2997885f73 Muchun Song 2022-06-28 41 -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-08-29 3:48 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20230825190436.55045-11-mike.kravetz@oracle.com>
2023-08-26 5:56 ` [PATCH 10/12] hugetlb: batch PMD split for bulk vmemmap dedup kernel test robot
2023-08-28 9:42 ` Joao Martins
2023-08-28 16:44 ` Mike Kravetz
2023-08-29 3:47 ` Muchun Song
2023-08-26 18:14 ` kernel test robot
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.