Re: [PATCH 05/11] mm/hugetlb: drop boot-time HVO handling for gigantic folios

[sashiko-bot@kernel.org]

Thank you for your contribution! Sashiko AI review found no new issues
in this patch, but noticed 2 related pre-existing issues it does not
introduce:
- [High] mm/memory-failure: race between hugetlb_update_hwpoison() and
  HVO head remap [1]
- [Critical] mm/hugetlb_vmemmap: race between deferred PMD split flush
  and PTE remap in vmemmap_split_pmd() [2]
--

commit da0d2d6e0b849f9a259d74fcb047aaceb9617411
Author: Muchun Song <muchun.song@linux.dev>

mm/hugetlb: drop boot-time HVO handling for gigantic folios

This commit removes the boot-time bulk vmemmap optimization call and tail
page initialization fallback for gigantic folios, relying instead on the
section-based vmemmap optimization infrastructure during bootmem gathering.

> diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
> index ba8e11e399b24..d73ceeef392eb 100644
> --- a/mm/hugetlb_vmemmap.c
> +++ b/mm/hugetlb_vmemmap.c
> @@ -552,7 +552,6 @@ static int hugetlb_vmemmap_pteify_folio(const struct hstate *h, struct folio *fo
>  void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_list)
>  {
>  	struct folio *folio;
> -	unsigned long nr_to_optimize = 0;
>  	LIST_HEAD(vmemmap_pages);
>  	unsigned long flags = VMEMMAP_DEFER_TLB_FLUSH;

[Severity: High]
This problem wasn't introduced by this patch, but can a concurrent memory
failure race with vmemmap optimization [1] and lose hardware poison
state on a newly allocated hugetlb folio?

After init_new_hugetlb_folio() sets the hugetlb flag on a fresh folio
(for example in prep_and_add_allocated_folios()), both
folio_test_hugetlb_freed() and folio_test_hugetlb_migratable() remain
false while hugetlb_vmemmap_optimize_folios() remaps its vmemmap pages:

CPU0                                  CPU1
hugetlb_vmemmap_optimize_folios()
  __hugetlb_vmemmap_optimize_folio()
    vmemmap_optimize_folio()
      vmemmap_optimize_pte()
        vmemmap_dup_page_from_list()
          copy_page(dst, src)
                                      try_memory_failure_hugetlb()
                                        get_huge_page_for_hwpoison()
                                          // returns MF_HUGETLB_RETRY
                                        // retries with MF_NO_RETRY:
                                        get_huge_page_for_hwpoison()
                                          spin_lock_irq(&hugetlb_lock)
                                          hugetlb_update_hwpoison()
                                            folio_test_set_hwpoison(folio)
                                            llist_add(&raw_hwp->node, head)
                                          spin_unlock_irq(&hugetlb_lock)
        set_pte_at(..., dst, ...)
  flush_tlb_all()
  free_vmemmap_page_list()  <-- old head page freed, hwpoison lost

Because vmemmap_optimize_pte() copies and remaps the head struct page
without synchronizing with hugetlb_lock (and defers the TLB flush across
the batch), hugetlb_update_hwpoison() can write PG_hwpoison and the
raw_hwp_page list entry to the old head struct page after copy_page() has
already finished.  Once the remap and TLB flush complete, the raw_hwp_page
allocation is leaked and the poisoned hugepage is added to the pool
unmarked.

> @@ -560,18 +559,7 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
>  		return;
>  
>  	list_for_each_entry(folio, folio_list, lru) {
> -		int ret;
> -
> -		/*
> -		 * Bootmem gigantic folios may already be marked optimized when
> -		 * their vmemmap layout was prepared earlier, so skip them here.
> -		 */
> -		if (folio_test_hugetlb_vmemmap_optimized(folio))
> -			continue;
> -
> -		nr_to_optimize++;
> -
> -		ret = hugetlb_vmemmap_pteify_folio(h, folio, flags);
> +		int ret = hugetlb_vmemmap_pteify_folio(h, folio, flags);
>  
>  		/*
>  		 * Splitting the PMD requires allocating a page, thus let's fail
> @@ -583,9 +571,6 @@ void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_l
>  			break;
>  	}
>  
> -	if (!nr_to_optimize)
> -		return;
> -
>  	flush_tlb_all();

[Severity: Critical]
This problem wasn't introduced by this patch, but can deferring the TLB
flush across PMD splits here race with a concurrent single-folio vmemmap
optimization [2] and remap PTEs before the 2MB PMD split is
flushed from the TLB?

A single 2MB vmemmap PMD covers 64 2MB folios.  While
hugetlb_vmemmap_optimize_folios() splits PMDs with VMEMMAP_DEFER_TLB_FLUSH
during bulk pool resize or demotion, a concurrent
hugetlb_vmemmap_optimize_folio() (such as from
alloc_surplus_hugetlb_folio()) can optimize another folio covered by the
same vmemmap PMD:

CPU0                                  CPU1
hugetlb_vmemmap_optimize_folios()
  hugetlb_vmemmap_pteify_folio()
    ...
      vmemmap_split_pmd()
        pmd_populate_kernel()
        // skips TLB flush
                                      hugetlb_vmemmap_optimize_folio()
                                        ...
                                          vmemmap_pmd_entry()
                                            // !pmd_leaf(*pmd), returns 0
                                          vmemmap_optimize_pte()
                                            set_pte_at()
                                        free_vmemmap_page_list()
                                          <-- old vmemmap pages freed with
                                              stale 2MB PMD TLB entry live
  flush_tlb_all()

Because CPU0 has already replaced the leaf PMD under
init_mm.page_table_lock, CPU1 sees !pmd_leaf(*pmd) in vmemmap_pmd_entry()
and proceeds directly to vmemmap_optimize_pte() without a PMD-range TLB
flush.  CPU1 then installs 4KB PTEs pointing to different physical pages,
flushes only its own folio's vmemmap range, and frees the displaced
vmemmap pages while stale 2MB PMD TLB entries remain active, which can
cause overlapping-TLB machine checks or use-after-free writes to the freed
vmemmap pages.

[1] https://sashiko.dev/#/bug/linux-18e93f17-529b-4d84-ba44-a41829f4be64
[2] https://sashiko.dev/#/bug/linux-fbdc3410-a4cf-457e-88e6-684380fbadfe

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261109095807.41547-1-songmuchun@bytedance.com?part=5