[PATCH v8 00/15] Use generic_file_read_iter() in hugetlbfs

[PATCH v8 00/15] Use generic_file_read_iter() in hugetlbfs

[Matthew Wilcox (Oracle)]

This set of patches are separated out from Jane's earlier series.
https://lore.kernel.org/linux-mm/20260617172534.1740152-1-jane.chu@oracle.com/

We need three things to use filemap_read_iter() instead of a hugetlbfs
specific read_iter: hwpoison handling, treating gaps in the pagecache as
holes, and having hugetlb set the folio size in the mapping.  hwopison
handling turned out to need a lot of prep work; the other two things
are fairly straightforward.

Sashiko complains that the implementation of AS_AUTHORITATIVE is
incomplete, but hugetlbfs doesn't use any of that functionality.
We'll add it when we need it, either when hugetlbfs uses it, or when we
convert shmem to use filemap.

v7 here:
https://lore.kernel.org/linux-mm/20260728204409.3396238-1-willy@infradead.org/

Changes from v7:
 - Split preventing freeing of a hugetlb folio into its own patch
 - Add hugetlb_lock_irq() abstraction to fix compile bug with HUGETLB disabled
 - Use is_page_poisoned() in kpageflags
 - Collect tags from various people

Jane Chu (2):
  filemap: Add hwpoison handling to filemap_read()
  hugetlb: replace hugetlbfs_read_iter() with generic_file_read_iter()

Matthew Wilcox (Oracle) (13):
  memory-failure: Fix hardware poison check in unpoison_memory() again
  memory-failure: Prevent hugetlb freeing during unpoisoning
  mm: Rename folio_contain_hwpoison_page() to folio_has_hwpoison_page()
  hugetlb: Mark some function arguments as const
  guest_memfd: Use folio_has_hwpoisoned_page()
  kpageflags: Use is_page_hwpoison() to set KPF_HWPOISON
  hugetlb: Move poison to pages before clearing hugetlb page type
  hugetlb: Use the has_hwpoisoned flag
  mm: Remove locking mf_mutex in is_raw_hwpoison_page_in_hugepage()
  mm: Check individual hugetlb pages for poison
  filemap: Remove checks in mapping_set_folio_order_range()
  hugetlb: Set mapping folio order
  filemap: Add support for authoritative mappings

 fs/Kconfig                 |   2 +-
 fs/hugetlbfs/inode.c       | 123 +++--------------------
 fs/proc/page.c             |   8 +-
 include/linux/hugetlb.h    |  43 +++++---
 include/linux/mm_types.h   |   4 +-
 include/linux/page-flags.h |  63 ++++++++++--
 include/linux/pagemap.h    |  45 +++++----
 mm/Kconfig                 |   6 +-
 mm/filemap.c               |  51 +++++++++-
 mm/huge_memory.c           |   2 +-
 mm/hugetlb.c               |  41 ++++----
 mm/memory-failure.c        | 197 +++++++++++++++++++++++++------------
 mm/memory_hotplug.c        |   2 +-
 mm/rmap.c                  |  43 +++++---
 mm/shmem.c                 |   2 +-
 mm/vmscan.c                |   2 +-
 virt/kvm/guest_memfd.c     |   2 +-
 17 files changed, 367 insertions(+), 269 deletions(-)

-- 
2.47.3

[PATCH v8 01/15] memory-failure: Fix hardware poison check in unpoison_memory() again

[Matthew Wilcox (Oracle)]

The earlier patch (6c54312f9689) that attempted to fix unpoison_memory()
was incorrect for hugetlb folios.  Before a6fddef49eef, we checked
the head page for poison (which was correct for hugetlb and incorrect
for THP).  We are currently incapable of unpoisoning pages (other than
the first page) in a hugetlb folio.

Use is_page_hwpoison() which handles hugetlb pages sufficiently well for
this purpose.  This is racy as we don't have a reference on the folio at
this point, but fixing that properly requires deeper surgery and this
is a CAP_SYS_ADMIN path only so this will do for the moment and can be
easily backported.  It's no worse than the situation before a6fddef49eef.

The other bug in a6fddef49eef is that we currently clear the HWPoison
flag on the precise page, which is wrong for hugetlb.  Fix that too.

Fixes: a6fddef49eef ("mm/memory-failure: convert unpoison_memory() to folios")
Cc: stable@vger.kernel.org
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
---
 mm/memory-failure.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 51508a55c405..944e6e1d4971 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -2695,8 +2695,8 @@ int unpoison_memory(unsigned long pfn)
 		goto unlock_mutex;
 	}
 
-	if (!PageHWPoison(p)) {
-		unpoison_pr_info("%#lx: page was already unpoisoned\n",
+	if (!is_page_hwpoison(p)) {
+		unpoison_pr_info("%#lx: page is not poisoned\n",
 				 pfn, &unpoison_rs);
 		goto unlock_mutex;
 	}
@@ -2748,6 +2748,7 @@ int unpoison_memory(unsigned long pfn)
 				folio_put(folio);
 				goto unlock_mutex;
 			}
+			p = &folio->page;
 		}
 
 		folio_put(folio);
-- 
2.47.3

[PATCH v8 02/15] memory-failure: Prevent hugetlb freeing during unpoisoning

[Matthew Wilcox (Oracle)]

If we fail to get a reference on the hugetlb folio, then it might be
freed as we operate on it.  Prevent the freeing and the attendant races
around manipulation of the raw_hwp list by holding the hugetlb_lock,
which is also held by the hugetlb code when freeing hugetlb folios.

Fixes: ac5fcde0a96a ("mm, hwpoison: make unpoison aware of raw error info in hwpoisoned hugepage")
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
---
 include/linux/hugetlb.h | 19 +++++++++++++++++++
 mm/memory-failure.c     |  6 +++++-
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 2abaf99321e9..50eab2c23299 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -110,6 +110,17 @@ extern struct resv_map *resv_map_alloc(void);
 void resv_map_release(struct kref *ref);
 
 extern spinlock_t hugetlb_lock;
+
+static inline void hugetlb_lock_irq(void)
+{
+	spin_lock_irq(&hugetlb_lock);
+}
+
+static inline void hugetlb_unlock_irq(void)
+{
+	spin_unlock_irq(&hugetlb_lock);
+}
+
 extern int hugetlb_max_hstate __read_mostly;
 #define for_each_hstate(h) \
 	for ((h) = hstates; (h) < &hstates[hugetlb_max_hstate]; (h)++)
@@ -279,6 +290,14 @@ unsigned int arch_hugetlb_cma_order(void);
 
 #else /* !CONFIG_HUGETLB_PAGE */
 
+static inline void hugetlb_lock_irq(void)
+{
+}
+
+static inline void hugetlb_unlock_irq(void)
+{
+}
+
 static inline void hugetlb_dup_vma_private(struct vm_area_struct *vma)
 {
 }
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 944e6e1d4971..1dd0e7b99bb1 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -2725,13 +2725,17 @@ int unpoison_memory(unsigned long pfn)
 
 	ghp = get_hwpoison_page(p, MF_UNPOISON);
 	if (!ghp) {
+		hugetlb_lock_irq();
 		if (folio_test_hugetlb(folio)) {
 			huge = true;
 			count = folio_free_raw_hwp(folio, false);
-			if (count == 0)
+			if (count == 0) {
+				hugetlb_unlock_irq();
 				goto unlock_mutex;
+			}
 		}
 		ret = folio_test_clear_hwpoison(folio) ? 0 : -EBUSY;
+		hugetlb_unlock_irq();
 	} else if (ghp < 0) {
 		if (ghp == -EHWPOISON) {
 			ret = put_page_back_buddy(p) ? 0 : -EBUSY;
-- 
2.47.3

[PATCH v8 03/15] mm: Rename folio_contain_hwpoison_page() to folio_has_hwpoison_page()

[Matthew Wilcox (Oracle)]

This is a better name for this predicate.  Also make the argument const
to allow other callers to be made const.  No functional change.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Usama Arif <usama.arif@linux.dev>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
Reviewed-by: Jane Chu <jane.chu@oracle.com>
---
 include/linux/page-flags.h | 4 ++--
 mm/huge_memory.c           | 2 +-
 mm/memory_hotplug.c        | 2 +-
 mm/shmem.c                 | 2 +-
 mm/vmscan.c                | 2 +-
 5 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 7223f6f4e2b4..4185a03a45cf 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -1083,10 +1083,10 @@ static inline bool is_page_hwpoison(const struct page *page)
 	return folio_test_hugetlb(folio) && PageHWPoison(&folio->page);
 }
 
-static inline bool folio_contain_hwpoisoned_page(struct folio *folio)
+static inline bool folio_has_hwpoisoned_page(const struct folio *folio)
 {
 	return folio_test_hwpoison(folio) ||
-	    (folio_test_large(folio) && folio_test_has_hwpoisoned(folio));
+	       (folio_test_large(folio) && folio_test_has_hwpoisoned(folio));
 }
 
 bool is_free_buddy_page(const struct page *page);
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 2bccb0a53a0a..353340309107 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -4413,7 +4413,7 @@ static bool thp_underused(struct folio *folio)
 	if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
 		return false;
 
-	if (folio_contain_hwpoisoned_page(folio))
+	if (folio_has_hwpoisoned_page(folio))
 		return false;
 
 	for (i = 0; i < folio_nr_pages(folio); i++) {
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 7ac19fab2263..33007e8e546a 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -1813,7 +1813,7 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 		if (folio_test_large(folio))
 			pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1;
 
-		if (folio_contain_hwpoisoned_page(folio)) {
+		if (folio_has_hwpoisoned_page(folio)) {
 			/*
 			 * unmap_poisoned_folio() cannot handle large folios
 			 * in all cases yet.
diff --git a/mm/shmem.c b/mm/shmem.c
index b51f83c970bb..492c9bef3620 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -3240,7 +3240,7 @@ shmem_write_begin(const struct kiocb *iocb, struct address_space *mapping,
 	if (ret)
 		return ret;
 
-	if (folio_contain_hwpoisoned_page(folio)) {
+	if (folio_has_hwpoisoned_page(folio)) {
 		folio_unlock(folio);
 		folio_put(folio);
 		return -EIO;
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 35c3bb15ae96..70e54ace0e74 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1086,7 +1086,7 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 		if (!folio_trylock(folio))
 			goto keep;
 
-		if (folio_contain_hwpoisoned_page(folio)) {
+		if (folio_has_hwpoisoned_page(folio)) {
 			/*
 			 * unmap_poisoned_folio() can't handle large
 			 * folio, just skip it. memory_failure() will
-- 
2.47.3

[PATCH v8 04/15] hugetlb: Mark some function arguments as const

[Matthew Wilcox (Oracle)]

These functions do not modify their folio argument, so we can mark them
as being const which will allow for some minor optimisations and let us
mark other function arguments as being const in the future.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Usama Arif <usama.arif@linux.dev>
Reviewed-by: Gregory Price <gourry@gourry.net>
Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
---
 include/linux/hugetlb.h | 19 +++++++++----------
 mm/hugetlb.c            |  2 +-
 2 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 50eab2c23299..e200c93088bc 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -176,7 +176,7 @@ bool hugetlbfs_pagecache_present(struct hstate *h,
 				 struct vm_area_struct *vma,
 				 unsigned long address);
 
-struct address_space *hugetlb_folio_mapping_lock_write(struct folio *folio);
+struct address_space *hugetlb_folio_mapping_lock_write(const struct folio *folio);
 
 extern int movable_gigantic_pages __read_mostly;
 extern int sysctl_hugetlb_shm_group __read_mostly;
@@ -311,8 +311,7 @@ static inline unsigned long hugetlb_total_pages(void)
 	return 0;
 }
 
-static inline struct address_space *hugetlb_folio_mapping_lock_write(
-							struct folio *folio)
+static inline struct address_space *hugetlb_folio_mapping_lock_write(const struct folio *folio)
 {
 	return NULL;
 }
@@ -614,8 +613,8 @@ enum hugetlb_page_flags {
 #ifdef CONFIG_HUGETLB_PAGE
 #define TESTHPAGEFLAG(uname, flname)				\
 static __always_inline						\
-bool folio_test_hugetlb_##flname(struct folio *folio)		\
-	{	void *private = &folio->private;		\
+bool folio_test_hugetlb_##flname(const struct folio *folio)	\
+	{	const void *private = &folio->private;		\
 		return test_bit(HPG_##flname, private);		\
 	}
 
@@ -635,7 +634,7 @@ void folio_clear_hugetlb_##flname(struct folio *folio)		\
 #else
 #define TESTHPAGEFLAG(uname, flname)				\
 static inline bool						\
-folio_test_hugetlb_##flname(struct folio *folio)		\
+folio_test_hugetlb_##flname(const struct folio *folio)		\
 	{ return 0; }
 
 #define SETHPAGEFLAG(uname, flname)				\
@@ -746,7 +745,7 @@ static inline struct hugepage_subpool *subpool_inode(struct inode *inode)
 	return HUGETLBFS_SB(inode->i_sb)->spool;
 }
 
-static inline struct hugepage_subpool *hugetlb_folio_subpool(struct folio *folio)
+static inline struct hugepage_subpool *hugetlb_folio_subpool(const struct folio *folio)
 {
 	return folio->_hugetlb_subpool;
 }
@@ -876,7 +875,7 @@ static inline bool arch_has_huge_bootmem_alloc(void)
 }
 #endif
 
-static inline struct hstate *folio_hstate(struct folio *folio)
+static inline struct hstate *folio_hstate(const struct folio *folio)
 {
 	VM_BUG_ON_FOLIO(!folio_test_hugetlb(folio), folio);
 	return size_to_hstate(folio_size(folio));
@@ -1107,7 +1106,7 @@ static inline unsigned long huge_page_mask_align(struct file *file)
 	return 0;
 }
 
-static inline struct hugepage_subpool *hugetlb_folio_subpool(struct folio *folio)
+static inline struct hugepage_subpool *hugetlb_folio_subpool(const struct folio *folio)
 {
 	return NULL;
 }
@@ -1176,7 +1175,7 @@ static inline struct hstate *hstate_vma(struct vm_area_struct *vma)
 	return NULL;
 }
 
-static inline struct hstate *folio_hstate(struct folio *folio)
+static inline struct hstate *folio_hstate(const struct folio *folio)
 {
 	return NULL;
 }
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 571212b80835..cc8d98399913 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1775,7 +1775,7 @@ void init_new_hugetlb_folio(struct folio *folio)
  * stable.  Due to locking order, we can only trylock_write.  If we can
  * not get the lock, simply return NULL to caller.
  */
-struct address_space *hugetlb_folio_mapping_lock_write(struct folio *folio)
+struct address_space *hugetlb_folio_mapping_lock_write(const struct folio *folio)
 {
 	struct address_space *mapping = folio_mapping(folio);
 
-- 
2.47.3

[PATCH v8 05/15] guest_memfd: Use folio_has_hwpoisoned_page()

[Matthew Wilcox (Oracle)]

folio_test_hwpoison() only works on hugetlb folios today and will give
the wrong answer for other large folios.  folio_has_hwpoisoned_page()
works on both.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Gregory Price <gourry@gourry.net>
---
 virt/kvm/guest_memfd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index db57c5766ab6..94eedcc7f1a9 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -774,7 +774,7 @@ static struct folio *__kvm_gmem_get_pfn(struct file *file,
 	if (IS_ERR(folio))
 		return folio;
 
-	if (folio_test_hwpoison(folio)) {
+	if (folio_has_hwpoisoned_page(folio)) {
 		folio_unlock(folio);
 		folio_put(folio);
 		return ERR_PTR(-EHWPOISON);
-- 
2.47.3

[PATCH v8 06/15] kpageflags: Use is_page_hwpoison() to set KPF_HWPOISON

[Matthew Wilcox (Oracle)]

Instead of knowing how hugetlb handles hwpoison, just ask
is_page_hwpoison().  This gives us the flexibility to change how hwpoison
is handled without updating this function in the future.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 fs/proc/page.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/fs/proc/page.c b/fs/proc/page.c
index 7d9387143435..28c9a7b724f3 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -233,12 +233,8 @@ u64 stable_page_flags(const struct page *page)
 	u |= kpf_copy_bit(k, KPF_UNEVICTABLE,	PG_unevictable);
 	u |= kpf_copy_bit(k, KPF_MLOCKED,	PG_mlocked);
 
-#ifdef CONFIG_MEMORY_FAILURE
-	if (u & (1 << KPF_HUGE))
-		u |= kpf_copy_bit(k, KPF_HWPOISON,	PG_hwpoison);
-	else
-		u |= kpf_copy_bit(ps.page_snapshot.flags.f, KPF_HWPOISON, PG_hwpoison);
-#endif
+	if (is_page_hwpoison(page))
+		u |= KPF_HWPOISON;
 
 	u |= kpf_copy_bit(k, KPF_RESERVED,	PG_reserved);
 	u |= kpf_copy_bit(k, KPF_OWNER_2,	PG_owner_2);
-- 
2.47.3

[PATCH v8 07/15] hugetlb: Move poison to pages before clearing hugetlb page type

[Matthew Wilcox (Oracle)]

For is_page_hwpoison() to be reliable, we need to set the hwpoison
flag on each page before clearing the hugetlb flag.  Introduce
folio_clear_hugetlb() to make this easier.  This does not by
itself make is_page_hwpoison() reliable; see subsequent patches.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
---
 mm/hugetlb.c | 31 ++++++++++++++++++-------------
 1 file changed, 18 insertions(+), 13 deletions(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index cc8d98399913..47403d02be88 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1381,6 +1381,18 @@ static struct folio *alloc_gigantic_frozen_folio(int order, gfp_t gfp_mask, int
 }
 #endif
 
+static void folio_clear_hugetlb(struct folio *folio)
+{
+	/*
+	 * Move HWPoison flag to each error page
+	 * which makes any healthy pages reusable.
+	 */
+	if (unlikely(folio_test_hwpoison(folio)))
+		folio_clear_hugetlb_hwpoison(folio);
+
+	__folio_clear_hugetlb(folio);
+}
+
 /*
  * Remove hugetlb folio from lists.
  * If vmemmap exists for the folio, clear the hugetlb flag so that the
@@ -1419,7 +1431,7 @@ void remove_hugetlb_folio(struct hstate *h, struct folio *folio,
 	 * to tail struct pages.
 	 */
 	if (!folio_test_hugetlb_vmemmap_optimized(folio))
-		__folio_clear_hugetlb(folio);
+		folio_clear_hugetlb(folio);
 
 	h->nr_huge_pages--;
 	h->nr_huge_pages_node[nid]--;
@@ -1493,17 +1505,10 @@ static void __update_and_free_hugetlb_folio(struct hstate *h,
 	 */
 	if (folio_test_hugetlb(folio)) {
 		spin_lock_irq(&hugetlb_lock);
-		__folio_clear_hugetlb(folio);
+		folio_clear_hugetlb(folio);
 		spin_unlock_irq(&hugetlb_lock);
 	}
 
-	/*
-	 * Move PageHWPoison flag from head page to the raw error pages,
-	 * which makes any healthy subpages reusable.
-	 */
-	if (unlikely(folio_test_hwpoison(folio)))
-		folio_clear_hugetlb_hwpoison(folio);
-
 	VM_BUG_ON_FOLIO(folio_ref_count(folio), folio);
 	if (folio_test_hugetlb_cma(folio))
 		hugetlb_cma_free_frozen_folio(folio);
@@ -1595,7 +1600,7 @@ static void bulk_vmemmap_restore_error(struct hstate *h,
 		list_for_each_entry_safe(folio, t_folio, non_hvo_folios, lru) {
 			list_del(&folio->lru);
 			spin_lock_irq(&hugetlb_lock);
-			__folio_clear_hugetlb(folio);
+			folio_clear_hugetlb(folio);
 			spin_unlock_irq(&hugetlb_lock);
 			update_and_free_hugetlb_folio(h, folio, false);
 			cond_resched();
@@ -1620,7 +1625,7 @@ static void bulk_vmemmap_restore_error(struct hstate *h,
 			} else {
 				list_del(&folio->lru);
 				spin_lock_irq(&hugetlb_lock);
-				__folio_clear_hugetlb(folio);
+				folio_clear_hugetlb(folio);
 				spin_unlock_irq(&hugetlb_lock);
 				update_and_free_hugetlb_folio(h, folio, false);
 				cond_resched();
@@ -1653,14 +1658,14 @@ static void update_and_free_pages_bulk(struct hstate *h,
 	 * should only be pages on the non_hvo_folios list.
 	 * Do note that the non_hvo_folios list could be empty.
 	 * Without HVO enabled, ret will be 0 and there is no need to call
-	 * __folio_clear_hugetlb as this was done previously.
+	 * folio_clear_hugetlb as this was done previously.
 	 */
 	VM_WARN_ON(!list_empty(folio_list));
 	VM_WARN_ON(ret < 0);
 	if (!list_empty(&non_hvo_folios) && ret) {
 		spin_lock_irq(&hugetlb_lock);
 		list_for_each_entry(folio, &non_hvo_folios, lru)
-			__folio_clear_hugetlb(folio);
+			folio_clear_hugetlb(folio);
 		spin_unlock_irq(&hugetlb_lock);
 	}
 
-- 
2.47.3

[PATCH v8 08/15] hugetlb: Use the has_hwpoisoned flag

[Matthew Wilcox (Oracle)]

Other large folios use the has_hwpoisoned flag.  Convert hugetlb to match.
This will help us use the per-page hwpoison flag in the future.

Also introduce a folio_test_huge_poison().  This has exactly the same
meaning as folio_test_has_hwpoisoned() but can be used when we don't have
a reference to the folio containing the page.  folio_test_has_hwpoisoned()
can race with folio splitting / reallocation and trip the assertions
in const_folio_flags().

This closes a gap where a page in a previously-poisoned hugetlb folio
could be observed to not have the hwpoison bit set.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 fs/Kconfig                 |  2 +-
 fs/hugetlbfs/inode.c       |  2 +-
 include/linux/page-flags.h | 46 ++++++++++++++++++++++++------
 mm/Kconfig                 |  6 +++-
 mm/hugetlb.c               | 10 +++----
 mm/memory-failure.c        | 58 +++++++++++++++++++++++++++-----------
 mm/rmap.c                  | 43 +++++++++++++++++-----------
 7 files changed, 119 insertions(+), 48 deletions(-)

diff --git a/fs/Kconfig b/fs/Kconfig
index cf6ae64776e6..eddac4ed214b 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -272,7 +272,7 @@ endif # HUGETLBFS
 
 config HUGETLB_PAGE
 	def_bool HUGETLBFS
-	select XARRAY_MULTI
+	select LARGE_FOLIO
 
 config HUGETLB_PAGE_OPTIMIZE_VMEMMAP
 	def_bool HUGETLB_PAGE
diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 216e1a0dd0b2..fbac554886c3 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -258,7 +258,7 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to)
 		} else {
 			folio_unlock(folio);
 
-			if (!folio_test_hwpoison(folio))
+			if (!folio_test_has_hwpoisoned(folio))
 				want = nr;
 			else {
 				/*
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 4185a03a45cf..4fab3fbfd430 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -893,14 +893,19 @@ static inline int PageTransCompound(const struct page *page)
 TESTPAGEFLAG_FALSE(TransCompound, transcompound)
 #endif
 
-#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_TRANSPARENT_HUGEPAGE)
+#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_LARGE_FOLIO)
 /*
- * PageHasHWPoisoned indicates that at least one subpage is hwpoisoned in the
- * compound page.
+ * folio_has_hwpoisoned indicates that at least one page is hwpoisoned in the
+ * folio.  That page will usually also have the HWPoison flag set, but this
+ * is not possible for folios which have HVO (see memory-failure for the
+ * scheme used in that case).  You probably don't want to call this directly;
+ * use folio_has_hwpoisoned_page() instead.
  *
  * This flag is set by hwpoison handler.  Cleared by THP split or free page.
  */
 FOLIO_FLAG(has_hwpoisoned, FOLIO_SECOND_PAGE)
+FOLIO_TEST_SET_FLAG(has_hwpoisoned, FOLIO_SECOND_PAGE)
+FOLIO_TEST_CLEAR_FLAG(has_hwpoisoned, FOLIO_SECOND_PAGE)
 #else
 FOLIO_FLAG_FALSE(has_hwpoisoned)
 #endif
@@ -1041,8 +1046,29 @@ PAGE_TYPE_OPS(Slab, slab, slab)
 
 #ifdef CONFIG_HUGETLB_PAGE
 FOLIO_TYPE_OPS(hugetlb, hugetlb)
+
+#ifdef CONFIG_MEMORY_FAILURE
+static inline bool folio_test_huge_poison(const struct folio *folio)
+{
+	return (READ_ONCE(folio->page.page_type) >> 23) ==
+		((PGTY_hugetlb << 1) | 1);
+}
+
+static inline void folio_set_huge_poison(struct folio *folio)
+{
+	folio->page.page_type |= (1 << 23);
+}
+
+static inline void folio_clear_huge_poison(struct folio *folio)
+{
+	folio->page.page_type &= ~(1 << 23);
+}
+#else
+FOLIO_TEST_FLAG_FALSE(huge_poison)
+#endif
 #else
 FOLIO_TEST_FLAG_FALSE(hugetlb)
+FOLIO_TEST_FLAG_FALSE(huge_poison)
 #endif
 
 PAGE_TYPE_OPS(Zsmalloc, zsmalloc, zsmalloc)
@@ -1069,9 +1095,10 @@ static inline bool PageHuge(const struct page *page)
 }
 
 /*
- * Check if a page is currently marked HWPoisoned. Note that this check is
- * best effort only and inherently racy: there is no way to synchronize with
- * failing hardware.
+ * Check if a page is currently marked HWPoisoned.  This check is best
+ * effort only and inherently racy: there is no way to synchronize with
+ * failing hardware.  The caller may not have a refcount on the folio
+ * containing the page, so we must be careful to not trip any assertions.
  */
 static inline bool is_page_hwpoison(const struct page *page)
 {
@@ -1080,12 +1107,15 @@ static inline bool is_page_hwpoison(const struct page *page)
 	if (PageHWPoison(page))
 		return true;
 	folio = page_folio(page);
-	return folio_test_hugetlb(folio) && PageHWPoison(&folio->page);
+	if (folio_test_huge_poison(folio))
+		return true;
+	/* In case we raced with hugetlb transferring flags */
+	return PageHWPoison(page);
 }
 
 static inline bool folio_has_hwpoisoned_page(const struct folio *folio)
 {
-	return folio_test_hwpoison(folio) ||
+	return PageHWPoison(&folio->page) ||
 	       (folio_test_large(folio) && folio_test_has_hwpoisoned(folio));
 }
 
diff --git a/mm/Kconfig b/mm/Kconfig
index 9e0ca4824905..e666dd14ca0c 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -843,11 +843,15 @@ config PERSISTENT_HUGE_ZERO_FOLIO
 config MM_ID
 	def_bool n
 
+config LARGE_FOLIO
+	def_bool n
+	select XARRAY_MULTI
+
 menuconfig TRANSPARENT_HUGEPAGE
 	bool "Transparent Hugepage Support"
 	depends on HAVE_ARCH_TRANSPARENT_HUGEPAGE && !PREEMPT_RT
 	select COMPACTION
-	select XARRAY_MULTI
+	select LARGE_FOLIO
 	select MM_ID
 	help
 	  Transparent Hugepages allows the kernel to use huge pages and
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 47403d02be88..40ae967b9ecc 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1255,7 +1255,7 @@ static struct folio *dequeue_hugetlb_folio_node_exact(struct hstate *h,
 		if (pin && !folio_is_longterm_pinnable(folio))
 			continue;
 
-		if (folio_test_hwpoison(folio))
+		if (folio_test_has_hwpoisoned(folio))
 			continue;
 
 		if (is_migrate_isolate_page(&folio->page))
@@ -1387,7 +1387,7 @@ static void folio_clear_hugetlb(struct folio *folio)
 	 * Move HWPoison flag to each error page
 	 * which makes any healthy pages reusable.
 	 */
-	if (unlikely(folio_test_hwpoison(folio)))
+	if (unlikely(folio_test_has_hwpoisoned(folio)))
 		folio_clear_hugetlb_hwpoison(folio);
 
 	__folio_clear_hugetlb(folio);
@@ -4003,7 +4003,7 @@ long demote_pool_huge_page(struct hstate *src, nodemask_t *nodes_allowed,
 		struct folio *folio, *next;
 
 		list_for_each_entry_safe(folio, next, &src->hugepage_freelists[node], lru) {
-			if (folio_test_hwpoison(folio))
+			if (folio_test_has_hwpoisoned(folio))
 				continue;
 
 			remove_hugetlb_folio(src, folio, false);
@@ -5814,7 +5814,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping,
 		 * don't have hwpoisoned swap entry for errored virtual address.
 		 * So we need to block hugepage fault by PG_hwpoison bit check.
 		 */
-		if (unlikely(folio_test_hwpoison(folio))) {
+		if (unlikely(folio_test_has_hwpoisoned(folio))) {
 			ret = VM_FAULT_HWPOISON_LARGE |
 				VM_FAULT_SET_HINDEX(hstate_index(h));
 			goto backout_unlocked;
@@ -6323,7 +6323,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
 	ptl = huge_pte_lock(h, dst_mm, dst_pte);
 
 	ret = -EIO;
-	if (folio_test_hwpoison(folio))
+	if (folio_test_has_hwpoisoned(folio))
 		goto out_release_unlock;
 
 	ret = -EEXIST;
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 1dd0e7b99bb1..714e1b398f2c 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1829,7 +1829,7 @@ bool is_raw_hwpoison_page_in_hugepage(struct page *page)
 	struct folio *folio = page_folio(page);
 	bool ret = false;
 
-	if (!folio_test_hwpoison(folio))
+	if (!folio_test_has_hwpoisoned(folio))
 		return false;
 
 	if (!folio_test_hugetlb(folio))
@@ -1881,6 +1881,24 @@ static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag)
 #define	MF_HUGETLB_FOLIO_PRE_POISONED	3	/* folio already poisoned */
 #define	MF_HUGETLB_PAGE_PRE_POISONED	4	/* exact page already poisoned */
 #define	MF_HUGETLB_RETRY		5	/* hugepage is busy, retry */
+
+static inline int hugetlb_set_poison(struct folio *folio)
+{
+	if (folio_test_set_has_hwpoisoned(folio))
+		return MF_HUGETLB_FOLIO_PRE_POISONED;
+	folio_set_huge_poison(folio);
+	return 0;
+}
+
+static inline int hugetlb_clear_poison(struct folio *folio)
+{
+	if (!folio_test_has_hwpoisoned(folio))
+		return -EBUSY;
+	folio_clear_huge_poison(folio);
+	folio_clear_has_hwpoisoned(folio);
+	return 0;
+}
+
 /*
  * Set hugetlb folio as hwpoisoned, update folio private raw hwpoison list
  * to keep track of the poisoned pages.
@@ -1890,12 +1908,12 @@ static int hugetlb_update_hwpoison(struct folio *folio, struct page *page)
 	struct llist_head *head;
 	struct raw_hwp_page *raw_hwp;
 	struct raw_hwp_page *p;
-	int ret = folio_test_set_hwpoison(folio) ? MF_HUGETLB_FOLIO_PRE_POISONED : 0;
+	int ret = hugetlb_set_poison(folio);
 
 	/*
 	 * Once the hwpoison hugepage has lost reliable raw error info,
-	 * there is little meaning to keep additional error info precisely,
-	 * so skip to add additional raw error info.
+	 * there is no point in keeping additional error info precisely,
+	 * so skip adding additional raw error info.
 	 */
 	if (folio_test_hugetlb_raw_hwp_unreliable(folio))
 		return MF_HUGETLB_FOLIO_PRE_POISONED;
@@ -1950,8 +1968,8 @@ void folio_clear_hugetlb_hwpoison(struct folio *folio)
 		return;
 	if (folio_test_hugetlb_vmemmap_optimized(folio))
 		return;
-	folio_clear_hwpoison(folio);
 	folio_free_raw_hwp(folio, true);
+	folio_clear_has_hwpoisoned(folio);
 }
 
 static int get_huge_page_for_hwpoison(unsigned long pfn, int flags,
@@ -2104,6 +2122,11 @@ static inline unsigned long folio_free_raw_hwp(struct folio *folio, bool flag)
 {
 	return 0;
 }
+
+static inline int hugetlb_clear_poison(struct folio *folio)
+{
+	return 0;
+}
 #endif	/* CONFIG_HUGETLB_PAGE */
 
 /* Drop the extra refcount in case we come from madvise() */
@@ -2733,8 +2756,10 @@ int unpoison_memory(unsigned long pfn)
 				hugetlb_unlock_irq();
 				goto unlock_mutex;
 			}
+			ret = hugetlb_clear_poison(folio);
+		} else {
+			ret = TestClearPageHWPoison(p) ? 0 : -EBUSY;
 		}
-		ret = folio_test_clear_hwpoison(folio) ? 0 : -EBUSY;
 		hugetlb_unlock_irq();
 	} else if (ghp < 0) {
 		if (ghp == -EHWPOISON) {
@@ -2744,17 +2769,18 @@ int unpoison_memory(unsigned long pfn)
 			unpoison_pr_info("%#lx: failed to grab page\n",
 					 pfn, &unpoison_rs);
 		}
-	} else {
-		if (folio_test_hugetlb(folio)) {
-			huge = true;
-			count = folio_free_raw_hwp(folio, false);
-			if (count == 0) {
-				folio_put(folio);
-				goto unlock_mutex;
-			}
-			p = &folio->page;
+	} else if (folio_test_hugetlb(folio)) {
+		huge = true;
+		count = folio_free_raw_hwp(folio, false);
+		if (count == 0) {
+			folio_put(folio);
+			goto unlock_mutex;
 		}
-
+		ret = hugetlb_clear_poison(folio);
+		folio_put(folio);
+		if (!ret)
+			folio_put(folio);
+	} else {
 		folio_put(folio);
 		if (TestClearPageHWPoison(p)) {
 			folio_put(folio);
diff --git a/mm/rmap.c b/mm/rmap.c
index 1c77d5dc06e9..fd19a0bfbfe7 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -1978,6 +1978,22 @@ static inline unsigned int folio_unmap_pte_batch(struct folio *folio,
 				     FPB_RESPECT_WRITE | FPB_RESPECT_SOFT_DIRTY);
 }
 
+/*
+ * Since we cannot split a hugetlb folio, we want to insert a poison
+ * entry into the page table for the whole folio even if only one page
+ * is poisoned.  Otherwise, we've split down to the PTE level and we only
+ * want to poison the precise page
+ */
+static bool ttu_create_hwpoison(const struct folio *folio,
+		const struct page *page, enum ttu_flags flags)
+{
+	if (!(flags & TTU_HWPOISON))
+		return false;
+	if (folio_test_hugetlb(folio))
+		return folio_test_has_hwpoisoned(folio);
+	return PageHWPoison(page);
+}
+
 /*
  * @arg: enum ttu_flags will be passed to this argument
  */
@@ -1993,7 +2009,6 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
 	enum ttu_flags flags = (enum ttu_flags)(long)arg;
 	unsigned long nr_pages = 1, end_addr;
 	unsigned long pfn;
-	unsigned long hsz = 0;
 	int ptes = 0;
 
 	/*
@@ -2023,9 +2038,6 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
 		 */
 		adjust_range_if_pmd_sharing_possible(vma, &range.start,
 						     &range.end);
-
-		/* We need the huge page size for set_huge_pte_at() */
-		hsz = huge_page_size(hstate_vma(vma));
 	}
 	mmu_notifier_invalidate_range_start(&range);
 
@@ -2121,7 +2133,8 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
 			 * The try_to_unmap() is only passed a hugetlb page
 			 * in the case where the hugetlb page is poisoned.
 			 */
-			VM_BUG_ON_PAGE(!PageHWPoison(subpage), subpage);
+			VM_BUG_ON_FOLIO(!folio_has_hwpoisoned_page(folio),
+					folio);
 			/*
 			 * huge_pmd_unshare may unmap an entire PMD page.
 			 * There is no way of knowing exactly which PMDs may
@@ -2200,12 +2213,12 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
 		/* Update high watermark before we lower rss */
 		update_hiwater_rss(mm);
 
-		if (PageHWPoison(subpage) && (flags & TTU_HWPOISON)) {
+		if (ttu_create_hwpoison(folio, subpage, flags)) {
 			pteval = swp_entry_to_pte(make_hwpoison_entry(subpage));
 			if (folio_test_hugetlb(folio)) {
 				hugetlb_count_sub(folio_nr_pages(folio), mm);
 				set_huge_pte_at(mm, address, pvmw.pte, pteval,
-						hsz);
+						folio_size(folio));
 			} else {
 				dec_mm_counter(mm, mm_counter(folio));
 				set_pte_at(mm, address, pvmw.pte, pteval);
@@ -2423,7 +2436,6 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
 	struct mmu_notifier_range range;
 	enum ttu_flags flags = (enum ttu_flags)(long)arg;
 	unsigned long pfn;
-	unsigned long hsz = 0;
 
 	/*
 	 * When racing against e.g. zap_pte_range() on another cpu,
@@ -2452,9 +2464,6 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
 		 */
 		adjust_range_if_pmd_sharing_possible(vma, &range.start,
 						     &range.end);
-
-		/* We need the huge page size for set_huge_pte_at() */
-		hsz = huge_page_size(hstate_vma(vma));
 	}
 	mmu_notifier_invalidate_range_start(&range);
 
@@ -2607,14 +2616,14 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
 		/* Update high watermark before we lower rss */
 		update_hiwater_rss(mm);
 
-		if (PageHWPoison(subpage)) {
+		if (ttu_create_hwpoison(folio, subpage, TTU_HWPOISON)) {
 			VM_WARN_ON_FOLIO(folio_is_device_private(folio), folio);
 
 			pteval = swp_entry_to_pte(make_hwpoison_entry(subpage));
 			if (folio_test_hugetlb(folio)) {
 				hugetlb_count_sub(folio_nr_pages(folio), mm);
 				set_huge_pte_at(mm, address, pvmw.pte, pteval,
-						hsz);
+						folio_size(folio));
 			} else {
 				dec_mm_counter(mm, mm_counter(folio));
 				set_pte_at(mm, address, pvmw.pte, pteval);
@@ -2644,7 +2653,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
 			if (arch_unmap_one(mm, vma, address, pteval) < 0) {
 				if (folio_test_hugetlb(folio))
 					set_huge_pte_at(mm, address, pvmw.pte,
-							pteval, hsz);
+							pteval,
+							folio_size(folio));
 				else
 					set_pte_at(mm, address, pvmw.pte, pteval);
 				ret = false;
@@ -2657,7 +2667,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
 				if (anon_exclusive &&
 				    hugetlb_try_share_anon_rmap(folio)) {
 					set_huge_pte_at(mm, address, pvmw.pte,
-							pteval, hsz);
+							pteval,
+							folio_size(folio));
 					ret = false;
 					page_vma_mapped_walk_done(&pvmw);
 					break;
@@ -2703,7 +2714,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
 			}
 			if (folio_test_hugetlb(folio))
 				set_huge_pte_at(mm, address, pvmw.pte, swp_pte,
-						hsz);
+						folio_size(folio));
 			else
 				set_pte_at(mm, address, pvmw.pte, swp_pte);
 			trace_set_migration_pte(address, pte_val(swp_pte),
-- 
2.47.3

[PATCH v8 09/15] mm: Remove locking mf_mutex in is_raw_hwpoison_page_in_hugepage()

[Matthew Wilcox (Oracle)]

Sleeping in this kind of predicate is unexpected.  Add a new spinlock
to protect access to the list, and turn it into a normal singly linked
list now that it doesn't need to be a lockless list.

Rename is_raw_hwpoison_page_in_hugepage() to hugetlb_page_hwpoison()
and make it take the folio (since the callers naturally have the folio).
Also remove the handling of non-hugetlb folios and make the arguments
const.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 fs/hugetlbfs/inode.c     |   4 +-
 include/linux/hugetlb.h  |   5 +-
 include/linux/mm_types.h |   4 +-
 mm/memory-failure.c      | 101 ++++++++++++++++++++++-----------------
 4 files changed, 61 insertions(+), 53 deletions(-)

diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index fbac554886c3..dbca3f713bbf 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -198,7 +198,7 @@ static size_t adjust_range_hwpoison(struct folio *folio, size_t offset,
 	struct page *page = folio_page(folio, offset / PAGE_SIZE);
 	size_t safe_bytes;
 
-	if (is_raw_hwpoison_page_in_hugepage(page))
+	if (hugetlb_page_hwpoison(folio, page))
 		return 0;
 	/* Safe to read the remaining bytes in this page. */
 	safe_bytes = PAGE_SIZE - (offset % PAGE_SIZE);
@@ -206,7 +206,7 @@ static size_t adjust_range_hwpoison(struct folio *folio, size_t offset,
 
 	/* Check each remaining page as long as we are not done yet. */
 	for (; safe_bytes < bytes; safe_bytes += PAGE_SIZE, page++)
-		if (is_raw_hwpoison_page_in_hugepage(page))
+		if (hugetlb_page_hwpoison(folio, page))
 			break;
 
 	return min(safe_bytes, bytes);
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index e200c93088bc..ec604cee8d22 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -1088,10 +1088,7 @@ void hugetlb_register_node(struct node *node);
 void hugetlb_unregister_node(struct node *node);
 #endif
 
-/*
- * Check if a given raw @page in a hugepage is HWPOISON.
- */
-bool is_raw_hwpoison_page_in_hugepage(struct page *page);
+bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page);
 
 static inline unsigned long huge_page_mask_align(struct file *file)
 {
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index b18c2b2e7d2c..86a2d0fee557 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -387,7 +387,7 @@ typedef unsigned short mm_id_t;
  * @_hugetlb_subpool: Do not use directly, use accessor in hugetlb.h.
  * @_hugetlb_cgroup: Do not use directly, use accessor in hugetlb_cgroup.h.
  * @_hugetlb_cgroup_rsvd: Do not use directly, use accessor in hugetlb_cgroup.h.
- * @_hugetlb_hwpoison: Do not use directly, call raw_hwp_list_head().
+ * @hugetlb_hwpoison: List of pages with hwpoison.
  * @_deferred_list: Folios to be split under memory pressure.
  * @_unused_slab_obj_exts: Placeholder to match obj_exts in struct slab.
  *
@@ -499,7 +499,7 @@ struct folio {
 			void *_hugetlb_subpool;
 			void *_hugetlb_cgroup;
 			void *_hugetlb_cgroup_rsvd;
-			void *_hugetlb_hwpoison;
+			struct hwp_page *hugetlb_hwpoison;
 	/* private: the union with struct page is transitional */
 		};
 		struct page __page_3;
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 714e1b398f2c..437a2be6c58c 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1808,63 +1808,60 @@ EXPORT_SYMBOL_GPL(mf_dax_kill_procs);
 
 #ifdef CONFIG_HUGETLB_PAGE
 
+/* Protects all lists of hwp_pages */
+static DEFINE_SPINLOCK(hwp_page_lock);
+
 /*
- * Struct raw_hwp_page represents information about "raw error page",
- * constructing singly linked list from ->_hugetlb_hwpoison field of folio.
+ * hwp_page represents information about "error page",
+ * constructing singly linked list from folio->hugetlb_hwpoison field.
  */
-struct raw_hwp_page {
-	struct llist_node node;
+struct hwp_page {
+	struct hwp_page *next;
 	struct page *page;
 };
 
-static inline struct llist_head *raw_hwp_list_head(struct folio *folio)
-{
-	return (struct llist_head *)&folio->_hugetlb_hwpoison;
-}
-
-bool is_raw_hwpoison_page_in_hugepage(struct page *page)
+/*
+ * Check if a given @page in a hugetlb folio is HWPOISON.
+ */
+bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page)
 {
-	struct llist_head *raw_hwp_head;
-	struct raw_hwp_page *p;
-	struct folio *folio = page_folio(page);
-	bool ret = false;
+	const struct hwp_page *p;
+	unsigned long flags;
 
 	if (!folio_test_has_hwpoisoned(folio))
 		return false;
 
-	if (!folio_test_hugetlb(folio))
-		return PageHWPoison(page);
+	spin_lock_irqsave(&hwp_page_lock, flags);
 
 	/*
-	 * When RawHwpUnreliable is set, kernel lost track of which subpages
-	 * are HWPOISON. So return as if ALL subpages are HWPOISONed.
+	 * When RawHwpUnreliable is set, kernel lost track of which pages
+	 * are HWPOISON. So return as if ALL pages are HWPOISONed.
 	 */
-	if (folio_test_hugetlb_raw_hwp_unreliable(folio))
+	if (folio_test_hugetlb_raw_hwp_unreliable(folio)) {
+		spin_unlock_irqrestore(&hwp_page_lock, flags);
 		return true;
+	}
 
-	mutex_lock(&mf_mutex);
-
-	raw_hwp_head = raw_hwp_list_head(folio);
-	llist_for_each_entry(p, raw_hwp_head->first, node) {
-		if (page == p->page) {
-			ret = true;
+	for (p = folio->hugetlb_hwpoison; p; p = p->next) {
+		if (page == p->page)
 			break;
-		}
 	}
+	spin_unlock_irqrestore(&hwp_page_lock, flags);
 
-	mutex_unlock(&mf_mutex);
-
-	return ret;
+	return p != NULL;
 }
 
 static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag)
 {
-	struct llist_node *head;
-	struct raw_hwp_page *p, *next;
+	struct hwp_page *p, *next;
 	unsigned long count = 0;
 
-	head = llist_del_all(raw_hwp_list_head(folio));
-	llist_for_each_entry_safe(p, next, head, node) {
+	next = folio->hugetlb_hwpoison;
+	folio->hugetlb_hwpoison = NULL;
+
+	while (next) {
+		p = next;
+		next = p->next;
 		if (move_flag)
 			SetPageHWPoison(p->page);
 		else
@@ -1905,9 +1902,8 @@ static inline int hugetlb_clear_poison(struct folio *folio)
  */
 static int hugetlb_update_hwpoison(struct folio *folio, struct page *page)
 {
-	struct llist_head *head;
-	struct raw_hwp_page *raw_hwp;
-	struct raw_hwp_page *p;
+	struct hwp_page *p;
+	unsigned long flags;
 	int ret = hugetlb_set_poison(folio);
 
 	/*
@@ -1917,16 +1913,23 @@ static int hugetlb_update_hwpoison(struct folio *folio, struct page *page)
 	 */
 	if (folio_test_hugetlb_raw_hwp_unreliable(folio))
 		return MF_HUGETLB_FOLIO_PRE_POISONED;
-	head = raw_hwp_list_head(folio);
-	llist_for_each_entry(p, head->first, node) {
+
+	spin_lock_irqsave(&hwp_page_lock, flags);
+	for (p = folio->hugetlb_hwpoison; p; p = p->next) {
 		if (p->page == page)
-			return MF_HUGETLB_PAGE_PRE_POISONED;
+			break;
 	}
 
-	raw_hwp = kmalloc_obj(struct raw_hwp_page, GFP_ATOMIC);
-	if (raw_hwp) {
-		raw_hwp->page = page;
-		llist_add(&raw_hwp->node, head);
+	if (p) {
+		spin_unlock_irqrestore(&hwp_page_lock, flags);
+		return MF_HUGETLB_PAGE_PRE_POISONED;
+	}
+
+	p = kmalloc_obj(*p, GFP_ATOMIC);
+	if (p) {
+		p->page = page;
+		p->next = folio->hugetlb_hwpoison;
+		folio->hugetlb_hwpoison = p;
 	} else {
 		/*
 		 * Failed to save raw error info.  We no longer trace all
@@ -1935,16 +1938,20 @@ static int hugetlb_update_hwpoison(struct folio *folio, struct page *page)
 		 */
 		folio_set_hugetlb_raw_hwp_unreliable(folio);
 		/*
-		 * Once hugetlb_raw_hwp_unreliable is set, raw_hwp_page is not
+		 * Once hugetlb_raw_hwp_unreliable is set, hwp_page is not
 		 * used any more, so free it.
 		 */
 		__folio_free_raw_hwp(folio, false);
 	}
+	spin_unlock_irqrestore(&hwp_page_lock, flags);
 	return ret;
 }
 
 static unsigned long folio_free_raw_hwp(struct folio *folio, bool move_flag)
 {
+	unsigned long count;
+	unsigned long flags;
+
 	/*
 	 * hugetlb_vmemmap_optimized hugepages can't be freed because struct
 	 * pages for tail pages are required but they don't exist.
@@ -1959,7 +1966,11 @@ static unsigned long folio_free_raw_hwp(struct folio *folio, bool move_flag)
 	if (folio_test_hugetlb_raw_hwp_unreliable(folio))
 		return 0;
 
-	return __folio_free_raw_hwp(folio, move_flag);
+	spin_lock_irqsave(&hwp_page_lock, flags);
+	count = __folio_free_raw_hwp(folio, move_flag);
+	spin_unlock_irqrestore(&hwp_page_lock, flags);
+
+	return count;
 }
 
 void folio_clear_hugetlb_hwpoison(struct folio *folio)
-- 
2.47.3

[PATCH v8 10/15] mm: Check individual hugetlb pages for poison

[Matthew Wilcox (Oracle)]

So far, is_page_hwpoison() has returned true for the entire poisoned
hugetlb folio, but we can usually allow access to the pages which
aren't poisoned.  It's slightly tricky due to not having a reference
to the folio, but the hugetlb lock can save us here.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 include/linux/page-flags.h |  4 +++-
 mm/memory-failure.c        | 43 +++++++++++++++++++++++++++++++-------
 2 files changed, 39 insertions(+), 8 deletions(-)

diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 4fab3fbfd430..ee026ba9b0b5 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -1094,6 +1094,8 @@ static inline bool PageHuge(const struct page *page)
 	return folio_test_hugetlb(page_folio(page));
 }
 
+bool hugetlb_unref_page_hwpoison(const struct page *page);
+
 /*
  * Check if a page is currently marked HWPoisoned.  This check is best
  * effort only and inherently racy: there is no way to synchronize with
@@ -1108,7 +1110,7 @@ static inline bool is_page_hwpoison(const struct page *page)
 		return true;
 	folio = page_folio(page);
 	if (folio_test_huge_poison(folio))
-		return true;
+		return hugetlb_unref_page_hwpoison(page);
 	/* In case we raced with hugetlb transferring flags */
 	return PageHWPoison(page);
 }
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index 437a2be6c58c..c0a402522c21 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -1820,17 +1820,12 @@ struct hwp_page {
 	struct page *page;
 };
 
-/*
- * Check if a given @page in a hugetlb folio is HWPOISON.
- */
-bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page)
+static bool precise_page_poisoned(const struct folio *folio,
+		const struct page *page)
 {
 	const struct hwp_page *p;
 	unsigned long flags;
 
-	if (!folio_test_has_hwpoisoned(folio))
-		return false;
-
 	spin_lock_irqsave(&hwp_page_lock, flags);
 
 	/*
@@ -1851,6 +1846,40 @@ bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page)
 	return p != NULL;
 }
 
+/*
+ * Check if a given @page in a hugetlb folio is HWPOISON.
+ */
+bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page)
+{
+	if (!folio_test_has_hwpoisoned(folio))
+		return false;
+
+	return precise_page_poisoned(folio, page);
+}
+
+/*
+ * We have no reference on the folio containing this page.
+ * The hugetlb_lock keeps hugetlb folios from being freed.
+ */
+bool hugetlb_unref_page_hwpoison(const struct page *page)
+{
+	const struct folio *folio;
+	unsigned long flags;
+	bool ret;
+
+	spin_lock_irqsave(&hugetlb_lock, flags);
+	folio = page_folio(page);
+	if (!folio_test_huge_poison(folio)) {
+		ret = PageHWPoison(page);
+		goto unlock;
+	}
+
+	ret = precise_page_poisoned(folio, page);
+unlock:
+	spin_unlock_irqrestore(&hugetlb_lock, flags);
+	return ret;
+}
+
 static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag)
 {
 	struct hwp_page *p, *next;
-- 
2.47.3

[PATCH v8 11/15] filemap: Add hwpoison handling to filemap_read()

[Matthew Wilcox (Oracle)]

From: Jane Chu <jane.chu@oracle.com>

Add hwpoison handling to filemap_read() such that .read_iter() could
make best effort copying data out of clean pages without risking
MCE in case page cache contains HWpoison.

Signed-off-by: Jane Chu <jane.chu@oracle.com>
Co-developed-by: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Matthew Wilcox <willy@infradead.org>
---
 include/linux/hugetlb.h    |  2 --
 include/linux/page-flags.h | 11 +++++++++++
 mm/filemap.c               | 33 +++++++++++++++++++++++++++++++--
 3 files changed, 42 insertions(+), 4 deletions(-)

diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index ec604cee8d22..639c0a772856 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -1088,8 +1088,6 @@ void hugetlb_register_node(struct node *node);
 void hugetlb_unregister_node(struct node *node);
 #endif
 
-bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page);
-
 static inline unsigned long huge_page_mask_align(struct file *file)
 {
 	return PAGE_MASK & ~huge_page_mask(hstate_file(file));
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index ee026ba9b0b5..07cc7854b2de 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -1094,6 +1094,7 @@ static inline bool PageHuge(const struct page *page)
 	return folio_test_hugetlb(page_folio(page));
 }
 
+bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page);
 bool hugetlb_unref_page_hwpoison(const struct page *page);
 
 /*
@@ -1115,6 +1116,16 @@ static inline bool is_page_hwpoison(const struct page *page)
 	return PageHWPoison(page);
 }
 
+static inline bool is_ref_page_hwpoison(const struct folio *folio,
+		const struct page *page)
+{
+	if (PageHWPoison(page))
+		return true;
+	if (folio_test_hugetlb(folio))
+		return hugetlb_page_hwpoison(folio, page);
+	return false;
+}
+
 static inline bool folio_has_hwpoisoned_page(const struct folio *folio)
 {
 	return PageHWPoison(&folio->page) ||
diff --git a/mm/filemap.c b/mm/filemap.c
index 58eb9d240643..26a5f18121f9 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -2483,6 +2483,8 @@ static void filemap_get_read_batch(struct address_space *mapping,
 
 		if (!folio_batch_add(fbatch, folio))
 			break;
+		if (folio_has_hwpoisoned_page(folio))
+			break;
 		if (!folio_test_uptodate(folio))
 			break;
 		if (folio_test_readahead(folio))
@@ -2749,6 +2751,29 @@ static inline bool pos_same_folio(loff_t pos1, loff_t pos2, struct folio *folio)
 	return (pos1 >> shift == pos2 >> shift);
 }
 
+static size_t adjust_range_hwpoison(const struct folio *folio, size_t offset,
+		size_t bytes)
+{
+	const struct page *page = folio_page(folio, offset / PAGE_SIZE);
+	size_t safe_bytes;
+
+	if (!folio_has_hwpoisoned_page(folio))
+		return bytes;
+	if (is_ref_page_hwpoison(folio, page))
+		return 0;
+
+	/* Safe to read the remaining bytes in this page. */
+	safe_bytes = PAGE_SIZE - (offset % PAGE_SIZE);
+	page++;
+
+	/* Check each remaining page as long as we are not done yet. */
+	for (; safe_bytes < bytes; safe_bytes += PAGE_SIZE, page++)
+		if (is_ref_page_hwpoison(folio, page))
+			break;
+
+	return min(safe_bytes, bytes);
+}
+
 static void filemap_end_dropbehind_read(struct folio *folio)
 {
 	if (!folio_test_dropbehind(folio))
@@ -2862,14 +2887,18 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter,
 			if (writably_mapped)
 				flush_dcache_folio(folio);
 
-			copied = copy_folio_to_iter(folio, offset, bytes, iter);
+			copied = adjust_range_hwpoison(folio, offset, bytes);
+			if (copied < bytes)
+				error = -EIO;
+			copied = copy_folio_to_iter(folio, offset, copied, iter);
 
 			already_read += copied;
 			iocb->ki_pos += copied;
 			last_pos = iocb->ki_pos;
 
 			if (copied < bytes) {
-				error = -EFAULT;
+				if (!error)
+					error = -EFAULT;
 				break;
 			}
 		}
-- 
2.47.3

[PATCH v8 12/15] filemap: Remove checks in mapping_set_folio_order_range()

[Matthew Wilcox (Oracle)]

These checks make it impossible to use mapping_set_folio_order_range()
in hugetlb.  They add very little value and can simply be removed.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 include/linux/pagemap.h | 34 ++++++++++++++--------------------
 1 file changed, 14 insertions(+), 20 deletions(-)

diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 2c3718d592d6..968b791cfd14 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -373,7 +373,7 @@ static inline void mapping_set_gfp_mask(struct address_space *m, gfp_t mask)
  * assumptions about maximum order if THP are disabled, but 8 seems like
  * a good order (that's 1MB if you're using 4kB pages)
  */
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#ifdef CONFIG_LARGE_FOLIO
 #define PREFERRED_MAX_PAGECACHE_ORDER	HPAGE_PMD_ORDER
 #else
 #define PREFERRED_MAX_PAGECACHE_ORDER	8
@@ -394,7 +394,7 @@ static inline void mapping_set_gfp_mask(struct address_space *m, gfp_t mask)
  */
 static inline size_t mapping_max_folio_size_supported(void)
 {
-	if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
+	if (IS_ENABLED(CONFIG_LARGE_FOLIO))
 		return 1U << (PAGE_SHIFT + MAX_PAGECACHE_ORDER);
 	return PAGE_SIZE;
 }
@@ -405,29 +405,23 @@ static inline size_t mapping_max_folio_size_supported(void)
  * @min: Minimum folio order (between 0-MAX_PAGECACHE_ORDER inclusive).
  * @max: Maximum folio order (between @min-MAX_PAGECACHE_ORDER inclusive).
  *
- * The filesystem should call this function in its inode constructor to
- * indicate which base size (min) and maximum size (max) of folio the VFS
- * can use to cache the contents of the file.  This should only be used
- * if the filesystem needs special handling of folio sizes (ie there is
- * something the core cannot know).
+ * The filesystem should call this function in its inode constructor
+ * to indicate which size folios can be used to cache the contents of
+ * the inode.  This should only be used if the filesystem needs special
+ * handling of folio sizes (ie there is something the core cannot know).
  * Do not tune it based on, eg, i_size.
  *
+ * hugetlb calls this with orders larger than MAX_PAGECACHE_ORDER.
+ * Normal filesystems should not do this.
+ *
  * Context: This should not be called while the inode is active as it
  * is non-atomic.
  */
 static inline void mapping_set_folio_order_range(struct address_space *mapping,
-						 unsigned int min,
-						 unsigned int max)
+		unsigned int min, unsigned int max)
 {
-	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
+	if (!IS_ENABLED(CONFIG_LARGE_FOLIO))
 		return;
-
-	if (min > MAX_PAGECACHE_ORDER)
-		min = MAX_PAGECACHE_ORDER;
-
-	if (max > MAX_PAGECACHE_ORDER)
-		max = MAX_PAGECACHE_ORDER;
-
 	if (max < min)
 		max = min;
 
@@ -460,7 +454,7 @@ static inline void mapping_set_large_folios(struct address_space *mapping)
 static inline unsigned int
 mapping_max_folio_order(const struct address_space *mapping)
 {
-	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
+	if (!IS_ENABLED(CONFIG_LARGE_FOLIO))
 		return 0;
 	return (mapping->flags & AS_FOLIO_ORDER_MAX_MASK) >> AS_FOLIO_ORDER_MAX;
 }
@@ -468,7 +462,7 @@ mapping_max_folio_order(const struct address_space *mapping)
 static inline unsigned int
 mapping_min_folio_order(const struct address_space *mapping)
 {
-	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
+	if (!IS_ENABLED(CONFIG_LARGE_FOLIO))
 		return 0;
 	return (mapping->flags & AS_FOLIO_ORDER_MIN_MASK) >> AS_FOLIO_ORDER_MIN;
 }
@@ -524,7 +518,7 @@ static inline bool mapping_large_folio_support(const struct address_space *mappi
  *
  * Return: True if PMD-sized folios are supported, otherwise false.
  */
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#ifdef CONFIG_LARGE_FOLIO
 static inline bool mapping_pmd_folio_support(const struct address_space *mapping)
 {
 	/* AS_FOLIO_ORDER is only reasonable for pagecache folios */
-- 
2.47.3

[PATCH v8 13/15] hugetlb: Set mapping folio order

[Matthew Wilcox (Oracle)]

hugetlbfs currently uses an fs-specific way to determine the size of
the folios in its page cache.  Thanks to the support of block sizes
larger than page sizes, we now support this as part of the page cache.
It's somewhat more efficient as we have this information directly in the
mapping rather than going from inode->i_sb->s_fs_info.  We can convert
the rest of the hugetlb code to use this at our leisure; this is needed
now so that we can use filemap_get_pages() unmodified.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Gregory Price <gourry@gourry.net>
---
 fs/hugetlbfs/inode.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index dbca3f713bbf..8d15413a67ee 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -883,6 +883,16 @@ static struct inode *hugetlbfs_get_root(struct super_block *sb,
 	return inode;
 }
 
+static void hugetlbfs_init_regular_inode(struct inode *inode)
+{
+	struct hstate *hstate = hstate_inode(inode);
+	unsigned int order = huge_page_order(hstate);
+
+	inode->i_op = &hugetlbfs_inode_operations;
+	inode->i_fop = &hugetlbfs_file_operations;
+	mapping_set_folio_order_range(inode->i_mapping, order, order);
+}
+
 /*
  * Hugetlbfs is not reclaimable; therefore its i_mmap_rwsem will never
  * be taken from reclaim -- unlike regular filesystems. This needs an
@@ -926,8 +936,7 @@ static struct inode *hugetlbfs_get_inode(struct super_block *sb,
 			init_special_inode(inode, mode, dev);
 			break;
 		case S_IFREG:
-			inode->i_op = &hugetlbfs_inode_operations;
-			inode->i_fop = &hugetlbfs_file_operations;
+			hugetlbfs_init_regular_inode(inode);
 			break;
 		case S_IFDIR:
 			inode->i_op = &hugetlbfs_dir_inode_operations;
-- 
2.47.3

[PATCH v8 14/15] filemap: Add support for authoritative mappings

[Matthew Wilcox (Oracle)]

An authoritative mapping knows about all the folios in the mapping.
If read() finds a missing folio, there's no reason to allocate one and
try to read it because we know it's a zero region of the file.  We can
just call iov_iter_zero() instead.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 include/linux/pagemap.h | 11 +++++++++++
 mm/filemap.c            | 18 ++++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 968b791cfd14..22d48935ffda 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -210,6 +210,7 @@ enum mapping_flags {
 	AS_WRITEBACK_MAY_DEADLOCK_ON_RECLAIM = 9,
 	AS_KERNEL_FILE = 10,	/* mapping for a fake kernel file that shouldn't
 				   account usage to user cgroups */
+	AS_AUTHORITATIVE = 11,	/* If we miss in the page cache, it's a hole */
 	/* Bits 16-25 are used for FOLIO_ORDER */
 	AS_FOLIO_ORDER_BITS = 5,
 	AS_FOLIO_ORDER_MIN = 16,
@@ -345,6 +346,16 @@ static inline bool mapping_writeback_may_deadlock_on_reclaim(const struct addres
 	return test_bit(AS_WRITEBACK_MAY_DEADLOCK_ON_RECLAIM, &mapping->flags);
 }
 
+static inline void mapping_set_authoritative(struct address_space *mapping)
+{
+	set_bit(AS_AUTHORITATIVE, &mapping->flags);
+}
+
+static inline bool mapping_is_authoritative(const struct address_space *mapping)
+{
+	return test_bit(AS_AUTHORITATIVE, &mapping->flags);
+}
+
 static inline gfp_t mapping_gfp_mask(const struct address_space *mapping)
 {
 	return mapping->gfp_mask;
diff --git a/mm/filemap.c b/mm/filemap.c
index 26a5f18121f9..5a8cc20e624e 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -2697,6 +2697,8 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count,
 	if (!folio_batch_count(fbatch)) {
 		DEFINE_READAHEAD(ractl, filp, &filp->f_ra, mapping, index);
 
+		if (mapping_is_authoritative(mapping))
+			return 0;
 		if (iocb->ki_flags & IOCB_NOIO)
 			return -EAGAIN;
 		if (iocb->ki_flags & IOCB_NOWAIT)
@@ -2853,6 +2855,22 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter,
 			goto put_folios;
 		end_offset = min_t(loff_t, isize, iocb->ki_pos + iter->count);
 
+		if (!folio_batch_count(&fbatch)) {
+			size_t fsize = mapping_min_folio_nrbytes(mapping);
+			size_t offset = iocb->ki_pos & (fsize - 1);
+			size_t bytes = min_t(loff_t, end_offset - iocb->ki_pos,
+					     fsize - offset);
+			size_t copied = iov_iter_zero(bytes, iter);
+
+			already_read += copied;
+			iocb->ki_pos += copied;
+			last_pos = iocb->ki_pos;
+
+			if (copied < bytes)
+				error = -EFAULT;
+			continue;
+		}
+
 		/*
 		 * Once we start copying data, we don't want to be touching any
 		 * cachelines that might be contended:
-- 
2.47.3

[PATCH v8 15/15] hugetlb: replace hugetlbfs_read_iter() with generic_file_read_iter()

[Matthew Wilcox (Oracle)]

From: Jane Chu <jane.chu@oracle.com>

generic_file_read_iter() now handles hugetlb poison, hugetlb page
size and missing folios correctly, so we no longer need a separate
hugetlbfs_read_iter().

Signed-off-by: Jane Chu <jane.chu@oracle.com>
Co-developed-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 fs/hugetlbfs/inode.c | 110 +------------------------------------------
 1 file changed, 2 insertions(+), 108 deletions(-)

diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 8d15413a67ee..cef7e61b827d 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -187,113 +187,6 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr,
 	return mm_get_unmapped_area_vmflags(file, addr0, len, pgoff, flags, 0);
 }
 
-/*
- * Someone wants to read @bytes from a HWPOISON hugetlb @folio from @offset.
- * Returns the maximum number of bytes one can read without touching the 1st raw
- * HWPOISON page.
- */
-static size_t adjust_range_hwpoison(struct folio *folio, size_t offset,
-		size_t bytes)
-{
-	struct page *page = folio_page(folio, offset / PAGE_SIZE);
-	size_t safe_bytes;
-
-	if (hugetlb_page_hwpoison(folio, page))
-		return 0;
-	/* Safe to read the remaining bytes in this page. */
-	safe_bytes = PAGE_SIZE - (offset % PAGE_SIZE);
-	page++;
-
-	/* Check each remaining page as long as we are not done yet. */
-	for (; safe_bytes < bytes; safe_bytes += PAGE_SIZE, page++)
-		if (hugetlb_page_hwpoison(folio, page))
-			break;
-
-	return min(safe_bytes, bytes);
-}
-
-/*
- * Support for read() - Find the page attached to f_mapping and copy out the
- * data. This provides functionality similar to filemap_read().
- */
-static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to)
-{
-	struct file *file = iocb->ki_filp;
-	struct hstate *h = hstate_file(file);
-	struct address_space *mapping = file->f_mapping;
-	struct inode *inode = mapping->host;
-	unsigned long index = iocb->ki_pos >> huge_page_shift(h);
-	unsigned long offset = iocb->ki_pos & ~huge_page_mask(h);
-	unsigned long end_index;
-	loff_t isize;
-	ssize_t retval = 0;
-
-	while (iov_iter_count(to)) {
-		struct folio *folio;
-		size_t nr, copied, want;
-
-		/* nr is the maximum number of bytes to copy from this page */
-		nr = huge_page_size(h);
-		isize = i_size_read(inode);
-		if (!isize)
-			break;
-		end_index = (isize - 1) >> huge_page_shift(h);
-		if (index > end_index)
-			break;
-		if (index == end_index) {
-			nr = ((isize - 1) & ~huge_page_mask(h)) + 1;
-			if (nr <= offset)
-				break;
-		}
-		nr = nr - offset;
-
-		/* Find the folio */
-		folio = filemap_lock_hugetlb_folio(h, mapping, index);
-		if (IS_ERR(folio)) {
-			/*
-			 * We have a HOLE, zero out the user-buffer for the
-			 * length of the hole or request.
-			 */
-			copied = iov_iter_zero(nr, to);
-		} else {
-			folio_unlock(folio);
-
-			if (!folio_test_has_hwpoisoned(folio))
-				want = nr;
-			else {
-				/*
-				 * Adjust how many bytes safe to read without
-				 * touching the 1st raw HWPOISON page after
-				 * offset.
-				 */
-				want = adjust_range_hwpoison(folio, offset, nr);
-				if (want == 0) {
-					folio_put(folio);
-					retval = -EIO;
-					break;
-				}
-			}
-
-			/*
-			 * We have the folio, copy it to user space buffer.
-			 */
-			copied = copy_folio_to_iter(folio, offset, want, to);
-			folio_put(folio);
-		}
-		offset += copied;
-		retval += copied;
-		if (copied != nr && iov_iter_count(to)) {
-			if (!retval)
-				retval = -EFAULT;
-			break;
-		}
-		index += offset >> huge_page_shift(h);
-		offset &= ~huge_page_mask(h);
-	}
-	iocb->ki_pos = ((loff_t)index << huge_page_shift(h)) + offset;
-	return retval;
-}
-
 static int hugetlbfs_write_begin(const struct kiocb *iocb,
 			struct address_space *mapping,
 			loff_t pos, unsigned len,
@@ -891,6 +784,7 @@ static void hugetlbfs_init_regular_inode(struct inode *inode)
 	inode->i_op = &hugetlbfs_inode_operations;
 	inode->i_fop = &hugetlbfs_file_operations;
 	mapping_set_folio_order_range(inode->i_mapping, order, order);
+	mapping_set_authoritative(inode->i_mapping);
 }
 
 /*
@@ -1215,7 +1109,7 @@ static void init_once(void *foo)
 }
 
 static const struct file_operations hugetlbfs_file_operations = {
-	.read_iter		= hugetlbfs_read_iter,
+	.read_iter		= generic_file_read_iter,
 	.mmap			= hugetlbfs_file_mmap,
 	.fsync			= noop_fsync,
 	.get_unmapped_area	= hugetlb_get_unmapped_area,
-- 
2.47.3

Re: [PATCH v8 02/15] memory-failure: Prevent hugetlb freeing during unpoisoning

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> If we fail to get a reference on the hugetlb folio, then it might be 
> freed as we operate on it. Prevent the freeing and the attendant races 
> around manipulation of the raw_hwp list by holding the hugetlb_lock, 
> which is also held by the hugetlb
> 
> 
> If we fail to get a reference on the hugetlb folio, then it might be
> freed as we operate on it.  Prevent the freeing and the attendant races
> around manipulation of the raw_hwp list by holding the hugetlb_lock,
> which is also held by the hugetlb code when freeing hugetlb folios.
> 
> Fixes: ac5fcde0a96a ("mm, hwpoison: make unpoison aware of raw error info in hwpoisoned hugepage")
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
> ---
>   include/linux/hugetlb.h | 19 +++++++++++++++++++
>   mm/memory-failure.c     |  6 +++++-
>   2 files changed, 24 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index 2abaf99321e9..50eab2c23299 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -110,6 +110,17 @@ extern struct resv_map *resv_map_alloc(void);
>   void resv_map_release(struct kref *ref);
>   
>   extern spinlock_t hugetlb_lock;
> +
> +static inline void hugetlb_lock_irq(void)
> +{
> +	spin_lock_irq(&hugetlb_lock);
> +}
> +
> +static inline void hugetlb_unlock_irq(void)
> +{
> +	spin_unlock_irq(&hugetlb_lock);
> +}
> +
>   extern int hugetlb_max_hstate __read_mostly;
>   #define for_each_hstate(h) \
>   	for ((h) = hstates; (h) < &hstates[hugetlb_max_hstate]; (h)++)
> @@ -279,6 +290,14 @@ unsigned int arch_hugetlb_cma_order(void);
>   
>   #else /* !CONFIG_HUGETLB_PAGE */
>   
> +static inline void hugetlb_lock_irq(void)
> +{
> +}
> +
> +static inline void hugetlb_unlock_irq(void)
> +{
> +}
> +
>   static inline void hugetlb_dup_vma_private(struct vm_area_struct *vma)
>   {
>   }
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 944e6e1d4971..1dd0e7b99bb1 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -2725,13 +2725,17 @@ int unpoison_memory(unsigned long pfn)
>   
>   	ghp = get_hwpoison_page(p, MF_UNPOISON);
>   	if (!ghp) {
> +		hugetlb_lock_irq();
>   		if (folio_test_hugetlb(folio)) {
>   			huge = true;
>   			count = folio_free_raw_hwp(folio, false);
> -			if (count == 0)
> +			if (count == 0) {
> +				hugetlb_unlock_irq();
>   				goto unlock_mutex;
> +			}
>   		}
>   		ret = folio_test_clear_hwpoison(folio) ? 0 : -EBUSY;
> +		hugetlb_unlock_irq();
>   	} else if (ghp < 0) {
>   		if (ghp == -EHWPOISON) {
>   			ret = put_page_back_buddy(p) ? 0 : -EBUSY;
> -- 
> 2.47.3
> 

Patch itself looks good, so Reviewed-by: Jane Chu <jane.chu@oracle.com>

That said, there is a pre-existing issue:
folio_free_raw_hwp() should check HPG_raw_hwp_unreliable, and fail the 
act of unpoison just like what __update_and_free_hugetlb_folio() does -

static void __update_and_free_hugetlb_folio(struct hstate *h,
                                                 struct folio *folio)
{
         bool clear_flag = folio_test_hugetlb_vmemmap_optimized(folio);

         if (hstate_is_gigantic_no_runtime(h))
                 return;

         /*
          * If we don't know which subpages are hwpoisoned, we can't free
          * the hugepage, so it's leaked intentionally.
          */
         if (folio_test_hugetlb_raw_hwp_unreliable(folio))
                 return;


thanks,
-jane

Re: [PATCH v8 04/15] hugetlb: Mark some function arguments as const

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> These functions do not modify their folio argument, so we can mark them 
> as being const which will allow for some minor optimisations and let us 
> mark other function arguments as being const in the future. Signed-off- 
> by: Matthew Wilcox (Oracle)
> 
> 
> These functions do not modify their folio argument, so we can mark them
> as being const which will allow for some minor optimisations and let us
> mark other function arguments as being const in the future.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Acked-by: Usama Arif <usama.arif@linux.dev>
> Reviewed-by: Gregory Price <gourry@gourry.net>
> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
> ---
>   include/linux/hugetlb.h | 19 +++++++++----------
>   mm/hugetlb.c            |  2 +-
>   2 files changed, 10 insertions(+), 11 deletions(-)
> 
> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index 50eab2c23299..e200c93088bc 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -176,7 +176,7 @@ bool hugetlbfs_pagecache_present(struct hstate *h,
>   				 struct vm_area_struct *vma,
>   				 unsigned long address);
>   
> -struct address_space *hugetlb_folio_mapping_lock_write(struct folio *folio);
> +struct address_space *hugetlb_folio_mapping_lock_write(const struct folio *folio);
>   
>   extern int movable_gigantic_pages __read_mostly;
>   extern int sysctl_hugetlb_shm_group __read_mostly;
> @@ -311,8 +311,7 @@ static inline unsigned long hugetlb_total_pages(void)
>   	return 0;
>   }
>   
> -static inline struct address_space *hugetlb_folio_mapping_lock_write(
> -							struct folio *folio)
> +static inline struct address_space *hugetlb_folio_mapping_lock_write(const struct folio *folio)
>   {
>   	return NULL;
>   }
> @@ -614,8 +613,8 @@ enum hugetlb_page_flags {
>   #ifdef CONFIG_HUGETLB_PAGE
>   #define TESTHPAGEFLAG(uname, flname)				\
>   static __always_inline						\
> -bool folio_test_hugetlb_##flname(struct folio *folio)		\
> -	{	void *private = &folio->private;		\
> +bool folio_test_hugetlb_##flname(const struct folio *folio)	\
> +	{	const void *private = &folio->private;		\
>   		return test_bit(HPG_##flname, private);		\
>   	}
>   
> @@ -635,7 +634,7 @@ void folio_clear_hugetlb_##flname(struct folio *folio)		\
>   #else
>   #define TESTHPAGEFLAG(uname, flname)				\
>   static inline bool						\
> -folio_test_hugetlb_##flname(struct folio *folio)		\
> +folio_test_hugetlb_##flname(const struct folio *folio)		\
>   	{ return 0; }
>   
>   #define SETHPAGEFLAG(uname, flname)				\
> @@ -746,7 +745,7 @@ static inline struct hugepage_subpool *subpool_inode(struct inode *inode)
>   	return HUGETLBFS_SB(inode->i_sb)->spool;
>   }
>   
> -static inline struct hugepage_subpool *hugetlb_folio_subpool(struct folio *folio)
> +static inline struct hugepage_subpool *hugetlb_folio_subpool(const struct folio *folio)
>   {
>   	return folio->_hugetlb_subpool;
>   }
> @@ -876,7 +875,7 @@ static inline bool arch_has_huge_bootmem_alloc(void)
>   }
>   #endif
>   
> -static inline struct hstate *folio_hstate(struct folio *folio)
> +static inline struct hstate *folio_hstate(const struct folio *folio)
>   {
>   	VM_BUG_ON_FOLIO(!folio_test_hugetlb(folio), folio);
>   	return size_to_hstate(folio_size(folio));
> @@ -1107,7 +1106,7 @@ static inline unsigned long huge_page_mask_align(struct file *file)
>   	return 0;
>   }
>   
> -static inline struct hugepage_subpool *hugetlb_folio_subpool(struct folio *folio)
> +static inline struct hugepage_subpool *hugetlb_folio_subpool(const struct folio *folio)
>   {
>   	return NULL;
>   }
> @@ -1176,7 +1175,7 @@ static inline struct hstate *hstate_vma(struct vm_area_struct *vma)
>   	return NULL;
>   }
>   
> -static inline struct hstate *folio_hstate(struct folio *folio)
> +static inline struct hstate *folio_hstate(const struct folio *folio)
>   {
>   	return NULL;
>   }
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 571212b80835..cc8d98399913 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -1775,7 +1775,7 @@ void init_new_hugetlb_folio(struct folio *folio)
>    * stable.  Due to locking order, we can only trylock_write.  If we can
>    * not get the lock, simply return NULL to caller.
>    */
> -struct address_space *hugetlb_folio_mapping_lock_write(struct folio *folio)
> +struct address_space *hugetlb_folio_mapping_lock_write(const struct folio *folio)
>   {
>   	struct address_space *mapping = folio_mapping(folio);
>   
> -- 
> 2.47.3
> 

Reviewed-by: Jane Chu <jane.chu@oracle.com>

thanks,
-jane

Re: [PATCH v8 05/15] guest_memfd: Use folio_has_hwpoisoned_page()

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> folio_test_hwpoison() only works on hugetlb folios today and will give 
> the wrong answer for other large folios. folio_has_hwpoisoned_page() 
> works on both. Signed-off-by: Matthew Wilcox (Oracle) <willy@  
> infradead. org> Acked-by: Sean Christopherson
> 
> 
> folio_test_hwpoison() only works on hugetlb folios today and will give
> the wrong answer for other large folios.  folio_has_hwpoisoned_page()
> works on both.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Acked-by: Sean Christopherson <seanjc@google.com>
> Reviewed-by: Ackerley Tng <ackerleytng@google.com>
> Reviewed-by: Gregory Price <gourry@gourry.net>
> ---
>   virt/kvm/guest_memfd.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index db57c5766ab6..94eedcc7f1a9 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c
> @@ -774,7 +774,7 @@ static struct folio *__kvm_gmem_get_pfn(struct file *file,
>   	if (IS_ERR(folio))
>   		return folio;
>   
> -	if (folio_test_hwpoison(folio)) {
> +	if (folio_has_hwpoisoned_page(folio)) {
>   		folio_unlock(folio);
>   		folio_put(folio);
>   		return ERR_PTR(-EHWPOISON);
> -- 
> 2.47.3
> 

Reviewed-by: Jane Chu <jane.chu@oracle.com>

thanks,
-jane

Re: [PATCH v8 06/15] kpageflags: Use is_page_hwpoison() to set KPF_HWPOISON

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> Instead of knowing how hugetlb handles hwpoison, just ask 
> is_page_hwpoison(). This gives us the flexibility to change how hwpoison 
> is handled without updating this function in the future. Signed-off-by: 
> Matthew Wilcox (Oracle) <willy@ infradead. org>
> 
> 
> Instead of knowing how hugetlb handles hwpoison, just ask
> is_page_hwpoison().  This gives us the flexibility to change how hwpoison
> is handled without updating this function in the future.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
>   fs/proc/page.c | 8 ++------
>   1 file changed, 2 insertions(+), 6 deletions(-)
> 
> diff --git a/fs/proc/page.c b/fs/proc/page.c
> index 7d9387143435..28c9a7b724f3 100644
> --- a/fs/proc/page.c
> +++ b/fs/proc/page.c
> @@ -233,12 +233,8 @@ u64 stable_page_flags(const struct page *page)
>   	u |= kpf_copy_bit(k, KPF_UNEVICTABLE,	PG_unevictable);
>   	u |= kpf_copy_bit(k, KPF_MLOCKED,	PG_mlocked);
>   
> -#ifdef CONFIG_MEMORY_FAILURE
> -	if (u & (1 << KPF_HUGE))
> -		u |= kpf_copy_bit(k, KPF_HWPOISON,	PG_hwpoison);
> -	else
> -		u |= kpf_copy_bit(ps.page_snapshot.flags.f, KPF_HWPOISON, PG_hwpoison);
> -#endif
> +	if (is_page_hwpoison(page))
> +		u |= KPF_HWPOISON;
>   
>   	u |= kpf_copy_bit(k, KPF_RESERVED,	PG_reserved);
>   	u |= kpf_copy_bit(k, KPF_OWNER_2,	PG_owner_2);
> -- 
> 2.47.3
> 

Good idea, but agreed with Sushiko about the bit setting issue,
with that fixed,

Reviewed-by: Jane Chu <jane.chu@oracle.com>

thanks,
-jane

Re: [PATCH v8 07/15] hugetlb: Move poison to pages before clearing hugetlb page type

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> For is_page_hwpoison() to be reliable, we need to set the hwpoison flag 
> on each page before clearing the hugetlb flag. Introduce 
> folio_clear_hugetlb() to make this easier. This does not by itself make 
> is_page_hwpoison() reliable; see subsequent
> 
> 
> For is_page_hwpoison() to be reliable, we need to set the hwpoison
> flag on each page before clearing the hugetlb flag.  Introduce
> folio_clear_hugetlb() to make this easier.  This does not by
> itself make is_page_hwpoison() reliable; see subsequent patches.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
> ---
>   mm/hugetlb.c | 31 ++++++++++++++++++-------------
>   1 file changed, 18 insertions(+), 13 deletions(-)
> 
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index cc8d98399913..47403d02be88 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -1381,6 +1381,18 @@ static struct folio *alloc_gigantic_frozen_folio(int order, gfp_t gfp_mask, int
>   }
>   #endif
>   
> +static void folio_clear_hugetlb(struct folio *folio)
> +{
> +	/*
> +	 * Move HWPoison flag to each error page
> +	 * which makes any healthy pages reusable.
> +	 */
> +	if (unlikely(folio_test_hwpoison(folio)))
> +		folio_clear_hugetlb_hwpoison(folio);
> +
> +	__folio_clear_hugetlb(folio);
> +}
> +
>   /*
>    * Remove hugetlb folio from lists.
>    * If vmemmap exists for the folio, clear the hugetlb flag so that the
> @@ -1419,7 +1431,7 @@ void remove_hugetlb_folio(struct hstate *h, struct folio *folio,
>   	 * to tail struct pages.
>   	 */
>   	if (!folio_test_hugetlb_vmemmap_optimized(folio))
> -		__folio_clear_hugetlb(folio);
> +		folio_clear_hugetlb(folio);
>   
>   	h->nr_huge_pages--;
>   	h->nr_huge_pages_node[nid]--;
> @@ -1493,17 +1505,10 @@ static void __update_and_free_hugetlb_folio(struct hstate *h,
>   	 */
>   	if (folio_test_hugetlb(folio)) {
>   		spin_lock_irq(&hugetlb_lock);
> -		__folio_clear_hugetlb(folio);
> +		folio_clear_hugetlb(folio);
>   		spin_unlock_irq(&hugetlb_lock);
>   	}
>   
> -	/*
> -	 * Move PageHWPoison flag from head page to the raw error pages,
> -	 * which makes any healthy subpages reusable.
> -	 */
> -	if (unlikely(folio_test_hwpoison(folio)))
> -		folio_clear_hugetlb_hwpoison(folio);
> -
>   	VM_BUG_ON_FOLIO(folio_ref_count(folio), folio);
>   	if (folio_test_hugetlb_cma(folio))
>   		hugetlb_cma_free_frozen_folio(folio);
> @@ -1595,7 +1600,7 @@ static void bulk_vmemmap_restore_error(struct hstate *h,
>   		list_for_each_entry_safe(folio, t_folio, non_hvo_folios, lru) {
>   			list_del(&folio->lru);
>   			spin_lock_irq(&hugetlb_lock);
> -			__folio_clear_hugetlb(folio);
> +			folio_clear_hugetlb(folio);
>   			spin_unlock_irq(&hugetlb_lock);
>   			update_and_free_hugetlb_folio(h, folio, false);
>   			cond_resched();
> @@ -1620,7 +1625,7 @@ static void bulk_vmemmap_restore_error(struct hstate *h,
>   			} else {
>   				list_del(&folio->lru);
>   				spin_lock_irq(&hugetlb_lock);
> -				__folio_clear_hugetlb(folio);
> +				folio_clear_hugetlb(folio);
>   				spin_unlock_irq(&hugetlb_lock);
>   				update_and_free_hugetlb_folio(h, folio, false);
>   				cond_resched();
> @@ -1653,14 +1658,14 @@ static void update_and_free_pages_bulk(struct hstate *h,
>   	 * should only be pages on the non_hvo_folios list.
>   	 * Do note that the non_hvo_folios list could be empty.
>   	 * Without HVO enabled, ret will be 0 and there is no need to call
> -	 * __folio_clear_hugetlb as this was done previously.
> +	 * folio_clear_hugetlb as this was done previously.
>   	 */
>   	VM_WARN_ON(!list_empty(folio_list));
>   	VM_WARN_ON(ret < 0);
>   	if (!list_empty(&non_hvo_folios) && ret) {
>   		spin_lock_irq(&hugetlb_lock);
>   		list_for_each_entry(folio, &non_hvo_folios, lru)
> -			__folio_clear_hugetlb(folio);
> +			folio_clear_hugetlb(folio);
>   		spin_unlock_irq(&hugetlb_lock);
>   	}
>   
> -- 
> 2.47.3
> 

With the patch, we're dissolving the raw_hwp_list at the earliest 
opportunity upon freeing hugetlb, looks fine to me.

Reviewed-by: Jane Chu <jane.chu@oracle.com>

thanks,
-jane

Re: [PATCH v8 08/15] hugetlb: Use the has_hwpoisoned flag

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> Other large folios use the has_hwpoisoned flag. Convert hugetlb to 
> match. This will help us use the per-page hwpoison flag in the future. 
> Also introduce a folio_test_huge_poison(). This has exactly the same 
> meaning as folio_test_has_hwpoisoned()
> 
> 
> Other large folios use the has_hwpoisoned flag.  Convert hugetlb to match.
> This will help us use the per-page hwpoison flag in the future.
> 
> Also introduce a folio_test_huge_poison().  This has exactly the same
> meaning as folio_test_has_hwpoisoned() but can be used when we don't have
> a reference to the folio containing the page.  folio_test_has_hwpoisoned()
> can race with folio splitting / reallocation and trip the assertions
> in const_folio_flags().
> 
> This closes a gap where a page in a previously-poisoned hugetlb folio
> could be observed to not have the hwpoison bit set.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
>   fs/Kconfig                 |  2 +-
>   fs/hugetlbfs/inode.c       |  2 +-
>   include/linux/page-flags.h | 46 ++++++++++++++++++++++++------
>   mm/Kconfig                 |  6 +++-
>   mm/hugetlb.c               | 10 +++----
>   mm/memory-failure.c        | 58 +++++++++++++++++++++++++++-----------
>   mm/rmap.c                  | 43 +++++++++++++++++-----------
>   7 files changed, 119 insertions(+), 48 deletions(-)
> 
> diff --git a/fs/Kconfig b/fs/Kconfig
> index cf6ae64776e6..eddac4ed214b 100644
> --- a/fs/Kconfig
> +++ b/fs/Kconfig
> @@ -272,7 +272,7 @@ endif # HUGETLBFS
>   
>   config HUGETLB_PAGE
>   	def_bool HUGETLBFS
> -	select XARRAY_MULTI
> +	select LARGE_FOLIO
>   
>   config HUGETLB_PAGE_OPTIMIZE_VMEMMAP
>   	def_bool HUGETLB_PAGE
> diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
> index 216e1a0dd0b2..fbac554886c3 100644
> --- a/fs/hugetlbfs/inode.c
> +++ b/fs/hugetlbfs/inode.c
> @@ -258,7 +258,7 @@ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to)
>   		} else {
>   			folio_unlock(folio);
>   
> -			if (!folio_test_hwpoison(folio))
> +			if (!folio_test_has_hwpoisoned(folio))
>   				want = nr;
>   			else {
>   				/*
> diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
> index 4185a03a45cf..4fab3fbfd430 100644
> --- a/include/linux/page-flags.h
> +++ b/include/linux/page-flags.h
> @@ -893,14 +893,19 @@ static inline int PageTransCompound(const struct page *page)
>   TESTPAGEFLAG_FALSE(TransCompound, transcompound)
>   #endif
>   
> -#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_TRANSPARENT_HUGEPAGE)
> +#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_LARGE_FOLIO)
>   /*
> - * PageHasHWPoisoned indicates that at least one subpage is hwpoisoned in the
> - * compound page.
> + * folio_has_hwpoisoned indicates that at least one page is hwpoisoned in the
> + * folio.  That page will usually also have the HWPoison flag set, but this
> + * is not possible for folios which have HVO (see memory-failure for the
> + * scheme used in that case).  You probably don't want to call this directly;
> + * use folio_has_hwpoisoned_page() instead.
>    *
>    * This flag is set by hwpoison handler.  Cleared by THP split or free page.
>    */
>   FOLIO_FLAG(has_hwpoisoned, FOLIO_SECOND_PAGE)
> +FOLIO_TEST_SET_FLAG(has_hwpoisoned, FOLIO_SECOND_PAGE)
> +FOLIO_TEST_CLEAR_FLAG(has_hwpoisoned, FOLIO_SECOND_PAGE)
>   #else
>   FOLIO_FLAG_FALSE(has_hwpoisoned)
>   #endif
> @@ -1041,8 +1046,29 @@ PAGE_TYPE_OPS(Slab, slab, slab)
>   
>   #ifdef CONFIG_HUGETLB_PAGE
>   FOLIO_TYPE_OPS(hugetlb, hugetlb)
> +
> +#ifdef CONFIG_MEMORY_FAILURE
> +static inline bool folio_test_huge_poison(const struct folio *folio)
> +{
> +	return (READ_ONCE(folio->page.page_type) >> 23) ==
> +		((PGTY_hugetlb << 1) | 1);
> +}
> +
> +static inline void folio_set_huge_poison(struct folio *folio)
> +{
> +	folio->page.page_type |= (1 << 23);
> +}
> +
> +static inline void folio_clear_huge_poison(struct folio *folio)
> +{
> +	folio->page.page_type &= ~(1 << 23);
> +}
> +#else
> +FOLIO_TEST_FLAG_FALSE(huge_poison)
> +#endif
>   #else
>   FOLIO_TEST_FLAG_FALSE(hugetlb)
> +FOLIO_TEST_FLAG_FALSE(huge_poison)
>   #endif
>   
>   PAGE_TYPE_OPS(Zsmalloc, zsmalloc, zsmalloc)
> @@ -1069,9 +1095,10 @@ static inline bool PageHuge(const struct page *page)
>   }
>   
>   /*
> - * Check if a page is currently marked HWPoisoned. Note that this check is
> - * best effort only and inherently racy: there is no way to synchronize with
> - * failing hardware.
> + * Check if a page is currently marked HWPoisoned.  This check is best
> + * effort only and inherently racy: there is no way to synchronize with
> + * failing hardware.  The caller may not have a refcount on the folio
> + * containing the page, so we must be careful to not trip any assertions.
>    */
>   static inline bool is_page_hwpoison(const struct page *page)
>   {
> @@ -1080,12 +1107,15 @@ static inline bool is_page_hwpoison(const struct page *page)
>   	if (PageHWPoison(page))
>   		return true;
>   	folio = page_folio(page);
> -	return folio_test_hugetlb(folio) && PageHWPoison(&folio->page);
> +	if (folio_test_huge_poison(folio))
> +		return true;
> +	/* In case we raced with hugetlb transferring flags */
> +	return PageHWPoison(page);
>   }
>   
>   static inline bool folio_has_hwpoisoned_page(const struct folio *folio)
>   {
> -	return folio_test_hwpoison(folio) ||
> +	return PageHWPoison(&folio->page) ||
>   	       (folio_test_large(folio) && folio_test_has_hwpoisoned(folio));
>   }
>   
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 9e0ca4824905..e666dd14ca0c 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -843,11 +843,15 @@ config PERSISTENT_HUGE_ZERO_FOLIO
>   config MM_ID
>   	def_bool n
>   
> +config LARGE_FOLIO
> +	def_bool n
> +	select XARRAY_MULTI
> +
>   menuconfig TRANSPARENT_HUGEPAGE
>   	bool "Transparent Hugepage Support"
>   	depends on HAVE_ARCH_TRANSPARENT_HUGEPAGE && !PREEMPT_RT
>   	select COMPACTION
> -	select XARRAY_MULTI
> +	select LARGE_FOLIO
>   	select MM_ID
>   	help
>   	  Transparent Hugepages allows the kernel to use huge pages and
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 47403d02be88..40ae967b9ecc 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -1255,7 +1255,7 @@ static struct folio *dequeue_hugetlb_folio_node_exact(struct hstate *h,
>   		if (pin && !folio_is_longterm_pinnable(folio))
>   			continue;
>   
> -		if (folio_test_hwpoison(folio))
> +		if (folio_test_has_hwpoisoned(folio))
>   			continue;
>   
>   		if (is_migrate_isolate_page(&folio->page))
> @@ -1387,7 +1387,7 @@ static void folio_clear_hugetlb(struct folio *folio)
>   	 * Move HWPoison flag to each error page
>   	 * which makes any healthy pages reusable.
>   	 */
> -	if (unlikely(folio_test_hwpoison(folio)))
> +	if (unlikely(folio_test_has_hwpoisoned(folio)))
>   		folio_clear_hugetlb_hwpoison(folio);
>   
>   	__folio_clear_hugetlb(folio);
> @@ -4003,7 +4003,7 @@ long demote_pool_huge_page(struct hstate *src, nodemask_t *nodes_allowed,
>   		struct folio *folio, *next;
>   
>   		list_for_each_entry_safe(folio, next, &src->hugepage_freelists[node], lru) {
> -			if (folio_test_hwpoison(folio))
> +			if (folio_test_has_hwpoisoned(folio))
>   				continue;
>   
>   			remove_hugetlb_folio(src, folio, false);
> @@ -5814,7 +5814,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping,
>   		 * don't have hwpoisoned swap entry for errored virtual address.
>   		 * So we need to block hugepage fault by PG_hwpoison bit check.
>   		 */
> -		if (unlikely(folio_test_hwpoison(folio))) {
> +		if (unlikely(folio_test_has_hwpoisoned(folio))) {
>   			ret = VM_FAULT_HWPOISON_LARGE |
>   				VM_FAULT_SET_HINDEX(hstate_index(h));
>   			goto backout_unlocked;
> @@ -6323,7 +6323,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
>   	ptl = huge_pte_lock(h, dst_mm, dst_pte);
>   
>   	ret = -EIO;
> -	if (folio_test_hwpoison(folio))
> +	if (folio_test_has_hwpoisoned(folio))
>   		goto out_release_unlock;
>   
>   	ret = -EEXIST;
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 1dd0e7b99bb1..714e1b398f2c 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -1829,7 +1829,7 @@ bool is_raw_hwpoison_page_in_hugepage(struct page *page)
>   	struct folio *folio = page_folio(page);
>   	bool ret = false;
>   
> -	if (!folio_test_hwpoison(folio))
> +	if (!folio_test_has_hwpoisoned(folio))
>   		return false;
>   
>   	if (!folio_test_hugetlb(folio))
> @@ -1881,6 +1881,24 @@ static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag)
>   #define	MF_HUGETLB_FOLIO_PRE_POISONED	3	/* folio already poisoned */
>   #define	MF_HUGETLB_PAGE_PRE_POISONED	4	/* exact page already poisoned */
>   #define	MF_HUGETLB_RETRY		5	/* hugepage is busy, retry */
> +
> +static inline int hugetlb_set_poison(struct folio *folio)
> +{
> +	if (folio_test_set_has_hwpoisoned(folio))
> +		return MF_HUGETLB_FOLIO_PRE_POISONED;
> +	folio_set_huge_poison(folio);
> +	return 0;
> +}
> +
> +static inline int hugetlb_clear_poison(struct folio *folio)
> +{
> +	if (!folio_test_has_hwpoisoned(folio))
> +		return -EBUSY;
> +	folio_clear_huge_poison(folio);
> +	folio_clear_has_hwpoisoned(folio);
> +	return 0;
> +}
> +
>   /*
>    * Set hugetlb folio as hwpoisoned, update folio private raw hwpoison list
>    * to keep track of the poisoned pages.
> @@ -1890,12 +1908,12 @@ static int hugetlb_update_hwpoison(struct folio *folio, struct page *page)
>   	struct llist_head *head;
>   	struct raw_hwp_page *raw_hwp;
>   	struct raw_hwp_page *p;
> -	int ret = folio_test_set_hwpoison(folio) ? MF_HUGETLB_FOLIO_PRE_POISONED : 0;
> +	int ret = hugetlb_set_poison(folio);
>   
>   	/*
>   	 * Once the hwpoison hugepage has lost reliable raw error info,
> -	 * there is little meaning to keep additional error info precisely,
> -	 * so skip to add additional raw error info.
> +	 * there is no point in keeping additional error info precisely,
> +	 * so skip adding additional raw error info.
>   	 */
>   	if (folio_test_hugetlb_raw_hwp_unreliable(folio))
>   		return MF_HUGETLB_FOLIO_PRE_POISONED;
> @@ -1950,8 +1968,8 @@ void folio_clear_hugetlb_hwpoison(struct folio *folio)
>   		return;
>   	if (folio_test_hugetlb_vmemmap_optimized(folio))
>   		return;
> -	folio_clear_hwpoison(folio);
>   	folio_free_raw_hwp(folio, true);
> +	folio_clear_has_hwpoisoned(folio);
>   }
>   
>   static int get_huge_page_for_hwpoison(unsigned long pfn, int flags,
> @@ -2104,6 +2122,11 @@ static inline unsigned long folio_free_raw_hwp(struct folio *folio, bool flag)
>   {
>   	return 0;
>   }
> +
> +static inline int hugetlb_clear_poison(struct folio *folio)
> +{
> +	return 0;
> +}
>   #endif	/* CONFIG_HUGETLB_PAGE */
>   
>   /* Drop the extra refcount in case we come from madvise() */
> @@ -2733,8 +2756,10 @@ int unpoison_memory(unsigned long pfn)
>   				hugetlb_unlock_irq();
>   				goto unlock_mutex;
>   			}
> +			ret = hugetlb_clear_poison(folio);
> +		} else {
> +			ret = TestClearPageHWPoison(p) ? 0 : -EBUSY;
>   		}
> -		ret = folio_test_clear_hwpoison(folio) ? 0 : -EBUSY;
>   		hugetlb_unlock_irq();
>   	} else if (ghp < 0) {
>   		if (ghp == -EHWPOISON) {
> @@ -2744,17 +2769,18 @@ int unpoison_memory(unsigned long pfn)
>   			unpoison_pr_info("%#lx: failed to grab page\n",
>   					 pfn, &unpoison_rs);
>   		}
> -	} else {
> -		if (folio_test_hugetlb(folio)) {
> -			huge = true;
> -			count = folio_free_raw_hwp(folio, false);
> -			if (count == 0) {
> -				folio_put(folio);
> -				goto unlock_mutex;
> -			}
> -			p = &folio->page;
> +	} else if (folio_test_hugetlb(folio)) {
> +		huge = true;
> +		count = folio_free_raw_hwp(folio, false);
> +		if (count == 0) {
> +			folio_put(folio);
> +			goto unlock_mutex;
>   		}
> -
> +		ret = hugetlb_clear_poison(folio);
> +		folio_put(folio);
> +		if (!ret)
> +			folio_put(folio);
> +	} else {
>   		folio_put(folio);
>   		if (TestClearPageHWPoison(p)) {
>   			folio_put(folio);
> diff --git a/mm/rmap.c b/mm/rmap.c
> index 1c77d5dc06e9..fd19a0bfbfe7 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -1978,6 +1978,22 @@ static inline unsigned int folio_unmap_pte_batch(struct folio *folio,
>   				     FPB_RESPECT_WRITE | FPB_RESPECT_SOFT_DIRTY);
>   }
>   
> +/*
> + * Since we cannot split a hugetlb folio, we want to insert a poison
> + * entry into the page table for the whole folio even if only one page
> + * is poisoned.  Otherwise, we've split down to the PTE level and we only
> + * want to poison the precise page
> + */
> +static bool ttu_create_hwpoison(const struct folio *folio,
> +		const struct page *page, enum ttu_flags flags)
> +{
> +	if (!(flags & TTU_HWPOISON))
> +		return false;
> +	if (folio_test_hugetlb(folio))
> +		return folio_test_has_hwpoisoned(folio);
> +	return PageHWPoison(page);
> +}
> +
>   /*
>    * @arg: enum ttu_flags will be passed to this argument
>    */
> @@ -1993,7 +2009,6 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
>   	enum ttu_flags flags = (enum ttu_flags)(long)arg;
>   	unsigned long nr_pages = 1, end_addr;
>   	unsigned long pfn;
> -	unsigned long hsz = 0;
>   	int ptes = 0;
>   
>   	/*
> @@ -2023,9 +2038,6 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
>   		 */
>   		adjust_range_if_pmd_sharing_possible(vma, &range.start,
>   						     &range.end);
> -
> -		/* We need the huge page size for set_huge_pte_at() */
> -		hsz = huge_page_size(hstate_vma(vma));
>   	}
>   	mmu_notifier_invalidate_range_start(&range);
>   
> @@ -2121,7 +2133,8 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
>   			 * The try_to_unmap() is only passed a hugetlb page
>   			 * in the case where the hugetlb page is poisoned.
>   			 */
> -			VM_BUG_ON_PAGE(!PageHWPoison(subpage), subpage);
> +			VM_BUG_ON_FOLIO(!folio_has_hwpoisoned_page(folio),
> +					folio);
>   			/*
>   			 * huge_pmd_unshare may unmap an entire PMD page.
>   			 * There is no way of knowing exactly which PMDs may
> @@ -2200,12 +2213,12 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
>   		/* Update high watermark before we lower rss */
>   		update_hiwater_rss(mm);
>   
> -		if (PageHWPoison(subpage) && (flags & TTU_HWPOISON)) {
> +		if (ttu_create_hwpoison(folio, subpage, flags)) {
>   			pteval = swp_entry_to_pte(make_hwpoison_entry(subpage));
>   			if (folio_test_hugetlb(folio)) {
>   				hugetlb_count_sub(folio_nr_pages(folio), mm);
>   				set_huge_pte_at(mm, address, pvmw.pte, pteval,
> -						hsz);
> +						folio_size(folio));
>   			} else {
>   				dec_mm_counter(mm, mm_counter(folio));
>   				set_pte_at(mm, address, pvmw.pte, pteval);
> @@ -2423,7 +2436,6 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
>   	struct mmu_notifier_range range;
>   	enum ttu_flags flags = (enum ttu_flags)(long)arg;
>   	unsigned long pfn;
> -	unsigned long hsz = 0;
>   
>   	/*
>   	 * When racing against e.g. zap_pte_range() on another cpu,
> @@ -2452,9 +2464,6 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
>   		 */
>   		adjust_range_if_pmd_sharing_possible(vma, &range.start,
>   						     &range.end);
> -
> -		/* We need the huge page size for set_huge_pte_at() */
> -		hsz = huge_page_size(hstate_vma(vma));
>   	}
>   	mmu_notifier_invalidate_range_start(&range);
>   
> @@ -2607,14 +2616,14 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
>   		/* Update high watermark before we lower rss */
>   		update_hiwater_rss(mm);
>   
> -		if (PageHWPoison(subpage)) {
> +		if (ttu_create_hwpoison(folio, subpage, TTU_HWPOISON)) {
>   			VM_WARN_ON_FOLIO(folio_is_device_private(folio), folio);
>   
>   			pteval = swp_entry_to_pte(make_hwpoison_entry(subpage));
>   			if (folio_test_hugetlb(folio)) {
>   				hugetlb_count_sub(folio_nr_pages(folio), mm);
>   				set_huge_pte_at(mm, address, pvmw.pte, pteval,
> -						hsz);
> +						folio_size(folio));
>   			} else {
>   				dec_mm_counter(mm, mm_counter(folio));
>   				set_pte_at(mm, address, pvmw.pte, pteval);
> @@ -2644,7 +2653,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
>   			if (arch_unmap_one(mm, vma, address, pteval) < 0) {
>   				if (folio_test_hugetlb(folio))
>   					set_huge_pte_at(mm, address, pvmw.pte,
> -							pteval, hsz);
> +							pteval,
> +							folio_size(folio));
>   				else
>   					set_pte_at(mm, address, pvmw.pte, pteval);
>   				ret = false;
> @@ -2657,7 +2667,8 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
>   				if (anon_exclusive &&
>   				    hugetlb_try_share_anon_rmap(folio)) {
>   					set_huge_pte_at(mm, address, pvmw.pte,
> -							pteval, hsz);
> +							pteval,
> +							folio_size(folio));
>   					ret = false;
>   					page_vma_mapped_walk_done(&pvmw);
>   					break;
> @@ -2703,7 +2714,7 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,
>   			}
>   			if (folio_test_hugetlb(folio))
>   				set_huge_pte_at(mm, address, pvmw.pte, swp_pte,
> -						hsz);
> +						folio_size(folio));
>   			else
>   				set_pte_at(mm, address, pvmw.pte, swp_pte);
>   			trace_set_migration_pte(address, pte_val(swp_pte),
> -- 
> 2.47.3
> 

So, we now have two sets keeping track of folio level hugetlb hwpoison 
status: one with PG_has_hwpoison, another with (PGTY_hugetlb + bit23) in 
the .page_type field.  I don't see a better way though.

Reviewed-by: Jane Chu <jane.chu@oracle.com>

thanks,
-jane

Re: [PATCH v8 09/15] mm: Remove locking mf_mutex in is_raw_hwpoison_page_in_hugepage()

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> Sleeping in this kind of predicate is unexpected. Add a new spinlock to 
> protect access to the list, and turn it into a normal singly linked list 
> now that it doesn't need to be a lockless list. Rename 
> is_raw_hwpoison_page_in_hugepage() to hugetlb_page_hwpoison()
> 
> 
> Sleeping in this kind of predicate is unexpected.  Add a new spinlock
> to protect access to the list, and turn it into a normal singly linked
> list now that it doesn't need to be a lockless list.
> 
> Rename is_raw_hwpoison_page_in_hugepage() to hugetlb_page_hwpoison()
> and make it take the folio (since the callers naturally have the folio).
> Also remove the handling of non-hugetlb folios and make the arguments
> const.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
>   fs/hugetlbfs/inode.c     |   4 +-
>   include/linux/hugetlb.h  |   5 +-
>   include/linux/mm_types.h |   4 +-
>   mm/memory-failure.c      | 101 ++++++++++++++++++++++-----------------
>   4 files changed, 61 insertions(+), 53 deletions(-)
> 
> diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
> index fbac554886c3..dbca3f713bbf 100644
> --- a/fs/hugetlbfs/inode.c
> +++ b/fs/hugetlbfs/inode.c
> @@ -198,7 +198,7 @@ static size_t adjust_range_hwpoison(struct folio *folio, size_t offset,
>   	struct page *page = folio_page(folio, offset / PAGE_SIZE);
>   	size_t safe_bytes;
>   
> -	if (is_raw_hwpoison_page_in_hugepage(page))
> +	if (hugetlb_page_hwpoison(folio, page))
>   		return 0;
>   	/* Safe to read the remaining bytes in this page. */
>   	safe_bytes = PAGE_SIZE - (offset % PAGE_SIZE);
> @@ -206,7 +206,7 @@ static size_t adjust_range_hwpoison(struct folio *folio, size_t offset,
>   
>   	/* Check each remaining page as long as we are not done yet. */
>   	for (; safe_bytes < bytes; safe_bytes += PAGE_SIZE, page++)
> -		if (is_raw_hwpoison_page_in_hugepage(page))
> +		if (hugetlb_page_hwpoison(folio, page))
>   			break;
>   
>   	return min(safe_bytes, bytes);
> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index e200c93088bc..ec604cee8d22 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -1088,10 +1088,7 @@ void hugetlb_register_node(struct node *node);
>   void hugetlb_unregister_node(struct node *node);
>   #endif
>   
> -/*
> - * Check if a given raw @page in a hugepage is HWPOISON.
> - */
> -bool is_raw_hwpoison_page_in_hugepage(struct page *page);
> +bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page);
>   
>   static inline unsigned long huge_page_mask_align(struct file *file)
>   {
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index b18c2b2e7d2c..86a2d0fee557 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -387,7 +387,7 @@ typedef unsigned short mm_id_t;
>    * @_hugetlb_subpool: Do not use directly, use accessor in hugetlb.h.
>    * @_hugetlb_cgroup: Do not use directly, use accessor in hugetlb_cgroup.h.
>    * @_hugetlb_cgroup_rsvd: Do not use directly, use accessor in hugetlb_cgroup.h.
> - * @_hugetlb_hwpoison: Do not use directly, call raw_hwp_list_head().
> + * @hugetlb_hwpoison: List of pages with hwpoison.
>    * @_deferred_list: Folios to be split under memory pressure.
>    * @_unused_slab_obj_exts: Placeholder to match obj_exts in struct slab.
>    *
> @@ -499,7 +499,7 @@ struct folio {
>   			void *_hugetlb_subpool;
>   			void *_hugetlb_cgroup;
>   			void *_hugetlb_cgroup_rsvd;
> -			void *_hugetlb_hwpoison;
> +			struct hwp_page *hugetlb_hwpoison;
>   	/* private: the union with struct page is transitional */
>   		};
>   		struct page __page_3;
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 714e1b398f2c..437a2be6c58c 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -1808,63 +1808,60 @@ EXPORT_SYMBOL_GPL(mf_dax_kill_procs);
>   
>   #ifdef CONFIG_HUGETLB_PAGE
>   
> +/* Protects all lists of hwp_pages */
> +static DEFINE_SPINLOCK(hwp_page_lock);
> +
>   /*
> - * Struct raw_hwp_page represents information about "raw error page",
> - * constructing singly linked list from ->_hugetlb_hwpoison field of folio.
> + * hwp_page represents information about "error page",
> + * constructing singly linked list from folio->hugetlb_hwpoison field.
>    */
> -struct raw_hwp_page {
> -	struct llist_node node;
> +struct hwp_page {
> +	struct hwp_page *next;
>   	struct page *page;
>   };
>   
> -static inline struct llist_head *raw_hwp_list_head(struct folio *folio)
> -{
> -	return (struct llist_head *)&folio->_hugetlb_hwpoison;
> -}
> -
> -bool is_raw_hwpoison_page_in_hugepage(struct page *page)
> +/*
> + * Check if a given @page in a hugetlb folio is HWPOISON.
> + */
> +bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page)
>   {
> -	struct llist_head *raw_hwp_head;
> -	struct raw_hwp_page *p;
> -	struct folio *folio = page_folio(page);
> -	bool ret = false;
> +	const struct hwp_page *p;
> +	unsigned long flags;
>   
>   	if (!folio_test_has_hwpoisoned(folio))
>   		return false;
>   
> -	if (!folio_test_hugetlb(folio))
> -		return PageHWPoison(page);
> +	spin_lock_irqsave(&hwp_page_lock, flags);
>   
>   	/*
> -	 * When RawHwpUnreliable is set, kernel lost track of which subpages
> -	 * are HWPOISON. So return as if ALL subpages are HWPOISONed.
> +	 * When RawHwpUnreliable is set, kernel lost track of which pages
> +	 * are HWPOISON. So return as if ALL pages are HWPOISONed.
>   	 */
> -	if (folio_test_hugetlb_raw_hwp_unreliable(folio))
> +	if (folio_test_hugetlb_raw_hwp_unreliable(folio)) {
> +		spin_unlock_irqrestore(&hwp_page_lock, flags);
>   		return true;
> +	}
>   
> -	mutex_lock(&mf_mutex);
> -
> -	raw_hwp_head = raw_hwp_list_head(folio);
> -	llist_for_each_entry(p, raw_hwp_head->first, node) {
> -		if (page == p->page) {
> -			ret = true;
> +	for (p = folio->hugetlb_hwpoison; p; p = p->next) {
> +		if (page == p->page)
>   			break;
> -		}
>   	}
> +	spin_unlock_irqrestore(&hwp_page_lock, flags);
>   
> -	mutex_unlock(&mf_mutex);
> -
> -	return ret;
> +	return p != NULL;
>   }
>   
>   static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag)
>   {
> -	struct llist_node *head;
> -	struct raw_hwp_page *p, *next;
> +	struct hwp_page *p, *next;
>   	unsigned long count = 0;
>   
> -	head = llist_del_all(raw_hwp_list_head(folio));
> -	llist_for_each_entry_safe(p, next, head, node) {
> +	next = folio->hugetlb_hwpoison;
> +	folio->hugetlb_hwpoison = NULL;
> +
> +	while (next) {
> +		p = next;
> +		next = p->next;
>   		if (move_flag)
>   			SetPageHWPoison(p->page);
>   		else
> @@ -1905,9 +1902,8 @@ static inline int hugetlb_clear_poison(struct folio *folio)
>    */
>   static int hugetlb_update_hwpoison(struct folio *folio, struct page *page)
>   {
> -	struct llist_head *head;
> -	struct raw_hwp_page *raw_hwp;
> -	struct raw_hwp_page *p;
> +	struct hwp_page *p;
> +	unsigned long flags;
>   	int ret = hugetlb_set_poison(folio);
>   
>   	/*
> @@ -1917,16 +1913,23 @@ static int hugetlb_update_hwpoison(struct folio *folio, struct page *page)
>   	 */
>   	if (folio_test_hugetlb_raw_hwp_unreliable(folio))
>   		return MF_HUGETLB_FOLIO_PRE_POISONED;
> -	head = raw_hwp_list_head(folio);
> -	llist_for_each_entry(p, head->first, node) {
> +
> +	spin_lock_irqsave(&hwp_page_lock, flags);
> +	for (p = folio->hugetlb_hwpoison; p; p = p->next) {
>   		if (p->page == page)
> -			return MF_HUGETLB_PAGE_PRE_POISONED;
> +			break;
>   	}
>   
> -	raw_hwp = kmalloc_obj(struct raw_hwp_page, GFP_ATOMIC);
> -	if (raw_hwp) {
> -		raw_hwp->page = page;
> -		llist_add(&raw_hwp->node, head);
> +	if (p) {
> +		spin_unlock_irqrestore(&hwp_page_lock, flags);
> +		return MF_HUGETLB_PAGE_PRE_POISONED;
> +	}
> +
> +	p = kmalloc_obj(*p, GFP_ATOMIC);
> +	if (p) {
> +		p->page = page;
> +		p->next = folio->hugetlb_hwpoison;
> +		folio->hugetlb_hwpoison = p;
>   	} else {
>   		/*
>   		 * Failed to save raw error info.  We no longer trace all
> @@ -1935,16 +1938,20 @@ static int hugetlb_update_hwpoison(struct folio *folio, struct page *page)
>   		 */
>   		folio_set_hugetlb_raw_hwp_unreliable(folio);
>   		/*
> -		 * Once hugetlb_raw_hwp_unreliable is set, raw_hwp_page is not
> +		 * Once hugetlb_raw_hwp_unreliable is set, hwp_page is not
>   		 * used any more, so free it.
>   		 */
>   		__folio_free_raw_hwp(folio, false);
>   	}
> +	spin_unlock_irqrestore(&hwp_page_lock, flags);
>   	return ret;
>   }
>   
>   static unsigned long folio_free_raw_hwp(struct folio *folio, bool move_flag)
>   {
> +	unsigned long count;
> +	unsigned long flags;
> +
>   	/*
>   	 * hugetlb_vmemmap_optimized hugepages can't be freed because struct
>   	 * pages for tail pages are required but they don't exist.
> @@ -1959,7 +1966,11 @@ static unsigned long folio_free_raw_hwp(struct folio *folio, bool move_flag)
>   	if (folio_test_hugetlb_raw_hwp_unreliable(folio))
>   		return 0;
>   
> -	return __folio_free_raw_hwp(folio, move_flag);
> +	spin_lock_irqsave(&hwp_page_lock, flags);
> +	count = __folio_free_raw_hwp(folio, move_flag);
> +	spin_unlock_irqrestore(&hwp_page_lock, flags);
> +
> +	return count;
>   }
>   
>   void folio_clear_hugetlb_hwpoison(struct folio *folio)
> -- 
> 2.47.3
> 

Reviewed-by: Jane Chu <jane.chu@oracle.com>

thanks,
-jane

Re: [PATCH v8 10/15] mm: Check individual hugetlb pages for poison

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> So far, is_page_hwpoison() has returned true for the entire poisoned 
> hugetlb folio, but we can usually allow access to the pages which aren't 
> poisoned. It's slightly tricky due to not having a reference to the 
> folio, but the hugetlb lock can
> 
> 
> So far, is_page_hwpoison() has returned true for the entire poisoned
> hugetlb folio, but we can usually allow access to the pages which
> aren't poisoned.  It's slightly tricky due to not having a reference
> to the folio, but the hugetlb lock can save us here.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
>   include/linux/page-flags.h |  4 +++-
>   mm/memory-failure.c        | 43 +++++++++++++++++++++++++++++++-------
>   2 files changed, 39 insertions(+), 8 deletions(-)
> 
> diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
> index 4fab3fbfd430..ee026ba9b0b5 100644
> --- a/include/linux/page-flags.h
> +++ b/include/linux/page-flags.h
> @@ -1094,6 +1094,8 @@ static inline bool PageHuge(const struct page *page)
>   	return folio_test_hugetlb(page_folio(page));
>   }
>   
> +bool hugetlb_unref_page_hwpoison(const struct page *page);
> +
>   /*
>    * Check if a page is currently marked HWPoisoned.  This check is best
>    * effort only and inherently racy: there is no way to synchronize with
> @@ -1108,7 +1110,7 @@ static inline bool is_page_hwpoison(const struct page *page)
>   		return true;
>   	folio = page_folio(page);
>   	if (folio_test_huge_poison(folio))
> -		return true;
> +		return hugetlb_unref_page_hwpoison(page);
>   	/* In case we raced with hugetlb transferring flags */
>   	return PageHWPoison(page);
>   }
> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
> index 437a2be6c58c..c0a402522c21 100644
> --- a/mm/memory-failure.c
> +++ b/mm/memory-failure.c
> @@ -1820,17 +1820,12 @@ struct hwp_page {
>   	struct page *page;
>   };
>   
> -/*
> - * Check if a given @page in a hugetlb folio is HWPOISON.
> - */
> -bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page)
> +static bool precise_page_poisoned(const struct folio *folio,
> +		const struct page *page)
>   {
>   	const struct hwp_page *p;
>   	unsigned long flags;
>   
> -	if (!folio_test_has_hwpoisoned(folio))
> -		return false;
> -
>   	spin_lock_irqsave(&hwp_page_lock, flags);
>   
>   	/*
> @@ -1851,6 +1846,40 @@ bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page)
>   	return p != NULL;
>   }
>   
> +/*
> + * Check if a given @page in a hugetlb folio is HWPOISON.
> + */
> +bool hugetlb_page_hwpoison(const struct folio *folio, const struct page *page)
> +{
> +	if (!folio_test_has_hwpoisoned(folio))
> +		return false;
> +
> +	return precise_page_poisoned(folio, page);
> +}
> +
> +/*
> + * We have no reference on the folio containing this page.
> + * The hugetlb_lock keeps hugetlb folios from being freed.
> + */
> +bool hugetlb_unref_page_hwpoison(const struct page *page)
> +{
> +	const struct folio *folio;
> +	unsigned long flags;
> +	bool ret;
> +
> +	spin_lock_irqsave(&hugetlb_lock, flags);
> +	folio = page_folio(page);
> +	if (!folio_test_huge_poison(folio)) {
> +		ret = PageHWPoison(page);
> +		goto unlock;
> +	}
> +
> +	ret = precise_page_poisoned(folio, page);
> +unlock:
> +	spin_unlock_irqrestore(&hugetlb_lock, flags);
> +	return ret;
> +}
> +
>   static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag)
>   {
>   	struct hwp_page *p, *next;
> -- 
> 2.47.3
> 

Reviewed-by: Jane Chu <jane.chu@oracle.com>

thanks,
-jane

Re: [PATCH v8 12/15] filemap: Remove checks in mapping_set_folio_order_range()

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> These checks make it impossible to use mapping_set_folio_order_range() 
> in hugetlb. They add very little value and can simply be removed. 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@ infradead. org> --- 
> include/linux/pagemap. h | 34 ++++++++++++++--------------------
> 
> 
> These checks make it impossible to use mapping_set_folio_order_range()
> in hugetlb.  They add very little value and can simply be removed.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
>   include/linux/pagemap.h | 34 ++++++++++++++--------------------
>   1 file changed, 14 insertions(+), 20 deletions(-)
> 
> diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
> index 2c3718d592d6..968b791cfd14 100644
> --- a/include/linux/pagemap.h
> +++ b/include/linux/pagemap.h
> @@ -373,7 +373,7 @@ static inline void mapping_set_gfp_mask(struct address_space *m, gfp_t mask)
>    * assumptions about maximum order if THP are disabled, but 8 seems like
>    * a good order (that's 1MB if you're using 4kB pages)
>    */
> -#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> +#ifdef CONFIG_LARGE_FOLIO
>   #define PREFERRED_MAX_PAGECACHE_ORDER	HPAGE_PMD_ORDER
>   #else
>   #define PREFERRED_MAX_PAGECACHE_ORDER	8
> @@ -394,7 +394,7 @@ static inline void mapping_set_gfp_mask(struct address_space *m, gfp_t mask)
>    */
>   static inline size_t mapping_max_folio_size_supported(void)
>   {
> -	if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
> +	if (IS_ENABLED(CONFIG_LARGE_FOLIO))
>   		return 1U << (PAGE_SHIFT + MAX_PAGECACHE_ORDER);
>   	return PAGE_SIZE;
>   }
> @@ -405,29 +405,23 @@ static inline size_t mapping_max_folio_size_supported(void)
>    * @min: Minimum folio order (between 0-MAX_PAGECACHE_ORDER inclusive).
>    * @max: Maximum folio order (between @min-MAX_PAGECACHE_ORDER inclusive).
>    *
> - * The filesystem should call this function in its inode constructor to
> - * indicate which base size (min) and maximum size (max) of folio the VFS
> - * can use to cache the contents of the file.  This should only be used
> - * if the filesystem needs special handling of folio sizes (ie there is
> - * something the core cannot know).
> + * The filesystem should call this function in its inode constructor
> + * to indicate which size folios can be used to cache the contents of
> + * the inode.  This should only be used if the filesystem needs special
> + * handling of folio sizes (ie there is something the core cannot know).
>    * Do not tune it based on, eg, i_size.
>    *
> + * hugetlb calls this with orders larger than MAX_PAGECACHE_ORDER.
> + * Normal filesystems should not do this.
> + *
>    * Context: This should not be called while the inode is active as it
>    * is non-atomic.
>    */
>   static inline void mapping_set_folio_order_range(struct address_space *mapping,
> -						 unsigned int min,
> -						 unsigned int max)
> +		unsigned int min, unsigned int max)
>   {
> -	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
> +	if (!IS_ENABLED(CONFIG_LARGE_FOLIO))
>   		return;
> -
> -	if (min > MAX_PAGECACHE_ORDER)
> -		min = MAX_PAGECACHE_ORDER;
> -
> -	if (max > MAX_PAGECACHE_ORDER)
> -		max = MAX_PAGECACHE_ORDER;
> -
>   	if (max < min)
>   		max = min;
>   
> @@ -460,7 +454,7 @@ static inline void mapping_set_large_folios(struct address_space *mapping)
>   static inline unsigned int
>   mapping_max_folio_order(const struct address_space *mapping)
>   {
> -	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
> +	if (!IS_ENABLED(CONFIG_LARGE_FOLIO))
>   		return 0;
>   	return (mapping->flags & AS_FOLIO_ORDER_MAX_MASK) >> AS_FOLIO_ORDER_MAX;
>   }
> @@ -468,7 +462,7 @@ mapping_max_folio_order(const struct address_space *mapping)
>   static inline unsigned int
>   mapping_min_folio_order(const struct address_space *mapping)
>   {
> -	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE))
> +	if (!IS_ENABLED(CONFIG_LARGE_FOLIO))
>   		return 0;
>   	return (mapping->flags & AS_FOLIO_ORDER_MIN_MASK) >> AS_FOLIO_ORDER_MIN;
>   }
> @@ -524,7 +518,7 @@ static inline bool mapping_large_folio_support(const struct address_space *mappi
>    *
>    * Return: True if PMD-sized folios are supported, otherwise false.
>    */
> -#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> +#ifdef CONFIG_LARGE_FOLIO
>   static inline bool mapping_pmd_folio_support(const struct address_space *mapping)
>   {
>   	/* AS_FOLIO_ORDER is only reasonable for pagecache folios */
> -- 
> 2.47.3
> 

Nice.
Reviewed-by: Jane Chu <jane.chu@oracle.com>

thanks,
-jane

Re: [PATCH v8 13/15] hugetlb: Set mapping folio order

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> hugetlbfs currently uses an fs-specific way to determine the size of the 
> folios in its page cache. Thanks to the support of block sizes larger 
> than page sizes, we now support this as part of the page cache. It's 
> somewhat more efficient as we
> 
> 
> hugetlbfs currently uses an fs-specific way to determine the size of
> the folios in its page cache.  Thanks to the support of block sizes
> larger than page sizes, we now support this as part of the page cache.
> It's somewhat more efficient as we have this information directly in the
> mapping rather than going from inode->i_sb->s_fs_info.  We can convert
> the rest of the hugetlb code to use this at our leisure; this is needed
> now so that we can use filemap_get_pages() unmodified.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Reviewed-by: Gregory Price <gourry@gourry.net>
> ---
>   fs/hugetlbfs/inode.c | 13 +++++++++++--
>   1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
> index dbca3f713bbf..8d15413a67ee 100644
> --- a/fs/hugetlbfs/inode.c
> +++ b/fs/hugetlbfs/inode.c
> @@ -883,6 +883,16 @@ static struct inode *hugetlbfs_get_root(struct super_block *sb,
>   	return inode;
>   }
>   
> +static void hugetlbfs_init_regular_inode(struct inode *inode)
> +{
> +	struct hstate *hstate = hstate_inode(inode);
> +	unsigned int order = huge_page_order(hstate);
> +
> +	inode->i_op = &hugetlbfs_inode_operations;
> +	inode->i_fop = &hugetlbfs_file_operations;
> +	mapping_set_folio_order_range(inode->i_mapping, order, order);
> +}
> +
>   /*
>    * Hugetlbfs is not reclaimable; therefore its i_mmap_rwsem will never
>    * be taken from reclaim -- unlike regular filesystems. This needs an
> @@ -926,8 +936,7 @@ static struct inode *hugetlbfs_get_inode(struct super_block *sb,
>   			init_special_inode(inode, mode, dev);
>   			break;
>   		case S_IFREG:
> -			inode->i_op = &hugetlbfs_inode_operations;
> -			inode->i_fop = &hugetlbfs_file_operations;
> +			hugetlbfs_init_regular_inode(inode);
>   			break;
>   		case S_IFDIR:
>   			inode->i_op = &hugetlbfs_dir_inode_operations;
> -- 
> 2.47.3
> 

Good idea!

Reviewed-by: Jane Chu <jane.chu@oracle.com>

thanks,
-jane

Re: [PATCH v8 14/15] filemap: Add support for authoritative mappings

[jane.chu]

On 7/31/2026 1:07 PM, Matthew Wilcox (Oracle) wrote:
> An authoritative mapping knows about all the folios in the mapping. If 
> read() finds a missing folio, there's no reason to allocate one and try 
> to read it because we know it's a zero region of the file. We can just 
> call iov_iter_zero() instead. 
> 
> 
> An authoritative mapping knows about all the folios in the mapping.
> If read() finds a missing folio, there's no reason to allocate one and
> try to read it because we know it's a zero region of the file.  We can
> just call iov_iter_zero() instead.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
>   include/linux/pagemap.h | 11 +++++++++++
>   mm/filemap.c            | 18 ++++++++++++++++++
>   2 files changed, 29 insertions(+)
> 
> diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
> index 968b791cfd14..22d48935ffda 100644
> --- a/include/linux/pagemap.h
> +++ b/include/linux/pagemap.h
> @@ -210,6 +210,7 @@ enum mapping_flags {
>   	AS_WRITEBACK_MAY_DEADLOCK_ON_RECLAIM = 9,
>   	AS_KERNEL_FILE = 10,	/* mapping for a fake kernel file that shouldn't
>   				   account usage to user cgroups */
> +	AS_AUTHORITATIVE = 11,	/* If we miss in the page cache, it's a hole */
>   	/* Bits 16-25 are used for FOLIO_ORDER */
>   	AS_FOLIO_ORDER_BITS = 5,
>   	AS_FOLIO_ORDER_MIN = 16,
> @@ -345,6 +346,16 @@ static inline bool mapping_writeback_may_deadlock_on_reclaim(const struct addres
>   	return test_bit(AS_WRITEBACK_MAY_DEADLOCK_ON_RECLAIM, &mapping->flags);
>   }
>   
> +static inline void mapping_set_authoritative(struct address_space *mapping)
> +{
> +	set_bit(AS_AUTHORITATIVE, &mapping->flags);
> +}
> +
> +static inline bool mapping_is_authoritative(const struct address_space *mapping)
> +{
> +	return test_bit(AS_AUTHORITATIVE, &mapping->flags);
> +}
> +
>   static inline gfp_t mapping_gfp_mask(const struct address_space *mapping)
>   {
>   	return mapping->gfp_mask;
> diff --git a/mm/filemap.c b/mm/filemap.c
> index 26a5f18121f9..5a8cc20e624e 100644
> --- a/mm/filemap.c
> +++ b/mm/filemap.c
> @@ -2697,6 +2697,8 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count,
>   	if (!folio_batch_count(fbatch)) {
>   		DEFINE_READAHEAD(ractl, filp, &filp->f_ra, mapping, index);
>   
> +		if (mapping_is_authoritative(mapping))
> +			return 0;
>   		if (iocb->ki_flags & IOCB_NOIO)
>   			return -EAGAIN;
>   		if (iocb->ki_flags & IOCB_NOWAIT)
> @@ -2853,6 +2855,22 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter,
>   			goto put_folios;
>   		end_offset = min_t(loff_t, isize, iocb->ki_pos + iter->count);
>   
> +		if (!folio_batch_count(&fbatch)) {
> +			size_t fsize = mapping_min_folio_nrbytes(mapping);
> +			size_t offset = iocb->ki_pos & (fsize - 1);
> +			size_t bytes = min_t(loff_t, end_offset - iocb->ki_pos,
> +					     fsize - offset);
> +			size_t copied = iov_iter_zero(bytes, iter);
> +
> +			already_read += copied;
> +			iocb->ki_pos += copied;
> +			last_pos = iocb->ki_pos;
> +
> +			if (copied < bytes)
> +				error = -EFAULT;
> +			continue;
> +		}
> +
>   		/*
>   		 * Once we start copying data, we don't want to be touching any
>   		 * cachelines that might be contended:
> -- 
> 2.47.3
> 

Very nice change.

Reviewed-by: Jane Chu <jane.chu@oracle.com>

thanks,
-jane

Re: [PATCH v8 02/15] memory-failure: Prevent hugetlb freeing during unpoisoning

[Matthew Wilcox]

On Mon, Aug 03, 2026 at 11:40:25PM -0700, jane.chu@oracle.com wrote:
> Patch itself looks good, so Reviewed-by: Jane Chu <jane.chu@oracle.com>

Thanks!

> That said, there is a pre-existing issue:
> folio_free_raw_hwp() should check HPG_raw_hwp_unreliable, and fail the act
> of unpoison just like what __update_and_free_hugetlb_folio() does -
> 
> static void __update_and_free_hugetlb_folio(struct hstate *h,
>                                                 struct folio *folio)
> {
>         bool clear_flag = folio_test_hugetlb_vmemmap_optimized(folio);
> 
>         if (hstate_is_gigantic_no_runtime(h))
>                 return;
> 
>         /*
>          * If we don't know which subpages are hwpoisoned, we can't free
>          * the hugepage, so it's leaked intentionally.
>          */
>         if (folio_test_hugetlb_raw_hwp_unreliable(folio))
>                 return;

I'm not sure I agree with that, but I'm new to hugetlb & hwpoison
handling, so please check my reasoning.

We set the unreliable bit when memory allocation fails (ie we can't
allocate a raw_hwp_page).  That does mean that if we have a real hardware
error, we can't know which page was poisoned, but for a simulated hwpoison
which we are now clearing, it doesn't matter which page specifically
contained the hwpoison.  We're going to unpoison all of the pages in
this hugetlb folio anyway.

Maybe there's more to it than that.

Re: [PATCH v8 06/15] kpageflags: Use is_page_hwpoison() to set KPF_HWPOISON

[Matthew Wilcox]

On Mon, Aug 03, 2026 at 11:44:28PM -0700, jane.chu@oracle.com wrote:
> Good idea, but agreed with Sushiko about the bit setting issue,
> with that fixed,
> 
> Reviewed-by: Jane Chu <jane.chu@oracle.com>

Thanks.  That's the only diff I have between v8 and v9  at this point:

        if (is_page_hwpoison(page))
-               u |= KPF_HWPOISON;
+               u |= 1 << KPF_HWPOISON;

Re: [PATCH v8 08/15] hugetlb: Use the has_hwpoisoned flag

[Matthew Wilcox]

On Mon, Aug 03, 2026 at 11:51:42PM -0700, jane.chu@oracle.com wrote:
> So, we now have two sets keeping track of folio level hugetlb hwpoison
> status: one with PG_has_hwpoison, another with (PGTY_hugetlb + bit23) in the
> .page_type field.  I don't see a better way though.

Yep, we end up marking the folio two different ways.  I don't love it
either, but I haven't come up with anything better.

> Reviewed-by: Jane Chu <jane.chu@oracle.com>
> 
> thanks,
> -jane

Re: [PATCH v8 06/15] kpageflags: Use is_page_hwpoison() to set KPF_HWPOISON

[Gregory Price]

On Fri, Jul 31, 2026 at 09:07:51PM +0100, Matthew Wilcox (Oracle) wrote:
> Instead of knowing how hugetlb handles hwpoison, just ask
> is_page_hwpoison().  This gives us the flexibility to change how hwpoison
> is handled without updating this function in the future.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>

Nothing to add, love -ifdef's :]

Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>

> ---
>  fs/proc/page.c | 8 ++------
>  1 file changed, 2 insertions(+), 6 deletions(-)
> 
> diff --git a/fs/proc/page.c b/fs/proc/page.c
> index 7d9387143435..28c9a7b724f3 100644
> --- a/fs/proc/page.c
> +++ b/fs/proc/page.c
> @@ -233,12 +233,8 @@ u64 stable_page_flags(const struct page *page)
>  	u |= kpf_copy_bit(k, KPF_UNEVICTABLE,	PG_unevictable);
>  	u |= kpf_copy_bit(k, KPF_MLOCKED,	PG_mlocked);
>  
> -#ifdef CONFIG_MEMORY_FAILURE
> -	if (u & (1 << KPF_HUGE))
> -		u |= kpf_copy_bit(k, KPF_HWPOISON,	PG_hwpoison);
> -	else
> -		u |= kpf_copy_bit(ps.page_snapshot.flags.f, KPF_HWPOISON, PG_hwpoison);
> -#endif
> +	if (is_page_hwpoison(page))
> +		u |= KPF_HWPOISON;
>  
>  	u |= kpf_copy_bit(k, KPF_RESERVED,	PG_reserved);
>  	u |= kpf_copy_bit(k, KPF_OWNER_2,	PG_owner_2);
> -- 
> 2.47.3
> 
> 

Re: [PATCH v8 08/15] hugetlb: Use the has_hwpoisoned flag

[Gregory Price]

On Fri, Jul 31, 2026 at 09:07:53PM +0100, Matthew Wilcox (Oracle) wrote:
> Other large folios use the has_hwpoisoned flag.  Convert hugetlb to match.
> This will help us use the per-page hwpoison flag in the future.
> 
> Also introduce a folio_test_huge_poison().  This has exactly the same
> meaning as folio_test_has_hwpoisoned() but can be used when we don't have
> a reference to the folio containing the page.  folio_test_has_hwpoisoned()
> can race with folio splitting / reallocation and trip the assertions
> in const_folio_flags().
> 
> This closes a gap where a page in a previously-poisoned hugetlb folio
> could be observed to not have the hwpoison bit set.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>

one question inline, but this seems straightforward.

Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>

> diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
> index 4185a03a45cf..4fab3fbfd430 100644
> --- a/include/linux/page-flags.h
> +++ b/include/linux/page-flags.h
> @@ -893,14 +893,19 @@ static inline int PageTransCompound(const struct page *page)
>  TESTPAGEFLAG_FALSE(TransCompound, transcompound)
>  #endif
>  
> -#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_TRANSPARENT_HUGEPAGE)
> +#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_LARGE_FOLIO)
>  /*
> - * PageHasHWPoisoned indicates that at least one subpage is hwpoisoned in the
> - * compound page.
> + * folio_has_hwpoisoned indicates that at least one page is hwpoisoned in the
> + * folio.  That page will usually also have the HWPoison flag set, but this
> + * is not possible for folios which have HVO (see memory-failure for the
> + * scheme used in that case).  You probably don't want to call this directly;
> + * use folio_has_hwpoisoned_page() instead.
>   *
>   * This flag is set by hwpoison handler.  Cleared by THP split or free page.
>   */

Had to google HVO, and didn't see HVO directly mentioned in
memory-failure.  Maybe worth expanding HVO to its actual definition and
a sentence on the quirk.

Does "Hugepage Vmapmem Optimization" just report poison for the head and
we lose the sub-page poison information in general?

~Gregory

Re: [PATCH v8 09/15] mm: Remove locking mf_mutex in is_raw_hwpoison_page_in_hugepage()

[Gregory Price]

On Fri, Jul 31, 2026 at 09:07:54PM +0100, Matthew Wilcox (Oracle) wrote:
> Sleeping in this kind of predicate is unexpected.  Add a new spinlock
> to protect access to the list, and turn it into a normal singly linked
> list now that it doesn't need to be a lockless list.
> 

Out of curiosity, wouldn't sleeping in this context not just be
unexpected but theoretically cause issues? (mid-poison, sleep, scheduled
process generates more poison on the same page...) - at a minimum this
takes a thread out of the future potential poison-generating pool?

I don't see any particular issue with the patch, just curious the
implication of removable sleepability from this context.

> Rename is_raw_hwpoison_page_in_hugepage() to hugetlb_page_hwpoison()
> and make it take the folio (since the callers naturally have the folio).
> Also remove the handling of non-hugetlb folios and make the arguments
> const.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>

Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>

Re: [PATCH v8 08/15] hugetlb: Use the has_hwpoisoned flag

[Matthew Wilcox]

On Tue, Aug 04, 2026 at 12:28:06PM -0400, Gregory Price wrote:
> Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>

Thanks, will add.

> > -#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_TRANSPARENT_HUGEPAGE)
> > +#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_LARGE_FOLIO)
> >  /*
> > - * PageHasHWPoisoned indicates that at least one subpage is hwpoisoned in the
> > - * compound page.
> > + * folio_has_hwpoisoned indicates that at least one page is hwpoisoned in the
> > + * folio.  That page will usually also have the HWPoison flag set, but this
> > + * is not possible for folios which have HVO (see memory-failure for the
> > + * scheme used in that case).  You probably don't want to call this directly;
> > + * use folio_has_hwpoisoned_page() instead.
> >   *
> >   * This flag is set by hwpoison handler.  Cleared by THP split or free page.
> >   */
> 
> Had to google HVO, and didn't see HVO directly mentioned in
> memory-failure.  Maybe worth expanding HVO to its actual definition and
> a sentence on the quirk.

It is mentioned earlier in the same file:

include/linux/page-flags.h:      * Limit mask usage to HugeTLB vmemmap optimization (HVO) where it

but that's not obvious when reading a patch.  I'll just expand it here:

 * The has_hwpoisoned flag indicates that at least one page is hwpoisoned
 * in the folio.  That page will usually also have the HWPoison flag
 * set, but this is not possible for folios which have HugeTLB vmemmap
 * optimization (see hugetlb_update_hwpoison() for the scheme used
 * in that case).  You probably don't want to call this directly; use
 * folio_has_hwpoisoned_page() instead.

> Does "Hugepage Vmapmem Optimization" just report poison for the head and
> we lose the sub-page poison information in general?

We only lose the sub-folio poison information if kmalloc fails in
hugetlb_update_hwpoison().  In the happy case, we retain knowledge
of which pages contain hwpoison (and later in this series, we actually
expose that information instead of just pretending the whole folio is
poisoned)

Re: [PATCH v8 09/15] mm: Remove locking mf_mutex in is_raw_hwpoison_page_in_hugepage()

[Matthew Wilcox]

On Tue, Aug 04, 2026 at 12:36:00PM -0400, Gregory Price wrote:
> On Fri, Jul 31, 2026 at 09:07:54PM +0100, Matthew Wilcox (Oracle) wrote:
> > Sleeping in this kind of predicate is unexpected.  Add a new spinlock
> > to protect access to the list, and turn it into a normal singly linked
> > list now that it doesn't need to be a lockless list.
> 
> Out of curiosity, wouldn't sleeping in this context not just be
> unexpected but theoretically cause issues? (mid-poison, sleep, scheduled
> process generates more poison on the same page...) - at a minimum this
> takes a thread out of the future potential poison-generating pool?

A reasonable question.  We don't try to handle races terribly well --
hwpoison can be generated asynchronously by patrol scrub, so it's always
going to be possible for a thread to hit poison on something that the
kernel didn't know about.

What we're trying to do is avoid touching memory which we already knew
to be poisoned, while also permitting touching memory which we haven't
been told is poison.

I see the locking in here as preventing list corruption and UAF rather
than preventing races.  I'm also considering that we're now exposing this
mutex fairly directly to userspace -- before it was only being touched
through some fairly weird mechanisms like reading kcore.  With this
patch series, every call to read() potentially touches this mutex.
And that could hold off recording any hwpoison for some time, not to
mention serialising all other calls to read() on a given folio.

So we have all manner of protections that prevent us from taking this mutex
-- the folio must have a hwpoison page in it and must be hugetlb, but at
the end of the day, one can still be preempted while holding a mutex and
we never know when we might want to call this from a non-sleepable context.

I'd be open to arguments that this should be an rwlock rather than
a spinlock.  Or figure out a way to shard the lock per folio (lockdep
etc make it very hard to embed a spinlock in struct folio).  I think
it'd also be profitable to change how we record poison for hugetlb to
not need locking or memory allocation.

But this is all a very long way from where I want to be working -- cleaning
up the page fault path.  I think I've made a reasonable set of improvements
here, and somebody else can come along later to make the code even better.

Re: [PATCH v8 08/15] hugetlb: Use the has_hwpoisoned flag

[Gregory Price]

On Tue, Aug 04, 2026 at 05:42:28PM +0100, Matthew Wilcox wrote:
> On Tue, Aug 04, 2026 at 12:28:06PM -0400, Gregory Price wrote:
> > Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
> 
> Thanks, will add.
> 
> > > -#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_TRANSPARENT_HUGEPAGE)
> > > +#if defined(CONFIG_MEMORY_FAILURE) && defined(CONFIG_LARGE_FOLIO)
> > >  /*
> > > - * PageHasHWPoisoned indicates that at least one subpage is hwpoisoned in the
> > > - * compound page.
> > > + * folio_has_hwpoisoned indicates that at least one page is hwpoisoned in the
> > > + * folio.  That page will usually also have the HWPoison flag set, but this
> > > + * is not possible for folios which have HVO (see memory-failure for the
> > > + * scheme used in that case).  You probably don't want to call this directly;
> > > + * use folio_has_hwpoisoned_page() instead.
> > >   *
> > >   * This flag is set by hwpoison handler.  Cleared by THP split or free page.
> > >   */
> > 
> > Had to google HVO, and didn't see HVO directly mentioned in
> > memory-failure.  Maybe worth expanding HVO to its actual definition and
> > a sentence on the quirk.
> 
> It is mentioned earlier in the same file:
> 
> include/linux/page-flags.h:      * Limit mask usage to HugeTLB vmemmap optimization (HVO) where it
>
> but that's not obvious when reading a patch.  I'll just expand it here:
> 
>  * The has_hwpoisoned flag indicates that at least one page is hwpoisoned
>  * in the folio.  That page will usually also have the HWPoison flag
>  * set, but this is not possible for folios which have HugeTLB vmemmap
>  * optimization (see hugetlb_update_hwpoison() for the scheme used
>  * in that case).  You probably don't want to call this directly; use
>  * folio_has_hwpoisoned_page() instead.
> 
> > Does "Hugepage Vmapmem Optimization" just report poison for the head and
> > we lose the sub-page poison information in general?
> 
> We only lose the sub-folio poison information if kmalloc fails in
> hugetlb_update_hwpoison().  In the happy case, we retain knowledge
> of which pages contain hwpoison (and later in this series, we actually
> expose that information instead of just pretending the whole folio is
> poisoned)
> 

Gotcha, thank you for the additional context.

~Gregory

Re: [PATCH v8 09/15] mm: Remove locking mf_mutex in is_raw_hwpoison_page_in_hugepage()

[Gregory Price]

On Tue, Aug 04, 2026 at 05:58:26PM +0100, Matthew Wilcox wrote:
> On Tue, Aug 04, 2026 at 12:36:00PM -0400, Gregory Price wrote:
> 
> So we have all manner of protections that prevent us from taking this mutex
> -- the folio must have a hwpoison page in it and must be hugetlb, but at
> the end of the day, one can still be preempted while holding a mutex and
> we never know when we might want to call this from a non-sleepable context.
> 
> I'd be open to arguments that this should be an rwlock rather than
> a spinlock.  Or figure out a way to shard the lock per folio (lockdep
> etc make it very hard to embed a spinlock in struct folio).  I think
> it'd also be profitable to change how we record poison for hugetlb to
> not need locking or memory allocation.
>

I think such complexity needs to be driven by an actual demonstration
the spinlock is problematic.  It seems like if you're hitting devastating
contention here, you have much bigger (hardware) problems.

> But this is all a very long way from where I want to be working -- cleaning
> up the page fault path.  I think I've made a reasonable set of improvements
> here, and somebody else can come along later to make the code even better.
> 

Agreed.

~Gregory

Re: [PATCH v8 10/15] mm: Check individual hugetlb pages for poison

[Gregory Price]

On Fri, Jul 31, 2026 at 09:07:55PM +0100, Matthew Wilcox (Oracle) wrote:
> +/*
> + * We have no reference on the folio containing this page.
> + * The hugetlb_lock keeps hugetlb folios from being freed.
> + */
> +bool hugetlb_unref_page_hwpoison(const struct page *page)
> +{
> +	const struct folio *folio;
> +	unsigned long flags;
> +	bool ret;
> +
> +	spin_lock_irqsave(&hugetlb_lock, flags);
> +	folio = page_folio(page);
> +	if (!folio_test_huge_poison(folio)) {
> +		ret = PageHWPoison(page);
> +		goto unlock;
> +	}
> +
> +	ret = precise_page_poisoned(folio, page);
> +unlock:
> +	spin_unlock_irqrestore(&hugetlb_lock, flags);
> +	return ret;
> +}
> +

I ended up with the same question as sashiko - i think this behavior
implies we must hold a reference on the page, otherwise the folio can
be invalid and all these accesses are unsafe.

But that must be the existing behavior for poison checks like this, so
this is at least no worse. Not sure it's worth addressing.

Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>

Re: [PATCH v8 06/15] kpageflags: Use is_page_hwpoison() to set KPF_HWPOISON

[jane.chu]

On 8/4/2026 8:46 AM, Matthew Wilcox wrote:
> On Mon, Aug 03, 2026 at 11: 44: 28PM -0700, jane. chu@ oracle. com 
> wrote: > Good idea, but agreed with Sushiko about the bit setting issue, 
>  > with that fixed, > > Reviewed-by: Jane Chu <jane. chu@ oracle. com> 
> Thanks. That's the only
> 
> 
> On Mon, Aug 03, 2026 at 11:44:28PM -0700, jane.chu@oracle.com wrote:
>> Good idea, but agreed with Sushiko about the bit setting issue,
>> with that fixed,
>> 
>> Reviewed-by: Jane Chu <jane.chu@oracle.com>
> 
> Thanks.  That's the only diff I have between v8 and v9  at this point:
> 
>          if (is_page_hwpoison(page))
> -               u |= KPF_HWPOISON;
> +               u |= 1 << KPF_HWPOISON;
> 
> 

Looks good.

Thanks!
-jane

Re: [PATCH v8 09/15] mm: Remove locking mf_mutex in is_raw_hwpoison_page_in_hugepage()

[jane.chu]

On 8/4/2026 9:58 AM, Matthew Wilcox wrote:
> On Tue, Aug 04, 2026 at 12: 36: 00PM -0400, Gregory Price wrote: > On 
> Fri, Jul 31, 2026 at 09: 07: 54PM +0100, Matthew Wilcox (Oracle) wrote: 
>  > > Sleeping in this kind of predicate is unexpected. Add a new 
> spinlock > > to protect
> 
> 
> On Tue, Aug 04, 2026 at 12:36:00PM -0400, Gregory Price wrote:
>> On Fri, Jul 31, 2026 at 09:07:54PM +0100, Matthew Wilcox (Oracle) wrote:
>> > Sleeping in this kind of predicate is unexpected.  Add a new spinlock
>> > to protect access to the list, and turn it into a normal singly linked
>> > list now that it doesn't need to be a lockless list.
>> 
>> Out of curiosity, wouldn't sleeping in this context not just be
>> unexpected but theoretically cause issues? (mid-poison, sleep, scheduled
>> process generates more poison on the same page...) - at a minimum this
>> takes a thread out of the future potential poison-generating pool?
> 
> A reasonable question.  We don't try to handle races terribly well --
> hwpoison can be generated asynchronously by patrol scrub, so it's always
> going to be possible for a thread to hit poison on something that the
> kernel didn't know about.
> 
> What we're trying to do is avoid touching memory which we already knew
> to be poisoned, while also permitting touching memory which we haven't
> been told is poison.
> 
> I see the locking in here as preventing list corruption and UAF rather
> than preventing races.  I'm also considering that we're now exposing this
> mutex fairly directly to userspace -- before it was only being touched
> through some fairly weird mechanisms like reading kcore.  With this
> patch series, every call to read() potentially touches this mutex.
> And that could hold off recording any hwpoison for some time, not to
> mention serialising all other calls to read() on a given folio.
> 
> So we have all manner of protections that prevent us from taking this mutex
> -- the folio must have a hwpoison page in it and must be hugetlb, but at
> the end of the day, one can still be preempted while holding a mutex and
> we never know when we might want to call this from a non-sleepable context.
> 
> I'd be open to arguments that this should be an rwlock rather than
> a spinlock.  Or figure out a way to shard the lock per folio (lockdep
> etc make it very hard to embed a spinlock in struct folio).  I think
> it'd also be profitable to change how we record poison for hugetlb to
> not need locking or memory allocation.
> 
> But this is all a very long way from where I want to be working -- cleaning
> up the page fault path.  I think I've made a reasonable set of improvements
> here, and somebody else can come along later to make the code even better.
> 

We don't go down the path examining the raw_hwp_list unless the hugetlb 
folio is poisoned which is a rare event, given that, I felt the lock is 
fine.

thanks,
-jane

Re: [PATCH v8 10/15] mm: Check individual hugetlb pages for poison

[Matthew Wilcox]

On Tue, Aug 04, 2026 at 03:15:37PM -0400, Gregory Price wrote:
> On Fri, Jul 31, 2026 at 09:07:55PM +0100, Matthew Wilcox (Oracle) wrote:
> > +/*
> > + * We have no reference on the folio containing this page.
> > + * The hugetlb_lock keeps hugetlb folios from being freed.
> > + */
> > +bool hugetlb_unref_page_hwpoison(const struct page *page)
> > +{
> > +	const struct folio *folio;
> > +	unsigned long flags;
> > +	bool ret;
> > +
> > +	spin_lock_irqsave(&hugetlb_lock, flags);
> > +	folio = page_folio(page);
> > +	if (!folio_test_huge_poison(folio)) {
> > +		ret = PageHWPoison(page);
> > +		goto unlock;
> > +	}
> > +
> > +	ret = precise_page_poisoned(folio, page);
> > +unlock:
> > +	spin_unlock_irqrestore(&hugetlb_lock, flags);
> > +	return ret;
> > +}
> > +
> 
> I ended up with the same question as sashiko - i think this behavior
> implies we must hold a reference on the page, otherwise the folio can
> be invalid and all these accesses are unsafe.
> 
> But that must be the existing behavior for poison checks like this, so
> this is at least no worse. Not sure it's worth addressing.

Let me just paste Sashiko's comment in here so we preserve it in the
lore archives and so we're definitely talking about the same thing:

> This is a pre-existing issue, but does calling page_folio()
> here without holding a reference to the page risk a panic during
> lockless PFN scanning?  If a monitoring tool reads /proc/kpageflags or
> /proc/kcore, pfn_to_online_page() retrieves the page without acquiring
> a reference. If the page is concurrently freed and reallocated for
> driver use, compound_info (which aliases lru.next) could be modified.
> Could this cause page_folio() to misidentify the page as a tail page
> and return an invalid folio pointer, leading to an out-of-bounds memory
> read when folio_test_huge_poison() accesses folio->page.page_type?
> While the hugetlb_lock is held here, can it stabilize unreferenced
> pages that might have already been repurposed into non-hugetlb pages?

Sashiko is mistaken (... and fair enough, I suspect you aren't the only
human who's confused by this either)

It's quite right that we can observe a page in literally any state since
we do not hold a refcount.  But it is forbidden to use bit 0 of lru.next
for any purpose other than indicating "this is a tail page":

       /*
         * Five words (20/40 bytes) are available in this union.
         * WARNING: bit 0 of the first word is used for PageTail(). That
         * means the other users of this union MUST NOT use the bit to
         * avoid collision and false-positive PageTail().
         */

That's been the rule since 2015 with commit 1d798ca3f164

So the pointer we get back from page_folio() must have been a folio _at
some point_.  It may not be a folio now.  It may be a folio, but not one
that contains this page.  But it's not a wild pointer, and treating it
as if it's a folio won't cause any harm (as long as we're really careful).

Specifically, we call:

> +	if (!folio_test_huge_poison(folio)) {

and all that does is access folio->page.page_type aka mapcount.  So if
the pointer we have is not a current folio, it'll just return false and
we'll check PageHWPoison.

So the only case this can return 'true' is if the folio was hugetlb
at that exact point.  And we've got the hugetlb lock, so it can't stop
being a hugetlb folio.  At this point it's safe to walk the list.

That's my reasoning, and I think Sashiko has explained enough of its
reasoning to be fairly sure Sashiko is wrong about this.  But hey,
you're not Sashiko.  Maybe you've found a gap in my logic.

Re: [PATCH v8 02/15] memory-failure: Prevent hugetlb freeing during unpoisoning

[jane.chu]

On 8/4/2026 8:35 AM, Matthew Wilcox wrote:
> On Mon, Aug 03, 2026 at 11: 40: 25PM -0700, jane. chu@ oracle. com 
> wrote: > Patch itself looks good, so Reviewed-by: Jane Chu <jane. chu@  
> oracle. com> Thanks! > That said, there is a pre-existing issue: > 
> folio_free_raw_hwp() should check
> 
> 
> On Mon, Aug 03, 2026 at 11:40:25PM -0700, jane.chu@oracle.com wrote:
>> Patch itself looks good, so Reviewed-by: Jane Chu <jane.chu@oracle.com>
> 
> Thanks!
> 
>> That said, there is a pre-existing issue:
>> folio_free_raw_hwp() should check HPG_raw_hwp_unreliable, and fail the act
>> of unpoison just like what __update_and_free_hugetlb_folio() does -
>> 
>> static void __update_and_free_hugetlb_folio(struct hstate *h,
>>                                                 struct folio *folio)
>> {
>>         bool clear_flag = folio_test_hugetlb_vmemmap_optimized(folio);
>> 
>>         if (hstate_is_gigantic_no_runtime(h))
>>                 return;
>> 
>>         /*
>>          * If we don't know which subpages are hwpoisoned, we can't free
>>          * the hugepage, so it's leaked intentionally.
>>          */
>>         if (folio_test_hugetlb_raw_hwp_unreliable(folio))
>>                 return;
> 
> I'm not sure I agree with that, but I'm new to hugetlb & hwpoison
> handling, so please check my reasoning.
> 
> We set the unreliable bit when memory allocation fails (ie we can't
> allocate a raw_hwp_page).  That does mean that if we have a real hardware
> error, we can't know which page was poisoned, but for a simulated hwpoison
> which we are now clearing, it doesn't matter which page specifically
> contained the hwpoison.  We're going to unpoison all of the pages in
> this hugetlb folio anyway.
> 
> Maybe there's more to it than that.
> 

First, I take back my comment about a pre-existing issue, because
folio_free_raw_hwp() does call if 
(folio_test_hugetlb_raw_hwp_unreliable(folio)) and bail out if true,
before it calls __folio_free_raw_hwp() to dissolve the list.
So, apology for the false alarm.

Also agreed with you that once a non-simulated hwpoison occurs, unpoison 
will be disallowed for the remaining life time of the kernel.

But I think the reason we don't dissolve the raw_hwp_list if 
PG_raw_hwp_unreliable is set is that, doing so would screw the 
validation tests and maybe some body's monitoring mechanism.
Had PG_raw_hwp_unreliable got set in a non-simulation scenario, 
immediately, OOM could happen, but then, we still lose the hugetlb folio 
for the remaining life of the kernel. Unless someone come up with a 
better solution to replace the raw_hwp_list. :)

thanks,
-jane

Re: [PATCH v8 12/15] filemap: Remove checks in mapping_set_folio_order_range()

[Gregory Price]

On Fri, Jul 31, 2026 at 09:07:57PM +0100, Matthew Wilcox (Oracle) wrote:
>   *
> + * hugetlb calls this with orders larger than MAX_PAGECACHE_ORDER.
> + * Normal filesystems should not do this.

Any static way to enforce this?

Otherwise

Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>

Re: [PATCH v8 15/15] hugetlb: replace hugetlbfs_read_iter() with generic_file_read_iter()

[Gregory Price]

On Fri, Jul 31, 2026 at 09:08:00PM +0100, Matthew Wilcox (Oracle) wrote:
>  
>  static const struct file_operations hugetlbfs_file_operations = {
> -	.read_iter		= hugetlbfs_read_iter,
> +	.read_iter		= generic_file_read_iter,
>  	.mmap			= hugetlbfs_file_mmap,
>  	.fsync			= noop_fsync,
>  	.get_unmapped_area	= hugetlb_get_unmapped_area,

Still trying to wrap my head around 11 and 14, but this is nice.

Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>

Re: [PATCH v8 10/15] mm: Check individual hugetlb pages for poison

[Gregory Price]

On Tue, Aug 04, 2026 at 10:21:29PM +0100, Matthew Wilcox wrote:
> On Tue, Aug 04, 2026 at 03:15:37PM -0400, Gregory Price wrote:
> 
> So the pointer we get back from page_folio() must have been a folio _at
> some point_.  It may not be a folio now.  It may be a folio, but not one
> that contains this page.  But it's not a wild pointer, and treating it
> as if it's a folio won't cause any harm (as long as we're really careful).
> 
> Specifically, we call:
> 
> > +	if (!folio_test_huge_poison(folio)) {
> 
> and all that does is access folio->page.page_type aka mapcount.  So if
> the pointer we have is not a current folio, it'll just return false and
> we'll check PageHWPoison.
> 
> So the only case this can return 'true' is if the folio was hugetlb
> at that exact point.  And we've got the hugetlb lock, so it can't stop
> being a hugetlb folio.  At this point it's safe to walk the list.
>

This is basically the reasoning i came to, ergo "not worth addressing"

> That's my reasoning, and I think Sashiko has explained enough of its
> reasoning to be fairly sure Sashiko is wrong about this.  But hey,
> you're not Sashiko.  Maybe you've found a gap in my logic.

I think sashiko is essentially just saying:

"But if someone does a dumb thing with this interface in the future dumb
 things will happen"

To which we say "Don't do the blatantly dumb thing :]"

~Gregory

Re: [PATCH v8 12/15] filemap: Remove checks in mapping_set_folio_order_range()

[Matthew Wilcox]

On Tue, Aug 04, 2026 at 05:23:27PM -0400, Gregory Price wrote:
> On Fri, Jul 31, 2026 at 09:07:57PM +0100, Matthew Wilcox (Oracle) wrote:
> >   *
> > + * hugetlb calls this with orders larger than MAX_PAGECACHE_ORDER.
> > + * Normal filesystems should not do this.
> 
> Any static way to enforce this?

Not easily.  We could look at mapping->a_ops and compare them to
hugetlbfs_aops, but this seems like overkill?  For such a low-level
interface with so few callers, I think the level of protection we had
earlier was just too much.

> Otherwise
> 
> Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>
>