[PATCH v2] mm/highmem: refuse to copy hwpoisoned folios in the !copy_mc fallbacks

[PATCH v2] mm/highmem: refuse to copy hwpoisoned folios in the !copy_mc fallbacks

[Yuanhe Shu]

The !copy_mc fallbacks of copy_mc_user_highpage() and
copy_mc_highpage() return 0 after a plain copy.  With PG_hwpoison
set while the data stays readable, there is no fault to recover
from: the copy completes, the destination folio carries no poison
marker, and the poison is consumed silently.

Check the source page with is_page_hwpoison() in both fallbacks,
before and after the copy, and return PAGE_SIZE, the
not-copied-bytes convention of the real copy_mc paths, so the
callers take their existing error paths.  On archs with copy_mc
this code does not compile.

Measured on arm64, where the fallback code compiles (10 rounds
each): COW copy throughput 1043.67 MB/s before and 1043.26 MB/s
after the patch, fork latency 348 vs 359 us, forced migration 57396
vs 57654 MB/s - all within run-to-run noise.

Suggested-by: "Huang, Ying" <ying.huang@linux.alibaba.com>
Suggested-by: Kaitao Cheng <kaitao.cheng@linux.dev>
Assisted-by: LLM
Signed-off-by: Yuanhe Shu <xiangzao@linux.alibaba.com>
Link: https://lore.kernel.org/r/877bjowf9i.fsf@DESKTOP-5N7EMDA/
---

Changes since v1:

v1: https://lore.kernel.org/r/20261026025724.1430747-1-xiangzao@linux.alibaba.com/

- Check before the copy as well, so data the flag already
  condemns is not read (Huang Ying, David Hildenbrand).
- Use is_page_hwpoison() instead of folio_contain_hwpoisoned_page()
  (sashiko review, Shivank Garg): the folio-level predicate misses a
  poisoned tail page in the memory_failure() window, and refuses
  clean siblings of a poisoned subpage.

 include/linux/highmem.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index d7aac9de1c8a..d0e33cacae90 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -473,13 +473,25 @@ static inline int copy_mc_highpage(struct page *to, struct page *from)
 static inline int copy_mc_user_highpage(struct page *to, struct page *from,
 					unsigned long vaddr, struct vm_area_struct *vma)
 {
+	/* Don't touch data that is already marked poisoned. */
+	if (is_page_hwpoison(from))
+		return PAGE_SIZE;
 	copy_user_highpage(to, from, vaddr, vma);
+	/* The flag can also land while the copy runs. */
+	if (is_page_hwpoison(from))
+		return PAGE_SIZE;
 	return 0;
 }
 
 static inline int copy_mc_highpage(struct page *to, struct page *from)
 {
+	/* Don't touch data that is already marked poisoned. */
+	if (is_page_hwpoison(from))
+		return PAGE_SIZE;
 	copy_highpage(to, from);
+	/* The flag can also land while the copy runs. */
+	if (is_page_hwpoison(from))
+		return PAGE_SIZE;
 	return 0;
 }
 #endif
-- 
2.43.7

Re: [PATCH v2] mm/highmem: refuse to copy hwpoisoned folios in the !copy_mc fallbacks

[Huang, Ying]

Yuanhe Shu <xiangzao@linux.alibaba.com> writes:

> The !copy_mc fallbacks of copy_mc_user_highpage() and
> copy_mc_highpage() return 0 after a plain copy.  With PG_hwpoison
> set while the data stays readable, there is no fault to recover
> from: the copy completes, the destination folio carries no poison
> marker, and the poison is consumed silently.

IIUC, in all cases except debugging, reading poisoned memory will cause
an MCE (x86), an SEA (arm), or similar.  Otherwise, copy_mc on x86
would not work either.

If so, we should only check is_page_hwpoison() before copying.

> Check the source page with is_page_hwpoison() in both fallbacks,
> before and after the copy, and return PAGE_SIZE, the
> not-copied-bytes convention of the real copy_mc paths, so the
> callers take their existing error paths.  On archs with copy_mc
> this code does not compile.
>
> Measured on arm64, where the fallback code compiles (10 rounds
> each): COW copy throughput 1043.67 MB/s before and 1043.26 MB/s
> after the patch, fork latency 348 vs 359 us, forced migration 57396
> vs 57654 MB/s - all within run-to-run noise.

Could you implement a micro-benchmark in kernel to test your copy_mc
functions?

[snip]

---
Best Regards,
Huang, Ying