[v3 06/24] mm: thp: handle split failure in wp_huge_pmd()

Usama Arif
2026-03-27T02:14:50Z

wp_huge_pmd() splits the PMD when COW or write-notify must be handled at
PTE level (e.g., shared/file VMAs, userfaultfd). It then returns
VM_FAULT_FALLBACK so the fault handler retries at PTE granularity.
If the split fails, the PMD is still huge. The PTE fault path cannot
handle a huge PMD entry.
Return VM_FAULT_OOM on split failure, which signals the fault handler to
invoke the OOM killer or return -ENOMEM to userspace. This is similar to
what __handle_mm_fault would do if p4d_alloc or pud_alloc fails.

Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
mm/memory.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index caf97c48cb166..b99ec3ffc18d1 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -6328,8 +6328,13 @@ static inline vm_fault_t wp_huge_pmd(struct vm_fault *vmf)
}

split:
- /* COW or write-notify handled on pte level: split pmd. */
- __split_huge_pmd(vma, vmf->pmd, vmf->address, false);
+ /*
+ * COW or write-notify handled on pte level: split pmd.
+ * If split fails, the PMD is still huge so falling back
+ * to PTE handling would be incorrect.
+ */
+ if (__split_huge_pmd(vma, vmf->pmd, vmf->address, false))
+ return VM_FAULT_OOM;

return VM_FAULT_FALLBACK;
}
--
2.52.0