[v3 12/24] mm: thp: handle split failure in device migration

Usama Arif
2026-03-27T02:15:30Z

Device memory migration has two call sites that split huge PMDs:

migrate_vma_split_unmapped_folio():
Called from migrate_vma_pages() when migrating a PMD-mapped THP to a
destination that doesn't support compound pages. It splits the PMD
then splits the folio via folio_split_unmapped().

If the PMD split fails, folio_split_unmapped() would operate on an
unsplit folio with inconsistent page table state. Propagate -ENOMEM
to skip this page's migration. This is safe as folio_split_unmapped
failure would be propagated in a similar way.

migrate_vma_insert_page():
Called from migrate_vma_pages() when inserting a page into a VMA
during migration back from device memory. If a huge zero PMD exists
at the target address, it must be split before PTE insertion.

If the split fails, the subsequent pte_alloc() and set_pte_at() would
operate on a PMD slot still occupied by the huge zero entry. Use
goto abort, consistent with other allocation failures in this function.

Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
mm/migrate_device.c | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/mm/migrate_device.c b/mm/migrate_device.c
index 2912eba575d5e..00003fbe803df 100644
--- a/mm/migrate_device.c
+++ b/mm/migrate_device.c
@@ -919,7 +919,19 @@ static int migrate_vma_split_unmapped_folio(struct migrate_vma *migrate,
* drops a reference at the end.
*/
folio_get(folio);
- split_huge_pmd_address(migrate->vma, addr, true);
+ /*
+ * If PMD split fails, folio_split_unmapped would operate on an
+ * unsplit folio with inconsistent page table state.
+ */
+ ret = split_huge_pmd_address(migrate->vma, addr, true);
+ if (ret) {
+ /*
+ * folio_get above was not consumed by split_huge_pmd_address.
+ * put back that reference.
+ */
+ folio_put(folio);
+ return ret;
+ }
ret = folio_split_unmapped(folio, 0);
if (ret)
return ret;
@@ -1015,7 +1027,13 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate,
if (pmd_trans_huge(*pmdp)) {
if (!is_huge_zero_pmd(*pmdp))
goto abort;
- split_huge_pmd(vma, pmdp, addr);
+ /*
+ * If split fails, the huge zero PMD remains and
+ * pte_alloc/PTE insertion that follows would be
+ * incorrect.
+ */
+ if (split_huge_pmd(vma, pmdp, addr))
+ goto abort;
} else if (pmd_leaf(*pmdp))
goto abort;
}
--
2.52.0