From: Wandun Chen <chenwandun1@gmail.com>
To: david@kernel.org, akpm@linux-foundation.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org
Cc: ljs@kernel.org, liam@infradead.org, vbabka@kernel.org,
rppt@kernel.org, surenb@google.com, mhocko@suse.com,
ziy@nvidia.com, matthew.brost@intel.com, joshua.hahnjy@gmail.com,
byungchul@sk.com, gourry@gourry.net,
ying.huang@linux.alibaba.com, apopple@nvidia.com
Subject: [PATCH 2/3] mm/migrate: batch PTE restoration in remove_migration_pte()
Date: Fri, 20 Nov 2026 11:29:48 +0800 [thread overview]
Message-ID: <20261120032949.2036731-3-chenwandun1@gmail.com> (raw)
In-Reply-To: <20261120032949.2036731-1-chenwandun1@gmail.com>
From: Wandun Chen <chenwandun@lixiang.com>
remove_migration_pte() restores PTE-mapped large folios one PTE at a
time, constructing each new PTE and updating the folio references and
rmap separately.
Add migration_pte_batch() to find adjacent migration entries with
consecutive PFNs and matching type and flags.
When a batch covers the whole folio, the rmap helpers already call
mlock_vma_folio(). Skip the explicit call to avoid double mlock
accounting.
Device-private PTEs encode PFNs as swap offsets. set_ptes() does not
handle this encoding when constructing subsequent PTEs, such as on
arm64 with a 16KB page granule, leaving device-private destinations
unbatched. The next patch enables batching for these entries by
advancing their softleaf offsets.
Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
Link: https://lore.kernel.org/lkml/5f9c449e-43b3-4732-9c14-8199418cace0@kernel.org/
Signed-off-by: Wandun Chen <chenwandun@lixiang.com>
---
mm/internal.h | 6 ++++++
mm/migrate.c | 35 +++++++++++++++++++++++++++++------
2 files changed, 35 insertions(+), 6 deletions(-)
diff --git a/mm/internal.h b/mm/internal.h
index 9f78c12882cb..e6ecf307d532 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -552,6 +552,12 @@ static inline int swap_pte_batch(pte_t *start_ptep, int max_nr, pte_t pte)
return __softleaf_pte_batch(start_ptep, max_nr, pte);
}
+static inline int migration_pte_batch(pte_t *start_ptep, int max_nr, pte_t pte)
+{
+ VM_WARN_ON(!softleaf_is_migration(softleaf_from_pte(pte)));
+
+ return __softleaf_pte_batch(start_ptep, max_nr, pte);
+}
#endif /* CONFIG_MMU */
void __acct_reclaim_writeback(pg_data_t *pgdat, struct folio *folio,
diff --git a/mm/migrate.c b/mm/migrate.c
index 3c39d309e6bd..e000ef77ea87 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -358,6 +358,7 @@ static bool remove_migration_pte(struct folio *folio,
while (page_vma_mapped_walk(&pvmw)) {
rmap_t rmap_flags = RMAP_NONE;
unsigned long idx = 0;
+ int nr = 1;
softleaf_t entry;
struct page *new;
pte_t old_pte;
@@ -441,17 +442,34 @@ static bool remove_migration_pte(struct folio *folio,
} else
#endif
{
+ if (folio_test_large(rmap_walk_arg->folio) &&
+ !is_device_private_page(new)) {
+ unsigned long end_addr = pmd_addr_end(pvmw.address, vma->vm_end);
+ unsigned long max_nr = (end_addr - pvmw.address) >> PAGE_SHIFT;
+
+ max_nr = min_t(unsigned long, max_nr,
+ PTRS_PER_PTE - pte_index(pvmw.address));
+
+ max_nr = min_t(unsigned long, max_nr,
+ folio_nr_pages(rmap_walk_arg->folio) - idx);
+
+ nr = migration_pte_batch(pvmw.pte, max_nr, old_pte);
+ }
+ if (nr > 1)
+ folio_ref_add(folio, nr - 1);
+
if (folio_test_anon(folio))
- folio_add_anon_rmap_pte(folio, new, vma,
- pvmw.address, rmap_flags);
+ folio_add_anon_rmap_ptes(folio, new, nr, vma,
+ pvmw.address, rmap_flags);
else
- folio_add_file_rmap_pte(folio, new, vma);
- set_pte_at(vma->vm_mm, pvmw.address, pvmw.pte, pte);
+ folio_add_file_rmap_ptes(folio, new, nr, vma);
+ set_ptes(vma->vm_mm, pvmw.address, pvmw.pte, pte, nr);
- ptes++;
+ ptes += nr;
}
if (READ_ONCE(vma->vm_flags) & VM_LOCKED) {
if (folio_test_large(folio) &&
+ nr < folio_nr_pages(folio) &&
ptes == folio_nr_pages(folio) &&
!(pvmw.flags & PVMW_PGTABLE_CROSSED))
mlock_vma_folio(folio, vma);
@@ -462,7 +480,12 @@ static bool remove_migration_pte(struct folio *folio,
compound_order(new));
/* No need to invalidate - it was non-present before */
- update_mmu_cache(vma, pvmw.address, pvmw.pte);
+ update_mmu_cache_range(NULL, vma, pvmw.address, pvmw.pte, nr);
+
+ if (nr == folio_nr_pages(folio)) {
+ page_vma_mapped_walk_done(&pvmw);
+ break;
+ }
}
return true;
--
2.43.0
next prev parent reply other threads:[~2026-11-20 3:30 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-20 3:29 [PATCH 0/3] mm/migrate: batch PTE restoration for large folios Wandun Chen
2026-11-20 3:29 ` [PATCH 1/3] mm: factor out common softleaf offset PTE batch scanning Wandun Chen
2026-11-20 15:34 ` Lorenzo Stoakes (ARM)
2026-11-20 3:29 ` Wandun Chen [this message]
2026-11-20 15:22 ` [PATCH 2/3] mm/migrate: batch PTE restoration in remove_migration_pte() Lorenzo Stoakes (ARM)
2026-11-20 3:29 ` [PATCH 3/3] mm/migrate: batch PTE restoration for device-private folios Wandun Chen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261120032949.2036731-3-chenwandun1@gmail.com \
--to=chenwandun1@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=apopple@nvidia.com \
--cc=byungchul@sk.com \
--cc=david@kernel.org \
--cc=gourry@gourry.net \
--cc=joshua.hahnjy@gmail.com \
--cc=liam@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=matthew.brost@intel.com \
--cc=mhocko@suse.com \
--cc=rppt@kernel.org \
--cc=surenb@google.com \
--cc=vbabka@kernel.org \
--cc=ying.huang@linux.alibaba.com \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.