From: Kiryl Shutsemau <kirill@shutemov.name>
To: akpm@linux-foundation.org, david@kernel.org, ljs@kernel.org,
nico.pache@linux.dev
Cc: baolin.wang@linux.alibaba.com, baohua@kernel.org,
dev.jain@arm.com, hughd@google.com, lance.yang@linux.dev,
liam@infradead.org, mhocko@suse.com, rppt@kernel.org,
ryan.roberts@arm.com, shuah@kernel.org, surenb@google.com,
usama.arif@linux.dev, vbabka@kernel.org, ziy@nvidia.com,
usama.anjum@arm.com, agordeev@linux.ibm.com, linux-mm@kvack.org,
linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org,
kas@kernel.org, jannh@google.com, willy@infradead.org,
pfalcato@suse.de, rostedt@goodmis.org, mhiramat@kernel.org,
linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org
Subject: [RFC PATCH 17/57] mm/collapse: copy the sources into the destinations
Date: Sun, 16 Aug 2026 23:45:29 +0100 [thread overview]
Message-ID: <20260816224609.308019-18-kirill@shutemov.name> (raw)
In-Reply-To: <20260816224609.308019-1-kirill@shutemov.name>
From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
Fill in the copy. For each candidate that both froze and has a
destination folio, the page mapped at slot k of its window becomes page k
of that folio. A slot with no source -- a hole, or a zeropage the freeze
cleared -- is zero-filled instead.
Neither condition implies the other. A reserve before the lock can leave
a folio on a candidate that then fails to freeze, and the provision
inside the window can decline one for a candidate that froze.
Nothing else is carried across: the destination's PTE bits are not
derived from the sources, and the install rebuilds them the way a fault
would.
A machine check reading a source is the only failure the copy can report,
and only where the architecture provides an MC-safe copy. The candidate
records SCAN_COPY_MC and the install undoes it, that being where the ptl
the undoing needs is held.
Assisted-by: Claude-Code:claude-opus-5
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
mm/collapse.c | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/mm/collapse.c b/mm/collapse.c
index cf2b9b3640ae..a3882d897d11 100644
--- a/mm/collapse.c
+++ b/mm/collapse.c
@@ -1040,6 +1040,46 @@ static void collapse_provision(struct mm_struct *mm,
static void collapse_copy(struct vm_area_struct *vma,
struct collapse_control *cc)
{
+ unsigned int i;
+
+ for (i = 0; i < cc->nr_candidates; i++) {
+ struct collapse_candidate *cand = &cc->candidates[i];
+ const unsigned int nr_pages = candidate_nr_pages(cand);
+ unsigned long addr = cand->addr;
+ unsigned int k;
+
+ /* A folio does not imply a freeze: reserve runs before the lock */
+ if (cand->state != CAND_FROZEN)
+ continue;
+
+ /* A freeze does not imply a folio: provision may have declined */
+ if (!cand->new_folio)
+ continue;
+
+ /* Each source lands where its address puts it: slot k, page k */
+ for (k = 0; k < nr_pages; k++, addr += PAGE_SIZE) {
+ struct page *dst = folio_page(cand->new_folio, k);
+ struct page *src;
+
+ /* No source: a hole, or a zeropage the freeze cleared */
+ if (pte_none_or_zero(cand->saved_ptes[k])) {
+ clear_user_highpage(dst, addr);
+ continue;
+ }
+
+ src = pte_page(cand->saved_ptes[k]);
+
+ /*
+ * A machine check on a source is the only way this
+ * fails, and the install is what undoes the candidate:
+ * that is where the ptl the undoing needs is held.
+ */
+ if (copy_mc_user_highpage(dst, src, addr, vma)) {
+ cand->result = SCAN_COPY_MC;
+ break;
+ }
+ }
+ }
}
/* Publish each destination folio in place of the sources it replaces */
--
2.54.0
next prev parent reply other threads:[~2026-08-16 22:46 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-16 22:45 [RFC PATCH 00/57] mm/collapse: rebuild collapse on migration primitives Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 01/57] mm: add pte_folio() Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 02/57] mm: add pte_none_or_zero() Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 03/57] mm/collapse: add collapse.h for the shared collapse state Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 04/57] mm/collapse: rename mthp_present_ptes to eligible_ptes Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 05/57] mm/collapse: state what a collapse may do in the policy Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 06/57] mm/collapse: move the smallest collapse order to collapse.h Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 07/57] mm/collapse: sketch the new anonymous collapse engine Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 08/57] mm/collapse: scan a table for what a collapse could use Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 09/57] mm/collapse: collect candidate windows into a round Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 10/57] mm/collapse: run a round and feed the outcomes back Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 11/57] mm/collapse: sketch the passes of a round Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 12/57] mm/collapse: allocate a destination per candidate Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 13/57] mm/collapse: revalidate a round against the VMA Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 14/57] mm/collapse: fault the sources in before the freeze Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 15/57] mm/collapse: check what a candidate would freeze Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 16/57] mm/collapse: freeze the sources behind migration entries Kiryl Shutsemau
2026-08-16 22:45 ` Kiryl Shutsemau [this message]
2026-08-16 22:45 ` [RFC PATCH 18/57] mm/collapse: install the destinations at PTE level Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 19/57] mm/collapse: install a PMD leaf as the terminal layer Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 20/57] mm/collapse: put the sources back Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 21/57] mm/collapse: settle whatever the round reached Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 22/57] mm/collapse: walk a table with a selection cursor Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 23/57] mm/collapse: give a refused region a second chance Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 24/57] mm/collapse: report each candidate's outcome to tracing Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 25/57] mm/collapse: collapse anonymous memory with the new engine Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 26/57] mm/collapse: give collapse_single_pmd() the range to work on Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 27/57] mm/collapse: scan the windows a VMA can hold Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 28/57] mm/collapse: remove the mechanism the engine replaces Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 29/57] mm/collapse: move what a collapse is judged on into collapse.c Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 30/57] mm/collapse: name the max_ptes ceiling after collapse Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 31/57] mm/khugepaged: count collapses where khugepaged makes them Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 32/57] mm/collapse: move the file collapse into collapse.c Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 33/57] mm/collapse: split collapse into a scan and a run Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 34/57] mm/collapse: implement MADV_COLLAPSE in madvise.c Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 35/57] mm/madvise: drop MADV_COLLAPSE's redundant mm reference Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 36/57] mm/collapse: report what the scan found Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 37/57] mm/collapse: report what the fault-in pass paid Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 38/57] mm/collapse: report the round, and what it made faulters wait Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 39/57] mm/collapse: name the file collapse's tracepoints after collapse Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 40/57] mm/collapse: remove the tracepoints of the mechanism that is gone Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 41/57] mm/collapse: give collapse its own trace header Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 42/57] mm/collapse: allow error injection into the freeze Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 43/57] mm/khugepaged: check the scan budget before the work, not after Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 44/57] mm/khugepaged: hold the address space open across a scan Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 45/57] mm/collapse: take a per-VMA read lock for the round Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 46/57] mm/khugepaged: scan under a per-VMA read lock Kiryl Shutsemau
2026-08-16 22:45 ` [RFC PATCH 47/57] mm/madvise: collapse " Kiryl Shutsemau
2026-08-16 22:46 ` [RFC PATCH 48/57] mm/collapse: assert the mm reference the engine relies on Kiryl Shutsemau
2026-08-16 22:46 ` [RFC PATCH 49/57] mm/khugepaged: drop the mmap_lock barrier from __khugepaged_exit() Kiryl Shutsemau
2026-08-16 22:46 ` [RFC PATCH 50/57] selftests/mm: attribute collapses by candidate event alone Kiryl Shutsemau
2026-08-16 22:46 ` [RFC PATCH 51/57] selftests/mm: cover collapse inside a sub-PMD VMA Kiryl Shutsemau
2026-08-16 22:46 ` [RFC PATCH 52/57] selftests/mm: cover a hole-y window in " Kiryl Shutsemau
2026-08-16 22:46 ` [RFC PATCH 53/57] selftests/mm: cover collapse of mlocked ranges Kiryl Shutsemau
2026-08-16 22:46 ` [RFC PATCH 54/57] selftests/mm: cover collapse beside a MADV_FREE'd page Kiryl Shutsemau
2026-08-16 22:46 ` [RFC PATCH 55/57] selftests/mm: cover collapse beside a pinned page Kiryl Shutsemau
2026-08-16 22:46 ` [RFC PATCH 56/57] selftests/mm: cover the scaled max_ptes_shared limit Kiryl Shutsemau
2026-08-16 22:46 ` [RFC PATCH 57/57] MAINTAINERS: add an entry for collapse Kiryl Shutsemau
2026-08-17 8:04 ` Lorenzo Stoakes (ARM)
2026-08-17 8:08 ` David Hildenbrand (Arm)
2026-08-17 10:12 ` Kiryl Shutsemau
2026-08-17 2:02 ` [RFC PATCH 00/57] mm/collapse: rebuild collapse on migration primitives Zi Yan
2026-08-17 10:07 ` Kiryl Shutsemau
2026-08-17 8:52 ` Lorenzo Stoakes (ARM)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260816224609.308019-18-kirill@shutemov.name \
--to=kirill@shutemov.name \
--cc=agordeev@linux.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=baohua@kernel.org \
--cc=baolin.wang@linux.alibaba.com \
--cc=bpf@vger.kernel.org \
--cc=david@kernel.org \
--cc=dev.jain@arm.com \
--cc=hughd@google.com \
--cc=jannh@google.com \
--cc=kas@kernel.org \
--cc=lance.yang@linux.dev \
--cc=liam@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=ljs@kernel.org \
--cc=mhiramat@kernel.org \
--cc=mhocko@suse.com \
--cc=nico.pache@linux.dev \
--cc=pfalcato@suse.de \
--cc=rostedt@goodmis.org \
--cc=rppt@kernel.org \
--cc=ryan.roberts@arm.com \
--cc=shuah@kernel.org \
--cc=surenb@google.com \
--cc=usama.anjum@arm.com \
--cc=usama.arif@linux.dev \
--cc=vbabka@kernel.org \
--cc=willy@infradead.org \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox