* [to-be-updated] mm-collapse-separate-scanning-a-pte-table-from-collapsing-it.patch removed from -mm tree
@ 2026-09-16 22:42 Andrew Morton
0 siblings, 0 replies; 2+ messages in thread
From: Andrew Morton @ 2026-09-16 22:42 UTC (permalink / raw)
To: mm-commits, kas, akpm
The quilt patch titled
Subject: mm/collapse: separate scanning a PTE table from collapsing it
has been removed from the -mm tree. Its filename was
mm-collapse-separate-scanning-a-pte-table-from-collapsing-it.patch
This patch was dropped because an updated version will be issued
------------------------------------------------------
From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
Subject: mm/collapse: separate scanning a PTE table from collapsing it
Date: Thu, 10 Sep 2026 13:02:28 +0100
A collapse is two jobs. One reads a PTE table under mmap_lock and decides
whether the range is worth collapsing. The other allocates, isolates,
copies and flushes, and wants the lock given up first.
collapse_single_pmd() did both, so the boundary between them was somewhere
in the middle of a function.
Give each half its own function:
- collapse_scan_pmd() scans one table and only reads. The anonymous
scan that used to carry that name keeps its body as
collapse_scan_anon_pmd(), and collapse_scan_pmd() is now the entry
that picks the anonymous or the file side.
- collapse_run_pmd() does the collapse the scan asked for.
SCAN_SUCCEED from the scan means there is something to run; anything
else is why there is not.
collapse_single_pmd() is now the two of them with the mmap_lock drop in
between, so its callers see what they saw before.
What the scan found and the run needs travels in collapse_control. For an
anonymous table that is the orders and the referenced and swapped-out
counts. For a file it is the file itself, the offset in it, and whether
the PMD folio is already in the page cache.
The file side moves with the anonymous one. collapse_scan_file() used to
run with mmap_lock already given up, and called collapse_file() itself
when the page cache looked worth it. It now runs under the lock like the
anonymous scan and only judges; the run does the collapse. A file
collapse works on the page cache and never sees a VMA, so the scan takes
the file reference while it still has one and the run gives it back.
That changes what a refused file table costs khugepaged. Every file table
it scanned used to end its pass over that mm, because the lock had been
dropped to scan it; now only a table it goes on to collapse does.
Two things on the file side stop being rescanned. When the page cache
already holds the PMD folio, the scan says so and the run goes straight to
retracting the PTE table. A run that refuses dirty pages and may write
them back retries collapse_file() alone. The checks the scan makes ahead
of it are ones collapse_file() repeats under the page cache lock.
Tracing changes with it. mm_khugepaged_scan_pmd and
mm_khugepaged_scan_file used to fire after the collapse, so for an
accepted table their status field carried what the collapse made of it.
They now fire before it and read SCAN_SUCCEED for an accepted table. What
the collapse then made of it is for mm_collapse_huge_page and
mm_khugepaged_collapse_file to report.
Assisted-by: LLM
Link: https://lore.kernel.org/20260910120238.2529819-9-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jann Horn <jannh@google.com>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
---
mm/collapse.h | 16 +++++
mm/khugepaged.c | 147 +++++++++++++++++++++++++++++++++++-----------
2 files changed, 128 insertions(+), 35 deletions(-)
--- a/mm/collapse.h~mm-collapse-separate-scanning-a-pte-table-from-collapsing-it
+++ a/mm/collapse.h
@@ -88,6 +88,22 @@ struct collapse_control {
/* Each bit marks a PTE the scan accepted as a collapse source */
DECLARE_BITMAP(eligible_ptes, MAX_PTRS_PER_PTE);
+
+ /*
+ * What a scan found and the run after it needs. Live only between the
+ * two, and read by nobody else.
+ *
+ * The file side takes a reference while it still has the VMA, since a
+ * file collapse works on the page cache and never sees one; the run is
+ * what gives it back. A scan that found the PMD folio already in the
+ * cache leaves only the PTE table to retract.
+ */
+ unsigned long scan_orders;
+ int scan_referenced;
+ int scan_unmapped;
+ struct file *scan_file;
+ pgoff_t scan_pgoff;
+ bool scan_retract_only;
};
#endif /* __MM_COLLAPSE_H */
--- a/mm/khugepaged.c~mm-collapse-separate-scanning-a-pte-table-from-collapsing-it
+++ a/mm/khugepaged.c
@@ -1550,14 +1550,14 @@ done:
return last_result;
}
-static enum scan_result collapse_scan_pmd(struct mm_struct *mm,
- struct vm_area_struct *vma, unsigned long start_addr,
- bool *lock_dropped, struct collapse_control *cc)
+static enum scan_result collapse_scan_anon_pmd(struct vm_area_struct *vma,
+ unsigned long start_addr, struct collapse_control *cc)
{
const unsigned int max_ptes_shared = collapse_max_ptes_shared(cc, HPAGE_PMD_ORDER);
const unsigned int max_ptes_swap = collapse_max_ptes_swap(cc, HPAGE_PMD_ORDER);
unsigned int max_ptes_none = collapse_max_ptes_none(cc, vma, HPAGE_PMD_ORDER);
enum tva_type tva_flags = cc->policy.tva_type;
+ struct mm_struct *mm = vma->vm_mm;
pmd_t *pmd;
pte_t *pte, *_pte, pteval;
int i;
@@ -1737,12 +1737,9 @@ static enum scan_result collapse_scan_pm
out_unmap:
pte_unmap_unlock(pte, ptl);
if (result == SCAN_SUCCEED) {
- /* collapse_huge_page() expects the lock to be dropped before calling */
- mmap_read_unlock(mm);
- result = mthp_collapse(mm, start_addr, referenced,
- unmapped, cc, enabled_orders);
- /* mmap_lock was released above, set lock_dropped */
- *lock_dropped = true;
+ cc->scan_orders = enabled_orders;
+ cc->scan_referenced = referenced;
+ cc->scan_unmapped = unmapped;
}
out:
trace_mm_khugepaged_scan_pmd(mm, failed_pfn, referenced,
@@ -2732,45 +2729,95 @@ static enum scan_result collapse_scan_fi
else
cc->progress += HPAGE_PMD_NR;
- if (result == SCAN_SUCCEED) {
- if (present < HPAGE_PMD_NR - max_ptes_none) {
- result = SCAN_EXCEED_NONE_PTE;
- count_vm_event(THP_SCAN_EXCEED_NONE_PTE);
- } else {
- result = collapse_file(mm, addr, file, start, cc);
- }
+ if (result == SCAN_SUCCEED && present < HPAGE_PMD_NR - max_ptes_none) {
+ result = SCAN_EXCEED_NONE_PTE;
+ count_vm_event(THP_SCAN_EXCEED_NONE_PTE);
}
- trace_mm_khugepaged_scan_file(mm, failed_pfn, file, present, swap, result);
+ trace_mm_khugepaged_scan_file(mm, failed_pfn, file, present, swap,
+ result);
return result;
}
-/*
- * Try to collapse a single PMD starting at a PMD aligned addr, and return
- * the results.
- */
-static enum scan_result collapse_single_pmd(unsigned long addr,
- struct vm_area_struct *vma, bool *lock_dropped,
- struct collapse_control *cc)
+static void collapse_control_init(struct collapse_control *cc)
+{
+ cc->progress = 0;
+ cc->scan_file = NULL;
+}
+
+static void collapse_control_release(struct collapse_control *cc)
+{
+ /* A scan that took a file reference should have been run */
+ if (WARN_ON_ONCE(cc->scan_file)) {
+ fput(cc->scan_file);
+ cc->scan_file = NULL;
+ }
+}
+
+static enum scan_result collapse_scan_pmd(struct vm_area_struct *vma,
+ unsigned long addr, struct collapse_control *cc)
{
- struct mm_struct *mm = vma->vm_mm;
- bool triggered_wb = false;
enum scan_result result;
- struct file *file;
pgoff_t pgoff;
- mmap_assert_locked(mm);
+ mmap_assert_locked(vma->vm_mm);
+ /* Whatever the last scan found has to have been run by now */
+ if (WARN_ON_ONCE(cc->scan_file)) {
+ fput(cc->scan_file);
+ cc->scan_file = NULL;
+ }
if (vma_is_anonymous(vma))
- return collapse_scan_pmd(mm, vma, addr, lock_dropped, cc);
+ return collapse_scan_anon_pmd(vma, addr, cc);
- file = get_file(vma->vm_file);
pgoff = linear_page_index(vma, addr);
+ result = collapse_scan_file(vma->vm_mm, addr, vma->vm_file, pgoff, cc);
+ switch (result) {
+ case SCAN_SUCCEED:
+ cc->scan_retract_only = false;
+ break;
+ case SCAN_PTE_MAPPED_HUGEPAGE:
+ /*
+ * The page cache already holds the PMD folio; what is left is
+ * to retract the PTE table, which is the run's job.
+ */
+ cc->scan_retract_only = true;
+ result = SCAN_SUCCEED;
+ break;
+ default:
+ return result;
+ }
- mmap_read_unlock(mm);
- *lock_dropped = true;
+ /*
+ * A file collapse works on the page cache and never sees a VMA, so take
+ * what it needs from this one while it is still here.
+ */
+ cc->scan_file = get_file(vma->vm_file);
+ cc->scan_pgoff = pgoff;
+ return result;
+}
+
+static enum scan_result collapse_run_pmd(struct mm_struct *mm,
+ unsigned long addr, struct collapse_control *cc)
+{
+ struct file *file = cc->scan_file;
+ bool triggered_wb = false;
+ enum scan_result result;
+ pgoff_t pgoff;
+
+ if (!file)
+ return mthp_collapse(mm, addr, cc->scan_referenced,
+ cc->scan_unmapped, cc, cc->scan_orders);
+
+ cc->scan_file = NULL;
+ pgoff = cc->scan_pgoff;
+
+ if (cc->scan_retract_only) {
+ result = SCAN_PTE_MAPPED_HUGEPAGE;
+ goto retract;
+ }
retry:
- result = collapse_scan_file(mm, addr, file, pgoff, cc);
+ result = collapse_file(mm, addr, file, pgoff, cc);
/* Dirty pages are worth a writeback and one more try, if asked for */
if (cc->policy.writeback_dirty && result == SCAN_PAGE_DIRTY_OR_WRITEBACK &&
@@ -2782,8 +2829,13 @@ retry:
triggered_wb = true;
goto retry;
}
+retract:
fput(file);
+ /*
+ * A PMD folio is in the page cache, whether the collapse just put it
+ * there or found it: retract the PTE table, and map the PMD if asked.
+ */
if (result == SCAN_PTE_MAPPED_HUGEPAGE) {
mmap_read_lock(mm);
if (collapse_test_exit_or_disable(mm))
@@ -2798,6 +2850,28 @@ retry:
return result;
}
+/*
+ * Try to collapse a single PMD starting at a PMD aligned addr, and return
+ * the results.
+ */
+static enum scan_result collapse_single_pmd(unsigned long addr,
+ struct vm_area_struct *vma, bool *lock_dropped,
+ struct collapse_control *cc)
+{
+ struct mm_struct *mm = vma->vm_mm;
+ enum scan_result result;
+
+ result = collapse_scan_pmd(vma, addr, cc);
+ if (result != SCAN_SUCCEED)
+ return result;
+
+ /* The collapse takes its own locks, so give this up */
+ mmap_read_unlock(mm);
+ *lock_dropped = true;
+
+ return collapse_run_pmd(mm, addr, cc);
+}
+
static void collapse_scan_mm_slot(unsigned int progress_max,
enum scan_result *result, struct collapse_control *cc)
__releases(&khugepaged_mm_lock)
@@ -2940,10 +3014,10 @@ static void khugepaged_do_scan(struct co
lru_add_drain_all();
+ collapse_control_init(cc);
/* One policy for the whole pass, so every table is judged the same */
collapse_policy_khugepaged(&cc->policy);
- cc->progress = 0;
while (true) {
cond_resched();
@@ -2974,6 +3048,8 @@ static void khugepaged_do_scan(struct co
khugepaged_alloc_sleep();
}
}
+
+ collapse_control_release(cc);
}
static bool khugepaged_should_wakeup(void)
@@ -3170,8 +3246,8 @@ int madvise_collapse(struct vm_area_stru
cc = kmalloc_obj(*cc);
if (!cc)
return -ENOMEM;
+ collapse_control_init(cc);
collapse_policy_forced(&cc->policy);
- cc->progress = 0;
lru_add_drain_all();
@@ -3228,6 +3304,7 @@ out_maybelock:
}
out_nolock:
mmap_assert_locked(mm);
+ collapse_control_release(cc);
kfree(cc);
return thps == ((hend - hstart) >> HPAGE_PMD_SHIFT) ? 0
_
Patches currently in -mm which might be from kas@kernel.org are
mm-huge_memory-do-not-touch-frozen-folios-in-deferred_split_isolate.patch
mm-huge_memory-dequeue-the-deferred-split-after-the-split-freeze.patch
mm-huge_memory-add-folio_reset_partially_mapped.patch
mm-collapse-open-code-collapse_single_pmd-in-its-two-callers.patch
mm-collapse-work-out-the-orders-a-vma-allows-once-per-vma.patch
mm-collapse-declare-the-collapse-interface-in-collapseh.patch
mm-collapse-implement-madv_collapse-in-madvisec.patch
a.patch
^ permalink raw reply [flat|nested] 2+ messages in thread
* [to-be-updated] mm-collapse-separate-scanning-a-pte-table-from-collapsing-it.patch removed from -mm tree
@ 2026-09-28 21:51 Andrew Morton
0 siblings, 0 replies; 2+ messages in thread
From: Andrew Morton @ 2026-09-28 21:51 UTC (permalink / raw)
To: mm-commits, kas, akpm
The quilt patch titled
Subject: mm/collapse: separate scanning a PTE table from collapsing it
has been removed from the -mm tree. Its filename was
mm-collapse-separate-scanning-a-pte-table-from-collapsing-it.patch
This patch was dropped because an updated version will be issued
------------------------------------------------------
From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
Subject: mm/collapse: separate scanning a PTE table from collapsing it
Date: Wed, 16 Sep 2026 10:31:35 +0100
A collapse is two jobs. One reads a PTE table under mmap_lock and decides
whether the range is worth collapsing. The other allocates, isolates,
copies and flushes, and wants the lock given up first.
collapse_single_pmd() did both, so the boundary between them was somewhere
in the middle of a function.
Give each half its own function:
- collapse_scan_pmd() scans one table and only reads. The anonymous
scan that used to carry that name keeps its body as
collapse_scan_anon_pmd(), and collapse_scan_pmd() is now the entry
that picks the anonymous or the file side.
- collapse_run_pmd() does the collapse the scan asked for, and is
handed what the scan returned. SCAN_SUCCEED means there is
something to collapse. SCAN_PTE_MAPPED_HUGEPAGE means the page
cache already holds the PMD folio and only the PTE table is left to
retract. Both are work for the run; anything else is why there is
nothing to do.
collapse_single_pmd() is now the two of them with the mmap_lock drop in
between, so its callers see what they saw before.
What the scan found and the run needs travels in collapse_control. For an
anonymous table that is the orders and the referenced and swapped-out
counts, which mthp_collapse() and collapse_huge_page() now read from there
instead of taking as arguments. For a file it is the file itself and the
offset in it.
The file side moves with the anonymous one. collapse_scan_file() used to
run with mmap_lock already given up, and called collapse_file() itself
when the page cache looked worth it. It now runs under the lock like the
anonymous scan and only reads; the run does the collapse. A file collapse
works on the page cache and never sees a VMA, so the scan takes the file
reference while it still has one and the run gives it back.
That changes what a refused file table costs khugepaged. Every file table
it scanned used to end its pass over that mm, because the lock had been
dropped to scan it; now only a table it goes on to collapse does.
Two things on the file side stop being rescanned. When the page cache
already holds the PMD folio, the scan says so and the run goes straight to
retracting the PTE table. A run that refuses dirty pages and may write
them back retries collapse_file() alone. The checks the scan makes ahead
of it are ones collapse_file() repeats under the page cache lock.
Tracing changes with it. mm_khugepaged_scan_pmd and
mm_khugepaged_scan_file used to fire after the collapse, so for an
accepted table their status field carried what the collapse made of it.
They now fire before it and read SCAN_SUCCEED for an accepted table. What
the collapse then made of it is for mm_collapse_huge_page and
mm_khugepaged_collapse_file to report.
Link: https://lore.kernel.org/20260916093145.4022188-9-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Assisted-by: LLM
Cc: Barry Song <baohua@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jann Horn <jannh@google.com>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
---
mm/collapse.h | 14 +++
mm/khugepaged.c | 166 ++++++++++++++++++++++++++++++++--------------
2 files changed, 130 insertions(+), 50 deletions(-)
--- a/mm/collapse.h~mm-collapse-separate-scanning-a-pte-table-from-collapsing-it
+++ a/mm/collapse.h
@@ -88,6 +88,20 @@ struct collapse_control {
/* Each bit marks a PTE the scan accepted as a collapse source */
DECLARE_BITMAP(eligible_ptes, MAX_PTRS_PER_PTE);
+
+ /*
+ * What a scan found and the run after it needs. Live only between the
+ * two, and read by nobody else.
+ *
+ * The file side takes a reference while it still has the VMA, since a
+ * file collapse works on the page cache and never sees one; the run is
+ * what gives it back.
+ */
+ unsigned long scan_orders;
+ int scan_referenced;
+ int scan_unmapped;
+ struct file *scan_file;
+ pgoff_t scan_pgoff;
};
#endif /* __MM_COLLAPSE_H */
--- a/mm/khugepaged.c~mm-collapse-separate-scanning-a-pte-table-from-collapsing-it
+++ a/mm/khugepaged.c
@@ -1254,8 +1254,8 @@ static pgtable_t alloc_deposit_pte_table
* while allocating a THP, as that could trigger direct reclaim/compaction.
* Note that the VMA must be rechecked after grabbing the mmap_lock again.
*/
-static enum scan_result collapse_huge_page(struct mm_struct *mm, unsigned long start_addr,
- int referenced, int unmapped, struct collapse_control *cc,
+static enum scan_result collapse_huge_page(struct mm_struct *mm,
+ unsigned long start_addr, struct collapse_control *cc,
unsigned int order)
{
const unsigned long pmd_addr = start_addr & HPAGE_PMD_MASK;
@@ -1302,14 +1302,14 @@ static enum scan_result collapse_huge_pa
goto out_nolock;
}
- if (unmapped) {
+ if (cc->scan_unmapped) {
/*
* __collapse_huge_page_swapin() will return with mmap_lock
* released when it fails. So we jump out_nolock directly in
* that case. Continuing to collapse causes inconsistency.
*/
result = __collapse_huge_page_swapin(mm, vma, start_addr, pmd,
- referenced, order);
+ cc->scan_referenced, order);
if (result != SCAN_SUCCEED)
goto out_nolock;
}
@@ -1478,9 +1478,8 @@ static unsigned int max_order_from_offse
* If a collapse is permitted, we attempt to collapse the PTE range into a
* mTHP.
*/
-static enum scan_result mthp_collapse(struct mm_struct *mm,
- unsigned long address, int referenced, int unmapped,
- struct collapse_control *cc, unsigned long enabled_orders)
+static enum scan_result mthp_collapse(struct mm_struct *mm, unsigned long address,
+ struct collapse_control *cc)
{
unsigned int nr_eligible_ptes, nr_ptes, max_ptes_none;
enum scan_result last_result = SCAN_FAIL;
@@ -1493,7 +1492,7 @@ static enum scan_result mthp_collapse(st
while (offset < HPAGE_PMD_NR) {
nr_ptes = 1UL << order;
- if (!test_bit(order, &enabled_orders))
+ if (!test_bit(order, &cc->scan_orders))
goto next_order;
max_ptes_none = collapse_max_ptes_none(cc, NULL, order);
@@ -1501,19 +1500,18 @@ static enum scan_result mthp_collapse(st
offset + nr_ptes);
/*
- * Swap PTEs accepted during the scan are counted in @unmapped,
- * not in cc->eligible_ptes. Account them for the PMD-order
- * candidate.
+ * Swap PTEs accepted during the scan are counted in
+ * cc->scan_unmapped, not in cc->eligible_ptes. Account them for
+ * the PMD-order candidate.
*/
if (is_pmd_order(order))
- nr_eligible_ptes += unmapped;
+ nr_eligible_ptes += cc->scan_unmapped;
if (nr_eligible_ptes >= nr_ptes - max_ptes_none) {
enum scan_result ret;
collapse_address = address + offset * PAGE_SIZE;
- ret = collapse_huge_page(mm, collapse_address, referenced,
- unmapped, cc, order);
+ ret = collapse_huge_page(mm, collapse_address, cc, order);
switch (ret) {
/* Cases where we continue to next collapse candidate */
@@ -1555,7 +1553,7 @@ next_order:
* we must always move to the next offset.
*/
if (order > COLLAPSE_MIN_MTHP_ORDER &&
- (enabled_orders & GENMASK(order - 1, 0))) {
+ (cc->scan_orders & GENMASK(order - 1, 0))) {
order--;
continue;
}
@@ -1580,14 +1578,14 @@ done:
return last_result;
}
-static enum scan_result collapse_scan_pmd(struct mm_struct *mm,
- struct vm_area_struct *vma, unsigned long start_addr,
- bool *lock_dropped, struct collapse_control *cc)
+static enum scan_result collapse_scan_anon_pmd(struct vm_area_struct *vma,
+ unsigned long start_addr, struct collapse_control *cc)
{
const unsigned int max_ptes_shared = collapse_max_ptes_shared(cc, HPAGE_PMD_ORDER);
const unsigned int max_ptes_swap = collapse_max_ptes_swap(cc, HPAGE_PMD_ORDER);
unsigned int max_ptes_none = collapse_max_ptes_none(cc, vma, HPAGE_PMD_ORDER);
enum tva_type tva_flags = cc->policy.tva_type;
+ struct mm_struct *mm = vma->vm_mm;
pmd_t *pmd;
pte_t *pte, *_pte, pteval;
int i;
@@ -1767,12 +1765,9 @@ static enum scan_result collapse_scan_pm
out_unmap:
pte_unmap_unlock(pte, ptl);
if (result == SCAN_SUCCEED) {
- /* collapse_huge_page() expects the lock to be dropped before calling */
- mmap_read_unlock(mm);
- result = mthp_collapse(mm, start_addr, referenced,
- unmapped, cc, enabled_orders);
- /* mmap_lock was released above, set lock_dropped */
- *lock_dropped = true;
+ cc->scan_orders = enabled_orders;
+ cc->scan_referenced = referenced;
+ cc->scan_unmapped = unmapped;
}
out:
trace_mm_khugepaged_scan_pmd(mm, failed_pfn, referenced,
@@ -2762,45 +2757,86 @@ static enum scan_result collapse_scan_fi
else
cc->progress += HPAGE_PMD_NR;
- if (result == SCAN_SUCCEED) {
- if (present < HPAGE_PMD_NR - max_ptes_none) {
- result = SCAN_EXCEED_NONE_PTE;
- count_vm_event(THP_SCAN_EXCEED_NONE_PTE);
- } else {
- result = collapse_file(mm, addr, file, start, cc);
- }
+ if (result == SCAN_SUCCEED && present < HPAGE_PMD_NR - max_ptes_none) {
+ result = SCAN_EXCEED_NONE_PTE;
+ count_vm_event(THP_SCAN_EXCEED_NONE_PTE);
}
- trace_mm_khugepaged_scan_file(mm, failed_pfn, file, present, swap, result);
+ trace_mm_khugepaged_scan_file(mm, failed_pfn, file, present, swap,
+ result);
return result;
}
-/*
- * Try to collapse a single PMD starting at a PMD aligned addr, and return
- * the results.
- */
-static enum scan_result collapse_single_pmd(unsigned long addr,
- struct vm_area_struct *vma, bool *lock_dropped,
- struct collapse_control *cc)
+static void collapse_control_init(struct collapse_control *cc)
+{
+ cc->progress = 0;
+ cc->scan_file = NULL;
+}
+
+/* A scan that took a file reference should have been run */
+static void collapse_put_scan_file(struct collapse_control *cc)
+{
+ if (WARN_ON_ONCE(cc->scan_file)) {
+ fput(cc->scan_file);
+ cc->scan_file = NULL;
+ }
+}
+
+static void collapse_control_release(struct collapse_control *cc)
+{
+ collapse_put_scan_file(cc);
+}
+
+static enum scan_result collapse_scan_pmd(struct vm_area_struct *vma,
+ unsigned long addr, struct collapse_control *cc)
{
- struct mm_struct *mm = vma->vm_mm;
- bool triggered_wb = false;
enum scan_result result;
- struct file *file;
pgoff_t pgoff;
- mmap_assert_locked(mm);
+ mmap_assert_locked(vma->vm_mm);
+ /* Whatever the last scan found has to have been run by now */
+ collapse_put_scan_file(cc);
if (vma_is_anonymous(vma))
- return collapse_scan_pmd(mm, vma, addr, lock_dropped, cc);
+ return collapse_scan_anon_pmd(vma, addr, cc);
- file = get_file(vma->vm_file);
pgoff = linear_page_index(vma, addr);
+ result = collapse_scan_file(vma->vm_mm, addr, vma->vm_file, pgoff, cc);
+ /*
+ * SCAN_PTE_MAPPED_HUGEPAGE is work too: the page cache already holds
+ * the PMD folio, and retracting the PTE table is the run's job.
+ */
+ if (result != SCAN_SUCCEED && result != SCAN_PTE_MAPPED_HUGEPAGE)
+ return result;
- mmap_read_unlock(mm);
- *lock_dropped = true;
+ /*
+ * A file collapse works on the page cache and never sees a VMA, so take
+ * what it needs from this one while it is still here.
+ */
+ cc->scan_file = get_file(vma->vm_file);
+ cc->scan_pgoff = pgoff;
+ return result;
+}
+
+static enum scan_result collapse_run_pmd(struct mm_struct *mm,
+ unsigned long addr, enum scan_result result,
+ struct collapse_control *cc)
+{
+ struct file *file = cc->scan_file;
+ bool triggered_wb = false;
+ pgoff_t pgoff;
+
+ if (!file)
+ return mthp_collapse(mm, addr, cc);
+
+ cc->scan_file = NULL;
+ pgoff = cc->scan_pgoff;
+
+ /* The scan found the PMD folio in place: nothing to collapse */
+ if (result == SCAN_PTE_MAPPED_HUGEPAGE)
+ goto retract;
retry:
- result = collapse_scan_file(mm, addr, file, pgoff, cc);
+ result = collapse_file(mm, addr, file, pgoff, cc);
/* Dirty pages are worth a writeback and one more try, if asked for */
if (cc->policy.writeback_dirty && result == SCAN_PAGE_DIRTY_OR_WRITEBACK &&
@@ -2812,8 +2848,13 @@ retry:
triggered_wb = true;
goto retry;
}
+retract:
fput(file);
+ /*
+ * A PMD folio is in the page cache, whether the collapse just put it
+ * there or found it: retract the PTE table, and map the PMD if asked.
+ */
if (result == SCAN_PTE_MAPPED_HUGEPAGE) {
mmap_read_lock(mm);
if (collapse_test_exit_or_disable(mm))
@@ -2828,6 +2869,28 @@ retry:
return result;
}
+/*
+ * Try to collapse a single PMD starting at a PMD aligned addr, and return
+ * the results.
+ */
+static enum scan_result collapse_single_pmd(unsigned long addr,
+ struct vm_area_struct *vma, bool *lock_dropped,
+ struct collapse_control *cc)
+{
+ struct mm_struct *mm = vma->vm_mm;
+ enum scan_result result;
+
+ result = collapse_scan_pmd(vma, addr, cc);
+ if (result != SCAN_SUCCEED && result != SCAN_PTE_MAPPED_HUGEPAGE)
+ return result;
+
+ /* The collapse takes its own locks, so give this up */
+ mmap_read_unlock(mm);
+ *lock_dropped = true;
+
+ return collapse_run_pmd(mm, addr, result, cc);
+}
+
static void collapse_scan_mm_slot(unsigned int progress_max,
enum scan_result *result, struct collapse_control *cc)
__releases(&khugepaged_mm_lock)
@@ -2970,10 +3033,10 @@ static void khugepaged_do_scan(struct co
lru_add_drain_all();
+ collapse_control_init(cc);
/* One policy for the whole pass, so every table is treated the same */
collapse_policy_khugepaged(&cc->policy);
- cc->progress = 0;
while (true) {
cond_resched();
@@ -3004,6 +3067,8 @@ static void khugepaged_do_scan(struct co
khugepaged_alloc_sleep();
}
}
+
+ collapse_control_release(cc);
}
static bool khugepaged_should_wakeup(void)
@@ -3200,8 +3265,8 @@ int madvise_collapse(struct vm_area_stru
cc = kmalloc_obj(*cc);
if (!cc)
return -ENOMEM;
+ collapse_control_init(cc);
collapse_policy_forced(&cc->policy);
- cc->progress = 0;
lru_add_drain_all();
@@ -3258,6 +3323,7 @@ out_maybelock:
}
out_nolock:
mmap_assert_locked(mm);
+ collapse_control_release(cc);
kfree(cc);
return thps == ((hend - hstart) >> HPAGE_PMD_SHIFT) ? 0
_
Patches currently in -mm which might be from kas@kernel.org are
mm-huge_memory-do-not-touch-frozen-folios-in-deferred_split_isolate.patch
mm-huge_memory-dequeue-the-deferred-split-after-the-split-freeze.patch
mm-huge_memory-add-folio_reset_partially_mapped.patch
mm-collapse-open-code-collapse_single_pmd-in-its-two-callers.patch
mm-collapse-work-out-the-orders-a-vma-allows-once-per-vma.patch
mm-collapse-declare-the-collapse-interface-in-collapseh.patch
mm-collapse-implement-madv_collapse-in-madvisec.patch
selftests-mm-raise-the-khugepaged-test-case-cap.patch
selftests-mm-skip-collapse_compound_extreme-where-the-pmd-is-too-large.patch
selftests-mm-scale-khugepageds-collapse-wait-with-the-pmd-size.patch
selftests-mm-skip-khugepaged-page-cache-cases-without-a-pmd-folio.patch
selftests-mm-make-the-swap-cases-swapout-reliable.patch
selftests-mm-stop-khugepaged-during-the-madv_collapse-cases.patch
selftests-mm-move-is_backed_by_folio-into-vm_util.patch
selftests-mm-add-folio-order-check-for-address-ranges.patch
selftests-mm-add-folio-order-detection-self-check.patch
selftests-mm-add-khugepaged-completion-barrier-helper.patch
selftests-mm-add-order-parameterized-khugepaged-collapse-cases.patch
selftests-mm-parameterize-the-mixed-source-collapse-case-by-source-order.patch
selftests-mm-cover-a-shared-source-collapse-write-race.patch
selftests-mm-run-every-supported-collapse-order-by-default.patch
selftests-mm-check-that-one-khugepaged-pass-collapses-one-window.patch
selftests-mm-add-khugepaged-race-harness.patch
selftests-mm-race-the-collapse-of-windows-with-holes.patch
selftests-mm-add-memory-pressure-threads-to-the-khugepaged-race-harness.patch
selftests-mm-zap-whole-pte-tables-in-the-khugepaged-race-harness.patch
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2026-09-28 21:51 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-28 21:51 [to-be-updated] mm-collapse-separate-scanning-a-pte-table-from-collapsing-it.patch removed from -mm tree Andrew Morton
-- strict thread matches above, loose matches on Subject: below --
2026-09-16 22:42 Andrew Morton
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.