* [RFC 0/1] mm/vmscan: reduce lru_lock contention via vmstat-derived scan-balance cost
@ 2026-06-26 12:19 Usama Arif
2026-06-26 12:19 ` [RFC 1/1] " Usama Arif
0 siblings, 1 reply; 2+ messages in thread
From: Usama Arif @ 2026-06-26 12:19 UTC (permalink / raw)
To: Andrew Morton, david, ljs, liam, vbabka, rppt, surenb, mhocko,
kasong, qi.zheng, shakeel.butt, axelrasmussen, yuanchu, weixugc,
chrisl, nphamcs, baoquan.he, youngjun.park, hannes,
roman.gushchin, muchun.song, linux-mm, linux-kernel, cgroups,
rientjes, kernel-team
Cc: Usama Arif
The anon/file scan balance heuristic in get_scan_count() is fed by two
scalars in struct lruvec (anon_cost, file_cost) that every reclaim
producer updates under lruvec->lru_lock. The cost-recording work
itself is trivial, but it both contends for and contributes to
contention on lru_lock - which is often a contention point
on memory-pressured workloads. Specifically:
- shrink_inactive_list() re-acquires lru_lock at function exit just
to call lru_note_cost_unlock_irq().
- shrink_active_list() does the same after rotation accounting.
- workingset_refault() takes folio_lruvec_lock_irq() purely to
record the refault cost.
- prepare_scan_control() snapshots anon_cost/file_cost under
lru_lock.
- lru_note_cost_unlock_irq() itself walks parent_lruvec() and
re-acquires lru_lock on every ancestor, multiplying the cost
of every update by memcg-hierarchy depth.
This patch removes those producer-side acquisitions entirely. The
producer-local inputs (PGROTATE_*, PGRECLAIM_PAGEOUT_*) become
per-LRU vmstat counters; WORKINGSET_RESTORE_* already captures the
refault input. prepare_scan_control() reads the raw cost signal
lock-free from those vmstats and folds the delta into a per-lruvec
accumulator. A dedicated per-lruvec cost_lock — not touched by
isolate_lru_folios(), move_folios_to_lru(), or folio_add_lru() —
serialises the accumulator RMW and the lrusize/4 halving check.
Hierarchy aggregation is implicit in rstat propagation, so the
parent_lruvec() walk and the lru_reparent_memcg() cost-splice both
disappear.
Trade-offs:
- Signal freshness is slightly worse: cost reads see rstat-
aggregated values that can lag until periodic / reader-triggered
flushing. Decay timing is also coarser since multiple producer
events may be batched into one read-side halving check.
The cost signal is a heuristic feeding the anon-vs-file split,
it's not a precise control loop — it's deliberately smoothed by
the lrusize/4 halving. Producing/consuming it with a tiny lag should
not be perceptible.
- Per-lruvec footprint grows by 2 unsigned longs + a spinlock,
its a small cost.
== Numbers ==
Tested on a 176-core, 256 GB host. The benchmark drives sustained
swap-out/refault inside a tight memcg using vm-scalability/usemem:
usemem -n 16 --prealloc --prefault --random $((256*1024*1024))
run inside a two-level memcg with memory.max=512M on the leaf
(4 GB anon working set has to fit in 512 MB -> continuous
shrink_inactive_list + workingset_refault). A 16 GB swap file
is used. Measurement is a 30 s `perf lock record -a` window
over otherwise-idle hardware.
Workload rates are identical on both kernels (the bench drives the
same memory pressure):
baseline patched delta
pgscan_direct / s 172,662 171,817 ~0%
pgsteal_direct / s 67,162 66,306 ~0%
workingset_refault_anon / s
40,696 39,830 ~0%
perf lock contention (total wait per 30 s window):
Lock Name Before After % change
shrink_lruvec+0x770 722.84 ms 0 -100% (eliminated)
(= lru_note_cost_unlock_irq)
workingset_refault+0x167 385.26 ms 0 -100% (eliminated)
(= lru_note_cost_refault)
shrink_node+0x4ad 689.43 ms 26.95 ms -96%
shrink_active_list 208.34 ms 15.97 ms -92%
lru_add_drain_cpu+0x34 1.96 s 917.71 ms -53%
Total LRU lock wait ~4.23 s ~1.66 s -61%
The two specific contention sites the patch removes
(shrink_lruvec+0x770 = lru_note_cost_unlock_irq;
workingset_refault+0x167 = lru_note_cost_refault) are completely
absent from the patched perf-lock-contention output.
Secondary reductions in shrink_node, shrink_active_list,
lru_add_drain_cpu and pgrefill/pgactivate look like knock-on
effects from removing the cost-recording overhead and the
parent_lruvec walk.
The remaining ~1.66 s of LRU lock wait on the patched kernel is
dominated by the per-CPU pagevec drain (lru_add_drain_cpu) and the
main reclaim path in shrink_lruvec.
The numbers above can be reproduced using the script in [1].
== Alternatives considered ==
1. cost_lock for both producer and consumer (no vmstat indirection):
Keep the producer loop, just swap lru_lock for a new per-lruvec
cost_lock. Decouples cost from LRU manipulation, but producers
still synchronously contend on cost_lock, the parent_lruvec()
walk is still required (O(memcg-depth) acquisitions per recording,
now on cost_lock), and lru_reparent_memcg() still needs explicit
cost-splice. We can do much better and this series removes the
producer lock entirely and gets hierarchy propagation for "free"
via rstat.
2. Attempt to switch to using MGLRU's scan model:
MGLRU has no anon_cost/file_cost at all. It replaces the cost
heuristic with generation-based aging: per-LRU sequence numbers
(min_seq/max_seq) age folios into generations, and the
older-generation type is the one to scan. So
lru_note_cost_unlock_irq() / lru_note_cost_refault() are simply
not called when lru_gen_enabled() — by design it sidesteps every
concern this patch addresses.
But MGLRU is not a substitute for fixing classic LRU:
- It relies on a lot of things including per-lruvec generation
lists, bloom filters, mm_struct walk infrastructure, working-set
protection tiers and a whole sysfs interface. Replacing
classic LRU's cost recording with the MGLRU model would
mean dragging in all of that.
- It changes scan-balance semantics, not just the locking, so
it's a heuristic change we would need to evaluate separately.
There are known regressions (database/anon-heavy workloads
sensitive to swappiness, or file-cache-dominated workloads
where MGLRU's bloom-filter protection differs from classic
refault tracking).
This patch preserves classic-LRU semantics.
3. Atomic cost counter:
lrusize/4 halving has no clean atomic form, and the parent
walk still has to run explicitly. Reusing vmstats gives per-CPU
aggregation AND rstat hierarchy propagation for free.
4. Drop cost_lock from the existing patch and reuse lru_lock in the
consumer (prepare_scan_control()):
Saves 1 lock space per lruvec but re-couples the cost path to LRU
manipulation, though just from the consumer side this time.
prepare_scan_control() runs at the start of every shrink_lruvec()
cycle, so under sustained memory pressure it would take lru_lock
on the hot path and block isolate_lru_folios() /
move_folios_to_lru() / folio_add_lru() i.e. when reclaim is
in flight. A dedicated cost_lock is never taken by anyone except
the consumer cost calucation.
[1] https://gist.github.com/uarif1/a4eb33a86c5b2d7bbc55b42f0956e884
Usama Arif (1):
mm/vmscan: reduce lru_lock contention via vmstat-derived scan-balance
cost
include/linux/mmzone.h | 11 +++++--
include/linux/swap.h | 3 --
mm/memcontrol-v1.c | 4 +--
mm/memcontrol.c | 4 +++
mm/mmzone.c | 1 +
mm/swap.c | 69 ------------------------------------------
mm/vmscan.c | 64 +++++++++++++++++++++++++++++++++------
mm/vmstat.c | 4 +++
mm/workingset.c | 5 ---
9 files changed, 74 insertions(+), 91 deletions(-)
--
2.53.0-Meta
^ permalink raw reply [flat|nested] 2+ messages in thread
* [RFC 1/1] mm/vmscan: reduce lru_lock contention via vmstat-derived scan-balance cost
2026-06-26 12:19 [RFC 0/1] mm/vmscan: reduce lru_lock contention via vmstat-derived scan-balance cost Usama Arif
@ 2026-06-26 12:19 ` Usama Arif
0 siblings, 0 replies; 2+ messages in thread
From: Usama Arif @ 2026-06-26 12:19 UTC (permalink / raw)
To: Andrew Morton, david, ljs, liam, vbabka, rppt, surenb, mhocko,
kasong, qi.zheng, shakeel.butt, axelrasmussen, yuanchu, weixugc,
chrisl, nphamcs, baoquan.he, youngjun.park, hannes,
roman.gushchin, muchun.song, linux-mm, linux-kernel, cgroups,
rientjes, kernel-team
Cc: Usama Arif
The anon/file scan balance in get_scan_count() is driven by two scalars
in struct lruvec, anon_cost and file_cost, accumulated by every reclaim
producer under lruvec->lru_lock. The acquisition sites for cost work
specifically are:
- shrink_inactive_list() re-takes lru_lock at function exit purely
to call lru_note_cost_unlock_irq() with (nr_pageout, nr_scanned -
nr_reclaimed). One acquisition per inactive shrink.
- shrink_active_list() does the same with (0, nr_rotated). One
acquisition per active shrink.
- workingset_refault() takes the lock via folio_lruvec_lock_irq()
purely to record the refault cost. One acquisition per refault.
- prepare_scan_control() takes lru_lock just to snapshot the two
scalars into sc->{anon,file}_cost.
- lru_note_cost_unlock_irq() itself walks parent_lruvec and
re-acquires lru_lock on each ancestor to propagate the update,
adding O(memcg-depth) acquisitions per producer call.
This hurts because lru_lock is already a heavy contention point on
memory-heavy workloads: every isolate_lru_folios(), move_folios_to_lru()
and folio_add_lru() takes it. The cost work itself is trivial (two
scalar bumps and one comparison), but it contends with and causes
contention for actual LRU manipulation. The parent_lruvec() also walks
multiplies cost-update overhead by memcg hierarchy depth.
Replace the producer-side accumulators with a read-side accumulator fed
from per-LRU vmstat counters. The old producer formula was:
cost = nr_io * SWAP_CLUSTER_MAX + nr_rotated
Add explicit node_stat counters for the producer-local inputs:
PGRECLAIM_PAGEOUT_{ANON,FILE} - reclaim-driven pageout submissions
(formerly stat.nr_pageout, weighted
by SWAP_CLUSTER_MAX).
PGROTATE_{ANON,FILE} - reclaim-driven rotations, bumped from
both shrink_inactive_list (by
nr_scanned - nr_reclaimed) and
shrink_active_list (by nr_rotated),
unweighted.
WORKINGSET_RESTORE_{ANON,FILE} already captures the refault IO that
lru_note_cost_refault() used to bill.
In prepare_scan_control() the raw cost signal is recomputed lock-free of
lru_lock from monotonic counters:
now = (PGRECLAIM_PAGEOUT_X + WORKINGSET_RESTORE_X) * SWAP_CLUSTER_MAX
+ PGROTATE_X
The delta against a per-lruvec prev_cost[] snapshot is folded into
cost_accum[]. The lrusize/4 halving threshold is preserved, but the
decay check now happens at the read site instead of on every producer
update.
A dedicated per-lruvec spinlock, cost_lock, serialises the prev_cost
RMW, the accumulator update, and the halving check against concurrent
reclaimers in the same memcg+node.
Hierarchy aggregation is now implicit in the vmstat accounting. The
producer-side parent_lruvec() walk and lru_reparent_memcg() cost splice
existed only because anon_cost/file_cost were private lruvec fields. With
the cost expressed as lruvec vmstats, rstat propagates the underlying
counters through the memcg hierarchy and prepare_scan_control() consumes
the same ratelimited rstat view as the surrounding reclaim heuristics.
Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
include/linux/mmzone.h | 11 +++++--
include/linux/swap.h | 3 --
mm/memcontrol-v1.c | 4 +--
mm/memcontrol.c | 4 +++
mm/mmzone.c | 1 +
mm/swap.c | 69 ------------------------------------------
mm/vmscan.c | 64 +++++++++++++++++++++++++++++++++------
mm/vmstat.c | 4 +++
mm/workingset.c | 5 ---
9 files changed, 74 insertions(+), 91 deletions(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index ca2712187147..0627622a5184 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -323,6 +323,10 @@ enum node_stat_item {
PGSCAN_PROACTIVE,
PGSCAN_ANON,
PGSCAN_FILE,
+ PGRECLAIM_PAGEOUT_ANON,
+ PGRECLAIM_PAGEOUT_FILE,
+ PGROTATE_ANON,
+ PGROTATE_FILE,
PGREFILL,
#ifdef CONFIG_HUGETLB_PAGE
NR_HUGETLB,
@@ -763,9 +767,12 @@ struct lruvec {
* These track the cost of reclaiming one LRU - file or anon -
* over the other. As the observed cost of reclaiming one LRU
* increases, the reclaim scan balance tips toward the other.
+ * Updated and decayed at prepare_scan_control() time; cost_lock
+ * serialises that update.
*/
- unsigned long anon_cost;
- unsigned long file_cost;
+ unsigned long prev_cost[ANON_AND_FILE];
+ unsigned long cost_accum[ANON_AND_FILE];
+ spinlock_t cost_lock;
/* Non-resident age, driven by LRU movement */
atomic_long_t nonresident_age;
/* Refaults at the time of last reclaim cycle */
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 6d72778e6cc3..d35a4761ebd7 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -309,9 +309,6 @@ extern unsigned long totalreserve_pages;
/* linux/mm/swap.c */
-void lru_note_cost_unlock_irq(struct lruvec *lruvec, bool file,
- unsigned int nr_io, unsigned int nr_rotated);
-void lru_note_cost_refault(struct folio *);
void folio_add_lru(struct folio *);
void folio_add_lru_vma(struct folio *, struct vm_area_struct *);
void mark_page_accessed(struct page *);
diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c
index 765069211567..c7a52bb68f4c 100644
--- a/mm/memcontrol-v1.c
+++ b/mm/memcontrol-v1.c
@@ -1988,8 +1988,8 @@ void memcg1_stat_format(struct mem_cgroup *memcg, struct seq_buf *s)
for_each_online_pgdat(pgdat) {
mz = memcg->nodeinfo[pgdat->node_id];
- anon_cost += mz->lruvec.anon_cost;
- file_cost += mz->lruvec.file_cost;
+ anon_cost += mz->lruvec.cost_accum[WORKINGSET_ANON];
+ file_cost += mz->lruvec.cost_accum[WORKINGSET_FILE];
}
seq_buf_printf(s, "anon_cost %lu\n", anon_cost);
seq_buf_printf(s, "file_cost %lu\n", file_cost);
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 56cd4af08232..3c068ebefd97 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -419,6 +419,10 @@ static const unsigned int memcg_node_stat_items[] = {
PGSCAN_PROACTIVE,
PGSCAN_ANON,
PGSCAN_FILE,
+ PGRECLAIM_PAGEOUT_ANON,
+ PGRECLAIM_PAGEOUT_FILE,
+ PGROTATE_ANON,
+ PGROTATE_FILE,
PGREFILL,
#ifdef CONFIG_HUGETLB_PAGE
NR_HUGETLB,
diff --git a/mm/mmzone.c b/mm/mmzone.c
index 0c8f181d9d50..17139db4d291 100644
--- a/mm/mmzone.c
+++ b/mm/mmzone.c
@@ -78,6 +78,7 @@ void lruvec_init(struct lruvec *lruvec)
memset(lruvec, 0, sizeof(struct lruvec));
spin_lock_init(&lruvec->lru_lock);
+ spin_lock_init(&lruvec->cost_lock);
zswap_lruvec_state_init(lruvec);
for_each_lru(lru)
diff --git a/mm/swap.c b/mm/swap.c
index 588f50d8f1a8..74b281778cbc 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -272,73 +272,6 @@ void folio_rotate_reclaimable(struct folio *folio)
folio_batch_add_and_move(folio, lru_move_tail);
}
-void lru_note_cost_unlock_irq(struct lruvec *lruvec, bool file,
- unsigned int nr_io, unsigned int nr_rotated)
- __releases(lruvec->lru_lock)
- __releases(rcu)
-{
- unsigned long cost;
-
- /*
- * Reflect the relative cost of incurring IO and spending CPU
- * time on rotations. This doesn't attempt to make a precise
- * comparison, it just says: if reloads are about comparable
- * between the LRU lists, or rotations are overwhelmingly
- * different between them, adjust scan balance for CPU work.
- */
- cost = nr_io * SWAP_CLUSTER_MAX + nr_rotated;
- if (!cost) {
- spin_unlock_irq(&lruvec->lru_lock);
- rcu_read_unlock();
- return;
- }
-
- for (;;) {
- unsigned long lrusize;
-
- /* Record cost event */
- if (file)
- lruvec->file_cost += cost;
- else
- lruvec->anon_cost += cost;
-
- /*
- * Decay previous events
- *
- * Because workloads change over time (and to avoid
- * overflow) we keep these statistics as a floating
- * average, which ends up weighing recent refaults
- * more than old ones.
- */
- lrusize = lruvec_page_state(lruvec, NR_INACTIVE_ANON) +
- lruvec_page_state(lruvec, NR_ACTIVE_ANON) +
- lruvec_page_state(lruvec, NR_INACTIVE_FILE) +
- lruvec_page_state(lruvec, NR_ACTIVE_FILE);
-
- if (lruvec->file_cost + lruvec->anon_cost > lrusize / 4) {
- lruvec->file_cost /= 2;
- lruvec->anon_cost /= 2;
- }
-
- spin_unlock_irq(&lruvec->lru_lock);
- lruvec = parent_lruvec(lruvec);
- if (!lruvec) {
- rcu_read_unlock();
- break;
- }
- spin_lock_irq(&lruvec->lru_lock);
- }
-}
-
-void lru_note_cost_refault(struct folio *folio)
-{
- struct lruvec *lruvec;
-
- lruvec = folio_lruvec_lock_irq(folio);
- lru_note_cost_unlock_irq(lruvec, folio_is_file_lru(folio),
- folio_nr_pages(folio), 0);
-}
-
static void lru_activate(struct lruvec *lruvec, struct folio *folio)
{
long nr_pages = folio_nr_pages(folio);
@@ -1164,8 +1097,6 @@ void lru_reparent_memcg(struct mem_cgroup *memcg, struct mem_cgroup *parent, int
child_lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid));
parent_lruvec = mem_cgroup_lruvec(parent, NODE_DATA(nid));
- parent_lruvec->anon_cost += child_lruvec->anon_cost;
- parent_lruvec->file_cost += child_lruvec->file_cost;
for_each_lru(lru)
lruvec_reparent_lru(child_lruvec, parent_lruvec, lru, nid);
diff --git a/mm/vmscan.c b/mm/vmscan.c
index e8a90911bf88..6d187f0f8bf8 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2043,10 +2043,13 @@ static unsigned long shrink_inactive_list(unsigned long nr_to_scan,
item = PGSTEAL_KSWAPD + reclaimer_offset(sc);
mod_lruvec_state(lruvec, item, nr_reclaimed);
mod_lruvec_state(lruvec, PGSTEAL_ANON + file, nr_reclaimed);
+ if (stat.nr_pageout)
+ mod_lruvec_state(lruvec, PGRECLAIM_PAGEOUT_ANON + file,
+ stat.nr_pageout);
+ if (nr_scanned > nr_reclaimed)
+ mod_lruvec_state(lruvec, PGROTATE_ANON + file,
+ nr_scanned - nr_reclaimed);
- lruvec_lock_irq(lruvec);
- lru_note_cost_unlock_irq(lruvec, file, stat.nr_pageout,
- nr_scanned - nr_reclaimed);
handle_reclaim_writeback(nr_taken, pgdat, sc, &stat);
trace_mm_vmscan_lru_shrink_inactive(pgdat->node_id,
nr_scanned, nr_reclaimed, &stat, sc->priority, file);
@@ -2152,9 +2155,9 @@ static void shrink_active_list(unsigned long nr_to_scan,
count_vm_events(PGDEACTIVATE, nr_deactivate);
count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, nr_deactivate);
mod_node_page_state(pgdat, NR_ISOLATED_ANON + file, -nr_taken);
+ if (nr_rotated)
+ mod_lruvec_state(lruvec, PGROTATE_ANON + file, nr_rotated);
- lruvec_lock_irq(lruvec);
- lru_note_cost_unlock_irq(lruvec, file, 0, nr_rotated);
trace_mm_vmscan_lru_shrink_active(pgdat->node_id, nr_taken, nr_activate,
nr_deactivate, nr_rotated, sc->priority, file);
}
@@ -2303,12 +2306,53 @@ static void prepare_scan_control(pg_data_t *pgdat, struct scan_control *sc)
mem_cgroup_flush_stats_ratelimited(sc->target_mem_cgroup);
/*
- * Determine the scan balance between anon and file LRUs.
+ * Determine the scan balance between anon and file LRUs from per-LRU
+ * vmstat counters. The raw cost per side is:
+ *
+ * PGROTATE - reclaim-driven rotations, bumped from both
+ * shrink_inactive_list and shrink_active_list
+ * (CPU work).
+ * PGRECLAIM_PAGEOUT - reclaim-driven pageout IO.
+ * WORKINGSET_RESTORE - refaults of previously-workingset pages.
+ *
+ * The two IO terms are weighted by SWAP_CLUSTER_MAX to reflect the
+ * higher cost of an IO over a rotation.
+ *
+ * Reads are lock-free per-cpu sum collations, rstat-aggregated up
+ * the memcg hierarchy by mem_cgroup_flush_stats_ratelimited() above.
+ *
+ * The delta against prev_cost is folded into cost_accum, which is
+ * halved on both sides whenever their sum exceeds lrusize/4.
+ * cost_lock serialises concurrent reclaimers in the same memcg+node.
*/
- spin_lock_irq(&target_lruvec->lru_lock);
- sc->anon_cost = target_lruvec->anon_cost;
- sc->file_cost = target_lruvec->file_cost;
- spin_unlock_irq(&target_lruvec->lru_lock);
+ spin_lock(&target_lruvec->cost_lock);
+ for (int f = 0; f <= 1; f++) {
+ unsigned long now, delta;
+
+ now = lruvec_page_state(target_lruvec, PGROTATE_ANON + f) +
+ (lruvec_page_state(target_lruvec,
+ PGRECLAIM_PAGEOUT_ANON + f) +
+ lruvec_page_state(target_lruvec,
+ WORKINGSET_RESTORE_BASE + f)) *
+ SWAP_CLUSTER_MAX;
+ delta = now - target_lruvec->prev_cost[f];
+ target_lruvec->prev_cost[f] = now;
+ target_lruvec->cost_accum[f] += delta;
+ }
+ unsigned long lrusize =
+ lruvec_page_state(target_lruvec, NR_INACTIVE_ANON) +
+ lruvec_page_state(target_lruvec, NR_ACTIVE_ANON) +
+ lruvec_page_state(target_lruvec, NR_INACTIVE_FILE) +
+ lruvec_page_state(target_lruvec, NR_ACTIVE_FILE);
+
+ if (target_lruvec->cost_accum[WORKINGSET_ANON] +
+ target_lruvec->cost_accum[WORKINGSET_FILE] > lrusize / 4) {
+ target_lruvec->cost_accum[WORKINGSET_ANON] /= 2;
+ target_lruvec->cost_accum[WORKINGSET_FILE] /= 2;
+ }
+ sc->anon_cost = target_lruvec->cost_accum[WORKINGSET_ANON];
+ sc->file_cost = target_lruvec->cost_accum[WORKINGSET_FILE];
+ spin_unlock(&target_lruvec->cost_lock);
/*
* Target desirable inactive:active list ratios for the anon
diff --git a/mm/vmstat.c b/mm/vmstat.c
index f534972f517d..2dfed5e9b64d 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -1289,6 +1289,10 @@ const char * const vmstat_text[] = {
[I(PGSCAN_PROACTIVE)] = "pgscan_proactive",
[I(PGSCAN_ANON)] = "pgscan_anon",
[I(PGSCAN_FILE)] = "pgscan_file",
+ [I(PGRECLAIM_PAGEOUT_ANON)] = "pgreclaim_pageout_anon",
+ [I(PGRECLAIM_PAGEOUT_FILE)] = "pgreclaim_pageout_file",
+ [I(PGROTATE_ANON)] = "pgrotate_anon",
+ [I(PGROTATE_FILE)] = "pgrotate_file",
[I(PGREFILL)] = "pgrefill",
#ifdef CONFIG_HUGETLB_PAGE
[I(NR_HUGETLB)] = "nr_hugetlb",
diff --git a/mm/workingset.c b/mm/workingset.c
index f351798e723a..7ac2b88c80ae 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -584,11 +584,6 @@ void workingset_refault(struct folio *folio, void *shadow)
/* Folio was active prior to eviction */
if (workingset) {
folio_set_workingset(folio);
- /*
- * XXX: Move to folio_add_lru() when it supports new vs
- * putback
- */
- lru_note_cost_refault(folio);
mod_lruvec_state(lruvec, WORKINGSET_RESTORE_BASE + file, nr);
}
out:
--
2.53.0-Meta
^ permalink raw reply related [flat|nested] 2+ messages in thread
end of thread, other threads:[~2026-06-26 12:20 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-26 12:19 [RFC 0/1] mm/vmscan: reduce lru_lock contention via vmstat-derived scan-balance cost Usama Arif
2026-06-26 12:19 ` [RFC 1/1] " Usama Arif
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox