[RFC 0/2] mm: add explicit goals to memcg memory.reclaim

[RFC 0/2] mm: add explicit goals to memcg memory.reclaim

[Li Zhe]

memory.reclaim is used by userspace controllers as a proactive memcg
reclaim interface. On systems without memory tiering, a completed request
is normally interpreted as eviction-oriented progress: pages were freed
and the cgroup's charged footprint can drop.

Memory tiering makes that interpretation ambiguous. The reclaim scanner
may satisfy the requested amount by demoting cold folios from a faster
tier to a slower tier. That is useful aging behavior, but it means a
request can complete even though the cgroup's charged footprint is
unchanged. Userspace controllers that rely on memory.reclaim for actual
cgroup footprint reduction therefore lose the ability to request the same
kind of outcome they get on non-tiered systems.

At the same time, some controllers want the opposite operation: use the
reclaim scanner as a best-effort placement mechanism to move cold folios
from DRAM to CXL or other lower tiers, without evicting them from memory.
This can be used to maintain top-tier headroom or a desired
DRAM/CXL occupancy ratio according to workload policy.

This series keeps the existing behavior as the default, and adds explicit
opt-in goals for userspace controllers that want a more specific outcome
from memcg memory.reclaim:

  echo "1G goal=evict" > memory.reclaim

    Run the normal reclaim scanner and keep demotion enabled as part of
    tier aging, but do not count successful demotion toward completing the
    requested amount. This lets callers ask memory.reclaim to continue
    until eviction progress, rather than demotion-only progress, satisfies
    the request. If the cgroup has no usable demotion target, goal=evict
    falls back to the default goal=progress behavior.

  echo "1G goal=demote" > memory.reclaim

    Use the existing reclaim scanner for best-effort memory tier demotion
    without LRU folio eviction. This is intended for placement control,
    such as maintaining top-tier headroom or a desired DRAM/CXL occupancy
    ratio. Demoted folios remain charged to the cgroup, so this goal does
    not imply memory.current reduction. If the cgroup has no usable
    demotion target, the request fails early with -EAGAIN.

The default remains goal=progress, which is the current memory.reclaim
behavior. In that mode, the kernel's existing reclaim progress accounting
is unchanged and successful demotion can satisfy the requested amount.

Patch 1 preserves the tiered LRU aging model. It changes the completion
accounting for an eviction-oriented request, while still allowing demotion
to happen as an intermediate step. After enough demotion has been done in
a batch, reclaim avoids repeatedly scanning the top tiers in the cgroup's
allowed demotion hierarchy just to produce more demotion-only progress,
while still allowing lower tiers to age or reclaim normally.

Patch 2 constrains a request in the other direction: it allows best-effort
demotion as the requested action, while keeping folios that are not queued
for demotion instead of evicting them from the LRU. It also skips slab
shrinking, because slab reclaim cannot contribute demotion progress and
would otherwise be retried aggressively for an underfilled demotion
request.

This also keeps the ABI scoped to the existing memory.reclaim interface
instead of adding a new cgroup file. The goal key describes what kind of
progress the caller wants from the request; it does not select a source or
target tier, does not provide strict placement guarantees, and remains
subject to cgroup protection, swappiness balancing, reclaim eligibility,
and demotion target availability.

This series is memcg-only. The per-node reclaim interface shares the
parser but has different semantics: node reclaim can legitimately treat
demotion as the desired way to relieve pressure on a node. For that
reason, the new goals are rejected from the per-node path rather than
being given a different meaning there.

Li Zhe (2):
  mm: add goal=evict to memory.reclaim
  mm: add goal=demote to memory.reclaim

 Documentation/admin-guide/cgroup-v2.rst |  26 ++++
 include/linux/memory-tiers.h            |  13 ++
 mm/internal.h                           |   2 +
 mm/memory-tiers.c                       |  35 +++++-
 mm/vmscan.c                             | 154 ++++++++++++++++++++++--
 5 files changed, 219 insertions(+), 11 deletions(-)

-- 
2.20.1

[RFC 1/2] mm: add goal=evict to memory.reclaim

[Li Zhe]

On tiered memory systems, reclaim can demote cold folios from a faster
memory tier to a slower memory tier before they are eventually evicted.
This is part of the aging pipeline: reclaiming directly from a faster
source tier can skip colder folios that have already aged into lower tiers.
Counting demotion as reclaim progress keeps the default memory.reclaim
behavior aligned with that tiered aging model.

However, this makes memory.reclaim completion observably different from
systems without demotion. Without demotion, completing a proactive memcg
reclaim request usually means that the charged footprint was reduced by
eviction. With memory tiering enabled, the same request can complete on
successful demotion alone: folios move to a lower tier, but memory.current
does not drop. Callers that use memory.reclaim to drive memcg footprint
reduction then have to issue repeated short requests to make eviction
progress.

Add goal=evict as an explicit opt-in for eviction-oriented memcg proactive
reclaim. Demotion remains enabled and can still happen as part of reclaim
aging, but successfully demoted folios do not count toward completing the
requested reclaim amount. The default remains goal=progress, preserving
the existing tiered aging behavior and using the existing nr_reclaimed
accounting unchanged. nr_demoted is a subset of nr_reclaimed and is
subtracted only for the evict-goal stop condition.

If the target cgroup has no usable demotion target, treat goal=evict as
goal=progress before reclaim starts. In that case demotion cannot satisfy
the request anyway, so the new key does not need special completion
accounting and reclaim keeps the same behavior as today's memory.reclaim.

goal=evict does not change folio selection or disable demotion. When a
selected folio has an allowed lower-tier target, shrink_folio_list() still
tries to demote it first. The new accounting only prevents those demoted
folios from satisfying an eviction-oriented request by themselves.

The scan limit is deliberately narrow. After an inner reclaim batch has
already demoted at least the batch target, reclaim skips only the top
tier(s) in the cgroup's allowed demotion hierarchy: tiers that can demote
to a lower allowed tier, but are not themselves targets of any higher
allowed tier. Intermediate tiers continue to be scanned so pages already
demoted there can age further downward, and the lowest tier(s) continue
to be scanned so eviction can satisfy the request.

goal=evict is only accepted for memcg memory.reclaim. The per-node
proactive reclaim path uses the same parser, but its reclaim target is a
node capacity target where demotion progress can be the desired outcome.
Reject the new goal key there rather than giving it a different meaning.

Signed-off-by: Li Zhe <lizhe.67@bytedance.com>
---
 Documentation/admin-guide/cgroup-v2.rst |  15 ++++
 include/linux/memory-tiers.h            |  13 +++
 mm/internal.h                           |   1 +
 mm/memory-tiers.c                       |  35 +++++++-
 mm/vmscan.c                             | 110 ++++++++++++++++++++++--
 5 files changed, 166 insertions(+), 8 deletions(-)

diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 7c2a8ed80071..ed87d45cfe21 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -1448,6 +1448,7 @@ The following nested keys are defined.
 
 	  ==========            ================================
 	  swappiness            Swappiness value to reclaim with
+	  goal                  Reclaim request goal
 	  ==========            ================================
 
 	Specifying a swappiness value instructs the kernel to perform
@@ -1458,6 +1459,20 @@ The following nested keys are defined.
 	The valid range for swappiness is [0-200, max], setting
 	swappiness=max exclusively reclaims anonymous memory.
 
+	The goal key controls what kind of progress satisfies the requested
+	reclaim amount. The default is goal=progress, which uses the kernel's
+	normal reclaim progress accounting. On tiered memory systems this can
+	include both eviction and demotion progress.
+
+	goal=evict keeps memory tier demotion enabled as part of reclaim aging,
+	but does not count successfully demoted pages toward completing the
+	requested reclaim amount. Demotion may still be performed on the way,
+	but demotion-only progress is not sufficient to satisfy the request.
+	If no demotion target is available to the cgroup, this behaves like
+	goal=progress. This does not select source or target memory tiers and
+	does not disable demotion. This is best-effort and does not bypass memory
+	protection or other reclaim constraints.
+
   memory.peak
 	A read-write single value file which exists on non-root cgroups.
 
diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.h
index 7999c58629ee..5d869f74deab 100644
--- a/include/linux/memory-tiers.h
+++ b/include/linux/memory-tiers.h
@@ -55,6 +55,7 @@ void mt_put_memory_types(struct list_head *memory_types);
 #ifdef CONFIG_NUMA_MIGRATION
 int next_demotion_node(int node, const nodemask_t *allowed_mask);
 void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets);
+void node_get_allowed_sources(pg_data_t *pgdat, nodemask_t *sources);
 bool node_is_toptier(int node);
 #else
 static inline int next_demotion_node(int node, const nodemask_t *allowed_mask)
@@ -67,6 +68,12 @@ static inline void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *target
 	*targets = NODE_MASK_NONE;
 }
 
+static inline void node_get_allowed_sources(pg_data_t *pgdat,
+					    nodemask_t *sources)
+{
+	*sources = NODE_MASK_NONE;
+}
+
 static inline bool node_is_toptier(int node)
 {
 	return true;
@@ -111,6 +118,12 @@ static inline void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *target
 	*targets = NODE_MASK_NONE;
 }
 
+static inline void node_get_allowed_sources(pg_data_t *pgdat,
+					    nodemask_t *sources)
+{
+	*sources = NODE_MASK_NONE;
+}
+
 static inline bool node_is_toptier(int node)
 {
 	return true;
diff --git a/mm/internal.h b/mm/internal.h
index 38b1165212c9..40cb5900f857 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -74,6 +74,7 @@ unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru,
 
 #define MEMCG_RECLAIM_MAY_SWAP (1 << 1)
 #define MEMCG_RECLAIM_PROACTIVE (1 << 2)
+#define MEMCG_RECLAIM_GOAL_EVICT (1 << 3)
 #define MIN_SWAPPINESS 0
 #define MAX_SWAPPINESS 200
 
diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c
index 54851d8a195b..ec0560228b62 100644
--- a/mm/memory-tiers.c
+++ b/mm/memory-tiers.c
@@ -24,6 +24,8 @@ struct memory_tier {
 	struct device dev;
 	/* All the nodes that are part of all the lower memory tiers. */
 	nodemask_t lower_tier_mask;
+	/* All the nodes that are part of all the higher memory tiers. */
+	nodemask_t higher_tier_mask;
 };
 
 struct demotion_nodes {
@@ -317,6 +319,22 @@ void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets)
 	rcu_read_unlock();
 }
 
+void node_get_allowed_sources(pg_data_t *pgdat, nodemask_t *sources)
+{
+	struct memory_tier *memtier;
+
+	/*
+	 * See node_get_allowed_targets() for the lifetime rules.
+	 */
+	rcu_read_lock();
+	memtier = rcu_dereference(pgdat->memtier);
+	if (memtier)
+		*sources = memtier->higher_tier_mask;
+	else
+		*sources = NODE_MASK_NONE;
+	rcu_read_unlock();
+}
+
 /**
  * next_demotion_node() - Get the next node in the demotion path
  * @node: The starting node to lookup the next node
@@ -385,8 +403,10 @@ static void disable_all_demotion_targets(void)
 		 * to access pgda->memtier.
 		 */
 		memtier = __node_get_memory_tier(node);
-		if (memtier)
+		if (memtier) {
 			memtier->lower_tier_mask = NODE_MASK_NONE;
+			memtier->higher_tier_mask = NODE_MASK_NONE;
+		}
 	}
 	/*
 	 * Ensure that the "disable" is visible across the system.
@@ -428,7 +448,7 @@ static void establish_demotion_targets(void)
 	struct demotion_nodes *nd;
 	int target = NUMA_NO_NODE, node;
 	int distance, best_distance;
-	nodemask_t tier_nodes, lower_tier;
+	nodemask_t tier_nodes, lower_tier, higher_tier;
 
 	lockdep_assert_held_once(&memory_tier_lock);
 
@@ -514,6 +534,17 @@ static void establish_demotion_targets(void)
 		memtier->lower_tier_mask = lower_tier;
 	}
 
+	/*
+	 * Build the higher_tier mask for each node collecting node mask from
+	 * all memory tiers above it.
+	 */
+	higher_tier = NODE_MASK_NONE;
+	list_for_each_entry(memtier, &memory_tiers, list) {
+		tier_nodes = get_memtier_nodemask(memtier);
+		memtier->higher_tier_mask = higher_tier;
+		nodes_or(higher_tier, higher_tier, tier_nodes);
+	}
+
 	dump_demotion_targets();
 }
 
diff --git a/mm/vmscan.c b/mm/vmscan.c
index f11491ee9ed5..cdaab8ad54ac 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -123,6 +123,9 @@ struct scan_control {
 	/* Proactive reclaim invoked by userspace */
 	unsigned int proactive:1;
 
+	/* Eviction-oriented proactive reclaim goal */
+	unsigned int evict_goal:1;
+
 	/*
 	 * Cgroup memory below memory.low is protected as long as we
 	 * don't threaten to OOM. If any cgroup is reclaimed at
@@ -166,9 +169,12 @@ struct scan_control {
 	/* Incremented by the number of inactive pages that were scanned */
 	unsigned long nr_scanned;
 
-	/* Number of pages freed so far during a call to shrink_zones() */
+	/* Number of pages reclaimed, including demotions */
 	unsigned long nr_reclaimed;
 
+	/* Number of reclaimed pages that were demoted */
+	unsigned long nr_demoted;
+
 	struct {
 		unsigned int dirty;
 		unsigned int congested;
@@ -181,6 +187,17 @@ struct scan_control {
 	struct reclaim_state reclaim_state;
 };
 
+static unsigned long reclaim_progress(struct scan_control *sc)
+{
+	if (!sc->evict_goal)
+		return sc->nr_reclaimed;
+
+	if (WARN_ON_ONCE(sc->nr_demoted > sc->nr_reclaimed))
+		return 0;
+
+	return sc->nr_reclaimed - sc->nr_demoted;
+}
+
 #ifdef ARCH_HAS_PREFETCHW
 #define prefetchw_prev_lru_folio(_folio, _base, _field)			\
 	do {								\
@@ -342,6 +359,53 @@ static bool can_demote(int nid, struct scan_control *sc,
 	return !nodes_empty(allowed_mask);
 }
 
+static bool memcg_has_demotion_target(struct mem_cgroup *memcg)
+{
+	int nid;
+
+	for_each_node_state(nid, N_MEMORY) {
+		if (can_demote(nid, NULL, memcg))
+			return true;
+	}
+
+	return false;
+}
+
+static bool memcg_node_is_demotion_target(int target_nid,
+					  struct mem_cgroup *memcg)
+{
+	struct pglist_data *pgdat = NODE_DATA(target_nid);
+	nodemask_t source_mask, target_mask;
+
+	if (!pgdat || !numa_demotion_enabled)
+		return false;
+
+	target_mask = nodemask_of_node(target_nid);
+	mem_cgroup_node_filter_allowed(memcg, &target_mask);
+	if (nodes_empty(target_mask))
+		return false;
+
+	node_get_allowed_sources(pgdat, &source_mask);
+	mem_cgroup_node_filter_allowed(memcg, &source_mask);
+	return !nodes_empty(source_mask);
+}
+
+static bool reclaim_skip_node(pg_data_t *pgdat, struct scan_control *sc)
+{
+	if (!sc->evict_goal || sc->nr_demoted < sc->nr_to_reclaim)
+		return false;
+	if (!can_demote(pgdat->node_id, sc, sc->target_mem_cgroup))
+		return false;
+
+	/*
+	 * Skip tiers that can demote but are not demotion targets from any
+	 * higher tier allowed to the cgroup once this batch's demotion work is
+	 * done. Other tiers keep aging toward lower tiers or eviction.
+	 */
+	return !memcg_node_is_demotion_target(pgdat->node_id,
+					      sc->target_mem_cgroup);
+}
+
 static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
 					  int nid,
 					  struct scan_control *sc)
@@ -1546,6 +1610,7 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 	nr_demoted = demote_folio_list(&demote_folios, pgdat, memcg);
 	nr_reclaimed += nr_demoted;
 	stat->nr_demoted += nr_demoted;
+	sc->nr_demoted += nr_demoted;
 	/* Folios that could not be demoted are still in @demote_folios */
 	if (!list_empty(&demote_folios)) {
 		/* Folios which weren't demoted go back on @folio_list */
@@ -5022,7 +5087,11 @@ static bool should_abort_scan(struct lruvec *lruvec, struct scan_control *sc)
 	if (unlikely(sc->proactive && signal_pending(current)))
 		return true;
 
-	if (sc->nr_reclaimed >= max(sc->nr_to_reclaim, compact_gap(sc->order)))
+	if (reclaim_progress(sc) >=
+	    max(sc->nr_to_reclaim, compact_gap(sc->order)))
+		return true;
+
+	if (reclaim_skip_node(lruvec_pgdat(lruvec), sc))
 		return true;
 
 	/* check the order to exclude compaction-induced reclaim */
@@ -6228,8 +6297,13 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc)
 				   sc->nr_scanned - scanned,
 				   sc->nr_reclaimed - reclaimed);
 
+		if (reclaim_skip_node(pgdat, sc)) {
+			mem_cgroup_iter_break(target_memcg, memcg);
+			break;
+		}
+
 		/* If partial walks are allowed, bail once goal is reached */
-		if (partial && sc->nr_reclaimed >= sc->nr_to_reclaim) {
+		if (partial && reclaim_progress(sc) >= sc->nr_to_reclaim) {
 			mem_cgroup_iter_break(target_memcg, memcg);
 			break;
 		}
@@ -6500,6 +6574,8 @@ static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc)
 		if (zone->zone_pgdat == last_pgdat)
 			continue;
 		last_pgdat = zone->zone_pgdat;
+		if (reclaim_skip_node(last_pgdat, sc))
+			continue;
 		shrink_node(zone->zone_pgdat, sc);
 	}
 
@@ -6564,7 +6640,7 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist,
 		sc->nr_scanned = 0;
 		shrink_zones(zonelist, sc);
 
-		if (sc->nr_reclaimed >= sc->nr_to_reclaim)
+		if (reclaim_progress(sc) >= sc->nr_to_reclaim)
 			break;
 
 		if (sc->compaction_ready)
@@ -6591,8 +6667,8 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist,
 
 	delayacct_freepages_end();
 
-	if (sc->nr_reclaimed)
-		return sc->nr_reclaimed;
+	if (reclaim_progress(sc))
+		return reclaim_progress(sc);
 
 	/* Aborted reclaim to try compaction? don't OOM, then */
 	if (sc->compaction_ready)
@@ -6872,6 +6948,8 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg,
 		.may_unmap = 1,
 		.may_swap = !!(reclaim_options & MEMCG_RECLAIM_MAY_SWAP),
 		.proactive = !!(reclaim_options & MEMCG_RECLAIM_PROACTIVE),
+		.evict_goal = !!(reclaim_options &
+					MEMCG_RECLAIM_GOAL_EVICT),
 	};
 	/*
 	 * Traverse the ZONELIST_FALLBACK zonelist of the current node to put
@@ -7947,11 +8025,15 @@ static unsigned long __node_reclaim(struct pglist_data *pgdat,
 enum {
 	MEMORY_RECLAIM_SWAPPINESS = 0,
 	MEMORY_RECLAIM_SWAPPINESS_MAX,
+	MEMORY_RECLAIM_GOAL_PROGRESS,
+	MEMORY_RECLAIM_GOAL_EVICT,
 	MEMORY_RECLAIM_NULL,
 };
 static const match_table_t tokens = {
 	{ MEMORY_RECLAIM_SWAPPINESS, "swappiness=%d"},
 	{ MEMORY_RECLAIM_SWAPPINESS_MAX, "swappiness=max"},
+	{ MEMORY_RECLAIM_GOAL_PROGRESS, "goal=progress"},
+	{ MEMORY_RECLAIM_GOAL_EVICT, "goal=evict"},
 	{ MEMORY_RECLAIM_NULL, NULL },
 };
 
@@ -7960,6 +8042,7 @@ int user_proactive_reclaim(char *buf,
 {
 	unsigned int nr_retries = MAX_RECLAIM_RETRIES;
 	unsigned long nr_to_reclaim, nr_reclaimed = 0;
+	bool evict_goal = false;
 	int swappiness = -1;
 	char *old_buf, *start;
 	substring_t args[MAX_OPT_ARGS];
@@ -7991,11 +8074,24 @@ int user_proactive_reclaim(char *buf,
 		case MEMORY_RECLAIM_SWAPPINESS_MAX:
 			swappiness = SWAPPINESS_ANON_ONLY;
 			break;
+		case MEMORY_RECLAIM_GOAL_PROGRESS:
+			if (!memcg)
+				return -EINVAL;
+			evict_goal = false;
+			break;
+		case MEMORY_RECLAIM_GOAL_EVICT:
+			if (!memcg)
+				return -EINVAL;
+			evict_goal = true;
+			break;
 		default:
 			return -EINVAL;
 		}
 	}
 
+	if (nr_to_reclaim && evict_goal && !memcg_has_demotion_target(memcg))
+		evict_goal = false;
+
 	while (nr_reclaimed < nr_to_reclaim) {
 		/* Will converge on zero, but reclaim enforces a minimum */
 		unsigned long batch_size = (nr_to_reclaim - nr_reclaimed) / 4;
@@ -8027,6 +8123,8 @@ int user_proactive_reclaim(char *buf,
 
 			reclaim_options = MEMCG_RECLAIM_MAY_SWAP |
 					  MEMCG_RECLAIM_PROACTIVE;
+			if (evict_goal)
+				reclaim_options |= MEMCG_RECLAIM_GOAL_EVICT;
 			reclaimed = try_to_free_mem_cgroup_pages(memcg,
 						 batch_size, gfp_mask,
 						 reclaim_options,
-- 
2.20.1

[RFC 2/2] mm: add goal=demote to memory.reclaim

[Li Zhe]

memory.reclaim normally runs the reclaim engine with the normal set of
reclaim actions. On tiered memory systems this can age cold folios by
demoting them first, but the same request can continue into eviction when
demotion is unavailable or when lower-tier nodes are scanned. That makes
memory.reclaim unsuitable for callers that want to use the existing
reclaim scanner only to move cold folios from DRAM to CXL. For example,
swappiness=0 biases selection toward file-LRU folios, but it still allows
file cache to be evicted if the request keeps reclaiming after demotion
progress.

Add goal=demote as an explicit memcg memory.reclaim goal for best-effort
tier demotion without LRU folio eviction. Successful demotions count
toward the requested amount. The reclaim walk skips nodes that do not have
demotion targets and keeps folios that are not queued for demotion instead
of reclaiming them. The default remains goal=progress, preserving the
existing behavior.

Fail goal=demote with -EAGAIN before reclaim starts when the target cgroup
has no usable demotion target. The option is valid in that case, but the
requested goal cannot make progress; returning early avoids retrying the
reclaim loop just to discover that every node is skipped.

This is useful for userspace controllers that want to manage top-tier
headroom and maintain a desired DRAM/CXL occupancy ratio according to
application policy and current system state. The controller can
proactively move cold folios to lower tiers and adjust the demotion amount
as workload placement, tier occupancy, bandwidth pressure, and latency
targets change. Demoted folios remain charged to the cgroup, so the goal
is placement control rather than memory.current reduction. It does not
provide strict file-only or target-tier placement; it remains best effort
and subject to reclaim eligibility and demotion target availability.

Signed-off-by: Li Zhe <lizhe.67@bytedance.com>
---
 Documentation/admin-guide/cgroup-v2.rst | 11 ++++++
 mm/internal.h                           |  1 +
 mm/vmscan.c                             | 48 ++++++++++++++++++++++---
 3 files changed, 55 insertions(+), 5 deletions(-)

diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index ed87d45cfe21..518a3eafe5d9 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -1473,6 +1473,17 @@ The following nested keys are defined.
 	does not disable demotion. This is best-effort and does not bypass memory
 	protection or other reclaim constraints.
 
+	goal=demote performs best-effort memory tier demotion for LRU folios:
+	successful demotion counts toward the requested amount, nodes without
+	demotion targets are skipped, and folios that cannot be demoted are kept
+	on the LRU instead of being evicted by this request. Slab shrinking is
+	also skipped, as it cannot contribute demotion progress. Demoted folios
+	remain charged to the cgroup, so this is intended for placement control
+	rather than memory.current reduction. It can be combined with swappiness
+	to use the existing anon/file reclaim balancing, for example
+	swappiness=0 to prefer file-LRU folios. If no demotion target is available
+	to the cgroup, this returns -EAGAIN.
+
   memory.peak
 	A read-write single value file which exists on non-root cgroups.
 
diff --git a/mm/internal.h b/mm/internal.h
index 40cb5900f857..33d75f6a5c3d 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -75,6 +75,7 @@ unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru,
 #define MEMCG_RECLAIM_MAY_SWAP (1 << 1)
 #define MEMCG_RECLAIM_PROACTIVE (1 << 2)
 #define MEMCG_RECLAIM_GOAL_EVICT (1 << 3)
+#define MEMCG_RECLAIM_GOAL_DEMOTE (1 << 4)
 #define MIN_SWAPPINESS 0
 #define MAX_SWAPPINESS 200
 
diff --git a/mm/vmscan.c b/mm/vmscan.c
index cdaab8ad54ac..1a98e83f35c4 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -126,6 +126,9 @@ struct scan_control {
 	/* Eviction-oriented proactive reclaim goal */
 	unsigned int evict_goal:1;
 
+	/* Demotion-only proactive reclaim goal */
+	unsigned int demote_goal:1;
+
 	/*
 	 * Cgroup memory below memory.low is protected as long as we
 	 * don't threaten to OOM. If any cgroup is reclaimed at
@@ -189,6 +192,9 @@ struct scan_control {
 
 static unsigned long reclaim_progress(struct scan_control *sc)
 {
+	if (sc->demote_goal)
+		return sc->nr_demoted;
+
 	if (!sc->evict_goal)
 		return sc->nr_reclaimed;
 
@@ -392,6 +398,12 @@ static bool memcg_node_is_demotion_target(int target_nid,
 
 static bool reclaim_skip_node(pg_data_t *pgdat, struct scan_control *sc)
 {
+	if (sc->demote_goal) {
+		if (sc->nr_demoted >= sc->nr_to_reclaim)
+			return true;
+		return !can_demote(pgdat->node_id, sc, sc->target_mem_cgroup);
+	}
+
 	if (!sc->evict_goal || sc->nr_demoted < sc->nr_to_reclaim)
 		return false;
 	if (!can_demote(pgdat->node_id, sc, sc->target_mem_cgroup))
@@ -1316,6 +1328,9 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 			continue;
 		}
 
+		if (sc->demote_goal)
+			goto keep_locked;
+
 		/*
 		 * Anonymous process memory has backing store?
 		 * Try to allocate it some swap space here.
@@ -5185,7 +5200,8 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc)
 
 	need_rotate = try_to_shrink_lruvec(lruvec, sc);
 
-	shrink_slab(sc->gfp_mask, pgdat->node_id, memcg, sc->priority);
+	if (!sc->demote_goal)
+		shrink_slab(sc->gfp_mask, pgdat->node_id, memcg, sc->priority);
 
 	if (!sc->proactive)
 		vmpressure(sc->gfp_mask, sc->order, memcg, false,
@@ -6288,8 +6304,9 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc)
 
 		shrink_lruvec(lruvec, sc);
 
-		shrink_slab(sc->gfp_mask, pgdat->node_id, memcg,
-			    sc->priority);
+		if (!sc->demote_goal)
+			shrink_slab(sc->gfp_mask, pgdat->node_id, memcg,
+				    sc->priority);
 
 		/* Record the group's reclaim efficiency */
 		if (!sc->proactive)
@@ -6950,6 +6967,7 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg,
 		.proactive = !!(reclaim_options & MEMCG_RECLAIM_PROACTIVE),
 		.evict_goal = !!(reclaim_options &
 					MEMCG_RECLAIM_GOAL_EVICT),
+		.demote_goal = !!(reclaim_options & MEMCG_RECLAIM_GOAL_DEMOTE),
 	};
 	/*
 	 * Traverse the ZONELIST_FALLBACK zonelist of the current node to put
@@ -8027,6 +8045,7 @@ enum {
 	MEMORY_RECLAIM_SWAPPINESS_MAX,
 	MEMORY_RECLAIM_GOAL_PROGRESS,
 	MEMORY_RECLAIM_GOAL_EVICT,
+	MEMORY_RECLAIM_GOAL_DEMOTE,
 	MEMORY_RECLAIM_NULL,
 };
 static const match_table_t tokens = {
@@ -8034,6 +8053,7 @@ static const match_table_t tokens = {
 	{ MEMORY_RECLAIM_SWAPPINESS_MAX, "swappiness=max"},
 	{ MEMORY_RECLAIM_GOAL_PROGRESS, "goal=progress"},
 	{ MEMORY_RECLAIM_GOAL_EVICT, "goal=evict"},
+	{ MEMORY_RECLAIM_GOAL_DEMOTE, "goal=demote"},
 	{ MEMORY_RECLAIM_NULL, NULL },
 };
 
@@ -8043,6 +8063,7 @@ int user_proactive_reclaim(char *buf,
 	unsigned int nr_retries = MAX_RECLAIM_RETRIES;
 	unsigned long nr_to_reclaim, nr_reclaimed = 0;
 	bool evict_goal = false;
+	bool demote_goal = false;
 	int swappiness = -1;
 	char *old_buf, *start;
 	substring_t args[MAX_OPT_ARGS];
@@ -8078,19 +8099,34 @@ int user_proactive_reclaim(char *buf,
 			if (!memcg)
 				return -EINVAL;
 			evict_goal = false;
+			demote_goal = false;
 			break;
 		case MEMORY_RECLAIM_GOAL_EVICT:
 			if (!memcg)
 				return -EINVAL;
 			evict_goal = true;
 			break;
+		case MEMORY_RECLAIM_GOAL_DEMOTE:
+			if (!memcg)
+				return -EINVAL;
+			demote_goal = true;
+			break;
 		default:
 			return -EINVAL;
 		}
 	}
 
-	if (nr_to_reclaim && evict_goal && !memcg_has_demotion_target(memcg))
-		evict_goal = false;
+	if (evict_goal && demote_goal)
+		return -EINVAL;
+
+	if (nr_to_reclaim && (evict_goal || demote_goal)) {
+		bool has_demotion_target = memcg_has_demotion_target(memcg);
+
+		if (demote_goal && !has_demotion_target)
+			return -EAGAIN;
+		if (evict_goal && !has_demotion_target)
+			evict_goal = false;
+	}
 
 	while (nr_reclaimed < nr_to_reclaim) {
 		/* Will converge on zero, but reclaim enforces a minimum */
@@ -8125,6 +8161,8 @@ int user_proactive_reclaim(char *buf,
 					  MEMCG_RECLAIM_PROACTIVE;
 			if (evict_goal)
 				reclaim_options |= MEMCG_RECLAIM_GOAL_EVICT;
+			if (demote_goal)
+				reclaim_options |= MEMCG_RECLAIM_GOAL_DEMOTE;
 			reclaimed = try_to_free_mem_cgroup_pages(memcg,
 						 batch_size, gfp_mask,
 						 reclaim_options,
-- 
2.20.1