[RFC PATCH v2 0/3] mm: avoid large folio splits when swap is unavailable

[RFC PATCH v2 0/3] mm: avoid large folio splits when swap is unavailable

[Xueyuan Chen]

This is an RFC v2 of Barry's original RFC patch, "mm: Avoiding split
large folios if swap has no space":

https://lore.kernel.org/r/20260618221720.71768-1-baohua@kernel.org

Barry's RFC showed the no-swap case with MADV_PAGEOUT on 16KB mTHP: the
large-folio split counter increased by 1024 even though no swapout
progress was possible. Skipping the split in that case kept the counter
at 0.

This version keeps that behavior, but makes folio_alloc_swap() classify
the failure. The helper has both the swap allocation result and the memcg
swap charge result, so vmscan only needs to split when folio_alloc_swap()
reports that a smaller folio might still be swapped out.

Patch #1 adds page_counter_margin(), a small helper that computes the
minimum remaining chargeable space across a page_counter hierarchy.

Patch #2 uses that helper in the memcg swap path and lets
folio_alloc_swap() distinguish large-folio swap allocation failures:

  - -E2BIG: splitting may let smaller folios make progress
  - -ENOSPC: no global swap space is available
  - -ENOMEM: splitting is not expected to help, including memcg swap
    charge failures with no remaining swap capacity

Patch #3 makes vmscan split a large folio only when folio_alloc_swap()
returns -E2BIG. Other failures keep the existing activation path and avoid
destroying the large folio when no smaller part can be backed by swap
either.

RFC v1 -> RFC v2:
- Split the RFC into helper, swap allocation, and vmscan patches.
- Add page_counter_margin() and use it for hierarchical memcg swap
  capacity checks.
- Make folio_alloc_swap() return -E2BIG only when a smaller folio may
  still be swapped out.
- Return -ENOSPC for no global swap space and -ENOMEM when splitting is
  not expected to help, including memcg swap exhaustion.
- Make vmscan split large folios only on -E2BIG from folio_alloc_swap().

Barry Song (Xiaomi) (1):
  mm/vmscan: avoid pointless large folio splits without swap

Xueyuan Chen (2):
  mm: add page_counter_margin()
  mm: distinguish large folio swap allocation failures

 include/linux/page_counter.h |  1 +
 include/linux/swap.h         | 10 ++++++----
 mm/memcontrol.c              | 15 ++++++++++-----
 mm/page_counter.c            | 20 ++++++++++++++++++++
 mm/swapfile.c                | 21 +++++++++++++++------
 mm/vmscan.c                  |  7 +++++--
 6 files changed, 57 insertions(+), 17 deletions(-)

-- 
2.47.3

[RFC PATCH v2 1/3] mm: add page_counter_margin()

[Xueyuan Chen]

Add a helper to report the minimum remaining chargeable space from a
page counter up to the root of its hierarchy.

This makes the hierarchical margin calculation reusable by callers that
need to know whether a smaller charge could still fit after a larger
charge failed.

Suggested-by: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
 include/linux/page_counter.h |  1 +
 mm/page_counter.c            | 20 ++++++++++++++++++++
 2 files changed, 21 insertions(+)

diff --git a/include/linux/page_counter.h b/include/linux/page_counter.h
index d649b6bbbc87..07b7cb12249c 100644
--- a/include/linux/page_counter.h
+++ b/include/linux/page_counter.h
@@ -68,6 +68,7 @@ static inline unsigned long page_counter_read(struct page_counter *counter)
 	return atomic_long_read(&counter->usage);
 }
 
+long page_counter_margin(struct page_counter *counter);
 void page_counter_cancel(struct page_counter *counter, unsigned long nr_pages);
 void page_counter_charge(struct page_counter *counter, unsigned long nr_pages);
 bool page_counter_try_charge(struct page_counter *counter,
diff --git a/mm/page_counter.c b/mm/page_counter.c
index 661e0f2a5127..6e55c7628025 100644
--- a/mm/page_counter.c
+++ b/mm/page_counter.c
@@ -171,6 +171,26 @@ bool page_counter_try_charge(struct page_counter *counter,
 	return false;
 }
 
+/**
+ * page_counter_margin - remaining chargeable pages within hierarchical limits
+ * @counter: counter
+ *
+ * Returns the smallest remaining margin between @counter and the root.
+ */
+long page_counter_margin(struct page_counter *counter)
+{
+	long margin = PAGE_COUNTER_MAX;
+
+	for (; counter; counter = counter->parent) {
+		long limit = READ_ONCE(counter->max);
+		long usage = page_counter_read(counter);
+
+		margin = min(margin, limit - usage);
+	}
+
+	return margin;
+}
+
 /**
  * page_counter_uncharge - hierarchically uncharge pages
  * @counter: counter
-- 
2.47.3

[RFC PATCH v2 2/3] mm: distinguish large folio swap allocation failures

[Xueyuan Chen]

folio_alloc_swap() reports most allocation failures with a generic
negative error code. Reclaim cannot tell whether splitting a large folio
could make progress or whether there is no backing space at all.

Track the global free swap count around the allocation attempt and let the
memcg swap charge path cap it by the remaining hierarchical swap margin.
Return -E2BIG for large folios when a smaller allocation might still fit,
-ENOSPC when no swap space is available, and -ENOMEM when the failure is
not helped by splitting.

This only refines folio_alloc_swap() return codes. The reclaim caller is
updated separately.

Signed-off-by: Xueyuan Chen <xueyuan.chen21@gmail.com>
---
 include/linux/swap.h | 10 ++++++----
 mm/memcontrol.c      | 15 ++++++++++-----
 mm/swapfile.c        | 21 +++++++++++++++------
 3 files changed, 31 insertions(+), 15 deletions(-)

diff --git a/include/linux/swap.h b/include/linux/swap.h
index 7a09df6977a5..0695ac56457f 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -571,13 +571,14 @@ static inline void folio_throttle_swaprate(struct folio *folio, gfp_t gfp)
 #endif
 
 #if defined(CONFIG_MEMCG) && defined(CONFIG_SWAP)
-int __mem_cgroup_try_charge_swap(struct folio *folio, swp_entry_t entry);
+int __mem_cgroup_try_charge_swap(struct folio *folio, swp_entry_t entry,
+				 long *nr_swap_pages);
 static inline int mem_cgroup_try_charge_swap(struct folio *folio,
-		swp_entry_t entry)
+		swp_entry_t entry, long *nr_swap_pages)
 {
 	if (mem_cgroup_disabled())
 		return 0;
-	return __mem_cgroup_try_charge_swap(folio, entry);
+	return __mem_cgroup_try_charge_swap(folio, entry, nr_swap_pages);
 }
 
 extern void __mem_cgroup_uncharge_swap(swp_entry_t entry, unsigned int nr_pages);
@@ -592,7 +593,8 @@ extern long mem_cgroup_get_nr_swap_pages(struct mem_cgroup *memcg);
 extern bool mem_cgroup_swap_full(struct folio *folio);
 #else
 static inline int mem_cgroup_try_charge_swap(struct folio *folio,
-					     swp_entry_t entry)
+					     swp_entry_t entry,
+					     long *nr_swap_pages)
 {
 	return 0;
 }
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 177732fef010..71600d41958e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -5472,12 +5472,14 @@ int __init mem_cgroup_init(void)
  * __mem_cgroup_try_charge_swap - try charging swap space for a folio
  * @folio: folio being added to swap
  * @entry: swap entry to charge
+ * @nr_swap_pages: optional swap availability to cap by memcg margin
  *
  * Try to charge @folio's memcg for the swap space at @entry.
  *
  * Returns 0 on success, -ENOMEM on failure.
  */
-int __mem_cgroup_try_charge_swap(struct folio *folio, swp_entry_t entry)
+int __mem_cgroup_try_charge_swap(struct folio *folio, swp_entry_t entry,
+				 long *nr_swap_pages)
 {
 	unsigned int nr_pages = folio_nr_pages(folio);
 	struct page_counter *counter;
@@ -5495,6 +5497,9 @@ int __mem_cgroup_try_charge_swap(struct folio *folio, swp_entry_t entry)
 	rcu_read_lock();
 	memcg = obj_cgroup_memcg(objcg);
 	if (!entry.val) {
+		if (nr_swap_pages && !mem_cgroup_is_root(memcg))
+			*nr_swap_pages = min(*nr_swap_pages,
+					     page_counter_margin(&memcg->swap));
 		memcg_memory_event(memcg, MEMCG_SWAP_FAIL);
 		rcu_read_unlock();
 		return 0;
@@ -5509,6 +5514,9 @@ int __mem_cgroup_try_charge_swap(struct folio *folio, swp_entry_t entry)
 		memcg_memory_event(memcg, MEMCG_SWAP_MAX);
 		memcg_memory_event(memcg, MEMCG_SWAP_FAIL);
 		mem_cgroup_private_id_put(memcg, nr_pages);
+		if (nr_swap_pages)
+			*nr_swap_pages = min(*nr_swap_pages,
+					     page_counter_margin(counter));
 		return -ENOMEM;
 	}
 	mod_memcg_state(memcg, MEMCG_SWAP, nr_pages);
@@ -5550,10 +5558,7 @@ long mem_cgroup_get_nr_swap_pages(struct mem_cgroup *memcg)
 
 	if (mem_cgroup_disabled() || do_memsw_account())
 		return nr_swap_pages;
-	for (; !mem_cgroup_is_root(memcg); memcg = parent_mem_cgroup(memcg))
-		nr_swap_pages = min_t(long, nr_swap_pages,
-				      READ_ONCE(memcg->swap.max) -
-				      page_counter_read(&memcg->swap));
+	nr_swap_pages = min(nr_swap_pages, page_counter_margin(&memcg->swap));
 	return nr_swap_pages;
 }
 
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 9174f1eeffb0..53a921ca099a 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -1690,12 +1690,14 @@ static int swap_dup_entries_cluster(struct swap_info_struct *si,
  * swap cache.
  *
  * Context: Caller needs to hold the folio lock.
- * Return: Whether the folio was added to the swap cache.
+ * Return: 0 on success, -E2BIG if splitting the folio might allow swapout,
+ * or another negative error code if splitting would not help.
  */
 int folio_alloc_swap(struct folio *folio)
 {
 	unsigned int order = folio_order(folio);
 	unsigned int size = 1 << order;
+	long nr_swap_pages;
 
 	VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
 	VM_BUG_ON_FOLIO(!folio_test_uptodate(folio), folio);
@@ -1706,7 +1708,7 @@ int folio_alloc_swap(struct folio *folio)
 		 * the caller should split the folio and try again.
 		 */
 		if (!IS_ENABLED(CONFIG_THP_SWAP))
-			return -EAGAIN;
+			return -E2BIG;
 
 		/*
 		 * Allocation size should never exceed cluster size
@@ -1714,10 +1716,12 @@ int folio_alloc_swap(struct folio *folio)
 		 */
 		if (size > SWAPFILE_CLUSTER) {
 			VM_WARN_ON_ONCE(1);
-			return -EINVAL;
+			return -E2BIG;
 		}
 	}
 
+	nr_swap_pages = get_nr_swap_pages();
+
 again:
 	local_lock(&percpu_swap_cluster.lock);
 	if (!swap_alloc_fast(folio))
@@ -1730,11 +1734,16 @@ int folio_alloc_swap(struct folio *folio)
 	}
 
 	/* Need to call this even if allocation failed, for MEMCG_SWAP_FAIL. */
-	if (unlikely(mem_cgroup_try_charge_swap(folio, folio->swap)))
+	if (unlikely(mem_cgroup_try_charge_swap(folio, folio->swap,
+						&nr_swap_pages))) {
 		swap_cache_del_folio(folio);
+		return order && nr_swap_pages > 0 ? -E2BIG : -ENOMEM;
+	}
 
-	if (unlikely(!folio_test_swapcache(folio)))
-		return -ENOMEM;
+	if (unlikely(!folio_test_swapcache(folio))) {
+		nr_swap_pages = get_nr_swap_pages();
+		return order && nr_swap_pages > 0 ? -E2BIG : -ENOSPC;
+	}
 
 	return 0;
 }
-- 
2.47.3

[RFC PATCH v2 3/3] mm/vmscan: avoid pointless large folio splits without swap

[Xueyuan Chen]

From: "Barry Song (Xiaomi)" <baohua@kernel.org>

When swap is disabled, exhausted, or unavailable due to memcg swap
limits, splitting a large anonymous folio cannot make swapout progress.
The fallback only destroys the large folio and inflates split statistics.

Use -E2BIG from folio_alloc_swap() as the explicit signal that splitting
the folio might allow swapout of smaller pieces. For other allocation
failures, keep the existing activation path and avoid the split.

This preserves the split fallback for fragmented or partially available
swap, while avoiding it when there is no backing space for any part of the
folio.

Reported-by: Nanzhe Zhao <zhaonanzhe@xiaomi.com>
Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
---
 mm/vmscan.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index bd1b1aa12581..40340a88f78e 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1260,6 +1260,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 		 */
 		if (folio_test_anon(folio) && folio_test_swapbacked(folio) &&
 				!folio_test_swapcache(folio)) {
+			int ret;
+
 			if (!(sc->gfp_mask & __GFP_IO))
 				goto keep_locked;
 			if (folio_maybe_dma_pinned(folio))
@@ -1278,10 +1280,11 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 				    split_folio_to_list(folio, folio_list))
 					goto activate_locked;
 			}
-			if (folio_alloc_swap(folio)) {
+			ret = folio_alloc_swap(folio);
+			if (ret) {
 				int __maybe_unused order = folio_order(folio);
 
-				if (!folio_test_large(folio))
+				if (!folio_test_large(folio) || ret != -E2BIG)
 					goto activate_locked_split;
 				/* Fallback to swap normal pages */
 				if (split_folio_to_list(folio, folio_list))
-- 
2.47.3

Re: [RFC PATCH v2 2/3] mm: distinguish large folio swap allocation failures

[Johannes Weiner]

On Thu, Jul 09, 2026 at 10:51:23PM +0800, Xueyuan Chen wrote:
> @@ -5550,10 +5558,7 @@ long mem_cgroup_get_nr_swap_pages(struct mem_cgroup *memcg)
>  
>  	if (mem_cgroup_disabled() || do_memsw_account())
>  		return nr_swap_pages;
> -	for (; !mem_cgroup_is_root(memcg); memcg = parent_mem_cgroup(memcg))
> -		nr_swap_pages = min_t(long, nr_swap_pages,
> -				      READ_ONCE(memcg->swap.max) -
> -				      page_counter_read(&memcg->swap));
> +	nr_swap_pages = min(nr_swap_pages, page_counter_margin(&memcg->swap));

This hunk is unrelated to this patch. Don't mix refactor work with new
functionality. Make the previous patch a pure refactor job (where you
add page_counter_margin() and use it here ^), like I had proposed.

Re: [RFC PATCH v2 2/3] mm: distinguish large folio swap allocation failures

[Johannes Weiner]

On Thu, Jul 09, 2026 at 11:01:12AM -0400, Johannes Weiner wrote:
> On Thu, Jul 09, 2026 at 10:51:23PM +0800, Xueyuan Chen wrote:
> > @@ -5550,10 +5558,7 @@ long mem_cgroup_get_nr_swap_pages(struct mem_cgroup *memcg)
> >  
> >  	if (mem_cgroup_disabled() || do_memsw_account())
> >  		return nr_swap_pages;
> > -	for (; !mem_cgroup_is_root(memcg); memcg = parent_mem_cgroup(memcg))
> > -		nr_swap_pages = min_t(long, nr_swap_pages,
> > -				      READ_ONCE(memcg->swap.max) -
> > -				      page_counter_read(&memcg->swap));
> > +	nr_swap_pages = min(nr_swap_pages, page_counter_margin(&memcg->swap));
> 
> This hunk is unrelated to this patch. Don't mix refactor work with new
> functionality. Make the previous patch a pure refactor job (where you
> add page_counter_margin() and use it here ^), like I had proposed.

I also liked my version of mem_cgroup_get_nr_swap_pages() a bit
better. Please just use my patch, keep the From: and you can add

Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>

Re: [RFC PATCH v2 2/3] mm: distinguish large folio swap allocation failures

[Xueyuan Chen]

On Thu, Jul 09, 2026 at 11:04:23AM -0400, Johannes Weiner wrote:

Hi Johannes,

>On Thu, Jul 09, 2026 at 11:01:12AM -0400, Johannes Weiner wrote:
>> On Thu, Jul 09, 2026 at 10:51:23PM +0800, Xueyuan Chen wrote:
>> > @@ -5550,10 +5558,7 @@ long mem_cgroup_get_nr_swap_pages(struct mem_cgroup *memcg)
>> >  
>> >  	if (mem_cgroup_disabled() || do_memsw_account())
>> >  		return nr_swap_pages;
>> > -	for (; !mem_cgroup_is_root(memcg); memcg = parent_mem_cgroup(memcg))
>> > -		nr_swap_pages = min_t(long, nr_swap_pages,
>> > -				      READ_ONCE(memcg->swap.max) -
>> > -				      page_counter_read(&memcg->swap));
>> > +	nr_swap_pages = min(nr_swap_pages, page_counter_margin(&memcg->swap));
>> 
>> This hunk is unrelated to this patch. Don't mix refactor work with new
>> functionality. Make the previous patch a pure refactor job (where you
>> add page_counter_margin() and use it here ^), like I had proposed.

You're right, that refactor doesn't belong in patch 2.

>I also liked my version of mem_cgroup_get_nr_swap_pages() a bit
>better. Please just use my patch, keep the From: and you can add
>
>Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
>

I'll use your version in v3, keep your From: and Signed-off-by, and make
patch 1 a pure refactor patch that adds page_counter_margin() and
converts mem_cgroup_get_nr_swap_pages() to use it.

Patch 2 will only keep the functional folio_alloc_swap()/memcg charge
changes.

Thanks,
Xueyuan

Re: [RFC PATCH v2 3/3] mm/vmscan: avoid pointless large folio splits without swap

[Baolin Wang]

On 7/9/26 10:51 PM, Xueyuan Chen wrote:
> From: "Barry Song (Xiaomi)" <baohua@kernel.org>
> 
> When swap is disabled, exhausted, or unavailable due to memcg swap
> limits, splitting a large anonymous folio cannot make swapout progress.
> The fallback only destroys the large folio and inflates split statistics.
> 
> Use -E2BIG from folio_alloc_swap() as the explicit signal that splitting
> the folio might allow swapout of smaller pieces. For other allocation
> failures, keep the existing activation path and avoid the split.
> 
> This preserves the split fallback for fragmented or partially available
> swap, while avoiding it when there is no backing space for any part of the
> folio.
> 
> Reported-by: Nanzhe Zhao <zhaonanzhe@xiaomi.com>
> Signed-off-by: Barry Song (Xiaomi) <baohua@kernel.org>
> ---
>   mm/vmscan.c | 7 +++++--
>   1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index bd1b1aa12581..40340a88f78e 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -1260,6 +1260,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
>   		 */
>   		if (folio_test_anon(folio) && folio_test_swapbacked(folio) &&
>   				!folio_test_swapcache(folio)) {
> +			int ret;
> +
>   			if (!(sc->gfp_mask & __GFP_IO))
>   				goto keep_locked;
>   			if (folio_maybe_dma_pinned(folio))
> @@ -1278,10 +1280,11 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
>   				    split_folio_to_list(folio, folio_list))
>   					goto activate_locked;
>   			}
> -			if (folio_alloc_swap(folio)) {
> +			ret = folio_alloc_swap(folio);
> +			if (ret) {
>   				int __maybe_unused order = folio_order(folio);
>   
> -				if (!folio_test_large(folio))
> +				if (!folio_test_large(folio) || ret != -E2BIG)
>   					goto activate_locked_split;

Like I said in v1 [1], please apply the same change to shmem swap as well.

[1] 
https://lore.kernel.org/all/6e89f868-ca7a-484f-aeea-5d8d029714f2@linux.alibaba.com/