[PATCH mm-next] mm/swap: free swap cache folios the zap path gave up on

[PATCH mm-next] mm/swap: free swap cache folios the zap path gave up on

[Jiayuan Chen]

From: Jiayuan Chen <jiayuan.chen@shopee.com>

On our production hosts we saw swap cache only pages keep growing for days:
on a box with a 500 GiB swap partition, 250 GiB of RAM and 250 GiB of
swap slots were held by folios that no process mapped any more, charged
to cgroups whose workloads had long exited. The only way to get them
back was swapoff, and with other services actively swapping on the
same host that is not an option.

Such a slot has swap count 0 and an unmapped folio pinned in the swap
cache, left there after a process exited or did MADV_DONTNEED. No page
table holds its swap entry any more, so nothing comes back for the
slot; LRU reclaim drops it eventually, and the allocator only reclaims
such slots once swap is half full. The growth stopped exactly there:
at 50% of the device, where vm_swap_full() becomes true.

The race is readahead against a zap in a neighbouring thread. Readahead
works on virtual addresses, so a fault at the end of thread A's range
pulls in the entries at the start of thread B's range:

  thread A                               thread B
  do_swap_page()
    swap_vma_readahead()
      swap_cache_read_folio(B's entry)
        folio locked, added to cache
        swap_read_folio()                MADV_DONTNEED
          zswap_load() / bio in flight     zap_nonpresent_ptes()
          .                                  swap_put_entries_cluster()
          .                                    swap count 1 -> 0
          .                                    __try_to_reclaim_swap()
          .                                      folio_trylock() fails
          .                                      gives up
          folio_unlock()
      folio_put()
  -> slot: swap count 0, folio in swap cache, not mapped, never freed

The lock is one reason it gives up. The other is writeback: with a block
device as backing store, B's own PAGEOUT leaves the folio in the swap
cache under writeback, and when the DONTNEED comes __try_to_reclaim_swap()
gets the lock but folio_swapcache_freeable() refuses. The write
completes, nobody frees the slot. The present-PTE side has the same two
outcomes through free_swap_cache().

Options considered:

1. Fix it on the readahead side: after the read, check the swap count
   and free the slot. Covers the lock, not writeback, and the
   asynchronous read path unlocks in the bio completion where the slot
   cannot be freed. Two places to patch, neither clean.
2. swapoff/swapon clears it, as does memory.reclaim on the right cgroup.
   That is a workaround, not a fix.
3. Have the side that gives up come back later. This patch.

The zap runs under the PTL and the fault path takes the folio lock
before the PTL, so waiting there is out; a worker has to do it. On a
lost trylock or a writeback refusal under TTRS_UNMAPPED,
__try_to_reclaim_swap() schedules si->sweep_work. The worker walks the
in-use clusters and, for every slot with swap count 0 whose folio is
unmapped, calls __try_to_reclaim_swap() with TTRS_WAIT, which takes the
folio lock and waits for writeback instead of giving up.
free_swap_cache() now calls __try_to_reclaim_swap() as well, so both
paths give up, and wake the worker, in the same place.

For now the worker scans every in-use cluster of the device; marking
the clusters that need it can reduce that later.

Cc: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Jiayuan Chen <jiayuan.chen@shopee.com>
---
 include/linux/swap.h |   1 +
 mm/swap.h            |   1 +
 mm/swap_state.c      |   9 ++--
 mm/swapfile.c        | 101 ++++++++++++++++++++++++++++++++++++++++++-
 4 files changed, 104 insertions(+), 8 deletions(-)

diff --git a/include/linux/swap.h b/include/linux/swap.h
index 4f686709d71f..5777f10e9aa4 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -267,6 +267,7 @@ struct swap_info_struct {
 					 */
 	struct work_struct discard_work; /* discard worker */
 	struct work_struct reclaim_work; /* reclaim worker */
+	struct work_struct sweep_work;	/* frees swap cache only slots a zap gave up on */
 	struct list_head discard_clusters; /* discard clusters list */
 	struct plist_node avail_list;   /* entry in swap_avail_head */
 	const struct swap_ops *ops;
diff --git a/mm/swap.h b/mm/swap.h
index d5bf21f517dc..f62f36c0ce5c 100644
--- a/mm/swap.h
+++ b/mm/swap.h
@@ -309,6 +309,7 @@ static inline bool folio_matches_swap_entry(const struct folio *folio,
  * - See the comment of get_swap_device() for more complex usage.
  */
 bool swap_cache_has_folio(swp_entry_t entry);
+void swap_reclaim_entry(swp_entry_t entry);
 struct folio *swap_cache_get_folio(swp_entry_t entry);
 void *swap_cache_get_shadow(swp_entry_t entry);
 void swap_cache_del_folio(struct folio *folio);
diff --git a/mm/swap_state.c b/mm/swap_state.c
index ebef568cd62e..99708f149b25 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -598,16 +598,13 @@ void swap_writeback_dropbehind_folio(struct folio *folio)
  *
  * Its ok to check the swapcache flag without the folio lock
  * here because we are going to recheck again inside
- * folio_free_swap() _with_ the lock.
+ * __try_to_reclaim_swap() _with_ the lock.
  * 					- Marcelo
  */
 void free_swap_cache(struct folio *folio)
 {
-	if (folio_test_swapcache(folio) && !folio_mapped(folio) &&
-	    folio_trylock(folio)) {
-		folio_free_swap(folio);
-		folio_unlock(folio);
-	}
+	if (folio_test_swapcache(folio) && !folio_mapped(folio))
+		swap_reclaim_entry(folio->swap);
 }
 
 /*
diff --git a/mm/swapfile.c b/mm/swapfile.c
index c3288910b3e3..ee29085152fe 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -172,6 +172,8 @@ static long swap_usage_in_pages(struct swap_info_struct *si)
 #define TTRS_UNMAPPED		0x2
 /* Reclaim the swap entry if swap is getting full */
 #define TTRS_FULL		0x4
+/* May sleep: wait for the folio lock and for writeback instead of giving up */
+#define TTRS_WAIT		0x8
 
 static bool swap_only_has_cache(struct swap_cluster_info *ci,
 				unsigned long offset, int nr_pages)
@@ -217,9 +219,19 @@ static int __try_to_reclaim_swap(struct swap_info_struct *si,
 	 * avoiding deadlock. This is a special case and you should
 	 * use folio_free_swap() with explicit folio_lock() in usual
 	 * operations.
+	 *
+	 * The zap path (TTRS_UNMAPPED) is the one caller for which giving
+	 * up means the slot is never looked at again: it stays allocated
+	 * with count 0 and an unmapped folio pinned in the swap cache. Let
+	 * the sweep worker, which can wait, come back for it.
 	 */
-	if (!folio_trylock(folio))
+	if (flags & TTRS_WAIT) {
+		folio_lock(folio);
+	} else if (!folio_trylock(folio)) {
+		if (flags & TTRS_UNMAPPED)
+			schedule_work(&si->sweep_work);
 		goto out;
+	}
 
 	/*
 	 * Offset could point to the middle of a large folio, or folio
@@ -232,11 +244,20 @@ static int __try_to_reclaim_swap(struct swap_info_struct *si,
 	}
 	offset = swp_offset(folio->swap);
 
+	if (flags & TTRS_WAIT)
+		folio_wait_writeback(folio);
+
 	need_reclaim = ((flags & TTRS_ANYWAY) ||
 			((flags & TTRS_UNMAPPED) && !folio_mapped(folio)) ||
 			((flags & TTRS_FULL) && mem_cgroup_swap_full(folio)));
-	if (!need_reclaim || !folio_swapcache_freeable(folio))
+	if (!need_reclaim)
+		goto out_unlock;
+	if (!folio_swapcache_freeable(folio)) {
+		/* Same for a folio the zap caught under writeback. */
+		if ((flags & TTRS_UNMAPPED) && folio_test_writeback(folio))
+			schedule_work(&si->sweep_work);
 		goto out_unlock;
+	}
 
 	/*
 	 * It's safe to delete the folio from swap cache only if the folio
@@ -1099,6 +1120,80 @@ static void swap_reclaim_work(struct work_struct *work)
 	swap_reclaim_full_clusters(si, true);
 }
 
+/*
+ * Free the slots a zap could not take back: swap cache only, count 0, folio
+ * not mapped. The zap runs under the PTL, so when the folio was locked (a
+ * readahead or an asynchronous swap-in still reading into it) or under
+ * writeback, __try_to_reclaim_swap() had to give up and scheduled us. Such
+ * a slot is reachable by nobody but LRU reclaim: no page table holds its
+ * entry, and the allocator skips it until swap is half full. Here we can
+ * wait for the lock and the writeback.
+ */
+static void swap_sweep_work(struct work_struct *work)
+{
+	struct swap_info_struct *si;
+	struct swap_cluster_info *ci;
+	unsigned long offset, end, swp_tb;
+	unsigned long nr_clusters;
+	int nr_reclaim;
+
+	si = container_of(work, struct swap_info_struct, sweep_work);
+	nr_clusters = DIV_ROUND_UP(si->max, SWAPFILE_CLUSTER);
+
+	for (unsigned long i = 0; i < nr_clusters; i++) {
+		ci = si->cluster_info + i;
+		offset = i * SWAPFILE_CLUSTER;
+		end = min(si->max, offset + SWAPFILE_CLUSTER);
+
+		spin_lock(&ci->lock);
+		if (!ci->count) {
+			spin_unlock(&ci->lock);
+			continue;
+		}
+		while (offset < end) {
+			swp_tb = swap_table_get(ci, offset % SWAPFILE_CLUSTER);
+			/*
+			 * A mapped folio with count 0 is a page swapped in and
+			 * kept in the swap cache: useful, and not ours. The
+			 * folio cannot leave the cache while we hold ci->lock,
+			 * so peeking at it is safe; the real check is done
+			 * under the folio lock in __try_to_reclaim_swap().
+			 */
+			if (swp_tb_is_folio(swp_tb) && !__swp_tb_get_count(swp_tb) &&
+			    !folio_mapped(swp_tb_to_folio(swp_tb))) {
+				spin_unlock(&ci->lock);
+				nr_reclaim = __try_to_reclaim_swap(si, offset,
+								   TTRS_UNMAPPED | TTRS_WAIT);
+				spin_lock(&ci->lock);
+				if (nr_reclaim) {
+					offset += abs(nr_reclaim);
+					continue;
+				}
+			}
+			offset++;
+		}
+		spin_unlock(&ci->lock);
+		cond_resched();
+	}
+}
+
+/*
+ * The present-PTE zap path drops a folio that was mapped from the swap
+ * cache. Same rules as the swap-entry zap: free the slot if the folio is
+ * unmapped and unused, let the worker handle it if that is not possible
+ * right now. @entry may already be stale, get_swap_device() sorts that out.
+ */
+void swap_reclaim_entry(swp_entry_t entry)
+{
+	struct swap_info_struct *si;
+
+	si = get_swap_device(entry);
+	if (IS_ERR_OR_NULL(si))
+		return;
+	__try_to_reclaim_swap(si, swp_offset(entry), TTRS_UNMAPPED);
+	put_swap_device(si);
+}
+
 /*
  * Try to allocate swap entries with specified order and try set a new
  * cluster for current CPU too.
@@ -3264,6 +3359,7 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
 
 	flush_work(&p->discard_work);
 	flush_work(&p->reclaim_work);
+	flush_work(&p->sweep_work);
 	flush_percpu_swap_cluster(p);
 
 	destroy_swap_extents(p, p->swap_file);
@@ -3744,6 +3840,7 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags)
 
 	INIT_WORK(&si->discard_work, swap_discard_work);
 	INIT_WORK(&si->reclaim_work, swap_reclaim_work);
+	INIT_WORK(&si->sweep_work, swap_sweep_work);
 
 	CLASS(filename, name)(specialfile);
 	swap_file = file_open_name(name, O_RDWR | O_LARGEFILE | O_EXCL, 0);
-- 
2.43.0

Re: [PATCH mm-next] mm/swap: free swap cache folios the zap path gave up on

[Andrew Morton]

On Wed, 14 Oct 2026 17:08:19 +0800 Jiayuan Chen <jiayuan.chen@linux.dev> wrote:

> From: Jiayuan Chen <jiayuan.chen@shopee.com>
> 
> On our production hosts we saw swap cache only pages keep growing for days:
> on a box with a 500 GiB swap partition, 250 GiB of RAM and 250 GiB of
> swap slots were held by folios that no process mapped any more, charged
> to cgroups whose workloads had long exited. The only way to get them
> back was swapoff, and with other services actively swapping on the
> same host that is not an option.
> 
> Such a slot has swap count 0 and an unmapped folio pinned in the swap
> cache, left there after a process exited or did MADV_DONTNEED. No page
> table holds its swap entry any more, so nothing comes back for the
> slot; LRU reclaim drops it eventually, and the allocator only reclaims
> such slots once swap is half full. The growth stopped exactly there:
> at 50% of the device, where vm_swap_full() becomes true.

That's a pretty nasty problem - thanks for working on it.

> Options considered:
> 
> 1. Fix it on the readahead side: after the read, check the swap count
>    and free the slot. Covers the lock, not writeback, and the
>    asynchronous read path unlocks in the bio completion where the slot
>    cannot be freed. Two places to patch, neither clean.
> 2. swapoff/swapon clears it, as does memory.reclaim on the right cgroup.
>    That is a workaround, not a fix.
> 3. Have the side that gives up come back later. This patch.
> 
> The zap runs under the PTL and the fault path takes the folio lock
> before the PTL, so waiting there is out; a worker has to do it. On a
> lost trylock or a writeback refusal under TTRS_UNMAPPED,
> __try_to_reclaim_swap() schedules si->sweep_work. The worker walks the
> in-use clusters and, for every slot with swap count 0 whose folio is
> unmapped, calls __try_to_reclaim_swap() with TTRS_WAIT, which takes the
> folio lock and waits for writeback instead of giving up.
> free_swap_cache() now calls __try_to_reclaim_swap() as well, so both
> paths give up, and wake the worker, in the same place.
> 
> For now the worker scans every in-use cluster of the device; marking
> the clusters that need it can reduce that later.

It's a substantial patch.  There's no quick-n-dirty fixup we can
prepare for backporting into older kernels?

>  include/linux/swap.h |   1 +
>  mm/swap.h            |   1 +
>  mm/swap_state.c      |   9 ++--
>  mm/swapfile.c        | 101 ++++++++++++++++++++++++++++++++++++++++++-
>  4 files changed, 104 insertions(+), 8 deletions(-)

Sashiko had a few things to say:
	https://sashiko.dev/#/patchset/20261014090821.227366-1-jiayuan.chen@linux.dev
	

Re: [PATCH mm-next] mm/swap: free swap cache folios the zap path gave up on

[Jiayuan Chen]

On 10/15/26 7:02 AM, Andrew Morton wrote:
> On Wed, 14 Oct 2026 17:08:19 +0800 Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
>
>> From: Jiayuan Chen <jiayuan.chen@shopee.com>
>>
>> On our production hosts we saw swap cache only pages keep growing for days:
>> on a box with a 500 GiB swap partition, 250 GiB of RAM and 250 GiB of
>> swap slots were held by folios that no process mapped any more, charged
>> to cgroups whose workloads had long exited. The only way to get them
>> back was swapoff, and with other services actively swapping on the
>> same host that is not an option.
>>
>> Such a slot has swap count 0 and an unmapped folio pinned in the swap
>> cache, left there after a process exited or did MADV_DONTNEED. No page
>> table holds its swap entry any more, so nothing comes back for the
>> slot; LRU reclaim drops it eventually, and the allocator only reclaims
>> such slots once swap is half full. The growth stopped exactly there:
>> at 50% of the device, where vm_swap_full() becomes true.
> That's a pretty nasty problem - thanks for working on it.
>
>> Options considered:
>>
>> 1. Fix it on the readahead side: after the read, check the swap count
>>     and free the slot. Covers the lock, not writeback, and the
>>     asynchronous read path unlocks in the bio completion where the slot
>>     cannot be freed. Two places to patch, neither clean.
>> 2. swapoff/swapon clears it, as does memory.reclaim on the right cgroup.
>>     That is a workaround, not a fix.
>> 3. Have the side that gives up come back later. This patch.
>>
>> The zap runs under the PTL and the fault path takes the folio lock
>> before the PTL, so waiting there is out; a worker has to do it. On a
>> lost trylock or a writeback refusal under TTRS_UNMAPPED,
>> __try_to_reclaim_swap() schedules si->sweep_work. The worker walks the
>> in-use clusters and, for every slot with swap count 0 whose folio is
>> unmapped, calls __try_to_reclaim_swap() with TTRS_WAIT, which takes the
>> folio lock and waits for writeback instead of giving up.
>> free_swap_cache() now calls __try_to_reclaim_swap() as well, so both
>> paths give up, and wake the worker, in the same place.
>>
>> For now the worker scans every in-use cluster of the device; marking
>> the clusters that need it can reduce that later.
> It's a substantial patch.  There's no quick-n-dirty fixup we can
> prepare for backporting into older kernels?


I couldn't find a small one. I tried the other options listed in the
changelog, and each was either incomplete or spread over many places.


The zap can only trylock, and the folio can be held by many things at
that moment: readahead (sync and async reads), writeback, zswap
writeback, reclaim, migration, folio split and so on. Fixing it on that
side means patching each of them, and missing one leaves the leak. This
patch fixes it in the one place they all have in common, the zap that
gives up.

As for older kernels, I found this on 6.6 and I'm happy to do the
backports.

>>   include/linux/swap.h |   1 +
>>   mm/swap.h            |   1 +
>>   mm/swap_state.c      |   9 ++--
>>   mm/swapfile.c        | 101 ++++++++++++++++++++++++++++++++++++++++++-
>>   4 files changed, 104 insertions(+), 8 deletions(-)
> Sashiko had a few things to say:
> 	https://sashiko.dev/#/patchset/20261014090821.227366-1-jiayuan.chen@linux.dev
> 	


Thanks. The large folio split and the offset step are real and come

from TTRS_WAIT.


The swap_cache_get_folio() race is pre-existing, I could reproduce it 
and will send a separate fix.

Re: [PATCH mm-next] mm/swap: free swap cache folios the zap path gave up on

[Nhat Pham]

On Wed, Oct 14, 2026 at 2:08 AM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
>
> From: Jiayuan Chen <jiayuan.chen@shopee.com>
>
> On our production hosts we saw swap cache only pages keep growing for days:
> on a box with a 500 GiB swap partition, 250 GiB of RAM and 250 GiB of
> swap slots were held by folios that no process mapped any more, charged

That's a really nasty situation.

> to cgroups whose workloads had long exited. The only way to get them
> back was swapoff, and with other services actively swapping on the
> same host that is not an option.

Is it not similar to cold file caches?

The recommendation for these kind of scenarios (for both file and
anon) is to adopt a proactive reclaimer (in the same vein as senpai).
That's what we do internally.

>
> Cc: Jiayuan Chen <jiayuan.chen@linux.dev>
> Signed-off-by: Jiayuan Chen <jiayuan.chen@shopee.com>

Code looks solid to me otherwise. Maybe include some numbers from your
testing in the cover letter to verify that the swap cache usage drop.

Re: [PATCH mm-next] mm/swap: free swap cache folios the zap path gave up on

[Jiayuan Chen]

Hi Nhat


On 10/16/26 1:11 AM, Nhat Pham wrote:
> On Wed, Oct 14, 2026 at 2:08 AM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
>> From: Jiayuan Chen <jiayuan.chen@shopee.com>
>>
>> On our production hosts we saw swap cache only pages keep growing for days:
>> on a box with a 500 GiB swap partition, 250 GiB of RAM and 250 GiB of
>> swap slots were held by folios that no process mapped any more, charged
> That's a really nasty situation.
>
>> to cgroups whose workloads had long exited. The only way to get them
>> back was swapoff, and with other services actively swapping on the
>> same host that is not an option.
> Is it not similar to cold file caches?


Not quite. A cold file cache can still be hit later. These folios
can't: no page table holds the swap entry any more, so nothing will
ever look them up again. They also hold swap slots, not just RAM, and
the allocator skips those slots until swap is half full.


> The recommendation for these kind of scenarios (for both file and
> anon) is to adopt a proactive reclaimer (in the same vein as senpai).
> That's what we do internally.


Proactive reclaim does free them, but it works around the leak rather
than fixing it. In our case they were charged to cgroups that had
already been removed, so there was no cgroup left to reclaim from, and
reclaiming from the parent pushed out live anon first



>> Cc: Jiayuan Chen <jiayuan.chen@linux.dev>
>> Signed-off-by: Jiayuan Chen <jiayuan.chen@shopee.com>
> Code looks solid to me otherwise. Maybe include some numbers from your
> testing in the cover letter to verify that the swap cache usage drop.


Thanks, will do in next version.

Re: [PATCH mm-next] mm/swap: free swap cache folios the zap path gave up on

[Kairui Song]

On Wed, Oct 14, 2026 at 5:10 PM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
>
> From: Jiayuan Chen <jiayuan.chen@shopee.com>
>
> On our production hosts we saw swap cache only pages keep growing for days:
> on a box with a 500 GiB swap partition, 250 GiB of RAM and 250 GiB of
> swap slots were held by folios that no process mapped any more, charged
> to cgroups whose workloads had long exited. The only way to get them
> back was swapoff, and with other services actively swapping on the
> same host that is not an option.

Hi, thanks for the report.

> Such a slot has swap count 0 and an unmapped folio pinned in the swap
> cache, left there after a process exited or did MADV_DONTNEED. No page
> table holds its swap entry any more, so nothing comes back for the
> slot; LRU reclaim drops it eventually, and the allocator only reclaims
> such slots once swap is half full. The growth stopped exactly there:
> at 50% of the device, where vm_swap_full() becomes true.

Hmm, I think this indicates a much cleaner fix is available.
vm_swap_full is inherited from the old allocator, we can instead just
let the allocator reclaim the slot if the slot have a count of 0 and
ignore vm_swap_full at that point.

Will this work? (Just a minimal implementation for reference, will be
even better if we can avoid the overhead).

diff --git a/mm/swapfile.c b/mm/swapfile.c
index c3288910b3e3..94670fe82f31 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -212,6 +212,9 @@ static int __try_to_reclaim_swap(struct
swap_info_struct *si,
        nr_pages = folio_nr_pages(folio);
        ret = -nr_pages;

+       if (folio_mapped(folio) && !vm_swap_full())
+               goto out;
+
        /*
         * We hold a folio lock here. We have to use trylock for
         * avoiding deadlock. This is a special case and you should
@@ -903,8 +906,6 @@ static bool cluster_scan_range(struct swap_info_struct *si,
                if (swp_tb_is_null(swp_tb))
                        continue;
                if (swp_tb_is_folio(swp_tb) && !__swp_tb_get_count(swp_tb)) {
-                       if (!vm_swap_full())
-                               return false;
                        *need_reclaim = true;
                        continue;
                }


Or, from the LRU's side, put swap readahead folios at the tail. I'm
actually trying to do that and it's not very easily doable at the
moment but will soon be easy to do.

> Options considered:
>
> 1. Fix it on the readahead side: after the read, check the swap count
>    and free the slot. Covers the lock, not writeback, and the
>    asynchronous read path unlocks in the bio completion where the slot
>    cannot be freed. Two places to patch, neither clean.

In fact, readahead checks the count before starts the read. It should
prevent some of the problems but seems not enough.

And another idea, what if we just add dropbehind flag or rotate the
folio, if folio_try_lock failed in __try_to_reclaim_swap?

Re: [PATCH mm-next] mm/swap: free swap cache folios the zap path gave up on

[Jiayuan Chen]

On 10/16/26 11:05 AM, Kairui Song wrote:
> On Wed, Oct 14, 2026 at 5:10 PM Jiayuan Chen <jiayuan.chen@linux.dev> wrote:
>> From: Jiayuan Chen <jiayuan.chen@shopee.com>
>>
>> On our production hosts we saw swap cache only pages keep growing for days:
>> on a box with a 500 GiB swap partition, 250 GiB of RAM and 250 GiB of
>> swap slots were held by folios that no process mapped any more, charged
>> to cgroups whose workloads had long exited. The only way to get them
>> back was swapoff, and with other services actively swapping on the
>> same host that is not an option.
> Hi, thanks for the report.
>
>> Such a slot has swap count 0 and an unmapped folio pinned in the swap
>> cache, left there after a process exited or did MADV_DONTNEED. No page
>> table holds its swap entry any more, so nothing comes back for the
>> slot; LRU reclaim drops it eventually, and the allocator only reclaims
>> such slots once swap is half full. The growth stopped exactly there:
>> at 50% of the device, where vm_swap_full() becomes true.
> Hmm, I think this indicates a much cleaner fix is available.
> vm_swap_full is inherited from the old allocator, we can instead just
> let the allocator reclaim the slot if the slot have a count of 0 and
> ignore vm_swap_full at that point.
>
> Will this work? (Just a minimal implementation for reference, will be
> even better if we can avoid the overhead).
>
> diff --git a/mm/swapfile.c b/mm/swapfile.c
> index c3288910b3e3..94670fe82f31 100644
> --- a/mm/swapfile.c
> +++ b/mm/swapfile.c
> @@ -212,6 +212,9 @@ static int __try_to_reclaim_swap(struct
> swap_info_struct *si,
>          nr_pages = folio_nr_pages(folio);
>          ret = -nr_pages;
>
> +       if (folio_mapped(folio) && !vm_swap_full())
> +               goto out;
> +
>          /*
>           * We hold a folio lock here. We have to use trylock for
>           * avoiding deadlock. This is a special case and you should
> @@ -903,8 +906,6 @@ static bool cluster_scan_range(struct swap_info_struct *si,
>                  if (swp_tb_is_null(swp_tb))
>                          continue;
>                  if (swp_tb_is_folio(swp_tb) && !__swp_tb_get_count(swp_tb)) {
> -                       if (!vm_swap_full())
> -                               return false;
>                          *need_reclaim = true;
>                          continue;
>                  }
>
Hi Kairui,

I tried it on our box (224 CPUs, 931 GiB swap partition). It doesn't
help there: after 100 rounds of the reproducer the leftover is 291 GB
with it and 309 GB without, and both stop growing at the same place,
488 GB, half of the device:

# cat /proc/swaps
Filename                Type        Size        Used Priority
/dev/nvme1n1p1                          partition    976561148 
489006260    -1

# cat /proc/meminfo | grep Swap
SwapCached:     489006260 kB
SwapTotal:      976561148 kB
SwapFree:       487554888 kB

cluster_alloc_swap_entry() picks clusters from nonfull_clusters first,
then free_clusters; it looks at frag_clusters only when both are empty,
and at full_clusters only when vm_swap_full(). A cluster gets onto
nonfull_clusters in partial_free_cluster(), that is when
__swap_cluster_free_entries() frees a slot in it; after
alloc_swap_scan_cluster() has used it, relocate_cluster() moves it to
frag or full.

A leftover slot has count 0 and a folio, so __swap_cluster_free_entries()
never runs for it. Once the live slots of a cluster are gone, nothing
calls partial_free_cluster() on it again, it stays on frag or full, and
cluster_alloc_swap_entry() doesn't pick it until the device is half
full. That is what the swap table looked like after the run:

list      clusters   leftover slots
FULL        106415       54484480   (all 512 slots leftover)
FRAG         44822       18471614
NONFULL          1              0

Making cluster_alloc_swap_entry() revisit those clusters means changing
the list policy.



> Or, from the LRU's side, put swap readahead folios at the tail. I'm
> actually trying to do that and it's not very easily doable at the
> moment but will soon be easy to do.


That helps the RAM side under pressure but doesn't give the slots
back.


>> Options considered:
>>
>> 1. Fix it on the readahead side: after the read, check the swap count
>>     and free the slot. Covers the lock, not writeback, and the
>>     asynchronous read path unlocks in the bio completion where the slot
>>     cannot be freed. Two places to patch, neither clean.
> In fact, readahead checks the count before starts the read. It should
> prevent some of the problems but seems not enough.


Right, the count is still 1 at that point. The zap drops it to 0 while
the read is in flight, after the folio is already in the swap cache.


> And another idea, what if we just add dropbehind flag or rotate the
> folio, if folio_try_lock failed in __try_to_reclaim_swap?


dropbehind would work for readahead and writeback if the read
completion honoured it too; today only folio_end_writeback() does, for
zswap writeback folios, and swap_writeback_dropbehind_folio() expects
the folio off the LRU. It still misses other lock holders (migration,
split, ...). Rotating only helps under memory pressure.