Re: [PATCH mm-next] mm/swap: free swap cache folios the zap path gave up on

[Jiayuan Chen <jiayuan.chen@linux.dev>]

On 10/14/26 5:08 PM, Jiayuan Chen wrote:
> From: Jiayuan Chen <jiayuan.chen@shopee.com>
>
> On our production hosts we saw swap cache only pages keep growing for days:
> on a box with a 500 GiB swap partition, 250 GiB of RAM and 250 GiB of
> swap slots were held by folios that no process mapped any more, charged
> to cgroups whose workloads had long exited. The only way to get them
> back was swapoff, and with other services actively swapping on the
> same host that is not an option.
>
> Such a slot has swap count 0 and an unmapped folio pinned in the swap
> cache, left there after a process exited or did MADV_DONTNEED. No page
> table holds its swap entry any more, so nothing comes back for the
> slot; LRU reclaim drops it eventually, and the allocator only reclaims
> such slots once swap is half full. The growth stopped exactly there:
> at 50% of the device, where vm_swap_full() becomes true.
>
> The race is readahead against a zap in a neighbouring thread. Readahead
> works on virtual addresses, so a fault at the end of thread A's range
> pulls in the entries at the start of thread B's range:
>
>    thread A                               thread B
>    do_swap_page()
>      swap_vma_readahead()
>        swap_cache_read_folio(B's entry)
>          folio locked, added to cache
>          swap_read_folio()                MADV_DONTNEED
>            zswap_load() / bio in flight     zap_nonpresent_ptes()
>            .                                  swap_put_entries_cluster()
>            .                                    swap count 1 -> 0
>            .                                    __try_to_reclaim_swap()
>            .                                      folio_trylock() fails
>            .                                      gives up
>            folio_unlock()
>        folio_put()
>    -> slot: swap count 0, folio in swap cache, not mapped, never freed
>
> The lock is one reason it gives up. The other is writeback: with a block
> device as backing store, B's own PAGEOUT leaves the folio in the swap
> cache under writeback, and when the DONTNEED comes __try_to_reclaim_swap()
> gets the lock but folio_swapcache_freeable() refuses. The write
> completes, nobody frees the slot. The present-PTE side has the same two
> outcomes through free_swap_cache().
>
> Options considered:
>
> 1. Fix it on the readahead side: after the read, check the swap count
>     and free the slot. Covers the lock, not writeback, and the
>     asynchronous read path unlocks in the bio completion where the slot
>     cannot be freed. Two places to patch, neither clean.
> 2. swapoff/swapon clears it, as does memory.reclaim on the right cgroup.
>     That is a workaround, not a fix.
> 3. Have the side that gives up come back later. This patch.
>
> The zap runs under the PTL and the fault path takes the folio lock
> before the PTL, so waiting there is out; a worker has to do it. On a
> lost trylock or a writeback refusal under TTRS_UNMAPPED,
> __try_to_reclaim_swap() schedules si->sweep_work. The worker walks the
> in-use clusters and, for every slot with swap count 0 whose folio is
> unmapped, calls __try_to_reclaim_swap() with TTRS_WAIT, which takes the
> folio lock and waits for writeback instead of giving up.
> free_swap_cache() now calls __try_to_reclaim_swap() as well, so both
> paths give up, and wake the worker, in the same place.
>
> For now the worker scans every in-use cluster of the device; marking
> the clusters that need it can reduce that later.
>
> Cc: Jiayuan Chen <jiayuan.chen@linux.dev>
> Signed-off-by: Jiayuan Chen <jiayuan.chen@shopee.com>
> ---
>   include/linux/swap.h |   1 +
>   mm/swap.h            |   1 +
>   mm/swap_state.c      |   9 ++--
>   mm/swapfile.c        | 101 ++++++++++++++++++++++++++++++++++++++++++-
>   4 files changed, 104 insertions(+), 8 deletions(-)
>
> diff --git a/include/linux/swap.h b/include/linux/swap.h
> index 4f686709d71f..5777f10e9aa4 100644
> --- a/include/linux/swap.h
> +++ b/include/linux/swap.h
> @@ -267,6 +267,7 @@ struct swap_info_struct {
>   					 */
>   	struct work_struct discard_work; /* discard worker */
>   	struct work_struct reclaim_work; /* reclaim worker */
> +	struct work_struct sweep_work;	/* frees swap cache only slots a zap gave up on */
>   	struct list_head discard_clusters; /* discard clusters list */
>   	struct plist_node avail_list;   /* entry in swap_avail_head */
>   	const struct swap_ops *ops;
> diff --git a/mm/swap.h b/mm/swap.h
> index d5bf21f517dc..f62f36c0ce5c 100644
> --- a/mm/swap.h
> +++ b/mm/swap.h
> @@ -309,6 +309,7 @@ static inline bool folio_matches_swap_entry(const struct folio *folio,
>    * - See the comment of get_swap_device() for more complex usage.
>    */
>   bool swap_cache_has_folio(swp_entry_t entry);
> +void swap_reclaim_entry(swp_entry_t entry);
>   struct folio *swap_cache_get_folio(swp_entry_t entry);
>   void *swap_cache_get_shadow(swp_entry_t entry);
>   void swap_cache_del_folio(struct folio *folio);
> diff --git a/mm/swap_state.c b/mm/swap_state.c
> index ebef568cd62e..99708f149b25 100644
> --- a/mm/swap_state.c
> +++ b/mm/swap_state.c
> @@ -598,16 +598,13 @@ void swap_writeback_dropbehind_folio(struct folio *folio)
>    *
>    * Its ok to check the swapcache flag without the folio lock
>    * here because we are going to recheck again inside
> - * folio_free_swap() _with_ the lock.
> + * __try_to_reclaim_swap() _with_ the lock.
>    * 					- Marcelo
>    */
>   void free_swap_cache(struct folio *folio)
>   {
> -	if (folio_test_swapcache(folio) && !folio_mapped(folio) &&
> -	    folio_trylock(folio)) {
> -		folio_free_swap(folio);
> -		folio_unlock(folio);
> -	}
> +	if (folio_test_swapcache(folio) && !folio_mapped(folio))
> +		swap_reclaim_entry(folio->swap);
>   }
>   



I spent some more time on this and I'm not happy with any of the
options I have. Advice welcome.

   1. Trigger the worker where the zap gives up (this patch). On the
      swap entry side that works, __try_to_reclaim_swap() has the si.
      On the present PTE side it doesn't: free_swap_cache() lost the
      trylock, so it can't read folio->swap to find the si, as Kairui
      pointed out. And that trylock-and-give-up is not in one place,
      free_swap_cache(), do_wp_page() and __uprobe_write() all have
      their own copy. Chasing every one of them doesn't feel right.

   2. Clean up at memcg offline. Doesn't cover the root cgroup or a
      cgroup that stays alive, and on mm-new the folios are reparented
      at offline anyway, so there is nothing left to find there.

   3. A swapon flag, say --clean-cache, that enables the cleanup.
      Pro: covers everything, and the cleanup can live in its own
      function without touching the existing paths. Con: we know to
      turn it on because we hit this, others won't. Needs util-linux
      too.

   4. A per-device worker that walks the device on its own, say 1024
      clusters per second, backing off when a pass finds nothing.
      Pro: same as 3, and nothing to turn on. Con: a worker that wakes
      up on its own for this, I expect objections.

   5. A sysctl (or a drop_caches bit) to kick the cleanup by hand.
      Pro: same as 4, all in the worker. Con: same as 3, someone has
      to know to poke it.