[PATCH v3 0/3] mm: zswap: free cold writeback folios promptly

[PATCH v3 0/3] mm: zswap: free cold writeback folios promptly

[Alexandre Ghiti]

When zswap writes an entry back, it allocates an order-0 swap cache folio,
decompresses into it, and issues the write. The folio is cold by
construction, yet today it is left on the LRU for page reclaim to find and
free later. That wastes a reclaim scan and keeps cold memory resident
longer than necessary.

Rather than implement this in zswap, extend the existing dropbehind
mechanism to swap cache folios and have zswap opt into it (Yosry). A
PG_dropbehind folio is already dropped from its cache once writeback
completes instead of being left for reclaim; for a swap cache folio that
"drop" is removing it from the swap cache.

  Patch 1 - move LRU insertion out of the swap cache allocator into its
            callers, so a folio that will be dropped on writeback completion
            (zswap writeback) can be allocated off the LRU without a flag.

  Patch 2 - drop dropbehind swap cache folios on writeback completion. The
            drop needs the folio and swap cluster locks and may sleep, but
            writeback can complete in interrupt context, so it is deferred
            to a worker via a per-CPU llist, which frees them in batches.

  Patch 3 - zswap allocates its writeback folio off the LRU and marks it
            dropbehind, opting into the mechanism above.

v1: https://lore.kernel.org/linux-mm/20260718093723.153324-1-alex@ghiti.fr/
v2: https://lore.kernel.org/linux-mm/20260727143618.1582318-1-alex@ghiti.fr/

Changes in v3:
- Drop the synchronous-IO special case in zswap writeback: those folios are
  now marked dropbehind like any other and dropped from folio_end_writeback()
  instead of inline. It was only an optimisation for zswap on a sync backend,
  which is not worth the extra code (Yosry, Nhat). This also removes
  swap_dropbehind_free_one_folio(), which had no other caller.
- Queue the deferred drop on a dedicated WQ_MEM_RECLAIM workqueue instead of
  system_wq: the work item runs on the reclaim path and the folios it frees
  stay off the LRU until it does, so it needs a rescuer to guarantee forward
  progress under memory pressure.
- Use mem_cgroup_tryget()/mem_cgroup_put() rather than css_tryget()/css_put():
  struct mem_cgroup is only defined under CONFIG_MEMCG, so the latter failed
  to build with CONFIG_MEMCG=n.
- Use raw_cpu_ptr() for the per-CPU llist: now that synchronous-IO swap also
  marks its folios dropbehind, writeback can complete in task context and the
  enqueue can run preemptible.

Changes in v2:
- Make swap dropbehind a generic core-mm mechanism that zswap opts into,
  rather than a zswap-specific implementation (Yosry).
- Allocate off the LRU by moving folio_add_lru() out of the swap cache
  allocator into its callers, instead of a skip_lru flag; rename it to
  __swap_cache_alloc_folio() (Kairui).
- Batch the freed folios in the deferred worker (folios_put) instead of a
  put per folio (Nhat).
- Skip the folio in the free path if it is still under writeback
  (defensive) (Nhat).

Results
-------
Paired baseline vs series on async swap (NVMe, Patch 2 deferred path). Each
workload runs confined to a memory cgroup (memory.max) small enough to force
zswap shrinker writeback.

Kernel build (defconfig, make -j4; memory.max = 600M):

  metric            baseline       series      delta
  pgrotated           441028         2521     -99.4%
  pgsteal_direct     3524343      2869004     -18.6%
  pgscan_direct      8393791      7765008      -7.5%
  zswpwb              705129       699019      -0.9%
  build time (s)        1155         1114      -3.6%

Of the 699019 folios written back, 698990 (99.996%) were freed promptly on
writeback completion; only 28 fell back to reclaim.

MySQL/OLTP (sysbench, 10 tables x 1M rows, 512M buffer pool, 8 threads, 300s;
memory.max = 256M):

  metric            baseline       series      delta
  transactions/s      153.87       163.30      +6.1%
  p95 latency (ms)    157.42       145.82      -7.4%
  avg latency (ms)     52.09        49.05      -5.9%
  pgrotated           743738        22460     -97.0%
  pgsteal_direct     6886490      5445278     -20.9%
  pgscan_direct     13462510     10820730     -19.6%

Future work
-----------
Barry suggested extending this to MADV_PAGEOUT and general reclaim, since on
asynchronous swap devices the memory is not freed until a later reclaim scan
even after the folio has been written out. I prototyped dropbehind for all
reclaimed swap folios (not included here) and it regressed sysbench OLTP
throughput by ~15% on NVMe swap. This matches Kairui's point that for common
reclaim the swap cache is worth keeping to catch fast refaults, unlike
synchronous-IO or zswap writeback where the reclaimer already waits.

So this series stays scoped to zswap cold writeback. I still need time to work
out whether general reclaim / MADV_PAGEOUT dropbehind is worth pursuing for
everything or only for known-cold cases, and how; I will follow up.

Alexandre Ghiti (3):
  mm: swap: move LRU insertion out of the swap cache allocator
  mm: swap: drop dropbehind swap cache folios on writeback completion
  mm: zswap: drop cold writeback folios via swap dropbehind

 fs/splice.c          |   2 +-
 include/linux/swap.h |   6 ++-
 mm/filemap.c         |  19 ++++++++
 mm/swap.h            |   6 +--
 mm/swap_state.c      | 114 ++++++++++++++++++++++++++++++++++++++++---
 mm/truncate.c        |   2 +-
 mm/vmscan.c          |  13 +++--
 mm/zswap.c           |   7 ++-
 8 files changed, 148 insertions(+), 21 deletions(-)


base-commit: 626acb37cd445144f321f1b64cac9a93760fa716
-- 
2.53.0-Meta

[PATCH v3 1/3] mm: swap: move LRU insertion out of the swap cache allocator

[Alexandre Ghiti]

zswap writeback wants a swap cache folio it can free directly once
writeback completes, i.e. one that is not on the LRU (dropping the last
reference on an LRU folio would trip the free-time page-flag checks).

So defer the LRU addition to the callers of __swap_cache_alloc_folio(),
no functional change intended.

Suggested-by: Kairui Song <kasong@tencent.com>
Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
---
 mm/swap.h       |  6 +++---
 mm/swap_state.c | 20 ++++++++++++--------
 mm/zswap.c      |  5 +++--
 3 files changed, 18 insertions(+), 13 deletions(-)

diff --git a/mm/swap.h b/mm/swap.h
index 77d2d14eda42..fc44daae1de1 100644
--- a/mm/swap.h
+++ b/mm/swap.h
@@ -304,9 +304,9 @@ bool swap_cache_has_folio(swp_entry_t entry);
 struct folio *swap_cache_get_folio(swp_entry_t entry);
 void *swap_cache_get_shadow(swp_entry_t entry);
 void swap_cache_del_folio(struct folio *folio);
-struct folio *swap_cache_alloc_folio(swp_entry_t target_entry, gfp_t gfp_mask,
-				     unsigned long orders, struct vm_fault *vmf,
-				     struct mempolicy *mpol, pgoff_t ilx);
+struct folio *__swap_cache_alloc_folio(swp_entry_t target_entry, gfp_t gfp_mask,
+				       unsigned long orders, struct vm_fault *vmf,
+				       struct mempolicy *mpol, pgoff_t ilx);
 /* Below helpers require the caller to lock and pass in the swap cluster. */
 void __swap_cache_add_folio(struct swap_cluster_info *ci,
 			    struct folio *folio, swp_entry_t entry);
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 9c3a5cf99778..90638a8d7232 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -483,13 +483,11 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
 	node_stat_mod_folio(folio, NR_FILE_PAGES, nr_pages);
 	lruvec_stat_mod_folio(folio, NR_SWAPCACHE, nr_pages);
 
-	/* Caller will initiate read into locked new_folio */
-	folio_add_lru(folio);
 	return folio;
 }
 
 /**
- * swap_cache_alloc_folio - Allocate folio for swapped out slot in swap cache.
+ * __swap_cache_alloc_folio - Allocate folio for swapped out slot in swap cache.
  * @targ_entry: swap entry indicating the target slot
  * @gfp: memory allocation flags
  * @orders: allocation orders, must be non zero
@@ -501,13 +499,17 @@ static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci,
  * doing IO (e.g. swap in or zswap writeback). The swap slot indicated by
  * @targ_entry must have a non-zero swap count (swapped out).
  *
+ * The returned folio is locked and is NOT on the LRU. The caller must either
+ * add it to the LRU with folio_add_lru() so page reclaim can find it, or free
+ * it directly once done; a folio left off the LRU is unreclaimable and leaks.
+ *
  * Context: Caller must protect the swap device with reference count or locks.
  * Return: Returns the folio if allocation succeeded and folio is in the swap
  * cache. Returns error code if failed due to race, OOM or invalid arguments.
  */
-struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
-				     unsigned long orders, struct vm_fault *vmf,
-				     struct mempolicy *mpol, pgoff_t ilx)
+struct folio *__swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
+				       unsigned long orders, struct vm_fault *vmf,
+				       struct mempolicy *mpol, pgoff_t ilx)
 {
 	int order, err;
 	struct folio *ret;
@@ -643,12 +645,13 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
 		folio = swap_cache_get_folio(entry);
 		if (folio)
 			return folio;
-		folio = swap_cache_alloc_folio(entry, gfp, BIT(0), NULL, mpol, ilx);
+		folio = __swap_cache_alloc_folio(entry, gfp, BIT(0), NULL, mpol, ilx);
 	} while (PTR_ERR(folio) == -EEXIST);
 
 	if (IS_ERR_OR_NULL(folio))
 		return NULL;
 
+	folio_add_lru(folio);
 	swap_read_folio(folio, plug);
 	if (readahead) {
 		folio_set_readahead(folio);
@@ -683,12 +686,13 @@ struct folio *swapin_sync(swp_entry_t entry, gfp_t gfp, unsigned long orders,
 		folio = swap_cache_get_folio(entry);
 		if (folio)
 			return folio;
-		folio = swap_cache_alloc_folio(entry, gfp, orders, vmf, mpol, ilx);
+		folio = __swap_cache_alloc_folio(entry, gfp, orders, vmf, mpol, ilx);
 	} while (PTR_ERR(folio) == -EEXIST);
 
 	if (IS_ERR(folio))
 		return folio;
 
+	folio_add_lru(folio);
 	swap_read_folio(folio, NULL);
 	return folio;
 }
diff --git a/mm/zswap.c b/mm/zswap.c
index 761cd699e0a3..8163e6c5f76c 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -1000,8 +1000,8 @@ static int zswap_writeback_entry(struct zswap_entry *entry,
 		return -EEXIST;
 
 	mpol = get_task_policy(current);
-	folio = swap_cache_alloc_folio(swpentry, GFP_KERNEL, BIT(0), NULL, mpol,
-				       NO_INTERLEAVE_INDEX);
+	folio = __swap_cache_alloc_folio(swpentry, GFP_KERNEL, BIT(0), NULL, mpol,
+					 NO_INTERLEAVE_INDEX);
 	put_swap_device(si);
 
 	/*
@@ -1013,6 +1013,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry,
 	 */
 	if (IS_ERR(folio))
 		return PTR_ERR(folio);
+	folio_add_lru(folio);
 
 	/*
 	 * folio is locked, and the swapcache is now secured against
-- 
2.53.0-Meta

[PATCH v3 2/3] mm: swap: drop dropbehind swap cache folios on writeback completion

[Alexandre Ghiti]

A PG_dropbehind folio is dropped from its cache once writeback completes
rather than left for reclaim to find later; this is implemented for file
folios in folio_end_dropbehind(). Extend it to swap cache folios.

The drop needs the folio and swap cluster locks and may sleep, but
writeback can complete in interrupt context, so defer the work to a
workqueue that runs once writeback has completed and frees the folios in
batches.

Suggested-by: Yosry Ahmed <yosry@kernel.org>
Suggested-by: Johannes Weiner <hannes@cmpxchg.org>
Suggested-by: Nhat Pham <nphamcs@gmail.com>
Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
---
 fs/splice.c          |  2 +-
 include/linux/swap.h |  6 ++-
 mm/filemap.c         | 19 +++++++++
 mm/swap_state.c      | 94 ++++++++++++++++++++++++++++++++++++++++++++
 mm/truncate.c        |  2 +-
 mm/vmscan.c          | 13 ++++--
 6 files changed, 130 insertions(+), 6 deletions(-)

diff --git a/fs/splice.c b/fs/splice.c
index 9d8f63e2fd1a..79424bbee1b0 100644
--- a/fs/splice.c
+++ b/fs/splice.c
@@ -90,7 +90,7 @@ static bool page_cache_pipe_buf_try_steal(struct pipe_inode_info *pipe,
 		 * If we succeeded in removing the mapping, set LRU flag
 		 * and return good.
 		 */
-		if (remove_mapping(mapping, folio)) {
+		if (remove_mapping(mapping, folio, false, NULL)) {
 			buf->flags |= PIPE_BUF_FLAG_LRU;
 			return true;
 		}
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 6d72778e6cc3..25470634dd9d 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -373,7 +373,8 @@ extern unsigned long mem_cgroup_shrink_node(struct mem_cgroup *mem,
 						unsigned long *nr_scanned);
 extern unsigned long shrink_all_memory(unsigned long nr_pages);
 extern int vm_swappiness;
-long remove_mapping(struct address_space *mapping, struct folio *folio);
+long remove_mapping(struct address_space *mapping, struct folio *folio,
+		    bool reclaimed, struct mem_cgroup *target_memcg);
 
 #if defined(CONFIG_SYSFS) && defined(CONFIG_NUMA)
 extern int reclaim_register_node(struct node *node);
@@ -467,6 +468,8 @@ void swap_put_entries_direct(swp_entry_t entry, int nr);
  */
 bool folio_free_swap(struct folio *folio);
 
+void swap_writeback_dropbehind_folio(struct folio *folio);
+
 /* Allocate / free (hibernation) exclusive entries */
 swp_entry_t swap_alloc_hibernation_slot(int type);
 void swap_free_hibernation_slot(swp_entry_t entry);
@@ -477,6 +480,7 @@ static inline void put_swap_device(struct swap_info_struct *si)
 }
 
 #else /* CONFIG_SWAP */
+static inline void swap_writeback_dropbehind_folio(struct folio *folio) {}
 static inline struct swap_info_struct *get_swap_device(swp_entry_t entry)
 {
 	return NULL;
diff --git a/mm/filemap.c b/mm/filemap.c
index dc3a0e960b9f..aaffacdf5b1f 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -1680,6 +1680,8 @@ EXPORT_SYMBOL_GPL(folio_end_writeback_no_dropbehind);
  */
 void folio_end_writeback(struct folio *folio)
 {
+	bool swap_dropbehind;
+
 	VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio);
 
 	/*
@@ -1689,7 +1691,24 @@ void folio_end_writeback(struct folio *folio)
 	 * reused before the folio_wake_bit().
 	 */
 	folio_get(folio);
+
+	/*
+	 * Dropbehind swap cache folios are off-LRU, so we must prevent a racing
+	 * swapin from removing the folio from the swap cache and keeping it
+	 * off-LRU: the writeback flag allows that. Afterwards a swapin may win
+	 * the race, but the folio is already queued and the worker puts it back
+	 * on the LRU in that case.
+	 */
+	swap_dropbehind = folio_test_swapcache(folio) &&
+			  folio_test_dropbehind(folio);
+
 	folio_end_writeback_no_dropbehind(folio);
+
+	if (swap_dropbehind) {
+		swap_writeback_dropbehind_folio(folio);
+		return;
+	}
+
 	folio_end_dropbehind(folio);
 	folio_put(folio);
 }
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 90638a8d7232..b93dd607d24c 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -16,6 +16,8 @@
 #include <linux/init.h>
 #include <linux/pagemap.h>
 #include <linux/folio_batch.h>
+#include <linux/llist.h>
+#include <linux/workqueue.h>
 #include <linux/backing-dev.h>
 #include <linux/blkdev.h>
 #include <linux/migrate.h>
@@ -537,6 +539,98 @@ struct folio *__swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp,
 	return ret;
 }
 
+static DEFINE_PER_CPU(struct llist_head, swap_dropbehind_llist);
+
+static bool swap_dropbehind_drop_folio(struct folio *folio)
+{
+	struct mem_cgroup *memcg;
+	bool dropped = false;
+
+	folio_lock(folio);
+
+	/* The folio was allocated off the LRU and nothing re-adds it here. */
+	VM_WARN_ON_ONCE_FOLIO(folio_test_lru(folio), folio);
+
+	rcu_read_lock();
+	memcg = folio_memcg(folio);
+	if (!mem_cgroup_tryget(memcg))
+		memcg = NULL;
+	rcu_read_unlock();
+
+	/*
+	 * Gate remove_mapping() on folio_test_swapcache(): a racing swapin may
+	 * have freed the swap slot (folio_free_swap()) and dropped the folio from
+	 * the cache, and remove_mapping() must not run on a non-swapcache folio
+	 * (it would trip __remove_mapping()'s mapping == folio_mapping() check).
+	 */
+	if (folio_test_swapcache(folio) && !folio_test_writeback(folio) &&
+	    remove_mapping(swap_address_space(folio->swap), folio, true, memcg)) {
+		dropped = true;
+	} else {
+		/* Raced: the folio is now owned by the swapin; put it back. */
+		folio_clear_dropbehind(folio);
+		folio_add_lru(folio);
+	}
+
+	mem_cgroup_put(memcg);
+
+	folio_unlock(folio);
+	if (!dropped)
+		folio_put(folio);
+	return dropped;
+}
+
+/**
+ * swap_dropbehind_free_batch_folio - free a dropbehind swap cache folio into a batch
+ * @folio: the off-LRU folio whose writeback has completed
+ * @fbatch: batch of folios to free, flushed when full
+ */
+static void swap_dropbehind_free_batch_folio(struct folio *folio,
+					     struct folio_batch *fbatch)
+{
+	if (swap_dropbehind_drop_folio(folio) && !folio_batch_add(fbatch, folio))
+		folios_put(fbatch);
+}
+
+static void swap_dropbehind_workfn(struct work_struct *work)
+{
+	struct folio_batch fbatch;
+	struct llist_node *pos, *next;
+	int cpu;
+
+	folio_batch_init(&fbatch);
+	for_each_possible_cpu(cpu) {
+		pos = llist_del_all(per_cpu_ptr(&swap_dropbehind_llist, cpu));
+		llist_for_each_safe(pos, next, pos) {
+			struct folio *folio = container_of((struct list_head *)pos,
+							   struct folio, lru);
+			swap_dropbehind_free_batch_folio(folio, &fbatch);
+		}
+	}
+	if (fbatch.nr)
+		folios_put(&fbatch);
+}
+
+static DECLARE_WORK(swap_dropbehind_work, swap_dropbehind_workfn);
+static struct workqueue_struct *swap_dropbehind_wq;
+
+void swap_writeback_dropbehind_folio(struct folio *folio)
+{
+	llist_add((struct llist_node *)&folio->lru,
+		  raw_cpu_ptr(&swap_dropbehind_llist));
+	queue_work(swap_dropbehind_wq, &swap_dropbehind_work);
+}
+
+static int __init swap_dropbehind_init(void)
+{
+	swap_dropbehind_wq = alloc_workqueue("swap_dropbehind",
+					     WQ_MEM_RECLAIM | WQ_PERCPU, 0);
+	if (!swap_dropbehind_wq)
+		return -ENOMEM;
+	return 0;
+}
+core_initcall(swap_dropbehind_init);
+
 /*
  * If we are the only user, then try to free up the swap cache.
  *
diff --git a/mm/truncate.c b/mm/truncate.c
index b58ba940be47..37f7d8278c1c 100644
--- a/mm/truncate.c
+++ b/mm/truncate.c
@@ -336,7 +336,7 @@ long mapping_evict_folio(struct address_space *mapping, struct folio *folio)
 	if (!filemap_release_folio(folio, 0))
 		return 0;
 
-	return remove_mapping(mapping, folio);
+	return remove_mapping(mapping, folio, false, NULL);
 }
 
 /**
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 3f3ff25e561a..37b093c4628c 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -789,17 +789,24 @@ static int __remove_mapping(struct address_space *mapping, struct folio *folio,
  * remove_mapping() - Attempt to remove a folio from its mapping.
  * @mapping: The address space.
  * @folio: The folio to remove.
+ * @reclaimed: Whether the folio is being reclaimed (record a shadow).
+ * @target_memcg: The memcg to charge the eviction shadow to when @reclaimed;
+ *                the caller must keep it alive across the call. Ignored (may
+ *                be NULL) when @reclaimed is false.
  *
  * If the folio is dirty, under writeback or if someone else has a ref
- * on it, removal will fail.
+ * on it, removal will fail. When @reclaimed is true, a workingset eviction
+ * shadow is stored (like page reclaim does) so that a later refault can be
+ * detected and the folio re-activated.
  * Return: The number of pages removed from the mapping.  0 if the folio
  * could not be removed.
  * Context: The caller should have a single refcount on the folio and
  * hold its lock.
  */
-long remove_mapping(struct address_space *mapping, struct folio *folio)
+long remove_mapping(struct address_space *mapping, struct folio *folio,
+		    bool reclaimed, struct mem_cgroup *target_memcg)
 {
-	if (__remove_mapping(mapping, folio, false, NULL)) {
+	if (__remove_mapping(mapping, folio, reclaimed, target_memcg)) {
 		/*
 		 * Unfreezing the refcount with 1 effectively
 		 * drops the pagecache ref for us without requiring another
-- 
2.53.0-Meta

[PATCH v3 3/3] mm: zswap: drop cold writeback folios via swap dropbehind

[Alexandre Ghiti]

zswap writeback decompresses an entry into a fresh swap cache folio and
writes it back. The folio is cold by construction, yet it is left on the
LRU for reclaim to find and free later, wasting a reclaim scan and keeping
cold memory resident longer than necessary.

Allocate the folio off the LRU and mark it PG_dropbehind so the swap
dropbehind path frees it from the swap cache once writeback completes.

Suggested-by: Johannes Weiner <hannes@cmpxchg.org>
Suggested-by: Nhat Pham <nphamcs@gmail.com>
Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
---
 mm/zswap.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/mm/zswap.c b/mm/zswap.c
index 8163e6c5f76c..a35671e837f8 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -1013,7 +1013,6 @@ static int zswap_writeback_entry(struct zswap_entry *entry,
 	 */
 	if (IS_ERR(folio))
 		return PTR_ERR(folio);
-	folio_add_lru(folio);
 
 	/*
 	 * folio is locked, and the swapcache is now secured against
@@ -1046,8 +1045,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry,
 	/* folio is up to date */
 	folio_mark_uptodate(folio);
 
-	/* move it to the tail of the inactive list after end_writeback */
-	folio_set_reclaim(folio);
+	folio_set_dropbehind(folio);
 
 	/* start writeback */
 	__swap_writepage(folio, NULL);
-- 
2.53.0-Meta

Re: [PATCH v3 3/3] mm: zswap: drop cold writeback folios via swap dropbehind

[Alexandre Ghiti]

I am pasting and answering Sashiko's review inline below.

On Tue, Aug 18, 2026 at 6:35 PM Alexandre Ghiti <alex@ghiti.fr> wrote:
>
> >
> zswap writeback decompresses an entry into a fresh swap cache folio and
> writes it back. The folio is cold by construction, yet it is left on the
> LRU for reclaim to find and free later, wasting a reclaim scan and keeping
> cold memory resident longer than necessary.
>
> Allocate the folio off the LRU and mark it PG_dropbehind so the swap
> dropbehind path frees it from the swap cache once writeback completes.
>
> Suggested-by: Johannes Weiner <hannes@cmpxchg.org>
> Suggested-by: Nhat Pham <nphamcs@gmail.com>
> Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
> ---
>  mm/zswap.c | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/mm/zswap.c b/mm/zswap.c
> index 8163e6c5f76c..a35671e837f8 100644
> --- a/mm/zswap.c
> +++ b/mm/zswap.c
> @@ -1013,7 +1013,6 @@ static int zswap_writeback_entry(struct zswap_entry *entry,
>          */
>         if (IS_ERR(folio))
>                 return PTR_ERR(folio);
> -       folio_add_lru(folio);
>
>         /*
>          * folio is locked, and the swapcache is now secured against
> @@ -1046,8 +1045,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry,
>         /* folio is up to date */
>         folio_mark_uptodate(folio);
>
> -       /* move it to the tail of the inactive list after end_writeback */
> -       folio_set_reclaim(folio);
> +       folio_set_dropbehind(folio);

Sashiko: "When setting dropbehind here, what happens if a concurrent swapoff
(try_to_unuse) removes the folio from the swap cache before the dropbehind
worker runs?
Looking at swap_dropbehind_drop_folio() in mm/swap_state.c:
if (folio_test_swapcache(folio) && !folio_test_writeback(folio) &&
remove_mapping(swap_address_space(folio->swap), folio, true, memcg)) {
dropped = true;
} else {
/* Raced: the folio is now owned by the swapin; put it back. */
folio_clear_dropbehind(folio);
folio_add_lru(folio);
}
If try_to_unuse() removes the folio from the swap cache, folio_test_swapcache()
will be false. The dropbehind worker will then blindly add the unmapped
anonymous folio to the LRU.
When reclaimed later, shrink_folio_list() will see it has PG_swapbacked but
no swap cache, causing it to allocate a new swap entry, mark it dirty, and
needlessly write the data back out to disk.
Could this race cause ghost anonymous folios to be added to the LRU?"

If swapoff frees the swap slot while the folio is queued for the dropbehind
drop, then yes, the worker takes the fallback and puts the folio back on the
LRU and that is exactly what has to happen: swapoff has just invalidated the
on-disk copy, so the folio now holds the only copy of the data. Leaving it off
the LRU would make it unreclaimable. Note the folio is marked dirty by
folio_free_swap() itself, for the same reason.

This is also not new: in the current code zswap's writeback folio is already on
the LRU when try_to_unuse() calls folio_free_swap() on it, so the resulting
state (unmapped, dirty, anon, on the LRU, no swap entry) is the same with and
without this series. Only the caller of folio_add_lru() differs.

>
>         /* start writeback */
>         __swap_writepage(folio, NULL);

Sashiko: "Does holding a reference to the folio until after
__swap_writepage() returns
prevent the dropbehind optimization from working on fast or synchronous swap
devices?
Because zswap_writeback_entry() holds a reference to the folio that isn't
released until it calls folio_put() at the end of the function, a race
can occur with synchronous I/O.
For synchronous devices (like zram), folio_end_writeback() is called and the
dropbehind worker is queued before __swap_writepage() even returns.
If the dropbehind worker calls remove_mapping() before zswap_writeback_entry()
calls folio_put(), remove_mapping() will see an unexpectedly high refcount
and fail.
The worker will then falsely assume a racing swapin and add the cold folio to
the LRU, defeating the purpose of the patch."

Yes, that race is real but I estimated it would be rare enough since
the folio_put() call is right after __swap_writepage(), and the time
the worker wakes up, the window is very small to me. I haven't
measured the actual rate, but I can measure it if needed. And if the
race is rare with sync devices, it is even more unlikely with async
devices.

In the end, I don't think any modifications are needed based on
Sashiko's review, let me know what you think.

Thanks,

Alex

> --
> 2.53.0-Meta
>
>

Re: [PATCH v3 2/3] mm: swap: drop dropbehind swap cache folios on writeback completion

[Barry Song]

On Wed, Aug 19, 2026 at 12:34 AM Alexandre Ghiti <alex@ghiti.fr> wrote:
>
> A PG_dropbehind folio is dropped from its cache once writeback completes
> rather than left for reclaim to find later; this is implemented for file
> folios in folio_end_dropbehind(). Extend it to swap cache folios.
>
> The drop needs the folio and swap cluster locks and may sleep, but
> writeback can complete in interrupt context, so defer the work to a
> workqueue that runs once writeback has completed and frees the folios in
> batches.
>
> Suggested-by: Yosry Ahmed <yosry@kernel.org>
> Suggested-by: Johannes Weiner <hannes@cmpxchg.org>
> Suggested-by: Nhat Pham <nphamcs@gmail.com>
> Signed-off-by: Alexandre Ghiti <alex@ghiti.fr>
> ---
[...]
>
> +static DEFINE_PER_CPU(struct llist_head, swap_dropbehind_llist);
> +
> +static bool swap_dropbehind_drop_folio(struct folio *folio)
> +{
> +       struct mem_cgroup *memcg;
> +       bool dropped = false;
> +
> +       folio_lock(folio);
> +
> +       /* The folio was allocated off the LRU and nothing re-adds it here. */
> +       VM_WARN_ON_ONCE_FOLIO(folio_test_lru(folio), folio);

Hi Alexandre,
Thanks very much for your patch!

I don't quite understand why we only support NON-LRU folios. Does this
restriction prevent potential optimizations for madv_pageout?

Also, zram might benefit from your patch in the future with
asynchronous hardware compression and software compression such as
kcompressd[1].

[1] https://lore.kernel.org/linux-mm/20250430082651.3152444-1-qun-wei.lin@mediatek.com/

[...]
> +
> +/**
> + * swap_dropbehind_free_batch_folio - free a dropbehind swap cache folio into a batch
> + * @folio: the off-LRU folio whose writeback has completed
> + * @fbatch: batch of folios to free, flushed when full
> + */
> +static void swap_dropbehind_free_batch_folio(struct folio *folio,
> +                                            struct folio_batch *fbatch)
> +{
> +       if (swap_dropbehind_drop_folio(folio) && !folio_batch_add(fbatch, folio))
> +               folios_put(fbatch);
> +}
> +
> +static void swap_dropbehind_workfn(struct work_struct *work)
> +{
> +       struct folio_batch fbatch;
> +       struct llist_node *pos, *next;
> +       int cpu;
> +
> +       folio_batch_init(&fbatch);
> +       for_each_possible_cpu(cpu) {
> +               pos = llist_del_all(per_cpu_ptr(&swap_dropbehind_llist, cpu));
> +               llist_for_each_safe(pos, next, pos) {
> +                       struct folio *folio = container_of((struct list_head *)pos,
> +                                                          struct folio, lru);
> +                       swap_dropbehind_free_batch_folio(folio, &fbatch);
> +               }
> +       }
> +       if (fbatch.nr)
> +               folios_put(&fbatch);
> +}
> +
> +static DECLARE_WORK(swap_dropbehind_work, swap_dropbehind_workfn);
> +static struct workqueue_struct *swap_dropbehind_wq;
> +
> +void swap_writeback_dropbehind_folio(struct folio *folio)
> +{
> +       llist_add((struct llist_node *)&folio->lru,
> +                 raw_cpu_ptr(&swap_dropbehind_llist));
> +       queue_work(swap_dropbehind_wq, &swap_dropbehind_work);
> +}
> +

This is good. For file dropbehind, I see both F2FS [1] and block
devices [2] reinventing workqueues in their own filesystems or
drivers, while folio_end_dropbehind() simply benefits from its safe,
sleepable context. This seems a bit odd.

So I'm really curious whether this could be extended to file-backed
folios as well, so that those filesystems and device drivers don't
have to deal with this themselves.

[1] https://lore.kernel.org/all/20260820071438.695893-1-qiwenjie@xiaomi.com/
[2] https://lore.kernel.org/all/20260730-blk-dontcache-v7-2-3e8e6850068d@columbia.edu/

Thanks
Barry

Re: [PATCH v3 2/3] mm: swap: drop dropbehind swap cache folios on writeback completion

[Matthew Wilcox]

On Thu, Aug 20, 2026 at 07:36:42PM +0800, Barry Song wrote:
> > +void swap_writeback_dropbehind_folio(struct folio *folio)
> > +{
> > +       llist_add((struct llist_node *)&folio->lru,
> > +                 raw_cpu_ptr(&swap_dropbehind_llist));
> > +       queue_work(swap_dropbehind_wq, &swap_dropbehind_work);
> > +}
> > +
> 
> This is good. For file dropbehind, I see both F2FS [1] and block
> devices [2] reinventing workqueues in their own filesystems or
> drivers, while folio_end_dropbehind() simply benefits from its safe,
> sleepable context. This seems a bit odd.
> 
> So I'm really curious whether this could be extended to file-backed
> folios as well, so that those filesystems and device drivers don't
> have to deal with this themselves.

You should probably both be aware of Tal's series:

https://lore.kernel.org/linux-mm/20260730-blk-dontcache-v7-0-3e8e6850068d@columbia.edu/

In fact, I'd base this series on Tal's.  It would be considerably
easier.