[PATCH 0/5 v2] fs: Deferred inode reclaim

[PATCH 0/5 v2] fs: Deferred inode reclaim

[Jan Kara]

Hello,

after a long pause here is a second revision of my patches implementing
deferred inode reclaim to deal with MM warnings due to GFP_NOFAIL allocations
from reclaim paths. This happens because to reclaim some inodes, filesystems
have to do IO including complex operations using journalling and forward
progress of these depends on successful memory allocations (which is impossible
to guarantee from reclaim context).

This time I have fully tested the patch set. In particular it survives full
fstests run on ext4 both as is as well as when marking all loaded ext4 inodes
for deferred reclaim to stress the deferred reclaim paths. I have also tested
(when marking all inodes for deferred reclaim) the patches by creating a memcg
with 128m memory limit and then scanning from 10 processes 600k inodes in total
which heavily exercises deferred inode reclaim paths. This also verified
efficiency of throttling of marking of inodes for deferred reclaim. Basically
we cannot guarantee any particular limit on the number of inodes queued for
deferred reclaim as that is bound only by their number in memory and memory
pressure. But as the number of queued inodes grows, the tasks creating them
get slowed down so eventually some equilibrium is hit. In my VM this was
at ~10k inodes (only slightly above the 8k limit when throttling kicks in)
but this all very much depends on the reclaim pressure, speed of deferred
reclaim etc. so I don't think this is some representative number.

The first patch is a pure fix for a problem I've hit when running fstests.  The
second patch deals with lazy timestamp updates which I've decided to handle
better directly inside the writeback infrastructure instead of deferring
reclaim. The remaining patches implement the inode reclaim deferal. Ext4 use of
deferred inode reclaim is there mostly as a demonstration. Other filesystems
need to determine which inodes need deferred reclaim and mark them as such
which generally requires good internal knowledge of the filesystem. I'm hoping
that once the infrastructure is there, fs developers prompted by MM warnings
will start using it :). I know for a fact that besides ext4 e.g. btrfs is
hitting MM warnings in inode reclaim as well.

								Honza

Changes since v1:
* Multiple workers to reclaim deferred inodes, related to that I've
  moved lists of deferred inodes from per-sb to global ones
* Added missed lazytime sync fix

Previous versions:
Link: http://lore.kernel.org/r/20260429174850.18223-1-jack@suse.cz # v1

[PATCH v2 1/5] writeback: Fix missed lazytime flush

[Jan Kara]

move_expired_inodes() moves to b_io list inodes which were dirtied at
current_time - dirtytime_expire_interval or before.
__writeback_single_inode() however calls sync_lazytime() only if the
inode was dirtied strictly before current_time -
dirtytime_expire_interval. It can thus happen that inode with dirty
timestamps gets queued to b_io list, then fails the check in
__writeback_single_inode() and as a result is just placed back to
b_dirty_time list with a refreshed timestamp and so timestamps aren't
written when they should have been.

This is a very difficult race to hit in practice but changes to how
dirty time expiry is handled on last iput() will make this much easier
to hit.

Fixes: 5fcd57505c00 ("writeback: Drop I_DIRTY_TIME_EXPIRE")
Signed-off-by: Jan Kara <jack@suse.cz>
---
 fs/fs-writeback.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index e744f9f9d43f..bb4546d147dc 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -1807,8 +1807,8 @@ __writeback_single_inode(struct inode *inode, struct writeback_control *wbc)
 	 */
 	if ((inode_state_read_once(inode) & I_DIRTY_TIME) &&
 	    (wbc->sync_mode == WB_SYNC_ALL ||
-	     time_after(jiffies, inode->dirtied_time_when +
-			dirtytime_expire_interval * HZ)))
+	     time_after_eq(jiffies, inode->dirtied_time_when +
+			   dirtytime_expire_interval * HZ)))
 		sync_lazytime(inode);
 
 	/*
-- 
2.51.0

[PATCH v2 2/5] fs: Avoid inode dirtying on last iput

[Jan Kara]

When inode has dirtied timestamps, we currently call sync_lazytime() on
last iput. This is done because inode with any dirty bit set is not
inserted into LRU and dirty timestamps expire only after many (12 by
default) hours so these inodes would be sitting outside of LRU aging for
a really long time. However this can result in doing IO and consequently
GFP_NOFAIL allocations from dentry reclaim making MM complain. Sample
trace for ext4 is:

prune_dcache_sb
shrink_dentry_list
__dentry_kill
iput
sync_lazytime
__mark_inode_dirty
ext4_dirty_inode
__ext4_mark_inode_dirty
ext4_reserve_inode_write
ext4_get_inode_loc
bdev_getblk
__filemap_get_folio_mpol

Avoid this dirtying on last iput by reshuffling unused inodes to the
beginning of b_dirty_time list and clobbering dirtied_time_when instead
so that they get written during next periodic writeback.

Signed-off-by: Jan Kara <jack@suse.cz>
---
 fs/fs-writeback.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 fs/inode.c        | 15 +++++++--------
 fs/internal.h     |  1 +
 3 files changed, 50 insertions(+), 8 deletions(-)

diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
index bb4546d147dc..762e310e808f 100644
--- a/fs/fs-writeback.c
+++ b/fs/fs-writeback.c
@@ -2837,6 +2837,48 @@ void __mark_inode_dirty(struct inode *inode, int flags)
 }
 EXPORT_SYMBOL(__mark_inode_dirty);
 
+/*
+ * If inode has dirty timestamps to write out, make sure flush worker writes
+ * them out during its next periodic writeback writeout.
+ */
+void queue_dirtytime_writeback(struct inode *inode)
+{
+	struct bdi_writeback *wb;
+	unsigned long new_time;
+
+	lockdep_assert_held(&inode->i_lock);
+
+	wb = locked_inode_to_wb_and_lock_list(inode);
+	spin_lock(&inode->i_lock);
+	/*
+	 * If inode writeback is already queued or inode got dirty, we have
+	 * nothing to do and we mustn't touch writeback lists anyway.
+	 */
+	if (inode_state_read(inode) & (I_SYNC_QUEUED | I_DIRTY))
+		goto out_wb_lock;
+	/* Written back while we dropped i_lock? */
+	if (!(inode_state_read(inode) & I_DIRTY_TIME))
+		goto out_wb_lock;
+
+	/*
+	 * Move inode to the beginning of dirty queue and clobber dirtied time
+	 * so that it gets written out during the next periodic writeback.
+	 */
+	new_time = jiffies - dirtytime_expire_interval * HZ;
+	if (!list_empty(&wb->b_dirty_time)) {
+		struct inode *first = wb_inode(wb->b_dirty_time.prev);
+		unsigned long first_time = READ_ONCE(first->dirtied_time_when);
+
+		if (time_before(first_time, new_time))
+			new_time = first_time;
+	}
+	inode->dirtied_when = new_time;
+	inode->dirtied_time_when = new_time;
+	list_move_tail(&inode->i_io_list, &wb->b_dirty_time);
+out_wb_lock:
+	spin_unlock(&wb->list_lock);
+}
+
 /*
  * The @s_sync_lock is used to serialise concurrent sync operations
  * to avoid lock contention problems with concurrent wait_sb_inodes() calls.
diff --git a/fs/inode.c b/fs/inode.c
index ba7da39be4a3..5555a513a0a3 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -2030,7 +2030,6 @@ void iput(struct inode *inode)
 	if (unlikely(!inode))
 		return;
 
-retry:
 	lockdep_assert_not_held(&inode->i_lock);
 	VFS_BUG_ON_INODE(inode_state_read_once(inode) & (I_FREEING | I_CLEAR), inode);
 	/*
@@ -2043,14 +2042,14 @@ void iput(struct inode *inode)
 	if (atomic_add_unless(&inode->i_count, -1, 1))
 		return;
 
-	if (inode->i_nlink && sync_lazytime(inode))
-		goto retry;
-
 	spin_lock(&inode->i_lock);
-	if (unlikely((inode_state_read(inode) & I_DIRTY_TIME) && inode->i_nlink)) {
-		spin_unlock(&inode->i_lock);
-		goto retry;
-	}
+	/*
+	 * If inode has timestamp updates pending, queue flushing them now as
+	 * otherwise the dirtiness could be preventing the inode from entering
+	 * LRU for hours.
+	 */
+	if (inode->i_nlink && inode_state_read(inode) & I_DIRTY_TIME)
+		queue_dirtytime_writeback(inode);
 
 	if (!atomic_dec_and_test(&inode->i_count)) {
 		spin_unlock(&inode->i_lock);
diff --git a/fs/internal.h b/fs/internal.h
index c658c8a5ebd5..eae0e2bb5d1c 100644
--- a/fs/internal.h
+++ b/fs/internal.h
@@ -219,6 +219,7 @@ bool in_group_or_capable(struct mnt_idmap *idmap,
  */
 long get_nr_dirty_inodes(void);
 bool sync_lazytime(struct inode *inode);
+void queue_dirtytime_writeback(struct inode *inode);
 
 /*
  * dcache.c
-- 
2.51.0

[PATCH v2 3/5] fs: Basic infrastructure for offloading inode reclaim

[Jan Kara]

Reclaim of some inodes is rather complex requiring running transactions
or doing other IO. Consequently filesystems end up doing GFP_NOFAIL
allocations from kswapd or even direct reclaim which is problematic
because forward progress of these allocations isn't guaranteed. Add
infrastructure for marking inodes whose reclaim is difficult and offload
reclaim of such inodes into a workqueue to not block kswapd with
difficult inode reclaim. The processing of the queued inodes happens in
parallel from several work items so that the reclaim has higher chances
of keeping up with the inflow of inodes if reclaim of some inodes
progresses slowly.

Signed-off-by: Jan Kara <jack@suse.cz>
---
 fs/inode.c                       | 197 ++++++++++++++++++++++++++++++-
 fs/super.c                       |   2 +
 include/linux/fs.h               |   3 +
 include/linux/fs/super_types.h   |   3 +
 include/trace/events/writeback.h |   3 +-
 5 files changed, 201 insertions(+), 7 deletions(-)

diff --git a/fs/inode.c b/fs/inode.c
index 5555a513a0a3..e902095cf39b 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -74,6 +74,25 @@ static DEFINE_PER_CPU(unsigned long, nr_unused);
 
 static struct kmem_cache *inode_cachep __ro_after_init;
 
+struct deferred_reclaim_queue {
+	struct list_head list;
+	spinlock_t lock;
+};
+
+/* Inodes for deferred reclaim */
+struct inode_deferred_reclaim {
+	int workers;			/* Number of parallel workers */
+	struct workqueue_struct *wq;	/* Workqueue for workers */
+	struct work_struct *work;	/* Workers */
+	/*
+	 * Queue of deferred inodes. The number of queues is the same number
+	 * as the number of workers
+	 */
+	struct deferred_reclaim_queue *queues;
+};
+
+static struct inode_deferred_reclaim deferred_reclaim;
+
 static long get_nr_inodes(void)
 {
 	int i;
@@ -919,6 +938,12 @@ void evict_inodes(struct super_block *sb)
 }
 EXPORT_SYMBOL_GPL(evict_inodes);
 
+struct inodes_to_prune {
+	struct list_head freeable;
+	struct list_head deferred;
+	unsigned int deferred_count;
+};
+
 /*
  * Isolate the inode from the LRU in preparation for freeing it.
  *
@@ -933,7 +958,7 @@ EXPORT_SYMBOL_GPL(evict_inodes);
 static enum lru_status inode_lru_isolate(struct list_head *item,
 		struct list_lru_one *lru, void *arg)
 {
-	struct list_head *freeable = arg;
+	struct inodes_to_prune *lists = arg;
 	struct inode	*inode = container_of(item, struct inode, i_lru);
 
 	/*
@@ -950,7 +975,7 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
 	 * sync, or the last page cache deletion will requeue them.
 	 */
 	if (icount_read(inode) ||
-	    (inode_state_read(inode) & ~I_REFERENCED) ||
+	    inode_state_read(inode) & ~(I_REFERENCED | I_DEFER_RECLAIM) ||
 	    !mapping_shrinkable(&inode->i_data)) {
 		list_lru_isolate(lru, &inode->i_lru);
 		spin_unlock(&inode->i_lock);
@@ -988,13 +1013,27 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
 
 	WARN_ON(inode_state_read(inode) & I_NEW);
 	inode_state_set(inode, I_FREEING);
-	list_lru_isolate_move(lru, &inode->i_lru, freeable);
+	/* Inode will take long time to cleanup. Offload that to worker. */
+	if (inode_state_read(inode) & I_DEFER_RECLAIM) {
+		list_lru_isolate_move(lru, &inode->i_lru, &lists->deferred);
+		lists->deferred_count++;
+	} else {
+		list_lru_isolate_move(lru, &inode->i_lru, &lists->freeable);
+	}
 	spin_unlock(&inode->i_lock);
 
 	this_cpu_dec(nr_unused);
 	return LRU_REMOVED;
 }
 
+/* Number of inodes to process in one reclaim batch */
+#define INODE_RECLAIM_BATCH_SIZE 16
+
+static int get_deferred_sb_id(struct super_block *sb)
+{
+	return hash_ptr(sb, 32) % deferred_reclaim.workers;
+}
+
 /*
  * Walk the superblock inode LRU for freeable inodes and attempt to free them.
  * This is called from the superblock shrinker function with a number of inodes
@@ -1003,15 +1042,158 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
  */
 long prune_icache_sb(struct super_block *sb, struct shrink_control *sc)
 {
-	LIST_HEAD(freeable);
+	struct inodes_to_prune lists = {
+		.freeable = LIST_HEAD_INIT(lists.freeable),
+		.deferred = LIST_HEAD_INIT(lists.deferred),
+	};
 	long freed;
 
 	freed = list_lru_shrink_walk(&sb->s_inode_lru, sc,
-				     inode_lru_isolate, &freeable);
-	dispose_list(&freeable);
+				     inode_lru_isolate, &lists);
+	dispose_list(&lists.freeable);
+	if (!list_empty(&lists.deferred)) {
+		int id = get_deferred_sb_id(sb);
+		int i, wake_count;
+
+		spin_lock(&deferred_reclaim.queues[id].lock);
+		list_splice_tail(&lists.deferred,
+				 &deferred_reclaim.queues[id].list);
+		atomic_add(lists.deferred_count, &sb->s_deferred_reclaim_count);
+		spin_unlock(&deferred_reclaim.queues[id].lock);
+
+		/* Queue works to process inodes we've added to the list */
+		wake_count = (lists.deferred_count + INODE_RECLAIM_BATCH_SIZE)
+						/ INODE_RECLAIM_BATCH_SIZE;
+		if (wake_count > deferred_reclaim.workers)
+			wake_count = deferred_reclaim.workers;
+		for (i = 0; i < wake_count; i++) {
+			queue_work(deferred_reclaim.wq,
+				   &deferred_reclaim.work[id]);
+			id = (id + 1) % deferred_reclaim.workers;
+		}
+	}
 	return freed;
 }
 
+static void __get_inode_reclaim_batch(int *start, int end,
+				      struct list_head *list, int *count)
+{
+	int i;
+
+	for (i = *start; i < end; i++) {
+		if (list_empty_careful(&deferred_reclaim.queues[i].list))
+			continue;
+		spin_lock(&deferred_reclaim.queues[i].lock);
+		while (*count < INODE_RECLAIM_BATCH_SIZE &&
+		       !list_empty(&deferred_reclaim.queues[i].list)) {
+			list_move(deferred_reclaim.queues[i].list.next, list);
+			(*count)++;
+		}
+		spin_unlock(&deferred_reclaim.queues[i].lock);
+		if (*count >= INODE_RECLAIM_BATCH_SIZE)
+			break;
+	}
+	*start = i;
+}
+
+/* Move a batch of inodes from queued lists to our private list */
+static int get_inode_reclaim_batch(int *id, struct list_head *list)
+{
+	int count = 0;
+	int orig_id = *id;
+
+	__get_inode_reclaim_batch(id, deferred_reclaim.workers, list, &count);
+	if (count >= INODE_RECLAIM_BATCH_SIZE)
+		return count;
+	/* Wrap around */
+	if (orig_id > 0) {
+		*id = 0;
+		__get_inode_reclaim_batch(id, orig_id, list, &count);
+	}
+	return count;
+}
+
+
+static void inode_reclaim_deferred(struct work_struct *work)
+{
+	struct inode *inode;
+	LIST_HEAD(inode_batch);
+	/*
+	 * We start with the list corresponding to the work but rolling a dice
+	 * would work as well
+	 */
+	int id = work - deferred_reclaim.work;
+
+	while (1) {
+		if (!get_inode_reclaim_batch(&id, &inode_batch))
+			break;
+		while (!list_empty(&inode_batch)) {
+			struct super_block *sb;
+
+			/*
+			 * inode_batch list is private and I_FREEING flags
+			 * protect us from anybody else trying to remove the
+			 * inode from the LRU list. No locking needed.
+			 */
+			inode = list_first_entry(&inode_batch, struct inode,
+						 i_lru);
+			list_del_init(&inode->i_lru);
+			sb = inode->i_sb;
+			evict(inode);
+			/*
+			 * The elevated s_deferred_reclaim_count keeps sb alive
+			 * until we drop it
+			 */
+			if (atomic_dec_and_test(&sb->s_deferred_reclaim_count))
+				wake_up_var(&sb->s_deferred_reclaim_count);
+			cond_resched();
+		}
+	}
+}
+
+/* Maximum number of inode reclaim workers */
+#define MAX_RECLAIM_WORKERS 16
+
+static void __init deferred_reclaim_init(void)
+{
+	int workers = MAX_RECLAIM_WORKERS;
+	int i;
+
+	/* Scale down the number of workers for small systems */
+	if (workers > num_possible_cpus())
+		workers = num_possible_cpus();
+
+	deferred_reclaim.workers = workers;
+	deferred_reclaim.work = kmalloc_objs(struct work_struct, workers);
+	deferred_reclaim.queues =
+			kmalloc_objs(struct deferred_reclaim_queue, workers);
+	if (!deferred_reclaim.work || !deferred_reclaim.queues)
+		panic("Failed to allocate deferred inode queues");
+
+	deferred_reclaim.wq = alloc_workqueue("deferred-inodegc",
+			WQ_FREEZABLE | WQ_MEM_RECLAIM | WQ_UNBOUND,
+			workers);
+	if (!deferred_reclaim.wq)
+		panic("Failed to allocate deferred inode reclaim workqueue");
+
+	for (i = 0; i < workers; i++) {
+		INIT_WORK(&deferred_reclaim.work[i], inode_reclaim_deferred);
+		spin_lock_init(&deferred_reclaim.queues[i].lock);
+		INIT_LIST_HEAD(&deferred_reclaim.queues[i].list);
+	}
+}
+
+void mark_inode_reclaim_deferred(struct inode *inode)
+{
+	if (inode_state_read_once(inode) & I_DEFER_RECLAIM)
+		return;
+
+	spin_lock(&inode->i_lock);
+	inode_state_set(inode, I_DEFER_RECLAIM);
+	spin_unlock(&inode->i_lock);
+}
+EXPORT_SYMBOL_GPL(mark_inode_reclaim_deferred);
+
 static void __wait_on_freeing_inode(struct inode *inode, bool hash_locked, bool rcu_locked);
 static bool igrab_from_hash(struct inode *inode);
 
@@ -2659,6 +2841,9 @@ void __init inode_init(void)
 					 SLAB_ACCOUNT),
 					 init_once);
 
+	/* Deferred inode reclaim infrastructure */
+	deferred_reclaim_init();
+
 	/* Hash may have been set up in inode_init_early */
 	if (!hashdist)
 		return;
diff --git a/fs/super.c b/fs/super.c
index 05e443173038..0852a40b2635 100644
--- a/fs/super.c
+++ b/fs/super.c
@@ -600,6 +600,8 @@ void deactivate_locked_super(struct super_block *s)
 	struct file_system_type *fs = s->s_type;
 	if (atomic_dec_and_test(&s->s_active)) {
 		shrinker_free(s->s_shrink);
+		wait_var_event(&s->s_deferred_reclaim_count,
+			       !atomic_read(&s->s_deferred_reclaim_count));
 		fs->kill_sb(s);
 
 		kill_super_notify(s);
diff --git a/include/linux/fs.h b/include/linux/fs.h
index f9d1e05e8ae6..a289402563f8 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -741,6 +741,7 @@ enum inode_state_flags_enum {
 	I_SYNC_QUEUED		= (1U << 17),
 	I_PINNING_NETFS_WB	= (1U << 18),
 	I_METADATA_WRITEBACK	= (1U << 19),
+	I_DEFER_RECLAIM		= (1U << 20),
 };
 
 #define I_DIRTY_INODE (I_DIRTY_SYNC | I_DIRTY_DATASYNC)
@@ -2218,6 +2219,8 @@ static inline void set_inode_metadata_writeback(struct inode *inode)
 	spin_unlock(&inode->i_lock);
 }
 
+void mark_inode_reclaim_deferred(struct inode *inode);
+
 /*
  * returns the refcount on the inode. it can change arbitrarily.
  */
diff --git a/include/linux/fs/super_types.h b/include/linux/fs/super_types.h
index ecd96aeb1cee..a41bd57fc2e6 100644
--- a/include/linux/fs/super_types.h
+++ b/include/linux/fs/super_types.h
@@ -232,6 +232,9 @@ struct super_block {
 
 	struct shrinker				*s_shrink;	/* per-sb shrinker handle */
 
+	/* Number of inodes queued for deferred reclaim */
+	atomic_t				s_deferred_reclaim_count;
+
 	/* Number of inodes with nlink == 0 but still referenced */
 	atomic_long_t				s_remove_count;
 
diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
index 13ee076ccd16..4a9cfe458820 100644
--- a/include/trace/events/writeback.h
+++ b/include/trace/events/writeback.h
@@ -28,7 +28,8 @@
 		{I_DONTCACHE,		"I_DONTCACHE"},		\
 		{I_SYNC_QUEUED,		"I_SYNC_QUEUED"},	\
 		{I_PINNING_NETFS_WB,	"I_PINNING_NETFS_WB"},	\
-		{I_LRU_ISOLATING,	"I_LRU_ISOLATING"}	\
+		{I_LRU_ISOLATING,	"I_LRU_ISOLATING"},	\
+		{I_DEFER_RECLAIM,	"I_DEFER_RECLAIM"}	\
 	)
 
 /* enums need to be exported to user space */
-- 
2.51.0

[PATCH v2 4/5] fs: Add throttling to deferred inode reclaim

[Jan Kara]

Deferring difficult inode reclaim from prune_icache_sb() to a workqueue
removes the natural feedback loop of blocking tasks in direct reclaim
until they make space for new allocations. This can result in the list
of deferred inodes to grow beyond any bounds and possibly push the
machine to a reclaim storm or OOM.

Add a throttling mechanism slowing down tasks in
mark_inode_reclaim_deferred() if the number of deferred inodes to
reclaim for a superblock grows over limit. We measure average time it
takes to reclaim inode on deferred list for a superblock and block tasks
proportionally to that.

Signed-off-by: Jan Kara <jack@suse.cz>
---
 fs/inode.c                       | 102 +++++++++++++++++++++++++++----
 include/linux/fs/super_types.h   |   2 +
 include/trace/events/writeback.h |  51 ++++++++++++++++
 3 files changed, 144 insertions(+), 11 deletions(-)

diff --git a/fs/inode.c b/fs/inode.c
index e902095cf39b..0491300c5ccb 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -1113,23 +1113,53 @@ static int get_inode_reclaim_batch(int *id, struct list_head *list)
 	return count;
 }
 
+static void inode_reclaim_update_stat(struct super_block *sb, unsigned int n,
+				      u64 start)
+{
+	u64 delay;
+
+	if (!sb)
+		return;
+
+	delay = div_u64(ktime_get_ns() - start, n);
+	/*
+	 * Smooth delay updates with exponential moving average. Updates can
+	 * get lost if workers race but we don't really care.
+	 */
+	WRITE_ONCE(sb->s_deferred_reclaim_delay,
+		   (63 * READ_ONCE(sb->s_deferred_reclaim_delay) + delay) / 64);
+
+	trace_inode_reclaim_update_stat(sb, n, delay,
+				READ_ONCE(sb->s_deferred_reclaim_delay));
+
+	/*
+	 * The elevated s_deferred_reclaim_count keeps sb alive until we drop
+	 * it
+	 */
+	if (!atomic_sub_return(n, &sb->s_deferred_reclaim_count))
+		wake_up_var(&sb->s_deferred_reclaim_count);
+}
 
 static void inode_reclaim_deferred(struct work_struct *work)
 {
 	struct inode *inode;
 	LIST_HEAD(inode_batch);
+	u64 start;
+	int id, count;
+	struct super_block *sb;
+
 	/*
 	 * We start with the list corresponding to the work but rolling a dice
 	 * would work as well
 	 */
-	int id = work - deferred_reclaim.work;
+	id = work - deferred_reclaim.work;
 
 	while (1) {
 		if (!get_inode_reclaim_batch(&id, &inode_batch))
 			break;
-		while (!list_empty(&inode_batch)) {
-			struct super_block *sb;
+		sb = NULL;
 
+		while (!list_empty(&inode_batch)) {
 			/*
 			 * inode_batch list is private and I_FREEING flags
 			 * protect us from anybody else trying to remove the
@@ -1137,17 +1167,20 @@ static void inode_reclaim_deferred(struct work_struct *work)
 			 */
 			inode = list_first_entry(&inode_batch, struct inode,
 						 i_lru);
+			if (inode->i_sb != sb) {
+				inode_reclaim_update_stat(sb, count, start);
+
+				sb = inode->i_sb;
+				count = 0;
+				start = ktime_get_ns();
+			}
+			count++;
 			list_del_init(&inode->i_lru);
-			sb = inode->i_sb;
 			evict(inode);
-			/*
-			 * The elevated s_deferred_reclaim_count keeps sb alive
-			 * until we drop it
-			 */
-			if (atomic_dec_and_test(&sb->s_deferred_reclaim_count))
-				wake_up_var(&sb->s_deferred_reclaim_count);
 			cond_resched();
 		}
+
+		inode_reclaim_update_stat(sb, count, start);
 	}
 }
 
@@ -1183,14 +1216,61 @@ static void __init deferred_reclaim_init(void)
 	}
 }
 
+/*
+ * Size of deferred reclaim list from which we start throttling tasks creating
+ * inodes marked for deferred reclaim.
+ */
+#define INODE_DEFERRED_RECLAIM_LIMIT 8192
+
+static void throttle_inode_deferred_reclaim(struct inode *inode)
+{
+	struct super_block *sb = inode->i_sb;
+	unsigned int len;
+
+	/*
+	 * If inodes with deferred reclaim are accumulating too much, slow down
+	 * tasks creating them. This doesn't provide any kind of guarantee on
+	 * the length of the deferred list since lots of inodes with
+	 * I_DEFER_RECLAIM can be already present in the inode cache and we
+	 * have no control when they reach the deferred list. But if the
+	 * pressure on the deferred list is sustained, the balance should
+	 * eventually be established.
+	 */
+	len = atomic_read(&sb->s_deferred_reclaim_count);
+	if (len > INODE_DEFERRED_RECLAIM_LIMIT) {
+		u64 delay = READ_ONCE(sb->s_deferred_reclaim_delay);
+
+		if (!delay)
+			return;
+		/*
+		 * Scale the delay based on how much we exceed the limit. Wait
+		 * at most 4x as long as estimated time to reclaim the inode.
+		 */
+		len = min(len, 5 * INODE_DEFERRED_RECLAIM_LIMIT);
+		delay = div_u64(delay * (len - INODE_DEFERRED_RECLAIM_LIMIT),
+				INODE_DEFERRED_RECLAIM_LIMIT);
+		trace_mark_inode_reclaim_deferred_throttle(inode, len, delay);
+
+		schedule_timeout_killable(nsecs_to_jiffies(delay));
+	}
+}
+
 void mark_inode_reclaim_deferred(struct inode *inode)
 {
+	bool throttle = false;
+
 	if (inode_state_read_once(inode) & I_DEFER_RECLAIM)
 		return;
 
 	spin_lock(&inode->i_lock);
-	inode_state_set(inode, I_DEFER_RECLAIM);
+	if (!(inode_state_read(inode) & I_DEFER_RECLAIM)) {
+		inode_state_set(inode, I_DEFER_RECLAIM);
+		throttle = true;
+	}
 	spin_unlock(&inode->i_lock);
+
+	if (throttle)
+		throttle_inode_deferred_reclaim(inode);
 }
 EXPORT_SYMBOL_GPL(mark_inode_reclaim_deferred);
 
diff --git a/include/linux/fs/super_types.h b/include/linux/fs/super_types.h
index a41bd57fc2e6..6db5df6fd35f 100644
--- a/include/linux/fs/super_types.h
+++ b/include/linux/fs/super_types.h
@@ -234,6 +234,8 @@ struct super_block {
 
 	/* Number of inodes queued for deferred reclaim */
 	atomic_t				s_deferred_reclaim_count;
+	/* Average time to reclaim one deferred inode */
+	u64					s_deferred_reclaim_delay;
 
 	/* Number of inodes with nlink == 0 but still referenced */
 	atomic_long_t				s_remove_count;
diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
index 4a9cfe458820..aec38eaeb10e 100644
--- a/include/trace/events/writeback.h
+++ b/include/trace/events/writeback.h
@@ -881,6 +881,57 @@ DEFINE_EVENT(writeback_inode_template, sb_clear_inode_writeback,
 	TP_ARGS(inode)
 );
 
+TRACE_EVENT(inode_reclaim_update_stat,
+	TP_PROTO(
+		struct super_block *sb,
+		unsigned int n,
+		u64 batch_delay,
+		u64 avg_delay
+	),
+	TP_ARGS(sb, n, batch_delay, avg_delay),
+
+	TP_STRUCT__entry(
+		__field(dev_t,		dev)
+		__field(unsigned int,	n)
+		__field(u64,		batch_delay)
+		__field(u64,		avg_delay)
+	),
+
+	TP_fast_assign(
+		__entry->dev = sb->s_dev;
+		__entry->n = n;
+		__entry->batch_delay = batch_delay;
+		__entry->avg_delay = avg_delay;
+	),
+
+	TP_printk("dev %d,%d batch size %u batch delay %llu ns avg delay %llu ns",
+		  MAJOR(__entry->dev), MINOR(__entry->dev), __entry->n,
+		  __entry->batch_delay, __entry->avg_delay)
+);
+
+TRACE_EVENT(mark_inode_reclaim_deferred_throttle,
+	TP_PROTO(struct inode *inode, unsigned int len, u64 delay),
+	TP_ARGS(inode, len, delay),
+
+	TP_STRUCT__entry(
+		__field(u64,		ino)
+		__field(dev_t,		dev)
+		__field(unsigned int,	len)
+		__field(u64,		delay)
+	),
+
+	TP_fast_assign(
+		__entry->ino = inode->i_ino;
+		__entry->dev = inode->i_sb->s_dev;
+		__entry->len = len;
+		__entry->delay = delay;
+	),
+
+	TP_printk("dev %d,%d ino %llu deferred list len %u delay %llu ns",
+		  MAJOR(__entry->dev), MINOR(__entry->dev),
+		  __entry->ino, __entry->len, __entry->delay)
+);
+
 #endif /* _TRACE_WRITEBACK_H */
 
 /* This part must be outside protection */
-- 
2.51.0

[PATCH v2 5/5] ext4: Defer inode reclaim if it has preallocations

[Jan Kara]

When we have to free preallocations during inode eviction, we need to
load block bitmaps and run transaction to modify them. This takes time
and also requires GFP_NOFAIL allocations. Mark inodes with preallocated
blocks as needing offloading of inode reclaim to a workqueue so that we
don't block reclaim for long and potentially deadlock MM subsystem.

Signed-off-by: Jan Kara <jack@suse.cz>
---
 fs/ext4/mballoc.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index 06171a11db12..4891f252105e 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -3115,6 +3115,13 @@ ext4_mb_regular_allocator(struct ext4_allocation_context *ac)
 	if (ac->ac_prefetch_nr)
 		ext4_mb_prefetch_fini(sb, ac->ac_prefetch_grp, ac->ac_prefetch_nr);
 
+	/*
+	 * Freeing preallocations requires loading bitmaps and running
+	 * transactions. Defer inode reclaim to a workqueue.
+	 */
+	if (!RB_EMPTY_ROOT(&EXT4_I(ac->ac_inode)->i_prealloc_node))
+		mark_inode_reclaim_deferred(ac->ac_inode);
+
 	return err;
 }
 
-- 
2.51.0

Re: [PATCH 0/5 v2] fs: Deferred inode reclaim

[Theodore Tso]

On Fri, Sep 11, 2026 at 10:51:36AM -0500, Jan Kara wrote:
> 
> after a long pause here is a second revision of my patches implementing
> deferred inode reclaim to deal with MM warnings due to GFP_NOFAIL allocations
> from reclaim paths. This happens because to reclaim some inodes, filesystems
> have to do IO including complex operations using journalling and forward
> progress of these depends on successful memory allocations (which is impossible
> to guarantee from reclaim context).

Thanks for the patch set!  I wonder if it is worthwhile to wait for
inode reclaim before freezing a file system for suspend.  On the plus
side, it leaves the file system in a more cleaned-up state, and
reduces the amount of memory that might need to be written in a
hibernation scenario.  On the other side of the argument there's more
opportunity for freeze/suspend deadlocks, and more complexity.  So
maybe it's not worth it.

What do folks think?

						- Ted

Re: [PATCH 0/5 v2] fs: Deferred inode reclaim

[Darrick J. Wong]

On Fri, Sep 11, 2026 at 10:55:53AM -0400, Theodore Tso wrote:
> On Fri, Sep 11, 2026 at 10:51:36AM -0500, Jan Kara wrote:
> > 
> > after a long pause here is a second revision of my patches implementing
> > deferred inode reclaim to deal with MM warnings due to GFP_NOFAIL allocations
> > from reclaim paths. This happens because to reclaim some inodes, filesystems
> > have to do IO including complex operations using journalling and forward
> > progress of these depends on successful memory allocations (which is impossible
> > to guarantee from reclaim context).
> 
> Thanks for the patch set!  I wonder if it is worthwhile to wait for
> inode reclaim before freezing a file system for suspend.  On the plus
> side, it leaves the file system in a more cleaned-up state, and
> reduces the amount of memory that might need to be written in a
> hibernation scenario.  On the other side of the argument there's more
> opportunity for freeze/suspend deadlocks, and more complexity.  So
> maybe it's not worth it.
> 
> What do folks think?

xfs doesn't flush any pending inode reclaim work on freeze, since the
log will be dirty and log recovery (if the frozen fs image gets
remounted) will take care of that.  Freezing to make a snapshot is fast,
mounting the snapshot is not.

--D

> 						- Ted
> 

Re: [PATCH 0/5 v2] fs: Deferred inode reclaim

[Christian Brauner]

> Hello,
> 
> after a long pause here is a second revision of my patches implementing
> deferred inode reclaim to deal with MM warnings due to GFP_NOFAIL allocations
> from reclaim paths. This happens because to reclaim some inodes, filesystems
> have to do IO including complex operations using journalling and forward
> progress of these depends on successful memory allocations (which is impossible
> to guarantee from reclaim context).
> 
> This time I have fully tested the patch set. In particular it survives full
> fstests run on ext4 both as is as well as when marking all loaded ext4 inodes
> for deferred reclaim to stress the deferred reclaim paths. I have also tested
> (when marking all inodes for deferred reclaim) the patches by creating a memcg
> with 128m memory limit and then scanning from 10 processes 600k inodes in total
> which heavily exercises deferred inode reclaim paths. This also verified
> efficiency of throttling of marking of inodes for deferred reclaim. Basically
> we cannot guarantee any particular limit on the number of inodes queued for
> deferred reclaim as that is bound only by their number in memory and memory
> pressure. But as the number of queued inodes grows, the tasks creating them
> get slowed down so eventually some equilibrium is hit. In my VM this was
> at ~10k inodes (only slightly above the 8k limit when throttling kicks in)
> but this all very much depends on the reclaim pressure, speed of deferred
> reclaim etc. so I don't think this is some representative number.
> 
> The first patch is a pure fix for a problem I've hit when running fstests.  The
> second patch deals with lazy timestamp updates which I've decided to handle
> better directly inside the writeback infrastructure instead of deferring
> reclaim. The remaining patches implement the inode reclaim deferal. Ext4 use of
> deferred inode reclaim is there mostly as a demonstration. Other filesystems
> need to determine which inodes need deferred reclaim and mark them as such
> which generally requires good internal knowledge of the filesystem. I'm hoping
> that once the infrastructure is there, fs developers prompted by MM warnings
> will start using it :). I know for a fact that besides ext4 e.g. btrfs is
> hitting MM warnings in inode reclaim as well.

This all looks worthwhile to me!
Note that Sashiko is complaining about long-term stalls introduced by
this patchset. If this is intended or bs then please let us know.

-- 

Re: [PATCH v2 4/5] fs: Add throttling to deferred inode reclaim

[Andreas Gruenbacher]

On Fri, Sep 11, 2026 at 10:51 AM Jan Kara <jack@suse.cz> wrote:
> Deferring difficult inode reclaim from prune_icache_sb() to a workqueue
> removes the natural feedback loop of blocking tasks in direct reclaim
> until they make space for new allocations. This can result in the list
> of deferred inodes to grow beyond any bounds and possibly push the
> machine to a reclaim storm or OOM.
>
> Add a throttling mechanism slowing down tasks in
> mark_inode_reclaim_deferred() if the number of deferred inodes to
> reclaim for a superblock grows over limit. We measure average time it
> takes to reclaim inode on deferred list for a superblock and block tasks
> proportionally to that.
>
> Signed-off-by: Jan Kara <jack@suse.cz>
> ---
>  fs/inode.c                       | 102 +++++++++++++++++++++++++++----
>  include/linux/fs/super_types.h   |   2 +
>  include/trace/events/writeback.h |  51 ++++++++++++++++
>  3 files changed, 144 insertions(+), 11 deletions(-)
>
> diff --git a/fs/inode.c b/fs/inode.c
> index e902095cf39b..0491300c5ccb 100644
> --- a/fs/inode.c
> +++ b/fs/inode.c
> @@ -1113,23 +1113,53 @@ static int get_inode_reclaim_batch(int *id, struct list_head *list)
>         return count;
>  }
>
> +static void inode_reclaim_update_stat(struct super_block *sb, unsigned int n,
> +                                     u64 start)
> +{
> +       u64 delay;
> +
> +       if (!sb)
> +               return;
> +
> +       delay = div_u64(ktime_get_ns() - start, n);
> +       /*
> +        * Smooth delay updates with exponential moving average. Updates can
> +        * get lost if workers race but we don't really care.
> +        */
> +       WRITE_ONCE(sb->s_deferred_reclaim_delay,
> +                  (63 * READ_ONCE(sb->s_deferred_reclaim_delay) + delay) / 64);
> +
> +       trace_inode_reclaim_update_stat(sb, n, delay,
> +                               READ_ONCE(sb->s_deferred_reclaim_delay));
> +
> +       /*
> +        * The elevated s_deferred_reclaim_count keeps sb alive until we drop
> +        * it
> +        */
> +       if (!atomic_sub_return(n, &sb->s_deferred_reclaim_count))
> +               wake_up_var(&sb->s_deferred_reclaim_count);
> +}
>
>  static void inode_reclaim_deferred(struct work_struct *work)
>  {
>         struct inode *inode;
>         LIST_HEAD(inode_batch);
> +       u64 start;
> +       int id, count;
> +       struct super_block *sb;
> +
>         /*
>          * We start with the list corresponding to the work but rolling a dice
>          * would work as well
>          */
> -       int id = work - deferred_reclaim.work;
> +       id = work - deferred_reclaim.work;
>
>         while (1) {
>                 if (!get_inode_reclaim_batch(&id, &inode_batch))
>                         break;
> -               while (!list_empty(&inode_batch)) {
> -                       struct super_block *sb;
> +               sb = NULL;
>
> +               while (!list_empty(&inode_batch)) {
>                         /*
>                          * inode_batch list is private and I_FREEING flags
>                          * protect us from anybody else trying to remove the
> @@ -1137,17 +1167,20 @@ static void inode_reclaim_deferred(struct work_struct *work)
>                          */
>                         inode = list_first_entry(&inode_batch, struct inode,
>                                                  i_lru);
> +                       if (inode->i_sb != sb) {
> +                               inode_reclaim_update_stat(sb, count, start);
> +
> +                               sb = inode->i_sb;
> +                               count = 0;
> +                               start = ktime_get_ns();
> +                       }
> +                       count++;
>                         list_del_init(&inode->i_lru);
> -                       sb = inode->i_sb;
>                         evict(inode);
> -                       /*
> -                        * The elevated s_deferred_reclaim_count keeps sb alive
> -                        * until we drop it
> -                        */
> -                       if (atomic_dec_and_test(&sb->s_deferred_reclaim_count))
> -                               wake_up_var(&sb->s_deferred_reclaim_count);
>                         cond_resched();
>                 }
> +
> +               inode_reclaim_update_stat(sb, count, start);
>         }
>  }
>
> @@ -1183,14 +1216,61 @@ static void __init deferred_reclaim_init(void)
>         }
>  }
>
> +/*
> + * Size of deferred reclaim list from which we start throttling tasks creating
> + * inodes marked for deferred reclaim.
> + */
> +#define INODE_DEFERRED_RECLAIM_LIMIT 8192
> +
> +static void throttle_inode_deferred_reclaim(struct inode *inode)
> +{
> +       struct super_block *sb = inode->i_sb;
> +       unsigned int len;
> +
> +       /*
> +        * If inodes with deferred reclaim are accumulating too much, slow down
> +        * tasks creating them. This doesn't provide any kind of guarantee on
> +        * the length of the deferred list since lots of inodes with
> +        * I_DEFER_RECLAIM can be already present in the inode cache and we
> +        * have no control when they reach the deferred list. But if the
> +        * pressure on the deferred list is sustained, the balance should
> +        * eventually be established.
> +        */
> +       len = atomic_read(&sb->s_deferred_reclaim_count);
> +       if (len > INODE_DEFERRED_RECLAIM_LIMIT) {
> +               u64 delay = READ_ONCE(sb->s_deferred_reclaim_delay);
> +
> +               if (!delay)
> +                       return;
> +               /*
> +                * Scale the delay based on how much we exceed the limit. Wait
> +                * at most 4x as long as estimated time to reclaim the inode.
> +                */
> +               len = min(len, 5 * INODE_DEFERRED_RECLAIM_LIMIT);
> +               delay = div_u64(delay * (len - INODE_DEFERRED_RECLAIM_LIMIT),
> +                               INODE_DEFERRED_RECLAIM_LIMIT);
> +               trace_mark_inode_reclaim_deferred_throttle(inode, len, delay);
> +
> +               schedule_timeout_killable(nsecs_to_jiffies(delay));
> +       }
> +}
> +
>  void mark_inode_reclaim_deferred(struct inode *inode)
>  {
> +       bool throttle = false;
> +
>         if (inode_state_read_once(inode) & I_DEFER_RECLAIM)
>                 return;
>
>         spin_lock(&inode->i_lock);
> -       inode_state_set(inode, I_DEFER_RECLAIM);
> +       if (!(inode_state_read(inode) & I_DEFER_RECLAIM)) {
> +               inode_state_set(inode, I_DEFER_RECLAIM);
> +               throttle = true;
> +       }
>         spin_unlock(&inode->i_lock);
> +
> +       if (throttle)
> +               throttle_inode_deferred_reclaim(inode);

I would expect the throttling to happen when too many inodes marked
I_DEFER_RECLAIM are evicted, but not here. In my use case [*], inodes
would only require deferred reclaim intermittently, while they have
outstanding revokes. Once there are no more outstanding revokes, those
inodes can just be evicted directly.

[*] https://lore.kernel.org/linux-fsdevel/CAHc6FU4Rf9-r8m8SSb=e0k0zyoFhQAdjupHMWvT6rStBshntug@mail.gmail.com/T/#t

>  }
>  EXPORT_SYMBOL_GPL(mark_inode_reclaim_deferred);
>
> diff --git a/include/linux/fs/super_types.h b/include/linux/fs/super_types.h
> index a41bd57fc2e6..6db5df6fd35f 100644
> --- a/include/linux/fs/super_types.h
> +++ b/include/linux/fs/super_types.h
> @@ -234,6 +234,8 @@ struct super_block {
>
>         /* Number of inodes queued for deferred reclaim */
>         atomic_t                                s_deferred_reclaim_count;
> +       /* Average time to reclaim one deferred inode */
> +       u64                                     s_deferred_reclaim_delay;
>
>         /* Number of inodes with nlink == 0 but still referenced */
>         atomic_long_t                           s_remove_count;
> diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
> index 4a9cfe458820..aec38eaeb10e 100644
> --- a/include/trace/events/writeback.h
> +++ b/include/trace/events/writeback.h
> @@ -881,6 +881,57 @@ DEFINE_EVENT(writeback_inode_template, sb_clear_inode_writeback,
>         TP_ARGS(inode)
>  );
>
> +TRACE_EVENT(inode_reclaim_update_stat,
> +       TP_PROTO(
> +               struct super_block *sb,
> +               unsigned int n,
> +               u64 batch_delay,
> +               u64 avg_delay
> +       ),
> +       TP_ARGS(sb, n, batch_delay, avg_delay),
> +
> +       TP_STRUCT__entry(
> +               __field(dev_t,          dev)
> +               __field(unsigned int,   n)
> +               __field(u64,            batch_delay)
> +               __field(u64,            avg_delay)
> +       ),
> +
> +       TP_fast_assign(
> +               __entry->dev = sb->s_dev;
> +               __entry->n = n;
> +               __entry->batch_delay = batch_delay;
> +               __entry->avg_delay = avg_delay;
> +       ),
> +
> +       TP_printk("dev %d,%d batch size %u batch delay %llu ns avg delay %llu ns",
> +                 MAJOR(__entry->dev), MINOR(__entry->dev), __entry->n,
> +                 __entry->batch_delay, __entry->avg_delay)
> +);
> +
> +TRACE_EVENT(mark_inode_reclaim_deferred_throttle,
> +       TP_PROTO(struct inode *inode, unsigned int len, u64 delay),
> +       TP_ARGS(inode, len, delay),
> +
> +       TP_STRUCT__entry(
> +               __field(u64,            ino)
> +               __field(dev_t,          dev)
> +               __field(unsigned int,   len)
> +               __field(u64,            delay)
> +       ),
> +
> +       TP_fast_assign(
> +               __entry->ino = inode->i_ino;
> +               __entry->dev = inode->i_sb->s_dev;
> +               __entry->len = len;
> +               __entry->delay = delay;
> +       ),
> +
> +       TP_printk("dev %d,%d ino %llu deferred list len %u delay %llu ns",
> +                 MAJOR(__entry->dev), MINOR(__entry->dev),
> +                 __entry->ino, __entry->len, __entry->delay)
> +);
> +
>  #endif /* _TRACE_WRITEBACK_H */
>
>  /* This part must be outside protection */
> --
> 2.51.0
>
>

Thanks,
Andreas

Re: [PATCH v2 4/5] fs: Add throttling to deferred inode reclaim

[Jan Kara]

On Wed 23-09-26 23:24:48, Andreas Gruenbacher wrote:
> On Fri, Sep 11, 2026 at 10:51 AM Jan Kara <jack@suse.cz> wrote:
> > Deferring difficult inode reclaim from prune_icache_sb() to a workqueue
> > removes the natural feedback loop of blocking tasks in direct reclaim
> > until they make space for new allocations. This can result in the list
> > of deferred inodes to grow beyond any bounds and possibly push the
> > machine to a reclaim storm or OOM.
> >
> > Add a throttling mechanism slowing down tasks in
> > mark_inode_reclaim_deferred() if the number of deferred inodes to
> > reclaim for a superblock grows over limit. We measure average time it
> > takes to reclaim inode on deferred list for a superblock and block tasks
> > proportionally to that.
> >
> > Signed-off-by: Jan Kara <jack@suse.cz>
...
> >  void mark_inode_reclaim_deferred(struct inode *inode)
> >  {
> > +       bool throttle = false;
> > +
> >         if (inode_state_read_once(inode) & I_DEFER_RECLAIM)
> >                 return;
> >
> >         spin_lock(&inode->i_lock);
> > -       inode_state_set(inode, I_DEFER_RECLAIM);
> > +       if (!(inode_state_read(inode) & I_DEFER_RECLAIM)) {
> > +               inode_state_set(inode, I_DEFER_RECLAIM);
> > +               throttle = true;
> > +       }
> >         spin_unlock(&inode->i_lock);
> > +
> > +       if (throttle)
> > +               throttle_inode_deferred_reclaim(inode);
> 
> I would expect the throttling to happen when too many inodes marked
> I_DEFER_RECLAIM are evicted, but not here. In my use case [*], inodes
> would only require deferred reclaim intermittently, while they have
> outstanding revokes. Once there are no more outstanding revokes, those
> inodes can just be evicted directly.
> 
> [*] https://lore.kernel.org/linux-fsdevel/CAHc6FU4Rf9-r8m8SSb=e0k0zyoFhQAdjupHMWvT6rStBshntug@mail.gmail.com/T/#t

Throttling during eviction doesn't make sense - you are punishing the wrong
task (the one trying to free memory, not the one actually making that
memory hard to reclaim).

Also note that the throttling kicks in only when the queue of inodes that
we want to reclaim but worker didn't get to them yet grows above threshold.
This means there's significant memory pressure, significant amount of
deferred inodes, *and* workers cannot keep up. Throttling marking of inodes
for deferred reclaim is the best thing I was able to come up with to
eventually reduce the pressure on the cleanup workers. But I'm open to
better ideas.

I understand your objection about the need for deferal being only temporary
but let's keep that discussion to the other thread.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH 0/5 v2] fs: Deferred inode reclaim

[Jan Kara]

On Thu 17-09-26 11:56:17, Christian Brauner wrote:
> > Hello,
> > 
> > after a long pause here is a second revision of my patches implementing
> > deferred inode reclaim to deal with MM warnings due to GFP_NOFAIL allocations
> > from reclaim paths. This happens because to reclaim some inodes, filesystems
> > have to do IO including complex operations using journalling and forward
> > progress of these depends on successful memory allocations (which is impossible
> > to guarantee from reclaim context).
> > 
> > This time I have fully tested the patch set. In particular it survives full
> > fstests run on ext4 both as is as well as when marking all loaded ext4 inodes
> > for deferred reclaim to stress the deferred reclaim paths. I have also tested
> > (when marking all inodes for deferred reclaim) the patches by creating a memcg
> > with 128m memory limit and then scanning from 10 processes 600k inodes in total
> > which heavily exercises deferred inode reclaim paths. This also verified
> > efficiency of throttling of marking of inodes for deferred reclaim. Basically
> > we cannot guarantee any particular limit on the number of inodes queued for
> > deferred reclaim as that is bound only by their number in memory and memory
> > pressure. But as the number of queued inodes grows, the tasks creating them
> > get slowed down so eventually some equilibrium is hit. In my VM this was
> > at ~10k inodes (only slightly above the 8k limit when throttling kicks in)
> > but this all very much depends on the reclaim pressure, speed of deferred
> > reclaim etc. so I don't think this is some representative number.
> > 
> > The first patch is a pure fix for a problem I've hit when running fstests.  The
> > second patch deals with lazy timestamp updates which I've decided to handle
> > better directly inside the writeback infrastructure instead of deferring
> > reclaim. The remaining patches implement the inode reclaim deferal. Ext4 use of
> > deferred inode reclaim is there mostly as a demonstration. Other filesystems
> > need to determine which inodes need deferred reclaim and mark them as such
> > which generally requires good internal knowledge of the filesystem. I'm hoping
> > that once the infrastructure is there, fs developers prompted by MM warnings
> > will start using it :). I know for a fact that besides ext4 e.g. btrfs is
> > hitting MM warnings in inode reclaim as well.
> 
> This all looks worthwhile to me!
> Note that Sashiko is complaining about long-term stalls introduced by
> this patchset. If this is intended or bs then please let us know.

Yeah, there are some good suggestions from Sashiko. I'll address them and
respin.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH v2 0/5] fs: Deferred inode reclaim

[Matthias Goergens]

Hi Honza,

I ran v2 against its base in QEMU, on syzbot's disk image with a config
close to syzbot's (KASAN, lockdep):

- Lazytime churn on ext4 under memory pressure: the PF_MEMALLOC warning
  fired with the sync_lazytime() stack in all 5 base runs, four times
  from kswapd, and in none of 5 runs with the series, which evicted as
  many inodes in reclaim context without reading ext4 metadata there.
  The workload also runs drop_caches=2 under memalloc_noreclaim_save()
  from a debug patch.

- Hibernation cycles with /sys/power/pm_test set to devices under that
  load, with inodes that 5/5 defers queued in each cycle, and with
  freeze_filesystems 0 and 1: no freezer timeouts and no hang involving
  the queue.  The stalls in throttle_direct_reclaim() from
  hibernate_preallocate_memory(), and lockdep's warning in
  __set_task_frozen() with freeze_filesystems=1, happen on the base
  kernel too.

- One path the series does not cover, see below: with the series
  applied, unlinked ext4 inodes under an overlay are still deleted in
  reclaim context, and the warning fired in 2 of 3 runs, as without it.

The reproducers and the debug patch are at
https://github.com/matthiasgoergens/linux/tree/deferred-reclaim-repro
and I can post them here if you prefer.  The Tested-by below covers the
lazytime and hibernation runs.

In 3/5:

> @@ -988,13 +1013,27 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
>  	inode_state_set(inode, I_FREEING);
> -	list_lru_isolate_move(lru, &inode->i_lru, freeable);
> +	/* Inode will take long time to cleanup. Offload that to worker. */
> +	if (inode_state_read(inode) & I_DEFER_RECLAIM) {
> +		list_lru_isolate_move(lru, &inode->i_lru, &lists->deferred);

This is the only place I_DEFER_RECLAIM queues an inode, also with
Andreas's simplification folded in.  An inode that ->drop_inode() drops
on its last iput(), which for ext4 means every unlinked inode, never
gets here: iput_final() calls evict() directly, in whatever context the
iput() came from.

syzbot hits that under reclaim with overlayfs on top of ext4, in the
same bug as the lazytime trace in your 2/5:

  https://syzkaller.appspot.com/bug?extid=7f94fe3ce0f6613e12b8
  https://syzkaller.appspot.com/text?tag=CrashReport&x=12764a15580000
  https://syzkaller.appspot.com/text?tag=CrashReport&x=13530905580000

Direct reclaim prunes overlayfs dentries, and dropping the overlay inode
puts the last reference to an unlinked lower ext4 inode, so
ext4_evict_inode() runs its deletion path under PF_MEMALLOC.  My
reproducer above does the same: stat files through an overlay on ext4,
unlink them in the lower or upper directory, then apply memory pressure.

Is that path meant to be covered?  Handing such inodes to the queue from
iput_final() under PF_MEMALLOC, as gfs2_drop_inode() does for gfs2,
mostly just delays things: blocks and quota come back later, lookups by
inode number wait in __wait_on_freeing_inode(), and umount would need
the inode counted in s_deferred_reclaim_count, as 3/5 does for its own.
But it does not work as is with freezing: ext4_evict_inode() takes
sb_start_intwrite() on the deletion path, so a worker evicting an
unlinked inode of a frozen filesystem sleeps until the thaw, holding a
batch of up to 16 inodes taken from any queue, and enough of those stop
the queue.  XFS keeps inodegc per mount and stops it in xfs_fs_sync_fs()
before SB_FREEZE_FS; this queue would need something like that before it
could take unlinked inodes.

On Ted's question (https://lore.kernel.org/r/aqQV6Uwma_KjUH0O@mit.edu):
with this WQ_FREEZABLE queue I would not wait either.  A non-freezable
queue would need draining before anything is frozen, because an eviction
after the hibernation snapshot writes to disk behind the image, and
draining later can wait on kjournald2 or on sb_start_intwrite().
Freezing the workers with the kernel threads avoids both.

One catch: a work item already blocked on something frozen earlier keeps
freeze_workqueues_busy() true until the freezer times out, and the
suspend or hibernation is aborted.  With /sys/power/freeze_filesystems=1
the filesystems are frozen before the freezer runs, so an eviction
waiting in sb_start_intwrite() is enough.  ext4 only takes that on the
deletion path, so 5/5 is not affected, but unlinked inodes routed
through the queue would be.

Tested-by: Matthias Goergens <matthias.goergens@gmail.com>

Thanks,
Matthias

Re: [PATCH v2 0/5] fs: Deferred inode reclaim

[Jan Kara]

Hi Matthias!

On Sat 26-09-26 10:48:29, Matthias Goergens wrote:
> I ran v2 against its base in QEMU, on syzbot's disk image with a config
> close to syzbot's (KASAN, lockdep):
> 
> - Lazytime churn on ext4 under memory pressure: the PF_MEMALLOC warning
>   fired with the sync_lazytime() stack in all 5 base runs, four times
>   from kswapd, and in none of 5 runs with the series, which evicted as
>   many inodes in reclaim context without reading ext4 metadata there.
>   The workload also runs drop_caches=2 under memalloc_noreclaim_save()
>   from a debug patch.
> 
> - Hibernation cycles with /sys/power/pm_test set to devices under that
>   load, with inodes that 5/5 defers queued in each cycle, and with
>   freeze_filesystems 0 and 1: no freezer timeouts and no hang involving
>   the queue.  The stalls in throttle_direct_reclaim() from
>   hibernate_preallocate_memory(), and lockdep's warning in
>   __set_task_frozen() with freeze_filesystems=1, happen on the base
>   kernel too.
> 
> - One path the series does not cover, see below: with the series
>   applied, unlinked ext4 inodes under an overlay are still deleted in
>   reclaim context, and the warning fired in 2 of 3 runs, as without it.
> 
> The reproducers and the debug patch are at
> https://github.com/matthiasgoergens/linux/tree/deferred-reclaim-repro
> and I can post them here if you prefer.  The Tested-by below covers the
> lazytime and hibernation runs.

Thanks for testing!

> > @@ -988,13 +1013,27 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
> >  	inode_state_set(inode, I_FREEING);
> > -	list_lru_isolate_move(lru, &inode->i_lru, freeable);
> > +	/* Inode will take long time to cleanup. Offload that to worker. */
> > +	if (inode_state_read(inode) & I_DEFER_RECLAIM) {
> > +		list_lru_isolate_move(lru, &inode->i_lru, &lists->deferred);
> 
> This is the only place I_DEFER_RECLAIM queues an inode, also with
> Andreas's simplification folded in.  An inode that ->drop_inode() drops
> on its last iput(), which for ext4 means every unlinked inode, never
> gets here: iput_final() calls evict() directly, in whatever context the
> iput() came from.
> 
> syzbot hits that under reclaim with overlayfs on top of ext4, in the
> same bug as the lazytime trace in your 2/5:
> 
>   https://syzkaller.appspot.com/bug?extid=7f94fe3ce0f6613e12b8
>   https://syzkaller.appspot.com/text?tag=CrashReport&x=12764a15580000
>   https://syzkaller.appspot.com/text?tag=CrashReport&x=13530905580000
> 
> Direct reclaim prunes overlayfs dentries, and dropping the overlay inode
> puts the last reference to an unlinked lower ext4 inode, so
> ext4_evict_inode() runs its deletion path under PF_MEMALLOC.  My
> reproducer above does the same: stat files through an overlay on ext4,
> unlink them in the lower or upper directory, then apply memory pressure.

This is an interesting case. When I was writing the patch set I was
thinking whether unlinked inodes should be marked for deferred reclaim and
decided for "no" - unlink or file close should be happening from context
where synchronous ->evict() should be possible (and is desirable!
Offloading to async context always has these inconvenient questions about
throttling associated with it and space being freed later than userspace
expects it etc.). But I didn't think of overlayfs - I assume syzkaller
unlinked upper layer inode under the overlayfs' hands - which can indeed
lead to overlayfs dentry being effectively the last holder of inode
reference and thus responsible for inode deletion.

Inherently it is kind of nasty that overlayfs dentries can be the last
thing protecting the inode from deletion but at the same time I don't see
an easy way around it. Amir, maybe you have an idea?

Anyway, if we cannot change how overlayfs behaves, what we could do is to
mark unlinked inodes for deferred reclaim but offload the reclaim from
iput_final() only if PF_MEMALLOC is set in VFS. At least then we know we
are in async context anyway so nobody can be relying on when exactly the
inode gets deleted when somebody left it to memory reclaim.

> Is that path meant to be covered?  Handing such inodes to the queue from
> iput_final() under PF_MEMALLOC, as gfs2_drop_inode() does for gfs2,
> mostly just delays things: blocks and quota come back later, lookups by
> inode number wait in __wait_on_freeing_inode(), and umount would need
> the inode counted in s_deferred_reclaim_count, as 3/5 does for its own.
> But it does not work as is with freezing: ext4_evict_inode() takes
> sb_start_intwrite() on the deletion path, so a worker evicting an
> unlinked inode of a frozen filesystem sleeps until the thaw, holding a
> batch of up to 16 inodes taken from any queue, and enough of those stop
> the queue.  XFS keeps inodegc per mount and stops it in xfs_fs_sync_fs()
> before SB_FREEZE_FS; this queue would need something like that before it
> could take unlinked inodes.
> 
> On Ted's question (https://lore.kernel.org/r/aqQV6Uwma_KjUH0O@mit.edu):
> with this WQ_FREEZABLE queue I would not wait either.  A non-freezable
> queue would need draining before anything is frozen, because an eviction
> after the hibernation snapshot writes to disk behind the image, and
> draining later can wait on kjournald2 or on sb_start_intwrite().
> Freezing the workers with the kernel threads avoids both.
> 
> One catch: a work item already blocked on something frozen earlier keeps
> freeze_workqueues_busy() true until the freezer times out, and the
> suspend or hibernation is aborted.  With /sys/power/freeze_filesystems=1
> the filesystems are frozen before the freezer runs, so an eviction
> waiting in sb_start_intwrite() is enough.  ext4 only takes that on the
> deletion path, so 5/5 is not affected, but unlinked inodes routed
> through the queue would be.

You (and Ted) have a good point here that if ->evict() is going to write to
the filesystem (and currently ext4 doesn't offload such case but I
would like the mechanism to be usable in such situations since e.g. XFS
needs it even if we solve the overlayfs case differently) we need to
be careful about interactions with filesystem freezing / system suspend.
Also in that case Sashiko's concern about possible deadlocks of throttling
with ext4 transaction machinery becomes real.

So as the patch set currently is, it is correct but I think we need to
somehow deal with the overlayfs issues you reported as well. At this point
I don't have a clear picture in my mind how to cleanly deal with the
lockups that would be created by it. I'll think about it more and also
maybe Amir will come up with some clever idea from overlayfs side :).

> Tested-by: Matthias Goergens <matthias.goergens@gmail.com>

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH v2 4/5] fs: Add throttling to deferred inode reclaim

[Andreas Gruenbacher]

On Thu, Sep 24, 2026 at 7:09 PM Jan Kara <jack@suse.cz> wrote:
> On Wed 23-09-26 23:24:48, Andreas Gruenbacher wrote:
> > On Fri, Sep 11, 2026 at 10:51 AM Jan Kara <jack@suse.cz> wrote:
> > > Deferring difficult inode reclaim from prune_icache_sb() to a workqueue
> > > removes the natural feedback loop of blocking tasks in direct reclaim
> > > until they make space for new allocations. This can result in the list
> > > of deferred inodes to grow beyond any bounds and possibly push the
> > > machine to a reclaim storm or OOM.
> > >
> > > Add a throttling mechanism slowing down tasks in
> > > mark_inode_reclaim_deferred() if the number of deferred inodes to
> > > reclaim for a superblock grows over limit. We measure average time it
> > > takes to reclaim inode on deferred list for a superblock and block tasks
> > > proportionally to that.
> > >
> > > Signed-off-by: Jan Kara <jack@suse.cz>
> ...
> > >  void mark_inode_reclaim_deferred(struct inode *inode)
> > >  {
> > > +       bool throttle = false;
> > > +
> > >         if (inode_state_read_once(inode) & I_DEFER_RECLAIM)
> > >                 return;
> > >
> > >         spin_lock(&inode->i_lock);
> > > -       inode_state_set(inode, I_DEFER_RECLAIM);
> > > +       if (!(inode_state_read(inode) & I_DEFER_RECLAIM)) {
> > > +               inode_state_set(inode, I_DEFER_RECLAIM);
> > > +               throttle = true;
> > > +       }
> > >         spin_unlock(&inode->i_lock);
> > > +
> > > +       if (throttle)
> > > +               throttle_inode_deferred_reclaim(inode);
> >
> > I would expect the throttling to happen when too many inodes marked
> > I_DEFER_RECLAIM are evicted, but not here. In my use case [*], inodes
> > would only require deferred reclaim intermittently, while they have
> > outstanding revokes. Once there are no more outstanding revokes, those
> > inodes can just be evicted directly.
> >
> > [*] https://lore.kernel.org/linux-fsdevel/CAHc6FU4Rf9-r8m8SSb=e0k0zyoFhQAdjupHMWvT6rStBshntug@mail.gmail.com/T/#t
>
> Throttling during eviction doesn't make sense - you are punishing the wrong
> task (the one trying to free memory, not the one actually making that
> memory hard to reclaim).
>
> Also note that the throttling kicks in only when the queue of inodes that
> we want to reclaim but worker didn't get to them yet grows above threshold.
> This means there's significant memory pressure, significant amount of
> deferred inodes, *and* workers cannot keep up. Throttling marking of inodes
> for deferred reclaim is the best thing I was able to come up with to
> eventually reduce the pressure on the cleanup workers. But I'm open to
> better ideas.

Alright, I guess this should work. But the change from throttling in
prune_icache_sb() to trottling in mark_inode_reclaim_deferred() should
be rationalized in the commit message of this patch.

> I understand your objection about the need for deferal being only temporary
> but let's keep that discussion to the other thread.

Maybe we can get away without it.

Thanks,
Andreas

Re: [PATCH v2 0/5] fs: Deferred inode reclaim

[Amir Goldstein]

On Tue, Sep 29, 2026 at 12:38 PM Jan Kara <jack@suse.cz> wrote:
>
> Hi Matthias!
>
> On Sat 26-09-26 10:48:29, Matthias Goergens wrote:
> > I ran v2 against its base in QEMU, on syzbot's disk image with a config
> > close to syzbot's (KASAN, lockdep):
> >
> > - Lazytime churn on ext4 under memory pressure: the PF_MEMALLOC warning
> >   fired with the sync_lazytime() stack in all 5 base runs, four times
> >   from kswapd, and in none of 5 runs with the series, which evicted as
> >   many inodes in reclaim context without reading ext4 metadata there.
> >   The workload also runs drop_caches=2 under memalloc_noreclaim_save()
> >   from a debug patch.
> >
> > - Hibernation cycles with /sys/power/pm_test set to devices under that
> >   load, with inodes that 5/5 defers queued in each cycle, and with
> >   freeze_filesystems 0 and 1: no freezer timeouts and no hang involving
> >   the queue.  The stalls in throttle_direct_reclaim() from
> >   hibernate_preallocate_memory(), and lockdep's warning in
> >   __set_task_frozen() with freeze_filesystems=1, happen on the base
> >   kernel too.
> >
> > - One path the series does not cover, see below: with the series
> >   applied, unlinked ext4 inodes under an overlay are still deleted in
> >   reclaim context, and the warning fired in 2 of 3 runs, as without it.
> >
> > The reproducers and the debug patch are at
> > https://github.com/matthiasgoergens/linux/tree/deferred-reclaim-repro
> > and I can post them here if you prefer.  The Tested-by below covers the
> > lazytime and hibernation runs.
>
> Thanks for testing!
>
> > > @@ -988,13 +1013,27 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
> > >     inode_state_set(inode, I_FREEING);
> > > -   list_lru_isolate_move(lru, &inode->i_lru, freeable);
> > > +   /* Inode will take long time to cleanup. Offload that to worker. */
> > > +   if (inode_state_read(inode) & I_DEFER_RECLAIM) {
> > > +           list_lru_isolate_move(lru, &inode->i_lru, &lists->deferred);
> >
> > This is the only place I_DEFER_RECLAIM queues an inode, also with
> > Andreas's simplification folded in.  An inode that ->drop_inode() drops
> > on its last iput(), which for ext4 means every unlinked inode, never
> > gets here: iput_final() calls evict() directly, in whatever context the
> > iput() came from.
> >
> > syzbot hits that under reclaim with overlayfs on top of ext4, in the
> > same bug as the lazytime trace in your 2/5:
> >
> >   https://syzkaller.appspot.com/bug?extid=7f94fe3ce0f6613e12b8
> >   https://syzkaller.appspot.com/text?tag=CrashReport&x=12764a15580000
> >   https://syzkaller.appspot.com/text?tag=CrashReport&x=13530905580000
> >
> > Direct reclaim prunes overlayfs dentries, and dropping the overlay inode
> > puts the last reference to an unlinked lower ext4 inode, so
> > ext4_evict_inode() runs its deletion path under PF_MEMALLOC.  My
> > reproducer above does the same: stat files through an overlay on ext4,
> > unlink them in the lower or upper directory, then apply memory pressure.
>
> This is an interesting case. When I was writing the patch set I was
> thinking whether unlinked inodes should be marked for deferred reclaim and
> decided for "no" - unlink or file close should be happening from context
> where synchronous ->evict() should be possible (and is desirable!
> Offloading to async context always has these inconvenient questions about
> throttling associated with it and space being freed later than userspace
> expects it etc.). But I didn't think of overlayfs - I assume syzkaller
> unlinked upper layer inode under the overlayfs' hands - which can indeed
> lead to overlayfs dentry being effectively the last holder of inode
> reference and thus responsible for inode deletion.
>
> Inherently it is kind of nasty that overlayfs dentries can be the last
> thing protecting the inode from deletion but at the same time I don't see
> an easy way around it. Amir, maybe you have an idea?
>

No, not really. If you could use the ovl_destroy_inode() to mark the
ext4 inodes for deferred reclaim, but with current design this does not
seem possible.

FYI, I assume the same situation could unfold from dropping an
inode with cachefiles backing inode.

I don't think the issue exists with FUSE passthrough backing inode
at the moment, but I do have a plan for a similar construct of
backing inode that detaches on evict of fuse inode as in overlayfs.

Thanks,
Amir.
overlayfs