[PATCH] fs: rotate inodes whose page cache could not be dropped

[PATCH] fs: rotate inodes whose page cache could not be dropped

[Nikolaos Karaolidis]

When inode_lru_isolate() drops the page cache of an unused inode, it
has to unlock the list, so it returns LRU_RETRY and the walk restarts
at the head of the list, where the inode still is. If
invalidate_mapping_pages() could not empty the mapping, the restarted
walk strips the same inode again, and so on until nr_to_walk runs out.
Every walk of that list then ends on this inode, and nothing behind it
is reclaimed for as long as its cache stays.

Before commit 2a0629834cd8 ("vfs: Don't evict inode under the inode lru
traversing context") the walker pinned the inode with a reference, and
dropping it in iput_final() set I_REFERENCED, so the restarted walk
rotated the inode to the tail. Set I_REFERENCED again when the mapping
is still populated after the invalidation, in the same i_lock section
that clears I_LRU_ISOLATING. inode_unpin_lru_isolating() now expects
i_lock to be held, like inode_pin_lru_isolating().

On CONFIG_HIGHMEM kernels this can happen to any unused inode: a folio
is left behind if it is locked, under writeback, spliced into a pipe,
or still in another CPU's LRU-add batch after being read. Without
HIGHMEM it happens to DAX inodes whose only entry is at index 0, which
mapping_shrinkable() lets onto the LRU and invalidate_mapping_pages()
never removes.

Fixes: 2a0629834cd8 ("vfs: Don't evict inode under the inode lru traversing context")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Nikolaos Karaolidis <nick@karaolidis.com>
---

Found while working on reclaim of inodes pinned by page cache in
ZONE_MOVABLE.

Tested in QEMU/KVM on master af32da41b032 (v7.3-rc6-232), with lockdep
and KASAN enabled. A small init program makes one unused inode whose
page can't be invalidated, lets it reach the head of the inode LRU,
opens 2000 other files, does one "echo 2 > /proc/sys/vm/drop_caches"
and counts the inodes left via umount; kprobes count the
invalidate_mapping_pages() calls:

                              without patch           with patch
  i386 HIGHMEM, file read     2000 of 2000 left,      none left,
  on another CPU              10300 strips, 1 inode   4 strips
  i386 HIGHMEM, page          same                    same
  spliced into a pipe
  x86_64, ext4 -o dax,        same                    same
  file mmap()ed
  controls (no leftover,      none left               none left
  shadow entry at index 0)

2a0629834cd8^ behaves like the patched kernel, 2a0629834cd8 like
master. On 5.15.y x86_64, where every unused inode with page cache is
on the LRU, the read-on-another-CPU case blocks the walk as well.

With "echo 64M > /sys/fs/cgroup/memory.reclaim" instead of
drop_caches, the i386 pipe case does 262450 strips and frees nothing
without the patch, 40 strips and frees everything with it. In the DAX
case that write takes ~400ms with the patch instead of ~4ms:
invalidate_mapping_pages() counts the DAX entry it leaves behind as
reclaimed, so unpatched, the 64M are "reclaimed" by re-stripping one
inode in a tight loop while nothing is freed.

The reproducer is a single C file; I can post it.

An LLM was used to check the analysis against the code, write the
reproducer and draft the changelog; I have reviewed all of it.

 fs/inode.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/fs/inode.c b/fs/inode.c
index a9d37be390a1..1614a97ce953 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -593,12 +593,11 @@ static void inode_pin_lru_isolating(struct inode *inode)
 
 static void inode_unpin_lru_isolating(struct inode *inode)
 {
-	spin_lock(&inode->i_lock);
+	lockdep_assert_held(&inode->i_lock);
 	WARN_ON(!(inode_state_read(inode) & I_LRU_ISOLATING));
 	inode_state_clear(inode, I_LRU_ISOLATING);
 	/* Called with inode->i_lock which ensures memory ordering. */
 	inode_wake_up_bit(inode, __I_LRU_ISOLATING);
-	spin_unlock(&inode->i_lock);
 }
 
 static void inode_wait_for_lru_isolating(struct inode *inode)
@@ -981,7 +980,13 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
 		else
 			__count_vm_events(PGINODESTEAL, reap);
 		mm_account_reclaimed_pages(reap);
+
+		spin_lock(&inode->i_lock);
+		/* The walk restarts here: don't retry leftovers right away */
+		if (!mapping_empty(&inode->i_data))
+			inode_state_set(inode, I_REFERENCED);
 		inode_unpin_lru_isolating(inode);
+		spin_unlock(&inode->i_lock);
 		return LRU_RETRY;
 	}
 

base-commit: af32da41b0327b9c6a37856ba82b6760d6c8d10e
-- 
2.55.0

Re: [PATCH] fs: rotate inodes whose page cache could not be dropped

[Zhihao Cheng]

在 2026/10/16 18:25, Nikolaos Karaolidis 写道:
> When inode_lru_isolate() drops the page cache of an unused inode, it
> has to unlock the list, so it returns LRU_RETRY and the walk restarts
> at the head of the list, where the inode still is. If
> invalidate_mapping_pages() could not empty the mapping, the restarted
> walk strips the same inode again, and so on until nr_to_walk runs out.
> Every walk of that list then ends on this inode, and nothing behind it
> is reclaimed for as long as its cache stays.
[...]
>   static void inode_wait_for_lru_isolating(struct inode *inode)
> @@ -981,7 +980,13 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
>   		else
>   			__count_vm_events(PGINODESTEAL, reap);
>   		mm_account_reclaimed_pages(reap);
> +
> +		spin_lock(&inode->i_lock);
> +		/* The walk restarts here: don't retry leftovers right away */
> +		if (!mapping_empty(&inode->i_data))
> +			inode_state_set(inode, I_REFERENCED);
>   		inode_unpin_lru_isolating(inode);
> +		spin_unlock(&inode->i_lock);
>   		return LRU_RETRY;
>   	}

Hi Nikolaos,
I find a possible BUG_ON if this patch applied, details shown as:
    P1                          P2
prune_icache_sb
  lock(lru->lock)
  inode_lru_isolate
   if (!mapping_empty(&inode->i_data))
    unlock(lru->lock)
    spin_lock(&inode->i_lock)
    inode_state_set(inode, I_REFERENCED)
    inode_unpin_lru_isolating
    spin_unlock(&inode->i_lock)
                          unlink
                           iget_locked // find_inode_fast gets the inode
                           iput
                            iput_final
                             drop = 1
                             evict
                              BUG_ON(inode_state_read_once(inode) != 
(I_FREEING | I_CLEAR))  // inode state with I_REFERENCED

Could that be possible?

Re: [PATCH] fs: rotate inodes whose page cache could not be dropped

[Nick Karaolidis]

On Saturday, October 17th, 2026 at 8:43 AM, Zhihao Cheng <chengzhihao1@huawei.com> wrote:

> Hi Nikolaos,
> I find a possible BUG_ON if this patch applied, details shown as:
>     P1                          P2
> prune_icache_sb
>   lock(lru->lock)
>   inode_lru_isolate
>    if (!mapping_empty(&inode->i_data))
>     unlock(lru->lock)
>     spin_lock(&inode->i_lock)
>     inode_state_set(inode, I_REFERENCED)
>     inode_unpin_lru_isolating
>     spin_unlock(&inode->i_lock)
>                           unlink
>                            iget_locked // find_inode_fast gets the inode
>                            iput
>                             iput_final
>                              drop = 1
>                              evict
>                               BUG_ON(inode_state_read_once(inode) !=
> (I_FREEING | I_CLEAR))  // inode state with I_REFERENCED
> 
> Could that be possible?

Hi Zhihao,

Thanks for looking at this. I don't think that BUG_ON can trigger.

- The patch sets I_REFERENCED in the same i_lock section that clears
  I_LRU_ISOLATING, and evict() waits in inode_wait_for_lru_isolating()
  before calling ->evict_inode(), so the walker can only set the flag
  before the filesystem's eviction runs.

- clear_inode(), which every ->evict_inode() calls, sets i_state to
  I_FREEING | I_CLEAR.

- Evicting an inode that has I_REFERENCED set already happens.
  iput_final() sets it through __inode_lru_list_add(inode, true) when
  the inode is already on the LRU.

Did I miss a path that checks i_state before clear_inode()?

Thanks,
Nikolaos

Re: [PATCH] fs: rotate inodes whose page cache could not be dropped

[Jan Kara]

On Fri 16-10-26 10:25:15, Nikolaos Karaolidis wrote:
> When inode_lru_isolate() drops the page cache of an unused inode, it
> has to unlock the list, so it returns LRU_RETRY and the walk restarts
> at the head of the list, where the inode still is. If
> invalidate_mapping_pages() could not empty the mapping, the restarted
> walk strips the same inode again, and so on until nr_to_walk runs out.
> Every walk of that list then ends on this inode, and nothing behind it
> is reclaimed for as long as its cache stays.
> 
> Before commit 2a0629834cd8 ("vfs: Don't evict inode under the inode lru
> traversing context") the walker pinned the inode with a reference, and
> dropping it in iput_final() set I_REFERENCED, so the restarted walk
> rotated the inode to the tail. Set I_REFERENCED again when the mapping
> is still populated after the invalidation, in the same i_lock section
> that clears I_LRU_ISOLATING. inode_unpin_lru_isolating() now expects
> i_lock to be held, like inode_pin_lru_isolating().
> 
> On CONFIG_HIGHMEM kernels this can happen to any unused inode: a folio
> is left behind if it is locked, under writeback, spliced into a pipe,
> or still in another CPU's LRU-add batch after being read. Without
> HIGHMEM it happens to DAX inodes whose only entry is at index 0, which
> mapping_shrinkable() lets onto the LRU and invalidate_mapping_pages()
> never removes.
> 
> Fixes: 2a0629834cd8 ("vfs: Don't evict inode under the inode lru traversing context")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Nikolaos Karaolidis <nick@karaolidis.com>

Looks sensible. Feel free to add:

Reviewed-by: Jan Kara <jack@suse.cz>

								Honza


> diff --git a/fs/inode.c b/fs/inode.c
> index a9d37be390a1..1614a97ce953 100644
> --- a/fs/inode.c
> +++ b/fs/inode.c
> @@ -593,12 +593,11 @@ static void inode_pin_lru_isolating(struct inode *inode)
>  
>  static void inode_unpin_lru_isolating(struct inode *inode)
>  {
> -	spin_lock(&inode->i_lock);
> +	lockdep_assert_held(&inode->i_lock);
>  	WARN_ON(!(inode_state_read(inode) & I_LRU_ISOLATING));
>  	inode_state_clear(inode, I_LRU_ISOLATING);
>  	/* Called with inode->i_lock which ensures memory ordering. */
>  	inode_wake_up_bit(inode, __I_LRU_ISOLATING);
> -	spin_unlock(&inode->i_lock);
>  }
>  
>  static void inode_wait_for_lru_isolating(struct inode *inode)
> @@ -981,7 +980,13 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
>  		else
>  			__count_vm_events(PGINODESTEAL, reap);
>  		mm_account_reclaimed_pages(reap);
> +
> +		spin_lock(&inode->i_lock);
> +		/* The walk restarts here: don't retry leftovers right away */
> +		if (!mapping_empty(&inode->i_data))
> +			inode_state_set(inode, I_REFERENCED);
>  		inode_unpin_lru_isolating(inode);
> +		spin_unlock(&inode->i_lock);
>  		return LRU_RETRY;
>  	}
>  
> 
> base-commit: af32da41b0327b9c6a37856ba82b6760d6c8d10e
> -- 
> 2.55.0
> 
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH] fs: rotate inodes whose page cache could not be dropped

[Zhihao Cheng]

在 2026/10/19 16:50, Nick Karaolidis 写道:
> On Saturday, October 17th, 2026 at 8:43 AM, Zhihao Cheng <chengzhihao1@huawei.com> wrote:
> 
>> Hi Nikolaos,
>> I find a possible BUG_ON if this patch applied, details shown as:
>>      P1                          P2
>> prune_icache_sb
>>    lock(lru->lock)
>>    inode_lru_isolate
>>     if (!mapping_empty(&inode->i_data))
>>      unlock(lru->lock)
>>      spin_lock(&inode->i_lock)
>>      inode_state_set(inode, I_REFERENCED)
>>      inode_unpin_lru_isolating
>>      spin_unlock(&inode->i_lock)
>>                            unlink
>>                             iget_locked // find_inode_fast gets the inode
>>                             iput
>>                              iput_final
>>                               drop = 1
>>                               evict
>>                                BUG_ON(inode_state_read_once(inode) !=
>> (I_FREEING | I_CLEAR))  // inode state with I_REFERENCED
>>
>> Could that be possible?
> 
> Hi Zhihao,
> 
> Thanks for looking at this. I don't think that BUG_ON can trigger.
> 
> - The patch sets I_REFERENCED in the same i_lock section that clears
>    I_LRU_ISOLATING, and evict() waits in inode_wait_for_lru_isolating()
>    before calling ->evict_inode(), so the walker can only set the flag
>    before the filesystem's eviction runs.
> 
> - clear_inode(), which every ->evict_inode() calls, sets i_state to
>    I_FREEING | I_CLEAR.
OK, I missunderstand the realization of 'inode_state_assign_raw(inode, 
I_FREEING | I_CLEAR)' in the clear_inode().
> 
> - Evicting an inode that has I_REFERENCED set already happens.
>    iput_final() sets it through __inode_lru_list_add(inode, true) when
>    the inode is already on the LRU.
> 
> Did I miss a path that checks i_state before clear_inode()?
> 
> Thanks,
> Nikolaos
> 
> .
> 

Re: [PATCH] fs: rotate inodes whose page cache could not be dropped

[Zhihao Cheng]

在 2026/10/16 18:25, Nikolaos Karaolidis 写道:
> When inode_lru_isolate() drops the page cache of an unused inode, it
> has to unlock the list, so it returns LRU_RETRY and the walk restarts
> at the head of the list, where the inode still is. If
> invalidate_mapping_pages() could not empty the mapping, the restarted
> walk strips the same inode again, and so on until nr_to_walk runs out.
> Every walk of that list then ends on this inode, and nothing behind it
> is reclaimed for as long as its cache stays.
> 
> Before commit 2a0629834cd8 ("vfs: Don't evict inode under the inode lru
> traversing context") the walker pinned the inode with a reference, and
> dropping it in iput_final() set I_REFERENCED, so the restarted walk
> rotated the inode to the tail. Set I_REFERENCED again when the mapping
> is still populated after the invalidation, in the same i_lock section
> that clears I_LRU_ISOLATING. inode_unpin_lru_isolating() now expects
> i_lock to be held, like inode_pin_lru_isolating().
> 
> On CONFIG_HIGHMEM kernels this can happen to any unused inode: a folio
> is left behind if it is locked, under writeback, spliced into a pipe,
> or still in another CPU's LRU-add batch after being read. Without
> HIGHMEM it happens to DAX inodes whose only entry is at index 0, which
> mapping_shrinkable() lets onto the LRU and invalidate_mapping_pages()
> never removes.
> 
> Fixes: 2a0629834cd8 ("vfs: Don't evict inode under the inode lru traversing context")
> Cc: stable@vger.kernel.org
> Assisted-by: LLM
> Signed-off-by: Nikolaos Karaolidis <nick@karaolidis.com>
> ---
> 
> Found while working on reclaim of inodes pinned by page cache in
> ZONE_MOVABLE.
> 
> Tested in QEMU/KVM on master af32da41b032 (v7.3-rc6-232), with lockdep
> and KASAN enabled. A small init program makes one unused inode whose
> page can't be invalidated, lets it reach the head of the inode LRU,
> opens 2000 other files, does one "echo 2 > /proc/sys/vm/drop_caches"
> and counts the inodes left via umount; kprobes count the
> invalidate_mapping_pages() calls:
> 
>                                without patch           with patch
>    i386 HIGHMEM, file read     2000 of 2000 left,      none left,
>    on another CPU              10300 strips, 1 inode   4 strips
>    i386 HIGHMEM, page          same                    same
>    spliced into a pipe
>    x86_64, ext4 -o dax,        same                    same
>    file mmap()ed
>    controls (no leftover,      none left               none left
>    shadow entry at index 0)
> 
> 2a0629834cd8^ behaves like the patched kernel, 2a0629834cd8 like
> master. On 5.15.y x86_64, where every unused inode with page cache is
> on the LRU, the read-on-another-CPU case blocks the walk as well.
> 
> With "echo 64M > /sys/fs/cgroup/memory.reclaim" instead of
> drop_caches, the i386 pipe case does 262450 strips and frees nothing
> without the patch, 40 strips and frees everything with it. In the DAX
> case that write takes ~400ms with the patch instead of ~4ms:
> invalidate_mapping_pages() counts the DAX entry it leaves behind as
> reclaimed, so unpatched, the 64M are "reclaimed" by re-stripping one
> inode in a tight loop while nothing is freed.
> 
> The reproducer is a single C file; I can post it.
> 
> An LLM was used to check the analysis against the code, write the
> reproducer and draft the changelog; I have reviewed all of it.
> 
>   fs/inode.c | 9 +++++++--
>   1 file changed, 7 insertions(+), 2 deletions(-)
> 
Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
> diff --git a/fs/inode.c b/fs/inode.c
> index a9d37be390a1..1614a97ce953 100644
> --- a/fs/inode.c
> +++ b/fs/inode.c
> @@ -593,12 +593,11 @@ static void inode_pin_lru_isolating(struct inode *inode)
>   
>   static void inode_unpin_lru_isolating(struct inode *inode)
>   {
> -	spin_lock(&inode->i_lock);
> +	lockdep_assert_held(&inode->i_lock);
>   	WARN_ON(!(inode_state_read(inode) & I_LRU_ISOLATING));
>   	inode_state_clear(inode, I_LRU_ISOLATING);
>   	/* Called with inode->i_lock which ensures memory ordering. */
>   	inode_wake_up_bit(inode, __I_LRU_ISOLATING);
> -	spin_unlock(&inode->i_lock);
>   }
>   
>   static void inode_wait_for_lru_isolating(struct inode *inode)
> @@ -981,7 +980,13 @@ static enum lru_status inode_lru_isolate(struct list_head *item,
>   		else
>   			__count_vm_events(PGINODESTEAL, reap);
>   		mm_account_reclaimed_pages(reap);
> +
> +		spin_lock(&inode->i_lock);
> +		/* The walk restarts here: don't retry leftovers right away */
> +		if (!mapping_empty(&inode->i_data))
> +			inode_state_set(inode, I_REFERENCED);
>   		inode_unpin_lru_isolating(inode);
> +		spin_unlock(&inode->i_lock);
>   		return LRU_RETRY;
>   	}
>   
> 
> base-commit: af32da41b0327b9c6a37856ba82b6760d6c8d10e
> 

Re: [PATCH] fs: rotate inodes whose page cache could not be dropped

[Christian Brauner]

On Fri, 16 Oct 2026 10:25:15 +0000, Nikolaos Karaolidis wrote:
> fs: rotate inodes whose page cache could not be dropped

Applied to the vfs.fixes branch of the vfs/vfs.git tree.
Patches in the vfs.fixes branch should appear in linux-next soon.

Please report any outstanding bugs that were missed during review in a
new review to the original patch series allowing us to drop it.

It's encouraged to provide Acked-bys and Reviewed-bys even though the
patch has now been applied. If possible patch trailers will be updated.

Note that commit hashes shown below are subject to change due to rebase,
trailer updates or similar. If in doubt, please check the listed branch.

tree:   https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
branch: vfs.fixes

[1/1] fs: rotate inodes whose page cache could not be dropped
      https://git.kernel.org/vfs/vfs/c/efe3ca110fdb