linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Tejun Heo <tj@kernel.org>
To: Jan Kara <jack@suse.cz>
Cc: Christian Brauner <brauner@kernel.org>, linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH 3/4] writeback: Avoid excessively long inode switching times
Date: Tue, 9 Sep 2025 06:56:59 -1000	[thread overview]
Message-ID: <aMBcW0yZJcuXpjW0@slm.duckdns.org> (raw)
In-Reply-To: <20250909144400.2901-7-jack@suse.cz>

On Tue, Sep 09, 2025 at 04:44:04PM +0200, Jan Kara wrote:
> With lazytime mount option enabled we can be switching many dirty inodes
> on cgroup exit to the parent cgroup. The numbers observed in practice
> when systemd slice of a large cron job exits can easily reach hundreds
> of thousands or millions. The logic in inode_do_switch_wbs() which sorts
> the inode into appropriate place in b_dirty list of the target wb
> however has linear complexity in the number of dirty inodes thus overall
> time complexity of switching all the inodes is quadratic leading to
> workers being pegged for hours consuming 100% of the CPU and switching
> inodes to the parent wb.
> 
> Simple reproducer of the issue:
>   FILES=10000
>   # Filesystem mounted with lazytime mount option
>   MNT=/mnt/
>   echo "Creating files and switching timestamps"
>   for (( j = 0; j < 50; j ++ )); do
>       mkdir $MNT/dir$j
>       for (( i = 0; i < $FILES; i++ )); do
>           echo "foo" >$MNT/dir$j/file$i
>       done
>       touch -a -t 202501010000 $MNT/dir$j/file*
>   done
>   wait
>   echo "Syncing and flushing"
>   sync
>   echo 3 >/proc/sys/vm/drop_caches
> 
>   echo "Reading all files from a cgroup"
>   mkdir /sys/fs/cgroup/unified/mycg1 || exit
>   echo $$ >/sys/fs/cgroup/unified/mycg1/cgroup.procs || exit
>   for (( j = 0; j < 50; j ++ )); do
>       cat /mnt/dir$j/file* >/dev/null &
>   done
>   wait
>   echo "Switching wbs"
>   # Now rmdir the cgroup after the script exits
> 
> We need to maintain b_dirty list ordering to keep writeback happy so
> instead of sorting inode into appropriate place just append it at the
> end of the list and clobber dirtied_time_when. This may result in inode
> writeback starting later after cgroup switch however cgroup switches are
> rare so it shouldn't matter much. Since the cgroup had write access to
> the inode, there are no practical concerns of the possible DoS issues.
> 
> Signed-off-by: Jan Kara <jack@suse.cz>

Acked-by: Tejun Heo <tj@kernel.org>

Thanks.

-- 
tejun

  reply	other threads:[~2025-09-09 16:57 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-09-09 14:44 [PATCH 0/4] writeback: Avoid lockups when switching inodes Jan Kara
2025-09-09 14:44 ` [PATCH 1/4] writeback: Avoid contention on wb->list_lock " Jan Kara
2025-09-09 16:52   ` Tejun Heo
2025-09-10  8:19     ` Jan Kara
2025-09-10 17:10       ` Tejun Heo
2025-09-11 11:30         ` Jan Kara
2025-09-11 12:01           ` Jan Kara
2025-09-12 10:39             ` Jan Kara
2025-09-09 14:44 ` [PATCH 2/4] writeback: Avoid softlockup when switching many inodes Jan Kara
2025-09-09 16:55   ` Tejun Heo
2025-09-09 14:44 ` [PATCH 3/4] writeback: Avoid excessively long inode switching times Jan Kara
2025-09-09 16:56   ` Tejun Heo [this message]
2025-09-09 14:44 ` [PATCH 4/4] writeback: Add tracepoint to track pending inode switches Jan Kara
2025-09-09 16:57   ` Tejun Heo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aMBcW0yZJcuXpjW0@slm.duckdns.org \
    --to=tj@kernel.org \
    --cc=brauner@kernel.org \
    --cc=jack@suse.cz \
    --cc=linux-fsdevel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).