From: Tejun Heo <tj@kernel.org>
To: Jan Kara <jack@suse.cz>
Cc: Christian Brauner <brauner@kernel.org>, linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH 3/4] writeback: Avoid excessively long inode switching times
Date: Tue, 9 Sep 2025 06:56:59 -1000 [thread overview]
Message-ID: <aMBcW0yZJcuXpjW0@slm.duckdns.org> (raw)
In-Reply-To: <20250909144400.2901-7-jack@suse.cz>
On Tue, Sep 09, 2025 at 04:44:04PM +0200, Jan Kara wrote:
> With lazytime mount option enabled we can be switching many dirty inodes
> on cgroup exit to the parent cgroup. The numbers observed in practice
> when systemd slice of a large cron job exits can easily reach hundreds
> of thousands or millions. The logic in inode_do_switch_wbs() which sorts
> the inode into appropriate place in b_dirty list of the target wb
> however has linear complexity in the number of dirty inodes thus overall
> time complexity of switching all the inodes is quadratic leading to
> workers being pegged for hours consuming 100% of the CPU and switching
> inodes to the parent wb.
>
> Simple reproducer of the issue:
> FILES=10000
> # Filesystem mounted with lazytime mount option
> MNT=/mnt/
> echo "Creating files and switching timestamps"
> for (( j = 0; j < 50; j ++ )); do
> mkdir $MNT/dir$j
> for (( i = 0; i < $FILES; i++ )); do
> echo "foo" >$MNT/dir$j/file$i
> done
> touch -a -t 202501010000 $MNT/dir$j/file*
> done
> wait
> echo "Syncing and flushing"
> sync
> echo 3 >/proc/sys/vm/drop_caches
>
> echo "Reading all files from a cgroup"
> mkdir /sys/fs/cgroup/unified/mycg1 || exit
> echo $$ >/sys/fs/cgroup/unified/mycg1/cgroup.procs || exit
> for (( j = 0; j < 50; j ++ )); do
> cat /mnt/dir$j/file* >/dev/null &
> done
> wait
> echo "Switching wbs"
> # Now rmdir the cgroup after the script exits
>
> We need to maintain b_dirty list ordering to keep writeback happy so
> instead of sorting inode into appropriate place just append it at the
> end of the list and clobber dirtied_time_when. This may result in inode
> writeback starting later after cgroup switch however cgroup switches are
> rare so it shouldn't matter much. Since the cgroup had write access to
> the inode, there are no practical concerns of the possible DoS issues.
>
> Signed-off-by: Jan Kara <jack@suse.cz>
Acked-by: Tejun Heo <tj@kernel.org>
Thanks.
--
tejun
next prev parent reply other threads:[~2025-09-09 16:57 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-09 14:44 [PATCH 0/4] writeback: Avoid lockups when switching inodes Jan Kara
2025-09-09 14:44 ` [PATCH 1/4] writeback: Avoid contention on wb->list_lock " Jan Kara
2025-09-09 16:52 ` Tejun Heo
2025-09-10 8:19 ` Jan Kara
2025-09-10 17:10 ` Tejun Heo
2025-09-11 11:30 ` Jan Kara
2025-09-11 12:01 ` Jan Kara
2025-09-12 10:39 ` Jan Kara
2025-09-09 14:44 ` [PATCH 2/4] writeback: Avoid softlockup when switching many inodes Jan Kara
2025-09-09 16:55 ` Tejun Heo
2025-09-09 14:44 ` [PATCH 3/4] writeback: Avoid excessively long inode switching times Jan Kara
2025-09-09 16:56 ` Tejun Heo [this message]
2025-09-09 14:44 ` [PATCH 4/4] writeback: Add tracepoint to track pending inode switches Jan Kara
2025-09-09 16:57 ` Tejun Heo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aMBcW0yZJcuXpjW0@slm.duckdns.org \
--to=tj@kernel.org \
--cc=brauner@kernel.org \
--cc=jack@suse.cz \
--cc=linux-fsdevel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).