From: Eric Biggers <ebiggers@kernel.org>
To: Christoph Hellwig <hch@lst.de>
Cc: Al Viro <viro@zeniv.linux.org.uk>,
Christian Brauner <brauner@kernel.org>, Jan Kara <jack@suse.cz>,
David Sterba <dsterba@suse.com>, Theodore Ts'o <tytso@mit.edu>,
Jaegeuk Kim <jaegeuk@kernel.org>, Chao Yu <chao@kernel.org>,
Andrey Albershteyn <aalbersh@redhat.com>,
Matthew Wilcox <willy@infradead.org>,
linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org,
linux-ext4@vger.kernel.org,
linux-f2fs-devel@lists.sourceforge.net, fsverity@lists.linux.dev
Subject: Re: [PATCH 08/15] fsverity: kick off hash readahead at data I/O submission time
Date: Wed, 28 Jan 2026 14:56:02 -0800 [thread overview]
Message-ID: <20260128225602.GB2024@quark> (raw)
In-Reply-To: <20260128152630.627409-9-hch@lst.de>
On Wed, Jan 28, 2026 at 04:26:20PM +0100, Christoph Hellwig wrote:
> Currently all reads of the fsverity hashes is kicked off from the data
> I/O completion handler, leading to needlessly dependent I/O. This is
> worked around a bit by performing readahead on the level 0 nodes, but
> still fairly ineffective.
>
> Switch to a model where the ->read_folio and ->readahead methods instead
> kick off explicit readahead of the fsverity hashed so they are usually
> available at I/O completion time.
>
> For 64k sequential reads on my test VM this improves read performance
> from 2.4GB/s - 2.6GB/s to 3.5GB/s - 3.9GB/s. The improvements for
> random reads are likely to be even bigger.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Acked-by: David Sterba <dsterba@suse.com> [btrfs]
Unfortunately, this patch causes recursive down_read() of
address_space::invalidate_lock. How was this meant to work?
[ 20.563185] ============================================
[ 20.564179] WARNING: possible recursive locking detected
[ 20.565170] 6.19.0-rc7-00041-g7bd72c6393ab #2 Not tainted
[ 20.566180] --------------------------------------------
[ 20.567169] cmp/2320 is trying to acquire lock:
[ 20.568019] ffff888108465030 (mapping.invalidate_lock#2){++++}-{4:4}, at: page_cache_ra_unbounded+0x6f/0x280
[ 20.569828]
[ 20.569828] but task is already holding lock:
[ 20.570914] ffff888108465030 (mapping.invalidate_lock#2){++++}-{4:4}, at: page_cache_ra_unbounded+0x6f/0x280
[ 20.572739]
[ 20.572739] other info that might help us debug this:
[ 20.573938] Possible unsafe locking scenario:
[ 20.573938]
[ 20.575042] CPU0
[ 20.575522] ----
[ 20.576003] lock(mapping.invalidate_lock#2);
[ 20.576849] lock(mapping.invalidate_lock#2);
[ 20.577698]
[ 20.577698] *** DEADLOCK ***
[ 20.577698]
[ 20.578795] May be due to missing lock nesting notation
[ 20.578795]
[ 20.580045] 1 lock held by cmp/2320:
[ 20.580726] #0: ffff888108465030 (mapping.invalidate_lock#2){++++}-{4:4}, at: page_cache_ra_unbounded+0x6f/0x20
[ 20.582596]
[ 20.582596] stack backtrace:
[ 20.583428] CPU: 0 UID: 0 PID: 2320 Comm: cmp Not tainted 6.19.0-rc7-00041-g7bd72c6393ab #2 PREEMPT(none)
[ 20.583433] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Arch Linux 1.17.0-2-2 04/01/2014
[ 20.583435] Call Trace:
[ 20.583437] <TASK>
[ 20.583438] show_stack+0x48/0x60
[ 20.583446] dump_stack_lvl+0x75/0xb0
[ 20.583451] dump_stack+0x14/0x1a
[ 20.583452] print_deadlock_bug.cold+0xc0/0xca
[ 20.583457] validate_chain+0x4ca/0x970
[ 20.583463] __lock_acquire+0x587/0xc40
[ 20.583465] ? find_held_lock+0x31/0x90
[ 20.583470] lock_acquire.part.0+0xaf/0x230
[ 20.583472] ? page_cache_ra_unbounded+0x6f/0x280
[ 20.583474] ? debug_smp_processor_id+0x1b/0x30
[ 20.583481] lock_acquire+0x67/0x140
[ 20.583483] ? page_cache_ra_unbounded+0x6f/0x280
[ 20.583484] down_read+0x40/0x180
[ 20.583487] ? page_cache_ra_unbounded+0x6f/0x280
[ 20.583489] page_cache_ra_unbounded+0x6f/0x280
[ 20.583491] ? lock_acquire.part.0+0xaf/0x230
[ 20.583492] ? __this_cpu_preempt_check+0x17/0x20
[ 20.583495] generic_readahead_merkle_tree+0x133/0x140
[ 20.583501] ext4_readahead_merkle_tree+0x2a/0x30
[ 20.583507] fsverity_readahead+0x9d/0xc0
[ 20.583510] ext4_mpage_readpages+0x194/0x9b0
[ 20.583515] ? __lock_release.isra.0+0x5e/0x160
[ 20.583517] ext4_readahead+0x3a/0x40
[ 20.583521] read_pages+0x84/0x370
[ 20.583523] page_cache_ra_unbounded+0x16c/0x280
[ 20.583525] page_cache_ra_order+0x10c/0x170
[ 20.583527] page_cache_sync_ra+0x1a1/0x360
[ 20.583528] filemap_get_pages+0x141/0x4c0
[ 20.583532] ? __this_cpu_preempt_check+0x17/0x20
[ 20.583534] filemap_read+0x11f/0x540
[ 20.583536] ? __folio_batch_add_and_move+0x7c/0x330
[ 20.583539] ? __this_cpu_preempt_check+0x17/0x20
[ 20.583541] generic_file_read_iter+0xc1/0x110
[ 20.583543] ? do_pte_missing+0x13a/0x450
[ 20.583547] ext4_file_read_iter+0x51/0x17
WARNING: multiple messages have this Message-ID (diff)
From: Eric Biggers via Linux-f2fs-devel <linux-f2fs-devel@lists.sourceforge.net>
To: Christoph Hellwig <hch@lst.de>
Cc: fsverity@lists.linux.dev, Christian Brauner <brauner@kernel.org>,
Theodore Ts'o <tytso@mit.edu>,
Andrey Albershteyn <aalbersh@redhat.com>,
Matthew Wilcox <willy@infradead.org>,
linux-f2fs-devel@lists.sourceforge.net,
linux-fsdevel@vger.kernel.org, Al Viro <viro@zeniv.linux.org.uk>,
Jaegeuk Kim <jaegeuk@kernel.org>, David Sterba <dsterba@suse.com>,
Jan Kara <jack@suse.cz>,
linux-ext4@vger.kernel.org, linux-btrfs@vger.kernel.org
Subject: Re: [f2fs-dev] [PATCH 08/15] fsverity: kick off hash readahead at data I/O submission time
Date: Wed, 28 Jan 2026 14:56:02 -0800 [thread overview]
Message-ID: <20260128225602.GB2024@quark> (raw)
In-Reply-To: <20260128152630.627409-9-hch@lst.de>
On Wed, Jan 28, 2026 at 04:26:20PM +0100, Christoph Hellwig wrote:
> Currently all reads of the fsverity hashes is kicked off from the data
> I/O completion handler, leading to needlessly dependent I/O. This is
> worked around a bit by performing readahead on the level 0 nodes, but
> still fairly ineffective.
>
> Switch to a model where the ->read_folio and ->readahead methods instead
> kick off explicit readahead of the fsverity hashed so they are usually
> available at I/O completion time.
>
> For 64k sequential reads on my test VM this improves read performance
> from 2.4GB/s - 2.6GB/s to 3.5GB/s - 3.9GB/s. The improvements for
> random reads are likely to be even bigger.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Acked-by: David Sterba <dsterba@suse.com> [btrfs]
Unfortunately, this patch causes recursive down_read() of
address_space::invalidate_lock. How was this meant to work?
[ 20.563185] ============================================
[ 20.564179] WARNING: possible recursive locking detected
[ 20.565170] 6.19.0-rc7-00041-g7bd72c6393ab #2 Not tainted
[ 20.566180] --------------------------------------------
[ 20.567169] cmp/2320 is trying to acquire lock:
[ 20.568019] ffff888108465030 (mapping.invalidate_lock#2){++++}-{4:4}, at: page_cache_ra_unbounded+0x6f/0x280
[ 20.569828]
[ 20.569828] but task is already holding lock:
[ 20.570914] ffff888108465030 (mapping.invalidate_lock#2){++++}-{4:4}, at: page_cache_ra_unbounded+0x6f/0x280
[ 20.572739]
[ 20.572739] other info that might help us debug this:
[ 20.573938] Possible unsafe locking scenario:
[ 20.573938]
[ 20.575042] CPU0
[ 20.575522] ----
[ 20.576003] lock(mapping.invalidate_lock#2);
[ 20.576849] lock(mapping.invalidate_lock#2);
[ 20.577698]
[ 20.577698] *** DEADLOCK ***
[ 20.577698]
[ 20.578795] May be due to missing lock nesting notation
[ 20.578795]
[ 20.580045] 1 lock held by cmp/2320:
[ 20.580726] #0: ffff888108465030 (mapping.invalidate_lock#2){++++}-{4:4}, at: page_cache_ra_unbounded+0x6f/0x20
[ 20.582596]
[ 20.582596] stack backtrace:
[ 20.583428] CPU: 0 UID: 0 PID: 2320 Comm: cmp Not tainted 6.19.0-rc7-00041-g7bd72c6393ab #2 PREEMPT(none)
[ 20.583433] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Arch Linux 1.17.0-2-2 04/01/2014
[ 20.583435] Call Trace:
[ 20.583437] <TASK>
[ 20.583438] show_stack+0x48/0x60
[ 20.583446] dump_stack_lvl+0x75/0xb0
[ 20.583451] dump_stack+0x14/0x1a
[ 20.583452] print_deadlock_bug.cold+0xc0/0xca
[ 20.583457] validate_chain+0x4ca/0x970
[ 20.583463] __lock_acquire+0x587/0xc40
[ 20.583465] ? find_held_lock+0x31/0x90
[ 20.583470] lock_acquire.part.0+0xaf/0x230
[ 20.583472] ? page_cache_ra_unbounded+0x6f/0x280
[ 20.583474] ? debug_smp_processor_id+0x1b/0x30
[ 20.583481] lock_acquire+0x67/0x140
[ 20.583483] ? page_cache_ra_unbounded+0x6f/0x280
[ 20.583484] down_read+0x40/0x180
[ 20.583487] ? page_cache_ra_unbounded+0x6f/0x280
[ 20.583489] page_cache_ra_unbounded+0x6f/0x280
[ 20.583491] ? lock_acquire.part.0+0xaf/0x230
[ 20.583492] ? __this_cpu_preempt_check+0x17/0x20
[ 20.583495] generic_readahead_merkle_tree+0x133/0x140
[ 20.583501] ext4_readahead_merkle_tree+0x2a/0x30
[ 20.583507] fsverity_readahead+0x9d/0xc0
[ 20.583510] ext4_mpage_readpages+0x194/0x9b0
[ 20.583515] ? __lock_release.isra.0+0x5e/0x160
[ 20.583517] ext4_readahead+0x3a/0x40
[ 20.583521] read_pages+0x84/0x370
[ 20.583523] page_cache_ra_unbounded+0x16c/0x280
[ 20.583525] page_cache_ra_order+0x10c/0x170
[ 20.583527] page_cache_sync_ra+0x1a1/0x360
[ 20.583528] filemap_get_pages+0x141/0x4c0
[ 20.583532] ? __this_cpu_preempt_check+0x17/0x20
[ 20.583534] filemap_read+0x11f/0x540
[ 20.583536] ? __folio_batch_add_and_move+0x7c/0x330
[ 20.583539] ? __this_cpu_preempt_check+0x17/0x20
[ 20.583541] generic_file_read_iter+0xc1/0x110
[ 20.583543] ? do_pte_missing+0x13a/0x450
[ 20.583547] ext4_file_read_iter+0x51/0x17
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
next prev parent reply other threads:[~2026-01-28 22:56 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-01-28 15:26 fsverity cleanups, speedup and memory usage optimization v4 Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 01/15] fs,fsverity: reject size changes on fsverity files in setattr_prepare Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] [PATCH 01/15] fs, fsverity: " Christoph Hellwig
2026-02-17 21:14 ` patchwork-bot+f2fs
2026-02-17 21:14 ` patchwork-bot+f2fs--- via Linux-f2fs-devel
2026-01-28 15:26 ` [PATCH 02/15] fs,fsverity: clear out fsverity_info from common code Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] [PATCH 02/15] fs, fsverity: " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 03/15] ext4: don't build the fsverity work handler for !CONFIG_FS_VERITY Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 04/15] f2fs: " Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 05/15] fsverity: pass struct file to ->write_merkle_tree_block Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 06/15] fsverity: start consolidating pagecache code Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 07/15] fsverity: don't issue readahead for non-ENOENT errors from __filemap_get_folio Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 16:32 ` Darrick J. Wong
2026-01-28 16:32 ` [f2fs-dev] " Darrick J. Wong via Linux-f2fs-devel
2026-01-28 23:48 ` Eric Biggers
2026-01-28 23:48 ` [f2fs-dev] " Eric Biggers via Linux-f2fs-devel
2026-01-30 5:48 ` Christoph Hellwig
2026-01-30 5:48 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 08/15] fsverity: kick off hash readahead at data I/O submission time Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 16:33 ` Darrick J. Wong
2026-01-28 16:33 ` [f2fs-dev] " Darrick J. Wong via Linux-f2fs-devel
2026-01-28 22:56 ` Eric Biggers [this message]
2026-01-28 22:56 ` Eric Biggers via Linux-f2fs-devel
2026-01-28 23:22 ` Darrick J. Wong
2026-01-28 23:22 ` [f2fs-dev] " Darrick J. Wong via Linux-f2fs-devel
2026-01-30 5:55 ` Christoph Hellwig
2026-01-30 5:55 ` [f2fs-dev] " Christoph Hellwig
2026-01-30 5:51 ` Christoph Hellwig
2026-01-30 5:51 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 09/15] fsverity: deconstify the inode pointer in struct fsverity_info Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 10/15] fsverity: push out fsverity_info lookup Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 11/15] fs: consolidate fsverity_info lookup in buffer.c Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 12/15] ext4: consolidate fsverity_info lookup Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 16:28 ` Jan Kara
2026-01-28 16:28 ` [f2fs-dev] " Jan Kara
2026-01-28 15:26 ` [PATCH 13/15] f2fs: " Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 14/15] btrfs: " Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-28 15:26 ` [PATCH 15/15] fsverity: use a hashtable to find the fsverity_info Christoph Hellwig
2026-01-28 15:26 ` [f2fs-dev] " Christoph Hellwig
2026-01-29 0:07 ` fsverity cleanups, speedup and memory usage optimization v4 Eric Biggers
2026-01-29 0:07 ` [f2fs-dev] " Eric Biggers via Linux-f2fs-devel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260128225602.GB2024@quark \
--to=ebiggers@kernel.org \
--cc=aalbersh@redhat.com \
--cc=brauner@kernel.org \
--cc=chao@kernel.org \
--cc=dsterba@suse.com \
--cc=fsverity@lists.linux.dev \
--cc=hch@lst.de \
--cc=jack@suse.cz \
--cc=jaegeuk@kernel.org \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-f2fs-devel@lists.sourceforge.net \
--cc=linux-fsdevel@vger.kernel.org \
--cc=tytso@mit.edu \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.