Re: Why a lot of fses are using bdev's page cache to do super block read/write?

Darrick J. Wong
2025-07-09T15:04:38Z

On Wed, Jul 09, 2025 at 06:35:00PM +0930, Qu Wenruo wrote:
> Hi,
>
> Recently I'm trying to remove direct bdev's page cache usage from btrfs
> super block IOs.
>
> And replace it with common bio interface (mostly with bdev_rw_virt()).
>
> However I'm hitting random generic/492 failure where sometimes blkid failed
> to detect any useful super block signature of btrfs.

Yes, you need to invalidate_bdev() after writing the superblock directly
to disk via submit_bio.

> This leads more digging, and to my surprise using bdev's page cache to do
> superblock IOs is not an exception, in fact f2fs is doing exactly the same
> thing.
>
>
> This makes me wonder:
>
> - Should a fs use bdev's page cache directly?
> I thought a fs shouldn't do this, and bio interface should be
> enough for most if not all cases.
>
> Or am I wrong in the first place?

As willy said, most filesystems use the bdev pagecache because then they
don't have to implement their own (metadata) buffer cache. The downside
is that any filesystem that does so must be prepared to handle the
buffer_head contents changing any time they cycle the bh lock because
anyone can write to the block device of a mounted fs ala tune2fs.

Effectively this means that you have to (a) revalidate the entire buffer
contents every time you lock_buffer(); and (b) you can't make decisions
based on superblock feature bits in the superblock bh directly.

I made that mistake when adding metadata_csum support to ext4 -- we'd
only connect to the crc32c "crypto" module if checksums were enabled in
the ondisk super at mount time, but then there were a couple of places
that looked at the ondisk super bits at runtime, so you could flip the
bit on and crash the kernel almost immediately.

Nowadays you could protect against malicious writes with the
BLK_DEV_WRITE_MOUNTED=n so at least that's mitigated a little bit.
Note (a) implies that the use of BH_Verified is a giant footgun.

Catherine Hoang [now cc'd] has prototyped a generic buffer cache so that
we can fix these vulnerabilities in ext2:
https://lore.kernel.org/linux-ext4/20250326014928.61507-1-catherine.hoang@oracle.com/

> - What is keeping fs super block update from racing with user space
> device scan?
>
> I guess it's the regular page/folio locking of the bdev page cache.
> But that also means, pure bio based IO will always race with buffered
> read of a block device.

Right. In theory you could take the posix advisory lock (aka flock)
from inside the kernel for the duration of the sb write, and that would
prevent libblkid/udev from seeing torn/stale contents because they take
LOCK_SH.

> - If so, is there any special bio flag to prevent such race?
> So far I am unable to find out such flag.

No.

--D

> Thanks,
> Qu
>