Re: Possible regression after NFS eof page pollution fix (ext4 checksum errors)

[Trond Myklebust <trondmy@kernel.org>]

On Mon, 2026-01-05 at 16:00 +0200, Mark Bloch wrote:
> 
> 
> On 04/01/2026 17:36, Trond Myklebust wrote:
> > On Sun, 2026-01-04 at 11:16 +0200, Mark Bloch wrote:
> > > Hi Trond,
> > > 
> > > We’ve recently started seeing filesystem issues in our internal
> > > regression runs, and we were able to bisect the problem down to
> > > the following commit:
> > > 
> > > commit b1817b18ff20e69f5accdccefaf78bf5454bede2
> > > Author: Trond Myklebust <trond.myklebust@hammerspace.com>
> > > Date:   Thu Sep 4 18:46:16 2025 -0400
> > > 
> > >     NFS: Protect against 'eof page pollution'
> > > 
> > >     This commit fixes the failing xfstest 'generic/363'.
> > > 
> > >     When the user mmaps() an area that extends beyond the end of
> > > file, and
> > >     proceeds to write data into the folio that straddles that
> > > eof,
> > > we're
> > >     required to discard that folio data if the user calls some
> > > function that
> > >     extends the file length.
> > > 
> > >     Signed-off-by: Trond Myklebust
> > > <trond.myklebust@hammerspace.com>
> > > 
> > > 
> > > After this change, we intermittently see EXT4 checksum-related
> > > errors
> > > during boot.
> > > A representative dmesg excerpt is below:
> > > 
> > >  [ 1908.365537] EXT4-fs warning (device vda2):
> > > ext4_dirblock_csum_verify:375: inode #263414: comm updatedb: No
> > > space
> > > for directory leaf checksum. Please run e2fsck -D.
> > >  [ 1908.375449] EXT4-fs error (device vda2):
> > > __ext4_find_entry:1624:
> > > inode #263414: comm updatedb: checksumming directory block 0
> > >  [ 1908.382985] EXT4-fs warning (device vda2):
> > > ext4_dirblock_csum_verify:375: inode #263414: comm updatedb: No
> > > space
> > > for directory leaf checksum. Please run e2fsck -D.
> > >  [ 1908.389289] EXT4-fs error (device vda2):
> > > __ext4_find_entry:1624:
> > > inode #263414: comm updatedb: checksumming directory block 0
> > >  [ 1909.598811] EXT4-fs warning (device vda2):
> > > ext4_dirblock_csum_verify:375: inode #423753: comm updatedb: No
> > > space
> > > for directory leaf checksum. Please run e2fsck -D.
> > >  [ 1909.604308] EXT4-fs error (device vda2):
> > > htree_dirblock_to_tree:1051: inode #423753: comm updatedb:
> > > Directory
> > > block failed checksum
> > >  [ 1909.958470] EXT4-fs warning (device vda2):
> > > ext4_dirblock_csum_verify:375: inode #423759: comm updatedb: No
> > > space
> > > for directory leaf checksum. Please run e2fsck -D.
> > >  [ 1909.963825] EXT4-fs error (device vda2):
> > > htree_dirblock_to_tree:1051: inode #423759: comm updatedb:
> > > Directory
> > > block failed checksum
> > >  [ 1909.985956] EXT4-fs warning (device vda2):
> > > ext4_dirblock_csum_verify:375: inode #303617: comm updatedb: No
> > > space
> > > for directory leaf checksum. Please run e2fsck -D.
> > >  [ 1909.991371] EXT4-fs error (device vda2):
> > > __ext4_find_entry:1624:
> > > inode #303617: comm updatedb: checksumming directory block 0
> > >  [ 1910.156415] EXT4-fs warning (device vda2):
> > > ext4_dirblock_csum_verify:375: inode #423761: comm updatedb: No
> > > space
> > > for directory leaf checksum. Please run e2fsck -D.
> > >  [ 1910.161959] EXT4-fs error (device vda2):
> > > htree_dirblock_to_tree:1051: inode #423761: comm updatedb:
> > > Directory
> > > block failed checksum
> > >  [ 1910.171364] EXT4-fs warning (device vda2):
> > > ext4_dirblock_csum_verify:375: inode #423735: comm updatedb: No
> > > space
> > > for directory leaf checksum. Please run e2fsck -D.
> > >  [ 1910.177292] EXT4-fs error (device vda2):
> > > htree_dirblock_to_tree:1051: inode #423735: comm updatedb:
> > > Directory
> > > block failed checksum
> > >  [ 1910.267721] EXT4-fs warning (device vda2):
> > > ext4_dirblock_csum_verify:375: inode #423744: comm updatedb: No
> > > space
> > > for directory leaf checksum. Please run e2fsck -D.
> > >  [ 1910.281838] EXT4-fs error (device vda2):
> > > htree_dirblock_to_tree:1051: inode #423744: comm updatedb:
> > > Directory
> > > block failed checksum
> > >  [ 1910.476906] EXT4-fs warning (device vda2):
> > > ext4_dirblock_csum_verify:375: inode #423751: comm updatedb: No
> > > space
> > > for directory leaf checksum. Please run e2fsck -D.
> > >  [ 1910.482403] EXT4-fs error (device vda2):
> > > htree_dirblock_to_tree:1051: inode #423751: comm updatedb:
> > > Directory
> > > block failed checksum
> > > 
> > > The issue has so far only been observed in tests that use a
> > > nested VM
> > > setup.
> > > It does not reproduce deterministically, roughly half of the
> > > nested
> > > VM boots trigger the problem.
> > > 
> > > Would you mind taking a look or pointing us in the right
> > > direction?
> > > Please let us know if additional information, testing,
> > > or instrumentation would be helpful.
> > > 
> > > Thanks,
> > > Mark
> > 
> > I'm having trouble seeing how those issues can be related unless
> > ext4
> > and NFS are somehow sharing the same folios. Does reverting just 
> > commit b1817b18ff20 and b2036bb65114 actually fix the ext4 problem?
> 
> Yes, after reverting those two commits we no longer can reproduce it.
> 
> > 
> > What does "nested VM" mean in this situation, and what is the
> > storage
> > for the ext4 filesystem that is being corrupted?
> > 
> 
> Probably should have explained better, let me do that now.
> Say we have host A.
> On host A we run VM B.
> Inside VM B we run VM C.
> 
> Inside VM B we have a mount (nfs one)
> X:/images/.libvirt on /images/.libvirt type nfs4
> (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,fat
> al_neterrors=none,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=Y,
> local_lock=none,addr=X)
> 
> which holds the .img files. We launce QEMU with something like this:
> 
> {"driver":"file","filename":"/images/.libvirt/linux-VAGRANTSLASH-
> upstream_Z.img","node-name":"libvirt-2-storage","auto-read-
> only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-
> format","read-only":true,"driver":"qcow2","file":"libvirt-2-
> storage","backing":null} -blockdev
> {"driver":"file","filename":"/images/.libvirt/Y.img","node-
> name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}
> 
> inside VM C, it's a regular ext4 mount:
> /dev/vda2 on / type ext4 (rw,relatime)
> 
> Mark
> 

OK so if I'm understanding correctly, this is organised as ext4
partitions that are stored in qcow2 images that are again stored on a
NFSv4.2 partition.

Do these qcow2 images have a file size that is fixed at creation time,
or is the file size dynamic?
Also, does changing the "discard" option from "unmap" to "ignore" make
any difference to the outcome?

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trondmy@kernel.org, trond.myklebust@hammerspace.com