Re: e2scrub finds corruption immediately after mounting

["Darrick J. Wong" <djwong@kernel.org>]

On Wed, Jan 03, 2024 at 04:14:36PM -0500, Brian J. Murrell wrote:
> I am trying to migrate from lvcheck
> (https://github.com/BryanKadzban/lvcheck) to using the officially
> supported e2scrub[_all] kit.
> 
> I am finding that e2scrub very often (much more than lvcheck even)
> finds corruption and wants me to do an offline e2fsck.  Not only does
> it do this immediately after booting a system that includes filesystem
> checks (that were caused by e2scrub previously setting a filesystem to
> be checked on next boot), but it happens immediately after I run an
> e2fsck and then mount the filesystem, even without any activity on it.
> Observe:
> 
> # umount /opt
> # e2fsck -y /dev/rootvol_tmp/almalinux8_opt 
> e2fsck 1.45.6 (20-Mar-2020)
> /dev/mapper/rootvol_tmp-almalinux8_opt: clean, 1698/178816 files,
> 482404/716800 blocks
> # e2scrub /dev/rootvol_tmp/almalinux8_opt 
>   Logical volume "almalinux8_opt.e2scrub" created.
> e2fsck 1.45.6 (20-Mar-2020)
> Pass 1: Checking inodes, blocks, and sizes
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> /dev/rootvol_tmp/almalinux8_opt.e2scrub: 1698/178816 files (86.9% non-
> contiguous), 482404/716800 blocks
> /dev/rootvol_tmp/almalinux8_opt: Scrub succeeded.
> tune2fs 1.45.6 (20-Mar-2020)
> Setting current mount count to 0
> Setting time filesystem last checked to Wed Jan  3 11:37:04 2024
> 
>   Logical volume "almalinux8_opt.e2scrub" successfully removed.
> # mount /opt
> # e2scrub /dev/rootvol_tmp/almalinux8_opt 
>   Logical volume "almalinux8_opt.e2scrub" created.
> e2fsck 1.45.6 (20-Mar-2020)
> Pass 1: Checking inodes, blocks, and sizes
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> /dev/rootvol_tmp/almalinux8_opt.e2scrub: 1698/178816 files (86.9% non-
> contiguous), 482404/716800 blocks
> /dev/rootvol_tmp/almalinux8_opt: Scrub FAILED due to corruption! 

Curious.  Normally e2scrub will run e2fsck twice: Once in journal-only
preen mode to replay the journal, then again with -fy to perform the
full filesystem (snapshot) check.

I wonder if you would paste the output of
"bash -x e2scrub /dev/rootvol_tmp/almalinux8_opt" here?  I'd be curious
to see what the command flow is.

Assuming that 1.47.0 doesn't magically fix it. :)

> Unmount and run e2fsck -y.
> tune2fs 1.45.6 (20-Mar-2020)
> Setting filesystem error flag to force fsck.
>   Logical volume "almalinux8_opt.e2scrub" successfully removed.
> 
> So as you can see, I unmount /opt, run an e2fsck -y on it to clean it
> and then before mounting run e2scrub and it finds the filesystem clean.
> Good so far.
> 
> I then mount it and then immediately run another e2scrub on it and that
> finds it dirty and wants me to unmount and run another e2fsck -y on it.
> But how can that be?  Surely an e2scrub on a freshly cleaned and
> mounted filesystem (with no activity on it in between) should be clean,
> yes?

Right.  Unless something's broken in e2fsck. :/

--D

> 
> Cheers,
> b.
>