[GIT PULL 16/22] xfs: merge bmap records for faster scrubs

[GIT PULL 16/22] xfs: merge bmap records for faster scrubs

[Darrick J. Wong]

Hi Dave,

Please pull this branch with changes for xfs.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit 6bb9209ceebb07fd07cec25af04eed1809c654de:

xfs: always check the existence of a dirent's child inode (2023-04-11 19:00:18 -0700)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-merge-bmap-records-6.4_2023-04-11

for you to fetch changes up to 1e59fdb7d6157ff685a250e0873a015a2b16a4f2:

xfs: don't call xchk_bmap_check_rmaps for btree-format file forks (2023-04-11 19:00:26 -0700)

----------------------------------------------------------------
xfs: merge bmap records for faster scrubs [v24.5]

I started looking into performance problems with the data fork scrubber
in generic/333, and noticed a few things that needed improving.  First,
due to design reasons, it's possible for file forks btrees to contain
multiple contiguous mappings to the same physical space.  Instead of
checking each ondisk mapping individually, it's much faster to combine
them when possible and check the combined mapping because that's fewer
trips through the rmap btree, and we can drop this check-around
behavior that it does when an rmapbt lookup produces a record that
starts before or ends after a particular bmbt mapping.

Second, I noticed that the bmbt scrubber decides to walk every reverse
mapping in the filesystem if the file fork is in btree format.  This is
very costly, and only necessary if the inode repair code had to zap a
fork to convince iget to work.  Constraining the full-rmap scan to this
one case means we can skip it for normal files, which drives the runtime
of this test from 8 hours down to 45 minutes (observed with realtime
reflink and rebuild-all mode.)

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (14):
xfs: remove xchk_parent_count_parent_dentries
xfs: simplify xchk_parent_validate
xfs: fix parent pointer scrub racing with subdirectory reparenting
xfs: manage inode DONTCACHE status at irele time
xfs: fix an inode lookup race in xchk_get_inode
xfs: rename xchk_get_inode -> xchk_iget_for_scrubbing
xfs: retain the AGI when we can't iget an inode to scrub the core
xfs: don't take the MMAPLOCK when scrubbing file metadata
xfs: change bmap scrubber to store the previous mapping
xfs: accumulate iextent records when checking bmap
xfs: split xchk_bmap_xref_rmap into two functions
xfs: alert the user about data/attr fork mappings that could be merged
xfs: split the xchk_bmap_check_rmaps into a predicate
xfs: don't call xchk_bmap_check_rmaps for btree-format file forks

fs/xfs/libxfs/xfs_bmap.h |   2 +-
fs/xfs/scrub/bmap.c      | 388 ++++++++++++++++++++++++++++++-----------------
fs/xfs/scrub/common.c    | 328 ++++++++++++++++++++++++++++++---------
fs/xfs/scrub/common.h    |  11 +-
fs/xfs/scrub/dir.c       |  14 +-
fs/xfs/scrub/inode.c     | 191 +++++++++++++++++++----
fs/xfs/scrub/parent.c    | 237 ++++++++++-------------------
fs/xfs/scrub/scrub.c     |   2 +-
fs/xfs/xfs_icache.c      |   3 +-
fs/xfs/xfs_icache.h      |  11 +-
10 files changed, 765 insertions(+), 422 deletions(-)

[GIT PULL v2 16/22] xfs: merge bmap records for faster scrubs

[Darrick J. Wong]

Hi Dave,

Please pull this branch with changes for xfs.

As usual, I did a test-merge with the main upstream branch as of a few
minutes ago, and didn't see any conflicts.  Please let me know if you
encounter any problems.

--D

The following changes since commit 1fc7a0597d237c17b6501f8c33b76d3eaaae9079:

xfs: don't take the MMAPLOCK when scrubbing file metadata (2023-04-11 19:00:22 -0700)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-merge-bmap-records-6.4_2023-04-12

for you to fetch changes up to 1e59fdb7d6157ff685a250e0873a015a2b16a4f2:

xfs: don't call xchk_bmap_check_rmaps for btree-format file forks (2023-04-11 19:00:26 -0700)

----------------------------------------------------------------
xfs: merge bmap records for faster scrubs [v24.5]

I started looking into performance problems with the data fork scrubber
in generic/333, and noticed a few things that needed improving.  First,
due to design reasons, it's possible for file forks btrees to contain
multiple contiguous mappings to the same physical space.  Instead of
checking each ondisk mapping individually, it's much faster to combine
them when possible and check the combined mapping because that's fewer
trips through the rmap btree, and we can drop this check-around
behavior that it does when an rmapbt lookup produces a record that
starts before or ends after a particular bmbt mapping.

Second, I noticed that the bmbt scrubber decides to walk every reverse
mapping in the filesystem if the file fork is in btree format.  This is
very costly, and only necessary if the inode repair code had to zap a
fork to convince iget to work.  Constraining the full-rmap scan to this
one case means we can skip it for normal files, which drives the runtime
of this test from 8 hours down to 45 minutes (observed with realtime
reflink and rebuild-all mode.)

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (6):
xfs: change bmap scrubber to store the previous mapping
xfs: accumulate iextent records when checking bmap
xfs: split xchk_bmap_xref_rmap into two functions
xfs: alert the user about data/attr fork mappings that could be merged
xfs: split the xchk_bmap_check_rmaps into a predicate
xfs: don't call xchk_bmap_check_rmaps for btree-format file forks

fs/xfs/libxfs/xfs_bmap.h |   2 +-
fs/xfs/scrub/bmap.c      | 379 ++++++++++++++++++++++++++++++-----------------
2 files changed, 242 insertions(+), 139 deletions(-)