[PATCHSET v29.0 33/40] xfs_scrub: use free space histograms to reduce fstrim runtime

Darrick J. Wong
2023-12-31T19:48:18Z

Hi all,

This patchset dramatically reduces the runtime of the FITRIM calls made
during phase 8 of xfs_scrub. It turns out that phase 8 can really get
bogged down if the free space contains a large number of very small
extents. In these cases, the runtime can increase by an order of
magnitude to free less than 1% of the free space. This is not worth the
time, since we're spending a lot of time to do very little work. The
FITRIM ioctl allows us to specify a minimum extent length, so we can use
statistical methods to compute a minlen parameter.

It turns out xfs_db/spaceman already have the code needed to create
histograms of free space extent lengths. We add the ability to compute
a CDF of the extent lengths, which make it easy to pick a minimum length
corresponding to 99% of the free space. In most cases, this results in
dramatic reductions in phase 8 runtime. Hence, move the histogram code
to libfrog, and wire up xfs_scrub, since phase 7 already walks the
fsmap.

We also add a new -o suboption to xfs_scrub so that people who /do/ want
to examine every free extent can do so.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems. Enjoy!
Comments and questions are, as always, welcome.

--D

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-fstrim-minlen-freesp-histogram

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=scrub-fstrim-minlen-freesp-histogram
---
db/freesp.c | 83 +++-------------
libfrog/Makefile | 2
libfrog/histogram.c | 252 ++++++++++++++++++++++++++++++++++++++++++++++++++
libfrog/histogram.h | 54 +++++++++++
man/man8/xfs_scrub.8 | 16 +++
scrub/phase7.c | 47 +++++++++
scrub/phase8.c | 75 ++++++++++++++-
scrub/spacemap.c | 11 +-
scrub/vfs.c | 4 +
scrub/vfs.h | 2
scrub/xfs_scrub.c | 45 +++++++++
scrub/xfs_scrub.h | 16 +++
spaceman/freesp.c | 93 +++++-------------
13 files changed, 544 insertions(+), 156 deletions(-)
create mode 100644 libfrog/histogram.c
create mode 100644 libfrog/histogram.h