[GIT PULL 08/11] mkfs: scale shards on ssds

Darrick J. Wong
2024-04-17T22:09:25Z

The following changes since commit 53bf0604e104bc053778495faef94b3570582aac:

mkfs: use a sensible log sector size default (2024-04-17 14:06:27 -0700)

are available in the Git repository at:

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git tags/mkfs-scale-geo-on-ssds-6.8_2024-04-17

for you to fetch changes up to c02a18733fc0a0e1b607f75e90962b3adc27c8fa:

mkfs: allow sizing internal logs for concurrency (2024-04-17 14:06:27 -0700)

----------------------------------------------------------------
mkfs: scale shards on ssds [08/20]

For a long time, the maintainers have had a gut feeling that we could
optimize performance of XFS filesystems on non-mechanical storage by
scaling the number of allocation groups to be a multiple of the CPU
count.

With modern ~2022 hardware, it is common for systems to have more than
four CPU cores and non-striped SSDs ranging in size from 256GB to 4TB.
The default mkfs geometry still defaults to 4 AGs regardless of core
count, which was settled on in the age of spinning rust.

This patchset adds a different computation for AG count and log size
that is based entirely on a desired level of concurrency. If we detect
storage that is non-rotational (or the sysadmin provides a CLI option),
then we will try to match the AG count to the CPU count to minimize AGF
contention and make the log large enough to minimize grant head
contention.

This has been running on the djcloud for months with no problems. Enjoy!

Signed-off-by: Darrick J. Wong <djwong@kernel.org>

----------------------------------------------------------------
Darrick J. Wong (2):
mkfs: allow sizing allocation groups for concurrency
mkfs: allow sizing internal logs for concurrency

man/man8/mkfs.xfs.8.in | 46 +++++++++
mkfs/xfs_mkfs.c | 251 +++++++++++++++++++++++++++++++++++++++++++++++--
2 files changed, 291 insertions(+), 6 deletions(-)