Re: [PATCHSET 06/10] mkfs: scale shards on ssds

Bill O'Donnell
2024-03-13T18:17:39Z

On Tue, Mar 12, 2024 at 06:48:36PM -0700, Darrick J. Wong wrote:
> Hi all,
>
> For a long time, the maintainers have had a gut feeling that we could
> optimize performance of XFS filesystems on non-mechanical storage by
> scaling the number of allocation groups to be a multiple of the CPU
> count.
>
> With modern ~2022 hardware, it is common for systems to have more than
> four CPU cores and non-striped SSDs ranging in size from 256GB to 4TB.
> The default mkfs geometry still defaults to 4 AGs regardless of core
> count, which was settled on in the age of spinning rust.
>
> This patchset adds a different computation for AG count and log size
> that is based entirely on a desired level of concurrency. If we detect
> storage that is non-rotational (or the sysadmin provides a CLI option),
> then we will try to match the AG count to the CPU count to minimize AGF
> contention and make the log large enough to minimize grant head
> contention.
>
> If you're going to start using this code, I strongly recommend pulling
> from my git trees, which are linked below.
>
> This has been running on the djcloud for months with no problems. Enjoy!
> Comments and questions are, as always, welcome.
>
> --D

Series...
Reviewed-by: Bill O'Donnell <bodonnel@redhat.com>

>
> xfsprogs git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=mkfs-scale-geo-on-ssds
>
> fstests git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=mkfs-scale-geo-on-ssds
> ---
> Commits in this patchset:
> * mkfs: allow sizing allocation groups for concurrency
> * mkfs: allow sizing internal logs for concurrency
> ---
> man/man8/mkfs.xfs.8.in | 46 +++++++++
> mkfs/xfs_mkfs.c | 251 +++++++++++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 291 insertions(+), 6 deletions(-)
>
>