Re: [PATCH 0/6] xfs: CPU usage optimizations for realtime allocator

Omar Sandoval
2023-07-06T21:39:13Z

On Tue, Jun 20, 2023 at 02:32:10PM -0700, Omar Sandoval wrote:
> Hello,
>
> Our distributed storage system uses XFS's realtime device support as a
> way to split an XFS filesystem between an SSD and an HDD -- we configure
> the HDD as the realtime device so that metadata goes on the SSD and data
> goes on the HDD.
>
> We've been running this in production for a few years now, so we have
> some fairly fragmented filesystems. This has exposed various CPU
> inefficiencies in the realtime allocator. These became even worse when
> we experimented with using XFS_XFLAG_EXTSIZE to force files to be
> allocated contiguously.
>
> This series adds several optimizations that don't change the realtime
> allocator's decisions, but make them happen more efficiently, mainly by
> avoiding redundant work. We've tested these in production and measured
> ~10% lower CPU utilization. Furthermore, it made it possible to use
> XFS_XFLAG_EXTSIZE to force contiguous allocations -- without these
> patches, our most fragmented systems would become unresponsive due to
> high CPU usage in the realtime allocator, but with them, CPU utilization
> is actually ~4-6% lower than before, and disk I/O utilization is 15-20%
> lower.
>
> Patches 2 and 3 are preparations for later optimizations; the remaining
> patches are the optimizations themselves.
>
> This is based on Linus' tree as of today (commit
> 692b7dc87ca6d55ab254f8259e6f970171dc9d01).
>
> Thanks!
>
> Omar Sandoval (6):
> xfs: cache last bitmap block in realtime allocator
> xfs: invert the realtime summary cache
> xfs: return maximum free size from xfs_rtany_summary()
> xfs: limit maxlen based on available space in
> xfs_rtallocate_extent_near()
> xfs: don't try redundant allocations in xfs_rtallocate_extent_near()
> xfs: don't look for end of extent further than necessary in
> xfs_rtallocate_extent_near()
>
> fs/xfs/libxfs/xfs_rtbitmap.c | 173 ++++++++++++++--------------
> fs/xfs/xfs_mount.h | 6 +-
> fs/xfs/xfs_rtalloc.c | 215 ++++++++++++++++-------------------
> fs/xfs/xfs_rtalloc.h | 28 +++--
> 4 files changed, 207 insertions(+), 215 deletions(-)

Gentle ping.