[PATCHBOMB 6.13] xfs: metadata directories and realtime groups

Darrick J. Wong
2024-10-11T00:24:03Z

Hi everyone,

Christoph and I have been working on getting the long-delayed metadata
directory tree patchset into mergeable shape, and I think we're now
satisfied that we've gotten the code to where we want it for 6.13.
This time around we've included a ton of cleanups and refactorings that
Dave requested during the 6.12 cycle.

The metadata directory tree sets us up for much more flexible metadata
within an XFS filesystem. Instead of rooting inodes in the superblock
which has very limited space, we instead create a directory tree that
can contain arbitrary numbers of metadata files.

Having done that, we can now shard the realtime volume into multiple
allocation groups, much as we do with AGs for the data device. However,
the realtime volume has a fun twist -- each rtgroup gets its own space
metadata files, and for that we need a metadata directory tree.
Note that we also implement busy free(d) extent tracking, which means
that we can do discards asynchronously.

Metadata directory trees and realtime groups also enable us to complete
the realtime modernization project, which will add reverse mapping
btrees, reflink, quota support, and zoned storage support for rt
volumes.

Finally, quota inodes now live in the metadata directory tree, which is
a pretty simple conversion. However, we added yet another new feature,
which is that xfs will now remember the quota accounting and enforcement
state across unmounts. You can still tweak them via mount options, but
not specifying any is no longer interpreted the same as 'noquota'.
Quotas for the realtime are now supported.

I'm only sending the kernel patches to the list for now, but please have
a look at the git tree links for xfsprogs and fstests changes.

https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/tag/?h=metadir_2024-10-10
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfsprogs-dev.git/tag/?h=metadir_2024-10-10
https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/tag/?h=metadir-quotas_2024-10-10

This is a list of the kernel patches that remain unreviewed:

[PATCHSET v5.0 3/9] xfs: metadata inode directory trees
[PATCH 01/28] xfs: constify the xfs_sb predicates
[PATCH 02/28] xfs: constify the xfs_inode predicates
[PATCH 04/28] xfs: undefine the sb_bad_features2 when metadir is
[PATCHSET v5.0 4/9] xfs: create incore rt allocation groups
[PATCH 08/21] xfs: add a xfs_qm_unmount_rt helper
[PATCH 09/21] xfs: factor out a xfs_growfs_rt_alloc_blocks helper
[PATCH 10/21] xfs: cleanup xfs_getfsmap_rtdev_rtbitmap
[PATCH 11/21] xfs: split xfs_trim_rtdev_extents
[PATCHSET v5.0 5/9] xfs: preparation for realtime allocation groups
[PATCH 1/2] xfs: fix rt device offset calculations for FITRIM
[PATCHSET v5.0 6/9] xfs: shard the realtime section
[PATCH 15/36] xfs: store rtgroup information with a bmap intent
[PATCH 27/36] xfs: create helpers to deal with rounding xfs_fileoff_t
[PATCH 28/36] xfs: create helpers to deal with rounding xfs_filblks_t
[PATCH 29/36] xfs: make xfs_rtblock_t a segmented address like
[PATCH 32/36] xfs: fix minor bug in xfs_verify_agbno
[PATCH 33/36] xfs: move the min and max group block numbers to
[PATCH 34/36] xfs: port the perag discard code to handle generic
[PATCH 35/36] xfs: implement busy extent tracking for rtgroups
[PATCH 36/36] xfs: use rtgroup busy extent list for FITRIM
[PATCHSET v5.0 8/9] xfs: enable quota for realtime volumes
[PATCH 1/6] xfs: fix chown with rt quota
[PATCH 2/6] xfs: advertise realtime quota support in the xqm stat
[PATCH 3/6] xfs: report realtime block quota limits on realtime
[PATCH 4/6] xfs: create quota preallocation watermarks for realtime
[PATCH 5/6] xfs: reserve quota for realtime files correctly
[PATCH 6/6] xfs: enable realtime quota again

None of the userspace and fstests patches have been reviewed.

--D