[RFD] xfsprogs/mkfs: prototype XFS image mode format for scalable AG growth

Brian Foster
2024-08-12T13:56:05Z

Tweak a few checks to facilitate experimentation with an agcount=1
filesystem format with a larger agsize than the filesystem data
size. The purpose of this is to POC a filesystem image mode format
for XFS that better supports the typical cloud filesystem image
deployment use case where a very small fs image is created and then
immediately grown orders of magnitude in size once deployed to
container environments. The large grow size delta produces
filesystems with excessive AG counts, which leads to various other
functional problems that eventually derive from this sort of
pathological geometry.

To experiment with this patch, format a small fs with something like
the following:

mkfs.xfs -f -lsize=64m -dsize=512m,agcount=1,agsize=8g <imgfile>

Increase the underlying image file size, mount and grow. The
filesystem will grow according to the format time AG size as if the
AG was a typical runt AG on a traditional multi-AG fs.

This means that the filesystem remains with an AG count of 1 until
fs size grows beyond AG size. Since the typical deployment workflow
is an immediate very small -> very large, one-time grow, the image
fs can set a reasonable enough default or configurable AG size
(based on user input) that ensures deployed filesystems end up in a
generally supportable geometry (i.e. with multiple AGs for
superblock redundancy) before seeing production workloads.

Further optional changes are possible on the kernel side to help
provide some simple guardrails against misuse of this mechanism. For
example, the kernel could do anything from warn/fail or restrict
runtime functionality for an insufficient grow. The image mode
itself could set a backwards incompat feature bit that requires a
mount option to enable full functionality (with the exception of
growfs). More discussion is required to determine whether this
provides a usable solution for the common cloud workflows that
exhibit this problem and what the right interface and/or limitations
are to ensure it is used correctly.

Not-Signed-off-by: Brian Foster <bfoster@redhat.com>
---

Hi all,

This is a followup to the idea Darrick brought up in the expansion
discussion here [1]. I poked through the code a bit and found it
somewhat amusing how little was in the way of experimenting with this,
so threw this against an fstests run over the weekend. I see maybe
around ~10 or so test failures, most of which look like simple failures
related to either not expecting agcount == 1 fs' or my generally
hacky/experimental changes. There are a couple or so that require a bit
more investigation to properly characterize before I would consider this
fully sane.

I'm posting this separately from the expansion discussion to hopefully
avoid further conflating the two. My current sense is that if this turns
out to be a fundamentally workable approach, mkfs would more look
something like 'mkfs --image-size 40g ...' and the kernel side may grow
some optional guardrail logic mentioned above and in the previous
discussion here [2], but others might have different ideas.

Darrick, you originally raised this idea and then Eric brought up some
legitimate technical concerns in the expansion design thread. I'm
curious if either of you have any further thoughts/ideas on this.

Brian

[1] https://lore.kernel.org/linux-xfs/20240721230100.4159699-1-david@fromorbit.com/
[2] https://lore.kernel.org/linux-xfs/ZqzMay58f0SvdWxV@bfoster/

mkfs/xfs_mkfs.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index 6d2469c3c..50a874a03 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -325,8 +325,7 @@ static struct opt_params dopts = {
},
.subopt_params = {
{ .index = D_AGCOUNT,
- .conflicts = { { &dopts, D_AGSIZE },
- { &dopts, D_CONCURRENCY },
+ .conflicts = { { &dopts, D_CONCURRENCY },
{ NULL, LAST_CONFLICT } },
.minval = 1,
.maxval = XFS_MAX_AGNUMBER,
@@ -368,8 +367,7 @@ static struct opt_params dopts = {
.defaultval = SUBOPT_NEEDS_VAL,
},
{ .index = D_AGSIZE,
- .conflicts = { { &dopts, D_AGCOUNT },
- { &dopts, D_CONCURRENCY },
+ .conflicts = { { &dopts, D_CONCURRENCY },
{ NULL, LAST_CONFLICT } },
.convert = true,
.minval = XFS_AG_MIN_BYTES,
@@ -1233,7 +1231,7 @@ validate_ag_geometry(
usage();
}

- if (agsize > dblocks) {
+ if (agsize > dblocks && agcount != 1) {
fprintf(stderr,
_("agsize (%lld blocks) too big, data area is %lld blocks\n"),
(long long)agsize, (long long)dblocks);
@@ -2703,7 +2701,8 @@ validate_supported(
* Filesystems should not have fewer than two AGs, because we need to
* have redundant superblocks.
*/
- if (mp->m_sb.sb_agcount < 2) {
+ if (mp->m_sb.sb_agcount < 2 &&
+ mp->m_sb.sb_agblocks <= mp->m_sb.sb_dblocks) {
fprintf(stderr,
_("Filesystem must have at least 2 superblocks for redundancy!\n"));
usage();
--
2.45.0