[PATCH 2/2] xfs: consistent low ag space behavior for sparse inode chunk allocs

[Brian Foster <bfoster@redhat.com>]

Matt Fleming reports a filesystem shutdown due to inobt block
allocation failure during sparse chunk allocation. Inode creation
can involve multiple allocations in a transaction via the initial
chunk allocation and inode btree growth via the subsequent inobt
record insertion. Technically this should be safe as the chunk
allocation sets the allocation minleft parameter to the max depth of
the inode btree, which means the allocation selects an AG only if
there is enough free space for record insertion after the
allocation. The record insertion naturally occurs in the same AG as
the allocation and the associated AGF is locked and held by the
current transaction.

The situation that reproduces this failure is a bit of a corner
case. The allocation and inode btrees are all completely full and
require a split on the next insertion. The AG has just enough
available space to satisfy inode creation through sparse allocation
(i.e. 7 blocks in this example). The block allocation occurs within
an existing free space record, splitting the free space record into
two and triggering the aforementioned allocbt splits.

The sparse chunk allocation consumes 4 blocks for the chunk, and 4
blocks from the AGFL for the two allocbt splits. This leaves the
AGFL with 4 remaining blocks and 3 available blocks in the AG. The
record insertion attempts block allocation for the inobt split, but
even though space is available, the geometry change has changed the
minimum allocation requirements enforced by
xfs_alloc_fix_freelist(). The min freelist value jumps from 8 to 12
due to the alloc btree level increases, so the available space
calculation goes from something like this before the allocation:

	free + agfl - res - minfree - minleft = avail
	2514 + 8 - 2505 - 8 - 2 = 7

... to this after it:

	2510 + 4 - 2505 - 12 - 0 = -3

Essentially the minleft value of 2 for the chunk alloc is not
sufficient to overcome the additional requirements imposed on the
subsequent inobt alloc due to the allocbt geometry changes caused by
the first.

Technically this shouldn't be a concern because a full split of any
of the allocation btrees in one transaction means that we're not
going to see another full split in the lifetime of that transaction,
but the allocation path does not behave that way.

Absent of a clear way to lock in the agfl state for the particular
case of multiple allocations into a single AG, this patch works
around the problem by increasing the minleft requirement for sparse
chunk allocations. Specifically, we increase minleft by the number
of blocks that the alloc length was reduced by. The reasoning behind
this is to generally preserve the AG level no space behavior between
full and sparse inode chunk allocations.

The purpose of sparse inode chunks is to facilitate allocation in
AGs under severe free space fragmentation and thus prevent premature
-ENOSPC conditions across the broader fs, not necessarily squeeze
every last block out of any single AG. Therefore, this helps ensure
the low AG space boundary conditions for sparse inode chunks are
relatively well tested compared to full inode chunks.

Fixes: 56d1115c9bc7 ("xfs: allocate sparse inode chunks on full chunk allocation failure")
Reported-by: Matt Fleming <matt@readmodwrite.com>
Signed-off-by: Brian Foster <bfoster@redhat.com>
---
 fs/xfs/libxfs/xfs_ialloc.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/fs/xfs/libxfs/xfs_ialloc.c b/fs/xfs/libxfs/xfs_ialloc.c
index 633b2d6e42c5..7e71c33fbefa 100644
--- a/fs/xfs/libxfs/xfs_ialloc.c
+++ b/fs/xfs/libxfs/xfs_ialloc.c
@@ -840,6 +840,14 @@ xfs_ialloc_ag_alloc(
 		args.minlen = igeo->ialloc_min_blks;
 		args.maxlen = args.minlen;
 
+		/*
+		 * Bump minleft by the alloc size delta to maintain consistent
+		 * out of space behavior with normal sized chunks. This isn't a
+		 * requirement, but helps avoid sparse chunk and inobt block
+		 * allocation quirks at or close to AG depletion.
+		 */
+		args.minleft += igeo->ialloc_blks - igeo->ialloc_min_blks;
+
 		/*
 		 * The inode record will be aligned to full chunk size. We must
 		 * prevent sparse allocation from AG boundaries that result in
-- 
2.55.0