Re: [PATCH 1/3] xfs: simplify extent allocation alignment

John Garry
2024-04-03T08:49:35Z

On 02/04/2024 22:26, Dave Chinner wrote:
>> And then this gives:
>> args->minlen=48 blen=64
> Which is perfectly reasonable - it fits the bounds specified just
> fine, and we'll get a 64 block allocation if that free space is
> exactly aligned. Otherwise we'll get a 48 block allocation.
>
>> But xfs_alloc_vextent_start_ag() -> xfs_alloc_vextent_iterate_ags() does not
>> seem to find something suitable.
> Entirely possible. The AGFL might have needed refilling, so there
> really wasn't enough blocks remaining for an aligned allocation to
> take place after doing that. That's a real ENOSPC condition, despite
> the best length sampling indicating that the allocation could be
> done.
>
> Remember, bestlen sampling is racy - it does not hold the AGF
> locked from the point of sampling to the point of allocation.

ok, I assumed that some lock was held.

> Hence
> when we finally go to do the allocation after setting it all up, we
> might have raced with another allocation that took the free space
> sampled during the bestlen pass and so then the allocation fails
> despite the setup saying it should succeed.

My test is single threaded, so I did not think that would be an issue.

>
> Fundamentally, if the filesystem's best free space length is the
> same size as the requested allocation,*failure is expected* and the
> fallback attempts progressively remove restrictions (such as
> alignment) to allow sub-optimal extents to be allocated down to
> minlen in size. Forced alignment turns off these fallbacks, so you
> are going to see hard ENOSPC errors the moment the filesystem runs
> out of contiguous free space extents large enough to hold aligned
> allocations.
>
> This can happen a -long- way away from a real enospc condition -
> depending on alignment constraints, you can start seeing this sort
> of behaviour (IIRC my math correctly) at around 70% full. The larger
> the alignment and the older the filesystem, the earlier this sort of
> ENOSPC event will occur.

For this scenario, statvfs returns - as a sample - f_blocks=73216,
f_bfree=19950, f_bavail=19950

So ~27% free.

>
> Use `xfs_spaceman -c 'freesp'` to dump the free space extent size
> histogram. That will quickly tell you if there is no remaining free
> extents large enough for an aligned 48 block allocation to succeed.
> With an alignment of 16 blocks, this requires at least a 63 block
> free space extent to succeed.

# xfs_spaceman -c 'freesp' /root/mnt2/
from to extents blocks pct
4 7 4 25 0.10
16 31 90 1440 5.77
32 63 12 400 1.60
64 127 1 64 0.26
512 1023 1 640 2.56
16384 22400 1 22390 89.71

>
> IOWs, we should expect ENOSPC to occur long before the filesystem
> reports that it is out of space when we are doing forced alignment
> allocation.

For my test, once ENOSPC occurs and statvfs tells us more than 10% space
free, we exit as something seems wrong. As you say, deducing an error
for this condition may not be the proper thing to do.

I do also note that if I then manually attempt to write the same data to
that same empty file after the test exits, it succeeds. So something
racy. I also notice that the FSB range we scan in
xfs_alloc_ag_vextent_size() -> xfs_alloc_compute_aligned() ->
xfs_extent_busy_trim() returns busy=1 when ENOSPC occurs - I have not
checked that further.

Thanks,
John