Re: "No space left on device" issue

[Dave Chinner <david@fromorbit.com>]

On Thu, Feb 17, 2022 at 11:52:36AM -0800, Manish Adkar wrote:
> Hello,
> 
> We have a 16 MB XFS test partition with both plenty of free space

Single AG filesystems are not a supported configuration.

> and inodes (419 out of 424), but we’re still getting the error "No
> space left on device" when trying to create new files (After the
> free space reaches 184320 bytes).

And there's your problem. Not enough free space for a inode
allocation transaction to guarantee success. Transaction
reservations need to take worst case reservations to guarantee
forwards progress, so if there's not enough space for a worst case
space usage operation, it will fail up front.

> It can then only edit and append
> any existing files. (There are good reasons why we want to avoid a
> larger partition.)

XFS is pretty much the worst choice you could make for this
filesystem.

> In the below “freesp” output, we can see that there are plenty of
> contiguous blocks for inode cluster allocation and hence the disk
> is not fragmented.
> 
> # xfs_db -r -c "freesp -s -a 0" /dev/ram1
>    from      to extents  blocks    pct
>       1       1       4       4   1.44
>       4       7       1       6   2.16
>     256     511       1     268  96.40
> total free extents 6
> total free blocks 278
> average free extent size 46.3333
> 
> I came across an email thread from the linux-xfs mailing list that
> described a similar issue. It mentioned, inodes are allocated in
> contiguous chunks of 64. Here, in this case, 1 inode takes 512 bytes
> of space and 1 block has 4096 bytes capacity. Hence there would be 8
> inodes per block (4096 / 512). Now, to allocate 64 inodes, 8 blocks
> would be needed (64 / 8).

Plus whatever is needed to update the inobt, the finbot, the 2 free
space btrees, the rmap btree and filling the AGFL just to allocate
and track the new inode cluster. THen we also have to take into
account the directory update, too, whihc means we might need to
allocate blocks for the directory data segment, the dir hash btree,i
the dir free space index, all the free space btree, rmap and AGFL
updates that dir block allocation requires, etc.

When we add up the *worst case* space requirement for a file create
operation, it quickly balloons out to be much larger than just the
space an inode cluster consumes.

> Looking at the above “freesp” output, we
> can see there are 8 contiguous blocks available so the issue must not
> be a fragmentation one. But the thread suggests, the allocation has
> an alignment requirement, and here, the blocks must be aligned to an

Your understanding is flawed - large contiguous extents can be cut
down to smaller aligned allocations.

Regardless, there are very good reasons we do not support such tiny
XFS filesystems nor attmept to use every last block available in the
filesystem.

If you truly need a 16MB filesystem for your application, *do not
use XFS*. Use a filesystem that is optimised for tiny devices.

Cheers,

Dave.

-- 
Dave Chinner
david@fromorbit.com