Re: Inconsistent "EXPERIMENTAL online shrink feature in use. Use at your own risk" alert

[Dave Chinner <david@fromorbit.com>]

On Wed, Mar 09, 2022 at 12:22:00PM -0600, Eric Sandeen wrote:
> So a weird thing here is that I think your logs show the xfs_growfs command
> happening immediately after mount, and it doesn't have any size specified, 
> so I can't tell if the intent was to shrink - but I don't think so:
> 
> Mar  6 19:59:21 tdm emhttpd: shcmd (81): mkdir -p /mnt/disk1
> Mar  6 19:59:21 tdm emhttpd: shcmd (82): mount -t xfs -o noatime /dev/md1 /mnt/disk1
> Mar  6 19:59:21 tdm kernel: SGI XFS with ACLs, security attributes, no debug enabled
> Mar  6 19:59:21 tdm kernel: XFS (md1): Mounting V5 Filesystem
> Mar  6 19:59:21 tdm kernel: XFS (md1): Ending clean mount
> Mar  6 19:59:21 tdm emhttpd: shcmd (83): xfs_growfs /mnt/disk1
> Mar  6 19:59:21 tdm kernel: xfs filesystem being mounted at /mnt/disk1 supports timestamps until 2038 (0x7fffffff)
> Mar  6 19:59:21 tdm root: xfs_growfs: XFS_IOC_FSGROWFSDATA xfsctl failed: No space left on device
> Mar  6 19:59:21 tdm root: meta-data=/dev/md1               isize=512  agcount=32, agsize=137330687 blks
> Mar  6 19:59:21 tdm root:          =                       sectsz=512  attr=2, projid32bit=1
> Mar  6 19:59:21 tdm root:          =                       crc=1  finobt=1, sparse=1, rmapbt=0
> Mar  6 19:59:21 tdm root:          =                       reflink=1  bigtime=0 inobtcount=0
> Mar  6 19:59:21 tdm root: data     =                       bsize=4096  blocks=4394581984, imaxpct=5
> Mar  6 19:59:21 tdm root:          =                       sunit=1  swidth=32 blks
> Mar  6 19:59:21 tdm root: naming   =version 2              bsize=4096  ascii-ci=0, ftype=1
> Mar  6 19:59:21 tdm root: log      =internal log           bsize=4096  blocks=521728, version=2
> Mar  6 19:59:21 tdm root:          =                       sectsz=512  sunit=1 blks, lazy-count=1
> Mar  6 19:59:21 tdm root: realtime =none                   extsz=4096  blocks=0, rtextents=0
> Mar  6 19:59:21 tdm emhttpd: shcmd (83): exit status: 1
> Mar  6 19:59:21 tdm kernel: XFS (md1): EXPERIMENTAL online shrink feature in use. Use at your own risk!
> 
> 
> We issue the EXPERIMENTAL message if the block delta is <= 0 (I'm not sure why
> it's done if delta == 0 and I wonder if it should instead be < 0).

Because if the growfs size is unchanged (i.e delta will be zero), we
don't even call xfs_growfs_data_private(), so the warning will not
be emitted:

        if (in->newblocks != mp->m_sb.sb_dblocks) {
		error = xfs_growfs_data_private(mp, in);
		if (error)
			goto out_error;
	}

If it is getting ENOSPC as an error then either the filesystem is
either:

- 100% full and xfs_trans_alloc() fails (unlikely)
- it is a single AG shrink, the remaining space is not contiguous
  and so fails allocation that removes it from the free space tree:

xfs_ag_shrink_space()
...
        /* internal log shouldn't also show up in the free space btrees */
        error = xfs_alloc_vextent(&args);
        if (!error && args.agbno == NULLAGBLOCK)
                error = -ENOSPC;

- after shrink of the AG, the call to xfs_ag_resv_init() fails
  because there isn't enough free space for metadata reservations
  in that AG anymore. i.e. it will only allow freeing from the last
  AG until reservation space has been regained.

So, yeah, a single AG shrink can give ENOSPC for several reasons,
which leads me to think that the unraid device underlying the
filesystem has changed size (for whatever reason) and growfs is just
saying "you haven't emptied the space at the end of the filesystem
before you tried to shrink the fs"...

> I'm wondering if we have some path through xfs_growfs_data_private() that calculates
> a delta < 0 unintentionally, or if we get there with delta == 0 and generate the
> warning message.

Nope, we're not even getting there for the delta == 0 case...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com