Re: [PATCH] xfs: update superblock counters correctly for !lazysbcount

[Brian Foster <bfoster@redhat.com>]

On Tue, Apr 27, 2021 at 09:13:18PM +0800, Gao Xiang wrote:
> On Tue, Apr 27, 2021 at 08:46:25AM -0400, Brian Foster wrote:
> > On Tue, Apr 27, 2021 at 09:12:01AM +0800, Gao Xiang wrote:
> > > From: Dave Chinner <dchinner@redhat.com>
> > > 
> > > Keep the mount superblock counters up to date for !lazysbcount
> > > filesystems so that when we log the superblock they do not need
> > > updating in any way because they are already correct.
> > > 
> > > It's found by what Zorro reported:
> > > 1. mkfs.xfs -f -l lazy-count=0 -m crc=0 $dev
> > > 2. mount $dev $mnt
> > > 3. fsstress -d $mnt -p 100 -n 1000 (maybe need more or less io load)
> > > 4. umount $mnt
> > > 5. xfs_repair -n $dev
> > > and I've seen no problem with this patch.
> > > 
> > > Signed-off-by: Dave Chinner <dchinner@redhat.com>
> > > Reported-by: Zorro Lang <zlang@redhat.com>
> > > Reviewed-by: Gao Xiang <hsiangkao@redhat.com>
> > > Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
> > > ---
> > 
> > Could you provide a bit more detail on the problem in the commit log?
> > From the description and code change, it seems like there is some
> > problem with doing the percpu aggregation in xfs_log_sb() on
> > !lazysbcount filesystems. Therefore this patch reserves that behavior
> > for lazysbcount, and instead enables per-transaction updates in the
> > !lazysbcount specific cleanup path. Am I following that correctly?
> 
> This patch inherited from Dave's patch [1] (and I added reproduable
> steps),
> https://lore.kernel.org/r/20210422014446.GZ63242@dread.disaster.area
> 
> More details see my original patch v2:
> https://lore.kernel.org/r/20210420110855.2961626-1-hsiangkao@redhat.com
> 

Ok, thanks. So the bit about xfs_log_sb() is to avoid an incorrect
overwrite of the in-core sb counters from the percpu counters on
!lazysbcount. The xfs_trans_apply_sb_deltas() function already applies
the transaction deltas to the on-disk superblock buffer, so the change
to xfs_trans_unreserve_and_mod_sb() is basically to apply those same
deltas to the in-core superblock so they are consistent in the
!lazysbcount case... yes? If I'm following that correctly, this looks
good to me:

Reviewed-by: Brian Foster <bfoster@redhat.com>

> Thanks,
> Gao Xiang
> 
> > 
> > Brian
> > 
> > > 
> > > As per discussion earilier [1], use the way Dave suggested instead.
> > > Also update the line to
> > > 	mp->m_sb.sb_fdblocks += tp->t_fdblocks_delta + tp->t_res_fdblocks_delta;
> > > so it can fix the case above.
> > > 
> > > with XFS debug off, xfstests auto testcases fail on my loop-device-based
> > > testbed with this patch and Darrick's [2]:
> > > 
> > > generic/095 generic/300 generic/600 generic/607 xfs/073 xfs/148 xfs/273
> > > xfs/293 xfs/491 xfs/492 xfs/495 xfs/503 xfs/505 xfs/506 xfs/514 xfs/515
> > > 
> > > MKFS_OPTIONS="-mcrc=0 -llazy-count=0"
> > > 
> > > and these testcases above still fail without these patches or with
> > > XFS debug on, so I've seen no regression due to this patch.
> > > 
> > > [1] https://lore.kernel.org/r/20210422030102.GA63242@dread.disaster.area/
> > > [2] https://lore.kernel.org/r/20210425154634.GZ3122264@magnolia/
> > > 
> > >  fs/xfs/libxfs/xfs_sb.c | 16 +++++++++++++---
> > >  fs/xfs/xfs_trans.c     |  3 +++
> > >  2 files changed, 16 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/fs/xfs/libxfs/xfs_sb.c b/fs/xfs/libxfs/xfs_sb.c
> > > index 60e6d255e5e2..dfbbcbd448c1 100644
> > > --- a/fs/xfs/libxfs/xfs_sb.c
> > > +++ b/fs/xfs/libxfs/xfs_sb.c
> > > @@ -926,9 +926,19 @@ xfs_log_sb(
> > >  	struct xfs_mount	*mp = tp->t_mountp;
> > >  	struct xfs_buf		*bp = xfs_trans_getsb(tp);
> > >  
> > > -	mp->m_sb.sb_icount = percpu_counter_sum(&mp->m_icount);
> > > -	mp->m_sb.sb_ifree = percpu_counter_sum(&mp->m_ifree);
> > > -	mp->m_sb.sb_fdblocks = percpu_counter_sum(&mp->m_fdblocks);
> > > +	/*
> > > +	 * Lazy sb counters don't update the in-core superblock so do that now.
> > > +	 * If this is at unmount, the counters will be exactly correct, but at
> > > +	 * any other time they will only be ballpark correct because of
> > > +	 * reservations that have been taken out percpu counters. If we have an
> > > +	 * unclean shutdown, this will be corrected by log recovery rebuilding
> > > +	 * the counters from the AGF block counts.
> > > +	 */
> > > +	if (xfs_sb_version_haslazysbcount(&mp->m_sb)) {
> > > +		mp->m_sb.sb_icount = percpu_counter_sum(&mp->m_icount);
> > > +		mp->m_sb.sb_ifree = percpu_counter_sum(&mp->m_ifree);
> > > +		mp->m_sb.sb_fdblocks = percpu_counter_sum(&mp->m_fdblocks);
> > > +	}
> > >  
> > >  	xfs_sb_to_disk(bp->b_addr, &mp->m_sb);
> > >  	xfs_trans_buf_set_type(tp, bp, XFS_BLFT_SB_BUF);
> > > diff --git a/fs/xfs/xfs_trans.c b/fs/xfs/xfs_trans.c
> > > index bcc978011869..1e37aa8eca5a 100644
> > > --- a/fs/xfs/xfs_trans.c
> > > +++ b/fs/xfs/xfs_trans.c
> > > @@ -629,6 +629,9 @@ xfs_trans_unreserve_and_mod_sb(
> > >  
> > >  	/* apply remaining deltas */
> > >  	spin_lock(&mp->m_sb_lock);
> > > +	mp->m_sb.sb_fdblocks += tp->t_fdblocks_delta + tp->t_res_fdblocks_delta;
> > > +	mp->m_sb.sb_icount += idelta;
> > > +	mp->m_sb.sb_ifree += ifreedelta;
> > >  	mp->m_sb.sb_frextents += rtxdelta;
> > >  	mp->m_sb.sb_dblocks += tp->t_dblocks_delta;
> > >  	mp->m_sb.sb_agcount += tp->t_agcount_delta;
> > > -- 
> > > 2.27.0
> > > 
> > 
>