Re: [PATCH 02/11] xfs: Check for extent overflow when trivally adding a new extent

Darrick J. Wong
2020-11-18T02:30:48Z

On Tue, Nov 17, 2020 at 07:42:45PM +0530, Chandan Babu R wrote:
> On Saturday 14 November 2020 5:54:20 AM IST Darrick J. Wong wrote:
> > On Fri, Nov 13, 2020 at 04:56:54PM +0530, Chandan Babu R wrote:
> > > This test verifies that XFS does not cause inode fork's extent count to
> > > overflow when adding a single extent while there's no possibility of
> > > splitting an existing mapping (limited to non-realtime files).
> > >
> > > Signed-off-by: Chandan Babu R <chandanrlinux@gmail.com>
> > > ---
> > > tests/xfs/522 | 214 ++++++++++++++++++++++++++++++++++++++++++++++
> > > tests/xfs/522.out | 24 ++++++
> > > tests/xfs/group | 1 +
> > > 3 files changed, 239 insertions(+)
> > > create mode 100755 tests/xfs/522
> > > create mode 100644 tests/xfs/522.out
> > >
> > > diff --git a/tests/xfs/522 b/tests/xfs/522
> > > new file mode 100755
> > > index 00000000..a54fe136
> > > --- /dev/null
> > > +++ b/tests/xfs/522
> > > @@ -0,0 +1,214 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2020 Chandan Babu R. All Rights Reserved.
> > > +#
> > > +# FS QA Test 522
> > > +#
> > > +# Verify that XFS does not cause inode fork's extent count to overflow when
> > > +# adding a single extent while there's no possibility of splitting an existing
> > > +# mapping (limited to non-realtime files).
> > > +
> > > +seq=`basename $0`
> > > +seqres=$RESULT_DIR/$seq
> > > +echo "QA output created by $seq"
> > > +
> > > +here=`pwd`
> > > +tmp=/tmp/$$
> > > +status=1 # failure is the default!
> > > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > > +
> > > +_cleanup()
> > > +{
> > > + cd /
> > > + rm -f $tmp.*
> > > +}
> > > +
> > > +# get standard environment, filters and checks
> > > +. ./common/rc
> > > +. ./common/filter
> > > +. ./common/quota
> > > +. ./common/inject
> > > +
> > > +# remove previous $seqres.full before test
> > > +rm -f $seqres.full
> > > +
> > > +# real QA test starts here
> > > +
> > > +_supported_fs xfs
> > > +_require_scratch
> > > +_require_xfs_quota
> > > +_require_xfs_debug
> > > +_require_xfs_io_command "falloc"
> > > +_require_xfs_io_error_injection "reduce_max_iextents"
> > > +_require_xfs_io_error_injection "bmap_alloc_minlen_extent"
> > > +
> > > +delalloc_to_written_extent()
> > > +{
> > > + echo "* Delalloc to written extent conversion"
> > > +
> > > + echo "Format and mount fs"
> > > + _scratch_mkfs >> $seqres.full
> > > + _scratch_mount >> $seqres.full
> > > +
> > > + testfile=$SCRATCH_MNT/testfile
> > > + bsize=$(_get_block_size $SCRATCH_MNT)
> > > +
> > > + echo "Inject reduce_max_iextents error tag"
> > > + xfs_io -x -c 'inject reduce_max_iextents' $SCRATCH_MNT
> > > +
> > > + nr_blks=$((15 * 2))
> > > +
> > > + echo "Create fragmented file"
> > > + for i in $(seq 0 2 $((nr_blks - 1))); do
> > > + xfs_io -f -c "pwrite $((i * bsize)) $bsize" -c fsync $testfile >> $seqres.full 2>&1
> > > + [[ $? != 0 ]] && break
> > > + done
> > > +
> > > + testino=$(stat -c "%i" $testfile)
> > > +
> > > + _scratch_unmount >> $seqres.full
> > > +
> > > + echo "Verify \$testfile's extent count"
> > > +
> > > + nextents=$(_scratch_get_iext_count $testino data || \
> > > + _fail "Unable to obtain inode fork's extent count")
> > > + if (( $nextents > 10 )); then
> > > + echo "Extent count overflow check failed: nextents = $nextents"
> > > + exit 1
> > > + fi
> > > +}
> > > +
> > > +falloc_unwritten_extent()
> > > +{
> > > + echo "* Fallocate of unwritten extents"
> > > +
> > > + echo "Format and mount fs"
> > > + _scratch_mkfs >> $seqres.full
> > > + _scratch_mount >> $seqres.full
> > > +
> > > + testfile=$SCRATCH_MNT/testfile
> > > + bsize=$(_get_block_size $SCRATCH_MNT)
> > > +
> > > + echo "Inject reduce_max_iextents error tag"
> > > + xfs_io -x -c 'inject reduce_max_iextents' $SCRATCH_MNT
> > > +
> > > + nr_blks=$((15 * 2))
> > > +
> > > + echo "Fallocate fragmented file"
> > > + for i in $(seq 0 2 $((nr_blks - 1))); do
> > > + xfs_io -f -c "falloc $((i * bsize)) $bsize" $testfile >> $seqres.full 2>&1
> > > + [[ $? != 0 ]] && break
> > > + done
> > > +
> > > + testino=$(stat -c "%i" $testfile)
> > > +
> > > + _scratch_unmount >> $seqres.full
> > > +
> > > + echo "Verify \$testfile's extent count"
> > > +
> > > + nextents=$(_scratch_get_iext_count $testino data || \
> > > + _fail "Unable to obtain inode fork's extent count")
> > > + if (( $nextents > 10 )); then
> > > + echo "Extent count overflow check failed: nextents = $nextents"
> > > + exit 1
> > > + fi
> > > +}
> > > +
> > > +quota_inode_extend()
> > > +{
> > > + echo "* Extend quota inodes"
> > > +
> > > + echo "Format and mount fs"
> > > + _scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seqres.full
> > > + _scratch_mount -o uquota >> $seqres.full
> > > +
> > > + testfile=$SCRATCH_MNT/testfile
> > > + bsize=$(_get_block_size $SCRATCH_MNT)
> > > +
> > > + echo "Consume free space"
> > > + dd if=/dev/zero of=${testfile} bs=${bsize} >> $seqres.full 2>&1
> > > + sync
> > > +
> > > + echo "Create fragmented filesystem"
> > > + $here/src/punch-alternating $testfile >> $seqres.full
> > > + sync
> > > +
> > > + echo "Inject reduce_max_iextents error tag"
> > > + xfs_io -x -c 'inject reduce_max_iextents' $SCRATCH_MNT
> > > +
> > > + echo "Inject bmap_alloc_minlen_extent error tag"
> > > + xfs_io -x -c 'inject bmap_alloc_minlen_extent' $SCRATCH_MNT
> > > +
> > > + nr_blks=20
> > > +
> > > + # This is a rough calculation; It doesn't take block headers into
> > > + # consideration.
> > > + # gdb -batch vmlinux -ex 'print sizeof(struct xfs_disk_dquot)'
> > > + # $1 = 104
> > > + nr_quotas_per_block=$((bsize / 104))
> >
> > That's sizeof(xfs_dqblk_t) you want, and it's 136 bytes long.
>
> Ah ok. I had missed that. Thanks for pointing that out.
>
> >
> > > + nr_quotas=$((nr_quotas_per_block * nr_blks))
> > > +
> > > + echo "Extend uquota file"
> > > + for i in $(seq 0 $nr_quotas); do
> >
> > You only have to initialize the first dquot in a dquot file block in
> > order to allocate the whole block, so you could speed this up with
> > "seq 0 $nr_quotas_per_block $nr_quotas".
>
> Ok. Thanks for the suggestion.
>
> >
> > > + chown $i $testfile >> $seqres.full 2>&1
> > > + [[ $? != 0 ]] && break
> > > + done
> > > +
> > > + _scratch_unmount >> $seqres.full
> > > +
> > > + echo "Verify uquota inode's extent count"
> > > + uquotino=$(_scratch_xfs_db -c sb -c "print uquotino")
> > > + uquotino=${uquotino##uquotino = }
> > > +
> > > + nextents=$(_scratch_get_iext_count $uquotino data || \
> > > + _fail "Unable to obtain inode fork's extent count")
> > > + if (( $nextents > 10 )); then
> > > + echo "Extent count overflow check failed: nextents = $nextents"
> > > + exit 1
> > > + fi
> > > +}
> > > +
> > > +directio_write()
> > > +{
> > > + echo "* Directio write"
> > > +
> > > + echo "Format and mount fs"
> > > + _scratch_mkfs >> $seqres.full
> > > + _scratch_mount >> $seqres.full
> > > +
> > > + testfile=$SCRATCH_MNT/testfile
> > > + bsize=$(_get_block_size $SCRATCH_MNT)
> > > +
> > > + echo "Inject reduce_max_iextents error tag"
> > > + xfs_io -x -c 'inject reduce_max_iextents' $SCRATCH_MNT
> > > +
> > > + nr_blks=$((15 * 2))
> > > +
> > > + echo "Create fragmented file via directio writes"
> > > + for i in $(seq 0 2 $((nr_blks - 1))); do
> > > + xfs_io -d -f -c "pwrite $((i * bsize)) $bsize" -c fsync $testfile >> $seqres.full 2>&1
> >
> > $XFS_IO_PROG -d -f -s -c "pwrite ..." $testfile
> >
> > "-s" is an undocumented flag that makes the writes synchronous.
>
> Ok. I will fix that.
>
> >
> > > + [[ $? != 0 ]] && break
> > > + done
> > > +
> > > + testino=$(stat -c "%i" $testfile)
> > > +
> > > + _scratch_unmount >> $seqres.full
> > > +
> > > + echo "Verify \$testfile's extent count"
> > > +
> > > + nextents=$(_scratch_get_iext_count $testino data || \
> >
> > $XFS_IO_PROG -c 'stat' $testfile | grep nextents ?
>
> I agree. I will replace the above with xfs_io's stat command. But for special
> inodes like quota and realtime bitmap and summary inodes we would still
> require the helper function to obtain the extent count.

<nod>

> >
> > > + _fail "Unable to obtain inode fork's extent count")
> > > + if (( $nextents > 10 )); then
> > > + echo "Extent count overflow check failed: nextents = $nextents"
> > > + exit 1
> > > + fi
> > > +}
> > > +
> > > +delalloc_to_written_extent
> > > +falloc_unwritten_extent
> > > +quota_inode_extend
> > > +directio_write
> >
> > I wonder if these should be separate tests, since they each format the
> > scratch fs? Or could you format the scratch fs once and test four
> > different files?
>
> Quota inode test is the only one which requires bmap_alloc_minlen_extent
> error tag. Other tests here have the following pattern,
> - mkfs and mount fs
> - Fragment fs space
> - Inject reduce_max_iextents error tag
> - Work on a test file
> Here we allocate individual blocks.
> - Umount
> - Check inode fork extent count.
> So we can have just one mkfs/mount overall by removing the testfile created by
> each test and also quota inode test can be moved to the last so that
> bmap_alloc_minlen_extent error tag can be applied only to that test.

<nod> sounds good to me.

I think you can disable errortags too, so the quota inode extend test
doesn't necessarily have to go last.

--D

> >
> > --D
> >
> > > +
> > > +# success, all done
> > > +status=0
> > > +exit
> > > diff --git a/tests/xfs/522.out b/tests/xfs/522.out
> > > new file mode 100644
> > > index 00000000..98791aae
> > > --- /dev/null
> > > +++ b/tests/xfs/522.out
> > > @@ -0,0 +1,24 @@
> > > +QA output created by 522
> > > +* Delalloc to written extent conversion
> > > +Format and mount fs
> > > +Inject reduce_max_iextents error tag
> > > +Create fragmented file
> > > +Verify $testfile's extent count
> > > +* Fallocate of unwritten extents
> > > +Format and mount fs
> > > +Inject reduce_max_iextents error tag
> > > +Fallocate fragmented file
> > > +Verify $testfile's extent count
> > > +* Extend quota inodes
> > > +Format and mount fs
> > > +Consume free space
> > > +Create fragmented filesystem
> > > +Inject reduce_max_iextents error tag
> > > +Inject bmap_alloc_minlen_extent error tag
> > > +Extend uquota file
> > > +Verify uquota inode's extent count
> > > +* Directio write
> > > +Format and mount fs
> > > +Inject reduce_max_iextents error tag
> > > +Create fragmented file via directio writes
> > > +Verify $testfile's extent count
> > > diff --git a/tests/xfs/group b/tests/xfs/group
> > > index b89c0a4e..1831f0b5 100644
> > > --- a/tests/xfs/group
> > > +++ b/tests/xfs/group
> > > @@ -519,3 +519,4 @@
> > > 519 auto quick reflink
> > > 520 auto quick reflink
> > > 521 auto quick realtime growfs
> > > +522 auto quick quota
> >
>
>
> --
> chandan
>
>
>