Re: [PATCH v2] nfs: test files written size as expected

Zorro Lang
2023-08-02T17:25:13Z

On Wed, Aug 02, 2023 at 09:36:40AM -0700, Darrick J. Wong wrote:
> On Wed, Aug 02, 2023 at 01:46:46PM +0800, Zorro Lang wrote:
> > Test nfs and its underlying fs, make sure file size as expected
> > after writting a file, and the speculative allocation space can
> > be shrunken.
> >
> > Signed-off-by: Zorro Lang <zlang@kernel.org>
> > ---
> >
> > Last year I sent a patch to fstests@, but it sometimes fails on the upstream
> > kernel that year:
> >
> > https://lore.kernel.org/fstests/Y3vTbHqT64gsQ573@magnolia/
> >
> > And we didn't get a proper reason for that, so that patch was blocked. Now
> > I found this case test passed on current upstream linux [1] (after loop
> > running it a whole night). So I think it's time to rebase and re-send this
> > patch to get review.
> >
> > Thanks,
> > Zorro
> >
> > [1]
> > FSTYP -- nfs
> > PLATFORM -- Linux/x86_64 xxxx 6.5.0-rc4 #1 SMP PREEMPT_DYNAMIC Tue Aug 1 15:32:55 EDT 2023
> > MKFS_OPTIONS -- xxxx.redhat.com:/mnt/xfstests/scratch/nfs-server
> > MOUNT_OPTIONS -- -o vers=4.2 -o context=system_u:object_r:root_t:s0 xxxx.redhat.com:/mnt/xfstests/scratch/nfs-server /mnt/xfstests/scratch/nfs-client
> >
> > nfs/002 4s ... 4s
> > Ran: nfs/002
> > Passed all 1 tests
> >
> > tests/nfs/002 | 46 ++++++++++++++++++++++++++++++++++++++++++++++
> > tests/nfs/002.out | 2 ++
> > 2 files changed, 48 insertions(+)
> > create mode 100755 tests/nfs/002
> > create mode 100644 tests/nfs/002.out
> >
> > diff --git a/tests/nfs/002 b/tests/nfs/002
> > new file mode 100755
> > index 00000000..b4b6554c
> > --- /dev/null
> > +++ b/tests/nfs/002
> > @@ -0,0 +1,46 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2023 Red Hat, Inc. All Rights Reserved.
> > +#
> > +# FS QA Test 002
> > +#
> > +# Make sure nfs gets expected file size after writting a big sized file. It's
> > +# not only testing nfs, test its underlying fs too. For example a known old bug
> > +# on xfs (underlying fs) caused nfs get larger file size (e.g. 16M) after
> > +# writting 10M data to a file. It's fixed by a series of patches around
> > +# 579b62faa5fb16 ("xfs: add background scanning to clear eofblocks inodes")
>
> Er... has this been banging around in the trunk for 11 years? ;)

Yeah, that's an old enough test case :-D I tried to tidy our internal test cases,
felt this case can be in fstests.

>
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto quick rw
> > +
> > +# real QA test starts here
> > +_supported_fs nfs
> > +# Need a series of patches related with this patch
> > +_fixed_by_kernel_commit 579b62faa5fb16 \
> > + "xfs: add background scanning to clear eofblocks inodes"
> > +_require_test
> > +
> > +localfile=$TEST_DIR/testfile.$seq
> > +rm -rf $localfile
> > +
> > +$XFS_IO_PROG -f -t -c "pwrite 0 10m" -c "fsync" $localfile >>$seqres.full 2>&1
> > +block_size=`stat -c '%B' $localfile`
> > +iblocks_expected=$((10 * 1024 * 1024 / $block_size))
> > +# Try several times for the speculative allocated file size can be shrunken
> > +res=1
> > +for ((i=0; i<10; i++));do
> > + iblocks_real=`stat -c '%b' $localfile`
> > + if [ "$iblocks_expected" = "$iblocks_real" ];then
>
> What happens if real < expected? Should there be some sort of bail out
> for unexpected things like that?

Hmm... I never thought that. I saw the real >= expected, is there any
chance to get real < expected?

>
> > + res=0
> > + break
> > + fi
> > + sleep 10
> > +done
>
> Though I guess the runtime is capped at ~100s so maybe it doesn't
> matter practically.

Mostly the test done in several seconds in my testing:

FSTYP -- nfs
PLATFORM -- Linux/x86_64 hp-dl360g9-06 6.5.0-rc4 #1 SMP PREEMPT_DYNAMIC Tue Aug 1 15:32:55 EDT 2023
MKFS_OPTIONS -- hp-dl360g9-06.rhts.eng.pek2.redhat.com:/mnt/xfstests/scratch/nfs-server
MOUNT_OPTIONS -- -o vers=4.2 -o context=system_u:object_r:root_t:s0 hp-dl360g9-06.rhts.eng.pek2.redhat.com:/mnt/xfstests/scratch/nfs-server /mnt/xfstests/scratch/nfs-client

nfs/002 5s ... 4s
Ran: nfs/002
Passed all 1 tests

>
> (What happens if xfs blockgc only runs every 5 minutes?)

How can can make that happen? If the 100s isn't enough, is there an upper
limit, or how to make an upper limit?

Thanks,
Zorro

>
> --D
>
> > +if [ $res -ne 0 ];then
> > + echo "Write $iblocks_expected blocks, but get $iblocks_real blocks"
> > +fi
> > +
> > +echo "Silence is golden"
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/nfs/002.out b/tests/nfs/002.out
> > new file mode 100644
> > index 00000000..61705c7c
> > --- /dev/null
> > +++ b/tests/nfs/002.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 002
> > +Silence is golden
> > --
> > 2.40.1
> >
>