Re: [PATCH v2 1/4] generic/377: Add copy to new file test

["Darrick J. Wong" <darrick.wong@oracle.com>]

On Fri, Sep 09, 2016 at 02:21:36PM +0800, Eryu Guan wrote:
> On Wed, Sep 07, 2016 at 03:56:16PM -0400, Anna Schumaker wrote:
> > This test copies data from various points in a source file to a new
> > file.  This is useful for testing the basics of copy_file_range().
> > 
> > Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
> > ---
> >  tests/generic/377     | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/generic/377.out |  21 +++++++++++
> >  tests/generic/group   |   1 +
> >  3 files changed, 124 insertions(+)
> >  create mode 100644 tests/generic/377
> 
> File mode of test case should be 755, i.e. it has x permission. So "git
> diff" won't complain the mode change after running these tests.
> 
> >  create mode 100644 tests/generic/377.out
> > 
> > diff --git a/tests/generic/377 b/tests/generic/377
> > new file mode 100644
> > index 0000000..352b9bb
> > --- /dev/null
> > +++ b/tests/generic/377
> > @@ -0,0 +1,102 @@
> > +#!/bin/bash
> > +# FS QA Test No. 377
> > +#
> > +# Tests vfs_copy_file_range():
> > +#   - Copy a file
> > +#   - Copy beginning of original to new file
> > +#   - Copy middle of original to a new file
> > +#   - Copy end of original to new file
> > +#-----------------------------------------------------------------------
> > +# Copyright (c) 2016 Netapp, Inc. All rights reserved.
> > +#
> > +# This program is free software; you can redistribute it and/or
> > +# modify it under the terms of the GNU General Public License as
> > +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it would be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +# GNU General Public License for more details.
> > +#
> > +# You should have received a copy of the GNU General Public License
> > +# along with this program; if not, write the Free Software Foundation,
> > +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> > +#-----------------------------------------------------------------------
> > +#
> > +
> > +seq=`basename $0`
> > +seqres=$RESULT_DIR/$seq
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1    # failure is the default!
> > +trap "_cleanup; exit \$status" 0 1 2 3 15
> > +
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -rf $tmp.*
> > +}
> > +
> > +# get standard environment
> > +. common/rc
> > +. common/filter
> > +
> > +# real QA test starts here
> > +_supported_os Linux
> 
> Need "_supported_fs generic" in all these tests.
> 
> > +
> > +_require_xfs_io_command "copy_range"
> 
> Do we need to test the support status on kernel side? e.g. what happens
> if filesystems have no "copy_file_range" implemented? Seems
> copy_file_range falls back to splice in this case, but I'm not sure. If
> so I think it's OK to have no kernel side detection.

But... it's a totally new syscall, so _require_io_command should actually try
calling it so that we can _notrun on old kernels.

> > +_require_test
> > +
> > +testdir=$TEST_DIR/test-$seq
> > +rm -rf $testdir
> > +mkdir $testdir
> > +
> > +_checksum_files()
> 
> We usually name local functions without the leading "_".
> 
> > +{
> > +	for f in $*; do
> > +		md5sum $testdir/$f | _filter_test_dir
> > +	done
> > +}

Or perhaps just "md5sum file copy | _filter_test_dir" directly?

--D

> > +
> > +rm -f $seqres.full
> > +
> > +echo "Create the original file and then copy"
> > +$XFS_IO_PROG -f -c 'pwrite -S 0x61 0    1000' $testdir/file >> $seqres.full 2>&1
> > +$XFS_IO_PROG -f -c 'pwrite -S 0x62 1000 1000' $testdir/file >> $seqres.full 2>&1
> > +$XFS_IO_PROG -f -c 'pwrite -S 0x63 2000 1000' $testdir/file >> $seqres.full 2>&1
> > +$XFS_IO_PROG -f -c 'pwrite -S 0x64 3000 1000' $testdir/file >> $seqres.full 2>&1
> > +$XFS_IO_PROG -f -c 'pwrite -S 0x65 4000 1000' $testdir/file >> $seqres.full 2>&1
> > +$XFS_IO_PROG -f -c "copy_range $testdir/file" "$testdir/copy"
> > +cmp $testdir/file $testdir/copy
> > +echo "Original md5sums:"
> > +_checksum_files file copy
> 
> I saw test failures when testing on XFS and btrfs, 4.8-rc3 kernel, and
> similar failures in other tests. Are these failures expected?
> 
> [root@dhcp-66-86-11 xfstests]# diff -u tests/generic/377.out /root/workspace/xfstests/results//xfs_4k_crc/generic/377.out.bad
> --- tests/generic/377.out       2016-09-09 12:17:40.790000000 +0800
> +++ /root/workspace/xfstests/results//xfs_4k_crc/generic/377.out.bad    2016-09-09 14:14:48.483000000 +0800
> @@ -4,18 +4,22 @@
>  e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/file
>  e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/copy
>  Copy beginning of original file
> +cmp: EOF on /mnt/testarea/test/test-377/beginning
>  md5sums after copying beginning:
>  e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/file
> -cabe45dcc9ae5b66ba86600cca6b8ba8  TEST_DIR/test-377/beginning
> +d41d8cd98f00b204e9800998ecf8427e  TEST_DIR/test-377/beginning
>  Copy middle of original file
> +cmp: EOF on /mnt/testarea/test/test-377/middle
>  md5sums after copying middle:
>  e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/file
> -4197de9da5badfc302715486b82bcdf1  TEST_DIR/test-377/middle
> +d41d8cd98f00b204e9800998ecf8427e  TEST_DIR/test-377/middle
>  Copy end of original file
> +cmp: EOF on /mnt/testarea/test/test-377/end
>  md5sums after copying end:
>  e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/file
> -e68d4a150c4e42f4f9ea3ffe4c9cf4ed  TEST_DIR/test-377/end
> +d41d8cd98f00b204e9800998ecf8427e  TEST_DIR/test-377/end
>  Copy beyond end of original file
> +cmp: EOF on /mnt/testarea/test/test-377/end
>  md5sums after copying beyond:
>  e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/file
> -e68d4a150c4e42f4f9ea3ffe4c9cf4ed  TEST_DIR/test-377/beyond
> +d41d8cd98f00b204e9800998ecf8427e  TEST_DIR/test-377/beyond
> 
> > +
> > +echo "Copy beginning of original file"
> > +$XFS_IO_PROG -f -c "copy_range -l 1000 $testdir/file" "$testdir/beginning"
> > +cmp -n 1000 $testdir/file $testdir/beginning
> > +echo "md5sums after copying beginning:"
> > +_checksum_files file beginning
> > +
> > +echo "Copy middle of original file"
> > +$XFS_IO_PROG -f -c "copy_range -s 1000 -l 3000 $testdir/file" "$testdir/middle"
> > +cmp -n 3000 $testdir/file $testdir/middle 1000
> > +echo "md5sums after copying middle:"
> > +_checksum_files file middle
> > +
> > +echo "Copy end of original file"
> > +$XFS_IO_PROG -f -c "copy_range -s 4000 -l 1000 $testdir/file" "$testdir/end"
> > +cmp -n 1000 $testdir/file $testdir/end 4000
> > +echo "md5sums after copying end:"
> > +_checksum_files file end
> > +
> > +echo "Copy beyond end of original file"
> > +$XFS_IO_PROG -f -c "copy_range -s 4000 -l 2000 $testdir/file" "$testdir/beyond"
> 
> I noticed that xfs_io hung on copying beyond EOF when testing with ext4
> and NFSv4.1, xfs_io was in "R" state for 10 minutes until I killed it.
> Is it kernel bug or test is not suitable on such filesystems?
> 
> Thanks,
> Eryu
> 
> > +cmp -n 1000 $testdir/file $testdir/end 4000
> > +echo "md5sums after copying beyond:"
> > +_checksum_files file beyond
> > +
> > +#success, all done
> > +status=0
> > +exit
> > diff --git a/tests/generic/377.out b/tests/generic/377.out
> > new file mode 100644
> > index 0000000..8ad8b53
> > --- /dev/null
> > +++ b/tests/generic/377.out
> > @@ -0,0 +1,21 @@
> > +QA output created by 377
> > +Create the original file and then copy
> > +Original md5sums:
> > +e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/file
> > +e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/copy
> > +Copy beginning of original file
> > +md5sums after copying beginning:
> > +e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/file
> > +cabe45dcc9ae5b66ba86600cca6b8ba8  TEST_DIR/test-377/beginning
> > +Copy middle of original file
> > +md5sums after copying middle:
> > +e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/file
> > +4197de9da5badfc302715486b82bcdf1  TEST_DIR/test-377/middle
> > +Copy end of original file
> > +md5sums after copying end:
> > +e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/file
> > +e68d4a150c4e42f4f9ea3ffe4c9cf4ed  TEST_DIR/test-377/end
> > +Copy beyond end of original file
> > +md5sums after copying beyond:
> > +e11fbace556cba26bf0076e74cab90a3  TEST_DIR/test-377/file
> > +e68d4a150c4e42f4f9ea3ffe4c9cf4ed  TEST_DIR/test-377/beyond
> > diff --git a/tests/generic/group b/tests/generic/group
> > index bad71bc..dec0d64 100644
> > --- a/tests/generic/group
> > +++ b/tests/generic/group
> > @@ -379,3 +379,4 @@
> >  374 auto quick clone dedupe
> >  375 auto quick acl
> >  376 auto quick metadata
> > +377 auto quick copy
> > -- 
> > 2.9.3
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe fstests" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html