[PATCH 0/3] O_TMPFILE tests, v2

[PATCH 0/3] O_TMPFILE tests, v2

[Christoph Hellwig]

Now has proper feature support checking for xfs_io and the kernel, addressed
Dave's various minor comments for the second test.  I've left the repair
output as-is for the move of the zero link count files to lost+found.  While
I plan to fix that it's something that is generic for all zero link count
files and shouldn't cause the O_TMPFILE test to fail.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 1/3] common: add flink support to _require_xfs_io_command

[Christoph Hellwig]

[-- Attachment #1: 0001-xfstests-add-flink-support-to-_require_xfs_io_comman.patch --]
[-- Type: text/plain, Size: 643 bytes --]

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 common/rc |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/common/rc b/common/rc
index 7f9db56..3bd4bcb 100644
--- a/common/rc
+++ b/common/rc
@@ -1318,6 +1318,10 @@ _require_xfs_io_command()
 		testio=`$XFS_IO_PROG -F -f -c "pwrite 0 20k" -c "fsync" \
 			-c "fiemap -v" $testfile 2>&1`
 		;;
+	"flink" )
+		testio=`$XFS_IO_PROG -T -F -c "flink $testfile" \
+			$TEST_DIR 2>&1`
+		;;
 	*)
 		testio=`$XFS_IO_PROG -c "$command help" 2>&1`
 	esac
-- 
1.7.10.4


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 2/3] generic: add a basic O_TMPFILE test

[Christoph Hellwig]

[-- Attachment #1: 0002-generic-add-a-basic-O_TMPFILE-test.patch --]
[-- Type: text/plain, Size: 3179 bytes --]

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 tests/generic/004     |   70 +++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/004.out |    6 +++++
 tests/generic/group   |    1 +
 3 files changed, 77 insertions(+)
 create mode 100755 tests/generic/004
 create mode 100644 tests/generic/004.out

diff --git a/tests/generic/004 b/tests/generic/004
new file mode 100755
index 0000000..c653530
--- /dev/null
+++ b/tests/generic/004
@@ -0,0 +1,70 @@
+#! /bin/bash
+# FS QA Test No. 004
+#
+# Test O_TMPFILE opens
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2014 Christoph Hellwig.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f ${testfile}
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs generic
+_supported_os Linux
+
+_require_xfs_io_command "flink"
+
+testfile="${TEST_DIR}/tst-tmpfile-flink"
+
+# test creating a r/w tmpfile, do I/O and link it into the namespace
+$XFS_IO_PROG -T \
+	-c "pwrite 0 4096" \
+	-c "pread 0 4096" \
+	-c "flink ${testfile}" \
+	${TEST_DIR} | _filter_xfs_io
+
+if [ ! -f "${testfile}" ]; then
+    echo "failed to link testfile into place"
+    exit 1
+fi
+
+rm -f ${tmpfile}
+
+# test creating a r/o tmpfile.  Should fail
+$XFS_IO_PROG -Tr ${TEST_DIR} -c "close" | _filter_test_dir
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/004.out b/tests/generic/004.out
new file mode 100644
index 0000000..b85c11c
--- /dev/null
+++ b/tests/generic/004.out
@@ -0,0 +1,6 @@
+QA output created by 004
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+/mnt/test: Invalid argument
diff --git a/tests/generic/group b/tests/generic/group
index a99b6a1..ad86d11 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -6,6 +6,7 @@
 001 rw dir udf auto quick
 002 metadata udf auto quick
 003 atime auto quick
+004 auto quick
 005 dir udf auto quick
 006 dir udf auto quick
 007 dir udf auto quick
-- 
1.7.10.4


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

[PATCH 3/3] xfs: add a test for unlinked inodes due to O_TMPFILE

[Christoph Hellwig]

[-- Attachment #1: 0003-xfs-add-a-test-for-unlinked-inodes-due-to-O_TMPFILE.patch --]
[-- Type: text/plain, Size: 5360 bytes --]

Make sure that we see unlinked inodes when shutting a filesystem
down that has an open O_TMPFILE descriptor, and make sure that it
has been deleted after a mount/umount cycle.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 tests/xfs/006     |   89 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/006.out |   43 ++++++++++++++++++++++++++
 tests/xfs/group   |    1 +
 3 files changed, 133 insertions(+)
 create mode 100755 tests/xfs/006
 create mode 100644 tests/xfs/006.out

diff --git a/tests/xfs/006 b/tests/xfs/006
new file mode 100755
index 0000000..7dae43d
--- /dev/null
+++ b/tests/xfs/006
@@ -0,0 +1,89 @@
+#! /bin/bash
+# XFS QA Test No. 006
+#
+# Test O_TMPFILE interaction with log recovery and repair.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2014 Christoph Hellwig.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+    rm -f ${testfile}
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/repair
+
+# real QA test starts here
+_supported_fs xfs
+_supported_os Linux
+
+_require_scratch
+_require_xfs_io_command "flink"
+
+testfile="${SCRATCH_MNT}/tst-tmpfile-flink"
+
+_scratch_mount
+
+#
+# Test creating a r/w tmpfile, make sure it goes out to disk while still
+# open using a freeze/thaw, and then shut down the filesystem.
+#
+$XFS_IO_PROG -x -T \
+	-c "pwrite 0 4096" \
+	-c "pread 0 4096" \
+	-c "freeze" \
+	-c "thaw" \
+	-c "shutdown" \
+	${SCRATCH_MNT} | _filter_xfs_io
+
+_scratch_unmount
+
+# repair should find an unlinked inode now
+_scratch_xfs_repair -n 2>&1 | \
+	_filter_repair > $tmp.repair
+
+ino=`grep 'disconnected inode ' $tmp.repair | \
+	head -1 | \
+	awk '{print $3}' | \
+	sed 's/,//'`
+
+cat $tmp.repair | sed "s/$ino/XXX/g"
+
+# filesystem should be clean after an unmount/mount cycle
+_scratch_mount
+_scratch_unmount
+
+_scratch_xfs_repair -n 2>&1 | _filter_repair
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/006.out b/tests/xfs/006.out
new file mode 100644
index 0000000..46f3b73
--- /dev/null
+++ b/tests/xfs/006.out
@@ -0,0 +1,43 @@
+QA output created by 006
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Phase 1 - find and verify superblock...
+Phase 2 - using <TYPEOF> log
+        - scan filesystem freespace and inode maps...
+        - found root inode chunk
+Phase 3 - for each AG...
+        - scan (but don't clear) agi unlinked lists...
+        - process known inodes and perform inode discovery...
+        - process newly discovered inodes...
+Phase 4 - check for duplicate blocks...
+        - setting up duplicate extent list...
+        - check for inodes claiming duplicate blocks...
+No modify flag set, skipping phase 5
+Phase 6 - check inode connectivity...
+        - traversing filesystem ...
+        - traversal finished ...
+        - moving disconnected inodes to lost+found ...
+disconnected inode XXX, would move to lost+found
+Phase 7 - verify link counts...
+would have reset inode XXX nlinks from 0 to 1
+No modify flag set, skipping filesystem flush and exiting.
+Phase 1 - find and verify superblock...
+Phase 2 - using <TYPEOF> log
+        - scan filesystem freespace and inode maps...
+        - found root inode chunk
+Phase 3 - for each AG...
+        - scan (but don't clear) agi unlinked lists...
+        - process known inodes and perform inode discovery...
+        - process newly discovered inodes...
+Phase 4 - check for duplicate blocks...
+        - setting up duplicate extent list...
+        - check for inodes claiming duplicate blocks...
+No modify flag set, skipping phase 5
+Phase 6 - check inode connectivity...
+        - traversing filesystem ...
+        - traversal finished ...
+        - moving disconnected inodes to lost+found ...
+Phase 7 - verify link counts...
+No modify flag set, skipping filesystem flush and exiting.
diff --git a/tests/xfs/group b/tests/xfs/group
index ba34650..2f896e1 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -3,6 +3,7 @@
 003 db auto quick
 004 db auto quick
 005 auto quick
+006 auto quick
 008 rw ioctl auto quick
 009 rw ioctl auto prealloc quick
 012 rw auto quick
-- 
1.7.10.4


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 1/3] common: add flink support to _require_xfs_io_command

[Dave Chinner]

On Thu, Mar 13, 2014 at 03:13:03AM -0700, Christoph Hellwig wrote:
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  common/rc |    4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/common/rc b/common/rc
> index 7f9db56..3bd4bcb 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -1318,6 +1318,10 @@ _require_xfs_io_command()
>  		testio=`$XFS_IO_PROG -F -f -c "pwrite 0 20k" -c "fsync" \
>  			-c "fiemap -v" $testfile 2>&1`
>  		;;
> +	"flink" )
> +		testio=`$XFS_IO_PROG -T -F -c "flink $testfile" \
> +			$TEST_DIR 2>&1`
> +		;;
>  	*)
>  		testio=`$XFS_IO_PROG -c "$command help" 2>&1`
>  	esac

Looks good.

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 2/3] generic: add a basic O_TMPFILE test

[Dave Chinner]

On Thu, Mar 13, 2014 at 03:13:04AM -0700, Christoph Hellwig wrote:
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  tests/generic/004     |   70 +++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/004.out |    6 +++++
>  tests/generic/group   |    1 +
>  3 files changed, 77 insertions(+)
>  create mode 100755 tests/generic/004
>  create mode 100644 tests/generic/004.out
> 
> diff --git a/tests/generic/004 b/tests/generic/004
> new file mode 100755
> index 0000000..c653530
> --- /dev/null
> +++ b/tests/generic/004
> @@ -0,0 +1,70 @@
> +#! /bin/bash
> +# FS QA Test No. 004
> +#
> +# Test O_TMPFILE opens

it also tests linking a tmpfile...

> +testfile="${TEST_DIR}/tst-tmpfile-flink"
> +
> +# test creating a r/w tmpfile, do I/O and link it into the namespace
> +$XFS_IO_PROG -T \
> +	-c "pwrite 0 4096" \
> +	-c "pread 0 4096" \
> +	-c "flink ${testfile}" \
> +	${TEST_DIR} | _filter_xfs_io
> +
> +if [ ! -f "${testfile}" ]; then
> +    echo "failed to link testfile into place"
> +    exit 1
> +fi
> +
> +rm -f ${tmpfile}

You could just replace the  check for the $testfile existing and
then the removal with:

rm $testfile

because if it doesn't exist that will thrown an ENOENT error like:

rm: cannot remove "foo": No such file or directory

And that will fail the test. There's also no reason to exit if there
is an error, because the rest of the test does not depend on
$testfile being removed. The golden output comparison will then
cause the test to fail...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 3/3] xfs: add a test for unlinked inodes due to O_TMPFILE

[Dave Chinner]

On Thu, Mar 13, 2014 at 03:13:05AM -0700, Christoph Hellwig wrote:
> Make sure that we see unlinked inodes when shutting a filesystem
> down that has an open O_TMPFILE descriptor, and make sure that it
> has been deleted after a mount/umount cycle.
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
>  tests/xfs/006     |   89 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/006.out |   43 ++++++++++++++++++++++++++
>  tests/xfs/group   |    1 +
>  3 files changed, 133 insertions(+)
>  create mode 100755 tests/xfs/006
>  create mode 100644 tests/xfs/006.out
> 
> diff --git a/tests/xfs/006 b/tests/xfs/006
> new file mode 100755
> index 0000000..7dae43d
> --- /dev/null
> +++ b/tests/xfs/006
> @@ -0,0 +1,89 @@
> +#! /bin/bash
> +# XFS QA Test No. 006
> +#
> +# Test O_TMPFILE interaction with log recovery and repair.
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2014 Christoph Hellwig.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +    cd /
> +    rm -f $tmp.*
> +    rm -f ${testfile}

No need to remove the testfile on a scratch device here.

...

> +testfile="${SCRATCH_MNT}/tst-tmpfile-flink"

isn't used.

> +
> +_scratch_mount
> +
> +#
> +# Test creating a r/w tmpfile, make sure it goes out to disk while still
> +# open using a freeze/thaw, and then shut down the filesystem.
> +#
> +$XFS_IO_PROG -x -T \
> +	-c "pwrite 0 4096" \
> +	-c "pread 0 4096" \
> +	-c "freeze" \
> +	-c "thaw" \
> +	-c "shutdown" \
> +	${SCRATCH_MNT} | _filter_xfs_io

So that's an inode on the unlinked list with a link count of zero.

> +_scratch_unmount
> +
> +# repair should find an unlinked inode now
> +_scratch_xfs_repair -n 2>&1 | \
> +	_filter_repair > $tmp.repair

The filter removes the AGI unlinked bucket inode listings, which is
what we really want to see here. The checking for disconnected
inodes is a side effect of using no-modify mode - if it was a real
repair that inode would be trashed, not seen as a disconnected
inode.

Hence I think this is really checking for the wrong thing...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 3/3] xfs: add a test for unlinked inodes due to O_TMPFILE

[Christoph Hellwig]

Skipping this for now, filtering the repair output properly is going to
be a lot of work, and I'd rather do a generic unlinked inode list
recovery test instead of tying it to O_TMPFILE.

With the first patch already reviewed and the second updated those
should be ready to apply now.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH 3/3] xfs: add a test for unlinked inodes due to O_TMPFILE

[Dave Chinner]

On Thu, Apr 03, 2014 at 04:07:30AM -0700, Christoph Hellwig wrote:
> Skipping this for now, filtering the repair output properly is going to
> be a lot of work, and I'd rather do a generic unlinked inode list
> recovery test instead of tying it to O_TMPFILE.
> 
> With the first patch already reviewed and the second updated those
> should be ready to apply now.

OK, Thanks for the update.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs