[PATCH] xfstests: test orphan inode recovery on RO mount

[PATCH] xfstests: test orphan inode recovery on RO mount

[Eric Sandeen]

There's a bug in xfs where the orphan inode list
is not processed on a readonly mount, and is still not
processed even if it gets mounted rw.  This test exercises
that, and makes sure that the unlinked list is empty after
a transition to rw and an unmount.

While we're at it, sanity check the same thing for pure ro
and rw unmounts.

Now that shutdown is generic, we can test other filesystems
as well, so this is in the generic group.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
---

diff --git a/tests/generic/416 b/tests/generic/416
new file mode 100755
index 0000000..b4a3b6e
--- /dev/null
+++ b/tests/generic/416
@@ -0,0 +1,118 @@
+#! /bin/bash
+# FS QA Test 416
+#
+# Test orphan inode / unlinked list processing on RO mount & RW transition
+#
+# A filesystem that crashes with open but unlinked inodes should
+# be consistent after a ro, ro->rw, or rw mount cycle.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2017 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs generic
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+_require_scratch_shutdown
+
+function create_dirty_orphans() {
+
+	_scratch_mount
+
+	num_files=200
+	num_eas=1
+	ea_val_size=1024
+
+	# sleep for ages and we will kill this pid when we are ready
+	delay=100
+
+	echo "open and unlink $num_files files with EAs"
+	src/multi_open_unlink -f $SCRATCH_MNT/test_file \
+		-n $num_files -s $delay -e $num_eas -v $ea_val_size &
+	pid=$!
+
+	# time to create and unlink all the files
+	sleep 10
+
+	echo "godown"
+	src/godown -v -f $SCRATCH_MNT >> $seqres.full
+
+	# kill the multi_open_unlink
+	kill $pid 2>/dev/null
+	wait $pid 2>/dev/null
+	pid=""
+
+	_scratch_unmount
+}
+
+# Does a regular rw mount handle the orphan list?
+echo "mount dirty orphans rw, then unmount"
+create_dirty_orphans
+_scratch_mount
+_scratch_unmount
+# We should be clean at this point
+echo "check fs consistency"
+_check_scratch_fs
+
+# Does a ro mount handle the orphan list?
+echo "mount dirty orphans ro, then unmount"
+create_dirty_orphans
+_scratch_mount -o ro
+_scratch_unmount
+# We should be clean at this point
+echo "check fs consistency"
+_check_scratch_fs
+
+# Does a mount with ro->rw transition handle orphan list?
+echo "mount dirty orphans ro and remount rw, then unmount"
+create_dirty_orphans
+_scratch_mount -o ro
+_scratch_remount rw
+_scratch_unmount
+# We should be clean at this point
+echo "check fs consistency"
+_check_scratch_fs
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/416.out b/tests/generic/416.out
new file mode 100644
index 0000000..f79a25d
--- /dev/null
+++ b/tests/generic/416.out
@@ -0,0 +1,13 @@
+QA output created by 416
+mount dirty orphans rw, then unmount
+open and unlink 200 files with EAs
+godown
+check fs consistency
+mount dirty orphans ro, then unmount
+open and unlink 200 files with EAs
+godown
+check fs consistency
+mount dirty orphans ro and remount rw, then unmount
+open and unlink 200 files with EAs
+godown
+check fs consistency
diff --git a/tests/generic/group b/tests/generic/group
index b510d41..e267a92 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -418,3 +418,4 @@
 413 auto quick
 414 auto quick clone
 415 auto clone
+416 auto quick shutdown log

Re: [PATCH] xfstests: test orphan inode recovery on RO mount

[Eryu Guan]

On Thu, Mar 09, 2017 at 03:31:43PM -0600, Eric Sandeen wrote:
> There's a bug in xfs where the orphan inode list
> is not processed on a readonly mount, and is still not
> processed even if it gets mounted rw.  This test exercises
> that, and makes sure that the unlinked list is empty after
> a transition to rw and an unmount.
> 
> While we're at it, sanity check the same thing for pure ro
> and rw unmounts.
> 
> Now that shutdown is generic, we can test other filesystems
> as well, so this is in the generic group.
> 
> Signed-off-by: Eric Sandeen <sandeen@redhat.com>
> ---
> 
> diff --git a/tests/generic/416 b/tests/generic/416
> new file mode 100755
> index 0000000..b4a3b6e
> --- /dev/null
> +++ b/tests/generic/416
> @@ -0,0 +1,118 @@
> +#! /bin/bash
> +# FS QA Test 416
> +#
> +# Test orphan inode / unlinked list processing on RO mount & RW transition
> +#
> +# A filesystem that crashes with open but unlinked inodes should
> +# be consistent after a ro, ro->rw, or rw mount cycle.
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2017 Red Hat, Inc.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs generic
> +_supported_fs generic

duplicated _supported_fs :)

> +_supported_os Linux
> +_require_scratch
> +_require_scratch_shutdown

Need a "_require_metadata_journaling $SCRATCH_DEV", otherwise ext2 will
fail fsck when driven by ext4 driver.

> +
> +function create_dirty_orphans() {
> +
> +	_scratch_mount
> +
> +	num_files=200
> +	num_eas=1
> +	ea_val_size=1024
> +
> +	# sleep for ages and we will kill this pid when we are ready
> +	delay=100
> +
> +	echo "open and unlink $num_files files with EAs"
> +	src/multi_open_unlink -f $SCRATCH_MNT/test_file \
> +		-n $num_files -s $delay -e $num_eas -v $ea_val_size &
> +	pid=$!
> +
> +	# time to create and unlink all the files
> +	sleep 10

Do we really need 10s? That seems a bit long to me, and I tested with
sleep 3, it also reproduced on XFS (4.11-rc1).

Otherwise look good to me. I can fix them all at commit time if you're
OK with "sleep 3".

Thanks,
Eryu

Re: [PATCH] xfstests: test orphan inode recovery on RO mount

[Eric Sandeen]

On 3/9/17 10:36 PM, Eryu Guan wrote:
> On Thu, Mar 09, 2017 at 03:31:43PM -0600, Eric Sandeen wrote:
>> There's a bug in xfs where the orphan inode list
>> is not processed on a readonly mount, and is still not
>> processed even if it gets mounted rw.  This test exercises
>> that, and makes sure that the unlinked list is empty after
>> a transition to rw and an unmount.
>>
>> While we're at it, sanity check the same thing for pure ro
>> and rw unmounts.
>>
>> Now that shutdown is generic, we can test other filesystems
>> as well, so this is in the generic group.
>>
>> Signed-off-by: Eric Sandeen <sandeen@redhat.com>

...

> Do we really need 10s? That seems a bit long to me, and I tested with
> sleep 3, it also reproduced on XFS (4.11-rc1).
> 
> Otherwise look good to me. I can fix them all at commit time if you're
> OK with "sleep 3".

Sure, thanks.  The 10s came from another test w/ the same setup.
Happy to resend too, if you'd prefer.

-Eric
 
> Thanks,
> Eryu
> 

Re: [PATCH] xfstests: test orphan inode recovery on RO mount

[Eryu Guan]

On Fri, Mar 10, 2017 at 08:20:36AM -0600, Eric Sandeen wrote:
> On 3/9/17 10:36 PM, Eryu Guan wrote:
> > On Thu, Mar 09, 2017 at 03:31:43PM -0600, Eric Sandeen wrote:
> >> There's a bug in xfs where the orphan inode list
> >> is not processed on a readonly mount, and is still not
> >> processed even if it gets mounted rw.  This test exercises
> >> that, and makes sure that the unlinked list is empty after
> >> a transition to rw and an unmount.
> >>
> >> While we're at it, sanity check the same thing for pure ro
> >> and rw unmounts.
> >>
> >> Now that shutdown is generic, we can test other filesystems
> >> as well, so this is in the generic group.
> >>
> >> Signed-off-by: Eric Sandeen <sandeen@redhat.com>
> 
> ...
> 
> > Do we really need 10s? That seems a bit long to me, and I tested with
> > sleep 3, it also reproduced on XFS (4.11-rc1).
> > 
> > Otherwise look good to me. I can fix them all at commit time if you're
> > OK with "sleep 3".
> 
> Sure, thanks.  The 10s came from another test w/ the same setup.
> Happy to resend too, if you'd prefer.

No worries, I've fixed them up and gone through release testing.

Thanks,
Eryu