Re: [PATCH] xfs: log record crc mismatch test

Re: [PATCH] xfs: log record crc mismatch test

[Dave Chinner]

On Mon, Apr 27, 2015 at 06:25:57PM +0300, Alexander Tsvetkov wrote:
> Added new test case for simulation of V5 filesystem
> log crc record corruption using of external log.
> 
> Signed-off-by: Alexander Tsvetkov <alexander.tsvetkov@oracle.com>

Hi Alexander,

In future, can you please post xfstests patches to
fstests@vger.kernel.org? I've cc'd that list on my reply.

> ---
>  tests/xfs/005     | 28 ++++++++++++++++++++++++++--
>  tests/xfs/005.out |  3 +++
>  2 files changed, 29 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/xfs/005 b/tests/xfs/005
> index fade4bb..ba0dcbb 100755
> --- a/tests/xfs/005
> +++ b/tests/xfs/005
> @@ -1,7 +1,7 @@
>  #! /bin/bash
>  # FS QA Test No. 005
>  #
> -# Test that a bad crc on a primary V5 superblock will fail the mount
> +# Test that a bad crc on a V5 filesystem will fail the mount
>  #
>  # 10e6e65 xfs: be more forgiving of a v4 secondary sb w/ junk in v5 fields
>  # inadvertently caused primary SB CRC failures to not error out, this
> @@ -55,10 +55,12 @@ _supported_os Linux
> 
>  _require_scratch_nocheck
>  _require_xfs_mkfs_crc
> +_require_logdev
> +logsize=2560

As eric pointed out, this fundamentally changes the context of the
test, and so should be written as a new test rather than a change to
an existing test.

Also, you don't need an external log device to do this - you can use
xfs_db to zero parts of the internal log just fine.

$ sudo mkfs.xfs -f -m crc=1,finobt=1 /dev/ram0
meta-data=/dev/ram0              isize=512    agcount=4, agsize=250000 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1
data     =                       bsize=4096   blocks=1000000, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
$ sudo xfs_db -x /dev/ram0
xfs_db> sb 0
xfs_db> p logstart
logstart = 524296
xfs_db> a logstart
xfs_db> type data
xfs_db> write fill 0xa5a5a5a5 0 4096
xfs_db> p
000: a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5
020: a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5
.....
fe0: a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5 a5a5a5a5
xfs_db> q
$ sudo mount /dev/ram0 /mnt/test
[346491.179086] XFS (ram0): Mounting V5 Filesystem
[346491.180844] XFS (ram0): Log inconsistent or not a log (last==0, first!=1)
[346491.182946] XFS (ram0): empty log check failed
[346491.184403] XFS (ram0): log mount/recovery failed: error -22
[346491.186276] XFS (ram0): log mount failed
.....

Note that if you have an external log device, logstart will be zero,
and you'll need to use xfs_io to overwrite it so you'll need to
explicitly test for that case before calling xfs_db to modify the
log.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH v2] xfs: log record crc mismatch test

[Alexander Tsvetkov]

Hi Eric,

On 04/27/2015 07:43 PM, Eric Sandeen wrote:
> On 4/27/15 10:25 AM, Alexander Tsvetkov wrote:
>> Added new test case for simulation of V5 filesystem
>> log crc record corruption using of external log.
>>
>> Signed-off-by: Alexander Tsvetkov <alexander.tsvetkov@oracle.com>
>> ---
>>   tests/xfs/005     | 28 ++++++++++++++++++++++++++--
>>   tests/xfs/005.out |  3 +++
>>   2 files changed, 29 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/xfs/005 b/tests/xfs/005
>> index fade4bb..ba0dcbb 100755
>> --- a/tests/xfs/005
>> +++ b/tests/xfs/005
>> @@ -1,7 +1,7 @@
>>   #! /bin/bash
>>   # FS QA Test No. 005
>>   #
>> -# Test that a bad crc on a primary V5 superblock will fail the mount
>> +# Test that a bad crc on a V5 filesystem will fail the mount
> bad crc on what?
>
>>   #
>>   # 10e6e65 xfs: be more forgiving of a v4 secondary sb w/ junk in v5 fields
>>   # inadvertently caused primary SB CRC failures to not error out, this
>> @@ -55,10 +55,12 @@ _supported_os Linux
>>
>>   _require_scratch_nocheck
>>   _require_xfs_mkfs_crc
>> +_require_logdev
>> +logsize=2560
> No, this means that xfs/05 will no longer work without an external logdev,
> right?  So you've stopped testing the internal case, and we've lost that
> coverage.
>
> And, we generally don't want to add new testcases to old tests because then
> it gets harder to investigate regressions.  It'd probably be best to
> create a new test which specifically tests log crc corruption, and make
> it handle both internal and external logs automatically.
I've prepared the test with both cases, see at the bottom of this mail


> Thanks,
> -Eric
>
>>   rm -f $seqres.full
>>
>> -_scratch_mkfs_xfs -m crc=1 >> $seqres.full 2>&1 || _fail "mkfs failed"
>> +_scratch_mkfs_xfs -m crc=1 -lsize=${logsize}b >> $seqres.full 2>&1 || _fail "mkfs failed"
>>
>>   # Zap the crc.  xfs_db updates the CRC post-write, so poke it directly
>>   $XFS_IO_PROG -c "pwrite 224 4" -c fsync $SCRATCH_DEV | _filter_xfs_io
>> @@ -66,6 +68,28 @@ $XFS_IO_PROG -c "pwrite 224 4" -c fsync $SCRATCH_DEV | _filter_xfs_io
>>   # should FAIL, the crc is bad; golden output contains mount failure
>>   _scratch_mount 2>&1 | filter_mount
>>
>> +_scratch_unmount &> /dev/null
>> +_scratch_mkfs_xfs -m crc=1 -lsize=${logsize}b  >> $seqres.full 2>&1 || _fail "mkfs failed"
>> +_scratch_mount >> $seqres.full 2>&1 || _fail "mount failed"
>> +
>> +# Run filesytem exercising for some time
>> +_param="-f link=10 -f creat=10 -f mkdir=10 -f truncate=5 -f symlink=10"
>> +$FSSTRESS_PROG $_param -s 1 $FSSTRESS_AVOID -n 1000 -d $SCRATCH_MNT &> /dev/null
>> +sleep 3
>> +
>> +# Shutdown filesystem and unmount
>> +src/godown -v $SCRATCH_MNT >> $seqres.full
>> +killall -q fsstress
>> +wait
>> +sleep 3
>> +_scratch_unmount
>> +
>> +# Zap log record crc.
>> +$XFS_IO_PROG -c "pwrite 5000 5" $SCRATCH_LOGDEV | _filter_xfs_io
>> +
>> +# should FAIL, log record crc is bad; golden output contains mount failure
>> +_scratch_mount 2>&1 | filter_mount
>> +
>>   # success, all done
>>   status=0
>>   exit
>> diff --git a/tests/xfs/005.out b/tests/xfs/005.out
>> index 13d0125..8907301 100644
>> --- a/tests/xfs/005.out
>> +++ b/tests/xfs/005.out
>> @@ -2,3 +2,6 @@ QA output created by 005
>>   wrote 4/4 bytes at offset 224
>>   XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>>   mount: Structure needs cleaning
>> +wrote 5/5 bytes at offset 5000
>> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>> +mount: Structure needs cleaning
Added new test for simulation of V5 filesystem
log crc record corruption in internal and external logs

Signed-off-by: Alexander Tsvetkov <alexander.tsvetkov@oracle.com>
---
  tests/xfs/306     | 110 
++++++++++++++++++++++++++++++++++++++++++++++++++++++
  tests/xfs/306.out |   7 ++++
  tests/xfs/group   |   1 +
  3 files changed, 118 insertions(+)
  create mode 100755 tests/xfs/306
  create mode 100644 tests/xfs/306.out

diff --git a/tests/xfs/306 b/tests/xfs/306
new file mode 100755
index 0000000..6e025fe
--- /dev/null
+++ b/tests/xfs/306
@@ -0,0 +1,110 @@
+#! /bin/bash
+# FS QA Test No. 306
+#
+# Test that bad crc on a V5 log record will fail the mount
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2015 Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+filter_mount()
+{
+    sed -e "s/mount .* failed: //"
+}
+
+filter_io()
+{
+    sed -e "s/at offset .*/at offset XXX/"
+}
+
+# $1 - log start offset
+# $2 - log device
+do_test()
+{
+    off=$1
+    dev=$2
+
+    _scratch_unmount >/dev/null 2>&1
+    _scratch_mount >> $seqres.full 2>&1 || _fail "mount failed"
+
+    # Run filesytem exercising for some time
+    param="-f link=10 -f creat=10 -f mkdir=10 -f truncate=5 -f symlink=10"
+    $FSSTRESS_PROG $param -s 1 $FSSTRESS_AVOID -n 1000 -d $SCRATCH_MNT 
&> /dev/null
+    sleep 3
+
+    # Shutdown filesystem and unmount
+    src/godown -v $SCRATCH_MNT >> $seqres.full
+    killall -q fsstress
+    wait
+    sleep 3
+    _scratch_unmount
+
+    # Zap log record crc
+    off=$((off + 5000))
+    $XFS_IO_PROG -c "pwrite $off 5" $dev | _filter_xfs_io | filter_io
+
+    # should FAIL, log record crc is bad; golden output contains mount 
failure
+    _scratch_mount 2>&1 | filter_mount
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+_supported_fs xfs
+_supported_os Linux
+
+_require_scratch_nocheck
+_require_xfs_mkfs_crc
+_require_logdev
+
+rm -f $seqres.full
+fssize=$((32 * 1024 * 1024))
+MKFS_OPTIONS="-m crc=1 $MKFS_OPTIONS"
+_scratch_mkfs_sized $fssize 2> /dev/null | _filter_mkfs 2> $tmp.mkfs > 
/dev/null
+. $tmp.mkfs
+
+# internal log corruption
+unset USE_EXTERNAL
+_scratch_mkfs_sized $fssize >> $seqres.full 2>&1 || _fail "mkfs failed"
+logstart=$($XFS_DB_PROG -c "sb 0" -c "print" $SCRATCH_DEV | grep 
logstart | cut -f3 -d' ')
+offset=$((logstart * lbsize))
+do_test $offset $SCRATCH_DEV
+
+# external log corruption
+USE_EXTERNAL="yes"
+_scratch_mkfs_sized $fssize >> $seqres.full 2>&1 || _fail "mkfs failed"
+do_test 0 $SCRATCH_LOGDEV
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/306.out b/tests/xfs/306.out
new file mode 100644
index 0000000..aef98f2
--- /dev/null
+++ b/tests/xfs/306.out
@@ -0,0 +1,7 @@
+QA output created by 306
+wrote 5/5 bytes at offset XXX
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+mount: Structure needs cleaning
+wrote 5/5 bytes at offset XXX
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+mount: Structure needs cleaning
diff --git a/tests/xfs/group b/tests/xfs/group
index 58144d2..f108377 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -191,3 +191,4 @@
  303 auto quick quota
  304 auto quick quota
  305 auto quota
+306 auto quick
-- 
1.9.3