[PATCH v7 0/3] dm error based test cases

[PATCH v7 0/3] dm error based test cases

[Anand Jain]

This is v7 of this patch set.
Mainly accepts Dave's latest review comments with thanks.
For actual change-list please ref to the individial patch. 

Anand Jain (3):
  xfstests: btrfs: add functions to create dm-error device
  xfstests: btrfs: test device replace, with EIO on the src dev
  xfstests: btrfs: test device delete with EIO on src dev

 common/dmerror      | 71 ++++++++++++++++++++++++++++++++++++++++++++++
 common/rc           |  7 +++++
 tests/btrfs/098     | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/098.out | 11 +++++++
 tests/btrfs/099     | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/099.out | 11 +++++++
 tests/btrfs/group   |  2 ++
 7 files changed, 265 insertions(+)
 create mode 100644 common/dmerror
 create mode 100755 tests/btrfs/098
 create mode 100644 tests/btrfs/098.out
 create mode 100755 tests/btrfs/099
 create mode 100644 tests/btrfs/099.out

-- 
2.4.1

[PATCH v7 1/3] xfstests: btrfs: add functions to create dm-error device

[Anand Jain]

From: Anand Jain <Anand.Jain@oracle.com>

Controlled EIO from the device is achieved using the dm device.
Helper functions are at common/dmerror.

Broadly steps will include calling _dmerror_init().
_dmerror_init() will use SCRATCH_DEV to create dm linear device and assign
DMERROR_DEV to /dev/mapper/error-test.

When test script is ready to get EIO, the test cases can call
_dmerror_load_table() which then it will load the dm error.
so that reading DMERROR_DEV will cause EIO. After the test case is
complete, cleanup must be done by calling _dmerror_cleanup().

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
---
v6->v7:
 rename _init_dmerror() to _dmerror_init()
 remove _scratch_mkfs_dmerror()
 rename _mount_dmerror() to _dmerror_mount()
 rename _cleaup_dmerror() to _dmerror_cleanup()
 rename _load_dmerror_table() to _dmerror_load_table()
 rename BLK_DEV_SIZE to blk_dev_size
 remove _unmount_dmerror there were no consumer of it
 use _fail instead of _fatal in rc/dmerror
 update error log to make crisp sense
 move _require_dmerror() from common/rc to common/dmerror and rename
   it to dmerror_required() so that its consistent with other function
   names with in the file

v5->v6: accepts Eryu's comments, thanks
  (. added missing $MKFS_OPTIONS at _scratch_mkfs_dmerror()
   . used $MOUNT_PROG instead of mount at _mount_dmerror()
   . correct typo $UMOUNT_PROG, no S at the end in _unmount_dmerror()
  )

v4->v5: No Change. keep up with the patch set

v3->v4: rebase on latest xfstests code

v2.1->v3: accepts Filipe Manana's review comments, thanks
   (. correct if else statement in _require_dm_error()
    . fix indent 
      a missed Dave comment in v1. looks like I goofed with git cli
   )

v2->v2.1: fixed missed typo error fixup in the commit.


v1->v2: accepts Dave Chinner's review comments, thanks
   (. use SCRATCH_DEV for dmerror backing device
    . remove duplicate check of DM_BLK_DEV in _init_dm_error_dev()
    . remove a wrong check when reading block size in _init_dm_error_dev()
    . remove a wrong check with blockdev --setra in _init_dm_error_dev()
    . remove unnecessary check in _load_dm_error_table()
    . remove unnecessary dmerror device test by using dd
   )

 common/dmerror | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 71 insertions(+)
 create mode 100644 common/dmerror

diff --git a/common/dmerror b/common/dmerror
new file mode 100644
index 0000000..f8b9ce6
--- /dev/null
+++ b/common/dmerror
@@ -0,0 +1,71 @@
+##/bin/bash
+#
+# Copyright (c) 2015 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#
+# common functions for setting up and tearing down a dmerror device
+
+# this test requires the device mapper error target
+#
+_dmerror_required()
+{
+	_require_command "$DMSETUP_PROG" dmsetup
+	$DMSETUP_PROG targets | grep error >/dev/null 2>&1
+	[ $? -ne 0 ] && _notrun "This test requires dm error support"
+}
+
+_dmerror_init()
+{
+	local dm_backing_dev=$SCRATCH_DEV
+
+	$DMSETUP_PROG remove error-test > /dev/null 2>&1
+
+	local blk_dev_size=`blockdev --getsz $dm_backing_dev`
+
+	DMERROR_DEV='/dev/mapper/error-test'
+
+	DMLINEAR_TABLE="0 $blk_dev_size linear $dm_backing_dev 0"
+
+	$DMSETUP_PROG create error-test --table "$DMLINEAR_TABLE" || \
+		_fatal "failed to create dm linear device"
+
+	DMERROR_TABLE="0 $blk_dev_size error $dm_backing_dev 0"
+}
+
+_dmerror_mount()
+{
+	_scratch_options mount
+	_mount -t $FSTYP $SCRATCH_OPTIONS $MOUNT_OPTIONS $SELINUX_MOUNT_OPTIONS $* \
+						$DMERROR_DEV $SCRATCH_MNT
+}
+
+_dmerror_cleanup()
+{
+	$UMOUNT_PROG $SCRATCH_MNT > /dev/null 2>&1
+	$DMSETUP_PROG remove error-test > /dev/null 2>&1
+}
+
+_dmerror_load_table()
+{
+	$DMSETUP_PROG suspend error-test
+	[ $? -ne 0 ] && _fail  "dmsetup suspend failed"
+
+	$DMSETUP_PROG load error-test --table "$DMERROR_TABLE"
+	[ $? -ne 0 ] && _fail "dmsetup failed to load error table"
+
+	$DMSETUP_PROG resume error-test
+	[ $? -ne 0 ] && _fail  "dmsetup resume failed"
+}
-- 
2.4.1

[PATCH v7 2/3] xfstests: btrfs: test device replace, with EIO on the src dev

[Anand Jain]

From: Anand Jain <Anand.Jain@oracle.com>

This test case will test to confirm the replace works with
the failed (EIO) replacing source device. EIO condition is
achieved using the DM device.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
---
v6->v7: accepts Dave's comments
    (. in line with changes as in 1/3)

v5->v6: accepts Eryu and Filipe's comments
    (. commit update
     . correct cleanup with tmp.*
     .  don't redirect run_check $FSFTRESS_PROG to dev null
    )

v4->v5: rebase on latest xfstests code and accepts Filipe comment
    (. cut line length to 80 char
     . accommodate to the regression introduced by a092363bbdfa6cc6e44353136bae9d3aa81baae2
    )
      
v3->v4: rebase on latest xfstests code

v2->v3: accepts Filipe Manana's review comments, thanks
    (. induce FS load using FSSTRESS_PROG, actually I missed it in v2
    )

v1->v2: accepts Dave Chinner's review comments, thanks
    (. move 'start with the clean devices' code to actually at the
       start of the script - damn why I didn't think like this before.
       Thanks Dave.
     . add .out file to contain the logs to verify.
    )
 tests/btrfs/098     | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/098.out | 11 ++++++++
 tests/btrfs/group   |  1 +
 3 files changed, 93 insertions(+)
 create mode 100755 tests/btrfs/098
 create mode 100644 tests/btrfs/098.out

diff --git a/tests/btrfs/098 b/tests/btrfs/098
new file mode 100755
index 0000000..8891655
--- /dev/null
+++ b/tests/btrfs/098
@@ -0,0 +1,81 @@
+#! /bin/bash
+# FS QA Test No. btrfs/098
+#
+# Test device replace works when the source device has EIO
+#
+# Copyright (c) 2015 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+
+_cleanup()
+{
+	_dmerror_cleanup
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/filter.btrfs
+. ./common/dmerror
+
+_supported_fs btrfs
+_supported_os Linux
+_need_to_be_root
+_require_scratch_dev_pool 3
+_dmerror_required
+
+rm -f $seqres.full
+
+dev1="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $2}'`"
+dev2="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $3}'`"
+
+_dmerror_init
+_mkfs_dev "-f -d raid1 -m raid1 $dev1 $DMERROR_DEV"
+_dmerror_mount
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+error_devid=`$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT |\
+			egrep $DMERROR_DEV | $AWK_PROG '{print $2}'`
+
+snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
+snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
+run_check $FSSTRESS_PROG -d $SCRATCH_MNT -n 200 -p 8 $FSSTRESS_AVOID -x \
+							"$snapshot_cmd" -X 50
+
+# now load the error into the DMERROR_DEV
+_dmerror_load_table
+
+_run_btrfs_util_prog replace start -B $error_devid $dev2 $SCRATCH_MNT
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+echo "=== device replace completed"
+
+status=0; exit
diff --git a/tests/btrfs/098.out b/tests/btrfs/098.out
new file mode 100644
index 0000000..eb2f87f
--- /dev/null
+++ b/tests/btrfs/098.out
@@ -0,0 +1,11 @@
+QA output created by 098
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+	devid <DEVID> size <SIZE> used <SIZE> path /dev/mapper/error-test
+
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+
+=== device replace completed
diff --git a/tests/btrfs/group b/tests/btrfs/group
index e13865a..c8a53b5 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -100,3 +100,4 @@
 095 auto quick metadata
 096 auto quick clone
 097 auto quick send clone
+098 auto quick replace
-- 
2.4.1

[PATCH v7 3/3] xfstests: btrfs: test device delete with EIO on src dev

[Anand Jain]

From: Anand Jain <Anand.Jain@oracle.com>

This test case tests if the device delete works with
the failed (EIO) source device. EIO errors are achieved
usign the DM device.

This test would need following btrfs-progs and btrfs
kernel patch
   btrfs-progs: device delete to accept devid
   Btrfs: device delete by devid

However when btrfs-progs patch is not found this test will
not run, and when kernel patch is not found btrfs-progs
will fail gracefully and thus the test script.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
v6->v7: accepts Dave's comments
    (. in line with changes as in 1/3)

v5->v6: accepts Eryu and Filipe's comments, thanks
    (. correct cleanup with tmp.*
     .  don't redirect run_check $FSFTRESS_PROG to dev null
    )

v4->v5: rebase on latest xfstests code, and accepts Filipe comment
    (. cut line length to 80 char
     . accommodate to the regression introduced by a092363bbdfa6cc6e44353136bae9d3aa81baae2
    )

v3->v4: rebase on latest xfstests code

v2->v3: accepts Filipe Manana's review comments, thanks
    ( . induce FS load using FSSTRESS_PROG
    )

v1->v2: accepts Dave Chinner's review comments, thanks
    (. move 'start with the clean devices' code to actually at the
       start of the script - damn why I didn't think like this
       before. Thanks Dave.
     . update .out file to contain the logs to verify.
    )
 common/rc           |  7 +++++
 tests/btrfs/099     | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/099.out | 11 +++++++
 tests/btrfs/group   |  1 +
 4 files changed, 101 insertions(+)
 create mode 100755 tests/btrfs/099
 create mode 100644 tests/btrfs/099.out

diff --git a/common/rc b/common/rc
index 70d2fa8..86c5192 100644
--- a/common/rc
+++ b/common/rc
@@ -2728,6 +2728,13 @@ _require_meta_uuid()
 	umount $SCRATCH_MNT
 }
 
+_require_btrfs_dev_del_by_devid()
+{
+	$BTRFS_UTIL_PROG device delete --help | egrep devid > /dev/null 2>&1
+	[ $? -eq 0 ] || _notrun "$BTRFS_UTIL_PROG too old "\
+			"(must support 'btrfs device delete <devid> /<mnt>')"
+}
+
 _get_total_inode()
 {
 	if [ -z "$1" ]; then
diff --git a/tests/btrfs/099 b/tests/btrfs/099
new file mode 100755
index 0000000..9793c9b
--- /dev/null
+++ b/tests/btrfs/099
@@ -0,0 +1,82 @@
+#! /bin/bash
+# FS QA Test No. btrfs/099
+#
+# Test device delete when the source device has EIO
+#
+# Copyright (c) 2015 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+
+_cleanup()
+{
+	_dmerror_cleanup
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/filter.btrfs
+. ./common/dmerror
+
+_supported_fs btrfs
+_supported_os Linux
+_need_to_be_root
+_require_scratch_dev_pool 3
+_require_btrfs_dev_del_by_devid
+_dmerror_required
+
+rm -f $seqres.full
+
+dev1="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $2}'`"
+dev2="`echo $SCRATCH_DEV_POOL | $AWK_PROG '{print $3}'`"
+
+_dmerror_init
+_mkfs_dev -f -d raid1 -m raid1 $dev1 $dev2 $DMERROR_DEV
+_dmerror_mount
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+error_devid=`$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT |\
+			egrep $DMERROR_DEV | $AWK_PROG '{print $2}'`
+
+snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
+snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
+run_check $FSSTRESS_PROG -d $SCRATCH_MNT -n 200 -p 8 $FSSTRESS_AVOID -x \
+							"$snapshot_cmd" -X 50
+
+# now load the error into the DMERROR_DEV
+_dmerror_load_table
+
+_run_btrfs_util_prog device delete $error_devid $SCRATCH_MNT
+
+_run_btrfs_util_prog filesystem show -m $SCRATCH_MNT
+$BTRFS_UTIL_PROG filesystem show -m $SCRATCH_MNT | _filter_btrfs_filesystem_show
+
+echo "=== device delete completed"
+
+status=0; exit
diff --git a/tests/btrfs/099.out b/tests/btrfs/099.out
new file mode 100644
index 0000000..ec74e45
--- /dev/null
+++ b/tests/btrfs/099.out
@@ -0,0 +1,11 @@
+QA output created by 099
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+	devid <DEVID> size <SIZE> used <SIZE> path /dev/mapper/error-test
+
+Label: none  uuid:  <UUID>
+	Total devices <NUM> FS bytes used <SIZE>
+	devid <DEVID> size <SIZE> used <SIZE> path SCRATCH_DEV
+
+=== device delete completed
diff --git a/tests/btrfs/group b/tests/btrfs/group
index c8a53b5..968ee63 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -101,3 +101,4 @@
 096 auto quick clone
 097 auto quick send clone
 098 auto quick replace
+099 auto quick replace
-- 
2.4.1