[xfstests PATCH v6 0/8] overlay: add fsck.overlay basic tests

[xfstests PATCH v6 0/8] overlay: add fsck.overlay basic tests

[zhangyi (F)]

Hi All:

Here is the sixth version of test cases for the fsck.overlay[1].

[1] https://github.com/hisilicon/overlayfs-progs

Changes since v5:
- Modify _is_mounted helper to check fstype and mount point.
- Fix overlay mount check in overlay fs check helpers.

-----------

Changes since v4:
- Add _overlay_check_scratch_dirs helper to check optionally dirs
  for tests use _require_overlay_scratch_dirs.
- Modify _overlay_check_fs, make sure overlay is not mounted when
  checking filesystem.

Changes since v3:
- Modify minor suggestions in 202.
- Add _require_overlay_scratch_dirs helper.
- Replace _require_scratch helper in test cases that finished with
  corrupt filesystem and use optionally test dirs.

Changes since v2:
- Modify some minor suggestions from Amir and Vivek, and simplify code
  in all three test cases.
- Fix duplicate redirect xattr test in 202.
- Add impure xattr test case of general merge directory in 'no impure'
  parent directory.
- Fix fs check failure in other overlay fs test cases. PS: The
  multi-lowers problem in overlay/010 also appears in many other tests,
  _require_scratch() will mis-check the pre-defined underlying dirs
  (not the real test dirs). I think they also need to be modified as this
  patch(06/07) does to check the correct dirs, and I can fix them if you
  think necessary.

Changes since v1:

- Fix _check_scratch_fs hook.
- Remove valid/invalid opaque xattr test.
- Add whiteout test cases of valid/invalid whiteouts in opaque/redirect
  parent directory.
- Add impure xattr test.


zhangyi (F) (8):
  common/rc: modify mounted check helper
  overlay: add filesystem check helper
  overlay: add fsck.overlay whiteout test
  overlay: add fsck.overlay redirect directory test
  overlay: add fsck.overlay impure xattr test
  overlay/003: fix fs check failure
  overlay: skip check for tests finished with corrupt filesystem
  overlay: correct scratch dirs check

 common/config         |   1 +
 common/overlay        | 132 +++++++++++++++++++++++++
 common/rc             |  34 +++----
 tests/overlay/003     |   1 -
 tests/overlay/005     |   6 +-
 tests/overlay/010     |   6 +-
 tests/overlay/014     |  10 +-
 tests/overlay/019     |   2 +-
 tests/overlay/031     |   2 +-
 tests/overlay/035     |   6 +-
 tests/overlay/036     |   5 +-
 tests/overlay/037     |   6 +-
 tests/overlay/038     |  10 +-
 tests/overlay/041     |  10 +-
 tests/overlay/043     |   6 +-
 tests/overlay/044     |   6 +-
 tests/overlay/201     | 232 +++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/201.out |  10 ++
 tests/overlay/202     | 269 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/202.out |  11 +++
 tests/overlay/203     | 143 +++++++++++++++++++++++++++
 tests/overlay/203.out |   4 +
 tests/overlay/group   |   3 +
 23 files changed, 885 insertions(+), 30 deletions(-)
 create mode 100755 tests/overlay/201
 create mode 100644 tests/overlay/201.out
 create mode 100755 tests/overlay/202
 create mode 100644 tests/overlay/202.out
 create mode 100755 tests/overlay/203
 create mode 100644 tests/overlay/203.out

-- 
2.9.5

[xfstests PATCH v6 1/8] common/rc: modify mounted check helper

[zhangyi (F)]

Modify _is_mounted() to accept a dir and fstype as input, and check
whether this dir is a mount point of specified fstype. This patch also
fix the problem of invalid fstype check and remove invalid error
message.

Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
---
 common/rc | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/common/rc b/common/rc
index 77a4eb4..2e7aee5 100644
--- a/common/rc
+++ b/common/rc
@@ -2372,27 +2372,27 @@ _scratch_mkfs_richacl()
 	esac
 }
 
-# check that a FS on a device is mounted
+# check that a FS on a device is mounted or a dir is a mount point
 # if so, return mount point
 #
 _is_mounted()
 {
-    if [ $# -ne 1 ]
-    then
-	echo "Usage: _is_mounted device" 1>&2
-	exit 1
-    fi
+	local name
+	local fstype
 
-    device=$1
+	if [ $# -eq 2 ]; then
+		name=$1
+		fstype=$2
+	elif [ $# -eq 1 ]; then
+		name=$1
+	else
+		echo "Usage: _is_mounted <device|mountpoint> [fstype]" 1>&2
+		exit 1
+	fi
 
-    if _mount | grep "$device " | $AWK_PROG -v pattern="type $FSTYP" '
-        pattern        { print $3 ; exit 0 }
-        END            { exit 1 }
-    '
-    then
-        echo "_is_mounted: $device is not a mounted $FSTYP FS"
-        exit 1
-    fi
+	_mount | grep "$name " | $AWK_PROG -v pattern="type $fstype" '
+		$0 ~ pattern	{ print $3 }
+	'
 }
 
 # remount a FS to a new mode (ro or rw)
-- 
2.9.5

[xfstests PATCH v6 2/8] overlay: add filesystem check helper

[zhangyi (F)]

Add filesystem check helpers for the upcoming fsck.overlay utility.
Hook these helpers to _check_test_fs and _check_scratch_fs for constants
underlying dirs of overlay filesystem, and introduce scratch check helpers
for optionally lower/upper/work dirs. These helpers works only if
fsck.overlay exists.

[ _check_test_fs/_check_scratch_fs part picked from Amir's patch, thanks ]

Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
---
 common/config  |   1 +
 common/overlay | 132 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 common/rc      |   4 +-
 3 files changed, 135 insertions(+), 2 deletions(-)

diff --git a/common/config b/common/config
index 5f40413..71115bd 100644
--- a/common/config
+++ b/common/config
@@ -236,6 +236,7 @@ case "$HOSTOS" in
         export MKFS_REISER4_PROG="`set_prog_path mkfs.reiser4`"
 	export E2FSCK_PROG="`set_prog_path e2fsck`"
 	export TUNE2FS_PROG="`set_prog_path tune2fs`"
+	export FSCK_OVERLAY_PROG="`set_prog_path fsck.overlay`"
         ;;
 esac
 
diff --git a/common/overlay b/common/overlay
index 1da4ab1..ce3f279 100644
--- a/common/overlay
+++ b/common/overlay
@@ -151,3 +151,135 @@ _require_scratch_overlay_feature()
 	        _notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}"
 	_scratch_unmount
 }
+
+# Require the same scratch device as _require_scratch, but do not check
+# the constants OVL_LOWER/OVL_UPPER/OVL_WORK dirs, should use together
+# with optionally lower/upper/work dirs and do check explicitly after test.
+_require_overlay_scratch_dirs()
+{
+	_require_scratch_nocheck
+}
+
+_overlay_fsck_dirs()
+{
+	local lowerdir=$1
+	local upperdir=$2
+	local workdir=$3
+	local options=$4
+
+	[[ ! -x "$FSCK_OVERLAY_PROG" ]] && return 0
+
+	$FSCK_OVERLAY_PROG -o lowerdir=$lowerdir -o upperdir=$upperdir \
+			   -o workdir=$workdir $options
+}
+
+_overlay_check_dirs()
+{
+	local lowerdir=$1
+	local upperdir=$2
+	local workdir=$3
+	local err=0
+
+	_overlay_fsck_dirs $* $FSCK_OPTIONS >>$tmp.fsck 2>&1
+	if [ $? -ne 0 ]; then
+		_log_err "_overlay_check_fs: overlayfs on $lowerdir,$upperdir,$workdir is inconsistent"
+
+		echo "*** fsck.overlay output ***"	>>$seqres.full
+		cat $tmp.fsck				>>$seqres.full
+		echo "*** end fsck.overlay output"	>>$seqres.full
+
+		echo "*** mount output ***"		>>$seqres.full
+		_mount					>>$seqres.full
+		echo "*** end mount output"		>>$seqres.full
+
+		err=1
+	fi
+	rm -f $tmp.fsck
+
+	return $err
+}
+
+# Check the same mnt/dev of _check_overlay_scratch_fs, but check optionally
+# lower/upper/work dirs of overlay filesystem, should use together with
+# _require_overlay_scratch_dirs
+_overlay_check_scratch_dirs()
+{
+	local lowerdir=$1
+	local upperdir=$2
+	local workdir=$3
+
+	# Need to umount overlay for scratch dir check
+	local ovl_mounted=`_is_mounted $SCRATCH_MNT overlay`
+	[ -z "$ovl_mounted" ] || $UMOUNT_PROG $SCRATCH_MNT
+
+	_overlay_check_dirs $lowerdir $upperdir $workdir
+	local ret=$?
+
+	if [ $ret -eq 0 -a -n "$ovl_mounted" ]; then
+		# overlay was mounted, remount besides extra mount options
+		_overlay_scratch_mount_dirs $lowerdir $upperdir $workdir
+		ret=$?
+	fi
+
+	return $ret
+}
+
+_overlay_check_fs()
+{
+	local ovl_mnt=$1
+	local base_dev=$4
+	local base_mnt=$5
+	shift 1
+
+	[ "$FSTYP" = overlay ] || return 0
+
+	# Base fs needs to be mounted to check overlay dirs
+	local base_mounted=""
+	[ -z "$base_dev" ] || base_mounted=`_is_mounted $base_dev`
+
+	if [ -z "$base_mounted" ]; then
+		_overlay_base_mount $*
+	else
+		# Need to umount overlay for dir check
+		local ovl_mounted=`_is_mounted $ovl_mnt overlay`
+		[ -z "$ovl_mounted" ] || $UMOUNT_PROG $ovl_mnt
+	fi
+
+	_overlay_check_dirs $base_mnt/$OVL_LOWER $base_mnt/$OVL_UPPER \
+			    $base_mnt/$OVL_WORK
+	local ret=$?
+
+	if [ -z "$base_mounted" ]; then
+		_overlay_base_unmount "$base_dev" "$base_mnt"
+	elif [ $ret -eq 0 -a -n "$ovl_mounted" ]; then
+		# overlay was mounted, remount besides extra mount options
+		_overlay_mount $base_mnt $ovl_mnt
+		ret=$?
+	fi
+
+	if [ $ret != 0 ]; then
+		status=1
+		if [ "$iam" != "check" ]; then
+			exit 1
+		fi
+		return 1
+	fi
+
+	return 0
+}
+
+_check_overlay_test_fs()
+{
+	_overlay_check_fs "$TEST_DIR" \
+		OVL_BASE_TEST_DEV OVL_BASE_TEST_DIR \
+		"$OVL_BASE_TEST_DEV" "$OVL_BASE_TEST_DIR" \
+		$TEST_FS_MOUNT_OPTS $SELINUX_MOUNT_OPTIONS
+}
+
+_check_overlay_scratch_fs()
+{
+	_overlay_check_fs "$SCRATCH_MNT" \
+		OVL_BASE_SCRATCH_DEV OVL_BASE_SCRATCH_MNT \
+		"$OVL_BASE_SCRATCH_DEV" "$OVL_BASE_SCRATCH_MNT" \
+		$OVL_BASE_MOUNT_OPTIONS $SELINUX_MOUNT_OPTIONS
+}
diff --git a/common/rc b/common/rc
index 2e7aee5..b2faf9a 100644
--- a/common/rc
+++ b/common/rc
@@ -2590,7 +2590,7 @@ _check_test_fs()
 	# no way to check consistency for GlusterFS
 	;;
     overlay)
-	# no way to check consistency for overlay
+	_check_overlay_test_fs
 	;;
     pvfs2)
 	;;
@@ -2648,7 +2648,7 @@ _check_scratch_fs()
 	# no way to check consistency for GlusterFS
 	;;
     overlay)
-	# no way to check consistency for overlay
+	_check_overlay_scratch_fs
 	;;
     pvfs2)
 	;;
-- 
2.9.5

[xfstests PATCH v6 3/8] overlay: add fsck.overlay whiteout test

[zhangyi (F)]

Add fsck.overlay test case to test it how to deal with orphan/valid
whiteouts in underlying directories of overlayfs.

Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/overlay/201     | 232 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/201.out |  10 +++
 tests/overlay/group   |   1 +
 3 files changed, 243 insertions(+)
 create mode 100755 tests/overlay/201
 create mode 100644 tests/overlay/201.out

diff --git a/tests/overlay/201 b/tests/overlay/201
new file mode 100755
index 0000000..3198c14
--- /dev/null
+++ b/tests/overlay/201
@@ -0,0 +1,232 @@
+#! /bin/bash
+# FS QA Test 201
+#
+# Test fsck.overlay how to deal with whiteouts in overlayfs.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2017 Huawei.  All Rights Reserved.
+# Author: zhangyi (F) <yi.zhang@huawei.com>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1        # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs overlay
+_supported_os Linux
+_require_overlay_scratch_dirs
+_require_command "$FSCK_OVERLAY_PROG" fsck.overlay
+
+# remove all files from previous tests
+_scratch_mkfs
+
+OVL_REDIRECT_XATTR="trusted.overlay.redirect"
+OVL_OPAQUE_XATTR="trusted.overlay.opaque"
+OVL_OPAQUE_XATTR_VAL="y"
+
+# Check whiteout
+check_whiteout()
+{
+	for arg in $*
+	do
+		local ttype=`stat -c "%F:%t,%T" $arg`
+
+		[[ $ttype == "character special file:0,0" ]] || \
+			echo "Valid whiteout removed incorrectly"
+	done
+}
+
+# Create a whiteout
+make_whiteout()
+{
+	for arg in $*
+	do
+		mknod $arg c 0 0
+	done
+}
+
+# Create an opaque directory
+make_opaque_dir()
+{
+	local target=$1
+
+	mkdir -p $target
+	$SETFATTR_PROG -n $OVL_OPAQUE_XATTR -v $OVL_OPAQUE_XATTR_VAL $target
+}
+
+# Create a redirect directory
+make_redirect_dir()
+{
+	local target=$1
+	local value=$2
+
+	mkdir -p $target
+	$SETFATTR_PROG -n $OVL_REDIRECT_XATTR -v $value $target
+}
+
+# Create test directories
+lowerdir=$OVL_BASE_SCRATCH_MNT/lower
+lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2
+upperdir=$OVL_BASE_SCRATCH_MNT/upper
+workdir=$OVL_BASE_SCRATCH_MNT/workdir
+
+make_test_dirs()
+{
+	rm -rf $lowerdir $lowerdir2 $upperdir $workdir
+	mkdir -p $lowerdir $lowerdir2 $upperdir $workdir
+}
+
+# Test orphan whiteout in lower and upper layer, should remove
+echo "+ Orphan whiteout"
+make_test_dirs
+make_whiteout $lowerdir/foo $upperdir/{foo,bar}
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+ls $lowerdir
+ls $upperdir
+
+# Test valid whiteout covering lower target, should not remove
+echo "+ Valid whiteout"
+make_test_dirs
+touch $lowerdir2/{foo,bar}
+make_whiteout $upperdir/foo $lowerdir/bar
+
+_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p >> \
+	 $seqres.full 2>&1 || _fail "fsck should not fail"
+check_whiteout $upperdir/foo $lowerdir/bar
+
+# Test orphan whiteout in opaque directory, should remove
+echo "+ Orphan whiteout(2)"
+make_test_dirs
+mkdir $lowerdir/testdir
+touch $lowerdir/testdir/foo
+make_opaque_dir $upperdir/testdir
+make_whiteout $upperdir/testdir/foo
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+ls $upperdir/testdir
+
+# Test orphan whiteout whose parent path is not an merged directory,
+# should remove
+echo "+ Orphan whiteout(3)"
+make_test_dirs
+mkdir $lowerdir2/{testdir1,testdir2,testdir3}
+touch $lowerdir2/{testdir1/foo,testdir2/foo,testdir3/foo}
+mkdir $upperdir/{testdir1,testdir2,testdir3,testdir4}
+touch $lowerdir/testdir1
+make_whiteout $lowerdir/testdir2
+make_opaque_dir $lowerdir/testdir3
+make_whiteout $upperdir/{testdir1/foo,/testdir2/foo,testdir3/foo,testdir4/foo}
+
+_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p >> \
+	$seqres.full 2>&1 || _fail "fsck should not fail"
+ls $upperdir/testdir1
+ls $upperdir/testdir2
+ls $upperdir/testdir3
+ls $upperdir/testdir4
+
+# Test orphan whiteout in redirect directory, should remove
+echo "+ Orphan whiteout(4)"
+make_test_dirs
+mkdir $lowerdir/{testdir,origin}
+touch $lowerdir/testdir/foo
+make_redirect_dir $upperdir/testdir "origin"
+make_whiteout $upperdir/testdir/foo
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+ls $upperdir/testdir
+
+# Test valid whiteout in redirect directory cover file in lower
+# redirect origin directory, should not remove
+echo "+ Valid whiteout(2)"
+make_test_dirs
+mkdir $lowerdir/origin
+touch $lowerdir/origin/foo
+make_redirect_dir $upperdir/testdir "origin"
+make_whiteout $upperdir/testdir/foo
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_whiteout $upperdir/testdir/foo
+
+# Test valid whiteout covering lower target whose parent directory
+# merge with a redirect directory in the middle layer, should not remove.
+echo "+ Valid whiteout(3)"
+make_test_dirs
+mkdir -p $lowerdir2/origin/subdir
+touch $lowerdir2/origin/subdir/foo
+make_redirect_dir $lowerdir/testdir "origin"
+mkdir -p $upperdir/testdir/subdir
+make_whiteout $upperdir/testdir/subdir/foo
+
+_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p \
+	>> $seqres.full 2>&1 || _fail "fsck should not fail"
+check_whiteout $upperdir/testdir/subdir/foo
+
+# Test invalid whiteout in opaque subdirectory in a redirect directory,
+# should remove
+echo "+ Orphan whiteout(5)"
+make_test_dirs
+mkdir -p $lowerdir/origin/subdir
+touch $lowerdir/origin/subdir/foo
+make_redirect_dir $upperdir/testdir "origin"
+make_opaque_dir $upperdir/testdir/subdir
+make_whiteout $upperdir/testdir/subdir/foo
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+ls $upperdir/testdir/subdir
+
+# Test valid whiteout in reidrect subdirectory in a opaque directory
+# covering lower target, should not remove
+echo "+ Valid whiteout(4)"
+make_test_dirs
+mkdir $lowerdir/origin
+touch $lowerdir/origin/foo
+make_opaque_dir $upperdir/testdir
+make_redirect_dir $upperdir/testdir/subdir "/origin"
+make_whiteout $upperdir/testdir/subdir/foo
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+        _fail "fsck should not fail"
+check_whiteout $upperdir/testdir/subdir/foo
+
+# success, all done
+status=0
+exit
diff --git a/tests/overlay/201.out b/tests/overlay/201.out
new file mode 100644
index 0000000..157bb85
--- /dev/null
+++ b/tests/overlay/201.out
@@ -0,0 +1,10 @@
+QA output created by 201
++ Orphan whiteout
++ Valid whiteout
++ Orphan whiteout(2)
++ Orphan whiteout(3)
++ Orphan whiteout(4)
++ Valid whiteout(2)
++ Valid whiteout(3)
++ Orphan whiteout(5)
++ Valid whiteout(4)
diff --git a/tests/overlay/group b/tests/overlay/group
index 7e541e4..7c5fcbb 100644
--- a/tests/overlay/group
+++ b/tests/overlay/group
@@ -49,3 +49,4 @@
 044 auto quick copyup hardlink nonsamefs
 047 auto quick copyup hardlink
 048 auto quick copyup hardlink
+201 auto quick fsck
-- 
2.9.5

[xfstests PATCH v6 4/8] overlay: add fsck.overlay redirect directory test

[zhangyi (F)]

Add fsck.overlay test case to test it how to deal with invalid/valid/
duplicate redirect xattr in underlying directories of overlayfs.

Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/overlay/202     | 269 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/202.out |  11 +++
 tests/overlay/group   |   1 +
 3 files changed, 281 insertions(+)
 create mode 100755 tests/overlay/202
 create mode 100644 tests/overlay/202.out

diff --git a/tests/overlay/202 b/tests/overlay/202
new file mode 100755
index 0000000..43e17bc
--- /dev/null
+++ b/tests/overlay/202
@@ -0,0 +1,269 @@
+#! /bin/bash
+# FS QA Test 202
+#
+# Test fsck.overlay how to deal with redirect xattr in overlayfs.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2017 Huawei.  All Rights Reserved.
+# Author: zhangyi (F) <yi.zhang@huawei.com>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1        # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/attr
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs overlay
+_supported_os Linux
+_require_overlay_scratch_dirs
+_require_attrs
+_require_command "$FSCK_OVERLAY_PROG" fsck.overlay
+
+# remove all files from previous tests
+_scratch_mkfs
+
+OVL_REDIRECT_XATTR="trusted.overlay.redirect"
+OVL_OPAQUE_XATTR="trusted.overlay.opaque"
+OVL_OPAQUE_XATTR_VAL="y"
+
+# Create a redirect directory
+make_redirect_dir()
+{
+	local target=$1
+	local value=$2
+
+	mkdir -p $target
+	$SETFATTR_PROG -n $OVL_REDIRECT_XATTR -v $value $target
+}
+
+# Check redirect xattr
+check_redirect()
+{
+	local target=$1
+	local expect=$2
+
+	value=$($GETFATTR_PROG --absolute-names --only-values -n \
+		$OVL_REDIRECT_XATTR $target)
+
+	[[ $value == $expect ]]	|| echo "Redirect xattr incorrect"
+}
+
+check_no_redirect()
+{
+	local target=$1
+
+	value=$($GETFATTR_PROG --absolute-names -d -m \
+		$OVL_REDIRECT_XATTR $target)
+
+	[[ -z "$value" ]] || echo "Redirect xattr not empty"
+}
+
+# Check opaque xattr
+check_opaque()
+{
+	local target=$1
+
+	value=$($GETFATTR_PROG --absolute-names --only-values -n \
+		$OVL_OPAQUE_XATTR $target)
+
+	[[ $value == $OVL_OPAQUE_XATTR_VAL ]] || echo "Opaque xattr incorrect"
+}
+
+# Create a whiteout
+make_whiteout()
+{
+	for arg in $*
+	do
+		mknod $arg c 0 0
+	done
+}
+
+# Check whiteout
+check_whiteout()
+{
+	for arg in $*
+	do
+		local ttype=`stat -c "%F:%t,%T" $arg`
+
+		[[ $ttype == "character special file:0,0" ]] || \
+			echo "Valid whiteout removed incorrectly"
+	done
+}
+
+# Create test directories
+lowerdir=$OVL_BASE_SCRATCH_MNT/lower
+lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2
+upperdir=$OVL_BASE_SCRATCH_MNT/upper
+workdir=$OVL_BASE_SCRATCH_MNT/workdir
+
+make_test_dirs()
+{
+	rm -rf $lowerdir $lowerdir2 $upperdir $workdir
+	mkdir -p $lowerdir $lowerdir2 $upperdir $workdir
+}
+
+# Test invalid redirect xattr point to a nonexistent origin, should remove
+echo "+ Invalid redirect"
+make_test_dirs
+make_redirect_dir $upperdir/testdir "invalid"
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_no_redirect $upperdir/testdir
+
+# Test invalid redirect xattr point to a file origin, should remove
+echo "+ Invalid redirect(2)"
+make_test_dirs
+touch $lowerdir/origin
+make_redirect_dir $upperdir/testdir "origin"
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_no_redirect $upperdir/testdir
+
+# Test valid redirect xattr point to a directory origin in the same directory,
+# should not remove
+echo "+ Valid redirect"
+make_test_dirs
+mkdir $lowerdir/origin
+make_whiteout $upperdir/origin
+make_redirect_dir $upperdir/testdir "origin"
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_redirect $upperdir/testdir "origin"
+
+# Test valid redirect xattr point to a directory origin in different directories
+# should not remove
+echo "+ Valid redirect(2)"
+make_test_dirs
+mkdir $lowerdir/origin
+make_whiteout $upperdir/origin
+make_redirect_dir $upperdir/testdir1/testdir2 "/origin"
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_redirect $upperdir/testdir1/testdir2 "/origin"
+
+# Test valid redirect xattr but missing whiteout to cover lower target,
+# should fix whiteout
+echo "+ Missing whiteout"
+make_test_dirs
+mkdir $lowerdir/origin
+make_redirect_dir $upperdir/testdir "origin"
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_redirect $upperdir/testdir "origin"
+check_whiteout $upperdir/origin
+
+# Test valid redirect xattrs exchanged by rename, should not remove
+echo "+ Valid redirect(3)"
+make_test_dirs
+mkdir $lowerdir/{testdir1,testdir2}
+make_redirect_dir $upperdir/testdir1 "testdir2"
+make_redirect_dir $upperdir/testdir2 "testdir1"
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_redirect $upperdir/testdir1 "testdir2"
+check_redirect $upperdir/testdir2 "testdir1"
+
+# Test invalid redirect xattr with lower same name directory exists,
+# should remove invalid redirect xattr and set opaque in yes mode
+echo "+ Invalid redirect(3)"
+make_test_dirs
+mkdir $lowerdir/testdir
+make_redirect_dir $upperdir/testdir "invalid"
+
+# Question get yes answer: Should set opaque dir ?
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -y >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_no_redirect $upperdir/testdir
+check_opaque $upperdir/testdir
+
+# Test duplicate redirect xattrs point to one origin, should fail in
+# auto mode, and should remove either of the duplicates in yes mode
+echo "+ Duplicate redirect"
+make_test_dirs
+mkdir $lowerdir2/origin
+make_redirect_dir $lowerdir/testdir1 "origin"
+make_redirect_dir $lowerdir/testdir2 "origin"
+make_redirect_dir $upperdir/testdir3 "origin"
+
+_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -p >> \
+	$seqres.full 2>&1 && _fail "fsck should fail"
+
+# Question get yes answer: Duplicate redirect directory, remove xattr ?
+_overlay_fsck_dirs "$lowerdir:$lowerdir2" $upperdir $workdir -y >> \
+	$seqres.full 2>&1 || _fail "fsck should not fail"
+redirect_1=`check_redirect $lowerdir/testdir1 "origin" 2>/dev/null`
+redirect_2=`check_redirect $lowerdir/testdir2 "origin" 2>/dev/null`
+[[ $redirect_1 == $redirect_2 ]] && echo "Redirect xattr incorrect"
+check_no_redirect $upperdir/testdir3
+
+# Test duplicate redirect xattr duplicate with merge directory, should
+# fail in auto mode, and should remove the redirect xattr in yes mode
+echo "+ Duplicate redirect(2)"
+make_test_dirs
+mkdir $lowerdir/origin $upperdir/origin
+make_redirect_dir $upperdir/testdir "origin"
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 && \
+	_fail "fsck should fail"
+
+# Question get yes answer: Duplicate redirect directory, remove xattr ?
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -y >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_no_redirect $upperdir/testdir
+
+# Test duplicate redirect xattr with lower same name directory exists,
+# should remove the duplicate redirect xattr and set opaque in yes mode
+echo "+ Duplicate redirect(3)"
+make_test_dirs
+mkdir $lowerdir/{origin,testdir} $upperdir/origin
+make_redirect_dir $upperdir/testdir "invalid"
+
+# Question one get yes answer: Duplicate redirect directory, remove xattr?
+# Question two get yes answer: Should set opaque dir ?
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -y >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_no_redirect $upperdir/testdir
+check_opaque $upperdir/testdir
+
+# success, all done
+status=0
+exit
diff --git a/tests/overlay/202.out b/tests/overlay/202.out
new file mode 100644
index 0000000..99d2b32
--- /dev/null
+++ b/tests/overlay/202.out
@@ -0,0 +1,11 @@
+QA output created by 202
++ Invalid redirect
++ Invalid redirect(2)
++ Valid redirect
++ Valid redirect(2)
++ Missing whiteout
++ Valid redirect(3)
++ Invalid redirect(3)
++ Duplicate redirect
++ Duplicate redirect(2)
++ Duplicate redirect(3)
diff --git a/tests/overlay/group b/tests/overlay/group
index 7c5fcbb..e39b5e0 100644
--- a/tests/overlay/group
+++ b/tests/overlay/group
@@ -50,3 +50,4 @@
 047 auto quick copyup hardlink
 048 auto quick copyup hardlink
 201 auto quick fsck
+202 auto quick fsck
-- 
2.9.5

[xfstests PATCH v6 5/8] overlay: add fsck.overlay impure xattr test

[zhangyi (F)]

Add fsck.overlay test case to test it how to deal with impure
xattr in underlying directories of overlayfs.

Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/overlay/203     | 143 ++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/203.out |   4 ++
 tests/overlay/group   |   1 +
 3 files changed, 148 insertions(+)
 create mode 100755 tests/overlay/203
 create mode 100644 tests/overlay/203.out

diff --git a/tests/overlay/203 b/tests/overlay/203
new file mode 100755
index 0000000..75b2097
--- /dev/null
+++ b/tests/overlay/203
@@ -0,0 +1,143 @@
+#! /bin/bash
+# FS QA Test 203
+#
+# Test fsck.overlay how to deal with impure xattr in overlayfs.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2017 Huawei.  All Rights Reserved.
+# Author: zhangyi (F) <yi.zhang@huawei.com>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1        # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs overlay
+_supported_os Linux
+_require_overlay_scratch_dirs
+_require_command "$FSCK_OVERLAY_PROG" fsck.overlay
+
+# remove all files from previous tests
+_scratch_mkfs
+
+OVL_REDIRECT_XATTR="trusted.overlay.redirect"
+OVL_IMPURE_XATTR="trusted.overlay.impure"
+OVL_IMPURE_XATTR_VAL="y"
+
+# Create a redirect directory
+make_redirect_dir()
+{
+	local target=$1
+	local value=$2
+
+	mkdir -p $target
+	$SETFATTR_PROG -n $OVL_REDIRECT_XATTR -v $value $target
+}
+
+# Remove impure xattr
+remove_impure()
+{
+	local target=$1
+
+	$SETFATTR_PROG -x $OVL_IMPURE_XATTR $target
+}
+
+# Check impure xattr
+check_impure()
+{
+	local target=$1
+
+	value=$($GETFATTR_PROG --absolute-names --only-values -n \
+		$OVL_IMPURE_XATTR $target)
+
+	[[ $value == $OVL_IMPURE_XATTR_VAL ]] || echo "Missing impure xattr"
+}
+
+# Create test directories
+lowerdir=$OVL_BASE_SCRATCH_MNT/lower
+lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2
+upperdir=$OVL_BASE_SCRATCH_MNT/upper
+workdir=$OVL_BASE_SCRATCH_MNT/workdir
+
+make_test_dirs()
+{
+	rm -rf $lowerdir $lowerdir2 $upperdir $workdir
+	mkdir -p $lowerdir $lowerdir2 $upperdir $workdir
+}
+
+# Test missing impure xattr in directory which has origin targets, should fix
+echo "+ Missing impure"
+make_test_dirs
+mkdir $lowerdir/{testdir1,testdir2}
+mkdir $upperdir/{testdir1,testdir2}
+touch $lowerdir/testdir1/foo
+mkdir $lowerdir/testdir2/subdir
+_overlay_scratch_mount_dirs $lowerdir $upperdir $workdir
+touch $SCRATCH_MNT/testdir1/foo
+touch $SCRATCH_MNT/testdir2/subdir
+$UMOUNT_PROG $SCRATCH_MNT
+remove_impure $upperdir/testdir1
+remove_impure $upperdir/testdir2
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_impure $upperdir/testdir1
+check_impure $upperdir/testdir2
+
+# Test missing impure xattr in directory which has redirect directories,
+# should fix
+echo "+ Missing impure(2)"
+make_test_dirs
+mkdir $lowerdir/origin
+make_redirect_dir $upperdir/testdir/subdir "/origin"
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_impure $upperdir/testdir
+
+# Test missing impure xattr in directory which has merge directories,
+# should fix
+echo "+ Missing impure(3)"
+make_test_dirs
+mkdir $lowerdir/testdir $upperdir/testdir
+
+_overlay_fsck_dirs $lowerdir $upperdir $workdir -p >> $seqres.full 2>&1 || \
+	_fail "fsck should not fail"
+check_impure $upperdir
+
+# success, all done
+status=0
+exit
diff --git a/tests/overlay/203.out b/tests/overlay/203.out
new file mode 100644
index 0000000..dd81788
--- /dev/null
+++ b/tests/overlay/203.out
@@ -0,0 +1,4 @@
+QA output created by 203
++ Missing impure
++ Missing impure(2)
++ Missing impure(3)
diff --git a/tests/overlay/group b/tests/overlay/group
index e39b5e0..f99d89e 100644
--- a/tests/overlay/group
+++ b/tests/overlay/group
@@ -51,3 +51,4 @@
 048 auto quick copyup hardlink
 201 auto quick fsck
 202 auto quick fsck
+203 auto quick fsck
-- 
2.9.5

[xfstests PATCH v6 6/8] overlay/003: fix fs check failure

[zhangyi (F)]

_check_overlay_scratch_fs() will check lowerdir of overlay filesystem,
this case remove this directory after test will lead to check failure,
and it is not really necessary to remove this directory, so keep this
directory.

Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/overlay/003 | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tests/overlay/003 b/tests/overlay/003
index f980edb..154531e 100755
--- a/tests/overlay/003
+++ b/tests/overlay/003
@@ -92,7 +92,6 @@ ls ${SCRATCH_MNT}/
 # unmount overlayfs but not base fs
 $UMOUNT_PROG $SCRATCH_MNT
 
-rm -rf $lowerdir
 echo "Silence is golden"
 # success, all done
 status=0
-- 
2.9.5

[xfstests PATCH v6 7/8] overlay: skip check for tests finished with corrupt filesystem

[zhangyi (F)]

No post-test check of the overlay dirs is required if case leaves
corrupt filesystem after test. We shoud use _require_scratch_nocheck()
instead of _require_scratch() in these cases.

Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
Reviewed-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/overlay/019 | 2 +-
 tests/overlay/031 | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/overlay/019 b/tests/overlay/019
index 3e2bc4e..575aaf7 100755
--- a/tests/overlay/019
+++ b/tests/overlay/019
@@ -46,7 +46,7 @@ rm -f $seqres.full
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_scratch_nocheck
 
 # Remove all files from previous tests
 _scratch_mkfs
diff --git a/tests/overlay/031 b/tests/overlay/031
index 186b409..90a06af 100755
--- a/tests/overlay/031
+++ b/tests/overlay/031
@@ -67,7 +67,7 @@ rm -f $seqres.full
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_scratch_nocheck
 
 # remove all files from previous runs
 _scratch_mkfs
-- 
2.9.5

[xfstests PATCH v6 8/8] overlay: correct scratch dirs check

[zhangyi (F)]

Tests that use _overlay_scratch_mount_dirs instead of _scratch_mount
should use _require_overlay_scratch_dirs instead of _require_scratch.
Those tests can optionally call _overlay_check_scratch_dirs at the end
of the test or before _scratch_umount to umount $SCRATCH_MNT and
run the checker.

Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
---
 tests/overlay/005 |  6 +++++-
 tests/overlay/010 |  6 +++++-
 tests/overlay/014 | 10 +++++++++-
 tests/overlay/035 |  6 +++++-
 tests/overlay/036 |  5 ++++-
 tests/overlay/037 |  6 +++++-
 tests/overlay/038 | 10 +++++++++-
 tests/overlay/041 | 10 +++++++++-
 tests/overlay/043 |  6 +++++-
 tests/overlay/044 |  6 +++++-
 10 files changed, 61 insertions(+), 10 deletions(-)

diff --git a/tests/overlay/005 b/tests/overlay/005
index 17992a3..f502d3f 100755
--- a/tests/overlay/005
+++ b/tests/overlay/005
@@ -54,7 +54,7 @@ rm -f $seqres.full
 # Modify as appropriate.
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_overlay_scratch_dirs
 _require_loop
 
 # Remove all files from previous tests
@@ -102,6 +102,10 @@ $XFS_IO_PROG -f -c "o" ${SCRATCH_MNT}/test_file \
 # unmount overlayfs
 $UMOUNT_PROG $SCRATCH_MNT
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check optionally dirs after test
+_overlay_check_scratch_dirs $lowerd $upperd $workd
+
 # unmount undelying xfs, this tiggers panic if memleak happens
 $UMOUNT_PROG ${OVL_BASE_SCRATCH_MNT}/uppermnt
 $UMOUNT_PROG ${OVL_BASE_SCRATCH_MNT}/lowermnt
diff --git a/tests/overlay/010 b/tests/overlay/010
index f55ebec..f3b50d7 100755
--- a/tests/overlay/010
+++ b/tests/overlay/010
@@ -48,7 +48,7 @@ rm -f $seqres.full
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_overlay_scratch_dirs
 
 # Remove all files from previous tests
 _scratch_mkfs
@@ -70,6 +70,10 @@ mknod $lowerdir2/testdir/a c 0 0
 _overlay_scratch_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir
 rm -rf $SCRATCH_MNT/testdir
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check optionally dirs after test
+_overlay_check_scratch_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir
+
 # success, all done
 echo "Silence is golden"
 status=0
diff --git a/tests/overlay/014 b/tests/overlay/014
index 9f308d3..069d8b6 100755
--- a/tests/overlay/014
+++ b/tests/overlay/014
@@ -53,7 +53,7 @@ rm -f $seqres.full
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_overlay_scratch_dirs
 
 # Remove all files from previous tests
 _scratch_mkfs
@@ -78,6 +78,10 @@ mkdir -p $SCRATCH_MNT/testdir/visibledir
 # unmount overlayfs but not base fs
 $UMOUNT_PROG $SCRATCH_MNT
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check optionally dirs after each test
+_overlay_check_scratch_dirs $lowerdir1 $lowerdir2 $workdir2
+
 # mount overlay again, with lowerdir1 and lowerdir2 as multiple lowerdirs,
 # and create a new file in testdir, triggers copyup from lowerdir,
 # copyup should not copy overlayfs private xattr
@@ -90,6 +94,10 @@ $UMOUNT_PROG $SCRATCH_MNT
 _overlay_scratch_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir
 ls $SCRATCH_MNT/testdir
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check optionally dirs after each test
+_overlay_check_scratch_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir
+
 # success, all done
 status=0
 exit
diff --git a/tests/overlay/035 b/tests/overlay/035
index 0544774..bc73013 100755
--- a/tests/overlay/035
+++ b/tests/overlay/035
@@ -51,7 +51,7 @@ rm -f $seqres.full
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_overlay_scratch_dirs
 _require_chattr i
 
 # Remove all files from previous tests
@@ -81,6 +81,10 @@ _overlay_scratch_mount_dirs $lowerdir2 $upperdir $workdir
 touch $SCRATCH_MNT/bar 2>&1 | _filter_scratch
 _scratch_remount rw 2>&1 | _filter_ro_mount
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check optionally dirs after test
+_overlay_check_scratch_dirs $lowerdir2 $upperdir $workdir
+
 # success, all done
 status=0
 exit
diff --git a/tests/overlay/036 b/tests/overlay/036
index e0c13ae..ca5adbd 100755
--- a/tests/overlay/036
+++ b/tests/overlay/036
@@ -69,7 +69,7 @@ rm -f $seqres.full
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_overlay_scratch_dirs
 _require_scratch_feature index
 
 # Remove all files from previous tests
@@ -110,6 +110,9 @@ _overlay_mount_dirs $lowerdir2 $upperdir $workdir2 \
 _overlay_mount_dirs $lowerdir2 $upperdir2 $workdir \
 	    overlay3 $SCRATCH_MNT -oindex=on 2>&1 | _filter_busy_mount
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# but which will not lead to inconsistency after this mount test,
+# so do not check optionally dirs after test
 
 # success, all done
 status=0
diff --git a/tests/overlay/037 b/tests/overlay/037
index 6710dda..60290a1 100755
--- a/tests/overlay/037
+++ b/tests/overlay/037
@@ -55,7 +55,7 @@ rm -f $seqres.full
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_overlay_scratch_dirs
 _require_scratch_feature index
 
 # Remove all files from previous tests
@@ -87,6 +87,10 @@ $UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
 # Mount overlay with original lowerdir, upperdir, workdir and index=on - expect success
 _overlay_scratch_mount_dirs $lowerdir $upperdir $workdir -oindex=on
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# but which will not lead to inconsistency after this mount test,
+# so do not check optionally dirs after test
+
 # success, all done
 status=0
 exit
diff --git a/tests/overlay/038 b/tests/overlay/038
index bd87156..aa20cbc 100755
--- a/tests/overlay/038
+++ b/tests/overlay/038
@@ -46,7 +46,7 @@ _cleanup()
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_overlay_scratch_dirs
 _require_attrs
 _require_test_program "t_dir_type"
 
@@ -161,6 +161,10 @@ subdir_d=$($here/src/t_dir_type $pure_lower_dir $pure_lower_subdir_st_ino)
 [[ $subdir_d == "subdir d" ]] || \
 	echo "Merged dir: Invalid d_ino reported for subdir"
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check after each test
+_check_scratch_fs
+
 _scratch_unmount
 
 # Verify pure lower residing in dir which has another lower layer
@@ -202,6 +206,10 @@ subdir_d=$($here/src/t_dir_type $pure_lower_dir $pure_lower_subdir_st_ino)
 [[ $subdir_d == "subdir d" ]] || \
 	echo "Pure lower in dir which has another lower layer: Invalid d_ino reported for subdir"
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check optionally dirs after each test
+_overlay_check_scratch_dirs "$middir:$lowerdir" $upperdir $workdir
+
 echo "Silence is golden"
 status=0
 exit
diff --git a/tests/overlay/041 b/tests/overlay/041
index 4152107..607908a 100755
--- a/tests/overlay/041
+++ b/tests/overlay/041
@@ -48,7 +48,7 @@ _cleanup()
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_overlay_scratch_dirs
 _require_test
 _require_attrs
 _require_test_program "t_dir_type"
@@ -167,6 +167,10 @@ subdir_d=$($here/src/t_dir_type $pure_lower_dir $pure_lower_subdir_st_ino)
 [[ $subdir_d == "subdir d" ]] || \
 	echo "Merged dir: Invalid d_ino reported for subdir"
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check optionally dirs after each test
+_overlay_check_scratch_dirs $lowerdir $upperdir $workdir
+
 _scratch_unmount
 
 # Verify pure lower residing in dir which has another lower layer
@@ -208,6 +212,10 @@ subdir_d=$($here/src/t_dir_type $pure_lower_dir $pure_lower_subdir_st_ino)
 [[ $subdir_d == "subdir d" ]] || \
 	echo "Pure lower in dir which has another lower layer: Invalid d_ino reported for subdir"
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check optionally dirs after each test
+_overlay_check_scratch_dirs "$middir:$lowerdir" $upperdir $workdir
+
 echo "Silence is golden"
 status=0
 exit
diff --git a/tests/overlay/043 b/tests/overlay/043
index 858b6a9..76e0dd9 100755
--- a/tests/overlay/043
+++ b/tests/overlay/043
@@ -56,7 +56,7 @@ _cleanup()
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_overlay_scratch_dirs
 _require_test
 _require_test_program "af_unix"
 _require_test_program "t_dir_type"
@@ -155,6 +155,10 @@ _overlay_scratch_mount_dirs $lowerdir $upperdir $workdir \
 # Compare inode numbers before/after mount cycle
 check_inode_numbers $testdir $tmp.after_move $tmp.after_cycle
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check optionally dirs after test
+_overlay_check_scratch_dirs $lowerdir $upperdir $workdir
+
 echo "Silence is golden"
 status=0
 exit
diff --git a/tests/overlay/044 b/tests/overlay/044
index 9c0ff04..e929068 100755
--- a/tests/overlay/044
+++ b/tests/overlay/044
@@ -49,7 +49,7 @@ _cleanup()
 # real QA test starts here
 _supported_fs overlay
 _supported_os Linux
-_require_scratch
+_require_overlay_scratch_dirs
 _require_test
 _require_scratch_feature index
 _require_test_program "t_dir_type"
@@ -141,5 +141,9 @@ echo "== After write two =="
 cat $FILES
 check_ino_nlink $SCRATCH_MNT $tmp.after_one $tmp.after_two
 
+# We called _require_overlay_scratch_dirs instead of _require_scratch,
+# do check optionally dirs after test
+_overlay_check_scratch_dirs $lowerdir $upperdir $workdir
+
 status=0
 exit
-- 
2.9.5

Re: [xfstests PATCH v6 1/8] common/rc: modify mounted check helper

[Amir Goldstein]

On Wed, Jan 24, 2018 at 9:33 AM, zhangyi (F) <yi.zhang@huawei.com> wrote:
> Modify _is_mounted() to accept a dir and fstype as input, and check
> whether this dir is a mount point of specified fstype. This patch also
> fix the problem of invalid fstype check and remove invalid error
> message.
>
> Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
> ---
>  common/rc | 30 +++++++++++++++---------------
>  1 file changed, 15 insertions(+), 15 deletions(-)
>
> diff --git a/common/rc b/common/rc
> index 77a4eb4..2e7aee5 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -2372,27 +2372,27 @@ _scratch_mkfs_richacl()
>         esac
>  }
>
> -# check that a FS on a device is mounted
> +# check that a FS on a device is mounted or a dir is a mount point
>  # if so, return mount point
>  #
>  _is_mounted()
>  {
> -    if [ $# -ne 1 ]
> -    then
> -       echo "Usage: _is_mounted device" 1>&2
> -       exit 1
> -    fi
> +       local name
> +       local fstype
>
> -    device=$1
> +       if [ $# -eq 2 ]; then
> +               name=$1
> +               fstype=$2
> +       elif [ $# -eq 1 ]; then
> +               name=$1

missing fstype=$FSTYP here?
probably need to check better that this change did not break _is_mounted users

suggesting:

    if [ $# -lt 1 ]
...
       local name=$1
       local fstype=${2-$FSTYP}

Re: [xfstests PATCH v6 2/8] overlay: add filesystem check helper

[Amir Goldstein]

On Wed, Jan 24, 2018 at 9:33 AM, zhangyi (F) <yi.zhang@huawei.com> wrote:
> Add filesystem check helpers for the upcoming fsck.overlay utility.
> Hook these helpers to _check_test_fs and _check_scratch_fs for constants
> underlying dirs of overlay filesystem, and introduce scratch check helpers
> for optionally lower/upper/work dirs. These helpers works only if
> fsck.overlay exists.
>
> [ _check_test_fs/_check_scratch_fs part picked from Amir's patch, thanks ]
>
> Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>
> ---
>  common/config  |   1 +
>  common/overlay | 132 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  common/rc      |   4 +-
>  3 files changed, 135 insertions(+), 2 deletions(-)
>
> diff --git a/common/config b/common/config
> index 5f40413..71115bd 100644
> --- a/common/config
> +++ b/common/config
> @@ -236,6 +236,7 @@ case "$HOSTOS" in
>          export MKFS_REISER4_PROG="`set_prog_path mkfs.reiser4`"
>         export E2FSCK_PROG="`set_prog_path e2fsck`"
>         export TUNE2FS_PROG="`set_prog_path tune2fs`"
> +       export FSCK_OVERLAY_PROG="`set_prog_path fsck.overlay`"
>          ;;
>  esac
>
> diff --git a/common/overlay b/common/overlay
> index 1da4ab1..ce3f279 100644
> --- a/common/overlay
> +++ b/common/overlay
> @@ -151,3 +151,135 @@ _require_scratch_overlay_feature()
>                 _notrun "${FSTYP} feature '${feature}' cannot be enabled on ${SCRATCH_DEV}"
>         _scratch_unmount
>  }
> +
> +# Require the same scratch device as _require_scratch, but do not check
> +# the constants OVL_LOWER/OVL_UPPER/OVL_WORK dirs, should use together
> +# with optionally lower/upper/work dirs and do check explicitly after test.
> +_require_overlay_scratch_dirs()
> +{
> +       _require_scratch_nocheck
> +}
> +
> +_overlay_fsck_dirs()
> +{
> +       local lowerdir=$1
> +       local upperdir=$2
> +       local workdir=$3
> +       local options=$4
> +
> +       [[ ! -x "$FSCK_OVERLAY_PROG" ]] && return 0
> +
> +       $FSCK_OVERLAY_PROG -o lowerdir=$lowerdir -o upperdir=$upperdir \
> +                          -o workdir=$workdir $options
> +}
> +
> +_overlay_check_dirs()
> +{
> +       local lowerdir=$1
> +       local upperdir=$2
> +       local workdir=$3
> +       local err=0
> +
> +       _overlay_fsck_dirs $* $FSCK_OPTIONS >>$tmp.fsck 2>&1
> +       if [ $? -ne 0 ]; then
> +               _log_err "_overlay_check_fs: overlayfs on $lowerdir,$upperdir,$workdir is inconsistent"
> +
> +               echo "*** fsck.overlay output ***"      >>$seqres.full
> +               cat $tmp.fsck                           >>$seqres.full
> +               echo "*** end fsck.overlay output"      >>$seqres.full
> +
> +               echo "*** mount output ***"             >>$seqres.full
> +               _mount                                  >>$seqres.full
> +               echo "*** end mount output"             >>$seqres.full
> +
> +               err=1
> +       fi
> +       rm -f $tmp.fsck
> +
> +       return $err
> +}
> +
> +# Check the same mnt/dev of _check_overlay_scratch_fs, but check optionally
> +# lower/upper/work dirs of overlay filesystem, should use together with
> +# _require_overlay_scratch_dirs
> +_overlay_check_scratch_dirs()
> +{
> +       local lowerdir=$1
> +       local upperdir=$2
> +       local workdir=$3
> +
> +       # Need to umount overlay for scratch dir check
> +       local ovl_mounted=`_is_mounted $SCRATCH_MNT overlay`

Here the argument 'overlay' seem redundant because _is_mounted
can check $FSTYP

> +       [ -z "$ovl_mounted" ] || $UMOUNT_PROG $SCRATCH_MNT
> +
> +       _overlay_check_dirs $lowerdir $upperdir $workdir
> +       local ret=$?
> +
> +       if [ $ret -eq 0 -a -n "$ovl_mounted" ]; then
> +               # overlay was mounted, remount besides extra mount options
> +               _overlay_scratch_mount_dirs $lowerdir $upperdir $workdir
> +               ret=$?
> +       fi
> +
> +       return $ret
> +}
> +
> +_overlay_check_fs()
> +{
> +       local ovl_mnt=$1
> +       local base_dev=$4
> +       local base_mnt=$5
> +       shift 1
> +
> +       [ "$FSTYP" = overlay ] || return 0
> +
> +       # Base fs needs to be mounted to check overlay dirs
> +       local base_mounted=""
> +       [ -z "$base_dev" ] || base_mounted=`_is_mounted $base_dev`

That looks like I have a bug in my original implementation of _overlay_check_fs
because $FSTYP" = overlay _is_mounted here will not actually find a mounted
base dev. Need to pass $OVL_BASE_FSTYP to your improved _is_mounted
helper.
No?

> +
> +       if [ -z "$base_mounted" ]; then
> +               _overlay_base_mount $*
> +       else
> +               # Need to umount overlay for dir check
> +               local ovl_mounted=`_is_mounted $ovl_mnt overlay`
> +               [ -z "$ovl_mounted" ] || $UMOUNT_PROG $ovl_mnt
> +       fi
> +
> +       _overlay_check_dirs $base_mnt/$OVL_LOWER $base_mnt/$OVL_UPPER \
> +                           $base_mnt/$OVL_WORK
> +       local ret=$?
> +
> +       if [ -z "$base_mounted" ]; then
> +               _overlay_base_unmount "$base_dev" "$base_mnt"
> +       elif [ $ret -eq 0 -a -n "$ovl_mounted" ]; then
> +               # overlay was mounted, remount besides extra mount options
> +               _overlay_mount $base_mnt $ovl_mnt
> +               ret=$?
> +       fi
> +
> +       if [ $ret != 0 ]; then
> +               status=1
> +               if [ "$iam" != "check" ]; then
> +                       exit 1
> +               fi
> +               return 1
> +       fi
> +
> +       return 0
> +}
> +
> +_check_overlay_test_fs()
> +{
> +       _overlay_check_fs "$TEST_DIR" \
> +               OVL_BASE_TEST_DEV OVL_BASE_TEST_DIR \
> +               "$OVL_BASE_TEST_DEV" "$OVL_BASE_TEST_DIR" \
> +               $TEST_FS_MOUNT_OPTS $SELINUX_MOUNT_OPTIONS
> +}
> +
> +_check_overlay_scratch_fs()
> +{
> +       _overlay_check_fs "$SCRATCH_MNT" \
> +               OVL_BASE_SCRATCH_DEV OVL_BASE_SCRATCH_MNT \
> +               "$OVL_BASE_SCRATCH_DEV" "$OVL_BASE_SCRATCH_MNT" \
> +               $OVL_BASE_MOUNT_OPTIONS $SELINUX_MOUNT_OPTIONS
> +}
> diff --git a/common/rc b/common/rc
> index 2e7aee5..b2faf9a 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -2590,7 +2590,7 @@ _check_test_fs()
>         # no way to check consistency for GlusterFS
>         ;;
>      overlay)
> -       # no way to check consistency for overlay
> +       _check_overlay_test_fs
>         ;;
>      pvfs2)
>         ;;
> @@ -2648,7 +2648,7 @@ _check_scratch_fs()
>         # no way to check consistency for GlusterFS
>         ;;
>      overlay)
> -       # no way to check consistency for overlay
> +       _check_overlay_scratch_fs
>         ;;
>      pvfs2)
>         ;;
> --
> 2.9.5
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [xfstests PATCH v6 8/8] overlay: correct scratch dirs check

[Amir Goldstein]

On Wed, Jan 24, 2018 at 9:33 AM, zhangyi (F) <yi.zhang@huawei.com> wrote:
> Tests that use _overlay_scratch_mount_dirs instead of _scratch_mount
> should use _require_overlay_scratch_dirs instead of _require_scratch.
> Those tests can optionally call _overlay_check_scratch_dirs at the end
> of the test or before _scratch_umount to umount $SCRATCH_MNT and
> run the checker.
>
> Signed-off-by: zhangyi (F) <yi.zhang@huawei.com>

Reviewed-by: Amir Goldstein <amir73il@gmail.com>

> ---
>  tests/overlay/005 |  6 +++++-
>  tests/overlay/010 |  6 +++++-
>  tests/overlay/014 | 10 +++++++++-
>  tests/overlay/035 |  6 +++++-
>  tests/overlay/036 |  5 ++++-
>  tests/overlay/037 |  6 +++++-
>  tests/overlay/038 | 10 +++++++++-
>  tests/overlay/041 | 10 +++++++++-
>  tests/overlay/043 |  6 +++++-
>  tests/overlay/044 |  6 +++++-
>  10 files changed, 61 insertions(+), 10 deletions(-)
>
> diff --git a/tests/overlay/005 b/tests/overlay/005
> index 17992a3..f502d3f 100755
> --- a/tests/overlay/005
> +++ b/tests/overlay/005
> @@ -54,7 +54,7 @@ rm -f $seqres.full
>  # Modify as appropriate.
>  _supported_fs overlay
>  _supported_os Linux
> -_require_scratch
> +_require_overlay_scratch_dirs
>  _require_loop
>
>  # Remove all files from previous tests
> @@ -102,6 +102,10 @@ $XFS_IO_PROG -f -c "o" ${SCRATCH_MNT}/test_file \
>  # unmount overlayfs
>  $UMOUNT_PROG $SCRATCH_MNT
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check optionally dirs after test
> +_overlay_check_scratch_dirs $lowerd $upperd $workd
> +
>  # unmount undelying xfs, this tiggers panic if memleak happens
>  $UMOUNT_PROG ${OVL_BASE_SCRATCH_MNT}/uppermnt
>  $UMOUNT_PROG ${OVL_BASE_SCRATCH_MNT}/lowermnt
> diff --git a/tests/overlay/010 b/tests/overlay/010
> index f55ebec..f3b50d7 100755
> --- a/tests/overlay/010
> +++ b/tests/overlay/010
> @@ -48,7 +48,7 @@ rm -f $seqres.full
>  # real QA test starts here
>  _supported_fs overlay
>  _supported_os Linux
> -_require_scratch
> +_require_overlay_scratch_dirs
>
>  # Remove all files from previous tests
>  _scratch_mkfs
> @@ -70,6 +70,10 @@ mknod $lowerdir2/testdir/a c 0 0
>  _overlay_scratch_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir
>  rm -rf $SCRATCH_MNT/testdir
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check optionally dirs after test
> +_overlay_check_scratch_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir
> +
>  # success, all done
>  echo "Silence is golden"
>  status=0
> diff --git a/tests/overlay/014 b/tests/overlay/014
> index 9f308d3..069d8b6 100755
> --- a/tests/overlay/014
> +++ b/tests/overlay/014
> @@ -53,7 +53,7 @@ rm -f $seqres.full
>  # real QA test starts here
>  _supported_fs overlay
>  _supported_os Linux
> -_require_scratch
> +_require_overlay_scratch_dirs
>
>  # Remove all files from previous tests
>  _scratch_mkfs
> @@ -78,6 +78,10 @@ mkdir -p $SCRATCH_MNT/testdir/visibledir
>  # unmount overlayfs but not base fs
>  $UMOUNT_PROG $SCRATCH_MNT
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check optionally dirs after each test
> +_overlay_check_scratch_dirs $lowerdir1 $lowerdir2 $workdir2
> +
>  # mount overlay again, with lowerdir1 and lowerdir2 as multiple lowerdirs,
>  # and create a new file in testdir, triggers copyup from lowerdir,
>  # copyup should not copy overlayfs private xattr
> @@ -90,6 +94,10 @@ $UMOUNT_PROG $SCRATCH_MNT
>  _overlay_scratch_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir
>  ls $SCRATCH_MNT/testdir
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check optionally dirs after each test
> +_overlay_check_scratch_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir
> +
>  # success, all done
>  status=0
>  exit
> diff --git a/tests/overlay/035 b/tests/overlay/035
> index 0544774..bc73013 100755
> --- a/tests/overlay/035
> +++ b/tests/overlay/035
> @@ -51,7 +51,7 @@ rm -f $seqres.full
>  # real QA test starts here
>  _supported_fs overlay
>  _supported_os Linux
> -_require_scratch
> +_require_overlay_scratch_dirs
>  _require_chattr i
>
>  # Remove all files from previous tests
> @@ -81,6 +81,10 @@ _overlay_scratch_mount_dirs $lowerdir2 $upperdir $workdir
>  touch $SCRATCH_MNT/bar 2>&1 | _filter_scratch
>  _scratch_remount rw 2>&1 | _filter_ro_mount
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check optionally dirs after test
> +_overlay_check_scratch_dirs $lowerdir2 $upperdir $workdir
> +
>  # success, all done
>  status=0
>  exit
> diff --git a/tests/overlay/036 b/tests/overlay/036
> index e0c13ae..ca5adbd 100755
> --- a/tests/overlay/036
> +++ b/tests/overlay/036
> @@ -69,7 +69,7 @@ rm -f $seqres.full
>  # real QA test starts here
>  _supported_fs overlay
>  _supported_os Linux
> -_require_scratch
> +_require_overlay_scratch_dirs
>  _require_scratch_feature index
>
>  # Remove all files from previous tests
> @@ -110,6 +110,9 @@ _overlay_mount_dirs $lowerdir2 $upperdir $workdir2 \
>  _overlay_mount_dirs $lowerdir2 $upperdir2 $workdir \
>             overlay3 $SCRATCH_MNT -oindex=on 2>&1 | _filter_busy_mount
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# but which will not lead to inconsistency after this mount test,
> +# so do not check optionally dirs after test
>
>  # success, all done
>  status=0
> diff --git a/tests/overlay/037 b/tests/overlay/037
> index 6710dda..60290a1 100755
> --- a/tests/overlay/037
> +++ b/tests/overlay/037
> @@ -55,7 +55,7 @@ rm -f $seqres.full
>  # real QA test starts here
>  _supported_fs overlay
>  _supported_os Linux
> -_require_scratch
> +_require_overlay_scratch_dirs
>  _require_scratch_feature index
>
>  # Remove all files from previous tests
> @@ -87,6 +87,10 @@ $UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
>  # Mount overlay with original lowerdir, upperdir, workdir and index=on - expect success
>  _overlay_scratch_mount_dirs $lowerdir $upperdir $workdir -oindex=on
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# but which will not lead to inconsistency after this mount test,
> +# so do not check optionally dirs after test
> +
>  # success, all done
>  status=0
>  exit
> diff --git a/tests/overlay/038 b/tests/overlay/038
> index bd87156..aa20cbc 100755
> --- a/tests/overlay/038
> +++ b/tests/overlay/038
> @@ -46,7 +46,7 @@ _cleanup()
>  # real QA test starts here
>  _supported_fs overlay
>  _supported_os Linux
> -_require_scratch
> +_require_overlay_scratch_dirs
>  _require_attrs
>  _require_test_program "t_dir_type"
>
> @@ -161,6 +161,10 @@ subdir_d=$($here/src/t_dir_type $pure_lower_dir $pure_lower_subdir_st_ino)
>  [[ $subdir_d == "subdir d" ]] || \
>         echo "Merged dir: Invalid d_ino reported for subdir"
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check after each test
> +_check_scratch_fs
> +
>  _scratch_unmount
>
>  # Verify pure lower residing in dir which has another lower layer
> @@ -202,6 +206,10 @@ subdir_d=$($here/src/t_dir_type $pure_lower_dir $pure_lower_subdir_st_ino)
>  [[ $subdir_d == "subdir d" ]] || \
>         echo "Pure lower in dir which has another lower layer: Invalid d_ino reported for subdir"
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check optionally dirs after each test
> +_overlay_check_scratch_dirs "$middir:$lowerdir" $upperdir $workdir
> +
>  echo "Silence is golden"
>  status=0
>  exit
> diff --git a/tests/overlay/041 b/tests/overlay/041
> index 4152107..607908a 100755
> --- a/tests/overlay/041
> +++ b/tests/overlay/041
> @@ -48,7 +48,7 @@ _cleanup()
>  # real QA test starts here
>  _supported_fs overlay
>  _supported_os Linux
> -_require_scratch
> +_require_overlay_scratch_dirs
>  _require_test
>  _require_attrs
>  _require_test_program "t_dir_type"
> @@ -167,6 +167,10 @@ subdir_d=$($here/src/t_dir_type $pure_lower_dir $pure_lower_subdir_st_ino)
>  [[ $subdir_d == "subdir d" ]] || \
>         echo "Merged dir: Invalid d_ino reported for subdir"
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check optionally dirs after each test
> +_overlay_check_scratch_dirs $lowerdir $upperdir $workdir
> +
>  _scratch_unmount
>
>  # Verify pure lower residing in dir which has another lower layer
> @@ -208,6 +212,10 @@ subdir_d=$($here/src/t_dir_type $pure_lower_dir $pure_lower_subdir_st_ino)
>  [[ $subdir_d == "subdir d" ]] || \
>         echo "Pure lower in dir which has another lower layer: Invalid d_ino reported for subdir"
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check optionally dirs after each test
> +_overlay_check_scratch_dirs "$middir:$lowerdir" $upperdir $workdir
> +
>  echo "Silence is golden"
>  status=0
>  exit
> diff --git a/tests/overlay/043 b/tests/overlay/043
> index 858b6a9..76e0dd9 100755
> --- a/tests/overlay/043
> +++ b/tests/overlay/043
> @@ -56,7 +56,7 @@ _cleanup()
>  # real QA test starts here
>  _supported_fs overlay
>  _supported_os Linux
> -_require_scratch
> +_require_overlay_scratch_dirs
>  _require_test
>  _require_test_program "af_unix"
>  _require_test_program "t_dir_type"
> @@ -155,6 +155,10 @@ _overlay_scratch_mount_dirs $lowerdir $upperdir $workdir \
>  # Compare inode numbers before/after mount cycle
>  check_inode_numbers $testdir $tmp.after_move $tmp.after_cycle
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check optionally dirs after test
> +_overlay_check_scratch_dirs $lowerdir $upperdir $workdir
> +
>  echo "Silence is golden"
>  status=0
>  exit
> diff --git a/tests/overlay/044 b/tests/overlay/044
> index 9c0ff04..e929068 100755
> --- a/tests/overlay/044
> +++ b/tests/overlay/044
> @@ -49,7 +49,7 @@ _cleanup()
>  # real QA test starts here
>  _supported_fs overlay
>  _supported_os Linux
> -_require_scratch
> +_require_overlay_scratch_dirs
>  _require_test
>  _require_scratch_feature index
>  _require_test_program "t_dir_type"
> @@ -141,5 +141,9 @@ echo "== After write two =="
>  cat $FILES
>  check_ino_nlink $SCRATCH_MNT $tmp.after_one $tmp.after_two
>
> +# We called _require_overlay_scratch_dirs instead of _require_scratch,
> +# do check optionally dirs after test
> +_overlay_check_scratch_dirs $lowerdir $upperdir $workdir
> +
>  status=0
>  exit
> --
> 2.9.5
>

Re: [xfstests PATCH v6 0/8] overlay: add fsck.overlay basic tests

[Amir Goldstein]

On Wed, Jan 24, 2018 at 9:33 AM, zhangyi (F) <yi.zhang@huawei.com> wrote:
> Hi All:
>
> Here is the sixth version of test cases for the fsck.overlay[1].

You know, I wanted to say this earlier.
This series is a mix of 2 independent series and I don't know if
it is best for you to post them together.

The 3 tests added to overlay/fsck group only require the minimal
helper _overlay_fsck_dirs.

The rest of the patches really do something very different than testing
overlay.fsck itself.

If you hadn't combined the 2 series together, the overlay/fsck series
would have completed review 2-3 revisions ago.

I'll leave it to you to decide if to post the next revision as one series
or two, but I think it would be better if you at least re-order the
patches so that the already reviewed overlay/fsck patches
with the simple _overlay_fsck_dirs helper are before the rest of
the patches.

Thanks,
Amir.

Re: [xfstests PATCH v6 0/8] overlay: add fsck.overlay basic tests

[zhangyi (F)]

On 2018/1/24 17:41, Amir Goldstein Wrote:
> On Wed, Jan 24, 2018 at 9:33 AM, zhangyi (F) <yi.zhang@huawei.com> wrote:
>> Hi All:
>>
>> Here is the sixth version of test cases for the fsck.overlay[1].
> 
> You know, I wanted to say this earlier.
> This series is a mix of 2 independent series and I don't know if
> it is best for you to post them together.
> 
> The 3 tests added to overlay/fsck group only require the minimal
> helper _overlay_fsck_dirs.
> 
> The rest of the patches really do something very different than testing
> overlay.fsck itself.
> 
> If you hadn't combined the 2 series together, the overlay/fsck series
> would have completed review 2-3 revisions ago.
> 
> I'll leave it to you to decide if to post the next revision as one series
> or two, but I think it would be better if you at least re-order the
> patches so that the already reviewed overlay/fsck patches
> with the simple _overlay_fsck_dirs helper are before the rest of
> the patches.
> 
Sure, it's better to split into two independent series, will do.

Thanks,
Yi.

Re: [xfstests PATCH v6 2/8] overlay: add filesystem check helper

[zhangyi (F)]

On 2018/1/24 17:31, Amir Goldstein Write:
> On Wed, Jan 24, 2018 at 9:33 AM, zhangyi (F) <yi.zhang@huawei.com> wrote:
>> +_overlay_check_fs()
>> +{
>> +       local ovl_mnt=$1
>> +       local base_dev=$4
>> +       local base_mnt=$5
>> +       shift 1
>> +
>> +       [ "$FSTYP" = overlay ] || return 0
>> +
>> +       # Base fs needs to be mounted to check overlay dirs
>> +       local base_mounted=""
>> +       [ -z "$base_dev" ] || base_mounted=`_is_mounted $base_dev`
> 
> That looks like I have a bug in my original implementation of _overlay_check_fs
> because $FSTYP" = overlay _is_mounted here will not actually find a mounted
> base dev. Need to pass $OVL_BASE_FSTYP to your improved _is_mounted
> helper.
> No?
> 
Yes, we not found this bug becasue previous _is_mounted missing $FSTYP check,
I will improve it as you suggested.

Thanks,
Yi.