[PATCH v5 0/10] fstests: new way to run overlay tests

public inbox for fstests@vger.kernel.org
 help / color / mirror / Atom feed

* [PATCH v5 0/10] fstests: new way to run overlay tests
@ 2017-02-28 12:18 Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 01/10] fstests: sanity check that test partitions are not mounted elsewhere Amir Goldstein
                   ` (10 more replies)
  0 siblings, 11 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

Hi Eryu,

I rebased my branch to your master and made some minor adaptations
to Xiong's new tests. Also reordered the patches:

- patches 1-6 are generic fixes and re-factoring - if you wish,
  you can merge them before the overlay config changes.

- patches 7-10 implement the new overlay config. patches 7 and 10
  include adaptations of Xiong's new tests to new config.

This is the 5th revision of new overlayfs config.
The main motivation of this work is to help catch overlayfs bugs
related to leaking objects in underlying (base) fs.

With this change, all you have to do to run overlay tests if you
already have a local.config setup to test a local file system is:
 ./check -overlay

See README.overlay for some examples of config files to use with -overlay.

The legacy overlayfs configuration, where TEST_DEV is a directory
still works, but it should be deprecated.

Tested ./check -overlay -g quick with both legacy overlay configuration
and the new base fs configuration.

Eryu wished to keep these changes soaking on the list for while,
so for those who can help with testing, you can get pull the branch
for testing from my github tree [1].

Thanks,
Amir.

[1] https://github.com/amir73il/xfstests/tree/ovl_base_fs

v5:
- Rebased to weekly update
- Adapted new overlay tests by Xiong to new config
- Reordered patches "fixes and re-factoring first"

v4:
- Address review comments from Eryu
- Fix sanity checks for already mounted scratch base fs
- Fix handling of SCRATCH_DEV_POOL
- Fix problems when run in kvm-xfstests
- Add README.overlay with example config files
- Remove documentation for overlay config sections

v3:
- Mount cycle base test fs
- Fix bugs in non overlay specific sanity checks
- Run -overlay test with existing config file of base fs
- Run overlay tests per base fs by adding overlay config sections

v2:
- Test and scratch base dirs each have thier own base fs
- Support mount cycles of base fs for scratch tests

v1:
- Both test and scratch base dirs on a single base fs


Amir Goldstein (10):
  fstests: sanity check that test partitions are not mounted elsewhere
  fstests: use _test_mount() consistently
  fstests: canonicalize mount points on every config section
  fstests: fix test and scratch filters for overlapping DEV/MNT paths
  fstests: allow overlay SCRATCH_DEV to be a base fs mount point
  generic/064: access SCRATCH_MNT after _scratch_mount
  overlay: rename OVERLAY_LOWER/UPPER/WORK_DIR
  overlay: configure TEST/SCRATCH vars to base fs
  overlay: mount/unmount base fs before/after running tests
  overlay: use OVL_BASE_SCRATCH_MNT instead of SCRATCH_DEV

 README                |  16 ++--
 README.overlay        |  46 +++++++++++
 check                 |  18 ++---
 common/config         | 160 ++++++++++++++++++++++++++++++++++-----
 common/filter         |  33 +++++++-
 common/rc             | 206 ++++++++++++++++++++++++++++++++++++--------------
 tests/generic/064     |  10 ++-
 tests/overlay/001     |   7 +-
 tests/overlay/002     |   2 +-
 tests/overlay/003     |   5 +-
 tests/overlay/004     |   7 +-
 tests/overlay/005     |  30 ++++----
 tests/overlay/006     |  10 +--
 tests/overlay/008     |   8 +-
 tests/overlay/009     |   2 +-
 tests/overlay/010     |  10 +--
 tests/overlay/011     |   6 +-
 tests/overlay/012     |   4 +-
 tests/overlay/013     |   4 +-
 tests/overlay/014     |  19 ++---
 tests/overlay/015     |   2 +-
 tests/overlay/016     |   2 +-
 tests/overlay/017     |   2 +-
 tests/overlay/018     |   2 +-
 tests/overlay/019     |   2 +-
 tests/overlay/020     |   2 +-
 tests/overlay/021     |   6 +-
 tests/overlay/022     |   2 +-
 tests/overlay/023     |   4 +-
 tests/overlay/023.out |   2 +-
 tests/overlay/024     |   2 +-
 tests/overlay/027     |   2 +-
 tests/overlay/028     |   2 +-
 tests/overlay/029     |   4 +-
 34 files changed, 466 insertions(+), 173 deletions(-)
 create mode 100644 README.overlay

-- 
2.7.4


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH v5 01/10] fstests: sanity check that test partitions are not mounted elsewhere
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
@ 2017-02-28 12:18 ` Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 02/10] fstests: use _test_mount() consistently Amir Goldstein
                   ` (9 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

When $TEST_DEV is mounted at a different location then $TEST_DIR,
_require_test() aborts the test with an error:
 TEST_DEV=/dev/sda5 is mounted but not on TEST_DIR=/mnt/test

There are several problems with current sanity check:
1. the output of the error is mixed into out.bad and hard to see
2. the test partition is unmounted at the end of the test regardless
   of the fact that it not pass the sanity that we have exclusivity
3. scratch partition has a similar sanity check in _require_scratch(),
   but we may not get to it, because $SCRATCH_DEV is unmounted prior
   to running the tests (which could unmount another mount point).

To solve all these problems, introduce a helper _check_mounted_on().
It checks if a device is mounted on a given mount point and optionally
checks the mounted fs type.

The sanity checks in _require_scratch() and _require_test() are
converted to use the helper and gain the check for correct fs type.

The helper is used in init_rc() to sanity check both test and scratch
partitions, before tests are run and before $SCRATCH_DEV is unmounted.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 common/rc | 90 +++++++++++++++++++++++++++++++++++++++------------------------
 1 file changed, 56 insertions(+), 34 deletions(-)

diff --git a/common/rc b/common/rc
index 8f23334..dab3cb2 100644
--- a/common/rc
+++ b/common/rc
@@ -1354,6 +1354,43 @@ _supported_os()
     _notrun "not suitable for this OS: $HOSTOS"
 }
 
+# check if a FS on a device is mounted
+# if so, verify that it is mounted on mount point
+# if fstype is given as argument, verify that it is also
+# mounted with correct fs type
+#
+_check_mounted_on()
+{
+	local devname=$1
+	local dev=$2
+	local mntname=$3
+	local mnt=$4
+	local type=$5
+
+	# Note that we use -F here so grep doesn't try to interpret an NFS over
+	# IPv6 server as a regular expression
+	local mount_rec=`_mount | grep -F "$dev"`
+	[ -n "$mount_rec" ] || return 1 # 1 = not mounted
+
+	# if it's mounted, make sure its on $mnt
+	if ! (echo $mount_rec | grep -q "$mnt")
+	then
+		echo "$devname=$dev is mounted but not on $mntname=$mnt - aborting"
+		echo "Already mounted result:"
+		echo $mount_rec
+		return 2 # 2 = mounted on wrong mnt
+	fi
+
+	if [ -n "$type" -a "`_fs_type $dev`" != "$type" ]
+	then
+		echo "$devname=$dev is mounted but not a type $type filesystem"
+		# raw $DF_PROG cannot handle NFS/CIFS/overlay correctly
+		_df_device $dev
+		return 3 # 3 = mounted as wrong type
+	fi
+	return 0 # 0 = mounted as expected
+}
+
 # this test needs a scratch partition - check we're ok & unmount it
 # No post-test check of the device is required. e.g. the test intentionally
 # finishes the test with the filesystem in a corrupt state
@@ -1408,21 +1445,12 @@ _require_scratch_nocheck()
 		 ;;
     esac
 
-    # mounted?
-    # Note that we use -F here so grep doesn't try to interpret an NFS over
-    # IPv6 server as a regular expression.
-    mount_rec=`_mount | grep -F $SCRATCH_DEV`
-    if [ "$mount_rec" ]
+    _check_mounted_on SCRATCH_DEV $SCRATCH_DEV SCRATCH_MNT $SCRATCH_MNT
+    local err=$?
+    [ $err -le 1 ] || exit 1
+    if [ $err -eq 0 ]
     then
-        # if it's mounted, make sure its on $SCRATCH_MNT
-        if ! echo $mount_rec | grep -q $SCRATCH_MNT
-        then
-            echo "\$SCRATCH_DEV=$SCRATCH_DEV is mounted but not on \$SCRATCH_MNT=$SCRATCH_MNT - aborting"
-            echo "Already mounted result:"
-            echo $mount_rec
-            exit 1
-        fi
-        # and then unmount it
+        # if it's mounted, unmount it
         if ! _scratch_unmount
         then
             echo "failed to unmount $SCRATCH_DEV"
@@ -1493,21 +1521,11 @@ _require_test()
 		 ;;
     esac
 
-    # mounted?
-    # Note that we use -F here so grep doesn't try to interpret an NFS over
-    # IPv6 server as a regular expression.
-    mount_rec=`_mount | grep -F $TEST_DEV`
-    if [ "$mount_rec" ]
+    _check_mounted_on TEST_DEV $TEST_DEV TEST_DIR $TEST_DIR
+    local err=$?
+    [ $err -le 1 ] || exit 1
+    if [ $err -ne 0 ]
     then
-        # if it's mounted, make sure its on $TEST_DIR
-        if ! echo $mount_rec | grep -q $TEST_DIR
-        then
-            echo "\$TEST_DEV=$TEST_DEV is mounted but not on \$TEST_DIR=$TEST_DIR - aborting"
-            echo "Already mounted result:"
-            echo $mount_rec
-            exit 1
-        fi
-    else
 	out=`_mount_or_remount_rw "$MOUNT_OPTIONS" $TEST_DEV $TEST_DIR`
 	if [ $? -ne 1 ]; then
 		echo $out
@@ -3132,13 +3150,17 @@ init_rc()
 		fi
 	fi
 
-	if [ "`_fs_type $TEST_DEV`" != "$FSTYP" ]
-	then
-		echo "common/rc: Error: \$TEST_DEV ($TEST_DEV) is not a MOUNTED $FSTYP filesystem"
-		# raw $DF_PROG cannot handle NFS/CIFS/overlay correctly
-		_df_device $TEST_DEV
-		exit 1
+	# Sanity check that TEST partition is not mounted at another mount point
+	# or as another fs type
+	_check_mounted_on TEST_DEV $TEST_DEV TEST_DIR $TEST_DIR $FSTYP || exit 1
+	if [ -n "$SCRATCH_DEV" ]; then
+		# Sanity check that SCRATCH partition is not mounted at another
+		# mount point, because it is about to be unmounted and formatted.
+		# Another fs type for scratch is fine (bye bye old fs type).
+		_check_mounted_on SCRATCH_DEV $SCRATCH_DEV SCRATCH_MNT $SCRATCH_MNT
+		[ $? -le 1 ] || exit 1
 	fi
+
 	# Figure out if we need to add -F ("foreign", deprecated) option to xfs_io
 	$XFS_IO_PROG -c stat $TEST_DIR 2>&1 | grep -q "is not on an XFS filesystem" && \
 		export XFS_IO_PROG="$XFS_IO_PROG -F"
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v5 02/10] fstests: use _test_mount() consistently
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 01/10] fstests: sanity check that test partitions are not mounted elsewhere Amir Goldstein
@ 2017-02-28 12:18 ` Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 03/10] fstests: canonicalize mount points on every config section Amir Goldstein
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

On start of every test run and on every test, in init_rc() helper,
the test partition is verified to be mounted, or is mounted by
the helper _test_mount().

_test_mount() uses mount options $TEST_FS_MOUNT_OPTS and not
$MOUNT_OPTIONS like _scratch_mount() does.

_test_cycle_mount(), which is called by some tests uses the
_test_mount() helper as well.

Contrary to those cases, in _require_test() helper, if test
partition is not mounted, the helper _mount_or_remount_rw()
is called to mount the test partition with $MOUNT_OPTIONS.
Although this should never happen, because of the test in
init_rc(), this case is inconsistent with the rest of the code,
so it has been changed to use _test_mount() as it should.

When running tests with a multi section configuration, and
either FSTYP or MOUNT_OPTIONS change between sections, the
helper _test_unmount() is called to unmount the old test mount
and then _mount_or_remount_rw() is called to mount it again
with new FSTYP and/or MOUNT_OPTIONS.
This is again inconsistent with the rest of the code, so
was changed to use _test_mount() and instead of checking
if MOUNT_OPTIONS have changed between sections, we check if
TEST_FS_MOUNT_OPTS were changed between sections.
Otherwise, we can leave the test partition mounted.

This change is needed to support overlay base fs mount
and for multi section config files which include overlay FSTYP.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 check     | 16 ++++++++--------
 common/rc |  6 +++---
 2 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/check b/check
index 5d7f75c..fbd12e9 100755
--- a/check
+++ b/check
@@ -474,7 +474,7 @@ fi

 for section in $HOST_OPTIONS_SECTIONS; do
 	OLD_FSTYP=$FSTYP
-	OLD_MOUNT_OPTIONS=$MOUNT_OPTIONS
+	OLD_TEST_FS_MOUNT_OPTS=$TEST_FS_MOUNT_OPTS
 	get_next_config $section

 	# Do we need to run only some sections ?
@@ -527,18 +527,18 @@ for section in $HOST_OPTIONS_SECTIONS; do
 			status=1
 			exit
 		fi
-		out=`_mount_or_remount_rw "$MOUNT_OPTIONS" $TEST_DEV $TEST_DIR`
-		if [ $? -ne 1 ]; then
-			echo $out
+		if ! _test_mount
+		then
+			echo "check: failed to mount $TEST_DEV on $TEST_DIR"
 			status=1
 			exit
 		fi
 		_prepare_test_list
-	elif [ "$OLD_MOUNT_OPTIONS" != "$MOUNT_OPTIONS" ]; then
+	elif [ "$OLD_TEST_FS_MOUNT_OPTS" != "$TEST_FS_MOUNT_OPTS" ]; then
 		_test_unmount 2> /dev/null
-		out=`_mount_or_remount_rw "$MOUNT_OPTIONS" $TEST_DEV $TEST_DIR`
-		if [ $? -ne 1 ]; then
-			echo $out
+		if ! _test_mount
+		then
+			echo "check: failed to mount $TEST_DEV on $TEST_DIR"
 			status=1
 			exit
 		fi
diff --git a/common/rc b/common/rc
index dab3cb2..4fa211a 100644
--- a/common/rc
+++ b/common/rc
@@ -1526,9 +1526,9 @@ _require_test()
     [ $err -le 1 ] || exit 1
     if [ $err -ne 0 ]
     then
-	out=`_mount_or_remount_rw "$MOUNT_OPTIONS" $TEST_DEV $TEST_DIR`
-	if [ $? -ne 1 ]; then
-		echo $out
+	if ! _test_mount
+	then
+		echo "!!! failed to mount $TEST_DEV on $TEST_DIR"
 		exit 1
 	fi
     fi
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v5 03/10] fstests: canonicalize mount points on every config section
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 01/10] fstests: sanity check that test partitions are not mounted elsewhere Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 02/10] fstests: use _test_mount() consistently Amir Goldstein
@ 2017-02-28 12:18 ` Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 04/10] fstests: fix test and scratch filters for overlapping DEV/MNT paths Amir Goldstein
                   ` (7 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

Create helper _canonicalize_mountpoint() to check and canonicalize
a mount point path.
Use helper to canonicalize TEST_DIR and SCRATCH_MNT after parse
of every config section.

This is needed for overlay base fs mount.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 common/config | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git a/common/config b/common/config
index 3ef401b..c402576 100644
--- a/common/config
+++ b/common/config
@@ -455,6 +455,20 @@ _check_device()
 	fi
 }
 
+# check and return a canonical mount point path
+_canonicalize_mountpoint()
+{
+	local name=$1
+	local dir=$2
+
+	if [ ! -d "$dir" ]; then
+		_fatal "common/config: $name ($dir) is not a directory"
+	fi
+
+	# this follows symlinks and removes all trailing "/"s
+	readlink -e "$dir"
+}
+
 # Parse config section options. This function will parse all the configuration
 # within a single section which name is passed as an argument. For section
 # name format see comments in get_config_sections().
@@ -543,10 +557,7 @@ get_next_config() {
 	fi
 
 	_check_device TEST_DEV required $TEST_DEV
-	if [ ! -d "$TEST_DIR" ]; then
-		echo "common/config: Error: \$TEST_DIR ($TEST_DIR) is not a directory"
-		exit 1
-	fi
+	export TEST_DIR=`_canonicalize_mountpoint TEST_DIR $TEST_DIR`
 
 	# a btrfs tester will set only SCRATCH_DEV_POOL, we will put first of its dev
 	# to SCRATCH_DEV and rest to SCRATCH_DEV_POOL to maintain the backward compatibility
@@ -561,10 +572,7 @@ get_next_config() {
 	fi
 
 	_check_device SCRATCH_DEV optional $SCRATCH_DEV
-	if [ ! -z "$SCRATCH_MNT" -a ! -d "$SCRATCH_MNT" ]; then
-		echo "common/config: Error: \$SCRATCH_MNT ($SCRATCH_MNT) is not a directory"
-		exit 1
-	fi
+	export SCRATCH_MNT=`_canonicalize_mountpoint SCRATCH_MNT $SCRATCH_MNT`
 
 	if [ -n "$USE_EXTERNAL" ]; then
 		_check_device TEST_RTDEV optional $TEST_RTDEV
@@ -591,12 +599,12 @@ if [ -z "$CONFIG_INCLUDED" ]; then
 	[ -z "$TEST_FS_MOUNT_OPTS" ] && _test_mount_opts
 	[ -z "$MKFS_OPTIONS" ] && _mkfs_opts
 	[ -z "$FSCK_OPTIONS" ] && _fsck_opts
+else
+	# canonicalize the mount points
+	# this follows symlinks and removes all trailing "/"s
+	export TEST_DIR=`_canonicalize_mountpoint TEST_DIR $TEST_DIR`
+	export SCRATCH_MNT=`_canonicalize_mountpoint SCRATCH_MNT $SCRATCH_MNT`
 fi
 
-# canonicalize the mount points
-# this follows symlinks and removes all trailing "/"s
-export TEST_DIR=`readlink -e "$TEST_DIR"`
-export SCRATCH_MNT=`readlink -e "$SCRATCH_MNT"`
-
 # make sure this script returns success
 /bin/true
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v5 04/10] fstests: fix test and scratch filters for overlapping DEV/MNT paths
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
                   ` (2 preceding siblings ...)
  2017-02-28 12:18 ` [PATCH v5 03/10] fstests: canonicalize mount points on every config section Amir Goldstein
@ 2017-02-28 12:18 ` Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 05/10] fstests: allow overlay SCRATCH_DEV to be a base fs mount point Amir Goldstein
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

When configuring overlay base fs, TEST_DEV/DIR and SCRATCH_DEV/MNT
are derived from the base fs mount points, where *_DEV are the
path of the base fs mount point and TEST_DIR/SCRATCH_MNT are
a directory under the base fs mount point.

This means that the overlay DEV paths are prefixes of the overlay
mount points.
Fix the test and sctach filters to check if TEST_DEV/SCRATCH_DEV is
a substring of TEST_DIR/SCRATCH_MNT and try and match the longer
string first.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 common/filter | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)

diff --git a/common/filter b/common/filter
index 4328159..010f080 100644
--- a/common/filter
+++ b/common/filter
@@ -280,14 +280,32 @@ _filter_xfs_io_pages_modified()
 
 _filter_test_dir()
 {
-	sed -e "s,$TEST_DEV,TEST_DEV,g" -e "s,$TEST_DIR,TEST_DIR,g"
+	if ( echo $TEST_DIR | grep -q $TEST_DEV ); then
+		# TEST_DEV is substr of TEST_DIR (e.g. /mnt and /mnt/ovl-mnt)
+		# substitute TEST_DIR first
+		sed -e "s,$TEST_DIR,TEST_DIR,g" \
+		    -e "s,$TEST_DEV,TEST_DEV,g"
+	else
+		# TEST_DIR maybe a substr of TEST_DIR (e.g. /vdc and /dev/vdc)
+		# substitute TEST_DEV first
+		sed -e "s,$TEST_DEV,TEST_DEV,g" \
+		    -e "s,$TEST_DIR,TEST_DIR,g"
+	fi
 }
 
 _filter_scratch()
 {
-	sed -e "s,$SCRATCH_DEV,SCRATCH_DEV,g" \
-	    -e "s,$SCRATCH_MNT,SCRATCH_MNT,g" \
-	    -e "/.use_space/d"
+	if ( echo $SCRATCH_MNT | grep -q $SCRATCH_DEV ); then
+		# SCRATCH_DEV is substr of SCRATCH_MNT
+		sed -e "s,$SCRATCH_MNT,SCRATCH_MNT,g" \
+		    -e "s,$SCRATCH_DEV,SCRATCH_DEV,g" \
+		    -e "/.use_space/d"
+	else
+		# SCRATCH_MNT maybe a substr of SCRATCH_DEV
+		sed -e "s,$SCRATCH_DEV,SCRATCH_DEV,g" \
+		    -e "s,$SCRATCH_MNT,SCRATCH_MNT,g" \
+		    -e "/.use_space/d"
+	fi
 }
 
 # Turn any device in the scratch pool into SCRATCH_DEV
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v5 05/10] fstests: allow overlay SCRATCH_DEV to be a base fs mount point
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
                   ` (3 preceding siblings ...)
  2017-02-28 12:18 ` [PATCH v5 04/10] fstests: fix test and scratch filters for overlapping DEV/MNT paths Amir Goldstein
@ 2017-02-28 12:18 ` Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 06/10] generic/064: access SCRATCH_MNT after _scratch_mount Amir Goldstein
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

When configure SCRATCH_DEV to a mount point (and not to a directory therein)
then user will get a false positive error in scratch tests:

 $SCRATCH_DEV=/mnt/base/scratch is mounted but not on $SCRATCH_MNT=/mnt/scratch
 Already mounted result:
 /dev/sda6 on /mnt/base/scratch type xfs (rw,relatime,attr2,inode64,noquota)

This is due to the wrong `grep -F $SCRATCH_DEV` which matches the mount
point instead of the device in that mount.
Fix _check_mounted_on() to grep the pattern "$dev on " and "$dev on $mnt"
instead of just grepping for "$dev" and "$mnt" without the " on " anchor.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 common/rc | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/common/rc b/common/rc
index 4fa211a..fb88ffa 100644
--- a/common/rc
+++ b/common/rc
@@ -1368,12 +1368,14 @@ _check_mounted_on()
 	local type=$5
 
 	# Note that we use -F here so grep doesn't try to interpret an NFS over
-	# IPv6 server as a regular expression
-	local mount_rec=`_mount | grep -F "$dev"`
+	# IPv6 server as a regular expression.  Because of that, we cannot use
+	# ^$dev so we use "$dev on " to avoid matching $dev to mount point field
+	# for overlay case, where $dev is a directory.
+	local mount_rec=`_mount | grep -F "$dev on "`
 	[ -n "$mount_rec" ] || return 1 # 1 = not mounted
 
 	# if it's mounted, make sure its on $mnt
-	if ! (echo $mount_rec | grep -q "$mnt")
+	if ! (echo $mount_rec | grep -q "$dev on $mnt")
 	then
 		echo "$devname=$dev is mounted but not on $mntname=$mnt - aborting"
 		echo "Already mounted result:"
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v5 06/10] generic/064: access SCRATCH_MNT after _scratch_mount
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
                   ` (4 preceding siblings ...)
  2017-02-28 12:18 ` [PATCH v5 05/10] fstests: allow overlay SCRATCH_DEV to be a base fs mount point Amir Goldstein
@ 2017-02-28 12:18 ` Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 07/10] overlay: rename OVERLAY_LOWER/UPPER/WORK_DIR Amir Goldstein
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

Test is calling _get_block_size $SCRATCH_MNT before
_scratch_mount. This results in block size of the
wrong fs and a failure with overlay base fs setup.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/generic/064 | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/tests/generic/064 b/tests/generic/064
index 8cd31b3..156485e 100755
--- a/tests/generic/064
+++ b/tests/generic/064
@@ -46,14 +46,16 @@ _require_scratch
 _require_xfs_io_command "fiemap"
 _require_xfs_io_command "finsert"
 _require_xfs_io_command "fcollapse"
-src=$SCRATCH_MNT/testfile
-dest=$SCRATCH_MNT/testfile.dest
-BLOCKS=100
-BSIZE=`_get_block_size $SCRATCH_MNT`
+
 rm -f $seqres.full
 
 _scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed"
 _scratch_mount || _fail "mount failed"
+
+src=$SCRATCH_MNT/testfile
+dest=$SCRATCH_MNT/testfile.dest
+BLOCKS=100
+BSIZE=`_get_block_size $SCRATCH_MNT`
 length=$(($BLOCKS * $BSIZE))
 
 # Write file
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v5 07/10] overlay: rename OVERLAY_LOWER/UPPER/WORK_DIR
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
                   ` (5 preceding siblings ...)
  2017-02-28 12:18 ` [PATCH v5 06/10] generic/064: access SCRATCH_MNT after _scratch_mount Amir Goldstein
@ 2017-02-28 12:18 ` Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 08/10] overlay: configure TEST/SCRATCH vars to base fs Amir Goldstein
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

As preparation step for configuring test/scratch base fs
for overlay tests:

- Rename OVERLAY_LOWER/UPPER/WORK_DIR => OVL_LOWER/UPPER/WORK
  because we want to use OVL_ prefix for all base fs vars

- Prepend "ovl-" prefix to lower/upper/work path values to
  isolate the overlay test dirs when running on a base fs
  that is also used to run non overlay tests

- Make those vars values non-configurable, because lower/upper/work
  directory names are an internal test detail which should
  not concern the user and because we wish to simplify
  and document the overlay tests setup

- Add helper _filter_ovl_dirs and use it to filter output
  of test overlay/023 which matches the constant string "work"

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 common/config         |  8 +++++---
 common/filter         |  7 +++++++
 common/rc             | 19 +++++++++++++------
 tests/overlay/001     |  2 +-
 tests/overlay/002     |  2 +-
 tests/overlay/003     |  2 +-
 tests/overlay/004     |  4 ++--
 tests/overlay/006     | 10 +++++-----
 tests/overlay/008     |  8 ++++----
 tests/overlay/009     |  2 +-
 tests/overlay/010     |  8 ++++----
 tests/overlay/011     |  4 ++--
 tests/overlay/012     |  4 ++--
 tests/overlay/013     |  4 ++--
 tests/overlay/015     |  2 +-
 tests/overlay/016     |  2 +-
 tests/overlay/017     |  2 +-
 tests/overlay/018     |  2 +-
 tests/overlay/019     |  2 +-
 tests/overlay/020     |  2 +-
 tests/overlay/021     |  2 +-
 tests/overlay/022     |  2 +-
 tests/overlay/023     |  4 ++--
 tests/overlay/023.out |  2 +-
 tests/overlay/024     |  2 +-
 tests/overlay/027     |  2 +-
 tests/overlay/028     |  2 +-
 tests/overlay/029     |  4 ++--
 28 files changed, 66 insertions(+), 50 deletions(-)

diff --git a/common/config b/common/config
index c402576..d09cac1 100644
--- a/common/config
+++ b/common/config
@@ -77,9 +77,11 @@ export XFS_MKFS_OPTIONS=${XFS_MKFS_OPTIONS:=-bsize=4096}
 export TIME_FACTOR=${TIME_FACTOR:=1}
 export LOAD_FACTOR=${LOAD_FACTOR:=1}
 export DEBUGFS_MNT=${DEBUGFS_MNT:="/sys/kernel/debug"}
-export OVERLAY_UPPER_DIR=${OVERLAY_UPPER_DIR:="upper"}
-export OVERLAY_LOWER_DIR=${OVERLAY_LOWER_DIR:="lower"}
-export OVERLAY_WORK_DIR=${OVERLAY_WORK_DIR:="work"}
+
+# some constants for overlayfs setup
+export OVL_UPPER="ovl-upper"
+export OVL_LOWER="ovl-lower"
+export OVL_WORK="ovl-work"
 
 export PWD=`pwd`
 #export DEBUG=${DEBUG:=...} # arbitrary CFLAGS really.
diff --git a/common/filter b/common/filter
index 010f080..1ceb346 100644
--- a/common/filter
+++ b/common/filter
@@ -424,5 +424,12 @@ _filter_lostfound()
 	sed -e '/^lost+found$/d'
 }
 
+_filter_ovl_dirs()
+{
+	sed -e "s,$OVL_LOWER,OVL_LOWER,g" \
+	    -e "s,$OVL_UPPER,OVL_UPPER,g" \
+	    -e "s,$OVL_WORK,OVL_WORK,g"
+}
+
 # make sure this script returns success
 /bin/true
diff --git a/common/rc b/common/rc
index fb88ffa..cca046b 100644
--- a/common/rc
+++ b/common/rc
@@ -277,7 +277,7 @@ _common_dev_mount_options()
 
 _overlay_basic_mount_options()
 {
-	echo "-o lowerdir=$1/$OVERLAY_LOWER_DIR,upperdir=$1/$OVERLAY_UPPER_DIR,workdir=$1/$OVERLAY_WORK_DIR"
+	echo "-o lowerdir=$1/$OVL_LOWER,upperdir=$1/$OVL_UPPER,workdir=$1/$OVL_WORK"
 }
 
 _overlay_mount_options()
@@ -336,6 +336,15 @@ _overlay_mount_dirs()
 		    -o workdir=$workdir $*
 }
 
+_overlay_mkdirs()
+{
+	local dir=$1
+
+	mkdir -p $dir/$OVL_UPPER
+	mkdir -p $dir/$OVL_LOWER
+	mkdir -p $dir/$OVL_WORK
+}
+
 # Given a dir, set up 3 subdirectories and mount on the given mnt.
 # The dir is used as the mount device so it can be seen from df or mount
 _overlay_mount()
@@ -346,12 +355,10 @@ _overlay_mount()
 
 	_supports_filetype $dir || _notrun "upper fs needs to support d_type"
 
-	mkdir -p $dir/$OVERLAY_UPPER_DIR
-	mkdir -p $dir/$OVERLAY_LOWER_DIR
-	mkdir -p $dir/$OVERLAY_WORK_DIR
+	_overlay_mkdirs $dir
 
-	_overlay_mount_dirs $dir/$OVERLAY_LOWER_DIR $dir/$OVERLAY_UPPER_DIR \
-			    $dir/$OVERLAY_WORK_DIR $OVERLAY_MOUNT_OPTIONS \
+	_overlay_mount_dirs $dir/$OVL_LOWER $dir/$OVL_UPPER \
+			    $dir/$OVL_WORK $OVERLAY_MOUNT_OPTIONS \
 			    $SELINUX_MOUNT_OPTIONS $* $dir $mnt
 }
 
diff --git a/tests/overlay/001 b/tests/overlay/001
index 7be9ae5..099ddd5 100755
--- a/tests/overlay/001
+++ b/tests/overlay/001
@@ -60,7 +60,7 @@ _scratch_mkfs
 _require_fs_space $SCRATCH_DEV $((4*1024*1024*2 + 8))
 
 # Create test files with different sizes in lower dir
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/zero_size
 $XFS_IO_PROG -fc "pwrite 0 4k" $lowerdir/less_than_4g >>$seqres.full
diff --git a/tests/overlay/002 b/tests/overlay/002
index ec7874e..eaf9a91 100755
--- a/tests/overlay/002
+++ b/tests/overlay/002
@@ -59,7 +59,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create our test file.
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/foobar
 
diff --git a/tests/overlay/003 b/tests/overlay/003
index fbc242a..5e610c8 100755
--- a/tests/overlay/003
+++ b/tests/overlay/003
@@ -60,7 +60,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create test files dirs in lower dir
-lowerdir=${SCRATCH_DEV}/${OVERLAY_LOWER_DIR}
+lowerdir=${SCRATCH_DEV}/${OVL_LOWER}
 mkdir -p $lowerdir
 
 touch ${lowerdir}/test_file
diff --git a/tests/overlay/004 b/tests/overlay/004
index 4075094..8da5170 100755
--- a/tests/overlay/004
+++ b/tests/overlay/004
@@ -53,8 +53,8 @@ _require_user
 _scratch_mkfs
 
 # Create test file in lower dir
-lowerdir=${SCRATCH_DEV}/${OVERLAY_LOWER_DIR}
-upperdir=${SCRATCH_DEV}/${OVERLAY_UPPER_DIR}
+lowerdir=${SCRATCH_DEV}/${OVL_LOWER}
+upperdir=${SCRATCH_DEV}/${OVL_UPPER}
 mkdir -p $lowerdir
 touch ${lowerdir}/attr_file1
 chmod 000 ${lowerdir}/attr_file1
diff --git a/tests/overlay/006 b/tests/overlay/006
index 31f11ef..55fc2dd 100755
--- a/tests/overlay/006
+++ b/tests/overlay/006
@@ -57,15 +57,15 @@ echo "Silence is golden"
 _scratch_mkfs
 
 # Create test file/dir before mount
-mkdir $SCRATCH_DEV/$OVERLAY_LOWER_DIR
-mkdir $SCRATCH_DEV/$OVERLAY_UPPER_DIR
-touch $SCRATCH_DEV/$OVERLAY_LOWER_DIR/lowertestfile
-mkdir $SCRATCH_DEV/$OVERLAY_UPPER_DIR/uppertestdir
+mkdir -p $SCRATCH_DEV/$OVL_LOWER
+mkdir -p $SCRATCH_DEV/$OVL_UPPER
+touch $SCRATCH_DEV/$OVL_LOWER/lowertestfile
+mkdir $SCRATCH_DEV/$OVL_UPPER/uppertestdir
 
 _scratch_mount
 
 # rename lowertestfile to uppertestdir, this triggers copyup and creates
-# whiteout in $OVERLAY_UPPER_DIR
+# whiteout in $OVL_UPPER
 mv $SCRATCH_MNT/lowertestfile $SCRATCH_MNT/uppertestdir
 # the lowertestfile can be removed
 rm $SCRATCH_MNT/uppertestdir/lowertestfile
diff --git a/tests/overlay/008 b/tests/overlay/008
index cb8667c..b81cff8 100755
--- a/tests/overlay/008
+++ b/tests/overlay/008
@@ -56,14 +56,14 @@ _require_user
 _scratch_mkfs
 
 # Create test file on lower dir, and chown to fsgqa user
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
-mkdir $lowerdir
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
+mkdir -p $lowerdir
 touch $lowerdir/testfile
 chown fsgqa:fsgqa $lowerdir/testfile
 
 # chown upperdir to fsgqa user, so new file/dir can be created by the user
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
-mkdir $upperdir
+upperdir=$SCRATCH_DEV/$OVL_UPPER
+mkdir -p $upperdir
 chown fsgqa:fsgqa $upperdir
 
 _scratch_mount
diff --git a/tests/overlay/009 b/tests/overlay/009
index de94ca4..bdac6b7 100755
--- a/tests/overlay/009
+++ b/tests/overlay/009
@@ -54,7 +54,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create test file in lowerdir
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/testfile
 
diff --git a/tests/overlay/010 b/tests/overlay/010
index a302d74..5882db9 100755
--- a/tests/overlay/010
+++ b/tests/overlay/010
@@ -55,10 +55,10 @@ _scratch_mkfs
 
 # Need two lower dirs in this test, and we mount overlay ourselves,
 # create upper and workdir as well
-lowerdir1=$SCRATCH_DEV/$OVERLAY_LOWER_DIR.1
-lowerdir2=$SCRATCH_DEV/$OVERLAY_LOWER_DIR.2
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
-workdir=$SCRATCH_DEV/$OVERLAY_WORK_DIR
+lowerdir1=$SCRATCH_DEV/$OVL_LOWER.1
+lowerdir2=$SCRATCH_DEV/$OVL_LOWER.2
+upperdir=$SCRATCH_DEV/$OVL_UPPER
+workdir=$SCRATCH_DEV/$OVL_WORK
 mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir
 
 # One lowerdir contains test dir and test files, the other contains whiteout
diff --git a/tests/overlay/011 b/tests/overlay/011
index 4b697b8..1d6c44a 100755
--- a/tests/overlay/011
+++ b/tests/overlay/011
@@ -58,8 +58,8 @@ _require_attrs
 _scratch_mkfs
 
 # Create test dir on upper and make it opaque by setting proper xattr
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
+upperdir=$SCRATCH_DEV/$OVL_UPPER
 mkdir -p $lowerdir/testdir
 mkdir -p $upperdir/testdir
 $SETFATTR_PROG -n "trusted.overlay.opaque" -v "y" $upperdir/testdir
diff --git a/tests/overlay/012 b/tests/overlay/012
index cfe16f2..1c4f7c6 100755
--- a/tests/overlay/012
+++ b/tests/overlay/012
@@ -55,8 +55,8 @@ _require_scratch
 # remove all files from previous runs
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
+upperdir=$SCRATCH_DEV/$OVL_UPPER
 mkdir -p $lowerdir/test
 
 _scratch_mount
diff --git a/tests/overlay/013 b/tests/overlay/013
index e99e10a..536e981 100755
--- a/tests/overlay/013
+++ b/tests/overlay/013
@@ -54,8 +54,8 @@ _require_test_program "t_truncate_self"
 _scratch_mkfs
 
 # copy test program to lower and upper dir
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
+upperdir=$SCRATCH_DEV/$OVL_UPPER
 mkdir -p $lowerdir
 mkdir -p $upperdir
 cp $here/src/t_truncate_self $lowerdir/test_lower
diff --git a/tests/overlay/015 b/tests/overlay/015
index c39caed..0e09e0c 100755
--- a/tests/overlay/015
+++ b/tests/overlay/015
@@ -57,7 +57,7 @@ _scratch_mkfs
 umask 022
 
 # Create test dir in lower dir and set sgid bit
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir/dir
 chown $qa_user:$qa_group $lowerdir/dir
 chmod 2775 $lowerdir/dir
diff --git a/tests/overlay/016 b/tests/overlay/016
index cffcde7..c678ea4 100755
--- a/tests/overlay/016
+++ b/tests/overlay/016
@@ -57,7 +57,7 @@ rm -f $seqres.full
 _scratch_mkfs >>$seqres.full 2>&1
 
 # Create our test files.
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 echo "This is old news" > $lowerdir/foo
 echo "This is old news" > $lowerdir/bar
diff --git a/tests/overlay/017 b/tests/overlay/017
index 5330de2..6b12722 100755
--- a/tests/overlay/017
+++ b/tests/overlay/017
@@ -60,7 +60,7 @@ _scratch_mkfs >>$seqres.full 2>&1
 # Not dealing with hardlinks here, because there is more to test
 # then stable inode number.
 # Hardlinks will get a test of their own.
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 mkdir $lowerdir/dir
 touch $lowerdir/file
diff --git a/tests/overlay/018 b/tests/overlay/018
index df631fc..37bf11c 100755
--- a/tests/overlay/018
+++ b/tests/overlay/018
@@ -55,7 +55,7 @@ rm -f $seqres.full
 _scratch_mkfs >>$seqres.full 2>&1
 
 # Create 2 hardlinked files in lower
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 echo "patient zero" >> $lowerdir/foo
 ln $lowerdir/foo $lowerdir/bar
diff --git a/tests/overlay/019 b/tests/overlay/019
index 41ce63b..9177acb 100755
--- a/tests/overlay/019
+++ b/tests/overlay/019
@@ -51,7 +51,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 
 _scratch_mount
diff --git a/tests/overlay/020 b/tests/overlay/020
index 4afd40a..3cd0f05 100755
--- a/tests/overlay/020
+++ b/tests/overlay/020
@@ -55,7 +55,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir/dir
 
 _scratch_mount
diff --git a/tests/overlay/021 b/tests/overlay/021
index c79bb01..c6d68aa 100755
--- a/tests/overlay/021
+++ b/tests/overlay/021
@@ -60,7 +60,7 @@ _scratch_mkfs
 # conservative and reserve space for 16 data copy ups per directory.
 _require_fs_space $SCRATCH_DEV $((16*(16+32+64+128)*1024))
 
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 
 testdir=arena
diff --git a/tests/overlay/022 b/tests/overlay/022
index 817ee63..46034ea 100755
--- a/tests/overlay/022
+++ b/tests/overlay/022
@@ -59,7 +59,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
+upperdir=$SCRATCH_DEV/$OVL_UPPER
 mkdir -p $upperdir/upper
 mkdir -p $upperdir/work
 # mount overlay with dirs in upper
diff --git a/tests/overlay/023 b/tests/overlay/023
index ab49e18..c855d16 100755
--- a/tests/overlay/023
+++ b/tests/overlay/023
@@ -63,14 +63,14 @@ _require_scratch
 _scratch_mkfs
 
 # setting acls before mount
-wkdir=$SCRATCH_DEV/$OVERLAY_WORK_DIR
+wkdir=$SCRATCH_DEV/$OVL_WORK
 mkdir -p $wkdir
 setfacl -d -m o::rwx $wkdir
 
 _scratch_mount
 
 # getting acls, ACL set on workdir should be cleaned
-getfacl -p $wkdir/work 2>&1 | _filter_scratch
+getfacl -p $wkdir/work 2>&1 | _filter_scratch | _filter_ovl_dirs
 
 # success, all done
 status=0
diff --git a/tests/overlay/023.out b/tests/overlay/023.out
index a1a3c37..4a243c5 100644
--- a/tests/overlay/023.out
+++ b/tests/overlay/023.out
@@ -1,5 +1,5 @@
 QA output created by 023
-# file: SCRATCH_DEV/work/work
+# file: SCRATCH_DEV/OVL_WORK/work
 # owner: root
 # group: root
 user::---
diff --git a/tests/overlay/024 b/tests/overlay/024
index 2dbef12..25195f7 100755
--- a/tests/overlay/024
+++ b/tests/overlay/024
@@ -61,7 +61,7 @@ _require_scratch
 _scratch_mkfs
 
 # making workdir
-wkdir=$SCRATCH_DEV/$OVERLAY_WORK_DIR
+wkdir=$SCRATCH_DEV/$OVL_WORK
 mkdir -p $wkdir/work/foo
 
 _scratch_mount
diff --git a/tests/overlay/027 b/tests/overlay/027
index 4bb0329..1c77956 100755
--- a/tests/overlay/027
+++ b/tests/overlay/027
@@ -63,7 +63,7 @@ _require_chattr
 _scratch_mkfs
 
 # Preparing immutable file
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
+upperdir=$SCRATCH_DEV/$OVL_UPPER
 mkdir -p $upperdir
 touch $upperdir/foo
 $CHATTR_PROG +i $upperdir/foo
diff --git a/tests/overlay/028 b/tests/overlay/028
index 6045e09..ad610e2 100755
--- a/tests/overlay/028
+++ b/tests/overlay/028
@@ -61,7 +61,7 @@ _require_command "$FLOCK_PROG" flock
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/foo
 
diff --git a/tests/overlay/029 b/tests/overlay/029
index 902a84c..8dce6b3 100755
--- a/tests/overlay/029
+++ b/tests/overlay/029
@@ -65,8 +65,8 @@ _require_scratch
 _scratch_mkfs
 
 # Preparing files
-upperdir=$SCRATCH_DEV/$OVERLAY_UPPER_DIR
-lowerdir=$SCRATCH_DEV/$OVERLAY_LOWER_DIR
+upperdir=$SCRATCH_DEV/$OVL_UPPER
+lowerdir=$SCRATCH_DEV/$OVL_LOWER
 mkdir -p $upperdir/up
 echo foo > $upperdir/up/foo
 mkdir -p $lowerdir/low
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v5 08/10] overlay: configure TEST/SCRATCH vars to base fs
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
                   ` (6 preceding siblings ...)
  2017-02-28 12:18 ` [PATCH v5 07/10] overlay: rename OVERLAY_LOWER/UPPER/WORK_DIR Amir Goldstein
@ 2017-02-28 12:18 ` Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 09/10] overlay: mount/unmount base fs before/after running tests Amir Goldstein
                   ` (2 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

Instead of setting the vars TEST/SCRATCH_DEV to overlay base dirs,
allow setting them to block devices to configure the base fs partition,
where overlay dirs will be created.

For example, the following config file can be used to run tests on
xfs test/scratch partitions:

 TEST_DEV=/dev/sda5
 TEST_DIR=/mnt/test
 SCRATCH_DEV=/dev/sda6
 SCRATCH_MNT=/mnt/scratch
 FSTYP=xfs

Using the same config file, but executing './check -overlay' will
use the same partitions as base fs for overlayfs directories
and set TEST_DIR/SCRATCH_MNT values to overlay mount points, i.e.:
/mnt/test/ovl-mnt and /mnt/scratch/ovl-mnt.

The base fs should be pre-formatted and mounted when starting the test.
An upcoming change is going to support mount/umount of base fs.

The new vars OVL_BASE_SCRATCH_MNT/TEST_DIR are set to point at the
overlayfs base dirs in either legacy or new config method.
Tests should always use these vars and not the legacy SCRATCH/TEST_DEV
vars when referring to overlay base dir.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 README         |  16 ++++----
 README.overlay |  18 +++++++++
 check          |   2 +-
 common/config  | 124 +++++++++++++++++++++++++++++++++++++++++++++++++++++----
 common/rc      |  28 ++++++++-----
 5 files changed, 162 insertions(+), 26 deletions(-)
 create mode 100644 README.overlay

diff --git a/README b/README
index bb42985..842e835 100644
--- a/README
+++ b/README
@@ -117,16 +117,18 @@ Running tests:
     - By default the tests suite will run xfs tests:
     - ./check '*/001' '*/002' '*/003'
     - ./check '*/06?'
-    - You can explicitly specify NFS/CIFS/UDF, otherwise the filesystem type will
-      be autodetected from $TEST_DEV:
-      ./check -nfs [test(s)]
     - Groups of tests maybe ran by: ./check -g [group(s)]
       See the 'group' file for details on groups
-    - for udf tests: ./check -udf [test(s)]
-      Running all the udf tests: ./check -udf -g udf
-    - for running nfs tests: ./check -nfs [test(s)]
-    - for running cifs/smb3 tests: ./check -cifs [test(s)]
     - To randomize test order: ./check -r [test(s)]
+    - You can explicitly specify NFS/CIFS/UDF/OVERLAY, otherwise
+      the filesystem type will be autodetected from $TEST_DEV:
+        - for running nfs tests: ./check -nfs [test(s)]
+        - for running cifs/smb3 tests: ./check -cifs [test(s)]
+        - for udf tests: ./check -udf [test(s)]
+          Running all the udf tests: ./check -udf -g udf
+        - for overlay tests: ./check -overlay [test(s)]
+          The TEST and SCRATCH partitions should be pre-formatted
+          with another base fs, where the overlay dirs will be created
 
     
     The check script tests the return value of each script, and
diff --git a/README.overlay b/README.overlay
new file mode 100644
index 0000000..647f9ea
--- /dev/null
+++ b/README.overlay
@@ -0,0 +1,18 @@
+
+To run xfstest on overlayfs, configure the variables of TEST and SCRATCH
+partitions to be used as the "base fs" and run './check -overlay'.
+
+For example, the following config file can be used to run tests on
+xfs test/scratch partitions:
+
+ TEST_DEV=/dev/sda5
+ TEST_DIR=/mnt/test
+ SCRATCH_DEV=/dev/sda6
+ SCRATCH_MNT=/mnt/scratch
+ FSTYP=xfs
+
+Using the same config file, but executing './check -overlay' will
+use the same partitions as base fs for overlayfs directories
+and set TEST_DIR/SCRATCH_MNT values to overlay mount points, i.e.:
+/mnt/test/ovl-mnt and /mnt/scratch/ovl-mnt, for the context of
+individual tests.
diff --git a/check b/check
index fbd12e9..66a349a 100755
--- a/check
+++ b/check
@@ -258,7 +258,7 @@ while [ $# -gt 0 ]; do
 
 	-nfs)		FSTYP=nfs ;;
 	-cifs)		FSTYP=cifs ;;
-	-overlay)	FSTYP=overlay ;;
+	-overlay)	FSTYP=overlay; export OVERLAY=true ;;
 	-tmpfs)		FSTYP=tmpfs ;;
 
 	-g)	group=$2 ; shift ;
diff --git a/common/config b/common/config
index d09cac1..37b0189 100644
--- a/common/config
+++ b/common/config
@@ -82,6 +82,8 @@ export DEBUGFS_MNT=${DEBUGFS_MNT:="/sys/kernel/debug"}
 export OVL_UPPER="ovl-upper"
 export OVL_LOWER="ovl-lower"
 export OVL_WORK="ovl-work"
+# overlay mount point parent must be the base fs root
+export OVL_MNT="ovl-mnt"
 
 export PWD=`pwd`
 #export DEBUG=${DEBUG:=...} # arbitrary CFLAGS really.
@@ -443,16 +445,19 @@ _check_device()
 		if [ "$dev_needed" == "required" ]; then
 			_fatal "common/config: $name is required but not defined!"
 		fi
-		return
+		return 0
+	fi
+
+	if [ -b "$dev" ] || ( echo $dev | grep -qE ":|//" ); then
+		# block device or a network url
+		return 0
 	fi
 
-	echo $dev | grep -qE ":|//" > /dev/null 2>&1
-	network_dev=$?
 	if [ "$FSTYP" == "overlay" ]; then
 		if [ ! -d "$dev" ]; then
 			_fatal "common/config: $name ($dev) is not a directory for overlay"
 		fi
-	elif [ ! -b "$dev" -a "$network_dev" != "0" ]; then
+	else
 		_fatal "common/config: $name ($dev) is not a block device or a network filesystem"
 	fi
 }
@@ -463,12 +468,89 @@ _canonicalize_mountpoint()
 	local name=$1
 	local dir=$2
 
-	if [ ! -d "$dir" ]; then
+	if [ -d "$dir" ]; then
+		# this follows symlinks and removes all trailing "/"s
+		readlink -e "$dir"
+		return 0
+	fi
+
+	if [ "$FSTYP" != "overlay" ] || [[ "$name" == OVL_BASE_* ]]; then
 		_fatal "common/config: $name ($dir) is not a directory"
 	fi
 
-	# this follows symlinks and removes all trailing "/"s
-	readlink -e "$dir"
+	# base fs may not be mounted yet, so just check that parent dir
+	# exists (where base fs will be mounted) because we are going to
+	# mkdir the overlay mount point dir anyway
+	local base=`basename $dir`
+	local parent=`dirname $dir`
+	parent=`_canonicalize_mountpoint OVL_BASE_$name "$parent"`
+
+	# prepend the overlay mount point to canonical parent path
+	echo "$parent/$base"
+}
+
+# On check -overlay, for the non multi section config case, this
+# function is called on every test, before init_rc().
+# When SCRATCH/TEST_* vars are defined in config file, config file
+# is sourced on every test and this function overrides the vars
+# every time.
+# When SCRATCH/TEST_* vars are defined in evironment and not
+# in config file, this function is called after vars have already
+# been overriden in the previous test.
+# In that case, TEST_DEV is a directory and not a blockdev and
+# the function will return without overriding the SCRATCH/TEST_* vars.
+_overlay_config_override()
+{
+	# There are 2 options for configuring overlayfs tests:
+	#
+	# 1. (legacy) SCRATCH/TEST_DEV point to existing directories
+	#    on an already mounted fs.  In this case, the new
+	#    OVL_BASE_SCRATCH/TEST_* vars are set to use the legacy
+	#    vars values (even though they may not be mount points).
+	#
+	[ ! -d "$TEST_DEV" ] || export OVL_BASE_TEST_DIR="$TEST_DEV"
+	[ ! -d "$SCRATCH_DEV" ] || export OVL_BASE_SCRATCH_MNT="$SCRATCH_DEV"
+
+	# 2. SCRATCH/TEST_DEV point to the base fs partitions.  In this case,
+	#    the new OVL_BASE_SCRATCH/TEST_DEV/MNT vars are set to the values
+	#    of the configured base fs and SCRATCH/TEST_DEV vars are set to the
+	#    overlayfs base and mount dirs inside base fs mount.
+	[ -b "$TEST_DEV" ] || return 0
+
+	# Config file may specify base fs type, but we obay -overlay flag
+	export OVL_BASE_FSTYP="$FSTYP"
+	export FSTYP=overlay
+
+	# Store original base fs vars
+	export OVL_BASE_TEST_DEV="$TEST_DEV"
+	export OVL_BASE_TEST_DIR="$TEST_DIR"
+	export OVL_BASE_MOUNT_OPTIONS="$MOUNT_OPTIONS"
+
+	# Set TEST vars to overlay base and mount dirs inside base fs
+	export TEST_DEV="$OVL_BASE_TEST_DIR"
+	export TEST_DIR="$OVL_BASE_TEST_DIR/$OVL_MNT"
+	export MOUNT_OPTIONS="$OVERLAY_MOUNT_OPTIONS"
+
+	[ -b "$SCRATCH_DEV" ] || return 0
+
+	# Store original base fs vars
+	export OVL_BASE_SCRATCH_DEV="$SCRATCH_DEV"
+	export OVL_BASE_SCRATCH_MNT="$SCRATCH_MNT"
+
+	# Set SCRATCH vars to overlay base and mount dirs inside base fs
+	export SCRATCH_DEV="$OVL_BASE_SCRATCH_MNT"
+	export SCRATCH_MNT="$OVL_BASE_SCRATCH_MNT/$OVL_MNT"
+}
+
+_overlay_config_restore()
+{
+	export OVERLAY=true
+	[ -z "$OVL_BASE_FSTYP" ] || export FSTYP=$OVL_BASE_FSTYP
+	[ -z "$OVL_BASE_TEST_DEV" ] || export TEST_DEV=$OVL_BASE_TEST_DEV
+	[ -z "$OVL_BASE_TEST_DIR" ] || export TEST_DIR=$OVL_BASE_TEST_DIR
+	[ -z "$OVL_BASE_SCRATCH_DEV" ] || export SCRATCH_DEV=$OVL_BASE_SCRATCH_DEV
+	[ -z "$OVL_BASE_SCRATCH_MNT" ] || export SCRATCH_MNT=$OVL_BASE_SCRATCH_MNT
+	[ -z "$OVL_BASE_MOUNT_OPTIONS" ] || export MOUNT_OPTIONS=$OVL_BASE_MOUNT_OPTIONS
 }
 
 # Parse config section options. This function will parse all the configuration
@@ -499,6 +581,15 @@ get_next_config() {
 		return 0
 	fi
 
+	# We might have overriden FSTYP and TEST/SCRATCH vars with overlay values
+	# in the previous section, so restore them to original values stored in
+	# OVL_BASE_*.
+	# We need to do this *before* old FSTYP and MOUNT_OPTIONS are recorded
+	# and *before* SCRATCH_DEV and MOUNT_OPTIONS are unset
+	if [ "$FSTYP" == "overlay" ]; then
+		_overlay_config_restore
+	fi
+
 	local OLD_FSTYP=$FSTYP
 	local OLD_MOUNT_OPTIONS=$MOUNT_OPTIONS
 	local OLD_TEST_FS_MOUNT_OPTS=$TEST_FS_MOUNT_OPTS
@@ -582,6 +673,14 @@ get_next_config() {
 		_check_device SCRATCH_RTDEV optional $SCRATCH_RTDEV
 		_check_device SCRATCH_LOGDEV optional $SCRATCH_LOGDEV
 	fi
+
+	# Override FSTYP from config when running ./check -overlay
+	# and maybe override base fs TEST/SCRATCH_DEV with overlay base dirs.
+	# We need to do this *after* default mount options are set by base FSTYP
+	# and *after* SCRATCH_DEV is deduced from SCRATCH_DEV_POOL
+	if [ "$OVERLAY" == "true" -o "$FSTYP" == "overlay" ]; then
+		_overlay_config_override
+	fi
 }
 
 if [ -z "$CONFIG_INCLUDED" ]; then
@@ -602,10 +701,21 @@ if [ -z "$CONFIG_INCLUDED" ]; then
 	[ -z "$MKFS_OPTIONS" ] && _mkfs_opts
 	[ -z "$FSCK_OPTIONS" ] && _fsck_opts
 else
+	# We get here for the non multi section case, on every test that sources
+	# common/rc after re-sourcing the HOST_OPTIONS config file.
+	# Because of this re-sourcing, we need to re-canonicalize the configured
+	# mount points and re-override TEST/SCRATCH_DEV overlay vars.
+
 	# canonicalize the mount points
 	# this follows symlinks and removes all trailing "/"s
 	export TEST_DIR=`_canonicalize_mountpoint TEST_DIR $TEST_DIR`
 	export SCRATCH_MNT=`_canonicalize_mountpoint SCRATCH_MNT $SCRATCH_MNT`
+
+	# Override FSTYP from config when running ./check -overlay
+	# and maybe override base fs TEST/SCRATCH_DEV with overlay base dirs
+	if [ "$OVERLAY" == "true" -o "$FSTYP" == "overlay" ]; then
+		_overlay_config_override
+	fi
 fi
 
 # make sure this script returns success
diff --git a/common/rc b/common/rc
index cca046b..fa4ca5e 100644
--- a/common/rc
+++ b/common/rc
@@ -292,7 +292,7 @@ _scratch_mount_options()
 	_scratch_options mount
 
 	if [ "$FSTYP" == "overlay" ]; then
-		echo `_overlay_mount_options $SCRATCH_DEV`
+		echo `_overlay_mount_options $OVL_BASE_SCRATCH_MNT`
 		return 0
 	fi
 	echo `_common_dev_mount_options $*` $SCRATCH_OPTIONS \
@@ -343,9 +343,10 @@ _overlay_mkdirs()
 	mkdir -p $dir/$OVL_UPPER
 	mkdir -p $dir/$OVL_LOWER
 	mkdir -p $dir/$OVL_WORK
+	mkdir -p $dir/$OVL_MNT
 }
 
-# Given a dir, set up 3 subdirectories and mount on the given mnt.
+# Given a base fs dir, set up overlay directories and mount on the given mnt.
 # The dir is used as the mount device so it can be seen from df or mount
 _overlay_mount()
 {
@@ -364,12 +365,12 @@ _overlay_mount()
 
 _overlay_test_mount()
 {
-	_overlay_mount $TEST_DEV $TEST_DIR $*
+	_overlay_mount $OVL_BASE_TEST_DIR $TEST_DIR $*
 }
 
 _overlay_scratch_mount()
 {
-	_overlay_mount $SCRATCH_DEV $SCRATCH_MNT $*
+	_overlay_mount $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT $*
 }
 
 _overlay_test_unmount()
@@ -684,10 +685,12 @@ _scratch_cleanup_files()
 {
 	case $FSTYP in
 	overlay)
-		# $SCRATCH_DEV is a valid directory in overlay case
-		rm -rf $SCRATCH_DEV/*
+		# Avoid rm -rf /* if we messed up
+		[ -n "$OVL_BASE_SCRATCH_MNT" ] || return 1
+		rm -rf $OVL_BASE_SCRATCH_MNT/*
 		;;
 	*)
+		[ -n "$SCRATCH_MNT" ] || return 1
 		_scratch_mount
 		rm -rf $SCRATCH_MNT/*
 		_scratch_unmount
@@ -1425,10 +1428,13 @@ _require_scratch_nocheck()
 		fi
 		;;
 	overlay)
-		if [ -z "$SCRATCH_DEV" -o ! -d "$SCRATCH_DEV" ]; then
-			_notrun "this test requires a valid \$SCRATCH_DEV as ovl base dir"
+		if [ -z "$OVL_BASE_SCRATCH_MNT" -o ! -d "$OVL_BASE_SCRATCH_MNT" ]; then
+			_notrun "this test requires a valid \$OVL_BASE_SCRATCH_MNT as ovl base dir"
 		fi
-		if [ ! -d "$SCRATCH_MNT" ]; then
+		# if $SCRATCH_MNT is derived from $OVL_BASE_SCRATCH_MNT then
+		# don't check $SCRATCH_MNT dir here because base fs may not be mounted
+		# and we will create the mount point anyway on _overlay_mount
+		if [ "$SCRATCH_MNT" != "$OVL_BASE_SCRATCH_MNT/$OVL_MNT" -a ! -d "$SCRATCH_MNT" ]; then
 			_notrun "this test requires a valid \$SCRATCH_MNT"
 		fi
 		;;
@@ -1501,8 +1507,8 @@ _require_test()
 		fi
 		;;
 	overlay)
-		if [ -z "$TEST_DEV" -o ! -d "$TEST_DEV" ]; then
-			_notrun "this test requires a valid \$TEST_DEV as ovl base dir"
+		if [ -z "$OVL_BASE_TEST_DIR" -o ! -d "$OVL_BASE_TEST_DIR" ]; then
+			_notrun "this test requires a valid \$TEST_DIR as ovl base dir"
 		fi
 		if [ ! -d "$TEST_DIR" ]; then
 			_notrun "this test requires a valid \$TEST_DIR"
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v5 09/10] overlay: mount/unmount base fs before/after running tests
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
                   ` (7 preceding siblings ...)
  2017-02-28 12:18 ` [PATCH v5 08/10] overlay: configure TEST/SCRATCH vars to base fs Amir Goldstein
@ 2017-02-28 12:18 ` Amir Goldstein
  2017-02-28 12:18 ` [PATCH v5 10/10] overlay: use OVL_BASE_SCRATCH_MNT instead of SCRATCH_DEV Amir Goldstein
  2017-03-01  3:47 ` [PATCH v5 0/10] fstests: new way to run overlay tests Eryu Guan
  10 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

When TEST/SCRATCH_DEV are configured to the base fs block device,
use this information to mount base fs before running tests,
unmount it after running tests and cycle on _test_cycle_mount
along with the overlay mounts.

This helps catching overlayfs bugs related to leaking objects in
underlying (base) fs.

To preserve expected tests behavior, the semantics are:
- _scratch_mkfs mounts the base fs, cleans all files, creates
  lower/upper dirs and keeps base fs mounted
- _scratch_mount mounts base fs (if needed) and mounts overlay
- _scratch_unmount unmounts overlay and base fs

Tests that use _scratch_unmount to unmount a custom overlay mount
and expect to have access to overlay base dir, were fixed to use
explicit umount $SCRATCH_MNT instead.

The overlay test itself, does not support formatting the base fs,
so config options like MKFS_OPTIONS and FSCK_OPTIONS are ignored
on 'check -overlay'.
The config option MOUNT_OPTIONS is used to mount the base scratch fs.
The config option TEST_FS_MOUNT_OPTS is used to mount the base test fs.
The config option OVERLAY_MOUNT_OPTIONS is used to mount both test and
scratch overlay mounts.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 README.overlay    | 28 +++++++++++++++++++++++
 common/rc         | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++----
 tests/overlay/003 |  3 ++-
 tests/overlay/004 |  3 ++-
 tests/overlay/014 |  5 +++--
 5 files changed, 98 insertions(+), 8 deletions(-)

diff --git a/README.overlay b/README.overlay
index 647f9ea..dfb8234 100644
--- a/README.overlay
+++ b/README.overlay
@@ -16,3 +16,31 @@ use the same partitions as base fs for overlayfs directories
 and set TEST_DIR/SCRATCH_MNT values to overlay mount points, i.e.:
 /mnt/test/ovl-mnt and /mnt/scratch/ovl-mnt, for the context of
 individual tests.
+
+'./check -overlay' does not support mkfs and fsck on the base fs, so
+the base fs should be pre-formatted before starting the -overlay run.
+An easy way to accomplish this is by running './check <some test>' once,
+before running './check -overlay'.
+
+Because of the lack of mkfs support, multi-section config files are only
+partly supported with './check -overlay'. Only multi-section files that
+do not change FSTYP and MKFS_OPTIONS can be safely used with -overlay.
+
+For example, the following multi-section config file can be used to
+run overlay tests on the same base fs, but with different mount options:
+
+ [xfs]
+ TEST_DEV=/dev/sda5
+ TEST_DIR=/mnt/test
+ SCRATCH_DEV=/dev/sda6
+ SCRATCH_MNT=/mnt/scratch
+ FSTYP=xfs
+
+ [xfs_pquota]
+ MOUNT_OPTIONS="-o pquota"
+ TEST_FS_MOUNT_OPTS="-o noatime"
+ OVERLAY_MOUNT_OPTIONS="-o redirect_dir=off"
+
+In the example above, MOUNT_OPTIONS will be used to mount the base scratch fs,
+TEST_FS_MOUNT_OPTS will be used to mount the base test fs and
+OVERLAY_MOUNT_OPTIONS will be used to mount both test and scratch overlays.
diff --git a/common/rc b/common/rc
index fa4ca5e..7577223 100644
--- a/common/rc
+++ b/common/rc
@@ -363,24 +363,80 @@ _overlay_mount()
 			    $SELINUX_MOUNT_OPTIONS $* $dir $mnt
 }
 
+_overlay_base_test_mount()
+{
+	if [ -z "$OVL_BASE_TEST_DEV" -o -z "$OVL_BASE_TEST_DIR" ] || \
+		_check_mounted_on OVL_BASE_TEST_DEV $OVL_BASE_TEST_DEV \
+				OVL_BASE_TEST_DIR $OVL_BASE_TEST_DIR
+	then
+		# no base fs or already mounted
+		return 0
+	elif [ $? -ne 1 ]
+	then
+		# base fs mounted but not on mount point
+		return 1
+	fi
+
+	_mount $TEST_FS_MOUNT_OPTS \
+		$SELINUX_MOUNT_OPTIONS \
+		$OVL_BASE_TEST_DEV $OVL_BASE_TEST_DIR
+}
+
 _overlay_test_mount()
 {
-	_overlay_mount $OVL_BASE_TEST_DIR $TEST_DIR $*
+	_overlay_base_test_mount && \
+		_overlay_mount $OVL_BASE_TEST_DIR $TEST_DIR $*
+}
+
+_overlay_base_scratch_mount()
+{
+	if [ -z "$OVL_BASE_SCRATCH_DEV" -o -z "$OVL_BASE_SCRATCH_MNT" ] || \
+		_check_mounted_on OVL_BASE_SCRATCH_DEV $OVL_BASE_SCRATCH_DEV \
+				OVL_BASE_SCRATCH_MNT $OVL_BASE_SCRATCH_MNT
+	then
+		# no base fs or already mounted
+		return 0
+	elif [ $? -ne 1 ]
+	then
+		# base fs mounted but not on mount point
+		return 1
+	fi
+
+	_mount $OVL_BASE_MOUNT_OPTIONS \
+		$SELINUX_MOUNT_OPTIONS \
+		$OVL_BASE_SCRATCH_DEV $OVL_BASE_SCRATCH_MNT
+}
+
+_overlay_base_scratch_unmount()
+{
+	[ -n "$OVL_BASE_SCRATCH_DEV" -a -n "$OVL_BASE_SCRATCH_MNT" ] || return 0
+
+	$UMOUNT_PROG $OVL_BASE_SCRATCH_MNT
 }
 
 _overlay_scratch_mount()
 {
-	_overlay_mount $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT $*
+	_overlay_base_scratch_mount && \
+		_overlay_mount $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT $*
+}
+
+_overlay_base_test_unmount()
+{
+	[ -n "$OVL_BASE_TEST_DEV" -a -n "$OVL_BASE_TEST_DIR" ] || return 0
+
+	$UMOUNT_PROG $OVL_BASE_TEST_DIR
 }
 
 _overlay_test_unmount()
 {
 	$UMOUNT_PROG $TEST_DIR
+	_overlay_base_test_unmount
 }
 
 _overlay_scratch_unmount()
 {
 	$UMOUNT_PROG $SCRATCH_MNT
+	_overlay_base_scratch_unmount
 }
 
 _scratch_mount()
@@ -687,7 +743,10 @@ _scratch_cleanup_files()
 	overlay)
 		# Avoid rm -rf /* if we messed up
 		[ -n "$OVL_BASE_SCRATCH_MNT" ] || return 1
-		rm -rf $OVL_BASE_SCRATCH_MNT/*
+		_overlay_base_scratch_mount || return 1
+		rm -rf $OVL_BASE_SCRATCH_MNT/* || return 1
+		_overlay_mkdirs $OVL_BASE_SCRATCH_MNT
+		# leave base fs mouted so tests can setup lower/upper dir files
 		;;
 	*)
 		[ -n "$SCRATCH_MNT" ] || return 1
@@ -710,7 +769,7 @@ _scratch_mkfs()
 		# $SCRATCH_MNT to avoid EEXIST caused by the leftover files
 		# created in previous runs
 		_scratch_cleanup_files
-		return 0
+		return $?
 		;;
 	tmpfs)
 		# do nothing for tmpfs
diff --git a/tests/overlay/003 b/tests/overlay/003
index 5e610c8..7625273 100755
--- a/tests/overlay/003
+++ b/tests/overlay/003
@@ -89,7 +89,8 @@ rm -rf ${SCRATCH_MNT}/*
 # nothing should be listed
 ls ${SCRATCH_MNT}/
 
-_scratch_unmount
+# unmount overlayfs but not base fs
+$UMOUNT_PROG $SCRATCH_MNT
 
 rm -rf $lowerdir
 echo "Silence is golden"
diff --git a/tests/overlay/004 b/tests/overlay/004
index 8da5170..72b0ab3 100755
--- a/tests/overlay/004
+++ b/tests/overlay/004
@@ -85,7 +85,8 @@ _user_do "chmod g+t ${SCRATCH_MNT}/attr_file2 > /dev/null 2>&1"
 _user_do "chmod u-X ${SCRATCH_MNT}/attr_file2 > /dev/null 2>&1"
 stat -c %a ${SCRATCH_MNT}/attr_file2
 
-_scratch_unmount
+# unmount overlayfs but not base fs
+$UMOUNT_PROG $SCRATCH_MNT
 
 # check mode bits of the file that has been copied up, and
 # the file that should not have been copied up.
diff --git a/tests/overlay/014 b/tests/overlay/014
index 36d7077..c95a892 100755
--- a/tests/overlay/014
+++ b/tests/overlay/014
@@ -73,7 +73,8 @@ mkdir -p $lowerdir1/testdir/d
 _overlay_mount_dirs $lowerdir1 $lowerdir2 $workdir $SCRATCH_DEV $SCRATCH_MNT
 rm -rf $SCRATCH_MNT/testdir
 mkdir -p $SCRATCH_MNT/testdir/visibledir
-_scratch_unmount
+# unmount overlayfs but not base fs
+$UMOUNT_PROG $SCRATCH_MNT
 
 # mount overlay again, with lowerdir1 and lowerdir2 as multiple lowerdirs,
 # and create a new file in testdir, triggers copyup from lowerdir,
@@ -84,7 +85,7 @@ touch $SCRATCH_MNT/testdir/visiblefile
 
 # umount and mount overlay again, buggy kernel treats the copied-up dir as
 # opaque, visibledir is not seen in merged dir.
-_scratch_unmount
+$UMOUNT_PROG $SCRATCH_MNT
 _overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
 		    $SCRATCH_DEV $SCRATCH_MNT
 ls $SCRATCH_MNT/testdir
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v5 10/10] overlay: use OVL_BASE_SCRATCH_MNT instead of SCRATCH_DEV
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
                   ` (8 preceding siblings ...)
  2017-02-28 12:18 ` [PATCH v5 09/10] overlay: mount/unmount base fs before/after running tests Amir Goldstein
@ 2017-02-28 12:18 ` Amir Goldstein
  2017-03-01  3:47 ` [PATCH v5 0/10] fstests: new way to run overlay tests Eryu Guan
  10 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2017-02-28 12:18 UTC (permalink / raw)
  To: Eryu Guan; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

Use the new var OVL_BASE_SCRATCH_MNT to refer to overlay
base dir instead of the legacy SCRATCH_DEV var.

In overlay/011, when using MOUNT_PROG directly, provide
OVL_BASE_SCRATCH_MNT as device argument instead of 'none'.
This fix is required to support mount of base fs.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 tests/overlay/001 |  7 +++----
 tests/overlay/002 |  2 +-
 tests/overlay/003 |  2 +-
 tests/overlay/004 |  4 ++--
 tests/overlay/005 | 30 +++++++++++++++---------------
 tests/overlay/006 |  8 ++++----
 tests/overlay/008 |  4 ++--
 tests/overlay/009 |  2 +-
 tests/overlay/010 | 10 +++++-----
 tests/overlay/011 |  6 +++---
 tests/overlay/012 |  4 ++--
 tests/overlay/013 |  4 ++--
 tests/overlay/014 | 14 +++++++-------
 tests/overlay/015 |  2 +-
 tests/overlay/016 |  2 +-
 tests/overlay/017 |  2 +-
 tests/overlay/018 |  2 +-
 tests/overlay/019 |  2 +-
 tests/overlay/020 |  2 +-
 tests/overlay/021 |  6 +++---
 tests/overlay/022 |  2 +-
 tests/overlay/023 |  2 +-
 tests/overlay/024 |  2 +-
 tests/overlay/027 |  2 +-
 tests/overlay/028 |  2 +-
 tests/overlay/029 |  4 ++--
 26 files changed, 64 insertions(+), 65 deletions(-)

diff --git a/tests/overlay/001 b/tests/overlay/001
index 099ddd5..c0abbfb 100755
--- a/tests/overlay/001
+++ b/tests/overlay/001
@@ -55,12 +55,11 @@ _require_scratch
 _scratch_mkfs
 
 # overlay copy_up doesn't deal with sparse file well, holes will be filled by
-# zeros, so at least (4G + 4G + 8k) free space is needed on $SCRATCH_DEV,
-# where $SCRATCH_DEV is actually a test dir used in overlay testing
-_require_fs_space $SCRATCH_DEV $((4*1024*1024*2 + 8))
+# zeros, so at least (4G + 4G + 8k) free space is needed on scratch base fs
+_require_fs_space $OVL_BASE_SCRATCH_MNT $((4*1024*1024*2 + 8))
 
 # Create test files with different sizes in lower dir
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/zero_size
 $XFS_IO_PROG -fc "pwrite 0 4k" $lowerdir/less_than_4g >>$seqres.full
diff --git a/tests/overlay/002 b/tests/overlay/002
index eaf9a91..ffdec18 100755
--- a/tests/overlay/002
+++ b/tests/overlay/002
@@ -59,7 +59,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create our test file.
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/foobar
 
diff --git a/tests/overlay/003 b/tests/overlay/003
index 7625273..f980edb 100755
--- a/tests/overlay/003
+++ b/tests/overlay/003
@@ -60,7 +60,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create test files dirs in lower dir
-lowerdir=${SCRATCH_DEV}/${OVL_LOWER}
+lowerdir=${OVL_BASE_SCRATCH_MNT}/${OVL_LOWER}
 mkdir -p $lowerdir
 
 touch ${lowerdir}/test_file
diff --git a/tests/overlay/004 b/tests/overlay/004
index 72b0ab3..611847a 100755
--- a/tests/overlay/004
+++ b/tests/overlay/004
@@ -53,8 +53,8 @@ _require_user
 _scratch_mkfs
 
 # Create test file in lower dir
-lowerdir=${SCRATCH_DEV}/${OVL_LOWER}
-upperdir=${SCRATCH_DEV}/${OVL_UPPER}
+lowerdir=${OVL_BASE_SCRATCH_MNT}/${OVL_LOWER}
+upperdir=${OVL_BASE_SCRATCH_MNT}/${OVL_UPPER}
 mkdir -p $lowerdir
 touch ${lowerdir}/attr_file1
 chmod 000 ${lowerdir}/attr_file1
diff --git a/tests/overlay/005 b/tests/overlay/005
index baadb69..f885fdc 100755
--- a/tests/overlay/005
+++ b/tests/overlay/005
@@ -61,31 +61,31 @@ _require_loop
 _scratch_mkfs
 
 # setup loop images fs for overlayfs
-lower_img=${SCRATCH_DEV}/${seq}.$$.img
+lower_img=${OVL_BASE_SCRATCH_MNT}/${seq}.$$.img
 $XFS_IO_PROG -f -c "truncate 120m" $lower_img >>$seqres.full 2>&1
 fs_loop_dev=`_create_loop_device $lower_img`
 $MKFS_XFS_PROG -f -n ftype=1 $fs_loop_dev >>$seqres.full 2>&1
 
 # only 20m for upper dir
-upper_img=${SCRATCH_DEV}/$$.${seq}.img
+upper_img=${OVL_BASE_SCRATCH_MNT}/$$.${seq}.img
 $XFS_IO_PROG -f -c "truncate 20m" $upper_img >>$seqres.full 2>&1
 upper_loop_dev=`_create_loop_device $upper_img`
 $MKFS_XFS_PROG -f -n ftype=1 $upper_loop_dev >>$seqres.full 2>&1
 
 # mount underlying xfs
-mkdir -p ${SCRATCH_DEV}/lowermnt
-mkdir -p ${SCRATCH_DEV}/uppermnt
-$MOUNT_PROG $fs_loop_dev ${SCRATCH_DEV}/lowermnt
-$MOUNT_PROG $upper_loop_dev ${SCRATCH_DEV}/uppermnt
+mkdir -p ${OVL_BASE_SCRATCH_MNT}/lowermnt
+mkdir -p ${OVL_BASE_SCRATCH_MNT}/uppermnt
+$MOUNT_PROG $fs_loop_dev ${OVL_BASE_SCRATCH_MNT}/lowermnt
+$MOUNT_PROG $upper_loop_dev ${OVL_BASE_SCRATCH_MNT}/uppermnt
 
 # prepare dirs
-mkdir -p ${SCRATCH_DEV}/lowermnt/lower
-mkdir -p ${SCRATCH_DEV}/uppermnt/upper
-mkdir -p ${SCRATCH_DEV}/uppermnt/work
+mkdir -p ${OVL_BASE_SCRATCH_MNT}/lowermnt/lower
+mkdir -p ${OVL_BASE_SCRATCH_MNT}/uppermnt/upper
+mkdir -p ${OVL_BASE_SCRATCH_MNT}/uppermnt/work
 
-lowerd=${SCRATCH_DEV}/lowermnt/lower
-upperd=${SCRATCH_DEV}/uppermnt/upper
-workd=${SCRATCH_DEV}/uppermnt/work
+lowerd=${OVL_BASE_SCRATCH_MNT}/lowermnt/lower
+upperd=${OVL_BASE_SCRATCH_MNT}/uppermnt/upper
+workd=${OVL_BASE_SCRATCH_MNT}/uppermnt/work
 
 # Create test file in lower dir, with too big a size for
 # upper dir to copy up.
@@ -93,7 +93,7 @@ $XFS_IO_PROG -f -c "truncate 48m" ${lowerd}/test_file \
 	>>$seqres.full 2>&1
 
 # mount new overlayfs
-_overlay_mount_dirs $lowerd $upperd $workd $SCRATCH_DEV $SCRATCH_MNT
+_overlay_mount_dirs $lowerd $upperd $workd $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 
 # the open call triggers copy-up and it will fail ENOSPC
 $XFS_IO_PROG -f -c "o" ${SCRATCH_MNT}/test_file \
@@ -103,8 +103,8 @@ $XFS_IO_PROG -f -c "o" ${SCRATCH_MNT}/test_file \
 $UMOUNT_PROG $SCRATCH_MNT
 
 # unmount undelying xfs, this tiggers panic if memleak happens
-$UMOUNT_PROG ${SCRATCH_DEV}/uppermnt
-$UMOUNT_PROG ${SCRATCH_DEV}/lowermnt
+$UMOUNT_PROG ${OVL_BASE_SCRATCH_MNT}/uppermnt
+$UMOUNT_PROG ${OVL_BASE_SCRATCH_MNT}/lowermnt
 
 # success, all done
 echo "Silence is golden"
diff --git a/tests/overlay/006 b/tests/overlay/006
index 55fc2dd..815657f 100755
--- a/tests/overlay/006
+++ b/tests/overlay/006
@@ -57,10 +57,10 @@ echo "Silence is golden"
 _scratch_mkfs
 
 # Create test file/dir before mount
-mkdir -p $SCRATCH_DEV/$OVL_LOWER
-mkdir -p $SCRATCH_DEV/$OVL_UPPER
-touch $SCRATCH_DEV/$OVL_LOWER/lowertestfile
-mkdir $SCRATCH_DEV/$OVL_UPPER/uppertestdir
+mkdir -p $OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+mkdir -p $OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+touch $OVL_BASE_SCRATCH_MNT/$OVL_LOWER/lowertestfile
+mkdir $OVL_BASE_SCRATCH_MNT/$OVL_UPPER/uppertestdir
 
 _scratch_mount
 
diff --git a/tests/overlay/008 b/tests/overlay/008
index b81cff8..d5bc1df 100755
--- a/tests/overlay/008
+++ b/tests/overlay/008
@@ -56,13 +56,13 @@ _require_user
 _scratch_mkfs
 
 # Create test file on lower dir, and chown to fsgqa user
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/testfile
 chown fsgqa:fsgqa $lowerdir/testfile
 
 # chown upperdir to fsgqa user, so new file/dir can be created by the user
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $upperdir
 chown fsgqa:fsgqa $upperdir
 
diff --git a/tests/overlay/009 b/tests/overlay/009
index bdac6b7..f367887 100755
--- a/tests/overlay/009
+++ b/tests/overlay/009
@@ -54,7 +54,7 @@ _require_scratch
 _scratch_mkfs
 
 # Create test file in lowerdir
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/testfile
 
diff --git a/tests/overlay/010 b/tests/overlay/010
index 5882db9..dee99cf 100755
--- a/tests/overlay/010
+++ b/tests/overlay/010
@@ -55,10 +55,10 @@ _scratch_mkfs
 
 # Need two lower dirs in this test, and we mount overlay ourselves,
 # create upper and workdir as well
-lowerdir1=$SCRATCH_DEV/$OVL_LOWER.1
-lowerdir2=$SCRATCH_DEV/$OVL_LOWER.2
-upperdir=$SCRATCH_DEV/$OVL_UPPER
-workdir=$SCRATCH_DEV/$OVL_WORK
+lowerdir1=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER.1
+lowerdir2=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER.2
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+workdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
 mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir
 
 # One lowerdir contains test dir and test files, the other contains whiteout
@@ -68,7 +68,7 @@ mknod $lowerdir2/testdir/a c 0 0
 
 # Mount overlayfs and remove testdir, which led to kernel crash
 _overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
-		    $SCRATCH_DEV $SCRATCH_MNT
+		    $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 rm -rf $SCRATCH_MNT/testdir
 
 # success, all done
diff --git a/tests/overlay/011 b/tests/overlay/011
index 1d6c44a..87ac17b 100755
--- a/tests/overlay/011
+++ b/tests/overlay/011
@@ -58,8 +58,8 @@ _require_attrs
 _scratch_mkfs
 
 # Create test dir on upper and make it opaque by setting proper xattr
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $lowerdir/testdir
 mkdir -p $upperdir/testdir
 $SETFATTR_PROG -n "trusted.overlay.opaque" -v "y" $upperdir/testdir
@@ -68,7 +68,7 @@ $SETFATTR_PROG -n "trusted.overlay.opaque" -v "y" $upperdir/testdir
 # $upperdir overlaid on top of $lowerdir, so that "trusted.overlay.opaque"
 # xattr should be honored and should not be listed
 # mount readonly, because there's no upper and workdir
-$MOUNT_PROG -t overlay -o ro -o lowerdir=$upperdir:$lowerdir none $SCRATCH_MNT
+$MOUNT_PROG -t overlay -o ro -o lowerdir=$upperdir:$lowerdir $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 
 # Dump trusted.overlay xattr, we should not see the "opaque" xattr
 $GETFATTR_PROG -d -m overlay $SCRATCH_MNT/testdir
diff --git a/tests/overlay/012 b/tests/overlay/012
index 1c4f7c6..9cf1526 100755
--- a/tests/overlay/012
+++ b/tests/overlay/012
@@ -55,8 +55,8 @@ _require_scratch
 # remove all files from previous runs
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $lowerdir/test
 
 _scratch_mount
diff --git a/tests/overlay/013 b/tests/overlay/013
index 536e981..de0fc26 100755
--- a/tests/overlay/013
+++ b/tests/overlay/013
@@ -54,8 +54,8 @@ _require_test_program "t_truncate_self"
 _scratch_mkfs
 
 # copy test program to lower and upper dir
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $lowerdir
 mkdir -p $upperdir
 cp $here/src/t_truncate_self $lowerdir/test_lower
diff --git a/tests/overlay/014 b/tests/overlay/014
index c95a892..653b7d6 100755
--- a/tests/overlay/014
+++ b/tests/overlay/014
@@ -61,16 +61,16 @@ _require_scratch
 _scratch_mkfs
 
 # Create multiple lowerdirs and upperdir, workdir, and testdir on lowerdir
-lowerdir1=$SCRATCH_DEV/lower1
-lowerdir2=$SCRATCH_DEV/lower2
-upperdir=$SCRATCH_DEV/upper
-workdir=$SCRATCH_DEV/workdir
+lowerdir1=$OVL_BASE_SCRATCH_MNT/lower1
+lowerdir2=$OVL_BASE_SCRATCH_MNT/lower2
+upperdir=$OVL_BASE_SCRATCH_MNT/upper
+workdir=$OVL_BASE_SCRATCH_MNT/workdir
 mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir
 mkdir -p $lowerdir1/testdir/d
 
 # mount overlay with $lowerdir2 as upperdir, and remove & recreate testdir,
 # make testdir on $lowerdir2 opaque
-_overlay_mount_dirs $lowerdir1 $lowerdir2 $workdir $SCRATCH_DEV $SCRATCH_MNT
+_overlay_mount_dirs $lowerdir1 $lowerdir2 $workdir $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 rm -rf $SCRATCH_MNT/testdir
 mkdir -p $SCRATCH_MNT/testdir/visibledir
 # unmount overlayfs but not base fs
@@ -80,14 +80,14 @@ $UMOUNT_PROG $SCRATCH_MNT
 # and create a new file in testdir, triggers copyup from lowerdir,
 # copyup should not copy overlayfs private xattr
 _overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
-		    $SCRATCH_DEV $SCRATCH_MNT
+		    $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 touch $SCRATCH_MNT/testdir/visiblefile
 
 # umount and mount overlay again, buggy kernel treats the copied-up dir as
 # opaque, visibledir is not seen in merged dir.
 $UMOUNT_PROG $SCRATCH_MNT
 _overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
-		    $SCRATCH_DEV $SCRATCH_MNT
+		    $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 ls $SCRATCH_MNT/testdir
 
 # success, all done
diff --git a/tests/overlay/015 b/tests/overlay/015
index 0e09e0c..4894097 100755
--- a/tests/overlay/015
+++ b/tests/overlay/015
@@ -57,7 +57,7 @@ _scratch_mkfs
 umask 022
 
 # Create test dir in lower dir and set sgid bit
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir/dir
 chown $qa_user:$qa_group $lowerdir/dir
 chmod 2775 $lowerdir/dir
diff --git a/tests/overlay/016 b/tests/overlay/016
index c678ea4..86a7d28 100755
--- a/tests/overlay/016
+++ b/tests/overlay/016
@@ -57,7 +57,7 @@ rm -f $seqres.full
 _scratch_mkfs >>$seqres.full 2>&1
 
 # Create our test files.
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 echo "This is old news" > $lowerdir/foo
 echo "This is old news" > $lowerdir/bar
diff --git a/tests/overlay/017 b/tests/overlay/017
index 6b12722..5ca3040 100755
--- a/tests/overlay/017
+++ b/tests/overlay/017
@@ -60,7 +60,7 @@ _scratch_mkfs >>$seqres.full 2>&1
 # Not dealing with hardlinks here, because there is more to test
 # then stable inode number.
 # Hardlinks will get a test of their own.
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 mkdir $lowerdir/dir
 touch $lowerdir/file
diff --git a/tests/overlay/018 b/tests/overlay/018
index 37bf11c..7e47732 100755
--- a/tests/overlay/018
+++ b/tests/overlay/018
@@ -55,7 +55,7 @@ rm -f $seqres.full
 _scratch_mkfs >>$seqres.full 2>&1
 
 # Create 2 hardlinked files in lower
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 echo "patient zero" >> $lowerdir/foo
 ln $lowerdir/foo $lowerdir/bar
diff --git a/tests/overlay/019 b/tests/overlay/019
index 9177acb..3e2bc4e 100755
--- a/tests/overlay/019
+++ b/tests/overlay/019
@@ -51,7 +51,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 
 _scratch_mount
diff --git a/tests/overlay/020 b/tests/overlay/020
index 3cd0f05..de3816c 100755
--- a/tests/overlay/020
+++ b/tests/overlay/020
@@ -55,7 +55,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir/dir
 
 _scratch_mount
diff --git a/tests/overlay/021 b/tests/overlay/021
index c6d68aa..1347f52 100755
--- a/tests/overlay/021
+++ b/tests/overlay/021
@@ -54,13 +54,13 @@ _scratch_mkfs
 
 # overlay copy_up doesn't deal with sparse file well, holes will be filled by
 # zeros, so for the worst case of hitting all the copy up bomb files, we need
-# (64*(16+32+64+128)M) free space on $SCRATCH_DEV.
+# (64*(16+32+64+128)M) free space on $OVL_BASE_SCRATCH_MNT.
 # However, triggering more than a total of 16 copy up bombs would require
 # really fast data copy (clone up doesn't take up space at all), so let's be
 # conservative and reserve space for 16 data copy ups per directory.
-_require_fs_space $SCRATCH_DEV $((16*(16+32+64+128)*1024))
+_require_fs_space $OVL_BASE_SCRATCH_MNT $((16*(16+32+64+128)*1024))
 
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 
 testdir=arena
diff --git a/tests/overlay/022 b/tests/overlay/022
index 46034ea..ef4d73f 100755
--- a/tests/overlay/022
+++ b/tests/overlay/022
@@ -59,7 +59,7 @@ _require_scratch
 # Remove all files from previous tests
 _scratch_mkfs
 
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $upperdir/upper
 mkdir -p $upperdir/work
 # mount overlay with dirs in upper
diff --git a/tests/overlay/023 b/tests/overlay/023
index c855d16..8da3c20 100755
--- a/tests/overlay/023
+++ b/tests/overlay/023
@@ -63,7 +63,7 @@ _require_scratch
 _scratch_mkfs
 
 # setting acls before mount
-wkdir=$SCRATCH_DEV/$OVL_WORK
+wkdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
 mkdir -p $wkdir
 setfacl -d -m o::rwx $wkdir
 
diff --git a/tests/overlay/024 b/tests/overlay/024
index 25195f7..75bd067 100755
--- a/tests/overlay/024
+++ b/tests/overlay/024
@@ -61,7 +61,7 @@ _require_scratch
 _scratch_mkfs
 
 # making workdir
-wkdir=$SCRATCH_DEV/$OVL_WORK
+wkdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
 mkdir -p $wkdir/work/foo
 
 _scratch_mount
diff --git a/tests/overlay/027 b/tests/overlay/027
index 1c77956..10111b7 100755
--- a/tests/overlay/027
+++ b/tests/overlay/027
@@ -63,7 +63,7 @@ _require_chattr
 _scratch_mkfs
 
 # Preparing immutable file
-upperdir=$SCRATCH_DEV/$OVL_UPPER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
 mkdir -p $upperdir
 touch $upperdir/foo
 $CHATTR_PROG +i $upperdir/foo
diff --git a/tests/overlay/028 b/tests/overlay/028
index ad610e2..4b8a3a1 100755
--- a/tests/overlay/028
+++ b/tests/overlay/028
@@ -61,7 +61,7 @@ _require_command "$FLOCK_PROG" flock
 # Remove all files from previous tests
 _scratch_mkfs
 
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $lowerdir
 touch $lowerdir/foo
 
diff --git a/tests/overlay/029 b/tests/overlay/029
index 8dce6b3..2ac674f 100755
--- a/tests/overlay/029
+++ b/tests/overlay/029
@@ -65,8 +65,8 @@ _require_scratch
 _scratch_mkfs
 
 # Preparing files
-upperdir=$SCRATCH_DEV/$OVL_UPPER
-lowerdir=$SCRATCH_DEV/$OVL_LOWER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
 mkdir -p $upperdir/up
 echo foo > $upperdir/up/foo
 mkdir -p $lowerdir/low
-- 
2.7.4


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [PATCH v5 0/10] fstests: new way to run overlay tests
  2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
                   ` (9 preceding siblings ...)
  2017-02-28 12:18 ` [PATCH v5 10/10] overlay: use OVL_BASE_SCRATCH_MNT instead of SCRATCH_DEV Amir Goldstein
@ 2017-03-01  3:47 ` Eryu Guan
  10 siblings, 0 replies; 12+ messages in thread
From: Eryu Guan @ 2017-03-01  3:47 UTC (permalink / raw)
  To: Amir Goldstein; +Cc: Miklos Szeredi, Xiong Zhou, linux-unionfs, fstests

On Tue, Feb 28, 2017 at 02:18:27PM +0200, Amir Goldstein wrote:
> Hi Eryu,
> 
> I rebased my branch to your master and made some minor adaptations
> to Xiong's new tests. Also reordered the patches:

Thanks for the resending! I was about to ask for it :)

> 
> - patches 1-6 are generic fixes and re-factoring - if you wish,
>   you can merge them before the overlay config changes.
> 
> - patches 7-10 implement the new overlay config. patches 7 and 10
>   include adaptations of Xiong's new tests to new config.
> 
> This is the 5th revision of new overlayfs config.
> The main motivation of this work is to help catch overlayfs bugs
> related to leaking objects in underlying (base) fs.
> 
> With this change, all you have to do to run overlay tests if you
> already have a local.config setup to test a local file system is:
>  ./check -overlay
> 
> See README.overlay for some examples of config files to use with -overlay.
> 
> The legacy overlayfs configuration, where TEST_DEV is a directory
> still works, but it should be deprecated.
> 
> Tested ./check -overlay -g quick with both legacy overlay configuration
> and the new base fs configuration.
> 
> Eryu wished to keep these changes soaking on the list for while,
> so for those who can help with testing, you can get pull the branch
> for testing from my github tree [1].

I was planning to pull them in via this week's update, if there's no
objections.

Thanks again for the work!

Eryu

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2017-03-01  4:47 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-02-28 12:18 [PATCH v5 0/10] fstests: new way to run overlay tests Amir Goldstein
2017-02-28 12:18 ` [PATCH v5 01/10] fstests: sanity check that test partitions are not mounted elsewhere Amir Goldstein
2017-02-28 12:18 ` [PATCH v5 02/10] fstests: use _test_mount() consistently Amir Goldstein
2017-02-28 12:18 ` [PATCH v5 03/10] fstests: canonicalize mount points on every config section Amir Goldstein
2017-02-28 12:18 ` [PATCH v5 04/10] fstests: fix test and scratch filters for overlapping DEV/MNT paths Amir Goldstein
2017-02-28 12:18 ` [PATCH v5 05/10] fstests: allow overlay SCRATCH_DEV to be a base fs mount point Amir Goldstein
2017-02-28 12:18 ` [PATCH v5 06/10] generic/064: access SCRATCH_MNT after _scratch_mount Amir Goldstein
2017-02-28 12:18 ` [PATCH v5 07/10] overlay: rename OVERLAY_LOWER/UPPER/WORK_DIR Amir Goldstein
2017-02-28 12:18 ` [PATCH v5 08/10] overlay: configure TEST/SCRATCH vars to base fs Amir Goldstein
2017-02-28 12:18 ` [PATCH v5 09/10] overlay: mount/unmount base fs before/after running tests Amir Goldstein
2017-02-28 12:18 ` [PATCH v5 10/10] overlay: use OVL_BASE_SCRATCH_MNT instead of SCRATCH_DEV Amir Goldstein
2017-03-01  3:47 ` [PATCH v5 0/10] fstests: new way to run overlay tests Eryu Guan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox