[PATCH v3 1/2] common/rc: introduce _overlay_mount_dirs helper

[PATCH v3 1/2] common/rc: introduce _overlay_mount_dirs helper

[Eryu Guan]

Introduce a new _overlay_mount_dirs() helper to do the actual
overlay mount operation and factor _overlay_scratch_mount and
_overlay_mount code to use a single implemention of
_overlay_mount_dirs(). Also convert all bare mount of overlay in
tests to use it.

Suggested-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Eryu Guan <eguan@redhat.com>
---
v3:
- mount with SELINUX_MOUNT_OPTIONS in _overlay_mount_dirs

 common/rc         | 21 +++++++++++++++------
 tests/overlay/005 |  5 +----
 tests/overlay/010 |  5 ++---
 3 files changed, 18 insertions(+), 13 deletions(-)

diff --git a/common/rc b/common/rc
index 04039a4..899bef4 100644
--- a/common/rc
+++ b/common/rc
@@ -266,6 +266,18 @@ _scratch_mount_options()
 					$SCRATCH_DEV $SCRATCH_MNT
 }
 
+# helper function to do the actual overlayfs mount operation
+_overlay_mount_dirs()
+{
+	local lowerdir=$1
+	local upperdir=$2
+	local workdir=$3
+	shift 3
+
+	$MOUNT_PROG -t overlay -o lowerdir=$lowerdir -o upperdir=$upperdir \
+		    -o workdir=$workdir $*
+}
+
 # Given a dir, set up 3 subdirectories and mount on the given mnt.
 # The dir is used as the mount device so it can be seen from df or mount
 _overlay_mount()
@@ -293,12 +305,9 @@ _overlay_mount()
 	mkdir -p $dir/$OVERLAY_LOWER_DIR
 	mkdir -p $dir/$OVERLAY_WORK_DIR
 
-	$MOUNT_PROG -t overlay $SELINUX_MOUNT_OPTIONS \
-		    -o lowerdir=$dir/$OVERLAY_LOWER_DIR \
-		    -o upperdir=$dir/$OVERLAY_UPPER_DIR \
-		    -o workdir=$dir/$OVERLAY_WORK_DIR   \
-		    $OVERLAY_MOUNT_OPTIONS $* \
-		    $dir $mnt
+	_overlay_mount_dirs $dir/$OVERLAY_LOWER_DIR $dir/$OVERLAY_UPPER_DIR \
+			    $dir/$OVERLAY_WORK_DIR $OVERLAY_MOUNT_OPTIONS \
+			    $SELINUX_MOUNT_OPTIONS $* $dir $mnt
 }
 
 _overlay_test_mount()
diff --git a/tests/overlay/005 b/tests/overlay/005
index 495e7b5..baadb69 100755
--- a/tests/overlay/005
+++ b/tests/overlay/005
@@ -93,10 +93,7 @@ $XFS_IO_PROG -f -c "truncate 48m" ${lowerd}/test_file \
 	>>$seqres.full 2>&1
 
 # mount new overlayfs
-$MOUNT_PROG -t overlay $SELINUX_MOUNT_OPTIONS \
-	    -o lowerdir=$lowerd -o upperdir=$upperd \
-	    -o workdir=$workd \
-	    $SCRATCH_DEV $SCRATCH_MNT
+_overlay_mount_dirs $lowerd $upperd $workd $SCRATCH_DEV $SCRATCH_MNT
 
 # the open call triggers copy-up and it will fail ENOSPC
 $XFS_IO_PROG -f -c "o" ${SCRATCH_MNT}/test_file \
diff --git a/tests/overlay/010 b/tests/overlay/010
index 5d1c2bd..a302d74 100755
--- a/tests/overlay/010
+++ b/tests/overlay/010
@@ -67,9 +67,8 @@ touch $lowerdir1/testdir/a $lowerdir1/testdir/b
 mknod $lowerdir2/testdir/a c 0 0
 
 # Mount overlayfs and remove testdir, which led to kernel crash
-$MOUNT_PROG -t overlay -o lowerdir=$lowerdir2:$lowerdir1 \
-	    -o upperdir=$upperdir,workdir=$workdir \
-	    $SCRATCH_DEV $SCRATCH_MNT
+_overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
+		    $SCRATCH_DEV $SCRATCH_MNT
 rm -rf $SCRATCH_MNT/testdir
 
 # success, all done
-- 
2.7.4

[PATCH v3 2/2] overlay: copy-up lowerdir which has opaque xattr set

[Eryu Guan]

When copying up a directory with opaque xattr set, the xattr was
copied up as well, after a remount the opaque flag is honored so the
copied-up directory is treated as opaque, which causes files created
in the lowerdir to be hidden when using multiple lower directories.

This is fixed by "ovl: don't copy up opaqueness".

Signed-off-by: Eryu Guan <eguan@redhat.com>
---
v3:
- replace mounted device "none" by $SCRATCH_DEV

v2:
- use _overlay_mount_dirs helper to do the overlay mount
- use _scratch_unmount instead of raw $UMOUNT_PROG
- remove unused _require_attrs

 tests/overlay/014     | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/overlay/014.out |  3 ++
 tests/overlay/group   |  1 +
 3 files changed, 98 insertions(+)
 create mode 100755 tests/overlay/014
 create mode 100644 tests/overlay/014.out

diff --git a/tests/overlay/014 b/tests/overlay/014
new file mode 100755
index 0000000..36d7077
--- /dev/null
+++ b/tests/overlay/014
@@ -0,0 +1,94 @@
+#! /bin/bash
+# FS QA Test 014
+#
+# Test copy-up of lowerdir with opaque xattr set.
+#
+# When copying up a directory with opaque xattr set, the xattr was copied up as
+# well, after a remount the opaque flag is honored so the copied-up directory
+# is treated as opaque, which causes files created in the lowerdir to be hidden
+# when using multiple lower directories.
+#
+# This is fixed by "ovl: don't copy up opaqueness".
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Red Hat Inc. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/attr
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs overlay
+_supported_os Linux
+_require_test
+_require_scratch
+
+# Remove all files from previous tests
+_scratch_mkfs
+
+# Create multiple lowerdirs and upperdir, workdir, and testdir on lowerdir
+lowerdir1=$SCRATCH_DEV/lower1
+lowerdir2=$SCRATCH_DEV/lower2
+upperdir=$SCRATCH_DEV/upper
+workdir=$SCRATCH_DEV/workdir
+mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir
+mkdir -p $lowerdir1/testdir/d
+
+# mount overlay with $lowerdir2 as upperdir, and remove & recreate testdir,
+# make testdir on $lowerdir2 opaque
+_overlay_mount_dirs $lowerdir1 $lowerdir2 $workdir $SCRATCH_DEV $SCRATCH_MNT
+rm -rf $SCRATCH_MNT/testdir
+mkdir -p $SCRATCH_MNT/testdir/visibledir
+_scratch_unmount
+
+# mount overlay again, with lowerdir1 and lowerdir2 as multiple lowerdirs,
+# and create a new file in testdir, triggers copyup from lowerdir,
+# copyup should not copy overlayfs private xattr
+_overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
+		    $SCRATCH_DEV $SCRATCH_MNT
+touch $SCRATCH_MNT/testdir/visiblefile
+
+# umount and mount overlay again, buggy kernel treats the copied-up dir as
+# opaque, visibledir is not seen in merged dir.
+_scratch_unmount
+_overlay_mount_dirs "$lowerdir2:$lowerdir1" $upperdir $workdir \
+		    $SCRATCH_DEV $SCRATCH_MNT
+ls $SCRATCH_MNT/testdir
+
+# success, all done
+status=0
+exit
diff --git a/tests/overlay/014.out b/tests/overlay/014.out
new file mode 100644
index 0000000..ec09c4d
--- /dev/null
+++ b/tests/overlay/014.out
@@ -0,0 +1,3 @@
+QA output created by 014
+visibledir
+visiblefile
diff --git a/tests/overlay/group b/tests/overlay/group
index 5c80566..71eaadb 100644
--- a/tests/overlay/group
+++ b/tests/overlay/group
@@ -16,3 +16,4 @@
 011 auto quick
 012 auto quick
 013 auto quick
+014 auto quick copyup
-- 
2.7.4

Re: [PATCH v3 1/2] common/rc: introduce _overlay_mount_dirs helper

[Dave Chinner]

On Thu, Sep 01, 2016 at 01:06:23PM +0800, Eryu Guan wrote:
> Introduce a new _overlay_mount_dirs() helper to do the actual
> overlay mount operation and factor _overlay_scratch_mount and
> _overlay_mount code to use a single implemention of
> _overlay_mount_dirs(). Also convert all bare mount of overlay in
> tests to use it.
> 
> Suggested-by: Dave Chinner <dchinner@redhat.com>
> Signed-off-by: Eryu Guan <eguan@redhat.com>

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH v3 2/2] overlay: copy-up lowerdir which has opaque xattr set

[Dave Chinner]

On Thu, Sep 01, 2016 at 01:06:24PM +0800, Eryu Guan wrote:
> When copying up a directory with opaque xattr set, the xattr was
> copied up as well, after a remount the opaque flag is honored so the
> copied-up directory is treated as opaque, which causes files created
> in the lowerdir to be hidden when using multiple lower directories.
> 
> This is fixed by "ovl: don't copy up opaqueness".
> 
> Signed-off-by: Eryu Guan <eguan@redhat.com>

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com