* [PATCH 1/4] overlay: Add tests for nesting private xattrs
2023-11-14 6:48 [PATCH 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
@ 2023-11-14 6:48 ` Amir Goldstein
2023-11-14 6:48 ` [PATCH 2/4] overlay: prepare for new lowerdir+,datadir+ tests Amir Goldstein
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Amir Goldstein @ 2023-11-14 6:48 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
From: Alexander Larsson <alexl@redhat.com>
If overlayfs xattr escaping is supported, ensure:
* We can create "overlay.*" xattrs on a file in the overlayfs
* We can create an xwhiteout file in the overlayfs
We check for nesting support by trying to getattr an "overlay.*" xattr
in an overlayfs mount, which will return ENOSUPP in older kernels.
Signed-off-by: Alexander Larsson <alexl@redhat.com>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
common/overlay | 12 +++
tests/overlay/084 | 169 ++++++++++++++++++++++++++++++++++++++++++
tests/overlay/084.out | 61 +++++++++++++++
3 files changed, 242 insertions(+)
create mode 100755 tests/overlay/084
create mode 100644 tests/overlay/084.out
diff --git a/common/overlay b/common/overlay
index 7004187f..f6017e4e 100644
--- a/common/overlay
+++ b/common/overlay
@@ -201,6 +201,18 @@ _require_scratch_overlay_features()
_scratch_unmount
}
+_require_scratch_overlay_xattr_escapes()
+{
+ _scratch_mkfs > /dev/null 2>&1
+ _overlay_scratch_mount_dirs $OVL_BASE_SCRATCH_MNT/$OVL_LOWER $OVL_BASE_SCRATCH_MNT/$OVL_UPPER $OVL_BASE_SCRATCH_MNT/$OVL_WORK -o rw
+
+ touch $SCRATCH_MNT/file
+ (getfattr -n trusted.overlay.foo $SCRATCH_MNT/file 2>&1 | grep -q "not supported") && \
+ _notrun "xattr escaping is not supported by overlay"
+
+ _scratch_unmount
+}
+
_require_scratch_overlay_verity()
{
local lowerdirs="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER:$OVL_BASE_SCRATCH_MNT/$OVL_LOWER"
diff --git a/tests/overlay/084 b/tests/overlay/084
new file mode 100755
index 00000000..ff451f38
--- /dev/null
+++ b/tests/overlay/084
@@ -0,0 +1,169 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2023 Red Hat, Inc. All Rights Reserved.
+# Copyright (C) 2023 CTERA Networks. All Rights Reserved.
+#
+# FS QA Test No. 084
+#
+# Test advanded nesting functionallity
+#
+. ./common/preamble
+_begin_fstest auto quick nested
+
+# Override the default cleanup function.
+_cleanup()
+{
+ cd /
+ # Unmount nested mounts if things fail
+ $UMOUNT_PROG $OVL_BASE_SCRATCH_MNT/nested 2>/dev/null
+ rm -rf $tmp
+}
+
+# Import common functions.
+. ./common/filter
+. ./common/attr
+
+# real QA test starts here
+_supported_fs overlay
+# We use non-default scratch underlying overlay dirs, we need to check
+# them explicity after test.
+_require_scratch_nocheck
+_require_scratch_overlay_xattr_escapes
+
+# remove all files from previous tests
+_scratch_mkfs
+
+lowerdir=$OVL_BASE_SCRATCH_MNT/lower
+middir=$OVL_BASE_SCRATCH_MNT/mid
+upperdir=$OVL_BASE_SCRATCH_MNT/upper
+workdir=$OVL_BASE_SCRATCH_MNT/workdir
+nesteddir=$OVL_BASE_SCRATCH_MNT/nested
+
+umount_overlay()
+{
+ $UMOUNT_PROG $SCRATCH_MNT
+}
+
+test_escape()
+{
+ local prefix=$1
+
+ echo -e "\n== Check xattr escape $prefix =="
+
+ local extra_options=""
+ if [ "$prefix" == "user" ]; then
+ extra_options="-o userxattr"
+ fi
+
+ _scratch_mkfs
+ mkdir -p $lowerdir $middir $upperdir $workdir $nesteddir
+
+ _overlay_scratch_mount_dirs $lowerdir $middir $workdir $extra_options
+
+ mkdir -p $SCRATCH_MNT/layer1/dir/ $SCRATCH_MNT/layer2/dir
+
+ touch $SCRATCH_MNT/layer1/dir/file
+
+ # Make layer2/dir an opaque file
+ # Only one of these will be escaped, but both should succeed
+ setfattr -n user.overlay.opaque -v "y" $SCRATCH_MNT/layer2/dir
+ setfattr -n trusted.overlay.opaque -v "y" $SCRATCH_MNT/layer2/dir
+
+ getfattr -m "overlay\\." --absolute-names -d $SCRATCH_MNT/layer2/dir | _filter_scratch
+
+ umount_overlay
+
+ getfattr -m "overlay\\." --absolute-names -d $middir/layer2/dir | _filter_scratch
+
+ # Remount as lower and try again
+ _overlay_scratch_mount_dirs $middir:$lowerdir $upperdir $workdir $extra_options
+
+ getfattr -m "overlay\\." --absolute-names -d $SCRATCH_MNT/layer2/dir | _filter_scratch
+
+ # Recursively mount and ensure the opaque dir is working with both trusted and user xattrs
+ echo "nested xattr mount with trusted.overlay"
+ _overlay_mount_dirs $SCRATCH_MNT/layer2:$SCRATCH_MNT/layer1 - - overlayfs $nesteddir
+ stat $nesteddir/dir/file 2>&1 | _filter_scratch
+ $UMOUNT_PROG $nesteddir
+
+ echo "nested xattr mount with user.overlay"
+ _overlay_mount_dirs $SCRATCH_MNT/layer2:$SCRATCH_MNT/layer1 - - -o userxattr overlayfs $nesteddir
+ stat $nesteddir/dir/file 2>&1 | _filter_scratch
+ $UMOUNT_PROG $nesteddir
+
+ # Also ensure propagate the escaped xattr when we copy-up layer2/dir
+ echo "copy-up of escaped xattrs"
+ touch $SCRATCH_MNT/layer2/dir/other_file
+ getfattr -m "$prefix.overlay\\.overlay" --absolute-names -d $upperdir/layer2/dir | _filter_scratch
+
+ umount_overlay
+}
+
+test_escape trusted
+test_escape user
+
+do_test_xwhiteout()
+{
+ local prefix=$1
+ local basedir=$2
+
+ local extra_options=""
+ if [ "$prefix" == "user" ]; then
+ extra_options="-o userxattr"
+ fi
+
+ mkdir -p $basedir/lower $basedir/upper $basedir/work
+ touch $basedir/lower/regular $basedir/lower/hidden $basedir/upper/hidden
+ setfattr -n $prefix.overlay.whiteouts -v "y" $basedir/upper
+ setfattr -n $prefix.overlay.whiteout -v "y" $basedir/upper/hidden
+
+ # Test the hidden is invisible
+ _overlay_scratch_mount_dirs $basedir/upper:$basedir/lower - - $extra_options
+ ls $SCRATCH_MNT
+ stat $SCRATCH_MNT/hidden 2>&1 | _filter_scratch
+ umount_overlay
+}
+
+# Validate that xwhiteouts work like whiteouts
+test_xwhiteout()
+{
+ local prefix=$1
+
+ echo -e "\n== Check xwhiteout $prefix =="
+
+ _scratch_mkfs
+
+ do_test_xwhiteout $prefix $OVL_BASE_SCRATCH_MNT
+}
+
+test_xwhiteout trusted
+test_xwhiteout user
+
+# Validate that (escaped) xwhiteouts work inside a nested overlayfs mount
+test_escaped_xwhiteout()
+{
+ local prefix=$1
+
+ echo -e "\n== Check escaped xwhiteout $prefix =="
+
+ local extra_options=""
+ if [ "$prefix" == "user" ]; then
+ extra_options="-o userxattr"
+ fi
+
+ _scratch_mkfs
+ mkdir -p $lowerdir $upperdir $workdir $nesteddir
+
+ _overlay_mount_dirs $lowerdir $upperdir $workdir $extra_options overlayfs $nesteddir
+
+ do_test_xwhiteout $prefix $nesteddir
+
+ $UMOUNT_PROG $nesteddir
+}
+
+test_escaped_xwhiteout trusted
+test_escaped_xwhiteout user
+
+# success, all done
+status=0
+exit
diff --git a/tests/overlay/084.out b/tests/overlay/084.out
new file mode 100644
index 00000000..54b890de
--- /dev/null
+++ b/tests/overlay/084.out
@@ -0,0 +1,61 @@
+QA output created by 084
+
+== Check xattr escape trusted ==
+# file: SCRATCH_MNT/layer2/dir
+trusted.overlay.opaque="y"
+user.overlay.opaque="y"
+
+# file: SCRATCH_DEV/mid/layer2/dir
+trusted.overlay.overlay.opaque="y"
+user.overlay.opaque="y"
+
+# file: SCRATCH_MNT/layer2/dir
+trusted.overlay.opaque="y"
+user.overlay.opaque="y"
+
+nested xattr mount with trusted.overlay
+stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
+nested xattr mount with user.overlay
+stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
+copy-up of escaped xattrs
+# file: SCRATCH_DEV/upper/layer2/dir
+trusted.overlay.overlay.opaque="y"
+
+
+== Check xattr escape user ==
+# file: SCRATCH_MNT/layer2/dir
+trusted.overlay.opaque="y"
+user.overlay.opaque="y"
+
+# file: SCRATCH_DEV/mid/layer2/dir
+trusted.overlay.opaque="y"
+user.overlay.overlay.opaque="y"
+
+# file: SCRATCH_MNT/layer2/dir
+trusted.overlay.opaque="y"
+user.overlay.opaque="y"
+
+nested xattr mount with trusted.overlay
+stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
+nested xattr mount with user.overlay
+stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
+copy-up of escaped xattrs
+# file: SCRATCH_DEV/upper/layer2/dir
+user.overlay.overlay.opaque="y"
+
+
+== Check xwhiteout trusted ==
+regular
+stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
+
+== Check xwhiteout user ==
+regular
+stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
+
+== Check escaped xwhiteout trusted ==
+regular
+stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
+
+== Check escaped xwhiteout user ==
+regular
+stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 2/4] overlay: prepare for new lowerdir+,datadir+ tests
2023-11-14 6:48 [PATCH 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
2023-11-14 6:48 ` [PATCH 1/4] overlay: Add tests for nesting private xattrs Amir Goldstein
@ 2023-11-14 6:48 ` Amir Goldstein
2023-11-14 6:48 ` [PATCH 3/4] overlay: test data-only lowerdirs with datadir+ mount option Amir Goldstein
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Amir Goldstein @ 2023-11-14 6:48 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
In preparation to forking tests for new lowerdir+,datadir+ mount options,
prepare a helper to test kernel support and pass datadirs into mount
helpers in overlay/079 test.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
common/overlay | 15 +++++++++++++++
tests/overlay/079 | 36 +++++++++++++++++++++---------------
2 files changed, 36 insertions(+), 15 deletions(-)
diff --git a/common/overlay b/common/overlay
index f6017e4e..028019ce 100644
--- a/common/overlay
+++ b/common/overlay
@@ -240,6 +240,21 @@ _require_scratch_overlay_lowerdata_layers()
_scratch_unmount
}
+# Check kernel support for lowerdir+=<lowerdir>,datadir+=<lowerdatadir> format
+_require_scratch_overlay_lowerdir_add_layers()
+{
+ local lowerdir="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER"
+ local datadir="$OVL_BASE_SCRATCH_MNT/$OVL_LOWER"
+
+ _scratch_mkfs > /dev/null 2>&1
+ $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+ -o"lowerdir+=$lowerdir,datadir+=$datadir" \
+ -o"redirect_dir=follow,metacopy=on" > /dev/null 2>&1 || \
+ _notrun "overlay lowerdir+,datadir+ not supported on ${SCRATCH_DEV}"
+
+ _scratch_unmount
+}
+
# Helper function to check underlying dirs of overlay filesystem
_overlay_fsck_dirs()
{
diff --git a/tests/overlay/079 b/tests/overlay/079
index 77f94598..078ee816 100755
--- a/tests/overlay/079
+++ b/tests/overlay/079
@@ -139,16 +139,21 @@ check_file_size_contents()
mount_overlay()
{
- local _lowerdir=$1
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
- _overlay_scratch_mount_dirs "$_lowerdir" $upperdir $workdir -o redirect_dir=on,index=on,metacopy=on
+ $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+ -o"lowerdir=$_lowerdir::$_datadir2::$_datadir" \
+ -o"upperdir=$upperdir,workdir=$workdir" \
+ -o redirect_dir=on,metacopy=on
}
mount_ro_overlay()
{
- local _lowerdir=$1
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
- _overlay_scratch_mount_dirs "$_lowerdir" "-" "-" -o ro,redirect_dir=follow,metacopy=on
+ $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+ -o"lowerdir=$_lowerdir::$_datadir2::$_datadir" \
+ -o redirect_dir=follow,metacopy=on
}
umount_overlay()
@@ -160,14 +165,14 @@ test_no_access()
{
local _target=$1
- mount_ro_overlay "$lowerdir::$datadir2::$datadir"
+ mount_ro_overlay "$lowerdir" "$datadir2" "$datadir"
stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \
echo "No access to lowerdata layer $_target"
echo "Unmount and Mount rw"
umount_overlay
- mount_overlay "$lowerdir::$datadir2::$datadir"
+ mount_overlay "$lowerdir" "$datadir2" "$datadir"
stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \
echo "No access to lowerdata layer $_target"
umount_overlay
@@ -175,11 +180,12 @@ test_no_access()
test_common()
{
- local _lowerdirs=$1 _target=$2 _size=$3 _blocks=$4 _data="$5"
- local _redirect=$6
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
+ local _target=$4 _size=$5 _blocks=$6 _data="$7"
+ local _redirect=$8
echo "Mount ro"
- mount_ro_overlay $_lowerdirs
+ mount_ro_overlay $_lowerdir $_datadir2 $_datadir
# Check redirect xattr to lowerdata
[ -n "$_redirect" ] && check_redirect $lowerdir/$_target "$_redirect"
@@ -191,7 +197,7 @@ test_common()
# Do a mount cycle and check size and contents again.
echo "Unmount and Mount rw"
umount_overlay
- mount_overlay $_lowerdirs
+ mount_overlay $_lowerdir $_datadir2 $_datadir
echo "check properties of metadata copied up file $_target"
check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
check_file_blocks $SCRATCH_MNT/$_target $_blocks
@@ -203,7 +209,7 @@ test_common()
check_file_size_contents $upperdir/$_target $_size ""
# Trigger data copy up and check absence of metacopy xattr.
- mount_overlay $_lowerdirs
+ mount_overlay $_lowerdir $_datadir2 $_datadir
$XFS_IO_PROG -c "falloc 0 1" $SCRATCH_MNT/$_target >> $seqres.full
echo "check properties of data copied up file $_target"
check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
@@ -216,7 +222,7 @@ test_lazy()
{
local _target=$1
- mount_overlay "$lowerdir::$datadir2::$datadir"
+ mount_overlay "$lowerdir" "$datadir2" "$datadir"
# Metadata should be valid
check_file_size $SCRATCH_MNT/$_target $datasize
@@ -305,12 +311,12 @@ test_no_access "$sharedname"
echo -e "\n== Check follow to lowerdata layer with absolute redirect =="
prepare_midlayer "/subdir/$dataname"
-test_common "$lowerdir::$datadir2::$datadir" "$dataname" $datasize $datablocks \
+test_common "$lowerdir" "$datadir2" "$datadir" "$dataname" $datasize $datablocks \
"$datacontent" "/subdir/$dataname"
-test_common "$lowerdir::$datadir2::$datadir" "$dataname2" $datasize $datablocks \
+test_common "$lowerdir" "$datadir2" "$datadir" "$dataname2" $datasize $datablocks \
"$datacontent2" "/subdir/$dataname.2"
# Shared file should be picked from upper datadir
-test_common "$lowerdir::$datadir2::$datadir" "$sharedname" $datasize $datablocks \
+test_common "$lowerdir" "$datadir2" "$datadir" "$sharedname" $datasize $datablocks \
"$datacontent2" "/subdir/$dataname.shared"
echo -e "\n== Check lazy follow to lowerdata layer =="
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 3/4] overlay: test data-only lowerdirs with datadir+ mount option
2023-11-14 6:48 [PATCH 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
2023-11-14 6:48 ` [PATCH 1/4] overlay: Add tests for nesting private xattrs Amir Goldstein
2023-11-14 6:48 ` [PATCH 2/4] overlay: prepare for new lowerdir+,datadir+ tests Amir Goldstein
@ 2023-11-14 6:48 ` Amir Goldstein
2023-11-14 6:48 ` [PATCH 4/4] overlay: test parsing of lowerdir+,datadir+ mount options Amir Goldstein
2023-12-04 14:31 ` [PATCH 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
4 siblings, 0 replies; 6+ messages in thread
From: Amir Goldstein @ 2023-11-14 6:48 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
Fork test overlay/079 to use the new lowerdir+,datadir+ mount options.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
tests/overlay/085 | 332 ++++++++++++++++++++++++++++++++++++++++++
tests/overlay/085.out | 42 ++++++
2 files changed, 374 insertions(+)
create mode 100755 tests/overlay/085
create mode 100644 tests/overlay/085.out
diff --git a/tests/overlay/085 b/tests/overlay/085
new file mode 100755
index 00000000..07a32c24
--- /dev/null
+++ b/tests/overlay/085
@@ -0,0 +1,332 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2018 Red Hat, Inc. All Rights Reserved.
+# Copyright (C) 2023 CTERA Networks. All Rights Reserved.
+#
+# FS QA Test No. 085
+#
+# Test data-only layers functionality.
+# This is a variant of test overlay/079 with lowerdir+,datadir+ mount options
+#
+. ./common/preamble
+_begin_fstest auto quick metacopy redirect prealloc
+
+# Import common functions.
+. ./common/filter
+. ./common/attr
+
+# real QA test starts here
+_supported_fs overlay
+# We use non-default scratch underlying overlay dirs, we need to check
+# them explicity after test.
+_require_scratch_nocheck
+_require_scratch_overlay_features redirect_dir metacopy
+_require_scratch_overlay_lowerdir_add_layers
+_require_xfs_io_command "falloc"
+
+# remove all files from previous tests
+_scratch_mkfs
+
+# File size on lower
+dataname="datafile"
+sharedname="shared"
+datacontent="data"
+dataname2="datafile2"
+datacontent2="data2"
+datasize="4096"
+
+# Number of blocks allocated by filesystem on lower. Will be queried later.
+datarblocks=""
+datarblocksize=""
+estimated_datablocks=""
+
+udirname="pureupper"
+ufile="upperfile"
+
+# Check metacopy xattr
+check_metacopy()
+{
+ local target=$1 exist=$2
+ local out_f target_f
+ local msg
+
+ out_f=$(_getfattr --absolute-names --only-values -n \
+ $OVL_XATTR_METACOPY $target 2>&1 | _filter_scratch)
+
+ if [ "$exist" == "y" ];then
+ [ "$out_f" == "" ] && return
+ echo "Metacopy xattr does not exist on ${target}. stdout=$out_f"
+ return
+ fi
+
+ if [ "$out_f" == "" ];then
+ echo "Metacopy xattr exists on ${target} unexpectedly."
+ return
+ fi
+
+ target_f=`echo $target | _filter_scratch`
+ msg="$target_f: trusted.overlay.metacopy: No such attribute"
+
+ [ "$out_f" == "$msg" ] && return
+
+ echo "Error while checking xattr on ${target}. stdout=$out"
+}
+
+# Check redirect xattr
+check_redirect()
+{
+ local target=$1
+ local expect=$2
+
+ value=$(_getfattr --absolute-names --only-values -n \
+ $OVL_XATTR_REDIRECT $target)
+
+ [[ "$value" == "$expect" ]] || echo "Redirect xattr incorrect. Expected=\"$expect\", actual=\"$value\""
+}
+
+# Check size
+check_file_size()
+{
+ local target=$1 expected_size=$2 actual_size
+
+ actual_size=$(_get_filesize $target)
+
+ [ "$actual_size" == "$expected_size" ] || echo "Expected file size $expected_size but actual size is $actual_size"
+}
+
+check_file_blocks()
+{
+ local target=$1 expected_blocks=$2 nr_blocks
+
+ nr_blocks=$(stat -c "%b" $target)
+
+ [ "$nr_blocks" == "$expected_blocks" ] || echo "Expected $expected_blocks blocks but actual number of blocks is ${nr_blocks}."
+}
+
+check_file_contents()
+{
+ local target=$1 expected=$2
+ local actual target_f
+
+ target_f=`echo $target | _filter_scratch`
+
+ read actual<$target
+
+ [ "$actual" == "$expected" ] || echo "Expected file $target_f contents to be \"$expected\" but actual contents are \"$actual\""
+}
+
+check_no_file_contents()
+{
+ local target=$1
+ local actual target_f out_f
+
+ target_f=`echo $target | _filter_scratch`
+ out_f=`cat $target 2>&1 | _filter_scratch`
+ msg="cat: $target_f: No such file or directory"
+
+ [ "$out_f" == "$msg" ] && return
+
+ echo "$target_f unexpectedly has content"
+}
+
+
+check_file_size_contents()
+{
+ local target=$1 expected_size=$2 expected_content=$3
+
+ check_file_size $target $expected_size
+ check_file_contents $target $expected_content
+}
+
+mount_overlay()
+{
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
+
+ $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+ -o"lowerdir+=$_lowerdir,datadir+=$_datadir2,datadir+=$_datadir" \
+ -o"upperdir=$upperdir,workdir=$workdir" \
+ -o redirect_dir=on,metacopy=on
+}
+
+mount_ro_overlay()
+{
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
+
+ $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+ -o"lowerdir+=$_lowerdir,datadir+=$_datadir2,datadir+=$_datadir" \
+ -o redirect_dir=follow,metacopy=on
+}
+
+umount_overlay()
+{
+ $UMOUNT_PROG $SCRATCH_MNT
+}
+
+test_no_access()
+{
+ local _target=$1
+
+ mount_ro_overlay "$lowerdir" "$datadir2" "$datadir"
+
+ stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \
+ echo "No access to lowerdata layer $_target"
+
+ echo "Unmount and Mount rw"
+ umount_overlay
+ mount_overlay "$lowerdir" "$datadir2" "$datadir"
+ stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \
+ echo "No access to lowerdata layer $_target"
+ umount_overlay
+}
+
+test_common()
+{
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
+ local _target=$4 _size=$5 _blocks=$6 _data="$7"
+ local _redirect=$8
+
+ echo "Mount ro"
+ mount_ro_overlay $_lowerdir $_datadir2 $_datadir
+
+ # Check redirect xattr to lowerdata
+ [ -n "$_redirect" ] && check_redirect $lowerdir/$_target "$_redirect"
+
+ echo "check properties of metadata copied up file $_target"
+ check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
+ check_file_blocks $SCRATCH_MNT/$_target $_blocks
+
+ # Do a mount cycle and check size and contents again.
+ echo "Unmount and Mount rw"
+ umount_overlay
+ mount_overlay $_lowerdir $_datadir2 $_datadir
+ echo "check properties of metadata copied up file $_target"
+ check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
+ check_file_blocks $SCRATCH_MNT/$_target $_blocks
+
+ # Trigger metadata copy up and check existence of metacopy xattr.
+ chmod 400 $SCRATCH_MNT/$_target
+ umount_overlay
+ check_metacopy $upperdir/$_target "y"
+ check_file_size_contents $upperdir/$_target $_size ""
+
+ # Trigger data copy up and check absence of metacopy xattr.
+ mount_overlay $_lowerdir $_datadir2 $_datadir
+ $XFS_IO_PROG -c "falloc 0 1" $SCRATCH_MNT/$_target >> $seqres.full
+ echo "check properties of data copied up file $_target"
+ check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
+ umount_overlay
+ check_metacopy $upperdir/$_target "n"
+ check_file_size_contents $upperdir/$_target $_size "$_data"
+}
+
+test_lazy()
+{
+ local _target=$1
+
+ mount_overlay "$lowerdir" "$datadir2" "$datadir"
+
+ # Metadata should be valid
+ check_file_size $SCRATCH_MNT/$_target $datasize
+ check_file_blocks $SCRATCH_MNT/$_target $estimated_datablocks
+
+ # But have no content
+ check_no_file_contents $SCRATCH_MNT/$_target
+
+ umount_overlay
+}
+
+create_basic_files()
+{
+ _scratch_mkfs
+ mkdir -p $datadir/subdir $datadir2/subdir $lowerdir $lowerdir2 $upperdir $workdir $workdir2
+ mkdir -p $upperdir/$udirname
+ echo "$datacontent" > $datadir/$dataname
+ chmod 600 $datadir/$dataname
+ echo "$datacontent2" > $datadir2/$dataname2
+ chmod 600 $datadir2/$dataname2
+
+ echo "$datacontent" > $datadir/$sharedname
+ echo "$datacontent2" > $datadir2/$sharedname
+ chmod 600 $datadir/$sharedname $datadir2/$sharedname
+
+ # Create files of size datasize.
+ for f in $datadir/$dataname $datadir2/$dataname2 $datadir/$sharedname $datadir2/$sharedname; do
+ $XFS_IO_PROG -c "falloc 0 $datasize" $f
+ $XFS_IO_PROG -c "fsync" $f
+ done
+
+ # Query number of block
+ datablocks=$(stat -c "%b" $datadir/$dataname)
+
+ # For lazy lookup file the block count is estimated based on size and block size
+ datablocksize=$(stat -c "%B" $datadir/$dataname)
+ estimated_datablocks=$(( ($datasize + $datablocksize - 1)/$datablocksize ))
+}
+
+prepare_midlayer()
+{
+ local _redirect=$1
+
+ _scratch_mkfs
+ create_basic_files
+ if [ -n "$_redirect" ]; then
+ mv "$datadir/$dataname" "$datadir/$_redirect"
+ mv "$datadir2/$dataname2" "$datadir2/$_redirect.2"
+ mv "$datadir/$sharedname" "$datadir/$_redirect.shared"
+ mv "$datadir2/$sharedname" "$datadir2/$_redirect.shared"
+ fi
+ # Create midlayer
+ _overlay_scratch_mount_dirs $datadir2:$datadir $lowerdir $workdir2 -o redirect_dir=on,index=on,metacopy=on
+ # Trigger a metacopy with or without redirect
+ if [ -n "$_redirect" ]; then
+ mv "$SCRATCH_MNT/$_redirect" "$SCRATCH_MNT/$dataname"
+ mv "$SCRATCH_MNT/$_redirect.2" "$SCRATCH_MNT/$dataname2"
+ mv "$SCRATCH_MNT/$_redirect.shared" "$SCRATCH_MNT/$sharedname"
+ else
+ chmod 400 $SCRATCH_MNT/$dataname
+ chmod 400 $SCRATCH_MNT/$dataname2
+ chmod 400 $SCRATCH_MNT/$sharedname
+ fi
+ umount_overlay
+}
+
+# Create test directories
+datadir=$OVL_BASE_SCRATCH_MNT/data
+datadir2=$OVL_BASE_SCRATCH_MNT/data2
+lowerdir=$OVL_BASE_SCRATCH_MNT/lower
+upperdir=$OVL_BASE_SCRATCH_MNT/upper
+workdir=$OVL_BASE_SCRATCH_MNT/workdir
+workdir2=$OVL_BASE_SCRATCH_MNT/workdir2
+
+echo -e "\n== Check no follow to lowerdata layer without redirect =="
+prepare_midlayer
+test_no_access "$dataname"
+test_no_access "$dataname2"
+test_no_access "$sharedname"
+
+echo -e "\n== Check no follow to lowerdata layer with relative redirect =="
+prepare_midlayer "$dataname.renamed"
+test_no_access "$dataname"
+test_no_access "$dataname2"
+test_no_access "$sharedname"
+
+echo -e "\n== Check follow to lowerdata layer with absolute redirect =="
+prepare_midlayer "/subdir/$dataname"
+test_common "$lowerdir" "$datadir2" "$datadir" "$dataname" $datasize $datablocks \
+ "$datacontent" "/subdir/$dataname"
+test_common "$lowerdir" "$datadir2" "$datadir" "$dataname2" $datasize $datablocks \
+ "$datacontent2" "/subdir/$dataname.2"
+# Shared file should be picked from upper datadir
+test_common "$lowerdir" "$datadir2" "$datadir" "$sharedname" $datasize $datablocks \
+ "$datacontent2" "/subdir/$dataname.shared"
+
+echo -e "\n== Check lazy follow to lowerdata layer =="
+
+prepare_midlayer "/subdir/$dataname"
+rm $datadir/subdir/$dataname
+test_lazy $dataname
+
+
+# success, all done
+status=0
+exit
diff --git a/tests/overlay/085.out b/tests/overlay/085.out
new file mode 100644
index 00000000..4b9b2d7c
--- /dev/null
+++ b/tests/overlay/085.out
@@ -0,0 +1,42 @@
+QA output created by 085
+
+== Check no follow to lowerdata layer without redirect ==
+No access to lowerdata layer datafile
+Unmount and Mount rw
+No access to lowerdata layer datafile
+No access to lowerdata layer datafile2
+Unmount and Mount rw
+No access to lowerdata layer datafile2
+No access to lowerdata layer shared
+Unmount and Mount rw
+No access to lowerdata layer shared
+
+== Check no follow to lowerdata layer with relative redirect ==
+No access to lowerdata layer datafile
+Unmount and Mount rw
+No access to lowerdata layer datafile
+No access to lowerdata layer datafile2
+Unmount and Mount rw
+No access to lowerdata layer datafile2
+No access to lowerdata layer shared
+Unmount and Mount rw
+No access to lowerdata layer shared
+
+== Check follow to lowerdata layer with absolute redirect ==
+Mount ro
+check properties of metadata copied up file datafile
+Unmount and Mount rw
+check properties of metadata copied up file datafile
+check properties of data copied up file datafile
+Mount ro
+check properties of metadata copied up file datafile2
+Unmount and Mount rw
+check properties of metadata copied up file datafile2
+check properties of data copied up file datafile2
+Mount ro
+check properties of metadata copied up file shared
+Unmount and Mount rw
+check properties of metadata copied up file shared
+check properties of data copied up file shared
+
+== Check lazy follow to lowerdata layer ==
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 4/4] overlay: test parsing of lowerdir+,datadir+ mount options
2023-11-14 6:48 [PATCH 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
` (2 preceding siblings ...)
2023-11-14 6:48 ` [PATCH 3/4] overlay: test data-only lowerdirs with datadir+ mount option Amir Goldstein
@ 2023-11-14 6:48 ` Amir Goldstein
2023-12-04 14:31 ` [PATCH 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
4 siblings, 0 replies; 6+ messages in thread
From: Amir Goldstein @ 2023-11-14 6:48 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
Fork test overlay/083 to test parsing of lowerdir+,datadir+ mount options.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
tests/overlay/086 | 81 +++++++++++++++++++++++++++++++++++++++++++
tests/overlay/086.out | 2 ++
2 files changed, 83 insertions(+)
create mode 100755 tests/overlay/086
create mode 100644 tests/overlay/086.out
diff --git a/tests/overlay/086 b/tests/overlay/086
new file mode 100755
index 00000000..b5960517
--- /dev/null
+++ b/tests/overlay/086
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2023 CTERA Networks. All Rights Reserved.
+#
+# FS QA Test 086
+#
+# Test lowerdir+,datadir+ mount option restrictions.
+#
+
+. ./common/preamble
+_begin_fstest auto quick mount
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+_supported_fs overlay
+
+# _overlay_check_* helpers do not handle special chars well
+_require_scratch_nocheck
+_require_scratch_overlay_lowerdir_add_layers
+
+# Remove all files from previous tests
+_scratch_mkfs
+
+# Create lowerdirs with special characters
+lowerdir_spaces="$OVL_BASE_SCRATCH_MNT/lower1 with spaces"
+lowerdir_colons="$OVL_BASE_SCRATCH_MNT/lower2:with::colons"
+lowerdir_colons_esc="$OVL_BASE_SCRATCH_MNT/lower2\:with\:\:colons"
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+workdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
+mkdir -p "$lowerdir_spaces" "$lowerdir_colons"
+
+# _overlay_mount_* helpers do not handle lowerdir+,datadir+, so execute mount directly.
+
+# check illegal combinations and order of lowerdir,lowerdir+,datadir+
+$MOUNT_PROG -t overlay none $SCRATCH_MNT \
+ -o"lowerdir=$lowerdir,lowerdir+=$lowerdir_colons" \
+ 2>> $seqres.full && \
+ echo "ERROR: invalid combination of lowerdir and lowerdir+ mount options"
+
+$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
+
+$MOUNT_PROG -t overlay none $SCRATCH_MNT \
+ -o"lowerdir=$lowerdir,datadir+=$lowerdir_colons" \
+ -o redirect_dir=follow,metacopy=on 2>> $seqres.full && \
+ echo "ERROR: invalid combination of lowerdir and datadir+ mount options"
+
+$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
+
+$MOUNT_PROG -t overlay none $SCRATCH_MNT \
+ -o"datadir+=$lowerdir,lowerdir+=$lowerdir_colons" \
+ -o redirect_dir=follow,metacopy=on 2>> $seqres.full && \
+ echo "ERROR: invalid order of lowerdir+ and datadir+ mount options"
+
+$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
+
+# mount is expected to fail with escaped colons.
+$MOUNT_PROG -t overlay none $SCRATCH_MNT \
+ -o"lowerdir+=$lowerdir_colons_esc" \
+ 2>> $seqres.full && \
+ echo "ERROR: incorrect parsing of escaped colons in lowerdir+ mount option"
+
+$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
+
+# mount is expected to succeed without escaped colons.
+$MOUNT_PROG -t overlay ovl_esc_test $SCRATCH_MNT \
+ -o"lowerdir+=$lowerdir_colons,datadir+=$lowerdir_spaces" \
+ -o redirect_dir=follow,metacopy=on \
+ 2>&1 | tee -a $seqres.full
+
+# if spaces are not escaped when showing mount options,
+# mount command will not show the word 'spaces' after the spaces
+$MOUNT_PROG -t overlay | grep ovl_esc_test | tee -a $seqres.full | \
+ grep -q 'datadir+'.*spaces || \
+ echo "ERROR: escaped spaces truncated from datadir+ mount option"
+
+echo "Silence is golden"
+status=0
+exit
diff --git a/tests/overlay/086.out b/tests/overlay/086.out
new file mode 100644
index 00000000..b34758fd
--- /dev/null
+++ b/tests/overlay/086.out
@@ -0,0 +1,2 @@
+QA output created by 086
+Silence is golden
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 0/4] Overlayfs tests for 6.7-rc1
2023-11-14 6:48 [PATCH 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
` (3 preceding siblings ...)
2023-11-14 6:48 ` [PATCH 4/4] overlay: test parsing of lowerdir+,datadir+ mount options Amir Goldstein
@ 2023-12-04 14:31 ` Amir Goldstein
4 siblings, 0 replies; 6+ messages in thread
From: Amir Goldstein @ 2023-12-04 14:31 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
On Tue, Nov 14, 2023 at 8:49 AM Amir Goldstein <amir73il@gmail.com> wrote:
>
> Zorro,
>
> This update contains 3 new overlayfs tests for new features added
> in v6.7-rc1.
>
> overlay/084, written by Alexander, tests the new nested xattrs feature.
> overlay/{085,086} test the new lowerdir+,datadir+ mount options.
>
> overlay/086 was partly forked from overlay/083, but overlay/083 is not
> sensitive to libmount version, because the escaped commas test is not
> related to any specific mount option, so it wasn't copied over.
>
> All the new tests do not run on older kernels.
>
Ping.
> Thanks,
> Amir.
>
> Alexander Larsson (1):
> overlay: Add tests of nesting
>
> Amir Goldstein (3):
> overlay: prepare for new lowerdir+,datadir+ tests
> overlay: test data-only lowerdirs with datadir+ mount option
> overlay: test parsing of lowerdir+,datadir+ mount options
>
> common/overlay | 27 ++++
> tests/overlay/079 | 36 +++--
> tests/overlay/084 | 169 +++++++++++++++++++++
> tests/overlay/084.out | 61 ++++++++
> tests/overlay/085 | 332 ++++++++++++++++++++++++++++++++++++++++++
> tests/overlay/085.out | 42 ++++++
> tests/overlay/086 | 81 +++++++++++
> tests/overlay/086.out | 2 +
> 8 files changed, 735 insertions(+), 15 deletions(-)
> create mode 100755 tests/overlay/084
> create mode 100644 tests/overlay/084.out
> create mode 100755 tests/overlay/085
> create mode 100644 tests/overlay/085.out
> create mode 100755 tests/overlay/086
> create mode 100644 tests/overlay/086.out
>
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 6+ messages in thread