* [PATCH v2 1/4] overlay: Add tests for nesting private xattrs
2023-12-04 18:58 [PATCH v2 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
@ 2023-12-04 18:58 ` Amir Goldstein
2023-12-10 13:35 ` Zorro Lang
2023-12-04 18:58 ` [PATCH v2 2/4] overlay: prepare for new lowerdir+,datadir+ tests Amir Goldstein
` (3 subsequent siblings)
4 siblings, 1 reply; 13+ messages in thread
From: Amir Goldstein @ 2023-12-04 18:58 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
If overlayfs xattr escaping is supported, ensure:
* We can create "overlay.*" xattrs on a file in the overlayfs
* We can create an xwhiteout file in the overlayfs
We check for nesting support by trying to getattr an "overlay.*" xattr
in an overlayfs mount, which will return ENOSUPP in older kernels.
Signed-off-by: Alexander Larsson <alexl@redhat.com>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
tests/overlay/084 | 169 ++++++++++++++++++++++++++++++++++++++++++
tests/overlay/084.out | 61 +++++++++++++++
2 files changed, 230 insertions(+)
create mode 100755 tests/overlay/084
create mode 100644 tests/overlay/084.out
diff --git a/tests/overlay/084 b/tests/overlay/084
new file mode 100755
index 00000000..ff451f38
--- /dev/null
+++ b/tests/overlay/084
@@ -0,0 +1,169 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2023 Red Hat, Inc. All Rights Reserved.
+# Copyright (C) 2023 CTERA Networks. All Rights Reserved.
+#
+# FS QA Test No. 084
+#
+# Test advanded nesting functionallity
+#
+. ./common/preamble
+_begin_fstest auto quick nested
+
+# Override the default cleanup function.
+_cleanup()
+{
+ cd /
+ # Unmount nested mounts if things fail
+ $UMOUNT_PROG $OVL_BASE_SCRATCH_MNT/nested 2>/dev/null
+ rm -rf $tmp
+}
+
+# Import common functions.
+. ./common/filter
+. ./common/attr
+
+# real QA test starts here
+_supported_fs overlay
+# We use non-default scratch underlying overlay dirs, we need to check
+# them explicity after test.
+_require_scratch_nocheck
+_require_scratch_overlay_xattr_escapes
+
+# remove all files from previous tests
+_scratch_mkfs
+
+lowerdir=$OVL_BASE_SCRATCH_MNT/lower
+middir=$OVL_BASE_SCRATCH_MNT/mid
+upperdir=$OVL_BASE_SCRATCH_MNT/upper
+workdir=$OVL_BASE_SCRATCH_MNT/workdir
+nesteddir=$OVL_BASE_SCRATCH_MNT/nested
+
+umount_overlay()
+{
+ $UMOUNT_PROG $SCRATCH_MNT
+}
+
+test_escape()
+{
+ local prefix=$1
+
+ echo -e "\n== Check xattr escape $prefix =="
+
+ local extra_options=""
+ if [ "$prefix" == "user" ]; then
+ extra_options="-o userxattr"
+ fi
+
+ _scratch_mkfs
+ mkdir -p $lowerdir $middir $upperdir $workdir $nesteddir
+
+ _overlay_scratch_mount_dirs $lowerdir $middir $workdir $extra_options
+
+ mkdir -p $SCRATCH_MNT/layer1/dir/ $SCRATCH_MNT/layer2/dir
+
+ touch $SCRATCH_MNT/layer1/dir/file
+
+ # Make layer2/dir an opaque file
+ # Only one of these will be escaped, but both should succeed
+ setfattr -n user.overlay.opaque -v "y" $SCRATCH_MNT/layer2/dir
+ setfattr -n trusted.overlay.opaque -v "y" $SCRATCH_MNT/layer2/dir
+
+ getfattr -m "overlay\\." --absolute-names -d $SCRATCH_MNT/layer2/dir | _filter_scratch
+
+ umount_overlay
+
+ getfattr -m "overlay\\." --absolute-names -d $middir/layer2/dir | _filter_scratch
+
+ # Remount as lower and try again
+ _overlay_scratch_mount_dirs $middir:$lowerdir $upperdir $workdir $extra_options
+
+ getfattr -m "overlay\\." --absolute-names -d $SCRATCH_MNT/layer2/dir | _filter_scratch
+
+ # Recursively mount and ensure the opaque dir is working with both trusted and user xattrs
+ echo "nested xattr mount with trusted.overlay"
+ _overlay_mount_dirs $SCRATCH_MNT/layer2:$SCRATCH_MNT/layer1 - - overlayfs $nesteddir
+ stat $nesteddir/dir/file 2>&1 | _filter_scratch
+ $UMOUNT_PROG $nesteddir
+
+ echo "nested xattr mount with user.overlay"
+ _overlay_mount_dirs $SCRATCH_MNT/layer2:$SCRATCH_MNT/layer1 - - -o userxattr overlayfs $nesteddir
+ stat $nesteddir/dir/file 2>&1 | _filter_scratch
+ $UMOUNT_PROG $nesteddir
+
+ # Also ensure propagate the escaped xattr when we copy-up layer2/dir
+ echo "copy-up of escaped xattrs"
+ touch $SCRATCH_MNT/layer2/dir/other_file
+ getfattr -m "$prefix.overlay\\.overlay" --absolute-names -d $upperdir/layer2/dir | _filter_scratch
+
+ umount_overlay
+}
+
+test_escape trusted
+test_escape user
+
+do_test_xwhiteout()
+{
+ local prefix=$1
+ local basedir=$2
+
+ local extra_options=""
+ if [ "$prefix" == "user" ]; then
+ extra_options="-o userxattr"
+ fi
+
+ mkdir -p $basedir/lower $basedir/upper $basedir/work
+ touch $basedir/lower/regular $basedir/lower/hidden $basedir/upper/hidden
+ setfattr -n $prefix.overlay.whiteouts -v "y" $basedir/upper
+ setfattr -n $prefix.overlay.whiteout -v "y" $basedir/upper/hidden
+
+ # Test the hidden is invisible
+ _overlay_scratch_mount_dirs $basedir/upper:$basedir/lower - - $extra_options
+ ls $SCRATCH_MNT
+ stat $SCRATCH_MNT/hidden 2>&1 | _filter_scratch
+ umount_overlay
+}
+
+# Validate that xwhiteouts work like whiteouts
+test_xwhiteout()
+{
+ local prefix=$1
+
+ echo -e "\n== Check xwhiteout $prefix =="
+
+ _scratch_mkfs
+
+ do_test_xwhiteout $prefix $OVL_BASE_SCRATCH_MNT
+}
+
+test_xwhiteout trusted
+test_xwhiteout user
+
+# Validate that (escaped) xwhiteouts work inside a nested overlayfs mount
+test_escaped_xwhiteout()
+{
+ local prefix=$1
+
+ echo -e "\n== Check escaped xwhiteout $prefix =="
+
+ local extra_options=""
+ if [ "$prefix" == "user" ]; then
+ extra_options="-o userxattr"
+ fi
+
+ _scratch_mkfs
+ mkdir -p $lowerdir $upperdir $workdir $nesteddir
+
+ _overlay_mount_dirs $lowerdir $upperdir $workdir $extra_options overlayfs $nesteddir
+
+ do_test_xwhiteout $prefix $nesteddir
+
+ $UMOUNT_PROG $nesteddir
+}
+
+test_escaped_xwhiteout trusted
+test_escaped_xwhiteout user
+
+# success, all done
+status=0
+exit
diff --git a/tests/overlay/084.out b/tests/overlay/084.out
new file mode 100644
index 00000000..54b890de
--- /dev/null
+++ b/tests/overlay/084.out
@@ -0,0 +1,61 @@
+QA output created by 084
+
+== Check xattr escape trusted ==
+# file: SCRATCH_MNT/layer2/dir
+trusted.overlay.opaque="y"
+user.overlay.opaque="y"
+
+# file: SCRATCH_DEV/mid/layer2/dir
+trusted.overlay.overlay.opaque="y"
+user.overlay.opaque="y"
+
+# file: SCRATCH_MNT/layer2/dir
+trusted.overlay.opaque="y"
+user.overlay.opaque="y"
+
+nested xattr mount with trusted.overlay
+stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
+nested xattr mount with user.overlay
+stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
+copy-up of escaped xattrs
+# file: SCRATCH_DEV/upper/layer2/dir
+trusted.overlay.overlay.opaque="y"
+
+
+== Check xattr escape user ==
+# file: SCRATCH_MNT/layer2/dir
+trusted.overlay.opaque="y"
+user.overlay.opaque="y"
+
+# file: SCRATCH_DEV/mid/layer2/dir
+trusted.overlay.opaque="y"
+user.overlay.overlay.opaque="y"
+
+# file: SCRATCH_MNT/layer2/dir
+trusted.overlay.opaque="y"
+user.overlay.opaque="y"
+
+nested xattr mount with trusted.overlay
+stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
+nested xattr mount with user.overlay
+stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
+copy-up of escaped xattrs
+# file: SCRATCH_DEV/upper/layer2/dir
+user.overlay.overlay.opaque="y"
+
+
+== Check xwhiteout trusted ==
+regular
+stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
+
+== Check xwhiteout user ==
+regular
+stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
+
+== Check escaped xwhiteout trusted ==
+regular
+stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
+
+== Check escaped xwhiteout user ==
+regular
+stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH v2 1/4] overlay: Add tests for nesting private xattrs
2023-12-04 18:58 ` [PATCH v2 1/4] overlay: Add tests for nesting private xattrs Amir Goldstein
@ 2023-12-10 13:35 ` Zorro Lang
2023-12-10 15:28 ` Amir Goldstein
0 siblings, 1 reply; 13+ messages in thread
From: Zorro Lang @ 2023-12-10 13:35 UTC (permalink / raw)
To: Amir Goldstein; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
On Mon, Dec 04, 2023 at 08:58:56PM +0200, Amir Goldstein wrote:
> If overlayfs xattr escaping is supported, ensure:
> * We can create "overlay.*" xattrs on a file in the overlayfs
> * We can create an xwhiteout file in the overlayfs
>
> We check for nesting support by trying to getattr an "overlay.*" xattr
> in an overlayfs mount, which will return ENOSUPP in older kernels.
>
> Signed-off-by: Alexander Larsson <alexl@redhat.com>
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> ---
Hi Amir,
This test passed with below kernel configuration at first:
CONFIG_OVERLAY_FS=m
# CONFIG_OVERLAY_FS_REDIRECT_DIR is not set
CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
# CONFIG_OVERLAY_FS_INDEX is not set
# CONFIG_OVERLAY_FS_XINO_AUTO is not set
# CONFIG_OVERLAY_FS_METACOPY is not set
But then I found it fails if I enabled below configurations:
CONFIG_OVERLAY_FS_REDIRECT_DIR=y
CONFIG_OVERLAY_FS_INDEX=y
CONFIG_OVERLAY_FS_XINO_AUTO=y
CONFIG_OVERLAY_FS_METACOPY=y
Without these configures, this test passed. But with them, it fails as [1].
The underlying fs is xfs (with default mkfs options), there're not specific
MOUNT_OPTIONS and MKFS_OPTIONS to use.
I'll delay merging this patchset temporarily, please check.
Thanks,
Zorro
[1]
QA output created by 084
== Check xattr escape trusted ==
# file: SCRATCH_MNT/layer2/dir
trusted.overlay.opaque="y"
user.overlay.opaque="y"
# file: SCRATCH_DEV/mid/layer2/dir
trusted.overlay.overlay.opaque="y"
user.overlay.opaque="y"
mount: /mnt/fstests/SCRATCH_DIR/ovl-mnt: mount(2) system call failed: Stale file handle.
getfattr: /mnt/fstests/SCRATCH_DIR/ovl-mnt/layer2/dir: No such file or directory
nested xattr mount with trusted.overlay
mount: /mnt/fstests/SCRATCH_DIR/nested: special device overlayfs does not exist.
stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/nested: not mounted.
nested xattr mount with user.overlay
mount: /mnt/fstests/SCRATCH_DIR/nested: special device overlayfs does not exist.
stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/nested: not mounted.
copy-up of escaped xattrs
touch: cannot touch '/mnt/fstests/SCRATCH_DIR/ovl-mnt/layer2/dir/other_file': No such file or directory
getfattr: /mnt/fstests/SCRATCH_DIR/upper/layer2/dir: No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/ovl-mnt: not mounted.
== Check xattr escape user ==
# file: SCRATCH_MNT/layer2/dir
trusted.overlay.opaque="y"
user.overlay.opaque="y"
# file: SCRATCH_DEV/mid/layer2/dir
trusted.overlay.opaque="y"
user.overlay.overlay.opaque="y"
mount: /mnt/fstests/SCRATCH_DIR/ovl-mnt: mount(2) system call failed: Stale file handle.
getfattr: /mnt/fstests/SCRATCH_DIR/ovl-mnt/layer2/dir: No such file or directory
nested xattr mount with trusted.overlay
mount: /mnt/fstests/SCRATCH_DIR/nested: special device overlayfs does not exist.
stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/nested: not mounted.
nested xattr mount with user.overlay
mount: /mnt/fstests/SCRATCH_DIR/nested: special device overlayfs does not exist.
stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/nested: not mounted.
copy-up of escaped xattrs
touch: cannot touch '/mnt/fstests/SCRATCH_DIR/ovl-mnt/layer2/dir/other_file': No such file or directory
getfattr: /mnt/fstests/SCRATCH_DIR/upper/layer2/dir: No such file or directory
umount: /mnt/fstests/SCRATCH_DIR/ovl-mnt: not mounted.
== Check xwhiteout trusted ==
regular
stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
== Check xwhiteout user ==
regular
stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
== Check escaped xwhiteout trusted ==
regular
stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
== Check escaped xwhiteout user ==
regular
stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
> tests/overlay/084 | 169 ++++++++++++++++++++++++++++++++++++++++++
> tests/overlay/084.out | 61 +++++++++++++++
> 2 files changed, 230 insertions(+)
> create mode 100755 tests/overlay/084
> create mode 100644 tests/overlay/084.out
>
> diff --git a/tests/overlay/084 b/tests/overlay/084
> new file mode 100755
> index 00000000..ff451f38
> --- /dev/null
> +++ b/tests/overlay/084
> @@ -0,0 +1,169 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2023 Red Hat, Inc. All Rights Reserved.
> +# Copyright (C) 2023 CTERA Networks. All Rights Reserved.
> +#
> +# FS QA Test No. 084
> +#
> +# Test advanded nesting functionallity
> +#
> +. ./common/preamble
> +_begin_fstest auto quick nested
> +
> +# Override the default cleanup function.
> +_cleanup()
> +{
> + cd /
> + # Unmount nested mounts if things fail
> + $UMOUNT_PROG $OVL_BASE_SCRATCH_MNT/nested 2>/dev/null
> + rm -rf $tmp
> +}
> +
> +# Import common functions.
> +. ./common/filter
> +. ./common/attr
> +
> +# real QA test starts here
> +_supported_fs overlay
> +# We use non-default scratch underlying overlay dirs, we need to check
> +# them explicity after test.
> +_require_scratch_nocheck
> +_require_scratch_overlay_xattr_escapes
> +
> +# remove all files from previous tests
> +_scratch_mkfs
> +
> +lowerdir=$OVL_BASE_SCRATCH_MNT/lower
> +middir=$OVL_BASE_SCRATCH_MNT/mid
> +upperdir=$OVL_BASE_SCRATCH_MNT/upper
> +workdir=$OVL_BASE_SCRATCH_MNT/workdir
> +nesteddir=$OVL_BASE_SCRATCH_MNT/nested
> +
> +umount_overlay()
> +{
> + $UMOUNT_PROG $SCRATCH_MNT
> +}
> +
> +test_escape()
> +{
> + local prefix=$1
> +
> + echo -e "\n== Check xattr escape $prefix =="
> +
> + local extra_options=""
> + if [ "$prefix" == "user" ]; then
> + extra_options="-o userxattr"
> + fi
> +
> + _scratch_mkfs
> + mkdir -p $lowerdir $middir $upperdir $workdir $nesteddir
> +
> + _overlay_scratch_mount_dirs $lowerdir $middir $workdir $extra_options
> +
> + mkdir -p $SCRATCH_MNT/layer1/dir/ $SCRATCH_MNT/layer2/dir
> +
> + touch $SCRATCH_MNT/layer1/dir/file
> +
> + # Make layer2/dir an opaque file
> + # Only one of these will be escaped, but both should succeed
> + setfattr -n user.overlay.opaque -v "y" $SCRATCH_MNT/layer2/dir
> + setfattr -n trusted.overlay.opaque -v "y" $SCRATCH_MNT/layer2/dir
> +
> + getfattr -m "overlay\\." --absolute-names -d $SCRATCH_MNT/layer2/dir | _filter_scratch
> +
> + umount_overlay
> +
> + getfattr -m "overlay\\." --absolute-names -d $middir/layer2/dir | _filter_scratch
> +
> + # Remount as lower and try again
> + _overlay_scratch_mount_dirs $middir:$lowerdir $upperdir $workdir $extra_options
> +
> + getfattr -m "overlay\\." --absolute-names -d $SCRATCH_MNT/layer2/dir | _filter_scratch
> +
> + # Recursively mount and ensure the opaque dir is working with both trusted and user xattrs
> + echo "nested xattr mount with trusted.overlay"
> + _overlay_mount_dirs $SCRATCH_MNT/layer2:$SCRATCH_MNT/layer1 - - overlayfs $nesteddir
> + stat $nesteddir/dir/file 2>&1 | _filter_scratch
> + $UMOUNT_PROG $nesteddir
> +
> + echo "nested xattr mount with user.overlay"
> + _overlay_mount_dirs $SCRATCH_MNT/layer2:$SCRATCH_MNT/layer1 - - -o userxattr overlayfs $nesteddir
> + stat $nesteddir/dir/file 2>&1 | _filter_scratch
> + $UMOUNT_PROG $nesteddir
> +
> + # Also ensure propagate the escaped xattr when we copy-up layer2/dir
> + echo "copy-up of escaped xattrs"
> + touch $SCRATCH_MNT/layer2/dir/other_file
> + getfattr -m "$prefix.overlay\\.overlay" --absolute-names -d $upperdir/layer2/dir | _filter_scratch
> +
> + umount_overlay
> +}
> +
> +test_escape trusted
> +test_escape user
> +
> +do_test_xwhiteout()
> +{
> + local prefix=$1
> + local basedir=$2
> +
> + local extra_options=""
> + if [ "$prefix" == "user" ]; then
> + extra_options="-o userxattr"
> + fi
> +
> + mkdir -p $basedir/lower $basedir/upper $basedir/work
> + touch $basedir/lower/regular $basedir/lower/hidden $basedir/upper/hidden
> + setfattr -n $prefix.overlay.whiteouts -v "y" $basedir/upper
> + setfattr -n $prefix.overlay.whiteout -v "y" $basedir/upper/hidden
> +
> + # Test the hidden is invisible
> + _overlay_scratch_mount_dirs $basedir/upper:$basedir/lower - - $extra_options
> + ls $SCRATCH_MNT
> + stat $SCRATCH_MNT/hidden 2>&1 | _filter_scratch
> + umount_overlay
> +}
> +
> +# Validate that xwhiteouts work like whiteouts
> +test_xwhiteout()
> +{
> + local prefix=$1
> +
> + echo -e "\n== Check xwhiteout $prefix =="
> +
> + _scratch_mkfs
> +
> + do_test_xwhiteout $prefix $OVL_BASE_SCRATCH_MNT
> +}
> +
> +test_xwhiteout trusted
> +test_xwhiteout user
> +
> +# Validate that (escaped) xwhiteouts work inside a nested overlayfs mount
> +test_escaped_xwhiteout()
> +{
> + local prefix=$1
> +
> + echo -e "\n== Check escaped xwhiteout $prefix =="
> +
> + local extra_options=""
> + if [ "$prefix" == "user" ]; then
> + extra_options="-o userxattr"
> + fi
> +
> + _scratch_mkfs
> + mkdir -p $lowerdir $upperdir $workdir $nesteddir
> +
> + _overlay_mount_dirs $lowerdir $upperdir $workdir $extra_options overlayfs $nesteddir
> +
> + do_test_xwhiteout $prefix $nesteddir
> +
> + $UMOUNT_PROG $nesteddir
> +}
> +
> +test_escaped_xwhiteout trusted
> +test_escaped_xwhiteout user
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/overlay/084.out b/tests/overlay/084.out
> new file mode 100644
> index 00000000..54b890de
> --- /dev/null
> +++ b/tests/overlay/084.out
> @@ -0,0 +1,61 @@
> +QA output created by 084
> +
> +== Check xattr escape trusted ==
> +# file: SCRATCH_MNT/layer2/dir
> +trusted.overlay.opaque="y"
> +user.overlay.opaque="y"
> +
> +# file: SCRATCH_DEV/mid/layer2/dir
> +trusted.overlay.overlay.opaque="y"
> +user.overlay.opaque="y"
> +
> +# file: SCRATCH_MNT/layer2/dir
> +trusted.overlay.opaque="y"
> +user.overlay.opaque="y"
> +
> +nested xattr mount with trusted.overlay
> +stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
> +nested xattr mount with user.overlay
> +stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
> +copy-up of escaped xattrs
> +# file: SCRATCH_DEV/upper/layer2/dir
> +trusted.overlay.overlay.opaque="y"
> +
> +
> +== Check xattr escape user ==
> +# file: SCRATCH_MNT/layer2/dir
> +trusted.overlay.opaque="y"
> +user.overlay.opaque="y"
> +
> +# file: SCRATCH_DEV/mid/layer2/dir
> +trusted.overlay.opaque="y"
> +user.overlay.overlay.opaque="y"
> +
> +# file: SCRATCH_MNT/layer2/dir
> +trusted.overlay.opaque="y"
> +user.overlay.opaque="y"
> +
> +nested xattr mount with trusted.overlay
> +stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
> +nested xattr mount with user.overlay
> +stat: cannot statx 'SCRATCH_DEV/nested/dir/file': No such file or directory
> +copy-up of escaped xattrs
> +# file: SCRATCH_DEV/upper/layer2/dir
> +user.overlay.overlay.opaque="y"
> +
> +
> +== Check xwhiteout trusted ==
> +regular
> +stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
> +
> +== Check xwhiteout user ==
> +regular
> +stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
> +
> +== Check escaped xwhiteout trusted ==
> +regular
> +stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
> +
> +== Check escaped xwhiteout user ==
> +regular
> +stat: cannot statx 'SCRATCH_MNT/hidden': No such file or directory
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v2 1/4] overlay: Add tests for nesting private xattrs
2023-12-10 13:35 ` Zorro Lang
@ 2023-12-10 15:28 ` Amir Goldstein
2023-12-10 20:45 ` Zorro Lang
0 siblings, 1 reply; 13+ messages in thread
From: Amir Goldstein @ 2023-12-10 15:28 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
On Sun, Dec 10, 2023 at 3:35 PM Zorro Lang <zlang@redhat.com> wrote:
>
> On Mon, Dec 04, 2023 at 08:58:56PM +0200, Amir Goldstein wrote:
> > If overlayfs xattr escaping is supported, ensure:
> > * We can create "overlay.*" xattrs on a file in the overlayfs
> > * We can create an xwhiteout file in the overlayfs
> >
> > We check for nesting support by trying to getattr an "overlay.*" xattr
> > in an overlayfs mount, which will return ENOSUPP in older kernels.
> >
> > Signed-off-by: Alexander Larsson <alexl@redhat.com>
> > Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> > ---
>
> Hi Amir,
>
> This test passed with below kernel configuration at first:
> CONFIG_OVERLAY_FS=m
> # CONFIG_OVERLAY_FS_REDIRECT_DIR is not set
> CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
> # CONFIG_OVERLAY_FS_INDEX is not set
> # CONFIG_OVERLAY_FS_XINO_AUTO is not set
> # CONFIG_OVERLAY_FS_METACOPY is not set
>
> But then I found it fails if I enabled below configurations:
> CONFIG_OVERLAY_FS_REDIRECT_DIR=y
> CONFIG_OVERLAY_FS_INDEX=y
> CONFIG_OVERLAY_FS_XINO_AUTO=y
> CONFIG_OVERLAY_FS_METACOPY=y
>
> Without these configures, this test passed. But with them, it fails as [1].
> The underlying fs is xfs (with default mkfs options), there're not specific
> MOUNT_OPTIONS and MKFS_OPTIONS to use.
>
> I'll delay merging this patchset temporarily, please check.
>
good spotting!
Here is a fix if you want to fix and test it in your tree:
diff --git a/tests/overlay/084 b/tests/overlay/084
index ff451f38..8465caeb 100755
--- a/tests/overlay/084
+++ b/tests/overlay/084
@@ -50,9 +50,10 @@ test_escape()
echo -e "\n== Check xattr escape $prefix =="
- local extra_options=""
+ # index feature would require nfs_export on $nesteddir mount
+ local extra_options="-o index=off"
if [ "$prefix" == "user" ]; then
- extra_options="-o userxattr"
+ extra_options+=",userxattr"
fi
_scratch_mkfs
@@ -146,9 +147,10 @@ test_escaped_xwhiteout()
echo -e "\n== Check escaped xwhiteout $prefix =="
- local extra_options=""
+ # index feature would require nfs_export on $nesteddir mount
+ local extra_options="-o index=off"
if [ "$prefix" == "user" ]; then
- extra_options="-o userxattr"
+ extra_options+=",userxattr"
fi
_scratch_mkfs
Thanks,
Amir.
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH v2 1/4] overlay: Add tests for nesting private xattrs
2023-12-10 15:28 ` Amir Goldstein
@ 2023-12-10 20:45 ` Zorro Lang
2023-12-11 6:47 ` Amir Goldstein
0 siblings, 1 reply; 13+ messages in thread
From: Zorro Lang @ 2023-12-10 20:45 UTC (permalink / raw)
To: Amir Goldstein; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
On Sun, Dec 10, 2023 at 05:28:34PM +0200, Amir Goldstein wrote:
> On Sun, Dec 10, 2023 at 3:35 PM Zorro Lang <zlang@redhat.com> wrote:
> >
> > On Mon, Dec 04, 2023 at 08:58:56PM +0200, Amir Goldstein wrote:
> > > If overlayfs xattr escaping is supported, ensure:
> > > * We can create "overlay.*" xattrs on a file in the overlayfs
> > > * We can create an xwhiteout file in the overlayfs
> > >
> > > We check for nesting support by trying to getattr an "overlay.*" xattr
> > > in an overlayfs mount, which will return ENOSUPP in older kernels.
> > >
> > > Signed-off-by: Alexander Larsson <alexl@redhat.com>
> > > Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> > > ---
> >
> > Hi Amir,
> >
> > This test passed with below kernel configuration at first:
> > CONFIG_OVERLAY_FS=m
> > # CONFIG_OVERLAY_FS_REDIRECT_DIR is not set
> > CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
> > # CONFIG_OVERLAY_FS_INDEX is not set
> > # CONFIG_OVERLAY_FS_XINO_AUTO is not set
> > # CONFIG_OVERLAY_FS_METACOPY is not set
> >
> > But then I found it fails if I enabled below configurations:
> > CONFIG_OVERLAY_FS_REDIRECT_DIR=y
> > CONFIG_OVERLAY_FS_INDEX=y
> > CONFIG_OVERLAY_FS_XINO_AUTO=y
> > CONFIG_OVERLAY_FS_METACOPY=y
> >
> > Without these configures, this test passed. But with them, it fails as [1].
> > The underlying fs is xfs (with default mkfs options), there're not specific
> > MOUNT_OPTIONS and MKFS_OPTIONS to use.
> >
> > I'll delay merging this patchset temporarily, please check.
> >
>
> good spotting!
>
> Here is a fix if you want to fix and test it in your tree:
>
> diff --git a/tests/overlay/084 b/tests/overlay/084
> index ff451f38..8465caeb 100755
> --- a/tests/overlay/084
> +++ b/tests/overlay/084
> @@ -50,9 +50,10 @@ test_escape()
>
> echo -e "\n== Check xattr escape $prefix =="
>
> - local extra_options=""
> + # index feature would require nfs_export on $nesteddir mount
> + local extra_options="-o index=off"
> if [ "$prefix" == "user" ]; then
> - extra_options="-o userxattr"
> + extra_options+=",userxattr"
> fi
>
> _scratch_mkfs
> @@ -146,9 +147,10 @@ test_escaped_xwhiteout()
>
> echo -e "\n== Check escaped xwhiteout $prefix =="
>
> - local extra_options=""
> + # index feature would require nfs_export on $nesteddir mount
> + local extra_options="-o index=off"
> if [ "$prefix" == "user" ]; then
> - extra_options="-o userxattr"
> + extra_options+=",userxattr"
It works, so it's about the CONFIG_OVERLAY_FS_INDEX=y. I've releated fstests
v2023.12.10 version, this patchset will be in next release. Will send a new
version with this change?
Thanks,
Zorro
> fi
>
> _scratch_mkfs
>
>
> Thanks,
> Amir.
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v2 1/4] overlay: Add tests for nesting private xattrs
2023-12-10 20:45 ` Zorro Lang
@ 2023-12-11 6:47 ` Amir Goldstein
0 siblings, 0 replies; 13+ messages in thread
From: Amir Goldstein @ 2023-12-11 6:47 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
On Sun, Dec 10, 2023 at 10:45 PM Zorro Lang <zlang@redhat.com> wrote:
>
> On Sun, Dec 10, 2023 at 05:28:34PM +0200, Amir Goldstein wrote:
> > On Sun, Dec 10, 2023 at 3:35 PM Zorro Lang <zlang@redhat.com> wrote:
> > >
> > > On Mon, Dec 04, 2023 at 08:58:56PM +0200, Amir Goldstein wrote:
> > > > If overlayfs xattr escaping is supported, ensure:
> > > > * We can create "overlay.*" xattrs on a file in the overlayfs
> > > > * We can create an xwhiteout file in the overlayfs
> > > >
> > > > We check for nesting support by trying to getattr an "overlay.*" xattr
> > > > in an overlayfs mount, which will return ENOSUPP in older kernels.
> > > >
> > > > Signed-off-by: Alexander Larsson <alexl@redhat.com>
> > > > Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> > > > ---
> > >
> > > Hi Amir,
> > >
> > > This test passed with below kernel configuration at first:
> > > CONFIG_OVERLAY_FS=m
> > > # CONFIG_OVERLAY_FS_REDIRECT_DIR is not set
> > > CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
> > > # CONFIG_OVERLAY_FS_INDEX is not set
> > > # CONFIG_OVERLAY_FS_XINO_AUTO is not set
> > > # CONFIG_OVERLAY_FS_METACOPY is not set
> > >
> > > But then I found it fails if I enabled below configurations:
> > > CONFIG_OVERLAY_FS_REDIRECT_DIR=y
> > > CONFIG_OVERLAY_FS_INDEX=y
> > > CONFIG_OVERLAY_FS_XINO_AUTO=y
> > > CONFIG_OVERLAY_FS_METACOPY=y
> > >
> > > Without these configures, this test passed. But with them, it fails as [1].
> > > The underlying fs is xfs (with default mkfs options), there're not specific
> > > MOUNT_OPTIONS and MKFS_OPTIONS to use.
> > >
> > > I'll delay merging this patchset temporarily, please check.
> > >
> >
> > good spotting!
> >
> > Here is a fix if you want to fix and test it in your tree:
> >
> > diff --git a/tests/overlay/084 b/tests/overlay/084
> > index ff451f38..8465caeb 100755
> > --- a/tests/overlay/084
> > +++ b/tests/overlay/084
> > @@ -50,9 +50,10 @@ test_escape()
> >
> > echo -e "\n== Check xattr escape $prefix =="
> >
> > - local extra_options=""
> > + # index feature would require nfs_export on $nesteddir mount
> > + local extra_options="-o index=off"
> > if [ "$prefix" == "user" ]; then
> > - extra_options="-o userxattr"
> > + extra_options+=",userxattr"
> > fi
> >
> > _scratch_mkfs
> > @@ -146,9 +147,10 @@ test_escaped_xwhiteout()
> >
> > echo -e "\n== Check escaped xwhiteout $prefix =="
> >
> > - local extra_options=""
> > + # index feature would require nfs_export on $nesteddir mount
> > + local extra_options="-o index=off"
> > if [ "$prefix" == "user" ]; then
> > - extra_options="-o userxattr"
> > + extra_options+=",userxattr"
>
> It works, so it's about the CONFIG_OVERLAY_FS_INDEX=y.
Yes.
the nested overlayfs setup requires that either the inner
overlayfs has nfs_export enabled, as is done in tests
overlay/068,069,070,071
or that the outer overlayfs has index disabled.
The latter is easier for this test, because there is no need
for the index feature in these test cases.
> I've released fstests
> v2023.12.10 version, this patchset will be in next release. Will send a new
> version with this change?
>
Ok, I will send a new version of test 084.
Thanks,
Amir.
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v2 2/4] overlay: prepare for new lowerdir+,datadir+ tests
2023-12-04 18:58 [PATCH v2 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
2023-12-04 18:58 ` [PATCH v2 1/4] overlay: Add tests for nesting private xattrs Amir Goldstein
@ 2023-12-04 18:58 ` Amir Goldstein
2023-12-06 8:37 ` Zorro Lang
2023-12-04 18:58 ` [PATCH v2 3/4] overlay: test data-only lowerdirs with datadir+ mount option Amir Goldstein
` (2 subsequent siblings)
4 siblings, 1 reply; 13+ messages in thread
From: Amir Goldstein @ 2023-12-04 18:58 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
In preparation to forking tests for new lowerdir+,datadir+ mount options,
prepare a helper to test kernel support and pass datadirs into mount
helpers in overlay/079 test.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
common/overlay | 15 +++++++++++++++
tests/overlay/079 | 36 +++++++++++++++++++++---------------
2 files changed, 36 insertions(+), 15 deletions(-)
diff --git a/common/overlay b/common/overlay
index 8f275228..ea1eb7b1 100644
--- a/common/overlay
+++ b/common/overlay
@@ -247,6 +247,21 @@ _require_scratch_overlay_lowerdata_layers()
_scratch_unmount
}
+# Check kernel support for lowerdir+=<lowerdir>,datadir+=<lowerdatadir> format
+_require_scratch_overlay_lowerdir_add_layers()
+{
+ local lowerdir="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER"
+ local datadir="$OVL_BASE_SCRATCH_MNT/$OVL_LOWER"
+
+ _scratch_mkfs > /dev/null 2>&1
+ $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+ -o"lowerdir+=$lowerdir,datadir+=$datadir" \
+ -o"redirect_dir=follow,metacopy=on" > /dev/null 2>&1 || \
+ _notrun "overlay lowerdir+,datadir+ not supported on ${SCRATCH_DEV}"
+
+ _scratch_unmount
+}
+
# Helper function to check underlying dirs of overlay filesystem
_overlay_fsck_dirs()
{
diff --git a/tests/overlay/079 b/tests/overlay/079
index 77f94598..078ee816 100755
--- a/tests/overlay/079
+++ b/tests/overlay/079
@@ -139,16 +139,21 @@ check_file_size_contents()
mount_overlay()
{
- local _lowerdir=$1
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
- _overlay_scratch_mount_dirs "$_lowerdir" $upperdir $workdir -o redirect_dir=on,index=on,metacopy=on
+ $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+ -o"lowerdir=$_lowerdir::$_datadir2::$_datadir" \
+ -o"upperdir=$upperdir,workdir=$workdir" \
+ -o redirect_dir=on,metacopy=on
}
mount_ro_overlay()
{
- local _lowerdir=$1
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
- _overlay_scratch_mount_dirs "$_lowerdir" "-" "-" -o ro,redirect_dir=follow,metacopy=on
+ $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+ -o"lowerdir=$_lowerdir::$_datadir2::$_datadir" \
+ -o redirect_dir=follow,metacopy=on
}
umount_overlay()
@@ -160,14 +165,14 @@ test_no_access()
{
local _target=$1
- mount_ro_overlay "$lowerdir::$datadir2::$datadir"
+ mount_ro_overlay "$lowerdir" "$datadir2" "$datadir"
stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \
echo "No access to lowerdata layer $_target"
echo "Unmount and Mount rw"
umount_overlay
- mount_overlay "$lowerdir::$datadir2::$datadir"
+ mount_overlay "$lowerdir" "$datadir2" "$datadir"
stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \
echo "No access to lowerdata layer $_target"
umount_overlay
@@ -175,11 +180,12 @@ test_no_access()
test_common()
{
- local _lowerdirs=$1 _target=$2 _size=$3 _blocks=$4 _data="$5"
- local _redirect=$6
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
+ local _target=$4 _size=$5 _blocks=$6 _data="$7"
+ local _redirect=$8
echo "Mount ro"
- mount_ro_overlay $_lowerdirs
+ mount_ro_overlay $_lowerdir $_datadir2 $_datadir
# Check redirect xattr to lowerdata
[ -n "$_redirect" ] && check_redirect $lowerdir/$_target "$_redirect"
@@ -191,7 +197,7 @@ test_common()
# Do a mount cycle and check size and contents again.
echo "Unmount and Mount rw"
umount_overlay
- mount_overlay $_lowerdirs
+ mount_overlay $_lowerdir $_datadir2 $_datadir
echo "check properties of metadata copied up file $_target"
check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
check_file_blocks $SCRATCH_MNT/$_target $_blocks
@@ -203,7 +209,7 @@ test_common()
check_file_size_contents $upperdir/$_target $_size ""
# Trigger data copy up and check absence of metacopy xattr.
- mount_overlay $_lowerdirs
+ mount_overlay $_lowerdir $_datadir2 $_datadir
$XFS_IO_PROG -c "falloc 0 1" $SCRATCH_MNT/$_target >> $seqres.full
echo "check properties of data copied up file $_target"
check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
@@ -216,7 +222,7 @@ test_lazy()
{
local _target=$1
- mount_overlay "$lowerdir::$datadir2::$datadir"
+ mount_overlay "$lowerdir" "$datadir2" "$datadir"
# Metadata should be valid
check_file_size $SCRATCH_MNT/$_target $datasize
@@ -305,12 +311,12 @@ test_no_access "$sharedname"
echo -e "\n== Check follow to lowerdata layer with absolute redirect =="
prepare_midlayer "/subdir/$dataname"
-test_common "$lowerdir::$datadir2::$datadir" "$dataname" $datasize $datablocks \
+test_common "$lowerdir" "$datadir2" "$datadir" "$dataname" $datasize $datablocks \
"$datacontent" "/subdir/$dataname"
-test_common "$lowerdir::$datadir2::$datadir" "$dataname2" $datasize $datablocks \
+test_common "$lowerdir" "$datadir2" "$datadir" "$dataname2" $datasize $datablocks \
"$datacontent2" "/subdir/$dataname.2"
# Shared file should be picked from upper datadir
-test_common "$lowerdir::$datadir2::$datadir" "$sharedname" $datasize $datablocks \
+test_common "$lowerdir" "$datadir2" "$datadir" "$sharedname" $datasize $datablocks \
"$datacontent2" "/subdir/$dataname.shared"
echo -e "\n== Check lazy follow to lowerdata layer =="
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH v2 2/4] overlay: prepare for new lowerdir+,datadir+ tests
2023-12-04 18:58 ` [PATCH v2 2/4] overlay: prepare for new lowerdir+,datadir+ tests Amir Goldstein
@ 2023-12-06 8:37 ` Zorro Lang
2023-12-06 10:29 ` Amir Goldstein
0 siblings, 1 reply; 13+ messages in thread
From: Zorro Lang @ 2023-12-06 8:37 UTC (permalink / raw)
To: Amir Goldstein; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
On Mon, Dec 04, 2023 at 08:58:57PM +0200, Amir Goldstein wrote:
> In preparation to forking tests for new lowerdir+,datadir+ mount options,
> prepare a helper to test kernel support and pass datadirs into mount
> helpers in overlay/079 test.
>
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> ---
> common/overlay | 15 +++++++++++++++
> tests/overlay/079 | 36 +++++++++++++++++++++---------------
> 2 files changed, 36 insertions(+), 15 deletions(-)
>
> diff --git a/common/overlay b/common/overlay
> index 8f275228..ea1eb7b1 100644
> --- a/common/overlay
> +++ b/common/overlay
> @@ -247,6 +247,21 @@ _require_scratch_overlay_lowerdata_layers()
> _scratch_unmount
> }
>
> +# Check kernel support for lowerdir+=<lowerdir>,datadir+=<lowerdatadir> format
> +_require_scratch_overlay_lowerdir_add_layers()
> +{
> + local lowerdir="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER"
> + local datadir="$OVL_BASE_SCRATCH_MNT/$OVL_LOWER"
> +
> + _scratch_mkfs > /dev/null 2>&1
> + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
> + -o"lowerdir+=$lowerdir,datadir+=$datadir" \
> + -o"redirect_dir=follow,metacopy=on" > /dev/null 2>&1 || \
> + _notrun "overlay lowerdir+,datadir+ not supported on ${SCRATCH_DEV}"
Hi Amir,
I found overlay cases don't use helpers in common/overlay recently, always
use raw $MOUNT_PROG directly (not only in this patchset). Although overlay
supports new mount format, can we improve the mount helpers in common/overlay
to support that? It would be to good to use common helpers to do common
operation.
Anyway, that can be changed in another patch, if it takes too much time or
you don't want to do it at here. What do you think?
Thanks,
Zorro
> +
> + _scratch_unmount
> +}
> +
> # Helper function to check underlying dirs of overlay filesystem
> _overlay_fsck_dirs()
> {
> diff --git a/tests/overlay/079 b/tests/overlay/079
> index 77f94598..078ee816 100755
> --- a/tests/overlay/079
> +++ b/tests/overlay/079
> @@ -139,16 +139,21 @@ check_file_size_contents()
>
> mount_overlay()
> {
> - local _lowerdir=$1
> + local _lowerdir=$1 _datadir2=$2 _datadir=$3
>
> - _overlay_scratch_mount_dirs "$_lowerdir" $upperdir $workdir -o redirect_dir=on,index=on,metacopy=on
> + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
> + -o"lowerdir=$_lowerdir::$_datadir2::$_datadir" \
> + -o"upperdir=$upperdir,workdir=$workdir" \
> + -o redirect_dir=on,metacopy=on
> }
>
> mount_ro_overlay()
> {
> - local _lowerdir=$1
> + local _lowerdir=$1 _datadir2=$2 _datadir=$3
>
> - _overlay_scratch_mount_dirs "$_lowerdir" "-" "-" -o ro,redirect_dir=follow,metacopy=on
> + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
> + -o"lowerdir=$_lowerdir::$_datadir2::$_datadir" \
> + -o redirect_dir=follow,metacopy=on
> }
>
> umount_overlay()
> @@ -160,14 +165,14 @@ test_no_access()
> {
> local _target=$1
>
> - mount_ro_overlay "$lowerdir::$datadir2::$datadir"
> + mount_ro_overlay "$lowerdir" "$datadir2" "$datadir"
>
> stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \
> echo "No access to lowerdata layer $_target"
>
> echo "Unmount and Mount rw"
> umount_overlay
> - mount_overlay "$lowerdir::$datadir2::$datadir"
> + mount_overlay "$lowerdir" "$datadir2" "$datadir"
> stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \
> echo "No access to lowerdata layer $_target"
> umount_overlay
> @@ -175,11 +180,12 @@ test_no_access()
>
> test_common()
> {
> - local _lowerdirs=$1 _target=$2 _size=$3 _blocks=$4 _data="$5"
> - local _redirect=$6
> + local _lowerdir=$1 _datadir2=$2 _datadir=$3
> + local _target=$4 _size=$5 _blocks=$6 _data="$7"
> + local _redirect=$8
>
> echo "Mount ro"
> - mount_ro_overlay $_lowerdirs
> + mount_ro_overlay $_lowerdir $_datadir2 $_datadir
>
> # Check redirect xattr to lowerdata
> [ -n "$_redirect" ] && check_redirect $lowerdir/$_target "$_redirect"
> @@ -191,7 +197,7 @@ test_common()
> # Do a mount cycle and check size and contents again.
> echo "Unmount and Mount rw"
> umount_overlay
> - mount_overlay $_lowerdirs
> + mount_overlay $_lowerdir $_datadir2 $_datadir
> echo "check properties of metadata copied up file $_target"
> check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
> check_file_blocks $SCRATCH_MNT/$_target $_blocks
> @@ -203,7 +209,7 @@ test_common()
> check_file_size_contents $upperdir/$_target $_size ""
>
> # Trigger data copy up and check absence of metacopy xattr.
> - mount_overlay $_lowerdirs
> + mount_overlay $_lowerdir $_datadir2 $_datadir
> $XFS_IO_PROG -c "falloc 0 1" $SCRATCH_MNT/$_target >> $seqres.full
> echo "check properties of data copied up file $_target"
> check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
> @@ -216,7 +222,7 @@ test_lazy()
> {
> local _target=$1
>
> - mount_overlay "$lowerdir::$datadir2::$datadir"
> + mount_overlay "$lowerdir" "$datadir2" "$datadir"
>
> # Metadata should be valid
> check_file_size $SCRATCH_MNT/$_target $datasize
> @@ -305,12 +311,12 @@ test_no_access "$sharedname"
>
> echo -e "\n== Check follow to lowerdata layer with absolute redirect =="
> prepare_midlayer "/subdir/$dataname"
> -test_common "$lowerdir::$datadir2::$datadir" "$dataname" $datasize $datablocks \
> +test_common "$lowerdir" "$datadir2" "$datadir" "$dataname" $datasize $datablocks \
> "$datacontent" "/subdir/$dataname"
> -test_common "$lowerdir::$datadir2::$datadir" "$dataname2" $datasize $datablocks \
> +test_common "$lowerdir" "$datadir2" "$datadir" "$dataname2" $datasize $datablocks \
> "$datacontent2" "/subdir/$dataname.2"
> # Shared file should be picked from upper datadir
> -test_common "$lowerdir::$datadir2::$datadir" "$sharedname" $datasize $datablocks \
> +test_common "$lowerdir" "$datadir2" "$datadir" "$sharedname" $datasize $datablocks \
> "$datacontent2" "/subdir/$dataname.shared"
>
> echo -e "\n== Check lazy follow to lowerdata layer =="
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v2 2/4] overlay: prepare for new lowerdir+,datadir+ tests
2023-12-06 8:37 ` Zorro Lang
@ 2023-12-06 10:29 ` Amir Goldstein
2023-12-06 13:35 ` Zorro Lang
0 siblings, 1 reply; 13+ messages in thread
From: Amir Goldstein @ 2023-12-06 10:29 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
On Wed, Dec 6, 2023 at 10:37 AM Zorro Lang <zlang@redhat.com> wrote:
>
> On Mon, Dec 04, 2023 at 08:58:57PM +0200, Amir Goldstein wrote:
> > In preparation to forking tests for new lowerdir+,datadir+ mount options,
> > prepare a helper to test kernel support and pass datadirs into mount
> > helpers in overlay/079 test.
> >
> > Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> > ---
> > common/overlay | 15 +++++++++++++++
> > tests/overlay/079 | 36 +++++++++++++++++++++---------------
> > 2 files changed, 36 insertions(+), 15 deletions(-)
> >
> > diff --git a/common/overlay b/common/overlay
> > index 8f275228..ea1eb7b1 100644
> > --- a/common/overlay
> > +++ b/common/overlay
> > @@ -247,6 +247,21 @@ _require_scratch_overlay_lowerdata_layers()
> > _scratch_unmount
> > }
> >
> > +# Check kernel support for lowerdir+=<lowerdir>,datadir+=<lowerdatadir> format
> > +_require_scratch_overlay_lowerdir_add_layers()
> > +{
> > + local lowerdir="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER"
> > + local datadir="$OVL_BASE_SCRATCH_MNT/$OVL_LOWER"
> > +
> > + _scratch_mkfs > /dev/null 2>&1
> > + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
> > + -o"lowerdir+=$lowerdir,datadir+=$datadir" \
> > + -o"redirect_dir=follow,metacopy=on" > /dev/null 2>&1 || \
> > + _notrun "overlay lowerdir+,datadir+ not supported on ${SCRATCH_DEV}"
>
> Hi Amir,
>
> I found overlay cases don't use helpers in common/overlay recently, always
> use raw $MOUNT_PROG directly (not only in this patchset). Although overlay
> supports new mount format, can we improve the mount helpers in common/overlay
> to support that? It would be to good to use common helpers to do common
> operation.
>
> Anyway, that can be changed in another patch, if it takes too much time or
> you don't want to do it at here. What do you think?
I agree. I wouldn't improve the existing helpers to support the new
lowerdir+,datadir+ options as positional argument like in
_overlay_scratch_mount_dirs(), but there is an opportunity to reduce
dedupe of this common line with a helper:
# Mount with mnt/dev of scratch mount and custom mount options
_overlay_scratch_mount_opts()
{
$MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT $*
}
I will work on this cleanup and post a patch when I get to it.
No need to block this series for the cleanup.
Thanks,
Amir.
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v2 2/4] overlay: prepare for new lowerdir+,datadir+ tests
2023-12-06 10:29 ` Amir Goldstein
@ 2023-12-06 13:35 ` Zorro Lang
0 siblings, 0 replies; 13+ messages in thread
From: Zorro Lang @ 2023-12-06 13:35 UTC (permalink / raw)
To: Amir Goldstein; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
On Wed, Dec 06, 2023 at 12:29:54PM +0200, Amir Goldstein wrote:
> On Wed, Dec 6, 2023 at 10:37 AM Zorro Lang <zlang@redhat.com> wrote:
> >
> > On Mon, Dec 04, 2023 at 08:58:57PM +0200, Amir Goldstein wrote:
> > > In preparation to forking tests for new lowerdir+,datadir+ mount options,
> > > prepare a helper to test kernel support and pass datadirs into mount
> > > helpers in overlay/079 test.
> > >
> > > Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> > > ---
> > > common/overlay | 15 +++++++++++++++
> > > tests/overlay/079 | 36 +++++++++++++++++++++---------------
> > > 2 files changed, 36 insertions(+), 15 deletions(-)
> > >
> > > diff --git a/common/overlay b/common/overlay
> > > index 8f275228..ea1eb7b1 100644
> > > --- a/common/overlay
> > > +++ b/common/overlay
> > > @@ -247,6 +247,21 @@ _require_scratch_overlay_lowerdata_layers()
> > > _scratch_unmount
> > > }
> > >
> > > +# Check kernel support for lowerdir+=<lowerdir>,datadir+=<lowerdatadir> format
> > > +_require_scratch_overlay_lowerdir_add_layers()
> > > +{
> > > + local lowerdir="$OVL_BASE_SCRATCH_MNT/$OVL_UPPER"
> > > + local datadir="$OVL_BASE_SCRATCH_MNT/$OVL_LOWER"
> > > +
> > > + _scratch_mkfs > /dev/null 2>&1
> > > + $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
> > > + -o"lowerdir+=$lowerdir,datadir+=$datadir" \
> > > + -o"redirect_dir=follow,metacopy=on" > /dev/null 2>&1 || \
> > > + _notrun "overlay lowerdir+,datadir+ not supported on ${SCRATCH_DEV}"
> >
> > Hi Amir,
> >
> > I found overlay cases don't use helpers in common/overlay recently, always
> > use raw $MOUNT_PROG directly (not only in this patchset). Although overlay
> > supports new mount format, can we improve the mount helpers in common/overlay
> > to support that? It would be to good to use common helpers to do common
> > operation.
> >
> > Anyway, that can be changed in another patch, if it takes too much time or
> > you don't want to do it at here. What do you think?
>
> I agree. I wouldn't improve the existing helpers to support the new
> lowerdir+,datadir+ options as positional argument like in
> _overlay_scratch_mount_dirs(), but there is an opportunity to reduce
> dedupe of this common line with a helper:
>
> # Mount with mnt/dev of scratch mount and custom mount options
> _overlay_scratch_mount_opts()
> {
> $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT $*
> }
>
> I will work on this cleanup and post a patch when I get to it.
> No need to block this series for the cleanup.
Agree, thanks for doing this!
>
> Thanks,
> Amir.
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v2 3/4] overlay: test data-only lowerdirs with datadir+ mount option
2023-12-04 18:58 [PATCH v2 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
2023-12-04 18:58 ` [PATCH v2 1/4] overlay: Add tests for nesting private xattrs Amir Goldstein
2023-12-04 18:58 ` [PATCH v2 2/4] overlay: prepare for new lowerdir+,datadir+ tests Amir Goldstein
@ 2023-12-04 18:58 ` Amir Goldstein
2023-12-04 18:58 ` [PATCH v2 4/4] overlay: test parsing of lowerdir+,datadir+ mount options Amir Goldstein
2023-12-09 4:49 ` [PATCH v2 0/4] Overlayfs tests for 6.7-rc1 Zorro Lang
4 siblings, 0 replies; 13+ messages in thread
From: Amir Goldstein @ 2023-12-04 18:58 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
Fork test overlay/079 to use the new lowerdir+,datadir+ mount options.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
tests/overlay/085 | 332 ++++++++++++++++++++++++++++++++++++++++++
tests/overlay/085.out | 42 ++++++
2 files changed, 374 insertions(+)
create mode 100755 tests/overlay/085
create mode 100644 tests/overlay/085.out
diff --git a/tests/overlay/085 b/tests/overlay/085
new file mode 100755
index 00000000..07a32c24
--- /dev/null
+++ b/tests/overlay/085
@@ -0,0 +1,332 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2018 Red Hat, Inc. All Rights Reserved.
+# Copyright (C) 2023 CTERA Networks. All Rights Reserved.
+#
+# FS QA Test No. 085
+#
+# Test data-only layers functionality.
+# This is a variant of test overlay/079 with lowerdir+,datadir+ mount options
+#
+. ./common/preamble
+_begin_fstest auto quick metacopy redirect prealloc
+
+# Import common functions.
+. ./common/filter
+. ./common/attr
+
+# real QA test starts here
+_supported_fs overlay
+# We use non-default scratch underlying overlay dirs, we need to check
+# them explicity after test.
+_require_scratch_nocheck
+_require_scratch_overlay_features redirect_dir metacopy
+_require_scratch_overlay_lowerdir_add_layers
+_require_xfs_io_command "falloc"
+
+# remove all files from previous tests
+_scratch_mkfs
+
+# File size on lower
+dataname="datafile"
+sharedname="shared"
+datacontent="data"
+dataname2="datafile2"
+datacontent2="data2"
+datasize="4096"
+
+# Number of blocks allocated by filesystem on lower. Will be queried later.
+datarblocks=""
+datarblocksize=""
+estimated_datablocks=""
+
+udirname="pureupper"
+ufile="upperfile"
+
+# Check metacopy xattr
+check_metacopy()
+{
+ local target=$1 exist=$2
+ local out_f target_f
+ local msg
+
+ out_f=$(_getfattr --absolute-names --only-values -n \
+ $OVL_XATTR_METACOPY $target 2>&1 | _filter_scratch)
+
+ if [ "$exist" == "y" ];then
+ [ "$out_f" == "" ] && return
+ echo "Metacopy xattr does not exist on ${target}. stdout=$out_f"
+ return
+ fi
+
+ if [ "$out_f" == "" ];then
+ echo "Metacopy xattr exists on ${target} unexpectedly."
+ return
+ fi
+
+ target_f=`echo $target | _filter_scratch`
+ msg="$target_f: trusted.overlay.metacopy: No such attribute"
+
+ [ "$out_f" == "$msg" ] && return
+
+ echo "Error while checking xattr on ${target}. stdout=$out"
+}
+
+# Check redirect xattr
+check_redirect()
+{
+ local target=$1
+ local expect=$2
+
+ value=$(_getfattr --absolute-names --only-values -n \
+ $OVL_XATTR_REDIRECT $target)
+
+ [[ "$value" == "$expect" ]] || echo "Redirect xattr incorrect. Expected=\"$expect\", actual=\"$value\""
+}
+
+# Check size
+check_file_size()
+{
+ local target=$1 expected_size=$2 actual_size
+
+ actual_size=$(_get_filesize $target)
+
+ [ "$actual_size" == "$expected_size" ] || echo "Expected file size $expected_size but actual size is $actual_size"
+}
+
+check_file_blocks()
+{
+ local target=$1 expected_blocks=$2 nr_blocks
+
+ nr_blocks=$(stat -c "%b" $target)
+
+ [ "$nr_blocks" == "$expected_blocks" ] || echo "Expected $expected_blocks blocks but actual number of blocks is ${nr_blocks}."
+}
+
+check_file_contents()
+{
+ local target=$1 expected=$2
+ local actual target_f
+
+ target_f=`echo $target | _filter_scratch`
+
+ read actual<$target
+
+ [ "$actual" == "$expected" ] || echo "Expected file $target_f contents to be \"$expected\" but actual contents are \"$actual\""
+}
+
+check_no_file_contents()
+{
+ local target=$1
+ local actual target_f out_f
+
+ target_f=`echo $target | _filter_scratch`
+ out_f=`cat $target 2>&1 | _filter_scratch`
+ msg="cat: $target_f: No such file or directory"
+
+ [ "$out_f" == "$msg" ] && return
+
+ echo "$target_f unexpectedly has content"
+}
+
+
+check_file_size_contents()
+{
+ local target=$1 expected_size=$2 expected_content=$3
+
+ check_file_size $target $expected_size
+ check_file_contents $target $expected_content
+}
+
+mount_overlay()
+{
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
+
+ $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+ -o"lowerdir+=$_lowerdir,datadir+=$_datadir2,datadir+=$_datadir" \
+ -o"upperdir=$upperdir,workdir=$workdir" \
+ -o redirect_dir=on,metacopy=on
+}
+
+mount_ro_overlay()
+{
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
+
+ $MOUNT_PROG -t overlay $OVL_BASE_SCRATCH_MNT $SCRATCH_MNT \
+ -o"lowerdir+=$_lowerdir,datadir+=$_datadir2,datadir+=$_datadir" \
+ -o redirect_dir=follow,metacopy=on
+}
+
+umount_overlay()
+{
+ $UMOUNT_PROG $SCRATCH_MNT
+}
+
+test_no_access()
+{
+ local _target=$1
+
+ mount_ro_overlay "$lowerdir" "$datadir2" "$datadir"
+
+ stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \
+ echo "No access to lowerdata layer $_target"
+
+ echo "Unmount and Mount rw"
+ umount_overlay
+ mount_overlay "$lowerdir" "$datadir2" "$datadir"
+ stat $SCRATCH_MNT/$_target >> $seqres.full 2>&1 || \
+ echo "No access to lowerdata layer $_target"
+ umount_overlay
+}
+
+test_common()
+{
+ local _lowerdir=$1 _datadir2=$2 _datadir=$3
+ local _target=$4 _size=$5 _blocks=$6 _data="$7"
+ local _redirect=$8
+
+ echo "Mount ro"
+ mount_ro_overlay $_lowerdir $_datadir2 $_datadir
+
+ # Check redirect xattr to lowerdata
+ [ -n "$_redirect" ] && check_redirect $lowerdir/$_target "$_redirect"
+
+ echo "check properties of metadata copied up file $_target"
+ check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
+ check_file_blocks $SCRATCH_MNT/$_target $_blocks
+
+ # Do a mount cycle and check size and contents again.
+ echo "Unmount and Mount rw"
+ umount_overlay
+ mount_overlay $_lowerdir $_datadir2 $_datadir
+ echo "check properties of metadata copied up file $_target"
+ check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
+ check_file_blocks $SCRATCH_MNT/$_target $_blocks
+
+ # Trigger metadata copy up and check existence of metacopy xattr.
+ chmod 400 $SCRATCH_MNT/$_target
+ umount_overlay
+ check_metacopy $upperdir/$_target "y"
+ check_file_size_contents $upperdir/$_target $_size ""
+
+ # Trigger data copy up and check absence of metacopy xattr.
+ mount_overlay $_lowerdir $_datadir2 $_datadir
+ $XFS_IO_PROG -c "falloc 0 1" $SCRATCH_MNT/$_target >> $seqres.full
+ echo "check properties of data copied up file $_target"
+ check_file_size_contents $SCRATCH_MNT/$_target $_size "$_data"
+ umount_overlay
+ check_metacopy $upperdir/$_target "n"
+ check_file_size_contents $upperdir/$_target $_size "$_data"
+}
+
+test_lazy()
+{
+ local _target=$1
+
+ mount_overlay "$lowerdir" "$datadir2" "$datadir"
+
+ # Metadata should be valid
+ check_file_size $SCRATCH_MNT/$_target $datasize
+ check_file_blocks $SCRATCH_MNT/$_target $estimated_datablocks
+
+ # But have no content
+ check_no_file_contents $SCRATCH_MNT/$_target
+
+ umount_overlay
+}
+
+create_basic_files()
+{
+ _scratch_mkfs
+ mkdir -p $datadir/subdir $datadir2/subdir $lowerdir $lowerdir2 $upperdir $workdir $workdir2
+ mkdir -p $upperdir/$udirname
+ echo "$datacontent" > $datadir/$dataname
+ chmod 600 $datadir/$dataname
+ echo "$datacontent2" > $datadir2/$dataname2
+ chmod 600 $datadir2/$dataname2
+
+ echo "$datacontent" > $datadir/$sharedname
+ echo "$datacontent2" > $datadir2/$sharedname
+ chmod 600 $datadir/$sharedname $datadir2/$sharedname
+
+ # Create files of size datasize.
+ for f in $datadir/$dataname $datadir2/$dataname2 $datadir/$sharedname $datadir2/$sharedname; do
+ $XFS_IO_PROG -c "falloc 0 $datasize" $f
+ $XFS_IO_PROG -c "fsync" $f
+ done
+
+ # Query number of block
+ datablocks=$(stat -c "%b" $datadir/$dataname)
+
+ # For lazy lookup file the block count is estimated based on size and block size
+ datablocksize=$(stat -c "%B" $datadir/$dataname)
+ estimated_datablocks=$(( ($datasize + $datablocksize - 1)/$datablocksize ))
+}
+
+prepare_midlayer()
+{
+ local _redirect=$1
+
+ _scratch_mkfs
+ create_basic_files
+ if [ -n "$_redirect" ]; then
+ mv "$datadir/$dataname" "$datadir/$_redirect"
+ mv "$datadir2/$dataname2" "$datadir2/$_redirect.2"
+ mv "$datadir/$sharedname" "$datadir/$_redirect.shared"
+ mv "$datadir2/$sharedname" "$datadir2/$_redirect.shared"
+ fi
+ # Create midlayer
+ _overlay_scratch_mount_dirs $datadir2:$datadir $lowerdir $workdir2 -o redirect_dir=on,index=on,metacopy=on
+ # Trigger a metacopy with or without redirect
+ if [ -n "$_redirect" ]; then
+ mv "$SCRATCH_MNT/$_redirect" "$SCRATCH_MNT/$dataname"
+ mv "$SCRATCH_MNT/$_redirect.2" "$SCRATCH_MNT/$dataname2"
+ mv "$SCRATCH_MNT/$_redirect.shared" "$SCRATCH_MNT/$sharedname"
+ else
+ chmod 400 $SCRATCH_MNT/$dataname
+ chmod 400 $SCRATCH_MNT/$dataname2
+ chmod 400 $SCRATCH_MNT/$sharedname
+ fi
+ umount_overlay
+}
+
+# Create test directories
+datadir=$OVL_BASE_SCRATCH_MNT/data
+datadir2=$OVL_BASE_SCRATCH_MNT/data2
+lowerdir=$OVL_BASE_SCRATCH_MNT/lower
+upperdir=$OVL_BASE_SCRATCH_MNT/upper
+workdir=$OVL_BASE_SCRATCH_MNT/workdir
+workdir2=$OVL_BASE_SCRATCH_MNT/workdir2
+
+echo -e "\n== Check no follow to lowerdata layer without redirect =="
+prepare_midlayer
+test_no_access "$dataname"
+test_no_access "$dataname2"
+test_no_access "$sharedname"
+
+echo -e "\n== Check no follow to lowerdata layer with relative redirect =="
+prepare_midlayer "$dataname.renamed"
+test_no_access "$dataname"
+test_no_access "$dataname2"
+test_no_access "$sharedname"
+
+echo -e "\n== Check follow to lowerdata layer with absolute redirect =="
+prepare_midlayer "/subdir/$dataname"
+test_common "$lowerdir" "$datadir2" "$datadir" "$dataname" $datasize $datablocks \
+ "$datacontent" "/subdir/$dataname"
+test_common "$lowerdir" "$datadir2" "$datadir" "$dataname2" $datasize $datablocks \
+ "$datacontent2" "/subdir/$dataname.2"
+# Shared file should be picked from upper datadir
+test_common "$lowerdir" "$datadir2" "$datadir" "$sharedname" $datasize $datablocks \
+ "$datacontent2" "/subdir/$dataname.shared"
+
+echo -e "\n== Check lazy follow to lowerdata layer =="
+
+prepare_midlayer "/subdir/$dataname"
+rm $datadir/subdir/$dataname
+test_lazy $dataname
+
+
+# success, all done
+status=0
+exit
diff --git a/tests/overlay/085.out b/tests/overlay/085.out
new file mode 100644
index 00000000..4b9b2d7c
--- /dev/null
+++ b/tests/overlay/085.out
@@ -0,0 +1,42 @@
+QA output created by 085
+
+== Check no follow to lowerdata layer without redirect ==
+No access to lowerdata layer datafile
+Unmount and Mount rw
+No access to lowerdata layer datafile
+No access to lowerdata layer datafile2
+Unmount and Mount rw
+No access to lowerdata layer datafile2
+No access to lowerdata layer shared
+Unmount and Mount rw
+No access to lowerdata layer shared
+
+== Check no follow to lowerdata layer with relative redirect ==
+No access to lowerdata layer datafile
+Unmount and Mount rw
+No access to lowerdata layer datafile
+No access to lowerdata layer datafile2
+Unmount and Mount rw
+No access to lowerdata layer datafile2
+No access to lowerdata layer shared
+Unmount and Mount rw
+No access to lowerdata layer shared
+
+== Check follow to lowerdata layer with absolute redirect ==
+Mount ro
+check properties of metadata copied up file datafile
+Unmount and Mount rw
+check properties of metadata copied up file datafile
+check properties of data copied up file datafile
+Mount ro
+check properties of metadata copied up file datafile2
+Unmount and Mount rw
+check properties of metadata copied up file datafile2
+check properties of data copied up file datafile2
+Mount ro
+check properties of metadata copied up file shared
+Unmount and Mount rw
+check properties of metadata copied up file shared
+check properties of data copied up file shared
+
+== Check lazy follow to lowerdata layer ==
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH v2 4/4] overlay: test parsing of lowerdir+,datadir+ mount options
2023-12-04 18:58 [PATCH v2 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
` (2 preceding siblings ...)
2023-12-04 18:58 ` [PATCH v2 3/4] overlay: test data-only lowerdirs with datadir+ mount option Amir Goldstein
@ 2023-12-04 18:58 ` Amir Goldstein
2023-12-09 4:49 ` [PATCH v2 0/4] Overlayfs tests for 6.7-rc1 Zorro Lang
4 siblings, 0 replies; 13+ messages in thread
From: Amir Goldstein @ 2023-12-04 18:58 UTC (permalink / raw)
To: Zorro Lang; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
Fork test overlay/083 to test parsing of lowerdir+,datadir+ mount options.
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
tests/overlay/086 | 81 +++++++++++++++++++++++++++++++++++++++++++
tests/overlay/086.out | 2 ++
2 files changed, 83 insertions(+)
create mode 100755 tests/overlay/086
create mode 100644 tests/overlay/086.out
diff --git a/tests/overlay/086 b/tests/overlay/086
new file mode 100755
index 00000000..b5960517
--- /dev/null
+++ b/tests/overlay/086
@@ -0,0 +1,81 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2023 CTERA Networks. All Rights Reserved.
+#
+# FS QA Test 086
+#
+# Test lowerdir+,datadir+ mount option restrictions.
+#
+
+. ./common/preamble
+_begin_fstest auto quick mount
+
+# Import common functions.
+. ./common/filter
+
+# real QA test starts here
+_supported_fs overlay
+
+# _overlay_check_* helpers do not handle special chars well
+_require_scratch_nocheck
+_require_scratch_overlay_lowerdir_add_layers
+
+# Remove all files from previous tests
+_scratch_mkfs
+
+# Create lowerdirs with special characters
+lowerdir_spaces="$OVL_BASE_SCRATCH_MNT/lower1 with spaces"
+lowerdir_colons="$OVL_BASE_SCRATCH_MNT/lower2:with::colons"
+lowerdir_colons_esc="$OVL_BASE_SCRATCH_MNT/lower2\:with\:\:colons"
+lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER
+upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER
+workdir=$OVL_BASE_SCRATCH_MNT/$OVL_WORK
+mkdir -p "$lowerdir_spaces" "$lowerdir_colons"
+
+# _overlay_mount_* helpers do not handle lowerdir+,datadir+, so execute mount directly.
+
+# check illegal combinations and order of lowerdir,lowerdir+,datadir+
+$MOUNT_PROG -t overlay none $SCRATCH_MNT \
+ -o"lowerdir=$lowerdir,lowerdir+=$lowerdir_colons" \
+ 2>> $seqres.full && \
+ echo "ERROR: invalid combination of lowerdir and lowerdir+ mount options"
+
+$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
+
+$MOUNT_PROG -t overlay none $SCRATCH_MNT \
+ -o"lowerdir=$lowerdir,datadir+=$lowerdir_colons" \
+ -o redirect_dir=follow,metacopy=on 2>> $seqres.full && \
+ echo "ERROR: invalid combination of lowerdir and datadir+ mount options"
+
+$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
+
+$MOUNT_PROG -t overlay none $SCRATCH_MNT \
+ -o"datadir+=$lowerdir,lowerdir+=$lowerdir_colons" \
+ -o redirect_dir=follow,metacopy=on 2>> $seqres.full && \
+ echo "ERROR: invalid order of lowerdir+ and datadir+ mount options"
+
+$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
+
+# mount is expected to fail with escaped colons.
+$MOUNT_PROG -t overlay none $SCRATCH_MNT \
+ -o"lowerdir+=$lowerdir_colons_esc" \
+ 2>> $seqres.full && \
+ echo "ERROR: incorrect parsing of escaped colons in lowerdir+ mount option"
+
+$UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
+
+# mount is expected to succeed without escaped colons.
+$MOUNT_PROG -t overlay ovl_esc_test $SCRATCH_MNT \
+ -o"lowerdir+=$lowerdir_colons,datadir+=$lowerdir_spaces" \
+ -o redirect_dir=follow,metacopy=on \
+ 2>&1 | tee -a $seqres.full
+
+# if spaces are not escaped when showing mount options,
+# mount command will not show the word 'spaces' after the spaces
+$MOUNT_PROG -t overlay | grep ovl_esc_test | tee -a $seqres.full | \
+ grep -q 'datadir+'.*spaces || \
+ echo "ERROR: escaped spaces truncated from datadir+ mount option"
+
+echo "Silence is golden"
+status=0
+exit
diff --git a/tests/overlay/086.out b/tests/overlay/086.out
new file mode 100644
index 00000000..b34758fd
--- /dev/null
+++ b/tests/overlay/086.out
@@ -0,0 +1,2 @@
+QA output created by 086
+Silence is golden
--
2.34.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH v2 0/4] Overlayfs tests for 6.7-rc1
2023-12-04 18:58 [PATCH v2 0/4] Overlayfs tests for 6.7-rc1 Amir Goldstein
` (3 preceding siblings ...)
2023-12-04 18:58 ` [PATCH v2 4/4] overlay: test parsing of lowerdir+,datadir+ mount options Amir Goldstein
@ 2023-12-09 4:49 ` Zorro Lang
4 siblings, 0 replies; 13+ messages in thread
From: Zorro Lang @ 2023-12-09 4:49 UTC (permalink / raw)
To: Amir Goldstein; +Cc: Alexander Larsson, Miklos Szeredi, linux-unionfs, fstests
On Mon, Dec 04, 2023 at 08:58:55PM +0200, Amir Goldstein wrote:
> Zorro,
>
> This update contains 3 new overlayfs tests for new features added
> in v6.7-rc1.
>
> overlay/084, written by Alexander, tests the new nested xattrs feature.
> overlay/{085,086} test the new lowerdir+,datadir+ mount options.
>
> overlay/086 was partly forked from overlay/083, but overlay/083 is not
> sensitive to libmount version, because the escaped commas test is not
> related to any specific mount option, so it wasn't copied over.
>
> All the new tests do not run on older kernels.
>
> Thanks,
> Amir.
>
> Changed since v1:
> - Helper _require_scratch_overlay_xattr_escapes() already added by
> "overlay/026: Fix test expectation for newer kernels"
This version looks good to me, let's have this feature test coverage
at first.
Reviewed-by: Zorro Lang <zlang@redhat.com>
>
> Amir Goldstein (4):
> overlay: Add tests for nesting private xattrs
> overlay: prepare for new lowerdir+,datadir+ tests
> overlay: test data-only lowerdirs with datadir+ mount option
> overlay: test parsing of lowerdir+,datadir+ mount options
>
> common/overlay | 15 ++
> tests/overlay/079 | 36 +++--
> tests/overlay/084 | 169 +++++++++++++++++++++
> tests/overlay/084.out | 61 ++++++++
> tests/overlay/085 | 332 ++++++++++++++++++++++++++++++++++++++++++
> tests/overlay/085.out | 42 ++++++
> tests/overlay/086 | 81 +++++++++++
> tests/overlay/086.out | 2 +
> 8 files changed, 723 insertions(+), 15 deletions(-)
> create mode 100755 tests/overlay/084
> create mode 100644 tests/overlay/084.out
> create mode 100755 tests/overlay/085
> create mode 100644 tests/overlay/085.out
> create mode 100755 tests/overlay/086
> create mode 100644 tests/overlay/086.out
>
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 13+ messages in thread