[PATCHBOMB] fstests: random fixes and 6.14 stragglers

[PATCHBOMB] fstests: random fixes and 6.14 stragglers

[Darrick J. Wong]

Hi everyone,

Here's a fairly small series of patches for fstests -- the first two add
functional testing for some new utilities that went into xfsprogs 6.14,
and the last one is all bug fixes.

--D

[PATCHSET 1/3] fstests: test dumping fs directory trees

[Darrick J. Wong]

Hi all,

Here's a functional test for recovering parts or entire filesystem
trees with xfs_db.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=rdump

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=rdump
---
Commits in this patchset:
 * xfs: test filesystem recovery with rdump
---
 tests/xfs/1895     |  153 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1895.out |    6 ++
 2 files changed, 159 insertions(+)
 create mode 100755 tests/xfs/1895
 create mode 100644 tests/xfs/1895.out

[PATCH 1/1] xfs: test filesystem recovery with rdump

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Test how well we can dump a fully populated filesystem's contents.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 tests/xfs/1895     |  153 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1895.out |    6 ++
 2 files changed, 159 insertions(+)
 create mode 100755 tests/xfs/1895
 create mode 100644 tests/xfs/1895.out


diff --git a/tests/xfs/1895 b/tests/xfs/1895
new file mode 100755
index 00000000000000..18b534d328e9fd
--- /dev/null
+++ b/tests/xfs/1895
@@ -0,0 +1,153 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Oracle, Inc.  All Rights Reserved.
+#
+# FS QA Test No. 1895
+#
+# Populate a XFS filesystem, ensure that rdump can "recover" the contents to
+# another directory, and compare the contents.
+#
+. ./common/preamble
+_begin_fstest auto scrub
+
+_cleanup()
+{
+	command -v _kill_fsstress &>/dev/null && _kill_fsstress
+	cd /
+	test -e "$testfiles" && rm -r -f $testfiles
+}
+
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+_require_xfs_db_command "rdump"
+_require_test
+_require_scratch
+_require_scrub
+_require_populate_commands
+
+make_md5()
+{
+	(cd $1 ; find . -type f -print0 | xargs -0 md5sum) > $tmp.md5.$2
+}
+
+cmp_md5()
+{
+	(cd $1 ; md5sum --quiet -c $tmp.md5.$2)
+}
+
+make_stat()
+{
+	# columns:	raw mode in hex,
+	# 		major rdev for special
+	# 		minor rdev for special
+	# 		uid of owner
+	# 		gid of owner
+	# 		file type
+	# 		total size
+	# 		mtime
+	# 		name
+	# We can't directly control directory sizes so filter them.
+	# Too many things can bump (or not) atime so don't test that.
+	(cd $1 ; find . -print0 |
+		xargs -0 stat -c '%f %t:%T %u %g %F %s %Y %n' |
+		sed -e 's/ directory [1-9][0-9]* / directory SIZE /g' |
+		sort) > $tmp.stat.$2
+}
+
+cmp_stat()
+{
+	diff -u $tmp.stat.$1 $tmp.stat.$2
+}
+
+make_stat_files() {
+	for file in "${FILES[@]}"; do
+		find "$1/$file" -print0 | xargs -0 stat -c '%f %t:%T %u %g %F %s %Y %n'
+	done | sed \
+		-e 's/ directory [1-9][0-9]* / directory SIZE /g' \
+		-e "s| $1| DUMPDIR|g" \
+		| sort > $tmp.stat.files.$2
+}
+
+cmp_stat_files()
+{
+	diff -u $tmp.stat.files.$1 $tmp.stat.files.$2
+}
+
+make_stat_dir() {
+	find "$1" -print0 | \
+		xargs -0 stat -c '%f %t:%T %u %g %F %s %Y %n' | sed \
+		-e 's/ directory [1-9][0-9]* / directory SIZE /g' \
+		-e "s| $1| DUMPDIR|g" \
+		| sort > $tmp.stat.dir.$2
+}
+
+cmp_stat_dir()
+{
+	diff -u $tmp.stat.dir.$1 $tmp.stat.dir.$2
+}
+
+FILES=(
+	"/S_IFDIR.FMT_INLINE"
+	"/S_IFBLK"
+	"/S_IFCHR"
+	"/S_IFLNK.FMT_LOCAL"
+	"/S_IFIFO"
+	"/S_IFDIR.FMT_INLINE/00000001"
+	"/ATTR.FMT_EXTENTS_REMOTE3K"
+	"/S_IFREG.FMT_EXTENTS"
+	"/S_IFREG.FMT_BTREE"
+	"/BNOBT"
+	"/S_IFDIR.FMT_BLOCK"
+)
+DIR="/S_IFDIR.FMT_LEAF"
+
+testfiles=$TEST_DIR/$seq
+mkdir -p $testfiles
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+_scratch_mount
+
+_run_fsstress -n 500 -d $SCRATCH_MNT/newfiles
+
+make_stat $SCRATCH_MNT before
+make_md5 $SCRATCH_MNT before
+make_stat_files $SCRATCH_MNT before
+make_stat_dir $SCRATCH_MNT/$DIR before
+_scratch_unmount
+
+echo "Recover filesystem"
+dumpdir1=$testfiles/rdump
+dumpdir2=$testfiles/sdump
+dumpdir3=$testfiles/tdump
+rm -r -f $dumpdir1 $dumpdir2 $dumpdir3
+
+# as of linux 6.12 fchownat does not work on symlinks
+_scratch_xfs_db -c "rdump / $dumpdir1" | sed -e '/could not be set/d'
+_scratch_xfs_db -c "rdump ${FILES[*]} $dumpdir2" | sed -e '/could not be set/d'
+_scratch_xfs_db -c "rdump $DIR $dumpdir3" | sed -e '/could not be set/d'
+
+echo "Check file contents"
+make_stat $dumpdir1 after
+cmp_stat before after
+cmp_md5 $dumpdir1 before
+
+echo "Check selected files contents"
+make_stat_files $dumpdir2 after
+cmp_stat_files before after
+
+echo "Check single dir extraction contents"
+make_stat_dir $dumpdir3 after
+cmp_stat_dir before after
+
+# remount so we can check this fs
+_scratch_mount
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1895.out b/tests/xfs/1895.out
new file mode 100644
index 00000000000000..de639ed3fc7e38
--- /dev/null
+++ b/tests/xfs/1895.out
@@ -0,0 +1,6 @@
+QA output created by 1895
+Format and populate
+Recover filesystem
+Check file contents
+Check selected files contents
+Check single dir extraction contents

[PATCHSET 2/3] fstests: test handling emoji filenames better

[Darrick J. Wong]

Hi all,

Ted told me about some bugs that the ext4 Unicode casefolding code has
suffered over the past year -- they tried stripping out zero width
joiner (ZWJ) codepoints to try to eliminate casefolded lookup comparison
issues, but doing so corrupts compound emoji handling in filenames.

This short series amends the Unicode testing in generic/453 and 454 to
examine these compound emoji.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-emoji-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=scrub-emoji-fixes
---
Commits in this patchset:
 * generic/45[34]: add colored emoji variants to unicode tests
---
 tests/generic/453 |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/454 |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 110 insertions(+)

[PATCH 1/1] generic/45[34]: add colored emoji variants to unicode tests

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Ted told me this morning about a recent problem with kernel Unicode name
casefolding vs. emoji -- initially, someone decided that zero-width
joiners should be stripped out of filenames during comparisons, which
lead to malicious git pulls of branches containing "<zwj>.git/config"
files overwriting git repo config files.  A quick fix was to stop
ignoring the "ignorable" code points, but that broke emoji in filenames,
because emoji use zero-width joiners to combine simpler emoji into more
complex ones, or alter skin tones, or colors, etc.  Reportedly the
casefolding code will also fold a red heart into a black one.

So.  To our filename support test, let's add various colors of heart
emoji and various skin tones of heart-hands; and compound emoji
consisting of multiple emoji glued together with zero width joiners.
This actually caused a buffer overflow in the string-escaping functions
of xfs_scrub phase 5 because I hadn't anticipated that we'd end up with
a filename consisting *entirely* of nonprinting bytes.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 tests/generic/453 |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/454 |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 110 insertions(+)


diff --git a/tests/generic/453 b/tests/generic/453
index 04945ad1085b2d..bd5ce8b2bb11d9 100755
--- a/tests/generic/453
+++ b/tests/generic/453
@@ -203,6 +203,36 @@ setf "job offer.pdf" "actual period"
 setf "llamapirate\xf3\xa0\x80\x81\xf3\xa0\x81\x94\xf3\xa0\x81\xa8\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb3\xf3\xa0\x81\xa1\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x81\xb3\xf3\xa0\x80\xa0\xf3\xa0\x81\xa6\xf3\xa0\x81\xaf\xf3\xa0\x81\xb2\xf3\xa0\x80\xa0\xf3\xa0\x81\x93\xf3\xa0\x81\xa5\xf3\xa0\x81\xa1\xf3\xa0\x81\xb4\xf3\xa0\x81\xb4\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb7\xf3\xa0\x81\xa5\xf3\xa0\x81\xb2\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\x95\xf3\xa0\x81\x93\xf3\xa0\x81\x84\xf3\xa0\x80\xa0\xf3\xa0\x80\xb1\xf3\xa0\x80\xb2\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x81\xbf"
 setf "llamapirate"
 
+# colored heart emoji to check if casefolding whacks emoji
+setf "\xf0\x9f\x92\x9c" "purple"
+setf "\xf0\x9f\x92\x99" "blue"
+setf "\xf0\x9f\x92\x9a" "green"
+setf "\xf0\x9f\x92\x9b" "yellow"
+setf "\xf0\x9f\xab\x80" "heart"
+setf "\xe2\x9d\xa4\xef\xb8\x8f" "red"
+setf "\xf0\x9f\xa4\x8e" "brown"
+setf "\xf0\x9f\xa4\x8d" "white"
+setf "\xf0\x9f\x96\xa4" "black"
+setf "\xf0\x9f\xa7\xa1" "orange"
+setf "\xe2\x99\xa5\xef\xb8\x8f" "red suit"
+
+# zero width joiners exist in the middle of emoji sequences aren't supposed
+# to be normalized to nothing, but apparently this caused issues with
+# casefolding on ext4; also the mending heart caused a crash in xfs_scrub
+setf "\xf0\x9f\x92\x94" "broken heart"
+setf "\xe2\x9d\xa4\xef\xb8\x8f\xe2\x80\x8d\xf0\x9f\xa9\xb9" "mending heart"
+setf "\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbb\xe2\x80\x8d\xe2\x9d\xa4\xef\xb8
+\x8f\xe2\x80\x8d\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbc" "couple with heart"
+setf "\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbb\xe2\x80\x8d\xe2\x9d\xa4\xef\xb8\x8f\xe2\x80\x8d\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbf" "couple with heart, light and dark skin tone"
+
+# emoji heart hands with skin tone variations
+setf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbf" "dark"
+setf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbe" "medium dark"
+setf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbd" "medium"
+setf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbc" "medium light"
+setf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbb" "light"
+setf "\xf0\x9f\xab\xb6" "neutral"
+
 ls -laR $testdir >> $seqres.full
 
 echo "Test files"
@@ -276,6 +306,31 @@ testf "job offer.pdf" "actual period"
 testf "llamapirate\xf3\xa0\x80\x81\xf3\xa0\x81\x94\xf3\xa0\x81\xa8\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb3\xf3\xa0\x81\xa1\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x81\xb3\xf3\xa0\x80\xa0\xf3\xa0\x81\xa6\xf3\xa0\x81\xaf\xf3\xa0\x81\xb2\xf3\xa0\x80\xa0\xf3\xa0\x81\x93\xf3\xa0\x81\xa5\xf3\xa0\x81\xa1\xf3\xa0\x81\xb4\xf3\xa0\x81\xb4\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb7\xf3\xa0\x81\xa5\xf3\xa0\x81\xb2\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\x95\xf3\xa0\x81\x93\xf3\xa0\x81\x84\xf3\xa0\x80\xa0\xf3\xa0\x80\xb1\xf3\xa0\x80\xb2\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x81\xbf"
 testf "llamapirate"
 
+testf "\xf0\x9f\x92\x9c" "purple"
+testf "\xf0\x9f\x92\x99" "blue"
+testf "\xf0\x9f\x92\x9a" "green"
+testf "\xf0\x9f\x92\x9b" "yellow"
+testf "\xf0\x9f\xab\x80" "heart"
+testf "\xe2\x9d\xa4\xef\xb8\x8f" "red"
+testf "\xf0\x9f\xa4\x8e" "brown"
+testf "\xf0\x9f\xa4\x8d" "white"
+testf "\xf0\x9f\x96\xa4" "black"
+testf "\xf0\x9f\xa7\xa1" "orange"
+testf "\xe2\x99\xa5\xef\xb8\x8f" "red suit"
+
+testf "\xf0\x9f\x92\x94" "broken heart"
+testf "\xe2\x9d\xa4\xef\xb8\x8f\xe2\x80\x8d\xf0\x9f\xa9\xb9" "mending heart"
+testf "\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbb\xe2\x80\x8d\xe2\x9d\xa4\xef\xb8
+\x8f\xe2\x80\x8d\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbc" "couple with heart"
+testf "\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbb\xe2\x80\x8d\xe2\x9d\xa4\xef\xb8\x8f\xe2\x80\x8d\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbf" "couple with heart, light and dark skin tone"
+
+testf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbf" "dark"
+testf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbe" "medium dark"
+testf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbd" "medium"
+testf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbc" "medium light"
+testf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbb" "light"
+testf "\xf0\x9f\xab\xb6" "neutral"
+
 echo "Uniqueness of inodes?"
 stat -c '%i' "${testdir}/"* | sort | uniq -c | while read nr inum; do
 	if [ "${nr}" -gt 1 ]; then
diff --git a/tests/generic/454 b/tests/generic/454
index aec8beb8b43ca0..9f6ddb4a0e48b2 100755
--- a/tests/generic/454
+++ b/tests/generic/454
@@ -124,6 +124,36 @@ setf "combmark_\xe1\x80\x9c\xe1\x80\xaf\xe1\x80\xad.txt" "combining marks"
 setf "llamapirate\xf3\xa0\x80\x81\xf3\xa0\x81\x94\xf3\xa0\x81\xa8\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb3\xf3\xa0\x81\xa1\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x81\xb3\xf3\xa0\x80\xa0\xf3\xa0\x81\xa6\xf3\xa0\x81\xaf\xf3\xa0\x81\xb2\xf3\xa0\x80\xa0\xf3\xa0\x81\x93\xf3\xa0\x81\xa5\xf3\xa0\x81\xa1\xf3\xa0\x81\xb4\xf3\xa0\x81\xb4\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb7\xf3\xa0\x81\xa5\xf3\xa0\x81\xb2\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\x95\xf3\xa0\x81\x93\xf3\xa0\x81\x84\xf3\xa0\x80\xa0\xf3\xa0\x80\xb1\xf3\xa0\x80\xb2\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x81\xbf" "secret instructions"
 setf "llamapirate" "no secret instructions"
 
+# colored heart emoji to check if casefolding whacks emoji
+setf "\xf0\x9f\x92\x9c" "purple"
+setf "\xf0\x9f\x92\x99" "blue"
+setf "\xf0\x9f\x92\x9a" "green"
+setf "\xf0\x9f\x92\x9b" "yellow"
+setf "\xf0\x9f\xab\x80" "heart"
+setf "\xe2\x9d\xa4\xef\xb8\x8f" "red"
+setf "\xf0\x9f\xa4\x8e" "brown"
+setf "\xf0\x9f\xa4\x8d" "white"
+setf "\xf0\x9f\x96\xa4" "black"
+setf "\xf0\x9f\xa7\xa1" "orange"
+setf "\xe2\x99\xa5\xef\xb8\x8f" "red suit"
+
+# zero width joiners exist in the middle of emoji sequences aren't supposed
+# to be normalized to nothing, but apparently this caused issues with
+# casefolding on ext4; also the mending heart caused a crash in xfs_scrub
+setf "\xf0\x9f\x92\x94" "broken heart"
+setf "\xe2\x9d\xa4\xef\xb8\x8f\xe2\x80\x8d\xf0\x9f\xa9\xb9" "mending heart"
+setf "\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbb\xe2\x80\x8d\xe2\x9d\xa4\xef\xb8
+\x8f\xe2\x80\x8d\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbc" "couple with heart"
+setf "\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbb\xe2\x80\x8d\xe2\x9d\xa4\xef\xb8\x8f\xe2\x80\x8d\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbf" "couple with heart, light and dark skin tone"
+
+# emoji heart hands with skin tone variations
+setf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbf" "dark"
+setf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbe" "medium dark"
+setf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbd" "medium"
+setf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbc" "medium light"
+setf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbb" "light"
+setf "\xf0\x9f\xab\xb6" "neutral"
+
 _getfattr --absolute-names -d "${testfile}" >> $seqres.full
 
 echo "Test files"
@@ -174,6 +204,31 @@ testf "combmark_\xe1\x80\x9c\xe1\x80\xaf\xe1\x80\xad.txt" "combining marks"
 testf "llamapirate\xf3\xa0\x80\x81\xf3\xa0\x81\x94\xf3\xa0\x81\xa8\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb3\xf3\xa0\x81\xa1\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x81\xb3\xf3\xa0\x80\xa0\xf3\xa0\x81\xa6\xf3\xa0\x81\xaf\xf3\xa0\x81\xb2\xf3\xa0\x80\xa0\xf3\xa0\x81\x93\xf3\xa0\x81\xa5\xf3\xa0\x81\xa1\xf3\xa0\x81\xb4\xf3\xa0\x81\xb4\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb7\xf3\xa0\x81\xa5\xf3\xa0\x81\xb2\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\x95\xf3\xa0\x81\x93\xf3\xa0\x81\x84\xf3\xa0\x80\xa0\xf3\xa0\x80\xb1\xf3\xa0\x80\xb2\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x81\xbf" "secret instructions"
 testf "llamapirate" "no secret instructions"
 
+testf "\xf0\x9f\x92\x9c" "purple"
+testf "\xf0\x9f\x92\x99" "blue"
+testf "\xf0\x9f\x92\x9a" "green"
+testf "\xf0\x9f\x92\x9b" "yellow"
+testf "\xf0\x9f\xab\x80" "heart"
+testf "\xe2\x9d\xa4\xef\xb8\x8f" "red"
+testf "\xf0\x9f\xa4\x8e" "brown"
+testf "\xf0\x9f\xa4\x8d" "white"
+testf "\xf0\x9f\x96\xa4" "black"
+testf "\xf0\x9f\xa7\xa1" "orange"
+testf "\xe2\x99\xa5\xef\xb8\x8f" "red suit"
+
+testf "\xf0\x9f\x92\x94" "broken heart"
+testf "\xe2\x9d\xa4\xef\xb8\x8f\xe2\x80\x8d\xf0\x9f\xa9\xb9" "mending heart"
+testf "\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbb\xe2\x80\x8d\xe2\x9d\xa4\xef\xb8
+\x8f\xe2\x80\x8d\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbc" "couple with heart"
+testf "\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbb\xe2\x80\x8d\xe2\x9d\xa4\xef\xb8\x8f\xe2\x80\x8d\xf0\x9f\xa7\x91\xf0\x9f\x8f\xbf" "couple with heart, light and dark skin tone"
+
+testf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbf" "dark"
+testf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbe" "medium dark"
+testf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbd" "medium"
+testf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbc" "medium light"
+testf "\xf0\x9f\xab\xb6\xf0\x9f\x8f\xbb" "light"
+testf "\xf0\x9f\xab\xb6" "neutral"
+
 echo "Uniqueness of keys?"
 crazy_keys="$(_getfattr --absolute-names -d "${testfile}" | grep -E -c '(french_|chinese_|greek_|arabic_|urk)')"
 expected_keys=11

[PATCHSET 3/3] fstests: more random fixes for v2025.03.09

[Darrick J. Wong]

Hi all,

Here's the usual odd fixes for fstests.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

With a bit of luck, this should all go splendidly.
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
Commits in this patchset:
 * xfs/614: determine the sector size of the fs image by doing a test format
 * generic/537: disable quota mount options for pre-metadir rt filesystems
 * check: don't allow TEST_DIR/SCRATCH_MNT to be in /tmp
---
 check             |   14 ++++++++++++++
 tests/generic/537 |   17 +++++++++++++++++
 tests/xfs/614     |   13 ++++++++-----
 3 files changed, 39 insertions(+), 5 deletions(-)

[PATCH 1/3] xfs/614: determine the sector size of the fs image by doing a test format

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

In some cases (such as xfs always_cow=1), the configuration of the test
filesystem determines the sector size of the filesystem that we're going
to simulate formatting.  Concretely, even if TEST_DEV is a block device
with 512b sectors, the directio geometry can specify 4k writes to avoid
nasty RMW cycles.  When this happens, mkfs.xfs will set the sector size
to that 4k accordingly, but the golden output selection is wrong.  Fix
this.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 tests/xfs/614 |   13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)


diff --git a/tests/xfs/614 b/tests/xfs/614
index 2a799fbf3ed71c..e182f073fddd64 100755
--- a/tests/xfs/614
+++ b/tests/xfs/614
@@ -25,13 +25,16 @@ _require_test
 $MKFS_XFS_PROG 2>&1 | grep -q concurrency || \
 	_notrun "mkfs does not support concurrency options"
 
-test_dev_lbasize=$(blockdev --getss $TEST_DEV)
-seqfull=$0
-_link_out_file "lba${test_dev_lbasize}"
-
+# Figure out what sector size mkfs will use to format, which might be dependent
+# upon the directio write geometry of the test filesystem.
 loop_file=$TEST_DIR/$seq.loop
-
 rm -f "$loop_file"
+truncate -s 16M "$loop_file"
+$MKFS_XFS_PROG -f -N "$loop_file" | _filter_mkfs 2>$tmp.mkfs >/dev/null
+. $tmp.mkfs
+seqfull=$0
+_link_out_file "lba${sectsz}"
+
 for sz in 16M 512M 1G 2G 16G 64G 256G 512G 1T 2T 4T 16T 64T 256T 512T 1P; do
 	for cpus in 2 4 8 16 32 40 64 96 160 512; do
 		truncate -s "$sz" "$loop_file"

[PATCH 2/3] generic/537: disable quota mount options for pre-metadir rt filesystems

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix this regression in generic/537:

mount: /opt: permission denied.
       dmesg(1) may have more information after failed mount system call.
mount -o uquota,gquota,pquota, -o ro,norecovery -ortdev=/dev/sdb4 /dev/sda4 /opt failed
mount -o uquota,gquota,pquota, -o ro,norecovery -ortdev=/dev/sdb4 /dev/sda4 /opt failed
(see /var/tmp/fstests/generic/537.full for details)

for reasons explained in the giant comment.  TLDR: quota and rt aren't
compatible on older xfs filesystems so we have to work around that.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/generic/537 |   17 +++++++++++++++++
 1 file changed, 17 insertions(+)


diff --git a/tests/generic/537 b/tests/generic/537
index f57bc1561dd57e..3be743c4133f4f 100755
--- a/tests/generic/537
+++ b/tests/generic/537
@@ -18,6 +18,7 @@ _begin_fstest auto quick trim
 
 # Import common functions.
 . ./common/filter
+. ./common/quota
 
 _require_scratch
 _require_fstrim
@@ -36,6 +37,22 @@ _scratch_mount -o ro >> $seqres.full 2>&1
 $FSTRIM_PROG -v $SCRATCH_MNT >> $seqres.full 2>&1
 _scratch_unmount
 
+# As of kernel commit 9f0902091c332b ("xfs: Do not allow norecovery mount with
+# quotacheck"), it is no longer possible to mount with "norecovery" and any
+# quota mount option if the quota mount options would require a metadata update
+# such as quotacheck.  For a pre-metadir XFS filesystem with a realtime volume
+# and quota-enabling options, the first two mount attempts will have succeeded
+# but with quotas disabled.  The mount option parsing for this next mount
+# attempt will see the same quota-enabling options and a lack of qflags in the
+# ondisk metadata and reject the mount because it thinks that will require
+# quotacheck.  Edit out the quota mount options for this specific
+# configuration.
+if [ "$FSTYP" = "xfs" ]; then
+	if [ "$USE_EXTERNAL" = "yes" ] && [ -n "$SCRATCH_RTDEV" ]; then
+		_qmount_option ""
+	fi
+fi
+
 echo "fstrim on ro mount with no log replay"
 norecovery="norecovery"
 test $FSTYP = "btrfs" && norecovery=nologreplay

Re: [PATCH 2/3] generic/537: disable quota mount options for pre-metadir rt filesystems

[Darrick J. Wong]

On Wed, Mar 12, 2025 at 04:11:48PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Fix this regression in generic/537:
> 
> mount: /opt: permission denied.
>        dmesg(1) may have more information after failed mount system call.
> mount -o uquota,gquota,pquota, -o ro,norecovery -ortdev=/dev/sdb4 /dev/sda4 /opt failed
> mount -o uquota,gquota,pquota, -o ro,norecovery -ortdev=/dev/sdb4 /dev/sda4 /opt failed
> (see /var/tmp/fstests/generic/537.full for details)
> 
> for reasons explained in the giant comment.  TLDR: quota and rt aren't
> compatible on older xfs filesystems so we have to work around that.
> 
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>

Ping?

--D

> ---
>  tests/generic/537 |   17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
> 
> 
> diff --git a/tests/generic/537 b/tests/generic/537
> index f57bc1561dd57e..3be743c4133f4f 100755
> --- a/tests/generic/537
> +++ b/tests/generic/537
> @@ -18,6 +18,7 @@ _begin_fstest auto quick trim
>  
>  # Import common functions.
>  . ./common/filter
> +. ./common/quota
>  
>  _require_scratch
>  _require_fstrim
> @@ -36,6 +37,22 @@ _scratch_mount -o ro >> $seqres.full 2>&1
>  $FSTRIM_PROG -v $SCRATCH_MNT >> $seqres.full 2>&1
>  _scratch_unmount
>  
> +# As of kernel commit 9f0902091c332b ("xfs: Do not allow norecovery mount with
> +# quotacheck"), it is no longer possible to mount with "norecovery" and any
> +# quota mount option if the quota mount options would require a metadata update
> +# such as quotacheck.  For a pre-metadir XFS filesystem with a realtime volume
> +# and quota-enabling options, the first two mount attempts will have succeeded
> +# but with quotas disabled.  The mount option parsing for this next mount
> +# attempt will see the same quota-enabling options and a lack of qflags in the
> +# ondisk metadata and reject the mount because it thinks that will require
> +# quotacheck.  Edit out the quota mount options for this specific
> +# configuration.
> +if [ "$FSTYP" = "xfs" ]; then
> +	if [ "$USE_EXTERNAL" = "yes" ] && [ -n "$SCRATCH_RTDEV" ]; then
> +		_qmount_option ""
> +	fi
> +fi
> +
>  echo "fstrim on ro mount with no log replay"
>  norecovery="norecovery"
>  test $FSTYP = "btrfs" && norecovery=nologreplay
> 
> 

[PATCH 3/3] check: don't allow TEST_DIR/SCRATCH_MNT to be in /tmp

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

If we're running in a private mount namespace, /tmp is a private tmpfs
mount.  Using TEST_DIR/SCRATCH_MNT that point there is a bad idea
because anyone can write to there.  Let's just stop that.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 check |   14 ++++++++++++++
 1 file changed, 14 insertions(+)


diff --git a/check b/check
index 33eb3e0859e578..09134ee63e41e2 100755
--- a/check
+++ b/check
@@ -815,6 +815,20 @@ function run_section()
 		echo "SECTION       -- $section"
 	fi
 
+	# If we're running in a private mount namespace, /tmp is a private
+	# directory.  We /could/ just mkdir it, but we'd rather have people
+	# set those paths elsewhere.
+	if [ "$HAVE_PRIVATENS" = yes ] && [[ $TEST_DIR =~ ^\/tmp ]]; then
+		echo "$TEST_DIR: TEST_DIR must not be in /tmp"
+		status=1
+		exit
+	fi
+	if [ "$HAVE_PRIVATENS" = yes ] && [[ $SCRATCH_MNT =~ ^\/tmp ]]; then
+		echo "$SCRATCH_MNT: SCRATCH_MNT must not be in /tmp"
+		status=1
+		exit
+	fi
+
 	sect_start=`_wallclock`
 	if $RECREATE_TEST_DEV || [ "$OLD_FSTYP" != "$FSTYP" ]; then
 		echo "RECREATING    -- $FSTYP on $TEST_DEV"

Re: [PATCH 3/3] check: don't allow TEST_DIR/SCRATCH_MNT to be in /tmp

[Darrick J. Wong]

On Wed, Mar 12, 2025 at 04:12:04PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> If we're running in a private mount namespace, /tmp is a private tmpfs

Since we're dropping privatens for ./check, there's no need for this
patch anymore.  I'll roll up and resend this series without this patch
and with the rvbs that have trickled in since then.

--D

> mount.  Using TEST_DIR/SCRATCH_MNT that point there is a bad idea
> because anyone can write to there.  Let's just stop that.
> 
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> ---
>  check |   14 ++++++++++++++
>  1 file changed, 14 insertions(+)
> 
> 
> diff --git a/check b/check
> index 33eb3e0859e578..09134ee63e41e2 100755
> --- a/check
> +++ b/check
> @@ -815,6 +815,20 @@ function run_section()
>  		echo "SECTION       -- $section"
>  	fi
>  
> +	# If we're running in a private mount namespace, /tmp is a private
> +	# directory.  We /could/ just mkdir it, but we'd rather have people
> +	# set those paths elsewhere.
> +	if [ "$HAVE_PRIVATENS" = yes ] && [[ $TEST_DIR =~ ^\/tmp ]]; then
> +		echo "$TEST_DIR: TEST_DIR must not be in /tmp"
> +		status=1
> +		exit
> +	fi
> +	if [ "$HAVE_PRIVATENS" = yes ] && [[ $SCRATCH_MNT =~ ^\/tmp ]]; then
> +		echo "$SCRATCH_MNT: SCRATCH_MNT must not be in /tmp"
> +		status=1
> +		exit
> +	fi
> +
>  	sect_start=`_wallclock`
>  	if $RECREATE_TEST_DEV || [ "$OLD_FSTYP" != "$FSTYP" ]; then
>  		echo "RECREATING    -- $FSTYP on $TEST_DEV"
> 
> 

[PATCH 4/3] common/populate: drop fallocate mode 0 requirement

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

None of the _scratch_$FSTYP_populate functions use fallocate mode 0 (aka
preallocation) to run, so drop the _require check.  This enables xfs/349
and friends to work on always-cow xfs filesystems.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/populate |    1 -
 1 file changed, 1 deletion(-)

diff --git a/common/populate b/common/populate
index a1be26d5b24adf..7352f598a0c700 100644
--- a/common/populate
+++ b/common/populate
@@ -8,7 +8,6 @@
 . ./common/quota
 
 _require_populate_commands() {
-	_require_xfs_io_command "falloc"
 	_require_xfs_io_command "fpunch"
 	_require_test_program "punch-alternating"
 	_require_test_program "popdir.pl"

Re: [PATCH 4/3] common/populate: drop fallocate mode 0 requirement

[Christoph Hellwig]

On Fri, Mar 14, 2025 at 10:54:49AM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> None of the _scratch_$FSTYP_populate functions use fallocate mode 0 (aka
> preallocation) to run, so drop the _require check.  This enables xfs/349
> and friends to work on always-cow xfs filesystems.

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 5/3] xfs/818: fix some design issues

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

While QA'ing zoned filesystem support, I discovered some design errors
in this test:

1) Since we're test formatting a sparse file on an xfs filesystem,
there's no need to play games with optimal device size; we can create
a totally sparse file that's the same size as SCRATCH_DEV.

2) mkfs.xfs cannot create realtime files, so if it fails with that,
there's no need to continue the test.

3) If mkfs -p fails for none of the proscribed reasons, it should exit
the test.  The final cat $tmp.mkfs will take care of tweaking the golden
output to register the test failure for further investigation.

Cc: <fstests@vger.kernel.org> # v2025.03.09
Fixes: 6d39dc34e61e11 ("xfs: test filesystem creation with xfs_protofile")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/818 |   19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/tests/xfs/818 b/tests/xfs/818
index aeb462353df7e9..bc809390b9e340 100755
--- a/tests/xfs/818
+++ b/tests/xfs/818
@@ -75,9 +75,8 @@ _run_fsstress -n 1000 -d $SCRATCH_MNT/newfiles
 make_stat $SCRATCH_MNT before
 make_md5 $SCRATCH_MNT before
 
-kb_needed=$(du -k -s $SCRATCH_MNT | awk '{print $1}')
-img_size=$((kb_needed * 2))
-test "$img_size" -lt $((300 * 1024)) && img_size=$((300 * 1024))
+scratch_sectors="$(blockdev --getsz $SCRATCH_DEV)"
+img_size=$((scratch_sectors * 512 / 1024))
 
 echo "Clone image with protofile"
 $XFS_PROTOFILE_PROG $SCRATCH_MNT > $testfiles/protofile
@@ -99,7 +98,21 @@ if ! _try_mkfs_dev -p $testfiles/protofile $testfiles/image &> $tmp.mkfs; then
 	if grep -q 'No space left on device' $tmp.mkfs; then
 		_notrun "not enough space in filesystem"
 	fi
+
+	# mkfs cannot create realtime files.
+	#
+	# If zoned=1 is in MKFS_OPTIONS, mkfs will create an internal realtime
+	# volume with rtinherit=1 and fail, so we need to _notrun that case.
+	#
+	# If zoned=1 is /not/ in MKFS_OPTIONS, we didn't pass a realtime device
+	# to mkfs so it will not create realtime files.  The format should work
+	# just fine.
+	if grep -q 'creating realtime files from proto file not supported' $tmp.mkfs; then
+		_notrun "mkfs cannot create realtime files"
+	fi
+
 	cat $tmp.mkfs
+	exit
 fi
 
 _mount $testfiles/image $testfiles/mount

Re: [PATCH 5/3] xfs/818: fix some design issues

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>