[PATCHBOMB] fstests: fix check-parallel and get all the xfs 6.13 changes merged

[PATCHBOMB] fstests: fix check-parallel and get all the xfs 6.13 changes merged

[Darrick J. Wong]

Hi everyone,

This patchbomb starts with a very large set of bug fixes that try to fix
all the problems that I found in last year's overhastily merged
check-parallel patchset.  There's really too many of them to get into
here.

Next comes some cleanups for the logwrites code to try to reduce the
instances of people writing logwrites replay tests that break on xfs due
to weird subtleties.

After that are new testscases for the metadata directory tree, realtime
allocation group, realtime quota, and persistent quota options features
that landed in xfs in 6.13.

None of these patches have been reviewed, but I thank you all for your
continued attention.

--Darrick

[PATCHSET 1/7] fstests: random fixes for v2025.01.12

[Darrick J. Wong]

Hi all,

This is an unusually large set of bug fixes for fstests.  The first 20
patches in this patchset are corrections for that RFC series.

The most significant change is that I made ./check run each test with
its own Unix process session id.  This means that a test can use pkill
to kill all of its own subprocesses, without killing anyone else's
subprocesses.  I'm not completely sold on that approach, but it works
for me.  It seems to work for check-parallel as well, though I've not
tested that all that much.

Note: I am /not/ happy about Dave's RFC going straight to for-next
without even a complete review right before everyone went on PTO for
several weeks for xmas/solar new year.  But in the interests of getting
QA back on line for myself and everyone else who's having problems, here
it is.

The last two patches are bugs that have lurked in fstests for ages.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

With a bit of luck, this should all go splendidly.
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=random-fixes

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=random-fixes

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
---
Commits in this patchset:
 * generic/476: fix fsstress process management
 * metadump: make non-local function variables more obvious
 * metadump: fix cleanup for v1 metadump testing
 * generic/482: _run_fsstress needs the test filesystem
 * generic/019: don't fail if fio crashes while shutting down
 * fuzzy: do not set _FSSTRESS_PID when exercising fsx
 * common/rc: create a wrapper for the su command
 * common: fix pkill by running test program in a separate session
 * unmount: resume logging of stdout and stderr for filtering
 * mkfs: don't hardcode log size
 * common/xfs: find loop devices for non-blockdevs passed to _prepare_for_eio_shutdown
 * preamble: fix missing _kill_fsstress
 * generic/650: revert SOAK DURATION changes
 * generic/032: fix pinned mount failure
 * fuzzy: stop __stress_scrub_fsx_loop if fsx fails
 * fuzzy: don't use readarray for xfsfind output
 * fuzzy: always stop the scrub fsstress loop on error
 * fuzzy: port fsx and fsstress loop to use --duration
 * common/rc: don't copy fsstress to $TEST_DIR
 * fix _require_scratch_duperemove ordering
 * fsstress: fix a memory leak
 * fsx: fix leaked log file pointer
 * build: initialize stack variables to zero by default
---
 check                 |   43 +++++++++++++++++++---
 common/fuzzy          |   95 ++++++++++++++++++++++++++++++++-----------------
 common/metadump       |   42 ++++++++++------------
 common/preamble       |    2 +
 common/quota          |    2 +
 common/rc             |   93 +++++++++++++++++++++++++++++++++++++++++-------
 common/reflink        |    6 ++-
 common/xfs            |    6 +++
 configure.ac          |    1 +
 include/builddefs.in  |    3 +-
 ltp/fsstress.c        |    1 +
 ltp/fsx.c             |    1 +
 m4/package_libcdev.m4 |   14 +++++++
 src/xfsfind.c         |   14 ++++++-
 tests/generic/019     |    2 +
 tests/generic/032     |    9 +++++
 tests/generic/050     |    2 +
 tests/generic/085     |    2 +
 tests/generic/093     |    2 +
 tests/generic/125     |    2 +
 tests/generic/128     |    2 +
 tests/generic/193     |   36 +++++++++----------
 tests/generic/230     |   14 ++++---
 tests/generic/231     |    2 +
 tests/generic/233     |    2 +
 tests/generic/270     |   12 ++++--
 tests/generic/310     |    6 ++-
 tests/generic/314     |    2 +
 tests/generic/327     |    2 +
 tests/generic/328     |    4 +-
 tests/generic/355     |    2 +
 tests/generic/361     |    4 +-
 tests/generic/453     |    6 ++-
 tests/generic/476     |    2 +
 tests/generic/514     |    2 +
 tests/generic/530     |    6 +--
 tests/generic/531     |    6 +--
 tests/generic/561     |    2 +
 tests/generic/573     |    2 +
 tests/generic/590     |    2 +
 tests/generic/600     |    2 +
 tests/generic/601     |    2 +
 tests/generic/603     |   10 +++--
 tests/generic/650     |    5 +--
 tests/generic/673     |    2 +
 tests/generic/674     |    2 +
 tests/generic/675     |    2 +
 tests/generic/680     |    2 +
 tests/generic/681     |    2 +
 tests/generic/682     |    2 +
 tests/generic/683     |    2 +
 tests/generic/684     |    2 +
 tests/generic/685     |    2 +
 tests/generic/686     |    2 +
 tests/generic/687     |    2 +
 tests/generic/688     |    2 +
 tests/generic/691     |    8 ++--
 tests/generic/721     |   10 +++--
 tests/generic/726     |    2 +
 tests/generic/727     |    2 +
 tests/generic/746     |    2 +
 tests/xfs/149         |    2 +
 tests/xfs/501         |    2 +
 tests/xfs/502         |    2 +
 tests/xfs/530         |    2 +
 tests/xfs/720         |    2 +
 tests/xfs/795         |    2 +
 tests/xfs/803         |    2 +
 68 files changed, 347 insertions(+), 192 deletions(-)

[PATCHSET 2/7] fstests: fix logwrites zeroing

[Darrick J. Wong]

Hi all,

This patchset attempts to fix the behavior of the logwrites replay tests
when running on XFS.  In short, the logwrites code logs every write made
to a block device and can set tags at arbitrary points in that log.  The
replay tests then erase the block device and replay up to those tags,
and then invoke the filesystem to see if it can recover the filesystem.

However, the logwrites replay has long had subtle problems.  XFS log
recovery won't replay a metadata block from the log if the ondisk block
has a newer log sequence number (LSNs), which rests on the assumption
that ondisk filesystem metadata do not have LSNs greater than that of
the head of the log.  This is not true if we're replaying to a point in
the past, so log recovery fails to produce a consistent filesystem.

The logwrites replay code works around this issue by erasing the block
device before replaying.  Unfortunately, it uses BLKDISCARD, which has
never guaranteed the results of a read after the discard completes.
Some devices return zeroes, others do nothing.  This inconsistency has
been worked around by installing the one block device that guarantees
that reads after a mass discard always return zeros -- dm-thinp.

Unfortunately that leaves a subtle landmine for test authors.  If their
filesystem's recovery doesn't behave like XFS's then they might not need
thinp.  If it does, it's all too easy to write a broken testcase.

Fix this whole situation by adding a warning to the logwrites setup code
if the block device is not known to guarantee that reads after discard
returns zeroes; fix the replay program to use BLKZEROOUT so that the
block device is zeroed before replay begins; and then fix the logwrites
setup to tell the replay program that it can use BLKDISCARD for the
devices where we know that will work, because discard is much faster
than writing zeroes.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

With a bit of luck, this should all go splendidly.
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=logwrites-fix-zeroing
---
Commits in this patchset:
 * logwrites: warn if we don't think read after discard returns zeroes
 * logwrites: use BLKZEROOUT if it's available
 * logwrites: only use BLKDISCARD if we know discard zeroes data
---
 common/dmlogwrites          |   39 ++++++++++++++++++++++++++++++++++++++-
 src/log-writes/log-writes.c |   10 ++++++++++
 src/log-writes/log-writes.h |    1 +
 src/log-writes/replay-log.c |    8 ++++++++
 4 files changed, 57 insertions(+), 1 deletion(-)

[PATCHSET v6.2 3/7] fstests: enable metadir

[Darrick J. Wong]

Hi all,

Adjust fstests as needed to support the XFS metadata directory feature,
and add some new tests for online fsck and fuzz testing of the ondisk
metadata.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=metadir

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=metadir

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=metadir

xfsdocs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-documentation.git/log/?h=metadir
---
Commits in this patchset:
 * various: fix finding metadata inode numbers when metadir is enabled
 * xfs/{030,033,178}: forcibly disable metadata directory trees
 * common/repair: patch up repair sb inode value complaints
 * xfs/206: update for metadata directory support
 * xfs/{050,144,153,299,330}: update quota reports to handle metadir trees
 * xfs/509: adjust inumbers accounting for metadata directories
 * xfs: create fuzz tests for metadata directories
 * xfs/163: bigger fs for metadir
 * xfs/122: disable this test for any codebase that knows about metadir
 * scrub: race metapath online fsck with fsstress
 * xfs: test metapath repairs
---
 common/filter      |    7 ++-
 common/repair      |    4 ++
 common/xfs         |  100 +++++++++++++++++++++++++++++++++++++++++++-
 tests/xfs/007      |   16 ++++---
 tests/xfs/030      |    1 
 tests/xfs/033      |    1 
 tests/xfs/050      |    5 ++
 tests/xfs/122      |    6 +++
 tests/xfs/153      |    5 ++
 tests/xfs/1546     |   34 +++++++++++++++
 tests/xfs/1546.out |    4 ++
 tests/xfs/1547     |   34 +++++++++++++++
 tests/xfs/1547.out |    4 ++
 tests/xfs/1548     |   34 +++++++++++++++
 tests/xfs/1548.out |    4 ++
 tests/xfs/1549     |   35 +++++++++++++++
 tests/xfs/1549.out |    4 ++
 tests/xfs/1550     |   34 +++++++++++++++
 tests/xfs/1550.out |    4 ++
 tests/xfs/1551     |   34 +++++++++++++++
 tests/xfs/1551.out |    4 ++
 tests/xfs/1552     |   34 +++++++++++++++
 tests/xfs/1552.out |    4 ++
 tests/xfs/1553     |   35 +++++++++++++++
 tests/xfs/1553.out |    4 ++
 tests/xfs/163      |    2 -
 tests/xfs/178      |    1 
 tests/xfs/1874     |  119 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1874.out |   19 ++++++++
 tests/xfs/1892     |   66 +++++++++++++++++++++++++++++
 tests/xfs/1892.out |    2 +
 tests/xfs/1893     |   67 +++++++++++++++++++++++++++++
 tests/xfs/1893.out |    2 +
 tests/xfs/206      |    1 
 tests/xfs/299      |    1 
 tests/xfs/330      |    6 ++-
 tests/xfs/509      |   23 +++++++++-
 tests/xfs/529      |    5 +-
 tests/xfs/530      |    6 +--
 tests/xfs/739      |    9 +---
 tests/xfs/740      |    9 +---
 tests/xfs/741      |    9 +---
 tests/xfs/742      |    9 +---
 tests/xfs/743      |    9 +---
 tests/xfs/744      |    9 +---
 tests/xfs/745      |    9 +---
 tests/xfs/746      |    9 +---
 47 files changed, 765 insertions(+), 78 deletions(-)
 create mode 100755 tests/xfs/1546
 create mode 100644 tests/xfs/1546.out
 create mode 100755 tests/xfs/1547
 create mode 100644 tests/xfs/1547.out
 create mode 100755 tests/xfs/1548
 create mode 100644 tests/xfs/1548.out
 create mode 100755 tests/xfs/1549
 create mode 100644 tests/xfs/1549.out
 create mode 100755 tests/xfs/1550
 create mode 100644 tests/xfs/1550.out
 create mode 100755 tests/xfs/1551
 create mode 100644 tests/xfs/1551.out
 create mode 100755 tests/xfs/1552
 create mode 100644 tests/xfs/1552.out
 create mode 100755 tests/xfs/1553
 create mode 100644 tests/xfs/1553.out
 create mode 100755 tests/xfs/1874
 create mode 100644 tests/xfs/1874.out
 create mode 100755 tests/xfs/1892
 create mode 100644 tests/xfs/1892.out
 create mode 100755 tests/xfs/1893
 create mode 100644 tests/xfs/1893.out

[PATCHSET v6.2 4/7] fstests: make protofiles less janky

[Darrick J. Wong]

Hi all,

Add a new regression test for xattr support of xfs protofiles.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=protofiles

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=protofiles
---
Commits in this patchset:
 * fstests: test mkfs.xfs protofiles with xattr support
---
 tests/xfs/1937     |  144 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1937.out |  102 +++++++++++++++++++++++++++++++++++++
 2 files changed, 246 insertions(+)
 create mode 100755 tests/xfs/1937
 create mode 100644 tests/xfs/1937.out

[PATCHSET v6.2 5/7] fstests: shard the realtime section

[Darrick J. Wong]

Hi all,

Right now, the realtime section uses a single pair of metadata inodes to
store the free space information.  This presents a scalability problem
since every thread trying to allocate or free rt extents have to lock
these files.  It would be very useful if we could begin to tackle these
problems by sharding the realtime section, so create the notion of
realtime groups, which are similar to allocation groups on the data
section.

While we're at it, define a superblock to be stamped into the start of
each rt section.  This enables utilities such as blkid to identify block
devices containing realtime sections, and helpfully avoids the situation
where a file extent can cross an rtgroup boundary.

The best advantage for rtgroups will become evident later when we get to
adding rmap and reflink to the realtime volume, since the geometry
constraints are the same for rt groups and AGs.  Hence we can reuse all
that code directly.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=realtime-groups

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=realtime-groups

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=realtime-groups

xfsdocs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-documentation.git/log/?h=realtime-groups
---
Commits in this patchset:
 * common/populate: refactor caching of metadumps to a helper
 * common/{fuzzy,populate}: use _scratch_xfs_mdrestore
 * fuzzy: stress data and rt sections of xfs filesystems equally
 * common/ext4: reformat external logs during mdrestore operations
 * common/populate: use metadump v2 format by default for fs metadata snapshots
 * punch-alternating: detect xfs realtime files with large allocation units
 * xfs/206: update mkfs filtering for rt groups feature
 * common: pass the realtime device to xfs_db when possible
 * xfs/185: update for rtgroups
 * xfs/449: update test to know about xfs_db -R
 * xfs/271,xfs/556: fix tests to deal with rtgroups output in bmap/fsmap commands
 * common/xfs: capture realtime devices during metadump/mdrestore
 * common/fuzzy: adapt the scrub stress tests to support rtgroups
 * xfs: fix fuzz tests of rtgroups bitmap and summary files
---
 common/ext4             |   17 +++++++++
 common/fuzzy            |   84 +++++++++++++++++++++++++++++++++------------
 common/metadump         |   22 ++++++++++--
 common/populate         |   85 ++++++++++++++++++++++++++--------------------
 common/xfs              |   87 +++++++++++++++++++++++++++++++++++++++++++----
 src/punch-alternating.c |   28 +++++++++++++++
 tests/xfs/114           |    4 ++
 tests/xfs/146           |    2 +
 tests/xfs/185           |   65 +++++++++++++++++++++++++++--------
 tests/xfs/187           |    3 +-
 tests/xfs/206           |    1 +
 tests/xfs/271           |    4 ++
 tests/xfs/341           |    4 +-
 tests/xfs/449           |    6 +++
 tests/xfs/556           |   16 +++++----
 tests/xfs/581           |    9 ++++-
 tests/xfs/582           |   14 ++++----
 tests/xfs/720           |    2 +
 tests/xfs/739           |    6 +++
 tests/xfs/740           |    6 +++
 tests/xfs/741           |    6 +++
 tests/xfs/742           |    6 +++
 tests/xfs/743           |    6 +++
 tests/xfs/744           |    6 +++
 tests/xfs/745           |    6 +++
 tests/xfs/746           |    6 +++
 tests/xfs/793           |   14 ++++----
 tests/xfs/795           |    2 +
 28 files changed, 393 insertions(+), 124 deletions(-)

[PATCHSET v6.2 6/7] fstests: store quota files in the metadir

[Darrick J. Wong]

Hi all,

Store the quota files in the metadata directory tree instead of the superblock.
Since we're introducing a new incompat feature flag, let's also make the mount
process bring up quotas in whatever state they were when the filesystem was
last unmounted, instead of requiring sysadmins to remember that themselves.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=metadir-quotas

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=metadir-quotas

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=metadir-quotas
---
Commits in this patchset:
 * xfs: update tests for quota files in the metadir
 * xfs: test persistent quota flags
 * xfs: fix quota detection in fuzz tests
 * xfs: fix tests for persistent qflags
---
 common/quota              |    1 
 common/rc                 |    1 
 common/xfs                |   21 ++++++
 tests/generic/563         |    8 ++
 tests/xfs/007             |    2 -
 tests/xfs/096             |    1 
 tests/xfs/096.out         |    2 -
 tests/xfs/106             |    2 -
 tests/xfs/116             |   13 ++++
 tests/xfs/116.cfg         |    1 
 tests/xfs/116.out.default |    0 
 tests/xfs/116.out.metadir |    3 +
 tests/xfs/152             |    2 -
 tests/xfs/1891            |  128 +++++++++++++++++++++++++++++++++++++++
 tests/xfs/1891.out        |  147 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/263             |    1 
 tests/xfs/263.out         |    2 -
 tests/xfs/425             |    5 +-
 tests/xfs/426             |    5 +-
 tests/xfs/427             |    5 +-
 tests/xfs/428             |    5 +-
 tests/xfs/429             |    5 +-
 tests/xfs/430             |    5 +-
 tests/xfs/487             |    5 +-
 tests/xfs/488             |    5 +-
 tests/xfs/489             |    5 +-
 tests/xfs/779             |    5 +-
 tests/xfs/780             |    5 +-
 tests/xfs/781             |    5 +-
 29 files changed, 376 insertions(+), 19 deletions(-)
 create mode 100644 tests/xfs/116.cfg
 rename tests/xfs/{116.out => 116.out.default} (100%)
 create mode 100644 tests/xfs/116.out.metadir
 create mode 100755 tests/xfs/1891
 create mode 100644 tests/xfs/1891.out

[PATCHSET v6.2 7/7] fstests: enable quota for realtime volumes

[Darrick J. Wong]

Hi all,

The sole patch in this series sets up functional testing for quota on
the xfs realtime device.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=realtime-quotas

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=realtime-quotas

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=realtime-quotas
---
Commits in this patchset:
 * common: enable testing of realtime quota when supported
 * xfs: fix quota tests to adapt to realtime quota
 * xfs: regression testing of quota on the realtime device
---
 common/populate    |   10 ++-
 common/quota       |   45 ++++++++-----
 common/xfs         |   62 +++++++++++++++++++
 tests/generic/219  |    1 
 tests/generic/230  |    1 
 tests/generic/305  |    1 
 tests/generic/326  |    1 
 tests/generic/327  |    1 
 tests/generic/328  |    1 
 tests/generic/566  |    4 +
 tests/generic/587  |    1 
 tests/generic/603  |    1 
 tests/generic/691  |    2 +
 tests/generic/710  |    4 +
 tests/xfs/050      |    2 +
 tests/xfs/096      |    4 +
 tests/xfs/106      |    1 
 tests/xfs/108      |    2 +
 tests/xfs/152      |    1 
 tests/xfs/153      |    2 +
 tests/xfs/161      |    1 
 tests/xfs/1858     |  174 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1858.out |   47 ++++++++++++++
 tests/xfs/213      |    1 
 tests/xfs/214      |    1 
 tests/xfs/220      |    2 +
 tests/xfs/299      |    2 +
 tests/xfs/330      |    1 
 tests/xfs/434      |    1 
 tests/xfs/435      |    1 
 tests/xfs/440      |    3 +
 tests/xfs/441      |    1 
 tests/xfs/442      |    1 
 tests/xfs/508      |    2 +
 tests/xfs/511      |   10 +++
 tests/xfs/720      |    5 +
 36 files changed, 377 insertions(+), 23 deletions(-)
 create mode 100755 tests/xfs/1858
 create mode 100644 tests/xfs/1858.out

[PATCH 01/23] generic/476: fix fsstress process management

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

generic/476 never used to run fsstress in the background, but
8973af00ec212f made it do that.  This is incorrect, because now 476 runs
for three seconds (i.e. long enough to fall out the bottom of the test
and end up in _cleanup), ignoring any SOAK_DURATION/TIME_FACTOR
settings.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/generic/476 |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/tests/generic/476 b/tests/generic/476
index 769b3745f75689..4537fcd77d2f07 100755
--- a/tests/generic/476
+++ b/tests/generic/476
@@ -22,7 +22,7 @@ nr_ops=$((25000 * nr_cpus * TIME_FACTOR))
 fsstress_args=(-w -d $SCRATCH_MNT -n $nr_ops -p $nr_cpus)
 test -n "$SOAK_DURATION" && fsstress_args+=(--duration="$SOAK_DURATION")
 
-_run_fsstress_bg "${fsstress_args[@]}"
+_run_fsstress "${fsstress_args[@]}"
 
 # success, all done
 status=0

[PATCH 02/23] metadump: make non-local function variables more obvious

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

In _xfs_verify_metadump_v2(), we want to set up some loop devices,
record the names of those loop devices, and then leave the variables in
the global namespace so that _xfs_cleanup_verify_metadump can dispose of
them.

Elsewhere in fstests the convention for global variables is to put them
in all caps to make it obvious that they're global and not local
variables, so do that here too.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: ce79de11337e38 ("fstests: clean up loop device instantiation")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/metadump |   28 +++++++++++++---------------
 1 file changed, 13 insertions(+), 15 deletions(-)


diff --git a/common/metadump b/common/metadump
index 3aa7adf76da4bb..493f8b6379dc0b 100644
--- a/common/metadump
+++ b/common/metadump
@@ -25,8 +25,8 @@ _xfs_cleanup_verify_metadump()
 	test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE"
 
 	if [ -n "$XFS_METADUMP_IMG" ]; then
-		[ -b "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev
-		[ -b "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev
+		[ -b "$METADUMP_DATA_LOOP_DEV" ] && _destroy_loop_device $METADUMP_DATA_LOOP_DEV
+		[ -b "$METADUMP_LOG_LOOP_DEV" ] && _destroy_loop_device $METADUMP_LOG_LOOP_DEV
 		for img in "$XFS_METADUMP_IMG"*; do
 			test -e "$img" && rm -f "$img"
 		done
@@ -100,9 +100,7 @@ _xfs_verify_metadump_v2()
 	local metadump_file="$XFS_METADUMP_FILE"
 	local version="-v 2"
 	local data_img="$XFS_METADUMP_IMG.data"
-	local data_loop
 	local log_img=""
-	local log_loop
 
 	# Capture metadump, which creates metadump_file
 	_scratch_xfs_metadump $metadump_file $metadump_args $version
@@ -118,27 +116,27 @@ _xfs_verify_metadump_v2()
 		_scratch_xfs_mdrestore $metadump_file
 
 	# Create loopdev for data device so we can mount the fs
-	md_data_loop_dev=$(_create_loop_device $data_img)
+	METADUMP_DATA_LOOP_DEV=$(_create_loop_device $data_img)
 
 	# Create loopdev for log device if we recovered anything
-	test -s "$log_img" && md_log_loop_dev=$(_create_loop_device $log_img)
+	test -s "$log_img" && METADUMP_LOG_LOOP_DEV=$(_create_loop_device $log_img)
 
 	# Mount fs, run an extra test, fsck, and unmount
-	SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _scratch_mount
+	SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV _scratch_mount
 	if [ -n "$extra_test" ]; then
-		SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev $extra_test
+		SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV $extra_test
 	fi
-	SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _check_xfs_scratch_fs
-	_unmount $md_data_loop_dev
+	SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV _check_xfs_scratch_fs
+	_unmount $METADUMP_DATA_LOOP_DEV
 
 	# Tear down what we created
-	if [ -b "$md_log_loop_dev" ]; then
-		_destroy_loop_device $md_log_loop_dev
-		unset md_log_loop_dev
+	if [ -b "$METADUMP_LOG_LOOP_DEV" ]; then
+		_destroy_loop_device $METADUMP_LOG_LOOP_DEV
+		unset METADUMP_LOG_LOOP_DEV
 		rm -f $log_img
 	fi
-	_destroy_loop_device $md_data_loop_dev
-	unset md_data_loop_dev
+	_destroy_loop_device $METADUMP_DATA_LOOP_DEV
+	unset METADUMP_DATA_LOOP_DEV
 	rm -f $data_img
 }
 

[PATCH 03/23] metadump: fix cleanup for v1 metadump testing

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

In commit ce79de11337e38, the metadump v2 tests were updated to leave
the names of loop devices in some global variables so that the cleanup
method can find them and remove the loop devices.  Inexplicably, the
metadump v1 test function was not upgraded.  Do so now.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: ce79de11337e38 ("fstests: clean up loop device instantiation")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/metadump |   14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)


diff --git a/common/metadump b/common/metadump
index 493f8b6379dc0b..a4ec9a7f921acf 100644
--- a/common/metadump
+++ b/common/metadump
@@ -61,7 +61,6 @@ _xfs_verify_metadump_v1()
 	local metadump_file="$XFS_METADUMP_FILE"
 	local version=""
 	local data_img="$XFS_METADUMP_IMG.data"
-	local data_loop
 
 	# Force v1 if we detect v2 support
 	if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then
@@ -75,18 +74,19 @@ _xfs_verify_metadump_v1()
 	SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file
 
 	# Create loopdev for data device so we can mount the fs
-	data_loop=$(_create_loop_device $data_img)
+	METADUMP_DATA_LOOP_DEV=$(_create_loop_device $data_img)
 
 	# Mount fs, run an extra test, fsck, and unmount
-	SCRATCH_DEV=$data_loop _scratch_mount
+	SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV _scratch_mount
 	if [ -n "$extra_test" ]; then
-		SCRATCH_DEV=$data_loop $extra_test
+		SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV $extra_test
 	fi
-	SCRATCH_DEV=$data_loop _check_xfs_scratch_fs
-	SCRATCH_DEV=$data_loop _scratch_unmount
+	SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV _check_xfs_scratch_fs
+	_unmount $METADUMP_DATA_LOOP_DEV
 
 	# Tear down what we created
-	_destroy_loop_device $data_loop
+	_destroy_loop_device $METADUMP_DATA_LOOP_DEV
+	unset METADUMP_DATA_LOOP_DEV
 	rm -f $data_img
 }
 

[PATCH 04/23] generic/482: _run_fsstress needs the test filesystem

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The test filesystem is now a hard dependency of _run_fsstress because
the latter copies the fsstress binary to a different name on the test
filesystem:

generic/482       - output mismatch (see /var/tmp/fstests/generic/482.out.bad)
    --- tests/generic/482.out   2024-02-28 16:20:24.262888854 -0800
    +++ /var/tmp/fstests/generic/482.out.bad    2025-01-03 15:00:43.107625116 -0800
    @@ -1,2 +1,3 @@
     QA output created by 482
    +cp: cannot create regular file '/mnt/482.fsstress': Read-only file system
     Silence is golden
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/482.out /var/tmp/fstests/generic/482.out.bad'  to see the entire diff)

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/generic/482 |    1 -
 1 file changed, 1 deletion(-)


diff --git a/tests/generic/482 b/tests/generic/482
index 8c114ee03058c6..0efc026a160040 100755
--- a/tests/generic/482
+++ b/tests/generic/482
@@ -68,7 +68,6 @@ lowspace=$((1024*1024 / 512))		# 1m low space threshold
 
 # Use a thin device to provide deterministic discard behavior. Discards are used
 # by the log replay tool for fast zeroing to prevent out-of-order replay issues.
-_test_unmount
 _dmthin_init $devsize $devsize $csize $lowspace
 _log_writes_init $DMTHIN_VOL_DEV
 _log_writes_mkfs >> $seqres.full 2>&1

[PATCH 05/23] generic/019: don't fail if fio crashes while shutting down

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

My system (Debian 12) has fio 3.33.  Once in a while, fio crashes while
shutting down after it receives a SIGBUS on account of the filesystem
going down.  This causes the test to fail with:

generic/019       - output mismatch (see /var/tmp/fstests/generic/019.out.bad)
    --- tests/generic/019.out   2024-02-28 16:20:24.130889521 -0800
    +++ /var/tmp/fstests/generic/019.out.bad    2025-01-03 15:00:35.903564431 -0800
    @@ -5,5 +5,6 @@

     Start fio..
     Force SCRATCH_DEV device failure
    +/tmp/fstests/tests/generic/019: line 112: 90841 Segmentation fault      $FIO_PROG $fio_config >> $seqres.full 2>&1
     Make SCRATCH_DEV device operable again
     Disallow global fail_make_request feature
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/019.out /var/tmp/fstests/generic/019.out.bad'  to see the entire diff)

because the wait command will dutifully report fatal signals that kill
the fio process.  Unfortunately, a core dump shows that we blew up in
some library's exit handler somewhere:

(gdb) where
#0  unlink_chunk (p=p@entry=0x55b31cb9a430, av=0x7f8b4475ec60 <main_arena>) at ./malloc/malloc.c:1628
#1  0x00007f8b446222ff in _int_free (av=0x7f8b4475ec60 <main_arena>, p=0x55b31cb9a430, have_lock=<optimized out>, have_lock@entry=0) at ./malloc/malloc.c:4603
#2  0x00007f8b44624f1f in __GI___libc_free (mem=<optimized out>) at ./malloc/malloc.c:3385
#3  0x00007f8b3a71cf0e in ?? () from /lib/x86_64-linux-gnu/libtasn1.so.6
#4  0x00007f8b4426447c in ?? () from /lib/x86_64-linux-gnu/libgnutls.so.30
#5  0x00007f8b4542212a in _dl_call_fini (closure_map=closure_map@entry=0x7f8b44465620) at ./elf/dl-call_fini.c:43
#6  0x00007f8b4542581e in _dl_fini () at ./elf/dl-fini.c:114
#7  0x00007f8b445ca55d in __run_exit_handlers (status=0, listp=0x7f8b4475e820 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true)
    at ./stdlib/exit.c:116
#8  0x00007f8b445ca69a in __GI_exit (status=<optimized out>) at ./stdlib/exit.c:146
#9  0x00007f8b445b3251 in __libc_start_call_main (main=main@entry=0x55b319278e10 <main>, argc=argc@entry=2, argv=argv@entry=0x7ffec6f8b468) at ../sysdeps/nptl/libc_start_call_main.h:74
#10 0x00007f8b445b3305 in __libc_start_main_impl (main=0x55b319278e10 <main>, argc=2, argv=0x7ffec6f8b468, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
    stack_end=0x7ffec6f8b458) at ../csu/libc-start.c:360
#11 0x000055b319278ed1 in _start ()

This isn't a filesystem failure, so mask this by shovelling the output
to seqres.full.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/generic/019 |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/tests/generic/019 b/tests/generic/019
index bed916b53f98e5..676ff27dab8062 100755
--- a/tests/generic/019
+++ b/tests/generic/019
@@ -109,7 +109,7 @@ _workout()
 	    _fail "failed: still able to perform integrity fsync on $SCRATCH_MNT"
 
 	_kill_fsstress
-	wait $fio_pid
+	wait $fio_pid &>> $seqres.full # old fio can crash on EIO, ignore segfault reporting
 	unset fio_pid
 
 	# We expect that broken FS still can be umounted

[PATCH 06/23] fuzzy: do not set _FSSTRESS_PID when exercising fsx

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

If we're not running fsstress as the scrub exerciser, don't set
_FSSTRESS_PID because the _kill_fsstress call in the cleanup function
will think that it has to wait for a nonexistant fsstress process.
This fixes the problem of xfs/565 runtime increasing from 30s to 800s
because it tries to kill a nonexistent "565.fsstress" process and then
waits for the fsx loop control process, which hasn't been sent any
signals.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/fuzzy |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)


diff --git a/common/fuzzy b/common/fuzzy
index 534e91dedbbb43..0a2d91542b561e 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -1392,7 +1392,11 @@ _scratch_xfs_stress_scrub() {
 
 	"__stress_scrub_${exerciser}_loop" "$end" "$runningfile" \
 			"$remount_period" "$stress_tgt" &
-	_FSSTRESS_PID=$!
+	# The loop is a background process, so _FSSTRESS_PID is set in that
+	# child.  Unfortunately, this process doesn't know about it.  Therefore
+	# we need to set _FSSTRESS_PID ourselves so that cleanup tries to kill
+	# fsstress.
+	test "${exerciser}" = "fsstress" && _FSSTRESS_PID=$!
 
 	if [ -n "$freeze" ]; then
 		__stress_scrub_freeze_loop "$end" "$runningfile" &

[PATCH 07/23] common/rc: create a wrapper for the su command

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Create a _su wrapper around the su command so that the next patch can
fix all the pkill isolation code so that this test can only kill
processes started for this test.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/rc         |    9 +++++++--
 tests/generic/093 |    2 +-
 tests/generic/125 |    2 +-
 tests/generic/128 |    2 +-
 tests/generic/193 |   36 ++++++++++++++++++------------------
 tests/generic/230 |   14 +++++++-------
 tests/generic/231 |    2 +-
 tests/generic/233 |    2 +-
 tests/generic/270 |    4 ++--
 tests/generic/314 |    2 +-
 tests/generic/327 |    2 +-
 tests/generic/328 |    4 ++--
 tests/generic/355 |    2 +-
 tests/generic/453 |    6 +++---
 tests/generic/514 |    2 +-
 tests/generic/573 |    2 +-
 tests/generic/600 |    2 +-
 tests/generic/601 |    2 +-
 tests/generic/603 |   10 +++++-----
 tests/generic/673 |    2 +-
 tests/generic/674 |    2 +-
 tests/generic/675 |    2 +-
 tests/generic/680 |    2 +-
 tests/generic/681 |    2 +-
 tests/generic/682 |    2 +-
 tests/generic/683 |    2 +-
 tests/generic/684 |    2 +-
 tests/generic/685 |    2 +-
 tests/generic/686 |    2 +-
 tests/generic/687 |    2 +-
 tests/generic/688 |    2 +-
 tests/generic/691 |    8 ++++----
 tests/generic/721 |   10 +++++-----
 tests/generic/726 |    2 +-
 tests/generic/727 |    2 +-
 tests/xfs/720     |    2 +-
 tests/xfs/795     |    2 +-
 tests/xfs/803     |    2 +-
 38 files changed, 82 insertions(+), 77 deletions(-)


diff --git a/common/rc b/common/rc
index c5421c9454fa62..459be11c11c405 100644
--- a/common/rc
+++ b/common/rc
@@ -2740,6 +2740,11 @@ _require_user_exists()
 	[ "$?" == "0" ] || _notrun "$user user not defined."
 }
 
+_su()
+{
+	su "$@"
+}
+
 # check if a user exists and is able to execute commands.
 # Uses 'fsgqa' user as default.
 #
@@ -2750,7 +2755,7 @@ _require_user()
 		qa_user=$1
 	fi
 	_require_user_exists $qa_user
-	echo /bin/true | su $qa_user
+	echo /bin/true | _su $qa_user
 	[ "$?" == "0" ] || _notrun "$qa_user cannot execute commands."
 }
 
@@ -2812,7 +2817,7 @@ s,^\s*$,,;
 
 _user_do()
 {
-	echo $1 | su -s /bin/bash $qa_user 2>&1 | _filter_user_do
+	echo $1 | _su -s /bin/bash $qa_user 2>&1 | _filter_user_do
 }
 
 _require_xfs_io_command()
diff --git a/tests/generic/093 b/tests/generic/093
index c4e866da1c18eb..047cc821e0e608 100755
--- a/tests/generic/093
+++ b/tests/generic/093
@@ -62,7 +62,7 @@ chmod ugo+w $TEST_DIR
 # don't use $here/src/writemod, as we're running it as a regular user, and
 # $here may contain path component that a regular user doesn't have search
 # permission
-su $qa_user -c "src/writemod $file" | filefilter
+_su $qa_user -c "src/writemod $file" | filefilter
 cat $file
 
 # success, all done
diff --git a/tests/generic/125 b/tests/generic/125
index e2bc5fa125da6e..011a51e8fa21c2 100755
--- a/tests/generic/125
+++ b/tests/generic/125
@@ -32,7 +32,7 @@ chmod a+rw $TESTFILE
 
 # don't use $here/src/ftrunc, as we're running it as a regular user, and $here
 # may contain path component that a regular user doesn't have search permission
-su $qa_user -c "./src/ftrunc -f $TESTFILE"
+_su $qa_user -c "./src/ftrunc -f $TESTFILE"
 
 if [ "$?" != "0" ];  then
     echo src/ftrunc returned non 0 status!
diff --git a/tests/generic/128 b/tests/generic/128
index f931ca0639687b..5a497cdea7382b 100755
--- a/tests/generic/128
+++ b/tests/generic/128
@@ -26,7 +26,7 @@ cp "$(type -P ls)" $SCRATCH_MNT
 chmod 700 $SCRATCH_MNT/nosuid
 chmod 4755 $SCRATCH_MNT/ls
 
-su -s/bin/bash - $qa_user -c "$SCRATCH_MNT/ls $SCRATCH_MNT/nosuid >/dev/null 2>&1"
+_su -s/bin/bash - $qa_user -c "$SCRATCH_MNT/ls $SCRATCH_MNT/nosuid >/dev/null 2>&1"
 if [ $? -eq 0 ] ; then
 	echo "Error: we shouldn't be able to ls the directory"
 fi
diff --git a/tests/generic/193 b/tests/generic/193
index d251d3a5c4f8ee..ba557428f77b9c 100755
--- a/tests/generic/193
+++ b/tests/generic/193
@@ -69,17 +69,17 @@ echo
 _create_files
 
 echo "user: chown root owned file to qa_user (should fail)"
-su ${qa_user} -c "chown ${qa_user} $test_root" 2>&1 | _filter_files
+_su ${qa_user} -c "chown ${qa_user} $test_root" 2>&1 | _filter_files
 
 echo "user: chown root owned file to root (should fail)"
-su ${qa_user} -c "chown root $test_root" 2>&1 | _filter_files
+_su ${qa_user} -c "chown root $test_root" 2>&1 | _filter_files
 
 echo "user: chown qa_user owned file to qa_user (should succeed)"
-su ${qa_user} -c "chown ${qa_user} $test_user"
+_su ${qa_user} -c "chown ${qa_user} $test_user"
 
 # this would work without _POSIX_CHOWN_RESTRICTED
 echo "user: chown qa_user owned file to root (should fail)"
-su ${qa_user} -c "chown root $test_user" 2>&1 | _filter_files
+_su ${qa_user} -c "chown root $test_user" 2>&1 | _filter_files
 
 _cleanup_files
 
@@ -93,19 +93,19 @@ echo
 _create_files
 
 echo "user: chgrp root owned file to root (should fail)"
-su ${qa_user} -c "chgrp root $test_root" 2>&1 | _filter_files
+_su ${qa_user} -c "chgrp root $test_root" 2>&1 | _filter_files
 
 echo "user: chgrp qa_user owned file to root (should fail)"
-su ${qa_user} -c "chgrp root $test_user" 2>&1 | _filter_files
+_su ${qa_user} -c "chgrp root $test_user" 2>&1 | _filter_files
 
 echo "user: chgrp root owned file to qa_user (should fail)"
-su ${qa_user} -c "chgrp ${qa_user} $test_root" 2>&1 | _filter_files
+_su ${qa_user} -c "chgrp ${qa_user} $test_root" 2>&1 | _filter_files
 
 echo "user: chgrp qa_user owned file to qa_user (should succeed)"
-su ${qa_user} -c "chgrp ${qa_user} $test_user"
+_su ${qa_user} -c "chgrp ${qa_user} $test_user"
 
 #echo "user: chgrp qa_user owned file to secondary group (should succeed)"
-#su ${qa_user} -c "chgrp ${group2} $test_user"
+#_su ${qa_user} -c "chgrp ${group2} $test_user"
 
 _cleanup_files
 
@@ -119,10 +119,10 @@ echo
 _create_files
 
 echo "user: chmod a+r on qa_user owned file (should succeed)"
-su ${qa_user} -c "chmod a+r $test_user"
+_su ${qa_user} -c "chmod a+r $test_user"
 
 echo "user: chmod a+r on root owned file (should fail)"
-su ${qa_user} -c "chmod a+r $test_root" 2>&1 | _filter_files
+_su ${qa_user} -c "chmod a+r $test_root" 2>&1 | _filter_files
 
 #
 # Setup a file owned by the qa_user, but with a group ID that
@@ -143,7 +143,7 @@ chown ${qa_user}:root $test_user
 chmod g+s $test_user
 
 # and let the qa_user change permission bits
-su ${qa_user} -c "chmod a+w $test_user"
+_su ${qa_user} -c "chmod a+w $test_user"
 stat -c '%A' $test_user
 
 #
@@ -220,7 +220,7 @@ echo "with no exec perm"
 echo frobnozzle >> $test_user
 chmod ug+s $test_user
 echo -n "before: "; stat -c '%A' $test_user
-su ${qa_user} -c "echo > $test_user"
+_su ${qa_user} -c "echo > $test_user"
 echo -n "after:  "; stat -c '%A' $test_user
 
 echo "with user exec perm"
@@ -228,7 +228,7 @@ echo frobnozzle >> $test_user
 chmod ug+s $test_user
 chmod u+x $test_user
 echo -n "before: "; stat -c '%A' $test_user
-su ${qa_user} -c "echo > $test_user"
+_su ${qa_user} -c "echo > $test_user"
 echo -n "after:  "; stat -c '%A' $test_user
 
 echo "with group exec perm"
@@ -237,7 +237,7 @@ chmod ug+s $test_user
 chmod g+x $test_user
 chmod u-x $test_user
 echo -n "before: "; stat -c '%A' $test_user
-su ${qa_user} -c "echo > $test_user"
+_su ${qa_user} -c "echo > $test_user"
 echo -n "after:  "; stat -c '%A' $test_user
 
 echo "with user+group exec perm"
@@ -245,7 +245,7 @@ echo frobnozzle >> $test_user
 chmod ug+s $test_user
 chmod ug+x $test_user
 echo -n "before: "; stat -c '%A' $test_user
-su ${qa_user} -c "echo > $test_user"
+_su ${qa_user} -c "echo > $test_user"
 echo -n "after:  "; stat -c '%A' $test_user
 
 #
@@ -258,10 +258,10 @@ echo
 _create_files
 
 echo "user: touch qa_user file (should succeed)"
-su ${qa_user} -c "touch $test_user"
+_su ${qa_user} -c "touch $test_user"
 
 echo "user: touch root file (should fail)"
-su ${qa_user} -c "touch $test_root" 2>&1 | _filter_files
+_su ${qa_user} -c "touch $test_root" 2>&1 | _filter_files
 
 _cleanup_files
 
diff --git a/tests/generic/230 b/tests/generic/230
index ba95fbe725ad28..18e20f4b2e9439 100755
--- a/tests/generic/230
+++ b/tests/generic/230
@@ -33,13 +33,13 @@ test_enforcement()
 	echo "--- initiating IO..." >>$seqres.full
 	# Firstly fit below block soft limit
 	echo "Write 225 blocks..."
-	su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((225 * $BLOCK_SIZE))' -c fsync \
+	_su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((225 * $BLOCK_SIZE))' -c fsync \
 		$SCRATCH_MNT/file1" 2>&1 >>$seqres.full | \
 		_filter_xfs_io_error | tee -a $seqres.full
 	repquota -$type $SCRATCH_MNT  | grep -v "^root" >>$seqres.full 2>&1
 	# Secondly overcome block soft limit
 	echo "Rewrite 250 blocks plus 1 byte..."
-	su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE + 1))' -c fsync \
+	_su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE + 1))' -c fsync \
 		$SCRATCH_MNT/file1" 2>&1 >>$seqres.full | \
 		_filter_xfs_io_error | tee -a $seqres.full
 	repquota -$type $SCRATCH_MNT  | grep -v "^root" >>$seqres.full 2>&1
@@ -49,14 +49,14 @@ test_enforcement()
 	# scratch device that may leed to grace time exceed.
 	setquota -$type $qa_user -T $grace $grace $SCRATCH_MNT 2>/dev/null
 	echo "Write 250 blocks..."
-	su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE))' -c fsync \
+	_su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE))' -c fsync \
 		$SCRATCH_MNT/file2" 2>&1 >>$seqres.full | \
 		_filter_xfs_io_error | tee -a $seqres.full
 	repquota -$type $SCRATCH_MNT  | grep -v "^root" >>$seqres.full 2>&1
 	# Now sleep for grace time and check that softlimit got enforced
 	sleep $((grace+1))
 	echo "Write 1 block..."
-	su $qa_user -c "$XFS_IO_PROG -c 'truncate 0' -c 'pwrite 0 $BLOCK_SIZE' \
+	_su $qa_user -c "$XFS_IO_PROG -c 'truncate 0' -c 'pwrite 0 $BLOCK_SIZE' \
 		$SCRATCH_MNT/file2" 2>&1 >>$seqres.full | \
 		_filter_xfs_io_error | tee -a $seqres.full
 	repquota -$type $SCRATCH_MNT  | grep -v "^root" >>$seqres.full 2>&1
@@ -65,7 +65,7 @@ test_enforcement()
 	# space reservations on XFS
 	setquota -$type $qa_user 0 0 3 5 $SCRATCH_MNT
 	echo "Touch 3+4"
-	su $qa_user -c "touch $SCRATCH_MNT/file3 $SCRATCH_MNT/file4" \
+	_su $qa_user -c "touch $SCRATCH_MNT/file3 $SCRATCH_MNT/file4" \
 		2>&1 >>$seqres.full | _filter_scratch | tee -a $seqres.full
 	repquota -$type $SCRATCH_MNT  | grep -v "^root" >>$seqres.full 2>&1
 	# Try to exceed inode hardlimit
@@ -74,14 +74,14 @@ test_enforcement()
 	# scratch device that may leed to grace time exceed.
 	setquota -$type $qa_user -T $grace $grace $SCRATCH_MNT 2>/dev/null
 	echo "Touch 5+6"
-	su $qa_user -c "touch $SCRATCH_MNT/file5 $SCRATCH_MNT/file6" \
+	_su $qa_user -c "touch $SCRATCH_MNT/file5 $SCRATCH_MNT/file6" \
 		2>&1 >>$seqres.full | _filter_scratch | tee -a $seqres.full
 	repquota -$type $SCRATCH_MNT  | grep -v "^root" >>$seqres.full 2>&1
 	# Wait and check grace time enforcement
 	rm -f $SCRATCH_MNT/file5 >>$seqres.full 2>&1
 	sleep $((grace+1))
 	echo "Touch 5"
-	su $qa_user -c "touch $SCRATCH_MNT/file5" 2>&1 >>$seqres.full |
+	_su $qa_user -c "touch $SCRATCH_MNT/file5" 2>&1 >>$seqres.full |
 		_filter_scratch | tee -a $seqres.full
 	repquota -$type $SCRATCH_MNT  | grep -v "^root" >>$seqres.full 2>&1
 	echo "--- completed IO ($type)" >>$seqres.full
diff --git a/tests/generic/231 b/tests/generic/231
index ef3ea45d45d736..8dda926d875e88 100755
--- a/tests/generic/231
+++ b/tests/generic/231
@@ -24,7 +24,7 @@ _fsx()
 	for (( i = 1; i <= $tasks; i++ )); do
 		SEED=$RANDOM
 		echo "ltp/fsx $FSX_ARGS -S $SEED $SCRATCH_MNT/fsx_file$i" >>$seqres.full
-		su $qa_user -c "ltp/fsx $FSX_ARGS -S $SEED \
+		_su $qa_user -c "ltp/fsx $FSX_ARGS -S $SEED \
 			$FSX_AVOID $SCRATCH_MNT/fsx_file$i" >$tmp.output$i 2>&1 &
 	done
 
diff --git a/tests/generic/233 b/tests/generic/233
index b4c804ff217fbb..df67b39092b7cb 100755
--- a/tests/generic/233
+++ b/tests/generic/233
@@ -43,7 +43,7 @@ _fsstress()
 	# temporarily.
 	ulimit -l unlimited
 	echo "fsstress $args" >> $seqres.full
-	if ! su $qa_user -c "$FSSTRESS_PROG $args" | tee -a $seqres.full | _filter_num
+	if ! _su $qa_user -c "$FSSTRESS_PROG $args" | tee -a $seqres.full | _filter_num
 	then
 		echo "    fsstress $args returned $?"
 		cat $tmp.out | tee -a $seqres.full
diff --git a/tests/generic/270 b/tests/generic/270
index 342ac8b5d3d963..d74971bb535239 100755
--- a/tests/generic/270
+++ b/tests/generic/270
@@ -37,14 +37,14 @@ _workout()
 	# io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited
 	# temporarily.
 	ulimit -l unlimited
-	su $qa_user -c "$_FSSTRESS_PROG $args" > /dev/null 2>&1 &
+	_su $qa_user -c "$_FSSTRESS_PROG $args" > /dev/null 2>&1 &
 	_FSSTRESS_PID=$!
 
 	echo "Run dd writers in parallel"
 	for ((i=0; i < num_iterations; i++))
 	do
 		# File will be opened with O_TRUNC each time
-		su $qa_user -c "dd if=/dev/zero \
+		_su $qa_user -c "dd if=/dev/zero \
 			of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M " \
 			>> $seqres.full 2>&1
 		sleep $enospc_time
diff --git a/tests/generic/314 b/tests/generic/314
index 5fbc6424de3ab4..65f7f9d9619f2e 100755
--- a/tests/generic/314
+++ b/tests/generic/314
@@ -27,7 +27,7 @@ chown $qa_user:12345 $TEST_DIR/$seq-dir
 chmod 2775 $TEST_DIR/$seq-dir
 
 # Make subdir
-su $qa_user -c "umask 022; mkdir $TEST_DIR/$seq-dir/subdir"
+_su $qa_user -c "umask 022; mkdir $TEST_DIR/$seq-dir/subdir"
 
 # Subdir should have inherited sgid
 _ls_l $TEST_DIR/$seq-dir/ | grep -v total | _filter_test_dir | awk '{print $1,$NF}'
diff --git a/tests/generic/327 b/tests/generic/327
index 2323e1e6a12ec8..18cfcd1f655bd7 100755
--- a/tests/generic/327
+++ b/tests/generic/327
@@ -47,7 +47,7 @@ _report_quota_blocks $SCRATCH_MNT
 echo "Try to reflink again"
 touch $testdir/file3
 chown $qa_user $testdir/file3
-su $qa_user -c "cp --reflink=always -f $testdir/file1 $testdir/file3" 2>&1 | _filter_scratch
+_su $qa_user -c "cp --reflink=always -f $testdir/file1 $testdir/file3" 2>&1 | _filter_scratch
 _report_quota_blocks $SCRATCH_MNT
 
 # success, all done
diff --git a/tests/generic/328 b/tests/generic/328
index 7071ded259ddb3..fa33bdb78dba12 100755
--- a/tests/generic/328
+++ b/tests/generic/328
@@ -47,12 +47,12 @@ setquota -u $qa_user 0 1024 0 0 $SCRATCH_MNT
 _report_quota_blocks $SCRATCH_MNT
 
 echo "Try to dio write the whole file"
-su $qa_user -c '$XFS_IO_PROG -d -c "pwrite 0 '$((sz+65536))'" '$testdir'/file1' 2>&1 >> $seqres.full | \
+_su $qa_user -c '$XFS_IO_PROG -d -c "pwrite 0 '$((sz+65536))'" '$testdir'/file1' 2>&1 >> $seqres.full | \
 	_filter_xfs_io_error
 _report_quota_blocks $SCRATCH_MNT
 
 echo "Try to write the whole file"
-su $qa_user -c '$XFS_IO_PROG -c "pwrite 0 '$((sz+65536))'" '$testdir'/file1' 2>&1 >> $seqres.full | \
+_su $qa_user -c '$XFS_IO_PROG -c "pwrite 0 '$((sz+65536))'" '$testdir'/file1' 2>&1 >> $seqres.full | \
 	_filter_xfs_io_error
 _report_quota_blocks $SCRATCH_MNT
 
diff --git a/tests/generic/355 b/tests/generic/355
index b0f8fc45d2b5f4..6b4f7ebae86405 100755
--- a/tests/generic/355
+++ b/tests/generic/355
@@ -22,7 +22,7 @@ rm -f $testfile
 
 do_io()
 {
-	su $qa_user -c "$XFS_IO_PROG -d -c 'pwrite 0 4k' $testfile" \
+	_su $qa_user -c "$XFS_IO_PROG -d -c 'pwrite 0 4k' $testfile" \
 		>>$seqres.full
 }
 
diff --git a/tests/generic/453 b/tests/generic/453
index b7e686f37100da..04945ad1085b2d 100755
--- a/tests/generic/453
+++ b/tests/generic/453
@@ -196,7 +196,7 @@ setf "job offer\xef\xb9\x92pdf" "small full stop"
 setf "job offer\xef\xbc\x8epdf" "fullwidth full stop"
 setf "job offer\xdc\x81pdf" "syriac supralinear full stop"
 setf "job offer\xdc\x82pdf" "syriac sublinear full stop"
-setf "job offer\xea\x93\xb8pdf" "lisu letter tone mya ti"
+setf "job offer\xea\x93\xb8pdf" "li_su letter tone mya ti"
 setf "job offer.pdf" "actual period"
 
 # encoding hidden tag characters in filenames to create confusing names
@@ -270,7 +270,7 @@ testf "job offer\xef\xb9\x92pdf" "small full stop"
 testf "job offer\xef\xbc\x8epdf" "fullwidth full stop"
 testf "job offer\xdc\x81pdf" "syriac supralinear full stop"
 testf "job offer\xdc\x82pdf" "syriac sublinear full stop"
-testf "job offer\xea\x93\xb8pdf" "lisu letter tone mya ti"
+testf "job offer\xea\x93\xb8pdf" "li_su letter tone mya ti"
 testf "job offer.pdf" "actual period"
 
 testf "llamapirate\xf3\xa0\x80\x81\xf3\xa0\x81\x94\xf3\xa0\x81\xa8\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb3\xf3\xa0\x81\xa1\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x81\xb3\xf3\xa0\x80\xa0\xf3\xa0\x81\xa6\xf3\xa0\x81\xaf\xf3\xa0\x81\xb2\xf3\xa0\x80\xa0\xf3\xa0\x81\x93\xf3\xa0\x81\xa5\xf3\xa0\x81\xa1\xf3\xa0\x81\xb4\xf3\xa0\x81\xb4\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb7\xf3\xa0\x81\xa5\xf3\xa0\x81\xb2\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\x95\xf3\xa0\x81\x93\xf3\xa0\x81\x84\xf3\xa0\x80\xa0\xf3\xa0\x80\xb1\xf3\xa0\x80\xb2\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x81\xbf"
@@ -307,7 +307,7 @@ if _check_xfs_scrub_does_unicode "$SCRATCH_MNT" "$SCRATCH_DEV"; then
 		grep -q "job offer.xef.xbc.x8epdf" $tmp.scrub || echo "No complaints about fullwidth full stop?"
 		grep -q "job offer.xdc.x81pdf" $tmp.scrub || echo "No complaints about syriac supralinear full stop?"
 		grep -q "job offer.xdc.x82pdf" $tmp.scrub || echo "No complaints about syriac sublinear full stop?"
-		grep -q "job offer.xea.x93.xb8pdf" $tmp.scrub || echo "No complaints about lisu letter tone mya ti?"
+		grep -q "job offer.xea.x93.xb8pdf" $tmp.scrub || echo "No complaints about li_su letter tone mya ti?"
 		grep -q "job offer.*could be confused with" $tmp.scrub || echo "No complaints about confusing job offers?"
 		grep -q "job offer.xe2.x80.xa4.xe2.x80.x8dpdf" $tmp.scrub || echo "No complaints about one dot leader with invisible space?"
 		grep -q "llamapirate" $tmp.scrub || echo "No complaints about hidden llm instructions in filenames?"
diff --git a/tests/generic/514 b/tests/generic/514
index 7f3d9c16cb70a4..a2086a255c77c6 100755
--- a/tests/generic/514
+++ b/tests/generic/514
@@ -21,7 +21,7 @@ _scratch_mount
 
 chmod a+rwx $SCRATCH_MNT
 $XFS_IO_PROG -f -c "pwrite -S 0x18 0 1m" $SCRATCH_MNT/foo >>$seqres.full
-su -s/bin/bash - $qa_user -c "ulimit -f 64 ; $XFS_IO_PROG -f -c \"reflink $SCRATCH_MNT/foo\" $SCRATCH_MNT/bar" >> $seqres.full 2>&1
+_su -s/bin/bash - $qa_user -c "ulimit -f 64 ; $XFS_IO_PROG -f -c \"reflink $SCRATCH_MNT/foo\" $SCRATCH_MNT/bar" >> $seqres.full 2>&1
 
 sz="$(_get_filesize $SCRATCH_MNT/bar)"
 if [ "$sz" -ne 0 ] && [ "$sz" -ne 65536 ]; then
diff --git a/tests/generic/573 b/tests/generic/573
index b310fccbddda56..d3f3296cb6bafa 100755
--- a/tests/generic/573
+++ b/tests/generic/573
@@ -56,7 +56,7 @@ $CHATTR_PROG -i $fsv_file
 _fsv_scratch_begin_subtest "FS_IOC_MEASURE_VERITY doesn't require root"
 _fsv_create_enable_file $fsv_file >> $seqres.full
 chmod 444 $fsv_file
-su $qa_user -c "$FSVERITY_PROG measure $fsv_file" >> $seqres.full
+_su $qa_user -c "$FSVERITY_PROG measure $fsv_file" >> $seqres.full
 
 # success, all done
 status=0
diff --git a/tests/generic/600 b/tests/generic/600
index 43f75376a10efc..31c832251ebb6f 100755
--- a/tests/generic/600
+++ b/tests/generic/600
@@ -33,7 +33,7 @@ setquota -t -u 0 1 $SCRATCH_MNT
 # Soft inode limit 1, hard limit 5
 setquota -u $qa_user 0 0 1 5 $SCRATCH_MNT
 # Run qa user over soft limit and go over grace period
-su $qa_user -c "touch $SCRATCH_MNT/file1 $SCRATCH_MNT/file2"
+_su $qa_user -c "touch $SCRATCH_MNT/file1 $SCRATCH_MNT/file2"
 sleep 3
 # Extend grace to now + 100s
 now=`date +%s`
diff --git a/tests/generic/601 b/tests/generic/601
index 78b6a4aaa13748..320ac0c758b766 100755
--- a/tests/generic/601
+++ b/tests/generic/601
@@ -42,7 +42,7 @@ $XFS_QUOTA_PROG -x -c "timer -u -i -d 1" $SCRATCH_MNT
 # Soft inode limit 1, hard limit 5
 $XFS_QUOTA_PROG -x -c "limit -u isoft=1 ihard=5 $qa_user" $SCRATCH_MNT
 # Run qa user over soft limit and go over grace period
-su $qa_user -c "touch $SCRATCH_MNT/file1 $SCRATCH_MNT/file2"
+_su $qa_user -c "touch $SCRATCH_MNT/file1 $SCRATCH_MNT/file2"
 sleep 3
 # Extend grace to now + 100s
 now=`date +%s`
diff --git a/tests/generic/603 b/tests/generic/603
index 2a75cf9e022750..b199f801a8f03f 100755
--- a/tests/generic/603
+++ b/tests/generic/603
@@ -66,13 +66,13 @@ test_grace()
 	echo "--- Test block quota ---"
 	# Firstly fit below block soft limit
 	echo "Write 225 blocks..."
-	su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((225 * $BLOCK_SIZE))' \
+	_su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((225 * $BLOCK_SIZE))' \
 		-c fsync $dir/file1" 2>&1 >>$seqres.full | \
 		_filter_xfs_io_error | tee -a $seqres.full
 	repquota -v -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1
 	# Secondly overcome block soft limit
 	echo "Rewrite 250 blocks plus 1 byte, over the block softlimit..."
-	su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE + 1))' \
+	_su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE + 1))' \
 		-c fsync $dir/file1" 2>&1 >>$seqres.full | \
 		_filter_xfs_io_error | tee -a $seqres.full
 	repquota -v -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1
@@ -81,7 +81,7 @@ test_grace()
 	# Now sleep enough grace time and check that softlimit got enforced
 	sleep $((bgrace + 1))
 	echo "Try to write 1 one more block after grace..."
-	su $qa_user -c "$XFS_IO_PROG -c 'truncate 0' -c 'pwrite 0 $BLOCK_SIZE' \
+	_su $qa_user -c "$XFS_IO_PROG -c 'truncate 0' -c 'pwrite 0 $BLOCK_SIZE' \
 		$dir/file2" 2>&1 >>$seqres.full | _filter_xfs_io_error | \
 		filter_enospc_edquot $type | tee -a $seqres.full
 	repquota -v -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1
@@ -91,7 +91,7 @@ test_grace()
 	# space reservations on XFS
 	setquota -$type $qa_user 0 0 3 100 $SCRATCH_MNT 
 	echo "Create 2 more files, over the inode softlimit..."
-	su $qa_user -c "touch $dir/file3 $dir/file4" 2>&1 >>$seqres.full | \
+	_su $qa_user -c "touch $dir/file3 $dir/file4" 2>&1 >>$seqres.full | \
 		_filter_scratch | tee -a $seqres.full
 	repquota -v -$type $SCRATCH_MNT  | grep -v "^root" >>$seqres.full 2>&1
 	# Reset grace time here, make below grace time test more accurate
@@ -99,7 +99,7 @@ test_grace()
 	# Wait and check grace time enforcement
 	sleep $((igrace+1))
 	echo "Try to create one more inode after grace..."
-	su $qa_user -c "touch $dir/file5" 2>&1 >>$seqres.full | \
+	_su $qa_user -c "touch $dir/file5" 2>&1 >>$seqres.full | \
 		filter_enospc_edquot $type | _filter_scratch | \
 		tee -a $seqres.full
 	repquota -v -$type $SCRATCH_MNT  | grep -v "^root" >>$seqres.full 2>&1
diff --git a/tests/generic/673 b/tests/generic/673
index 8f6def9c78881a..6c54ade81f0cec 100755
--- a/tests/generic/673
+++ b/tests/generic/673
@@ -39,7 +39,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c 'reflink $SCRATCH_MNT/b 0 0 1m' $SCRATCH_MNT/a"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/generic/674 b/tests/generic/674
index 1b711f27f39ed1..41fbdeb7d9eb17 100755
--- a/tests/generic/674
+++ b/tests/generic/674
@@ -42,7 +42,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c 'dedupe $SCRATCH_MNT/b 0 0 1m' $SCRATCH_MNT/a"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/generic/675 b/tests/generic/675
index e66de84b546a25..87dfbdfe278dd2 100755
--- a/tests/generic/675
+++ b/tests/generic/675
@@ -44,7 +44,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c 'reflink $SCRATCH_MNT/b 0 0 1m' $SCRATCH_MNT/a"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/generic/680 b/tests/generic/680
index 07048db5cc39a9..1a418fa3b61b0b 100755
--- a/tests/generic/680
+++ b/tests/generic/680
@@ -38,7 +38,7 @@ chmod 0644 $localfile
 cp $here/src/splice2pipe $tmp.splice2pipe
 # Test unprivileged user's privilege escalation
 echo "Test unprivileged user:"
-su ${qa_user} -c "$tmp.splice2pipe $localfile 1 AAAAAAAABBBBBBBB"
+_su ${qa_user} -c "$tmp.splice2pipe $localfile 1 AAAAAAAABBBBBBBB"
 _hexdump $localfile
 
 # success, all done
diff --git a/tests/generic/681 b/tests/generic/681
index aef54205d26f3a..dc4252013fc058 100755
--- a/tests/generic/681
+++ b/tests/generic/681
@@ -55,7 +55,7 @@ ls -sld $scratchdir  >> $seqres.full
 echo "fail quota" >> $seqres.full
 for ((i = 0; i < dirents; i++)); do
 	name=$(printf "y%0254d" $i)
-	su - "$qa_user" -c "ln $scratchfile $scratchdir/$name" 2>&1 | \
+	_su - "$qa_user" -c "ln $scratchfile $scratchdir/$name" 2>&1 | \
 		_filter_scratch | sed -e 's/y[0-9]*/yXXX/g'
 	test "${PIPESTATUS[0]}" -ne 0 && break
 done
diff --git a/tests/generic/682 b/tests/generic/682
index 3572af173cbe63..6914a549dc0975 100755
--- a/tests/generic/682
+++ b/tests/generic/682
@@ -65,7 +65,7 @@ echo "fail quota" >> $seqres.full
 for ((i = 0; i < dirents; i++)); do
 	name=$(printf "y%0254d" $i)
 	ln $scratchfile $stagedir/$name
-	su - "$qa_user" -c "mv $stagedir/$name $scratchdir/$name" 2>&1 | \
+	_su - "$qa_user" -c "mv $stagedir/$name $scratchdir/$name" 2>&1 | \
 		_filter_scratch | _filter_mv_output
 	test "${PIPESTATUS[0]}" -ne 0 && break
 done
diff --git a/tests/generic/683 b/tests/generic/683
index cc9a9786bde4bf..883905da5f9082 100755
--- a/tests/generic/683
+++ b/tests/generic/683
@@ -49,7 +49,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c '$command $start $end' $junk_file"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/generic/684 b/tests/generic/684
index 2ca036fe518050..9cdfe4ab4f4463 100755
--- a/tests/generic/684
+++ b/tests/generic/684
@@ -49,7 +49,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c '$command $start $end' $junk_file"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/generic/685 b/tests/generic/685
index de07a798a68594..5567255032e1c7 100755
--- a/tests/generic/685
+++ b/tests/generic/685
@@ -49,7 +49,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c '$command $start $end' $junk_file"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/generic/686 b/tests/generic/686
index fc6761fe61a91e..a3fa8e060aeab6 100755
--- a/tests/generic/686
+++ b/tests/generic/686
@@ -49,7 +49,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c '$command $start $end' $junk_file"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/generic/687 b/tests/generic/687
index 82dce88b85ef6d..0c4b09d29fe5e6 100755
--- a/tests/generic/687
+++ b/tests/generic/687
@@ -49,7 +49,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c '$command $start $end' $junk_file"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/generic/688 b/tests/generic/688
index e491d5cf30af23..77db29461415b3 100755
--- a/tests/generic/688
+++ b/tests/generic/688
@@ -51,7 +51,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c 'falloc 0 64k' $junk_file"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/generic/691 b/tests/generic/691
index f33d6edf18a1d1..30ae4a1e384a05 100755
--- a/tests/generic/691
+++ b/tests/generic/691
@@ -75,11 +75,11 @@ exercise()
 	setquota -${type} -t 86400 86400 $SCRATCH_MNT
 	repquota -v -${type} $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1
 	# Exceed the soft quota limit a bit at first
-	su $qa_user -c "$XFS_IO_PROG -f -t -c 'pwrite 0 2m' -c fsync ${file}.0" >>$seqres.full
+	_su $qa_user -c "$XFS_IO_PROG -f -t -c 'pwrite 0 2m' -c fsync ${file}.0" >>$seqres.full
 	# Write more data more times under soft quota limit exhausted condition,
 	# but not reach hard limit. To make sure each write won't trigger EDQUOT.
 	for ((i=1; i<=100; i++));do
-		su "$qa_user" -c "$XFS_IO_PROG -f -c 'pwrite 0 1m' -c fsync ${file}.$i" >>$seqres.full
+		_su "$qa_user" -c "$XFS_IO_PROG -f -c 'pwrite 0 1m' -c fsync ${file}.$i" >>$seqres.full
 		if [ $? -ne 0 ];then
 			echo "Unexpected error (type=$type)!"
 			break
@@ -89,9 +89,9 @@ exercise()
 
 	# As we've tested soft limit, now exceed the hard limit and give it a
 	# test in passing.
-	su $qa_user -c "$XFS_IO_PROG -f -t -c 'pwrite 0 100m' -c fsync ${file}.hard.0" 2>&1 >/dev/null | filter_quota $type
+	_su $qa_user -c "$XFS_IO_PROG -f -t -c 'pwrite 0 100m' -c fsync ${file}.hard.0" 2>&1 >/dev/null | filter_quota $type
 	for ((i=1; i<=10; i++));do
-		su "$qa_user" -c "$XFS_IO_PROG -f -c 'pwrite 0 1m' -c fsync ${file}.hard.$i" 2>&1 | filter_quota $type
+		_su "$qa_user" -c "$XFS_IO_PROG -f -c 'pwrite 0 1m' -c fsync ${file}.hard.$i" 2>&1 | filter_quota $type
 	done
 }
 
diff --git a/tests/generic/721 b/tests/generic/721
index a9565f18917831..75d5063c2d1701 100755
--- a/tests/generic/721
+++ b/tests/generic/721
@@ -52,7 +52,7 @@ cmd="$XFS_IO_PROG \
 	-c 'pwrite -S 0x60 44k 55k -b 1m' \
 	-c 'commitupdate -q' \
 	\"$dir/a\""
-su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir
+_su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir
 
 filesnap "after commit" $dir/a
 echo
@@ -67,7 +67,7 @@ cmd="$XFS_IO_PROG \
 	-c 'pwrite -S 0x60 0 55k' \
 	-c 'commitupdate -q' \
 	\"$dir/a\""
-su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir
+_su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir
 
 filesnap "after shorten commit" $dir/a
 echo
@@ -81,7 +81,7 @@ cmd="$XFS_IO_PROG \
 	-c \"pwrite -S 0x60 0 $(( (blksz * nrblks) + 37373 ))\" \
 	-c 'commitupdate -q' \
 	\"$dir/a\""
-su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir
+_su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir
 
 filesnap "after lengthen commit" $dir/a
 echo
@@ -96,7 +96,7 @@ cmd="$XFS_IO_PROG \
 	-c 'pwrite -S 0x60 44k 55k -b 1m' \
 	-c 'cancelupdate' \
 	\"$dir/a\""
-su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir
+_su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir
 
 filesnap "after cancel" $dir/a
 echo
@@ -115,7 +115,7 @@ cmd="$XFS_IO_PROG \
 	-c 'pwrite -S 0x61 22k 11k -b 1m' \
 	-c 'commitupdate -q' \
 	\"$dir/a\""
-su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir
+_su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir
 
 filesnap "after fail commit" $dir/a
 echo
diff --git a/tests/generic/726 b/tests/generic/726
index 131ac5b503e1a4..d2a2a2cebe522e 100755
--- a/tests/generic/726
+++ b/tests/generic/726
@@ -46,7 +46,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c 'startupdate' -c 'pwrite -S 0x57 0 1m' -c 'commitupdate' $SCRATCH_MNT/a"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/generic/727 b/tests/generic/727
index ee7ed9760a165a..9551e47cb13b06 100755
--- a/tests/generic/727
+++ b/tests/generic/727
@@ -54,7 +54,7 @@ commit_and_check() {
 
 	local cmd="$XFS_IO_PROG -c 'startupdate' -c 'pwrite -S 0x57 0 1m' -c 'commitupdate' $SCRATCH_MNT/a"
 	if [ -n "$user" ]; then
-		su - "$user" -c "$cmd" >> $seqres.full
+		_su - "$user" -c "$cmd" >> $seqres.full
 	else
 		$SHELL -c "$cmd" >> $seqres.full
 	fi
diff --git a/tests/xfs/720 b/tests/xfs/720
index f928cc43d3bc54..68a6c7f6e2d584 100755
--- a/tests/xfs/720
+++ b/tests/xfs/720
@@ -61,7 +61,7 @@ $XFS_QUOTA_PROG -x -c 'report -u' $SCRATCH_MNT >> $seqres.full
 
 # Fail at appending the file as qa_user to ensure quota enforcement works
 echo "fail quota" >> $seqres.full
-su - "$qa_user" -c "$XFS_IO_PROG -c 'pwrite 10g 1' $scratchfile" >> $seqres.full
+_su - "$qa_user" -c "$XFS_IO_PROG -c 'pwrite 10g 1' $scratchfile" >> $seqres.full
 $XFS_QUOTA_PROG -x -c 'report -u' $SCRATCH_MNT >> $seqres.full
 
 # success, all done
diff --git a/tests/xfs/795 b/tests/xfs/795
index 5a67f02ec92eca..217f96092a4c42 100755
--- a/tests/xfs/795
+++ b/tests/xfs/795
@@ -63,7 +63,7 @@ $XFS_QUOTA_PROG -x -c 'report -u' $SCRATCH_MNT >> $seqres.full
 echo "fail quota" >> $seqres.full
 for ((i = 0; i < dirents; i++)); do
 	name=$(printf "y%0254d" $i)
-	su - "$qa_user" -c "ln $scratchfile $scratchdir/$name" 2>&1 | \
+	_su - "$qa_user" -c "ln $scratchfile $scratchdir/$name" 2>&1 | \
 		_filter_scratch | sed -e 's/y[0-9]*/yXXX/g'
 	test "${PIPESTATUS[0]}" -ne 0 && break
 done
diff --git a/tests/xfs/803 b/tests/xfs/803
index e3277181b41af0..4e5a58c4a3328e 100755
--- a/tests/xfs/803
+++ b/tests/xfs/803
@@ -88,7 +88,7 @@ echo set too long value
 $XFS_IO_PROG -c "setfsprops $propname=$longpropval" $TEST_DIR
 
 echo not enough permissions
-su - "$qa_user" -c "$XFS_IO_PROG -c \"setfsprops $propname=$propval\" $TEST_DIR" 2>&1 | _filter_test_dir
+_su - "$qa_user" -c "$XFS_IO_PROG -c \"setfsprops $propname=$propval\" $TEST_DIR" 2>&1 | _filter_test_dir
 
 echo "*** DB TEST ***"
 

[PATCH 08/23] common: fix pkill by running test program in a separate session

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Run each test program with a separate session id so that we can tell
pkill to kill all processes of a given name, but only within our own
session id.  This /should/ suffice to run multiple fstests on the same
machine without one instance shooting down processes of another
instance.

This fixes a general problem with using "pkill --parent" -- if the
process being targeted is not a direct descendant of the bash script
calling pkill, then pkill will not do anything.  The scrub stress tests
make use of multiple background subshells, which is how a ^C in the
parent process fails to result in fsx/fsstress being killed.

This is necessary to fix SOAK_DURATION runtime constraints for all the
scrub stress tests.  However, there is a cost -- the test program no
longer runs with the same controlling tty as ./check, which means that
^Z doesn't work and SIGINT/SIGQUIT are set to SIG_IGN.  IOWs, if a test
wants to kill its subprocesses, it must use another signal such as
SIGPIPE.  Fortunately, bash doesn't whine about children dying due to
fatal signals if the children run in a different session id.

I also explored alternate designs, and this was the least unsatisfying:

a) Setting the process group didn't work because background subshells
are assigned a new group id.

b) Constraining the pkill/pgrep search to a cgroup could work, but we'd
have to set up a cgroup in which to run the fstest.

c) Putting test subprocesses in a systemd sub-scope and telling systemd
to kill the sub-scope could work because ./check can already use it to
ensure that all child processes of a test are killed.  However, this is
an *optional* feature, which means that we'd have to require systemd.

d) Constraining the pkill/pgrep search to a particular mount namespace
could work, but we already have tests that set up their own mount
namespaces, which means the constrained pgrep will not find all child
processes of a test.

e) Constraining to any other type of namespace (uts, pid, etc) might not
work because those namespaces might not be enabled.

f) Revert check-parallel and go back to one fstests instance per system.
Zorro already chose not to revert.

So.  Change _run_seq to create a the ./$seq process with a new session
id, update _su calls to use the same session as the parent test, update
all the pkill sites to use a wrapper so that we only target processes
created by *this* instance of fstests, and update SIGINT to SIGPIPE.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 check             |   33 ++++++++++++++++++++++++++++-----
 common/fuzzy      |   17 ++++++++---------
 common/rc         |   12 ++++++++++--
 tests/generic/310 |    6 +++---
 tests/generic/561 |    2 +-
 5 files changed, 50 insertions(+), 20 deletions(-)


diff --git a/check b/check
index 607d2456e6a1fe..bafe48bf12db67 100755
--- a/check
+++ b/check
@@ -698,18 +698,41 @@ _adjust_oom_score -500
 # systemd doesn't automatically remove transient scopes that fail to terminate
 # when systemd tells them to terminate (e.g. programs stuck in D state when
 # systemd sends SIGKILL), so we use reset-failed to tear down the scope.
+#
+# Use setsid to run the test program with a separate session id so that we
+# can pkill only the processes started by this test.
 _run_seq() {
-	local cmd=(bash -c "test -w ${OOM_SCORE_ADJ} && echo 250 > ${OOM_SCORE_ADJ}; exec ./$seq")
+	local cmd=(bash -c "test -w ${OOM_SCORE_ADJ} && echo 250 > ${OOM_SCORE_ADJ}; exec setsid bash ./$seq")
 
 	if [ -n "${HAVE_SYSTEMD_SCOPES}" ]; then
 		local unit="$(systemd-escape "fs$seq").scope"
 		systemctl reset-failed "${unit}" &> /dev/null
-		systemd-run --quiet --unit "${unit}" --scope "${cmd[@]}"
+		systemd-run --quiet --unit "${unit}" --scope "${cmd[@]}" &
+		CHILDPID=$!
+		wait
 		res=$?
+		unset CHILDPID
 		systemctl stop "${unit}" &> /dev/null
 		return "${res}"
 	else
-		"${cmd[@]}"
+		# bash won't run the SIGINT trap handler while there are
+		# foreground children in a separate session, so we must run
+		# the test in the background and wait for it.
+		"${cmd[@]}" &
+		CHILDPID=$!
+		wait
+		unset CHILDPID
+	fi
+}
+
+_kill_seq() {
+	if [ -n "$CHILDPID" ]; then
+		# SIGPIPE will kill all the children (including fsstress)
+		# without bash logging fatal signal termination messages to the
+		# console
+		pkill -PIPE --session "$CHILDPID"
+		wait
+		unset CHILDPID
 	fi
 }
 
@@ -718,9 +741,9 @@ _prepare_test_list
 fstests_start_time="$(date +"%F %T")"
 
 if $OPTIONS_HAVE_SECTIONS; then
-	trap "_summary; exit \$status" 0 1 2 3 15
+	trap "_kill_seq; _summary; exit \$status" 0 1 2 3 15
 else
-	trap "_wrapup; exit \$status" 0 1 2 3 15
+	trap "_kill_seq; _wrapup; exit \$status" 0 1 2 3 15
 fi
 
 function run_section()
diff --git a/common/fuzzy b/common/fuzzy
index 0a2d91542b561e..772ce7ddcff6d8 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -891,7 +891,7 @@ __stress_xfs_scrub_loop() {
 	local runningfile="$2"
 	local scrub_startat="$3"
 	shift; shift; shift
-	local sigint_ret="$(( $(kill -l SIGINT) + 128 ))"
+	local signal_ret="$(( $(kill -l SIGPIPE) + 128 ))"
 	local scrublog="$tmp.scrub"
 
 	while __stress_scrub_running "$scrub_startat" "$runningfile"; do
@@ -901,8 +901,8 @@ __stress_xfs_scrub_loop() {
 	while __stress_scrub_running "$end" "$runningfile"; do
 		_scratch_scrub "$@" &> $scrublog
 		res=$?
-		if [ "$res" -eq "$sigint_ret" ]; then
-			# Ignore SIGINT because the cleanup function sends
+		if [ "$res" -eq "$signal_ret" ]; then
+			# Ignore SIGPIPE because the cleanup function sends
 			# that to terminate xfs_scrub
 			res=0
 		fi
@@ -1173,13 +1173,11 @@ _scratch_xfs_stress_scrub_cleanup() {
 	rm -f "$runningfile"
 	echo "Cleaning up scrub stress run at $(date)" >> $seqres.full
 
-	# Send SIGINT so that bash won't print a 'Terminated' message that
-	# distorts the golden output.
 	echo "Killing stressor processes at $(date)" >> $seqres.full
-	_kill_fsstress
-	pkill -INT --parent $$ xfs_io >> $seqres.full 2>&1
-	pkill -INT --parent $$ fsx >> $seqres.full 2>&1
-	pkill -INT --parent $$ xfs_scrub >> $seqres.full 2>&1
+	_pkill --echo -PIPE fsstress >> $seqres.full 2>&1
+	_pkill --echo -PIPE xfs_io >> $seqres.full 2>&1
+	_pkill --echo -PIPE fsx >> $seqres.full 2>&1
+	_pkill --echo -PIPE xfs_scrub >> $seqres.full 2>&1
 
 	# Tests are not allowed to exit with the scratch fs frozen.  If we
 	# started a fs freeze/thaw background loop, wait for that loop to exit
@@ -1209,6 +1207,7 @@ _scratch_xfs_stress_scrub_cleanup() {
 	# Wait for the remaining children to exit.
 	echo "Waiting for children to exit at $(date)" >> $seqres.full
 	wait
+	echo "Children exited as of $(date)" >> $seqres.full
 
 	# Ensure the scratch fs is also writable before we exit.
 	if [ -n "$__SCRUB_STRESS_REMOUNT_LOOP" ]; then
diff --git a/common/rc b/common/rc
index 459be11c11c405..d143ba36265c6c 100644
--- a/common/rc
+++ b/common/rc
@@ -30,6 +30,12 @@ _test_sync()
 	_sync_fs $TEST_DIR
 }
 
+# Kill only the test processes started by this test
+_pkill()
+{
+	pkill --session 0 "$@"
+}
+
 # Common execution handling for fsstress invocation.
 #
 # We need per-test fsstress binaries because of the way fsstress forks and
@@ -69,7 +75,7 @@ _kill_fsstress()
 	if [ -n "$_FSSTRESS_PID" ]; then
 		# use SIGPIPE to avoid "Killed" messages from bash
 		echo "killing $_FSSTRESS_BIN" >> $seqres.full
-		pkill -PIPE $_FSSTRESS_BIN >> $seqres.full 2>&1
+		_pkill -PIPE $_FSSTRESS_BIN >> $seqres.full 2>&1
 		_wait_for_fsstress
 		return $?
 	fi
@@ -2740,9 +2746,11 @@ _require_user_exists()
 	[ "$?" == "0" ] || _notrun "$user user not defined."
 }
 
+# Run all non-root processes in the same session as the root.  Believe it or
+# not, passing $SHELL in this manner works both for "su" and "su -c cmd".
 _su()
 {
-	su "$@"
+	su --session-command $SHELL "$@"
 }
 
 # check if a user exists and is able to execute commands.
diff --git a/tests/generic/310 b/tests/generic/310
index 52babfdc803a21..570cc5f3859548 100755
--- a/tests/generic/310
+++ b/tests/generic/310
@@ -29,7 +29,7 @@ _begin_fstest auto
 # Override the default cleanup function.
 _cleanup()
 {
-	pkill -9 $seq.t_readdir > /dev/null 2>&1
+	_pkill -9 $seq.t_readdir > /dev/null 2>&1
 	wait
 	rm -rf $TEST_DIR/tmp
 	rm -f $tmp.*
@@ -83,7 +83,7 @@ _test_read()
 {
 	 $TEST_DIR/$seq.t_readdir_1 $SEQ_DIR > /dev/null 2>&1 &
 	sleep $RUN_TIME
-	pkill -PIPE $seq.t_readdir_1
+	_pkill -PIPE $seq.t_readdir_1
 	wait
 
 	check_kernel_bug
@@ -97,7 +97,7 @@ _test_lseek()
 	$TEST_DIR/$seq.t_readdir_2 $SEQ_DIR > /dev/null 2>&1 &
 	readdir_pid=$!
 	sleep $RUN_TIME
-	pkill -PIPE $seq.t_readdir_2
+	_pkill -PIPE $seq.t_readdir_2
 	wait
 
 	check_kernel_bug
diff --git a/tests/generic/561 b/tests/generic/561
index afe727ac56cbd9..b260aaf16c9256 100755
--- a/tests/generic/561
+++ b/tests/generic/561
@@ -40,7 +40,7 @@ function end_test()
 	# stop duperemove running
 	if [ -e $dupe_run ]; then
 		rm -f $dupe_run
-		pkill $dedup_bin >/dev/null 2>&1
+		_pkill $dedup_bin >/dev/null 2>&1
 		wait $dedup_pids
 		rm -f $dedup_prog
 	fi

[PATCH 09/23] unmount: resume logging of stdout and stderr for filtering

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

There's a number of places where a test program calls a variant of
_unmount but then pipes the output through a _filter script or
something.  The new _unmount helper redirects stdout and stderr to
seqres.full, which means that those error messages (some of which are
encoded in the golden outputs) are suppressed.  This leads to test
regressions in generic/050 and other places, so let's resume logging.

This also undoes all the changes that removed /dev/null redirection of
unmount calls.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: 4c6bc4565105e6 ("fstests: clean up mount and unmount operations")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 check             |   10 ++++++++--
 common/quota      |    2 +-
 common/rc         |   21 +++++++++++++++++++--
 tests/generic/050 |    2 +-
 tests/generic/085 |    2 +-
 tests/generic/361 |    4 ++--
 tests/generic/590 |    2 +-
 tests/generic/746 |    2 +-
 tests/xfs/149     |    2 +-
 tests/xfs/530     |    2 +-
 10 files changed, 36 insertions(+), 13 deletions(-)


diff --git a/check b/check
index bafe48bf12db67..126a77441d700d 100755
--- a/check
+++ b/check
@@ -1026,8 +1026,8 @@ function run_section()
 
 		if [ $sts -ne 0 ]; then
 			_dump_err_cont "[failed, exit status $sts]"
-			_test_unmount 2>> $seqres.full
-			_scratch_unmount 2>> $seqres.full
+			_test_unmount 2> /dev/null
+			_scratch_unmount 2> /dev/null
 			rm -f ${RESULT_DIR}/require_test*
 			rm -f ${RESULT_DIR}/require_scratch*
 			# Even though we failed, there may be something interesting in
@@ -1113,6 +1113,12 @@ function run_section()
 		_stash_test_status "$seqnum" "$tc_status"
 	done
 
+	# Reset these three variables so that unmount output doesn't get
+	# written to $seqres.full of the last test to run.
+	seq="check.$$"
+	check="$RESULT_BASE/check"
+	seqres="$check"
+
 	sect_stop=`_wallclock`
 	interrupt=false
 	_wrapup
diff --git a/common/quota b/common/quota
index 8688116c6547a9..4dad9b79a27a7f 100644
--- a/common/quota
+++ b/common/quota
@@ -274,7 +274,7 @@ _choose_prid()
 
 _qmount()
 {
-    _scratch_unmount
+    _scratch_unmount >/dev/null 2>&1
     _try_scratch_mount || _fail "qmount failed"
     # xfs doesn't need these setups and quotacheck even fails on xfs
     # redirect the output to $seqres.full for debug purpose and ignore results
diff --git a/common/rc b/common/rc
index d143ba36265c6c..9e34c301b0deb0 100644
--- a/common/rc
+++ b/common/rc
@@ -480,11 +480,28 @@ _scratch_mount_idmapped()
 }
 
 # Unmount the filesystem based on the directory or device passed.
+# Log everything that happens to seqres.full, and use BASHPID because
+# background subshells have the same $$ as the parent but not the same
+# $BASHPID.
 _unmount()
 {
-	local args="$*"
+	local outlog="$tmp.$BASHPID.umount"
+	local errlog="$tmp.$BASHPID.umount.err"
 
-	$UMOUNT_PROG $args >> $seqres.full 2>&1
+	rm -f "$outlog" "$errlog"
+	$UMOUNT_PROG "$@" 2> "$errlog" > "$outlog"
+	local res="${PIPESTATUS[0]}"
+
+	if [ -s "$outlog" ]; then
+		cat "$outlog" >> $seqres.full
+		cat "$outlog"
+	fi
+	if [ -s "$errlog" ]; then
+		cat "$errlog" >> $seqres.full
+		>&2 cat "$errlog"
+	fi
+	rm -f "$outlog" "$errlog"
+	return $res
 }
 
 _scratch_unmount()
diff --git a/tests/generic/050 b/tests/generic/050
index 8e9456db279003..affb072df5969f 100755
--- a/tests/generic/050
+++ b/tests/generic/050
@@ -89,7 +89,7 @@ _try_scratch_mount 2>&1 | _filter_ro_mount | _filter_scratch
 
 # expects an error, so open code the unmount
 echo "unmounting read-only filesystem"
-$UMOUNT_PROG $SCRATCH_DEV 2>&1 | _filter_scratch | _filter_ending_dot
+_scratch_unmount 2>&1 | _filter_scratch | _filter_ending_dot
 
 #
 # This is the way out if the underlying device really is read-only.
diff --git a/tests/generic/085 b/tests/generic/085
index 7671a36ab9524f..d3fa10be9ccace 100755
--- a/tests/generic/085
+++ b/tests/generic/085
@@ -29,7 +29,7 @@ cleanup_dmdev()
 	fi
 	# in case it's still suspended and/or mounted
 	$DMSETUP_PROG resume $lvdev >> $seqres.full 2>&1
-	_unmount -q $SCRATCH_MNT
+	_unmount -q $SCRATCH_MNT >/dev/null 2>&1
 	_dmsetup_remove $node
 }
 
diff --git a/tests/generic/361 b/tests/generic/361
index e2b7984361e87c..b584af47540020 100755
--- a/tests/generic/361
+++ b/tests/generic/361
@@ -16,7 +16,7 @@ _begin_fstest auto quick
 # Override the default cleanup function.
 _cleanup()
 {
-	_unmount $fs_mnt
+	_unmount $fs_mnt &>> /dev/null
 	[ -n "$loop_dev" ] && _destroy_loop_device $loop_dev
 	cd /
 	rm -f $tmp.*
@@ -54,7 +54,7 @@ $XFS_IO_PROG -fc "pwrite 0 520m" $fs_mnt/testfile >>$seqres.full 2>&1
 # remount should not hang
 $MOUNT_PROG -o remount,ro $fs_mnt >>$seqres.full 2>&1
 
-_unmount $fs_mnt
+_unmount $fs_mnt &>/dev/null
 _destroy_loop_device $loop_dev
 unset loop_dev
 
diff --git a/tests/generic/590 b/tests/generic/590
index 1adeef4c2ad52c..ba1337a856f15d 100755
--- a/tests/generic/590
+++ b/tests/generic/590
@@ -15,7 +15,7 @@ _begin_fstest auto prealloc preallocrw
 # Override the default cleanup function.
 _cleanup()
 {
-	_scratch_unmount
+	_scratch_unmount &>/dev/null
 	[ -n "$loop_dev" ] && _destroy_loop_device $loop_dev
 	cd /
 	rm -f $tmp.*
diff --git a/tests/generic/746 b/tests/generic/746
index ba8ed25e845776..6f02b1cc354782 100755
--- a/tests/generic/746
+++ b/tests/generic/746
@@ -223,7 +223,7 @@ while read line; do
 done < $fiemap_after
 echo "done."
 
-_unmount $loop_mnt
+_unmount $loop_mnt &>/dev/null
 _destroy_loop_device $loop_dev
 unset loop_dev
 
diff --git a/tests/xfs/149 b/tests/xfs/149
index 9a96f82ede1761..28dfc7f04c1773 100755
--- a/tests/xfs/149
+++ b/tests/xfs/149
@@ -22,7 +22,7 @@ loop_symlink=$TEST_DIR/loop_symlink.$$
 # Override the default cleanup function.
 _cleanup()
 {
-    _unmount $mntdir
+    _unmount $mntdir &>/dev/null
     [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev
     rmdir $mntdir
     rm -f $loop_symlink
diff --git a/tests/xfs/530 b/tests/xfs/530
index d0d0e2665070f8..95ab32f1e1f828 100755
--- a/tests/xfs/530
+++ b/tests/xfs/530
@@ -116,7 +116,7 @@ done
 echo "Check filesystem"
 _check_scratch_fs
 
-_scratch_unmount
+_scratch_unmount &> /dev/null
 _destroy_loop_device $rt_loop_dev
 unset rt_loop_dev
 

[PATCH 10/23] mkfs: don't hardcode log size

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Commit 000813899afb46 hardcoded a log size of 256MB into xfs/501,
xfs/502, and generic/530.  This seems to be an attempt to reduce test
run times by increasing the log size so that more background threads can
run in parallel.  Unfortunately, this breaks a couple of my test
configurations:

 - External logs smaller than 256MB
 - Internal logs where the AG size is less than 256MB

For example, here's seqres.full from a failed xfs/501 invocation:

** mkfs failed with extra mkfs options added to " -m metadir=2,autofsck=1,uquota,gquota,pquota, -d rtinherit=1," by test 501 **
** attempting to mkfs using only test 501 options: -l size=256m **
size 256m specified for log subvolume is too large, maximum is 32768 blocks
<snip>
mount -ortdev=/dev/sdb4 -ologdev=/dev/sdb2 /dev/sda4 /opt failed
umount: /dev/sda4: not mounted.

Note that there's some formatting error here, so we jettison the entire
rt configuration to force the log size option, but then mount fails
because we didn't edit out the rtdev option there too.

Fortunately, mkfs.xfs already /has/ a few options to try to improve
parallelism in the filesystem by avoiding contention on the log grant
heads by scaling up the log size.  These options are aware of log and AG
size constraints so they won't conflict with other geometry options.

Use them.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: 000813899afb46 ("fstests: scale some tests for high CPU count sanity")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/rc         |   27 +++++++++++++++++++++++++++
 tests/generic/530 |    6 +-----
 tests/generic/531 |    6 +-----
 tests/xfs/501     |    2 +-
 tests/xfs/502     |    2 +-
 5 files changed, 31 insertions(+), 12 deletions(-)


diff --git a/common/rc b/common/rc
index 9e34c301b0deb0..885669beeb5e26 100644
--- a/common/rc
+++ b/common/rc
@@ -689,6 +689,33 @@ _test_cycle_mount()
     _test_mount
 }
 
+# Are there mkfs options to try to improve concurrency?
+_scratch_mkfs_concurrency_options()
+{
+	local nr_cpus="$(( $1 * LOAD_FACTOR ))"
+
+	case "$FSTYP" in
+	xfs)
+		# If any concurrency options are already specified, don't
+		# compute our own conflicting ones.
+		echo "$SCRATCH_OPTIONS $MKFS_OPTIONS" | \
+			grep -q 'concurrency=' &&
+			return
+
+		local sections=(d r)
+
+		# -l concurrency does not work with external logs
+		test _has_logdev || sections+=(l)
+
+		for section in "${sections[@]}"; do
+			$MKFS_XFS_PROG -$section concurrency=$nr_cpus 2>&1 | \
+				grep -q "unknown option -$section" ||
+				echo "-$section concurrency=$nr_cpus "
+		done
+		;;
+	esac
+}
+
 _scratch_mkfs_options()
 {
     _scratch_options mkfs
diff --git a/tests/generic/530 b/tests/generic/530
index f2513156a920e8..7413840476b588 100755
--- a/tests/generic/530
+++ b/tests/generic/530
@@ -25,11 +25,7 @@ _require_test_program "t_open_tmpfiles"
 # For XFS, pushing 50000 unlinked inode inactivations through a small xfs log
 # can result in bottlenecks on the log grant heads, so try to make the log
 # larger to reduce runtime.
-if [ "$FSTYP" = "xfs" ] && ! _has_logdev; then
-    _scratch_mkfs "-l size=256m" >> $seqres.full 2>&1
-else
-    _scratch_mkfs >> $seqres.full 2>&1
-fi
+_scratch_mkfs $(_scratch_mkfs_concurrency_options 32) >> $seqres.full 2>&1
 _scratch_mount
 
 # Set ULIMIT_NOFILE to min(file-max / 2, 50000 files per LOAD_FACTOR)
diff --git a/tests/generic/531 b/tests/generic/531
index ed6c3f91153ecc..3ba2790c923464 100755
--- a/tests/generic/531
+++ b/tests/generic/531
@@ -23,11 +23,7 @@ _require_test_program "t_open_tmpfiles"
 
 # On high CPU count machines, this runs a -lot- of create and unlink
 # concurrency. Set the filesytsem up to handle this.
-if [ $FSTYP = "xfs" ]; then
-	_scratch_mkfs "-d agcount=32" >> $seqres.full 2>&1
-else
-	_scratch_mkfs >> $seqres.full 2>&1
-fi
+_scratch_mkfs $(_scratch_mkfs_concurrency_options 32) >> $seqres.full 2>&1
 _scratch_mount
 
 # Try to load up all the CPUs, two threads per CPU.
diff --git a/tests/xfs/501 b/tests/xfs/501
index 678c51b52948c5..4b29ef97d36c1a 100755
--- a/tests/xfs/501
+++ b/tests/xfs/501
@@ -33,7 +33,7 @@ _require_xfs_sysfs debug/log_recovery_delay
 _require_scratch
 _require_test_program "t_open_tmpfiles"
 
-_scratch_mkfs "-l size=256m" >> $seqres.full 2>&1
+_scratch_mkfs $(_scratch_mkfs_concurrency_options 32) >> $seqres.full 2>&1
 _scratch_mount
 
 # Set ULIMIT_NOFILE to min(file-max / 2, 30000 files per LOAD_FACTOR)
diff --git a/tests/xfs/502 b/tests/xfs/502
index 10b0017f6b2eb2..df3e7bcb17872d 100755
--- a/tests/xfs/502
+++ b/tests/xfs/502
@@ -23,7 +23,7 @@ _require_xfs_io_error_injection "iunlink_fallback"
 _require_scratch
 _require_test_program "t_open_tmpfiles"
 
-_scratch_mkfs "-l size=256m" | _filter_mkfs 2> $tmp.mkfs > /dev/null
+_scratch_mkfs $(_scratch_mkfs_concurrency_options 32) | _filter_mkfs 2> $tmp.mkfs > /dev/null
 cat $tmp.mkfs >> $seqres.full
 . $tmp.mkfs
 

[PATCH 11/23] common/xfs: find loop devices for non-blockdevs passed to _prepare_for_eio_shutdown

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

xfs/336 does this somewhat sketchy thing where it mdrestores into a
regular file, and then does this to validate the restored metadata:

SCRATCH_DEV=$TEST_DIR/image _scratch_mount

Unfortunately, commit 1a49022fab9b4d causes the following regression:

 --- /tmp/fstests/tests/xfs/336.out      2024-11-12 16:17:36.733447713 -0800
 +++ /var/tmp/fstests/xfs/336.out.bad    2025-01-04 19:10:39.861871114 -0800
 @@ -5,4 +5,5 @@ Create big file
  Explode the rtrmapbt
  Create metadump file
  Restore metadump
 -Check restored fs
 +Usage: _set_fs_sysfs_attr <mounted_device> <attr> <content>
 +(see /var/tmp/fstests/xfs/336.full for details)

This is due to the fact that SCRATCH_DEV is temporarily reassigned to
the regular file.  That path is passed straight through _scratch_mount
to _xfs_prepare_for_eio_shutdown, but that helper _fails because the
"dev" argument isn't actually a path to a block device.

Fix this by detecting non-bdevs and finding (we hope) the loop device
that was created to handle the mount.  While we're at it, have the
helper return the exit code from mount, not _prepare_for_eio_shutdown.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: 1a49022fab9b4d ("fstests: always use fail-at-unmount semantics for XFS")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/rc  |    8 ++++++++
 common/xfs |    6 ++++++
 2 files changed, 14 insertions(+)


diff --git a/common/rc b/common/rc
index 885669beeb5e26..4419cfc3188374 100644
--- a/common/rc
+++ b/common/rc
@@ -441,6 +441,7 @@ _try_scratch_mount()
 	[ $mount_ret -ne 0 ] && return $mount_ret
 	_idmapped_mount $SCRATCH_DEV $SCRATCH_MNT
 	_prepare_for_eio_shutdown $SCRATCH_DEV
+	return $mount_ret
 }
 
 # mount scratch device with given options and _fail if mount fails
@@ -658,6 +659,7 @@ _test_mount()
     [ $mount_ret -ne 0 ] && return $mount_ret
     _idmapped_mount $TEST_DEV $TEST_DIR
     _prepare_for_eio_shutdown $TEST_DEV
+    return $mount_ret
 }
 
 _test_unmount()
@@ -4469,6 +4471,12 @@ _destroy_loop_device()
 	losetup -d $dev || _fail "Cannot destroy loop device $dev"
 }
 
+# Find the loop bdev for a given file, if there is one.
+_find_loop_device()
+{
+	losetup --list -n -O NAME -j "$1"
+}
+
 _scale_fsstress_args()
 {
     local args=""
diff --git a/common/xfs b/common/xfs
index 0417a40adba3e2..c68bd6d7c773ac 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1110,6 +1110,12 @@ _xfs_prepare_for_eio_shutdown()
 	local dev="$1"
 	local ctlfile="error/fail_at_unmount"
 
+	# Is this a regular file?  Check if there's a loop device somewhere.
+	# Hopefully that lines up with a mounted filesystem.
+	if [ ! -b "$dev" ]; then
+		dev=$(_find_loop_device "$1" | tail -n 1)
+	fi
+
 	# Once we enable IO errors, it's possible that a writer thread will
 	# trip over EIO, cancel the transaction, and shut down the system.
 	# This is expected behavior, so we need to remove the "Internal error"

[PATCH 12/23] preamble: fix missing _kill_fsstress

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Commit 8973af00ec212f added a _kill_fsstress to the standard _cleanup
function.  However, if something breaks during test program
initialization before it gets to sourcing common/rc, then you get
failures that look like this:

 --- /tmp/fstests/tests/generic/556.out  2024-09-25 12:09:52.938797554 -0700
 +++ /var/tmp/fstests/generic/556.out.bad        2025-01-04 22:34:01.268327003 -0800
 @@ -1,16 +1,3 @@
  QA output created by 556
 -SCRATCH_MNT/basic Casefold
 -SCRATCH_MNT/basic
 -SCRATCH_MNT/casefold_flag_removal Casefold
 -SCRATCH_MNT/casefold_flag_removal Casefold
 -SCRATCH_MNT/flag_inheritance/d1/d2/d3 Casefold
 -SCRATCH_MNT/symlink/ind1/TARGET
 -mv: 'SCRATCH_MNT/rename/rename' and 'SCRATCH_MNT/rename/RENAME' are the same file
 -# file: SCRATCH_MNT/xattrs/x
 -user.foo="bar"
 -
 -# file: SCRATCH_MNT/xattrs/x/f1
 -user.foo="bar"
 -
 -touch: 'SCRATCH_MNT/strict/corac'$'\314\247\303': Invalid argument
 -touch: 'SCRATCH_MNT/strict/cora'$'\303\247\303': Invalid argument
 +./tests/generic/556: 108: common/config: Syntax error: "&" unexpected
 +./tests/generic/556: 10: _kill_fsstress: not found

It's that last line that's unnecessary.  Fix this by checking for the
presence of a _kill_fsstress before invoking it.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/preamble |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/common/preamble b/common/preamble
index 78e45d522f482c..0c9ee2e0377dd5 100644
--- a/common/preamble
+++ b/common/preamble
@@ -7,7 +7,7 @@
 # Standard cleanup function.  Individual tests can override this.
 _cleanup()
 {
-	_kill_fsstress
+	command -v _kill_fsstress &>/dev/null && _kill_fsstress
 	cd /
 	rm -r -f $tmp.*
 }

[PATCH 13/23] generic/650: revert SOAK DURATION changes

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Prior to commit 8973af00ec21, in the absence of an explicit
SOAK_DURATION, this test would run 2500 fsstress operations each of ten
times through the loop body.  On the author's machines, this kept the
runtime to about 30s total.  Oddly, this was changed to 30s per loop
body with no specific justification in the middle of an fsstress process
management change.

On the author's machine, this explodes the runtime from ~30s to 420s.
Put things back the way they were.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/generic/650 |    5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)


diff --git a/tests/generic/650 b/tests/generic/650
index 60f86fdf518961..d376488f2fedeb 100755
--- a/tests/generic/650
+++ b/tests/generic/650
@@ -68,11 +68,8 @@ test "$nr_cpus" -gt 1024 && nr_cpus="$nr_hotplug_cpus"
 fsstress_args+=(-p $nr_cpus)
 if [ -n "$SOAK_DURATION" ]; then
 	test "$SOAK_DURATION" -lt 10 && SOAK_DURATION=10
-else
-	# run for 30s per iteration max
-	SOAK_DURATION=300
+	fsstress_args+=(--duration="$((SOAK_DURATION / 10))")
 fi
-fsstress_args+=(--duration="$((SOAK_DURATION / 10))")
 
 nr_ops=$((2500 * TIME_FACTOR))
 fsstress_args+=(-n $nr_ops)

[PATCH 14/23] generic/032: fix pinned mount failure

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

generic/032 now periodically fails with:

 --- /tmp/fstests/tests/generic/032.out	2025-01-05 11:42:14.427388698 -0800
 +++ /var/tmp/fstests/generic/032.out.bad	2025-01-06 18:20:17.122818195 -0800
 @@ -1,5 +1,7 @@
  QA output created by 032
  100 iterations
 -000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  >................<
 -*
 -100000
 +umount: /opt: target is busy.
 +mount: /opt: /dev/sda4 already mounted on /opt.
 +       dmesg(1) may have more information after failed mount system call.
 +cycle mount failed
 +(see /var/tmp/fstests/generic/032.full for details)

The root cause of this regression is the _syncloop subshell.  This
background process runs _scratch_sync, which is actually an xfs_io
process that calls syncfs on the scratch mount.

Unfortunately, while the test kills the _syncloop subshell, it doesn't
actually kill the xfs_io process.  If the xfs_io process is in D state
running the syncfs, it won't react to the signal, but it will pin the
mount.  Then the _scratch_cycle_mount fails because the mount is pinned.

Prior to commit 8973af00ec212f the _syncloop ran sync(1) which avoided
pinning the scratch filesystem.

Fix this by pgrepping for the xfs_io process and killing and waiting for
it if necessary.

Cc: <fstests@vger.kernel.org> # v2024.12.08
Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/rc         |    6 ++++++
 tests/generic/032 |    9 +++++++++
 2 files changed, 15 insertions(+)


diff --git a/common/rc b/common/rc
index 4419cfc3188374..d7f3c48eafe590 100644
--- a/common/rc
+++ b/common/rc
@@ -36,6 +36,12 @@ _pkill()
 	pkill --session 0 "$@"
 }
 
+# Find only the test processes started by this test
+_pgrep()
+{
+	pgrep --session 0 "$@"
+}
+
 # Common execution handling for fsstress invocation.
 #
 # We need per-test fsstress binaries because of the way fsstress forks and
diff --git a/tests/generic/032 b/tests/generic/032
index 30290c7225a2fa..48d594fe9315b8 100755
--- a/tests/generic/032
+++ b/tests/generic/032
@@ -81,6 +81,15 @@ echo $iters iterations
 kill $syncpid
 wait
 
+# The xfs_io instance started by _scratch_sync could be stuck in D state when
+# the subshell running _syncloop & is killed.  That xfs_io process pins the
+# mount so we must kill it and wait for it to die before cycling the mount.
+dead_syncfs_pid=$(_pgrep xfs_io)
+if [ -n "$dead_syncfs_pid" ]; then
+	_pkill xfs_io
+	wait $dead_syncfs_pid
+fi
+
 # clear page cache and dump the file
 _scratch_cycle_mount
 _hexdump $SCRATCH_MNT/file

[PATCH 15/23] fuzzy: stop __stress_scrub_fsx_loop if fsx fails

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Stop the fsx scrub stress loop if fsx returns a nonzero error code.

Cc: <fstests@vger.kernel.org> # v2023.01.15
Fixes: a2056ca8917bc8 ("fuzzy: enhance scrub stress testing to use fsx")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/fuzzy |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)


diff --git a/common/fuzzy b/common/fuzzy
index 772ce7ddcff6d8..46771b1d117106 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -942,6 +942,7 @@ __stress_scrub_fsx_loop() {
 	local remount_period="$3"
 	local stress_tgt="$4"	# ignored
 	local focus=(-q -X)	# quiet, validate file contents
+	local res
 
 	# As of November 2022, 2 million fsx ops should be enough to keep
 	# any filesystem busy for a couple of hours.
@@ -993,7 +994,9 @@ __stress_scrub_fsx_loop() {
 		# Need to recheck running conditions if we cleared anything
 		__stress_scrub_clean_scratch && continue
 		$here/ltp/fsx $args >> $seqres.full
-		echo "fsx exits with $? at $(date)" >> $seqres.full
+		res=$?
+		echo "fsx exits with $res at $(date)" >> $seqres.full
+		test "$res" -ne 0 && break
 	done
 	rm -f "$runningfile"
 }

[PATCH 16/23] fuzzy: don't use readarray for xfsfind output

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Some of the scrub stress tests (e.g. xfs/796) walk the directory tree to
find filepaths to scrub, and load the entire list of paths into a bash
array.  On a large filesystem or a long-running test this is hugely
wasteful of memory because we use each path exactly once.

Fix __stress_one_scrub_loop to avoid this by reading lines directly from
the output of the xfsfind utility.  However, we play some games with fd
77 so that the processes in the loop body will use the same stdin as the
test and /not/ the piped stdout of xfsfind.

To avoid read(1) becoming confused by newlines in the file paths, adapt
xfsfind to print nulls between pathnames, and the bash code to recognize
them.

This was a debugging patch while I was trying to figure out why xfs/286
and other scrub soak tests started OOMing after the v2024.12.08 changes,
though in the end the OOMs were the result of memory leaks in fsstress.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/fuzzy  |   22 +++++++++++++---------
 src/xfsfind.c |   14 +++++++++++---
 2 files changed, 24 insertions(+), 12 deletions(-)


diff --git a/common/fuzzy b/common/fuzzy
index 46771b1d117106..f97c92340ba9ae 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -861,14 +861,14 @@ __stress_one_scrub_loop() {
 		;;
 	esac
 
-	local target_cmd=(echo "$scrub_tgt")
+	local target_cmd=(echo -en "$scrub_tgt\0")
 	case "$scrub_tgt" in
-	"%file%")	target_cmd=($here/src/xfsfind -q  "$SCRATCH_MNT");;
-	"%attrfile%")	target_cmd=($here/src/xfsfind -qa "$SCRATCH_MNT");;
-	"%datafile%")	target_cmd=($here/src/xfsfind -qb "$SCRATCH_MNT");;
-	"%dir%")	target_cmd=($here/src/xfsfind -qd "$SCRATCH_MNT");;
-	"%regfile%")	target_cmd=($here/src/xfsfind -qr "$SCRATCH_MNT");;
-	"%cowfile%")	target_cmd=($here/src/xfsfind -qs "$SCRATCH_MNT");;
+	"%file%")	target_cmd=($here/src/xfsfind -0q  "$SCRATCH_MNT");;
+	"%attrfile%")	target_cmd=($here/src/xfsfind -0qa "$SCRATCH_MNT");;
+	"%datafile%")	target_cmd=($here/src/xfsfind -0qb "$SCRATCH_MNT");;
+	"%dir%")	target_cmd=($here/src/xfsfind -0qd "$SCRATCH_MNT");;
+	"%regfile%")	target_cmd=($here/src/xfsfind -0qr "$SCRATCH_MNT");;
+	"%cowfile%")	target_cmd=($here/src/xfsfind -0qs "$SCRATCH_MNT");;
 	esac
 
 	while __stress_scrub_running "$scrub_startat" "$runningfile"; do
@@ -876,12 +876,16 @@ __stress_one_scrub_loop() {
 	done
 
 	while __stress_scrub_running "$end" "$runningfile"; do
-		readarray -t fnames < <("${target_cmd[@]}" 2>> $seqres.full)
-		for fname in "${fnames[@]}"; do
+		# Attach the stdout of xfsfind to fd 77 so that we can read
+		# pathnames from that file descriptor without passing the pipe
+		# to the loop body as stdin.
+		exec 77< <("${target_cmd[@]}" 2>> $seqres.full)
+		while read -u 77 -d '' fname; do
 			$XFS_IO_PROG -x "${xfs_io_args[@]}" "$fname" 2>&1 | \
 				__stress_scrub_filter_output "${extra_filters[@]}"
 			__stress_scrub_running "$end" "$runningfile" || break
 		done
+		exec 77<&-
 	done
 }
 
diff --git a/src/xfsfind.c b/src/xfsfind.c
index c81deaf64f57e9..2043d01ded3210 100644
--- a/src/xfsfind.c
+++ b/src/xfsfind.c
@@ -20,6 +20,7 @@ static int want_dir;
 static int want_regfile;
 static int want_sharedfile;
 static int report_errors = 1;
+static int print0;
 
 static int
 check_datafile(
@@ -115,6 +116,7 @@ print_help(
 	printf("\n");
 	printf("Print all file paths matching any of the given predicates.\n");
 	printf("\n");
+	printf("-0	Print nulls between paths instead of newlines.\n");
 	printf("-a	Match files with xattrs.\n");
 	printf("-b	Match files with data blocks.\n");
 	printf("-d	Match directories.\n");
@@ -208,8 +210,13 @@ visit(
 out_fd:
 	close(fd);
 out:
-	if (printme)
-		printf("%s\n", path);
+	if (printme) {
+		if (print0)
+			printf("%s%c", path, 0);
+		else
+			printf("%s\n", path);
+		fflush(stdout);
+	}
 	return retval;
 }
 
@@ -236,8 +243,9 @@ main(
 	int			c;
 	int			ret;
 
-	while ((c = getopt(argc, argv, "abdqrs")) >= 0) {
+	while ((c = getopt(argc, argv, "0abdqrs")) >= 0) {
 		switch (c) {
+		case '0':	print0 = 1;          break;
 		case 'a':	want_attrfile = 1;   break;
 		case 'b':	want_datafile = 1;   break;
 		case 'd':	want_dir = 1;        break;

[PATCH 17/23] fuzzy: always stop the scrub fsstress loop on error

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Always abort the non-remount scrub fsstress loop in
__stress_scrub_fsstress_loop if fsstress returns a nonzero exit code.

Cc: <fstests@vger.kernel.org> # v2023.01.15
Fixes: 20df87599f66d0 ("fuzzy: make scrub stress loop control more robust")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/fuzzy |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)


diff --git a/common/fuzzy b/common/fuzzy
index f97c92340ba9ae..e9150173a5d723 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -1012,6 +1012,7 @@ __stress_scrub_fsstress_loop() {
 	local remount_period="$3"
 	local stress_tgt="$4"
 	local focus=()
+	local res
 
 	case "$stress_tgt" in
 	"parent")
@@ -1143,7 +1144,9 @@ __stress_scrub_fsstress_loop() {
 		# Need to recheck running conditions if we cleared anything
 		__stress_scrub_clean_scratch && continue
 		_run_fsstress $args >> $seqres.full
-		echo "fsstress exits with $? at $(date)" >> $seqres.full
+		res=$?
+		echo "$mode fsstress exits with $res at $(date)" >> $seqres.full
+		[ "$res" -ne 0 ] && break;
 	done
 	rm -f "$runningfile"
 }

[PATCH 18/23] fuzzy: port fsx and fsstress loop to use --duration

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Quite a while ago, I added --duration= arguments to fsx and fsstress,
and apparently I forgot to update the scrub stress loops to use them.
Replace the usage of timeout(1) for the remount_period versions of the
loop to clean up that code; and convert the non-remount loop so that
they don't run over time.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/fuzzy |   40 ++++++++++++++++++++++++++++------------
 1 file changed, 28 insertions(+), 12 deletions(-)


diff --git a/common/fuzzy b/common/fuzzy
index e9150173a5d723..331bf5ad7bbafa 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -939,6 +939,22 @@ __stress_scrub_clean_scratch() {
 	return 0
 }
 
+# Compute a --duration= interval for fsx and fsstress
+___stress_scrub_duration()
+{
+	local end="$1"
+	local remount_period="$2"
+	local now="$(date +%s)"
+	local delta="$((end - now))"
+
+	test "$delta" -lt 0 && delta=0
+
+	test -n "$remount_period" && test "$remount_period" -lt "$delta" && \
+		delta="$remount_period"
+
+	echo "--duration=$delta"
+}
+
 # Run fsx while we're testing online fsck.
 __stress_scrub_fsx_loop() {
 	local end="$1"
@@ -946,6 +962,7 @@ __stress_scrub_fsx_loop() {
 	local remount_period="$3"
 	local stress_tgt="$4"	# ignored
 	local focus=(-q -X)	# quiet, validate file contents
+	local duration
 	local res
 
 	# As of November 2022, 2 million fsx ops should be enough to keep
@@ -965,17 +982,12 @@ __stress_scrub_fsx_loop() {
 			# anything.
 			test "$mode" = "rw" && __stress_scrub_clean_scratch && continue
 
-			timeout -s TERM "$remount_period" $here/ltp/fsx \
-					$args $rw_arg >> $seqres.full
+			duration=$(___stress_scrub_duration "$end" "$remount_period")
+			$here/ltp/fsx $duration $args $rw_arg >> $seqres.full
 			res=$?
 			echo "$mode fsx exits with $res at $(date)" >> $seqres.full
-			if [ "$res" -ne 0 ] && [ "$res" -ne 124 ]; then
-				# Stop if fsx returns error.  Mask off
-				# the magic code 124 because that is how the
-				# timeout(1) program communicates that we ran
-				# out of time.
-				break;
-			fi
+			test "$res" -ne 0 && break
+
 			if [ "$mode" = "rw" ]; then
 				mode="ro"
 				rw_arg="-t 0 -w 0 -FHzCIJBE0"
@@ -997,7 +1009,8 @@ __stress_scrub_fsx_loop() {
 	while __stress_scrub_running "$end" "$runningfile"; do
 		# Need to recheck running conditions if we cleared anything
 		__stress_scrub_clean_scratch && continue
-		$here/ltp/fsx $args >> $seqres.full
+		duration=$(___stress_scrub_duration "$end" "$remount_period")
+		$here/ltp/fsx $duration $args >> $seqres.full
 		res=$?
 		echo "fsx exits with $res at $(date)" >> $seqres.full
 		test "$res" -ne 0 && break
@@ -1013,6 +1026,7 @@ __stress_scrub_fsstress_loop() {
 	local stress_tgt="$4"
 	local focus=()
 	local res
+	local duration
 
 	case "$stress_tgt" in
 	"parent")
@@ -1115,7 +1129,8 @@ __stress_scrub_fsstress_loop() {
 			# anything.
 			test "$mode" = "rw" && __stress_scrub_clean_scratch && continue
 
-			_run_fsstress_bg $args $rw_arg >> $seqres.full
+			duration=$(___stress_scrub_duration "$end" "$remount_period")
+			_run_fsstress_bg $duration $args $rw_arg >> $seqres.full
 			sleep $remount_period
 			_kill_fsstress
 			res=$?
@@ -1143,7 +1158,8 @@ __stress_scrub_fsstress_loop() {
 	while __stress_scrub_running "$end" "$runningfile"; do
 		# Need to recheck running conditions if we cleared anything
 		__stress_scrub_clean_scratch && continue
-		_run_fsstress $args >> $seqres.full
+		duration=$(___stress_scrub_duration "$end" "$remount_period")
+		_run_fsstress $duration $args >> $seqres.full
 		res=$?
 		echo "$mode fsstress exits with $res at $(date)" >> $seqres.full
 		[ "$res" -ne 0 ] && break;

[PATCH 19/23] common/rc: don't copy fsstress to $TEST_DIR

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Now that we can pkill only processes that were started by this test, we
don't need to copy the fsstress binary to $TEST_DIR to avoid killing the
wrong program instances.  This avoids a whole slew of ETXTBSY problems
with scrub stress tests that run multiple copies of fsstress in the
background.

Revert most of the changes to generic/270, because it wants to do
something fancy with the fsstress binary, so it needs to control the
process directly.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/rc         |   14 +++++---------
 tests/generic/270 |   10 ++++++----
 tests/generic/482 |    1 +
 3 files changed, 12 insertions(+), 13 deletions(-)


diff --git a/common/rc b/common/rc
index d7f3c48eafe590..25eb13ab2c5a48 100644
--- a/common/rc
+++ b/common/rc
@@ -56,11 +56,9 @@ _pgrep()
 # task name to kill.
 #
 # If tasks want to start fsstress themselves (e.g. under a different uid) then
-# they can set up _FSSTRESS_BIN and record _FSSTRESS_PID themselves. Then if the
-# test is killed then it will get cleaned up automatically.
+# they can record _FSSTRESS_PID themselves. Then if the test is killed then it
+# will get cleaned up automatically.
 
-_FSSTRESS_BIN="$seq.fsstress"
-_FSSTRESS_PROG="$TEST_DIR/$seq.fsstress"
 _FSSTRESS_PID=""
 _wait_for_fsstress()
 {
@@ -71,7 +69,6 @@ _wait_for_fsstress()
 		ret=$?
 		unset _FSSTRESS_PID
 	fi
-	rm -f $_FSSTRESS_PROG
 	return $ret
 }
 
@@ -80,8 +77,8 @@ _kill_fsstress()
 {
 	if [ -n "$_FSSTRESS_PID" ]; then
 		# use SIGPIPE to avoid "Killed" messages from bash
-		echo "killing $_FSSTRESS_BIN" >> $seqres.full
-		_pkill -PIPE $_FSSTRESS_BIN >> $seqres.full 2>&1
+		echo "killing fsstress" >> $seqres.full
+		_pkill -PIPE fsstress >> $seqres.full 2>&1
 		_wait_for_fsstress
 		return $?
 	fi
@@ -89,8 +86,7 @@ _kill_fsstress()
 
 _run_fsstress_bg()
 {
-	cp -f $FSSTRESS_PROG $_FSSTRESS_PROG
-	$_FSSTRESS_PROG $FSSTRESS_AVOID "$@" >> $seqres.full 2>&1 &
+	$FSSTRESS_PROG $FSSTRESS_AVOID "$@" >> $seqres.full 2>&1 &
 	_FSSTRESS_PID=$!
 }
 
diff --git a/tests/generic/270 b/tests/generic/270
index d74971bb535239..ce51592004fe77 100755
--- a/tests/generic/270
+++ b/tests/generic/270
@@ -28,8 +28,8 @@ _workout()
 	args=`_scale_fsstress_args -p128 -n999999999 -f setattr=1 $FSSTRESS_AVOID -d $out`
 	echo "fsstress $args" >> $seqres.full
 	# Grant chown capability 
-	cp $FSSTRESS_PROG $_FSSTRESS_PROG
-	$SETCAP_PROG cap_chown=epi $_FSSTRESS_PROG
+	cp $FSSTRESS_PROG $tmp.fsstress.bin
+	$SETCAP_PROG cap_chown=epi $tmp.fsstress.bin
 
 	# io_uring accounts memory it needs under the rlimit memlocked option,
 	# which can be quite low on some setups (especially 64K pagesize). root
@@ -37,7 +37,7 @@ _workout()
 	# io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited
 	# temporarily.
 	ulimit -l unlimited
-	_su $qa_user -c "$_FSSTRESS_PROG $args" > /dev/null 2>&1 &
+	_su $qa_user -c "$tmp.fsstress.bin $args" > /dev/null 2>&1 &
 	_FSSTRESS_PID=$!
 
 	echo "Run dd writers in parallel"
@@ -50,7 +50,9 @@ _workout()
 		sleep $enospc_time
 	done
 
-	_kill_fsstress
+	_pkill -PIPE -f fsstress
+	pidwait $_FSSTRESS_PID
+	return 0
 }
 
 _require_quota
diff --git a/tests/generic/482 b/tests/generic/482
index 0efc026a160040..8c114ee03058c6 100755
--- a/tests/generic/482
+++ b/tests/generic/482
@@ -68,6 +68,7 @@ lowspace=$((1024*1024 / 512))		# 1m low space threshold
 
 # Use a thin device to provide deterministic discard behavior. Discards are used
 # by the log replay tool for fast zeroing to prevent out-of-order replay issues.
+_test_unmount
 _dmthin_init $devsize $devsize $csize $lowspace
 _log_writes_init $DMTHIN_VOL_DEV
 _log_writes_mkfs >> $seqres.full 2>&1

[PATCH 20/23] fix _require_scratch_duperemove ordering

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Zorro complained that generic/559 stopped working, and I noticed that
the duperemove invocation in the _require_scratch_duperemove function
would fail with:

 Error 2: No such file or directory while getting path to file /opt/file1. Skipping.
 Error 2: No such file or directory while getting path to file /opt/file2. Skipping.
 No dedupe candidates found.
 Gathering file list...

The cause of this is the incorrect placement of _require_scratch_dedupe
after a _scratch_mount.  _require_scratch_dedupe formats, mounts, tests,
and unmounts the scratch filesystem, which means that it should not come
between a _scratch_mount call and code that uses $SCRATCH_MNT.

Cc: <fstests@vger.kernel.org> # v2024.12.22
Fixes: 3b9f5fc7d7d853 ("common: call _require_scratch_dedupe from _require_scratch_duperemove")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/reflink |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)


diff --git a/common/reflink b/common/reflink
index 9177c45e70bb37..757f06c1c69fa7 100644
--- a/common/reflink
+++ b/common/reflink
@@ -80,11 +80,11 @@ _require_scratch_duperemove()
 {
 	_require_scratch
 	_require_command "$DUPEREMOVE_PROG" duperemove
-
-	_scratch_mkfs > /dev/null
-	_scratch_mount
 	_require_scratch_dedupe
 
+	_scratch_mkfs > /dev/null
+	_scratch_mount
+
 	dd if=/dev/zero of="$SCRATCH_MNT/file1" bs=128k count=1 >& /dev/null
 	dd if=/dev/zero of="$SCRATCH_MNT/file2" bs=128k count=1 >& /dev/null
 	if ! "$DUPEREMOVE_PROG" -d "$SCRATCH_MNT/file1" \

[PATCH 21/23] fsstress: fix a memory leak

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Someone forgot to free the iovec that readv_f allocates.

Fixes: 80499d8f5f251e ("fsstress: new writev and readv operations test")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 ltp/fsstress.c |    1 +
 1 file changed, 1 insertion(+)


diff --git a/ltp/fsstress.c b/ltp/fsstress.c
index 9e8eaa6d8656da..14c29921e8b0f8 100644
--- a/ltp/fsstress.c
+++ b/ltp/fsstress.c
@@ -4652,6 +4652,7 @@ readv_f(opnum_t opno, long r)
 	}
 
 	e = readv(fd, iov, iovcnt) < 0 ? errno : 0;
+	free(iov);
 	free(buf);
 	if (v)
 		printf("%d/%lld: readv %s%s [%lld,%d,%d] %d\n",

[PATCH 22/23] fsx: fix leaked log file pointer

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix a resource leaks in fsx, where we fail to close the fsx logfile,
because the C library could have some buffered contents that aren't
flushed when the program terminates.  glibc seems to do this for us, but
I wouldn't be so sure about the others.

Fixes: 3f742550dfed84 ("fsx: add support for recording operations to a file")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 ltp/fsx.c |    1 +
 1 file changed, 1 insertion(+)


diff --git a/ltp/fsx.c b/ltp/fsx.c
index 9efd2f5c86d11c..2c19fff880330a 100644
--- a/ltp/fsx.c
+++ b/ltp/fsx.c
@@ -3345,6 +3345,7 @@ main(int argc, char **argv)
 	if (recordops)
 		logdump();
 
+	fclose(fsxlogf);
 	exit(0);
 	return 0;
 }

[PATCH 23/23] build: initialize stack variables to zero by default

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Newer versions of gcc and clang can include the ability to zero stack
variables by default.  Let's enable it so that we (a) reduce the risk of
writing stack contents to disk somewhere and (b) try to reduce
unpredictable program behavior based on random stack contents.  The
kernel added this 6 years ago, so I think it's mature enough for
fstests.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 configure.ac          |    1 +
 include/builddefs.in  |    3 ++-
 m4/package_libcdev.m4 |   14 ++++++++++++++
 3 files changed, 17 insertions(+), 1 deletion(-)


diff --git a/configure.ac b/configure.ac
index c81411e735a90d..f3c8c643f0eb9e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -72,6 +72,7 @@ AC_HAVE_NFTW
 AC_HAVE_RLIMIT_NOFILE
 AC_NEED_INTERNAL_XFS_IOC_EXCHANGE_RANGE
 AC_HAVE_FICLONE
+AC_HAVE_TRIVIAL_AUTO_VAR_INIT
 
 AC_CHECK_FUNCS([renameat2])
 AC_CHECK_FUNCS([reallocarray])
diff --git a/include/builddefs.in b/include/builddefs.in
index 7274cde8d0814c..5b5864278682a4 100644
--- a/include/builddefs.in
+++ b/include/builddefs.in
@@ -76,6 +76,7 @@ NEED_INTERNAL_XFS_IOC_EXCHANGE_RANGE = @need_internal_xfs_ioc_exchange_range@
 HAVE_FICLONE = @have_ficlone@
 
 GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall
+SANITIZER_CFLAGS += @autovar_init_cflags@
 
 ifeq ($(PKG_PLATFORM),linux)
 PCFLAGS = -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 $(GCCFLAGS)
@@ -90,7 +91,7 @@ GCFLAGS = $(OPTIMIZER) $(DEBUG) $(CPPFLAGS) \
 	-I$(TOPDIR)/include -DVERSION=\"$(PKG_VERSION)\"
 
 # Global, Platform, Local CFLAGS
-CFLAGS += $(GCFLAGS) $(PCFLAGS) $(LCFLAGS)
+CFLAGS += $(GCFLAGS) $(PCFLAGS) $(LCFLAGS) $(SANITIZER_CFLAGS)
 
 include $(TOPDIR)/include/buildmacros
 
diff --git a/m4/package_libcdev.m4 b/m4/package_libcdev.m4
index a0d50f4d9b68e4..ed8fe6e32ae00a 100644
--- a/m4/package_libcdev.m4
+++ b/m4/package_libcdev.m4
@@ -72,3 +72,17 @@ AC_DEFUN([AC_HAVE_FICLONE],
        AC_MSG_RESULT(yes)],[AC_MSG_RESULT(no)])
     AC_SUBST(have_ficlone)
   ])
+
+# Check if we have -ftrivial-auto-var-init=zero
+AC_DEFUN([AC_HAVE_TRIVIAL_AUTO_VAR_INIT],
+  [ AC_MSG_CHECKING([if C compiler supports zeroing automatic vars])
+    OLD_CFLAGS="$CFLAGS"
+    TEST_CFLAGS="-ftrivial-auto-var-init=zero"
+    CFLAGS="$CFLAGS $TEST_CFLAGS"
+    AC_LINK_IFELSE([AC_LANG_PROGRAM([])],
+        [AC_MSG_RESULT([yes])]
+        [autovar_init_cflags=$TEST_CFLAGS],
+        [AC_MSG_RESULT([no])])
+    CFLAGS="${OLD_CFLAGS}"
+    AC_SUBST(autovar_init_cflags)
+  ])

[PATCH 1/3] logwrites: warn if we don't think read after discard returns zeroes

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The logwrites replay program expects that it can issue a DISCARD against
the block device passed to _log_writes_init and that will cause all
subsequent reads to return zeroes.  This is required for correct log
recovery on filesystems such as XFS that skip recovering buffers if
newer ones are found on disk.

Unfortunately, there's no way to discover if a device's discard
implementation actually guarantees zeroes.  There used to be a sysfs
knob keyed to an allowlist, but it is now hardwired to return 0.  So
either we need a magic device that does discard-and-zero, or we need to
do the zeroing ourselves.  The logwrites program does its own zeroing if
there is no discard support, and some tests do their own zeroing.

The only devices we know to work reliably are the software defined ones
that are provided by the kernel itself -- which means dm-thinp.  Warn if
we have a device that supports discard that isn't thinp and the test
fails.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/dmlogwrites |   31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)


diff --git a/common/dmlogwrites b/common/dmlogwrites
index a27e1966a933a6..96101d53c38b4a 100644
--- a/common/dmlogwrites
+++ b/common/dmlogwrites
@@ -59,6 +59,35 @@ _require_log_writes_dax_mountopt()
 	fi
 }
 
+_log_writes_check_bdev()
+{
+	local sysfs="/sys/block/$(_short_dev $1)"
+
+	# Some filesystems (e.g. XFS) optimize log recovery by assuming that
+	# they can elide replay of metadata blocks if the block has a higher
+	# log serial number than the transaction being recovered.  This is a
+	# problem if the filesystem log contents can go back in time, which is
+	# what the logwrites replay program does.
+	#
+	# The logwrites replay program begins by erasing the block device's
+	# contents.  This can be done very quickly with DISCARD provided the
+	# device guarantees that all reads after a DISCARD return zeroes, or
+	# very slowly by writing zeroes to the device.  Fast is preferable, but
+	# there's no longer any way to detect that DISCARD actually unmaps
+	# zeroes, so warn the user about this requirement if the test happens
+	# to fail.
+
+	# No discard support means the logwrites will do its own zeroing
+	test "$(cat "$sysfs/queue/discard_max_bytes")" -eq 0 && return
+
+	# dm-thinp guarantees that reads after discards return zeroes
+	dmsetup status "$blkdev" 2>/dev/null | grep -q '^0.* thin ' && return
+
+	echo "HINT: $blkdev doesn't guarantee that reads after DISCARD will return zeroes" >> $seqres.hints
+	echo "      This is required for correct journal replay on some filesystems (e.g. xfs)" >> $seqres.hints
+	echo >> $seqres.hints
+}
+
 # Set up a dm-log-writes device
 #
 # blkdev: the specified target device
@@ -84,6 +113,8 @@ _log_writes_init()
 	LOGWRITES_NAME=logwrites-test
 	LOGWRITES_DMDEV=/dev/mapper/$LOGWRITES_NAME
 	LOGWRITES_TABLE="0 $BLK_DEV_SIZE log-writes $blkdev $LOGWRITES_DEV"
+
+	_log_writes_check_bdev "$blkdev"
 	_dmsetup_create $LOGWRITES_NAME --table "$LOGWRITES_TABLE" || \
 		_fail "failed to create log-writes device"
 }

[PATCH 2/3] logwrites: use BLKZEROOUT if it's available

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Use the BLKZEROOUT ioctl instead of writing zeroed buffers if the kernel
supports it.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 src/log-writes/log-writes.c |   10 ++++++++++
 src/log-writes/log-writes.h |    1 +
 2 files changed, 11 insertions(+)


diff --git a/src/log-writes/log-writes.c b/src/log-writes/log-writes.c
index aa53473974d9e8..8f94ae5629e085 100644
--- a/src/log-writes/log-writes.c
+++ b/src/log-writes/log-writes.c
@@ -42,6 +42,7 @@ static int discard_range(struct log *log, u64 start, u64 len)
 
 static int zero_range(struct log *log, u64 start, u64 len)
 {
+	u64 range[2] = { start, len };
 	u64 bufsize = len;
 	ssize_t ret;
 	char *buf = NULL;
@@ -54,6 +55,15 @@ static int zero_range(struct log *log, u64 start, u64 len)
 		return 0;
 	}
 
+	if (!(log->flags & LOG_ZEROOUT_NOT_SUPP)) {
+		if (ioctl(log->replayfd, BLKZEROOUT, &range) < 0) {
+			if (log_writes_verbose)
+				printf(
+ "replay device doesn't support zeroout, switching to writing zeros\n");
+			log->flags |= LOG_ZEROOUT_NOT_SUPP;
+		}
+	}
+
 	while (!buf) {
 		buf = malloc(bufsize);
 		if (!buf)
diff --git a/src/log-writes/log-writes.h b/src/log-writes/log-writes.h
index b9f571ac3b2384..f659931634e64a 100644
--- a/src/log-writes/log-writes.h
+++ b/src/log-writes/log-writes.h
@@ -63,6 +63,7 @@ struct log_write_entry {
 
 #define LOG_IGNORE_DISCARD (1 << 0)
 #define LOG_DISCARD_NOT_SUPP (1 << 1)
+#define LOG_ZEROOUT_NOT_SUPP (1 << 2)
 
 struct log {
 	int logfd;

[PATCH 3/3] logwrites: only use BLKDISCARD if we know discard zeroes data

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Building off the checks established in the previous patch, only enable
the use of BLKDISCARD if we know that the logwrites device guarantees
that reads after a discard return zeroes.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/dmlogwrites          |   10 ++++++++--
 src/log-writes/replay-log.c |    8 ++++++++
 2 files changed, 16 insertions(+), 2 deletions(-)


diff --git a/common/dmlogwrites b/common/dmlogwrites
index 96101d53c38b4a..fbc8beb5ce597e 100644
--- a/common/dmlogwrites
+++ b/common/dmlogwrites
@@ -81,7 +81,10 @@ _log_writes_check_bdev()
 	test "$(cat "$sysfs/queue/discard_max_bytes")" -eq 0 && return
 
 	# dm-thinp guarantees that reads after discards return zeroes
-	dmsetup status "$blkdev" 2>/dev/null | grep -q '^0.* thin ' && return
+	if dmsetup status "$blkdev" 2>/dev/null | grep -q '^0.* thin '; then
+		LOGWRITES_REPLAY_ARGS+=(--discard-zeroes-data)
+		return
+	fi
 
 	echo "HINT: $blkdev doesn't guarantee that reads after DISCARD will return zeroes" >> $seqres.hints
 	echo "      This is required for correct journal replay on some filesystems (e.g. xfs)" >> $seqres.hints
@@ -110,6 +113,7 @@ _log_writes_init()
 		BLK_DEV_SIZE=$((length / blksz))
 	fi
 
+	LOGWRITES_REPLAY_ARGS=()
 	LOGWRITES_NAME=logwrites-test
 	LOGWRITES_DMDEV=/dev/mapper/$LOGWRITES_NAME
 	LOGWRITES_TABLE="0 $BLK_DEV_SIZE log-writes $blkdev $LOGWRITES_DEV"
@@ -161,7 +165,8 @@ _log_writes_replay_log()
 	[ $? -ne 0 ] && _fail "mark '$_mark' does not exist"
 
 	$here/src/log-writes/replay-log --log $LOGWRITES_DEV --replay $_blkdev \
-		--end-mark $_mark >> $seqres.full 2>&1
+		--end-mark $_mark "${LOGWRITES_REPLAY_ARGS[@]}" \
+		>> $seqres.full 2>&1
 	[ $? -ne 0 ] && _fail "replay failed"
 }
 
@@ -231,6 +236,7 @@ _log_writes_replay_log_range()
 	echo "=== replay to $end ===" >> $seqres.full
 	$here/src/log-writes/replay-log -vv --log $LOGWRITES_DEV \
 		--replay $blkdev --limit $(($end + 1)) \
+		"${LOGWRITES_REPLAY_ARGS[@]}" \
 		>> $seqres.full 2>&1
 	[ $? -ne 0 ] && _fail "replay failed"
 }
diff --git a/src/log-writes/replay-log.c b/src/log-writes/replay-log.c
index 968c82ab64a9ad..e07401f63af573 100644
--- a/src/log-writes/replay-log.c
+++ b/src/log-writes/replay-log.c
@@ -18,6 +18,7 @@ enum option_indexes {
 	FIND,
 	NUM_ENTRIES,
 	NO_DISCARD,
+	DISCARD_ZEROES_DATA,
 	FSCK,
 	CHECK,
 	START_MARK,
@@ -37,6 +38,7 @@ static struct option long_options[] = {
 	{"find", no_argument, NULL, 0},
 	{"num-entries", no_argument, NULL, 0},
 	{"no-discard", no_argument, NULL, 0},
+	{"discard-zeroes-data", no_argument, NULL, 0},
 	{"fsck", required_argument, NULL, 0},
 	{"check", required_argument, NULL, 0},
 	{"start-mark", required_argument, NULL, 0},
@@ -155,6 +157,7 @@ int main(int argc, char **argv)
 	int ret;
 	int print_num_entries = 0;
 	int discard = 1;
+	int use_kernel_discard = 0;
 	enum log_replay_check_mode check_mode = 0;
 
 	while ((c = getopt_long(argc, argv, "v", long_options,
@@ -242,6 +245,9 @@ int main(int argc, char **argv)
 		case NO_DISCARD:
 			discard = 0;
 			break;
+		case DISCARD_ZEROES_DATA:
+			use_kernel_discard = 1;
+			break;
 		case FSCK:
 			fsck_command = strdup(optarg);
 			if (!fsck_command) {
@@ -299,6 +305,8 @@ int main(int argc, char **argv)
 
 	if (!discard)
 		log->flags |= LOG_IGNORE_DISCARD;
+	if (!use_kernel_discard)
+		log->flags |= LOG_DISCARD_NOT_SUPP;
 
 	log->start_sector = start_sector;
 	log->end_sector = end_sector;

[PATCH 01/11] various: fix finding metadata inode numbers when metadir is enabled

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

There are a number of tests that use xfs_db to examine the contents of
metadata inodes to check correct functioning.  The logic is scattered
everywhere and won't work with metadata directory trees, so make a
shared helper to find these inodes.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/xfs    |   21 +++++++++++++++++++--
 tests/xfs/007 |   16 +++++++++-------
 tests/xfs/529 |    5 ++---
 tests/xfs/530 |    6 ++----
 tests/xfs/739 |    9 ++-------
 tests/xfs/740 |    9 ++-------
 tests/xfs/741 |    9 ++-------
 tests/xfs/742 |    9 ++-------
 tests/xfs/743 |    9 ++-------
 tests/xfs/744 |    9 ++-------
 tests/xfs/745 |    9 ++-------
 tests/xfs/746 |    9 ++-------
 12 files changed, 48 insertions(+), 72 deletions(-)


diff --git a/common/xfs b/common/xfs
index c68bd6d7c773ac..092b3dc6f3bdc5 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1406,7 +1406,7 @@ _scratch_get_bmx_prefix() {
 
 _scratch_get_iext_count()
 {
-	local ino=$1
+	local selector=$1
 	local whichfork=$2
 	local field=""
 
@@ -1421,7 +1421,7 @@ _scratch_get_iext_count()
 			return 1
 	esac
 
-	_scratch_xfs_get_metadata_field $field "inode $ino"
+	_scratch_xfs_get_metadata_field $field "$selector"
 }
 
 #
@@ -1891,3 +1891,20 @@ _wipe_xfs_properties()
 		setfattr --remove="$name" "$1"
 	done
 }
+
+# Return the xfs_db selector for a superblock-rooted metadata file on the
+# scratch filesystem.  The sole argument is the name of the field within the
+# superblock.  This helper cannot be used to find files under the metadata
+# directory tree.
+_scratch_xfs_find_metafile()
+{
+	local metafile="$1"
+	local sb_field
+
+	sb_field="$(_scratch_xfs_get_sb_field "$metafile")"
+	if echo "$sb_field" | grep -q -w 'not found'; then
+		return 1
+	fi
+	echo "inode $sb_field"
+	return 0
+}
diff --git a/tests/xfs/007 b/tests/xfs/007
index 2535f04cac36a5..e35a069f9bd5c5 100755
--- a/tests/xfs/007
+++ b/tests/xfs/007
@@ -21,6 +21,11 @@ _require_xfs_quota
 _scratch_mkfs_xfs | _filter_mkfs > /dev/null 2> $tmp.mkfs
 . $tmp.mkfs
 
+get_qfile_nblocks() {
+	local selector="$(_scratch_xfs_find_metafile "$1")"
+	_scratch_xfs_db -c "$selector" -c "p core.nblocks"
+}
+
 do_test()
 {
 	qino_1=$1
@@ -30,12 +35,9 @@ do_test()
 	echo "*** umount"
 	_scratch_unmount
 
-	QINO_1=`_scratch_xfs_get_sb_field $qino_1`
-	QINO_2=`_scratch_xfs_get_sb_field $qino_2`
-
 	echo "*** Usage before quotarm ***"
-	_scratch_xfs_db -c "inode $QINO_1" -c "p core.nblocks"
-	_scratch_xfs_db -c "inode $QINO_2" -c "p core.nblocks"
+	get_qfile_nblocks $qino_1
+	get_qfile_nblocks $qino_2
 
 	_qmount
 	echo "*** turn off $off_opts quotas"
@@ -65,8 +67,8 @@ do_test()
 	_scratch_unmount
 
 	echo "*** Usage after quotarm ***"
-	_scratch_xfs_db -c "inode $QINO_1" -c "p core.nblocks"
-	_scratch_xfs_db -c "inode $QINO_2" -c "p core.nblocks"
+	get_qfile_nblocks $qino_1
+	get_qfile_nblocks $qino_2
 }
 
 # Test user and group first
diff --git a/tests/xfs/529 b/tests/xfs/529
index 14bdd2eebf7047..aab8668c76f46c 100755
--- a/tests/xfs/529
+++ b/tests/xfs/529
@@ -161,9 +161,8 @@ done
 _scratch_unmount >> $seqres.full
 
 echo "Verify uquota inode's extent count"
-uquotino=$(_scratch_xfs_get_metadata_field 'uquotino' 'sb 0')
-
-nextents=$(_scratch_get_iext_count $uquotino data || \
+selector="$(_scratch_xfs_find_metafile uquotino)"
+nextents=$(_scratch_get_iext_count "$selector" data || \
 		   _fail "Unable to obtain inode fork's extent count")
 if (( $nextents > 10 )); then
 	echo "Extent count overflow check failed: nextents = $nextents"
diff --git a/tests/xfs/530 b/tests/xfs/530
index 95ab32f1e1f828..4a41127e3b820f 100755
--- a/tests/xfs/530
+++ b/tests/xfs/530
@@ -102,10 +102,8 @@ _scratch_unmount >> $seqres.full
 
 echo "Verify rbmino's and rsumino's extent count"
 for rtino in rbmino rsumino; do
-	ino=$(_scratch_xfs_get_metadata_field $rtino "sb 0")
-	echo "$rtino = $ino" >> $seqres.full
-
-	nextents=$(_scratch_get_iext_count $ino data || \
+	selector="$(_scratch_xfs_find_metafile "$rtino")"
+	nextents=$(_scratch_get_iext_count "$selector" data || \
 			_fail "Unable to obtain inode fork's extent count")
 	if (( $nextents > 10 )); then
 		echo "Extent count overflow check failed: nextents = $nextents"
diff --git a/tests/xfs/739 b/tests/xfs/739
index 52c90b91b218df..5fd6caa5bce2f8 100755
--- a/tests/xfs/739
+++ b/tests/xfs/739
@@ -25,13 +25,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.bitmap')
-else
-	path=('sb' 'addr rbmino')
-fi
-_scratch_xfs_fuzz_metadata '' 'online' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rbmino)"
+_scratch_xfs_fuzz_metadata '' 'online' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
 # success, all done
diff --git a/tests/xfs/740 b/tests/xfs/740
index fb616a16362a6c..c8990034773b32 100755
--- a/tests/xfs/740
+++ b/tests/xfs/740
@@ -25,13 +25,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.summary')
-else
-	path=('sb' 'addr rsumino')
-fi
-_scratch_xfs_fuzz_metadata '' 'online' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rsumino)"
+_scratch_xfs_fuzz_metadata '' 'online' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
 # success, all done
diff --git a/tests/xfs/741 b/tests/xfs/741
index ea1c2d98516fdb..96c2315c524311 100755
--- a/tests/xfs/741
+++ b/tests/xfs/741
@@ -25,13 +25,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.bitmap')
-else
-	path=('sb' 'addr rbmino')
-fi
-_scratch_xfs_fuzz_metadata '' 'offline' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rbmino)"
+_scratch_xfs_fuzz_metadata '' 'offline' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
 # success, all done
diff --git a/tests/xfs/742 b/tests/xfs/742
index bbc186f8472b64..301ae7b9574320 100755
--- a/tests/xfs/742
+++ b/tests/xfs/742
@@ -25,13 +25,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.summary')
-else
-	path=('sb' 'addr rsumino')
-fi
-_scratch_xfs_fuzz_metadata '' 'offline' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rsumino)"
+_scratch_xfs_fuzz_metadata '' 'offline' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
 # success, all done
diff --git a/tests/xfs/743 b/tests/xfs/743
index 8021d7ecdc9009..039624f711c0a6 100755
--- a/tests/xfs/743
+++ b/tests/xfs/743
@@ -26,13 +26,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.bitmap')
-else
-	path=('sb' 'addr rbmino')
-fi
-_scratch_xfs_fuzz_metadata '' 'both' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rbmino)"
+_scratch_xfs_fuzz_metadata '' 'both' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
 # success, all done
diff --git a/tests/xfs/744 b/tests/xfs/744
index 0e719dcebf03d2..13f63b9ceb1756 100755
--- a/tests/xfs/744
+++ b/tests/xfs/744
@@ -26,13 +26,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.summary')
-else
-	path=('sb' 'addr rsumino')
-fi
-_scratch_xfs_fuzz_metadata '' 'both' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rsumino)"
+_scratch_xfs_fuzz_metadata '' 'both' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
 # success, all done
diff --git a/tests/xfs/745 b/tests/xfs/745
index 3549ad08772c2c..56a6d58ef9f4ca 100755
--- a/tests/xfs/745
+++ b/tests/xfs/745
@@ -25,13 +25,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.bitmap')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.bitmap')
-else
-	path=('sb' 'addr rbmino')
-fi
-_scratch_xfs_fuzz_metadata '' 'none' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rbmino)"
+_scratch_xfs_fuzz_metadata '' 'none' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
 # success, all done
diff --git a/tests/xfs/746 b/tests/xfs/746
index 5afc71ad19c6e6..935b2e5acba5d4 100755
--- a/tests/xfs/746
+++ b/tests/xfs/746
@@ -25,13 +25,8 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-is_metadir=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /realtime/0.summary')
-if [ -n "$is_metadir" ]; then
-	path=('path -m /realtime/0.summary')
-else
-	path=('sb' 'addr rsumino')
-fi
-_scratch_xfs_fuzz_metadata '' 'none' "${path[@]}" 'dblock 0' >> $seqres.full
+path="$(_scratch_xfs_find_metafile rsumino)"
+_scratch_xfs_fuzz_metadata '' 'none' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
 # success, all done

[PATCH 02/11] xfs/{030,033,178}: forcibly disable metadata directory trees

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The golden output for thests tests encode the xfs_repair output when we
fuzz various parts of the filesystem.  With metadata directory trees
enabled, however, the golden output changes dramatically to reflect
reconstruction of the metadata directory tree.

To avoid regressions, add a helper to force metadata directories off via
MKFS_OPTIONS.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/xfs    |   25 +++++++++++++++++++++++++
 tests/xfs/030 |    1 +
 tests/xfs/033 |    1 +
 tests/xfs/178 |    1 +
 4 files changed, 28 insertions(+)


diff --git a/common/xfs b/common/xfs
index 092b3dc6f3bdc5..ee7fe7b92a4950 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1908,3 +1908,28 @@ _scratch_xfs_find_metafile()
 	echo "inode $sb_field"
 	return 0
 }
+
+# Force metadata directories off.
+_scratch_xfs_force_no_metadir()
+{
+	# Remove any mkfs-time quota options because those are only supported
+	# with metadir=1
+	for opt in uquota gquota pquota; do
+		echo "$MKFS_OPTIONS" | grep -q -w "$opt" || continue
+
+		MKFS_OPTIONS="$(echo "$MKFS_OPTIONS" | sed -e "s/,$opt//g" -e "s/ $opt/ /g")"
+		MOUNT_OPTIONS="$MOUNT_OPTIONS -o $opt"
+	done
+
+	# Replace any explicit metadir option with metadir=0
+	if echo "$MKFS_OPTIONS" | grep -q 'metadir='; then
+		MKFS_OPTIONS="$(echo "$MKFS_OPTIONS" | sed -e 's/metadir=[0-9]*/metadir=0/g' -e 's/metadir\([, ]\)/metadir=0\1/g')"
+		return
+	fi
+
+	# Inject metadir=0 if there isn't one in MKFS_OPTIONS and mkfs supports
+	# that option.
+	if grep -q 'metadir=' $MKFS_XFS_PROG; then
+		MKFS_OPTIONS="-m metadir=0 $MKFS_OPTIONS"
+	fi
+}
diff --git a/tests/xfs/030 b/tests/xfs/030
index 7ce5ffce38693c..22fbdb2fdbc999 100755
--- a/tests/xfs/030
+++ b/tests/xfs/030
@@ -48,6 +48,7 @@ _check_ag()
 
 _require_scratch
 _require_no_large_scratch_dev
+_scratch_xfs_force_no_metadir
 
 DSIZE="-dsize=100m,agcount=6"
 
diff --git a/tests/xfs/033 b/tests/xfs/033
index d7b02a9c51b3f0..e0b0dd58212d61 100755
--- a/tests/xfs/033
+++ b/tests/xfs/033
@@ -51,6 +51,7 @@ _filter_bad_ids()
 
 _require_scratch
 _require_no_large_scratch_dev
+_scratch_xfs_force_no_metadir
 
 # devzero blows away 512byte blocks, so make 512byte inodes (at least)
 _scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs >/dev/null
diff --git a/tests/xfs/178 b/tests/xfs/178
index a22e626706ec49..0cc0e3f5bb88b4 100755
--- a/tests/xfs/178
+++ b/tests/xfs/178
@@ -50,6 +50,7 @@ _dd_repair_check()
 #             fix filesystem, new mkfs.xfs will be fine.
 
 _require_scratch
+_scratch_xfs_force_no_metadir
 _scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
 
 # By executing the followint tmp file, will get on the mkfs options stored in

[PATCH 03/11] common/repair: patch up repair sb inode value complaints

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Now that we've refactored xfs_repair to be more consistent in how it
reports unexpected superblock inode pointer values, we have to fix up
the fstests repair filters to emulate the old golden output.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/repair |    4 ++++
 1 file changed, 4 insertions(+)


diff --git a/common/repair b/common/repair
index 0dae830520f1e9..a79f9b2b3571ce 100644
--- a/common/repair
+++ b/common/repair
@@ -28,6 +28,10 @@ _filter_repair()
 	perl -ne '
 # for sb
 /- agno = / && next;	# remove each AG line (variable number)
+s/realtime bitmap inode pointer/realtime bitmap ino pointer/;
+s/sb realtime bitmap inode value/sb realtime bitmap inode/;
+s/realtime summary inode pointer/realtime summary ino pointer/;
+s/sb realtime summary inode value/sb realtime summary inode/;
 s/(pointer to) (\d+)/\1 INO/;
 # Changed inode output in 5.5.0
 s/sb root inode value /sb root inode /;

[PATCH 04/11] xfs/206: update for metadata directory support

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Filter 'metadir=' out of the golden output so that metadata directories
don't cause this test to regress.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/206 |    1 +
 1 file changed, 1 insertion(+)


diff --git a/tests/xfs/206 b/tests/xfs/206
index 1297433188e868..ef5f4868e9bdca 100755
--- a/tests/xfs/206
+++ b/tests/xfs/206
@@ -63,6 +63,7 @@ mkfs_filter()
 	    -e "s/, lazy-count=[0-9]//" \
 	    -e "/.*crc=/d" \
 	    -e "/exchange=/d" \
+	    -e '/metadir=.*/d' \
 	    -e 's/, parent=[01]//' \
 	    -e "/^Default configuration/d"
 }

[PATCH 05/11] xfs/{050,144,153,299,330}: update quota reports to handle metadir trees

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Prior to the new metadir feature in XFS, the rtbitmap and rtsummary
files were included in icount, though their bcount contribution is zero
due to rt and quota not being supported together.  With the new metadir
feature in XFS, no files in the metadata directory tree are counted in
quota.

Hence we must adjust the icount of any quota report down by two to avoid
breaking golden outputs.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/filter |    7 +++++--
 common/xfs    |   29 +++++++++++++++++++++++++++++
 tests/xfs/050 |    5 +++++
 tests/xfs/153 |    5 +++++
 tests/xfs/299 |    1 +
 tests/xfs/330 |    6 +++++-
 6 files changed, 50 insertions(+), 3 deletions(-)


diff --git a/common/filter b/common/filter
index 7e02ded377cc9b..1ebfd27e898e01 100644
--- a/common/filter
+++ b/common/filter
@@ -624,11 +624,14 @@ _filter_getcap()
 
 # Filter user/group/project id numbers out of quota reports, and standardize
 # the block counts to use filesystem block size.  Callers must set the id and
-# bsize variables before calling this function.
+# bsize variables before calling this function.  The HIDDEN_QUOTA_FILES variable
+# (by default zero) is the number of root files to filter out of the inode
+# count part of the quota report.
 _filter_quota_report()
 {
 	test -n "$id" || echo "id must be set"
 	test -n "$bsize" || echo "block size must be set"
+	test -n "$HIDDEN_QUOTA_FILES" || HIDDEN_QUOTA_FILES=0
 
 	tr -s '[:space:]' | \
 	perl -npe '
@@ -636,7 +639,7 @@ _filter_quota_report()
 		s/^\#0 \d+ /[ROOT] 0 /g;
 		s/6 days/7 days/g' |
 	perl -npe '
-		$val = 0;
+		$val = '"$HIDDEN_QUOTA_FILES"';
 		if ($ENV{'LARGE_SCRATCH_DEV'}) {
 			$val = $ENV{'NUM_SPACE_FILES'};
 		}
diff --git a/common/xfs b/common/xfs
index ee7fe7b92a4950..a16ec56df0f665 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1933,3 +1933,32 @@ _scratch_xfs_force_no_metadir()
 		MKFS_OPTIONS="-m metadir=0 $MKFS_OPTIONS"
 	fi
 }
+
+# Decide if a mount filesystem has metadata directory trees.
+_xfs_mount_has_metadir() {
+	local mount="$1"
+
+	# spaceman (and its info command) predate metadir
+	test ! -e "$XFS_SPACEMAN_PROG" && return 1
+	$XFS_SPACEMAN_PROG -c "info" "$mount" | grep -q 'metadir=1'
+}
+
+# Compute the number of files that are not counted in quotas.
+_xfs_calc_hidden_quota_files() {
+	local mount="$1"
+
+	if _xfs_mount_has_metadir "$mount"; then
+		# Prior to the metadir feature, the realtime bitmap and summary
+		# file were "owned" by root and hence accounted to the root
+		# dquots.  The metadata directory feature stopped accounting
+		# metadata files to quotas, so we must subtract 2 inodes from
+		# the repquota golden outputs to keep the tests going.
+		#
+		# We needn't adjust the block counts because the kernel doesn't
+		# support rt quota and hence the rt metadata files will always
+		# be zero length.
+		echo -2
+	else
+		echo 0
+	fi
+}
diff --git a/tests/xfs/050 b/tests/xfs/050
index 78303bf784d05e..1e40ab90a843e8 100755
--- a/tests/xfs/050
+++ b/tests/xfs/050
@@ -30,9 +30,14 @@ _require_scratch
 _require_xfs_quota
 
 _scratch_mkfs >/dev/null 2>&1
+orig_mntopts="$MOUNT_OPTIONS"
+_qmount_option "uquota"
 _scratch_mount
 bsize=$(_get_file_block_size $SCRATCH_MNT)
+# needs quota enabled to compute the number of metadata dir files
+HIDDEN_QUOTA_FILES=$(_xfs_calc_hidden_quota_files $SCRATCH_MNT)
 _scratch_unmount
+MOUNT_OPTIONS="$orig_mntopts"
 
 bsoft=$(( 200 * $bsize ))
 bhard=$(( 1000 * $bsize ))
diff --git a/tests/xfs/153 b/tests/xfs/153
index d5e43082c1cd35..2ce22b8c44b298 100755
--- a/tests/xfs/153
+++ b/tests/xfs/153
@@ -35,9 +35,14 @@ _require_idmapped_mounts
 _require_test_program "vfs/mount-idmapped"
 
 _scratch_mkfs >/dev/null 2>&1
+orig_mntopts="$MOUNT_OPTIONS"
+_qmount_option "uquota"
 _scratch_mount
 bsize=$(_get_file_block_size $SCRATCH_MNT)
+# needs quota enabled to compute the number of metadata dir files
+HIDDEN_QUOTA_FILES=$(_xfs_calc_hidden_quota_files $SCRATCH_MNT)
 _scratch_unmount
+MOUNT_OPTIONS="$orig_mntopts"
 
 bsoft=$(( 200 * $bsize ))
 bhard=$(( 1000 * $bsize ))
diff --git a/tests/xfs/299 b/tests/xfs/299
index 710eb89c2ac0e7..3986f8fb904e5d 100755
--- a/tests/xfs/299
+++ b/tests/xfs/299
@@ -155,6 +155,7 @@ _qmount_option "uquota,gquota,pquota"
 _qmount
 
 bsize=$(_get_file_block_size $SCRATCH_MNT)
+HIDDEN_QUOTA_FILES=$(_xfs_calc_hidden_quota_files $SCRATCH_MNT)
 
 bsoft=$(( 100 * $bsize ))
 bhard=$(( 500 * $bsize ))
diff --git a/tests/xfs/330 b/tests/xfs/330
index d239a64085c76c..30c09ff5906e12 100755
--- a/tests/xfs/330
+++ b/tests/xfs/330
@@ -24,7 +24,10 @@ _require_nobody
 
 do_repquota()
 {
-	repquota $SCRATCH_MNT | grep -E '^(fsgqa|root|nobody)' | sort -r
+	repquota $SCRATCH_MNT | grep -E '^(fsgqa|root|nobody)' | sort -r | \
+	perl -npe '
+		$val = '"$HIDDEN_QUOTA_FILES"';
+		s/(^root\s+--\s+\S+\s+\S+\s+\S+\s+)(\S+)/$1@{[$2 - $val]}/g'
 }
 
 rm -f "$seqres.full"
@@ -33,6 +36,7 @@ echo "Format and mount"
 _scratch_mkfs > "$seqres.full" 2>&1
 export MOUNT_OPTIONS="-o usrquota,grpquota $MOUNT_OPTIONS"
 _scratch_mount >> "$seqres.full" 2>&1
+HIDDEN_QUOTA_FILES=$(_xfs_calc_hidden_quota_files $SCRATCH_MNT)
 quotacheck -u -g $SCRATCH_MNT 2> /dev/null
 quotaon $SCRATCH_MNT 2> /dev/null
 

[PATCH 06/11] xfs/509: adjust inumbers accounting for metadata directories

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The INUMBERS ioctl exports data from the inode btree directly -- the
number of inodes it reports is taken from ir_freemask and includes all
the files in the metadata directory tree.  BULKSTAT, on the other hand,
only reports non-metadata files.  When metadir is enabled, this will
(eventually) cause a discrepancy in the inode counts that is large
enough to exceed the tolerances, thereby causing a test failure.

Correct this by counting the files in the metadata directory and
subtracting that from the INUMBERS totals.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/509 |   23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)


diff --git a/tests/xfs/509 b/tests/xfs/509
index 53c6bd9c0772a1..9b07fecc5d1a10 100755
--- a/tests/xfs/509
+++ b/tests/xfs/509
@@ -91,13 +91,13 @@ inumbers_count()
 	bstat_versions | while read v_tag v_flag; do
 		echo -n "inumbers all($v_tag): "
 		nr=$(inumbers_fs $SCRATCH_MNT $v_flag)
-		_within_tolerance "inumbers" $nr $expect $tolerance -v
+		_within_tolerance "inumbers" $((nr - METADATA_FILES)) $expect $tolerance -v
 
 		local agcount=$(_xfs_mount_agcount $SCRATCH_MNT)
 		for batchsize in 71 2 1; do
 			echo -n "inumbers $batchsize($v_tag): "
 			nr=$(inumbers_ag $agcount $batchsize $SCRATCH_MNT $v_flag)
-			_within_tolerance "inumbers" $nr $expect $tolerance -v
+			_within_tolerance "inumbers" $((nr - METADATA_FILES)) $expect $tolerance -v
 		done
 	done
 }
@@ -142,9 +142,28 @@ _require_xfs_io_command inumbers
 DIRCOUNT=8
 INOCOUNT=$((2048 / DIRCOUNT))
 
+# Count everything in the metadata directory tree.
+count_metadir_files() {
+	# Each possible path in the metadata directory tree must be listed
+	# here.
+	local metadirs=('/rtgroups')
+	local db_args=('-f')
+
+	for m in "${metadirs[@]}"; do
+		db_args+=('-c' "ls -m $m")
+	done
+
+	local ret=$(_scratch_xfs_db "${db_args[@]}" 2>/dev/null | grep regular | wc -l)
+	test -z "$ret" && ret=0
+	echo $ret
+}
+
 _scratch_mkfs "-d agcount=$DIRCOUNT" >> $seqres.full 2>&1 || _fail "mkfs failed"
 _scratch_mount
 
+METADATA_FILES=$(count_metadir_files)
+echo "found $METADATA_FILES metadata files" >> $seqres.full
+
 # Figure out if we have v5 bulkstat/inumbers ioctls.
 has_v5=
 bs_root_out="$($XFS_IO_PROG -c 'bulkstat_single root' $SCRATCH_MNT 2>>$seqres.full)"

[PATCH 07/11] xfs: create fuzz tests for metadata directories

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Create fuzz tests to make sure that all the validation works for
metadata directories and subdirectories.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/xfs         |   22 ++++++++++++++++++++++
 tests/xfs/1546     |   34 ++++++++++++++++++++++++++++++++++
 tests/xfs/1546.out |    4 ++++
 tests/xfs/1547     |   34 ++++++++++++++++++++++++++++++++++
 tests/xfs/1547.out |    4 ++++
 tests/xfs/1548     |   34 ++++++++++++++++++++++++++++++++++
 tests/xfs/1548.out |    4 ++++
 tests/xfs/1549     |   35 +++++++++++++++++++++++++++++++++++
 tests/xfs/1549.out |    4 ++++
 tests/xfs/1550     |   34 ++++++++++++++++++++++++++++++++++
 tests/xfs/1550.out |    4 ++++
 tests/xfs/1551     |   34 ++++++++++++++++++++++++++++++++++
 tests/xfs/1551.out |    4 ++++
 tests/xfs/1552     |   34 ++++++++++++++++++++++++++++++++++
 tests/xfs/1552.out |    4 ++++
 tests/xfs/1553     |   35 +++++++++++++++++++++++++++++++++++
 tests/xfs/1553.out |    4 ++++
 17 files changed, 328 insertions(+)
 create mode 100755 tests/xfs/1546
 create mode 100644 tests/xfs/1546.out
 create mode 100755 tests/xfs/1547
 create mode 100644 tests/xfs/1547.out
 create mode 100755 tests/xfs/1548
 create mode 100644 tests/xfs/1548.out
 create mode 100755 tests/xfs/1549
 create mode 100644 tests/xfs/1549.out
 create mode 100755 tests/xfs/1550
 create mode 100644 tests/xfs/1550.out
 create mode 100755 tests/xfs/1551
 create mode 100644 tests/xfs/1551.out
 create mode 100755 tests/xfs/1552
 create mode 100644 tests/xfs/1552.out
 create mode 100755 tests/xfs/1553
 create mode 100644 tests/xfs/1553.out


diff --git a/common/xfs b/common/xfs
index a16ec56df0f665..1c5daaa977aef1 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1962,3 +1962,25 @@ _xfs_calc_hidden_quota_files() {
 		echo 0
 	fi
 }
+
+_require_xfs_mkfs_metadir()
+{
+	_scratch_mkfs_xfs_supported -m metadir=1 >/dev/null 2>&1 || \
+		_notrun "mkfs.xfs doesn't have metadir features"
+}
+
+_require_xfs_scratch_metadir()
+{
+	_require_xfs_mkfs_metadir
+	_require_scratch
+
+	_scratch_mkfs -m metadir=1 &> /dev/null
+	_require_scratch_xfs_features METADIR
+	_try_scratch_mount
+	res=$?
+	if [ $res -ne 0 ]; then
+		_notrun "mounting with metadir not supported by filesystem type: $FSTYP"
+	else
+		_scratch_unmount
+	fi
+}
diff --git a/tests/xfs/1546 b/tests/xfs/1546
new file mode 100755
index 00000000000000..482ea0ef6ea6be
--- /dev/null
+++ b/tests/xfs/1546
@@ -0,0 +1,34 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1546
+#
+# Populate a XFS filesystem and fuzz every metadir root field.
+# Use xfs_scrub to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers scrub fuzzers_online_repair realtime
+
+_register_cleanup "_cleanup" BUS
+
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /')
+
+echo "Fuzz metadir root"
+_scratch_xfs_fuzz_metadata '' 'online' 'path -m /' >> $seqres.full
+echo "Done fuzzing metadir root"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1546.out b/tests/xfs/1546.out
new file mode 100644
index 00000000000000..b72891a7583c04
--- /dev/null
+++ b/tests/xfs/1546.out
@@ -0,0 +1,4 @@
+QA output created by 1546
+Format and populate
+Fuzz metadir root
+Done fuzzing metadir root
diff --git a/tests/xfs/1547 b/tests/xfs/1547
new file mode 100755
index 00000000000000..f9ad773b6c7de6
--- /dev/null
+++ b/tests/xfs/1547
@@ -0,0 +1,34 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1547
+#
+# Populate a XFS filesystem and fuzz every metadir root field.
+# Use xfs_repair to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers repair fuzzers_repair
+
+_register_cleanup "_cleanup" BUS
+
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /')
+
+echo "Fuzz metadir root"
+_scratch_xfs_fuzz_metadata '' 'offline' 'path -m /' >> $seqres.full
+echo "Done fuzzing metadir root"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1547.out b/tests/xfs/1547.out
new file mode 100644
index 00000000000000..983cc01343e5f4
--- /dev/null
+++ b/tests/xfs/1547.out
@@ -0,0 +1,4 @@
+QA output created by 1547
+Format and populate
+Fuzz metadir root
+Done fuzzing metadir root
diff --git a/tests/xfs/1548 b/tests/xfs/1548
new file mode 100755
index 00000000000000..010841b6cbf955
--- /dev/null
+++ b/tests/xfs/1548
@@ -0,0 +1,34 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1548
+#
+# Populate a XFS filesystem and fuzz every metadir root field.
+# Do not fix the filesystem, to test metadata verifiers.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers fuzzers_norepair
+
+_register_cleanup "_cleanup" BUS
+
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /')
+
+echo "Fuzz metadir root"
+_scratch_xfs_fuzz_metadata '' 'none' 'path -m /' >> $seqres.full
+echo "Done fuzzing metadir root"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1548.out b/tests/xfs/1548.out
new file mode 100644
index 00000000000000..9e395bb059436d
--- /dev/null
+++ b/tests/xfs/1548.out
@@ -0,0 +1,4 @@
+QA output created by 1548
+Format and populate
+Fuzz metadir root
+Done fuzzing metadir root
diff --git a/tests/xfs/1549 b/tests/xfs/1549
new file mode 100755
index 00000000000000..f02513a0afd225
--- /dev/null
+++ b/tests/xfs/1549
@@ -0,0 +1,35 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1549
+#
+# Populate a XFS filesystem and fuzz every metadir root field.
+# Try online repair and, if necessary, offline repair,
+# to test the most likely usage pattern.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers scrub repair fuzzers_bothrepair
+
+_register_cleanup "_cleanup" BUS
+
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /')
+
+echo "Fuzz metadir root"
+_scratch_xfs_fuzz_metadata '' 'both' 'path -m /' >> $seqres.full
+echo "Done fuzzing metadir root"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1549.out b/tests/xfs/1549.out
new file mode 100644
index 00000000000000..22b3d215e32e7b
--- /dev/null
+++ b/tests/xfs/1549.out
@@ -0,0 +1,4 @@
+QA output created by 1549
+Format and populate
+Fuzz metadir root
+Done fuzzing metadir root
diff --git a/tests/xfs/1550 b/tests/xfs/1550
new file mode 100755
index 00000000000000..cbd6c7207a36f4
--- /dev/null
+++ b/tests/xfs/1550
@@ -0,0 +1,34 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1550
+#
+# Populate a XFS filesystem and fuzz every metadir subdir field.
+# Use xfs_scrub to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers scrub fuzzers_online_repair realtime
+
+_register_cleanup "_cleanup" BUS
+
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /rtgroups')
+
+echo "Fuzz metadir subdir"
+_scratch_xfs_fuzz_metadata '' 'online' 'path -m /rtgroups' >> $seqres.full
+echo "Done fuzzing metadir subdir"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1550.out b/tests/xfs/1550.out
new file mode 100644
index 00000000000000..7694cd670bd25b
--- /dev/null
+++ b/tests/xfs/1550.out
@@ -0,0 +1,4 @@
+QA output created by 1550
+Format and populate
+Fuzz metadir subdir
+Done fuzzing metadir subdir
diff --git a/tests/xfs/1551 b/tests/xfs/1551
new file mode 100755
index 00000000000000..fbce8c9fdc2a75
--- /dev/null
+++ b/tests/xfs/1551
@@ -0,0 +1,34 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1551
+#
+# Populate a XFS filesystem and fuzz every metadir subdir field.
+# Use xfs_repair to fix the corruption.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers repair fuzzers_repair
+
+_register_cleanup "_cleanup" BUS
+
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /rtgroups')
+
+echo "Fuzz metadir subdir"
+_scratch_xfs_fuzz_metadata '' 'offline' 'path -m /rtgroups' >> $seqres.full
+echo "Done fuzzing metadir subdir"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1551.out b/tests/xfs/1551.out
new file mode 100644
index 00000000000000..4c3360d08b34f8
--- /dev/null
+++ b/tests/xfs/1551.out
@@ -0,0 +1,4 @@
+QA output created by 1551
+Format and populate
+Fuzz metadir subdir
+Done fuzzing metadir subdir
diff --git a/tests/xfs/1552 b/tests/xfs/1552
new file mode 100755
index 00000000000000..023284dfca8e2c
--- /dev/null
+++ b/tests/xfs/1552
@@ -0,0 +1,34 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1552
+#
+# Populate a XFS filesystem and fuzz every metadir subdir field.
+# Do not fix the filesystem, to test metadata verifiers.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers fuzzers_norepair
+
+_register_cleanup "_cleanup" BUS
+
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /rtgroups')
+
+echo "Fuzz metadir subdir"
+_scratch_xfs_fuzz_metadata '' 'none' 'path -m /rtgroups' >> $seqres.full
+echo "Done fuzzing metadir subdir"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1552.out b/tests/xfs/1552.out
new file mode 100644
index 00000000000000..6636b1b656c926
--- /dev/null
+++ b/tests/xfs/1552.out
@@ -0,0 +1,4 @@
+QA output created by 1552
+Format and populate
+Fuzz metadir subdir
+Done fuzzing metadir subdir
diff --git a/tests/xfs/1553 b/tests/xfs/1553
new file mode 100755
index 00000000000000..a82b3be2f59b5f
--- /dev/null
+++ b/tests/xfs/1553
@@ -0,0 +1,35 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1553
+#
+# Populate a XFS filesystem and fuzz every metadir subdir field.
+# Try online repair and, if necessary, offline repair,
+# to test the most likely usage pattern.
+
+. ./common/preamble
+_begin_fstest dangerous_fuzzers scrub repair fuzzers_bothrepair
+
+_register_cleanup "_cleanup" BUS
+
+. ./common/filter
+. ./common/populate
+. ./common/fuzzy
+
+_require_xfs_scratch_metadir
+_require_scratch_xfs_fuzz_fields
+_disable_dmesg_check
+
+echo "Format and populate"
+_scratch_populate_cached nofill > $seqres.full 2>&1
+
+inode_ver=$(_scratch_xfs_get_metadata_field "core.version" 'path -m /rtgroups')
+
+echo "Fuzz metadir subdir"
+_scratch_xfs_fuzz_metadata '' 'both' 'path -m /rtgroups' >> $seqres.full
+echo "Done fuzzing metadir subdir"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1553.out b/tests/xfs/1553.out
new file mode 100644
index 00000000000000..0298fcfddbf15a
--- /dev/null
+++ b/tests/xfs/1553.out
@@ -0,0 +1,4 @@
+QA output created by 1553
+Format and populate
+Fuzz metadir subdir
+Done fuzzing metadir subdir

[PATCH 08/11] xfs/163: bigger fs for metadir

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Adjust filesystem size up so we can pass this test even with metadir
and rtgroups enabled.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/163 |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


diff --git a/tests/xfs/163 b/tests/xfs/163
index 75c3113dc2fd03..015a82cdffaa68 100755
--- a/tests/xfs/163
+++ b/tests/xfs/163
@@ -42,7 +42,7 @@ echo "Format and mount"
 
 # agcount = 1 is forbidden on purpose, and need to ensure shrinking to
 # 2 AGs isn't feasible yet. So agcount = 3 is the minimum number now.
-_scratch_mkfs -dsize="$((512 * 1024 * 1024))" -dagcount=3 2>&1 | \
+_scratch_mkfs -dsize="$((900 * 1024 * 1024))" -dagcount=3 2>&1 | \
 	tee -a $seqres.full | _filter_mkfs 2>$tmp.mkfs >/dev/null
 . $tmp.mkfs
 t_dblocks=$dblocks

[PATCH 09/11] xfs/122: disable this test for any codebase that knows about metadir

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

All of the ondisk structure size checks from this test were copied to
the build time checks in xfs_ondisk.h.  This means that the kernel and
xfsprogs build processes check the structure sizes, which means that
fstests no longer needs to do that.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/122 |    6 ++++++
 1 file changed, 6 insertions(+)


diff --git a/tests/xfs/122 b/tests/xfs/122
index a968948841de14..e96ef2fe93042b 100755
--- a/tests/xfs/122
+++ b/tests/xfs/122
@@ -15,6 +15,12 @@ _begin_fstest other auto quick clone realtime
 
 _require_command "$INDENT_PROG" indent
 
+# We ported all the ondisk size checks to xfs_ondisk.h in both the kernel and
+# xfsprogs libxfs when we added the metadir feature.  If mkfs supports metadir
+# then we don't have to run this test anymore.
+$MKFS_XFS_PROG --help 2>&1 | grep -q metadir && \
+	_notrun "struct size checks moved to libxfs/xfs_ondisk.h"
+
 # Starting in Linux 6.1, the EFI log formats were adjusted away from using
 # single-element arrays as flex arrays.
 _wants_kernel_commit 03a7485cd701 \

[PATCH 10/11] scrub: race metapath online fsck with fsstress

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Add a pair of new tests to exercise fsstress vs. metapath repairs.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/xfs         |    3 ++
 tests/xfs/1892     |   66 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1892.out |    2 ++
 tests/xfs/1893     |   67 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1893.out |    2 ++
 5 files changed, 140 insertions(+)
 create mode 100755 tests/xfs/1892
 create mode 100644 tests/xfs/1892.out
 create mode 100755 tests/xfs/1893
 create mode 100644 tests/xfs/1893.out


diff --git a/common/xfs b/common/xfs
index 1c5daaa977aef1..b8e24c2e0ce8fe 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1664,6 +1664,9 @@ _xfs_filter_mkfs()
 	if (/^realtime\s+=([\w|\/.-]+)\s+extsz=(\d+)\s+blocks=(\d+), rtextents=(\d+)/) {
 		print STDERR "rtdev=$1\nrtextsz=$2\nrtblocks=$3\nrtextents=$4\n";
 		print STDOUT "realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX\n";
+	}
+	if (/^\s+=\s+rgcount=(\d+)\s+rgsize=(\d+) extents/) {
+		print STDERR "rgcount=$1\nrgextents=$2\n";
 	}'
 }
 
diff --git a/tests/xfs/1892 b/tests/xfs/1892
new file mode 100755
index 00000000000000..13310353564554
--- /dev/null
+++ b/tests/xfs/1892
@@ -0,0 +1,66 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1892
+#
+# Race fsstress and metadata directory tree path corruption detector for a
+# while to see if we crash or livelock.
+#
+. ./common/preamble
+_begin_fstest scrub fsstress_scrub
+
+_cleanup() {
+	_scratch_xfs_stress_scrub_cleanup &> /dev/null
+	cd /
+	rm -r -f $tmp.*
+}
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/fuzzy
+. ./common/inject
+. ./common/xfs
+
+_require_scratch
+_require_xfs_stress_scrub
+
+_scratch_mkfs | _filter_mkfs 2>$tmp.mkfs >/dev/null
+. $tmp.mkfs
+_scratch_mount
+
+verbs=()
+
+try_verb()
+{
+	$XFS_IO_PROG -x -c "scrub metapath $*" "$SCRATCH_MNT" 2>&1
+}
+
+# Metapath verbs that don't require arguments
+for v in quotadir usrquota grpquota prjquota rtdir; do
+	testio=$(try_verb "$v")
+	test -z "$testio" && verbs+=("$v")
+done
+
+# Metapath verbs that take a rt group number
+for ((rgno = 0; rgno < rgcount; rgno++)); do
+	for v in rtbitmap rtsummary rtrmapbt rtrefcbt; do
+		testio=$(try_verb "$v" "$rgno")
+		test -z "$testio" && verbs+=("$v $rgno")
+	done
+done
+test "${#verbs[@]}" -gt 0 || _notrun "no metapath verbs detected"
+
+args=()
+for v in "${verbs[@]}"; do
+	args+=("scrub metapath $v")
+done
+
+echo "${verbs[@]}" >> $seqres.full
+_scratch_xfs_stress_scrub "${args[@]}"
+
+# success, all done
+echo Silence is golden
+status=0
+exit
diff --git a/tests/xfs/1892.out b/tests/xfs/1892.out
new file mode 100644
index 00000000000000..afc84c43ed2dcc
--- /dev/null
+++ b/tests/xfs/1892.out
@@ -0,0 +1,2 @@
+QA output created by 1892
+Silence is golden
diff --git a/tests/xfs/1893 b/tests/xfs/1893
new file mode 100755
index 00000000000000..d06687fa2a1087
--- /dev/null
+++ b/tests/xfs/1893
@@ -0,0 +1,67 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1893
+#
+# Race fsstress and metadata directory tree path repair for a while to see if
+# we crash or livelock.
+#
+. ./common/preamble
+_begin_fstest online_repair fsstress_online_repair
+
+_cleanup() {
+	_scratch_xfs_stress_scrub_cleanup &> /dev/null
+	cd /
+	rm -r -f $tmp.*
+}
+_register_cleanup "_cleanup" BUS
+
+# Import common functions.
+. ./common/filter
+. ./common/fuzzy
+. ./common/inject
+. ./common/xfs
+
+_require_scratch
+_require_xfs_stress_online_repair
+
+_scratch_mkfs | _filter_mkfs 2>$tmp.mkfs >/dev/null
+. $tmp.mkfs
+_scratch_mount
+
+verbs=()
+
+try_verb()
+{
+	$XFS_IO_PROG -x -c "repair metapath $*" "$SCRATCH_MNT" 2>&1 | \
+		sed -e '/did not need repair/d'
+}
+
+# Metapath verbs that don't require arguments
+for v in quotadir usrquota grpquota prjquota rtdir; do
+	testio=$(try_verb "$v")
+	test -z "$testio" && verbs+=("$v")
+done
+
+# Metapath verbs that take a rt group number
+for ((rgno = 0; rgno < rgcount; rgno++)); do
+	for v in rtbitmap rtsummary; do
+		testio=$(try_verb "$v" "$rgno")
+		test -z "$testio" && verbs+=("$v $rgno")
+	done
+done
+test "${#verbs[@]}" -gt 0 || _notrun "no metapath verbs detected"
+
+args=()
+for v in "${verbs[@]}"; do
+	args+=("repair metapath $v")
+done
+
+echo "${verbs[@]}" >> $seqres.full
+_scratch_xfs_stress_online_repair "${args[@]}"
+
+# success, all done
+echo Silence is golden
+status=0
+exit
diff --git a/tests/xfs/1893.out b/tests/xfs/1893.out
new file mode 100644
index 00000000000000..64d6b1b2bcd09e
--- /dev/null
+++ b/tests/xfs/1893.out
@@ -0,0 +1,2 @@
+QA output created by 1893
+Silence is golden

[PATCH 11/11] xfs: test metapath repairs

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Functional testing for metadir path checking and repairs.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/1874     |  119 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1874.out |   19 ++++++++
 2 files changed, 138 insertions(+)
 create mode 100755 tests/xfs/1874
 create mode 100644 tests/xfs/1874.out


diff --git a/tests/xfs/1874 b/tests/xfs/1874
new file mode 100755
index 00000000000000..2e1af9222fa0bf
--- /dev/null
+++ b/tests/xfs/1874
@@ -0,0 +1,119 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test 1874
+#
+# Functional test of using online repair to fix metadir paths.
+#
+. ./common/preamble
+_begin_fstest auto online_repair
+
+. ./common/filter
+. ./common/inject
+. ./common/fuzzy
+. ./common/quota
+
+_require_xfs_db_command "link"
+_require_xfs_db_command "unlink"
+_require_scratch
+_require_xfs_stress_online_repair
+
+prepare_fs() {
+	# Format filesystem
+	_scratch_mkfs | _filter_mkfs 2> $tmp.mkfs >> $seqres.full
+	_scratch_mount
+
+	_require_xfs_has_feature "$SCRATCH_MNT" rmapbt
+	_require_xfs_has_feature "$SCRATCH_MNT" realtime
+	_require_xfs_has_feature "$SCRATCH_MNT" metadir
+	_require_xfs_has_feature "$SCRATCH_MNT" parent
+
+	root_inum="$(stat -c '%i' $SCRATCH_MNT)"
+	__stress_scrub_check_commands "%dir%" '' '' 'scrub metapath'
+	_scratch_unmount
+
+	# Stash the /rtgroups inode number and gen
+	rt_metadir_inum=$(_scratch_xfs_get_metadata_field v3.inumber 'path -m /rtgroups')
+	rt_metadir_gen=$(_scratch_xfs_get_metadata_field core.gen 'path -m /rtgroups')
+
+	# Stash the /rtgroups/0.rmap inode number and gen
+	rbm_inum=$(_scratch_xfs_get_metadata_field v3.inumber 'path -m /rtgroups/0.rmap')
+	rbm_gen=$(_scratch_xfs_get_metadata_field core.gen 'path -m /rtgroups/0.rmap')
+
+	# Fuzz parent pointer in rtgroup 0 rmap file
+	_scratch_xfs_db -x \
+		-c 'path -m /rtgroups/0.rmap' \
+		-c "write -d a.sfattr.list[0].parent_dir.inumber $root_inum" >> $seqres.full
+}
+
+simple_online_repair() {
+	echo "check /rtgroups dir" | _tee_kernlog
+	$XFS_IO_PROG -c "scrub directory $rt_metadir_inum $rt_metadir_gen" $SCRATCH_MNT
+
+	echo "check /rtgroups/0.rmap pptr" | _tee_kernlog
+	$XFS_IO_PROG -c "scrub parent $rbm_inum $rbm_gen" $SCRATCH_MNT
+
+	echo "check /rtgroups/0.rmap metapath" | _tee_kernlog
+	$XFS_IO_PROG -c "scrub metapath rtrmapbt 0" $SCRATCH_MNT
+
+	echo "check nlinks" | _tee_kernlog
+	$XFS_IO_PROG -c "scrub nlinks" $SCRATCH_MNT
+
+	# Destroying a metadir path (e.g. /rtgroups/0.rmap) cannot be done
+	# offline because then the mount will fail.  Hence we must use a
+	# specific sequence of online repairs to remove the metadir path link.
+	# Only then can we use the metapath scrubber to restore the link.
+
+	# Force repair the parent directory.  Since /rtgroups/0.rmap has a bad
+	# parent pointer, the "0.rmap" entry in /rtgroups will not be created.
+	echo "fix /rtgroups dir" | _tee_kernlog
+	$XFS_IO_PROG -x -c "repair -R directory $rt_metadir_inum $rt_metadir_gen" $SCRATCH_MNT
+
+	# Force repair the parent pointer.  Since the "0.rmap" entry in
+	# /rtgroups no longer exists and no other directories count the
+	# rtgroup 0 rmap as a parent, this will fail cross-referencing after
+	# the repair.
+	echo "fix /rtgroups/0.rmap pptr" | _tee_kernlog
+	$XFS_IO_PROG -x -c "repair -R parent $rbm_inum $rbm_gen" $SCRATCH_MNT
+
+	# Now that we've completely erased the /rtgroups/0.rmap path, check
+	# that the link is indeed lost, and restore the link.
+	echo "fix /rtgroups/0.rmap metapath" | _tee_kernlog
+	$XFS_IO_PROG -x -c "repair metapath rtrmapbt 0" $SCRATCH_MNT
+
+	# Make sure we're not missing any link count
+	echo "fix nlinks" | _tee_kernlog
+	$XFS_IO_PROG -x -c "repair nlinks" $SCRATCH_MNT
+}
+
+echo Part 1: Use raw ioctls to detect the error and fix it.
+prepare_fs
+_scratch_mount
+simple_online_repair
+_check_scratch_fs
+_scratch_unmount
+
+echo Part 2: Use xfs_scrub to detect the error and fix it.
+prepare_fs
+_scratch_mount
+echo "fix with xfs_scrub" | _tee_kernlog
+_scratch_scrub &>> $seqres.full
+echo "xfs_scrub returned $?" >> $seqres.full
+_check_scratch_fs
+_scratch_unmount
+
+echo Part 3: Use xfs_repair to detect the error and fix it.
+prepare_fs
+echo "fix with xfs_repair" | _tee_kernlog
+echo repair?? >> $seqres.full
+_scratch_xfs_repair &>> $seqres.full
+echo "xfs_repair returned $?" >> $seqres.full
+_scratch_mount
+_check_scratch_fs
+_scratch_unmount
+
+echo "done with test" | _tee_kernlog
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1874.out b/tests/xfs/1874.out
new file mode 100644
index 00000000000000..ff4497363d8063
--- /dev/null
+++ b/tests/xfs/1874.out
@@ -0,0 +1,19 @@
+QA output created by 1874
+Part 1: Use raw ioctls to detect the error and fix it.
+check /rtgroups dir
+Corruption detected during cross-referencing.
+check /rtgroups/0.rmap pptr
+Corruption detected during cross-referencing.
+check /rtgroups/0.rmap metapath
+check nlinks
+fix /rtgroups dir
+fix /rtgroups/0.rmap pptr
+Corruption remains.
+Corruption still detected during cross-referencing.
+fix /rtgroups/0.rmap metapath
+fix nlinks
+Part 2: Use xfs_scrub to detect the error and fix it.
+fix with xfs_scrub
+Part 3: Use xfs_repair to detect the error and fix it.
+fix with xfs_repair
+done with test

[PATCH 1/1] fstests: test mkfs.xfs protofiles with xattr support

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Make sure we can do protofiles with xattr support.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/1937     |  144 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1937.out |  102 +++++++++++++++++++++++++++++++++++++
 2 files changed, 246 insertions(+)
 create mode 100755 tests/xfs/1937
 create mode 100644 tests/xfs/1937.out


diff --git a/tests/xfs/1937 b/tests/xfs/1937
new file mode 100755
index 00000000000000..aa4143a75ef643
--- /dev/null
+++ b/tests/xfs/1937
@@ -0,0 +1,144 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2025 Oracle.  All Rights Reserved.
+# Copyright (c) 2000-2004 Silicon Graphics, Inc.  All Rights Reserved.
+#
+# FS QA Test No. 1937
+#
+# mkfs protofile with xattrs test
+#
+. ./common/preamble
+_begin_fstest mkfs auto quick
+
+seqfull="$seqres.full"
+rm -f $seqfull
+
+. ./common/filter
+
+_cleanup()
+{
+	echo "*** unmount"
+	_scratch_unmount 2>/dev/null
+	rm -f $tmp.*
+	rm -f $TEST_DIR/$seq.file
+}
+
+_full()
+{
+	echo ""            >>$seqfull
+	echo "*** $* ***"  >>$seqfull
+	echo ""            >>$seqfull
+}
+
+_filter_stat()
+{
+	sed '
+		/^Access:/d;
+		/^Modify:/d;
+		/^Change:/d;
+		s/Device: *[0-9][0-9]*,[0-9][0-9]*/Device: <DEVICE>/;
+		s/Inode: *[0-9][0-9]*/Inode: <INODE>/;
+		s/Size: *[0-9][0-9]* *Filetype: Dir/Size: <DSIZE> Filetype: Dir/;
+	' | tr -s ' '
+}
+
+_require_command $ATTR_PROG "attr"
+_require_scratch
+
+# mkfs cannot create a filesystem with protofiles if realtime is enabled, so
+# don't run this test if the rtinherit is anywhere in the mkfs options.
+echo "$MKFS_OPTIONS" | grep -q "rtinherit" && \
+	_notrun "Cannot mkfs with a protofile and -d rtinherit."
+
+protofile=$tmp.proto
+tempfile=$TEST_DIR/$seq.file
+
+$XFS_IO_PROG -f -c 'pwrite 64k 28k' -c 'pwrite 1280k 37960' $tempfile >> $seqres.full
+$here/src/devzero -b 2048 -n 2 -c -v 44 $tempfile.2 
+
+$ATTR_PROG -R -s rootdata -V 0test $tempfile &>> $seqres.full
+$ATTR_PROG -S -s acldata -V 1test $tempfile &>> $seqres.full
+$ATTR_PROG -s userdata -V 2test $tempfile &>> $seqres.full
+perl -e 'print "x" x 37960;' | $ATTR_PROG -s bigdata $tempfile &>> $seqres.full
+
+cat >$protofile <<EOF
+DUMMY1
+0 0
+: root directory
+d--777 3 1
+: a directory
+directory d--755 3 1 
+test ---755 3 1 $tempfile
+file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_0 ---755 3 1 $tempfile
+file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1 ---755 3 1 $tempfile
+file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2 ---755 3 1 $tempfile
+file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3 ---755 3 1 $tempfile
+file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_4 ---755 3 1 $tempfile
+$
+: back in the root
+setuid -u-666 0 0 $tempfile
+setgid --g666 0 0 $tempfile
+setugid -ug666 0 0 $tempfile
+directory_setgid d-g755 3 2
+file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_5 ---755 3 1 $tempfile
+$
+: back in the root
+block_device b--012 3 1 161 162 
+char_device c--345 3 1 177 178
+pipe p--670 0 0
+symlink l--123 0 0 bigfile
+: a file we actually read
+bigfile ---666 3 0 $tempfile.2
+: done
+$
+EOF
+
+if [ $? -ne 0 ]
+then
+	_fail "failed to create test protofile"
+fi
+
+_verify_fs()
+{
+	echo "*** create FS version $1"
+	VERSION="-n version=$1"
+
+	_scratch_unmount >/dev/null 2>&1
+
+	_full "mkfs"
+	_scratch_mkfs_xfs $VERSION -p $protofile >>$seqfull 2>&1
+
+	echo "*** check FS"
+	_check_scratch_fs
+
+	echo "*** mount FS"
+	_full " mount"
+	_try_scratch_mount >>$seqfull 2>&1 \
+		|| _fail "mount failed"
+
+	$ATTR_PROG -l $SCRATCH_MNT/directory/test | \
+		grep -q 'Attribute.*has a ' || \
+		_notrun "mkfs.xfs protofile does not support xattrs"
+
+	echo "*** verify FS"
+	(cd $SCRATCH_MNT ; find . | LC_COLLATE=POSIX sort \
+		| grep -v ".use_space" \
+		| xargs $here/src/lstat64 | _filter_stat)
+	diff -q $SCRATCH_MNT/bigfile $tempfile.2 \
+		|| _fail "bigfile corrupted"
+	diff -q $SCRATCH_MNT/symlink $tempfile.2 \
+		|| _fail "symlink broken"
+
+	$ATTR_PROG -l $SCRATCH_MNT/directory/test | _filter_scratch
+
+	echo "*** unmount FS"
+	_full "umount"
+	_scratch_unmount >>$seqfull 2>&1 \
+		|| _fail "umount failed"
+}
+
+_verify_fs 2
+
+echo "*** done"
+status=0
+exit
diff --git a/tests/xfs/1937.out b/tests/xfs/1937.out
new file mode 100644
index 00000000000000..050c8318b1abca
--- /dev/null
+++ b/tests/xfs/1937.out
@@ -0,0 +1,102 @@
+QA output created by 1937
+Wrote 2048.00Kb (value 0x2c)
+*** create FS version 2
+*** check FS
+*** mount FS
+*** verify FS
+ File: "."
+ Size: <DSIZE> Filetype: Directory
+ Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 4 
+
+ File: "./bigfile"
+ Size: 2097152 Filetype: Regular File
+ Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./block_device"
+ Size: 0 Filetype: Block Device
+ Mode: (0012/b-----x-w-) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 1 Device type: 161,162
+
+ File: "./char_device"
+ Size: 0 Filetype: Character Device
+ Mode: (0345/c-wxr--r-x) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 1 Device type: 177,178
+
+ File: "./directory"
+ Size: <DSIZE> Filetype: Directory
+ Mode: (0755/drwxr-xr-x) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 2 
+
+ File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_0"
+ Size: 1348680 Filetype: Regular File
+ Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1"
+ Size: 1348680 Filetype: Regular File
+ Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2"
+ Size: 1348680 Filetype: Regular File
+ Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3"
+ Size: 1348680 Filetype: Regular File
+ Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_4"
+ Size: 1348680 Filetype: Regular File
+ Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./directory/test"
+ Size: 1348680 Filetype: Regular File
+ Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./directory_setgid"
+ Size: <DSIZE> Filetype: Directory
+ Mode: (2755/drwxr-sr-x) Uid: (3) Gid: (2)
+Device: <DEVICE> Inode: <INODE> Links: 2 
+
+ File: "./directory_setgid/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_5"
+ Size: 1348680 Filetype: Regular File
+ Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./pipe"
+ Size: 0 Filetype: Fifo File
+ Mode: (0670/frw-rwx---) Uid: (0) Gid: (0)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./setgid"
+ Size: 1348680 Filetype: Regular File
+ Mode: (2666/-rw-rwsrw-) Uid: (0) Gid: (0)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./setugid"
+ Size: 1348680 Filetype: Regular File
+ Mode: (6666/-rwsrwsrw-) Uid: (0) Gid: (0)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./setuid"
+ Size: 1348680 Filetype: Regular File
+ Mode: (4666/-rwsrw-rw-) Uid: (0) Gid: (0)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+
+ File: "./symlink"
+ Size: 7 Filetype: Symbolic Link
+ Mode: (0123/l--x-w--wx) Uid: (0) Gid: (0)
+Device: <DEVICE> Inode: <INODE> Links: 1 
+Attribute "userdata" has a 5 byte value for SCRATCH_MNT/directory/test
+Attribute "rootdata" has a 5 byte value for SCRATCH_MNT/directory/test
+Attribute "bigdata" has a 37960 byte value for SCRATCH_MNT/directory/test
+Attribute "acldata" has a 5 byte value for SCRATCH_MNT/directory/test
+*** unmount FS
+*** done
+*** unmount

[PATCH 01/14] common/populate: refactor caching of metadumps to a helper

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Hoist out of _scratch_populate_cached all the code that we use to save a
metadump of the populated filesystem.  We're going to make this more
involved for XFS in the next few patches so that we can take advantage
of the new support for external devices in metadump/mdrestore.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/populate |   37 ++++++++++++++++++++++++++++---------
 1 file changed, 28 insertions(+), 9 deletions(-)


diff --git a/common/populate b/common/populate
index 4cf9c0691956a3..4cc4415697ac78 100644
--- a/common/populate
+++ b/common/populate
@@ -1051,6 +1051,31 @@ _scratch_populate_restore_cached() {
 	return 1
 }
 
+# Take a metadump of the scratch filesystem and cache it for later.
+_scratch_populate_save_metadump()
+{
+	local metadump_file="$1"
+
+	case "${FSTYP}" in
+	"xfs")
+		local logdev=none
+		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
+			logdev=$SCRATCH_LOGDEV
+
+		_xfs_metadump "$metadump_file" "$SCRATCH_DEV" "$logdev" \
+				compress -a -o
+		res=$?
+		;;
+	"ext2"|"ext3"|"ext4")
+		_ext4_metadump "${SCRATCH_DEV}" "${metadump_file}" compress
+		res=$?
+		;;
+	*)
+		_fail "Don't know how to save a ${FSTYP} filesystem."
+	esac
+	return $res
+}
+
 # Populate a scratch FS from scratch or from a cached image.
 _scratch_populate_cached() {
 	local meta_descr="$(_scratch_populate_cache_tag "$@")"
@@ -1074,26 +1099,20 @@ _scratch_populate_cached() {
 
 	# Oh well, just create one from scratch
 	_scratch_mkfs
-	echo "${meta_descr}" > "${populate_metadump_descr}"
 	case "${FSTYP}" in
 	"xfs")
 		_scratch_xfs_populate $@
 		_scratch_xfs_populate_check
-
-		local logdev=none
-		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
-			logdev=$SCRATCH_LOGDEV
-
-		_xfs_metadump "$POPULATE_METADUMP" "$SCRATCH_DEV" "$logdev" \
-			compress -a -o
 		;;
 	"ext2"|"ext3"|"ext4")
 		_scratch_ext4_populate $@
 		_scratch_ext4_populate_check
-		_ext4_metadump "${SCRATCH_DEV}" "${POPULATE_METADUMP}" compress
 		;;
 	*)
 		_fail "Don't know how to populate a ${FSTYP} filesystem."
 		;;
 	esac
+
+	_scratch_populate_save_metadump "${POPULATE_METADUMP}" && \
+			echo "${meta_descr}" > "${populate_metadump_descr}"
 }

[PATCH 02/14] common/{fuzzy,populate}: use _scratch_xfs_mdrestore

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Port the fuzzing and populated filesystem cache code to use this helper
to pick up external log devices for the scratch filesystem.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/fuzzy    |    2 +-
 common/populate |   15 ++-------------
 2 files changed, 3 insertions(+), 14 deletions(-)


diff --git a/common/fuzzy b/common/fuzzy
index 331bf5ad7bbafa..9fcaf9b6ee55a9 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -306,7 +306,7 @@ __scratch_xfs_fuzz_unmount()
 __scratch_xfs_fuzz_mdrestore()
 {
 	__scratch_xfs_fuzz_unmount
-	_xfs_mdrestore "${POPULATE_METADUMP}" "${SCRATCH_DEV}" || \
+	_scratch_xfs_mdrestore "${POPULATE_METADUMP}" || \
 		_fail "${POPULATE_METADUMP}: Could not find metadump to restore?"
 }
 
diff --git a/common/populate b/common/populate
index 4cc4415697ac78..814f28a41c9bea 100644
--- a/common/populate
+++ b/common/populate
@@ -1017,19 +1017,8 @@ _scratch_populate_restore_cached() {
 
 	case "${FSTYP}" in
 	"xfs")
-		_xfs_mdrestore "${metadump}" "${SCRATCH_DEV}"
-		res=$?
-		test $res -ne 0 && return $res
-
-		# Cached images should have been unmounted cleanly, so if
-		# there's an external log we need to wipe it and run repair to
-		# format it to match this filesystem.
-		if [ -n "${SCRATCH_LOGDEV}" ]; then
-			$WIPEFS_PROG -a "${SCRATCH_LOGDEV}"
-			_scratch_xfs_repair
-			res=$?
-		fi
-		return $res
+		_scratch_xfs_mdrestore "${metadump}"
+		return $?
 		;;
 	"ext2"|"ext3"|"ext4")
 		_ext4_mdrestore "${metadump}" "${SCRATCH_DEV}"

[PATCH 03/14] fuzzy: stress data and rt sections of xfs filesystems equally

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

If we're stress-testing scrub on a realtime filesystem, make sure that
we run fsstress on separate directory trees for data and realtime
workouts.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/fuzzy |   55 +++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 39 insertions(+), 16 deletions(-)


diff --git a/common/fuzzy b/common/fuzzy
index 9fcaf9b6ee55a9..a7e28dda137e8a 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -1018,15 +1018,28 @@ __stress_scrub_fsx_loop() {
 	rm -f "$runningfile"
 }
 
+# Run fsstress and record outcome
+__run_fsstress() {
+	_run_fsstress $*
+	local res=$?
+	echo "fsstress $* exits with $res at $(date)" >> $seqres.full
+	test "$res" -ne 0 && touch "$tmp.killstress"
+}
+
 # Run fsstress while we're testing online fsck.
 __stress_scrub_fsstress_loop() {
 	local end="$1"
 	local runningfile="$2"
 	local remount_period="$3"
 	local stress_tgt="$4"
-	local focus=()
-	local res
+	local focus=(-p 4 -n 2000000)
+	local res res2
 	local duration
+	local has_rt
+	local d_args r_args
+
+	test $FSTYP = "xfs" && _xfs_has_feature "$SCRATCH_MNT" realtime && \
+		has_rt=1
 
 	case "$stress_tgt" in
 	"parent")
@@ -1118,9 +1131,21 @@ __stress_scrub_fsstress_loop() {
 
 	# As of March 2022, 2 million fsstress ops should be enough to keep
 	# any filesystem busy for a couple of hours.
-	local args=$(_scale_fsstress_args -p 4 -d $SCRATCH_MNT -n 2000000 "${focus[@]}")
-	echo "Running $FSSTRESS_PROG $args" >> $seqres.full
+	if [ -n "$has_rt" ]; then
+		mkdir -p $SCRATCH_MNT/rt $SCRATCH_MNT/data
+		$XFS_IO_PROG -c 'chattr +rt' $SCRATCH_MNT/rt
+		$XFS_IO_PROG -c 'chattr -rt' $SCRATCH_MNT/data
 
+		r_args=$(_scale_fsstress_args -d $SCRATCH_MNT/rt "${focus[@]}")
+		d_args=$(_scale_fsstress_args -d $SCRATCH_MNT/data "${focus[@]}")
+		echo "Running $FSSTRESS_PROG $d_args" >> $seqres.full
+		echo "Running $FSSTRESS_PROG $r_args" >> $seqres.full
+	else
+		d_args=$(_scale_fsstress_args -d $SCRATCH_MNT "${focus[@]}")
+		echo "Running $FSSTRESS_PROG $d_args" >> $seqres.full
+	fi
+
+	rm -f "$tmp.killstress"
 	if [ -n "$remount_period" ]; then
 		local mode="rw"
 		local rw_arg=""
@@ -1130,12 +1155,10 @@ __stress_scrub_fsstress_loop() {
 			test "$mode" = "rw" && __stress_scrub_clean_scratch && continue
 
 			duration=$(___stress_scrub_duration "$end" "$remount_period")
-			_run_fsstress_bg $duration $args $rw_arg >> $seqres.full
-			sleep $remount_period
-			_kill_fsstress
-			res=$?
-			echo "$mode fsstress exits with $res at $(date)" >> $seqres.full
-			[ "$res" -ne 0 ] && break;
+			__run_fsstress $duration $d_args $rw_arg &
+			test -n "$has_rt" && __run_fsstress $duration $r_args $rw_arg &
+			wait
+			test -e "$tmp.killstress" && break
 
 			if [ "$mode" = "rw" ]; then
 				mode="ro"
@@ -1151,7 +1174,7 @@ __stress_scrub_fsstress_loop() {
 				sleep 0.2
 			done
 		done
-		rm -f "$runningfile"
+		rm -f "$runningfile" "$tmp.killstress"
 		return 0
 	fi
 
@@ -1159,12 +1182,12 @@ __stress_scrub_fsstress_loop() {
 		# Need to recheck running conditions if we cleared anything
 		__stress_scrub_clean_scratch && continue
 		duration=$(___stress_scrub_duration "$end" "$remount_period")
-		_run_fsstress $duration $args >> $seqres.full
-		res=$?
-		echo "$mode fsstress exits with $res at $(date)" >> $seqres.full
-		[ "$res" -ne 0 ] && break;
+		__run_fsstress $duration $d_args &
+		test -n "$has_rt" && __run_fsstress $duration $r_args &
+		wait
+		test -e "$tmp.killstress" && break
 	done
-	rm -f "$runningfile"
+	rm -f "$runningfile" "$tmp.killstress"
 }
 
 # Make sure we have everything we need to run stress and scrub

[PATCH 04/14] common/ext4: reformat external logs during mdrestore operations

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The e2image file format doesn't support the capture of external log
devices, which means that mdrestore ought to reformat the external log
to get the restored filesystem to work again.  The common/populate code
could already do this, so push it to the common ext4 helper.

While we're at it, fix the uncareful usage of SCRATCH_LOGDEV in the
populate code.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/ext4     |   17 ++++++++++++++++-
 common/populate |   18 +++++-------------
 2 files changed, 21 insertions(+), 14 deletions(-)


diff --git a/common/ext4 b/common/ext4
index 13921bb8165a4d..e1b336d3d20cba 100644
--- a/common/ext4
+++ b/common/ext4
@@ -134,7 +134,8 @@ _ext4_mdrestore()
 {
 	local metadump="$1"
 	local device="$2"
-	shift; shift
+	local logdev="$3"
+	shift; shift; shift
 	local options="$@"
 
 	# If we're configured for compressed dumps and there isn't already an
@@ -148,6 +149,20 @@ _ext4_mdrestore()
 	test -r "$metadump" || return 1
 
 	$E2IMAGE_PROG $options -r "${metadump}" "${SCRATCH_DEV}"
+	res=$?
+	test $res -ne 0 && return $res
+
+	# ext4 cannot e2image external logs, so we have to reformat the log
+	# device to match the restored fs
+	if [ "${logdev}" != "none" ]; then
+		local fsuuid="$($DUMPE2FS_PROG -h "${SCRATCH_DEV}" 2>/dev/null | \
+				grep 'Journal UUID:' | \
+				sed -e 's/Journal UUID:[[:space:]]*//g')"
+		$MKFS_EXT4_PROG -O journal_dev "${logdev}" \
+				-F -U "${fsuuid}"
+		res=$?
+	fi
+	return $res
 }
 
 # this test requires the ext4 kernel support crc feature on scratch device
diff --git a/common/populate b/common/populate
index 814f28a41c9bea..65fbd19b30e4e1 100644
--- a/common/populate
+++ b/common/populate
@@ -1021,20 +1021,12 @@ _scratch_populate_restore_cached() {
 		return $?
 		;;
 	"ext2"|"ext3"|"ext4")
-		_ext4_mdrestore "${metadump}" "${SCRATCH_DEV}"
-		ret=$?
-		test $ret -ne 0 && return $ret
+		local logdev=none
+		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
+			logdev=$SCRATCH_LOGDEV
 
-		# ext4 cannot e2image external logs, so we have to reformat
-		# the scratch device to match the restored fs
-		if [ -n "${SCRATCH_LOGDEV}" ]; then
-			local fsuuid="$($DUMPE2FS_PROG -h "${SCRATCH_DEV}" 2>/dev/null | \
-					grep 'Journal UUID:' | \
-					sed -e 's/Journal UUID:[[:space:]]*//g')"
-			$MKFS_EXT4_PROG -O journal_dev "${SCRATCH_LOGDEV}" \
-					-F -U "${fsuuid}"
-		fi
-		return 0
+		_ext4_mdrestore "${metadump}" "${SCRATCH_DEV}" "${logdev}"
+		return $?
 		;;
 	esac
 	return 1

[PATCH 05/14] common/populate: use metadump v2 format by default for fs metadata snapshots

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

When we're snapshotting filesystem metadata after creating a populated
filesystem, force the creation of metadump v2 files by default to
exercise the new format, since xfs_metadump continues to use the v1
format unless told otherwise.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/populate |   13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)


diff --git a/common/populate b/common/populate
index 65fbd19b30e4e1..96fc13875df503 100644
--- a/common/populate
+++ b/common/populate
@@ -55,7 +55,12 @@ __populate_fail() {
 	case "$FSTYP" in
 	xfs)
 		_scratch_unmount
-		_scratch_xfs_metadump "$metadump" -a -o
+
+		mdargs=('-a' '-o')
+		test "$(_xfs_metadump_max_version)" -gt 1 && \
+			mdargs+=('-v' '2')
+
+		_scratch_xfs_metadump "$metadump" "${mdargs[@]}"
 		;;
 	ext4)
 		_scratch_unmount
@@ -1043,8 +1048,12 @@ _scratch_populate_save_metadump()
 		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
 			logdev=$SCRATCH_LOGDEV
 
+		mdargs=('-a' '-o')
+		test "$(_xfs_metadump_max_version)" -gt 1 && \
+			mdargs+=('-v' '2')
+
 		_xfs_metadump "$metadump_file" "$SCRATCH_DEV" "$logdev" \
-				compress -a -o
+				compress "${mdargs[@]}"
 		res=$?
 		;;
 	"ext2"|"ext3"|"ext4")

[PATCH 06/14] punch-alternating: detect xfs realtime files with large allocation units

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

For files on the XFS realtime volume, it's possible that the file
allocation unit (aka the minimum size we have to punch to deallocate
file blocks) could be greater than a single fs block.  This utility
assumed that it's always possible to punch a single fs block, but for
these types of files, all that does is zeroes the page cache.  While
that's what most *user applications* want, fstests uses punching to
fragment file mapping metadata and/or fragment free space, so adapt this
test for that purpose by detecting realtime files.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 src/punch-alternating.c |   28 +++++++++++++++++++++++++++-
 tests/xfs/114           |    4 ++++
 tests/xfs/146           |    2 +-
 tests/xfs/187           |    3 ++-
 tests/xfs/341           |    4 ++--
 5 files changed, 36 insertions(+), 5 deletions(-)


diff --git a/src/punch-alternating.c b/src/punch-alternating.c
index 18dd215197db2b..d2bb4b6a2276c9 100644
--- a/src/punch-alternating.c
+++ b/src/punch-alternating.c
@@ -20,6 +20,28 @@ void usage(char *cmd)
 	exit(1);
 }
 
+/* Compute the file allocation unit size for an XFS file. */
+static int detect_xfs_alloc_unit(int fd)
+{
+	struct fsxattr fsx;
+	struct xfs_fsop_geom fsgeom;
+	int ret;
+
+	ret = ioctl(fd, XFS_IOC_FSGEOMETRY, &fsgeom);
+	if (ret)
+		return -1;
+
+	ret = ioctl(fd, XFS_IOC_FSGETXATTR, &fsx);
+	if (ret)
+		return -1;
+
+	ret = fsgeom.blocksize;
+	if (fsx.fsx_xflags & XFS_XFLAG_REALTIME)
+		ret *= fsgeom.rtextsize;
+
+	return ret;
+}
+
 int main(int argc, char *argv[])
 {
 	struct stat	s;
@@ -82,7 +104,11 @@ int main(int argc, char *argv[])
 		goto err;
 
 	sz = s.st_size;
-	blksz = sf.f_bsize;
+	c = detect_xfs_alloc_unit(fd);
+	if (c > 0)
+		blksz = c;
+	else
+		blksz = sf.f_bsize;
 
 	mode = FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE;
 	for (offset = start_offset * blksz;
diff --git a/tests/xfs/114 b/tests/xfs/114
index 510d31a4028598..f764cad73babb7 100755
--- a/tests/xfs/114
+++ b/tests/xfs/114
@@ -47,6 +47,10 @@ $XFS_IO_PROG -f \
 	-c "pwrite -S 0x68 -b 1048576 0 $len2" \
 	$SCRATCH_MNT/f2 >> $seqres.full
 
+# The arguments to punch-alternating must be specified in units of file
+# allocation units, so we divide the argument by $file_blksz.  We already
+# verified that $blksz is congruent with $file_blksz, so the fpunch parameters
+# will always align with the file allocation unit.
 $here/src/punch-alternating -o $((16 * blksz / file_blksz)) \
 	-s $((blksz / file_blksz)) \
 	-i $((blksz * 2 / file_blksz)) \
diff --git a/tests/xfs/146 b/tests/xfs/146
index b6f4c2bd093d45..1cd7076d2426ee 100755
--- a/tests/xfs/146
+++ b/tests/xfs/146
@@ -67,7 +67,7 @@ _xfs_force_bdev realtime $SCRATCH_MNT
 # Allocate some stuff at the start, to force the first falloc of the ouch file
 # to happen somewhere in the middle of the rt volume
 $XFS_IO_PROG -f -c 'falloc 0 64m' "$SCRATCH_MNT/b"
-$here/src/punch-alternating -i $((rextblks * 2)) -s $((rextblks)) "$SCRATCH_MNT/b"
+$here/src/punch-alternating "$SCRATCH_MNT/b"
 
 avail="$(df -P "$SCRATCH_MNT" | awk 'END {print $4}')"1
 toobig="$((avail * 2))"
diff --git a/tests/xfs/187 b/tests/xfs/187
index 56a9adc164eab2..1d32d702f629c9 100755
--- a/tests/xfs/187
+++ b/tests/xfs/187
@@ -130,7 +130,8 @@ $XFS_IO_PROG -f -c "truncate $required_sz" -c "falloc 0 $remap_sz" $SCRATCH_MNT/
 # Punch out every other extent of the last two sections, to fragment free space.
 frag_sz=$((remap_sz * 3))
 punch_off=$((bigfile_sz - frag_sz))
-$here/src/punch-alternating $SCRATCH_MNT/bigfile -o $((punch_off / fsbsize)) -i $((rtextsize_blks * 2)) -s $rtextsize_blks
+rtextsize_bytes=$((fsbsize * rtextsize_blks))
+$here/src/punch-alternating $SCRATCH_MNT/bigfile -o $((punch_off / rtextsize_bytes))
 
 # Make sure we have some free rtextents.
 free_rtx=$(_xfs_statfs_field "$SCRATCH_MNT" statfs.f_bavail)
diff --git a/tests/xfs/341 b/tests/xfs/341
index 6e25549b2b3d08..9b12febf8d5c49 100755
--- a/tests/xfs/341
+++ b/tests/xfs/341
@@ -41,8 +41,8 @@ len=$((blocks * rtextsz))
 echo "Create some files"
 $XFS_IO_PROG -f -R -c "falloc 0 $len" -c "pwrite -S 0x68 -b 1048576 0 $len" $SCRATCH_MNT/f1 >> $seqres.full
 $XFS_IO_PROG -f -R -c "falloc 0 $len" -c "pwrite -S 0x68 -b 1048576 0 $len" $SCRATCH_MNT/f2 >> $seqres.full
-$here/src/punch-alternating -i $((2 * rtextsz_blks)) -s $rtextsz_blks $SCRATCH_MNT/f1 >> "$seqres.full"
-$here/src/punch-alternating -i $((2 * rtextsz_blks)) -s $rtextsz_blks $SCRATCH_MNT/f2 >> "$seqres.full"
+$here/src/punch-alternating $SCRATCH_MNT/f1 >> "$seqres.full"
+$here/src/punch-alternating $SCRATCH_MNT/f2 >> "$seqres.full"
 echo garbage > $SCRATCH_MNT/f3
 ino=$(stat -c '%i' $SCRATCH_MNT/f3)
 _scratch_unmount

[PATCH 07/14] xfs/206: update mkfs filtering for rt groups feature

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Filter out the new mkfs lines that show the rtgroup information, since
this test is heavily dependent on old mkfs output.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/206 |    1 +
 1 file changed, 1 insertion(+)


diff --git a/tests/xfs/206 b/tests/xfs/206
index ef5f4868e9bdca..01531e1f08c37e 100755
--- a/tests/xfs/206
+++ b/tests/xfs/206
@@ -65,6 +65,7 @@ mkfs_filter()
 	    -e "/exchange=/d" \
 	    -e '/metadir=.*/d' \
 	    -e 's/, parent=[01]//' \
+	    -e '/rgcount=/d' \
 	    -e "/^Default configuration/d"
 }
 

[PATCH 08/14] common: pass the realtime device to xfs_db when possible

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Teach xfstests to pass the realtime device to xfs_db when it supports
that option.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/xfs |    4 ++++
 1 file changed, 4 insertions(+)


diff --git a/common/xfs b/common/xfs
index b8e24c2e0ce8fe..6e2070b5647b38 100644
--- a/common/xfs
+++ b/common/xfs
@@ -308,6 +308,10 @@ _scratch_xfs_db_options()
 	SCRATCH_OPTIONS=""
 	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
 		SCRATCH_OPTIONS="-l$SCRATCH_LOGDEV"
+	if [ "$USE_EXTERNAL" = yes ] && [ ! -z "$SCRATCH_RTDEV" ]; then
+		$XFS_DB_PROG --help 2>&1 | grep -q -- '-R rtdev' && \
+			SCRATCH_OPTIONS="$SCRATCH_OPTIONS -R$SCRATCH_RTDEV"
+	fi
 	echo $SCRATCH_OPTIONS $* $SCRATCH_DEV
 }
 

[PATCH 09/14] xfs/185: update for rtgroups

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

This old test is a bit too fixated on exact rt allocator behavior.  With
rtgroups enabled, we can end up with one large contiguous region that's
split into multiple bmbt mappings to avoid crossing rtgroup boundaries.
The realtime superblock throws another twist into the mix because the
first rtx will always be in use, which can shift the start of the
physical space mappings by up to 1 rtx.

Also fix a bug where we'd try to fallocate the total number of rtx,
whereas we should be asking for the number of free rtx to avoid ENOSPC
errors.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/185 |   65 ++++++++++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 50 insertions(+), 15 deletions(-)


diff --git a/tests/xfs/185 b/tests/xfs/185
index f3601a5292ef0b..7aceb383ce4609 100755
--- a/tests/xfs/185
+++ b/tests/xfs/185
@@ -97,10 +97,17 @@ test "$ddbytes" -lt "$((rtbytes + (10 * rtextsize) ))" || \
 # higher than the size of the data device.  For realtime files this is really
 # easy because fallocate for the first rt file always starts allocating at
 # physical offset zero.
-alloc_rtx="$((rtbytes / rtextsize))"
+rtfreebytes="$(stat -f -c '%S * %a' $rtfile | bc)"
+alloc_rtx="$((rtfreebytes / rtextsize))"
 $XFS_IO_PROG -c "falloc 0 $((alloc_rtx * rtextsize))" $rtfile
 
-expected_end="$(( (alloc_rtx * rtextsize - 1) / 512 ))"
+# log a bunch of geometry data to the full file for debugging
+echo "rtbytes $rtbytes rtfreebytes $rtfreebytes rtextsize $rtextsize" >> $seqres.full
+echo "allocrtx $alloc_rtx falloc $((alloc_rtx * rtextsize))" >> $seqres.full
+$XFS_IO_PROG -c statfs $SCRATCH_MNT >> $seqres.full
+
+total_rtx=$(_xfs_statfs_field $SCRATCH_MNT geom.rtextents)
+expected_end="$(( (total_rtx * rtextsize - 1) / 512 ))"
 
 # Print extent mapping of rtfile in format:
 # file_offset file_end physical_offset physical_end
@@ -113,13 +120,28 @@ rtfile_exts() {
 		done
 }
 
-# Make sure that we actually got the entire device.
-rtfile_exts | $AWK_PROG -v end=$expected_end '
+# Make sure that fallocate actually managed to map the entire rt device.  The
+# realtime superblock may consume the first rtx, so we allow for that here.
+# Allow for multiple contiguous mappings if the rtgroups are very small.
+allowed_start=$((rtextsize / 512))
+rtfile_exts | $AWK_PROG -v exp_start=$allowed_start -v exp_end=$expected_end '
+BEGIN {
+	start = -1;
+	end = -1;
+}
 {
-	if ($3 != 0)
-		printf("%s: unexpected physical offset %s, wanted 0\n", $0, $3);
-	if ($4 != end)
-		printf("%s: unexpected physical end %s, wanted %d\n", $0, $4, end);
+	if (end >= 0 && ($3 != end + 1))
+		printf("%s: non-contiguous allocation should have started at %s\n", $0, end + 1);
+	if (start < 0 || $3 < start)
+		start = $3;
+	if (end < 0 || $4 > end)
+		end = $4;
+}
+END {
+	if (start > exp_start)
+		printf("%s: unexpected physical offset %d, wanted <= %d\n", $0, start, exp_start);
+	if (end != exp_end)
+		printf("%s: unexpected physical end %d, wanted %d\n", $0, end, exp_end);
 }'
 
 # Now punch out a range that is slightly larger than the size of the data
@@ -132,14 +154,27 @@ expected_offset="$((punch_rtx * rtextsize / 512))"
 
 echo "rtfile should have physical extent from $expected_offset to $expected_end sectors" >> $seqres.full
 
-# Make sure that the realtime file now has only one extent at the end of the
-# realtime device
-rtfile_exts | $AWK_PROG -v offset=$expected_offset -v end=$expected_end '
+# Make sure that the realtime file now maps one large extent at the end of the
+# realtime device.  Due to rtgroups boundary rules, there may be multiple
+# contiguous mappings.
+rtfile_exts | $AWK_PROG -v exp_start=$expected_offset -v exp_end=$expected_end '
+BEGIN {
+	start = -1;
+	end = -1;
+}
 {
-	if ($3 != offset)
-		printf("%s: unexpected physical offset %s, wanted %d\n", $0, $3, offset);
-	if ($4 != end)
-		printf("%s: unexpected physical end %s, wanted %d\n", $0, $4, end);
+	if (end >= 0 && ($3 != end + 1))
+		printf("%s: non-contiguous allocation should have started at %s\n", $0, end + 1);
+	if (start < 0 || $3 < start)
+		start = $3;
+	if (end < 0 || $4 > end)
+		end = $4;
+}
+END {
+	if (start < exp_start)
+		printf("%s: unexpected physical offset %d, wanted >= %d\n", $0, start, exp_start);
+	if (end != exp_end)
+		printf("%s: unexpected physical end %d, wanted %d\n", $0, end, exp_end);
 }'
 
 $XFS_IO_PROG -c 'bmap -elpv' $rtfile >> $seqres.full

[PATCH 10/14] xfs/449: update test to know about xfs_db -R

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The realtime groups feature added a -R flag to xfs_db so that users can
pass in the realtime device.  Since we've now modified the
_scratch_xfs_db to use this facility, we can update the test to do exact
comparisons of the xfs_db info command against the mkfs output.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/449 |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)


diff --git a/tests/xfs/449 b/tests/xfs/449
index 3d528e03a483fb..a739df50e319c5 100755
--- a/tests/xfs/449
+++ b/tests/xfs/449
@@ -30,7 +30,11 @@ echo DB >> $seqres.full
 cat $tmp.dbinfo >> $seqres.full
 # xfs_db doesn't take a rtdev argument, so it reports "realtime=external".
 # mkfs does, so make a quick substitution
-diff -u <(cat $tmp.mkfs | sed -e 's/realtime =\/.*extsz=/realtime =external               extsz=/g') $tmp.dbinfo
+if $XFS_DB_PROG --help 2>&1 | grep -q -- '-R rtdev'; then
+	diff -u $tmp.mkfs $tmp.dbinfo
+else
+	diff -u <(cat $tmp.mkfs | sed -e 's/realtime =\/.*extsz=/realtime =external               extsz=/g') $tmp.dbinfo
+fi
 
 _scratch_mount
 

[PATCH 11/14] xfs/271,xfs/556: fix tests to deal with rtgroups output in bmap/fsmap commands

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix these tests to deal with the xfs_io bmap and fsmap commands printing
out realtime group numbers if the feature is enabled.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/xfs    |    7 +++++++
 tests/xfs/271 |    4 +++-
 tests/xfs/556 |   16 ++++++++++------
 3 files changed, 20 insertions(+), 7 deletions(-)


diff --git a/common/xfs b/common/xfs
index 6e2070b5647b38..744785279df97b 100644
--- a/common/xfs
+++ b/common/xfs
@@ -419,6 +419,13 @@ _xfs_has_feature()
 		feat="rtextents"
 		feat_regex="[1-9][0-9]*"
 		;;
+	"rtgroups")
+		# any fs with rtgroups enabled will have a nonzero rt group
+		# size, even if there is no rt device (and hence zero actual
+		# groups)
+		feat="rgsize"
+		feat_regex="[1-9][0-9]*"
+		;;
 	esac
 
 	local answer="$($XFS_INFO_PROG "$fs" 2>&1 | grep -E -w -c "$feat=$feat_regex")"
diff --git a/tests/xfs/271 b/tests/xfs/271
index 420f4e7479220a..8a71746d6eaede 100755
--- a/tests/xfs/271
+++ b/tests/xfs/271
@@ -29,6 +29,8 @@ _scratch_mkfs > "$seqres.full" 2>&1
 _scratch_mount
 
 agcount=$(_xfs_mount_agcount $SCRATCH_MNT)
+agcount_wiggle=0
+_xfs_has_feature $SCRATCH_MNT rtgroups && agcount_wiggle=1
 
 # mkfs lays out btree root blocks in the order bnobt, cntbt, inobt, finobt,
 # rmapbt, refcountbt, and then allocates AGFL blocks.  Since GETFSMAP has the
@@ -46,7 +48,7 @@ cat $TEST_DIR/fsmap >> $seqres.full
 
 echo "Check AG header" | tee -a $seqres.full
 grep 'static fs metadata[[:space:]]*[0-9]*[[:space:]]*(0\.\.' $TEST_DIR/fsmap | tee -a $seqres.full > $TEST_DIR/testout
-_within_tolerance "AG header count" $(wc -l < $TEST_DIR/testout) $agcount 0 -v
+_within_tolerance "AG header count" $(wc -l < $TEST_DIR/testout) $agcount $agcount_wiggle -v
 
 echo "Check freesp/rmap btrees" | tee -a $seqres.full
 grep 'per-AG metadata[[:space:]]*[0-9]*[[:space:]]*([0-9]*\.\.' $TEST_DIR/fsmap | tee -a $seqres.full > $TEST_DIR/testout
diff --git a/tests/xfs/556 b/tests/xfs/556
index 79e03caf40a0a5..83d5022e700c8b 100755
--- a/tests/xfs/556
+++ b/tests/xfs/556
@@ -45,16 +45,20 @@ victim=$SCRATCH_MNT/a
 file_blksz=$(_get_file_block_size $SCRATCH_MNT)
 $XFS_IO_PROG -f -c "pwrite -S 0x58 0 $((4 * file_blksz))" -c "fsync" $victim >> $seqres.full
 unset errordev
-_xfs_is_realtime_file $victim && errordev="RT"
+
+awk_len_prog='{print $6}'
+if _xfs_is_realtime_file $victim; then
+	if ! _xfs_has_feature $SCRATCH_MNT rtgroups; then
+		awk_len_prog='{print $4}'
+	fi
+	errordev="RT"
+fi
 bmap_str="$($XFS_IO_PROG -c "bmap -elpv" $victim | grep "^[[:space:]]*0:")"
 echo "$errordev:$bmap_str" >> $seqres.full
 
 phys="$(echo "$bmap_str" | $AWK_PROG '{print $3}')"
-if [ "$errordev" = "RT" ]; then
-	len="$(echo "$bmap_str" | $AWK_PROG '{print $4}')"
-else
-	len="$(echo "$bmap_str" | $AWK_PROG '{print $6}')"
-fi
+len="$(echo "$bmap_str" | $AWK_PROG "$awk_len_prog")"
+
 fs_blksz=$(_get_block_size $SCRATCH_MNT)
 echo "file_blksz:$file_blksz:fs_blksz:$fs_blksz" >> $seqres.full
 kernel_sectors_per_fs_block=$((fs_blksz / 512))

[PATCH 12/14] common/xfs: capture realtime devices during metadump/mdrestore

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

If xfs_metadump supports the -r switch to capture the contents of
realtime devices and there is a realtime device, add the option to the
command line to enable preservation.

Similarly, if the dump file could restore to an external scratch rtdev,
pass the -r option to mdrestore so that we can restore rtdev contents.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/metadump |   22 ++++++++++++++++++----
 common/populate |    6 +++++-
 common/xfs      |   48 ++++++++++++++++++++++++++++++++++++++++--------
 3 files changed, 63 insertions(+), 13 deletions(-)


diff --git a/common/metadump b/common/metadump
index a4ec9a7f921acf..61ba3cbb91647c 100644
--- a/common/metadump
+++ b/common/metadump
@@ -27,6 +27,7 @@ _xfs_cleanup_verify_metadump()
 	if [ -n "$XFS_METADUMP_IMG" ]; then
 		[ -b "$METADUMP_DATA_LOOP_DEV" ] && _destroy_loop_device $METADUMP_DATA_LOOP_DEV
 		[ -b "$METADUMP_LOG_LOOP_DEV" ] && _destroy_loop_device $METADUMP_LOG_LOOP_DEV
+		[ -b "$METADUMP_RT_LOOP_DEV" ] && _destroy_loop_device $METADUMP_RT_LOOP_DEV
 		for img in "$XFS_METADUMP_IMG"*; do
 			test -e "$img" && rm -f "$img"
 		done
@@ -101,6 +102,7 @@ _xfs_verify_metadump_v2()
 	local version="-v 2"
 	local data_img="$XFS_METADUMP_IMG.data"
 	local log_img=""
+	local rt_img=""
 
 	# Capture metadump, which creates metadump_file
 	_scratch_xfs_metadump $metadump_file $metadump_args $version
@@ -111,8 +113,12 @@ _xfs_verify_metadump_v2()
 	# from such a metadump file.
 	test -n "$SCRATCH_LOGDEV" && log_img="$XFS_METADUMP_IMG.log"
 
+	# Use a temporary file to hold restored rt device contents
+	test -n "$SCRATCH_RTDEV" && _xfs_metadump_supports_rt && \
+		rt_img="$XFS_METADUMP_IMG.rt"
+
 	# Restore metadump, which creates data_img and log_img
-	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img \
+	SCRATCH_DEV=$data_img SCRATCH_LOGDEV=$log_img SCRATCH_RTDEV=$rt_img \
 		_scratch_xfs_mdrestore $metadump_file
 
 	# Create loopdev for data device so we can mount the fs
@@ -121,12 +127,15 @@ _xfs_verify_metadump_v2()
 	# Create loopdev for log device if we recovered anything
 	test -s "$log_img" && METADUMP_LOG_LOOP_DEV=$(_create_loop_device $log_img)
 
+	# Create loopdev for rt device if we recovered anything
+	test -s "$rt_img" && METADUMP_RT_LOOP_DEV=$(_create_loop_device $rt_img)
+
 	# Mount fs, run an extra test, fsck, and unmount
-	SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV _scratch_mount
+	SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV SCRATCH_RTDEV=$METADUMP_RT_LOOP_DEV _scratch_mount
 	if [ -n "$extra_test" ]; then
-		SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV $extra_test
+		SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV SCRATCH_RTDEV=$METADUMP_RT_LOOP_DEV $extra_test
 	fi
-	SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV _check_xfs_scratch_fs
+	SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV SCRATCH_RTDEV=$METADUMP_RT_LOOP_DEV _check_xfs_scratch_fs
 	_unmount $METADUMP_DATA_LOOP_DEV
 
 	# Tear down what we created
@@ -135,6 +144,11 @@ _xfs_verify_metadump_v2()
 		unset METADUMP_LOG_LOOP_DEV
 		rm -f $log_img
 	fi
+	if [ -b "$METADUMP_RT_LOOP_DEV" ]; then
+		_destroy_loop_device $METADUMP_RT_LOOP_DEV
+		unset METADUMP_RT_LOOP_DEV
+		rm -f $rt_img
+	fi
 	_destroy_loop_device $METADUMP_DATA_LOOP_DEV
 	unset METADUMP_DATA_LOOP_DEV
 	rm -f $data_img
diff --git a/common/populate b/common/populate
index 96fc13875df503..9c3d49efebb3a4 100644
--- a/common/populate
+++ b/common/populate
@@ -1048,12 +1048,16 @@ _scratch_populate_save_metadump()
 		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
 			logdev=$SCRATCH_LOGDEV
 
+		local rtdev=none
+		[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
+			rtdev=$SCRATCH_RTDEV
+
 		mdargs=('-a' '-o')
 		test "$(_xfs_metadump_max_version)" -gt 1 && \
 			mdargs+=('-v' '2')
 
 		_xfs_metadump "$metadump_file" "$SCRATCH_DEV" "$logdev" \
-				compress "${mdargs[@]}"
+				"$rtdev" compress "${mdargs[@]}"
 		res=$?
 		;;
 	"ext2"|"ext3"|"ext4")
diff --git a/common/xfs b/common/xfs
index 744785279df97b..81b080b7d8088f 100644
--- a/common/xfs
+++ b/common/xfs
@@ -625,14 +625,19 @@ _xfs_metadump() {
 	local metadump="$1"
 	local device="$2"
 	local logdev="$3"
-	local compressopt="$4"
-	shift; shift; shift; shift
+	local rtdev="$4"
+	local compressopt="$5"
+	shift; shift; shift; shift; shift
 	local options="$@"
 
 	if [ "$logdev" != "none" ]; then
 		options="$options -l $logdev"
 	fi
 
+	if [ "$rtdev" != "none" ] && _xfs_metadump_supports_rt; then
+		options="$options -r $rtdev"
+	fi
+
 	$XFS_METADUMP_PROG $options "$device" "$metadump"
 	res=$?
 	[ "$compressopt" = "compress" ] && [ -n "$DUMP_COMPRESSOR" ] &&
@@ -656,7 +661,8 @@ _xfs_mdrestore() {
 	local metadump="$1"
 	local device="$2"
 	local logdev="$3"
-	shift; shift; shift
+	local rtdev="$4"
+	shift; shift; shift; shift
 	local options="$@"
 	local dumpfile_ver
 
@@ -684,6 +690,18 @@ _xfs_mdrestore() {
 		options="$options -l $logdev"
 	fi
 
+	if [ "$rtdev" != "none" ] && [[ $dumpfile_ver > 1 ]] && _xfs_metadump_supports_rt; then
+		# metadump and mdrestore capture and restore metadata on the
+		# realtime volume by turning on metadump v2 format.  This is
+		# only done if the realtime volume contains metadata such as
+		# rtgroup superblocks.  The -r option to mdrestore wasn't added
+		# until the creation of rtgroups.
+		#
+		# Hence it only makes sense to specify -r here if the dump file
+		# itself is in v2 format.
+		options="$options -r $rtdev"
+	fi
+
 	$XFS_MDRESTORE_PROG $options "${metadump}" "${device}"
 }
 
@@ -697,17 +715,27 @@ _xfs_metadump_max_version()
 	fi
 }
 
+# Do xfs_metadump/mdrestore support the -r switch for realtime devices?
+_xfs_metadump_supports_rt()
+{
+	$XFS_METADUMP_PROG --help 2>&1 | grep -q -- '-r rtdev'
+}
+
 # Snapshot the metadata on the scratch device
 _scratch_xfs_metadump()
 {
 	local metadump=$1
 	shift
 	local logdev=none
+	local rtdev=none
 
 	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
 		logdev=$SCRATCH_LOGDEV
 
-	_xfs_metadump "$metadump" "$SCRATCH_DEV" "$logdev" nocompress "$@"
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
+		rtdev=$SCRATCH_RTDEV
+
+	_xfs_metadump "$metadump" "$SCRATCH_DEV" "$logdev" "$rtdev" nocompress "$@"
 }
 
 # Restore snapshotted metadata on the scratch device
@@ -716,6 +744,7 @@ _scratch_xfs_mdrestore()
 	local metadump=$1
 	shift
 	local logdev=none
+	local rtdev=none
 	local options="$@"
 
 	# $SCRATCH_LOGDEV should have a non-zero length value only when all of
@@ -726,7 +755,10 @@ _scratch_xfs_mdrestore()
 		logdev=$SCRATCH_LOGDEV
 	fi
 
-	_xfs_mdrestore "$metadump" "$SCRATCH_DEV" "$logdev" "$@"
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ] && \
+		rtdev=$SCRATCH_RTDEV
+
+	_xfs_mdrestore "$metadump" "$SCRATCH_DEV" "$logdev" "$rtdev" "$@"
 }
 
 # Do not use xfs_repair (offline fsck) to rebuild the filesystem
@@ -847,7 +879,7 @@ _check_xfs_filesystem()
 	if [ "$ok" -ne 1 ] && [ "$DUMP_CORRUPT_FS" = "1" ]; then
 		local flatdev="$(basename "$device")"
 		_xfs_metadump "$seqres.$flatdev.check.md" "$device" "$logdev" \
-			compress -a -o >> $seqres.full
+			"$rtdev" compress -a -o >> $seqres.full
 	fi
 
 	# Optionally test the index rebuilding behavior.
@@ -880,7 +912,7 @@ _check_xfs_filesystem()
 		if [ "$rebuild_ok" -ne 1 ] && [ "$DUMP_CORRUPT_FS" = "1" ]; then
 			local flatdev="$(basename "$device")"
 			_xfs_metadump "$seqres.$flatdev.rebuild.md" "$device" \
-				"$logdev" compress -a -o >> $seqres.full
+				"$logdev" "$rtdev" compress -a -o >> $seqres.full
 		fi
 	fi
 
@@ -964,7 +996,7 @@ _check_xfs_filesystem()
 		if [ "$orebuild_ok" -ne 1 ] && [ "$DUMP_CORRUPT_FS" = "1" ]; then
 			local flatdev="$(basename "$device")"
 			_xfs_metadump "$seqres.$flatdev.orebuild.md" "$device" \
-				"$logdev" compress -a -o >> $seqres.full
+				"$logdev" "$rtdev" compress -a -o >> $seqres.full
 		fi
 	fi
 

[PATCH 13/14] common/fuzzy: adapt the scrub stress tests to support rtgroups

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Adapt the scrub stress testing framework to support checking realtime
group metadata.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/fuzzy  |   27 ++++++++++++++++++++++-----
 common/xfs    |    9 +++++++++
 tests/xfs/581 |    2 +-
 tests/xfs/720 |    2 +-
 tests/xfs/795 |    2 +-
 5 files changed, 34 insertions(+), 8 deletions(-)


diff --git a/common/fuzzy b/common/fuzzy
index a7e28dda137e8a..ad28ba9275a09f 100644
--- a/common/fuzzy
+++ b/common/fuzzy
@@ -829,8 +829,10 @@ __stress_one_scrub_loop() {
 	local scrub_tgt="$3"
 	local scrub_startat="$4"
 	local start_agno="$5"
-	shift; shift; shift; shift; shift
+	local start_rgno="$6"
+	shift; shift; shift; shift; shift; shift
 	local agcount="$(_xfs_mount_agcount $SCRATCH_MNT)"
+	local rgcount="$(_xfs_mount_rgcount $SCRATCH_MNT)"
 
 	local xfs_io_args=()
 	for arg in "$@"; do
@@ -843,6 +845,12 @@ __stress_one_scrub_loop() {
 				local ag_arg="$(echo "$arg" | sed -e "s|%agno%|$agno|g")"
 				xfs_io_args+=('-c' "$ag_arg")
 			done
+		elif echo "$arg" | grep -q -w '%rgno%'; then
+			# Substitute the rtgroup number
+			for ((rgno = start_rgno; rgno < rgcount; rgno++)); do
+				local rg_arg="$(echo "$arg" | sed -e "s|%rgno%|$rgno|g")"
+				xfs_io_args+=('-c' "$rg_arg")
+			done
 		else
 			xfs_io_args+=('-c' "$arg")
 		fi
@@ -1273,7 +1281,9 @@ _scratch_xfs_stress_scrub_cleanup() {
 __stress_scrub_check_commands() {
 	local scrub_tgt="$1"
 	local start_agno="$2"
-	shift; shift
+	local start_rgno="$3"
+	shift; shift; shift
+	local rgcount="$(_xfs_mount_rgcount $SCRATCH_MNT)"
 
 	local cooked_tgt="$scrub_tgt"
 	case "$scrub_tgt" in
@@ -1303,6 +1313,10 @@ __stress_scrub_check_commands() {
 			cooked_arg="$(echo "$cooked_arg" | sed -e 's/^repair/repair -R/g')"
 		fi
 		cooked_arg="$(echo "$cooked_arg" | sed -e "s/%agno%/$start_agno/g")"
+		if echo "$cooked_arg" | grep -q -w '%rgno%'; then
+			test "$rgcount" -eq 0 && continue
+			cooked_arg="$(echo "$cooked_arg" | sed -e "s/%rgno%/$start_rgno/g")"
+		fi
 		testio=`$XFS_IO_PROG -x -c "$cooked_arg" "$cooked_tgt" 2>&1`
 		echo $testio | grep -q "Unknown type" && \
 			_notrun "xfs_io scrub subcommand support is missing"
@@ -1328,6 +1342,7 @@ __stress_scrub_check_commands() {
 #	in a separate loop.  If zero -i options are specified, do not run.
 #	Callers must check each of these commands (via _require_xfs_io_command)
 #	before calling here.
+# -R	For %rgno% substitution, start with this rtgroup instead of rtgroup 0.
 # -r	Run fsstress for this amount of time, then remount the fs ro or rw.
 #	The default is to run fsstress continuously with no remount, unless
 #	XFS_SCRUB_STRESS_REMOUNT_PERIOD is set.
@@ -1374,6 +1389,7 @@ _scratch_xfs_stress_scrub() {
 	local remount_period="${XFS_SCRUB_STRESS_REMOUNT_PERIOD}"
 	local stress_tgt="${XFS_SCRUB_STRESS_TARGET:-default}"
 	local start_agno=0
+	local start_rgno=0
 
 	__SCRUB_STRESS_FREEZE_PID=""
 	__SCRUB_STRESS_REMOUNT_LOOP=""
@@ -1381,12 +1397,13 @@ _scratch_xfs_stress_scrub() {
 	touch "$runningfile"
 
 	OPTIND=1
-	while getopts "a:fi:r:s:S:t:w:x:X:" c; do
+	while getopts "a:fi:r:R:s:S:t:w:x:X:" c; do
 		case "$c" in
 			a) start_agno="$OPTARG";;
 			f) freeze=yes;;
 			i) io_args+=("$OPTARG");;
 			r) remount_period="$OPTARG";;
+			R) start_rgno="$OPTARG";;
 			s) one_scrub_args+=("$OPTARG");;
 			S) xfs_scrub_args+=("$OPTARG");;
 			t) scrub_tgt="$OPTARG";;
@@ -1397,7 +1414,7 @@ _scratch_xfs_stress_scrub() {
 		esac
 	done
 
-	__stress_scrub_check_commands "$scrub_tgt" "$start_agno" \
+	__stress_scrub_check_commands "$scrub_tgt" "$start_agno" "$start_rgno" \
 			"${one_scrub_args[@]}"
 
 	if ! command -v "__stress_scrub_${exerciser}_loop" &>/dev/null; then
@@ -1458,7 +1475,7 @@ _scratch_xfs_stress_scrub() {
 
 	if [ "${#one_scrub_args[@]}" -gt 0 ]; then
 		__stress_one_scrub_loop "$end" "$runningfile" "$scrub_tgt" \
-				"$scrub_startat" "$start_agno" \
+				"$scrub_startat" "$start_agno" "$start_rgno" \
 				"${one_scrub_args[@]}" &
 	fi
 
diff --git a/common/xfs b/common/xfs
index 81b080b7d8088f..49fb2ea7e7894e 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1496,6 +1496,15 @@ _xfs_mount_agcount()
 	$XFS_INFO_PROG "$1" | sed -n "s/^.*agcount=\([[:digit:]]*\).*/\1/p"
 }
 
+# Find rtgroup count of mounted filesystem
+_xfs_mount_rgcount()
+{
+	local rtgroups="$($XFS_INFO_PROG "$1" | grep rgcount= | sed -e 's/^.*rgcount=\([0-9]*\).*$/\1/g')"
+
+	test -z "$rtgroups" && rtgroups=0
+	echo "$rtgroups"
+}
+
 # Wipe the superblock of each XFS AGs
 _try_wipe_scratch_xfs()
 {
diff --git a/tests/xfs/581 b/tests/xfs/581
index 39eb42da4b10c5..6aa360b37b90c5 100755
--- a/tests/xfs/581
+++ b/tests/xfs/581
@@ -30,7 +30,7 @@ _require_xfs_stress_scrub
 _scratch_mkfs > "$seqres.full" 2>&1
 _scratch_mount
 _require_xfs_has_feature "$SCRATCH_MNT" realtime
-_scratch_xfs_stress_scrub -s "scrub rtbitmap"
+_scratch_xfs_stress_scrub -s "scrub rtbitmap"  -s "scrub rgbitmap %rgno%"
 
 # success, all done
 echo Silence is golden
diff --git a/tests/xfs/720 b/tests/xfs/720
index 68a6c7f6e2d584..97b3d2579cbd7f 100755
--- a/tests/xfs/720
+++ b/tests/xfs/720
@@ -37,7 +37,7 @@ alloc_unit=$(_get_file_block_size $SCRATCH_MNT)
 scratchfile=$SCRATCH_MNT/file
 touch $scratchfile
 $XFS_IO_PROG -x -c 'inject force_repair' $SCRATCH_MNT
-__stress_scrub_check_commands "$scratchfile" "" 'repair bmapbtd'
+__stress_scrub_check_commands "$scratchfile" "" "" 'repair bmapbtd'
 
 # Compute the number of extent records needed to guarantee btree format,
 # assuming 16 bytes for each ondisk extent record
diff --git a/tests/xfs/795 b/tests/xfs/795
index 217f96092a4c42..e7004705b526a5 100755
--- a/tests/xfs/795
+++ b/tests/xfs/795
@@ -37,7 +37,7 @@ scratchfile=$SCRATCH_MNT/file
 mkdir $scratchdir
 touch $scratchfile
 $XFS_IO_PROG -x -c 'inject force_repair' $SCRATCH_MNT
-__stress_scrub_check_commands "$scratchdir" "" 'repair directory'
+__stress_scrub_check_commands "$scratchdir" "" "" 'repair directory'
 
 # Create a 2-dirblock directory
 total_size=$((alloc_unit * 2))

[PATCH 14/14] xfs: fix fuzz tests of rtgroups bitmap and summary files

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

With rtgroups, the rt bitmap and summary files are now per-group, so
adjust the fuzz and fsck tests to find the new locations.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/xfs    |   19 +++++++++++++++++++
 tests/xfs/581 |    9 ++++++++-
 tests/xfs/582 |   14 +++++++-------
 tests/xfs/739 |    6 +++++-
 tests/xfs/740 |    6 +++++-
 tests/xfs/741 |    6 +++++-
 tests/xfs/742 |    6 +++++-
 tests/xfs/743 |    6 +++++-
 tests/xfs/744 |    6 +++++-
 tests/xfs/745 |    6 +++++-
 tests/xfs/746 |    6 +++++-
 tests/xfs/793 |   14 +++++++-------
 12 files changed, 81 insertions(+), 23 deletions(-)


diff --git a/common/xfs b/common/xfs
index 49fb2ea7e7894e..aa635f8b4745fa 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1956,6 +1956,25 @@ _scratch_xfs_find_metafile()
 	local metafile="$1"
 	local sb_field
 
+	# With metadir=1, the realtime volume is sharded into allocation
+	# groups.  Each rtgroup has its own bitmap and summary file.  Tests
+	# should pick a particular file, but this compatibility shim still
+	# exists to keep old tests working.
+	case "$metafile" in
+	"rbmino")
+		if _xfs_has_feature "$SCRATCH_DEV" rtgroups; then
+			echo "path -m /rtgroups/0.bitmap"
+			return 0
+		fi
+		;;
+	"rsumino")
+		if _xfs_has_feature "$SCRATCH_DEV" rtgroups; then
+			echo "path -m /rtgroups/0.summary"
+			return 0
+		fi
+		;;
+	esac
+
 	sb_field="$(_scratch_xfs_get_sb_field "$metafile")"
 	if echo "$sb_field" | grep -q -w 'not found'; then
 		return 1
diff --git a/tests/xfs/581 b/tests/xfs/581
index 6aa360b37b90c5..7d79dbcad70149 100755
--- a/tests/xfs/581
+++ b/tests/xfs/581
@@ -30,7 +30,14 @@ _require_xfs_stress_scrub
 _scratch_mkfs > "$seqres.full" 2>&1
 _scratch_mount
 _require_xfs_has_feature "$SCRATCH_MNT" realtime
-_scratch_xfs_stress_scrub -s "scrub rtbitmap"  -s "scrub rgbitmap %rgno%"
+
+if _xfs_has_feature "$SCRATCH_MNT" rtgroups; then
+	_scratch_xfs_stress_scrub -s "scrub rtbitmap %rgno%"
+elif xfs_io -c 'help scrub' | grep -q rgsuper; then
+	_scratch_xfs_stress_scrub -s "scrub rtbitmap 0"
+else
+	_scratch_xfs_stress_scrub -s "scrub rtbitmap"
+fi
 
 # success, all done
 echo Silence is golden
diff --git a/tests/xfs/582 b/tests/xfs/582
index e92f128f8a5695..a2cc58c04bf8d2 100755
--- a/tests/xfs/582
+++ b/tests/xfs/582
@@ -31,13 +31,13 @@ _scratch_mkfs > "$seqres.full" 2>&1
 _scratch_mount
 _require_xfs_has_feature "$SCRATCH_MNT" realtime
 
-# XXX the realtime summary scrubber isn't currently implemented upstream.
-# Don't bother trying to test it on those kernels
-$XFS_IO_PROG -c 'scrub rtsummary' -c 'scrub rtsummary' "$SCRATCH_MNT" 2>&1 | \
-	grep -q 'Scan was not complete' && \
-	_notrun "rtsummary scrub is incomplete"
-
-_scratch_xfs_stress_scrub -s "scrub rtsummary"
+if _xfs_has_feature "$SCRATCH_MNT" rtgroups; then
+	_scratch_xfs_stress_scrub -s "scrub rtsummary %rgno%"
+elif xfs_io -c 'help scrub' | grep -q rgsuper; then
+	_scratch_xfs_stress_scrub -s "scrub rtsummary 0"
+else
+	_scratch_xfs_stress_scrub -s "scrub rtsummary"
+fi
 
 # success, all done
 echo Silence is golden
diff --git a/tests/xfs/739 b/tests/xfs/739
index 5fd6caa5bce2f8..77cceac7ac7f02 100755
--- a/tests/xfs/739
+++ b/tests/xfs/739
@@ -25,7 +25,11 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-path="$(_scratch_xfs_find_metafile rbmino)"
+if _xfs_has_feature "$SCRATCH_DEV" rtgroups; then
+	path="path -m /rtgroups/0.bitmap"
+else
+	path="$(_scratch_xfs_find_metafile rbmino)"
+fi
 _scratch_xfs_fuzz_metadata '' 'online' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
diff --git a/tests/xfs/740 b/tests/xfs/740
index c8990034773b32..fe5d054956d5e2 100755
--- a/tests/xfs/740
+++ b/tests/xfs/740
@@ -25,7 +25,11 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-path="$(_scratch_xfs_find_metafile rsumino)"
+if _xfs_has_feature "$SCRATCH_DEV" rtgroups; then
+	path="path -m /rtgroups/0.summary"
+else
+	path="$(_scratch_xfs_find_metafile rsumino)"
+fi
 _scratch_xfs_fuzz_metadata '' 'online' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
diff --git a/tests/xfs/741 b/tests/xfs/741
index 96c2315c524311..35f78aab7faead 100755
--- a/tests/xfs/741
+++ b/tests/xfs/741
@@ -25,7 +25,11 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-path="$(_scratch_xfs_find_metafile rbmino)"
+if _xfs_has_feature "$SCRATCH_DEV" rtgroups; then
+	path="path -m /rtgroups/0.bitmap"
+else
+	path="$(_scratch_xfs_find_metafile rbmino)"
+fi
 _scratch_xfs_fuzz_metadata '' 'offline' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
diff --git a/tests/xfs/742 b/tests/xfs/742
index 301ae7b9574320..04087c1a224558 100755
--- a/tests/xfs/742
+++ b/tests/xfs/742
@@ -25,7 +25,11 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-path="$(_scratch_xfs_find_metafile rsumino)"
+if _xfs_has_feature "$SCRATCH_DEV" rtgroups; then
+	path="path -m /rtgroups/0.summary"
+else
+	path="$(_scratch_xfs_find_metafile rsumino)"
+fi
 _scratch_xfs_fuzz_metadata '' 'offline' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
diff --git a/tests/xfs/743 b/tests/xfs/743
index 039624f711c0a6..1e70147fa3bcef 100755
--- a/tests/xfs/743
+++ b/tests/xfs/743
@@ -26,7 +26,11 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-path="$(_scratch_xfs_find_metafile rbmino)"
+if _xfs_has_feature "$SCRATCH_DEV" rtgroups; then
+	path="path -m /rtgroups/0.bitmap"
+else
+	path="$(_scratch_xfs_find_metafile rbmino)"
+fi
 _scratch_xfs_fuzz_metadata '' 'both' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
diff --git a/tests/xfs/744 b/tests/xfs/744
index 13f63b9ceb1756..0db3dd617c2d1f 100755
--- a/tests/xfs/744
+++ b/tests/xfs/744
@@ -26,7 +26,11 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-path="$(_scratch_xfs_find_metafile rsumino)"
+if _xfs_has_feature "$SCRATCH_DEV" rtgroups; then
+	path="path -m /rtgroups/0.summary"
+else
+	path="$(_scratch_xfs_find_metafile rsumino)"
+fi
 _scratch_xfs_fuzz_metadata '' 'both' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
diff --git a/tests/xfs/745 b/tests/xfs/745
index 56a6d58ef9f4ca..acfbe72597fd5a 100755
--- a/tests/xfs/745
+++ b/tests/xfs/745
@@ -25,7 +25,11 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtbitmap"
-path="$(_scratch_xfs_find_metafile rbmino)"
+if _xfs_has_feature "$SCRATCH_DEV" rtgroups; then
+	path="path -m /rtgroups/0.bitmap"
+else
+	path="$(_scratch_xfs_find_metafile rbmino)"
+fi
 _scratch_xfs_fuzz_metadata '' 'none' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtbitmap"
 
diff --git a/tests/xfs/746 b/tests/xfs/746
index 935b2e5acba5d4..0d1ab895aacd62 100755
--- a/tests/xfs/746
+++ b/tests/xfs/746
@@ -25,7 +25,11 @@ echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
 
 echo "Fuzz rtsummary"
-path="$(_scratch_xfs_find_metafile rsumino)"
+if _xfs_has_feature "$SCRATCH_DEV" rtgroups; then
+	path="path -m /rtgroups/0.summary"
+else
+	path="$(_scratch_xfs_find_metafile rsumino)"
+fi
 _scratch_xfs_fuzz_metadata '' 'none' "$path" 'dblock 0' >> $seqres.full
 echo "Done fuzzing rtsummary"
 
diff --git a/tests/xfs/793 b/tests/xfs/793
index a779bf81738537..07c64e7a3f9744 100755
--- a/tests/xfs/793
+++ b/tests/xfs/793
@@ -32,13 +32,13 @@ _scratch_mount
 _require_xfs_has_feature "$SCRATCH_MNT" realtime
 _xfs_force_bdev realtime $SCRATCH_MNT
 
-# XXX the realtime summary scrubber isn't currently implemented upstream.
-# Don't bother trying to fix it on those kernels
-$XFS_IO_PROG -c 'scrub rtsummary' -c 'scrub rtsummary' "$SCRATCH_MNT" 2>&1 | \
-	grep -q 'Scan was not complete' && \
-	_notrun "rtsummary scrub is incomplete"
-
-_scratch_xfs_stress_online_repair -s "repair rtsummary"
+if _xfs_has_feature "$SCRATCH_MNT" rtgroups; then
+	_scratch_xfs_stress_online_repair -s "repair rtsummary %rgno%"
+elif xfs_io -c 'help scrub' | grep -q rgsuper; then
+	_scratch_xfs_stress_online_repair -s "repair rtsummary 0"
+else
+	_scratch_xfs_stress_online_repair -s "repair rtsummary"
+fi
 
 # success, all done
 echo Silence is golden

[PATCH 1/4] xfs: update tests for quota files in the metadir

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Update fstests to handle quota files in the metadir.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/xfs |   18 ++++++++++++++++++
 1 file changed, 18 insertions(+)


diff --git a/common/xfs b/common/xfs
index aa635f8b4745fa..0f29d5c5a6b963 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1973,6 +1973,24 @@ _scratch_xfs_find_metafile()
 			return 0
 		fi
 		;;
+	"uquotino")
+		if _xfs_has_feature "$SCRATCH_DEV" metadir; then
+			echo "path -m /quota/user"
+			return 0
+		fi
+		;;
+	"gquotino")
+		if _xfs_has_feature "$SCRATCH_DEV" metadir; then
+			echo "path -m /quota/group"
+			return 0
+		fi
+		;;
+	"pquotino")
+		if _xfs_has_feature "$SCRATCH_DEV" metadir; then
+			echo "path -m /quota/project"
+			return 0
+		fi
+		;;
 	esac
 
 	sb_field="$(_scratch_xfs_get_sb_field "$metafile")"

[PATCH 2/4] xfs: test persistent quota flags

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Test the persistent quota flags that come with the metadir feature.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/generic/563  |    8 ++-
 tests/xfs/1891     |  128 +++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1891.out |  147 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 282 insertions(+), 1 deletion(-)
 create mode 100755 tests/xfs/1891
 create mode 100644 tests/xfs/1891.out


diff --git a/tests/generic/563 b/tests/generic/563
index 95a928fba5627e..89a71aa44938ea 100755
--- a/tests/generic/563
+++ b/tests/generic/563
@@ -96,7 +96,13 @@ smajor=$((0x`stat -L -c %t $loop_dev`))
 sminor=$((0x`stat -L -c %T $loop_dev`))
 
 _mkfs_dev $loop_dev >> $seqres.full 2>&1
-_mount $loop_dev $SCRATCH_MNT || _fail "mount failed"
+if [ $FSTYP = "xfs" ]; then
+	# Writes to the quota file are captured in cgroup metrics on XFS, so
+	# we require that quota is not enabled at all.
+	_mount $loop_dev -o noquota $SCRATCH_MNT || _fail "mount failed"
+else
+	_mount $loop_dev $SCRATCH_MNT || _fail "mount failed"
+fi
 
 blksize=$(_get_block_size "$SCRATCH_MNT")
 
diff --git a/tests/xfs/1891 b/tests/xfs/1891
new file mode 100755
index 00000000000000..7db94e0976527e
--- /dev/null
+++ b/tests/xfs/1891
@@ -0,0 +1,128 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2024-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test 1891
+#
+# Functionality test for persistent quota accounting and enforcement flags in
+# XFS when metadata directories are enabled.
+#
+. ./common/preamble
+_begin_fstest auto quick quota
+
+. ./common/filter
+. ./common/quota
+
+$MKFS_XFS_PROG 2>&1 | grep -q 'uquota' || \
+	_notrun "mkfs does not support uquota option"
+
+_require_scratch
+_require_xfs_quota
+
+filter_quota_state() {
+	sed -e 's/Inode: #[0-9]\+/Inode #XXX/g' \
+	    -e '/max warnings:/d' \
+	    -e '/Blocks grace time:/d' \
+	    -e '/Inodes grace time:/d' \
+		| _filter_scratch
+}
+
+qerase_mkfs_options() {
+	echo "$MKFS_OPTIONS" | sed \
+		-e 's/uquota//g' \
+		-e 's/gquota//g' \
+		-e 's/pquota//g' \
+		-e 's/uqnoenforce//g' \
+		-e 's/gqnoenforce//g' \
+		-e 's/pqnoenforce//g' \
+		-e 's/,,*/,/g'
+}
+
+confirm() {
+	echo "$MOUNT_OPTIONS" | grep -E -q '(qnoenforce|quota)' && \
+		echo "saw quota mount options"
+	_scratch_mount
+	$XFS_QUOTA_PROG -x -c "state -ugp" $SCRATCH_MNT | filter_quota_state
+	_check_xfs_scratch_fs
+	_scratch_unmount
+}
+
+ORIG_MOUNT_OPTIONS="$MOUNT_OPTIONS"
+MKFS_OPTIONS="$(qerase_mkfs_options)"
+
+echo "Test 0: formatting a subset"
+_scratch_mkfs -m uquota,gqnoenforce &>> $seqres.full
+MOUNT_OPTIONS="$ORIG_MOUNT_OPTIONS"
+_qmount_option	# blank out quota options
+confirm
+
+echo "Test 1: formatting"
+_scratch_mkfs -m uquota,gquota,pquota &>> $seqres.full
+MOUNT_OPTIONS="$ORIG_MOUNT_OPTIONS"
+_qmount_option	# blank out quota options
+confirm
+
+echo "Test 2: only grpquota"
+MOUNT_OPTIONS="$ORIG_MOUNT_OPTIONS"
+_qmount_option grpquota
+confirm
+
+echo "Test 3: repair"
+_scratch_xfs_repair &>> $seqres.full || echo "repair failed?"
+MOUNT_OPTIONS="$ORIG_MOUNT_OPTIONS"
+_qmount_option	# blank out quota options
+confirm
+
+echo "Test 4: weird options"
+MOUNT_OPTIONS="$ORIG_MOUNT_OPTIONS"
+_qmount_option pqnoenforce,uquota
+confirm
+
+echo "Test 5: simple recovery"
+_scratch_mkfs -m uquota,gquota,pquota &>> $seqres.full
+MOUNT_OPTIONS="$ORIG_MOUNT_OPTIONS"
+_qmount_option	# blank out quota options
+echo "$MOUNT_OPTIONS" | grep -E -q '(qnoenforce|quota)' && \
+	echo "saw quota mount options"
+_scratch_mount
+$XFS_QUOTA_PROG -x -c "state -ugp" $SCRATCH_MNT | filter_quota_state
+touch $SCRATCH_MNT/a
+_scratch_shutdown -v -f >> $seqres.full
+echo shutdown
+_scratch_unmount
+confirm
+
+echo "Test 6: simple recovery with mount options"
+_scratch_mkfs -m uquota,gquota,pquota &>> $seqres.full
+MOUNT_OPTIONS="$ORIG_MOUNT_OPTIONS"
+_qmount_option	# blank out quota options
+echo "$MOUNT_OPTIONS" | grep -E -q '(qnoenforce|quota)' && \
+	echo "saw quota mount options"
+_scratch_mount
+$XFS_QUOTA_PROG -x -c "state -ugp" $SCRATCH_MNT | filter_quota_state
+touch $SCRATCH_MNT/a
+_scratch_shutdown -v -f >> $seqres.full
+echo shutdown
+_scratch_unmount
+MOUNT_OPTIONS="$ORIG_MOUNT_OPTIONS"
+_qmount_option gqnoenforce
+confirm
+
+echo "Test 7: user quotaoff recovery"
+_scratch_mkfs -m uquota,gquota,pquota &>> $seqres.full
+MOUNT_OPTIONS="$ORIG_MOUNT_OPTIONS"
+_qmount_option	# blank out quota options
+echo "$MOUNT_OPTIONS" | grep -E -q '(qnoenforce|quota)' && \
+	echo "saw quota mount options"
+_scratch_mount
+$XFS_QUOTA_PROG -x -c "state -ugp" $SCRATCH_MNT | filter_quota_state
+touch $SCRATCH_MNT/a
+$XFS_QUOTA_PROG -x -c 'off -u' $SCRATCH_MNT
+_scratch_shutdown -v -f >> $seqres.full
+echo shutdown
+_scratch_unmount
+confirm
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/1891.out b/tests/xfs/1891.out
new file mode 100644
index 00000000000000..7e8894088042bb
--- /dev/null
+++ b/tests/xfs/1891.out
@@ -0,0 +1,147 @@
+QA output created by 1891
+Test 0: formatting a subset
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: OFF
+  Enforcement: OFF
+  Inode: N/A
+Test 1: formatting
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Test 2: only grpquota
+saw quota mount options
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: OFF
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: OFF
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Test 3: repair
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: OFF
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: OFF
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Test 4: weird options
+saw quota mount options
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: OFF
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Test 5: simple recovery
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+shutdown
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Test 6: simple recovery with mount options
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+shutdown
+saw quota mount options
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: OFF
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: OFF
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Test 7: user quotaoff recovery
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+shutdown
+User quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: OFF
+  Inode #XXX (1 blocks, 1 extents)
+Group quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)
+Project quota state on SCRATCH_MNT (SCRATCH_DEV)
+  Accounting: ON
+  Enforcement: ON
+  Inode #XXX (1 blocks, 1 extents)

[PATCH 3/4] xfs: fix quota detection in fuzz tests

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

With metadir, quota options persist until they are changed by mount
options.  Therefore, we can set the quota flags in MKFS_OPTIONS and
needn't supply them in MOUNT_OPTIONS.  Unfortunately, this means that we
cannot grep the MOUNT_OPTIONS anymore; we must mount the fs and run
src/feature to determine if quotas are enabled.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/425 |    5 ++++-
 tests/xfs/426 |    5 ++++-
 tests/xfs/427 |    5 ++++-
 tests/xfs/428 |    5 ++++-
 tests/xfs/429 |    5 ++++-
 tests/xfs/430 |    5 ++++-
 tests/xfs/487 |    5 ++++-
 tests/xfs/488 |    5 ++++-
 tests/xfs/489 |    5 ++++-
 tests/xfs/779 |    5 ++++-
 tests/xfs/780 |    5 ++++-
 tests/xfs/781 |    5 ++++-
 12 files changed, 48 insertions(+), 12 deletions(-)


diff --git a/tests/xfs/425 b/tests/xfs/425
index 7ad53f97a6940c..77f86bcc398312 100755
--- a/tests/xfs/425
+++ b/tests/xfs/425
@@ -23,7 +23,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'usrquota' || _notrun "user quota disabled"
+
+_scratch_mount
+$here/src/feature -U $SCRATCH_DEV || _notrun "user quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/426 b/tests/xfs/426
index 53bfd0d637fcb5..80f572eb8068a2 100755
--- a/tests/xfs/426
+++ b/tests/xfs/426
@@ -23,7 +23,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'usrquota' || _notrun "user quota disabled"
+
+_scratch_mount
+$here/src/feature -U $SCRATCH_DEV || _notrun "user quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/427 b/tests/xfs/427
index 38de1360af6262..48b8d5e935abfe 100755
--- a/tests/xfs/427
+++ b/tests/xfs/427
@@ -23,7 +23,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'grpquota' || _notrun "group quota disabled"
+
+_scratch_mount
+$here/src/feature -G $SCRATCH_DEV || _notrun "group quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/428 b/tests/xfs/428
index e112ccf84646c1..f87f0a98bae3e9 100755
--- a/tests/xfs/428
+++ b/tests/xfs/428
@@ -23,7 +23,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'grpquota' || _notrun "group quota disabled"
+
+_scratch_mount
+$here/src/feature -G $SCRATCH_DEV || _notrun "group quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/429 b/tests/xfs/429
index ded8c3944a2648..426e716aa079bf 100755
--- a/tests/xfs/429
+++ b/tests/xfs/429
@@ -23,7 +23,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'prjquota' || _notrun "project quota disabled"
+
+_scratch_mount
+$here/src/feature -P $SCRATCH_DEV || _notrun "project quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/430 b/tests/xfs/430
index 3e6527851069a9..b3a2c6dd1a5f6a 100755
--- a/tests/xfs/430
+++ b/tests/xfs/430
@@ -23,7 +23,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'prjquota' || _notrun "project quota disabled"
+
+_scratch_mount
+$here/src/feature -P $SCRATCH_DEV || _notrun "project quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/487 b/tests/xfs/487
index 0a5403a25dfd82..809a4c6fd621cd 100755
--- a/tests/xfs/487
+++ b/tests/xfs/487
@@ -24,7 +24,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'usrquota' || _notrun "user quota disabled"
+
+_scratch_mount
+$here/src/feature -U $SCRATCH_DEV || _notrun "user quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/488 b/tests/xfs/488
index 0e67889f26f7a0..a8144c9ce39b91 100755
--- a/tests/xfs/488
+++ b/tests/xfs/488
@@ -24,7 +24,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'grpquota' || _notrun "group quota disabled"
+
+_scratch_mount
+$here/src/feature -G $SCRATCH_DEV || _notrun "group quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/489 b/tests/xfs/489
index ef65525c224764..cb24b444bcc919 100755
--- a/tests/xfs/489
+++ b/tests/xfs/489
@@ -24,7 +24,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'prjquota' || _notrun "project quota disabled"
+
+_scratch_mount
+$here/src/feature -P $SCRATCH_DEV || _notrun "project quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/779 b/tests/xfs/779
index acce522995c693..e1e44c1928c67b 100755
--- a/tests/xfs/779
+++ b/tests/xfs/779
@@ -25,7 +25,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'usrquota' || _notrun "user quota disabled"
+
+_scratch_mount
+$here/src/feature -U $SCRATCH_DEV || _notrunn "user quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/780 b/tests/xfs/780
index efcbeb8e147353..ebd25f8a13bab4 100755
--- a/tests/xfs/780
+++ b/tests/xfs/780
@@ -25,7 +25,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'grpquota' || _notrun "group quota disabled"
+
+_scratch_mount
+$here/src/feature -G $SCRATCH_DEV || _notrun "group quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 
diff --git a/tests/xfs/781 b/tests/xfs/781
index 09d63bfeceb6e7..49920cd1215ec1 100755
--- a/tests/xfs/781
+++ b/tests/xfs/781
@@ -25,7 +25,10 @@ _require_quota
 
 echo "Format and populate"
 _scratch_populate_cached nofill > $seqres.full 2>&1
-echo "${MOUNT_OPTIONS}" | grep -q 'prjquota' || _notrun "project quota disabled"
+
+_scratch_mount
+$here/src/feature -P $SCRATCH_DEV || _notrun "project quota disabled"
+_scratch_unmount
 
 _scratch_xfs_set_quota_fuzz_ids
 

[PATCH 4/4] xfs: fix tests for persistent qflags

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix the tests that now break with persistent quota flags.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/quota              |    1 +
 common/rc                 |    1 +
 common/xfs                |    3 +++
 tests/xfs/007             |    2 +-
 tests/xfs/096             |    1 +
 tests/xfs/096.out         |    2 +-
 tests/xfs/106             |    2 +-
 tests/xfs/116             |   13 ++++++++++++-
 tests/xfs/116.cfg         |    1 +
 tests/xfs/116.out.default |    0 
 tests/xfs/116.out.metadir |    3 +++
 tests/xfs/152             |    2 +-
 tests/xfs/263             |    1 +
 tests/xfs/263.out         |    2 +-
 14 files changed, 28 insertions(+), 6 deletions(-)
 create mode 100644 tests/xfs/116.cfg
 rename tests/xfs/{116.out => 116.out.default} (100%)
 create mode 100644 tests/xfs/116.out.metadir


diff --git a/common/quota b/common/quota
index 4dad9b79a27a7f..4ef0d4775067ee 100644
--- a/common/quota
+++ b/common/quota
@@ -312,6 +312,7 @@ _qmount_option()
 		-e 's/grpjquota=[^, ]*/QUOTA/g' \
 		-e 's/\bpquota/QUOTA/g'    \
 		-e 's/prjquota/QUOTA/g'    \
+		-e 's/noquota/QUOTA/g'     \
 		-e 's/quota/QUOTA/g'       \
 		-e 's/uqnoenforce/QUOTA/g' \
 		-e 's/gqnoenforce/QUOTA/g' \
diff --git a/common/rc b/common/rc
index 25eb13ab2c5a48..824fc061e84a99 100644
--- a/common/rc
+++ b/common/rc
@@ -3721,6 +3721,7 @@ _get_os_name()
 _link_out_file_named()
 {
 	test -n "$seqfull" || _fail "need to set seqfull"
+	test -r "$seqfull.cfg" || _fail "need $seqfull.cfg"
 
 	local features=$2
 	local suffix=$(FEATURES="$features" perl -e '
diff --git a/common/xfs b/common/xfs
index 0f29d5c5a6b963..32a048b15efc04 100644
--- a/common/xfs
+++ b/common/xfs
@@ -1710,6 +1710,9 @@ _xfs_filter_mkfs()
 		print STDERR "ldev=\"$1\"\nlbsize=$2\nlblocks=$3\nlversion=$4\n";
 		print STDOUT "log      =LDEV bsize=XXX blocks=XXX\n";
 	}
+	if (/^\s+=\s+exchange=(\d+)\s+metadir=(\d+)/) {
+		print STDERR "exchange=$1\nmetadir=$2\n\n";
+	}
 	if (/^\s+=\s+sectsz=(\d+)\s+sunit=(\d+) blks/) {
 		print STDERR "logsectsz=$1\nlogsunit=$2\n\n";
 	}
diff --git a/tests/xfs/007 b/tests/xfs/007
index e35a069f9bd5c5..4721bc832b3fe4 100755
--- a/tests/xfs/007
+++ b/tests/xfs/007
@@ -47,7 +47,7 @@ do_test()
 	# This takes care of newer kernels where quotaoff clears the superblock
 	# quota enforcement flags but doesn't shut down accounting.
 	_scratch_unmount
-	_qmount_option ""
+	_qmount_option "noquota"
 	_scratch_mount
 
 	rm_commands=(-x -c "remove -$off_opts")
diff --git a/tests/xfs/096 b/tests/xfs/096
index 57a05a8ffefbd1..f1f5d562d4fa18 100755
--- a/tests/xfs/096
+++ b/tests/xfs/096
@@ -28,6 +28,7 @@ function option_string()
 	if [ "$((VAL & 4))" -ne "0" ]; then OPT=prjquota,${OPT}; fi;
 	if [ "$((VAL & 2))" -ne "0" ]; then OPT=grpquota,${OPT}; fi;
 	if [ "$((VAL & 1))" -ne "0" ]; then OPT=usrquota,${OPT}; fi;
+	if [ "$VAL" = "0" ]; then OPT=noquota; fi;
 	echo $OPT
 }
 
diff --git a/tests/xfs/096.out b/tests/xfs/096.out
index 1deb7a8c302374..20f68d3870c4a0 100644
--- a/tests/xfs/096.out
+++ b/tests/xfs/096.out
@@ -1,5 +1,5 @@
 QA output created by 096
-== Options: rw ==
+== Options: noquota ==
 == Options: usrquota,rw ==
 User quota state on SCRATCH_MNT (SCRATCH_DEV)
   Accounting: ON
diff --git a/tests/xfs/106 b/tests/xfs/106
index 066efef1181b8a..10cbd1052bbc89 100755
--- a/tests/xfs/106
+++ b/tests/xfs/106
@@ -155,7 +155,7 @@ test_off()
 {
 	echo "turning quota off by remounting"
 	_scratch_unmount
-	_qmount_option ""
+	_qmount_option "noquota"
 	_qmount
 }
 
diff --git a/tests/xfs/116 b/tests/xfs/116
index c5e7508f8862ed..3ef6f5ddfdb2d6 100755
--- a/tests/xfs/116
+++ b/tests/xfs/116
@@ -23,7 +23,18 @@ _require_xfs_quota
 # Only mount with the quota options we specify below
 _qmount_option "defaults"
 
-_scratch_mkfs >/dev/null 2>&1
+_scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs >/dev/null
+. $tmp.mkfs
+cat $tmp.mkfs >> $seqres.full
+
+# link correct .out file, see $seqfull.cfg
+seqfull=$0
+if [ "$metadir" = 1 ]; then
+	_link_out_file_named $seqfull.out "metadir"
+else
+	_link_out_file_named $seqfull.out
+fi
+
 _scratch_mount "-o uquota"
 _scratch_unmount
 _scratch_xfs_db -r -c sb -c print  | grep qflags
diff --git a/tests/xfs/116.cfg b/tests/xfs/116.cfg
new file mode 100644
index 00000000000000..571f542faef97d
--- /dev/null
+++ b/tests/xfs/116.cfg
@@ -0,0 +1 @@
+metadir: metadir
diff --git a/tests/xfs/116.out b/tests/xfs/116.out.default
similarity index 100%
rename from tests/xfs/116.out
rename to tests/xfs/116.out.default
diff --git a/tests/xfs/116.out.metadir b/tests/xfs/116.out.metadir
new file mode 100644
index 00000000000000..3a58fb919b1e39
--- /dev/null
+++ b/tests/xfs/116.out.metadir
@@ -0,0 +1,3 @@
+QA output created by 116
+qflags = 0x7
+qflags = 0x7
diff --git a/tests/xfs/152 b/tests/xfs/152
index 6c052cbc9b31f5..94428b35d22a87 100755
--- a/tests/xfs/152
+++ b/tests/xfs/152
@@ -194,7 +194,7 @@ test_off()
 {
 	echo "checking off command (type=$type)"
 	_scratch_unmount
-	_qmount_option ""
+	_qmount_option "noquota"
 	_qmount
 }
 
diff --git a/tests/xfs/263 b/tests/xfs/263
index aedbc4795296d3..83ec8b959fa9de 100755
--- a/tests/xfs/263
+++ b/tests/xfs/263
@@ -27,6 +27,7 @@ function option_string()
 	if [ "$((VAL & 4))" -ne "0" ]; then OPT=prjquota,${OPT}; fi;
 	if [ "$((VAL & 2))" -ne "0" ]; then OPT=grpquota,${OPT}; fi;
 	if [ "$((VAL & 1))" -ne "0" ]; then OPT=usrquota,${OPT}; fi;
+	if [ "$VAL" = "0" ]; then OPT=noquota; fi;
 	echo $OPT
 }
 
diff --git a/tests/xfs/263.out b/tests/xfs/263.out
index 64c1a5876cfa24..8682eee2680728 100644
--- a/tests/xfs/263.out
+++ b/tests/xfs/263.out
@@ -1,5 +1,5 @@
 QA output created by 263
-== Options: rw ==
+== Options: noquota ==
 == Options: usrquota,rw ==
 User quota state on SCRATCH_MNT (SCRATCH_DEV)
   Accounting: ON

[PATCH 1/3] common: enable testing of realtime quota when supported

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

If the kernel advertises realtime quota support, test it.

However, this has a plot twist -- because rt quota only works if the xfs
is formatted with rtgroups, we have to mount a filesystem to see if
rtquota is actually supported.  Since it's time consuming to format and
mount the scratch filesystem, we'll assume that the test and scratch
fses have the same support.

This will cause problems if one sets SCRATCH_RTDEV but not TEST_RTDEV.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/populate |   10 ++++++---
 common/quota    |   20 +++---------------
 common/xfs      |   62 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 72 insertions(+), 20 deletions(-)


diff --git a/common/populate b/common/populate
index 9c3d49efebb3a4..41f5941070efc8 100644
--- a/common/populate
+++ b/common/populate
@@ -245,9 +245,13 @@ _populate_xfs_qmount_option()
 	if [ ! -f /proc/fs/xfs/xqmstat ]; then
 		# No quota support
 		return
-	elif [ "${USE_EXTERNAL}" = "yes" ] && [ ! -z "${SCRATCH_RTDEV}" ]; then
-		# Quotas not supported on rt filesystems
-		return
+	elif [ "${USE_EXTERNAL}" = "yes" ] && [ -n "${SCRATCH_RTDEV}" ]; then
+		# We have not mounted the scratch fs, so we can only check
+		# rtquota support from the test fs.  Skip the quota options if
+		# the test fs does not have an rt section.
+		test -n "${TEST_RTDEV}" || return
+		_xfs_kmod_supports_rtquota || return
+		_xfs_test_supports_rtquota || return
 	elif [ -z "${XFS_QUOTA_PROG}" ]; then
 		# xfs quota tools not installed
 		return
diff --git a/common/quota b/common/quota
index 4ef0d4775067ee..6735d0fec48991 100644
--- a/common/quota
+++ b/common/quota
@@ -23,12 +23,7 @@ _require_quota()
 	if [ ! -f /proc/fs/xfs/xqmstat ]; then
 	    _notrun "Installed kernel does not support XFS quotas"
         fi
-	if [ "$USE_EXTERNAL" = yes -a ! -z "$TEST_RTDEV" ]; then
-	    _notrun "Quotas not supported on realtime test device"
-	fi
-	if [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ]; then
-	    _notrun "Quotas not supported on realtime scratch device"
-	fi
+	_require_xfs_rtquota_if_rtdev
 	;;
     *)
 	_notrun "disk quotas not supported by this filesystem type: $FSTYP"
@@ -44,12 +39,7 @@ _require_xfs_quota()
 {
     $here/src/feature -q $TEST_DEV
     [ $? -ne 0 ] && _notrun "Installed kernel does not support XFS quota"
-    if [ "$USE_EXTERNAL" = yes -a ! -z "$TEST_RTDEV" ]; then
-	_notrun "Quotas not supported on realtime test device"
-    fi
-    if [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_RTDEV" ]; then
-	_notrun "Quotas not supported on realtime scratch device"
-    fi
+    _require_xfs_rtquota_if_rtdev
     [ -n "$XFS_QUOTA_PROG" ] || _notrun "XFS quota user tools not installed"
 }
 
@@ -153,11 +143,7 @@ _require_prjquota()
     fi
     $here/src/feature -P $_dev
     [ $? -ne 0 ] && _notrun "Installed kernel does not support project quotas"
-    if [ "$USE_EXTERNAL" = yes ]; then
-	if [ -n "$TEST_RTDEV" -o -n "$SCRATCH_RTDEV" ]; then
-	    _notrun "Project quotas not supported on realtime filesystem"
-	fi
-    fi
+    test "$FSTYP" = "xfs" && _require_xfs_rtquota_if_rtdev
 }
 
 #
diff --git a/common/xfs b/common/xfs
index 32a048b15efc04..282fd7b931c3ad 100644
--- a/common/xfs
+++ b/common/xfs
@@ -2079,3 +2079,65 @@ _require_xfs_scratch_metadir()
 		_scratch_unmount
 	fi
 }
+
+# Does the xfs kernel module support realtime quota?
+_xfs_kmod_supports_rtquota() {
+	local xqmfile="/proc/fs/xfs/xqm"
+
+	test -e "$xqmfile" || modprobe xfs
+	test -e "$xqmfile" || return 1
+
+	grep -q -w rtquota "$xqmfile"
+}
+
+# Does this mounted filesystem support realtime quota?  This is the only way
+# to check that the fs really supports it because the kernel ignores quota
+# mount options for pre-rtgroups realtime filesystems.
+_xfs_fs_supports_rtquota() {
+	local mntpt="$1"
+	local dev="$2"
+	local rtdev="$3"
+
+	test -d "$mntpt" || \
+		echo "_xfs_fs_supports_rtquota needs a mountpoint"
+	test "$USE_EXTERNAL" == "yes" || \
+		echo "_xfs_fs_supports_rtquota needs USE_EXTERNAL=yes"
+	test -n "$rtdev" || \
+		echo "_xfs_fs_supports_rtquota needs an rtdev"
+
+	$here/src/feature -U $dev || \
+		$here/src/feature -G $dev || \
+		$here/src/feature -P $dev
+}
+
+# Do we support realtime quotas on the (mounted) test filesystem?
+_xfs_test_supports_rtquota() {
+	_xfs_fs_supports_rtquota "$TEST_DIR" "$TEST_DEV" "$TEST_RTDEV"
+}
+
+# Do we support realtime quotas on the (mounted) scratch filesystem?
+_xfs_scratch_supports_rtquota() {
+	_xfs_fs_supports_rtquota "$SCRATCH_MNT" "$SCRATCH_DEV" "$SCRATCH_RTDEV"
+}
+
+# Make sure that we're set up for realtime quotas if external rt devices are
+# configured.  The test filesystem has to be mounted before each test, so we
+# can check that quickly, and we make the bold assumption that the same will
+# apply to any scratch fs that might be created.
+_require_xfs_rtquota_if_rtdev() {
+	test "$USE_EXTERNAL" = "yes" || return
+
+	if [ -n "$TEST_RTDEV$SCRATCH_RTDEV" ]; then
+		_xfs_kmod_supports_rtquota || \
+			_notrun "Kernel driver does not support rt quota"
+	fi
+
+	if [ -n "$TEST_RTDEV" ]; then
+		_xfs_test_supports_rtquota || \
+			_notrun "Quotas not supported on realtime device"
+	fi
+
+	if [ -n "$SCRATCH_RTDEV" ] && [ -z "$TEST_RTDEV" ]; then
+		_notrun "Quotas probably not supported on realtime scratch device; set TEST_RTDEV"
+	fi
+}

[PATCH 2/3] xfs: fix quota tests to adapt to realtime quota

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Fix or limit the scope of tests so that we can turn on testing for
realtime quotas.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 common/quota      |   25 +++++++++++++++++++++++++
 tests/generic/219 |    1 +
 tests/generic/230 |    1 +
 tests/generic/305 |    1 +
 tests/generic/326 |    1 +
 tests/generic/327 |    1 +
 tests/generic/328 |    1 +
 tests/generic/566 |    4 +++-
 tests/generic/587 |    1 +
 tests/generic/603 |    1 +
 tests/generic/691 |    2 ++
 tests/generic/710 |    4 +++-
 tests/xfs/050     |    2 ++
 tests/xfs/096     |    4 ++++
 tests/xfs/106     |    1 +
 tests/xfs/108     |    2 ++
 tests/xfs/152     |    1 +
 tests/xfs/153     |    2 ++
 tests/xfs/161     |    1 +
 tests/xfs/213     |    1 +
 tests/xfs/214     |    1 +
 tests/xfs/220     |    2 ++
 tests/xfs/299     |    2 ++
 tests/xfs/330     |    1 +
 tests/xfs/434     |    1 +
 tests/xfs/435     |    1 +
 tests/xfs/440     |    3 +++
 tests/xfs/441     |    1 +
 tests/xfs/442     |    1 +
 tests/xfs/508     |    2 ++
 tests/xfs/511     |   10 +++++++++-
 tests/xfs/720     |    5 +++++
 32 files changed, 84 insertions(+), 3 deletions(-)


diff --git a/common/quota b/common/quota
index 6735d0fec48991..2f5ccd2fa17363 100644
--- a/common/quota
+++ b/common/quota
@@ -436,5 +436,30 @@ _restore_project_quota()
 	sed -i "/:$id$/d" /etc/projid
 }
 
+# Reconfigure the mounted fs as needed so that we can test the VFS quota
+# utilities.  They do not support realtime block limits or reporting, so
+# we forcibly inhibit rtinherit on XFS filesystems.
+_force_vfs_quota_testing()
+{
+	local mount="${1:-$TEST_DIR}"
+
+	test "$FSTYP" = "xfs" && _xfs_force_bdev data "$mount"
+}
+
+# Does the mounted scratch filesystem have a realtime volume where quota works?
+# Returns nonzero if any of the above are false.
+_scratch_supports_rtquota()
+{
+	case "$FSTYP" in
+	"xfs")
+		if [ "$USE_EXTERNAL" = yes ] && [ -n "$SCRATCH_RTDEV" ]; then
+			_xfs_scratch_supports_rtquota
+			return
+		fi
+	esac
+
+	return 1
+}
+
 # make sure this script returns success
 /bin/true
diff --git a/tests/generic/219 b/tests/generic/219
index d72aa745fdfcf1..cc2ec119eb4298 100755
--- a/tests/generic/219
+++ b/tests/generic/219
@@ -92,6 +92,7 @@ test_accounting()
 _scratch_unmount 2>/dev/null
 _scratch_mkfs >> $seqres.full 2>&1
 _scratch_mount "-o usrquota,grpquota"
+_force_vfs_quota_testing $SCRATCH_MNT
 quotacheck -u -g $SCRATCH_MNT 2>/dev/null
 quotaon $SCRATCH_MNT 2>/dev/null
 _scratch_unmount
diff --git a/tests/generic/230 b/tests/generic/230
index 18e20f4b2e9439..a8caf5a808c3c7 100755
--- a/tests/generic/230
+++ b/tests/generic/230
@@ -100,6 +100,7 @@ _qmount_option 'defaults'
 
 _scratch_mkfs >> $seqres.full 2>&1
 _scratch_mount "-o usrquota,grpquota"
+_force_vfs_quota_testing $SCRATCH_MNT
 BLOCK_SIZE=$(_get_file_block_size $SCRATCH_MNT)
 quotacheck -u -g $SCRATCH_MNT 2>/dev/null
 quotaon $SCRATCH_MNT 2>/dev/null
diff --git a/tests/generic/305 b/tests/generic/305
index 6ccbb3d07c70c2..373abf57037ed6 100755
--- a/tests/generic/305
+++ b/tests/generic/305
@@ -26,6 +26,7 @@ echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
 export MOUNT_OPTIONS="-o usrquota,grpquota $MOUNT_OPTIONS"
 _scratch_mount >> $seqres.full 2>&1
+_force_vfs_quota_testing $SCRATCH_MNT
 quotacheck -u -g $SCRATCH_MNT 2> /dev/null
 quotaon $SCRATCH_MNT 2> /dev/null
 
diff --git a/tests/generic/326 b/tests/generic/326
index 321e7dc6a8e54c..d16e95aabaecd5 100755
--- a/tests/generic/326
+++ b/tests/generic/326
@@ -27,6 +27,7 @@ echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
 export MOUNT_OPTIONS="-o usrquota,grpquota $MOUNT_OPTIONS"
 _scratch_mount >> $seqres.full 2>&1
+_force_vfs_quota_testing $SCRATCH_MNT
 quotacheck -u -g $SCRATCH_MNT 2> /dev/null
 quotaon $SCRATCH_MNT 2> /dev/null
 
diff --git a/tests/generic/327 b/tests/generic/327
index 18cfcd1f655bd7..9102cbd6841072 100755
--- a/tests/generic/327
+++ b/tests/generic/327
@@ -25,6 +25,7 @@ echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
 export MOUNT_OPTIONS="-o usrquota,grpquota $MOUNT_OPTIONS"
 _scratch_mount >> $seqres.full 2>&1
+_force_vfs_quota_testing $SCRATCH_MNT
 quotacheck -u -g $SCRATCH_MNT 2> /dev/null
 quotaon $SCRATCH_MNT 2> /dev/null
 
diff --git a/tests/generic/328 b/tests/generic/328
index fa33bdb78dba12..db785475db87d6 100755
--- a/tests/generic/328
+++ b/tests/generic/328
@@ -26,6 +26,7 @@ echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
 export MOUNT_OPTIONS="-o usrquota,grpquota $MOUNT_OPTIONS"
 _scratch_mount >> $seqres.full 2>&1
+_force_vfs_quota_testing $SCRATCH_MNT
 quotacheck -u -g $SCRATCH_MNT 2> /dev/null
 quotaon $SCRATCH_MNT 2> /dev/null
 
diff --git a/tests/generic/566 b/tests/generic/566
index a41e04852ed88c..a6ec82fd36d8bb 100755
--- a/tests/generic/566
+++ b/tests/generic/566
@@ -35,7 +35,9 @@ _qmount
 dir="$SCRATCH_MNT/dummy"
 mkdir -p $dir
 chown $qa_user $dir
-$XFS_QUOTA_PROG -x -c "limit -g bsoft=100k bhard=100k $qa_user" $SCRATCH_MNT
+_scratch_supports_rtquota && \
+	extra_limits="rtbsoft=100k rtbhard=100k"
+$XFS_QUOTA_PROG -x -c "limit -g bsoft=100k bhard=100k $extra_limits $qa_user" $SCRATCH_MNT
 
 $XFS_IO_PROG -f -c 'pwrite -S 0x58 0 1m' $dir/foo >> $seqres.full
 chown $qa_user "${dir}/foo"
diff --git a/tests/generic/587 b/tests/generic/587
index 466596799431f7..49fcade72790d6 100755
--- a/tests/generic/587
+++ b/tests/generic/587
@@ -56,6 +56,7 @@ _scratch_mkfs > $seqres.full
 # This test must have user quota enabled
 _qmount_option usrquota
 _qmount >> $seqres.full
+_force_vfs_quota_testing $SCRATCH_MNT
 
 testfile=$SCRATCH_MNT/test-$seq
 touch $testfile
diff --git a/tests/generic/603 b/tests/generic/603
index b199f801a8f03f..9b5e824577b382 100755
--- a/tests/generic/603
+++ b/tests/generic/603
@@ -120,6 +120,7 @@ _scratch_mkfs >$seqres.full 2>&1
 _scratch_enable_pquota
 _qmount_option "usrquota,grpquota,prjquota"
 _qmount
+_force_vfs_quota_testing $SCRATCH_MNT
 _require_prjquota $SCRATCH_DEV
 BLOCK_SIZE=$(_get_file_block_size $SCRATCH_MNT)
 rm -rf $SCRATCH_MNT/t
diff --git a/tests/generic/691 b/tests/generic/691
index 30ae4a1e384a05..fa682fb73b0417 100755
--- a/tests/generic/691
+++ b/tests/generic/691
@@ -39,6 +39,7 @@ _scratch_mkfs >$seqres.full 2>&1
 _scratch_enable_pquota
 _qmount_option "prjquota"
 _qmount
+_force_vfs_quota_testing $SCRATCH_MNT
 _require_prjquota $SCRATCH_DEV
 
 filter_quota()
@@ -66,6 +67,7 @@ exercise()
 		_scratch_enable_pquota
 	fi
 	_qmount
+	_force_vfs_quota_testing $SCRATCH_MNT
 	if [ "$type" = "P" ];then
 		_create_project_quota $SCRATCH_MNT/t $projid $qa_user
 		file=$SCRATCH_MNT/t/testfile
diff --git a/tests/generic/710 b/tests/generic/710
index 072cddf570f444..16f7276d3d4b04 100755
--- a/tests/generic/710
+++ b/tests/generic/710
@@ -32,7 +32,9 @@ $XFS_IO_PROG -f -c 'pwrite -S 0x59 0 64k -b 64k' -c 'truncate 256k' $SCRATCH_MNT
 chown nobody $SCRATCH_MNT/b
 
 # Set up a quota limit
-$XFS_QUOTA_PROG -x -c "limit -u bhard=70k nobody" $SCRATCH_MNT
+_scratch_supports_rtquota && \
+	extra_limits="rtbhard=70k"
+$XFS_QUOTA_PROG -x -c "limit -u bhard=70k $extra_limits nobody" $SCRATCH_MNT
 
 echo before exchangerange >> $seqres.full
 $XFS_QUOTA_PROG -x -c 'report -a' $SCRATCH_MNT >> $seqres.full
diff --git a/tests/xfs/050 b/tests/xfs/050
index 1e40ab90a843e8..46c60a4bdd6b66 100755
--- a/tests/xfs/050
+++ b/tests/xfs/050
@@ -33,6 +33,7 @@ _scratch_mkfs >/dev/null 2>&1
 orig_mntopts="$MOUNT_OPTIONS"
 _qmount_option "uquota"
 _scratch_mount
+_force_vfs_quota_testing $SCRATCH_MNT   # golden output encodes block usage
 bsize=$(_get_file_block_size $SCRATCH_MNT)
 # needs quota enabled to compute the number of metadata dir files
 HIDDEN_QUOTA_FILES=$(_xfs_calc_hidden_quota_files $SCRATCH_MNT)
@@ -75,6 +76,7 @@ _exercise()
 	. $tmp.mkfs
 
 	_qmount
+	_force_vfs_quota_testing $SCRATCH_MNT   # golden output encodes block usage
 
 	# Figure out whether we're doing large allocations
 	# (bail out if they're so large they stuff the test up)
diff --git a/tests/xfs/096 b/tests/xfs/096
index f1f5d562d4fa18..4c4fdfa12ef840 100755
--- a/tests/xfs/096
+++ b/tests/xfs/096
@@ -20,6 +20,10 @@ _require_scratch
 _require_xfs_quota
 _require_xfs_nocrc
 
+if [ "$USE_EXTERNAL" = yes ] && [ -n "$SCRATCH_RTDEV" ]; then
+	_notrun "Realtime quotas not supported on V4 filesystems"
+fi
+
 function option_string()
 {
 	VAL=$1
diff --git a/tests/xfs/106 b/tests/xfs/106
index 10cbd1052bbc89..108cd0b8c3061a 100755
--- a/tests/xfs/106
+++ b/tests/xfs/106
@@ -195,6 +195,7 @@ test_xfs_quota()
 {
 	_qmount_option $1
 	_qmount
+	_force_vfs_quota_testing $SCRATCH_MNT	 # golden output encodes block usage
 
 	if [ $type == "p" ]; then
 		_require_prjquota $SCRATCH_DEV
diff --git a/tests/xfs/108 b/tests/xfs/108
index 149d76eeb17a42..8adc63d4e56e4f 100755
--- a/tests/xfs/108
+++ b/tests/xfs/108
@@ -63,6 +63,7 @@ test_accounting()
 export MOUNT_OPTIONS="-opquota"
 _scratch_mkfs_xfs >> $seqres.full
 _qmount
+_force_vfs_quota_testing $SCRATCH_MNT	 # golden output encodes block usage
 _require_prjquota $SCRATCH_DEV
 
 rm -f $tmp.projects $seqres.full
@@ -70,6 +71,7 @@ _scratch_unmount 2>/dev/null
 _scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs
 cat $tmp.mkfs >>$seqres.full
 _scratch_mount
+_force_vfs_quota_testing $SCRATCH_MNT	 # golden output encodes block usage
 
 uid=1
 gid=2
diff --git a/tests/xfs/152 b/tests/xfs/152
index 94428b35d22a87..7ba00c4bfac9ff 100755
--- a/tests/xfs/152
+++ b/tests/xfs/152
@@ -240,6 +240,7 @@ qmount_idmapped()
 {
 	wipe_mounts
 	_try_scratch_mount || _fail "qmount failed"
+	_force_vfs_quota_testing $SCRATCH_MNT	 # golden output encodes block usage
 
 	mkdir -p "${SCRATCH_MNT}/unmapped"
 	mkdir -p "${SCRATCH_MNT}/idmapped"
diff --git a/tests/xfs/153 b/tests/xfs/153
index 2ce22b8c44b298..27db71e7738d08 100755
--- a/tests/xfs/153
+++ b/tests/xfs/153
@@ -38,6 +38,7 @@ _scratch_mkfs >/dev/null 2>&1
 orig_mntopts="$MOUNT_OPTIONS"
 _qmount_option "uquota"
 _scratch_mount
+_force_vfs_quota_testing $SCRATCH_MNT	 # golden output encodes block usage
 bsize=$(_get_file_block_size $SCRATCH_MNT)
 # needs quota enabled to compute the number of metadata dir files
 HIDDEN_QUOTA_FILES=$(_xfs_calc_hidden_quota_files $SCRATCH_MNT)
@@ -80,6 +81,7 @@ run_tests()
 	. $tmp.mkfs
 
 	_qmount
+	_force_vfs_quota_testing $SCRATCH_MNT   # golden output encodes block usage
 
 	# Figure out whether we're doing large allocations
 	# (bail out if they're so large they stuff the test up)
diff --git a/tests/xfs/161 b/tests/xfs/161
index e13a646a5053bd..c35bcabb5eda90 100755
--- a/tests/xfs/161
+++ b/tests/xfs/161
@@ -36,6 +36,7 @@ _qmount_option "usrquota"
 _scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
 _scratch_mount >> $seqres.full
 
+_xfs_force_bdev data $SCRATCH_MNT
 
 min_blksz=65536
 file_blksz=$(_get_file_block_size "$SCRATCH_MNT")
diff --git a/tests/xfs/213 b/tests/xfs/213
index 7b27e3c09a815a..a9e97ee01cabce 100755
--- a/tests/xfs/213
+++ b/tests/xfs/213
@@ -28,6 +28,7 @@ echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
 export MOUNT_OPTIONS="-o usrquota,grpquota $MOUNT_OPTIONS"
 _scratch_mount >> $seqres.full 2>&1
+_force_vfs_quota_testing $SCRATCH_MNT	# repquota
 quotacheck -u -g $SCRATCH_MNT 2> /dev/null
 quotaon $SCRATCH_MNT 2> /dev/null
 
diff --git a/tests/xfs/214 b/tests/xfs/214
index f2f23b3fb33f5d..c316a92baa4404 100755
--- a/tests/xfs/214
+++ b/tests/xfs/214
@@ -29,6 +29,7 @@ echo "Format and mount"
 _scratch_mkfs > $seqres.full 2>&1
 export MOUNT_OPTIONS="-o usrquota,grpquota $MOUNT_OPTIONS"
 _scratch_mount >> $seqres.full 2>&1
+_force_vfs_quota_testing $SCRATCH_MNT	# repquota
 quotacheck -u -g $SCRATCH_MNT 2> /dev/null
 quotaon $SCRATCH_MNT 2> /dev/null
 
diff --git a/tests/xfs/220 b/tests/xfs/220
index f89c976fb850af..d34764301b9e06 100755
--- a/tests/xfs/220
+++ b/tests/xfs/220
@@ -37,6 +37,7 @@ _scratch_mkfs_xfs >/dev/null 2>&1
 
 # mount  with quotas enabled
 _scratch_mount -o uquota
+_force_vfs_quota_testing $SCRATCH_MNT	 # golden output encodes block usage
 
 # turn off quota
 $XFS_QUOTA_PROG -x -c off $SCRATCH_DEV
@@ -49,6 +50,7 @@ _scratch_mkfs_xfs >/dev/null 2>&1
 
 # mount  with quotas enabled
 _scratch_mount -o uquota
+_force_vfs_quota_testing $SCRATCH_MNT	 # golden output encodes block usage
 
 # turn off quota accounting...
 $XFS_QUOTA_PROG -x -c off $SCRATCH_DEV
diff --git a/tests/xfs/299 b/tests/xfs/299
index 3986f8fb904e5d..9f8b6631ccabfb 100755
--- a/tests/xfs/299
+++ b/tests/xfs/299
@@ -153,6 +153,7 @@ projid_file="$tmp.projid"
 echo "*** user, group, and project"
 _qmount_option "uquota,gquota,pquota"
 _qmount
+_force_vfs_quota_testing $SCRATCH_MNT	 # golden output encodes block usage
 
 bsize=$(_get_file_block_size $SCRATCH_MNT)
 HIDDEN_QUOTA_FILES=$(_xfs_calc_hidden_quota_files $SCRATCH_MNT)
@@ -180,6 +181,7 @@ cat $tmp.mkfs >>$seqres.full
 echo "*** uqnoenforce, gqnoenforce, and pqnoenforce"
 _qmount_option "uqnoenforce,gqnoenforce,pqnoenforce"
 _qmount
+_force_vfs_quota_testing $SCRATCH_MNT	 # golden output encodes block usage
 _exercise uno
 _exercise gno
 _exercise pno
diff --git a/tests/xfs/330 b/tests/xfs/330
index 30c09ff5906e12..ca74929b1d26fc 100755
--- a/tests/xfs/330
+++ b/tests/xfs/330
@@ -36,6 +36,7 @@ echo "Format and mount"
 _scratch_mkfs > "$seqres.full" 2>&1
 export MOUNT_OPTIONS="-o usrquota,grpquota $MOUNT_OPTIONS"
 _scratch_mount >> "$seqres.full" 2>&1
+_force_vfs_quota_testing $SCRATCH_MNT	 # golden output encodes block usage
 HIDDEN_QUOTA_FILES=$(_xfs_calc_hidden_quota_files $SCRATCH_MNT)
 quotacheck -u -g $SCRATCH_MNT 2> /dev/null
 quotaon $SCRATCH_MNT 2> /dev/null
diff --git a/tests/xfs/434 b/tests/xfs/434
index fe609b138d732b..2dec1b084e0801 100755
--- a/tests/xfs/434
+++ b/tests/xfs/434
@@ -44,6 +44,7 @@ _scratch_mount -o noquota >> "$seqres.full" 2>&1
 
 testdir="$SCRATCH_MNT/test-$seq"
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 blks=3
 mkdir "$testdir"
 
diff --git a/tests/xfs/435 b/tests/xfs/435
index 22c02fbd1289bb..c1d6a40e00fc1f 100755
--- a/tests/xfs/435
+++ b/tests/xfs/435
@@ -35,6 +35,7 @@ _scratch_mount -o quota >> "$seqres.full" 2>&1
 
 testdir="$SCRATCH_MNT/test-$seq"
 blksz=65536
+_require_congruent_file_oplen $SCRATCH_MNT $blksz
 blks=3
 mkdir "$testdir"
 
diff --git a/tests/xfs/440 b/tests/xfs/440
index c0b6756ba97665..8c283fd28bbf9e 100755
--- a/tests/xfs/440
+++ b/tests/xfs/440
@@ -32,6 +32,9 @@ echo "Format and mount"
 _scratch_mkfs > "$seqres.full" 2>&1
 _scratch_mount "-o usrquota,grpquota" >> "$seqres.full" 2>&1
 
+# Make sure all our files are on the data device
+_xfs_force_bdev data $SCRATCH_MNT
+
 echo "Create files"
 $XFS_IO_PROG -c "cowextsize 1m" $SCRATCH_MNT
 touch $SCRATCH_MNT/a $SCRATCH_MNT/force_fsgqa
diff --git a/tests/xfs/441 b/tests/xfs/441
index ca3c576ad9cc66..a64da1d815047b 100755
--- a/tests/xfs/441
+++ b/tests/xfs/441
@@ -31,6 +31,7 @@ check_quota() {
 echo "Format and mount (noquota)"
 _scratch_mkfs > "$seqres.full" 2>&1
 _scratch_mount "-o noquota" >> $seqres.full 2>&1
+_force_vfs_quota_testing $SCRATCH_MNT	 # _check_quota_usage uses repquota
 
 echo "Create files"
 _pwrite_byte 0x58 0 1m $SCRATCH_MNT/a >> $seqres.full
diff --git a/tests/xfs/442 b/tests/xfs/442
index 08f0aac40433ee..9bb055ce4d02de 100755
--- a/tests/xfs/442
+++ b/tests/xfs/442
@@ -60,6 +60,7 @@ _qmount_option "usrquota,grpquota,prjquota"
 # tests now have separate faster-running regression tests.
 _scratch_mkfs_sized $((1600 * 1048576)) > $seqres.full 2>&1
 _scratch_mount >> $seqres.full 2>&1
+_force_vfs_quota_testing $SCRATCH_MNT	 # _check_quota_usage uses repquota
 
 nr_cpus=$((LOAD_FACTOR * 4))
 nr_ops=$((25000 * TIME_FACTOR))
diff --git a/tests/xfs/508 b/tests/xfs/508
index 1bd13e98c9f641..df38dd3fbd50e4 100755
--- a/tests/xfs/508
+++ b/tests/xfs/508
@@ -57,6 +57,8 @@ _require_prjquota $SCRATCH_DEV
 mkdir $SCRATCH_MNT/dir
 $QUOTA_CMD -x -c 'project -s test' $SCRATCH_MNT >>$seqres.full 2>&1
 $QUOTA_CMD -x -c 'limit -p bsoft=10m bhard=20m test' $SCRATCH_MNT
+_scratch_supports_rtquota && \
+	$QUOTA_CMD -x -c 'limit -p rtbsoft=10m rtbhard=20m test' $SCRATCH_MNT
 
 # test the Project inheritance bit is a directory only flag, and it's set on
 # directory by default. Expect no complain about "project inheritance flag is
diff --git a/tests/xfs/511 b/tests/xfs/511
index a942e92e3af32d..0c7e137203ddb7 100755
--- a/tests/xfs/511
+++ b/tests/xfs/511
@@ -38,11 +38,19 @@ $XFS_IO_PROG -f -c "pwrite 0 65536" -c syncfs $SCRATCH_MNT/t/file >>$seqres.full
 quota_cmd="$XFS_QUOTA_PROG -x"
 $quota_cmd -c "project -s -p $SCRATCH_MNT/t 42" $SCRATCH_MNT >/dev/null 2>&1
 $quota_cmd -c 'limit -p isoft=53 bsoft=100m 42' $SCRATCH_MNT
+_scratch_supports_rtquota && \
+	$quota_cmd -c 'limit -p rtbsoft=100m 42' $SCRATCH_MNT
+
+# The golden output for this test was written with the assumption that the file
+# allocation unit divides 64k evenly, so the file block usage would be exactly
+# 64k.  On realtime filesystems this isn't always true (e.g. -rextsize=28k) so
+# we'll accept the space usage being the same as what du reports for the file.
+file_nblocks=$(du -B 1024 $SCRATCH_MNT/t/file | awk '{print $1}')
 
 # The itotal and size should be 53 and 102400(k), as above project quota limit.
 # The isued and used should be 2 and 64(k), as this case takes.
 df -k --output=file,itotal,iused,size,used $SCRATCH_MNT/t | \
-	_filter_scratch | _filter_spaces
+	_filter_scratch | _filter_spaces | sed -e "s|$file_nblocks$|64|"
 
 # success, all done
 status=0
diff --git a/tests/xfs/720 b/tests/xfs/720
index 97b3d2579cbd7f..565373bef71e88 100755
--- a/tests/xfs/720
+++ b/tests/xfs/720
@@ -30,6 +30,11 @@ _scratch_mkfs > "$seqres.full" 2>&1
 _qmount_option usrquota
 _qmount
 
+# This test tries to exceed quota limits by creating an N>2 block bmbt, setting
+# the block limit to 2N, and rebuilding the bmbt.  Hence we must force the
+# files to be created on the data device.
+_xfs_force_bdev data $SCRATCH_MNT
+
 blocksize=$(_get_block_size $SCRATCH_MNT)
 alloc_unit=$(_get_file_block_size $SCRATCH_MNT)
 

[PATCH 3/3] xfs: regression testing of quota on the realtime device

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Make sure that quota accounting and enforcement work correctly for
realtime volumes on XFS.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 tests/xfs/1858     |  174 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/1858.out |   47 ++++++++++++++
 2 files changed, 221 insertions(+)
 create mode 100755 tests/xfs/1858
 create mode 100644 tests/xfs/1858.out


diff --git a/tests/xfs/1858 b/tests/xfs/1858
new file mode 100755
index 00000000000000..64c536024f6ec0
--- /dev/null
+++ b/tests/xfs/1858
@@ -0,0 +1,174 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2022-2025 Oracle.  All Rights Reserved.
+#
+# FS QA Test No. 1858
+#
+# Functional testing for realtime quotas.
+
+. ./common/preamble
+_begin_fstest auto quick quota realtime
+
+. ./common/quota
+. ./common/filter
+
+_require_test_program "punch-alternating"
+_require_scratch
+_require_user
+
+echo "Format filesystem" | tee -a $seqres.full
+_scratch_mkfs > $seqres.full
+_try_scratch_mount -o usrquota || \
+	_notrun "Can't mount with realtime quota enabled"
+
+_require_xfs_has_feature "$SCRATCH_MNT" realtime
+_scratch_supports_rtquota || _notrun "Requires realtime quota"
+
+# Make sure all our files are on the rt device
+_xfs_force_bdev realtime $SCRATCH_MNT
+chmod a+rwx $SCRATCH_MNT
+
+# Record rt geometry
+bmbt_blksz=$(_get_block_size $SCRATCH_MNT)
+file_blksz=$(_get_file_block_size $SCRATCH_MNT)
+rextsize=$((file_blksz / bmbt_blksz))
+echo "bmbt_blksz $bmbt_blksz" >> $seqres.full
+echo "file_blksz $file_blksz" >> $seqres.full
+echo "rextsize $rextsize" >> $seqres.full
+
+note() {
+	echo -e "\n$*" | tee -a $seqres.full
+}
+
+# Report on the user's block and rt block usage, soft limit, hard limit, and
+# warning count for rt volumes
+report_rtusage() {
+	local user="$1"
+	local timeout_arg="$2"
+	local print_timeout=0
+
+	test -z "$user" && user=$qa_user
+	test -n "$timeout_arg" && print_timeout=1
+
+	$XFS_QUOTA_PROG -c "quota -u -r -n -N $user" $SCRATCH_MNT | \
+		sed -e 's/ days/_days/g' >> $seqres.full
+
+	$XFS_QUOTA_PROG -c "quota -u -r -n -N $user" $SCRATCH_MNT | \
+		sed -e 's/ days/_days/g' | \
+		awk -v user=$user -v print_timeout=$print_timeout -v file_blksz=$file_blksz \
+			'{printf("%s[real] %d %d %d %d %s\n", user, $2 * 1024 / file_blksz, $3 * 1024 / file_blksz, $4 * 1024 / file_blksz, $5, print_timeout ? $6 : "---");}'
+}
+
+note "Write 128rx to root"
+$XFS_IO_PROG -f -c "pwrite 0 $((128 * file_blksz))" $SCRATCH_MNT/file1 > /dev/null
+chmod a+r $SCRATCH_MNT/file1
+_scratch_sync
+report_rtusage 0
+
+note "Write 64rx to root, 4444, and 5555."
+$XFS_IO_PROG -f -c "pwrite 0 $((64 * file_blksz))" $SCRATCH_MNT/file3.5555 > /dev/null
+chown 5555 $SCRATCH_MNT/file3.5555
+$XFS_IO_PROG -f -c "pwrite 0 $((64 * file_blksz))" $SCRATCH_MNT/file3.4444 > /dev/null
+chown 4444 $SCRATCH_MNT/file3.4444
+$XFS_IO_PROG -f -c "pwrite 0 $((64 * file_blksz))" $SCRATCH_MNT/file3 > /dev/null
+_scratch_sync
+report_rtusage 0
+report_rtusage 4444
+report_rtusage 5555
+
+note "Move 64rx from root to 5555"
+chown 5555 $SCRATCH_MNT/file3
+report_rtusage 0
+report_rtusage 4444
+report_rtusage 5555
+
+note "Move 64rx from 5555 to 4444"
+chown 4444 $SCRATCH_MNT/file3
+report_rtusage 0
+report_rtusage 4444
+report_rtusage 5555
+
+note "Set hard limit of 1024rx and check enforcement"
+$XFS_QUOTA_PROG -x -c "limit -u rtbhard=$((1024 * file_blksz)) $qa_user" $SCRATCH_MNT
+# fsync (-w) after pwrite because enforcement only begins when space usage is
+# committed.  If delalloc is enabled, this doesn't happen until writeback.
+_su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -w 0 $((2048 * file_blksz))' $SCRATCH_MNT/file2"
+report_rtusage
+
+note "Set soft limit of 512rx and check timelimit enforcement"
+rm -f $SCRATCH_MNT/file2 $SCRATCH_MNT/file2.1
+period=6	# seconds
+$XFS_QUOTA_PROG -x \
+	-c "limit -u rtbsoft=$((512 * file_blksz)) rtbhard=0 $qa_user" \
+	-c "timer -u -r -d $period" \
+	-c 'state -u' $SCRATCH_MNT >> $seqres.full
+
+_su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -w 0 $((512 * file_blksz))' $SCRATCH_MNT/file2" > /dev/null
+report_rtusage
+
+overflow=$(date +%s)
+_su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -w -b $file_blksz 0 $file_blksz' $SCRATCH_MNT/file2.1" > /dev/null
+report_rtusage
+sleep $((period / 2))
+note "Try again after $((period / 2))s"
+_su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -w -b $file_blksz $file_blksz $file_blksz' $SCRATCH_MNT/file2.1" > /dev/null
+report_rtusage
+sleep $period
+note "Try again after another ${period}s"
+_su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -w -b $file_blksz $((2 * file_blksz)) $file_blksz' $SCRATCH_MNT/file2.1" > /dev/null
+report_rtusage
+
+note "Extend time limits and warnings"
+rm -f $SCRATCH_MNT/file2 $SCRATCH_MNT/file2.1
+$XFS_QUOTA_PROG -x \
+	-c "limit -u rtbsoft=$((512 * file_blksz)) rtbhard=0 $qa_user" \
+	-c "timer -u -r -d 49h" $SCRATCH_MNT \
+	-c 'state -u' $SCRATCH_MNT >> $seqres.full
+
+_su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -w 0 $((512 * file_blksz))' $SCRATCH_MNT/file2" > /dev/null
+report_rtusage $qa_user want_timeout
+
+note "Try to trip a 2 day grace period"
+_su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -w -b $file_blksz 0 $file_blksz' $SCRATCH_MNT/file2.1" > /dev/null
+report_rtusage $qa_user want_timeout
+
+$XFS_QUOTA_PROG -x -c "timer -u -r 73h $qa_user" $SCRATCH_MNT
+
+note "Try to trip a 3 day grace period"
+_su $qa_user -c "$XFS_IO_PROG -f -c 'pwrite -w -b $file_blksz $file_blksz $file_blksz' $SCRATCH_MNT/file2.1" > /dev/null
+report_rtusage $qa_user want_timeout
+
+note "Test quota applied to bmbt"
+
+# Testing quota enforcement for bmbt shape changes is tricky.  The block
+# reservation will be for enough blocks to handle the maximal btree split.
+# This is (approximately) 9 blocks no matter the size of the existing extent
+# map structure, so we set the hard limit to one more than this quantity.
+#
+# However, that means that we need to make a file of at least twice that size
+# to ensure that we create enough extent records even in the rextsize==1 case
+# where punching doesn't just create unwritten records.
+#
+# Unfortunately, it's very difficult to predict when exactly the EDQUOT will
+# come down, so we just look for the error message.
+extent_records=$(( (25 * bmbt_blksz) / 16))
+echo "extent_records $extent_records" >> $seqres.full
+
+rm -f $SCRATCH_MNT/file2
+$XFS_QUOTA_PROG -x \
+	-c "limit -u rtbsoft=0 rtbhard=0 $qa_user" \
+	-c "limit -u bhard=$((bmbt_blksz * 10)) bsoft=0 $qa_user" \
+	-c 'state -u' $SCRATCH_MNT >> $seqres.full
+$XFS_IO_PROG -f -c "pwrite -S 0x58 -b 64m 0 $((extent_records * file_blksz))" $SCRATCH_MNT/file2 > /dev/null
+_scratch_sync
+chown $qa_user $SCRATCH_MNT/file2
+$here/src/punch-alternating $SCRATCH_MNT/file2 2>&1 | _filter_scratch
+
+$XFS_QUOTA_PROG -c "quota -u -r -n -N $qa_user" -c "quota -u -b -n -N $qa_user" $SCRATCH_MNT >> $seqres.full
+$XFS_IO_PROG -c "bmap -e -l -p -v" $SCRATCH_MNT/file2 >> $seqres.full
+
+# success, all done
+$XFS_QUOTA_PROG -x -c 'report -a -u' -c 'report -a -u -r' $SCRATCH_MNT >> $seqres.full
+ls -latr $SCRATCH_MNT >> $seqres.full
+status=0
+exit
diff --git a/tests/xfs/1858.out b/tests/xfs/1858.out
new file mode 100644
index 00000000000000..85298618d26e0f
--- /dev/null
+++ b/tests/xfs/1858.out
@@ -0,0 +1,47 @@
+QA output created by 1858
+Format filesystem
+
+Write 128rx to root
+0[real] 128 0 0 0 ---
+
+Write 64rx to root, 4444, and 5555.
+0[real] 192 0 0 0 ---
+4444[real] 64 0 0 0 ---
+5555[real] 64 0 0 0 ---
+
+Move 64rx from root to 5555
+0[real] 128 0 0 0 ---
+4444[real] 64 0 0 0 ---
+5555[real] 128 0 0 0 ---
+
+Move 64rx from 5555 to 4444
+0[real] 128 0 0 0 ---
+4444[real] 128 0 0 0 ---
+5555[real] 64 0 0 0 ---
+
+Set hard limit of 1024rx and check enforcement
+pwrite: Disk quota exceeded
+fsgqa[real] 1024 0 1024 0 ---
+
+Set soft limit of 512rx and check timelimit enforcement
+fsgqa[real] 512 512 0 0 ---
+fsgqa[real] 513 512 0 0 ---
+
+Try again after 3s
+fsgqa[real] 514 512 0 0 ---
+
+Try again after another 6s
+pwrite: Disk quota exceeded
+fsgqa[real] 514 512 0 0 ---
+
+Extend time limits and warnings
+fsgqa[real] 512 512 0 0 [--------]
+
+Try to trip a 2 day grace period
+fsgqa[real] 513 512 0 0 [2_days]
+
+Try to trip a 3 day grace period
+fsgqa[real] 514 512 0 0 [3_days]
+
+Test quota applied to bmbt
+SCRATCH_MNT/file2: Disk quota exceeded

Re: [PATCH 01/23] generic/476: fix fsstress process management

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:25:26PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> generic/476 never used to run fsstress in the background, but
> 8973af00ec212f made it do that.  This is incorrect, because now 476 runs
> for three seconds (i.e. long enough to fall out the bottom of the test
> and end up in _cleanup), ignoring any SOAK_DURATION/TIME_FACTOR
> settings.
> 
> Cc: <fstests@vger.kernel.org> # v2024.12.08
> Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> ---
>  tests/generic/476 |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Looks fine.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 02/23] metadump: make non-local function variables more obvious

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:25:42PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> In _xfs_verify_metadump_v2(), we want to set up some loop devices,
> record the names of those loop devices, and then leave the variables in
> the global namespace so that _xfs_cleanup_verify_metadump can dispose of
> them.
> 
> Elsewhere in fstests the convention for global variables is to put them
> in all caps to make it obvious that they're global and not local
> variables, so do that here too.

I did this because they are local to the file and nothing external
accesses them. So I converted them to be the same as all other
file-scope loop device names which are all lower case.

But it really doesn't matter at all, so if that's what you want
then:

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 03/23] metadump: fix cleanup for v1 metadump testing

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:25:57PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> In commit ce79de11337e38, the metadump v2 tests were updated to leave
> the names of loop devices in some global variables so that the cleanup
> method can find them and remove the loop devices.  Inexplicably, the
> metadump v1 test function was not upgraded.  Do so now.

Probably because the xfsprogs version I was using defaulted to v2
formats and so I didn't notice I hadn't converted the v1 format
code.

> Cc: <fstests@vger.kernel.org> # v2024.12.08
> Fixes: ce79de11337e38 ("fstests: clean up loop device instantiation")
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> ---
>  common/metadump |   14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 04/23] generic/482: _run_fsstress needs the test filesystem

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:26:13PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> The test filesystem is now a hard dependency of _run_fsstress because
> the latter copies the fsstress binary to a different name on the test
> filesystem:
> 
> generic/482       - output mismatch (see /var/tmp/fstests/generic/482.out.bad)
>     --- tests/generic/482.out   2024-02-28 16:20:24.262888854 -0800
>     +++ /var/tmp/fstests/generic/482.out.bad    2025-01-03 15:00:43.107625116 -0800
>     @@ -1,2 +1,3 @@
>      QA output created by 482
>     +cp: cannot create regular file '/mnt/482.fsstress': Read-only file system
>      Silence is golden
>     ...
>     (Run 'diff -u /tmp/fstests/tests/generic/482.out /var/tmp/fstests/generic/482.out.bad'  to see the entire diff)

Ah, because I hadn't added dm-logwrite support to check-parallel
this test wasn't being run....

However, this patch doesn't need to exist - this dependency is
removed  later in the series by using the changes to use a unique
session ID for each test and so the fsstress binary doesn't need to
be rename. The change in this patch is then reverted....

I'd just drop this patch (and the later revert).

-Dave.

> 
> Cc: <fstests@vger.kernel.org> # v2024.12.08
> Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> ---
>  tests/generic/482 |    1 -
>  1 file changed, 1 deletion(-)
> 
> 
> diff --git a/tests/generic/482 b/tests/generic/482
> index 8c114ee03058c6..0efc026a160040 100755
> --- a/tests/generic/482
> +++ b/tests/generic/482
> @@ -68,7 +68,6 @@ lowspace=$((1024*1024 / 512))		# 1m low space threshold
>  
>  # Use a thin device to provide deterministic discard behavior. Discards are used
>  # by the log replay tool for fast zeroing to prevent out-of-order replay issues.
> -_test_unmount
>  _dmthin_init $devsize $devsize $csize $lowspace
>  _log_writes_init $DMTHIN_VOL_DEV
>  _log_writes_mkfs >> $seqres.full 2>&1
> 
> 
> 

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 05/23] generic/019: don't fail if fio crashes while shutting down

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:26:28PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> My system (Debian 12) has fio 3.33.  Once in a while, fio crashes while
> shutting down after it receives a SIGBUS on account of the filesystem
> going down.  This causes the test to fail with:
> 
> generic/019       - output mismatch (see /var/tmp/fstests/generic/019.out.bad)
>     --- tests/generic/019.out   2024-02-28 16:20:24.130889521 -0800
>     +++ /var/tmp/fstests/generic/019.out.bad    2025-01-03 15:00:35.903564431 -0800
>     @@ -5,5 +5,6 @@
> 
>      Start fio..
>      Force SCRATCH_DEV device failure
>     +/tmp/fstests/tests/generic/019: line 112: 90841 Segmentation fault      $FIO_PROG $fio_config >> $seqres.full 2>&1
>      Make SCRATCH_DEV device operable again
>      Disallow global fail_make_request feature
>     ...
>     (Run 'diff -u /tmp/fstests/tests/generic/019.out /var/tmp/fstests/generic/019.out.bad'  to see the entire diff)
> 
> because the wait command will dutifully report fatal signals that kill
> the fio process.  Unfortunately, a core dump shows that we blew up in
> some library's exit handler somewhere:
> 
> (gdb) where
> #0  unlink_chunk (p=p@entry=0x55b31cb9a430, av=0x7f8b4475ec60 <main_arena>) at ./malloc/malloc.c:1628
> #1  0x00007f8b446222ff in _int_free (av=0x7f8b4475ec60 <main_arena>, p=0x55b31cb9a430, have_lock=<optimized out>, have_lock@entry=0) at ./malloc/malloc.c:4603
> #2  0x00007f8b44624f1f in __GI___libc_free (mem=<optimized out>) at ./malloc/malloc.c:3385
> #3  0x00007f8b3a71cf0e in ?? () from /lib/x86_64-linux-gnu/libtasn1.so.6
> #4  0x00007f8b4426447c in ?? () from /lib/x86_64-linux-gnu/libgnutls.so.30
> #5  0x00007f8b4542212a in _dl_call_fini (closure_map=closure_map@entry=0x7f8b44465620) at ./elf/dl-call_fini.c:43
> #6  0x00007f8b4542581e in _dl_fini () at ./elf/dl-fini.c:114
> #7  0x00007f8b445ca55d in __run_exit_handlers (status=0, listp=0x7f8b4475e820 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true)
>     at ./stdlib/exit.c:116
> #8  0x00007f8b445ca69a in __GI_exit (status=<optimized out>) at ./stdlib/exit.c:146
> #9  0x00007f8b445b3251 in __libc_start_call_main (main=main@entry=0x55b319278e10 <main>, argc=argc@entry=2, argv=argv@entry=0x7ffec6f8b468) at ../sysdeps/nptl/libc_start_call_main.h:74
> #10 0x00007f8b445b3305 in __libc_start_main_impl (main=0x55b319278e10 <main>, argc=2, argv=0x7ffec6f8b468, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
>     stack_end=0x7ffec6f8b458) at ../csu/libc-start.c:360
> #11 0x000055b319278ed1 in _start ()
> 
> This isn't a filesystem failure, so mask this by shovelling the output
> to seqres.full.

looks fine.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 06/23] fuzzy: do not set _FSSTRESS_PID when exercising fsx

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:26:44PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> If we're not running fsstress as the scrub exerciser, don't set
> _FSSTRESS_PID because the _kill_fsstress call in the cleanup function
> will think that it has to wait for a nonexistant fsstress process.
> This fixes the problem of xfs/565 runtime increasing from 30s to 800s
> because it tries to kill a nonexistent "565.fsstress" process and then
> waits for the fsx loop control process, which hasn't been sent any
> signals.
> 
> Cc: <fstests@vger.kernel.org> # v2024.12.08
> Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> ---
>  common/fuzzy |    6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> 
> diff --git a/common/fuzzy b/common/fuzzy
> index 534e91dedbbb43..0a2d91542b561e 100644
> --- a/common/fuzzy
> +++ b/common/fuzzy
> @@ -1392,7 +1392,11 @@ _scratch_xfs_stress_scrub() {
>  
>  	"__stress_scrub_${exerciser}_loop" "$end" "$runningfile" \
>  			"$remount_period" "$stress_tgt" &
> -	_FSSTRESS_PID=$!
> +	# The loop is a background process, so _FSSTRESS_PID is set in that
> +	# child.  Unfortunately, this process doesn't know about it.  Therefore
> +	# we need to set _FSSTRESS_PID ourselves so that cleanup tries to kill
> +	# fsstress.
> +	test "${exerciser}" = "fsstress" && _FSSTRESS_PID=$!

Yup, looks fine.

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:27:15PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Run each test program with a separate session id so that we can tell
> pkill to kill all processes of a given name, but only within our own
> session id.  This /should/ suffice to run multiple fstests on the same
> machine without one instance shooting down processes of another
> instance.
> 
> This fixes a general problem with using "pkill --parent" -- if the
> process being targeted is not a direct descendant of the bash script
> calling pkill, then pkill will not do anything.  The scrub stress tests
> make use of multiple background subshells, which is how a ^C in the
> parent process fails to result in fsx/fsstress being killed.

Yeah, 'pkill --parent' was the best I had managed to come up that
mostly worked, not because it perfect. That was something I wanted
feedback on before merge because it still had problems...

> This is necessary to fix SOAK_DURATION runtime constraints for all the
> scrub stress tests.  However, there is a cost -- the test program no
> longer runs with the same controlling tty as ./check, which means that
> ^Z doesn't work and SIGINT/SIGQUIT are set to SIG_IGN.  IOWs, if a test
> wants to kill its subprocesses, it must use another signal such as
> SIGPIPE.  Fortunately, bash doesn't whine about children dying due to
> fatal signals if the children run in a different session id.
> 
> I also explored alternate designs, and this was the least unsatisfying:
> 
> a) Setting the process group didn't work because background subshells
> are assigned a new group id.

Yup, tried that.

> b) Constraining the pkill/pgrep search to a cgroup could work, but we'd
> have to set up a cgroup in which to run the fstest.

thought about that, too, and considered if systemd scopes could do
that, but ...

> 
> c) Putting test subprocesses in a systemd sub-scope and telling systemd
> to kill the sub-scope could work because ./check can already use it to
> ensure that all child processes of a test are killed.  However, this is
> an *optional* feature, which means that we'd have to require systemd.

... requiring systemd was somewhat of a show-stopper for testing
older distros.

> d) Constraining the pkill/pgrep search to a particular mount namespace
> could work, but we already have tests that set up their own mount
> namespaces, which means the constrained pgrep will not find all child
> processes of a test.

*nod*.

> e) Constraining to any other type of namespace (uts, pid, etc) might not
> work because those namespaces might not be enabled.

*nod*

I also tried modifying fsstress to catch and propagate signals and a
couple of other ways of managing processes in the stress code, but
all ended up having significantly worse downsides than using 'pkill
--parent'.

I was aware of session IDs, but I've never used them before and
hadn't gone down the rabbit hole of working out how to use them when
I posted the initial RFC patchset.

> f) Revert check-parallel and go back to one fstests instance per system.
> Zorro already chose not to revert.
> 
> So.  Change _run_seq to create a the ./$seq process with a new session
> id, update _su calls to use the same session as the parent test, update
> all the pkill sites to use a wrapper so that we only target processes
> created by *this* instance of fstests, and update SIGINT to SIGPIPE.

Yeah, that's definitely cleaner.

.....

> @@ -1173,13 +1173,11 @@ _scratch_xfs_stress_scrub_cleanup() {
>  	rm -f "$runningfile"
>  	echo "Cleaning up scrub stress run at $(date)" >> $seqres.full
>  
> -	# Send SIGINT so that bash won't print a 'Terminated' message that
> -	# distorts the golden output.
>  	echo "Killing stressor processes at $(date)" >> $seqres.full
> -	_kill_fsstress
> -	pkill -INT --parent $$ xfs_io >> $seqres.full 2>&1
> -	pkill -INT --parent $$ fsx >> $seqres.full 2>&1
> -	pkill -INT --parent $$ xfs_scrub >> $seqres.full 2>&1
> +	_pkill --echo -PIPE fsstress >> $seqres.full 2>&1
> +	_pkill --echo -PIPE xfs_io >> $seqres.full 2>&1
> +	_pkill --echo -PIPE fsx >> $seqres.full 2>&1
> +	_pkill --echo -PIPE xfs_scrub >> $seqres.full 2>&1

Removing _kill_fsstress is wrong - the fsstress process has already
been renamed, so open coding 'pkill fsstress' may not match. The
_kill_fsstress() code gets changed to do the right thing here:

> @@ -69,7 +75,7 @@ _kill_fsstress()
>  	if [ -n "$_FSSTRESS_PID" ]; then
>  		# use SIGPIPE to avoid "Killed" messages from bash
>  		echo "killing $_FSSTRESS_BIN" >> $seqres.full
> -		pkill -PIPE $_FSSTRESS_BIN >> $seqres.full 2>&1
> +		_pkill -PIPE $_FSSTRESS_BIN >> $seqres.full 2>&1
>  		_wait_for_fsstress
>  		return $?
>  	fi

Then in the next patch when the _FSSTRESS_BIN workaround goes away,
_kill_fsstress() is exactly what you open coded in
_scratch_xfs_stress_scrub_cleanup()....

i.e. common/fuzzy really shouldn't open code the fsstress process
management - it should use the wrapper like everything else does.

Everything else in the patch looks good.

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 09/23] unmount: resume logging of stdout and stderr for filtering

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:27:31PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> There's a number of places where a test program calls a variant of
> _unmount but then pipes the output through a _filter script or
> something.  The new _unmount helper redirects stdout and stderr to
> seqres.full, which means that those error messages (some of which are
> encoded in the golden outputs) are suppressed.  This leads to test
> regressions in generic/050 and other places, so let's resume logging.

Huh. g/050 hasn't been failing in my test runs. Bizarre.

Anyway, change looks fine.

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 10/23] mkfs: don't hardcode log size

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:27:46PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Commit 000813899afb46 hardcoded a log size of 256MB into xfs/501,
> xfs/502, and generic/530.  This seems to be an attempt to reduce test
> run times by increasing the log size so that more background threads can
> run in parallel.  Unfortunately, this breaks a couple of my test
> configurations:
> 
>  - External logs smaller than 256MB
>  - Internal logs where the AG size is less than 256MB
....

> diff --git a/common/rc b/common/rc
> index 9e34c301b0deb0..885669beeb5e26 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -689,6 +689,33 @@ _test_cycle_mount()
>      _test_mount
>  }
>  
> +# Are there mkfs options to try to improve concurrency?
> +_scratch_mkfs_concurrency_options()
> +{
> +	local nr_cpus="$(( $1 * LOAD_FACTOR ))"

caller does not need to pass a number of CPUs. This function can
simply do:

	local nr_cpus=$(getconf _NPROCESSORS_CONF)

And that will set concurrency to be "optimal" for the number of CPUs
in the machine the test is going to run on. That way tests don't
need to hard code some number that is going to be too large for
small systems and to small for large systems...

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 11/23] common/xfs: find loop devices for non-blockdevs passed to _prepare_for_eio_shutdown

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:28:02PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> xfs/336 does this somewhat sketchy thing where it mdrestores into a
> regular file, and then does this to validate the restored metadata:
> 
> SCRATCH_DEV=$TEST_DIR/image _scratch_mount

That's a canonical example of what is called "stepping on a
landmine".

We validate that the SCRATCH_DEV is a block device at the start of
check and each section it reads and runs (via common/config), and
then make the assumption in all the infrastructure that SCRATCH_DEV
always points to a valid block device.

Now we have one new test that overwrites SCRATCH_DEV temporarily
with a file and so we have to add checks all through the
infrastructure to handle this one whacky test?

> Unfortunately, commit 1a49022fab9b4d causes the following regression:
> 
>  --- /tmp/fstests/tests/xfs/336.out      2024-11-12 16:17:36.733447713 -0800
>  +++ /var/tmp/fstests/xfs/336.out.bad    2025-01-04 19:10:39.861871114 -0800
>  @@ -5,4 +5,5 @@ Create big file
>   Explode the rtrmapbt
>   Create metadump file
>   Restore metadump
>  -Check restored fs
>  +Usage: _set_fs_sysfs_attr <mounted_device> <attr> <content>
>  +(see /var/tmp/fstests/xfs/336.full for details)
> 
> This is due to the fact that SCRATCH_DEV is temporarily reassigned to
> the regular file.  That path is passed straight through _scratch_mount
> to _xfs_prepare_for_eio_shutdown, but that helper _fails because the
> "dev" argument isn't actually a path to a block device.

_scratch_mount assumes that SCRATCH_DEV points to a valid block
device. xfs/336 is the problem here, not the code that assumes
SCRATCH_DEV points to a block device....

Why are these hacks needed? Why can't _xfs_verify_metadumps()
loopdev usage be extended to handle the new rt rmap code that this
test is supposed to be exercising? 

> Fix this by detecting non-bdevs and finding (we hope) the loop device
> that was created to handle the mount. 

What loop device? xfs/336 doesn't use loop devices at all.

Oh, this is assuming that mount will silently do a loopback mount
when passed a file rather than a block device. IOWs, it's relying on
some third party to do the loop device creation and hence allow it
to be mounted.

IOWs, this change is addressing a landmine by adding another
landmine.

I really think that xfs/336 needs to be fixed - one off test hacks
like this, while they may work, only make modifying and maintaining
the fstests infrastructure that much harder....

> While we're at it, have the
> helper return the exit code from mount, not _prepare_for_eio_shutdown.

That should be a separate patch.

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 12/23] preamble: fix missing _kill_fsstress

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:28:18PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Commit 8973af00ec212f added a _kill_fsstress to the standard _cleanup
> function.  However, if something breaks during test program
> initialization before it gets to sourcing common/rc, then you get
> failures that look like this:
> 
>  --- /tmp/fstests/tests/generic/556.out  2024-09-25 12:09:52.938797554 -0700
>  +++ /var/tmp/fstests/generic/556.out.bad        2025-01-04 22:34:01.268327003 -0800
>  @@ -1,16 +1,3 @@
>   QA output created by 556
>  -SCRATCH_MNT/basic Casefold
>  -SCRATCH_MNT/basic
>  -SCRATCH_MNT/casefold_flag_removal Casefold
>  -SCRATCH_MNT/casefold_flag_removal Casefold
>  -SCRATCH_MNT/flag_inheritance/d1/d2/d3 Casefold
>  -SCRATCH_MNT/symlink/ind1/TARGET
>  -mv: 'SCRATCH_MNT/rename/rename' and 'SCRATCH_MNT/rename/RENAME' are the same file
>  -# file: SCRATCH_MNT/xattrs/x
>  -user.foo="bar"
>  -
>  -# file: SCRATCH_MNT/xattrs/x/f1
>  -user.foo="bar"
>  -
>  -touch: 'SCRATCH_MNT/strict/corac'$'\314\247\303': Invalid argument
>  -touch: 'SCRATCH_MNT/strict/cora'$'\303\247\303': Invalid argument
>  +./tests/generic/556: 108: common/config: Syntax error: "&" unexpected
>  +./tests/generic/556: 10: _kill_fsstress: not found
> 
> It's that last line that's unnecessary.  Fix this by checking for the
> presence of a _kill_fsstress before invoking it.
> 
> Cc: <fstests@vger.kernel.org> # v2024.12.08
> Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> ---
>  common/preamble |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> 
> diff --git a/common/preamble b/common/preamble
> index 78e45d522f482c..0c9ee2e0377dd5 100644
> --- a/common/preamble
> +++ b/common/preamble
> @@ -7,7 +7,7 @@
>  # Standard cleanup function.  Individual tests can override this.
>  _cleanup()
>  {
> -	_kill_fsstress
> +	command -v _kill_fsstress &>/dev/null && _kill_fsstress
>  	cd /
>  	rm -r -f $tmp.*
>  }

Looks fine.

Reviewed-by: Dave Chinner <dchinner@redhat.com>
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:28:33PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Prior to commit 8973af00ec21, in the absence of an explicit
> SOAK_DURATION, this test would run 2500 fsstress operations each of ten
> times through the loop body.  On the author's machines, this kept the
> runtime to about 30s total.  Oddly, this was changed to 30s per loop
> body with no specific justification in the middle of an fsstress process
> management change.

I'm pretty sure that was because when you run g/650 on a machine
with 64p, the number of ops performed on the filesystem is
nr_cpus * 2500 * nr_loops.

In that case, each loop was taking over 90s to run, so the overall
runtime was up in the 15-20 minute mark. I wanted to cap the runtime
of each loop to min(nr_ops, SOAK_DURATION) so that it ran in about 5
minutes in the worst case i.e. (nr_loops * SOAK_DURATION).

I probably misunderstood how -n nr_ops vs --duration=30 interact;
I expected it to run until either were exhausted, not for duration
to override nr_ops as implied by this:

> On the author's machine, this explodes the runtime from ~30s to 420s.
> Put things back the way they were.

Yeah, OK, that's exactly waht keep_running() does - duration
overrides nr_ops.

Ok, so keeping or reverting the change will simply make different
people unhappy because of the excessive runtime the test has at
either ends of the CPU count spectrum - what's the best way to go
about providing the desired min(nr_ops, max loop time) behaviour?
Do we simply cap the maximum process count to keep the number of ops
down to something reasonable (e.g. 16), or something else?

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 14/23] generic/032: fix pinned mount failure

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:28:49PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> generic/032 now periodically fails with:
> 
>  --- /tmp/fstests/tests/generic/032.out	2025-01-05 11:42:14.427388698 -0800
>  +++ /var/tmp/fstests/generic/032.out.bad	2025-01-06 18:20:17.122818195 -0800
>  @@ -1,5 +1,7 @@
>   QA output created by 032
>   100 iterations
>  -000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  >................<
>  -*
>  -100000
>  +umount: /opt: target is busy.
>  +mount: /opt: /dev/sda4 already mounted on /opt.
>  +       dmesg(1) may have more information after failed mount system call.
>  +cycle mount failed
>  +(see /var/tmp/fstests/generic/032.full for details)
> 
> The root cause of this regression is the _syncloop subshell.  This
> background process runs _scratch_sync, which is actually an xfs_io
> process that calls syncfs on the scratch mount.
> 
> Unfortunately, while the test kills the _syncloop subshell, it doesn't
> actually kill the xfs_io process.  If the xfs_io process is in D state
> running the syncfs, it won't react to the signal, but it will pin the
> mount.  Then the _scratch_cycle_mount fails because the mount is pinned.
> 
> Prior to commit 8973af00ec212f the _syncloop ran sync(1) which avoided
> pinning the scratch filesystem.

How does running sync(1) prevent this? they run the same kernel
code, so I'm a little confused as to why this is a problem caused
by using the syncfs() syscall rather than the sync() syscall...

> Fix this by pgrepping for the xfs_io process and killing and waiting for
> it if necessary.

Change looks fine, though.

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 19/23] common/rc: don't copy fsstress to $TEST_DIR

[Dave Chinner]

On Thu, Jan 16, 2025 at 03:30:07PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Now that we can pkill only processes that were started by this test, we
> don't need to copy the fsstress binary to $TEST_DIR to avoid killing the
> wrong program instances.  This avoids a whole slew of ETXTBSY problems
> with scrub stress tests that run multiple copies of fsstress in the
> background.
> 
> Revert most of the changes to generic/270, because it wants to do
> something fancy with the fsstress binary, so it needs to control the
> process directly.
> 
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>

Change looks fine, though this should be a lot further up near the
top of the patchset so that the change to g/482 and the revert here
is not necessary at all.

With that reordering done:

Reviewed-by: Dave Chinner <dchinner@redhat.com>

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 10/23] mkfs: don't hardcode log size

[Theodore Ts'o]

On Tue, Jan 21, 2025 at 02:58:25PM +1100, Dave Chinner wrote:
> > +# Are there mkfs options to try to improve concurrency?
> > +_scratch_mkfs_concurrency_options()
> > +{
> > +	local nr_cpus="$(( $1 * LOAD_FACTOR ))"
> 
> caller does not need to pass a number of CPUs. This function can
> simply do:
> 
> 	local nr_cpus=$(getconf _NPROCESSORS_CONF)
> 
> And that will set concurrency to be "optimal" for the number of CPUs
> in the machine the test is going to run on. That way tests don't
> need to hard code some number that is going to be too large for
> small systems and to small for large systems...

Hmm, but is this the right thing if you are using check-parallel?  If
you are running multiple tests that are all running some kind of load
or stress-testing antagonist at the same time, then having 3x to 5x
the number of necessary antagonist threads is going to unnecessarily
slow down the test run, which goes against the original goal of what
we were hoping to achieve with check-parallel.

How many tests are you currently able to run in parallel today, and
what's the ultimate goal?  We could have some kind of antagonist load
which is shared across multiple tests, but it's not clear to me that
it's worth the complexity.  (And note that it's not just fs and cpu
load antagonistsw; there could also be memory stress antagonists, where
having multiple antagonists could lead to OOM kills...)

							- Ted

Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

[Theodore Ts'o]

On Tue, Jan 21, 2025 at 03:57:23PM +1100, Dave Chinner wrote:
> I probably misunderstood how -n nr_ops vs --duration=30 interact;
> I expected it to run until either were exhausted, not for duration
> to override nr_ops as implied by this:

There are (at least) two ways that a soak duration is being used
today; one is where someone wants to run a very long soak for hours
and where if you go long by an hour or two it's no big deals.  The
other is where you are specifying a soak duration as part of a smoke
test (using the smoketest group), where you might be hoping to keep
the overall run time to 15-20 minutes and so you set SOAK_DURATION to
3m.

(This was based on some research that Darrick did which showed that
running the original 5 tests in the smoketest group gave you most of
the code coverage of running all of the quick group, which had
ballooned from 15 minutes many years ago to an hour or more.  I just
noticed that we've since added two more tests to the smoketest group;
it might be worth checking whether those two new tests addded to thhe
smoketest groups significantly improves code coverage or not.  It
would be unfortunate if the runtime bloat that happened to the quick
group also happens to the smoketest group...)

The bottom line is in addition to trying to design semantics for users
who might be at either end of the CPU count spectrum, we should also
consider that SOAK_DURATION could be set for values ranging from
minutes to hours.

Thanks,

						- Ted

Re: [PATCH 10/23] mkfs: don't hardcode log size

[Dave Chinner]

On Tue, Jan 21, 2025 at 07:44:30AM -0500, Theodore Ts'o wrote:
> On Tue, Jan 21, 2025 at 02:58:25PM +1100, Dave Chinner wrote:
> > > +# Are there mkfs options to try to improve concurrency?
> > > +_scratch_mkfs_concurrency_options()
> > > +{
> > > +	local nr_cpus="$(( $1 * LOAD_FACTOR ))"
> > 
> > caller does not need to pass a number of CPUs. This function can
> > simply do:
> > 
> > 	local nr_cpus=$(getconf _NPROCESSORS_CONF)
> > 
> > And that will set concurrency to be "optimal" for the number of CPUs
> > in the machine the test is going to run on. That way tests don't
> > need to hard code some number that is going to be too large for
> > small systems and to small for large systems...
> 
> Hmm, but is this the right thing if you are using check-parallel?

Yes. The whole point of check-parallel is to run the tests in such a
way as to max out the resources of the test machine for the entire
test run. Everything that can be run concurrently should be run
concurrently, and we should not be cutting down on the concurrency
just because we are running check-parallel.

> If
> you are running multiple tests that are all running some kind of load
> or stress-testing antagonist at the same time, then having 3x to 5x
> the number of necessary antagonist threads is going to unnecessarily
> slow down the test run, which goes against the original goal of what
> we were hoping to achieve with check-parallel.

There are tests that run a thousand concurrent fsstress processes -
check-parallel still runs all those thousand fsstress processes.

> How many tests are you currently able to run in parallel today,

All of them if I wanted. Default is to run one test per CPU at a
time, but also to allow tests that use concurrency to maximise it.

> and
> what's the ultimate goal?

My initial goal was to maximise the utilisation of the machine when
testing XFS. If I can't max out a 64p server with 1.5 million
IOPS/7GB/s IO and 64GB RAM with check-parallel, then check-parallel
is not running enough tests in parallel.

Right now with 64 runner threads (one per CPU), I'm seeing an
average utilisation across the whole auto group XFS test run of:

-50 CPUs
- 2.5GB/s IO @ 30k IOPS
- 40GB RAM

The utilisation on ext4 is much lower and runtimes are much longer
for (as yet) unknown reasons. Concurrent fsstress loads, in
particular, appear to run much slower on ext4 than XFS...

> We could have some kind of antagonist load
> which is shared across multiple tests, but it's not clear to me that
> it's worth the complexity.

Yes, that's the plan further down the track - stuff like background
CPU hotplug (instead of a test that specifically runs hotplug with
fsstress that takes about 5 minutes to run), cache dropping to add
memory reclaim during tests, etc

> (And note that it's not just fs and cpu
> load antagonistsw; there could also be memory stress antagonists, where
> having multiple antagonists could lead to OOM kills...)

Yes, I eventually plan to use the src/usemem.c memory locker to
create changing levels of background memory stress to the test
runs...

Right now "perturbations" are exercised as a side effect of random
tests performing these actions. I want to make them controllable by
check-parallel so we can exercise the system functionality across
the entire range of correctness tests we have, not just an isolated
test case.

IOWs, the whole point of check-parallel is to make use of large
machines to stress the whole OS at the same time as we are testing
for filesystem behavioural correctness.

I also want to do it in as short a time period as possible - outside
of dedicated QE environments, I don't beleive that long running
stress tests actually provide value for the machine time they
consume. i.e. returns rapidly diminish because stress tests
cover 99.99% of the code paths they are going to exercise in the
first few minutes of running.

Yes, letting them run for longer will -eventually- cover rarely
travelled code paths, but for developers, CI systems and
first/second level QE verification of bug fixes we don't need
extended stress tests.

Further, when we run fstests in the normal way, we never cover
things like memory reclaim racing against unmount, freeze, sync,
etc. And we never cover them when the system is under extremely
heavy load running multiple GB/s of IO whilst CPU hotplug is running
whilst the scheduler is running at nearly a million context
switches/s, etc.

That's exactly the sort of loads that check-parallel is generating
on a machine just running the correctness tests in parallel. It
combines correctness testing with a dynamic, stressful environment,
and it runs the tests -fast-. The coverage I get in a single 10
minute auto-group run of check-parallel is -much higher- than I get
in a single auto-group run of check that takes 4 hours on the same
hardware to complete....

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

[Dave Chinner]

On Tue, Jan 21, 2025 at 08:00:27AM -0500, Theodore Ts'o wrote:
> On Tue, Jan 21, 2025 at 03:57:23PM +1100, Dave Chinner wrote:
> > I probably misunderstood how -n nr_ops vs --duration=30 interact;
> > I expected it to run until either were exhausted, not for duration
> > to override nr_ops as implied by this:
> 
> There are (at least) two ways that a soak duration is being used
> today; one is where someone wants to run a very long soak for hours
> and where if you go long by an hour or two it's no big deals.  The
> other is where you are specifying a soak duration as part of a smoke
> test (using the smoketest group), where you might be hoping to keep
> the overall run time to 15-20 minutes and so you set SOAK_DURATION to
> 3m.

check-parallel on my 64p machine runs the full auto group test in
under 10 minutes.

i.e. if you have a typical modern server (64-128p, 256GB RAM and a
couple of NVMe SSDs), then check-parallel allows a full test run in
the same time that './check -g smoketest' will run....

> (This was based on some research that Darrick did which showed that
> running the original 5 tests in the smoketest group gave you most of
> the code coverage of running all of the quick group, which had
> ballooned from 15 minutes many years ago to an hour or more.  I just
> noticed that we've since added two more tests to the smoketest group;
> it might be worth checking whether those two new tests addded to thhe
> smoketest groups significantly improves code coverage or not.  It
> would be unfortunate if the runtime bloat that happened to the quick
> group also happens to the smoketest group...)

Yes, and I've previously made the point about how check-parallel
changes the way we should be looking at dev-test cycles. We no
longer have to care that auto group testing takes 4 hours to run and
have to work around that with things like smoketest groups. If you
can run the whole auto test group in 10-15 minutes, then we don't
need "quick", "smoketest", etc to reduce dev-test cycle time
anymore...

> The bottom line is in addition to trying to design semantics for users
> who might be at either end of the CPU count spectrum, we should also
> consider that SOAK_DURATION could be set for values ranging from
> minutes to hours.

I don't see much point in testing for hours with check-parallel. The
whole point of it is to enable iteration across the entire fs test
matrix as fast as possible.

If you want to do long running soak tests, then keep using check for
that. If you want to run the auto group test across 100 different
mkfs option combinations, then that is where check-parallel comes in
- it'll take a few hours to do this instead of a week.

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 04/23] generic/482: _run_fsstress needs the test filesystem

[Darrick J. Wong]

On Tue, Jan 21, 2025 at 02:12:05PM +1100, Dave Chinner wrote:
> On Thu, Jan 16, 2025 at 03:26:13PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > The test filesystem is now a hard dependency of _run_fsstress because
> > the latter copies the fsstress binary to a different name on the test
> > filesystem:
> > 
> > generic/482       - output mismatch (see /var/tmp/fstests/generic/482.out.bad)
> >     --- tests/generic/482.out   2024-02-28 16:20:24.262888854 -0800
> >     +++ /var/tmp/fstests/generic/482.out.bad    2025-01-03 15:00:43.107625116 -0800
> >     @@ -1,2 +1,3 @@
> >      QA output created by 482
> >     +cp: cannot create regular file '/mnt/482.fsstress': Read-only file system
> >      Silence is golden
> >     ...
> >     (Run 'diff -u /tmp/fstests/tests/generic/482.out /var/tmp/fstests/generic/482.out.bad'  to see the entire diff)
> 
> Ah, because I hadn't added dm-logwrite support to check-parallel
> this test wasn't being run....
> 
> However, this patch doesn't need to exist - this dependency is
> removed  later in the series by using the changes to use a unique
> session ID for each test and so the fsstress binary doesn't need to
> be rename. The change in this patch is then reverted....
> 
> I'd just drop this patch (and the later revert).

Done, thanks for reviewing.

--D

> -Dave.
> 
> > 
> > Cc: <fstests@vger.kernel.org> # v2024.12.08
> > Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management")
> > Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> > ---
> >  tests/generic/482 |    1 -
> >  1 file changed, 1 deletion(-)
> > 
> > 
> > diff --git a/tests/generic/482 b/tests/generic/482
> > index 8c114ee03058c6..0efc026a160040 100755
> > --- a/tests/generic/482
> > +++ b/tests/generic/482
> > @@ -68,7 +68,6 @@ lowspace=$((1024*1024 / 512))		# 1m low space threshold
> >  
> >  # Use a thin device to provide deterministic discard behavior. Discards are used
> >  # by the log replay tool for fast zeroing to prevent out-of-order replay issues.
> > -_test_unmount
> >  _dmthin_init $devsize $devsize $csize $lowspace
> >  _log_writes_init $DMTHIN_VOL_DEV
> >  _log_writes_mkfs >> $seqres.full 2>&1
> > 
> > 
> > 
> 
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 09/23] unmount: resume logging of stdout and stderr for filtering

[Darrick J. Wong]

On Tue, Jan 21, 2025 at 02:52:35PM +1100, Dave Chinner wrote:
> On Thu, Jan 16, 2025 at 03:27:31PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > There's a number of places where a test program calls a variant of
> > _unmount but then pipes the output through a _filter script or
> > something.  The new _unmount helper redirects stdout and stderr to
> > seqres.full, which means that those error messages (some of which are
> > encoded in the golden outputs) are suppressed.  This leads to test
> > regressions in generic/050 and other places, so let's resume logging.
> 
> Huh. g/050 hasn't been failing in my test runs. Bizarre.

Yeah, it's bombing out here on the ro block device while trying to
enable quotas.

> Anyway, change looks fine.
> 
> Reviewed-by: Dave Chinner <dchinner@redhat.com>

Thanks!

--D

> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 10/23] mkfs: don't hardcode log size

[Darrick J. Wong]

On Tue, Jan 21, 2025 at 02:58:25PM +1100, Dave Chinner wrote:
> On Thu, Jan 16, 2025 at 03:27:46PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Commit 000813899afb46 hardcoded a log size of 256MB into xfs/501,
> > xfs/502, and generic/530.  This seems to be an attempt to reduce test
> > run times by increasing the log size so that more background threads can
> > run in parallel.  Unfortunately, this breaks a couple of my test
> > configurations:
> > 
> >  - External logs smaller than 256MB
> >  - Internal logs where the AG size is less than 256MB
> ....
> 
> > diff --git a/common/rc b/common/rc
> > index 9e34c301b0deb0..885669beeb5e26 100644
> > --- a/common/rc
> > +++ b/common/rc
> > @@ -689,6 +689,33 @@ _test_cycle_mount()
> >      _test_mount
> >  }
> >  
> > +# Are there mkfs options to try to improve concurrency?
> > +_scratch_mkfs_concurrency_options()
> > +{
> > +	local nr_cpus="$(( $1 * LOAD_FACTOR ))"
> 
> caller does not need to pass a number of CPUs. This function can
> simply do:
> 
> 	local nr_cpus=$(getconf _NPROCESSORS_CONF)
> 
> And that will set concurrency to be "optimal" for the number of CPUs
> in the machine the test is going to run on. That way tests don't
> need to hard code some number that is going to be too large for
> small systems and to small for large systems...

Sounds good to me.

--D

> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 10/23] mkfs: don't hardcode log size

[Darrick J. Wong]

On Tue, Jan 21, 2025 at 07:44:30AM -0500, Theodore Ts'o wrote:
> On Tue, Jan 21, 2025 at 02:58:25PM +1100, Dave Chinner wrote:
> > > +# Are there mkfs options to try to improve concurrency?
> > > +_scratch_mkfs_concurrency_options()
> > > +{
> > > +	local nr_cpus="$(( $1 * LOAD_FACTOR ))"
> > 
> > caller does not need to pass a number of CPUs. This function can
> > simply do:
> > 
> > 	local nr_cpus=$(getconf _NPROCESSORS_CONF)
> > 
> > And that will set concurrency to be "optimal" for the number of CPUs
> > in the machine the test is going to run on. That way tests don't
> > need to hard code some number that is going to be too large for
> > small systems and to small for large systems...
> 
> Hmm, but is this the right thing if you are using check-parallel?  If
> you are running multiple tests that are all running some kind of load
> or stress-testing antagonist at the same time, then having 3x to 5x
> the number of necessary antagonist threads is going to unnecessarily
> slow down the test run, which goes against the original goal of what
> we were hoping to achieve with check-parallel.

<shrug> Maybe a more appropriate thing to do is:

	local nr_cpus=$(grep Cpus_allowed /proc/self/status | hweight)

So a check-parallel could (if they see such problems) constrain the
parallelism through cpu pinning.  I think getconf _NPROCESSORS_CONF is
probably fine for now.

(The other day I /did/ see some program in either util-linux or
coreutils that told you the number of "available" cpus based on checking
the affinity mask and whatever cgroups constraints are applied.  I can't
find it now, alas...)

> How many tests are you currently able to run in parallel today, and
> what's the ultimate goal?  We could have some kind of antagonist load
> which is shared across multiple tests, but it's not clear to me that
> it's worth the complexity.  (And note that it's not just fs and cpu
> load antagonistsw; there could also be memory stress antagonists, where
> having multiple antagonists could lead to OOM kills...)

On the other hand, perhaps having random antagonistic processes from
other ./check instances is exactly the kind of stress testing that we
want to shake out weirder bugs?  It's clear from Dave's RFC that the
generic/650 cpu hotplug shenanigans had some effect. ;)

--D

> 							- Ted
> 

Re: [PATCH 10/23] mkfs: don't hardcode log size

[Darrick J. Wong]

On Wed, Jan 22, 2025 at 09:05:18AM +1100, Dave Chinner wrote:
> On Tue, Jan 21, 2025 at 07:44:30AM -0500, Theodore Ts'o wrote:
> > On Tue, Jan 21, 2025 at 02:58:25PM +1100, Dave Chinner wrote:
> > > > +# Are there mkfs options to try to improve concurrency?
> > > > +_scratch_mkfs_concurrency_options()
> > > > +{
> > > > +	local nr_cpus="$(( $1 * LOAD_FACTOR ))"
> > > 
> > > caller does not need to pass a number of CPUs. This function can
> > > simply do:
> > > 
> > > 	local nr_cpus=$(getconf _NPROCESSORS_CONF)
> > > 
> > > And that will set concurrency to be "optimal" for the number of CPUs
> > > in the machine the test is going to run on. That way tests don't
> > > need to hard code some number that is going to be too large for
> > > small systems and to small for large systems...
> > 
> > Hmm, but is this the right thing if you are using check-parallel?
> 
> Yes. The whole point of check-parallel is to run the tests in such a
> way as to max out the resources of the test machine for the entire
> test run. Everything that can be run concurrently should be run
> concurrently, and we should not be cutting down on the concurrency
> just because we are running check-parallel.
> 
> > If
> > you are running multiple tests that are all running some kind of load
> > or stress-testing antagonist at the same time, then having 3x to 5x
> > the number of necessary antagonist threads is going to unnecessarily
> > slow down the test run, which goes against the original goal of what
> > we were hoping to achieve with check-parallel.
> 
> There are tests that run a thousand concurrent fsstress processes -
> check-parallel still runs all those thousand fsstress processes.
> 
> > How many tests are you currently able to run in parallel today,
> 
> All of them if I wanted. Default is to run one test per CPU at a
> time, but also to allow tests that use concurrency to maximise it.
> 
> > and
> > what's the ultimate goal?
> 
> My initial goal was to maximise the utilisation of the machine when
> testing XFS. If I can't max out a 64p server with 1.5 million
> IOPS/7GB/s IO and 64GB RAM with check-parallel, then check-parallel
> is not running enough tests in parallel.
> 
> Right now with 64 runner threads (one per CPU), I'm seeing an
> average utilisation across the whole auto group XFS test run of:
> 
> -50 CPUs
> - 2.5GB/s IO @ 30k IOPS
> - 40GB RAM
> 
> The utilisation on ext4 is much lower and runtimes are much longer
> for (as yet) unknown reasons. Concurrent fsstress loads, in
> particular, appear to run much slower on ext4 than XFS...
> 
> > We could have some kind of antagonist load
> > which is shared across multiple tests, but it's not clear to me that
> > it's worth the complexity.
> 
> Yes, that's the plan further down the track - stuff like background
> CPU hotplug (instead of a test that specifically runs hotplug with
> fsstress that takes about 5 minutes to run), cache dropping to add
> memory reclaim during tests, etc
> 
> > (And note that it's not just fs and cpu
> > load antagonistsw; there could also be memory stress antagonists, where
> > having multiple antagonists could lead to OOM kills...)
> 
> Yes, I eventually plan to use the src/usemem.c memory locker to
> create changing levels of background memory stress to the test
> runs...
> 
> Right now "perturbations" are exercised as a side effect of random
> tests performing these actions. I want to make them controllable by
> check-parallel so we can exercise the system functionality across
> the entire range of correctness tests we have, not just an isolated
> test case.
> 
> IOWs, the whole point of check-parallel is to make use of large
> machines to stress the whole OS at the same time as we are testing
> for filesystem behavioural correctness.
> 
> I also want to do it in as short a time period as possible - outside
> of dedicated QE environments, I don't beleive that long running
> stress tests actually provide value for the machine time they
> consume. i.e. returns rapidly diminish because stress tests
> cover 99.99% of the code paths they are going to exercise in the
> first few minutes of running.
> 
> Yes, letting them run for longer will -eventually- cover rarely
> travelled code paths, but for developers, CI systems and
> first/second level QE verification of bug fixes we don't need
> extended stress tests.

Admittedly the long soak tests probably don't add much once the scratch
device has filled up and been cleaned out a few times.  Maybe that
sacrificial usemem would be useful sooner than later.

ATM the online repair vs fsstress soak test is still pretty useful for
probing just how bad things can get in terms of system stress and
stalling, but that's only because repairs are resource intensive. :)

--D

> Further, when we run fstests in the normal way, we never cover
> things like memory reclaim racing against unmount, freeze, sync,
> etc. And we never cover them when the system is under extremely
> heavy load running multiple GB/s of IO whilst CPU hotplug is running
> whilst the scheduler is running at nearly a million context
> switches/s, etc.
> 
> That's exactly the sort of loads that check-parallel is generating
> on a machine just running the correctness tests in parallel. It
> combines correctness testing with a dynamic, stressful environment,
> and it runs the tests -fast-. The coverage I get in a single 10
> minute auto-group run of check-parallel is -much higher- than I get
> in a single auto-group run of check that takes 4 hours on the same
> hardware to complete....
> 
> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

[Darrick J. Wong]

On Tue, Jan 21, 2025 at 03:57:23PM +1100, Dave Chinner wrote:
> On Thu, Jan 16, 2025 at 03:28:33PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Prior to commit 8973af00ec21, in the absence of an explicit
> > SOAK_DURATION, this test would run 2500 fsstress operations each of ten
> > times through the loop body.  On the author's machines, this kept the
> > runtime to about 30s total.  Oddly, this was changed to 30s per loop
> > body with no specific justification in the middle of an fsstress process
> > management change.
> 
> I'm pretty sure that was because when you run g/650 on a machine
> with 64p, the number of ops performed on the filesystem is
> nr_cpus * 2500 * nr_loops.

Where does that happen?

Oh, heh.  -n is the number of ops *per process*.

> In that case, each loop was taking over 90s to run, so the overall
> runtime was up in the 15-20 minute mark. I wanted to cap the runtime
> of each loop to min(nr_ops, SOAK_DURATION) so that it ran in about 5
> minutes in the worst case i.e. (nr_loops * SOAK_DURATION).
> 
> I probably misunderstood how -n nr_ops vs --duration=30 interact;
> I expected it to run until either were exhausted, not for duration
> to override nr_ops as implied by this:

Yeah, SOAK_DURATION overrides pretty much everything.

> > On the author's machine, this explodes the runtime from ~30s to 420s.
> > Put things back the way they were.
> 
> Yeah, OK, that's exactly waht keep_running() does - duration
> overrides nr_ops.
> 
> Ok, so keeping or reverting the change will simply make different
> people unhappy because of the excessive runtime the test has at
> either ends of the CPU count spectrum - what's the best way to go
> about providing the desired min(nr_ops, max loop time) behaviour?
> Do we simply cap the maximum process count to keep the number of ops
> down to something reasonable (e.g. 16), or something else?

How about running fsstress with --duration=3 if SOAK_DURATION isn't set?
That should keep the runtime to 30 seconds or so even on larger
machines:

if [ -n "$SOAK_DURATION" ]; then
	test "$SOAK_DURATION" -lt 10 && SOAK_DURATION=10
	fsstress_args+=(--duration="$((SOAK_DURATION / 10))")
else
	# run for 3s per iteration max for a default runtime of ~30s.
	fsstress_args+=(--duration=3)
fi

--D

> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

[Darrick J. Wong]

On Wed, Jan 22, 2025 at 09:15:48AM +1100, Dave Chinner wrote:
> On Tue, Jan 21, 2025 at 08:00:27AM -0500, Theodore Ts'o wrote:
> > On Tue, Jan 21, 2025 at 03:57:23PM +1100, Dave Chinner wrote:
> > > I probably misunderstood how -n nr_ops vs --duration=30 interact;
> > > I expected it to run until either were exhausted, not for duration
> > > to override nr_ops as implied by this:
> > 
> > There are (at least) two ways that a soak duration is being used
> > today; one is where someone wants to run a very long soak for hours
> > and where if you go long by an hour or two it's no big deals.  The
> > other is where you are specifying a soak duration as part of a smoke
> > test (using the smoketest group), where you might be hoping to keep
> > the overall run time to 15-20 minutes and so you set SOAK_DURATION to
> > 3m.
> 
> check-parallel on my 64p machine runs the full auto group test in
> under 10 minutes.
> 
> i.e. if you have a typical modern server (64-128p, 256GB RAM and a
> couple of NVMe SSDs), then check-parallel allows a full test run in
> the same time that './check -g smoketest' will run....
> 
> > (This was based on some research that Darrick did which showed that
> > running the original 5 tests in the smoketest group gave you most of
> > the code coverage of running all of the quick group, which had
> > ballooned from 15 minutes many years ago to an hour or more.  I just
> > noticed that we've since added two more tests to the smoketest group;
> > it might be worth checking whether those two new tests addded to thhe
> > smoketest groups significantly improves code coverage or not.  It
> > would be unfortunate if the runtime bloat that happened to the quick
> > group also happens to the smoketest group...)
> 
> Yes, and I've previously made the point about how check-parallel
> changes the way we should be looking at dev-test cycles. We no
> longer have to care that auto group testing takes 4 hours to run and
> have to work around that with things like smoketest groups. If you
> can run the whole auto test group in 10-15 minutes, then we don't
> need "quick", "smoketest", etc to reduce dev-test cycle time
> anymore...
> 
> > The bottom line is in addition to trying to design semantics for users
> > who might be at either end of the CPU count spectrum, we should also
> > consider that SOAK_DURATION could be set for values ranging from
> > minutes to hours.
> 
> I don't see much point in testing for hours with check-parallel. The
> whole point of it is to enable iteration across the entire fs test
> matrix as fast as possible.

I do -- running all the soak tests in parallel on a (probably old lower
spec) machine.  Parallelism is all right for a lot of things.

--D

> If you want to do long running soak tests, then keep using check for
> that. If you want to run the auto group test across 100 different
> mkfs option combinations, then that is where check-parallel comes in
> - it'll take a few hours to do this instead of a week.
> 
> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 19/23] common/rc: don't copy fsstress to $TEST_DIR

[Darrick J. Wong]

On Tue, Jan 21, 2025 at 04:05:38PM +1100, Dave Chinner wrote:
> On Thu, Jan 16, 2025 at 03:30:07PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Now that we can pkill only processes that were started by this test, we
> > don't need to copy the fsstress binary to $TEST_DIR to avoid killing the
> > wrong program instances.  This avoids a whole slew of ETXTBSY problems
> > with scrub stress tests that run multiple copies of fsstress in the
> > background.
> > 
> > Revert most of the changes to generic/270, because it wants to do
> > something fancy with the fsstress binary, so it needs to control the
> > process directly.
> > 
> > Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> 
> Change looks fine, though this should be a lot further up near the
> top of the patchset so that the change to g/482 and the revert here
> is not necessary at all.
> 
> With that reordering done:
> 
> Reviewed-by: Dave Chinner <dchinner@redhat.com>

Done, and thanks for responding to the big patch set.

--D

> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 11/23] common/xfs: find loop devices for non-blockdevs passed to _prepare_for_eio_shutdown

[Darrick J. Wong]

On Tue, Jan 21, 2025 at 03:37:17PM +1100, Dave Chinner wrote:
> On Thu, Jan 16, 2025 at 03:28:02PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > xfs/336 does this somewhat sketchy thing where it mdrestores into a
> > regular file, and then does this to validate the restored metadata:
> > 
> > SCRATCH_DEV=$TEST_DIR/image _scratch_mount
> 
> That's a canonical example of what is called "stepping on a
> landmine".

60% of fstests is written in bash, all of it is a friggin land mine
because bash totally lets us do variable substitution at any time, and
any time you make a change you have to exhaustively test the whole mess
to make sure nothing broke...

(Yeah, I hate bash)

> We validate that the SCRATCH_DEV is a block device at the start of
> check and each section it reads and runs (via common/config), and
> then make the assumption in all the infrastructure that SCRATCH_DEV
> always points to a valid block device.

We do?  Can you point me to the sentence in doc/ that says this
explicitly?  There's nothing I can find in the any docs and
_try_scratch_mount does not check SCRATCH_DEV is a bdev for XFS.
That needs to be documented.

> Now we have one new test that overwrites SCRATCH_DEV temporarily
> with a file and so we have to add checks all through the
> infrastructure to handle this one whacky test?
> 
> > Unfortunately, commit 1a49022fab9b4d causes the following regression:
> > 
> >  --- /tmp/fstests/tests/xfs/336.out      2024-11-12 16:17:36.733447713 -0800
> >  +++ /var/tmp/fstests/xfs/336.out.bad    2025-01-04 19:10:39.861871114 -0800
> >  @@ -5,4 +5,5 @@ Create big file
> >   Explode the rtrmapbt
> >   Create metadump file
> >   Restore metadump
> >  -Check restored fs
> >  +Usage: _set_fs_sysfs_attr <mounted_device> <attr> <content>
> >  +(see /var/tmp/fstests/xfs/336.full for details)
> > 
> > This is due to the fact that SCRATCH_DEV is temporarily reassigned to
> > the regular file.  That path is passed straight through _scratch_mount
> > to _xfs_prepare_for_eio_shutdown, but that helper _fails because the
> > "dev" argument isn't actually a path to a block device.
> 
> _scratch_mount assumes that SCRATCH_DEV points to a valid block
> device. xfs/336 is the problem here, not the code that assumes
> SCRATCH_DEV points to a block device....
> 
> Why are these hacks needed? Why can't _xfs_verify_metadumps()
> loopdev usage be extended to handle the new rt rmap code that this
> test is supposed to be exercising? 

Because _xfs_verify_metadumps came long after xfs/336.  336 itself was
merged long ago when I was naïve and didn't think it would take quite so
long to merge the rtrmap code.

> > Fix this by detecting non-bdevs and finding (we hope) the loop device
> > that was created to handle the mount. 
> 
> What loop device? xfs/336 doesn't use loop devices at all.
> 
> Oh, this is assuming that mount will silently do a loopback mount
> when passed a file rather than a block device. IOWs, it's relying on
> some third party to do the loop device creation and hence allow it
> to be mounted.
> 
> IOWs, this change is addressing a landmine by adding another
> landmine.

Some would say that mount adding the ability to set up a loop dev was
itself *avoiding* a landmine from 90s era util-linux.

> I really think that xfs/336 needs to be fixed - one off test hacks
> like this, while they may work, only make modifying and maintaining
> the fstests infrastructure that much harder....

Yeah, it'll get cleaned up for the rtrmap fstests merge.

> > While we're at it, have the
> > helper return the exit code from mount, not _prepare_for_eio_shutdown.
> 
> That should be a separate patch.

Ok.

--D

> 
> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 14/23] generic/032: fix pinned mount failure

[Darrick J. Wong]

On Tue, Jan 21, 2025 at 04:03:23PM +1100, Dave Chinner wrote:
> On Thu, Jan 16, 2025 at 03:28:49PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > generic/032 now periodically fails with:
> > 
> >  --- /tmp/fstests/tests/generic/032.out	2025-01-05 11:42:14.427388698 -0800
> >  +++ /var/tmp/fstests/generic/032.out.bad	2025-01-06 18:20:17.122818195 -0800
> >  @@ -1,5 +1,7 @@
> >   QA output created by 032
> >   100 iterations
> >  -000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  >................<
> >  -*
> >  -100000
> >  +umount: /opt: target is busy.
> >  +mount: /opt: /dev/sda4 already mounted on /opt.
> >  +       dmesg(1) may have more information after failed mount system call.
> >  +cycle mount failed
> >  +(see /var/tmp/fstests/generic/032.full for details)
> > 
> > The root cause of this regression is the _syncloop subshell.  This
> > background process runs _scratch_sync, which is actually an xfs_io
> > process that calls syncfs on the scratch mount.
> > 
> > Unfortunately, while the test kills the _syncloop subshell, it doesn't
> > actually kill the xfs_io process.  If the xfs_io process is in D state
> > running the syncfs, it won't react to the signal, but it will pin the
> > mount.  Then the _scratch_cycle_mount fails because the mount is pinned.
> > 
> > Prior to commit 8973af00ec212f the _syncloop ran sync(1) which avoided
> > pinning the scratch filesystem.
> 
> How does running sync(1) prevent this? they run the same kernel
> code, so I'm a little confused as to why this is a problem caused
> by using the syncfs() syscall rather than the sync() syscall...

Instead of:
_scratch_sync -> _sync_fs $SCRATCH_MNT -> $XFS_IO_PROG -rxc "syncfs" $SCRATCH_MNT

sync(1) just calls sync(2) with no open files other than
std{in,out,err}.

--D

> > Fix this by pgrepping for the xfs_io process and killing and waiting for
> > it if necessary.
> 
> Change looks fine, though.
> 
> Reviewed-by: Dave Chinner <dchinner@redhat.com>
> 
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

[Theodore Ts'o]

On Wed, Jan 22, 2025 at 09:15:48AM +1100, Dave Chinner wrote:
> check-parallel on my 64p machine runs the full auto group test in
> under 10 minutes.
> 
> i.e. if you have a typical modern server (64-128p, 256GB RAM and a
> couple of NVMe SSDs), then check-parallel allows a full test run in
> the same time that './check -g smoketest' will run....

Interesting.  I would have thought that even with NVMe SSD's, you'd be
I/O speed constrained, especially given that some of the tests
(especially the ENOSPC hitters) can take quite a lot of time to fill
the storage device, even if they are using fallocate.

How do you have your test and scratch devices configured?

> Yes, and I've previously made the point about how check-parallel
> changes the way we should be looking at dev-test cycles. We no
> longer have to care that auto group testing takes 4 hours to run and
> have to work around that with things like smoketest groups. If you
> can run the whole auto test group in 10-15 minutes, then we don't
> need "quick", "smoketest", etc to reduce dev-test cycle time
> anymore...

Well, yes, if the only consideration is test run time latency.

I can think of two off-setting considerations.  The first is if you
care about cost.  The cheapest you can get a 64 CPU, 24 GiB VM on
Google Cloud is $3.04 USD/hour (n1-stndard-64 in a Iowa data center),
so ten minutes of run time is about 51 cents USD (ignoring the storage
costs).  Not bad.  But running xfs/4k on the auto group on an
e2-standard-2 VM takes 3.2 hours; but the e2-standard-2 VM is much
cheaper, coming in at $0.087651 USD/ hour.  So that translates to 28
cents for the VM, and that's not taking into account the fact you
almost certainly much more expensive, high-performance storge to
support the 64 CPU VM.  So if you don't care about time to run
completion (for example, if I'm monitoring the 5.15, 6.1, 6.6, and
6.12 LTS LTS rc git trees, and kicking off a build whenever Greg or
Sasha updates them), using a serialized xfstests is going to be
cheaper because you can use less expensive cloud resources.

The second concern is that for certain class of failures (UBSAN,
KCSAN, Lockdep, RCU soft lockups, WARN_ON, BUG_ON, and other
panics/OOPS), if you are runnig 64 tests in parllel it might not be
obvious which test caused the failure.  Today, even if the test VM
crashes or hangs, I can have test manager (which runs on a e2-small VM
costing $0.021913 USD/hour and can manage dozens of test VM's all at the
same time), can restart the test VM, and we know which test is at at
fault, and we mark that a particular test with the Junit XML status of
"error" (as distinct from "success" or "failure").  If there are 64
test runs in parallel, if I wanted to have automated recovery if the
test appliance hangs or crashes, life gets a lot more complicated.....
I suppose we could have the human (or test automation) try run each
individual test that had been running at the time of the crash but
that's a lot more complicated, and what if the tests pass when run
once at a time?  I guess we should happen that check-parallel found a
bug that plain check didn't find, but the human being still has to
root cause the failure.

Cheers,

						- Ted

Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

[Dave Chinner]

On Tue, Jan 21, 2025 at 07:49:44PM -0800, Darrick J. Wong wrote:
> On Tue, Jan 21, 2025 at 03:57:23PM +1100, Dave Chinner wrote:
> > On Thu, Jan 16, 2025 at 03:28:33PM -0800, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > > 
> > > Prior to commit 8973af00ec21, in the absence of an explicit
> > > SOAK_DURATION, this test would run 2500 fsstress operations each of ten
> > > times through the loop body.  On the author's machines, this kept the
> > > runtime to about 30s total.  Oddly, this was changed to 30s per loop
> > > body with no specific justification in the middle of an fsstress process
> > > management change.
> > 
> > I'm pretty sure that was because when you run g/650 on a machine
> > with 64p, the number of ops performed on the filesystem is
> > nr_cpus * 2500 * nr_loops.
> 
> Where does that happen?
> 
> Oh, heh.  -n is the number of ops *per process*.

Yeah, I just noticed another case of this:

Ten slowest tests - runtime in seconds:
generic/750 559
generic/311 486
.....

generic/750 does:

nr_cpus=$((LOAD_FACTOR * 4))
nr_ops=$((25000 * nr_cpus * TIME_FACTOR))
fsstress_args=(-w -d $SCRATCH_MNT -n $nr_ops -p $nr_cpus)

So the actual load factor increase is exponential:

Load factor	nr_cpus		nr_ops		total ops
1		4		100k		400k
2		8		200k		1.6M
3		12		300k		3.6M
4		16		400k		6.4M

and so on.

I suspect that there are other similar cpu scaling issues
lurking across the many fsstress tests...

> > > On the author's machine, this explodes the runtime from ~30s to 420s.
> > > Put things back the way they were.
> > 
> > Yeah, OK, that's exactly waht keep_running() does - duration
> > overrides nr_ops.
> > 
> > Ok, so keeping or reverting the change will simply make different
> > people unhappy because of the excessive runtime the test has at
> > either ends of the CPU count spectrum - what's the best way to go
> > about providing the desired min(nr_ops, max loop time) behaviour?
> > Do we simply cap the maximum process count to keep the number of ops
> > down to something reasonable (e.g. 16), or something else?
> 
> How about running fsstress with --duration=3 if SOAK_DURATION isn't set?
> That should keep the runtime to 30 seconds or so even on larger
> machines:
> 
> if [ -n "$SOAK_DURATION" ]; then
> 	test "$SOAK_DURATION" -lt 10 && SOAK_DURATION=10
> 	fsstress_args+=(--duration="$((SOAK_DURATION / 10))")
> else
> 	# run for 3s per iteration max for a default runtime of ~30s.
> 	fsstress_args+=(--duration=3)
> fi

Yeah, that works for me.

As a rainy day project, perhaps we should look to convert all the
fsstress invocations to be time bound rather than running a specific
number of ops. i.e. hard code nr_ops=<some huge number> in
_run_fstress_bg() and the tests only need to define parallelism and
runtime.

This would make the test runtimes more deterministic across machines
with vastly different capabilities and and largely make "test xyz is
slow on my test machine" reports largely go away.

Thoughts?

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 14/23] generic/032: fix pinned mount failure

[Dave Chinner]

On Tue, Jan 21, 2025 at 08:08:34PM -0800, Darrick J. Wong wrote:
> On Tue, Jan 21, 2025 at 04:03:23PM +1100, Dave Chinner wrote:
> > On Thu, Jan 16, 2025 at 03:28:49PM -0800, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > > 
> > > generic/032 now periodically fails with:
> > > 
> > >  --- /tmp/fstests/tests/generic/032.out	2025-01-05 11:42:14.427388698 -0800
> > >  +++ /var/tmp/fstests/generic/032.out.bad	2025-01-06 18:20:17.122818195 -0800
> > >  @@ -1,5 +1,7 @@
> > >   QA output created by 032
> > >   100 iterations
> > >  -000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd  >................<
> > >  -*
> > >  -100000
> > >  +umount: /opt: target is busy.
> > >  +mount: /opt: /dev/sda4 already mounted on /opt.
> > >  +       dmesg(1) may have more information after failed mount system call.
> > >  +cycle mount failed
> > >  +(see /var/tmp/fstests/generic/032.full for details)
> > > 
> > > The root cause of this regression is the _syncloop subshell.  This
> > > background process runs _scratch_sync, which is actually an xfs_io
> > > process that calls syncfs on the scratch mount.
> > > 
> > > Unfortunately, while the test kills the _syncloop subshell, it doesn't
> > > actually kill the xfs_io process.  If the xfs_io process is in D state
> > > running the syncfs, it won't react to the signal, but it will pin the
> > > mount.  Then the _scratch_cycle_mount fails because the mount is pinned.
> > > 
> > > Prior to commit 8973af00ec212f the _syncloop ran sync(1) which avoided
> > > pinning the scratch filesystem.
> > 
> > How does running sync(1) prevent this? they run the same kernel
> > code, so I'm a little confused as to why this is a problem caused
> > by using the syncfs() syscall rather than the sync() syscall...
> 
> Instead of:
> _scratch_sync -> _sync_fs $SCRATCH_MNT -> $XFS_IO_PROG -rxc "syncfs" $SCRATCH_MNT
> 
> sync(1) just calls sync(2) with no open files other than
> std{in,out,err}.

Sure, but while sync(2) is writing back a superblock it pins the
superblock by holding the s_umount lock. So even if the sync process
is killable, it still pins the dirty superblock for the same
amount of time as syncfs.

Oh, in the sync case unmount blocks on the s_umount lock rather than
returns -EBUSY because of the elevated open file count with syncfs.
Ok, gotcha, we've been using different definitions for the phrase
"mount is pinned". :/

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Darrick J. Wong]

On Tue, Jan 21, 2025 at 02:28:26PM +1100, Dave Chinner wrote:
> On Thu, Jan 16, 2025 at 03:27:15PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Run each test program with a separate session id so that we can tell
> > pkill to kill all processes of a given name, but only within our own
> > session id.  This /should/ suffice to run multiple fstests on the same
> > machine without one instance shooting down processes of another
> > instance.
> > 
> > This fixes a general problem with using "pkill --parent" -- if the
> > process being targeted is not a direct descendant of the bash script
> > calling pkill, then pkill will not do anything.  The scrub stress tests
> > make use of multiple background subshells, which is how a ^C in the
> > parent process fails to result in fsx/fsstress being killed.
> 
> Yeah, 'pkill --parent' was the best I had managed to come up that
> mostly worked, not because it perfect. That was something I wanted
> feedback on before merge because it still had problems...
> 
> > This is necessary to fix SOAK_DURATION runtime constraints for all the
> > scrub stress tests.  However, there is a cost -- the test program no
> > longer runs with the same controlling tty as ./check, which means that
> > ^Z doesn't work and SIGINT/SIGQUIT are set to SIG_IGN.  IOWs, if a test
> > wants to kill its subprocesses, it must use another signal such as
> > SIGPIPE.  Fortunately, bash doesn't whine about children dying due to
> > fatal signals if the children run in a different session id.
> > 
> > I also explored alternate designs, and this was the least unsatisfying:
> > 
> > a) Setting the process group didn't work because background subshells
> > are assigned a new group id.
> 
> Yup, tried that.
> 
> > b) Constraining the pkill/pgrep search to a cgroup could work, but we'd
> > have to set up a cgroup in which to run the fstest.
> 
> thought about that, too, and considered if systemd scopes could do
> that, but ...
> 
> > 
> > c) Putting test subprocesses in a systemd sub-scope and telling systemd
> > to kill the sub-scope could work because ./check can already use it to
> > ensure that all child processes of a test are killed.  However, this is
> > an *optional* feature, which means that we'd have to require systemd.
> 
> ... requiring systemd was somewhat of a show-stopper for testing
> older distros.

Isn't RHEL7 the oldest one at this point?  And it does systemd.  At this
point the only reason I didn't go full systemd is out of consideration
for Devuan, since they probably need QA.

> > d) Constraining the pkill/pgrep search to a particular mount namespace
> > could work, but we already have tests that set up their own mount
> > namespaces, which means the constrained pgrep will not find all child
> > processes of a test.
> 
> *nod*.
> 
> > e) Constraining to any other type of namespace (uts, pid, etc) might not
> > work because those namespaces might not be enabled.
> 
> *nod*
> 
> I also tried modifying fsstress to catch and propagate signals and a
> couple of other ways of managing processes in the stress code, but
> all ended up having significantly worse downsides than using 'pkill
> --parent'.

Yeah, and then you'd still have to figure out fsx and any other random
little utility that a test might run in a background.

> I was aware of session IDs, but I've never used them before and
> hadn't gone down the rabbit hole of working out how to use them when
> I posted the initial RFC patchset.

<nod> Session IDs kinda suck, but they suck the least for nearly minimal
extra effort.

> > f) Revert check-parallel and go back to one fstests instance per system.
> > Zorro already chose not to revert.
> > 
> > So.  Change _run_seq to create a the ./$seq process with a new session
> > id, update _su calls to use the same session as the parent test, update
> > all the pkill sites to use a wrapper so that we only target processes
> > created by *this* instance of fstests, and update SIGINT to SIGPIPE.
> 
> Yeah, that's definitely cleaner.
> 
> .....
> 
> > @@ -1173,13 +1173,11 @@ _scratch_xfs_stress_scrub_cleanup() {
> >  	rm -f "$runningfile"
> >  	echo "Cleaning up scrub stress run at $(date)" >> $seqres.full
> >  
> > -	# Send SIGINT so that bash won't print a 'Terminated' message that
> > -	# distorts the golden output.
> >  	echo "Killing stressor processes at $(date)" >> $seqres.full
> > -	_kill_fsstress
> > -	pkill -INT --parent $$ xfs_io >> $seqres.full 2>&1
> > -	pkill -INT --parent $$ fsx >> $seqres.full 2>&1
> > -	pkill -INT --parent $$ xfs_scrub >> $seqres.full 2>&1
> > +	_pkill --echo -PIPE fsstress >> $seqres.full 2>&1
> > +	_pkill --echo -PIPE xfs_io >> $seqres.full 2>&1
> > +	_pkill --echo -PIPE fsx >> $seqres.full 2>&1
> > +	_pkill --echo -PIPE xfs_scrub >> $seqres.full 2>&1
> 
> Removing _kill_fsstress is wrong - the fsstress process has already
> been renamed, so open coding 'pkill fsstress' may not match. The
> _kill_fsstress() code gets changed to do the right thing here:
> 
> > @@ -69,7 +75,7 @@ _kill_fsstress()
> >  	if [ -n "$_FSSTRESS_PID" ]; then
> >  		# use SIGPIPE to avoid "Killed" messages from bash
> >  		echo "killing $_FSSTRESS_BIN" >> $seqres.full
> > -		pkill -PIPE $_FSSTRESS_BIN >> $seqres.full 2>&1
> > +		_pkill -PIPE $_FSSTRESS_BIN >> $seqres.full 2>&1
> >  		_wait_for_fsstress
> >  		return $?
> >  	fi
> 
> Then in the next patch when the _FSSTRESS_BIN workaround goes away,
> _kill_fsstress() is exactly what you open coded in
> _scratch_xfs_stress_scrub_cleanup()....
> 
> i.e. common/fuzzy really shouldn't open code the fsstress process
> management - it should use the wrapper like everything else does.

Ok will change.  I suppose I did go fix up the setting (or not) of
_FSSTRESS_PID.

> Everything else in the patch looks good.

Cool!

--D

> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

[Darrick J. Wong]

On Wed, Jan 22, 2025 at 03:12:11PM +1100, Dave Chinner wrote:
> On Tue, Jan 21, 2025 at 07:49:44PM -0800, Darrick J. Wong wrote:
> > On Tue, Jan 21, 2025 at 03:57:23PM +1100, Dave Chinner wrote:
> > > On Thu, Jan 16, 2025 at 03:28:33PM -0800, Darrick J. Wong wrote:
> > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > 
> > > > Prior to commit 8973af00ec21, in the absence of an explicit
> > > > SOAK_DURATION, this test would run 2500 fsstress operations each of ten
> > > > times through the loop body.  On the author's machines, this kept the
> > > > runtime to about 30s total.  Oddly, this was changed to 30s per loop
> > > > body with no specific justification in the middle of an fsstress process
> > > > management change.
> > > 
> > > I'm pretty sure that was because when you run g/650 on a machine
> > > with 64p, the number of ops performed on the filesystem is
> > > nr_cpus * 2500 * nr_loops.
> > 
> > Where does that happen?
> > 
> > Oh, heh.  -n is the number of ops *per process*.
> 
> Yeah, I just noticed another case of this:
> 
> Ten slowest tests - runtime in seconds:
> generic/750 559
> generic/311 486
> .....
> 
> generic/750 does:
> 
> nr_cpus=$((LOAD_FACTOR * 4))
> nr_ops=$((25000 * nr_cpus * TIME_FACTOR))
> fsstress_args=(-w -d $SCRATCH_MNT -n $nr_ops -p $nr_cpus)
> 
> So the actual load factor increase is exponential:
> 
> Load factor	nr_cpus		nr_ops		total ops
> 1		4		100k		400k
> 2		8		200k		1.6M
> 3		12		300k		3.6M
> 4		16		400k		6.4M
> 
> and so on.
> 
> I suspect that there are other similar cpu scaling issues
> lurking across the many fsstress tests...
> 
> > > > On the author's machine, this explodes the runtime from ~30s to 420s.
> > > > Put things back the way they were.
> > > 
> > > Yeah, OK, that's exactly waht keep_running() does - duration
> > > overrides nr_ops.
> > > 
> > > Ok, so keeping or reverting the change will simply make different
> > > people unhappy because of the excessive runtime the test has at
> > > either ends of the CPU count spectrum - what's the best way to go
> > > about providing the desired min(nr_ops, max loop time) behaviour?
> > > Do we simply cap the maximum process count to keep the number of ops
> > > down to something reasonable (e.g. 16), or something else?
> > 
> > How about running fsstress with --duration=3 if SOAK_DURATION isn't set?
> > That should keep the runtime to 30 seconds or so even on larger
> > machines:
> > 
> > if [ -n "$SOAK_DURATION" ]; then
> > 	test "$SOAK_DURATION" -lt 10 && SOAK_DURATION=10
> > 	fsstress_args+=(--duration="$((SOAK_DURATION / 10))")
> > else
> > 	# run for 3s per iteration max for a default runtime of ~30s.
> > 	fsstress_args+=(--duration=3)
> > fi
> 
> Yeah, that works for me.
> 
> As a rainy day project, perhaps we should look to convert all the
> fsstress invocations to be time bound rather than running a specific
> number of ops. i.e. hard code nr_ops=<some huge number> in
> _run_fstress_bg() and the tests only need to define parallelism and
> runtime.

I /think/ the only ones that do that are generic/1220 generic/476
generic/642 generic/750.  I could drop the nr_cpus term from the nr_ops
calculation.

> This would make the test runtimes more deterministic across machines
> with vastly different capabilities and and largely make "test xyz is
> slow on my test machine" reports largely go away.
> 
> Thoughts?

I'm fine with _run_fsstress injecting --duration=30 if no other duration
argument is passed in.

--D

> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 11/23] common/xfs: find loop devices for non-blockdevs passed to _prepare_for_eio_shutdown

[Dave Chinner]

On Tue, Jan 21, 2025 at 08:05:42PM -0800, Darrick J. Wong wrote:
> On Tue, Jan 21, 2025 at 03:37:17PM +1100, Dave Chinner wrote:
> > On Thu, Jan 16, 2025 at 03:28:02PM -0800, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > > 
> > > xfs/336 does this somewhat sketchy thing where it mdrestores into a
> > > regular file, and then does this to validate the restored metadata:
> > > 
> > > SCRATCH_DEV=$TEST_DIR/image _scratch_mount
> > 
> > That's a canonical example of what is called "stepping on a
> > landmine".
> 
> 60% of fstests is written in bash, all of it is a friggin land mine
> because bash totally lets us do variable substitution at any time, and
> any time you make a change you have to exhaustively test the whole mess
> to make sure nothing broke...

Yes, I know, which is why the moment I saw xfs/336 I called it out -
it has never run on my machines, ever...

> (Yeah, I hate bash)

Not a great fan of it myself. But it's no worse than other scripting
languages that use JIT based syntax checking from the "if it wasn't
run it ain't tested" perspective.

> > We validate that the SCRATCH_DEV is a block device at the start of
> > check and each section it reads and runs (via common/config), and
> > then make the assumption in all the infrastructure that SCRATCH_DEV
> > always points to a valid block device.
> 
> We do?

fstests configurations for block based filesystems have always been
based on block devices and mount points, not image files. 

Yes, you can pass an image file to XFS utilities and they will do
the right thing, but not all filesystems or all the infrastructure
in fstests can handle an image file masquerading as a device.

I certainly didn't expect it.....

> Can you point me to the sentence in doc/ that says this
> explicitly? 

fstests is woefully under-documented - especially when it comes to
configuration constraints and behaviours - so I doubt it is actually
specified anywhere. AFAIA it has never been raised in discussion
for a long time (not since we added network filesystem support a
long time ago, IIRC)

However, the code is pretty explicit - common/config is responsible
for setting up and validating the runtime config before any test can
run. All test and scratch devices are passed through this
validation:

_check_device()
{
        local name=$1
        local dev_needed=$2
        local dev=$3

        if [ -z "$dev" ]; then
                if [ "$dev_needed" == "required" ]; then
                        _fatal "common/config: $name is required but not defined!"
                fi
                return 0
        fi

        if [ -b "$dev" ] || ( echo $dev | grep -qE ":|//" ); then
                # block device or a network url
                return 0
	fi

	case "$FSTYP" in
        9p|fuse|tmpfs|virtiofs|afs)
	.....
	*)
                _fatal "common/config: $name ($dev) is not a block device or a network filesystem"
        esac
}
....

Basically, it says that all the test and scratch devices (including
the external ones) must be either a block device, a network URL, or
a string that the specific filesystem under test must recognise and
accept (e.g. a directory for overlay filesystems). Otherwise fstests
will fail to run with an explicit error message that says:

	<device> is not a block device or network filesystem

Nowhere in this config validation process does fstests consider
image files as a valid device configuration for a block based
filesystem.

If we need to do stuff with a image files, we have the
infrastructure to create loop devices and then operate directly on
that dynamic loop device(s) (e.g. _mkfs_dev, _mount, _unmount,
_check_xfs_filesystem, etc) that are created.

> There's nothing I can find in the any docs and
> _try_scratch_mount does not check SCRATCH_DEV is a bdev for XFS.

That's because it's validated before we start running tests and the
assumption is that nobody is screwing with SCRATCH_DEV in a way
that makes it behave vastly differently.

Consider what it means to have to do runtime checking of the
device validity in common code before we do anything with the
device. We'd have to sprinkle _check_device calls -everywhere-.

We'd also have to check logdev and rtdev variables if USE_EXTERNAL
is set, too.

That's not a viable development strategy, nor is it a maintainable
solution to the issue at hand. It's far simpler to fix one test not
to use this trick than it is to declare "nobody can trust TEST_DEV
or SCRATCH_DEV to be a valid block device" and have to handle that
everywhere those variables are used...

> That needs to be documented.

Sure.

> > > Fix this by detecting non-bdevs and finding (we hope) the loop device
> > > that was created to handle the mount. 
> > 
> > What loop device? xfs/336 doesn't use loop devices at all.
> > 
> > Oh, this is assuming that mount will silently do a loopback mount
> > when passed a file rather than a block device. IOWs, it's relying on
> > some third party to do the loop device creation and hence allow it
> > to be mounted.
> > 
> > IOWs, this change is addressing a landmine by adding another
> > landmine.
> 
> Some would say that mount adding the ability to set up a loop dev was
> itself *avoiding* a landmine from 90s era util-linux.

True.

But in the case of fstests we explicitly create loop
devices so that we don't have to play whacky games to find the
random loop device that mount magically creates when you pass it a
file.

Making all the image file and loop device usage consistent across
all of fstests was part of the infrastructure changes in my initial
check-parallel patchset. This was necessary because killing tests
with ctrl-c would randomly leave dangling mounts and loop devices
because many tests did not have _cleanup routines to tear down
mounts that auto-created loop devices or clean up loop
devices they created themselves properly.

Part of those changes was fixing up the mess in some XFS tests
where that mixed loop device and image file based operations
interchangably. I didn't notice x/336 because it wasn't
running on my test system and so didn't attempt to fix it at the
same time...

> > I really think that xfs/336 needs to be fixed - one off test hacks
> > like this, while they may work, only make modifying and maintaining
> > the fstests infrastructure that much harder....
> 
> Yeah, it'll get cleaned up for the rtrmap fstests merge.

Thanks!

-Dave.

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

[Dave Chinner]

On Tue, Jan 21, 2025 at 11:08:39PM -0500, Theodore Ts'o wrote:
> On Wed, Jan 22, 2025 at 09:15:48AM +1100, Dave Chinner wrote:
> > check-parallel on my 64p machine runs the full auto group test in
> > under 10 minutes.
> > 
> > i.e. if you have a typical modern server (64-128p, 256GB RAM and a
> > couple of NVMe SSDs), then check-parallel allows a full test run in
> > the same time that './check -g smoketest' will run....
> 
> Interesting.  I would have thought that even with NVMe SSD's, you'd be
> I/O speed constrained, especially given that some of the tests
> (especially the ENOSPC hitters) can take quite a lot of time to fill
> the storage device, even if they are using fallocate.

You haven't looked at how check-parallel works, have you? :/

> How do you have your test and scratch devices configured?

Please go and read the check-parallel script. It does all the
per-runner process test and scratch device configuration itself
using loop devices.

> > Yes, and I've previously made the point about how check-parallel
> > changes the way we should be looking at dev-test cycles. We no
> > longer have to care that auto group testing takes 4 hours to run and
> > have to work around that with things like smoketest groups. If you
> > can run the whole auto test group in 10-15 minutes, then we don't
> > need "quick", "smoketest", etc to reduce dev-test cycle time
> > anymore...
> 
> Well, yes, if the only consideration is test run time latency.

Sure.

> I can think of two off-setting considerations.  The first is if you
> care about cost.

Which I really don't care about.

That's something for a QE organisation to worry about, and it's up
to them to make the best use of the tools they have within the
budget they have.

> The second concern is that for certain class of failures (UBSAN,
> KCSAN, Lockdep, RCU soft lockups, WARN_ON, BUG_ON, and other
> panics/OOPS), if you are runnig 64 tests in parllel it might not be
> obvious which test caused the failure.

Then multiple tests will fail with the same dmesg error, but it's
generally pretty clear which of the tests caused it. Yes, it's a bit
more work to isolate the specific test, but it's not hard or any
different to how a test failure is debugged now.

If you want to automate such failures, then my process is to grep
the log files for all the tests that failed with a dmesg error then
run them again using check instead of check-parallel.  Then I get
exactly which test generated the dmesg output without having to put
time into trying to work out which test triggered the failure.

> Today, even if the test VM
> crashes or hangs, I can have test manager (which runs on a e2-small VM
> costing $0.021913 USD/hour and can manage dozens of test VM's all at the
> same time), can restart the test VM, and we know which test is at at
> fault, and we mark that a particular test with the Junit XML status of
> "error" (as distinct from "success" or "failure").  If there are 64
> test runs in parallel, if I wanted to have automated recovery if the
> test appliance hangs or crashes, life gets a lot more complicated.....

Not really. Both dmesg and the results files will have tracked all
the tests inflight when the system crashes, so it's just an extra
step to extract all those tests and run them again using check
and/or check-parallel to further isolate which test caused the
failure....

I'm sure this could be automated eventually, but that's way down my
priority list right now.

> I suppose we could have the human (or test automation) try run each
> individual test that had been running at the time of the crash but
> that's a lot more complicated, and what if the tests pass when run
> once at a time?  I guess we should happen that check-parallel found a
> bug that plain check didn't find, but the human being still has to
> root cause the failure.

Yes. This is no different to a test that is flakey or compeltely
fails when run serially by check multiple times. You still need a
human to find the root cause of the failure.

Nobody is being forced to change their tooling or processes to use
check-parallel if they don't want or need to. It is an alternative
method for running the tests within the fstests suite - if using
check meets your needs, there is no reason to use check-parallel or
even care that it exists...

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Dave Chinner]

On Tue, Jan 21, 2025 at 08:24:00PM -0800, Darrick J. Wong wrote:
> On Tue, Jan 21, 2025 at 02:28:26PM +1100, Dave Chinner wrote:
> > On Thu, Jan 16, 2025 at 03:27:15PM -0800, Darrick J. Wong wrote:
> > > c) Putting test subprocesses in a systemd sub-scope and telling systemd
> > > to kill the sub-scope could work because ./check can already use it to
> > > ensure that all child processes of a test are killed.  However, this is
> > > an *optional* feature, which means that we'd have to require systemd.
> > 
> > ... requiring systemd was somewhat of a show-stopper for testing
> > older distros.
> 
> Isn't RHEL7 the oldest one at this point?  And it does systemd.  At this
> point the only reason I didn't go full systemd is out of consideration
> for Devuan, since they probably need QA.

I have no idea what is out there in distro land vs what fstests
"supports". All I know is that there are distros out there that
don't use systemd.

It feels like poor form to prevent generic filesystem QA
infrastructure from running on those distros by making an avoidable
choice to tie the infrastructure exclusively to systemd-based
functionality....

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 13/23] generic/650: revert SOAK DURATION changes

[Darrick J. Wong]

On Wed, Jan 22, 2025 at 05:01:47PM +1100, Dave Chinner wrote:
> On Tue, Jan 21, 2025 at 11:08:39PM -0500, Theodore Ts'o wrote:
> > On Wed, Jan 22, 2025 at 09:15:48AM +1100, Dave Chinner wrote:
> > > check-parallel on my 64p machine runs the full auto group test in
> > > under 10 minutes.
> > > 
> > > i.e. if you have a typical modern server (64-128p, 256GB RAM and a
> > > couple of NVMe SSDs), then check-parallel allows a full test run in
> > > the same time that './check -g smoketest' will run....
> > 
> > Interesting.  I would have thought that even with NVMe SSD's, you'd be
> > I/O speed constrained, especially given that some of the tests
> > (especially the ENOSPC hitters) can take quite a lot of time to fill
> > the storage device, even if they are using fallocate.
> 
> You haven't looked at how check-parallel works, have you? :/
> 
> > How do you have your test and scratch devices configured?
> 
> Please go and read the check-parallel script. It does all the
> per-runner process test and scratch device configuration itself
> using loop devices.
> 
> > > Yes, and I've previously made the point about how check-parallel
> > > changes the way we should be looking at dev-test cycles. We no
> > > longer have to care that auto group testing takes 4 hours to run and
> > > have to work around that with things like smoketest groups. If you
> > > can run the whole auto test group in 10-15 minutes, then we don't
> > > need "quick", "smoketest", etc to reduce dev-test cycle time
> > > anymore...
> > 
> > Well, yes, if the only consideration is test run time latency.
> 
> Sure.
> 
> > I can think of two off-setting considerations.  The first is if you
> > care about cost.
> 
> Which I really don't care about.
> 
> That's something for a QE organisation to worry about, and it's up
> to them to make the best use of the tools they have within the
> budget they have.
> 
> > The second concern is that for certain class of failures (UBSAN,
> > KCSAN, Lockdep, RCU soft lockups, WARN_ON, BUG_ON, and other
> > panics/OOPS), if you are runnig 64 tests in parllel it might not be
> > obvious which test caused the failure.
> 
> Then multiple tests will fail with the same dmesg error, but it's
> generally pretty clear which of the tests caused it. Yes, it's a bit
> more work to isolate the specific test, but it's not hard or any
> different to how a test failure is debugged now.
> 
> If you want to automate such failures, then my process is to grep
> the log files for all the tests that failed with a dmesg error then
> run them again using check instead of check-parallel.  Then I get
> exactly which test generated the dmesg output without having to put
> time into trying to work out which test triggered the failure.
> 
> > Today, even if the test VM
> > crashes or hangs, I can have test manager (which runs on a e2-small VM
> > costing $0.021913 USD/hour and can manage dozens of test VM's all at the
> > same time), can restart the test VM, and we know which test is at at
> > fault, and we mark that a particular test with the Junit XML status of
> > "error" (as distinct from "success" or "failure").  If there are 64
> > test runs in parallel, if I wanted to have automated recovery if the
> > test appliance hangs or crashes, life gets a lot more complicated.....
> 
> Not really. Both dmesg and the results files will have tracked all
> the tests inflight when the system crashes, so it's just an extra
> step to extract all those tests and run them again using check
> and/or check-parallel to further isolate which test caused the
> failure....

That reminds me to go see if ./check actually fsyncs the state and
report files and whatnot between tests, so that we have a better chance
of figuring out where exactly fstests blew up the machine.

(Luckily xfs is stable enough I haven't had a machine explode in quite
some time, good job everyone! :))

--D

> I'm sure this could be automated eventually, but that's way down my
> priority list right now.
> 
> > I suppose we could have the human (or test automation) try run each
> > individual test that had been running at the time of the crash but
> > that's a lot more complicated, and what if the tests pass when run
> > once at a time?  I guess we should happen that check-parallel found a
> > bug that plain check didn't find, but the human being still has to
> > root cause the failure.
> 
> Yes. This is no different to a test that is flakey or compeltely
> fails when run serially by check multiple times. You still need a
> human to find the root cause of the failure.
> 
> Nobody is being forced to change their tooling or processes to use
> check-parallel if they don't want or need to. It is an alternative
> method for running the tests within the fstests suite - if using
> check meets your needs, there is no reason to use check-parallel or
> even care that it exists...
> 
> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Darrick J. Wong]

On Wed, Jan 22, 2025 at 05:08:17PM +1100, Dave Chinner wrote:
> On Tue, Jan 21, 2025 at 08:24:00PM -0800, Darrick J. Wong wrote:
> > On Tue, Jan 21, 2025 at 02:28:26PM +1100, Dave Chinner wrote:
> > > On Thu, Jan 16, 2025 at 03:27:15PM -0800, Darrick J. Wong wrote:
> > > > c) Putting test subprocesses in a systemd sub-scope and telling systemd
> > > > to kill the sub-scope could work because ./check can already use it to
> > > > ensure that all child processes of a test are killed.  However, this is
> > > > an *optional* feature, which means that we'd have to require systemd.
> > > 
> > > ... requiring systemd was somewhat of a show-stopper for testing
> > > older distros.
> > 
> > Isn't RHEL7 the oldest one at this point?  And it does systemd.  At this
> > point the only reason I didn't go full systemd is out of consideration
> > for Devuan, since they probably need QA.
> 
> I have no idea what is out there in distro land vs what fstests
> "supports". All I know is that there are distros out there that
> don't use systemd.
> 
> It feels like poor form to prevent generic filesystem QA
> infrastructure from running on those distros by making an avoidable
> choice to tie the infrastructure exclusively to systemd-based
> functionality....

Agreed, though at some point after these bugfixes are merged I'll see if
I can build on the existing "if you have systemd then ___ else here's
your shabby opencoded version" logic in fstests to isolate the ./checks
from each other a little better.  It'd be kinda nice if we could
actually just put them in something resembling a modernish container,
albeit with the same underlying fs.

<shrug> Anyone else interested in that?

--D

> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Dave Chinner]

On Tue, Jan 21, 2025 at 11:05:20PM -0800, Darrick J. Wong wrote:
> On Wed, Jan 22, 2025 at 05:08:17PM +1100, Dave Chinner wrote:
> > On Tue, Jan 21, 2025 at 08:24:00PM -0800, Darrick J. Wong wrote:
> > > On Tue, Jan 21, 2025 at 02:28:26PM +1100, Dave Chinner wrote:
> > > > On Thu, Jan 16, 2025 at 03:27:15PM -0800, Darrick J. Wong wrote:
> > > > > c) Putting test subprocesses in a systemd sub-scope and telling systemd
> > > > > to kill the sub-scope could work because ./check can already use it to
> > > > > ensure that all child processes of a test are killed.  However, this is
> > > > > an *optional* feature, which means that we'd have to require systemd.
> > > > 
> > > > ... requiring systemd was somewhat of a show-stopper for testing
> > > > older distros.
> > > 
> > > Isn't RHEL7 the oldest one at this point?  And it does systemd.  At this
> > > point the only reason I didn't go full systemd is out of consideration
> > > for Devuan, since they probably need QA.
> > 
> > I have no idea what is out there in distro land vs what fstests
> > "supports". All I know is that there are distros out there that
> > don't use systemd.
> > 
> > It feels like poor form to prevent generic filesystem QA
> > infrastructure from running on those distros by making an avoidable
> > choice to tie the infrastructure exclusively to systemd-based
> > functionality....
> 
> Agreed, though at some point after these bugfixes are merged I'll see if
> I can build on the existing "if you have systemd then ___ else here's
> your shabby opencoded version" logic in fstests to isolate the ./checks
> from each other a little better.  It'd be kinda nice if we could
> actually just put them in something resembling a modernish container,
> albeit with the same underlying fs.

Agreed, but I don't think we need to depend on systemd for that,
either.

> <shrug> Anyone else interested in that?

check-parallel has already started down that road with the
mount namespace isolation it uses for the runner tasks via
src/nsexec.c.

My plan has been to factor more of the check test running code
(similar to what I did with the test list parsing) so that the
check-parallel can iterate sections itself and runners can execute
individual tests directly, rather than bouncing them through check
to execute a set of tests serially. Then check-parallel could do
whatever it needed to isolate individual tests from each other and
nothing in check would need to change.

Now I'm wondering if I can just run each runner's check instance
in it's own private PID namespace as easily as I'm running them in
their own private mount namespace...

Hmmm - looks like src/nsexec.c can create new PID namespaces via
the "-p" option. I haven't used that before - I wonder if that's a
better solution that using per-test session IDs to solve the pkill
--parent problem? Something to look into in the morning....

-Dave.

-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Darrick J. Wong]

On Wed, Jan 22, 2025 at 08:42:49PM +1100, Dave Chinner wrote:
> On Tue, Jan 21, 2025 at 11:05:20PM -0800, Darrick J. Wong wrote:
> > On Wed, Jan 22, 2025 at 05:08:17PM +1100, Dave Chinner wrote:
> > > On Tue, Jan 21, 2025 at 08:24:00PM -0800, Darrick J. Wong wrote:
> > > > On Tue, Jan 21, 2025 at 02:28:26PM +1100, Dave Chinner wrote:
> > > > > On Thu, Jan 16, 2025 at 03:27:15PM -0800, Darrick J. Wong wrote:
> > > > > > c) Putting test subprocesses in a systemd sub-scope and telling systemd
> > > > > > to kill the sub-scope could work because ./check can already use it to
> > > > > > ensure that all child processes of a test are killed.  However, this is
> > > > > > an *optional* feature, which means that we'd have to require systemd.
> > > > > 
> > > > > ... requiring systemd was somewhat of a show-stopper for testing
> > > > > older distros.
> > > > 
> > > > Isn't RHEL7 the oldest one at this point?  And it does systemd.  At this
> > > > point the only reason I didn't go full systemd is out of consideration
> > > > for Devuan, since they probably need QA.
> > > 
> > > I have no idea what is out there in distro land vs what fstests
> > > "supports". All I know is that there are distros out there that
> > > don't use systemd.
> > > 
> > > It feels like poor form to prevent generic filesystem QA
> > > infrastructure from running on those distros by making an avoidable
> > > choice to tie the infrastructure exclusively to systemd-based
> > > functionality....
> > 
> > Agreed, though at some point after these bugfixes are merged I'll see if
> > I can build on the existing "if you have systemd then ___ else here's
> > your shabby opencoded version" logic in fstests to isolate the ./checks
> > from each other a little better.  It'd be kinda nice if we could
> > actually just put them in something resembling a modernish container,
> > albeit with the same underlying fs.
> 
> Agreed, but I don't think we need to depend on systemd for that,
> either.
> 
> > <shrug> Anyone else interested in that?
> 
> check-parallel has already started down that road with the
> mount namespace isolation it uses for the runner tasks via
> src/nsexec.c.
> 
> My plan has been to factor more of the check test running code
> (similar to what I did with the test list parsing) so that the
> check-parallel can iterate sections itself and runners can execute
> individual tests directly, rather than bouncing them through check
> to execute a set of tests serially. Then check-parallel could do
> whatever it needed to isolate individual tests from each other and
> nothing in check would need to change.
> 
> Now I'm wondering if I can just run each runner's check instance
> in it's own private PID namespace as easily as I'm running them in
> their own private mount namespace...
> 
> Hmmm - looks like src/nsexec.c can create new PID namespaces via
> the "-p" option. I haven't used that before - I wonder if that's a
> better solution that using per-test session IDs to solve the pkill
> --parent problem? Something to look into in the morning....

I tried that -- it appears to work, but then:

# ./src/nsexec -p bash
Current time: Wed Jan 22 13:43:33 PST 2025; Terminal: /dev/pts/0
# ps
fatal library error, lookup self
# 

Regular processes and pkill/pgrep seem to work, but that didn't seem
especially encouraging so I stopped. :/

--D

> -Dave.
> 
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Dave Chinner]

On Wed, Jan 22, 2025 at 01:46:09PM -0800, Darrick J. Wong wrote:
> On Wed, Jan 22, 2025 at 08:42:49PM +1100, Dave Chinner wrote:
> > On Tue, Jan 21, 2025 at 11:05:20PM -0800, Darrick J. Wong wrote:
> > > On Wed, Jan 22, 2025 at 05:08:17PM +1100, Dave Chinner wrote:
> > > > On Tue, Jan 21, 2025 at 08:24:00PM -0800, Darrick J. Wong wrote:
> > > > > On Tue, Jan 21, 2025 at 02:28:26PM +1100, Dave Chinner wrote:
> > > > > > On Thu, Jan 16, 2025 at 03:27:15PM -0800, Darrick J. Wong wrote:
> > > > > > > c) Putting test subprocesses in a systemd sub-scope and telling systemd
> > > > > > > to kill the sub-scope could work because ./check can already use it to
> > > > > > > ensure that all child processes of a test are killed.  However, this is
> > > > > > > an *optional* feature, which means that we'd have to require systemd.
> > > > > > 
> > > > > > ... requiring systemd was somewhat of a show-stopper for testing
> > > > > > older distros.
> > > > > 
> > > > > Isn't RHEL7 the oldest one at this point?  And it does systemd.  At this
> > > > > point the only reason I didn't go full systemd is out of consideration
> > > > > for Devuan, since they probably need QA.
> > > > 
> > > > I have no idea what is out there in distro land vs what fstests
> > > > "supports". All I know is that there are distros out there that
> > > > don't use systemd.
> > > > 
> > > > It feels like poor form to prevent generic filesystem QA
> > > > infrastructure from running on those distros by making an avoidable
> > > > choice to tie the infrastructure exclusively to systemd-based
> > > > functionality....
> > > 
> > > Agreed, though at some point after these bugfixes are merged I'll see if
> > > I can build on the existing "if you have systemd then ___ else here's
> > > your shabby opencoded version" logic in fstests to isolate the ./checks
> > > from each other a little better.  It'd be kinda nice if we could
> > > actually just put them in something resembling a modernish container,
> > > albeit with the same underlying fs.
> > 
> > Agreed, but I don't think we need to depend on systemd for that,
> > either.
> > 
> > > <shrug> Anyone else interested in that?
> > 
> > check-parallel has already started down that road with the
> > mount namespace isolation it uses for the runner tasks via
> > src/nsexec.c.
> > 
> > My plan has been to factor more of the check test running code
> > (similar to what I did with the test list parsing) so that the
> > check-parallel can iterate sections itself and runners can execute
> > individual tests directly, rather than bouncing them through check
> > to execute a set of tests serially. Then check-parallel could do
> > whatever it needed to isolate individual tests from each other and
> > nothing in check would need to change.
> > 
> > Now I'm wondering if I can just run each runner's check instance
> > in it's own private PID namespace as easily as I'm running them in
> > their own private mount namespace...
> > 
> > Hmmm - looks like src/nsexec.c can create new PID namespaces via
> > the "-p" option. I haven't used that before - I wonder if that's a
> > better solution that using per-test session IDs to solve the pkill
> > --parent problem? Something to look into in the morning....
> 
> I tried that -- it appears to work, but then:
> 
> # ./src/nsexec -p bash
> Current time: Wed Jan 22 13:43:33 PST 2025; Terminal: /dev/pts/0
> # ps
> fatal library error, lookup self
> # 

That looks like a bug in whatever distro you are using - it works
as it should here on a recent debian unstable userspace.

Note, however, that ps will show all processes in both the parent
and the child namespace the shell is running on because the contents
of /proc are the same for both.

However, because we are also using private mount namespaces for the
check process, pid_namespaces(7) tells us:

/proc and PID namespaces

       A /proc filesystem shows (in the /proc/pid directories) only
       processes visible in the PID namespace of the process that
       performed the mount, even if the /proc filesystem is viewed
       from processes in other namespaces.

       After creating a new PID namespace, it is useful for the
       child to change its root directory and mount a new procfs
       instance at /proc so that tools  such  as  ps(1) work
>>>    correctly.  If a new mount namespace is simultaneously
>>>    created by including CLONE_NEWNS in the flags argument of
>>>    clone(2) or unshare(2), then it isn't necessary to change the
>>>    root directory: a new procfs instance can be mounted directly
>>>    over /proc.

       From a shell, the command to mount /proc is:

           $ mount -t proc proc /proc

       Calling readlink(2) on the path /proc/self yields the process
       ID of the caller in the PID namespace of the procfs mount
       (i.e., the PID name‐ space of the process that mounted the
       procfs).  This can be useful for introspection purposes, when
       a process wants to discover its  PID  in other namespaces.

This appears to give us an environment that only shows the processes
within the current PID namespace:

$ sudo src/nsexec -p -m bash
# mount -t proc proc /proc
# ps waux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           1  0.0  0.0   7384  3744 pts/1    S    11:55   0:00 bash
root          72  0.0  0.0   8300  3736 pts/1    R+   12:04   0:00 ps waux
# pstree -N pid
[4026538173]
bash───pstree
#

Yup, there we go - we have full PID isolation for this shell.

OK, I suspect this is a better way to proceed for check-parallel than
to need session IDs for individual tests and wrappers for
pgrep/pkill/pidwait. Let me see what breaks when I use this.....

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Dave Chinner]

On Thu, Jan 23, 2025 at 12:16:50PM +1100, Dave Chinner wrote:
> On Wed, Jan 22, 2025 at 01:46:09PM -0800, Darrick J. Wong wrote:
> > > > Agreed, though at some point after these bugfixes are merged I'll see if
> > > > I can build on the existing "if you have systemd then ___ else here's
> > > > your shabby opencoded version" logic in fstests to isolate the ./checks
> > > > from each other a little better.  It'd be kinda nice if we could
> > > > actually just put them in something resembling a modernish container,
> > > > albeit with the same underlying fs.
> > > 
> > > Agreed, but I don't think we need to depend on systemd for that,
> > > either.
> > > 
> > > > <shrug> Anyone else interested in that?
> > > 
> > > check-parallel has already started down that road with the
> > > mount namespace isolation it uses for the runner tasks via
> > > src/nsexec.c.
.....
> > > Hmmm - looks like src/nsexec.c can create new PID namespaces via
> > > the "-p" option. I haven't used that before - I wonder if that's a
> > > better solution that using per-test session IDs to solve the pkill
> > > --parent problem? Something to look into in the morning....

.....

> Note, however, that ps will show all processes in both the parent
> and the child namespace the shell is running on because the contents
> of /proc are the same for both.
> 
> However, because we are also using private mount namespaces for the
> check process, pid_namespaces(7) tells us:
> 
> /proc and PID namespaces
> 
>        A /proc filesystem shows (in the /proc/pid directories) only
>        processes visible in the PID namespace of the process that
>        performed the mount, even if the /proc filesystem is viewed
>        from processes in other namespaces.
> 
>        After creating a new PID namespace, it is useful for the
>        child to change its root directory and mount a new procfs
>        instance at /proc so that tools  such  as  ps(1) work
> >>>    correctly.  If a new mount namespace is simultaneously
> >>>    created by including CLONE_NEWNS in the flags argument of
> >>>    clone(2) or unshare(2), then it isn't necessary to change the
> >>>    root directory: a new procfs instance can be mounted directly
> >>>    over /proc.
> 
>        From a shell, the command to mount /proc is:
> 
>            $ mount -t proc proc /proc
> 
>        Calling readlink(2) on the path /proc/self yields the process
>        ID of the caller in the PID namespace of the procfs mount
>        (i.e., the PID name‐ space of the process that mounted the
>        procfs).  This can be useful for introspection purposes, when
>        a process wants to discover its  PID  in other namespaces.
> 
> This appears to give us an environment that only shows the processes
> within the current PID namespace:
> 
> $ sudo src/nsexec -p -m bash
> # mount -t proc proc /proc
> # ps waux
> USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> root           1  0.0  0.0   7384  3744 pts/1    S    11:55   0:00 bash
> root          72  0.0  0.0   8300  3736 pts/1    R+   12:04   0:00 ps waux
> # pstree -N pid
> [4026538173]
> bash───pstree
> #
> 
> Yup, there we go - we have full PID isolation for this shell.

Ok, it took me some time to get this to work reliably - the way was
full of landmines with little documentation to guide around them.

1. If multiple commands are needed to be run from nsexec, a helper
   script is needed (I called it check-helper).

2. You have to `mount --make-private /proc` before doing anything to
   make the mounts of /proc inside the pid namespace work correctly.
   If you don't do this, every other mount namespace will also see
   only the newest mount, and that means every runner but the last
   one to start with the wrong mount and PID namespace view in /proc.

3. if you get the system into the state described by 1), unmounting
   /proc in each runner then races to remove the top /proc mount.
   Many threads get -EBUSY failures from unmount, leaving many stale
   mounts of /proc behind.

4. the top level shell where check-parallel was invoked is left with
   the view where /proc has been unmounted from a PID/mount
   namespace that has gone away. Hence /proc now has no processes or
   mounts being reported and nothing works until you mount a new
   instance /proc in that namespace.

5. After mounting proc again there are lots of mounts of stale /proc
   mounts still reported. They cannot be unmounted as the mount
   namespaces that created them have gone away and unmounting /proc
   in the current shell simply removes the last mounted one and we
   goto 4).

4. /tmp is still shared across all runner instances so all the

   concurrent runners dump all their tmp files in /tmp. However, the
   runners no longer have unique PIDs (i.e. check runs as PID 3 in
   all runner instaces). This means using /tmp/tmp.$$ as the
   check/test temp file definition results is instant tmp file name
   collisions and random things in check and tests fail.  check and
   common/preamble have to be converted to use `mktemp` to provide
   unique tempfile name prefixes again.

5. Don't forget to revert the parent /proc mount back to shared
   after check has finished running (or was aborted).

I think with this (current prototype patch below), we can use PID
namespaces rather than process session IDs for check-parallel safe
process management.

Thoughts?

-Dave.
-- 
Dave Chinner
david@fromorbit.com


check-parallel: use PID namespaces for runner process isolation

From: Dave Chinner <dchinner@redhat.com>

This provides isolation between individual runners so that they
cannot see the processes that other test runners have created.
This means tools like pkill will only find processes run by the test
that is calling it, hence there is no danger taht it might kill
processes owned by a different test in a different runner context.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 check           |  6 +++++-
 check-helper    | 23 +++++++++++++++++++++++
 check-parallel  | 25 +++++++++++++++++++------
 common/preamble |  5 ++++-
 4 files changed, 51 insertions(+), 8 deletions(-)

diff --git a/check b/check
index 4dc266dcf..314436667 100755
--- a/check
+++ b/check
@@ -4,7 +4,11 @@
 #
 # Control script for QA
 #
-tmp=/tmp/$$
+
+# When run from a pid namespace, /tmp/tmp.$$ is not a unique identifier.
+# Use mktemp instead.
+tmp=`mktemp`
+
 status=0
 needwrap=true
 needsum=true
diff --git a/check-helper b/check-helper
new file mode 100755
index 000000000..47a92de8b
--- /dev/null
+++ b/check-helper
@@ -0,0 +1,23 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Red Hat, Inc.  All Rights Reserved.
+#
+
+# When check is run from check-parallel, it is run in private mount and PID
+# namespaces. We need to set up /proc to only show processes in this PID
+# namespace, so we mount a new instance of /proc over the top of the existing
+# version. Because we are in a private mount namespace, the does not propagate
+# outside this context and hence does not conflict with the other test runners
+# that are performing the same setup actions.
+
+args="$@"
+
+#echo $args
+mount -t proc proc /proc
+ret=$?
+if [ $ret -eq 0 ]; then
+	./check $args
+	umount -l /proc
+else
+	echo failed to mount /proc, ret $ret!
+fi
diff --git a/check-parallel b/check-parallel
index 2fbf0fdbe..6082baf5e 100755
--- a/check-parallel
+++ b/check-parallel
@@ -259,14 +259,14 @@ runner_go()
 	rm -f $RESULT_BASE/check.*
 
 	# Only supports default mkfs parameters right now
-	wipefs -a $TEST_DEV > /dev/null 2>&1
-	yes | mkfs -t $FSTYP $TEST_MKFS_OPTS $TEST_DEV > /dev/null 2>&1
+	wipefs -a $TEST_DEV > $me/log 2>&1
+	yes | mkfs -t $FSTYP $TEST_MKFS_OPTS $TEST_DEV >> $me/log 2>&1
 
 #	export DUMP_CORRUPT_FS=1
 
-	# Run the tests in it's own mount namespace, as per the comment below
-	# that precedes making the basedir a private mount.
-	./src/nsexec -m ./check $run_section -x unreliable_in_parallel --exact-order ${runner_list[$id]} > $me/log 2>&1
+	# Run the tests in it's own mount and PID namespace, as per the comment
+	# below that precedes making the basedir a private mount.
+	./src/nsexec -m -p ./check-helper $run_section -x unreliable_in_parallel --exact-order ${runner_list[$id]} >> $me/log 2>&1
 
 	wait
 	sleep 1
@@ -291,6 +291,8 @@ cleanup()
 	umount -R $basedir/*/test 2> /dev/null
 	umount -R $basedir/*/scratch 2> /dev/null
 	losetup --detach-all
+	mount --make-shared /proc
+	mount --make-shared $basedir
 }
 
 trap "cleanup; exit" HUP INT QUIT TERM
@@ -311,10 +313,17 @@ fi
 # controls the mount to succeed without actually unmounting the filesytsem
 # because a mount namespace still holds a reference to it. This causes other
 # operations on the block device to fail as it is still busy (e.g. fsck, mkfs,
-# etc). Hence we make the basedir private here and then run each check instance
+# etc).
+#
+# Hence we make the basedir private here and then run each check instance
 # in it's own mount namespace so that they cannot see mounts that other tests
 # are performing.
+#
+# We also need to make /proc private so that the runners can be run cleanly in
+# a PID namespace. This requires an new mount of /proc inside the PID namespace,
+# and this requires a private mount namespace to work correctly.
 mount --make-private $basedir
+mount --make-private /proc
 
 now=`date +%Y-%m-%d-%H:%M:%S`
 for ((i = 0; i < $runners; i++)); do
@@ -328,6 +337,10 @@ for ((i = 0; i < $runners; i++)); do
 done;
 wait
 
+# Restore the parents back to shared mount namespace behaviour.
+mount --make-shared /proc
+mount --make-shared $basedir
+
 if [ -n "$show_test_list" ]; then
 	exit 0
 fi
diff --git a/common/preamble b/common/preamble
index 78e45d522..8f47b172a 100644
--- a/common/preamble
+++ b/common/preamble
@@ -43,8 +43,11 @@ _begin_fstest()
 	seqres=$RESULT_DIR/$seq
 	echo "QA output created by $seq"
 
+	# When run from a pid namespace, /tmp/tmp.$$ is not a unique identifier.
+	# Use mktemp instead.
+	tmp=`mktemp`
+
 	here=`pwd`
-	tmp=/tmp/$$
 	status=1	# failure is the default!
 
 	_register_cleanup _cleanup

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Darrick J. Wong]

On Tue, Jan 28, 2025 at 03:34:50PM +1100, Dave Chinner wrote:
> On Thu, Jan 23, 2025 at 12:16:50PM +1100, Dave Chinner wrote:
> > On Wed, Jan 22, 2025 at 01:46:09PM -0800, Darrick J. Wong wrote:
> > > > > Agreed, though at some point after these bugfixes are merged I'll see if
> > > > > I can build on the existing "if you have systemd then ___ else here's
> > > > > your shabby opencoded version" logic in fstests to isolate the ./checks
> > > > > from each other a little better.  It'd be kinda nice if we could
> > > > > actually just put them in something resembling a modernish container,
> > > > > albeit with the same underlying fs.
> > > > 
> > > > Agreed, but I don't think we need to depend on systemd for that,
> > > > either.
> > > > 
> > > > > <shrug> Anyone else interested in that?
> > > > 
> > > > check-parallel has already started down that road with the
> > > > mount namespace isolation it uses for the runner tasks via
> > > > src/nsexec.c.
> .....
> > > > Hmmm - looks like src/nsexec.c can create new PID namespaces via
> > > > the "-p" option. I haven't used that before - I wonder if that's a
> > > > better solution that using per-test session IDs to solve the pkill
> > > > --parent problem? Something to look into in the morning....
> 
> .....
> 
> > Note, however, that ps will show all processes in both the parent
> > and the child namespace the shell is running on because the contents
> > of /proc are the same for both.
> > 
> > However, because we are also using private mount namespaces for the
> > check process, pid_namespaces(7) tells us:
> > 
> > /proc and PID namespaces
> > 
> >        A /proc filesystem shows (in the /proc/pid directories) only
> >        processes visible in the PID namespace of the process that
> >        performed the mount, even if the /proc filesystem is viewed
> >        from processes in other namespaces.
> > 
> >        After creating a new PID namespace, it is useful for the
> >        child to change its root directory and mount a new procfs
> >        instance at /proc so that tools  such  as  ps(1) work
> > >>>    correctly.  If a new mount namespace is simultaneously
> > >>>    created by including CLONE_NEWNS in the flags argument of
> > >>>    clone(2) or unshare(2), then it isn't necessary to change the
> > >>>    root directory: a new procfs instance can be mounted directly
> > >>>    over /proc.
> > 
> >        From a shell, the command to mount /proc is:
> > 
> >            $ mount -t proc proc /proc
> > 
> >        Calling readlink(2) on the path /proc/self yields the process
> >        ID of the caller in the PID namespace of the procfs mount
> >        (i.e., the PID name‐ space of the process that mounted the
> >        procfs).  This can be useful for introspection purposes, when
> >        a process wants to discover its  PID  in other namespaces.
> > 
> > This appears to give us an environment that only shows the processes
> > within the current PID namespace:
> > 
> > $ sudo src/nsexec -p -m bash
> > # mount -t proc proc /proc
> > # ps waux
> > USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > root           1  0.0  0.0   7384  3744 pts/1    S    11:55   0:00 bash
> > root          72  0.0  0.0   8300  3736 pts/1    R+   12:04   0:00 ps waux
> > # pstree -N pid
> > [4026538173]
> > bash───pstree
> > #
> > 
> > Yup, there we go - we have full PID isolation for this shell.
> 
> Ok, it took me some time to get this to work reliably - the way was
> full of landmines with little documentation to guide around them.
> 
> 1. If multiple commands are needed to be run from nsexec, a helper
>    script is needed (I called it check-helper).
> 
> 2. You have to `mount --make-private /proc` before doing anything to
>    make the mounts of /proc inside the pid namespace work correctly.
>    If you don't do this, every other mount namespace will also see
>    only the newest mount, and that means every runner but the last
>    one to start with the wrong mount and PID namespace view in /proc.
> 
> 3. if you get the system into the state described by 1), unmounting
>    /proc in each runner then races to remove the top /proc mount.
>    Many threads get -EBUSY failures from unmount, leaving many stale
>    mounts of /proc behind.
> 
> 4. the top level shell where check-parallel was invoked is left with
>    the view where /proc has been unmounted from a PID/mount
>    namespace that has gone away. Hence /proc now has no processes or
>    mounts being reported and nothing works until you mount a new
>    instance /proc in that namespace.
> 
> 5. After mounting proc again there are lots of mounts of stale /proc
>    mounts still reported. They cannot be unmounted as the mount
>    namespaces that created them have gone away and unmounting /proc
>    in the current shell simply removes the last mounted one and we
>    goto 4).
> 
> 4. /tmp is still shared across all runner instances so all the
> 
>    concurrent runners dump all their tmp files in /tmp. However, the
>    runners no longer have unique PIDs (i.e. check runs as PID 3 in
>    all runner instaces). This means using /tmp/tmp.$$ as the
>    check/test temp file definition results is instant tmp file name
>    collisions and random things in check and tests fail.  check and
>    common/preamble have to be converted to use `mktemp` to provide
>    unique tempfile name prefixes again.
> 
> 5. Don't forget to revert the parent /proc mount back to shared
>    after check has finished running (or was aborted).
> 
> I think with this (current prototype patch below), we can use PID
> namespaces rather than process session IDs for check-parallel safe
> process management.
> 
> Thoughts?

Was about to go to bed, but can we also start a new mount namespace,
create a private (or at least non-global) /tmp to put files into, and
then each test instance is isolated from clobbering the /tmpfiles of
other ./check instances *and* the rest of the system?

--D

> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 
> 
> check-parallel: use PID namespaces for runner process isolation
> 
> From: Dave Chinner <dchinner@redhat.com>
> 
> This provides isolation between individual runners so that they
> cannot see the processes that other test runners have created.
> This means tools like pkill will only find processes run by the test
> that is calling it, hence there is no danger taht it might kill
> processes owned by a different test in a different runner context.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>  check           |  6 +++++-
>  check-helper    | 23 +++++++++++++++++++++++
>  check-parallel  | 25 +++++++++++++++++++------
>  common/preamble |  5 ++++-
>  4 files changed, 51 insertions(+), 8 deletions(-)
> 
> diff --git a/check b/check
> index 4dc266dcf..314436667 100755
> --- a/check
> +++ b/check
> @@ -4,7 +4,11 @@
>  #
>  # Control script for QA
>  #
> -tmp=/tmp/$$
> +
> +# When run from a pid namespace, /tmp/tmp.$$ is not a unique identifier.
> +# Use mktemp instead.
> +tmp=`mktemp`
> +
>  status=0
>  needwrap=true
>  needsum=true
> diff --git a/check-helper b/check-helper
> new file mode 100755
> index 000000000..47a92de8b
> --- /dev/null
> +++ b/check-helper
> @@ -0,0 +1,23 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2025 Red Hat, Inc.  All Rights Reserved.
> +#
> +
> +# When check is run from check-parallel, it is run in private mount and PID
> +# namespaces. We need to set up /proc to only show processes in this PID
> +# namespace, so we mount a new instance of /proc over the top of the existing
> +# version. Because we are in a private mount namespace, the does not propagate
> +# outside this context and hence does not conflict with the other test runners
> +# that are performing the same setup actions.
> +
> +args="$@"
> +
> +#echo $args
> +mount -t proc proc /proc
> +ret=$?
> +if [ $ret -eq 0 ]; then
> +	./check $args
> +	umount -l /proc
> +else
> +	echo failed to mount /proc, ret $ret!
> +fi
> diff --git a/check-parallel b/check-parallel
> index 2fbf0fdbe..6082baf5e 100755
> --- a/check-parallel
> +++ b/check-parallel
> @@ -259,14 +259,14 @@ runner_go()
>  	rm -f $RESULT_BASE/check.*
>  
>  	# Only supports default mkfs parameters right now
> -	wipefs -a $TEST_DEV > /dev/null 2>&1
> -	yes | mkfs -t $FSTYP $TEST_MKFS_OPTS $TEST_DEV > /dev/null 2>&1
> +	wipefs -a $TEST_DEV > $me/log 2>&1
> +	yes | mkfs -t $FSTYP $TEST_MKFS_OPTS $TEST_DEV >> $me/log 2>&1
>  
>  #	export DUMP_CORRUPT_FS=1
>  
> -	# Run the tests in it's own mount namespace, as per the comment below
> -	# that precedes making the basedir a private mount.
> -	./src/nsexec -m ./check $run_section -x unreliable_in_parallel --exact-order ${runner_list[$id]} > $me/log 2>&1
> +	# Run the tests in it's own mount and PID namespace, as per the comment
> +	# below that precedes making the basedir a private mount.
> +	./src/nsexec -m -p ./check-helper $run_section -x unreliable_in_parallel --exact-order ${runner_list[$id]} >> $me/log 2>&1
>  
>  	wait
>  	sleep 1
> @@ -291,6 +291,8 @@ cleanup()
>  	umount -R $basedir/*/test 2> /dev/null
>  	umount -R $basedir/*/scratch 2> /dev/null
>  	losetup --detach-all
> +	mount --make-shared /proc
> +	mount --make-shared $basedir
>  }
>  
>  trap "cleanup; exit" HUP INT QUIT TERM
> @@ -311,10 +313,17 @@ fi
>  # controls the mount to succeed without actually unmounting the filesytsem
>  # because a mount namespace still holds a reference to it. This causes other
>  # operations on the block device to fail as it is still busy (e.g. fsck, mkfs,
> -# etc). Hence we make the basedir private here and then run each check instance
> +# etc).
> +#
> +# Hence we make the basedir private here and then run each check instance
>  # in it's own mount namespace so that they cannot see mounts that other tests
>  # are performing.
> +#
> +# We also need to make /proc private so that the runners can be run cleanly in
> +# a PID namespace. This requires an new mount of /proc inside the PID namespace,
> +# and this requires a private mount namespace to work correctly.
>  mount --make-private $basedir
> +mount --make-private /proc
>  
>  now=`date +%Y-%m-%d-%H:%M:%S`
>  for ((i = 0; i < $runners; i++)); do
> @@ -328,6 +337,10 @@ for ((i = 0; i < $runners; i++)); do
>  done;
>  wait
>  
> +# Restore the parents back to shared mount namespace behaviour.
> +mount --make-shared /proc
> +mount --make-shared $basedir
> +
>  if [ -n "$show_test_list" ]; then
>  	exit 0
>  fi
> diff --git a/common/preamble b/common/preamble
> index 78e45d522..8f47b172a 100644
> --- a/common/preamble
> +++ b/common/preamble
> @@ -43,8 +43,11 @@ _begin_fstest()
>  	seqres=$RESULT_DIR/$seq
>  	echo "QA output created by $seq"
>  
> +	# When run from a pid namespace, /tmp/tmp.$$ is not a unique identifier.
> +	# Use mktemp instead.
> +	tmp=`mktemp`
> +
>  	here=`pwd`
> -	tmp=/tmp/$$
>  	status=1	# failure is the default!
>  
>  	_register_cleanup _cleanup

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Dave Chinner]

On Mon, Jan 27, 2025 at 11:23:52PM -0800, Darrick J. Wong wrote:
> On Tue, Jan 28, 2025 at 03:34:50PM +1100, Dave Chinner wrote:
> > On Thu, Jan 23, 2025 at 12:16:50PM +1100, Dave Chinner wrote:
> > 4. /tmp is still shared across all runner instances so all the
> > 
> >    concurrent runners dump all their tmp files in /tmp. However, the
> >    runners no longer have unique PIDs (i.e. check runs as PID 3 in
> >    all runner instaces). This means using /tmp/tmp.$$ as the
> >    check/test temp file definition results is instant tmp file name
> >    collisions and random things in check and tests fail.  check and
> >    common/preamble have to be converted to use `mktemp` to provide
> >    unique tempfile name prefixes again.
> > 
> > 5. Don't forget to revert the parent /proc mount back to shared
> >    after check has finished running (or was aborted).
> > 
> > I think with this (current prototype patch below), we can use PID
> > namespaces rather than process session IDs for check-parallel safe
> > process management.
> > 
> > Thoughts?
> 
> Was about to go to bed, but can we also start a new mount namespace,
> create a private (or at least non-global) /tmp to put files into, and
> then each test instance is isolated from clobbering the /tmpfiles of
> other ./check instances *and* the rest of the system?

We probably can. I didn't want to go down that rat hole straight
away, because then I'd have to make a decision about what to mount
there. One thing at a time....

I suspect that I can just use a tmpfs filesystem for it - there's
heaps of memory available on my test machines and we don't use /tmp
to hold large files, so that should work fine for me.  However, I'm
a little concerned about what will happen when testing under memory
pressure situations if /tmp needs memory to operate correctly.

I'll have a look at what is needed for private tmpfs /tmp instances
to work - it should work just fine.

However, if check-parallel has taught me anything, it is that trying
to use "should work" features on a modern system tends to mean "this
is a poorly documented rat-hole that with many dead-ends that will
be explored before a working solution is found"...

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Darrick J. Wong]

On Wed, Jan 29, 2025 at 07:39:22AM +1100, Dave Chinner wrote:
> On Mon, Jan 27, 2025 at 11:23:52PM -0800, Darrick J. Wong wrote:
> > On Tue, Jan 28, 2025 at 03:34:50PM +1100, Dave Chinner wrote:
> > > On Thu, Jan 23, 2025 at 12:16:50PM +1100, Dave Chinner wrote:
> > > 4. /tmp is still shared across all runner instances so all the
> > > 
> > >    concurrent runners dump all their tmp files in /tmp. However, the
> > >    runners no longer have unique PIDs (i.e. check runs as PID 3 in
> > >    all runner instaces). This means using /tmp/tmp.$$ as the
> > >    check/test temp file definition results is instant tmp file name
> > >    collisions and random things in check and tests fail.  check and
> > >    common/preamble have to be converted to use `mktemp` to provide
> > >    unique tempfile name prefixes again.
> > > 
> > > 5. Don't forget to revert the parent /proc mount back to shared
> > >    after check has finished running (or was aborted).
> > > 
> > > I think with this (current prototype patch below), we can use PID
> > > namespaces rather than process session IDs for check-parallel safe
> > > process management.
> > > 
> > > Thoughts?
> > 
> > Was about to go to bed, but can we also start a new mount namespace,
> > create a private (or at least non-global) /tmp to put files into, and
> > then each test instance is isolated from clobbering the /tmpfiles of
> > other ./check instances *and* the rest of the system?
> 
> We probably can. I didn't want to go down that rat hole straight
> away, because then I'd have to make a decision about what to mount
> there. One thing at a time....
> 
> I suspect that I can just use a tmpfs filesystem for it - there's
> heaps of memory available on my test machines and we don't use /tmp
> to hold large files, so that should work fine for me.  However, I'm
> a little concerned about what will happen when testing under memory
> pressure situations if /tmp needs memory to operate correctly.
> 
> I'll have a look at what is needed for private tmpfs /tmp instances
> to work - it should work just fine.
> 
> However, if check-parallel has taught me anything, it is that trying
> to use "should work" features on a modern system tends to mean "this
> is a poorly documented rat-hole that with many dead-ends that will
> be explored before a working solution is found"...

<nod> I'm running an experiment overnight with the following patch to
get rid of the session id grossness.  AFAICT it can also be used by
check-parallel to start its subprocesses in separate pid namespaces,
though I didn't investigate thoroughly.

I'm also not sure it's required for check-helper to unmount the /proc
that it creates; merely exiting seems to clean everything up? <shrug>

I also tried using systemd-nspawn to run fstests inside a "container"
but very quickly discovered that you can't pass block devices to the
container which makes fstests pretty useless for testing real scsi
devices. :/

--D

check: run tests in a private pid/mount namespace

Experiment with running tests in a private pid/mount namespace for
better isolation of the scheduler (check) vs the test (./$seq).  This
also makes it cleaner to adjust the oom score and is a convenient place
to set up the environment before invoking the test.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 check        |   30 +++++-------------------------
 check-helper |   31 +++++++++++++++++++++++++++++++
 common/rc    |    8 +++-----
 3 files changed, 39 insertions(+), 30 deletions(-)
 create mode 100755 check-helper

diff --git a/check b/check
index 00ee5af2a31e34..9c70f6f1e10110 100755
--- a/check
+++ b/check
@@ -698,41 +698,21 @@ _adjust_oom_score -500
 # systemd doesn't automatically remove transient scopes that fail to terminate
 # when systemd tells them to terminate (e.g. programs stuck in D state when
 # systemd sends SIGKILL), so we use reset-failed to tear down the scope.
-#
-# Use setsid to run the test program with a separate session id so that we
-# can pkill only the processes started by this test.
 _run_seq() {
-	local cmd=(bash -c "test -w ${OOM_SCORE_ADJ} && echo 250 > ${OOM_SCORE_ADJ}; exec setsid bash ./$seq")
+	local cmd=("./check-helper" "./$seq")
 
 	if [ -n "${HAVE_SYSTEMD_SCOPES}" ]; then
 		local unit="$(systemd-escape "fs$seq").scope"
 		systemctl reset-failed "${unit}" &> /dev/null
-		systemd-run --quiet --unit "${unit}" --scope "${cmd[@]}" &
-		CHILDPID=$!
-		wait
+		systemd-run --quiet --unit "${unit}" --scope "${cmd[@]}"
 		res=$?
-		unset CHILDPID
 		systemctl stop "${unit}" &> /dev/null
 		return "${res}"
 	else
 		# bash won't run the SIGINT trap handler while there are
 		# foreground children in a separate session, so we must run
 		# the test in the background and wait for it.
-		"${cmd[@]}" &
-		CHILDPID=$!
-		wait
-		unset CHILDPID
-	fi
-}
-
-_kill_seq() {
-	if [ -n "$CHILDPID" ]; then
-		# SIGPIPE will kill all the children (including fsstress)
-		# without bash logging fatal signal termination messages to the
-		# console
-		pkill -PIPE --session "$CHILDPID"
-		wait
-		unset CHILDPID
+		"${cmd[@]}"
 	fi
 }
 
@@ -741,9 +721,9 @@ _prepare_test_list
 fstests_start_time="$(date +"%F %T")"
 
 if $OPTIONS_HAVE_SECTIONS; then
-	trap "_kill_seq; _summary; exit \$status" 0 1 2 3 15
+	trap "_summary; exit \$status" 0 1 2 3 15
 else
-	trap "_kill_seq; _wrapup; exit \$status" 0 1 2 3 15
+	trap "_wrapup; exit \$status" 0 1 2 3 15
 fi
 
 function run_section()
diff --git a/check-helper b/check-helper
new file mode 100755
index 00000000000000..2cc8dbe5cfc791
--- /dev/null
+++ b/check-helper
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Oracle.  All Rights Reserved.
+#
+# Try starting things in a private pid/mount namespace with a private /tmp
+# and /proc so that child process trees cannot interfere with each other.
+
+if [ -n "${IN_NSEXEC}" ]; then
+	for path in /proc /tmp; do
+		mount --make-private "$path"
+	done
+	unset IN_NSEXEC
+	mount -t proc proc /proc
+	mount -t tmpfs tmpfs /tmp
+
+	oom_knob="/proc/self/oom_score_adj"
+	test -w "${oom_knob}" && echo 250 > "${oom_knob}"
+
+	# Run the test, but don't let it be pid 1 because that will confuse
+	# the filter functions in common/dump.
+	"$@"
+	exit
+fi
+
+if [ -z "$1" ] || [ "$1" = "--help" ]; then
+	echo "Usage: $0 command [args...]"
+	exit 1
+fi
+
+IN_NSEXEC=1 exec "$(dirname "$0")/src/nsexec" -m -p $0 "$@"
diff --git a/common/rc b/common/rc
index 1c28a2d190f5a0..cc080ecaa9f801 100644
--- a/common/rc
+++ b/common/rc
@@ -33,13 +33,13 @@ _test_sync()
 # Kill only the test processes started by this test
 _pkill()
 {
-	pkill --session 0 "$@"
+	pkill "$@"
 }
 
 # Find only the test processes started by this test
 _pgrep()
 {
-	pgrep --session 0 "$@"
+	pgrep "$@"
 }
 
 # Common execution handling for fsstress invocation.
@@ -2817,11 +2817,9 @@ _require_user_exists()
 	[ "$?" == "0" ] || _notrun "$user user not defined."
 }
 
-# Run all non-root processes in the same session as the root.  Believe it or
-# not, passing $SHELL in this manner works both for "su" and "su -c cmd".
 _su()
 {
-	su --session-command $SHELL "$@"
+	su "$@"
 }
 
 # check if a user exists and is able to execute commands.

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Dave Chinner]

On Tue, Jan 28, 2025 at 07:13:13PM -0800, Darrick J. Wong wrote:
> On Wed, Jan 29, 2025 at 07:39:22AM +1100, Dave Chinner wrote:
> > On Mon, Jan 27, 2025 at 11:23:52PM -0800, Darrick J. Wong wrote:
> > > On Tue, Jan 28, 2025 at 03:34:50PM +1100, Dave Chinner wrote:
> > > > On Thu, Jan 23, 2025 at 12:16:50PM +1100, Dave Chinner wrote:
> > > > 4. /tmp is still shared across all runner instances so all the
> > > > 
> > > >    concurrent runners dump all their tmp files in /tmp. However, the
> > > >    runners no longer have unique PIDs (i.e. check runs as PID 3 in
> > > >    all runner instaces). This means using /tmp/tmp.$$ as the
> > > >    check/test temp file definition results is instant tmp file name
> > > >    collisions and random things in check and tests fail.  check and
> > > >    common/preamble have to be converted to use `mktemp` to provide
> > > >    unique tempfile name prefixes again.
> > > > 
> > > > 5. Don't forget to revert the parent /proc mount back to shared
> > > >    after check has finished running (or was aborted).
> > > > 
> > > > I think with this (current prototype patch below), we can use PID
> > > > namespaces rather than process session IDs for check-parallel safe
> > > > process management.
> > > > 
> > > > Thoughts?
> > > 
> > > Was about to go to bed, but can we also start a new mount namespace,
> > > create a private (or at least non-global) /tmp to put files into, and
> > > then each test instance is isolated from clobbering the /tmpfiles of
> > > other ./check instances *and* the rest of the system?
> > 
> > We probably can. I didn't want to go down that rat hole straight
> > away, because then I'd have to make a decision about what to mount
> > there. One thing at a time....
> > 
> > I suspect that I can just use a tmpfs filesystem for it - there's
> > heaps of memory available on my test machines and we don't use /tmp
> > to hold large files, so that should work fine for me.  However, I'm
> > a little concerned about what will happen when testing under memory
> > pressure situations if /tmp needs memory to operate correctly.
> > 
> > I'll have a look at what is needed for private tmpfs /tmp instances
> > to work - it should work just fine.
> > 
> > However, if check-parallel has taught me anything, it is that trying
> > to use "should work" features on a modern system tends to mean "this
> > is a poorly documented rat-hole that with many dead-ends that will
> > be explored before a working solution is found"...
> 
> <nod> I'm running an experiment overnight with the following patch to
> get rid of the session id grossness.  AFAICT it can also be used by
> check-parallel to start its subprocesses in separate pid namespaces,
> though I didn't investigate thoroughly.

I don't think check-parallel needs to start each check instance in
it's own PID namespace - it's the tests themselves that need the
isolation from each other.

However, the check instances require a private mount namespace, as
they mount and unmount test/scratch devices themselves and we do not
want other check instances seeing those mounts.

Hence I think the current check-parallel code doing mount namespace
isolation as it already does should work with this patch enabling
per-test process isolation inside check itself.

> I'm also not sure it's required for check-helper to unmount the /proc
> that it creates; merely exiting seems to clean everything up? <shrug>

Yeah, I think tearing down the mount namespace (i.e. exiting the
process that nsexec created) drops the last active reference to the
mounts inside the private namespace and so it gets torn down that
way.

So from my perspective, I think your check-helper namespace patch is
a good improvement and I'll build/fix anything I come across on top
of it. Once your series of fixes goes in, I'll rebase all the stuff
I've got on top it and go from there...

> I also tried using systemd-nspawn to run fstests inside a "container"
> but very quickly discovered that you can't pass block devices to the
> container which makes fstests pretty useless for testing real scsi
> devices. :/

Yet another dead-end in the poorly sign-posted rat-hole, eh?

-Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 08/23] common: fix pkill by running test program in a separate session

[Darrick J. Wong]

On Wed, Jan 29, 2025 at 05:06:52PM +1100, Dave Chinner wrote:
> On Tue, Jan 28, 2025 at 07:13:13PM -0800, Darrick J. Wong wrote:
> > On Wed, Jan 29, 2025 at 07:39:22AM +1100, Dave Chinner wrote:
> > > On Mon, Jan 27, 2025 at 11:23:52PM -0800, Darrick J. Wong wrote:
> > > > On Tue, Jan 28, 2025 at 03:34:50PM +1100, Dave Chinner wrote:
> > > > > On Thu, Jan 23, 2025 at 12:16:50PM +1100, Dave Chinner wrote:
> > > > > 4. /tmp is still shared across all runner instances so all the
> > > > > 
> > > > >    concurrent runners dump all their tmp files in /tmp. However, the
> > > > >    runners no longer have unique PIDs (i.e. check runs as PID 3 in
> > > > >    all runner instaces). This means using /tmp/tmp.$$ as the
> > > > >    check/test temp file definition results is instant tmp file name
> > > > >    collisions and random things in check and tests fail.  check and
> > > > >    common/preamble have to be converted to use `mktemp` to provide
> > > > >    unique tempfile name prefixes again.
> > > > > 
> > > > > 5. Don't forget to revert the parent /proc mount back to shared
> > > > >    after check has finished running (or was aborted).
> > > > > 
> > > > > I think with this (current prototype patch below), we can use PID
> > > > > namespaces rather than process session IDs for check-parallel safe
> > > > > process management.
> > > > > 
> > > > > Thoughts?
> > > > 
> > > > Was about to go to bed, but can we also start a new mount namespace,
> > > > create a private (or at least non-global) /tmp to put files into, and
> > > > then each test instance is isolated from clobbering the /tmpfiles of
> > > > other ./check instances *and* the rest of the system?
> > > 
> > > We probably can. I didn't want to go down that rat hole straight
> > > away, because then I'd have to make a decision about what to mount
> > > there. One thing at a time....
> > > 
> > > I suspect that I can just use a tmpfs filesystem for it - there's
> > > heaps of memory available on my test machines and we don't use /tmp
> > > to hold large files, so that should work fine for me.  However, I'm
> > > a little concerned about what will happen when testing under memory
> > > pressure situations if /tmp needs memory to operate correctly.
> > > 
> > > I'll have a look at what is needed for private tmpfs /tmp instances
> > > to work - it should work just fine.
> > > 
> > > However, if check-parallel has taught me anything, it is that trying
> > > to use "should work" features on a modern system tends to mean "this
> > > is a poorly documented rat-hole that with many dead-ends that will
> > > be explored before a working solution is found"...
> > 
> > <nod> I'm running an experiment overnight with the following patch to
> > get rid of the session id grossness.  AFAICT it can also be used by
> > check-parallel to start its subprocesses in separate pid namespaces,
> > though I didn't investigate thoroughly.
> 
> I don't think check-parallel needs to start each check instance in
> it's own PID namespace - it's the tests themselves that need the
> isolation from each other.
> 
> However, the check instances require a private mount namespace, as
> they mount and unmount test/scratch devices themselves and we do not
> want other check instances seeing those mounts.
> 
> Hence I think the current check-parallel code doing mount namespace
> isolation as it already does should work with this patch enabling
> per-test process isolation inside check itself.
> 
> > I'm also not sure it's required for check-helper to unmount the /proc
> > that it creates; merely exiting seems to clean everything up? <shrug>
> 
> Yeah, I think tearing down the mount namespace (i.e. exiting the
> process that nsexec created) drops the last active reference to the
> mounts inside the private namespace and so it gets torn down that
> way.
> 
> So from my perspective, I think your check-helper namespace patch is
> a good improvement and I'll build/fix anything I come across on top
> of it. Once your series of fixes goes in, I'll rebase all the stuff
> I've got on top it and go from there...

<nod> I might reformulate the pkill code to use nsexec (and not systemd)
if it's available; systemd scopes if those are available (I figured out
how to get systemd to tell me the cgroup name); or worst case fall back
to process sessions if neither are available.

I don't know how ancient of a userspace we realistically have to support
since (afaict) namespaces and systemd both showed up around the 2.6.24
era?  But I also don't know if Devuan at least does pid/mount
namespaces.

--D

> > I also tried using systemd-nspawn to run fstests inside a "container"
> > but very quickly discovered that you can't pass block devices to the
> > container which makes fstests pretty useless for testing real scsi
> > devices. :/
> 
> Yet another dead-end in the poorly sign-posted rat-hole, eh?

Yup.

--D

> -Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

Re: [PATCH 1/1] fstests: test mkfs.xfs protofiles with xattr support

[Zorro Lang]

On Thu, Jan 16, 2025 at 03:35:04PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Make sure we can do protofiles with xattr support.
> 
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> ---

This test always fails on my side, as below (diff output):

   ...
   Attribute "rootdata" has a 5 byte value for SCRATCH_MNT/directory/test
   Attribute "bigdata" has a 37960 byte value for SCRATCH_MNT/directory/test
   Attribute "acldata" has a 5 byte value for SCRATCH_MNT/directory/test
  +Attribute "selinux" has a 28 byte value for SCRATCH_MNT/directory/test
   *** unmount FS
   *** done
   *** unmount

Looks like the $SELINUX_MOUNT_OPTIONS doesn't help the mkfs protofile
with xattrs.

Thanks,
Zorro


>  tests/xfs/1937     |  144 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/1937.out |  102 +++++++++++++++++++++++++++++++++++++
>  2 files changed, 246 insertions(+)
>  create mode 100755 tests/xfs/1937
>  create mode 100644 tests/xfs/1937.out
> 
> 
> diff --git a/tests/xfs/1937 b/tests/xfs/1937
> new file mode 100755
> index 00000000000000..aa4143a75ef643
> --- /dev/null
> +++ b/tests/xfs/1937
> @@ -0,0 +1,144 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2024-2025 Oracle.  All Rights Reserved.
> +# Copyright (c) 2000-2004 Silicon Graphics, Inc.  All Rights Reserved.
> +#
> +# FS QA Test No. 1937
> +#
> +# mkfs protofile with xattrs test
> +#
> +. ./common/preamble
> +_begin_fstest mkfs auto quick
> +
> +seqfull="$seqres.full"
> +rm -f $seqfull
> +
> +. ./common/filter
> +
> +_cleanup()
> +{
> +	echo "*** unmount"
> +	_scratch_unmount 2>/dev/null
> +	rm -f $tmp.*
> +	rm -f $TEST_DIR/$seq.file
> +}
> +
> +_full()
> +{
> +	echo ""            >>$seqfull
> +	echo "*** $* ***"  >>$seqfull
> +	echo ""            >>$seqfull
> +}
> +
> +_filter_stat()
> +{
> +	sed '
> +		/^Access:/d;
> +		/^Modify:/d;
> +		/^Change:/d;
> +		s/Device: *[0-9][0-9]*,[0-9][0-9]*/Device: <DEVICE>/;
> +		s/Inode: *[0-9][0-9]*/Inode: <INODE>/;
> +		s/Size: *[0-9][0-9]* *Filetype: Dir/Size: <DSIZE> Filetype: Dir/;
> +	' | tr -s ' '
> +}
> +
> +_require_command $ATTR_PROG "attr"
> +_require_scratch
> +
> +# mkfs cannot create a filesystem with protofiles if realtime is enabled, so
> +# don't run this test if the rtinherit is anywhere in the mkfs options.
> +echo "$MKFS_OPTIONS" | grep -q "rtinherit" && \
> +	_notrun "Cannot mkfs with a protofile and -d rtinherit."
> +
> +protofile=$tmp.proto
> +tempfile=$TEST_DIR/$seq.file
> +
> +$XFS_IO_PROG -f -c 'pwrite 64k 28k' -c 'pwrite 1280k 37960' $tempfile >> $seqres.full
> +$here/src/devzero -b 2048 -n 2 -c -v 44 $tempfile.2 
> +
> +$ATTR_PROG -R -s rootdata -V 0test $tempfile &>> $seqres.full
> +$ATTR_PROG -S -s acldata -V 1test $tempfile &>> $seqres.full
> +$ATTR_PROG -s userdata -V 2test $tempfile &>> $seqres.full
> +perl -e 'print "x" x 37960;' | $ATTR_PROG -s bigdata $tempfile &>> $seqres.full
> +
> +cat >$protofile <<EOF
> +DUMMY1
> +0 0
> +: root directory
> +d--777 3 1
> +: a directory
> +directory d--755 3 1 
> +test ---755 3 1 $tempfile
> +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_0 ---755 3 1 $tempfile
> +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1 ---755 3 1 $tempfile
> +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2 ---755 3 1 $tempfile
> +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3 ---755 3 1 $tempfile
> +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_4 ---755 3 1 $tempfile
> +$
> +: back in the root
> +setuid -u-666 0 0 $tempfile
> +setgid --g666 0 0 $tempfile
> +setugid -ug666 0 0 $tempfile
> +directory_setgid d-g755 3 2
> +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_5 ---755 3 1 $tempfile
> +$
> +: back in the root
> +block_device b--012 3 1 161 162 
> +char_device c--345 3 1 177 178
> +pipe p--670 0 0
> +symlink l--123 0 0 bigfile
> +: a file we actually read
> +bigfile ---666 3 0 $tempfile.2
> +: done
> +$
> +EOF
> +
> +if [ $? -ne 0 ]
> +then
> +	_fail "failed to create test protofile"
> +fi
> +
> +_verify_fs()
> +{
> +	echo "*** create FS version $1"
> +	VERSION="-n version=$1"
> +
> +	_scratch_unmount >/dev/null 2>&1
> +
> +	_full "mkfs"
> +	_scratch_mkfs_xfs $VERSION -p $protofile >>$seqfull 2>&1
> +
> +	echo "*** check FS"
> +	_check_scratch_fs
> +
> +	echo "*** mount FS"
> +	_full " mount"
> +	_try_scratch_mount >>$seqfull 2>&1 \
> +		|| _fail "mount failed"
> +
> +	$ATTR_PROG -l $SCRATCH_MNT/directory/test | \
> +		grep -q 'Attribute.*has a ' || \
> +		_notrun "mkfs.xfs protofile does not support xattrs"
> +
> +	echo "*** verify FS"
> +	(cd $SCRATCH_MNT ; find . | LC_COLLATE=POSIX sort \
> +		| grep -v ".use_space" \
> +		| xargs $here/src/lstat64 | _filter_stat)
> +	diff -q $SCRATCH_MNT/bigfile $tempfile.2 \
> +		|| _fail "bigfile corrupted"
> +	diff -q $SCRATCH_MNT/symlink $tempfile.2 \
> +		|| _fail "symlink broken"
> +
> +	$ATTR_PROG -l $SCRATCH_MNT/directory/test | _filter_scratch
> +
> +	echo "*** unmount FS"
> +	_full "umount"
> +	_scratch_unmount >>$seqfull 2>&1 \
> +		|| _fail "umount failed"
> +}
> +
> +_verify_fs 2
> +
> +echo "*** done"
> +status=0
> +exit
> diff --git a/tests/xfs/1937.out b/tests/xfs/1937.out
> new file mode 100644
> index 00000000000000..050c8318b1abca
> --- /dev/null
> +++ b/tests/xfs/1937.out
> @@ -0,0 +1,102 @@
> +QA output created by 1937
> +Wrote 2048.00Kb (value 0x2c)
> +*** create FS version 2
> +*** check FS
> +*** mount FS
> +*** verify FS
> + File: "."
> + Size: <DSIZE> Filetype: Directory
> + Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 4 
> +
> + File: "./bigfile"
> + Size: 2097152 Filetype: Regular File
> + Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./block_device"
> + Size: 0 Filetype: Block Device
> + Mode: (0012/b-----x-w-) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 1 Device type: 161,162
> +
> + File: "./char_device"
> + Size: 0 Filetype: Character Device
> + Mode: (0345/c-wxr--r-x) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 1 Device type: 177,178
> +
> + File: "./directory"
> + Size: <DSIZE> Filetype: Directory
> + Mode: (0755/drwxr-xr-x) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 2 
> +
> + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_0"
> + Size: 1348680 Filetype: Regular File
> + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1"
> + Size: 1348680 Filetype: Regular File
> + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2"
> + Size: 1348680 Filetype: Regular File
> + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3"
> + Size: 1348680 Filetype: Regular File
> + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_4"
> + Size: 1348680 Filetype: Regular File
> + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./directory/test"
> + Size: 1348680 Filetype: Regular File
> + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./directory_setgid"
> + Size: <DSIZE> Filetype: Directory
> + Mode: (2755/drwxr-sr-x) Uid: (3) Gid: (2)
> +Device: <DEVICE> Inode: <INODE> Links: 2 
> +
> + File: "./directory_setgid/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_5"
> + Size: 1348680 Filetype: Regular File
> + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./pipe"
> + Size: 0 Filetype: Fifo File
> + Mode: (0670/frw-rwx---) Uid: (0) Gid: (0)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./setgid"
> + Size: 1348680 Filetype: Regular File
> + Mode: (2666/-rw-rwsrw-) Uid: (0) Gid: (0)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./setugid"
> + Size: 1348680 Filetype: Regular File
> + Mode: (6666/-rwsrwsrw-) Uid: (0) Gid: (0)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./setuid"
> + Size: 1348680 Filetype: Regular File
> + Mode: (4666/-rwsrw-rw-) Uid: (0) Gid: (0)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +
> + File: "./symlink"
> + Size: 7 Filetype: Symbolic Link
> + Mode: (0123/l--x-w--wx) Uid: (0) Gid: (0)
> +Device: <DEVICE> Inode: <INODE> Links: 1 
> +Attribute "userdata" has a 5 byte value for SCRATCH_MNT/directory/test
> +Attribute "rootdata" has a 5 byte value for SCRATCH_MNT/directory/test
> +Attribute "bigdata" has a 37960 byte value for SCRATCH_MNT/directory/test
> +Attribute "acldata" has a 5 byte value for SCRATCH_MNT/directory/test
> +*** unmount FS
> +*** done
> +*** unmount
> 

Re: [PATCH 1/1] fstests: test mkfs.xfs protofiles with xattr support

[Darrick J. Wong]

On Sun, Mar 02, 2025 at 09:15:44PM +0800, Zorro Lang wrote:
> On Thu, Jan 16, 2025 at 03:35:04PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > Make sure we can do protofiles with xattr support.
> > 
> > Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> > ---
> 
> This test always fails on my side, as below (diff output):
> 
>    ...
>    Attribute "rootdata" has a 5 byte value for SCRATCH_MNT/directory/test
>    Attribute "bigdata" has a 37960 byte value for SCRATCH_MNT/directory/test
>    Attribute "acldata" has a 5 byte value for SCRATCH_MNT/directory/test
>   +Attribute "selinux" has a 28 byte value for SCRATCH_MNT/directory/test
>    *** unmount FS
>    *** done
>    *** unmount
> 
> Looks like the $SELINUX_MOUNT_OPTIONS doesn't help the mkfs protofile
> with xattrs.

Oops.  Ok then, I'll filter them out below...

> Thanks,
> Zorro
> 
> 
> >  tests/xfs/1937     |  144 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tests/xfs/1937.out |  102 +++++++++++++++++++++++++++++++++++++
> >  2 files changed, 246 insertions(+)
> >  create mode 100755 tests/xfs/1937
> >  create mode 100644 tests/xfs/1937.out
> > 
> > 
> > diff --git a/tests/xfs/1937 b/tests/xfs/1937
> > new file mode 100755
> > index 00000000000000..aa4143a75ef643
> > --- /dev/null
> > +++ b/tests/xfs/1937
> > @@ -0,0 +1,144 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2024-2025 Oracle.  All Rights Reserved.
> > +# Copyright (c) 2000-2004 Silicon Graphics, Inc.  All Rights Reserved.
> > +#
> > +# FS QA Test No. 1937
> > +#
> > +# mkfs protofile with xattrs test
> > +#
> > +. ./common/preamble
> > +_begin_fstest mkfs auto quick
> > +
> > +seqfull="$seqres.full"
> > +rm -f $seqfull
> > +
> > +. ./common/filter
> > +
> > +_cleanup()
> > +{
> > +	echo "*** unmount"
> > +	_scratch_unmount 2>/dev/null
> > +	rm -f $tmp.*
> > +	rm -f $TEST_DIR/$seq.file
> > +}
> > +
> > +_full()
> > +{
> > +	echo ""            >>$seqfull
> > +	echo "*** $* ***"  >>$seqfull
> > +	echo ""            >>$seqfull
> > +}
> > +
> > +_filter_stat()
> > +{
> > +	sed '
> > +		/^Access:/d;
> > +		/^Modify:/d;
> > +		/^Change:/d;
> > +		s/Device: *[0-9][0-9]*,[0-9][0-9]*/Device: <DEVICE>/;
> > +		s/Inode: *[0-9][0-9]*/Inode: <INODE>/;
> > +		s/Size: *[0-9][0-9]* *Filetype: Dir/Size: <DSIZE> Filetype: Dir/;
> > +	' | tr -s ' '
> > +}
> > +
> > +_require_command $ATTR_PROG "attr"
> > +_require_scratch
> > +
> > +# mkfs cannot create a filesystem with protofiles if realtime is enabled, so
> > +# don't run this test if the rtinherit is anywhere in the mkfs options.
> > +echo "$MKFS_OPTIONS" | grep -q "rtinherit" && \
> > +	_notrun "Cannot mkfs with a protofile and -d rtinherit."
> > +
> > +protofile=$tmp.proto
> > +tempfile=$TEST_DIR/$seq.file
> > +
> > +$XFS_IO_PROG -f -c 'pwrite 64k 28k' -c 'pwrite 1280k 37960' $tempfile >> $seqres.full
> > +$here/src/devzero -b 2048 -n 2 -c -v 44 $tempfile.2 
> > +
> > +$ATTR_PROG -R -s rootdata -V 0test $tempfile &>> $seqres.full
> > +$ATTR_PROG -S -s acldata -V 1test $tempfile &>> $seqres.full
> > +$ATTR_PROG -s userdata -V 2test $tempfile &>> $seqres.full
> > +perl -e 'print "x" x 37960;' | $ATTR_PROG -s bigdata $tempfile &>> $seqres.full
> > +
> > +cat >$protofile <<EOF
> > +DUMMY1
> > +0 0
> > +: root directory
> > +d--777 3 1
> > +: a directory
> > +directory d--755 3 1 
> > +test ---755 3 1 $tempfile
> > +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_0 ---755 3 1 $tempfile
> > +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1 ---755 3 1 $tempfile
> > +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2 ---755 3 1 $tempfile
> > +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3 ---755 3 1 $tempfile
> > +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_4 ---755 3 1 $tempfile
> > +$
> > +: back in the root
> > +setuid -u-666 0 0 $tempfile
> > +setgid --g666 0 0 $tempfile
> > +setugid -ug666 0 0 $tempfile
> > +directory_setgid d-g755 3 2
> > +file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_5 ---755 3 1 $tempfile
> > +$
> > +: back in the root
> > +block_device b--012 3 1 161 162 
> > +char_device c--345 3 1 177 178
> > +pipe p--670 0 0
> > +symlink l--123 0 0 bigfile
> > +: a file we actually read
> > +bigfile ---666 3 0 $tempfile.2
> > +: done
> > +$
> > +EOF
> > +
> > +if [ $? -ne 0 ]
> > +then
> > +	_fail "failed to create test protofile"
> > +fi
> > +
> > +_verify_fs()
> > +{
> > +	echo "*** create FS version $1"
> > +	VERSION="-n version=$1"
> > +
> > +	_scratch_unmount >/dev/null 2>&1
> > +
> > +	_full "mkfs"
> > +	_scratch_mkfs_xfs $VERSION -p $protofile >>$seqfull 2>&1
> > +
> > +	echo "*** check FS"
> > +	_check_scratch_fs
> > +
> > +	echo "*** mount FS"
> > +	_full " mount"
> > +	_try_scratch_mount >>$seqfull 2>&1 \
> > +		|| _fail "mount failed"
> > +
> > +	$ATTR_PROG -l $SCRATCH_MNT/directory/test | \
> > +		grep -q 'Attribute.*has a ' || \
> > +		_notrun "mkfs.xfs protofile does not support xattrs"
> > +
> > +	echo "*** verify FS"
> > +	(cd $SCRATCH_MNT ; find . | LC_COLLATE=POSIX sort \
> > +		| grep -v ".use_space" \
> > +		| xargs $here/src/lstat64 | _filter_stat)
> > +	diff -q $SCRATCH_MNT/bigfile $tempfile.2 \
> > +		|| _fail "bigfile corrupted"
> > +	diff -q $SCRATCH_MNT/symlink $tempfile.2 \
> > +		|| _fail "symlink broken"
> > +
> > +	$ATTR_PROG -l $SCRATCH_MNT/directory/test | _filter_scratch

...so they don't spill into the golden output.  As this is already in
patches-in-queue, do you want me to send a fixpatch on top of that?

--D

> > +
> > +	echo "*** unmount FS"
> > +	_full "umount"
> > +	_scratch_unmount >>$seqfull 2>&1 \
> > +		|| _fail "umount failed"
> > +}
> > +
> > +_verify_fs 2
> > +
> > +echo "*** done"
> > +status=0
> > +exit
> > diff --git a/tests/xfs/1937.out b/tests/xfs/1937.out
> > new file mode 100644
> > index 00000000000000..050c8318b1abca
> > --- /dev/null
> > +++ b/tests/xfs/1937.out
> > @@ -0,0 +1,102 @@
> > +QA output created by 1937
> > +Wrote 2048.00Kb (value 0x2c)
> > +*** create FS version 2
> > +*** check FS
> > +*** mount FS
> > +*** verify FS
> > + File: "."
> > + Size: <DSIZE> Filetype: Directory
> > + Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 4 
> > +
> > + File: "./bigfile"
> > + Size: 2097152 Filetype: Regular File
> > + Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./block_device"
> > + Size: 0 Filetype: Block Device
> > + Mode: (0012/b-----x-w-) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 Device type: 161,162
> > +
> > + File: "./char_device"
> > + Size: 0 Filetype: Character Device
> > + Mode: (0345/c-wxr--r-x) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 Device type: 177,178
> > +
> > + File: "./directory"
> > + Size: <DSIZE> Filetype: Directory
> > + Mode: (0755/drwxr-xr-x) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 2 
> > +
> > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_0"
> > + Size: 1348680 Filetype: Regular File
> > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1"
> > + Size: 1348680 Filetype: Regular File
> > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2"
> > + Size: 1348680 Filetype: Regular File
> > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3"
> > + Size: 1348680 Filetype: Regular File
> > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_4"
> > + Size: 1348680 Filetype: Regular File
> > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./directory/test"
> > + Size: 1348680 Filetype: Regular File
> > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./directory_setgid"
> > + Size: <DSIZE> Filetype: Directory
> > + Mode: (2755/drwxr-sr-x) Uid: (3) Gid: (2)
> > +Device: <DEVICE> Inode: <INODE> Links: 2 
> > +
> > + File: "./directory_setgid/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_5"
> > + Size: 1348680 Filetype: Regular File
> > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./pipe"
> > + Size: 0 Filetype: Fifo File
> > + Mode: (0670/frw-rwx---) Uid: (0) Gid: (0)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./setgid"
> > + Size: 1348680 Filetype: Regular File
> > + Mode: (2666/-rw-rwsrw-) Uid: (0) Gid: (0)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./setugid"
> > + Size: 1348680 Filetype: Regular File
> > + Mode: (6666/-rwsrwsrw-) Uid: (0) Gid: (0)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./setuid"
> > + Size: 1348680 Filetype: Regular File
> > + Mode: (4666/-rwsrw-rw-) Uid: (0) Gid: (0)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +
> > + File: "./symlink"
> > + Size: 7 Filetype: Symbolic Link
> > + Mode: (0123/l--x-w--wx) Uid: (0) Gid: (0)
> > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > +Attribute "userdata" has a 5 byte value for SCRATCH_MNT/directory/test
> > +Attribute "rootdata" has a 5 byte value for SCRATCH_MNT/directory/test
> > +Attribute "bigdata" has a 37960 byte value for SCRATCH_MNT/directory/test
> > +Attribute "acldata" has a 5 byte value for SCRATCH_MNT/directory/test
> > +*** unmount FS
> > +*** done
> > +*** unmount
> > 
> 

Re: [PATCH 1/1] fstests: test mkfs.xfs protofiles with xattr support

[Zorro Lang]

On Tue, Mar 04, 2025 at 09:42:42AM -0800, Darrick J. Wong wrote:
> On Sun, Mar 02, 2025 at 09:15:44PM +0800, Zorro Lang wrote:
> > On Thu, Jan 16, 2025 at 03:35:04PM -0800, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > > 
> > > Make sure we can do protofiles with xattr support.
> > > 
> > > Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> > > ---
> > 
> > This test always fails on my side, as below (diff output):
> > 
> >    ...
> >    Attribute "rootdata" has a 5 byte value for SCRATCH_MNT/directory/test
> >    Attribute "bigdata" has a 37960 byte value for SCRATCH_MNT/directory/test
> >    Attribute "acldata" has a 5 byte value for SCRATCH_MNT/directory/test
> >   +Attribute "selinux" has a 28 byte value for SCRATCH_MNT/directory/test
> >    *** unmount FS
> >    *** done
> >    *** unmount
> > 
> > Looks like the $SELINUX_MOUNT_OPTIONS doesn't help the mkfs protofile
> > with xattrs.
> 
> Oops.  Ok then, I'll filter them out below...
> 
> > Thanks,
> > Zorro
> > 
> > 
> > >  tests/xfs/1937     |  144 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > >  tests/xfs/1937.out |  102 +++++++++++++++++++++++++++++++++++++
> > >  2 files changed, 246 insertions(+)
> > >  create mode 100755 tests/xfs/1937
> > >  create mode 100644 tests/xfs/1937.out
> > >
 
[snip]

> > > +	echo "*** verify FS"
> > > +	(cd $SCRATCH_MNT ; find . | LC_COLLATE=POSIX sort \
> > > +		| grep -v ".use_space" \
> > > +		| xargs $here/src/lstat64 | _filter_stat)
> > > +	diff -q $SCRATCH_MNT/bigfile $tempfile.2 \
> > > +		|| _fail "bigfile corrupted"
> > > +	diff -q $SCRATCH_MNT/symlink $tempfile.2 \
> > > +		|| _fail "symlink broken"
> > > +
> > > +	$ATTR_PROG -l $SCRATCH_MNT/directory/test | _filter_scratch
> 
> ...so they don't spill into the golden output.  As this is already in
> patches-in-queue, do you want me to send a fixpatch on top of that?

If you just need a simple filter, I think you can tell me what do you
want to change, I can amend the commit simply. Or if you need to change
more, only re-send this patch is good to me :)

Thanks,
Zorro

> 
> --D
> 
> > > +
> > > +	echo "*** unmount FS"
> > > +	_full "umount"
> > > +	_scratch_unmount >>$seqfull 2>&1 \
> > > +		|| _fail "umount failed"
> > > +}
> > > +
> > > +_verify_fs 2
> > > +
> > > +echo "*** done"
> > > +status=0
> > > +exit
> > > diff --git a/tests/xfs/1937.out b/tests/xfs/1937.out
> > > new file mode 100644
> > > index 00000000000000..050c8318b1abca
> > > --- /dev/null
> > > +++ b/tests/xfs/1937.out
> > > @@ -0,0 +1,102 @@
> > > +QA output created by 1937
> > > +Wrote 2048.00Kb (value 0x2c)
> > > +*** create FS version 2
> > > +*** check FS
> > > +*** mount FS
> > > +*** verify FS
> > > + File: "."
> > > + Size: <DSIZE> Filetype: Directory
> > > + Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 4 
> > > +
> > > + File: "./bigfile"
> > > + Size: 2097152 Filetype: Regular File
> > > + Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./block_device"
> > > + Size: 0 Filetype: Block Device
> > > + Mode: (0012/b-----x-w-) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 Device type: 161,162
> > > +
> > > + File: "./char_device"
> > > + Size: 0 Filetype: Character Device
> > > + Mode: (0345/c-wxr--r-x) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 Device type: 177,178
> > > +
> > > + File: "./directory"
> > > + Size: <DSIZE> Filetype: Directory
> > > + Mode: (0755/drwxr-xr-x) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 2 
> > > +
> > > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_0"
> > > + Size: 1348680 Filetype: Regular File
> > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1"
> > > + Size: 1348680 Filetype: Regular File
> > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2"
> > > + Size: 1348680 Filetype: Regular File
> > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3"
> > > + Size: 1348680 Filetype: Regular File
> > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_4"
> > > + Size: 1348680 Filetype: Regular File
> > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./directory/test"
> > > + Size: 1348680 Filetype: Regular File
> > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./directory_setgid"
> > > + Size: <DSIZE> Filetype: Directory
> > > + Mode: (2755/drwxr-sr-x) Uid: (3) Gid: (2)
> > > +Device: <DEVICE> Inode: <INODE> Links: 2 
> > > +
> > > + File: "./directory_setgid/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_5"
> > > + Size: 1348680 Filetype: Regular File
> > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./pipe"
> > > + Size: 0 Filetype: Fifo File
> > > + Mode: (0670/frw-rwx---) Uid: (0) Gid: (0)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./setgid"
> > > + Size: 1348680 Filetype: Regular File
> > > + Mode: (2666/-rw-rwsrw-) Uid: (0) Gid: (0)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./setugid"
> > > + Size: 1348680 Filetype: Regular File
> > > + Mode: (6666/-rwsrwsrw-) Uid: (0) Gid: (0)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./setuid"
> > > + Size: 1348680 Filetype: Regular File
> > > + Mode: (4666/-rwsrw-rw-) Uid: (0) Gid: (0)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +
> > > + File: "./symlink"
> > > + Size: 7 Filetype: Symbolic Link
> > > + Mode: (0123/l--x-w--wx) Uid: (0) Gid: (0)
> > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > +Attribute "userdata" has a 5 byte value for SCRATCH_MNT/directory/test
> > > +Attribute "rootdata" has a 5 byte value for SCRATCH_MNT/directory/test
> > > +Attribute "bigdata" has a 37960 byte value for SCRATCH_MNT/directory/test
> > > +Attribute "acldata" has a 5 byte value for SCRATCH_MNT/directory/test
> > > +*** unmount FS
> > > +*** done
> > > +*** unmount
> > > 
> > 
> 

Re: [PATCH 1/1] fstests: test mkfs.xfs protofiles with xattr support

[Darrick J. Wong]

On Wed, Mar 05, 2025 at 02:00:40AM +0800, Zorro Lang wrote:
> On Tue, Mar 04, 2025 at 09:42:42AM -0800, Darrick J. Wong wrote:
> > On Sun, Mar 02, 2025 at 09:15:44PM +0800, Zorro Lang wrote:
> > > On Thu, Jan 16, 2025 at 03:35:04PM -0800, Darrick J. Wong wrote:
> > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > 
> > > > Make sure we can do protofiles with xattr support.
> > > > 
> > > > Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> > > > ---
> > > 
> > > This test always fails on my side, as below (diff output):
> > > 
> > >    ...
> > >    Attribute "rootdata" has a 5 byte value for SCRATCH_MNT/directory/test
> > >    Attribute "bigdata" has a 37960 byte value for SCRATCH_MNT/directory/test
> > >    Attribute "acldata" has a 5 byte value for SCRATCH_MNT/directory/test
> > >   +Attribute "selinux" has a 28 byte value for SCRATCH_MNT/directory/test
> > >    *** unmount FS
> > >    *** done
> > >    *** unmount
> > > 
> > > Looks like the $SELINUX_MOUNT_OPTIONS doesn't help the mkfs protofile
> > > with xattrs.
> > 
> > Oops.  Ok then, I'll filter them out below...
> > 
> > > Thanks,
> > > Zorro
> > > 
> > > 
> > > >  tests/xfs/1937     |  144 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  tests/xfs/1937.out |  102 +++++++++++++++++++++++++++++++++++++
> > > >  2 files changed, 246 insertions(+)
> > > >  create mode 100755 tests/xfs/1937
> > > >  create mode 100644 tests/xfs/1937.out
> > > >
>  
> [snip]
> 
> > > > +	echo "*** verify FS"
> > > > +	(cd $SCRATCH_MNT ; find . | LC_COLLATE=POSIX sort \
> > > > +		| grep -v ".use_space" \
> > > > +		| xargs $here/src/lstat64 | _filter_stat)
> > > > +	diff -q $SCRATCH_MNT/bigfile $tempfile.2 \
> > > > +		|| _fail "bigfile corrupted"
> > > > +	diff -q $SCRATCH_MNT/symlink $tempfile.2 \
> > > > +		|| _fail "symlink broken"
> > > > +
> > > > +	$ATTR_PROG -l $SCRATCH_MNT/directory/test | _filter_scratch
> > 
> > ...so they don't spill into the golden output.  As this is already in
> > patches-in-queue, do you want me to send a fixpatch on top of that?
> 
> If you just need a simple filter, I think you can tell me what do you
> want to change, I can amend the commit simply. Or if you need to change
> more, only re-send this patch is good to me :)

I /think/ changing the above line to this will fix it:

	$ATTR_PROG -l $SCRATCH_MNT/directory/test | \
		sed -e '/Attribute..selinux..has/d' | \
		_filter_scratch

--D

> Thanks,
> Zorro
> 
> > 
> > --D
> > 
> > > > +
> > > > +	echo "*** unmount FS"
> > > > +	_full "umount"
> > > > +	_scratch_unmount >>$seqfull 2>&1 \
> > > > +		|| _fail "umount failed"
> > > > +}
> > > > +
> > > > +_verify_fs 2
> > > > +
> > > > +echo "*** done"
> > > > +status=0
> > > > +exit
> > > > diff --git a/tests/xfs/1937.out b/tests/xfs/1937.out
> > > > new file mode 100644
> > > > index 00000000000000..050c8318b1abca
> > > > --- /dev/null
> > > > +++ b/tests/xfs/1937.out
> > > > @@ -0,0 +1,102 @@
> > > > +QA output created by 1937
> > > > +Wrote 2048.00Kb (value 0x2c)
> > > > +*** create FS version 2
> > > > +*** check FS
> > > > +*** mount FS
> > > > +*** verify FS
> > > > + File: "."
> > > > + Size: <DSIZE> Filetype: Directory
> > > > + Mode: (0777/drwxrwxrwx) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 4 
> > > > +
> > > > + File: "./bigfile"
> > > > + Size: 2097152 Filetype: Regular File
> > > > + Mode: (0666/-rw-rw-rw-) Uid: (3) Gid: (0)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./block_device"
> > > > + Size: 0 Filetype: Block Device
> > > > + Mode: (0012/b-----x-w-) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 Device type: 161,162
> > > > +
> > > > + File: "./char_device"
> > > > + Size: 0 Filetype: Character Device
> > > > + Mode: (0345/c-wxr--r-x) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 Device type: 177,178
> > > > +
> > > > + File: "./directory"
> > > > + Size: <DSIZE> Filetype: Directory
> > > > + Mode: (0755/drwxr-xr-x) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 2 
> > > > +
> > > > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_0"
> > > > + Size: 1348680 Filetype: Regular File
> > > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1"
> > > > + Size: 1348680 Filetype: Regular File
> > > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_2"
> > > > + Size: 1348680 Filetype: Regular File
> > > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_3"
> > > > + Size: 1348680 Filetype: Regular File
> > > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./directory/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_4"
> > > > + Size: 1348680 Filetype: Regular File
> > > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./directory/test"
> > > > + Size: 1348680 Filetype: Regular File
> > > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./directory_setgid"
> > > > + Size: <DSIZE> Filetype: Directory
> > > > + Mode: (2755/drwxr-sr-x) Uid: (3) Gid: (2)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 2 
> > > > +
> > > > + File: "./directory_setgid/file_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_5"
> > > > + Size: 1348680 Filetype: Regular File
> > > > + Mode: (0755/-rwxr-xr-x) Uid: (3) Gid: (1)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./pipe"
> > > > + Size: 0 Filetype: Fifo File
> > > > + Mode: (0670/frw-rwx---) Uid: (0) Gid: (0)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./setgid"
> > > > + Size: 1348680 Filetype: Regular File
> > > > + Mode: (2666/-rw-rwsrw-) Uid: (0) Gid: (0)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./setugid"
> > > > + Size: 1348680 Filetype: Regular File
> > > > + Mode: (6666/-rwsrwsrw-) Uid: (0) Gid: (0)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./setuid"
> > > > + Size: 1348680 Filetype: Regular File
> > > > + Mode: (4666/-rwsrw-rw-) Uid: (0) Gid: (0)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +
> > > > + File: "./symlink"
> > > > + Size: 7 Filetype: Symbolic Link
> > > > + Mode: (0123/l--x-w--wx) Uid: (0) Gid: (0)
> > > > +Device: <DEVICE> Inode: <INODE> Links: 1 
> > > > +Attribute "userdata" has a 5 byte value for SCRATCH_MNT/directory/test
> > > > +Attribute "rootdata" has a 5 byte value for SCRATCH_MNT/directory/test
> > > > +Attribute "bigdata" has a 37960 byte value for SCRATCH_MNT/directory/test
> > > > +Attribute "acldata" has a 5 byte value for SCRATCH_MNT/directory/test
> > > > +*** unmount FS
> > > > +*** done
> > > > +*** unmount
> > > > 
> > > 
> > 
> 
>