From: "Darrick J. Wong" <djwong@kernel.org>
To: Zorro Lang <zlang@redhat.com>
Cc: Dave Chinner <david@fromorbit.com>, fstests@vger.kernel.org
Subject: Re: [RFC PATCH 00/40] fstests: concurrent test execution
Date: Fri, 6 Dec 2024 16:09:17 -0800 [thread overview]
Message-ID: <20241207000917.GA7824@frogsfrogsfrogs> (raw)
In-Reply-To: <20241129042216.u2f2sl5pc553gxqr@dell-per750-06-vm-08.rhts.eng.pek2.redhat.com>
On Fri, Nov 29, 2024 at 12:22:16PM +0800, Zorro Lang wrote:
> On Wed, Nov 27, 2024 at 03:51:30PM +1100, Dave Chinner wrote:
> > Hi folks,
> >
> > This patchset introduces the ability to run fstests concurrently
> > instead of serially as the current check script does. A git branch
> > containing this patchset can be pulled from here:
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfstests-dev.git check-parallel
>
> Hi Dave,
>
> I've merged your "check-parallel" branch, and rebase on fstests'
> patches-in-queue branch (which is nearly the next release). I just
> pushed a new branch "for-dave-check-parallel" which fixed all
> conflicts. It'll be "next next" release, feel free to update base
> on that. I'll test that branch too :)
I ran this through my test infrastructure at zorro's request. I saw a
bunch of loop dev errors trickle out:
--- xfs/129.out
+++ xfs/129.out.bad
@@ -2,3 +2,6 @@
Create the original file blocks
Reflink every other block
Create metadump file, restore it and check restored fs
+losetup: /dev/loop0: detach failed: No such device or address
+Cannot destroy loop device /dev/loop0
+(see /var/tmp/fstests/xfs/129.full for details)
and I noticed the runtimes for running serially went way up. Not sure
if that was because my dev tree has a bunch of metadir fixes in it or
not; will run that again over the weekend with upstream tot to see if it
that brings the total runtime back down.
--D
> Thanks,
> Zorro
>
> >
> > The motivation for this is the ever growing runtime of fstests as
> > more tests are added, along with the extremely low resource usage of
> > individual tests. This means that a typical machine running fstests
> > is under-utilised and a single auto group test set execution takes
> > hours.
> >
> > Yes, I know that I could scale out testing by running lots of little
> > VMs at once, and I already do that. The problem is that I don't get
> > back a complete auto test run result for hours. Want to check that a
> > one-line fix has not caused any regressions? That's, at minimum, an
> > overnight wait for the test farm to crunch through a dozen configs.
> >
> > On my 64p/128GB RAM VM, 'check -g auto -s xfs -x dump' typically
> > takes around 230-240 minutes to run. With this patchset applied, it
> > runs the same set of tests in 8-10 minutes. I can run ~25 complete
> > auto group test sets with check-parallel in the same time it takes
> > check to run one.
> >
> > IOWs, I can have the most common config complete a full regression
> > test run as fast as I can turn around a new kernel with a new change
> > to test. I have CPU, memory and IO to burn in my test machines, but
> > what I lack is instant feedback for the change I just made.
> > check-parallel is fast enough that it gives me pretty much instant
> > feedback....
> >
> > Most of this patchset is preparing infrastructure for concurrent
> > test execution and fixing bugs in tests that I've found whilst
> > getting concurrent execution working reliably. The infrastructure
> > changes center around:
> >
> > - getting rid of killall - there's nothing quite like one test
> > killing the processes of 15 other tests at the same time...
> > - cleaning up background process instantiation and reaping, which is
> > a lot more important when an interrupt needs to kill hundreds of
> > processes instead of just a couple.
> > - isolating error reporting (e.g. dmesg filtering) so that one test
> > failure doesn't trigger lots of other false failure detections
> > - sanitising the use of loopback devices
> > - avoiding the use of fixed device names - multiple tests need to
> > use dm-flakey, dm-error, etc devices at the same time, so each
> > test needs a unique device name
> > - marking tests that are unreliable when outside perturbations
> > occur. These include tests that expect to find delalloc extents,
> > write specific offset patterns in memory then sync them to create
> > specific layouts, etc. e.g. If another test runs sync(1), then
> > those write patterns no longer produce the expected output.
> > - taming tests that weren't designed for high CPU count machines
> > - replacing `udevadm settle` calls because udev is -never- idle when
> > there are tens to hundreds of block devices and filesystems being
> > rapidly set up and torn down.
> > - converting sync(1) and sync(2) to syncfs(2) to avoid ihaving
> > hundreds of concurrent superblock list traversals lock-stepping
> > with multiple mount/unmounts every second.
> >
> > There are lots of little other things, but those are the main test
> > and test infrastructure changes. Some of these are big - the
> > fsstress execution rework touches 105 files, but it now means that
> > every single fsstress execution in fstests is controlled by 4 helper
> > functions:
> >
> > _run_fsstress() - run fsstress synchronously
> > _run_fsstress_bg - run fsstress in background
> > _wait_for_fsstress - wait for background fsstress
> > _kill_fsstress - kill+wait for background fsstress
> >
> > The test infrastructure also automatically handles cleanup of
> > fsstress processes when the test is interrupted, so tests using
> > fsstress don't need a custom _cleanup() function just to call
> > _kill_fsstress(). This is something that should have been done a
> > long time ago, but now it is critical for being able to manage
> > multiple independent concurrent fsstress invocations sanely.
> >
> > There are some tests that just can't be run reliably in a concurrent
> > environment - if there is outside interference in, say, page cache
> > flushing then the tests fail. These tests have been added to the
> > "unreliable_in_parallel" test group with a comment explaining why
> > they are unreliable. The check-parallel script automatically
> > excludes this test group.
> >
> > The only remaining set of tests that are somewhat flakey are the
> > tests that exercise quotas. Quota tests randomly fail for various
> > reasons. Sometimes they don't detect EDQUOT conditions. Sometimes
> > repquota emits weird "device not found" errors. Sometimes grace
> > periods don't start, sometimes they don't time out, or time out and
> > then don't trigger EDQUOT. I don't know why these weird things are
> > happening yet, and until I properly integrate test group selection
> > into check-parallel I can't really isolate the quota tests to focus
> > on them alone.
> >
> > There are several patches that speed up test runtime. There were
> > several tests that were taking 12-15 minutes to run each, and these
> > made up >95% of the entire check-parallel runtime. In general, these
> > tests have been made to run with more concurrency to speed them up.
> > the result is that the longest individual test runtime has dropped
> > to around 7 minutes, and the elapsed runtime for check-parallel has
> > dropped to 8-10 minutes.
> >
> > Hence there are 39 patches that are doing prep work on tests and
> > infrastructure to make tests run reliabily in concurrent test
> > contexts. The last patch is the check-parallel runner script that
> > runs the tests concurrently.
> >
> > The check-parallel script is still very rudimentary. I hard coded
> > the tests it runs (generic+xfs auto tests) and the concurrency so
> > that I could run explicit sets of tests with './check --exact-order
> > <list>'. It is pointed at a mounted directory, and it creates all
> > the infrastructure it needs to run the tests within that directory.
> >
> > This enabled me to break up the tests across a set of identical
> > runner process contexts. Each runner does:
> >
> > - create runner directory
> > - create test and scratch image files
> > - create loopback devices for test and scratch devices
> > - sets up results directories
> > - execute check in it's own private mount namespace so it can't see
> > any of the mounts that other runners are using.
> > - tears down loopback devices
> > - reports test failures.
> >
> > If you run with the same directory over and over again, then it
> > reuses the same runner infrastructure and test and scratch image
> > files. The results directories are not overwritten as they are
> > date-stamped, hence using the same mount point automatically creates
> > a result archive for later data mining.
> >
> > A typical target directory (/mnt/xfs) looks like:
> >
> > /mnt/xfs/runner-0/...
> > /mnt/xfs/runner-1/...
> > ....
> > /mnt/xfs/runner-63/...
> >
> > And each runner directory:
> >
> > log
> > results-2024-11-19-11:22:25/...
> > results-2024-11-19-12:36:28/...
> > .....
> > results-2024-11-27-13:32:42/...
> > scratch/
> > scratch.img
> > test/
> > test.img
> >
> > The log file is the check output for that runner and should look
> > familiar:
> >
> > SECTION -- xfs
> > FSTYP -- xfs (debug)
> > PLATFORM -- Linux/x86_64 test1 6.12.0-dgc+ #297 SMP PREEMPT_DYNAMIC Wed Nov 27 08:13:06 AEDT 2024
> > MKFS_OPTIONS -- -f -m rmapbt=1 /dev/loop10
> > MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/loop10 /mnt/xfs/runner-10/scratch
> >
> > generic/387 309s
> > generic/551 49s
> > generic/289 2s
> > xfs/274 3s
> > generic/677 3s
> > generic/143 5s
> > generic/304 1s
> > generic/390 4s
> > generic/427 4s
> > xfs/103 1s
> > generic/252 3s
> > xfs/045 1s
> > generic/374 2s
> > generic/002 1s
> > generic/534 1s
> > generic/039 1s
> > generic/595 [not run] No encryption support for xfs
> > xfs/122 [not run] Could not compile test program (see end of /mnt/xfs/runner-10/results-2024-11-27-13:32:42/xfs/xfs/122.full)
> > xfs/556 [not run] xfs_scrub not found
> > Ran: generic/387 generic/551 generic/289 xfs/274 generic/677 generic/143 generic/304 generic/390 generic/427 xfs/103 generic/252 xfs/045 generic/374 generic/002 generic/534 generic/039 generic/595 xfs/122 xfs/556
> > Not run: generic/595 xfs/122 xfs/556
> > Passed all 19 tests
> >
> > SECTION -- xfs
> > =========================
> > Ran: generic/387 generic/551 generic/289 xfs/274 generic/677 generic/143 generic/304 generic/390 generic/427 xfs/103 generic/252 xfs/045 generic/374 generic/002 generic/534 generic/039 generic/595 xfs/122 xfs/556
> > Not run: generic/595 xfs/122 xfs/556
> > Passed all 19 tests
> >
> > Doing something with all the log files from a run can be done with
> > "/mnt/xfs/*/log". e.g. grep, vi, etc.
> >
> > The results directory contains the same check.{full,log,time} and
> > test results directories as per a normal check invocation, so
> > there's little difference in checking/analysing results with a
> > parallel execution run.
> >
> > Because the tests run in private mount namespaces, it's easy to see
> > what check-parallel is running at any point in time using 'pstree -N
> > mnt'. Here's a run that is hung on an unmount not completing:
> >
> > $ pstree -N mnt
> > [4026531841]
> > bash
> > bash───pstree
> > [0]
> > sudo───sudo───check-parallel─┬─check-parallel───nsexec───check───311───fsync-tester
> > ├─check-parallel───nsexec───check───467───open_by_handle
> > ├─check-parallel───nsexec───check───338
> > ├─check-parallel───nsexec───check───421
> > ├─check-parallel───nsexec───check───441
> > ├─check-parallel───nsexec───check───232
> > ├─check-parallel───nsexec───check───477───open_by_handle
> > ├─check-parallel───nsexec───check───420
> > ├─check-parallel───nsexec───check───426───open_by_handle
> > ├─check-parallel───nsexec───check───756───open_by_handle
> > ├─check-parallel───nsexec───check───231
> > ├─check-parallel───nsexec───check───475───475.fsstress───475.fsstress───{475.fsstress}
> > ├─check-parallel───nsexec───check───388───388.fsstress───388.fsstress───{388.fsstress}
> > ├─check-parallel───nsexec───check───259───sync
> > ├─check-parallel───nsexec───check───622───sync
> > ├─check-parallel───nsexec───check───318───sync
> > ├─check-parallel───nsexec───check───753───umount
> > ├─check-parallel───nsexec───check───086
> > ├─check-parallel───nsexec───check───648───648.fsstress───648.fsstress───{648.fsstress}
> > ├─check-parallel───nsexec───check───391
> > ├─check-parallel───nsexec───check───315───sync
> > └─check-parallel───nsexec───check───183───bulkstat_unlink
> >
> > All the other processes are in sync or dropping caches and stuck
> > waiting for the superblock s_umount lock that the unmount holds.
> > Finding where it is stuck:
> >
> > $ pgrep [u]mount
> > 1081885
> > $ sudo cat /proc/1081885/stack
> > [<0>] xfs_ail_push_all_sync+0x9c/0xf0
> > [<0>] xfs_unmount_flush_inodes+0x41/0x70
> > [<0>] xfs_unmountfs+0x59/0x190
> > [<0>] xfs_fs_put_super+0x3b/0x90
> > [<0>] generic_shutdown_super+0x77/0x160
> > [<0>] kill_block_super+0x1b/0x40
> > [<0>] xfs_kill_sb+0x12/0x30
> > [<0>] deactivate_locked_super+0x38/0x100
> > [<0>] deactivate_super+0x41/0x50
> > [<0>] cleanup_mnt+0x9f/0x160
> > [<0>] __cleanup_mnt+0x12/0x20
> > [<0>] task_work_run+0x89/0xb0
> > [<0>] resume_user_mode_work+0x4f/0x60
> > [<0>] syscall_exit_to_user_mode+0x76/0xb0
> > [<0>] do_syscall_64+0x74/0x130
> > [<0>] entry_SYSCALL_64_after_hwframe+0x76/0x7e
> > $
> >
> > Yup, known bug - a shutdown vs XFS_ISTALE inode cluster freeing
> > issue that leaves pinned, stale inodes in the AIL.
> >
> > The point I'm making here is that running tests concurrently doesn't
> > change anything material in how you'd go about discovering and
> > diagnosing failures. The only difference is in what that initial
> > failure might look like. e.g. Failures that result in dmesg output
> > will cause a huge number of tests to all fail with "check dmesg for
> > failure output" reports. Hence there is a bit of sifting to find
> > which test triggered the dmesg output, but failure detection still
> > works just fine.
> >
> > The check-parallel script is really only at proof-of-concept stage.
> > It is sufficient to run tests in parallel, but that's about it. most
> > of the work so far has gone into making the generic and XFS tests
> > run reliably in parallel. I have not run any other tests, and
> > haven't done full conversions of other test directories or C code in
> > src/. e.g. the sync -> syncfs was only done for the common dir and
> > the generic and xfs tests dirs
> >
> > I do not plan to do these conversions for ext4/btrfs/overlay/etc any
> > time soon as these conversions are mostly about improving execution
> > time, not test correctness. Hence they are not a priority for me -
> > the priority is further developing the concurrent execution
> > environment. i.e. check-parallel.
> >
> > To that end, I need to factor all the test selection and exclusion
> > code out of check, and do the same with the actual test list runner
> > loop. That way I can reuse all the existing code from within the
> > check-parallel context rather than having to call check to do all of
> > that work itself. I would like to get check-parallel to the point
> > where it is mostly just s/check/check-parallel/ on the command line
> > to move from serial to concurrent test execution.
> >
> > I also want to try to integrate the config section stuff into
> > check-parallel. This is more a case of defining what devices the
> > config needs to create (i.e. as loop devices) rather than what
> > devices it should be using. I think I can do this just by defining a
> > different set of environment variables (e.g. NEED_SCRATCHDEV,
> > NEED_LOGDEV, etc) and triggering the loop device creation from these
> > variables.
> >
> > In a way, the fact that check-parallel bootstraps it's own runtime
> > environment almost makes it entirely zero-config. As it stands right
> > now, you should be able to pull this patchset, create your base test
> > directory (make sure you have at least 100GB of free disk space),
> > run './check-parallel <test_dir> -x dump' and it Should Just Work.
> >
> > At this point, the test infrastructure problems are largely solved.
> > My focus is now on further development of the check-parallel script
> > and integrating it tightly into the existing check infrastructure
> > rather than open-coding test lists and configuration information.
> > This will probably take a bit of time, so I'd like to get the bug
> > fixes, improvements and infrastructure changes underway so I'm not
> > left carrying a huge patchset for months....
> >
> > ----------------------------------------------------------------
> > Dave Chinner (40):
> > xfs/448: get rid of assert-on-failure
> > fstests: cleanup fsstress process management
> > fuzzy: don't use killall
> > fstests: per-test dmflakey instances
> > fstests: per-test dmerror instances
> > fstests: per-test dmhuge instances
> > fstests: per-test dmthin instances
> > fstests: per-test dmdust instances
> > fstests: per-test dmdelay instances
> > fstests: fix DM device creation/removal vs udev races
> > fstests: use syncfs rather than sync
> > fstests: clean up mount and unmount operations
> > fstests: clean up loop device instantiation
> > fstests: xfs/227 is really slow
> > fstests: mark tests that are unreliable when run in parallel
> > fstests: use udevadm wait in preference to settle
> > xfs/442: rescale load so it's not exponential
> > xfs/176: fix broken setup code
> > xfs/177: remove unused slab object count location checks
> > fstests: remove uses of killall where possible
> > generic/127: reduce runtime
> > quota: system project quota files need to be shared
> > dmesg: reduce noise from other tests
> > fstests: stop using /tmp directly
> > fstests: scale some tests for high CPU count sanity
> > generic/310: cleanup killing background processes
> > filter: handle mount errors from CONFIG_BLK_DEV_WRITE_MOUNTED=y
> > filters: add a filter that accepts EIO instead of other errors
> > generic/085: general cleanup for reliability and debugging
> > fstests: don't use directory stacks
> > fstests: clean up a couple of dm-flakey tests
> > fstests: clean up termination of various tests
> > vfstests: some tests require the testdir to be shared
> > xfs/629: single extent files should be within tolerance
> > xfs/076: fix broken mkfs filtering
> > fstests: capture some failures to seqres.full
> > fstests: always use fail-at-unmount semantics for XFS
> > generic/062: don't leave debug files in $here on failure
> > fstests: quota grace periods unreliable under load
> > fstests: check-parallel
> >
> > check | 12 -
> > check-parallel | 205 ++++++++++++++++++
> > common/btrfs | 4
> > common/config | 36 ++-
> > common/dmdelay | 24 +-
> > common/dmdust | 14 -
> > common/dmerror | 74 +++---
> > common/dmflakey | 60 ++---
> > common/dmhugedisk | 21 +
> > common/dmlogwrites | 4
> > common/dmthin | 12 -
> > common/encrypt | 2
> > common/filter | 17 +
> > common/fuzzy | 37 +--
> > common/log | 2
> > common/metadump | 32 +-
> > common/overlay | 10
> > common/populate | 8
> > common/preamble | 1
> > common/quota | 37 ---
> > common/rc | 166 +++++++++++---
> > common/repair | 2
> > common/report | 2
> > common/verity | 2
> > common/xfs | 2
> > doc/group-names.txt | 1
> > doc/requirement-checking.txt | 6
> > ltp/fsstress.c | 28 ++
> > src/aio-dio-regress/aio-last-ref-held-by-io.c | 5
> > src/dmerror | 6
> > tests/btrfs/004 | 11
> > tests/btrfs/007 | 3
> > tests/btrfs/012 | 4
> > tests/btrfs/028 | 6
> > tests/btrfs/049 | 4
> > tests/btrfs/057 | 4
> > tests/btrfs/060 | 14 -
> > tests/btrfs/061 | 13 -
> > tests/btrfs/062 | 13 -
> > tests/btrfs/063 | 13 -
> > tests/btrfs/064 | 13 -
> > tests/btrfs/065 | 14 -
> > tests/btrfs/066 | 14 -
> > tests/btrfs/067 | 14 -
> > tests/btrfs/068 | 14 -
> > tests/btrfs/069 | 13 -
> > tests/btrfs/070 | 13 -
> > tests/btrfs/071 | 13 -
> > tests/btrfs/072 | 14 -
> > tests/btrfs/073 | 13 -
> > tests/btrfs/074 | 13 -
> > tests/btrfs/078 | 12 -
> > tests/btrfs/100 | 4
> > tests/btrfs/101 | 4
> > tests/btrfs/136 | 6
> > tests/btrfs/160 | 3
> > tests/btrfs/192 | 12 -
> > tests/btrfs/195 | 2
> > tests/btrfs/212 | 16 -
> > tests/btrfs/232 | 4
> > tests/btrfs/252 | 5
> > tests/btrfs/261 | 2
> > tests/btrfs/284 | 4
> > tests/btrfs/286 | 2
> > tests/btrfs/291 | 5
> > tests/btrfs/320 | 6
> > tests/btrfs/332 | 4
> > tests/ext4/004 | 4
> > tests/ext4/057 | 14 -
> > tests/ext4/058 | 3
> > tests/ext4/307 | 4
> > tests/generic/013 | 22 -
> > tests/generic/015 | 2
> > tests/generic/019 | 13 -
> > tests/generic/029 | 2
> > tests/generic/030 | 2
> > tests/generic/032 | 2
> > tests/generic/034 | 2
> > tests/generic/039 | 2
> > tests/generic/040 | 2
> > tests/generic/041 | 2
> > tests/generic/042 | 4
> > tests/generic/048 | 2
> > tests/generic/049 | 4
> > tests/generic/050 | 3
> > tests/generic/051 | 24 --
> > tests/generic/054 | 2
> > tests/generic/055 | 6
> > tests/generic/057 | 2
> > tests/generic/059 | 2
> > tests/generic/062 | 2
> > tests/generic/065 | 2
> > tests/generic/066 | 5
> > tests/generic/067 | 17 -
> > tests/generic/068 | 7
> > tests/generic/070 | 12 -
> > tests/generic/073 | 2
> > tests/generic/076 | 10
> > tests/generic/076.out | 1
> > tests/generic/081 | 8
> > tests/generic/083 | 13 -
> > tests/generic/083.out | 1
> > tests/generic/084 | 12 -
> > tests/generic/085 | 21 +
> > tests/generic/090 | 4
> > tests/generic/092 | 2
> > tests/generic/098 | 4
> > tests/generic/099 | 8
> > tests/generic/101 | 2
> > tests/generic/104 | 2
> > tests/generic/106 | 2
> > tests/generic/107 | 2
> > tests/generic/108 | 9
> > tests/generic/109 | 5
> > tests/generic/117 | 4
> > tests/generic/127 | 67 +++--
> > tests/generic/127.out | 6
> > tests/generic/135 | 24 --
> > tests/generic/150 | 2
> > tests/generic/151 | 2
> > tests/generic/152 | 2
> > tests/generic/157 | 3
> > tests/generic/158 | 3
> > tests/generic/159 | 2
> > tests/generic/160 | 2
> > tests/generic/162 | 4
> > tests/generic/163 | 4
> > tests/generic/164 | 4
> > tests/generic/165 | 4
> > tests/generic/166 | 6
> > tests/generic/167 | 4
> > tests/generic/168 | 7
> > tests/generic/170 | 7
> > tests/generic/171 | 6
> > tests/generic/172 | 6
> > tests/generic/173 | 6
> > tests/generic/174 | 6
> > tests/generic/204 | 8
> > tests/generic/232 | 5
> > tests/generic/232.out | 1
> > tests/generic/247 | 2
> > tests/generic/250 | 2
> > tests/generic/251 | 5
> > tests/generic/252 | 5
> > tests/generic/265 | 2
> > tests/generic/266 | 2
> > tests/generic/267 | 2
> > tests/generic/268 | 2
> > tests/generic/269 | 8
> > tests/generic/270 | 10
> > tests/generic/271 | 2
> > tests/generic/272 | 2
> > tests/generic/273 | 2
> > tests/generic/274 | 6
> > tests/generic/275 | 6
> > tests/generic/276 | 2
> > tests/generic/278 | 2
> > tests/generic/279 | 2
> > tests/generic/281 | 2
> > tests/generic/282 | 2
> > tests/generic/283 | 2
> > tests/generic/306 | 2
> > tests/generic/310 | 24 +-
> > tests/generic/315 | 2
> > tests/generic/317 | 2
> > tests/generic/318 | 2
> > tests/generic/321 | 4
> > tests/generic/323 | 7
> > tests/generic/325 | 2
> > tests/generic/328 | 4
> > tests/generic/329 | 5
> > tests/generic/330 | 2
> > tests/generic/331 | 7
> > tests/generic/332 | 4
> > tests/generic/333 | 6
> > tests/generic/334 | 6
> > tests/generic/335 | 2
> > tests/generic/336 | 9
> > tests/generic/341 | 2
> > tests/generic/342 | 2
> > tests/generic/343 | 2
> > tests/generic/347 | 2
> > tests/generic/348 | 2
> > tests/generic/353 | 2
> > tests/generic/361 | 8
> > tests/generic/373 | 4
> > tests/generic/374 | 4
> > tests/generic/376 | 2
> > tests/generic/382 | 2
> > tests/generic/387 | 2
> > tests/generic/388 | 24 --
> > tests/generic/390 | 11
> > tests/generic/391 | 2
> > tests/generic/395 | 2
> > tests/generic/409 | 11
> > tests/generic/410 | 11
> > tests/generic/411 | 11
> > tests/generic/416 | 2
> > tests/generic/422 | 4
> > tests/generic/425 | 2
> > tests/generic/441 | 2
> > tests/generic/459 | 8
> > tests/generic/461 | 17 -
> > tests/generic/464 | 10
> > tests/generic/474 | 4
> > tests/generic/475 | 17 -
> > tests/generic/476 | 15 -
> > tests/generic/479 | 2
> > tests/generic/480 | 2
> > tests/generic/482 | 12 -
> > tests/generic/483 | 2
> > tests/generic/484 | 3
> > tests/generic/489 | 2
> > tests/generic/502 | 2
> > tests/generic/505 | 2
> > tests/generic/506 | 2
> > tests/generic/507 | 2
> > tests/generic/508 | 2
> > tests/generic/510 | 2
> > tests/generic/520 | 2
> > tests/generic/526 | 2
> > tests/generic/527 | 2
> > tests/generic/530 | 2
> > tests/generic/531 | 8
> > tests/generic/535 | 2
> > tests/generic/546 | 2
> > tests/generic/547 | 5
> > tests/generic/556 | 2
> > tests/generic/560 | 7
> > tests/generic/561 | 25 +-
> > tests/generic/563 | 24 +-
> > tests/generic/564 | 12 -
> > tests/generic/579 | 8
> > tests/generic/585 | 4
> > tests/generic/589 | 11
> > tests/generic/590 | 9
> > tests/generic/599 | 2
> > tests/generic/601 | 7
> > tests/generic/603 | 14 -
> > tests/generic/604 | 2
> > tests/generic/610 | 2
> > tests/generic/620 | 1
> > tests/generic/628 | 4
> > tests/generic/629 | 4
> > tests/generic/631 | 2
> > tests/generic/632 | 2
> > tests/generic/640 | 2
> > tests/generic/642 | 10
> > tests/generic/648 | 25 --
> > tests/generic/650 | 19 -
> > tests/generic/670 | 2
> > tests/generic/671 | 2
> > tests/generic/672 | 2
> > tests/generic/673 | 2
> > tests/generic/674 | 2
> > tests/generic/675 | 2
> > tests/generic/677 | 2
> > tests/generic/683 | 2
> > tests/generic/684 | 2
> > tests/generic/685 | 2
> > tests/generic/686 | 2
> > tests/generic/687 | 2
> > tests/generic/688 | 2
> > tests/generic/690 | 2
> > tests/generic/691 | 6
> > tests/generic/694 | 2
> > tests/generic/695 | 2
> > tests/generic/698 | 4
> > tests/generic/699 | 8
> > tests/generic/703 | 2
> > tests/generic/704 | 2
> > tests/generic/707 | 7
> > tests/generic/716 | 2
> > tests/generic/717 | 2
> > tests/generic/718 | 2
> > tests/generic/719 | 2
> > tests/generic/721 | 2
> > tests/generic/722 | 15 -
> > tests/generic/725 | 2
> > tests/generic/726 | 2
> > tests/generic/727 | 2
> > tests/generic/730 | 2
> > tests/generic/731 | 2
> > tests/generic/732 | 4
> > tests/generic/735 | 2
> > tests/generic/738 | 2
> > tests/generic/741 | 2
> > tests/generic/743 | 4
> > tests/generic/744 | 10
> > tests/generic/745 | 4
> > tests/generic/746 | 18 -
> > tests/generic/747 | 4
> > tests/generic/749 | 4
> > tests/generic/750 | 10
> > tests/generic/751 | 1
> > tests/generic/753 | 17 -
> > tests/overlay/019 | 48 ++--
> > tests/overlay/021 | 8
> > tests/overlay/058 | 12 -
> > tests/xfs/006 | 7
> > tests/xfs/011 | 11
> > tests/xfs/013 | 22 -
> > tests/xfs/014 | 11
> > tests/xfs/016 | 4
> > tests/xfs/017 | 13 -
> > tests/xfs/017.out | 1
> > tests/xfs/032 | 2
> > tests/xfs/049 | 39 ++-
> > tests/xfs/050 | 5
> > tests/xfs/051 | 10
> > tests/xfs/052 | 2
> > tests/xfs/057 | 9
> > tests/xfs/070 | 9
> > tests/xfs/073 | 36 +--
> > tests/xfs/074 | 23 +-
> > tests/xfs/076 | 8
> > tests/xfs/077 | 2
> > tests/xfs/078 | 20 -
> > tests/xfs/079 | 17 -
> > tests/xfs/104 | 14 -
> > tests/xfs/110 | 2
> > tests/xfs/118 | 4
> > tests/xfs/119 | 2
> > tests/xfs/128 | 2
> > tests/xfs/133 | 2
> > tests/xfs/134 | 2
> > tests/xfs/137 | 2
> > tests/xfs/141 | 13 -
> > tests/xfs/148 | 29 +-
> > tests/xfs/149 | 8
> > tests/xfs/154 | 1
> > tests/xfs/158 | 4
> > tests/xfs/161 | 2
> > tests/xfs/167 | 17 -
> > tests/xfs/168 | 8
> > tests/xfs/176 | 10
> > tests/xfs/177 | 15 -
> > tests/xfs/186 | 4
> > tests/xfs/195 | 2
> > tests/xfs/201 | 4
> > tests/xfs/212 | 2
> > tests/xfs/216 | 23 +-
> > tests/xfs/217 | 24 +-
> > tests/xfs/227 | 59 +++--
> > tests/xfs/231 | 4
> > tests/xfs/232 | 10
> > tests/xfs/234 | 2
> > tests/xfs/236 | 2
> > tests/xfs/237 | 13 -
> > tests/xfs/239 | 2
> > tests/xfs/240 | 7
> > tests/xfs/241 | 2
> > tests/xfs/243 | 11
> > tests/xfs/246 | 2
> > tests/xfs/250 | 19 +
> > tests/xfs/259 | 13 -
> > tests/xfs/264 | 4
> > tests/xfs/265 | 2
> > tests/xfs/270 | 2
> > tests/xfs/272 | 2
> > tests/xfs/274 | 2
> > tests/xfs/276 | 2
> > tests/xfs/289 | 4
> > tests/xfs/291 | 4
> > tests/xfs/297 | 12 -
> > tests/xfs/300 | 8
> > tests/xfs/305 | 8
> > tests/xfs/309 | 2
> > tests/xfs/312 | 2
> > tests/xfs/313 | 2
> > tests/xfs/314 | 2
> > tests/xfs/315 | 4
> > tests/xfs/316 | 2
> > tests/xfs/317 | 2
> > tests/xfs/318 | 4
> > tests/xfs/319 | 2
> > tests/xfs/320 | 2
> > tests/xfs/321 | 2
> > tests/xfs/322 | 2
> > tests/xfs/323 | 2
> > tests/xfs/324 | 2
> > tests/xfs/325 | 4
> > tests/xfs/326 | 4
> > tests/xfs/327 | 2
> > tests/xfs/420 | 2
> > tests/xfs/421 | 2
> > tests/xfs/423 | 2
> > tests/xfs/438 | 2
> > tests/xfs/440 | 8
> > tests/xfs/442 | 15 -
> > tests/xfs/448 | 6
> > tests/xfs/495 | 2
> > tests/xfs/501 | 2
> > tests/xfs/502 | 2
> > tests/xfs/507 | 2
> > tests/xfs/511 | 2
> > tests/xfs/513 | 48 +---
> > tests/xfs/519 | 2
> > tests/xfs/520 | 2
> > tests/xfs/521 | 8
> > tests/xfs/527 | 5
> > tests/xfs/528 | 10
> > tests/xfs/530 | 11
> > tests/xfs/538 | 4
> > tests/xfs/541 | 2
> > tests/xfs/544 | 2
> > tests/xfs/553 | 4
> > tests/xfs/558 | 7
> > tests/xfs/601 | 2
> > tests/xfs/606 | 14 -
> > tests/xfs/607 | 6
> > tests/xfs/609 | 20 -
> > tests/xfs/610 | 20 -
> > tests/xfs/613 | 44 +--
> > tests/xfs/613.out | 1
> > tests/xfs/617 | 2
> > tests/xfs/629 | 6
> > tests/xfs/630 | 2
> > tests/xfs/631 | 9
> > tests/xfs/790 | 2
> > tests/xfs/791 | 2
> > tests/xfs/792 | 2
> > tests/xfs/802 | 7
> > 423 files changed, 1827 insertions(+), 1629 deletions(-)
> >
> >
>
>
next prev parent reply other threads:[~2024-12-07 0:09 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-27 4:51 [RFC PATCH 00/40] fstests: concurrent test execution Dave Chinner
2024-11-27 4:51 ` [PATCH 01/40] xfs/448: get rid of assert-on-failure Dave Chinner
2024-11-27 4:51 ` [PATCH 02/40] fstests: cleanup fsstress process management Dave Chinner
2024-11-29 4:03 ` Zorro Lang
2024-12-04 17:57 ` Zorro Lang
2024-12-05 4:42 ` Dave Chinner
2024-12-05 9:57 ` Zorro Lang
2024-12-04 18:04 ` Zorro Lang
2024-12-05 4:55 ` Dave Chinner
2024-12-05 10:05 ` Zorro Lang
2024-11-27 4:51 ` [PATCH 03/40] fuzzy: don't use killall Dave Chinner
2024-11-27 4:51 ` [PATCH 04/40] fstests: per-test dmflakey instances Dave Chinner
2024-11-27 4:51 ` [PATCH 05/40] fstests: per-test dmerror instances Dave Chinner
2024-11-27 4:51 ` [PATCH 06/40] fstests: per-test dmhuge instances Dave Chinner
2024-11-27 4:51 ` [PATCH 07/40] fstests: per-test dmthin instances Dave Chinner
2024-11-27 4:51 ` [PATCH 08/40] fstests: per-test dmdust instances Dave Chinner
2024-11-27 4:51 ` [PATCH 09/40] fstests: per-test dmdelay instances Dave Chinner
2024-11-27 4:51 ` [PATCH 10/40] fstests: fix DM device creation/removal vs udev races Dave Chinner
2024-11-27 4:51 ` [PATCH 11/40] fstests: use syncfs rather than sync Dave Chinner
2024-11-27 4:51 ` [PATCH 12/40] fstests: clean up mount and unmount operations Dave Chinner
2024-11-27 4:51 ` [PATCH 13/40] fstests: clean up loop device instantiation Dave Chinner
2024-12-01 12:31 ` Zorro Lang
2024-12-01 12:50 ` Zorro Lang
2024-12-07 12:44 ` Zorro Lang
2024-12-07 18:59 ` Zorro Lang
2024-12-07 19:51 ` Zorro Lang
2024-11-27 4:51 ` [PATCH 14/40] fstests: xfs/227 is really slow Dave Chinner
2024-11-27 4:51 ` [PATCH 15/40] fstests: mark tests that are unreliable when run in parallel Dave Chinner
2024-11-27 4:51 ` [PATCH 16/40] fstests: use udevadm wait in preference to settle Dave Chinner
2024-11-29 17:10 ` Darrick J. Wong
2024-11-29 22:33 ` Dave Chinner
2024-11-30 2:34 ` Zorro Lang
2024-11-27 4:51 ` [PATCH 17/40] xfs/442: rescale load so it's not exponential Dave Chinner
2024-11-27 4:51 ` [PATCH 18/40] xfs/176: fix broken setup code Dave Chinner
2024-11-27 4:51 ` [PATCH 19/40] xfs/177: remove unused slab object count location checks Dave Chinner
2024-11-27 4:51 ` [PATCH 20/40] fstests: remove uses of killall where possible Dave Chinner
2024-11-27 4:51 ` [PATCH 21/40] generic/127: reduce runtime Dave Chinner
2024-11-27 4:51 ` [PATCH 22/40] quota: system project quota files need to be shared Dave Chinner
2024-11-27 4:51 ` [PATCH 23/40] dmesg: reduce noise from other tests Dave Chinner
2024-11-27 4:51 ` [PATCH 24/40] fstests: stop using /tmp directly Dave Chinner
2024-11-27 4:51 ` [PATCH 25/40] fstests: scale some tests for high CPU count sanity Dave Chinner
2024-11-29 3:34 ` Zorro Lang
2024-11-27 4:51 ` [PATCH 26/40] generic/310: cleanup killing background processes Dave Chinner
2024-11-27 4:51 ` [PATCH 27/40] filter: handle mount errors from CONFIG_BLK_DEV_WRITE_MOUNTED=y Dave Chinner
2024-11-27 4:51 ` [PATCH 28/40] filters: add a filter that accepts EIO instead of other errors Dave Chinner
2024-11-27 4:51 ` [PATCH 29/40] generic/085: general cleanup for reliability and debugging Dave Chinner
2024-11-27 4:52 ` [PATCH 30/40] fstests: don't use directory stacks Dave Chinner
2024-12-01 12:10 ` Zorro Lang
2024-12-01 21:37 ` Dave Chinner
2024-11-27 4:52 ` [PATCH 31/40] fstests: clean up a couple of dm-flakey tests Dave Chinner
2024-11-27 4:52 ` [PATCH 32/40] fstests: clean up termination of various tests Dave Chinner
2024-11-27 4:52 ` [PATCH 33/40] vfstests: some tests require the testdir to be shared Dave Chinner
2024-11-27 4:52 ` [PATCH 34/40] xfs/629: single extent files should be within tolerance Dave Chinner
2024-11-27 4:52 ` [PATCH 35/40] xfs/076: fix broken mkfs filtering Dave Chinner
2024-11-27 4:52 ` [PATCH 36/40] fstests: capture some failures to seqres.full Dave Chinner
2024-11-27 4:52 ` [PATCH 37/40] fstests: always use fail-at-unmount semantics for XFS Dave Chinner
2024-11-27 4:52 ` [PATCH 38/40] generic/062: don't leave debug files in $here on failure Dave Chinner
2024-11-27 4:52 ` [PATCH 39/40] fstests: quota grace periods unreliable under load Dave Chinner
2024-11-27 4:52 ` [PATCH 40/40] fstests: check-parallel Dave Chinner
2024-11-29 4:22 ` [RFC PATCH 00/40] fstests: concurrent test execution Zorro Lang
2024-12-07 0:09 ` Darrick J. Wong [this message]
2024-12-07 9:38 ` Zorro Lang
2024-12-08 0:02 ` Dave Chinner
2024-12-08 6:15 ` Zorro Lang
2024-12-10 0:55 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20241207000917.GA7824@frogsfrogsfrogs \
--to=djwong@kernel.org \
--cc=david@fromorbit.com \
--cc=fstests@vger.kernel.org \
--cc=zlang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox