From: "Theodore Ts'o" <tytso@mit.edu>
To: "Darrick J. Wong" <djwong@kernel.org>
Cc: zlang@redhat.com, linux-xfs@vger.kernel.org,
fstests@vger.kernel.org, guan@eryu.me
Subject: Re: [PATCH 1/2] check: add a -smoketest option
Date: Thu, 27 Jul 2023 15:04:10 -0400 [thread overview]
Message-ID: <20230727190410.GI30264@mit.edu> (raw)
In-Reply-To: <169033660570.3222210.3010411210438664310.stgit@frogsfrogsfrogs>
On Tue, Jul 25, 2023 at 06:56:45PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> Create a "-smoketest" parameter to check that will run generic
> filesystem smoke testing for five minutes apiece. Since there are only
> five smoke tests, this is effectively a 16min super-quick test.
The code is setting SOAK_DURATION to 4 minutes, not 5 minutes.
However, when I ran the moral equivalent:
kvm-xfstests --soak-duration 4m --fail-loop-count 0 -c ext4/4k \
generic/475 generic/476 generic/521 generic/522 generic/642
It overall took 17 minutes to run, with just under a minute of test
infrastructure overhead (in the check script and my wrapper scripts),
with the actual test time as follows:
ext4/4k: 5 tests, 975 seconds
generic/475 Pass 242s
generic/476 Pass 244s
generic/521 Pass 241s
generic/522 Pass 241s
generic/642 Pass 7s
Totals: 5 tests, 0 skipped, 0 failures, 0 errors, 975s
The time which generic/642 ran was surprising so I took a closer look.
It does claim to be in group "soak", and it even tries to canonicalize
SOAK_DURATION (I'm not sure why, since the check script does this
already). But generic/642 doesn't seem to use SOAK_DURATION. It does
caculate a DURATION value, but it doesn't actually use SOAK_DURATION.
So that sounds like a bug in the generic/642 test?
There was also a bug xfstests's "make install" in that it doesn't
actually install src/soak_duration.awk, but I'll send that a patch
fixing that under separate cover.
Darrick -- suppose changed the SOAK_DURATION down to 2 minutes? How
much do you think that would materially affect the code coverage
metrics, and the overall effectiveness of the smoke test? If we get
generci/642 to honor SOAK_DURATION, using an overall 2 minute soak for
each test would translate to the smoke test taking about 13 minutes,
which would be great from a drive-by patch submitter perspective.
- Ted
> With gcov enabled, running these tests yields about ~75% coverage for
> iomap and ~60% for xfs; or ~50% for ext4 and ~75% for ext4; and ~45% for
> btrfs. Coverage was about ~65% for the pagecache.
next prev parent reply other threads:[~2023-07-27 19:04 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-26 1:56 [PATCHSET v2 0/2] fstests: testing improvements Darrick J. Wong
2023-07-26 1:56 ` [PATCH 1/2] check: add a -smoketest option Darrick J. Wong
2023-07-27 19:04 ` Theodore Ts'o [this message]
2023-07-26 1:56 ` [PATCH 2/2] check: generate gcov code coverage reports at the end of each section Darrick J. Wong
2023-07-27 14:57 ` Zorro Lang
-- strict thread matches above, loose matches on Subject: below --
2023-07-19 1:10 [PATCHSET 0/2] fstests: testing improvements Darrick J. Wong
2023-07-19 1:10 ` [PATCH 1/2] check: add a -smoketest option Darrick J. Wong
2023-07-19 15:10 ` Zorro Lang
2023-07-19 15:29 ` Darrick J. Wong
2023-07-19 16:11 ` Zorro Lang
2023-07-20 2:27 ` Darrick J. Wong
2023-07-20 14:34 ` Zorro Lang
2023-07-26 0:05 ` Darrick J. Wong
2023-07-26 6:01 ` Theodore Ts'o
2023-07-26 14:54 ` Zorro Lang
2023-07-26 20:59 ` Theodore Ts'o
2023-07-27 1:36 ` Theodore Ts'o
2023-07-27 1:54 ` Darrick J. Wong
2023-07-27 3:25 ` Zorro Lang
2023-07-27 14:33 ` Theodore Ts'o
2023-07-27 15:30 ` Zorro Lang
2023-07-28 15:53 ` Theodore Ts'o
2023-07-28 19:52 ` Zorro Lang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230727190410.GI30264@mit.edu \
--to=tytso@mit.edu \
--cc=djwong@kernel.org \
--cc=fstests@vger.kernel.org \
--cc=guan@eryu.me \
--cc=linux-xfs@vger.kernel.org \
--cc=zlang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.