Re: [PATCH] fstests: provides smoketest template

["Darrick J. Wong" <djwong@kernel.org>]

On Thu, Jul 27, 2023 at 04:03:27AM +0800, Zorro Lang wrote:
> Darrick suggests that fstests can provide a simple smoketest, by running
> several generic filesystem smoke testing for five minutes apiece. Since
> there are only five smoke tests, this is effectively a 16min super-quick
> test.
> 
> With gcov enabled, running these tests yields about ~75% coverage for
> iomap and ~60% for xfs; or ~50% for ext4 and ~75% for ext4; and ~45% for
> btrfs.  Coverage was about ~65% for the pagecache.
> 
> To implement that, this patch add a new "-t" option to ./check, and a
> new directory "template" under xfstests/, then we can have smoketest
> template, also can have more other testing templates.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Zorro Lang <zlang@kernel.org>
> ---
> 
> Hi,
> 
> This patch uses another way to achieve the smoketest requirement[1]. When
> I reviewed the patch from Darrick [2], I thought "smoketest" might not be
> the last one requirement likes that. I don't want to give each kind of
> tests a separated option. But we might can leave a hook for more fs-devel
> who have good testing templates for fstests.
> 
> Although we have test groups, but those group names are too complex for
> the users who not always use fstests. So I'm thinking about providing
> some simple templates to run fstests, these templates base on test groups
> and some fstests global parameters, help users to know what kind of
> test they can do.
> 
> Feel free to discuss, and if most of you prefer the original patch [2],
> I'll also think about merging the original one :)
> 
> Thanks,
> Zorro
> 
> [1]
> https://lore.kernel.org/fstests/20230726145441.lbzzokwigrztimyq@zlang-mailbox/T/#mabc0de98699f1b877c87caccb13809c9283c0606
> [2]
> https://lore.kernel.org/fstests/169033660570.3222210.3010411210438664310.stgit@frogsfrogsfrogs/T/#u
> 
> 
>  check               |  8 ++++++++
>  doc/group-names.txt |  1 +
>  templates/smoketest | 16 ++++++++++++++++
>  tests/generic/475   |  2 +-
>  tests/generic/476   |  2 +-
>  tests/generic/521   |  2 +-
>  tests/generic/522   |  2 +-
>  tests/generic/642   |  2 +-
>  8 files changed, 30 insertions(+), 5 deletions(-)
>  create mode 100644 templates/smoketest
> 
> diff --git a/check b/check
> index 89e7e7bf..7100aae4 100755
> --- a/check
> +++ b/check
> @@ -335,6 +335,14 @@ while [ $# -gt 0 ]; do
>  		;;
>  	-i)	iterations=$2; shift ;;
>  	-I) 	iterations=$2; istop=true; shift ;;
> +	-t)
> +		source templates/$2
> +		if [ $? -ne 0 ];then
> +			echo "Cannot import the templates/$2"
> +			exit 1
> +		fi
> +		shift
> +		;;
>  	-T)	timestamp=true ;;
>  	-d)	DUMP_OUTPUT=true ;;
>  	-b)	brief_test_summary=true;;
> diff --git a/doc/group-names.txt b/doc/group-names.txt
> index 1c35a394..c3dcca37 100644
> --- a/doc/group-names.txt
> +++ b/doc/group-names.txt
> @@ -118,6 +118,7 @@ selftest		tests with fixed results, used to validate testing setup
>  send			btrfs send/receive
>  shrinkfs		decreasing the size of a filesystem
>  shutdown		FS_IOC_SHUTDOWN ioctl
> +smoketest		Simple smoke tests
>  snapshot		btrfs snapshots
>  soak			long running soak tests whose runtime can be controlled
>                          directly by setting the SOAK_DURATION variable
> diff --git a/templates/smoketest b/templates/smoketest
> new file mode 100644
> index 00000000..40a0104b
> --- /dev/null
> +++ b/templates/smoketest
> @@ -0,0 +1,16 @@
> +##/bin/bash
> +# For infrequent filesystem developers who simply want to run a quick test
> +# of the most commonly used filesystem functionality, use this command:
> +#
> +#     ./check -t smoketest <other config options>
> +#
> +# This template helps fstests to run several tests to exercise the file I/O,
> +# metadata, and crash recovery exercisers for four minutes apiece.  This
> +# should complete in approximately 20 minutes.
> +
> +echo "**********************"
> +echo "* A Quick Smoke Test *"
> +echo "**********************"
> +
> +[ -z "$SOAK_DURATION" ] && SOAK_DURATION="4m"
> +GROUP_LIST="smoketest"

Why not simply put

SOAK_DURATION=4m
GROUP_LIST=smoketest

in configs/smoketest.config and tell people to run

HOST_OPTIONS=configs/smoketest.config ./check

?

--D

> diff --git a/tests/generic/475 b/tests/generic/475
> index 0cbf5131..ce7fe013 100755
> --- a/tests/generic/475
> +++ b/tests/generic/475
> @@ -12,7 +12,7 @@
>  # testing efforts.
>  #
>  . ./common/preamble
> -_begin_fstest shutdown auto log metadata eio recoveryloop
> +_begin_fstest shutdown auto log metadata eio recoveryloop smoketest
>  
>  # Override the default cleanup function.
>  _cleanup()
> diff --git a/tests/generic/476 b/tests/generic/476
> index 8e93b734..b1ae4df4 100755
> --- a/tests/generic/476
> +++ b/tests/generic/476
> @@ -8,7 +8,7 @@
>  # bugs in the write path.
>  #
>  . ./common/preamble
> -_begin_fstest auto rw long_rw stress soak
> +_begin_fstest auto rw long_rw stress soak smoketest
>  
>  # Override the default cleanup function.
>  _cleanup()
> diff --git a/tests/generic/521 b/tests/generic/521
> index 22dd31a8..0956e501 100755
> --- a/tests/generic/521
> +++ b/tests/generic/521
> @@ -7,7 +7,7 @@
>  # Long-soak directio fsx test
>  #
>  . ./common/preamble
> -_begin_fstest soak long_rw
> +_begin_fstest soak long_rw smoketest
>  
>  # Import common functions.
>  . ./common/filter
> diff --git a/tests/generic/522 b/tests/generic/522
> index f0cbcb24..0e4e6009 100755
> --- a/tests/generic/522
> +++ b/tests/generic/522
> @@ -7,7 +7,7 @@
>  # Long-soak buffered fsx test
>  #
>  . ./common/preamble
> -_begin_fstest soak long_rw
> +_begin_fstest soak long_rw smoketest
>  
>  # Import common functions.
>  . ./common/filter
> diff --git a/tests/generic/642 b/tests/generic/642
> index eba90903..e6a475a8 100755
> --- a/tests/generic/642
> +++ b/tests/generic/642
> @@ -8,7 +8,7 @@
>  # bugs in the xattr code.
>  #
>  . ./common/preamble
> -_begin_fstest auto soak attr long_rw stress
> +_begin_fstest auto soak attr long_rw stress smoketest
>  
>  _cleanup()
>  {
> -- 
> 2.40.1
>