Re: [PATCH v2 02/18] selftest/firmware: Add selftest timeout in settings

From: Scott Branden <scott.branden@broadcom.com>
To: Kees Cook <keescook@chromium.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: linux-efi@vger.kernel.org,
	"Rafael J. Wysocki" <rafael@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	linux-fsdevel@vger.kernel.org,
	Stephen Boyd <stephen.boyd@linaro.org>,
	SeongJae Park <sjpark@amazon.de>,
	Mimi Zohar <zohar@linux.ibm.com>,
	David Howells <dhowells@redhat.com>,
	Tushar Sugandhi <tusharsu@linux.microsoft.com>,
	Peter Jones <pjones@redhat.com>,
	linux-kselftest@vger.kernel.org,
	"Joel Fernandes (Google)" <joel@joelfernandes.org>,
	Shuah Khan <shuah@kernel.org>, Ard Biesheuvel <ardb@kernel.org>,
	Thomas Cedeno <thomascedeno@google.com>,
	linux-security-module@vger.kernel.org,
	Anders Roxell <anders.roxell@linaro.org>,
	Paul Moore <paul@paul-moore.com>,
	Mauro Carvalho Chehab <mchehab+huawei@kernel.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	Nayna Jain <nayna@linux.ibm.com>,
	Matthew Garrett <matthewgarrett@google.com>,
	James Morris <jmorris@namei.org>,
	Lakshmi Ramasubramanian <nramas@linux.microsoft.com>,
	Aaron Goidel <acgoide@tycho.nsa.gov>,
	"Serge E. Hallyn" <serge@hallyn.com>,
	Wenwen Wang <wenwen@cs.uga.edu>,
	selinux@vger.kernel.org, Hans de Goede <hdegoede@redhat.com>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	Matthieu Baerts <matthieu.baerts@tessares.net>,
	KP Singh <kpsingh@chromium.org>,
	Eric Paris <eparis@parisplace.org>,
	linux-integrity@vger.kernel.org,
	Florent Revest <revest@google.com>,
	Andrea Righi <andrea.righi@canonical.com>,
	Dmitry Kasatkin <dmitry.kasatkin@gmail.com>,
	Stephen Smalley <stephen.smalley.work@gmail.com>,
	Randy Dunlap <rdunlap@infradead.org>,
	kexec@lists.infradead.org, linux-kernel@vger.kernel.org,
	Luis Chamberlain <mcgrof@kernel.org>,
	Eric Biederman <ebiederm@xmission.com>,
	Dave Olsthoorn <dave@bewaar.me>, Jessica Yu <jeyu@kernel.org>,
	Casey Schaufler <casey@schaufler-ca.com>,
	Joe Perches <joe@perches.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Thiago Jung Bauermann <bauerman@linux.ibm.com>
Subject: Re: [PATCH v2 02/18] selftest/firmware: Add selftest timeout in settings
Date: Thu, 23 Jul 2020 10:34:06 -0700	[thread overview]
Message-ID: <577f52ba-a514-558d-7041-e65ea9974917@broadcom.com> (raw)
In-Reply-To: <20200722193020.2676422-3-keescook@chromium.org>

works.

On 2020-07-22 12:30 p.m., Kees Cook wrote:
> The firmware tests would always time out for me. Add a correct timeout,
> including details on how the value was reached. Additionally allow the
> test harness to skip comments in settings files and report how long a
> given timeout was.
>
> Signed-off-by: Kees Cook <keescook@chromium.org>
Acked-by: Scott Branden <scott.branden@broadcom.com>
> ---
>   tools/testing/selftests/firmware/settings   | 8 ++++++++
>   tools/testing/selftests/kselftest/runner.sh | 6 +++++-
>   2 files changed, 13 insertions(+), 1 deletion(-)
>   create mode 100644 tools/testing/selftests/firmware/settings
>
> diff --git a/tools/testing/selftests/firmware/settings b/tools/testing/selftests/firmware/settings
> new file mode 100644
> index 000000000000..085e664ee093
> --- /dev/null
> +++ b/tools/testing/selftests/firmware/settings
> @@ -0,0 +1,8 @@
> +# The async firmware timeout is set to 1 second (but ends up being effectively
> +# 2 seconds). There are 3 test configs, each done with and without firmware
> +# present, each with 2 "nowait" functions tested 5 times. Expected time for a
> +# normal execution should be 2 * 3 * 2 * 2 * 5 = 120 seconds for those alone.
> +# Additionally, fw_fallback may take 5 seconds for internal timeouts in each
> +# of the 3 configs, so at least another 15 seconds are needed. Add another
> +# 10 seconds for each testing config: 120 + 15 + 30
> +timeout=165
> diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
> index 676b3a8b114d..cd5ddf979f15 100644
> --- a/tools/testing/selftests/kselftest/runner.sh
> +++ b/tools/testing/selftests/kselftest/runner.sh
> @@ -53,6 +53,10 @@ run_one()
>   	settings="$BASE_DIR/$DIR/settings"
>   	if [ -r "$settings" ] ; then
>   		while read line ; do
> +			# Skip comments.
> +			if echo "$line" | grep -q '^#'; then
> +				continue
> +			fi
>   			field=$(echo "$line" | cut -d= -f1)
>   			value=$(echo "$line" | cut -d= -f2-)
>   			eval "kselftest_$field"="$value"
> @@ -80,7 +84,7 @@ run_one()
>   			echo "not ok $test_num $TEST_HDR_MSG # SKIP"
>   		elif [ $rc -eq $timeout_rc ]; then \
>   			echo "#"
> -			echo "not ok $test_num $TEST_HDR_MSG # TIMEOUT"
> +			echo "not ok $test_num $TEST_HDR_MSG # TIMEOUT $kselftest_timeout seconds"
>   		else
>   			echo "not ok $test_num $TEST_HDR_MSG # exit=$rc"
>   		fi)

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec