Re: [PATCH v2 02/18] selftest/firmware: Add selftest timeout in settings

Scott Branden
2020-07-23T17:34:16Z

works.

On 2020-07-22 12:30 p.m., Kees Cook wrote:
> The firmware tests would always time out for me. Add a correct timeout,
> including details on how the value was reached. Additionally allow the
> test harness to skip comments in settings files and report how long a
> given timeout was.
>
> Signed-off-by: Kees Cook <keescook@chromium.org>
Acked-by: Scott Branden <scott.branden@broadcom.com>
> ---
> tools/testing/selftests/firmware/settings | 8 ++++++++
> tools/testing/selftests/kselftest/runner.sh | 6 +++++-
> 2 files changed, 13 insertions(+), 1 deletion(-)
> create mode 100644 tools/testing/selftests/firmware/settings
>
> diff --git a/tools/testing/selftests/firmware/settings b/tools/testing/selftests/firmware/settings
> new file mode 100644
> index 000000000000..085e664ee093
> --- /dev/null
> +++ b/tools/testing/selftests/firmware/settings
> @@ -0,0 +1,8 @@
> +# The async firmware timeout is set to 1 second (but ends up being effectively
> +# 2 seconds). There are 3 test configs, each done with and without firmware
> +# present, each with 2 "nowait" functions tested 5 times. Expected time for a
> +# normal execution should be 2 * 3 * 2 * 2 * 5 = 120 seconds for those alone.
> +# Additionally, fw_fallback may take 5 seconds for internal timeouts in each
> +# of the 3 configs, so at least another 15 seconds are needed. Add another
> +# 10 seconds for each testing config: 120 + 15 + 30
> +timeout=165
> diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
> index 676b3a8b114d..cd5ddf979f15 100644
> --- a/tools/testing/selftests/kselftest/runner.sh
> +++ b/tools/testing/selftests/kselftest/runner.sh
> @@ -53,6 +53,10 @@ run_one()
> settings="$BASE_DIR/$DIR/settings"
> if [ -r "$settings" ] ; then
> while read line ; do
> + # Skip comments.
> + if echo "$line" | grep -q '^#'; then
> + continue
> + fi
> field=$(echo "$line" | cut -d= -f1)
> value=$(echo "$line" | cut -d= -f2-)
> eval "kselftest_$field"="$value"
> @@ -80,7 +84,7 @@ run_one()
> echo "not ok $test_num $TEST_HDR_MSG # SKIP"
> elif [ $rc -eq $timeout_rc ]; then \
> echo "#"
> - echo "not ok $test_num $TEST_HDR_MSG # TIMEOUT"
> + echo "not ok $test_num $TEST_HDR_MSG # TIMEOUT $kselftest_timeout seconds"
> else
> echo "not ok $test_num $TEST_HDR_MSG # exit=$rc"
> fi)