[PATCH] automation: introduce TEST_TIMEOUT_OVERRIDE

[PATCH] automation: introduce TEST_TIMEOUT_OVERRIDE

[Stefano Stabellini]

TEST_TIME is set as a CI/CD project variable, as it should be, to match
the capability and speed of the testing infrastructure.

As it turns out, TEST_TIME defined in test.yaml cannot override
TEST_TIME defined as CI/CD project variable. As a consequence, today the
TEST_TIME setting in test.yaml for the Xilinx jobs is ignored.

Instead, rename TEST_TIMEOUT to TEST_TIMEOUT_OVERRIDE in test.yaml and
check for TEST_TIMEOUT_OVERRIDE first in console.exp.

Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 8675016b6a..e947736195 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -84,7 +84,7 @@
   variables:
     CONTAINER: ubuntu:xenial-xilinx
     LOGFILE: qemu-smoke-xilinx.log
-    TEST_TIMEOUT: 120
+    TEST_TIMEOUT_OVERRIDE: 120
   artifacts:
     paths:
       - smoke.serial
@@ -104,7 +104,7 @@
     LOGFILE: xilinx-smoke-x86_64.log
     XEN_CMD_CONSOLE: "console=com2 com2=57600,8n1,0x2F8,4"
     TEST_BOARD: "crater"
-    TEST_TIMEOUT: 1000
+    TEST_TIMEOUT_OVERRIDE: 1000
   artifacts:
     paths:
       - smoke.serial
diff --git a/automation/scripts/console.exp b/automation/scripts/console.exp
index f538aa6bd0..310543c33e 100755
--- a/automation/scripts/console.exp
+++ b/automation/scripts/console.exp
@@ -1,6 +1,8 @@
 #!/usr/bin/expect -f
 
-if {[info exists env(TEST_TIMEOUT)]} {
+if {[info exists env(TEST_TIMEOUT_OVERRIDE)]} {
+    set timeout $env(TEST_TIMEOUT_OVERRIDE)
+} elseif {[info exists env(TEST_TIMEOUT)]} {
     set timeout $env(TEST_TIMEOUT)
 } else {
     set timeout 1500

Re: [PATCH] automation: introduce TEST_TIMEOUT_OVERRIDE

[Marek Marczykowski-Górecki]

[-- Attachment #1: Type: text/plain, Size: 2053 bytes --]

On Thu, Oct 03, 2024 at 01:22:51PM -0700, Stefano Stabellini wrote:
> TEST_TIME is set as a CI/CD project variable, as it should be, to match
> the capability and speed of the testing infrastructure.
> 
> As it turns out, TEST_TIME defined in test.yaml cannot override
> TEST_TIME defined as CI/CD project variable. As a consequence, today the
> TEST_TIME setting in test.yaml for the Xilinx jobs is ignored.

s/TEST_TIME/\0OUT/

> 
> Instead, rename TEST_TIMEOUT to TEST_TIMEOUT_OVERRIDE in test.yaml and
> check for TEST_TIMEOUT_OVERRIDE first in console.exp.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>

with commit message fixed:
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 8675016b6a..e947736195 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -84,7 +84,7 @@
>    variables:
>      CONTAINER: ubuntu:xenial-xilinx
>      LOGFILE: qemu-smoke-xilinx.log
> -    TEST_TIMEOUT: 120
> +    TEST_TIMEOUT_OVERRIDE: 120
>    artifacts:
>      paths:
>        - smoke.serial
> @@ -104,7 +104,7 @@
>      LOGFILE: xilinx-smoke-x86_64.log
>      XEN_CMD_CONSOLE: "console=com2 com2=57600,8n1,0x2F8,4"
>      TEST_BOARD: "crater"
> -    TEST_TIMEOUT: 1000
> +    TEST_TIMEOUT_OVERRIDE: 1000
>    artifacts:
>      paths:
>        - smoke.serial
> diff --git a/automation/scripts/console.exp b/automation/scripts/console.exp
> index f538aa6bd0..310543c33e 100755
> --- a/automation/scripts/console.exp
> +++ b/automation/scripts/console.exp
> @@ -1,6 +1,8 @@
>  #!/usr/bin/expect -f
>  
> -if {[info exists env(TEST_TIMEOUT)]} {
> +if {[info exists env(TEST_TIMEOUT_OVERRIDE)]} {
> +    set timeout $env(TEST_TIMEOUT_OVERRIDE)
> +} elseif {[info exists env(TEST_TIMEOUT)]} {
>      set timeout $env(TEST_TIMEOUT)
>  } else {
>      set timeout 1500

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]