[PATCH 1/1] tests/avocado: update firmware to enable sbsa-ref/max

[PATCH 1/1] tests/avocado: update firmware to enable sbsa-ref/max

[Marcin Juszkiewicz]

Update prebuilt firmware images to have TF-A with FEAT_FGT support
enabled. This allowed us to enable test for "max" cpu in sbsa-ref
machine.

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
---
 tests/avocado/machine_aarch64_sbsaref.py | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py
index 0a79fa7ab6..35f8042416 100644
--- a/tests/avocado/machine_aarch64_sbsaref.py
+++ b/tests/avocado/machine_aarch64_sbsaref.py
@@ -29,23 +29,23 @@ def fetch_firmware(self):
         """
         Flash volumes generated using:
 
-        - Fedora GNU Toolchain version 12.2.1 20220819 (Red Hat Cross 12.2.1-2)
+        - Fedora GNU Toolchain version 13.1.1 20230511 (Red Hat 13.1.1-2)
 
         - Trusted Firmware-A
-          https://github.com/ARM-software/arm-trusted-firmware/tree/5fdb2e54
+          https://github.com/ARM-software/arm-trusted-firmware/tree/c0d8ee38
 
         - Tianocore EDK II
-          https://github.com/tianocore/edk2/tree/494127613b
-          https://github.com/tianocore/edk2-non-osi/tree/41876073
-          https://github.com/tianocore/edk2-platforms/tree/8efa4f42
+          https://github.com/tianocore/edk2/tree/0f9283429dd4
+          https://github.com/tianocore/edk2-non-osi/tree/f0bb00937ad6
+          https://github.com/tianocore/edk2-platforms/tree/7880b92e2a04
         """
 
         # Secure BootRom (TF-A code)
         fs0_xz_url = (
-            "https://fileserver.linaro.org/s/ATnSmq6k8SoXgbH/"
+            "https://fileserver.linaro.org/s/HrYMCjP7MEccjRP/"
             "download/SBSA_FLASH0.fd.xz"
         )
-        fs0_xz_hash = "a210a09692bcbe0a3743ffd0df44e80e0c7ad8ab"
+        fs0_xz_hash = "447eff64a90b84ce47703c6ec41fbfc25befaaea"
         tar_xz_path = self.fetch_asset(fs0_xz_url, asset_hash=fs0_xz_hash)
         archive.extract(tar_xz_path, self.workdir)
         fs0_path = os.path.join(self.workdir, "SBSA_FLASH0.fd")
@@ -93,15 +93,15 @@ def test_sbsaref_edk2_firmware(self):
 
         # AP Trusted ROM
         wait_for_console_pattern(self, "Booting Trusted Firmware")
-        wait_for_console_pattern(self, "BL1: v2.8(release):v2.8")
+        wait_for_console_pattern(self, "BL1: v2.9(release):v2.9")
         wait_for_console_pattern(self, "BL1: Booting BL2")
 
         # Trusted Boot Firmware
-        wait_for_console_pattern(self, "BL2: v2.8(release)")
+        wait_for_console_pattern(self, "BL2: v2.9(release)")
         wait_for_console_pattern(self, "Booting BL31")
 
         # EL3 Runtime Software
-        wait_for_console_pattern(self, "BL31: v2.8(release)")
+        wait_for_console_pattern(self, "BL31: v2.9(release)")
 
         # Non-trusted Firmware
         wait_for_console_pattern(self, "UEFI firmware (version 1.0")
@@ -150,7 +150,7 @@ def test_sbsaref_alpine_linux_neoverse_n1(self):
         """
         self.boot_alpine_linux("neoverse-n1")
 
-    @skip("requires TF-A update to handle FEAT_FGT")
+    @skipUnless(os.getenv("AVOCADO_TIMEOUT_EXPECTED"), "Test might timeout")
     def test_sbsaref_alpine_linux_max(self):
         """
         :avocado: tags=cpu:max
-- 
2.40.1

Re: [PATCH 1/1] tests/avocado: update firmware to enable sbsa-ref/max

[Alex Bennée]

Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> writes:

> Update prebuilt firmware images to have TF-A with FEAT_FGT support
> enabled. This allowed us to enable test for "max" cpu in sbsa-ref
> machine.
>
> Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>

Queued to testing/next, thanks.
-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH 1/1] tests/avocado: update firmware to enable sbsa-ref/max

[Philippe Mathieu-Daudé]

On 30/5/23 17:22, Marcin Juszkiewicz wrote:
> Update prebuilt firmware images to have TF-A with FEAT_FGT support
> enabled. This allowed us to enable test for "max" cpu in sbsa-ref
> machine.
> 
> Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
> ---
>   tests/avocado/machine_aarch64_sbsaref.py | 22 +++++++++++-----------
>   1 file changed, 11 insertions(+), 11 deletions(-)
> 
> diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py
> index 0a79fa7ab6..35f8042416 100644
> --- a/tests/avocado/machine_aarch64_sbsaref.py
> +++ b/tests/avocado/machine_aarch64_sbsaref.py
> @@ -29,23 +29,23 @@ def fetch_firmware(self):
>           """
>           Flash volumes generated using:
>   
> -        - Fedora GNU Toolchain version 12.2.1 20220819 (Red Hat Cross 12.2.1-2)
> +        - Fedora GNU Toolchain version 13.1.1 20230511 (Red Hat 13.1.1-2)
>   
>           - Trusted Firmware-A
> -          https://github.com/ARM-software/arm-trusted-firmware/tree/5fdb2e54
> +          https://github.com/ARM-software/arm-trusted-firmware/tree/c0d8ee38
>   
>           - Tianocore EDK II
> -          https://github.com/tianocore/edk2/tree/494127613b
> -          https://github.com/tianocore/edk2-non-osi/tree/41876073
> -          https://github.com/tianocore/edk2-platforms/tree/8efa4f42
> +          https://github.com/tianocore/edk2/tree/0f9283429dd4
> +          https://github.com/tianocore/edk2-non-osi/tree/f0bb00937ad6
> +          https://github.com/tianocore/edk2-platforms/tree/7880b92e2a04

Thanks for updating this comment!

> -    @skip("requires TF-A update to handle FEAT_FGT")
> +    @skipUnless(os.getenv("AVOCADO_TIMEOUT_EXPECTED"), "Test might timeout")

Can it still timeout?

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [PATCH 1/1] tests/avocado: update firmware to enable sbsa-ref/max

[Marcin Juszkiewicz]

W dniu 7.06.2023 o 16:33, Philippe Mathieu-Daudé pisze:
> On 30/5/23 17:22, Marcin Juszkiewicz wrote:
>> Update prebuilt firmware images to have TF-A with FEAT_FGT support
>> enabled. This allowed us to enable test for "max" cpu in sbsa-ref
>> machine.
>>
>> Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
>> ---
>>   tests/avocado/machine_aarch64_sbsaref.py | 22 +++++++++++-----------
>>   1 file changed, 11 insertions(+), 11 deletions(-)
>>
>> diff --git a/tests/avocado/machine_aarch64_sbsaref.py 
>> b/tests/avocado/machine_aarch64_sbsaref.py
>> index 0a79fa7ab6..35f8042416 100644
>> --- a/tests/avocado/machine_aarch64_sbsaref.py
>> +++ b/tests/avocado/machine_aarch64_sbsaref.py
>> @@ -29,23 +29,23 @@ def fetch_firmware(self):
>>           """
>>           Flash volumes generated using:
>> -        - Fedora GNU Toolchain version 12.2.1 20220819 (Red Hat Cross 
>> 12.2.1-2)
>> +        - Fedora GNU Toolchain version 13.1.1 20230511 (Red Hat 
>> 13.1.1-2)
>>           - Trusted Firmware-A
>> -          
>> https://github.com/ARM-software/arm-trusted-firmware/tree/5fdb2e54
>> +          
>> https://github.com/ARM-software/arm-trusted-firmware/tree/c0d8ee38
>>           - Tianocore EDK II
>> -          https://github.com/tianocore/edk2/tree/494127613b
>> -          https://github.com/tianocore/edk2-non-osi/tree/41876073
>> -          https://github.com/tianocore/edk2-platforms/tree/8efa4f42
>> +          https://github.com/tianocore/edk2/tree/0f9283429dd4
>> +          https://github.com/tianocore/edk2-non-osi/tree/f0bb00937ad6
>> +          https://github.com/tianocore/edk2-platforms/tree/7880b92e2a04
> 
> Thanks for updating this comment!

Having a way to reproduce is crucial for CI.

>> -    @skip("requires TF-A update to handle FEAT_FGT")
>> +    @skipUnless(os.getenv("AVOCADO_TIMEOUT_EXPECTED"), "Test might 
>> timeout")
> 
> Can it still timeout?

All Linux based tests in this file have that @skipUnless as they take 
some time:

test_sbsaref_edk2_firmware: PASS (2.72 s)
test_sbsaref_alpine_linux_cortex_a57: PASS (23.71 s)
test_sbsaref_alpine_linux_neoverse_n1: PASS (23.53 s)
test_sbsaref_alpine_linux_max: PASS (28.16 s)

Re: [PATCH 1/1] tests/avocado: update firmware to enable sbsa-ref/max

[Philippe Mathieu-Daudé]

On 7/6/23 17:29, Marcin Juszkiewicz wrote:
> W dniu 7.06.2023 o 16:33, Philippe Mathieu-Daudé pisze:
>> On 30/5/23 17:22, Marcin Juszkiewicz wrote:
>>> Update prebuilt firmware images to have TF-A with FEAT_FGT support
>>> enabled. This allowed us to enable test for "max" cpu in sbsa-ref
>>> machine.
>>>
>>> Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
>>> ---
>>>   tests/avocado/machine_aarch64_sbsaref.py | 22 +++++++++++-----------
>>>   1 file changed, 11 insertions(+), 11 deletions(-)
>>>
>>> diff --git a/tests/avocado/machine_aarch64_sbsaref.py 
>>> b/tests/avocado/machine_aarch64_sbsaref.py
>>> index 0a79fa7ab6..35f8042416 100644
>>> --- a/tests/avocado/machine_aarch64_sbsaref.py
>>> +++ b/tests/avocado/machine_aarch64_sbsaref.py
>>> @@ -29,23 +29,23 @@ def fetch_firmware(self):
>>>           """
>>>           Flash volumes generated using:
>>> -        - Fedora GNU Toolchain version 12.2.1 20220819 (Red Hat 
>>> Cross 12.2.1-2)
>>> +        - Fedora GNU Toolchain version 13.1.1 20230511 (Red Hat 
>>> 13.1.1-2)
>>>           - Trusted Firmware-A
>>> - https://github.com/ARM-software/arm-trusted-firmware/tree/5fdb2e54
>>> + https://github.com/ARM-software/arm-trusted-firmware/tree/c0d8ee38
>>>           - Tianocore EDK II
>>> -          https://github.com/tianocore/edk2/tree/494127613b
>>> -          https://github.com/tianocore/edk2-non-osi/tree/41876073
>>> -          https://github.com/tianocore/edk2-platforms/tree/8efa4f42
>>> +          https://github.com/tianocore/edk2/tree/0f9283429dd4
>>> +          https://github.com/tianocore/edk2-non-osi/tree/f0bb00937ad6
>>> +          https://github.com/tianocore/edk2-platforms/tree/7880b92e2a04
>>
>> Thanks for updating this comment!
> 
> Having a way to reproduce is crucial for CI.
> 
>>> -    @skip("requires TF-A update to handle FEAT_FGT")
>>> +    @skipUnless(os.getenv("AVOCADO_TIMEOUT_EXPECTED"), "Test might 
>>> timeout")
>>
>> Can it still timeout?
> 
> All Linux based tests in this file have that @skipUnless as they take 
> some time:
> 
> test_sbsaref_edk2_firmware: PASS (2.72 s)
> test_sbsaref_alpine_linux_cortex_a57: PASS (23.71 s)
> test_sbsaref_alpine_linux_neoverse_n1: PASS (23.53 s)
> test_sbsaref_alpine_linux_max: PASS (28.16 s)

I suppose this was due to a bug we had with Avocado consuming QEMU's
console. I don't remember recent complains. Alex, do you know if this
was fixed?
We define the class timeout to 180s, so all tests inherit it. In your
run all tests take <30sec, it should be fine to run them on CI.
Adding ~1min30s extra on the job running these Avocado tests seem
reasonable to me, but I have Cc'ed Thomas who took care to reduce
testing time.

Regards,

Phil.