[PATCH 1/3] tests/avocado: update AArch64 tests to Alpine 3.17.2

[PATCH 1/3] tests/avocado: update AArch64 tests to Alpine 3.17.2

[Marcin Juszkiewicz]

To test Alpine boot on SBSA-Ref target we need Alpine Linux
'standard' image as 'virt' one lacks kernel modules.

So to minimalize Avocado cache I move test to 'standard' image.

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
---
 tests/avocado/machine_aarch64_virt.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tests/avocado/machine_aarch64_virt.py b/tests/avocado/machine_aarch64_virt.py
index 25dab8dc00..a90dc6ff4b 100644
--- a/tests/avocado/machine_aarch64_virt.py
+++ b/tests/avocado/machine_aarch64_virt.py
@@ -38,11 +38,11 @@ def test_alpine_virt_tcg_gic_max(self):
         :avocado: tags=accel:tcg
         """
         iso_url = ('https://dl-cdn.alpinelinux.org/'
-                   'alpine/v3.16/releases/aarch64/'
-                   'alpine-virt-3.16.3-aarch64.iso')
+                   'alpine/v3.17/releases/aarch64/'
+                   'alpine-standard-3.17.2-aarch64.iso')
 
         # Alpine use sha256 so I recalculated this myself
-        iso_sha1 = '0683bc089486d55c91bf6607d5ecb93925769bc0'
+        iso_sha1 = '76284fcd7b41fe899b0c2375ceb8470803eea839'
         iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha1)
 
         self.vm.set_console()
@@ -65,7 +65,7 @@ def test_alpine_virt_tcg_gic_max(self):
         self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom')
 
         self.vm.launch()
-        self.wait_for_console_pattern('Welcome to Alpine Linux 3.16')
+        self.wait_for_console_pattern('Welcome to Alpine Linux 3.17')
 
 
     def common_aarch64_virt(self, machine):
-- 
2.39.2

[PATCH 2/3] tests/avocado: Add set of boot tests on SBSA-ref

[Marcin Juszkiewicz]

From: Philippe Mathieu-Daudé <philmd@linaro.org>

This change adds set of boot tests on SBSA-ref machine:

1. boot TF-A up to the EDK2 banner
2. boot Linux kernel/initrd up to Busybox shell

Prebuilt flash volumes are included, built using upstream documentation.
EDK2 and TF-A are updated to have SVE/SME properly enabled for 'max'
cpu.

Linux kernel/initrd are from Debian 'bookworm' installer with
"init=/bin/sh" kernel argument.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
---
 MAINTAINERS                              |   1 +
 tests/avocado/machine_aarch64_sbsaref.py | 150 +++++++++++++++++++++++
 2 files changed, 151 insertions(+)
 create mode 100644 tests/avocado/machine_aarch64_sbsaref.py

diff --git a/MAINTAINERS b/MAINTAINERS
index 76662969d7..df0dfd4bc3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -916,6 +916,7 @@ L: qemu-arm@nongnu.org
 S: Maintained
 F: hw/arm/sbsa-ref.c
 F: docs/system/arm/sbsa.rst
+F: tests/avocado/machine_aarch64_sbsaref.py
 
 Sharp SL-5500 (Collie) PDA
 M: Peter Maydell <peter.maydell@linaro.org>
diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py
new file mode 100644
index 0000000000..e36e53d4d5
--- /dev/null
+++ b/tests/avocado/machine_aarch64_sbsaref.py
@@ -0,0 +1,150 @@
+# Functional test that boots a Linux kernel and checks the console
+#
+# SPDX-FileCopyrightText: 2023 Linaro Ltd.
+# SPDX-FileContributor: Philippe Mathieu-Daudé <philmd@linaro.org>
+# SPDX-FileContributor: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+import shutil
+
+from avocado.utils import archive
+
+from avocado_qemu import QemuSystemTest
+from avocado_qemu import wait_for_console_pattern
+from avocado_qemu import interrupt_interactive_console_until_pattern
+
+
+class Aarch64SbsarefMachine(QemuSystemTest):
+    """
+    :avocado: tags=arch:aarch64
+    :avocado: tags=machine:sbsa-ref
+    """
+
+    def fetch_firmware(self):
+        """
+        Flash volumes generated using:
+
+        - Fedora GNU Toolchain version 12.2.1 20220819 (Red Hat Cross 12.2.1-2)
+
+        - Trusted Firmware-A
+          https://github.com/ARM-software/arm-trusted-firmware/tree/6264643a
+
+        - Tianocore EDK II
+          https://github.com/tianocore/edk2/tree/f6ce1a5c
+          https://github.com/tianocore/edk2-non-osi/tree/74d4da60
+          https://github.com/tianocore/edk2-platforms/tree/0540e1a2
+        """
+
+        # Secure BootRom (TF-A code)
+        fs0_xz_url = ('https://fileserver.linaro.org/s/sZay4ZCCfHSXPKj/'
+                      'download/SBSA_FLASH0.fd.xz')
+        fs0_xz_hash = 'e74778cbb8e1aa0b77f8883565b9a18db638f6bb'
+        tar_xz_path = self.fetch_asset(fs0_xz_url, asset_hash=fs0_xz_hash)
+        archive.extract(tar_xz_path, self.workdir)
+        fs0_path = os.path.join(self.workdir, 'SBSA_FLASH0.fd')
+
+        # Non-secure rom (UEFI and EFI variables)
+        fs1_xz_url = ('https://fileserver.linaro.org/s/osHNaypByLa9xDK/'
+                      'download/SBSA_FLASH1.fd.xz')
+        fs1_xz_hash = '7d9f1a6b8964b8b99144f7e905a4083f31e31ad3'
+        tar_xz_path = self.fetch_asset(fs1_xz_url, asset_hash=fs1_xz_hash)
+        archive.extract(tar_xz_path, self.workdir)
+        fs1_path = os.path.join(self.workdir, 'SBSA_FLASH1.fd')
+
+        for path in [fs0_path, fs1_path]:
+            with open(path, 'ab+') as fd:
+                fd.truncate(256 << 20)  # Expand volumes to 256MiB
+
+        self.vm.set_console()
+        self.vm.add_args('-drive', f'if=pflash,file={fs0_path},format=raw',
+                         '-drive', f'if=pflash,file={fs1_path},format=raw')
+
+    def test_sbsaref_tfa_v2_8(self):
+        """
+        :avocado: tags=cpu:cortex-a57
+        """
+
+        self.fetch_firmware()
+        self.vm.launch()
+
+        # TF-A boot sequence:
+        #
+        # https://github.com/ARM-software/arm-trusted-firmware/blob/v2.8.0/\
+        #     docs/design/trusted-board-boot.rst#trusted-board-boot-sequence
+        # https://trustedfirmware-a.readthedocs.io/en/v2.8/\
+        #     design/firmware-design.html#cold-boot
+
+        # AP Trusted ROM
+        wait_for_console_pattern(self, 'Booting Trusted Firmware')
+        wait_for_console_pattern(self, 'BL1: v2.8(release):v2.8')
+        wait_for_console_pattern(self, 'BL1: Booting BL2')
+
+        # Trusted Boot Firmware
+        wait_for_console_pattern(self, 'BL2: v2.8(release)')
+        wait_for_console_pattern(self, 'Booting BL31')
+
+        # EL3 Runtime Software
+        wait_for_console_pattern(self, 'BL31: v2.8(release)')
+
+        # Non-trusted Firmware
+        wait_for_console_pattern(self, 'UEFI firmware (version 1.0')
+        interrupt_interactive_console_until_pattern(self,
+                                                    'QEMU SBSA-REF Machine')
+
+    def boot_linux(self, cpu='cortex-a57'):
+        """
+        :avocado: tags=cpu:cortex-a57
+        """
+        self.fetch_firmware()
+
+        os.makedirs(f'{self.workdir}/vfat/efi/boot')
+
+        # UEFI shell binary
+        shell_url = ('https://fileserver.linaro.org/s/SGoyRrEzkmW8C8Y/'
+                     'download/bootaa64.efi')
+        shell_sha1 = '5a8791eb130406d1a659e538b1a194a604a29a78'
+        shell_path = self.fetch_asset(shell_url, shell_sha1)
+        shutil.copyfile(shell_path,
+                        f'{self.workdir}/vfat/efi/boot/bootaa64.efi')
+
+        # Debian 'bookworm' d-i kernel from 8th Feb 2023
+        linux_url = ('https://fileserver.linaro.org/s/L8JMwEZQK8SDR39/'
+                     'download/linux')
+        linux_sha1 = '39a75284783ab63626642228fbac1863492d30b5'
+        linux_path = self.fetch_asset(linux_url, linux_sha1)
+        shutil.copyfile(linux_path, f'{self.workdir}/vfat/linux')
+
+        # Debian 'bookworm' d-i initrd.gz from 8th Feb 2023
+        initrd_url = ('https://fileserver.linaro.org/s/NmYTxezZNKGF5P4/'
+                      'download/initrd.gz')
+        initrd_sha1 = '1404d0129cbd0bff7aaa589ddbea3cdb7c0d4c1d'
+        initrd_path = self.fetch_asset(initrd_url, initrd_sha1)
+        shutil.copyfile(initrd_path, f'{self.workdir}/vfat/initrd.gz')
+
+        with open(f'{self.workdir}/vfat/startup.nsh', 'w') as script:
+            script.write('fs0:\\linux initrd=\\initrd.gz init=/bin/sh')
+
+        self.vm.add_args('-cpu', cpu,
+                         '-drive',
+                         f'file=fat:rw:{self.workdir}/vfat/,format=raw')
+        self.vm.launch()
+
+        # Exit UEFI
+        wait_for_console_pattern(self, 'EFI stub: Exiting boot services...')
+
+        # init=/bin/sh started
+        wait_for_console_pattern(self, 'BusyBox v1.35.0 (Debian')
+
+    def test_sbsaref_linux_a57(self):
+        """
+        :avocado: tags=cpu:cortex-a57
+        """
+        self.boot_linux('cortex-a57')
+
+    def test_sbsaref_linux_max(self):
+        """
+        :avocado: tags=cpu:max
+        """
+        self.boot_linux('max')
-- 
2.39.2

Re: [PATCH 2/3] tests/avocado: Add set of boot tests on SBSA-ref

[Alex Bennée]

Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> writes:

> From: Philippe Mathieu-Daudé <philmd@linaro.org>
>
> This change adds set of boot tests on SBSA-ref machine:
>
> 1. boot TF-A up to the EDK2 banner
> 2. boot Linux kernel/initrd up to Busybox shell
>
> Prebuilt flash volumes are included, built using upstream documentation.
> EDK2 and TF-A are updated to have SVE/SME properly enabled for 'max'
> cpu.
>
> Linux kernel/initrd are from Debian 'bookworm' installer with
> "init=/bin/sh" kernel argument.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
> ---
>  MAINTAINERS                              |   1 +
>  tests/avocado/machine_aarch64_sbsaref.py | 150 +++++++++++++++++++++++
>  2 files changed, 151 insertions(+)
>  create mode 100644 tests/avocado/machine_aarch64_sbsaref.py
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 76662969d7..df0dfd4bc3 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -916,6 +916,7 @@ L: qemu-arm@nongnu.org
>  S: Maintained
>  F: hw/arm/sbsa-ref.c
>  F: docs/system/arm/sbsa.rst
> +F: tests/avocado/machine_aarch64_sbsaref.py
>  
>  Sharp SL-5500 (Collie) PDA
>  M: Peter Maydell <peter.maydell@linaro.org>
> diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py
> new file mode 100644
> index 0000000000..e36e53d4d5
> --- /dev/null
> +++ b/tests/avocado/machine_aarch64_sbsaref.py
> @@ -0,0 +1,150 @@
> +# Functional test that boots a Linux kernel and checks the console
> +#
> +# SPDX-FileCopyrightText: 2023 Linaro Ltd.
> +# SPDX-FileContributor: Philippe Mathieu-Daudé <philmd@linaro.org>
> +# SPDX-FileContributor: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
> +#
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +import os
> +import shutil
> +
> +from avocado.utils import archive
> +
> +from avocado_qemu import QemuSystemTest
> +from avocado_qemu import wait_for_console_pattern
> +from avocado_qemu import interrupt_interactive_console_until_pattern
> +
> +
> +class Aarch64SbsarefMachine(QemuSystemTest):
> +    """
> +    :avocado: tags=arch:aarch64
> +    :avocado: tags=machine:sbsa-ref
> +    """
> +
> +    def fetch_firmware(self):
> +        """
> +        Flash volumes generated using:
> +
> +        - Fedora GNU Toolchain version 12.2.1 20220819 (Red Hat Cross 12.2.1-2)
> +
> +        - Trusted Firmware-A
> +          https://github.com/ARM-software/arm-trusted-firmware/tree/6264643a
> +
> +        - Tianocore EDK II
> +          https://github.com/tianocore/edk2/tree/f6ce1a5c
> +          https://github.com/tianocore/edk2-non-osi/tree/74d4da60
> +          https://github.com/tianocore/edk2-platforms/tree/0540e1a2
> +        """
> +
> +        # Secure BootRom (TF-A code)
> +        fs0_xz_url = ('https://fileserver.linaro.org/s/sZay4ZCCfHSXPKj/'
> +                      'download/SBSA_FLASH0.fd.xz')
> +        fs0_xz_hash = 'e74778cbb8e1aa0b77f8883565b9a18db638f6bb'
> +        tar_xz_path = self.fetch_asset(fs0_xz_url, asset_hash=fs0_xz_hash)
> +        archive.extract(tar_xz_path, self.workdir)
> +        fs0_path = os.path.join(self.workdir, 'SBSA_FLASH0.fd')
> +
> +        # Non-secure rom (UEFI and EFI variables)
> +        fs1_xz_url = ('https://fileserver.linaro.org/s/osHNaypByLa9xDK/'
> +                      'download/SBSA_FLASH1.fd.xz')
> +        fs1_xz_hash = '7d9f1a6b8964b8b99144f7e905a4083f31e31ad3'
> +        tar_xz_path = self.fetch_asset(fs1_xz_url, asset_hash=fs1_xz_hash)
> +        archive.extract(tar_xz_path, self.workdir)
> +        fs1_path = os.path.join(self.workdir, 'SBSA_FLASH1.fd')
> +
> +        for path in [fs0_path, fs1_path]:
> +            with open(path, 'ab+') as fd:
> +                fd.truncate(256 << 20)  # Expand volumes to 256MiB
> +
> +        self.vm.set_console()
> +        self.vm.add_args('-drive', f'if=pflash,file={fs0_path},format=raw',
> +                         '-drive', f'if=pflash,file={fs1_path},format=raw')
> +
> +    def test_sbsaref_tfa_v2_8(self):
> +        """
> +        :avocado: tags=cpu:cortex-a57
> +        """
> +
> +        self.fetch_firmware()
> +        self.vm.launch()
> +
> +        # TF-A boot sequence:
> +        #
> +        # https://github.com/ARM-software/arm-trusted-firmware/blob/v2.8.0/\
> +        #     docs/design/trusted-board-boot.rst#trusted-board-boot-sequence
> +        # https://trustedfirmware-a.readthedocs.io/en/v2.8/\
> +        #     design/firmware-design.html#cold-boot
> +
> +        # AP Trusted ROM
> +        wait_for_console_pattern(self, 'Booting Trusted Firmware')
> +        wait_for_console_pattern(self, 'BL1: v2.8(release):v2.8')
> +        wait_for_console_pattern(self, 'BL1: Booting BL2')
> +
> +        # Trusted Boot Firmware
> +        wait_for_console_pattern(self, 'BL2: v2.8(release)')
> +        wait_for_console_pattern(self, 'Booting BL31')
> +
> +        # EL3 Runtime Software
> +        wait_for_console_pattern(self, 'BL31: v2.8(release)')
> +
> +        # Non-trusted Firmware
> +        wait_for_console_pattern(self, 'UEFI firmware (version 1.0')
> +        interrupt_interactive_console_until_pattern(self,
> +                                                    'QEMU SBSA-REF Machine')
> +
> +    def boot_linux(self, cpu='cortex-a57'):
> +        """
> +        :avocado: tags=cpu:cortex-a57
> +        """
> +        self.fetch_firmware()
> +
> +        os.makedirs(f'{self.workdir}/vfat/efi/boot')
> +
> +        # UEFI shell binary
> +        shell_url = ('https://fileserver.linaro.org/s/SGoyRrEzkmW8C8Y/'
> +                     'download/bootaa64.efi')
> +        shell_sha1 = '5a8791eb130406d1a659e538b1a194a604a29a78'
> +        shell_path = self.fetch_asset(shell_url, shell_sha1)
> +        shutil.copyfile(shell_path,
> +                        f'{self.workdir}/vfat/efi/boot/bootaa64.efi')
> +
> +        # Debian 'bookworm' d-i kernel from 8th Feb 2023
> +        linux_url = ('https://fileserver.linaro.org/s/L8JMwEZQK8SDR39/'
> +                     'download/linux')
> +        linux_sha1 = '39a75284783ab63626642228fbac1863492d30b5'
> +        linux_path = self.fetch_asset(linux_url, linux_sha1)
> +        shutil.copyfile(linux_path, f'{self.workdir}/vfat/linux')
> +
> +        # Debian 'bookworm' d-i initrd.gz from 8th Feb 2023
> +        initrd_url = ('https://fileserver.linaro.org/s/NmYTxezZNKGF5P4/'
> +                      'download/initrd.gz')
> +        initrd_sha1 = '1404d0129cbd0bff7aaa589ddbea3cdb7c0d4c1d'
> +        initrd_path = self.fetch_asset(initrd_url, initrd_sha1)
> +        shutil.copyfile(initrd_path, f'{self.workdir}/vfat/initrd.gz')
> +
> +        with open(f'{self.workdir}/vfat/startup.nsh', 'w') as script:
> +            script.write('fs0:\\linux initrd=\\initrd.gz init=/bin/sh')
> +
> +        self.vm.add_args('-cpu', cpu,
> +                         '-drive',
> +                         f'file=fat:rw:{self.workdir}/vfat/,format=raw')
> +        self.vm.launch()
> +
> +        # Exit UEFI
> +        wait_for_console_pattern(self, 'EFI stub: Exiting boot services...')
> +
> +        # init=/bin/sh started
> +        wait_for_console_pattern(self, 'BusyBox v1.35.0 (Debian')
> +
> +    def test_sbsaref_linux_a57(self):
> +        """
> +        :avocado: tags=cpu:cortex-a57
> +        """
> +        self.boot_linux('cortex-a57')
> +
> +    def test_sbsaref_linux_max(self):
> +        """
> +        :avocado: tags=cpu:max
> +        """
> +        self.boot_linux('max')

You probably want to encode the pauth-impef here so we don't spend all
our time on the cryptography. e.g.

     self.boot_linux('max,pauth-impdef=on')

but even with that it seems to be timing out for me:

  2023-03-03 11:32:30,573 __init__         L0153 DEBUG| NOTICE:  BL1: Booting BL31
  2023-03-03 11:32:30,578 __init__         L0153 DEBUG| NOTICE:  BL31: v2.8(release):v2.8-335-g6264643a0
  2023-03-03 11:32:30,578 __init__         L0153 DEBUG| NOTICE:  BL31: Built : 17:51:50, Feb  8 2023
  2023-03-03 11:32:30,583 __init__         L0153 DEBUG| UEFI firmware (version 1.0 built at 16:17:37 on Feb  9 2023)
  2023-03-03 11:32:34,698 __init__         L0153 DEBUG| ESC[2JESC[04DESC[=3hESC[2JESC[09DESC[2JESC[04DUEFI Interactive Shell v2.2
  2023-03-03 11:32:34,699 __init__         L0153 DEBUG| EDK II
  2023-03-03 11:32:34,702 __init__         L0153 DEBUG| UEFI v2.70 (EFI Development Kit II / SbsaQemu, 0x00010000)
  2023-03-03 11:32:34,708 __init__         L0153 DEBUG| ESC[1mESC[33mESC[40mMapping tableESC[0mESC[37mESC[40m
  2023-03-03 11:32:34,716 __init__         L0153 DEBUG| ESC[1mESC[33mESC[40m      FS0:ESC[0mESC[37mESC[40m ESC[1mESC[37mESC[40mAlias(s):ESC[0mESC[37mESC[40mHD0a65535a1:;BLK1:
  2023-03-03 11:32:34,716 __init__         L0153 DEBUG| VenHw(0D51905B-B77E-452A-A2C0-ECA0CC8D514A,000010600000000000)/Sata(0x
  2023-03-03 11:32:34,716 __init__         L0153 DEBUG| 0,0xFFFF,0x0)/HD(1,MBR,0xBE1AFDFA,0x3F,0xFBFC1)
  2023-03-03 11:32:34,722 __init__         L0153 DEBUG| ESC[1mESC[33mESC[40m     BLK0:ESC[0mESC[37mESC[40m ESC[1mESC[37mESC[40mAlias(s):ESC[0mESC[37mESC[40m
  2023-03-03 11:32:34,722 __init__         L0153 DEBUG| VenHw(0D51905B-B77E-452A-A2C0-ECA0CC8D514A,000010600000000000)/Sata(0x
  2023-03-03 11:32:34,722 __init__         L0153 DEBUG| 0,0xFFFF,0x0)
  2023-03-03 11:32:39,749 __init__         L0153 DEBUG| Press ESC[1mESC[37mESC[40mESCESC[0mESC[37mESC[40m in 5 seconds to skip ESC[1mESC[33mESC[40mstartup.nshESC[0mESC[37mESC[40m or any other key to continue.ESC[72DPress ESC[1mESC[37mESC[40mESCESC[0mESC[37mESC[40m in 4 seconds to skip ESC[1mESC[33mESC[40mstartup.nshESC[0mESC[37mESC[40m or any other key to continue.ESC[72DPress ESC[1mESC[37mESC[40mESCESC[0mESC[37mESC[40m in 3 seconds to skip ESC[1mESC[33mESC[40mstartup.nshESC[0mESC[37mESC[40m or any other key to continue.ESC[72DPress ESC[1mESC[37mESC[40mESCESC[0mESC[37mESC[40m in 2 seconds to skip ESC[1mESC[33mESC[40mstartup.nshESC[0mESC[37mESC[40m or any other key to continue.ESC[72DPress ESC[1mESC[37mESC[40mESCESC[0mESC[37mESC[40m in 1 seconds to skip ESC[1mESC[33mESC[40mstartup.nshESC[0mESC[37mESC[40m or any other key to continue.
  2023-03-03 11:32:39,766 __init__         L0153 DEBUG| ESC[1mESC[33mESC[40mShell> ESC[0mESC[37mESC[40mfs0:\linux initrd=\initrd.gz init=/bin/sh
  2023-03-03 11:32:40,432 __init__         L0153 DEBUG| EFI stub: Booting Linux Kernel...
  2023-03-03 11:32:40,434 __init__         L0153 DEBUG| EFI stub: EFI_RNG_PROTOCOL unavailable
  2023-03-03 11:32:41,051 __init__         L0153 DEBUG| EFI stub: Loaded initrd from command line option
  2023-03-03 11:32:41,053 __init__         L0153 DEBUG| EFI stub: Generating empty DTB
  2023-03-03 11:32:41,054 __init__         L0153 DEBUG| EFI stub: Exiting boot services...
  2023-03-03 11:34:27,869 stacktrace       L0039 ERROR| 
  2023-03-03 11:34:27,870 stacktrace       L0041 ERROR| Reproduced traceback from: /home/alex/lsrc/qemu.git/builds/arm.all/tests/venv/lib/python3.9/site-packages/avocado/core/test.py:770
  2023-03-03 11:34:27,871 stacktrace       L0045 ERROR| Traceback (most recent call last):
  2023-03-03 11:34:27,871 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/arm.all/tests/avocado/machine_aarch64_sbsaref.py", line 150, in test_sbsaref_linux_max
  2023-03-03 11:34:27,871 stacktrace       L0045 ERROR|     self.boot_linux('max,pauth-impdef=on')

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH 2/3] tests/avocado: Add set of boot tests on SBSA-ref

[Marcin Juszkiewicz]

W dniu 3.03.2023 o 12:27, Alex Bennée pisze:
>> +    def test_sbsaref_linux_max(self):
>> +        """
>> +        :avocado: tags=cpu:max
>> +        """
>> +        self.boot_linux('max')
> You probably want to encode the pauth-impef here so we don't spend all
> our time on the cryptography. e.g.
> 
>       self.boot_linux('max,pauth-impdef=on')

Thanks! Moved from 33s to 20s here:

from:

  (3/4) tests/avocado/machine_aarch64_sbsaref.py:Aarch64SbsarefMachine.test_sbsaref_linux_max: PASS (33.75 s)

to:

  (3/4) tests/avocado/machine_aarch64_sbsaref.py:Aarch64SbsarefMachine.test_sbsaref_linux_max: PASS (20.34 s)
  
> but even with that it seems to be timing out for me:

JOB ID     : 030a40bf6c2523b25bed4ca7a630b380d4dba39d
JOB LOG    : /home/marcin/devel/linaro/sbsa-qemu/code/qemu/build/tests/results/job-2023-03-03T16.04-030a40b/job.log
  (1/4) tests/avocado/machine_aarch64_sbsaref.py:Aarch64SbsarefMachine.test_sbsaref_tfa_v2_8: PASS (3.00 s)
  (2/4) tests/avocado/machine_aarch64_sbsaref.py:Aarch64SbsarefMachine.test_sbsaref_linux_a57: PASS (19.37 s)
  (3/4) tests/avocado/machine_aarch64_sbsaref.py:Aarch64SbsarefMachine.test_sbsaref_linux_max: PASS (20.34 s)
  (4/4) tests/avocado/machine_aarch64_sbsaref.py:Aarch64SbsarefMachine.test_alpine_sbsaref_max: PASS (71.96 s)
RESULTS    : PASS 4 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 115.00 s

[PATCH 3/3] tests/avocado: add Alpine Linux boot on AArch64/sbsa-ref

[Marcin Juszkiewicz]

To unify tests for AArch64/virt and AArch64/sbsa-ref we boot
same Alpine Linux image on both.

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
---
 tests/avocado/machine_aarch64_sbsaref.py | 29 ++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py
index e36e53d4d5..fdbde4b891 100644
--- a/tests/avocado/machine_aarch64_sbsaref.py
+++ b/tests/avocado/machine_aarch64_sbsaref.py
@@ -148,3 +148,32 @@ def test_sbsaref_linux_max(self):
         :avocado: tags=cpu:max
         """
         self.boot_linux('max')
+
+    # This tests the whole boot chain from EFI to Userspace
+    # We only boot a whole OS for the current top level CPU and GIC
+    # Other test profiles should use more minimal boots
+    def test_alpine_sbsaref_max(self):
+        """
+        :avocado: tags=arch:aarch64
+        :avocado: tags=machine:sbsa-ref
+        """
+        self.fetch_firmware()
+
+        iso_url = ('https://dl-cdn.alpinelinux.org/'
+                   'alpine/v3.17/releases/aarch64/'
+                   'alpine-standard-3.17.2-aarch64.iso')
+
+        # Alpine use sha256 so I recalculated this myself
+        iso_sha1 = '76284fcd7b41fe899b0c2375ceb8470803eea839'
+        iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha1)
+
+        self.vm.set_console()
+        self.vm.add_args("-cpu", "max,pauth-impdef=on")
+        self.vm.add_args("-machine", "sbsa-ref")
+        self.vm.add_args("-drive", f"file={iso_path},format=raw")
+        self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
+        self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom')
+
+        self.vm.launch()
+        wait_for_console_pattern(self, 'Welcome to Alpine Linux 3.17')
+
-- 
2.39.2

Re: [PATCH 3/3] tests/avocado: add Alpine Linux boot on AArch64/sbsa-ref

[Alex Bennée]

Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> writes:

> To unify tests for AArch64/virt and AArch64/sbsa-ref we boot
> same Alpine Linux image on both.
>
> Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
> ---
>  tests/avocado/machine_aarch64_sbsaref.py | 29 ++++++++++++++++++++++++
>  1 file changed, 29 insertions(+)
>
> diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py
> index e36e53d4d5..fdbde4b891 100644
> --- a/tests/avocado/machine_aarch64_sbsaref.py
> +++ b/tests/avocado/machine_aarch64_sbsaref.py
> @@ -148,3 +148,32 @@ def test_sbsaref_linux_max(self):
>          :avocado: tags=cpu:max
>          """
>          self.boot_linux('max')
> +
> +    # This tests the whole boot chain from EFI to Userspace
> +    # We only boot a whole OS for the current top level CPU and GIC
> +    # Other test profiles should use more minimal boots
> +    def test_alpine_sbsaref_max(self):
> +        """
> +        :avocado: tags=arch:aarch64
> +        :avocado: tags=machine:sbsa-ref
> +        """
> +        self.fetch_firmware()
> +
> +        iso_url = ('https://dl-cdn.alpinelinux.org/'
> +                   'alpine/v3.17/releases/aarch64/'
> +                   'alpine-standard-3.17.2-aarch64.iso')
> +
> +        # Alpine use sha256 so I recalculated this myself
> +        iso_sha1 = '76284fcd7b41fe899b0c2375ceb8470803eea839'
> +        iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha1)
> +
> +        self.vm.set_console()
> +        self.vm.add_args("-cpu", "max,pauth-impdef=on")
> +        self.vm.add_args("-machine", "sbsa-ref")
> +        self.vm.add_args("-drive", f"file={iso_path},format=raw")
> +        self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
> +        self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom')
> +
> +        self.vm.launch()
> +        wait_for_console_pattern(self, 'Welcome to Alpine Linux 3.17')
> +

This seems to get to grub before hanging:

  2023-03-03 11:35:53,696 protocol         L0495 DEBUG| Transitioning from 'Runstate.CONNECTING' to 'Runstate.RUNNING'.
  2023-03-03 11:35:53,700 __init__         L0153 DEBUG| NOTICE:  Booting Trusted Firmware
  2023-03-03 11:35:53,700 __init__         L0153 DEBUG| NOTICE:  BL1: v2.8(release):v2.8-335-g6264643a0
  2023-03-03 11:35:53,700 __init__         L0153 DEBUG| NOTICE:  BL1: Built : 17:51:50, Feb  8 2023
  2023-03-03 11:35:53,703 __init__         L0153 DEBUG| NOTICE:  BL1: Booting BL2
  2023-03-03 11:35:53,707 __init__         L0153 DEBUG| NOTICE:  BL2: v2.8(release):v2.8-335-g6264643a0
  2023-03-03 11:35:53,707 __init__         L0153 DEBUG| NOTICE:  BL2: Built : 17:51:50, Feb  8 2023
  2023-03-03 11:35:53,722 __init__         L0153 DEBUG| NOTICE:  BL1: Booting BL31
  2023-03-03 11:35:53,727 __init__         L0153 DEBUG| NOTICE:  BL31: v2.8(release):v2.8-335-g6264643a0
  2023-03-03 11:35:53,727 __init__         L0153 DEBUG| NOTICE:  BL31: Built : 17:51:50, Feb  8 2023
  2023-03-03 11:35:53,732 __init__         L0153 DEBUG| UEFI firmware (version 1.0 built at 16:17:37 on Feb  9 2023)
  2023-03-03 11:35:57,556 __init__         L0153 DEBUG| ESC[2JESC[04DESC[=3hESC[2JESC[09DESC[0mESC[30mESC[47mWelcome to GRUB!
  2023-03-03 11:35:57,599 __init__         L0153 DEBUG| ESC[0mESC[37mESC[40mESC[0mESC[30mESC[40mESC[2JESC[01;01HESC[0mESC[37mESC[40mESC[02;30HGNU GRUB  version 2.06
  2023-03-03 11:35:57,624 __init__         L0153 DEBUG| ESC[01C/----------------------------------------------------------------------------\ESC[05;02H|ESC[76C|ESC[06;02H|ESC[76C|ESC[07;02H|ESC[76C|ESC[08;02H|ESC[76C|ESC[09;02H|ESC[76C|ESC[10;02H|ESC[76C|ESC[11;02H|ESC[76C|ESC[12;02H|ESC[76C|ESC[13;02H|ESC[76C|ESC[14;02H|ESC[76C|ESC[15;02H|ESC[76C|ESC[16;02H|ESC[76C|ESC[17;02H|ESC[76C|ESC[18;02H\----------------------------------------------------------------------------/ESC[19;02HESC[20;02H     Use the ^ and v keys to select which entry is highlighted.
  2023-03-03 11:35:57,628 __init__         L0153 DEBUG| Press enter to boot the selected OS, `e' to edit the commands
  2023-03-03 11:35:57,632 __init__         L0153 DEBUG| before booting or `c' for a command-line.                           ESC[05;80H
  2023-03-03 11:35:57,698 __init__         L0153 DEBUG| ESC[0mESC[30mESC[47mESC[05;03H*Linux lts                                                                  ESC[0mESC[37mESC[40mESC[01DESC[06;03H                                                                            ESC[01DESC[07;03H                                                                            ESC[01DESC[08;03H                                                                            ESC[01DESC[09;03H                                                                            ESC[01DESC[10;03H                                                                            ESC[01DESC[11;03H                                                                            ESC[01DESC[12;03H                                                                            ESC[01DESC[13;03H                                                                            ESC[01DESC[14;03H                                                                            ESC[01DESC[15;03H                                                                            ESC[01DESC[16;03H                                                                            ESC[01D
  ESC[17;03H                                                                            ESC[01DESC[02C
  2023-03-03 11:35:58,724 __init__         L0153 DEBUG| ESC[05;78HESC[23;01H   The highlighted entry will be executed automatically in 1s.                 ESC[05;78HESC[23;01H   The highlighted entry will be executed automatically in 0s.                 ESC[05;78HESC[0mESC[30mESC[40mESC[2JESC[01;01HESC[0mESC[37mESC[40mESC[0mESC[30mESC[40mESC[2J
  ESC[04DESC[0mESC[37mESC[40m  Booting `Linux lts'
  2023-03-03 11:37:51,367 stacktrace       L0039 ERROR| 
  2023-03-03 11:37:51,368 stacktrace       L0041 ERROR| Reproduced traceback from: /home/alex/lsrc/qemu.git/builds/arm.all/tests/venv/lib/python3.9/site-packages/avocado/core/test.py:770
  2023-03-03 11:37:51,368 stacktrace       L0045 ERROR| Traceback (most recent call last):
  2023-03-03 11:37:51,368 stacktrace       L0045 ERROR|   File "/home/alex/lsrc/qemu.git/builds/arm.all/tests/avocado/machine_aarch64_sbsaref.py", line 178, in test_alpine_sbsaref_max


-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH 1/3] tests/avocado: update AArch64 tests to Alpine 3.17.2

[Alex Bennée]

Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> writes:

Next time please ensure you have a cover letter for a multi-patch series
as the various bits of tooling get confused without one.

> To test Alpine boot on SBSA-Ref target we need Alpine Linux
> 'standard' image as 'virt' one lacks kernel modules.

Is this kernel modules we will need for SBSA "hw" choices?

> So to minimalize Avocado cache I move test to 'standard' image.
>
> Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  tests/avocado/machine_aarch64_virt.py | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/tests/avocado/machine_aarch64_virt.py b/tests/avocado/machine_aarch64_virt.py
> index 25dab8dc00..a90dc6ff4b 100644
> --- a/tests/avocado/machine_aarch64_virt.py
> +++ b/tests/avocado/machine_aarch64_virt.py
> @@ -38,11 +38,11 @@ def test_alpine_virt_tcg_gic_max(self):
>          :avocado: tags=accel:tcg
>          """
>          iso_url = ('https://dl-cdn.alpinelinux.org/'
> -                   'alpine/v3.16/releases/aarch64/'
> -                   'alpine-virt-3.16.3-aarch64.iso')
> +                   'alpine/v3.17/releases/aarch64/'
> +                   'alpine-standard-3.17.2-aarch64.iso')
>  
>          # Alpine use sha256 so I recalculated this myself
> -        iso_sha1 = '0683bc089486d55c91bf6607d5ecb93925769bc0'
> +        iso_sha1 = '76284fcd7b41fe899b0c2375ceb8470803eea839'
>          iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha1)
>  
>          self.vm.set_console()
> @@ -65,7 +65,7 @@ def test_alpine_virt_tcg_gic_max(self):
>          self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom')
>  
>          self.vm.launch()
> -        self.wait_for_console_pattern('Welcome to Alpine Linux 3.16')
> +        self.wait_for_console_pattern('Welcome to Alpine Linux 3.17')
>  
>  
>      def common_aarch64_virt(self, machine):


-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH 1/3] tests/avocado: update AArch64 tests to Alpine 3.17.2

[Marcin Juszkiewicz]

W dniu 3.03.2023 o 12:24, Alex Bennée pisze:
> 
> Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> writes:
> 
> Next time please ensure you have a cover letter for a multi-patch series
> as the various bits of tooling get confused without one.
> 
>> To test Alpine boot on SBSA-Ref target we need Alpine Linux
>> 'standard' image as 'virt' one lacks kernel modules.
> 
> Is this kernel modules we will need for SBSA "hw" choices?

Yes. Virt image fails to boot on sbsa-ref:

   Booting `Linux virt'
-console: Mounting boot media failed.
console: initramfs emergency recovery shell launched. Type 'exit' to 
continue boot
console: sh: can't access tty; job control turned off