Re: [PATCH] tests/functional: Convert most Aspeed machine tests

["Philippe Mathieu-Daudé" <philmd@linaro.org>]

On 16/10/24 06:30, Daniel P. Berrangé wrote:
> On Wed, Oct 16, 2024 at 11:07:41AM +0200, Cédric Le Goater wrote:
>> This is a simple conversion of the tests with some cleanups and
>> adjustments to match the new test framework. Replace the zephyr image
>> MD5 hashes with SHA256 hashes while at it.
>>
>> The SDK tests depend on a ssh class from avocado.utils which is
>> difficult to replace. To be addressed separately.
>>
>> Signed-off-by: Cédric Le Goater <clg@redhat.com>
>> ---
>>   tests/avocado/machine_aspeed.py     | 252 --------------------------
>>   tests/functional/meson.build        |   2 +
>>   tests/functional/test_arm_aspeed.py | 269 ++++++++++++++++++++++++++++
>>   3 files changed, 271 insertions(+), 252 deletions(-)
>>   create mode 100644 tests/functional/test_arm_aspeed.py
>>
>> diff --git a/tests/functional/test_arm_aspeed.py b/tests/functional/test_arm_aspeed.py
>> new file mode 100644
>> index 000000000000..2f9a90f64d8f
>> --- /dev/null
>> +++ b/tests/functional/test_arm_aspeed.py
>> @@ -0,0 +1,269 @@
>> +#!/usr/bin/env python3
>> +#
>> +# Functional test that boots the ASPEED SoCs with firmware
>> +#
>> +# Copyright (C) 2022 ASPEED Technology Inc
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +import os
>> +import time
>> +import subprocess
>> +import tempfile
>> +
>> +from qemu_test import QemuSystemTest, Asset
>> +from qemu_test import wait_for_console_pattern
>> +from qemu_test import exec_command_and_wait_for_pattern
>> +from qemu_test import interrupt_interactive_console_until_pattern
>> +from qemu_test import exec_command
>> +from qemu_test import has_cmd
>> +from qemu_test.utils import archive_extract
>> +from zipfile import ZipFile
>> +from unittest import skipUnless
>> +
>> +class AST1030Machine(QemuSystemTest):
>> +
>> +    def test_ast1030_zephyros_1_04(self):
>> +        asset_url = Asset(
>> +            ('https://github.com/AspeedTech-BMC'
>> +             '/zephyr/releases/download/v00.01.04/ast1030-evb-demo.zip'),
>> +            '4ac6210adcbc61294927918707c6762483fd844dde5e07f3ba834ad1f91434d3')
> 
> Don't instantiate assets inline to test code, as this makes them
> invisible to the logic that pre-downloads assets prior to running
> tests. As a result you're liable to have test timeouts if the
> on-the-fly asset download takes too long.
> 
> Instead You should use a class level varible with an "ASSET_" name
> prefix eg
> 
>     ASSET_<BLAH> = Asset(....)

This is somehow documented in docs/devel/testing/functional.rst but
could be clarified:

   The second problem with downloading files from the internet are time
   constraints. The time for downloading files should not be taken into
   account when the test is running and the timeout of the test is
   ticking (since downloading can be very slow, depending on the network
   bandwidth).
   This problem is solved by downloading the assets ahead of time, before
   the tests are run. This pre-caching is done with the qemu_test.Asset
   class. To use it in your test, declare an asset in your test class
   with its URL and SHA256 checksum like this::

       ASSET_somename = (
           ('https://www.qemu.org/assets/images/qemu_head_200.png'),
 
'34b74cad46ea28a2966c1d04e102510daf1fd73e6582b6b74523940d5da029dd')


> 
> See other converted tests for examples of the pattern.
> 
>> +        kernel_name = "ast1030-evb-demo/zephyr.elf"
>> +        zip_file = asset_url.fetch()
>> +        with ZipFile(zip_file, 'r') as zf:
>> +                     zf.extract(kernel_name, path=self.workdir)
>> +        kernel_file = os.path.join(self.workdir, kernel_name)
>> +
>> +        self.set_machine('ast1030-evb')
>> +        self.vm.set_console()
>> +        self.vm.add_args('-kernel', kernel_file, '-nographic')
>> +        self.vm.launch()
>> +        wait_for_console_pattern(self, "Booting Zephyr OS")
>> +        exec_command_and_wait_for_pattern(self, "help",
>> +                                          "Available commands")
>> +
> 
> With regards,
> Daniel