Re: [PATCH v2 15/21] tests/acceptance: add a new set of tests to exercise plugins

[Wainer dos Santos Moschetta <wainersm@redhat.com>]

On 2/11/21 7:31 AM, Philippe Mathieu-Daudé wrote:
> On 2/10/21 11:10 PM, Alex Bennée wrote:
>> This is just a simple test to count the instructions executed by a
>> kernel. However a later test will detect a failure condition when
>> icount is enabled.
>>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Message-Id: <20210209182749.31323-7-alex.bennee@linaro.org>
>> ---
>>   tests/acceptance/tcg_plugins.py | 103 ++++++++++++++++++++++++++++++++
>>   1 file changed, 103 insertions(+)
>>   create mode 100644 tests/acceptance/tcg_plugins.py
> Can you add this file to "TCG Plugins" in MAINTAINERS please?
>
>> diff --git a/tests/acceptance/tcg_plugins.py b/tests/acceptance/tcg_plugins.py
>> new file mode 100644
>> index 0000000000..b512979769
>> --- /dev/null
>> +++ b/tests/acceptance/tcg_plugins.py
>> @@ -0,0 +1,103 @@
>> +# TCG Plugins tests
>> +#
>> +# These are a little more involved than the basic tests run by check-tcg.
>> +#
>> +# Copyright (c) 2021 Linaro
>> +#
>> +# Author:
>> +#  Alex Bennée <alex.bennee@linaro.org>
>> +#
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +
>> +import logging
>> +import time
>> +import tempfile
>> +import mmap
>> +import re
>> +
>> +from boot_linux_console import LinuxKernelTest
>> +
>> +
>> +class PluginKernelBase(LinuxKernelTest):
>> +    """
>> +    Boots a Linux kernel with a TCG plugin enabled.
>> +    """
>> +
>> +    timeout = 120
>> +    KERNEL_COMMON_COMMAND_LINE = 'printk.time=1 panic=-1 '
>> +
>> +    def run_vm(self, kernel_path, kernel_command_line,
>> +               plugin, plugin_log, console_pattern, args):
>> +
>> +        logger = logging.getLogger('plugin')
>> +        start_time = time.time()
>> +        vm = self.get_vm()
>> +        vm.set_console()
>> +        vm.add_args('-kernel', kernel_path,
>> +                    '-append', kernel_command_line,
>> +                    '-plugin', plugin,
>> +                    '-d', 'plugin',
>> +                    '-D', plugin_log,
>> +                    '-net', 'none',
>> +                    '-no-reboot')
>> +        if args:
>> +            vm.add_args(*args)
>> +
>> +        try:
>> +            vm.launch()
>> +        except:
>> +            # fails if plugins not enabled
>> +            self.cancel("TCG Plugins not enabled")
> The test could fail for other reasons you want to catch...
>
> We don't have yet the possibility to query the binary if it
> has the plugin feature builtin. Can you add a TODO comment
> so we fix that once we got it?
>
>> +
>> +        self.wait_for_console_pattern(console_pattern, vm)
>> +        elapsed = time.time() - start_time
>> +        logger.info('elapsed time %.2f sec' % elapsed)
>> +        # ensure logs are flushed
>> +        vm.shutdown()
>> +        return elapsed
>> +
>> +
>> +class PluginKernelNormal(PluginKernelBase):
>> +
>> +    def _grab_aarch64_kernel(self):
>> +        kernel_url = ('http://security.debian.org/'
>> +                      'debian-security/pool/updates/main/l/linux-signed-arm64/'
>> +                      'linux-image-4.19.0-12-arm64_4.19.152-1_arm64.deb')
>> +        kernel_sha1 = '2036c2792f80ac9c4ccaae742b2e0a28385b6010'
>> +        kernel_deb = self.fetch_asset(kernel_url, asset_hash=kernel_sha1)
>> +        kernel_path = self.extract_from_deb(kernel_deb,
>> +                                            "/boot/vmlinuz-4.19.0-12-arm64")
>> +        return kernel_path
>> +
>> +    def test_aarch64_virt_insn(self):
>> +        """
>> +        :avocado: tags=accel:tcg
>> +        :avocado: tags=arch:aarch64
>> +        :avocado: tags=machine:virt
>> +        :avocado: tags=cpu:cortex-a57
>> +        """
>> +        kernel_path = self._grab_aarch64_kernel()
>> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
>> +                               'console=ttyAMA0')
>> +        console_pattern = 'Kernel panic - not syncing: VFS:'
>> +
>> +        plugin_log = tempfile.NamedTemporaryFile(mode="r+t", prefix="plugin",
>> +                                                 suffix=".log")
>> +
>> +        self.run_vm(kernel_path, kernel_command_line,
>> +                    "tests/plugin/libinsn.so", plugin_log.name,
>> +                    console_pattern,
>> +                    args=('-cpu', 'cortex-a53'))
> I had the understanding that by using 'tags=cpu' QEMU would be called
> with "-cpu tags['cpu']". Cleber can you confirm please?

Currently only the 'machine' tag is automatically converted to QEMU 
arguments (in tests/acceptance/avocado_qemu/__init__.py::get_vm()). 
Having the same behavior for other tags like 'cpu' and 'accel' should be 
helpful IMO.


- Wainer

>
>> +
>> +        logger = logging.getLogger()
>> +
>> +        with plugin_log as lf, \
>> +             mmap.mmap(lf.fileno(), 0, access=mmap.ACCESS_READ) as s:
>> +
>> +            m = re.search(br"insns: (?P<count>\d+)", s)
>> +            if "count" in m.groupdict():
>> +                logger.debug("reported %d instructions",
>> +                             int(m.group("count")))
>> +            else:
>> +                logger.debug("Failed to find instruction count")
>> +                self.fail
>>
> With MAINTAINERS & TODO:
> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
> Thanks,
>
> Phil.
>