Re: [PATCH 5/6] tests/acceptance/virtio_seg_max_adjust: Restrict to X86 architecture

[Cornelia Huck <cohuck@redhat.com>]

On Wed, 22 Jan 2020 23:32:46 +0100
Philippe Mathieu-Daudé <philmd@redhat.com> wrote:

> Running on mainstream KVM architectures, we get:
> 
> - Aarch64
> 
>   Timeout.
> 
>   job.log:
>   -------
>   No machine specified, and there is no default
>   Use -machine help to list supported machines

The code probably needs to be made more clever to find the machines to
run?

> 
> - MIPS:
> 
>    (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: argument of type 'NoneType' is not iterable (0.14 s)
> 
>   job.log:
>   -------
>   Could not load MIPS bios 'mipsel_bios.bin', and no -kernel argument was specified

Probably needs some hint from mips folks how this can be set up.

> 
> - PowerPC
> 
>    (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: invalid literal for int() with base 10: 'sxxm' (0.16 s)
> 
>   job.log:
>   -------
>   >>> {'execute': 'query-machines'}  
>   <<< {'return': [{'hotpluggable-cpus': True, 'name': 'pseries-2.12-sxxm', 'numa-mem-supported': True, 'default-cpu-type': 'power8_v2.0-powerpc64-cpu', 'cpu-max': 1024, 'deprecated': False}, ...

This seems to be because the machine type parsing code cannot deal with
the format used here.

> 
> - S390X:
> 
>    (1/1) VirtioMaxSegSettingsCheck.test_machine_types: ERROR: invalid literal for int() with base 10: 'virtio' (0.14 s)
> 
>   job.log:
>   -------
>   Traceback (most recent call last):
>     File "virtio_seg_max_adjust.py", line 139, in test_machine_types
>       if self.seg_max_adjust_enabled(m):
>     File "virtio_seg_max_adjust.py", line 113, in seg_max_adjust_enabled
>       major = int(ver[0])
>   ValueError: invalid literal for int() with base 10: 'virtio'
>   >>> {'execute': 'query-machines'}  
>   <<< {'return': [{'hotpluggable-cpus': True, 'name': 's390-ccw-virtio-4.0', 'numa-mem-supported': False, 'default-cpu-type': 'qemu-s390x-cpu', 'cpu-max': 248, 'deprecated': False}, ...

Same here.

> 
> Assuming this test is only expected to run on the X86 architecture,
> restrict the test to this particular architecture.
> 
> When this test is run on other architecture, the tests will be skipped.
> 
> Examples:
> 
> - running on S390X:
> 
>  (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types: SKIP: Architecture 's390' unsupported
> 
> - running on Aarch64 setting the QEMU binary path:
> 
>   $ uname -m && avocado --show=app run -p qemu_bin=x86_64-softmmu/qemu-system-x86_64 tests/acceptance/virtio_seg_max_adjust.py
>   aarch64
>   JOB ID     : 92b7fae8868920aada0cb143f9571dffdf60931d
>   JOB LOG    : job-results/job-2020-01-22T17.54-92b7fae/job.log
>    (1/1) tests/acceptance/virtio_seg_max_adjust.py:VirtioMaxSegSettingsCheck.test_machine_types: PASS (25.99 s)
>   RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>   JOB TIME   : 26.13 s
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/acceptance/virtio_seg_max_adjust.py | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/tests/acceptance/virtio_seg_max_adjust.py b/tests/acceptance/virtio_seg_max_adjust.py
> index ad736bcda3..2fc6bfcbd8 100755
> --- a/tests/acceptance/virtio_seg_max_adjust.py
> +++ b/tests/acceptance/virtio_seg_max_adjust.py
> @@ -26,6 +26,7 @@ import logging
>  sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
>  from qemu.machine import QEMUMachine
>  from avocado_qemu import Test
> +from avocado.core.exceptions import TestSkipError
>  
>  #list of machine types and virtqueue properties to test
>  VIRTIO_SCSI_PROPS = {'seg_max_adjust': 'seg_max_adjust'}
> @@ -117,12 +118,22 @@ class VirtioMaxSegSettingsCheck(Test):
>          return False
>  
>      def test_machine_types(self):
> +        """
> +        :avocado: tags=arch:i386
> +        :avocado: tags=arch:x86_64
> +        """
>          EXCLUDED_MACHINES = ['none', 'isapc', 'microvm']
>          if os.geteuid() != 0:
>              EXCLUDED_MACHINES += ['xenfv', 'xenpv']
>          # collect all machine types except the ones in EXCLUDED_MACHINES
>          with QEMUMachine(self.qemu_bin) as vm:
>              vm.launch()
> +            # Skip test if target is not X86
> +            # TODO: Move this check to Avocado (based on the test tags)
> +            target_arch = vm.command('query-target')['arch']
> +            if target_arch not in ['i386', 'x86_64']:
> +                errmsg = "Architecture '%s' unsupported" % target_arch
> +                raise TestSkipError(errmsg)

I think we should rather fix the machine parsing code, and only then
exclude architectures out of the box. (Sorry, my python-fu is lacking,
or I would try it myself.)

There does not seem to be anything that is really architecture specific
in there. Just explicitly requesting the -pci versions of virtio-blk
and virtio-scsi seems wrong, though, as this looks like a generic
property, and should work on -ccw as well. (And probably also for
virtio-mmio devices.) If not, I'd like to know :)

>              machines = [m['name'] for m in vm.command('query-machines')]
>              vm.shutdown()
>          for m in EXCLUDED_MACHINES: