Re: [PATCH v3 1/4] tests/acceptance: avocado_qemu: Introduce the 'accel' test parameter

[Wainer dos Santos Moschetta <wainersm@redhat.com>]

On 1/30/20 8:51 PM, Philippe Mathieu-Daudé wrote:
> On 1/27/20 8:28 PM, Wainer dos Santos Moschetta wrote:
>> On 1/24/20 7:36 AM, Philippe Mathieu-Daudé wrote:
>>> On 1/22/20 2:27 AM, Wainer dos Santos Moschetta wrote:
>>>> The test case may need to boot the VM with an accelerator that
>>>> isn't actually enabled on the QEMU binary and/or present in the 
>>>> host. In
>>>> this case the test behavior is undefined, and the best course of
>>>> action is to skip its execution.
>>>>
>>>> This change introduced the 'accel' parameter (and the handler of
>>>> tag with same name) used to indicate the test case requires a
>>>> given accelerator available. It was implemented a mechanism to
>>>> skip the test case if the accelerator is not available. Moreover,
>>>>   the QEMU -accel argument is set automatically to any VM
>>>> launched if the parameter is present.
>>>>
>>>> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>>>> ---
>>>>   docs/devel/testing.rst                    | 16 ++++++++++++++++
>>>>   tests/acceptance/avocado_qemu/__init__.py | 23 
>>>> +++++++++++++++++++++++
>>>>   2 files changed, 39 insertions(+)
>>>>
>>>> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
>>>> index ab5be0c729..d17d0e90aa 100644
>>>> --- a/docs/devel/testing.rst
>>>> +++ b/docs/devel/testing.rst
>>>> @@ -759,6 +759,17 @@ name.  If one is not given explicitly, it will 
>>>> either be set to
>>>>   ``None``, or, if the test is tagged with one (and only one)
>>>>   ``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``.
>>>>   +accel
>>>> +~~~~~
>>>> +The accelerator that will be set to all QEMUMachine instances created
>>>> +by the test.
>>>> +
>>>> +The ``accel`` attribute will be set to the test parameter of the same
>>>> +name.  If one is not given explicitly, it will either be set to
>>>> +``None``, or, if the test is tagged with one (and only one)
>>>> +``:avocado: tags=accel:VALUE`` tag, it will be set to ``VALUE``. 
>>>> Currently
>>>> +``VALUE`` should be either ``kvm`` or ``tcg``.
>>>> +
>>>>   qemu_bin
>>>>   ~~~~~~~~
>>>>   @@ -800,6 +811,11 @@ machine
>>>>   The machine type that will be set to all QEMUMachine instances 
>>>> created
>>>>   by the test.
>>>>   +accel
>>>> +~~~~~
>>>> +The accelerator that will be set to all QEMUMachine instances created
>>>> +by the test. In case the accelerator is not available (both QEMU
>>>> +binary and the host system are checked) then the test is canceled.
>>>>     qemu_bin
>>>>   ~~~~~~~~
>>>> diff --git a/tests/acceptance/avocado_qemu/__init__.py 
>>>> b/tests/acceptance/avocado_qemu/__init__.py
>>>> index 6618ea67c1..c83a75ccbc 100644
>>>> --- a/tests/acceptance/avocado_qemu/__init__.py
>>>> +++ b/tests/acceptance/avocado_qemu/__init__.py
>>>> @@ -20,6 +20,7 @@ SRC_ROOT_DIR = 
>>>> os.path.join(os.path.dirname(__file__), '..', '..', '..')
>>>>   sys.path.append(os.path.join(SRC_ROOT_DIR, 'python'))
>>>>     from qemu.machine import QEMUMachine
>>>> +from qemu.accel import kvm_available, tcg_available
>>>>     def is_readable_executable_file(path):
>>>>       return os.path.isfile(path) and os.access(path, os.R_OK | 
>>>> os.X_OK)
>>>> @@ -111,6 +112,8 @@ class Test(avocado.Test):
>>>>         def setUp(self):
>>>>           self._vms = {}
>>>> +        # VM argumments that are mapped from parameters
>>>> +        self._param_to_vm_args = []
>>>>             self.arch = self.params.get('arch',
>>>> default=self._get_unique_tag_val('arch'))
>>>> @@ -124,10 +127,30 @@ class Test(avocado.Test):
>>>>           if self.qemu_bin is None:
>>>>               self.cancel("No QEMU binary defined or found in the 
>>>> source tree")
>>>>   +        self.accel = self.params.get('accel',
>>>> + default=self._get_unique_tag_val('accel'))
>>>> +        if self.accel:
>>>> +            avail = False
>>>> +            if self.accel == 'kvm':
>>>> +                if kvm_available(self.arch, self.qemu_bin):
>>>> +                    avail = True
>>>> +            elif self.accel == 'tcg':
>>>> +                if tcg_available(self.qemu_bin):
>>>> +                    avail = True
>>>> +            else:
>>>> +                self.cancel("Unknown accelerator: %s" % self.accel)
>>>> +
>>>> +            if avail:
>>>> +                self._param_to_vm_args.extend(['-accel', self.accel])
>>>> +            else:
>>>> +                self.cancel("%s is not available" % self.accel)
>>>
>>> Why refuse to test the other accelerators?
>>>
>>> Isn't it better to QMP-ask QEMU which accelerator it supports, and 
>>> SKIP if it isn't available?
>>
>>
>> python/qemu/accel.py:list_accel(qemu_bin) can be used for that end. 
>> For example:
>>
>> if self.accel not in list_accel(self.qemu_bin):
>>
>>      self.cancel("%s is not available" % self.accel)
>>
>> However checking the support only on QEMU's binary may be very weak 
>> (take KVM as an example). I implemented checkers for kvm and tcg on 
>> python/qemu/accel.py. Given that I have zero knowledge on the other 
>> accelerators,  I simply did not touch on them.
>>
>> That said, IMHO it needs to implement checkers for those others 
>> accelerator before they get reliably handled by avocado_qemu 
>> automatically. And test writers can still run tests with those 
>> accelerators as long as 'accel' tag is not used.
>>
>> What do you think Philippe?
>
> Can you have a look at the binary_get_accels() method which aims to be 
> more generic?
>
> https://lists.gnu.org/archive/html/qemu-devel/2020-01/msg07312.html
> "python/qemu: Add binutils::binary_get_accels()"
>
> https://lists.gnu.org/archive/html/qemu-devel/2020-01/msg07313.html
> "python/qemu/accel: Use binutils::binary_get_accels()"


Sure, I will have a look at those.

But I prefer to stick with the implementation proposed on this patch. 
IMHO if avocado_qemu is going to set the accelerator from the tag, then 
it should/could go further with additional checks (e.g. if kvm is 
enabled at host, not just at QEMU binary).

- Wainer

>
>>
>> Thanks, good point!
>>
>> - Wainer
>>
>>
>>>
>>>
>>>> +
>>>>       def _new_vm(self, *args):
>>>>           vm = QEMUMachine(self.qemu_bin, sock_dir=tempfile.mkdtemp())
>>>>           if args:
>>>>               vm.add_args(*args)
>>>> +        if self._param_to_vm_args:
>>>> +            vm.add_args(*self._param_to_vm_args)
>>>>           return vm
>>>>         @property
>>>>
>>>
>>
>