Re: [PATCH 1/6] tests/boot_linux_console: add microvm acceptance test

[Wainer dos Santos Moschetta <wainersm@redhat.com>]

On 1/30/20 9:51 PM, Philippe Mathieu-Daudé wrote:
> On 1/30/20 6:41 PM, Wainer dos Santos Moschetta wrote:
>> On 1/27/20 2:36 PM, Liam Merwick wrote:
>>> Refactor test_x86_64_pc() to test_x86_64_machine() so that separate
>>> functions which specify the Avocado tag of ':avocado: tags=machine:'
>>> as being either 'pc' or 'microvm' can be used to test booting a
>>> compressed kernel using either machine class.
>>>
>>> Signed-off-by: Liam Merwick <liam.merwick@oracle.com>
>>> ---
>>>   tests/acceptance/boot_linux_console.py | 15 +++++++++++++--
>>>   1 file changed, 13 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/tests/acceptance/boot_linux_console.py 
>>> b/tests/acceptance/boot_linux_console.py
>>> index e40b84651b0b..aa5b07b1c609 100644
>>> --- a/tests/acceptance/boot_linux_console.py
>>> +++ b/tests/acceptance/boot_linux_console.py
>>> @@ -51,10 +51,9 @@ class BootLinuxConsole(Test):
>>>           os.chdir(cwd)
>>>           return self.workdir + path
>>> -    def test_x86_64_pc(self):
>>> +    def do_test_x86_64_machine(self):
>>>           """
>>>           :avocado: tags=arch:x86_64
>>> -        :avocado: tags=machine:pc
>>>           """
>>>           kernel_url = 
>>> ('https://archives.fedoraproject.org/pub/archive/fedora'
>>> '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
>>> @@ -70,6 +69,18 @@ class BootLinuxConsole(Test):
>>>           console_pattern = 'Kernel command line: %s' % 
>>> kernel_command_line
>>>           self.wait_for_console_pattern(console_pattern)
>>> +    def test_x86_64_pc(self):
>>> +        """
>>> +        :avocado: tags=machine:pc
>>> +        """
>>
>> The test method won't inherit the 'arch' tag from 
>> `do_test_x86_64_machine()`, so you need to explicitly 'arch' tag each 
>> test you created in this series. If you don't do so, Avocado won't 
>> filter out those x86_64 tests in case QEMU is built with non-x86_64 
>> targets.
>>
>> Follows an example, I built QEMU with '--target-list=arm-softmmu'. I 
>> got:
>>
>> ```
>>
>> (02/18) 
>> tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc: 
>> CANCEL: No QEMU binary defined or found in the source tree (0.00 s)
>>   (03/18) 
>> tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_microvm: 
>> CANCEL: No QEMU binary defined or found in the source tree (0.00 s)
>>   (04/18) 
>> tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_virt: 
>> PASS (1.25 s)
>>
>> ```
>>
>> OK, avocado_qemu was smart enough to skip the tests, but ideally it 
>> should not even consider running them in the first place.
>
> This should be solved by this patch:
> https://lists.gnu.org/archive/html/qemu-devel/2020-01/msg07311.html
> "tests/acceptance: Use 'machine' tag to check if available in QEMU binary"


That patch is useful**  but does not solve the problem that I point out 
here. If you have a test case which is arch-specific and the arch-QEMU 
was not built then it should not be executed. Tagging the test with 
'arch' ensures that.

** BTW, I'm going to review it soon

- Wainer


>>
>> Thanks!
>>
>> - Wainer
>>
>>> +        self.do_test_x86_64_machine()
>>> +
>>> +    def test_x86_64_microvm(self):
>>> +        """
>>> +        :avocado: tags=machine:microvm
>>> +        """
>>> +        self.do_test_x86_64_machine()
>>> +
>>>       def test_mips_malta(self):
>>>           """
>>>           :avocado: tags=arch:mips
>>
>
>