Re: [PATCH v3 00/11] Record/replay acceptance tests

["Philippe Mathieu-Daudé" <philmd@redhat.com>]

Hi Pavel,

On 5/31/20 5:09 PM, Philippe Mathieu-Daudé wrote:
> On 5/29/20 9:04 AM, Pavel Dovgalyuk wrote:
>> The following series adds record/replay tests to the acceptance group.
>> Test pass successfully with the latest submitted record/replay fixes:
>>  - replay: notify the main loop when there are no instructions
>>  - replay: synchronize on every virtual timer callback
>>
>> The provided tests perform kernel boot and disk image boot scenarios.
>> For all of them recording and replaying phases are executed.
>> Tests were borrowed from existing boot_linux*.py tests. But some
>> of the platforms and images were excluded, because icount for them
>> still has some issues.
>>
>> Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
>>
>> v3 changes:
>>  - Added record/replay logging (suggested by Philippe Mathieu-Daudé)
>>  - Changed the console pattern to get longer kernel execution (suggested by Alex Bennée)
>>  - Coding style fixes
>>  - Other minor changes
>> v2 changes:
>>  - Some test structure refactoring (suggested by Willian Rampazzo)
>>
>> ---
>>
>> Pavel Dovgaluk (11):
>>       tests/acceptance: allow console interaction with specific VMs
>>       tests/acceptance: refactor boot_linux_console test to allow code reuse
>>       tests/acceptance: add base class record/replay kernel tests
>>       tests/acceptance: add kernel record/replay test for x86_64
>>       tests/acceptance: add record/replay test for aarch64
>>       tests/acceptance: add record/replay test for arm
>>       tests/acceptance: add record/replay test for ppc64
>>       tests/acceptance: add record/replay test for m68k

You forgot to Cc the maintainers for each target/machine tested.

Anyway this series has been on the list for 3 weeks, so I'm
queuing it for next acceptance-testing pull request.

Since the x86 tests generate some timeout errors on Travis-CI,
I squashed the following changes:

-- >8 --
diff --git a/tests/acceptance/replay_kernel.py
b/tests/acceptance/replay_kernel.py
index a25eb3d3ee..8fa6b517c5 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -12,6 +12,7 @@
 import logging
 import time

+from avocado import skipIf
 from avocado_qemu import wait_for_console_pattern
 from avocado.utils import archive
 from avocado.utils import process
@@ -72,6 +73,7 @@ def run_rr(self, kernel_path, kernel_command_line,
console_pattern,
         logger = logging.getLogger('replay')
         logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))

+    @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
     def test_x86_64_pc(self):
         """
         :avocado: tags=arch:x86_64
diff --git a/tests/acceptance/replay_linux.py
b/tests/acceptance/replay_linux.py
index 472ab730cd..8e43e282a7 100644
--- a/tests/acceptance/replay_linux.py
+++ b/tests/acceptance/replay_linux.py
@@ -12,6 +12,7 @@
 import logging
 import time

+from avocado import skipIf
 from avocado.utils import cloudinit
 from avocado.utils import datadrainer
 from avocado.utils import network
@@ -92,6 +93,7 @@ def run_rr(self, args=None, shift=7):
         logger = logging.getLogger('replay')
         logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1))

+@skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI')
 class ReplayLinuxX8664(ReplayLinux):
     """
     :avocado: tags=arch:x86_64
---

Regards,

Phil.

>>       tests/acceptance: record/replay tests with advcal images
>>       tests/acceptance: refactor boot_linux to allow code reuse
>>       tests/acceptance: Linux boot test for record/replay
> 
> Thanks, patches 1/2/10 applied to my python-next tree:
> https://gitlab.com/philmd/qemu/commits/python-next
> 
> - tests/acceptance: allow console interaction with specific VMs
> - tests/acceptance: refactor boot_linux_console test to allow code reuse
> - tests/acceptance: refactor boot_linux to allow code reuse
>