Re: [PATCH v2 03/11] tests/acceptance: add base class record/replay kernel tests

["Alex Bennée" <alex.bennee@linaro.org>]

Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:

> On 27.05.2020 18:20, Alex Bennée wrote:
>> Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
>>
>>> This patch adds a base for testing kernel boot recording and replaying.
>>> Each test has the phase of recording and phase of replaying.
>>> Virtual machines just boot the kernel and do not interact with
>>> the network.
>>> Structure and image links for the tests are borrowed from boot_linux_console.py
>>> Testing controls the message pattern at the end of the kernel
>>> boot for both record and replay modes. In replay mode QEMU is also
>>> intended to finish the execution automatically.
>>>
>>> Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 47ef3139e6..e9a9ce4f66 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -2497,6 +2497,7 @@ F: net/filter-replay.c
>>   F: include/sysemu/replay.h
>>   F: docs/replay.txt
>>   F: stubs/replay.c
>> +F: tests/acceptance/replay_kernel.py
>>     IOVA Tree
>>   M: Peter Xu <peterx@redhat.com>
>> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
>> new file mode 100644
>> index 0000000000..b8b277ad2f
>> --- /dev/null
>> +++ b/tests/acceptance/replay_kernel.py
>> @@ -0,0 +1,57 @@
>> +# Record/replay test that boots a Linux kernel
>> +#
>> +# Copyright (c) 2020 ISP RAS
>> +#
>> +# Author:
>> +#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later.  See the COPYING file in the top-level directory.
>> +
>> +import os
>> +import gzip
>>
>> Do we actually use gzip in this test?
>
> Removed that, thanks.
>
>>
>> +
>> +from avocado_qemu import wait_for_console_pattern
>> +from avocado.utils import process
>> +from avocado.utils import archive
>> +from boot_linux_console import LinuxKernelUtils
>> +
>> +class ReplayKernel(LinuxKernelUtils):
>> +    """
>> +    Boots a Linux kernel in record mode and checks that the console
>> +    is operational and the kernel command line is properly passed
>> +    from QEMU to the kernel.
>> +    Then replays the same scenario and verifies, that QEMU correctly
>> +    terminates.
>>
>> Shouldn't we be doing more to verify the replay behaved the same as the
>> recorded session? What happens if things go wrong? Does QEMU barf out or
>> just deviate from the previous run?
>
> We hardly can compare vCPU states during record and replay.
>
> But in the most cases it is not needed. When control flow goes in the
> wrong direction, it affects the interrupts and exceptions.
>
> And interrupts and exceptions are the synchronization points in the
> replay log. Therefore when the executions differ, QEMU replay just
> hangs.

Maybe we should fix that and exit with a more definitive error? Hangs
are just plain ugly to debug because your first step has to be to start
poking around with a debugger.

-- 
Alex Bennée