[PATCH v3 0/4] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Gustavo Romero <gustavo.romero@linaro.org>]

tests/functional: Adapt reverse_debugging to run w/o Avocado

The goal of this series is to remove Avocado as a dependency for running
the reverse_debugging functional test.

After several rounds of discussions about v1 and v2, and experiments
done by Daniel and Thomas (thanks for all the experiments and comments
so far), I've taken a new approach and moved away from using a runner
for GDB. The changes, I believe, are much simpler now.

This new series uses GDB's machine interface (MI) via the pygdbmi module
(thanks Manos and Peter for the inputs). pygdbmi provides a controller
to start GDB and communicate with it through MI, so there is no longer a
risk of version clashes between libpython in GDB and Python modules in
the pyvenv, as it could, in theory, happen when GDB executes the test
script via -x option.

Also, as Daniel pointed out, the overall test output is pretty bad and
currently does not allow one to easily follow the sequence of GDB
commands used in the test. I took this opportunity to improve the output
and it now prints the sequence in a format that can be copied and pasted
directly into GDB. For instance, the test output for STEPS = 4 is:

TAP version 13
# $ set debug remote 1
# $ target remote localhost:62464
# Remote debugging using localhost:62464
# 0x0000000040000000 in ?? ()
# $ set debug remote 0
# $ print $pc
# $1 = (void (*)()) 0x40000000
# $ stepi
# 0x0000000040000004 in ?? ()
# $ print $pc
# $2 = (void (*)()) 0x40000004
# $ stepi
# 0x0000000040000008 in ?? ()
# $ print $pc
# $3 = (void (*)()) 0x40000008
# $ stepi
# 0x000000004000000c in ?? ()
# $ print $pc
# $4 = (void (*)()) 0x4000000c
# $ stepi
# 0x0000000040000010 in ?? ()
# $ reverse-stepi
# 0x000000004000000c in ?? ()
# $ print $pc
# $5 = (void (*)()) 0x4000000c
# $ reverse-stepi
# 0x0000000040000008 in ?? ()
# $ print $pc
# $6 = (void (*)()) 0x40000008
# $ reverse-stepi
# 0x0000000040000004 in ?? ()
# $ print $pc
# $7 = (void (*)()) 0x40000004
# $ reverse-stepi
# 0x0000000040000000 in ?? ()
# $ print $pc
# $8 = (void (*)()) 0x40000000
# $ print $pc
# $9 = (void (*)()) 0x40000000
# $ stepi
# 0x0000000040000004 in ?? ()
# $ print $pc
# $10 = (void (*)()) 0x40000004
# $ stepi
# 0x0000000040000008 in ?? ()
# $ print $pc
# $11 = (void (*)()) 0x40000008
# $ stepi
# 0x000000004000000c in ?? ()
# $ print $pc
# $12 = (void (*)()) 0x4000000c
# $ stepi
# 0x0000000040000010 in ?? ()
# $ break *0x40000000
# Breakpoint 1 at 0x40000000
# $ break *0x40000004
# Breakpoint 2 at 0x40000004
# $ break *0x40000008
# Breakpoint 3 at 0x40000008
# $ break *0x4000000c
# Breakpoint 4 at 0x4000000c
# $ continue
# Continuing.
# Program received signal SIGINT, Interrupt.
# 0xffff45a2feaa2050 in ?? ()
# $ print $pc
# $13 = (void (*)()) 0xffff45a2feaa2050
# **** Hit replay-break at icount=3691561, pc=0xffff45a2feaa2050 ****
# $ reverse-continue
# Continuing.
# Breakpoint 4, 0x000000004000000c in ?? ()
# $ print $pc
# $14 = (void (*)()) 0x4000000c
# **** Hit breakpoint at the first PC in reverse order (0x4000000c) ****
ok 1 test_reverse_debug.ReverseDebugging_AArch64.test_aarch64_virt
1..1

As can be observed above, the TAP protocol is respected, and Meson
correctly displays GDB's test output in testlog-thorough.txt.

Because the pygdbmi "shim" is so thin, I had to write a trivial GDB
class around it to easily capture and print the payloads returned by its
write() method. The GDB class allows clean, single-line commands to be
used in the tests through method chaining, making them easier to follow,
for example:

pc = gdb.cli("print $pc").get_add()

The test is kept “skipped” for aarch64, ppc64, and x86_64, so it is
necessary to set QEMU_TEST_FLAKY_TESTS=1 in the test environment to
effectively run the test on these archs.

On aarch64, the test is flaky, but there is a fix that I’ve tested while
writing this series [0] that resolves it. On ppc64 and x86_64, the test
always fails: on ppc64, GDB gets a bogus PC, and on x86_64, the last
part of the test (reverse-continue) does not hit the last executed PC
(as it should) but instead jumps to the beginning of the code (first PC
in forward order).

Thus, to effectively run the reverse_debugging test on aarch64:

$ export QEMU_TEST_FLAKY_TESTS=1
$ make check-functional

or even, to run only the reverse_debug test:
$ ./pyvenv/bin/meson test --verbose --no-rebuild -t 1 --setup thorough --suite func-thorough func-aarch64-reverse_debug


Cheers,
Gustavo

v1:
https://patchew.org/QEMU/20250819143916.4138035-1-gustavo.romero@linaro.org/

v2:
https://patchew.org/QEMU/20250904154640.52687-1-gustavo.romero@linaro.org/

v3:
- Use pygdbmi instead of run-test.py or any other GDB runner
- No changes in meson.build, except to set QEMU_TEST_GDB in the test env. 
- Improved test output to show all GDB commands used in the test


Gustavo Romero (4):
  python: Install pygdbmi in venv
  tests/functional: Provide GDB to the functional tests
  tests/functional: Adapt reverse_debugging to run w/o Avocado
  tests/functional: Adapt arches to reverse_debugging w/o Avocado

 configure                                     |   2 +
 meson_options.txt                             |   2 +
 .../wheels/pygdbmi-0.11.0.0-py3-none-any.whl  | Bin 0 -> 21258 bytes
 pythondeps.toml                               |   1 +
 scripts/meson-buildoptions.sh                 |   2 +
 .../functional/aarch64/test_reverse_debug.py  |  13 +-
 tests/functional/meson.build                  |   6 +
 tests/functional/ppc64/test_reverse_debug.py  |  15 +-
 tests/functional/reverse_debugging.py         | 308 +++++++++++-------
 tests/functional/x86_64/test_reverse_debug.py |  19 +-
 10 files changed, 217 insertions(+), 151 deletions(-)
 create mode 100644 python/wheels/pygdbmi-0.11.0.0-py3-none-any.whl

-- 
2.34.1