[PATCH v5 0/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[PATCH v5 0/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Gustavo Romero]

The goal of this series is to remove Avocado as a dependency for running
the reverse_debugging functional test.

After several rounds of discussions about v1 and v2, and experiments
done by Daniel and Thomas (thanks for all the experiments and comments
so far), I've taken a new approach and moved away from using a runner
for GDB. The changes, I believe, are much simpler now.

This new series uses GDB's machine interface (MI) via the pygdbmi module
(thanks Manos and Peter for the inputs). pygdbmi provides a controller
to start GDB and communicate with it through MI, so there is no longer a
risk of version clashes between libpython in GDB and Python modules in
the pyvenv, as it could, in theory, happen when GDB executes the test
script via -x option.

Also, as Daniel pointed out, the overall test output is pretty bad and
currently does not allow one to easily follow the sequence of GDB
commands used in the test. I took this opportunity to improve the output
and it now prints the sequence in a format that can be copied and pasted
directly into GDB.

The TAP protocol is respected, and Meson correctly displays GDB's test
output in testlog-thorough.txt.

Because the pygdbmi "shim" is so thin, I had to write a trivial GDB
class around it to easily capture and print the payloads returned by its
write() method. The GDB class allows clean, single-line commands to be
used in the tests through method chaining, making them easier to follow,
for example:

pc = gdb.cli("print $pc").get_add()

The test is kept “skipped” for aarch64, ppc64, and x86_64, so it is
necessary to set QEMU_TEST_FLAKY_TESTS=1 in the test environment to
effectively run the test on these archs.

On aarch64, the test is flaky, but there is a fix that I’ve tested while
writing this series [0] that resolves it. On ppc64 and x86_64, the test
always fails: on ppc64, GDB gets a bogus PC, and on x86_64, the last
part of the test (reverse-continue) does not hit the last executed PC
(as it should happen) but instead jumps to the beginning of the code
(first PC in forward order).

Thus, to effectively run the reverse_debugging test on aarch64:

$ export QEMU_TEST_FLAKY_TESTS=1
$ make check-functional

or:
$ make check-functional-aarch64

or even, to run only the reverse_debug test after 'make check-functional':
$ ./pyvenv/bin/meson test --verbose --no-rebuild -t 1 --setup thorough --suite func-thorough func-aarch64-reverse_debug


Cheers,
Gustavo

v1:
https://patchew.org/QEMU/20250819143916.4138035-1-gustavo.romero@linaro.org/

v2:
https://patchew.org/QEMU/20250904154640.52687-1-gustavo.romero@linaro.org/

v3:
https://patchew.org/QEMU/20250922054351.14289-1-gustavo.romero@linaro.org/

v4:
https://patchew.org/QEMU/20250926051542.104432-1-gustavo.romero@linaro.org/

v5:
- Fixed installation of "pygdbmi" in the test env for
  "make check-functional-<ARCH>" (thuth)
- Increased GDB command timeout from 4 to 32 seconds (thuth)
- Reverted use of get_pc() in reverse_debugging.py (thuth)
- Removed $gdb_arches from configure and checked if GDB supports the
  arch to be tested in reverse_debugging.py instead. If the arch is not
  supported by GDB the test is skipped (thuth)
- Corrected grammar in GDB timeout message (thuth)
- Skipped test if "pygdbmi" is not installed in the test environment
  (thuth, danpb)
- Improved error handling in try: block when GDB commands fail, time out,
  or when the test fails, so there is no "double exception"


Daniel P. Berrangé (2):
  tests/functional: replace avocado process with subprocess
  tests/functional: drop datadrainer class in reverse debugging

Gustavo Romero (7):
  tests/functional: Re-activate the check-venv target
  python: Install pygdbmi in meson's venv
  tests/functional: Provide GDB to the functional tests
  tests/functional: Add GDB class
  tests/functional: Add decorator to skip test on missing env vars
  tests/functional: Adapt reverse_debugging to run w/o Avocado
  tests/functional: Adapt arches to reverse_debugging w/o Avocado

 configure                                     |   2 +
 meson_options.txt                             |   2 +
 pythondeps.toml                               |   1 +
 scripts/meson-buildoptions.sh                 |   2 +
 tests/Makefile.include                        |   4 +-
 .../functional/aarch64/test_reverse_debug.py  |  11 +-
 tests/functional/meson.build                  |   6 +
 tests/functional/ppc64/test_reverse_debug.py  |  15 +-
 tests/functional/qemu_test/__init__.py        |   4 +-
 tests/functional/qemu_test/decorators.py      |  18 +++
 tests/functional/qemu_test/gdb.py             |  88 ++++++++++
 tests/functional/reverse_debugging.py         | 151 +++++++++---------
 tests/functional/x86_64/test_reverse_debug.py |  15 +-
 13 files changed, 218 insertions(+), 101 deletions(-)
 create mode 100644 tests/functional/qemu_test/gdb.py

-- 
2.34.1

[PATCH v5 1/9] tests/functional: Re-activate the check-venv target

[Gustavo Romero]

Add check-venv target as a dependency for the functional tests. This
causes Python modules listed in pythondeps.toml, under the testdeps
group, to be installed when 'make check-functional{-<ARCH>}' is executed
to prepare and run the functional tests.

Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
 tests/Makefile.include | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 62a4fc8ed3..e47ef4d45c 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -105,11 +105,11 @@ check-venv: $(TESTS_VENV_TOKEN)
 
 FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS)))
 .PHONY: $(FUNCTIONAL_TARGETS)
-$(FUNCTIONAL_TARGETS):
+$(FUNCTIONAL_TARGETS): check-venv
 	@$(MAKE) SPEED=thorough $(subst -functional,-func,$@)
 
 .PHONY: check-functional
-check-functional:
+check-functional: check-venv
 	@$(NINJA) precache-functional
 	@QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func check-func-quick
 
-- 
2.34.1

Re: [PATCH v5 1/9] tests/functional: Re-activate the check-venv target

[Daniel P. Berrangé]

On Thu, Oct 02, 2025 at 02:04:24AM +0000, Gustavo Romero wrote:
> Add check-venv target as a dependency for the functional tests. This
> causes Python modules listed in pythondeps.toml, under the testdeps
> group, to be installed when 'make check-functional{-<ARCH>}' is executed
> to prepare and run the functional tests.
> 
> Suggested-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
>  tests/Makefile.include | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v5 1/9] tests/functional: Re-activate the check-venv target

[Thomas Huth]

On 02/10/2025 04.04, Gustavo Romero wrote:
> Add check-venv target as a dependency for the functional tests. This
> causes Python modules listed in pythondeps.toml, under the testdeps
> group, to be installed when 'make check-functional{-<ARCH>}' is executed
> to prepare and run the functional tests.
> 
> Suggested-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
>   tests/Makefile.include | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 62a4fc8ed3..e47ef4d45c 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -105,11 +105,11 @@ check-venv: $(TESTS_VENV_TOKEN)
>   
>   FUNCTIONAL_TARGETS=$(patsubst %-softmmu,check-functional-%, $(filter %-softmmu,$(TARGETS)))
>   .PHONY: $(FUNCTIONAL_TARGETS)
> -$(FUNCTIONAL_TARGETS):
> +$(FUNCTIONAL_TARGETS): check-venv
>   	@$(MAKE) SPEED=thorough $(subst -functional,-func,$@)
>   
>   .PHONY: check-functional
> -check-functional:
> +check-functional: check-venv
>   	@$(NINJA) precache-functional
>   	@QEMU_TEST_NO_DOWNLOAD=1 $(MAKE) SPEED=thorough check-func check-func-quick
>   

Reviewed-by: Thomas Huth <thuth@redhat.com>

[PATCH v5 2/9] python: Install pygdbmi in meson's venv

[Gustavo Romero]

The upcoming changes in the reverse_debugging functional test to remove
Avocado as a dependency will require pygdbmi for interacting with GDB,
so install it in meson's venv (located in the build dir's pyvenv/).

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
 pythondeps.toml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/pythondeps.toml b/pythondeps.toml
index 16fb2a989c..98e99e7900 100644
--- a/pythondeps.toml
+++ b/pythondeps.toml
@@ -33,3 +33,4 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" }
 
 [testdeps]
 qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" }
+pygdbmi = { accepted = ">=0.11.0.0", installed = "0.11.0.0" }
-- 
2.34.1

Re: [PATCH v5 2/9] python: Install pygdbmi in meson's venv

[Daniel P. Berrangé]

On Thu, Oct 02, 2025 at 02:04:25AM +0000, Gustavo Romero wrote:
> The upcoming changes in the reverse_debugging functional test to remove
> Avocado as a dependency will require pygdbmi for interacting with GDB,
> so install it in meson's venv (located in the build dir's pyvenv/).
> 
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
>  pythondeps.toml | 1 +
>  1 file changed, 1 insertion(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v5 2/9] python: Install pygdbmi in meson's venv

[John Snow]

On Wed, Oct 1, 2025 at 10:09 PM Gustavo Romero
<gustavo.romero@linaro.org> wrote:
>
> The upcoming changes in the reverse_debugging functional test to remove
> Avocado as a dependency will require pygdbmi for interacting with GDB,
> so install it in meson's venv (located in the build dir's pyvenv/).
>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
>  pythondeps.toml | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/pythondeps.toml b/pythondeps.toml
> index 16fb2a989c..98e99e7900 100644
> --- a/pythondeps.toml
> +++ b/pythondeps.toml
> @@ -33,3 +33,4 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" }
>
>  [testdeps]
>  qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" }
> +pygdbmi = { accepted = ">=0.11.0.0", installed = "0.11.0.0" }
> --
> 2.34.1
>

Do we need to vendor pygdbmi? Currently, mkvenv does not consult
online sources and so if this package is missing, pulling in testdeps
will fail.

We can either:

(1) vendor pygdbmi, or
(2) change mkvenv arguments for the testdeps group specifically to
allow it to download missing deps.

--js

Re: [PATCH v5 2/9] python: Install pygdbmi in meson's venv

[Thomas Huth]

On 14/10/2025 19.39, John Snow wrote:
> On Wed, Oct 1, 2025 at 10:09 PM Gustavo Romero
> <gustavo.romero@linaro.org> wrote:
>>
>> The upcoming changes in the reverse_debugging functional test to remove
>> Avocado as a dependency will require pygdbmi for interacting with GDB,
>> so install it in meson's venv (located in the build dir's pyvenv/).
>>
>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
>> ---
>>   pythondeps.toml | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/pythondeps.toml b/pythondeps.toml
>> index 16fb2a989c..98e99e7900 100644
>> --- a/pythondeps.toml
>> +++ b/pythondeps.toml
>> @@ -33,3 +33,4 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" }
>>
>>   [testdeps]
>>   qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" }
>> +pygdbmi = { accepted = ">=0.11.0.0", installed = "0.11.0.0" }
>> --
>> 2.34.1
>>
> 
> Do we need to vendor pygdbmi? Currently, mkvenv does not consult
> online sources and so if this package is missing, pulling in testdeps
> will fail.

This is handled by the "check-venv" target which is called now when you do a 
"make check-functional". See:

  https://gitlab.com/qemu-project/qemu/-/commit/75b1786996c422878d09bd12f166004a7d32e459

Works fine for me, did you hit any issues here?

  Thomas

Re: [PATCH v5 2/9] python: Install pygdbmi in meson's venv

[John Snow]

On Tue, Oct 14, 2025 at 2:21 PM Thomas Huth <thuth@redhat.com> wrote:
>
> On 14/10/2025 19.39, John Snow wrote:
> > On Wed, Oct 1, 2025 at 10:09 PM Gustavo Romero
> > <gustavo.romero@linaro.org> wrote:
> >>
> >> The upcoming changes in the reverse_debugging functional test to remove
> >> Avocado as a dependency will require pygdbmi for interacting with GDB,
> >> so install it in meson's venv (located in the build dir's pyvenv/).
> >>
> >> Reviewed-by: Thomas Huth <thuth@redhat.com>
> >> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> >> ---
> >>   pythondeps.toml | 1 +
> >>   1 file changed, 1 insertion(+)
> >>
> >> diff --git a/pythondeps.toml b/pythondeps.toml
> >> index 16fb2a989c..98e99e7900 100644
> >> --- a/pythondeps.toml
> >> +++ b/pythondeps.toml
> >> @@ -33,3 +33,4 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" }
> >>
> >>   [testdeps]
> >>   qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" }
> >> +pygdbmi = { accepted = ">=0.11.0.0", installed = "0.11.0.0" }
> >> --
> >> 2.34.1
> >>
> >
> > Do we need to vendor pygdbmi? Currently, mkvenv does not consult
> > online sources and so if this package is missing, pulling in testdeps
> > will fail.
>
> This is handled by the "check-venv" target which is called now when you do a
> "make check-functional". See:
>
>   https://gitlab.com/qemu-project/qemu/-/commit/75b1786996c422878d09bd12f166004a7d32e459
>
> Works fine for me, did you hit any issues here?
>
>   Thomas

Ah-ha. I am working on dropping qemu.qmp from the tree and had moved
this machinery into mkvenv.py - I didn't realize that the Makefile
invocation here passes `--online`. The core groups for mkvenv still
operate offline only, I didn't realize we let it go online for tests.

Before these dependencies get more numerous: do we accept that "make
check" might only work if you are online, or do we want to enforce the
idea that "make check" should always work offline only?

(like "make check" works offline and "make check-extras" may require
internet for deps?)

--js

Re: [PATCH v5 2/9] python: Install pygdbmi in meson's venv

[Thomas Huth]

On 14/10/2025 20.55, John Snow wrote:
> On Tue, Oct 14, 2025 at 2:21 PM Thomas Huth <thuth@redhat.com> wrote:
>>
>> On 14/10/2025 19.39, John Snow wrote:
>>> On Wed, Oct 1, 2025 at 10:09 PM Gustavo Romero
>>> <gustavo.romero@linaro.org> wrote:
>>>>
>>>> The upcoming changes in the reverse_debugging functional test to remove
>>>> Avocado as a dependency will require pygdbmi for interacting with GDB,
>>>> so install it in meson's venv (located in the build dir's pyvenv/).
>>>>
>>>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>>>> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
>>>> ---
>>>>    pythondeps.toml | 1 +
>>>>    1 file changed, 1 insertion(+)
>>>>
>>>> diff --git a/pythondeps.toml b/pythondeps.toml
>>>> index 16fb2a989c..98e99e7900 100644
>>>> --- a/pythondeps.toml
>>>> +++ b/pythondeps.toml
>>>> @@ -33,3 +33,4 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" }
>>>>
>>>>    [testdeps]
>>>>    qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" }
>>>> +pygdbmi = { accepted = ">=0.11.0.0", installed = "0.11.0.0" }
>>>> --
>>>> 2.34.1
>>>>
>>>
>>> Do we need to vendor pygdbmi? Currently, mkvenv does not consult
>>> online sources and so if this package is missing, pulling in testdeps
>>> will fail.
>>
>> This is handled by the "check-venv" target which is called now when you do a
>> "make check-functional". See:
>>
>>    https://gitlab.com/qemu-project/qemu/-/commit/75b1786996c422878d09bd12f166004a7d32e459
>>
>> Works fine for me, did you hit any issues here?
>>
>>    Thomas
> 
> Ah-ha. I am working on dropping qemu.qmp from the tree and had moved
> this machinery into mkvenv.py - I didn't realize that the Makefile
> invocation here passes `--online`. The core groups for mkvenv still
> operate offline only, I didn't realize we let it go online for tests.
> 
> Before these dependencies get more numerous: do we accept that "make
> check" might only work if you are online, or do we want to enforce the
> idea that "make check" should always work offline only?
> 
> (like "make check" works offline and "make check-extras" may require
> internet for deps?)

"make check" works offline (it does not include "check-functional"). It's 
only "make check-functional" that requires an internet connection. It 
required this for downloading the assets for the functional tests already, 
so adding "check-venv" here did not hurt.

  Thomas

Re: [PATCH v5 2/9] python: Install pygdbmi in meson's venv

[John Snow]

On Tue, Oct 14, 2025 at 2:59 PM Thomas Huth <thuth@redhat.com> wrote:
>
> On 14/10/2025 20.55, John Snow wrote:
> > On Tue, Oct 14, 2025 at 2:21 PM Thomas Huth <thuth@redhat.com> wrote:
> >>
> >> On 14/10/2025 19.39, John Snow wrote:
> >>> On Wed, Oct 1, 2025 at 10:09 PM Gustavo Romero
> >>> <gustavo.romero@linaro.org> wrote:
> >>>>
> >>>> The upcoming changes in the reverse_debugging functional test to remove
> >>>> Avocado as a dependency will require pygdbmi for interacting with GDB,
> >>>> so install it in meson's venv (located in the build dir's pyvenv/).
> >>>>
> >>>> Reviewed-by: Thomas Huth <thuth@redhat.com>
> >>>> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> >>>> ---
> >>>>    pythondeps.toml | 1 +
> >>>>    1 file changed, 1 insertion(+)
> >>>>
> >>>> diff --git a/pythondeps.toml b/pythondeps.toml
> >>>> index 16fb2a989c..98e99e7900 100644
> >>>> --- a/pythondeps.toml
> >>>> +++ b/pythondeps.toml
> >>>> @@ -33,3 +33,4 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" }
> >>>>
> >>>>    [testdeps]
> >>>>    qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" }
> >>>> +pygdbmi = { accepted = ">=0.11.0.0", installed = "0.11.0.0" }
> >>>> --
> >>>> 2.34.1
> >>>>
> >>>
> >>> Do we need to vendor pygdbmi? Currently, mkvenv does not consult
> >>> online sources and so if this package is missing, pulling in testdeps
> >>> will fail.
> >>
> >> This is handled by the "check-venv" target which is called now when you do a
> >> "make check-functional". See:
> >>
> >>    https://gitlab.com/qemu-project/qemu/-/commit/75b1786996c422878d09bd12f166004a7d32e459
> >>
> >> Works fine for me, did you hit any issues here?
> >>
> >>    Thomas
> >
> > Ah-ha. I am working on dropping qemu.qmp from the tree and had moved
> > this machinery into mkvenv.py - I didn't realize that the Makefile
> > invocation here passes `--online`. The core groups for mkvenv still
> > operate offline only, I didn't realize we let it go online for tests.
> >
> > Before these dependencies get more numerous: do we accept that "make
> > check" might only work if you are online, or do we want to enforce the
> > idea that "make check" should always work offline only?
> >
> > (like "make check" works offline and "make check-extras" may require
> > internet for deps?)
>
> "make check" works offline (it does not include "check-functional"). It's
> only "make check-functional" that requires an internet connection. It
> required this for downloading the assets for the functional tests already,
> so adding "check-venv" here did not hurt.
>
>   Thomas

OK, got it. For *my* purposes (and this has nothing to do with this
patch anymore, I see that it's fine as-is) I need to pull qemu.qmp
into the testing venv /offline/ for the default "make check"
invocation and I intended to vendor qemu.qmp to accomplish that.

With this dep here, installing "testdeps" offline now fails, so either:

(1) I vendor pygdbmi so it can be installed offline, or
(2) I split the test deps into two groups; offline and online.

Re: [PATCH v5 2/9] python: Install pygdbmi in meson's venv

[Thomas Huth]

On 14/10/2025 21.05, John Snow wrote:
> On Tue, Oct 14, 2025 at 2:59 PM Thomas Huth <thuth@redhat.com> wrote:
>>
>> On 14/10/2025 20.55, John Snow wrote:
>>> On Tue, Oct 14, 2025 at 2:21 PM Thomas Huth <thuth@redhat.com> wrote:
>>>>
>>>> On 14/10/2025 19.39, John Snow wrote:
>>>>> On Wed, Oct 1, 2025 at 10:09 PM Gustavo Romero
>>>>> <gustavo.romero@linaro.org> wrote:
>>>>>>
>>>>>> The upcoming changes in the reverse_debugging functional test to remove
>>>>>> Avocado as a dependency will require pygdbmi for interacting with GDB,
>>>>>> so install it in meson's venv (located in the build dir's pyvenv/).
>>>>>>
>>>>>> Reviewed-by: Thomas Huth <thuth@redhat.com>
>>>>>> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
>>>>>> ---
>>>>>>     pythondeps.toml | 1 +
>>>>>>     1 file changed, 1 insertion(+)
>>>>>>
>>>>>> diff --git a/pythondeps.toml b/pythondeps.toml
>>>>>> index 16fb2a989c..98e99e7900 100644
>>>>>> --- a/pythondeps.toml
>>>>>> +++ b/pythondeps.toml
>>>>>> @@ -33,3 +33,4 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" }
>>>>>>
>>>>>>     [testdeps]
>>>>>>     qemu.qmp = { accepted = ">=0.0.3", installed = "0.0.3" }
>>>>>> +pygdbmi = { accepted = ">=0.11.0.0", installed = "0.11.0.0" }
>>>>>> --
>>>>>> 2.34.1
>>>>>>
>>>>>
>>>>> Do we need to vendor pygdbmi? Currently, mkvenv does not consult
>>>>> online sources and so if this package is missing, pulling in testdeps
>>>>> will fail.
>>>>
>>>> This is handled by the "check-venv" target which is called now when you do a
>>>> "make check-functional". See:
>>>>
>>>>     https://gitlab.com/qemu-project/qemu/-/commit/75b1786996c422878d09bd12f166004a7d32e459
>>>>
>>>> Works fine for me, did you hit any issues here?
>>>>
>>>>     Thomas
>>>
>>> Ah-ha. I am working on dropping qemu.qmp from the tree and had moved
>>> this machinery into mkvenv.py - I didn't realize that the Makefile
>>> invocation here passes `--online`. The core groups for mkvenv still
>>> operate offline only, I didn't realize we let it go online for tests.
>>>
>>> Before these dependencies get more numerous: do we accept that "make
>>> check" might only work if you are online, or do we want to enforce the
>>> idea that "make check" should always work offline only?
>>>
>>> (like "make check" works offline and "make check-extras" may require
>>> internet for deps?)
>>
>> "make check" works offline (it does not include "check-functional"). It's
>> only "make check-functional" that requires an internet connection. It
>> required this for downloading the assets for the functional tests already,
>> so adding "check-venv" here did not hurt.
>>
>>    Thomas
> 
> OK, got it. For *my* purposes (and this has nothing to do with this
> patch anymore, I see that it's fine as-is) I need to pull qemu.qmp
> into the testing venv /offline/ for the default "make check"
> invocation and I intended to vendor qemu.qmp to accomplish that.

Makes sense. I did the same for pycotap a year ago.

> With this dep here, installing "testdeps" offline now fails, so either:
> 
> (1) I vendor pygdbmi so it can be installed offline, or
> (2) I split the test deps into two groups; offline and online.

Please go with option (2). We likely should add more deps for the functional 
tests one day (e.g. a bunch of tests are using "numpy" and "cv2" - and 
currently these need to be installed in the host env manually), so adding 
all these via option (1) won't scale in the long run.

  Thomas

[PATCH v5 3/9] tests/functional: Provide GDB to the functional tests

[Gustavo Romero]

The probe of GDB is done in 'configure' and the full path is passed to
meson.build via the -Dgdb=option.

Meson then can pass the location of GDB to the functional tests via an
environment variable: QEMU_TEST_GDB.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
 configure                     | 2 ++
 meson_options.txt             | 2 ++
 scripts/meson-buildoptions.sh | 2 ++
 tests/functional/meson.build  | 6 ++++++
 4 files changed, 12 insertions(+)

diff --git a/configure b/configure
index 0f7eb95586..7226ddd589 100755
--- a/configure
+++ b/configure
@@ -1984,6 +1984,8 @@ if test "$skip_meson" = no; then
   test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE"
   test "$plugins" = yes && meson_option_add "-Dplugins=true"
   test "$tcg" != enabled && meson_option_add "-Dtcg=$tcg"
+  test -n "$gdb_bin" && meson_option_add "-Dgdb=$gdb_bin"
+
   run_meson() {
     NINJA=$ninja $meson setup "$@" "$PWD" "$source_path"
   }
diff --git a/meson_options.txt b/meson_options.txt
index fff1521e58..5bb41bcbc4 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -36,6 +36,8 @@ option('trace_file', type: 'string', value: 'trace',
 option('coroutine_backend', type: 'combo',
        choices: ['ucontext', 'sigaltstack', 'windows', 'wasm', 'auto'],
        value: 'auto', description: 'coroutine backend to use')
+option('gdb', type: 'string', value: '',
+       description: 'Path to GDB')
 
 # Everything else can be set via --enable/--disable-* option
 # on the configure script command line.  After adding an option
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 0ebe6bc52a..f4bd21220e 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -58,6 +58,7 @@ meson_options_help() {
   printf "%s\n" '  --enable-ubsan           enable undefined behaviour sanitizer'
   printf "%s\n" '  --firmwarepath=VALUES    search PATH for firmware files [share/qemu-'
   printf "%s\n" '                           firmware]'
+  printf "%s\n" '  --gdb=VALUE              Path to GDB'
   printf "%s\n" '  --iasl=VALUE             Path to ACPI disassembler'
   printf "%s\n" '  --includedir=VALUE       Header file directory [include]'
   printf "%s\n" '  --interp-prefix=VALUE    where to find shared libraries etc., use %M for'
@@ -323,6 +324,7 @@ _meson_option_parse() {
     --disable-fuzzing) printf "%s" -Dfuzzing=false ;;
     --enable-gcrypt) printf "%s" -Dgcrypt=enabled ;;
     --disable-gcrypt) printf "%s" -Dgcrypt=disabled ;;
+    --gdb=*) quote_sh "-Dgdb=$2" ;;
     --enable-gettext) printf "%s" -Dgettext=enabled ;;
     --disable-gettext) printf "%s" -Dgettext=disabled ;;
     --enable-gio) printf "%s" -Dgio=enabled ;;
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 2a0c5aa141..725630d308 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -77,6 +77,12 @@ foreach speed : ['quick', 'thorough']
     test_env.set('PYTHONPATH', meson.project_source_root() / 'python:' +
                                meson.current_source_dir())
 
+    # Define the GDB environment variable if gdb is available.
+    gdb = get_option('gdb')
+    if gdb != ''
+      test_env.set('QEMU_TEST_GDB', gdb)
+    endif
+
     foreach test : target_tests
       testname = '@0@-@1@'.format(target_base, test)
       if fs.exists('generic' / 'test_' + test + '.py')
-- 
2.34.1

Re: [PATCH v5 3/9] tests/functional: Provide GDB to the functional tests

[Daniel P. Berrangé]

On Thu, Oct 02, 2025 at 02:04:26AM +0000, Gustavo Romero wrote:
> The probe of GDB is done in 'configure' and the full path is passed to
> meson.build via the -Dgdb=option.
> 
> Meson then can pass the location of GDB to the functional tests via an
> environment variable: QEMU_TEST_GDB.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  configure                     | 2 ++
>  meson_options.txt             | 2 ++
>  scripts/meson-buildoptions.sh | 2 ++
>  tests/functional/meson.build  | 6 ++++++
>  4 files changed, 12 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH v5 4/9] tests/functional: Add GDB class

[Gustavo Romero]

Add GDB class, which provides methods to run GDB commands and capture
their output. The GDB class is a wrapper around the pygdbmi module and
interacts with GDB via GDB's machine interface (MI).

Acked-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
 tests/functional/qemu_test/__init__.py |  1 +
 tests/functional/qemu_test/gdb.py      | 88 ++++++++++++++++++++++++++
 2 files changed, 89 insertions(+)
 create mode 100644 tests/functional/qemu_test/gdb.py

diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
index 6e666a059f..60d19891bf 100644
--- a/tests/functional/qemu_test/__init__.py
+++ b/tests/functional/qemu_test/__init__.py
@@ -18,3 +18,4 @@
     skipIfMissingImports, skipIfOperatingSystem, skipLockedMemoryTest
 from .archive import archive_extract
 from .uncompress import uncompress
+from .gdb import GDB
diff --git a/tests/functional/qemu_test/gdb.py b/tests/functional/qemu_test/gdb.py
new file mode 100644
index 0000000000..05e4c29c2a
--- /dev/null
+++ b/tests/functional/qemu_test/gdb.py
@@ -0,0 +1,88 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+#
+# A simple interface module built around pygdbmi for handling GDB commands.
+#
+# Copyright (c) 2025 Linaro Limited
+#
+# Author:
+#  Gustavo Romero <gustavo.romero@linaro.org>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+import re
+
+
+class GDB:
+    """Provides methods to run and capture GDB command output."""
+
+
+    def __init__(self, gdb_path, echo=True, suffix='# ', prompt="$ "):
+        from pygdbmi.gdbcontroller import GdbController
+        from pygdbmi.constants import GdbTimeoutError
+        type(self).TimeoutError = GdbTimeoutError
+
+        gdb_cmd = [gdb_path, "-q", "--interpreter=mi2"]
+        self.gdbmi = GdbController(gdb_cmd)
+        self.echo = echo
+        self.suffix = suffix
+        self.prompt = prompt
+        self.response = None
+        self.cmd_output = None
+
+
+    def get_payload(self, response, kind):
+        output = []
+        for o in response:
+            # Unpack payloads of the same type.
+            _type, _, payload, *_ = o.values()
+            if _type == kind:
+                output += [payload]
+
+        # Some output lines do not end with \n but begin with it,
+        # so remove the leading \n and merge them with the next line
+        # that ends with \n.
+        lines = [line.lstrip('\n') for line in output]
+        lines = "".join(lines)
+        lines = lines.splitlines(keepends=True)
+
+        return lines
+
+
+    def cli(self, cmd, timeout=32.0):
+        self.response = self.gdbmi.write(cmd, timeout_sec=timeout)
+        self.cmd_output = self.get_payload(self.response, kind="console")
+        if self.echo:
+            print(self.suffix + self.prompt + cmd)
+
+            if len(self.cmd_output) > 0:
+                cmd_output = self.suffix.join(self.cmd_output)
+                print(self.suffix + cmd_output, end="")
+
+        return self
+
+
+    def get_addr(self):
+        address_pattern = r"0x[0-9A-Fa-f]+"
+        cmd_output = "".join(self.cmd_output) # Concat output lines.
+
+        match = re.search(address_pattern, cmd_output)
+
+        return int(match[0], 16) if match else None
+
+
+    def get_log(self):
+        r = self.get_payload(self.response, kind="log")
+        r = "".join(r)
+
+        return r
+
+
+    def get_console(self):
+        r = "".join(self.cmd_output)
+
+        return r
+
+
+    def exit(self):
+        self.gdbmi.exit()
-- 
2.34.1

Re: [PATCH v5 4/9] tests/functional: Add GDB class

[Daniel P. Berrangé]

On Thu, Oct 02, 2025 at 02:04:27AM +0000, Gustavo Romero wrote:
> Add GDB class, which provides methods to run GDB commands and capture
> their output. The GDB class is a wrapper around the pygdbmi module and
> interacts with GDB via GDB's machine interface (MI).
> 
> Acked-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
>  tests/functional/qemu_test/__init__.py |  1 +
>  tests/functional/qemu_test/gdb.py      | 88 ++++++++++++++++++++++++++
>  2 files changed, 89 insertions(+)
>  create mode 100644 tests/functional/qemu_test/gdb.py


> diff --git a/tests/functional/qemu_test/gdb.py b/tests/functional/qemu_test/gdb.py
> new file mode 100644
> index 0000000000..05e4c29c2a
> --- /dev/null
> +++ b/tests/functional/qemu_test/gdb.py
> @@ -0,0 +1,88 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +#
> +# A simple interface module built around pygdbmi for handling GDB commands.
> +#
> +# Copyright (c) 2025 Linaro Limited
> +#
> +# Author:
> +#  Gustavo Romero <gustavo.romero@linaro.org>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later.  See the COPYING file in the top-level directory.

Please remove these two lines - they should not be included in any
newly authored files - SPDX alone is desired.

If that is removed, then

 Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v5 4/9] tests/functional: Add GDB class

[Alex Bennée]

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Thu, Oct 02, 2025 at 02:04:27AM +0000, Gustavo Romero wrote:
>> Add GDB class, which provides methods to run GDB commands and capture
>> their output. The GDB class is a wrapper around the pygdbmi module and
>> interacts with GDB via GDB's machine interface (MI).
>> 
>> Acked-by: Thomas Huth <thuth@redhat.com>
>> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
>> ---
>>  tests/functional/qemu_test/__init__.py |  1 +
>>  tests/functional/qemu_test/gdb.py      | 88 ++++++++++++++++++++++++++
>>  2 files changed, 89 insertions(+)
>>  create mode 100644 tests/functional/qemu_test/gdb.py
>
>
>> diff --git a/tests/functional/qemu_test/gdb.py b/tests/functional/qemu_test/gdb.py
>> new file mode 100644
>> index 0000000000..05e4c29c2a
>> --- /dev/null
>> +++ b/tests/functional/qemu_test/gdb.py
>> @@ -0,0 +1,88 @@
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +#
>> +# A simple interface module built around pygdbmi for handling GDB commands.
>> +#
>> +# Copyright (c) 2025 Linaro Limited
>> +#
>> +# Author:
>> +#  Gustavo Romero <gustavo.romero@linaro.org>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later.  See the COPYING file in the top-level directory.
>
> Please remove these two lines - they should not be included in any
> newly authored files - SPDX alone is desired.

I can clean that up when I pull this to save another round trip from
Gustavo.

>
> If that is removed, then
>
>  Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
>
>
> With regards,
> Daniel

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

[PATCH v5 5/9] tests/functional: replace avocado process with subprocess

[Gustavo Romero]

From: Daniel P. Berrangé <berrange@redhat.com>

The standard python subprocess.check_call method is better than
avocado.utils.process as it doesn't require stuffing all args
into a single string.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/reverse_debugging.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/tests/functional/reverse_debugging.py b/tests/functional/reverse_debugging.py
index f9a1d395f1..a7ff47cb90 100644
--- a/tests/functional/reverse_debugging.py
+++ b/tests/functional/reverse_debugging.py
@@ -11,6 +11,7 @@
 # later.  See the COPYING file in the top-level directory.
 import os
 import logging
+from subprocess import check_output
 
 from qemu_test import LinuxKernelTest, get_qemu_img
 from qemu_test.ports import Ports
@@ -100,7 +101,6 @@ def vm_get_icount(vm):
 
     def reverse_debugging(self, shift=7, args=None):
         from avocado.utils import gdb
-        from avocado.utils import process
 
         logger = logging.getLogger('replay')
 
@@ -111,8 +111,9 @@ def reverse_debugging(self, shift=7, args=None):
         if qemu_img is None:
             self.skipTest('Could not find "qemu-img", which is required to '
                           'create the temporary qcow2 image')
-        cmd = '%s create -f qcow2 %s 128M' % (qemu_img, image_path)
-        process.run(cmd)
+        out = check_output([qemu_img, 'create', '-f', 'qcow2', image_path, '128M'],
+                           encoding='utf8')
+        logger.info("qemu-img: %s" % out)
 
         replay_path = os.path.join(self.workdir, 'replay.bin')
 
-- 
2.34.1

[PATCH v5 6/9] tests/functional: drop datadrainer class in reverse debugging

[Gustavo Romero]

From: Daniel P. Berrangé <berrange@redhat.com>

The reverse debugging test uses the avocado datadrainer class to
create a background thread that reads from the console socket and
dumps it via python logger.

Most tests log console output as a side effect of doing calls
to match strings, but this test never tries to match anything.

This isn't critical, so just drop the functionality.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/reverse_debugging.py | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/tests/functional/reverse_debugging.py b/tests/functional/reverse_debugging.py
index a7ff47cb90..7fd8c7607f 100644
--- a/tests/functional/reverse_debugging.py
+++ b/tests/functional/reverse_debugging.py
@@ -34,8 +34,6 @@ class ReverseDebugging(LinuxKernelTest):
     endian_is_le = True
 
     def run_vm(self, record, shift, args, replay_path, image_path, port):
-        from avocado.utils import datadrainer
-
         logger = logging.getLogger('replay')
         vm = self.get_vm(name='record' if record else 'replay')
         vm.set_console()
@@ -53,10 +51,6 @@ def run_vm(self, record, shift, args, replay_path, image_path, port):
         if args:
             vm.add_args(*args)
         vm.launch()
-        console_drainer = datadrainer.LineLogger(vm.console_socket.fileno(),
-                                    logger=self.log.getChild('console'),
-                                    stop_check=(lambda : not vm.is_running()))
-        console_drainer.start()
         return vm
 
     @staticmethod
-- 
2.34.1

[PATCH v5 7/9] tests/functional: Add decorator to skip test on missing env vars

[Gustavo Romero]

Add a decorator to skip tests on missing env variable(s). Multiple
variable names can be provided and if one or more of them are not set in
the test environment the test is skipped and the missing vars are
printed out.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
 tests/functional/qemu_test/__init__.py   |  3 ++-
 tests/functional/qemu_test/decorators.py | 18 ++++++++++++++++++
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/tests/functional/qemu_test/__init__.py b/tests/functional/qemu_test/__init__.py
index 60d19891bf..320193591b 100644
--- a/tests/functional/qemu_test/__init__.py
+++ b/tests/functional/qemu_test/__init__.py
@@ -15,7 +15,8 @@
 from .linuxkernel import LinuxKernelTest
 from .decorators import skipIfMissingCommands, skipIfNotMachine, \
     skipFlakyTest, skipUntrustedTest, skipBigDataTest, skipSlowTest, \
-    skipIfMissingImports, skipIfOperatingSystem, skipLockedMemoryTest
+    skipIfMissingImports, skipIfOperatingSystem, skipLockedMemoryTest, \
+    skipIfMissingEnv
 from .archive import archive_extract
 from .uncompress import uncompress
 from .gdb import GDB
diff --git a/tests/functional/qemu_test/decorators.py b/tests/functional/qemu_test/decorators.py
index c0d1567b14..b239295804 100644
--- a/tests/functional/qemu_test/decorators.py
+++ b/tests/functional/qemu_test/decorators.py
@@ -11,6 +11,24 @@
 from .cmd import which
 
 '''
+Decorator to skip execution of a test if the provided
+environment variables are not set.
+Example:
+
+  @skipIfMissingEnv("QEMU_ENV_VAR0", "QEMU_ENV_VAR1")
+'''
+def skipIfMissingEnv(*vars_):
+    missing_vars = []
+    for var in vars_:
+        if os.getenv(var) == None:
+            missing_vars.append(var)
+
+    has_vars = True if len(missing_vars) == 0 else False
+
+    return skipUnless(has_vars, f"Missing env var(s): {', '.join(missing_vars)}")
+
+'''
+
 Decorator to skip execution of a test if the list
 of command binaries is not available in $PATH.
 Example:
-- 
2.34.1

Re: [PATCH v5 7/9] tests/functional: Add decorator to skip test on missing env vars

[Daniel P. Berrangé]

On Thu, Oct 02, 2025 at 02:04:30AM +0000, Gustavo Romero wrote:
> Add a decorator to skip tests on missing env variable(s). Multiple
> variable names can be provided and if one or more of them are not set in
> the test environment the test is skipped and the missing vars are
> printed out.
> 
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
>  tests/functional/qemu_test/__init__.py   |  3 ++-
>  tests/functional/qemu_test/decorators.py | 18 ++++++++++++++++++
>  2 files changed, 20 insertions(+), 1 deletion(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH v5 8/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Gustavo Romero]

This commit removes Avocado as a dependency for running the
reverse_debugging test.

The main benefit, beyond eliminating an extra dependency, is that there
is no longer any need to handle GDB packets manually. This removes the
need for ad-hoc functions dealing with endianness and arch-specific
register numbers, making the test easier to read. The timeout variable
is also removed, since Meson now manages timeouts automatically.

reverse_debugging now uses the pygdbmi module to interact with GDB, if
it's available in the test environment, otherwise the test is skipped.
GDB is detect via the QEMU_TEST_GDB env. variable.

This commit also significantly improves the output for the test and
now prints all the GDB commands used in sequence. It also adds
some clarifications to existing comments, for example, clarifying that
once the replay-break is reached, a SIGINT is captured in GDB.

reverse_debugging is kept "skipped" for aarch64, ppc64, and x86_64, so
won't run unless QEMU_TEST_FLAKY_TESTS=1 is set in the test environment,
before running 'make check-functional' or 'meson test [...]'.

Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
 tests/functional/reverse_debugging.py | 138 ++++++++++++++------------
 1 file changed, 74 insertions(+), 64 deletions(-)

diff --git a/tests/functional/reverse_debugging.py b/tests/functional/reverse_debugging.py
index 7fd8c7607f..f06996089a 100644
--- a/tests/functional/reverse_debugging.py
+++ b/tests/functional/reverse_debugging.py
@@ -1,19 +1,22 @@
-# Reverse debugging test
-#
 # SPDX-License-Identifier: GPL-2.0-or-later
 #
+# Reverse debugging test
+#
 # Copyright (c) 2020 ISP RAS
+# Copyright (c) 2025 Linaro Limited
 #
 # Author:
 #  Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+#  Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
 #
 # This work is licensed under the terms of the GNU GPL, version 2 or
 # later.  See the COPYING file in the top-level directory.
-import os
+
 import logging
+import os
 from subprocess import check_output
 
-from qemu_test import LinuxKernelTest, get_qemu_img
+from qemu_test import LinuxKernelTest, get_qemu_img, GDB, skipIfMissingEnv, skipIfMissingImports
 from qemu_test.ports import Ports
 
 
@@ -29,9 +32,7 @@ class ReverseDebugging(LinuxKernelTest):
     that the execution is stopped at the last of them.
     """
 
-    timeout = 10
     STEPS = 10
-    endian_is_le = True
 
     def run_vm(self, record, shift, args, replay_path, image_path, port):
         logger = logging.getLogger('replay')
@@ -54,47 +55,16 @@ def run_vm(self, record, shift, args, replay_path, image_path, port):
         return vm
 
     @staticmethod
-    def get_reg_le(g, reg):
-        res = g.cmd(b'p%x' % reg)
-        num = 0
-        for i in range(len(res))[-2::-2]:
-            num = 0x100 * num + int(res[i:i + 2], 16)
-        return num
-
-    @staticmethod
-    def get_reg_be(g, reg):
-        res = g.cmd(b'p%x' % reg)
-        return int(res, 16)
-
-    def get_reg(self, g, reg):
-        # value may be encoded in BE or LE order
-        if self.endian_is_le:
-            return self.get_reg_le(g, reg)
-        else:
-            return self.get_reg_be(g, reg)
-
-    def get_pc(self, g):
-        return self.get_reg(g, self.REG_PC)
-
-    def check_pc(self, g, addr):
-        pc = self.get_pc(g)
-        if pc != addr:
-            self.fail('Invalid PC (read %x instead of %x)' % (pc, addr))
-
-    @staticmethod
-    def gdb_step(g):
-        g.cmd(b's', b'T05thread:01;')
-
-    @staticmethod
-    def gdb_bstep(g):
-        g.cmd(b'bs', b'T05thread:01;')
+    def get_pc(gdb: GDB):
+        return gdb.cli("print $pc").get_addr()
 
     @staticmethod
     def vm_get_icount(vm):
         return vm.qmp('query-replay')['return']['icount']
 
-    def reverse_debugging(self, shift=7, args=None):
-        from avocado.utils import gdb
+    @skipIfMissingImports("pygdbmi") # Required by GDB class
+    def reverse_debugging(self, gdb_arch, shift=7, args=None):
+        from qemu_test import GDB
 
         logger = logging.getLogger('replay')
 
@@ -124,68 +94,108 @@ def reverse_debugging(self, shift=7, args=None):
         with Ports() as ports:
             port = ports.find_free_port()
             vm = self.run_vm(False, shift, args, replay_path, image_path, port)
-        logger.info('connecting to gdbstub')
-        g = gdb.GDBRemote('127.0.0.1', port, False, False)
-        g.connect()
-        r = g.cmd(b'qSupported')
-        if b'qXfer:features:read+' in r:
-            g.cmd(b'qXfer:features:read:target.xml:0,ffb')
-        if b'ReverseStep+' not in r:
+
+        try:
+            logger.info('Connecting to gdbstub...')
+            self.reverse_debugging_run(vm, port, gdb_arch, last_icount)
+            logger.info('Test passed.')
+        except GDB.TimeoutError:
+            # Convert a GDB timeout exception into a unittest failure exception.
+            raise self.failureException("Timeout while connecting to or "
+                                        "communicating with gdbstub...") from None
+        except Exception:
+            # Re-throw exceptions from unittest, like the ones caused by fail(),
+            # skipTest(), etc.
+            raise
+
+    @skipIfMissingEnv("QEMU_TEST_GDB")
+    def reverse_debugging_run(self, vm, port, gdb_arch, last_icount):
+        logger = logging.getLogger('replay')
+
+        gdb_cmd = os.getenv('QEMU_TEST_GDB')
+        gdb = GDB(gdb_cmd)
+
+        r = gdb.cli("set architecture").get_log()
+        if gdb_arch not in r:
+            self.skipTest(f"GDB does not support arch '{gdb_arch}'")
+
+        gdb.cli("set debug remote 1")
+
+        c = gdb.cli(f"target remote localhost:{port}").get_console()
+        if not f"Remote debugging using localhost:{port}" in c:
+            self.fail("Could not connect to gdbstub!")
+
+        # Remote debug messages are in 'log' payloads.
+        r = gdb.get_log()
+        if 'ReverseStep+' not in r:
             self.fail('Reverse step is not supported by QEMU')
-        if b'ReverseContinue+' not in r:
+        if 'ReverseContinue+' not in r:
             self.fail('Reverse continue is not supported by QEMU')
 
+        gdb.cli("set debug remote 0")
+
         logger.info('stepping forward')
         steps = []
         # record first instruction addresses
         for _ in range(self.STEPS):
-            pc = self.get_pc(g)
+            pc = self.get_pc(gdb)
             logger.info('saving position %x' % pc)
             steps.append(pc)
-            self.gdb_step(g)
+            gdb.cli("stepi")
 
         # visit the recorded instruction in reverse order
         logger.info('stepping backward')
         for addr in steps[::-1]:
-            self.gdb_bstep(g)
-            self.check_pc(g, addr)
             logger.info('found position %x' % addr)
+            gdb.cli("reverse-stepi")
+            pc = self.get_pc(gdb)
+            if pc != addr:
+                logger.info('Invalid PC (read %x instead of %x)' % (pc, addr))
+                self.fail('Reverse stepping failed!')
 
         # visit the recorded instruction in forward order
         logger.info('stepping forward')
         for addr in steps:
-            self.check_pc(g, addr)
-            self.gdb_step(g)
             logger.info('found position %x' % addr)
+            pc = self.get_pc(gdb)
+            if pc != addr:
+                logger.info('Invalid PC (read %x instead of %x)' % (pc, addr))
+                self.fail('Forward stepping failed!')
+            gdb.cli("stepi")
 
         # set breakpoints for the instructions just stepped over
         logger.info('setting breakpoints')
         for addr in steps:
-            # hardware breakpoint at addr with len=1
-            g.cmd(b'Z1,%x,1' % addr, b'OK')
+            gdb.cli(f"break *{hex(addr)}")
 
         # this may hit a breakpoint if first instructions are executed
         # again
         logger.info('continuing execution')
         vm.qmp('replay-break', icount=last_icount - 1)
         # continue - will return after pausing
-        # This could stop at the end and get a T02 return, or by
-        # re-executing one of the breakpoints and get a T05 return.
-        g.cmd(b'c')
+        # This can stop at the end of the replay-break and gdb gets a SIGINT,
+        # or by re-executing one of the breakpoints and gdb stops at a
+        # breakpoint.
+        gdb.cli("continue")
+
         if self.vm_get_icount(vm) == last_icount - 1:
             logger.info('reached the end (icount %s)' % (last_icount - 1))
         else:
             logger.info('hit a breakpoint again at %x (icount %s)' %
-                        (self.get_pc(g), self.vm_get_icount(vm)))
+                        (self.get_pc(gdb), self.vm_get_icount(vm)))
 
         logger.info('running reverse continue to reach %x' % steps[-1])
         # reverse continue - will return after stopping at the breakpoint
-        g.cmd(b'bc', b'T05thread:01;')
+        gdb.cli("reverse-continue")
 
         # assume that none of the first instructions is executed again
         # breaking the order of the breakpoints
-        self.check_pc(g, steps[-1])
+        pc = self.get_pc(gdb)
+        if pc != steps[-1]:
+            self.fail("'reverse-continue' did not hit the first PC in reverse order!")
+
         logger.info('successfully reached %x' % steps[-1])
 
         logger.info('exiting gdb and qemu')
+        gdb.exit()
         vm.shutdown()
-- 
2.34.1

Re: [PATCH v5 8/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Daniel P. Berrangé]

On Thu, Oct 02, 2025 at 02:04:31AM +0000, Gustavo Romero wrote:
> This commit removes Avocado as a dependency for running the
> reverse_debugging test.
> 
> The main benefit, beyond eliminating an extra dependency, is that there
> is no longer any need to handle GDB packets manually. This removes the
> need for ad-hoc functions dealing with endianness and arch-specific
> register numbers, making the test easier to read. The timeout variable
> is also removed, since Meson now manages timeouts automatically.
> 
> reverse_debugging now uses the pygdbmi module to interact with GDB, if
> it's available in the test environment, otherwise the test is skipped.
> GDB is detect via the QEMU_TEST_GDB env. variable.
> 
> This commit also significantly improves the output for the test and
> now prints all the GDB commands used in sequence. It also adds
> some clarifications to existing comments, for example, clarifying that
> once the replay-break is reached, a SIGINT is captured in GDB.
> 
> reverse_debugging is kept "skipped" for aarch64, ppc64, and x86_64, so
> won't run unless QEMU_TEST_FLAKY_TESTS=1 is set in the test environment,
> before running 'make check-functional' or 'meson test [...]'.
> 
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
>  tests/functional/reverse_debugging.py | 138 ++++++++++++++------------
>  1 file changed, 74 insertions(+), 64 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH v5 9/9] tests/functional: Adapt arches to reverse_debugging w/o Avocado

[Gustavo Romero]

reverse_debugging no longer depends on Avocado, so remove the import
checks for Avocado, the per-arch endianness tweaks, and the per-arch
register settings. All of these are now handled in the ReverseDebugging
class, automatically.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
 tests/functional/aarch64/test_reverse_debug.py | 11 +++++------
 tests/functional/ppc64/test_reverse_debug.py   | 15 ++++++---------
 tests/functional/x86_64/test_reverse_debug.py  | 15 +++++----------
 3 files changed, 16 insertions(+), 25 deletions(-)

diff --git a/tests/functional/aarch64/test_reverse_debug.py b/tests/functional/aarch64/test_reverse_debug.py
index 8bc91ccfde..906f10aec5 100755
--- a/tests/functional/aarch64/test_reverse_debug.py
+++ b/tests/functional/aarch64/test_reverse_debug.py
@@ -2,25 +2,24 @@
 #
 # SPDX-License-Identifier: GPL-2.0-or-later
 #
-# Reverse debugging test
+# Reverse debugging test for aarch64
 #
 # Copyright (c) 2020 ISP RAS
+# Copyright (c) 2025 Linaro Limited
 #
 # Author:
 #  Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+#  Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
 #
 # This work is licensed under the terms of the GNU GPL, version 2 or
 # later.  See the COPYING file in the top-level directory.
 
-from qemu_test import Asset, skipIfMissingImports, skipFlakyTest
+from qemu_test import Asset, skipFlakyTest
 from reverse_debugging import ReverseDebugging
 
 
-@skipIfMissingImports('avocado.utils')
 class ReverseDebugging_AArch64(ReverseDebugging):
 
-    REG_PC = 32
-
     ASSET_KERNEL = Asset(
         ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
          'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
@@ -31,7 +30,7 @@ def test_aarch64_virt(self):
         self.set_machine('virt')
         self.cpu = 'cortex-a53'
         kernel_path = self.ASSET_KERNEL.fetch()
-        self.reverse_debugging(args=('-kernel', kernel_path))
+        self.reverse_debugging(gdb_arch='aarch64', args=('-kernel', kernel_path))
 
 
 if __name__ == '__main__':
diff --git a/tests/functional/ppc64/test_reverse_debug.py b/tests/functional/ppc64/test_reverse_debug.py
index 5931adef5a..69551fb84d 100755
--- a/tests/functional/ppc64/test_reverse_debug.py
+++ b/tests/functional/ppc64/test_reverse_debug.py
@@ -2,39 +2,36 @@
 #
 # SPDX-License-Identifier: GPL-2.0-or-later
 #
-# Reverse debugging test
+# Reverse debugging test for ppc64
 #
 # Copyright (c) 2020 ISP RAS
+# Copyright (c) 2025 Linaro Limited
 #
 # Author:
 #  Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+#  Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
 #
 # This work is licensed under the terms of the GNU GPL, version 2 or
 # later.  See the COPYING file in the top-level directory.
 
-from qemu_test import skipIfMissingImports, skipFlakyTest
+from qemu_test import skipFlakyTest
 from reverse_debugging import ReverseDebugging
 
 
-@skipIfMissingImports('avocado.utils')
 class ReverseDebugging_ppc64(ReverseDebugging):
 
-    REG_PC = 0x40
-
     @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
     def test_ppc64_pseries(self):
         self.set_machine('pseries')
         # SLOF branches back to its entry point, which causes this test
         # to take the 'hit a breakpoint again' path. That's not a problem,
         # just slightly different than the other machines.
-        self.endian_is_le = False
-        self.reverse_debugging()
+        self.reverse_debugging(gdb_arch='powerpc:common64')
 
     @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
     def test_ppc64_powernv(self):
         self.set_machine('powernv')
-        self.endian_is_le = False
-        self.reverse_debugging()
+        self.reverse_debugging(gdb_arch='powerpc:common64')
 
 
 if __name__ == '__main__':
diff --git a/tests/functional/x86_64/test_reverse_debug.py b/tests/functional/x86_64/test_reverse_debug.py
index d713e91e14..2b31ae8724 100755
--- a/tests/functional/x86_64/test_reverse_debug.py
+++ b/tests/functional/x86_64/test_reverse_debug.py
@@ -2,34 +2,29 @@
 #
 # SPDX-License-Identifier: GPL-2.0-or-later
 #
-# Reverse debugging test
+# Reverse debugging test for x86_64
 #
 # Copyright (c) 2020 ISP RAS
+# Copyright (c) 2025 Linaro Limited
 #
 # Author:
 #  Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+#  Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
 #
 # This work is licensed under the terms of the GNU GPL, version 2 or
 # later.  See the COPYING file in the top-level directory.
 
-from qemu_test import skipIfMissingImports, skipFlakyTest
+from qemu_test import skipFlakyTest
 from reverse_debugging import ReverseDebugging
 
 
-@skipIfMissingImports('avocado.utils')
 class ReverseDebugging_X86_64(ReverseDebugging):
 
-    REG_PC = 0x10
-    REG_CS = 0x12
-    def get_pc(self, g):
-        return self.get_reg_le(g, self.REG_PC) \
-            + self.get_reg_le(g, self.REG_CS) * 0x10
-
     @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2922")
     def test_x86_64_pc(self):
         self.set_machine('pc')
         # start with BIOS only
-        self.reverse_debugging()
+        self.reverse_debugging(gdb_arch='x86-64')
 
 
 if __name__ == '__main__':
-- 
2.34.1

Re: [PATCH v5 9/9] tests/functional: Adapt arches to reverse_debugging w/o Avocado

[Daniel P. Berrangé]

On Thu, Oct 02, 2025 at 02:04:32AM +0000, Gustavo Romero wrote:
> reverse_debugging no longer depends on Avocado, so remove the import
> checks for Avocado, the per-arch endianness tweaks, and the per-arch
> register settings. All of these are now handled in the ReverseDebugging
> class, automatically.
> 
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
>  tests/functional/aarch64/test_reverse_debug.py | 11 +++++------
>  tests/functional/ppc64/test_reverse_debug.py   | 15 ++++++---------
>  tests/functional/x86_64/test_reverse_debug.py  | 15 +++++----------
>  3 files changed, 16 insertions(+), 25 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v5 0/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Thomas Huth]

On 02/10/2025 04.04, Gustavo Romero wrote:
> The goal of this series is to remove Avocado as a dependency for running
> the reverse_debugging functional test.
> 
> After several rounds of discussions about v1 and v2, and experiments
> done by Daniel and Thomas (thanks for all the experiments and comments
> so far), I've taken a new approach and moved away from using a runner
> for GDB. The changes, I believe, are much simpler now.

  Hi Gustavo,

unfortunately, this still seems to be broken on Fedora. After applying your 
series, I get:

stderr:
Traceback (most recent call last):
   File "/home/thuth/devel/qemu/tests/functional/reverse_debugging.py", line 
100, in reverse_debugging
     self.reverse_debugging_run(vm, port, gdb_arch, last_icount)
     ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/lib64/python3.13/unittest/case.py", line 156, in skip_wrapper
     raise SkipTest(reason)
unittest.case.SkipTest: Missing env var(s): QEMU_TEST_GDB

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File 
"/home/thuth/devel/qemu/tests/functional/aarch64/test_reverse_debug.py", 
line 32, in test_aarch64_virt
     self.reverse_debugging(gdb_arch='aarch64', args=('-kernel', kernel_path))
     ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/thuth/devel/qemu/tests/functional/reverse_debugging.py", line 
102, in reverse_debugging
     except GDB.TimeoutError:
            ^^^^^^^^^^^^^^^^
AttributeError: type object 'GDB' has no attribute 'TimeoutError'

More information on 
test_reverse_debug.ReverseDebugging_AArch64.test_aarch64_virt could be found 
here:
  /home/thuth/tmp/qemu-build/tests/functional/aarch64/test_reverse_debug.ReverseDebugging_AArch64.test_aarch64_virt/base.log
  /home/thuth/tmp/qemu-build/tests/functional/aarch64/test_reverse_debug.ReverseDebugging_AArch64.test_aarch64_virt/console.log

(test program exited with status code 1)


... something with the environment variable passing still seems to be 
somehow broken ...?

  Thomas

Re: [PATCH v5 0/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Thomas Huth]

On 02/10/2025 18.53, Thomas Huth wrote:
> On 02/10/2025 04.04, Gustavo Romero wrote:
>> The goal of this series is to remove Avocado as a dependency for running
>> the reverse_debugging functional test.
>>
>> After several rounds of discussions about v1 and v2, and experiments
>> done by Daniel and Thomas (thanks for all the experiments and comments
>> so far), I've taken a new approach and moved away from using a runner
>> for GDB. The changes, I believe, are much simpler now.
> 
>   Hi Gustavo,
> 
> unfortunately, this still seems to be broken on Fedora. After applying your 
> series, I get:
> 
> stderr:
> Traceback (most recent call last):
>    File "/home/thuth/devel/qemu/tests/functional/reverse_debugging.py", line 
> 100, in reverse_debugging
>      self.reverse_debugging_run(vm, port, gdb_arch, last_icount)
>      ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>    File "/usr/lib64/python3.13/unittest/case.py", line 156, in skip_wrapper
>      raise SkipTest(reason)
> unittest.case.SkipTest: Missing env var(s): QEMU_TEST_GDB

Looks like it's required to explicitly use the --gdb=... parameter of 
configure to make it work, and it does not work without that paramter? Could 
you please have a look whether it works without --gdb with the auto-detected 
gdb for you?

  Thomas

Re: [PATCH v5 0/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Gustavo Romero]

Hi Thomas,

On 10/2/25 14:52, Thomas Huth wrote:
> On 02/10/2025 18.53, Thomas Huth wrote:
>> On 02/10/2025 04.04, Gustavo Romero wrote:
>>> The goal of this series is to remove Avocado as a dependency for running
>>> the reverse_debugging functional test.
>>>
>>> After several rounds of discussions about v1 and v2, and experiments
>>> done by Daniel and Thomas (thanks for all the experiments and comments
>>> so far), I've taken a new approach and moved away from using a runner
>>> for GDB. The changes, I believe, are much simpler now.
>>
>>   Hi Gustavo,
>>
>> unfortunately, this still seems to be broken on Fedora. After applying your series, I get:
>>
>> stderr:
>> Traceback (most recent call last):
>>    File "/home/thuth/devel/qemu/tests/functional/reverse_debugging.py", line 100, in reverse_debugging
>>      self.reverse_debugging_run(vm, port, gdb_arch, last_icount)
>>      ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>    File "/usr/lib64/python3.13/unittest/case.py", line 156, in skip_wrapper
>>      raise SkipTest(reason)
>> unittest.case.SkipTest: Missing env var(s): QEMU_TEST_GDB
> 
> Looks like it's required to explicitly use the --gdb=... parameter of configure to make it work, and it does not work without that paramter? Could you please have a look whether it works without --gdb with the auto-detected gdb for you?

Yes, it works without passing --gdb on Ubuntu. I'm checking it on Fedora.

But at least have you got the test skipped properly (displayed on Meson as skipped) since QEMU_TEST_GDB is missing?


Cheers,
Gustavo

Re: [PATCH v5 0/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Gustavo Romero]

Hi Thomas,

On 10/3/25 10:30, Gustavo Romero wrote:
> Hi Thomas,
> 
> On 10/2/25 14:52, Thomas Huth wrote:
>> On 02/10/2025 18.53, Thomas Huth wrote:
>>> On 02/10/2025 04.04, Gustavo Romero wrote:
>>>> The goal of this series is to remove Avocado as a dependency for running
>>>> the reverse_debugging functional test.
>>>>
>>>> After several rounds of discussions about v1 and v2, and experiments
>>>> done by Daniel and Thomas (thanks for all the experiments and comments
>>>> so far), I've taken a new approach and moved away from using a runner
>>>> for GDB. The changes, I believe, are much simpler now.
>>>
>>>   Hi Gustavo,
>>>
>>> unfortunately, this still seems to be broken on Fedora. After applying your series, I get:
>>>
>>> stderr:
>>> Traceback (most recent call last):
>>>    File "/home/thuth/devel/qemu/tests/functional/reverse_debugging.py", line 100, in reverse_debugging
>>>      self.reverse_debugging_run(vm, port, gdb_arch, last_icount)
>>>      ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>    File "/usr/lib64/python3.13/unittest/case.py", line 156, in skip_wrapper
>>>      raise SkipTest(reason)
>>> unittest.case.SkipTest: Missing env var(s): QEMU_TEST_GDB
>>
>> Looks like it's required to explicitly use the --gdb=... parameter of configure to make it work, and it does not work without that paramter? Could you please have a look whether it works without --gdb with the auto-detected gdb for you?
> 
> Yes, it works without passing --gdb on Ubuntu. I'm checking it on Fedora.
> 
> But at least have you got the test skipped properly (displayed on Meson as skipped) since QEMU_TEST_GDB is missing?

hmm actually, no it's not skipped correctly. The @skipIfMissingEnv() annotation is in the wrong method.

I've just sent a v6 with:

diff --git a/tests/functional/reverse_debugging.py b/tests/functional/reverse_debugging.py
index f06996089a..2a2e51b912 100644
--- a/tests/functional/reverse_debugging.py
+++ b/tests/functional/reverse_debugging.py
@@ -63,6 +63,7 @@ def vm_get_icount(vm):
          return vm.qmp('query-replay')['return']['icount']
  
      @skipIfMissingImports("pygdbmi") # Required by GDB class
+    @skipIfMissingEnv("QEMU_TEST_GDB")
      def reverse_debugging(self, gdb_arch, shift=7, args=None):
          from qemu_test import GDB
  
@@ -108,7 +109,6 @@ def reverse_debugging(self, gdb_arch, shift=7, args=None):
              # skipTest(), etc.
              raise
  
-    @skipIfMissingEnv("QEMU_TEST_GDB")
      def reverse_debugging_run(self, vm, port, gdb_arch, last_icount):
          logger = logging.getLogger('replay')


Now, I have not idea why GDB is not detected in Fedora. I'm setting up a Fedora env. to try it.

Are you sure GDB is installed in your test env?

Do mind checking if:

gromero@gromero0:/mnt/git/qemu_$ gdb_bin=$(command -v "gdb-multiarch" || command -v "gdb")
gromero@gromero0:/mnt/git/qemu_$ echo $gdb_bin
/usr/bin/gdb

works in your env and if QEMU_TEST_GDB is in:

$ ./pyvenv/bin/meson test  --verbose --no-rebuild -t 1 --setup thorough  --suite func-thorough  func-aarch64-reverse_debug

output?


Cheers,
Gustavo

Re: [PATCH v5 0/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Gustavo Romero]

Hi Thomas,

On 10/3/25 11:38, Gustavo Romero wrote:
> Hi Thomas,
> 
> On 10/3/25 10:30, Gustavo Romero wrote:
>> Hi Thomas,
>>
>> On 10/2/25 14:52, Thomas Huth wrote:
>>> On 02/10/2025 18.53, Thomas Huth wrote:
>>>> On 02/10/2025 04.04, Gustavo Romero wrote:
>>>>> The goal of this series is to remove Avocado as a dependency for running
>>>>> the reverse_debugging functional test.
>>>>>
>>>>> After several rounds of discussions about v1 and v2, and experiments
>>>>> done by Daniel and Thomas (thanks for all the experiments and comments
>>>>> so far), I've taken a new approach and moved away from using a runner
>>>>> for GDB. The changes, I believe, are much simpler now.
>>>>
>>>>   Hi Gustavo,
>>>>
>>>> unfortunately, this still seems to be broken on Fedora. After applying your series, I get:
>>>>
>>>> stderr:
>>>> Traceback (most recent call last):
>>>>    File "/home/thuth/devel/qemu/tests/functional/reverse_debugging.py", line 100, in reverse_debugging
>>>>      self.reverse_debugging_run(vm, port, gdb_arch, last_icount)
>>>>      ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>>    File "/usr/lib64/python3.13/unittest/case.py", line 156, in skip_wrapper
>>>>      raise SkipTest(reason)
>>>> unittest.case.SkipTest: Missing env var(s): QEMU_TEST_GDB
>>>
>>> Looks like it's required to explicitly use the --gdb=... parameter of configure to make it work, and it does not work without that paramter? Could you please have a look whether it works without --gdb with the auto-detected gdb for you?
>>
>> Yes, it works without passing --gdb on Ubuntu. I'm checking it on Fedora.
>>
>> But at least have you got the test skipped properly (displayed on Meson as skipped) since QEMU_TEST_GDB is missing?
> 
> hmm actually, no it's not skipped correctly. The @skipIfMissingEnv() annotation is in the wrong method.
> 
> I've just sent a v6 with:
> 
> diff --git a/tests/functional/reverse_debugging.py b/tests/functional/reverse_debugging.py
> index f06996089a..2a2e51b912 100644
> --- a/tests/functional/reverse_debugging.py
> +++ b/tests/functional/reverse_debugging.py
> @@ -63,6 +63,7 @@ def vm_get_icount(vm):
>           return vm.qmp('query-replay')['return']['icount']
> 
>       @skipIfMissingImports("pygdbmi") # Required by GDB class
> +    @skipIfMissingEnv("QEMU_TEST_GDB")
>       def reverse_debugging(self, gdb_arch, shift=7, args=None):
>           from qemu_test import GDB
> 
> @@ -108,7 +109,6 @@ def reverse_debugging(self, gdb_arch, shift=7, args=None):
>               # skipTest(), etc.
>               raise
> 
> -    @skipIfMissingEnv("QEMU_TEST_GDB")
>       def reverse_debugging_run(self, vm, port, gdb_arch, last_icount):
>           logger = logging.getLogger('replay')
> 
> 
> Now, I have not idea why GDB is not detected in Fedora. I'm setting up a Fedora env. to try it.

I've just tried it on Fedora 42 and GDB is correctly detected:

$ ../configure --target-list=aarch64-softmmu --disable-docs
$ make -j check-functional-aarch64
   VENVPIP install -e /home/gromero/git/qemu/python/
/home/gromero/git/qemu/build/pyvenv/bin/python3 -B /home/gromero/git/qemu/python/scripts/mkvenv.py ensuregroup  --online /home/gromero/git/qemu/pythondeps.toml testdeps
mkvenv: checking for qemu
mkvenv: checking for pygdbmi>=0.11.0.0
mkvenv: installing qemu, pygdbmi==0.11.0.0
make[1]: Entering directory '/home/gromero/git/qemu/build'
[1/6] Generating qemu-version.h with a custom command (wrapped by meson to capture output)
[2/6] Compiling C object libqmp.a.p/monitor_qmp-cmds-control.c.o
[3/6] Compiling C object libsystem.a.p/system_vl.c.o
[4/6] Compiling C object qemu-img.p/qemu-img.c.o
[5/6] Linking target qemu-img
[6/6] Linking target qemu-system-aarch64
/home/gromero/git/qemu/build/pyvenv/bin/meson test  --no-rebuild -t 1 --setup thorough   --print-errorlogs  --suite func-aarch64  --suite func-aarch64-thorough
  1/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-rme_sbsaref                SKIP              0.22s   0 subtests passed
  2/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-rme_virt                   SKIP              0.25s   0 subtests passed
  3/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-virt_gpu                   SKIP              0.22s   0 subtests passed
  4/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-tuxrun                     SKIP              0.22s   0 subtests passed
  5/29 qemu:func-quick+func-aarch64 / func-aarch64-info_usernet                                    SKIP              0.29s   0 subtests passed
  6/29 qemu:func-quick+func-aarch64 / func-aarch64-empty_cpu_model                                 OK                0.32s   1 subtests passed
  7/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-smmu                       SKIP              0.38s   0 subtests passed
  8/29 qemu:func-quick+func-aarch64 / func-aarch64-version                                         OK                0.35s   1 subtests passed
  9/29 qemu:func-quick+func-aarch64 / func-aarch64-vmstate                                         OK                0.42s   1 subtests passed
10/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-multiprocess               SKIP              0.27s   0 subtests passed
11/29 qemu:func-quick+func-aarch64 / func-aarch64-vnc                                             OK                0.53s   3 subtests passed
12/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-replay                     SKIP              0.26s   0 subtests passed
13/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-raspi3                     OK                2.91s   1 subtests passed
14/29 qemu:func-quick+func-aarch64 / func-aarch64-migration                                       OK                3.60s   3 subtests passed
15/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-hotplug_pci                OK               12.90s   1 subtests passed
16/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-xlnx_versal                OK               27.11s   1 subtests passed
17/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-raspi4                     OK               35.83s   2 subtests passed
18/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-sbsaref                    OK               36.41s   1 subtests passed
19/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-reverse_debug              OK               36.98s   1 subtests passed   <===
20/29 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-device_passthrough         OK               41.88s   1 subtests passed
[...]


also:


gromero@gromero13:~/git/qemu/build$ ./pyvenv/bin/meson test --no-rebuild -t 1 --setup thorough  --suite func-thorough  func-aarch64-reverse_debug
1/1 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-reverse_debug        OK              16.68s   1 subtests passed

Ok:                 1
Expected Fail:      0
Fail:               0
Unexpected Pass:    0
Skipped:            0
Timeout:            0

Full log written to /home/gromero/git/qemu/build/meson-logs/testlog-thorough.txt
gromero@gromero13:~/git/qemu/build$

I have installed:


gromero@gromero13:~/git/qemu/build$ dnf info gdb
Updating and loading repositories:
Repositories loaded.
Installed packages
Name            : gdb
Epoch           : 0
Version         : 16.3
Release         : 1.fc42
Architecture    : x86_64
Installed size  : 455.3 KiB
Source          : gdb-16.3-1.fc42.src.rpm
 From repository : updates
Summary         : A GNU source-level debugger for C, C++, Fortran, Go and other languages
URL             : https://gnu.org/software/gdb/
License         : GPL-3.0-or-later AND BSD-3-Clause AND FSFAP AND LGPL-2.1-or-later AND GPL-2.0-or-later AND LGPL-2.0-or-later AND LicenseRef-Fedora-Public-Domain AND GFDL-1.3-or-later AND LGPL-2.0-or-later WITH
                 :  GCC-exception-2.0 AND GPL-3.0-or-later WITH GCC-exception-3.1 AND GPL-2.0-or-later WITH GNU-compiler-exception AND MIT
Description     : GDB, the GNU debugger, allows you to debug programs written in C, C++,
                 : Fortran, Go, and other languages, by executing them in a controlled
                 : fashion and printing their data.
                 :
                 : If you want to use GDB for development purposes, you should install
                 : the 'gdb' package which will install 'gdb-headless' and possibly other
                 : useful packages too.
Vendor          : Fedora Project
gromero@gromero13:~/git/qemu/build$


I'm wondering if the skipIfMissingEnv bug (fixed in v6) have acted as a red herring and
in fact no GDB was installed in your environment the first time you tried it without passing --gdb?


Cheers,
Gustavo

> Are you sure GDB is installed in your test env?
> 
> Do mind checking if:
> 
> gromero@gromero0:/mnt/git/qemu_$ gdb_bin=$(command -v "gdb-multiarch" || command -v "gdb")
> gromero@gromero0:/mnt/git/qemu_$ echo $gdb_bin
> /usr/bin/gdb
> 
> works in your env and if QEMU_TEST_GDB is in:
> 
> $ ./pyvenv/bin/meson test  --verbose --no-rebuild -t 1 --setup thorough  --suite func-thorough  func-aarch64-reverse_debug
> 
> output?
> 
> 
> Cheers,
> Gustavo
> 
> 

Re: [PATCH v5 0/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Thomas Huth]

  Hi Gustavo!

On 03/10/2025 17.07, Gustavo Romero wrote:
> Hi Thomas,
> 
> On 10/3/25 11:38, Gustavo Romero wrote:
>> Hi Thomas,
>>
>> On 10/3/25 10:30, Gustavo Romero wrote:
>>> Hi Thomas,
>>>
>>> On 10/2/25 14:52, Thomas Huth wrote:
>>>> On 02/10/2025 18.53, Thomas Huth wrote:
>>>>> On 02/10/2025 04.04, Gustavo Romero wrote:
>>>>>> The goal of this series is to remove Avocado as a dependency for running
>>>>>> the reverse_debugging functional test.
>>>>>>
>>>>>> After several rounds of discussions about v1 and v2, and experiments
>>>>>> done by Daniel and Thomas (thanks for all the experiments and comments
>>>>>> so far), I've taken a new approach and moved away from using a runner
>>>>>> for GDB. The changes, I believe, are much simpler now.
>>>>>
>>>>>   Hi Gustavo,
>>>>>
>>>>> unfortunately, this still seems to be broken on Fedora. After applying 
>>>>> your series, I get:
>>>>>
>>>>> stderr:
>>>>> Traceback (most recent call last):
>>>>>    File "/home/thuth/devel/qemu/tests/functional/reverse_debugging.py", 
>>>>> line 100, in reverse_debugging
>>>>>      self.reverse_debugging_run(vm, port, gdb_arch, last_icount)
>>>>>      ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>>>    File "/usr/lib64/python3.13/unittest/case.py", line 156, in 
>>>>> skip_wrapper
>>>>>      raise SkipTest(reason)
>>>>> unittest.case.SkipTest: Missing env var(s): QEMU_TEST_GDB
>>>>
>>>> Looks like it's required to explicitly use the --gdb=... parameter of 
>>>> configure to make it work, and it does not work without that paramter? 
>>>> Could you please have a look whether it works without --gdb with the 
>>>> auto-detected gdb for you?
...
> I've just tried it on Fedora 42 and GDB is correctly detected:
> 
> $ ../configure --target-list=aarch64-softmmu --disable-docs
 > $ make -j check-functional-aarch64

I just tried it again, and it just seems to be broken in my build folder 
where I do incremental builds. It works when I build QEMU in a new folder 
from scratch - quite weird.

> I have installed:
> 
> gromero@gromero13:~/git/qemu/build$ dnf info gdb
> Updating and loading repositories:
> Repositories loaded.
> Installed packages
> Name            : gdb
> Epoch           : 0
> Version         : 16.3
> Release         : 1.fc42
> Architecture    : x86_64
> Installed size  : 455.3 KiB
> Source          : gdb-16.3-1.fc42.src.rpm

I've got the very same package installed here.

>> Are you sure GDB is installed in your test env?
>>
>> Do mind checking if:
>>
>> gromero@gromero0:/mnt/git/qemu_$ gdb_bin=$(command -v "gdb-multiarch" || 
>> command -v "gdb")
>> gromero@gromero0:/mnt/git/qemu_$ echo $gdb_bin
>> /usr/bin/gdb
>>
>> works in your env and if QEMU_TEST_GDB is in:
>>
>> $ ./pyvenv/bin/meson test  --verbose --no-rebuild -t 1 --setup thorough  
>> --suite func-thorough  func-aarch64-reverse_debug
>>
>> output?

$ echo $gdb_bin
/usr/bin/gdb

  Thomas

Re: [PATCH v5 0/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

>  Hi Gustavo!
>
> On 03/10/2025 17.07, Gustavo Romero wrote:
>> Hi Thomas,
>> On 10/3/25 11:38, Gustavo Romero wrote:
>>> Hi Thomas,
>>>
>>> On 10/3/25 10:30, Gustavo Romero wrote:
>>>> Hi Thomas,
>>>>
>>>> On 10/2/25 14:52, Thomas Huth wrote:
>>>>> On 02/10/2025 18.53, Thomas Huth wrote:
>>>>>> On 02/10/2025 04.04, Gustavo Romero wrote:
>>>>>>> The goal of this series is to remove Avocado as a dependency for running
>>>>>>> the reverse_debugging functional test.
>>>>>>>
>>>>>>> After several rounds of discussions about v1 and v2, and experiments
>>>>>>> done by Daniel and Thomas (thanks for all the experiments and comments
>>>>>>> so far), I've taken a new approach and moved away from using a runner
>>>>>>> for GDB. The changes, I believe, are much simpler now.
>>>>>>
>>>>>>   Hi Gustavo,
>>>>>>
>>>>>> unfortunately, this still seems to be broken on Fedora. After
>>>>>> applying your series, I get:
>>>>>>
>>>>>> stderr:
>>>>>> Traceback (most recent call last):
>>>>>>    File
>>>>>> "/home/thuth/devel/qemu/tests/functional/reverse_debugging.py",
>>>>>> line 100, in reverse_debugging
>>>>>>      self.reverse_debugging_run(vm, port, gdb_arch, last_icount)
>>>>>>      ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>>>>    File "/usr/lib64/python3.13/unittest/case.py", line 156, in
>>>>>> skip_wrapper
>>>>>>      raise SkipTest(reason)
>>>>>> unittest.case.SkipTest: Missing env var(s): QEMU_TEST_GDB
>>>>>
>>>>> Looks like it's required to explicitly use the --gdb=...
>>>>> parameter of configure to make it work, and it does not work
>>>>> without that paramter? Could you please have a look whether it
>>>>> works without --gdb with the auto-detected gdb for you?
> ...
>> I've just tried it on Fedora 42 and GDB is correctly detected:
>> $ ../configure --target-list=aarch64-softmmu --disable-docs
>> $ make -j check-functional-aarch64
>
> I just tried it again, and it just seems to be broken in my build
> folder where I do incremental builds. It works when I build QEMU in a
> new folder from scratch - quite weird.
>
>> I have installed:
>> gromero@gromero13:~/git/qemu/build$ dnf info gdb
>> Updating and loading repositories:
>> Repositories loaded.
>> Installed packages
>> Name            : gdb
>> Epoch           : 0
>> Version         : 16.3
>> Release         : 1.fc42
>> Architecture    : x86_64
>> Installed size  : 455.3 KiB
>> Source          : gdb-16.3-1.fc42.src.rpm
>
> I've got the very same package installed here.
>
>>> Are you sure GDB is installed in your test env?
>>>
>>> Do mind checking if:
>>>
>>> gromero@gromero0:/mnt/git/qemu_$ gdb_bin=$(command -v
>>> "gdb-multiarch" || command -v "gdb")
>>> gromero@gromero0:/mnt/git/qemu_$ echo $gdb_bin
>>> /usr/bin/gdb
>>>
>>> works in your env and if QEMU_TEST_GDB is in:
>>>
>>> $ ./pyvenv/bin/meson test  --verbose --no-rebuild -t 1 --setup
>>> thorough  --suite func-thorough  func-aarch64-reverse_debug
>>>
>>> output?
>
> $ echo $gdb_bin
> /usr/bin/gdb
>
>  Thomas

Do you want me to drop the series until we figure out whats going on?

FWIW it works fine in my "all" build directory and I incrementally build
it all the time.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH v5 0/9] tests/functional: Adapt reverse_debugging to run w/o Avocado

[Thomas Huth]

On 06/10/2025 11.58, Alex Bennée wrote:
> Thomas Huth <thuth@redhat.com> writes:
> 
>>   Hi Gustavo!
>>
>> On 03/10/2025 17.07, Gustavo Romero wrote:
>>> Hi Thomas,
>>> On 10/3/25 11:38, Gustavo Romero wrote:
>>>> Hi Thomas,
>>>>
>>>> On 10/3/25 10:30, Gustavo Romero wrote:
>>>>> Hi Thomas,
>>>>>
>>>>> On 10/2/25 14:52, Thomas Huth wrote:
>>>>>> On 02/10/2025 18.53, Thomas Huth wrote:
>>>>>>> On 02/10/2025 04.04, Gustavo Romero wrote:
>>>>>>>> The goal of this series is to remove Avocado as a dependency for running
>>>>>>>> the reverse_debugging functional test.
>>>>>>>>
>>>>>>>> After several rounds of discussions about v1 and v2, and experiments
>>>>>>>> done by Daniel and Thomas (thanks for all the experiments and comments
>>>>>>>> so far), I've taken a new approach and moved away from using a runner
>>>>>>>> for GDB. The changes, I believe, are much simpler now.
>>>>>>>
>>>>>>>    Hi Gustavo,
>>>>>>>
>>>>>>> unfortunately, this still seems to be broken on Fedora. After
>>>>>>> applying your series, I get:
>>>>>>>
>>>>>>> stderr:
>>>>>>> Traceback (most recent call last):
>>>>>>>     File
>>>>>>> "/home/thuth/devel/qemu/tests/functional/reverse_debugging.py",
>>>>>>> line 100, in reverse_debugging
>>>>>>>       self.reverse_debugging_run(vm, port, gdb_arch, last_icount)
>>>>>>>       ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>>>>>>     File "/usr/lib64/python3.13/unittest/case.py", line 156, in
>>>>>>> skip_wrapper
>>>>>>>       raise SkipTest(reason)
>>>>>>> unittest.case.SkipTest: Missing env var(s): QEMU_TEST_GDB
>>>>>>
>>>>>> Looks like it's required to explicitly use the --gdb=...
>>>>>> parameter of configure to make it work, and it does not work
>>>>>> without that paramter? Could you please have a look whether it
>>>>>> works without --gdb with the auto-detected gdb for you?
>> ...
>>> I've just tried it on Fedora 42 and GDB is correctly detected:
>>> $ ../configure --target-list=aarch64-softmmu --disable-docs
>>> $ make -j check-functional-aarch64
>>
>> I just tried it again, and it just seems to be broken in my build
>> folder where I do incremental builds. It works when I build QEMU in a
>> new folder from scratch - quite weird.
>>
>>> I have installed:
>>> gromero@gromero13:~/git/qemu/build$ dnf info gdb
>>> Updating and loading repositories:
>>> Repositories loaded.
>>> Installed packages
>>> Name            : gdb
>>> Epoch           : 0
>>> Version         : 16.3
>>> Release         : 1.fc42
>>> Architecture    : x86_64
>>> Installed size  : 455.3 KiB
>>> Source          : gdb-16.3-1.fc42.src.rpm
>>
>> I've got the very same package installed here.
>>
>>>> Are you sure GDB is installed in your test env?
>>>>
>>>> Do mind checking if:
>>>>
>>>> gromero@gromero0:/mnt/git/qemu_$ gdb_bin=$(command -v
>>>> "gdb-multiarch" || command -v "gdb")
>>>> gromero@gromero0:/mnt/git/qemu_$ echo $gdb_bin
>>>> /usr/bin/gdb
>>>>
>>>> works in your env and if QEMU_TEST_GDB is in:
>>>>
>>>> $ ./pyvenv/bin/meson test  --verbose --no-rebuild -t 1 --setup
>>>> thorough  --suite func-thorough  func-aarch64-reverse_debug
>>>>
>>>> output?
>>
>> $ echo $gdb_bin
>> /usr/bin/gdb
>>
>>   Thomas
> 
> Do you want me to drop the series until we figure out whats going on?

No, I think it's ok. There is no error anymore with v6, the test just got 
skipped. Since the test is normally skipped due to the FLAKY marker anyway, 
I think that's fine.
I can keep an eye on it and try to come up with a fix in case I can figure 
it out, but otherwise, I guess time will simply fix the problem when I 
occasionally blow away my build directory anyway due to other reasons.
So please go ahead and pick up v6.

  Thanks,
   Thomas