* [PATCH v2 1/5] tests/guest-debug: Make QEMU optional in run-test.py
2025-09-04 15:46 [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
@ 2025-09-04 15:46 ` Gustavo Romero
2025-09-05 7:22 ` Alex Bennée
2025-09-08 9:02 ` Thomas Huth
2025-09-04 15:46 ` [PATCH v2 2/5] tests/guest-debug: Format comments Gustavo Romero
` (5 subsequent siblings)
6 siblings, 2 replies; 25+ messages in thread
From: Gustavo Romero @ 2025-09-04 15:46 UTC (permalink / raw)
To: qemu-devel, alex.bennee, thuth, berrange
Cc: qemu-arm, 1844144, gustavo.romero
This commit makes QEMU optional in run-test.py, allowing it to be used
as a GDB runner, i.e., to call GDB and pass a test script to it without
launching QEMU. In this configuration, it is the test script’s duty to
configure and run the VMs that GDB connects to.
sys.argv passed via -ex now includes the full path to the test script in
addition to the script’s arguments, which allows unittest introspection
to work properly in case it is used in the test script.
The --binary option continues to be required when --qemu is passed.
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
tests/guest-debug/run-test.py | 60 +++++++++++++++++++----------------
1 file changed, 33 insertions(+), 27 deletions(-)
diff --git a/tests/guest-debug/run-test.py b/tests/guest-debug/run-test.py
index 75e9c92e03..d21a5dda77 100755
--- a/tests/guest-debug/run-test.py
+++ b/tests/guest-debug/run-test.py
@@ -15,6 +15,7 @@
import subprocess
import shutil
import shlex
+import sys
import os
from time import sleep
from tempfile import TemporaryDirectory
@@ -22,10 +23,10 @@
def get_args():
parser = argparse.ArgumentParser(description="A gdbstub test runner")
parser.add_argument("--qemu", help="Qemu binary for test",
- required=True)
+ required=False)
parser.add_argument("--qargs", help="Qemu arguments for test")
parser.add_argument("--binary", help="Binary to debug",
- required=True)
+ required='--qemu' in sys.argv)
parser.add_argument("--test", help="GDB test script")
parser.add_argument('test_args', nargs='*',
help="Additional args for GDB test script. "
@@ -73,27 +74,30 @@ def log(output, msg):
socket_dir = TemporaryDirectory("qemu-gdbstub")
socket_name = os.path.join(socket_dir.name, "gdbstub.socket")
- # Launch QEMU with binary
- if "system" in args.qemu:
- if args.no_suspend:
- suspend = ''
+ if args.qemu:
+ # Launch QEMU with binary.
+ if "system" in args.qemu:
+ if args.no_suspend:
+ suspend = ''
+ else:
+ suspend = ' -S'
+ cmd = f'{args.qemu} {args.qargs} {args.binary}' \
+ f'{suspend} -gdb unix:path={socket_name},server=on'
else:
- suspend = ' -S'
- cmd = f'{args.qemu} {args.qargs} {args.binary}' \
- f'{suspend} -gdb unix:path={socket_name},server=on'
- else:
- if args.no_suspend:
- suspend = ',suspend=n'
- else:
- suspend = ''
- cmd = f'{args.qemu} {args.qargs} -g {socket_name}{suspend}' \
- f' {args.binary}'
+ if args.no_suspend:
+ suspend = ',suspend=n'
+ else:
+ suspend = ''
+ cmd = f'{args.qemu} {args.qargs} -g {socket_name}{suspend}' \
+ f' {args.binary}'
- log(output, "QEMU CMD: %s" % (cmd))
- inferior = subprocess.Popen(shlex.split(cmd))
+ log(output, "QEMU CMD: %s" % (cmd))
+ inferior = subprocess.Popen(shlex.split(cmd))
# Now launch gdb with our test and collect the result
- gdb_cmd = "%s %s" % (args.gdb, args.binary)
+ gdb_cmd = args.gdb
+ if args.binary:
+ gdb_cmd += " %s" % (args.binary)
if args.gdb_args:
gdb_cmd += " %s" % (args.gdb_args)
# run quietly and ignore .gdbinit
@@ -103,11 +107,12 @@ def log(output, msg):
# disable prompts in case of crash
gdb_cmd += " -ex 'set confirm off'"
# connect to remote
- gdb_cmd += " -ex 'target remote %s'" % (socket_name)
+ if args.qemu:
+ gdb_cmd += " -ex 'target remote %s'" % (socket_name)
# finally the test script itself
if args.test:
- if args.test_args:
- gdb_cmd += f" -ex \"py sys.argv={args.test_args}\""
+ argv = [args.test] + args.test_args
+ gdb_cmd += f" -ex \"py sys.argv={argv}\""
gdb_cmd += " -x %s" % (args.test)
@@ -129,10 +134,11 @@ def log(output, msg):
log(output, "GDB crashed? (%d, %d) SKIPPING" % (result, result - 128))
exit(0)
- try:
- inferior.wait(2)
- except subprocess.TimeoutExpired:
- log(output, "GDB never connected? Killed guest")
- inferior.kill()
+ if args.qemu:
+ try:
+ inferior.wait(2)
+ except subprocess.TimeoutExpired:
+ log(output, "GDB never connected? Killed guest")
+ inferior.kill()
exit(result)
--
2.34.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v2 1/5] tests/guest-debug: Make QEMU optional in run-test.py
2025-09-04 15:46 ` [PATCH v2 1/5] tests/guest-debug: Make QEMU optional in run-test.py Gustavo Romero
@ 2025-09-05 7:22 ` Alex Bennée
2025-09-08 9:02 ` Thomas Huth
1 sibling, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2025-09-05 7:22 UTC (permalink / raw)
To: Gustavo Romero; +Cc: qemu-devel, thuth, berrange, qemu-arm, 1844144
Gustavo Romero <gustavo.romero@linaro.org> writes:
> This commit makes QEMU optional in run-test.py, allowing it to be used
> as a GDB runner, i.e., to call GDB and pass a test script to it without
> launching QEMU. In this configuration, it is the test script’s duty to
> configure and run the VMs that GDB connects to.
>
> sys.argv passed via -ex now includes the full path to the test script in
> addition to the script’s arguments, which allows unittest introspection
> to work properly in case it is used in the test script.
>
> The --binary option continues to be required when --qemu is passed.
>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/guest-debug/run-test.py | 60 +++++++++++++++++++----------------
> 1 file changed, 33 insertions(+), 27 deletions(-)
>
> diff --git a/tests/guest-debug/run-test.py b/tests/guest-debug/run-test.py
> index 75e9c92e03..d21a5dda77 100755
> --- a/tests/guest-debug/run-test.py
> +++ b/tests/guest-debug/run-test.py
> @@ -15,6 +15,7 @@
> import subprocess
> import shutil
> import shlex
> +import sys
> import os
> from time import sleep
> from tempfile import TemporaryDirectory
> @@ -22,10 +23,10 @@
> def get_args():
> parser = argparse.ArgumentParser(description="A gdbstub test runner")
> parser.add_argument("--qemu", help="Qemu binary for test",
> - required=True)
> + required=False)
> parser.add_argument("--qargs", help="Qemu arguments for test")
> parser.add_argument("--binary", help="Binary to debug",
> - required=True)
> + required='--qemu' in sys.argv)
> parser.add_argument("--test", help="GDB test script")
> parser.add_argument('test_args', nargs='*',
> help="Additional args for GDB test script. "
> @@ -73,27 +74,30 @@ def log(output, msg):
> socket_dir = TemporaryDirectory("qemu-gdbstub")
> socket_name = os.path.join(socket_dir.name, "gdbstub.socket")
>
> - # Launch QEMU with binary
> - if "system" in args.qemu:
> - if args.no_suspend:
> - suspend = ''
> + if args.qemu:
> + # Launch QEMU with binary.
> + if "system" in args.qemu:
> + if args.no_suspend:
> + suspend = ''
> + else:
> + suspend = ' -S'
> + cmd = f'{args.qemu} {args.qargs} {args.binary}' \
> + f'{suspend} -gdb unix:path={socket_name},server=on'
> else:
> - suspend = ' -S'
> - cmd = f'{args.qemu} {args.qargs} {args.binary}' \
> - f'{suspend} -gdb unix:path={socket_name},server=on'
> - else:
> - if args.no_suspend:
> - suspend = ',suspend=n'
> - else:
> - suspend = ''
> - cmd = f'{args.qemu} {args.qargs} -g {socket_name}{suspend}' \
> - f' {args.binary}'
> + if args.no_suspend:
> + suspend = ',suspend=n'
> + else:
> + suspend = ''
> + cmd = f'{args.qemu} {args.qargs} -g {socket_name}{suspend}' \
> + f' {args.binary}'
>
> - log(output, "QEMU CMD: %s" % (cmd))
> - inferior = subprocess.Popen(shlex.split(cmd))
> + log(output, "QEMU CMD: %s" % (cmd))
> + inferior = subprocess.Popen(shlex.split(cmd))
>
> # Now launch gdb with our test and collect the result
> - gdb_cmd = "%s %s" % (args.gdb, args.binary)
> + gdb_cmd = args.gdb
> + if args.binary:
> + gdb_cmd += " %s" % (args.binary)
> if args.gdb_args:
> gdb_cmd += " %s" % (args.gdb_args)
> # run quietly and ignore .gdbinit
> @@ -103,11 +107,12 @@ def log(output, msg):
> # disable prompts in case of crash
> gdb_cmd += " -ex 'set confirm off'"
> # connect to remote
> - gdb_cmd += " -ex 'target remote %s'" % (socket_name)
> + if args.qemu:
> + gdb_cmd += " -ex 'target remote %s'" % (socket_name)
> # finally the test script itself
> if args.test:
> - if args.test_args:
> - gdb_cmd += f" -ex \"py sys.argv={args.test_args}\""
> + argv = [args.test] + args.test_args
> + gdb_cmd += f" -ex \"py sys.argv={argv}\""
> gdb_cmd += " -x %s" % (args.test)
>
>
> @@ -129,10 +134,11 @@ def log(output, msg):
> log(output, "GDB crashed? (%d, %d) SKIPPING" % (result, result - 128))
> exit(0)
>
> - try:
> - inferior.wait(2)
> - except subprocess.TimeoutExpired:
> - log(output, "GDB never connected? Killed guest")
> - inferior.kill()
> + if args.qemu:
> + try:
> + inferior.wait(2)
> + except subprocess.TimeoutExpired:
> + log(output, "GDB never connected? Killed guest")
> + inferior.kill()
>
> exit(result)
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 1/5] tests/guest-debug: Make QEMU optional in run-test.py
2025-09-04 15:46 ` [PATCH v2 1/5] tests/guest-debug: Make QEMU optional in run-test.py Gustavo Romero
2025-09-05 7:22 ` Alex Bennée
@ 2025-09-08 9:02 ` Thomas Huth
1 sibling, 0 replies; 25+ messages in thread
From: Thomas Huth @ 2025-09-08 9:02 UTC (permalink / raw)
To: Gustavo Romero, qemu-devel, alex.bennee, berrange; +Cc: qemu-arm, 1844144
On 04/09/2025 17.46, Gustavo Romero wrote:
> This commit makes QEMU optional in run-test.py, allowing it to be used
> as a GDB runner, i.e., to call GDB and pass a test script to it without
> launching QEMU. In this configuration, it is the test script’s duty to
> configure and run the VMs that GDB connects to.
>
> sys.argv passed via -ex now includes the full path to the test script in
> addition to the script’s arguments, which allows unittest introspection
> to work properly in case it is used in the test script.
>
> The --binary option continues to be required when --qemu is passed.
>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
> tests/guest-debug/run-test.py | 60 +++++++++++++++++++----------------
> 1 file changed, 33 insertions(+), 27 deletions(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v2 2/5] tests/guest-debug: Format comments
2025-09-04 15:46 [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
2025-09-04 15:46 ` [PATCH v2 1/5] tests/guest-debug: Make QEMU optional in run-test.py Gustavo Romero
@ 2025-09-04 15:46 ` Gustavo Romero
2025-09-05 7:22 ` Alex Bennée
2025-09-08 9:03 ` Thomas Huth
2025-09-04 15:46 ` [PATCH v2 3/5] tests/guest-debug: Add quiet option to run-tests.py Gustavo Romero
` (4 subsequent siblings)
6 siblings, 2 replies; 25+ messages in thread
From: Gustavo Romero @ 2025-09-04 15:46 UTC (permalink / raw)
To: qemu-devel, alex.bennee, thuth, berrange
Cc: qemu-arm, 1844144, gustavo.romero
Capitalize and add periods to comments.
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
tests/guest-debug/run-test.py | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/tests/guest-debug/run-test.py b/tests/guest-debug/run-test.py
index d21a5dda77..e051e8947c 100755
--- a/tests/guest-debug/run-test.py
+++ b/tests/guest-debug/run-test.py
@@ -54,7 +54,7 @@ def log(output, msg):
if __name__ == '__main__':
args = get_args()
- # Search for a gdb we can use
+ # Search for a gdb we can use.
if not args.gdb:
args.gdb = shutil.which("gdb-multiarch")
if not args.gdb:
@@ -94,22 +94,22 @@ def log(output, msg):
log(output, "QEMU CMD: %s" % (cmd))
inferior = subprocess.Popen(shlex.split(cmd))
- # Now launch gdb with our test and collect the result
+ # Now launch gdb with our test and collect the result.
gdb_cmd = args.gdb
if args.binary:
gdb_cmd += " %s" % (args.binary)
if args.gdb_args:
gdb_cmd += " %s" % (args.gdb_args)
- # run quietly and ignore .gdbinit
+ # Run quietly and ignore .gdbinit.
gdb_cmd += " -q -n -batch"
- # disable pagination
+ # Disable pagination.
gdb_cmd += " -ex 'set pagination off'"
- # disable prompts in case of crash
+ # Disable prompts in case of crash.
gdb_cmd += " -ex 'set confirm off'"
- # connect to remote
+ # Connect automatically to remote only if QEMU is launched.
if args.qemu:
gdb_cmd += " -ex 'target remote %s'" % (socket_name)
- # finally the test script itself
+ # Finally the test script itself.
if args.test:
argv = [args.test] + args.test_args
gdb_cmd += f" -ex \"py sys.argv={argv}\""
--
2.34.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v2 2/5] tests/guest-debug: Format comments
2025-09-04 15:46 ` [PATCH v2 2/5] tests/guest-debug: Format comments Gustavo Romero
@ 2025-09-05 7:22 ` Alex Bennée
2025-09-08 9:03 ` Thomas Huth
1 sibling, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2025-09-05 7:22 UTC (permalink / raw)
To: Gustavo Romero; +Cc: qemu-devel, thuth, berrange, qemu-arm, 1844144
Gustavo Romero <gustavo.romero@linaro.org> writes:
> Capitalize and add periods to comments.
>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/guest-debug/run-test.py | 14 +++++++-------
> 1 file changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/tests/guest-debug/run-test.py b/tests/guest-debug/run-test.py
> index d21a5dda77..e051e8947c 100755
> --- a/tests/guest-debug/run-test.py
> +++ b/tests/guest-debug/run-test.py
> @@ -54,7 +54,7 @@ def log(output, msg):
> if __name__ == '__main__':
> args = get_args()
>
> - # Search for a gdb we can use
> + # Search for a gdb we can use.
> if not args.gdb:
> args.gdb = shutil.which("gdb-multiarch")
> if not args.gdb:
> @@ -94,22 +94,22 @@ def log(output, msg):
> log(output, "QEMU CMD: %s" % (cmd))
> inferior = subprocess.Popen(shlex.split(cmd))
>
> - # Now launch gdb with our test and collect the result
> + # Now launch gdb with our test and collect the result.
> gdb_cmd = args.gdb
> if args.binary:
> gdb_cmd += " %s" % (args.binary)
> if args.gdb_args:
> gdb_cmd += " %s" % (args.gdb_args)
> - # run quietly and ignore .gdbinit
> + # Run quietly and ignore .gdbinit.
> gdb_cmd += " -q -n -batch"
> - # disable pagination
> + # Disable pagination.
> gdb_cmd += " -ex 'set pagination off'"
> - # disable prompts in case of crash
> + # Disable prompts in case of crash.
> gdb_cmd += " -ex 'set confirm off'"
> - # connect to remote
> + # Connect automatically to remote only if QEMU is launched.
> if args.qemu:
> gdb_cmd += " -ex 'target remote %s'" % (socket_name)
> - # finally the test script itself
> + # Finally the test script itself.
> if args.test:
> argv = [args.test] + args.test_args
> gdb_cmd += f" -ex \"py sys.argv={argv}\""
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 2/5] tests/guest-debug: Format comments
2025-09-04 15:46 ` [PATCH v2 2/5] tests/guest-debug: Format comments Gustavo Romero
2025-09-05 7:22 ` Alex Bennée
@ 2025-09-08 9:03 ` Thomas Huth
1 sibling, 0 replies; 25+ messages in thread
From: Thomas Huth @ 2025-09-08 9:03 UTC (permalink / raw)
To: Gustavo Romero, qemu-devel, alex.bennee, berrange; +Cc: qemu-arm, 1844144
On 04/09/2025 17.46, Gustavo Romero wrote:
> Capitalize and add periods to comments.
>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
> tests/guest-debug/run-test.py | 14 +++++++-------
> 1 file changed, 7 insertions(+), 7 deletions(-)
Looks like it's currently a mix in the script, so it might be a good idea
indeed to standardize on one format here.
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v2 3/5] tests/guest-debug: Add quiet option to run-tests.py
2025-09-04 15:46 [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
2025-09-04 15:46 ` [PATCH v2 1/5] tests/guest-debug: Make QEMU optional in run-test.py Gustavo Romero
2025-09-04 15:46 ` [PATCH v2 2/5] tests/guest-debug: Format comments Gustavo Romero
@ 2025-09-04 15:46 ` Gustavo Romero
2025-09-05 7:21 ` Alex Bennée
2025-09-08 9:12 ` Thomas Huth
2025-09-04 15:46 ` [PATCH v2 4/5] tests/functional: Support tests that require a runner Gustavo Romero
` (3 subsequent siblings)
6 siblings, 2 replies; 25+ messages in thread
From: Gustavo Romero @ 2025-09-04 15:46 UTC (permalink / raw)
To: qemu-devel, alex.bennee, thuth, berrange
Cc: qemu-arm, 1844144, gustavo.romero
Add a --quiet option to run-tests.py so it can run without printing any
messages to the stdout.
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
tests/guest-debug/run-test.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/tests/guest-debug/run-test.py b/tests/guest-debug/run-test.py
index e051e8947c..cf8b37b8b8 100755
--- a/tests/guest-debug/run-test.py
+++ b/tests/guest-debug/run-test.py
@@ -39,15 +39,17 @@ def get_args():
parser.add_argument("--stderr", help="A file to redirect stderr to")
parser.add_argument("--no-suspend", action="store_true",
help="Ask the binary to not wait for GDB connection")
+ parser.add_argument("--quiet", action="store_true", default=False,
+ help="Don't print any messages to stdout")
return parser.parse_args()
-def log(output, msg):
+def log(output, msg, quiet):
if output:
output.write(msg + "\n")
output.flush()
- else:
+ elif not quiet:
print(msg)
@@ -91,7 +93,7 @@ def log(output, msg):
cmd = f'{args.qemu} {args.qargs} -g {socket_name}{suspend}' \
f' {args.binary}'
- log(output, "QEMU CMD: %s" % (cmd))
+ log(output, "QEMU CMD: %s" % (cmd), args.quiet)
inferior = subprocess.Popen(shlex.split(cmd))
# Now launch gdb with our test and collect the result.
@@ -117,7 +119,7 @@ def log(output, msg):
sleep(1)
- log(output, "GDB CMD: %s" % (gdb_cmd))
+ log(output, "GDB CMD: %s" % (gdb_cmd), args.quiet)
gdb_env = dict(os.environ)
gdb_pythonpath = gdb_env.get("PYTHONPATH", "").split(os.pathsep)
--
2.34.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v2 3/5] tests/guest-debug: Add quiet option to run-tests.py
2025-09-04 15:46 ` [PATCH v2 3/5] tests/guest-debug: Add quiet option to run-tests.py Gustavo Romero
@ 2025-09-05 7:21 ` Alex Bennée
2025-09-08 9:12 ` Thomas Huth
1 sibling, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2025-09-05 7:21 UTC (permalink / raw)
To: Gustavo Romero; +Cc: qemu-devel, thuth, berrange, qemu-arm, 1844144
Gustavo Romero <gustavo.romero@linaro.org> writes:
> Add a --quiet option to run-tests.py so it can run without printing any
> messages to the stdout.
>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/guest-debug/run-test.py | 10 ++++++----
> 1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/tests/guest-debug/run-test.py b/tests/guest-debug/run-test.py
> index e051e8947c..cf8b37b8b8 100755
> --- a/tests/guest-debug/run-test.py
> +++ b/tests/guest-debug/run-test.py
> @@ -39,15 +39,17 @@ def get_args():
> parser.add_argument("--stderr", help="A file to redirect stderr to")
> parser.add_argument("--no-suspend", action="store_true",
> help="Ask the binary to not wait for GDB connection")
> + parser.add_argument("--quiet", action="store_true", default=False,
> + help="Don't print any messages to stdout")
>
> return parser.parse_args()
>
>
> -def log(output, msg):
> +def log(output, msg, quiet):
> if output:
> output.write(msg + "\n")
> output.flush()
> - else:
> + elif not quiet:
> print(msg)
>
>
> @@ -91,7 +93,7 @@ def log(output, msg):
> cmd = f'{args.qemu} {args.qargs} -g {socket_name}{suspend}' \
> f' {args.binary}'
>
> - log(output, "QEMU CMD: %s" % (cmd))
> + log(output, "QEMU CMD: %s" % (cmd), args.quiet)
> inferior = subprocess.Popen(shlex.split(cmd))
>
> # Now launch gdb with our test and collect the result.
> @@ -117,7 +119,7 @@ def log(output, msg):
>
>
> sleep(1)
> - log(output, "GDB CMD: %s" % (gdb_cmd))
> + log(output, "GDB CMD: %s" % (gdb_cmd), args.quiet)
>
> gdb_env = dict(os.environ)
> gdb_pythonpath = gdb_env.get("PYTHONPATH", "").split(os.pathsep)
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 3/5] tests/guest-debug: Add quiet option to run-tests.py
2025-09-04 15:46 ` [PATCH v2 3/5] tests/guest-debug: Add quiet option to run-tests.py Gustavo Romero
2025-09-05 7:21 ` Alex Bennée
@ 2025-09-08 9:12 ` Thomas Huth
1 sibling, 0 replies; 25+ messages in thread
From: Thomas Huth @ 2025-09-08 9:12 UTC (permalink / raw)
To: Gustavo Romero, qemu-devel, alex.bennee, berrange; +Cc: qemu-arm, 1844144
On 04/09/2025 17.46, Gustavo Romero wrote:
> Add a --quiet option to run-tests.py so it can run without printing any
> messages to the stdout.
>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
> tests/guest-debug/run-test.py | 10 ++++++----
> 1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/tests/guest-debug/run-test.py b/tests/guest-debug/run-test.py
> index e051e8947c..cf8b37b8b8 100755
> --- a/tests/guest-debug/run-test.py
> +++ b/tests/guest-debug/run-test.py
> @@ -39,15 +39,17 @@ def get_args():
> parser.add_argument("--stderr", help="A file to redirect stderr to")
> parser.add_argument("--no-suspend", action="store_true",
> help="Ask the binary to not wait for GDB connection")
> + parser.add_argument("--quiet", action="store_true", default=False,
> + help="Don't print any messages to stdout")
>
> return parser.parse_args()
>
>
> -def log(output, msg):
> +def log(output, msg, quiet):
> if output:
> output.write(msg + "\n")
> output.flush()
> - else:
> + elif not quiet:
> print(msg)
>
>
> @@ -91,7 +93,7 @@ def log(output, msg):
> cmd = f'{args.qemu} {args.qargs} -g {socket_name}{suspend}' \
> f' {args.binary}'
>
> - log(output, "QEMU CMD: %s" % (cmd))
> + log(output, "QEMU CMD: %s" % (cmd), args.quiet)
> inferior = subprocess.Popen(shlex.split(cmd))
>
> # Now launch gdb with our test and collect the result.
> @@ -117,7 +119,7 @@ def log(output, msg):
>
>
> sleep(1)
> - log(output, "GDB CMD: %s" % (gdb_cmd))
> + log(output, "GDB CMD: %s" % (gdb_cmd), args.quiet)
>
> gdb_env = dict(os.environ)
> gdb_pythonpath = gdb_env.get("PYTHONPATH", "").split(os.pathsep)
I can see two more calls to log() in that script, don't you need to change
these spots, too:
$ grep log tests/guest-debug/run-test.py
def log(output, msg):
log(output, "QEMU CMD: %s" % (cmd))
log(output, "GDB CMD: %s" % (gdb_cmd))
log(output, "GDB crashed? (%d, %d) SKIPPING" % (result, result - 128))
log(output, "GDB never connected? Killed guest")
?
Maybe you could declare the new parameter with quiet=False by default, so
that you don't have to worry?
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v2 4/5] tests/functional: Support tests that require a runner
2025-09-04 15:46 [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
` (2 preceding siblings ...)
2025-09-04 15:46 ` [PATCH v2 3/5] tests/guest-debug: Add quiet option to run-tests.py Gustavo Romero
@ 2025-09-04 15:46 ` Gustavo Romero
2025-09-08 9:21 ` Thomas Huth
2025-09-04 15:46 ` [PATCH v2 5/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
` (2 subsequent siblings)
6 siblings, 1 reply; 25+ messages in thread
From: Gustavo Romero @ 2025-09-04 15:46 UTC (permalink / raw)
To: qemu-devel, alex.bennee, thuth, berrange
Cc: qemu-arm, 1844144, gustavo.romero
Add support for running tests that require a specific runner.
The test is specified via a tuple (name, runner, protocol), where name
is the test name as found in the tests/functional directory without the
'test_' prefix and the .py extension, runner is an array containing the
runner and any arguments required by the runner, and protocol is
the test protocol used by Meson to determine whether the test passed or
failed.
The test tuples are added to arrays that follow the current naming
logic but with the suffix '_with_runner' appended to their names. In
Meson it's not easy to select an element in an array at runtime based on
its type, so it's simpler to have a new array for these new test types
than use the current ones from the tests that don't require a runner,
and so avoid mixing strings and tuples in the same array.
Currently there is only one runner, the GDB runner, but more runners can
be defined and associated to a test via the tuple.
The GDB runner is only defined if GDB is detected. The probe is done
in 'configure' and the full path is passed to meson.build via the -Dgdb=
option.
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
configure | 2 ++
meson.build | 4 +++
meson_options.txt | 2 ++
scripts/meson-buildoptions.sh | 2 ++
tests/functional/meson.build | 65 +++++++++++++++++++++++++++++++++++
5 files changed, 75 insertions(+)
diff --git a/configure b/configure
index 274a778764..8e2e2cd562 100755
--- a/configure
+++ b/configure
@@ -1978,6 +1978,8 @@ if test "$skip_meson" = no; then
test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE"
test "$plugins" = yes && meson_option_add "-Dplugins=true"
test "$tcg" != enabled && meson_option_add "-Dtcg=$tcg"
+ test -n "$gdb_bin" && meson_option_add "-Dgdb=$gdb_bin"
+
run_meson() {
NINJA=$ninja $meson setup "$@" "$PWD" "$source_path"
}
diff --git a/meson.build b/meson.build
index 0d42de61ae..7e0afe8288 100644
--- a/meson.build
+++ b/meson.build
@@ -75,6 +75,10 @@ have_user = have_linux_user or have_bsd_user
sh = find_program('sh')
python = import('python').find_installation()
+# Meson python.get_path() on 'purelib' or 'platlib' doesn't properly return the
+# site-packages dir in pyvenv, so it is built manually.
+python_ver = python.language_version()
+python_site_packages = meson.build_root() / 'pyvenv/lib/python' + python_ver / 'site-packages'
cc = meson.get_compiler('c')
all_languages = ['c']
diff --git a/meson_options.txt b/meson_options.txt
index fff1521e58..5bb41bcbc4 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -36,6 +36,8 @@ option('trace_file', type: 'string', value: 'trace',
option('coroutine_backend', type: 'combo',
choices: ['ucontext', 'sigaltstack', 'windows', 'wasm', 'auto'],
value: 'auto', description: 'coroutine backend to use')
+option('gdb', type: 'string', value: '',
+ description: 'Path to GDB')
# Everything else can be set via --enable/--disable-* option
# on the configure script command line. After adding an option
diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
index 0ebe6bc52a..f4bd21220e 100644
--- a/scripts/meson-buildoptions.sh
+++ b/scripts/meson-buildoptions.sh
@@ -58,6 +58,7 @@ meson_options_help() {
printf "%s\n" ' --enable-ubsan enable undefined behaviour sanitizer'
printf "%s\n" ' --firmwarepath=VALUES search PATH for firmware files [share/qemu-'
printf "%s\n" ' firmware]'
+ printf "%s\n" ' --gdb=VALUE Path to GDB'
printf "%s\n" ' --iasl=VALUE Path to ACPI disassembler'
printf "%s\n" ' --includedir=VALUE Header file directory [include]'
printf "%s\n" ' --interp-prefix=VALUE where to find shared libraries etc., use %M for'
@@ -323,6 +324,7 @@ _meson_option_parse() {
--disable-fuzzing) printf "%s" -Dfuzzing=false ;;
--enable-gcrypt) printf "%s" -Dgcrypt=enabled ;;
--disable-gcrypt) printf "%s" -Dgcrypt=disabled ;;
+ --gdb=*) quote_sh "-Dgdb=$2" ;;
--enable-gettext) printf "%s" -Dgettext=enabled ;;
--disable-gettext) printf "%s" -Dgettext=disabled ;;
--enable-gio) printf "%s" -Dgio=enabled ;;
diff --git a/tests/functional/meson.build b/tests/functional/meson.build
index 2a0c5aa141..febd31a263 100644
--- a/tests/functional/meson.build
+++ b/tests/functional/meson.build
@@ -9,6 +9,25 @@ if get_option('tcg_interpreter')
subdir_done()
endif
+# Define the GDB runner if GDB is available.
+gdb = get_option('gdb')
+if gdb != ''
+ gdb_runner_script = meson.project_source_root() + '/tests/guest-debug/run-test.py'
+ gdb_runner = [gdb_runner_script, '--quiet', '--gdb', gdb, '--test']
+
+ message('GDB runner defined based on GDB found at: ', gdb)
+
+ # A test with a runner is a tuple (name, runner, protocol).
+ # The tests must be elements of an array named like:
+ #
+ # test_<arch>_<mode=[system|linuxuser|bsduser]>_<speed=[quick|thorough]>_with_runner = [
+ # ['test0', gdb_runner, 'exitcode'],
+ # ...
+ # ]
+else
+ message('GDB not found, skipping functional tests that rely on it.')
+endif
+
subdir('aarch64')
subdir('alpha')
subdir('arm')
@@ -61,9 +80,11 @@ foreach speed : ['quick', 'thorough']
suites = ['func-quick', 'func-' + target_base]
target_tests = get_variable('tests_' + target_base + '_' + sysmode + '_quick', []) \
+ get_variable('tests_generic_' + sysmode)
+ target_tests_r = get_variable('tests_' + target_base + '_' + sysmode + '_quick_with_runner', [])
else
suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed]
target_tests = get_variable('tests_' + target_base + '_' + sysmode + '_' + speed, [])
+ target_tests_r = get_variable('tests_' + target_base + '_' + sysmode + '_' + speed + '_with_runner', [])
endif
test_deps = [roms, keymap_targets]
@@ -121,6 +142,50 @@ foreach speed : ['quick', 'thorough']
priority: time_out,
suite: suites)
endforeach
+
+ # Prepare tests that require a specific runner.
+ foreach test : target_tests_r
+ testname = '@0@-@1@'.format(target_base, test[0])
+ testfile = target_base / 'test_' + test[0] + '.py'
+ testpath = meson.current_source_dir() / testfile
+ teststamp = testname + '.tstamp'
+ testrunner = test[1]
+ testproto = test[2]
+
+ # python_site_packages, i.e., site packages from Python in pyvenv, is
+ # added to PYTHONPATH because some runners can run a program that has its
+ # own Python hooks that, by its turn, will search for modules based on
+ # PYTHONPATH independently of the Python used by the runner, like, for
+ # example, GDB using libpython.
+ test_r_precache_env = test_precache_env
+ test_r_precache_env.append('PYTHONPATH', python_site_packages)
+ # For similar reasons, PYTHONPATH must also include the path to the test
+ # scripts, otherwise unittest's introspection will failed.
+ test_r_precache_env.append('PYTHONPATH', meson.current_source_dir() / target_base)
+
+ precache = custom_target('func-precache-' + testname,
+ output: teststamp,
+ command: [testrunner, testpath],
+ depend_files: files(testpath),
+ build_by_default: false,
+ env: test_r_precache_env)
+ precache_all += precache
+
+ # See comments above about PYTHONPATH in test_r_precache_env.
+ test_r_env = test_env
+ test_r_env.append('PYTHONPATH', python_site_packages)
+ test_r_env.append('PYTHONPATH', meson.current_source_dir() / target_base)
+
+ test('func-' + testname,
+ python,
+ depends: [test_deps, test_emulator, emulator_modules, plugin_modules],
+ env: test_r_env,
+ args: [testrunner, testpath],
+ protocol: testproto,
+ timeout: time_out,
+ priority: time_out,
+ suite: suites)
+ endforeach
endforeach
endforeach
--
2.34.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v2 4/5] tests/functional: Support tests that require a runner
2025-09-04 15:46 ` [PATCH v2 4/5] tests/functional: Support tests that require a runner Gustavo Romero
@ 2025-09-08 9:21 ` Thomas Huth
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Huth @ 2025-09-08 9:21 UTC (permalink / raw)
To: Gustavo Romero, qemu-devel, alex.bennee, berrange; +Cc: qemu-arm, 1844144
On 04/09/2025 17.46, Gustavo Romero wrote:
> Add support for running tests that require a specific runner.
>
> The test is specified via a tuple (name, runner, protocol), where name
> is the test name as found in the tests/functional directory without the
> 'test_' prefix and the .py extension, runner is an array containing the
> runner and any arguments required by the runner, and protocol is
> the test protocol used by Meson to determine whether the test passed or
> failed.
>
> The test tuples are added to arrays that follow the current naming
> logic but with the suffix '_with_runner' appended to their names. In
> Meson it's not easy to select an element in an array at runtime based on
> its type, so it's simpler to have a new array for these new test types
> than use the current ones from the tests that don't require a runner,
> and so avoid mixing strings and tuples in the same array.
>
> Currently there is only one runner, the GDB runner, but more runners can
> be defined and associated to a test via the tuple.
>
> The GDB runner is only defined if GDB is detected. The probe is done
> in 'configure' and the full path is passed to meson.build via the -Dgdb=
> option.
>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
> configure | 2 ++
> meson.build | 4 +++
> meson_options.txt | 2 ++
> scripts/meson-buildoptions.sh | 2 ++
> tests/functional/meson.build | 65 +++++++++++++++++++++++++++++++++++
> 5 files changed, 75 insertions(+)
>
> diff --git a/configure b/configure
> index 274a778764..8e2e2cd562 100755
> --- a/configure
> +++ b/configure
> @@ -1978,6 +1978,8 @@ if test "$skip_meson" = no; then
> test -n "${LIB_FUZZING_ENGINE+xxx}" && meson_option_add "-Dfuzzing_engine=$LIB_FUZZING_ENGINE"
> test "$plugins" = yes && meson_option_add "-Dplugins=true"
> test "$tcg" != enabled && meson_option_add "-Dtcg=$tcg"
> + test -n "$gdb_bin" && meson_option_add "-Dgdb=$gdb_bin"
> +
> run_meson() {
> NINJA=$ninja $meson setup "$@" "$PWD" "$source_path"
> }
> diff --git a/meson.build b/meson.build
> index 0d42de61ae..7e0afe8288 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -75,6 +75,10 @@ have_user = have_linux_user or have_bsd_user
>
> sh = find_program('sh')
> python = import('python').find_installation()
> +# Meson python.get_path() on 'purelib' or 'platlib' doesn't properly return the
> +# site-packages dir in pyvenv, so it is built manually.
> +python_ver = python.language_version()
> +python_site_packages = meson.build_root() / 'pyvenv/lib/python' + python_ver / 'site-packages'
>
> cc = meson.get_compiler('c')
> all_languages = ['c']
> diff --git a/meson_options.txt b/meson_options.txt
> index fff1521e58..5bb41bcbc4 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -36,6 +36,8 @@ option('trace_file', type: 'string', value: 'trace',
> option('coroutine_backend', type: 'combo',
> choices: ['ucontext', 'sigaltstack', 'windows', 'wasm', 'auto'],
> value: 'auto', description: 'coroutine backend to use')
> +option('gdb', type: 'string', value: '',
> + description: 'Path to GDB')
>
> # Everything else can be set via --enable/--disable-* option
> # on the configure script command line. After adding an option
> diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh
> index 0ebe6bc52a..f4bd21220e 100644
> --- a/scripts/meson-buildoptions.sh
> +++ b/scripts/meson-buildoptions.sh
> @@ -58,6 +58,7 @@ meson_options_help() {
> printf "%s\n" ' --enable-ubsan enable undefined behaviour sanitizer'
> printf "%s\n" ' --firmwarepath=VALUES search PATH for firmware files [share/qemu-'
> printf "%s\n" ' firmware]'
> + printf "%s\n" ' --gdb=VALUE Path to GDB'
> printf "%s\n" ' --iasl=VALUE Path to ACPI disassembler'
> printf "%s\n" ' --includedir=VALUE Header file directory [include]'
> printf "%s\n" ' --interp-prefix=VALUE where to find shared libraries etc., use %M for'
> @@ -323,6 +324,7 @@ _meson_option_parse() {
> --disable-fuzzing) printf "%s" -Dfuzzing=false ;;
> --enable-gcrypt) printf "%s" -Dgcrypt=enabled ;;
> --disable-gcrypt) printf "%s" -Dgcrypt=disabled ;;
> + --gdb=*) quote_sh "-Dgdb=$2" ;;
> --enable-gettext) printf "%s" -Dgettext=enabled ;;
> --disable-gettext) printf "%s" -Dgettext=disabled ;;
> --enable-gio) printf "%s" -Dgio=enabled ;;
> diff --git a/tests/functional/meson.build b/tests/functional/meson.build
> index 2a0c5aa141..febd31a263 100644
> --- a/tests/functional/meson.build
> +++ b/tests/functional/meson.build
> @@ -9,6 +9,25 @@ if get_option('tcg_interpreter')
> subdir_done()
> endif
>
> +# Define the GDB runner if GDB is available.
> +gdb = get_option('gdb')
> +if gdb != ''
> + gdb_runner_script = meson.project_source_root() + '/tests/guest-debug/run-test.py'
> + gdb_runner = [gdb_runner_script, '--quiet', '--gdb', gdb, '--test']
> +
> + message('GDB runner defined based on GDB found at: ', gdb)
> +
> + # A test with a runner is a tuple (name, runner, protocol).
> + # The tests must be elements of an array named like:
> + #
> + # test_<arch>_<mode=[system|linuxuser|bsduser]>_<speed=[quick|thorough]>_with_runner = [
> + # ['test0', gdb_runner, 'exitcode'],
> + # ...
> + # ]
> +else
> + message('GDB not found, skipping functional tests that rely on it.')
> +endif
> +
> subdir('aarch64')
> subdir('alpha')
> subdir('arm')
> @@ -61,9 +80,11 @@ foreach speed : ['quick', 'thorough']
> suites = ['func-quick', 'func-' + target_base]
> target_tests = get_variable('tests_' + target_base + '_' + sysmode + '_quick', []) \
> + get_variable('tests_generic_' + sysmode)
> + target_tests_r = get_variable('tests_' + target_base + '_' + sysmode + '_quick_with_runner', [])
> else
> suites = ['func-' + speed, 'func-' + target_base + '-' + speed, speed]
> target_tests = get_variable('tests_' + target_base + '_' + sysmode + '_' + speed, [])
> + target_tests_r = get_variable('tests_' + target_base + '_' + sysmode + '_' + speed + '_with_runner', [])
> endif
>
> test_deps = [roms, keymap_targets]
> @@ -121,6 +142,50 @@ foreach speed : ['quick', 'thorough']
> priority: time_out,
> suite: suites)
> endforeach
> +
> + # Prepare tests that require a specific runner.
> + foreach test : target_tests_r
> + testname = '@0@-@1@'.format(target_base, test[0])
> + testfile = target_base / 'test_' + test[0] + '.py'
> + testpath = meson.current_source_dir() / testfile
> + teststamp = testname + '.tstamp'
> + testrunner = test[1]
> + testproto = test[2]
> +
> + # python_site_packages, i.e., site packages from Python in pyvenv, is
> + # added to PYTHONPATH because some runners can run a program that has its
> + # own Python hooks that, by its turn, will search for modules based on
> + # PYTHONPATH independently of the Python used by the runner, like, for
> + # example, GDB using libpython.
> + test_r_precache_env = test_precache_env
> + test_r_precache_env.append('PYTHONPATH', python_site_packages)
> + # For similar reasons, PYTHONPATH must also include the path to the test
> + # scripts, otherwise unittest's introspection will failed.
> + test_r_precache_env.append('PYTHONPATH', meson.current_source_dir() / target_base)
> +
> + precache = custom_target('func-precache-' + testname,
> + output: teststamp,
> + command: [testrunner, testpath],
> + depend_files: files(testpath),
> + build_by_default: false,
> + env: test_r_precache_env)
> + precache_all += precache
> +
> + # See comments above about PYTHONPATH in test_r_precache_env.
> + test_r_env = test_env
> + test_r_env.append('PYTHONPATH', python_site_packages)
> + test_r_env.append('PYTHONPATH', meson.current_source_dir() / target_base)
> +
> + test('func-' + testname,
> + python,
> + depends: [test_deps, test_emulator, emulator_modules, plugin_modules],
> + env: test_r_env,
> + args: [testrunner, testpath],
> + protocol: testproto,
> + timeout: time_out,
> + priority: time_out,
> + suite: suites)
> + endforeach
> endforeach
> endforeach
It's a little bit unfortunate that we get quite a bit of almost duplicated
code in meson.build this way, but I currently also lack an idea of doing it
in a better way... So from my side:
Acked-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v2 5/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-04 15:46 [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
` (3 preceding siblings ...)
2025-09-04 15:46 ` [PATCH v2 4/5] tests/functional: Support tests that require a runner Gustavo Romero
@ 2025-09-04 15:46 ` Gustavo Romero
2025-09-05 7:21 ` Alex Bennée
2025-09-08 9:16 ` Daniel P. Berrangé
2025-09-08 11:49 ` [PATCH v2 0/5] " Daniel P. Berrangé
2025-09-15 8:29 ` Daniel P. Berrangé
6 siblings, 2 replies; 25+ messages in thread
From: Gustavo Romero @ 2025-09-04 15:46 UTC (permalink / raw)
To: qemu-devel, alex.bennee, thuth, berrange
Cc: qemu-arm, 1844144, gustavo.romero
This commit removes Avocado as a dependency for running the
reverse_debugging test.
The main benefit, beyond eliminating an extra dependency, is that there
is no longer any need to handle GDB packets manually. This removes the
need for ad-hoc functions dealing with endianness and arch-specific
register numbers, making the test easier to read. The timeout variable
is also removed, since Meson now manages timeouts automatically.
The reverse_debugging test is now executed through a runner, because it
requires GDB in addition to QMP. The runner is responsible for invoking
GDB with the appropriate arguments and for passing the test script to
GDB.
Since the test now runs inside GDB, its output, particularly from
'stepi' commands, which cannot be disabled, interleaves with the TAP
output from unittest. To avoid this conflict, the test no longer uses
Meson’s 'tap' protocol and instead relies on the simpler 'exitcode'
protocol.
reverse_debugging is kept "skipped" for aarch64, ppc64, and x86_64, so
won't run unless QEMU_TEST_FLAKY_TESTS=1 is set in the test environment,
before running 'make check-functional' or 'meson test [...]'.
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
---
tests/functional/aarch64/meson.build | 7 +-
.../functional/aarch64/test_reverse_debug.py | 15 +-
tests/functional/ppc64/meson.build | 7 +-
tests/functional/ppc64/test_reverse_debug.py | 17 +-
tests/functional/reverse_debugging.py | 175 +++++++++---------
tests/functional/x86_64/meson.build | 7 +-
tests/functional/x86_64/test_reverse_debug.py | 21 +--
7 files changed, 131 insertions(+), 118 deletions(-)
diff --git a/tests/functional/aarch64/meson.build b/tests/functional/aarch64/meson.build
index 04846c6eb1..4f3168dd55 100644
--- a/tests/functional/aarch64/meson.build
+++ b/tests/functional/aarch64/meson.build
@@ -32,7 +32,6 @@ tests_aarch64_system_thorough = [
'raspi3',
'raspi4',
'replay',
- 'reverse_debug',
'rme_virt',
'rme_sbsaref',
'sbsaref',
@@ -46,3 +45,9 @@ tests_aarch64_system_thorough = [
'xen',
'xlnx_versal',
]
+
+if is_variable('gdb_runner')
+tests_aarch64_system_thorough_with_runner = [
+ ['reverse_debug', gdb_runner, 'exitcode'],
+]
+endif
diff --git a/tests/functional/aarch64/test_reverse_debug.py b/tests/functional/aarch64/test_reverse_debug.py
index 8bc91ccfde..8b6f82c227 100755
--- a/tests/functional/aarch64/test_reverse_debug.py
+++ b/tests/functional/aarch64/test_reverse_debug.py
@@ -1,26 +1,25 @@
-#!/usr/bin/env python3
-#
# SPDX-License-Identifier: GPL-2.0-or-later
#
-# Reverse debugging test
+# Reverse debugging test for aarch64
#
# Copyright (c) 2020 ISP RAS
+# Copyright (c) 2025 Linaro Limited
#
# Author:
# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+# Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from qemu_test import Asset, skipIfMissingImports, skipFlakyTest
+# ReverseDebugging must be imported always first because of the check
+# in it for not running this test without the GDB runner.
from reverse_debugging import ReverseDebugging
+from qemu_test import Asset, skipFlakyTest
-@skipIfMissingImports('avocado.utils')
class ReverseDebugging_AArch64(ReverseDebugging):
- REG_PC = 32
-
ASSET_KERNEL = Asset(
('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
@@ -35,4 +34,4 @@ def test_aarch64_virt(self):
if __name__ == '__main__':
- ReverseDebugging.main()
+ ReverseDebugging_AArch64.main()
diff --git a/tests/functional/ppc64/meson.build b/tests/functional/ppc64/meson.build
index 842fe0fc71..e81f8b0003 100644
--- a/tests/functional/ppc64/meson.build
+++ b/tests/functional/ppc64/meson.build
@@ -20,6 +20,11 @@ tests_ppc64_system_thorough = [
'powernv',
'pseries',
'replay',
- 'reverse_debug',
'tuxrun',
]
+
+if is_variable('gdb_runner')
+tests_ppc64_system_thorough_with_runner = [
+ ['reverse_debug', gdb_runner, 'exitcode'],
+]
+endif
diff --git a/tests/functional/ppc64/test_reverse_debug.py b/tests/functional/ppc64/test_reverse_debug.py
index 5931adef5a..e70ca85d0a 100755
--- a/tests/functional/ppc64/test_reverse_debug.py
+++ b/tests/functional/ppc64/test_reverse_debug.py
@@ -1,41 +1,38 @@
-#!/usr/bin/env python3
-#
# SPDX-License-Identifier: GPL-2.0-or-later
#
-# Reverse debugging test
+# Reverse debugging test for ppc64
#
# Copyright (c) 2020 ISP RAS
+# Copyright (c) 2025 Linaro Limited
#
# Author:
# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+# Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from qemu_test import skipIfMissingImports, skipFlakyTest
+# ReverseDebugging must be imported always first because of the check
+# in it for not running this test without the GDB runner.
from reverse_debugging import ReverseDebugging
+from qemu_test import skipFlakyTest
-@skipIfMissingImports('avocado.utils')
class ReverseDebugging_ppc64(ReverseDebugging):
- REG_PC = 0x40
-
@skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
def test_ppc64_pseries(self):
self.set_machine('pseries')
# SLOF branches back to its entry point, which causes this test
# to take the 'hit a breakpoint again' path. That's not a problem,
# just slightly different than the other machines.
- self.endian_is_le = False
self.reverse_debugging()
@skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
def test_ppc64_powernv(self):
self.set_machine('powernv')
- self.endian_is_le = False
self.reverse_debugging()
if __name__ == '__main__':
- ReverseDebugging.main()
+ ReverseDebugging_ppc64.main()
diff --git a/tests/functional/reverse_debugging.py b/tests/functional/reverse_debugging.py
index f9a1d395f1..bb0b176579 100644
--- a/tests/functional/reverse_debugging.py
+++ b/tests/functional/reverse_debugging.py
@@ -1,16 +1,26 @@
-# Reverse debugging test
-#
# SPDX-License-Identifier: GPL-2.0-or-later
#
+# Reverse debugging test
+#
# Copyright (c) 2020 ISP RAS
+# Copyright (c) 2025 Linaro Limited
#
# Author:
# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+# Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-import os
+
+try:
+ import gdb
+except ModuleNotFoundError:
+ from sys import exit
+ exit("This script must be launched via tests/guest-debug/run-test.py!")
import logging
+import os
+import subprocess
+
from qemu_test import LinuxKernelTest, get_qemu_img
from qemu_test.ports import Ports
@@ -28,13 +38,9 @@ class ReverseDebugging(LinuxKernelTest):
that the execution is stopped at the last of them.
"""
- timeout = 10
STEPS = 10
- endian_is_le = True
def run_vm(self, record, shift, args, replay_path, image_path, port):
- from avocado.utils import datadrainer
-
logger = logging.getLogger('replay')
vm = self.get_vm(name='record' if record else 'replay')
vm.set_console()
@@ -52,59 +58,48 @@ def run_vm(self, record, shift, args, replay_path, image_path, port):
if args:
vm.add_args(*args)
vm.launch()
- console_drainer = datadrainer.LineLogger(vm.console_socket.fileno(),
- logger=self.log.getChild('console'),
- stop_check=(lambda : not vm.is_running()))
- console_drainer.start()
+
return vm
@staticmethod
- def get_reg_le(g, reg):
- res = g.cmd(b'p%x' % reg)
- num = 0
- for i in range(len(res))[-2::-2]:
- num = 0x100 * num + int(res[i:i + 2], 16)
- return num
+ def gdb_connect(host, port):
+ # Set debug on connection to get the qSupport string.
+ gdb.execute("set debug remote 1")
+ r = gdb.execute(f"target remote {host}:{port}", False, True)
+ gdb.execute("set debug remote 0")
+
+ return r
@staticmethod
- def get_reg_be(g, reg):
- res = g.cmd(b'p%x' % reg)
- return int(res, 16)
-
- def get_reg(self, g, reg):
- # value may be encoded in BE or LE order
- if self.endian_is_le:
- return self.get_reg_le(g, reg)
- else:
- return self.get_reg_be(g, reg)
+ def get_pc():
+ val = gdb.parse_and_eval("$pc")
+ pc = int(val)
- def get_pc(self, g):
- return self.get_reg(g, self.REG_PC)
+ return pc
- def check_pc(self, g, addr):
- pc = self.get_pc(g)
+ def check_pc(self, addr):
+ logger = logging.getLogger('reply')
+ pc = self.get_pc()
if pc != addr:
- self.fail('Invalid PC (read %x instead of %x)' % (pc, addr))
+ logger.info('Invalid PC (read %x instead of %x)' % (pc, addr))
+ gdb.execute("exit 1")
@staticmethod
- def gdb_step(g):
- g.cmd(b's', b'T05thread:01;')
+ def gdb_step():
+ gdb.execute("stepi")
@staticmethod
- def gdb_bstep(g):
- g.cmd(b'bs', b'T05thread:01;')
+ def gdb_bstep():
+ gdb.execute("reverse-stepi")
@staticmethod
def vm_get_icount(vm):
return vm.qmp('query-replay')['return']['icount']
def reverse_debugging(self, shift=7, args=None):
- from avocado.utils import gdb
- from avocado.utils import process
-
logger = logging.getLogger('replay')
- # create qcow2 for snapshots
+ # Create qcow2 for snapshots
logger.info('creating qcow2 image for VM snapshots')
image_path = os.path.join(self.workdir, 'disk.qcow2')
qemu_img = get_qemu_img(self)
@@ -112,11 +107,11 @@ def reverse_debugging(self, shift=7, args=None):
self.skipTest('Could not find "qemu-img", which is required to '
'create the temporary qcow2 image')
cmd = '%s create -f qcow2 %s 128M' % (qemu_img, image_path)
- process.run(cmd)
+ subprocess.run(cmd, shell=True)
replay_path = os.path.join(self.workdir, 'replay.bin')
- # record the log
+ # Record the log.
vm = self.run_vm(True, shift, args, replay_path, image_path, -1)
while self.vm_get_icount(vm) <= self.STEPS:
pass
@@ -125,72 +120,84 @@ def reverse_debugging(self, shift=7, args=None):
logger.info("recorded log with %s+ steps" % last_icount)
- # replay and run debug commands
+ # Replay and run debug commands.
with Ports() as ports:
port = ports.find_free_port()
vm = self.run_vm(False, shift, args, replay_path, image_path, port)
- logger.info('connecting to gdbstub')
- g = gdb.GDBRemote('127.0.0.1', port, False, False)
- g.connect()
- r = g.cmd(b'qSupported')
- if b'qXfer:features:read+' in r:
- g.cmd(b'qXfer:features:read:target.xml:0,ffb')
- if b'ReverseStep+' not in r:
+ logger.info('Connecting to gdbstub')
+ r = self.gdb_connect('127.0.0.1', port)
+ if 'ReverseStep+' not in r:
self.fail('Reverse step is not supported by QEMU')
- if b'ReverseContinue+' not in r:
+ if 'ReverseContinue+' not in r:
self.fail('Reverse continue is not supported by QEMU')
- logger.info('stepping forward')
+ logger.info('Stepping forward')
steps = []
- # record first instruction addresses
+ # Record first instruction addresses.
for _ in range(self.STEPS):
- pc = self.get_pc(g)
- logger.info('saving position %x' % pc)
+ pc = self.get_pc()
+ logger.info('Saving position %x' % pc)
steps.append(pc)
- self.gdb_step(g)
+ self.gdb_step()
- # visit the recorded instruction in reverse order
- logger.info('stepping backward')
+ # Visit the recorded instruction in reverse order.
+ logger.info('Stepping backward')
for addr in steps[::-1]:
- self.gdb_bstep(g)
- self.check_pc(g, addr)
- logger.info('found position %x' % addr)
+ self.gdb_bstep()
+ self.check_pc(addr)
+ logger.info('Found position %x' % addr)
- # visit the recorded instruction in forward order
- logger.info('stepping forward')
+ # Visit the recorded instruction in forward order.
+ logger.info('Stepping forward')
for addr in steps:
- self.check_pc(g, addr)
- self.gdb_step(g)
- logger.info('found position %x' % addr)
+ self.check_pc(addr)
+ self.gdb_step()
+ logger.info('Found position %x' % addr)
- # set breakpoints for the instructions just stepped over
- logger.info('setting breakpoints')
+ # Set breakpoints for the instructions just stepped over.
+ logger.info('Setting breakpoints')
for addr in steps:
# hardware breakpoint at addr with len=1
- g.cmd(b'Z1,%x,1' % addr, b'OK')
+ gdb.execute(f"break *{hex(addr)}")
- # this may hit a breakpoint if first instructions are executed
- # again
- logger.info('continuing execution')
+ # This may hit a breakpoint if first instructions are executed again.
+ logger.info('Continuing execution')
vm.qmp('replay-break', icount=last_icount - 1)
- # continue - will return after pausing
+ # continue - will return after pausing.
# This could stop at the end and get a T02 return, or by
# re-executing one of the breakpoints and get a T05 return.
- g.cmd(b'c')
+ gdb.execute("continue")
if self.vm_get_icount(vm) == last_icount - 1:
- logger.info('reached the end (icount %s)' % (last_icount - 1))
+ logger.info('Reached the end (icount %s)' % (last_icount - 1))
else:
- logger.info('hit a breakpoint again at %x (icount %s)' %
+ logger.info('Hit a breakpoint again at %x (icount %s)' %
(self.get_pc(g), self.vm_get_icount(vm)))
- logger.info('running reverse continue to reach %x' % steps[-1])
- # reverse continue - will return after stopping at the breakpoint
- g.cmd(b'bc', b'T05thread:01;')
+ logger.info('Running reverse continue to reach %x' % steps[-1])
+ # reverse continue - will return after stopping at the breakpoint.
+ gdb.execute("reverse-continue")
- # assume that none of the first instructions is executed again
- # breaking the order of the breakpoints
- self.check_pc(g, steps[-1])
- logger.info('successfully reached %x' % steps[-1])
+ # Assume that none of the first instructions is executed again
+ # breaking the order of the breakpoints.
+ # steps[-1] is the first saved $pc in reverse order.
+ self.check_pc(steps[-1])
+ logger.info('Successfully reached %x' % steps[-1])
- logger.info('exiting gdb and qemu')
+ logger.info('Exiting GDB and QEMU...')
+ # Disconnect from the VM.
+ gdb.execute("disconnect")
+ # Guarantee VM is shutdown.
vm.shutdown()
+ # Gently exit from GDB.
+ gdb.execute("exit 0")
+
+ def main():
+ try:
+ LinuxKernelTest.main()
+ except SystemExit:
+ # If the test is marked with @skipFlakyTest, then it will be exited
+ # via sys.exit() before we have the chance to exit from GDB gently.
+ # Because recent versions of GDB will return a failure value if this
+ # happens, we catch the SystemExit and exit from GDB gently with 77,
+ # which meson interprets correctly as a skipped test.
+ gdb.execute("exit 77")
diff --git a/tests/functional/x86_64/meson.build b/tests/functional/x86_64/meson.build
index d0b4667bb8..b5e0c9a0d7 100644
--- a/tests/functional/x86_64/meson.build
+++ b/tests/functional/x86_64/meson.build
@@ -29,8 +29,13 @@ tests_x86_64_system_thorough = [
'multiprocess',
'netdev_ethtool',
'replay',
- 'reverse_debug',
'tuxrun',
'virtio_balloon',
'virtio_gpu',
]
+
+if is_variable('gdb_runner')
+ tests_x86_64_system_thorough_with_runner = [
+ ['reverse_debug', gdb_runner, 'exitcode'],
+ ]
+endif
diff --git a/tests/functional/x86_64/test_reverse_debug.py b/tests/functional/x86_64/test_reverse_debug.py
index d713e91e14..465f7e0abb 100755
--- a/tests/functional/x86_64/test_reverse_debug.py
+++ b/tests/functional/x86_64/test_reverse_debug.py
@@ -1,36 +1,31 @@
-#!/usr/bin/env python3
-#
# SPDX-License-Identifier: GPL-2.0-or-later
#
-# Reverse debugging test
+# Reverse debugging test for x86_64
#
# Copyright (c) 2020 ISP RAS
+# Copyright (c) 2025 Linaro Limited
#
# Author:
# Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
+# Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
-from qemu_test import skipIfMissingImports, skipFlakyTest
+# ReverseDebugging must be imported always first because of the check
+# in it for not running this test without the GDB runner.
from reverse_debugging import ReverseDebugging
+from qemu_test import skipFlakyTest
-@skipIfMissingImports('avocado.utils')
class ReverseDebugging_X86_64(ReverseDebugging):
- REG_PC = 0x10
- REG_CS = 0x12
- def get_pc(self, g):
- return self.get_reg_le(g, self.REG_PC) \
- + self.get_reg_le(g, self.REG_CS) * 0x10
-
@skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2922")
def test_x86_64_pc(self):
self.set_machine('pc')
- # start with BIOS only
+ # Start with BIOS only
self.reverse_debugging()
if __name__ == '__main__':
- ReverseDebugging.main()
+ ReverseDebugging_X86_64.main()
--
2.34.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v2 5/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-04 15:46 ` [PATCH v2 5/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
@ 2025-09-05 7:21 ` Alex Bennée
2025-09-08 9:16 ` Daniel P. Berrangé
1 sibling, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2025-09-05 7:21 UTC (permalink / raw)
To: Gustavo Romero; +Cc: qemu-devel, thuth, berrange, qemu-arm, 1844144
Gustavo Romero <gustavo.romero@linaro.org> writes:
> This commit removes Avocado as a dependency for running the
> reverse_debugging test.
>
> The main benefit, beyond eliminating an extra dependency, is that there
> is no longer any need to handle GDB packets manually. This removes the
> need for ad-hoc functions dealing with endianness and arch-specific
> register numbers, making the test easier to read. The timeout variable
> is also removed, since Meson now manages timeouts automatically.
>
> The reverse_debugging test is now executed through a runner, because it
> requires GDB in addition to QMP. The runner is responsible for invoking
> GDB with the appropriate arguments and for passing the test script to
> GDB.
>
> Since the test now runs inside GDB, its output, particularly from
> 'stepi' commands, which cannot be disabled, interleaves with the TAP
> output from unittest. To avoid this conflict, the test no longer uses
> Meson’s 'tap' protocol and instead relies on the simpler 'exitcode'
> protocol.
>
> reverse_debugging is kept "skipped" for aarch64, ppc64, and x86_64, so
> won't run unless QEMU_TEST_FLAKY_TESTS=1 is set in the test environment,
> before running 'make check-functional' or 'meson test [...]'.
>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
Could we split this between the architectures to make for better
bisectability. Otherwise looks good:
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/functional/aarch64/meson.build | 7 +-
> .../functional/aarch64/test_reverse_debug.py | 15 +-
> tests/functional/ppc64/meson.build | 7 +-
> tests/functional/ppc64/test_reverse_debug.py | 17 +-
> tests/functional/reverse_debugging.py | 175 +++++++++---------
> tests/functional/x86_64/meson.build | 7 +-
> tests/functional/x86_64/test_reverse_debug.py | 21 +--
> 7 files changed, 131 insertions(+), 118 deletions(-)
>
> diff --git a/tests/functional/aarch64/meson.build b/tests/functional/aarch64/meson.build
> index 04846c6eb1..4f3168dd55 100644
> --- a/tests/functional/aarch64/meson.build
> +++ b/tests/functional/aarch64/meson.build
> @@ -32,7 +32,6 @@ tests_aarch64_system_thorough = [
> 'raspi3',
> 'raspi4',
> 'replay',
> - 'reverse_debug',
> 'rme_virt',
> 'rme_sbsaref',
> 'sbsaref',
> @@ -46,3 +45,9 @@ tests_aarch64_system_thorough = [
> 'xen',
> 'xlnx_versal',
> ]
> +
> +if is_variable('gdb_runner')
> +tests_aarch64_system_thorough_with_runner = [
> + ['reverse_debug', gdb_runner, 'exitcode'],
> +]
> +endif
> diff --git a/tests/functional/aarch64/test_reverse_debug.py b/tests/functional/aarch64/test_reverse_debug.py
> index 8bc91ccfde..8b6f82c227 100755
> --- a/tests/functional/aarch64/test_reverse_debug.py
> +++ b/tests/functional/aarch64/test_reverse_debug.py
> @@ -1,26 +1,25 @@
> -#!/usr/bin/env python3
> -#
> # SPDX-License-Identifier: GPL-2.0-or-later
> #
> -# Reverse debugging test
> +# Reverse debugging test for aarch64
> #
> # Copyright (c) 2020 ISP RAS
> +# Copyright (c) 2025 Linaro Limited
> #
> # Author:
> # Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
> +# Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
> #
> # This work is licensed under the terms of the GNU GPL, version 2 or
> # later. See the COPYING file in the top-level directory.
>
> -from qemu_test import Asset, skipIfMissingImports, skipFlakyTest
> +# ReverseDebugging must be imported always first because of the check
> +# in it for not running this test without the GDB runner.
> from reverse_debugging import ReverseDebugging
> +from qemu_test import Asset, skipFlakyTest
>
>
> -@skipIfMissingImports('avocado.utils')
> class ReverseDebugging_AArch64(ReverseDebugging):
>
> - REG_PC = 32
> -
> ASSET_KERNEL = Asset(
> ('https://archives.fedoraproject.org/pub/archive/fedora/linux/'
> 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz'),
> @@ -35,4 +34,4 @@ def test_aarch64_virt(self):
>
>
> if __name__ == '__main__':
> - ReverseDebugging.main()
> + ReverseDebugging_AArch64.main()
> diff --git a/tests/functional/ppc64/meson.build b/tests/functional/ppc64/meson.build
> index 842fe0fc71..e81f8b0003 100644
> --- a/tests/functional/ppc64/meson.build
> +++ b/tests/functional/ppc64/meson.build
> @@ -20,6 +20,11 @@ tests_ppc64_system_thorough = [
> 'powernv',
> 'pseries',
> 'replay',
> - 'reverse_debug',
> 'tuxrun',
> ]
> +
> +if is_variable('gdb_runner')
> +tests_ppc64_system_thorough_with_runner = [
> + ['reverse_debug', gdb_runner, 'exitcode'],
> +]
> +endif
> diff --git a/tests/functional/ppc64/test_reverse_debug.py b/tests/functional/ppc64/test_reverse_debug.py
> index 5931adef5a..e70ca85d0a 100755
> --- a/tests/functional/ppc64/test_reverse_debug.py
> +++ b/tests/functional/ppc64/test_reverse_debug.py
> @@ -1,41 +1,38 @@
> -#!/usr/bin/env python3
> -#
> # SPDX-License-Identifier: GPL-2.0-or-later
> #
> -# Reverse debugging test
> +# Reverse debugging test for ppc64
> #
> # Copyright (c) 2020 ISP RAS
> +# Copyright (c) 2025 Linaro Limited
> #
> # Author:
> # Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
> +# Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
> #
> # This work is licensed under the terms of the GNU GPL, version 2 or
> # later. See the COPYING file in the top-level directory.
>
> -from qemu_test import skipIfMissingImports, skipFlakyTest
> +# ReverseDebugging must be imported always first because of the check
> +# in it for not running this test without the GDB runner.
> from reverse_debugging import ReverseDebugging
> +from qemu_test import skipFlakyTest
>
>
> -@skipIfMissingImports('avocado.utils')
> class ReverseDebugging_ppc64(ReverseDebugging):
>
> - REG_PC = 0x40
> -
> @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
> def test_ppc64_pseries(self):
> self.set_machine('pseries')
> # SLOF branches back to its entry point, which causes this test
> # to take the 'hit a breakpoint again' path. That's not a problem,
> # just slightly different than the other machines.
> - self.endian_is_le = False
> self.reverse_debugging()
>
> @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/1992")
> def test_ppc64_powernv(self):
> self.set_machine('powernv')
> - self.endian_is_le = False
> self.reverse_debugging()
>
>
> if __name__ == '__main__':
> - ReverseDebugging.main()
> + ReverseDebugging_ppc64.main()
> diff --git a/tests/functional/reverse_debugging.py b/tests/functional/reverse_debugging.py
> index f9a1d395f1..bb0b176579 100644
> --- a/tests/functional/reverse_debugging.py
> +++ b/tests/functional/reverse_debugging.py
> @@ -1,16 +1,26 @@
> -# Reverse debugging test
> -#
> # SPDX-License-Identifier: GPL-2.0-or-later
> #
> +# Reverse debugging test
> +#
> # Copyright (c) 2020 ISP RAS
> +# Copyright (c) 2025 Linaro Limited
> #
> # Author:
> # Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
> +# Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
> #
> # This work is licensed under the terms of the GNU GPL, version 2 or
> # later. See the COPYING file in the top-level directory.
> -import os
> +
> +try:
> + import gdb
> +except ModuleNotFoundError:
> + from sys import exit
> + exit("This script must be launched via tests/guest-debug/run-test.py!")
> import logging
> +import os
> +import subprocess
> +
>
> from qemu_test import LinuxKernelTest, get_qemu_img
> from qemu_test.ports import Ports
> @@ -28,13 +38,9 @@ class ReverseDebugging(LinuxKernelTest):
> that the execution is stopped at the last of them.
> """
>
> - timeout = 10
> STEPS = 10
> - endian_is_le = True
>
> def run_vm(self, record, shift, args, replay_path, image_path, port):
> - from avocado.utils import datadrainer
> -
> logger = logging.getLogger('replay')
> vm = self.get_vm(name='record' if record else 'replay')
> vm.set_console()
> @@ -52,59 +58,48 @@ def run_vm(self, record, shift, args, replay_path, image_path, port):
> if args:
> vm.add_args(*args)
> vm.launch()
> - console_drainer = datadrainer.LineLogger(vm.console_socket.fileno(),
> - logger=self.log.getChild('console'),
> - stop_check=(lambda : not vm.is_running()))
> - console_drainer.start()
> +
> return vm
>
> @staticmethod
> - def get_reg_le(g, reg):
> - res = g.cmd(b'p%x' % reg)
> - num = 0
> - for i in range(len(res))[-2::-2]:
> - num = 0x100 * num + int(res[i:i + 2], 16)
> - return num
> + def gdb_connect(host, port):
> + # Set debug on connection to get the qSupport string.
> + gdb.execute("set debug remote 1")
> + r = gdb.execute(f"target remote {host}:{port}", False, True)
> + gdb.execute("set debug remote 0")
> +
> + return r
>
> @staticmethod
> - def get_reg_be(g, reg):
> - res = g.cmd(b'p%x' % reg)
> - return int(res, 16)
> -
> - def get_reg(self, g, reg):
> - # value may be encoded in BE or LE order
> - if self.endian_is_le:
> - return self.get_reg_le(g, reg)
> - else:
> - return self.get_reg_be(g, reg)
> + def get_pc():
> + val = gdb.parse_and_eval("$pc")
> + pc = int(val)
>
> - def get_pc(self, g):
> - return self.get_reg(g, self.REG_PC)
> + return pc
>
> - def check_pc(self, g, addr):
> - pc = self.get_pc(g)
> + def check_pc(self, addr):
> + logger = logging.getLogger('reply')
> + pc = self.get_pc()
> if pc != addr:
> - self.fail('Invalid PC (read %x instead of %x)' % (pc, addr))
> + logger.info('Invalid PC (read %x instead of %x)' % (pc, addr))
> + gdb.execute("exit 1")
>
> @staticmethod
> - def gdb_step(g):
> - g.cmd(b's', b'T05thread:01;')
> + def gdb_step():
> + gdb.execute("stepi")
>
> @staticmethod
> - def gdb_bstep(g):
> - g.cmd(b'bs', b'T05thread:01;')
> + def gdb_bstep():
> + gdb.execute("reverse-stepi")
>
> @staticmethod
> def vm_get_icount(vm):
> return vm.qmp('query-replay')['return']['icount']
>
> def reverse_debugging(self, shift=7, args=None):
> - from avocado.utils import gdb
> - from avocado.utils import process
> -
> logger = logging.getLogger('replay')
>
> - # create qcow2 for snapshots
> + # Create qcow2 for snapshots
> logger.info('creating qcow2 image for VM snapshots')
> image_path = os.path.join(self.workdir, 'disk.qcow2')
> qemu_img = get_qemu_img(self)
> @@ -112,11 +107,11 @@ def reverse_debugging(self, shift=7, args=None):
> self.skipTest('Could not find "qemu-img", which is required to '
> 'create the temporary qcow2 image')
> cmd = '%s create -f qcow2 %s 128M' % (qemu_img, image_path)
> - process.run(cmd)
> + subprocess.run(cmd, shell=True)
>
> replay_path = os.path.join(self.workdir, 'replay.bin')
>
> - # record the log
> + # Record the log.
> vm = self.run_vm(True, shift, args, replay_path, image_path, -1)
> while self.vm_get_icount(vm) <= self.STEPS:
> pass
> @@ -125,72 +120,84 @@ def reverse_debugging(self, shift=7, args=None):
>
> logger.info("recorded log with %s+ steps" % last_icount)
>
> - # replay and run debug commands
> + # Replay and run debug commands.
> with Ports() as ports:
> port = ports.find_free_port()
> vm = self.run_vm(False, shift, args, replay_path, image_path, port)
> - logger.info('connecting to gdbstub')
> - g = gdb.GDBRemote('127.0.0.1', port, False, False)
> - g.connect()
> - r = g.cmd(b'qSupported')
> - if b'qXfer:features:read+' in r:
> - g.cmd(b'qXfer:features:read:target.xml:0,ffb')
> - if b'ReverseStep+' not in r:
> + logger.info('Connecting to gdbstub')
> + r = self.gdb_connect('127.0.0.1', port)
> + if 'ReverseStep+' not in r:
> self.fail('Reverse step is not supported by QEMU')
> - if b'ReverseContinue+' not in r:
> + if 'ReverseContinue+' not in r:
> self.fail('Reverse continue is not supported by QEMU')
>
> - logger.info('stepping forward')
> + logger.info('Stepping forward')
> steps = []
> - # record first instruction addresses
> + # Record first instruction addresses.
> for _ in range(self.STEPS):
> - pc = self.get_pc(g)
> - logger.info('saving position %x' % pc)
> + pc = self.get_pc()
> + logger.info('Saving position %x' % pc)
> steps.append(pc)
> - self.gdb_step(g)
> + self.gdb_step()
>
> - # visit the recorded instruction in reverse order
> - logger.info('stepping backward')
> + # Visit the recorded instruction in reverse order.
> + logger.info('Stepping backward')
> for addr in steps[::-1]:
> - self.gdb_bstep(g)
> - self.check_pc(g, addr)
> - logger.info('found position %x' % addr)
> + self.gdb_bstep()
> + self.check_pc(addr)
> + logger.info('Found position %x' % addr)
>
> - # visit the recorded instruction in forward order
> - logger.info('stepping forward')
> + # Visit the recorded instruction in forward order.
> + logger.info('Stepping forward')
> for addr in steps:
> - self.check_pc(g, addr)
> - self.gdb_step(g)
> - logger.info('found position %x' % addr)
> + self.check_pc(addr)
> + self.gdb_step()
> + logger.info('Found position %x' % addr)
>
> - # set breakpoints for the instructions just stepped over
> - logger.info('setting breakpoints')
> + # Set breakpoints for the instructions just stepped over.
> + logger.info('Setting breakpoints')
> for addr in steps:
> # hardware breakpoint at addr with len=1
> - g.cmd(b'Z1,%x,1' % addr, b'OK')
> + gdb.execute(f"break *{hex(addr)}")
>
> - # this may hit a breakpoint if first instructions are executed
> - # again
> - logger.info('continuing execution')
> + # This may hit a breakpoint if first instructions are executed again.
> + logger.info('Continuing execution')
> vm.qmp('replay-break', icount=last_icount - 1)
> - # continue - will return after pausing
> + # continue - will return after pausing.
> # This could stop at the end and get a T02 return, or by
> # re-executing one of the breakpoints and get a T05 return.
> - g.cmd(b'c')
> + gdb.execute("continue")
> if self.vm_get_icount(vm) == last_icount - 1:
> - logger.info('reached the end (icount %s)' % (last_icount - 1))
> + logger.info('Reached the end (icount %s)' % (last_icount - 1))
> else:
> - logger.info('hit a breakpoint again at %x (icount %s)' %
> + logger.info('Hit a breakpoint again at %x (icount %s)' %
> (self.get_pc(g), self.vm_get_icount(vm)))
>
> - logger.info('running reverse continue to reach %x' % steps[-1])
> - # reverse continue - will return after stopping at the breakpoint
> - g.cmd(b'bc', b'T05thread:01;')
> + logger.info('Running reverse continue to reach %x' % steps[-1])
> + # reverse continue - will return after stopping at the breakpoint.
> + gdb.execute("reverse-continue")
>
> - # assume that none of the first instructions is executed again
> - # breaking the order of the breakpoints
> - self.check_pc(g, steps[-1])
> - logger.info('successfully reached %x' % steps[-1])
> + # Assume that none of the first instructions is executed again
> + # breaking the order of the breakpoints.
> + # steps[-1] is the first saved $pc in reverse order.
> + self.check_pc(steps[-1])
> + logger.info('Successfully reached %x' % steps[-1])
>
> - logger.info('exiting gdb and qemu')
> + logger.info('Exiting GDB and QEMU...')
> + # Disconnect from the VM.
> + gdb.execute("disconnect")
> + # Guarantee VM is shutdown.
> vm.shutdown()
> + # Gently exit from GDB.
> + gdb.execute("exit 0")
> +
> + def main():
> + try:
> + LinuxKernelTest.main()
> + except SystemExit:
> + # If the test is marked with @skipFlakyTest, then it will be exited
> + # via sys.exit() before we have the chance to exit from GDB gently.
> + # Because recent versions of GDB will return a failure value if this
> + # happens, we catch the SystemExit and exit from GDB gently with 77,
> + # which meson interprets correctly as a skipped test.
> + gdb.execute("exit 77")
> diff --git a/tests/functional/x86_64/meson.build b/tests/functional/x86_64/meson.build
> index d0b4667bb8..b5e0c9a0d7 100644
> --- a/tests/functional/x86_64/meson.build
> +++ b/tests/functional/x86_64/meson.build
> @@ -29,8 +29,13 @@ tests_x86_64_system_thorough = [
> 'multiprocess',
> 'netdev_ethtool',
> 'replay',
> - 'reverse_debug',
> 'tuxrun',
> 'virtio_balloon',
> 'virtio_gpu',
> ]
> +
> +if is_variable('gdb_runner')
> + tests_x86_64_system_thorough_with_runner = [
> + ['reverse_debug', gdb_runner, 'exitcode'],
> + ]
> +endif
> diff --git a/tests/functional/x86_64/test_reverse_debug.py b/tests/functional/x86_64/test_reverse_debug.py
> index d713e91e14..465f7e0abb 100755
> --- a/tests/functional/x86_64/test_reverse_debug.py
> +++ b/tests/functional/x86_64/test_reverse_debug.py
> @@ -1,36 +1,31 @@
> -#!/usr/bin/env python3
> -#
> # SPDX-License-Identifier: GPL-2.0-or-later
> #
> -# Reverse debugging test
> +# Reverse debugging test for x86_64
> #
> # Copyright (c) 2020 ISP RAS
> +# Copyright (c) 2025 Linaro Limited
> #
> # Author:
> # Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
> +# Gustavo Romero <gustavo.romero@linaro.org> (Run without Avocado)
> #
> # This work is licensed under the terms of the GNU GPL, version 2 or
> # later. See the COPYING file in the top-level directory.
>
> -from qemu_test import skipIfMissingImports, skipFlakyTest
> +# ReverseDebugging must be imported always first because of the check
> +# in it for not running this test without the GDB runner.
> from reverse_debugging import ReverseDebugging
> +from qemu_test import skipFlakyTest
>
>
> -@skipIfMissingImports('avocado.utils')
> class ReverseDebugging_X86_64(ReverseDebugging):
>
> - REG_PC = 0x10
> - REG_CS = 0x12
> - def get_pc(self, g):
> - return self.get_reg_le(g, self.REG_PC) \
> - + self.get_reg_le(g, self.REG_CS) * 0x10
> -
> @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/2922")
> def test_x86_64_pc(self):
> self.set_machine('pc')
> - # start with BIOS only
> + # Start with BIOS only
> self.reverse_debugging()
>
>
> if __name__ == '__main__':
> - ReverseDebugging.main()
> + ReverseDebugging_X86_64.main()
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 5/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-04 15:46 ` [PATCH v2 5/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
2025-09-05 7:21 ` Alex Bennée
@ 2025-09-08 9:16 ` Daniel P. Berrangé
2025-09-11 23:50 ` Gustavo Romero
1 sibling, 1 reply; 25+ messages in thread
From: Daniel P. Berrangé @ 2025-09-08 9:16 UTC (permalink / raw)
To: Gustavo Romero; +Cc: qemu-devel, alex.bennee, thuth, qemu-arm, 1844144
On Thu, Sep 04, 2025 at 03:46:40PM +0000, Gustavo Romero wrote:
> This commit removes Avocado as a dependency for running the
> reverse_debugging test.
>
> The main benefit, beyond eliminating an extra dependency, is that there
> is no longer any need to handle GDB packets manually. This removes the
> need for ad-hoc functions dealing with endianness and arch-specific
> register numbers, making the test easier to read. The timeout variable
> is also removed, since Meson now manages timeouts automatically.
>
> The reverse_debugging test is now executed through a runner, because it
> requires GDB in addition to QMP. The runner is responsible for invoking
> GDB with the appropriate arguments and for passing the test script to
> GDB.
>
> Since the test now runs inside GDB, its output, particularly from
> 'stepi' commands, which cannot be disabled, interleaves with the TAP
> output from unittest. To avoid this conflict, the test no longer uses
> Meson’s 'tap' protocol and instead relies on the simpler 'exitcode'
> protocol.
>
> reverse_debugging is kept "skipped" for aarch64, ppc64, and x86_64, so
> won't run unless QEMU_TEST_FLAKY_TESTS=1 is set in the test environment,
> before running 'make check-functional' or 'meson test [...]'.
>
> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
> ---
> tests/functional/aarch64/meson.build | 7 +-
> .../functional/aarch64/test_reverse_debug.py | 15 +-
> tests/functional/ppc64/meson.build | 7 +-
> tests/functional/ppc64/test_reverse_debug.py | 17 +-
> tests/functional/reverse_debugging.py | 175 +++++++++---------
> tests/functional/x86_64/meson.build | 7 +-
> tests/functional/x86_64/test_reverse_debug.py | 21 +--
> 7 files changed, 131 insertions(+), 118 deletions(-)
>
> diff --git a/tests/functional/aarch64/meson.build b/tests/functional/aarch64/meson.build
> index 04846c6eb1..4f3168dd55 100644
> --- a/tests/functional/aarch64/meson.build
> +++ b/tests/functional/aarch64/meson.build
> @@ -32,7 +32,6 @@ tests_aarch64_system_thorough = [
> 'raspi3',
> 'raspi4',
> 'replay',
> - 'reverse_debug',
> 'rme_virt',
> 'rme_sbsaref',
> 'sbsaref',
> @@ -46,3 +45,9 @@ tests_aarch64_system_thorough = [
> 'xen',
> 'xlnx_versal',
> ]
> +
> +if is_variable('gdb_runner')
> +tests_aarch64_system_thorough_with_runner = [
> + ['reverse_debug', gdb_runner, 'exitcode'],
> +]
> +endif
Why do you need to replace the of 'tap' with 'exitcode' for this script ?
I don't like the idea that a subset of our functional tests are not using
the TAP protocol.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 5/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-08 9:16 ` Daniel P. Berrangé
@ 2025-09-11 23:50 ` Gustavo Romero
0 siblings, 0 replies; 25+ messages in thread
From: Gustavo Romero @ 2025-09-11 23:50 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: qemu-devel, alex.bennee, thuth, qemu-arm
Hi Daniel!
On 9/8/25 06:16, Daniel P. Berrangé wrote:
> On Thu, Sep 04, 2025 at 03:46:40PM +0000, Gustavo Romero wrote:
>> This commit removes Avocado as a dependency for running the
>> reverse_debugging test.
>>
>> The main benefit, beyond eliminating an extra dependency, is that there
>> is no longer any need to handle GDB packets manually. This removes the
>> need for ad-hoc functions dealing with endianness and arch-specific
>> register numbers, making the test easier to read. The timeout variable
>> is also removed, since Meson now manages timeouts automatically.
>>
>> The reverse_debugging test is now executed through a runner, because it
>> requires GDB in addition to QMP. The runner is responsible for invoking
>> GDB with the appropriate arguments and for passing the test script to
>> GDB.
>>
>> Since the test now runs inside GDB, its output, particularly from
>> 'stepi' commands, which cannot be disabled, interleaves with the TAP
>> output from unittest. To avoid this conflict, the test no longer uses
>> Meson’s 'tap' protocol and instead relies on the simpler 'exitcode'
>> protocol.
>>
>> reverse_debugging is kept "skipped" for aarch64, ppc64, and x86_64, so
>> won't run unless QEMU_TEST_FLAKY_TESTS=1 is set in the test environment,
>> before running 'make check-functional' or 'meson test [...]'.
>>
>> Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
>> ---
>> tests/functional/aarch64/meson.build | 7 +-
>> .../functional/aarch64/test_reverse_debug.py | 15 +-
>> tests/functional/ppc64/meson.build | 7 +-
>> tests/functional/ppc64/test_reverse_debug.py | 17 +-
>> tests/functional/reverse_debugging.py | 175 +++++++++---------
>> tests/functional/x86_64/meson.build | 7 +-
>> tests/functional/x86_64/test_reverse_debug.py | 21 +--
>> 7 files changed, 131 insertions(+), 118 deletions(-)
>>
>> diff --git a/tests/functional/aarch64/meson.build b/tests/functional/aarch64/meson.build
>> index 04846c6eb1..4f3168dd55 100644
>> --- a/tests/functional/aarch64/meson.build
>> +++ b/tests/functional/aarch64/meson.build
>> @@ -32,7 +32,6 @@ tests_aarch64_system_thorough = [
>> 'raspi3',
>> 'raspi4',
>> 'replay',
>> - 'reverse_debug',
>> 'rme_virt',
>> 'rme_sbsaref',
>> 'sbsaref',
>> @@ -46,3 +45,9 @@ tests_aarch64_system_thorough = [
>> 'xen',
>> 'xlnx_versal',
>> ]
>> +
>> +if is_variable('gdb_runner')
>> +tests_aarch64_system_thorough_with_runner = [
>> + ['reverse_debug', gdb_runner, 'exitcode'],
>> +]
>> +endif
>
> Why do you need to replace the of 'tap' with 'exitcode' for this script ?
>
> I don't like the idea that a subset of our functional tests are not using
> the TAP protocol.
Unfortunately, the GDB Python API seems to be broken regarding being able
to redirect GDB's output to a file (as it's possible to do via GDB's prompt)
so we can't redirect the printed addresses that are printed after each "stepi"
command in the test script. As a consequence Meson's TAP gets confused with
it and fails the test. Since I could not find any directive in QEMU functional
tests that forbid using the classic exitcode, I used it, at least while
the GDB redirect is not fixed.
Why don't you like it?
Cheers,
Gustavo
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-04 15:46 [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
` (4 preceding siblings ...)
2025-09-04 15:46 ` [PATCH v2 5/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
@ 2025-09-08 11:49 ` Daniel P. Berrangé
2025-09-11 23:51 ` Gustavo Romero
2025-09-15 8:29 ` Daniel P. Berrangé
6 siblings, 1 reply; 25+ messages in thread
From: Daniel P. Berrangé @ 2025-09-08 11:49 UTC (permalink / raw)
To: Gustavo Romero; +Cc: qemu-devel, alex.bennee, thuth, qemu-arm, 1844144
On Thu, Sep 04, 2025 at 03:46:35PM +0000, Gustavo Romero wrote:
> In this series, we leveraged the run-test.py script used in the
> check-tcg tests, making it a GDB runner capable of calling a test script
> without spawning any VMs. In this configuration, the test scripts can
> manage the VM and also import gdb, making the GDB Python API inside the
> functional test scripts.
>
> A --quiet option has been added to run-test.py so it doesn't print the
> command line used to execute GDB to the stdout. This ensures that users
> don't get confused about how to re-run the tests. One can re-run the
> test simply by copying and pasting the command line shown by Meson when
> V=1 is passed:
>
> $ make -j check-functional V=1
>
> or, alternatively, once the test run completes, the exact command found
> in the 'command:' field of the build/meson-logs/testlog-thorough.txt
> file generated by Meson. Both methods provide the correct environment
> variables required to run the test, such as the proper $PYTHONPATH.
While I like the conceptual idea of just sending human GDB commands,
instead of working with GDB protocol packets, I really dislike the
effect this has on the execution / startup of the functional tests
via use of the custom runner for a number of reasons
* The command line for launching the test outside of meson is very
complicated, so not memorable
* It makes the meson.build rules much more complicated
* Running standalone there is no TAP output available making the
test hard to debug on failure or timeout
I understand the need to spawn the test via gdb, in order to be able
to import the 'gdb' python module. Looking at what reverse_debugging.py
does, however, makes me question whether we actually need to directly
use the 'gdb' python module.
The only APIs we use are 'gdb.execute' and 'gdb.parse_and_eval'.
The latter is only used once as
gdb.parse_and_eval("$pc")
and I believe that can be changed to
gdb.execute("printf \"0x%x\", $pc", to_string=True)
IOW, all we need is 'gdb.execute("....", to_string=True)'
With a little extra helper proxy script, we can achieve this without
changing the way scripts are launched.
The script needs to listen on a UNIX socket path. When a client
connects, it should read lines of data from the client and pass
them to 'gdb.execute(..., to_string=True)' and whatever data
gdb returns should be written back to the client.
A very very crude example with no error handling would be:
#!/usr/bin/python3
import gdb
import os
import socket
sock = os.environ.get("QEMU_PROXY", "/tmp/qemu.gdb.proxy")
try:
os.unlink(sock)
except:
pass
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
s.bind(sock)
s.listen()
conn, addr = s.accept()
fh = conn.makefile('rw')
with conn:
while True:
line = fh.readline()
if not line:
break
data = gdb.execute(line, to_string=True)
fh.write(data)
fh.flush()
In the functional test suite, we should have a helper file
tests/functional/qemu_test/gdb.py that provides an API for
launching GDB to execute this proxy script, and an API to
execute commands by talking over this UNIX socket path.
With this, we will need no changes in the way we execute the
reverse debugging script from a test runner POV, thus avoiding
all the downsides of use of the run-test.py script. IOW, the
first 4 patches in this series go away completely. Instead we
need a patch to create the proxy script and a patch to create
the helper APIs in tests/functional/qemu_test/gdb.py, whereupon
the last patch can replace
try:
import gdb
except ModuleNotFoundError:
from sys import exit
exit("This script must be launched via tests/guest-debug/run-test.py!")
with
from qemu_test import gdb
and the earlier mentioned replacement of parse_and_eval()
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-08 11:49 ` [PATCH v2 0/5] " Daniel P. Berrangé
@ 2025-09-11 23:51 ` Gustavo Romero
2025-09-12 14:49 ` Daniel P. Berrangé
0 siblings, 1 reply; 25+ messages in thread
From: Gustavo Romero @ 2025-09-11 23:51 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: qemu-devel, alex.bennee, thuth, qemu-arm
Hi Daniel,
Thanks a lot for review and the suggestions.
On 9/8/25 08:49, Daniel P. Berrangé wrote:
> On Thu, Sep 04, 2025 at 03:46:35PM +0000, Gustavo Romero wrote:
>> In this series, we leveraged the run-test.py script used in the
>> check-tcg tests, making it a GDB runner capable of calling a test script
>> without spawning any VMs. In this configuration, the test scripts can
>> manage the VM and also import gdb, making the GDB Python API inside the
>> functional test scripts.
>>
>> A --quiet option has been added to run-test.py so it doesn't print the
>> command line used to execute GDB to the stdout. This ensures that users
>> don't get confused about how to re-run the tests. One can re-run the
>> test simply by copying and pasting the command line shown by Meson when
>> V=1 is passed:
>>
>> $ make -j check-functional V=1
>>
>> or, alternatively, once the test run completes, the exact command found
>> in the 'command:' field of the build/meson-logs/testlog-thorough.txt
>> file generated by Meson. Both methods provide the correct environment
>> variables required to run the test, such as the proper $PYTHONPATH.
>
> While I like the conceptual idea of just sending human GDB commands,
> instead of working with GDB protocol packets, I really dislike the
> effect this has on the execution / startup of the functional tests
> via use of the custom runner for a number of reasons
>
> * The command line for launching the test outside of meson is very
> complicated, so not memorable
Why very complicated? It calls a simple runner instead of calling the
test script directly, but it doesn't change the way to re-run a single
test. One just have to pass V=1 to see make's command line and copy
and paste the full command line to re-run the test. I mentioned
inspecting 'testlog-thorough.txt' just for completeness.
> * It makes the meson.build rules much more complicated
Do we want to never augment functional tests' meson.build? Nothing
complicated is being added. Basically, just a new variable suffixed with
'_with_runner' which holds a tuple (test, runner) that tell the test
to be executed, following the same logic we already have for all the other
variables that specify the tests per arch/mode/speed.
Another option would be to select a runner based on a suffix in the test
name, for instance, 'reverse_debug_with_runner.py'.
> * Running standalone there is no TAP output available making the
> test hard to debug on failure or timeout
This is because of an unfortunate GDB Python API issue, please see my
reply in your comment on patch 5/5. This can be solved but needs more
investigation on GDB side.
> I understand the need to spawn the test via gdb, in order to be able
> to import the 'gdb' python module. Looking at what reverse_debugging.py
> does, however, makes me question whether we actually need to directly
> use the 'gdb' python module.
>
> The only APIs we use are 'gdb.execute' and 'gdb.parse_and_eval'.
>
> The latter is only used once as
>
> gdb.parse_and_eval("$pc")
>
> and I believe that can be changed to
>
> gdb.execute("printf \"0x%x\", $pc", to_string=True)
>
> IOW, all we need is 'gdb.execute("....", to_string=True)'
Yes, I do want to directly use the 'gdb' python module directly in the
tests. We shouldn't look at a solution only for reverse_debug.py but also
think of any future tests that will require the GDB Python API, so I don't
want to specialize here and reduce the API to a single method.
> With a little extra helper proxy script, we can achieve this without
> changing the way scripts are launched.
>
> The script needs to listen on a UNIX socket path. When a client
> connects, it should read lines of data from the client and pass
> them to 'gdb.execute(..., to_string=True)' and whatever data
> gdb returns should be written back to the client.
>
> A very very crude example with no error handling would be:
>
> #!/usr/bin/python3
>
> import gdb
> import os
> import socket
>
> sock = os.environ.get("QEMU_PROXY", "/tmp/qemu.gdb.proxy")
>
> try:
> os.unlink(sock)
> except:
> pass
>
> with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
> s.bind(sock)
> s.listen()
> conn, addr = s.accept()
> fh = conn.makefile('rw')
> with conn:
> while True:
> line = fh.readline()
> if not line:
> break
> data = gdb.execute(line, to_string=True)
> fh.write(data)
> fh.flush()
>
>
> In the functional test suite, we should have a helper file
> tests/functional/qemu_test/gdb.py that provides an API for
> launching GDB to execute this proxy script, and an API to
> execute commands by talking over this UNIX socket path.
>
> With this, we will need no changes in the way we execute the
> reverse debugging script from a test runner POV, thus avoiding
> all the downsides of use of the run-test.py script. IOW, the
> first 4 patches in this series go away completely. Instead we
> need a patch to create the proxy script and a patch to create
> the helper APIs in tests/functional/qemu_test/gdb.py, whereupon
> the last patch can replace
>
> try:
> import gdb
> except ModuleNotFoundError:
> from sys import exit
> exit("This script must be launched via tests/guest-debug/run-test.py!")
>
> with
>
> from qemu_test import gdb
>
> and the earlier mentioned replacement of parse_and_eval()
For the sake of not adding a few lines into meson.build, we are going
to design a new ad-hoc API for the functional tests on top of the GDB
Python API, which will communicate with the test script via a socket
and will _still require a runner anyway_ (just now hidden under a
module/API)? This is far more complicated than having a simple runner
to call GDB and pass the test script.
In fact, I think that if the test script had any clue in its name,
like the "_with_runner" suffix I mentioned above, maybe Meson's could
take care of calling GDB itself without calling any runner. Would that
address your first comment in the bullets (and maybe the second one too,
but not sure without trying it) and get this series accepted by you,
since the third one, about the exitcode, is related to GDB's odd behavior?
Cheers,
Gustavo
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-11 23:51 ` Gustavo Romero
@ 2025-09-12 14:49 ` Daniel P. Berrangé
2025-09-12 16:04 ` Alex Bennée
0 siblings, 1 reply; 25+ messages in thread
From: Daniel P. Berrangé @ 2025-09-12 14:49 UTC (permalink / raw)
To: Gustavo Romero; +Cc: qemu-devel, alex.bennee, thuth, qemu-arm
On Thu, Sep 11, 2025 at 08:51:08PM -0300, Gustavo Romero wrote:
> Hi Daniel,
>
> Thanks a lot for review and the suggestions.
>
> On 9/8/25 08:49, Daniel P. Berrangé wrote:
> > On Thu, Sep 04, 2025 at 03:46:35PM +0000, Gustavo Romero wrote:
> > > In this series, we leveraged the run-test.py script used in the
> > > check-tcg tests, making it a GDB runner capable of calling a test script
> > > without spawning any VMs. In this configuration, the test scripts can
> > > manage the VM and also import gdb, making the GDB Python API inside the
> > > functional test scripts.
> > >
> > > A --quiet option has been added to run-test.py so it doesn't print the
> > > command line used to execute GDB to the stdout. This ensures that users
> > > don't get confused about how to re-run the tests. One can re-run the
> > > test simply by copying and pasting the command line shown by Meson when
> > > V=1 is passed:
> > >
> > > $ make -j check-functional V=1
> > >
> > > or, alternatively, once the test run completes, the exact command found
> > > in the 'command:' field of the build/meson-logs/testlog-thorough.txt
> > > file generated by Meson. Both methods provide the correct environment
> > > variables required to run the test, such as the proper $PYTHONPATH.
> >
> > While I like the conceptual idea of just sending human GDB commands,
> > instead of working with GDB protocol packets, I really dislike the
> > effect this has on the execution / startup of the functional tests
> > via use of the custom runner for a number of reasons
> >
> > * The command line for launching the test outside of meson is very
> > complicated, so not memorable
>
> Why very complicated? It calls a simple runner instead of calling the
> test script directly, but it doesn't change the way to re-run a single
> test. One just have to pass V=1 to see make's command line and copy
> and paste the full command line to re-run the test. I mentioned
> inspecting 'testlog-thorough.txt' just for completeness.
Today we can run the individual tests directly
# ./tests/functional/x86_64/test_reverse_debug.py
TAP version 13
ok 1 test_reverse_debug.ReverseDebugging_X86_64.test_x86_64_pc
1..1
(assuming you have PYTHONPATH and QEMU_TEST_QEMU_BINARY env set)
This gives you a very easy way to interact with the test, see
its progress, understand what failed, and debug it with strace,
etc.
This change looses all that. It appears I can run it with
# ./tests/guest-debug/run-test.py --quiet --gdb gdb --test \
./tests/functional/x86_64/test_reverse_debug.py
but the output is unintelligible
TAP version 13
0x000000000000fff0 in ?? ()
0x000000000000e05b in ?? ()
0x000000000000e062 in ?? ()
0x000000000000e066 in ?? ()
0x000000000000e068 in ?? ()
0x000000000000e06a in ?? ()
0x000000000000e070 in ?? ()
0x000000000000e076 in ?? ()
0x000000000000cf30 in ?? ()
0x000000000000cf31 in ?? ()
0x000000000000cf32 in ?? ()
0x000000000000cf31 in ?? ()
0x000000000000cf30 in ?? ()
0x000000000000e076 in ?? ()
0x000000000000e070 in ?? ()
0x000000000000e06a in ?? ()
0x000000000000e068 in ?? ()
0x000000000000e066 in ?? ()
0x000000000000e062 in ?? ()
0x000000000000e05b in ?? ()
0x000000000000fff0 in ?? ()
0x000000000000e05b in ?? ()
0x000000000000e062 in ?? ()
0x000000000000e066 in ?? ()
0x000000000000e068 in ?? ()
0x000000000000e06a in ?? ()
0x000000000000e070 in ?? ()
0x000000000000e076 in ?? ()
0x000000000000cf30 in ?? ()
0x000000000000cf31 in ?? ()
0x000000000000cf32 in ?? ()
Breakpoint 1 at 0xfff0
Breakpoint 2 at 0xe05b
Breakpoint 3 at 0xe062
Breakpoint 4 at 0xe066
Breakpoint 5 at 0xe068
Breakpoint 6 at 0xe06a
Breakpoint 7 at 0xe070
Breakpoint 8 at 0xe076
Breakpoint 9 at 0xcf30
Breakpoint 10 at 0xcf31
Program received signal SIGINT, Interrupt.
0x00000000000d80dc in ?? ()
Breakpoint 1, 0x000000000000fff0 in ?? ()
[Inferior 1 (process 1) detached]
This undermines the core goals of what we aimed to achieve with
the new functional test harness.
>
> > * It makes the meson.build rules much more complicated
>
> Do we want to never augment functional tests' meson.build? Nothing
> complicated is being added. Basically, just a new variable suffixed with
> '_with_runner' which holds a tuple (test, runner) that tell the test
> to be executed, following the same logic we already have for all the other
> variables that specify the tests per arch/mode/speed.
>
> Another option would be to select a runner based on a suffix in the test
> name, for instance, 'reverse_debug_with_runner.py'.
IMHO the overall concept of using the run-test.py runner for launching
the tests is flawed and not viable. It adds too much complexity to the
use of the tests, and harms the output.
> > * Running standalone there is no TAP output available making the
> > test hard to debug on failure or timeout
>
> This is because of an unfortunate GDB Python API issue, please see my
> reply in your comment on patch 5/5. This can be solved but needs more
> investigation on GDB side.
>
>
> > I understand the need to spawn the test via gdb, in order to be able
> > to import the 'gdb' python module. Looking at what reverse_debugging.py
> > does, however, makes me question whether we actually need to directly
> > use the 'gdb' python module.
> >
> > The only APIs we use are 'gdb.execute' and 'gdb.parse_and_eval'.
> >
> > The latter is only used once as
> >
> > gdb.parse_and_eval("$pc")
> >
> > and I believe that can be changed to
> >
> > gdb.execute("printf \"0x%x\", $pc", to_string=True)
> >
> > IOW, all we need is 'gdb.execute("....", to_string=True)'
>
> Yes, I do want to directly use the 'gdb' python module directly in the
> tests. We shouldn't look at a solution only for reverse_debug.py but also
> think of any future tests that will require the GDB Python API, so I don't
> want to specialize here and reduce the API to a single method.
If any other tests needing GDB arrive int he future we can consider
them at that time.
I like the idea of the test being able to execute human gdb commands,
but I don't think the GDB provided 'gdb' module is viable to use
directly. We need to retain control over how we launch our tests
without intermediate runners present.
> > With a little extra helper proxy script, we can achieve this without
> > changing the way scripts are launched.
> >
> > The script needs to listen on a UNIX socket path. When a client
> > connects, it should read lines of data from the client and pass
> > them to 'gdb.execute(..., to_string=True)' and whatever data
> > gdb returns should be written back to the client.
> >
> > A very very crude example with no error handling would be:
> >
> > #!/usr/bin/python3
> >
> > import gdb
> > import os
> > import socket
> >
> > sock = os.environ.get("QEMU_PROXY", "/tmp/qemu.gdb.proxy")
> >
> > try:
> > os.unlink(sock)
> > except:
> > pass
> >
> > with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
> > s.bind(sock)
> > s.listen()
> > conn, addr = s.accept()
> > fh = conn.makefile('rw')
> > with conn:
> > while True:
> > line = fh.readline()
> > if not line:
> > break
> > data = gdb.execute(line, to_string=True)
> > fh.write(data)
> > fh.flush()
> >
> >
> > In the functional test suite, we should have a helper file
> > tests/functional/qemu_test/gdb.py that provides an API for
> > launching GDB to execute this proxy script, and an API to
> > execute commands by talking over this UNIX socket path.
> >
> > With this, we will need no changes in the way we execute the
> > reverse debugging script from a test runner POV, thus avoiding
> > all the downsides of use of the run-test.py script. IOW, the
> > first 4 patches in this series go away completely. Instead we
> > need a patch to create the proxy script and a patch to create
> > the helper APIs in tests/functional/qemu_test/gdb.py, whereupon
> > the last patch can replace
> >
> > try:
> > import gdb
> > except ModuleNotFoundError:
> > from sys import exit
> > exit("This script must be launched via tests/guest-debug/run-test.py!")
> >
> > with
> >
> > from qemu_test import gdb
> >
> > and the earlier mentioned replacement of parse_and_eval()
>
> For the sake of not adding a few lines into meson.build, we are going
> to design a new ad-hoc API for the functional tests on top of the GDB
> Python API, which will communicate with the test script via a socket
> and will _still require a runner anyway_ (just now hidden under a
> module/API)? This is far more complicated than having a simple runner
> to call GDB and pass the test script.
This is not exclusively about the meson.build changes. It is about the
overall execution environment of the tests being *simple* and easy to
understand. That is the overrriding goal of how we approached design
of the new functional test harness that made it valuable to spend the
time to replace avocado. The GDB runner usage undermines the benefits
we have achieved.
> In fact, I think that if the test script had any clue in its name,
> like the "_with_runner" suffix I mentioned above, maybe Meson's could
> take care of calling GDB itself without calling any runner. Would that
> address your first comment in the bullets (and maybe the second one too,
> but not sure without trying it) and get this series accepted by you,
> since the third one, about the exitcode, is related to GDB's odd behavior?
The tests need to be runnable directly as standalone python programs,
with well formed TAP output.
Given the limitations of GDB, if we want to use its python module, then
the proxy idea I describe above is the only way forward I see.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-12 14:49 ` Daniel P. Berrangé
@ 2025-09-12 16:04 ` Alex Bennée
2025-09-12 16:27 ` Daniel P. Berrangé
0 siblings, 1 reply; 25+ messages in thread
From: Alex Bennée @ 2025-09-12 16:04 UTC (permalink / raw)
To: Daniel P. Berrangé; +Cc: Gustavo Romero, qemu-devel, thuth, qemu-arm
Daniel P. Berrangé <berrange@redhat.com> writes:
> On Thu, Sep 11, 2025 at 08:51:08PM -0300, Gustavo Romero wrote:
>> Hi Daniel,
>>
>> Thanks a lot for review and the suggestions.
>>
>> On 9/8/25 08:49, Daniel P. Berrangé wrote:
>> > On Thu, Sep 04, 2025 at 03:46:35PM +0000, Gustavo Romero wrote:
>> > > In this series, we leveraged the run-test.py script used in the
>> > > check-tcg tests, making it a GDB runner capable of calling a test script
>> > > without spawning any VMs. In this configuration, the test scripts can
>> > > manage the VM and also import gdb, making the GDB Python API inside the
>> > > functional test scripts.
>> > >
>> > > A --quiet option has been added to run-test.py so it doesn't print the
>> > > command line used to execute GDB to the stdout. This ensures that users
>> > > don't get confused about how to re-run the tests. One can re-run the
>> > > test simply by copying and pasting the command line shown by Meson when
>> > > V=1 is passed:
>> > >
>> > > $ make -j check-functional V=1
>> > >
>> > > or, alternatively, once the test run completes, the exact command found
>> > > in the 'command:' field of the build/meson-logs/testlog-thorough.txt
>> > > file generated by Meson. Both methods provide the correct environment
>> > > variables required to run the test, such as the proper $PYTHONPATH.
>> >
>> > While I like the conceptual idea of just sending human GDB commands,
>> > instead of working with GDB protocol packets, I really dislike the
>> > effect this has on the execution / startup of the functional tests
>> > via use of the custom runner for a number of reasons
>> >
>> > * The command line for launching the test outside of meson is very
>> > complicated, so not memorable
>>
>> Why very complicated? It calls a simple runner instead of calling the
>> test script directly, but it doesn't change the way to re-run a single
>> test. One just have to pass V=1 to see make's command line and copy
>> and paste the full command line to re-run the test. I mentioned
>> inspecting 'testlog-thorough.txt' just for completeness.
>
> Today we can run the individual tests directly
>
> # ./tests/functional/x86_64/test_reverse_debug.py
> TAP version 13
> ok 1 test_reverse_debug.ReverseDebugging_X86_64.test_x86_64_pc
> 1..1
>
>
> (assuming you have PYTHONPATH and QEMU_TEST_QEMU_BINARY env set)
and the old version of Avocado...
> This gives you a very easy way to interact with the test, see
> its progress, understand what failed, and debug it with strace,
> etc.
>
> This change looses all that. It appears I can run it with
>
> # ./tests/guest-debug/run-test.py --quiet --gdb gdb --test \
> ./tests/functional/x86_64/test_reverse_debug.py
>
<snip>
>
>
> This undermines the core goals of what we aimed to achieve with
> the new functional test harness.
>
>>
>> > * It makes the meson.build rules much more complicated
>>
>> Do we want to never augment functional tests' meson.build? Nothing
>> complicated is being added. Basically, just a new variable suffixed with
>> '_with_runner' which holds a tuple (test, runner) that tell the test
>> to be executed, following the same logic we already have for all the other
>> variables that specify the tests per arch/mode/speed.
>>
>> Another option would be to select a runner based on a suffix in the test
>> name, for instance, 'reverse_debug_with_runner.py'.
>
> IMHO the overall concept of using the run-test.py runner for launching
> the tests is flawed and not viable. It adds too much complexity to the
> use of the tests, and harms the output.
>
>> > * Running standalone there is no TAP output available making the
>> > test hard to debug on failure or timeout
>>
>> This is because of an unfortunate GDB Python API issue, please see my
>> reply in your comment on patch 5/5. This can be solved but needs more
>> investigation on GDB side.
>>
>>
>> > I understand the need to spawn the test via gdb, in order to be able
>> > to import the 'gdb' python module. Looking at what reverse_debugging.py
>> > does, however, makes me question whether we actually need to directly
>> > use the 'gdb' python module.
>> >
>> > The only APIs we use are 'gdb.execute' and 'gdb.parse_and_eval'.
>> >
>> > The latter is only used once as
>> >
>> > gdb.parse_and_eval("$pc")
>> >
>> > and I believe that can be changed to
>> >
>> > gdb.execute("printf \"0x%x\", $pc", to_string=True)
>> >
>> > IOW, all we need is 'gdb.execute("....", to_string=True)'
>>
>> Yes, I do want to directly use the 'gdb' python module directly in the
>> tests. We shouldn't look at a solution only for reverse_debug.py but also
>> think of any future tests that will require the GDB Python API, so I don't
>> want to specialize here and reduce the API to a single method.
>
> If any other tests needing GDB arrive int he future we can consider
> them at that time.
We already have a whole chunk of gdb tests under check-tcg. Maybe it
would be easier just to re-write the tests to use the check-tcg system
tests rather than jumping through hoops to fit in with the
check-functional requirements.
The only downside is that we miss out on having async device events as
the check-tcg tests are super simple. But the record/replay tests should
pick up on sync errors so perhaps its orthogonal to defending the
reverse-step/continue functionality?
> I like the idea of the test being able to execute human gdb commands,
> but I don't think the GDB provided 'gdb' module is viable to use
> directly. We need to retain control over how we launch our tests
> without intermediate runners present.
>
>> > With a little extra helper proxy script, we can achieve this without
>> > changing the way scripts are launched.
>> >
>> > The script needs to listen on a UNIX socket path. When a client
>> > connects, it should read lines of data from the client and pass
>> > them to 'gdb.execute(..., to_string=True)' and whatever data
>> > gdb returns should be written back to the client.
>> >
>> > A very very crude example with no error handling would be:
My concern is it probably isn't quite that simple - and we have just
invented YAGMI (Yet Another GDB Machine Interface) module. The fact that
we have no widely packaged python gdb interface is probably a testament
to the edge cases that exist.
>> >
>> > #!/usr/bin/python3
>> >
>> > import gdb
>> > import os
>> > import socket
>> >
>> > sock = os.environ.get("QEMU_PROXY", "/tmp/qemu.gdb.proxy")
>> >
>> > try:
>> > os.unlink(sock)
>> > except:
>> > pass
>> >
>> > with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
>> > s.bind(sock)
>> > s.listen()
>> > conn, addr = s.accept()
>> > fh = conn.makefile('rw')
>> > with conn:
>> > while True:
>> > line = fh.readline()
>> > if not line:
>> > break
>> > data = gdb.execute(line, to_string=True)
>> > fh.write(data)
>> > fh.flush()
>> >
>> >
>> > In the functional test suite, we should have a helper file
>> > tests/functional/qemu_test/gdb.py that provides an API for
>> > launching GDB to execute this proxy script, and an API to
>> > execute commands by talking over this UNIX socket path.
>> >
>> > With this, we will need no changes in the way we execute the
>> > reverse debugging script from a test runner POV, thus avoiding
>> > all the downsides of use of the run-test.py script. IOW, the
>> > first 4 patches in this series go away completely. Instead we
>> > need a patch to create the proxy script and a patch to create
>> > the helper APIs in tests/functional/qemu_test/gdb.py, whereupon
>> > the last patch can replace
>> >
>> > try:
>> > import gdb
>> > except ModuleNotFoundError:
>> > from sys import exit
>> > exit("This script must be launched via tests/guest-debug/run-test.py!")
>> >
>> > with
>> >
>> > from qemu_test import gdb
>> >
>> > and the earlier mentioned replacement of parse_and_eval()
>>
>> For the sake of not adding a few lines into meson.build, we are going
>> to design a new ad-hoc API for the functional tests on top of the GDB
>> Python API, which will communicate with the test script via a socket
>> and will _still require a runner anyway_ (just now hidden under a
>> module/API)? This is far more complicated than having a simple runner
>> to call GDB and pass the test script.
>
> This is not exclusively about the meson.build changes. It is about the
> overall execution environment of the tests being *simple* and easy to
> understand. That is the overrriding goal of how we approached design
> of the new functional test harness that made it valuable to spend the
> time to replace avocado. The GDB runner usage undermines the benefits
> we have achieved.
>
>> In fact, I think that if the test script had any clue in its name,
>> like the "_with_runner" suffix I mentioned above, maybe Meson's could
>> take care of calling GDB itself without calling any runner. Would that
>> address your first comment in the bullets (and maybe the second one too,
>> but not sure without trying it) and get this series accepted by you,
>> since the third one, about the exitcode, is related to GDB's odd behavior?
>
> The tests need to be runnable directly as standalone python programs,
> with well formed TAP output.
>
> Given the limitations of GDB, if we want to use its python module, then
> the proxy idea I describe above is the only way forward I see.
>
> With regards,
> Daniel
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-12 16:04 ` Alex Bennée
@ 2025-09-12 16:27 ` Daniel P. Berrangé
2025-09-12 16:50 ` Peter Maydell
2025-09-15 12:49 ` Thomas Huth
0 siblings, 2 replies; 25+ messages in thread
From: Daniel P. Berrangé @ 2025-09-12 16:27 UTC (permalink / raw)
To: Alex Bennée; +Cc: Gustavo Romero, qemu-devel, thuth, qemu-arm
On Fri, Sep 12, 2025 at 05:04:40PM +0100, Alex Bennée wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
>
> > On Thu, Sep 11, 2025 at 08:51:08PM -0300, Gustavo Romero wrote:
> >> Hi Daniel,
> >>
> >> Thanks a lot for review and the suggestions.
> >>
> >> On 9/8/25 08:49, Daniel P. Berrangé wrote:
> >> > On Thu, Sep 04, 2025 at 03:46:35PM +0000, Gustavo Romero wrote:
> >> > > In this series, we leveraged the run-test.py script used in the
> >> > > check-tcg tests, making it a GDB runner capable of calling a test script
> >> > > without spawning any VMs. In this configuration, the test scripts can
> >> > > manage the VM and also import gdb, making the GDB Python API inside the
> >> > > functional test scripts.
> >> > >
> >> > > A --quiet option has been added to run-test.py so it doesn't print the
> >> > > command line used to execute GDB to the stdout. This ensures that users
> >> > > don't get confused about how to re-run the tests. One can re-run the
> >> > > test simply by copying and pasting the command line shown by Meson when
> >> > > V=1 is passed:
> >> > >
> >> > > $ make -j check-functional V=1
> >> > >
> >> > > or, alternatively, once the test run completes, the exact command found
> >> > > in the 'command:' field of the build/meson-logs/testlog-thorough.txt
> >> > > file generated by Meson. Both methods provide the correct environment
> >> > > variables required to run the test, such as the proper $PYTHONPATH.
> >> >
> >> > While I like the conceptual idea of just sending human GDB commands,
> >> > instead of working with GDB protocol packets, I really dislike the
> >> > effect this has on the execution / startup of the functional tests
> >> > via use of the custom runner for a number of reasons
> >> >
> >> > * The command line for launching the test outside of meson is very
> >> > complicated, so not memorable
> >>
> >> Why very complicated? It calls a simple runner instead of calling the
> >> test script directly, but it doesn't change the way to re-run a single
> >> test. One just have to pass V=1 to see make's command line and copy
> >> and paste the full command line to re-run the test. I mentioned
> >> inspecting 'testlog-thorough.txt' just for completeness.
> >
> > Today we can run the individual tests directly
> >
> > # ./tests/functional/x86_64/test_reverse_debug.py
> > TAP version 13
> > ok 1 test_reverse_debug.ReverseDebugging_X86_64.test_x86_64_pc
> > 1..1
> >
> >
> > (assuming you have PYTHONPATH and QEMU_TEST_QEMU_BINARY env set)
>
> and the old version of Avocado...
>
> > This gives you a very easy way to interact with the test, see
> > its progress, understand what failed, and debug it with strace,
> > etc.
> >
> > This change looses all that. It appears I can run it with
> >
> > # ./tests/guest-debug/run-test.py --quiet --gdb gdb --test \
> > ./tests/functional/x86_64/test_reverse_debug.py
> >
> <snip>
> >
> >
> > This undermines the core goals of what we aimed to achieve with
> > the new functional test harness.
> >
> >>
> >> > * It makes the meson.build rules much more complicated
> >>
> >> Do we want to never augment functional tests' meson.build? Nothing
> >> complicated is being added. Basically, just a new variable suffixed with
> >> '_with_runner' which holds a tuple (test, runner) that tell the test
> >> to be executed, following the same logic we already have for all the other
> >> variables that specify the tests per arch/mode/speed.
> >>
> >> Another option would be to select a runner based on a suffix in the test
> >> name, for instance, 'reverse_debug_with_runner.py'.
> >
> > IMHO the overall concept of using the run-test.py runner for launching
> > the tests is flawed and not viable. It adds too much complexity to the
> > use of the tests, and harms the output.
>
>
>
> >
> >> > * Running standalone there is no TAP output available making the
> >> > test hard to debug on failure or timeout
> >>
> >> This is because of an unfortunate GDB Python API issue, please see my
> >> reply in your comment on patch 5/5. This can be solved but needs more
> >> investigation on GDB side.
> >>
> >>
> >> > I understand the need to spawn the test via gdb, in order to be able
> >> > to import the 'gdb' python module. Looking at what reverse_debugging.py
> >> > does, however, makes me question whether we actually need to directly
> >> > use the 'gdb' python module.
> >> >
> >> > The only APIs we use are 'gdb.execute' and 'gdb.parse_and_eval'.
> >> >
> >> > The latter is only used once as
> >> >
> >> > gdb.parse_and_eval("$pc")
> >> >
> >> > and I believe that can be changed to
> >> >
> >> > gdb.execute("printf \"0x%x\", $pc", to_string=True)
> >> >
> >> > IOW, all we need is 'gdb.execute("....", to_string=True)'
> >>
> >> Yes, I do want to directly use the 'gdb' python module directly in the
> >> tests. We shouldn't look at a solution only for reverse_debug.py but also
> >> think of any future tests that will require the GDB Python API, so I don't
> >> want to specialize here and reduce the API to a single method.
> >
> > If any other tests needing GDB arrive int he future we can consider
> > them at that time.
>
> We already have a whole chunk of gdb tests under check-tcg. Maybe it
> would be easier just to re-write the tests to use the check-tcg system
> tests rather than jumping through hoops to fit in with the
> check-functional requirements.
Well if 'easy' is our goal, then we can just copy the gdbmi.py
file from avocado into our test suite. It didn't originate in
avocado to begin with, they copied it from its orignal repo
to then port it to py3. The only real downside with gdbmi is
that the machine level protocol is ugly to read, but I'll take
that over this current patch, as I think the reverse debugging
stuff is a match for the functional test suite, and moving to
the tcg tests still leaves us with the unpleasant interaction
and debugging issue that I've described - they're just hidden
from the functional test suite, but still impacting QEMU as a
whole.
> >> > With a little extra helper proxy script, we can achieve this without
> >> > changing the way scripts are launched.
> >> >
> >> > The script needs to listen on a UNIX socket path. When a client
> >> > connects, it should read lines of data from the client and pass
> >> > them to 'gdb.execute(..., to_string=True)' and whatever data
> >> > gdb returns should be written back to the client.
> >> >
> >> > A very very crude example with no error handling would be:
>
> My concern is it probably isn't quite that simple - and we have just
> invented YAGMI (Yet Another GDB Machine Interface) module. The fact that
> we have no widely packaged python gdb interface is probably a testament
> to the edge cases that exist.
I agree it isn't simple if the TCG tests are in scope, but from the POV
to the reverse debugging functional test it looks simple.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-12 16:27 ` Daniel P. Berrangé
@ 2025-09-12 16:50 ` Peter Maydell
2025-09-15 12:49 ` Thomas Huth
1 sibling, 0 replies; 25+ messages in thread
From: Peter Maydell @ 2025-09-12 16:50 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Alex Bennée, Gustavo Romero, qemu-devel, thuth, qemu-arm
On Fri, 12 Sept 2025 at 17:27, Daniel P. Berrangé <berrange@redhat.com> wrote:
> Well if 'easy' is our goal, then we can just copy the gdbmi.py
> file from avocado into our test suite. It didn't originate in
> avocado to begin with, they copied it from its orignal repo
> to then port it to py3. The only real downside with gdbmi is
> that the machine level protocol is ugly to read, but I'll take
> that over this current patch, as I think the reverse debugging
> stuff is a match for the functional test suite, and moving to
> the tcg tests still leaves us with the unpleasant interaction
> and debugging issue that I've described - they're just hidden
> from the functional test suite, but still impacting QEMU as a
> whole.
If we're doing programmatic driving of GDB then the machine
interface sounds like the better choice for the task anyway...
-- PMM
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-12 16:27 ` Daniel P. Berrangé
2025-09-12 16:50 ` Peter Maydell
@ 2025-09-15 12:49 ` Thomas Huth
2025-09-15 22:11 ` Gustavo Romero
1 sibling, 1 reply; 25+ messages in thread
From: Thomas Huth @ 2025-09-15 12:49 UTC (permalink / raw)
To: Daniel P. Berrangé, Alex Bennée
Cc: Gustavo Romero, qemu-devel, qemu-arm
On 12/09/2025 18.27, Daniel P. Berrangé wrote:
> On Fri, Sep 12, 2025 at 05:04:40PM +0100, Alex Bennée wrote:
>> Daniel P. Berrangé <berrange@redhat.com> writes:
>>
>>> On Thu, Sep 11, 2025 at 08:51:08PM -0300, Gustavo Romero wrote:
>>>> Hi Daniel,
>>>>
>>>> Thanks a lot for review and the suggestions.
>>>>
>>>> On 9/8/25 08:49, Daniel P. Berrangé wrote:
>>>>> On Thu, Sep 04, 2025 at 03:46:35PM +0000, Gustavo Romero wrote:
>>>>>> In this series, we leveraged the run-test.py script used in the
>>>>>> check-tcg tests, making it a GDB runner capable of calling a test script
>>>>>> without spawning any VMs. In this configuration, the test scripts can
>>>>>> manage the VM and also import gdb, making the GDB Python API inside the
>>>>>> functional test scripts.
>>>>>>
>>>>>> A --quiet option has been added to run-test.py so it doesn't print the
>>>>>> command line used to execute GDB to the stdout. This ensures that users
>>>>>> don't get confused about how to re-run the tests. One can re-run the
>>>>>> test simply by copying and pasting the command line shown by Meson when
>>>>>> V=1 is passed:
>>>>>>
>>>>>> $ make -j check-functional V=1
>>>>>>
>>>>>> or, alternatively, once the test run completes, the exact command found
>>>>>> in the 'command:' field of the build/meson-logs/testlog-thorough.txt
>>>>>> file generated by Meson. Both methods provide the correct environment
>>>>>> variables required to run the test, such as the proper $PYTHONPATH.
>>>>>
>>>>> While I like the conceptual idea of just sending human GDB commands,
>>>>> instead of working with GDB protocol packets, I really dislike the
>>>>> effect this has on the execution / startup of the functional tests
>>>>> via use of the custom runner for a number of reasons
>>>>>
>>>>> * The command line for launching the test outside of meson is very
>>>>> complicated, so not memorable
>>>>
>>>> Why very complicated? It calls a simple runner instead of calling the
>>>> test script directly, but it doesn't change the way to re-run a single
>>>> test. One just have to pass V=1 to see make's command line and copy
>>>> and paste the full command line to re-run the test. I mentioned
>>>> inspecting 'testlog-thorough.txt' just for completeness.
>>>
>>> Today we can run the individual tests directly
>>>
>>> # ./tests/functional/x86_64/test_reverse_debug.py
>>> TAP version 13
>>> ok 1 test_reverse_debug.ReverseDebugging_X86_64.test_x86_64_pc
>>> 1..1
>>>
>>>
>>> (assuming you have PYTHONPATH and QEMU_TEST_QEMU_BINARY env set)
>>
>> and the old version of Avocado...
>>
>>> This gives you a very easy way to interact with the test, see
>>> its progress, understand what failed, and debug it with strace,
>>> etc.
>>>
>>> This change looses all that. It appears I can run it with
>>>
>>> # ./tests/guest-debug/run-test.py --quiet --gdb gdb --test \
>>> ./tests/functional/x86_64/test_reverse_debug.py
>>>
>> <snip>
>>>
>>>
>>> This undermines the core goals of what we aimed to achieve with
>>> the new functional test harness.
>>>
>>>>
>>>>> * It makes the meson.build rules much more complicated
>>>>
>>>> Do we want to never augment functional tests' meson.build? Nothing
>>>> complicated is being added. Basically, just a new variable suffixed with
>>>> '_with_runner' which holds a tuple (test, runner) that tell the test
>>>> to be executed, following the same logic we already have for all the other
>>>> variables that specify the tests per arch/mode/speed.
>>>>
>>>> Another option would be to select a runner based on a suffix in the test
>>>> name, for instance, 'reverse_debug_with_runner.py'.
>>>
>>> IMHO the overall concept of using the run-test.py runner for launching
>>> the tests is flawed and not viable. It adds too much complexity to the
>>> use of the tests, and harms the output.
>>
>>
>>
>>>
>>>>> * Running standalone there is no TAP output available making the
>>>>> test hard to debug on failure or timeout
>>>>
>>>> This is because of an unfortunate GDB Python API issue, please see my
>>>> reply in your comment on patch 5/5. This can be solved but needs more
>>>> investigation on GDB side.
>>>>
>>>>
>>>>> I understand the need to spawn the test via gdb, in order to be able
>>>>> to import the 'gdb' python module. Looking at what reverse_debugging.py
>>>>> does, however, makes me question whether we actually need to directly
>>>>> use the 'gdb' python module.
>>>>>
>>>>> The only APIs we use are 'gdb.execute' and 'gdb.parse_and_eval'.
>>>>>
>>>>> The latter is only used once as
>>>>>
>>>>> gdb.parse_and_eval("$pc")
>>>>>
>>>>> and I believe that can be changed to
>>>>>
>>>>> gdb.execute("printf \"0x%x\", $pc", to_string=True)
>>>>>
>>>>> IOW, all we need is 'gdb.execute("....", to_string=True)'
>>>>
>>>> Yes, I do want to directly use the 'gdb' python module directly in the
>>>> tests. We shouldn't look at a solution only for reverse_debug.py but also
>>>> think of any future tests that will require the GDB Python API, so I don't
>>>> want to specialize here and reduce the API to a single method.
>>>
>>> If any other tests needing GDB arrive int he future we can consider
>>> them at that time.
>>
>> We already have a whole chunk of gdb tests under check-tcg. Maybe it
>> would be easier just to re-write the tests to use the check-tcg system
>> tests rather than jumping through hoops to fit in with the
>> check-functional requirements.
>
> Well if 'easy' is our goal, then we can just copy the gdbmi.py
> file from avocado into our test suite.
But maintaining that stuff in the QEMU repository is also kind of ugly.
I took another stab at the problem:
https://lore.kernel.org/qemu-devel/20250915124207.42053-1-thuth@redhat.com/
It's basically Gustavo's patches, but I removed all the stuff around
tests/guest-debug/run-test.py and the cumbersome code for running a test
through gdb via meson.build.
The test start in normal pycotap mode, then call a new function called
reverse_debug() which then takes care of calling gdb with the right
arguments (i.e. this logic has been copied from the run-test.py script).
It seems to work fine with the aarch64 test already, but details and other
architectures still need some more love.
WDYT? Is it worth to pursue that approach?
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-15 12:49 ` Thomas Huth
@ 2025-09-15 22:11 ` Gustavo Romero
0 siblings, 0 replies; 25+ messages in thread
From: Gustavo Romero @ 2025-09-15 22:11 UTC (permalink / raw)
To: Thomas Huth, Daniel P. Berrangé, Alex Bennée
Cc: qemu-devel, qemu-arm
Hi Thomas and folks,
On 9/15/25 09:49, Thomas Huth wrote:
> On 12/09/2025 18.27, Daniel P. Berrangé wrote:
>> On Fri, Sep 12, 2025 at 05:04:40PM +0100, Alex Bennée wrote:
>>> Daniel P. Berrangé <berrange@redhat.com> writes:
>>>
>>>> On Thu, Sep 11, 2025 at 08:51:08PM -0300, Gustavo Romero wrote:
>>>>> Hi Daniel,
>>>>>
>>>>> Thanks a lot for review and the suggestions.
>>>>>
>>>>> On 9/8/25 08:49, Daniel P. Berrangé wrote:
>>>>>> On Thu, Sep 04, 2025 at 03:46:35PM +0000, Gustavo Romero wrote:
>>>>>>> In this series, we leveraged the run-test.py script used in the
>>>>>>> check-tcg tests, making it a GDB runner capable of calling a test script
>>>>>>> without spawning any VMs. In this configuration, the test scripts can
>>>>>>> manage the VM and also import gdb, making the GDB Python API inside the
>>>>>>> functional test scripts.
>>>>>>>
>>>>>>> A --quiet option has been added to run-test.py so it doesn't print the
>>>>>>> command line used to execute GDB to the stdout. This ensures that users
>>>>>>> don't get confused about how to re-run the tests. One can re-run the
>>>>>>> test simply by copying and pasting the command line shown by Meson when
>>>>>>> V=1 is passed:
>>>>>>>
>>>>>>> $ make -j check-functional V=1
>>>>>>>
>>>>>>> or, alternatively, once the test run completes, the exact command found
>>>>>>> in the 'command:' field of the build/meson-logs/testlog-thorough.txt
>>>>>>> file generated by Meson. Both methods provide the correct environment
>>>>>>> variables required to run the test, such as the proper $PYTHONPATH.
>>>>>>
>>>>>> While I like the conceptual idea of just sending human GDB commands,
>>>>>> instead of working with GDB protocol packets, I really dislike the
>>>>>> effect this has on the execution / startup of the functional tests
>>>>>> via use of the custom runner for a number of reasons
>>>>>>
>>>>>> * The command line for launching the test outside of meson is very
>>>>>> complicated, so not memorable
>>>>>
>>>>> Why very complicated? It calls a simple runner instead of calling the
>>>>> test script directly, but it doesn't change the way to re-run a single
>>>>> test. One just have to pass V=1 to see make's command line and copy
>>>>> and paste the full command line to re-run the test. I mentioned
>>>>> inspecting 'testlog-thorough.txt' just for completeness.
>>>>
>>>> Today we can run the individual tests directly
>>>>
>>>> # ./tests/functional/x86_64/test_reverse_debug.py
>>>> TAP version 13
>>>> ok 1 test_reverse_debug.ReverseDebugging_X86_64.test_x86_64_pc
>>>> 1..1
>>>>
>>>>
>>>> (assuming you have PYTHONPATH and QEMU_TEST_QEMU_BINARY env set)
>>>
>>> and the old version of Avocado...
>>>
>>>> This gives you a very easy way to interact with the test, see
>>>> its progress, understand what failed, and debug it with strace,
>>>> etc.
>>>>
>>>> This change looses all that. It appears I can run it with
>>>>
>>>> # ./tests/guest-debug/run-test.py --quiet --gdb gdb --test \
>>>> ./tests/functional/x86_64/test_reverse_debug.py
>>>>
>>> <snip>
>>>>
>>>>
>>>> This undermines the core goals of what we aimed to achieve with
>>>> the new functional test harness.
>>>>
>>>>>
>>>>>> * It makes the meson.build rules much more complicated
>>>>>
>>>>> Do we want to never augment functional tests' meson.build? Nothing
>>>>> complicated is being added. Basically, just a new variable suffixed with
>>>>> '_with_runner' which holds a tuple (test, runner) that tell the test
>>>>> to be executed, following the same logic we already have for all the other
>>>>> variables that specify the tests per arch/mode/speed.
>>>>>
>>>>> Another option would be to select a runner based on a suffix in the test
>>>>> name, for instance, 'reverse_debug_with_runner.py'.
>>>>
>>>> IMHO the overall concept of using the run-test.py runner for launching
>>>> the tests is flawed and not viable. It adds too much complexity to the
>>>> use of the tests, and harms the output.
>>>
>>>
>>>
>>>>
>>>>>> * Running standalone there is no TAP output available making the
>>>>>> test hard to debug on failure or timeout
>>>>>
>>>>> This is because of an unfortunate GDB Python API issue, please see my
>>>>> reply in your comment on patch 5/5. This can be solved but needs more
>>>>> investigation on GDB side.
>>>>>
>>>>>
>>>>>> I understand the need to spawn the test via gdb, in order to be able
>>>>>> to import the 'gdb' python module. Looking at what reverse_debugging.py
>>>>>> does, however, makes me question whether we actually need to directly
>>>>>> use the 'gdb' python module.
>>>>>>
>>>>>> The only APIs we use are 'gdb.execute' and 'gdb.parse_and_eval'.
>>>>>>
>>>>>> The latter is only used once as
>>>>>>
>>>>>> gdb.parse_and_eval("$pc")
>>>>>>
>>>>>> and I believe that can be changed to
>>>>>>
>>>>>> gdb.execute("printf \"0x%x\", $pc", to_string=True)
>>>>>>
>>>>>> IOW, all we need is 'gdb.execute("....", to_string=True)'
>>>>>
>>>>> Yes, I do want to directly use the 'gdb' python module directly in the
>>>>> tests. We shouldn't look at a solution only for reverse_debug.py but also
>>>>> think of any future tests that will require the GDB Python API, so I don't
>>>>> want to specialize here and reduce the API to a single method.
>>>>
>>>> If any other tests needing GDB arrive int he future we can consider
>>>> them at that time.
>>>
>>> We already have a whole chunk of gdb tests under check-tcg. Maybe it
>>> would be easier just to re-write the tests to use the check-tcg system
>>> tests rather than jumping through hoops to fit in with the
>>> check-functional requirements.
>>
>> Well if 'easy' is our goal, then we can just copy the gdbmi.py
>> file from avocado into our test suite.
>
> But maintaining that stuff in the QEMU repository is also kind of ugly.
I totally agree.
> I took another stab at the problem:
>
> https://lore.kernel.org/qemu-devel/20250915124207.42053-1-thuth@redhat.com/
>
> It's basically Gustavo's patches, but I removed all the stuff around tests/guest-debug/run-test.py and the cumbersome code for running a test through gdb via meson.build.
>
> The test start in normal pycotap mode, then call a new function called reverse_debug() which then takes care of calling gdb with the right arguments (i.e. this logic has been copied from the run-test.py script).
>
> It seems to work fine with the aarch64 test already, but details and other architectures still need some more love.
>
> WDYT? Is it worth to pursue that approach?
Thanks Thomas, I prefer your approach. I've reviewed your
series and had some minor comments on it but overall it
looks good! :)
Cheers,
Gustavo
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado
2025-09-04 15:46 [PATCH v2 0/5] tests/functional: Adapt reverse_debugging to run w/o Avocado Gustavo Romero
` (5 preceding siblings ...)
2025-09-08 11:49 ` [PATCH v2 0/5] " Daniel P. Berrangé
@ 2025-09-15 8:29 ` Daniel P. Berrangé
6 siblings, 0 replies; 25+ messages in thread
From: Daniel P. Berrangé @ 2025-09-15 8:29 UTC (permalink / raw)
To: Gustavo Romero; +Cc: qemu-devel, alex.bennee, thuth, qemu-arm, 1844144
On Thu, Sep 04, 2025 at 03:46:35PM +0000, Gustavo Romero wrote:
> The goal of this series is to remove Avocado as a dependency for running
> the reverse_debugging functional test.
>
> This test, the last one I’m aware of that relies on Avocado, requires it
> because of the need for GDB to test reverse stepping, continue, etc.
>
> In this series, we leveraged the run-test.py script used in the
> check-tcg tests, making it a GDB runner capable of calling a test script
> without spawning any VMs. In this configuration, the test scripts can
> manage the VM and also import gdb, making the GDB Python API inside the
> functional test scripts.
I've posted an alternate series that removes the avocado
dependency by importing the gdb py code it relied upon.
This avoids the changes to the way the tests must be run
which I don't think are a desirable approach for the
functional tests
https://lists.nongnu.org/archive/html/qemu-devel/2025-09/msg02499.html
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 25+ messages in thread